2024-03-18T20:13:47.189 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-18T20:13:47.190 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_yuriw@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --archive /home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382 --name yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi -- /home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382/orig.config.yaml 2024-03-18T20:13:47.250 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T20:13:47.346 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382 branch: wip-yuri10-testing-2024-03-15-1653-reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: yweinste@redhat.com first_in_suite: false job_id: '7606382' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri10-testing-2024-03-15-1653-reef ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 49bdc51142759622a281d3e5327eebd07cbb8bb3 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 49bdc51142759622a281d3e5327eebd07cbb8bb3 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri10-testing-2024-03-15-1653-reef sha1: 49bdc51142759622a281d3e5327eebd07cbb8bb3 owner: scheduled_yuriw@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8313 sha1: 49bdc51142759622a281d3e5327eebd07cbb8bb3 sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri10-testing-2024-03-15-1653-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_49bdc51142759622a281d3e5327eebd07cbb8bb3/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 49bdc51142759622a281d3e5327eebd07cbb8bb3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJjbaIubNluIMJ5WYE2UsGKZYMdPrlAlHELCOZgn5CAEzD7NPU+mrkUwcL48e7pZz1x+aY1v/0ku468yC0+qBXU= smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFEsrw1C5H2lCkAnrn/V52JlpCJU99u6LHS4CKEwe7In5E93NlG7KT7uJI4l+G5YLeA6Jsm/yMh9TwxiRdXb5+s= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH3GcrcPA+A6q8OGeEMHtFZt1ALZ47FuwpBd94qmQw+xP7LNWuj5zeeEHZBfgI+nZzGsrtajzFUEjRTnGRf2qO8= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-16_15:03:17 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-18T20:13:47.347 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_49bdc51142759622a281d3e5327eebd07cbb8bb3/qa; will attempt to use it 2024-03-18T20:13:47.347 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_49bdc51142759622a281d3e5327eebd07cbb8bb3/qa/tasks 2024-03-18T20:13:47.348 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-18T20:13:47.349 INFO:teuthology.task.internal:Checking packages... 2024-03-18T20:13:47.373 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '49bdc51142759622a281d3e5327eebd07cbb8bb3' 2024-03-18T20:13:47.373 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T20:13:47.373 INFO:teuthology.packaging:ref: None 2024-03-18T20:13:47.373 INFO:teuthology.packaging:tag: None 2024-03-18T20:13:47.373 INFO:teuthology.packaging:branch: wip-yuri10-testing-2024-03-15-1653-reef 2024-03-18T20:13:47.373 INFO:teuthology.packaging:sha1: 49bdc51142759622a281d3e5327eebd07cbb8bb3 2024-03-18T20:13:47.374 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri10-testing-2024-03-15-1653-reef 2024-03-18T20:13:47.601 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-914.g49bdc511 2024-03-18T20:13:47.602 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-18T20:13:47.610 INFO:teuthology.task.internal:no buildpackages task found 2024-03-18T20:13:47.610 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-18T20:13:47.634 INFO:teuthology.task.internal:Saving configuration 2024-03-18T20:13:47.656 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-18T20:13:47.663 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T20:13:47.688 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 20:08:32.955827', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJjbaIubNluIMJ5WYE2UsGKZYMdPrlAlHELCOZgn5CAEzD7NPU+mrkUwcL48e7pZz1x+aY1v/0ku468yC0+qBXU='} 2024-03-18T20:13:47.714 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 20:08:32.957238', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFEsrw1C5H2lCkAnrn/V52JlpCJU99u6LHS4CKEwe7In5E93NlG7KT7uJI4l+G5YLeA6Jsm/yMh9TwxiRdXb5+s='} 2024-03-18T20:13:47.735 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 20:08:32.954227', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH3GcrcPA+A6q8OGeEMHtFZt1ALZ47FuwpBd94qmQw+xP7LNWuj5zeeEHZBfgI+nZzGsrtajzFUEjRTnGRf2qO8='} 2024-03-18T20:13:47.735 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-18T20:13:47.742 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-18T20:13:47.742 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-18T20:13:47.742 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-18T20:13:47.743 INFO:teuthology.run_tasks:Running task console_log... 2024-03-18T20:13:47.857 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fecb0db3160>, signals=[15]) 2024-03-18T20:13:47.857 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-18T20:13:47.866 INFO:teuthology.task.internal:Opening connections... 2024-03-18T20:13:47.866 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-03-18T20:13:47.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T20:13:47.946 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2024-03-18T20:13:47.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T20:13:48.021 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-03-18T20:13:48.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T20:13:48.094 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-18T20:13:48.101 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-03-18T20:13:48.124 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-03-18T20:13:48.124 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-03-18T20:13:48.185 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Red Hat Enterprise Linux" 2024-03-18T20:13:48.185 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="8.6 (Ootpa)" 2024-03-18T20:13:48.185 INFO:teuthology.orchestra.run.smithi046.stdout:ID="rhel" 2024-03-18T20:13:48.186 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="fedora" 2024-03-18T20:13:48.186 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="8.6" 2024-03-18T20:13:48.186 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el8" 2024-03-18T20:13:48.186 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-18T20:13:48.186 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-03-18T20:13:48.186 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-18T20:13:48.186 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.redhat.com/" 2024-03-18T20:13:48.186 INFO:teuthology.orchestra.run.smithi046.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-18T20:13:48.186 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T20:13:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:13:48.186 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-18T20:13:48.186 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-18T20:13:48.187 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-18T20:13:48.187 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-18T20:13:48.187 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-03-18T20:13:48.210 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-03-18T20:13:48.231 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-03-18T20:13:48.231 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-03-18T20:13:48.293 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="Red Hat Enterprise Linux" 2024-03-18T20:13:48.293 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="8.6 (Ootpa)" 2024-03-18T20:13:48.293 INFO:teuthology.orchestra.run.smithi073.stdout:ID="rhel" 2024-03-18T20:13:48.293 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="fedora" 2024-03-18T20:13:48.293 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="8.6" 2024-03-18T20:13:48.293 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el8" 2024-03-18T20:13:48.293 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-18T20:13:48.293 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2024-03-18T20:13:48.294 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-18T20:13:48.294 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://www.redhat.com/" 2024-03-18T20:13:48.294 INFO:teuthology.orchestra.run.smithi073.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-18T20:13:48.294 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T20:13:48.294 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:13:48.294 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-18T20:13:48.294 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-18T20:13:48.294 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-18T20:13:48.294 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-18T20:13:48.294 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-03-18T20:13:48.380 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-03-18T20:13:48.401 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-03-18T20:13:48.402 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-03-18T20:13:48.461 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Red Hat Enterprise Linux" 2024-03-18T20:13:48.461 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8.6 (Ootpa)" 2024-03-18T20:13:48.461 INFO:teuthology.orchestra.run.smithi155.stdout:ID="rhel" 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="fedora" 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8.6" 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8" 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.redhat.com/" 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-18T20:13:48.462 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-18T20:13:48.463 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-18T20:13:48.463 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-03-18T20:13:48.493 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-18T20:13:48.504 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-18T20:13:48.511 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-18T20:13:48.512 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-03-18T20:13:48.514 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2024-03-18T20:13:48.517 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-03-18T20:13:48.536 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-18T20:13:48.542 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-18T20:13:48.542 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-03-18T20:13:48.574 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2024-03-18T20:13:48.577 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-03-18T20:13:48.604 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-18T20:13:48.684 INFO:teuthology.run_tasks:Running task kernel... 2024-03-18T20:13:48.698 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-18T20:13:48.699 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-18T20:13:48.699 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-18T20:13:48.699 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T20:13:48.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T20:13:48.720 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-03-18T20:13:48.781 INFO:teuthology.orchestra.run.smithi046.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-18T20:13:48.782 INFO:teuthology.task.kernel:Running kernel on smithi046: 4.18.0-513.9.1.el8_9.x86_64 2024-03-18T20:13:48.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum install -y kernel 2024-03-18T20:13:50.318 INFO:teuthology.orchestra.run.smithi046.stdout:Updating Subscription Management repositories. 2024-03-18T20:13:50.318 INFO:teuthology.orchestra.run.smithi046.stdout:Unable to read consumer identity 2024-03-18T20:13:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:13:50.623 INFO:teuthology.orchestra.run.smithi046.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-18T20:13:50.623 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:13:52.271 INFO:teuthology.orchestra.run.smithi046.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-03-18T20:13:54.056 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 16 MB 00:01 2024-03-18T20:13:58.180 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 651 kB/s | 24 kB 00:00 2024-03-18T20:13:58.933 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-03-18T20:13:58.933 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-03-18T20:13:59.018 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-03-18T20:13:59.019 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-03-18T20:13:59.019 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-03-18T20:13:59.214 DEBUG:teuthology.orchestra.run.smithi046:> echo no | sudo yum reinstall kernel || true 2024-03-18T20:13:59.600 INFO:teuthology.orchestra.run.smithi046.stdout:Updating Subscription Management repositories. 2024-03-18T20:13:59.600 INFO:teuthology.orchestra.run.smithi046.stdout:Unable to read consumer identity 2024-03-18T20:13:59.786 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:13:59.786 INFO:teuthology.orchestra.run.smithi046.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-18T20:13:59.786 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:14:00.148 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:02 ago on Mon 18 Mar 2024 08:13:58 PM UTC. 2024-03-18T20:14:00.232 INFO:teuthology.orchestra.run.smithi046.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-03-18T20:14:00.248 INFO:teuthology.orchestra.run.smithi046.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-03-18T20:14:00.303 INFO:teuthology.orchestra.run.smithi046.stderr:Error: No packages marked for reinstall. 2024-03-18T20:14:00.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum reinstall -y kernel || true 2024-03-18T20:14:00.741 INFO:teuthology.orchestra.run.smithi046.stdout:Updating Subscription Management repositories. 2024-03-18T20:14:00.742 INFO:teuthology.orchestra.run.smithi046.stdout:Unable to read consumer identity 2024-03-18T20:14:00.974 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:14:00.974 INFO:teuthology.orchestra.run.smithi046.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-18T20:14:00.974 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:14:01.325 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:03 ago on Mon 18 Mar 2024 08:13:58 PM UTC. 2024-03-18T20:14:01.398 INFO:teuthology.orchestra.run.smithi046.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-03-18T20:14:01.398 INFO:teuthology.orchestra.run.smithi046.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-03-18T20:14:01.441 INFO:teuthology.orchestra.run.smithi046.stderr:Error: No packages marked for reinstall. 2024-03-18T20:14:01.495 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T20:14:01.679 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-03-18T20:14:01.679 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-03-18T20:14:01.679 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-03-18T20:14:01.679 DEBUG:teuthology.parallel:result is None 2024-03-18T20:14:01.680 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-18T20:14:01.687 INFO:teuthology.task.internal:Creating test directory... 2024-03-18T20:14:01.687 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T20:14:01.689 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T20:14:01.691 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T20:14:01.712 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-18T20:14:01.718 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-18T20:14:01.753 INFO:teuthology.task.internal:Creating archive directory... 2024-03-18T20:14:01.753 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T20:14:01.755 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T20:14:01.757 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T20:14:01.980 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-18T20:14:01.987 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-18T20:14:01.987 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T20:14:02.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T20:14:02.006 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T20:14:02.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T20:14:02.026 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T20:14:02.028 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T20:14:02.049 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T20:14:02.067 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T20:14:02.082 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T20:14:02.084 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T20:14:02.102 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T20:14:02.110 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T20:14:02.128 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T20:14:02.129 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-18T20:14:02.136 INFO:teuthology.task.internal:Configuring sudo... 2024-03-18T20:14:02.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T20:14:02.139 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T20:14:02.146 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T20:14:02.175 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-18T20:14:02.184 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-18T20:14:02.184 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T20:14:02.214 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T20:14:02.216 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T20:14:02.235 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T20:14:02.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T20:14:02.402 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T20:14:02.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T20:14:02.500 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:14:02.501 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T20:14:02.575 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T20:14:02.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T20:14:02.669 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T20:14:02.755 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T20:14:02.799 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-18T20:14:02.801 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T20:14:02.867 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T20:14:02.915 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T20:14:02.972 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T20:14:03.058 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T20:14:03.104 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-18T20:14:03.104 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T20:14:03.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-03-18T20:14:03.176 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2024-03-18T20:14:03.179 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-03-18T20:14:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T20:14:03.227 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T20:14:03.261 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T20:14:03.561 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-18T20:14:03.569 INFO:teuthology.task.internal:Starting timer... 2024-03-18T20:14:03.570 INFO:teuthology.run_tasks:Running task pcp... 2024-03-18T20:14:03.596 INFO:teuthology.run_tasks:Running task selinux... 2024-03-18T20:14:03.614 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2024-03-18T20:14:03.615 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-03-18T20:14:03.701 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-03-18T20:14:03.703 DEBUG:teuthology.orchestra.run.smithi073:> sudo service auditd rotate 2024-03-18T20:14:03.788 INFO:teuthology.orchestra.run.smithi073.stdout:Rotating logs: 2024-03-18T20:14:03.790 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2024-03-18T20:14:03.856 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2024-03-18T20:14:03.858 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-18T20:14:03.858 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-03-18T20:14:03.882 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-03-18T20:14:03.882 DEBUG:teuthology.orchestra.run.smithi073:> /usr/sbin/getenforce 2024-03-18T20:14:03.918 INFO:teuthology.orchestra.run.smithi073.stdout:Permissive 2024-03-18T20:14:03.918 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2024-03-18T20:14:03.941 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2024-03-18T20:14:03.941 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi073.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive'} 2024-03-18T20:14:03.942 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-18T20:14:03.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T20:14:03.976 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-18T20:14:04.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T20:14:04.005 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-18T20:14:04.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T20:14:04.032 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-18T20:14:04.032 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-03-18T20:14:04.064 DEBUG:teuthology.orchestra.run.smithi073:> sudo /usr/sbin/setenforce permissive 2024-03-18T20:14:04.092 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2024-03-18T20:14:04.119 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-18T20:14:04.129 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-18T20:14:04.134 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-18T20:14:04.246 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-18T20:14:04.262 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-18T20:14:04.263 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi073.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-18T20:24:21.416 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi073.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-03-18T20:24:21.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-03-18T20:24:21.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T20:24:21.500 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-03-18T20:24:21.531 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-03-18T20:24:21.531 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-03-18T20:24:21.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T20:24:21.606 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-03-18T20:24:21.677 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-03-18T20:24:21.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-03-18T20:24:21.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T20:24:21.755 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-03-18T20:24:21.824 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-03-18T20:24:21.825 INFO:teuthology.run_tasks:Running task clock... 2024-03-18T20:24:21.837 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-18T20:24:21.837 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T20:24:21.838 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T20:24:21.840 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T20:24:21.840 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T20:24:21.843 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T20:24:21.843 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T20:24:21.875 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T20:24:21.879 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T20:24:21.890 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T20:24:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T20:24:21.900 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T20:24:21.915 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T20:24:21.956 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: ntpd: command not found 2024-03-18T20:24:21.968 INFO:teuthology.orchestra.run.smithi073.stdout:506 Cannot talk to daemon 2024-03-18T20:24:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-03-18T20:24:21.982 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2024-03-18T20:24:21.983 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T20:24:21.988 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-03-18T20:24:21.995 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2024-03-18T20:24:21.997 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T20:24:22.006 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T20:24:22.010 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T20:24:22.023 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T20:24:22.025 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T20:24:22.073 INFO:teuthology.orchestra.run.smithi073.stderr:bash: ntpq: command not found 2024-03-18T20:24:22.077 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T20:24:22.077 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-03-18T20:24:22.077 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T20:24:22.077 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T20:24:22.077 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T20:24:22.077 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T20:24:22.096 INFO:teuthology.orchestra.run.smithi046.stderr:bash: ntpq: command not found 2024-03-18T20:24:22.101 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T20:24:22.101 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-03-18T20:24:22.101 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T20:24:22.101 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T20:24:22.101 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T20:24:22.101 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T20:24:22.111 INFO:teuthology.orchestra.run.smithi155.stderr:bash: ntpq: command not found 2024-03-18T20:24:22.115 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T20:24:22.115 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-03-18T20:24:22.115 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T20:24:22.115 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T20:24:22.115 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T20:24:22.115 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T20:24:22.116 INFO:teuthology.run_tasks:Running task pexec... 2024-03-18T20:24:22.125 INFO:teuthology.task.pexec:Executing custom commands... 2024-03-18T20:24:22.126 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi046.front.sepia.ceph.com 2024-03-18T20:24:22.126 DEBUG:teuthology.orchestra.run.smithi046:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-18T20:24:22.126 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi073.front.sepia.ceph.com 2024-03-18T20:24:22.126 DEBUG:teuthology.orchestra.run.smithi073:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-18T20:24:22.127 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi155.front.sepia.ceph.com 2024-03-18T20:24:22.127 DEBUG:teuthology.orchestra.run.smithi155:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-18T20:24:22.505 INFO:teuthology.orchestra.run.smithi073.stdout:Updating Subscription Management repositories. 2024-03-18T20:24:22.533 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2024-03-18T20:24:22.577 INFO:teuthology.orchestra.run.smithi046.stdout:Updating Subscription Management repositories. 2024-03-18T20:24:24.320 INFO:teuthology.orchestra.run.smithi073.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 36 kB/s | 4.1 kB 00:00 2024-03-18T20:24:24.418 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-03-18T20:24:24.626 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-03-18T20:24:24.673 INFO:teuthology.orchestra.run.smithi073.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-03-18T20:24:24.752 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-03-18T20:24:24.932 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 63 kB/s | 4.5 kB 00:00 2024-03-18T20:24:27.652 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-18T20:24:27.653 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-18T20:24:27.653 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-03-18T20:24:27.653 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-18T20:24:27.653 INFO:teuthology.orchestra.run.smithi155.stdout:Resetting modules: 2024-03-18T20:24:27.653 INFO:teuthology.orchestra.run.smithi155.stdout: container-tools 2024-03-18T20:24:27.653 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:24:27.653 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-18T20:24:27.653 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-18T20:24:27.653 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:24:27.770 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-03-18T20:24:27.772 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-03-18T20:24:27.772 INFO:teuthology.orchestra.run.smithi073.stdout: Package Architecture Version Repository Size 2024-03-18T20:24:27.772 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-03-18T20:24:27.772 INFO:teuthology.orchestra.run.smithi073.stdout:Resetting modules: 2024-03-18T20:24:27.772 INFO:teuthology.orchestra.run.smithi073.stdout: container-tools 2024-03-18T20:24:27.772 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:24:27.772 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-03-18T20:24:27.772 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-03-18T20:24:27.772 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:24:28.331 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-03-18T20:24:28.333 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-03-18T20:24:28.333 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-03-18T20:24:28.333 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-03-18T20:24:28.333 INFO:teuthology.orchestra.run.smithi046.stdout:Resetting modules: 2024-03-18T20:24:28.333 INFO:teuthology.orchestra.run.smithi046.stdout: container-tools 2024-03-18T20:24:28.333 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:24:28.333 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-03-18T20:24:28.333 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-03-18T20:24:28.333 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:24:39.104 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-03-18T20:24:39.639 INFO:teuthology.orchestra.run.smithi046.stdout:Updating Subscription Management repositories. 2024-03-18T20:24:39.724 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-03-18T20:24:40.216 INFO:teuthology.orchestra.run.smithi073.stdout:Updating Subscription Management repositories. 2024-03-18T20:24:40.641 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-18T20:24:41.129 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2024-03-18T20:24:41.705 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-03-18T20:24:41.994 INFO:teuthology.orchestra.run.smithi073.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-03-18T20:24:42.058 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2024-03-18T20:24:42.336 INFO:teuthology.orchestra.run.smithi073.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2024-03-18T20:24:42.800 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 102 kB/s | 4.1 kB 00:00 2024-03-18T20:24:43.120 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 kB/s | 4.5 kB 00:00 2024-03-18T20:24:45.378 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-03-18T20:24:45.418 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-03-18T20:24:45.543 INFO:teuthology.orchestra.run.smithi073.stdout:========================================================================================================== 2024-03-18T20:24:45.543 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-03-18T20:24:45.543 INFO:teuthology.orchestra.run.smithi073.stdout:========================================================================================================== 2024-03-18T20:24:45.543 INFO:teuthology.orchestra.run.smithi073.stdout:Installing group/module packages: 2024-03-18T20:24:45.543 INFO:teuthology.orchestra.run.smithi073.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-18T20:24:45.543 INFO:teuthology.orchestra.run.smithi073.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-18T20:24:45.543 INFO:teuthology.orchestra.run.smithi073.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-18T20:24:45.543 INFO:teuthology.orchestra.run.smithi073.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-18T20:24:45.543 INFO:teuthology.orchestra.run.smithi073.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-18T20:24:45.544 INFO:teuthology.orchestra.run.smithi073.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-18T20:24:45.544 INFO:teuthology.orchestra.run.smithi073.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-18T20:24:45.544 INFO:teuthology.orchestra.run.smithi073.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-18T20:24:45.544 INFO:teuthology.orchestra.run.smithi073.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-18T20:24:45.544 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-03-18T20:24:45.544 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-18T20:24:45.544 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-18T20:24:45.544 INFO:teuthology.orchestra.run.smithi073.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-18T20:24:45.544 INFO:teuthology.orchestra.run.smithi073.stdout:Installing weak dependencies: 2024-03-18T20:24:45.544 INFO:teuthology.orchestra.run.smithi073.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-18T20:24:45.544 INFO:teuthology.orchestra.run.smithi073.stdout:Installing module profiles: 2024-03-18T20:24:45.544 INFO:teuthology.orchestra.run.smithi073.stdout: container-tools/common 2024-03-18T20:24:45.545 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:24:45.545 INFO:teuthology.orchestra.run.smithi073.stdout:Enabling module streams: 2024-03-18T20:24:45.545 INFO:teuthology.orchestra.run.smithi073.stdout: container-tools rhel8 2024-03-18T20:24:45.545 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:24:45.545 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-03-18T20:24:45.545 INFO:teuthology.orchestra.run.smithi073.stdout:========================================================================================================== 2024-03-18T20:24:45.545 INFO:teuthology.orchestra.run.smithi073.stdout:Install 13 Packages 2024-03-18T20:24:45.545 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:24:45.546 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 26 M 2024-03-18T20:24:45.546 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 87 M 2024-03-18T20:24:45.546 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-03-18T20:24:45.597 INFO:teuthology.orchestra.run.smithi046.stdout:========================================================================================================== 2024-03-18T20:24:45.598 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-03-18T20:24:45.598 INFO:teuthology.orchestra.run.smithi046.stdout:========================================================================================================== 2024-03-18T20:24:45.598 INFO:teuthology.orchestra.run.smithi046.stdout:Installing group/module packages: 2024-03-18T20:24:45.598 INFO:teuthology.orchestra.run.smithi046.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-18T20:24:45.598 INFO:teuthology.orchestra.run.smithi046.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-18T20:24:45.598 INFO:teuthology.orchestra.run.smithi046.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-18T20:24:45.598 INFO:teuthology.orchestra.run.smithi046.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-18T20:24:45.598 INFO:teuthology.orchestra.run.smithi046.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-18T20:24:45.598 INFO:teuthology.orchestra.run.smithi046.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-18T20:24:45.598 INFO:teuthology.orchestra.run.smithi046.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout:Installing module profiles: 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout: container-tools/common 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling module streams: 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout: container-tools rhel8 2024-03-18T20:24:45.599 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:24:45.600 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-03-18T20:24:45.600 INFO:teuthology.orchestra.run.smithi046.stdout:========================================================================================================== 2024-03-18T20:24:45.600 INFO:teuthology.orchestra.run.smithi046.stdout:Install 13 Packages 2024-03-18T20:24:45.600 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:24:45.600 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 26 M 2024-03-18T20:24:45.600 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 87 M 2024-03-18T20:24:45.601 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-03-18T20:24:45.911 INFO:teuthology.orchestra.run.smithi046.stdout:(1/13): criu-libs-3.18-4.module+el8.9.0+20326+3 127 kB/s | 39 kB 00:00 2024-03-18T20:24:45.924 INFO:teuthology.orchestra.run.smithi073.stdout:(1/13): criu-libs-3.18-4.module+el8.9.0+20326+3 104 kB/s | 39 kB 00:00 2024-03-18T20:24:45.941 INFO:teuthology.orchestra.run.smithi073.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 63 kB/s | 25 kB 00:00 2024-03-18T20:24:45.958 INFO:teuthology.orchestra.run.smithi073.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 228 kB/s | 94 kB 00:00 2024-03-18T20:24:45.994 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-18T20:24:46.020 INFO:teuthology.orchestra.run.smithi046.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 224 kB/s | 94 kB 00:00 2024-03-18T20:24:46.045 INFO:teuthology.orchestra.run.smithi046.stdout:(3/13): python3-pytoml-0.1.14-5.git7dea353.el8. 56 kB/s | 25 kB 00:00 2024-03-18T20:24:46.066 INFO:teuthology.orchestra.run.smithi073.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 288 kB/s | 41 kB 00:00 2024-03-18T20:24:46.070 INFO:teuthology.orchestra.run.smithi046.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 257 kB/s | 41 kB 00:00 2024-03-18T20:24:46.157 INFO:teuthology.orchestra.run.smithi155.stdout:========================================================================================================== 2024-03-18T20:24:46.157 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-18T20:24:46.157 INFO:teuthology.orchestra.run.smithi155.stdout:========================================================================================================== 2024-03-18T20:24:46.157 INFO:teuthology.orchestra.run.smithi155.stdout:Installing group/module packages: 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-18T20:24:46.158 INFO:teuthology.orchestra.run.smithi155.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-18T20:24:46.159 INFO:teuthology.orchestra.run.smithi155.stdout:Installing weak dependencies: 2024-03-18T20:24:46.159 INFO:teuthology.orchestra.run.smithi155.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-18T20:24:46.159 INFO:teuthology.orchestra.run.smithi155.stdout:Installing module profiles: 2024-03-18T20:24:46.159 INFO:teuthology.orchestra.run.smithi155.stdout: container-tools/common 2024-03-18T20:24:46.159 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:24:46.159 INFO:teuthology.orchestra.run.smithi155.stdout:Enabling module streams: 2024-03-18T20:24:46.159 INFO:teuthology.orchestra.run.smithi155.stdout: container-tools rhel8 2024-03-18T20:24:46.159 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:24:46.159 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-18T20:24:46.159 INFO:teuthology.orchestra.run.smithi155.stdout:========================================================================================================== 2024-03-18T20:24:46.159 INFO:teuthology.orchestra.run.smithi155.stdout:Install 13 Packages 2024-03-18T20:24:46.159 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:24:46.160 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 26 M 2024-03-18T20:24:46.160 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 87 M 2024-03-18T20:24:46.160 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-18T20:24:46.275 INFO:teuthology.orchestra.run.smithi073.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2124 3.5 MB/s | 739 kB 00:00 2024-03-18T20:24:46.279 INFO:teuthology.orchestra.run.smithi046.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2124 3.5 MB/s | 739 kB 00:00 2024-03-18T20:24:46.313 INFO:teuthology.orchestra.run.smithi046.stdout:(6/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 3.5 MB/s | 1.0 MB 00:00 2024-03-18T20:24:46.326 INFO:teuthology.orchestra.run.smithi073.stdout:(6/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 2.6 MB/s | 1.0 MB 00:00 2024-03-18T20:24:46.352 INFO:teuthology.orchestra.run.smithi155.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 490 kB/s | 94 kB 00:00 2024-03-18T20:24:46.377 INFO:teuthology.orchestra.run.smithi155.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 114 kB/s | 25 kB 00:00 2024-03-18T20:24:46.402 INFO:teuthology.orchestra.run.smithi155.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 162 kB/s | 39 kB 00:00 2024-03-18T20:24:46.495 INFO:teuthology.orchestra.run.smithi155.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 287 kB/s | 41 kB 00:00 2024-03-18T20:24:46.517 INFO:teuthology.orchestra.run.smithi073.stdout:(7/13): buildah-1.31.3-3.module+el8.9.0+21243+a 16 MB/s | 8.8 MB 00:00 2024-03-18T20:24:46.536 INFO:teuthology.orchestra.run.smithi155.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 6.3 MB/s | 1.0 MB 00:00 2024-03-18T20:24:46.542 INFO:teuthology.orchestra.run.smithi046.stdout:(7/13): buildah-1.31.3-3.module+el8.9.0+21243+a 18 MB/s | 8.8 MB 00:00 2024-03-18T20:24:46.543 INFO:teuthology.orchestra.run.smithi073.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 895 kB/s | 239 kB 00:00 2024-03-18T20:24:46.571 INFO:teuthology.orchestra.run.smithi046.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 819 kB/s | 239 kB 00:00 2024-03-18T20:24:46.686 INFO:teuthology.orchestra.run.smithi155.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2124 3.8 MB/s | 739 kB 00:00 2024-03-18T20:24:46.721 INFO:teuthology.orchestra.run.smithi155.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 1.3 MB/s | 239 kB 00:00 2024-03-18T20:24:46.763 INFO:teuthology.orchestra.run.smithi046.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21243+a 8.3 MB/s | 3.7 MB 00:00 2024-03-18T20:24:46.776 INFO:teuthology.orchestra.run.smithi073.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21243+a 8.2 MB/s | 3.7 MB 00:00 2024-03-18T20:24:46.802 INFO:teuthology.orchestra.run.smithi073.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 542 kB/s | 154 kB 00:00 2024-03-18T20:24:46.821 INFO:teuthology.orchestra.run.smithi155.stdout:(8/13): buildah-1.31.3-3.module+el8.9.0+21243+a 21 MB/s | 8.8 MB 00:00 2024-03-18T20:24:46.871 INFO:teuthology.orchestra.run.smithi155.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 1.0 MB/s | 154 kB 00:00 2024-03-18T20:24:46.922 INFO:teuthology.orchestra.run.smithi046.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 405 kB/s | 154 kB 00:00 2024-03-18T20:24:46.969 INFO:teuthology.orchestra.run.smithi073.stdout:(11/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 19 MB/s | 8.1 MB 00:00 2024-03-18T20:24:46.971 INFO:teuthology.orchestra.run.smithi155.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21243+ 13 MB/s | 3.7 MB 00:00 2024-03-18T20:24:47.022 INFO:teuthology.orchestra.run.smithi046.stdout:(11/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 18 MB/s | 8.1 MB 00:00 2024-03-18T20:24:47.052 INFO:teuthology.orchestra.run.smithi073.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21243+a5 198 kB/s | 49 kB 00:00 2024-03-18T20:24:47.122 INFO:teuthology.orchestra.run.smithi046.stdout:(12/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 7.1 MB/s | 2.5 MB 00:00 2024-03-18T20:24:47.139 INFO:teuthology.orchestra.run.smithi046.stdout:(13/13): udica-0.2.6-20.module+el8.9.0+21243+a5 228 kB/s | 49 kB 00:00 2024-03-18T20:24:47.140 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-03-18T20:24:47.140 INFO:teuthology.orchestra.run.smithi046.stdout:Total 17 MB/s | 26 MB 00:01 2024-03-18T20:24:47.155 INFO:teuthology.orchestra.run.smithi155.stdout:(11/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 24 MB/s | 8.1 MB 00:00 2024-03-18T20:24:47.180 INFO:teuthology.orchestra.run.smithi155.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21243+a5 237 kB/s | 49 kB 00:00 2024-03-18T20:24:47.222 INFO:teuthology.orchestra.run.smithi155.stdout:(13/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 7.2 MB/s | 2.5 MB 00:00 2024-03-18T20:24:47.222 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-18T20:24:47.222 INFO:teuthology.orchestra.run.smithi155.stdout:Total 24 MB/s | 26 MB 00:01 2024-03-18T20:24:47.285 INFO:teuthology.orchestra.run.smithi073.stdout:(13/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 5.0 MB/s | 2.5 MB 00:00 2024-03-18T20:24:47.286 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-03-18T20:24:47.286 INFO:teuthology.orchestra.run.smithi073.stdout:Total 15 MB/s | 26 MB 00:01 2024-03-18T20:24:47.445 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-03-18T20:24:47.470 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-03-18T20:24:47.470 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-03-18T20:24:47.506 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-18T20:24:47.527 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-18T20:24:47.527 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-18T20:24:47.571 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-03-18T20:24:47.592 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-03-18T20:24:47.592 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-03-18T20:24:47.724 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-03-18T20:24:47.734 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-03-18T20:24:47.757 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-18T20:24:47.766 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-18T20:24:47.822 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-03-18T20:24:47.831 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-03-18T20:24:48.445 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-18T20:24:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-03-18T20:24:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-03-18T20:24:48.973 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-18T20:24:48.978 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-18T20:24:48.983 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-18T20:24:49.157 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-18T20:24:49.158 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-18T20:24:49.167 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-18T20:24:49.310 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-18T20:24:49.320 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-18T20:24:49.324 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-18T20:24:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-18T20:24:49.484 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-18T20:24:49.501 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-18T20:24:49.624 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-18T20:24:49.654 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-18T20:24:49.660 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-18T20:24:49.837 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-18T20:24:49.854 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-18T20:24:49.869 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-18T20:24:50.044 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-18T20:24:50.105 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-18T20:24:50.118 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-18T20:24:50.921 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-18T20:24:51.002 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-18T20:24:51.047 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-18T20:24:51.129 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-18T20:24:51.256 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-18T20:24:51.271 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-18T20:24:51.502 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-18T20:24:51.650 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-18T20:24:51.651 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-18T20:24:52.412 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-18T20:24:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-18T20:24:52.586 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-18T20:24:52.622 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-18T20:24:52.770 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-18T20:24:52.786 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-18T20:24:52.788 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-18T20:24:52.957 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-18T20:24:52.961 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-18T20:24:53.225 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-18T20:24:53.382 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-18T20:24:53.382 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-18T20:24:53.382 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-18T20:24:53.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-18T20:24:53.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-18T20:24:53.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-18T20:24:53.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-18T20:24:53.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-18T20:24:53.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-18T20:24:53.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-18T20:24:53.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-18T20:24:53.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-18T20:24:53.383 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-18T20:24:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-18T20:24:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-18T20:24:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-18T20:24:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-18T20:24:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-18T20:24:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-18T20:24:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-18T20:24:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-18T20:24:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-18T20:24:53.417 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-18T20:24:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-18T20:24:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-18T20:24:53.418 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-18T20:25:03.281 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-18T20:25:03.281 INFO:teuthology.orchestra.run.smithi046.stdout:Installed products updated. 2024-03-18T20:25:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:25:03.920 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-03-18T20:25:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-18T20:25:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-18T20:25:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:03.920 INFO:teuthology.orchestra.run.smithi046.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-18T20:25:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-18T20:25:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-18T20:25:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-18T20:25:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-18T20:25:03.921 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:25:03.921 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-03-18T20:25:04.078 DEBUG:teuthology.parallel:result is None 2024-03-18T20:25:05.084 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-18T20:25:05.084 INFO:teuthology.orchestra.run.smithi155.stdout:Installed products updated. 2024-03-18T20:25:05.645 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:25:05.646 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-18T20:25:05.646 INFO:teuthology.orchestra.run.smithi155.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:05.646 INFO:teuthology.orchestra.run.smithi155.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:05.646 INFO:teuthology.orchestra.run.smithi155.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-18T20:25:05.646 INFO:teuthology.orchestra.run.smithi155.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-18T20:25:05.646 INFO:teuthology.orchestra.run.smithi155.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:05.646 INFO:teuthology.orchestra.run.smithi155.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:05.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-18T20:25:05.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-18T20:25:05.646 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-18T20:25:05.646 INFO:teuthology.orchestra.run.smithi155.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:05.647 INFO:teuthology.orchestra.run.smithi155.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:05.647 INFO:teuthology.orchestra.run.smithi155.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-18T20:25:05.647 INFO:teuthology.orchestra.run.smithi155.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-18T20:25:05.647 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:25:05.647 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-18T20:25:05.798 DEBUG:teuthology.parallel:result is None 2024-03-18T20:25:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-18T20:25:06.155 INFO:teuthology.orchestra.run.smithi073.stdout:Installed products updated. 2024-03-18T20:25:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-18T20:25:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:25:06.712 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-03-18T20:25:06.855 DEBUG:teuthology.parallel:result is None 2024-03-18T20:25:06.855 INFO:teuthology.run_tasks:Running task install... 2024-03-18T20:25:06.865 DEBUG:teuthology.task.install:project ceph 2024-03-18T20:25:06.865 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '49bdc51142759622a281d3e5327eebd07cbb8bb3'}} 2024-03-18T20:25:06.865 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '49bdc51142759622a281d3e5327eebd07cbb8bb3'} 2024-03-18T20:25:06.865 INFO:teuthology.task.install:Using flavor: default 2024-03-18T20:25:06.871 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-18T20:25:06.871 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-18T20:25:06.871 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49bdc51142759622a281d3e5327eebd07cbb8bb3', 'tag': None, 'wait_for_package': False} 2024-03-18T20:25:06.872 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49bdc51142759622a281d3e5327eebd07cbb8bb3 2024-03-18T20:25:06.874 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49bdc51142759622a281d3e5327eebd07cbb8bb3', 'tag': None, 'wait_for_package': False} 2024-03-18T20:25:06.874 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49bdc51142759622a281d3e5327eebd07cbb8bb3 2024-03-18T20:25:06.875 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49bdc51142759622a281d3e5327eebd07cbb8bb3', 'tag': None, 'wait_for_package': False} 2024-03-18T20:25:06.876 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49bdc51142759622a281d3e5327eebd07cbb8bb3 2024-03-18T20:25:07.021 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/ 2024-03-18T20:25:07.021 INFO:teuthology.task.install.rpm:Package version is 18.2.2-914.g49bdc511 2024-03-18T20:25:07.091 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/ 2024-03-18T20:25:07.091 INFO:teuthology.task.install.rpm:Package version is 18.2.2-914.g49bdc511 2024-03-18T20:25:07.099 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/ 2024-03-18T20:25:07.099 INFO:teuthology.task.install.rpm:Package version is 18.2.2-914.g49bdc511 2024-03-18T20:25:07.139 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T20:25:07.140 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-18T20:25:07.140 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T20:25:07.179 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-18T20:25:07.179 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49bdc51142759622a281d3e5327eebd07cbb8bb3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T20:25:07.253 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T20:25:07.259 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T20:25:07.259 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:25:07.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T20:25:07.270 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T20:25:07.270 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-18T20:25:07.270 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T20:25:07.294 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-18T20:25:07.294 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49bdc51142759622a281d3e5327eebd07cbb8bb3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T20:25:07.299 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-18T20:25:07.299 DEBUG:teuthology.orchestra.run.smithi073:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49bdc51142759622a281d3e5327eebd07cbb8bb3/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T20:25:07.336 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T20:25:07.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T20:25:07.373 DEBUG:teuthology.orchestra.run.smithi073:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T20:25:07.404 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-03-18T20:25:07.405 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-03-18T20:25:07.451 DEBUG:teuthology.orchestra.run.smithi073:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T20:25:07.459 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T20:25:07.490 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-03-18T20:25:07.492 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-03-18T20:25:07.518 INFO:teuthology.orchestra.run.smithi073.stdout:check_obsoletes = 1 2024-03-18T20:25:07.519 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum clean all 2024-03-18T20:25:07.787 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2024-03-18T20:25:07.895 INFO:teuthology.orchestra.run.smithi073.stdout:Updating Subscription Management repositories. 2024-03-18T20:25:07.930 INFO:teuthology.orchestra.run.smithi046.stdout:Updating Subscription Management repositories. 2024-03-18T20:25:09.096 INFO:teuthology.orchestra.run.smithi155.stdout:61 files removed 2024-03-18T20:25:09.150 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-18T20:25:09.306 INFO:teuthology.orchestra.run.smithi073.stdout:61 files removed 2024-03-18T20:25:09.357 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-18T20:25:09.475 INFO:teuthology.orchestra.run.smithi046.stdout:61 files removed 2024-03-18T20:25:09.492 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2024-03-18T20:25:09.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-18T20:25:09.696 INFO:teuthology.orchestra.run.smithi073.stdout:Updating Subscription Management repositories. 2024-03-18T20:25:09.917 INFO:teuthology.orchestra.run.smithi046.stdout:Updating Subscription Management repositories. 2024-03-18T20:25:11.162 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 221 kB/s | 80 kB 00:00 2024-03-18T20:25:11.443 INFO:teuthology.orchestra.run.smithi073.stdout:ceph packages for x86_64 267 kB/s | 80 kB 00:00 2024-03-18T20:25:11.480 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 35 kB/s | 9.9 kB 00:00 2024-03-18T20:25:11.520 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 270 kB/s | 80 kB 00:00 2024-03-18T20:25:11.764 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 5.8 kB/s | 1.5 kB 00:00 2024-03-18T20:25:11.788 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 44 kB/s | 9.9 kB 00:00 2024-03-18T20:25:11.836 INFO:teuthology.orchestra.run.smithi073.stdout:ceph noarch packages 28 kB/s | 9.9 kB 00:00 2024-03-18T20:25:11.906 INFO:teuthology.orchestra.run.smithi155.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2024-03-18T20:25:12.138 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 4.7 kB/s | 1.5 kB 00:00 2024-03-18T20:25:12.186 INFO:teuthology.orchestra.run.smithi073.stdout:ceph source packages 4.6 kB/s | 1.5 kB 00:00 2024-03-18T20:25:12.281 INFO:teuthology.orchestra.run.smithi046.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2024-03-18T20:25:12.337 INFO:teuthology.orchestra.run.smithi073.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2024-03-18T20:25:13.467 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2024-03-18T20:25:13.746 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2024-03-18T20:25:17.238 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 534 kB/s | 24 kB 00:00 2024-03-18T20:25:17.431 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 409 kB/s | 24 kB 00:00 2024-03-18T20:25:18.015 INFO:teuthology.orchestra.run.smithi073.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-03-18T20:25:18.116 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 15 MB/s | 8.8 MB 00:00 2024-03-18T20:25:20.899 INFO:teuthology.orchestra.run.smithi073.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 66 MB 00:01 2024-03-18T20:25:21.543 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 38 MB/s | 66 MB 00:01 2024-03-18T20:25:22.623 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 1.5 MB/s | 16 MB 00:10 2024-03-18T20:25:26.207 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 428 kB/s | 24 kB 00:00 2024-03-18T20:25:26.926 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-03-18T20:25:29.702 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 52 MB/s | 66 MB 00:01 2024-03-18T20:25:31.775 INFO:teuthology.orchestra.run.smithi073.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 59 MB 00:01 2024-03-18T20:25:33.611 INFO:teuthology.orchestra.run.smithi046.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 59 MB 00:01 2024-03-18T20:25:40.594 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 MB/s | 59 MB 00:01 2024-03-18T20:25:46.995 INFO:teuthology.orchestra.run.smithi073.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-18T20:25:46.996 INFO:teuthology.orchestra.run.smithi073.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-18T20:25:47.014 INFO:teuthology.orchestra.run.smithi073.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-18T20:25:47.015 INFO:teuthology.orchestra.run.smithi073.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-18T20:25:47.017 INFO:teuthology.orchestra.run.smithi073.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-18T20:25:47.018 INFO:teuthology.orchestra.run.smithi073.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-18T20:25:47.021 INFO:teuthology.orchestra.run.smithi073.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-18T20:25:47.025 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-18T20:25:47.026 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-18T20:25:47.245 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-03-18T20:25:47.259 INFO:teuthology.orchestra.run.smithi073.stdout:=================================================================================================================================== 2024-03-18T20:25:47.259 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-03-18T20:25:47.259 INFO:teuthology.orchestra.run.smithi073.stdout:=================================================================================================================================== 2024-03-18T20:25:47.259 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-03-18T20:25:47.260 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-18T20:25:47.260 INFO:teuthology.orchestra.run.smithi073.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-18T20:25:47.260 INFO:teuthology.orchestra.run.smithi073.stdout: ceph x86_64 2:18.2.2-914.g49bdc511.el8 ceph 6.4 k 2024-03-18T20:25:47.260 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base x86_64 2:18.2.2-914.g49bdc511.el8 ceph 5.4 M 2024-03-18T20:25:47.260 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse x86_64 2:18.2.2-914.g49bdc511.el8 ceph 854 k 2024-03-18T20:25:47.260 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-914.g49bdc511.el8 ceph 165 k 2024-03-18T20:25:47.260 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr x86_64 2:18.2.2-914.g49bdc511.el8 ceph 1.6 M 2024-03-18T20:25:47.260 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 138 k 2024-03-18T20:25:47.260 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 1.7 M 2024-03-18T20:25:47.260 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 7.4 M 2024-03-18T20:25:47.261 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 49 k 2024-03-18T20:25:47.261 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw x86_64 2:18.2.2-914.g49bdc511.el8 ceph 13 M 2024-03-18T20:25:47.261 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test x86_64 2:18.2.2-914.g49bdc511.el8 ceph 47 M 2024-03-18T20:25:47.261 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 263 k 2024-03-18T20:25:47.261 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 172 k 2024-03-18T20:25:47.261 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror x86_64 2:18.2.2-914.g49bdc511.el8 ceph 261 k 2024-03-18T20:25:47.261 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 18 k 2024-03-18T20:25:47.261 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-18T20:25:47.261 INFO:teuthology.orchestra.run.smithi073.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-18T20:25:47.261 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-18T20:25:47.262 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel x86_64 2:18.2.2-914.g49bdc511.el8 ceph 31 k 2024-03-18T20:25:47.262 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 775 k 2024-03-18T20:25:47.262 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel x86_64 2:18.2.2-914.g49bdc511.el8 ceph 139 k 2024-03-18T20:25:47.262 INFO:teuthology.orchestra.run.smithi073.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-18T20:25:47.262 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-18T20:25:47.262 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-18T20:25:47.262 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-18T20:25:47.262 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-18T20:25:47.262 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-18T20:25:47.262 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs x86_64 2:18.2.2-914.g49bdc511.el8 ceph 196 k 2024-03-18T20:25:47.263 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados x86_64 2:18.2.2-914.g49bdc511.el8 ceph 370 k 2024-03-18T20:25:47.263 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd x86_64 2:18.2.2-914.g49bdc511.el8 ceph 365 k 2024-03-18T20:25:47.263 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw x86_64 2:18.2.2-914.g49bdc511.el8 ceph 107 k 2024-03-18T20:25:47.263 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse x86_64 2:18.2.2-914.g49bdc511.el8 ceph 95 k 2024-03-18T20:25:47.263 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror x86_64 2:18.2.2-914.g49bdc511.el8 ceph 3.5 M 2024-03-18T20:25:47.263 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd x86_64 2:18.2.2-914.g49bdc511.el8 ceph 182 k 2024-03-18T20:25:47.263 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-18T20:25:47.263 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrading: 2024-03-18T20:25:47.263 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-18T20:25:47.264 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-18T20:25:47.264 INFO:teuthology.orchestra.run.smithi073.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-18T20:25:47.264 INFO:teuthology.orchestra.run.smithi073.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-18T20:25:47.264 INFO:teuthology.orchestra.run.smithi073.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-18T20:25:47.264 INFO:teuthology.orchestra.run.smithi073.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-18T20:25:47.264 INFO:teuthology.orchestra.run.smithi073.stdout: librados2 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 3.7 M 2024-03-18T20:25:47.264 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 3.6 M 2024-03-18T20:25:47.264 INFO:teuthology.orchestra.run.smithi073.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-18T20:25:47.264 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-18T20:25:47.264 INFO:teuthology.orchestra.run.smithi073.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-18T20:25:47.264 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-18T20:25:47.265 INFO:teuthology.orchestra.run.smithi073.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-18T20:25:47.265 INFO:teuthology.orchestra.run.smithi073.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-18T20:25:47.265 INFO:teuthology.orchestra.run.smithi073.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-18T20:25:47.265 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-18T20:25:47.265 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-03-18T20:25:47.265 INFO:teuthology.orchestra.run.smithi073.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-18T20:25:47.265 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common x86_64 2:18.2.2-914.g49bdc511.el8 ceph 22 M 2024-03-18T20:25:47.265 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 23 k 2024-03-18T20:25:47.265 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds x86_64 2:18.2.2-914.g49bdc511.el8 ceph 2.3 M 2024-03-18T20:25:47.265 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 245 k 2024-03-18T20:25:47.266 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon x86_64 2:18.2.2-914.g49bdc511.el8 ceph 4.6 M 2024-03-18T20:25:47.266 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd x86_64 2:18.2.2-914.g49bdc511.el8 ceph 18 M 2024-03-18T20:25:47.266 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 15 k 2024-03-18T20:25:47.266 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux x86_64 2:18.2.2-914.g49bdc511.el8 ceph 24 k 2024-03-18T20:25:47.266 INFO:teuthology.orchestra.run.smithi073.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-18T20:25:47.266 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-18T20:25:47.266 INFO:teuthology.orchestra.run.smithi073.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-18T20:25:47.266 INFO:teuthology.orchestra.run.smithi073.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-18T20:25:47.266 INFO:teuthology.orchestra.run.smithi073.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-18T20:25:47.266 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite x86_64 2:18.2.2-914.g49bdc511.el8 ceph 142 k 2024-03-18T20:25:47.266 INFO:teuthology.orchestra.run.smithi073.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-18T20:25:47.267 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-18T20:25:47.267 INFO:teuthology.orchestra.run.smithi073.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-18T20:25:47.267 INFO:teuthology.orchestra.run.smithi073.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-18T20:25:47.267 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-18T20:25:47.267 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-18T20:25:47.267 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 474 k 2024-03-18T20:25:47.267 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-18T20:25:47.267 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 6.5 M 2024-03-18T20:25:47.267 INFO:teuthology.orchestra.run.smithi073.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-18T20:25:47.267 INFO:teuthology.orchestra.run.smithi073.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-18T20:25:47.267 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-18T20:25:47.267 INFO:teuthology.orchestra.run.smithi073.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-18T20:25:47.268 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse x86_64 2:18.2.2-914.g49bdc511.el8 ceph 45 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common x86_64 2:18.2.2-914.g49bdc511.el8 ceph 126 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-18T20:25:47.269 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-18T20:25:47.270 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-18T20:25:47.270 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-18T20:25:47.270 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-18T20:25:47.270 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-18T20:25:47.270 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-18T20:25:47.270 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-18T20:25:47.270 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-18T20:25:47.270 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-18T20:25:47.270 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-18T20:25:47.270 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-18T20:25:47.270 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-18T20:25:47.271 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-18T20:25:47.272 INFO:teuthology.orchestra.run.smithi073.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-18T20:25:47.273 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-18T20:25:47.273 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-18T20:25:47.273 INFO:teuthology.orchestra.run.smithi073.stdout:Enabling module streams: 2024-03-18T20:25:47.273 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql 10 2024-03-18T20:25:47.273 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:25:47.273 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-03-18T20:25:47.273 INFO:teuthology.orchestra.run.smithi073.stdout:=================================================================================================================================== 2024-03-18T20:25:47.273 INFO:teuthology.orchestra.run.smithi073.stdout:Install 130 Packages 2024-03-18T20:25:47.273 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrade 16 Packages 2024-03-18T20:25:47.273 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:25:47.274 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 212 M 2024-03-18T20:25:47.274 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-03-18T20:25:47.517 INFO:teuthology.orchestra.run.smithi073.stdout:(1/146): ceph-18.2.2-914.g49bdc511.el8.x86_64.r 31 kB/s | 6.4 kB 00:00 2024-03-18T20:25:47.884 INFO:teuthology.orchestra.run.smithi073.stdout:(2/146): ceph-fuse-18.2.2-914.g49bdc511.el8.x86 2.3 MB/s | 854 kB 00:00 2024-03-18T20:25:47.976 INFO:teuthology.orchestra.run.smithi073.stdout:(3/146): ceph-immutable-object-cache-18.2.2-914 1.8 MB/s | 165 kB 00:00 2024-03-18T20:25:48.059 INFO:teuthology.orchestra.run.smithi073.stdout:(4/146): ceph-base-18.2.2-914.g49bdc511.el8.x86 7.2 MB/s | 5.4 MB 00:00 2024-03-18T20:25:48.251 INFO:teuthology.orchestra.run.smithi073.stdout:(5/146): ceph-mgr-18.2.2-914.g49bdc511.el8.x86_ 8.1 MB/s | 1.6 MB 00:00 2024-03-18T20:25:48.478 INFO:teuthology.orchestra.run.smithi073.stdout:(6/146): ceph-mds-18.2.2-914.g49bdc511.el8.x86_ 4.6 MB/s | 2.3 MB 00:00 2024-03-18T20:25:48.802 INFO:teuthology.orchestra.run.smithi073.stdout:(7/146): ceph-mon-18.2.2-914.g49bdc511.el8.x86_ 8.4 MB/s | 4.6 MB 00:00 2024-03-18T20:25:49.294 INFO:teuthology.orchestra.run.smithi073.stdout:(8/146): ceph-common-18.2.2-914.g49bdc511.el8.x 11 MB/s | 22 MB 00:01 2024-03-18T20:25:49.587 INFO:teuthology.orchestra.run.smithi073.stdout:(9/146): ceph-selinux-18.2.2-914.g49bdc511.el8. 83 kB/s | 24 kB 00:00 2024-03-18T20:25:50.094 INFO:teuthology.orchestra.run.smithi073.stdout:(10/146): ceph-radosgw-18.2.2-914.g49bdc511.el8 10 MB/s | 13 MB 00:01 2024-03-18T20:25:50.186 INFO:teuthology.orchestra.run.smithi073.stdout:(11/146): cephfs-mirror-18.2.2-914.g49bdc511.el 2.8 MB/s | 261 kB 00:00 2024-03-18T20:25:50.286 INFO:teuthology.orchestra.run.smithi073.stdout:(12/146): libcephfs-devel-18.2.2-914.g49bdc511. 310 kB/s | 31 kB 00:00 2024-03-18T20:25:50.420 INFO:teuthology.orchestra.run.smithi073.stdout:(13/146): libcephfs2-18.2.2-914.g49bdc511.el8.x 5.7 MB/s | 775 kB 00:00 2024-03-18T20:25:50.537 INFO:teuthology.orchestra.run.smithi073.stdout:(14/146): libcephsqlite-18.2.2-914.g49bdc511.el 1.2 MB/s | 142 kB 00:00 2024-03-18T20:25:50.654 INFO:teuthology.orchestra.run.smithi073.stdout:(15/146): librados-devel-18.2.2-914.g49bdc511.e 1.2 MB/s | 139 kB 00:00 2024-03-18T20:25:50.700 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-18T20:25:50.702 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-18T20:25:50.726 INFO:teuthology.orchestra.run.smithi046.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-18T20:25:50.726 INFO:teuthology.orchestra.run.smithi046.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-18T20:25:50.726 INFO:teuthology.orchestra.run.smithi046.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-18T20:25:50.727 INFO:teuthology.orchestra.run.smithi046.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-18T20:25:50.731 INFO:teuthology.orchestra.run.smithi046.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-18T20:25:50.735 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-18T20:25:50.737 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-18T20:25:50.762 INFO:teuthology.orchestra.run.smithi073.stdout:(16/146): libradosstriper1-18.2.2-914.g49bdc511 4.3 MB/s | 474 kB 00:00 2024-03-18T20:25:50.988 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-03-18T20:25:51.004 INFO:teuthology.orchestra.run.smithi046.stdout:=================================================================================================================================== 2024-03-18T20:25:51.004 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-03-18T20:25:51.004 INFO:teuthology.orchestra.run.smithi046.stdout:=================================================================================================================================== 2024-03-18T20:25:51.004 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-03-18T20:25:51.004 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-18T20:25:51.004 INFO:teuthology.orchestra.run.smithi046.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-18T20:25:51.004 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:18.2.2-914.g49bdc511.el8 ceph 6.4 k 2024-03-18T20:25:51.004 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:18.2.2-914.g49bdc511.el8 ceph 5.4 M 2024-03-18T20:25:51.004 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:18.2.2-914.g49bdc511.el8 ceph 854 k 2024-03-18T20:25:51.004 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-914.g49bdc511.el8 ceph 165 k 2024-03-18T20:25:51.004 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:18.2.2-914.g49bdc511.el8 ceph 1.6 M 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 138 k 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 1.7 M 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 7.4 M 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 49 k 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:18.2.2-914.g49bdc511.el8 ceph 13 M 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:18.2.2-914.g49bdc511.el8 ceph 47 M 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 263 k 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 172 k 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror x86_64 2:18.2.2-914.g49bdc511.el8 ceph 261 k 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 18 k 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-18T20:25:51.005 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:18.2.2-914.g49bdc511.el8 ceph 31 k 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 775 k 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:18.2.2-914.g49bdc511.el8 ceph 139 k 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:18.2.2-914.g49bdc511.el8 ceph 196 k 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:18.2.2-914.g49bdc511.el8 ceph 370 k 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:18.2.2-914.g49bdc511.el8 ceph 365 k 2024-03-18T20:25:51.006 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:18.2.2-914.g49bdc511.el8 ceph 107 k 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:18.2.2-914.g49bdc511.el8 ceph 95 k 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:18.2.2-914.g49bdc511.el8 ceph 3.5 M 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:18.2.2-914.g49bdc511.el8 ceph 182 k 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 3.7 M 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 3.6 M 2024-03-18T20:25:51.007 INFO:teuthology.orchestra.run.smithi046.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:18.2.2-914.g49bdc511.el8 ceph 22 M 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 23 k 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:18.2.2-914.g49bdc511.el8 ceph 2.3 M 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 245 k 2024-03-18T20:25:51.008 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:18.2.2-914.g49bdc511.el8 ceph 4.6 M 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:18.2.2-914.g49bdc511.el8 ceph 18 M 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 15 k 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:18.2.2-914.g49bdc511.el8 ceph 24 k 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:18.2.2-914.g49bdc511.el8 ceph 142 k 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-18T20:25:51.009 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 474 k 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 6.5 M 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-18T20:25:51.010 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:18.2.2-914.g49bdc511.el8 ceph 45 k 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:18.2.2-914.g49bdc511.el8 ceph 126 k 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-18T20:25:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-18T20:25:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-18T20:25:51.013 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-18T20:25:51.014 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling module streams: 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql 10 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:=================================================================================================================================== 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:Install 130 Packages 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 16 Packages 2024-03-18T20:25:51.015 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 212 M 2024-03-18T20:25:51.016 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-03-18T20:25:51.586 INFO:teuthology.orchestra.run.smithi046.stdout:(1/146): ceph-18.2.2-914.g49bdc511.el8.x86_64.r 18 kB/s | 6.4 kB 00:00 2024-03-18T20:25:51.804 INFO:teuthology.orchestra.run.smithi073.stdout:(17/146): librgw2-18.2.2-914.g49bdc511.el8.x86_ 6.3 MB/s | 6.5 MB 00:01 2024-03-18T20:25:51.819 INFO:teuthology.orchestra.run.smithi046.stdout:(2/146): ceph-base-18.2.2-914.g49bdc511.el8.x86 9.2 MB/s | 5.4 MB 00:00 2024-03-18T20:25:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:(3/146): ceph-immutable-object-cache-18.2.2-914 3.9 MB/s | 165 kB 00:00 2024-03-18T20:25:51.980 INFO:teuthology.orchestra.run.smithi073.stdout:(18/146): ceph-osd-18.2.2-914.g49bdc511.el8.x86 5.1 MB/s | 18 MB 00:03 2024-03-18T20:25:52.005 INFO:teuthology.orchestra.run.smithi073.stdout:(19/146): python3-ceph-argparse-18.2.2-914.g49b 223 kB/s | 45 kB 00:00 2024-03-18T20:25:52.028 INFO:teuthology.orchestra.run.smithi046.stdout:(4/146): ceph-mds-18.2.2-914.g49bdc511.el8.x86_ 14 MB/s | 2.3 MB 00:00 2024-03-18T20:25:52.055 INFO:teuthology.orchestra.run.smithi073.stdout:(20/146): python3-ceph-common-18.2.2-914.g49bdc 1.6 MB/s | 126 kB 00:00 2024-03-18T20:25:52.079 INFO:teuthology.orchestra.run.smithi046.stdout:(5/146): ceph-fuse-18.2.2-914.g49bdc511.el8.x86 1.7 MB/s | 854 kB 00:00 2024-03-18T20:25:52.080 INFO:teuthology.orchestra.run.smithi073.stdout:(21/146): python3-cephfs-18.2.2-914.g49bdc511.e 2.5 MB/s | 196 kB 00:00 2024-03-18T20:25:52.171 INFO:teuthology.orchestra.run.smithi046.stdout:(6/146): ceph-mgr-18.2.2-914.g49bdc511.el8.x86_ 11 MB/s | 1.6 MB 00:00 2024-03-18T20:25:52.231 INFO:teuthology.orchestra.run.smithi073.stdout:(22/146): python3-rados-18.2.2-914.g49bdc511.el 2.1 MB/s | 370 kB 00:00 2024-03-18T20:25:52.914 INFO:teuthology.orchestra.run.smithi073.stdout:(23/146): python3-rgw-18.2.2-914.g49bdc511.el8. 156 kB/s | 107 kB 00:00 2024-03-18T20:25:52.939 INFO:teuthology.orchestra.run.smithi073.stdout:(24/146): python3-rbd-18.2.2-914.g49bdc511.el8. 425 kB/s | 365 kB 00:00 2024-03-18T20:25:53.031 INFO:teuthology.orchestra.run.smithi073.stdout:(25/146): rbd-fuse-18.2.2-914.g49bdc511.el8.x86 813 kB/s | 95 kB 00:00 2024-03-18T20:25:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:(26/146): rbd-nbd-18.2.2-914.g49bdc511.el8.x86_ 1.3 MB/s | 182 kB 00:00 2024-03-18T20:25:53.204 INFO:teuthology.orchestra.run.smithi046.stdout:(7/146): ceph-mon-18.2.2-914.g49bdc511.el8.x86_ 4.1 MB/s | 4.6 MB 00:01 2024-03-18T20:25:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:(27/146): ceph-grafana-dashboards-18.2.2-914.g4 279 kB/s | 23 kB 00:00 2024-03-18T20:25:53.332 INFO:teuthology.orchestra.run.smithi073.stdout:(28/146): ceph-mgr-cephadm-18.2.2-914.g49bdc511 1.6 MB/s | 138 kB 00:00 2024-03-18T20:25:53.380 INFO:teuthology.orchestra.run.smithi046.stdout:(8/146): ceph-osd-18.2.2-914.g49bdc511.el8.x86_ 15 MB/s | 18 MB 00:01 2024-03-18T20:25:53.447 INFO:teuthology.orchestra.run.smithi046.stdout:(9/146): ceph-selinux-18.2.2-914.g49bdc511.el8. 367 kB/s | 24 kB 00:00 2024-03-18T20:25:53.999 INFO:teuthology.orchestra.run.smithi073.stdout:(29/146): ceph-mgr-dashboard-18.2.2-914.g49bdc5 2.6 MB/s | 1.7 MB 00:00 2024-03-18T20:25:54.933 INFO:teuthology.orchestra.run.smithi073.stdout:(30/146): rbd-mirror-18.2.2-914.g49bdc511.el8.x 1.7 MB/s | 3.5 MB 00:01 2024-03-18T20:25:55.075 INFO:teuthology.orchestra.run.smithi073.stdout:(31/146): ceph-mgr-modules-core-18.2.2-914.g49b 1.7 MB/s | 245 kB 00:00 2024-03-18T20:25:55.175 INFO:teuthology.orchestra.run.smithi073.stdout:(32/146): ceph-mgr-rook-18.2.2-914.g49bdc511.el 492 kB/s | 49 kB 00:00 2024-03-18T20:25:55.280 INFO:teuthology.orchestra.run.smithi046.stdout:(10/146): ceph-radosgw-18.2.2-914.g49bdc511.el8 6.3 MB/s | 13 MB 00:02 2024-03-18T20:25:55.364 INFO:teuthology.orchestra.run.smithi046.stdout:(11/146): cephfs-mirror-18.2.2-914.g49bdc511.el 3.1 MB/s | 261 kB 00:00 2024-03-18T20:25:55.430 INFO:teuthology.orchestra.run.smithi046.stdout:(12/146): libcephfs-devel-18.2.2-914.g49bdc511. 466 kB/s | 31 kB 00:00 2024-03-18T20:25:55.572 INFO:teuthology.orchestra.run.smithi046.stdout:(13/146): libcephfs2-18.2.2-914.g49bdc511.el8.x 5.4 MB/s | 775 kB 00:00 2024-03-18T20:25:55.584 INFO:teuthology.orchestra.run.smithi073.stdout:(33/146): ceph-test-18.2.2-914.g49bdc511.el8.x8 7.8 MB/s | 47 MB 00:05 2024-03-18T20:25:55.609 INFO:teuthology.orchestra.run.smithi073.stdout:(34/146): ceph-prometheus-alerts-18.2.2-914.g49 34 kB/s | 15 kB 00:00 2024-03-18T20:25:55.673 INFO:teuthology.orchestra.run.smithi046.stdout:(14/146): libcephsqlite-18.2.2-914.g49bdc511.el 1.4 MB/s | 142 kB 00:00 2024-03-18T20:25:55.701 INFO:teuthology.orchestra.run.smithi073.stdout:(35/146): cephadm-18.2.2-914.g49bdc511.el8.noar 1.8 MB/s | 172 kB 00:00 2024-03-18T20:25:55.726 INFO:teuthology.orchestra.run.smithi073.stdout:(36/146): ceph-volume-18.2.2-914.g49bdc511.el8. 1.8 MB/s | 263 kB 00:00 2024-03-18T20:25:55.764 INFO:teuthology.orchestra.run.smithi046.stdout:(15/146): librados-devel-18.2.2-914.g49bdc511.e 1.5 MB/s | 139 kB 00:00 2024-03-18T20:25:55.784 INFO:teuthology.orchestra.run.smithi073.stdout:(37/146): cephfs-top-18.2.2-914.g49bdc511.el8.n 217 kB/s | 18 kB 00:00 2024-03-18T20:25:55.835 INFO:teuthology.orchestra.run.smithi073.stdout:(38/146): gperftools-libs-2.7-9.el8.x86_64.rpm 2.8 MB/s | 306 kB 00:00 2024-03-18T20:25:55.860 INFO:teuthology.orchestra.run.smithi073.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 781 kB/s | 59 kB 00:00 2024-03-18T20:25:55.870 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-18T20:25:55.872 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-18T20:25:55.873 INFO:teuthology.orchestra.run.smithi046.stdout:(16/146): libradosstriper1-18.2.2-914.g49bdc511 4.3 MB/s | 474 kB 00:00 2024-03-18T20:25:55.885 INFO:teuthology.orchestra.run.smithi073.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2024-03-18T20:25:55.890 INFO:teuthology.orchestra.run.smithi155.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-18T20:25:55.892 INFO:teuthology.orchestra.run.smithi155.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-18T20:25:55.894 INFO:teuthology.orchestra.run.smithi155.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-18T20:25:55.895 INFO:teuthology.orchestra.run.smithi155.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-18T20:25:55.898 INFO:teuthology.orchestra.run.smithi155.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-18T20:25:55.902 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-18T20:25:55.904 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-18T20:25:55.911 INFO:teuthology.orchestra.run.smithi073.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.7 MB/s | 44 kB 00:00 2024-03-18T20:25:55.936 INFO:teuthology.orchestra.run.smithi073.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 6.2 MB/s | 476 kB 00:00 2024-03-18T20:25:55.961 INFO:teuthology.orchestra.run.smithi073.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2024-03-18T20:25:55.987 INFO:teuthology.orchestra.run.smithi073.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 656 kB/s | 33 kB 00:00 2024-03-18T20:25:56.004 INFO:teuthology.orchestra.run.smithi073.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 390 kB/s | 16 kB 00:00 2024-03-18T20:25:56.029 INFO:teuthology.orchestra.run.smithi073.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 4.0 MB/s | 173 kB 00:00 2024-03-18T20:25:56.055 INFO:teuthology.orchestra.run.smithi073.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 7.5 MB/s | 384 kB 00:00 2024-03-18T20:25:56.079 INFO:teuthology.orchestra.run.smithi073.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2024-03-18T20:25:56.105 INFO:teuthology.orchestra.run.smithi073.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2024-03-18T20:25:56.119 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-18T20:25:56.130 INFO:teuthology.orchestra.run.smithi073.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2024-03-18T20:25:56.133 INFO:teuthology.orchestra.run.smithi155.stdout:=================================================================================================================================== 2024-03-18T20:25:56.133 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-18T20:25:56.133 INFO:teuthology.orchestra.run.smithi155.stdout:=================================================================================================================================== 2024-03-18T20:25:56.133 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-18T20:25:56.133 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-18T20:25:56.133 INFO:teuthology.orchestra.run.smithi155.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-18T20:25:56.133 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:18.2.2-914.g49bdc511.el8 ceph 6.4 k 2024-03-18T20:25:56.133 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:18.2.2-914.g49bdc511.el8 ceph 5.4 M 2024-03-18T20:25:56.134 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:18.2.2-914.g49bdc511.el8 ceph 854 k 2024-03-18T20:25:56.134 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-914.g49bdc511.el8 ceph 165 k 2024-03-18T20:25:56.134 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:18.2.2-914.g49bdc511.el8 ceph 1.6 M 2024-03-18T20:25:56.134 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 138 k 2024-03-18T20:25:56.134 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 1.7 M 2024-03-18T20:25:56.134 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 7.4 M 2024-03-18T20:25:56.134 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 49 k 2024-03-18T20:25:56.134 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:18.2.2-914.g49bdc511.el8 ceph 13 M 2024-03-18T20:25:56.134 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:18.2.2-914.g49bdc511.el8 ceph 47 M 2024-03-18T20:25:56.134 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 263 k 2024-03-18T20:25:56.134 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 172 k 2024-03-18T20:25:56.134 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror x86_64 2:18.2.2-914.g49bdc511.el8 ceph 261 k 2024-03-18T20:25:56.135 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 18 k 2024-03-18T20:25:56.135 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-18T20:25:56.135 INFO:teuthology.orchestra.run.smithi155.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-18T20:25:56.135 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-18T20:25:56.135 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:18.2.2-914.g49bdc511.el8 ceph 31 k 2024-03-18T20:25:56.135 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 775 k 2024-03-18T20:25:56.135 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:18.2.2-914.g49bdc511.el8 ceph 139 k 2024-03-18T20:25:56.135 INFO:teuthology.orchestra.run.smithi155.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-18T20:25:56.135 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-18T20:25:56.135 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-18T20:25:56.135 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-18T20:25:56.135 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:18.2.2-914.g49bdc511.el8 ceph 196 k 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:18.2.2-914.g49bdc511.el8 ceph 370 k 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:18.2.2-914.g49bdc511.el8 ceph 365 k 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:18.2.2-914.g49bdc511.el8 ceph 107 k 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:18.2.2-914.g49bdc511.el8 ceph 95 k 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:18.2.2-914.g49bdc511.el8 ceph 3.5 M 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:18.2.2-914.g49bdc511.el8 ceph 182 k 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-18T20:25:56.136 INFO:teuthology.orchestra.run.smithi155.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 3.7 M 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 3.6 M 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-18T20:25:56.137 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-18T20:25:56.138 INFO:teuthology.orchestra.run.smithi155.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-18T20:25:56.138 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:18.2.2-914.g49bdc511.el8 ceph 22 M 2024-03-18T20:25:56.138 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 23 k 2024-03-18T20:25:56.138 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:18.2.2-914.g49bdc511.el8 ceph 2.3 M 2024-03-18T20:25:56.138 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 245 k 2024-03-18T20:25:56.138 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:18.2.2-914.g49bdc511.el8 ceph 4.6 M 2024-03-18T20:25:56.138 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:18.2.2-914.g49bdc511.el8 ceph 18 M 2024-03-18T20:25:56.138 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:18.2.2-914.g49bdc511.el8 ceph-noarch 15 k 2024-03-18T20:25:56.138 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:18.2.2-914.g49bdc511.el8 ceph 24 k 2024-03-18T20:25:56.138 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-18T20:25:56.138 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-18T20:25:56.139 INFO:teuthology.orchestra.run.smithi155.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-18T20:25:56.139 INFO:teuthology.orchestra.run.smithi155.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-18T20:25:56.139 INFO:teuthology.orchestra.run.smithi155.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-18T20:25:56.139 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:18.2.2-914.g49bdc511.el8 ceph 142 k 2024-03-18T20:25:56.139 INFO:teuthology.orchestra.run.smithi155.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-18T20:25:56.139 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-18T20:25:56.139 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-18T20:25:56.139 INFO:teuthology.orchestra.run.smithi155.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-18T20:25:56.139 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-18T20:25:56.139 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-18T20:25:56.139 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 474 k 2024-03-18T20:25:56.139 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:18.2.2-914.g49bdc511.el8 ceph 6.5 M 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-18T20:25:56.140 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:18.2.2-914.g49bdc511.el8 ceph 45 k 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:18.2.2-914.g49bdc511.el8 ceph 126 k 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-18T20:25:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-18T20:25:56.142 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-18T20:25:56.143 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-18T20:25:56.144 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-18T20:25:56.145 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-18T20:25:56.145 INFO:teuthology.orchestra.run.smithi155.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-18T20:25:56.145 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-18T20:25:56.145 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-18T20:25:56.145 INFO:teuthology.orchestra.run.smithi155.stdout:Enabling module streams: 2024-03-18T20:25:56.145 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql 10 2024-03-18T20:25:56.145 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:25:56.145 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-18T20:25:56.145 INFO:teuthology.orchestra.run.smithi155.stdout:=================================================================================================================================== 2024-03-18T20:25:56.145 INFO:teuthology.orchestra.run.smithi155.stdout:Install 130 Packages 2024-03-18T20:25:56.145 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 16 Packages 2024-03-18T20:25:56.145 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:25:56.146 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 212 M 2024-03-18T20:25:56.146 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-18T20:25:56.164 INFO:teuthology.orchestra.run.smithi073.stdout:(51/146): python3-kubernetes-11.0.0-6.el8.noarc 20 MB/s | 1.2 MB 00:00 2024-03-18T20:25:56.264 INFO:teuthology.orchestra.run.smithi073.stdout:(52/146): ceph-mgr-diskprediction-local-18.2.2- 3.3 MB/s | 7.4 MB 00:02 2024-03-18T20:25:56.289 INFO:teuthology.orchestra.run.smithi073.stdout:(53/146): python3-logutils-0.3.5-11.el8.noarch. 307 kB/s | 49 kB 00:00 2024-03-18T20:25:56.315 INFO:teuthology.orchestra.run.smithi073.stdout:(54/146): python3-more-itertools-7.2.0-3.el8.no 390 kB/s | 59 kB 00:00 2024-03-18T20:25:56.340 INFO:teuthology.orchestra.run.smithi073.stdout:(55/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2024-03-18T20:25:56.365 INFO:teuthology.orchestra.run.smithi073.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2024-03-18T20:25:56.390 INFO:teuthology.orchestra.run.smithi073.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 672 kB/s | 34 kB 00:00 2024-03-18T20:25:56.424 INFO:teuthology.orchestra.run.smithi073.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2024-03-18T20:25:56.449 INFO:teuthology.orchestra.run.smithi073.stdout:(59/146): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2024-03-18T20:25:56.475 INFO:teuthology.orchestra.run.smithi073.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 372 kB/s | 19 kB 00:00 2024-03-18T20:25:56.500 INFO:teuthology.orchestra.run.smithi073.stdout:(61/146): python3-rsa-4.9-2.el8.noarch.rpm 587 kB/s | 64 kB 00:00 2024-03-18T20:25:56.510 INFO:teuthology.orchestra.run.smithi155.stdout:(1/146): ceph-18.2.2-914.g49bdc511.el8.x86_64.r 29 kB/s | 6.4 kB 00:00 2024-03-18T20:25:56.525 INFO:teuthology.orchestra.run.smithi073.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 318 kB/s | 24 kB 00:00 2024-03-18T20:25:56.550 INFO:teuthology.orchestra.run.smithi073.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 388 kB/s | 29 kB 00:00 2024-03-18T20:25:56.576 INFO:teuthology.orchestra.run.smithi073.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 549 kB/s | 27 kB 00:00 2024-03-18T20:25:56.601 INFO:teuthology.orchestra.run.smithi073.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2024-03-18T20:25:56.626 INFO:teuthology.orchestra.run.smithi073.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2024-03-18T20:25:56.676 INFO:teuthology.orchestra.run.smithi073.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 613 kB/s | 61 kB 00:00 2024-03-18T20:25:56.699 INFO:teuthology.orchestra.run.smithi046.stdout:(17/146): librgw2-18.2.2-914.g49bdc511.el8.x86_ 7.9 MB/s | 6.5 MB 00:00 2024-03-18T20:25:56.702 INFO:teuthology.orchestra.run.smithi073.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2024-03-18T20:25:56.719 INFO:teuthology.orchestra.run.smithi073.stdout:(69/146): python3-webtest-2.0.33-1.el8.noarch.r 728 kB/s | 85 kB 00:00 2024-03-18T20:25:56.744 INFO:teuthology.orchestra.run.smithi073.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2024-03-18T20:25:56.786 INFO:teuthology.orchestra.run.smithi073.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 16 MB/s | 1.7 MB 00:00 2024-03-18T20:25:56.815 INFO:teuthology.orchestra.run.smithi046.stdout:(18/146): python3-ceph-argparse-18.2.2-914.g49b 384 kB/s | 45 kB 00:00 2024-03-18T20:25:56.869 INFO:teuthology.orchestra.run.smithi155.stdout:(2/146): ceph-fuse-18.2.2-914.g49bdc511.el8.x86 2.3 MB/s | 854 kB 00:00 2024-03-18T20:25:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:(72/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 92 kB/s | 18 kB 00:00 2024-03-18T20:25:56.961 INFO:teuthology.orchestra.run.smithi155.stdout:(3/146): ceph-immutable-object-cache-18.2.2-914 1.8 MB/s | 165 kB 00:00 2024-03-18T20:25:56.970 INFO:teuthology.orchestra.run.smithi073.stdout:(73/146): python3-jwt-1.6.1-2.el8.noarch.rpm 239 kB/s | 43 kB 00:00 2024-03-18T20:25:56.982 INFO:teuthology.orchestra.run.smithi046.stdout:(19/146): python3-ceph-common-18.2.2-914.g49bdc 754 kB/s | 126 kB 00:00 2024-03-18T20:25:56.995 INFO:teuthology.orchestra.run.smithi073.stdout:(74/146): python3-packaging-16.8-9.el8.noarch.r 175 kB/s | 48 kB 00:00 2024-03-18T20:25:57.054 INFO:teuthology.orchestra.run.smithi073.stdout:(75/146): python3-oauthlib-2.1.0-1.el8.noarch.r 1.4 MB/s | 155 kB 00:00 2024-03-18T20:25:57.121 INFO:teuthology.orchestra.run.smithi073.stdout:(76/146): python3-pycparser-2.14-14.el8.noarch. 725 kB/s | 109 kB 00:00 2024-03-18T20:25:57.146 INFO:teuthology.orchestra.run.smithi073.stdout:(77/146): python3-requests-oauthlib-1.0.0-1.el8 284 kB/s | 43 kB 00:00 2024-03-18T20:25:57.188 INFO:teuthology.orchestra.run.smithi073.stdout:(78/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 139 kB/s | 19 kB 00:00 2024-03-18T20:25:57.214 INFO:teuthology.orchestra.run.smithi155.stdout:(4/146): ceph-base-18.2.2-914.g49bdc511.el8.x86 5.8 MB/s | 5.4 MB 00:00 2024-03-18T20:25:57.247 INFO:teuthology.orchestra.run.smithi073.stdout:(79/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 350 kB/s | 44 kB 00:00 2024-03-18T20:25:57.286 INFO:teuthology.orchestra.run.smithi155.stdout:(5/146): ceph-mds-18.2.2-914.g49bdc511.el8.x86_ 7.0 MB/s | 2.3 MB 00:00 2024-03-18T20:25:57.297 INFO:teuthology.orchestra.run.smithi073.stdout:(80/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2024-03-18T20:25:57.339 INFO:teuthology.orchestra.run.smithi073.stdout:(81/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2024-03-18T20:25:57.364 INFO:teuthology.orchestra.run.smithi073.stdout:(82/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2024-03-18T20:25:57.395 INFO:teuthology.orchestra.run.smithi155.stdout:(6/146): ceph-mgr-18.2.2-914.g49bdc511.el8.x86_ 8.6 MB/s | 1.6 MB 00:00 2024-03-18T20:25:57.431 INFO:teuthology.orchestra.run.smithi073.stdout:(83/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.5 MB/s | 200 kB 00:00 2024-03-18T20:25:57.481 INFO:teuthology.orchestra.run.smithi073.stdout:(84/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 338 kB/s | 48 kB 00:00 2024-03-18T20:25:57.531 INFO:teuthology.orchestra.run.smithi073.stdout:(85/146): libsepol-devel-2.9-3.el8.x86_64.rpm 520 kB/s | 87 kB 00:00 2024-03-18T20:25:57.607 INFO:teuthology.orchestra.run.smithi073.stdout:(86/146): libcom_err-devel-1.45.6-4.el8.x86_64. 308 kB/s | 39 kB 00:00 2024-03-18T20:25:57.623 INFO:teuthology.orchestra.run.smithi073.stdout:(87/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 299 kB/s | 58 kB 00:00 2024-03-18T20:25:57.641 INFO:teuthology.orchestra.run.smithi073.stdout:(88/146): fuse-2.9.7-17.el8.x86_64.rpm 761 kB/s | 83 kB 00:00 2024-03-18T20:25:57.642 INFO:teuthology.orchestra.run.smithi046.stdout:(20/146): ceph-test-18.2.2-914.g49bdc511.el8.x8 11 MB/s | 47 MB 00:04 2024-03-18T20:25:57.666 INFO:teuthology.orchestra.run.smithi046.stdout:(21/146): python3-cephfs-18.2.2-914.g49bdc511.e 287 kB/s | 196 kB 00:00 2024-03-18T20:25:57.724 INFO:teuthology.orchestra.run.smithi073.stdout:(89/146): libgfortran-8.5.0-20.el8.x86_64.rpm 5.4 MB/s | 645 kB 00:00 2024-03-18T20:25:57.745 INFO:teuthology.orchestra.run.smithi155.stdout:(7/146): ceph-mon-18.2.2-914.g49bdc511.el8.x86_ 10 MB/s | 4.6 MB 00:00 2024-03-18T20:25:57.758 INFO:teuthology.orchestra.run.smithi073.stdout:(90/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-03-18T20:25:57.783 INFO:teuthology.orchestra.run.smithi073.stdout:(91/146): libuuid-devel-2.32.1-43.el8.x86_64.rp 701 kB/s | 100 kB 00:00 2024-03-18T20:25:57.784 INFO:teuthology.orchestra.run.smithi046.stdout:(22/146): python3-rados-18.2.2-914.g49bdc511.el 2.6 MB/s | 370 kB 00:00 2024-03-18T20:25:57.809 INFO:teuthology.orchestra.run.smithi046.stdout:(23/146): python3-rbd-18.2.2-914.g49bdc511.el8. 2.5 MB/s | 365 kB 00:00 2024-03-18T20:25:57.842 INFO:teuthology.orchestra.run.smithi073.stdout:(92/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.0 MB/s | 238 kB 00:00 2024-03-18T20:25:57.859 INFO:teuthology.orchestra.run.smithi046.stdout:(24/146): python3-rgw-18.2.2-914.g49bdc511.el8. 1.4 MB/s | 107 kB 00:00 2024-03-18T20:25:57.901 INFO:teuthology.orchestra.run.smithi073.stdout:(93/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 511 kB/s | 73 kB 00:00 2024-03-18T20:25:57.917 INFO:teuthology.orchestra.run.smithi046.stdout:(25/146): rbd-fuse-18.2.2-914.g49bdc511.el8.x86 877 kB/s | 95 kB 00:00 2024-03-18T20:25:57.926 INFO:teuthology.orchestra.run.smithi073.stdout:(94/146): NetworkManager-initscripts-updown-1.4 1.0 MB/s | 144 kB 00:00 2024-03-18T20:25:57.960 INFO:teuthology.orchestra.run.smithi073.stdout:(95/146): elfutils-libelf-devel-0.189-3.el8.x86 526 kB/s | 62 kB 00:00 2024-03-18T20:25:58.010 INFO:teuthology.orchestra.run.smithi073.stdout:(96/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 5.0 MB/s | 562 kB 00:00 2024-03-18T20:25:58.035 INFO:teuthology.orchestra.run.smithi046.stdout:(26/146): rbd-nbd-18.2.2-914.g49bdc511.el8.x86_ 1.5 MB/s | 182 kB 00:00 2024-03-18T20:25:58.043 INFO:teuthology.orchestra.run.smithi073.stdout:(97/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 563 kB/s | 47 kB 00:00 2024-03-18T20:25:58.119 INFO:teuthology.orchestra.run.smithi073.stdout:(98/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-03-18T20:25:58.143 INFO:teuthology.orchestra.run.smithi046.stdout:(27/146): ceph-grafana-dashboards-18.2.2-914.g4 215 kB/s | 23 kB 00:00 2024-03-18T20:25:58.152 INFO:teuthology.orchestra.run.smithi073.stdout:(99/146): python3-cryptography-3.2.1-7.el8_9.x8 3.8 MB/s | 559 kB 00:00 2024-03-18T20:25:58.236 INFO:teuthology.orchestra.run.smithi073.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 104 kB/s | 20 kB 00:00 2024-03-18T20:25:58.251 INFO:teuthology.orchestra.run.smithi046.stdout:(28/146): ceph-mgr-cephadm-18.2.2-914.g49bdc511 1.2 MB/s | 138 kB 00:00 2024-03-18T20:25:58.261 INFO:teuthology.orchestra.run.smithi073.stdout:(101/146): python3-pyasn1-modules-0.3.7-6.el8.n 1.0 MB/s | 110 kB 00:00 2024-03-18T20:25:58.286 INFO:teuthology.orchestra.run.smithi073.stdout:(102/146): python3-prettytable-0.7.2-14.el8.noa 263 kB/s | 44 kB 00:00 2024-03-18T20:25:58.345 INFO:teuthology.orchestra.run.smithi073.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 244 kB/s | 27 kB 00:00 2024-03-18T20:25:58.379 INFO:teuthology.orchestra.run.smithi073.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.1 MB/s | 126 kB 00:00 2024-03-18T20:25:58.412 INFO:teuthology.orchestra.run.smithi073.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 3.6 MB/s | 457 kB 00:00 2024-03-18T20:25:58.454 INFO:teuthology.orchestra.run.smithi073.stdout:(106/146): bison-3.0.4-10.el8.x86_64.rpm 6.2 MB/s | 688 kB 00:00 2024-03-18T20:25:58.513 INFO:teuthology.orchestra.run.smithi073.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2024-03-18T20:25:58.555 INFO:teuthology.orchestra.run.smithi073.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 4.9 MB/s | 709 kB 00:00 2024-03-18T20:25:58.597 INFO:teuthology.orchestra.run.smithi073.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 273 kB/s | 39 kB 00:00 2024-03-18T20:25:58.630 INFO:teuthology.orchestra.run.smithi073.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2024-03-18T20:25:58.706 INFO:teuthology.orchestra.run.smithi073.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 425 kB/s | 64 kB 00:00 2024-03-18T20:25:58.735 INFO:teuthology.orchestra.run.smithi046.stdout:(29/146): ceph-mgr-dashboard-18.2.2-914.g49bdc5 3.6 MB/s | 1.7 MB 00:00 2024-03-18T20:25:58.739 INFO:teuthology.orchestra.run.smithi073.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 729 kB/s | 103 kB 00:00 2024-03-18T20:25:58.823 INFO:teuthology.orchestra.run.smithi073.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 4.5 MB/s | 538 kB 00:00 2024-03-18T20:25:58.948 INFO:teuthology.orchestra.run.smithi073.stdout:(114/146): python3-numpy-1.14.3-10.el8.x86_64.r 18 MB/s | 3.7 MB 00:00 2024-03-18T20:25:58.977 INFO:teuthology.orchestra.run.smithi046.stdout:(30/146): rbd-mirror-18.2.2-914.g49bdc511.el8.x 3.1 MB/s | 3.5 MB 00:01 2024-03-18T20:25:58.982 INFO:teuthology.orchestra.run.smithi073.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.0 MB/s | 323 kB 00:00 2024-03-18T20:25:59.057 INFO:teuthology.orchestra.run.smithi073.stdout:(116/146): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2024-03-18T20:25:59.074 INFO:teuthology.orchestra.run.smithi073.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.8 MB/s | 225 kB 00:00 2024-03-18T20:25:59.091 INFO:teuthology.orchestra.run.smithi073.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2024-03-18T20:25:59.102 INFO:teuthology.orchestra.run.smithi046.stdout:(31/146): ceph-mgr-modules-core-18.2.2-914.g49b 1.9 MB/s | 245 kB 00:00 2024-03-18T20:25:59.166 INFO:teuthology.orchestra.run.smithi073.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2024-03-18T20:25:59.177 INFO:teuthology.orchestra.run.smithi046.stdout:(32/146): ceph-mgr-rook-18.2.2-914.g49bdc511.el 658 kB/s | 49 kB 00:00 2024-03-18T20:25:59.204 INFO:teuthology.orchestra.run.smithi155.stdout:(8/146): ceph-radosgw-18.2.2-914.g49bdc511.el8. 9.0 MB/s | 13 MB 00:01 2024-03-18T20:25:59.219 INFO:teuthology.orchestra.run.smithi046.stdout:(33/146): ceph-prometheus-alerts-18.2.2-914.g49 354 kB/s | 15 kB 00:00 2024-03-18T20:25:59.278 INFO:teuthology.orchestra.run.smithi046.stdout:(34/146): ceph-volume-18.2.2-914.g49bdc511.el8. 4.4 MB/s | 263 kB 00:00 2024-03-18T20:25:59.325 INFO:teuthology.orchestra.run.smithi073.stdout:(120/146): openblas-threads-0.3.15-6.el8.x86_64 19 MB/s | 5.2 MB 00:00 2024-03-18T20:25:59.328 INFO:teuthology.orchestra.run.smithi046.stdout:(35/146): cephadm-18.2.2-914.g49bdc511.el8.noar 3.4 MB/s | 172 kB 00:00 2024-03-18T20:25:59.371 INFO:teuthology.orchestra.run.smithi046.stdout:(36/146): cephfs-top-18.2.2-914.g49bdc511.el8.n 432 kB/s | 18 kB 00:00 2024-03-18T20:25:59.371 INFO:teuthology.orchestra.run.smithi155.stdout:(9/146): ceph-osd-18.2.2-914.g49bdc511.el8.x86_ 9.0 MB/s | 18 MB 00:01 2024-03-18T20:25:59.392 INFO:teuthology.orchestra.run.smithi073.stdout:(121/146): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-03-18T20:25:59.396 INFO:teuthology.orchestra.run.smithi155.stdout:(10/146): ceph-selinux-18.2.2-914.g49bdc511.el8 127 kB/s | 24 kB 00:00 2024-03-18T20:25:59.434 INFO:teuthology.orchestra.run.smithi073.stdout:(122/146): automake-1.16.1-8.el8.noarch.rpm 2.6 MB/s | 714 kB 00:00 2024-03-18T20:25:59.455 INFO:teuthology.orchestra.run.smithi155.stdout:(11/146): cephfs-mirror-18.2.2-914.g49bdc511.el 4.4 MB/s | 261 kB 00:00 2024-03-18T20:25:59.490 INFO:teuthology.orchestra.run.smithi073.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2024-03-18T20:25:59.505 INFO:teuthology.orchestra.run.smithi155.stdout:(12/146): libcephfs-devel-18.2.2-914.g49bdc511. 619 kB/s | 31 kB 00:00 2024-03-18T20:25:59.518 INFO:teuthology.orchestra.run.smithi073.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2024-03-18T20:25:59.601 INFO:teuthology.orchestra.run.smithi073.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 324 kB/s | 54 kB 00:00 2024-03-18T20:25:59.622 INFO:teuthology.orchestra.run.smithi155.stdout:(13/146): libcephfs2-18.2.2-914.g49bdc511.el8.x 6.5 MB/s | 775 kB 00:00 2024-03-18T20:25:59.668 INFO:teuthology.orchestra.run.smithi073.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-03-18T20:25:59.681 INFO:teuthology.orchestra.run.smithi155.stdout:(14/146): libcephsqlite-18.2.2-914.g49bdc511.el 2.4 MB/s | 142 kB 00:00 2024-03-18T20:25:59.704 INFO:teuthology.orchestra.run.smithi046.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 919 kB/s | 306 kB 00:00 2024-03-18T20:25:59.739 INFO:teuthology.orchestra.run.smithi155.stdout:(15/146): librados-devel-18.2.2-914.g49bdc511.e 2.3 MB/s | 139 kB 00:00 2024-03-18T20:25:59.779 INFO:teuthology.orchestra.run.smithi046.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 787 kB/s | 59 kB 00:00 2024-03-18T20:25:59.802 INFO:teuthology.orchestra.run.smithi073.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 7.5 MB/s | 1.5 MB 00:00 2024-03-18T20:25:59.838 INFO:teuthology.orchestra.run.smithi155.stdout:(16/146): libradosstriper1-18.2.2-914.g49bdc511 4.7 MB/s | 474 kB 00:00 2024-03-18T20:25:59.895 INFO:teuthology.orchestra.run.smithi046.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 649 kB/s | 75 kB 00:00 2024-03-18T20:25:59.962 INFO:teuthology.orchestra.run.smithi073.stdout:(128/146): python3-scipy-1.0.0-21.module+el8.9. 29 MB/s | 14 MB 00:00 2024-03-18T20:26:00.002 INFO:teuthology.orchestra.run.smithi073.stdout:(129/146): postgresql-contrib-10.23-4.module+el 2.4 MB/s | 812 kB 00:00 2024-03-18T20:26:00.080 INFO:teuthology.orchestra.run.smithi046.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.5 MB/s | 476 kB 00:00 2024-03-18T20:26:00.122 INFO:teuthology.orchestra.run.smithi046.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2024-03-18T20:26:00.136 INFO:teuthology.orchestra.run.smithi073.stdout:(130/146): postgresql-server-10.23-4.module+el8 15 MB/s | 5.1 MB 00:00 2024-03-18T20:26:00.189 INFO:teuthology.orchestra.run.smithi046.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2024-03-18T20:26:00.231 INFO:teuthology.orchestra.run.smithi046.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 796 kB/s | 33 kB 00:00 2024-03-18T20:26:00.273 INFO:teuthology.orchestra.run.smithi046.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 393 kB/s | 16 kB 00:00 2024-03-18T20:26:00.320 INFO:teuthology.orchestra.run.smithi073.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.2 MB/s | 232 kB 00:00 2024-03-18T20:26:00.340 INFO:teuthology.orchestra.run.smithi046.stdout:(45/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2024-03-18T20:26:00.432 INFO:teuthology.orchestra.run.smithi046.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2024-03-18T20:26:00.466 INFO:teuthology.orchestra.run.smithi046.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 2.4 MB/s | 81 kB 00:00 2024-03-18T20:26:00.470 INFO:teuthology.orchestra.run.smithi073.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.0 MB/s | 303 kB 00:00 2024-03-18T20:26:00.508 INFO:teuthology.orchestra.run.smithi046.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 267 kB/s | 11 kB 00:00 2024-03-18T20:26:00.532 INFO:teuthology.orchestra.run.smithi155.stdout:(17/146): librgw2-18.2.2-914.g49bdc511.el8.x86_ 9.4 MB/s | 6.5 MB 00:00 2024-03-18T20:26:00.578 INFO:teuthology.orchestra.run.smithi073.stdout:(133/146): libmount-2.32.1-43.el8.x86_64.rpm 2.1 MB/s | 237 kB 00:00 2024-03-18T20:26:00.616 INFO:teuthology.orchestra.run.smithi046.stdout:(49/146): ceph-mgr-diskprediction-local-18.2.2- 3.9 MB/s | 7.4 MB 00:01 2024-03-18T20:26:00.624 INFO:teuthology.orchestra.run.smithi155.stdout:(18/146): python3-ceph-argparse-18.2.2-914.g49b 488 kB/s | 45 kB 00:00 2024-03-18T20:26:00.633 INFO:teuthology.orchestra.run.smithi046.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2024-03-18T20:26:00.682 INFO:teuthology.orchestra.run.smithi155.stdout:(19/146): python3-ceph-common-18.2.2-914.g49bdc 2.1 MB/s | 126 kB 00:00 2024-03-18T20:26:00.700 INFO:teuthology.orchestra.run.smithi046.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 731 kB/s | 49 kB 00:00 2024-03-18T20:26:00.720 INFO:teuthology.orchestra.run.smithi073.stdout:(134/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.2 MB/s | 179 kB 00:00 2024-03-18T20:26:00.758 INFO:teuthology.orchestra.run.smithi155.stdout:(20/146): python3-cephfs-18.2.2-914.g49bdc511.e 2.6 MB/s | 196 kB 00:00 2024-03-18T20:26:00.776 INFO:teuthology.orchestra.run.smithi046.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 784 kB/s | 59 kB 00:00 2024-03-18T20:26:00.821 INFO:teuthology.orchestra.run.smithi073.stdout:(135/146): libuuid-2.32.1-43.el8.x86_64.rpm 991 kB/s | 99 kB 00:00 2024-03-18T20:26:00.841 INFO:teuthology.orchestra.run.smithi155.stdout:(21/146): python3-rados-18.2.2-914.g49bdc511.el 4.3 MB/s | 370 kB 00:00 2024-03-18T20:26:00.842 INFO:teuthology.orchestra.run.smithi046.stdout:(53/146): python3-natsort-7.1.1-2.el8.noarch.rp 904 kB/s | 60 kB 00:00 2024-03-18T20:26:00.933 INFO:teuthology.orchestra.run.smithi155.stdout:(22/146): python3-rbd-18.2.2-914.g49bdc511.el8. 3.9 MB/s | 365 kB 00:00 2024-03-18T20:26:00.988 INFO:teuthology.orchestra.run.smithi073.stdout:(136/146): util-linux-2.32.1-43.el8.x86_64.rpm 15 MB/s | 2.5 MB 00:00 2024-03-18T20:26:01.000 INFO:teuthology.orchestra.run.smithi155.stdout:(23/146): python3-rgw-18.2.2-914.g49bdc511.el8. 1.6 MB/s | 107 kB 00:00 2024-03-18T20:26:01.067 INFO:teuthology.orchestra.run.smithi155.stdout:(24/146): rbd-fuse-18.2.2-914.g49bdc511.el8.x86 1.4 MB/s | 95 kB 00:00 2024-03-18T20:26:01.154 INFO:teuthology.orchestra.run.smithi073.stdout:(137/146): util-linux-user-2.32.1-43.el8.x86_64 617 kB/s | 103 kB 00:00 2024-03-18T20:26:01.159 INFO:teuthology.orchestra.run.smithi046.stdout:(54/146): ceph-common-18.2.2-914.g49bdc511.el8. 2.2 MB/s | 22 MB 00:09 2024-03-18T20:26:01.185 INFO:teuthology.orchestra.run.smithi046.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 827 kB/s | 283 kB 00:00 2024-03-18T20:26:01.243 INFO:teuthology.orchestra.run.smithi046.stdout:(56/146): python3-repoze-lru-0.7-6.el8.noarch.r 580 kB/s | 34 kB 00:00 2024-03-18T20:26:01.280 INFO:teuthology.orchestra.run.smithi073.stdout:(138/146): librbd1-18.2.2-914.g49bdc511.el8.x86 2.8 MB/s | 3.6 MB 00:01 2024-03-18T20:26:01.322 INFO:teuthology.orchestra.run.smithi073.stdout:(139/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.0 MB/s | 854 kB 00:00 2024-03-18T20:26:01.369 INFO:teuthology.orchestra.run.smithi046.stdout:(57/146): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2024-03-18T20:26:01.380 INFO:teuthology.orchestra.run.smithi073.stdout:(140/146): libblkid-2.32.1-43.el8.x86_64.rpm 2.2 MB/s | 220 kB 00:00 2024-03-18T20:26:01.394 INFO:teuthology.orchestra.run.smithi046.stdout:(58/146): python3-portend-2.6-1.el8.noarch.rpm 67 kB/s | 16 kB 00:00 2024-03-18T20:26:01.409 INFO:teuthology.orchestra.run.smithi155.stdout:(25/146): rbd-mirror-18.2.2-914.g49bdc511.el8.x 10 MB/s | 3.5 MB 00:00 2024-03-18T20:26:01.427 INFO:teuthology.orchestra.run.smithi046.stdout:(59/146): python3-kubernetes-11.0.0-6.el8.noarc 1.4 MB/s | 1.2 MB 00:00 2024-03-18T20:26:01.468 INFO:teuthology.orchestra.run.smithi155.stdout:(26/146): rbd-nbd-18.2.2-914.g49bdc511.el8.x86_ 3.1 MB/s | 182 kB 00:00 2024-03-18T20:26:01.478 INFO:teuthology.orchestra.run.smithi046.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 224 kB/s | 19 kB 00:00 2024-03-18T20:26:01.489 INFO:teuthology.orchestra.run.smithi073.stdout:(141/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.5 MB/s | 254 kB 00:00 2024-03-18T20:26:01.511 INFO:teuthology.orchestra.run.smithi046.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 286 kB/s | 24 kB 00:00 2024-03-18T20:26:01.527 INFO:teuthology.orchestra.run.smithi155.stdout:(27/146): ceph-grafana-dashboards-18.2.2-914.g4 399 kB/s | 23 kB 00:00 2024-03-18T20:26:01.531 INFO:teuthology.orchestra.run.smithi073.stdout:(142/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.6 MB/s | 711 kB 00:00 2024-03-18T20:26:01.553 INFO:teuthology.orchestra.run.smithi046.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 388 kB/s | 29 kB 00:00 2024-03-18T20:26:01.585 INFO:teuthology.orchestra.run.smithi155.stdout:(28/146): ceph-mgr-cephadm-18.2.2-914.g49bdc511 2.3 MB/s | 138 kB 00:00 2024-03-18T20:26:01.595 INFO:teuthology.orchestra.run.smithi046.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 329 kB/s | 27 kB 00:00 2024-03-18T20:26:01.623 INFO:teuthology.orchestra.run.smithi073.stdout:(143/146): librados2-18.2.2-914.g49bdc511.el8.x 2.2 MB/s | 3.7 MB 00:01 2024-03-18T20:26:01.648 INFO:teuthology.orchestra.run.smithi073.stdout:(144/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.8 MB/s | 333 kB 00:00 2024-03-18T20:26:01.687 INFO:teuthology.orchestra.run.smithi046.stdout:(64/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2024-03-18T20:26:01.690 INFO:teuthology.orchestra.run.smithi073.stdout:(145/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.3 MB/s | 1.5 MB 00:00 2024-03-18T20:26:01.704 INFO:teuthology.orchestra.run.smithi046.stdout:(65/146): python3-rsa-4.9-2.el8.noarch.rpm 191 kB/s | 64 kB 00:00 2024-03-18T20:26:01.729 INFO:teuthology.orchestra.run.smithi046.stdout:(66/146): python3-waitress-1.4.3-1.el8.noarch.r 1.4 MB/s | 255 kB 00:00 2024-03-18T20:26:01.755 INFO:teuthology.orchestra.run.smithi046.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 911 kB/s | 61 kB 00:00 2024-03-18T20:26:01.777 INFO:teuthology.orchestra.run.smithi155.stdout:(29/146): ceph-mgr-dashboard-18.2.2-914.g49bdc5 9.1 MB/s | 1.7 MB 00:00 2024-03-18T20:26:01.780 INFO:teuthology.orchestra.run.smithi046.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2024-03-18T20:26:01.781 INFO:teuthology.orchestra.run.smithi073.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.7 MB/s | 1.1 MB 00:00 2024-03-18T20:26:01.785 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-03-18T20:26:01.785 INFO:teuthology.orchestra.run.smithi073.stdout:Total 15 MB/s | 212 MB 00:14 2024-03-18T20:26:01.805 INFO:teuthology.orchestra.run.smithi046.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2024-03-18T20:26:01.822 INFO:teuthology.orchestra.run.smithi046.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2024-03-18T20:26:02.006 INFO:teuthology.orchestra.run.smithi046.stdout:(71/146): python3-packaging-16.8-9.el8.noarch.r 241 kB/s | 48 kB 00:00 2024-03-18T20:26:02.098 INFO:teuthology.orchestra.run.smithi046.stdout:(72/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 67 kB/s | 18 kB 00:00 2024-03-18T20:26:02.265 INFO:teuthology.orchestra.run.smithi046.stdout:(73/146): thrift-0.13.0-2.el8.x86_64.rpm 3.4 MB/s | 1.7 MB 00:00 2024-03-18T20:26:02.290 INFO:teuthology.orchestra.run.smithi046.stdout:(74/146): python3-jwt-1.6.1-2.el8.noarch.rpm 150 kB/s | 43 kB 00:00 2024-03-18T20:26:02.324 INFO:teuthology.orchestra.run.smithi046.stdout:(75/146): python3-oauthlib-2.1.0-1.el8.noarch.r 687 kB/s | 155 kB 00:00 2024-03-18T20:26:02.388 INFO:teuthology.orchestra.run.smithi155.stdout:(30/146): ceph-mgr-diskprediction-local-18.2.2- 12 MB/s | 7.4 MB 00:00 2024-03-18T20:26:02.416 INFO:teuthology.orchestra.run.smithi046.stdout:(76/146): python3-pycparser-2.14-14.el8.noarch. 726 kB/s | 109 kB 00:00 2024-03-18T20:26:02.441 INFO:teuthology.orchestra.run.smithi046.stdout:(77/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 160 kB/s | 19 kB 00:00 2024-03-18T20:26:02.453 INFO:teuthology.orchestra.run.smithi155.stdout:(31/146): ceph-mgr-modules-core-18.2.2-914.g49b 3.7 MB/s | 245 kB 00:00 2024-03-18T20:26:02.491 INFO:teuthology.orchestra.run.smithi046.stdout:(78/146): python3-requests-oauthlib-1.0.0-1.el8 222 kB/s | 43 kB 00:00 2024-03-18T20:26:02.512 INFO:teuthology.orchestra.run.smithi155.stdout:(32/146): ceph-mgr-rook-18.2.2-914.g49bdc511.el 845 kB/s | 49 kB 00:00 2024-03-18T20:26:02.550 INFO:teuthology.orchestra.run.smithi046.stdout:(79/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 328 kB/s | 44 kB 00:00 2024-03-18T20:26:02.570 INFO:teuthology.orchestra.run.smithi155.stdout:(33/146): ceph-prometheus-alerts-18.2.2-914.g49 254 kB/s | 15 kB 00:00 2024-03-18T20:26:02.584 INFO:teuthology.orchestra.run.smithi046.stdout:(80/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2024-03-18T20:26:02.634 INFO:teuthology.orchestra.run.smithi046.stdout:(81/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.1 MB/s | 605 kB 00:00 2024-03-18T20:26:02.637 INFO:teuthology.orchestra.run.smithi155.stdout:(34/146): ceph-volume-18.2.2-914.g49bdc511.el8. 3.9 MB/s | 263 kB 00:00 2024-03-18T20:26:02.704 INFO:teuthology.orchestra.run.smithi155.stdout:(35/146): cephadm-18.2.2-914.g49bdc511.el8.noar 2.5 MB/s | 172 kB 00:00 2024-03-18T20:26:02.717 INFO:teuthology.orchestra.run.smithi046.stdout:(82/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2024-03-18T20:26:02.743 INFO:teuthology.orchestra.run.smithi046.stdout:(83/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.2 MB/s | 200 kB 00:00 2024-03-18T20:26:02.754 INFO:teuthology.orchestra.run.smithi155.stdout:(36/146): cephfs-top-18.2.2-914.g49bdc511.el8.n 363 kB/s | 18 kB 00:00 2024-03-18T20:26:02.811 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-03-18T20:26:02.834 INFO:teuthology.orchestra.run.smithi046.stdout:(84/146): libsepol-devel-2.9-3.el8.x86_64.rpm 744 kB/s | 87 kB 00:00 2024-03-18T20:26:02.860 INFO:teuthology.orchestra.run.smithi046.stdout:(85/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 213 kB/s | 48 kB 00:00 2024-03-18T20:26:02.885 INFO:teuthology.orchestra.run.smithi046.stdout:(86/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 405 kB/s | 58 kB 00:00 2024-03-18T20:26:02.969 INFO:teuthology.orchestra.run.smithi046.stdout:(87/146): fuse-2.9.7-17.el8.x86_64.rpm 764 kB/s | 83 kB 00:00 2024-03-18T20:26:02.994 INFO:teuthology.orchestra.run.smithi046.stdout:(88/146): libcom_err-devel-1.45.6-4.el8.x86_64. 242 kB/s | 39 kB 00:00 2024-03-18T20:26:03.036 INFO:teuthology.orchestra.run.smithi046.stdout:(89/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.2 MB/s | 645 kB 00:00 2024-03-18T20:26:03.095 INFO:teuthology.orchestra.run.smithi046.stdout:(90/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-03-18T20:26:03.161 INFO:teuthology.orchestra.run.smithi046.stdout:(91/146): libuuid-devel-2.32.1-43.el8.x86_64.rp 596 kB/s | 100 kB 00:00 2024-03-18T20:26:03.187 INFO:teuthology.orchestra.run.smithi046.stdout:(92/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-03-18T20:26:03.188 INFO:teuthology.orchestra.run.smithi155.stdout:(37/146): ceph-test-18.2.2-914.g49bdc511.el8.x8 12 MB/s | 47 MB 00:03 2024-03-18T20:26:03.202 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-03-18T20:26:03.202 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-03-18T20:26:03.221 INFO:teuthology.orchestra.run.smithi046.stdout:(93/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 577 kB/s | 73 kB 00:00 2024-03-18T20:26:03.304 INFO:teuthology.orchestra.run.smithi046.stdout:(94/146): elfutils-libelf-devel-0.189-3.el8.x86 527 kB/s | 62 kB 00:00 2024-03-18T20:26:03.330 INFO:teuthology.orchestra.run.smithi046.stdout:(95/146): NetworkManager-initscripts-updown-1.4 859 kB/s | 144 kB 00:00 2024-03-18T20:26:03.408 INFO:teuthology.orchestra.run.smithi046.stdout:(96/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 2.9 MB/s | 562 kB 00:00 2024-03-18T20:26:03.417 INFO:teuthology.orchestra.run.smithi155.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-nyc.osuosl.org/pub/fedora-epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 443: Network is unreachable] 2024-03-18T20:26:03.447 INFO:teuthology.orchestra.run.smithi046.stdout:(97/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 403 kB/s | 47 kB 00:00 2024-03-18T20:26:03.522 INFO:teuthology.orchestra.run.smithi046.stdout:(98/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-03-18T20:26:03.556 INFO:teuthology.orchestra.run.smithi046.stdout:(99/146): python3-jsonpointer-1.10-11.el8.noarc 184 kB/s | 20 kB 00:00 2024-03-18T20:26:03.589 INFO:teuthology.orchestra.run.smithi046.stdout:(100/146): python3-cryptography-3.2.1-7.el8_9.x 3.0 MB/s | 559 kB 00:00 2024-03-18T20:26:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 437 kB/s | 44 kB 00:00 2024-03-18T20:26:03.706 INFO:teuthology.orchestra.run.smithi046.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 731 kB/s | 110 kB 00:00 2024-03-18T20:26:03.732 INFO:teuthology.orchestra.run.smithi046.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 187 kB/s | 27 kB 00:00 2024-03-18T20:26:03.799 INFO:teuthology.orchestra.run.smithi046.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 720 kB/s | 126 kB 00:00 2024-03-18T20:26:03.840 INFO:teuthology.orchestra.run.smithi046.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 3.3 MB/s | 457 kB 00:00 2024-03-18T20:26:03.891 INFO:teuthology.orchestra.run.smithi046.stdout:(106/146): bison-3.0.4-10.el8.x86_64.rpm 4.2 MB/s | 688 kB 00:00 2024-03-18T20:26:03.924 INFO:teuthology.orchestra.run.smithi046.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 2.5 MB/s | 320 kB 00:00 2024-03-18T20:26:03.983 INFO:teuthology.orchestra.run.smithi046.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 4.9 MB/s | 709 kB 00:00 2024-03-18T20:26:04.042 INFO:teuthology.orchestra.run.smithi046.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 258 kB/s | 39 kB 00:00 2024-03-18T20:26:04.067 INFO:teuthology.orchestra.run.smithi046.stdout:(110/146): uuid-1.6.2-43.el8.x86_64.rpm 766 kB/s | 64 kB 00:00 2024-03-18T20:26:04.109 INFO:teuthology.orchestra.run.smithi046.stdout:(111/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2024-03-18T20:26:04.159 INFO:teuthology.orchestra.run.smithi046.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 884 kB/s | 103 kB 00:00 2024-03-18T20:26:04.230 INFO:teuthology.orchestra.run.smithi155.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-nyc.osuosl.org/pub/fedora-epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 443: No route to host] 2024-03-18T20:26:04.368 INFO:teuthology.orchestra.run.smithi046.stdout:(113/146): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2024-03-18T20:26:04.392 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-03-18T20:26:04.401 INFO:teuthology.orchestra.run.smithi046.stdout:(114/146): python3-jinja2-2.10.1-3.el8.noarch.r 1.8 MB/s | 538 kB 00:00 2024-03-18T20:26:04.402 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-03-18T20:26:04.493 INFO:teuthology.orchestra.run.smithi046.stdout:(115/146): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2024-03-18T20:26:04.519 INFO:teuthology.orchestra.run.smithi046.stdout:(116/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.1 MB/s | 323 kB 00:00 2024-03-18T20:26:04.544 INFO:teuthology.orchestra.run.smithi046.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.5 MB/s | 225 kB 00:00 2024-03-18T20:26:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.3 MB/s | 354 kB 00:00 2024-03-18T20:26:04.694 INFO:teuthology.orchestra.run.smithi046.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2024-03-18T20:26:04.853 INFO:teuthology.orchestra.run.smithi046.stdout:(120/146): openblas-threads-0.3.15-6.el8.x86_64 16 MB/s | 5.2 MB 00:00 2024-03-18T20:26:04.895 INFO:teuthology.orchestra.run.smithi046.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 3.5 MB/s | 714 kB 00:00 2024-03-18T20:26:04.995 INFO:teuthology.orchestra.run.smithi046.stdout:(122/146): openblas-0.3.15-6.el8.x86_64.rpm 14 MB/s | 5.0 MB 00:00 2024-03-18T20:26:05.029 INFO:teuthology.orchestra.run.smithi046.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2024-03-18T20:26:05.063 INFO:teuthology.orchestra.run.smithi046.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-03-18T20:26:05.097 INFO:teuthology.orchestra.run.smithi155.stdout:(38/146): ceph-common-18.2.2-914.g49bdc511.el8. 2.5 MB/s | 22 MB 00:08 2024-03-18T20:26:05.121 INFO:teuthology.orchestra.run.smithi046.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 431 kB/s | 54 kB 00:00 2024-03-18T20:26:05.139 INFO:teuthology.orchestra.run.smithi155.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for https://ftp-nyc.osuosl.org/pub/fedora-epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 443: Network is unreachable] 2024-03-18T20:26:05.197 INFO:teuthology.orchestra.run.smithi046.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-03-18T20:26:05.272 INFO:teuthology.orchestra.run.smithi046.stdout:(127/146): postgresql-contrib-10.23-4.module+el 11 MB/s | 812 kB 00:00 2024-03-18T20:26:05.330 INFO:teuthology.orchestra.run.smithi046.stdout:(128/146): postgresql-10.23-4.module+el8.9.0+21 7.2 MB/s | 1.5 MB 00:00 2024-03-18T20:26:05.565 INFO:teuthology.orchestra.run.smithi046.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 26 MB/s | 14 MB 00:00 2024-03-18T20:26:05.698 INFO:teuthology.orchestra.run.smithi046.stdout:(130/146): postgresql-server-10.23-4.module+el8 12 MB/s | 5.1 MB 00:00 2024-03-18T20:26:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.2 MB/s | 232 kB 00:00 2024-03-18T20:26:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:(132/146): librados2-18.2.2-914.g49bdc511.el8.x 5.3 MB/s | 3.7 MB 00:00 2024-03-18T20:26:06.057 INFO:teuthology.orchestra.run.smithi046.stdout:(133/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2024-03-18T20:26:06.116 INFO:teuthology.orchestra.run.smithi046.stdout:(134/146): librbd1-18.2.2-914.g49bdc511.el8.x86 6.5 MB/s | 3.6 MB 00:00 2024-03-18T20:26:06.191 INFO:teuthology.orchestra.run.smithi046.stdout:(135/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.3 MB/s | 179 kB 00:00 2024-03-18T20:26:06.214 INFO:teuthology.orchestra.run.smithi155.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-nyc.osuosl.org/pub/fedora-epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: No route to host] 2024-03-18T20:26:06.216 INFO:teuthology.orchestra.run.smithi046.stdout:(136/146): libmount-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 237 kB 00:00 2024-03-18T20:26:06.274 INFO:teuthology.orchestra.run.smithi046.stdout:(137/146): libuuid-2.32.1-43.el8.x86_64.rpm 624 kB/s | 99 kB 00:00 2024-03-18T20:26:06.350 INFO:teuthology.orchestra.run.smithi046.stdout:(138/146): util-linux-user-2.32.1-43.el8.x86_64 770 kB/s | 103 kB 00:00 2024-03-18T20:26:06.408 INFO:teuthology.orchestra.run.smithi046.stdout:(139/146): util-linux-2.32.1-43.el8.x86_64.rpm 11 MB/s | 2.5 MB 00:00 2024-03-18T20:26:06.450 INFO:teuthology.orchestra.run.smithi046.stdout:(140/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.7 MB/s | 854 kB 00:00 2024-03-18T20:26:06.501 INFO:teuthology.orchestra.run.smithi046.stdout:(141/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-03-18T20:26:06.593 INFO:teuthology.orchestra.run.smithi046.stdout:(142/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.9 MB/s | 711 kB 00:00 2024-03-18T20:26:06.659 INFO:teuthology.orchestra.run.smithi046.stdout:(143/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.0 MB/s | 254 kB 00:00 2024-03-18T20:26:06.710 INFO:teuthology.orchestra.run.smithi046.stdout:(144/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.1 MB/s | 1.5 MB 00:00 2024-03-18T20:26:06.743 INFO:teuthology.orchestra.run.smithi046.stdout:(145/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.2 MB/s | 333 kB 00:00 2024-03-18T20:26:06.844 INFO:teuthology.orchestra.run.smithi046.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.8 MB/s | 1.1 MB 00:00 2024-03-18T20:26:06.849 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-03-18T20:26:06.849 INFO:teuthology.orchestra.run.smithi046.stdout:Total 13 MB/s | 212 MB 00:15 2024-03-18T20:26:07.147 INFO:teuthology.orchestra.run.smithi155.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 37 kB/s | 75 kB 00:02 2024-03-18T20:26:07.166 INFO:teuthology.orchestra.run.smithi155.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Curl error (7): Couldn't connect to server for http://ftp-nyc.osuosl.org/pub/fedora-epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: Network is unreachable] 2024-03-18T20:26:07.983 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-03-18T20:26:08.236 INFO:teuthology.orchestra.run.smithi155.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Curl error (7): Couldn't connect to server for https://ftp-nyc.osuosl.org/pub/fedora-epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm [Failed to connect to ftp-nyc.osuosl.org port 443: No route to host] 2024-03-18T20:26:08.430 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-03-18T20:26:08.431 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-03-18T20:26:08.555 INFO:teuthology.orchestra.run.smithi155.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 338 kB/s | 476 kB 00:01 2024-03-18T20:26:09.608 INFO:teuthology.orchestra.run.smithi155.stdout:[MIRROR] python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-nyc.osuosl.org/pub/fedora-epel/8/Everything/x86_64/Packages/p/python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: No route to host] 2024-03-18T20:26:09.740 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-03-18T20:26:09.751 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-03-18T20:26:09.814 INFO:teuthology.orchestra.run.smithi155.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 35 kB/s | 44 kB 00:01 2024-03-18T20:26:09.884 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-03-18T20:26:09.939 INFO:teuthology.orchestra.run.smithi155.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2024-03-18T20:26:10.018 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-18T20:26:10.039 INFO:teuthology.orchestra.run.smithi155.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2024-03-18T20:26:10.140 INFO:teuthology.orchestra.run.smithi155.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2024-03-18T20:26:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-18T20:26:10.256 INFO:teuthology.orchestra.run.smithi155.stdout:(45/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2024-03-18T20:26:10.406 INFO:teuthology.orchestra.run.smithi155.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.5 MB/s | 384 kB 00:00 2024-03-18T20:26:10.447 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-18T20:26:10.524 INFO:teuthology.orchestra.run.smithi155.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 696 kB/s | 81 kB 00:00 2024-03-18T20:26:10.566 INFO:teuthology.orchestra.run.smithi155.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-nyc.osuosl.org/pub/fedora-epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: No route to host] 2024-03-18T20:26:10.604 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-18T20:26:10.624 INFO:teuthology.orchestra.run.smithi155.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-03-18T20:26:10.724 INFO:teuthology.orchestra.run.smithi155.stdout:(49/146): python3-jaraco-functools-2.0-4.el8.no 176 kB/s | 18 kB 00:00 2024-03-18T20:26:10.803 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-18T20:26:10.950 INFO:teuthology.orchestra.run.smithi155.stdout:(50/146): python3-kubernetes-11.0.0-6.el8.noarc 5.2 MB/s | 1.2 MB 00:00 2024-03-18T20:26:10.990 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-18T20:26:11.042 INFO:teuthology.orchestra.run.smithi155.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 532 kB/s | 49 kB 00:00 2024-03-18T20:26:11.134 INFO:teuthology.orchestra.run.smithi155.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 640 kB/s | 59 kB 00:00 2024-03-18T20:26:11.159 INFO:teuthology.orchestra.run.smithi155.stdout:(53/146): gperftools-libs-2.7-9.el8.x86_64.rpm 36 kB/s | 306 kB 00:08 2024-03-18T20:26:11.234 INFO:teuthology.orchestra.run.smithi155.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 601 kB/s | 60 kB 00:00 2024-03-18T20:26:11.310 INFO:teuthology.orchestra.run.smithi155.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2024-03-18T20:26:11.336 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-18T20:26:11.343 INFO:teuthology.orchestra.run.smithi155.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2024-03-18T20:26:11.347 INFO:teuthology.orchestra.run.smithi155.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://ftp-nyc.osuosl.org/pub/fedora-epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm [Failed to connect to ftp-nyc.osuosl.org port 80: No route to host] 2024-03-18T20:26:11.410 INFO:teuthology.orchestra.run.smithi155.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 335 kB/s | 34 kB 00:00 2024-03-18T20:26:11.412 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-18T20:26:11.502 INFO:teuthology.orchestra.run.smithi155.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2024-03-18T20:26:11.527 INFO:teuthology.orchestra.run.smithi155.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 548 kB/s | 64 kB 00:00 2024-03-18T20:26:11.603 INFO:teuthology.orchestra.run.smithi155.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 188 kB/s | 19 kB 00:00 2024-03-18T20:26:11.607 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-18T20:26:11.628 INFO:teuthology.orchestra.run.smithi155.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2024-03-18T20:26:11.712 INFO:teuthology.orchestra.run.smithi155.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2024-03-18T20:26:11.737 INFO:teuthology.orchestra.run.smithi155.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2024-03-18T20:26:11.762 INFO:teuthology.orchestra.run.smithi155.stdout:(64/146): liboath-2.6.2-3.el8.x86_64.rpm 6.9 kB/s | 59 kB 00:08 2024-03-18T20:26:11.802 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-03-18T20:26:11.846 INFO:teuthology.orchestra.run.smithi155.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2024-03-18T20:26:11.871 INFO:teuthology.orchestra.run.smithi155.stdout:(66/146): python3-waitress-1.4.3-1.el8.noarch.r 1.6 MB/s | 255 kB 00:00 2024-03-18T20:26:11.897 INFO:teuthology.orchestra.run.smithi155.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 457 kB/s | 61 kB 00:00 2024-03-18T20:26:11.947 INFO:teuthology.orchestra.run.smithi155.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 850 kB/s | 85 kB 00:00 2024-03-18T20:26:11.972 INFO:teuthology.orchestra.run.smithi155.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 228 kB/s | 23 kB 00:00 2024-03-18T20:26:11.989 INFO:teuthology.orchestra.run.smithi155.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2024-03-18T20:26:11.994 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-18T20:26:12.084 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-18T20:26:12.139 INFO:teuthology.orchestra.run.smithi155.stdout:(71/146): python3-packaging-16.8-9.el8.noarch.r 289 kB/s | 48 kB 00:00 2024-03-18T20:26:12.190 INFO:teuthology.orchestra.run.smithi155.stdout:(72/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 92 kB/s | 18 kB 00:00 2024-03-18T20:26:12.257 INFO:teuthology.orchestra.run.smithi155.stdout:(73/146): python3-jwt-1.6.1-2.el8.noarch.rpm 364 kB/s | 43 kB 00:00 2024-03-18T20:26:12.286 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-18T20:26:12.349 INFO:teuthology.orchestra.run.smithi155.stdout:(74/146): thrift-0.13.0-2.el8.x86_64.rpm 3.8 MB/s | 1.7 MB 00:00 2024-03-18T20:26:12.374 INFO:teuthology.orchestra.run.smithi155.stdout:(75/146): python3-oauthlib-2.1.0-1.el8.noarch.r 843 kB/s | 155 kB 00:00 2024-03-18T20:26:12.441 INFO:teuthology.orchestra.run.smithi155.stdout:(76/146): python3-pycparser-2.14-14.el8.noarch. 593 kB/s | 109 kB 00:00 2024-03-18T20:26:12.508 INFO:teuthology.orchestra.run.smithi155.stdout:(77/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 145 kB/s | 19 kB 00:00 2024-03-18T20:26:12.558 INFO:teuthology.orchestra.run.smithi155.stdout:(78/146): python3-requests-oauthlib-1.0.0-1.el8 204 kB/s | 43 kB 00:00 2024-03-18T20:26:12.584 INFO:teuthology.orchestra.run.smithi155.stdout:(79/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 308 kB/s | 44 kB 00:00 2024-03-18T20:26:12.617 INFO:teuthology.orchestra.run.smithi155.stdout:(80/146): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2024-03-18T20:26:12.668 INFO:teuthology.orchestra.run.smithi155.stdout:(81/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2024-03-18T20:26:12.693 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-18T20:26:12.701 INFO:teuthology.orchestra.run.smithi155.stdout:(82/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2024-03-18T20:26:12.726 INFO:teuthology.orchestra.run.smithi155.stdout:(83/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2024-03-18T20:26:12.777 INFO:teuthology.orchestra.run.smithi155.stdout:(84/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 442 kB/s | 48 kB 00:00 2024-03-18T20:26:12.827 INFO:teuthology.orchestra.run.smithi155.stdout:(85/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 575 kB/s | 58 kB 00:00 2024-03-18T20:26:12.844 INFO:teuthology.orchestra.run.smithi155.stdout:(86/146): libsepol-devel-2.9-3.el8.x86_64.rpm 611 kB/s | 87 kB 00:00 2024-03-18T20:26:12.894 INFO:teuthology.orchestra.run.smithi155.stdout:(87/146): libcom_err-devel-1.45.6-4.el8.x86_64. 329 kB/s | 39 kB 00:00 2024-03-18T20:26:12.895 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librados2-2:18.2.2-914.g49bdc511.el8.x86_64 9/162 2024-03-18T20:26:12.994 INFO:teuthology.orchestra.run.smithi155.stdout:(88/146): fuse-2.9.7-17.el8.x86_64.rpm 497 kB/s | 83 kB 00:00 2024-03-18T20:26:13.036 INFO:teuthology.orchestra.run.smithi155.stdout:(89/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.3 MB/s | 645 kB 00:00 2024-03-18T20:26:13.062 INFO:teuthology.orchestra.run.smithi155.stdout:(90/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.0 MB/s | 172 kB 00:00 2024-03-18T20:26:13.120 INFO:teuthology.orchestra.run.smithi155.stdout:(91/146): libuuid-devel-2.32.1-43.el8.x86_64.rp 793 kB/s | 100 kB 00:00 2024-03-18T20:26:13.187 INFO:teuthology.orchestra.run.smithi155.stdout:(92/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-03-18T20:26:13.230 INFO:teuthology.orchestra.run.smithi155.stdout:(93/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 434 kB/s | 73 kB 00:00 2024-03-18T20:26:13.281 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:18.2.2-914.g49bdc511.el8.x86_64 9/162 2024-03-18T20:26:13.296 INFO:teuthology.orchestra.run.smithi155.stdout:(94/146): NetworkManager-initscripts-updown-1.4 821 kB/s | 144 kB 00:00 2024-03-18T20:26:13.355 INFO:teuthology.orchestra.run.smithi155.stdout:(95/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 4.4 MB/s | 562 kB 00:00 2024-03-18T20:26:13.398 INFO:teuthology.orchestra.run.smithi155.stdout:(96/146): elfutils-libelf-devel-0.189-3.el8.x86 294 kB/s | 62 kB 00:00 2024-03-18T20:26:13.494 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librbd1-2:18.2.2-914.g49bdc511.el8.x86_64 10/162 2024-03-18T20:26:13.505 INFO:teuthology.orchestra.run.smithi155.stdout:(97/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 313 kB/s | 47 kB 00:00 2024-03-18T20:26:13.547 INFO:teuthology.orchestra.run.smithi155.stdout:(98/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 9.3 MB/s | 2.3 MB 00:00 2024-03-18T20:26:13.573 INFO:teuthology.orchestra.run.smithi155.stdout:(99/146): python3-cryptography-3.2.1-7.el8_9.x8 3.1 MB/s | 559 kB 00:00 2024-03-18T20:26:13.631 INFO:teuthology.orchestra.run.smithi155.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 159 kB/s | 20 kB 00:00 2024-03-18T20:26:13.637 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:18.2.2-914.g49bdc511.el8.x86_64 10/162 2024-03-18T20:26:13.682 INFO:teuthology.orchestra.run.smithi155.stdout:(101/146): python3-pyasn1-modules-0.3.7-6.el8.n 1.0 MB/s | 110 kB 00:00 2024-03-18T20:26:13.699 INFO:teuthology.orchestra.run.smithi155.stdout:(102/146): python3-prettytable-0.7.2-14.el8.noa 291 kB/s | 44 kB 00:00 2024-03-18T20:26:13.749 INFO:teuthology.orchestra.run.smithi155.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 226 kB/s | 27 kB 00:00 2024-03-18T20:26:13.783 INFO:teuthology.orchestra.run.smithi155.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.2 MB/s | 126 kB 00:00 2024-03-18T20:26:13.819 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs2-2:18.2.2-914.g49bdc511.el8.x86_64 11/162 2024-03-18T20:26:13.858 INFO:teuthology.orchestra.run.smithi155.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 2.8 MB/s | 457 kB 00:00 2024-03-18T20:26:13.900 INFO:teuthology.orchestra.run.smithi155.stdout:(106/146): bison-3.0.4-10.el8.x86_64.rpm 4.5 MB/s | 688 kB 00:00 2024-03-18T20:26:13.915 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephfs2-2:18.2.2-914.g49bdc511.el8.x86_64 11/162 2024-03-18T20:26:13.925 INFO:teuthology.orchestra.run.smithi155.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2024-03-18T20:26:13.975 INFO:teuthology.orchestra.run.smithi155.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 5.9 MB/s | 709 kB 00:00 2024-03-18T20:26:14.058 INFO:teuthology.orchestra.run.smithi155.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 246 kB/s | 39 kB 00:00 2024-03-18T20:26:14.118 INFO:teuthology.orchestra.run.smithi155.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2024-03-18T20:26:14.143 INFO:teuthology.orchestra.run.smithi155.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 382 kB/s | 64 kB 00:00 2024-03-18T20:26:14.168 INFO:teuthology.orchestra.run.smithi155.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 943 kB/s | 103 kB 00:00 2024-03-18T20:26:14.237 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rados-2:18.2.2-914.g49bdc511.el8.x86_64 12/162 2024-03-18T20:26:14.327 INFO:teuthology.orchestra.run.smithi155.stdout:(113/146): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2024-03-18T20:26:14.361 INFO:teuthology.orchestra.run.smithi155.stdout:(114/146): python3-jinja2-2.10.1-3.el8.noarch.r 2.4 MB/s | 538 kB 00:00 2024-03-18T20:26:14.436 INFO:teuthology.orchestra.run.smithi155.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2024-03-18T20:26:14.486 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-18T20:26:14.494 INFO:teuthology.orchestra.run.smithi155.stdout:(116/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.6 MB/s | 225 kB 00:00 2024-03-18T20:26:14.553 INFO:teuthology.orchestra.run.smithi155.stdout:(117/146): python3-numpy-1.14.3-10.el8.x86_64.r 9.7 MB/s | 3.7 MB 00:00 2024-03-18T20:26:14.557 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-18T20:26:14.579 INFO:teuthology.orchestra.run.smithi155.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2024-03-18T20:26:14.621 INFO:teuthology.orchestra.run.smithi155.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2024-03-18T20:26:14.742 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-18T20:26:14.762 INFO:teuthology.orchestra.run.smithi155.stdout:(120/146): openblas-threads-0.3.15-6.el8.x86_64 19 MB/s | 5.2 MB 00:00 2024-03-18T20:26:14.804 INFO:teuthology.orchestra.run.smithi155.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 3.8 MB/s | 714 kB 00:00 2024-03-18T20:26:14.913 INFO:teuthology.orchestra.run.smithi155.stdout:(122/146): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2024-03-18T20:26:14.938 INFO:teuthology.orchestra.run.smithi155.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2024-03-18T20:26:14.963 INFO:teuthology.orchestra.run.smithi155.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-03-18T20:26:14.982 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-18T20:26:15.022 INFO:teuthology.orchestra.run.smithi155.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 499 kB/s | 54 kB 00:00 2024-03-18T20:26:15.064 INFO:teuthology.orchestra.run.smithi155.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-03-18T20:26:15.147 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-18T20:26:15.265 INFO:teuthology.orchestra.run.smithi155.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 6.2 MB/s | 1.5 MB 00:00 2024-03-18T20:26:15.337 INFO:teuthology.orchestra.run.smithi155.stdout:(128/146): postgresql-contrib-10.23-4.module+el 2.9 MB/s | 812 kB 00:00 2024-03-18T20:26:15.507 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-18T20:26:15.532 INFO:teuthology.orchestra.run.smithi155.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 23 MB/s | 14 MB 00:00 2024-03-18T20:26:15.640 INFO:teuthology.orchestra.run.smithi155.stdout:(130/146): postgresql-server-10.23-4.module+el8 13 MB/s | 5.1 MB 00:00 2024-03-18T20:26:15.680 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-03-18T20:26:15.749 INFO:teuthology.orchestra.run.smithi155.stdout:(131/146): librados2-18.2.2-914.g49bdc511.el8.x 8.9 MB/s | 3.7 MB 00:00 2024-03-18T20:26:15.767 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-18T20:26:15.791 INFO:teuthology.orchestra.run.smithi155.stdout:(132/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.5 MB/s | 232 kB 00:00 2024-03-18T20:26:15.827 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-18T20:26:15.866 INFO:teuthology.orchestra.run.smithi155.stdout:(133/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.5 MB/s | 303 kB 00:00 2024-03-18T20:26:15.938 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephsqlite-2:18.2.2-914.g49bdc511.el8.x86_64 19/162 2024-03-18T20:26:15.972 INFO:teuthology.orchestra.run.smithi155.stdout:(134/146): librbd1-18.2.2-914.g49bdc511.el8.x86 8.1 MB/s | 3.6 MB 00:00 2024-03-18T20:26:16.009 INFO:teuthology.orchestra.run.smithi155.stdout:(135/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.2 MB/s | 179 kB 00:00 2024-03-18T20:26:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephsqlite-2:18.2.2-914.g49bdc511.el8.x86_64 19/162 2024-03-18T20:26:16.034 INFO:teuthology.orchestra.run.smithi155.stdout:(136/146): libmount-2.32.1-43.el8.x86_64.rpm 974 kB/s | 237 kB 00:00 2024-03-18T20:26:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-18T20:26:16.118 INFO:teuthology.orchestra.run.smithi155.stdout:(137/146): libuuid-2.32.1-43.el8.x86_64.rpm 683 kB/s | 99 kB 00:00 2024-03-18T20:26:16.168 INFO:teuthology.orchestra.run.smithi155.stdout:(138/146): util-linux-user-2.32.1-43.el8.x86_64 767 kB/s | 103 kB 00:00 2024-03-18T20:26:16.185 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libradosstriper1-2:18.2.2-914.g49bdc511.el8.x86_ 20/162 2024-03-18T20:26:16.256 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libradosstriper1-2:18.2.2-914.g49bdc511.el8.x86_ 20/162 2024-03-18T20:26:16.258 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-18T20:26:16.268 INFO:teuthology.orchestra.run.smithi155.stdout:(139/146): util-linux-2.32.1-43.el8.x86_64.rpm 9.6 MB/s | 2.5 MB 00:00 2024-03-18T20:26:16.302 INFO:teuthology.orchestra.run.smithi155.stdout:(140/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.5 MB/s | 854 kB 00:00 2024-03-18T20:26:16.327 INFO:teuthology.orchestra.run.smithi155.stdout:(141/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-03-18T20:26:16.369 INFO:teuthology.orchestra.run.smithi155.stdout:(142/146): libfdisk-2.32.1-43.el8.x86_64.rpm 2.5 MB/s | 254 kB 00:00 2024-03-18T20:26:16.420 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-18T20:26:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-03-18T20:26:16.486 INFO:teuthology.orchestra.run.smithi155.stdout:(143/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.8 MB/s | 333 kB 00:00 2024-03-18T20:26:16.511 INFO:teuthology.orchestra.run.smithi155.stdout:(144/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.3 MB/s | 711 kB 00:00 2024-03-18T20:26:16.562 INFO:teuthology.orchestra.run.smithi155.stdout:(145/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 6.3 MB/s | 1.5 MB 00:00 2024-03-18T20:26:16.572 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-18T20:26:16.656 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-18T20:26:16.686 INFO:teuthology.orchestra.run.smithi155.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.3 MB/s | 1.1 MB 00:00 2024-03-18T20:26:16.689 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-18T20:26:16.689 INFO:teuthology.orchestra.run.smithi155.stdout:Total 10 MB/s | 212 MB 00:20 2024-03-18T20:26:16.855 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-18T20:26:16.925 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-18T20:26:17.000 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-03-18T20:26:17.210 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-18T20:26:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-03-18T20:26:17.382 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-03-18T20:26:17.394 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-18T20:26:17.519 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-18T20:26:17.622 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-18T20:26:17.652 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-18T20:26:17.721 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-18T20:26:17.727 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-18T20:26:17.905 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-18T20:26:18.122 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-18T20:26:18.122 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-18T20:26:18.276 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-18T20:26:18.349 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-18T20:26:18.460 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-18T20:26:18.597 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:18.2.2-914.g49bdc511.el8.x86_64 9/162 2024-03-18T20:26:19.032 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:18.2.2-914.g49bdc511.el8.x86_64 9/162 2024-03-18T20:26:19.064 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-18T20:26:19.248 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:18.2.2-914.g49bdc511.el8.x86_64 10/162 2024-03-18T20:26:19.285 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-03-18T20:26:19.307 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-18T20:26:19.318 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-18T20:26:19.372 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:18.2.2-914.g49bdc511.el8.x86_64 10/162 2024-03-18T20:26:19.555 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:18.2.2-914.g49bdc511.el8.x86_64 11/162 2024-03-18T20:26:19.641 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:18.2.2-914.g49bdc511.el8.x86_64 11/162 2024-03-18T20:26:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:18.2.2-914.g49bdc511.el8.x86_64 12/162 2024-03-18T20:26:19.968 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-03-18T20:26:20.163 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-18T20:26:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-03-18T20:26:20.256 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-18T20:26:20.415 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-03-18T20:26:20.473 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-18T20:26:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-18T20:26:20.710 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-18T20:26:20.824 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-common-2:18.2.2-914.g49bdc511.el8.x 31/162 2024-03-18T20:26:20.869 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-18T20:26:20.962 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-argparse-2:18.2.2-914.g49bdc511.el8 32/162 2024-03-18T20:26:21.196 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cephfs-2:18.2.2-914.g49bdc511.el8.x86_64 33/162 2024-03-18T20:26:21.258 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-18T20:26:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-18T20:26:21.473 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-18T20:26:21.642 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:18.2.2-914.g49bdc511.el8.x86_64 19/162 2024-03-18T20:26:21.751 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:18.2.2-914.g49bdc511.el8.x86_64 19/162 2024-03-18T20:26:21.880 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:18.2.2-914.g49bdc511.el8.x86_ 20/162 2024-03-18T20:26:21.963 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:18.2.2-914.g49bdc511.el8.x86_ 20/162 2024-03-18T20:26:22.120 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-03-18T20:26:22.370 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-03-18T20:26:22.637 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-03-18T20:26:22.788 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-18T20:26:22.917 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-18T20:26:23.054 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-18T20:26:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-18T20:26:23.419 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-03-18T20:26:23.741 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-18T20:26:23.954 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-18T20:26:24.328 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-03-18T20:26:24.619 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-18T20:26:24.638 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-03-18T20:26:24.813 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-03-18T20:26:24.884 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-03-18T20:26:25.105 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-18T20:26:25.111 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-18T20:26:25.240 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-18T20:26:25.320 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-18T20:26:25.461 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-18T20:26:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rbd-2:18.2.2-914.g49bdc511.el8.x86_64 41/162 2024-03-18T20:26:25.589 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-03-18T20:26:25.665 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librados-devel-2:18.2.2-914.g49bdc511.el8.x86_64 42/162 2024-03-18T20:26:25.677 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-18T20:26:25.837 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-03-18T20:26:25.853 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-18T20:26:25.859 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-18T20:26:25.923 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-18T20:26:26.046 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-03-18T20:26:26.150 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-18T20:26:26.189 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-18T20:26:26.216 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-18T20:26:26.264 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-18T20:26:26.306 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-18T20:26:26.437 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-18T20:26:26.470 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:18.2.2-914.g49bdc511.el8.x 31/162 2024-03-18T20:26:26.498 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-18T20:26:26.566 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-18T20:26:26.593 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:18.2.2-914.g49bdc511.el8 32/162 2024-03-18T20:26:26.644 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-18T20:26:26.735 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-18T20:26:26.851 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-18T20:26:26.870 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:18.2.2-914.g49bdc511.el8.x86_64 33/162 2024-03-18T20:26:26.913 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-03-18T20:26:27.045 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-18T20:26:27.115 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-18T20:26:27.198 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-18T20:26:27.297 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-18T20:26:27.381 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-18T20:26:27.576 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-18T20:26:27.603 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-18T20:26:27.782 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-18T20:26:27.792 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-18T20:26:27.871 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-18T20:26:27.989 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:18.2.2-914.g49bdc511.el8.x86_64 9/162 2024-03-18T20:26:28.135 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-03-18T20:26:28.326 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-03-18T20:26:28.384 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:18.2.2-914.g49bdc511.el8.x86_64 9/162 2024-03-18T20:26:28.517 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-03-18T20:26:28.585 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:18.2.2-914.g49bdc511.el8.x86_64 10/162 2024-03-18T20:26:28.589 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-03-18T20:26:28.711 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:18.2.2-914.g49bdc511.el8.x86_64 10/162 2024-03-18T20:26:28.757 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-03-18T20:26:28.907 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-03-18T20:26:28.914 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:18.2.2-914.g49bdc511.el8.x86_64 11/162 2024-03-18T20:26:29.007 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:18.2.2-914.g49bdc511.el8.x86_64 11/162 2024-03-18T20:26:29.009 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-18T20:26:29.109 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-03-18T20:26:29.296 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/162 2024-03-18T20:26:29.307 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:18.2.2-914.g49bdc511.el8.x86_64 12/162 2024-03-18T20:26:29.334 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-03-18T20:26:29.496 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/162 2024-03-18T20:26:29.580 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-18T20:26:29.654 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/162 2024-03-18T20:26:29.664 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-18T20:26:29.836 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/162 2024-03-18T20:26:29.862 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-18T20:26:30.020 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/162 2024-03-18T20:26:30.101 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-18T20:26:30.284 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-18T20:26:30.296 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/162 2024-03-18T20:26:30.419 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-03-18T20:26:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/162 2024-03-18T20:26:30.660 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-18T20:26:30.743 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-03-18T20:26:30.797 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/162 2024-03-18T20:26:30.949 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-18T20:26:30.954 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/162 2024-03-18T20:26:30.968 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-03-18T20:26:31.121 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:18.2.2-914.g49bdc511.el8.x86_64 19/162 2024-03-18T20:26:31.143 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/162 2024-03-18T20:26:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-18T20:26:31.223 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:18.2.2-914.g49bdc511.el8.x86_64 19/162 2024-03-18T20:26:31.372 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:18.2.2-914.g49bdc511.el8.x86_ 20/162 2024-03-18T20:26:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-18T20:26:31.464 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/162 2024-03-18T20:26:31.471 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:18.2.2-914.g49bdc511.el8.x86_ 20/162 2024-03-18T20:26:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:18.2.2-914.g49bdc511.el8.x86_64 41/162 2024-03-18T20:26:31.634 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-03-18T20:26:31.736 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 67/162 2024-03-18T20:26:31.745 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:18.2.2-914.g49bdc511.el8.x86_64 42/162 2024-03-18T20:26:31.872 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-03-18T20:26:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-18T20:26:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-18T20:26:32.032 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 68/162 2024-03-18T20:26:32.135 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-03-18T20:26:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-18T20:26:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 69/162 2024-03-18T20:26:32.289 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-18T20:26:32.290 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-18T20:26:32.424 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-18T20:26:32.434 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 70/162 2024-03-18T20:26:32.563 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-18T20:26:32.572 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-18T20:26:32.666 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 71/162 2024-03-18T20:26:32.757 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-18T20:26:32.887 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 72/162 2024-03-18T20:26:32.928 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-18T20:26:33.083 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 73/162 2024-03-18T20:26:33.197 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-18T20:26:33.310 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/162 2024-03-18T20:26:33.380 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-18T20:26:33.391 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-18T20:26:33.496 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/162 2024-03-18T20:26:33.654 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-18T20:26:33.685 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-18T20:26:33.751 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/162 2024-03-18T20:26:33.862 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-18T20:26:33.934 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-03-18T20:26:33.946 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-18T20:26:33.971 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-18T20:26:34.082 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-18T20:26:34.155 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-03-18T20:26:34.204 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-03-18T20:26:34.252 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-18T20:26:34.534 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-18T20:26:34.540 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-03-18T20:26:34.695 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-03-18T20:26:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-03-18T20:26:34.756 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-18T20:26:34.836 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-03-18T20:26:34.920 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-18T20:26:34.936 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-03-18T20:26:35.077 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-18T20:26:35.077 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-03-18T20:26:35.104 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-03-18T20:26:35.270 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-18T20:26:35.277 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-03-18T20:26:35.323 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-03-18T20:26:35.445 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-18T20:26:35.508 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/162 2024-03-18T20:26:35.556 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-18T20:26:35.662 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-18T20:26:35.721 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/162 2024-03-18T20:26:35.727 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:18.2.2-914.g49bdc511.el8.x 31/162 2024-03-18T20:26:35.844 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-18T20:26:35.866 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:18.2.2-914.g49bdc511.el8 32/162 2024-03-18T20:26:35.894 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/162 2024-03-18T20:26:36.022 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-18T20:26:36.067 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/162 2024-03-18T20:26:36.160 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:18.2.2-914.g49bdc511.el8.x86_64 33/162 2024-03-18T20:26:36.234 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/162 2024-03-18T20:26:36.261 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-18T20:26:36.330 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-18T20:26:36.469 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-18T20:26:36.528 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/162 2024-03-18T20:26:36.638 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-18T20:26:36.823 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/162 2024-03-18T20:26:36.878 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-18T20:26:37.025 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/162 2024-03-18T20:26:37.115 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-18T20:26:37.180 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/162 2024-03-18T20:26:37.282 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-18T20:26:37.430 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/162 2024-03-18T20:26:37.565 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-18T20:26:37.796 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/162 2024-03-18T20:26:38.082 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-18T20:26:38.093 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 67/162 2024-03-18T20:26:38.381 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 68/162 2024-03-18T20:26:38.406 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-03-18T20:26:38.562 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 69/162 2024-03-18T20:26:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-18T20:26:38.759 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 70/162 2024-03-18T20:26:38.894 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-18T20:26:39.053 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 71/162 2024-03-18T20:26:39.114 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-18T20:26:39.271 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 72/162 2024-03-18T20:26:39.379 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-03-18T20:26:39.399 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-18T20:26:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 73/162 2024-03-18T20:26:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-18T20:26:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/162 2024-03-18T20:26:39.710 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-03-18T20:26:39.856 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-18T20:26:39.885 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/162 2024-03-18T20:26:39.963 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-03-18T20:26:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/162 2024-03-18T20:26:40.190 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-18T20:26:40.331 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-03-18T20:26:40.406 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-18T20:26:40.479 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-18T20:26:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-18T20:26:40.649 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:18.2.2-914.g49bdc511.el8.x86_64 41/162 2024-03-18T20:26:40.660 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-18T20:26:40.782 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librgw2-2:18.2.2-914.g49bdc511.el8.x86_64 103/162 2024-03-18T20:26:40.801 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:18.2.2-914.g49bdc511.el8.x86_64 42/162 2024-03-18T20:26:40.855 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librgw2-2:18.2.2-914.g49bdc511.el8.x86_64 103/162 2024-03-18T20:26:40.974 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-18T20:26:40.979 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-18T20:26:41.032 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rgw-2:18.2.2-914.g49bdc511.el8.x86_64 104/162 2024-03-18T20:26:41.085 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-18T20:26:41.186 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-18T20:26:41.287 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-18T20:26:41.349 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-18T20:26:41.356 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-18T20:26:41.507 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-18T20:26:41.583 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-18T20:26:41.721 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-18T20:26:41.806 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-18T20:26:41.909 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-18T20:26:42.004 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-18T20:26:42.268 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-18T20:26:42.422 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-18T20:26:42.441 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-18T20:26:42.567 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-18T20:26:42.746 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-18T20:26:42.773 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-18T20:26:42.787 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-18T20:26:42.920 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-18T20:26:43.001 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-18T20:26:43.007 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-18T20:26:43.158 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-18T20:26:43.266 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-03-18T20:26:43.390 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-18T20:26:43.454 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-03-18T20:26:43.612 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-03-18T20:26:43.619 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-18T20:26:43.704 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-03-18T20:26:43.820 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-18T20:26:43.877 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-03-18T20:26:44.027 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-03-18T20:26:44.047 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-18T20:26:44.196 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-03-18T20:26:44.433 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/162 2024-03-18T20:26:44.653 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/162 2024-03-18T20:26:44.663 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 105/162 2024-03-18T20:26:44.809 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/162 2024-03-18T20:26:44.882 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 105/162 2024-03-18T20:26:44.998 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/162 2024-03-18T20:26:45.183 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/162 2024-03-18T20:26:45.213 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-18T20:26:45.459 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/162 2024-03-18T20:26:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 105/162 2024-03-18T20:26:45.604 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-18T20:26:45.725 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-base-2:18.2.2-914.g49bdc511.el8.x86_64 106/162 2024-03-18T20:26:45.775 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/162 2024-03-18T20:26:45.807 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-base-2:18.2.2-914.g49bdc511.el8.x86_64 106/162 2024-03-18T20:26:45.832 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-18T20:26:45.973 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-selinux-2:18.2.2-914.g49bdc511.el8.x86_64 107/162 2024-03-18T20:26:45.983 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/162 2024-03-18T20:26:46.132 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-18T20:26:46.132 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/162 2024-03-18T20:26:46.379 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-18T20:26:46.381 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/162 2024-03-18T20:26:46.598 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-18T20:26:46.727 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/162 2024-03-18T20:26:47.031 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 67/162 2024-03-18T20:26:47.311 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 68/162 2024-03-18T20:26:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-18T20:26:47.552 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 69/162 2024-03-18T20:26:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:18.2.2-914.g49bdc511.el8.x86_64 103/162 2024-03-18T20:26:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:18.2.2-914.g49bdc511.el8.x86_64 103/162 2024-03-18T20:26:47.712 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 70/162 2024-03-18T20:26:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:18.2.2-914.g49bdc511.el8.x86_64 104/162 2024-03-18T20:26:47.963 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 71/162 2024-03-18T20:26:48.232 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 72/162 2024-03-18T20:26:48.437 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 73/162 2024-03-18T20:26:48.638 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/162 2024-03-18T20:26:48.859 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/162 2024-03-18T20:26:49.130 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/162 2024-03-18T20:26:49.312 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-03-18T20:26:49.503 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-18T20:26:49.692 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-18T20:26:49.976 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-18T20:26:50.260 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-18T20:26:50.424 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-18T20:26:50.581 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-18T20:26:50.801 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-18T20:26:50.990 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-18T20:26:51.225 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-18T20:26:51.447 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-18T20:26:51.658 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-18T20:26:51.890 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-18T20:26:52.131 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-18T20:26:52.291 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-18T20:26:52.401 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 105/162 2024-03-18T20:26:52.524 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-18T20:26:52.635 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 105/162 2024-03-18T20:26:52.768 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-18T20:26:52.926 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-18T20:26:53.143 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-18T20:26:53.279 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 105/162 2024-03-18T20:26:53.568 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:18.2.2-914.g49bdc511.el8.x86_64 106/162 2024-03-18T20:26:53.694 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:18.2.2-914.g49bdc511.el8.x86_64 106/162 2024-03-18T20:26:53.826 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:18.2.2-914.g49bdc511.el8.x86_64 107/162 2024-03-18T20:26:54.254 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-18T20:26:54.556 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-18T20:26:54.801 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-18T20:26:55.077 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-18T20:26:55.369 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-18T20:26:55.560 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-18T20:26:56.341 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-18T20:26:56.502 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:18.2.2-914.g49bdc511.el8.x86_64 103/162 2024-03-18T20:26:56.585 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:18.2.2-914.g49bdc511.el8.x86_64 103/162 2024-03-18T20:26:57.220 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:18.2.2-914.g49bdc511.el8.x86_64 104/162 2024-03-18T20:27:00.948 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 105/162 2024-03-18T20:27:01.201 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 105/162 2024-03-18T20:27:01.781 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 105/162 2024-03-18T20:27:02.056 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:18.2.2-914.g49bdc511.el8.x86_64 106/162 2024-03-18T20:27:02.144 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:18.2.2-914.g49bdc511.el8.x86_64 106/162 2024-03-18T20:27:02.589 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:18.2.2-914.g49bdc511.el8.x86_64 107/162 2024-03-18T20:27:05.369 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-selinux-2:18.2.2-914.g49bdc511.el8.x86_64 107/162 2024-03-18T20:27:05.369 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /sys 2024-03-18T20:27:05.370 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /proc 2024-03-18T20:27:05.370 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /dev 2024-03-18T20:27:05.370 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /run 2024-03-18T20:27:05.370 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /mnt 2024-03-18T20:27:05.370 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /var/tmp 2024-03-18T20:27:05.370 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /home 2024-03-18T20:27:05.370 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /tmp 2024-03-18T20:27:05.370 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /dev 2024-03-18T20:27:05.370 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:27:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-volume-2:18.2.2-914.g49bdc511.el8.noarch 108/162 2024-03-18T20:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-volume-2:18.2.2-914.g49bdc511.el8.noarch 108/162 2024-03-18T20:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-osd-2:18.2.2-914.g49bdc511.el8.x86_64 109/162 2024-03-18T20:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-osd-2:18.2.2-914.g49bdc511.el8.x86_64 109/162 2024-03-18T20:27:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mds-2:18.2.2-914.g49bdc511.el8.x86_64 110/162 2024-03-18T20:27:08.268 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mds-2:18.2.2-914.g49bdc511.el8.x86_64 110/162 2024-03-18T20:27:08.420 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mon-2:18.2.2-914.g49bdc511.el8.x86_64 111/162 2024-03-18T20:27:08.437 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mon-2:18.2.2-914.g49bdc511.el8.x86_64 111/162 2024-03-18T20:27:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephadm-2:18.2.2-914.g49bdc511.el8.noarch 112/162 2024-03-18T20:27:10.365 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephadm-2:18.2.2-914.g49bdc511.el8.noarch 112/162 2024-03-18T20:27:10.483 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-914.g49bdc511.el 113/162 2024-03-18T20:27:10.645 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-914.g49bdc511.e 114/162 2024-03-18T20:27:10.836 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-914.g49bdc511.el8.noar 115/162 2024-03-18T20:27:11.059 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-914.g49bdc511.el8.noar 115/162 2024-03-18T20:27:11.346 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-914.g49bdc511.el8.no 116/162 2024-03-18T20:27:12.312 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-914.g49bdc511.el8.no 116/162 2024-03-18T20:27:12.468 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-914.g49bd 117/162 2024-03-18T20:27:12.561 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-914.g49bd 117/162 2024-03-18T20:27:12.900 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-914.g49bdc511.el8 118/162 2024-03-18T20:27:13.063 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-2:18.2.2-914.g49bdc511.el8.x86_64 119/162 2024-03-18T20:27:13.100 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-2:18.2.2-914.g49bdc511.el8.x86_64 119/162 2024-03-18T20:27:13.252 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-rook-2:18.2.2-914.g49bdc511.el8.noarch 120/162 2024-03-18T20:27:13.264 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-914.g49bdc511.el8.noarch 120/162 2024-03-18T20:27:14.522 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-2:18.2.2-914.g49bdc511.el8.x86_64 121/162 2024-03-18T20:27:14.695 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-radosgw-2:18.2.2-914.g49bdc511.el8.x86_64 122/162 2024-03-18T20:27:14.729 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-914.g49bdc511.el8.x86_64 122/162 2024-03-18T20:27:14.894 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-914.g49bdc5 123/162 2024-03-18T20:27:14.936 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-914.g49bdc5 123/162 2024-03-18T20:27:15.094 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 124/162 2024-03-18T20:27:15.441 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 124/162 2024-03-18T20:27:15.610 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 125/162 2024-03-18T20:27:15.946 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:18.2.2-914.g49bdc511.el8.x86_64 107/162 2024-03-18T20:27:15.946 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-03-18T20:27:15.947 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-03-18T20:27:15.947 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /dev 2024-03-18T20:27:15.947 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /run 2024-03-18T20:27:15.947 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-03-18T20:27:15.947 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-03-18T20:27:15.947 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-03-18T20:27:15.947 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-03-18T20:27:15.947 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /dev 2024-03-18T20:27:15.947 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:27:16.273 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-volume-2:18.2.2-914.g49bdc511.el8.noarch 108/162 2024-03-18T20:27:18.003 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-volume-2:18.2.2-914.g49bdc511.el8.noarch 108/162 2024-03-18T20:27:18.165 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:18.2.2-914.g49bdc511.el8.x86_64 109/162 2024-03-18T20:27:18.408 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:18.2.2-914.g49bdc511.el8.x86_64 109/162 2024-03-18T20:27:18.598 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:18.2.2-914.g49bdc511.el8.x86_64 110/162 2024-03-18T20:27:19.104 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:18.2.2-914.g49bdc511.el8.x86_64 110/162 2024-03-18T20:27:19.298 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:18.2.2-914.g49bdc511.el8.x86_64 111/162 2024-03-18T20:27:19.320 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:18.2.2-914.g49bdc511.el8.x86_64 111/162 2024-03-18T20:27:20.055 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 125/162 2024-03-18T20:27:21.112 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-test-2:18.2.2-914.g49bdc511.el8.x86_64 126/162 2024-03-18T20:27:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-03-18T20:27:21.438 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:18.2.2-914.g49bdc511.el8.noarch 112/162 2024-03-18T20:27:21.576 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:18.2.2-914.g49bdc511.el8.noarch 112/162 2024-03-18T20:27:21.641 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-03-18T20:27:21.688 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-914.g49bdc511.el 113/162 2024-03-18T20:27:21.846 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-914.g49bdc511.e 114/162 2024-03-18T20:27:22.017 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 129/162 2024-03-18T20:27:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-914.g49bdc511.el8.noar 115/162 2024-03-18T20:27:22.099 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:18.2.2-914.g49bdc511.el8.x86_64 107/162 2024-03-18T20:27:22.100 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2024-03-18T20:27:22.100 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2024-03-18T20:27:22.100 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /dev 2024-03-18T20:27:22.100 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /run 2024-03-18T20:27:22.100 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2024-03-18T20:27:22.100 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2024-03-18T20:27:22.100 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2024-03-18T20:27:22.100 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2024-03-18T20:27:22.100 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /dev 2024-03-18T20:27:22.100 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:27:22.312 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-914.g49bdc511.el8.noar 115/162 2024-03-18T20:27:22.319 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-03-18T20:27:22.456 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:18.2.2-914.g49bdc511.el8.noarch 108/162 2024-03-18T20:27:22.546 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-03-18T20:27:22.559 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-03-18T20:27:22.617 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-914.g49bdc511.el8.no 116/162 2024-03-18T20:27:22.803 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-18T20:27:22.963 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-18T20:27:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs-devel-2:18.2.2-914.g49bdc511.el8.x86_6 134/162 2024-03-18T20:27:23.695 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-914.g49bdc511.el8.no 116/162 2024-03-18T20:27:23.838 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-914.g49bd 117/162 2024-03-18T20:27:23.948 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-914.g49bd 117/162 2024-03-18T20:27:24.018 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:18.2.2-914.g49bdc511.el8.noarch 108/162 2024-03-18T20:27:24.186 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:18.2.2-914.g49bdc511.el8.x86_64 109/162 2024-03-18T20:27:24.333 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-914.g49bdc511.el8 118/162 2024-03-18T20:27:24.406 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:18.2.2-914.g49bdc511.el8.x86_64 109/162 2024-03-18T20:27:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:18.2.2-914.g49bdc511.el8.x86_64 119/162 2024-03-18T20:27:24.551 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:18.2.2-914.g49bdc511.el8.x86_64 119/162 2024-03-18T20:27:24.581 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:18.2.2-914.g49bdc511.el8.x86_64 110/162 2024-03-18T20:27:24.680 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:18.2.2-914.g49bdc511.el8.noarch 120/162 2024-03-18T20:27:24.695 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-914.g49bdc511.el8.noarch 120/162 2024-03-18T20:27:25.036 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:18.2.2-914.g49bdc511.el8.x86_64 110/162 2024-03-18T20:27:25.214 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:18.2.2-914.g49bdc511.el8.x86_64 111/162 2024-03-18T20:27:25.232 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:18.2.2-914.g49bdc511.el8.x86_64 111/162 2024-03-18T20:27:25.388 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-18T20:27:25.676 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-18T20:27:25.713 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-18T20:27:26.113 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-18T20:27:26.114 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:18.2.2-914.g49bdc511.el8.x86_64 121/162 2024-03-18T20:27:26.331 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:18.2.2-914.g49bdc511.el8.x86_64 122/162 2024-03-18T20:27:26.371 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-914.g49bdc511.el8.x86_64 122/162 2024-03-18T20:27:26.525 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-18T20:27:26.581 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-914.g49bdc5 123/162 2024-03-18T20:27:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-914.g49bdc5 123/162 2024-03-18T20:27:26.762 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-18T20:27:26.822 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 124/162 2024-03-18T20:27:26.927 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:18.2.2-914.g49bdc511.el8.noarch 112/162 2024-03-18T20:27:26.942 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-top-2:18.2.2-914.g49bdc511.el8.noarch 139/162 2024-03-18T20:27:27.105 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:18.2.2-914.g49bdc511.el8.noarch 112/162 2024-03-18T20:27:27.158 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 140/162 2024-03-18T20:27:27.209 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 124/162 2024-03-18T20:27:27.236 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-914.g49bdc511.el 113/162 2024-03-18T20:27:27.372 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 125/162 2024-03-18T20:27:27.397 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-914.g49bdc511.e 114/162 2024-03-18T20:27:27.423 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-nbd-2:18.2.2-914.g49bdc511.el8.x86_64 141/162 2024-03-18T20:27:27.613 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-914.g49bdc511.el8.noar 115/162 2024-03-18T20:27:27.701 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-18T20:27:27.836 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-914.g49bdc511.el8.noar 115/162 2024-03-18T20:27:27.916 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-03-18T20:27:27.991 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-03-18T20:27:28.118 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-914.g49bdc511.el8.no 116/162 2024-03-18T20:27:28.200 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-03-18T20:27:28.212 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-03-18T20:27:28.391 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-18T20:27:28.400 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-18T20:27:28.546 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-03-18T20:27:28.705 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-18T20:27:28.876 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-18T20:27:29.053 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-18T20:27:29.053 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-18T20:27:29.081 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-914.g49bdc511.el8.no 116/162 2024-03-18T20:27:29.238 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-914.g49bd 117/162 2024-03-18T20:27:29.310 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-18T20:27:29.311 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-18T20:27:29.330 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-914.g49bd 117/162 2024-03-18T20:27:29.520 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-18T20:27:29.520 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-18T20:27:29.700 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-914.g49bdc511.el8 118/162 2024-03-18T20:27:29.747 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-18T20:27:29.868 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:18.2.2-914.g49bdc511.el8.x86_64 119/162 2024-03-18T20:27:29.907 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:18.2.2-914.g49bdc511.el8.x86_64 119/162 2024-03-18T20:27:30.009 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-18T20:27:30.054 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:18.2.2-914.g49bdc511.el8.noarch 120/162 2024-03-18T20:27:30.066 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-914.g49bdc511.el8.noarch 120/162 2024-03-18T20:27:30.201 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-18T20:27:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-18T20:27:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-18T20:27:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-18T20:27:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-18T20:27:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-18T20:27:30.877 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-18T20:27:31.118 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-18T20:27:31.119 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-18T20:27:31.322 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:18.2.2-914.g49bdc511.el8.x86_64 121/162 2024-03-18T20:27:31.360 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-18T20:27:31.360 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-18T20:27:31.560 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-18T20:27:31.701 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:18.2.2-914.g49bdc511.el8.x86_64 122/162 2024-03-18T20:27:31.727 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-18T20:27:31.727 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-18T20:27:31.737 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-914.g49bdc511.el8.x86_64 122/162 2024-03-18T20:27:31.930 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-914.g49bdc5 123/162 2024-03-18T20:27:31.974 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-914.g49bdc5 123/162 2024-03-18T20:27:32.171 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 124/162 2024-03-18T20:27:32.281 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 125/162 2024-03-18T20:27:32.519 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 124/162 2024-03-18T20:27:32.617 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-18T20:27:32.617 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-2:18.2.2-914.g49bdc511.el8.x86_64 1/162 2024-03-18T20:27:32.617 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-base-2:18.2.2-914.g49bdc511.el8.x86_64 2/162 2024-03-18T20:27:32.617 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 3/162 2024-03-18T20:27:32.617 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 4/162 2024-03-18T20:27:32.617 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-914.g49bdc5 5/162 2024-03-18T20:27:32.617 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mds-2:18.2.2-914.g49bdc511.el8.x86_64 6/162 2024-03-18T20:27:32.617 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-2:18.2.2-914.g49bdc511.el8.x86_64 7/162 2024-03-18T20:27:32.617 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mon-2:18.2.2-914.g49bdc511.el8.x86_64 8/162 2024-03-18T20:27:32.617 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-osd-2:18.2.2-914.g49bdc511.el8.x86_64 9/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-radosgw-2:18.2.2-914.g49bdc511.el8.x86_64 10/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-selinux-2:18.2.2-914.g49bdc511.el8.x86_64 11/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-test-2:18.2.2-914.g49bdc511.el8.x86_64 12/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 13/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs-devel-2:18.2.2-914.g49bdc511.el8.x86_6 14/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs2-2:18.2.2-914.g49bdc511.el8.x86_64 15/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephsqlite-2:18.2.2-914.g49bdc511.el8.x86_64 16/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados-devel-2:18.2.2-914.g49bdc511.el8.x86_64 17/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libradosstriper1-2:18.2.2-914.g49bdc511.el8.x86_ 18/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librgw2-2:18.2.2-914.g49bdc511.el8.x86_64 19/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-argparse-2:18.2.2-914.g49bdc511.el8 20/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-common-2:18.2.2-914.g49bdc511.el8.x 21/162 2024-03-18T20:27:32.618 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cephfs-2:18.2.2-914.g49bdc511.el8.x86_64 22/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rados-2:18.2.2-914.g49bdc511.el8.x86_64 23/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rbd-2:18.2.2-914.g49bdc511.el8.x86_64 24/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rgw-2:18.2.2-914.g49bdc511.el8.x86_64 25/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 26/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 27/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-nbd-2:18.2.2-914.g49bdc511.el8.x86_64 28/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-914.g49bdc511.e 29/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-914.g49bdc511.el8.noar 30/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-914.g49bdc511.el8.no 31/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-914.g49bd 32/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-914.g49bdc511.el8 33/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-rook-2:18.2.2-914.g49bdc511.el8.noarch 34/162 2024-03-18T20:27:32.619 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-914.g49bdc511.el 35/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-volume-2:18.2.2-914.g49bdc511.el8.noarch 36/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephadm-2:18.2.2-914.g49bdc511.el8.noarch 37/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-top-2:18.2.2-914.g49bdc511.el8.noarch 38/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-18T20:27:32.620 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-18T20:27:32.621 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-18T20:27:32.622 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-18T20:27:32.622 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-18T20:27:32.622 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-18T20:27:32.622 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-18T20:27:32.622 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-18T20:27:32.622 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-18T20:27:32.622 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-18T20:27:32.622 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-18T20:27:32.622 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-18T20:27:32.622 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-03-18T20:27:32.622 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/162 2024-03-18T20:27:32.622 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/162 2024-03-18T20:27:32.623 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/162 2024-03-18T20:27:32.623 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/162 2024-03-18T20:27:32.623 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/162 2024-03-18T20:27:32.623 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/162 2024-03-18T20:27:32.623 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/162 2024-03-18T20:27:32.623 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/162 2024-03-18T20:27:32.623 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/162 2024-03-18T20:27:32.623 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/162 2024-03-18T20:27:32.623 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/162 2024-03-18T20:27:32.623 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/162 2024-03-18T20:27:32.623 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/162 2024-03-18T20:27:32.623 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 91/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 92/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 93/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 94/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 95/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 96/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 97/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 98/162 2024-03-18T20:27:32.624 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 99/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-03-18T20:27:32.625 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-03-18T20:27:32.626 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:18.2.2-914.g49bdc511.el8.x86_64 131/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:18.2.2-914.g49bdc511.el8.x86_64 133/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-18T20:27:32.627 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-18T20:27:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-18T20:27:32.629 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-18T20:27:32.629 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-18T20:27:32.629 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-18T20:27:32.629 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-18T20:27:32.629 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-18T20:27:32.629 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-18T20:27:32.629 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-18T20:27:32.629 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-18T20:27:32.705 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 125/162 2024-03-18T20:27:33.375 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:18.2.2-914.g49bdc511.el8.x86_64 126/162 2024-03-18T20:27:33.660 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-03-18T20:27:33.922 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-03-18T20:27:34.328 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 129/162 2024-03-18T20:27:34.632 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-03-18T20:27:34.856 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-03-18T20:27:34.872 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-03-18T20:27:35.059 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-18T20:27:35.218 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-18T20:27:35.380 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:18.2.2-914.g49bdc511.el8.x86_6 134/162 2024-03-18T20:27:37.149 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 125/162 2024-03-18T20:27:38.025 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-18T20:27:38.154 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:18.2.2-914.g49bdc511.el8.x86_64 126/162 2024-03-18T20:27:38.329 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-18T20:27:38.370 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-18T20:27:38.408 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-03-18T20:27:38.685 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-03-18T20:27:38.774 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-18T20:27:39.065 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 129/162 2024-03-18T20:27:39.172 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-18T20:27:39.397 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-03-18T20:27:39.425 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-18T20:27:39.608 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-03-18T20:27:39.621 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-03-18T20:27:39.639 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-top-2:18.2.2-914.g49bdc511.el8.noarch 139/162 2024-03-18T20:27:39.831 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 140/162 2024-03-18T20:27:39.839 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-18T20:27:40.008 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-18T20:27:40.118 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:18.2.2-914.g49bdc511.el8.x86_64 141/162 2024-03-18T20:27:40.163 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:18.2.2-914.g49bdc511.el8.x86_6 134/162 2024-03-18T20:27:40.428 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-18T20:27:40.611 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-03-18T20:27:40.698 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-03-18T20:27:40.878 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-03-18T20:27:40.894 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-03-18T20:27:41.087 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-18T20:27:41.098 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-18T20:27:41.256 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-03-18T20:27:41.426 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-18T20:27:41.606 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-18T20:27:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-18T20:27:41.791 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-18T20:27:42.040 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-18T20:27:42.040 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-18T20:27:42.266 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-18T20:27:42.266 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-18T20:27:42.371 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-18T20:27:42.527 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-18T20:27:42.645 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-18T20:27:42.682 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-18T20:27:42.714 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-18T20:27:42.875 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-18T20:27:43.093 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-18T20:27:43.130 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-18T20:27:43.131 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-18T20:27:43.356 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-18T20:27:43.356 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-18T20:27:43.487 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-18T20:27:43.576 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-18T20:27:43.577 INFO:teuthology.orchestra.run.smithi073.stdout:Installed products updated. 2024-03-18T20:27:43.589 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-18T20:27:43.589 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-18T20:27:43.674 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-18T20:27:43.839 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-18T20:27:43.840 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-18T20:27:43.861 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-top-2:18.2.2-914.g49bdc511.el8.noarch 139/162 2024-03-18T20:27:44.044 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 140/162 2024-03-18T20:27:44.055 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-18T20:27:44.055 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-18T20:27:44.183 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:27:44.183 INFO:teuthology.orchestra.run.smithi073.stdout:Upgraded: 2024-03-18T20:27:44.183 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-18T20:27:44.183 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-18T20:27:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-18T20:27:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-18T20:27:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-18T20:27:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-18T20:27:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: librados2-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-18T20:27:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-18T20:27:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-18T20:27:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-18T20:27:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-18T20:27:44.184 INFO:teuthology.orchestra.run.smithi073.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: automake-1.16.1-8.el8.noarch 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: bison-3.0.4-10.el8.x86_64 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.185 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:44.186 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: flex-2.6.1-9.el8.x86_64 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: jq-1.6-7.el8.x86_64 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.187 INFO:teuthology.orchestra.run.smithi073.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: libpq-13.11-1.el8.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-18T20:27:44.188 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-18T20:27:44.189 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-18T20:27:44.190 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-18T20:27:44.190 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-18T20:27:44.190 INFO:teuthology.orchestra.run.smithi073.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-18T20:27:44.190 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-18T20:27:44.190 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.190 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.190 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.190 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-18T20:27:44.190 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-18T20:27:44.190 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-18T20:27:44.190 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-18T20:27:44.190 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-18T20:27:44.191 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend-2.6-1.el8.noarch 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.192 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-18T20:27:44.193 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-18T20:27:44.194 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-18T20:27:44.194 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.194 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.194 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:44.194 INFO:teuthology.orchestra.run.smithi073.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-18T20:27:44.194 INFO:teuthology.orchestra.run.smithi073.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-18T20:27:44.194 INFO:teuthology.orchestra.run.smithi073.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-18T20:27:44.194 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-18T20:27:44.194 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-18T20:27:44.195 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-18T20:27:44.195 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-18T20:27:44.195 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-03-18T20:27:44.296 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:18.2.2-914.g49bdc511.el8.x86_64 141/162 2024-03-18T20:27:44.298 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-18T20:27:44.388 DEBUG:teuthology.parallel:result is None 2024-03-18T20:27:44.439 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-18T20:27:44.439 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-18T20:27:44.596 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-18T20:27:44.785 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-03-18T20:27:44.861 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-03-18T20:27:45.077 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-03-18T20:27:45.091 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-03-18T20:27:45.261 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-18T20:27:45.269 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-18T20:27:45.370 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-18T20:27:45.370 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:18.2.2-914.g49bdc511.el8.x86_64 1/162 2024-03-18T20:27:45.370 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:18.2.2-914.g49bdc511.el8.x86_64 2/162 2024-03-18T20:27:45.370 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 3/162 2024-03-18T20:27:45.370 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 4/162 2024-03-18T20:27:45.370 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-914.g49bdc5 5/162 2024-03-18T20:27:45.370 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:18.2.2-914.g49bdc511.el8.x86_64 6/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:18.2.2-914.g49bdc511.el8.x86_64 7/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:18.2.2-914.g49bdc511.el8.x86_64 8/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:18.2.2-914.g49bdc511.el8.x86_64 9/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:18.2.2-914.g49bdc511.el8.x86_64 10/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:18.2.2-914.g49bdc511.el8.x86_64 11/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:18.2.2-914.g49bdc511.el8.x86_64 12/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 13/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:18.2.2-914.g49bdc511.el8.x86_6 14/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:18.2.2-914.g49bdc511.el8.x86_64 15/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:18.2.2-914.g49bdc511.el8.x86_64 16/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:18.2.2-914.g49bdc511.el8.x86_64 17/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:18.2.2-914.g49bdc511.el8.x86_ 18/162 2024-03-18T20:27:45.371 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:18.2.2-914.g49bdc511.el8.x86_64 19/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:18.2.2-914.g49bdc511.el8 20/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:18.2.2-914.g49bdc511.el8.x 21/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:18.2.2-914.g49bdc511.el8.x86_64 22/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:18.2.2-914.g49bdc511.el8.x86_64 23/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:18.2.2-914.g49bdc511.el8.x86_64 24/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:18.2.2-914.g49bdc511.el8.x86_64 25/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 26/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 27/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:18.2.2-914.g49bdc511.el8.x86_64 28/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-914.g49bdc511.e 29/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-914.g49bdc511.el8.noar 30/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-914.g49bdc511.el8.no 31/162 2024-03-18T20:27:45.372 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-914.g49bd 32/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-914.g49bdc511.el8 33/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:18.2.2-914.g49bdc511.el8.noarch 34/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-914.g49bdc511.el 35/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-volume-2:18.2.2-914.g49bdc511.el8.noarch 36/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:18.2.2-914.g49bdc511.el8.noarch 37/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-top-2:18.2.2-914.g49bdc511.el8.noarch 38/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-18T20:27:45.373 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-18T20:27:45.374 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-18T20:27:45.374 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-18T20:27:45.374 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-18T20:27:45.374 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-18T20:27:45.374 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-18T20:27:45.374 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-18T20:27:45.374 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-18T20:27:45.374 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-18T20:27:45.374 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-18T20:27:45.374 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-18T20:27:45.374 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-18T20:27:45.374 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-18T20:27:45.375 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/162 2024-03-18T20:27:45.376 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 91/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 92/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 93/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 94/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 95/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 96/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 97/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 98/162 2024-03-18T20:27:45.377 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 99/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-03-18T20:27:45.378 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-03-18T20:27:45.379 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:18.2.2-914.g49bdc511.el8.x86_64 131/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:18.2.2-914.g49bdc511.el8.x86_64 133/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-18T20:27:45.380 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-18T20:27:45.381 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-18T20:27:45.382 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-18T20:27:45.382 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-18T20:27:45.382 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-18T20:27:45.382 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-18T20:27:45.382 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-18T20:27:45.382 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-18T20:27:45.382 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-18T20:27:45.382 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-18T20:27:45.382 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-18T20:27:45.441 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-03-18T20:27:45.594 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-18T20:27:45.796 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-18T20:27:45.955 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-18T20:27:45.956 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-18T20:27:46.221 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-18T20:27:46.221 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-18T20:27:46.456 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-18T20:27:46.456 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-18T20:27:46.699 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-18T20:27:46.954 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-18T20:27:47.122 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-18T20:27:47.329 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-18T20:27:47.329 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-18T20:27:47.588 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-18T20:27:47.588 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-18T20:27:47.813 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-18T20:27:47.813 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-18T20:27:48.030 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-18T20:27:48.030 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-18T20:27:48.279 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-18T20:27:48.280 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-18T20:27:48.505 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-18T20:27:48.655 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-18T20:27:48.655 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-18T20:27:49.551 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-18T20:27:49.551 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:18.2.2-914.g49bdc511.el8.x86_64 1/162 2024-03-18T20:27:49.551 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:18.2.2-914.g49bdc511.el8.x86_64 2/162 2024-03-18T20:27:49.551 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 3/162 2024-03-18T20:27:49.551 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 4/162 2024-03-18T20:27:49.551 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-914.g49bdc5 5/162 2024-03-18T20:27:49.551 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:18.2.2-914.g49bdc511.el8.x86_64 6/162 2024-03-18T20:27:49.551 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:18.2.2-914.g49bdc511.el8.x86_64 7/162 2024-03-18T20:27:49.551 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:18.2.2-914.g49bdc511.el8.x86_64 8/162 2024-03-18T20:27:49.551 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:18.2.2-914.g49bdc511.el8.x86_64 9/162 2024-03-18T20:27:49.551 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:18.2.2-914.g49bdc511.el8.x86_64 10/162 2024-03-18T20:27:49.551 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:18.2.2-914.g49bdc511.el8.x86_64 11/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:18.2.2-914.g49bdc511.el8.x86_64 12/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 13/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:18.2.2-914.g49bdc511.el8.x86_6 14/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:18.2.2-914.g49bdc511.el8.x86_64 15/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:18.2.2-914.g49bdc511.el8.x86_64 16/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:18.2.2-914.g49bdc511.el8.x86_64 17/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:18.2.2-914.g49bdc511.el8.x86_ 18/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:18.2.2-914.g49bdc511.el8.x86_64 19/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:18.2.2-914.g49bdc511.el8 20/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:18.2.2-914.g49bdc511.el8.x 21/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:18.2.2-914.g49bdc511.el8.x86_64 22/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:18.2.2-914.g49bdc511.el8.x86_64 23/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:18.2.2-914.g49bdc511.el8.x86_64 24/162 2024-03-18T20:27:49.552 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:18.2.2-914.g49bdc511.el8.x86_64 25/162 2024-03-18T20:27:49.564 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 26/162 2024-03-18T20:27:49.564 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 27/162 2024-03-18T20:27:49.564 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:18.2.2-914.g49bdc511.el8.x86_64 28/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-914.g49bdc511.e 29/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-914.g49bdc511.el8.noar 30/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-914.g49bdc511.el8.no 31/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-914.g49bd 32/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-914.g49bdc511.el8 33/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:18.2.2-914.g49bdc511.el8.noarch 34/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-914.g49bdc511.el 35/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:18.2.2-914.g49bdc511.el8.noarch 36/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:18.2.2-914.g49bdc511.el8.noarch 37/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-top-2:18.2.2-914.g49bdc511.el8.noarch 38/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-18T20:27:49.565 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-18T20:27:49.566 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-18T20:27:49.567 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/162 2024-03-18T20:27:49.568 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 91/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 92/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 93/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 94/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 95/162 2024-03-18T20:27:49.569 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 96/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 97/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 98/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 99/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-03-18T20:27:49.570 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-03-18T20:27:49.571 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:18.2.2-914.g49bdc511.el8.x86_64 131/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:18.2.2-914.g49bdc511.el8.x86_64 133/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-18T20:27:49.572 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-18T20:27:49.573 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-18T20:27:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-18T20:27:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-18T20:27:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-18T20:27:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-18T20:27:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-18T20:27:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-18T20:27:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-18T20:27:49.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-18T20:27:56.845 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-18T20:27:56.846 INFO:teuthology.orchestra.run.smithi046.stdout:Installed products updated. 2024-03-18T20:27:58.154 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:27:58.203 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-03-18T20:27:58.203 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-18T20:27:58.203 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-18T20:27:58.203 INFO:teuthology.orchestra.run.smithi046.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-18T20:27:58.203 INFO:teuthology.orchestra.run.smithi046.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-18T20:27:58.203 INFO:teuthology.orchestra.run.smithi046.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-18T20:27:58.203 INFO:teuthology.orchestra.run.smithi046.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-18T20:27:58.203 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.203 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.203 INFO:teuthology.orchestra.run.smithi046.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-18T20:27:58.203 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-18T20:27:58.203 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: automake-1.16.1-8.el8.noarch 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: bison-3.0.4-10.el8.x86_64 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.204 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.205 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: flex-2.6.1-9.el8.x86_64 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: jq-1.6-7.el8.x86_64 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-18T20:27:58.206 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: libpq-13.11-1.el8.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-18T20:27:58.207 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-18T20:27:58.208 INFO:teuthology.orchestra.run.smithi046.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-18T20:27:58.209 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-18T20:27:58.210 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-18T20:27:58.210 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-18T20:27:58.210 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-18T20:27:58.210 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-18T20:27:58.210 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-18T20:27:58.210 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-18T20:27:58.210 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-18T20:27:58.210 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-18T20:27:58.210 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-18T20:27:58.210 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-18T20:27:58.210 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-18T20:27:58.210 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-2.6-1.el8.noarch 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-18T20:27:58.211 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-18T20:27:58.212 INFO:teuthology.orchestra.run.smithi046.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-18T20:27:58.213 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-18T20:27:58.214 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-18T20:27:58.214 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:27:58.214 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-03-18T20:27:58.540 DEBUG:teuthology.parallel:result is None 2024-03-18T20:27:59.793 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-18T20:27:59.793 INFO:teuthology.orchestra.run.smithi155.stdout:Installed products updated. 2024-03-18T20:28:00.395 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-18T20:28:00.396 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: automake-1.16.1-8.el8.noarch 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: bison-3.0.4-10.el8.x86_64 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:28:00.397 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.398 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.398 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.398 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:28:00.398 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:28:00.398 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:28:00.398 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:28:00.398 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:28:00.398 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.398 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.398 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:28:00.398 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.398 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top-2:18.2.2-914.g49bdc511.el8.noarch 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: flex-2.6.1-9.el8.x86_64 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: jq-1.6-7.el8.x86_64 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-18T20:28:00.399 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: libpq-13.11-1.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-18T20:28:00.400 INFO:teuthology.orchestra.run.smithi155.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-18T20:28:00.401 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-18T20:28:00.402 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-18T20:28:00.402 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-18T20:28:00.402 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-18T20:28:00.402 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-18T20:28:00.402 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-18T20:28:00.402 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-18T20:28:00.402 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.402 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.402 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.402 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-18T20:28:00.402 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-18T20:28:00.402 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-18T20:28:00.403 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-2.6-1.el8.noarch 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-18T20:28:00.404 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-18T20:28:00.405 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-18T20:28:00.406 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-18T20:28:00.406 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-18T20:28:00.406 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-18T20:28:00.406 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-18T20:28:00.406 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.406 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.406 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:18.2.2-914.g49bdc511.el8.x86_64 2024-03-18T20:28:00.406 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-18T20:28:00.406 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-18T20:28:00.406 INFO:teuthology.orchestra.run.smithi155.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-18T20:28:00.406 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-18T20:28:00.406 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-18T20:28:00.407 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-18T20:28:00.407 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:28:00.407 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-18T20:28:00.613 DEBUG:teuthology.parallel:result is None 2024-03-18T20:28:00.614 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49bdc51142759622a281d3e5327eebd07cbb8bb3 2024-03-18T20:28:00.756 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T20:28:00.787 INFO:teuthology.orchestra.run.smithi046.stdout:18.2.2-914.g49bdc511.el8 2024-03-18T20:28:00.787 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-914.g49bdc511.el8 2024-03-18T20:28:00.788 INFO:teuthology.task.install:The correct ceph version 18.2.2-914.g49bdc511 is installed. 2024-03-18T20:28:00.789 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49bdc51142759622a281d3e5327eebd07cbb8bb3 2024-03-18T20:28:00.925 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T20:28:00.955 INFO:teuthology.orchestra.run.smithi073.stdout:18.2.2-914.g49bdc511.el8 2024-03-18T20:28:00.955 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-914.g49bdc511.el8 2024-03-18T20:28:00.956 INFO:teuthology.task.install:The correct ceph version 18.2.2-914.g49bdc511 is installed. 2024-03-18T20:28:00.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49bdc51142759622a281d3e5327eebd07cbb8bb3 2024-03-18T20:28:01.103 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T20:28:01.127 INFO:teuthology.orchestra.run.smithi155.stdout:18.2.2-914.g49bdc511.el8 2024-03-18T20:28:01.128 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-914.g49bdc511.el8 2024-03-18T20:28:01.128 INFO:teuthology.task.install:The correct ceph version 18.2.2-914.g49bdc511 is installed. 2024-03-18T20:28:01.130 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-18T20:28:01.130 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:28:01.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T20:28:01.183 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-18T20:28:01.183 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T20:28:01.232 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-18T20:28:01.232 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T20:28:01.275 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-18T20:28:01.275 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:28:01.275 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T20:28:01.311 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T20:28:01.385 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-18T20:28:01.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T20:28:01.416 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T20:28:01.485 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-18T20:28:01.485 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T20:28:01.513 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T20:28:01.580 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-18T20:28:01.580 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:28:01.580 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T20:28:01.614 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T20:28:01.689 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-18T20:28:01.689 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T20:28:01.718 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T20:28:01.787 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-18T20:28:01.787 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T20:28:01.814 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T20:28:01.880 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-18T20:28:01.880 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:28:01.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T20:28:01.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T20:28:01.991 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-18T20:28:01.991 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T20:28:02.023 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T20:28:02.092 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-18T20:28:02.093 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T20:28:02.119 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T20:28:02.187 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-18T20:28:02.264 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '49bdc51142759622a281d3e5327eebd07cbb8bb3', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-18T20:28:02.264 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 2024-03-18T20:28:02.265 INFO:tasks.cephadm:Cluster fsid is 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:28:02.265 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-18T20:28:02.265 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.46', 'mon.b': '172.21.15.73', 'mon.c': '172.21.15.155'} 2024-03-18T20:28:02.265 INFO:tasks.cephadm:First mon is mon.a on smithi046 2024-03-18T20:28:02.265 INFO:tasks.cephadm:First mgr is x 2024-03-18T20:28:02.265 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-18T20:28:02.266 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname $(hostname -s) 2024-03-18T20:28:02.301 DEBUG:teuthology.orchestra.run.smithi073:> sudo hostname $(hostname -s) 2024-03-18T20:28:02.333 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2024-03-18T20:28:02.360 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-18T20:28:02.360 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49bdc51142759622a281d3e5327eebd07cbb8bb3 2024-03-18T20:28:02.497 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '49bdc51142759622a281d3e5327eebd07cbb8bb3', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/77701/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-914-g49bdc511', 'node_name': '172.21.4.64+confusa02', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-914.g49bdc511'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-03-16 02:21:43.411752', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-yuri10-testing-2024-03-15-1653-reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}] 2024-03-18T20:28:02.632 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-yuri10-testing-2024-03-15-1653-reef, sha1 49bdc51142759622a281d3e5327eebd07cbb8bb3 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=49bdc51142759622a281d3e5327eebd07cbb8bb3 2024-03-18T20:28:02.634 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/x86_64/flavors/default/cephadm 2024-03-18T20:28:02.634 DEBUG:teuthology.orchestra.run.smithi046:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T20:28:04.329 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-rw-r--. 1 ubuntu ubuntu 702420 Mar 18 20:28 /home/ubuntu/cephtest/cephadm 2024-03-18T20:28:04.330 DEBUG:teuthology.orchestra.run.smithi073:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T20:28:04.959 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-rw-r--. 1 ubuntu ubuntu 702420 Mar 18 20:28 /home/ubuntu/cephtest/cephadm 2024-03-18T20:28:04.959 DEBUG:teuthology.orchestra.run.smithi155:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-yuri10-testing-2024-03-15-1653-reef/49bdc51142759622a281d3e5327eebd07cbb8bb3/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T20:28:05.709 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-rw-r--. 1 ubuntu ubuntu 702420 Mar 18 20:28 /home/ubuntu/cephtest/cephadm 2024-03-18T20:28:05.710 DEBUG:teuthology.orchestra.run.smithi046:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T20:28:05.733 DEBUG:teuthology.orchestra.run.smithi073:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T20:28:05.755 DEBUG:teuthology.orchestra.run.smithi155:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T20:28:05.793 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 on all hosts... 2024-03-18T20:28:05.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 pull 2024-03-18T20:28:05.796 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 pull 2024-03-18T20:28:05.799 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 pull 2024-03-18T20:28:06.649 INFO:teuthology.orchestra.run.smithi073.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3... 2024-03-18T20:28:07.175 INFO:teuthology.orchestra.run.smithi046.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3... 2024-03-18T20:28:07.971 INFO:teuthology.orchestra.run.smithi155.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3... 2024-03-18T20:28:32.154 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:28:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-914-g49bdc511 (49bdc51142759622a281d3e5327eebd07cbb8bb3) reef (stable)", 2024-03-18T20:28:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "image_id": "015bdba7dcef68659e3352b62f3bbf78033bb173f439b9af177f23d5c5943567", 2024-03-18T20:28:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "repo_digests": [ 2024-03-18T20:28:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1b5dd227b04788a41794658b6571950ee4d311fa0172206cd8a725de73702aee" 2024-03-18T20:28:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-18T20:28:32.155 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:28:36.017 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-03-18T20:28:36.017 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_version": "ceph version 18.2.2-914-g49bdc511 (49bdc51142759622a281d3e5327eebd07cbb8bb3) reef (stable)", 2024-03-18T20:28:36.017 INFO:teuthology.orchestra.run.smithi073.stdout: "image_id": "015bdba7dcef68659e3352b62f3bbf78033bb173f439b9af177f23d5c5943567", 2024-03-18T20:28:36.018 INFO:teuthology.orchestra.run.smithi073.stdout: "repo_digests": [ 2024-03-18T20:28:36.018 INFO:teuthology.orchestra.run.smithi073.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1b5dd227b04788a41794658b6571950ee4d311fa0172206cd8a725de73702aee" 2024-03-18T20:28:36.018 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-03-18T20:28:36.018 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-03-18T20:28:38.307 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-03-18T20:28:38.307 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.2.2-914-g49bdc511 (49bdc51142759622a281d3e5327eebd07cbb8bb3) reef (stable)", 2024-03-18T20:28:38.307 INFO:teuthology.orchestra.run.smithi155.stdout: "image_id": "015bdba7dcef68659e3352b62f3bbf78033bb173f439b9af177f23d5c5943567", 2024-03-18T20:28:38.307 INFO:teuthology.orchestra.run.smithi155.stdout: "repo_digests": [ 2024-03-18T20:28:38.307 INFO:teuthology.orchestra.run.smithi155.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1b5dd227b04788a41794658b6571950ee4d311fa0172206cd8a725de73702aee" 2024-03-18T20:28:38.307 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-03-18T20:28:38.308 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-03-18T20:28:38.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph 2024-03-18T20:28:38.367 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph 2024-03-18T20:28:38.404 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2024-03-18T20:28:38.435 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /etc/ceph 2024-03-18T20:28:38.470 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /etc/ceph 2024-03-18T20:28:38.505 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2024-03-18T20:28:38.531 INFO:tasks.cephadm:Writing seed config... 2024-03-18T20:28:38.532 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-18T20:28:38.533 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-18T20:28:38.534 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-18T20:28:38.535 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-18T20:28:38.536 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-18T20:28:38.536 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-18T20:28:38.536 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-18T20:28:38.536 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-18T20:28:38.536 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:28:38.537 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-18T20:28:38.558 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 04a8ae18-e566-11ee-95c9-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-18T20:28:38.559 DEBUG:teuthology.orchestra.run.smithi046:mon.a> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.a.service 2024-03-18T20:28:38.601 DEBUG:teuthology.orchestra.run.smithi046:mgr.x> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.x.service 2024-03-18T20:28:38.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:-- Logs begin at Mon 2024-03-18 20:12:57 UTC. -- 2024-03-18T20:28:38.637 INFO:tasks.cephadm:Bootstrapping... 2024-03-18T20:28:38.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 -v bootstrap --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.46 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-18T20:28:38.668 INFO:journalctl@ceph.mgr.x.smithi046.stdout:-- Logs begin at Mon 2024-03-18 20:12:57 UTC. -- 2024-03-18T20:28:38.826 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-03-18T20:28:38.826 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3', '-v', 'bootstrap', '--fsid', '04a8ae18-e566-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.46', '--skip-admin-label'] 2024-03-18T20:28:38.863 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 4.6.1 2024-03-18T20:28:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-18T20:28:38.864 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying podman|docker is present... 2024-03-18T20:28:38.909 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 4.6.1 2024-03-18T20:28:38.909 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying lvm2 is present... 2024-03-18T20:28:38.909 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying time synchronization is in place... 2024-03-18T20:28:38.917 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-18T20:28:38.917 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-18T20:28:38.923 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-18T20:28:38.924 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-03-18T20:28:38.930 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-03-18T20:28:38.937 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-03-18T20:28:38.937 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-03-18T20:28:38.937 INFO:teuthology.orchestra.run.smithi046.stdout:Repeating the final host check... 2024-03-18T20:28:38.969 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 4.6.1 2024-03-18T20:28:38.969 INFO:teuthology.orchestra.run.smithi046.stdout:podman (/bin/podman) version 4.6.1 is present 2024-03-18T20:28:38.969 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl is present 2024-03-18T20:28:38.969 INFO:teuthology.orchestra.run.smithi046.stdout:lvcreate is present 2024-03-18T20:28:38.976 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-18T20:28:38.976 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-18T20:28:38.982 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-18T20:28:38.982 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-03-18T20:28:38.988 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-03-18T20:28:38.995 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-03-18T20:28:38.995 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-03-18T20:28:38.995 INFO:teuthology.orchestra.run.smithi046.stdout:Host looks OK 2024-03-18T20:28:38.995 INFO:teuthology.orchestra.run.smithi046.stdout:Cluster fsid: 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:28:38.996 INFO:teuthology.orchestra.run.smithi046.stdout:Acquiring lock 139980580985320 on /run/cephadm/04a8ae18-e566-11ee-95c9-87774f69a715.lock 2024-03-18T20:28:38.996 INFO:teuthology.orchestra.run.smithi046.stdout:Lock 139980580985320 acquired on /run/cephadm/04a8ae18-e566-11ee-95c9-87774f69a715.lock 2024-03-18T20:28:38.996 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 3300 ... 2024-03-18T20:28:38.996 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 6789 ... 2024-03-18T20:28:38.997 INFO:teuthology.orchestra.run.smithi046.stdout:Base mon IP(s) is [172.21.15.46:3300, 172.21.15.46:6789], mon addrv is [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-03-18T20:28:39.001 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2024-03-18T20:28:39.001 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.46 metric 100 2024-03-18T20:28:39.005 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-18T20:28:39.005 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-03-18T20:28:39.005 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1770sec hoplimit 64 pref medium 2024-03-18T20:28:39.007 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-18T20:28:39.007 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-18T20:28:39.007 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-18T20:28:39.007 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-18T20:28:39.007 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-18T20:28:39.007 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-18T20:28:39.008 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-03-18T20:28:39.008 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-03-18T20:28:39.008 INFO:teuthology.orchestra.run.smithi046.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-18T20:28:39.009 INFO:teuthology.orchestra.run.smithi046.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-18T20:28:39.010 INFO:teuthology.orchestra.run.smithi046.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3... 2024-03-18T20:28:39.418 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 015bdba7dcef68659e3352b62f3bbf78033bb173f439b9af177f23d5c5943567 2024-03-18T20:28:39.418 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3... 2024-03-18T20:28:39.418 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Getting image source signatures 2024-03-18T20:28:39.418 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:4bbf42c993c119ce89c30a67b341883c7867a88bea0dc325a65e3df8d508a4ef 2024-03-18T20:28:39.418 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-18T20:28:39.418 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying config sha256:015bdba7dcef68659e3352b62f3bbf78033bb173f439b9af177f23d5c5943567 2024-03-18T20:28:39.419 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-18T20:28:40.154 INFO:teuthology.orchestra.run.smithi046.stdout:ceph: stdout ceph version 18.2.2-914-g49bdc511 (49bdc51142759622a281d3e5327eebd07cbb8bb3) reef (stable) 2024-03-18T20:28:40.154 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph version: ceph version 18.2.2-914-g49bdc511 (49bdc51142759622a281d3e5327eebd07cbb8bb3) reef (stable) 2024-03-18T20:28:40.154 INFO:teuthology.orchestra.run.smithi046.stdout:Extracting ceph user uid/gid from container image... 2024-03-18T20:28:40.743 INFO:teuthology.orchestra.run.smithi046.stdout:stat: stdout 167 167 2024-03-18T20:28:40.743 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial keys... 2024-03-18T20:28:41.338 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQD5o/hl6TdSABAADQbuF9RTnn7j+h9VCialbQ== 2024-03-18T20:28:41.956 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQD5o/hlFMaJJBAAvvfDPaYaia/wMG7/Icxs9A== 2024-03-18T20:28:42.587 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQD6o/hldd0wDxAA2tYngaYui414GYOnvIiyIQ== 2024-03-18T20:28:42.588 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial monmap... 2024-03-18T20:28:43.188 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-18T20:28:43.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-18T20:28:43.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:28:43.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-18T20:28:43.189 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool for a [v2:172.21.15.46:3300,v1:172.21.15.46:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-18T20:28:43.189 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = pacific 2024-03-18T20:28:43.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: set fsid to 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:28:43.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-18T20:28:43.189 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:28:43.189 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mon... 2024-03-18T20:28:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:create mon.a on 2024-03-18T20:28:44.237 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-03-18T20:28:44.337 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-18T20:28:44.753 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-04a8ae18-e566-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-04a8ae18-e566-11ee-95c9-87774f69a715.target. 2024-03-18T20:28:44.754 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-04a8ae18-e566-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-04a8ae18-e566-11ee-95c9-87774f69a715.target. 2024-03-18T20:28:45.183 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.a 2024-03-18T20:28:45.183 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.a.service: Unit ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.a.service not loaded. 2024-03-18T20:28:45.284 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-04a8ae18-e566-11ee-95c9-87774f69a715.target.wants/ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-04a8ae18-e566-11ee-95c9-87774f69a715@.service. 2024-03-18T20:28:45.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:45 smithi046 systemd[1]: Starting Ceph mon.a for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:28:46.110 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T20:28:46.110 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-03-18T20:28:46.119 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T20:28:46.119 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-03-18T20:28:46.119 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-03-18T20:28:46.119 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-03-18T20:28:46.120 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon to start... 2024-03-18T20:28:46.120 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon... 2024-03-18T20:28:46.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:46 smithi046 bash[42973]: e9ddf2edde40daeff700a77904fff946b5e02f04d5b145fe14f6b8c990dca457 2024-03-18T20:28:46.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:46 smithi046 systemd[1]: Started Ceph mon.a for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:28:46.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:46 smithi046 ceph-mon[43010]: mkfs 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:28:46.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:46 smithi046 ceph-mon[43010]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-18T20:28:46.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout cluster: 2024-03-18T20:28:46.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout id: 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:28:46.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-18T20:28:46.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:28:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout services: 2024-03-18T20:28:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.489077s) 2024-03-18T20:28:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-18T20:28:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-18T20:28:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:28:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout data: 2024-03-18T20:28:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-18T20:28:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-18T20:28:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-18T20:28:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pgs: 2024-03-18T20:28:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:28:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.540+0000 7f4ccfb19700 1 Processor -- start 2024-03-18T20:28:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.540+0000 7f4ccfb19700 1 -- start start 2024-03-18T20:28:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.541+0000 7f4ccfb19700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 0x7f4cc81058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.541+0000 7f4ccfb19700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc8105e90 con 0x7f4cc81054a0 2024-03-18T20:28:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.541+0000 7f4ccd8b5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 0x7f4cc81058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.541+0000 7f4ccd8b5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 0x7f4cc81058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36492/0 (socket says 172.21.15.46:36492) 2024-03-18T20:28:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.541+0000 7f4ccd8b5700 1 -- 172.21.15.46:0/513461878 learned_addr learned my addr 172.21.15.46:0/513461878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.541+0000 7f4ccd8b5700 1 -- 172.21.15.46:0/513461878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc81066a0 con 0x7f4cc81054a0 2024-03-18T20:28:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.542+0000 7f4ccd8b5700 1 --2- 172.21.15.46:0/513461878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 0x7f4cc81058c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80098a0 tx=0x7f4cb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa6d026f0bf80a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.542+0000 7f4ccc8b3700 1 -- 172.21.15.46:0/513461878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cb8004160 con 0x7f4cc81054a0 2024-03-18T20:28:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.542+0000 7f4ccc8b3700 1 -- 172.21.15.46:0/513461878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb80042c0 con 0x7f4cc81054a0 2024-03-18T20:28:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.542+0000 7f4ccc8b3700 1 -- 172.21.15.46:0/513461878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cb80045a0 con 0x7f4cc81054a0 2024-03-18T20:28:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.542+0000 7f4ccfb19700 1 -- 172.21.15.46:0/513461878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 msgr2=0x7f4cc81058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.542+0000 7f4ccfb19700 1 --2- 172.21.15.46:0/513461878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 0x7f4cc81058c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80098a0 tx=0x7f4cb8009bb0 comp rx=0 tx=0).stop 2024-03-18T20:28:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.542+0000 7f4ccfb19700 1 -- 172.21.15.46:0/513461878 shutdown_connections 2024-03-18T20:28:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.542+0000 7f4ccfb19700 1 --2- 172.21.15.46:0/513461878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 0x7f4cc81058c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.542+0000 7f4ccfb19700 1 -- 172.21.15.46:0/513461878 >> 172.21.15.46:0/513461878 conn(0x7f4cc8100a20 msgr2=0x7f4cc8102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.543+0000 7f4ccfb19700 1 -- 172.21.15.46:0/513461878 shutdown_connections 2024-03-18T20:28:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.543+0000 7f4ccfb19700 1 -- 172.21.15.46:0/513461878 wait complete. 2024-03-18T20:28:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.543+0000 7f4ccfb19700 1 Processor -- start 2024-03-18T20:28:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.543+0000 7f4ccfb19700 1 -- start start 2024-03-18T20:28:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.544+0000 7f4ccfb19700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 0x7f4cc8079a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.544+0000 7f4ccfb19700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc8079f60 con 0x7f4cc81054a0 2024-03-18T20:28:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.544+0000 7f4ccd8b5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 0x7f4cc8079a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.544+0000 7f4ccd8b5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 0x7f4cc8079a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36502/0 (socket says 172.21.15.46:36502) 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.544+0000 7f4ccd8b5700 1 -- 172.21.15.46:0/2062892000 learned_addr learned my addr 172.21.15.46:0/2062892000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.544+0000 7f4ccd8b5700 1 -- 172.21.15.46:0/2062892000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb8009550 con 0x7f4cc81054a0 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.544+0000 7f4ccd8b5700 1 --2- 172.21.15.46:0/2062892000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 0x7f4cc8079a20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8004000 tx=0x7f4cb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.544+0000 7f4cbeffd700 1 -- 172.21.15.46:0/2062892000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cb8011630 con 0x7f4cc81054a0 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.544+0000 7f4cbeffd700 1 -- 172.21.15.46:0/2062892000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb8011790 con 0x7f4cc81054a0 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.544+0000 7f4cbeffd700 1 -- 172.21.15.46:0/2062892000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cb8011a30 con 0x7f4cc81054a0 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.544+0000 7f4ccfb19700 1 -- 172.21.15.46:0/2062892000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc819a5e0 con 0x7f4cc81054a0 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.544+0000 7f4ccfb19700 1 -- 172.21.15.46:0/2062892000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc819ee40 con 0x7f4cc81054a0 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.546+0000 7f4cbeffd700 1 -- 172.21.15.46:0/2062892000 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4cb8022070 con 0x7f4cc81054a0 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.546+0000 7f4cbeffd700 1 -- 172.21.15.46:0/2062892000 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4cb801c450 con 0x7f4cc81054a0 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.546+0000 7f4ccfb19700 1 -- 172.21.15.46:0/2062892000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc8192710 con 0x7f4cc81054a0 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.548+0000 7f4cbeffd700 1 -- 172.21.15.46:0/2062892000 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f4cb801b450 con 0x7f4cc81054a0 2024-03-18T20:28:46.988 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.624+0000 7f4ccfb19700 1 -- 172.21.15.46:0/2062892000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f4cc802ce80 con 0x7f4cc81054a0 2024-03-18T20:28:46.989 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.624+0000 7f4cbeffd700 1 -- 172.21.15.46:0/2062892000 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f4cb801b5f0 con 0x7f4cc81054a0 2024-03-18T20:28:46.989 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.625+0000 7f4ccfb19700 1 -- 172.21.15.46:0/2062892000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 msgr2=0x7f4cc8079a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:46.989 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.625+0000 7f4ccfb19700 1 --2- 172.21.15.46:0/2062892000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 0x7f4cc8079a20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8004000 tx=0x7f4cb8004750 comp rx=0 tx=0).stop 2024-03-18T20:28:46.989 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.625+0000 7f4ccfb19700 1 -- 172.21.15.46:0/2062892000 shutdown_connections 2024-03-18T20:28:46.989 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.625+0000 7f4ccfb19700 1 --2- 172.21.15.46:0/2062892000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cc81054a0 0x7f4cc8079a20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:46.989 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.625+0000 7f4ccfb19700 1 -- 172.21.15.46:0/2062892000 >> 172.21.15.46:0/2062892000 conn(0x7f4cc8100a20 msgr2=0x7f4cc818f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:46.989 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.625+0000 7f4ccfb19700 1 -- 172.21.15.46:0/2062892000 shutdown_connections 2024-03-18T20:28:46.989 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:46.625+0000 7f4ccfb19700 1 -- 172.21.15.46:0/2062892000 wait complete. 2024-03-18T20:28:46.989 INFO:teuthology.orchestra.run.smithi046.stdout:mon is available 2024-03-18T20:28:46.989 INFO:teuthology.orchestra.run.smithi046.stdout:Assimilating anything we can from ceph.conf... 2024-03-18T20:28:47.514 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:47 smithi046 ceph-mon[43010]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-18T20:28:47.515 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:47 smithi046 ceph-mon[43010]: monmap e1: 1 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T20:28:47.515 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:47 smithi046 ceph-mon[43010]: fsmap 2024-03-18T20:28:47.515 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:47 smithi046 ceph-mon[43010]: osdmap e1: 0 total, 0 up, 0 in 2024-03-18T20:28:47.515 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:47 smithi046 ceph-mon[43010]: mgrmap e1: no daemons active 2024-03-18T20:28:47.515 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:47 smithi046 ceph-mon[43010]: from='client.? 172.21.15.46:0/2062892000' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-18T20:28:47.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:28:47.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-03-18T20:28:47.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:28:47.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-03-18T20:28:47.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-18T20:28:47.837 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-18T20:28:47.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-18T20:28:47.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-18T20:28:47.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:28:47.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-18T20:28:47.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-18T20:28:47.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:28:47.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-03-18T20:28:47.838 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-18T20:28:47.845 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-18T20:28:47.845 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-18T20:28:47.845 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-18T20:28:47.845 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-18T20:28:47.845 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.406+0000 7f6407eef700 1 Processor -- start 2024-03-18T20:28:47.845 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.406+0000 7f6407eef700 1 -- start start 2024-03-18T20:28:47.845 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.406+0000 7f6407eef700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 0x7f6400109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:47.845 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.406+0000 7f6407eef700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6400079ed0 con 0x7f6400107040 2024-03-18T20:28:47.845 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.406+0000 7f6405c8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 0x7f6400109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:47.845 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.406+0000 7f6405c8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 0x7f6400109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36516/0 (socket says 172.21.15.46:36516) 2024-03-18T20:28:47.845 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.406+0000 7f6405c8b700 1 -- 172.21.15.46:0/762125258 learned_addr learned my addr 172.21.15.46:0/762125258 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:47.845 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.407+0000 7f6405c8b700 1 -- 172.21.15.46:0/762125258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6400109970 con 0x7f6400107040 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.407+0000 7f6405c8b700 1 --2- 172.21.15.46:0/762125258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 0x7f6400109430 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0098a0 tx=0x7f63fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbb05072ba35f5a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.407+0000 7f6404c89700 1 -- 172.21.15.46:0/762125258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63fc004160 con 0x7f6400107040 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.407+0000 7f6404c89700 1 -- 172.21.15.46:0/762125258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc0042c0 con 0x7f6400107040 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.407+0000 7f6404c89700 1 -- 172.21.15.46:0/762125258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63fc0045a0 con 0x7f6400107040 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.407+0000 7f6407eef700 1 -- 172.21.15.46:0/762125258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 msgr2=0x7f6400109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.408+0000 7f6407eef700 1 --2- 172.21.15.46:0/762125258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 0x7f6400109430 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0098a0 tx=0x7f63fc009bb0 comp rx=0 tx=0).stop 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.408+0000 7f6407eef700 1 -- 172.21.15.46:0/762125258 shutdown_connections 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.408+0000 7f6407eef700 1 --2- 172.21.15.46:0/762125258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 0x7f6400109430 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.408+0000 7f6407eef700 1 -- 172.21.15.46:0/762125258 >> 172.21.15.46:0/762125258 conn(0x7f6400100ab0 msgr2=0x7f6400102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.408+0000 7f6407eef700 1 -- 172.21.15.46:0/762125258 shutdown_connections 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.408+0000 7f6407eef700 1 -- 172.21.15.46:0/762125258 wait complete. 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.408+0000 7f6407eef700 1 Processor -- start 2024-03-18T20:28:47.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.408+0000 7f6407eef700 1 -- start start 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.409+0000 7f6407eef700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 0x7f6400106870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.409+0000 7f6407eef700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6400106db0 con 0x7f6400107040 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.409+0000 7f6405c8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 0x7f6400106870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.409+0000 7f6405c8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 0x7f6400106870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36524/0 (socket says 172.21.15.46:36524) 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.409+0000 7f6405c8b700 1 -- 172.21.15.46:0/646808417 learned_addr learned my addr 172.21.15.46:0/646808417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.409+0000 7f6405c8b700 1 -- 172.21.15.46:0/646808417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc009550 con 0x7f6400107040 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.409+0000 7f6405c8b700 1 --2- 172.21.15.46:0/646808417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 0x7f6400106870 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6400079310 tx=0x7f63fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.409+0000 7f63f6ffd700 1 -- 172.21.15.46:0/646808417 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63fc011630 con 0x7f6400107040 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.409+0000 7f63f6ffd700 1 -- 172.21.15.46:0/646808417 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc011790 con 0x7f6400107040 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.409+0000 7f6407eef700 1 -- 172.21.15.46:0/646808417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6400104ff0 con 0x7f6400107040 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.409+0000 7f63f6ffd700 1 -- 172.21.15.46:0/646808417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63fc011a30 con 0x7f6400107040 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.410+0000 7f6407eef700 1 -- 172.21.15.46:0/646808417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6400105490 con 0x7f6400107040 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.410+0000 7f63f6ffd700 1 -- 172.21.15.46:0/646808417 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f63fc021070 con 0x7f6400107040 2024-03-18T20:28:47.847 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.410+0000 7f63f6ffd700 1 -- 172.21.15.46:0/646808417 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f63fc01b440 con 0x7f6400107040 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.411+0000 7f6407eef700 1 -- 172.21.15.46:0/646808417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64000623e0 con 0x7f6400107040 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.413+0000 7f63f6ffd700 1 -- 172.21.15.46:0/646808417 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f63fc01a460 con 0x7f6400107040 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.485+0000 7f6407eef700 1 -- 172.21.15.46:0/646808417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f64001938f0 con 0x7f6400107040 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.489+0000 7f63f6ffd700 1 -- 172.21.15.46:0/646808417 <== mon.0 v2:172.21.15.46:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f63fc01a600 con 0x7f6400107040 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.489+0000 7f63f6ffd700 1 -- 172.21.15.46:0/646808417 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f63fc0334e0 con 0x7f6400107040 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.490+0000 7f6407eef700 1 -- 172.21.15.46:0/646808417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 msgr2=0x7f6400106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.490+0000 7f6407eef700 1 --2- 172.21.15.46:0/646808417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 0x7f6400106870 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6400079310 tx=0x7f63fc004750 comp rx=0 tx=0).stop 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.490+0000 7f6407eef700 1 -- 172.21.15.46:0/646808417 shutdown_connections 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.490+0000 7f6407eef700 1 --2- 172.21.15.46:0/646808417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6400107040 0x7f6400106870 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.490+0000 7f6407eef700 1 -- 172.21.15.46:0/646808417 >> 172.21.15.46:0/646808417 conn(0x7f6400100ab0 msgr2=0x7f6400100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.490+0000 7f6407eef700 1 -- 172.21.15.46:0/646808417 shutdown_connections 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:47.490+0000 7f6407eef700 1 -- 172.21.15.46:0/646808417 wait complete. 2024-03-18T20:28:47.848 INFO:teuthology.orchestra.run.smithi046.stdout:Generating new minimal ceph.conf... 2024-03-18T20:28:48.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:48 smithi046 ceph-mon[43010]: from='client.? 172.21.15.46:0/646808417' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-18T20:28:48.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:48 smithi046 ceph-mon[43010]: from='client.? 172.21.15.46:0/646808417' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-18T20:28:48.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.202+0000 7f27e0476700 1 Processor -- start 2024-03-18T20:28:48.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.202+0000 7f27e0476700 1 -- start start 2024-03-18T20:28:48.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.203+0000 7f27e0476700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 0x7f27d81058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:48.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.203+0000 7f27e0476700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d8105ec0 con 0x7f27d81054d0 2024-03-18T20:28:48.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.203+0000 7f27de212700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 0x7f27d81058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:48.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.203+0000 7f27de212700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 0x7f27d81058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36536/0 (socket says 172.21.15.46:36536) 2024-03-18T20:28:48.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.203+0000 7f27de212700 1 -- 172.21.15.46:0/2163180493 learned_addr learned my addr 172.21.15.46:0/2163180493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:48.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.203+0000 7f27de212700 1 -- 172.21.15.46:0/2163180493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d81066d0 con 0x7f27d81054d0 2024-03-18T20:28:48.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.203+0000 7f27de212700 1 --2- 172.21.15.46:0/2163180493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 0x7f27d81058f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f27d40098a0 tx=0x7f27d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc119fc221dc27f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:48.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.204+0000 7f27dd210700 1 -- 172.21.15.46:0/2163180493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27d4004160 con 0x7f27d81054d0 2024-03-18T20:28:48.650 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.204+0000 7f27dd210700 1 -- 172.21.15.46:0/2163180493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27d40042c0 con 0x7f27d81054d0 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.204+0000 7f27dd210700 1 -- 172.21.15.46:0/2163180493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27d4011a00 con 0x7f27d81054d0 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.204+0000 7f27e0476700 1 -- 172.21.15.46:0/2163180493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 msgr2=0x7f27d81058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.204+0000 7f27e0476700 1 --2- 172.21.15.46:0/2163180493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 0x7f27d81058f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f27d40098a0 tx=0x7f27d4009bb0 comp rx=0 tx=0).stop 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.204+0000 7f27e0476700 1 -- 172.21.15.46:0/2163180493 shutdown_connections 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.204+0000 7f27e0476700 1 --2- 172.21.15.46:0/2163180493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 0x7f27d81058f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.204+0000 7f27e0476700 1 -- 172.21.15.46:0/2163180493 >> 172.21.15.46:0/2163180493 conn(0x7f27d8100ab0 msgr2=0x7f27d8102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.204+0000 7f27e0476700 1 -- 172.21.15.46:0/2163180493 shutdown_connections 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.204+0000 7f27e0476700 1 -- 172.21.15.46:0/2163180493 wait complete. 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.205+0000 7f27e0476700 1 Processor -- start 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.205+0000 7f27e0476700 1 -- start start 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.205+0000 7f27e0476700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 0x7f27d819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.205+0000 7f27e0476700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d819a8f0 con 0x7f27d81054d0 2024-03-18T20:28:48.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.205+0000 7f27de212700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 0x7f27d819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.205+0000 7f27de212700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 0x7f27d819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36540/0 (socket says 172.21.15.46:36540) 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.205+0000 7f27de212700 1 -- 172.21.15.46:0/1816006425 learned_addr learned my addr 172.21.15.46:0/1816006425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.206+0000 7f27de212700 1 -- 172.21.15.46:0/1816006425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d4009550 con 0x7f27d81054d0 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.206+0000 7f27de212700 1 --2- 172.21.15.46:0/1816006425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 0x7f27d819a3b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f27d4011710 tx=0x7f27d4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.206+0000 7f27cf7fe700 1 -- 172.21.15.46:0/1816006425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27d4003aa0 con 0x7f27d81054d0 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.206+0000 7f27cf7fe700 1 -- 172.21.15.46:0/1816006425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27d4003c00 con 0x7f27d81054d0 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.206+0000 7f27e0476700 1 -- 172.21.15.46:0/1816006425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d819aaf0 con 0x7f27d81054d0 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.206+0000 7f27cf7fe700 1 -- 172.21.15.46:0/1816006425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27d4024800 con 0x7f27d81054d0 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.206+0000 7f27e0476700 1 -- 172.21.15.46:0/1816006425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d819af90 con 0x7f27d81054d0 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.206+0000 7f27cf7fe700 1 -- 172.21.15.46:0/1816006425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f27d4024d30 con 0x7f27d81054d0 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.206+0000 7f27cf7fe700 1 -- 172.21.15.46:0/1816006425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f27d40193c0 con 0x7f27d81054d0 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.207+0000 7f27e0476700 1 -- 172.21.15.46:0/1816006425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d81944b0 con 0x7f27d81054d0 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.209+0000 7f27cf7fe700 1 -- 172.21.15.46:0/1816006425 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f27d401abd0 con 0x7f27d81054d0 2024-03-18T20:28:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.282+0000 7f27e0476700 1 -- 172.21.15.46:0/1816006425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f27d80623e0 con 0x7f27d81054d0 2024-03-18T20:28:48.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.282+0000 7f27cf7fe700 1 -- 172.21.15.46:0/1816006425 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f27d401ad70 con 0x7f27d81054d0 2024-03-18T20:28:48.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.283+0000 7f27e0476700 1 -- 172.21.15.46:0/1816006425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 msgr2=0x7f27d819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:48.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.283+0000 7f27e0476700 1 --2- 172.21.15.46:0/1816006425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 0x7f27d819a3b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f27d4011710 tx=0x7f27d4011ad0 comp rx=0 tx=0).stop 2024-03-18T20:28:48.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.283+0000 7f27e0476700 1 -- 172.21.15.46:0/1816006425 shutdown_connections 2024-03-18T20:28:48.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.283+0000 7f27e0476700 1 --2- 172.21.15.46:0/1816006425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27d81054d0 0x7f27d819a3b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:48.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.283+0000 7f27e0476700 1 -- 172.21.15.46:0/1816006425 >> 172.21.15.46:0/1816006425 conn(0x7f27d8100ab0 msgr2=0x7f27d8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:48.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.283+0000 7f27e0476700 1 -- 172.21.15.46:0/1816006425 shutdown_connections 2024-03-18T20:28:48.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:48.283+0000 7f27e0476700 1 -- 172.21.15.46:0/1816006425 wait complete. 2024-03-18T20:28:48.653 INFO:teuthology.orchestra.run.smithi046.stdout:Restarting the monitor... 2024-03-18T20:28:48.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:48 smithi046 systemd[1]: Stopping Ceph mon.a for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:28:49.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:48 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a[42987]: 2024-03-18T20:28:48.868+0000 7f31e4651700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T20:28:49.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:48 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a[42987]: 2024-03-18T20:28:48.868+0000 7f31e4651700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-18T20:28:49.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 bash[43336]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a 2024-03-18T20:28:49.624 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 systemd[1]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.a.service: Succeeded. 2024-03-18T20:28:49.624 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 systemd[1]: Stopped Ceph mon.a for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:28:49.624 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 systemd[1]: Starting Ceph mon.a for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:28:49.671 INFO:teuthology.orchestra.run.smithi046.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-18T20:28:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 bash[43456]: 8f8069fe096e7be015e8ff11daa2049c9b318bb3afd2c299f4ed619786ef9e4b 2024-03-18T20:28:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 systemd[1]: Started Ceph mon.a for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:28:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: set uid:gid to 167:167 (ceph:ceph) 2024-03-18T20:28:50.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: ceph version 18.2.2-914-g49bdc511 (49bdc51142759622a281d3e5327eebd07cbb8bb3) reef (stable), process ceph-mon, pid 7 2024-03-18T20:28:50.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: pidfile_write: ignore empty --pid-file 2024-03-18T20:28:50.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: load: jerasure load: lrc 2024-03-18T20:28:50.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: RocksDB version: 7.9.2 2024-03-18T20:28:50.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Git sha 0 2024-03-18T20:28:50.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Compile date 2024-03-16 00:14:49 2024-03-18T20:28:50.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: DB SUMMARY 2024-03-18T20:28:50.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: DB Session ID: 64SSBO6IIHYTRR92L6LW 2024-03-18T20:28:50.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: CURRENT file: CURRENT 2024-03-18T20:28:50.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: IDENTITY file: IDENTITY 2024-03-18T20:28:50.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-18T20:28:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-18T20:28:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98238 ; 2024-03-18T20:28:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.error_if_exists: 0 2024-03-18T20:28:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.create_if_missing: 0 2024-03-18T20:28:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.paranoid_checks: 1 2024-03-18T20:28:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-18T20:28:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-18T20:28:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-18T20:28:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.env: 0x561f7f30f720 2024-03-18T20:28:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.fs: PosixFileSystem 2024-03-18T20:28:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.info_log: 0x561f827cb4e0 2024-03-18T20:28:50.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_file_opening_threads: 16 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.statistics: (nil) 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.use_fsync: 0 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_log_file_size: 0 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.keep_log_file_num: 1000 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.recycle_log_file_num: 0 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.allow_fallocate: 1 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.allow_mmap_reads: 0 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.allow_mmap_writes: 0 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.use_direct_reads: 0 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-18T20:28:50.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.create_missing_column_families: 0 2024-03-18T20:28:50.051 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.db_log_dir: 2024-03-18T20:28:50.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.wal_dir: 2024-03-18T20:28:50.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-18T20:28:50.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-18T20:28:50.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-18T20:28:50.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-18T20:28:50.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-18T20:28:50.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-18T20:28:50.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.advise_random_on_open: 1 2024-03-18T20:28:50.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.db_write_buffer_size: 0 2024-03-18T20:28:50.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.write_buffer_manager: 0x561f81a64320 2024-03-18T20:28:50.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-18T20:28:50.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.rate_limiter: (nil) 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.wal_recovery_mode: 2 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.enable_thread_tracking: 0 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.enable_pipelined_write: 0 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.unordered_write: 0 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.row_cache: None 2024-03-18T20:28:50.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.wal_filter: None 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.allow_ingest_behind: 0 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.two_write_queues: 0 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.manual_wal_flush: 0 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.wal_compression: 0 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.atomic_flush: 0 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.log_readahead_size: 0 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T20:28:50.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_background_jobs: 2 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_background_compactions: -1 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_subcompactions: 1 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T20:28:50.056 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T20:28:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T20:28:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T20:28:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_open_files: -1 2024-03-18T20:28:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T20:28:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T20:28:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T20:28:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T20:28:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_background_flushes: -1 2024-03-18T20:28:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Compression algorithms supported: 2024-03-18T20:28:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T20:28:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: kZSTD supported: 0 2024-03-18T20:28:50.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: kXpressCompression supported: 0 2024-03-18T20:28:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T20:28:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: kZlibCompression supported: 1 2024-03-18T20:28:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: kSnappyCompression supported: 1 2024-03-18T20:28:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: kLZ4Compression supported: 1 2024-03-18T20:28:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: kBZip2Compression supported: 0 2024-03-18T20:28:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T20:28:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-18T20:28:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-18T20:28:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-18T20:28:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T20:28:50.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.merge_operator: 2024-03-18T20:28:50.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_filter: None 2024-03-18T20:28:50.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T20:28:50.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T20:28:50.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T20:28:50.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T20:28:50.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561f827cb5e0) 2024-03-18T20:28:50.059 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks: 1 2024-03-18T20:28:50.059 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T20:28:50.059 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T20:28:50.059 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_top_level_index_and_filter: 1 2024-03-18T20:28:50.059 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_type: 0 2024-03-18T20:28:50.060 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_index_type: 0 2024-03-18T20:28:50.060 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_shortening: 1 2024-03-18T20:28:50.060 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T20:28:50.060 INFO:journalctl@ceph.mon.a.smithi046.stdout: checksum: 4 2024-03-18T20:28:50.060 INFO:journalctl@ceph.mon.a.smithi046.stdout: no_block_cache: 0 2024-03-18T20:28:50.060 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache: 0x561f81ae51f0 2024-03-18T20:28:50.060 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_name: BinnedLRUCache 2024-03-18T20:28:50.060 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_options: 2024-03-18T20:28:50.060 INFO:journalctl@ceph.mon.a.smithi046.stdout: capacity : 536870912 2024-03-18T20:28:50.060 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_shard_bits : 4 2024-03-18T20:28:50.060 INFO:journalctl@ceph.mon.a.smithi046.stdout: strict_capacity_limit : 0 2024-03-18T20:28:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout: high_pri_pool_ratio: 0.000 2024-03-18T20:28:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_compressed: (nil) 2024-03-18T20:28:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout: persistent_cache: (nil) 2024-03-18T20:28:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size: 4096 2024-03-18T20:28:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size_deviation: 10 2024-03-18T20:28:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_restart_interval: 16 2024-03-18T20:28:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_block_restart_interval: 1 2024-03-18T20:28:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout: metadata_block_size: 4096 2024-03-18T20:28:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout: partition_filters: 0 2024-03-18T20:28:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout: use_delta_encoding: 1 2024-03-18T20:28:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout: filter_policy: bloomfilter 2024-03-18T20:28:50.061 INFO:journalctl@ceph.mon.a.smithi046.stdout: whole_key_filtering: 1 2024-03-18T20:28:50.062 INFO:journalctl@ceph.mon.a.smithi046.stdout: verify_compression: 0 2024-03-18T20:28:50.062 INFO:journalctl@ceph.mon.a.smithi046.stdout: read_amp_bytes_per_bit: 0 2024-03-18T20:28:50.062 INFO:journalctl@ceph.mon.a.smithi046.stdout: format_version: 5 2024-03-18T20:28:50.062 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_index_compression: 1 2024-03-18T20:28:50.062 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_align: 0 2024-03-18T20:28:50.062 INFO:journalctl@ceph.mon.a.smithi046.stdout: max_auto_readahead_size: 262144 2024-03-18T20:28:50.062 INFO:journalctl@ceph.mon.a.smithi046.stdout: prepopulate_block_cache: 0 2024-03-18T20:28:50.062 INFO:journalctl@ceph.mon.a.smithi046.stdout: initial_auto_readahead_size: 8192 2024-03-18T20:28:50.062 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_file_reads_for_auto_readahead: 2 2024-03-18T20:28:50.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T20:28:50.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T20:28:50.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compression: NoCompression 2024-03-18T20:28:50.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T20:28:50.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T20:28:50.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T20:28:50.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.num_levels: 7 2024-03-18T20:28:50.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T20:28:50.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T20:28:50.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T20:28:50.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T20:28:50.063 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T20:28:50.064 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T20:28:50.064 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T20:28:50.064 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T20:28:50.064 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T20:28:50.064 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T20:28:50.064 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-18T20:28:50.064 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-18T20:28:50.064 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T20:28:50.064 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T20:28:50.064 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T20:28:50.064 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T20:28:50.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T20:28:50.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T20:28:50.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T20:28:50.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T20:28:50.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T20:28:50.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T20:28:50.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T20:28:50.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T20:28:50.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T20:28:50.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T20:28:50.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T20:28:50.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T20:28:50.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T20:28:50.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T20:28:50.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T20:28:50.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T20:28:50.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T20:28:50.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T20:28:50.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T20:28:50.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T20:28:50.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T20:28:50.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T20:28:50.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T20:28:50.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T20:28:50.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T20:28:50.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T20:28:50.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T20:28:50.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T20:28:50.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T20:28:50.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T20:28:50.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T20:28:50.068 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T20:28:50.068 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T20:28:50.068 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T20:28:50.068 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T20:28:50.068 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T20:28:50.068 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T20:28:50.068 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.table_properties_collectors: 2024-03-18T20:28:50.068 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.inplace_update_support: 0 2024-03-18T20:28:50.068 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T20:28:50.069 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T20:28:50.069 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T20:28:50.069 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T20:28:50.069 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.bloom_locality: 0 2024-03-18T20:28:50.069 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.max_successive_merges: 0 2024-03-18T20:28:50.069 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T20:28:50.069 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T20:28:50.069 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T20:28:50.069 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T20:28:50.069 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.ttl: 2592000 2024-03-18T20:28:50.069 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T20:28:50.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T20:28:50.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T20:28:50.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.enable_blob_files: false 2024-03-18T20:28:50.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.min_blob_size: 0 2024-03-18T20:28:50.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T20:28:50.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T20:28:50.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T20:28:50.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T20:28:50.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T20:28:50.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T20:28:50.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T20:28:50.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T20:28:50.071 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-18T20:28:50.071 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-18T20:28:50.071 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 36d6d0bd-ab1f-412c-93f9-7b9c1db66d7f 2024-03-18T20:28:50.071 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710793729680453, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-18T20:28:50.071 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-18T20:28:50.071 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710793729681682, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93565, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 311, "table_properties": {"data_size": 91556, "index_size": 291, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13802, "raw_average_key_size": 50, "raw_value_size": 84054, "raw_average_value_size": 306, "num_data_blocks": 11, "num_entries": 274, "num_filter_entries": 274, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710793729, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "36d6d0bd-ab1f-412c-93f9-7b9c1db66d7f", "db_session_id": "64SSBO6IIHYTRR92L6LW", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-18T20:28:50.071 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710793729681773, "job": 1, "event": "recovery_finished"} 2024-03-18T20:28:50.071 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-18T20:28:50.071 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T20:28:50.071 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561f81b82000 2024-03-18T20:28:50.071 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: DB pointer 0x561f81b70000 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** DB Stats ** 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-03-18T20:28:50.072 INFO:journalctl@ceph.mon.a.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T20:28:50.073 INFO:journalctl@ceph.mon.a.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T20:28:50.073 INFO:journalctl@ceph.mon.a.smithi046.stdout: L0 2/0 93.21 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T20:28:50.073 INFO:journalctl@ceph.mon.a.smithi046.stdout: Sum 2/0 93.21 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T20:28:50.073 INFO:journalctl@ceph.mon.a.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T20:28:50.073 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-03-18T20:28:50.073 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-03-18T20:28:50.073 INFO:journalctl@ceph.mon.a.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T20:28:50.073 INFO:journalctl@ceph.mon.a.smithi046.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T20:28:50.073 INFO:journalctl@ceph.mon.a.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 117.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T20:28:50.073 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-03-18T20:28:50.073 INFO:journalctl@ceph.mon.a.smithi046.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative compaction: 0.00 GB write, 22.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval compaction: 0.00 GB write, 22.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache BinnedLRUCache@0x561f81ae51f0#7 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-18T20:28:50.074 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: starting mon.a rank 0 at public addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] at bind addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: mon.a@-1(???) e1 preinit fsid 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: mon.a@-1(???).mds e1 new map 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: mon.a@-1(???).mds e1 print_map 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout: e1 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout: legacy client fscid: -1 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout: No filesystems configured 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T20:28:50.075 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T20:28:50.076 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T20:28:50.076 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-18T20:28:50.076 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-18T20:28:50.076 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: monmap e1: 1 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T20:28:50.076 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: fsmap 2024-03-18T20:28:50.076 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: osdmap e1: 0 total, 0 up, 0 in 2024-03-18T20:28:50.076 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:49 smithi046 ceph-mon[43495]: mgrmap e1: no daemons active 2024-03-18T20:28:50.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.026+0000 7f3981587700 1 Processor -- start 2024-03-18T20:28:50.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.026+0000 7f3981587700 1 -- start start 2024-03-18T20:28:50.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.026+0000 7f3981587700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 0x7f397c105df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:50.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.026+0000 7f3981587700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c1063c0 con 0x7f397c1059d0 2024-03-18T20:28:50.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.026+0000 7f397affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 0x7f397c105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:50.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.026+0000 7f397affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 0x7f397c105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36546/0 (socket says 172.21.15.46:36546) 2024-03-18T20:28:50.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.026+0000 7f397affd700 1 -- 172.21.15.46:0/2785399518 learned_addr learned my addr 172.21.15.46:0/2785399518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:50.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.027+0000 7f397affd700 1 -- 172.21.15.46:0/2785399518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c1066f0 con 0x7f397c1059d0 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.027+0000 7f397affd700 1 --2- 172.21.15.46:0/2785399518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 0x7f397c105df0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f396c0098a0 tx=0x7f396c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54395545af6e395 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.027+0000 7f3979ffb700 1 -- 172.21.15.46:0/2785399518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f396c004160 con 0x7f397c1059d0 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.027+0000 7f3979ffb700 1 -- 172.21.15.46:0/2785399518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f396c0042c0 con 0x7f397c1059d0 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.027+0000 7f3979ffb700 1 -- 172.21.15.46:0/2785399518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f396c011a00 con 0x7f397c1059d0 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.028+0000 7f3981587700 1 -- 172.21.15.46:0/2785399518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 msgr2=0x7f397c105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.028+0000 7f3981587700 1 --2- 172.21.15.46:0/2785399518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 0x7f397c105df0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f396c0098a0 tx=0x7f396c009bb0 comp rx=0 tx=0).stop 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.028+0000 7f3981587700 1 -- 172.21.15.46:0/2785399518 shutdown_connections 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.028+0000 7f3981587700 1 --2- 172.21.15.46:0/2785399518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 0x7f397c105df0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.028+0000 7f3981587700 1 -- 172.21.15.46:0/2785399518 >> 172.21.15.46:0/2785399518 conn(0x7f397c100210 msgr2=0x7f397c102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.028+0000 7f3981587700 1 -- 172.21.15.46:0/2785399518 shutdown_connections 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.028+0000 7f3981587700 1 -- 172.21.15.46:0/2785399518 wait complete. 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.029+0000 7f3981587700 1 Processor -- start 2024-03-18T20:28:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.029+0000 7f3981587700 1 -- start start 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.029+0000 7f3981587700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 0x7f397c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.029+0000 7f3981587700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c19a840 con 0x7f397c1059d0 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.029+0000 7f397affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 0x7f397c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.029+0000 7f397affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 0x7f397c19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36556/0 (socket says 172.21.15.46:36556) 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.029+0000 7f397affd700 1 -- 172.21.15.46:0/1469366365 learned_addr learned my addr 172.21.15.46:0/1469366365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.030+0000 7f397affd700 1 -- 172.21.15.46:0/1469366365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c009550 con 0x7f397c1059d0 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.030+0000 7f397affd700 1 --2- 172.21.15.46:0/1469366365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 0x7f397c19a300 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f396c009130 tx=0x7f396c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.030+0000 7f3963fff700 1 -- 172.21.15.46:0/1469366365 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f396c003b70 con 0x7f397c1059d0 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.030+0000 7f3981587700 1 -- 172.21.15.46:0/1469366365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c19aa40 con 0x7f397c1059d0 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.030+0000 7f3963fff700 1 -- 172.21.15.46:0/1469366365 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f396c003cd0 con 0x7f397c1059d0 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.030+0000 7f3963fff700 1 -- 172.21.15.46:0/1469366365 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f396c0197d0 con 0x7f397c1059d0 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.030+0000 7f3981587700 1 -- 172.21.15.46:0/1469366365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c19aee0 con 0x7f397c1059d0 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.030+0000 7f3963fff700 1 -- 172.21.15.46:0/1469366365 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f396c019a10 con 0x7f397c1059d0 2024-03-18T20:28:50.559 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.030+0000 7f3963fff700 1 -- 172.21.15.46:0/1469366365 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f396c00b720 con 0x7f397c1059d0 2024-03-18T20:28:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.031+0000 7f3981587700 1 -- 172.21.15.46:0/1469366365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397c1944a0 con 0x7f397c1059d0 2024-03-18T20:28:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.034+0000 7f3963fff700 1 -- 172.21.15.46:0/1469366365 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f396c02a060 con 0x7f397c1059d0 2024-03-18T20:28:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.106+0000 7f3981587700 1 -- 172.21.15.46:0/1469366365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f397c02d0d0 con 0x7f397c1059d0 2024-03-18T20:28:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.108+0000 7f3963fff700 1 -- 172.21.15.46:0/1469366365 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f396c019e50 con 0x7f397c1059d0 2024-03-18T20:28:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.109+0000 7f3981587700 1 -- 172.21.15.46:0/1469366365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 msgr2=0x7f397c19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.109+0000 7f3981587700 1 --2- 172.21.15.46:0/1469366365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 0x7f397c19a300 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f396c009130 tx=0x7f396c011ad0 comp rx=0 tx=0).stop 2024-03-18T20:28:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.109+0000 7f3981587700 1 -- 172.21.15.46:0/1469366365 shutdown_connections 2024-03-18T20:28:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.109+0000 7f3981587700 1 --2- 172.21.15.46:0/1469366365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f397c1059d0 0x7f397c19a300 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.109+0000 7f3981587700 1 -- 172.21.15.46:0/1469366365 >> 172.21.15.46:0/1469366365 conn(0x7f397c100210 msgr2=0x7f397c101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.109+0000 7f3981587700 1 -- 172.21.15.46:0/1469366365 shutdown_connections 2024-03-18T20:28:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:50.109+0000 7f3981587700 1 -- 172.21.15.46:0/1469366365 wait complete. 2024-03-18T20:28:50.560 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-18T20:28:50.586 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-18T20:28:50.586 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mgr... 2024-03-18T20:28:50.587 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-18T20:28:50.587 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-18T20:28:50.788 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.x 2024-03-18T20:28:50.789 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.x.service: Unit ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.x.service not loaded. 2024-03-18T20:28:50.890 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-04a8ae18-e566-11ee-95c9-87774f69a715.target.wants/ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-04a8ae18-e566-11ee-95c9-87774f69a715@.service. 2024-03-18T20:28:51.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:50 smithi046 systemd[1]: Starting Ceph mgr.x for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:28:51.317 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T20:28:51.317 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-03-18T20:28:51.326 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T20:28:51.326 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-03-18T20:28:51.326 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-03-18T20:28:51.326 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-03-18T20:28:51.336 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T20:28:51.336 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-03-18T20:28:51.345 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T20:28:51.345 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-03-18T20:28:51.345 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-03-18T20:28:51.345 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-18T20:28:51.345 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr to start... 2024-03-18T20:28:51.345 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr... 2024-03-18T20:28:51.454 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:51 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1469366365' entity='client.admin' 2024-03-18T20:28:51.454 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:51 smithi046 bash[43729]: a5e0c557f3b2eceaef9a5261ca5e596832248d5969365c14b10ca47b3d25d5a4 2024-03-18T20:28:51.454 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:51 smithi046 systemd[1]: Started Ceph mgr.x for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:28:52.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:51 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:51.761+0000 7faf3b8fe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T20:28:52.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:51 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:51.871+0000 7faf3b8fe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T20:28:52.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:51 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:51.971+0000 7faf3b8fe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T20:28:52.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:28:52.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-03-18T20:28:52.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:28:52.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T20:28:52.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T20:28:52.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T20:28:52.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T20:28:52.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:52.163 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T20:28:52.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T20:28:52.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T20:28:52.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T20:28:52.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T20:28:52.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T20:28:52.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-03-18T20:28:52.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T20:28:52.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:52.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T20:28:52.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:28:52.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T20:28:46.137994+0000", 2024-03-18T20:28:52.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.719+0000 7fed760ad700 1 Processor -- start 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.719+0000 7fed760ad700 1 -- start start 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.720+0000 7fed760ad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 0x7fed70109460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.720+0000 7fed760ad700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed700744c0 con 0x7fed70107070 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.720+0000 7fed6f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 0x7fed70109460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.720+0000 7fed6f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 0x7fed70109460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36576/0 (socket says 172.21.15.46:36576) 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.720+0000 7fed6f7fe700 1 -- 172.21.15.46:0/3856713913 learned_addr learned my addr 172.21.15.46:0/3856713913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.721+0000 7fed6f7fe700 1 -- 172.21.15.46:0/3856713913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed701099a0 con 0x7fed70107070 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.721+0000 7fed6f7fe700 1 --2- 172.21.15.46:0/3856713913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 0x7fed70109460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fed600098a0 tx=0x7fed60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834d67441039f7e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:52.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.721+0000 7fed6e7fc700 1 -- 172.21.15.46:0/3856713913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed60004160 con 0x7fed70107070 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.721+0000 7fed6e7fc700 1 -- 172.21.15.46:0/3856713913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed600042c0 con 0x7fed70107070 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.721+0000 7fed6e7fc700 1 -- 172.21.15.46:0/3856713913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed60011a00 con 0x7fed70107070 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.722+0000 7fed760ad700 1 -- 172.21.15.46:0/3856713913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 msgr2=0x7fed70109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.722+0000 7fed760ad700 1 --2- 172.21.15.46:0/3856713913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 0x7fed70109460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fed600098a0 tx=0x7fed60009bb0 comp rx=0 tx=0).stop 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.722+0000 7fed760ad700 1 -- 172.21.15.46:0/3856713913 shutdown_connections 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.722+0000 7fed760ad700 1 --2- 172.21.15.46:0/3856713913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 0x7fed70109460 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.722+0000 7fed760ad700 1 -- 172.21.15.46:0/3856713913 >> 172.21.15.46:0/3856713913 conn(0x7fed70100aa0 msgr2=0x7fed70102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.722+0000 7fed760ad700 1 -- 172.21.15.46:0/3856713913 shutdown_connections 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.722+0000 7fed760ad700 1 -- 172.21.15.46:0/3856713913 wait complete. 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.723+0000 7fed760ad700 1 Processor -- start 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.723+0000 7fed760ad700 1 -- start start 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.723+0000 7fed760ad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 0x7fed7019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:52.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.723+0000 7fed760ad700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7019ca30 con 0x7fed70107070 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.723+0000 7fed6f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 0x7fed7019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.724+0000 7fed6f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 0x7fed7019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36590/0 (socket says 172.21.15.46:36590) 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.724+0000 7fed6f7fe700 1 -- 172.21.15.46:0/3917033927 learned_addr learned my addr 172.21.15.46:0/3917033927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.724+0000 7fed6f7fe700 1 -- 172.21.15.46:0/3917033927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed60009550 con 0x7fed70107070 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.724+0000 7fed6f7fe700 1 --2- 172.21.15.46:0/3917033927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 0x7fed7019c4f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fed60009130 tx=0x7fed60011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.724+0000 7fed6cff9700 1 -- 172.21.15.46:0/3917033927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed60003b70 con 0x7fed70107070 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.724+0000 7fed6cff9700 1 -- 172.21.15.46:0/3917033927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed60003cd0 con 0x7fed70107070 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.724+0000 7fed760ad700 1 -- 172.21.15.46:0/3917033927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7019cc30 con 0x7fed70107070 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.724+0000 7fed6cff9700 1 -- 172.21.15.46:0/3917033927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed600197d0 con 0x7fed70107070 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.724+0000 7fed760ad700 1 -- 172.21.15.46:0/3917033927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7019d0d0 con 0x7fed70107070 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.725+0000 7fed6cff9700 1 -- 172.21.15.46:0/3917033927 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fed600199c0 con 0x7fed70107070 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.725+0000 7fed6cff9700 1 -- 172.21.15.46:0/3917033927 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fed6000b720 con 0x7fed70107070 2024-03-18T20:28:52.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.725+0000 7fed760ad700 1 -- 172.21.15.46:0/3917033927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7010a740 con 0x7fed70107070 2024-03-18T20:28:52.170 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.728+0000 7fed6cff9700 1 -- 172.21.15.46:0/3917033927 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fed6002a060 con 0x7fed70107070 2024-03-18T20:28:52.170 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.804+0000 7fed760ad700 1 -- 172.21.15.46:0/3917033927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fed7010a900 con 0x7fed70107070 2024-03-18T20:28:52.170 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.805+0000 7fed6cff9700 1 -- 172.21.15.46:0/3917033927 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fed60019e00 con 0x7fed70107070 2024-03-18T20:28:52.170 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.806+0000 7fed760ad700 1 -- 172.21.15.46:0/3917033927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 msgr2=0x7fed7019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:52.170 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.806+0000 7fed760ad700 1 --2- 172.21.15.46:0/3917033927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 0x7fed7019c4f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fed60009130 tx=0x7fed60011ad0 comp rx=0 tx=0).stop 2024-03-18T20:28:52.170 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.806+0000 7fed760ad700 1 -- 172.21.15.46:0/3917033927 shutdown_connections 2024-03-18T20:28:52.170 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.806+0000 7fed760ad700 1 --2- 172.21.15.46:0/3917033927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed70107070 0x7fed7019c4f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:52.170 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.806+0000 7fed760ad700 1 -- 172.21.15.46:0/3917033927 >> 172.21.15.46:0/3917033927 conn(0x7fed70100aa0 msgr2=0x7fed701015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:52.170 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.806+0000 7fed760ad700 1 -- 172.21.15.46:0/3917033927 shutdown_connections 2024-03-18T20:28:52.170 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:51.806+0000 7fed760ad700 1 -- 172.21.15.46:0/3917033927 wait complete. 2024-03-18T20:28:52.170 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (1/15)... 2024-03-18T20:28:52.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:52 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3917033927' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T20:28:52.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:52 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:52.066+0000 7faf3b8fe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T20:28:53.093 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:52 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:52.796+0000 7faf3b8fe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:28:53.493 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:53 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:53.089+0000 7faf3b8fe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T20:28:53.493 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:53 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:53.185+0000 7faf3b8fe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T20:28:53.493 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:53 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:53.309+0000 7faf3b8fe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T20:28:53.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:53 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:53.489+0000 7faf3b8fe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T20:28:53.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:53 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:53.633+0000 7faf3b8fe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T20:28:53.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:53 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:53.729+0000 7faf3b8fe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T20:28:54.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:54 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:54.150+0000 7faf3b8fe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T20:28:54.903 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:54 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:54.607+0000 7faf3b8fe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T20:28:54.904 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:54 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:54.829+0000 7faf3b8fe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T20:28:54.904 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:54 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1022064287' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T20:28:54.970 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:28:54.970 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-03-18T20:28:54.970 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:28:54.970 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T20:28:54.970 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T20:28:54.970 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T20:28:54.970 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T20:28:54.970 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T20:28:54.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:28:54.972 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T20:28:54.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T20:28:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T20:28:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T20:28:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T20:28:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T20:28:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T20:28:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T20:28:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T20:28:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T20:28:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T20:28:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T20:28:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T20:28:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T20:28:46.137994+0000", 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.532+0000 7f6f6ab7b700 1 Processor -- start 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.532+0000 7f6f6ab7b700 1 -- start start 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.532+0000 7f6f6ab7b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 0x7f6f64105280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.532+0000 7f6f6ab7b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f640744c0 con 0x7f6f64104e60 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.532+0000 7f6f68917700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 0x7f6f64105280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.532+0000 7f6f68917700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 0x7f6f64105280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36606/0 (socket says 172.21.15.46:36606) 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.532+0000 7f6f68917700 1 -- 172.21.15.46:0/1877103700 learned_addr learned my addr 172.21.15.46:0/1877103700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.532+0000 7f6f68917700 1 -- 172.21.15.46:0/1877103700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f641057c0 con 0x7f6f64104e60 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.533+0000 7f6f68917700 1 --2- 172.21.15.46:0/1877103700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 0x7f6f64105280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6f580098a0 tx=0x7f6f58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fbcc27aa41f390 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.533+0000 7f6f637fe700 1 -- 172.21.15.46:0/1877103700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f58004160 con 0x7f6f64104e60 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.533+0000 7f6f637fe700 1 -- 172.21.15.46:0/1877103700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f580042c0 con 0x7f6f64104e60 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.533+0000 7f6f637fe700 1 -- 172.21.15.46:0/1877103700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f58011a00 con 0x7f6f64104e60 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.533+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1877103700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 msgr2=0x7f6f64105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.533+0000 7f6f6ab7b700 1 --2- 172.21.15.46:0/1877103700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 0x7f6f64105280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6f580098a0 tx=0x7f6f58009bb0 comp rx=0 tx=0).stop 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.534+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1877103700 shutdown_connections 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.534+0000 7f6f6ab7b700 1 --2- 172.21.15.46:0/1877103700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 0x7f6f64105280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.534+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1877103700 >> 172.21.15.46:0/1877103700 conn(0x7f6f64100aa0 msgr2=0x7f6f64102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:54.984 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.534+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1877103700 shutdown_connections 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.534+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1877103700 wait complete. 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.534+0000 7f6f6ab7b700 1 Processor -- start 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.534+0000 7f6f6ab7b700 1 -- start start 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.535+0000 7f6f6ab7b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 0x7f6f6419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.535+0000 7f6f6ab7b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6419ca50 con 0x7f6f64104e60 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.535+0000 7f6f68917700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 0x7f6f6419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.535+0000 7f6f68917700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 0x7f6f6419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36614/0 (socket says 172.21.15.46:36614) 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.535+0000 7f6f68917700 1 -- 172.21.15.46:0/1022064287 learned_addr learned my addr 172.21.15.46:0/1022064287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.535+0000 7f6f68917700 1 -- 172.21.15.46:0/1022064287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f58009550 con 0x7f6f64104e60 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.535+0000 7f6f68917700 1 --2- 172.21.15.46:0/1022064287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 0x7f6f6419c510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009130 tx=0x7f6f58011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.535+0000 7f6f61ffb700 1 -- 172.21.15.46:0/1022064287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f58003b70 con 0x7f6f64104e60 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.535+0000 7f6f61ffb700 1 -- 172.21.15.46:0/1022064287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f58003cd0 con 0x7f6f64104e60 2024-03-18T20:28:54.985 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.535+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1022064287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6419cc50 con 0x7f6f64104e60 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.536+0000 7f6f61ffb700 1 -- 172.21.15.46:0/1022064287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f580197d0 con 0x7f6f64104e60 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.536+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1022064287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6419d0f0 con 0x7f6f64104e60 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.536+0000 7f6f61ffb700 1 -- 172.21.15.46:0/1022064287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6f58019a10 con 0x7f6f64104e60 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.536+0000 7f6f61ffb700 1 -- 172.21.15.46:0/1022064287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6f5800b720 con 0x7f6f64104e60 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.537+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1022064287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f64196280 con 0x7f6f64104e60 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.539+0000 7f6f61ffb700 1 -- 172.21.15.46:0/1022064287 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f6f5802a060 con 0x7f6f64104e60 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.616+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1022064287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6f6402cf60 con 0x7f6f64104e60 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.616+0000 7f6f61ffb700 1 -- 172.21.15.46:0/1022064287 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f6f58019e50 con 0x7f6f64104e60 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.617+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1022064287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 msgr2=0x7f6f6419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.617+0000 7f6f6ab7b700 1 --2- 172.21.15.46:0/1022064287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 0x7f6f6419c510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009130 tx=0x7f6f58011ad0 comp rx=0 tx=0).stop 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.618+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1022064287 shutdown_connections 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.618+0000 7f6f6ab7b700 1 --2- 172.21.15.46:0/1022064287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f64104e60 0x7f6f6419c510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:54.986 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.618+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1022064287 >> 172.21.15.46:0/1022064287 conn(0x7f6f64100aa0 msgr2=0x7f6f641933c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:54.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.618+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1022064287 shutdown_connections 2024-03-18T20:28:54.987 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:54.618+0000 7f6f6ab7b700 1 -- 172.21.15.46:0/1022064287 wait complete. 2024-03-18T20:28:54.987 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (2/15)... 2024-03-18T20:28:55.297 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:54 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:54.935+0000 7faf3b8fe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T20:28:56.084 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:55 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:55.804+0000 7faf3b8fe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T20:28:56.084 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:55 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:55.913+0000 7faf3b8fe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T20:28:57.225 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:56 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:56.080+0000 7faf3b8fe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T20:28:57.225 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:56 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:56.796+0000 7faf3b8fe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T20:28:57.225 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:57 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:57.004+0000 7faf3b8fe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T20:28:57.225 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:57 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:57.221+0000 7faf3b8fe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T20:28:58.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:58 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3498064742' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T20:28:58.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:58 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:58.292+0000 7faf3b8fe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T20:28:58.936 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:58 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:58.618+0000 7faf3b8fe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:28:58.936 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:58 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:58.717+0000 7faf3b8fe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T20:28:59.216 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:28:58 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:28:58.932+0000 7faf3b8fe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T20:28:59.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: Activating manager daemon x 2024-03-18T20:28:59.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: mgrmap e2: x(active, starting, since 0.00339943s) 2024-03-18T20:28:59.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: from='mgr.14100 172.21.15.46:0/2738524787' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T20:28:59.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: from='mgr.14100 172.21.15.46:0/2738524787' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T20:28:59.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: from='mgr.14100 172.21.15.46:0/2738524787' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T20:28:59.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: from='mgr.14100 172.21.15.46:0/2738524787' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:28:59.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: from='mgr.14100 172.21.15.46:0/2738524787' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T20:28:59.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: Manager daemon x is now available 2024-03-18T20:28:59.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: from='mgr.14100 172.21.15.46:0/2738524787' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T20:28:59.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: from='mgr.14100 172.21.15.46:0/2738524787' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T20:28:59.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: from='mgr.14100 172.21.15.46:0/2738524787' entity='mgr.x' 2024-03-18T20:28:59.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: from='mgr.14100 172.21.15.46:0/2738524787' entity='mgr.x' 2024-03-18T20:28:59.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:28:59 smithi046 ceph-mon[43495]: from='mgr.14100 172.21.15.46:0/2738524787' entity='mgr.x' 2024-03-18T20:28:59.732 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:28:59.732 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-03-18T20:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-18T20:28:59.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T20:28:59.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:28:59.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-18T20:28:59.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T20:28:59.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:59.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T20:28:59.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:28:59.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T20:28:59.734 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T20:28:59.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T20:28:59.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:59.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T20:28:59.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T20:28:46.137994+0000", 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.076+0000 7ff312a88700 1 Processor -- start 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.077+0000 7ff312a88700 1 -- start start 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.077+0000 7ff312a88700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 0x7ff30c1058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.077+0000 7ff312a88700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c105e90 con 0x7ff30c1054a0 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.077+0000 7ff310824700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 0x7ff30c1058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.077+0000 7ff310824700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 0x7ff30c1058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51746/0 (socket says 172.21.15.46:51746) 2024-03-18T20:28:59.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.077+0000 7ff310824700 1 -- 172.21.15.46:0/1588582204 learned_addr learned my addr 172.21.15.46:0/1588582204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.077+0000 7ff310824700 1 -- 172.21.15.46:0/1588582204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c1066a0 con 0x7ff30c1054a0 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.077+0000 7ff310824700 1 --2- 172.21.15.46:0/1588582204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 0x7ff30c1058c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff3000098a0 tx=0x7ff300009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efda9832b1159e43 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.078+0000 7ff30b7fe700 1 -- 172.21.15.46:0/1588582204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff300004160 con 0x7ff30c1054a0 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.078+0000 7ff30b7fe700 1 -- 172.21.15.46:0/1588582204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3000042c0 con 0x7ff30c1054a0 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.078+0000 7ff30b7fe700 1 -- 172.21.15.46:0/1588582204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff300011a00 con 0x7ff30c1054a0 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.078+0000 7ff312a88700 1 -- 172.21.15.46:0/1588582204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 msgr2=0x7ff30c1058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.078+0000 7ff312a88700 1 --2- 172.21.15.46:0/1588582204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 0x7ff30c1058c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff3000098a0 tx=0x7ff300009bb0 comp rx=0 tx=0).stop 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.078+0000 7ff312a88700 1 -- 172.21.15.46:0/1588582204 shutdown_connections 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.078+0000 7ff312a88700 1 --2- 172.21.15.46:0/1588582204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 0x7ff30c1058c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.078+0000 7ff312a88700 1 -- 172.21.15.46:0/1588582204 >> 172.21.15.46:0/1588582204 conn(0x7ff30c100aa0 msgr2=0x7ff30c102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.078+0000 7ff312a88700 1 -- 172.21.15.46:0/1588582204 shutdown_connections 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.079+0000 7ff312a88700 1 -- 172.21.15.46:0/1588582204 wait complete. 2024-03-18T20:28:59.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.079+0000 7ff312a88700 1 Processor -- start 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.079+0000 7ff312a88700 1 -- start start 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.079+0000 7ff312a88700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 0x7ff30c1904b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.079+0000 7ff312a88700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c1909f0 con 0x7ff30c1054a0 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.079+0000 7ff310824700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 0x7ff30c1904b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.079+0000 7ff310824700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 0x7ff30c1904b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51758/0 (socket says 172.21.15.46:51758) 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.080+0000 7ff310824700 1 -- 172.21.15.46:0/3498064742 learned_addr learned my addr 172.21.15.46:0/3498064742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.080+0000 7ff310824700 1 -- 172.21.15.46:0/3498064742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300009550 con 0x7ff30c1054a0 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.080+0000 7ff310824700 1 --2- 172.21.15.46:0/3498064742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 0x7ff30c1904b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff300009130 tx=0x7ff300011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.080+0000 7ff309ffb700 1 -- 172.21.15.46:0/3498064742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff300003ab0 con 0x7ff30c1054a0 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.080+0000 7ff309ffb700 1 -- 172.21.15.46:0/3498064742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff300003c10 con 0x7ff30c1054a0 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.080+0000 7ff312a88700 1 -- 172.21.15.46:0/3498064742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30c18ec80 con 0x7ff30c1054a0 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.080+0000 7ff309ffb700 1 -- 172.21.15.46:0/3498064742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3000247d0 con 0x7ff30c1054a0 2024-03-18T20:28:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.081+0000 7ff312a88700 1 -- 172.21.15.46:0/3498064742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30c18f120 con 0x7ff30c1054a0 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.081+0000 7ff309ffb700 1 -- 172.21.15.46:0/3498064742 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff300024d00 con 0x7ff30c1054a0 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.081+0000 7ff309ffb700 1 -- 172.21.15.46:0/3498064742 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff3000193c0 con 0x7ff30c1054a0 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.082+0000 7ff312a88700 1 -- 172.21.15.46:0/3498064742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30c18fb30 con 0x7ff30c1054a0 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.084+0000 7ff309ffb700 1 -- 172.21.15.46:0/3498064742 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7ff30001abb0 con 0x7ff30c1054a0 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.160+0000 7ff312a88700 1 -- 172.21.15.46:0/3498064742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff30c194320 con 0x7ff30c1054a0 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.161+0000 7ff309ffb700 1 -- 172.21.15.46:0/3498064742 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7ff30001ad50 con 0x7ff30c1054a0 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.162+0000 7ff312a88700 1 -- 172.21.15.46:0/3498064742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 msgr2=0x7ff30c1904b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.162+0000 7ff312a88700 1 --2- 172.21.15.46:0/3498064742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 0x7ff30c1904b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff300009130 tx=0x7ff300011ad0 comp rx=0 tx=0).stop 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.162+0000 7ff312a88700 1 -- 172.21.15.46:0/3498064742 shutdown_connections 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.162+0000 7ff312a88700 1 --2- 172.21.15.46:0/3498064742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff30c1054a0 0x7ff30c1904b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.162+0000 7ff312a88700 1 -- 172.21.15.46:0/3498064742 >> 172.21.15.46:0/3498064742 conn(0x7ff30c100aa0 msgr2=0x7ff30c101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.162+0000 7ff312a88700 1 -- 172.21.15.46:0/3498064742 shutdown_connections 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:28:58.162+0000 7ff312a88700 1 -- 172.21.15.46:0/3498064742 wait complete. 2024-03-18T20:28:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (3/15)... 2024-03-18T20:29:01.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:00 smithi046 ceph-mon[43495]: mgrmap e3: x(active, since 1.00795s) 2024-03-18T20:29:03.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:02 smithi046 ceph-mon[43495]: mgrmap e4: x(active, since 3s) 2024-03-18T20:29:04.220 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:03 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1747325507' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T20:29:04.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:29:04.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-03-18T20:29:04.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:29:04.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T20:29:04.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T20:29:04.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T20:29:04.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T20:29:04.261 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:29:04.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T20:29:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T20:29:04.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T20:28:46.137994+0000", 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.586+0000 7fae6928b700 1 Processor -- start 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.586+0000 7fae6928b700 1 -- start start 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.586+0000 7fae6928b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 0x7fae64106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:04.265 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.586+0000 7fae6928b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae64106ff0 con 0x7fae64106600 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.586+0000 7fae62ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 0x7fae64106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.586+0000 7fae62ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 0x7fae64106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51826/0 (socket says 172.21.15.46:51826) 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.586+0000 7fae62ffd700 1 -- 172.21.15.46:0/1766820462 learned_addr learned my addr 172.21.15.46:0/1766820462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.587+0000 7fae62ffd700 1 -- 172.21.15.46:0/1766820462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae641077b0 con 0x7fae64106600 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.587+0000 7fae62ffd700 1 --2- 172.21.15.46:0/1766820462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 0x7fae64106a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fae54009e80 tx=0x7fae5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d879015e7f6d28 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.587+0000 7fae61ffb700 1 -- 172.21.15.46:0/1766820462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae5400e070 con 0x7fae64106600 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.587+0000 7fae61ffb700 1 -- 172.21.15.46:0/1766820462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae5400b8c0 con 0x7fae64106600 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.587+0000 7fae61ffb700 1 -- 172.21.15.46:0/1766820462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae540044c0 con 0x7fae64106600 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.588+0000 7fae6928b700 1 -- 172.21.15.46:0/1766820462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 msgr2=0x7fae64106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.588+0000 7fae6928b700 1 --2- 172.21.15.46:0/1766820462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 0x7fae64106a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fae54009e80 tx=0x7fae5400b330 comp rx=0 tx=0).stop 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.588+0000 7fae6928b700 1 -- 172.21.15.46:0/1766820462 shutdown_connections 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.588+0000 7fae6928b700 1 --2- 172.21.15.46:0/1766820462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 0x7fae64106a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.588+0000 7fae6928b700 1 -- 172.21.15.46:0/1766820462 >> 172.21.15.46:0/1766820462 conn(0x7fae64101be0 msgr2=0x7fae64104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.588+0000 7fae6928b700 1 -- 172.21.15.46:0/1766820462 shutdown_connections 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.588+0000 7fae6928b700 1 -- 172.21.15.46:0/1766820462 wait complete. 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.588+0000 7fae6928b700 1 Processor -- start 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.589+0000 7fae6928b700 1 -- start start 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.589+0000 7fae6928b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 0x7fae6407bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.589+0000 7fae6928b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6407c080 con 0x7fae64106600 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.589+0000 7fae62ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 0x7fae6407bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.589+0000 7fae62ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 0x7fae6407bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51836/0 (socket says 172.21.15.46:51836) 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.589+0000 7fae62ffd700 1 -- 172.21.15.46:0/1747325507 learned_addr learned my addr 172.21.15.46:0/1747325507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.589+0000 7fae62ffd700 1 -- 172.21.15.46:0/1747325507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae54009b30 con 0x7fae64106600 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.589+0000 7fae62ffd700 1 --2- 172.21.15.46:0/1747325507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 0x7fae6407bb40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fae54004110 tx=0x7fae54004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.590+0000 7fae5bfff700 1 -- 172.21.15.46:0/1747325507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae5400e040 con 0x7fae64106600 2024-03-18T20:29:04.267 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.590+0000 7fae5bfff700 1 -- 172.21.15.46:0/1747325507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae5400bd70 con 0x7fae64106600 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.590+0000 7fae6928b700 1 -- 172.21.15.46:0/1747325507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6407a2c0 con 0x7fae64106600 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.590+0000 7fae5bfff700 1 -- 172.21.15.46:0/1747325507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae5401c670 con 0x7fae64106600 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.590+0000 7fae6928b700 1 -- 172.21.15.46:0/1747325507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6407a7c0 con 0x7fae64106600 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.590+0000 7fae5bfff700 1 -- 172.21.15.46:0/1747325507 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) v1 ==== 45522+0+0 (secure 0 0 0) 0x7fae5401c8b0 con 0x7fae64106600 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.591+0000 7fae6928b700 1 -- 172.21.15.46:0/1747325507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae640623e0 con 0x7fae64106600 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.591+0000 7fae5bfff700 1 --2- 172.21.15.46:0/1747325507 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fae500386b0 0x7fae5003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.591+0000 7fae5bfff700 1 -- 172.21.15.46:0/1747325507 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fae5404c5d0 con 0x7fae64106600 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.591+0000 7fae627fc700 1 --2- 172.21.15.46:0/1747325507 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fae500386b0 0x7fae5003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.591+0000 7fae627fc700 1 --2- 172.21.15.46:0/1747325507 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fae500386b0 0x7fae5003ab70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fae4c006fd0 tx=0x7fae4c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.596+0000 7fae5bfff700 1 -- 172.21.15.46:0/1747325507 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fae54011980 con 0x7fae64106600 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.892+0000 7fae6928b700 1 -- 172.21.15.46:0/1747325507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fae6410a9b0 con 0x7fae64106600 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.893+0000 7fae5bfff700 1 -- 172.21.15.46:0/1747325507 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fae540204f0 con 0x7fae64106600 2024-03-18T20:29:04.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.895+0000 7fae6928b700 1 -- 172.21.15.46:0/1747325507 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fae500386b0 msgr2=0x7fae5003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:04.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.895+0000 7fae6928b700 1 --2- 172.21.15.46:0/1747325507 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fae500386b0 0x7fae5003ab70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fae4c006fd0 tx=0x7fae4c006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:04.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.895+0000 7fae6928b700 1 -- 172.21.15.46:0/1747325507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 msgr2=0x7fae6407bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:04.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.895+0000 7fae6928b700 1 --2- 172.21.15.46:0/1747325507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 0x7fae6407bb40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fae54004110 tx=0x7fae54004610 comp rx=0 tx=0).stop 2024-03-18T20:29:04.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.895+0000 7fae6928b700 1 -- 172.21.15.46:0/1747325507 shutdown_connections 2024-03-18T20:29:04.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.895+0000 7fae6928b700 1 --2- 172.21.15.46:0/1747325507 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fae500386b0 0x7fae5003ab70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:04.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.895+0000 7fae6928b700 1 --2- 172.21.15.46:0/1747325507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae64106600 0x7fae6407bb40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:04.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.895+0000 7fae6928b700 1 -- 172.21.15.46:0/1747325507 >> 172.21.15.46:0/1747325507 conn(0x7fae64101be0 msgr2=0x7fae64102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:04.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.895+0000 7fae6928b700 1 -- 172.21.15.46:0/1747325507 shutdown_connections 2024-03-18T20:29:04.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:03.895+0000 7fae6928b700 1 -- 172.21.15.46:0/1747325507 wait complete. 2024-03-18T20:29:04.269 INFO:teuthology.orchestra.run.smithi046.stdout:mgr is available 2024-03-18T20:29:05.189 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:29:05.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-03-18T20:29:05.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:29:05.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-18T20:29:05.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-18T20:29:05.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-18T20:29:05.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-18T20:29:05.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:29:05.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-18T20:29:05.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-18T20:29:05.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-03-18T20:29:05.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-03-18T20:29:05.190 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.600+0000 7fc89efc4700 1 Processor -- start 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.600+0000 7fc89efc4700 1 -- start start 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.600+0000 7fc89efc4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 0x7fc8981058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.600+0000 7fc89efc4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc898105e90 con 0x7fc8981054a0 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.601+0000 7fc89cd60700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 0x7fc8981058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.601+0000 7fc89cd60700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 0x7fc8981058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51850/0 (socket says 172.21.15.46:51850) 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.601+0000 7fc89cd60700 1 -- 172.21.15.46:0/179069171 learned_addr learned my addr 172.21.15.46:0/179069171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.601+0000 7fc89cd60700 1 -- 172.21.15.46:0/179069171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8981066a0 con 0x7fc8981054a0 2024-03-18T20:29:05.191 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.601+0000 7fc89cd60700 1 --2- 172.21.15.46:0/179069171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 0x7fc8981058c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc8880098a0 tx=0x7fc888009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49cebf1e21b7e59 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.601+0000 7fc8977fe700 1 -- 172.21.15.46:0/179069171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc888004160 con 0x7fc8981054a0 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.601+0000 7fc8977fe700 1 -- 172.21.15.46:0/179069171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8880042c0 con 0x7fc8981054a0 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.601+0000 7fc8977fe700 1 -- 172.21.15.46:0/179069171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc888011a00 con 0x7fc8981054a0 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.602+0000 7fc89efc4700 1 -- 172.21.15.46:0/179069171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 msgr2=0x7fc8981058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.602+0000 7fc89efc4700 1 --2- 172.21.15.46:0/179069171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 0x7fc8981058c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc8880098a0 tx=0x7fc888009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.602+0000 7fc89efc4700 1 -- 172.21.15.46:0/179069171 shutdown_connections 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.602+0000 7fc89efc4700 1 --2- 172.21.15.46:0/179069171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 0x7fc8981058c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.602+0000 7fc89efc4700 1 -- 172.21.15.46:0/179069171 >> 172.21.15.46:0/179069171 conn(0x7fc898100aa0 msgr2=0x7fc898102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.602+0000 7fc89efc4700 1 -- 172.21.15.46:0/179069171 shutdown_connections 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.602+0000 7fc89efc4700 1 -- 172.21.15.46:0/179069171 wait complete. 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.603+0000 7fc89efc4700 1 Processor -- start 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.603+0000 7fc89efc4700 1 -- start start 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.603+0000 7fc89efc4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 0x7fc8980740d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.603+0000 7fc89efc4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc898074610 con 0x7fc8981054a0 2024-03-18T20:29:05.192 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.603+0000 7fc89cd60700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 0x7fc8980740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.603+0000 7fc89cd60700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 0x7fc8980740d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51854/0 (socket says 172.21.15.46:51854) 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.603+0000 7fc89cd60700 1 -- 172.21.15.46:0/247004788 learned_addr learned my addr 172.21.15.46:0/247004788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.603+0000 7fc89cd60700 1 -- 172.21.15.46:0/247004788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc888009550 con 0x7fc8981054a0 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.604+0000 7fc89cd60700 1 --2- 172.21.15.46:0/247004788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 0x7fc8980740d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc888009130 tx=0x7fc888011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.604+0000 7fc895ffb700 1 -- 172.21.15.46:0/247004788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc888003ab0 con 0x7fc8981054a0 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.604+0000 7fc895ffb700 1 -- 172.21.15.46:0/247004788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc888003c10 con 0x7fc8981054a0 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.604+0000 7fc89efc4700 1 -- 172.21.15.46:0/247004788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89819d050 con 0x7fc8981054a0 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.604+0000 7fc895ffb700 1 -- 172.21.15.46:0/247004788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8880247d0 con 0x7fc8981054a0 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.604+0000 7fc89efc4700 1 -- 172.21.15.46:0/247004788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89819d4f0 con 0x7fc8981054a0 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.605+0000 7fc89efc4700 1 -- 172.21.15.46:0/247004788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8981098d0 con 0x7fc8981054a0 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.606+0000 7fc895ffb700 1 -- 172.21.15.46:0/247004788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) v1 ==== 45522+0+0 (secure 0 0 0) 0x7fc88801abb0 con 0x7fc8981054a0 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.606+0000 7fc895ffb700 1 --2- 172.21.15.46:0/247004788 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fc8800386b0 0x7fc88003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.606+0000 7fc895ffb700 1 -- 172.21.15.46:0/247004788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc88804c710 con 0x7fc8981054a0 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.606+0000 7fc897fff700 1 --2- 172.21.15.46:0/247004788 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fc8800386b0 0x7fc88003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.607+0000 7fc897fff700 1 --2- 172.21.15.46:0/247004788 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fc8800386b0 0x7fc88003ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc88c006fd0 tx=0x7fc88c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.610+0000 7fc895ffb700 1 -- 172.21.15.46:0/247004788 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc88802d3f0 con 0x7fc8981054a0 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.821+0000 7fc89efc4700 1 -- 172.21.15.46:0/247004788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc89819db50 con 0x7fc8981054a0 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.822+0000 7fc895ffb700 1 -- 172.21.15.46:0/247004788 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fc88802c3a0 con 0x7fc8981054a0 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.824+0000 7fc89efc4700 1 -- 172.21.15.46:0/247004788 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fc8800386b0 msgr2=0x7fc88003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.824+0000 7fc89efc4700 1 --2- 172.21.15.46:0/247004788 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fc8800386b0 0x7fc88003ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc88c006fd0 tx=0x7fc88c006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.824+0000 7fc89efc4700 1 -- 172.21.15.46:0/247004788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 msgr2=0x7fc8980740d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.824+0000 7fc89efc4700 1 --2- 172.21.15.46:0/247004788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 0x7fc8980740d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc888009130 tx=0x7fc888011ad0 comp rx=0 tx=0).stop 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.824+0000 7fc89efc4700 1 -- 172.21.15.46:0/247004788 shutdown_connections 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.825+0000 7fc89efc4700 1 --2- 172.21.15.46:0/247004788 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fc8800386b0 0x7fc88003ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.825+0000 7fc89efc4700 1 --2- 172.21.15.46:0/247004788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8981054a0 0x7fc8980740d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.825+0000 7fc89efc4700 1 -- 172.21.15.46:0/247004788 >> 172.21.15.46:0/247004788 conn(0x7fc898100aa0 msgr2=0x7fc898101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:05.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.825+0000 7fc89efc4700 1 -- 172.21.15.46:0/247004788 shutdown_connections 2024-03-18T20:29:05.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:04.825+0000 7fc89efc4700 1 -- 172.21.15.46:0/247004788 wait complete. 2024-03-18T20:29:05.195 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling cephadm module... 2024-03-18T20:29:06.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:05 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/247004788' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-18T20:29:07.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:06 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2333754702' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-18T20:29:07.036 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:06 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: ignoring --setuser ceph since I am not root 2024-03-18T20:29:07.036 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:06 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: ignoring --setgroup ceph since I am not root 2024-03-18T20:29:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.543+0000 7f91fe0c0700 1 Processor -- start 2024-03-18T20:29:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.543+0000 7f91fe0c0700 1 -- start start 2024-03-18T20:29:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.543+0000 7f91fe0c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 0x7f91f81058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.543+0000 7f91fe0c0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f8105ec0 con 0x7f91f81054d0 2024-03-18T20:29:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.543+0000 7f91f77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 0x7f91f81058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:07.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.544+0000 7f91f77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 0x7f91f81058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42690/0 (socket says 172.21.15.46:42690) 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.544+0000 7f91f77fe700 1 -- 172.21.15.46:0/1214109586 learned_addr learned my addr 172.21.15.46:0/1214109586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.544+0000 7f91f77fe700 1 -- 172.21.15.46:0/1214109586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f81066d0 con 0x7f91f81054d0 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.544+0000 7f91f77fe700 1 --2- 172.21.15.46:0/1214109586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 0x7f91f81058f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f91e80098a0 tx=0x7f91e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1f08d5207e9243 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.544+0000 7f91f67fc700 1 -- 172.21.15.46:0/1214109586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91e8004160 con 0x7f91f81054d0 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.544+0000 7f91f67fc700 1 -- 172.21.15.46:0/1214109586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f91e80042c0 con 0x7f91f81054d0 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.544+0000 7f91f67fc700 1 -- 172.21.15.46:0/1214109586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91e8011a00 con 0x7f91f81054d0 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.545+0000 7f91fe0c0700 1 -- 172.21.15.46:0/1214109586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 msgr2=0x7f91f81058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.545+0000 7f91fe0c0700 1 --2- 172.21.15.46:0/1214109586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 0x7f91f81058f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f91e80098a0 tx=0x7f91e8009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.545+0000 7f91fe0c0700 1 -- 172.21.15.46:0/1214109586 shutdown_connections 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.545+0000 7f91fe0c0700 1 --2- 172.21.15.46:0/1214109586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 0x7f91f81058f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.545+0000 7f91fe0c0700 1 -- 172.21.15.46:0/1214109586 >> 172.21.15.46:0/1214109586 conn(0x7f91f8100a70 msgr2=0x7f91f8102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.545+0000 7f91fe0c0700 1 -- 172.21.15.46:0/1214109586 shutdown_connections 2024-03-18T20:29:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.545+0000 7f91fe0c0700 1 -- 172.21.15.46:0/1214109586 wait complete. 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.546+0000 7f91fe0c0700 1 Processor -- start 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.546+0000 7f91fe0c0700 1 -- start start 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.546+0000 7f91fe0c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 0x7f91f8190530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.546+0000 7f91fe0c0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f8190a70 con 0x7f91f81054d0 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.546+0000 7f91f77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 0x7f91f8190530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.546+0000 7f91f77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 0x7f91f8190530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42696/0 (socket says 172.21.15.46:42696) 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.546+0000 7f91f77fe700 1 -- 172.21.15.46:0/2333754702 learned_addr learned my addr 172.21.15.46:0/2333754702 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.547+0000 7f91f77fe700 1 -- 172.21.15.46:0/2333754702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e8009550 con 0x7f91f81054d0 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.547+0000 7f91f77fe700 1 --2- 172.21.15.46:0/2333754702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 0x7f91f8190530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009fb0 tx=0x7f91e8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.547+0000 7f91f4ff9700 1 -- 172.21.15.46:0/2333754702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91e8003a40 con 0x7f91f81054d0 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.547+0000 7f91f4ff9700 1 -- 172.21.15.46:0/2333754702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f91e8003ba0 con 0x7f91f81054d0 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.547+0000 7f91f4ff9700 1 -- 172.21.15.46:0/2333754702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91e801a750 con 0x7f91f81054d0 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.547+0000 7f91fe0c0700 1 -- 172.21.15.46:0/2333754702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f818ecb0 con 0x7f91f81054d0 2024-03-18T20:29:07.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.547+0000 7f91fe0c0700 1 -- 172.21.15.46:0/2333754702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f818f150 con 0x7f91f81054d0 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.548+0000 7f91f4ff9700 1 -- 172.21.15.46:0/2333754702 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) v1 ==== 45522+0+0 (secure 0 0 0) 0x7f91e801a980 con 0x7f91f81054d0 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.548+0000 7f91f4ff9700 1 --2- 172.21.15.46:0/2333754702 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f91e00386b0 0x7f91e003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.548+0000 7f91f4ff9700 1 -- 172.21.15.46:0/2333754702 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f91e804c590 con 0x7f91f81054d0 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.548+0000 7f91de7fc700 1 -- 172.21.15.46:0/2333754702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f818fb60 con 0x7f91f81054d0 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.548+0000 7f91f6ffd700 1 --2- 172.21.15.46:0/2333754702 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f91e00386b0 0x7f91e003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.549+0000 7f91f6ffd700 1 --2- 172.21.15.46:0/2333754702 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f91e00386b0 0x7f91e003ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f91ec006fd0 tx=0x7f91ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.553+0000 7f91f4ff9700 1 -- 172.21.15.46:0/2333754702 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f91e801ac30 con 0x7f91f81054d0 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:05.826+0000 7f91de7fc700 1 -- 172.21.15.46:0/2333754702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f91f80623e0 con 0x7f91f81054d0 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:06.695+0000 7f91f4ff9700 1 -- 172.21.15.46:0/2333754702 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 5) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f91e802d960 con 0x7f91f81054d0 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:06.697+0000 7f91f4ff9700 1 -- 172.21.15.46:0/2333754702 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f91e801ae60 con 0x7f91f81054d0 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:06.700+0000 7f91de7fc700 1 -- 172.21.15.46:0/2333754702 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f91e00386b0 msgr2=0x7f91e003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:06.700+0000 7f91de7fc700 1 --2- 172.21.15.46:0/2333754702 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f91e00386b0 0x7f91e003ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f91ec006fd0 tx=0x7f91ec006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:06.700+0000 7f91de7fc700 1 -- 172.21.15.46:0/2333754702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 msgr2=0x7f91f8190530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:07.066 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:06.700+0000 7f91de7fc700 1 --2- 172.21.15.46:0/2333754702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 0x7f91f8190530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009fb0 tx=0x7f91e8011ad0 comp rx=0 tx=0).stop 2024-03-18T20:29:07.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:06.700+0000 7f91de7fc700 1 -- 172.21.15.46:0/2333754702 shutdown_connections 2024-03-18T20:29:07.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:06.700+0000 7f91de7fc700 1 --2- 172.21.15.46:0/2333754702 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f91e00386b0 0x7f91e003ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:07.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:06.700+0000 7f91de7fc700 1 --2- 172.21.15.46:0/2333754702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91f81054d0 0x7f91f8190530 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:07.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:06.700+0000 7f91de7fc700 1 -- 172.21.15.46:0/2333754702 >> 172.21.15.46:0/2333754702 conn(0x7f91f8100a70 msgr2=0x7f91f8101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:07.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:06.700+0000 7f91de7fc700 1 -- 172.21.15.46:0/2333754702 shutdown_connections 2024-03-18T20:29:07.067 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:06.700+0000 7f91de7fc700 1 -- 172.21.15.46:0/2333754702 wait complete. 2024-03-18T20:29:07.290 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:07.178+0000 7f24ff5d2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T20:29:07.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:07.284+0000 7f24ff5d2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T20:29:07.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:07.386+0000 7f24ff5d2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T20:29:07.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:07.484+0000 7f24ff5d2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T20:29:07.990 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:07 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2333754702' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-18T20:29:07.990 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:07 smithi046 ceph-mon[43495]: mgrmap e5: x(active, since 7s) 2024-03-18T20:29:07.990 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:07 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1045389368' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-18T20:29:08.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-03-18T20:29:08.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-18T20:29:08.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T20:29:08.047 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.404+0000 7f74f0fdd700 1 Processor -- start 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.404+0000 7f74f0fdd700 1 -- start start 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.405+0000 7f74f0fdd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 0x7f74ec109480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.405+0000 7f74f0fdd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec0744a0 con 0x7f74ec107090 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.405+0000 7f74ea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 0x7f74ec109480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.405+0000 7f74ea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 0x7f74ec109480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42730/0 (socket says 172.21.15.46:42730) 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.405+0000 7f74ea59c700 1 -- 172.21.15.46:0/3898283024 learned_addr learned my addr 172.21.15.46:0/3898283024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.405+0000 7f74ea59c700 1 -- 172.21.15.46:0/3898283024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec1099c0 con 0x7f74ec107090 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.405+0000 7f74ea59c700 1 --2- 172.21.15.46:0/3898283024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 0x7f74ec109480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0098a0 tx=0x7f74dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b85eaefdc25a60 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.405+0000 7f74e959a700 1 -- 172.21.15.46:0/3898283024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74dc004160 con 0x7f74ec107090 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.406+0000 7f74e959a700 1 -- 172.21.15.46:0/3898283024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f74dc0042c0 con 0x7f74ec107090 2024-03-18T20:29:08.048 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.406+0000 7f74e959a700 1 -- 172.21.15.46:0/3898283024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74dc011a00 con 0x7f74ec107090 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.406+0000 7f74f0fdd700 1 -- 172.21.15.46:0/3898283024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 msgr2=0x7f74ec109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.406+0000 7f74f0fdd700 1 --2- 172.21.15.46:0/3898283024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 0x7f74ec109480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0098a0 tx=0x7f74dc009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.406+0000 7f74f0fdd700 1 -- 172.21.15.46:0/3898283024 shutdown_connections 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.406+0000 7f74f0fdd700 1 --2- 172.21.15.46:0/3898283024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 0x7f74ec109480 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.406+0000 7f74f0fdd700 1 -- 172.21.15.46:0/3898283024 >> 172.21.15.46:0/3898283024 conn(0x7f74ec100aa0 msgr2=0x7f74ec102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.407+0000 7f74f0fdd700 1 -- 172.21.15.46:0/3898283024 shutdown_connections 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.407+0000 7f74f0fdd700 1 -- 172.21.15.46:0/3898283024 wait complete. 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.407+0000 7f74f0fdd700 1 Processor -- start 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.407+0000 7f74f0fdd700 1 -- start start 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.408+0000 7f74f0fdd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 0x7f74ec19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.408+0000 7f74f0fdd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec19ca80 con 0x7f74ec107090 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.408+0000 7f74ea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 0x7f74ec19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.408+0000 7f74ea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 0x7f74ec19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42738/0 (socket says 172.21.15.46:42738) 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.408+0000 7f74ea59c700 1 -- 172.21.15.46:0/1045389368 learned_addr learned my addr 172.21.15.46:0/1045389368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.408+0000 7f74ea59c700 1 -- 172.21.15.46:0/1045389368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc009550 con 0x7f74ec107090 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.408+0000 7f74ea59c700 1 --2- 172.21.15.46:0/1045389368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 0x7f74ec19c540 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009fb0 tx=0x7f74dc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.408+0000 7f74db7fe700 1 -- 172.21.15.46:0/1045389368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74dc003a40 con 0x7f74ec107090 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.409+0000 7f74db7fe700 1 -- 172.21.15.46:0/1045389368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f74dc003ba0 con 0x7f74ec107090 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.409+0000 7f74f0fdd700 1 -- 172.21.15.46:0/1045389368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec19cc80 con 0x7f74ec107090 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.409+0000 7f74db7fe700 1 -- 172.21.15.46:0/1045389368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74dc01a750 con 0x7f74ec107090 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.409+0000 7f74f0fdd700 1 -- 172.21.15.46:0/1045389368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec19d120 con 0x7f74ec107090 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.409+0000 7f74db7fe700 1 -- 172.21.15.46:0/1045389368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f74dc01a9d0 con 0x7f74ec107090 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.410+0000 7f74db7fe700 1 --2- 172.21.15.46:0/1045389368 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f74d40386b0 0x7f74d403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.410+0000 7f74f0fdd700 1 -- 172.21.15.46:0/1045389368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ec10a710 con 0x7f74ec107090 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.410+0000 7f74e9d9b700 1 -- 172.21.15.46:0/1045389368 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f74d40386b0 msgr2=0x7f74d403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1385957034 2024-03-18T20:29:08.050 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.410+0000 7f74db7fe700 1 -- 172.21.15.46:0/1045389368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f74dc04c3f0 con 0x7f74ec107090 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.410+0000 7f74e9d9b700 1 --2- 172.21.15.46:0/1045389368 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f74d40386b0 0x7f74d403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.415+0000 7f74db7fe700 1 -- 172.21.15.46:0/1045389368 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f74dc018db0 con 0x7f74ec107090 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.611+0000 7f74e9d9b700 1 -- 172.21.15.46:0/1045389368 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f74d40386b0 msgr2=0x7f74d403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1385957034 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.611+0000 7f74e9d9b700 1 --2- 172.21.15.46:0/1045389368 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f74d40386b0 0x7f74d403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.687+0000 7f74f0fdd700 1 -- 172.21.15.46:0/1045389368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f74ec19fdd0 con 0x7f74ec107090 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.687+0000 7f74db7fe700 1 -- 172.21.15.46:0/1045389368 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f74dc018350 con 0x7f74ec107090 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.689+0000 7f74f0fdd700 1 -- 172.21.15.46:0/1045389368 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f74d40386b0 msgr2=0x7f74d403ab70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.689+0000 7f74f0fdd700 1 --2- 172.21.15.46:0/1045389368 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f74d40386b0 0x7f74d403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.689+0000 7f74f0fdd700 1 -- 172.21.15.46:0/1045389368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 msgr2=0x7f74ec19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.689+0000 7f74f0fdd700 1 --2- 172.21.15.46:0/1045389368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 0x7f74ec19c540 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009fb0 tx=0x7f74dc011ad0 comp rx=0 tx=0).stop 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.690+0000 7f74f0fdd700 1 -- 172.21.15.46:0/1045389368 shutdown_connections 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.690+0000 7f74f0fdd700 1 --2- 172.21.15.46:0/1045389368 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7f74d40386b0 0x7f74d403ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.690+0000 7f74f0fdd700 1 --2- 172.21.15.46:0/1045389368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74ec107090 0x7f74ec19c540 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:08.051 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.690+0000 7f74f0fdd700 1 -- 172.21.15.46:0/1045389368 >> 172.21.15.46:0/1045389368 conn(0x7f74ec100aa0 msgr2=0x7f74ec1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:08.052 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.690+0000 7f74f0fdd700 1 -- 172.21.15.46:0/1045389368 shutdown_connections 2024-03-18T20:29:08.052 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:07.690+0000 7f74f0fdd700 1 -- 172.21.15.46:0/1045389368 wait complete. 2024-03-18T20:29:08.052 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-03-18T20:29:08.052 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 5... 2024-03-18T20:29:08.297 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:08 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:08.233+0000 7f24ff5d2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:29:08.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:08 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:08.524+0000 7f24ff5d2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T20:29:08.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:08 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:08.620+0000 7f24ff5d2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T20:29:08.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:08 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:08.742+0000 7f24ff5d2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T20:29:09.297 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:08 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:08.924+0000 7f24ff5d2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T20:29:09.297 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:09 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:09.069+0000 7f24ff5d2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T20:29:09.297 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:09 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:09.166+0000 7f24ff5d2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T20:29:10.032 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:09 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:09.587+0000 7f24ff5d2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T20:29:10.297 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:10 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:10.028+0000 7f24ff5d2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T20:29:10.297 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:10 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:10.247+0000 7f24ff5d2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T20:29:10.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:10 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:10.352+0000 7f24ff5d2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T20:29:11.501 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:11 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:11.221+0000 7f24ff5d2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T20:29:11.501 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:11 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:11.329+0000 7f24ff5d2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T20:29:11.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:11 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:11.497+0000 7f24ff5d2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T20:29:12.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:12 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:12.213+0000 7f24ff5d2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T20:29:12.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:12 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:12.419+0000 7f24ff5d2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T20:29:13.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:12 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:12.638+0000 7f24ff5d2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T20:29:14.019 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:13 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:13.693+0000 7f24ff5d2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T20:29:14.297 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:14.015+0000 7f24ff5d2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:29:14.297 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:14.115+0000 7f24ff5d2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T20:29:14.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:14.331+0000 7f24ff5d2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T20:29:14.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: Active manager daemon x restarted 2024-03-18T20:29:14.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: Activating manager daemon x 2024-03-18T20:29:14.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: osdmap e2: 0 total, 0 up, 0 in 2024-03-18T20:29:14.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: mgrmap e6: x(active, starting, since 0.0034971s) 2024-03-18T20:29:14.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:29:14.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T20:29:14.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T20:29:14.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T20:29:14.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T20:29:14.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: Manager daemon x is now available 2024-03-18T20:29:14.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:14.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:14.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:29:14.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:14 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:29:15.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:15 smithi046 ceph-mon[43495]: Found migration_current of "None". Setting to last migration. 2024-03-18T20:29:15.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:15 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:29:15.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:15 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T20:29:15.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:15 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T20:29:15.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:15 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:15.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:15 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:15.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:15 smithi046 ceph-mon[43495]: mgrmap e7: x(active, since 1.00559s) 2024-03-18T20:29:15.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-03-18T20:29:15.723 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-18T20:29:15.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-18T20:29:15.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-03-18T20:29:15.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.424+0000 7fbe81865700 1 Processor -- start 2024-03-18T20:29:15.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.424+0000 7fbe81865700 1 -- start start 2024-03-18T20:29:15.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.424+0000 7fbe81865700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 0x7fbe7c106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:15.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.424+0000 7fbe81865700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c106ff0 con 0x7fbe7c106600 2024-03-18T20:29:15.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.424+0000 7fbe7affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 0x7fbe7c106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:15.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.424+0000 7fbe7affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 0x7fbe7c106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42754/0 (socket says 172.21.15.46:42754) 2024-03-18T20:29:15.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.424+0000 7fbe7affd700 1 -- 172.21.15.46:0/1559846923 learned_addr learned my addr 172.21.15.46:0/1559846923 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:15.724 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.425+0000 7fbe7affd700 1 -- 172.21.15.46:0/1559846923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7c1077b0 con 0x7fbe7c106600 2024-03-18T20:29:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.425+0000 7fbe7affd700 1 --2- 172.21.15.46:0/1559846923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 0x7fbe7c106a20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbe640098a0 tx=0x7fbe64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245c600cc7dad1b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.425+0000 7fbe79ffb700 1 -- 172.21.15.46:0/1559846923 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe64004160 con 0x7fbe7c106600 2024-03-18T20:29:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.425+0000 7fbe79ffb700 1 -- 172.21.15.46:0/1559846923 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe640042c0 con 0x7fbe7c106600 2024-03-18T20:29:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.425+0000 7fbe79ffb700 1 -- 172.21.15.46:0/1559846923 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe64011a00 con 0x7fbe7c106600 2024-03-18T20:29:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.426+0000 7fbe81865700 1 -- 172.21.15.46:0/1559846923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 msgr2=0x7fbe7c106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.426+0000 7fbe81865700 1 --2- 172.21.15.46:0/1559846923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 0x7fbe7c106a20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbe640098a0 tx=0x7fbe64009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.426+0000 7fbe81865700 1 -- 172.21.15.46:0/1559846923 shutdown_connections 2024-03-18T20:29:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.426+0000 7fbe81865700 1 --2- 172.21.15.46:0/1559846923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 0x7fbe7c106a20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.426+0000 7fbe81865700 1 -- 172.21.15.46:0/1559846923 >> 172.21.15.46:0/1559846923 conn(0x7fbe7c101be0 msgr2=0x7fbe7c104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.426+0000 7fbe81865700 1 -- 172.21.15.46:0/1559846923 shutdown_connections 2024-03-18T20:29:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.426+0000 7fbe81865700 1 -- 172.21.15.46:0/1559846923 wait complete. 2024-03-18T20:29:15.725 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.427+0000 7fbe81865700 1 Processor -- start 2024-03-18T20:29:15.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.427+0000 7fbe81865700 1 -- start start 2024-03-18T20:29:15.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.427+0000 7fbe81865700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 0x7fbe7c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:15.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.427+0000 7fbe81865700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c19a890 con 0x7fbe7c106600 2024-03-18T20:29:15.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.428+0000 7fbe7affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 0x7fbe7c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:15.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.428+0000 7fbe7affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 0x7fbe7c19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42766/0 (socket says 172.21.15.46:42766) 2024-03-18T20:29:15.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.428+0000 7fbe7affd700 1 -- 172.21.15.46:0/2828527049 learned_addr learned my addr 172.21.15.46:0/2828527049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:15.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.428+0000 7fbe7affd700 1 -- 172.21.15.46:0/2828527049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe64009550 con 0x7fbe7c106600 2024-03-18T20:29:15.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.428+0000 7fbe7affd700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 0x7fbe7c19a350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbe64011710 tx=0x7fbe64011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:15.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.428+0000 7fbe80863700 1 -- 172.21.15.46:0/2828527049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe64003aa0 con 0x7fbe7c106600 2024-03-18T20:29:15.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.428+0000 7fbe80863700 1 -- 172.21.15.46:0/2828527049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe64003c00 con 0x7fbe7c106600 2024-03-18T20:29:15.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.428+0000 7fbe81865700 1 -- 172.21.15.46:0/2828527049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7c19aa90 con 0x7fbe7c106600 2024-03-18T20:29:15.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.428+0000 7fbe80863700 1 -- 172.21.15.46:0/2828527049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe64024800 con 0x7fbe7c106600 2024-03-18T20:29:15.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.428+0000 7fbe81865700 1 -- 172.21.15.46:0/2828527049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c19af30 con 0x7fbe7c106600 2024-03-18T20:29:15.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.429+0000 7fbe80863700 1 -- 172.21.15.46:0/2828527049 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fbe64024960 con 0x7fbe7c106600 2024-03-18T20:29:15.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.429+0000 7fbe80863700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 0x7fbe6803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:15.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.429+0000 7fbe81865700 1 -- 172.21.15.46:0/2828527049 --> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe7c0623e0 con 0x7fbe68038750 2024-03-18T20:29:15.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.429+0000 7fbe80863700 1 -- 172.21.15.46:0/2828527049 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbe6404c370 con 0x7fbe7c106600 2024-03-18T20:29:15.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.429+0000 7fbe7a7fc700 1 -- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 msgr2=0x7fbe6803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1385957034 2024-03-18T20:29:15.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.429+0000 7fbe7a7fc700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 0x7fbe6803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T20:29:15.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.630+0000 7fbe7a7fc700 1 -- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 msgr2=0x7fbe6803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1385957034 2024-03-18T20:29:15.727 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:08.630+0000 7fbe7a7fc700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 0x7fbe6803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T20:29:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:09.031+0000 7fbe7a7fc700 1 -- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 msgr2=0x7fbe6803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1385957034 2024-03-18T20:29:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:09.031+0000 7fbe7a7fc700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 0x7fbe6803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T20:29:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:09.832+0000 7fbe7a7fc700 1 -- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 msgr2=0x7fbe6803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1385957034 2024-03-18T20:29:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:09.832+0000 7fbe7a7fc700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 0x7fbe6803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T20:29:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:11.433+0000 7fbe7a7fc700 1 -- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 msgr2=0x7fbe6803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/1385957034 2024-03-18T20:29:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:11.433+0000 7fbe7a7fc700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 0x7fbe6803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T20:29:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:14.347+0000 7fbe80863700 1 -- 172.21.15.46:0/2828527049 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 6) v1 ==== 45300+0+0 (secure 0 0 0) 0x7fbe6402d9b0 con 0x7fbe7c106600 2024-03-18T20:29:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:14.347+0000 7fbe80863700 1 -- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 msgr2=0x7fbe6803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:29:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:14.347+0000 7fbe80863700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 0x7fbe6803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.349+0000 7fbe80863700 1 -- 172.21.15.46:0/2828527049 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 7) v1 ==== 45427+0+0 (secure 0 0 0) 0x7fbe6404df70 con 0x7fbe7c106600 2024-03-18T20:29:15.728 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.349+0000 7fbe80863700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fbe6803be80 0x7fbe6803e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.349+0000 7fbe80863700 1 -- 172.21.15.46:0/2828527049 --> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe7c0623e0 con 0x7fbe6803be80 2024-03-18T20:29:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.349+0000 7fbe7a7fc700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fbe6803be80 0x7fbe6803e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.350+0000 7fbe7a7fc700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fbe6803be80 0x7fbe6803e270 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c003a10 tx=0x7fbe6c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.351+0000 7fbe80863700 1 -- 172.21.15.46:0/2828527049 <== mgr.14120 v2:172.21.15.46:6800/2753187643 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fbe7c0623e0 con 0x7fbe6803be80 2024-03-18T20:29:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.357+0000 7fbe81865700 1 -- 172.21.15.46:0/2828527049 --> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fbe7c10a9e0 con 0x7fbe6803be80 2024-03-18T20:29:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.357+0000 7fbe80863700 1 -- 172.21.15.46:0/2828527049 <== mgr.14120 v2:172.21.15.46:6800/2753187643 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fbe7c10a9e0 con 0x7fbe6803be80 2024-03-18T20:29:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.358+0000 7fbe81865700 1 -- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fbe6803be80 msgr2=0x7fbe6803e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.358+0000 7fbe81865700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fbe6803be80 0x7fbe6803e270 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c003a10 tx=0x7fbe6c0092b0 comp rx=0 tx=0).stop 2024-03-18T20:29:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.358+0000 7fbe81865700 1 -- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 msgr2=0x7fbe7c19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.358+0000 7fbe81865700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 0x7fbe7c19a350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbe64011710 tx=0x7fbe64011ad0 comp rx=0 tx=0).stop 2024-03-18T20:29:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.358+0000 7fbe81865700 1 -- 172.21.15.46:0/2828527049 shutdown_connections 2024-03-18T20:29:15.729 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.358+0000 7fbe81865700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fbe6803be80 0x7fbe6803e270 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.358+0000 7fbe81865700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe7c106600 0x7fbe7c19a350 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.358+0000 7fbe81865700 1 --2- 172.21.15.46:0/2828527049 >> [v2:172.21.15.46:6800/1385957034,v1:172.21.15.46:6801/1385957034] conn(0x7fbe68038750 0x7fbe6803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.358+0000 7fbe81865700 1 -- 172.21.15.46:0/2828527049 >> 172.21.15.46:0/2828527049 conn(0x7fbe7c101be0 msgr2=0x7fbe7c1027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.358+0000 7fbe81865700 1 -- 172.21.15.46:0/2828527049 shutdown_connections 2024-03-18T20:29:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:15.358+0000 7fbe81865700 1 -- 172.21.15.46:0/2828527049 wait complete. 2024-03-18T20:29:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 5 is available 2024-03-18T20:29:15.730 INFO:teuthology.orchestra.run.smithi046.stdout:Setting orchestrator backend to cephadm... 2024-03-18T20:29:16.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:16 smithi046 ceph-mon[43495]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-18T20:29:16.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:16 smithi046 ceph-mon[43495]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-18T20:29:16.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:16 smithi046 ceph-mon[43495]: [18/Mar/2024:20:29:15] ENGINE Bus STARTING 2024-03-18T20:29:16.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:16 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:29:16.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:16 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:16.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:16 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:29:16.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.059+0000 7f092ddf5700 1 Processor -- start 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.059+0000 7f092ddf5700 1 -- start start 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.060+0000 7f092ddf5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 0x7f09281058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.060+0000 7f092ddf5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0928105ec0 con 0x7f09281054d0 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.060+0000 7f09277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 0x7f09281058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.060+0000 7f09277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 0x7f09281058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38142/0 (socket says 172.21.15.46:38142) 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.060+0000 7f09277fe700 1 -- 172.21.15.46:0/36431401 learned_addr learned my addr 172.21.15.46:0/36431401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.060+0000 7f09277fe700 1 -- 172.21.15.46:0/36431401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09281066d0 con 0x7f09281054d0 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.060+0000 7f09277fe700 1 --2- 172.21.15.46:0/36431401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 0x7f09281058f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09100098a0 tx=0x7f0910009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a1b868441f3a12 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.060+0000 7f09267fc700 1 -- 172.21.15.46:0/36431401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0910004160 con 0x7f09281054d0 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.061+0000 7f09267fc700 1 -- 172.21.15.46:0/36431401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f09100042c0 con 0x7f09281054d0 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.061+0000 7f09267fc700 1 -- 172.21.15.46:0/36431401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0910011a00 con 0x7f09281054d0 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.061+0000 7f092ddf5700 1 -- 172.21.15.46:0/36431401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 msgr2=0x7f09281058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:16.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.061+0000 7f092ddf5700 1 --2- 172.21.15.46:0/36431401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 0x7f09281058f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09100098a0 tx=0x7f0910009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.061+0000 7f092ddf5700 1 -- 172.21.15.46:0/36431401 shutdown_connections 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.061+0000 7f092ddf5700 1 --2- 172.21.15.46:0/36431401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 0x7f09281058f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.061+0000 7f092ddf5700 1 -- 172.21.15.46:0/36431401 >> 172.21.15.46:0/36431401 conn(0x7f0928100a70 msgr2=0x7f0928102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.061+0000 7f092ddf5700 1 -- 172.21.15.46:0/36431401 shutdown_connections 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.061+0000 7f092ddf5700 1 -- 172.21.15.46:0/36431401 wait complete. 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.062+0000 7f092ddf5700 1 Processor -- start 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.062+0000 7f092ddf5700 1 -- start start 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.062+0000 7f092ddf5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 0x7f092819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.062+0000 7f092ddf5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092819a890 con 0x7f09281054d0 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.063+0000 7f09277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 0x7f092819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.063+0000 7f09277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 0x7f092819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38152/0 (socket says 172.21.15.46:38152) 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.063+0000 7f09277fe700 1 -- 172.21.15.46:0/2006970005 learned_addr learned my addr 172.21.15.46:0/2006970005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.063+0000 7f09277fe700 1 -- 172.21.15.46:0/2006970005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0910009550 con 0x7f09281054d0 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.063+0000 7f09277fe700 1 --2- 172.21.15.46:0/2006970005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 0x7f092819a350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0910009130 tx=0x7f0910011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.063+0000 7f0924ff9700 1 -- 172.21.15.46:0/2006970005 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0910003ab0 con 0x7f09281054d0 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.063+0000 7f0924ff9700 1 -- 172.21.15.46:0/2006970005 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0910003c10 con 0x7f09281054d0 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.063+0000 7f092ddf5700 1 -- 172.21.15.46:0/2006970005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092819aa90 con 0x7f09281054d0 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.063+0000 7f0924ff9700 1 -- 172.21.15.46:0/2006970005 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09100247d0 con 0x7f09281054d0 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.063+0000 7f092ddf5700 1 -- 172.21.15.46:0/2006970005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092819af30 con 0x7f09281054d0 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.064+0000 7f092ddf5700 1 -- 172.21.15.46:0/2006970005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09281944b0 con 0x7f09281054d0 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.065+0000 7f0924ff9700 1 -- 172.21.15.46:0/2006970005 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 7) v1 ==== 45427+0+0 (secure 0 0 0) 0x7f091001abb0 con 0x7f09281054d0 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.065+0000 7f0924ff9700 1 --2- 172.21.15.46:0/2006970005 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f0914038620 0x7f091403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.065+0000 7f0924ff9700 1 -- 172.21.15.46:0/2006970005 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f091004ca20 con 0x7f09281054d0 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.065+0000 7f0926ffd700 1 --2- 172.21.15.46:0/2006970005 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f0914038620 0x7f091403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.065+0000 7f0926ffd700 1 --2- 172.21.15.46:0/2006970005 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f0914038620 0x7f091403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0918006fd0 tx=0x7f0918006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.069+0000 7f0924ff9700 1 -- 172.21.15.46:0/2006970005 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0910018760 con 0x7f09281054d0 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.297+0000 7f092ddf5700 1 -- 172.21.15.46:0/2006970005 --> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f09280611f0 con 0x7f0914038620 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.306+0000 7f0924ff9700 1 -- 172.21.15.46:0/2006970005 <== mgr.14120 v2:172.21.15.46:6800/2753187643 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f09280611f0 con 0x7f0914038620 2024-03-18T20:29:16.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.308+0000 7f092ddf5700 1 -- 172.21.15.46:0/2006970005 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f0914038620 msgr2=0x7f091403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:16.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.308+0000 7f092ddf5700 1 --2- 172.21.15.46:0/2006970005 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f0914038620 0x7f091403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0918006fd0 tx=0x7f0918006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:16.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.308+0000 7f092ddf5700 1 -- 172.21.15.46:0/2006970005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 msgr2=0x7f092819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:16.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.308+0000 7f092ddf5700 1 --2- 172.21.15.46:0/2006970005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 0x7f092819a350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0910009130 tx=0x7f0910011ad0 comp rx=0 tx=0).stop 2024-03-18T20:29:16.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.309+0000 7f092ddf5700 1 -- 172.21.15.46:0/2006970005 shutdown_connections 2024-03-18T20:29:16.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.309+0000 7f092ddf5700 1 --2- 172.21.15.46:0/2006970005 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f0914038620 0x7f091403aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:16.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.309+0000 7f092ddf5700 1 --2- 172.21.15.46:0/2006970005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09281054d0 0x7f092819a350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:16.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.309+0000 7f092ddf5700 1 -- 172.21.15.46:0/2006970005 >> 172.21.15.46:0/2006970005 conn(0x7f0928100a70 msgr2=0x7f0928101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:16.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.309+0000 7f092ddf5700 1 -- 172.21.15.46:0/2006970005 shutdown_connections 2024-03-18T20:29:16.682 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:16.309+0000 7f092ddf5700 1 -- 172.21.15.46:0/2006970005 wait complete. 2024-03-18T20:29:17.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:17 smithi046 ceph-mon[43495]: [18/Mar/2024:20:29:15] ENGINE Serving on https://172.21.15.46:7150 2024-03-18T20:29:17.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:17 smithi046 ceph-mon[43495]: [18/Mar/2024:20:29:15] ENGINE Serving on http://172.21.15.46:8765 2024-03-18T20:29:17.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:17 smithi046 ceph-mon[43495]: [18/Mar/2024:20:29:15] ENGINE Bus STARTED 2024-03-18T20:29:17.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:17 smithi046 ceph-mon[43495]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:29:17.585 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:17 smithi046 ceph-mon[43495]: mgrmap e8: x(active, since 2s) 2024-03-18T20:29:17.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-18T20:29:17.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.052+0000 7f9905282700 1 Processor -- start 2024-03-18T20:29:17.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.052+0000 7f9905282700 1 -- start start 2024-03-18T20:29:17.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.053+0000 7f9905282700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 0x7f9900108080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:17.637 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.053+0000 7f9905282700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9900108650 con 0x7f9900107c60 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.053+0000 7f98feffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 0x7f9900108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.053+0000 7f98feffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 0x7f9900108080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38156/0 (socket says 172.21.15.46:38156) 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.053+0000 7f98feffd700 1 -- 172.21.15.46:0/3211499903 learned_addr learned my addr 172.21.15.46:0/3211499903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.053+0000 7f98feffd700 1 -- 172.21.15.46:0/3211499903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99001061f0 con 0x7f9900107c60 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.053+0000 7f98feffd700 1 --2- 172.21.15.46:0/3211499903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 0x7f9900108080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f98e80098a0 tx=0x7f98e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7c4be334c4c2ce server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.053+0000 7f98fdffb700 1 -- 172.21.15.46:0/3211499903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98e8004160 con 0x7f9900107c60 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.053+0000 7f98fdffb700 1 -- 172.21.15.46:0/3211499903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f98e80042c0 con 0x7f9900107c60 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.054+0000 7f98fdffb700 1 -- 172.21.15.46:0/3211499903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98e8011a00 con 0x7f9900107c60 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.054+0000 7f9905282700 1 -- 172.21.15.46:0/3211499903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 msgr2=0x7f9900108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.054+0000 7f9905282700 1 --2- 172.21.15.46:0/3211499903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 0x7f9900108080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f98e80098a0 tx=0x7f98e8009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.054+0000 7f9905282700 1 -- 172.21.15.46:0/3211499903 shutdown_connections 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.054+0000 7f9905282700 1 --2- 172.21.15.46:0/3211499903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 0x7f9900108080 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:17.638 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.054+0000 7f9905282700 1 -- 172.21.15.46:0/3211499903 >> 172.21.15.46:0/3211499903 conn(0x7f9900101b70 msgr2=0x7f9900103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.054+0000 7f9905282700 1 -- 172.21.15.46:0/3211499903 shutdown_connections 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.054+0000 7f9905282700 1 -- 172.21.15.46:0/3211499903 wait complete. 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.055+0000 7f9905282700 1 Processor -- start 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.055+0000 7f9905282700 1 -- start start 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.056+0000 7f9905282700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 0x7f990019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.056+0000 7f9905282700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990019a930 con 0x7f9900107c60 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.056+0000 7f98feffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 0x7f990019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.056+0000 7f98feffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 0x7f990019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38164/0 (socket says 172.21.15.46:38164) 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.056+0000 7f98feffd700 1 -- 172.21.15.46:0/1339974861 learned_addr learned my addr 172.21.15.46:0/1339974861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.056+0000 7f98feffd700 1 -- 172.21.15.46:0/1339974861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e8009550 con 0x7f9900107c60 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.056+0000 7f98feffd700 1 --2- 172.21.15.46:0/1339974861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 0x7f990019a3f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009130 tx=0x7f98e8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.056+0000 7f98f7fff700 1 -- 172.21.15.46:0/1339974861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98e8003ab0 con 0x7f9900107c60 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.056+0000 7f98f7fff700 1 -- 172.21.15.46:0/1339974861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f98e8003c10 con 0x7f9900107c60 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.056+0000 7f98f7fff700 1 -- 172.21.15.46:0/1339974861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98e80247d0 con 0x7f9900107c60 2024-03-18T20:29:17.639 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.056+0000 7f9905282700 1 -- 172.21.15.46:0/1339974861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990019ab30 con 0x7f9900107c60 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.056+0000 7f9905282700 1 -- 172.21.15.46:0/1339974861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990019afd0 con 0x7f9900107c60 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.057+0000 7f98f7fff700 1 -- 172.21.15.46:0/1339974861 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 7) v1 ==== 45427+0+0 (secure 0 0 0) 0x7f98e801a430 con 0x7f9900107c60 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.057+0000 7f9905282700 1 -- 172.21.15.46:0/1339974861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99000623e0 con 0x7f9900107c60 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.058+0000 7f98f7fff700 1 --2- 172.21.15.46:0/1339974861 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f98ec038620 0x7f98ec03aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.058+0000 7f98f7fff700 1 -- 172.21.15.46:0/1339974861 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f98e804c5f0 con 0x7f9900107c60 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.058+0000 7f98fe7fc700 1 --2- 172.21.15.46:0/1339974861 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f98ec038620 0x7f98ec03aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.058+0000 7f98fe7fc700 1 --2- 172.21.15.46:0/1339974861 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f98ec038620 0x7f98ec03aae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98f0006fd0 tx=0x7f98f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.063+0000 7f98f7fff700 1 -- 172.21.15.46:0/1339974861 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f98e8018b40 con 0x7f9900107c60 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.277+0000 7f9905282700 1 -- 172.21.15.46:0/1339974861 --> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f9900102e80 con 0x7f98ec038620 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.278+0000 7f98f7fff700 1 -- 172.21.15.46:0/1339974861 <== mgr.14120 v2:172.21.15.46:6800/2753187643 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f9900102e80 con 0x7f98ec038620 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.280+0000 7f9905282700 1 -- 172.21.15.46:0/1339974861 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f98ec038620 msgr2=0x7f98ec03aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.280+0000 7f9905282700 1 --2- 172.21.15.46:0/1339974861 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f98ec038620 0x7f98ec03aae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98f0006fd0 tx=0x7f98f0006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:17.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.280+0000 7f9905282700 1 -- 172.21.15.46:0/1339974861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 msgr2=0x7f990019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:17.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.280+0000 7f9905282700 1 --2- 172.21.15.46:0/1339974861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 0x7f990019a3f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009130 tx=0x7f98e8011ad0 comp rx=0 tx=0).stop 2024-03-18T20:29:17.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.280+0000 7f9905282700 1 -- 172.21.15.46:0/1339974861 shutdown_connections 2024-03-18T20:29:17.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.280+0000 7f9905282700 1 --2- 172.21.15.46:0/1339974861 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f98ec038620 0x7f98ec03aae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:17.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.280+0000 7f9905282700 1 --2- 172.21.15.46:0/1339974861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9900107c60 0x7f990019a3f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:17.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.280+0000 7f9905282700 1 -- 172.21.15.46:0/1339974861 >> 172.21.15.46:0/1339974861 conn(0x7f9900101b70 msgr2=0x7f9900102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:17.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.280+0000 7f9905282700 1 -- 172.21.15.46:0/1339974861 shutdown_connections 2024-03-18T20:29:17.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.280+0000 7f9905282700 1 -- 172.21.15.46:0/1339974861 wait complete. 2024-03-18T20:29:17.641 INFO:teuthology.orchestra.run.smithi046.stdout:Generating ssh key... 2024-03-18T20:29:18.725 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-mon[43495]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:29:19.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: Generating public/private rsa key pair. 2024-03-18T20:29:19.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: Your identification has been saved in /tmp/tmpvfojfaiy/key. 2024-03-18T20:29:19.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: Your public key has been saved in /tmp/tmpvfojfaiy/key.pub. 2024-03-18T20:29:19.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: The key fingerprint is: 2024-03-18T20:29:19.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: SHA256:JhFyTq4krgbv3ZtshoVKKMfzbuRwhchu1Zv+CWEGPec ceph-04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:29:19.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: The key's randomart image is: 2024-03-18T20:29:19.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: +---[RSA 3072]----+ 2024-03-18T20:29:19.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: | . + | 2024-03-18T20:29:19.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: | B . | 2024-03-18T20:29:19.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: | ...oo* . | 2024-03-18T20:29:19.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: | .ooooo= | 2024-03-18T20:29:19.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: |.+...o=oE | 2024-03-18T20:29:19.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: |+oO +o++ | 2024-03-18T20:29:19.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: |o=.O +. | 2024-03-18T20:29:19.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: |....=o+o . | 2024-03-18T20:29:19.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: | .ooo=oo | 2024-03-18T20:29:19.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: +----[SHA256]-----+ 2024-03-18T20:29:19.125 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.984+0000 7f93fff17700 1 Processor -- start 2024-03-18T20:29:19.125 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.985+0000 7f93fff17700 1 -- start start 2024-03-18T20:29:19.125 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.985+0000 7f93fff17700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 0x7f93f8104cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:19.125 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.985+0000 7f93fff17700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f8105290 con 0x7f93f81048a0 2024-03-18T20:29:19.125 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.985+0000 7f93fdcb3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 0x7f93f8104cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:19.125 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.985+0000 7f93fdcb3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 0x7f93f8104cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38170/0 (socket says 172.21.15.46:38170) 2024-03-18T20:29:19.125 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.985+0000 7f93fdcb3700 1 -- 172.21.15.46:0/273217045 learned_addr learned my addr 172.21.15.46:0/273217045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:19.125 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.985+0000 7f93fdcb3700 1 -- 172.21.15.46:0/273217045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f8107830 con 0x7f93f81048a0 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.986+0000 7f93fdcb3700 1 --2- 172.21.15.46:0/273217045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 0x7f93f8104cc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f93f40098a0 tx=0x7f93f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5d2ea275cd0345 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.986+0000 7f93fccb1700 1 -- 172.21.15.46:0/273217045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93f4004160 con 0x7f93f81048a0 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.986+0000 7f93fccb1700 1 -- 172.21.15.46:0/273217045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f93f40042c0 con 0x7f93f81048a0 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.986+0000 7f93fccb1700 1 -- 172.21.15.46:0/273217045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93f4011a00 con 0x7f93f81048a0 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.986+0000 7f93fff17700 1 -- 172.21.15.46:0/273217045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 msgr2=0x7f93f8104cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.986+0000 7f93fff17700 1 --2- 172.21.15.46:0/273217045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 0x7f93f8104cc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f93f40098a0 tx=0x7f93f4009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.986+0000 7f93fff17700 1 -- 172.21.15.46:0/273217045 shutdown_connections 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.986+0000 7f93fff17700 1 --2- 172.21.15.46:0/273217045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 0x7f93f8104cc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.986+0000 7f93fff17700 1 -- 172.21.15.46:0/273217045 >> 172.21.15.46:0/273217045 conn(0x7f93f8100210 msgr2=0x7f93f8102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.987+0000 7f93fff17700 1 -- 172.21.15.46:0/273217045 shutdown_connections 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.987+0000 7f93fff17700 1 -- 172.21.15.46:0/273217045 wait complete. 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.987+0000 7f93fff17700 1 Processor -- start 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.987+0000 7f93fff17700 1 -- start start 2024-03-18T20:29:19.126 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.987+0000 7f93fff17700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 0x7f93f819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.987+0000 7f93fff17700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f819ca70 con 0x7f93f81048a0 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.988+0000 7f93fdcb3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 0x7f93f819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.988+0000 7f93fdcb3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 0x7f93f819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38182/0 (socket says 172.21.15.46:38182) 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.988+0000 7f93fdcb3700 1 -- 172.21.15.46:0/1156147831 learned_addr learned my addr 172.21.15.46:0/1156147831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.988+0000 7f93fdcb3700 1 -- 172.21.15.46:0/1156147831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f4009550 con 0x7f93f81048a0 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.988+0000 7f93fdcb3700 1 --2- 172.21.15.46:0/1156147831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 0x7f93f819c530 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009130 tx=0x7f93f4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.988+0000 7f93eeffd700 1 -- 172.21.15.46:0/1156147831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93f4003ab0 con 0x7f93f81048a0 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.988+0000 7f93eeffd700 1 -- 172.21.15.46:0/1156147831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f93f4003c10 con 0x7f93f81048a0 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.988+0000 7f93fff17700 1 -- 172.21.15.46:0/1156147831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f819cc70 con 0x7f93f81048a0 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.988+0000 7f93eeffd700 1 -- 172.21.15.46:0/1156147831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93f40247d0 con 0x7f93f81048a0 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.988+0000 7f93fff17700 1 -- 172.21.15.46:0/1156147831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f819d110 con 0x7f93f81048a0 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.989+0000 7f93eeffd700 1 -- 172.21.15.46:0/1156147831 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f93f401abb0 con 0x7f93f81048a0 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.989+0000 7f93fff17700 1 -- 172.21.15.46:0/1156147831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f810aa60 con 0x7f93f81048a0 2024-03-18T20:29:19.127 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.989+0000 7f93eeffd700 1 --2- 172.21.15.46:0/1156147831 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f93e4038750 0x7f93e403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.989+0000 7f93eeffd700 1 -- 172.21.15.46:0/1156147831 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f93f404c790 con 0x7f93f81048a0 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.990+0000 7f93fd4b2700 1 --2- 172.21.15.46:0/1156147831 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f93e4038750 0x7f93e403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.990+0000 7f93fd4b2700 1 --2- 172.21.15.46:0/1156147831 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f93e4038750 0x7f93e403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f93e8006fd0 tx=0x7f93e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:17.995+0000 7f93eeffd700 1 -- 172.21.15.46:0/1156147831 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f93f4018900 con 0x7f93f81048a0 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:18.205+0000 7f93fff17700 1 -- 172.21.15.46:0/1156147831 --> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f93f8101410 con 0x7f93e4038750 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:18.727+0000 7f93eeffd700 1 -- 172.21.15.46:0/1156147831 <== mgr.14120 v2:172.21.15.46:6800/2753187643 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f93f8101410 con 0x7f93e4038750 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:18.730+0000 7f93fff17700 1 -- 172.21.15.46:0/1156147831 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f93e4038750 msgr2=0x7f93e403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:18.730+0000 7f93fff17700 1 --2- 172.21.15.46:0/1156147831 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f93e4038750 0x7f93e403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f93e8006fd0 tx=0x7f93e8006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:18.730+0000 7f93fff17700 1 -- 172.21.15.46:0/1156147831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 msgr2=0x7f93f819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:18.730+0000 7f93fff17700 1 --2- 172.21.15.46:0/1156147831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 0x7f93f819c530 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009130 tx=0x7f93f4011ad0 comp rx=0 tx=0).stop 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:18.730+0000 7f93fff17700 1 -- 172.21.15.46:0/1156147831 shutdown_connections 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:18.730+0000 7f93fff17700 1 --2- 172.21.15.46:0/1156147831 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f93e4038750 0x7f93e403ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:18.730+0000 7f93fff17700 1 --2- 172.21.15.46:0/1156147831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93f81048a0 0x7f93f819c530 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:18.730+0000 7f93fff17700 1 -- 172.21.15.46:0/1156147831 >> 172.21.15.46:0/1156147831 conn(0x7f93f8100210 msgr2=0x7f93f8100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:19.128 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:18.730+0000 7f93fff17700 1 -- 172.21.15.46:0/1156147831 shutdown_connections 2024-03-18T20:29:19.129 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:18.730+0000 7f93fff17700 1 -- 172.21.15.46:0/1156147831 wait complete. 2024-03-18T20:29:20.001 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:19 smithi046 ceph-mon[43495]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:29:20.001 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:19 smithi046 ceph-mon[43495]: Generating ssh key... 2024-03-18T20:29:20.002 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:19 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:20.002 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:19 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:20.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMpsbkDnEggsBzNA/LKu/lCOv9Ylg1Nogzk5RP4zjiZvdhmcqhgPkG62T9xBnCrPYSrtI1vUg5vUGY/2WJZO9kee8ZKDncUDcSXh54wSI/QjKznsKVtpxijKycToAmLWel0ux/CcKAcaqE0WXTW2puFf/wfS62gl5uSmq7nqg2RsdUTR2EyT6mHpav7GuwKbE2eNFmiOwo1rRVUemSq5fI9L88oaGSRmb5gykaR5PPklrgOjzGwJKNKFTL87n4wPUjsqDhQBWUjKlCkssZHrOh9NMqmsAwnSnZDnl/enqLoy/CBr9Dl0oR/RmFtVSTnQbzv+LSx2i0GHwjZte9mCRFfQK0+0QkLW/de0dC750E0X4WzyNwxoYr7vnUXlteFEij+yWVB0mPPPgV56zokJ0PjEvsXglmNPOYnzDAz4yLPWLwqcK33wVnFbpGk3PbO/HWqqw0W6IrXvNDBJyG3gk/AWmvlcNnyEs9FYy/NGfvmy6dL6O2pnKZ3tqDHGt5Ln0= ceph-04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:29:20.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.494+0000 7f8a8ea31700 1 Processor -- start 2024-03-18T20:29:20.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.494+0000 7f8a8ea31700 1 -- start start 2024-03-18T20:29:20.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.494+0000 7f8a8ea31700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 0x7f8a88109400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:20.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.494+0000 7f8a8ea31700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a88074520 con 0x7f8a88107010 2024-03-18T20:29:20.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.494+0000 7f8a87fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 0x7f8a88109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:20.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.494+0000 7f8a87fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 0x7f8a88109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38198/0 (socket says 172.21.15.46:38198) 2024-03-18T20:29:20.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.494+0000 7f8a87fff700 1 -- 172.21.15.46:0/2835942565 learned_addr learned my addr 172.21.15.46:0/2835942565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:20.071 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.494+0000 7f8a87fff700 1 -- 172.21.15.46:0/2835942565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a88109940 con 0x7f8a88107010 2024-03-18T20:29:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.495+0000 7f8a87fff700 1 --2- 172.21.15.46:0/2835942565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 0x7f8a88109400 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8a780098a0 tx=0x7f8a78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f051a9d4c186df37 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.495+0000 7f8a86ffd700 1 -- 172.21.15.46:0/2835942565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a78004160 con 0x7f8a88107010 2024-03-18T20:29:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.495+0000 7f8a86ffd700 1 -- 172.21.15.46:0/2835942565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a780042c0 con 0x7f8a88107010 2024-03-18T20:29:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.495+0000 7f8a86ffd700 1 -- 172.21.15.46:0/2835942565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a78011a00 con 0x7f8a88107010 2024-03-18T20:29:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.495+0000 7f8a8ea31700 1 -- 172.21.15.46:0/2835942565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 msgr2=0x7f8a88109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.495+0000 7f8a8ea31700 1 --2- 172.21.15.46:0/2835942565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 0x7f8a88109400 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8a780098a0 tx=0x7f8a78009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.495+0000 7f8a8ea31700 1 -- 172.21.15.46:0/2835942565 shutdown_connections 2024-03-18T20:29:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.495+0000 7f8a8ea31700 1 --2- 172.21.15.46:0/2835942565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 0x7f8a88109400 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.495+0000 7f8a8ea31700 1 -- 172.21.15.46:0/2835942565 >> 172.21.15.46:0/2835942565 conn(0x7f8a88100aa0 msgr2=0x7f8a88102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.495+0000 7f8a8ea31700 1 -- 172.21.15.46:0/2835942565 shutdown_connections 2024-03-18T20:29:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.495+0000 7f8a8ea31700 1 -- 172.21.15.46:0/2835942565 wait complete. 2024-03-18T20:29:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.496+0000 7f8a8ea31700 1 Processor -- start 2024-03-18T20:29:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.496+0000 7f8a8ea31700 1 -- start start 2024-03-18T20:29:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.496+0000 7f8a8ea31700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 0x7f8a8819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.496+0000 7f8a8ea31700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8819ca50 con 0x7f8a88107010 2024-03-18T20:29:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.497+0000 7f8a87fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 0x7f8a8819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.497+0000 7f8a87fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 0x7f8a8819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38204/0 (socket says 172.21.15.46:38204) 2024-03-18T20:29:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.497+0000 7f8a87fff700 1 -- 172.21.15.46:0/1044042337 learned_addr learned my addr 172.21.15.46:0/1044042337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.497+0000 7f8a87fff700 1 -- 172.21.15.46:0/1044042337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78009550 con 0x7f8a88107010 2024-03-18T20:29:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.497+0000 7f8a87fff700 1 --2- 172.21.15.46:0/1044042337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 0x7f8a8819c510 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009fb0 tx=0x7f8a78011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.497+0000 7f8a857fa700 1 -- 172.21.15.46:0/1044042337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a78003a40 con 0x7f8a88107010 2024-03-18T20:29:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.497+0000 7f8a857fa700 1 -- 172.21.15.46:0/1044042337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a78003ba0 con 0x7f8a88107010 2024-03-18T20:29:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.497+0000 7f8a8ea31700 1 -- 172.21.15.46:0/1044042337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8819cc50 con 0x7f8a88107010 2024-03-18T20:29:20.073 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.497+0000 7f8a857fa700 1 -- 172.21.15.46:0/1044042337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a7801a750 con 0x7f8a88107010 2024-03-18T20:29:20.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.497+0000 7f8a8ea31700 1 -- 172.21.15.46:0/1044042337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8819d0f0 con 0x7f8a88107010 2024-03-18T20:29:20.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.498+0000 7f8a857fa700 1 -- 172.21.15.46:0/1044042337 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f8a7801a980 con 0x7f8a88107010 2024-03-18T20:29:20.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.498+0000 7f8a8ea31700 1 -- 172.21.15.46:0/1044042337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8810a740 con 0x7f8a88107010 2024-03-18T20:29:20.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.498+0000 7f8a857fa700 1 --2- 172.21.15.46:0/1044042337 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f8a70038700 0x7f8a7003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:20.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.498+0000 7f8a857fa700 1 -- 172.21.15.46:0/1044042337 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8a7804c6a0 con 0x7f8a88107010 2024-03-18T20:29:20.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.499+0000 7f8a877fe700 1 --2- 172.21.15.46:0/1044042337 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f8a70038700 0x7f8a7003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:20.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.499+0000 7f8a877fe700 1 --2- 172.21.15.46:0/1044042337 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f8a70038700 0x7f8a7003abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c006fd0 tx=0x7f8a7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:20.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.503+0000 7f8a857fa700 1 -- 172.21.15.46:0/1044042337 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8a7801ac30 con 0x7f8a88107010 2024-03-18T20:29:20.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.716+0000 7f8a8ea31700 1 -- 172.21.15.46:0/1044042337 --> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a88101d20 con 0x7f8a70038700 2024-03-18T20:29:20.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.716+0000 7f8a857fa700 1 -- 172.21.15.46:0/1044042337 <== mgr.14120 v2:172.21.15.46:6800/2753187643 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f8a88101d20 con 0x7f8a70038700 2024-03-18T20:29:20.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.719+0000 7f8a8ea31700 1 -- 172.21.15.46:0/1044042337 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f8a70038700 msgr2=0x7f8a7003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:20.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.719+0000 7f8a8ea31700 1 --2- 172.21.15.46:0/1044042337 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f8a70038700 0x7f8a7003abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c006fd0 tx=0x7f8a7c006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:20.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.719+0000 7f8a8ea31700 1 -- 172.21.15.46:0/1044042337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 msgr2=0x7f8a8819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:20.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.719+0000 7f8a8ea31700 1 --2- 172.21.15.46:0/1044042337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 0x7f8a8819c510 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009fb0 tx=0x7f8a78011ad0 comp rx=0 tx=0).stop 2024-03-18T20:29:20.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.719+0000 7f8a8ea31700 1 -- 172.21.15.46:0/1044042337 shutdown_connections 2024-03-18T20:29:20.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.719+0000 7f8a8ea31700 1 --2- 172.21.15.46:0/1044042337 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f8a70038700 0x7f8a7003abc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:20.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.719+0000 7f8a8ea31700 1 --2- 172.21.15.46:0/1044042337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a88107010 0x7f8a8819c510 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:20.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.719+0000 7f8a8ea31700 1 -- 172.21.15.46:0/1044042337 >> 172.21.15.46:0/1044042337 conn(0x7f8a88100aa0 msgr2=0x7f8a88101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:20.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.719+0000 7f8a8ea31700 1 -- 172.21.15.46:0/1044042337 shutdown_connections 2024-03-18T20:29:20.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:19.719+0000 7f8a8ea31700 1 -- 172.21.15.46:0/1044042337 wait complete. 2024-03-18T20:29:20.075 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-18T20:29:20.075 INFO:teuthology.orchestra.run.smithi046.stdout:Adding key to root@localhost authorized_keys... 2024-03-18T20:29:20.076 INFO:teuthology.orchestra.run.smithi046.stdout:Adding host smithi046... 2024-03-18T20:29:20.966 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:20 smithi046 ceph-mon[43495]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:29:21.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:21 smithi046 ceph-mon[43495]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:29:23.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:22 smithi046 ceph-mon[43495]: Deploying cephadm binary to smithi046 2024-03-18T20:29:23.388 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Added host 'smithi046' with addr '172.21.15.46' 2024-03-18T20:29:23.388 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.438+0000 7f254419a700 1 Processor -- start 2024-03-18T20:29:23.388 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.438+0000 7f254419a700 1 -- start start 2024-03-18T20:29:23.388 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.439+0000 7f254419a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 0x7f253c104ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.439+0000 7f254419a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1052b0 con 0x7f253c1048c0 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.439+0000 7f2541f36700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 0x7f253c104ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.439+0000 7f2541f36700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 0x7f253c104ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38208/0 (socket says 172.21.15.46:38208) 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.439+0000 7f2541f36700 1 -- 172.21.15.46:0/1535994851 learned_addr learned my addr 172.21.15.46:0/1535994851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.439+0000 7f2541f36700 1 -- 172.21.15.46:0/1535994851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c107850 con 0x7f253c1048c0 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.439+0000 7f2541f36700 1 --2- 172.21.15.46:0/1535994851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 0x7f253c104ce0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f25380098a0 tx=0x7f2538009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1085298eeac282ef server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.440+0000 7f2540f34700 1 -- 172.21.15.46:0/1535994851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2538004160 con 0x7f253c1048c0 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.440+0000 7f2540f34700 1 -- 172.21.15.46:0/1535994851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25380042c0 con 0x7f253c1048c0 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.440+0000 7f2540f34700 1 -- 172.21.15.46:0/1535994851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2538011a00 con 0x7f253c1048c0 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.440+0000 7f254419a700 1 -- 172.21.15.46:0/1535994851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 msgr2=0x7f253c104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.440+0000 7f254419a700 1 --2- 172.21.15.46:0/1535994851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 0x7f253c104ce0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f25380098a0 tx=0x7f2538009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.440+0000 7f254419a700 1 -- 172.21.15.46:0/1535994851 shutdown_connections 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.440+0000 7f254419a700 1 --2- 172.21.15.46:0/1535994851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 0x7f253c104ce0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:23.389 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.440+0000 7f254419a700 1 -- 172.21.15.46:0/1535994851 >> 172.21.15.46:0/1535994851 conn(0x7f253c100210 msgr2=0x7f253c102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.440+0000 7f254419a700 1 -- 172.21.15.46:0/1535994851 shutdown_connections 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.440+0000 7f254419a700 1 -- 172.21.15.46:0/1535994851 wait complete. 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.441+0000 7f254419a700 1 Processor -- start 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.441+0000 7f254419a700 1 -- start start 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.441+0000 7f254419a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 0x7f253c19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.441+0000 7f254419a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c19cac0 con 0x7f253c1048c0 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.441+0000 7f2541f36700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 0x7f253c19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.441+0000 7f2541f36700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 0x7f253c19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38218/0 (socket says 172.21.15.46:38218) 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.441+0000 7f2541f36700 1 -- 172.21.15.46:0/1554163628 learned_addr learned my addr 172.21.15.46:0/1554163628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.442+0000 7f2541f36700 1 -- 172.21.15.46:0/1554163628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2538009550 con 0x7f253c1048c0 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.442+0000 7f2541f36700 1 --2- 172.21.15.46:0/1554163628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 0x7f253c19c580 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2538009130 tx=0x7f2538011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.442+0000 7f2532ffd700 1 -- 172.21.15.46:0/1554163628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2538003ab0 con 0x7f253c1048c0 2024-03-18T20:29:23.390 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.442+0000 7f2532ffd700 1 -- 172.21.15.46:0/1554163628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2538003c10 con 0x7f253c1048c0 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.442+0000 7f2532ffd700 1 -- 172.21.15.46:0/1554163628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25380247d0 con 0x7f253c1048c0 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.442+0000 7f254419a700 1 -- 172.21.15.46:0/1554163628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c19ccc0 con 0x7f253c1048c0 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.442+0000 7f254419a700 1 -- 172.21.15.46:0/1554163628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c19d160 con 0x7f253c1048c0 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.443+0000 7f254419a700 1 -- 172.21.15.46:0/1554163628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c10aa80 con 0x7f253c1048c0 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.443+0000 7f2532ffd700 1 -- 172.21.15.46:0/1554163628 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f253801abb0 con 0x7f253c1048c0 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.443+0000 7f2532ffd700 1 --2- 172.21.15.46:0/1554163628 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f2528038750 0x7f252803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.443+0000 7f2532ffd700 1 -- 172.21.15.46:0/1554163628 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f253804c790 con 0x7f253c1048c0 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.443+0000 7f2541735700 1 --2- 172.21.15.46:0/1554163628 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f2528038750 0x7f252803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.444+0000 7f2541735700 1 --2- 172.21.15.46:0/1554163628 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f2528038750 0x7f252803ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f252c006fd0 tx=0x7f252c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.449+0000 7f2532ffd700 1 -- 172.21.15.46:0/1554163628 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2538018530 con 0x7f253c1048c0 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:20.665+0000 7f254419a700 1 -- 172.21.15.46:0/1554163628 --> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}) v1 -- 0x7f253c101430 con 0x7f2528038750 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.069+0000 7f2532ffd700 1 -- 172.21.15.46:0/1554163628 <== mgr.14120 v2:172.21.15.46:6800/2753187643 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f253c101430 con 0x7f2528038750 2024-03-18T20:29:23.391 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.072+0000 7f254419a700 1 -- 172.21.15.46:0/1554163628 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f2528038750 msgr2=0x7f252803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:23.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.072+0000 7f254419a700 1 --2- 172.21.15.46:0/1554163628 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f2528038750 0x7f252803ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f252c006fd0 tx=0x7f252c006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:23.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.072+0000 7f254419a700 1 -- 172.21.15.46:0/1554163628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 msgr2=0x7f253c19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:23.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.072+0000 7f254419a700 1 --2- 172.21.15.46:0/1554163628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 0x7f253c19c580 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2538009130 tx=0x7f2538011ad0 comp rx=0 tx=0).stop 2024-03-18T20:29:23.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.072+0000 7f254419a700 1 -- 172.21.15.46:0/1554163628 shutdown_connections 2024-03-18T20:29:23.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.072+0000 7f254419a700 1 --2- 172.21.15.46:0/1554163628 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f2528038750 0x7f252803ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:23.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.072+0000 7f254419a700 1 --2- 172.21.15.46:0/1554163628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c1048c0 0x7f253c19c580 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:23.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.072+0000 7f254419a700 1 -- 172.21.15.46:0/1554163628 >> 172.21.15.46:0/1554163628 conn(0x7f253c100210 msgr2=0x7f253c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:23.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.072+0000 7f254419a700 1 -- 172.21.15.46:0/1554163628 shutdown_connections 2024-03-18T20:29:23.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.072+0000 7f254419a700 1 -- 172.21.15.46:0/1554163628 wait complete. 2024-03-18T20:29:23.392 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mon service... 2024-03-18T20:29:24.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:24 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:24.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:24 smithi046 ceph-mon[43495]: Added host smithi046 2024-03-18T20:29:24.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:24 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:29:24.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:24 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:24.470 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-18T20:29:24.470 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.745+0000 7fa1bfe72700 1 Processor -- start 2024-03-18T20:29:24.470 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.745+0000 7fa1bfe72700 1 -- start start 2024-03-18T20:29:24.470 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.745+0000 7fa1bfe72700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 0x7fa1b8109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:24.470 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.745+0000 7fa1bfe72700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8074520 con 0x7fa1b8107040 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.745+0000 7fa1bdc0e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 0x7fa1b8109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.746+0000 7fa1bdc0e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 0x7fa1b8109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38234/0 (socket says 172.21.15.46:38234) 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.746+0000 7fa1bdc0e700 1 -- 172.21.15.46:0/26051623 learned_addr learned my addr 172.21.15.46:0/26051623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.746+0000 7fa1bdc0e700 1 -- 172.21.15.46:0/26051623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b8109970 con 0x7fa1b8107040 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.746+0000 7fa1bdc0e700 1 --2- 172.21.15.46:0/26051623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 0x7fa1b8109430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40098a0 tx=0x7fa1b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a9ecc9034ca64b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.746+0000 7fa1bcc0c700 1 -- 172.21.15.46:0/26051623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1b4004160 con 0x7fa1b8107040 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.746+0000 7fa1bcc0c700 1 -- 172.21.15.46:0/26051623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa1b40042c0 con 0x7fa1b8107040 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.746+0000 7fa1bcc0c700 1 -- 172.21.15.46:0/26051623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1b4011a00 con 0x7fa1b8107040 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.747+0000 7fa1bfe72700 1 -- 172.21.15.46:0/26051623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 msgr2=0x7fa1b8109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.747+0000 7fa1bfe72700 1 --2- 172.21.15.46:0/26051623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 0x7fa1b8109430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40098a0 tx=0x7fa1b4009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.747+0000 7fa1bfe72700 1 -- 172.21.15.46:0/26051623 shutdown_connections 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.747+0000 7fa1bfe72700 1 --2- 172.21.15.46:0/26051623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 0x7fa1b8109430 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:24.471 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.747+0000 7fa1bfe72700 1 -- 172.21.15.46:0/26051623 >> 172.21.15.46:0/26051623 conn(0x7fa1b8100a70 msgr2=0x7fa1b8102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.747+0000 7fa1bfe72700 1 -- 172.21.15.46:0/26051623 shutdown_connections 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.747+0000 7fa1bfe72700 1 -- 172.21.15.46:0/26051623 wait complete. 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.748+0000 7fa1bfe72700 1 Processor -- start 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.748+0000 7fa1bfe72700 1 -- start start 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.748+0000 7fa1bfe72700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 0x7fa1b819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.748+0000 7fa1bfe72700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b819a7f0 con 0x7fa1b8107040 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.748+0000 7fa1bdc0e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 0x7fa1b819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.748+0000 7fa1bdc0e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 0x7fa1b819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38242/0 (socket says 172.21.15.46:38242) 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.748+0000 7fa1bdc0e700 1 -- 172.21.15.46:0/1625799755 learned_addr learned my addr 172.21.15.46:0/1625799755 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.749+0000 7fa1bdc0e700 1 -- 172.21.15.46:0/1625799755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b4009550 con 0x7fa1b8107040 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.749+0000 7fa1bdc0e700 1 --2- 172.21.15.46:0/1625799755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 0x7fa1b819a2b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009fb0 tx=0x7fa1b4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.749+0000 7fa1aeffd700 1 -- 172.21.15.46:0/1625799755 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1b4003a40 con 0x7fa1b8107040 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.749+0000 7fa1bfe72700 1 -- 172.21.15.46:0/1625799755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b819a9f0 con 0x7fa1b8107040 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.749+0000 7fa1aeffd700 1 -- 172.21.15.46:0/1625799755 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa1b4003ba0 con 0x7fa1b8107040 2024-03-18T20:29:24.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.749+0000 7fa1aeffd700 1 -- 172.21.15.46:0/1625799755 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1b401a750 con 0x7fa1b8107040 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.749+0000 7fa1bfe72700 1 -- 172.21.15.46:0/1625799755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b819ae90 con 0x7fa1b8107040 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.750+0000 7fa1bfe72700 1 -- 172.21.15.46:0/1625799755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b810a770 con 0x7fa1b8107040 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.750+0000 7fa1aeffd700 1 -- 172.21.15.46:0/1625799755 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fa1b4024b60 con 0x7fa1b8107040 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.751+0000 7fa1aeffd700 1 --2- 172.21.15.46:0/1625799755 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fa1a4038700 0x7fa1a403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.751+0000 7fa1aeffd700 1 -- 172.21.15.46:0/1625799755 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa1b4055760 con 0x7fa1b8107040 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.751+0000 7fa1bd40d700 1 --2- 172.21.15.46:0/1625799755 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fa1a4038700 0x7fa1a403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.751+0000 7fa1bd40d700 1 --2- 172.21.15.46:0/1625799755 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fa1a4038700 0x7fa1a403abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8006fd0 tx=0x7fa1a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.756+0000 7fa1aeffd700 1 -- 172.21.15.46:0/1625799755 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa1b4052060 con 0x7fa1b8107040 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.987+0000 7fa1bfe72700 1 -- 172.21.15.46:0/1625799755 --> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa1b8101c60 con 0x7fa1a4038700 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.992+0000 7fa1aeffd700 1 -- 172.21.15.46:0/1625799755 <== mgr.14120 v2:172.21.15.46:6800/2753187643 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa1b8101c60 con 0x7fa1a4038700 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.994+0000 7fa1bfe72700 1 -- 172.21.15.46:0/1625799755 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fa1a4038700 msgr2=0x7fa1a403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.994+0000 7fa1bfe72700 1 --2- 172.21.15.46:0/1625799755 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fa1a4038700 0x7fa1a403abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8006fd0 tx=0x7fa1a8006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.994+0000 7fa1bfe72700 1 -- 172.21.15.46:0/1625799755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 msgr2=0x7fa1b819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:24.473 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.994+0000 7fa1bfe72700 1 --2- 172.21.15.46:0/1625799755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 0x7fa1b819a2b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009fb0 tx=0x7fa1b4011ad0 comp rx=0 tx=0).stop 2024-03-18T20:29:24.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.994+0000 7fa1bfe72700 1 -- 172.21.15.46:0/1625799755 shutdown_connections 2024-03-18T20:29:24.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.994+0000 7fa1bfe72700 1 --2- 172.21.15.46:0/1625799755 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fa1a4038700 0x7fa1a403abc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:24.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.994+0000 7fa1bfe72700 1 --2- 172.21.15.46:0/1625799755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1b8107040 0x7fa1b819a2b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:24.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.994+0000 7fa1bfe72700 1 -- 172.21.15.46:0/1625799755 >> 172.21.15.46:0/1625799755 conn(0x7fa1b8100a70 msgr2=0x7fa1b8101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:24.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.995+0000 7fa1bfe72700 1 -- 172.21.15.46:0/1625799755 shutdown_connections 2024-03-18T20:29:24.474 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:23.995+0000 7fa1bfe72700 1 -- 172.21.15.46:0/1625799755 wait complete. 2024-03-18T20:29:24.474 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mgr service... 2024-03-18T20:29:25.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:25 smithi046 ceph-mon[43495]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:29:25.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:25 smithi046 ceph-mon[43495]: Saving service mon spec with placement count:5 2024-03-18T20:29:25.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:25 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:25.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:25 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:25.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:25 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:25.631 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-18T20:29:25.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.825+0000 7f5daa6f3700 1 Processor -- start 2024-03-18T20:29:25.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.825+0000 7f5daa6f3700 1 -- start start 2024-03-18T20:29:25.642 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.825+0000 7f5daa6f3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 0x7f5da41058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:25.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.825+0000 7f5daa6f3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da4105ec0 con 0x7f5da41054d0 2024-03-18T20:29:25.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.826+0000 7f5da3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 0x7f5da41058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:25.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.826+0000 7f5da3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 0x7f5da41058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38248/0 (socket says 172.21.15.46:38248) 2024-03-18T20:29:25.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.826+0000 7f5da3fff700 1 -- 172.21.15.46:0/464925038 learned_addr learned my addr 172.21.15.46:0/464925038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:25.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.826+0000 7f5da3fff700 1 -- 172.21.15.46:0/464925038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da41066d0 con 0x7f5da41054d0 2024-03-18T20:29:25.643 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.826+0000 7f5da3fff700 1 --2- 172.21.15.46:0/464925038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 0x7f5da41058f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5d940098a0 tx=0x7f5d94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cc0415e7111c7b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:25.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.826+0000 7f5da2ffd700 1 -- 172.21.15.46:0/464925038 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d94004160 con 0x7f5da41054d0 2024-03-18T20:29:25.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.826+0000 7f5da2ffd700 1 -- 172.21.15.46:0/464925038 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d940042c0 con 0x7f5da41054d0 2024-03-18T20:29:25.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.826+0000 7f5da2ffd700 1 -- 172.21.15.46:0/464925038 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d940119b0 con 0x7f5da41054d0 2024-03-18T20:29:25.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.827+0000 7f5daa6f3700 1 -- 172.21.15.46:0/464925038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 msgr2=0x7f5da41058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:25.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.827+0000 7f5daa6f3700 1 --2- 172.21.15.46:0/464925038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 0x7f5da41058f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5d940098a0 tx=0x7f5d94009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:25.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.827+0000 7f5daa6f3700 1 -- 172.21.15.46:0/464925038 shutdown_connections 2024-03-18T20:29:25.644 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.827+0000 7f5daa6f3700 1 --2- 172.21.15.46:0/464925038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 0x7f5da41058f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:25.645 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.827+0000 7f5daa6f3700 1 -- 172.21.15.46:0/464925038 >> 172.21.15.46:0/464925038 conn(0x7f5da4100a70 msgr2=0x7f5da4102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:25.645 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.827+0000 7f5daa6f3700 1 -- 172.21.15.46:0/464925038 shutdown_connections 2024-03-18T20:29:25.645 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.827+0000 7f5daa6f3700 1 -- 172.21.15.46:0/464925038 wait complete. 2024-03-18T20:29:25.645 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.828+0000 7f5daa6f3700 1 Processor -- start 2024-03-18T20:29:25.645 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.828+0000 7f5daa6f3700 1 -- start start 2024-03-18T20:29:25.645 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.829+0000 7f5daa6f3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 0x7f5da419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:25.646 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.829+0000 7f5daa6f3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da419a8b0 con 0x7f5da41054d0 2024-03-18T20:29:25.646 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.829+0000 7f5da3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 0x7f5da419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:25.646 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.829+0000 7f5da3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 0x7f5da419a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38254/0 (socket says 172.21.15.46:38254) 2024-03-18T20:29:25.646 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.829+0000 7f5da3fff700 1 -- 172.21.15.46:0/3389748928 learned_addr learned my addr 172.21.15.46:0/3389748928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:25.646 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.829+0000 7f5da3fff700 1 -- 172.21.15.46:0/3389748928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d94009550 con 0x7f5da41054d0 2024-03-18T20:29:25.646 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.829+0000 7f5da3fff700 1 --2- 172.21.15.46:0/3389748928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 0x7f5da419a370 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5d94003e60 tx=0x7f5d94011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:25.646 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.830+0000 7f5da17fa700 1 -- 172.21.15.46:0/3389748928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d94003a60 con 0x7f5da41054d0 2024-03-18T20:29:25.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.830+0000 7f5da17fa700 1 -- 172.21.15.46:0/3389748928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d94003bc0 con 0x7f5da41054d0 2024-03-18T20:29:25.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.830+0000 7f5daa6f3700 1 -- 172.21.15.46:0/3389748928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da419aab0 con 0x7f5da41054d0 2024-03-18T20:29:25.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.830+0000 7f5da17fa700 1 -- 172.21.15.46:0/3389748928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d940196c0 con 0x7f5da41054d0 2024-03-18T20:29:25.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.830+0000 7f5daa6f3700 1 -- 172.21.15.46:0/3389748928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da419af50 con 0x7f5da41054d0 2024-03-18T20:29:25.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.831+0000 7f5da17fa700 1 -- 172.21.15.46:0/3389748928 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f5d9401a430 con 0x7f5da41054d0 2024-03-18T20:29:25.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.831+0000 7f5daa6f3700 1 -- 172.21.15.46:0/3389748928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da4194570 con 0x7f5da41054d0 2024-03-18T20:29:25.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.831+0000 7f5da17fa700 1 --2- 172.21.15.46:0/3389748928 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f5d8c0386e0 0x7f5d8c03aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:25.647 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.831+0000 7f5da17fa700 1 -- 172.21.15.46:0/3389748928 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5d9404c910 con 0x7f5da41054d0 2024-03-18T20:29:25.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.831+0000 7f5da37fe700 1 --2- 172.21.15.46:0/3389748928 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f5d8c0386e0 0x7f5d8c03aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:25.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.832+0000 7f5da37fe700 1 --2- 172.21.15.46:0/3389748928 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f5d8c0386e0 0x7f5d8c03aba0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5d98006fd0 tx=0x7f5d98006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:25.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:24.836+0000 7f5da17fa700 1 -- 172.21.15.46:0/3389748928 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5d94010560 con 0x7f5da41054d0 2024-03-18T20:29:25.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:25.107+0000 7f5daa6f3700 1 -- 172.21.15.46:0/3389748928 --> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5da40611f0 con 0x7f5d8c0386e0 2024-03-18T20:29:25.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:25.112+0000 7f5da17fa700 1 -- 172.21.15.46:0/3389748928 <== mgr.14120 v2:172.21.15.46:6800/2753187643 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5da40611f0 con 0x7f5d8c0386e0 2024-03-18T20:29:25.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:25.114+0000 7f5daa6f3700 1 -- 172.21.15.46:0/3389748928 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f5d8c0386e0 msgr2=0x7f5d8c03aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:25.648 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:25.114+0000 7f5daa6f3700 1 --2- 172.21.15.46:0/3389748928 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f5d8c0386e0 0x7f5d8c03aba0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5d98006fd0 tx=0x7f5d98006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:25.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:25.114+0000 7f5daa6f3700 1 -- 172.21.15.46:0/3389748928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 msgr2=0x7f5da419a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:25.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:25.114+0000 7f5daa6f3700 1 --2- 172.21.15.46:0/3389748928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 0x7f5da419a370 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5d94003e60 tx=0x7f5d94011a30 comp rx=0 tx=0).stop 2024-03-18T20:29:25.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:25.114+0000 7f5daa6f3700 1 -- 172.21.15.46:0/3389748928 shutdown_connections 2024-03-18T20:29:25.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:25.114+0000 7f5daa6f3700 1 --2- 172.21.15.46:0/3389748928 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f5d8c0386e0 0x7f5d8c03aba0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:25.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:25.114+0000 7f5daa6f3700 1 --2- 172.21.15.46:0/3389748928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da41054d0 0x7f5da419a370 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:25.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:25.114+0000 7f5daa6f3700 1 -- 172.21.15.46:0/3389748928 >> 172.21.15.46:0/3389748928 conn(0x7f5da4100a70 msgr2=0x7f5da4101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:25.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:25.114+0000 7f5daa6f3700 1 -- 172.21.15.46:0/3389748928 shutdown_connections 2024-03-18T20:29:25.649 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:25.114+0000 7f5daa6f3700 1 -- 172.21.15.46:0/3389748928 wait complete. 2024-03-18T20:29:26.677 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.053+0000 7f41a985e700 1 Processor -- start 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.053+0000 7f41a985e700 1 -- start start 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.054+0000 7f41a985e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 0x7f41a407a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.054+0000 7f41a985e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a407a8f0 con 0x7f41a407be70 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.054+0000 7f41a2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 0x7f41a407a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.054+0000 7f41a2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 0x7f41a407a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52920/0 (socket says 172.21.15.46:52920) 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.054+0000 7f41a2ffd700 1 -- 172.21.15.46:0/2568172015 learned_addr learned my addr 172.21.15.46:0/2568172015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.054+0000 7f41a2ffd700 1 -- 172.21.15.46:0/2568172015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a407aa30 con 0x7f41a407be70 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.055+0000 7f41a2ffd700 1 --2- 172.21.15.46:0/2568172015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 0x7f41a407a320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f41940098a0 tx=0x7f4194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa82bd1a30b38d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.055+0000 7f41a1ffb700 1 -- 172.21.15.46:0/2568172015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4194004160 con 0x7f41a407be70 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.055+0000 7f41a1ffb700 1 -- 172.21.15.46:0/2568172015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41940042c0 con 0x7f41a407be70 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.055+0000 7f41a1ffb700 1 -- 172.21.15.46:0/2568172015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41940119b0 con 0x7f41a407be70 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.056+0000 7f41a985e700 1 -- 172.21.15.46:0/2568172015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 msgr2=0x7f41a407a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.056+0000 7f41a985e700 1 --2- 172.21.15.46:0/2568172015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 0x7f41a407a320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f41940098a0 tx=0x7f4194009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.056+0000 7f41a985e700 1 -- 172.21.15.46:0/2568172015 shutdown_connections 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.056+0000 7f41a985e700 1 --2- 172.21.15.46:0/2568172015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 0x7f41a407a320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.056+0000 7f41a985e700 1 -- 172.21.15.46:0/2568172015 >> 172.21.15.46:0/2568172015 conn(0x7f41a4101bb0 msgr2=0x7f41a4103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.056+0000 7f41a985e700 1 -- 172.21.15.46:0/2568172015 shutdown_connections 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.056+0000 7f41a985e700 1 -- 172.21.15.46:0/2568172015 wait complete. 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.057+0000 7f41a985e700 1 Processor -- start 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.057+0000 7f41a985e700 1 -- start start 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.057+0000 7f41a985e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 0x7f41a419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.057+0000 7f41a985e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a419ca20 con 0x7f41a407be70 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.058+0000 7f41a2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 0x7f41a419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.058+0000 7f41a2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 0x7f41a419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52928/0 (socket says 172.21.15.46:52928) 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.058+0000 7f41a2ffd700 1 -- 172.21.15.46:0/719690623 learned_addr learned my addr 172.21.15.46:0/719690623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.058+0000 7f41a2ffd700 1 -- 172.21.15.46:0/719690623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4194009550 con 0x7f41a407be70 2024-03-18T20:29:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.058+0000 7f41a2ffd700 1 --2- 172.21.15.46:0/719690623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 0x7f41a419c4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4194011670 tx=0x7f4194011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.058+0000 7f41a885c700 1 -- 172.21.15.46:0/719690623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4194003b20 con 0x7f41a407be70 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.058+0000 7f41a985e700 1 -- 172.21.15.46:0/719690623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a419cc20 con 0x7f41a407be70 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.058+0000 7f41a885c700 1 -- 172.21.15.46:0/719690623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4194003c80 con 0x7f41a407be70 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.058+0000 7f41a885c700 1 -- 172.21.15.46:0/719690623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4194019770 con 0x7f41a407be70 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.058+0000 7f41a985e700 1 -- 172.21.15.46:0/719690623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a419d0c0 con 0x7f41a407be70 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.059+0000 7f41a985e700 1 -- 172.21.15.46:0/719690623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a410c9b0 con 0x7f41a407be70 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.059+0000 7f41a885c700 1 -- 172.21.15.46:0/719690623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f419401ab60 con 0x7f41a407be70 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.060+0000 7f41a885c700 1 --2- 172.21.15.46:0/719690623 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f418c038700 0x7f418c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.060+0000 7f41a885c700 1 -- 172.21.15.46:0/719690623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f419404c9b0 con 0x7f41a407be70 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.060+0000 7f41a27fc700 1 --2- 172.21.15.46:0/719690623 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f418c038700 0x7f418c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.060+0000 7f41a27fc700 1 --2- 172.21.15.46:0/719690623 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f418c038700 0x7f418c03abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4198006fd0 tx=0x7f4198006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.065+0000 7f41a885c700 1 -- 172.21.15.46:0/719690623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f419401a430 con 0x7f41a407be70 2024-03-18T20:29:26.680 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.290+0000 7f41a985e700 1 -- 172.21.15.46:0/719690623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f41a419fd70 con 0x7f41a407be70 2024-03-18T20:29:26.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.293+0000 7f41a885c700 1 -- 172.21.15.46:0/719690623 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f419402db40 con 0x7f41a407be70 2024-03-18T20:29:26.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.296+0000 7f41a985e700 1 -- 172.21.15.46:0/719690623 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f418c038700 msgr2=0x7f418c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:26.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.296+0000 7f41a985e700 1 --2- 172.21.15.46:0/719690623 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f418c038700 0x7f418c03abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4198006fd0 tx=0x7f4198006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:26.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.296+0000 7f41a985e700 1 -- 172.21.15.46:0/719690623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 msgr2=0x7f41a419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:26.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.296+0000 7f41a985e700 1 --2- 172.21.15.46:0/719690623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 0x7f41a419c4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4194011670 tx=0x7f4194011a30 comp rx=0 tx=0).stop 2024-03-18T20:29:26.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.296+0000 7f41a985e700 1 -- 172.21.15.46:0/719690623 shutdown_connections 2024-03-18T20:29:26.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.296+0000 7f41a985e700 1 --2- 172.21.15.46:0/719690623 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f418c038700 0x7f418c03abc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:26.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.296+0000 7f41a985e700 1 --2- 172.21.15.46:0/719690623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a407be70 0x7f41a419c4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:26.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.296+0000 7f41a985e700 1 -- 172.21.15.46:0/719690623 >> 172.21.15.46:0/719690623 conn(0x7f41a4101bb0 msgr2=0x7f41a4102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:26.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.296+0000 7f41a985e700 1 -- 172.21.15.46:0/719690623 shutdown_connections 2024-03-18T20:29:26.681 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:26.296+0000 7f41a985e700 1 -- 172.21.15.46:0/719690623 wait complete. 2024-03-18T20:29:27.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:26 smithi046 ceph-mon[43495]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:29:27.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:26 smithi046 ceph-mon[43495]: Saving service mgr spec with placement count:2 2024-03-18T20:29:27.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:26 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:27.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:26 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/719690623' entity='client.admin' 2024-03-18T20:29:27.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:26 smithi046 ceph-mon[43495]: from='mgr.14120 172.21.15.46:0/80292447' entity='mgr.x' 2024-03-18T20:29:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.041+0000 7fecf3baa700 1 Processor -- start 2024-03-18T20:29:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.041+0000 7fecf3baa700 1 -- start start 2024-03-18T20:29:27.911 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.041+0000 7fecf3baa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 0x7fecec106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.041+0000 7fecf3baa700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec106fd0 con 0x7fecec1065e0 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.041+0000 7fecf1946700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 0x7fecec106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.042+0000 7fecf1946700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 0x7fecec106a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52942/0 (socket says 172.21.15.46:52942) 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.042+0000 7fecf1946700 1 -- 172.21.15.46:0/865617635 learned_addr learned my addr 172.21.15.46:0/865617635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.042+0000 7fecf1946700 1 -- 172.21.15.46:0/865617635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec107790 con 0x7fecec1065e0 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.042+0000 7fecf1946700 1 --2- 172.21.15.46:0/865617635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 0x7fecec106a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fece80098a0 tx=0x7fece8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404ef9ef00bad4a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.042+0000 7fecf0944700 1 -- 172.21.15.46:0/865617635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fece8004160 con 0x7fecec1065e0 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.042+0000 7fecf0944700 1 -- 172.21.15.46:0/865617635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece80042c0 con 0x7fecec1065e0 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.042+0000 7fecf0944700 1 -- 172.21.15.46:0/865617635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fece80119b0 con 0x7fecec1065e0 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.043+0000 7fecf3baa700 1 -- 172.21.15.46:0/865617635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 msgr2=0x7fecec106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.043+0000 7fecf3baa700 1 --2- 172.21.15.46:0/865617635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 0x7fecec106a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fece80098a0 tx=0x7fece8009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.043+0000 7fecf3baa700 1 -- 172.21.15.46:0/865617635 shutdown_connections 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.043+0000 7fecf3baa700 1 --2- 172.21.15.46:0/865617635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 0x7fecec106a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:27.912 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.043+0000 7fecf3baa700 1 -- 172.21.15.46:0/865617635 >> 172.21.15.46:0/865617635 conn(0x7fecec101be0 msgr2=0x7fecec104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.043+0000 7fecf3baa700 1 -- 172.21.15.46:0/865617635 shutdown_connections 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.043+0000 7fecf3baa700 1 -- 172.21.15.46:0/865617635 wait complete. 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.044+0000 7fecf3baa700 1 Processor -- start 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.044+0000 7fecf3baa700 1 -- start start 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.044+0000 7fecf3baa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 0x7fecec19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.044+0000 7fecf3baa700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec19a870 con 0x7fecec1065e0 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.044+0000 7fecf1946700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 0x7fecec19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.044+0000 7fecf1946700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 0x7fecec19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52954/0 (socket says 172.21.15.46:52954) 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.044+0000 7fecf1946700 1 -- 172.21.15.46:0/3989169965 learned_addr learned my addr 172.21.15.46:0/3989169965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.045+0000 7fecf1946700 1 -- 172.21.15.46:0/3989169965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece8009550 con 0x7fecec1065e0 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.045+0000 7fecf1946700 1 --2- 172.21.15.46:0/3989169965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 0x7fecec19a330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fece8006af0 tx=0x7fece8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.045+0000 7fece2ffd700 1 -- 172.21.15.46:0/3989169965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fece8003a40 con 0x7fecec1065e0 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.045+0000 7fece2ffd700 1 -- 172.21.15.46:0/3989169965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece8003ba0 con 0x7fecec1065e0 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.045+0000 7fece2ffd700 1 -- 172.21.15.46:0/3989169965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fece8019690 con 0x7fecec1065e0 2024-03-18T20:29:27.913 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.045+0000 7fecf3baa700 1 -- 172.21.15.46:0/3989169965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec19aa70 con 0x7fecec1065e0 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.045+0000 7fecf3baa700 1 -- 172.21.15.46:0/3989169965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec19af10 con 0x7fecec1065e0 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.046+0000 7fece2ffd700 1 -- 172.21.15.46:0/3989169965 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fece801aa60 con 0x7fecec1065e0 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.046+0000 7fecf3baa700 1 -- 172.21.15.46:0/3989169965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec10a9c0 con 0x7fecec1065e0 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.046+0000 7fece2ffd700 1 --2- 172.21.15.46:0/3989169965 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fecd8038700 0x7fecd803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.046+0000 7fece2ffd700 1 -- 172.21.15.46:0/3989169965 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fece804c820 con 0x7fecec1065e0 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.047+0000 7fecf1145700 1 --2- 172.21.15.46:0/3989169965 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fecd8038700 0x7fecd803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.047+0000 7fecf1145700 1 --2- 172.21.15.46:0/3989169965 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fecd8038700 0x7fecd803abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fecdc006fd0 tx=0x7fecdc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.052+0000 7fece2ffd700 1 -- 172.21.15.46:0/3989169965 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fece801a430 con 0x7fecec1065e0 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.277+0000 7fecf3baa700 1 -- 172.21.15.46:0/3989169965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fecec10ab50 con 0x7fecec1065e0 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.281+0000 7fece2ffd700 1 -- 172.21.15.46:0/3989169965 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fece800b3e0 con 0x7fecec1065e0 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.283+0000 7fecf3baa700 1 -- 172.21.15.46:0/3989169965 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fecd8038700 msgr2=0x7fecd803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.283+0000 7fecf3baa700 1 --2- 172.21.15.46:0/3989169965 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fecd8038700 0x7fecd803abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fecdc006fd0 tx=0x7fecdc006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.283+0000 7fecf3baa700 1 -- 172.21.15.46:0/3989169965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 msgr2=0x7fecec19a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.283+0000 7fecf3baa700 1 --2- 172.21.15.46:0/3989169965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 0x7fecec19a330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fece8006af0 tx=0x7fece8011a10 comp rx=0 tx=0).stop 2024-03-18T20:29:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.283+0000 7fecf3baa700 1 -- 172.21.15.46:0/3989169965 shutdown_connections 2024-03-18T20:29:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.283+0000 7fecf3baa700 1 --2- 172.21.15.46:0/3989169965 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fecd8038700 0x7fecd803abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.283+0000 7fecf3baa700 1 --2- 172.21.15.46:0/3989169965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecec1065e0 0x7fecec19a330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.283+0000 7fecf3baa700 1 -- 172.21.15.46:0/3989169965 >> 172.21.15.46:0/3989169965 conn(0x7fecec101be0 msgr2=0x7fecec1027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.283+0000 7fecf3baa700 1 -- 172.21.15.46:0/3989169965 shutdown_connections 2024-03-18T20:29:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:27.284+0000 7fecf3baa700 1 -- 172.21.15.46:0/3989169965 wait complete. 2024-03-18T20:29:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling the dashboard module... 2024-03-18T20:29:28.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:28 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3989169965' entity='client.admin' 2024-03-18T20:29:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:29 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2788244321' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-18T20:29:29.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:29 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: ignoring --setuser ceph since I am not root 2024-03-18T20:29:29.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:29 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: ignoring --setgroup ceph since I am not root 2024-03-18T20:29:30.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:29 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:29.778+0000 7f80f795a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T20:29:30.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:29 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:29.882+0000 7f80f795a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T20:29:30.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:29 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:29.979+0000 7f80f795a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T20:29:30.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.284+0000 7f2c409cf700 1 Processor -- start 2024-03-18T20:29:30.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.284+0000 7f2c409cf700 1 -- start start 2024-03-18T20:29:30.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.284+0000 7f2c409cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 0x7f2c3c106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:30.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.285+0000 7f2c409cf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c106fd0 con 0x7f2c3c1065e0 2024-03-18T20:29:30.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.285+0000 7f2c3a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 0x7f2c3c106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:30.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.285+0000 7f2c3a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 0x7f2c3c106a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52964/0 (socket says 172.21.15.46:52964) 2024-03-18T20:29:30.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.285+0000 7f2c3a59c700 1 -- 172.21.15.46:0/1591038351 learned_addr learned my addr 172.21.15.46:0/1591038351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:30.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.285+0000 7f2c3a59c700 1 -- 172.21.15.46:0/1591038351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c107790 con 0x7f2c3c1065e0 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.285+0000 7f2c3a59c700 1 --2- 172.21.15.46:0/1591038351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 0x7f2c3c106a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c240098a0 tx=0x7f2c24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2437b3a4f9080525 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.285+0000 7f2c3959a700 1 -- 172.21.15.46:0/1591038351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c24004160 con 0x7f2c3c1065e0 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.285+0000 7f2c3959a700 1 -- 172.21.15.46:0/1591038351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c240042c0 con 0x7f2c3c1065e0 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.285+0000 7f2c3959a700 1 -- 172.21.15.46:0/1591038351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c240119b0 con 0x7f2c3c1065e0 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.286+0000 7f2c409cf700 1 -- 172.21.15.46:0/1591038351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 msgr2=0x7f2c3c106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.286+0000 7f2c409cf700 1 --2- 172.21.15.46:0/1591038351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 0x7f2c3c106a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c240098a0 tx=0x7f2c24009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.286+0000 7f2c409cf700 1 -- 172.21.15.46:0/1591038351 shutdown_connections 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.286+0000 7f2c409cf700 1 --2- 172.21.15.46:0/1591038351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 0x7f2c3c106a00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.286+0000 7f2c409cf700 1 -- 172.21.15.46:0/1591038351 >> 172.21.15.46:0/1591038351 conn(0x7f2c3c101be0 msgr2=0x7f2c3c104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.286+0000 7f2c409cf700 1 -- 172.21.15.46:0/1591038351 shutdown_connections 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.286+0000 7f2c409cf700 1 -- 172.21.15.46:0/1591038351 wait complete. 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.287+0000 7f2c409cf700 1 Processor -- start 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.287+0000 7f2c409cf700 1 -- start start 2024-03-18T20:29:30.153 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.287+0000 7f2c409cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 0x7f2c3c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.287+0000 7f2c409cf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c19ca60 con 0x7f2c3c1065e0 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.287+0000 7f2c3a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 0x7f2c3c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.287+0000 7f2c3a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 0x7f2c3c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52974/0 (socket says 172.21.15.46:52974) 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.287+0000 7f2c3a59c700 1 -- 172.21.15.46:0/2788244321 learned_addr learned my addr 172.21.15.46:0/2788244321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.288+0000 7f2c3a59c700 1 -- 172.21.15.46:0/2788244321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c24009550 con 0x7f2c3c1065e0 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.288+0000 7f2c3a59c700 1 --2- 172.21.15.46:0/2788244321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 0x7f2c3c19c520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c24003e60 tx=0x7f2c24011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.288+0000 7f2c337fe700 1 -- 172.21.15.46:0/2788244321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c24003a60 con 0x7f2c3c1065e0 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.288+0000 7f2c409cf700 1 -- 172.21.15.46:0/2788244321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3c19cc60 con 0x7f2c3c1065e0 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.288+0000 7f2c337fe700 1 -- 172.21.15.46:0/2788244321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c24003bc0 con 0x7f2c3c1065e0 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.288+0000 7f2c337fe700 1 -- 172.21.15.46:0/2788244321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c240196c0 con 0x7f2c3c1065e0 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.288+0000 7f2c409cf700 1 -- 172.21.15.46:0/2788244321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c19d100 con 0x7f2c3c1065e0 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.289+0000 7f2c337fe700 1 -- 172.21.15.46:0/2788244321 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f2c24019930 con 0x7f2c3c1065e0 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.289+0000 7f2c409cf700 1 -- 172.21.15.46:0/2788244321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3c10a9c0 con 0x7f2c3c1065e0 2024-03-18T20:29:30.154 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.289+0000 7f2c337fe700 1 --2- 172.21.15.46:0/2788244321 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f2c28038730 0x7f2c2803abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.289+0000 7f2c337fe700 1 -- 172.21.15.46:0/2788244321 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2c2404cb70 con 0x7f2c3c1065e0 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.289+0000 7f2c39d9b700 1 --2- 172.21.15.46:0/2788244321 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f2c28038730 0x7f2c2803abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.290+0000 7f2c39d9b700 1 --2- 172.21.15.46:0/2788244321 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f2c28038730 0x7f2c2803abf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c006fd0 tx=0x7f2c2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.294+0000 7f2c337fe700 1 -- 172.21.15.46:0/2788244321 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2c24010de0 con 0x7f2c3c1065e0 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:28.564+0000 7f2c409cf700 1 -- 172.21.15.46:0/2788244321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f2c3c10abe0 con 0x7f2c3c1065e0 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:29.284+0000 7f2c337fe700 1 -- 172.21.15.46:0/2788244321 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 9) v1 ==== 45546+0+0 (secure 0 0 0) 0x7f2c2400fd80 con 0x7f2c3c1065e0 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:29.286+0000 7f2c337fe700 1 -- 172.21.15.46:0/2788244321 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f2c2402bbf0 con 0x7f2c3c1065e0 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:29.289+0000 7f2c409cf700 1 -- 172.21.15.46:0/2788244321 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f2c28038730 msgr2=0x7f2c2803abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:29.289+0000 7f2c409cf700 1 --2- 172.21.15.46:0/2788244321 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f2c28038730 0x7f2c2803abf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c006fd0 tx=0x7f2c2c006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:29.289+0000 7f2c409cf700 1 -- 172.21.15.46:0/2788244321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 msgr2=0x7f2c3c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:29.289+0000 7f2c409cf700 1 --2- 172.21.15.46:0/2788244321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 0x7f2c3c19c520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c24003e60 tx=0x7f2c24011a30 comp rx=0 tx=0).stop 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:29.289+0000 7f2c409cf700 1 -- 172.21.15.46:0/2788244321 shutdown_connections 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:29.289+0000 7f2c409cf700 1 --2- 172.21.15.46:0/2788244321 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7f2c28038730 0x7f2c2803abf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:29.289+0000 7f2c409cf700 1 --2- 172.21.15.46:0/2788244321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c3c1065e0 0x7f2c3c19c520 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:30.156 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:29.289+0000 7f2c409cf700 1 -- 172.21.15.46:0/2788244321 >> 172.21.15.46:0/2788244321 conn(0x7f2c3c101be0 msgr2=0x7f2c3c102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:30.156 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:29.289+0000 7f2c409cf700 1 -- 172.21.15.46:0/2788244321 shutdown_connections 2024-03-18T20:29:30.156 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:29.289+0000 7f2c409cf700 1 -- 172.21.15.46:0/2788244321 wait complete. 2024-03-18T20:29:30.554 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:30 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2788244321' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-18T20:29:30.554 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:30 smithi046 ceph-mon[43495]: mgrmap e9: x(active, since 14s) 2024-03-18T20:29:30.555 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:30 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:30.075+0000 7f80f795a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T20:29:31.114 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:30 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:30.815+0000 7f80f795a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:29:31.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-03-18T20:29:31.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-18T20:29:31.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T20:29:31.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-18T20:29:31.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-18T20:29:31.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-03-18T20:29:31.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.499+0000 7fdef5722700 1 Processor -- start 2024-03-18T20:29:31.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.499+0000 7fdef5722700 1 -- start start 2024-03-18T20:29:31.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.499+0000 7fdef5722700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 0x7fdef0106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:31.194 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.499+0000 7fdef5722700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef0106ff0 con 0x7fdef0106600 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.499+0000 7fdeeeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 0x7fdef0106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.500+0000 7fdeeeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 0x7fdef0106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52990/0 (socket says 172.21.15.46:52990) 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.500+0000 7fdeeeffd700 1 -- 172.21.15.46:0/3991588690 learned_addr learned my addr 172.21.15.46:0/3991588690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.500+0000 7fdeeeffd700 1 -- 172.21.15.46:0/3991588690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef01077b0 con 0x7fdef0106600 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.500+0000 7fdeeeffd700 1 --2- 172.21.15.46:0/3991588690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 0x7fdef0106a20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdee00098a0 tx=0x7fdee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f553d90f52537ac3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.500+0000 7fdeedffb700 1 -- 172.21.15.46:0/3991588690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdee0004160 con 0x7fdef0106600 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.500+0000 7fdeedffb700 1 -- 172.21.15.46:0/3991588690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdee00042c0 con 0x7fdef0106600 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.500+0000 7fdeedffb700 1 -- 172.21.15.46:0/3991588690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdee00119b0 con 0x7fdef0106600 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.501+0000 7fdef5722700 1 -- 172.21.15.46:0/3991588690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 msgr2=0x7fdef0106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.501+0000 7fdef5722700 1 --2- 172.21.15.46:0/3991588690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 0x7fdef0106a20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdee00098a0 tx=0x7fdee0009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.501+0000 7fdef5722700 1 -- 172.21.15.46:0/3991588690 shutdown_connections 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.501+0000 7fdef5722700 1 --2- 172.21.15.46:0/3991588690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 0x7fdef0106a20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.501+0000 7fdef5722700 1 -- 172.21.15.46:0/3991588690 >> 172.21.15.46:0/3991588690 conn(0x7fdef0101be0 msgr2=0x7fdef0104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.501+0000 7fdef5722700 1 -- 172.21.15.46:0/3991588690 shutdown_connections 2024-03-18T20:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.501+0000 7fdef5722700 1 -- 172.21.15.46:0/3991588690 wait complete. 2024-03-18T20:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.502+0000 7fdef5722700 1 Processor -- start 2024-03-18T20:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.502+0000 7fdef5722700 1 -- start start 2024-03-18T20:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.502+0000 7fdef5722700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 0x7fdef019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.502+0000 7fdef5722700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef019a8b0 con 0x7fdef0106600 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.502+0000 7fdeeeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 0x7fdef019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.503+0000 7fdeeeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 0x7fdef019a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53006/0 (socket says 172.21.15.46:53006) 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.503+0000 7fdeeeffd700 1 -- 172.21.15.46:0/2432364591 learned_addr learned my addr 172.21.15.46:0/2432364591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.503+0000 7fdeeeffd700 1 -- 172.21.15.46:0/2432364591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee0009550 con 0x7fdef0106600 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.503+0000 7fdeeeffd700 1 --2- 172.21.15.46:0/2432364591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 0x7fdef019a370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdee0003e60 tx=0x7fdee0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.503+0000 7fded7fff700 1 -- 172.21.15.46:0/2432364591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdee00038b0 con 0x7fdef0106600 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.503+0000 7fded7fff700 1 -- 172.21.15.46:0/2432364591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdee0003a10 con 0x7fdef0106600 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.503+0000 7fdef5722700 1 -- 172.21.15.46:0/2432364591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef019aab0 con 0x7fdef0106600 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.503+0000 7fded7fff700 1 -- 172.21.15.46:0/2432364591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdee0019530 con 0x7fdef0106600 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.503+0000 7fdef5722700 1 -- 172.21.15.46:0/2432364591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef019af50 con 0x7fdef0106600 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.504+0000 7fded7fff700 1 -- 172.21.15.46:0/2432364591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fdee00197f0 con 0x7fdef0106600 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.504+0000 7fdef5722700 1 -- 172.21.15.46:0/2432364591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef010a9e0 con 0x7fdef0106600 2024-03-18T20:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.504+0000 7fded7fff700 1 --2- 172.21.15.46:0/2432364591 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fded8038700 0x7fded803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.505+0000 7fded7fff700 1 -- 172.21.15.46:0/2432364591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdee004c850 con 0x7fdef0106600 2024-03-18T20:29:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.505+0000 7fdeee7fc700 1 -- 172.21.15.46:0/2432364591 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fded8038700 msgr2=0x7fded803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2753187643 2024-03-18T20:29:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.505+0000 7fdeee7fc700 1 --2- 172.21.15.46:0/2432364591 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fded8038700 0x7fded803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T20:29:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.510+0000 7fded7fff700 1 -- 172.21.15.46:0/2432364591 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdee002b340 con 0x7fdef0106600 2024-03-18T20:29:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.706+0000 7fdeee7fc700 1 -- 172.21.15.46:0/2432364591 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fded8038700 msgr2=0x7fded803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2753187643 2024-03-18T20:29:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.706+0000 7fdeee7fc700 1 --2- 172.21.15.46:0/2432364591 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fded8038700 0x7fded803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T20:29:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.824+0000 7fdef5722700 1 -- 172.21.15.46:0/2432364591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fdef019dc00 con 0x7fdef0106600 2024-03-18T20:29:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.825+0000 7fded7fff700 1 -- 172.21.15.46:0/2432364591 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fdee004c1c0 con 0x7fdef0106600 2024-03-18T20:29:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.827+0000 7fdef5722700 1 -- 172.21.15.46:0/2432364591 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fded8038700 msgr2=0x7fded803abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:29:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.827+0000 7fdef5722700 1 --2- 172.21.15.46:0/2432364591 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fded8038700 0x7fded803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.827+0000 7fdef5722700 1 -- 172.21.15.46:0/2432364591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 msgr2=0x7fdef019a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:31.198 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.827+0000 7fdef5722700 1 --2- 172.21.15.46:0/2432364591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 0x7fdef019a370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdee0003e60 tx=0x7fdee0011a30 comp rx=0 tx=0).stop 2024-03-18T20:29:31.199 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.827+0000 7fdef5722700 1 -- 172.21.15.46:0/2432364591 shutdown_connections 2024-03-18T20:29:31.199 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.827+0000 7fdef5722700 1 --2- 172.21.15.46:0/2432364591 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fded8038700 0x7fded803abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:31.199 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.827+0000 7fdef5722700 1 --2- 172.21.15.46:0/2432364591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef0106600 0x7fdef019a370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:31.199 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.827+0000 7fdef5722700 1 -- 172.21.15.46:0/2432364591 >> 172.21.15.46:0/2432364591 conn(0x7fdef0101be0 msgr2=0x7fdef01027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:31.199 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.828+0000 7fdef5722700 1 -- 172.21.15.46:0/2432364591 shutdown_connections 2024-03-18T20:29:31.199 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:30.828+0000 7fdef5722700 1 -- 172.21.15.46:0/2432364591 wait complete. 2024-03-18T20:29:31.199 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-03-18T20:29:31.199 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 9... 2024-03-18T20:29:31.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:31 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2432364591' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-18T20:29:31.537 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:31 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:31.110+0000 7f80f795a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T20:29:31.537 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:31 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:31.209+0000 7f80f795a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T20:29:31.537 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:31 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:31.347+0000 7f80f795a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T20:29:31.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:31 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:31.532+0000 7f80f795a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T20:29:31.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:31 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:31.678+0000 7f80f795a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T20:29:31.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:31 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:31.775+0000 7f80f795a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T20:29:32.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:32 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:32.196+0000 7f80f795a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T20:29:32.963 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:32 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:32.636+0000 7f80f795a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T20:29:32.963 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:32 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:32.853+0000 7f80f795a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T20:29:33.297 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:32 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:32.959+0000 7f80f795a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T20:29:34.106 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:33 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:33.825+0000 7f80f795a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T20:29:34.107 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:33 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:33.934+0000 7f80f795a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T20:29:34.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:34 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:34.102+0000 7f80f795a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T20:29:35.248 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:34 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:34.817+0000 7f80f795a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T20:29:35.248 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:35 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:35.023+0000 7f80f795a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T20:29:35.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:35 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:35.244+0000 7f80f795a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T20:29:36.626 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:36 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:36.300+0000 7f80f795a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T20:29:36.942 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:36 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:36.622+0000 7f80f795a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:29:36.942 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:36 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:36.723+0000 7f80f795a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T20:29:37.297 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:29:36 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:29:36.938+0000 7f80f795a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T20:29:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:37 smithi046 ceph-mon[43495]: Active manager daemon x restarted 2024-03-18T20:29:37.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:37 smithi046 ceph-mon[43495]: Activating manager daemon x 2024-03-18T20:29:37.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:37 smithi046 ceph-mon[43495]: osdmap e3: 0 total, 0 up, 0 in 2024-03-18T20:29:37.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:37 smithi046 ceph-mon[43495]: mgrmap e10: x(active, starting, since 0.0033806s) 2024-03-18T20:29:37.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:29:37.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T20:29:37.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T20:29:37.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T20:29:37.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T20:29:37.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:37 smithi046 ceph-mon[43495]: Manager daemon x is now available 2024-03-18T20:29:37.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:29:37.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:29:38.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T20:29:38.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T20:29:38.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:38.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:38 smithi046 ceph-mon[43495]: mgrmap e11: x(active, since 1.00662s) 2024-03-18T20:29:38.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-03-18T20:29:38.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-18T20:29:38.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-18T20:29:38.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-03-18T20:29:38.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.545+0000 7fde846da700 1 Processor -- start 2024-03-18T20:29:38.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.545+0000 7fde846da700 1 -- start start 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.545+0000 7fde846da700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 0x7fde7c079210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.545+0000 7fde846da700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c0797e0 con 0x7fde7c07adb0 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.545+0000 7fde82476700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 0x7fde7c079210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.546+0000 7fde82476700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 0x7fde7c079210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53010/0 (socket says 172.21.15.46:53010) 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.546+0000 7fde82476700 1 -- 172.21.15.46:0/2762837588 learned_addr learned my addr 172.21.15.46:0/2762837588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.546+0000 7fde82476700 1 -- 172.21.15.46:0/2762837588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7c079920 con 0x7fde7c07adb0 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.546+0000 7fde82476700 1 --2- 172.21.15.46:0/2762837588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 0x7fde7c079210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fde780098a0 tx=0x7fde78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28abef0c3d06ea7c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.546+0000 7fde81474700 1 -- 172.21.15.46:0/2762837588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde78004160 con 0x7fde7c07adb0 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.546+0000 7fde81474700 1 -- 172.21.15.46:0/2762837588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde780042c0 con 0x7fde7c07adb0 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.546+0000 7fde81474700 1 -- 172.21.15.46:0/2762837588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde780119b0 con 0x7fde7c07adb0 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.547+0000 7fde846da700 1 -- 172.21.15.46:0/2762837588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 msgr2=0x7fde7c079210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.547+0000 7fde846da700 1 --2- 172.21.15.46:0/2762837588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 0x7fde7c079210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fde780098a0 tx=0x7fde78009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:38.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.547+0000 7fde846da700 1 -- 172.21.15.46:0/2762837588 shutdown_connections 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.547+0000 7fde846da700 1 --2- 172.21.15.46:0/2762837588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 0x7fde7c079210 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.547+0000 7fde846da700 1 -- 172.21.15.46:0/2762837588 >> 172.21.15.46:0/2762837588 conn(0x7fde7c101b70 msgr2=0x7fde7c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.547+0000 7fde846da700 1 -- 172.21.15.46:0/2762837588 shutdown_connections 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.547+0000 7fde846da700 1 -- 172.21.15.46:0/2762837588 wait complete. 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.548+0000 7fde846da700 1 Processor -- start 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.548+0000 7fde846da700 1 -- start start 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.548+0000 7fde846da700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 0x7fde7c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.548+0000 7fde846da700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c19caa0 con 0x7fde7c07adb0 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.548+0000 7fde82476700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 0x7fde7c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.548+0000 7fde82476700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 0x7fde7c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53026/0 (socket says 172.21.15.46:53026) 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.548+0000 7fde82476700 1 -- 172.21.15.46:0/4253429708 learned_addr learned my addr 172.21.15.46:0/4253429708 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.548+0000 7fde82476700 1 -- 172.21.15.46:0/4253429708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde78009550 con 0x7fde7c07adb0 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.549+0000 7fde82476700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 0x7fde7c19c560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fde78006af0 tx=0x7fde78011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:38.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.549+0000 7fde737fe700 1 -- 172.21.15.46:0/4253429708 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde78003a40 con 0x7fde7c07adb0 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.549+0000 7fde737fe700 1 -- 172.21.15.46:0/4253429708 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde78003ba0 con 0x7fde7c07adb0 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.549+0000 7fde846da700 1 -- 172.21.15.46:0/4253429708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7c19cca0 con 0x7fde7c07adb0 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.549+0000 7fde737fe700 1 -- 172.21.15.46:0/4253429708 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde78019690 con 0x7fde7c07adb0 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.549+0000 7fde846da700 1 -- 172.21.15.46:0/4253429708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7c19d140 con 0x7fde7c07adb0 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.550+0000 7fde737fe700 1 -- 172.21.15.46:0/4253429708 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fde78019950 con 0x7fde7c07adb0 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.550+0000 7fde737fe700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 0x7fde6803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.550+0000 7fde846da700 1 -- 172.21.15.46:0/4253429708 --> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde7c10c9d0 con 0x7fde68038700 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.550+0000 7fde737fe700 1 -- 172.21.15.46:0/4253429708 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fde7804c680 con 0x7fde7c07adb0 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.550+0000 7fde81c75700 1 -- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 msgr2=0x7fde6803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2753187643 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.550+0000 7fde81c75700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 0x7fde6803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.751+0000 7fde81c75700 1 -- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 msgr2=0x7fde6803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2753187643 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:31.751+0000 7fde81c75700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 0x7fde6803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T20:29:38.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:32.152+0000 7fde81c75700 1 -- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 msgr2=0x7fde6803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2753187643 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:32.152+0000 7fde81c75700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 0x7fde6803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:32.953+0000 7fde81c75700 1 -- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 msgr2=0x7fde6803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2753187643 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:32.953+0000 7fde81c75700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 0x7fde6803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:34.554+0000 7fde81c75700 1 -- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 msgr2=0x7fde6803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2753187643 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:34.554+0000 7fde81c75700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 0x7fde6803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:36.954+0000 7fde737fe700 1 -- 172.21.15.46:0/4253429708 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 10) v1 ==== 45313+0+0 (secure 0 0 0) 0x7fde7801a4d0 con 0x7fde7c07adb0 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:36.955+0000 7fde737fe700 1 -- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 msgr2=0x7fde6803abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:36.955+0000 7fde737fe700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 0x7fde6803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.956+0000 7fde737fe700 1 -- 172.21.15.46:0/4253429708 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 11) v1 ==== 45440+0+0 (secure 0 0 0) 0x7fde7804e4a0 con 0x7fde7c07adb0 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.956+0000 7fde737fe700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde6803bfc0 0x7fde6803e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.956+0000 7fde737fe700 1 -- 172.21.15.46:0/4253429708 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde7c10c9d0 con 0x7fde6803bfc0 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.957+0000 7fde81c75700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde6803bfc0 0x7fde6803e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:38.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.957+0000 7fde81c75700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde6803bfc0 0x7fde6803e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fde6c003a10 tx=0x7fde6c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.958+0000 7fde737fe700 1 -- 172.21.15.46:0/4253429708 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fde7c10c9d0 con 0x7fde6803bfc0 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.965+0000 7fde846da700 1 -- 172.21.15.46:0/4253429708 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fde7c10cb40 con 0x7fde6803bfc0 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.965+0000 7fde737fe700 1 -- 172.21.15.46:0/4253429708 <== mgr.14152 v2:172.21.15.46:6800/3400106473 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fde7c10cb40 con 0x7fde6803bfc0 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.966+0000 7fde846da700 1 -- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde6803bfc0 msgr2=0x7fde6803e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.966+0000 7fde846da700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde6803bfc0 0x7fde6803e3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fde6c003a10 tx=0x7fde6c0092b0 comp rx=0 tx=0).stop 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.966+0000 7fde846da700 1 -- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 msgr2=0x7fde7c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.966+0000 7fde846da700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 0x7fde7c19c560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fde78006af0 tx=0x7fde78011a10 comp rx=0 tx=0).stop 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.966+0000 7fde846da700 1 -- 172.21.15.46:0/4253429708 shutdown_connections 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.966+0000 7fde846da700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde6803bfc0 0x7fde6803e3b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.966+0000 7fde846da700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7c07adb0 0x7fde7c19c560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.966+0000 7fde846da700 1 --2- 172.21.15.46:0/4253429708 >> [v2:172.21.15.46:6800/2753187643,v1:172.21.15.46:6801/2753187643] conn(0x7fde68038700 0x7fde6803abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.966+0000 7fde846da700 1 -- 172.21.15.46:0/4253429708 >> 172.21.15.46:0/4253429708 conn(0x7fde7c101b70 msgr2=0x7fde7c102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.966+0000 7fde846da700 1 -- 172.21.15.46:0/4253429708 shutdown_connections 2024-03-18T20:29:38.316 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:37.966+0000 7fde846da700 1 -- 172.21.15.46:0/4253429708 wait complete. 2024-03-18T20:29:38.317 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 9 is available 2024-03-18T20:29:38.317 INFO:teuthology.orchestra.run.smithi046.stdout:Generating a dashboard self-signed certificate... 2024-03-18T20:29:39.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:39 smithi046 ceph-mon[43495]: [18/Mar/2024:20:29:37] ENGINE Bus STARTING 2024-03-18T20:29:39.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:39 smithi046 ceph-mon[43495]: [18/Mar/2024:20:29:37] ENGINE Serving on https://172.21.15.46:7150 2024-03-18T20:29:39.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:39 smithi046 ceph-mon[43495]: [18/Mar/2024:20:29:37] ENGINE Serving on http://172.21.15.46:8765 2024-03-18T20:29:39.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:39 smithi046 ceph-mon[43495]: [18/Mar/2024:20:29:37] ENGINE Bus STARTED 2024-03-18T20:29:39.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:39 smithi046 ceph-mon[43495]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-18T20:29:39.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:39 smithi046 ceph-mon[43495]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-18T20:29:39.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:39 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:39.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-18T20:29:39.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.676+0000 7f16f0e27700 1 Processor -- start 2024-03-18T20:29:39.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.676+0000 7f16f0e27700 1 -- start start 2024-03-18T20:29:39.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.677+0000 7f16f0e27700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 0x7f16ec109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:39.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.677+0000 7f16f0e27700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec079ed0 con 0x7f16ec107040 2024-03-18T20:29:39.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.677+0000 7f16ea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 0x7f16ec109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:39.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.677+0000 7f16ea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 0x7f16ec109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46722/0 (socket says 172.21.15.46:46722) 2024-03-18T20:29:39.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.677+0000 7f16ea59c700 1 -- 172.21.15.46:0/893115528 learned_addr learned my addr 172.21.15.46:0/893115528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:39.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.677+0000 7f16ea59c700 1 -- 172.21.15.46:0/893115528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec109970 con 0x7f16ec107040 2024-03-18T20:29:39.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.678+0000 7f16ea59c700 1 --2- 172.21.15.46:0/893115528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 0x7f16ec109430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0098a0 tx=0x7f16dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275c583a21d7da70 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.678+0000 7f16e959a700 1 -- 172.21.15.46:0/893115528 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16dc004160 con 0x7f16ec107040 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.678+0000 7f16e959a700 1 -- 172.21.15.46:0/893115528 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16dc0042c0 con 0x7f16ec107040 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.678+0000 7f16e959a700 1 -- 172.21.15.46:0/893115528 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16dc0119b0 con 0x7f16ec107040 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.679+0000 7f16f0e27700 1 -- 172.21.15.46:0/893115528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 msgr2=0x7f16ec109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.679+0000 7f16f0e27700 1 --2- 172.21.15.46:0/893115528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 0x7f16ec109430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0098a0 tx=0x7f16dc009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.679+0000 7f16f0e27700 1 -- 172.21.15.46:0/893115528 shutdown_connections 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.679+0000 7f16f0e27700 1 --2- 172.21.15.46:0/893115528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 0x7f16ec109430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.679+0000 7f16f0e27700 1 -- 172.21.15.46:0/893115528 >> 172.21.15.46:0/893115528 conn(0x7f16ec100a70 msgr2=0x7f16ec102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.679+0000 7f16f0e27700 1 -- 172.21.15.46:0/893115528 shutdown_connections 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.679+0000 7f16f0e27700 1 -- 172.21.15.46:0/893115528 wait complete. 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.680+0000 7f16f0e27700 1 Processor -- start 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.680+0000 7f16f0e27700 1 -- start start 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.680+0000 7f16f0e27700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 0x7f16ec19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:39.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.680+0000 7f16f0e27700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec19caf0 con 0x7f16ec107040 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.680+0000 7f16ea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 0x7f16ec19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.680+0000 7f16ea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 0x7f16ec19c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46724/0 (socket says 172.21.15.46:46724) 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.680+0000 7f16ea59c700 1 -- 172.21.15.46:0/1103378415 learned_addr learned my addr 172.21.15.46:0/1103378415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.680+0000 7f16ea59c700 1 -- 172.21.15.46:0/1103378415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc009550 con 0x7f16ec107040 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.681+0000 7f16ea59c700 1 --2- 172.21.15.46:0/1103378415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 0x7f16ec19c5b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f16dc003e60 tx=0x7f16dc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.681+0000 7f16db7fe700 1 -- 172.21.15.46:0/1103378415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16dc0038b0 con 0x7f16ec107040 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.681+0000 7f16db7fe700 1 -- 172.21.15.46:0/1103378415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16dc003a10 con 0x7f16ec107040 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.681+0000 7f16f0e27700 1 -- 172.21.15.46:0/1103378415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec19ccf0 con 0x7f16ec107040 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.681+0000 7f16db7fe700 1 -- 172.21.15.46:0/1103378415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16dc019530 con 0x7f16ec107040 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.681+0000 7f16f0e27700 1 -- 172.21.15.46:0/1103378415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec19d190 con 0x7f16ec107040 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.682+0000 7f16db7fe700 1 -- 172.21.15.46:0/1103378415 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) v1 ==== 45440+0+0 (secure 0 0 0) 0x7f16dc0197f0 con 0x7f16ec107040 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.682+0000 7f16db7fe700 1 --2- 172.21.15.46:0/1103378415 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f16d40385e0 0x7f16d403aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.682+0000 7f16db7fe700 1 -- 172.21.15.46:0/1103378415 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f16dc04d7b0 con 0x7f16ec107040 2024-03-18T20:29:39.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.682+0000 7f16e9d9b700 1 --2- 172.21.15.46:0/1103378415 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f16d40385e0 0x7f16d403aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.682+0000 7f16f0e27700 1 -- 172.21.15.46:0/1103378415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ec0623e0 con 0x7f16ec107040 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.683+0000 7f16e9d9b700 1 --2- 172.21.15.46:0/1103378415 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f16d40385e0 0x7f16d403aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f16e0006fd0 tx=0x7f16e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.688+0000 7f16db7fe700 1 -- 172.21.15.46:0/1103378415 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f16dc019b40 con 0x7f16ec107040 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:38.919+0000 7f16f0e27700 1 -- 172.21.15.46:0/1103378415 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f16ec101cc0 con 0x7f16d40385e0 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.073+0000 7f16db7fe700 1 -- 172.21.15.46:0/1103378415 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f16ec101cc0 con 0x7f16d40385e0 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.076+0000 7f16f0e27700 1 -- 172.21.15.46:0/1103378415 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f16d40385e0 msgr2=0x7f16d403aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.076+0000 7f16f0e27700 1 --2- 172.21.15.46:0/1103378415 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f16d40385e0 0x7f16d403aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f16e0006fd0 tx=0x7f16e0006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.076+0000 7f16f0e27700 1 -- 172.21.15.46:0/1103378415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 msgr2=0x7f16ec19c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.076+0000 7f16f0e27700 1 --2- 172.21.15.46:0/1103378415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 0x7f16ec19c5b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f16dc003e60 tx=0x7f16dc011a30 comp rx=0 tx=0).stop 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.076+0000 7f16f0e27700 1 -- 172.21.15.46:0/1103378415 shutdown_connections 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.076+0000 7f16f0e27700 1 --2- 172.21.15.46:0/1103378415 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f16d40385e0 0x7f16d403aaa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.076+0000 7f16f0e27700 1 --2- 172.21.15.46:0/1103378415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16ec107040 0x7f16ec19c5b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.076+0000 7f16f0e27700 1 -- 172.21.15.46:0/1103378415 >> 172.21.15.46:0/1103378415 conn(0x7f16ec100a70 msgr2=0x7f16ec1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.076+0000 7f16f0e27700 1 -- 172.21.15.46:0/1103378415 shutdown_connections 2024-03-18T20:29:39.497 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.076+0000 7f16f0e27700 1 -- 172.21.15.46:0/1103378415 wait complete. 2024-03-18T20:29:39.497 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial admin user... 2024-03-18T20:29:40.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:40 smithi046 ceph-mon[43495]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:29:40.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:40 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:40.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:40 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:40.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:40 smithi046 ceph-mon[43495]: mgrmap e12: x(active, since 2s) 2024-03-18T20:29:40.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$siYypXtDLiJOYveYhqLIK.yPVrJGokMJ8UuhlwBkOHfTUxI4iSCma", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710793780, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-18T20:29:40.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.923+0000 7fbb557b2700 1 Processor -- start 2024-03-18T20:29:40.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.923+0000 7fbb557b2700 1 -- start start 2024-03-18T20:29:40.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.924+0000 7fbb557b2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 0x7fbb50106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:40.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.924+0000 7fbb557b2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50106ff0 con 0x7fbb50106600 2024-03-18T20:29:40.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.924+0000 7fbb4effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 0x7fbb50106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:40.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.924+0000 7fbb4effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 0x7fbb50106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46730/0 (socket says 172.21.15.46:46730) 2024-03-18T20:29:40.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.924+0000 7fbb4effd700 1 -- 172.21.15.46:0/1987883804 learned_addr learned my addr 172.21.15.46:0/1987883804 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:40.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.924+0000 7fbb4effd700 1 -- 172.21.15.46:0/1987883804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb501077b0 con 0x7fbb50106600 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.924+0000 7fbb4effd700 1 --2- 172.21.15.46:0/1987883804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 0x7fbb50106a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb400098a0 tx=0x7fbb40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51450ac4d27945e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.925+0000 7fbb4dffb700 1 -- 172.21.15.46:0/1987883804 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb40004160 con 0x7fbb50106600 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.925+0000 7fbb4dffb700 1 -- 172.21.15.46:0/1987883804 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb400042c0 con 0x7fbb50106600 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.925+0000 7fbb4dffb700 1 -- 172.21.15.46:0/1987883804 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb400119b0 con 0x7fbb50106600 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.925+0000 7fbb557b2700 1 -- 172.21.15.46:0/1987883804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 msgr2=0x7fbb50106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.925+0000 7fbb557b2700 1 --2- 172.21.15.46:0/1987883804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 0x7fbb50106a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb400098a0 tx=0x7fbb40009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.926+0000 7fbb557b2700 1 -- 172.21.15.46:0/1987883804 shutdown_connections 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.926+0000 7fbb557b2700 1 --2- 172.21.15.46:0/1987883804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 0x7fbb50106a20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.926+0000 7fbb557b2700 1 -- 172.21.15.46:0/1987883804 >> 172.21.15.46:0/1987883804 conn(0x7fbb50101be0 msgr2=0x7fbb50104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.926+0000 7fbb557b2700 1 -- 172.21.15.46:0/1987883804 shutdown_connections 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.926+0000 7fbb557b2700 1 -- 172.21.15.46:0/1987883804 wait complete. 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.926+0000 7fbb557b2700 1 Processor -- start 2024-03-18T20:29:40.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.927+0000 7fbb557b2700 1 -- start start 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.927+0000 7fbb557b2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 0x7fbb5019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.927+0000 7fbb557b2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5019a8d0 con 0x7fbb50106600 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.927+0000 7fbb4effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 0x7fbb5019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.927+0000 7fbb4effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 0x7fbb5019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46732/0 (socket says 172.21.15.46:46732) 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.927+0000 7fbb4effd700 1 -- 172.21.15.46:0/4071735816 learned_addr learned my addr 172.21.15.46:0/4071735816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.927+0000 7fbb4effd700 1 -- 172.21.15.46:0/4071735816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb40009550 con 0x7fbb50106600 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.927+0000 7fbb4effd700 1 --2- 172.21.15.46:0/4071735816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 0x7fbb5019a390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbb40011670 tx=0x7fbb40011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.928+0000 7fbb37fff700 1 -- 172.21.15.46:0/4071735816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb40003af0 con 0x7fbb50106600 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.928+0000 7fbb37fff700 1 -- 172.21.15.46:0/4071735816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb40003c50 con 0x7fbb50106600 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.928+0000 7fbb557b2700 1 -- 172.21.15.46:0/4071735816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5019aad0 con 0x7fbb50106600 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.928+0000 7fbb37fff700 1 -- 172.21.15.46:0/4071735816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb40024820 con 0x7fbb50106600 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.928+0000 7fbb557b2700 1 -- 172.21.15.46:0/4071735816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5019af70 con 0x7fbb50106600 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.929+0000 7fbb37fff700 1 -- 172.21.15.46:0/4071735816 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fbb40024aa0 con 0x7fbb50106600 2024-03-18T20:29:40.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.929+0000 7fbb37fff700 1 --2- 172.21.15.46:0/4071735816 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbb38038750 0x7fbb3803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.929+0000 7fbb557b2700 1 -- 172.21.15.46:0/4071735816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5010a9e0 con 0x7fbb50106600 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.929+0000 7fbb37fff700 1 -- 172.21.15.46:0/4071735816 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbb4004c9a0 con 0x7fbb50106600 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.929+0000 7fbb4e7fc700 1 --2- 172.21.15.46:0/4071735816 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbb38038750 0x7fbb3803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.930+0000 7fbb4e7fc700 1 --2- 172.21.15.46:0/4071735816 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbb38038750 0x7fbb3803ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbb44006fd0 tx=0x7fbb44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:39.934+0000 7fbb37fff700 1 -- 172.21.15.46:0/4071735816 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbb40018ce0 con 0x7fbb50106600 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:40.162+0000 7fbb557b2700 1 -- 172.21.15.46:0/4071735816 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fbb50062200 con 0x7fbb38038750 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:40.415+0000 7fbb37fff700 1 -- 172.21.15.46:0/4071735816 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fbb50062200 con 0x7fbb38038750 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:40.417+0000 7fbb557b2700 1 -- 172.21.15.46:0/4071735816 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbb38038750 msgr2=0x7fbb3803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:40.417+0000 7fbb557b2700 1 --2- 172.21.15.46:0/4071735816 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbb38038750 0x7fbb3803ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbb44006fd0 tx=0x7fbb44006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:40.417+0000 7fbb557b2700 1 -- 172.21.15.46:0/4071735816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 msgr2=0x7fbb5019a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:40.417+0000 7fbb557b2700 1 --2- 172.21.15.46:0/4071735816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 0x7fbb5019a390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbb40011670 tx=0x7fbb40011a30 comp rx=0 tx=0).stop 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:40.417+0000 7fbb557b2700 1 -- 172.21.15.46:0/4071735816 shutdown_connections 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:40.417+0000 7fbb557b2700 1 --2- 172.21.15.46:0/4071735816 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbb38038750 0x7fbb3803ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:40.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:40.417+0000 7fbb557b2700 1 --2- 172.21.15.46:0/4071735816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50106600 0x7fbb5019a390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:40.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:40.417+0000 7fbb557b2700 1 -- 172.21.15.46:0/4071735816 >> 172.21.15.46:0/4071735816 conn(0x7fbb50101be0 msgr2=0x7fbb50102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:40.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:40.418+0000 7fbb557b2700 1 -- 172.21.15.46:0/4071735816 shutdown_connections 2024-03-18T20:29:40.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:40.418+0000 7fbb557b2700 1 -- 172.21.15.46:0/4071735816 wait complete. 2024-03-18T20:29:40.780 INFO:teuthology.orchestra.run.smithi046.stdout:Fetching dashboard port number... 2024-03-18T20:29:41.421 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:41 smithi046 ceph-mon[43495]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:29:41.422 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:41 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:41.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 8443 2024-03-18T20:29:41.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.170+0000 7faf6df50700 1 Processor -- start 2024-03-18T20:29:41.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.170+0000 7faf6df50700 1 -- start start 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.170+0000 7faf6df50700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 0x7faf68109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.170+0000 7faf6df50700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf68079ed0 con 0x7faf68107020 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.170+0000 7faf677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 0x7faf68109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.170+0000 7faf677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 0x7faf68109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46740/0 (socket says 172.21.15.46:46740) 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.170+0000 7faf677fe700 1 -- 172.21.15.46:0/1305851345 learned_addr learned my addr 172.21.15.46:0/1305851345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.171+0000 7faf677fe700 1 -- 172.21.15.46:0/1305851345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf68109950 con 0x7faf68107020 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.171+0000 7faf677fe700 1 --2- 172.21.15.46:0/1305851345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 0x7faf68109410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf580098a0 tx=0x7faf58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d36ffb436c7693 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.171+0000 7faf667fc700 1 -- 172.21.15.46:0/1305851345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf58004160 con 0x7faf68107020 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.171+0000 7faf667fc700 1 -- 172.21.15.46:0/1305851345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf580042c0 con 0x7faf68107020 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.171+0000 7faf667fc700 1 -- 172.21.15.46:0/1305851345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf580119b0 con 0x7faf68107020 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.172+0000 7faf6df50700 1 -- 172.21.15.46:0/1305851345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 msgr2=0x7faf68109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.172+0000 7faf6df50700 1 --2- 172.21.15.46:0/1305851345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 0x7faf68109410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf580098a0 tx=0x7faf58009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:41.777 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.172+0000 7faf6df50700 1 -- 172.21.15.46:0/1305851345 shutdown_connections 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.172+0000 7faf6df50700 1 --2- 172.21.15.46:0/1305851345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 0x7faf68109410 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.172+0000 7faf6df50700 1 -- 172.21.15.46:0/1305851345 >> 172.21.15.46:0/1305851345 conn(0x7faf68100a90 msgr2=0x7faf68102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.172+0000 7faf6df50700 1 -- 172.21.15.46:0/1305851345 shutdown_connections 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.172+0000 7faf6df50700 1 -- 172.21.15.46:0/1305851345 wait complete. 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.172+0000 7faf6df50700 1 Processor -- start 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.173+0000 7faf6df50700 1 -- start start 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.173+0000 7faf6df50700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 0x7faf6819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.173+0000 7faf6df50700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6819a860 con 0x7faf68107020 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.173+0000 7faf677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 0x7faf6819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.173+0000 7faf677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 0x7faf6819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46754/0 (socket says 172.21.15.46:46754) 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.173+0000 7faf677fe700 1 -- 172.21.15.46:0/3648720712 learned_addr learned my addr 172.21.15.46:0/3648720712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.173+0000 7faf677fe700 1 -- 172.21.15.46:0/3648720712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf58009550 con 0x7faf68107020 2024-03-18T20:29:41.778 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.173+0000 7faf677fe700 1 --2- 172.21.15.46:0/3648720712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 0x7faf6819a320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faf58011670 tx=0x7faf58011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.174+0000 7faf64ff9700 1 -- 172.21.15.46:0/3648720712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf58003b20 con 0x7faf68107020 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.174+0000 7faf64ff9700 1 -- 172.21.15.46:0/3648720712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf58003c80 con 0x7faf68107020 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.174+0000 7faf6df50700 1 -- 172.21.15.46:0/3648720712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6819aa60 con 0x7faf68107020 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.174+0000 7faf64ff9700 1 -- 172.21.15.46:0/3648720712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf58019770 con 0x7faf68107020 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.174+0000 7faf6df50700 1 -- 172.21.15.46:0/3648720712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6819af00 con 0x7faf68107020 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.175+0000 7faf64ff9700 1 -- 172.21.15.46:0/3648720712 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7faf5801a430 con 0x7faf68107020 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.175+0000 7faf6df50700 1 -- 172.21.15.46:0/3648720712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6810a790 con 0x7faf68107020 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.175+0000 7faf64ff9700 1 --2- 172.21.15.46:0/3648720712 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faf540387a0 0x7faf5403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.175+0000 7faf64ff9700 1 -- 172.21.15.46:0/3648720712 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faf5804ca50 con 0x7faf68107020 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.175+0000 7faf66ffd700 1 --2- 172.21.15.46:0/3648720712 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faf540387a0 0x7faf5403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.176+0000 7faf66ffd700 1 --2- 172.21.15.46:0/3648720712 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faf540387a0 0x7faf5403ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faf50006fd0 tx=0x7faf50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.180+0000 7faf64ff9700 1 -- 172.21.15.46:0/3648720712 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faf58019a30 con 0x7faf68107020 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.388+0000 7faf6df50700 1 -- 172.21.15.46:0/3648720712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7faf6819db60 con 0x7faf68107020 2024-03-18T20:29:41.779 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.388+0000 7faf64ff9700 1 -- 172.21.15.46:0/3648720712 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7faf5802d350 con 0x7faf68107020 2024-03-18T20:29:41.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.391+0000 7faf6df50700 1 -- 172.21.15.46:0/3648720712 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faf540387a0 msgr2=0x7faf5403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:41.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.391+0000 7faf6df50700 1 --2- 172.21.15.46:0/3648720712 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faf540387a0 0x7faf5403ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faf50006fd0 tx=0x7faf50006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:41.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.391+0000 7faf6df50700 1 -- 172.21.15.46:0/3648720712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 msgr2=0x7faf6819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:41.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.391+0000 7faf6df50700 1 --2- 172.21.15.46:0/3648720712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 0x7faf6819a320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faf58011670 tx=0x7faf58011a30 comp rx=0 tx=0).stop 2024-03-18T20:29:41.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.391+0000 7faf6df50700 1 -- 172.21.15.46:0/3648720712 shutdown_connections 2024-03-18T20:29:41.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.391+0000 7faf6df50700 1 --2- 172.21.15.46:0/3648720712 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faf540387a0 0x7faf5403ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:41.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.391+0000 7faf6df50700 1 --2- 172.21.15.46:0/3648720712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf68107020 0x7faf6819a320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:41.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.391+0000 7faf6df50700 1 -- 172.21.15.46:0/3648720712 >> 172.21.15.46:0/3648720712 conn(0x7faf68100a90 msgr2=0x7faf681015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:41.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.391+0000 7faf6df50700 1 -- 172.21.15.46:0/3648720712 shutdown_connections 2024-03-18T20:29:41.780 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:41.391+0000 7faf6df50700 1 -- 172.21.15.46:0/3648720712 wait complete. 2024-03-18T20:29:41.787 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T20:29:41.787 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-03-18T20:29:41.796 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T20:29:41.796 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-03-18T20:29:41.797 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-03-18T20:29:41.797 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-18T20:29:41.799 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph Dashboard is now available at: 2024-03-18T20:29:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:29:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: URL: https://smithi046:8443/ 2024-03-18T20:29:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: User: admin 2024-03-18T20:29:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: Password: f4lmow6zb0 2024-03-18T20:29:41.799 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:29:41.799 INFO:teuthology.orchestra.run.smithi046.stdout:Saving cluster configuration to /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config directory 2024-03-18T20:29:41.800 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling autotune for osd_memory_target 2024-03-18T20:29:42.439 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:42 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3648720712' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-18T20:29:42.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.181+0000 7f803321f700 1 Processor -- start 2024-03-18T20:29:42.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.181+0000 7f803321f700 1 -- start start 2024-03-18T20:29:42.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.182+0000 7f803321f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 0x7f802c1058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:42.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.182+0000 7f803321f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c105eb0 con 0x7f802c1054c0 2024-03-18T20:29:42.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.182+0000 7f8030fbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 0x7f802c1058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:42.772 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.182+0000 7f8030fbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 0x7f802c1058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46764/0 (socket says 172.21.15.46:46764) 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.182+0000 7f8030fbb700 1 -- 172.21.15.46:0/1162097271 learned_addr learned my addr 172.21.15.46:0/1162097271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.182+0000 7f8030fbb700 1 -- 172.21.15.46:0/1162097271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c1066c0 con 0x7f802c1054c0 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.182+0000 7f8030fbb700 1 --2- 172.21.15.46:0/1162097271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 0x7f802c1058e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f80200098a0 tx=0x7f8020009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44928f6651bff26b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.183+0000 7f802b7fe700 1 -- 172.21.15.46:0/1162097271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8020004160 con 0x7f802c1054c0 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.183+0000 7f802b7fe700 1 -- 172.21.15.46:0/1162097271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80200042c0 con 0x7f802c1054c0 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.183+0000 7f802b7fe700 1 -- 172.21.15.46:0/1162097271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80200119b0 con 0x7f802c1054c0 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.183+0000 7f803321f700 1 -- 172.21.15.46:0/1162097271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 msgr2=0x7f802c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.183+0000 7f803321f700 1 --2- 172.21.15.46:0/1162097271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 0x7f802c1058e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f80200098a0 tx=0x7f8020009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.183+0000 7f803321f700 1 -- 172.21.15.46:0/1162097271 shutdown_connections 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.183+0000 7f803321f700 1 --2- 172.21.15.46:0/1162097271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 0x7f802c1058e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.183+0000 7f803321f700 1 -- 172.21.15.46:0/1162097271 >> 172.21.15.46:0/1162097271 conn(0x7f802c100a80 msgr2=0x7f802c102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.183+0000 7f803321f700 1 -- 172.21.15.46:0/1162097271 shutdown_connections 2024-03-18T20:29:42.773 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.183+0000 7f803321f700 1 -- 172.21.15.46:0/1162097271 wait complete. 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.184+0000 7f803321f700 1 Processor -- start 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.184+0000 7f803321f700 1 -- start start 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.184+0000 7f803321f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 0x7f802c1905e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.184+0000 7f803321f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c190b20 con 0x7f802c1054c0 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.184+0000 7f8030fbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 0x7f802c1905e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.184+0000 7f8030fbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 0x7f802c1905e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46766/0 (socket says 172.21.15.46:46766) 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.184+0000 7f8030fbb700 1 -- 172.21.15.46:0/1163601153 learned_addr learned my addr 172.21.15.46:0/1163601153 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.185+0000 7f8030fbb700 1 -- 172.21.15.46:0/1163601153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8020009550 con 0x7f802c1054c0 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.185+0000 7f8030fbb700 1 --2- 172.21.15.46:0/1163601153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 0x7f802c1905e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8020003e60 tx=0x7f8020011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.185+0000 7f8029ffb700 1 -- 172.21.15.46:0/1163601153 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8020003a60 con 0x7f802c1054c0 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.185+0000 7f8029ffb700 1 -- 172.21.15.46:0/1163601153 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8020003bc0 con 0x7f802c1054c0 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.185+0000 7f8029ffb700 1 -- 172.21.15.46:0/1163601153 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80200196c0 con 0x7f802c1054c0 2024-03-18T20:29:42.774 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.185+0000 7f803321f700 1 -- 172.21.15.46:0/1163601153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c18ed60 con 0x7f802c1054c0 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.185+0000 7f803321f700 1 -- 172.21.15.46:0/1163601153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c18f200 con 0x7f802c1054c0 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.186+0000 7f8029ffb700 1 -- 172.21.15.46:0/1163601153 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7f8020019930 con 0x7f802c1054c0 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.186+0000 7f8029ffb700 1 --2- 172.21.15.46:0/1163601153 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f801c038780 0x7f801c03ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.186+0000 7f8029ffb700 1 -- 172.21.15.46:0/1163601153 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f802004cb70 con 0x7f802c1054c0 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.187+0000 7f802bfff700 1 --2- 172.21.15.46:0/1163601153 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f801c038780 0x7f801c03ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.187+0000 7f803321f700 1 -- 172.21.15.46:0/1163601153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c18fc10 con 0x7f802c1054c0 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.187+0000 7f802bfff700 1 --2- 172.21.15.46:0/1163601153 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f801c038780 0x7f801c03ac40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8018006fd0 tx=0x7f8018006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.192+0000 7f8029ffb700 1 -- 172.21.15.46:0/1163601153 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8020010560 con 0x7f802c1054c0 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.405+0000 7f803321f700 1 -- 172.21.15.46:0/1163601153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f802c0623e0 con 0x7f802c1054c0 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.406+0000 7f8029ffb700 1 -- 172.21.15.46:0/1163601153 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f8020031020 con 0x7f802c1054c0 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.408+0000 7f803321f700 1 -- 172.21.15.46:0/1163601153 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f801c038780 msgr2=0x7f801c03ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.408+0000 7f803321f700 1 --2- 172.21.15.46:0/1163601153 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f801c038780 0x7f801c03ac40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8018006fd0 tx=0x7f8018006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.408+0000 7f803321f700 1 -- 172.21.15.46:0/1163601153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 msgr2=0x7f802c1905e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:42.775 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.408+0000 7f803321f700 1 --2- 172.21.15.46:0/1163601153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 0x7f802c1905e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8020003e60 tx=0x7f8020011a30 comp rx=0 tx=0).stop 2024-03-18T20:29:42.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.408+0000 7f803321f700 1 -- 172.21.15.46:0/1163601153 shutdown_connections 2024-03-18T20:29:42.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.408+0000 7f803321f700 1 --2- 172.21.15.46:0/1163601153 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f801c038780 0x7f801c03ac40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:42.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.408+0000 7f803321f700 1 --2- 172.21.15.46:0/1163601153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f802c1054c0 0x7f802c1905e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:42.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.408+0000 7f803321f700 1 -- 172.21.15.46:0/1163601153 >> 172.21.15.46:0/1163601153 conn(0x7f802c100a80 msgr2=0x7f802c1019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:42.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.408+0000 7f803321f700 1 -- 172.21.15.46:0/1163601153 shutdown_connections 2024-03-18T20:29:42.776 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:42.408+0000 7f803321f700 1 -- 172.21.15.46:0/1163601153 wait complete. 2024-03-18T20:29:43.885 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.174+0000 7f933ad31700 1 Processor -- start 2024-03-18T20:29:43.885 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.174+0000 7f933ad31700 1 -- start start 2024-03-18T20:29:43.885 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.175+0000 7f933ad31700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 0x7f93341052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:43.885 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.175+0000 7f933ad31700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9334079e30 con 0x7f9334104ec0 2024-03-18T20:29:43.885 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.175+0000 7f9338acd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 0x7f93341052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:43.885 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.175+0000 7f9338acd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 0x7f93341052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46780/0 (socket says 172.21.15.46:46780) 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.175+0000 7f9338acd700 1 -- 172.21.15.46:0/3205846945 learned_addr learned my addr 172.21.15.46:0/3205846945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.175+0000 7f9338acd700 1 -- 172.21.15.46:0/3205846945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334079f70 con 0x7f9334104ec0 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.175+0000 7f9338acd700 1 --2- 172.21.15.46:0/3205846945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 0x7f93341052e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f93280098a0 tx=0x7f9328009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6ea754e3a0a02d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.175+0000 7f93337fe700 1 -- 172.21.15.46:0/3205846945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9328004160 con 0x7f9334104ec0 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.176+0000 7f93337fe700 1 -- 172.21.15.46:0/3205846945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93280042c0 con 0x7f9334104ec0 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.176+0000 7f93337fe700 1 -- 172.21.15.46:0/3205846945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93280119b0 con 0x7f9334104ec0 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.176+0000 7f933ad31700 1 -- 172.21.15.46:0/3205846945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 msgr2=0x7f93341052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.176+0000 7f933ad31700 1 --2- 172.21.15.46:0/3205846945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 0x7f93341052e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f93280098a0 tx=0x7f9328009bb0 comp rx=0 tx=0).stop 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.176+0000 7f933ad31700 1 -- 172.21.15.46:0/3205846945 shutdown_connections 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.176+0000 7f933ad31700 1 --2- 172.21.15.46:0/3205846945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 0x7f93341052e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.176+0000 7f933ad31700 1 -- 172.21.15.46:0/3205846945 >> 172.21.15.46:0/3205846945 conn(0x7f9334100a80 msgr2=0x7f9334102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.176+0000 7f933ad31700 1 -- 172.21.15.46:0/3205846945 shutdown_connections 2024-03-18T20:29:43.886 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.176+0000 7f933ad31700 1 -- 172.21.15.46:0/3205846945 wait complete. 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.177+0000 7f933ad31700 1 Processor -- start 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.177+0000 7f933ad31700 1 -- start start 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.177+0000 7f933ad31700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 0x7f933419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.177+0000 7f933ad31700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933419ca60 con 0x7f9334104ec0 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.177+0000 7f9338acd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 0x7f933419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.177+0000 7f9338acd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 0x7f933419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46782/0 (socket says 172.21.15.46:46782) 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.177+0000 7f9338acd700 1 -- 172.21.15.46:0/1274430169 learned_addr learned my addr 172.21.15.46:0/1274430169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.178+0000 7f9338acd700 1 -- 172.21.15.46:0/1274430169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9328009550 con 0x7f9334104ec0 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.178+0000 7f9338acd700 1 --2- 172.21.15.46:0/1274430169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 0x7f933419c520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9328006af0 tx=0x7f9328011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.178+0000 7f9331ffb700 1 -- 172.21.15.46:0/1274430169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9328003a40 con 0x7f9334104ec0 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.178+0000 7f9331ffb700 1 -- 172.21.15.46:0/1274430169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9328003ba0 con 0x7f9334104ec0 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.178+0000 7f933ad31700 1 -- 172.21.15.46:0/1274430169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933419cc60 con 0x7f9334104ec0 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.178+0000 7f9331ffb700 1 -- 172.21.15.46:0/1274430169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93280196c0 con 0x7f9334104ec0 2024-03-18T20:29:43.887 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.178+0000 7f933ad31700 1 -- 172.21.15.46:0/1274430169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933419d100 con 0x7f9334104ec0 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.179+0000 7f9331ffb700 1 -- 172.21.15.46:0/1274430169 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f932801a430 con 0x7f9334104ec0 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.179+0000 7f933ad31700 1 -- 172.21.15.46:0/1274430169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93341961f0 con 0x7f9334104ec0 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.179+0000 7f9331ffb700 1 --2- 172.21.15.46:0/1274430169 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93240387f0 0x7f932403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.180+0000 7f9331ffb700 1 -- 172.21.15.46:0/1274430169 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f932804cb60 con 0x7f9334104ec0 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.180+0000 7f9333fff700 1 --2- 172.21.15.46:0/1274430169 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93240387f0 0x7f932403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.180+0000 7f9333fff700 1 --2- 172.21.15.46:0/1274430169 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93240387f0 0x7f932403acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9320006fd0 tx=0x7f9320006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.185+0000 7f9331ffb700 1 -- 172.21.15.46:0/1274430169 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f932801ac60 con 0x7f9334104ec0 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.516+0000 7f933ad31700 1 -- 172.21.15.46:0/1274430169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f933402ccb0 con 0x7f9334104ec0 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.518+0000 7f9331ffb700 1 -- 172.21.15.46:0/1274430169 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f932800fb60 con 0x7f9334104ec0 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.521+0000 7f933ad31700 1 -- 172.21.15.46:0/1274430169 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93240387f0 msgr2=0x7f932403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.521+0000 7f933ad31700 1 --2- 172.21.15.46:0/1274430169 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93240387f0 0x7f932403acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9320006fd0 tx=0x7f9320006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.521+0000 7f933ad31700 1 -- 172.21.15.46:0/1274430169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 msgr2=0x7f933419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:43.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.521+0000 7f933ad31700 1 --2- 172.21.15.46:0/1274430169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 0x7f933419c520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9328006af0 tx=0x7f9328011a10 comp rx=0 tx=0).stop 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.521+0000 7f933ad31700 1 -- 172.21.15.46:0/1274430169 shutdown_connections 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.521+0000 7f933ad31700 1 --2- 172.21.15.46:0/1274430169 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93240387f0 0x7f932403acb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.521+0000 7f933ad31700 1 --2- 172.21.15.46:0/1274430169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9334104ec0 0x7f933419c520 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.521+0000 7f933ad31700 1 -- 172.21.15.46:0/1274430169 >> 172.21.15.46:0/1274430169 conn(0x7f9334100a80 msgr2=0x7f9334193330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.521+0000 7f933ad31700 1 -- 172.21.15.46:0/1274430169 shutdown_connections 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-03-18T20:29:43.521+0000 7f933ad31700 1 -- 172.21.15.46:0/1274430169 wait complete. 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout:Or, if you are only running a single cluster on this host: 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-18T20:29:43.889 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:29:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-18T20:29:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:29:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: ceph telemetry on 2024-03-18T20:29:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:29:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:For more information see: 2024-03-18T20:29:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:29:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-18T20:29:43.890 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:29:43.890 INFO:teuthology.orchestra.run.smithi046.stdout:Bootstrap complete. 2024-03-18T20:29:43.932 INFO:tasks.cephadm:Fetching config... 2024-03-18T20:29:43.932 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:29:43.932 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-18T20:29:43.952 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-18T20:29:43.952 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:29:43.952 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-18T20:29:44.014 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-18T20:29:44.014 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:29:44.014 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-18T20:29:44.094 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-18T20:29:44.094 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:29:44.094 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-18T20:29:44.156 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-18T20:29:44.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMpsbkDnEggsBzNA/LKu/lCOv9Ylg1Nogzk5RP4zjiZvdhmcqhgPkG62T9xBnCrPYSrtI1vUg5vUGY/2WJZO9kee8ZKDncUDcSXh54wSI/QjKznsKVtpxijKycToAmLWel0ux/CcKAcaqE0WXTW2puFf/wfS62gl5uSmq7nqg2RsdUTR2EyT6mHpav7GuwKbE2eNFmiOwo1rRVUemSq5fI9L88oaGSRmb5gykaR5PPklrgOjzGwJKNKFTL87n4wPUjsqDhQBWUjKlCkssZHrOh9NMqmsAwnSnZDnl/enqLoy/CBr9Dl0oR/RmFtVSTnQbzv+LSx2i0GHwjZte9mCRFfQK0+0QkLW/de0dC750E0X4WzyNwxoYr7vnUXlteFEij+yWVB0mPPPgV56zokJ0PjEvsXglmNPOYnzDAz4yLPWLwqcK33wVnFbpGk3PbO/HWqqw0W6IrXvNDBJyG3gk/AWmvlcNnyEs9FYy/NGfvmy6dL6O2pnKZ3tqDHGt5Ln0= ceph-04a8ae18-e566-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T20:29:44.223 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:44 smithi046 ceph-mon[43495]: mgrmap e13: x(active, since 6s) 2024-03-18T20:29:44.223 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:44 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1274430169' entity='client.admin' 2024-03-18T20:29:44.239 INFO:teuthology.orchestra.run.smithi046.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMpsbkDnEggsBzNA/LKu/lCOv9Ylg1Nogzk5RP4zjiZvdhmcqhgPkG62T9xBnCrPYSrtI1vUg5vUGY/2WJZO9kee8ZKDncUDcSXh54wSI/QjKznsKVtpxijKycToAmLWel0ux/CcKAcaqE0WXTW2puFf/wfS62gl5uSmq7nqg2RsdUTR2EyT6mHpav7GuwKbE2eNFmiOwo1rRVUemSq5fI9L88oaGSRmb5gykaR5PPklrgOjzGwJKNKFTL87n4wPUjsqDhQBWUjKlCkssZHrOh9NMqmsAwnSnZDnl/enqLoy/CBr9Dl0oR/RmFtVSTnQbzv+LSx2i0GHwjZte9mCRFfQK0+0QkLW/de0dC750E0X4WzyNwxoYr7vnUXlteFEij+yWVB0mPPPgV56zokJ0PjEvsXglmNPOYnzDAz4yLPWLwqcK33wVnFbpGk3PbO/HWqqw0W6IrXvNDBJyG3gk/AWmvlcNnyEs9FYy/NGfvmy6dL6O2pnKZ3tqDHGt5Ln0= ceph-04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:29:44.253 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMpsbkDnEggsBzNA/LKu/lCOv9Ylg1Nogzk5RP4zjiZvdhmcqhgPkG62T9xBnCrPYSrtI1vUg5vUGY/2WJZO9kee8ZKDncUDcSXh54wSI/QjKznsKVtpxijKycToAmLWel0ux/CcKAcaqE0WXTW2puFf/wfS62gl5uSmq7nqg2RsdUTR2EyT6mHpav7GuwKbE2eNFmiOwo1rRVUemSq5fI9L88oaGSRmb5gykaR5PPklrgOjzGwJKNKFTL87n4wPUjsqDhQBWUjKlCkssZHrOh9NMqmsAwnSnZDnl/enqLoy/CBr9Dl0oR/RmFtVSTnQbzv+LSx2i0GHwjZte9mCRFfQK0+0QkLW/de0dC750E0X4WzyNwxoYr7vnUXlteFEij+yWVB0mPPPgV56zokJ0PjEvsXglmNPOYnzDAz4yLPWLwqcK33wVnFbpGk3PbO/HWqqw0W6IrXvNDBJyG3gk/AWmvlcNnyEs9FYy/NGfvmy6dL6O2pnKZ3tqDHGt5Ln0= ceph-04a8ae18-e566-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T20:29:44.298 INFO:teuthology.orchestra.run.smithi073.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMpsbkDnEggsBzNA/LKu/lCOv9Ylg1Nogzk5RP4zjiZvdhmcqhgPkG62T9xBnCrPYSrtI1vUg5vUGY/2WJZO9kee8ZKDncUDcSXh54wSI/QjKznsKVtpxijKycToAmLWel0ux/CcKAcaqE0WXTW2puFf/wfS62gl5uSmq7nqg2RsdUTR2EyT6mHpav7GuwKbE2eNFmiOwo1rRVUemSq5fI9L88oaGSRmb5gykaR5PPklrgOjzGwJKNKFTL87n4wPUjsqDhQBWUjKlCkssZHrOh9NMqmsAwnSnZDnl/enqLoy/CBr9Dl0oR/RmFtVSTnQbzv+LSx2i0GHwjZte9mCRFfQK0+0QkLW/de0dC750E0X4WzyNwxoYr7vnUXlteFEij+yWVB0mPPPgV56zokJ0PjEvsXglmNPOYnzDAz4yLPWLwqcK33wVnFbpGk3PbO/HWqqw0W6IrXvNDBJyG3gk/AWmvlcNnyEs9FYy/NGfvmy6dL6O2pnKZ3tqDHGt5Ln0= ceph-04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:29:44.311 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMpsbkDnEggsBzNA/LKu/lCOv9Ylg1Nogzk5RP4zjiZvdhmcqhgPkG62T9xBnCrPYSrtI1vUg5vUGY/2WJZO9kee8ZKDncUDcSXh54wSI/QjKznsKVtpxijKycToAmLWel0ux/CcKAcaqE0WXTW2puFf/wfS62gl5uSmq7nqg2RsdUTR2EyT6mHpav7GuwKbE2eNFmiOwo1rRVUemSq5fI9L88oaGSRmb5gykaR5PPklrgOjzGwJKNKFTL87n4wPUjsqDhQBWUjKlCkssZHrOh9NMqmsAwnSnZDnl/enqLoy/CBr9Dl0oR/RmFtVSTnQbzv+LSx2i0GHwjZte9mCRFfQK0+0QkLW/de0dC750E0X4WzyNwxoYr7vnUXlteFEij+yWVB0mPPPgV56zokJ0PjEvsXglmNPOYnzDAz4yLPWLwqcK33wVnFbpGk3PbO/HWqqw0W6IrXvNDBJyG3gk/AWmvlcNnyEs9FYy/NGfvmy6dL6O2pnKZ3tqDHGt5Ln0= ceph-04a8ae18-e566-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T20:29:44.358 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMpsbkDnEggsBzNA/LKu/lCOv9Ylg1Nogzk5RP4zjiZvdhmcqhgPkG62T9xBnCrPYSrtI1vUg5vUGY/2WJZO9kee8ZKDncUDcSXh54wSI/QjKznsKVtpxijKycToAmLWel0ux/CcKAcaqE0WXTW2puFf/wfS62gl5uSmq7nqg2RsdUTR2EyT6mHpav7GuwKbE2eNFmiOwo1rRVUemSq5fI9L88oaGSRmb5gykaR5PPklrgOjzGwJKNKFTL87n4wPUjsqDhQBWUjKlCkssZHrOh9NMqmsAwnSnZDnl/enqLoy/CBr9Dl0oR/RmFtVSTnQbzv+LSx2i0GHwjZte9mCRFfQK0+0QkLW/de0dC750E0X4WzyNwxoYr7vnUXlteFEij+yWVB0mPPPgV56zokJ0PjEvsXglmNPOYnzDAz4yLPWLwqcK33wVnFbpGk3PbO/HWqqw0W6IrXvNDBJyG3gk/AWmvlcNnyEs9FYy/NGfvmy6dL6O2pnKZ3tqDHGt5Ln0= ceph-04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:29:44.371 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-18T20:29:44.622 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:29:45.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.788+0000 7f7a8d774700 1 -- 172.21.15.46:0/2033927771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a880ff630 msgr2=0x7f7a880ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:45.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.788+0000 7f7a8d774700 1 --2- 172.21.15.46:0/2033927771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a880ff630 0x7f7a880ffa50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009910 tx=0x7f7a78009c20 comp rx=0 tx=0).stop 2024-03-18T20:29:45.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.788+0000 7f7a8d774700 1 -- 172.21.15.46:0/2033927771 shutdown_connections 2024-03-18T20:29:45.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.788+0000 7f7a8d774700 1 --2- 172.21.15.46:0/2033927771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a880ff630 0x7f7a880ffa50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:45.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.788+0000 7f7a8d774700 1 -- 172.21.15.46:0/2033927771 >> 172.21.15.46:0/2033927771 conn(0x7f7a880fabd0 msgr2=0x7f7a880fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:45.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.788+0000 7f7a8d774700 1 -- 172.21.15.46:0/2033927771 shutdown_connections 2024-03-18T20:29:45.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.788+0000 7f7a8d774700 1 -- 172.21.15.46:0/2033927771 wait complete. 2024-03-18T20:29:45.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.789+0000 7f7a8d774700 1 Processor -- start 2024-03-18T20:29:45.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.789+0000 7f7a8d774700 1 -- start start 2024-03-18T20:29:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.790+0000 7f7a8d774700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a880ff630 0x7f7a880fca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.790+0000 7f7a8d774700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880fcf80 con 0x7f7a880ff630 2024-03-18T20:29:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.790+0000 7f7a86ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a880ff630 0x7f7a880fca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.790+0000 7f7a86ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a880ff630 0x7f7a880fca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55934/0 (socket says 172.21.15.46:55934) 2024-03-18T20:29:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.790+0000 7f7a86ffd700 1 -- 172.21.15.46:0/368996835 learned_addr learned my addr 172.21.15.46:0/368996835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.790+0000 7f7a86ffd700 1 -- 172.21.15.46:0/368996835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a780095f0 con 0x7f7a880ff630 2024-03-18T20:29:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.790+0000 7f7a86ffd700 1 --2- 172.21.15.46:0/368996835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a880ff630 0x7f7a880fca40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7a78004f70 tx=0x7f7a78004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.790+0000 7f7a6ffff700 1 -- 172.21.15.46:0/368996835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a7801d070 con 0x7f7a880ff630 2024-03-18T20:29:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.790+0000 7f7a6ffff700 1 -- 172.21.15.46:0/368996835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a7800be20 con 0x7f7a880ff630 2024-03-18T20:29:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.790+0000 7f7a8d774700 1 -- 172.21.15.46:0/368996835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a880fb1c0 con 0x7f7a880ff630 2024-03-18T20:29:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.790+0000 7f7a6ffff700 1 -- 172.21.15.46:0/368996835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a78022df0 con 0x7f7a880ff630 2024-03-18T20:29:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.790+0000 7f7a8d774700 1 -- 172.21.15.46:0/368996835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a880fb660 con 0x7f7a880ff630 2024-03-18T20:29:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.791+0000 7f7a8d774700 1 -- 172.21.15.46:0/368996835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a88105c80 con 0x7f7a880ff630 2024-03-18T20:29:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.792+0000 7f7a6ffff700 1 -- 172.21.15.46:0/368996835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f7a7800b960 con 0x7f7a880ff630 2024-03-18T20:29:45.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.792+0000 7f7a6ffff700 1 --2- 172.21.15.46:0/368996835 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7a70038780 0x7f7a7003ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:45.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.792+0000 7f7a6ffff700 1 -- 172.21.15.46:0/368996835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7a7804d100 con 0x7f7a880ff630 2024-03-18T20:29:45.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.792+0000 7f7a867fc700 1 --2- 172.21.15.46:0/368996835 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7a70038780 0x7f7a7003ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:45.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.793+0000 7f7a867fc700 1 --2- 172.21.15.46:0/368996835 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7a70038780 0x7f7a7003ac40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c006fd0 tx=0x7f7a7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:45.797+0000 7f7a6ffff700 1 -- 172.21.15.46:0/368996835 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7a7802bba0 con 0x7f7a880ff630 2024-03-18T20:29:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:46.010+0000 7f7a8d774700 1 -- 172.21.15.46:0/368996835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f7a880623e0 con 0x7f7a880ff630 2024-03-18T20:29:46.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:46.014+0000 7f7a6ffff700 1 -- 172.21.15.46:0/368996835 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7a78027030 con 0x7f7a880ff630 2024-03-18T20:29:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:46.016+0000 7f7a8d774700 1 -- 172.21.15.46:0/368996835 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7a70038780 msgr2=0x7f7a7003ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:46.016+0000 7f7a8d774700 1 --2- 172.21.15.46:0/368996835 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7a70038780 0x7f7a7003ac40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c006fd0 tx=0x7f7a7c006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:46.016+0000 7f7a8d774700 1 -- 172.21.15.46:0/368996835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a880ff630 msgr2=0x7f7a880fca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:46.016+0000 7f7a8d774700 1 --2- 172.21.15.46:0/368996835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a880ff630 0x7f7a880fca40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7a78004f70 tx=0x7f7a78004fa0 comp rx=0 tx=0).stop 2024-03-18T20:29:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:46.016+0000 7f7a8d774700 1 -- 172.21.15.46:0/368996835 shutdown_connections 2024-03-18T20:29:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:46.016+0000 7f7a8d774700 1 --2- 172.21.15.46:0/368996835 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7a70038780 0x7f7a7003ac40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:46.016+0000 7f7a8d774700 1 --2- 172.21.15.46:0/368996835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a880ff630 0x7f7a880fca40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:46.016+0000 7f7a8d774700 1 -- 172.21.15.46:0/368996835 >> 172.21.15.46:0/368996835 conn(0x7f7a880fabd0 msgr2=0x7f7a88069380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:46.016+0000 7f7a8d774700 1 -- 172.21.15.46:0/368996835 shutdown_connections 2024-03-18T20:29:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:46.016+0000 7f7a8d774700 1 -- 172.21.15.46:0/368996835 wait complete. 2024-03-18T20:29:46.463 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-18T20:29:46.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-18T20:29:46.717 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:29:47.185 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:47 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/368996835' entity='client.admin' 2024-03-18T20:29:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.808+0000 7f1a59576700 1 -- 172.21.15.46:0/2405950907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a54100720 msgr2=0x7f1a54100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.808+0000 7f1a59576700 1 --2- 172.21.15.46:0/2405950907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a54100720 0x7f1a54100b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009910 tx=0x7f1a44009c20 comp rx=0 tx=0).stop 2024-03-18T20:29:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.808+0000 7f1a59576700 1 -- 172.21.15.46:0/2405950907 shutdown_connections 2024-03-18T20:29:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.808+0000 7f1a59576700 1 --2- 172.21.15.46:0/2405950907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a54100720 0x7f1a54100b40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.808+0000 7f1a59576700 1 -- 172.21.15.46:0/2405950907 >> 172.21.15.46:0/2405950907 conn(0x7f1a540fbcc0 msgr2=0x7f1a540fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.808+0000 7f1a59576700 1 -- 172.21.15.46:0/2405950907 shutdown_connections 2024-03-18T20:29:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.808+0000 7f1a59576700 1 -- 172.21.15.46:0/2405950907 wait complete. 2024-03-18T20:29:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.809+0000 7f1a59576700 1 Processor -- start 2024-03-18T20:29:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.809+0000 7f1a59576700 1 -- start start 2024-03-18T20:29:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.809+0000 7f1a59576700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a54100720 0x7f1a54195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.809+0000 7f1a59576700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a541960e0 con 0x7f1a54100720 2024-03-18T20:29:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.809+0000 7f1a52ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a54100720 0x7f1a54195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.809+0000 7f1a52ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a54100720 0x7f1a54195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55960/0 (socket says 172.21.15.46:55960) 2024-03-18T20:29:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.809+0000 7f1a52ffd700 1 -- 172.21.15.46:0/2581068237 learned_addr learned my addr 172.21.15.46:0/2581068237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.809+0000 7f1a52ffd700 1 -- 172.21.15.46:0/2581068237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a440095f0 con 0x7f1a54100720 2024-03-18T20:29:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.810+0000 7f1a52ffd700 1 --2- 172.21.15.46:0/2581068237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a54100720 0x7f1a54195ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005950 tx=0x7f1a440057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.810+0000 7f1a3bfff700 1 -- 172.21.15.46:0/2581068237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a4401d070 con 0x7f1a54100720 2024-03-18T20:29:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.810+0000 7f1a59576700 1 -- 172.21.15.46:0/2581068237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a541962e0 con 0x7f1a54100720 2024-03-18T20:29:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.810+0000 7f1a3bfff700 1 -- 172.21.15.46:0/2581068237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a4400bc10 con 0x7f1a54100720 2024-03-18T20:29:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.810+0000 7f1a3bfff700 1 -- 172.21.15.46:0/2581068237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a44003a10 con 0x7f1a54100720 2024-03-18T20:29:47.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.810+0000 7f1a59576700 1 -- 172.21.15.46:0/2581068237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a54196780 con 0x7f1a54100720 2024-03-18T20:29:47.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.811+0000 7f1a3bfff700 1 -- 172.21.15.46:0/2581068237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f1a44003cd0 con 0x7f1a54100720 2024-03-18T20:29:47.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.811+0000 7f1a3bfff700 1 --2- 172.21.15.46:0/2581068237 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1a3c0387a0 0x7f1a3c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:47.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.811+0000 7f1a3bfff700 1 -- 172.21.15.46:0/2581068237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1a4404c550 con 0x7f1a54100720 2024-03-18T20:29:47.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.811+0000 7f1a527fc700 1 --2- 172.21.15.46:0/2581068237 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1a3c0387a0 0x7f1a3c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:47.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.811+0000 7f1a59576700 1 -- 172.21.15.46:0/2581068237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5418f440 con 0x7f1a54100720 2024-03-18T20:29:47.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.811+0000 7f1a527fc700 1 --2- 172.21.15.46:0/2581068237 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1a3c0387a0 0x7f1a3c03ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1a48006fd0 tx=0x7f1a48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:47.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:47.816+0000 7f1a3bfff700 1 -- 172.21.15.46:0/2581068237 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1a44026070 con 0x7f1a54100720 2024-03-18T20:29:48.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:48.031+0000 7f1a59576700 1 -- 172.21.15.46:0/2581068237 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a5402cf90 con 0x7f1a3c0387a0 2024-03-18T20:29:48.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:48.035+0000 7f1a3bfff700 1 -- 172.21.15.46:0/2581068237 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1a5402cf90 con 0x7f1a3c0387a0 2024-03-18T20:29:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:48.037+0000 7f1a59576700 1 -- 172.21.15.46:0/2581068237 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1a3c0387a0 msgr2=0x7f1a3c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:48.037+0000 7f1a59576700 1 --2- 172.21.15.46:0/2581068237 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1a3c0387a0 0x7f1a3c03ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1a48006fd0 tx=0x7f1a48006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:48.037+0000 7f1a59576700 1 -- 172.21.15.46:0/2581068237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a54100720 msgr2=0x7f1a54195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:48.037+0000 7f1a59576700 1 --2- 172.21.15.46:0/2581068237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a54100720 0x7f1a54195ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005950 tx=0x7f1a440057c0 comp rx=0 tx=0).stop 2024-03-18T20:29:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:48.037+0000 7f1a59576700 1 -- 172.21.15.46:0/2581068237 shutdown_connections 2024-03-18T20:29:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:48.037+0000 7f1a59576700 1 --2- 172.21.15.46:0/2581068237 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1a3c0387a0 0x7f1a3c03ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:48.037+0000 7f1a59576700 1 --2- 172.21.15.46:0/2581068237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a54100720 0x7f1a54195ba0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:48.037+0000 7f1a59576700 1 -- 172.21.15.46:0/2581068237 >> 172.21.15.46:0/2581068237 conn(0x7f1a540fbcc0 msgr2=0x7f1a540fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:48.037+0000 7f1a59576700 1 -- 172.21.15.46:0/2581068237 shutdown_connections 2024-03-18T20:29:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:48.038+0000 7f1a59576700 1 -- 172.21.15.46:0/2581068237 wait complete. 2024-03-18T20:29:48.425 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi073 2024-03-18T20:29:48.425 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-18T20:29:48.426 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/etc/ceph/ceph.conf 2024-03-18T20:29:48.447 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-18T20:29:48.447 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:29:48.507 INFO:tasks.cephadm:Adding host smithi073 to orchestrator... 2024-03-18T20:29:48.508 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch host add smithi073 2024-03-18T20:29:48.764 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:29:48.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:48.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:48.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi046", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:29:48.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:48.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:29:48.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:48.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:29:48.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:48.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:29:48.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:29:48.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:29:48.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: Updating smithi046:/etc/ceph/ceph.conf 2024-03-18T20:29:48.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:48 smithi046 ceph-mon[43495]: Updating smithi046:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:29:50.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.074+0000 7f91de49d700 1 -- 172.21.15.46:0/956049111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91d81066c0 msgr2=0x7f91d8106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:50.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.074+0000 7f91de49d700 1 --2- 172.21.15.46:0/956049111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91d81066c0 0x7f91d8106aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009910 tx=0x7f91c8009c20 comp rx=0 tx=0).stop 2024-03-18T20:29:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.074+0000 7f91de49d700 1 -- 172.21.15.46:0/956049111 shutdown_connections 2024-03-18T20:29:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.074+0000 7f91de49d700 1 --2- 172.21.15.46:0/956049111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91d81066c0 0x7f91d8106aa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.074+0000 7f91de49d700 1 -- 172.21.15.46:0/956049111 >> 172.21.15.46:0/956049111 conn(0x7f91d80fc1b0 msgr2=0x7f91d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.074+0000 7f91de49d700 1 -- 172.21.15.46:0/956049111 shutdown_connections 2024-03-18T20:29:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.074+0000 7f91de49d700 1 -- 172.21.15.46:0/956049111 wait complete. 2024-03-18T20:29:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.075+0000 7f91de49d700 1 Processor -- start 2024-03-18T20:29:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.075+0000 7f91de49d700 1 -- start start 2024-03-18T20:29:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.075+0000 7f91de49d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91d81066c0 0x7f91d8195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.075+0000 7f91de49d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d81964a0 con 0x7f91d81066c0 2024-03-18T20:29:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.075+0000 7f91d7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91d81066c0 0x7f91d8195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.075+0000 7f91d7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91d81066c0 0x7f91d8195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55982/0 (socket says 172.21.15.46:55982) 2024-03-18T20:29:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.075+0000 7f91d7fff700 1 -- 172.21.15.46:0/2614239763 learned_addr learned my addr 172.21.15.46:0/2614239763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.076+0000 7f91d7fff700 1 -- 172.21.15.46:0/2614239763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c80095f0 con 0x7f91d81066c0 2024-03-18T20:29:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.076+0000 7f91d7fff700 1 --2- 172.21.15.46:0/2614239763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91d81066c0 0x7f91d8195f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006010 tx=0x7f91c8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.076+0000 7f91d57fa700 1 -- 172.21.15.46:0/2614239763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91c801d070 con 0x7f91d81066c0 2024-03-18T20:29:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.076+0000 7f91de49d700 1 -- 172.21.15.46:0/2614239763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d8196700 con 0x7f91d81066c0 2024-03-18T20:29:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.076+0000 7f91d57fa700 1 -- 172.21.15.46:0/2614239763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91c800bbb0 con 0x7f91d81066c0 2024-03-18T20:29:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.076+0000 7f91d57fa700 1 -- 172.21.15.46:0/2614239763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91c8003810 con 0x7f91d81066c0 2024-03-18T20:29:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.076+0000 7f91de49d700 1 -- 172.21.15.46:0/2614239763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d8196b80 con 0x7f91d81066c0 2024-03-18T20:29:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.077+0000 7f91d57fa700 1 -- 172.21.15.46:0/2614239763 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f91c8003a10 con 0x7f91d81066c0 2024-03-18T20:29:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.077+0000 7f91d57fa700 1 --2- 172.21.15.46:0/2614239763 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f91c00387a0 0x7f91c003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.077+0000 7f91de49d700 1 -- 172.21.15.46:0/2614239763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d8101e00 con 0x7f91d81066c0 2024-03-18T20:29:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.077+0000 7f91d57fa700 1 -- 172.21.15.46:0/2614239763 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f91c804c3d0 con 0x7f91d81066c0 2024-03-18T20:29:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.077+0000 7f91d77fe700 1 --2- 172.21.15.46:0/2614239763 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f91c00387a0 0x7f91c003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.078+0000 7f91d77fe700 1 --2- 172.21.15.46:0/2614239763 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f91c00387a0 0x7f91c003ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f91cc006fd0 tx=0x7f91cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.082+0000 7f91d57fa700 1 -- 172.21.15.46:0/2614239763 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f91c8026020 con 0x7f91d81066c0 2024-03-18T20:29:50.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:50.299+0000 7f91de49d700 1 -- 172.21.15.46:0/2614239763 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi073", "target": ["mon-mgr", ""]}) v1 -- 0x7f91d8108a00 con 0x7f91c00387a0 2024-03-18T20:29:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:50 smithi046 ceph-mon[43495]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:29:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:50 smithi046 ceph-mon[43495]: Updating smithi046:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:29:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:51 smithi046 ceph-mon[43495]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi073", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:29:52.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:52 smithi046 ceph-mon[43495]: Deploying cephadm binary to smithi073 2024-03-18T20:29:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:52.613+0000 7f91d57fa700 1 -- 172.21.15.46:0/2614239763 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f91d8108a00 con 0x7f91c00387a0 2024-03-18T20:29:52.618 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi073' with addr '172.21.15.73' 2024-03-18T20:29:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:52.616+0000 7f91de49d700 1 -- 172.21.15.46:0/2614239763 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f91c00387a0 msgr2=0x7f91c003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:52.616+0000 7f91de49d700 1 --2- 172.21.15.46:0/2614239763 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f91c00387a0 0x7f91c003ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f91cc006fd0 tx=0x7f91cc006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:52.616+0000 7f91de49d700 1 -- 172.21.15.46:0/2614239763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91d81066c0 msgr2=0x7f91d8195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:52.616+0000 7f91de49d700 1 --2- 172.21.15.46:0/2614239763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91d81066c0 0x7f91d8195f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006010 tx=0x7f91c8004ff0 comp rx=0 tx=0).stop 2024-03-18T20:29:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:52.616+0000 7f91de49d700 1 -- 172.21.15.46:0/2614239763 shutdown_connections 2024-03-18T20:29:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:52.616+0000 7f91de49d700 1 --2- 172.21.15.46:0/2614239763 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f91c00387a0 0x7f91c003ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:52.616+0000 7f91de49d700 1 --2- 172.21.15.46:0/2614239763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91d81066c0 0x7f91d8195f60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:52.616+0000 7f91de49d700 1 -- 172.21.15.46:0/2614239763 >> 172.21.15.46:0/2614239763 conn(0x7f91d80fc1b0 msgr2=0x7f91d81082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:52.616+0000 7f91de49d700 1 -- 172.21.15.46:0/2614239763 shutdown_connections 2024-03-18T20:29:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:52.616+0000 7f91de49d700 1 -- 172.21.15.46:0/2614239763 wait complete. 2024-03-18T20:29:53.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-18T20:29:53.360 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:29:53.767 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:53.767 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:53 smithi046 ceph-mon[43495]: Added host smithi073 2024-03-18T20:29:53.767 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:29:53.767 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:53.767 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.464+0000 7f833096f700 1 -- 172.21.15.46:0/2326290042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105630 msgr2=0x7f832c105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.464+0000 7f833096f700 1 --2- 172.21.15.46:0/2326290042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105630 0x7f832c105a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f831c009910 tx=0x7f831c009c20 comp rx=0 tx=0).stop 2024-03-18T20:29:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.464+0000 7f833096f700 1 -- 172.21.15.46:0/2326290042 shutdown_connections 2024-03-18T20:29:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.464+0000 7f833096f700 1 --2- 172.21.15.46:0/2326290042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105630 0x7f832c105a10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.464+0000 7f833096f700 1 -- 172.21.15.46:0/2326290042 >> 172.21.15.46:0/2326290042 conn(0x7f832c075110 msgr2=0x7f832c075520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.464+0000 7f833096f700 1 -- 172.21.15.46:0/2326290042 shutdown_connections 2024-03-18T20:29:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.464+0000 7f833096f700 1 -- 172.21.15.46:0/2326290042 wait complete. 2024-03-18T20:29:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.465+0000 7f833096f700 1 Processor -- start 2024-03-18T20:29:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.465+0000 7f833096f700 1 -- start start 2024-03-18T20:29:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.466+0000 7f833096f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105630 0x7f832c195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.466+0000 7f833096f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c196470 con 0x7f832c105630 2024-03-18T20:29:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.466+0000 7f832a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105630 0x7f832c195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.466+0000 7f832a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105630 0x7f832c195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56008/0 (socket says 172.21.15.46:56008) 2024-03-18T20:29:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.466+0000 7f832a59c700 1 -- 172.21.15.46:0/795487102 learned_addr learned my addr 172.21.15.46:0/795487102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.466+0000 7f832a59c700 1 -- 172.21.15.46:0/795487102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c0095f0 con 0x7f832c105630 2024-03-18T20:29:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.466+0000 7f832a59c700 1 --2- 172.21.15.46:0/795487102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105630 0x7f832c195f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f831c006010 tx=0x7f831c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.466+0000 7f831b7fe700 1 -- 172.21.15.46:0/795487102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f831c01d070 con 0x7f832c105630 2024-03-18T20:29:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.466+0000 7f831b7fe700 1 -- 172.21.15.46:0/795487102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f831c00bc10 con 0x7f832c105630 2024-03-18T20:29:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.466+0000 7f833096f700 1 -- 172.21.15.46:0/795487102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c196670 con 0x7f832c105630 2024-03-18T20:29:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.466+0000 7f831b7fe700 1 -- 172.21.15.46:0/795487102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f831c0038a0 con 0x7f832c105630 2024-03-18T20:29:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.466+0000 7f833096f700 1 -- 172.21.15.46:0/795487102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c196b10 con 0x7f832c105630 2024-03-18T20:29:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.468+0000 7f831b7fe700 1 -- 172.21.15.46:0/795487102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f831c003a00 con 0x7f832c105630 2024-03-18T20:29:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.468+0000 7f833096f700 1 -- 172.21.15.46:0/795487102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c107ec0 con 0x7f832c105630 2024-03-18T20:29:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.468+0000 7f831b7fe700 1 --2- 172.21.15.46:0/795487102 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f83140387f0 0x7f831403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.468+0000 7f831b7fe700 1 -- 172.21.15.46:0/795487102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f831c04c360 con 0x7f832c105630 2024-03-18T20:29:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.468+0000 7f8329d9b700 1 --2- 172.21.15.46:0/795487102 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f83140387f0 0x7f831403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.468+0000 7f8329d9b700 1 --2- 172.21.15.46:0/795487102 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f83140387f0 0x7f831403acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8320006fd0 tx=0x7f8320006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:54.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.473+0000 7f831b7fe700 1 -- 172.21.15.46:0/795487102 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f831c00b6f0 con 0x7f832c105630 2024-03-18T20:29:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.693+0000 7f833096f700 1 -- 172.21.15.46:0/795487102 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f832c18f620 con 0x7f83140387f0 2024-03-18T20:29:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.694+0000 7f831b7fe700 1 -- 172.21.15.46:0/795487102 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f832c18f620 con 0x7f83140387f0 2024-03-18T20:29:54.699 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:29:54.699 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.73", "hostname": "smithi073", "labels": [], "status": ""}] 2024-03-18T20:29:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.696+0000 7f833096f700 1 -- 172.21.15.46:0/795487102 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f83140387f0 msgr2=0x7f831403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.697+0000 7f833096f700 1 --2- 172.21.15.46:0/795487102 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f83140387f0 0x7f831403acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8320006fd0 tx=0x7f8320006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.697+0000 7f833096f700 1 -- 172.21.15.46:0/795487102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105630 msgr2=0x7f832c195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.697+0000 7f833096f700 1 --2- 172.21.15.46:0/795487102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105630 0x7f832c195f30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f831c006010 tx=0x7f831c004ff0 comp rx=0 tx=0).stop 2024-03-18T20:29:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.697+0000 7f833096f700 1 -- 172.21.15.46:0/795487102 shutdown_connections 2024-03-18T20:29:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.697+0000 7f833096f700 1 --2- 172.21.15.46:0/795487102 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f83140387f0 0x7f831403acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.697+0000 7f833096f700 1 --2- 172.21.15.46:0/795487102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105630 0x7f832c195f30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.697+0000 7f833096f700 1 -- 172.21.15.46:0/795487102 >> 172.21.15.46:0/795487102 conn(0x7f832c075110 msgr2=0x7f832c18ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.697+0000 7f833096f700 1 -- 172.21.15.46:0/795487102 shutdown_connections 2024-03-18T20:29:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:54.697+0000 7f833096f700 1 -- 172.21.15.46:0/795487102 wait complete. 2024-03-18T20:29:55.082 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2024-03-18T20:29:55.082 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-18T20:29:55.082 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2024-03-18T20:29:55.103 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-18T20:29:55.103 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:29:55.164 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2024-03-18T20:29:55.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch host add smithi155 2024-03-18T20:29:55.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:54 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:55.417 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:29:56.214 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:55 smithi046 ceph-mon[43495]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:29:56.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.414+0000 7f8576bda700 1 -- 172.21.15.46:0/27865119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8570105660 msgr2=0x7f8570105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.414+0000 7f8576bda700 1 --2- 172.21.15.46:0/27865119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8570105660 0x7f8570105a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8560009910 tx=0x7f8560009c20 comp rx=0 tx=0).stop 2024-03-18T20:29:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.414+0000 7f8576bda700 1 -- 172.21.15.46:0/27865119 shutdown_connections 2024-03-18T20:29:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.414+0000 7f8576bda700 1 --2- 172.21.15.46:0/27865119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8570105660 0x7f8570105a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.414+0000 7f8576bda700 1 -- 172.21.15.46:0/27865119 >> 172.21.15.46:0/27865119 conn(0x7f85700fb0e0 msgr2=0x7f85700fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.414+0000 7f8576bda700 1 -- 172.21.15.46:0/27865119 shutdown_connections 2024-03-18T20:29:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.414+0000 7f8576bda700 1 -- 172.21.15.46:0/27865119 wait complete. 2024-03-18T20:29:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.415+0000 7f8576bda700 1 Processor -- start 2024-03-18T20:29:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.415+0000 7f8576bda700 1 -- start start 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.415+0000 7f8576bda700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8570105660 0x7f8570194df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.415+0000 7f8576bda700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8570195330 con 0x7f8570105660 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.415+0000 7f8574976700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8570105660 0x7f8570194df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.415+0000 7f8574976700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8570105660 0x7f8570194df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50916/0 (socket says 172.21.15.46:50916) 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.415+0000 7f8574976700 1 -- 172.21.15.46:0/3525144728 learned_addr learned my addr 172.21.15.46:0/3525144728 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.415+0000 7f8574976700 1 -- 172.21.15.46:0/3525144728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85600095f0 con 0x7f8570105660 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.416+0000 7f8574976700 1 --2- 172.21.15.46:0/3525144728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8570105660 0x7f8570194df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8560004f40 tx=0x7f8560005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.416+0000 7f856dffb700 1 -- 172.21.15.46:0/3525144728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f856001d070 con 0x7f8570105660 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.416+0000 7f856dffb700 1 -- 172.21.15.46:0/3525144728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f856000b6f0 con 0x7f8570105660 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.416+0000 7f8576bda700 1 -- 172.21.15.46:0/3525144728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8570195590 con 0x7f8570105660 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.416+0000 7f856dffb700 1 -- 172.21.15.46:0/3525144728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8560003810 con 0x7f8570105660 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.416+0000 7f8576bda700 1 -- 172.21.15.46:0/3525144728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8570195a90 con 0x7f8570105660 2024-03-18T20:29:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.417+0000 7f856dffb700 1 -- 172.21.15.46:0/3525144728 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f8560003ad0 con 0x7f8570105660 2024-03-18T20:29:56.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.417+0000 7f8576bda700 1 -- 172.21.15.46:0/3525144728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8570100d10 con 0x7f8570105660 2024-03-18T20:29:56.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.417+0000 7f856dffb700 1 --2- 172.21.15.46:0/3525144728 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f85580387a0 0x7f855803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:29:56.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.417+0000 7f856dffb700 1 -- 172.21.15.46:0/3525144728 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f856004c300 con 0x7f8570105660 2024-03-18T20:29:56.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.417+0000 7f856ffff700 1 --2- 172.21.15.46:0/3525144728 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f85580387a0 0x7f855803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:29:56.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.418+0000 7f856ffff700 1 --2- 172.21.15.46:0/3525144728 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f85580387a0 0x7f855803ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8564006fd0 tx=0x7f8564006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:29:56.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.422+0000 7f856dffb700 1 -- 172.21.15.46:0/3525144728 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f856002acf0 con 0x7f8570105660 2024-03-18T20:29:56.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:56.685+0000 7f8576bda700 1 -- 172.21.15.46:0/3525144728 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}) v1 -- 0x7f8570195a90 con 0x7f85580387a0 2024-03-18T20:29:57.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:57.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:57.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:57.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:57.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi073", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:29:57.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:29:57.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:29:57.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:57 smithi046 ceph-mon[43495]: Updating smithi073:/etc/ceph/ceph.conf 2024-03-18T20:29:58.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:58 smithi046 ceph-mon[43495]: Updating smithi073:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:29:58.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:58 smithi046 ceph-mon[43495]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:29:58.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:58 smithi046 ceph-mon[43495]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:29:58.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:58 smithi046 ceph-mon[43495]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:29:58.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:58 smithi046 ceph-mon[43495]: Updating smithi073:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:29:58.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:58 smithi046 ceph-mon[43495]: Deploying cephadm binary to smithi155 2024-03-18T20:29:58.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:58 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:58.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:58 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:58.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:29:58 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:29:59.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:59.047+0000 7f856dffb700 1 -- 172.21.15.46:0/3525144728 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8570195a90 con 0x7f85580387a0 2024-03-18T20:29:59.051 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi155' with addr '172.21.15.155' 2024-03-18T20:29:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:59.050+0000 7f8576bda700 1 -- 172.21.15.46:0/3525144728 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f85580387a0 msgr2=0x7f855803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:59.050+0000 7f8576bda700 1 --2- 172.21.15.46:0/3525144728 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f85580387a0 0x7f855803ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8564006fd0 tx=0x7f8564006e40 comp rx=0 tx=0).stop 2024-03-18T20:29:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:59.050+0000 7f8576bda700 1 -- 172.21.15.46:0/3525144728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8570105660 msgr2=0x7f8570194df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:29:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:59.050+0000 7f8576bda700 1 --2- 172.21.15.46:0/3525144728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8570105660 0x7f8570194df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8560004f40 tx=0x7f8560005020 comp rx=0 tx=0).stop 2024-03-18T20:29:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:59.050+0000 7f8576bda700 1 -- 172.21.15.46:0/3525144728 shutdown_connections 2024-03-18T20:29:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:59.050+0000 7f8576bda700 1 --2- 172.21.15.46:0/3525144728 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f85580387a0 0x7f855803ac60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:59.050+0000 7f8576bda700 1 --2- 172.21.15.46:0/3525144728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8570105660 0x7f8570194df0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:29:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:59.050+0000 7f8576bda700 1 -- 172.21.15.46:0/3525144728 >> 172.21.15.46:0/3525144728 conn(0x7f85700fb0e0 msgr2=0x7f85700fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:29:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:59.050+0000 7f8576bda700 1 -- 172.21.15.46:0/3525144728 shutdown_connections 2024-03-18T20:29:59.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:29:59.050+0000 7f8576bda700 1 -- 172.21.15.46:0/3525144728 wait complete. 2024-03-18T20:29:59.455 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-18T20:29:59.706 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:30:00.120 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:00 smithi046 ceph-mon[43495]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:00.120 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:00 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:00.120 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:00 smithi046 ceph-mon[43495]: Added host smithi155 2024-03-18T20:30:00.120 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:00 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:00.120 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:00 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:00.121 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:00 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.723+0000 7f3a7ea20700 1 -- 172.21.15.46:0/1393992049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a781066c0 msgr2=0x7f3a78106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.723+0000 7f3a7ea20700 1 --2- 172.21.15.46:0/1393992049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a781066c0 0x7f3a78106aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009910 tx=0x7f3a68009c20 comp rx=0 tx=0).stop 2024-03-18T20:30:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.723+0000 7f3a7ea20700 1 -- 172.21.15.46:0/1393992049 shutdown_connections 2024-03-18T20:30:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.723+0000 7f3a7ea20700 1 --2- 172.21.15.46:0/1393992049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a781066c0 0x7f3a78106aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.723+0000 7f3a7ea20700 1 -- 172.21.15.46:0/1393992049 >> 172.21.15.46:0/1393992049 conn(0x7f3a780fc1d0 msgr2=0x7f3a780fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.723+0000 7f3a7ea20700 1 -- 172.21.15.46:0/1393992049 shutdown_connections 2024-03-18T20:30:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.723+0000 7f3a7ea20700 1 -- 172.21.15.46:0/1393992049 wait complete. 2024-03-18T20:30:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.723+0000 7f3a7ea20700 1 Processor -- start 2024-03-18T20:30:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.723+0000 7f3a7ea20700 1 -- start start 2024-03-18T20:30:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.724+0000 7f3a7ea20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a781066c0 0x7f3a78195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.724+0000 7f3a7ea20700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a781964b0 con 0x7f3a781066c0 2024-03-18T20:30:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.724+0000 7f3a77fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a781066c0 0x7f3a78195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.724+0000 7f3a77fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a781066c0 0x7f3a78195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50930/0 (socket says 172.21.15.46:50930) 2024-03-18T20:30:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.724+0000 7f3a77fff700 1 -- 172.21.15.46:0/3792254961 learned_addr learned my addr 172.21.15.46:0/3792254961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:30:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.724+0000 7f3a77fff700 1 -- 172.21.15.46:0/3792254961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a680095f0 con 0x7f3a781066c0 2024-03-18T20:30:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.724+0000 7f3a77fff700 1 --2- 172.21.15.46:0/3792254961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a781066c0 0x7f3a78195f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3a68004f40 tx=0x7f3a68005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.725+0000 7f3a757fa700 1 -- 172.21.15.46:0/3792254961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6801d070 con 0x7f3a781066c0 2024-03-18T20:30:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.725+0000 7f3a757fa700 1 -- 172.21.15.46:0/3792254961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a6800bbb0 con 0x7f3a781066c0 2024-03-18T20:30:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.725+0000 7f3a7ea20700 1 -- 172.21.15.46:0/3792254961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a78196710 con 0x7f3a781066c0 2024-03-18T20:30:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.725+0000 7f3a757fa700 1 -- 172.21.15.46:0/3792254961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a68003810 con 0x7f3a781066c0 2024-03-18T20:30:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.725+0000 7f3a7ea20700 1 -- 172.21.15.46:0/3792254961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a78196b90 con 0x7f3a781066c0 2024-03-18T20:30:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.726+0000 7f3a757fa700 1 -- 172.21.15.46:0/3792254961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f3a68003a90 con 0x7f3a781066c0 2024-03-18T20:30:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.726+0000 7f3a5effd700 1 -- 172.21.15.46:0/3792254961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a580052f0 con 0x7f3a781066c0 2024-03-18T20:30:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.726+0000 7f3a757fa700 1 --2- 172.21.15.46:0/3792254961 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3a600387a0 0x7f3a6003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.726+0000 7f3a757fa700 1 -- 172.21.15.46:0/3792254961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3a6804c400 con 0x7f3a781066c0 2024-03-18T20:30:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.726+0000 7f3a777fe700 1 --2- 172.21.15.46:0/3792254961 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3a600387a0 0x7f3a6003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.726+0000 7f3a777fe700 1 --2- 172.21.15.46:0/3792254961 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3a600387a0 0x7f3a6003ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c006fd0 tx=0x7f3a6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.731+0000 7f3a757fa700 1 -- 172.21.15.46:0/3792254961 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3a68029770 con 0x7f3a781066c0 2024-03-18T20:30:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.946+0000 7f3a5effd700 1 -- 172.21.15.46:0/3792254961 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a58000bc0 con 0x7f3a600387a0 2024-03-18T20:30:00.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.947+0000 7f3a757fa700 1 -- 172.21.15.46:0/3792254961 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f3a58000bc0 con 0x7f3a600387a0 2024-03-18T20:30:00.951 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:30:00.951 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.73", "hostname": "smithi073", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2024-03-18T20:30:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.949+0000 7f3a5effd700 1 -- 172.21.15.46:0/3792254961 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3a600387a0 msgr2=0x7f3a6003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.949+0000 7f3a5effd700 1 --2- 172.21.15.46:0/3792254961 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3a600387a0 0x7f3a6003ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c006fd0 tx=0x7f3a6c006e40 comp rx=0 tx=0).stop 2024-03-18T20:30:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.949+0000 7f3a5effd700 1 -- 172.21.15.46:0/3792254961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a781066c0 msgr2=0x7f3a78195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.949+0000 7f3a5effd700 1 --2- 172.21.15.46:0/3792254961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a781066c0 0x7f3a78195f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3a68004f40 tx=0x7f3a68005020 comp rx=0 tx=0).stop 2024-03-18T20:30:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.950+0000 7f3a5effd700 1 -- 172.21.15.46:0/3792254961 shutdown_connections 2024-03-18T20:30:00.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.950+0000 7f3a5effd700 1 --2- 172.21.15.46:0/3792254961 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3a600387a0 0x7f3a6003ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:00.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.950+0000 7f3a5effd700 1 --2- 172.21.15.46:0/3792254961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a781066c0 0x7f3a78195f70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:00.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.950+0000 7f3a5effd700 1 -- 172.21.15.46:0/3792254961 >> 172.21.15.46:0/3792254961 conn(0x7f3a780fc1d0 msgr2=0x7f3a78108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:00.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.950+0000 7f3a5effd700 1 -- 172.21.15.46:0/3792254961 shutdown_connections 2024-03-18T20:30:00.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:00.950+0000 7f3a5effd700 1 -- 172.21.15.46:0/3792254961 wait complete. 2024-03-18T20:30:01.341 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-18T20:30:01.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-18T20:30:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:01 smithi046 ceph-mon[43495]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:30:01.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:01 smithi046 ceph-mon[43495]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:01.595 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:30:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.564+0000 7f696c41d700 1 -- 172.21.15.46:0/2268955559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6964105630 msgr2=0x7f6964105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.564+0000 7f696c41d700 1 --2- 172.21.15.46:0/2268955559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6964105630 0x7f6964105a10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6958009910 tx=0x7f6958009c20 comp rx=0 tx=0).stop 2024-03-18T20:30:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.564+0000 7f696c41d700 1 -- 172.21.15.46:0/2268955559 shutdown_connections 2024-03-18T20:30:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.564+0000 7f696c41d700 1 --2- 172.21.15.46:0/2268955559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6964105630 0x7f6964105a10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.564+0000 7f696c41d700 1 -- 172.21.15.46:0/2268955559 >> 172.21.15.46:0/2268955559 conn(0x7f6964075110 msgr2=0x7f6964075520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.564+0000 7f696c41d700 1 -- 172.21.15.46:0/2268955559 shutdown_connections 2024-03-18T20:30:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.564+0000 7f696c41d700 1 -- 172.21.15.46:0/2268955559 wait complete. 2024-03-18T20:30:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.565+0000 7f696c41d700 1 Processor -- start 2024-03-18T20:30:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.565+0000 7f696c41d700 1 -- start start 2024-03-18T20:30:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.565+0000 7f696c41d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6964105630 0x7f69640fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.565+0000 7f696c41d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69640feb10 con 0x7f6964105630 2024-03-18T20:30:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.565+0000 7f696a1b9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6964105630 0x7f69640fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.565+0000 7f696a1b9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6964105630 0x7f69640fcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50952/0 (socket says 172.21.15.46:50952) 2024-03-18T20:30:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.565+0000 7f696a1b9700 1 -- 172.21.15.46:0/3636110898 learned_addr learned my addr 172.21.15.46:0/3636110898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:30:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.565+0000 7f696a1b9700 1 -- 172.21.15.46:0/3636110898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69580095f0 con 0x7f6964105630 2024-03-18T20:30:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.566+0000 7f696a1b9700 1 --2- 172.21.15.46:0/3636110898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6964105630 0x7f69640fcc50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6958004f40 tx=0x7f6958005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.566+0000 7f69577fe700 1 -- 172.21.15.46:0/3636110898 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f695801d070 con 0x7f6964105630 2024-03-18T20:30:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.566+0000 7f69577fe700 1 -- 172.21.15.46:0/3636110898 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f695800bc10 con 0x7f6964105630 2024-03-18T20:30:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.566+0000 7f696c41d700 1 -- 172.21.15.46:0/3636110898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69640fd190 con 0x7f6964105630 2024-03-18T20:30:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.566+0000 7f69577fe700 1 -- 172.21.15.46:0/3636110898 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69580037d0 con 0x7f6964105630 2024-03-18T20:30:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.566+0000 7f696c41d700 1 -- 172.21.15.46:0/3636110898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69640fd5b0 con 0x7f6964105630 2024-03-18T20:30:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.567+0000 7f69577fe700 1 -- 172.21.15.46:0/3636110898 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f6958003a90 con 0x7f6964105630 2024-03-18T20:30:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.567+0000 7f696c41d700 1 -- 172.21.15.46:0/3636110898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6964100d00 con 0x7f6964105630 2024-03-18T20:30:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.567+0000 7f69577fe700 1 --2- 172.21.15.46:0/3636110898 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f69500387a0 0x7f695003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.567+0000 7f69577fe700 1 -- 172.21.15.46:0/3636110898 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f695804c540 con 0x7f6964105630 2024-03-18T20:30:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.567+0000 7f69699b8700 1 --2- 172.21.15.46:0/3636110898 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f69500387a0 0x7f695003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:02.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.568+0000 7f69699b8700 1 --2- 172.21.15.46:0/3636110898 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f69500387a0 0x7f695003ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6960006fd0 tx=0x7f6960006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:02.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.572+0000 7f69577fe700 1 -- 172.21.15.46:0/3636110898 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6958029770 con 0x7f6964105630 2024-03-18T20:30:02.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.787+0000 7f696c41d700 1 -- 172.21.15.46:0/3636110898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f69640fe240 con 0x7f6964105630 2024-03-18T20:30:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.840+0000 7f69577fe700 1 -- 172.21.15.46:0/3636110898 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6958026070 con 0x7f6964105630 2024-03-18T20:30:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.842+0000 7f696c41d700 1 -- 172.21.15.46:0/3636110898 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f69500387a0 msgr2=0x7f695003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.842+0000 7f696c41d700 1 --2- 172.21.15.46:0/3636110898 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f69500387a0 0x7f695003ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6960006fd0 tx=0x7f6960006e40 comp rx=0 tx=0).stop 2024-03-18T20:30:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.842+0000 7f696c41d700 1 -- 172.21.15.46:0/3636110898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6964105630 msgr2=0x7f69640fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.842+0000 7f696c41d700 1 --2- 172.21.15.46:0/3636110898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6964105630 0x7f69640fcc50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6958004f40 tx=0x7f6958005020 comp rx=0 tx=0).stop 2024-03-18T20:30:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.842+0000 7f696c41d700 1 -- 172.21.15.46:0/3636110898 shutdown_connections 2024-03-18T20:30:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.842+0000 7f696c41d700 1 --2- 172.21.15.46:0/3636110898 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f69500387a0 0x7f695003ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.842+0000 7f696c41d700 1 --2- 172.21.15.46:0/3636110898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6964105630 0x7f69640fcc50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.842+0000 7f696c41d700 1 -- 172.21.15.46:0/3636110898 >> 172.21.15.46:0/3636110898 conn(0x7f6964075110 msgr2=0x7f696418ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.842+0000 7f696c41d700 1 -- 172.21.15.46:0/3636110898 shutdown_connections 2024-03-18T20:30:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:02.842+0000 7f696c41d700 1 -- 172.21.15.46:0/3636110898 wait complete. 2024-03-18T20:30:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-03-18T20:30:03.095 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:02 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3636110898' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-18T20:30:03.221 INFO:tasks.cephadm:Adding mon.a on smithi046 2024-03-18T20:30:03.221 INFO:tasks.cephadm:Adding mon.b on smithi073 2024-03-18T20:30:03.222 INFO:tasks.cephadm:Adding mon.c on smithi155 2024-03-18T20:30:03.222 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch apply mon '3;smithi046:172.21.15.46=a;smithi073:172.21.15.73=b;smithi155:172.21.15.155=c' 2024-03-18T20:30:03.462 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:30:04.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:03 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:04.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:03 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3636110898' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-18T20:30:04.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:03 smithi046 ceph-mon[43495]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T20:30:04.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:03 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:04.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:03 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:04.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:03 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:04.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:03 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:30:04.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:03 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:04.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:03 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:04.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:03 smithi046 ceph-mon[43495]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-18T20:30:04.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:03 smithi046 ceph-mon[43495]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:04.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:03 smithi046 ceph-mon[43495]: Updating smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:30:04.412 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.408+0000 7f4f9488b700 1 -- 172.21.15.155:0/1688187684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f901066d0 msgr2=0x7f4f90106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:04.412 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.408+0000 7f4f9488b700 1 --2- 172.21.15.155:0/1688187684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f901066d0 0x7f4f90106ab0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009910 tx=0x7f4f80009c20 comp rx=0 tx=0).stop 2024-03-18T20:30:04.412 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.408+0000 7f4f9488b700 1 -- 172.21.15.155:0/1688187684 shutdown_connections 2024-03-18T20:30:04.412 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.408+0000 7f4f9488b700 1 --2- 172.21.15.155:0/1688187684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f901066d0 0x7f4f90106ab0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:04.412 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.408+0000 7f4f9488b700 1 -- 172.21.15.155:0/1688187684 >> 172.21.15.155:0/1688187684 conn(0x7f4f900fc200 msgr2=0x7f4f900fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:04.412 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.408+0000 7f4f9488b700 1 -- 172.21.15.155:0/1688187684 shutdown_connections 2024-03-18T20:30:04.412 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.408+0000 7f4f9488b700 1 -- 172.21.15.155:0/1688187684 wait complete. 2024-03-18T20:30:04.413 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.409+0000 7f4f9488b700 1 Processor -- start 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.409+0000 7f4f9488b700 1 -- start start 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.409+0000 7f4f9488b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f901066d0 0x7f4f90195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.409+0000 7f4f9488b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f90196490 con 0x7f4f901066d0 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.409+0000 7f4f8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f901066d0 0x7f4f90195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.410+0000 7f4f8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f901066d0 0x7f4f90195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.155:41366/0 (socket says 172.21.15.155:41366) 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.410+0000 7f4f8e59c700 1 -- 172.21.15.155:0/3399141049 learned_addr learned my addr 172.21.15.155:0/3399141049 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.410+0000 7f4f8e59c700 1 -- 172.21.15.155:0/3399141049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f800095f0 con 0x7f4f901066d0 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.410+0000 7f4f8e59c700 1 --2- 172.21.15.155:0/3399141049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f901066d0 0x7f4f90195f50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4f80004f40 tx=0x7f4f80005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.410+0000 7f4f877fe700 1 -- 172.21.15.155:0/3399141049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f8001d070 con 0x7f4f901066d0 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.410+0000 7f4f9488b700 1 -- 172.21.15.155:0/3399141049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f901966f0 con 0x7f4f901066d0 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.410+0000 7f4f877fe700 1 -- 172.21.15.155:0/3399141049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8000bbb0 con 0x7f4f901066d0 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.410+0000 7f4f877fe700 1 -- 172.21.15.155:0/3399141049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f80003810 con 0x7f4f901066d0 2024-03-18T20:30:04.414 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.410+0000 7f4f9488b700 1 -- 172.21.15.155:0/3399141049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f90196b70 con 0x7f4f901066d0 2024-03-18T20:30:04.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.411+0000 7f4f9488b700 1 -- 172.21.15.155:0/3399141049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f90101e10 con 0x7f4f901066d0 2024-03-18T20:30:04.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.411+0000 7f4f877fe700 1 -- 172.21.15.155:0/3399141049 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f4f8000bd20 con 0x7f4f901066d0 2024-03-18T20:30:04.415 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.411+0000 7f4f877fe700 1 --2- 172.21.15.155:0/3399141049 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4f7c0387a0 0x7f4f7c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:04.416 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.411+0000 7f4f877fe700 1 -- 172.21.15.155:0/3399141049 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4f8004c3b0 con 0x7f4f901066d0 2024-03-18T20:30:04.416 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.412+0000 7f4f8dd9b700 1 --2- 172.21.15.155:0/3399141049 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4f7c0387a0 0x7f4f7c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:04.416 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.412+0000 7f4f8dd9b700 1 --2- 172.21.15.155:0/3399141049 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4f7c0387a0 0x7f4f7c03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4f78006fd0 tx=0x7f4f78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:04.420 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.417+0000 7f4f877fe700 1 -- 172.21.15.155:0/3399141049 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4f80029360 con 0x7f4f901066d0 2024-03-18T20:30:04.622 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.618+0000 7f4f9488b700 1 -- 172.21.15.155:0/3399141049 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi073:172.21.15.73=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f90108b50 con 0x7f4f7c0387a0 2024-03-18T20:30:04.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.626+0000 7f4f877fe700 1 -- 172.21.15.155:0/3399141049 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4f90108b50 con 0x7f4f7c0387a0 2024-03-18T20:30:04.630 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mon update... 2024-03-18T20:30:04.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.627+0000 7f4f9488b700 1 -- 172.21.15.155:0/3399141049 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4f7c0387a0 msgr2=0x7f4f7c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:04.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.627+0000 7f4f9488b700 1 --2- 172.21.15.155:0/3399141049 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4f7c0387a0 0x7f4f7c03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4f78006fd0 tx=0x7f4f78006e40 comp rx=0 tx=0).stop 2024-03-18T20:30:04.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.627+0000 7f4f9488b700 1 -- 172.21.15.155:0/3399141049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f901066d0 msgr2=0x7f4f90195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:04.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.627+0000 7f4f9488b700 1 --2- 172.21.15.155:0/3399141049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f901066d0 0x7f4f90195f50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4f80004f40 tx=0x7f4f80005020 comp rx=0 tx=0).stop 2024-03-18T20:30:04.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.627+0000 7f4f9488b700 1 -- 172.21.15.155:0/3399141049 shutdown_connections 2024-03-18T20:30:04.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.627+0000 7f4f9488b700 1 --2- 172.21.15.155:0/3399141049 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4f7c0387a0 0x7f4f7c03ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:04.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.627+0000 7f4f9488b700 1 --2- 172.21.15.155:0/3399141049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f901066d0 0x7f4f90195f50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:04.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.627+0000 7f4f9488b700 1 -- 172.21.15.155:0/3399141049 >> 172.21.15.155:0/3399141049 conn(0x7f4f900fc200 msgr2=0x7f4f90108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:04.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.627+0000 7f4f9488b700 1 -- 172.21.15.155:0/3399141049 shutdown_connections 2024-03-18T20:30:04.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:04.627+0000 7f4f9488b700 1 -- 172.21.15.155:0/3399141049 wait complete. 2024-03-18T20:30:05.007 DEBUG:teuthology.orchestra.run.smithi073:mon.b> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.b.service 2024-03-18T20:30:05.010 DEBUG:teuthology.orchestra.run.smithi155:mon.c> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.c.service 2024-03-18T20:30:05.012 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T20:30:05.012 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T20:30:05.043 INFO:journalctl@ceph.mon.c.smithi155.stdout:-- Logs begin at Mon 2024-03-18 20:13:03 UTC. -- 2024-03-18T20:30:05.047 INFO:journalctl@ceph.mon.b.smithi073.stdout:-- Logs begin at Mon 2024-03-18 20:12:50 UTC. -- 2024-03-18T20:30:05.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:04 smithi046 ceph-mon[43495]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:30:05.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:04 smithi046 ceph-mon[43495]: Updating smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:30:05.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:05.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:05.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:05.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:05.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:05.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:05.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:05.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:05.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T20:30:05.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:05.389 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:30:06.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:05 smithi046 ceph-mon[43495]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi073:172.21.15.73=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:30:06.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:05 smithi046 ceph-mon[43495]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi073:172.21.15.73=b;smithi155:172.21.15.155=c;count:3 2024-03-18T20:30:06.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:05 smithi046 ceph-mon[43495]: Deploying daemon mon.c on smithi155 2024-03-18T20:30:06.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:05 smithi046 ceph-mon[43495]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:06.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.879+0000 7fa387fd9700 1 -- 172.21.15.155:0/3605482680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3801066d0 msgr2=0x7fa380106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:06.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.879+0000 7fa387fd9700 1 --2- 172.21.15.155:0/3605482680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3801066d0 0x7fa380106ab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009910 tx=0x7fa37c009c20 comp rx=0 tx=0).stop 2024-03-18T20:30:06.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.879+0000 7fa387fd9700 1 -- 172.21.15.155:0/3605482680 shutdown_connections 2024-03-18T20:30:06.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.879+0000 7fa387fd9700 1 --2- 172.21.15.155:0/3605482680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3801066d0 0x7fa380106ab0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:06.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.879+0000 7fa387fd9700 1 -- 172.21.15.155:0/3605482680 >> 172.21.15.155:0/3605482680 conn(0x7fa3800fc200 msgr2=0x7fa3800fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:06.883 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.879+0000 7fa387fd9700 1 -- 172.21.15.155:0/3605482680 shutdown_connections 2024-03-18T20:30:06.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.879+0000 7fa387fd9700 1 -- 172.21.15.155:0/3605482680 wait complete. 2024-03-18T20:30:06.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.880+0000 7fa387fd9700 1 Processor -- start 2024-03-18T20:30:06.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.880+0000 7fa387fd9700 1 -- start start 2024-03-18T20:30:06.884 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.880+0000 7fa387fd9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3801066d0 0x7fa380195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:06.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.880+0000 7fa387fd9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa380196490 con 0x7fa3801066d0 2024-03-18T20:30:06.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.880+0000 7fa385d75700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3801066d0 0x7fa380195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:06.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.880+0000 7fa385d75700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3801066d0 0x7fa380195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.155:41386/0 (socket says 172.21.15.155:41386) 2024-03-18T20:30:06.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.880+0000 7fa385d75700 1 -- 172.21.15.155:0/1671501081 learned_addr learned my addr 172.21.15.155:0/1671501081 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-18T20:30:06.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.881+0000 7fa385d75700 1 -- 172.21.15.155:0/1671501081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c0095f0 con 0x7fa3801066d0 2024-03-18T20:30:06.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.881+0000 7fa385d75700 1 --2- 172.21.15.155:0/1671501081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3801066d0 0x7fa380195f50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa37c004f40 tx=0x7fa37c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:06.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.884+0000 7fa376ffd700 1 -- 172.21.15.155:0/1671501081 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa37c01d070 con 0x7fa3801066d0 2024-03-18T20:30:06.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.884+0000 7fa376ffd700 1 -- 172.21.15.155:0/1671501081 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa37c00bbb0 con 0x7fa3801066d0 2024-03-18T20:30:06.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.884+0000 7fa376ffd700 1 -- 172.21.15.155:0/1671501081 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa37c003810 con 0x7fa3801066d0 2024-03-18T20:30:06.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.884+0000 7fa387fd9700 1 -- 172.21.15.155:0/1671501081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa380196690 con 0x7fa3801066d0 2024-03-18T20:30:06.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.884+0000 7fa387fd9700 1 -- 172.21.15.155:0/1671501081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa380196b10 con 0x7fa3801066d0 2024-03-18T20:30:06.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.887+0000 7fa376ffd700 1 -- 172.21.15.155:0/1671501081 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fa37c003970 con 0x7fa3801066d0 2024-03-18T20:30:06.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.887+0000 7fa376ffd700 1 --2- 172.21.15.155:0/1671501081 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa36c038890 0x7fa36c03ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:06.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.887+0000 7fa376ffd700 1 -- 172.21.15.155:0/1671501081 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa37c04d3d0 con 0x7fa3801066d0 2024-03-18T20:30:06.892 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.887+0000 7fa385574700 1 --2- 172.21.15.155:0/1671501081 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa36c038890 0x7fa36c03ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:06.893 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.889+0000 7fa387fd9700 1 -- 172.21.15.155:0/1671501081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3640052f0 con 0x7fa3801066d0 2024-03-18T20:30:06.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.890+0000 7fa385574700 1 --2- 172.21.15.155:0/1671501081 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa36c038890 0x7fa36c03ad50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa370006fd0 tx=0x7fa370006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:06.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:06.896+0000 7fa376ffd700 1 -- 172.21.15.155:0/1671501081 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa37c026070 con 0x7fa3801066d0 2024-03-18T20:30:07.229 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:07.226+0000 7fa387fd9700 1 -- 172.21.15.155:0/1671501081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa364005160 con 0x7fa3801066d0 2024-03-18T20:30:07.230 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:07.226+0000 7fa376ffd700 1 -- 172.21.15.155:0/1671501081 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7fa37c029360 con 0x7fa3801066d0 2024-03-18T20:30:07.230 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:30:07.230 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","modified":"2024-03-18T20:28:42.853236Z","created":"2024-03-18T20:28:42.853236Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T20:30:07.232 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:07.229+0000 7fa387fd9700 1 -- 172.21.15.155:0/1671501081 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa36c038890 msgr2=0x7fa36c03ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:07.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:07.229+0000 7fa387fd9700 1 --2- 172.21.15.155:0/1671501081 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa36c038890 0x7fa36c03ad50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa370006fd0 tx=0x7fa370006e40 comp rx=0 tx=0).stop 2024-03-18T20:30:07.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:07.229+0000 7fa387fd9700 1 -- 172.21.15.155:0/1671501081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3801066d0 msgr2=0x7fa380195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:07.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:07.229+0000 7fa387fd9700 1 --2- 172.21.15.155:0/1671501081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3801066d0 0x7fa380195f50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa37c004f40 tx=0x7fa37c005020 comp rx=0 tx=0).stop 2024-03-18T20:30:07.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:07.229+0000 7fa387fd9700 1 -- 172.21.15.155:0/1671501081 shutdown_connections 2024-03-18T20:30:07.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:07.229+0000 7fa387fd9700 1 --2- 172.21.15.155:0/1671501081 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa36c038890 0x7fa36c03ad50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:07.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:07.229+0000 7fa387fd9700 1 --2- 172.21.15.155:0/1671501081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3801066d0 0x7fa380195f50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:07.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:07.229+0000 7fa387fd9700 1 -- 172.21.15.155:0/1671501081 >> 172.21.15.155:0/1671501081 conn(0x7fa3800fc200 msgr2=0x7fa380108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:07.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:07.229+0000 7fa387fd9700 1 -- 172.21.15.155:0/1671501081 shutdown_connections 2024-03-18T20:30:07.233 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:07.229+0000 7fa387fd9700 1 -- 172.21.15.155:0/1671501081 wait complete. 2024-03-18T20:30:07.233 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-03-18T20:30:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 systemd[1]: Started Ceph mon.c for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:30:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T20:30:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T20:30:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T20:30:08.295 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.table_properties_collectors: 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.inplace_update_support: 0 2024-03-18T20:30:08.296 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.bloom_locality: 0 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.max_successive_merges: 0 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.ttl: 2592000 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T20:30:08.297 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.enable_blob_files: false 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.min_blob_size: 0 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5dd38eed-ffc6-407a-85a9-8dd03e3160bb 2024-03-18T20:30:08.298 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710793807882771, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710793807883352, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710793807, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5dd38eed-ffc6-407a-85a9-8dd03e3160bb", "db_session_id": "EHN8Y2GGDSIS7FCHNEY7", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710793807883441, "job": 1, "event": "recovery_finished"} 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56526d42c000 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: DB pointer 0x56526d41a000 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout: ** DB Stats ** 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T20:30:08.299 INFO:journalctl@ceph.mon.c.smithi155.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: ** Compaction Stats [default] ** 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: ** Compaction Stats [default] ** 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T20:30:08.300 INFO:journalctl@ceph.mon.c.smithi155.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: Cumulative compaction: 0.00 GB write, 0.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: Interval compaction: 0.00 GB write, 0.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: Block cache BinnedLRUCache@0x56526d38f1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-18T20:30:08.301 INFO:journalctl@ceph.mon.c.smithi155.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout: 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: using public_addr v2:172.21.15.155:0/0 -> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: starting mon.c rank -1 at public addrs [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] at bind addrs [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(???) e0 preinit fsid 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).mds e1 new map 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout: e1 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout: legacy client fscid: -1 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout: 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout: No filesystems configured 2024-03-18T20:30:08.302 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3636110898' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3636110898' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T20:30:08.303 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: Updating smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: Updating smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:08.304 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:08.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:08.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:08.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:08.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T20:30:08.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:08.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi073:172.21.15.73=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:30:08.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi073:172.21.15.73=b;smithi155:172.21.15.155=c;count:3 2024-03-18T20:30:08.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: Deploying daemon mon.c on smithi155 2024-03-18T20:30:08.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:08.305 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:07 smithi155 ceph-mon[48344]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-18T20:30:08.608 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T20:30:08.608 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T20:30:08.839 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T20:30:13.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:30:13.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: mon.a calling monitor election 2024-03-18T20:30:13.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:13.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:13.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:13.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:13.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: mon.c calling monitor election 2024-03-18T20:30:13.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:13.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:13.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:13.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:13.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T20:30:13.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: monmap e2: 2 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],c=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T20:30:13.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: fsmap 2024-03-18T20:30:13.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T20:30:13.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: mgrmap e13: x(active, since 36s) 2024-03-18T20:30:13.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: overall HEALTH_OK 2024-03-18T20:30:13.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:13.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:30:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: mon.a calling monitor election 2024-03-18T20:30:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: mon.c calling monitor election 2024-03-18T20:30:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T20:30:13.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: monmap e2: 2 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],c=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T20:30:13.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: fsmap 2024-03-18T20:30:13.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T20:30:13.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: mgrmap e13: x(active, since 36s) 2024-03-18T20:30:13.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: overall HEALTH_OK 2024-03-18T20:30:13.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:13.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:13.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.341+0000 7fe1ac681700 1 -- 172.21.15.155:0/1070488725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1900035c0 msgr2=0x7fe190005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:13.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.341+0000 7fe1ac681700 1 --2- 172.21.15.155:0/1070488725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1900035c0 0x7fe190005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40ff600 tx=0x7fe198014ec0 comp rx=0 tx=0).stop 2024-03-18T20:30:13.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.341+0000 7fe1ac681700 1 -- 172.21.15.155:0/1070488725 shutdown_connections 2024-03-18T20:30:13.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.341+0000 7fe1ac681700 1 --2- 172.21.15.155:0/1070488725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1900035c0 0x7fe190005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:13.346 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.341+0000 7fe1ac681700 1 -- 172.21.15.155:0/1070488725 >> 172.21.15.155:0/1070488725 conn(0x7fe1a40fb1c0 msgr2=0x7fe1a40fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:13.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.341+0000 7fe1ac681700 1 -- 172.21.15.155:0/1070488725 shutdown_connections 2024-03-18T20:30:13.347 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.341+0000 7fe1ac681700 1 -- 172.21.15.155:0/1070488725 wait complete. 2024-03-18T20:30:13.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.342+0000 7fe1ac681700 1 Processor -- start 2024-03-18T20:30:13.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.342+0000 7fe1ac681700 1 -- start start 2024-03-18T20:30:13.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1ac681700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1900035c0 0x7fe1a4196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:13.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1ac681700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1a4197460 0x7fe1a41978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:13.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1ac681700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a4069c00 con 0x7fe1900035c0 2024-03-18T20:30:13.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1ac681700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a419ff90 con 0x7fe1a4197460 2024-03-18T20:30:13.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1a9c1c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1a4197460 0x7fe1a41978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:13.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1aa41d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1900035c0 0x7fe1a4196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:13.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1a9c1c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1a4197460 0x7fe1a41978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:45586/0 (socket says 172.21.15.155:45586) 2024-03-18T20:30:13.348 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1aa41d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1900035c0 0x7fe1a4196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.155:51880/0 (socket says 172.21.15.155:51880) 2024-03-18T20:30:13.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1a9c1c700 1 -- 172.21.15.155:0/3062619606 learned_addr learned my addr 172.21.15.155:0/3062619606 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-18T20:30:13.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1a9c1c700 1 -- 172.21.15.155:0/3062619606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1a4197460 msgr2=0x7fe1a41978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-18T20:30:13.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1a9c1c700 1 -- 172.21.15.155:0/3062619606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1a4197460 msgr2=0x7fe1a41978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T20:30:13.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1a9c1c700 1 --2- 172.21.15.155:0/3062619606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1a4197460 0x7fe1a41978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T20:30:13.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1a9c1c700 1 --2- 172.21.15.155:0/3062619606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1a4197460 0x7fe1a41978e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T20:30:13.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1aa41d700 1 -- 172.21.15.155:0/3062619606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1a4197460 msgr2=0x7fe1a41978e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:30:13.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1aa41d700 1 --2- 172.21.15.155:0/3062619606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1a4197460 0x7fe1a41978e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:13.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.343+0000 7fe1aa41d700 1 -- 172.21.15.155:0/3062619606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe198014910 con 0x7fe1900035c0 2024-03-18T20:30:13.349 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.344+0000 7fe1aa41d700 1 --2- 172.21.15.155:0/3062619606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1900035c0 0x7fe1a4196f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe198010010 tx=0x7fe19800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:13.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.344+0000 7fe1977fe700 1 -- 172.21.15.155:0/3062619606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe198023070 con 0x7fe1900035c0 2024-03-18T20:30:13.350 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.344+0000 7fe1ac681700 1 -- 172.21.15.155:0/3062619606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a41a0190 con 0x7fe1900035c0 2024-03-18T20:30:13.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.344+0000 7fe1977fe700 1 -- 172.21.15.155:0/3062619606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe198004b90 con 0x7fe1900035c0 2024-03-18T20:30:13.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.344+0000 7fe1977fe700 1 -- 172.21.15.155:0/3062619606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe198019040 con 0x7fe1900035c0 2024-03-18T20:30:13.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.344+0000 7fe1ac681700 1 -- 172.21.15.155:0/3062619606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a41a06b0 con 0x7fe1900035c0 2024-03-18T20:30:13.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.345+0000 7fe1977fe700 1 -- 172.21.15.155:0/3062619606 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fe198012450 con 0x7fe1900035c0 2024-03-18T20:30:13.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.345+0000 7fe1ac681700 1 -- 172.21.15.155:0/3062619606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a418edc0 con 0x7fe1900035c0 2024-03-18T20:30:13.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.345+0000 7fe1977fe700 1 --2- 172.21.15.155:0/3062619606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe1880385d0 0x7fe18803aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:13.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.345+0000 7fe1977fe700 1 -- 172.21.15.155:0/3062619606 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe19804e630 con 0x7fe1900035c0 2024-03-18T20:30:13.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.346+0000 7fe1a9c1c700 1 --2- 172.21.15.155:0/3062619606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe1880385d0 0x7fe18803aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:13.351 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.346+0000 7fe1a9c1c700 1 --2- 172.21.15.155:0/3062619606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe1880385d0 0x7fe18803aa90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009180 tx=0x7fe1a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:13.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.351+0000 7fe1977fe700 1 -- 172.21.15.155:0/3062619606 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe198010790 con 0x7fe1900035c0 2024-03-18T20:30:13.625 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.621+0000 7fe1ac681700 1 -- 172.21.15.155:0/3062619606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe1a418e7a0 con 0x7fe1900035c0 2024-03-18T20:30:13.626 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.621+0000 7fe1977fe700 1 -- 172.21.15.155:0/3062619606 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fe198030cc0 con 0x7fe1900035c0 2024-03-18T20:30:13.626 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:30:13.626 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":2,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","modified":"2024-03-18T20:30:07.913053Z","created":"2024-03-18T20:28:42.853236Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-18T20:30:13.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.624+0000 7fe1ac681700 1 -- 172.21.15.155:0/3062619606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe1880385d0 msgr2=0x7fe18803aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:13.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.624+0000 7fe1ac681700 1 --2- 172.21.15.155:0/3062619606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe1880385d0 0x7fe18803aa90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009180 tx=0x7fe1a0008040 comp rx=0 tx=0).stop 2024-03-18T20:30:13.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.624+0000 7fe1ac681700 1 -- 172.21.15.155:0/3062619606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1900035c0 msgr2=0x7fe1a4196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:13.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.624+0000 7fe1ac681700 1 --2- 172.21.15.155:0/3062619606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1900035c0 0x7fe1a4196f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe198010010 tx=0x7fe19800fc60 comp rx=0 tx=0).stop 2024-03-18T20:30:13.628 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.624+0000 7fe1ac681700 1 -- 172.21.15.155:0/3062619606 shutdown_connections 2024-03-18T20:30:13.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.624+0000 7fe1ac681700 1 --2- 172.21.15.155:0/3062619606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe1880385d0 0x7fe18803aa90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:13.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.624+0000 7fe1ac681700 1 --2- 172.21.15.155:0/3062619606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1900035c0 0x7fe1a4196f20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:13.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.624+0000 7fe1ac681700 1 --2- 172.21.15.155:0/3062619606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1a4197460 0x7fe1a41978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:13.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.624+0000 7fe1ac681700 1 -- 172.21.15.155:0/3062619606 >> 172.21.15.155:0/3062619606 conn(0x7fe1a40fb1c0 msgr2=0x7fe1a40fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:13.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.624+0000 7fe1ac681700 1 -- 172.21.15.155:0/3062619606 shutdown_connections 2024-03-18T20:30:13.629 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:13.624+0000 7fe1ac681700 1 -- 172.21.15.155:0/3062619606 wait complete. 2024-03-18T20:30:13.629 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 2 2024-03-18T20:30:14.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:14 smithi155 ceph-mon[48344]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:14.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:14.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T20:30:14.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:14.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:14 smithi155 ceph-mon[48344]: Deploying daemon mon.b on smithi073 2024-03-18T20:30:14.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:14 smithi155 ceph-mon[48344]: from='client.? 172.21.15.155:0/3062619606' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T20:30:14.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:14.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:14 smithi046 ceph-mon[43495]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:14.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:14.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T20:30:14.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:14.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:14 smithi046 ceph-mon[43495]: Deploying daemon mon.b on smithi073 2024-03-18T20:30:14.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:14 smithi046 ceph-mon[43495]: from='client.? 172.21.15.155:0/3062619606' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T20:30:14.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:15.005 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T20:30:15.005 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T20:30:15.234 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T20:30:15.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 systemd[1]: Started Ceph mon.b for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:30:15.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T20:30:15.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T20:30:15.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T20:30:15.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.log_readahead_size: 0 2024-03-18T20:30:15.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T20:30:15.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T20:30:15.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T20:30:15.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T20:30:15.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T20:30:15.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T20:30:15.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-18T20:30:15.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_background_jobs: 2 2024-03-18T20:30:15.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_background_compactions: -1 2024-03-18T20:30:15.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_subcompactions: 1 2024-03-18T20:30:15.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_open_files: -1 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T20:30:15.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_background_flushes: -1 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Compression algorithms supported: 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: kZSTD supported: 0 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: kXpressCompression supported: 0 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: kZlibCompression supported: 1 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: kSnappyCompression supported: 1 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: kLZ4Compression supported: 1 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: kBZip2Compression supported: 0 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-18T20:30:15.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.merge_operator: 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_filter: None 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557770e4b5e0) 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout: cache_index_and_filter_blocks: 1 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout: pin_top_level_index_and_filter: 1 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout: index_type: 0 2024-03-18T20:30:15.975 INFO:journalctl@ceph.mon.b.smithi073.stdout: data_block_index_type: 0 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: index_shortening: 1 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: checksum: 4 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: no_block_cache: 0 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_cache: 0x5577701651f0 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_cache_name: BinnedLRUCache 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_cache_options: 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: capacity : 536870912 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: num_shard_bits : 4 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: strict_capacity_limit : 0 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: high_pri_pool_ratio: 0.000 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_cache_compressed: (nil) 2024-03-18T20:30:15.976 INFO:journalctl@ceph.mon.b.smithi073.stdout: persistent_cache: (nil) 2024-03-18T20:30:15.977 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_size: 4096 2024-03-18T20:30:15.977 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_size_deviation: 10 2024-03-18T20:30:15.977 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_restart_interval: 16 2024-03-18T20:30:15.977 INFO:journalctl@ceph.mon.b.smithi073.stdout: index_block_restart_interval: 1 2024-03-18T20:30:15.977 INFO:journalctl@ceph.mon.b.smithi073.stdout: metadata_block_size: 4096 2024-03-18T20:30:15.977 INFO:journalctl@ceph.mon.b.smithi073.stdout: partition_filters: 0 2024-03-18T20:30:15.977 INFO:journalctl@ceph.mon.b.smithi073.stdout: use_delta_encoding: 1 2024-03-18T20:30:15.977 INFO:journalctl@ceph.mon.b.smithi073.stdout: filter_policy: bloomfilter 2024-03-18T20:30:15.977 INFO:journalctl@ceph.mon.b.smithi073.stdout: whole_key_filtering: 1 2024-03-18T20:30:15.977 INFO:journalctl@ceph.mon.b.smithi073.stdout: verify_compression: 0 2024-03-18T20:30:15.978 INFO:journalctl@ceph.mon.b.smithi073.stdout: read_amp_bytes_per_bit: 0 2024-03-18T20:30:15.978 INFO:journalctl@ceph.mon.b.smithi073.stdout: format_version: 5 2024-03-18T20:30:15.978 INFO:journalctl@ceph.mon.b.smithi073.stdout: enable_index_compression: 1 2024-03-18T20:30:15.978 INFO:journalctl@ceph.mon.b.smithi073.stdout: block_align: 0 2024-03-18T20:30:15.978 INFO:journalctl@ceph.mon.b.smithi073.stdout: max_auto_readahead_size: 262144 2024-03-18T20:30:15.978 INFO:journalctl@ceph.mon.b.smithi073.stdout: prepopulate_block_cache: 0 2024-03-18T20:30:15.978 INFO:journalctl@ceph.mon.b.smithi073.stdout: initial_auto_readahead_size: 8192 2024-03-18T20:30:15.978 INFO:journalctl@ceph.mon.b.smithi073.stdout: num_file_reads_for_auto_readahead: 2 2024-03-18T20:30:15.978 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T20:30:15.978 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T20:30:15.978 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compression: NoCompression 2024-03-18T20:30:15.978 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T20:30:15.980 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T20:30:15.980 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T20:30:15.980 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.num_levels: 7 2024-03-18T20:30:15.980 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T20:30:15.980 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T20:30:15.980 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T20:30:15.980 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T20:30:15.980 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T20:30:15.980 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T20:30:15.980 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T20:30:15.980 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T20:30:15.981 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T20:30:15.982 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T20:30:15.983 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.table_properties_collectors: 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.inplace_update_support: 0 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.bloom_locality: 0 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.max_successive_merges: 0 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T20:30:15.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.ttl: 2592000 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.enable_blob_files: false 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.min_blob_size: 0 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T20:30:15.985 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 81f17a64-b16b-4a4b-802e-8098d8eebfa5 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710793815576598, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710793815577019, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710793815, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "81f17a64-b16b-4a4b-802e-8098d8eebfa5", "db_session_id": "1QEIUKSNCCD7323GLC98", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710793815577099, "job": 1, "event": "recovery_finished"} 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557770202000 2024-03-18T20:30:15.986 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: DB pointer 0x5577701f0000 2024-03-18T20:30:15.987 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T20:30:15.987 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T20:30:15.987 INFO:journalctl@ceph.mon.b.smithi073.stdout: ** DB Stats ** 2024-03-18T20:30:15.987 INFO:journalctl@ceph.mon.b.smithi073.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T20:30:15.987 INFO:journalctl@ceph.mon.b.smithi073.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T20:30:15.987 INFO:journalctl@ceph.mon.b.smithi073.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T20:30:15.987 INFO:journalctl@ceph.mon.b.smithi073.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T20:30:15.987 INFO:journalctl@ceph.mon.b.smithi073.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T20:30:15.987 INFO:journalctl@ceph.mon.b.smithi073.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T20:30:15.987 INFO:journalctl@ceph.mon.b.smithi073.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T20:30:15.987 INFO:journalctl@ceph.mon.b.smithi073.stdout: 2024-03-18T20:30:15.987 INFO:journalctl@ceph.mon.b.smithi073.stdout: ** Compaction Stats [default] ** 2024-03-18T20:30:15.988 INFO:journalctl@ceph.mon.b.smithi073.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T20:30:15.988 INFO:journalctl@ceph.mon.b.smithi073.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T20:30:15.988 INFO:journalctl@ceph.mon.b.smithi073.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T20:30:15.988 INFO:journalctl@ceph.mon.b.smithi073.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T20:30:15.988 INFO:journalctl@ceph.mon.b.smithi073.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T20:30:15.988 INFO:journalctl@ceph.mon.b.smithi073.stdout: 2024-03-18T20:30:15.988 INFO:journalctl@ceph.mon.b.smithi073.stdout: ** Compaction Stats [default] ** 2024-03-18T20:30:15.988 INFO:journalctl@ceph.mon.b.smithi073.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T20:30:15.988 INFO:journalctl@ceph.mon.b.smithi073.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T20:30:15.989 INFO:journalctl@ceph.mon.b.smithi073.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T20:30:15.989 INFO:journalctl@ceph.mon.b.smithi073.stdout: 2024-03-18T20:30:15.989 INFO:journalctl@ceph.mon.b.smithi073.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T20:30:15.989 INFO:journalctl@ceph.mon.b.smithi073.stdout: 2024-03-18T20:30:15.989 INFO:journalctl@ceph.mon.b.smithi073.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T20:30:15.989 INFO:journalctl@ceph.mon.b.smithi073.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T20:30:15.989 INFO:journalctl@ceph.mon.b.smithi073.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T20:30:15.989 INFO:journalctl@ceph.mon.b.smithi073.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T20:30:15.989 INFO:journalctl@ceph.mon.b.smithi073.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T20:30:15.989 INFO:journalctl@ceph.mon.b.smithi073.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T20:30:15.989 INFO:journalctl@ceph.mon.b.smithi073.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout: Block cache BinnedLRUCache@0x5577701651f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout: 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: using public_addr v2:172.21.15.73:0/0 -> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: starting mon.b rank -1 at public addrs [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] at bind addrs [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(???) e0 preinit fsid 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).mds e1 new map 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-18T20:30:15.990 INFO:journalctl@ceph.mon.b.smithi073.stdout: e1 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout: legacy client fscid: -1 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout: 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout: No filesystems configured 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T20:30:15.991 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3636110898' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3636110898' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:15.992 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: Updating smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: Updating smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi073:172.21.15.73=b;smithi155:172.21.15.155=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi073:172.21.15.73=b;smithi155:172.21.15.155=c;count:3 2024-03-18T20:30:15.993 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: Deploying daemon mon.c on smithi155 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.a calling monitor election 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.c calling monitor election 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:15.994 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: monmap e2: 2 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],c=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: fsmap 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mgrmap e13: x(active, since 36s) 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: overall HEALTH_OK 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: Deploying daemon mon.b on smithi073 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='client.? 172.21.15.155:0/3062619606' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:15.995 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:15 smithi073 ceph-mon[47384]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-18T20:30:21.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:30:21.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: mon.a calling monitor election 2024-03-18T20:30:21.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: mon.c calling monitor election 2024-03-18T20:30:21.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:21.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:21.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:21.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:21.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:21.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:21.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:21.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:21.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:21.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T20:30:21.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0],c=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T20:30:21.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: fsmap 2024-03-18T20:30:21.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T20:30:21.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: mgrmap e13: x(active, since 43s) 2024-03-18T20:30:21.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-18T20:30:21.043 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-18T20:30:21.043 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-18T20:30:21.043 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: mon.b (rank 2) addr [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] is down (out of quorum) 2024-03-18T20:30:21.043 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:21.043 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:21.043 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:21.043 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:21.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:30:21.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: mon.a calling monitor election 2024-03-18T20:30:21.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: mon.c calling monitor election 2024-03-18T20:30:21.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:21.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:21.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:21.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:21.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:21.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:21.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0],c=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: fsmap 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: mgrmap e13: x(active, since 43s) 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: mon.b (rank 2) addr [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] is down (out of quorum) 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:21.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:21.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:21.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:22.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:21 smithi155 ceph-mon[48344]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:22.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:22.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:22.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:22.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:22.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:21 smithi046 ceph-mon[43495]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:22.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:22.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:22.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:22.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.097+0000 7fa07a80a700 1 -- 172.21.15.155:0/10838732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa05c006230 msgr2=0x7fa05c0086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:22.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.097+0000 7fa07a80a700 1 --2- 172.21.15.155:0/10838732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa05c006230 0x7fa05c0086a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa074068ee0 tx=0x7fa06400fe90 comp rx=0 tx=0).stop 2024-03-18T20:30:22.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.097+0000 7fa07a80a700 1 -- 172.21.15.155:0/10838732 shutdown_connections 2024-03-18T20:30:22.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.097+0000 7fa07a80a700 1 --2- 172.21.15.155:0/10838732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa05c006230 0x7fa05c0086a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:22.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.097+0000 7fa07a80a700 1 -- 172.21.15.155:0/10838732 >> 172.21.15.155:0/10838732 conn(0x7fa0740fb1c0 msgr2=0x7fa0740fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:22.103 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.097+0000 7fa07a80a700 1 -- 172.21.15.155:0/10838732 shutdown_connections 2024-03-18T20:30:22.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.097+0000 7fa07a80a700 1 -- 172.21.15.155:0/10838732 wait complete. 2024-03-18T20:30:22.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.097+0000 7fa07a80a700 1 Processor -- start 2024-03-18T20:30:22.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa07a80a700 1 -- start start 2024-03-18T20:30:22.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa07a80a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa05c006230 0x7fa0740ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:22.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa07a80a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0740ffb50 0x7fa0740fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:22.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa07a80a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0741a2460 0x7fa0741a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:22.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa07a80a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0741a4e20 con 0x7fa0741a2460 2024-03-18T20:30:22.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa07a80a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0741a4f60 con 0x7fa0740ffb50 2024-03-18T20:30:22.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa07a80a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0741a5120 con 0x7fa05c006230 2024-03-18T20:30:22.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa073fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa05c006230 0x7fa0740ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:22.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa073fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa05c006230 0x7fa0740ff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:45594/0 (socket says 172.21.15.155:45594) 2024-03-18T20:30:22.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa073fff700 1 -- 172.21.15.155:0/1636761425 learned_addr learned my addr 172.21.15.155:0/1636761425 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa078da7700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0741a2460 0x7fa0741a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa073fff700 1 -- 172.21.15.155:0/1636761425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa05c006230 msgr2=0x7fa0740ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa073fff700 1 -- 172.21.15.155:0/1636761425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa05c006230 msgr2=0x7fa0740ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa073fff700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa05c006230 0x7fa0740ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa073fff700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa05c006230 0x7fa0740ff610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa0737fe700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0740ffb50 0x7fa0740fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa0737fe700 1 -- 172.21.15.155:0/1636761425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0740ffb50 msgr2=0x7fa0740fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa0737fe700 1 -- 172.21.15.155:0/1636761425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0740ffb50 msgr2=0x7fa0740fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa0737fe700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0740ffb50 0x7fa0740fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.098+0000 7fa0737fe700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0740ffb50 0x7fa0740fffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.099+0000 7fa078da7700 1 -- 172.21.15.155:0/1636761425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa05c006230 msgr2=0x7fa0740ff610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.099+0000 7fa078da7700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa05c006230 0x7fa0740ff610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.099+0000 7fa078da7700 1 -- 172.21.15.155:0/1636761425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0740ffb50 msgr2=0x7fa0740fffd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:30:22.105 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.099+0000 7fa078da7700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0740ffb50 0x7fa0740fffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:22.106 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.099+0000 7fa078da7700 1 -- 172.21.15.155:0/1636761425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06400f8e0 con 0x7fa0741a2460 2024-03-18T20:30:22.106 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.099+0000 7fa078da7700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0741a2460 0x7fa0741a4850 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00bd50 tx=0x7fa06c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:22.106 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.099+0000 7fa0717fa700 1 -- 172.21.15.155:0/1636761425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00cd10 con 0x7fa0741a2460 2024-03-18T20:30:22.106 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.099+0000 7fa0717fa700 1 -- 172.21.15.155:0/1636761425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa06c00ce70 con 0x7fa0741a2460 2024-03-18T20:30:22.106 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.099+0000 7fa0717fa700 1 -- 172.21.15.155:0/1636761425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c0127d0 con 0x7fa0741a2460 2024-03-18T20:30:22.106 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.099+0000 7fa07a80a700 1 -- 172.21.15.155:0/1636761425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0741a5400 con 0x7fa0741a2460 2024-03-18T20:30:22.106 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.099+0000 7fa07a80a700 1 -- 172.21.15.155:0/1636761425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0741a58b0 con 0x7fa0741a2460 2024-03-18T20:30:22.107 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.100+0000 7fa0717fa700 1 -- 172.21.15.155:0/1636761425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fa06c0129b0 con 0x7fa0741a2460 2024-03-18T20:30:22.107 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.100+0000 7fa07a80a700 1 -- 172.21.15.155:0/1636761425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa074069560 con 0x7fa0741a2460 2024-03-18T20:30:22.107 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.100+0000 7fa0717fa700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa060038c20 0x7fa06003b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:22.107 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.100+0000 7fa0717fa700 1 -- 172.21.15.155:0/1636761425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa06c04af30 con 0x7fa0741a2460 2024-03-18T20:30:22.107 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.101+0000 7fa073fff700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa060038c20 0x7fa06003b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:22.107 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.101+0000 7fa073fff700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa060038c20 0x7fa06003b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa064005b40 tx=0x7fa064005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:22.109 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.106+0000 7fa0717fa700 1 -- 172.21.15.155:0/1636761425 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa06c01d3b0 con 0x7fa0741a2460 2024-03-18T20:30:22.394 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.390+0000 7fa07a80a700 1 -- 172.21.15.155:0/1636761425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa0741a5b30 con 0x7fa0741a2460 2024-03-18T20:30:22.395 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.391+0000 7fa0717fa700 1 -- 172.21.15.155:0/1636761425 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7fa06c019030 con 0x7fa0741a2460 2024-03-18T20:30:22.395 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-18T20:30:22.395 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":3,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","modified":"2024-03-18T20:30:15.609960Z","created":"2024-03-18T20:28:42.853236Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:3300","nonce":0},{"type":"v1","addr":"172.21.15.73:6789","nonce":0}]},"addr":"172.21.15.73:6789/0","public_addr":"172.21.15.73:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-18T20:30:22.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.393+0000 7fa07a80a700 1 -- 172.21.15.155:0/1636761425 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa060038c20 msgr2=0x7fa06003b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:22.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.393+0000 7fa07a80a700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa060038c20 0x7fa06003b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa064005b40 tx=0x7fa064005a90 comp rx=0 tx=0).stop 2024-03-18T20:30:22.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.393+0000 7fa07a80a700 1 -- 172.21.15.155:0/1636761425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0741a2460 msgr2=0x7fa0741a4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:22.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.393+0000 7fa07a80a700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0741a2460 0x7fa0741a4850 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00bd50 tx=0x7fa06c00be30 comp rx=0 tx=0).stop 2024-03-18T20:30:22.397 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.393+0000 7fa07a80a700 1 -- 172.21.15.155:0/1636761425 shutdown_connections 2024-03-18T20:30:22.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.393+0000 7fa07a80a700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa060038c20 0x7fa06003b0e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:22.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.393+0000 7fa07a80a700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0741a2460 0x7fa0741a4850 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:22.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.393+0000 7fa07a80a700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa05c006230 0x7fa0740ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:22.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.393+0000 7fa07a80a700 1 --2- 172.21.15.155:0/1636761425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0740ffb50 0x7fa0740fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:22.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.393+0000 7fa07a80a700 1 -- 172.21.15.155:0/1636761425 >> 172.21.15.155:0/1636761425 conn(0x7fa0740fb1c0 msgr2=0x7fa074102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:22.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.393+0000 7fa07a80a700 1 -- 172.21.15.155:0/1636761425 shutdown_connections 2024-03-18T20:30:22.402 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:22.393+0000 7fa07a80a700 1 -- 172.21.15.155:0/1636761425 wait complete. 2024-03-18T20:30:22.402 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 3 2024-03-18T20:30:22.795 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-18T20:30:22.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-18T20:30:22.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:30:22.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: mon.a calling monitor election 2024-03-18T20:30:22.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: mon.c calling monitor election 2024-03-18T20:30:22.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:22.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:30:22.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0],c=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: fsmap 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: mgrmap e13: x(active, since 43s) 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-18T20:30:22.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-18T20:30:22.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: mon.b (rank 2) addr [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] is down (out of quorum) 2024-03-18T20:30:22.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:22.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:22.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:22.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:22.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:22.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:23.160 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:30:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: mon.b calling monitor election 2024-03-18T20:30:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: mon.b calling monitor election 2024-03-18T20:30:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: mon.c calling monitor election 2024-03-18T20:30:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: mon.a calling monitor election 2024-03-18T20:30:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-18T20:30:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0],c=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T20:30:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: fsmap 2024-03-18T20:30:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T20:30:23.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: mgrmap e13: x(active, since 45s) 2024-03-18T20:30:23.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-18T20:30:23.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: Cluster is now healthy 2024-03-18T20:30:23.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: overall HEALTH_OK 2024-03-18T20:30:23.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:23.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:23.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T20:30:23.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T20:30:23.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:23 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:23.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: mon.b calling monitor election 2024-03-18T20:30:23.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: mon.b calling monitor election 2024-03-18T20:30:23.602 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: mon.c calling monitor election 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: mon.a calling monitor election 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0],c=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: fsmap 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: mgrmap e13: x(active, since 45s) 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: Cluster is now healthy 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: overall HEALTH_OK 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T20:30:23.603 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:23 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:23.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: mon.b calling monitor election 2024-03-18T20:30:23.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: mon.b calling monitor election 2024-03-18T20:30:23.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: mon.c calling monitor election 2024-03-18T20:30:23.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: mon.a calling monitor election 2024-03-18T20:30:23.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-18T20:30:23.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0],c=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-18T20:30:23.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: fsmap 2024-03-18T20:30:23.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T20:30:23.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: mgrmap e13: x(active, since 45s) 2024-03-18T20:30:23.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-18T20:30:23.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: Cluster is now healthy 2024-03-18T20:30:23.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: overall HEALTH_OK 2024-03-18T20:30:23.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:23.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:23.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T20:30:23.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T20:30:23.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:23 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.125+0000 7f9419794700 1 -- 172.21.15.46:0/755453964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9414072fe0 msgr2=0x7f94140733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.125+0000 7f9419794700 1 --2- 172.21.15.46:0/755453964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9414072fe0 0x7f94140733c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9404009990 tx=0x7f9404009ca0 comp rx=0 tx=0).stop 2024-03-18T20:30:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.125+0000 7f9419794700 1 -- 172.21.15.46:0/755453964 shutdown_connections 2024-03-18T20:30:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.125+0000 7f9419794700 1 --2- 172.21.15.46:0/755453964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9414072fe0 0x7f94140733c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.125+0000 7f9419794700 1 --2- 172.21.15.46:0/755453964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f941410d7b0 0x7f941410fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.125+0000 7f9419794700 1 --2- 172.21.15.46:0/755453964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9414073900 0x7f941410d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.125+0000 7f9419794700 1 -- 172.21.15.46:0/755453964 >> 172.21.15.46:0/755453964 conn(0x7f9414077f10 msgr2=0x7f9414078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.125+0000 7f9419794700 1 -- 172.21.15.46:0/755453964 shutdown_connections 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.125+0000 7f9419794700 1 -- 172.21.15.46:0/755453964 wait complete. 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.126+0000 7f9419794700 1 Processor -- start 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.126+0000 7f9419794700 1 -- start start 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.126+0000 7f9419794700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9414072fe0 0x7f9414199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.126+0000 7f9419794700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9414073900 0x7f94141998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.126+0000 7f9419794700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f941410d7b0 0x7f941419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.126+0000 7f9419794700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941419e290 con 0x7f941410d7b0 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.126+0000 7f9419794700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941419e400 con 0x7f9414073900 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.126+0000 7f9419794700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941419e5c0 con 0x7f9414072fe0 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f94137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f941410d7b0 0x7f941419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f94127fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9414073900 0x7f94141998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f94137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f941410d7b0 0x7f941419dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51400/0 (socket says 172.21.15.46:51400) 2024-03-18T20:30:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f94137fe700 1 -- 172.21.15.46:0/2777255825 learned_addr learned my addr 172.21.15.46:0/2777255825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f9412ffd700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9414072fe0 0x7f9414199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f94137fe700 1 -- 172.21.15.46:0/2777255825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9414072fe0 msgr2=0x7f9414199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f94137fe700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9414072fe0 0x7f9414199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f94137fe700 1 -- 172.21.15.46:0/2777255825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9414073900 msgr2=0x7f94141998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f94137fe700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9414073900 0x7f94141998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f94137fe700 1 -- 172.21.15.46:0/2777255825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94040095f0 con 0x7f941410d7b0 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f94127fc700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9414073900 0x7f94141998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f94137fe700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f941410d7b0 0x7f941419dc80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f940c00ef90 tx=0x7f940c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f93fbfff700 1 -- 172.21.15.46:0/2777255825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c018070 con 0x7f941410d7b0 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f9419794700 1 -- 172.21.15.46:0/2777255825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941419e840 con 0x7f941410d7b0 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f93fbfff700 1 -- 172.21.15.46:0/2777255825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f940c00f040 con 0x7f941410d7b0 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f93fbfff700 1 -- 172.21.15.46:0/2777255825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c00cd60 con 0x7f941410d7b0 2024-03-18T20:30:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.127+0000 7f9419794700 1 -- 172.21.15.46:0/2777255825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94141a6100 con 0x7f941410d7b0 2024-03-18T20:30:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.128+0000 7f93fbfff700 1 -- 172.21.15.46:0/2777255825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f940c004000 con 0x7f941410d7b0 2024-03-18T20:30:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.129+0000 7f93fbfff700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93fc0389d0 0x7f93fc03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.129+0000 7f9419794700 1 -- 172.21.15.46:0/2777255825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94141089c0 con 0x7f941410d7b0 2024-03-18T20:30:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.129+0000 7f93fbfff700 1 -- 172.21.15.46:0/2777255825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f940c04ae00 con 0x7f941410d7b0 2024-03-18T20:30:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.129+0000 7f9412ffd700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93fc0389d0 0x7f93fc03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.129+0000 7f9412ffd700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93fc0389d0 0x7f93fc03ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f94040098e0 tx=0x7f9404005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:24.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.134+0000 7f93fbfff700 1 -- 172.21.15.46:0/2777255825 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f940c0051e0 con 0x7f941410d7b0 2024-03-18T20:30:24.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.359+0000 7f9419794700 1 -- 172.21.15.46:0/2777255825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9414074690 con 0x7f941410d7b0 2024-03-18T20:30:24.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.360+0000 7f93fbfff700 1 -- 172.21.15.46:0/2777255825 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f940c010660 con 0x7f941410d7b0 2024-03-18T20:30:24.364 INFO:teuthology.orchestra.run.smithi046.stdout:# minimal ceph.conf for 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:30:24.364 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-03-18T20:30:24.364 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:30:24.364 INFO:teuthology.orchestra.run.smithi046.stdout: mon_host = [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] 2024-03-18T20:30:24.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.362+0000 7f9419794700 1 -- 172.21.15.46:0/2777255825 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93fc0389d0 msgr2=0x7f93fc03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:24.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.362+0000 7f9419794700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93fc0389d0 0x7f93fc03ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f94040098e0 tx=0x7f9404005960 comp rx=0 tx=0).stop 2024-03-18T20:30:24.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.362+0000 7f9419794700 1 -- 172.21.15.46:0/2777255825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f941410d7b0 msgr2=0x7f941419dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:24.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.362+0000 7f9419794700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f941410d7b0 0x7f941419dc80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f940c00ef90 tx=0x7f940c00c3a0 comp rx=0 tx=0).stop 2024-03-18T20:30:24.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.362+0000 7f9419794700 1 -- 172.21.15.46:0/2777255825 shutdown_connections 2024-03-18T20:30:24.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.362+0000 7f9419794700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93fc0389d0 0x7f93fc03ae90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:24.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.362+0000 7f9419794700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f941410d7b0 0x7f941419dc80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:24.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.362+0000 7f9419794700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9414072fe0 0x7f9414199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:24.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.362+0000 7f9419794700 1 --2- 172.21.15.46:0/2777255825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9414073900 0x7f94141998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:24.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.362+0000 7f9419794700 1 -- 172.21.15.46:0/2777255825 >> 172.21.15.46:0/2777255825 conn(0x7f9414077f10 msgr2=0x7f94140fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:24.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.362+0000 7f9419794700 1 -- 172.21.15.46:0/2777255825 shutdown_connections 2024-03-18T20:30:24.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:24.362+0000 7f9419794700 1 -- 172.21.15.46:0/2777255825 wait complete. 2024-03-18T20:30:24.644 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:24 smithi155 ceph-mon[48344]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:24.644 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:24 smithi155 ceph-mon[48344]: Reconfiguring mon.b (monmap changed)... 2024-03-18T20:30:24.644 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:24 smithi155 ceph-mon[48344]: Reconfiguring daemon mon.b on smithi073 2024-03-18T20:30:24.644 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:24.644 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:24.645 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:24.645 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T20:30:24.645 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T20:30:24.645 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:24.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:24 smithi046 ceph-mon[43495]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:24.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:24 smithi046 ceph-mon[43495]: Reconfiguring mon.b (monmap changed)... 2024-03-18T20:30:24.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:24 smithi046 ceph-mon[43495]: Reconfiguring daemon mon.b on smithi073 2024-03-18T20:30:24.685 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:24.686 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:24.686 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:24.686 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T20:30:24.686 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T20:30:24.686 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:24.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:24 smithi073 ceph-mon[47384]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:24.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:24 smithi073 ceph-mon[47384]: Reconfiguring mon.b (monmap changed)... 2024-03-18T20:30:24.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:24 smithi073 ceph-mon[47384]: Reconfiguring daemon mon.b on smithi073 2024-03-18T20:30:24.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:30:24.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:24.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:24.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T20:30:24.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T20:30:24.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:24.768 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-18T20:30:24.769 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:30:24.769 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T20:30:24.806 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:30:24.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:30:24.881 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-18T20:30:24.882 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T20:30:24.916 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-18T20:30:24.916 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:30:24.985 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-18T20:30:24.985 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T20:30:25.016 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-18T20:30:25.016 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:30:25.083 INFO:tasks.cephadm:Adding mgr.x on smithi046 2024-03-18T20:30:25.083 INFO:tasks.cephadm:Adding mgr.y on smithi073 2024-03-18T20:30:25.083 INFO:tasks.cephadm:Adding mgr.z on smithi155 2024-03-18T20:30:25.084 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch apply mgr '3;smithi046=x;smithi073=y;smithi155=z' 2024-03-18T20:30:25.335 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T20:30:25.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:25 smithi073 ceph-mon[47384]: Reconfiguring mon.c (monmap changed)... 2024-03-18T20:30:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:25 smithi073 ceph-mon[47384]: Reconfiguring daemon mon.c on smithi155 2024-03-18T20:30:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:25 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2777255825' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:25.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:25 smithi155 ceph-mon[48344]: Reconfiguring mon.c (monmap changed)... 2024-03-18T20:30:25.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:25 smithi155 ceph-mon[48344]: Reconfiguring daemon mon.c on smithi155 2024-03-18T20:30:25.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:25 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2777255825' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:25.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:25.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:25.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:25.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:25.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:25.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:25.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:25 smithi046 ceph-mon[43495]: Reconfiguring mon.c (monmap changed)... 2024-03-18T20:30:25.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:25 smithi046 ceph-mon[43495]: Reconfiguring daemon mon.c on smithi155 2024-03-18T20:30:25.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:25 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2777255825' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:25.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:25.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:25.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:25.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:25.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:25.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:26.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.281+0000 7fc2946d5700 1 -- 172.21.15.155:0/1121684788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c0ff840 msgr2=0x7fc28c1093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:26.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.281+0000 7fc2946d5700 1 --2- 172.21.15.155:0/1121684788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c0ff840 0x7fc28c1093f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc28400b0a0 tx=0x7fc28400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:30:26.287 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.281+0000 7fc2946d5700 1 -- 172.21.15.155:0/1121684788 shutdown_connections 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.281+0000 7fc2946d5700 1 --2- 172.21.15.155:0/1121684788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28c102400 0x7fc28c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.281+0000 7fc2946d5700 1 --2- 172.21.15.155:0/1121684788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c0ff840 0x7fc28c1093f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.281+0000 7fc2946d5700 1 --2- 172.21.15.155:0/1121684788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc28c0fee80 0x7fc28c0ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.281+0000 7fc2946d5700 1 -- 172.21.15.155:0/1121684788 >> 172.21.15.155:0/1121684788 conn(0x7fc28c0faa40 msgr2=0x7fc28c0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.281+0000 7fc2946d5700 1 -- 172.21.15.155:0/1121684788 shutdown_connections 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.281+0000 7fc2946d5700 1 -- 172.21.15.155:0/1121684788 wait complete. 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.281+0000 7fc2946d5700 1 Processor -- start 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.281+0000 7fc2946d5700 1 -- start start 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc2946d5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28c0fee80 0x7fc28c101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc2946d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c0ff840 0x7fc28c100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc2946d5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc28c102400 0x7fc28c100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc2946d5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c100d70 con 0x7fc28c0fee80 2024-03-18T20:30:26.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc2946d5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c100ee0 con 0x7fc28c102400 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc2946d5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c1a0eb0 con 0x7fc28c0ff840 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc291c70700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c0ff840 0x7fc28c100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc291c70700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c0ff840 0x7fc28c100150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:56074/0 (socket says 172.21.15.155:56074) 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc291c70700 1 -- 172.21.15.155:0/224144923 learned_addr learned my addr 172.21.15.155:0/224144923 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc292471700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28c0fee80 0x7fc28c101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc292c72700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc28c102400 0x7fc28c100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc291c70700 1 -- 172.21.15.155:0/224144923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc28c102400 msgr2=0x7fc28c100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc291c70700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc28c102400 0x7fc28c100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc291c70700 1 -- 172.21.15.155:0/224144923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28c0fee80 msgr2=0x7fc28c101a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc291c70700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28c0fee80 0x7fc28c101a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc291c70700 1 -- 172.21.15.155:0/224144923 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc284009d00 con 0x7fc28c0ff840 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc292c72700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc28c102400 0x7fc28c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc291c70700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c0ff840 0x7fc28c100150 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc28800c970 tx=0x7fc28800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:26.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc27f7fe700 1 -- 172.21.15.155:0/224144923 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc288004220 con 0x7fc28c0ff840 2024-03-18T20:30:26.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc2946d5700 1 -- 172.21.15.155:0/224144923 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28c1a1110 con 0x7fc28c0ff840 2024-03-18T20:30:26.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc27f7fe700 1 -- 172.21.15.155:0/224144923 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc288007d60 con 0x7fc28c0ff840 2024-03-18T20:30:26.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc27f7fe700 1 -- 172.21.15.155:0/224144923 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc288003c90 con 0x7fc28c0ff840 2024-03-18T20:30:26.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.282+0000 7fc2946d5700 1 -- 172.21.15.155:0/224144923 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28c1a1630 con 0x7fc28c0ff840 2024-03-18T20:30:26.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.283+0000 7fc27f7fe700 1 -- 172.21.15.155:0/224144923 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fc2880078a0 con 0x7fc28c0ff840 2024-03-18T20:30:26.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.283+0000 7fc2946d5700 1 -- 172.21.15.155:0/224144923 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc28c108360 con 0x7fc28c0ff840 2024-03-18T20:30:26.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.283+0000 7fc27f7fe700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc2780389d0 0x7fc27803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:26.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.283+0000 7fc27f7fe700 1 -- 172.21.15.155:0/224144923 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc28804ae50 con 0x7fc28c0ff840 2024-03-18T20:30:26.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.284+0000 7fc292471700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc2780389d0 0x7fc27803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:26.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.284+0000 7fc292471700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc2780389d0 0x7fc27803ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc28c069180 tx=0x7fc28000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:26.292 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.288+0000 7fc27f7fe700 1 -- 172.21.15.155:0/224144923 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc288003df0 con 0x7fc28c0ff840 2024-03-18T20:30:26.494 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.490+0000 7fc2946d5700 1 -- 172.21.15.155:0/224144923 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi073=y;smithi155=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fc28c0611f0 con 0x7fc2780389d0 2024-03-18T20:30:26.500 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.497+0000 7fc27f7fe700 1 -- 172.21.15.155:0/224144923 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc28c0611f0 con 0x7fc2780389d0 2024-03-18T20:30:26.500 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mgr update... 2024-03-18T20:30:26.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.499+0000 7fc2946d5700 1 -- 172.21.15.155:0/224144923 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc2780389d0 msgr2=0x7fc27803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:26.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.499+0000 7fc2946d5700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc2780389d0 0x7fc27803ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc28c069180 tx=0x7fc28000b410 comp rx=0 tx=0).stop 2024-03-18T20:30:26.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.499+0000 7fc2946d5700 1 -- 172.21.15.155:0/224144923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c0ff840 msgr2=0x7fc28c100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:26.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.499+0000 7fc2946d5700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c0ff840 0x7fc28c100150 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc28800c970 tx=0x7fc28800cc80 comp rx=0 tx=0).stop 2024-03-18T20:30:26.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.499+0000 7fc2946d5700 1 -- 172.21.15.155:0/224144923 shutdown_connections 2024-03-18T20:30:26.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.499+0000 7fc2946d5700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc2780389d0 0x7fc27803ae90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:26.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.499+0000 7fc2946d5700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28c0fee80 0x7fc28c101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:26.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.499+0000 7fc2946d5700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc28c102400 0x7fc28c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:26.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.499+0000 7fc2946d5700 1 --2- 172.21.15.155:0/224144923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc28c0ff840 0x7fc28c100150 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:26.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.499+0000 7fc2946d5700 1 -- 172.21.15.155:0/224144923 >> 172.21.15.155:0/224144923 conn(0x7fc28c0faa40 msgr2=0x7fc28c1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:26.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.499+0000 7fc2946d5700 1 -- 172.21.15.155:0/224144923 shutdown_connections 2024-03-18T20:30:26.503 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:30:26.499+0000 7fc2946d5700 1 -- 172.21.15.155:0/224144923 wait complete. 2024-03-18T20:30:26.530 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:26 smithi155 ceph-mon[48344]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:26.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:26 smithi073 ceph-mon[47384]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:26.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:26 smithi046 ceph-mon[43495]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:26.883 DEBUG:teuthology.orchestra.run.smithi073:mgr.y> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.y.service 2024-03-18T20:30:26.886 DEBUG:teuthology.orchestra.run.smithi155:mgr.z> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.z.service 2024-03-18T20:30:26.888 INFO:tasks.cephadm:Deploying OSDs... 2024-03-18T20:30:26.888 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:30:26.888 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T20:30:26.911 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T20:30:26.911 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-03-18T20:30:26.917 INFO:journalctl@ceph.mgr.z.smithi155.stdout:-- Logs begin at Mon 2024-03-18 20:13:03 UTC. -- 2024-03-18T20:30:26.918 INFO:journalctl@ceph.mgr.y.smithi073.stdout:-- Logs begin at Mon 2024-03-18 20:12:50 UTC. -- 2024-03-18T20:30:26.973 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T20:30:26.973 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T20:30:26.974 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 171660 Links: 1 2024-03-18T20:30:26.974 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:30:26.974 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T20:30:26.974 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-03-18 20:29:46.601012408 +0000 2024-03-18T20:30:26.974 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-03-18 20:23:28.977044891 +0000 2024-03-18T20:30:26.974 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-03-18 20:23:28.977044891 +0000 2024-03-18T20:30:26.974 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-03-18T20:30:26.974 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T20:30:27.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-03-18T20:30:27.046 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-03-18T20:30:27.046 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000130839 s, 3.9 MB/s 2024-03-18T20:30:27.047 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T20:30:27.111 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-03-18T20:30:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T20:30:27.180 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T20:30:27.180 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 180586 Links: 1 2024-03-18T20:30:27.180 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:30:27.181 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T20:30:27.181 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-03-18 20:29:46.658011793 +0000 2024-03-18T20:30:27.181 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-03-18 20:23:29.385040706 +0000 2024-03-18T20:30:27.181 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-03-18 20:23:29.385040706 +0000 2024-03-18T20:30:27.181 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-03-18T20:30:27.183 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T20:30:27.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-03-18T20:30:27.269 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-03-18T20:30:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000113434 s, 4.5 MB/s 2024-03-18T20:30:27.273 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T20:30:27.342 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-03-18T20:30:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T20:30:27.406 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T20:30:27.406 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 177818 Links: 1 2024-03-18T20:30:27.406 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:30:27.406 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T20:30:27.406 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-03-18 20:29:46.714011190 +0000 2024-03-18T20:30:27.407 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-03-18 20:23:29.756036902 +0000 2024-03-18T20:30:27.407 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-03-18 20:23:29.756036902 +0000 2024-03-18T20:30:27.407 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-03-18T20:30:27.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T20:30:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-03-18T20:30:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-03-18T20:30:27.481 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000131461 s, 3.9 MB/s 2024-03-18T20:30:27.482 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T20:30:27.548 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-03-18T20:30:27.611 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T20:30:27.611 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T20:30:27.611 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 180684 Links: 1 2024-03-18T20:30:27.612 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:30:27.612 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T20:30:27.612 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-03-18 20:29:46.771010575 +0000 2024-03-18T20:30:27.612 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-03-18 20:23:30.138032985 +0000 2024-03-18T20:30:27.612 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-03-18 20:23:30.138032985 +0000 2024-03-18T20:30:27.612 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-03-18T20:30:27.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T20:30:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-03-18T20:30:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-03-18T20:30:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000124513 s, 4.1 MB/s 2024-03-18T20:30:27.686 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T20:30:27.749 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-18T20:30:27.750 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T20:30:27.771 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T20:30:27.771 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2024-03-18T20:30:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi073=y;smithi155=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:30:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: Saving service mgr spec with placement smithi046=x;smithi073=y;smithi155=z;count:3 2024-03-18T20:30:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T20:30:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T20:30:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T20:30:27.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:27.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: Deploying daemon mgr.z on smithi155 2024-03-18T20:30:27.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:27 smithi155 ceph-mon[48344]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi073=y;smithi155=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:30:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: Saving service mgr spec with placement smithi046=x;smithi073=y;smithi155=z;count:3 2024-03-18T20:30:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T20:30:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T20:30:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T20:30:27.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:27.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: Deploying daemon mgr.z on smithi155 2024-03-18T20:30:27.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:27 smithi046 ceph-mon[43495]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:27.830 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T20:30:27.831 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T20:30:27.831 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 6h/6d Inode: 181280 Links: 1 2024-03-18T20:30:27.831 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:30:27.831 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T20:30:27.831 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-03-18 20:30:15.191910968 +0000 2024-03-18T20:30:27.831 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-03-18 20:22:54.260126351 +0000 2024-03-18T20:30:27.831 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-03-18 20:22:54.260126351 +0000 2024-03-18T20:30:27.831 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-03-18T20:30:27.832 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T20:30:27.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi073=y;smithi155=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:30:27.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: Saving service mgr spec with placement smithi046=x;smithi073=y;smithi155=z;count:3 2024-03-18T20:30:27.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:27.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:27.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:27.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:27.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:27.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T20:30:27.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T20:30:27.899 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T20:30:27.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:27.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: Deploying daemon mgr.z on smithi155 2024-03-18T20:30:27.900 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:27 smithi073 ceph-mon[47384]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:27.901 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-03-18T20:30:27.901 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-03-18T20:30:27.901 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000212072 s, 2.4 MB/s 2024-03-18T20:30:27.903 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T20:30:27.964 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2024-03-18T20:30:28.024 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T20:30:28.025 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T20:30:28.025 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 6h/6d Inode: 175850 Links: 1 2024-03-18T20:30:28.025 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:30:28.025 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T20:30:28.025 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-03-18 20:30:15.191910968 +0000 2024-03-18T20:30:28.025 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-03-18 20:22:54.601118318 +0000 2024-03-18T20:30:28.025 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-03-18 20:22:54.601118318 +0000 2024-03-18T20:30:28.025 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-03-18T20:30:28.025 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T20:30:28.093 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-03-18T20:30:28.093 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-03-18T20:30:28.093 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000231419 s, 2.2 MB/s 2024-03-18T20:30:28.094 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T20:30:28.154 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2024-03-18T20:30:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T20:30:28.213 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T20:30:28.213 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 6h/6d Inode: 179559 Links: 1 2024-03-18T20:30:28.213 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:30:28.213 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T20:30:28.213 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-03-18 20:30:15.191910968 +0000 2024-03-18T20:30:28.213 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-03-18 20:22:54.948110145 +0000 2024-03-18T20:30:28.214 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-03-18 20:22:54.948110145 +0000 2024-03-18T20:30:28.214 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-03-18T20:30:28.214 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T20:30:28.281 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-03-18T20:30:28.281 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-03-18T20:30:28.281 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000197727 s, 2.6 MB/s 2024-03-18T20:30:28.282 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T20:30:28.340 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2024-03-18T20:30:28.398 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T20:30:28.398 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T20:30:28.398 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 6h/6d Inode: 181400 Links: 1 2024-03-18T20:30:28.398 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:30:28.398 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T20:30:28.398 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-03-18 20:30:15.191910968 +0000 2024-03-18T20:30:28.398 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-03-18 20:22:55.304101758 +0000 2024-03-18T20:30:28.398 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-03-18 20:22:55.304101758 +0000 2024-03-18T20:30:28.398 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-03-18T20:30:28.399 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T20:30:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-03-18T20:30:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-03-18T20:30:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000238099 s, 2.2 MB/s 2024-03-18T20:30:28.468 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T20:30:28.527 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-18T20:30:28.527 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T20:30:28.540 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:28 smithi155 bash[50194]: 9f876dfca0f2c330616f2d3a62311ce09ab9e9f1a58b006f197d7484f1dd48df 2024-03-18T20:30:28.540 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:28 smithi155 systemd[1]: Started Ceph mgr.z for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:30:28.546 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T20:30:28.546 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-03-18T20:30:28.604 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T20:30:28.604 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T20:30:28.604 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 172946 Links: 1 2024-03-18T20:30:28.604 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:30:28.604 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T20:30:28.604 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-03-18 20:30:27.751660491 +0000 2024-03-18T20:30:28.604 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-03-18 20:22:53.847272081 +0000 2024-03-18T20:30:28.604 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-03-18 20:22:53.847272081 +0000 2024-03-18T20:30:28.604 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-03-18T20:30:28.605 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T20:30:28.635 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:28 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:28.536+0000 7f49a4e42200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T20:30:28.665 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-03-18T20:30:28.665 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-03-18T20:30:28.665 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000260463 s, 2.0 MB/s 2024-03-18T20:30:28.666 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T20:30:28.724 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-03-18T20:30:28.781 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T20:30:28.782 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T20:30:28.782 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 180228 Links: 1 2024-03-18T20:30:28.782 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:30:28.782 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T20:30:28.782 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-03-18 20:30:27.751660491 +0000 2024-03-18T20:30:28.782 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-03-18 20:22:54.196265035 +0000 2024-03-18T20:30:28.782 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-03-18 20:22:54.196265035 +0000 2024-03-18T20:30:28.782 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-03-18T20:30:28.782 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T20:30:28.791 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:28 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:28.632+0000 7f49a4e42200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T20:30:28.791 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:28 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:28.724+0000 7f49a4e42200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T20:30:28.816 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-03-18T20:30:28.816 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-03-18T20:30:28.816 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000236149 s, 2.2 MB/s 2024-03-18T20:30:28.817 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T20:30:28.876 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-03-18T20:30:28.936 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T20:30:28.936 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T20:30:28.936 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 180320 Links: 1 2024-03-18T20:30:28.936 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:30:28.936 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T20:30:28.936 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-03-18 20:30:27.751660491 +0000 2024-03-18T20:30:28.936 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-03-18 20:22:54.532258251 +0000 2024-03-18T20:30:28.937 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-03-18 20:22:54.532258251 +0000 2024-03-18T20:30:28.937 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-03-18T20:30:28.937 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T20:30:29.000 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:28 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:28.813+0000 7f49a4e42200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T20:30:29.006 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-03-18T20:30:29.007 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-03-18T20:30:29.007 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000172483 s, 3.0 MB/s 2024-03-18T20:30:29.007 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T20:30:29.065 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-03-18T20:30:29.123 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T20:30:29.123 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T20:30:29.123 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 178945 Links: 1 2024-03-18T20:30:29.123 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:30:29.123 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T20:30:29.123 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-03-18 20:30:27.751660491 +0000 2024-03-18T20:30:29.123 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-03-18 20:22:54.877251286 +0000 2024-03-18T20:30:29.123 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-03-18 20:22:54.877251286 +0000 2024-03-18T20:30:29.124 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-03-18T20:30:29.124 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T20:30:29.189 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-03-18T20:30:29.189 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-03-18T20:30:29.189 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000198017 s, 2.6 MB/s 2024-03-18T20:30:29.190 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T20:30:29.254 INFO:tasks.cephadm:Deploying osd.0 on smithi046 with /dev/vg_nvme/lv_4... 2024-03-18T20:30:29.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T20:30:29.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:29.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:29.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:29.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T20:30:29.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T20:30:29.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T20:30:29.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:29.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:29 smithi155 ceph-mon[48344]: Deploying daemon mgr.y on smithi073 2024-03-18T20:30:29.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:29.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:29.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:29.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T20:30:29.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T20:30:29.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T20:30:29.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:29.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:29 smithi073 ceph-mon[47384]: Deploying daemon mgr.y on smithi073 2024-03-18T20:30:29.472 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:29 smithi073 systemd[1]: Starting Ceph mgr.y for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:30:29.505 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:30:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T20:30:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T20:30:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T20:30:29.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:29.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:29 smithi046 ceph-mon[43495]: Deploying daemon mgr.y on smithi073 2024-03-18T20:30:29.791 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:29 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:29.471+0000 7f49a4e42200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:30:29.791 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:29 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:29.721+0000 7f49a4e42200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T20:30:30.078 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:29 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:29.805+0000 7f49a4e42200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T20:30:30.078 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:29 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:29.915+0000 7f49a4e42200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T20:30:30.079 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:30 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:30.075+0000 7f49a4e42200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T20:30:30.220 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:29 smithi073 ceph-mgr[48882]: -- 172.21.15.73:0/1803538120 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5565a0a501a0 con 0x5565a0a26800 2024-03-18T20:30:30.221 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:29 smithi073 bash[48845]: 3f5f54dd80ecc4774dd52c2d72d51a546f6134de6ea4c39ce9291352066be75f 2024-03-18T20:30:30.221 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:29 smithi073 systemd[1]: Started Ceph mgr.y for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:30:30.269 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:30 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:30.242+0000 7f5ab682c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T20:30:30.535 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:30 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:30.340+0000 7f5ab682c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T20:30:30.535 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:30 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:30.440+0000 7f5ab682c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T20:30:30.535 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:30 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:30.530+0000 7f5ab682c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T20:30:30.535 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:30 smithi073 ceph-mon[47384]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:30.535 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:30.535 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:30.535 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:30.535 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:30.535 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:30.541 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:30 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:30.209+0000 7f49a4e42200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T20:30:30.541 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:30 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:30.299+0000 7f49a4e42200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T20:30:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:30 smithi155 ceph-mon[48344]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:30 smithi046 ceph-mon[43495]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:30.862 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T20:30:30.863 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T20:30:30.863 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-03-18T20:30:30.863 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-03-18T20:30:30.863 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113967 s, 920 MB/s 2024-03-18T20:30:30.863 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-03-18T20:30:31.041 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:30 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:30.717+0000 7f49a4e42200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T20:30:31.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi046:vg_nvme/lv_4 2024-03-18T20:30:31.408 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:31.115+0000 7f49a4e42200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T20:30:31.408 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:31.311+0000 7f49a4e42200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T20:30:31.470 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:31.189+0000 7f5ab682c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:30:31.471 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:31.437+0000 7f5ab682c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T20:30:31.786 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:31.521+0000 7f5ab682c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T20:30:31.786 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:31.627+0000 7f5ab682c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T20:30:31.786 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:31.782+0000 7f5ab682c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T20:30:31.791 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:31.404+0000 7f49a4e42200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T20:30:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:30:32.132 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.132 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.132 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:32.132 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.132 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.132 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:32.132 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:32.132 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.132 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: Reconfiguring mgr.x (unknown last config time)... 2024-03-18T20:30:32.133 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T20:30:32.133 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T20:30:32.133 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:32.133 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:31 smithi046 ceph-mon[43495]: Reconfiguring daemon mgr.x on smithi046 2024-03-18T20:30:32.168 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.169 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.169 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:32.169 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.169 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.169 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:32.169 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:32.169 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.169 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: Reconfiguring mgr.x (unknown last config time)... 2024-03-18T20:30:32.169 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T20:30:32.169 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T20:30:32.169 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:32.169 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:31 smithi155 ceph-mon[48344]: Reconfiguring daemon mgr.x on smithi046 2024-03-18T20:30:32.220 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:31.907+0000 7f5ab682c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T20:30:32.220 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:31.991+0000 7f5ab682c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T20:30:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: Reconfiguring mgr.x (unknown last config time)... 2024-03-18T20:30:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T20:30:32.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T20:30:32.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:32.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:31 smithi073 ceph-mon[47384]: Reconfiguring daemon mgr.x on smithi046 2024-03-18T20:30:32.541 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:32.165+0000 7f49a4e42200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T20:30:32.541 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:32.262+0000 7f49a4e42200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T20:30:32.541 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:32.414+0000 7f49a4e42200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T20:30:32.721 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:32 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:32.360+0000 7f5ab682c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T20:30:32.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.970+0000 7f02c37b7700 1 -- 172.21.15.46:0/3252061660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02bc100e00 msgr2=0x7f02bc101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:32.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.970+0000 7f02c37b7700 1 --2- 172.21.15.46:0/3252061660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02bc100e00 0x7f02bc101220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009990 tx=0x7f02b8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:30:32.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.971+0000 7f02c37b7700 1 -- 172.21.15.46:0/3252061660 shutdown_connections 2024-03-18T20:30:32.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.971+0000 7f02c37b7700 1 --2- 172.21.15.46:0/3252061660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02bc100e00 0x7f02bc101220 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:32.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.971+0000 7f02c37b7700 1 --2- 172.21.15.46:0/3252061660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02bc102920 0x7f02bc109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:32.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.971+0000 7f02c37b7700 1 --2- 172.21.15.46:0/3252061660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02bc101f60 0x7f02bc1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:32.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.971+0000 7f02c37b7700 1 -- 172.21.15.46:0/3252061660 >> 172.21.15.46:0/3252061660 conn(0x7f02bc0fc3a0 msgr2=0x7f02bc0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:32.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.971+0000 7f02c37b7700 1 -- 172.21.15.46:0/3252061660 shutdown_connections 2024-03-18T20:30:32.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.971+0000 7f02c37b7700 1 -- 172.21.15.46:0/3252061660 wait complete. 2024-03-18T20:30:32.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.971+0000 7f02c37b7700 1 Processor -- start 2024-03-18T20:30:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c37b7700 1 -- start start 2024-03-18T20:30:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c37b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02bc100e00 0x7f02bc196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c37b7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02bc101f60 0x7f02bc1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c37b7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02bc102920 0x7f02bc19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c37b7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc19e890 con 0x7f02bc100e00 2024-03-18T20:30:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c37b7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc19e9d0 con 0x7f02bc101f60 2024-03-18T20:30:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c37b7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc19eb90 con 0x7f02bc102920 2024-03-18T20:30:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c1553700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02bc100e00 0x7f02bc196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c1553700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02bc100e00 0x7f02bc196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37586/0 (socket says 172.21.15.46:37586) 2024-03-18T20:30:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c1553700 1 -- 172.21.15.46:0/4061329572 learned_addr learned my addr 172.21.15.46:0/4061329572 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:30:32.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c0d52700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02bc101f60 0x7f02bc1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c1d54700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02bc102920 0x7f02bc19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c1553700 1 -- 172.21.15.46:0/4061329572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02bc102920 msgr2=0x7f02bc19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c1553700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02bc102920 0x7f02bc19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c1553700 1 -- 172.21.15.46:0/4061329572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02bc101f60 msgr2=0x7f02bc1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c1553700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02bc101f60 0x7f02bc1972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c1553700 1 -- 172.21.15.46:0/4061329572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b80095f0 con 0x7f02bc100e00 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c1d54700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02bc102920 0x7f02bc19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c0d52700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02bc101f60 0x7f02bc1972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.972+0000 7f02c1553700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02bc100e00 0x7f02bc196d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02b800b620 tx=0x7f02b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.973+0000 7f02b27fc700 1 -- 172.21.15.46:0/4061329572 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b801d070 con 0x7f02bc100e00 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.973+0000 7f02c37b7700 1 -- 172.21.15.46:0/4061329572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02bc19ee10 con 0x7f02bc100e00 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.973+0000 7f02b27fc700 1 -- 172.21.15.46:0/4061329572 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02b8022470 con 0x7f02bc100e00 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.973+0000 7f02b27fc700 1 -- 172.21.15.46:0/4061329572 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8017650 con 0x7f02bc100e00 2024-03-18T20:30:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.973+0000 7f02c37b7700 1 -- 172.21.15.46:0/4061329572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02bc19f170 con 0x7f02bc100e00 2024-03-18T20:30:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.974+0000 7f02b27fc700 1 -- 172.21.15.46:0/4061329572 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f02b80177b0 con 0x7f02bc100e00 2024-03-18T20:30:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.974+0000 7f02c37b7700 1 -- 172.21.15.46:0/4061329572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02bc1082a0 con 0x7f02bc100e00 2024-03-18T20:30:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.974+0000 7f02b27fc700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f02a80389d0 0x7f02a803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.974+0000 7f02b27fc700 1 -- 172.21.15.46:0/4061329572 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f02b804c910 con 0x7f02bc100e00 2024-03-18T20:30:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.974+0000 7f02c0d52700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f02a80389d0 0x7f02a803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.974+0000 7f02c0d52700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f02a80389d0 0x7f02a803ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f02ac011490 tx=0x7f02ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:32.979+0000 7f02b27fc700 1 -- 172.21.15.46:0/4061329572 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f02b802a360 con 0x7f02bc100e00 2024-03-18T20:30:33.026 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:32 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:32.740+0000 7f5ab682c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T20:30:33.026 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:32 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:32.931+0000 7f5ab682c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T20:30:33.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:33 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:33.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:33 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:33.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:33 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:33.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:33 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:33.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:33 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:33.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:33 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:33.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:33 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:33.203+0000 7f02c37b7700 1 -- 172.21.15.46:0/4061329572 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f02bc0611f0 con 0x7f02a80389d0 2024-03-18T20:30:33.291 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:33 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:33.044+0000 7f49a4e42200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T20:30:33.291 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:33 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:33.226+0000 7f49a4e42200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T20:30:33.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:33 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:33.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:33 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:33.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:33 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:33.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:33 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:33.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:33 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:33.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:33 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:33.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:33 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:33.470 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:33 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:33.022+0000 7f5ab682c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T20:30:33.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:33 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:33.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:33 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:33.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:33 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:33.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:33 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:33.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:33 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:33.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:33 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:33.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:33 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:33.791 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:33 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:33.420+0000 7f49a4e42200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T20:30:34.220 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:33 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:33.779+0000 7f5ab682c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T20:30:34.221 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:33 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:33.873+0000 7f5ab682c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T20:30:34.221 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:34 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:34.019+0000 7f5ab682c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T20:30:34.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:34 smithi073 ceph-mon[47384]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:34.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:34 smithi073 ceph-mon[47384]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:30:34.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:34 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:30:34.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:34 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:30:34.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:34 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:34.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:34 smithi046 ceph-mon[43495]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:34.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:34 smithi046 ceph-mon[43495]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:30:34.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:34 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:30:34.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:34 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:30:34.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:34 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:34 smithi155 ceph-mon[48344]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:34 smithi155 ceph-mon[48344]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:30:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:34 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:30:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:34 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:30:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:34 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:34.631 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:34 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:34.344+0000 7f49a4e42200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T20:30:34.909 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:34 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:34.628+0000 7f49a4e42200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:30:34.909 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:34 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:34.716+0000 7f49a4e42200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T20:30:34.970 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:34 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:34.636+0000 7f5ab682c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T20:30:34.970 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:34 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:34.816+0000 7f5ab682c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T20:30:35.291 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:30:34 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:30:34.905+0000 7f49a4e42200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T20:30:35.470 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:35 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:35.005+0000 7f5ab682c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T20:30:35.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:35.716+0000 7f02b27fc700 1 -- 172.21.15.46:0/4061329572 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 14) v1 ==== 90699+0+0 (secure 0 0 0) 0x7f02b802c960 con 0x7f02bc100e00 2024-03-18T20:30:35.970 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:35 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:35.919+0000 7f5ab682c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T20:30:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:35 smithi073 ceph-mon[47384]: Standby manager daemon z started 2024-03-18T20:30:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:35 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.155:0/2840411137' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T20:30:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:35 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.155:0/2840411137' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T20:30:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:35 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.155:0/2840411137' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T20:30:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:35 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.155:0/2840411137' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T20:30:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:35 smithi073 ceph-mon[47384]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:35 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3327781975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "303e69be-0b32-4cff-b91e-ce6665f3a087"}]: dispatch 2024-03-18T20:30:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:35 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3327781975' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "303e69be-0b32-4cff-b91e-ce6665f3a087"}]': finished 2024-03-18T20:30:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:35 smithi073 ceph-mon[47384]: osdmap e5: 1 total, 0 up, 1 in 2024-03-18T20:30:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:35 smithi155 ceph-mon[48344]: Standby manager daemon z started 2024-03-18T20:30:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:35 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.155:0/2840411137' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T20:30:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:35 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.155:0/2840411137' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T20:30:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:35 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.155:0/2840411137' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T20:30:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:35 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.155:0/2840411137' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T20:30:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:35 smithi155 ceph-mon[48344]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:36.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:35 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3327781975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "303e69be-0b32-4cff-b91e-ce6665f3a087"}]: dispatch 2024-03-18T20:30:36.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:35 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3327781975' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "303e69be-0b32-4cff-b91e-ce6665f3a087"}]': finished 2024-03-18T20:30:36.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:35 smithi155 ceph-mon[48344]: osdmap e5: 1 total, 0 up, 1 in 2024-03-18T20:30:36.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:35 smithi046 ceph-mon[43495]: Standby manager daemon z started 2024-03-18T20:30:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:35 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.155:0/2840411137' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T20:30:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:35 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.155:0/2840411137' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T20:30:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:35 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.155:0/2840411137' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T20:30:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:35 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.155:0/2840411137' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T20:30:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:35 smithi046 ceph-mon[43495]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:35 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3327781975' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "303e69be-0b32-4cff-b91e-ce6665f3a087"}]: dispatch 2024-03-18T20:30:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:35 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3327781975' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "303e69be-0b32-4cff-b91e-ce6665f3a087"}]': finished 2024-03-18T20:30:36.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:35 smithi046 ceph-mon[43495]: osdmap e5: 1 total, 0 up, 1 in 2024-03-18T20:30:36.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:36.470 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:36 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:36.200+0000 7f5ab682c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:30:36.471 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:36 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:36.287+0000 7f5ab682c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T20:30:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:36.721+0000 7f02b27fc700 1 -- 172.21.15.46:0/4061329572 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7f02b802ac20 con 0x7f02bc100e00 2024-03-18T20:30:36.729 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:30:36 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:30:36.475+0000 7f5ab682c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T20:30:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:36 smithi155 ceph-mon[48344]: mgrmap e14: x(active, since 58s), standbys: z 2024-03-18T20:30:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T20:30:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:36 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2523308585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:30:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:36 smithi155 ceph-mon[48344]: Standby manager daemon y started 2024-03-18T20:30:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:36 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.73:0/2071005844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T20:30:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:36 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.73:0/2071005844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T20:30:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:36 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.73:0/2071005844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T20:30:37.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:36 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.73:0/2071005844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T20:30:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:36 smithi046 ceph-mon[43495]: mgrmap e14: x(active, since 58s), standbys: z 2024-03-18T20:30:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T20:30:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:36 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2523308585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:30:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:36 smithi046 ceph-mon[43495]: Standby manager daemon y started 2024-03-18T20:30:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:36 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.73:0/2071005844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T20:30:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:36 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.73:0/2071005844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T20:30:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:36 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.73:0/2071005844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T20:30:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:36 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.73:0/2071005844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T20:30:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:36 smithi073 ceph-mon[47384]: mgrmap e14: x(active, since 58s), standbys: z 2024-03-18T20:30:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T20:30:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:36 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2523308585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:30:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:36 smithi073 ceph-mon[47384]: Standby manager daemon y started 2024-03-18T20:30:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:36 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.73:0/2071005844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T20:30:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:36 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.73:0/2071005844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T20:30:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:36 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.73:0/2071005844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T20:30:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:36 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.73:0/2071005844' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T20:30:38.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:37 smithi155 ceph-mon[48344]: mgrmap e15: x(active, since 59s), standbys: z, y 2024-03-18T20:30:38.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:37 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T20:30:38.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:37 smithi155 ceph-mon[48344]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:38.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:37 smithi046 ceph-mon[43495]: mgrmap e15: x(active, since 59s), standbys: z, y 2024-03-18T20:30:38.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T20:30:38.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:37 smithi046 ceph-mon[43495]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:38.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:37 smithi073 ceph-mon[47384]: mgrmap e15: x(active, since 59s), standbys: z, y 2024-03-18T20:30:38.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:37 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T20:30:38.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:37 smithi073 ceph-mon[47384]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:40.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:40 smithi155 ceph-mon[48344]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:40.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:40 smithi046 ceph-mon[43495]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:40.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:40 smithi073 ceph-mon[47384]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:42.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:42 smithi046 ceph-mon[43495]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:42.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:42 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T20:30:42.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:42 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:42.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:42 smithi046 ceph-mon[43495]: Deploying daemon osd.0 on smithi046 2024-03-18T20:30:42.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:42 smithi155 ceph-mon[48344]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:42.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:42 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T20:30:42.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:42 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:42.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:42 smithi155 ceph-mon[48344]: Deploying daemon osd.0 on smithi046 2024-03-18T20:30:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:42 smithi073 ceph-mon[47384]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:42.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:42 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T20:30:42.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:42 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:42.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:42 smithi073 ceph-mon[47384]: Deploying daemon osd.0 on smithi046 2024-03-18T20:30:44.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:44 smithi155 ceph-mon[48344]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:44.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:44 smithi046 ceph-mon[43495]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:44 smithi073 ceph-mon[47384]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:46.156 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:45 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:46.156 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:45 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:46.156 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:45 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:46.156 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:45 smithi046 ceph-mon[43495]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:46.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:45 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:46.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:45 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:46.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:45 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:46.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:45 smithi073 ceph-mon[47384]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:46.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:45 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:46.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:45 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:46.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:45 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:46.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:45 smithi155 ceph-mon[48344]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:47.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.050+0000 7f02b27fc700 1 -- 172.21.15.46:0/4061329572 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f02bc0611f0 con 0x7f02a80389d0 2024-03-18T20:30:47.054 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 0 on host 'smithi046' 2024-03-18T20:30:47.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.053+0000 7f02c37b7700 1 -- 172.21.15.46:0/4061329572 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f02a80389d0 msgr2=0x7f02a803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:47.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.053+0000 7f02c37b7700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f02a80389d0 0x7f02a803ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f02ac011490 tx=0x7f02ac009260 comp rx=0 tx=0).stop 2024-03-18T20:30:47.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.053+0000 7f02c37b7700 1 -- 172.21.15.46:0/4061329572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02bc100e00 msgr2=0x7f02bc196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:47.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.053+0000 7f02c37b7700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02bc100e00 0x7f02bc196d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02b800b620 tx=0x7f02b8004730 comp rx=0 tx=0).stop 2024-03-18T20:30:47.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.053+0000 7f02c37b7700 1 -- 172.21.15.46:0/4061329572 shutdown_connections 2024-03-18T20:30:47.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.053+0000 7f02c37b7700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f02a80389d0 0x7f02a803ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:47.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.053+0000 7f02c37b7700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02bc100e00 0x7f02bc196d90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:47.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.053+0000 7f02c37b7700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02bc102920 0x7f02bc19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:47.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.053+0000 7f02c37b7700 1 --2- 172.21.15.46:0/4061329572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02bc101f60 0x7f02bc1972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:47.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.053+0000 7f02c37b7700 1 -- 172.21.15.46:0/4061329572 >> 172.21.15.46:0/4061329572 conn(0x7f02bc0fc3a0 msgr2=0x7f02bc1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:47.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.053+0000 7f02c37b7700 1 -- 172.21.15.46:0/4061329572 shutdown_connections 2024-03-18T20:30:47.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:30:47.053+0000 7f02c37b7700 1 -- 172.21.15.46:0/4061329572 wait complete. 2024-03-18T20:30:47.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:47.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:47.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:47.421 DEBUG:teuthology.orchestra.run.smithi046:osd.0> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.0.service 2024-03-18T20:30:47.423 INFO:tasks.cephadm:Deploying osd.1 on smithi073 with /dev/vg_nvme/lv_4... 2024-03-18T20:30:47.423 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T20:30:47.462 INFO:journalctl@ceph.osd.0.smithi046.stdout:-- Logs begin at Mon 2024-03-18 20:12:57 UTC. -- 2024-03-18T20:30:47.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:47.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:47.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T20:30:47.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:47.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:47.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:47.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:30:48.294 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:48 smithi046 ceph-mon[43495]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:48.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:48 smithi073 ceph-mon[47384]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:48.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:48 smithi155 ceph-mon[48344]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:48.988 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T20:30:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T20:30:48.989 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10+0 records in 2024-03-18T20:30:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:10+0 records out 2024-03-18T20:30:48.989 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112756 s, 930 MB/s 2024-03-18T20:30:48.989 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping successful for: 2024-03-18T20:30:49.254 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 20:30:48 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-0[61199]: 2024-03-18T20:30:48.863+0000 7fc445939640 -1 osd.0 0 log_to_monitors true 2024-03-18T20:30:49.317 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:49 smithi073 ceph-mon[47384]: from='osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T20:30:49.337 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi073:vg_nvme/lv_4 2024-03-18T20:30:49.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:49 smithi155 ceph-mon[48344]: from='osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T20:30:49.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:49 smithi046 ceph-mon[43495]: from='osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T20:30:49.551 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T20:30:50.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.514+0000 7fbde7c38700 1 -- 172.21.15.73:0/3563342108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde01041b0 msgr2=0x7fbde01065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.514+0000 7fbde7c38700 1 --2- 172.21.15.73:0/3563342108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde01041b0 0x7fbde01065a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009990 tx=0x7fbddc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.514+0000 7fbde7c38700 1 -- 172.21.15.73:0/3563342108 shutdown_connections 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.514+0000 7fbde7c38700 1 --2- 172.21.15.73:0/3563342108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0106ae0 0x7fbde0108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.514+0000 7fbde7c38700 1 --2- 172.21.15.73:0/3563342108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbde0101880 0x7fbde0103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.514+0000 7fbde7c38700 1 --2- 172.21.15.73:0/3563342108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde01041b0 0x7fbde01065a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.514+0000 7fbde7c38700 1 -- 172.21.15.73:0/3563342108 >> 172.21.15.73:0/3563342108 conn(0x7fbde00fb2b0 msgr2=0x7fbde00fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.514+0000 7fbde7c38700 1 -- 172.21.15.73:0/3563342108 shutdown_connections 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.514+0000 7fbde7c38700 1 -- 172.21.15.73:0/3563342108 wait complete. 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.514+0000 7fbde7c38700 1 Processor -- start 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.514+0000 7fbde7c38700 1 -- start start 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde7c38700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbde0101880 0x7fbde0196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde7c38700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde01041b0 0x7fbde01973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde7c38700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde0106ae0 0x7fbde019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:50.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde7c38700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde019e9a0 con 0x7fbde01041b0 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde7c38700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde019eae0 con 0x7fbde0106ae0 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde7c38700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde019ec70 con 0x7fbde0101880 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde61d5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde0106ae0 0x7fbde019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde61d5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde0106ae0 0x7fbde019e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42706/0 (socket says 172.21.15.73:42706) 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde61d5700 1 -- 172.21.15.73:0/3955894606 learned_addr learned my addr 172.21.15.73:0/3955894606 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde51d3700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde01041b0 0x7fbde01973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde59d4700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbde0101880 0x7fbde0196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde61d5700 1 -- 172.21.15.73:0/3955894606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbde0101880 msgr2=0x7fbde0196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde61d5700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbde0101880 0x7fbde0196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde61d5700 1 -- 172.21.15.73:0/3955894606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde01041b0 msgr2=0x7fbde01973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde61d5700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde01041b0 0x7fbde01973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde61d5700 1 -- 172.21.15.73:0/3955894606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc0095f0 con 0x7fbde0106ae0 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde61d5700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde0106ae0 0x7fbde019e3d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800b9e0 tx=0x7fbdd800bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbdd6ffd700 1 -- 172.21.15.73:0/3955894606 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800c930 con 0x7fbde0106ae0 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbdd6ffd700 1 -- 172.21.15.73:0/3955894606 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdd8014440 con 0x7fbde0106ae0 2024-03-18T20:30:50.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde7c38700 1 -- 172.21.15.73:0/3955894606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde019ef50 con 0x7fbde0106ae0 2024-03-18T20:30:50.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbdd6ffd700 1 -- 172.21.15.73:0/3955894606 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80125a0 con 0x7fbde0106ae0 2024-03-18T20:30:50.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.515+0000 7fbde7c38700 1 -- 172.21.15.73:0/3955894606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde019f370 con 0x7fbde0106ae0 2024-03-18T20:30:50.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.516+0000 7fbde7c38700 1 -- 172.21.15.73:0/3955894606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde00fce30 con 0x7fbde0106ae0 2024-03-18T20:30:50.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.517+0000 7fbdd6ffd700 1 -- 172.21.15.73:0/3955894606 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7fbdd80145b0 con 0x7fbde0106ae0 2024-03-18T20:30:50.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.518+0000 7fbdd6ffd700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbdcc0a1090 0x7fbdcc0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:30:50.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.518+0000 7fbdd6ffd700 1 -- 172.21.15.73:0/3955894606 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7fbdd8006e40 con 0x7fbde0106ae0 2024-03-18T20:30:50.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.518+0000 7fbde59d4700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbdcc0a1090 0x7fbdcc0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:30:50.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.519+0000 7fbde59d4700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbdcc0a1090 0x7fbdcc0a3550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0006fd0 tx=0x7fbdd0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:30:50.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.522+0000 7fbdd6ffd700 1 -- 172.21.15.73:0/3955894606 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbdd8095610 con 0x7fbde0106ae0 2024-03-18T20:30:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:50 smithi155 ceph-mon[48344]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:50 smithi155 ceph-mon[48344]: from='osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T20:30:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:50 smithi155 ceph-mon[48344]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T20:30:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:50 smithi155 ceph-mon[48344]: from='osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-03-18T20:30:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:30:50.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:50.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:50.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:50.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:50.547 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-0[61199]: 2024-03-18T20:30:50.259+0000 7fc43a79c700 -1 osd.0 0 waiting for initial osdmap 2024-03-18T20:30:50.547 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-0[61199]: 2024-03-18T20:30:50.262+0000 7fc43458d700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T20:30:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-mon[43495]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-mon[43495]: from='osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T20:30:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-mon[43495]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T20:30:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-mon[43495]: from='osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-03-18T20:30:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:30:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:50.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:50.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:50.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:50.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:50 smithi073 ceph-mon[47384]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:50.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:50 smithi073 ceph-mon[47384]: from='osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T20:30:50.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:50 smithi073 ceph-mon[47384]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T20:30:50.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:50.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:50 smithi073 ceph-mon[47384]: from='osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-03-18T20:30:50.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:50.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:50.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:30:50.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:50.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:50.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:30:50.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:30:50.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:30:50.726+0000 7fbde7c38700 1 -- 172.21.15.73:0/3955894606 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbde00611f0 con 0x7fbdcc0a1090 2024-03-18T20:30:51.447 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:51 smithi073 ceph-mon[47384]: Detected new or changed devices on smithi046 2024-03-18T20:30:51.447 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:51 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi046 to 17118M 2024-03-18T20:30:51.447 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:51 smithi073 ceph-mon[47384]: from='osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-03-18T20:30:51.447 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:51 smithi073 ceph-mon[47384]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T20:30:51.447 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:51 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:51.447 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:51 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:51.447 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:51 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:30:51.447 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:51 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:30:51.447 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:51 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:51.447 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:51 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:51 smithi155 ceph-mon[48344]: Detected new or changed devices on smithi046 2024-03-18T20:30:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:51 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi046 to 17118M 2024-03-18T20:30:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:51 smithi155 ceph-mon[48344]: from='osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-03-18T20:30:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:51 smithi155 ceph-mon[48344]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T20:30:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:51 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:51 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:51 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:30:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:51 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:30:51.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:51 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:51.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:51 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:51 smithi046 ceph-mon[43495]: Detected new or changed devices on smithi046 2024-03-18T20:30:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:51 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi046 to 17118M 2024-03-18T20:30:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:51 smithi046 ceph-mon[43495]: from='osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-03-18T20:30:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:51 smithi046 ceph-mon[43495]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T20:30:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:51 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:51 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:51 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:30:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:51 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:30:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:51 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:51 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:52 smithi155 ceph-mon[48344]: purged_snaps scrub starts 2024-03-18T20:30:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:52 smithi155 ceph-mon[48344]: purged_snaps scrub ok 2024-03-18T20:30:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:52 smithi155 ceph-mon[48344]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:30:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:52 smithi155 ceph-mon[48344]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:52 smithi155 ceph-mon[48344]: osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490] boot 2024-03-18T20:30:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:52 smithi155 ceph-mon[48344]: osdmap e8: 1 total, 1 up, 1 in 2024-03-18T20:30:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:52 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:52 smithi046 ceph-mon[43495]: purged_snaps scrub starts 2024-03-18T20:30:52.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:52 smithi046 ceph-mon[43495]: purged_snaps scrub ok 2024-03-18T20:30:52.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:52 smithi046 ceph-mon[43495]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:30:52.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:52 smithi046 ceph-mon[43495]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:52.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:52 smithi046 ceph-mon[43495]: osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490] boot 2024-03-18T20:30:52.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:52 smithi046 ceph-mon[43495]: osdmap e8: 1 total, 1 up, 1 in 2024-03-18T20:30:52.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:52 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:52.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:52 smithi073 ceph-mon[47384]: purged_snaps scrub starts 2024-03-18T20:30:52.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:52 smithi073 ceph-mon[47384]: purged_snaps scrub ok 2024-03-18T20:30:52.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:52 smithi073 ceph-mon[47384]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:30:52.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:52 smithi073 ceph-mon[47384]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T20:30:52.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:52 smithi073 ceph-mon[47384]: osd.0 [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490] boot 2024-03-18T20:30:52.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:52 smithi073 ceph-mon[47384]: osdmap e8: 1 total, 1 up, 1 in 2024-03-18T20:30:52.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:52 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:30:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:53 smithi155 ceph-mon[48344]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T20:30:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:53 smithi155 ceph-mon[48344]: from='client.? 172.21.15.73:0/2276674597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7b934a7-0679-4158-bc78-6f8cc7d107c9"}]: dispatch 2024-03-18T20:30:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:53 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7b934a7-0679-4158-bc78-6f8cc7d107c9"}]: dispatch 2024-03-18T20:30:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:53 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7b934a7-0679-4158-bc78-6f8cc7d107c9"}]': finished 2024-03-18T20:30:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:53 smithi155 ceph-mon[48344]: osdmap e10: 2 total, 1 up, 2 in 2024-03-18T20:30:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:53 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:30:53.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:53 smithi155 ceph-mon[48344]: from='client.? 172.21.15.73:0/3444623263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:30:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:53 smithi046 ceph-mon[43495]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T20:30:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:53 smithi046 ceph-mon[43495]: from='client.? 172.21.15.73:0/2276674597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7b934a7-0679-4158-bc78-6f8cc7d107c9"}]: dispatch 2024-03-18T20:30:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:53 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7b934a7-0679-4158-bc78-6f8cc7d107c9"}]: dispatch 2024-03-18T20:30:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:53 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7b934a7-0679-4158-bc78-6f8cc7d107c9"}]': finished 2024-03-18T20:30:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:53 smithi046 ceph-mon[43495]: osdmap e10: 2 total, 1 up, 2 in 2024-03-18T20:30:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:30:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:53 smithi046 ceph-mon[43495]: from='client.? 172.21.15.73:0/3444623263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:30:53.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:53 smithi073 ceph-mon[47384]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T20:30:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:53 smithi073 ceph-mon[47384]: from='client.? 172.21.15.73:0/2276674597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7b934a7-0679-4158-bc78-6f8cc7d107c9"}]: dispatch 2024-03-18T20:30:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:53 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7b934a7-0679-4158-bc78-6f8cc7d107c9"}]: dispatch 2024-03-18T20:30:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:53 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7b934a7-0679-4158-bc78-6f8cc7d107c9"}]': finished 2024-03-18T20:30:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:53 smithi073 ceph-mon[47384]: osdmap e10: 2 total, 1 up, 2 in 2024-03-18T20:30:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:53 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:30:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:53 smithi073 ceph-mon[47384]: from='client.? 172.21.15.73:0/3444623263' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:30:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:54 smithi155 ceph-mon[48344]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:30:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:54 smithi046 ceph-mon[43495]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:30:54.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:54 smithi073 ceph-mon[47384]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:30:56.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:55 smithi155 ceph-mon[48344]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:30:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:55 smithi046 ceph-mon[43495]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:30:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:55 smithi073 ceph-mon[47384]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:30:58.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:58 smithi073 ceph-mon[47384]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:30:58.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:58 smithi155 ceph-mon[48344]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:30:58.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:58 smithi046 ceph-mon[43495]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:30:59.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:59 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T20:30:59.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:30:59 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:59.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:59 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T20:30:59.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:30:59 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:30:59.388 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:59 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T20:30:59.388 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:30:59 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:00.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:00 smithi155 ceph-mon[48344]: Deploying daemon osd.1 on smithi073 2024-03-18T20:31:00.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:00 smithi155 ceph-mon[48344]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:00.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:00 smithi046 ceph-mon[43495]: Deploying daemon osd.1 on smithi073 2024-03-18T20:31:00.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:00 smithi046 ceph-mon[43495]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:00.357 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:00 smithi073 ceph-mon[47384]: Deploying daemon osd.1 on smithi073 2024-03-18T20:31:00.357 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:00 smithi073 ceph-mon[47384]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:02.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:02 smithi155 ceph-mon[48344]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:02.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:02 smithi046 ceph-mon[43495]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:02.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:02 smithi073 ceph-mon[47384]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:03 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:03 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:03 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:03 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:03 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:03 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:03.688 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:03 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:03.688 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:03 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:03.688 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:03 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:04.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:04 smithi073 ceph-mon[47384]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:04.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:04.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:04.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:04.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:04.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:04 smithi046 ceph-mon[43495]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:04.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:04.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.632+0000 7fbdd6ffd700 1 -- 172.21.15.73:0/3955894606 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbde00611f0 con 0x7fbdcc0a1090 2024-03-18T20:31:04.638 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 1 on host 'smithi073' 2024-03-18T20:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.634+0000 7fbde7c38700 1 -- 172.21.15.73:0/3955894606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbdcc0a1090 msgr2=0x7fbdcc0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.634+0000 7fbde7c38700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbdcc0a1090 0x7fbdcc0a3550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0006fd0 tx=0x7fbdd0008040 comp rx=0 tx=0).stop 2024-03-18T20:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.634+0000 7fbde7c38700 1 -- 172.21.15.73:0/3955894606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde0106ae0 msgr2=0x7fbde019e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.634+0000 7fbde7c38700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde0106ae0 0x7fbde019e3d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800b9e0 tx=0x7fbdd800bcf0 comp rx=0 tx=0).stop 2024-03-18T20:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.634+0000 7fbde7c38700 1 -- 172.21.15.73:0/3955894606 shutdown_connections 2024-03-18T20:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.634+0000 7fbde7c38700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbdcc0a1090 0x7fbdcc0a3550 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.634+0000 7fbde7c38700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbde0106ae0 0x7fbde019e3d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.634+0000 7fbde7c38700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbde0101880 0x7fbde0196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.634+0000 7fbde7c38700 1 --2- 172.21.15.73:0/3955894606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde01041b0 0x7fbde01973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.634+0000 7fbde7c38700 1 -- 172.21.15.73:0/3955894606 >> 172.21.15.73:0/3955894606 conn(0x7fbde00fb2b0 msgr2=0x7fbde01021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:31:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.634+0000 7fbde7c38700 1 -- 172.21.15.73:0/3955894606 shutdown_connections 2024-03-18T20:31:04.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:04.635+0000 7fbde7c38700 1 -- 172.21.15.73:0/3955894606 wait complete. 2024-03-18T20:31:04.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:04 smithi155 ceph-mon[48344]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:04.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:04.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:04.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:04.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:04.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:05.028 DEBUG:teuthology.orchestra.run.smithi073:osd.1> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.1.service 2024-03-18T20:31:05.030 INFO:tasks.cephadm:Deploying osd.2 on smithi155 with /dev/vg_nvme/lv_4... 2024-03-18T20:31:05.030 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T20:31:05.062 INFO:journalctl@ceph.osd.1.smithi073.stdout:-- Logs begin at Mon 2024-03-18 20:12:50 UTC. -- 2024-03-18T20:31:05.258 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T20:31:05.674 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:05 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:05.675 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:05 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:05.675 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:05 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:05.675 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:05 smithi155 ceph-mon[48344]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:05.825 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:05 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:05.826 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:05 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:05.826 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:05 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:05.826 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:05 smithi073 ceph-mon[47384]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:06.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:05 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:06.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:05 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:06.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:05 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:06.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:05 smithi046 ceph-mon[43495]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:06.563 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T20:31:06.563 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T20:31:06.563 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-03-18T20:31:06.563 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-03-18T20:31:06.563 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111667 s, 939 MB/s 2024-03-18T20:31:06.563 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-03-18T20:31:06.721 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 20:31:06 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-1[52371]: 2024-03-18T20:31:06.283+0000 7ff87c70b640 -1 osd.1 0 log_to_monitors true 2024-03-18T20:31:06.885 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:06 smithi155 ceph-mon[48344]: from='osd.1 [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T20:31:06.885 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:06 smithi155 ceph-mon[48344]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T20:31:06.886 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2024-03-18T20:31:07.023 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:06 smithi073 ceph-mon[47384]: from='osd.1 [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T20:31:07.023 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:06 smithi073 ceph-mon[47384]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T20:31:07.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:06 smithi046 ceph-mon[43495]: from='osd.1 [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T20:31:07.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:06 smithi046 ceph-mon[43495]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T20:31:07.101 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T20:31:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T20:31:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T20:31:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:31:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: from='osd.1 [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:31:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:31:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: Detected new or changed devices on smithi073 2024-03-18T20:31:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:31:07.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi073 to 17113M 2024-03-18T20:31:07.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:07.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:07.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:07.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:08.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T20:31:08.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T20:31:08.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:31:08.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: from='osd.1 [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:31:08.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:31:08.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:08.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: Detected new or changed devices on smithi073 2024-03-18T20:31:08.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:08.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:08.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:31:08.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi073 to 17113M 2024-03-18T20:31:08.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:08.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:08.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:08.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:08.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 -- 172.21.15.155:0/4097912743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f0101890 msgr2=0x7f81f0108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:08.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 --2- 172.21.15.155:0/4097912743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f0101890 0x7f81f0108140 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f81e800b0a0 tx=0x7f81e800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:31:08.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 -- 172.21.15.155:0/4097912743 shutdown_connections 2024-03-18T20:31:08.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 --2- 172.21.15.155:0/4097912743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f00ffcf0 0x7f81f0100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:08.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 --2- 172.21.15.155:0/4097912743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f0101890 0x7f81f0108140 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:08.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 --2- 172.21.15.155:0/4097912743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0100ef0 0x7f81f0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:08.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 -- 172.21.15.155:0/4097912743 >> 172.21.15.155:0/4097912743 conn(0x7f81f00fb2b0 msgr2=0x7f81f00fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:31:08.055 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 -- 172.21.15.155:0/4097912743 shutdown_connections 2024-03-18T20:31:08.055 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 -- 172.21.15.155:0/4097912743 wait complete. 2024-03-18T20:31:08.057 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 Processor -- start 2024-03-18T20:31:08.057 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 -- start start 2024-03-18T20:31:08.057 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f00ffcf0 0x7f81f010a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:31:08.057 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f0100ef0 0x7f81f0108c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:31:08.057 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0101890 0x7f81f0109190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:31:08.057 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.050+0000 7f81f714e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f0109870 con 0x7f81f0100ef0 2024-03-18T20:31:08.057 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f714e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f01099e0 con 0x7f81f0101890 2024-03-18T20:31:08.057 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f714e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f01a0e40 con 0x7f81f00ffcf0 2024-03-18T20:31:08.057 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f4eea700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f00ffcf0 0x7f81f010a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f4eea700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f00ffcf0 0x7f81f010a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:38090/0 (socket says 172.21.15.155:38090) 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f4eea700 1 -- 172.21.15.155:0/2603630951 learned_addr learned my addr 172.21.15.155:0/2603630951 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81effff700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f0100ef0 0x7f81f0108c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f56eb700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0101890 0x7f81f0109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f4eea700 1 -- 172.21.15.155:0/2603630951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0101890 msgr2=0x7f81f0109190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f4eea700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0101890 0x7f81f0109190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f4eea700 1 -- 172.21.15.155:0/2603630951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f0100ef0 msgr2=0x7f81f0108c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f4eea700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f0100ef0 0x7f81f0108c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f4eea700 1 -- 172.21.15.155:0/2603630951 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e8009d00 con 0x7f81f00ffcf0 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f56eb700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0101890 0x7f81f0109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81effff700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f0100ef0 0x7f81f0108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f4eea700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f00ffcf0 0x7f81f010a560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f81e400d980 tx=0x7f81e400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:31:08.058 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81edffb700 1 -- 172.21.15.155:0/2603630951 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e400de60 con 0x7f81f00ffcf0 2024-03-18T20:31:08.059 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81edffb700 1 -- 172.21.15.155:0/2603630951 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81e40044e0 con 0x7f81f00ffcf0 2024-03-18T20:31:08.059 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81f714e700 1 -- 172.21.15.155:0/2603630951 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f01a10a0 con 0x7f81f00ffcf0 2024-03-18T20:31:08.059 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.051+0000 7f81edffb700 1 -- 172.21.15.155:0/2603630951 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4003be0 con 0x7f81f00ffcf0 2024-03-18T20:31:08.059 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.052+0000 7f81f714e700 1 -- 172.21.15.155:0/2603630951 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f01a15c0 con 0x7f81f00ffcf0 2024-03-18T20:31:08.059 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.052+0000 7f81f714e700 1 -- 172.21.15.155:0/2603630951 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f01070b0 con 0x7f81f00ffcf0 2024-03-18T20:31:08.060 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.053+0000 7f81edffb700 1 -- 172.21.15.155:0/2603630951 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7f81e4009bb0 con 0x7f81f00ffcf0 2024-03-18T20:31:08.060 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.054+0000 7f81edffb700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f81e00a1110 0x7f81e00a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:31:08.060 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.054+0000 7f81edffb700 1 -- 172.21.15.155:0/2603630951 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7f81e40cac30 con 0x7f81f00ffcf0 2024-03-18T20:31:08.060 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.054+0000 7f81effff700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f81e00a1110 0x7f81e00a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:31:08.060 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.055+0000 7f81effff700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f81e00a1110 0x7f81e00a35d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f81dc005fd0 tx=0x7f81dc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:31:08.063 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.058+0000 7f81edffb700 1 -- 172.21.15.155:0/2603630951 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f81e4095d30 con 0x7f81f00ffcf0 2024-03-18T20:31:08.220 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-1[52371]: 2024-03-18T20:31:07.733+0000 7ff87156e700 -1 osd.1 0 waiting for initial osdmap 2024-03-18T20:31:08.221 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-1[52371]: 2024-03-18T20:31:07.739+0000 7ff86d363700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T20:31:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T20:31:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T20:31:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:31:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: from='osd.1 [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:31:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:31:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T20:31:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: Detected new or changed devices on smithi073 2024-03-18T20:31:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:31:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi073 to 17113M 2024-03-18T20:31:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:08.264 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:08.260+0000 7f81f714e700 1 -- 172.21.15.155:0/2603630951 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f81f002d030 con 0x7f81e00a1110 2024-03-18T20:31:08.949 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:08 smithi155 ceph-mon[48344]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-03-18T20:31:08.949 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:08 smithi155 ceph-mon[48344]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T20:31:08.949 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:08 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:31:08.949 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:08 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:31:08.949 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:08 smithi155 ceph-mon[48344]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:31:08.950 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:08 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:31:08.950 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:08 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:31:08.950 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:08 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:09.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:08 smithi046 ceph-mon[43495]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-03-18T20:31:09.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:08 smithi046 ceph-mon[43495]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T20:31:09.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:08 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:31:09.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:08 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:31:09.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:08 smithi046 ceph-mon[43495]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:31:09.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:08 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:31:09.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:08 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:31:09.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:08 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:09.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:08 smithi073 ceph-mon[47384]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-03-18T20:31:09.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:08 smithi073 ceph-mon[47384]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T20:31:09.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:08 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:31:09.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:08 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:31:09.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:08 smithi073 ceph-mon[47384]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:31:09.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:08 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:31:09.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:08 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:31:09.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:08 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:10.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:09 smithi155 ceph-mon[48344]: purged_snaps scrub starts 2024-03-18T20:31:10.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:09 smithi155 ceph-mon[48344]: purged_snaps scrub ok 2024-03-18T20:31:10.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:09 smithi155 ceph-mon[48344]: osd.1 [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822] boot 2024-03-18T20:31:10.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:09 smithi155 ceph-mon[48344]: osdmap e13: 2 total, 2 up, 2 in 2024-03-18T20:31:10.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:09 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:31:10.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:09 smithi155 ceph-mon[48344]: pgmap v49: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:10.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:09 smithi046 ceph-mon[43495]: purged_snaps scrub starts 2024-03-18T20:31:10.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:09 smithi046 ceph-mon[43495]: purged_snaps scrub ok 2024-03-18T20:31:10.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:09 smithi046 ceph-mon[43495]: osd.1 [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822] boot 2024-03-18T20:31:10.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:09 smithi046 ceph-mon[43495]: osdmap e13: 2 total, 2 up, 2 in 2024-03-18T20:31:10.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:09 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:31:10.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:09 smithi046 ceph-mon[43495]: pgmap v49: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:09 smithi073 ceph-mon[47384]: purged_snaps scrub starts 2024-03-18T20:31:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:09 smithi073 ceph-mon[47384]: purged_snaps scrub ok 2024-03-18T20:31:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:09 smithi073 ceph-mon[47384]: osd.1 [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822] boot 2024-03-18T20:31:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:09 smithi073 ceph-mon[47384]: osdmap e13: 2 total, 2 up, 2 in 2024-03-18T20:31:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:09 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:31:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:09 smithi073 ceph-mon[47384]: pgmap v49: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:11.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:10 smithi155 ceph-mon[48344]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T20:31:11.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:10 smithi155 ceph-mon[48344]: from='client.? 172.21.15.155:0/3234298478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10f51beb-1c27-4b01-9648-f296eddd43af"}]: dispatch 2024-03-18T20:31:11.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:10 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10f51beb-1c27-4b01-9648-f296eddd43af"}]: dispatch 2024-03-18T20:31:11.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:10 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10f51beb-1c27-4b01-9648-f296eddd43af"}]': finished 2024-03-18T20:31:11.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:10 smithi155 ceph-mon[48344]: osdmap e15: 3 total, 2 up, 3 in 2024-03-18T20:31:11.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:10 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:11.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:10 smithi155 ceph-mon[48344]: from='client.? 172.21.15.155:0/1120521662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:31:11.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:10 smithi046 ceph-mon[43495]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T20:31:11.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:10 smithi046 ceph-mon[43495]: from='client.? 172.21.15.155:0/3234298478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10f51beb-1c27-4b01-9648-f296eddd43af"}]: dispatch 2024-03-18T20:31:11.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:10 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10f51beb-1c27-4b01-9648-f296eddd43af"}]: dispatch 2024-03-18T20:31:11.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:10 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10f51beb-1c27-4b01-9648-f296eddd43af"}]': finished 2024-03-18T20:31:11.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:10 smithi046 ceph-mon[43495]: osdmap e15: 3 total, 2 up, 3 in 2024-03-18T20:31:11.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:10 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:11.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:10 smithi046 ceph-mon[43495]: from='client.? 172.21.15.155:0/1120521662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:31:11.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:10 smithi073 ceph-mon[47384]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T20:31:11.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:10 smithi073 ceph-mon[47384]: from='client.? 172.21.15.155:0/3234298478' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10f51beb-1c27-4b01-9648-f296eddd43af"}]: dispatch 2024-03-18T20:31:11.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:10 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10f51beb-1c27-4b01-9648-f296eddd43af"}]: dispatch 2024-03-18T20:31:11.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:10 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10f51beb-1c27-4b01-9648-f296eddd43af"}]': finished 2024-03-18T20:31:11.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:10 smithi073 ceph-mon[47384]: osdmap e15: 3 total, 2 up, 3 in 2024-03-18T20:31:11.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:10 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:11.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:10 smithi073 ceph-mon[47384]: from='client.? 172.21.15.155:0/1120521662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:31:12.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:11 smithi155 ceph-mon[48344]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:12.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:11 smithi046 ceph-mon[43495]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:12.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:11 smithi073 ceph-mon[47384]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:14.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:14 smithi155 ceph-mon[48344]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:14.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:14 smithi046 ceph-mon[43495]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:14 smithi073 ceph-mon[47384]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:16.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:15 smithi155 ceph-mon[48344]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:16.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:15 smithi046 ceph-mon[43495]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:16.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:15 smithi073 ceph-mon[47384]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:17.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:16 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T20:31:17.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:16 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:17.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:16 smithi155 ceph-mon[48344]: Deploying daemon osd.2 on smithi155 2024-03-18T20:31:17.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:16 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T20:31:17.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:16 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:17.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:16 smithi046 ceph-mon[43495]: Deploying daemon osd.2 on smithi155 2024-03-18T20:31:17.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:16 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T20:31:17.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:16 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:17.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:16 smithi073 ceph-mon[47384]: Deploying daemon osd.2 on smithi155 2024-03-18T20:31:18.000 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:17 smithi155 ceph-mon[48344]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:18.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:17 smithi046 ceph-mon[43495]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:18.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:17 smithi073 ceph-mon[47384]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:20 smithi155 ceph-mon[48344]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:20.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:20 smithi046 ceph-mon[43495]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:20.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:20 smithi073 ceph-mon[47384]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:20.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:20 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:20.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:20 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:20.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:20 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:21.819 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.816+0000 7f81edffb700 1 -- 172.21.15.155:0/2603630951 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f81f002d030 con 0x7f81e00a1110 2024-03-18T20:31:21.820 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 2 on host 'smithi155' 2024-03-18T20:31:21.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.818+0000 7f81f714e700 1 -- 172.21.15.155:0/2603630951 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f81e00a1110 msgr2=0x7f81e00a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:21.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.818+0000 7f81f714e700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f81e00a1110 0x7f81e00a35d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f81dc005fd0 tx=0x7f81dc005e30 comp rx=0 tx=0).stop 2024-03-18T20:31:21.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.818+0000 7f81f714e700 1 -- 172.21.15.155:0/2603630951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f00ffcf0 msgr2=0x7f81f010a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:21.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.818+0000 7f81f714e700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f00ffcf0 0x7f81f010a560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f81e400d980 tx=0x7f81e400dd40 comp rx=0 tx=0).stop 2024-03-18T20:31:21.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.818+0000 7f81f714e700 1 -- 172.21.15.155:0/2603630951 shutdown_connections 2024-03-18T20:31:21.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.818+0000 7f81f714e700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f81e00a1110 0x7f81e00a35d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:21.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.818+0000 7f81f714e700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0101890 0x7f81f0109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:21.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.818+0000 7f81f714e700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f00ffcf0 0x7f81f010a560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:21.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.818+0000 7f81f714e700 1 --2- 172.21.15.155:0/2603630951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f0100ef0 0x7f81f0108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:21.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.818+0000 7f81f714e700 1 -- 172.21.15.155:0/2603630951 >> 172.21.15.155:0/2603630951 conn(0x7f81f00fb2b0 msgr2=0x7f81f01041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:31:21.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.818+0000 7f81f714e700 1 -- 172.21.15.155:0/2603630951 shutdown_connections 2024-03-18T20:31:21.823 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:31:21.818+0000 7f81f714e700 1 -- 172.21.15.155:0/2603630951 wait complete. 2024-03-18T20:31:22.147 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.147 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.147 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:22.147 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:22.147 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.147 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:21 smithi155 ceph-mon[48344]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:22.147 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.147 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.147 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:22.208 DEBUG:teuthology.orchestra.run.smithi155:osd.2> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.2.service 2024-03-18T20:31:22.211 INFO:tasks.cephadm:Deploying osd.3 on smithi046 with /dev/vg_nvme/lv_3... 2024-03-18T20:31:22.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T20:31:22.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:22.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:22.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:21 smithi073 ceph-mon[47384]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:22.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:22.242 INFO:journalctl@ceph.osd.2.smithi155.stdout:-- Logs begin at Mon 2024-03-18 20:13:03 UTC. -- 2024-03-18T20:31:22.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:22.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:22.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:21 smithi046 ceph-mon[43495]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:22.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:22.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:22.467 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:31:23.904 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 20:31:23 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-2[54140]: 2024-03-18T20:31:23.614+0000 7fbfea2e3640 -1 osd.2 0 log_to_monitors true 2024-03-18T20:31:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T20:31:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T20:31:23.961 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-03-18T20:31:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-03-18T20:31:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00930334 s, 1.1 GB/s 2024-03-18T20:31:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-03-18T20:31:24.167 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:24 smithi155 ceph-mon[48344]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:24.167 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:24 smithi155 ceph-mon[48344]: from='osd.2 [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T20:31:24.167 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:24 smithi155 ceph-mon[48344]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T20:31:24.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:24 smithi046 ceph-mon[43495]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:24.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:24 smithi046 ceph-mon[43495]: from='osd.2 [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T20:31:24.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:24 smithi046 ceph-mon[43495]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T20:31:24.313 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi046:vg_nvme/lv_3 2024-03-18T20:31:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:24 smithi073 ceph-mon[47384]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:24 smithi073 ceph-mon[47384]: from='osd.2 [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T20:31:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:24 smithi073 ceph-mon[47384]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T20:31:24.605 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:31:25.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T20:31:25.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T20:31:25.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:25.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: from='osd.2 [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:31:25.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:31:25.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: Detected new or changed devices on smithi155 2024-03-18T20:31:25.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:25.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:25.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:31:25.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi155 to 17114M 2024-03-18T20:31:25.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:25.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:25.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:25.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:25.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T20:31:25.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T20:31:25.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:25.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: from='osd.2 [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:31:25.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:31:25.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: Detected new or changed devices on smithi155 2024-03-18T20:31:25.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:25.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:25.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:31:25.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi155 to 17114M 2024-03-18T20:31:25.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:25.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:25.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:25.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:25.541 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-2[54140]: 2024-03-18T20:31:25.048+0000 7fbfdf146700 -1 osd.2 0 waiting for initial osdmap 2024-03-18T20:31:25.541 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-2[54140]: 2024-03-18T20:31:25.053+0000 7fbfdb73c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T20:31:25.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T20:31:25.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T20:31:25.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:25.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: from='osd.2 [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:31:25.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:31:25.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: Detected new or changed devices on smithi155 2024-03-18T20:31:25.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:25.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:25.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:31:25.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi155 to 17114M 2024-03-18T20:31:25.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:25.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:25.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:25.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:25.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.589+0000 7fbd7d781700 1 -- 172.21.15.46:0/3915193097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd781009e0 msgr2=0x7fbd78100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:25.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.589+0000 7fbd7d781700 1 --2- 172.21.15.46:0/3915193097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd781009e0 0x7fbd78100e60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000b0a0 tx=0x7fbd7000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:31:25.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.589+0000 7fbd7d781700 1 -- 172.21.15.46:0/3915193097 shutdown_connections 2024-03-18T20:31:25.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.589+0000 7fbd7d781700 1 --2- 172.21.15.46:0/3915193097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd781009e0 0x7fbd78100e60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:25.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.589+0000 7fbd7d781700 1 --2- 172.21.15.46:0/3915193097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd780ff6c0 0x7fbd780ffae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.589+0000 7fbd7d781700 1 --2- 172.21.15.46:0/3915193097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd78100020 0x7fbd781004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.589+0000 7fbd7d781700 1 -- 172.21.15.46:0/3915193097 >> 172.21.15.46:0/3915193097 conn(0x7fbd780fb2a0 msgr2=0x7fbd780fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:31:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.589+0000 7fbd7d781700 1 -- 172.21.15.46:0/3915193097 shutdown_connections 2024-03-18T20:31:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.589+0000 7fbd7d781700 1 -- 172.21.15.46:0/3915193097 wait complete. 2024-03-18T20:31:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.590+0000 7fbd7d781700 1 Processor -- start 2024-03-18T20:31:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.590+0000 7fbd7d781700 1 -- start start 2024-03-18T20:31:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.590+0000 7fbd7d781700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd780ff6c0 0x7fbd78198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:31:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.590+0000 7fbd7d781700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd78100020 0x7fbd781994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.590+0000 7fbd7d781700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd781009e0 0x7fbd781a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.590+0000 7fbd7d781700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd781a0ae0 con 0x7fbd78100020 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.590+0000 7fbd7d781700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd781a0c20 con 0x7fbd780ff6c0 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.590+0000 7fbd7d781700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd781a0db0 con 0x7fbd781009e0 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.590+0000 7fbd6e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd78100020 0x7fbd781994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.590+0000 7fbd6e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd78100020 0x7fbd781994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58566/0 (socket says 172.21.15.46:58566) 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.590+0000 7fbd6e7fc700 1 -- 172.21.15.46:0/2646508419 learned_addr learned my addr 172.21.15.46:0/2646508419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.590+0000 7fbd777fe700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd781009e0 0x7fbd781a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.591+0000 7fbd76ffd700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd780ff6c0 0x7fbd78198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.591+0000 7fbd6e7fc700 1 -- 172.21.15.46:0/2646508419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd781009e0 msgr2=0x7fbd781a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.591+0000 7fbd6e7fc700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd781009e0 0x7fbd781a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.591+0000 7fbd6e7fc700 1 -- 172.21.15.46:0/2646508419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd780ff6c0 msgr2=0x7fbd78198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.591+0000 7fbd6e7fc700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd780ff6c0 0x7fbd78198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.591+0000 7fbd6e7fc700 1 -- 172.21.15.46:0/2646508419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd70009d00 con 0x7fbd78100020 2024-03-18T20:31:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.591+0000 7fbd6e7fc700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd78100020 0x7fbd781994e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000d980 tx=0x7fbd6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:31:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.591+0000 7fbd74ff9700 1 -- 172.21.15.46:0/2646508419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd600098d0 con 0x7fbd78100020 2024-03-18T20:31:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.591+0000 7fbd74ff9700 1 -- 172.21.15.46:0/2646508419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd60010460 con 0x7fbd78100020 2024-03-18T20:31:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.591+0000 7fbd7d781700 1 -- 172.21.15.46:0/2646508419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd781a1090 con 0x7fbd78100020 2024-03-18T20:31:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.591+0000 7fbd74ff9700 1 -- 172.21.15.46:0/2646508419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6000f620 con 0x7fbd78100020 2024-03-18T20:31:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.591+0000 7fbd7d781700 1 -- 172.21.15.46:0/2646508419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd781a1590 con 0x7fbd78100020 2024-03-18T20:31:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.592+0000 7fbd7d781700 1 -- 172.21.15.46:0/2646508419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7810a4a0 con 0x7fbd78100020 2024-03-18T20:31:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.593+0000 7fbd74ff9700 1 -- 172.21.15.46:0/2646508419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7fbd60010860 con 0x7fbd78100020 2024-03-18T20:31:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.594+0000 7fbd74ff9700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbd640a1090 0x7fbd640a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:31:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.594+0000 7fbd76ffd700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbd640a1090 0x7fbd640a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:31:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.594+0000 7fbd74ff9700 1 -- 172.21.15.46:0/2646508419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7fbd600caf00 con 0x7fbd78100020 2024-03-18T20:31:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.595+0000 7fbd76ffd700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbd640a1090 0x7fbd640a3550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009c60 tx=0x7fbd68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:31:25.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.599+0000 7fbd74ff9700 1 -- 172.21.15.46:0/2646508419 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbd60095de0 con 0x7fbd78100020 2024-03-18T20:31:25.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:25.820+0000 7fbd7d781700 1 -- 172.21.15.46:0/2646508419 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd7802d160 con 0x7fbd640a1090 2024-03-18T20:31:26.367 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:26 smithi046 ceph-mon[43495]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:26.367 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:26 smithi046 ceph-mon[43495]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-18T20:31:26.367 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:26 smithi046 ceph-mon[43495]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T20:31:26.367 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:26.367 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:26.367 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:31:26.368 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:31:26.368 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:26.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:26 smithi073 ceph-mon[47384]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:26.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:26 smithi073 ceph-mon[47384]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-18T20:31:26.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:26 smithi073 ceph-mon[47384]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T20:31:26.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:26.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:26.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:31:26.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:31:26.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:26 smithi155 ceph-mon[48344]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:26 smithi155 ceph-mon[48344]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-18T20:31:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:26 smithi155 ceph-mon[48344]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T20:31:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:31:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:31:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:27 smithi155 ceph-mon[48344]: purged_snaps scrub starts 2024-03-18T20:31:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:27 smithi155 ceph-mon[48344]: purged_snaps scrub ok 2024-03-18T20:31:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:27 smithi155 ceph-mon[48344]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:31:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:27 smithi155 ceph-mon[48344]: osd.2 [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296] boot 2024-03-18T20:31:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:27 smithi155 ceph-mon[48344]: osdmap e18: 3 total, 3 up, 3 in 2024-03-18T20:31:27.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:27 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:27 smithi046 ceph-mon[43495]: purged_snaps scrub starts 2024-03-18T20:31:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:27 smithi046 ceph-mon[43495]: purged_snaps scrub ok 2024-03-18T20:31:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:27 smithi046 ceph-mon[43495]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:31:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:27 smithi046 ceph-mon[43495]: osd.2 [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296] boot 2024-03-18T20:31:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:27 smithi046 ceph-mon[43495]: osdmap e18: 3 total, 3 up, 3 in 2024-03-18T20:31:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:27 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:27.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:27 smithi073 ceph-mon[47384]: purged_snaps scrub starts 2024-03-18T20:31:27.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:27 smithi073 ceph-mon[47384]: purged_snaps scrub ok 2024-03-18T20:31:27.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:27 smithi073 ceph-mon[47384]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:31:27.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:27 smithi073 ceph-mon[47384]: osd.2 [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296] boot 2024-03-18T20:31:27.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:27 smithi073 ceph-mon[47384]: osdmap e18: 3 total, 3 up, 3 in 2024-03-18T20:31:27.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:27 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:31:28.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:28 smithi155 ceph-mon[48344]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:28.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:28 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-18T20:31:28.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:28 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-18T20:31:28.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:28 smithi155 ceph-mon[48344]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T20:31:28.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:28 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T20:31:28.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:28 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/4239491012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e36074d6-5d67-4411-998e-42bfd7f85a98"}]: dispatch 2024-03-18T20:31:28.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:28 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T20:31:28.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:28 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/4239491012' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e36074d6-5d67-4411-998e-42bfd7f85a98"}]': finished 2024-03-18T20:31:28.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:28 smithi155 ceph-mon[48344]: osdmap e20: 4 total, 3 up, 4 in 2024-03-18T20:31:28.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:28 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:28.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:28 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/4057344557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:31:28.798 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 20:31:28 smithi046 sudo[67533]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T20:31:28.798 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 20:31:28 smithi046 sudo[67533]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T20:31:28.798 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 20:31:28 smithi046 sudo[67533]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T20:31:28.798 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 20:31:28 smithi046 sudo[67533]: pam_unix(sudo:session): session closed for user root 2024-03-18T20:31:28.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:28 smithi046 ceph-mon[43495]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:28.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:28 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-18T20:31:28.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:28 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-18T20:31:28.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:28 smithi046 ceph-mon[43495]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T20:31:28.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:28 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T20:31:28.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:28 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/4239491012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e36074d6-5d67-4411-998e-42bfd7f85a98"}]: dispatch 2024-03-18T20:31:28.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:28 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T20:31:28.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:28 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/4239491012' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e36074d6-5d67-4411-998e-42bfd7f85a98"}]': finished 2024-03-18T20:31:28.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:28 smithi046 ceph-mon[43495]: osdmap e20: 4 total, 3 up, 4 in 2024-03-18T20:31:28.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:28 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:28.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:28 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/4057344557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:31:28.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:28 smithi073 ceph-mon[47384]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-18T20:31:28.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:28 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-18T20:31:28.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:28 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-18T20:31:28.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:28 smithi073 ceph-mon[47384]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T20:31:28.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:28 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T20:31:28.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:28 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/4239491012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e36074d6-5d67-4411-998e-42bfd7f85a98"}]: dispatch 2024-03-18T20:31:28.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:28 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T20:31:28.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:28 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/4239491012' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e36074d6-5d67-4411-998e-42bfd7f85a98"}]': finished 2024-03-18T20:31:28.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:28 smithi073 ceph-mon[47384]: osdmap e20: 4 total, 3 up, 4 in 2024-03-18T20:31:28.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:28 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:28.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:28 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/4057344557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:31:29.297 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 20:31:28 smithi046 sudo[67588]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T20:31:29.297 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 20:31:28 smithi046 sudo[67588]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T20:31:29.297 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 20:31:28 smithi046 sudo[67588]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T20:31:29.297 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 20:31:28 smithi046 sudo[67588]: pam_unix(sudo:session): session closed for user root 2024-03-18T20:31:29.319 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 20:31:29 smithi073 sudo[57842]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T20:31:29.319 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 20:31:29 smithi073 sudo[57842]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T20:31:29.319 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 20:31:29 smithi073 sudo[57842]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T20:31:29.319 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 20:31:29 smithi073 sudo[57842]: pam_unix(sudo:session): session closed for user root 2024-03-18T20:31:29.319 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 20:31:29 smithi073 sudo[57845]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T20:31:29.319 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 20:31:29 smithi073 sudo[57845]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T20:31:29.319 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 20:31:29 smithi073 sudo[57845]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T20:31:29.319 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 20:31:29 smithi073 sudo[57845]: pam_unix(sudo:session): session closed for user root 2024-03-18T20:31:30.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:29 smithi155 ceph-mon[48344]: osdmap e21: 4 total, 3 up, 4 in 2024-03-18T20:31:30.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:30.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:30.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:29 smithi155 ceph-mon[48344]: pgmap v67: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-18T20:31:30.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:30.042 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 20:31:29 smithi155 sudo[59448]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T20:31:30.042 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 20:31:29 smithi155 sudo[59448]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T20:31:30.042 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 20:31:29 smithi155 sudo[59448]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T20:31:30.042 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 20:31:29 smithi155 sudo[59448]: pam_unix(sudo:session): session closed for user root 2024-03-18T20:31:30.042 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 20:31:29 smithi155 sudo[59451]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T20:31:30.042 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 20:31:29 smithi155 sudo[59451]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T20:31:30.042 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 20:31:29 smithi155 sudo[59451]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T20:31:30.042 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 20:31:29 smithi155 sudo[59451]: pam_unix(sudo:session): session closed for user root 2024-03-18T20:31:30.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:29 smithi046 ceph-mon[43495]: osdmap e21: 4 total, 3 up, 4 in 2024-03-18T20:31:30.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:30.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:30.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:29 smithi046 ceph-mon[43495]: pgmap v67: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-18T20:31:30.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:30.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:29 smithi073 ceph-mon[47384]: osdmap e21: 4 total, 3 up, 4 in 2024-03-18T20:31:30.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:30.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:30.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:29 smithi073 ceph-mon[47384]: pgmap v67: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-18T20:31:30.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:30.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:30 smithi046 sudo[68338]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T20:31:30.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:30 smithi046 sudo[68338]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T20:31:30.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:30 smithi046 sudo[68338]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T20:31:30.784 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:30 smithi046 sudo[68338]: pam_unix(sudo:session): session closed for user root 2024-03-18T20:31:30.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:30.794+0000 7fbd74ff9700 1 -- 172.21.15.46:0/2646508419 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fbd600961b0 con 0x7fbd78100020 2024-03-18T20:31:31.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:30 smithi046 sudo[68529]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T20:31:31.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:30 smithi046 sudo[68529]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T20:31:31.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:30 smithi046 sudo[68529]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T20:31:31.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:30 smithi046 sudo[68529]: pam_unix(sudo:session): session closed for user root 2024-03-18T20:31:31.279 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 sudo[59454]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T20:31:31.279 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 sudo[59454]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T20:31:31.279 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 sudo[59454]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T20:31:31.280 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 sudo[59454]: pam_unix(sudo:session): session closed for user root 2024-03-18T20:31:31.280 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 sudo[59457]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T20:31:31.280 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 sudo[59457]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T20:31:31.280 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 sudo[59457]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T20:31:31.280 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 sudo[59457]: pam_unix(sudo:session): session closed for user root 2024-03-18T20:31:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 ceph-mon[48344]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T20:31:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 ceph-mon[48344]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T20:31:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:31:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:31:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:31:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 ceph-mon[48344]: mgrmap e16: x(active, since 113s), standbys: z, y 2024-03-18T20:31:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:31:31.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:31:31.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:31:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:31 smithi046 ceph-mon[43495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T20:31:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:31 smithi046 ceph-mon[43495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T20:31:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:31:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:31:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:31:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:31 smithi046 ceph-mon[43495]: mgrmap e16: x(active, since 113s), standbys: z, y 2024-03-18T20:31:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:31:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:31:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:31:31.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 ceph-mon[47384]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 ceph-mon[47384]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 ceph-mon[47384]: mgrmap e16: x(active, since 113s), standbys: z, y 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 sudo[57848]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 sudo[57848]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 sudo[57848]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T20:31:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 sudo[57848]: pam_unix(sudo:session): session closed for user root 2024-03-18T20:31:32.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 sudo[57851]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T20:31:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 sudo[57851]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T20:31:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 sudo[57851]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T20:31:32.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:31 smithi073 sudo[57851]: pam_unix(sudo:session): session closed for user root 2024-03-18T20:31:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:32 smithi155 ceph-mon[48344]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T20:31:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:32 smithi155 ceph-mon[48344]: pgmap v68: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-18T20:31:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:32 smithi155 ceph-mon[48344]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T20:31:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:32 smithi155 ceph-mon[48344]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T20:31:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:31:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:31:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:31:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:32 smithi155 ceph-mon[48344]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T20:31:32.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:32 smithi046 ceph-mon[43495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T20:31:32.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:32 smithi046 ceph-mon[43495]: pgmap v68: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-18T20:31:32.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:32 smithi046 ceph-mon[43495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T20:31:32.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:32 smithi046 ceph-mon[43495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T20:31:32.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:31:32.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:31:32.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:31:32.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:32 smithi046 ceph-mon[43495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T20:31:32.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:32 smithi073 ceph-mon[47384]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T20:31:32.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:32 smithi073 ceph-mon[47384]: pgmap v68: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-18T20:31:32.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:32 smithi073 ceph-mon[47384]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T20:31:32.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:32 smithi073 ceph-mon[47384]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T20:31:32.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:31:32.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:31:32.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:31:32.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:32 smithi073 ceph-mon[47384]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T20:31:34.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:34 smithi046 ceph-mon[43495]: pgmap v69: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-18T20:31:34.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:34 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T20:31:34.506 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:34 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:34.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:34 smithi155 ceph-mon[48344]: pgmap v69: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-18T20:31:34.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:34 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T20:31:34.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:34 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:34 smithi073 ceph-mon[47384]: pgmap v69: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-18T20:31:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:34 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T20:31:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:34 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:35.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:35 smithi155 ceph-mon[48344]: Deploying daemon osd.3 on smithi046 2024-03-18T20:31:35.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:35 smithi046 ceph-mon[43495]: Deploying daemon osd.3 on smithi046 2024-03-18T20:31:35.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:35 smithi073 ceph-mon[47384]: Deploying daemon osd.3 on smithi046 2024-03-18T20:31:36.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:36 smithi155 ceph-mon[48344]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 6.4 KiB/s rd, 294 KiB/s wr, 20 op/s 2024-03-18T20:31:36.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:36 smithi046 ceph-mon[43495]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 6.4 KiB/s rd, 294 KiB/s wr, 20 op/s 2024-03-18T20:31:36.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:36 smithi073 ceph-mon[47384]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 6.4 KiB/s rd, 294 KiB/s wr, 20 op/s 2024-03-18T20:31:38.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:38 smithi046 ceph-mon[43495]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.9 KiB/s rd, 222 KiB/s wr, 15 op/s 2024-03-18T20:31:38.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:38.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:38.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:38 smithi155 ceph-mon[48344]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.9 KiB/s rd, 222 KiB/s wr, 15 op/s 2024-03-18T20:31:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:38 smithi073 ceph-mon[47384]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.9 KiB/s rd, 222 KiB/s wr, 15 op/s 2024-03-18T20:31:38.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:38.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:38.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:40.121 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 3 on host 'smithi046' 2024-03-18T20:31:40.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.116+0000 7fbd74ff9700 1 -- 172.21.15.46:0/2646508419 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbd7802d160 con 0x7fbd640a1090 2024-03-18T20:31:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.119+0000 7fbd7d781700 1 -- 172.21.15.46:0/2646508419 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbd640a1090 msgr2=0x7fbd640a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.119+0000 7fbd7d781700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbd640a1090 0x7fbd640a3550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009c60 tx=0x7fbd68009260 comp rx=0 tx=0).stop 2024-03-18T20:31:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.119+0000 7fbd7d781700 1 -- 172.21.15.46:0/2646508419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd78100020 msgr2=0x7fbd781994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.119+0000 7fbd7d781700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd78100020 0x7fbd781994e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000d980 tx=0x7fbd6000dc90 comp rx=0 tx=0).stop 2024-03-18T20:31:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.119+0000 7fbd7d781700 1 -- 172.21.15.46:0/2646508419 shutdown_connections 2024-03-18T20:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.119+0000 7fbd7d781700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbd640a1090 0x7fbd640a3550 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.119+0000 7fbd7d781700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd781009e0 0x7fbd781a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.119+0000 7fbd7d781700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd780ff6c0 0x7fbd78198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.120+0000 7fbd7d781700 1 --2- 172.21.15.46:0/2646508419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd78100020 0x7fbd781994e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.120+0000 7fbd7d781700 1 -- 172.21.15.46:0/2646508419 >> 172.21.15.46:0/2646508419 conn(0x7fbd780fb2a0 msgr2=0x7fbd780fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.120+0000 7fbd7d781700 1 -- 172.21.15.46:0/2646508419 shutdown_connections 2024-03-18T20:31:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:31:40.120+0000 7fbd7d781700 1 -- 172.21.15.46:0/2646508419 wait complete. 2024-03-18T20:31:40.287 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:40 smithi046 ceph-mon[43495]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 179 KiB/s wr, 12 op/s 2024-03-18T20:31:40.288 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:40 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.288 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:40 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.288 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:40 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:40.288 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:40 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:40.288 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:40 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.289 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:40 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.289 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:40 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.289 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:40 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:40.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:40 smithi073 ceph-mon[47384]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 179 KiB/s wr, 12 op/s 2024-03-18T20:31:40.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:40 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:40 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:40 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:40.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:40 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:40.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:40 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:40 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:40 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:40 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:40.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:40 smithi155 ceph-mon[48344]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 179 KiB/s wr, 12 op/s 2024-03-18T20:31:40.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:40 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:40 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:40 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:40.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:40 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:40.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:40 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:40 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:40 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:40.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:40 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:40.612 DEBUG:teuthology.orchestra.run.smithi046:osd.3> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.3.service 2024-03-18T20:31:40.615 INFO:tasks.cephadm:Deploying osd.4 on smithi073 with /dev/vg_nvme/lv_3... 2024-03-18T20:31:40.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T20:31:40.650 INFO:journalctl@ceph.osd.3.smithi046.stdout:-- Logs begin at Mon 2024-03-18 20:12:57 UTC. -- 2024-03-18T20:31:40.846 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T20:31:42.047 INFO:journalctl@ceph.osd.3.smithi046.stdout:Mar 18 20:31:41 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-3[69745]: 2024-03-18T20:31:41.753+0000 7f97ff336640 -1 osd.3 0 log_to_monitors true 2024-03-18T20:31:42.102 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T20:31:42.102 INFO:teuthology.orchestra.run.smithi073.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T20:31:42.102 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10+0 records in 2024-03-18T20:31:42.102 INFO:teuthology.orchestra.run.smithi073.stderr:10+0 records out 2024-03-18T20:31:42.102 INFO:teuthology.orchestra.run.smithi073.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113653 s, 923 MB/s 2024-03-18T20:31:42.102 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping successful for: 2024-03-18T20:31:42.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:42 smithi073 ceph-mon[47384]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-18T20:31:42.393 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:42 smithi073 ceph-mon[47384]: from='osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T20:31:42.493 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi073:vg_nvme/lv_3 2024-03-18T20:31:42.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:42 smithi155 ceph-mon[48344]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-18T20:31:42.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:42 smithi155 ceph-mon[48344]: from='osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T20:31:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:42 smithi046 ceph-mon[43495]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-18T20:31:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:42 smithi046 ceph-mon[43495]: from='osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T20:31:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T20:31:43.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:43 smithi073 ceph-mon[47384]: from='osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T20:31:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:43 smithi073 ceph-mon[47384]: osdmap e22: 4 total, 3 up, 4 in 2024-03-18T20:31:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:43 smithi073 ceph-mon[47384]: from='osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-03-18T20:31:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:31:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:31:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:43.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:43 smithi155 ceph-mon[48344]: from='osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T20:31:43.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:43 smithi155 ceph-mon[48344]: osdmap e22: 4 total, 3 up, 4 in 2024-03-18T20:31:43.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:43.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:43 smithi155 ceph-mon[48344]: from='osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-03-18T20:31:43.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:43.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:43.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:31:43.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:31:43.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:43.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:43.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:43.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:43.547 INFO:journalctl@ceph.osd.3.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-3[69745]: 2024-03-18T20:31:43.177+0000 7f97f4199700 -1 osd.3 0 waiting for initial osdmap 2024-03-18T20:31:43.547 INFO:journalctl@ceph.osd.3.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-3[69745]: 2024-03-18T20:31:43.185+0000 7f97ec787700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T20:31:43.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-mon[43495]: from='osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T20:31:43.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-mon[43495]: osdmap e22: 4 total, 3 up, 4 in 2024-03-18T20:31:43.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:43.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-mon[43495]: from='osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-03-18T20:31:43.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:43.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:43.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:31:43.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:31:43.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:43.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:43.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:43.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:43.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.646+0000 7fc1233a0700 1 -- 172.21.15.73:0/4047955695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc11c106ae0 msgr2=0x7fc11c108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:43.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.646+0000 7fc1233a0700 1 --2- 172.21.15.73:0/4047955695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc11c106ae0 0x7fc11c108f50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc11400b0a0 tx=0x7fc11400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:31:43.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.646+0000 7fc1233a0700 1 -- 172.21.15.73:0/4047955695 shutdown_connections 2024-03-18T20:31:43.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.646+0000 7fc1233a0700 1 --2- 172.21.15.73:0/4047955695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc11c101880 0x7fc11c103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:43.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.646+0000 7fc1233a0700 1 --2- 172.21.15.73:0/4047955695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc11c106ae0 0x7fc11c108f50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:43.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.646+0000 7fc1233a0700 1 --2- 172.21.15.73:0/4047955695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc11c1041b0 0x7fc11c1065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:43.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.646+0000 7fc1233a0700 1 -- 172.21.15.73:0/4047955695 >> 172.21.15.73:0/4047955695 conn(0x7fc11c0fb2b0 msgr2=0x7fc11c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:31:43.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.646+0000 7fc1233a0700 1 -- 172.21.15.73:0/4047955695 shutdown_connections 2024-03-18T20:31:43.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.646+0000 7fc1233a0700 1 -- 172.21.15.73:0/4047955695 wait complete. 2024-03-18T20:31:43.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc1233a0700 1 Processor -- start 2024-03-18T20:31:43.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc1233a0700 1 -- start start 2024-03-18T20:31:43.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc1233a0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc11c101880 0x7fc11c101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:31:43.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc1233a0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc11c1041b0 0x7fc11c0ff830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:31:43.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc1233a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc11c106ae0 0x7fc11c0ffd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:31:43.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc1233a0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c100510 con 0x7fc11c106ae0 2024-03-18T20:31:43.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc1233a0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c100680 con 0x7fc11c101880 2024-03-18T20:31:43.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc1233a0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c1a0f40 con 0x7fc11c1041b0 2024-03-18T20:31:43.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc12193d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc11c106ae0 0x7fc11c0ffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:31:43.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc12193d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc11c106ae0 0x7fc11c0ffd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.73:34458/0 (socket says 172.21.15.73:34458) 2024-03-18T20:31:43.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc12193d700 1 -- 172.21.15.73:0/1755814466 learned_addr learned my addr 172.21.15.73:0/1755814466 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-18T20:31:43.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc12113c700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc11c101880 0x7fc11c101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:31:43.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.647+0000 7fc12093b700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc11c1041b0 0x7fc11c0ff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:31:43.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.648+0000 7fc12193d700 1 -- 172.21.15.73:0/1755814466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc11c1041b0 msgr2=0x7fc11c0ff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:43.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.648+0000 7fc12193d700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc11c1041b0 0x7fc11c0ff830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:43.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.648+0000 7fc12193d700 1 -- 172.21.15.73:0/1755814466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc11c101880 msgr2=0x7fc11c101190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:43.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.648+0000 7fc12193d700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc11c101880 0x7fc11c101190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:43.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.648+0000 7fc12193d700 1 -- 172.21.15.73:0/1755814466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc114009d00 con 0x7fc11c106ae0 2024-03-18T20:31:43.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.648+0000 7fc12113c700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc11c101880 0x7fc11c101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:31:43.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.648+0000 7fc12193d700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc11c106ae0 0x7fc11c0ffd70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc114004d00 tx=0x7fc114004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:31:43.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.648+0000 7fc1127fc700 1 -- 172.21.15.73:0/1755814466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400e040 con 0x7fc11c106ae0 2024-03-18T20:31:43.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.648+0000 7fc1233a0700 1 -- 172.21.15.73:0/1755814466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c1a1170 con 0x7fc11c106ae0 2024-03-18T20:31:43.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.648+0000 7fc1127fc700 1 -- 172.21.15.73:0/1755814466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc11400cec0 con 0x7fc11c106ae0 2024-03-18T20:31:43.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.648+0000 7fc1127fc700 1 -- 172.21.15.73:0/1755814466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11401ba40 con 0x7fc11c106ae0 2024-03-18T20:31:43.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.648+0000 7fc1233a0700 1 -- 172.21.15.73:0/1755814466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c1a16b0 con 0x7fc11c106ae0 2024-03-18T20:31:43.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.649+0000 7fc1233a0700 1 -- 172.21.15.73:0/1755814466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c0fce30 con 0x7fc11c106ae0 2024-03-18T20:31:43.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.651+0000 7fc1127fc700 1 -- 172.21.15.73:0/1755814466 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc11401bba0 con 0x7fc11c106ae0 2024-03-18T20:31:43.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.651+0000 7fc1127fc700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc1080a1120 0x7fc1080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:31:43.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.651+0000 7fc1127fc700 1 -- 172.21.15.73:0/1755814466 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3389+0+0 (secure 0 0 0) 0x7fc1140cd7b0 con 0x7fc11c106ae0 2024-03-18T20:31:43.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.651+0000 7fc12113c700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc1080a1120 0x7fc1080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:31:43.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.652+0000 7fc12113c700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc1080a1120 0x7fc1080a35e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009de0 tx=0x7fc10c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:31:43.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.655+0000 7fc1127fc700 1 -- 172.21.15.73:0/1755814466 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc114098270 con 0x7fc11c106ae0 2024-03-18T20:31:43.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:43.856+0000 7fc1233a0700 1 -- 172.21.15.73:0/1755814466 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc11c0611f0 con 0x7fc1080a1120 2024-03-18T20:31:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:44 smithi073 ceph-mon[47384]: Detected new or changed devices on smithi046 2024-03-18T20:31:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:44 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi046 to 8559M 2024-03-18T20:31:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:44 smithi073 ceph-mon[47384]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-18T20:31:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:44 smithi073 ceph-mon[47384]: from='osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-03-18T20:31:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:44 smithi073 ceph-mon[47384]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T20:31:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:44 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:44 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:44 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:31:44.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:44 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:31:44.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:44 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:44 smithi155 ceph-mon[48344]: Detected new or changed devices on smithi046 2024-03-18T20:31:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:44 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi046 to 8559M 2024-03-18T20:31:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:44 smithi155 ceph-mon[48344]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-18T20:31:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:44 smithi155 ceph-mon[48344]: from='osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-03-18T20:31:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:44 smithi155 ceph-mon[48344]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T20:31:44.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:44 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:44.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:44 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:44.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:44 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:31:44.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:44 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:31:44.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:44 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:44.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:44 smithi046 ceph-mon[43495]: Detected new or changed devices on smithi046 2024-03-18T20:31:44.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:44 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi046 to 8559M 2024-03-18T20:31:44.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:44 smithi046 ceph-mon[43495]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-18T20:31:44.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:44 smithi046 ceph-mon[43495]: from='osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-03-18T20:31:44.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:44 smithi046 ceph-mon[43495]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T20:31:44.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:44 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:44.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:44 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:44.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:44 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:31:44.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:44 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:31:44.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:44 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:45.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:45 smithi073 ceph-mon[47384]: purged_snaps scrub starts 2024-03-18T20:31:45.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:45 smithi073 ceph-mon[47384]: purged_snaps scrub ok 2024-03-18T20:31:45.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:45 smithi073 ceph-mon[47384]: from='client.14325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:31:45.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:45 smithi073 ceph-mon[47384]: osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456] boot 2024-03-18T20:31:45.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:45 smithi073 ceph-mon[47384]: osdmap e24: 4 total, 4 up, 4 in 2024-03-18T20:31:45.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:45 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:45.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:45 smithi073 ceph-mon[47384]: osdmap e25: 4 total, 4 up, 4 in 2024-03-18T20:31:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:45 smithi155 ceph-mon[48344]: purged_snaps scrub starts 2024-03-18T20:31:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:45 smithi155 ceph-mon[48344]: purged_snaps scrub ok 2024-03-18T20:31:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:45 smithi155 ceph-mon[48344]: from='client.14325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:31:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:45 smithi155 ceph-mon[48344]: osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456] boot 2024-03-18T20:31:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:45 smithi155 ceph-mon[48344]: osdmap e24: 4 total, 4 up, 4 in 2024-03-18T20:31:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:45 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:45 smithi155 ceph-mon[48344]: osdmap e25: 4 total, 4 up, 4 in 2024-03-18T20:31:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:45 smithi046 ceph-mon[43495]: purged_snaps scrub starts 2024-03-18T20:31:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:45 smithi046 ceph-mon[43495]: purged_snaps scrub ok 2024-03-18T20:31:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:45 smithi046 ceph-mon[43495]: from='client.14325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:31:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:45 smithi046 ceph-mon[43495]: osd.3 [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456] boot 2024-03-18T20:31:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:45 smithi046 ceph-mon[43495]: osdmap e24: 4 total, 4 up, 4 in 2024-03-18T20:31:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:45 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:31:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:45 smithi046 ceph-mon[43495]: osdmap e25: 4 total, 4 up, 4 in 2024-03-18T20:31:46.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:46 smithi073 ceph-mon[47384]: pgmap v79: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-03-18T20:31:46.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:46 smithi073 ceph-mon[47384]: from='client.? 172.21.15.73:0/4246400055' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73dd25b5-ca0d-4ca4-b079-34487065b2c9"}]: dispatch 2024-03-18T20:31:46.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:46 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73dd25b5-ca0d-4ca4-b079-34487065b2c9"}]: dispatch 2024-03-18T20:31:46.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:46 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73dd25b5-ca0d-4ca4-b079-34487065b2c9"}]': finished 2024-03-18T20:31:46.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:46 smithi073 ceph-mon[47384]: osdmap e26: 5 total, 4 up, 5 in 2024-03-18T20:31:46.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:46 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:31:46.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:46 smithi155 ceph-mon[48344]: pgmap v79: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-03-18T20:31:46.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:46 smithi155 ceph-mon[48344]: from='client.? 172.21.15.73:0/4246400055' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73dd25b5-ca0d-4ca4-b079-34487065b2c9"}]: dispatch 2024-03-18T20:31:46.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:46 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73dd25b5-ca0d-4ca4-b079-34487065b2c9"}]: dispatch 2024-03-18T20:31:46.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:46 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73dd25b5-ca0d-4ca4-b079-34487065b2c9"}]': finished 2024-03-18T20:31:46.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:46 smithi155 ceph-mon[48344]: osdmap e26: 5 total, 4 up, 5 in 2024-03-18T20:31:46.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:46 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:31:46.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:46 smithi046 ceph-mon[43495]: pgmap v79: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-03-18T20:31:46.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:46 smithi046 ceph-mon[43495]: from='client.? 172.21.15.73:0/4246400055' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73dd25b5-ca0d-4ca4-b079-34487065b2c9"}]: dispatch 2024-03-18T20:31:46.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:46 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73dd25b5-ca0d-4ca4-b079-34487065b2c9"}]: dispatch 2024-03-18T20:31:46.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:46 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73dd25b5-ca0d-4ca4-b079-34487065b2c9"}]': finished 2024-03-18T20:31:46.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:46 smithi046 ceph-mon[43495]: osdmap e26: 5 total, 4 up, 5 in 2024-03-18T20:31:46.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:46 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:31:47.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:47 smithi073 ceph-mon[47384]: from='client.? 172.21.15.73:0/1335576019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:31:47.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:47 smithi073 ceph-mon[47384]: osdmap e27: 5 total, 4 up, 5 in 2024-03-18T20:31:47.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:31:47.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:47 smithi155 ceph-mon[48344]: from='client.? 172.21.15.73:0/1335576019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:31:47.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:47 smithi155 ceph-mon[48344]: osdmap e27: 5 total, 4 up, 5 in 2024-03-18T20:31:47.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:31:47.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:47 smithi046 ceph-mon[43495]: from='client.? 172.21.15.73:0/1335576019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:31:47.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:47 smithi046 ceph-mon[43495]: osdmap e27: 5 total, 4 up, 5 in 2024-03-18T20:31:47.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:31:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:48 smithi073 ceph-mon[47384]: pgmap v82: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-03-18T20:31:48.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:48 smithi073 ceph-mon[47384]: osdmap e28: 5 total, 4 up, 5 in 2024-03-18T20:31:48.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:48 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:31:48.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:48 smithi155 ceph-mon[48344]: pgmap v82: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-03-18T20:31:48.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:48 smithi155 ceph-mon[48344]: osdmap e28: 5 total, 4 up, 5 in 2024-03-18T20:31:48.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:48 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:31:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:48 smithi046 ceph-mon[43495]: pgmap v82: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-03-18T20:31:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:48 smithi046 ceph-mon[43495]: osdmap e28: 5 total, 4 up, 5 in 2024-03-18T20:31:48.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:31:50.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:50 smithi073 ceph-mon[47384]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-18T20:31:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:50 smithi155 ceph-mon[48344]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-18T20:31:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:50 smithi046 ceph-mon[43495]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-18T20:31:52.380 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:52 smithi073 ceph-mon[47384]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T20:31:52.380 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:52 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T20:31:52.380 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:52 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:52 smithi155 ceph-mon[48344]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T20:31:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:52 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T20:31:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:52 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:52 smithi046 ceph-mon[43495]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T20:31:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:52 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T20:31:52.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:52 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:53.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:53 smithi073 ceph-mon[47384]: Deploying daemon osd.4 on smithi073 2024-03-18T20:31:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:53 smithi155 ceph-mon[48344]: Deploying daemon osd.4 on smithi073 2024-03-18T20:31:53.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:53 smithi046 ceph-mon[43495]: Deploying daemon osd.4 on smithi073 2024-03-18T20:31:54.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:54 smithi073 ceph-mon[47384]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-18T20:31:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:54 smithi155 ceph-mon[48344]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-18T20:31:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:54 smithi046 ceph-mon[43495]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-18T20:31:56.380 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:56 smithi073 ceph-mon[47384]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-18T20:31:56.380 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:56 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:56.380 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:56 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:56.380 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:56 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:56.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:56 smithi155 ceph-mon[48344]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-18T20:31:56.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:56 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:56.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:56 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:56.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:56 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:56.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:56 smithi046 ceph-mon[43495]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-18T20:31:56.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:56 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:56.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:56 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:56.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:56 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:57.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.314+0000 7fc1127fc700 1 -- 172.21.15.73:0/1755814466 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc11c0611f0 con 0x7fc1080a1120 2024-03-18T20:31:57.320 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 4 on host 'smithi073' 2024-03-18T20:31:57.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.317+0000 7fc1233a0700 1 -- 172.21.15.73:0/1755814466 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc1080a1120 msgr2=0x7fc1080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:57.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.317+0000 7fc1233a0700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc1080a1120 0x7fc1080a35e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009de0 tx=0x7fc10c009190 comp rx=0 tx=0).stop 2024-03-18T20:31:57.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.317+0000 7fc1233a0700 1 -- 172.21.15.73:0/1755814466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc11c106ae0 msgr2=0x7fc11c0ffd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:31:57.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.317+0000 7fc1233a0700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc11c106ae0 0x7fc11c0ffd70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc114004d00 tx=0x7fc114004d30 comp rx=0 tx=0).stop 2024-03-18T20:31:57.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.317+0000 7fc1233a0700 1 -- 172.21.15.73:0/1755814466 shutdown_connections 2024-03-18T20:31:57.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.317+0000 7fc1233a0700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc1080a1120 0x7fc1080a35e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:57.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.317+0000 7fc1233a0700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc11c106ae0 0x7fc11c0ffd70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:57.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.317+0000 7fc1233a0700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc11c101880 0x7fc11c101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:57.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.317+0000 7fc1233a0700 1 --2- 172.21.15.73:0/1755814466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc11c1041b0 0x7fc11c0ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:31:57.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.317+0000 7fc1233a0700 1 -- 172.21.15.73:0/1755814466 >> 172.21.15.73:0/1755814466 conn(0x7fc11c0fb2b0 msgr2=0x7fc11c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:31:57.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.317+0000 7fc1233a0700 1 -- 172.21.15.73:0/1755814466 shutdown_connections 2024-03-18T20:31:57.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:31:57.317+0000 7fc1233a0700 1 -- 172.21.15.73:0/1755814466 wait complete. 2024-03-18T20:31:57.697 DEBUG:teuthology.orchestra.run.smithi073:osd.4> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.4.service 2024-03-18T20:31:57.699 INFO:tasks.cephadm:Deploying osd.5 on smithi155 with /dev/vg_nvme/lv_3... 2024-03-18T20:31:57.700 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T20:31:57.729 INFO:journalctl@ceph.osd.4.smithi073.stdout:-- Logs begin at Mon 2024-03-18 20:12:50 UTC. -- 2024-03-18T20:31:57.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:57 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:57.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:57 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:57.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:57 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:57.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:57 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:57.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:57 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:57.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:57 smithi073 ceph-mon[47384]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T20:31:57.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:57 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:57.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:57 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:57.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:31:57 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:57.929 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T20:31:58.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:57 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:58.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:57 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:58.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:57 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:58.033 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:57 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:58.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:57 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:58.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:57 smithi155 ceph-mon[48344]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T20:31:58.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:57 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:58.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:57 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:58.034 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:31:57 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:58.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:58.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:58.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:31:58.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:31:58.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:58.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:57 smithi046 ceph-mon[43495]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T20:31:58.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:58.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:31:58.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:31:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:31:59.178 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T20:31:59.179 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T20:31:59.179 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-03-18T20:31:59.179 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-03-18T20:31:59.179 INFO:teuthology.orchestra.run.smithi155.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0120692 s, 869 MB/s 2024-03-18T20:31:59.179 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-03-18T20:31:59.471 INFO:journalctl@ceph.osd.4.smithi073.stdout:Mar 18 20:31:59 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-4[60911]: 2024-03-18T20:31:59.195+0000 7f752b99c640 -1 osd.4 0 log_to_monitors true 2024-03-18T20:31:59.518 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2024-03-18T20:31:59.736 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T20:32:00.160 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:00 smithi155 ceph-mon[48344]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-18T20:32:00.160 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:00 smithi155 ceph-mon[48344]: from='osd.4 [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T20:32:00.160 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:00 smithi155 ceph-mon[48344]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T20:32:00.160 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:00 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:00.160 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:00 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:00.160 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:00 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:00.160 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:00 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:00.160 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:00 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:00.160 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:00 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:00.160 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:00 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:00.161 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:00 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:00.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:00 smithi046 ceph-mon[43495]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-18T20:32:00.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:00 smithi046 ceph-mon[43495]: from='osd.4 [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T20:32:00.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:00 smithi046 ceph-mon[43495]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T20:32:00.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:00 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:00.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:00 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:00.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:00 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:00.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:00 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:00.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:00 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:00.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:00 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:00.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:00 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:00.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:00 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:00.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:00 smithi073 ceph-mon[47384]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-18T20:32:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:00 smithi073 ceph-mon[47384]: from='osd.4 [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T20:32:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:00 smithi073 ceph-mon[47384]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T20:32:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:00 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:00 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:00 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:00 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:00 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:00 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:00 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:00 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:00.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.666+0000 7f28c754f700 1 -- 172.21.15.155:0/1937267457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c0102a40 msgr2=0x7f28c01092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:00.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.666+0000 7f28c754f700 1 --2- 172.21.15.155:0/1937267457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c0102a40 0x7f28c01092f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f28b800b0a0 tx=0x7f28b800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:32:00.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.666+0000 7f28c754f700 1 -- 172.21.15.155:0/1937267457 shutdown_connections 2024-03-18T20:32:00.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.666+0000 7f28c754f700 1 --2- 172.21.15.155:0/1937267457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c00ff0c0 0x7f28c00ff4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.666+0000 7f28c754f700 1 --2- 172.21.15.155:0/1937267457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c0102a40 0x7f28c01092f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.666+0000 7f28c754f700 1 --2- 172.21.15.155:0/1937267457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28c01020a0 0x7f28c0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.666+0000 7f28c754f700 1 -- 172.21.15.155:0/1937267457 >> 172.21.15.155:0/1937267457 conn(0x7f28c00faa40 msgr2=0x7f28c00fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.666+0000 7f28c754f700 1 -- 172.21.15.155:0/1937267457 shutdown_connections 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.666+0000 7f28c754f700 1 -- 172.21.15.155:0/1937267457 wait complete. 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.666+0000 7f28c754f700 1 Processor -- start 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.666+0000 7f28c754f700 1 -- start start 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c754f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c00ff0c0 0x7f28c0196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c754f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c01020a0 0x7f28c0197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c754f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28c0102a40 0x7f28c019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c754f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c019e900 con 0x7f28c00ff0c0 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c754f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c019ea40 con 0x7f28c0102a40 2024-03-18T20:32:00.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c754f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c019ec00 con 0x7f28c01020a0 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c4aea700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c01020a0 0x7f28c0197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c4aea700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c01020a0 0x7f28c0197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:38854/0 (socket says 172.21.15.155:38854) 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c4aea700 1 -- 172.21.15.155:0/276711543 learned_addr learned my addr 172.21.15.155:0/276711543 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c5aec700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28c0102a40 0x7f28c019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c4aea700 1 -- 172.21.15.155:0/276711543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28c0102a40 msgr2=0x7f28c019e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c4aea700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28c0102a40 0x7f28c019e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c4aea700 1 -- 172.21.15.155:0/276711543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c00ff0c0 msgr2=0x7f28c0196e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c4aea700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c00ff0c0 0x7f28c0196e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c4aea700 1 -- 172.21.15.155:0/276711543 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8009d00 con 0x7f28c01020a0 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c4aea700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c01020a0 0x7f28c0197390 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00c920 tx=0x7f28bc00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28b67fc700 1 -- 172.21.15.155:0/276711543 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc007b00 con 0x7f28c01020a0 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28b67fc700 1 -- 172.21.15.155:0/276711543 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28bc004cf0 con 0x7f28c01020a0 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c754f700 1 -- 172.21.15.155:0/276711543 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c019eee0 con 0x7f28c01020a0 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28b67fc700 1 -- 172.21.15.155:0/276711543 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc005610 con 0x7f28c01020a0 2024-03-18T20:32:00.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.667+0000 7f28c754f700 1 -- 172.21.15.155:0/276711543 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c019f2f0 con 0x7f28c01020a0 2024-03-18T20:32:00.674 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.668+0000 7f28c754f700 1 -- 172.21.15.155:0/276711543 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c0108260 con 0x7f28c01020a0 2024-03-18T20:32:00.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.669+0000 7f28b67fc700 1 -- 172.21.15.155:0/276711543 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f28bc004750 con 0x7f28c01020a0 2024-03-18T20:32:00.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.670+0000 7f28b67fc700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f28ac0a1120 0x7f28ac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:00.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.670+0000 7f28b67fc700 1 -- 172.21.15.155:0/276711543 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3805+0+0 (secure 0 0 0) 0x7f28bc0cb700 con 0x7f28c01020a0 2024-03-18T20:32:00.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.670+0000 7f28c52eb700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f28ac0a1120 0x7f28ac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:00.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.670+0000 7f28c52eb700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f28ac0a1120 0x7f28ac0a35e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f28c0198290 tx=0x7f28b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:32:00.677 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.674+0000 7f28b67fc700 1 -- 172.21.15.155:0/276711543 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f28bc096030 con 0x7f28c01020a0 2024-03-18T20:32:00.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:00.874+0000 7f28c754f700 1 -- 172.21.15.155:0/276711543 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f28c00611f0 con 0x7f28ac0a1120 2024-03-18T20:32:01.369 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:01 smithi155 ceph-mon[48344]: Detected new or changed devices on smithi073 2024-03-18T20:32:01.369 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:01 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi073 to 8556M 2024-03-18T20:32:01.369 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:01 smithi155 ceph-mon[48344]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T20:32:01.369 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:01 smithi155 ceph-mon[48344]: osdmap e29: 5 total, 4 up, 5 in 2024-03-18T20:32:01.369 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:01 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:32:01.369 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:01 smithi155 ceph-mon[48344]: from='osd.4 [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:32:01.369 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:01 smithi155 ceph-mon[48344]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:32:01.370 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:01 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:32:01.370 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:01 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:32:01.370 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:01 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:01.470 INFO:journalctl@ceph.osd.4.smithi073.stdout:Mar 18 20:32:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-4[60911]: 2024-03-18T20:32:01.050+0000 7f75207ff700 -1 osd.4 0 waiting for initial osdmap 2024-03-18T20:32:01.471 INFO:journalctl@ceph.osd.4.smithi073.stdout:Mar 18 20:32:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-4[60911]: 2024-03-18T20:32:01.058+0000 7f751b5f2700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T20:32:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:01 smithi073 ceph-mon[47384]: Detected new or changed devices on smithi073 2024-03-18T20:32:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:01 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi073 to 8556M 2024-03-18T20:32:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:01 smithi073 ceph-mon[47384]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T20:32:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:01 smithi073 ceph-mon[47384]: osdmap e29: 5 total, 4 up, 5 in 2024-03-18T20:32:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:01 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:32:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:01 smithi073 ceph-mon[47384]: from='osd.4 [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:32:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:01 smithi073 ceph-mon[47384]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:32:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:01 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:32:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:01 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:32:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:01 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:01 smithi046 ceph-mon[43495]: Detected new or changed devices on smithi073 2024-03-18T20:32:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:01 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi073 to 8556M 2024-03-18T20:32:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:01 smithi046 ceph-mon[43495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T20:32:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:01 smithi046 ceph-mon[43495]: osdmap e29: 5 total, 4 up, 5 in 2024-03-18T20:32:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:32:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:01 smithi046 ceph-mon[43495]: from='osd.4 [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:32:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:01 smithi046 ceph-mon[43495]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:32:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:32:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:32:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:02.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:02 smithi155 ceph-mon[48344]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:32:02.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:02 smithi155 ceph-mon[48344]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-18T20:32:02.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:02 smithi155 ceph-mon[48344]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-03-18T20:32:02.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:02 smithi155 ceph-mon[48344]: osdmap e30: 5 total, 4 up, 5 in 2024-03-18T20:32:02.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:02 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:32:02.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:02 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:32:02.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:02 smithi073 ceph-mon[47384]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:32:02.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:02 smithi073 ceph-mon[47384]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-18T20:32:02.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:02 smithi073 ceph-mon[47384]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-03-18T20:32:02.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:02 smithi073 ceph-mon[47384]: osdmap e30: 5 total, 4 up, 5 in 2024-03-18T20:32:02.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:02 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:32:02.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:02 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:32:02.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:02 smithi046 ceph-mon[43495]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:32:02.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:02 smithi046 ceph-mon[43495]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-18T20:32:02.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:02 smithi046 ceph-mon[43495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-03-18T20:32:02.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:02 smithi046 ceph-mon[43495]: osdmap e30: 5 total, 4 up, 5 in 2024-03-18T20:32:02.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:02 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:32:02.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:02 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:32:03.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:03 smithi073 ceph-mon[47384]: purged_snaps scrub starts 2024-03-18T20:32:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:03 smithi073 ceph-mon[47384]: purged_snaps scrub ok 2024-03-18T20:32:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:03 smithi073 ceph-mon[47384]: osd.4 [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749] boot 2024-03-18T20:32:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:03 smithi073 ceph-mon[47384]: osdmap e31: 5 total, 5 up, 5 in 2024-03-18T20:32:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:03 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:32:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:03 smithi073 ceph-mon[47384]: from='client.? 172.21.15.155:0/459291563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697"}]: dispatch 2024-03-18T20:32:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:03 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697"}]: dispatch 2024-03-18T20:32:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:03 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697"}]': finished 2024-03-18T20:32:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:03 smithi073 ceph-mon[47384]: osdmap e32: 6 total, 5 up, 6 in 2024-03-18T20:32:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:03 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:03 smithi155 ceph-mon[48344]: purged_snaps scrub starts 2024-03-18T20:32:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:03 smithi155 ceph-mon[48344]: purged_snaps scrub ok 2024-03-18T20:32:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:03 smithi155 ceph-mon[48344]: osd.4 [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749] boot 2024-03-18T20:32:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:03 smithi155 ceph-mon[48344]: osdmap e31: 5 total, 5 up, 5 in 2024-03-18T20:32:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:03 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:32:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:03 smithi155 ceph-mon[48344]: from='client.? 172.21.15.155:0/459291563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697"}]: dispatch 2024-03-18T20:32:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:03 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697"}]: dispatch 2024-03-18T20:32:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:03 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697"}]': finished 2024-03-18T20:32:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:03 smithi155 ceph-mon[48344]: osdmap e32: 6 total, 5 up, 6 in 2024-03-18T20:32:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:03 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:03 smithi046 ceph-mon[43495]: purged_snaps scrub starts 2024-03-18T20:32:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:03 smithi046 ceph-mon[43495]: purged_snaps scrub ok 2024-03-18T20:32:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:03 smithi046 ceph-mon[43495]: osd.4 [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749] boot 2024-03-18T20:32:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:03 smithi046 ceph-mon[43495]: osdmap e31: 5 total, 5 up, 5 in 2024-03-18T20:32:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:03 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:32:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:03 smithi046 ceph-mon[43495]: from='client.? 172.21.15.155:0/459291563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697"}]: dispatch 2024-03-18T20:32:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:03 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697"}]: dispatch 2024-03-18T20:32:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:03 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697"}]': finished 2024-03-18T20:32:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:03 smithi046 ceph-mon[43495]: osdmap e32: 6 total, 5 up, 6 in 2024-03-18T20:32:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:03 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:04 smithi073 ceph-mon[47384]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-18T20:32:04.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:04 smithi073 ceph-mon[47384]: from='client.? 172.21.15.155:0/3250358572' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:32:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:04 smithi155 ceph-mon[48344]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-18T20:32:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:04 smithi155 ceph-mon[48344]: from='client.? 172.21.15.155:0/3250358572' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:32:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:04 smithi046 ceph-mon[43495]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-18T20:32:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:04 smithi046 ceph-mon[43495]: from='client.? 172.21.15.155:0/3250358572' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:32:06.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:05 smithi155 ceph-mon[48344]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:06.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:05 smithi046 ceph-mon[43495]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:05 smithi073 ceph-mon[47384]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:08.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:08 smithi155 ceph-mon[48344]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:08.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:08 smithi046 ceph-mon[43495]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:08 smithi073 ceph-mon[47384]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:09 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T20:32:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:09 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:09.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:09 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T20:32:09.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:09 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:09.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:09 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T20:32:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:09 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:10.334 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:10 smithi155 ceph-mon[48344]: Deploying daemon osd.5 on smithi155 2024-03-18T20:32:10.335 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:10 smithi155 ceph-mon[48344]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:10 smithi073 ceph-mon[47384]: Deploying daemon osd.5 on smithi155 2024-03-18T20:32:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:10 smithi073 ceph-mon[47384]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:10 smithi046 ceph-mon[43495]: Deploying daemon osd.5 on smithi155 2024-03-18T20:32:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:10 smithi046 ceph-mon[43495]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:12 smithi073 ceph-mon[47384]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:12 smithi155 ceph-mon[48344]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:12 smithi046 ceph-mon[43495]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:13.516 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:13.517 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:13.517 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:13.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:13.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:13.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:13.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:13 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:13.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:13 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:13.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:13 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:14.390 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.386+0000 7f28b67fc700 1 -- 172.21.15.155:0/276711543 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f28c00611f0 con 0x7f28ac0a1120 2024-03-18T20:32:14.391 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 5 on host 'smithi155' 2024-03-18T20:32:14.392 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.389+0000 7f28c754f700 1 -- 172.21.15.155:0/276711543 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f28ac0a1120 msgr2=0x7f28ac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:14.392 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.389+0000 7f28c754f700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f28ac0a1120 0x7f28ac0a35e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f28c0198290 tx=0x7f28b0009260 comp rx=0 tx=0).stop 2024-03-18T20:32:14.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.389+0000 7f28c754f700 1 -- 172.21.15.155:0/276711543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c01020a0 msgr2=0x7f28c0197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:14.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.389+0000 7f28c754f700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c01020a0 0x7f28c0197390 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00c920 tx=0x7f28bc00cce0 comp rx=0 tx=0).stop 2024-03-18T20:32:14.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.389+0000 7f28c754f700 1 -- 172.21.15.155:0/276711543 shutdown_connections 2024-03-18T20:32:14.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.389+0000 7f28c754f700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f28ac0a1120 0x7f28ac0a35e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:14.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.389+0000 7f28c754f700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c00ff0c0 0x7f28c0196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:14.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.389+0000 7f28c754f700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28c0102a40 0x7f28c019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:14.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.389+0000 7f28c754f700 1 --2- 172.21.15.155:0/276711543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c01020a0 0x7f28c0197390 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:14.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.389+0000 7f28c754f700 1 -- 172.21.15.155:0/276711543 >> 172.21.15.155:0/276711543 conn(0x7f28c00faa40 msgr2=0x7f28c01053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:32:14.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.389+0000 7f28c754f700 1 -- 172.21.15.155:0/276711543 shutdown_connections 2024-03-18T20:32:14.393 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:14.389+0000 7f28c754f700 1 -- 172.21.15.155:0/276711543 wait complete. 2024-03-18T20:32:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:14 smithi155 ceph-mon[48344]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:14 smithi046 ceph-mon[43495]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:14.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:14 smithi073 ceph-mon[47384]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:14.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:14 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:14.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:14 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:14.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:14 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:14.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:14 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:14.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:14 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:14.911 DEBUG:teuthology.orchestra.run.smithi155:osd.5> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.5.service 2024-03-18T20:32:14.913 INFO:tasks.cephadm:Deploying osd.6 on smithi046 with /dev/vg_nvme/lv_2... 2024-03-18T20:32:14.914 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T20:32:14.943 INFO:journalctl@ceph.osd.5.smithi155.stdout:-- Logs begin at Mon 2024-03-18 20:13:03 UTC. -- 2024-03-18T20:32:15.168 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:32:15.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:15 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:15.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:15 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:15.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:15 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:15.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:15 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:15.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:15 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:15.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:15 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:15.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:15.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:15.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T20:32:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T20:32:16.514 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-03-18T20:32:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-03-18T20:32:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0089804 s, 1.2 GB/s 2024-03-18T20:32:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-03-18T20:32:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:16 smithi155 ceph-mon[48344]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:16 smithi155 ceph-mon[48344]: from='osd.5 [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T20:32:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:16 smithi155 ceph-mon[48344]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T20:32:16.541 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 20:32:16 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T20:32:16.272+0000 7fa64089a640 -1 osd.5 0 log_to_monitors true 2024-03-18T20:32:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:16 smithi046 ceph-mon[43495]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:16 smithi046 ceph-mon[43495]: from='osd.5 [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T20:32:16.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:16 smithi046 ceph-mon[43495]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T20:32:16.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:16 smithi073 ceph-mon[47384]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:16.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:16 smithi073 ceph-mon[47384]: from='osd.5 [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T20:32:16.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:16 smithi073 ceph-mon[47384]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T20:32:16.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi046:vg_nvme/lv_2 2024-03-18T20:32:17.190 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:32:17.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T20:32:17.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: osdmap e33: 6 total, 5 up, 6 in 2024-03-18T20:32:17.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:17.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: from='osd.5 [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:32:17.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:32:17.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: Detected new or changed devices on smithi155 2024-03-18T20:32:17.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:17.971 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:17.971 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:17.971 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:17.971 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi155 to 8557M 2024-03-18T20:32:17.971 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:17.971 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:17.971 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:17.971 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:17.971 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:17 smithi046 ceph-mon[43495]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:18.041 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T20:32:17.749+0000 7fa6356fd700 -1 osd.5 0 waiting for initial osdmap 2024-03-18T20:32:18.041 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T20:32:17.757+0000 7fa6304f0700 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T20:32:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T20:32:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: osdmap e33: 6 total, 5 up, 6 in 2024-03-18T20:32:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: from='osd.5 [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:32:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:32:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: Detected new or changed devices on smithi155 2024-03-18T20:32:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi155 to 8557M 2024-03-18T20:32:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:17 smithi155 ceph-mon[48344]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.156+0000 7ff171ac3700 1 -- 172.21.15.46:0/1345361954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c0ffcf0 msgr2=0x7ff16c100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.156+0000 7ff171ac3700 1 --2- 172.21.15.46:0/1345361954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c0ffcf0 0x7ff16c100110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0099c0 tx=0x7ff15c009cd0 comp rx=0 tx=0).stop 2024-03-18T20:32:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.156+0000 7ff171ac3700 1 -- 172.21.15.46:0/1345361954 shutdown_connections 2024-03-18T20:32:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.156+0000 7ff171ac3700 1 --2- 172.21.15.46:0/1345361954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c0ffcf0 0x7ff16c100110 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.156+0000 7ff171ac3700 1 --2- 172.21.15.46:0/1345361954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff16c101890 0x7ff16c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.156+0000 7ff171ac3700 1 --2- 172.21.15.46:0/1345361954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c100ef0 0x7ff16c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.156+0000 7ff171ac3700 1 -- 172.21.15.46:0/1345361954 >> 172.21.15.46:0/1345361954 conn(0x7ff16c0fb2b0 msgr2=0x7ff16c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:32:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.156+0000 7ff171ac3700 1 -- 172.21.15.46:0/1345361954 shutdown_connections 2024-03-18T20:32:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.156+0000 7ff171ac3700 1 -- 172.21.15.46:0/1345361954 wait complete. 2024-03-18T20:32:18.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.157+0000 7ff171ac3700 1 Processor -- start 2024-03-18T20:32:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.157+0000 7ff171ac3700 1 -- start start 2024-03-18T20:32:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.157+0000 7ff171ac3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c0ffcf0 0x7ff16c196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.157+0000 7ff171ac3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff16c100ef0 0x7ff16c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:18.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.157+0000 7ff171ac3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c101890 0x7ff16c19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.157+0000 7ff171ac3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c19e840 con 0x7ff16c0ffcf0 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.157+0000 7ff171ac3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c19e980 con 0x7ff16c100ef0 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.157+0000 7ff171ac3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c19eb40 con 0x7ff16c101890 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.157+0000 7ff16b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c0ffcf0 0x7ff16c196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.157+0000 7ff16b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c0ffcf0 0x7ff16c196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46932/0 (socket says 172.21.15.46:46932) 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.157+0000 7ff16b7fe700 1 -- 172.21.15.46:0/2550088602 learned_addr learned my addr 172.21.15.46:0/2550088602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff16bfff700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c101890 0x7ff16c19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff16affd700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff16c100ef0 0x7ff16c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff16b7fe700 1 -- 172.21.15.46:0/2550088602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c101890 msgr2=0x7ff16c19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff16b7fe700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c101890 0x7ff16c19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff16b7fe700 1 -- 172.21.15.46:0/2550088602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff16c100ef0 msgr2=0x7ff16c197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff16b7fe700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff16c100ef0 0x7ff16c197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:18.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff16b7fe700 1 -- 172.21.15.46:0/2550088602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c0095f0 con 0x7ff16c0ffcf0 2024-03-18T20:32:18.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff16affd700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff16c100ef0 0x7ff16c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:32:18.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff16b7fe700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c0ffcf0 0x7ff16c196d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff15c003820 tx=0x7ff15c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:32:18.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff168ff9700 1 -- 172.21.15.46:0/2550088602 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c01d070 con 0x7ff16c0ffcf0 2024-03-18T20:32:18.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff168ff9700 1 -- 172.21.15.46:0/2550088602 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff15c0054f0 con 0x7ff16c0ffcf0 2024-03-18T20:32:18.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff168ff9700 1 -- 172.21.15.46:0/2550088602 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c004a00 con 0x7ff16c0ffcf0 2024-03-18T20:32:18.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff171ac3700 1 -- 172.21.15.46:0/2550088602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c19edc0 con 0x7ff16c0ffcf0 2024-03-18T20:32:18.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.158+0000 7ff171ac3700 1 -- 172.21.15.46:0/2550088602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c19f1b0 con 0x7ff16c0ffcf0 2024-03-18T20:32:18.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.159+0000 7ff171ac3700 1 -- 172.21.15.46:0/2550088602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16c1071d0 con 0x7ff16c0ffcf0 2024-03-18T20:32:18.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.160+0000 7ff168ff9700 1 -- 172.21.15.46:0/2550088602 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7ff15c003b50 con 0x7ff16c0ffcf0 2024-03-18T20:32:18.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.161+0000 7ff168ff9700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff1540a1120 0x7ff1540a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:18.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.161+0000 7ff16affd700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff1540a1120 0x7ff1540a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:18.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.162+0000 7ff168ff9700 1 -- 172.21.15.46:0/2550088602 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4253+0+0 (secure 0 0 0) 0x7ff15c0cc750 con 0x7ff16c0ffcf0 2024-03-18T20:32:18.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.162+0000 7ff16affd700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff1540a1120 0x7ff1540a35e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff160005950 tx=0x7ff1600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:32:18.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.166+0000 7ff168ff9700 1 -- 172.21.15.46:0/2550088602 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff15c096f50 con 0x7ff16c0ffcf0 2024-03-18T20:32:18.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T20:32:18.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: osdmap e33: 6 total, 5 up, 6 in 2024-03-18T20:32:18.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:18.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: from='osd.5 [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:32:18.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:32:18.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: Detected new or changed devices on smithi155 2024-03-18T20:32:18.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:18.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:18.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:18.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:18.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi155 to 8557M 2024-03-18T20:32:18.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:18.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:18.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:18.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:18.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:17 smithi073 ceph-mon[47384]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:18.391+0000 7ff171ac3700 1 -- 172.21.15.46:0/2550088602 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff16c0611f0 con 0x7ff1540a1120 2024-03-18T20:32:19.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:18 smithi155 ceph-mon[48344]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-18T20:32:19.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:18 smithi155 ceph-mon[48344]: osdmap e34: 6 total, 5 up, 6 in 2024-03-18T20:32:19.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:18 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:19.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:18 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:19.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:18 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:32:19.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:18 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:32:19.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:18 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:19.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:18 smithi046 ceph-mon[43495]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-18T20:32:19.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:18 smithi046 ceph-mon[43495]: osdmap e34: 6 total, 5 up, 6 in 2024-03-18T20:32:19.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:18 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:19.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:18 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:19.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:18 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:32:19.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:18 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:32:19.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:18 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:19.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:18 smithi073 ceph-mon[47384]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-18T20:32:19.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:18 smithi073 ceph-mon[47384]: osdmap e34: 6 total, 5 up, 6 in 2024-03-18T20:32:19.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:18 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:19.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:18 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:19.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:18 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:32:19.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:18 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:32:19.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:18 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:20.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:19 smithi155 ceph-mon[48344]: purged_snaps scrub starts 2024-03-18T20:32:20.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:19 smithi155 ceph-mon[48344]: purged_snaps scrub ok 2024-03-18T20:32:20.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:19 smithi155 ceph-mon[48344]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:32:20.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:19 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:20.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:19 smithi155 ceph-mon[48344]: osd.5 [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215] boot 2024-03-18T20:32:20.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:19 smithi155 ceph-mon[48344]: osdmap e35: 6 total, 6 up, 6 in 2024-03-18T20:32:20.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:19 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:20.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:19 smithi155 ceph-mon[48344]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:20.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:19 smithi155 ceph-mon[48344]: osdmap e36: 6 total, 6 up, 6 in 2024-03-18T20:32:20.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:19 smithi046 ceph-mon[43495]: purged_snaps scrub starts 2024-03-18T20:32:20.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:19 smithi046 ceph-mon[43495]: purged_snaps scrub ok 2024-03-18T20:32:20.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:19 smithi046 ceph-mon[43495]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:32:20.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:19 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:20.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:19 smithi046 ceph-mon[43495]: osd.5 [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215] boot 2024-03-18T20:32:20.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:19 smithi046 ceph-mon[43495]: osdmap e35: 6 total, 6 up, 6 in 2024-03-18T20:32:20.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:19 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:20.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:19 smithi046 ceph-mon[43495]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:20.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:19 smithi046 ceph-mon[43495]: osdmap e36: 6 total, 6 up, 6 in 2024-03-18T20:32:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:19 smithi073 ceph-mon[47384]: purged_snaps scrub starts 2024-03-18T20:32:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:19 smithi073 ceph-mon[47384]: purged_snaps scrub ok 2024-03-18T20:32:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:19 smithi073 ceph-mon[47384]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:32:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:19 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:19 smithi073 ceph-mon[47384]: osd.5 [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215] boot 2024-03-18T20:32:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:19 smithi073 ceph-mon[47384]: osdmap e35: 6 total, 6 up, 6 in 2024-03-18T20:32:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:19 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:32:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:19 smithi073 ceph-mon[47384]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-18T20:32:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:19 smithi073 ceph-mon[47384]: osdmap e36: 6 total, 6 up, 6 in 2024-03-18T20:32:21.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:20 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/287144381' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb44585c-e16c-421c-981a-a7cb8b9ba58d"}]: dispatch 2024-03-18T20:32:21.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:20 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/287144381' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb44585c-e16c-421c-981a-a7cb8b9ba58d"}]': finished 2024-03-18T20:32:21.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:20 smithi155 ceph-mon[48344]: osdmap e37: 7 total, 6 up, 7 in 2024-03-18T20:32:21.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:21.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:20 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/287144381' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb44585c-e16c-421c-981a-a7cb8b9ba58d"}]: dispatch 2024-03-18T20:32:21.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:20 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/287144381' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb44585c-e16c-421c-981a-a7cb8b9ba58d"}]': finished 2024-03-18T20:32:21.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:20 smithi046 ceph-mon[43495]: osdmap e37: 7 total, 6 up, 7 in 2024-03-18T20:32:21.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:21.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:20 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/287144381' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb44585c-e16c-421c-981a-a7cb8b9ba58d"}]: dispatch 2024-03-18T20:32:21.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:20 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/287144381' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb44585c-e16c-421c-981a-a7cb8b9ba58d"}]': finished 2024-03-18T20:32:21.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:20 smithi073 ceph-mon[47384]: osdmap e37: 7 total, 6 up, 7 in 2024-03-18T20:32:21.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:20 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:22.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:21 smithi155 ceph-mon[48344]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-18T20:32:22.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:21 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3861397153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:32:22.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:21 smithi155 ceph-mon[48344]: osdmap e38: 7 total, 6 up, 7 in 2024-03-18T20:32:22.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:22.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:21 smithi046 ceph-mon[43495]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-18T20:32:22.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:21 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3861397153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:32:22.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:21 smithi046 ceph-mon[43495]: osdmap e38: 7 total, 6 up, 7 in 2024-03-18T20:32:22.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:22.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:21 smithi073 ceph-mon[47384]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-18T20:32:22.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:21 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3861397153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:32:22.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:21 smithi073 ceph-mon[47384]: osdmap e38: 7 total, 6 up, 7 in 2024-03-18T20:32:22.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:23.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:23 smithi073 ceph-mon[47384]: osdmap e39: 7 total, 6 up, 7 in 2024-03-18T20:32:23.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:23 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:23.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:23 smithi155 ceph-mon[48344]: osdmap e39: 7 total, 6 up, 7 in 2024-03-18T20:32:23.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:23 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:23.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:23 smithi046 ceph-mon[43495]: osdmap e39: 7 total, 6 up, 7 in 2024-03-18T20:32:23.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:23 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:24 smithi046 ceph-mon[43495]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-18T20:32:24.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:24 smithi073 ceph-mon[47384]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-18T20:32:24.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:24 smithi155 ceph-mon[48344]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-18T20:32:26.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:25 smithi155 ceph-mon[48344]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-03-18T20:32:26.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:25 smithi046 ceph-mon[43495]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-03-18T20:32:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:25 smithi073 ceph-mon[47384]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-03-18T20:32:27.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-18T20:32:27.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:27.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-18T20:32:27.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:27.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-18T20:32:27.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:27 smithi046 ceph-mon[43495]: Deploying daemon osd.6 on smithi046 2024-03-18T20:32:27.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:27 smithi046 ceph-mon[43495]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-18T20:32:28.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:27 smithi155 ceph-mon[48344]: Deploying daemon osd.6 on smithi046 2024-03-18T20:32:28.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:27 smithi155 ceph-mon[48344]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-18T20:32:28.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:27 smithi073 ceph-mon[47384]: Deploying daemon osd.6 on smithi046 2024-03-18T20:32:28.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:27 smithi073 ceph-mon[47384]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-18T20:32:30.152 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:30 smithi046 ceph-mon[43495]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T20:32:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:30 smithi073 ceph-mon[47384]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T20:32:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:30 smithi155 ceph-mon[48344]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-18T20:32:31.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:31.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:31.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:31.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:32.486 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:32 smithi046 ceph-mon[43495]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-18T20:32:32.486 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:32.486 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:32.486 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:32.486 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:32.486 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:32 smithi155 ceph-mon[48344]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-18T20:32:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.587+0000 7ff168ff9700 1 -- 172.21.15.46:0/2550088602 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff16c0611f0 con 0x7ff1540a1120 2024-03-18T20:32:32.592 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 6 on host 'smithi046' 2024-03-18T20:32:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.589+0000 7ff171ac3700 1 -- 172.21.15.46:0/2550088602 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff1540a1120 msgr2=0x7ff1540a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.589+0000 7ff171ac3700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff1540a1120 0x7ff1540a35e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff160005950 tx=0x7ff1600058e0 comp rx=0 tx=0).stop 2024-03-18T20:32:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.590+0000 7ff171ac3700 1 -- 172.21.15.46:0/2550088602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c0ffcf0 msgr2=0x7ff16c196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.590+0000 7ff171ac3700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c0ffcf0 0x7ff16c196d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff15c003820 tx=0x7ff15c004730 comp rx=0 tx=0).stop 2024-03-18T20:32:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.590+0000 7ff171ac3700 1 -- 172.21.15.46:0/2550088602 shutdown_connections 2024-03-18T20:32:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.590+0000 7ff171ac3700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff1540a1120 0x7ff1540a35e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.590+0000 7ff171ac3700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff16c0ffcf0 0x7ff16c196d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.590+0000 7ff171ac3700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff16c101890 0x7ff16c19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.590+0000 7ff171ac3700 1 --2- 172.21.15.46:0/2550088602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff16c100ef0 0x7ff16c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.590+0000 7ff171ac3700 1 -- 172.21.15.46:0/2550088602 >> 172.21.15.46:0/2550088602 conn(0x7ff16c0fb2b0 msgr2=0x7ff16c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:32:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.590+0000 7ff171ac3700 1 -- 172.21.15.46:0/2550088602 shutdown_connections 2024-03-18T20:32:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:32:32.590+0000 7ff171ac3700 1 -- 172.21.15.46:0/2550088602 wait complete. 2024-03-18T20:32:32.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:32 smithi073 ceph-mon[47384]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-18T20:32:32.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:32.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:32.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:32.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:32.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:33.018 DEBUG:teuthology.orchestra.run.smithi046:osd.6> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.6.service 2024-03-18T20:32:33.020 INFO:tasks.cephadm:Deploying osd.7 on smithi073 with /dev/vg_nvme/lv_2... 2024-03-18T20:32:33.020 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T20:32:33.059 INFO:journalctl@ceph.osd.6.smithi046.stdout:-- Logs begin at Mon 2024-03-18 20:12:57 UTC. -- 2024-03-18T20:32:33.251 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T20:32:33.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:33 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:33.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:33 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:33.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:33 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:33.665 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:33 smithi073 ceph-mon[47384]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-18T20:32:33.892 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:33 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:33.893 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:33 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:33.893 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:33 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:33.893 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:33 smithi046 ceph-mon[43495]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-18T20:32:34.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:33 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:34.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:33 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:34.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:33 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:34.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:33 smithi155 ceph-mon[48344]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-18T20:32:34.546 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T20:32:34.546 INFO:teuthology.orchestra.run.smithi073.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T20:32:34.547 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10+0 records in 2024-03-18T20:32:34.547 INFO:teuthology.orchestra.run.smithi073.stderr:10+0 records out 2024-03-18T20:32:34.547 INFO:teuthology.orchestra.run.smithi073.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.01127 s, 930 MB/s 2024-03-18T20:32:34.547 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 2024-03-18T20:32:34.547 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping successful for: 2024-03-18T20:32:34.547 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 20:32:34 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T20:32:34.250+0000 7f3021eea640 -1 osd.6 0 log_to_monitors true 2024-03-18T20:32:34.851 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:34 smithi073 ceph-mon[47384]: from='osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-18T20:32:34.936 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi073:vg_nvme/lv_2 2024-03-18T20:32:34.969 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:34 smithi046 ceph-mon[43495]: from='osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-18T20:32:35.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:34 smithi155 ceph-mon[48344]: from='osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-18T20:32:35.150 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T20:32:35.950 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: from='osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: osdmap e40: 7 total, 6 up, 7 in 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: from='osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: Detected new or changed devices on smithi046 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi046 to 5706M 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:35.951 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:35.952 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:35.952 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: from='osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-18T20:32:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: osdmap e40: 7 total, 6 up, 7 in 2024-03-18T20:32:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: from='osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-03-18T20:32:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-18T20:32:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: Detected new or changed devices on smithi046 2024-03-18T20:32:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:36.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:36.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi046 to 5706M 2024-03-18T20:32:36.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:36.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:36.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:36.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:36.046 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T20:32:35.602+0000 7f3016d4d700 -1 osd.6 0 waiting for initial osdmap 2024-03-18T20:32:36.047 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T20:32:35.615+0000 7f3013343700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T20:32:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: from='osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-18T20:32:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: osdmap e40: 7 total, 6 up, 7 in 2024-03-18T20:32:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: from='osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-03-18T20:32:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-18T20:32:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: Detected new or changed devices on smithi046 2024-03-18T20:32:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:36.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:36.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:36.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi046 to 5706M 2024-03-18T20:32:36.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:36.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:36.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:36.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:36.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.122+0000 7f58687b0700 1 -- 172.21.15.73:0/3289335125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5860101fe0 msgr2=0x7f5860102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:36.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.122+0000 7f58687b0700 1 --2- 172.21.15.73:0/3289335125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5860101fe0 0x7f5860102440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f585c0099c0 tx=0x7f585c009cd0 comp rx=0 tx=0).stop 2024-03-18T20:32:36.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.122+0000 7f58687b0700 1 -- 172.21.15.73:0/3289335125 shutdown_connections 2024-03-18T20:32:36.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.122+0000 7f58687b0700 1 --2- 172.21.15.73:0/3289335125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5860100de0 0x7f5860101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.122+0000 7f58687b0700 1 --2- 172.21.15.73:0/3289335125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5860102980 0x7f5860109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.122+0000 7f58687b0700 1 --2- 172.21.15.73:0/3289335125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5860101fe0 0x7f5860102440 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.122+0000 7f58687b0700 1 -- 172.21.15.73:0/3289335125 >> 172.21.15.73:0/3289335125 conn(0x7f58600fc380 msgr2=0x7f58600fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.123+0000 7f58687b0700 1 -- 172.21.15.73:0/3289335125 shutdown_connections 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.123+0000 7f58687b0700 1 -- 172.21.15.73:0/3289335125 wait complete. 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.123+0000 7f58687b0700 1 Processor -- start 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.123+0000 7f58687b0700 1 -- start start 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.123+0000 7f58687b0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5860100de0 0x7f5860196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.123+0000 7f58687b0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5860101fe0 0x7f5860197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.123+0000 7f58687b0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5860102980 0x7f586019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.123+0000 7f58687b0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586019e7f0 con 0x7f5860100de0 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f58687b0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586019e930 con 0x7f5860101fe0 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f58687b0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586019eaf0 con 0x7f5860102980 2024-03-18T20:32:36.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f5865d4b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5860101fe0 0x7f5860197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:36.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f5865d4b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5860101fe0 0x7f5860197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40076/0 (socket says 172.21.15.73:40076) 2024-03-18T20:32:36.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f5865d4b700 1 -- 172.21.15.73:0/3578242336 learned_addr learned my addr 172.21.15.73:0/3578242336 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-18T20:32:36.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f5866d4d700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5860102980 0x7f586019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:36.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f586654c700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5860100de0 0x7f5860196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:36.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f5865d4b700 1 -- 172.21.15.73:0/3578242336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5860102980 msgr2=0x7f586019e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:36.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f5865d4b700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5860102980 0x7f586019e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:36.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f5865d4b700 1 -- 172.21.15.73:0/3578242336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5860100de0 msgr2=0x7f5860196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:36.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f5865d4b700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5860100de0 0x7f5860196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:36.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f5865d4b700 1 -- 172.21.15.73:0/3578242336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c0095f0 con 0x7f5860101fe0 2024-03-18T20:32:36.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f586654c700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5860100de0 0x7f5860196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:32:36.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f5865d4b700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5860101fe0 0x7f5860197200 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f585c003730 tx=0x7f585c00f720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:32:36.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f58537fe700 1 -- 172.21.15.73:0/3578242336 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c01d070 con 0x7f5860101fe0 2024-03-18T20:32:36.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f58687b0700 1 -- 172.21.15.73:0/3578242336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586019ed70 con 0x7f5860101fe0 2024-03-18T20:32:36.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f58537fe700 1 -- 172.21.15.73:0/3578242336 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f585c0059a0 con 0x7f5860101fe0 2024-03-18T20:32:36.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f58537fe700 1 -- 172.21.15.73:0/3578242336 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c004b30 con 0x7f5860101fe0 2024-03-18T20:32:36.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.124+0000 7f58687b0700 1 -- 172.21.15.73:0/3578242336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586019f1c0 con 0x7f5860101fe0 2024-03-18T20:32:36.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.125+0000 7f58687b0700 1 -- 172.21.15.73:0/3578242336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58601082c0 con 0x7f5860101fe0 2024-03-18T20:32:36.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.126+0000 7f58537fe700 1 -- 172.21.15.73:0/3578242336 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f585c00fc10 con 0x7f5860101fe0 2024-03-18T20:32:36.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.127+0000 7f58537fe700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f584c0a1120 0x7f584c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:36.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.127+0000 7f58537fe700 1 -- 172.21.15.73:0/3578242336 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4685+0+0 (secure 0 0 0) 0x7f585c0cccb0 con 0x7f5860101fe0 2024-03-18T20:32:36.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.127+0000 7f586654c700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f584c0a1120 0x7f584c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:36.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.128+0000 7f586654c700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f584c0a1120 0x7f584c0a35e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f58600fe840 tx=0x7f5854009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:32:36.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.131+0000 7f58537fe700 1 -- 172.21.15.73:0/3578242336 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f585c097380 con 0x7f5860101fe0 2024-03-18T20:32:36.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:36.329+0000 7f58687b0700 1 -- 172.21.15.73:0/3578242336 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f58600611f0 con 0x7f584c0a1120 2024-03-18T20:32:36.869 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:36 smithi073 ceph-mon[47384]: from='osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-03-18T20:32:36.869 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:36 smithi073 ceph-mon[47384]: osdmap e41: 7 total, 6 up, 7 in 2024-03-18T20:32:36.869 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:36.869 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:36.869 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:36 smithi073 ceph-mon[47384]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:32:36.869 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:32:36.869 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:32:36.869 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:36 smithi155 ceph-mon[48344]: from='osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-03-18T20:32:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:36 smithi155 ceph-mon[48344]: osdmap e41: 7 total, 6 up, 7 in 2024-03-18T20:32:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:36 smithi155 ceph-mon[48344]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:32:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:32:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:32:37.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:36 smithi046 ceph-mon[43495]: from='osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-03-18T20:32:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:36 smithi046 ceph-mon[43495]: osdmap e41: 7 total, 6 up, 7 in 2024-03-18T20:32:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:36 smithi046 ceph-mon[43495]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:32:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:32:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:32:37.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:37.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:37 smithi073 ceph-mon[47384]: purged_snaps scrub starts 2024-03-18T20:32:37.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:37 smithi073 ceph-mon[47384]: purged_snaps scrub ok 2024-03-18T20:32:37.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:37 smithi073 ceph-mon[47384]: osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640] boot 2024-03-18T20:32:37.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:37 smithi073 ceph-mon[47384]: osdmap e42: 7 total, 7 up, 7 in 2024-03-18T20:32:37.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:37 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:37.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:37 smithi073 ceph-mon[47384]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-18T20:32:38.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:37 smithi155 ceph-mon[48344]: purged_snaps scrub starts 2024-03-18T20:32:38.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:37 smithi155 ceph-mon[48344]: purged_snaps scrub ok 2024-03-18T20:32:38.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:37 smithi155 ceph-mon[48344]: osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640] boot 2024-03-18T20:32:38.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:37 smithi155 ceph-mon[48344]: osdmap e42: 7 total, 7 up, 7 in 2024-03-18T20:32:38.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:37 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:38.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:37 smithi155 ceph-mon[48344]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-18T20:32:38.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:37 smithi046 ceph-mon[43495]: purged_snaps scrub starts 2024-03-18T20:32:38.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:37 smithi046 ceph-mon[43495]: purged_snaps scrub ok 2024-03-18T20:32:38.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:37 smithi046 ceph-mon[43495]: osd.6 [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640] boot 2024-03-18T20:32:38.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:37 smithi046 ceph-mon[43495]: osdmap e42: 7 total, 7 up, 7 in 2024-03-18T20:32:38.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:32:38.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:37 smithi046 ceph-mon[43495]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-18T20:32:38.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:38 smithi073 ceph-mon[47384]: osdmap e43: 7 total, 7 up, 7 in 2024-03-18T20:32:38.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:38 smithi073 ceph-mon[47384]: from='client.? 172.21.15.73:0/1712671598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85b71a49-23d9-4769-b084-7066e7bb38a0"}]: dispatch 2024-03-18T20:32:38.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:38 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85b71a49-23d9-4769-b084-7066e7bb38a0"}]: dispatch 2024-03-18T20:32:38.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:38 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85b71a49-23d9-4769-b084-7066e7bb38a0"}]': finished 2024-03-18T20:32:38.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:38 smithi073 ceph-mon[47384]: osdmap e44: 8 total, 7 up, 8 in 2024-03-18T20:32:38.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:39.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:38 smithi155 ceph-mon[48344]: osdmap e43: 7 total, 7 up, 7 in 2024-03-18T20:32:39.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:38 smithi155 ceph-mon[48344]: from='client.? 172.21.15.73:0/1712671598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85b71a49-23d9-4769-b084-7066e7bb38a0"}]: dispatch 2024-03-18T20:32:39.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:38 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85b71a49-23d9-4769-b084-7066e7bb38a0"}]: dispatch 2024-03-18T20:32:39.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:38 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85b71a49-23d9-4769-b084-7066e7bb38a0"}]': finished 2024-03-18T20:32:39.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:38 smithi155 ceph-mon[48344]: osdmap e44: 8 total, 7 up, 8 in 2024-03-18T20:32:39.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:39.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:38 smithi046 ceph-mon[43495]: osdmap e43: 7 total, 7 up, 7 in 2024-03-18T20:32:39.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:38 smithi046 ceph-mon[43495]: from='client.? 172.21.15.73:0/1712671598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85b71a49-23d9-4769-b084-7066e7bb38a0"}]: dispatch 2024-03-18T20:32:39.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:38 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "85b71a49-23d9-4769-b084-7066e7bb38a0"}]: dispatch 2024-03-18T20:32:39.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:38 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "85b71a49-23d9-4769-b084-7066e7bb38a0"}]': finished 2024-03-18T20:32:39.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:38 smithi046 ceph-mon[43495]: osdmap e44: 8 total, 7 up, 8 in 2024-03-18T20:32:39.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:39.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:39 smithi073 ceph-mon[47384]: from='client.? 172.21.15.73:0/4052209261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:32:39.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:39 smithi073 ceph-mon[47384]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:40.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:39 smithi155 ceph-mon[48344]: from='client.? 172.21.15.73:0/4052209261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:32:40.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:39 smithi155 ceph-mon[48344]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:40.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:39 smithi046 ceph-mon[43495]: from='client.? 172.21.15.73:0/4052209261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:32:40.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:39 smithi046 ceph-mon[43495]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:42.375 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:42 smithi073 ceph-mon[47384]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:42.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:42 smithi155 ceph-mon[48344]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:42 smithi046 ceph-mon[43495]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:44.183 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:44 smithi073 ceph-mon[47384]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:44 smithi155 ceph-mon[48344]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:44 smithi046 ceph-mon[43495]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:45.413 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:45 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-18T20:32:45.413 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:45 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:45.413 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:45 smithi073 ceph-mon[47384]: Deploying daemon osd.7 on smithi073 2024-03-18T20:32:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:45 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-18T20:32:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:45 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:45 smithi155 ceph-mon[48344]: Deploying daemon osd.7 on smithi073 2024-03-18T20:32:45.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:45 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-18T20:32:45.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:45 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:45 smithi046 ceph-mon[43495]: Deploying daemon osd.7 on smithi073 2024-03-18T20:32:46.401 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:46 smithi073 ceph-mon[47384]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:46.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:46 smithi155 ceph-mon[48344]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:46.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:46 smithi046 ceph-mon[43495]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:48 smithi073 ceph-mon[47384]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:48 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:48 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:48 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:48.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:48 smithi155 ceph-mon[48344]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:48.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:48 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:48.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:48 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:48.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:48 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:48 smithi046 ceph-mon[43495]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:48.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:48.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:48.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:50.158 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:50 smithi073 ceph-mon[47384]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:50.158 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:50.158 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:50.158 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:50.158 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:50.158 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:50.291 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 7 on host 'smithi073' 2024-03-18T20:32:50.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.286+0000 7f58537fe700 1 -- 172.21.15.73:0/3578242336 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f58600611f0 con 0x7f584c0a1120 2024-03-18T20:32:50.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.289+0000 7f58687b0700 1 -- 172.21.15.73:0/3578242336 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f584c0a1120 msgr2=0x7f584c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:50.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.289+0000 7f58687b0700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f584c0a1120 0x7f584c0a35e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f58600fe840 tx=0x7f5854009260 comp rx=0 tx=0).stop 2024-03-18T20:32:50.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.289+0000 7f58687b0700 1 -- 172.21.15.73:0/3578242336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5860101fe0 msgr2=0x7f5860197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:50.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.289+0000 7f58687b0700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5860101fe0 0x7f5860197200 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f585c003730 tx=0x7f585c00f720 comp rx=0 tx=0).stop 2024-03-18T20:32:50.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.289+0000 7f58687b0700 1 -- 172.21.15.73:0/3578242336 shutdown_connections 2024-03-18T20:32:50.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.289+0000 7f58687b0700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f584c0a1120 0x7f584c0a35e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:50.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.289+0000 7f58687b0700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5860100de0 0x7f5860196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:50.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.289+0000 7f58687b0700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5860102980 0x7f586019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:50.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.289+0000 7f58687b0700 1 --2- 172.21.15.73:0/3578242336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5860101fe0 0x7f5860197200 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:50.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.289+0000 7f58687b0700 1 -- 172.21.15.73:0/3578242336 >> 172.21.15.73:0/3578242336 conn(0x7f58600fc380 msgr2=0x7f5860105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:32:50.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.289+0000 7f58687b0700 1 -- 172.21.15.73:0/3578242336 shutdown_connections 2024-03-18T20:32:50.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:32:50.289+0000 7f58687b0700 1 -- 172.21.15.73:0/3578242336 wait complete. 2024-03-18T20:32:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:50 smithi155 ceph-mon[48344]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:50 smithi046 ceph-mon[43495]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:50.689 DEBUG:teuthology.orchestra.run.smithi073:osd.7> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.7.service 2024-03-18T20:32:50.691 INFO:tasks.cephadm:Deploying osd.8 on smithi155 with /dev/vg_nvme/lv_2... 2024-03-18T20:32:50.691 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T20:32:50.720 INFO:journalctl@ceph.osd.7.smithi073.stdout:-- Logs begin at Mon 2024-03-18 20:12:50 UTC. -- 2024-03-18T20:32:50.926 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T20:32:51.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:51 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:51.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:51 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:51.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:51 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:51.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:51 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:51.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:51 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:51.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:51 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:51 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:51 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:51 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:32:52.193 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T20:32:52.193 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T20:32:52.194 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-03-18T20:32:52.194 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-03-18T20:32:52.194 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0177932 s, 589 MB/s 2024-03-18T20:32:52.194 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-03-18T20:32:52.261 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 20:32:52 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T20:32:51.999+0000 7fd7c3df3640 -1 osd.7 0 log_to_monitors true 2024-03-18T20:32:52.512 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:52 smithi155 ceph-mon[48344]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:52.512 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:52 smithi155 ceph-mon[48344]: from='osd.7 [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T20:32:52.512 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:52 smithi155 ceph-mon[48344]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T20:32:52.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:52 smithi046 ceph-mon[43495]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:52.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:52 smithi046 ceph-mon[43495]: from='osd.7 [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T20:32:52.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:52 smithi046 ceph-mon[43495]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T20:32:52.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:52 smithi073 ceph-mon[47384]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:52.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:52 smithi073 ceph-mon[47384]: from='osd.7 [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T20:32:52.604 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:52 smithi073 ceph-mon[47384]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T20:32:52.605 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2024-03-18T20:32:52.824 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T20:32:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-18T20:32:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: osdmap e45: 8 total, 7 up, 8 in 2024-03-18T20:32:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='osd.7 [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:32:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:32:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:53.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:53.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:53.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:53.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:53.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:53.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:53 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: osdmap e45: 8 total, 7 up, 8 in 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='osd.7 [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:53.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:53.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:53 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:53.721 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T20:32:53.274+0000 7fd7b8c56700 -1 osd.7 0 waiting for initial osdmap 2024-03-18T20:32:53.721 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T20:32:53.286+0000 7fd7b1244700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T20:32:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-18T20:32:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: osdmap e45: 8 total, 7 up, 8 in 2024-03-18T20:32:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='osd.7 [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:32:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:32:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:53.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:53.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:53.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:32:53.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:53.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:53.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:32:53.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:53 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:32:53.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.792+0000 7f15ef907700 1 -- 172.21.15.155:0/2458453869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e81041f0 msgr2=0x7f15e81065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:53.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.792+0000 7f15ef907700 1 --2- 172.21.15.155:0/2458453869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e81041f0 0x7f15e81065e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009990 tx=0x7f15e4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:32:53.797 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.792+0000 7f15ef907700 1 -- 172.21.15.155:0/2458453869 shutdown_connections 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.792+0000 7f15ef907700 1 --2- 172.21.15.155:0/2458453869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8106b20 0x7f15e8108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.792+0000 7f15ef907700 1 --2- 172.21.15.155:0/2458453869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15e81018c0 0x7f15e8103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.792+0000 7f15ef907700 1 --2- 172.21.15.155:0/2458453869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e81041f0 0x7f15e81065e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.792+0000 7f15ef907700 1 -- 172.21.15.155:0/2458453869 >> 172.21.15.155:0/2458453869 conn(0x7f15e80fb2d0 msgr2=0x7f15e80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.792+0000 7f15ef907700 1 -- 172.21.15.155:0/2458453869 shutdown_connections 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.792+0000 7f15ef907700 1 -- 172.21.15.155:0/2458453869 wait complete. 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ef907700 1 Processor -- start 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ef907700 1 -- start start 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ef907700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15e81018c0 0x7f15e8196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ef907700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e81041f0 0x7f15e8197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ef907700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8106b20 0x7f15e819e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ef907700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e819e840 con 0x7f15e8106b20 2024-03-18T20:32:53.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ef907700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e819e980 con 0x7f15e81018c0 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ef907700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e819eb40 con 0x7f15e81041f0 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ecea2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e81041f0 0x7f15e8197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ecea2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e81041f0 0x7f15e8197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:35960/0 (socket says 172.21.15.155:35960) 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ecea2700 1 -- 172.21.15.155:0/1117883449 learned_addr learned my addr 172.21.15.155:0/1117883449 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15edea4700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8106b20 0x7f15e819e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ed6a3700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15e81018c0 0x7f15e8196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ecea2700 1 -- 172.21.15.155:0/1117883449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15e81018c0 msgr2=0x7f15e8196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ecea2700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15e81018c0 0x7f15e8196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ecea2700 1 -- 172.21.15.155:0/1117883449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8106b20 msgr2=0x7f15e819e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ecea2700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8106b20 0x7f15e819e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ecea2700 1 -- 172.21.15.155:0/1117883449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e40095f0 con 0x7f15e81041f0 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.793+0000 7f15ecea2700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e81041f0 0x7f15e8197280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f15e40095c0 tx=0x7f15e400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:32:53.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.794+0000 7f15de7fc700 1 -- 172.21.15.155:0/1117883449 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e401d070 con 0x7f15e81041f0 2024-03-18T20:32:53.800 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.794+0000 7f15de7fc700 1 -- 172.21.15.155:0/1117883449 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15e4022470 con 0x7f15e81041f0 2024-03-18T20:32:53.800 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.794+0000 7f15ef907700 1 -- 172.21.15.155:0/1117883449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e819edc0 con 0x7f15e81041f0 2024-03-18T20:32:53.800 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.794+0000 7f15de7fc700 1 -- 172.21.15.155:0/1117883449 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4017650 con 0x7f15e81041f0 2024-03-18T20:32:53.800 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.794+0000 7f15ef907700 1 -- 172.21.15.155:0/1117883449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e819f180 con 0x7f15e81041f0 2024-03-18T20:32:53.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.794+0000 7f15ef907700 1 -- 172.21.15.155:0/1117883449 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e80fce50 con 0x7f15e81041f0 2024-03-18T20:32:53.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.796+0000 7f15de7fc700 1 -- 172.21.15.155:0/1117883449 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f15e4022850 con 0x7f15e81041f0 2024-03-18T20:32:53.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.796+0000 7f15de7fc700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f15d40a1120 0x7f15d40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:32:53.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.796+0000 7f15de7fc700 1 -- 172.21.15.155:0/1117883449 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f15e40cd210 con 0x7f15e81041f0 2024-03-18T20:32:53.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.796+0000 7f15ed6a3700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f15d40a1120 0x7f15d40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:32:53.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.797+0000 7f15ed6a3700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f15d40a1120 0x7f15d40a35e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f15d8005950 tx=0x7f15d80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:32:53.804 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.801+0000 7f15de7fc700 1 -- 172.21.15.155:0/1117883449 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f15e40977d0 con 0x7f15e81041f0 2024-03-18T20:32:54.001 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:32:53.997+0000 7f15ef907700 1 -- 172.21.15.155:0/1117883449 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f15e80611f0 con 0x7f15d40a1120 2024-03-18T20:32:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:54 smithi155 ceph-mon[48344]: Detected new or changed devices on smithi073 2024-03-18T20:32:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:54 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi073 to 5704M 2024-03-18T20:32:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:54 smithi155 ceph-mon[48344]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:54 smithi155 ceph-mon[48344]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-03-18T20:32:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:54 smithi155 ceph-mon[48344]: osdmap e46: 8 total, 7 up, 8 in 2024-03-18T20:32:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:54 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:54 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:54 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:32:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:54 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:32:54.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:54 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:54.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:54 smithi046 ceph-mon[43495]: Detected new or changed devices on smithi073 2024-03-18T20:32:54.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:54 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi073 to 5704M 2024-03-18T20:32:54.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:54 smithi046 ceph-mon[43495]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:54.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:54 smithi046 ceph-mon[43495]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-03-18T20:32:54.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:54 smithi046 ceph-mon[43495]: osdmap e46: 8 total, 7 up, 8 in 2024-03-18T20:32:54.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:54 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:54.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:54 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:54.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:54 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:32:54.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:54 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:32:54.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:54 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:54 smithi073 ceph-mon[47384]: Detected new or changed devices on smithi073 2024-03-18T20:32:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:54 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi073 to 5704M 2024-03-18T20:32:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:54 smithi073 ceph-mon[47384]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:54 smithi073 ceph-mon[47384]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-03-18T20:32:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:54 smithi073 ceph-mon[47384]: osdmap e46: 8 total, 7 up, 8 in 2024-03-18T20:32:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:54 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:54 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:54 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:32:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:54 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:32:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:54 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:32:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:55 smithi155 ceph-mon[48344]: purged_snaps scrub starts 2024-03-18T20:32:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:55 smithi155 ceph-mon[48344]: purged_snaps scrub ok 2024-03-18T20:32:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:55 smithi155 ceph-mon[48344]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:32:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:55 smithi155 ceph-mon[48344]: osd.7 [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190] boot 2024-03-18T20:32:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:55 smithi155 ceph-mon[48344]: osdmap e47: 8 total, 8 up, 8 in 2024-03-18T20:32:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:55 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:55 smithi155 ceph-mon[48344]: osdmap e48: 8 total, 8 up, 8 in 2024-03-18T20:32:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:55 smithi046 ceph-mon[43495]: purged_snaps scrub starts 2024-03-18T20:32:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:55 smithi046 ceph-mon[43495]: purged_snaps scrub ok 2024-03-18T20:32:55.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:55 smithi046 ceph-mon[43495]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:32:55.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:55 smithi046 ceph-mon[43495]: osd.7 [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190] boot 2024-03-18T20:32:55.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:55 smithi046 ceph-mon[43495]: osdmap e47: 8 total, 8 up, 8 in 2024-03-18T20:32:55.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:55 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:55.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:55 smithi046 ceph-mon[43495]: osdmap e48: 8 total, 8 up, 8 in 2024-03-18T20:32:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:55 smithi073 ceph-mon[47384]: purged_snaps scrub starts 2024-03-18T20:32:55.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:55 smithi073 ceph-mon[47384]: purged_snaps scrub ok 2024-03-18T20:32:55.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:55 smithi073 ceph-mon[47384]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:32:55.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:55 smithi073 ceph-mon[47384]: osd.7 [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190] boot 2024-03-18T20:32:55.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:55 smithi073 ceph-mon[47384]: osdmap e47: 8 total, 8 up, 8 in 2024-03-18T20:32:55.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:55 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:32:55.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:55 smithi073 ceph-mon[47384]: osdmap e48: 8 total, 8 up, 8 in 2024-03-18T20:32:56.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:56 smithi155 ceph-mon[48344]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:56.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:56 smithi155 ceph-mon[48344]: osdmap e49: 8 total, 8 up, 8 in 2024-03-18T20:32:56.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:56 smithi155 ceph-mon[48344]: from='client.? 172.21.15.155:0/46989272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45ae5d47-1129-43f7-a5b5-d7e3336ae755"}]: dispatch 2024-03-18T20:32:56.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:56 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45ae5d47-1129-43f7-a5b5-d7e3336ae755"}]: dispatch 2024-03-18T20:32:56.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:56 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45ae5d47-1129-43f7-a5b5-d7e3336ae755"}]': finished 2024-03-18T20:32:56.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:56 smithi155 ceph-mon[48344]: osdmap e50: 9 total, 8 up, 9 in 2024-03-18T20:32:56.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:56 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:32:56.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:56 smithi046 ceph-mon[43495]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:56.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:56 smithi046 ceph-mon[43495]: osdmap e49: 8 total, 8 up, 8 in 2024-03-18T20:32:56.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:56 smithi046 ceph-mon[43495]: from='client.? 172.21.15.155:0/46989272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45ae5d47-1129-43f7-a5b5-d7e3336ae755"}]: dispatch 2024-03-18T20:32:56.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:56 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45ae5d47-1129-43f7-a5b5-d7e3336ae755"}]: dispatch 2024-03-18T20:32:56.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:56 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45ae5d47-1129-43f7-a5b5-d7e3336ae755"}]': finished 2024-03-18T20:32:56.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:56 smithi046 ceph-mon[43495]: osdmap e50: 9 total, 8 up, 9 in 2024-03-18T20:32:56.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:56 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:32:56.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:56 smithi073 ceph-mon[47384]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-03-18T20:32:56.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:56 smithi073 ceph-mon[47384]: osdmap e49: 8 total, 8 up, 8 in 2024-03-18T20:32:56.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:56 smithi073 ceph-mon[47384]: from='client.? 172.21.15.155:0/46989272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45ae5d47-1129-43f7-a5b5-d7e3336ae755"}]: dispatch 2024-03-18T20:32:56.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:56 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45ae5d47-1129-43f7-a5b5-d7e3336ae755"}]: dispatch 2024-03-18T20:32:56.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:56 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45ae5d47-1129-43f7-a5b5-d7e3336ae755"}]': finished 2024-03-18T20:32:56.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:56 smithi073 ceph-mon[47384]: osdmap e50: 9 total, 8 up, 9 in 2024-03-18T20:32:56.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:56 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:32:57.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:57 smithi155 ceph-mon[48344]: from='client.? 172.21.15.155:0/1457743894' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:32:57.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:57 smithi155 ceph-mon[48344]: osdmap e51: 9 total, 8 up, 9 in 2024-03-18T20:32:57.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:57 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:32:57.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:57 smithi046 ceph-mon[43495]: from='client.? 172.21.15.155:0/1457743894' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:32:57.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:57 smithi046 ceph-mon[43495]: osdmap e51: 9 total, 8 up, 9 in 2024-03-18T20:32:57.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:57 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:32:57.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:57 smithi073 ceph-mon[47384]: from='client.? 172.21.15.155:0/1457743894' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:32:57.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:57 smithi073 ceph-mon[47384]: osdmap e51: 9 total, 8 up, 9 in 2024-03-18T20:32:57.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:57 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:32:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:32:58 smithi155 ceph-mon[48344]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:32:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:32:58 smithi046 ceph-mon[43495]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:32:58.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:32:58 smithi073 ceph-mon[47384]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:00 smithi155 ceph-mon[48344]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:00.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:00 smithi046 ceph-mon[43495]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:00 smithi073 ceph-mon[47384]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:02.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:02 smithi155 ceph-mon[48344]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:02.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:02 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-18T20:33:02.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:02 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:02 smithi046 ceph-mon[43495]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:02 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-18T20:33:02.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:02 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:02.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:02 smithi073 ceph-mon[47384]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:02.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:02 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-18T20:33:02.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:02 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:03.585 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:03 smithi155 ceph-mon[48344]: Deploying daemon osd.8 on smithi155 2024-03-18T20:33:03.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:03 smithi073 ceph-mon[47384]: Deploying daemon osd.8 on smithi155 2024-03-18T20:33:03.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:03 smithi046 ceph-mon[43495]: Deploying daemon osd.8 on smithi155 2024-03-18T20:33:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:04 smithi155 ceph-mon[48344]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:04.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:04 smithi073 ceph-mon[47384]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:04.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:04 smithi046 ceph-mon[43495]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:06.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:05 smithi155 ceph-mon[48344]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:06.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:05 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:06.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:05 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:06.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:05 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:06.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:05 smithi046 ceph-mon[43495]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:06.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:05 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:06.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:05 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:06.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:05 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:05 smithi073 ceph-mon[47384]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:06.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:05 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:06.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:05 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:06.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:05 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:07.614 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.610+0000 7f15de7fc700 1 -- 172.21.15.155:0/1117883449 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f15e80611f0 con 0x7f15d40a1120 2024-03-18T20:33:07.614 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 8 on host 'smithi155' 2024-03-18T20:33:07.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.613+0000 7f15ef907700 1 -- 172.21.15.155:0/1117883449 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f15d40a1120 msgr2=0x7f15d40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:07.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.613+0000 7f15ef907700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f15d40a1120 0x7f15d40a35e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f15d8005950 tx=0x7f15d80058e0 comp rx=0 tx=0).stop 2024-03-18T20:33:07.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.613+0000 7f15ef907700 1 -- 172.21.15.155:0/1117883449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e81041f0 msgr2=0x7f15e8197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:07.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.613+0000 7f15ef907700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e81041f0 0x7f15e8197280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f15e40095c0 tx=0x7f15e400f690 comp rx=0 tx=0).stop 2024-03-18T20:33:07.616 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.613+0000 7f15ef907700 1 -- 172.21.15.155:0/1117883449 shutdown_connections 2024-03-18T20:33:07.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.613+0000 7f15ef907700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f15d40a1120 0x7f15d40a35e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:07.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.613+0000 7f15ef907700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8106b20 0x7f15e819e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:07.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.613+0000 7f15ef907700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15e81018c0 0x7f15e8196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:07.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.613+0000 7f15ef907700 1 --2- 172.21.15.155:0/1117883449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15e81041f0 0x7f15e8197280 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:07.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.613+0000 7f15ef907700 1 -- 172.21.15.155:0/1117883449 >> 172.21.15.155:0/1117883449 conn(0x7f15e80fb2d0 msgr2=0x7f15e8103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:33:07.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.613+0000 7f15ef907700 1 -- 172.21.15.155:0/1117883449 shutdown_connections 2024-03-18T20:33:07.617 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:07.613+0000 7f15ef907700 1 -- 172.21.15.155:0/1117883449 wait complete. 2024-03-18T20:33:07.923 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:07.924 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:07.924 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:07.924 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:07.924 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:07.924 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:07 smithi155 ceph-mon[48344]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:07.924 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:07.924 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:07.924 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:07 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:07.957 DEBUG:teuthology.orchestra.run.smithi155:osd.8> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.8.service 2024-03-18T20:33:07.959 INFO:tasks.cephadm:Deploying osd.9 on smithi046 with /dev/vg_nvme/lv_1... 2024-03-18T20:33:07.959 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-18T20:33:08.006 INFO:journalctl@ceph.osd.8.smithi155.stdout:-- Logs begin at Mon 2024-03-18 20:13:03 UTC. -- 2024-03-18T20:33:08.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:07 smithi073 ceph-mon[47384]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:07 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:08.229 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:33:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:07 smithi046 ceph-mon[43495]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:07 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-18T20:33:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-18T20:33:09.780 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-03-18T20:33:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-03-18T20:33:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00907451 s, 1.2 GB/s 2024-03-18T20:33:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-03-18T20:33:09.790 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 20:33:09 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T20:33:09.459+0000 7f0f8fa8f640 -1 osd.8 0 log_to_monitors true 2024-03-18T20:33:10.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:10 smithi046 ceph-mon[43495]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:10.098 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:10 smithi046 ceph-mon[43495]: from='osd.8 [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T20:33:10.098 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:10 smithi046 ceph-mon[43495]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T20:33:10.098 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:10 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:10.098 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:10 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:10.098 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:10 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:10.098 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:10 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:10.098 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:10 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:10.098 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:10 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:10.098 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:10 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:10.098 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:10 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:10.098 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:10 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:10.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi046:vg_nvme/lv_1 2024-03-18T20:33:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:33:10.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:10 smithi073 ceph-mon[47384]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:10.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:10 smithi073 ceph-mon[47384]: from='osd.8 [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T20:33:10.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:10 smithi073 ceph-mon[47384]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T20:33:10.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:10 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:10.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:10 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:10.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:10 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:10.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:10 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:10.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:10 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:10.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:10 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:10.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:10 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:10.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:10 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:10.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:10 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:10 smithi155 ceph-mon[48344]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:10 smithi155 ceph-mon[48344]: from='osd.8 [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T20:33:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:10 smithi155 ceph-mon[48344]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T20:33:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:10 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:10 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:10 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:10 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:10 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:10 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:10 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:10 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:10 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:11.205 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:11 smithi046 ceph-mon[43495]: Detected new or changed devices on smithi155 2024-03-18T20:33:11.205 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:11 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi155 to 5704M 2024-03-18T20:33:11.205 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:11 smithi046 ceph-mon[43495]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-18T20:33:11.205 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:11 smithi046 ceph-mon[43495]: osdmap e52: 9 total, 8 up, 9 in 2024-03-18T20:33:11.205 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:11 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:11.206 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:11 smithi046 ceph-mon[43495]: from='osd.8 [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:33:11.206 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:11 smithi046 ceph-mon[43495]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:33:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.400+0000 7f7de6872700 1 -- 172.21.15.46:0/908455247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de0101890 msgr2=0x7f7de0108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.400+0000 7f7de6872700 1 --2- 172.21.15.46:0/908455247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de0101890 0x7f7de0108140 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800b0d0 tx=0x7f7dd800b3e0 comp rx=0 tx=0).stop 2024-03-18T20:33:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.400+0000 7f7de6872700 1 -- 172.21.15.46:0/908455247 shutdown_connections 2024-03-18T20:33:11.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.400+0000 7f7de6872700 1 --2- 172.21.15.46:0/908455247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de0101890 0x7f7de0108140 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:11.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.400+0000 7f7de6872700 1 --2- 172.21.15.46:0/908455247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7de00ffcf0 0x7f7de0100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:11.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.400+0000 7f7de6872700 1 --2- 172.21.15.46:0/908455247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de0100ef0 0x7f7de0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:11.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.400+0000 7f7de6872700 1 -- 172.21.15.46:0/908455247 >> 172.21.15.46:0/908455247 conn(0x7f7de00fb2b0 msgr2=0x7f7de00fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:33:11.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.400+0000 7f7de6872700 1 -- 172.21.15.46:0/908455247 shutdown_connections 2024-03-18T20:33:11.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.400+0000 7f7de6872700 1 -- 172.21.15.46:0/908455247 wait complete. 2024-03-18T20:33:11.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.400+0000 7f7de6872700 1 Processor -- start 2024-03-18T20:33:11.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.401+0000 7f7de6872700 1 -- start start 2024-03-18T20:33:11.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.401+0000 7f7de6872700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de00ffcf0 0x7f7de0198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:33:11.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.401+0000 7f7de6872700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de0100ef0 0x7f7de0199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:33:11.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.401+0000 7f7de6872700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7de0101890 0x7f7de01a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:33:11.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.401+0000 7f7de6872700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de01a0b10 con 0x7f7de00ffcf0 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.401+0000 7f7de6872700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de01a0c50 con 0x7f7de0101890 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.401+0000 7f7de6872700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de01a0de0 con 0x7f7de0100ef0 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.401+0000 7f7ddffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de00ffcf0 0x7f7de0198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.401+0000 7f7ddffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de00ffcf0 0x7f7de0198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54362/0 (socket says 172.21.15.46:54362) 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.401+0000 7f7ddffff700 1 -- 172.21.15.46:0/638972920 learned_addr learned my addr 172.21.15.46:0/638972920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.401+0000 7f7ddf7fe700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de0100ef0 0x7f7de0199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.401+0000 7f7de4e0f700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7de0101890 0x7f7de01a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.402+0000 7f7ddffff700 1 -- 172.21.15.46:0/638972920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de0100ef0 msgr2=0x7f7de0199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.402+0000 7f7ddffff700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de0100ef0 0x7f7de0199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.402+0000 7f7ddffff700 1 -- 172.21.15.46:0/638972920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7de0101890 msgr2=0x7f7de01a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.402+0000 7f7ddffff700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7de0101890 0x7f7de01a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.402+0000 7f7ddffff700 1 -- 172.21.15.46:0/638972920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8009d00 con 0x7f7de00ffcf0 2024-03-18T20:33:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.402+0000 7f7de4e0f700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7de0101890 0x7f7de01a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:33:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.402+0000 7f7ddffff700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de00ffcf0 0x7f7de0198fd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000d9b0 tx=0x7f7dd000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:33:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.402+0000 7f7ddd7fa700 1 -- 172.21.15.46:0/638972920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000b950 con 0x7f7de00ffcf0 2024-03-18T20:33:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.402+0000 7f7ddd7fa700 1 -- 172.21.15.46:0/638972920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dd0007620 con 0x7f7de00ffcf0 2024-03-18T20:33:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.402+0000 7f7de6872700 1 -- 172.21.15.46:0/638972920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de01a10c0 con 0x7f7de00ffcf0 2024-03-18T20:33:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.402+0000 7f7ddd7fa700 1 -- 172.21.15.46:0/638972920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000f3c0 con 0x7f7de00ffcf0 2024-03-18T20:33:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.402+0000 7f7de6872700 1 -- 172.21.15.46:0/638972920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de01a15c0 con 0x7f7de00ffcf0 2024-03-18T20:33:11.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.403+0000 7f7de6872700 1 -- 172.21.15.46:0/638972920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de004ea80 con 0x7f7de00ffcf0 2024-03-18T20:33:11.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.404+0000 7f7ddd7fa700 1 -- 172.21.15.46:0/638972920 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7dd0010640 con 0x7f7de00ffcf0 2024-03-18T20:33:11.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.405+0000 7f7ddd7fa700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7dc80a1120 0x7f7dc80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:33:11.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.405+0000 7f7ddd7fa700 1 -- 172.21.15.46:0/638972920 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5549+0+0 (secure 0 0 0) 0x7f7dd00cc030 con 0x7f7de00ffcf0 2024-03-18T20:33:11.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.410+0000 7f7ddf7fe700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7dc80a1120 0x7f7dc80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:33:11.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.410+0000 7f7ddd7fa700 1 -- 172.21.15.46:0/638972920 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7dd0096290 con 0x7f7de00ffcf0 2024-03-18T20:33:11.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.410+0000 7f7ddf7fe700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7dc80a1120 0x7f7dc80a35e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4005950 tx=0x7f7dd400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:33:11.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:11 smithi073 ceph-mon[47384]: Detected new or changed devices on smithi155 2024-03-18T20:33:11.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:11 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi155 to 5704M 2024-03-18T20:33:11.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:11 smithi073 ceph-mon[47384]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-18T20:33:11.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:11 smithi073 ceph-mon[47384]: osdmap e52: 9 total, 8 up, 9 in 2024-03-18T20:33:11.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:11 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:11.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:11 smithi073 ceph-mon[47384]: from='osd.8 [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:33:11.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:11 smithi073 ceph-mon[47384]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:33:11.541 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 20:33:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T20:33:11.069+0000 7f0f848f2700 -1 osd.8 0 waiting for initial osdmap 2024-03-18T20:33:11.541 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 20:33:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T20:33:11.083+0000 7f0f7dee2700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T20:33:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:11 smithi155 ceph-mon[48344]: Detected new or changed devices on smithi155 2024-03-18T20:33:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:11 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi155 to 5704M 2024-03-18T20:33:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:11 smithi155 ceph-mon[48344]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-18T20:33:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:11 smithi155 ceph-mon[48344]: osdmap e52: 9 total, 8 up, 9 in 2024-03-18T20:33:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:11 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:11 smithi155 ceph-mon[48344]: from='osd.8 [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:33:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:11 smithi155 ceph-mon[48344]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:33:11.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:11.633+0000 7f7de6872700 1 -- 172.21.15.46:0/638972920 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7de00fd490 con 0x7f7dc80a1120 2024-03-18T20:33:12.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:12 smithi046 ceph-mon[43495]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:12.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:12 smithi046 ceph-mon[43495]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-18T20:33:12.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:12 smithi046 ceph-mon[43495]: osdmap e53: 9 total, 8 up, 9 in 2024-03-18T20:33:12.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:12.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:12.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:33:12.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:33:12.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:12 smithi073 ceph-mon[47384]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:12 smithi073 ceph-mon[47384]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-18T20:33:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:12 smithi073 ceph-mon[47384]: osdmap e53: 9 total, 8 up, 9 in 2024-03-18T20:33:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:33:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:33:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:12 smithi155 ceph-mon[48344]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:12 smithi155 ceph-mon[48344]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-18T20:33:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:12 smithi155 ceph-mon[48344]: osdmap e53: 9 total, 8 up, 9 in 2024-03-18T20:33:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:33:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:33:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:13.474 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:13 smithi073 ceph-mon[47384]: purged_snaps scrub starts 2024-03-18T20:33:13.474 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:13 smithi073 ceph-mon[47384]: purged_snaps scrub ok 2024-03-18T20:33:13.474 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:13 smithi073 ceph-mon[47384]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:33:13.474 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:13 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:13.474 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:13 smithi073 ceph-mon[47384]: osd.8 [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462] boot 2024-03-18T20:33:13.474 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:13 smithi073 ceph-mon[47384]: osdmap e54: 9 total, 9 up, 9 in 2024-03-18T20:33:13.474 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:13 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:13 smithi155 ceph-mon[48344]: purged_snaps scrub starts 2024-03-18T20:33:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:13 smithi155 ceph-mon[48344]: purged_snaps scrub ok 2024-03-18T20:33:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:13 smithi155 ceph-mon[48344]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:33:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:13 smithi155 ceph-mon[48344]: osd.8 [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462] boot 2024-03-18T20:33:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:13 smithi155 ceph-mon[48344]: osdmap e54: 9 total, 9 up, 9 in 2024-03-18T20:33:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:13.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:13 smithi046 ceph-mon[43495]: purged_snaps scrub starts 2024-03-18T20:33:13.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:13 smithi046 ceph-mon[43495]: purged_snaps scrub ok 2024-03-18T20:33:13.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:13 smithi046 ceph-mon[43495]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:33:13.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:13.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:13 smithi046 ceph-mon[43495]: osd.8 [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462] boot 2024-03-18T20:33:13.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:13 smithi046 ceph-mon[43495]: osdmap e54: 9 total, 9 up, 9 in 2024-03-18T20:33:13.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:33:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:14 smithi073 ceph-mon[47384]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:14 smithi073 ceph-mon[47384]: osdmap e55: 9 total, 9 up, 9 in 2024-03-18T20:33:14.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:14 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2579497713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8f8c910-b6f1-496a-b5ca-ef6f68fef718"}]: dispatch 2024-03-18T20:33:14.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:14 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2579497713' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8f8c910-b6f1-496a-b5ca-ef6f68fef718"}]': finished 2024-03-18T20:33:14.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:14 smithi073 ceph-mon[47384]: osdmap e56: 10 total, 9 up, 10 in 2024-03-18T20:33:14.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:14 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:14 smithi155 ceph-mon[48344]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:14 smithi155 ceph-mon[48344]: osdmap e55: 9 total, 9 up, 9 in 2024-03-18T20:33:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:14 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2579497713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8f8c910-b6f1-496a-b5ca-ef6f68fef718"}]: dispatch 2024-03-18T20:33:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:14 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2579497713' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8f8c910-b6f1-496a-b5ca-ef6f68fef718"}]': finished 2024-03-18T20:33:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:14 smithi155 ceph-mon[48344]: osdmap e56: 10 total, 9 up, 10 in 2024-03-18T20:33:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:14 smithi046 ceph-mon[43495]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-03-18T20:33:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:14 smithi046 ceph-mon[43495]: osdmap e55: 9 total, 9 up, 9 in 2024-03-18T20:33:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:14 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2579497713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8f8c910-b6f1-496a-b5ca-ef6f68fef718"}]: dispatch 2024-03-18T20:33:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:14 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2579497713' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8f8c910-b6f1-496a-b5ca-ef6f68fef718"}]': finished 2024-03-18T20:33:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:14 smithi046 ceph-mon[43495]: osdmap e56: 10 total, 9 up, 10 in 2024-03-18T20:33:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:15.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:15 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1717349905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:33:15.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:15 smithi073 ceph-mon[47384]: osdmap e57: 10 total, 9 up, 10 in 2024-03-18T20:33:15.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:15.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:15 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1717349905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:33:15.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:15 smithi155 ceph-mon[48344]: osdmap e57: 10 total, 9 up, 10 in 2024-03-18T20:33:15.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:15 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:15.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:15 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1717349905' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:33:15.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:15 smithi046 ceph-mon[43495]: osdmap e57: 10 total, 9 up, 10 in 2024-03-18T20:33:15.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:15 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:16.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:16 smithi073 ceph-mon[47384]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2024-03-18T20:33:16.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:16 smithi073 ceph-mon[47384]: osdmap e58: 10 total, 9 up, 10 in 2024-03-18T20:33:16.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:16 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:16 smithi155 ceph-mon[48344]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2024-03-18T20:33:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:16 smithi155 ceph-mon[48344]: osdmap e58: 10 total, 9 up, 10 in 2024-03-18T20:33:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:16 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:16.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:16 smithi046 ceph-mon[43495]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2024-03-18T20:33:16.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:16 smithi046 ceph-mon[43495]: osdmap e58: 10 total, 9 up, 10 in 2024-03-18T20:33:16.550 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:16 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:18.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:18 smithi046 ceph-mon[43495]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T20:33:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:18 smithi073 ceph-mon[47384]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T20:33:18.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:18 smithi155 ceph-mon[48344]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T20:33:20.267 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:20 smithi046 ceph-mon[43495]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-18T20:33:20.267 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-18T20:33:20.267 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:20 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:20.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:20 smithi073 ceph-mon[47384]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-18T20:33:20.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:20 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-18T20:33:20.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:20 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:20.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:20 smithi155 ceph-mon[48344]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-18T20:33:20.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-18T20:33:20.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:20 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:21.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:21 smithi073 ceph-mon[47384]: Deploying daemon osd.9 on smithi046 2024-03-18T20:33:21.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:21 smithi155 ceph-mon[48344]: Deploying daemon osd.9 on smithi046 2024-03-18T20:33:21.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:21 smithi046 ceph-mon[43495]: Deploying daemon osd.9 on smithi046 2024-03-18T20:33:22.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:22 smithi046 ceph-mon[43495]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-18T20:33:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:22 smithi073 ceph-mon[47384]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-18T20:33:22.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:22 smithi155 ceph-mon[48344]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-18T20:33:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:24 smithi073 ceph-mon[47384]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-18T20:33:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:24 smithi155 ceph-mon[48344]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-18T20:33:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:24 smithi046 ceph-mon[43495]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-18T20:33:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 9 on host 'smithi046' 2024-03-18T20:33:25.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.886+0000 7f7ddd7fa700 1 -- 172.21.15.46:0/638972920 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7de00fd490 con 0x7f7dc80a1120 2024-03-18T20:33:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.889+0000 7f7de6872700 1 -- 172.21.15.46:0/638972920 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7dc80a1120 msgr2=0x7f7dc80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.889+0000 7f7de6872700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7dc80a1120 0x7f7dc80a35e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4005950 tx=0x7f7dd400b410 comp rx=0 tx=0).stop 2024-03-18T20:33:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.889+0000 7f7de6872700 1 -- 172.21.15.46:0/638972920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de00ffcf0 msgr2=0x7f7de0198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.889+0000 7f7de6872700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de00ffcf0 0x7f7de0198fd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000d9b0 tx=0x7f7dd000dcc0 comp rx=0 tx=0).stop 2024-03-18T20:33:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.889+0000 7f7de6872700 1 -- 172.21.15.46:0/638972920 shutdown_connections 2024-03-18T20:33:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.889+0000 7f7de6872700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7dc80a1120 0x7f7dc80a35e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.889+0000 7f7de6872700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de00ffcf0 0x7f7de0198fd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.889+0000 7f7de6872700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7de0101890 0x7f7de01a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.889+0000 7f7de6872700 1 --2- 172.21.15.46:0/638972920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de0100ef0 0x7f7de0199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.889+0000 7f7de6872700 1 -- 172.21.15.46:0/638972920 >> 172.21.15.46:0/638972920 conn(0x7f7de00fb2b0 msgr2=0x7f7de00fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:33:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.889+0000 7f7de6872700 1 -- 172.21.15.46:0/638972920 shutdown_connections 2024-03-18T20:33:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:33:25.889+0000 7f7de6872700 1 -- 172.21.15.46:0/638972920 wait complete. 2024-03-18T20:33:26.395 DEBUG:teuthology.orchestra.run.smithi046:osd.9> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.9.service 2024-03-18T20:33:26.397 INFO:tasks.cephadm:Deploying osd.10 on smithi073 with /dev/vg_nvme/lv_1... 2024-03-18T20:33:26.398 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-18T20:33:26.425 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:26 smithi046 ceph-mon[43495]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T20:33:26.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:26.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:26.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:26.435 INFO:journalctl@ceph.osd.9.smithi046.stdout:-- Logs begin at Mon 2024-03-18 20:12:57 UTC. -- 2024-03-18T20:33:26.444 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:26 smithi073 ceph-mon[47384]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T20:33:26.445 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.445 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.445 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:26.445 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:26.445 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.445 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.446 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.446 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:26 smithi155 ceph-mon[48344]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T20:33:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T20:33:27.798 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 20:33:27 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T20:33:27.437+0000 7f4d01302640 -1 osd.9 0 log_to_monitors true 2024-03-18T20:33:28.003 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-18T20:33:28.003 INFO:teuthology.orchestra.run.smithi073.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-18T20:33:28.003 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10+0 records in 2024-03-18T20:33:28.003 INFO:teuthology.orchestra.run.smithi073.stderr:10+0 records out 2024-03-18T20:33:28.003 INFO:teuthology.orchestra.run.smithi073.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113852 s, 921 MB/s 2024-03-18T20:33:28.003 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping successful for: 2024-03-18T20:33:28.343 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:28 smithi073 ceph-mon[47384]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2024-03-18T20:33:28.343 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:28 smithi073 ceph-mon[47384]: from='osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T20:33:28.395 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi073:vg_nvme/lv_1 2024-03-18T20:33:28.487 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:28 smithi046 ceph-mon[43495]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2024-03-18T20:33:28.487 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:28 smithi046 ceph-mon[43495]: from='osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T20:33:28.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:28 smithi155 ceph-mon[48344]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2024-03-18T20:33:28.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:28 smithi155 ceph-mon[48344]: from='osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T20:33:28.614 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T20:33:29.416 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-18T20:33:29.416 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: osdmap e59: 10 total, 9 up, 10 in 2024-03-18T20:33:29.416 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:29.416 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-03-18T20:33:29.416 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:29.416 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:29.417 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:29.417 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:29.417 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:29.417 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:29.417 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:29.417 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:29.417 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:29.417 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:29 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:29.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-18T20:33:29.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: osdmap e59: 10 total, 9 up, 10 in 2024-03-18T20:33:29.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:29.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-03-18T20:33:29.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:29.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:29.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:29.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:29.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:29.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:29.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:29.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:29.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:29.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:29 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:29.546 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T20:33:29.167+0000 7f4cf6165700 -1 osd.9 0 waiting for initial osdmap 2024-03-18T20:33:29.547 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T20:33:29.187+0000 7f4cef755700 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T20:33:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-18T20:33:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: osdmap e59: 10 total, 9 up, 10 in 2024-03-18T20:33:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-03-18T20:33:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:29.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:29.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:29.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:29.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:29.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:29 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:29.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.590+0000 7f48b7126700 1 -- 172.21.15.73:0/882235350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48b0101fe0 msgr2=0x7f48b0102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:29.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.590+0000 7f48b7126700 1 --2- 172.21.15.73:0/882235350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48b0101fe0 0x7f48b0102440 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f489c0099c0 tx=0x7f489c009cd0 comp rx=0 tx=0).stop 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 -- 172.21.15.73:0/882235350 shutdown_connections 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 --2- 172.21.15.73:0/882235350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b0100de0 0x7f48b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 --2- 172.21.15.73:0/882235350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b0102980 0x7f48b0109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 --2- 172.21.15.73:0/882235350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48b0101fe0 0x7f48b0102440 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 -- 172.21.15.73:0/882235350 >> 172.21.15.73:0/882235350 conn(0x7f48b00fc380 msgr2=0x7f48b00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 -- 172.21.15.73:0/882235350 shutdown_connections 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 -- 172.21.15.73:0/882235350 wait complete. 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 Processor -- start 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 -- start start 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b0100de0 0x7f48b0196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48b0101fe0 0x7f48b0197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b0102980 0x7f48b019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:33:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b019e900 con 0x7f48b0100de0 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b019ea40 con 0x7f48b0101fe0 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48b7126700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b019ec00 con 0x7f48b0102980 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48affff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48b0101fe0 0x7f48b0197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48affff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48b0101fe0 0x7f48b0197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56282/0 (socket says 172.21.15.73:56282) 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.591+0000 7f48affff700 1 -- 172.21.15.73:0/285599412 learned_addr learned my addr 172.21.15.73:0/285599412 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48b4ec2700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b0100de0 0x7f48b0196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48b56c3700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b0102980 0x7f48b019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48affff700 1 -- 172.21.15.73:0/285599412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b0102980 msgr2=0x7f48b019e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48affff700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b0102980 0x7f48b019e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48affff700 1 -- 172.21.15.73:0/285599412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b0100de0 msgr2=0x7f48b0196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48affff700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b0100de0 0x7f48b0196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48affff700 1 -- 172.21.15.73:0/285599412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0095f0 con 0x7f48b0101fe0 2024-03-18T20:33:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48b56c3700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b0102980 0x7f48b019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:33:29.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48affff700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48b0101fe0 0x7f48b0197340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f489c0036f0 tx=0x7f489c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:33:29.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48adffb700 1 -- 172.21.15.73:0/285599412 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c01d070 con 0x7f48b0101fe0 2024-03-18T20:33:29.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48adffb700 1 -- 172.21.15.73:0/285599412 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f489c004200 con 0x7f48b0101fe0 2024-03-18T20:33:29.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48b7126700 1 -- 172.21.15.73:0/285599412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b019ee80 con 0x7f48b0101fe0 2024-03-18T20:33:29.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48adffb700 1 -- 172.21.15.73:0/285599412 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c00f460 con 0x7f48b0101fe0 2024-03-18T20:33:29.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.592+0000 7f48b7126700 1 -- 172.21.15.73:0/285599412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b019f290 con 0x7f48b0101fe0 2024-03-18T20:33:29.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.593+0000 7f48b7126700 1 -- 172.21.15.73:0/285599412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b01082b0 con 0x7f48b0101fe0 2024-03-18T20:33:29.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.594+0000 7f48adffb700 1 -- 172.21.15.73:0/285599412 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f489c022600 con 0x7f48b0101fe0 2024-03-18T20:33:29.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.595+0000 7f48adffb700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f48a00a11a0 0x7f48a00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:33:29.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.595+0000 7f48adffb700 1 -- 172.21.15.73:0/285599412 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5981+0+0 (secure 0 0 0) 0x7f489c0cd350 con 0x7f48b0101fe0 2024-03-18T20:33:29.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.595+0000 7f48b4ec2700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f48a00a11a0 0x7f48a00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:33:29.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.596+0000 7f48b4ec2700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f48a00a11a0 0x7f48a00a3660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48b0198290 tx=0x7f48a4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:33:29.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.600+0000 7f48adffb700 1 -- 172.21.15.73:0/285599412 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f489c05d9f0 con 0x7f48b0101fe0 2024-03-18T20:33:29.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:29.804+0000 7f48b7126700 1 -- 172.21.15.73:0/285599412 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f48b00611f0 con 0x7f48a00a11a0 2024-03-18T20:33:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:30 smithi073 ceph-mon[47384]: Detected new or changed devices on smithi046 2024-03-18T20:33:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:30 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi046 to 4279M 2024-03-18T20:33:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:30 smithi073 ceph-mon[47384]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2024-03-18T20:33:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:30 smithi073 ceph-mon[47384]: from='osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-03-18T20:33:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:30 smithi073 ceph-mon[47384]: osdmap e60: 10 total, 9 up, 10 in 2024-03-18T20:33:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:33:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:33:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:30 smithi155 ceph-mon[48344]: Detected new or changed devices on smithi046 2024-03-18T20:33:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:30 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi046 to 4279M 2024-03-18T20:33:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:30 smithi155 ceph-mon[48344]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2024-03-18T20:33:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:30 smithi155 ceph-mon[48344]: from='osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-03-18T20:33:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:30 smithi155 ceph-mon[48344]: osdmap e60: 10 total, 9 up, 10 in 2024-03-18T20:33:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:33:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:33:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:30 smithi046 ceph-mon[43495]: Detected new or changed devices on smithi046 2024-03-18T20:33:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:30 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi046 to 4279M 2024-03-18T20:33:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:30 smithi046 ceph-mon[43495]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2024-03-18T20:33:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:30 smithi046 ceph-mon[43495]: from='osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-03-18T20:33:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:30 smithi046 ceph-mon[43495]: osdmap e60: 10 total, 9 up, 10 in 2024-03-18T20:33:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:33:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:33:30.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:31.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:31 smithi073 ceph-mon[47384]: purged_snaps scrub starts 2024-03-18T20:33:31.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:31 smithi073 ceph-mon[47384]: purged_snaps scrub ok 2024-03-18T20:33:31.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:31 smithi073 ceph-mon[47384]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:33:31.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:31 smithi073 ceph-mon[47384]: osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015] boot 2024-03-18T20:33:31.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:31 smithi073 ceph-mon[47384]: osdmap e61: 10 total, 10 up, 10 in 2024-03-18T20:33:31.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:31 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:31 smithi155 ceph-mon[48344]: purged_snaps scrub starts 2024-03-18T20:33:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:31 smithi155 ceph-mon[48344]: purged_snaps scrub ok 2024-03-18T20:33:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:31 smithi155 ceph-mon[48344]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:33:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:31 smithi155 ceph-mon[48344]: osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015] boot 2024-03-18T20:33:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:31 smithi155 ceph-mon[48344]: osdmap e61: 10 total, 10 up, 10 in 2024-03-18T20:33:31.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:31 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:31.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:31 smithi046 ceph-mon[43495]: purged_snaps scrub starts 2024-03-18T20:33:31.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:31 smithi046 ceph-mon[43495]: purged_snaps scrub ok 2024-03-18T20:33:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:31 smithi046 ceph-mon[43495]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:33:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:31 smithi046 ceph-mon[43495]: osd.9 [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015] boot 2024-03-18T20:33:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:31 smithi046 ceph-mon[43495]: osdmap e61: 10 total, 10 up, 10 in 2024-03-18T20:33:31.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:31 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:33:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:32 smithi073 ceph-mon[47384]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2024-03-18T20:33:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:32 smithi073 ceph-mon[47384]: osdmap e62: 10 total, 10 up, 10 in 2024-03-18T20:33:32.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:32 smithi073 ceph-mon[47384]: from='client.? 172.21.15.73:0/481223831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9975d16-8806-494b-926e-2ba60d76ac1e"}]: dispatch 2024-03-18T20:33:32.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:32 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9975d16-8806-494b-926e-2ba60d76ac1e"}]: dispatch 2024-03-18T20:33:32.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:32 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9975d16-8806-494b-926e-2ba60d76ac1e"}]': finished 2024-03-18T20:33:32.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:32 smithi073 ceph-mon[47384]: osdmap e63: 11 total, 10 up, 11 in 2024-03-18T20:33:32.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:32 smithi155 ceph-mon[48344]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2024-03-18T20:33:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:32 smithi155 ceph-mon[48344]: osdmap e62: 10 total, 10 up, 10 in 2024-03-18T20:33:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:32 smithi155 ceph-mon[48344]: from='client.? 172.21.15.73:0/481223831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9975d16-8806-494b-926e-2ba60d76ac1e"}]: dispatch 2024-03-18T20:33:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:32 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9975d16-8806-494b-926e-2ba60d76ac1e"}]: dispatch 2024-03-18T20:33:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:32 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9975d16-8806-494b-926e-2ba60d76ac1e"}]': finished 2024-03-18T20:33:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:32 smithi155 ceph-mon[48344]: osdmap e63: 11 total, 10 up, 11 in 2024-03-18T20:33:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:32 smithi046 ceph-mon[43495]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2024-03-18T20:33:32.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:32 smithi046 ceph-mon[43495]: osdmap e62: 10 total, 10 up, 10 in 2024-03-18T20:33:32.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:32 smithi046 ceph-mon[43495]: from='client.? 172.21.15.73:0/481223831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9975d16-8806-494b-926e-2ba60d76ac1e"}]: dispatch 2024-03-18T20:33:32.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:32 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9975d16-8806-494b-926e-2ba60d76ac1e"}]: dispatch 2024-03-18T20:33:32.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:32 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9975d16-8806-494b-926e-2ba60d76ac1e"}]': finished 2024-03-18T20:33:32.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:32 smithi046 ceph-mon[43495]: osdmap e63: 11 total, 10 up, 11 in 2024-03-18T20:33:32.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:33.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:33 smithi073 ceph-mon[47384]: from='client.? 172.21.15.73:0/1801043194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:33:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:33 smithi155 ceph-mon[48344]: from='client.? 172.21.15.73:0/1801043194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:33:33.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:33 smithi046 ceph-mon[43495]: from='client.? 172.21.15.73:0/1801043194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:33:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:34 smithi073 ceph-mon[47384]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:34 smithi155 ceph-mon[48344]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:34.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:34 smithi046 ceph-mon[43495]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:35.925 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:35 smithi073 ceph-mon[47384]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:36.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:35 smithi155 ceph-mon[48344]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:36.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:35 smithi046 ceph-mon[43495]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:38.338 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:38 smithi073 ceph-mon[47384]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:38.339 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-18T20:33:38.339 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:38 smithi155 ceph-mon[48344]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-18T20:33:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:38 smithi046 ceph-mon[43495]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-18T20:33:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:39.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:39 smithi073 ceph-mon[47384]: Deploying daemon osd.10 on smithi073 2024-03-18T20:33:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:39 smithi155 ceph-mon[48344]: Deploying daemon osd.10 on smithi073 2024-03-18T20:33:39.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:39 smithi046 ceph-mon[43495]: Deploying daemon osd.10 on smithi073 2024-03-18T20:33:40.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:40 smithi073 ceph-mon[47384]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:40 smithi155 ceph-mon[48344]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:40.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:40 smithi046 ceph-mon[43495]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:42.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:42 smithi073 ceph-mon[47384]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:42.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:42 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:42.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:42 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:42.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:42 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:42.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:42 smithi155 ceph-mon[48344]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:42.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:42 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:42.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:42 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:42.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:42 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:42.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:42 smithi046 ceph-mon[43495]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:42.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:42 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:42.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:42 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:42.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:42 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:43.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.760+0000 7f48adffb700 1 -- 172.21.15.73:0/285599412 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f48b00611f0 con 0x7f48a00a11a0 2024-03-18T20:33:43.764 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 10 on host 'smithi073' 2024-03-18T20:33:43.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.762+0000 7f48b7126700 1 -- 172.21.15.73:0/285599412 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f48a00a11a0 msgr2=0x7f48a00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:43.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.762+0000 7f48b7126700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f48a00a11a0 0x7f48a00a3660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48b0198290 tx=0x7f48a4009190 comp rx=0 tx=0).stop 2024-03-18T20:33:43.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.762+0000 7f48b7126700 1 -- 172.21.15.73:0/285599412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48b0101fe0 msgr2=0x7f48b0197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:43.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.762+0000 7f48b7126700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48b0101fe0 0x7f48b0197340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f489c0036f0 tx=0x7f489c0047b0 comp rx=0 tx=0).stop 2024-03-18T20:33:43.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.762+0000 7f48b7126700 1 -- 172.21.15.73:0/285599412 shutdown_connections 2024-03-18T20:33:43.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.762+0000 7f48b7126700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f48a00a11a0 0x7f48a00a3660 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:43.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.762+0000 7f48b7126700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48b0100de0 0x7f48b0196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:43.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.762+0000 7f48b7126700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48b0102980 0x7f48b019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:43.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.762+0000 7f48b7126700 1 --2- 172.21.15.73:0/285599412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48b0101fe0 0x7f48b0197340 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:43.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.762+0000 7f48b7126700 1 -- 172.21.15.73:0/285599412 >> 172.21.15.73:0/285599412 conn(0x7f48b00fc380 msgr2=0x7f48b01053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:33:43.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.762+0000 7f48b7126700 1 -- 172.21.15.73:0/285599412 shutdown_connections 2024-03-18T20:33:43.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-18T20:33:43.762+0000 7f48b7126700 1 -- 172.21.15.73:0/285599412 wait complete. 2024-03-18T20:33:44.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:44.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:44.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:43 smithi155 ceph-mon[48344]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:44.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:44.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:43 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:44.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:44.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:43 smithi046 ceph-mon[43495]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:44.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:44.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:43 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:44.063 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:44.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:43 smithi073 ceph-mon[47384]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:44.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:33:44.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.064 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:43 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:44.146 DEBUG:teuthology.orchestra.run.smithi073:osd.10> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.10.service 2024-03-18T20:33:44.149 INFO:tasks.cephadm:Deploying osd.11 on smithi155 with /dev/vg_nvme/lv_1... 2024-03-18T20:33:44.149 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-18T20:33:44.179 INFO:journalctl@ceph.osd.10.smithi073.stdout:-- Logs begin at Mon 2024-03-18 20:12:50 UTC. -- 2024-03-18T20:33:44.380 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T20:33:45.678 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-18T20:33:45.678 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-18T20:33:45.678 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-03-18T20:33:45.679 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-03-18T20:33:45.679 INFO:teuthology.orchestra.run.smithi155.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114026 s, 920 MB/s 2024-03-18T20:33:45.679 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-03-18T20:33:45.720 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 20:33:45 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T20:33:45.448+0000 7f85acc4d640 -1 osd.10 0 log_to_monitors true 2024-03-18T20:33:46.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:45 smithi155 ceph-mon[48344]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:46.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:45 smithi155 ceph-mon[48344]: from='osd.10 [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T20:33:46.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:45 smithi155 ceph-mon[48344]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T20:33:46.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:45 smithi046 ceph-mon[43495]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:46.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:45 smithi046 ceph-mon[43495]: from='osd.10 [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T20:33:46.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:45 smithi046 ceph-mon[43495]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T20:33:46.055 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2024-03-18T20:33:46.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:45 smithi073 ceph-mon[47384]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:46.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:45 smithi073 ceph-mon[47384]: from='osd.10 [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T20:33:46.057 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:45 smithi073 ceph-mon[47384]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T20:33:46.276 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T20:33:46.998 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-18T20:33:46.998 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: osdmap e64: 11 total, 10 up, 11 in 2024-03-18T20:33:46.998 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:46.998 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='osd.10 [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:33:46.998 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:33:46.998 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: Detected new or changed devices on smithi073 2024-03-18T20:33:46.998 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:46.998 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:46.998 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:46.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:46.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:46.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:46.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi073 to 4278M 2024-03-18T20:33:46.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:46.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:46.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:46.999 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:46 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:47.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-18T20:33:47.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: osdmap e64: 11 total, 10 up, 11 in 2024-03-18T20:33:47.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:47.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='osd.10 [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:33:47.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:33:47.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: Detected new or changed devices on smithi073 2024-03-18T20:33:47.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:47.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:47.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:47.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:47.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:47.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:47.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi073 to 4278M 2024-03-18T20:33:47.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:47.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:47.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:47.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:46 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:47.170 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.166+0000 7f1bc20a9700 1 -- 172.21.15.155:0/115409717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc0ff0b0 msgr2=0x7f1bbc0ff4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:47.170 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.166+0000 7f1bc20a9700 1 --2- 172.21.15.155:0/115409717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc0ff0b0 0x7f1bbc0ff4d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009990 tx=0x7f1bac009ca0 comp rx=0 tx=0).stop 2024-03-18T20:33:47.170 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.166+0000 7f1bc20a9700 1 -- 172.21.15.155:0/115409717 shutdown_connections 2024-03-18T20:33:47.170 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.166+0000 7f1bc20a9700 1 --2- 172.21.15.155:0/115409717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc102a30 0x7f1bbc1092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:47.170 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.166+0000 7f1bc20a9700 1 --2- 172.21.15.155:0/115409717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc0ff0b0 0x7f1bbc0ff4d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:47.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.167+0000 7f1bc20a9700 1 --2- 172.21.15.155:0/115409717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bbc102090 0x7f1bbc1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:47.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.167+0000 7f1bc20a9700 1 -- 172.21.15.155:0/115409717 >> 172.21.15.155:0/115409717 conn(0x7f1bbc0faa10 msgr2=0x7f1bbc0fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:33:47.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.167+0000 7f1bc20a9700 1 -- 172.21.15.155:0/115409717 shutdown_connections 2024-03-18T20:33:47.171 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.167+0000 7f1bc20a9700 1 -- 172.21.15.155:0/115409717 wait complete. 2024-03-18T20:33:47.174 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.167+0000 7f1bc20a9700 1 Processor -- start 2024-03-18T20:33:47.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bc20a9700 1 -- start start 2024-03-18T20:33:47.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bc20a9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bbc0ff0b0 0x7f1bbc196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:33:47.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bc20a9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc102090 0x7f1bbc197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:33:47.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bc20a9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc102a30 0x7f1bbc19e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:33:47.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bc20a9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc19e770 con 0x7f1bbc102a30 2024-03-18T20:33:47.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bc20a9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc19e8b0 con 0x7f1bbc0ff0b0 2024-03-18T20:33:47.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bc20a9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc19ea40 con 0x7f1bbc102090 2024-03-18T20:33:47.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bbaffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc102090 0x7f1bbc197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:33:47.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bbaffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc102090 0x7f1bbc197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:42528/0 (socket says 172.21.15.155:42528) 2024-03-18T20:33:47.175 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bbaffd700 1 -- 172.21.15.155:0/3907879279 learned_addr learned my addr 172.21.15.155:0/3907879279 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-18T20:33:47.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bbbfff700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc102a30 0x7f1bbc19e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:33:47.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bbb7fe700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bbc0ff0b0 0x7f1bbc196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:33:47.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bbaffd700 1 -- 172.21.15.155:0/3907879279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bbc0ff0b0 msgr2=0x7f1bbc196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:47.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bbaffd700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bbc0ff0b0 0x7f1bbc196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:47.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bbaffd700 1 -- 172.21.15.155:0/3907879279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc102a30 msgr2=0x7f1bbc19e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:33:47.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bbaffd700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc102a30 0x7f1bbc19e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:33:47.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bbaffd700 1 -- 172.21.15.155:0/3907879279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac0095f0 con 0x7f1bbc102090 2024-03-18T20:33:47.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bbbfff700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc102a30 0x7f1bbc19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:33:47.176 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bbaffd700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc102090 0x7f1bbc197170 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000c9e0 tx=0x7f1bb000ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:33:47.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bb8ff9700 1 -- 172.21.15.155:0/3907879279 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0007ac0 con 0x7f1bbc102090 2024-03-18T20:33:47.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bb8ff9700 1 -- 172.21.15.155:0/3907879279 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bb0007c20 con 0x7f1bbc102090 2024-03-18T20:33:47.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.168+0000 7f1bc20a9700 1 -- 172.21.15.155:0/3907879279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bbc19ed20 con 0x7f1bbc102090 2024-03-18T20:33:47.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.169+0000 7f1bb8ff9700 1 -- 172.21.15.155:0/3907879279 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0018660 con 0x7f1bbc102090 2024-03-18T20:33:47.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.169+0000 7f1bc20a9700 1 -- 172.21.15.155:0/3907879279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bbc19f1c0 con 0x7f1bbc102090 2024-03-18T20:33:47.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.169+0000 7f1bc20a9700 1 -- 172.21.15.155:0/3907879279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bbc1082a0 con 0x7f1bbc102090 2024-03-18T20:33:47.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.170+0000 7f1bb8ff9700 1 -- 172.21.15.155:0/3907879279 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1bb000f450 con 0x7f1bbc102090 2024-03-18T20:33:47.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.171+0000 7f1bb8ff9700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1ba40a1120 0x7f1ba40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:33:47.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.171+0000 7f1bb8ff9700 1 -- 172.21.15.155:0/3907879279 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1bb00cc390 con 0x7f1bbc102090 2024-03-18T20:33:47.177 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.171+0000 7f1bbb7fe700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1ba40a1120 0x7f1ba40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:33:47.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.172+0000 7f1bbb7fe700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1ba40a1120 0x7f1ba40a35e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00b5c0 tx=0x7f1bac003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:33:47.178 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.174+0000 7f1bb8ff9700 1 -- 172.21.15.155:0/3907879279 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1bb0096320 con 0x7f1bbc102090 2024-03-18T20:33:47.220 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T20:33:46.774+0000 7f85a1ab0700 -1 osd.10 0 waiting for initial osdmap 2024-03-18T20:33:47.221 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T20:33:46.789+0000 7f859b0a0700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T20:33:47.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-18T20:33:47.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: osdmap e64: 11 total, 10 up, 11 in 2024-03-18T20:33:47.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:47.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='osd.10 [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:33:47.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-03-18T20:33:47.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: Detected new or changed devices on smithi073 2024-03-18T20:33:47.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:47.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:47.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:47.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:47.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:47.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:33:47.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi073 to 4278M 2024-03-18T20:33:47.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:47.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:47.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:33:47.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:46 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:33:47.377 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:33:47.374+0000 7f1bc20a9700 1 -- 172.21.15.155:0/3907879279 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bbc0611f0 con 0x7f1ba40a1120 2024-03-18T20:33:48.008 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:47 smithi155 ceph-mon[48344]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-03-18T20:33:48.008 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:47 smithi155 ceph-mon[48344]: osdmap e65: 11 total, 10 up, 11 in 2024-03-18T20:33:48.008 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:48.008 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:48.008 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:47 smithi155 ceph-mon[48344]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:48.008 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:47 smithi155 ceph-mon[48344]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:33:48.009 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:33:48.009 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:33:48.009 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:47 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:48.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:47 smithi046 ceph-mon[43495]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-03-18T20:33:48.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:47 smithi046 ceph-mon[43495]: osdmap e65: 11 total, 10 up, 11 in 2024-03-18T20:33:48.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:48.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:48.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:47 smithi046 ceph-mon[43495]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:48.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:47 smithi046 ceph-mon[43495]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:33:48.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:33:48.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:33:48.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:47 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:48.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:47 smithi073 ceph-mon[47384]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-03-18T20:33:48.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:47 smithi073 ceph-mon[47384]: osdmap e65: 11 total, 10 up, 11 in 2024-03-18T20:33:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:47 smithi073 ceph-mon[47384]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2024-03-18T20:33:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:47 smithi073 ceph-mon[47384]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:33:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T20:33:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T20:33:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:47 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:48.784 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:48 smithi155 ceph-mon[48344]: purged_snaps scrub starts 2024-03-18T20:33:48.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:48 smithi155 ceph-mon[48344]: purged_snaps scrub ok 2024-03-18T20:33:48.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:48 smithi155 ceph-mon[48344]: osd.10 [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025] boot 2024-03-18T20:33:48.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:48 smithi155 ceph-mon[48344]: osdmap e66: 11 total, 11 up, 11 in 2024-03-18T20:33:48.785 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:48 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:49.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:48 smithi046 ceph-mon[43495]: purged_snaps scrub starts 2024-03-18T20:33:49.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:48 smithi046 ceph-mon[43495]: purged_snaps scrub ok 2024-03-18T20:33:49.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:48 smithi046 ceph-mon[43495]: osd.10 [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025] boot 2024-03-18T20:33:49.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:48 smithi046 ceph-mon[43495]: osdmap e66: 11 total, 11 up, 11 in 2024-03-18T20:33:49.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:48 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:49.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:48 smithi073 ceph-mon[47384]: purged_snaps scrub starts 2024-03-18T20:33:49.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:48 smithi073 ceph-mon[47384]: purged_snaps scrub ok 2024-03-18T20:33:49.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:48 smithi073 ceph-mon[47384]: osd.10 [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025] boot 2024-03-18T20:33:49.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:48 smithi073 ceph-mon[47384]: osdmap e66: 11 total, 11 up, 11 in 2024-03-18T20:33:49.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:48 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:33:50.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:49 smithi155 ceph-mon[48344]: osdmap e67: 11 total, 11 up, 11 in 2024-03-18T20:33:50.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:49 smithi155 ceph-mon[48344]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2024-03-18T20:33:50.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:49 smithi155 ceph-mon[48344]: from='client.? 172.21.15.155:0/2329025764' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef1f614e-b094-4a41-a132-8c4db3d34b3b"}]: dispatch 2024-03-18T20:33:50.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:49 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef1f614e-b094-4a41-a132-8c4db3d34b3b"}]: dispatch 2024-03-18T20:33:50.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:49 smithi155 ceph-mon[48344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef1f614e-b094-4a41-a132-8c4db3d34b3b"}]': finished 2024-03-18T20:33:50.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:49 smithi155 ceph-mon[48344]: osdmap e68: 12 total, 11 up, 12 in 2024-03-18T20:33:50.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:49 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:33:50.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:49 smithi155 ceph-mon[48344]: osdmap e69: 12 total, 11 up, 12 in 2024-03-18T20:33:50.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:49 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:33:50.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:49 smithi046 ceph-mon[43495]: osdmap e67: 11 total, 11 up, 11 in 2024-03-18T20:33:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:49 smithi046 ceph-mon[43495]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2024-03-18T20:33:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:49 smithi046 ceph-mon[43495]: from='client.? 172.21.15.155:0/2329025764' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef1f614e-b094-4a41-a132-8c4db3d34b3b"}]: dispatch 2024-03-18T20:33:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:49 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef1f614e-b094-4a41-a132-8c4db3d34b3b"}]: dispatch 2024-03-18T20:33:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:49 smithi046 ceph-mon[43495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef1f614e-b094-4a41-a132-8c4db3d34b3b"}]': finished 2024-03-18T20:33:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:49 smithi046 ceph-mon[43495]: osdmap e68: 12 total, 11 up, 12 in 2024-03-18T20:33:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:49 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:33:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:49 smithi046 ceph-mon[43495]: osdmap e69: 12 total, 11 up, 12 in 2024-03-18T20:33:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:49 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:33:50.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:49 smithi073 ceph-mon[47384]: osdmap e67: 11 total, 11 up, 11 in 2024-03-18T20:33:50.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:49 smithi073 ceph-mon[47384]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2024-03-18T20:33:50.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:49 smithi073 ceph-mon[47384]: from='client.? 172.21.15.155:0/2329025764' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef1f614e-b094-4a41-a132-8c4db3d34b3b"}]: dispatch 2024-03-18T20:33:50.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:49 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef1f614e-b094-4a41-a132-8c4db3d34b3b"}]: dispatch 2024-03-18T20:33:50.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:49 smithi073 ceph-mon[47384]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef1f614e-b094-4a41-a132-8c4db3d34b3b"}]': finished 2024-03-18T20:33:50.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:49 smithi073 ceph-mon[47384]: osdmap e68: 12 total, 11 up, 12 in 2024-03-18T20:33:50.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:49 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:33:50.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:49 smithi073 ceph-mon[47384]: osdmap e69: 12 total, 11 up, 12 in 2024-03-18T20:33:50.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:49 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:33:51.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:50 smithi046 ceph-mon[43495]: from='client.? 172.21.15.155:0/925757295' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:33:51.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:50 smithi046 ceph-mon[43495]: osdmap e70: 12 total, 11 up, 12 in 2024-03-18T20:33:51.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:50 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:33:51.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:50 smithi073 ceph-mon[47384]: from='client.? 172.21.15.155:0/925757295' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:33:51.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:50 smithi073 ceph-mon[47384]: osdmap e70: 12 total, 11 up, 12 in 2024-03-18T20:33:51.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:50 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:33:51.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:50 smithi155 ceph-mon[48344]: from='client.? 172.21.15.155:0/925757295' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T20:33:51.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:50 smithi155 ceph-mon[48344]: osdmap e70: 12 total, 11 up, 12 in 2024-03-18T20:33:51.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:50 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:33:52.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:51 smithi046 ceph-mon[43495]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 178 KiB/s, 0 objects/s recovering 2024-03-18T20:33:52.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:51 smithi073 ceph-mon[47384]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 178 KiB/s, 0 objects/s recovering 2024-03-18T20:33:52.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:51 smithi155 ceph-mon[48344]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 178 KiB/s, 0 objects/s recovering 2024-03-18T20:33:54.107 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:54 smithi155 ceph-mon[48344]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-18T20:33:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:54 smithi073 ceph-mon[47384]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-18T20:33:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:54 smithi046 ceph-mon[43495]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-03-18T20:33:55.857 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:55 smithi155 ceph-mon[48344]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T20:33:55.857 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:55 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-18T20:33:55.857 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:55 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:55.857 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:55 smithi155 ceph-mon[48344]: Deploying daemon osd.11 on smithi155 2024-03-18T20:33:56.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:55 smithi046 ceph-mon[43495]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T20:33:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:55 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-18T20:33:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:55 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:55 smithi046 ceph-mon[43495]: Deploying daemon osd.11 on smithi155 2024-03-18T20:33:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:55 smithi073 ceph-mon[47384]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T20:33:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:55 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-18T20:33:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:55 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:33:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:55 smithi073 ceph-mon[47384]: Deploying daemon osd.11 on smithi155 2024-03-18T20:33:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:58 smithi073 ceph-mon[47384]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-18T20:33:58.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:58 smithi155 ceph-mon[48344]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-18T20:33:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:58 smithi046 ceph-mon[43495]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-18T20:34:00.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:59 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:00.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:59 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:00.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:59 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:00.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:33:59 smithi155 ceph-mon[48344]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-18T20:34:00.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:59 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:00.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:59 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:00.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:59 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:00.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:33:59 smithi046 ceph-mon[43495]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-18T20:34:00.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:59 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:00.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:59 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:00.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:59 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:00.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:33:59 smithi073 ceph-mon[47384]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-18T20:34:01.032 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 11 on host 'smithi155' 2024-03-18T20:34:01.032 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.028+0000 7f1bb8ff9700 1 -- 172.21.15.155:0/3907879279 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f1bbc0611f0 con 0x7f1ba40a1120 2024-03-18T20:34:01.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.030+0000 7f1bc20a9700 1 -- 172.21.15.155:0/3907879279 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1ba40a1120 msgr2=0x7f1ba40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:01.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.030+0000 7f1bc20a9700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1ba40a1120 0x7f1ba40a35e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00b5c0 tx=0x7f1bac003790 comp rx=0 tx=0).stop 2024-03-18T20:34:01.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.030+0000 7f1bc20a9700 1 -- 172.21.15.155:0/3907879279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc102090 msgr2=0x7f1bbc197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:01.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.030+0000 7f1bc20a9700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc102090 0x7f1bbc197170 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000c9e0 tx=0x7f1bb000ccf0 comp rx=0 tx=0).stop 2024-03-18T20:34:01.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.030+0000 7f1bc20a9700 1 -- 172.21.15.155:0/3907879279 shutdown_connections 2024-03-18T20:34:01.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.030+0000 7f1bc20a9700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1ba40a1120 0x7f1ba40a35e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:01.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.030+0000 7f1bc20a9700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bbc102a30 0x7f1bbc19e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:01.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.030+0000 7f1bc20a9700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bbc0ff0b0 0x7f1bbc196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:01.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.030+0000 7f1bc20a9700 1 --2- 172.21.15.155:0/3907879279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bbc102090 0x7f1bbc197170 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:01.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.030+0000 7f1bc20a9700 1 -- 172.21.15.155:0/3907879279 >> 172.21.15.155:0/3907879279 conn(0x7f1bbc0faa10 msgr2=0x7f1bbc1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:01.034 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.031+0000 7f1bc20a9700 1 -- 172.21.15.155:0/3907879279 shutdown_connections 2024-03-18T20:34:01.035 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:01.031+0000 7f1bc20a9700 1 -- 172.21.15.155:0/3907879279 wait complete. 2024-03-18T20:34:01.307 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:01 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.307 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:01 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.307 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:01 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:01.307 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:01 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:01.307 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:01 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.307 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:01 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.307 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:01 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.307 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:01 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:01.407 DEBUG:teuthology.orchestra.run.smithi155:osd.11> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.11.service 2024-03-18T20:34:01.409 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-18T20:34:01.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-18T20:34:01.441 INFO:journalctl@ceph.osd.11.smithi155.stdout:-- Logs begin at Mon 2024-03-18 20:13:03 UTC. -- 2024-03-18T20:34:01.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:01 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:01 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:01 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:01 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:01 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:01 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:01 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:01 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:01.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:01 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:01.668 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:02.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:02 smithi046 ceph-mon[43495]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2024-03-18T20:34:02.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:02 smithi155 ceph-mon[48344]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2024-03-18T20:34:02.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.658+0000 7f878eeca700 1 -- 172.21.15.46:0/335572376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788100df0 msgr2=0x7f8788101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:02.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.658+0000 7f878eeca700 1 --2- 172.21.15.46:0/335572376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788100df0 0x7f8788101210 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f877c0099c0 tx=0x7f877c009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:02.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.658+0000 7f878eeca700 1 -- 172.21.15.46:0/335572376 shutdown_connections 2024-03-18T20:34:02.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.658+0000 7f878eeca700 1 --2- 172.21.15.46:0/335572376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788100df0 0x7f8788101210 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:02.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.658+0000 7f878eeca700 1 --2- 172.21.15.46:0/335572376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788102990 0x7f8788109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:02.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.658+0000 7f878eeca700 1 --2- 172.21.15.46:0/335572376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8788101ff0 0x7f8788102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:02.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.658+0000 7f878eeca700 1 -- 172.21.15.46:0/335572376 >> 172.21.15.46:0/335572376 conn(0x7f87880fc390 msgr2=0x7f87880fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:02.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.658+0000 7f878eeca700 1 -- 172.21.15.46:0/335572376 shutdown_connections 2024-03-18T20:34:02.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.658+0000 7f878eeca700 1 -- 172.21.15.46:0/335572376 wait complete. 2024-03-18T20:34:02.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.658+0000 7f878eeca700 1 Processor -- start 2024-03-18T20:34:02.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.658+0000 7f878eeca700 1 -- start start 2024-03-18T20:34:02.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878eeca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788100df0 0x7f8788074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:02.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878eeca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788101ff0 0x7f87880731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:02.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878eeca700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8788102990 0x7f87880736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:02.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878eeca700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8788073e80 con 0x7f8788100df0 2024-03-18T20:34:02.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878eeca700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8788073ff0 con 0x7f8788101ff0 2024-03-18T20:34:02.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878eeca700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87881a0d70 con 0x7f8788102990 2024-03-18T20:34:02.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878cc66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788100df0 0x7f8788074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:02.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878cc66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788100df0 0x7f8788074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42468/0 (socket says 172.21.15.46:42468) 2024-03-18T20:34:02.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878cc66700 1 -- 172.21.15.46:0/770593463 learned_addr learned my addr 172.21.15.46:0/770593463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:02.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878d467700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8788102990 0x7f87880736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:02.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f8787fff700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788101ff0 0x7f87880731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:02.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878cc66700 1 -- 172.21.15.46:0/770593463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8788102990 msgr2=0x7f87880736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:02.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878cc66700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8788102990 0x7f87880736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:02.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878cc66700 1 -- 172.21.15.46:0/770593463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788101ff0 msgr2=0x7f87880731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:02.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878cc66700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788101ff0 0x7f87880731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:02.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878cc66700 1 -- 172.21.15.46:0/770593463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c0095f0 con 0x7f8788100df0 2024-03-18T20:34:02.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f878cc66700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788100df0 0x7f8788074b00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f877c009fd0 tx=0x7f877c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:02.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f8785ffb700 1 -- 172.21.15.46:0/770593463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c01d070 con 0x7f8788100df0 2024-03-18T20:34:02.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.659+0000 7f8785ffb700 1 -- 172.21.15.46:0/770593463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f877c0059a0 con 0x7f8788100df0 2024-03-18T20:34:02.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.660+0000 7f878eeca700 1 -- 172.21.15.46:0/770593463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87881a0fa0 con 0x7f8788100df0 2024-03-18T20:34:02.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.660+0000 7f8785ffb700 1 -- 172.21.15.46:0/770593463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c0173f0 con 0x7f8788100df0 2024-03-18T20:34:02.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.660+0000 7f878eeca700 1 -- 172.21.15.46:0/770593463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87881a1350 con 0x7f8788100df0 2024-03-18T20:34:02.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.661+0000 7f878eeca700 1 -- 172.21.15.46:0/770593463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87881082f0 con 0x7f8788100df0 2024-03-18T20:34:02.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.662+0000 7f8785ffb700 1 -- 172.21.15.46:0/770593463 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f877c022600 con 0x7f8788100df0 2024-03-18T20:34:02.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.663+0000 7f8785ffb700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f87780a1120 0x7f87780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:02.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.663+0000 7f8787fff700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f87780a1120 0x7f87780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:02.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.663+0000 7f8785ffb700 1 -- 172.21.15.46:0/770593463 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f877c0cdc50 con 0x7f8788100df0 2024-03-18T20:34:02.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.663+0000 7f8787fff700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f87780a1120 0x7f87780a35e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8774009c60 tx=0x7f8774009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:02.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.666+0000 7f8785ffb700 1 -- 172.21.15.46:0/770593463 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f877c0979b0 con 0x7f8788100df0 2024-03-18T20:34:02.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:02 smithi073 ceph-mon[47384]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2024-03-18T20:34:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.878+0000 7f878eeca700 1 -- 172.21.15.46:0/770593463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f878804ea80 con 0x7f8788100df0 2024-03-18T20:34:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.878+0000 7f8785ffb700 1 -- 172.21.15.46:0/770593463 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) v1 ==== 74+0+133 (secure 0 0 0) 0x7f877c09afd0 con 0x7f8788100df0 2024-03-18T20:34:02.882 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.880+0000 7f878eeca700 1 -- 172.21.15.46:0/770593463 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f87780a1120 msgr2=0x7f87780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.880+0000 7f878eeca700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f87780a1120 0x7f87780a35e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8774009c60 tx=0x7f8774009260 comp rx=0 tx=0).stop 2024-03-18T20:34:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.880+0000 7f878eeca700 1 -- 172.21.15.46:0/770593463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788100df0 msgr2=0x7f8788074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.880+0000 7f878eeca700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788100df0 0x7f8788074b00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f877c009fd0 tx=0x7f877c005f10 comp rx=0 tx=0).stop 2024-03-18T20:34:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.881+0000 7f878eeca700 1 -- 172.21.15.46:0/770593463 shutdown_connections 2024-03-18T20:34:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.881+0000 7f878eeca700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f87780a1120 0x7f87780a35e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.881+0000 7f878eeca700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788100df0 0x7f8788074b00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.881+0000 7f878eeca700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8788102990 0x7f87880736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.881+0000 7f878eeca700 1 --2- 172.21.15.46:0/770593463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788101ff0 0x7f87880731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.881+0000 7f878eeca700 1 -- 172.21.15.46:0/770593463 >> 172.21.15.46:0/770593463 conn(0x7f87880fc390 msgr2=0x7f8788105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.881+0000 7f878eeca700 1 -- 172.21.15.46:0/770593463 shutdown_connections 2024-03-18T20:34:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:02.881+0000 7f878eeca700 1 -- 172.21.15.46:0/770593463 wait complete. 2024-03-18T20:34:03.143 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 20:34:02 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T20:34:02.790+0000 7f6786353640 -1 osd.11 0 log_to_monitors true 2024-03-18T20:34:03.260 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1710794027,"num_in_osds":12,"osd_in_since":1710794029,"num_remapped_pgs":0} 2024-03-18T20:34:03.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:03 smithi155 ceph-mon[48344]: from='osd.11 [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T20:34:03.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:03 smithi155 ceph-mon[48344]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T20:34:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:03 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/770593463' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T20:34:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:03 smithi046 ceph-mon[43495]: from='osd.11 [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T20:34:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:03 smithi046 ceph-mon[43495]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T20:34:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:03 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/770593463' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T20:34:03.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:03 smithi073 ceph-mon[47384]: from='osd.11 [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T20:34:03.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:03 smithi073 ceph-mon[47384]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T20:34:03.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:03 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/770593463' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T20:34:04.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-18T20:34:04.516 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:04.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T20:34:04.248+0000 7f677b1b6700 -1 osd.11 0 waiting for initial osdmap 2024-03-18T20:34:04.541 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T20:34:04.266+0000 7f67757a8700 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T20:34:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2024-03-18T20:34:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-18T20:34:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: osdmap e71: 12 total, 11 up, 12 in 2024-03-18T20:34:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:34:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='osd.11 [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:34:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:34:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:04.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:34:04.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:34:04.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:34:04.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:34:04.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:04.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:04.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:04.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:04 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2024-03-18T20:34:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-18T20:34:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: osdmap e71: 12 total, 11 up, 12 in 2024-03-18T20:34:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:34:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='osd.11 [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:34:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:34:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:04.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:34:04.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:34:04.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:34:04.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:34:04.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:04.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:04.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:04.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:04 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:04.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2024-03-18T20:34:04.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-18T20:34:04.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: osdmap e71: 12 total, 11 up, 12 in 2024-03-18T20:34:04.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:34:04.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='osd.11 [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:34:04.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-18T20:34:04.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:04.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:04.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:34:04.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:34:04.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:34:04.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:34:04.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:04.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:04.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:04.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:04 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.491+0000 7f67b1374700 1 -- 172.21.15.46:0/3799189876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67ac1041a0 msgr2=0x7f67ac106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.491+0000 7f67b1374700 1 --2- 172.21.15.46:0/3799189876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67ac1041a0 0x7f67ac106590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009990 tx=0x7f67a0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:34:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.491+0000 7f67b1374700 1 -- 172.21.15.46:0/3799189876 shutdown_connections 2024-03-18T20:34:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.491+0000 7f67b1374700 1 --2- 172.21.15.46:0/3799189876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67ac106ad0 0x7f67ac108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.491+0000 7f67b1374700 1 --2- 172.21.15.46:0/3799189876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67ac101870 0x7f67ac103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.491+0000 7f67b1374700 1 --2- 172.21.15.46:0/3799189876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67ac1041a0 0x7f67ac106590 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.491+0000 7f67b1374700 1 -- 172.21.15.46:0/3799189876 >> 172.21.15.46:0/3799189876 conn(0x7f67ac0fb2a0 msgr2=0x7f67ac0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.491+0000 7f67b1374700 1 -- 172.21.15.46:0/3799189876 shutdown_connections 2024-03-18T20:34:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.491+0000 7f67b1374700 1 -- 172.21.15.46:0/3799189876 wait complete. 2024-03-18T20:34:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67b1374700 1 Processor -- start 2024-03-18T20:34:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67b1374700 1 -- start start 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67b1374700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67ac101870 0x7f67ac196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67b1374700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67ac1041a0 0x7f67ac197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67b1374700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67ac106ad0 0x7f67ac19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67b1374700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac19e820 con 0x7f67ac1041a0 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67b1374700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac19e960 con 0x7f67ac101870 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67b1374700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac19eb20 con 0x7f67ac106ad0 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67aa7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67ac1041a0 0x7f67ac197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67aa7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67ac1041a0 0x7f67ac197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56260/0 (socket says 172.21.15.46:56260) 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67aa7fc700 1 -- 172.21.15.46:0/98464443 learned_addr learned my addr 172.21.15.46:0/98464443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67aaffd700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67ac101870 0x7f67ac196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.492+0000 7f67ab7fe700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67ac106ad0 0x7f67ac19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.493+0000 7f67aa7fc700 1 -- 172.21.15.46:0/98464443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67ac106ad0 msgr2=0x7f67ac19e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.493+0000 7f67aa7fc700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67ac106ad0 0x7f67ac19e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.493+0000 7f67aa7fc700 1 -- 172.21.15.46:0/98464443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67ac101870 msgr2=0x7f67ac196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.493+0000 7f67aa7fc700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67ac101870 0x7f67ac196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.493+0000 7f67aa7fc700 1 -- 172.21.15.46:0/98464443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a00095f0 con 0x7f67ac1041a0 2024-03-18T20:34:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.493+0000 7f67aa7fc700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67ac1041a0 0x7f67ac197230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f67a0000c00 tx=0x7f67a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.493+0000 7f6793fff700 1 -- 172.21.15.46:0/98464443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a001d070 con 0x7f67ac1041a0 2024-03-18T20:34:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.493+0000 7f6793fff700 1 -- 172.21.15.46:0/98464443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67a00054f0 con 0x7f67ac1041a0 2024-03-18T20:34:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.493+0000 7f67b1374700 1 -- 172.21.15.46:0/98464443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ac19eda0 con 0x7f67ac1041a0 2024-03-18T20:34:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.493+0000 7f6793fff700 1 -- 172.21.15.46:0/98464443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0004bd0 con 0x7f67ac1041a0 2024-03-18T20:34:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.493+0000 7f67b1374700 1 -- 172.21.15.46:0/98464443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ac19f150 con 0x7f67ac1041a0 2024-03-18T20:34:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.494+0000 7f67b1374700 1 -- 172.21.15.46:0/98464443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ac0fce20 con 0x7f67ac1041a0 2024-03-18T20:34:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.495+0000 7f6793fff700 1 -- 172.21.15.46:0/98464443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f67a000fb90 con 0x7f67ac1041a0 2024-03-18T20:34:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.496+0000 7f6793fff700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f67940a1120 0x7f67940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.496+0000 7f67aaffd700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f67940a1120 0x7f67940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.496+0000 7f6793fff700 1 -- 172.21.15.46:0/98464443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f67a00cd660 con 0x7f67ac1041a0 2024-03-18T20:34:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.496+0000 7f67aaffd700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f67940a1120 0x7f67940a35e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f67ac102ff0 tx=0x7f679c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.501+0000 7f6793fff700 1 -- 172.21.15.46:0/98464443 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f67a0097280 con 0x7f67ac1041a0 2024-03-18T20:34:05.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:05 smithi155 ceph-mon[48344]: Detected new or changed devices on smithi155 2024-03-18T20:34:05.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:05 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi155 to 4278M 2024-03-18T20:34:05.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:05 smithi155 ceph-mon[48344]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-18T20:34:05.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:05 smithi155 ceph-mon[48344]: osdmap e72: 12 total, 11 up, 12 in 2024-03-18T20:34:05.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:05 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:34:05.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:05 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:34:05.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:05 smithi046 ceph-mon[43495]: Detected new or changed devices on smithi155 2024-03-18T20:34:05.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:05 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi155 to 4278M 2024-03-18T20:34:05.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:05 smithi046 ceph-mon[43495]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-18T20:34:05.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:05 smithi046 ceph-mon[43495]: osdmap e72: 12 total, 11 up, 12 in 2024-03-18T20:34:05.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:05 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:34:05.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:05 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:34:05.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.711+0000 7f67b1374700 1 -- 172.21.15.46:0/98464443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f67ac04ea80 con 0x7f67ac1041a0 2024-03-18T20:34:05.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.712+0000 7f6793fff700 1 -- 172.21.15.46:0/98464443 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) v1 ==== 74+0+133 (secure 0 0 0) 0x7f67a009a8a0 con 0x7f67ac1041a0 2024-03-18T20:34:05.716 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.714+0000 7f67b1374700 1 -- 172.21.15.46:0/98464443 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f67940a1120 msgr2=0x7f67940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.714+0000 7f67b1374700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f67940a1120 0x7f67940a35e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f67ac102ff0 tx=0x7f679c0073d0 comp rx=0 tx=0).stop 2024-03-18T20:34:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.714+0000 7f67b1374700 1 -- 172.21.15.46:0/98464443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67ac1041a0 msgr2=0x7f67ac197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.714+0000 7f67b1374700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67ac1041a0 0x7f67ac197230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f67a0000c00 tx=0x7f67a0004730 comp rx=0 tx=0).stop 2024-03-18T20:34:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.714+0000 7f67b1374700 1 -- 172.21.15.46:0/98464443 shutdown_connections 2024-03-18T20:34:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.714+0000 7f67b1374700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f67940a1120 0x7f67940a35e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.714+0000 7f67b1374700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67ac106ad0 0x7f67ac19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.714+0000 7f67b1374700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67ac101870 0x7f67ac196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.714+0000 7f67b1374700 1 --2- 172.21.15.46:0/98464443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67ac1041a0 0x7f67ac197230 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.714+0000 7f67b1374700 1 -- 172.21.15.46:0/98464443 >> 172.21.15.46:0/98464443 conn(0x7f67ac0fb2a0 msgr2=0x7f67ac103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.714+0000 7f67b1374700 1 -- 172.21.15.46:0/98464443 shutdown_connections 2024-03-18T20:34:05.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:05.714+0000 7f67b1374700 1 -- 172.21.15.46:0/98464443 wait complete. 2024-03-18T20:34:05.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:05 smithi073 ceph-mon[47384]: Detected new or changed devices on smithi155 2024-03-18T20:34:05.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:05 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi155 to 4278M 2024-03-18T20:34:05.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:05 smithi073 ceph-mon[47384]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-18T20:34:05.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:05 smithi073 ceph-mon[47384]: osdmap e72: 12 total, 11 up, 12 in 2024-03-18T20:34:05.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:05 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:34:05.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:05 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:34:06.090 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1710794045,"num_in_osds":12,"osd_in_since":1710794029,"num_remapped_pgs":0} 2024-03-18T20:34:06.091 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T20:34:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:06.461 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:06 smithi046 ceph-mon[43495]: purged_snaps scrub starts 2024-03-18T20:34:06.461 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:06 smithi046 ceph-mon[43495]: purged_snaps scrub ok 2024-03-18T20:34:06.461 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:06 smithi046 ceph-mon[43495]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2024-03-18T20:34:06.461 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:06 smithi046 ceph-mon[43495]: osd.11 [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764] boot 2024-03-18T20:34:06.461 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:06 smithi046 ceph-mon[43495]: osdmap e73: 12 total, 12 up, 12 in 2024-03-18T20:34:06.461 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:06 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:34:06.461 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:06 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/98464443' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T20:34:06.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:06 smithi155 ceph-mon[48344]: purged_snaps scrub starts 2024-03-18T20:34:06.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:06 smithi155 ceph-mon[48344]: purged_snaps scrub ok 2024-03-18T20:34:06.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:06 smithi155 ceph-mon[48344]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2024-03-18T20:34:06.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:06 smithi155 ceph-mon[48344]: osd.11 [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764] boot 2024-03-18T20:34:06.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:06 smithi155 ceph-mon[48344]: osdmap e73: 12 total, 12 up, 12 in 2024-03-18T20:34:06.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:06 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:34:06.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:06 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/98464443' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T20:34:06.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:06 smithi073 ceph-mon[47384]: purged_snaps scrub starts 2024-03-18T20:34:06.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:06 smithi073 ceph-mon[47384]: purged_snaps scrub ok 2024-03-18T20:34:06.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:06 smithi073 ceph-mon[47384]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2024-03-18T20:34:06.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:06 smithi073 ceph-mon[47384]: osd.11 [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764] boot 2024-03-18T20:34:06.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:06 smithi073 ceph-mon[47384]: osdmap e73: 12 total, 12 up, 12 in 2024-03-18T20:34:06.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:06 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:34:06.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:06 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/98464443' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T20:34:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.340+0000 7f1117109700 1 -- 172.21.15.46:0/1903257485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1110100660 msgr2=0x7f111010cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.340+0000 7f1117109700 1 --2- 172.21.15.46:0/1903257485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1110100660 0x7f111010cb00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f110800b0a0 tx=0x7f110800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.341+0000 7f1117109700 1 -- 172.21.15.46:0/1903257485 shutdown_connections 2024-03-18T20:34:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.341+0000 7f1117109700 1 --2- 172.21.15.46:0/1903257485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1110100660 0x7f111010cb00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.341+0000 7f1117109700 1 --2- 172.21.15.46:0/1903257485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1110105d50 0x7f1110106130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.341+0000 7f1117109700 1 --2- 172.21.15.46:0/1903257485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11100ffca0 0x7f1110100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.341+0000 7f1117109700 1 -- 172.21.15.46:0/1903257485 >> 172.21.15.46:0/1903257485 conn(0x7f1110076c80 msgr2=0x7f1110077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.341+0000 7f1117109700 1 -- 172.21.15.46:0/1903257485 shutdown_connections 2024-03-18T20:34:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.341+0000 7f1117109700 1 -- 172.21.15.46:0/1903257485 wait complete. 2024-03-18T20:34:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.341+0000 7f1117109700 1 Processor -- start 2024-03-18T20:34:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.341+0000 7f1117109700 1 -- start start 2024-03-18T20:34:07.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f1117109700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11100ffca0 0x7f1110101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:07.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f1117109700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1110100660 0x7f1110101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:07.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f1117109700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1110105d50 0x7f11101023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:07.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f1117109700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11101a5530 con 0x7f1110105d50 2024-03-18T20:34:07.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f1117109700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11101a56a0 con 0x7f11100ffca0 2024-03-18T20:34:07.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f1117109700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11101a5860 con 0x7f1110100660 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f11156a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1110105d50 0x7f11101023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f11156a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1110105d50 0x7f11101023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56278/0 (socket says 172.21.15.46:56278) 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f11156a6700 1 -- 172.21.15.46:0/3569811409 learned_addr learned my addr 172.21.15.46:0/3569811409 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f110ffff700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1110100660 0x7f1110101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f1114ea5700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11100ffca0 0x7f1110101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f11156a6700 1 -- 172.21.15.46:0/3569811409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1110100660 msgr2=0x7f1110101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f11156a6700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1110100660 0x7f1110101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f11156a6700 1 -- 172.21.15.46:0/3569811409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11100ffca0 msgr2=0x7f1110101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f11156a6700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11100ffca0 0x7f1110101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f11156a6700 1 -- 172.21.15.46:0/3569811409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1108009d00 con 0x7f1110105d50 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f11156a6700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1110105d50 0x7f11101023d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f110800b070 tx=0x7f1108007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.342+0000 7f110dffb700 1 -- 172.21.15.46:0/3569811409 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110800e040 con 0x7f1110105d50 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.343+0000 7f1117109700 1 -- 172.21.15.46:0/3569811409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11101a5ae0 con 0x7f1110105d50 2024-03-18T20:34:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.343+0000 7f110dffb700 1 -- 172.21.15.46:0/3569811409 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1108009440 con 0x7f1110105d50 2024-03-18T20:34:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.343+0000 7f110dffb700 1 -- 172.21.15.46:0/3569811409 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110801b450 con 0x7f1110105d50 2024-03-18T20:34:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.343+0000 7f1117109700 1 -- 172.21.15.46:0/3569811409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11101a5fe0 con 0x7f1110105d50 2024-03-18T20:34:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.344+0000 7f1117109700 1 -- 172.21.15.46:0/3569811409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1110101390 con 0x7f1110105d50 2024-03-18T20:34:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.345+0000 7f110dffb700 1 -- 172.21.15.46:0/3569811409 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f110801d480 con 0x7f1110105d50 2024-03-18T20:34:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.346+0000 7f110dffb700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f10fc0a1120 0x7f10fc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.346+0000 7f1114ea5700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f10fc0a1120 0x7f10fc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.346+0000 7f110dffb700 1 -- 172.21.15.46:0/3569811409 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f11080cd8f0 con 0x7f1110105d50 2024-03-18T20:34:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.346+0000 7f1114ea5700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f10fc0a1120 0x7f10fc0a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f110000ac50 tx=0x7f110000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:07.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.350+0000 7f110dffb700 1 -- 172.21.15.46:0/3569811409 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f11080975b0 con 0x7f1110105d50 2024-03-18T20:34:07.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.563+0000 7f1117109700 1 -- 172.21.15.46:0/3569811409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f111004ea80 con 0x7f1110105d50 2024-03-18T20:34:07.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.564+0000 7f110dffb700 1 -- 172.21.15.46:0/3569811409 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f110809abd0 con 0x7f1110105d50 2024-03-18T20:34:07.568 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:07.568 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":74,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:34:06.766970+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:34:07.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.567+0000 7f1117109700 1 -- 172.21.15.46:0/3569811409 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f10fc0a1120 msgr2=0x7f10fc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:07.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.567+0000 7f1117109700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f10fc0a1120 0x7f10fc0a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f110000ac50 tx=0x7f110000a380 comp rx=0 tx=0).stop 2024-03-18T20:34:07.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.567+0000 7f1117109700 1 -- 172.21.15.46:0/3569811409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1110105d50 msgr2=0x7f11101023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:07.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.567+0000 7f1117109700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1110105d50 0x7f11101023d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f110800b070 tx=0x7f1108007870 comp rx=0 tx=0).stop 2024-03-18T20:34:07.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.567+0000 7f1117109700 1 -- 172.21.15.46:0/3569811409 shutdown_connections 2024-03-18T20:34:07.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.567+0000 7f1117109700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f10fc0a1120 0x7f10fc0a35e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:07.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.567+0000 7f1117109700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1110105d50 0x7f11101023d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:07.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.567+0000 7f1117109700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11100ffca0 0x7f1110101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:07.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.567+0000 7f1117109700 1 --2- 172.21.15.46:0/3569811409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1110100660 0x7f1110101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:07.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.567+0000 7f1117109700 1 -- 172.21.15.46:0/3569811409 >> 172.21.15.46:0/3569811409 conn(0x7f1110076c80 msgr2=0x7f11100feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:07.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.567+0000 7f1117109700 1 -- 172.21.15.46:0/3569811409 shutdown_connections 2024-03-18T20:34:07.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:07.567+0000 7f1117109700 1 -- 172.21.15.46:0/3569811409 wait complete. 2024-03-18T20:34:07.882 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:07 smithi046 ceph-mon[43495]: osdmap e74: 12 total, 12 up, 12 in 2024-03-18T20:34:07.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:07 smithi046 ceph-mon[43495]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:07.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:07 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3569811409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:07.949 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T20:31:27.488318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T20:34:07.950 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-18T20:34:08.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:07 smithi155 ceph-mon[48344]: osdmap e74: 12 total, 12 up, 12 in 2024-03-18T20:34:08.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:07 smithi155 ceph-mon[48344]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:08.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:07 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3569811409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:08.206 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:08.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:07 smithi073 ceph-mon[47384]: osdmap e74: 12 total, 12 up, 12 in 2024-03-18T20:34:08.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:07 smithi073 ceph-mon[47384]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:08.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:07 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3569811409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.174+0000 7f7ff549d700 1 -- 172.21.15.46:0/2330001070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff0100e60 msgr2=0x7f7ff01012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.174+0000 7f7ff549d700 1 --2- 172.21.15.46:0/2330001070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff0100e60 0x7f7ff01012e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80099c0 tx=0x7f7fd8009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.175+0000 7f7ff549d700 1 -- 172.21.15.46:0/2330001070 shutdown_connections 2024-03-18T20:34:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.175+0000 7f7ff549d700 1 --2- 172.21.15.46:0/2330001070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff0101820 0x7f7ff010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.175+0000 7f7ff549d700 1 --2- 172.21.15.46:0/2330001070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ff0106e90 0x7f7ff0107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.175+0000 7f7ff549d700 1 --2- 172.21.15.46:0/2330001070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff0100e60 0x7f7ff01012e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.175+0000 7f7ff549d700 1 -- 172.21.15.46:0/2330001070 >> 172.21.15.46:0/2330001070 conn(0x7f7ff0077ed0 msgr2=0x7f7ff00782e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.175+0000 7f7ff549d700 1 -- 172.21.15.46:0/2330001070 shutdown_connections 2024-03-18T20:34:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.175+0000 7f7ff549d700 1 -- 172.21.15.46:0/2330001070 wait complete. 2024-03-18T20:34:09.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.175+0000 7f7ff549d700 1 Processor -- start 2024-03-18T20:34:09.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7ff549d700 1 -- start start 2024-03-18T20:34:09.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7ff549d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff0100e60 0x7f7ff0197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:09.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7ff549d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ff0101820 0x7f7ff01977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:09.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7ff549d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff0106e90 0x7f7ff019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:09.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7ff549d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff019c160 con 0x7f7ff0106e90 2024-03-18T20:34:09.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7ff549d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff019c2d0 con 0x7f7ff0101820 2024-03-18T20:34:09.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7ff549d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff019c490 con 0x7f7ff0100e60 2024-03-18T20:34:09.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7fef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff0106e90 0x7f7ff019bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:09.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7fef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff0106e90 0x7f7ff019bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56298/0 (socket says 172.21.15.46:56298) 2024-03-18T20:34:09.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7fef7fe700 1 -- 172.21.15.46:0/359230194 learned_addr learned my addr 172.21.15.46:0/359230194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7fee7fc700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ff0101820 0x7f7ff01977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7feeffd700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff0100e60 0x7f7ff0197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7fef7fe700 1 -- 172.21.15.46:0/359230194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff0100e60 msgr2=0x7f7ff0197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7fef7fe700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff0100e60 0x7f7ff0197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7fef7fe700 1 -- 172.21.15.46:0/359230194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ff0101820 msgr2=0x7f7ff01977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7fef7fe700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ff0101820 0x7f7ff01977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:09.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7fef7fe700 1 -- 172.21.15.46:0/359230194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd80095f0 con 0x7f7ff0106e90 2024-03-18T20:34:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.176+0000 7f7fef7fe700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff0106e90 0x7f7ff019bb50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800ada0 tx=0x7f7fe800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.177+0000 7f7fe7fff700 1 -- 172.21.15.46:0/359230194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe800f070 con 0x7f7ff0106e90 2024-03-18T20:34:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.177+0000 7f7fe7fff700 1 -- 172.21.15.46:0/359230194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fe8007b30 con 0x7f7ff0106e90 2024-03-18T20:34:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.177+0000 7f7ff549d700 1 -- 172.21.15.46:0/359230194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff019c770 con 0x7f7ff0106e90 2024-03-18T20:34:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.177+0000 7f7fe7fff700 1 -- 172.21.15.46:0/359230194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80053e0 con 0x7f7ff0106e90 2024-03-18T20:34:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.177+0000 7f7ff549d700 1 -- 172.21.15.46:0/359230194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff01a3fd0 con 0x7f7ff0106e90 2024-03-18T20:34:09.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.178+0000 7f7ff549d700 1 -- 172.21.15.46:0/359230194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff0109300 con 0x7f7ff0106e90 2024-03-18T20:34:09.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.179+0000 7f7fe7fff700 1 -- 172.21.15.46:0/359230194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7fe800c980 con 0x7f7ff0106e90 2024-03-18T20:34:09.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.180+0000 7f7fe7fff700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7fdc0a1120 0x7f7fdc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:09.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.180+0000 7f7feeffd700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7fdc0a1120 0x7f7fdc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.180+0000 7f7fe7fff700 1 -- 172.21.15.46:0/359230194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7fe80cbb00 con 0x7f7ff0106e90 2024-03-18T20:34:09.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.180+0000 7f7feeffd700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7fdc0a1120 0x7f7fdc0a35e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0006fd0 tx=0x7f7fe0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:09.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.184+0000 7f7fe7fff700 1 -- 172.21.15.46:0/359230194 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7fe80cde60 con 0x7f7ff0106e90 2024-03-18T20:34:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.394+0000 7f7ff549d700 1 -- 172.21.15.46:0/359230194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ff004ea80 con 0x7f7ff0106e90 2024-03-18T20:34:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.394+0000 7f7fe7fff700 1 -- 172.21.15.46:0/359230194 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fe8095720 con 0x7f7ff0106e90 2024-03-18T20:34:09.398 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-18T20:34:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.396+0000 7f7ff549d700 1 -- 172.21.15.46:0/359230194 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7fdc0a1120 msgr2=0x7f7fdc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.396+0000 7f7ff549d700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7fdc0a1120 0x7f7fdc0a35e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0006fd0 tx=0x7f7fe0008040 comp rx=0 tx=0).stop 2024-03-18T20:34:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.396+0000 7f7ff549d700 1 -- 172.21.15.46:0/359230194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff0106e90 msgr2=0x7f7ff019bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.396+0000 7f7ff549d700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff0106e90 0x7f7ff019bb50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800ada0 tx=0x7f7fe800c330 comp rx=0 tx=0).stop 2024-03-18T20:34:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.397+0000 7f7ff549d700 1 -- 172.21.15.46:0/359230194 shutdown_connections 2024-03-18T20:34:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.397+0000 7f7ff549d700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7fdc0a1120 0x7f7fdc0a35e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:09.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.397+0000 7f7ff549d700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff0106e90 0x7f7ff019bb50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:09.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.397+0000 7f7ff549d700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ff0100e60 0x7f7ff0197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:09.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.397+0000 7f7ff549d700 1 --2- 172.21.15.46:0/359230194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ff0101820 0x7f7ff01977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:09.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.397+0000 7f7ff549d700 1 -- 172.21.15.46:0/359230194 >> 172.21.15.46:0/359230194 conn(0x7f7ff0077ed0 msgr2=0x7f7ff00ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:09.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.397+0000 7f7ff549d700 1 -- 172.21.15.46:0/359230194 shutdown_connections 2024-03-18T20:34:09.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:09.397+0000 7f7ff549d700 1 -- 172.21.15.46:0/359230194 wait complete. 2024-03-18T20:34:09.775 INFO:tasks.cephadm:Adding mds.a on smithi046 2024-03-18T20:34:09.775 INFO:tasks.cephadm:Adding mds.d on smithi046 2024-03-18T20:34:09.775 INFO:tasks.cephadm:Adding mds.g on smithi046 2024-03-18T20:34:09.775 INFO:tasks.cephadm:Adding mds.j on smithi046 2024-03-18T20:34:09.775 INFO:tasks.cephadm:Adding mds.b on smithi073 2024-03-18T20:34:09.775 INFO:tasks.cephadm:Adding mds.e on smithi073 2024-03-18T20:34:09.775 INFO:tasks.cephadm:Adding mds.h on smithi073 2024-03-18T20:34:09.775 INFO:tasks.cephadm:Adding mds.k on smithi073 2024-03-18T20:34:09.775 INFO:tasks.cephadm:Adding mds.c on smithi155 2024-03-18T20:34:09.775 INFO:tasks.cephadm:Adding mds.f on smithi155 2024-03-18T20:34:09.776 INFO:tasks.cephadm:Adding mds.i on smithi155 2024-03-18T20:34:09.776 INFO:tasks.cephadm:Adding mds.l on smithi155 2024-03-18T20:34:09.776 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph orch apply mds all '12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l' 2024-03-18T20:34:10.013 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T20:34:10.388 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:10 smithi155 ceph-mon[48344]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:10.388 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:10 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/359230194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:34:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:10 smithi073 ceph-mon[47384]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:10 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/359230194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:34:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:10 smithi046 ceph-mon[43495]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:10 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/359230194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:34:10.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.897+0000 7f801fbb3700 1 -- 172.21.15.155:0/3790274417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80180ffcf0 msgr2=0x7f8018100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:10.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.897+0000 7f801fbb3700 1 --2- 172.21.15.155:0/3790274417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80180ffcf0 0x7f8018100110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8014009990 tx=0x7f8014009ca0 comp rx=0 tx=0).stop 2024-03-18T20:34:10.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.897+0000 7f801fbb3700 1 -- 172.21.15.155:0/3790274417 shutdown_connections 2024-03-18T20:34:10.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.897+0000 7f801fbb3700 1 --2- 172.21.15.155:0/3790274417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8018101890 0x7f8018108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:10.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.897+0000 7f801fbb3700 1 --2- 172.21.15.155:0/3790274417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80180ffcf0 0x7f8018100110 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:10.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.897+0000 7f801fbb3700 1 --2- 172.21.15.155:0/3790274417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8018100ef0 0x7f8018101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:10.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.897+0000 7f801fbb3700 1 -- 172.21.15.155:0/3790274417 >> 172.21.15.155:0/3790274417 conn(0x7f80180fb2b0 msgr2=0x7f80180fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:10.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.897+0000 7f801fbb3700 1 -- 172.21.15.155:0/3790274417 shutdown_connections 2024-03-18T20:34:10.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.897+0000 7f801fbb3700 1 -- 172.21.15.155:0/3790274417 wait complete. 2024-03-18T20:34:10.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801fbb3700 1 Processor -- start 2024-03-18T20:34:10.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801fbb3700 1 -- start start 2024-03-18T20:34:10.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801fbb3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80180ffcf0 0x7f8018195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:10.903 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801fbb3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8018100ef0 0x7f80181961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801fbb3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8018101890 0x7f801819d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801fbb3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801819d760 con 0x7f8018100ef0 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801fbb3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801819d8a0 con 0x7f80180ffcf0 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801fbb3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801819da60 con 0x7f8018101890 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801e150700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8018101890 0x7f801819d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801e150700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8018101890 0x7f801819d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:50290/0 (socket says 172.21.15.155:50290) 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801e150700 1 -- 172.21.15.155:0/1569412735 learned_addr learned my addr 172.21.15.155:0/1569412735 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801d14e700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8018100ef0 0x7f80181961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801d94f700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80180ffcf0 0x7f8018195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801e150700 1 -- 172.21.15.155:0/1569412735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80180ffcf0 msgr2=0x7f8018195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801e150700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80180ffcf0 0x7f8018195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801e150700 1 -- 172.21.15.155:0/1569412735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8018100ef0 msgr2=0x7f80181961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:10.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801e150700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8018100ef0 0x7f80181961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:10.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.898+0000 7f801e150700 1 -- 172.21.15.155:0/1569412735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80140095f0 con 0x7f8018101890 2024-03-18T20:34:10.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.899+0000 7f801e150700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8018101890 0x7f801819d190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f801000ada0 tx=0x7f801000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:10.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.899+0000 7f800effd700 1 -- 172.21.15.155:0/1569412735 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801000f070 con 0x7f8018101890 2024-03-18T20:34:10.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.899+0000 7f801fbb3700 1 -- 172.21.15.155:0/1569412735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80180687b0 con 0x7f8018101890 2024-03-18T20:34:10.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.899+0000 7f800effd700 1 -- 172.21.15.155:0/1569412735 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8010007b30 con 0x7f8018101890 2024-03-18T20:34:10.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.899+0000 7f800effd700 1 -- 172.21.15.155:0/1569412735 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010005280 con 0x7f8018101890 2024-03-18T20:34:10.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.899+0000 7f801fbb3700 1 -- 172.21.15.155:0/1569412735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8018068c60 con 0x7f8018101890 2024-03-18T20:34:10.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.900+0000 7f801fbb3700 1 -- 172.21.15.155:0/1569412735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801804ea80 con 0x7f8018101890 2024-03-18T20:34:10.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.901+0000 7f800effd700 1 -- 172.21.15.155:0/1569412735 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f801000a090 con 0x7f8018101890 2024-03-18T20:34:10.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.901+0000 7f800effd700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f80040a1120 0x7f80040a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:10.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.902+0000 7f800effd700 1 -- 172.21.15.155:0/1569412735 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f80100cbed0 con 0x7f8018101890 2024-03-18T20:34:10.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.902+0000 7f801d94f700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f80040a1120 0x7f80040a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:10.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.902+0000 7f801d94f700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f80040a1120 0x7f80040a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f80181970f0 tx=0x7f8014005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:10.908 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:10.905+0000 7f800effd700 1 -- 172.21.15.155:0/1569412735 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8010095b90 con 0x7f8018101890 2024-03-18T20:34:11.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.109+0000 7f801fbb3700 1 -- 172.21.15.155:0/1569412735 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f8018068e80 con 0x7f80040a1120 2024-03-18T20:34:11.120 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.116+0000 7f800effd700 1 -- 172.21.15.155:0/1569412735 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f8018068e80 con 0x7f80040a1120 2024-03-18T20:34:11.120 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mds.all update... 2024-03-18T20:34:11.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.119+0000 7f801fbb3700 1 -- 172.21.15.155:0/1569412735 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f80040a1120 msgr2=0x7f80040a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:11.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.119+0000 7f801fbb3700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f80040a1120 0x7f80040a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f80181970f0 tx=0x7f8014005fd0 comp rx=0 tx=0).stop 2024-03-18T20:34:11.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.119+0000 7f801fbb3700 1 -- 172.21.15.155:0/1569412735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8018101890 msgr2=0x7f801819d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:11.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.119+0000 7f801fbb3700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8018101890 0x7f801819d190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f801000ada0 tx=0x7f801000c330 comp rx=0 tx=0).stop 2024-03-18T20:34:11.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.119+0000 7f801fbb3700 1 -- 172.21.15.155:0/1569412735 shutdown_connections 2024-03-18T20:34:11.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.119+0000 7f801fbb3700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f80040a1120 0x7f80040a35e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:11.122 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.119+0000 7f801fbb3700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8018101890 0x7f801819d190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:11.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.119+0000 7f801fbb3700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80180ffcf0 0x7f8018195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:11.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.119+0000 7f801fbb3700 1 --2- 172.21.15.155:0/1569412735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8018100ef0 0x7f80181961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:11.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.119+0000 7f801fbb3700 1 -- 172.21.15.155:0/1569412735 >> 172.21.15.155:0/1569412735 conn(0x7f80180fb2b0 msgr2=0x7f8018104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:11.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.119+0000 7f801fbb3700 1 -- 172.21.15.155:0/1569412735 shutdown_connections 2024-03-18T20:34:11.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-18T20:34:11.119+0000 7f801fbb3700 1 -- 172.21.15.155:0/1569412735 wait complete. 2024-03-18T20:34:11.484 DEBUG:teuthology.orchestra.run.smithi046:mds.a> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.a.service 2024-03-18T20:34:11.487 DEBUG:teuthology.orchestra.run.smithi046:mds.d> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.d.service 2024-03-18T20:34:11.521 INFO:journalctl@ceph.mds.a.smithi046.stdout:-- Logs begin at Mon 2024-03-18 20:12:57 UTC. -- 2024-03-18T20:34:11.522 DEBUG:teuthology.orchestra.run.smithi046:mds.g> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.g.service 2024-03-18T20:34:11.553 INFO:journalctl@ceph.mds.d.smithi046.stdout:-- Logs begin at Mon 2024-03-18 20:12:57 UTC. -- 2024-03-18T20:34:11.555 DEBUG:teuthology.orchestra.run.smithi046:mds.j> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.j.service 2024-03-18T20:34:11.584 INFO:journalctl@ceph.mds.g.smithi046.stdout:-- Logs begin at Mon 2024-03-18 20:12:57 UTC. -- 2024-03-18T20:34:11.585 DEBUG:teuthology.orchestra.run.smithi073:mds.b> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.b.service 2024-03-18T20:34:11.587 DEBUG:teuthology.orchestra.run.smithi073:mds.e> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.e.service 2024-03-18T20:34:11.615 INFO:journalctl@ceph.mds.j.smithi046.stdout:-- Logs begin at Mon 2024-03-18 20:12:57 UTC. -- 2024-03-18T20:34:11.627 INFO:journalctl@ceph.mds.b.smithi073.stdout:-- Logs begin at Mon 2024-03-18 20:12:50 UTC. -- 2024-03-18T20:34:11.628 DEBUG:teuthology.orchestra.run.smithi073:mds.h> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.h.service 2024-03-18T20:34:11.657 INFO:journalctl@ceph.mds.e.smithi073.stdout:-- Logs begin at Mon 2024-03-18 20:12:50 UTC. -- 2024-03-18T20:34:11.658 DEBUG:teuthology.orchestra.run.smithi073:mds.k> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.k.service 2024-03-18T20:34:11.686 INFO:journalctl@ceph.mds.h.smithi073.stdout:-- Logs begin at Mon 2024-03-18 20:12:50 UTC. -- 2024-03-18T20:34:11.687 DEBUG:teuthology.orchestra.run.smithi155:mds.c> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.c.service 2024-03-18T20:34:11.690 DEBUG:teuthology.orchestra.run.smithi155:mds.f> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.f.service 2024-03-18T20:34:11.714 INFO:journalctl@ceph.mds.k.smithi073.stdout:-- Logs begin at Mon 2024-03-18 20:12:50 UTC. -- 2024-03-18T20:34:11.721 INFO:journalctl@ceph.mds.c.smithi155.stdout:-- Logs begin at Mon 2024-03-18 20:13:03 UTC. -- 2024-03-18T20:34:11.721 DEBUG:teuthology.orchestra.run.smithi155:mds.i> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.i.service 2024-03-18T20:34:11.748 INFO:journalctl@ceph.mds.f.smithi155.stdout:-- Logs begin at Mon 2024-03-18 20:13:03 UTC. -- 2024-03-18T20:34:11.749 DEBUG:teuthology.orchestra.run.smithi155:mds.l> sudo journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.l.service 2024-03-18T20:34:11.779 INFO:journalctl@ceph.mds.i.smithi155.stdout:-- Logs begin at Mon 2024-03-18 20:13:03 UTC. -- 2024-03-18T20:34:11.779 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-18T20:34:11.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:34:11.805 INFO:journalctl@ceph.mds.l.smithi155.stdout:-- Logs begin at Mon 2024-03-18 20:13:03 UTC. -- 2024-03-18T20:34:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.935+0000 7efc8051a700 1 -- 172.21.15.46:0/2011870916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc780ffb90 msgr2=0x7efc780fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.935+0000 7efc8051a700 1 --2- 172.21.15.46:0/2011870916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc780ffb90 0x7efc780fffb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efc740099c0 tx=0x7efc74009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.935+0000 7efc8051a700 1 -- 172.21.15.46:0/2011870916 shutdown_connections 2024-03-18T20:34:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.935+0000 7efc8051a700 1 --2- 172.21.15.46:0/2011870916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc780ffb90 0x7efc780fffb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.935+0000 7efc8051a700 1 --2- 172.21.15.46:0/2011870916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc78101720 0x7efc78107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.935+0000 7efc8051a700 1 --2- 172.21.15.46:0/2011870916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc78100d80 0x7efc781011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.935+0000 7efc8051a700 1 -- 172.21.15.46:0/2011870916 >> 172.21.15.46:0/2011870916 conn(0x7efc780fb100 msgr2=0x7efc780fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.935+0000 7efc8051a700 1 -- 172.21.15.46:0/2011870916 shutdown_connections 2024-03-18T20:34:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.935+0000 7efc8051a700 1 -- 172.21.15.46:0/2011870916 wait complete. 2024-03-18T20:34:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc8051a700 1 Processor -- start 2024-03-18T20:34:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc8051a700 1 -- start start 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc8051a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc780ffb90 0x7efc78198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc8051a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc78100d80 0x7efc78199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc8051a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc78101720 0x7efc781a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc8051a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc781a0930 con 0x7efc78101720 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc8051a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc781a0a70 con 0x7efc78100d80 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc8051a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc781a0c00 con 0x7efc780ffb90 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc7eab7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc78101720 0x7efc781a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc7e2b6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc780ffb90 0x7efc78198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc7eab7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc78101720 0x7efc781a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56306/0 (socket says 172.21.15.46:56306) 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc7eab7700 1 -- 172.21.15.46:0/2406302941 learned_addr learned my addr 172.21.15.46:0/2406302941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.936+0000 7efc7dab5700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc78100d80 0x7efc78199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.937+0000 7efc7e2b6700 1 -- 172.21.15.46:0/2406302941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc78100d80 msgr2=0x7efc78199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.937+0000 7efc7e2b6700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc78100d80 0x7efc78199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.937+0000 7efc7e2b6700 1 -- 172.21.15.46:0/2406302941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc78101720 msgr2=0x7efc781a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.937+0000 7efc7e2b6700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc78101720 0x7efc781a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.937+0000 7efc7e2b6700 1 -- 172.21.15.46:0/2406302941 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc740095f0 con 0x7efc780ffb90 2024-03-18T20:34:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.937+0000 7efc7eab7700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc78101720 0x7efc781a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.937+0000 7efc7e2b6700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc780ffb90 0x7efc78198e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efc740098e0 tx=0x7efc74005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.937+0000 7efc6f7fe700 1 -- 172.21.15.46:0/2406302941 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7401c070 con 0x7efc780ffb90 2024-03-18T20:34:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.937+0000 7efc6f7fe700 1 -- 172.21.15.46:0/2406302941 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc7400fd10 con 0x7efc780ffb90 2024-03-18T20:34:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.937+0000 7efc8051a700 1 -- 172.21.15.46:0/2406302941 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc781a0e80 con 0x7efc780ffb90 2024-03-18T20:34:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.937+0000 7efc6f7fe700 1 -- 172.21.15.46:0/2406302941 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74022c30 con 0x7efc780ffb90 2024-03-18T20:34:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.937+0000 7efc8051a700 1 -- 172.21.15.46:0/2406302941 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc781a11f0 con 0x7efc780ffb90 2024-03-18T20:34:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.938+0000 7efc8051a700 1 -- 172.21.15.46:0/2406302941 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc78067ed0 con 0x7efc780ffb90 2024-03-18T20:34:11.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.939+0000 7efc6f7fe700 1 -- 172.21.15.46:0/2406302941 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7efc74020070 con 0x7efc780ffb90 2024-03-18T20:34:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.940+0000 7efc6f7fe700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efc640a1120 0x7efc640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.940+0000 7efc7dab5700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efc640a1120 0x7efc640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.940+0000 7efc6f7fe700 1 -- 172.21.15.46:0/2406302941 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efc740cd2b0 con 0x7efc780ffb90 2024-03-18T20:34:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.940+0000 7efc7dab5700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efc640a1120 0x7efc640a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc68009c60 tx=0x7efc68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:11.943+0000 7efc6f7fe700 1 -- 172.21.15.46:0/2406302941 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efc74096ed0 con 0x7efc780ffb90 2024-03-18T20:34:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.176+0000 7efc8051a700 1 -- 172.21.15.46:0/2406302941 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc781a13e0 con 0x7efc780ffb90 2024-03-18T20:34:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.178+0000 7efc6f7fe700 1 -- 172.21.15.46:0/2406302941 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18374 (secure 0 0 0) 0x7efc7409a4f0 con 0x7efc780ffb90 2024-03-18T20:34:12.181 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:12.181 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":74,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:34:06.766970+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:34:12.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.180+0000 7efc8051a700 1 -- 172.21.15.46:0/2406302941 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efc640a1120 msgr2=0x7efc640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:12.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.180+0000 7efc8051a700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efc640a1120 0x7efc640a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc68009c60 tx=0x7efc68009260 comp rx=0 tx=0).stop 2024-03-18T20:34:12.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.180+0000 7efc8051a700 1 -- 172.21.15.46:0/2406302941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc780ffb90 msgr2=0x7efc78198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:12.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.180+0000 7efc8051a700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc780ffb90 0x7efc78198e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efc740098e0 tx=0x7efc74005b00 comp rx=0 tx=0).stop 2024-03-18T20:34:12.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.180+0000 7efc8051a700 1 -- 172.21.15.46:0/2406302941 shutdown_connections 2024-03-18T20:34:12.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.180+0000 7efc8051a700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efc640a1120 0x7efc640a35e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.180+0000 7efc8051a700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc78101720 0x7efc781a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.180+0000 7efc8051a700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc780ffb90 0x7efc78198e00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.180+0000 7efc8051a700 1 --2- 172.21.15.46:0/2406302941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc78100d80 0x7efc78199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.180+0000 7efc8051a700 1 -- 172.21.15.46:0/2406302941 >> 172.21.15.46:0/2406302941 conn(0x7efc780fb100 msgr2=0x7efc78108920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:12.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.180+0000 7efc8051a700 1 -- 172.21.15.46:0/2406302941 shutdown_connections 2024-03-18T20:34:12.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.180+0000 7efc8051a700 1 -- 172.21.15.46:0/2406302941 wait complete. 2024-03-18T20:34:12.198 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T20:31:27.488318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T20:34:12.199 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T20:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.335+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/4141920677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8102910 msgr2=0x7fc5e81091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.335+0000 7fc5ed0c1700 1 --2- 172.21.15.46:0/4141920677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8102910 0x7fc5e81091c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000b0d0 tx=0x7fc5e000b3e0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.335+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/4141920677 shutdown_connections 2024-03-18T20:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.335+0000 7fc5ed0c1700 1 --2- 172.21.15.46:0/4141920677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8102910 0x7fc5e81091c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.335+0000 7fc5ed0c1700 1 --2- 172.21.15.46:0/4141920677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e8100d80 0x7fc5e81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.335+0000 7fc5ed0c1700 1 --2- 172.21.15.46:0/4141920677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5e8101f70 0x7fc5e81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.335+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/4141920677 >> 172.21.15.46:0/4141920677 conn(0x7fc5e80fc2f0 msgr2=0x7fc5e80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.335+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/4141920677 shutdown_connections 2024-03-18T20:34:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.335+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/4141920677 wait complete. 2024-03-18T20:34:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.336+0000 7fc5ed0c1700 1 Processor -- start 2024-03-18T20:34:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.336+0000 7fc5ed0c1700 1 -- start start 2024-03-18T20:34:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5ed0c1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8100d80 0x7fc5e8196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5ed0c1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e8101f70 0x7fc5e81972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5ed0c1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5e8102910 0x7fc5e819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5ed0c1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e819e850 con 0x7fc5e8100d80 2024-03-18T20:34:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5ed0c1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e819e9c0 con 0x7fc5e8102910 2024-03-18T20:34:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5ed0c1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e819eb80 con 0x7fc5e8101f70 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5e6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8100d80 0x7fc5e8196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5e6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8100d80 0x7fc5e8196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56326/0 (socket says 172.21.15.46:56326) 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5e6d9d700 1 -- 172.21.15.46:0/3352587038 learned_addr learned my addr 172.21.15.46:0/3352587038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5de59c700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e8101f70 0x7fc5e81972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5e759e700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5e8102910 0x7fc5e819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5e6d9d700 1 -- 172.21.15.46:0/3352587038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e8101f70 msgr2=0x7fc5e81972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5e6d9d700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e8101f70 0x7fc5e81972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5e6d9d700 1 -- 172.21.15.46:0/3352587038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5e8102910 msgr2=0x7fc5e819e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5e6d9d700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5e8102910 0x7fc5e819e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5e6d9d700 1 -- 172.21.15.46:0/3352587038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e0009d00 con 0x7fc5e8100d80 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5de59c700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e8101f70 0x7fc5e81972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5e6d9d700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8100d80 0x7fc5e8196d60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800e970 tx=0x7fc5d800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.337+0000 7fc5e4d99700 1 -- 172.21.15.46:0/3352587038 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8009790 con 0x7fc5e8100d80 2024-03-18T20:34:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.338+0000 7fc5e4d99700 1 -- 172.21.15.46:0/3352587038 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5d800c7e0 con 0x7fc5e8100d80 2024-03-18T20:34:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.338+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/3352587038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e819ee60 con 0x7fc5e8100d80 2024-03-18T20:34:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.338+0000 7fc5e4d99700 1 -- 172.21.15.46:0/3352587038 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8003c70 con 0x7fc5e8100d80 2024-03-18T20:34:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.338+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/3352587038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e819f280 con 0x7fc5e8100d80 2024-03-18T20:34:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.339+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/3352587038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e8108130 con 0x7fc5e8100d80 2024-03-18T20:34:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.340+0000 7fc5e4d99700 1 -- 172.21.15.46:0/3352587038 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc5d800cc50 con 0x7fc5e8100d80 2024-03-18T20:34:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.341+0000 7fc5e4d99700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc5d40a1120 0x7fc5d40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:12.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.341+0000 7fc5de59c700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc5d40a1120 0x7fc5d40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:12.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.341+0000 7fc5e4d99700 1 -- 172.21.15.46:0/3352587038 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc5d8014070 con 0x7fc5e8100d80 2024-03-18T20:34:12.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.342+0000 7fc5de59c700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc5d40a1120 0x7fc5d40a35e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000afd0 tx=0x7fc5d000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:12.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.345+0000 7fc5e4d99700 1 -- 172.21.15.46:0/3352587038 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc5d8096320 con 0x7fc5e8100d80 2024-03-18T20:34:12.416 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:12 smithi155 ceph-mon[48344]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:12.416 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:12 smithi155 ceph-mon[48344]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:12.416 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:12 smithi155 ceph-mon[48344]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-03-18T20:34:12.416 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:12.417 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:12.417 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:12.417 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:12.417 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:12.417 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:12.417 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:12.417 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:12 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:12 smithi073 ceph-mon[47384]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:12 smithi073 ceph-mon[47384]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:12 smithi073 ceph-mon[47384]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-03-18T20:34:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:12 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:12 smithi046 ceph-mon[43495]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:12 smithi046 ceph-mon[43495]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:12 smithi046 ceph-mon[43495]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-03-18T20:34:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:12.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:12 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:12.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.559+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/3352587038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5e802d0f0 con 0x7fc5e8100d80 2024-03-18T20:34:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.560+0000 7fc5e4d99700 1 -- 172.21.15.46:0/3352587038 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5d8003dd0 con 0x7fc5e8100d80 2024-03-18T20:34:12.563 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-18T20:34:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.562+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/3352587038 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc5d40a1120 msgr2=0x7fc5d40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.562+0000 7fc5ed0c1700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc5d40a1120 0x7fc5d40a35e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000afd0 tx=0x7fc5d000a380 comp rx=0 tx=0).stop 2024-03-18T20:34:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.562+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/3352587038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8100d80 msgr2=0x7fc5e8196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.562+0000 7fc5ed0c1700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8100d80 0x7fc5e8196d60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800e970 tx=0x7fc5d800ec80 comp rx=0 tx=0).stop 2024-03-18T20:34:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.562+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/3352587038 shutdown_connections 2024-03-18T20:34:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.562+0000 7fc5ed0c1700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc5d40a1120 0x7fc5d40a35e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.562+0000 7fc5ed0c1700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e8100d80 0x7fc5e8196d60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.562+0000 7fc5ed0c1700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5e8102910 0x7fc5e819e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.562+0000 7fc5ed0c1700 1 --2- 172.21.15.46:0/3352587038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5e8101f70 0x7fc5e81972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.562+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/3352587038 >> 172.21.15.46:0/3352587038 conn(0x7fc5e80fc2f0 msgr2=0x7fc5e8105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.562+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/3352587038 shutdown_connections 2024-03-18T20:34:12.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.562+0000 7fc5ed0c1700 1 -- 172.21.15.46:0/3352587038 wait complete. 2024-03-18T20:34:12.580 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:34:12.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.713+0000 7f93239c8700 1 -- 172.21.15.46:0/2961299782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c1028d0 msgr2=0x7f931c109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:12.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.713+0000 7f93239c8700 1 --2- 172.21.15.46:0/2961299782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c1028d0 0x7f931c109180 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f931400b0a0 tx=0x7f931400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.713+0000 7f93239c8700 1 -- 172.21.15.46:0/2961299782 shutdown_connections 2024-03-18T20:34:12.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.713+0000 7f93239c8700 1 --2- 172.21.15.46:0/2961299782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c1028d0 0x7f931c109180 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.713+0000 7f93239c8700 1 --2- 172.21.15.46:0/2961299782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f931c0fef60 0x7f931c0ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.713+0000 7f93239c8700 1 --2- 172.21.15.46:0/2961299782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931c101f30 0x7f931c102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.713+0000 7f93239c8700 1 -- 172.21.15.46:0/2961299782 >> 172.21.15.46:0/2961299782 conn(0x7f931c0fa8b0 msgr2=0x7f931c0fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:12.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.713+0000 7f93239c8700 1 -- 172.21.15.46:0/2961299782 shutdown_connections 2024-03-18T20:34:12.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.713+0000 7f93239c8700 1 -- 172.21.15.46:0/2961299782 wait complete. 2024-03-18T20:34:12.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.713+0000 7f93239c8700 1 Processor -- start 2024-03-18T20:34:12.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f93239c8700 1 -- start start 2024-03-18T20:34:12.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f93239c8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f931c0fef60 0x7f931c101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:12.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f93239c8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931c101f30 0x7f931c0ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:12.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f93239c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c1028d0 0x7f931c100420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:12.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f93239c8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c100bc0 con 0x7f931c1028d0 2024-03-18T20:34:12.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f93239c8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c100d30 con 0x7f931c0fef60 2024-03-18T20:34:12.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f93239c8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c1a0d60 con 0x7f931c101f30 2024-03-18T20:34:12.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f9321f65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c1028d0 0x7f931c100420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:12.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f9320f63700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931c101f30 0x7f931c0ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:12.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f9321f65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c1028d0 0x7f931c100420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56344/0 (socket says 172.21.15.46:56344) 2024-03-18T20:34:12.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f9321f65700 1 -- 172.21.15.46:0/592766265 learned_addr learned my addr 172.21.15.46:0/592766265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f9321764700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f931c0fef60 0x7f931c101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f9320f63700 1 -- 172.21.15.46:0/592766265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f931c0fef60 msgr2=0x7f931c101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f9320f63700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f931c0fef60 0x7f931c101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f9320f63700 1 -- 172.21.15.46:0/592766265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c1028d0 msgr2=0x7f931c100420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f9320f63700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c1028d0 0x7f931c100420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f9320f63700 1 -- 172.21.15.46:0/592766265 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314009d00 con 0x7f931c101f30 2024-03-18T20:34:12.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.714+0000 7f9321f65700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c1028d0 0x7f931c100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:12.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.715+0000 7f9320f63700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931c101f30 0x7f931c0ffee0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f930c00e9f0 tx=0x7f930c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:12.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.715+0000 7f93127fc700 1 -- 172.21.15.46:0/592766265 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c00c9c0 con 0x7f931c101f30 2024-03-18T20:34:12.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.715+0000 7f93239c8700 1 -- 172.21.15.46:0/592766265 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c1a0f90 con 0x7f931c101f30 2024-03-18T20:34:12.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.715+0000 7f93127fc700 1 -- 172.21.15.46:0/592766265 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f930c010430 con 0x7f931c101f30 2024-03-18T20:34:12.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.715+0000 7f93127fc700 1 -- 172.21.15.46:0/592766265 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c018680 con 0x7f931c101f30 2024-03-18T20:34:12.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.715+0000 7f93239c8700 1 -- 172.21.15.46:0/592766265 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c1a13c0 con 0x7f931c101f30 2024-03-18T20:34:12.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.716+0000 7f93239c8700 1 -- 172.21.15.46:0/592766265 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c108140 con 0x7f931c101f30 2024-03-18T20:34:12.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.717+0000 7f93127fc700 1 -- 172.21.15.46:0/592766265 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f930c010860 con 0x7f931c101f30 2024-03-18T20:34:12.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.717+0000 7f93127fc700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93080a1120 0x7f93080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:12.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.718+0000 7f9321764700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93080a1120 0x7f93080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:12.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.718+0000 7f93127fc700 1 -- 172.21.15.46:0/592766265 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f930c014070 con 0x7f931c101f30 2024-03-18T20:34:12.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.718+0000 7f9321764700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93080a1120 0x7f93080a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f931c069070 tx=0x7f9318007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:12.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.723+0000 7f93127fc700 1 -- 172.21.15.46:0/592766265 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f930c0529f0 con 0x7f931c101f30 2024-03-18T20:34:12.734 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 18 20:34:12 smithi155 systemd[1]: Starting Ceph mds.l for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:34:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.987+0000 7f93239c8700 1 -- 172.21.15.46:0/592766265 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f931c02cf60 con 0x7f931c101f30 2024-03-18T20:34:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.987+0000 7f93127fc700 1 -- 172.21.15.46:0/592766265 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f930c096140 con 0x7f931c101f30 2024-03-18T20:34:12.991 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:12.991 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-18T20:34:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.990+0000 7f93239c8700 1 -- 172.21.15.46:0/592766265 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93080a1120 msgr2=0x7f93080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.990+0000 7f93239c8700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93080a1120 0x7f93080a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f931c069070 tx=0x7f9318007400 comp rx=0 tx=0).stop 2024-03-18T20:34:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.990+0000 7f93239c8700 1 -- 172.21.15.46:0/592766265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931c101f30 msgr2=0x7f931c0ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.990+0000 7f93239c8700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931c101f30 0x7f931c0ffee0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f930c00e9f0 tx=0x7f930c00ed00 comp rx=0 tx=0).stop 2024-03-18T20:34:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.990+0000 7f93239c8700 1 -- 172.21.15.46:0/592766265 shutdown_connections 2024-03-18T20:34:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.990+0000 7f93239c8700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f93080a1120 0x7f93080a35e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.990+0000 7f93239c8700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f931c1028d0 0x7f931c100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.990+0000 7f93239c8700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f931c0fef60 0x7f931c101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.990+0000 7f93239c8700 1 --2- 172.21.15.46:0/592766265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931c101f30 0x7f931c0ffee0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.990+0000 7f93239c8700 1 -- 172.21.15.46:0/592766265 >> 172.21.15.46:0/592766265 conn(0x7f931c0fa8b0 msgr2=0x7f931c105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.990+0000 7f93239c8700 1 -- 172.21.15.46:0/592766265 shutdown_connections 2024-03-18T20:34:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:12.990+0000 7f93239c8700 1 -- 172.21.15.46:0/592766265 wait complete. 2024-03-18T20:34:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 1 2024-03-18T20:34:13.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:13 smithi046 ceph-mon[43495]: Deploying daemon mds.l on smithi155 2024-03-18T20:34:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:13 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2406302941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:13 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3352587038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:34:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:13 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/592766265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:13 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6832/4207499009,v1:172.21.15.155:6833/4207499009] up:boot 2024-03-18T20:34:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:13 smithi046 ceph-mon[43495]: fsmap 1 up:standby 2024-03-18T20:34:13.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:13 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T20:34:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:13 smithi073 ceph-mon[47384]: Deploying daemon mds.l on smithi155 2024-03-18T20:34:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:13 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2406302941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:13 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3352587038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:34:13.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:13 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/592766265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:13.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:13 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6832/4207499009,v1:172.21.15.155:6833/4207499009] up:boot 2024-03-18T20:34:13.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:13 smithi073 ceph-mon[47384]: fsmap 1 up:standby 2024-03-18T20:34:13.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:13 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T20:34:13.541 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 18 20:34:13 smithi155 ceph-mds[84315]: -- 172.21.15.155:0/1543026596 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559ecced81a0 con 0x559ecceaec00 2024-03-18T20:34:13.541 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 18 20:34:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-l[84292]: starting mds.l at 2024-03-18T20:34:13.541 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 18 20:34:13 smithi155 bash[84277]: 348f0a5da15c6147f4e3f542e07cb5575522f6418d5b52e1f4a1fb29be08a257 2024-03-18T20:34:13.541 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 18 20:34:13 smithi155 systemd[1]: Started Ceph mds.l for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:34:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:13 smithi155 ceph-mon[48344]: Deploying daemon mds.l on smithi155 2024-03-18T20:34:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:13 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2406302941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:13 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3352587038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:34:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:13 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/592766265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:13 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6832/4207499009,v1:172.21.15.155:6833/4207499009] up:boot 2024-03-18T20:34:13.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:13 smithi155 ceph-mon[48344]: fsmap 1 up:standby 2024-03-18T20:34:13.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:13 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T20:34:14.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:14 smithi073 ceph-mon[47384]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:14.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:14 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:14.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:14 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:14.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:14 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:14.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:14 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:14.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:14 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:14.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:14 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:14.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:14 smithi073 ceph-mon[47384]: Deploying daemon mds.k on smithi073 2024-03-18T20:34:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:14 smithi155 ceph-mon[48344]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:14 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:14 smithi155 ceph-mon[48344]: Deploying daemon mds.k on smithi073 2024-03-18T20:34:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:14 smithi046 ceph-mon[43495]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:14 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:14 smithi046 ceph-mon[43495]: Deploying daemon mds.k on smithi073 2024-03-18T20:34:14.676 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:34:14 smithi073 systemd[1]: Starting Ceph mds.k for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:34:15.011 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:34:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.152+0000 7f767d372700 1 -- 172.21.15.46:0/1523030992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678104070 msgr2=0x7f7678106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.152+0000 7f767d372700 1 --2- 172.21.15.46:0/1523030992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678104070 0x7f7678106460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f766c0098f0 tx=0x7f766c009c00 comp rx=0 tx=0).stop 2024-03-18T20:34:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.152+0000 7f767d372700 1 -- 172.21.15.46:0/1523030992 shutdown_connections 2024-03-18T20:34:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.152+0000 7f767d372700 1 --2- 172.21.15.46:0/1523030992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76781069a0 0x7f7678108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.152+0000 7f767d372700 1 --2- 172.21.15.46:0/1523030992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7678101740 0x7f7678103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.152+0000 7f767d372700 1 --2- 172.21.15.46:0/1523030992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678104070 0x7f7678106460 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.152+0000 7f767d372700 1 -- 172.21.15.46:0/1523030992 >> 172.21.15.46:0/1523030992 conn(0x7f76780fb0d0 msgr2=0x7f76780fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.152+0000 7f767d372700 1 -- 172.21.15.46:0/1523030992 shutdown_connections 2024-03-18T20:34:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.152+0000 7f767d372700 1 -- 172.21.15.46:0/1523030992 wait complete. 2024-03-18T20:34:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.152+0000 7f767d372700 1 Processor -- start 2024-03-18T20:34:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f767d372700 1 -- start start 2024-03-18T20:34:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f767d372700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678101740 0x7f7678101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f767d372700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7678104070 0x7f76780ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f767d372700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76781069a0 0x7f76780ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f767d372700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76781003d0 con 0x7f76781069a0 2024-03-18T20:34:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f767d372700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7678100540 con 0x7f7678101740 2024-03-18T20:34:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f767d372700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76781a0cd0 con 0x7f7678104070 2024-03-18T20:34:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f76777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76781069a0 0x7f76780ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f76777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76781069a0 0x7f76780ffc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56350/0 (socket says 172.21.15.46:56350) 2024-03-18T20:34:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f76777fe700 1 -- 172.21.15.46:0/3581091663 learned_addr learned my addr 172.21.15.46:0/3581091663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f7676ffd700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678101740 0x7f7678101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f76767fc700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7678104070 0x7f76780ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f76777fe700 1 -- 172.21.15.46:0/3581091663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7678104070 msgr2=0x7f76780ff6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f76777fe700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7678104070 0x7f76780ff6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f76777fe700 1 -- 172.21.15.46:0/3581091663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678101740 msgr2=0x7f7678101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f76777fe700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678101740 0x7f7678101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f76777fe700 1 -- 172.21.15.46:0/3581091663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c009520 con 0x7f76781069a0 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.153+0000 7f7676ffd700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678101740 0x7f7678101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.154+0000 7f76777fe700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76781069a0 0x7f76780ffc30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f767000c110 tx=0x7f767000c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.154+0000 7f765ffff700 1 -- 172.21.15.46:0/3581091663 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670017070 con 0x7f76781069a0 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.154+0000 7f765ffff700 1 -- 172.21.15.46:0/3581091663 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f767000e050 con 0x7f76781069a0 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.154+0000 7f767d372700 1 -- 172.21.15.46:0/3581091663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76781a0f00 con 0x7f76781069a0 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.154+0000 7f765ffff700 1 -- 172.21.15.46:0/3581091663 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7670007840 con 0x7f76781069a0 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.154+0000 7f767d372700 1 -- 172.21.15.46:0/3581091663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76781a1330 con 0x7f76781069a0 2024-03-18T20:34:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.155+0000 7f767d372700 1 -- 172.21.15.46:0/3581091663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76780fcc50 con 0x7f76781069a0 2024-03-18T20:34:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.157+0000 7f765ffff700 1 -- 172.21.15.46:0/3581091663 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7670007a60 con 0x7f76781069a0 2024-03-18T20:34:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.157+0000 7f765ffff700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f76600a1120 0x7f76600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.157+0000 7f765ffff700 1 -- 172.21.15.46:0/3581091663 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f76700cc880 con 0x7f76781069a0 2024-03-18T20:34:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.157+0000 7f7676ffd700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f76600a1120 0x7f76600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:15.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.158+0000 7f7676ffd700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f76600a1120 0x7f76600a35e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7668010420 tx=0x7f76680073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:15.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.161+0000 7f765ffff700 1 -- 172.21.15.46:0/3581091663 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7670099ea0 con 0x7f76781069a0 2024-03-18T20:34:15.220 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:34:14 smithi073 ceph-mds[82810]: -- 172.21.15.73:0/1595684799 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5585aa3241a0 con 0x5585aa2fb000 2024-03-18T20:34:15.220 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:34:14 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: starting mds.k at 2024-03-18T20:34:15.220 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:34:14 smithi073 bash[82773]: c5e8c5baedf024e8bb2699a001aaa8a32049a28497235e7c969a67c1a4e013e2 2024-03-18T20:34:15.220 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:34:14 smithi073 systemd[1]: Started Ceph mds.k for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:34:15.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.437+0000 7f767d372700 1 -- 172.21.15.46:0/3581091663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f767802cf60 con 0x7f76781069a0 2024-03-18T20:34:15.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.438+0000 7f765ffff700 1 -- 172.21.15.46:0/3581091663 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f7670099a30 con 0x7f76781069a0 2024-03-18T20:34:15.441 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:15.441 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-18T20:34:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.440+0000 7f767d372700 1 -- 172.21.15.46:0/3581091663 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f76600a1120 msgr2=0x7f76600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.440+0000 7f767d372700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f76600a1120 0x7f76600a35e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7668010420 tx=0x7f76680073d0 comp rx=0 tx=0).stop 2024-03-18T20:34:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.440+0000 7f767d372700 1 -- 172.21.15.46:0/3581091663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76781069a0 msgr2=0x7f76780ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.440+0000 7f767d372700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76781069a0 0x7f76780ffc30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f767000c110 tx=0x7f767000c420 comp rx=0 tx=0).stop 2024-03-18T20:34:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.440+0000 7f767d372700 1 -- 172.21.15.46:0/3581091663 shutdown_connections 2024-03-18T20:34:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.440+0000 7f767d372700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f76600a1120 0x7f76600a35e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.440+0000 7f767d372700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76781069a0 0x7f76780ffc30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.440+0000 7f767d372700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678101740 0x7f7678101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.440+0000 7f767d372700 1 --2- 172.21.15.46:0/3581091663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7678104070 0x7f76780ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:15.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.440+0000 7f767d372700 1 -- 172.21.15.46:0/3581091663 >> 172.21.15.46:0/3581091663 conn(0x7f76780fb0d0 msgr2=0x7f7678103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:15.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.440+0000 7f767d372700 1 -- 172.21.15.46:0/3581091663 shutdown_connections 2024-03-18T20:34:15.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:15.440+0000 7f767d372700 1 -- 172.21.15.46:0/3581091663 wait complete. 2024-03-18T20:34:15.445 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 2 2024-03-18T20:34:16.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:15 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:16.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:15 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:16.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:15 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:16.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:15 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:16.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:15 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:16.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:15 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:16.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:15 smithi046 ceph-mon[43495]: Deploying daemon mds.j on smithi046 2024-03-18T20:34:16.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:15 smithi046 ceph-mon[43495]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:16.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:15 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3581091663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:16.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:15 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] up:boot 2024-03-18T20:34:16.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:15 smithi046 ceph-mon[43495]: fsmap 2 up:standby 2024-03-18T20:34:16.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:15 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T20:34:16.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:16.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:16.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:16.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:16.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:16.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:16.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:15 smithi073 ceph-mon[47384]: Deploying daemon mds.j on smithi046 2024-03-18T20:34:16.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:15 smithi073 ceph-mon[47384]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:16.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:15 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3581091663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:16.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:15 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] up:boot 2024-03-18T20:34:16.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:15 smithi073 ceph-mon[47384]: fsmap 2 up:standby 2024-03-18T20:34:16.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:15 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T20:34:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:15 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:15 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:15 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:15 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:15 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:15 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:15 smithi155 ceph-mon[48344]: Deploying daemon mds.j on smithi046 2024-03-18T20:34:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:15 smithi155 ceph-mon[48344]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:15 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3581091663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:15 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] up:boot 2024-03-18T20:34:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:15 smithi155 ceph-mon[48344]: fsmap 2 up:standby 2024-03-18T20:34:16.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:15 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T20:34:16.446 INFO:journalctl@ceph.mds.j.smithi046.stdout:Mar 18 20:34:16 smithi046 systemd[1]: Starting Ceph mds.j for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:34:17.047 INFO:journalctl@ceph.mds.j.smithi046.stdout:Mar 18 20:34:16 smithi046 bash[92483]: 96542963d0e35fbf5531c1191fdcd597b694c046a5dafbc286a112a63921652d 2024-03-18T20:34:17.047 INFO:journalctl@ceph.mds.j.smithi046.stdout:Mar 18 20:34:16 smithi046 ceph-mds[92521]: -- 172.21.15.46:0/1889889341 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560f58dd01a0 con 0x560f58da6c00 2024-03-18T20:34:17.047 INFO:journalctl@ceph.mds.j.smithi046.stdout:Mar 18 20:34:16 smithi046 systemd[1]: Started Ceph mds.j for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:34:17.047 INFO:journalctl@ceph.mds.j.smithi046.stdout:Mar 18 20:34:16 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-j[92498]: starting mds.j at 2024-03-18T20:34:17.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:34:17.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.612+0000 7fc70b725700 1 -- 172.21.15.46:0/1952944481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7040ffb60 msgr2=0x7fc7040fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:17.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.612+0000 7fc70b725700 1 --2- 172.21.15.46:0/1952944481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7040ffb60 0x7fc7040fff80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc7000099c0 tx=0x7fc700009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:17.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.612+0000 7fc70b725700 1 -- 172.21.15.46:0/1952944481 shutdown_connections 2024-03-18T20:34:17.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.612+0000 7fc70b725700 1 --2- 172.21.15.46:0/1952944481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7040ffb60 0x7fc7040fff80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:17.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.612+0000 7fc70b725700 1 --2- 172.21.15.46:0/1952944481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7041016f0 0x7fc704107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:17.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.612+0000 7fc70b725700 1 --2- 172.21.15.46:0/1952944481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc704100d50 0x7fc7041011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:17.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.612+0000 7fc70b725700 1 -- 172.21.15.46:0/1952944481 >> 172.21.15.46:0/1952944481 conn(0x7fc7040fb0d0 msgr2=0x7fc7040fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.612+0000 7fc70b725700 1 -- 172.21.15.46:0/1952944481 shutdown_connections 2024-03-18T20:34:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.612+0000 7fc70b725700 1 -- 172.21.15.46:0/1952944481 wait complete. 2024-03-18T20:34:17.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.613+0000 7fc70b725700 1 Processor -- start 2024-03-18T20:34:17.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.613+0000 7fc70b725700 1 -- start start 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.613+0000 7fc70b725700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7040ffb60 0x7fc704103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.613+0000 7fc70b725700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc704100d50 0x7fc704102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.613+0000 7fc70b725700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7041016f0 0x7fc704102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.613+0000 7fc70b725700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc704102e10 con 0x7fc7041016f0 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.613+0000 7fc70b725700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc704102f80 con 0x7fc704100d50 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.613+0000 7fc70b725700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7041a0bc0 con 0x7fc7040ffb60 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc7094c1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7040ffb60 0x7fc704103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc709cc2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7041016f0 0x7fc704102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc7094c1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7040ffb60 0x7fc704103aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33038/0 (socket says 172.21.15.46:33038) 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc7094c1700 1 -- 172.21.15.46:0/3227140857 learned_addr learned my addr 172.21.15.46:0/3227140857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc709cc2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7041016f0 0x7fc704102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60792/0 (socket says 172.21.15.46:60792) 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc708cc0700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc704100d50 0x7fc704102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc709cc2700 1 -- 172.21.15.46:0/3227140857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7040ffb60 msgr2=0x7fc704103aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc709cc2700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7040ffb60 0x7fc704103aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:17.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc709cc2700 1 -- 172.21.15.46:0/3227140857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc704100d50 msgr2=0x7fc704102130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:17.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc709cc2700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc704100d50 0x7fc704102130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:17.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc709cc2700 1 -- 172.21.15.46:0/3227140857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7000095f0 con 0x7fc7041016f0 2024-03-18T20:34:17.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc708cc0700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc704100d50 0x7fc704102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:34:17.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc709cc2700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7041016f0 0x7fc704102670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00cc50 tx=0x7fc6fc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:17.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc6fa7fc700 1 -- 172.21.15.46:0/3227140857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00aae0 con 0x7fc7041016f0 2024-03-18T20:34:17.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc6fa7fc700 1 -- 172.21.15.46:0/3227140857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6fc00ac40 con 0x7fc7041016f0 2024-03-18T20:34:17.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc70b725700 1 -- 172.21.15.46:0/3227140857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7041a0e50 con 0x7fc7041016f0 2024-03-18T20:34:17.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc6fa7fc700 1 -- 172.21.15.46:0/3227140857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc0115e0 con 0x7fc7041016f0 2024-03-18T20:34:17.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.614+0000 7fc70b725700 1 -- 172.21.15.46:0/3227140857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7041a1280 con 0x7fc7041016f0 2024-03-18T20:34:17.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.615+0000 7fc70b725700 1 -- 172.21.15.46:0/3227140857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc704067a10 con 0x7fc7041016f0 2024-03-18T20:34:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.617+0000 7fc6fa7fc700 1 -- 172.21.15.46:0/3227140857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc6fc004ad0 con 0x7fc7041016f0 2024-03-18T20:34:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.618+0000 7fc6fa7fc700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc6f00a1120 0x7fc6f00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.618+0000 7fc7094c1700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc6f00a1120 0x7fc6f00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.618+0000 7fc6fa7fc700 1 -- 172.21.15.46:0/3227140857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc6fc0cca10 con 0x7fc7041016f0 2024-03-18T20:34:17.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.618+0000 7fc7094c1700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc6f00a1120 0x7fc6f00a35e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc700003820 tx=0x7fc700005060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:17.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.625+0000 7fc6fa7fc700 1 -- 172.21.15.46:0/3227140857 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc6fc096640 con 0x7fc7041016f0 2024-03-18T20:34:17.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:17.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:17.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:17.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:17.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:17.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:17.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:17 smithi046 ceph-mon[43495]: Deploying daemon mds.d on smithi046 2024-03-18T20:34:17.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:17 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.46:6834/231081000,v1:172.21.15.46:6835/231081000] up:boot 2024-03-18T20:34:17.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:17 smithi046 ceph-mon[43495]: fsmap 3 up:standby 2024-03-18T20:34:17.704 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:17 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T20:34:17.705 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:17 smithi046 ceph-mon[43495]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.932+0000 7fc70b725700 1 -- 172.21.15.46:0/3227140857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7041a15e0 con 0x7fc7041016f0 2024-03-18T20:34:17.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.933+0000 7fc6fa7fc700 1 -- 172.21.15.46:0/3227140857 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2771 (secure 0 0 0) 0x7fc6fc099c60 con 0x7fc7041016f0 2024-03-18T20:34:17.936 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:17.937 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-18T20:34:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.935+0000 7fc70b725700 1 -- 172.21.15.46:0/3227140857 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc6f00a1120 msgr2=0x7fc6f00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.935+0000 7fc70b725700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc6f00a1120 0x7fc6f00a35e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc700003820 tx=0x7fc700005060 comp rx=0 tx=0).stop 2024-03-18T20:34:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.935+0000 7fc70b725700 1 -- 172.21.15.46:0/3227140857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7041016f0 msgr2=0x7fc704102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.935+0000 7fc70b725700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7041016f0 0x7fc704102670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00cc50 tx=0x7fc6fc00cf60 comp rx=0 tx=0).stop 2024-03-18T20:34:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.935+0000 7fc70b725700 1 -- 172.21.15.46:0/3227140857 shutdown_connections 2024-03-18T20:34:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.935+0000 7fc70b725700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc6f00a1120 0x7fc6f00a35e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.935+0000 7fc70b725700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7041016f0 0x7fc704102670 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.935+0000 7fc70b725700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7040ffb60 0x7fc704103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.935+0000 7fc70b725700 1 --2- 172.21.15.46:0/3227140857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc704100d50 0x7fc704102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.935+0000 7fc70b725700 1 -- 172.21.15.46:0/3227140857 >> 172.21.15.46:0/3227140857 conn(0x7fc7040fb0d0 msgr2=0x7fc7040fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.935+0000 7fc70b725700 1 -- 172.21.15.46:0/3227140857 shutdown_connections 2024-03-18T20:34:17.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:17.935+0000 7fc70b725700 1 -- 172.21.15.46:0/3227140857 wait complete. 2024-03-18T20:34:17.940 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 4 2024-03-18T20:34:17.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:17.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:17.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:17.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:17.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:17.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:17.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:17 smithi073 ceph-mon[47384]: Deploying daemon mds.d on smithi046 2024-03-18T20:34:17.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:17 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.46:6834/231081000,v1:172.21.15.46:6835/231081000] up:boot 2024-03-18T20:34:17.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:17 smithi073 ceph-mon[47384]: fsmap 3 up:standby 2024-03-18T20:34:17.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:17 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T20:34:17.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:17 smithi073 ceph-mon[47384]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:17.999 INFO:journalctl@ceph.mds.d.smithi046.stdout:Mar 18 20:34:17 smithi046 systemd[1]: Starting Ceph mds.d for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:34:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:17 smithi155 ceph-mon[48344]: Deploying daemon mds.d on smithi046 2024-03-18T20:34:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:17 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.46:6834/231081000,v1:172.21.15.46:6835/231081000] up:boot 2024-03-18T20:34:18.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:17 smithi155 ceph-mon[48344]: fsmap 3 up:standby 2024-03-18T20:34:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:17 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T20:34:18.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:17 smithi155 ceph-mon[48344]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:18.797 INFO:journalctl@ceph.mds.d.smithi046.stdout:Mar 18 20:34:18 smithi046 bash[92872]: 49531c10947a9d418a10cd31ed45faf6a2a23d72d829153ecaf3a0b2a4b2367a 2024-03-18T20:34:18.797 INFO:journalctl@ceph.mds.d.smithi046.stdout:Mar 18 20:34:18 smithi046 systemd[1]: Started Ceph mds.d for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:34:18.797 INFO:journalctl@ceph.mds.d.smithi046.stdout:Mar 18 20:34:18 smithi046 ceph-mds[92910]: -- 172.21.15.46:0/2067067858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555d3df241a0 con 0x555d3defb000 2024-03-18T20:34:18.797 INFO:journalctl@ceph.mds.d.smithi046.stdout:Mar 18 20:34:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-d[92887]: starting mds.d at 2024-03-18T20:34:19.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:18 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3227140857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:19.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:18 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:19.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:18 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:19.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:18 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:19.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:18 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:19.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:18 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:19.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:18 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:19.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:18 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3227140857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:19.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:18 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:19.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:18 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:19.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:18 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:19.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:18 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:19.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:18 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:19.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:18 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:19.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:18 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3227140857' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:19.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:18 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:19.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:18 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:19.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:18 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:19.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:18 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:19.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:18 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:19.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:18 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:19.923 INFO:journalctl@ceph.mds.e.smithi073.stdout:Mar 18 20:34:19 smithi073 systemd[1]: Starting Ceph mds.e for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:34:19.954 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:34:20.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.104+0000 7fabbc3dd700 1 -- 172.21.15.46:0/2287217868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb41069b0 msgr2=0x7fabb4108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:20.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.104+0000 7fabbc3dd700 1 --2- 172.21.15.46:0/2287217868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb41069b0 0x7fabb4108dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fabac00b0a0 tx=0x7fabac00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.104+0000 7fabbc3dd700 1 -- 172.21.15.46:0/2287217868 shutdown_connections 2024-03-18T20:34:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.104+0000 7fabbc3dd700 1 --2- 172.21.15.46:0/2287217868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb41069b0 0x7fabb4108dc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.104+0000 7fabbc3dd700 1 --2- 172.21.15.46:0/2287217868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb4101750 0x7fabb4103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.104+0000 7fabbc3dd700 1 --2- 172.21.15.46:0/2287217868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb4104080 0x7fabb4106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.104+0000 7fabbc3dd700 1 -- 172.21.15.46:0/2287217868 >> 172.21.15.46:0/2287217868 conn(0x7fabb40fb100 msgr2=0x7fabb40fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.104+0000 7fabbc3dd700 1 -- 172.21.15.46:0/2287217868 shutdown_connections 2024-03-18T20:34:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.104+0000 7fabbc3dd700 1 -- 172.21.15.46:0/2287217868 wait complete. 2024-03-18T20:34:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabbc3dd700 1 Processor -- start 2024-03-18T20:34:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabbc3dd700 1 -- start start 2024-03-18T20:34:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabbc3dd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb4101750 0x7fabb4101020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabbc3dd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4104080 0x7fabb40ff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabbc3dd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb41069b0 0x7fabb40ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:20.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabbc3dd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb4100340 con 0x7fabb4104080 2024-03-18T20:34:20.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabbc3dd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb41004b0 con 0x7fabb41069b0 2024-03-18T20:34:20.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabbc3dd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb41a0de0 con 0x7fabb4101750 2024-03-18T20:34:20.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabb9978700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4104080 0x7fabb40ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:20.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabb9978700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4104080 0x7fabb40ff700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60818/0 (socket says 172.21.15.46:60818) 2024-03-18T20:34:20.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabb9978700 1 -- 172.21.15.46:0/3257104548 learned_addr learned my addr 172.21.15.46:0/3257104548 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:20.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabba97a700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb41069b0 0x7fabb40ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:20.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.105+0000 7fabba179700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb4101750 0x7fabb4101020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabb9978700 1 -- 172.21.15.46:0/3257104548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb4101750 msgr2=0x7fabb4101020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabb9978700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb4101750 0x7fabb4101020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabb9978700 1 -- 172.21.15.46:0/3257104548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb41069b0 msgr2=0x7fabb40ffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabb9978700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb41069b0 0x7fabb40ffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabb9978700 1 -- 172.21.15.46:0/3257104548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac009d00 con 0x7fabb4104080 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabba179700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb4101750 0x7fabb4101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabba97a700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb41069b0 0x7fabb40ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabb9978700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4104080 0x7fabb40ff700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faba400e9a0 tx=0x7faba400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabab7fe700 1 -- 172.21.15.46:0/3257104548 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4009790 con 0x7fabb4104080 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabab7fe700 1 -- 172.21.15.46:0/3257104548 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faba400cbd0 con 0x7fabb4104080 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabbc3dd700 1 -- 172.21.15.46:0/3257104548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb41a0fe0 con 0x7fabb4104080 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabab7fe700 1 -- 172.21.15.46:0/3257104548 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4018680 con 0x7fabb4104080 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.106+0000 7fabbc3dd700 1 -- 172.21.15.46:0/3257104548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb41a1510 con 0x7fabb4104080 2024-03-18T20:34:20.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.107+0000 7fabbc3dd700 1 -- 172.21.15.46:0/3257104548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb40fcc80 con 0x7fabb4104080 2024-03-18T20:34:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.108+0000 7fabab7fe700 1 -- 172.21.15.46:0/3257104548 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7faba400cd40 con 0x7fabb4104080 2024-03-18T20:34:20.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.109+0000 7fabab7fe700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faba00a11a0 0x7faba00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:20.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.109+0000 7fabba179700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faba00a11a0 0x7faba00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:20.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.109+0000 7fabab7fe700 1 -- 172.21.15.46:0/3257104548 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faba4014070 con 0x7fabb4104080 2024-03-18T20:34:20.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.110+0000 7fabba179700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faba00a11a0 0x7faba00a3660 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009cb0 tx=0x7fabb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.113+0000 7fabab7fe700 1 -- 172.21.15.46:0/3257104548 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faba4096540 con 0x7fabb4104080 2024-03-18T20:34:20.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:19 smithi073 ceph-mon[47384]: Deploying daemon mds.e on smithi073 2024-03-18T20:34:20.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:19 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.46:6836/114990598,v1:172.21.15.46:6837/114990598] up:boot 2024-03-18T20:34:20.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:19 smithi073 ceph-mon[47384]: fsmap 4 up:standby 2024-03-18T20:34:20.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:19 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T20:34:20.179 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:19 smithi073 ceph-mon[47384]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:19 smithi155 ceph-mon[48344]: Deploying daemon mds.e on smithi073 2024-03-18T20:34:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:19 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.46:6836/114990598,v1:172.21.15.46:6837/114990598] up:boot 2024-03-18T20:34:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:19 smithi155 ceph-mon[48344]: fsmap 4 up:standby 2024-03-18T20:34:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:19 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T20:34:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:19 smithi155 ceph-mon[48344]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:20.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:19 smithi046 ceph-mon[43495]: Deploying daemon mds.e on smithi073 2024-03-18T20:34:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:19 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.46:6836/114990598,v1:172.21.15.46:6837/114990598] up:boot 2024-03-18T20:34:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:19 smithi046 ceph-mon[43495]: fsmap 4 up:standby 2024-03-18T20:34:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:19 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T20:34:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:19 smithi046 ceph-mon[43495]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.379+0000 7fabbc3dd700 1 -- 172.21.15.46:0/3257104548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabb4067e60 con 0x7fabb4104080 2024-03-18T20:34:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.380+0000 7fabab7fe700 1 -- 172.21.15.46:0/3257104548 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3525 (secure 0 0 0) 0x7faba4026de0 con 0x7fabb4104080 2024-03-18T20:34:20.384 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:20.384 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-18T20:34:20.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.382+0000 7fabbc3dd700 1 -- 172.21.15.46:0/3257104548 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faba00a11a0 msgr2=0x7faba00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:20.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.382+0000 7fabbc3dd700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faba00a11a0 0x7faba00a3660 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009cb0 tx=0x7fabb0009260 comp rx=0 tx=0).stop 2024-03-18T20:34:20.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.382+0000 7fabbc3dd700 1 -- 172.21.15.46:0/3257104548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4104080 msgr2=0x7fabb40ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:20.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.382+0000 7fabbc3dd700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4104080 0x7fabb40ff700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faba400e9a0 tx=0x7faba400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:34:20.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.382+0000 7fabbc3dd700 1 -- 172.21.15.46:0/3257104548 shutdown_connections 2024-03-18T20:34:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.382+0000 7fabbc3dd700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faba00a11a0 0x7faba00a3660 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.382+0000 7fabbc3dd700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb41069b0 0x7fabb40ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.382+0000 7fabbc3dd700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fabb4101750 0x7fabb4101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.382+0000 7fabbc3dd700 1 --2- 172.21.15.46:0/3257104548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4104080 0x7fabb40ff700 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.382+0000 7fabbc3dd700 1 -- 172.21.15.46:0/3257104548 >> 172.21.15.46:0/3257104548 conn(0x7fabb40fb100 msgr2=0x7fabb4101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:20.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.383+0000 7fabbc3dd700 1 -- 172.21.15.46:0/3257104548 shutdown_connections 2024-03-18T20:34:20.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:20.383+0000 7fabbc3dd700 1 -- 172.21.15.46:0/3257104548 wait complete. 2024-03-18T20:34:20.389 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 5 2024-03-18T20:34:20.471 INFO:journalctl@ceph.mds.e.smithi073.stdout:Mar 18 20:34:20 smithi073 ceph-mds[83150]: -- 172.21.15.73:0/2260116430 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5582a5ade1a0 con 0x5582a5ab4c00 2024-03-18T20:34:20.471 INFO:journalctl@ceph.mds.e.smithi073.stdout:Mar 18 20:34:20 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-e[83127]: starting mds.e at 2024-03-18T20:34:20.471 INFO:journalctl@ceph.mds.e.smithi073.stdout:Mar 18 20:34:20 smithi073 bash[83113]: 34b59532efea26ee810017ea81a7fb4a155ef81f273a43cc7baa33809cbd4da4 2024-03-18T20:34:20.471 INFO:journalctl@ceph.mds.e.smithi073.stdout:Mar 18 20:34:20 smithi073 systemd[1]: Started Ceph mds.e for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:34:21.288 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:21.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:21.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:21.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:21.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:21.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:21.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-mon[48344]: Deploying daemon mds.f on smithi155 2024-03-18T20:34:21.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3257104548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:21.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.73:6834/2049469125,v1:172.21.15.73:6835/2049469125] up:boot 2024-03-18T20:34:21.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-mon[48344]: fsmap 5 up:standby 2024-03-18T20:34:21.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T20:34:21.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:21.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:21.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:21.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:21.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:21.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:21.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:21 smithi046 ceph-mon[43495]: Deploying daemon mds.f on smithi155 2024-03-18T20:34:21.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:21 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3257104548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:21.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:21 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.73:6834/2049469125,v1:172.21.15.73:6835/2049469125] up:boot 2024-03-18T20:34:21.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:21 smithi046 ceph-mon[43495]: fsmap 5 up:standby 2024-03-18T20:34:21.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:21 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T20:34:21.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:21.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:21.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:21.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:21.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:21.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:21.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:21 smithi073 ceph-mon[47384]: Deploying daemon mds.f on smithi155 2024-03-18T20:34:21.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:21 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3257104548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:21.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:21 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.73:6834/2049469125,v1:172.21.15.73:6835/2049469125] up:boot 2024-03-18T20:34:21.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:21 smithi073 ceph-mon[47384]: fsmap 5 up:standby 2024-03-18T20:34:21.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:21 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T20:34:21.893 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 18 20:34:21 smithi155 systemd[1]: Starting Ceph mds.f for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:34:22.267 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 18 20:34:21 smithi155 bash[84620]: 436e410027f71401e104d96873822657e6e05e4e7d15c60a30c8f4b85b3c6a6e 2024-03-18T20:34:22.267 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 18 20:34:21 smithi155 systemd[1]: Started Ceph mds.f for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:34:22.267 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-mds[84658]: -- 172.21.15.155:0/2223815433 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c6356781a0 con 0x55c63564f000 2024-03-18T20:34:22.267 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 18 20:34:21 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-f[84635]: starting mds.f at 2024-03-18T20:34:22.401 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:34:22.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:22 smithi155 ceph-mon[48344]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:22.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:22 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:22.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:22 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:22.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:22 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:22.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:22 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:22.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:22 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:22.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:22 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:22 smithi046 ceph-mon[43495]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:22 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:22.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:22 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:22.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:22 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:22.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:22 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:22.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:22 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:22.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:22 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.545+0000 7f97082f6700 1 -- 172.21.15.46:0/3272430112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700100d50 msgr2=0x7f97001011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.545+0000 7f97082f6700 1 --2- 172.21.15.46:0/3272430112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700100d50 0x7f97001011b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f96f00099c0 tx=0x7f96f0009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.545+0000 7f97082f6700 1 -- 172.21.15.46:0/3272430112 shutdown_connections 2024-03-18T20:34:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.545+0000 7f97082f6700 1 --2- 172.21.15.46:0/3272430112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97001016f0 0x7f9700107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:22.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.545+0000 7f97082f6700 1 --2- 172.21.15.46:0/3272430112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97000ffb60 0x7f97000fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.545+0000 7f97082f6700 1 --2- 172.21.15.46:0/3272430112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700100d50 0x7f97001011b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.545+0000 7f97082f6700 1 -- 172.21.15.46:0/3272430112 >> 172.21.15.46:0/3272430112 conn(0x7f97000fb0d0 msgr2=0x7f97000fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.545+0000 7f97082f6700 1 -- 172.21.15.46:0/3272430112 shutdown_connections 2024-03-18T20:34:22.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.545+0000 7f97082f6700 1 -- 172.21.15.46:0/3272430112 wait complete. 2024-03-18T20:34:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.545+0000 7f97082f6700 1 Processor -- start 2024-03-18T20:34:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.545+0000 7f97082f6700 1 -- start start 2024-03-18T20:34:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f97082f6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97000ffb60 0x7f970010a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f97082f6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9700100d50 0x7f9700108ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:22.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f97082f6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97001016f0 0x7f9700108ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f97082f6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9700109790 con 0x7f97000ffb60 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f97082f6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9700109900 con 0x7f9700100d50 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f97082f6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97001a0c50 con 0x7f97001016f0 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f9706092700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97000ffb60 0x7f970010a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f9706092700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97000ffb60 0x7f970010a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60844/0 (socket says 172.21.15.46:60844) 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f9706092700 1 -- 172.21.15.46:0/3446403612 learned_addr learned my addr 172.21.15.46:0/3446403612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f9705891700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9700100d50 0x7f9700108ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f9706893700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97001016f0 0x7f9700108ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f9706092700 1 -- 172.21.15.46:0/3446403612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97001016f0 msgr2=0x7f9700108ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f9706092700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97001016f0 0x7f9700108ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f9706092700 1 -- 172.21.15.46:0/3446403612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9700100d50 msgr2=0x7f9700108ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f9706092700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9700100d50 0x7f9700108ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:22.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f9706092700 1 -- 172.21.15.46:0/3446403612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f00095f0 con 0x7f97000ffb60 2024-03-18T20:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f9706092700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97000ffb60 0x7f970010a420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00e970 tx=0x7f96fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f96f77fe700 1 -- 172.21.15.46:0/3446403612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc009790 con 0x7f97000ffb60 2024-03-18T20:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f96f77fe700 1 -- 172.21.15.46:0/3446403612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96fc00c780 con 0x7f97000ffb60 2024-03-18T20:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f97082f6700 1 -- 172.21.15.46:0/3446403612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97001a0ee0 con 0x7f97000ffb60 2024-03-18T20:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f96f77fe700 1 -- 172.21.15.46:0/3446403612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc003be0 con 0x7f97000ffb60 2024-03-18T20:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.546+0000 7f97082f6700 1 -- 172.21.15.46:0/3446403612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97001a1300 con 0x7f97000ffb60 2024-03-18T20:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.548+0000 7f97082f6700 1 -- 172.21.15.46:0/3446403612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9700107030 con 0x7f97000ffb60 2024-03-18T20:34:22.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.549+0000 7f96f77fe700 1 -- 172.21.15.46:0/3446403612 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f96fc00cbf0 con 0x7f97000ffb60 2024-03-18T20:34:22.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.549+0000 7f96f77fe700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f96ec0a1120 0x7f96ec0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:22.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.550+0000 7f96f77fe700 1 -- 172.21.15.46:0/3446403612 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f96fc014070 con 0x7f97000ffb60 2024-03-18T20:34:22.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.550+0000 7f9705891700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f96ec0a1120 0x7f96ec0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:22.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.550+0000 7f9705891700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f96ec0a1120 0x7f96ec0a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f96f0006010 tx=0x7f96f000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:22.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.553+0000 7f96f77fe700 1 -- 172.21.15.46:0/3446403612 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f96fc096310 con 0x7f97000ffb60 2024-03-18T20:34:22.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:22 smithi073 ceph-mon[47384]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:22.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:22.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:22.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:22.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:22.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:22.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:22 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:22.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.821+0000 7f97082f6700 1 -- 172.21.15.46:0/3446403612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9700067a10 con 0x7f97000ffb60 2024-03-18T20:34:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.822+0000 7f96f77fe700 1 -- 172.21.15.46:0/3446403612 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5042 (secure 0 0 0) 0x7f96fc099930 con 0x7f97000ffb60 2024-03-18T20:34:22.825 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:22.825 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2024-03-18T20:34:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.824+0000 7f97082f6700 1 -- 172.21.15.46:0/3446403612 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f96ec0a1120 msgr2=0x7f96ec0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.824+0000 7f97082f6700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f96ec0a1120 0x7f96ec0a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f96f0006010 tx=0x7f96f000b540 comp rx=0 tx=0).stop 2024-03-18T20:34:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.824+0000 7f97082f6700 1 -- 172.21.15.46:0/3446403612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97000ffb60 msgr2=0x7f970010a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.824+0000 7f97082f6700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97000ffb60 0x7f970010a420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00e970 tx=0x7f96fc00ec80 comp rx=0 tx=0).stop 2024-03-18T20:34:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.824+0000 7f97082f6700 1 -- 172.21.15.46:0/3446403612 shutdown_connections 2024-03-18T20:34:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.824+0000 7f97082f6700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f96ec0a1120 0x7f96ec0a35e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.824+0000 7f97082f6700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97000ffb60 0x7f970010a420 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.824+0000 7f97082f6700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97001016f0 0x7f9700108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.824+0000 7f97082f6700 1 --2- 172.21.15.46:0/3446403612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9700100d50 0x7f9700108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.824+0000 7f97082f6700 1 -- 172.21.15.46:0/3446403612 >> 172.21.15.46:0/3446403612 conn(0x7f97000fb0d0 msgr2=0x7f9700104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.824+0000 7f97082f6700 1 -- 172.21.15.46:0/3446403612 shutdown_connections 2024-03-18T20:34:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:22.824+0000 7f97082f6700 1 -- 172.21.15.46:0/3446403612 wait complete. 2024-03-18T20:34:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 7 2024-03-18T20:34:23.453 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:34:23 smithi155 systemd[1]: Starting Ceph mds.c for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:34:23.454 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:23 smithi155 ceph-mon[48344]: Deploying daemon mds.c on smithi155 2024-03-18T20:34:23.454 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:23 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6834/2573112997,v1:172.21.15.155:6835/2573112997] up:boot 2024-03-18T20:34:23.454 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:23 smithi155 ceph-mon[48344]: fsmap 6 up:standby 2024-03-18T20:34:23.454 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:23 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T20:34:23.454 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:23 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3446403612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:23 smithi046 ceph-mon[43495]: Deploying daemon mds.c on smithi155 2024-03-18T20:34:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:23 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6834/2573112997,v1:172.21.15.155:6835/2573112997] up:boot 2024-03-18T20:34:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:23 smithi046 ceph-mon[43495]: fsmap 6 up:standby 2024-03-18T20:34:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:23 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T20:34:23.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:23 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3446403612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:23.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:23 smithi073 ceph-mon[47384]: Deploying daemon mds.c on smithi155 2024-03-18T20:34:23.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:23 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6834/2573112997,v1:172.21.15.155:6835/2573112997] up:boot 2024-03-18T20:34:23.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:23 smithi073 ceph-mon[47384]: fsmap 6 up:standby 2024-03-18T20:34:23.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:23 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T20:34:23.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:23 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3446403612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:24.040 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:34:23 smithi155 ceph-mds[84998]: -- 172.21.15.155:0/2725477423 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562a2a1761a0 con 0x562a2a14d000 2024-03-18T20:34:24.041 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:34:23 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: starting mds.c at 2024-03-18T20:34:24.041 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:34:23 smithi155 bash[84960]: f86207f0aa4ae370899a28d7a1a7eebac3355fdb51bd1802ffd8af3cf67b3a5b 2024-03-18T20:34:24.041 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:34:23 smithi155 systemd[1]: Started Ceph mds.c for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:34:24.398 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:24 smithi073 ceph-mon[47384]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:24.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:24.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:24.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:24.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:24.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:24.399 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:24 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:24 smithi155 ceph-mon[48344]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:24 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:24 smithi046 ceph-mon[43495]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:24 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:24.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:34:24.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.983+0000 7fdfc9062700 1 -- 172.21.15.46:0/1087175792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfc4101720 msgr2=0x7fdfc4107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:24.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.983+0000 7fdfc9062700 1 --2- 172.21.15.46:0/1087175792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfc4101720 0x7fdfc4107fd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00b0d0 tx=0x7fdfbc00b3e0 comp rx=0 tx=0).stop 2024-03-18T20:34:24.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.984+0000 7fdfc9062700 1 -- 172.21.15.46:0/1087175792 shutdown_connections 2024-03-18T20:34:24.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.984+0000 7fdfc9062700 1 --2- 172.21.15.46:0/1087175792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfc4101720 0x7fdfc4107fd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:24.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.984+0000 7fdfc9062700 1 --2- 172.21.15.46:0/1087175792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfc40ffb90 0x7fdfc40fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:24.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.984+0000 7fdfc9062700 1 --2- 172.21.15.46:0/1087175792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfc4100d80 0x7fdfc41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:24.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.984+0000 7fdfc9062700 1 -- 172.21.15.46:0/1087175792 >> 172.21.15.46:0/1087175792 conn(0x7fdfc40fb100 msgr2=0x7fdfc40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:24.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.984+0000 7fdfc9062700 1 -- 172.21.15.46:0/1087175792 shutdown_connections 2024-03-18T20:34:24.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.984+0000 7fdfc9062700 1 -- 172.21.15.46:0/1087175792 wait complete. 2024-03-18T20:34:24.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.984+0000 7fdfc9062700 1 Processor -- start 2024-03-18T20:34:24.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfc9062700 1 -- start start 2024-03-18T20:34:24.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfc9062700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfc40ffb90 0x7fdfc4103a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:24.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfc9062700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfc4100d80 0x7fdfc4102160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfc9062700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfc4101720 0x7fdfc41026a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfc9062700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc4102da0 con 0x7fdfc4100d80 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfc9062700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc4102f10 con 0x7fdfc40ffb90 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfc9062700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc41a0d40 con 0x7fdfc4101720 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfba59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfc4100d80 0x7fdfc4102160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfba59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfc4100d80 0x7fdfc4102160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60866/0 (socket says 172.21.15.46:60866) 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfba59c700 1 -- 172.21.15.46:0/2337801553 learned_addr learned my addr 172.21.15.46:0/2337801553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfc2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfc40ffb90 0x7fdfc4103a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfc359e700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfc4101720 0x7fdfc41026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfba59c700 1 -- 172.21.15.46:0/2337801553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfc4101720 msgr2=0x7fdfc41026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfba59c700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfc4101720 0x7fdfc41026a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfba59c700 1 -- 172.21.15.46:0/2337801553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfc40ffb90 msgr2=0x7fdfc4103a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfba59c700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfc40ffb90 0x7fdfc4103a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfba59c700 1 -- 172.21.15.46:0/2337801553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb4009520 con 0x7fdfc4100d80 2024-03-18T20:34:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.985+0000 7fdfc2d9d700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfc40ffb90 0x7fdfc4103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.986+0000 7fdfba59c700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfc4100d80 0x7fdfc4102160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00d950 tx=0x7fdfac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.986+0000 7fdfc0d99700 1 -- 172.21.15.46:0/2337801553 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00f890 con 0x7fdfc4100d80 2024-03-18T20:34:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.986+0000 7fdfc0d99700 1 -- 172.21.15.46:0/2337801553 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfac010460 con 0x7fdfc4100d80 2024-03-18T20:34:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.986+0000 7fdfc9062700 1 -- 172.21.15.46:0/2337801553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfbc009d00 con 0x7fdfc4100d80 2024-03-18T20:34:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.986+0000 7fdfc0d99700 1 -- 172.21.15.46:0/2337801553 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00e610 con 0x7fdfc4100d80 2024-03-18T20:34:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.986+0000 7fdfc9062700 1 -- 172.21.15.46:0/2337801553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc41a1290 con 0x7fdfc4100d80 2024-03-18T20:34:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.987+0000 7fdfc9062700 1 -- 172.21.15.46:0/2337801553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc4067180 con 0x7fdfc4100d80 2024-03-18T20:34:24.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.989+0000 7fdfc0d99700 1 -- 172.21.15.46:0/2337801553 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fdfac0108d0 con 0x7fdfc4100d80 2024-03-18T20:34:24.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.989+0000 7fdfc0d99700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdfb00a1120 0x7fdfb00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:24.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.990+0000 7fdfc0d99700 1 -- 172.21.15.46:0/2337801553 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdfac0cc5f0 con 0x7fdfc4100d80 2024-03-18T20:34:24.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.990+0000 7fdfc2d9d700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdfb00a1120 0x7fdfb00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:24.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.990+0000 7fdfc2d9d700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdfb00a1120 0x7fdfb00a35e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009fd0 tx=0x7fdfb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:24.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:24.993+0000 7fdfc0d99700 1 -- 172.21.15.46:0/2337801553 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdfac096180 con 0x7fdfc4100d80 2024-03-18T20:34:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.256+0000 7fdfc9062700 1 -- 172.21.15.46:0/2337801553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfc41a1480 con 0x7fdfc4100d80 2024-03-18T20:34:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.256+0000 7fdfc0d99700 1 -- 172.21.15.46:0/2337801553 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5802 (secure 0 0 0) 0x7fdfac0997a0 con 0x7fdfc4100d80 2024-03-18T20:34:25.260 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:25.261 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-03-18T20:34:25.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.258+0000 7fdfc9062700 1 -- 172.21.15.46:0/2337801553 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdfb00a1120 msgr2=0x7fdfb00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:25.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.258+0000 7fdfc9062700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdfb00a1120 0x7fdfb00a35e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009fd0 tx=0x7fdfb4009260 comp rx=0 tx=0).stop 2024-03-18T20:34:25.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.258+0000 7fdfc9062700 1 -- 172.21.15.46:0/2337801553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfc4100d80 msgr2=0x7fdfc4102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:25.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.258+0000 7fdfc9062700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfc4100d80 0x7fdfc4102160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00d950 tx=0x7fdfac00dc60 comp rx=0 tx=0).stop 2024-03-18T20:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.259+0000 7fdfc9062700 1 -- 172.21.15.46:0/2337801553 shutdown_connections 2024-03-18T20:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.259+0000 7fdfc9062700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdfb00a1120 0x7fdfb00a35e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.259+0000 7fdfc9062700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfc4101720 0x7fdfc41026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.259+0000 7fdfc9062700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfc40ffb90 0x7fdfc4103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.259+0000 7fdfc9062700 1 --2- 172.21.15.46:0/2337801553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfc4100d80 0x7fdfc4102160 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.259+0000 7fdfc9062700 1 -- 172.21.15.46:0/2337801553 >> 172.21.15.46:0/2337801553 conn(0x7fdfc40fb100 msgr2=0x7fdfc40fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.259+0000 7fdfc9062700 1 -- 172.21.15.46:0/2337801553 shutdown_connections 2024-03-18T20:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:25.259+0000 7fdfc9062700 1 -- 172.21.15.46:0/2337801553 wait complete. 2024-03-18T20:34:25.263 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-03-18T20:34:25.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:25 smithi155 ceph-mon[48344]: Deploying daemon mds.h on smithi073 2024-03-18T20:34:25.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:25 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6836/1686098166,v1:172.21.15.155:6837/1686098166] up:boot 2024-03-18T20:34:25.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:25 smithi155 ceph-mon[48344]: fsmap 7 up:standby 2024-03-18T20:34:25.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:25 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T20:34:25.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:25 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2337801553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:25.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:25 smithi046 ceph-mon[43495]: Deploying daemon mds.h on smithi073 2024-03-18T20:34:25.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:25 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6836/1686098166,v1:172.21.15.155:6837/1686098166] up:boot 2024-03-18T20:34:25.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:25 smithi046 ceph-mon[43495]: fsmap 7 up:standby 2024-03-18T20:34:25.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:25 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T20:34:25.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:25 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2337801553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:25.720 INFO:journalctl@ceph.mds.h.smithi073.stdout:Mar 18 20:34:25 smithi073 bash[83455]: ea7390eda069b61c206b8b0e6c184c06cd6124709c42b1627ffa23653288db04 2024-03-18T20:34:25.720 INFO:journalctl@ceph.mds.h.smithi073.stdout:Mar 18 20:34:25 smithi073 systemd[1]: Started Ceph mds.h for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:34:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:25 smithi073 ceph-mon[47384]: Deploying daemon mds.h on smithi073 2024-03-18T20:34:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:25 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6836/1686098166,v1:172.21.15.155:6837/1686098166] up:boot 2024-03-18T20:34:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:25 smithi073 ceph-mon[47384]: fsmap 7 up:standby 2024-03-18T20:34:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:25 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T20:34:25.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:25 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2337801553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:26.422 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:26 smithi155 ceph-mon[48344]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:26.422 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:26.422 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:26.422 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:26.422 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:26.422 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:26.422 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:26.423 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:26 smithi155 ceph-mon[48344]: Deploying daemon mds.i on smithi155 2024-03-18T20:34:26.423 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:26 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] up:boot 2024-03-18T20:34:26.423 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:26 smithi155 ceph-mon[48344]: fsmap 8 up:standby 2024-03-18T20:34:26.423 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:26 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T20:34:26.672 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 20:34:26 smithi155 systemd[1]: Starting Ceph mds.i for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:34:26.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:26 smithi073 ceph-mon[47384]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:26.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:26.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:26.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:26.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:26.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:26.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:26.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:26 smithi073 ceph-mon[47384]: Deploying daemon mds.i on smithi155 2024-03-18T20:34:26.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:26 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] up:boot 2024-03-18T20:34:26.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:26 smithi073 ceph-mon[47384]: fsmap 8 up:standby 2024-03-18T20:34:26.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:26 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T20:34:26.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:26 smithi046 ceph-mon[43495]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:26.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:26.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:26.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:26.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:26.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:26.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:26.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:26 smithi046 ceph-mon[43495]: Deploying daemon mds.i on smithi155 2024-03-18T20:34:26.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:26 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] up:boot 2024-03-18T20:34:26.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:26 smithi046 ceph-mon[43495]: fsmap 8 up:standby 2024-03-18T20:34:26.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:26 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T20:34:27.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:34:27.291 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 20:34:27 smithi155 ceph-mds[85341]: -- 172.21.15.155:0/1266934983 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5561ff4381a0 con 0x5561ff40ec00 2024-03-18T20:34:27.291 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 20:34:27 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-i[85318]: starting mds.i at 2024-03-18T20:34:27.291 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 20:34:27 smithi155 bash[85303]: 00e981ff7207bafdfa5b710d5fbbb96919023188358398abb00cccc4f86165f6 2024-03-18T20:34:27.291 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 20:34:27 smithi155 systemd[1]: Started Ceph mds.i for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:34:27.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.418+0000 7fd671f20700 1 -- 172.21.15.46:0/3192970247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd66c1007f0 msgr2=0x7fd66c1080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.418+0000 7fd671f20700 1 --2- 172.21.15.46:0/3192970247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd66c1007f0 0x7fd66c1080b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd66400b0a0 tx=0x7fd66400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.418+0000 7fd671f20700 1 -- 172.21.15.46:0/3192970247 shutdown_connections 2024-03-18T20:34:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.418+0000 7fd671f20700 1 --2- 172.21.15.46:0/3192970247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd66c1007f0 0x7fd66c1080b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.418+0000 7fd671f20700 1 --2- 172.21.15.46:0/3192970247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd66c101120 0x7fd66c101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.418+0000 7fd671f20700 1 --2- 172.21.15.46:0/3192970247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd66c0ffe50 0x7fd66c1002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.418+0000 7fd671f20700 1 -- 172.21.15.46:0/3192970247 >> 172.21.15.46:0/3192970247 conn(0x7fd66c0fa880 msgr2=0x7fd66c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.418+0000 7fd671f20700 1 -- 172.21.15.46:0/3192970247 shutdown_connections 2024-03-18T20:34:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.418+0000 7fd671f20700 1 -- 172.21.15.46:0/3192970247 wait complete. 2024-03-18T20:34:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.419+0000 7fd671f20700 1 Processor -- start 2024-03-18T20:34:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.419+0000 7fd671f20700 1 -- start start 2024-03-18T20:34:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.419+0000 7fd671f20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd66c0ffe50 0x7fd66c195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.419+0000 7fd671f20700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd66c1007f0 0x7fd66c1960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.419+0000 7fd671f20700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd66c101120 0x7fd66c19d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.419+0000 7fd671f20700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c19d6c0 con 0x7fd66c0ffe50 2024-03-18T20:34:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.419+0000 7fd671f20700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c19d800 con 0x7fd66c1007f0 2024-03-18T20:34:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.419+0000 7fd671f20700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c19d990 con 0x7fd66c101120 2024-03-18T20:34:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.419+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd66c0ffe50 0x7fd66c195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd66c0ffe50 0x7fd66c195b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58308/0 (socket says 172.21.15.46:58308) 2024-03-18T20:34:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd66b7fe700 1 -- 172.21.15.46:0/3623957266 learned_addr learned my addr 172.21.15.46:0/3623957266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd66bfff700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd66c101120 0x7fd66c19d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd66affd700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd66c1007f0 0x7fd66c1960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd66b7fe700 1 -- 172.21.15.46:0/3623957266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd66c101120 msgr2=0x7fd66c19d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd66b7fe700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd66c101120 0x7fd66c19d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd66b7fe700 1 -- 172.21.15.46:0/3623957266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd66c1007f0 msgr2=0x7fd66c1960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd66b7fe700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd66c1007f0 0x7fd66c1960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd66b7fe700 1 -- 172.21.15.46:0/3623957266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd664009d00 con 0x7fd66c0ffe50 2024-03-18T20:34:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd66affd700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd66c1007f0 0x7fd66c1960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:34:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd66bfff700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd66c101120 0x7fd66c19d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd66b7fe700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd66c0ffe50 0x7fd66c195b90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00e970 tx=0x7fd65c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd668ff9700 1 -- 172.21.15.46:0/3623957266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c009790 con 0x7fd66c0ffe50 2024-03-18T20:34:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd668ff9700 1 -- 172.21.15.46:0/3623957266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd65c00eec0 con 0x7fd66c0ffe50 2024-03-18T20:34:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd671f20700 1 -- 172.21.15.46:0/3623957266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66c19dc70 con 0x7fd66c0ffe50 2024-03-18T20:34:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd668ff9700 1 -- 172.21.15.46:0/3623957266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c003cf0 con 0x7fd66c0ffe50 2024-03-18T20:34:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.420+0000 7fd671f20700 1 -- 172.21.15.46:0/3623957266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66c19e180 con 0x7fd66c0ffe50 2024-03-18T20:34:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.422+0000 7fd671f20700 1 -- 172.21.15.46:0/3623957266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66c107020 con 0x7fd66c0ffe50 2024-03-18T20:34:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.423+0000 7fd668ff9700 1 -- 172.21.15.46:0/3623957266 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fd65c00ccd0 con 0x7fd66c0ffe50 2024-03-18T20:34:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.423+0000 7fd668ff9700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fd6540a1120 0x7fd6540a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.424+0000 7fd66affd700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fd6540a1120 0x7fd6540a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.424+0000 7fd668ff9700 1 -- 172.21.15.46:0/3623957266 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd65c014070 con 0x7fd66c0ffe50 2024-03-18T20:34:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.424+0000 7fd66affd700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fd6540a1120 0x7fd6540a35e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd660005950 tx=0x7fd6600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.428+0000 7fd668ff9700 1 -- 172.21.15.46:0/3623957266 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd65c0962e0 con 0x7fd66c0ffe50 2024-03-18T20:34:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.708+0000 7fd671f20700 1 -- 172.21.15.46:0/3623957266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd66c02cf30 con 0x7fd66c0ffe50 2024-03-18T20:34:27.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.709+0000 7fd668ff9700 1 -- 172.21.15.46:0/3623957266 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7321 (secure 0 0 0) 0x7fd65c099900 con 0x7fd66c0ffe50 2024-03-18T20:34:27.712 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:27.712 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-03-18T20:34:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.711+0000 7fd671f20700 1 -- 172.21.15.46:0/3623957266 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fd6540a1120 msgr2=0x7fd6540a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.711+0000 7fd671f20700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fd6540a1120 0x7fd6540a35e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd660005950 tx=0x7fd6600058e0 comp rx=0 tx=0).stop 2024-03-18T20:34:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.711+0000 7fd671f20700 1 -- 172.21.15.46:0/3623957266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd66c0ffe50 msgr2=0x7fd66c195b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.711+0000 7fd671f20700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd66c0ffe50 0x7fd66c195b90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00e970 tx=0x7fd65c00ec80 comp rx=0 tx=0).stop 2024-03-18T20:34:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.711+0000 7fd671f20700 1 -- 172.21.15.46:0/3623957266 shutdown_connections 2024-03-18T20:34:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.711+0000 7fd671f20700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fd6540a1120 0x7fd6540a35e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.711+0000 7fd671f20700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd66c0ffe50 0x7fd66c195b90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.711+0000 7fd671f20700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd66c101120 0x7fd66c19d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.711+0000 7fd671f20700 1 --2- 172.21.15.46:0/3623957266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd66c1007f0 0x7fd66c1960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.711+0000 7fd671f20700 1 -- 172.21.15.46:0/3623957266 >> 172.21.15.46:0/3623957266 conn(0x7fd66c0fa880 msgr2=0x7fd66c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.711+0000 7fd671f20700 1 -- 172.21.15.46:0/3623957266 shutdown_connections 2024-03-18T20:34:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:27.711+0000 7fd671f20700 1 -- 172.21.15.46:0/3623957266 wait complete. 2024-03-18T20:34:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 10 2024-03-18T20:34:28.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:28 smithi046 ceph-mon[43495]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:28.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:28 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:28.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:28 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:28.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:28 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:28.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:28 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:28.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:28 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:28.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:28 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:28.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:28 smithi046 ceph-mon[43495]: Deploying daemon mds.g on smithi046 2024-03-18T20:34:28.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:28 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] up:boot 2024-03-18T20:34:28.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:28 smithi046 ceph-mon[43495]: fsmap 9 up:standby 2024-03-18T20:34:28.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:28 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T20:34:28.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:28 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3623957266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:28.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:28 smithi073 ceph-mon[47384]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:28.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:28 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:28.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:28 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:28.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:28 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:28.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:28 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:28.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:28 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:28.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:28 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:28.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:28 smithi073 ceph-mon[47384]: Deploying daemon mds.g on smithi046 2024-03-18T20:34:28.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:28 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] up:boot 2024-03-18T20:34:28.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:28 smithi073 ceph-mon[47384]: fsmap 9 up:standby 2024-03-18T20:34:28.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:28 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T20:34:28.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:28 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3623957266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:28.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:28 smithi155 ceph-mon[48344]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:28.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:28 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:28.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:28 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:28.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:28 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:28.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:28 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:28.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:28 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:28.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:28 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:28.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:28 smithi155 ceph-mon[48344]: Deploying daemon mds.g on smithi046 2024-03-18T20:34:28.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:28 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] up:boot 2024-03-18T20:34:28.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:28 smithi155 ceph-mon[48344]: fsmap 9 up:standby 2024-03-18T20:34:28.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:28 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T20:34:28.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:28 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3623957266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:29.026 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 20:34:28 smithi046 systemd[1]: Starting Ceph mds.g for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:34:29.546 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 20:34:29 smithi046 ceph-mds[93432]: -- 172.21.15.46:0/4291196612 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55902b2761a0 con 0x55902b24d000 2024-03-18T20:34:29.547 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 20:34:29 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-g[93409]: starting mds.g at 2024-03-18T20:34:29.547 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 20:34:29 smithi046 bash[93394]: ba4fdb9479a016270590d7c71dc6adc23d81daa439272260b31aff9edf973146 2024-03-18T20:34:29.547 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 20:34:29 smithi046 systemd[1]: Started Ceph mds.g for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:34:29.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:34:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.878+0000 7f3a4f95a700 1 -- 172.21.15.46:0/800158029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a480ffeb0 msgr2=0x7f3a48100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.878+0000 7f3a4f95a700 1 --2- 172.21.15.46:0/800158029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a480ffeb0 0x7f3a48100330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009990 tx=0x7f3a3c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:34:29.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.878+0000 7f3a4f95a700 1 -- 172.21.15.46:0/800158029 shutdown_connections 2024-03-18T20:34:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.878+0000 7f3a4f95a700 1 --2- 172.21.15.46:0/800158029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a48100870 0x7f3a48100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.878+0000 7f3a4f95a700 1 --2- 172.21.15.46:0/800158029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a480ff550 0x7f3a480ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.878+0000 7f3a4f95a700 1 --2- 172.21.15.46:0/800158029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a480ffeb0 0x7f3a48100330 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.878+0000 7f3a4f95a700 1 -- 172.21.15.46:0/800158029 >> 172.21.15.46:0/800158029 conn(0x7f3a480fb0d0 msgr2=0x7f3a480fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.878+0000 7f3a4f95a700 1 -- 172.21.15.46:0/800158029 shutdown_connections 2024-03-18T20:34:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.878+0000 7f3a4f95a700 1 -- 172.21.15.46:0/800158029 wait complete. 2024-03-18T20:34:29.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.879+0000 7f3a4f95a700 1 Processor -- start 2024-03-18T20:34:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.879+0000 7f3a4f95a700 1 -- start start 2024-03-18T20:34:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.879+0000 7f3a4f95a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a480ff550 0x7f3a48198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.879+0000 7f3a4f95a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a480ffeb0 0x7f3a48199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.879+0000 7f3a4f95a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a48100870 0x7f3a481a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.879+0000 7f3a4f95a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a481a07f0 con 0x7f3a480ffeb0 2024-03-18T20:34:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.879+0000 7f3a4f95a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a481a0960 con 0x7f3a48100870 2024-03-18T20:34:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.879+0000 7f3a4f95a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a481a0b20 con 0x7f3a480ff550 2024-03-18T20:34:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.879+0000 7f3a4cef5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a480ffeb0 0x7f3a48199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a4cef5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a480ffeb0 0x7f3a48199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58354/0 (socket says 172.21.15.46:58354) 2024-03-18T20:34:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a4cef5700 1 -- 172.21.15.46:0/1189453443 learned_addr learned my addr 172.21.15.46:0/1189453443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a4d6f6700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a480ff550 0x7f3a48198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a4def7700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a48100870 0x7f3a481a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a4cef5700 1 -- 172.21.15.46:0/1189453443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a480ff550 msgr2=0x7f3a48198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a4cef5700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a480ff550 0x7f3a48198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a4cef5700 1 -- 172.21.15.46:0/1189453443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a48100870 msgr2=0x7f3a481a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a4cef5700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a48100870 0x7f3a481a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a4cef5700 1 -- 172.21.15.46:0/1189453443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c0095f0 con 0x7f3a480ffeb0 2024-03-18T20:34:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a4cef5700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a480ffeb0 0x7f3a48199210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00b5c0 tx=0x7f3a3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:29.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a467fc700 1 -- 172.21.15.46:0/1189453443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c01d070 con 0x7f3a480ffeb0 2024-03-18T20:34:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a467fc700 1 -- 172.21.15.46:0/1189453443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a3c003f60 con 0x7f3a480ffeb0 2024-03-18T20:34:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a467fc700 1 -- 172.21.15.46:0/1189453443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c022d10 con 0x7f3a480ffeb0 2024-03-18T20:34:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a4f95a700 1 -- 172.21.15.46:0/1189453443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a481a0da0 con 0x7f3a480ffeb0 2024-03-18T20:34:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.880+0000 7f3a4f95a700 1 -- 172.21.15.46:0/1189453443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a481a11c0 con 0x7f3a480ffeb0 2024-03-18T20:34:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.881+0000 7f3a4f95a700 1 -- 172.21.15.46:0/1189453443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4810a4f0 con 0x7f3a480ffeb0 2024-03-18T20:34:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.882+0000 7f3a467fc700 1 -- 172.21.15.46:0/1189453443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f3a3c022470 con 0x7f3a480ffeb0 2024-03-18T20:34:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.883+0000 7f3a467fc700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3a300a11a0 0x7f3a300a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.883+0000 7f3a467fc700 1 -- 172.21.15.46:0/1189453443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3a3c0cd2d0 con 0x7f3a480ffeb0 2024-03-18T20:34:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.883+0000 7f3a4d6f6700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3a300a11a0 0x7f3a300a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:29.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.884+0000 7f3a4d6f6700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3a300a11a0 0x7f3a300a3660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a48105be0 tx=0x7f3a4000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:29.888+0000 7f3a467fc700 1 -- 172.21.15.46:0/1189453443 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3a3c096f90 con 0x7f3a480ffeb0 2024-03-18T20:34:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.163+0000 7f3a4f95a700 1 -- 172.21.15.46:0/1189453443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a48067a10 con 0x7f3a480ffeb0 2024-03-18T20:34:30.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.164+0000 7f3a467fc700 1 -- 172.21.15.46:0/1189453443 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8079 (secure 0 0 0) 0x7f3a3c09a5b0 con 0x7f3a480ffeb0 2024-03-18T20:34:30.179 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:30.180 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24521,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[]} 2024-03-18T20:34:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.166+0000 7f3a4f95a700 1 -- 172.21.15.46:0/1189453443 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3a300a11a0 msgr2=0x7f3a300a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.166+0000 7f3a4f95a700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3a300a11a0 0x7f3a300a3660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a48105be0 tx=0x7f3a4000a380 comp rx=0 tx=0).stop 2024-03-18T20:34:30.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.166+0000 7f3a4f95a700 1 -- 172.21.15.46:0/1189453443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a480ffeb0 msgr2=0x7f3a48199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:30.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.166+0000 7f3a4f95a700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a480ffeb0 0x7f3a48199210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00b5c0 tx=0x7f3a3c004730 comp rx=0 tx=0).stop 2024-03-18T20:34:30.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.166+0000 7f3a4f95a700 1 -- 172.21.15.46:0/1189453443 shutdown_connections 2024-03-18T20:34:30.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.166+0000 7f3a4f95a700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3a300a11a0 0x7f3a300a3660 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:30.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.166+0000 7f3a4f95a700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a48100870 0x7f3a481a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:30.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.166+0000 7f3a4f95a700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3a480ff550 0x7f3a48198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:30.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.166+0000 7f3a4f95a700 1 --2- 172.21.15.46:0/1189453443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a480ffeb0 0x7f3a48199210 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.166+0000 7f3a4f95a700 1 -- 172.21.15.46:0/1189453443 >> 172.21.15.46:0/1189453443 conn(0x7f3a480fb0d0 msgr2=0x7f3a480fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.166+0000 7f3a4f95a700 1 -- 172.21.15.46:0/1189453443 shutdown_connections 2024-03-18T20:34:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:30.166+0000 7f3a4f95a700 1 -- 172.21.15.46:0/1189453443 wait complete. 2024-03-18T20:34:30.184 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 11 2024-03-18T20:34:30.295 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mon[43495]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:30.295 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mon[43495]: Deploying daemon mds.a on smithi046 2024-03-18T20:34:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] up:boot 2024-03-18T20:34:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mon[43495]: fsmap 10 up:standby 2024-03-18T20:34:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T20:34:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1189453443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:30 smithi073 ceph-mon[47384]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:30.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:30.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:30.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:30.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:30 smithi073 ceph-mon[47384]: Deploying daemon mds.a on smithi046 2024-03-18T20:34:30.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:30 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] up:boot 2024-03-18T20:34:30.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:30 smithi073 ceph-mon[47384]: fsmap 10 up:standby 2024-03-18T20:34:30.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:30 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T20:34:30.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:30 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1189453443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:30 smithi155 ceph-mon[48344]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:30 smithi155 ceph-mon[48344]: Deploying daemon mds.a on smithi046 2024-03-18T20:34:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:30 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] up:boot 2024-03-18T20:34:30.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:30 smithi155 ceph-mon[48344]: fsmap 10 up:standby 2024-03-18T20:34:30.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:30 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T20:34:30.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:30 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1189453443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:30.847 INFO:journalctl@ceph.mds.a.smithi046.stdout:Mar 18 20:34:30 smithi046 systemd[1]: Starting Ceph mds.a for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:34:31.296 INFO:journalctl@ceph.mds.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-mds[93822]: -- 172.21.15.46:0/3569147623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561d24d821a0 con 0x561d24d58800 2024-03-18T20:34:31.297 INFO:journalctl@ceph.mds.a.smithi046.stdout:Mar 18 20:34:30 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-a[93799]: starting mds.a at 2024-03-18T20:34:31.297 INFO:journalctl@ceph.mds.a.smithi046.stdout:Mar 18 20:34:30 smithi046 bash[93784]: d0396efc47ef9e275053ffb66502e197829f8c2cc4b02192b2f6aadfc64191d3 2024-03-18T20:34:31.297 INFO:journalctl@ceph.mds.a.smithi046.stdout:Mar 18 20:34:30 smithi046 systemd[1]: Started Ceph mds.a for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:34:32.187 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:34:32.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:32.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:32 smithi155 ceph-mon[48344]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:32.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:32.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:32.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:32.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:32.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:32.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:32 smithi155 ceph-mon[48344]: Deploying daemon mds.b on smithi073 2024-03-18T20:34:32.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:32 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.46:6840/3470614253,v1:172.21.15.46:6841/3470614253] up:boot 2024-03-18T20:34:32.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:32 smithi155 ceph-mon[48344]: fsmap 11 up:standby 2024-03-18T20:34:32.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:32 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T20:34:32.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:32.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:32 smithi046 ceph-mon[43495]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:32.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:32.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:32.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:32.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:32.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:32.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:32 smithi046 ceph-mon[43495]: Deploying daemon mds.b on smithi073 2024-03-18T20:34:32.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:32 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.46:6840/3470614253,v1:172.21.15.46:6841/3470614253] up:boot 2024-03-18T20:34:32.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:32 smithi046 ceph-mon[43495]: fsmap 11 up:standby 2024-03-18T20:34:32.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:32 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T20:34:32.321 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:32.321 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-mon[47384]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:32.321 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:32.321 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:32.321 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T20:34:32.321 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T20:34:32.321 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:32.321 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-mon[47384]: Deploying daemon mds.b on smithi073 2024-03-18T20:34:32.321 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.46:6840/3470614253,v1:172.21.15.46:6841/3470614253] up:boot 2024-03-18T20:34:32.322 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-mon[47384]: fsmap 11 up:standby 2024-03-18T20:34:32.322 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T20:34:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.336+0000 7f922a051700 1 -- 172.21.15.46:0/2088012977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9224106950 msgr2=0x7f9224108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.336+0000 7f922a051700 1 --2- 172.21.15.46:0/2088012977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9224106950 0x7f9224108dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f921c00b0a0 tx=0x7f921c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.336+0000 7f922a051700 1 -- 172.21.15.46:0/2088012977 shutdown_connections 2024-03-18T20:34:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.336+0000 7f922a051700 1 --2- 172.21.15.46:0/2088012977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9224106950 0x7f9224108dc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.336+0000 7f922a051700 1 --2- 172.21.15.46:0/2088012977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92241016f0 0x7f9224103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.336+0000 7f922a051700 1 --2- 172.21.15.46:0/2088012977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9224104020 0x7f9224106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.336+0000 7f922a051700 1 -- 172.21.15.46:0/2088012977 >> 172.21.15.46:0/2088012977 conn(0x7f92240fb100 msgr2=0x7f92240fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.336+0000 7f922a051700 1 -- 172.21.15.46:0/2088012977 shutdown_connections 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.336+0000 7f922a051700 1 -- 172.21.15.46:0/2088012977 wait complete. 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.337+0000 7f922a051700 1 Processor -- start 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.337+0000 7f922a051700 1 -- start start 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.337+0000 7f922a051700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92241016f0 0x7f9224196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.337+0000 7f922a051700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9224104020 0x7f9224197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.337+0000 7f922a051700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9224106950 0x7f922419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.337+0000 7f922a051700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922419e7b0 con 0x7f92241016f0 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.337+0000 7f922a051700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922419e920 con 0x7f9224106950 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.337+0000 7f922a051700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922419eae0 con 0x7f9224104020 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.337+0000 7f92237fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92241016f0 0x7f9224196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.337+0000 7f92237fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92241016f0 0x7f9224196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58394/0 (socket says 172.21.15.46:58394) 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.337+0000 7f92237fe700 1 -- 172.21.15.46:0/1551094933 learned_addr learned my addr 172.21.15.46:0/1551094933 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f9222ffd700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9224104020 0x7f9224197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f92237fe700 1 -- 172.21.15.46:0/1551094933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9224104020 msgr2=0x7f9224197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f92237fe700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9224104020 0x7f9224197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f92237fe700 1 -- 172.21.15.46:0/1551094933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9224106950 msgr2=0x7f922419e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f92237fe700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9224106950 0x7f922419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f92237fe700 1 -- 172.21.15.46:0/1551094933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c009d00 con 0x7f92241016f0 2024-03-18T20:34:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f9222ffd700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9224104020 0x7f9224197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:34:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f92237fe700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92241016f0 0x7f9224196cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f921400c9b0 tx=0x7f921400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f9220ff9700 1 -- 172.21.15.46:0/1551094933 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214007b00 con 0x7f92241016f0 2024-03-18T20:34:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f922a051700 1 -- 172.21.15.46:0/1551094933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922419edc0 con 0x7f92241016f0 2024-03-18T20:34:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f9220ff9700 1 -- 172.21.15.46:0/1551094933 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9214007c60 con 0x7f92241016f0 2024-03-18T20:34:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f9220ff9700 1 -- 172.21.15.46:0/1551094933 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214018660 con 0x7f92241016f0 2024-03-18T20:34:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.338+0000 7f922a051700 1 -- 172.21.15.46:0/1551094933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922419f290 con 0x7f92241016f0 2024-03-18T20:34:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.339+0000 7f922a051700 1 -- 172.21.15.46:0/1551094933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92240fcc80 con 0x7f92241016f0 2024-03-18T20:34:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.340+0000 7f9220ff9700 1 -- 172.21.15.46:0/1551094933 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f9214018880 con 0x7f92241016f0 2024-03-18T20:34:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.341+0000 7f9220ff9700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f920c0a1120 0x7f920c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.341+0000 7f9222ffd700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f920c0a1120 0x7f920c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.341+0000 7f9220ff9700 1 -- 172.21.15.46:0/1551094933 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f92140cc750 con 0x7f92241016f0 2024-03-18T20:34:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.342+0000 7f9222ffd700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f920c0a1120 0x7f920c0a35e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9218007900 tx=0x7f9218008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.345+0000 7f9220ff9700 1 -- 172.21.15.46:0/1551094933 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9214096410 con 0x7f92241016f0 2024-03-18T20:34:32.574 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 20:34:32 smithi073 systemd[1]: Starting Ceph mds.b for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T20:34:32.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.610+0000 7f922a051700 1 -- 172.21.15.46:0/1551094933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f922402cf30 con 0x7f92241016f0 2024-03-18T20:34:32.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.610+0000 7f9220ff9700 1 -- 172.21.15.46:0/1551094933 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8837 (secure 0 0 0) 0x7f9214099a30 con 0x7f92241016f0 2024-03-18T20:34:32.614 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:32.614 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24521,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[]} 2024-03-18T20:34:32.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.613+0000 7f922a051700 1 -- 172.21.15.46:0/1551094933 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f920c0a1120 msgr2=0x7f920c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:32.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.613+0000 7f922a051700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f920c0a1120 0x7f920c0a35e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9218007900 tx=0x7f9218008040 comp rx=0 tx=0).stop 2024-03-18T20:34:32.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.613+0000 7f922a051700 1 -- 172.21.15.46:0/1551094933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92241016f0 msgr2=0x7f9224196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:32.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.613+0000 7f922a051700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92241016f0 0x7f9224196cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f921400c9b0 tx=0x7f921400cd70 comp rx=0 tx=0).stop 2024-03-18T20:34:32.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.613+0000 7f922a051700 1 -- 172.21.15.46:0/1551094933 shutdown_connections 2024-03-18T20:34:32.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.613+0000 7f922a051700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f920c0a1120 0x7f920c0a35e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:32.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.613+0000 7f922a051700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92241016f0 0x7f9224196cc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:32.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.613+0000 7f922a051700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9224106950 0x7f922419e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:32.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.613+0000 7f922a051700 1 --2- 172.21.15.46:0/1551094933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9224104020 0x7f9224197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:32.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.613+0000 7f922a051700 1 -- 172.21.15.46:0/1551094933 >> 172.21.15.46:0/1551094933 conn(0x7f92240fb100 msgr2=0x7f9224102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:32.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.613+0000 7f922a051700 1 -- 172.21.15.46:0/1551094933 shutdown_connections 2024-03-18T20:34:32.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:32.613+0000 7f922a051700 1 -- 172.21.15.46:0/1551094933 wait complete. 2024-03-18T20:34:32.617 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 12 2024-03-18T20:34:33.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:33 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1551094933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:33.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:33 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:33.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:33 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:33.221 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-mds[83832]: -- 172.21.15.73:0/795070249 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b6c245e1a0 con 0x55b6c2435000 2024-03-18T20:34:33.221 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 20:34:32 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: starting mds.b at 2024-03-18T20:34:33.221 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 20:34:32 smithi073 bash[83794]: 5a752c3a7d3aefed5a55f1d6de05bd742a6dd92ddd848d71b2c62dfff5970535 2024-03-18T20:34:33.221 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 20:34:32 smithi073 systemd[1]: Started Ceph mds.b for 04a8ae18-e566-11ee-95c9-87774f69a715. 2024-03-18T20:34:33.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:33 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1551094933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:33.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:33 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:33.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:33 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:33.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:33 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1551094933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:33.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:33 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:33.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:33 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:34 smithi155 ceph-mon[48344]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:34 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] up:boot 2024-03-18T20:34:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:34 smithi155 ceph-mon[48344]: fsmap 12 up:standby 2024-03-18T20:34:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:34 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:34 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T20:34:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:34 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:34 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:34.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:34 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:34.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:34 smithi046 ceph-mon[43495]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:34.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:34 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] up:boot 2024-03-18T20:34:34.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:34 smithi046 ceph-mon[43495]: fsmap 12 up:standby 2024-03-18T20:34:34.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:34 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:34.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:34 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T20:34:34.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:34 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:34.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:34 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:34.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:34 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:34 smithi073 ceph-mon[47384]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:34.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:34 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] up:boot 2024-03-18T20:34:34.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:34 smithi073 ceph-mon[47384]: fsmap 12 up:standby 2024-03-18T20:34:34.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:34 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:34.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:34 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T20:34:34.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:34 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:34.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:34 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:34.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:34 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:34.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:34:34.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.780+0000 7ff151c80700 1 -- 172.21.15.46:0/1772242799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff14c101760 msgr2=0x7ff14c107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.780+0000 7ff151c80700 1 --2- 172.21.15.46:0/1772242799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff14c101760 0x7ff14c107ff0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff14400b0a0 tx=0x7ff14400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.780+0000 7ff151c80700 1 -- 172.21.15.46:0/1772242799 shutdown_connections 2024-03-18T20:34:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.780+0000 7ff151c80700 1 --2- 172.21.15.46:0/1772242799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff14c101760 0x7ff14c107ff0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.780+0000 7ff151c80700 1 --2- 172.21.15.46:0/1772242799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff14c100090 0x7ff14c1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.780+0000 7ff151c80700 1 --2- 172.21.15.46:0/1772242799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff14c100da0 0x7ff14c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.780+0000 7ff151c80700 1 -- 172.21.15.46:0/1772242799 >> 172.21.15.46:0/1772242799 conn(0x7ff14c0fa880 msgr2=0x7ff14c0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.780+0000 7ff151c80700 1 -- 172.21.15.46:0/1772242799 shutdown_connections 2024-03-18T20:34:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.780+0000 7ff151c80700 1 -- 172.21.15.46:0/1772242799 wait complete. 2024-03-18T20:34:34.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.781+0000 7ff151c80700 1 Processor -- start 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.781+0000 7ff151c80700 1 -- start start 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.781+0000 7ff151c80700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff14c100090 0x7ff14c0ff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.781+0000 7ff151c80700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff14c100da0 0x7ff14c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.781+0000 7ff151c80700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff14c101760 0x7ff14c1a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.781+0000 7ff151c80700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c1a0a20 con 0x7ff14c100090 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.781+0000 7ff151c80700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c1a0b90 con 0x7ff14c100da0 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.781+0000 7ff151c80700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c1a0d50 con 0x7ff14c101760 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.781+0000 7ff14b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff14c100090 0x7ff14c0ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.781+0000 7ff14b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff14c100090 0x7ff14c0ff700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58410/0 (socket says 172.21.15.46:58410) 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.781+0000 7ff14b7fe700 1 -- 172.21.15.46:0/2107338215 learned_addr learned my addr 172.21.15.46:0/2107338215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff14bfff700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff14c101760 0x7ff14c1a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff14affd700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff14c100da0 0x7ff14c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff14b7fe700 1 -- 172.21.15.46:0/2107338215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff14c101760 msgr2=0x7ff14c1a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff14b7fe700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff14c101760 0x7ff14c1a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff14b7fe700 1 -- 172.21.15.46:0/2107338215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff14c100da0 msgr2=0x7ff14c19de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff14b7fe700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff14c100da0 0x7ff14c19de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff14b7fe700 1 -- 172.21.15.46:0/2107338215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff144009d00 con 0x7ff14c100090 2024-03-18T20:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff14bfff700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff14c101760 0x7ff14c1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff14b7fe700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff14c100090 0x7ff14c0ff700 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00e970 tx=0x7ff13c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff148ff9700 1 -- 172.21.15.46:0/2107338215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c009790 con 0x7ff14c100090 2024-03-18T20:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff148ff9700 1 -- 172.21.15.46:0/2107338215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff13c00c7e0 con 0x7ff14c100090 2024-03-18T20:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff151c80700 1 -- 172.21.15.46:0/2107338215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14c1a0fd0 con 0x7ff14c100090 2024-03-18T20:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff148ff9700 1 -- 172.21.15.46:0/2107338215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c003c70 con 0x7ff14c100090 2024-03-18T20:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.782+0000 7ff151c80700 1 -- 172.21.15.46:0/2107338215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14c1a1450 con 0x7ff14c100090 2024-03-18T20:34:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.783+0000 7ff151c80700 1 -- 172.21.15.46:0/2107338215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff14c106f60 con 0x7ff14c100090 2024-03-18T20:34:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.785+0000 7ff148ff9700 1 -- 172.21.15.46:0/2107338215 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7ff13c00cc50 con 0x7ff14c100090 2024-03-18T20:34:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.786+0000 7ff148ff9700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff1340a1120 0x7ff1340a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.786+0000 7ff148ff9700 1 -- 172.21.15.46:0/2107338215 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff13c014070 con 0x7ff14c100090 2024-03-18T20:34:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.786+0000 7ff14affd700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff1340a1120 0x7ff1340a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:34.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.787+0000 7ff14affd700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff1340a1120 0x7ff1340a35e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0ffcc0 tx=0x7ff140007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:34.789+0000 7ff148ff9700 1 -- 172.21.15.46:0/2107338215 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff13c096350 con 0x7ff14c100090 2024-03-18T20:34:35.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.063+0000 7ff151c80700 1 -- 172.21.15.46:0/2107338215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff14c067180 con 0x7ff14c100090 2024-03-18T20:34:35.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.064+0000 7ff148ff9700 1 -- 172.21.15.46:0/2107338215 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9595 (secure 0 0 0) 0x7ff13c003dd0 con 0x7ff14c100090 2024-03-18T20:34:35.068 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:35.068 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24521,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-03-18T20:34:35.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.066+0000 7ff151c80700 1 -- 172.21.15.46:0/2107338215 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff1340a1120 msgr2=0x7ff1340a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:35.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.066+0000 7ff151c80700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff1340a1120 0x7ff1340a35e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0ffcc0 tx=0x7ff140007400 comp rx=0 tx=0).stop 2024-03-18T20:34:35.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.066+0000 7ff151c80700 1 -- 172.21.15.46:0/2107338215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff14c100090 msgr2=0x7ff14c0ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:35.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.066+0000 7ff151c80700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff14c100090 0x7ff14c0ff700 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00e970 tx=0x7ff13c00ec80 comp rx=0 tx=0).stop 2024-03-18T20:34:35.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.066+0000 7ff151c80700 1 -- 172.21.15.46:0/2107338215 shutdown_connections 2024-03-18T20:34:35.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.066+0000 7ff151c80700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff1340a1120 0x7ff1340a35e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.066+0000 7ff151c80700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff14c100090 0x7ff14c0ff700 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.066+0000 7ff151c80700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff14c101760 0x7ff14c1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.066+0000 7ff151c80700 1 --2- 172.21.15.46:0/2107338215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff14c100da0 0x7ff14c19de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.066+0000 7ff151c80700 1 -- 172.21.15.46:0/2107338215 >> 172.21.15.46:0/2107338215 conn(0x7ff14c0fa880 msgr2=0x7ff14c1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:35.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.067+0000 7ff151c80700 1 -- 172.21.15.46:0/2107338215 shutdown_connections 2024-03-18T20:34:35.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.067+0000 7ff151c80700 1 -- 172.21.15.46:0/2107338215 wait complete. 2024-03-18T20:34:35.072 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2024-03-18T20:34:35.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:34:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.229+0000 7fada1b6e700 1 -- 172.21.15.46:0/1211832670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad9c101750 msgr2=0x7fad9c103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.229+0000 7fada1b6e700 1 --2- 172.21.15.46:0/1211832670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad9c101750 0x7fad9c103b40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0098f0 tx=0x7fad8c009c00 comp rx=0 tx=0).stop 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.229+0000 7fada1b6e700 1 -- 172.21.15.46:0/1211832670 shutdown_connections 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.229+0000 7fada1b6e700 1 --2- 172.21.15.46:0/1211832670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9c1069b0 0x7fad9c108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.229+0000 7fada1b6e700 1 --2- 172.21.15.46:0/1211832670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad9c101750 0x7fad9c103b40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.229+0000 7fada1b6e700 1 --2- 172.21.15.46:0/1211832670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad9c104080 0x7fad9c106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.229+0000 7fada1b6e700 1 -- 172.21.15.46:0/1211832670 >> 172.21.15.46:0/1211832670 conn(0x7fad9c0fb100 msgr2=0x7fad9c0fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.229+0000 7fada1b6e700 1 -- 172.21.15.46:0/1211832670 shutdown_connections 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.229+0000 7fada1b6e700 1 -- 172.21.15.46:0/1211832670 wait complete. 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.229+0000 7fada1b6e700 1 Processor -- start 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fada1b6e700 1 -- start start 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fada1b6e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad9c101750 0x7fad9c101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fada1b6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9c104080 0x7fad9c0ff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fada1b6e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad9c1069b0 0x7fad9c0ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fada1b6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c1003e0 con 0x7fad9c104080 2024-03-18T20:34:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fada1b6e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c100550 con 0x7fad9c1069b0 2024-03-18T20:34:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fada1b6e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c1a0c00 con 0x7fad9c101750 2024-03-18T20:34:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9c104080 0x7fad9c0ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad9c1069b0 0x7fad9c0ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad9c1069b0 0x7fad9c0ffc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33072/0 (socket says 172.21.15.46:33072) 2024-03-18T20:34:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9c104080 0x7fad9c0ff700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58414/0 (socket says 172.21.15.46:58414) 2024-03-18T20:34:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9bfff700 1 -- 172.21.15.46:0/2660620975 learned_addr learned my addr 172.21.15.46:0/2660620975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9b7fe700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad9c101750 0x7fad9c101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9bfff700 1 -- 172.21.15.46:0/2660620975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad9c101750 msgr2=0x7fad9c101070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9bfff700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad9c101750 0x7fad9c101070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9bfff700 1 -- 172.21.15.46:0/2660620975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9c104080 msgr2=0x7fad9c0ff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9bfff700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9c104080 0x7fad9c0ff700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9bfff700 1 -- 172.21.15.46:0/2660620975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad94009d00 con 0x7fad9c1069b0 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9b7fe700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad9c101750 0x7fad9c101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.230+0000 7fad9affd700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9c104080 0x7fad9c0ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.231+0000 7fad9bfff700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad9c1069b0 0x7fad9c0ffc40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fad9400e3c0 tx=0x7fad9400e6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.231+0000 7fad98ff9700 1 -- 172.21.15.46:0/2660620975 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad940090d0 con 0x7fad9c1069b0 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.231+0000 7fad98ff9700 1 -- 172.21.15.46:0/2660620975 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad9400f040 con 0x7fad9c1069b0 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.231+0000 7fada1b6e700 1 -- 172.21.15.46:0/2660620975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c009520 con 0x7fad9c1069b0 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.231+0000 7fad98ff9700 1 -- 172.21.15.46:0/2660620975 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94007bb0 con 0x7fad9c1069b0 2024-03-18T20:34:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.231+0000 7fada1b6e700 1 -- 172.21.15.46:0/2660620975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c1a1200 con 0x7fad9c1069b0 2024-03-18T20:34:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.232+0000 7fada1b6e700 1 -- 172.21.15.46:0/2660620975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9c0fcc80 con 0x7fad9c1069b0 2024-03-18T20:34:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.233+0000 7fad98ff9700 1 -- 172.21.15.46:0/2660620975 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fad94007d10 con 0x7fad9c1069b0 2024-03-18T20:34:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.234+0000 7fad98ff9700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fad840a11a0 0x7fad840a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.234+0000 7fad9b7fe700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fad840a11a0 0x7fad840a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.234+0000 7fad98ff9700 1 -- 172.21.15.46:0/2660620975 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fad940cc2e0 con 0x7fad9c1069b0 2024-03-18T20:34:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.234+0000 7fad9b7fe700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fad840a11a0 0x7fad840a3660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fad8c005e90 tx=0x7fad8c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:35.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.238+0000 7fad98ff9700 1 -- 172.21.15.46:0/2660620975 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fad94095fa0 con 0x7fad9c1069b0 2024-03-18T20:34:35.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.505+0000 7fada1b6e700 1 -- 172.21.15.46:0/2660620975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad9c02ccb0 con 0x7fad9c1069b0 2024-03-18T20:34:35.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.507+0000 7fad98ff9700 1 -- 172.21.15.46:0/2660620975 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18374 (secure 0 0 0) 0x7fad940995c0 con 0x7fad9c1069b0 2024-03-18T20:34:35.514 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:35.514 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":74,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:34:06.766970+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:34:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.509+0000 7fada1b6e700 1 -- 172.21.15.46:0/2660620975 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fad840a11a0 msgr2=0x7fad840a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.509+0000 7fada1b6e700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fad840a11a0 0x7fad840a3660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fad8c005e90 tx=0x7fad8c019040 comp rx=0 tx=0).stop 2024-03-18T20:34:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.509+0000 7fada1b6e700 1 -- 172.21.15.46:0/2660620975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad9c1069b0 msgr2=0x7fad9c0ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.509+0000 7fada1b6e700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad9c1069b0 0x7fad9c0ffc40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fad9400e3c0 tx=0x7fad9400e6d0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.509+0000 7fada1b6e700 1 -- 172.21.15.46:0/2660620975 shutdown_connections 2024-03-18T20:34:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.509+0000 7fada1b6e700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fad840a11a0 0x7fad840a3660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.509+0000 7fada1b6e700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad9c1069b0 0x7fad9c0ffc40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.509+0000 7fada1b6e700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad9c101750 0x7fad9c101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.509+0000 7fada1b6e700 1 --2- 172.21.15.46:0/2660620975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9c104080 0x7fad9c0ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.509+0000 7fada1b6e700 1 -- 172.21.15.46:0/2660620975 >> 172.21.15.46:0/2660620975 conn(0x7fad9c0fb100 msgr2=0x7fad9c101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.509+0000 7fada1b6e700 1 -- 172.21.15.46:0/2660620975 shutdown_connections 2024-03-18T20:34:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.509+0000 7fada1b6e700 1 -- 172.21.15.46:0/2660620975 wait complete. 2024-03-18T20:34:35.526 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:35 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:35.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:35 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2107338215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:35.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T20:31:27.488318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T20:34:35.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T20:34:35.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.682+0000 7fbc6dda8700 1 -- 172.21.15.46:0/603955107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc68102910 msgr2=0x7fbc681091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:35.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.682+0000 7fbc6dda8700 1 --2- 172.21.15.46:0/603955107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc68102910 0x7fbc681091c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbc600099c0 tx=0x7fbc60009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.682+0000 7fbc6dda8700 1 -- 172.21.15.46:0/603955107 shutdown_connections 2024-03-18T20:34:35.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.682+0000 7fbc6dda8700 1 --2- 172.21.15.46:0/603955107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc68102910 0x7fbc681091c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.682+0000 7fbc6dda8700 1 --2- 172.21.15.46:0/603955107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc68100d80 0x7fbc681011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.682+0000 7fbc6dda8700 1 --2- 172.21.15.46:0/603955107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc68101f70 0x7fbc681023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.682+0000 7fbc6dda8700 1 -- 172.21.15.46:0/603955107 >> 172.21.15.46:0/603955107 conn(0x7fbc680fc2f0 msgr2=0x7fbc680fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.682+0000 7fbc6dda8700 1 -- 172.21.15.46:0/603955107 shutdown_connections 2024-03-18T20:34:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.682+0000 7fbc6dda8700 1 -- 172.21.15.46:0/603955107 wait complete. 2024-03-18T20:34:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.683+0000 7fbc6dda8700 1 Processor -- start 2024-03-18T20:34:35.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.683+0000 7fbc6dda8700 1 -- start start 2024-03-18T20:34:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc6dda8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc68100d80 0x7fbc68196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc6dda8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc68101f70 0x7fbc68197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc6dda8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc68102910 0x7fbc6819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc6dda8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6819e7f0 con 0x7fbc68102910 2024-03-18T20:34:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc6dda8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6819e960 con 0x7fbc68101f70 2024-03-18T20:34:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc6dda8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6819eb20 con 0x7fbc68100d80 2024-03-18T20:34:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc66ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc68101f70 0x7fbc68197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc66ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc68101f70 0x7fbc68197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:53210/0 (socket says 172.21.15.46:53210) 2024-03-18T20:34:35.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc66ffd700 1 -- 172.21.15.46:0/1465526247 learned_addr learned my addr 172.21.15.46:0/1465526247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc677fe700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc68100d80 0x7fbc68196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc67fff700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc68102910 0x7fbc6819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc66ffd700 1 -- 172.21.15.46:0/1465526247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc68100d80 msgr2=0x7fbc68196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc66ffd700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc68100d80 0x7fbc68196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc66ffd700 1 -- 172.21.15.46:0/1465526247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc68102910 msgr2=0x7fbc6819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc66ffd700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc68102910 0x7fbc6819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc66ffd700 1 -- 172.21.15.46:0/1465526247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc58009c40 con 0x7fbc68101f70 2024-03-18T20:34:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.684+0000 7fbc677fe700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc68100d80 0x7fbc68196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.685+0000 7fbc66ffd700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc68101f70 0x7fbc68197210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00e9a0 tx=0x7fbc5c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:35.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.685+0000 7fbc64ff9700 1 -- 172.21.15.46:0/1465526247 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c009790 con 0x7fbc68101f70 2024-03-18T20:34:35.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.685+0000 7fbc64ff9700 1 -- 172.21.15.46:0/1465526247 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc5c010430 con 0x7fbc68101f70 2024-03-18T20:34:35.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.685+0000 7fbc6dda8700 1 -- 172.21.15.46:0/1465526247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc600095f0 con 0x7fbc68101f70 2024-03-18T20:34:35.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.685+0000 7fbc64ff9700 1 -- 172.21.15.46:0/1465526247 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c018680 con 0x7fbc68101f70 2024-03-18T20:34:35.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.685+0000 7fbc6dda8700 1 -- 172.21.15.46:0/1465526247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6819f1d0 con 0x7fbc68101f70 2024-03-18T20:34:35.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.686+0000 7fbc6dda8700 1 -- 172.21.15.46:0/1465526247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc68108130 con 0x7fbc68101f70 2024-03-18T20:34:35.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.687+0000 7fbc64ff9700 1 -- 172.21.15.46:0/1465526247 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fbc5c0105a0 con 0x7fbc68101f70 2024-03-18T20:34:35.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.688+0000 7fbc64ff9700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbc500a11a0 0x7fbc500a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:35.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.688+0000 7fbc677fe700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbc500a11a0 0x7fbc500a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:35.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.688+0000 7fbc64ff9700 1 -- 172.21.15.46:0/1465526247 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbc5c014070 con 0x7fbc68101f70 2024-03-18T20:34:35.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.688+0000 7fbc677fe700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbc500a11a0 0x7fbc500a3660 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbc680fdeb0 tx=0x7fbc5800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:35.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.691+0000 7fbc64ff9700 1 -- 172.21.15.46:0/1465526247 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbc5c0961e0 con 0x7fbc68101f70 2024-03-18T20:34:35.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:35.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:35.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:35 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:35.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:35 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2107338215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:35.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:35.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:35.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:35.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:35 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:34:35.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:35 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2107338215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.948+0000 7fbc6dda8700 1 -- 172.21.15.46:0/1465526247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc68067260 con 0x7fbc68101f70 2024-03-18T20:34:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.949+0000 7fbc64ff9700 1 -- 172.21.15.46:0/1465526247 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc5c099800 con 0x7fbc68101f70 2024-03-18T20:34:35.954 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-18T20:34:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.952+0000 7fbc6dda8700 1 -- 172.21.15.46:0/1465526247 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbc500a11a0 msgr2=0x7fbc500a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.952+0000 7fbc6dda8700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbc500a11a0 0x7fbc500a3660 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbc680fdeb0 tx=0x7fbc5800b040 comp rx=0 tx=0).stop 2024-03-18T20:34:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.952+0000 7fbc6dda8700 1 -- 172.21.15.46:0/1465526247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc68101f70 msgr2=0x7fbc68197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.952+0000 7fbc6dda8700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc68101f70 0x7fbc68197210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00e9a0 tx=0x7fbc5c00ecb0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.952+0000 7fbc6dda8700 1 -- 172.21.15.46:0/1465526247 shutdown_connections 2024-03-18T20:34:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.952+0000 7fbc6dda8700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fbc500a11a0 0x7fbc500a3660 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.952+0000 7fbc6dda8700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc68102910 0x7fbc6819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.952+0000 7fbc6dda8700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc68100d80 0x7fbc68196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.952+0000 7fbc6dda8700 1 --2- 172.21.15.46:0/1465526247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc68101f70 0x7fbc68197210 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.952+0000 7fbc6dda8700 1 -- 172.21.15.46:0/1465526247 >> 172.21.15.46:0/1465526247 conn(0x7fbc680fc2f0 msgr2=0x7fbc68105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.952+0000 7fbc6dda8700 1 -- 172.21.15.46:0/1465526247 shutdown_connections 2024-03-18T20:34:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:35.952+0000 7fbc6dda8700 1 -- 172.21.15.46:0/1465526247 wait complete. 2024-03-18T20:34:35.972 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-18T20:34:36.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.125+0000 7f3c027e2700 1 -- 172.21.15.46:0/997062099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc102960 msgr2=0x7f3bfc109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:36.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.125+0000 7f3c027e2700 1 --2- 172.21.15.46:0/997062099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc102960 0x7f3bfc109210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400b0a0 tx=0x7f3bf400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.125+0000 7f3c027e2700 1 -- 172.21.15.46:0/997062099 shutdown_connections 2024-03-18T20:34:36.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.125+0000 7f3c027e2700 1 --2- 172.21.15.46:0/997062099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc102960 0x7f3bfc109210 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.125+0000 7f3c027e2700 1 --2- 172.21.15.46:0/997062099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0fef50 0x7f3bfc0ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.125+0000 7f3c027e2700 1 --2- 172.21.15.46:0/997062099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bfc101fc0 0x7f3bfc102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.125+0000 7f3c027e2700 1 -- 172.21.15.46:0/997062099 >> 172.21.15.46:0/997062099 conn(0x7f3bfc0fa880 msgr2=0x7f3bfc0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.125+0000 7f3c027e2700 1 -- 172.21.15.46:0/997062099 shutdown_connections 2024-03-18T20:34:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.125+0000 7f3c027e2700 1 -- 172.21.15.46:0/997062099 wait complete. 2024-03-18T20:34:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.126+0000 7f3c027e2700 1 Processor -- start 2024-03-18T20:34:36.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.126+0000 7f3c027e2700 1 -- start start 2024-03-18T20:34:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.126+0000 7f3c027e2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0fef50 0x7f3bfc196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.126+0000 7f3c027e2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bfc101fc0 0x7f3bfc197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.126+0000 7f3c027e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc102960 0x7f3bfc19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.126+0000 7f3c027e2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc19e820 con 0x7f3bfc102960 2024-03-18T20:34:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.126+0000 7f3c027e2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc19e990 con 0x7f3bfc0fef50 2024-03-18T20:34:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.126+0000 7f3c027e2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc19eb50 con 0x7f3bfc101fc0 2024-03-18T20:34:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.126+0000 7f3c00d7f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc102960 0x7f3bfc19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.126+0000 7f3c00d7f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc102960 0x7f3bfc19e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44518/0 (socket says 172.21.15.46:44518) 2024-03-18T20:34:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.126+0000 7f3c00d7f700 1 -- 172.21.15.46:0/2211305115 learned_addr learned my addr 172.21.15.46:0/2211305115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.126+0000 7f3bfb7fe700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bfc101fc0 0x7f3bfc197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3bfbfff700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0fef50 0x7f3bfc196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:36.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3bfb7fe700 1 -- 172.21.15.46:0/2211305115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0fef50 msgr2=0x7f3bfc196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3bfb7fe700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0fef50 0x7f3bfc196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3bfb7fe700 1 -- 172.21.15.46:0/2211305115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc102960 msgr2=0x7f3bfc19e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3bfb7fe700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc102960 0x7f3bfc19e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3bfb7fe700 1 -- 172.21.15.46:0/2211305115 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf0009520 con 0x7f3bfc101fc0 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3bfbfff700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0fef50 0x7f3bfc196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3c00d7f700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc102960 0x7f3bfc19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3bfb7fe700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bfc101fc0 0x7f3bfc197240 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000e9f0 tx=0x7f3bf000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3bf97fa700 1 -- 172.21.15.46:0/2211305115 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf000cb70 con 0x7f3bfc101fc0 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3bf97fa700 1 -- 172.21.15.46:0/2211305115 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bf000ee90 con 0x7f3bfc101fc0 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3bf97fa700 1 -- 172.21.15.46:0/2211305115 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0003cf0 con 0x7f3bfc101fc0 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3c027e2700 1 -- 172.21.15.46:0/2211305115 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf4009d00 con 0x7f3bfc101fc0 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.127+0000 7f3c027e2700 1 -- 172.21.15.46:0/2211305115 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bfc19f1b0 con 0x7f3bfc101fc0 2024-03-18T20:34:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.129+0000 7f3c027e2700 1 -- 172.21.15.46:0/2211305115 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bfc108200 con 0x7f3bfc101fc0 2024-03-18T20:34:36.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.129+0000 7f3bf97fa700 1 -- 172.21.15.46:0/2211305115 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f3bf0018430 con 0x7f3bfc101fc0 2024-03-18T20:34:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.130+0000 7f3bf97fa700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3be40a11a0 0x7f3be40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.130+0000 7f3bfbfff700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3be40a11a0 0x7f3be40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.130+0000 7f3bf97fa700 1 -- 172.21.15.46:0/2211305115 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3bf0014070 con 0x7f3bfc101fc0 2024-03-18T20:34:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.130+0000 7f3bfbfff700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3be40a11a0 0x7f3be40a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0fc430 tx=0x7f3bec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.134+0000 7f3bf97fa700 1 -- 172.21.15.46:0/2211305115 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3bf00962a0 con 0x7f3bfc101fc0 2024-03-18T20:34:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.375+0000 7f3c027e2700 1 -- 172.21.15.46:0/2211305115 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3bfc02ccb0 con 0x7f3bfc101fc0 2024-03-18T20:34:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.376+0000 7f3bf97fa700 1 -- 172.21.15.46:0/2211305115 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3bf00998c0 con 0x7f3bfc101fc0 2024-03-18T20:34:36.380 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:36.381 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-18T20:34:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T20:34:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-03-18T20:34:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-03-18T20:34:36.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:34:36.381 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-18T20:34:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.378+0000 7f3c027e2700 1 -- 172.21.15.46:0/2211305115 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3be40a11a0 msgr2=0x7f3be40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.378+0000 7f3c027e2700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3be40a11a0 0x7f3be40a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0fc430 tx=0x7f3bec009260 comp rx=0 tx=0).stop 2024-03-18T20:34:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.378+0000 7f3c027e2700 1 -- 172.21.15.46:0/2211305115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bfc101fc0 msgr2=0x7f3bfc197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.378+0000 7f3c027e2700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bfc101fc0 0x7f3bfc197240 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000e9f0 tx=0x7f3bf000ed00 comp rx=0 tx=0).stop 2024-03-18T20:34:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.379+0000 7f3c027e2700 1 -- 172.21.15.46:0/2211305115 shutdown_connections 2024-03-18T20:34:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.379+0000 7f3c027e2700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3be40a11a0 0x7f3be40a3660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.379+0000 7f3c027e2700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc102960 0x7f3bfc19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.379+0000 7f3c027e2700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0fef50 0x7f3bfc196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.379+0000 7f3c027e2700 1 --2- 172.21.15.46:0/2211305115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bfc101fc0 0x7f3bfc197240 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.379+0000 7f3c027e2700 1 -- 172.21.15.46:0/2211305115 >> 172.21.15.46:0/2211305115 conn(0x7f3bfc0fa880 msgr2=0x7f3bfc105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:36.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.379+0000 7f3c027e2700 1 -- 172.21.15.46:0/2211305115 shutdown_connections 2024-03-18T20:34:36.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.379+0000 7f3c027e2700 1 -- 172.21.15.46:0/2211305115 wait complete. 2024-03-18T20:34:36.397 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-18T20:34:36.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-18T20:34:36.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.535+0000 7fb558be7700 1 -- 172.21.15.46:0/2236930745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5541017f0 msgr2=0x7fb554103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:36.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.535+0000 7fb558be7700 1 --2- 172.21.15.46:0/2236930745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5541017f0 0x7fb554103be0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb544009990 tx=0x7fb544009ca0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.535+0000 7fb558be7700 1 -- 172.21.15.46:0/2236930745 shutdown_connections 2024-03-18T20:34:36.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.535+0000 7fb558be7700 1 --2- 172.21.15.46:0/2236930745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5541017f0 0x7fb554103be0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.535+0000 7fb558be7700 1 --2- 172.21.15.46:0/2236930745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554106a50 0x7fb554108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.535+0000 7fb558be7700 1 --2- 172.21.15.46:0/2236930745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb554104120 0x7fb554106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.535+0000 7fb558be7700 1 -- 172.21.15.46:0/2236930745 >> 172.21.15.46:0/2236930745 conn(0x7fb5540fb200 msgr2=0x7fb5540fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:36.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.535+0000 7fb558be7700 1 -- 172.21.15.46:0/2236930745 shutdown_connections 2024-03-18T20:34:36.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.535+0000 7fb558be7700 1 -- 172.21.15.46:0/2236930745 wait complete. 2024-03-18T20:34:36.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb558be7700 1 Processor -- start 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb558be7700 1 -- start start 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb558be7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5541017f0 0x7fb554196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb558be7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb554104120 0x7fb5541971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb558be7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554106a50 0x7fb55419e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb558be7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55419e790 con 0x7fb5541017f0 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb558be7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55419e900 con 0x7fb554104120 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb558be7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55419eac0 con 0x7fb554106a50 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb55259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5541017f0 0x7fb554196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb55259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5541017f0 0x7fb554196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44536/0 (socket says 172.21.15.46:44536) 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb55259c700 1 -- 172.21.15.46:0/2130394343 learned_addr learned my addr 172.21.15.46:0/2130394343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb552d9d700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554106a50 0x7fb55419e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.536+0000 7fb551d9b700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb554104120 0x7fb5541971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb55259c700 1 -- 172.21.15.46:0/2130394343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554106a50 msgr2=0x7fb55419e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:36.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb55259c700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554106a50 0x7fb55419e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb55259c700 1 -- 172.21.15.46:0/2130394343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb554104120 msgr2=0x7fb5541971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb55259c700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb554104120 0x7fb5541971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb55259c700 1 -- 172.21.15.46:0/2130394343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5440095f0 con 0x7fb5541017f0 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb552d9d700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554106a50 0x7fb55419e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb551d9b700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb554104120 0x7fb5541971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb55259c700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5541017f0 0x7fb554196ca0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb544005910 tx=0x7fb544009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb5437fe700 1 -- 172.21.15.46:0/2130394343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54401d070 con 0x7fb5541017f0 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb5437fe700 1 -- 172.21.15.46:0/2130394343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54400fbc0 con 0x7fb5541017f0 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb558be7700 1 -- 172.21.15.46:0/2130394343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55419ed40 con 0x7fb5541017f0 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb5437fe700 1 -- 172.21.15.46:0/2130394343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544017650 con 0x7fb5541017f0 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.537+0000 7fb558be7700 1 -- 172.21.15.46:0/2130394343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55419f160 con 0x7fb5541017f0 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.538+0000 7fb558be7700 1 -- 172.21.15.46:0/2130394343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5540fcd80 con 0x7fb5541017f0 2024-03-18T20:34:36.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.539+0000 7fb5437fe700 1 -- 172.21.15.46:0/2130394343 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fb544021410 con 0x7fb5541017f0 2024-03-18T20:34:36.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.540+0000 7fb5437fe700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb53c0a1120 0x7fb53c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:36.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.540+0000 7fb551d9b700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb53c0a1120 0x7fb53c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:36.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.540+0000 7fb5437fe700 1 -- 172.21.15.46:0/2130394343 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb5440cdd90 con 0x7fb5541017f0 2024-03-18T20:34:36.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.541+0000 7fb551d9b700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb53c0a1120 0x7fb53c0a35e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb548009cb0 tx=0x7fb548009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.544+0000 7fb5437fe700 1 -- 172.21.15.46:0/2130394343 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb5440979f0 con 0x7fb5541017f0 2024-03-18T20:34:36.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:36 smithi073 ceph-mon[47384]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:36.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:36 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2660620975' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:36.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:36 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1465526247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:34:36.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:36.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:36.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:36 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:36.758+0000 7fb558be7700 1 -- 172.21.15.46:0/2130394343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb554067f40 con 0x7fb5541017f0 2024-03-18T20:34:36.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:36 smithi155 ceph-mon[48344]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:36.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:36 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2660620975' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:36.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:36 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1465526247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:34:36.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:36.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:36.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:36 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:36 smithi046 ceph-mon[43495]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:36.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:36 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2660620975' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:36.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:36 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1465526247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:34:36.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:36.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:34:36.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:34:36.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:36 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:37.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.316+0000 7fb5437fe700 1 -- 172.21.15.46:0/2130394343 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb54409b010 con 0x7fb5541017f0 2024-03-18T20:34:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.321+0000 7fb558be7700 1 -- 172.21.15.46:0/2130394343 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb53c0a1120 msgr2=0x7fb53c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.321+0000 7fb558be7700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb53c0a1120 0x7fb53c0a35e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb548009cb0 tx=0x7fb548009260 comp rx=0 tx=0).stop 2024-03-18T20:34:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.321+0000 7fb558be7700 1 -- 172.21.15.46:0/2130394343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5541017f0 msgr2=0x7fb554196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.321+0000 7fb558be7700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5541017f0 0x7fb554196ca0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb544005910 tx=0x7fb544009580 comp rx=0 tx=0).stop 2024-03-18T20:34:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.322+0000 7fb558be7700 1 -- 172.21.15.46:0/2130394343 shutdown_connections 2024-03-18T20:34:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.322+0000 7fb558be7700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb53c0a1120 0x7fb53c0a35e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.322+0000 7fb558be7700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5541017f0 0x7fb554196ca0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.322+0000 7fb558be7700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554106a50 0x7fb55419e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.322+0000 7fb558be7700 1 --2- 172.21.15.46:0/2130394343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb554104120 0x7fb5541971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.322+0000 7fb558be7700 1 -- 172.21.15.46:0/2130394343 >> 172.21.15.46:0/2130394343 conn(0x7fb5540fb200 msgr2=0x7fb554103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.323+0000 7fb558be7700 1 -- 172.21.15.46:0/2130394343 shutdown_connections 2024-03-18T20:34:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.323+0000 7fb558be7700 1 -- 172.21.15.46:0/2130394343 wait complete. 2024-03-18T20:34:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-03-18T20:34:37.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-18T20:34:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.484+0000 7f827d00c700 1 -- 172.21.15.46:0/2844285674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8278101710 msgr2=0x7f8278107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.484+0000 7f827d00c700 1 --2- 172.21.15.46:0/2844285674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8278101710 0x7f8278107fc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f827000b0d0 tx=0x7f827000b3e0 comp rx=0 tx=0).stop 2024-03-18T20:34:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.484+0000 7f827d00c700 1 -- 172.21.15.46:0/2844285674 shutdown_connections 2024-03-18T20:34:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.484+0000 7f827d00c700 1 --2- 172.21.15.46:0/2844285674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8278101710 0x7f8278107fc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.484+0000 7f827d00c700 1 --2- 172.21.15.46:0/2844285674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82780ffb80 0x7f82780fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.484+0000 7f827d00c700 1 --2- 172.21.15.46:0/2844285674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8278100d70 0x7f82781011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.484+0000 7f827d00c700 1 -- 172.21.15.46:0/2844285674 >> 172.21.15.46:0/2844285674 conn(0x7f82780fb0f0 msgr2=0x7f82780fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.484+0000 7f827d00c700 1 -- 172.21.15.46:0/2844285674 shutdown_connections 2024-03-18T20:34:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.484+0000 7f827d00c700 1 -- 172.21.15.46:0/2844285674 wait complete. 2024-03-18T20:34:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.485+0000 7f827d00c700 1 Processor -- start 2024-03-18T20:34:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827d00c700 1 -- start start 2024-03-18T20:34:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827d00c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82780ffb80 0x7f8278068910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827d00c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8278100d70 0x7f827819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827d00c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8278101710 0x7f827819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827d00c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827819e7b0 con 0x7f8278101710 2024-03-18T20:34:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827d00c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827819e920 con 0x7f82780ffb80 2024-03-18T20:34:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827d00c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827819eae0 con 0x7f8278100d70 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8278101710 0x7f827819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f8276d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82780ffb80 0x7f8278068910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8278101710 0x7f827819e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44564/0 (socket says 172.21.15.46:44564) 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827759e700 1 -- 172.21.15.46:0/3974286303 learned_addr learned my addr 172.21.15.46:0/3974286303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f826ffff700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8278100d70 0x7f827819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827759e700 1 -- 172.21.15.46:0/3974286303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8278100d70 msgr2=0x7f827819ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827759e700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8278100d70 0x7f827819ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827759e700 1 -- 172.21.15.46:0/3974286303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82780ffb80 msgr2=0x7f8278068910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827759e700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82780ffb80 0x7f8278068910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f827759e700 1 -- 172.21.15.46:0/3974286303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8270009d00 con 0x7f8278101710 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f826ffff700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8278100d70 0x7f827819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.486+0000 7f8276d9d700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82780ffb80 0x7f8278068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.487+0000 7f827759e700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8278101710 0x7f827819e010 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8270007b60 tx=0x7f8270008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.487+0000 7f8274d99700 1 -- 172.21.15.46:0/3974286303 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827000e040 con 0x7f8278101710 2024-03-18T20:34:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.487+0000 7f8274d99700 1 -- 172.21.15.46:0/3974286303 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827000be30 con 0x7f8278101710 2024-03-18T20:34:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.487+0000 7f827d00c700 1 -- 172.21.15.46:0/3974286303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827819ed60 con 0x7f8278101710 2024-03-18T20:34:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.487+0000 7f8274d99700 1 -- 172.21.15.46:0/3974286303 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827001bae0 con 0x7f8278101710 2024-03-18T20:34:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.487+0000 7f827d00c700 1 -- 172.21.15.46:0/3974286303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827819f190 con 0x7f8278101710 2024-03-18T20:34:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.488+0000 7f827d00c700 1 -- 172.21.15.46:0/3974286303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8278106f30 con 0x7f8278101710 2024-03-18T20:34:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.490+0000 7f8274d99700 1 -- 172.21.15.46:0/3974286303 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f827001d670 con 0x7f8278101710 2024-03-18T20:34:37.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.490+0000 7f8274d99700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f82640a1110 0x7f82640a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:37.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.490+0000 7f8276d9d700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f82640a1110 0x7f82640a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:37.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.490+0000 7f8274d99700 1 -- 172.21.15.46:0/3974286303 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f82700ce580 con 0x7f8278101710 2024-03-18T20:34:37.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.491+0000 7f8276d9d700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f82640a1110 0x7f82640a35d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8278100bd0 tx=0x7f826800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:37.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.494+0000 7f8274d99700 1 -- 172.21.15.46:0/3974286303 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f82700980d0 con 0x7f8278101710 2024-03-18T20:34:37.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:37.712+0000 7f827d00c700 1 -- 172.21.15.46:0/3974286303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f827802d130 con 0x7f8278101710 2024-03-18T20:34:37.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:37 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2211305115' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-18T20:34:37.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:37 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2130394343' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-18T20:34:37.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:37 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:37.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:37 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2211305115' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-18T20:34:37.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:37 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2130394343' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-18T20:34:37.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:37 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:37.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:37 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2211305115' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-18T20:34:37.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:37 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2130394343' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-18T20:34:37.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:34:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.323+0000 7f8274d99700 1 -- 172.21.15.46:0/3974286303 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) v1 ==== 170+0+0 (secure 0 0 0) 0x7f827802d130 con 0x7f8278101710 2024-03-18T20:34:38.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.336+0000 7f827d00c700 1 -- 172.21.15.46:0/3974286303 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f82640a1110 msgr2=0x7f82640a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:38.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.336+0000 7f827d00c700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f82640a1110 0x7f82640a35d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8278100bd0 tx=0x7f826800b410 comp rx=0 tx=0).stop 2024-03-18T20:34:38.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.336+0000 7f827d00c700 1 -- 172.21.15.46:0/3974286303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8278101710 msgr2=0x7f827819e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:38.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.336+0000 7f827d00c700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8278101710 0x7f827819e010 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8270007b60 tx=0x7f8270008f10 comp rx=0 tx=0).stop 2024-03-18T20:34:38.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.343+0000 7f827d00c700 1 -- 172.21.15.46:0/3974286303 shutdown_connections 2024-03-18T20:34:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.343+0000 7f827d00c700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f82640a1110 0x7f82640a35d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.343+0000 7f827d00c700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8278101710 0x7f827819e010 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.343+0000 7f827d00c700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82780ffb80 0x7f8278068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.343+0000 7f827d00c700 1 --2- 172.21.15.46:0/3974286303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8278100d70 0x7f827819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.343+0000 7f827d00c700 1 -- 172.21.15.46:0/3974286303 >> 172.21.15.46:0/3974286303 conn(0x7f82780fb0f0 msgr2=0x7f8278104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.343+0000 7f827d00c700 1 -- 172.21.15.46:0/3974286303 shutdown_connections 2024-03-18T20:34:38.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.343+0000 7f827d00c700 1 -- 172.21.15.46:0/3974286303 wait complete. 2024-03-18T20:34:38.351 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-03-18T20:34:38.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-18T20:34:38.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.503+0000 7fe87b710700 1 -- 172.21.15.46:0/2904794630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8740ffc90 msgr2=0x7fe8741000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.503+0000 7fe87b710700 1 --2- 172.21.15.46:0/2904794630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8740ffc90 0x7fe8741000b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe8700099c0 tx=0x7fe870009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.503+0000 7fe87b710700 1 -- 172.21.15.46:0/2904794630 shutdown_connections 2024-03-18T20:34:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.503+0000 7fe87b710700 1 --2- 172.21.15.46:0/2904794630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8740ffc90 0x7fe8741000b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.503+0000 7fe87b710700 1 --2- 172.21.15.46:0/2904794630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe874101820 0x7fe8741080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.503+0000 7fe87b710700 1 --2- 172.21.15.46:0/2904794630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe874100e80 0x7fe8741012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.503+0000 7fe87b710700 1 -- 172.21.15.46:0/2904794630 >> 172.21.15.46:0/2904794630 conn(0x7fe8740fb200 msgr2=0x7fe8740fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.503+0000 7fe87b710700 1 -- 172.21.15.46:0/2904794630 shutdown_connections 2024-03-18T20:34:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.503+0000 7fe87b710700 1 -- 172.21.15.46:0/2904794630 wait complete. 2024-03-18T20:34:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe87b710700 1 Processor -- start 2024-03-18T20:34:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe87b710700 1 -- start start 2024-03-18T20:34:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe87b710700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8740ffc90 0x7fe874069a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe87b710700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe874100e80 0x7fe87419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe87b710700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe874101820 0x7fe87419e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe87b710700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87419e800 con 0x7fe8740ffc90 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe87b710700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87419e970 con 0x7fe874101820 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe87b710700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87419eb30 con 0x7fe874100e80 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe8794ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8740ffc90 0x7fe874069a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe8794ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8740ffc90 0x7fe874069a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44572/0 (socket says 172.21.15.46:44572) 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe8794ac700 1 -- 172.21.15.46:0/1718720699 learned_addr learned my addr 172.21.15.46:0/1718720699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe878cab700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe874100e80 0x7fe87419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe879cad700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe874101820 0x7fe87419e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe8794ac700 1 -- 172.21.15.46:0/1718720699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe874100e80 msgr2=0x7fe87419ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe8794ac700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe874100e80 0x7fe87419ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe8794ac700 1 -- 172.21.15.46:0/1718720699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe874101820 msgr2=0x7fe87419e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:38.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe8794ac700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe874101820 0x7fe87419e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe8794ac700 1 -- 172.21.15.46:0/1718720699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8700095f0 con 0x7fe8740ffc90 2024-03-18T20:34:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.504+0000 7fe879cad700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe874101820 0x7fe87419e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:34:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.505+0000 7fe8794ac700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8740ffc90 0x7fe874069a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe870009fd0 tx=0x7fe870005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.505+0000 7fe86a7fc700 1 -- 172.21.15.46:0/1718720699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87001d070 con 0x7fe8740ffc90 2024-03-18T20:34:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.505+0000 7fe86a7fc700 1 -- 172.21.15.46:0/1718720699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8700059a0 con 0x7fe8740ffc90 2024-03-18T20:34:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.505+0000 7fe87b710700 1 -- 172.21.15.46:0/1718720699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87419edb0 con 0x7fe8740ffc90 2024-03-18T20:34:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.505+0000 7fe86a7fc700 1 -- 172.21.15.46:0/1718720699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8700173f0 con 0x7fe8740ffc90 2024-03-18T20:34:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.505+0000 7fe87b710700 1 -- 172.21.15.46:0/1718720699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87419f1f0 con 0x7fe8740ffc90 2024-03-18T20:34:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.506+0000 7fe87b710700 1 -- 172.21.15.46:0/1718720699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe874107160 con 0x7fe8740ffc90 2024-03-18T20:34:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.507+0000 7fe86a7fc700 1 -- 172.21.15.46:0/1718720699 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fe8700226e0 con 0x7fe8740ffc90 2024-03-18T20:34:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.508+0000 7fe86a7fc700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe8600a11a0 0x7fe8600a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.508+0000 7fe86a7fc700 1 -- 172.21.15.46:0/1718720699 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fe8700cda90 con 0x7fe8740ffc90 2024-03-18T20:34:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.508+0000 7fe878cab700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe8600a11a0 0x7fe8600a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.508+0000 7fe878cab700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe8600a11a0 0x7fe8600a3660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe87406a050 tx=0x7fe86400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:38.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.513+0000 7fe86a7fc700 1 -- 172.21.15.46:0/1718720699 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe8700974e0 con 0x7fe8740ffc90 2024-03-18T20:34:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:38 smithi073 ceph-mon[47384]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:38 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2130394343' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-18T20:34:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:38 smithi073 ceph-mon[47384]: osdmap e75: 12 total, 12 up, 12 in 2024-03-18T20:34:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:38 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3974286303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T20:34:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.782+0000 7fe87b710700 1 -- 172.21.15.46:0/1718720699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe87402ce60 con 0x7fe8740ffc90 2024-03-18T20:34:38.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:38 smithi155 ceph-mon[48344]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:38.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:38 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2130394343' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-18T20:34:38.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:38 smithi155 ceph-mon[48344]: osdmap e75: 12 total, 12 up, 12 in 2024-03-18T20:34:38.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:38 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3974286303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T20:34:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.789+0000 7fe86a7fc700 1 -- 172.21.15.46:0/1718720699 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe87009ab00 con 0x7fe8740ffc90 2024-03-18T20:34:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:38 smithi046 ceph-mon[43495]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:38 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2130394343' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-18T20:34:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:38 smithi046 ceph-mon[43495]: osdmap e75: 12 total, 12 up, 12 in 2024-03-18T20:34:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:38 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3974286303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T20:34:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.798+0000 7fe87b710700 1 -- 172.21.15.46:0/1718720699 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe8600a11a0 msgr2=0x7fe8600a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.798+0000 7fe87b710700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe8600a11a0 0x7fe8600a3660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe87406a050 tx=0x7fe86400b410 comp rx=0 tx=0).stop 2024-03-18T20:34:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.798+0000 7fe87b710700 1 -- 172.21.15.46:0/1718720699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8740ffc90 msgr2=0x7fe874069a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.798+0000 7fe87b710700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8740ffc90 0x7fe874069a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe870009fd0 tx=0x7fe870005f10 comp rx=0 tx=0).stop 2024-03-18T20:34:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.801+0000 7fe87b710700 1 -- 172.21.15.46:0/1718720699 shutdown_connections 2024-03-18T20:34:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.801+0000 7fe87b710700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe8600a11a0 0x7fe8600a3660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.801+0000 7fe87b710700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8740ffc90 0x7fe874069a90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.801+0000 7fe87b710700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe874101820 0x7fe87419e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.801+0000 7fe87b710700 1 --2- 172.21.15.46:0/1718720699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe874100e80 0x7fe87419ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.801+0000 7fe87b710700 1 -- 172.21.15.46:0/1718720699 >> 172.21.15.46:0/1718720699 conn(0x7fe8740fb200 msgr2=0x7fe8741042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.802+0000 7fe87b710700 1 -- 172.21.15.46:0/1718720699 shutdown_connections 2024-03-18T20:34:38.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.810+0000 7fe87b710700 1 -- 172.21.15.46:0/1718720699 wait complete. 2024-03-18T20:34:38.818 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2024-03-18T20:34:38.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:34:38.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.966+0000 7f4070894700 1 -- 172.21.15.46:0/135215319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c100d50 msgr2=0x7f406c1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:38.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.966+0000 7f4070894700 1 --2- 172.21.15.46:0/135215319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c100d50 0x7f406c1011b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f40540099c0 tx=0x7f4054009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.966+0000 7f4070894700 1 -- 172.21.15.46:0/135215319 shutdown_connections 2024-03-18T20:34:38.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.966+0000 7f4070894700 1 --2- 172.21.15.46:0/135215319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f406c1016f0 0x7f406c107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.966+0000 7f4070894700 1 --2- 172.21.15.46:0/135215319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406c0ffb60 0x7f406c0fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.966+0000 7f4070894700 1 --2- 172.21.15.46:0/135215319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c100d50 0x7f406c1011b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.966+0000 7f4070894700 1 -- 172.21.15.46:0/135215319 >> 172.21.15.46:0/135215319 conn(0x7f406c0fb0d0 msgr2=0x7f406c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:38.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.966+0000 7f4070894700 1 -- 172.21.15.46:0/135215319 shutdown_connections 2024-03-18T20:34:38.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.966+0000 7f4070894700 1 -- 172.21.15.46:0/135215319 wait complete. 2024-03-18T20:34:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.967+0000 7f4070894700 1 Processor -- start 2024-03-18T20:34:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.967+0000 7f4070894700 1 -- start start 2024-03-18T20:34:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.967+0000 7f4070894700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0ffb60 0x7f406c103a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.967+0000 7f4070894700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f406c100d50 0x7f406c102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.967+0000 7f4070894700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406c1016f0 0x7f406c102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.967+0000 7f4070894700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c102d70 con 0x7f406c0ffb60 2024-03-18T20:34:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.967+0000 7f4070894700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c102ee0 con 0x7f406c1016f0 2024-03-18T20:34:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.967+0000 7f4070894700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c1a0c40 con 0x7f406c100d50 2024-03-18T20:34:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.967+0000 7f406a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0ffb60 0x7f406c103a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:38.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.967+0000 7f406a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0ffb60 0x7f406c103a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44596/0 (socket says 172.21.15.46:44596) 2024-03-18T20:34:38.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.967+0000 7f406a59c700 1 -- 172.21.15.46:0/2459013228 learned_addr learned my addr 172.21.15.46:0/2459013228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:38.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f4061d9b700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f406c100d50 0x7f406c102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:38.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f406ad9d700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406c1016f0 0x7f406c102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:38.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f406a59c700 1 -- 172.21.15.46:0/2459013228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f406c100d50 msgr2=0x7f406c102130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:38.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f406a59c700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f406c100d50 0x7f406c102130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f406a59c700 1 -- 172.21.15.46:0/2459013228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406c1016f0 msgr2=0x7f406c102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:38.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f406a59c700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406c1016f0 0x7f406c102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:38.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f406a59c700 1 -- 172.21.15.46:0/2459013228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40540095f0 con 0x7f406c0ffb60 2024-03-18T20:34:38.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f406a59c700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0ffb60 0x7f406c103a50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f405c00d980 tx=0x7f405c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:38.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f4063fff700 1 -- 172.21.15.46:0/2459013228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c004020 con 0x7f406c0ffb60 2024-03-18T20:34:38.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f4063fff700 1 -- 172.21.15.46:0/2459013228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f405c00f690 con 0x7f406c0ffb60 2024-03-18T20:34:38.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f4070894700 1 -- 172.21.15.46:0/2459013228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406c1a0ea0 con 0x7f406c0ffb60 2024-03-18T20:34:38.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f4063fff700 1 -- 172.21.15.46:0/2459013228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c003a60 con 0x7f406c0ffb60 2024-03-18T20:34:38.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.968+0000 7f4070894700 1 -- 172.21.15.46:0/2459013228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c1a12c0 con 0x7f406c0ffb60 2024-03-18T20:34:38.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.969+0000 7f4070894700 1 -- 172.21.15.46:0/2459013228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406c067a10 con 0x7f406c0ffb60 2024-03-18T20:34:38.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.970+0000 7f4063fff700 1 -- 172.21.15.46:0/2459013228 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f405c00fb00 con 0x7f406c0ffb60 2024-03-18T20:34:38.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.971+0000 7f4063fff700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f40580a1120 0x7f40580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:38.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.971+0000 7f4061d9b700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f40580a1120 0x7f40580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:38.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.971+0000 7f4063fff700 1 -- 172.21.15.46:0/2459013228 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f405c0cc700 con 0x7f406c0ffb60 2024-03-18T20:34:38.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.972+0000 7f4061d9b700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f40580a1120 0x7f40580a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4054009990 tx=0x7f405400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:38.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:38.975+0000 7f4063fff700 1 -- 172.21.15.46:0/2459013228 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f405c095f40 con 0x7f406c0ffb60 2024-03-18T20:34:39.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.204+0000 7f4070894700 1 -- 172.21.15.46:0/2459013228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f406c1a1680 con 0x7f406c0ffb60 2024-03-18T20:34:39.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.206+0000 7f4063fff700 1 -- 172.21.15.46:0/2459013228 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+22084 (secure 0 0 0) 0x7f405c099560 con 0x7f406c0ffb60 2024-03-18T20:34:39.211 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:39.213 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:34:38.784699+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:34:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.208+0000 7f4070894700 1 -- 172.21.15.46:0/2459013228 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f40580a1120 msgr2=0x7f40580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.208+0000 7f4070894700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f40580a1120 0x7f40580a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4054009990 tx=0x7f405400b540 comp rx=0 tx=0).stop 2024-03-18T20:34:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.208+0000 7f4070894700 1 -- 172.21.15.46:0/2459013228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0ffb60 msgr2=0x7f406c103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.208+0000 7f4070894700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0ffb60 0x7f406c103a50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f405c00d980 tx=0x7f405c00dc90 comp rx=0 tx=0).stop 2024-03-18T20:34:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.208+0000 7f4070894700 1 -- 172.21.15.46:0/2459013228 shutdown_connections 2024-03-18T20:34:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.208+0000 7f4070894700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f40580a1120 0x7f40580a35e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.208+0000 7f4070894700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f406c0ffb60 0x7f406c103a50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.208+0000 7f4070894700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f406c1016f0 0x7f406c102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.208+0000 7f4070894700 1 --2- 172.21.15.46:0/2459013228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f406c100d50 0x7f406c102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.208+0000 7f4070894700 1 -- 172.21.15.46:0/2459013228 >> 172.21.15.46:0/2459013228 conn(0x7f406c0fb0d0 msgr2=0x7f406c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.209+0000 7f4070894700 1 -- 172.21.15.46:0/2459013228 shutdown_connections 2024-03-18T20:34:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.209+0000 7f4070894700 1 -- 172.21.15.46:0/2459013228 wait complete. 2024-03-18T20:34:39.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:34:39.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.360+0000 7efddf28e700 1 -- 172.21.15.46:0/414764828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd81069b0 msgr2=0x7efdd8108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:39.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.360+0000 7efddf28e700 1 --2- 172.21.15.46:0/414764828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd81069b0 0x7efdd8108dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efdd000b0a0 tx=0x7efdd000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.361+0000 7efddf28e700 1 -- 172.21.15.46:0/414764828 shutdown_connections 2024-03-18T20:34:39.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.361+0000 7efddf28e700 1 --2- 172.21.15.46:0/414764828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd81069b0 0x7efdd8108dc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.361+0000 7efddf28e700 1 --2- 172.21.15.46:0/414764828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdd8101750 0x7efdd8103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.361+0000 7efddf28e700 1 --2- 172.21.15.46:0/414764828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdd8104080 0x7efdd8106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.361+0000 7efddf28e700 1 -- 172.21.15.46:0/414764828 >> 172.21.15.46:0/414764828 conn(0x7efdd80fb140 msgr2=0x7efdd80fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.361+0000 7efddf28e700 1 -- 172.21.15.46:0/414764828 shutdown_connections 2024-03-18T20:34:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.361+0000 7efddf28e700 1 -- 172.21.15.46:0/414764828 wait complete. 2024-03-18T20:34:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.361+0000 7efddf28e700 1 Processor -- start 2024-03-18T20:34:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.361+0000 7efddf28e700 1 -- start start 2024-03-18T20:34:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddf28e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd8101750 0x7efdd8196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddf28e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdd8104080 0x7efdd8197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddf28e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdd81069b0 0x7efdd819e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddf28e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd819e7b0 con 0x7efdd8101750 2024-03-18T20:34:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddf28e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd819e920 con 0x7efdd81069b0 2024-03-18T20:34:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddf28e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd819eae0 con 0x7efdd8104080 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddd02a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd8101750 0x7efdd8196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddd02a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd8101750 0x7efdd8196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44612/0 (socket says 172.21.15.46:44612) 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddd02a700 1 -- 172.21.15.46:0/280358737 learned_addr learned my addr 172.21.15.46:0/280358737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddd82b700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdd81069b0 0x7efdd819e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddc829700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdd8104080 0x7efdd8197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddd02a700 1 -- 172.21.15.46:0/280358737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdd8104080 msgr2=0x7efdd8197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddd02a700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdd8104080 0x7efdd8197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddd02a700 1 -- 172.21.15.46:0/280358737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdd81069b0 msgr2=0x7efdd819e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddd02a700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdd81069b0 0x7efdd819e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddd02a700 1 -- 172.21.15.46:0/280358737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd0009d00 con 0x7efdd8101750 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddd02a700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd8101750 0x7efdd8196d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efdd400d980 tx=0x7efdd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efdce7fc700 1 -- 172.21.15.46:0/280358737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd4004220 con 0x7efdd8101750 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efdce7fc700 1 -- 172.21.15.46:0/280358737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdd4007640 con 0x7efdd8101750 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efdce7fc700 1 -- 172.21.15.46:0/280358737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd4005610 con 0x7efdd8101750 2024-03-18T20:34:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.362+0000 7efddf28e700 1 -- 172.21.15.46:0/280358737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd819edc0 con 0x7efdd8101750 2024-03-18T20:34:39.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.363+0000 7efddf28e700 1 -- 172.21.15.46:0/280358737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd819f1c0 con 0x7efdd8101750 2024-03-18T20:34:39.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.363+0000 7efddf28e700 1 -- 172.21.15.46:0/280358737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd80fccc0 con 0x7efdd8101750 2024-03-18T20:34:39.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.365+0000 7efdce7fc700 1 -- 172.21.15.46:0/280358737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7efdd4004750 con 0x7efdd8101750 2024-03-18T20:34:39.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.366+0000 7efdce7fc700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efdc40a1120 0x7efdc40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:39.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.366+0000 7efdce7fc700 1 -- 172.21.15.46:0/280358737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efdd40ccba0 con 0x7efdd8101750 2024-03-18T20:34:39.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.366+0000 7efddc829700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efdc40a1120 0x7efdc40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:39.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.366+0000 7efddc829700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efdc40a1120 0x7efdc40a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009bd0 tx=0x7efdc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:39.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.370+0000 7efdce7fc700 1 -- 172.21.15.46:0/280358737 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efdd40963e0 con 0x7efdd8101750 2024-03-18T20:34:39.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:39 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3974286303' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-18T20:34:39.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:39 smithi046 ceph-mon[43495]: osdmap e76: 12 total, 12 up, 12 in 2024-03-18T20:34:39.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:39 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1718720699' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-18T20:34:39.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:39 smithi046 ceph-mon[43495]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-18T20:34:39.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:39 smithi046 ceph-mon[43495]: osdmap e77: 12 total, 12 up, 12 in 2024-03-18T20:34:39.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:39 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1718720699' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-18T20:34:39.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:39 smithi046 ceph-mon[43495]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-18T20:34:39.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:39 smithi046 ceph-mon[43495]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-18T20:34:39.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:39 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2459013228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.594+0000 7efddf28e700 1 -- 172.21.15.46:0/280358737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdd8067e60 con 0x7efdd8101750 2024-03-18T20:34:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.595+0000 7efdce7fc700 1 -- 172.21.15.46:0/280358737 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+22084 (secure 0 0 0) 0x7efdd4016090 con 0x7efdd8101750 2024-03-18T20:34:39.600 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:39.600 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:34:38.784699+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:34:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.598+0000 7efddf28e700 1 -- 172.21.15.46:0/280358737 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efdc40a1120 msgr2=0x7efdc40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.598+0000 7efddf28e700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efdc40a1120 0x7efdc40a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009bd0 tx=0x7efdc8009260 comp rx=0 tx=0).stop 2024-03-18T20:34:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.598+0000 7efddf28e700 1 -- 172.21.15.46:0/280358737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd8101750 msgr2=0x7efdd8196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.598+0000 7efddf28e700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd8101750 0x7efdd8196d10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efdd400d980 tx=0x7efdd400dc90 comp rx=0 tx=0).stop 2024-03-18T20:34:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.598+0000 7efddf28e700 1 -- 172.21.15.46:0/280358737 shutdown_connections 2024-03-18T20:34:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.598+0000 7efddf28e700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efdc40a1120 0x7efdc40a35e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.598+0000 7efddf28e700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd8101750 0x7efdd8196d10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.598+0000 7efddf28e700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdd81069b0 0x7efdd819e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.598+0000 7efddf28e700 1 --2- 172.21.15.46:0/280358737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdd8104080 0x7efdd8197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.598+0000 7efddf28e700 1 -- 172.21.15.46:0/280358737 >> 172.21.15.46:0/280358737 conn(0x7efdd80fb140 msgr2=0x7efdd81020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.598+0000 7efddf28e700 1 -- 172.21.15.46:0/280358737 shutdown_connections 2024-03-18T20:34:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.598+0000 7efddf28e700 1 -- 172.21.15.46:0/280358737 wait complete. 2024-03-18T20:34:39.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-18T20:34:39.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:39 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3974286303' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-18T20:34:39.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:39 smithi073 ceph-mon[47384]: osdmap e76: 12 total, 12 up, 12 in 2024-03-18T20:34:39.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:39 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1718720699' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-18T20:34:39.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:39 smithi073 ceph-mon[47384]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-18T20:34:39.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:39 smithi073 ceph-mon[47384]: osdmap e77: 12 total, 12 up, 12 in 2024-03-18T20:34:39.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:39 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1718720699' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-18T20:34:39.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:39 smithi073 ceph-mon[47384]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-18T20:34:39.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:39 smithi073 ceph-mon[47384]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-18T20:34:39.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:39 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2459013228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.754+0000 7f886c57b700 1 -- 172.21.15.46:0/2657072762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8864101820 msgr2=0x7f88641080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.754+0000 7f886c57b700 1 --2- 172.21.15.46:0/2657072762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8864101820 0x7f88641080d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f885c00b0d0 tx=0x7f885c00b3e0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.754+0000 7f886c57b700 1 -- 172.21.15.46:0/2657072762 shutdown_connections 2024-03-18T20:34:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.754+0000 7f886c57b700 1 --2- 172.21.15.46:0/2657072762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8864101820 0x7f88641080d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.754+0000 7f886c57b700 1 --2- 172.21.15.46:0/2657072762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88640ffc90 0x7f88641000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.754+0000 7f886c57b700 1 --2- 172.21.15.46:0/2657072762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8864100e80 0x7f88641012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.754+0000 7f886c57b700 1 -- 172.21.15.46:0/2657072762 >> 172.21.15.46:0/2657072762 conn(0x7f88640fb200 msgr2=0x7f88640fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.754+0000 7f886c57b700 1 -- 172.21.15.46:0/2657072762 shutdown_connections 2024-03-18T20:34:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.754+0000 7f886c57b700 1 -- 172.21.15.46:0/2657072762 wait complete. 2024-03-18T20:34:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f886c57b700 1 Processor -- start 2024-03-18T20:34:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f886c57b700 1 -- start start 2024-03-18T20:34:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f886c57b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88640ffc90 0x7f8864195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f886c57b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8864100e80 0x7f8864196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f886c57b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8864101820 0x7f886419d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f886c57b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886419d800 con 0x7f8864100e80 2024-03-18T20:34:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f886c57b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886419d970 con 0x7f8864101820 2024-03-18T20:34:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f886c57b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886419db30 con 0x7f88640ffc90 2024-03-18T20:34:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f8869b16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8864100e80 0x7f8864196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f886a317700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88640ffc90 0x7f8864195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f886a317700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88640ffc90 0x7f8864195ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:50142/0 (socket says 172.21.15.46:50142) 2024-03-18T20:34:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f8869b16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8864100e80 0x7f8864196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44632/0 (socket says 172.21.15.46:44632) 2024-03-18T20:34:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f886a317700 1 -- 172.21.15.46:0/329472520 learned_addr learned my addr 172.21.15.46:0/329472520 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.755+0000 7f886ab18700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8864101820 0x7f886419d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.756+0000 7f886ab18700 1 -- 172.21.15.46:0/329472520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88640ffc90 msgr2=0x7f8864195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.756+0000 7f886ab18700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88640ffc90 0x7f8864195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.756+0000 7f886ab18700 1 -- 172.21.15.46:0/329472520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8864100e80 msgr2=0x7f8864196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.756+0000 7f886ab18700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8864100e80 0x7f8864196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.756+0000 7f886ab18700 1 -- 172.21.15.46:0/329472520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c009d00 con 0x7f8864101820 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.756+0000 7f886a317700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88640ffc90 0x7f8864195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.756+0000 7f886ab18700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8864101820 0x7f886419d230 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f885c007b60 tx=0x7f885c008e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.756+0000 7f885b7fe700 1 -- 172.21.15.46:0/329472520 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c00e040 con 0x7f8864101820 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.756+0000 7f886c57b700 1 -- 172.21.15.46:0/329472520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88640692b0 con 0x7f8864101820 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.756+0000 7f885b7fe700 1 -- 172.21.15.46:0/329472520 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f885c005070 con 0x7f8864101820 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.756+0000 7f885b7fe700 1 -- 172.21.15.46:0/329472520 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c01d6d0 con 0x7f8864101820 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.756+0000 7f886c57b700 1 -- 172.21.15.46:0/329472520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8864069760 con 0x7f8864101820 2024-03-18T20:34:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.757+0000 7f886c57b700 1 -- 172.21.15.46:0/329472520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8864067f40 con 0x7f8864101820 2024-03-18T20:34:39.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.758+0000 7f885b7fe700 1 -- 172.21.15.46:0/329472520 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f885c0051e0 con 0x7f8864101820 2024-03-18T20:34:39.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.759+0000 7f885b7fe700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f88500a1120 0x7f88500a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:39.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.759+0000 7f886a317700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f88500a1120 0x7f88500a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:39.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.759+0000 7f885b7fe700 1 -- 172.21.15.46:0/329472520 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f885c0cdda0 con 0x7f8864101820 2024-03-18T20:34:39.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.759+0000 7f886a317700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f88500a1120 0x7f88500a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f88640fcd30 tx=0x7f8860010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:39.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:39.763+0000 7f885b7fe700 1 -- 172.21.15.46:0/329472520 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f885c097670 con 0x7f8864101820 2024-03-18T20:34:39.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:39 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3974286303' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-18T20:34:39.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:39 smithi155 ceph-mon[48344]: osdmap e76: 12 total, 12 up, 12 in 2024-03-18T20:34:39.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:39 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1718720699' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-18T20:34:39.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:39 smithi155 ceph-mon[48344]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-18T20:34:39.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:39 smithi155 ceph-mon[48344]: osdmap e77: 12 total, 12 up, 12 in 2024-03-18T20:34:39.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:39 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1718720699' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-18T20:34:39.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:39 smithi155 ceph-mon[48344]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-18T20:34:39.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:39 smithi155 ceph-mon[48344]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-18T20:34:39.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:39 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2459013228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:40.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.042+0000 7f886c57b700 1 -- 172.21.15.46:0/329472520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8864106bb0 con 0x7f8864101820 2024-03-18T20:34:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:40 smithi073 ceph-mon[47384]: pgmap v218: 97 pgs: 21 creating+peering, 2 active+clean, 74 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:40 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/280358737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:40 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] up:active 2024-03-18T20:34:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:40 smithi073 ceph-mon[47384]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T20:34:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:40 smithi073 ceph-mon[47384]: osdmap e78: 12 total, 12 up, 12 in 2024-03-18T20:34:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:40 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/329472520' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T20:34:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:40 smithi073 ceph-mon[47384]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T20:34:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:40 smithi155 ceph-mon[48344]: pgmap v218: 97 pgs: 21 creating+peering, 2 active+clean, 74 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:40 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/280358737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:40 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] up:active 2024-03-18T20:34:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:40 smithi155 ceph-mon[48344]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T20:34:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:40 smithi155 ceph-mon[48344]: osdmap e78: 12 total, 12 up, 12 in 2024-03-18T20:34:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:40 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/329472520' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T20:34:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:40 smithi155 ceph-mon[48344]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T20:34:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:40 smithi046 ceph-mon[43495]: pgmap v218: 97 pgs: 21 creating+peering, 2 active+clean, 74 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:34:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:40 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/280358737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:40 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] up:active 2024-03-18T20:34:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:40 smithi046 ceph-mon[43495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T20:34:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:40 smithi046 ceph-mon[43495]: osdmap e78: 12 total, 12 up, 12 in 2024-03-18T20:34:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:40 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/329472520' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T20:34:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:40 smithi046 ceph-mon[43495]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T20:34:40.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.795+0000 7f885b7fe700 1 -- 172.21.15.46:0/329472520 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f885c09ac90 con 0x7f8864101820 2024-03-18T20:34:40.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.798+0000 7f886c57b700 1 -- 172.21.15.46:0/329472520 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f88500a1120 msgr2=0x7f88500a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:40.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.798+0000 7f886c57b700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f88500a1120 0x7f88500a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f88640fcd30 tx=0x7f8860010040 comp rx=0 tx=0).stop 2024-03-18T20:34:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.798+0000 7f886c57b700 1 -- 172.21.15.46:0/329472520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8864101820 msgr2=0x7f886419d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.798+0000 7f886c57b700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8864101820 0x7f886419d230 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f885c007b60 tx=0x7f885c008e90 comp rx=0 tx=0).stop 2024-03-18T20:34:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.798+0000 7f886c57b700 1 -- 172.21.15.46:0/329472520 shutdown_connections 2024-03-18T20:34:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.798+0000 7f886c57b700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f88500a1120 0x7f88500a35e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.798+0000 7f886c57b700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8864101820 0x7f886419d230 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.798+0000 7f886c57b700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88640ffc90 0x7f8864195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.798+0000 7f886c57b700 1 --2- 172.21.15.46:0/329472520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8864100e80 0x7f8864196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.798+0000 7f886c57b700 1 -- 172.21.15.46:0/329472520 >> 172.21.15.46:0/329472520 conn(0x7f88640fb200 msgr2=0x7f8864104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.798+0000 7f886c57b700 1 -- 172.21.15.46:0/329472520 shutdown_connections 2024-03-18T20:34:40.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.798+0000 7f886c57b700 1 -- 172.21.15.46:0/329472520 wait complete. 2024-03-18T20:34:40.818 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-18T20:34:40.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-18T20:34:40.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.960+0000 7fc08631d700 1 -- 172.21.15.46:0/3369417520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0800ff7c0 msgr2=0x7fc080109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.960+0000 7fc08631d700 1 --2- 172.21.15.46:0/3369417520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0800ff7c0 0x7fc080109390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b0a0 tx=0x7fc07800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.960+0000 7fc08631d700 1 -- 172.21.15.46:0/3369417520 shutdown_connections 2024-03-18T20:34:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.960+0000 7fc08631d700 1 --2- 172.21.15.46:0/3369417520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0800ff7c0 0x7fc080109390 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.960+0000 7fc08631d700 1 --2- 172.21.15.46:0/3369417520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0801023b0 0x7fc0801027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.960+0000 7fc08631d700 1 --2- 172.21.15.46:0/3369417520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0800fee20 0x7fc0800ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.960+0000 7fc08631d700 1 -- 172.21.15.46:0/3369417520 >> 172.21.15.46:0/3369417520 conn(0x7fc0800fa980 msgr2=0x7fc0800fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.960+0000 7fc08631d700 1 -- 172.21.15.46:0/3369417520 shutdown_connections 2024-03-18T20:34:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.960+0000 7fc08631d700 1 -- 172.21.15.46:0/3369417520 wait complete. 2024-03-18T20:34:40.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc08631d700 1 Processor -- start 2024-03-18T20:34:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc08631d700 1 -- start start 2024-03-18T20:34:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc08631d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0800fee20 0x7fc080196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc08631d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0800ff7c0 0x7fc080197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc08631d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0801023b0 0x7fc08019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc08631d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08019e8f0 con 0x7fc0800ff7c0 2024-03-18T20:34:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc08631d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08019ea60 con 0x7fc0801023b0 2024-03-18T20:34:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc08631d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08019ec20 con 0x7fc0800fee20 2024-03-18T20:34:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc07f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0800ff7c0 0x7fc080197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc07f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0800ff7c0 0x7fc080197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44654/0 (socket says 172.21.15.46:44654) 2024-03-18T20:34:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc07f7fe700 1 -- 172.21.15.46:0/3740757492 learned_addr learned my addr 172.21.15.46:0/3740757492 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc07ffff700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0800fee20 0x7fc080196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc0848ba700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0801023b0 0x7fc08019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc07f7fe700 1 -- 172.21.15.46:0/3740757492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0800fee20 msgr2=0x7fc080196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.961+0000 7fc07f7fe700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0800fee20 0x7fc080196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.962+0000 7fc07f7fe700 1 -- 172.21.15.46:0/3740757492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0801023b0 msgr2=0x7fc08019e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.962+0000 7fc07f7fe700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0801023b0 0x7fc08019e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.962+0000 7fc07f7fe700 1 -- 172.21.15.46:0/3740757492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078009d00 con 0x7fc0800ff7c0 2024-03-18T20:34:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.962+0000 7fc07f7fe700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0800ff7c0 0x7fc080197340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc07400d980 tx=0x7fc07400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.962+0000 7fc07d7fa700 1 -- 172.21.15.46:0/3740757492 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074004220 con 0x7fc0800ff7c0 2024-03-18T20:34:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.962+0000 7fc07d7fa700 1 -- 172.21.15.46:0/3740757492 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc074007640 con 0x7fc0800ff7c0 2024-03-18T20:34:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.962+0000 7fc08631d700 1 -- 172.21.15.46:0/3740757492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08019ef00 con 0x7fc0800ff7c0 2024-03-18T20:34:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.962+0000 7fc07d7fa700 1 -- 172.21.15.46:0/3740757492 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074005690 con 0x7fc0800ff7c0 2024-03-18T20:34:40.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.962+0000 7fc08631d700 1 -- 172.21.15.46:0/3740757492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08019f380 con 0x7fc0800ff7c0 2024-03-18T20:34:40.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.963+0000 7fc08631d700 1 -- 172.21.15.46:0/3740757492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc080108300 con 0x7fc0800ff7c0 2024-03-18T20:34:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.964+0000 7fc07d7fa700 1 -- 172.21.15.46:0/3740757492 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc074004750 con 0x7fc0800ff7c0 2024-03-18T20:34:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.965+0000 7fc07d7fa700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc0680a11a0 0x7fc0680a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.965+0000 7fc07d7fa700 1 -- 172.21.15.46:0/3740757492 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc0740cca40 con 0x7fc0800ff7c0 2024-03-18T20:34:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.965+0000 7fc07ffff700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc0680a11a0 0x7fc0680a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:40.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.965+0000 7fc07ffff700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc0680a11a0 0x7fc0680a3660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc080198280 tx=0x7fc07000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:40.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:40.968+0000 7fc07d7fa700 1 -- 172.21.15.46:0/3740757492 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc074096320 con 0x7fc0800ff7c0 2024-03-18T20:34:41.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.234+0000 7fc08631d700 1 -- 172.21.15.46:0/3740757492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fc080067f40 con 0x7fc0800ff7c0 2024-03-18T20:34:41.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.800+0000 7fc07d7fa700 1 -- 172.21.15.46:0/3740757492 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) v1 ==== 105+0+0 (secure 0 0 0) 0x7fc074016070 con 0x7fc0800ff7c0 2024-03-18T20:34:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.803+0000 7fc08631d700 1 -- 172.21.15.46:0/3740757492 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc0680a11a0 msgr2=0x7fc0680a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.803+0000 7fc08631d700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc0680a11a0 0x7fc0680a3660 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc080198280 tx=0x7fc07000b410 comp rx=0 tx=0).stop 2024-03-18T20:34:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.803+0000 7fc08631d700 1 -- 172.21.15.46:0/3740757492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0800ff7c0 msgr2=0x7fc080197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.803+0000 7fc08631d700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0800ff7c0 0x7fc080197340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc07400d980 tx=0x7fc07400dc90 comp rx=0 tx=0).stop 2024-03-18T20:34:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.803+0000 7fc08631d700 1 -- 172.21.15.46:0/3740757492 shutdown_connections 2024-03-18T20:34:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.803+0000 7fc08631d700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc0680a11a0 0x7fc0680a3660 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.803+0000 7fc08631d700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0801023b0 0x7fc08019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.803+0000 7fc08631d700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0800fee20 0x7fc080196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.803+0000 7fc08631d700 1 --2- 172.21.15.46:0/3740757492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0800ff7c0 0x7fc080197340 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.803+0000 7fc08631d700 1 -- 172.21.15.46:0/3740757492 >> 172.21.15.46:0/3740757492 conn(0x7fc0800fa980 msgr2=0x7fc080105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.804+0000 7fc08631d700 1 -- 172.21.15.46:0/3740757492 shutdown_connections 2024-03-18T20:34:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.804+0000 7fc08631d700 1 -- 172.21.15.46:0/3740757492 wait complete. 2024-03-18T20:34:41.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-18T20:34:41.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.968+0000 7fc0573dd700 1 -- 172.21.15.46:0/1732202069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050108b60 msgr2=0x7fc05010afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:41.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.968+0000 7fc0573dd700 1 --2- 172.21.15.46:0/1732202069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050108b60 0x7fc05010afd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc04800b0a0 tx=0x7fc04800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:41.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.968+0000 7fc0573dd700 1 -- 172.21.15.46:0/1732202069 shutdown_connections 2024-03-18T20:34:41.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.968+0000 7fc0573dd700 1 --2- 172.21.15.46:0/1732202069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050108b60 0x7fc05010afd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:41.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.968+0000 7fc0573dd700 1 --2- 172.21.15.46:0/1732202069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc050074d90 0x7fc0500731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:41.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.968+0000 7fc0573dd700 1 --2- 172.21.15.46:0/1732202069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0500737b0 0x7fc050073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:41.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.968+0000 7fc0573dd700 1 -- 172.21.15.46:0/1732202069 >> 172.21.15.46:0/1732202069 conn(0x7fc0500fc200 msgr2=0x7fc0500fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:41.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.968+0000 7fc0573dd700 1 -- 172.21.15.46:0/1732202069 shutdown_connections 2024-03-18T20:34:41.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.968+0000 7fc0573dd700 1 -- 172.21.15.46:0/1732202069 wait complete. 2024-03-18T20:34:41.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc0573dd700 1 Processor -- start 2024-03-18T20:34:41.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc0573dd700 1 -- start start 2024-03-18T20:34:41.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc0573dd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0500737b0 0x7fc050198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:41.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc0573dd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc050074d90 0x7fc050199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc0573dd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050108b60 0x7fc0501a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc0573dd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0501a0a10 con 0x7fc050108b60 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc0573dd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0501a0b50 con 0x7fc0500737b0 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc0573dd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0501a0ce0 con 0x7fc050074d90 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc05597a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050108b60 0x7fc0501a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc05597a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050108b60 0x7fc0501a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44676/0 (socket says 172.21.15.46:44676) 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc05597a700 1 -- 172.21.15.46:0/401219124 learned_addr learned my addr 172.21.15.46:0/401219124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc055179700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0500737b0 0x7fc050198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc054978700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc050074d90 0x7fc050199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc05597a700 1 -- 172.21.15.46:0/401219124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc050074d90 msgr2=0x7fc050199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc05597a700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc050074d90 0x7fc050199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc05597a700 1 -- 172.21.15.46:0/401219124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0500737b0 msgr2=0x7fc050198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc05597a700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0500737b0 0x7fc050198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc05597a700 1 -- 172.21.15.46:0/401219124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc048009d00 con 0x7fc050108b60 2024-03-18T20:34:41.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.969+0000 7fc05597a700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050108b60 0x7fc0501a0440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc048007ab0 tx=0x7fc048004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:41.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.970+0000 7fc0467fc700 1 -- 172.21.15.46:0/401219124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800e040 con 0x7fc050108b60 2024-03-18T20:34:41.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.970+0000 7fc0467fc700 1 -- 172.21.15.46:0/401219124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc048004e70 con 0x7fc050108b60 2024-03-18T20:34:41.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.970+0000 7fc0573dd700 1 -- 172.21.15.46:0/401219124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0501a0f60 con 0x7fc050108b60 2024-03-18T20:34:41.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.970+0000 7fc0467fc700 1 -- 172.21.15.46:0/401219124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04801da70 con 0x7fc050108b60 2024-03-18T20:34:41.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.970+0000 7fc0573dd700 1 -- 172.21.15.46:0/401219124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0501a1380 con 0x7fc050108b60 2024-03-18T20:34:41.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.971+0000 7fc0573dd700 1 -- 172.21.15.46:0/401219124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc050107970 con 0x7fc050108b60 2024-03-18T20:34:41.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.972+0000 7fc0467fc700 1 -- 172.21.15.46:0/401219124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc0480058a0 con 0x7fc050108b60 2024-03-18T20:34:41.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.973+0000 7fc0467fc700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc03c0a1190 0x7fc03c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:41.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.973+0000 7fc055179700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc03c0a1190 0x7fc03c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:41.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.973+0000 7fc0467fc700 1 -- 172.21.15.46:0/401219124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc0480ce0b0 con 0x7fc050108b60 2024-03-18T20:34:41.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.973+0000 7fc055179700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc03c0a1190 0x7fc03c0a3650 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc04c005950 tx=0x7fc04c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:41.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:41.977+0000 7fc0467fc700 1 -- 172.21.15.46:0/401219124 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc048097a20 con 0x7fc050108b60 2024-03-18T20:34:42.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:41 smithi073 ceph-mon[47384]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-18T20:34:42.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:41 smithi073 ceph-mon[47384]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T20:34:42.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:41 smithi073 ceph-mon[47384]: pgmap v220: 97 pgs: 21 creating+peering, 33 active+clean, 43 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-18T20:34:42.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:41 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3740757492' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-18T20:34:42.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.246+0000 7fc0573dd700 1 -- 172.21.15.46:0/401219124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fc050067180 con 0x7fc050108b60 2024-03-18T20:34:42.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:41 smithi155 ceph-mon[48344]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-18T20:34:42.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:41 smithi155 ceph-mon[48344]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T20:34:42.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:41 smithi155 ceph-mon[48344]: pgmap v220: 97 pgs: 21 creating+peering, 33 active+clean, 43 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-18T20:34:42.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:41 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3740757492' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-18T20:34:42.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:41 smithi046 ceph-mon[43495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-18T20:34:42.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:41 smithi046 ceph-mon[43495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T20:34:42.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:41 smithi046 ceph-mon[43495]: pgmap v220: 97 pgs: 21 creating+peering, 33 active+clean, 43 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-03-18T20:34:42.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:41 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3740757492' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-18T20:34:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.805+0000 7fc0467fc700 1 -- 172.21.15.46:0/401219124 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7fc04809b040 con 0x7fc050108b60 2024-03-18T20:34:42.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.809+0000 7fc0573dd700 1 -- 172.21.15.46:0/401219124 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc03c0a1190 msgr2=0x7fc03c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:42.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.809+0000 7fc0573dd700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc03c0a1190 0x7fc03c0a3650 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc04c005950 tx=0x7fc04c00b410 comp rx=0 tx=0).stop 2024-03-18T20:34:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.809+0000 7fc0573dd700 1 -- 172.21.15.46:0/401219124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050108b60 msgr2=0x7fc0501a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.809+0000 7fc0573dd700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050108b60 0x7fc0501a0440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc048007ab0 tx=0x7fc048004c80 comp rx=0 tx=0).stop 2024-03-18T20:34:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.809+0000 7fc0573dd700 1 -- 172.21.15.46:0/401219124 shutdown_connections 2024-03-18T20:34:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.809+0000 7fc0573dd700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc03c0a1190 0x7fc03c0a3650 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.809+0000 7fc0573dd700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050108b60 0x7fc0501a0440 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.809+0000 7fc0573dd700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0500737b0 0x7fc050198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.809+0000 7fc0573dd700 1 --2- 172.21.15.46:0/401219124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc050074d90 0x7fc050199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.809+0000 7fc0573dd700 1 -- 172.21.15.46:0/401219124 >> 172.21.15.46:0/401219124 conn(0x7fc0500fc200 msgr2=0x7fc050104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.809+0000 7fc0573dd700 1 -- 172.21.15.46:0/401219124 shutdown_connections 2024-03-18T20:34:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.809+0000 7fc0573dd700 1 -- 172.21.15.46:0/401219124 wait complete. 2024-03-18T20:34:42.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-18T20:34:42.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.972+0000 7fdb6447a700 1 -- 172.21.15.46:0/334269494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c104110 msgr2=0x7fdb5c106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.972+0000 7fdb6447a700 1 --2- 172.21.15.46:0/334269494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c104110 0x7fdb5c106500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009990 tx=0x7fdb4c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:34:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.972+0000 7fdb6447a700 1 -- 172.21.15.46:0/334269494 shutdown_connections 2024-03-18T20:34:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.972+0000 7fdb6447a700 1 --2- 172.21.15.46:0/334269494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c106a40 0x7fdb5c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.972+0000 7fdb6447a700 1 --2- 172.21.15.46:0/334269494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c1017e0 0x7fdb5c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.972+0000 7fdb6447a700 1 --2- 172.21.15.46:0/334269494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c104110 0x7fdb5c106500 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.972+0000 7fdb6447a700 1 -- 172.21.15.46:0/334269494 >> 172.21.15.46:0/334269494 conn(0x7fdb5c0fb1d0 msgr2=0x7fdb5c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.972+0000 7fdb6447a700 1 -- 172.21.15.46:0/334269494 shutdown_connections 2024-03-18T20:34:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.972+0000 7fdb6447a700 1 -- 172.21.15.46:0/334269494 wait complete. 2024-03-18T20:34:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb6447a700 1 Processor -- start 2024-03-18T20:34:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb6447a700 1 -- start start 2024-03-18T20:34:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb6447a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c1017e0 0x7fdb5c196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb6447a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c104110 0x7fdb5c197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb6447a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c106a40 0x7fdb5c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb6447a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19e760 con 0x7fdb5c104110 2024-03-18T20:34:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb6447a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19e8a0 con 0x7fdb5c1017e0 2024-03-18T20:34:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb6447a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19ea30 con 0x7fdb5c106a40 2024-03-18T20:34:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb61a15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c104110 0x7fdb5c197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb61a15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c104110 0x7fdb5c197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44688/0 (socket says 172.21.15.46:44688) 2024-03-18T20:34:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb61a15700 1 -- 172.21.15.46:0/4157129164 learned_addr learned my addr 172.21.15.46:0/4157129164 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb62216700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c1017e0 0x7fdb5c196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.973+0000 7fdb62a17700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c106a40 0x7fdb5c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:42.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.974+0000 7fdb61a15700 1 -- 172.21.15.46:0/4157129164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c106a40 msgr2=0x7fdb5c19e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.974+0000 7fdb61a15700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c106a40 0x7fdb5c19e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.974+0000 7fdb61a15700 1 -- 172.21.15.46:0/4157129164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c1017e0 msgr2=0x7fdb5c196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.974+0000 7fdb61a15700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c1017e0 0x7fdb5c196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.974+0000 7fdb61a15700 1 -- 172.21.15.46:0/4157129164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c0095f0 con 0x7fdb5c104110 2024-03-18T20:34:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.974+0000 7fdb62216700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c1017e0 0x7fdb5c196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:34:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.974+0000 7fdb61a15700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c104110 0x7fdb5c197170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0098e0 tx=0x7fdb4c0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.974+0000 7fdb537fe700 1 -- 172.21.15.46:0/4157129164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c01d070 con 0x7fdb5c104110 2024-03-18T20:34:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.974+0000 7fdb537fe700 1 -- 172.21.15.46:0/4157129164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb4c00fb80 con 0x7fdb5c104110 2024-03-18T20:34:42.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.974+0000 7fdb6447a700 1 -- 172.21.15.46:0/4157129164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c19ecb0 con 0x7fdb5c104110 2024-03-18T20:34:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.974+0000 7fdb537fe700 1 -- 172.21.15.46:0/4157129164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c017650 con 0x7fdb5c104110 2024-03-18T20:34:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.974+0000 7fdb6447a700 1 -- 172.21.15.46:0/4157129164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c19f0d0 con 0x7fdb5c104110 2024-03-18T20:34:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.975+0000 7fdb6447a700 1 -- 172.21.15.46:0/4157129164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c0fcd50 con 0x7fdb5c104110 2024-03-18T20:34:42.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.976+0000 7fdb537fe700 1 -- 172.21.15.46:0/4157129164 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fdb4c00fcf0 con 0x7fdb5c104110 2024-03-18T20:34:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.977+0000 7fdb537fe700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdb480a1120 0x7fdb480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.977+0000 7fdb62216700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdb480a1120 0x7fdb480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.977+0000 7fdb537fe700 1 -- 172.21.15.46:0/4157129164 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdb4c0ce4f0 con 0x7fdb5c104110 2024-03-18T20:34:42.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.978+0000 7fdb62216700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdb480a1120 0x7fdb480a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb58005fd0 tx=0x7fdb58005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:42.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:42.981+0000 7fdb537fe700 1 -- 172.21.15.46:0/4157129164 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdb4c097e60 con 0x7fdb5c104110 2024-03-18T20:34:43.220 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 20:34:43 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: 2024-03-18T20:34:43.033+0000 7f60b60db700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-18T20:34:43.220 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 20:34:43 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: 2024-03-18T20:34:43.033+0000 7f60b60db700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-18T20:34:43.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:42 smithi073 ceph-mon[47384]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T20:34:43.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:42 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3740757492' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-18T20:34:43.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:42 smithi073 ceph-mon[47384]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-18T20:34:43.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:42 smithi073 ceph-mon[47384]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T20:34:43.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:42 smithi073 ceph-mon[47384]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-03-18T20:34:43.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:42 smithi073 ceph-mon[47384]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-03-18T20:34:43.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:42 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/401219124' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T20:34:43.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.250+0000 7fdb6447a700 1 -- 172.21.15.46:0/4157129164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fdb5c02d0f0 con 0x7fdb5c104110 2024-03-18T20:34:43.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:42 smithi155 ceph-mon[48344]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T20:34:43.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:42 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3740757492' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-18T20:34:43.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:42 smithi155 ceph-mon[48344]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-18T20:34:43.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:42 smithi155 ceph-mon[48344]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T20:34:43.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:42 smithi155 ceph-mon[48344]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-03-18T20:34:43.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:42 smithi155 ceph-mon[48344]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-03-18T20:34:43.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:42 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/401219124' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T20:34:43.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:42 smithi046 ceph-mon[43495]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T20:34:43.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:42 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3740757492' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-18T20:34:43.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:42 smithi046 ceph-mon[43495]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-18T20:34:43.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:42 smithi046 ceph-mon[43495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-18T20:34:43.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:42 smithi046 ceph-mon[43495]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-03-18T20:34:43.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:42 smithi046 ceph-mon[43495]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-03-18T20:34:43.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:42 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/401219124' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T20:34:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.814+0000 7fdb537fe700 1 -- 172.21.15.46:0/4157129164 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7fdb4c0270b0 con 0x7fdb5c104110 2024-03-18T20:34:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.817+0000 7fdb6447a700 1 -- 172.21.15.46:0/4157129164 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdb480a1120 msgr2=0x7fdb480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.817+0000 7fdb6447a700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdb480a1120 0x7fdb480a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb58005fd0 tx=0x7fdb58005e30 comp rx=0 tx=0).stop 2024-03-18T20:34:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.817+0000 7fdb6447a700 1 -- 172.21.15.46:0/4157129164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c104110 msgr2=0x7fdb5c197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.817+0000 7fdb6447a700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c104110 0x7fdb5c197170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0098e0 tx=0x7fdb4c0057d0 comp rx=0 tx=0).stop 2024-03-18T20:34:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.817+0000 7fdb6447a700 1 -- 172.21.15.46:0/4157129164 shutdown_connections 2024-03-18T20:34:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.817+0000 7fdb6447a700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdb480a1120 0x7fdb480a35e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.817+0000 7fdb6447a700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb5c106a40 0x7fdb5c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.817+0000 7fdb6447a700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c1017e0 0x7fdb5c196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:43.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.817+0000 7fdb6447a700 1 --2- 172.21.15.46:0/4157129164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c104110 0x7fdb5c197170 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:43.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.817+0000 7fdb6447a700 1 -- 172.21.15.46:0/4157129164 >> 172.21.15.46:0/4157129164 conn(0x7fdb5c0fb1d0 msgr2=0x7fdb5c1022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:43.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.818+0000 7fdb6447a700 1 -- 172.21.15.46:0/4157129164 shutdown_connections 2024-03-18T20:34:43.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.818+0000 7fdb6447a700 1 -- 172.21.15.46:0/4157129164 wait complete. 2024-03-18T20:34:43.841 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-03-18T20:34:43.841 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-18T20:34:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.981+0000 7f5a0049f700 1 -- 172.21.15.46:0/105777940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f8100e80 msgr2=0x7f59f81012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.981+0000 7f5a0049f700 1 --2- 172.21.15.46:0/105777940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f8100e80 0x7f59f81012e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f59e80099c0 tx=0x7f59e8009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.981+0000 7f5a0049f700 1 -- 172.21.15.46:0/105777940 shutdown_connections 2024-03-18T20:34:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.981+0000 7f5a0049f700 1 --2- 172.21.15.46:0/105777940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59f8101820 0x7f59f81080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.981+0000 7f5a0049f700 1 --2- 172.21.15.46:0/105777940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59f80ffc90 0x7f59f81000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.981+0000 7f5a0049f700 1 --2- 172.21.15.46:0/105777940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f8100e80 0x7f59f81012e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:43.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.981+0000 7f5a0049f700 1 -- 172.21.15.46:0/105777940 >> 172.21.15.46:0/105777940 conn(0x7f59f80fb200 msgr2=0x7f59f80fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:43.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.981+0000 7f5a0049f700 1 -- 172.21.15.46:0/105777940 shutdown_connections 2024-03-18T20:34:43.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.981+0000 7f5a0049f700 1 -- 172.21.15.46:0/105777940 wait complete. 2024-03-18T20:34:43.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.981+0000 7f5a0049f700 1 Processor -- start 2024-03-18T20:34:43.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.981+0000 7f5a0049f700 1 -- start start 2024-03-18T20:34:43.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f5a0049f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59f80ffc90 0x7f59f8196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:43.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f5a0049f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59f8100e80 0x7f59f8197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:43.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f5a0049f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f8101820 0x7f59f819e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:43.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f5a0049f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f819e820 con 0x7f59f8101820 2024-03-18T20:34:43.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f5a0049f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f819e990 con 0x7f59f80ffc90 2024-03-18T20:34:43.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f5a0049f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f819eb50 con 0x7f59f8100e80 2024-03-18T20:34:43.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59fea3c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f8101820 0x7f59f819e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:43.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59fea3c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f8101820 0x7f59f819e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44706/0 (socket says 172.21.15.46:44706) 2024-03-18T20:34:43.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59fea3c700 1 -- 172.21.15.46:0/1410467876 learned_addr learned my addr 172.21.15.46:0/1410467876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:43.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59fe23b700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59f80ffc90 0x7f59f8196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:43.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59fda3a700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59f8100e80 0x7f59f8197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:43.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59fea3c700 1 -- 172.21.15.46:0/1410467876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59f8100e80 msgr2=0x7f59f8197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:43.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59fea3c700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59f8100e80 0x7f59f8197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:43.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59fea3c700 1 -- 172.21.15.46:0/1410467876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59f80ffc90 msgr2=0x7f59f8196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:43.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59fea3c700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59f80ffc90 0x7f59f8196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:43.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59fea3c700 1 -- 172.21.15.46:0/1410467876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e80095f0 con 0x7f59f8101820 2024-03-18T20:34:43.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59fda3a700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59f8100e80 0x7f59f8197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:34:43.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59fea3c700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f8101820 0x7f59f819e250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f59f000ef30 tx=0x7f59f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:43.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.982+0000 7f59ef7fe700 1 -- 172.21.15.46:0/1410467876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0018070 con 0x7f59f8101820 2024-03-18T20:34:43.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.983+0000 7f59ef7fe700 1 -- 172.21.15.46:0/1410467876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59f000f040 con 0x7f59f8101820 2024-03-18T20:34:43.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.983+0000 7f5a0049f700 1 -- 172.21.15.46:0/1410467876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f819edd0 con 0x7f59f8101820 2024-03-18T20:34:43.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.983+0000 7f59ef7fe700 1 -- 172.21.15.46:0/1410467876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f000cdb0 con 0x7f59f8101820 2024-03-18T20:34:43.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.983+0000 7f5a0049f700 1 -- 172.21.15.46:0/1410467876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f819f230 con 0x7f59f8101820 2024-03-18T20:34:43.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.984+0000 7f5a0049f700 1 -- 172.21.15.46:0/1410467876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f8067260 con 0x7f59f8101820 2024-03-18T20:34:43.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.985+0000 7f59ef7fe700 1 -- 172.21.15.46:0/1410467876 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f59f0003a00 con 0x7f59f8101820 2024-03-18T20:34:43.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.986+0000 7f59ef7fe700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f59e40a1120 0x7f59e40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:43.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.986+0000 7f59fe23b700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f59e40a1120 0x7f59e40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:43.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.986+0000 7f59ef7fe700 1 -- 172.21.15.46:0/1410467876 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f59f00cccd0 con 0x7f59f8101820 2024-03-18T20:34:43.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.986+0000 7f59fe23b700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f59e40a1120 0x7f59e40a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f59f80694d0 tx=0x7f59f400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:43.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:43.991+0000 7f59ef7fe700 1 -- 172.21.15.46:0/1410467876 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f59f0096640 con 0x7f59f8101820 2024-03-18T20:34:44.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:43 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] up:active 2024-03-18T20:34:44.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:43 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/401219124' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T20:34:44.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:43 smithi073 ceph-mon[47384]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-18T20:34:44.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:43 smithi073 ceph-mon[47384]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-03-18T20:34:44.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:43 smithi073 ceph-mon[47384]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-03-18T20:34:44.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:43 smithi073 ceph-mon[47384]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-18T20:34:44.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:43 smithi073 ceph-mon[47384]: pgmap v221: 97 pgs: 15 creating+peering, 76 active+clean, 6 unknown; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 5 op/s 2024-03-18T20:34:44.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:43 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/4157129164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-18T20:34:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.237+0000 7f5a0049f700 1 -- 172.21.15.46:0/1410467876 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f59f819f420 con 0x7f59e40a1120 2024-03-18T20:34:44.248 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:43 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] up:active 2024-03-18T20:34:44.248 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:43 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/401219124' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T20:34:44.248 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:43 smithi046 ceph-mon[43495]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-18T20:34:44.248 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:43 smithi046 ceph-mon[43495]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-03-18T20:34:44.248 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:43 smithi046 ceph-mon[43495]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-03-18T20:34:44.248 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:43 smithi046 ceph-mon[43495]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-18T20:34:44.249 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:43 smithi046 ceph-mon[43495]: pgmap v221: 97 pgs: 15 creating+peering, 76 active+clean, 6 unknown; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 5 op/s 2024-03-18T20:34:44.249 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:43 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/4157129164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-18T20:34:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.261+0000 7f59ef7fe700 1 -- 172.21.15.46:0/1410467876 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f59f819f420 con 0x7f59e40a1120 2024-03-18T20:34:44.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.263+0000 7f5a0049f700 1 -- 172.21.15.46:0/1410467876 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f59e40a1120 msgr2=0x7f59e40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:44.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.263+0000 7f5a0049f700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f59e40a1120 0x7f59e40a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f59f80694d0 tx=0x7f59f400b410 comp rx=0 tx=0).stop 2024-03-18T20:34:44.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.263+0000 7f5a0049f700 1 -- 172.21.15.46:0/1410467876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f8101820 msgr2=0x7f59f819e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.263+0000 7f5a0049f700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f8101820 0x7f59f819e250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f59f000ef30 tx=0x7f59f000c3a0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.263+0000 7f5a0049f700 1 -- 172.21.15.46:0/1410467876 shutdown_connections 2024-03-18T20:34:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.263+0000 7f5a0049f700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f59e40a1120 0x7f59e40a35e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.263+0000 7f5a0049f700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f8101820 0x7f59f819e250 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.263+0000 7f5a0049f700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59f80ffc90 0x7f59f8196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.263+0000 7f5a0049f700 1 --2- 172.21.15.46:0/1410467876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59f8100e80 0x7f59f8197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.263+0000 7f5a0049f700 1 -- 172.21.15.46:0/1410467876 >> 172.21.15.46:0/1410467876 conn(0x7f59f80fb200 msgr2=0x7f59f80fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.263+0000 7f5a0049f700 1 -- 172.21.15.46:0/1410467876 shutdown_connections 2024-03-18T20:34:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.263+0000 7f5a0049f700 1 -- 172.21.15.46:0/1410467876 wait complete. 2024-03-18T20:34:44.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-18T20:34:44.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:43 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] up:active 2024-03-18T20:34:44.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:43 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/401219124' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T20:34:44.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:43 smithi155 ceph-mon[48344]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-18T20:34:44.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:43 smithi155 ceph-mon[48344]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-03-18T20:34:44.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:43 smithi155 ceph-mon[48344]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-03-18T20:34:44.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:43 smithi155 ceph-mon[48344]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-18T20:34:44.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:43 smithi155 ceph-mon[48344]: pgmap v221: 97 pgs: 15 creating+peering, 76 active+clean, 6 unknown; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 5 op/s 2024-03-18T20:34:44.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:43 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/4157129164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-18T20:34:44.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.421+0000 7ff30997e700 1 -- 172.21.15.46:0/3296601279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3041001b0 msgr2=0x7ff3041005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:44.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.421+0000 7ff30997e700 1 --2- 172.21.15.46:0/3296601279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3041001b0 0x7ff3041005d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009990 tx=0x7ff2f4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.421+0000 7ff30997e700 1 -- 172.21.15.46:0/3296601279 shutdown_connections 2024-03-18T20:34:44.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.421+0000 7ff30997e700 1 --2- 172.21.15.46:0/3296601279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3041001b0 0x7ff3041005d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.421+0000 7ff30997e700 1 --2- 172.21.15.46:0/3296601279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff304101880 0x7ff304108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.421+0000 7ff30997e700 1 --2- 172.21.15.46:0/3296601279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff304100ec0 0x7ff304101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.421+0000 7ff30997e700 1 -- 172.21.15.46:0/3296601279 >> 172.21.15.46:0/3296601279 conn(0x7ff3040fa980 msgr2=0x7ff3040fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:44.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.421+0000 7ff30997e700 1 -- 172.21.15.46:0/3296601279 shutdown_connections 2024-03-18T20:34:44.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.421+0000 7ff30997e700 1 -- 172.21.15.46:0/3296601279 wait complete. 2024-03-18T20:34:44.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.422+0000 7ff30997e700 1 Processor -- start 2024-03-18T20:34:44.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.422+0000 7ff30997e700 1 -- start start 2024-03-18T20:34:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.422+0000 7ff30997e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3041001b0 0x7ff3040ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.422+0000 7ff30997e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff304100ec0 0x7ff30419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.422+0000 7ff30997e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff304101880 0x7ff30419e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:44.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.422+0000 7ff30997e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30419e820 con 0x7ff304100ec0 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.422+0000 7ff30997e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30419e990 con 0x7ff3041001b0 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.422+0000 7ff30997e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30419eb50 con 0x7ff304101880 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff2fa7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff304100ec0 0x7ff30419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff2fa7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff304100ec0 0x7ff30419ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44748/0 (socket says 172.21.15.46:44748) 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff2fa7fc700 1 -- 172.21.15.46:0/1025785232 learned_addr learned my addr 172.21.15.46:0/1025785232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff3037fe700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff304101880 0x7ff30419e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff302ffd700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3041001b0 0x7ff3040ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff2fa7fc700 1 -- 172.21.15.46:0/1025785232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff304101880 msgr2=0x7ff30419e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff2fa7fc700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff304101880 0x7ff30419e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff2fa7fc700 1 -- 172.21.15.46:0/1025785232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3041001b0 msgr2=0x7ff3040ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff2fa7fc700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3041001b0 0x7ff3040ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff2fa7fc700 1 -- 172.21.15.46:0/1025785232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f40095f0 con 0x7ff304100ec0 2024-03-18T20:34:44.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff2fa7fc700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff304100ec0 0x7ff30419ba90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00d980 tx=0x7ff2ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff30897c700 1 -- 172.21.15.46:0/1025785232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec004020 con 0x7ff304100ec0 2024-03-18T20:34:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff30897c700 1 -- 172.21.15.46:0/1025785232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2ec00f690 con 0x7ff304100ec0 2024-03-18T20:34:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff30997e700 1 -- 172.21.15.46:0/1025785232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30419ee30 con 0x7ff304100ec0 2024-03-18T20:34:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff30897c700 1 -- 172.21.15.46:0/1025785232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec003a60 con 0x7ff304100ec0 2024-03-18T20:34:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.423+0000 7ff30997e700 1 -- 172.21.15.46:0/1025785232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30419f2b0 con 0x7ff304100ec0 2024-03-18T20:34:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.424+0000 7ff30997e700 1 -- 172.21.15.46:0/1025785232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3041070a0 con 0x7ff304100ec0 2024-03-18T20:34:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.426+0000 7ff30897c700 1 -- 172.21.15.46:0/1025785232 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7ff2ec00fb00 con 0x7ff304100ec0 2024-03-18T20:34:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.426+0000 7ff30897c700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff2f00a1120 0x7ff2f00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.426+0000 7ff302ffd700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff2f00a1120 0x7ff2f00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.426+0000 7ff30897c700 1 -- 172.21.15.46:0/1025785232 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff2ec0cc300 con 0x7ff304100ec0 2024-03-18T20:34:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.427+0000 7ff302ffd700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff2f00a1120 0x7ff2f00a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.431+0000 7ff30897c700 1 -- 172.21.15.46:0/1025785232 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff2ec095be0 con 0x7ff304100ec0 2024-03-18T20:34:44.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:34:44.242+0000 7f805a0af700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:34:44.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:34:44.242+0000 7f805a0af700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:34:44.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:34:44.242+0000 7f805a0af700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:34:44.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:34:44.242+0000 7f805a0af700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:34:44.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:34:44.242+0000 7f805a0af700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:34:44.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.689+0000 7ff30997e700 1 -- 172.21.15.46:0/1025785232 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3040611f0 con 0x7ff2f00a1120 2024-03-18T20:34:44.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.711+0000 7ff30897c700 1 -- 172.21.15.46:0/1025785232 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff3040611f0 con 0x7ff2f00a1120 2024-03-18T20:34:44.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.714+0000 7ff30997e700 1 -- 172.21.15.46:0/1025785232 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff2f00a1120 msgr2=0x7ff2f00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.714+0000 7ff30997e700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff2f00a1120 0x7ff2f00a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f4005fb0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.714+0000 7ff30997e700 1 -- 172.21.15.46:0/1025785232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff304100ec0 msgr2=0x7ff30419ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.714+0000 7ff30997e700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff304100ec0 0x7ff30419ba90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00d980 tx=0x7ff2ec00dc90 comp rx=0 tx=0).stop 2024-03-18T20:34:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.714+0000 7ff30997e700 1 -- 172.21.15.46:0/1025785232 shutdown_connections 2024-03-18T20:34:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.714+0000 7ff30997e700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff2f00a1120 0x7ff2f00a35e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.714+0000 7ff30997e700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff304101880 0x7ff30419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.714+0000 7ff30997e700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3041001b0 0x7ff3040ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.714+0000 7ff30997e700 1 --2- 172.21.15.46:0/1025785232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff304100ec0 0x7ff30419ba90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.714+0000 7ff30997e700 1 -- 172.21.15.46:0/1025785232 >> 172.21.15.46:0/1025785232 conn(0x7ff3040fa980 msgr2=0x7ff3041041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.714+0000 7ff30997e700 1 -- 172.21.15.46:0/1025785232 shutdown_connections 2024-03-18T20:34:44.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.714+0000 7ff30997e700 1 -- 172.21.15.46:0/1025785232 wait complete. 2024-03-18T20:34:44.732 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-18T20:34:44.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.876+0000 7fc5be7b1700 1 -- 172.21.15.46:0/857058821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8101f70 msgr2=0x7fc5b81023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:44.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.876+0000 7fc5be7b1700 1 --2- 172.21.15.46:0/857058821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8101f70 0x7fc5b81023d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0099c0 tx=0x7fc5ac009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.876+0000 7fc5be7b1700 1 -- 172.21.15.46:0/857058821 shutdown_connections 2024-03-18T20:34:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.876+0000 7fc5be7b1700 1 --2- 172.21.15.46:0/857058821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5b8102910 0x7fc5b81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.876+0000 7fc5be7b1700 1 --2- 172.21.15.46:0/857058821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b8100d80 0x7fc5b81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.876+0000 7fc5be7b1700 1 --2- 172.21.15.46:0/857058821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8101f70 0x7fc5b81023d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.876+0000 7fc5be7b1700 1 -- 172.21.15.46:0/857058821 >> 172.21.15.46:0/857058821 conn(0x7fc5b80fc2f0 msgr2=0x7fc5b80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.876+0000 7fc5be7b1700 1 -- 172.21.15.46:0/857058821 shutdown_connections 2024-03-18T20:34:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.876+0000 7fc5be7b1700 1 -- 172.21.15.46:0/857058821 wait complete. 2024-03-18T20:34:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.877+0000 7fc5be7b1700 1 Processor -- start 2024-03-18T20:34:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.877+0000 7fc5be7b1700 1 -- start start 2024-03-18T20:34:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.877+0000 7fc5be7b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b8100d80 0x7fc5b8196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.877+0000 7fc5be7b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8101f70 0x7fc5b81971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.877+0000 7fc5be7b1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5b8102910 0x7fc5b819e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.877+0000 7fc5be7b1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b819e790 con 0x7fc5b8101f70 2024-03-18T20:34:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.877+0000 7fc5be7b1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b819e900 con 0x7fc5b8100d80 2024-03-18T20:34:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.877+0000 7fc5be7b1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b819eac0 con 0x7fc5b8102910 2024-03-18T20:34:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8101f70 0x7fc5b81971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8101f70 0x7fc5b81971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44766/0 (socket says 172.21.15.46:44766) 2024-03-18T20:34:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b77fe700 1 -- 172.21.15.46:0/2299073814 learned_addr learned my addr 172.21.15.46:0/2299073814 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5bcd4e700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5b8102910 0x7fc5b819e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b7fff700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b8100d80 0x7fc5b8196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b77fe700 1 -- 172.21.15.46:0/2299073814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5b8102910 msgr2=0x7fc5b819e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b77fe700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5b8102910 0x7fc5b819e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b77fe700 1 -- 172.21.15.46:0/2299073814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b8100d80 msgr2=0x7fc5b8196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b77fe700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b8100d80 0x7fc5b8196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b77fe700 1 -- 172.21.15.46:0/2299073814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac0095f0 con 0x7fc5b8101f70 2024-03-18T20:34:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b77fe700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8101f70 0x7fc5b81971e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0036f0 tx=0x7fc5ac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b57fa700 1 -- 172.21.15.46:0/2299073814 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac01d070 con 0x7fc5b8101f70 2024-03-18T20:34:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b57fa700 1 -- 172.21.15.46:0/2299073814 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5ac004090 con 0x7fc5b8101f70 2024-03-18T20:34:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5be7b1700 1 -- 172.21.15.46:0/2299073814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b819ed40 con 0x7fc5b8101f70 2024-03-18T20:34:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5b57fa700 1 -- 172.21.15.46:0/2299073814 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac022d40 con 0x7fc5b8101f70 2024-03-18T20:34:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.878+0000 7fc5be7b1700 1 -- 172.21.15.46:0/2299073814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b819f1a0 con 0x7fc5b8101f70 2024-03-18T20:34:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.879+0000 7fc5be7b1700 1 -- 172.21.15.46:0/2299073814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b8108200 con 0x7fc5b8101f70 2024-03-18T20:34:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.880+0000 7fc5b57fa700 1 -- 172.21.15.46:0/2299073814 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc5ac022470 con 0x7fc5b8101f70 2024-03-18T20:34:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.881+0000 7fc5b57fa700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc5a00a1120 0x7fc5a00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.881+0000 7fc5b57fa700 1 -- 172.21.15.46:0/2299073814 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc5ac0cdc80 con 0x7fc5b8101f70 2024-03-18T20:34:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.881+0000 7fc5b7fff700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc5a00a1120 0x7fc5a00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.882+0000 7fc5b7fff700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc5a00a1120 0x7fc5a00a35e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8198120 tx=0x7fc5a800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:44.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:44.885+0000 7fc5b57fa700 1 -- 172.21.15.46:0/2299073814 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc5ac0975f0 con 0x7fc5b8101f70 2024-03-18T20:34:45.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] up:active 2024-03-18T20:34:45.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/4157129164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-18T20:34:45.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-18T20:34:45.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-03-18T20:34:45.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-03-18T20:34:45.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-18T20:34:45.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:45.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] up:active 2024-03-18T20:34:45.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-18T20:34:45.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T20:34:45.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: Cluster is now healthy 2024-03-18T20:34:45.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2024-03-18T20:34:45.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-18T20:34:45.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:44 smithi046 ceph-mon[43495]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-03-18T20:34:45.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.141+0000 7fc5be7b1700 1 -- 172.21.15.46:0/2299073814 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5b802d160 con 0x7fc5a00a1120 2024-03-18T20:34:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.146+0000 7fc5b57fa700 1 -- 172.21.15.46:0/2299073814 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fc5b802d160 con 0x7fc5a00a1120 2024-03-18T20:34:45.150 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_0/4b1446e0-92e0-45b3-bb82-da65cb60a60c 2024-03-18T20:34:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.148+0000 7fc5be7b1700 1 -- 172.21.15.46:0/2299073814 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc5a00a1120 msgr2=0x7fc5a00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.148+0000 7fc5be7b1700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc5a00a1120 0x7fc5a00a35e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8198120 tx=0x7fc5a800a5e0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.148+0000 7fc5be7b1700 1 -- 172.21.15.46:0/2299073814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8101f70 msgr2=0x7fc5b81971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.148+0000 7fc5be7b1700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8101f70 0x7fc5b81971e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0036f0 tx=0x7fc5ac0047b0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.148+0000 7fc5be7b1700 1 -- 172.21.15.46:0/2299073814 shutdown_connections 2024-03-18T20:34:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.148+0000 7fc5be7b1700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc5a00a1120 0x7fc5a00a35e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.148+0000 7fc5be7b1700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5b8102910 0x7fc5b819e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.148+0000 7fc5be7b1700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b8100d80 0x7fc5b8196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.148+0000 7fc5be7b1700 1 --2- 172.21.15.46:0/2299073814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8101f70 0x7fc5b81971e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.148+0000 7fc5be7b1700 1 -- 172.21.15.46:0/2299073814 >> 172.21.15.46:0/2299073814 conn(0x7fc5b80fc2f0 msgr2=0x7fc5b8105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.148+0000 7fc5be7b1700 1 -- 172.21.15.46:0/2299073814 shutdown_connections 2024-03-18T20:34:45.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.148+0000 7fc5be7b1700 1 -- 172.21.15.46:0/2299073814 wait complete. 2024-03-18T20:34:45.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-18T20:34:45.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] up:active 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/4157129164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] up:active 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: Cluster is now healthy 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-18T20:34:45.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:44 smithi073 ceph-mon[47384]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-03-18T20:34:45.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] up:active 2024-03-18T20:34:45.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/4157129164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-18T20:34:45.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-18T20:34:45.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-03-18T20:34:45.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-03-18T20:34:45.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-18T20:34:45.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:45.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] up:active 2024-03-18T20:34:45.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-18T20:34:45.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T20:34:45.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: Cluster is now healthy 2024-03-18T20:34:45.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2024-03-18T20:34:45.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-18T20:34:45.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:44 smithi155 ceph-mon[48344]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-03-18T20:34:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.304+0000 7fde1c4bd700 1 -- 172.21.15.46:0/1670748910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde14101840 msgr2=0x7fde14103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.304+0000 7fde1c4bd700 1 --2- 172.21.15.46:0/1670748910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde14101840 0x7fde14103c30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fde10009990 tx=0x7fde10009ca0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.304+0000 7fde1c4bd700 1 -- 172.21.15.46:0/1670748910 shutdown_connections 2024-03-18T20:34:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.304+0000 7fde1c4bd700 1 --2- 172.21.15.46:0/1670748910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde14101840 0x7fde14103c30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.304+0000 7fde1c4bd700 1 --2- 172.21.15.46:0/1670748910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde14106aa0 0x7fde14108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.304+0000 7fde1c4bd700 1 --2- 172.21.15.46:0/1670748910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde14104170 0x7fde14106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.304+0000 7fde1c4bd700 1 -- 172.21.15.46:0/1670748910 >> 172.21.15.46:0/1670748910 conn(0x7fde140fb1d0 msgr2=0x7fde140fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.304+0000 7fde1c4bd700 1 -- 172.21.15.46:0/1670748910 shutdown_connections 2024-03-18T20:34:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.304+0000 7fde1c4bd700 1 -- 172.21.15.46:0/1670748910 wait complete. 2024-03-18T20:34:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.305+0000 7fde1c4bd700 1 Processor -- start 2024-03-18T20:34:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.305+0000 7fde1c4bd700 1 -- start start 2024-03-18T20:34:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.305+0000 7fde1c4bd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde14101840 0x7fde14196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.305+0000 7fde1c4bd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde14104170 0x7fde14197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.305+0000 7fde1c4bd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde14106aa0 0x7fde1419e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.305+0000 7fde1c4bd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1419e7c0 con 0x7fde14104170 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.305+0000 7fde1c4bd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1419e930 con 0x7fde14101840 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.305+0000 7fde1c4bd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1419eaf0 con 0x7fde14106aa0 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.305+0000 7fde19a58700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde14104170 0x7fde14197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.305+0000 7fde19a58700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde14104170 0x7fde14197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42748/0 (socket says 172.21.15.46:42748) 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.305+0000 7fde19a58700 1 -- 172.21.15.46:0/989393586 learned_addr learned my addr 172.21.15.46:0/989393586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.305+0000 7fde1a259700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde14101840 0x7fde14196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.306+0000 7fde1aa5a700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde14106aa0 0x7fde1419e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.306+0000 7fde19a58700 1 -- 172.21.15.46:0/989393586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde14106aa0 msgr2=0x7fde1419e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.306+0000 7fde19a58700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde14106aa0 0x7fde1419e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.306+0000 7fde19a58700 1 -- 172.21.15.46:0/989393586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde14101840 msgr2=0x7fde14196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.306+0000 7fde19a58700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde14101840 0x7fde14196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.306+0000 7fde19a58700 1 -- 172.21.15.46:0/989393586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde100095f0 con 0x7fde14104170 2024-03-18T20:34:45.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.306+0000 7fde19a58700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde14104170 0x7fde14197210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde0400e9a0 tx=0x7fde0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:45.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.306+0000 7fde0b7fe700 1 -- 172.21.15.46:0/989393586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04009790 con 0x7fde14104170 2024-03-18T20:34:45.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.306+0000 7fde0b7fe700 1 -- 172.21.15.46:0/989393586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde0400cbd0 con 0x7fde14104170 2024-03-18T20:34:45.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.306+0000 7fde1c4bd700 1 -- 172.21.15.46:0/989393586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1419edd0 con 0x7fde14104170 2024-03-18T20:34:45.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.306+0000 7fde0b7fe700 1 -- 172.21.15.46:0/989393586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04018680 con 0x7fde14104170 2024-03-18T20:34:45.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.306+0000 7fde1c4bd700 1 -- 172.21.15.46:0/989393586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1419f250 con 0x7fde14104170 2024-03-18T20:34:45.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.307+0000 7fde1c4bd700 1 -- 172.21.15.46:0/989393586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde140fcd50 con 0x7fde14104170 2024-03-18T20:34:45.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.308+0000 7fde0b7fe700 1 -- 172.21.15.46:0/989393586 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fde0400cd40 con 0x7fde14104170 2024-03-18T20:34:45.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.309+0000 7fde0b7fe700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde000a11a0 0x7fde000a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:45.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.309+0000 7fde0b7fe700 1 -- 172.21.15.46:0/989393586 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fde04014070 con 0x7fde14104170 2024-03-18T20:34:45.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.309+0000 7fde1a259700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde000a11a0 0x7fde000a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:45.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.310+0000 7fde1a259700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde000a11a0 0x7fde000a3660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fde1000b680 tx=0x7fde10005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:45.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.313+0000 7fde0b7fe700 1 -- 172.21.15.46:0/989393586 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fde04096560 con 0x7fde14104170 2024-03-18T20:34:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.567+0000 7fde1c4bd700 1 -- 172.21.15.46:0/989393586 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fde1402d120 con 0x7fde000a11a0 2024-03-18T20:34:45.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.590+0000 7fde0b7fe700 1 -- 172.21.15.46:0/989393586 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fde1402d120 con 0x7fde000a11a0 2024-03-18T20:34:45.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.592+0000 7fde1c4bd700 1 -- 172.21.15.46:0/989393586 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde000a11a0 msgr2=0x7fde000a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:45.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.592+0000 7fde1c4bd700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde000a11a0 0x7fde000a3660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fde1000b680 tx=0x7fde10005fd0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.592+0000 7fde1c4bd700 1 -- 172.21.15.46:0/989393586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde14104170 msgr2=0x7fde14197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:45.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.592+0000 7fde1c4bd700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde14104170 0x7fde14197210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde0400e9a0 tx=0x7fde0400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.593+0000 7fde1c4bd700 1 -- 172.21.15.46:0/989393586 shutdown_connections 2024-03-18T20:34:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.593+0000 7fde1c4bd700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde000a11a0 0x7fde000a3660 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.593+0000 7fde1c4bd700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde14106aa0 0x7fde1419e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.593+0000 7fde1c4bd700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde14101840 0x7fde14196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.593+0000 7fde1c4bd700 1 --2- 172.21.15.46:0/989393586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde14104170 0x7fde14197210 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.593+0000 7fde1c4bd700 1 -- 172.21.15.46:0/989393586 >> 172.21.15.46:0/989393586 conn(0x7fde140fb1d0 msgr2=0x7fde14102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.593+0000 7fde1c4bd700 1 -- 172.21.15.46:0/989393586 shutdown_connections 2024-03-18T20:34:45.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.593+0000 7fde1c4bd700 1 -- 172.21.15.46:0/989393586 wait complete. 2024-03-18T20:34:45.610 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-18T20:34:45.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.750+0000 7f3b641e8700 1 -- 172.21.15.46:0/1181273888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c100d80 msgr2=0x7f3b5c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:45.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.750+0000 7f3b641e8700 1 --2- 172.21.15.46:0/1181273888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c100d80 0x7f3b5c1011a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3b580099c0 tx=0x7f3b58009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.750+0000 7f3b641e8700 1 -- 172.21.15.46:0/1181273888 shutdown_connections 2024-03-18T20:34:45.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.750+0000 7f3b641e8700 1 --2- 172.21.15.46:0/1181273888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c100d80 0x7f3b5c1011a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.750+0000 7f3b641e8700 1 --2- 172.21.15.46:0/1181273888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b5c102910 0x7f3b5c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.750+0000 7f3b641e8700 1 --2- 172.21.15.46:0/1181273888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b5c101f70 0x7f3b5c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.750+0000 7f3b641e8700 1 -- 172.21.15.46:0/1181273888 >> 172.21.15.46:0/1181273888 conn(0x7f3b5c0fc2f0 msgr2=0x7f3b5c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:45.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.750+0000 7f3b641e8700 1 -- 172.21.15.46:0/1181273888 shutdown_connections 2024-03-18T20:34:45.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.750+0000 7f3b641e8700 1 -- 172.21.15.46:0/1181273888 wait complete. 2024-03-18T20:34:45.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.750+0000 7f3b641e8700 1 Processor -- start 2024-03-18T20:34:45.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.750+0000 7f3b641e8700 1 -- start start 2024-03-18T20:34:45.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b641e8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b5c100d80 0x7f3b5c196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:45.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b641e8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c101f70 0x7f3b5c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:45.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b641e8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b5c102910 0x7f3b5c19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:45.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b641e8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c19e820 con 0x7f3b5c101f70 2024-03-18T20:34:45.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b641e8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c19e990 con 0x7f3b5c102910 2024-03-18T20:34:45.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b641e8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c19eb50 con 0x7f3b5c100d80 2024-03-18T20:34:45.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b61783700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c101f70 0x7f3b5c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:45.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b61783700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c101f70 0x7f3b5c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42774/0 (socket says 172.21.15.46:42774) 2024-03-18T20:34:45.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b61783700 1 -- 172.21.15.46:0/974060658 learned_addr learned my addr 172.21.15.46:0/974060658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:45.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b62785700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b5c102910 0x7f3b5c19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:45.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b61f84700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b5c100d80 0x7f3b5c196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:45.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b61783700 1 -- 172.21.15.46:0/974060658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b5c100d80 msgr2=0x7f3b5c196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:45.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b61783700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b5c100d80 0x7f3b5c196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b61783700 1 -- 172.21.15.46:0/974060658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b5c102910 msgr2=0x7f3b5c19e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:45.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b61783700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b5c102910 0x7f3b5c19e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:45.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b61783700 1 -- 172.21.15.46:0/974060658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b580095f0 con 0x7f3b5c101f70 2024-03-18T20:34:45.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b61783700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c101f70 0x7f3b5c197240 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00d950 tx=0x7f3b4c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:45.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.751+0000 7f3b52ffd700 1 -- 172.21.15.46:0/974060658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c009940 con 0x7f3b5c101f70 2024-03-18T20:34:45.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.752+0000 7f3b52ffd700 1 -- 172.21.15.46:0/974060658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b4c010460 con 0x7f3b5c101f70 2024-03-18T20:34:45.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.752+0000 7f3b641e8700 1 -- 172.21.15.46:0/974060658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c19ee30 con 0x7f3b5c101f70 2024-03-18T20:34:45.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.752+0000 7f3b52ffd700 1 -- 172.21.15.46:0/974060658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c00f620 con 0x7f3b5c101f70 2024-03-18T20:34:45.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.752+0000 7f3b641e8700 1 -- 172.21.15.46:0/974060658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c19f2b0 con 0x7f3b5c101f70 2024-03-18T20:34:45.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.753+0000 7f3b641e8700 1 -- 172.21.15.46:0/974060658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5c108200 con 0x7f3b5c101f70 2024-03-18T20:34:45.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.754+0000 7f3b52ffd700 1 -- 172.21.15.46:0/974060658 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f3b4c0108d0 con 0x7f3b5c101f70 2024-03-18T20:34:45.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.754+0000 7f3b52ffd700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3b480a1120 0x7f3b480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:45.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.754+0000 7f3b61f84700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3b480a1120 0x7f3b480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:45.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.754+0000 7f3b52ffd700 1 -- 172.21.15.46:0/974060658 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3b4c0cc970 con 0x7f3b5c101f70 2024-03-18T20:34:45.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.755+0000 7f3b61f84700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3b480a1120 0x7f3b480a35e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0fdf00 tx=0x7f3b58009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:45.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.758+0000 7f3b52ffd700 1 -- 172.21.15.46:0/974060658 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3b4c096250 con 0x7f3b5c101f70 2024-03-18T20:34:45.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:45.790+0000 7f3b52ffd700 1 -- 172.21.15.46:0/974060658 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3b4c0105d0 con 0x7f3b5c101f70 2024-03-18T20:34:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.024+0000 7f3b641e8700 1 -- 172.21.15.46:0/974060658 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b5c02d160 con 0x7f3b480a1120 2024-03-18T20:34:46.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.028+0000 7f3b52ffd700 1 -- 172.21.15.46:0/974060658 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f3b5c02d160 con 0x7f3b480a1120 2024-03-18T20:34:46.032 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410 2024-03-18T20:34:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.031+0000 7f3b641e8700 1 -- 172.21.15.46:0/974060658 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3b480a1120 msgr2=0x7f3b480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.031+0000 7f3b641e8700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3b480a1120 0x7f3b480a35e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0fdf00 tx=0x7f3b58009f90 comp rx=0 tx=0).stop 2024-03-18T20:34:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.031+0000 7f3b641e8700 1 -- 172.21.15.46:0/974060658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c101f70 msgr2=0x7f3b5c197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.031+0000 7f3b641e8700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c101f70 0x7f3b5c197240 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00d950 tx=0x7f3b4c00dc60 comp rx=0 tx=0).stop 2024-03-18T20:34:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.031+0000 7f3b641e8700 1 -- 172.21.15.46:0/974060658 shutdown_connections 2024-03-18T20:34:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.031+0000 7f3b641e8700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3b480a1120 0x7f3b480a35e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.031+0000 7f3b641e8700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b5c102910 0x7f3b5c19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.031+0000 7f3b641e8700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b5c100d80 0x7f3b5c196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.031+0000 7f3b641e8700 1 --2- 172.21.15.46:0/974060658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c101f70 0x7f3b5c197240 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.031+0000 7f3b641e8700 1 -- 172.21.15.46:0/974060658 >> 172.21.15.46:0/974060658 conn(0x7f3b5c0fc2f0 msgr2=0x7f3b5c105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.031+0000 7f3b641e8700 1 -- 172.21.15.46:0/974060658 shutdown_connections 2024-03-18T20:34:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.031+0000 7f3b641e8700 1 -- 172.21.15.46:0/974060658 wait complete. 2024-03-18T20:34:46.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:45 smithi046 ceph-mon[43495]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:46.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:45 smithi046 ceph-mon[43495]: pgmap v222: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s wr, 14 op/s 2024-03-18T20:34:46.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:45 smithi046 ceph-mon[43495]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:46.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:45 smithi046 ceph-mon[43495]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-03-18T20:34:46.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:45 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] up:active 2024-03-18T20:34:46.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:45 smithi046 ceph-mon[43495]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T20:34:46.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:45 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:34:46.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:34:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.182+0000 7fcafb4b8700 1 -- 172.21.15.46:0/2777538211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf4100c80 msgr2=0x7fcaf41010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.182+0000 7fcafb4b8700 1 --2- 172.21.15.46:0/2777538211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf4100c80 0x7fcaf41010a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00099c0 tx=0x7fcaf0009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.182+0000 7fcafb4b8700 1 -- 172.21.15.46:0/2777538211 shutdown_connections 2024-03-18T20:34:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.182+0000 7fcafb4b8700 1 --2- 172.21.15.46:0/2777538211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf4100c80 0x7fcaf41010a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.182+0000 7fcafb4b8700 1 --2- 172.21.15.46:0/2777538211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcaf4102810 0x7fcaf41090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.182+0000 7fcafb4b8700 1 --2- 172.21.15.46:0/2777538211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaf4101e70 0x7fcaf41022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.182+0000 7fcafb4b8700 1 -- 172.21.15.46:0/2777538211 >> 172.21.15.46:0/2777538211 conn(0x7fcaf40fc210 msgr2=0x7fcaf40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.182+0000 7fcafb4b8700 1 -- 172.21.15.46:0/2777538211 shutdown_connections 2024-03-18T20:34:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.182+0000 7fcafb4b8700 1 -- 172.21.15.46:0/2777538211 wait complete. 2024-03-18T20:34:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcafb4b8700 1 Processor -- start 2024-03-18T20:34:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcafb4b8700 1 -- start start 2024-03-18T20:34:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcafb4b8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf4100c80 0x7fcaf4074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcafb4b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcaf4101e70 0x7fcaf40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcafb4b8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaf4102810 0x7fcaf40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcafb4b8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf4073e80 con 0x7fcaf4100c80 2024-03-18T20:34:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcafb4b8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf4073ff0 con 0x7fcaf4101e70 2024-03-18T20:34:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcafb4b8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf41a0c50 con 0x7fcaf4102810 2024-03-18T20:34:46.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcaf9254700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf4100c80 0x7fcaf4074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:46.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcaf9254700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf4100c80 0x7fcaf4074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42790/0 (socket says 172.21.15.46:42790) 2024-03-18T20:34:46.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcaf9254700 1 -- 172.21.15.46:0/1164868002 learned_addr learned my addr 172.21.15.46:0/1164868002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:46.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcaf9a55700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaf4102810 0x7fcaf40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:46.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcaf8a53700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcaf4101e70 0x7fcaf40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:46.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcaf9254700 1 -- 172.21.15.46:0/1164868002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaf4102810 msgr2=0x7fcaf40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:46.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcaf9254700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaf4102810 0x7fcaf40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcaf9254700 1 -- 172.21.15.46:0/1164868002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcaf4101e70 msgr2=0x7fcaf40731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcaf9254700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcaf4101e70 0x7fcaf40731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcaf9254700 1 -- 172.21.15.46:0/1164868002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf00095f0 con 0x7fcaf4100c80 2024-03-18T20:34:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.183+0000 7fcaf9254700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf4100c80 0x7fcaf4074b10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0003730 tx=0x7fcaf000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.184+0000 7fcaea7fc700 1 -- 172.21.15.46:0/1164868002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf001d070 con 0x7fcaf4100c80 2024-03-18T20:34:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.184+0000 7fcaea7fc700 1 -- 172.21.15.46:0/1164868002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaf00059a0 con 0x7fcaf4100c80 2024-03-18T20:34:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.184+0000 7fcafb4b8700 1 -- 172.21.15.46:0/1164868002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf41a0e80 con 0x7fcaf4100c80 2024-03-18T20:34:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.184+0000 7fcaea7fc700 1 -- 172.21.15.46:0/1164868002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0004990 con 0x7fcaf4100c80 2024-03-18T20:34:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.184+0000 7fcafb4b8700 1 -- 172.21.15.46:0/1164868002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf41a1240 con 0x7fcaf4100c80 2024-03-18T20:34:46.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.185+0000 7fcafb4b8700 1 -- 172.21.15.46:0/1164868002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf40671f0 con 0x7fcaf4100c80 2024-03-18T20:34:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.186+0000 7fcaea7fc700 1 -- 172.21.15.46:0/1164868002 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fcaf0005210 con 0x7fcaf4100c80 2024-03-18T20:34:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.186+0000 7fcaea7fc700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fcae00a1230 0x7fcae00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.187+0000 7fcaf8a53700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fcae00a1230 0x7fcae00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.187+0000 7fcaea7fc700 1 -- 172.21.15.46:0/1164868002 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcaf00cd430 con 0x7fcaf4100c80 2024-03-18T20:34:46.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.187+0000 7fcaf8a53700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fcae00a1230 0x7fcae00a36f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009c60 tx=0x7fcae4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.191+0000 7fcaea7fc700 1 -- 172.21.15.46:0/1164868002 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcaf40671f0 con 0x7fcaf4100c80 2024-03-18T20:34:46.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:45 smithi073 ceph-mon[47384]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:46.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:45 smithi073 ceph-mon[47384]: pgmap v222: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s wr, 14 op/s 2024-03-18T20:34:46.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:45 smithi073 ceph-mon[47384]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:46.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:45 smithi073 ceph-mon[47384]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-03-18T20:34:46.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:45 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] up:active 2024-03-18T20:34:46.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:45 smithi073 ceph-mon[47384]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T20:34:46.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:45 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:34:46.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:45 smithi155 ceph-mon[48344]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:46.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:45 smithi155 ceph-mon[48344]: pgmap v222: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s wr, 14 op/s 2024-03-18T20:34:46.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:45 smithi155 ceph-mon[48344]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:46.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:45 smithi155 ceph-mon[48344]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-03-18T20:34:46.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:45 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] up:active 2024-03-18T20:34:46.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:45 smithi155 ceph-mon[48344]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T20:34:46.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:45 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:34:46.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.460+0000 7fcafb4b8700 1 -- 172.21.15.46:0/1164868002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcaf40671f0 con 0x7fcaf4100c80 2024-03-18T20:34:46.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.461+0000 7fcaea7fc700 1 -- 172.21.15.46:0/1164868002 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10909 (secure 0 0 0) 0x7fcaf40671f0 con 0x7fcaf4100c80 2024-03-18T20:34:46.465 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:46.465 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:34:45.796438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:34:46.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.463+0000 7fcafb4b8700 1 -- 172.21.15.46:0/1164868002 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fcae00a1230 msgr2=0x7fcae00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:46.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.463+0000 7fcafb4b8700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fcae00a1230 0x7fcae00a36f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009c60 tx=0x7fcae4009260 comp rx=0 tx=0).stop 2024-03-18T20:34:46.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.463+0000 7fcafb4b8700 1 -- 172.21.15.46:0/1164868002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf4100c80 msgr2=0x7fcaf4074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:46.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.463+0000 7fcafb4b8700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf4100c80 0x7fcaf4074b10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0003730 tx=0x7fcaf000f670 comp rx=0 tx=0).stop 2024-03-18T20:34:46.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.464+0000 7fcafb4b8700 1 -- 172.21.15.46:0/1164868002 shutdown_connections 2024-03-18T20:34:46.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.464+0000 7fcafb4b8700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fcae00a1230 0x7fcae00a36f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.464+0000 7fcafb4b8700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcaf4100c80 0x7fcaf4074b10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.464+0000 7fcafb4b8700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaf4102810 0x7fcaf40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.464+0000 7fcafb4b8700 1 --2- 172.21.15.46:0/1164868002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcaf4101e70 0x7fcaf40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.464+0000 7fcafb4b8700 1 -- 172.21.15.46:0/1164868002 >> 172.21.15.46:0/1164868002 conn(0x7fcaf40fc210 msgr2=0x7fcaf4105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:46.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.464+0000 7fcafb4b8700 1 -- 172.21.15.46:0/1164868002 shutdown_connections 2024-03-18T20:34:46.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.464+0000 7fcafb4b8700 1 -- 172.21.15.46:0/1164868002 wait complete. 2024-03-18T20:34:46.468 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-03-18T20:34:46.481 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:34:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.615+0000 7fea760a1700 1 -- 172.21.15.46:0/1352622081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea700ffb90 msgr2=0x7fea700fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.615+0000 7fea760a1700 1 --2- 172.21.15.46:0/1352622081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea700ffb90 0x7fea700fffb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fea600099c0 tx=0x7fea60009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.615+0000 7fea760a1700 1 -- 172.21.15.46:0/1352622081 shutdown_connections 2024-03-18T20:34:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.615+0000 7fea760a1700 1 --2- 172.21.15.46:0/1352622081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea700ffb90 0x7fea700fffb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.615+0000 7fea760a1700 1 --2- 172.21.15.46:0/1352622081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea70101720 0x7fea70107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.615+0000 7fea760a1700 1 --2- 172.21.15.46:0/1352622081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea70100d80 0x7fea701011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.615+0000 7fea760a1700 1 -- 172.21.15.46:0/1352622081 >> 172.21.15.46:0/1352622081 conn(0x7fea700fb100 msgr2=0x7fea700fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.615+0000 7fea760a1700 1 -- 172.21.15.46:0/1352622081 shutdown_connections 2024-03-18T20:34:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.615+0000 7fea760a1700 1 -- 172.21.15.46:0/1352622081 wait complete. 2024-03-18T20:34:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea760a1700 1 Processor -- start 2024-03-18T20:34:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea760a1700 1 -- start start 2024-03-18T20:34:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea760a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea700ffb90 0x7fea70196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea760a1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea70100d80 0x7fea70197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea760a1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea70101720 0x7fea7019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea760a1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7019e720 con 0x7fea70101720 2024-03-18T20:34:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea760a1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7019e860 con 0x7fea700ffb90 2024-03-18T20:34:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea760a1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7019e9f0 con 0x7fea70100d80 2024-03-18T20:34:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea6ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea70101720 0x7fea7019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea6effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea70100d80 0x7fea70197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea6ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea70101720 0x7fea7019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42814/0 (socket says 172.21.15.46:42814) 2024-03-18T20:34:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea6effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea70100d80 0x7fea70197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:50320/0 (socket says 172.21.15.46:50320) 2024-03-18T20:34:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.616+0000 7fea6ffff700 1 -- 172.21.15.46:0/3888901103 learned_addr learned my addr 172.21.15.46:0/3888901103 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea6f7fe700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea700ffb90 0x7fea70196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea6ffff700 1 -- 172.21.15.46:0/3888901103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea70100d80 msgr2=0x7fea70197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea6ffff700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea70100d80 0x7fea70197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea6ffff700 1 -- 172.21.15.46:0/3888901103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea700ffb90 msgr2=0x7fea70196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea6ffff700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea700ffb90 0x7fea70196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea6ffff700 1 -- 172.21.15.46:0/3888901103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea600095f0 con 0x7fea70101720 2024-03-18T20:34:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea6effd700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea70100d80 0x7fea70197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:34:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea6ffff700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea70101720 0x7fea7019e150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fea68007ee0 tx=0x7fea6800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea6cff9700 1 -- 172.21.15.46:0/3888901103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6800dd20 con 0x7fea70101720 2024-03-18T20:34:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea6cff9700 1 -- 172.21.15.46:0/3888901103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea6800f040 con 0x7fea70101720 2024-03-18T20:34:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea760a1700 1 -- 172.21.15.46:0/3888901103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7019ecd0 con 0x7fea70101720 2024-03-18T20:34:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea6cff9700 1 -- 172.21.15.46:0/3888901103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea680048a0 con 0x7fea70101720 2024-03-18T20:34:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.617+0000 7fea760a1700 1 -- 172.21.15.46:0/3888901103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7019f190 con 0x7fea70101720 2024-03-18T20:34:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.618+0000 7fea760a1700 1 -- 172.21.15.46:0/3888901103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea70067180 con 0x7fea70101720 2024-03-18T20:34:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.619+0000 7fea6cff9700 1 -- 172.21.15.46:0/3888901103 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fea68003a00 con 0x7fea70101720 2024-03-18T20:34:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.620+0000 7fea6cff9700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fea580a1230 0x7fea580a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.620+0000 7fea6cff9700 1 -- 172.21.15.46:0/3888901103 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fea680cc8d0 con 0x7fea70101720 2024-03-18T20:34:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.620+0000 7fea6f7fe700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fea580a1230 0x7fea580a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.621+0000 7fea6f7fe700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fea580a1230 0x7fea580a36f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea60003730 tx=0x7fea60000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:46.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.624+0000 7fea6cff9700 1 -- 172.21.15.46:0/3888901103 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fea680960b0 con 0x7fea70101720 2024-03-18T20:34:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.855+0000 7fea760a1700 1 -- 172.21.15.46:0/3888901103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea7019f380 con 0x7fea70101720 2024-03-18T20:34:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.857+0000 7fea6cff9700 1 -- 172.21.15.46:0/3888901103 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+22071 (secure 0 0 0) 0x7fea68023020 con 0x7fea70101720 2024-03-18T20:34:46.861 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:46.861 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:34:39.786955+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:34:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.859+0000 7fea760a1700 1 -- 172.21.15.46:0/3888901103 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fea580a1230 msgr2=0x7fea580a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.859+0000 7fea760a1700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fea580a1230 0x7fea580a36f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea60003730 tx=0x7fea60000bc0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.859+0000 7fea760a1700 1 -- 172.21.15.46:0/3888901103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea70101720 msgr2=0x7fea7019e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.859+0000 7fea760a1700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea70101720 0x7fea7019e150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fea68007ee0 tx=0x7fea6800d3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.859+0000 7fea760a1700 1 -- 172.21.15.46:0/3888901103 shutdown_connections 2024-03-18T20:34:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.859+0000 7fea760a1700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fea580a1230 0x7fea580a36f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.859+0000 7fea760a1700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea70101720 0x7fea7019e150 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.859+0000 7fea760a1700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea700ffb90 0x7fea70196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.859+0000 7fea760a1700 1 --2- 172.21.15.46:0/3888901103 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea70100d80 0x7fea70197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.859+0000 7fea760a1700 1 -- 172.21.15.46:0/3888901103 >> 172.21.15.46:0/3888901103 conn(0x7fea700fb100 msgr2=0x7fea70068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.859+0000 7fea760a1700 1 -- 172.21.15.46:0/3888901103 shutdown_connections 2024-03-18T20:34:46.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:46.859+0000 7fea760a1700 1 -- 172.21.15.46:0/3888901103 wait complete. 2024-03-18T20:34:46.877 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-18T20:34:46.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T20:34:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.011+0000 7fa0521d1700 1 -- 172.21.15.46:0/602899325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa04c100d80 msgr2=0x7fa04c1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.011+0000 7fa0521d1700 1 --2- 172.21.15.46:0/602899325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa04c100d80 0x7fa04c1011e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0400099c0 tx=0x7fa040009cd0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.011+0000 7fa0521d1700 1 -- 172.21.15.46:0/602899325 shutdown_connections 2024-03-18T20:34:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.011+0000 7fa0521d1700 1 --2- 172.21.15.46:0/602899325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c101720 0x7fa04c107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.011+0000 7fa0521d1700 1 --2- 172.21.15.46:0/602899325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa04c0ffb90 0x7fa04c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.011+0000 7fa0521d1700 1 --2- 172.21.15.46:0/602899325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa04c100d80 0x7fa04c1011e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.011+0000 7fa0521d1700 1 -- 172.21.15.46:0/602899325 >> 172.21.15.46:0/602899325 conn(0x7fa04c0fb100 msgr2=0x7fa04c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:47.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.011+0000 7fa0521d1700 1 -- 172.21.15.46:0/602899325 shutdown_connections 2024-03-18T20:34:47.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.011+0000 7fa0521d1700 1 -- 172.21.15.46:0/602899325 wait complete. 2024-03-18T20:34:47.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.012+0000 7fa0521d1700 1 Processor -- start 2024-03-18T20:34:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.012+0000 7fa0521d1700 1 -- start start 2024-03-18T20:34:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.012+0000 7fa0521d1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa04c0ffb90 0x7fa04c103ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.012+0000 7fa0521d1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa04c100d80 0x7fa04c102160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.012+0000 7fa0521d1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c101720 0x7fa04c1026a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa0521d1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c102e40 con 0x7fa04c100d80 2024-03-18T20:34:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa0521d1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c102fb0 con 0x7fa04c101720 2024-03-18T20:34:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa0521d1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c1a0c60 con 0x7fa04c0ffb90 2024-03-18T20:34:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa04affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa04c100d80 0x7fa04c102160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa04affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa04c100d80 0x7fa04c102160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42828/0 (socket says 172.21.15.46:42828) 2024-03-18T20:34:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa04affd700 1 -- 172.21.15.46:0/49405940 learned_addr learned my addr 172.21.15.46:0/49405940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:47.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa04bfff700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c101720 0x7fa04c1026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:47.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa04b7fe700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa04c0ffb90 0x7fa04c103ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:47.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa04affd700 1 -- 172.21.15.46:0/49405940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa04c0ffb90 msgr2=0x7fa04c103ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:47.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa04affd700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa04c0ffb90 0x7fa04c103ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa04affd700 1 -- 172.21.15.46:0/49405940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c101720 msgr2=0x7fa04c1026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:47.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa04affd700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c101720 0x7fa04c1026a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa04affd700 1 -- 172.21.15.46:0/49405940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0400095f0 con 0x7fa04c100d80 2024-03-18T20:34:47.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa04affd700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa04c100d80 0x7fa04c102160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa040006010 tx=0x7fa04000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:47.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa048ff9700 1 -- 172.21.15.46:0/49405940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04001d070 con 0x7fa04c100d80 2024-03-18T20:34:47.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa048ff9700 1 -- 172.21.15.46:0/49405940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0400054f0 con 0x7fa04c100d80 2024-03-18T20:34:47.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa0521d1700 1 -- 172.21.15.46:0/49405940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04c1a0e90 con 0x7fa04c100d80 2024-03-18T20:34:47.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa048ff9700 1 -- 172.21.15.46:0/49405940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040004c70 con 0x7fa04c100d80 2024-03-18T20:34:47.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.013+0000 7fa0521d1700 1 -- 172.21.15.46:0/49405940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04c1a1200 con 0x7fa04c100d80 2024-03-18T20:34:47.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.014+0000 7fa0521d1700 1 -- 172.21.15.46:0/49405940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04c067180 con 0x7fa04c100d80 2024-03-18T20:34:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.016+0000 7fa048ff9700 1 -- 172.21.15.46:0/49405940 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa04000fbb0 con 0x7fa04c100d80 2024-03-18T20:34:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.016+0000 7fa048ff9700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa0340a11b0 0x7fa0340a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.016+0000 7fa04b7fe700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa0340a11b0 0x7fa0340a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.016+0000 7fa048ff9700 1 -- 172.21.15.46:0/49405940 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa0400cdab0 con 0x7fa04c100d80 2024-03-18T20:34:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.017+0000 7fa04b7fe700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa0340a11b0 0x7fa0340a3670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa04c100be0 tx=0x7fa03c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:47.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.020+0000 7fa048ff9700 1 -- 172.21.15.46:0/49405940 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa040097320 con 0x7fa04c100d80 2024-03-18T20:34:47.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:46 smithi073 ceph-mon[47384]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:47.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:46 smithi073 ceph-mon[47384]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:47.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:46 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1164868002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:47.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.233+0000 7fa0521d1700 1 -- 172.21.15.46:0/49405940 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa04c0fd340 con 0x7fa0340a11b0 2024-03-18T20:34:47.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.244+0000 7fa048ff9700 1 -- 172.21.15.46:0/49405940 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+328109 (secure 0 0 0) 0x7fa04c0fd340 con 0x7fa0340a11b0 2024-03-18T20:34:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.248+0000 7fa0521d1700 1 -- 172.21.15.46:0/49405940 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa0340a11b0 msgr2=0x7fa0340a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.248+0000 7fa0521d1700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa0340a11b0 0x7fa0340a3670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa04c100be0 tx=0x7fa03c009260 comp rx=0 tx=0).stop 2024-03-18T20:34:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.248+0000 7fa0521d1700 1 -- 172.21.15.46:0/49405940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa04c100d80 msgr2=0x7fa04c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.248+0000 7fa0521d1700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa04c100d80 0x7fa04c102160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa040006010 tx=0x7fa04000f740 comp rx=0 tx=0).stop 2024-03-18T20:34:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.248+0000 7fa0521d1700 1 -- 172.21.15.46:0/49405940 shutdown_connections 2024-03-18T20:34:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.248+0000 7fa0521d1700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa0340a11b0 0x7fa0340a3670 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.248+0000 7fa0521d1700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c101720 0x7fa04c1026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.248+0000 7fa0521d1700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa04c0ffb90 0x7fa04c103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.248+0000 7fa0521d1700 1 --2- 172.21.15.46:0/49405940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa04c100d80 0x7fa04c102160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.248+0000 7fa0521d1700 1 -- 172.21.15.46:0/49405940 >> 172.21.15.46:0/49405940 conn(0x7fa04c0fb100 msgr2=0x7fa04c0fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.248+0000 7fa0521d1700 1 -- 172.21.15.46:0/49405940 shutdown_connections 2024-03-18T20:34:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.248+0000 7fa0521d1700 1 -- 172.21.15.46:0/49405940 wait complete. 2024-03-18T20:34:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-18T20:34:47.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:46 smithi046 ceph-mon[43495]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:47.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:46 smithi046 ceph-mon[43495]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:47.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:46 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1164868002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:47.265 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-03-18T20:34:47.019338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596076,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":138,"num_read_kb":121,"num_write":311,"num_write_kb":2984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":254,"ondisk_log_size":254,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":283,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336012,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605812,"statfs":{"total":1151940427776,"available":1151596351488,"internally_reserved":0,"allocated":12079104,"data_stored":6640115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5708,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":130,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.231847"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.902054+0000","last_change":"2024-03-18T20:34:38.817028+0000","last_active":"2024-03-18T20:34:39.902054+0000","last_peered":"2024-03-18T20:34:39.902054+0000","last_clean":"2024-03-18T20:34:39.902054+0000","last_became_active":"2024-03-18T20:34:38.816711+0000","last_became_peered":"2024-03-18T20:34:38.816711+0000","last_unstale":"2024-03-18T20:34:39.902054+0000","last_undegraded":"2024-03-18T20:34:39.902054+0000","last_fullsized":"2024-03-18T20:34:39.902054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:32:16.827141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901345+0000","last_change":"2024-03-18T20:34:38.807350+0000","last_active":"2024-03-18T20:34:39.901345+0000","last_peered":"2024-03-18T20:34:39.901345+0000","last_clean":"2024-03-18T20:34:39.901345+0000","last_became_active":"2024-03-18T20:34:38.807008+0000","last_became_peered":"2024-03-18T20:34:38.807008+0000","last_unstale":"2024-03-18T20:34:39.901345+0000","last_undegraded":"2024-03-18T20:34:39.901345+0000","last_fullsized":"2024-03-18T20:34:39.901345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:35.667384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901221+0000","last_change":"2024-03-18T20:34:38.817045+0000","last_active":"2024-03-18T20:34:39.901221+0000","last_peered":"2024-03-18T20:34:39.901221+0000","last_clean":"2024-03-18T20:34:39.901221+0000","last_became_active":"2024-03-18T20:34:38.816659+0000","last_became_peered":"2024-03-18T20:34:38.816659+0000","last_unstale":"2024-03-18T20:34:39.901221+0000","last_undegraded":"2024-03-18T20:34:39.901221+0000","last_fullsized":"2024-03-18T20:34:39.901221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:04:49.316127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998431+0000","last_change":"2024-03-18T20:34:38.804833+0000","last_active":"2024-03-18T20:34:39.998431+0000","last_peered":"2024-03-18T20:34:39.998431+0000","last_clean":"2024-03-18T20:34:39.998431+0000","last_became_active":"2024-03-18T20:34:38.804486+0000","last_became_peered":"2024-03-18T20:34:38.804486+0000","last_unstale":"2024-03-18T20:34:39.998431+0000","last_undegraded":"2024-03-18T20:34:39.998431+0000","last_fullsized":"2024-03-18T20:34:39.998431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:19:44.266207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996571+0000","last_change":"2024-03-18T20:34:38.797660+0000","last_active":"2024-03-18T20:34:39.996571+0000","last_peered":"2024-03-18T20:34:39.996571+0000","last_clean":"2024-03-18T20:34:39.996571+0000","last_became_active":"2024-03-18T20:34:38.797155+0000","last_became_peered":"2024-03-18T20:34:38.797155+0000","last_unstale":"2024-03-18T20:34:39.996571+0000","last_undegraded":"2024-03-18T20:34:39.996571+0000","last_fullsized":"2024-03-18T20:34:39.996571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:22:11.363453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997430+0000","last_change":"2024-03-18T20:34:38.822891+0000","last_active":"2024-03-18T20:34:39.997430+0000","last_peered":"2024-03-18T20:34:39.997430+0000","last_clean":"2024-03-18T20:34:39.997430+0000","last_became_active":"2024-03-18T20:34:38.822574+0000","last_became_peered":"2024-03-18T20:34:38.822574+0000","last_unstale":"2024-03-18T20:34:39.997430+0000","last_undegraded":"2024-03-18T20:34:39.997430+0000","last_fullsized":"2024-03-18T20:34:39.997430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:42:05.727053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997617+0000","last_change":"2024-03-18T20:34:38.805378+0000","last_active":"2024-03-18T20:34:39.997617+0000","last_peered":"2024-03-18T20:34:39.997617+0000","last_clean":"2024-03-18T20:34:39.997617+0000","last_became_active":"2024-03-18T20:34:38.805130+0000","last_became_peered":"2024-03-18T20:34:38.805130+0000","last_unstale":"2024-03-18T20:34:39.997617+0000","last_undegraded":"2024-03-18T20:34:39.997617+0000","last_fullsized":"2024-03-18T20:34:39.997617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:46:19.420715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198520+0000","last_change":"2024-03-18T20:34:38.801034+0000","last_active":"2024-03-18T20:34:39.198520+0000","last_peered":"2024-03-18T20:34:39.198520+0000","last_clean":"2024-03-18T20:34:39.198520+0000","last_became_active":"2024-03-18T20:34:38.800706+0000","last_became_peered":"2024-03-18T20:34:38.800706+0000","last_unstale":"2024-03-18T20:34:39.198520+0000","last_undegraded":"2024-03-18T20:34:39.198520+0000","last_fullsized":"2024-03-18T20:34:39.198520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:10:22.641105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.823531+0000","last_change":"2024-03-18T20:34:38.822841+0000","last_active":"2024-03-18T20:34:38.823531+0000","last_peered":"2024-03-18T20:34:38.823531+0000","last_clean":"2024-03-18T20:34:38.823531+0000","last_became_active":"2024-03-18T20:34:38.822537+0000","last_became_peered":"2024-03-18T20:34:38.822537+0000","last_unstale":"2024-03-18T20:34:38.823531+0000","last_undegraded":"2024-03-18T20:34:38.823531+0000","last_fullsized":"2024-03-18T20:34:38.823531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:38.741995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997436+0000","last_change":"2024-03-18T20:34:38.814728+0000","last_active":"2024-03-18T20:34:39.997436+0000","last_peered":"2024-03-18T20:34:39.997436+0000","last_clean":"2024-03-18T20:34:39.997436+0000","last_became_active":"2024-03-18T20:34:38.814200+0000","last_became_peered":"2024-03-18T20:34:38.814200+0000","last_unstale":"2024-03-18T20:34:39.997436+0000","last_undegraded":"2024-03-18T20:34:39.997436+0000","last_fullsized":"2024-03-18T20:34:39.997436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:24:54.064770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997025+0000","last_change":"2024-03-18T20:34:38.818054+0000","last_active":"2024-03-18T20:34:39.997025+0000","last_peered":"2024-03-18T20:34:39.997025+0000","last_clean":"2024-03-18T20:34:39.997025+0000","last_became_active":"2024-03-18T20:34:38.817752+0000","last_became_peered":"2024-03-18T20:34:38.817752+0000","last_unstale":"2024-03-18T20:34:39.997025+0000","last_undegraded":"2024-03-18T20:34:39.997025+0000","last_fullsized":"2024-03-18T20:34:39.997025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:01:13.728434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998381+0000","last_change":"2024-03-18T20:34:38.801356+0000","last_active":"2024-03-18T20:34:39.998381+0000","last_peered":"2024-03-18T20:34:39.998381+0000","last_clean":"2024-03-18T20:34:39.998381+0000","last_became_active":"2024-03-18T20:34:38.800978+0000","last_became_peered":"2024-03-18T20:34:38.800978+0000","last_unstale":"2024-03-18T20:34:39.998381+0000","last_undegraded":"2024-03-18T20:34:39.998381+0000","last_fullsized":"2024-03-18T20:34:39.998381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:07.634686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793686+0000","last_change":"2024-03-18T20:34:38.803006+0000","last_active":"2024-03-18T20:34:39.793686+0000","last_peered":"2024-03-18T20:34:39.793686+0000","last_clean":"2024-03-18T20:34:39.793686+0000","last_became_active":"2024-03-18T20:34:38.802454+0000","last_became_peered":"2024-03-18T20:34:38.802454+0000","last_unstale":"2024-03-18T20:34:39.793686+0000","last_undegraded":"2024-03-18T20:34:39.793686+0000","last_fullsized":"2024-03-18T20:34:39.793686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:11:02.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901028+0000","last_change":"2024-03-18T20:34:38.807397+0000","last_active":"2024-03-18T20:34:39.901028+0000","last_peered":"2024-03-18T20:34:39.901028+0000","last_clean":"2024-03-18T20:34:39.901028+0000","last_became_active":"2024-03-18T20:34:38.807072+0000","last_became_peered":"2024-03-18T20:34:38.807072+0000","last_unstale":"2024-03-18T20:34:39.901028+0000","last_undegraded":"2024-03-18T20:34:39.901028+0000","last_fullsized":"2024-03-18T20:34:39.901028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:55:53.832825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.900962+0000","last_change":"2024-03-18T20:34:38.812925+0000","last_active":"2024-03-18T20:34:39.900962+0000","last_peered":"2024-03-18T20:34:39.900962+0000","last_clean":"2024-03-18T20:34:39.900962+0000","last_became_active":"2024-03-18T20:34:38.812597+0000","last_became_peered":"2024-03-18T20:34:38.812597+0000","last_unstale":"2024-03-18T20:34:39.900962+0000","last_undegraded":"2024-03-18T20:34:39.900962+0000","last_fullsized":"2024-03-18T20:34:39.900962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:14:35.103852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.820572+0000","last_change":"2024-03-18T20:34:38.817892+0000","last_active":"2024-03-18T20:34:38.820572+0000","last_peered":"2024-03-18T20:34:38.820572+0000","last_clean":"2024-03-18T20:34:38.820572+0000","last_became_active":"2024-03-18T20:34:38.816261+0000","last_became_peered":"2024-03-18T20:34:38.816261+0000","last_unstale":"2024-03-18T20:34:38.820572+0000","last_undegraded":"2024-03-18T20:34:38.820572+0000","last_fullsized":"2024-03-18T20:34:38.820572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:32.481210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.902148+0000","last_change":"2024-03-18T20:34:38.816966+0000","last_active":"2024-03-18T20:34:39.902148+0000","last_peered":"2024-03-18T20:34:39.902148+0000","last_clean":"2024-03-18T20:34:39.902148+0000","last_became_active":"2024-03-18T20:34:38.816645+0000","last_became_peered":"2024-03-18T20:34:38.816645+0000","last_unstale":"2024-03-18T20:34:39.902148+0000","last_undegraded":"2024-03-18T20:34:39.902148+0000","last_fullsized":"2024-03-18T20:34:39.902148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:39.904353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997076+0000","last_change":"2024-03-18T20:34:38.817005+0000","last_active":"2024-03-18T20:34:39.997076+0000","last_peered":"2024-03-18T20:34:39.997076+0000","last_clean":"2024-03-18T20:34:39.997076+0000","last_became_active":"2024-03-18T20:34:38.816710+0000","last_became_peered":"2024-03-18T20:34:38.816710+0000","last_unstale":"2024-03-18T20:34:39.997076+0000","last_undegraded":"2024-03-18T20:34:39.997076+0000","last_fullsized":"2024-03-18T20:34:39.997076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:30:43.483977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997574+0000","last_change":"2024-03-18T20:34:38.810957+0000","last_active":"2024-03-18T20:34:39.997574+0000","last_peered":"2024-03-18T20:34:39.997574+0000","last_clean":"2024-03-18T20:34:39.997574+0000","last_became_active":"2024-03-18T20:34:38.810615+0000","last_became_peered":"2024-03-18T20:34:38.810615+0000","last_unstale":"2024-03-18T20:34:39.997574+0000","last_undegraded":"2024-03-18T20:34:39.997574+0000","last_fullsized":"2024-03-18T20:34:39.997574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:26:36.544034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.794465+0000","last_change":"2024-03-18T20:34:38.337863+0000","last_active":"2024-03-18T20:34:44.794465+0000","last_peered":"2024-03-18T20:34:44.794465+0000","last_clean":"2024-03-18T20:34:44.794465+0000","last_became_active":"2024-03-18T20:34:38.337548+0000","last_became_peered":"2024-03-18T20:34:38.337548+0000","last_unstale":"2024-03-18T20:34:44.794465+0000","last_undegraded":"2024-03-18T20:34:44.794465+0000","last_fullsized":"2024-03-18T20:34:44.794465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:41:40.237882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198626+0000","last_change":"2024-03-18T20:34:38.814321+0000","last_active":"2024-03-18T20:34:39.198626+0000","last_peered":"2024-03-18T20:34:39.198626+0000","last_clean":"2024-03-18T20:34:39.198626+0000","last_became_active":"2024-03-18T20:34:38.813939+0000","last_became_peered":"2024-03-18T20:34:38.813939+0000","last_unstale":"2024-03-18T20:34:39.198626+0000","last_undegraded":"2024-03-18T20:34:39.198626+0000","last_fullsized":"2024-03-18T20:34:39.198626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:05:42.441099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996941+0000","last_change":"2024-03-18T20:34:38.347790+0000","last_active":"2024-03-18T20:34:39.996941+0000","last_peered":"2024-03-18T20:34:39.996941+0000","last_clean":"2024-03-18T20:34:39.996941+0000","last_became_active":"2024-03-18T20:34:38.347317+0000","last_became_peered":"2024-03-18T20:34:38.347317+0000","last_unstale":"2024-03-18T20:34:39.996941+0000","last_undegraded":"2024-03-18T20:34:39.996941+0000","last_fullsized":"2024-03-18T20:34:39.996941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.096480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901274+0000","last_change":"2024-03-18T20:34:38.817116+0000","last_active":"2024-03-18T20:34:39.901274+0000","last_peered":"2024-03-18T20:34:39.901274+0000","last_clean":"2024-03-18T20:34:39.901274+0000","last_became_active":"2024-03-18T20:34:38.816761+0000","last_became_peered":"2024-03-18T20:34:38.816761+0000","last_unstale":"2024-03-18T20:34:39.901274+0000","last_undegraded":"2024-03-18T20:34:39.901274+0000","last_fullsized":"2024-03-18T20:34:39.901274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:44.353533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.820176+0000","last_change":"2024-03-18T20:34:38.339236+0000","last_active":"2024-03-18T20:34:42.820176+0000","last_peered":"2024-03-18T20:34:42.820176+0000","last_clean":"2024-03-18T20:34:42.820176+0000","last_became_active":"2024-03-18T20:34:38.338879+0000","last_became_peered":"2024-03-18T20:34:38.338879+0000","last_unstale":"2024-03-18T20:34:42.820176+0000","last_undegraded":"2024-03-18T20:34:42.820176+0000","last_fullsized":"2024-03-18T20:34:42.820176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:33.340151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.820775+0000","last_change":"2024-03-18T20:34:38.817861+0000","last_active":"2024-03-18T20:34:38.820775+0000","last_peered":"2024-03-18T20:34:38.820775+0000","last_clean":"2024-03-18T20:34:38.820775+0000","last_became_active":"2024-03-18T20:34:38.815976+0000","last_became_peered":"2024-03-18T20:34:38.815976+0000","last_unstale":"2024-03-18T20:34:38.820775+0000","last_undegraded":"2024-03-18T20:34:38.820775+0000","last_fullsized":"2024-03-18T20:34:38.820775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:54.070067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.820328+0000","last_change":"2024-03-18T20:34:38.345758+0000","last_active":"2024-03-18T20:34:38.820328+0000","last_peered":"2024-03-18T20:34:38.820328+0000","last_clean":"2024-03-18T20:34:38.820328+0000","last_became_active":"2024-03-18T20:34:38.345417+0000","last_became_peered":"2024-03-18T20:34:38.345417+0000","last_unstale":"2024-03-18T20:34:38.820328+0000","last_undegraded":"2024-03-18T20:34:38.820328+0000","last_fullsized":"2024-03-18T20:34:38.820328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:25.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759430+0000","last_change":"2024-03-18T20:34:38.818745+0000","last_active":"2024-03-18T20:34:39.759430+0000","last_peered":"2024-03-18T20:34:39.759430+0000","last_clean":"2024-03-18T20:34:39.759430+0000","last_became_active":"2024-03-18T20:34:38.818415+0000","last_became_peered":"2024-03-18T20:34:38.818415+0000","last_unstale":"2024-03-18T20:34:39.759430+0000","last_undegraded":"2024-03-18T20:34:39.759430+0000","last_fullsized":"2024-03-18T20:34:39.759430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:45:49.454498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814903+0000","last_change":"2024-03-18T20:34:38.345804+0000","last_active":"2024-03-18T20:34:41.814903+0000","last_peered":"2024-03-18T20:34:41.814903+0000","last_clean":"2024-03-18T20:34:41.814903+0000","last_became_active":"2024-03-18T20:34:38.345409+0000","last_became_peered":"2024-03-18T20:34:38.345409+0000","last_unstale":"2024-03-18T20:34:41.814903+0000","last_undegraded":"2024-03-18T20:34:41.814903+0000","last_fullsized":"2024-03-18T20:34:41.814903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:00.085412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996703+0000","last_change":"2024-03-18T20:34:38.799821+0000","last_active":"2024-03-18T20:34:39.996703+0000","last_peered":"2024-03-18T20:34:39.996703+0000","last_clean":"2024-03-18T20:34:39.996703+0000","last_became_active":"2024-03-18T20:34:38.799457+0000","last_became_peered":"2024-03-18T20:34:38.799457+0000","last_unstale":"2024-03-18T20:34:39.996703+0000","last_undegraded":"2024-03-18T20:34:39.996703+0000","last_fullsized":"2024-03-18T20:34:39.996703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:39:35.912892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.805945+0000","last_change":"2024-03-18T20:34:38.341999+0000","last_active":"2024-03-18T20:34:38.805945+0000","last_peered":"2024-03-18T20:34:38.805945+0000","last_clean":"2024-03-18T20:34:38.805945+0000","last_became_active":"2024-03-18T20:34:38.341431+0000","last_became_peered":"2024-03-18T20:34:38.341431+0000","last_unstale":"2024-03-18T20:34:38.805945+0000","last_undegraded":"2024-03-18T20:34:38.805945+0000","last_fullsized":"2024-03-18T20:34:38.805945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:07:26.711113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759388+0000","last_change":"2024-03-18T20:34:38.804003+0000","last_active":"2024-03-18T20:34:39.759388+0000","last_peered":"2024-03-18T20:34:39.759388+0000","last_clean":"2024-03-18T20:34:39.759388+0000","last_became_active":"2024-03-18T20:34:38.803667+0000","last_became_peered":"2024-03-18T20:34:38.803667+0000","last_unstale":"2024-03-18T20:34:39.759388+0000","last_undegraded":"2024-03-18T20:34:39.759388+0000","last_fullsized":"2024-03-18T20:34:39.759388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:19:58.230790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.795285+0000","last_change":"2024-03-18T20:34:38.346493+0000","last_active":"2024-03-18T20:34:38.795285+0000","last_peered":"2024-03-18T20:34:38.795285+0000","last_clean":"2024-03-18T20:34:38.795285+0000","last_became_active":"2024-03-18T20:34:38.345925+0000","last_became_peered":"2024-03-18T20:34:38.345925+0000","last_unstale":"2024-03-18T20:34:38.795285+0000","last_undegraded":"2024-03-18T20:34:38.795285+0000","last_fullsized":"2024-03-18T20:34:38.795285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:52:47.773860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198488+0000","last_change":"2024-03-18T20:34:38.807284+0000","last_active":"2024-03-18T20:34:39.198488+0000","last_peered":"2024-03-18T20:34:39.198488+0000","last_clean":"2024-03-18T20:34:39.198488+0000","last_became_active":"2024-03-18T20:34:38.806844+0000","last_became_peered":"2024-03-18T20:34:38.806844+0000","last_unstale":"2024-03-18T20:34:39.198488+0000","last_undegraded":"2024-03-18T20:34:39.198488+0000","last_fullsized":"2024-03-18T20:34:39.198488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:44:35.469953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814733+0000","last_change":"2024-03-18T20:34:38.343090+0000","last_active":"2024-03-18T20:34:41.814733+0000","last_peered":"2024-03-18T20:34:41.814733+0000","last_clean":"2024-03-18T20:34:41.814733+0000","last_became_active":"2024-03-18T20:34:38.342712+0000","last_became_peered":"2024-03-18T20:34:38.342712+0000","last_unstale":"2024-03-18T20:34:41.814733+0000","last_undegraded":"2024-03-18T20:34:41.814733+0000","last_fullsized":"2024-03-18T20:34:41.814733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:14:05.909604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996769+0000","last_change":"2024-03-18T20:34:38.800616+0000","last_active":"2024-03-18T20:34:39.996769+0000","last_peered":"2024-03-18T20:34:39.996769+0000","last_clean":"2024-03-18T20:34:39.996769+0000","last_became_active":"2024-03-18T20:34:38.800340+0000","last_became_peered":"2024-03-18T20:34:38.800340+0000","last_unstale":"2024-03-18T20:34:39.996769+0000","last_undegraded":"2024-03-18T20:34:39.996769+0000","last_fullsized":"2024-03-18T20:34:39.996769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:18.263179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814549+0000","last_change":"2024-03-18T20:34:38.345634+0000","last_active":"2024-03-18T20:34:41.814549+0000","last_peered":"2024-03-18T20:34:41.814549+0000","last_clean":"2024-03-18T20:34:41.814549+0000","last_became_active":"2024-03-18T20:34:38.345275+0000","last_became_peered":"2024-03-18T20:34:38.345275+0000","last_unstale":"2024-03-18T20:34:41.814549+0000","last_undegraded":"2024-03-18T20:34:41.814549+0000","last_fullsized":"2024-03-18T20:34:41.814549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:29.939529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.823111+0000","last_change":"2024-03-18T20:34:38.820286+0000","last_active":"2024-03-18T20:34:38.823111+0000","last_peered":"2024-03-18T20:34:38.823111+0000","last_clean":"2024-03-18T20:34:38.823111+0000","last_became_active":"2024-03-18T20:34:38.819939+0000","last_became_peered":"2024-03-18T20:34:38.819939+0000","last_unstale":"2024-03-18T20:34:38.823111+0000","last_undegraded":"2024-03-18T20:34:38.823111+0000","last_fullsized":"2024-03-18T20:34:38.823111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:19.229339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.794842+0000","last_change":"2024-03-18T20:34:38.341699+0000","last_active":"2024-03-18T20:34:44.794842+0000","last_peered":"2024-03-18T20:34:44.794842+0000","last_clean":"2024-03-18T20:34:44.794842+0000","last_became_active":"2024-03-18T20:34:38.341390+0000","last_became_peered":"2024-03-18T20:34:38.341390+0000","last_unstale":"2024-03-18T20:34:44.794842+0000","last_undegraded":"2024-03-18T20:34:44.794842+0000","last_fullsized":"2024-03-18T20:34:44.794842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:16:02.239414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901314+0000","last_change":"2024-03-18T20:34:38.805316+0000","last_active":"2024-03-18T20:34:39.901314+0000","last_peered":"2024-03-18T20:34:39.901314+0000","last_clean":"2024-03-18T20:34:39.901314+0000","last_became_active":"2024-03-18T20:34:38.804967+0000","last_became_peered":"2024-03-18T20:34:38.804967+0000","last_unstale":"2024-03-18T20:34:39.901314+0000","last_undegraded":"2024-03-18T20:34:39.901314+0000","last_fullsized":"2024-03-18T20:34:39.901314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:54.591401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814531+0000","last_change":"2024-03-18T20:34:38.346821+0000","last_active":"2024-03-18T20:34:41.814531+0000","last_peered":"2024-03-18T20:34:41.814531+0000","last_clean":"2024-03-18T20:34:41.814531+0000","last_became_active":"2024-03-18T20:34:38.346314+0000","last_became_peered":"2024-03-18T20:34:38.346314+0000","last_unstale":"2024-03-18T20:34:41.814531+0000","last_undegraded":"2024-03-18T20:34:41.814531+0000","last_fullsized":"2024-03-18T20:34:41.814531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:12:04.552971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.823287+0000","last_change":"2024-03-18T20:34:38.820479+0000","last_active":"2024-03-18T20:34:38.823287+0000","last_peered":"2024-03-18T20:34:38.823287+0000","last_clean":"2024-03-18T20:34:38.823287+0000","last_became_active":"2024-03-18T20:34:38.820186+0000","last_became_peered":"2024-03-18T20:34:38.820186+0000","last_unstale":"2024-03-18T20:34:38.823287+0000","last_undegraded":"2024-03-18T20:34:38.823287+0000","last_fullsized":"2024-03-18T20:34:38.823287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:15.953764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'8","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814692+0000","last_change":"2024-03-18T20:34:38.344332+0000","last_active":"2024-03-18T20:34:41.814692+0000","last_peered":"2024-03-18T20:34:41.814692+0000","last_clean":"2024-03-18T20:34:41.814692+0000","last_became_active":"2024-03-18T20:34:38.343827+0000","last_became_peered":"2024-03-18T20:34:38.343827+0000","last_unstale":"2024-03-18T20:34:41.814692+0000","last_undegraded":"2024-03-18T20:34:41.814692+0000","last_fullsized":"2024-03-18T20:34:41.814692+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:13:21.725004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996827+0000","last_change":"2024-03-18T20:34:38.814794+0000","last_active":"2024-03-18T20:34:39.996827+0000","last_peered":"2024-03-18T20:34:39.996827+0000","last_clean":"2024-03-18T20:34:39.996827+0000","last_became_active":"2024-03-18T20:34:38.814434+0000","last_became_peered":"2024-03-18T20:34:38.814434+0000","last_unstale":"2024-03-18T20:34:39.996827+0000","last_undegraded":"2024-03-18T20:34:39.996827+0000","last_fullsized":"2024-03-18T20:34:39.996827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:43:19.083212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814504+0000","last_change":"2024-03-18T20:34:38.343894+0000","last_active":"2024-03-18T20:34:41.814504+0000","last_peered":"2024-03-18T20:34:41.814504+0000","last_clean":"2024-03-18T20:34:41.814504+0000","last_became_active":"2024-03-18T20:34:38.343493+0000","last_became_peered":"2024-03-18T20:34:38.343493+0000","last_unstale":"2024-03-18T20:34:41.814504+0000","last_undegraded":"2024-03-18T20:34:41.814504+0000","last_fullsized":"2024-03-18T20:34:41.814504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:13:32.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793766+0000","last_change":"2024-03-18T20:34:38.809968+0000","last_active":"2024-03-18T20:34:39.793766+0000","last_peered":"2024-03-18T20:34:39.793766+0000","last_clean":"2024-03-18T20:34:39.793766+0000","last_became_active":"2024-03-18T20:34:38.809641+0000","last_became_peered":"2024-03-18T20:34:38.809641+0000","last_unstale":"2024-03-18T20:34:39.793766+0000","last_undegraded":"2024-03-18T20:34:39.793766+0000","last_fullsized":"2024-03-18T20:34:39.793766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:50:52.925683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814608+0000","last_change":"2024-03-18T20:34:38.345621+0000","last_active":"2024-03-18T20:34:41.814608+0000","last_peered":"2024-03-18T20:34:41.814608+0000","last_clean":"2024-03-18T20:34:41.814608+0000","last_became_active":"2024-03-18T20:34:38.345255+0000","last_became_peered":"2024-03-18T20:34:38.345255+0000","last_unstale":"2024-03-18T20:34:41.814608+0000","last_undegraded":"2024-03-18T20:34:41.814608+0000","last_fullsized":"2024-03-18T20:34:41.814608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:17:03.737973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997875+0000","last_change":"2024-03-18T20:34:38.804390+0000","last_active":"2024-03-18T20:34:39.997875+0000","last_peered":"2024-03-18T20:34:39.997875+0000","last_clean":"2024-03-18T20:34:39.997875+0000","last_became_active":"2024-03-18T20:34:38.804033+0000","last_became_peered":"2024-03-18T20:34:38.804033+0000","last_unstale":"2024-03-18T20:34:39.997875+0000","last_undegraded":"2024-03-18T20:34:39.997875+0000","last_fullsized":"2024-03-18T20:34:39.997875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:09:45.407921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.818248+0000","last_change":"2024-03-18T20:34:38.340239+0000","last_active":"2024-03-18T20:34:42.818248+0000","last_peered":"2024-03-18T20:34:42.818248+0000","last_clean":"2024-03-18T20:34:42.818248+0000","last_became_active":"2024-03-18T20:34:38.339880+0000","last_became_peered":"2024-03-18T20:34:38.339880+0000","last_unstale":"2024-03-18T20:34:42.818248+0000","last_undegraded":"2024-03-18T20:34:42.818248+0000","last_fullsized":"2024-03-18T20:34:42.818248+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:55:46.864885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998003+0000","last_change":"2024-03-18T20:34:38.819223+0000","last_active":"2024-03-18T20:34:39.998003+0000","last_peered":"2024-03-18T20:34:39.998003+0000","last_clean":"2024-03-18T20:34:39.998003+0000","last_became_active":"2024-03-18T20:34:38.818879+0000","last_became_peered":"2024-03-18T20:34:38.818879+0000","last_unstale":"2024-03-18T20:34:39.998003+0000","last_undegraded":"2024-03-18T20:34:39.998003+0000","last_fullsized":"2024-03-18T20:34:39.998003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:45.645912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.821003+0000","last_change":"2024-03-18T20:34:38.336817+0000","last_active":"2024-03-18T20:34:42.821003+0000","last_peered":"2024-03-18T20:34:42.821003+0000","last_clean":"2024-03-18T20:34:42.821003+0000","last_became_active":"2024-03-18T20:34:38.336453+0000","last_became_peered":"2024-03-18T20:34:38.336453+0000","last_unstale":"2024-03-18T20:34:42.821003+0000","last_undegraded":"2024-03-18T20:34:42.821003+0000","last_fullsized":"2024-03-18T20:34:42.821003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:10.253435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759322+0000","last_change":"2024-03-18T20:34:38.818048+0000","last_active":"2024-03-18T20:34:39.759322+0000","last_peered":"2024-03-18T20:34:39.759322+0000","last_clean":"2024-03-18T20:34:39.759322+0000","last_became_active":"2024-03-18T20:34:38.817682+0000","last_became_peered":"2024-03-18T20:34:38.817682+0000","last_unstale":"2024-03-18T20:34:39.759322+0000","last_undegraded":"2024-03-18T20:34:39.759322+0000","last_fullsized":"2024-03-18T20:34:39.759322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:25:55.291815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.820282+0000","last_change":"2024-03-18T20:34:38.345773+0000","last_active":"2024-03-18T20:34:38.820282+0000","last_peered":"2024-03-18T20:34:38.820282+0000","last_clean":"2024-03-18T20:34:38.820282+0000","last_became_active":"2024-03-18T20:34:38.345432+0000","last_became_peered":"2024-03-18T20:34:38.345432+0000","last_unstale":"2024-03-18T20:34:38.820282+0000","last_undegraded":"2024-03-18T20:34:38.820282+0000","last_fullsized":"2024-03-18T20:34:38.820282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:56:57.657424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901903+0000","last_change":"2024-03-18T20:34:38.806138+0000","last_active":"2024-03-18T20:34:39.901903+0000","last_peered":"2024-03-18T20:34:39.901903+0000","last_clean":"2024-03-18T20:34:39.901903+0000","last_became_active":"2024-03-18T20:34:38.805821+0000","last_became_peered":"2024-03-18T20:34:38.805821+0000","last_unstale":"2024-03-18T20:34:39.901903+0000","last_undegraded":"2024-03-18T20:34:39.901903+0000","last_fullsized":"2024-03-18T20:34:39.901903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:25:57.887813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.173015+0000","last_change":"2024-03-18T20:34:38.338751+0000","last_active":"2024-03-18T20:34:41.173015+0000","last_peered":"2024-03-18T20:34:41.173015+0000","last_clean":"2024-03-18T20:34:41.173015+0000","last_became_active":"2024-03-18T20:34:38.337495+0000","last_became_peered":"2024-03-18T20:34:38.337495+0000","last_unstale":"2024-03-18T20:34:41.173015+0000","last_undegraded":"2024-03-18T20:34:41.173015+0000","last_fullsized":"2024-03-18T20:34:41.173015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:38:04.402380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997145+0000","last_change":"2024-03-18T20:34:38.803937+0000","last_active":"2024-03-18T20:34:39.997145+0000","last_peered":"2024-03-18T20:34:39.997145+0000","last_clean":"2024-03-18T20:34:39.997145+0000","last_became_active":"2024-03-18T20:34:38.803520+0000","last_became_peered":"2024-03-18T20:34:38.803520+0000","last_unstale":"2024-03-18T20:34:39.997145+0000","last_undegraded":"2024-03-18T20:34:39.997145+0000","last_fullsized":"2024-03-18T20:34:39.997145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:20:37.537265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.794027+0000","last_change":"2024-03-18T20:34:38.337481+0000","last_active":"2024-03-18T20:34:44.794027+0000","last_peered":"2024-03-18T20:34:44.794027+0000","last_clean":"2024-03-18T20:34:44.794027+0000","last_became_active":"2024-03-18T20:34:38.337152+0000","last_became_peered":"2024-03-18T20:34:38.337152+0000","last_unstale":"2024-03-18T20:34:44.794027+0000","last_undegraded":"2024-03-18T20:34:44.794027+0000","last_fullsized":"2024-03-18T20:34:44.794027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:07:59.593978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997194+0000","last_change":"2024-03-18T20:34:38.814240+0000","last_active":"2024-03-18T20:34:39.997194+0000","last_peered":"2024-03-18T20:34:39.997194+0000","last_clean":"2024-03-18T20:34:39.997194+0000","last_became_active":"2024-03-18T20:34:38.813177+0000","last_became_peered":"2024-03-18T20:34:38.813177+0000","last_unstale":"2024-03-18T20:34:39.997194+0000","last_undegraded":"2024-03-18T20:34:39.997194+0000","last_fullsized":"2024-03-18T20:34:39.997194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:22:30.076880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759286+0000","last_change":"2024-03-18T20:34:38.806133+0000","last_active":"2024-03-18T20:34:39.759286+0000","last_peered":"2024-03-18T20:34:39.759286+0000","last_clean":"2024-03-18T20:34:39.759286+0000","last_became_active":"2024-03-18T20:34:38.805257+0000","last_became_peered":"2024-03-18T20:34:38.805257+0000","last_unstale":"2024-03-18T20:34:39.759286+0000","last_undegraded":"2024-03-18T20:34:39.759286+0000","last_fullsized":"2024-03-18T20:34:39.759286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:05:15.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793220+0000","last_change":"2024-03-18T20:34:38.346271+0000","last_active":"2024-03-18T20:34:39.793220+0000","last_peered":"2024-03-18T20:34:39.793220+0000","last_clean":"2024-03-18T20:34:39.793220+0000","last_became_active":"2024-03-18T20:34:38.345054+0000","last_became_peered":"2024-03-18T20:34:38.345054+0000","last_unstale":"2024-03-18T20:34:39.793220+0000","last_undegraded":"2024-03-18T20:34:39.793220+0000","last_fullsized":"2024-03-18T20:34:39.793220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:42:52.510294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198582+0000","last_change":"2024-03-18T20:34:38.816834+0000","last_active":"2024-03-18T20:34:39.198582+0000","last_peered":"2024-03-18T20:34:39.198582+0000","last_clean":"2024-03-18T20:34:39.198582+0000","last_became_active":"2024-03-18T20:34:38.816516+0000","last_became_peered":"2024-03-18T20:34:38.816516+0000","last_unstale":"2024-03-18T20:34:39.198582+0000","last_undegraded":"2024-03-18T20:34:39.198582+0000","last_fullsized":"2024-03-18T20:34:39.198582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:33:54.529206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.813927+0000","last_change":"2024-03-18T20:34:38.339407+0000","last_active":"2024-03-18T20:34:41.813927+0000","last_peered":"2024-03-18T20:34:41.813927+0000","last_clean":"2024-03-18T20:34:41.813927+0000","last_became_active":"2024-03-18T20:34:38.339086+0000","last_became_peered":"2024-03-18T20:34:38.339086+0000","last_unstale":"2024-03-18T20:34:41.813927+0000","last_undegraded":"2024-03-18T20:34:41.813927+0000","last_fullsized":"2024-03-18T20:34:41.813927+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:18.479630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":330,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.801661+0000","last_change":"2024-03-18T20:33:50.777651+0000","last_active":"2024-03-18T20:34:38.801661+0000","last_peered":"2024-03-18T20:34:38.801661+0000","last_clean":"2024-03-18T20:34:38.801661+0000","last_became_active":"2024-03-18T20:33:50.777281+0000","last_became_peered":"2024-03-18T20:33:50.777281+0000","last_unstale":"2024-03-18T20:34:38.801661+0000","last_undegraded":"2024-03-18T20:34:38.801661+0000","last_fullsized":"2024-03-18T20:34:38.801661+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_clean_scrub_stamp":"2024-03-18T20:31:27.488744+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:28.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793222+0000","last_change":"2024-03-18T20:34:38.812714+0000","last_active":"2024-03-18T20:34:39.793222+0000","last_peered":"2024-03-18T20:34:39.793222+0000","last_clean":"2024-03-18T20:34:39.793222+0000","last_became_active":"2024-03-18T20:34:38.812464+0000","last_became_peered":"2024-03-18T20:34:38.812464+0000","last_unstale":"2024-03-18T20:34:39.793222+0000","last_undegraded":"2024-03-18T20:34:39.793222+0000","last_fullsized":"2024-03-18T20:34:39.793222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:03:13.853740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.820233+0000","last_change":"2024-03-18T20:34:38.343173+0000","last_active":"2024-03-18T20:34:42.820233+0000","last_peered":"2024-03-18T20:34:42.820233+0000","last_clean":"2024-03-18T20:34:42.820233+0000","last_became_active":"2024-03-18T20:34:38.342774+0000","last_became_peered":"2024-03-18T20:34:38.342774+0000","last_unstale":"2024-03-18T20:34:42.820233+0000","last_undegraded":"2024-03-18T20:34:42.820233+0000","last_fullsized":"2024-03-18T20:34:42.820233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:52.283033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996953+0000","last_change":"2024-03-18T20:34:38.344539+0000","last_active":"2024-03-18T20:34:39.996953+0000","last_peered":"2024-03-18T20:34:39.996953+0000","last_clean":"2024-03-18T20:34:39.996953+0000","last_became_active":"2024-03-18T20:34:38.344188+0000","last_became_peered":"2024-03-18T20:34:38.344188+0000","last_unstale":"2024-03-18T20:34:39.996953+0000","last_undegraded":"2024-03-18T20:34:39.996953+0000","last_fullsized":"2024-03-18T20:34:39.996953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:38.641928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793487+0000","last_change":"2024-03-18T20:34:38.806602+0000","last_active":"2024-03-18T20:34:39.793487+0000","last_peered":"2024-03-18T20:34:39.793487+0000","last_clean":"2024-03-18T20:34:39.793487+0000","last_became_active":"2024-03-18T20:34:38.806241+0000","last_became_peered":"2024-03-18T20:34:38.806241+0000","last_unstale":"2024-03-18T20:34:39.793487+0000","last_undegraded":"2024-03-18T20:34:39.793487+0000","last_fullsized":"2024-03-18T20:34:39.793487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:04.463311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'7","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:43.831715+0000","last_change":"2024-03-18T20:34:38.339695+0000","last_active":"2024-03-18T20:34:43.831715+0000","last_peered":"2024-03-18T20:34:43.831715+0000","last_clean":"2024-03-18T20:34:43.831715+0000","last_became_active":"2024-03-18T20:34:38.339331+0000","last_became_peered":"2024-03-18T20:34:38.339331+0000","last_unstale":"2024-03-18T20:34:43.831715+0000","last_undegraded":"2024-03-18T20:34:43.831715+0000","last_fullsized":"2024-03-18T20:34:43.831715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:57:06.434289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198695+0000","last_change":"2024-03-18T20:34:38.809754+0000","last_active":"2024-03-18T20:34:39.198695+0000","last_peered":"2024-03-18T20:34:39.198695+0000","last_clean":"2024-03-18T20:34:39.198695+0000","last_became_active":"2024-03-18T20:34:38.809415+0000","last_became_peered":"2024-03-18T20:34:38.809415+0000","last_unstale":"2024-03-18T20:34:39.198695+0000","last_undegraded":"2024-03-18T20:34:39.198695+0000","last_fullsized":"2024-03-18T20:34:39.198695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:34:30.761582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.813819+0000","last_change":"2024-03-18T20:34:38.342080+0000","last_active":"2024-03-18T20:34:41.813819+0000","last_peered":"2024-03-18T20:34:41.813819+0000","last_clean":"2024-03-18T20:34:41.813819+0000","last_became_active":"2024-03-18T20:34:38.340623+0000","last_became_peered":"2024-03-18T20:34:38.340623+0000","last_unstale":"2024-03-18T20:34:41.813819+0000","last_undegraded":"2024-03-18T20:34:41.813819+0000","last_fullsized":"2024-03-18T20:34:41.813819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:49:14.912391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901335+0000","last_change":"2024-03-18T20:34:38.806558+0000","last_active":"2024-03-18T20:34:39.901335+0000","last_peered":"2024-03-18T20:34:39.901335+0000","last_clean":"2024-03-18T20:34:39.901335+0000","last_became_active":"2024-03-18T20:34:38.806171+0000","last_became_peered":"2024-03-18T20:34:38.806171+0000","last_unstale":"2024-03-18T20:34:39.901335+0000","last_undegraded":"2024-03-18T20:34:39.901335+0000","last_fullsized":"2024-03-18T20:34:39.901335+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:46:26.530157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.813762+0000","last_change":"2024-03-18T20:34:38.336175+0000","last_active":"2024-03-18T20:34:41.813762+0000","last_peered":"2024-03-18T20:34:41.813762+0000","last_clean":"2024-03-18T20:34:41.813762+0000","last_became_active":"2024-03-18T20:34:38.335798+0000","last_became_peered":"2024-03-18T20:34:38.335798+0000","last_unstale":"2024-03-18T20:34:41.813762+0000","last_undegraded":"2024-03-18T20:34:41.813762+0000","last_fullsized":"2024-03-18T20:34:41.813762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:33:33.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996504+0000","last_change":"2024-03-18T20:34:38.807684+0000","last_active":"2024-03-18T20:34:39.996504+0000","last_peered":"2024-03-18T20:34:39.996504+0000","last_clean":"2024-03-18T20:34:39.996504+0000","last_became_active":"2024-03-18T20:34:38.807300+0000","last_became_peered":"2024-03-18T20:34:38.807300+0000","last_unstale":"2024-03-18T20:34:39.996504+0000","last_undegraded":"2024-03-18T20:34:39.996504+0000","last_fullsized":"2024-03-18T20:34:39.996504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:31:43.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'5","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:43.831864+0000","last_change":"2024-03-18T20:34:38.341802+0000","last_active":"2024-03-18T20:34:43.831864+0000","last_peered":"2024-03-18T20:34:43.831864+0000","last_clean":"2024-03-18T20:34:43.831864+0000","last_became_active":"2024-03-18T20:34:38.340594+0000","last_became_peered":"2024-03-18T20:34:38.340594+0000","last_unstale":"2024-03-18T20:34:43.831864+0000","last_undegraded":"2024-03-18T20:34:43.831864+0000","last_fullsized":"2024-03-18T20:34:43.831864+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:01:06.830323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901738+0000","last_change":"2024-03-18T20:34:38.809790+0000","last_active":"2024-03-18T20:34:39.901738+0000","last_peered":"2024-03-18T20:34:39.901738+0000","last_clean":"2024-03-18T20:34:39.901738+0000","last_became_active":"2024-03-18T20:34:38.809470+0000","last_became_peered":"2024-03-18T20:34:38.809470+0000","last_unstale":"2024-03-18T20:34:39.901738+0000","last_undegraded":"2024-03-18T20:34:39.901738+0000","last_fullsized":"2024-03-18T20:34:39.901738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:48.225759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997300+0000","last_change":"2024-03-18T20:34:38.343103+0000","last_active":"2024-03-18T20:34:39.997300+0000","last_peered":"2024-03-18T20:34:39.997300+0000","last_clean":"2024-03-18T20:34:39.997300+0000","last_became_active":"2024-03-18T20:34:38.342744+0000","last_became_peered":"2024-03-18T20:34:38.342744+0000","last_unstale":"2024-03-18T20:34:39.997300+0000","last_undegraded":"2024-03-18T20:34:39.997300+0000","last_fullsized":"2024-03-18T20:34:39.997300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:39.078219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198604+0000","last_change":"2024-03-18T20:34:38.809294+0000","last_active":"2024-03-18T20:34:39.198604+0000","last_peered":"2024-03-18T20:34:39.198604+0000","last_clean":"2024-03-18T20:34:39.198604+0000","last_became_active":"2024-03-18T20:34:38.808542+0000","last_became_peered":"2024-03-18T20:34:38.808542+0000","last_unstale":"2024-03-18T20:34:39.198604+0000","last_undegraded":"2024-03-18T20:34:39.198604+0000","last_fullsized":"2024-03-18T20:34:39.198604+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:11:30.669317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.813639+0000","last_change":"2024-03-18T20:34:38.339454+0000","last_active":"2024-03-18T20:34:41.813639+0000","last_peered":"2024-03-18T20:34:41.813639+0000","last_clean":"2024-03-18T20:34:41.813639+0000","last_became_active":"2024-03-18T20:34:38.339154+0000","last_became_peered":"2024-03-18T20:34:38.339154+0000","last_unstale":"2024-03-18T20:34:41.813639+0000","last_undegraded":"2024-03-18T20:34:41.813639+0000","last_fullsized":"2024-03-18T20:34:41.813639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:12:06.300600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998255+0000","last_change":"2024-03-18T20:34:38.818462+0000","last_active":"2024-03-18T20:34:39.998255+0000","last_peered":"2024-03-18T20:34:39.998255+0000","last_clean":"2024-03-18T20:34:39.998255+0000","last_became_active":"2024-03-18T20:34:38.817522+0000","last_became_peered":"2024-03-18T20:34:38.817522+0000","last_unstale":"2024-03-18T20:34:39.998255+0000","last_undegraded":"2024-03-18T20:34:39.998255+0000","last_fullsized":"2024-03-18T20:34:39.998255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:50:32.258583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997694+0000","last_change":"2024-03-18T20:34:38.341054+0000","last_active":"2024-03-18T20:34:39.997694+0000","last_peered":"2024-03-18T20:34:39.997694+0000","last_clean":"2024-03-18T20:34:39.997694+0000","last_became_active":"2024-03-18T20:34:38.340352+0000","last_became_peered":"2024-03-18T20:34:38.340352+0000","last_unstale":"2024-03-18T20:34:39.997694+0000","last_undegraded":"2024-03-18T20:34:39.997694+0000","last_fullsized":"2024-03-18T20:34:39.997694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:44.850244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997144+0000","last_change":"2024-03-18T20:34:38.818176+0000","last_active":"2024-03-18T20:34:39.997144+0000","last_peered":"2024-03-18T20:34:39.997144+0000","last_clean":"2024-03-18T20:34:39.997144+0000","last_became_active":"2024-03-18T20:34:38.817820+0000","last_became_peered":"2024-03-18T20:34:38.817820+0000","last_unstale":"2024-03-18T20:34:39.997144+0000","last_undegraded":"2024-03-18T20:34:39.997144+0000","last_fullsized":"2024-03-18T20:34:39.997144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:19:29.815174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.795014+0000","last_change":"2024-03-18T20:34:38.346591+0000","last_active":"2024-03-18T20:34:39.795014+0000","last_peered":"2024-03-18T20:34:39.795014+0000","last_clean":"2024-03-18T20:34:39.795014+0000","last_became_active":"2024-03-18T20:34:38.345897+0000","last_became_peered":"2024-03-18T20:34:38.345897+0000","last_unstale":"2024-03-18T20:34:39.795014+0000","last_undegraded":"2024-03-18T20:34:39.795014+0000","last_fullsized":"2024-03-18T20:34:39.795014+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:47:32.118905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997344+0000","last_change":"2024-03-18T20:34:38.804814+0000","last_active":"2024-03-18T20:34:39.997344+0000","last_peered":"2024-03-18T20:34:39.997344+0000","last_clean":"2024-03-18T20:34:39.997344+0000","last_became_active":"2024-03-18T20:34:38.804585+0000","last_became_peered":"2024-03-18T20:34:38.804585+0000","last_unstale":"2024-03-18T20:34:39.997344+0000","last_undegraded":"2024-03-18T20:34:39.997344+0000","last_fullsized":"2024-03-18T20:34:39.997344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:40:47.273070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.818161+0000","last_change":"2024-03-18T20:34:38.344615+0000","last_active":"2024-03-18T20:34:38.818161+0000","last_peered":"2024-03-18T20:34:38.818161+0000","last_clean":"2024-03-18T20:34:38.818161+0000","last_became_active":"2024-03-18T20:34:38.344242+0000","last_became_peered":"2024-03-18T20:34:38.344242+0000","last_unstale":"2024-03-18T20:34:38.818161+0000","last_undegraded":"2024-03-18T20:34:38.818161+0000","last_fullsized":"2024-03-18T20:34:38.818161+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.449808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997264+0000","last_change":"2024-03-18T20:34:38.820875+0000","last_active":"2024-03-18T20:34:39.997264+0000","last_peered":"2024-03-18T20:34:39.997264+0000","last_clean":"2024-03-18T20:34:39.997264+0000","last_became_active":"2024-03-18T20:34:38.820550+0000","last_became_peered":"2024-03-18T20:34:38.820550+0000","last_unstale":"2024-03-18T20:34:39.997264+0000","last_undegraded":"2024-03-18T20:34:39.997264+0000","last_fullsized":"2024-03-18T20:34:39.997264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:49:36.358734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901824+0000","last_change":"2024-03-18T20:34:38.812552+0000","last_active":"2024-03-18T20:34:39.901824+0000","last_peered":"2024-03-18T20:34:39.901824+0000","last_clean":"2024-03-18T20:34:39.901824+0000","last_became_active":"2024-03-18T20:34:38.811939+0000","last_became_peered":"2024-03-18T20:34:38.811939+0000","last_unstale":"2024-03-18T20:34:39.901824+0000","last_undegraded":"2024-03-18T20:34:39.901824+0000","last_fullsized":"2024-03-18T20:34:39.901824+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:41:52.096974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997308+0000","last_change":"2024-03-18T20:34:38.815691+0000","last_active":"2024-03-18T20:34:39.997308+0000","last_peered":"2024-03-18T20:34:39.997308+0000","last_clean":"2024-03-18T20:34:39.997308+0000","last_became_active":"2024-03-18T20:34:38.815370+0000","last_became_peered":"2024-03-18T20:34:38.815370+0000","last_unstale":"2024-03-18T20:34:39.997308+0000","last_undegraded":"2024-03-18T20:34:39.997308+0000","last_fullsized":"2024-03-18T20:34:39.997308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:59:41.260013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997260+0000","last_change":"2024-03-18T20:34:38.812651+0000","last_active":"2024-03-18T20:34:39.997260+0000","last_peered":"2024-03-18T20:34:39.997260+0000","last_clean":"2024-03-18T20:34:39.997260+0000","last_became_active":"2024-03-18T20:34:38.812322+0000","last_became_peered":"2024-03-18T20:34:38.812322+0000","last_unstale":"2024-03-18T20:34:39.997260+0000","last_undegraded":"2024-03-18T20:34:39.997260+0000","last_fullsized":"2024-03-18T20:34:39.997260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:34:55.658371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901247+0000","last_change":"2024-03-18T20:34:38.806475+0000","last_active":"2024-03-18T20:34:39.901247+0000","last_peered":"2024-03-18T20:34:39.901247+0000","last_clean":"2024-03-18T20:34:39.901247+0000","last_became_active":"2024-03-18T20:34:38.806027+0000","last_became_peered":"2024-03-18T20:34:38.806027+0000","last_unstale":"2024-03-18T20:34:39.901247+0000","last_undegraded":"2024-03-18T20:34:39.901247+0000","last_fullsized":"2024-03-18T20:34:39.901247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:57:17.140031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793496+0000","last_change":"2024-03-18T20:34:38.811057+0000","last_active":"2024-03-18T20:34:39.793496+0000","last_peered":"2024-03-18T20:34:39.793496+0000","last_clean":"2024-03-18T20:34:39.793496+0000","last_became_active":"2024-03-18T20:34:38.810316+0000","last_became_peered":"2024-03-18T20:34:38.810316+0000","last_unstale":"2024-03-18T20:34:39.793496+0000","last_undegraded":"2024-03-18T20:34:39.793496+0000","last_fullsized":"2024-03-18T20:34:39.793496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:29:22.426717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.902010+0000","last_change":"2024-03-18T20:34:38.812952+0000","last_active":"2024-03-18T20:34:39.902010+0000","last_peered":"2024-03-18T20:34:39.902010+0000","last_clean":"2024-03-18T20:34:39.902010+0000","last_became_active":"2024-03-18T20:34:38.812550+0000","last_became_peered":"2024-03-18T20:34:38.812550+0000","last_unstale":"2024-03-18T20:34:39.902010+0000","last_undegraded":"2024-03-18T20:34:39.902010+0000","last_fullsized":"2024-03-18T20:34:39.902010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:06:26.525078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997748+0000","last_change":"2024-03-18T20:34:38.817358+0000","last_active":"2024-03-18T20:34:39.997748+0000","last_peered":"2024-03-18T20:34:39.997748+0000","last_clean":"2024-03-18T20:34:39.997748+0000","last_became_active":"2024-03-18T20:34:38.816939+0000","last_became_peered":"2024-03-18T20:34:38.816939+0000","last_unstale":"2024-03-18T20:34:39.997748+0000","last_undegraded":"2024-03-18T20:34:39.997748+0000","last_fullsized":"2024-03-18T20:34:39.997748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:33.258162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198671+0000","last_change":"2024-03-18T20:34:38.807798+0000","last_active":"2024-03-18T20:34:39.198671+0000","last_peered":"2024-03-18T20:34:39.198671+0000","last_clean":"2024-03-18T20:34:39.198671+0000","last_became_active":"2024-03-18T20:34:38.807489+0000","last_became_peered":"2024-03-18T20:34:38.807489+0000","last_unstale":"2024-03-18T20:34:39.198671+0000","last_undegraded":"2024-03-18T20:34:39.198671+0000","last_fullsized":"2024-03-18T20:34:39.198671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:15.395763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759267+0000","last_change":"2024-03-18T20:34:38.806363+0000","last_active":"2024-03-18T20:34:39.759267+0000","last_peered":"2024-03-18T20:34:39.759267+0000","last_clean":"2024-03-18T20:34:39.759267+0000","last_became_active":"2024-03-18T20:34:38.805707+0000","last_became_peered":"2024-03-18T20:34:38.805707+0000","last_unstale":"2024-03-18T20:34:39.759267+0000","last_undegraded":"2024-03-18T20:34:39.759267+0000","last_fullsized":"2024-03-18T20:34:39.759267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:10.918979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198542+0000","last_change":"2024-03-18T20:34:38.814439+0000","last_active":"2024-03-18T20:34:39.198542+0000","last_peered":"2024-03-18T20:34:39.198542+0000","last_clean":"2024-03-18T20:34:39.198542+0000","last_became_active":"2024-03-18T20:34:38.813988+0000","last_became_peered":"2024-03-18T20:34:38.813988+0000","last_unstale":"2024-03-18T20:34:39.198542+0000","last_undegraded":"2024-03-18T20:34:39.198542+0000","last_fullsized":"2024-03-18T20:34:39.198542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:03:14.639044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.821809+0000","last_change":"2024-03-18T20:34:38.818550+0000","last_active":"2024-03-18T20:34:38.821809+0000","last_peered":"2024-03-18T20:34:38.821809+0000","last_clean":"2024-03-18T20:34:38.821809+0000","last_became_active":"2024-03-18T20:34:38.818209+0000","last_became_peered":"2024-03-18T20:34:38.818209+0000","last_unstale":"2024-03-18T20:34:38.821809+0000","last_undegraded":"2024-03-18T20:34:38.821809+0000","last_fullsized":"2024-03-18T20:34:38.821809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:40:48.395331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793586+0000","last_change":"2024-03-18T20:34:38.808439+0000","last_active":"2024-03-18T20:34:39.793586+0000","last_peered":"2024-03-18T20:34:39.793586+0000","last_clean":"2024-03-18T20:34:39.793586+0000","last_became_active":"2024-03-18T20:34:38.807995+0000","last_became_peered":"2024-03-18T20:34:38.807995+0000","last_unstale":"2024-03-18T20:34:39.793586+0000","last_undegraded":"2024-03-18T20:34:39.793586+0000","last_fullsized":"2024-03-18T20:34:39.793586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:54.414148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901907+0000","last_change":"2024-03-18T20:34:38.810793+0000","last_active":"2024-03-18T20:34:39.901907+0000","last_peered":"2024-03-18T20:34:39.901907+0000","last_clean":"2024-03-18T20:34:39.901907+0000","last_became_active":"2024-03-18T20:34:38.810496+0000","last_became_peered":"2024-03-18T20:34:38.810496+0000","last_unstale":"2024-03-18T20:34:39.901907+0000","last_undegraded":"2024-03-18T20:34:39.901907+0000","last_fullsized":"2024-03-18T20:34:39.901907+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:33.780675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5708,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":130,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":16538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612618,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":851968,"data_stored":405743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":10,"up_from":66,"seq":283467841549,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1466368,"data_stored":997156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]}]},{"osd":9,"up_from":61,"seq":261993005074,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":405721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.32200000000000001}]},{"osd":1,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.443}]},{"osd":2,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.308}]},{"osd":4,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.47699999999999998}]},{"osd":5,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.49099999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.39600000000000002}]},{"osd":7,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52100000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.67000000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]}]},{"osd":8,"up_from":54,"seq":231928234005,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":405225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.68700000000000006}]},{"osd":2,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.58999999999999997}]},{"osd":3,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.89000000000000001}]},{"osd":4,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.749},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.83299999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.80300000000000005}]},{"osd":6,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.93500000000000005}]},{"osd":7,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.85199999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.97299999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":7,"up_from":47,"seq":201863462936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1466368,"data_stored":998357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.439}]},{"osd":1,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.378}]},{"osd":2,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.66500000000000004}]},{"osd":3,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53900000000000003}]},{"osd":4,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.39900000000000002}]},{"osd":5,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.64200000000000002}]},{"osd":6,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.67600000000000005}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.749}]},{"osd":9,"last update":"Mon Mar 18 20:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.73499999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":6,"up_from":42,"seq":180388626459,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":404599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.33500000000000002}]},{"osd":1,"last update":"Mon Mar 18 20:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.36399999999999999}]},{"osd":2,"last update":"Mon Mar 18 20:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.627}]},{"osd":3,"last update":"Mon Mar 18 20:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54000000000000004}]},{"osd":4,"last update":"Mon Mar 18 20:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.436},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46999999999999997}]},{"osd":5,"last update":"Mon Mar 18 20:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.61499999999999999}]},{"osd":7,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.59999999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.71399999999999997}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.64100000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":406163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.54300000000000004},"min":{"1min":0.32400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.81799999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.432}]},{"osd":2,"last update":"Mon Mar 18 20:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.745,"5min":0.754,"15min":0.754},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71399999999999997}]},{"osd":3,"last update":"Mon Mar 18 20:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71599999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72999999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51400000000000001}]},{"osd":4,"last update":"Mon Mar 18 20:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.41499999999999998,"15min":0.40100000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.42399999999999999,"15min":0.41199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.61399999999999999}]},{"osd":5,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.749}]},{"osd":6,"last update":"Mon Mar 18 20:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0229999999999999,"5min":1.196,"15min":1.196},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.434,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79200000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56100000000000005}]},{"osd":7,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.52700000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.79400000000000004}]},{"osd":9,"last update":"Mon Mar 18 20:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.65600000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738415,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":851968,"data_stored":405293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.36699999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Mar 18 20:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.44700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.627},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92600000000000005,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.745}]},{"osd":3,"last update":"Mon Mar 18 20:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.32700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.38300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.61599999999999999}]},{"osd":4,"last update":"Mon Mar 18 20:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.33800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.64300000000000002}]},{"osd":5,"last update":"Mon Mar 18 20:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Mar 18 20:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.80800000000000005}]},{"osd":7,"last update":"Mon Mar 18 20:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.88700000000000001}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.78300000000000003}]},{"osd":9,"last update":"Mon Mar 18 20:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.85399999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411370,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":405528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.48699999999999999,"15min":0.46400000000000002},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.75,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.48399999999999999,"15min":0.46300000000000002},"min":{"1min":0.318,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57999999999999996}]},{"osd":1,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.56000000000000005,"15min":0.52900000000000003},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.6100000000000003,"5min":4.6100000000000003,"15min":4.6100000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.71599999999999997}]},{"osd":3,"last update":"Mon Mar 18 20:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52300000000000002,"15min":0.51300000000000001},"min":{"1min":0.39900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.52300000000000002},"min":{"1min":0.40799999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68999999999999995}]},{"osd":4,"last update":"Mon Mar 18 20:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.436,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.44500000000000001,"15min":0.438},"min":{"1min":0.307,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35099999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42699999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84999999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.626},{"interface":"front","average":{"1min":0.625,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.77100000000000002,"5min":1.083,"15min":1.083},"last":0.74199999999999999}]},{"osd":7,"last update":"Mon Mar 18 20:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.439},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45800000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.53900000000000003}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":3,"up_from":24,"seq":103079215141,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":404883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.53400000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.40600000000000003,"15min":0.40799999999999997},"min":{"1min":0.28799999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.503,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Mon Mar 18 20:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53300000000000003,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73499999999999999,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68200000000000005}]},{"osd":2,"last update":"Mon Mar 18 20:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72899999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.622},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.80800000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Mon Mar 18 20:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83199999999999996,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.01,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Mar 18 20:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.499,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.47399999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.79900000000000004}]},{"osd":6,"last update":"Mon Mar 18 20:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83999999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.755}]},{"osd":7,"last update":"Mon Mar 18 20:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73099999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.81899999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":404621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.38400000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.35799999999999998,"15min":0.35399999999999998},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.495,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.439}]},{"osd":2,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.41699999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.433,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.874,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65300000000000002}]},{"osd":3,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53600000000000003}]},{"osd":5,"last update":"Mon Mar 18 20:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77600000000000002,"5min":0.875,"15min":0.875},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Mar 18 20:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.48199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.747}]},{"osd":7,"last update":"Mon Mar 18 20:33:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58999999999999997}]},{"osd":8,"last update":"Mon Mar 18 20:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.81999999999999995}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77700000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]}]},{"osd":5,"up_from":35,"seq":150323855391,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1470464,"data_stored":996826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60999999999999999}]},{"osd":1,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69099999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.42999999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.41499999999999998}]},{"osd":2,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.27900000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.27900000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.35699999999999998}]},{"osd":3,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.35499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76300000000000001,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.433}]},{"osd":4,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.75800000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.66000000000000003}]},{"osd":6,"last update":"Mon Mar 18 20:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58499999999999996}]},{"osd":7,"last update":"Mon Mar 18 20:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.625},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46899999999999997}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.50900000000000001}]},{"osd":9,"last update":"Mon Mar 18 20:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67100000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T20:34:47.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T20:34:47.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:46 smithi155 ceph-mon[48344]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:47.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:46 smithi155 ceph-mon[48344]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:34:47.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:46 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1164868002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:34:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.442+0000 7fafc2c23700 1 -- 172.21.15.46:0/1852007601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafbc074d70 msgr2=0x7fafbc0731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.442+0000 7fafc2c23700 1 --2- 172.21.15.46:0/1852007601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafbc074d70 0x7fafbc0731c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009990 tx=0x7fafb0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.442+0000 7fafc2c23700 1 -- 172.21.15.46:0/1852007601 shutdown_connections 2024-03-18T20:34:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.442+0000 7fafc2c23700 1 --2- 172.21.15.46:0/1852007601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafbc074d70 0x7fafbc0731c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.442+0000 7fafc2c23700 1 --2- 172.21.15.46:0/1852007601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafbc108b60 0x7fafbc10aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.442+0000 7fafc2c23700 1 --2- 172.21.15.46:0/1852007601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafbc073790 0x7fafbc073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.442+0000 7fafc2c23700 1 -- 172.21.15.46:0/1852007601 >> 172.21.15.46:0/1852007601 conn(0x7fafbc0fc1e0 msgr2=0x7fafbc0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:47.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.442+0000 7fafc2c23700 1 -- 172.21.15.46:0/1852007601 shutdown_connections 2024-03-18T20:34:47.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.442+0000 7fafc2c23700 1 -- 172.21.15.46:0/1852007601 wait complete. 2024-03-18T20:34:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.443+0000 7fafc2c23700 1 Processor -- start 2024-03-18T20:34:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.443+0000 7fafc2c23700 1 -- start start 2024-03-18T20:34:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.443+0000 7fafc2c23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafbc073790 0x7fafbc198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.443+0000 7fafc2c23700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafbc074d70 0x7fafbc199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.443+0000 7fafc2c23700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafbc108b60 0x7fafbc1a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.443+0000 7fafc2c23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc1a0950 con 0x7fafbc073790 2024-03-18T20:34:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.443+0000 7fafc2c23700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc1a0a90 con 0x7fafbc108b60 2024-03-18T20:34:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.443+0000 7fafc2c23700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc1a0c20 con 0x7fafbc074d70 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.443+0000 7fafc09bf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafbc073790 0x7fafbc198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.443+0000 7fafc09bf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafbc073790 0x7fafbc198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42834/0 (socket says 172.21.15.46:42834) 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.443+0000 7fafc09bf700 1 -- 172.21.15.46:0/1181212430 learned_addr learned my addr 172.21.15.46:0/1181212430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafbbfff700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafbc074d70 0x7fafbc199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafc11c0700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafbc108b60 0x7fafbc1a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafc09bf700 1 -- 172.21.15.46:0/1181212430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafbc074d70 msgr2=0x7fafbc199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafc09bf700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafbc074d70 0x7fafbc199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafc09bf700 1 -- 172.21.15.46:0/1181212430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafbc108b60 msgr2=0x7fafbc1a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafc09bf700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafbc108b60 0x7fafbc1a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafc09bf700 1 -- 172.21.15.46:0/1181212430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb00095f0 con 0x7fafbc073790 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafc09bf700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafbc073790 0x7fafbc198e20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fafb00095c0 tx=0x7fafb000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafb9ffb700 1 -- 172.21.15.46:0/1181212430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb001d070 con 0x7fafbc073790 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafb9ffb700 1 -- 172.21.15.46:0/1181212430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafb0022470 con 0x7fafbc073790 2024-03-18T20:34:47.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafc2c23700 1 -- 172.21.15.46:0/1181212430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafbc1a0ea0 con 0x7fafbc073790 2024-03-18T20:34:47.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafb9ffb700 1 -- 172.21.15.46:0/1181212430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0017650 con 0x7fafbc073790 2024-03-18T20:34:47.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.444+0000 7fafc2c23700 1 -- 172.21.15.46:0/1181212430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafbc1a1210 con 0x7fafbc073790 2024-03-18T20:34:47.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.445+0000 7fafc2c23700 1 -- 172.21.15.46:0/1181212430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafbc107970 con 0x7fafbc073790 2024-03-18T20:34:47.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.447+0000 7fafb9ffb700 1 -- 172.21.15.46:0/1181212430 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fafb0022890 con 0x7fafbc073790 2024-03-18T20:34:47.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.447+0000 7fafb9ffb700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fafac0a11b0 0x7fafac0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:47.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.447+0000 7fafbbfff700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fafac0a11b0 0x7fafac0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:47.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.447+0000 7fafb9ffb700 1 -- 172.21.15.46:0/1181212430 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fafb00ce080 con 0x7fafbc073790 2024-03-18T20:34:47.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.448+0000 7fafbbfff700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fafac0a11b0 0x7fafac0a3670 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fafa8005950 tx=0x7fafa800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:47.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.451+0000 7fafb9ffb700 1 -- 172.21.15.46:0/1181212430 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fafb00978f0 con 0x7fafbc073790 2024-03-18T20:34:47.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.662+0000 7fafc2c23700 1 -- 172.21.15.46:0/1181212430 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fafbc0611f0 con 0x7fafac0a11b0 2024-03-18T20:34:47.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.672+0000 7fafb9ffb700 1 -- 172.21.15.46:0/1181212430 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+328109 (secure 0 0 0) 0x7fafbc0611f0 con 0x7fafac0a11b0 2024-03-18T20:34:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.677+0000 7fafc2c23700 1 -- 172.21.15.46:0/1181212430 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fafac0a11b0 msgr2=0x7fafac0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.677+0000 7fafc2c23700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fafac0a11b0 0x7fafac0a3670 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fafa8005950 tx=0x7fafa800b410 comp rx=0 tx=0).stop 2024-03-18T20:34:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.677+0000 7fafc2c23700 1 -- 172.21.15.46:0/1181212430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafbc073790 msgr2=0x7fafbc198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.677+0000 7fafc2c23700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafbc073790 0x7fafbc198e20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fafb00095c0 tx=0x7fafb000f690 comp rx=0 tx=0).stop 2024-03-18T20:34:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.677+0000 7fafc2c23700 1 -- 172.21.15.46:0/1181212430 shutdown_connections 2024-03-18T20:34:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.677+0000 7fafc2c23700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fafac0a11b0 0x7fafac0a3670 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.677+0000 7fafc2c23700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafbc073790 0x7fafbc198e20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.677+0000 7fafc2c23700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafbc108b60 0x7fafbc1a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.677+0000 7fafc2c23700 1 --2- 172.21.15.46:0/1181212430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fafbc074d70 0x7fafbc199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.677+0000 7fafc2c23700 1 -- 172.21.15.46:0/1181212430 >> 172.21.15.46:0/1181212430 conn(0x7fafbc0fc1e0 msgr2=0x7fafbc104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.677+0000 7fafc2c23700 1 -- 172.21.15.46:0/1181212430 shutdown_connections 2024-03-18T20:34:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:47.677+0000 7fafc2c23700 1 -- 172.21.15.46:0/1181212430 wait complete. 2024-03-18T20:34:47.684 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-18T20:34:47.694 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-03-18T20:34:47.019338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596076,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":138,"num_read_kb":121,"num_write":311,"num_write_kb":2984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":254,"ondisk_log_size":254,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":283,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336012,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605812,"statfs":{"total":1151940427776,"available":1151596351488,"internally_reserved":0,"allocated":12079104,"data_stored":6640115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5708,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":130,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.231847"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.902054+0000","last_change":"2024-03-18T20:34:38.817028+0000","last_active":"2024-03-18T20:34:39.902054+0000","last_peered":"2024-03-18T20:34:39.902054+0000","last_clean":"2024-03-18T20:34:39.902054+0000","last_became_active":"2024-03-18T20:34:38.816711+0000","last_became_peered":"2024-03-18T20:34:38.816711+0000","last_unstale":"2024-03-18T20:34:39.902054+0000","last_undegraded":"2024-03-18T20:34:39.902054+0000","last_fullsized":"2024-03-18T20:34:39.902054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:32:16.827141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901345+0000","last_change":"2024-03-18T20:34:38.807350+0000","last_active":"2024-03-18T20:34:39.901345+0000","last_peered":"2024-03-18T20:34:39.901345+0000","last_clean":"2024-03-18T20:34:39.901345+0000","last_became_active":"2024-03-18T20:34:38.807008+0000","last_became_peered":"2024-03-18T20:34:38.807008+0000","last_unstale":"2024-03-18T20:34:39.901345+0000","last_undegraded":"2024-03-18T20:34:39.901345+0000","last_fullsized":"2024-03-18T20:34:39.901345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:35.667384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901221+0000","last_change":"2024-03-18T20:34:38.817045+0000","last_active":"2024-03-18T20:34:39.901221+0000","last_peered":"2024-03-18T20:34:39.901221+0000","last_clean":"2024-03-18T20:34:39.901221+0000","last_became_active":"2024-03-18T20:34:38.816659+0000","last_became_peered":"2024-03-18T20:34:38.816659+0000","last_unstale":"2024-03-18T20:34:39.901221+0000","last_undegraded":"2024-03-18T20:34:39.901221+0000","last_fullsized":"2024-03-18T20:34:39.901221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:04:49.316127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998431+0000","last_change":"2024-03-18T20:34:38.804833+0000","last_active":"2024-03-18T20:34:39.998431+0000","last_peered":"2024-03-18T20:34:39.998431+0000","last_clean":"2024-03-18T20:34:39.998431+0000","last_became_active":"2024-03-18T20:34:38.804486+0000","last_became_peered":"2024-03-18T20:34:38.804486+0000","last_unstale":"2024-03-18T20:34:39.998431+0000","last_undegraded":"2024-03-18T20:34:39.998431+0000","last_fullsized":"2024-03-18T20:34:39.998431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:19:44.266207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996571+0000","last_change":"2024-03-18T20:34:38.797660+0000","last_active":"2024-03-18T20:34:39.996571+0000","last_peered":"2024-03-18T20:34:39.996571+0000","last_clean":"2024-03-18T20:34:39.996571+0000","last_became_active":"2024-03-18T20:34:38.797155+0000","last_became_peered":"2024-03-18T20:34:38.797155+0000","last_unstale":"2024-03-18T20:34:39.996571+0000","last_undegraded":"2024-03-18T20:34:39.996571+0000","last_fullsized":"2024-03-18T20:34:39.996571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:22:11.363453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997430+0000","last_change":"2024-03-18T20:34:38.822891+0000","last_active":"2024-03-18T20:34:39.997430+0000","last_peered":"2024-03-18T20:34:39.997430+0000","last_clean":"2024-03-18T20:34:39.997430+0000","last_became_active":"2024-03-18T20:34:38.822574+0000","last_became_peered":"2024-03-18T20:34:38.822574+0000","last_unstale":"2024-03-18T20:34:39.997430+0000","last_undegraded":"2024-03-18T20:34:39.997430+0000","last_fullsized":"2024-03-18T20:34:39.997430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:42:05.727053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997617+0000","last_change":"2024-03-18T20:34:38.805378+0000","last_active":"2024-03-18T20:34:39.997617+0000","last_peered":"2024-03-18T20:34:39.997617+0000","last_clean":"2024-03-18T20:34:39.997617+0000","last_became_active":"2024-03-18T20:34:38.805130+0000","last_became_peered":"2024-03-18T20:34:38.805130+0000","last_unstale":"2024-03-18T20:34:39.997617+0000","last_undegraded":"2024-03-18T20:34:39.997617+0000","last_fullsized":"2024-03-18T20:34:39.997617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:46:19.420715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198520+0000","last_change":"2024-03-18T20:34:38.801034+0000","last_active":"2024-03-18T20:34:39.198520+0000","last_peered":"2024-03-18T20:34:39.198520+0000","last_clean":"2024-03-18T20:34:39.198520+0000","last_became_active":"2024-03-18T20:34:38.800706+0000","last_became_peered":"2024-03-18T20:34:38.800706+0000","last_unstale":"2024-03-18T20:34:39.198520+0000","last_undegraded":"2024-03-18T20:34:39.198520+0000","last_fullsized":"2024-03-18T20:34:39.198520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:10:22.641105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.823531+0000","last_change":"2024-03-18T20:34:38.822841+0000","last_active":"2024-03-18T20:34:38.823531+0000","last_peered":"2024-03-18T20:34:38.823531+0000","last_clean":"2024-03-18T20:34:38.823531+0000","last_became_active":"2024-03-18T20:34:38.822537+0000","last_became_peered":"2024-03-18T20:34:38.822537+0000","last_unstale":"2024-03-18T20:34:38.823531+0000","last_undegraded":"2024-03-18T20:34:38.823531+0000","last_fullsized":"2024-03-18T20:34:38.823531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:38.741995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997436+0000","last_change":"2024-03-18T20:34:38.814728+0000","last_active":"2024-03-18T20:34:39.997436+0000","last_peered":"2024-03-18T20:34:39.997436+0000","last_clean":"2024-03-18T20:34:39.997436+0000","last_became_active":"2024-03-18T20:34:38.814200+0000","last_became_peered":"2024-03-18T20:34:38.814200+0000","last_unstale":"2024-03-18T20:34:39.997436+0000","last_undegraded":"2024-03-18T20:34:39.997436+0000","last_fullsized":"2024-03-18T20:34:39.997436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:24:54.064770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997025+0000","last_change":"2024-03-18T20:34:38.818054+0000","last_active":"2024-03-18T20:34:39.997025+0000","last_peered":"2024-03-18T20:34:39.997025+0000","last_clean":"2024-03-18T20:34:39.997025+0000","last_became_active":"2024-03-18T20:34:38.817752+0000","last_became_peered":"2024-03-18T20:34:38.817752+0000","last_unstale":"2024-03-18T20:34:39.997025+0000","last_undegraded":"2024-03-18T20:34:39.997025+0000","last_fullsized":"2024-03-18T20:34:39.997025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:01:13.728434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998381+0000","last_change":"2024-03-18T20:34:38.801356+0000","last_active":"2024-03-18T20:34:39.998381+0000","last_peered":"2024-03-18T20:34:39.998381+0000","last_clean":"2024-03-18T20:34:39.998381+0000","last_became_active":"2024-03-18T20:34:38.800978+0000","last_became_peered":"2024-03-18T20:34:38.800978+0000","last_unstale":"2024-03-18T20:34:39.998381+0000","last_undegraded":"2024-03-18T20:34:39.998381+0000","last_fullsized":"2024-03-18T20:34:39.998381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:07.634686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793686+0000","last_change":"2024-03-18T20:34:38.803006+0000","last_active":"2024-03-18T20:34:39.793686+0000","last_peered":"2024-03-18T20:34:39.793686+0000","last_clean":"2024-03-18T20:34:39.793686+0000","last_became_active":"2024-03-18T20:34:38.802454+0000","last_became_peered":"2024-03-18T20:34:38.802454+0000","last_unstale":"2024-03-18T20:34:39.793686+0000","last_undegraded":"2024-03-18T20:34:39.793686+0000","last_fullsized":"2024-03-18T20:34:39.793686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:11:02.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901028+0000","last_change":"2024-03-18T20:34:38.807397+0000","last_active":"2024-03-18T20:34:39.901028+0000","last_peered":"2024-03-18T20:34:39.901028+0000","last_clean":"2024-03-18T20:34:39.901028+0000","last_became_active":"2024-03-18T20:34:38.807072+0000","last_became_peered":"2024-03-18T20:34:38.807072+0000","last_unstale":"2024-03-18T20:34:39.901028+0000","last_undegraded":"2024-03-18T20:34:39.901028+0000","last_fullsized":"2024-03-18T20:34:39.901028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:55:53.832825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.900962+0000","last_change":"2024-03-18T20:34:38.812925+0000","last_active":"2024-03-18T20:34:39.900962+0000","last_peered":"2024-03-18T20:34:39.900962+0000","last_clean":"2024-03-18T20:34:39.900962+0000","last_became_active":"2024-03-18T20:34:38.812597+0000","last_became_peered":"2024-03-18T20:34:38.812597+0000","last_unstale":"2024-03-18T20:34:39.900962+0000","last_undegraded":"2024-03-18T20:34:39.900962+0000","last_fullsized":"2024-03-18T20:34:39.900962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:14:35.103852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.820572+0000","last_change":"2024-03-18T20:34:38.817892+0000","last_active":"2024-03-18T20:34:38.820572+0000","last_peered":"2024-03-18T20:34:38.820572+0000","last_clean":"2024-03-18T20:34:38.820572+0000","last_became_active":"2024-03-18T20:34:38.816261+0000","last_became_peered":"2024-03-18T20:34:38.816261+0000","last_unstale":"2024-03-18T20:34:38.820572+0000","last_undegraded":"2024-03-18T20:34:38.820572+0000","last_fullsized":"2024-03-18T20:34:38.820572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:32.481210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.902148+0000","last_change":"2024-03-18T20:34:38.816966+0000","last_active":"2024-03-18T20:34:39.902148+0000","last_peered":"2024-03-18T20:34:39.902148+0000","last_clean":"2024-03-18T20:34:39.902148+0000","last_became_active":"2024-03-18T20:34:38.816645+0000","last_became_peered":"2024-03-18T20:34:38.816645+0000","last_unstale":"2024-03-18T20:34:39.902148+0000","last_undegraded":"2024-03-18T20:34:39.902148+0000","last_fullsized":"2024-03-18T20:34:39.902148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:39.904353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997076+0000","last_change":"2024-03-18T20:34:38.817005+0000","last_active":"2024-03-18T20:34:39.997076+0000","last_peered":"2024-03-18T20:34:39.997076+0000","last_clean":"2024-03-18T20:34:39.997076+0000","last_became_active":"2024-03-18T20:34:38.816710+0000","last_became_peered":"2024-03-18T20:34:38.816710+0000","last_unstale":"2024-03-18T20:34:39.997076+0000","last_undegraded":"2024-03-18T20:34:39.997076+0000","last_fullsized":"2024-03-18T20:34:39.997076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:30:43.483977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997574+0000","last_change":"2024-03-18T20:34:38.810957+0000","last_active":"2024-03-18T20:34:39.997574+0000","last_peered":"2024-03-18T20:34:39.997574+0000","last_clean":"2024-03-18T20:34:39.997574+0000","last_became_active":"2024-03-18T20:34:38.810615+0000","last_became_peered":"2024-03-18T20:34:38.810615+0000","last_unstale":"2024-03-18T20:34:39.997574+0000","last_undegraded":"2024-03-18T20:34:39.997574+0000","last_fullsized":"2024-03-18T20:34:39.997574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:26:36.544034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.794465+0000","last_change":"2024-03-18T20:34:38.337863+0000","last_active":"2024-03-18T20:34:44.794465+0000","last_peered":"2024-03-18T20:34:44.794465+0000","last_clean":"2024-03-18T20:34:44.794465+0000","last_became_active":"2024-03-18T20:34:38.337548+0000","last_became_peered":"2024-03-18T20:34:38.337548+0000","last_unstale":"2024-03-18T20:34:44.794465+0000","last_undegraded":"2024-03-18T20:34:44.794465+0000","last_fullsized":"2024-03-18T20:34:44.794465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:41:40.237882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198626+0000","last_change":"2024-03-18T20:34:38.814321+0000","last_active":"2024-03-18T20:34:39.198626+0000","last_peered":"2024-03-18T20:34:39.198626+0000","last_clean":"2024-03-18T20:34:39.198626+0000","last_became_active":"2024-03-18T20:34:38.813939+0000","last_became_peered":"2024-03-18T20:34:38.813939+0000","last_unstale":"2024-03-18T20:34:39.198626+0000","last_undegraded":"2024-03-18T20:34:39.198626+0000","last_fullsized":"2024-03-18T20:34:39.198626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:05:42.441099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996941+0000","last_change":"2024-03-18T20:34:38.347790+0000","last_active":"2024-03-18T20:34:39.996941+0000","last_peered":"2024-03-18T20:34:39.996941+0000","last_clean":"2024-03-18T20:34:39.996941+0000","last_became_active":"2024-03-18T20:34:38.347317+0000","last_became_peered":"2024-03-18T20:34:38.347317+0000","last_unstale":"2024-03-18T20:34:39.996941+0000","last_undegraded":"2024-03-18T20:34:39.996941+0000","last_fullsized":"2024-03-18T20:34:39.996941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.096480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901274+0000","last_change":"2024-03-18T20:34:38.817116+0000","last_active":"2024-03-18T20:34:39.901274+0000","last_peered":"2024-03-18T20:34:39.901274+0000","last_clean":"2024-03-18T20:34:39.901274+0000","last_became_active":"2024-03-18T20:34:38.816761+0000","last_became_peered":"2024-03-18T20:34:38.816761+0000","last_unstale":"2024-03-18T20:34:39.901274+0000","last_undegraded":"2024-03-18T20:34:39.901274+0000","last_fullsized":"2024-03-18T20:34:39.901274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:44.353533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.820176+0000","last_change":"2024-03-18T20:34:38.339236+0000","last_active":"2024-03-18T20:34:42.820176+0000","last_peered":"2024-03-18T20:34:42.820176+0000","last_clean":"2024-03-18T20:34:42.820176+0000","last_became_active":"2024-03-18T20:34:38.338879+0000","last_became_peered":"2024-03-18T20:34:38.338879+0000","last_unstale":"2024-03-18T20:34:42.820176+0000","last_undegraded":"2024-03-18T20:34:42.820176+0000","last_fullsized":"2024-03-18T20:34:42.820176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:33.340151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.820775+0000","last_change":"2024-03-18T20:34:38.817861+0000","last_active":"2024-03-18T20:34:38.820775+0000","last_peered":"2024-03-18T20:34:38.820775+0000","last_clean":"2024-03-18T20:34:38.820775+0000","last_became_active":"2024-03-18T20:34:38.815976+0000","last_became_peered":"2024-03-18T20:34:38.815976+0000","last_unstale":"2024-03-18T20:34:38.820775+0000","last_undegraded":"2024-03-18T20:34:38.820775+0000","last_fullsized":"2024-03-18T20:34:38.820775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:54.070067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.820328+0000","last_change":"2024-03-18T20:34:38.345758+0000","last_active":"2024-03-18T20:34:38.820328+0000","last_peered":"2024-03-18T20:34:38.820328+0000","last_clean":"2024-03-18T20:34:38.820328+0000","last_became_active":"2024-03-18T20:34:38.345417+0000","last_became_peered":"2024-03-18T20:34:38.345417+0000","last_unstale":"2024-03-18T20:34:38.820328+0000","last_undegraded":"2024-03-18T20:34:38.820328+0000","last_fullsized":"2024-03-18T20:34:38.820328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:25.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759430+0000","last_change":"2024-03-18T20:34:38.818745+0000","last_active":"2024-03-18T20:34:39.759430+0000","last_peered":"2024-03-18T20:34:39.759430+0000","last_clean":"2024-03-18T20:34:39.759430+0000","last_became_active":"2024-03-18T20:34:38.818415+0000","last_became_peered":"2024-03-18T20:34:38.818415+0000","last_unstale":"2024-03-18T20:34:39.759430+0000","last_undegraded":"2024-03-18T20:34:39.759430+0000","last_fullsized":"2024-03-18T20:34:39.759430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:45:49.454498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814903+0000","last_change":"2024-03-18T20:34:38.345804+0000","last_active":"2024-03-18T20:34:41.814903+0000","last_peered":"2024-03-18T20:34:41.814903+0000","last_clean":"2024-03-18T20:34:41.814903+0000","last_became_active":"2024-03-18T20:34:38.345409+0000","last_became_peered":"2024-03-18T20:34:38.345409+0000","last_unstale":"2024-03-18T20:34:41.814903+0000","last_undegraded":"2024-03-18T20:34:41.814903+0000","last_fullsized":"2024-03-18T20:34:41.814903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:00.085412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996703+0000","last_change":"2024-03-18T20:34:38.799821+0000","last_active":"2024-03-18T20:34:39.996703+0000","last_peered":"2024-03-18T20:34:39.996703+0000","last_clean":"2024-03-18T20:34:39.996703+0000","last_became_active":"2024-03-18T20:34:38.799457+0000","last_became_peered":"2024-03-18T20:34:38.799457+0000","last_unstale":"2024-03-18T20:34:39.996703+0000","last_undegraded":"2024-03-18T20:34:39.996703+0000","last_fullsized":"2024-03-18T20:34:39.996703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:39:35.912892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.805945+0000","last_change":"2024-03-18T20:34:38.341999+0000","last_active":"2024-03-18T20:34:38.805945+0000","last_peered":"2024-03-18T20:34:38.805945+0000","last_clean":"2024-03-18T20:34:38.805945+0000","last_became_active":"2024-03-18T20:34:38.341431+0000","last_became_peered":"2024-03-18T20:34:38.341431+0000","last_unstale":"2024-03-18T20:34:38.805945+0000","last_undegraded":"2024-03-18T20:34:38.805945+0000","last_fullsized":"2024-03-18T20:34:38.805945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:07:26.711113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759388+0000","last_change":"2024-03-18T20:34:38.804003+0000","last_active":"2024-03-18T20:34:39.759388+0000","last_peered":"2024-03-18T20:34:39.759388+0000","last_clean":"2024-03-18T20:34:39.759388+0000","last_became_active":"2024-03-18T20:34:38.803667+0000","last_became_peered":"2024-03-18T20:34:38.803667+0000","last_unstale":"2024-03-18T20:34:39.759388+0000","last_undegraded":"2024-03-18T20:34:39.759388+0000","last_fullsized":"2024-03-18T20:34:39.759388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:19:58.230790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.795285+0000","last_change":"2024-03-18T20:34:38.346493+0000","last_active":"2024-03-18T20:34:38.795285+0000","last_peered":"2024-03-18T20:34:38.795285+0000","last_clean":"2024-03-18T20:34:38.795285+0000","last_became_active":"2024-03-18T20:34:38.345925+0000","last_became_peered":"2024-03-18T20:34:38.345925+0000","last_unstale":"2024-03-18T20:34:38.795285+0000","last_undegraded":"2024-03-18T20:34:38.795285+0000","last_fullsized":"2024-03-18T20:34:38.795285+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:52:47.773860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198488+0000","last_change":"2024-03-18T20:34:38.807284+0000","last_active":"2024-03-18T20:34:39.198488+0000","last_peered":"2024-03-18T20:34:39.198488+0000","last_clean":"2024-03-18T20:34:39.198488+0000","last_became_active":"2024-03-18T20:34:38.806844+0000","last_became_peered":"2024-03-18T20:34:38.806844+0000","last_unstale":"2024-03-18T20:34:39.198488+0000","last_undegraded":"2024-03-18T20:34:39.198488+0000","last_fullsized":"2024-03-18T20:34:39.198488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:44:35.469953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814733+0000","last_change":"2024-03-18T20:34:38.343090+0000","last_active":"2024-03-18T20:34:41.814733+0000","last_peered":"2024-03-18T20:34:41.814733+0000","last_clean":"2024-03-18T20:34:41.814733+0000","last_became_active":"2024-03-18T20:34:38.342712+0000","last_became_peered":"2024-03-18T20:34:38.342712+0000","last_unstale":"2024-03-18T20:34:41.814733+0000","last_undegraded":"2024-03-18T20:34:41.814733+0000","last_fullsized":"2024-03-18T20:34:41.814733+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:14:05.909604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996769+0000","last_change":"2024-03-18T20:34:38.800616+0000","last_active":"2024-03-18T20:34:39.996769+0000","last_peered":"2024-03-18T20:34:39.996769+0000","last_clean":"2024-03-18T20:34:39.996769+0000","last_became_active":"2024-03-18T20:34:38.800340+0000","last_became_peered":"2024-03-18T20:34:38.800340+0000","last_unstale":"2024-03-18T20:34:39.996769+0000","last_undegraded":"2024-03-18T20:34:39.996769+0000","last_fullsized":"2024-03-18T20:34:39.996769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:18.263179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814549+0000","last_change":"2024-03-18T20:34:38.345634+0000","last_active":"2024-03-18T20:34:41.814549+0000","last_peered":"2024-03-18T20:34:41.814549+0000","last_clean":"2024-03-18T20:34:41.814549+0000","last_became_active":"2024-03-18T20:34:38.345275+0000","last_became_peered":"2024-03-18T20:34:38.345275+0000","last_unstale":"2024-03-18T20:34:41.814549+0000","last_undegraded":"2024-03-18T20:34:41.814549+0000","last_fullsized":"2024-03-18T20:34:41.814549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:29.939529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.823111+0000","last_change":"2024-03-18T20:34:38.820286+0000","last_active":"2024-03-18T20:34:38.823111+0000","last_peered":"2024-03-18T20:34:38.823111+0000","last_clean":"2024-03-18T20:34:38.823111+0000","last_became_active":"2024-03-18T20:34:38.819939+0000","last_became_peered":"2024-03-18T20:34:38.819939+0000","last_unstale":"2024-03-18T20:34:38.823111+0000","last_undegraded":"2024-03-18T20:34:38.823111+0000","last_fullsized":"2024-03-18T20:34:38.823111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:19.229339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.794842+0000","last_change":"2024-03-18T20:34:38.341699+0000","last_active":"2024-03-18T20:34:44.794842+0000","last_peered":"2024-03-18T20:34:44.794842+0000","last_clean":"2024-03-18T20:34:44.794842+0000","last_became_active":"2024-03-18T20:34:38.341390+0000","last_became_peered":"2024-03-18T20:34:38.341390+0000","last_unstale":"2024-03-18T20:34:44.794842+0000","last_undegraded":"2024-03-18T20:34:44.794842+0000","last_fullsized":"2024-03-18T20:34:44.794842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:16:02.239414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901314+0000","last_change":"2024-03-18T20:34:38.805316+0000","last_active":"2024-03-18T20:34:39.901314+0000","last_peered":"2024-03-18T20:34:39.901314+0000","last_clean":"2024-03-18T20:34:39.901314+0000","last_became_active":"2024-03-18T20:34:38.804967+0000","last_became_peered":"2024-03-18T20:34:38.804967+0000","last_unstale":"2024-03-18T20:34:39.901314+0000","last_undegraded":"2024-03-18T20:34:39.901314+0000","last_fullsized":"2024-03-18T20:34:39.901314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:54.591401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814531+0000","last_change":"2024-03-18T20:34:38.346821+0000","last_active":"2024-03-18T20:34:41.814531+0000","last_peered":"2024-03-18T20:34:41.814531+0000","last_clean":"2024-03-18T20:34:41.814531+0000","last_became_active":"2024-03-18T20:34:38.346314+0000","last_became_peered":"2024-03-18T20:34:38.346314+0000","last_unstale":"2024-03-18T20:34:41.814531+0000","last_undegraded":"2024-03-18T20:34:41.814531+0000","last_fullsized":"2024-03-18T20:34:41.814531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:12:04.552971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.823287+0000","last_change":"2024-03-18T20:34:38.820479+0000","last_active":"2024-03-18T20:34:38.823287+0000","last_peered":"2024-03-18T20:34:38.823287+0000","last_clean":"2024-03-18T20:34:38.823287+0000","last_became_active":"2024-03-18T20:34:38.820186+0000","last_became_peered":"2024-03-18T20:34:38.820186+0000","last_unstale":"2024-03-18T20:34:38.823287+0000","last_undegraded":"2024-03-18T20:34:38.823287+0000","last_fullsized":"2024-03-18T20:34:38.823287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:15.953764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'8","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814692+0000","last_change":"2024-03-18T20:34:38.344332+0000","last_active":"2024-03-18T20:34:41.814692+0000","last_peered":"2024-03-18T20:34:41.814692+0000","last_clean":"2024-03-18T20:34:41.814692+0000","last_became_active":"2024-03-18T20:34:38.343827+0000","last_became_peered":"2024-03-18T20:34:38.343827+0000","last_unstale":"2024-03-18T20:34:41.814692+0000","last_undegraded":"2024-03-18T20:34:41.814692+0000","last_fullsized":"2024-03-18T20:34:41.814692+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:13:21.725004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996827+0000","last_change":"2024-03-18T20:34:38.814794+0000","last_active":"2024-03-18T20:34:39.996827+0000","last_peered":"2024-03-18T20:34:39.996827+0000","last_clean":"2024-03-18T20:34:39.996827+0000","last_became_active":"2024-03-18T20:34:38.814434+0000","last_became_peered":"2024-03-18T20:34:38.814434+0000","last_unstale":"2024-03-18T20:34:39.996827+0000","last_undegraded":"2024-03-18T20:34:39.996827+0000","last_fullsized":"2024-03-18T20:34:39.996827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:43:19.083212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814504+0000","last_change":"2024-03-18T20:34:38.343894+0000","last_active":"2024-03-18T20:34:41.814504+0000","last_peered":"2024-03-18T20:34:41.814504+0000","last_clean":"2024-03-18T20:34:41.814504+0000","last_became_active":"2024-03-18T20:34:38.343493+0000","last_became_peered":"2024-03-18T20:34:38.343493+0000","last_unstale":"2024-03-18T20:34:41.814504+0000","last_undegraded":"2024-03-18T20:34:41.814504+0000","last_fullsized":"2024-03-18T20:34:41.814504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:13:32.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793766+0000","last_change":"2024-03-18T20:34:38.809968+0000","last_active":"2024-03-18T20:34:39.793766+0000","last_peered":"2024-03-18T20:34:39.793766+0000","last_clean":"2024-03-18T20:34:39.793766+0000","last_became_active":"2024-03-18T20:34:38.809641+0000","last_became_peered":"2024-03-18T20:34:38.809641+0000","last_unstale":"2024-03-18T20:34:39.793766+0000","last_undegraded":"2024-03-18T20:34:39.793766+0000","last_fullsized":"2024-03-18T20:34:39.793766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:50:52.925683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.814608+0000","last_change":"2024-03-18T20:34:38.345621+0000","last_active":"2024-03-18T20:34:41.814608+0000","last_peered":"2024-03-18T20:34:41.814608+0000","last_clean":"2024-03-18T20:34:41.814608+0000","last_became_active":"2024-03-18T20:34:38.345255+0000","last_became_peered":"2024-03-18T20:34:38.345255+0000","last_unstale":"2024-03-18T20:34:41.814608+0000","last_undegraded":"2024-03-18T20:34:41.814608+0000","last_fullsized":"2024-03-18T20:34:41.814608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:17:03.737973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997875+0000","last_change":"2024-03-18T20:34:38.804390+0000","last_active":"2024-03-18T20:34:39.997875+0000","last_peered":"2024-03-18T20:34:39.997875+0000","last_clean":"2024-03-18T20:34:39.997875+0000","last_became_active":"2024-03-18T20:34:38.804033+0000","last_became_peered":"2024-03-18T20:34:38.804033+0000","last_unstale":"2024-03-18T20:34:39.997875+0000","last_undegraded":"2024-03-18T20:34:39.997875+0000","last_fullsized":"2024-03-18T20:34:39.997875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:09:45.407921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.818248+0000","last_change":"2024-03-18T20:34:38.340239+0000","last_active":"2024-03-18T20:34:42.818248+0000","last_peered":"2024-03-18T20:34:42.818248+0000","last_clean":"2024-03-18T20:34:42.818248+0000","last_became_active":"2024-03-18T20:34:38.339880+0000","last_became_peered":"2024-03-18T20:34:38.339880+0000","last_unstale":"2024-03-18T20:34:42.818248+0000","last_undegraded":"2024-03-18T20:34:42.818248+0000","last_fullsized":"2024-03-18T20:34:42.818248+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:55:46.864885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998003+0000","last_change":"2024-03-18T20:34:38.819223+0000","last_active":"2024-03-18T20:34:39.998003+0000","last_peered":"2024-03-18T20:34:39.998003+0000","last_clean":"2024-03-18T20:34:39.998003+0000","last_became_active":"2024-03-18T20:34:38.818879+0000","last_became_peered":"2024-03-18T20:34:38.818879+0000","last_unstale":"2024-03-18T20:34:39.998003+0000","last_undegraded":"2024-03-18T20:34:39.998003+0000","last_fullsized":"2024-03-18T20:34:39.998003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:45.645912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.821003+0000","last_change":"2024-03-18T20:34:38.336817+0000","last_active":"2024-03-18T20:34:42.821003+0000","last_peered":"2024-03-18T20:34:42.821003+0000","last_clean":"2024-03-18T20:34:42.821003+0000","last_became_active":"2024-03-18T20:34:38.336453+0000","last_became_peered":"2024-03-18T20:34:38.336453+0000","last_unstale":"2024-03-18T20:34:42.821003+0000","last_undegraded":"2024-03-18T20:34:42.821003+0000","last_fullsized":"2024-03-18T20:34:42.821003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:10.253435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759322+0000","last_change":"2024-03-18T20:34:38.818048+0000","last_active":"2024-03-18T20:34:39.759322+0000","last_peered":"2024-03-18T20:34:39.759322+0000","last_clean":"2024-03-18T20:34:39.759322+0000","last_became_active":"2024-03-18T20:34:38.817682+0000","last_became_peered":"2024-03-18T20:34:38.817682+0000","last_unstale":"2024-03-18T20:34:39.759322+0000","last_undegraded":"2024-03-18T20:34:39.759322+0000","last_fullsized":"2024-03-18T20:34:39.759322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:25:55.291815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.820282+0000","last_change":"2024-03-18T20:34:38.345773+0000","last_active":"2024-03-18T20:34:38.820282+0000","last_peered":"2024-03-18T20:34:38.820282+0000","last_clean":"2024-03-18T20:34:38.820282+0000","last_became_active":"2024-03-18T20:34:38.345432+0000","last_became_peered":"2024-03-18T20:34:38.345432+0000","last_unstale":"2024-03-18T20:34:38.820282+0000","last_undegraded":"2024-03-18T20:34:38.820282+0000","last_fullsized":"2024-03-18T20:34:38.820282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:56:57.657424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901903+0000","last_change":"2024-03-18T20:34:38.806138+0000","last_active":"2024-03-18T20:34:39.901903+0000","last_peered":"2024-03-18T20:34:39.901903+0000","last_clean":"2024-03-18T20:34:39.901903+0000","last_became_active":"2024-03-18T20:34:38.805821+0000","last_became_peered":"2024-03-18T20:34:38.805821+0000","last_unstale":"2024-03-18T20:34:39.901903+0000","last_undegraded":"2024-03-18T20:34:39.901903+0000","last_fullsized":"2024-03-18T20:34:39.901903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:25:57.887813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.173015+0000","last_change":"2024-03-18T20:34:38.338751+0000","last_active":"2024-03-18T20:34:41.173015+0000","last_peered":"2024-03-18T20:34:41.173015+0000","last_clean":"2024-03-18T20:34:41.173015+0000","last_became_active":"2024-03-18T20:34:38.337495+0000","last_became_peered":"2024-03-18T20:34:38.337495+0000","last_unstale":"2024-03-18T20:34:41.173015+0000","last_undegraded":"2024-03-18T20:34:41.173015+0000","last_fullsized":"2024-03-18T20:34:41.173015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:38:04.402380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997145+0000","last_change":"2024-03-18T20:34:38.803937+0000","last_active":"2024-03-18T20:34:39.997145+0000","last_peered":"2024-03-18T20:34:39.997145+0000","last_clean":"2024-03-18T20:34:39.997145+0000","last_became_active":"2024-03-18T20:34:38.803520+0000","last_became_peered":"2024-03-18T20:34:38.803520+0000","last_unstale":"2024-03-18T20:34:39.997145+0000","last_undegraded":"2024-03-18T20:34:39.997145+0000","last_fullsized":"2024-03-18T20:34:39.997145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:20:37.537265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.794027+0000","last_change":"2024-03-18T20:34:38.337481+0000","last_active":"2024-03-18T20:34:44.794027+0000","last_peered":"2024-03-18T20:34:44.794027+0000","last_clean":"2024-03-18T20:34:44.794027+0000","last_became_active":"2024-03-18T20:34:38.337152+0000","last_became_peered":"2024-03-18T20:34:38.337152+0000","last_unstale":"2024-03-18T20:34:44.794027+0000","last_undegraded":"2024-03-18T20:34:44.794027+0000","last_fullsized":"2024-03-18T20:34:44.794027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:07:59.593978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997194+0000","last_change":"2024-03-18T20:34:38.814240+0000","last_active":"2024-03-18T20:34:39.997194+0000","last_peered":"2024-03-18T20:34:39.997194+0000","last_clean":"2024-03-18T20:34:39.997194+0000","last_became_active":"2024-03-18T20:34:38.813177+0000","last_became_peered":"2024-03-18T20:34:38.813177+0000","last_unstale":"2024-03-18T20:34:39.997194+0000","last_undegraded":"2024-03-18T20:34:39.997194+0000","last_fullsized":"2024-03-18T20:34:39.997194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:22:30.076880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759286+0000","last_change":"2024-03-18T20:34:38.806133+0000","last_active":"2024-03-18T20:34:39.759286+0000","last_peered":"2024-03-18T20:34:39.759286+0000","last_clean":"2024-03-18T20:34:39.759286+0000","last_became_active":"2024-03-18T20:34:38.805257+0000","last_became_peered":"2024-03-18T20:34:38.805257+0000","last_unstale":"2024-03-18T20:34:39.759286+0000","last_undegraded":"2024-03-18T20:34:39.759286+0000","last_fullsized":"2024-03-18T20:34:39.759286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:05:15.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793220+0000","last_change":"2024-03-18T20:34:38.346271+0000","last_active":"2024-03-18T20:34:39.793220+0000","last_peered":"2024-03-18T20:34:39.793220+0000","last_clean":"2024-03-18T20:34:39.793220+0000","last_became_active":"2024-03-18T20:34:38.345054+0000","last_became_peered":"2024-03-18T20:34:38.345054+0000","last_unstale":"2024-03-18T20:34:39.793220+0000","last_undegraded":"2024-03-18T20:34:39.793220+0000","last_fullsized":"2024-03-18T20:34:39.793220+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:42:52.510294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198582+0000","last_change":"2024-03-18T20:34:38.816834+0000","last_active":"2024-03-18T20:34:39.198582+0000","last_peered":"2024-03-18T20:34:39.198582+0000","last_clean":"2024-03-18T20:34:39.198582+0000","last_became_active":"2024-03-18T20:34:38.816516+0000","last_became_peered":"2024-03-18T20:34:38.816516+0000","last_unstale":"2024-03-18T20:34:39.198582+0000","last_undegraded":"2024-03-18T20:34:39.198582+0000","last_fullsized":"2024-03-18T20:34:39.198582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:33:54.529206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.813927+0000","last_change":"2024-03-18T20:34:38.339407+0000","last_active":"2024-03-18T20:34:41.813927+0000","last_peered":"2024-03-18T20:34:41.813927+0000","last_clean":"2024-03-18T20:34:41.813927+0000","last_became_active":"2024-03-18T20:34:38.339086+0000","last_became_peered":"2024-03-18T20:34:38.339086+0000","last_unstale":"2024-03-18T20:34:41.813927+0000","last_undegraded":"2024-03-18T20:34:41.813927+0000","last_fullsized":"2024-03-18T20:34:41.813927+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:18.479630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":330,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.801661+0000","last_change":"2024-03-18T20:33:50.777651+0000","last_active":"2024-03-18T20:34:38.801661+0000","last_peered":"2024-03-18T20:34:38.801661+0000","last_clean":"2024-03-18T20:34:38.801661+0000","last_became_active":"2024-03-18T20:33:50.777281+0000","last_became_peered":"2024-03-18T20:33:50.777281+0000","last_unstale":"2024-03-18T20:34:38.801661+0000","last_undegraded":"2024-03-18T20:34:38.801661+0000","last_fullsized":"2024-03-18T20:34:38.801661+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_clean_scrub_stamp":"2024-03-18T20:31:27.488744+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:28.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793222+0000","last_change":"2024-03-18T20:34:38.812714+0000","last_active":"2024-03-18T20:34:39.793222+0000","last_peered":"2024-03-18T20:34:39.793222+0000","last_clean":"2024-03-18T20:34:39.793222+0000","last_became_active":"2024-03-18T20:34:38.812464+0000","last_became_peered":"2024-03-18T20:34:38.812464+0000","last_unstale":"2024-03-18T20:34:39.793222+0000","last_undegraded":"2024-03-18T20:34:39.793222+0000","last_fullsized":"2024-03-18T20:34:39.793222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:03:13.853740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.820233+0000","last_change":"2024-03-18T20:34:38.343173+0000","last_active":"2024-03-18T20:34:42.820233+0000","last_peered":"2024-03-18T20:34:42.820233+0000","last_clean":"2024-03-18T20:34:42.820233+0000","last_became_active":"2024-03-18T20:34:38.342774+0000","last_became_peered":"2024-03-18T20:34:38.342774+0000","last_unstale":"2024-03-18T20:34:42.820233+0000","last_undegraded":"2024-03-18T20:34:42.820233+0000","last_fullsized":"2024-03-18T20:34:42.820233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:52.283033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996953+0000","last_change":"2024-03-18T20:34:38.344539+0000","last_active":"2024-03-18T20:34:39.996953+0000","last_peered":"2024-03-18T20:34:39.996953+0000","last_clean":"2024-03-18T20:34:39.996953+0000","last_became_active":"2024-03-18T20:34:38.344188+0000","last_became_peered":"2024-03-18T20:34:38.344188+0000","last_unstale":"2024-03-18T20:34:39.996953+0000","last_undegraded":"2024-03-18T20:34:39.996953+0000","last_fullsized":"2024-03-18T20:34:39.996953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:38.641928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793487+0000","last_change":"2024-03-18T20:34:38.806602+0000","last_active":"2024-03-18T20:34:39.793487+0000","last_peered":"2024-03-18T20:34:39.793487+0000","last_clean":"2024-03-18T20:34:39.793487+0000","last_became_active":"2024-03-18T20:34:38.806241+0000","last_became_peered":"2024-03-18T20:34:38.806241+0000","last_unstale":"2024-03-18T20:34:39.793487+0000","last_undegraded":"2024-03-18T20:34:39.793487+0000","last_fullsized":"2024-03-18T20:34:39.793487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:04.463311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'7","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:43.831715+0000","last_change":"2024-03-18T20:34:38.339695+0000","last_active":"2024-03-18T20:34:43.831715+0000","last_peered":"2024-03-18T20:34:43.831715+0000","last_clean":"2024-03-18T20:34:43.831715+0000","last_became_active":"2024-03-18T20:34:38.339331+0000","last_became_peered":"2024-03-18T20:34:38.339331+0000","last_unstale":"2024-03-18T20:34:43.831715+0000","last_undegraded":"2024-03-18T20:34:43.831715+0000","last_fullsized":"2024-03-18T20:34:43.831715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:57:06.434289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198695+0000","last_change":"2024-03-18T20:34:38.809754+0000","last_active":"2024-03-18T20:34:39.198695+0000","last_peered":"2024-03-18T20:34:39.198695+0000","last_clean":"2024-03-18T20:34:39.198695+0000","last_became_active":"2024-03-18T20:34:38.809415+0000","last_became_peered":"2024-03-18T20:34:38.809415+0000","last_unstale":"2024-03-18T20:34:39.198695+0000","last_undegraded":"2024-03-18T20:34:39.198695+0000","last_fullsized":"2024-03-18T20:34:39.198695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:34:30.761582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.813819+0000","last_change":"2024-03-18T20:34:38.342080+0000","last_active":"2024-03-18T20:34:41.813819+0000","last_peered":"2024-03-18T20:34:41.813819+0000","last_clean":"2024-03-18T20:34:41.813819+0000","last_became_active":"2024-03-18T20:34:38.340623+0000","last_became_peered":"2024-03-18T20:34:38.340623+0000","last_unstale":"2024-03-18T20:34:41.813819+0000","last_undegraded":"2024-03-18T20:34:41.813819+0000","last_fullsized":"2024-03-18T20:34:41.813819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:49:14.912391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901335+0000","last_change":"2024-03-18T20:34:38.806558+0000","last_active":"2024-03-18T20:34:39.901335+0000","last_peered":"2024-03-18T20:34:39.901335+0000","last_clean":"2024-03-18T20:34:39.901335+0000","last_became_active":"2024-03-18T20:34:38.806171+0000","last_became_peered":"2024-03-18T20:34:38.806171+0000","last_unstale":"2024-03-18T20:34:39.901335+0000","last_undegraded":"2024-03-18T20:34:39.901335+0000","last_fullsized":"2024-03-18T20:34:39.901335+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:46:26.530157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.813762+0000","last_change":"2024-03-18T20:34:38.336175+0000","last_active":"2024-03-18T20:34:41.813762+0000","last_peered":"2024-03-18T20:34:41.813762+0000","last_clean":"2024-03-18T20:34:41.813762+0000","last_became_active":"2024-03-18T20:34:38.335798+0000","last_became_peered":"2024-03-18T20:34:38.335798+0000","last_unstale":"2024-03-18T20:34:41.813762+0000","last_undegraded":"2024-03-18T20:34:41.813762+0000","last_fullsized":"2024-03-18T20:34:41.813762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:33:33.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996504+0000","last_change":"2024-03-18T20:34:38.807684+0000","last_active":"2024-03-18T20:34:39.996504+0000","last_peered":"2024-03-18T20:34:39.996504+0000","last_clean":"2024-03-18T20:34:39.996504+0000","last_became_active":"2024-03-18T20:34:38.807300+0000","last_became_peered":"2024-03-18T20:34:38.807300+0000","last_unstale":"2024-03-18T20:34:39.996504+0000","last_undegraded":"2024-03-18T20:34:39.996504+0000","last_fullsized":"2024-03-18T20:34:39.996504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:31:43.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'5","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:43.831864+0000","last_change":"2024-03-18T20:34:38.341802+0000","last_active":"2024-03-18T20:34:43.831864+0000","last_peered":"2024-03-18T20:34:43.831864+0000","last_clean":"2024-03-18T20:34:43.831864+0000","last_became_active":"2024-03-18T20:34:38.340594+0000","last_became_peered":"2024-03-18T20:34:38.340594+0000","last_unstale":"2024-03-18T20:34:43.831864+0000","last_undegraded":"2024-03-18T20:34:43.831864+0000","last_fullsized":"2024-03-18T20:34:43.831864+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:01:06.830323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901738+0000","last_change":"2024-03-18T20:34:38.809790+0000","last_active":"2024-03-18T20:34:39.901738+0000","last_peered":"2024-03-18T20:34:39.901738+0000","last_clean":"2024-03-18T20:34:39.901738+0000","last_became_active":"2024-03-18T20:34:38.809470+0000","last_became_peered":"2024-03-18T20:34:38.809470+0000","last_unstale":"2024-03-18T20:34:39.901738+0000","last_undegraded":"2024-03-18T20:34:39.901738+0000","last_fullsized":"2024-03-18T20:34:39.901738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:48.225759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997300+0000","last_change":"2024-03-18T20:34:38.343103+0000","last_active":"2024-03-18T20:34:39.997300+0000","last_peered":"2024-03-18T20:34:39.997300+0000","last_clean":"2024-03-18T20:34:39.997300+0000","last_became_active":"2024-03-18T20:34:38.342744+0000","last_became_peered":"2024-03-18T20:34:38.342744+0000","last_unstale":"2024-03-18T20:34:39.997300+0000","last_undegraded":"2024-03-18T20:34:39.997300+0000","last_fullsized":"2024-03-18T20:34:39.997300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:39.078219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198604+0000","last_change":"2024-03-18T20:34:38.809294+0000","last_active":"2024-03-18T20:34:39.198604+0000","last_peered":"2024-03-18T20:34:39.198604+0000","last_clean":"2024-03-18T20:34:39.198604+0000","last_became_active":"2024-03-18T20:34:38.808542+0000","last_became_peered":"2024-03-18T20:34:38.808542+0000","last_unstale":"2024-03-18T20:34:39.198604+0000","last_undegraded":"2024-03-18T20:34:39.198604+0000","last_fullsized":"2024-03-18T20:34:39.198604+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:11:30.669317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.813639+0000","last_change":"2024-03-18T20:34:38.339454+0000","last_active":"2024-03-18T20:34:41.813639+0000","last_peered":"2024-03-18T20:34:41.813639+0000","last_clean":"2024-03-18T20:34:41.813639+0000","last_became_active":"2024-03-18T20:34:38.339154+0000","last_became_peered":"2024-03-18T20:34:38.339154+0000","last_unstale":"2024-03-18T20:34:41.813639+0000","last_undegraded":"2024-03-18T20:34:41.813639+0000","last_fullsized":"2024-03-18T20:34:41.813639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:12:06.300600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998255+0000","last_change":"2024-03-18T20:34:38.818462+0000","last_active":"2024-03-18T20:34:39.998255+0000","last_peered":"2024-03-18T20:34:39.998255+0000","last_clean":"2024-03-18T20:34:39.998255+0000","last_became_active":"2024-03-18T20:34:38.817522+0000","last_became_peered":"2024-03-18T20:34:38.817522+0000","last_unstale":"2024-03-18T20:34:39.998255+0000","last_undegraded":"2024-03-18T20:34:39.998255+0000","last_fullsized":"2024-03-18T20:34:39.998255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:50:32.258583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997694+0000","last_change":"2024-03-18T20:34:38.341054+0000","last_active":"2024-03-18T20:34:39.997694+0000","last_peered":"2024-03-18T20:34:39.997694+0000","last_clean":"2024-03-18T20:34:39.997694+0000","last_became_active":"2024-03-18T20:34:38.340352+0000","last_became_peered":"2024-03-18T20:34:38.340352+0000","last_unstale":"2024-03-18T20:34:39.997694+0000","last_undegraded":"2024-03-18T20:34:39.997694+0000","last_fullsized":"2024-03-18T20:34:39.997694+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:44.850244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997144+0000","last_change":"2024-03-18T20:34:38.818176+0000","last_active":"2024-03-18T20:34:39.997144+0000","last_peered":"2024-03-18T20:34:39.997144+0000","last_clean":"2024-03-18T20:34:39.997144+0000","last_became_active":"2024-03-18T20:34:38.817820+0000","last_became_peered":"2024-03-18T20:34:38.817820+0000","last_unstale":"2024-03-18T20:34:39.997144+0000","last_undegraded":"2024-03-18T20:34:39.997144+0000","last_fullsized":"2024-03-18T20:34:39.997144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:19:29.815174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.795014+0000","last_change":"2024-03-18T20:34:38.346591+0000","last_active":"2024-03-18T20:34:39.795014+0000","last_peered":"2024-03-18T20:34:39.795014+0000","last_clean":"2024-03-18T20:34:39.795014+0000","last_became_active":"2024-03-18T20:34:38.345897+0000","last_became_peered":"2024-03-18T20:34:38.345897+0000","last_unstale":"2024-03-18T20:34:39.795014+0000","last_undegraded":"2024-03-18T20:34:39.795014+0000","last_fullsized":"2024-03-18T20:34:39.795014+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:47:32.118905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997344+0000","last_change":"2024-03-18T20:34:38.804814+0000","last_active":"2024-03-18T20:34:39.997344+0000","last_peered":"2024-03-18T20:34:39.997344+0000","last_clean":"2024-03-18T20:34:39.997344+0000","last_became_active":"2024-03-18T20:34:38.804585+0000","last_became_peered":"2024-03-18T20:34:38.804585+0000","last_unstale":"2024-03-18T20:34:39.997344+0000","last_undegraded":"2024-03-18T20:34:39.997344+0000","last_fullsized":"2024-03-18T20:34:39.997344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:40:47.273070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.818161+0000","last_change":"2024-03-18T20:34:38.344615+0000","last_active":"2024-03-18T20:34:38.818161+0000","last_peered":"2024-03-18T20:34:38.818161+0000","last_clean":"2024-03-18T20:34:38.818161+0000","last_became_active":"2024-03-18T20:34:38.344242+0000","last_became_peered":"2024-03-18T20:34:38.344242+0000","last_unstale":"2024-03-18T20:34:38.818161+0000","last_undegraded":"2024-03-18T20:34:38.818161+0000","last_fullsized":"2024-03-18T20:34:38.818161+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.449808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997264+0000","last_change":"2024-03-18T20:34:38.820875+0000","last_active":"2024-03-18T20:34:39.997264+0000","last_peered":"2024-03-18T20:34:39.997264+0000","last_clean":"2024-03-18T20:34:39.997264+0000","last_became_active":"2024-03-18T20:34:38.820550+0000","last_became_peered":"2024-03-18T20:34:38.820550+0000","last_unstale":"2024-03-18T20:34:39.997264+0000","last_undegraded":"2024-03-18T20:34:39.997264+0000","last_fullsized":"2024-03-18T20:34:39.997264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:49:36.358734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901824+0000","last_change":"2024-03-18T20:34:38.812552+0000","last_active":"2024-03-18T20:34:39.901824+0000","last_peered":"2024-03-18T20:34:39.901824+0000","last_clean":"2024-03-18T20:34:39.901824+0000","last_became_active":"2024-03-18T20:34:38.811939+0000","last_became_peered":"2024-03-18T20:34:38.811939+0000","last_unstale":"2024-03-18T20:34:39.901824+0000","last_undegraded":"2024-03-18T20:34:39.901824+0000","last_fullsized":"2024-03-18T20:34:39.901824+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:41:52.096974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997308+0000","last_change":"2024-03-18T20:34:38.815691+0000","last_active":"2024-03-18T20:34:39.997308+0000","last_peered":"2024-03-18T20:34:39.997308+0000","last_clean":"2024-03-18T20:34:39.997308+0000","last_became_active":"2024-03-18T20:34:38.815370+0000","last_became_peered":"2024-03-18T20:34:38.815370+0000","last_unstale":"2024-03-18T20:34:39.997308+0000","last_undegraded":"2024-03-18T20:34:39.997308+0000","last_fullsized":"2024-03-18T20:34:39.997308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:59:41.260013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997260+0000","last_change":"2024-03-18T20:34:38.812651+0000","last_active":"2024-03-18T20:34:39.997260+0000","last_peered":"2024-03-18T20:34:39.997260+0000","last_clean":"2024-03-18T20:34:39.997260+0000","last_became_active":"2024-03-18T20:34:38.812322+0000","last_became_peered":"2024-03-18T20:34:38.812322+0000","last_unstale":"2024-03-18T20:34:39.997260+0000","last_undegraded":"2024-03-18T20:34:39.997260+0000","last_fullsized":"2024-03-18T20:34:39.997260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:34:55.658371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901247+0000","last_change":"2024-03-18T20:34:38.806475+0000","last_active":"2024-03-18T20:34:39.901247+0000","last_peered":"2024-03-18T20:34:39.901247+0000","last_clean":"2024-03-18T20:34:39.901247+0000","last_became_active":"2024-03-18T20:34:38.806027+0000","last_became_peered":"2024-03-18T20:34:38.806027+0000","last_unstale":"2024-03-18T20:34:39.901247+0000","last_undegraded":"2024-03-18T20:34:39.901247+0000","last_fullsized":"2024-03-18T20:34:39.901247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:57:17.140031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793496+0000","last_change":"2024-03-18T20:34:38.811057+0000","last_active":"2024-03-18T20:34:39.793496+0000","last_peered":"2024-03-18T20:34:39.793496+0000","last_clean":"2024-03-18T20:34:39.793496+0000","last_became_active":"2024-03-18T20:34:38.810316+0000","last_became_peered":"2024-03-18T20:34:38.810316+0000","last_unstale":"2024-03-18T20:34:39.793496+0000","last_undegraded":"2024-03-18T20:34:39.793496+0000","last_fullsized":"2024-03-18T20:34:39.793496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:29:22.426717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.902010+0000","last_change":"2024-03-18T20:34:38.812952+0000","last_active":"2024-03-18T20:34:39.902010+0000","last_peered":"2024-03-18T20:34:39.902010+0000","last_clean":"2024-03-18T20:34:39.902010+0000","last_became_active":"2024-03-18T20:34:38.812550+0000","last_became_peered":"2024-03-18T20:34:38.812550+0000","last_unstale":"2024-03-18T20:34:39.902010+0000","last_undegraded":"2024-03-18T20:34:39.902010+0000","last_fullsized":"2024-03-18T20:34:39.902010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:06:26.525078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997748+0000","last_change":"2024-03-18T20:34:38.817358+0000","last_active":"2024-03-18T20:34:39.997748+0000","last_peered":"2024-03-18T20:34:39.997748+0000","last_clean":"2024-03-18T20:34:39.997748+0000","last_became_active":"2024-03-18T20:34:38.816939+0000","last_became_peered":"2024-03-18T20:34:38.816939+0000","last_unstale":"2024-03-18T20:34:39.997748+0000","last_undegraded":"2024-03-18T20:34:39.997748+0000","last_fullsized":"2024-03-18T20:34:39.997748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:33.258162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198671+0000","last_change":"2024-03-18T20:34:38.807798+0000","last_active":"2024-03-18T20:34:39.198671+0000","last_peered":"2024-03-18T20:34:39.198671+0000","last_clean":"2024-03-18T20:34:39.198671+0000","last_became_active":"2024-03-18T20:34:38.807489+0000","last_became_peered":"2024-03-18T20:34:38.807489+0000","last_unstale":"2024-03-18T20:34:39.198671+0000","last_undegraded":"2024-03-18T20:34:39.198671+0000","last_fullsized":"2024-03-18T20:34:39.198671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:15.395763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759267+0000","last_change":"2024-03-18T20:34:38.806363+0000","last_active":"2024-03-18T20:34:39.759267+0000","last_peered":"2024-03-18T20:34:39.759267+0000","last_clean":"2024-03-18T20:34:39.759267+0000","last_became_active":"2024-03-18T20:34:38.805707+0000","last_became_peered":"2024-03-18T20:34:38.805707+0000","last_unstale":"2024-03-18T20:34:39.759267+0000","last_undegraded":"2024-03-18T20:34:39.759267+0000","last_fullsized":"2024-03-18T20:34:39.759267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:10.918979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198542+0000","last_change":"2024-03-18T20:34:38.814439+0000","last_active":"2024-03-18T20:34:39.198542+0000","last_peered":"2024-03-18T20:34:39.198542+0000","last_clean":"2024-03-18T20:34:39.198542+0000","last_became_active":"2024-03-18T20:34:38.813988+0000","last_became_peered":"2024-03-18T20:34:38.813988+0000","last_unstale":"2024-03-18T20:34:39.198542+0000","last_undegraded":"2024-03-18T20:34:39.198542+0000","last_fullsized":"2024-03-18T20:34:39.198542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:03:14.639044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.821809+0000","last_change":"2024-03-18T20:34:38.818550+0000","last_active":"2024-03-18T20:34:38.821809+0000","last_peered":"2024-03-18T20:34:38.821809+0000","last_clean":"2024-03-18T20:34:38.821809+0000","last_became_active":"2024-03-18T20:34:38.818209+0000","last_became_peered":"2024-03-18T20:34:38.818209+0000","last_unstale":"2024-03-18T20:34:38.821809+0000","last_undegraded":"2024-03-18T20:34:38.821809+0000","last_fullsized":"2024-03-18T20:34:38.821809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:40:48.395331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793586+0000","last_change":"2024-03-18T20:34:38.808439+0000","last_active":"2024-03-18T20:34:39.793586+0000","last_peered":"2024-03-18T20:34:39.793586+0000","last_clean":"2024-03-18T20:34:39.793586+0000","last_became_active":"2024-03-18T20:34:38.807995+0000","last_became_peered":"2024-03-18T20:34:38.807995+0000","last_unstale":"2024-03-18T20:34:39.793586+0000","last_undegraded":"2024-03-18T20:34:39.793586+0000","last_fullsized":"2024-03-18T20:34:39.793586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:54.414148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901907+0000","last_change":"2024-03-18T20:34:38.810793+0000","last_active":"2024-03-18T20:34:39.901907+0000","last_peered":"2024-03-18T20:34:39.901907+0000","last_clean":"2024-03-18T20:34:39.901907+0000","last_became_active":"2024-03-18T20:34:38.810496+0000","last_became_peered":"2024-03-18T20:34:38.810496+0000","last_unstale":"2024-03-18T20:34:39.901907+0000","last_undegraded":"2024-03-18T20:34:39.901907+0000","last_fullsized":"2024-03-18T20:34:39.901907+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:33.780675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5708,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":130,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":16538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612618,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":851968,"data_stored":405743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":10,"up_from":66,"seq":283467841549,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1466368,"data_stored":997156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]}]},{"osd":9,"up_from":61,"seq":261993005074,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":405721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.32200000000000001}]},{"osd":1,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.443}]},{"osd":2,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.308}]},{"osd":4,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.47699999999999998}]},{"osd":5,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.49099999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.39600000000000002}]},{"osd":7,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52100000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.67000000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]}]},{"osd":8,"up_from":54,"seq":231928234005,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":405225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.68700000000000006}]},{"osd":2,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.58999999999999997}]},{"osd":3,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.89000000000000001}]},{"osd":4,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.749},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.83299999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.80300000000000005}]},{"osd":6,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.93500000000000005}]},{"osd":7,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.85199999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.97299999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":7,"up_from":47,"seq":201863462936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1466368,"data_stored":998357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.439}]},{"osd":1,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.378}]},{"osd":2,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.66500000000000004}]},{"osd":3,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53900000000000003}]},{"osd":4,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.39900000000000002}]},{"osd":5,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.64200000000000002}]},{"osd":6,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.67600000000000005}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.749}]},{"osd":9,"last update":"Mon Mar 18 20:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.73499999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":6,"up_from":42,"seq":180388626459,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":404599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.33500000000000002}]},{"osd":1,"last update":"Mon Mar 18 20:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.36399999999999999}]},{"osd":2,"last update":"Mon Mar 18 20:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.627}]},{"osd":3,"last update":"Mon Mar 18 20:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54000000000000004}]},{"osd":4,"last update":"Mon Mar 18 20:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.436},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.46999999999999997}]},{"osd":5,"last update":"Mon Mar 18 20:33:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.61499999999999999}]},{"osd":7,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.59999999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.71399999999999997}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.64100000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":406163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.54300000000000004},"min":{"1min":0.32400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.81799999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.432}]},{"osd":2,"last update":"Mon Mar 18 20:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.745,"5min":0.754,"15min":0.754},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71399999999999997}]},{"osd":3,"last update":"Mon Mar 18 20:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71599999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72999999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51400000000000001}]},{"osd":4,"last update":"Mon Mar 18 20:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.41499999999999998,"15min":0.40100000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.42399999999999999,"15min":0.41199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.61399999999999999}]},{"osd":5,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.749}]},{"osd":6,"last update":"Mon Mar 18 20:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0229999999999999,"5min":1.196,"15min":1.196},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.434,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79200000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56100000000000005}]},{"osd":7,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.52700000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.79400000000000004}]},{"osd":9,"last update":"Mon Mar 18 20:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.65600000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738415,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":851968,"data_stored":405293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.36699999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Mar 18 20:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.44700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.627},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92600000000000005,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.745}]},{"osd":3,"last update":"Mon Mar 18 20:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.32700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.38300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.61599999999999999}]},{"osd":4,"last update":"Mon Mar 18 20:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.33800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.64300000000000002}]},{"osd":5,"last update":"Mon Mar 18 20:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Mar 18 20:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.80800000000000005}]},{"osd":7,"last update":"Mon Mar 18 20:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.88700000000000001}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.78300000000000003}]},{"osd":9,"last update":"Mon Mar 18 20:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.85399999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411370,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":405528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.48699999999999999,"15min":0.46400000000000002},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.75,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.48399999999999999,"15min":0.46300000000000002},"min":{"1min":0.318,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57999999999999996}]},{"osd":1,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.56000000000000005,"15min":0.52900000000000003},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.6100000000000003,"5min":4.6100000000000003,"15min":4.6100000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.71599999999999997}]},{"osd":3,"last update":"Mon Mar 18 20:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52300000000000002,"15min":0.51300000000000001},"min":{"1min":0.39900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.52300000000000002},"min":{"1min":0.40799999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68999999999999995}]},{"osd":4,"last update":"Mon Mar 18 20:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.436,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.44500000000000001,"15min":0.438},"min":{"1min":0.307,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35099999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42699999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84999999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.626},{"interface":"front","average":{"1min":0.625,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.77100000000000002,"5min":1.083,"15min":1.083},"last":0.74199999999999999}]},{"osd":7,"last update":"Mon Mar 18 20:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.439},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45800000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.53900000000000003}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":3,"up_from":24,"seq":103079215141,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":404883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.53400000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.40600000000000003,"15min":0.40799999999999997},"min":{"1min":0.28799999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.503,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.47399999999999998}]},{"osd":1,"last update":"Mon Mar 18 20:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53300000000000003,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73499999999999999,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68200000000000005}]},{"osd":2,"last update":"Mon Mar 18 20:33:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72899999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.622},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.80800000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Mon Mar 18 20:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83199999999999996,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.01,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Mar 18 20:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.499,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.47399999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.79900000000000004}]},{"osd":6,"last update":"Mon Mar 18 20:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83999999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.755}]},{"osd":7,"last update":"Mon Mar 18 20:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73099999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.81899999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":404621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.38400000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.35599999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.35799999999999998,"15min":0.35399999999999998},"min":{"1min":0.26400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.495,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.439}]},{"osd":2,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.41699999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.433,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.874,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65300000000000002}]},{"osd":3,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53600000000000003}]},{"osd":5,"last update":"Mon Mar 18 20:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77600000000000002,"5min":0.875,"15min":0.875},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Mar 18 20:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.48199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.747}]},{"osd":7,"last update":"Mon Mar 18 20:33:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58999999999999997}]},{"osd":8,"last update":"Mon Mar 18 20:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.81999999999999995}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77700000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]}]},{"osd":5,"up_from":35,"seq":150323855391,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1470464,"data_stored":996826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.60999999999999999}]},{"osd":1,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69099999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.42999999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.41499999999999998}]},{"osd":2,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.27900000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.27900000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.35699999999999998}]},{"osd":3,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.35499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76300000000000001,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.433}]},{"osd":4,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.75800000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.66000000000000003}]},{"osd":6,"last update":"Mon Mar 18 20:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58499999999999996}]},{"osd":7,"last update":"Mon Mar 18 20:33:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.625},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46899999999999997}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.50900000000000001}]},{"osd":9,"last update":"Mon Mar 18 20:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67100000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T20:34:47.700 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-18T20:34:47.700 INFO:tasks.cephadm:Setting up client nodes... 2024-03-18T20:34:47.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-18T20:34:47.988 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:48.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:47 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3888901103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:48.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:47 smithi046 ceph-mon[43495]: pgmap v223: 97 pgs: 97 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 248 B/s rd, 5.6 KiB/s wr, 16 op/s 2024-03-18T20:34:48.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:47 smithi046 ceph-mon[43495]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:34:48.220 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 20:34:48 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: 2024-03-18T20:34:48.033+0000 7f60b60db700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-18T20:34:48.220 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 20:34:48 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: 2024-03-18T20:34:48.033+0000 7f60b60db700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-18T20:34:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:47 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3888901103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:47 smithi073 ceph-mon[47384]: pgmap v223: 97 pgs: 97 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 248 B/s rd, 5.6 KiB/s wr, 16 op/s 2024-03-18T20:34:48.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:47 smithi073 ceph-mon[47384]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:34:48.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:47 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3888901103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:48.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:47 smithi155 ceph-mon[48344]: pgmap v223: 97 pgs: 97 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 248 B/s rd, 5.6 KiB/s wr, 16 op/s 2024-03-18T20:34:48.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:47 smithi155 ceph-mon[48344]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:34:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.975+0000 7f9bbded1700 1 -- 172.21.15.46:0/285103351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8106b70 msgr2=0x7f9bb8108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.975+0000 7f9bbded1700 1 --2- 172.21.15.46:0/285103351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8106b70 0x7f9bb8108f60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000b0a0 tx=0x7f9bb000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.975+0000 7f9bbded1700 1 -- 172.21.15.46:0/285103351 shutdown_connections 2024-03-18T20:34:48.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.975+0000 7f9bbded1700 1 --2- 172.21.15.46:0/285103351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8106b70 0x7f9bb8108f60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:48.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.975+0000 7f9bbded1700 1 --2- 172.21.15.46:0/285103351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8101910 0x7f9bb8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:48.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.975+0000 7f9bbded1700 1 --2- 172.21.15.46:0/285103351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bb8104240 0x7f9bb8106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:48.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.975+0000 7f9bbded1700 1 -- 172.21.15.46:0/285103351 >> 172.21.15.46:0/285103351 conn(0x7f9bb80fb280 msgr2=0x7f9bb80fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:48.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.975+0000 7f9bbded1700 1 -- 172.21.15.46:0/285103351 shutdown_connections 2024-03-18T20:34:48.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.975+0000 7f9bbded1700 1 -- 172.21.15.46:0/285103351 wait complete. 2024-03-18T20:34:48.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.976+0000 7f9bbded1700 1 Processor -- start 2024-03-18T20:34:48.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bbded1700 1 -- start start 2024-03-18T20:34:48.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bbded1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8101910 0x7f9bb8196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:48.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bbded1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8104240 0x7f9bb81973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:48.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bbded1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bb8106b70 0x7f9bb819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:48.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bbded1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb819e970 con 0x7f9bb8101910 2024-03-18T20:34:48.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bbded1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb819eab0 con 0x7f9bb8104240 2024-03-18T20:34:48.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bbded1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb819ec40 con 0x7f9bb8106b70 2024-03-18T20:34:48.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bb77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8101910 0x7f9bb8196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bb77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8101910 0x7f9bb8196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42860/0 (socket says 172.21.15.46:42860) 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bb77fe700 1 -- 172.21.15.46:0/3989180055 learned_addr learned my addr 172.21.15.46:0/3989180055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bb6ffd700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8104240 0x7f9bb81973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bb7fff700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bb8106b70 0x7f9bb819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bb77fe700 1 -- 172.21.15.46:0/3989180055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bb8106b70 msgr2=0x7f9bb819e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bb77fe700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bb8106b70 0x7f9bb819e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bb77fe700 1 -- 172.21.15.46:0/3989180055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8104240 msgr2=0x7f9bb81973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bb77fe700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8104240 0x7f9bb81973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bb77fe700 1 -- 172.21.15.46:0/3989180055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb0009d00 con 0x7f9bb8101910 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.977+0000 7f9bb6ffd700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8104240 0x7f9bb81973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.978+0000 7f9bb77fe700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8101910 0x7f9bb8196e80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800c950 tx=0x7f9ba800cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.978+0000 7f9bb4ff9700 1 -- 172.21.15.46:0/3989180055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8007ac0 con 0x7f9bb8101910 2024-03-18T20:34:48.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.978+0000 7f9bb4ff9700 1 -- 172.21.15.46:0/3989180055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ba8004cf0 con 0x7f9bb8101910 2024-03-18T20:34:48.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.978+0000 7f9bbded1700 1 -- 172.21.15.46:0/3989180055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb819ef20 con 0x7f9bb8101910 2024-03-18T20:34:48.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.978+0000 7f9bb4ff9700 1 -- 172.21.15.46:0/3989180055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8005550 con 0x7f9bb8101910 2024-03-18T20:34:48.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.978+0000 7f9bbded1700 1 -- 172.21.15.46:0/3989180055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb819f330 con 0x7f9bb8101910 2024-03-18T20:34:48.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.979+0000 7f9bbded1700 1 -- 172.21.15.46:0/3989180055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb80fce00 con 0x7f9bb8101910 2024-03-18T20:34:48.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.980+0000 7f9bb4ff9700 1 -- 172.21.15.46:0/3989180055 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f9ba8004750 con 0x7f9bb8101910 2024-03-18T20:34:48.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.981+0000 7f9bb4ff9700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9ba00a11b0 0x7f9ba00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:48.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.981+0000 7f9bb6ffd700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9ba00a11b0 0x7f9ba00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:48.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.981+0000 7f9bb4ff9700 1 -- 172.21.15.46:0/3989180055 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9ba80cc4c0 con 0x7f9bb8101910 2024-03-18T20:34:48.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.981+0000 7f9bb6ffd700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9ba00a11b0 0x7f9ba00a3670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0099d0 tx=0x7f9bac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:48.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:48.985+0000 7f9bb4ff9700 1 -- 172.21.15.46:0/3989180055 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9ba8095ca0 con 0x7f9bb8101910 2024-03-18T20:34:49.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:48 smithi046 ceph-mon[43495]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:34:49.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:48 smithi073 ceph-mon[47384]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:34:49.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.283+0000 7f9bbded1700 1 -- 172.21.15.46:0/3989180055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f9bb804ea80 con 0x7f9bb8101910 2024-03-18T20:34:49.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:48 smithi155 ceph-mon[48344]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:34:49.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.288+0000 7f9bb4ff9700 1 -- 172.21.15.46:0/3989180055 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f9ba80992c0 con 0x7f9bb8101910 2024-03-18T20:34:49.291 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-03-18T20:34:49.291 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBppfhlTYL7EBAA4hyCDHzycp6hM5H4aSQuTw== 2024-03-18T20:34:49.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.290+0000 7f9bbded1700 1 -- 172.21.15.46:0/3989180055 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9ba00a11b0 msgr2=0x7f9ba00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:49.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.290+0000 7f9bbded1700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9ba00a11b0 0x7f9ba00a3670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0099d0 tx=0x7f9bac008040 comp rx=0 tx=0).stop 2024-03-18T20:34:49.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.290+0000 7f9bbded1700 1 -- 172.21.15.46:0/3989180055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8101910 msgr2=0x7f9bb8196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:49.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.290+0000 7f9bbded1700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8101910 0x7f9bb8196e80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800c950 tx=0x7f9ba800cd10 comp rx=0 tx=0).stop 2024-03-18T20:34:49.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.290+0000 7f9bbded1700 1 -- 172.21.15.46:0/3989180055 shutdown_connections 2024-03-18T20:34:49.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.290+0000 7f9bbded1700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9ba00a11b0 0x7f9ba00a3670 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:49.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.290+0000 7f9bbded1700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8101910 0x7f9bb8196e80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:49.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.290+0000 7f9bbded1700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bb8106b70 0x7f9bb819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:49.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.290+0000 7f9bbded1700 1 --2- 172.21.15.46:0/3989180055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8104240 0x7f9bb81973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:49.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.290+0000 7f9bbded1700 1 -- 172.21.15.46:0/3989180055 >> 172.21.15.46:0/3989180055 conn(0x7f9bb80fb280 msgr2=0x7f9bb8102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:49.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.290+0000 7f9bbded1700 1 -- 172.21.15.46:0/3989180055 shutdown_connections 2024-03-18T20:34:49.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:49.290+0000 7f9bbded1700 1 -- 172.21.15.46:0/3989180055 wait complete. 2024-03-18T20:34:49.685 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:34:49.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-18T20:34:49.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-18T20:34:49.738 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-18T20:34:49.738 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-18T20:34:49.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-18T20:34:50.033 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:50.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:49 smithi046 ceph-mon[43495]: pgmap v224: 97 pgs: 97 active+clean; 620 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-03-18T20:34:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:49 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3989180055' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T20:34:50.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:49 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3989180055' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T20:34:50.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:49 smithi073 ceph-mon[47384]: pgmap v224: 97 pgs: 97 active+clean; 620 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-03-18T20:34:50.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:49 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3989180055' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T20:34:50.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:49 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3989180055' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T20:34:50.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:49 smithi155 ceph-mon[48344]: pgmap v224: 97 pgs: 97 active+clean; 620 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-03-18T20:34:50.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:49 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3989180055' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T20:34:50.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:49 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3989180055' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T20:34:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.129+0000 7f358ddd8700 1 -- 172.21.15.46:0/2131935736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35880ff750 msgr2=0x7f35880ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.129+0000 7f358ddd8700 1 --2- 172.21.15.46:0/2131935736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35880ff750 0x7f35880ffbb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f357c009990 tx=0x7f357c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:34:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.129+0000 7f358ddd8700 1 -- 172.21.15.46:0/2131935736 shutdown_connections 2024-03-18T20:34:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.129+0000 7f358ddd8700 1 --2- 172.21.15.46:0/2131935736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35881000f0 0x7f358810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.129+0000 7f358ddd8700 1 --2- 172.21.15.46:0/2131935736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3588105e90 0x7f3588106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.129+0000 7f358ddd8700 1 --2- 172.21.15.46:0/2131935736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35880ff750 0x7f35880ffbb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.129+0000 7f358ddd8700 1 -- 172.21.15.46:0/2131935736 >> 172.21.15.46:0/2131935736 conn(0x7f3588074fd0 msgr2=0x7f35880753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.129+0000 7f358ddd8700 1 -- 172.21.15.46:0/2131935736 shutdown_connections 2024-03-18T20:34:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.129+0000 7f358ddd8700 1 -- 172.21.15.46:0/2131935736 wait complete. 2024-03-18T20:34:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.130+0000 7f358ddd8700 1 Processor -- start 2024-03-18T20:34:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.130+0000 7f358ddd8700 1 -- start start 2024-03-18T20:34:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.130+0000 7f358ddd8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35880ff750 0x7f3588197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.130+0000 7f358ddd8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35881000f0 0x7f3588197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.130+0000 7f358ddd8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3588105e90 0x7f358819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.130+0000 7f358ddd8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358819c0e0 con 0x7f35880ff750 2024-03-18T20:34:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.130+0000 7f358ddd8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358819c250 con 0x7f35881000f0 2024-03-18T20:34:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.130+0000 7f358ddd8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358819c410 con 0x7f3588105e90 2024-03-18T20:34:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.130+0000 7f35877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35880ff750 0x7f3588197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.130+0000 7f35877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35880ff750 0x7f3588197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42864/0 (socket says 172.21.15.46:42864) 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.130+0000 7f35877fe700 1 -- 172.21.15.46:0/2503436300 learned_addr learned my addr 172.21.15.46:0/2503436300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f3586ffd700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35881000f0 0x7f3588197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f35877fe700 1 -- 172.21.15.46:0/2503436300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3588105e90 msgr2=0x7f358819bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f3587fff700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3588105e90 0x7f358819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f35877fe700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3588105e90 0x7f358819bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f35877fe700 1 -- 172.21.15.46:0/2503436300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35881000f0 msgr2=0x7f3588197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f35877fe700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35881000f0 0x7f3588197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f35877fe700 1 -- 172.21.15.46:0/2503436300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c0095f0 con 0x7f35880ff750 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f3586ffd700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35881000f0 0x7f3588197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f3587fff700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3588105e90 0x7f358819bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f35877fe700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35880ff750 0x7f3588197230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f357800d980 tx=0x7f357800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f3584ff9700 1 -- 172.21.15.46:0/2503436300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357800bb60 con 0x7f35880ff750 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f358ddd8700 1 -- 172.21.15.46:0/2503436300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358819c6f0 con 0x7f35880ff750 2024-03-18T20:34:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f3584ff9700 1 -- 172.21.15.46:0/2503436300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3578007620 con 0x7f35880ff750 2024-03-18T20:34:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f3584ff9700 1 -- 172.21.15.46:0/2503436300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357800f620 con 0x7f35880ff750 2024-03-18T20:34:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.131+0000 7f358ddd8700 1 -- 172.21.15.46:0/2503436300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35881a3f50 con 0x7f35880ff750 2024-03-18T20:34:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.132+0000 7f358ddd8700 1 -- 172.21.15.46:0/2503436300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35881092f0 con 0x7f35880ff750 2024-03-18T20:34:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.134+0000 7f3584ff9700 1 -- 172.21.15.46:0/2503436300 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3578010460 con 0x7f35880ff750 2024-03-18T20:34:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.135+0000 7f3584ff9700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f35700a11b0 0x7f35700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.135+0000 7f3586ffd700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f35700a11b0 0x7f35700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.135+0000 7f3584ff9700 1 -- 172.21.15.46:0/2503436300 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f35780cd070 con 0x7f35880ff750 2024-03-18T20:34:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.135+0000 7f3586ffd700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f35700a11b0 0x7f35700a3670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f357c00b2e0 tx=0x7f357c005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:51.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.138+0000 7f3584ff9700 1 -- 172.21.15.46:0/2503436300 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3578096850 con 0x7f35880ff750 2024-03-18T20:34:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.410+0000 7f358ddd8700 1 -- 172.21.15.46:0/2503436300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f35880681f0 con 0x7f35880ff750 2024-03-18T20:34:51.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.415+0000 7f3584ff9700 1 -- 172.21.15.46:0/2503436300 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260275 (secure 0 0 0) 0x7f357800de60 con 0x7f35880ff750 2024-03-18T20:34:51.419 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:51.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.418+0000 7f358ddd8700 1 -- 172.21.15.46:0/2503436300 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f35700a11b0 msgr2=0x7f35700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:51.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.418+0000 7f358ddd8700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f35700a11b0 0x7f35700a3670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f357c00b2e0 tx=0x7f357c005e80 comp rx=0 tx=0).stop 2024-03-18T20:34:51.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.418+0000 7f358ddd8700 1 -- 172.21.15.46:0/2503436300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35880ff750 msgr2=0x7f3588197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:51.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.418+0000 7f358ddd8700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35880ff750 0x7f3588197230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f357800d980 tx=0x7f357800dd40 comp rx=0 tx=0).stop 2024-03-18T20:34:51.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.419+0000 7f358ddd8700 1 -- 172.21.15.46:0/2503436300 shutdown_connections 2024-03-18T20:34:51.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.419+0000 7f358ddd8700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f35700a11b0 0x7f35700a3670 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:51.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.419+0000 7f358ddd8700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35880ff750 0x7f3588197230 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:51.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.419+0000 7f358ddd8700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3588105e90 0x7f358819bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:51.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.419+0000 7f358ddd8700 1 --2- 172.21.15.46:0/2503436300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35881000f0 0x7f3588197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:51.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.419+0000 7f358ddd8700 1 -- 172.21.15.46:0/2503436300 >> 172.21.15.46:0/2503436300 conn(0x7f3588074fd0 msgr2=0x7f35880fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:51.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.419+0000 7f358ddd8700 1 -- 172.21.15.46:0/2503436300 shutdown_connections 2024-03-18T20:34:51.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:51.419+0000 7f358ddd8700 1 -- 172.21.15.46:0/2503436300 wait complete. 2024-03-18T20:34:51.803 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3400106473},{"type":"v1","addr":"172.21.15.46:6801","nonce":3400106473}]},"active_addr":"172.21.15.46:6801/3400106473","active_change":"2024-03-18T20:29:36.953364+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.46:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2204627135}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":1017695887}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":604593037}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2587067898}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":3248305722}]}]} 2024-03-18T20:34:51.808 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-18T20:34:51.808 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-18T20:34:51.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T20:34:52.066 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:52.077 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:52 smithi046 ceph-mon[43495]: pgmap v225: 97 pgs: 97 active+clean; 620 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2024-03-18T20:34:52.077 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:52 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2503436300' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T20:34:52.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:52 smithi073 ceph-mon[47384]: pgmap v225: 97 pgs: 97 active+clean; 620 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2024-03-18T20:34:52.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:52 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2503436300' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T20:34:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:52 smithi155 ceph-mon[48344]: pgmap v225: 97 pgs: 97 active+clean; 620 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2024-03-18T20:34:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:52 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2503436300' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T20:34:53.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.132+0000 7f6a81a2d700 1 -- 172.21.15.46:0/504971376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c104f90 msgr2=0x7f6a7c10cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:53.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.132+0000 7f6a81a2d700 1 --2- 172.21.15.46:0/504971376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c104f90 0x7f6a7c10cb60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400b0a0 tx=0x7f6a7400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:53.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.132+0000 7f6a81a2d700 1 -- 172.21.15.46:0/504971376 shutdown_connections 2024-03-18T20:34:53.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.132+0000 7f6a81a2d700 1 --2- 172.21.15.46:0/504971376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c104f90 0x7f6a7c10cb60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:53.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.132+0000 7f6a81a2d700 1 --2- 172.21.15.46:0/504971376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a7c0fff70 0x7f6a7c100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:53.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.132+0000 7f6a81a2d700 1 --2- 172.21.15.46:0/504971376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a7c100920 0x7f6a7c1048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:53.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.132+0000 7f6a81a2d700 1 -- 172.21.15.46:0/504971376 >> 172.21.15.46:0/504971376 conn(0x7f6a7c075d90 msgr2=0x7f6a7c0761a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:53.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.132+0000 7f6a81a2d700 1 -- 172.21.15.46:0/504971376 shutdown_connections 2024-03-18T20:34:53.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.132+0000 7f6a81a2d700 1 -- 172.21.15.46:0/504971376 wait complete. 2024-03-18T20:34:53.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a81a2d700 1 Processor -- start 2024-03-18T20:34:53.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a81a2d700 1 -- start start 2024-03-18T20:34:53.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a81a2d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a7c0fff70 0x7f6a7c101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a81a2d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c100920 0x7f6a7c101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a81a2d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a7c104f90 0x7f6a7c1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a81a2d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1a5740 con 0x7f6a7c100920 2024-03-18T20:34:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a81a2d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1a58b0 con 0x7f6a7c104f90 2024-03-18T20:34:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a81a2d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1a5a70 con 0x7f6a7c0fff70 2024-03-18T20:34:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a7a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c100920 0x7f6a7c101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a7a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c100920 0x7f6a7c101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42886/0 (socket says 172.21.15.46:42886) 2024-03-18T20:34:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a7a7fc700 1 -- 172.21.15.46:0/1992154951 learned_addr learned my addr 172.21.15.46:0/1992154951 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a7b7fe700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a7c104f90 0x7f6a7c1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.133+0000 7f6a7affd700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a7c0fff70 0x7f6a7c101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.134+0000 7f6a7a7fc700 1 -- 172.21.15.46:0/1992154951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a7c0fff70 msgr2=0x7f6a7c101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.134+0000 7f6a7a7fc700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a7c0fff70 0x7f6a7c101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.134+0000 7f6a7a7fc700 1 -- 172.21.15.46:0/1992154951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a7c104f90 msgr2=0x7f6a7c1023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.134+0000 7f6a7a7fc700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a7c104f90 0x7f6a7c1023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.134+0000 7f6a7a7fc700 1 -- 172.21.15.46:0/1992154951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a74009d00 con 0x7f6a7c100920 2024-03-18T20:34:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.134+0000 7f6a7a7fc700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c100920 0x7f6a7c101e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000d950 tx=0x7f6a7000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.134+0000 7f6a80a2b700 1 -- 172.21.15.46:0/1992154951 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70004220 con 0x7f6a7c100920 2024-03-18T20:34:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.134+0000 7f6a81a2d700 1 -- 172.21.15.46:0/1992154951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c1a5d50 con 0x7f6a7c100920 2024-03-18T20:34:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.134+0000 7f6a80a2b700 1 -- 172.21.15.46:0/1992154951 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a70007620 con 0x7f6a7c100920 2024-03-18T20:34:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.134+0000 7f6a80a2b700 1 -- 172.21.15.46:0/1992154951 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70005670 con 0x7f6a7c100920 2024-03-18T20:34:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.134+0000 7f6a81a2d700 1 -- 172.21.15.46:0/1992154951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c1a61f0 con 0x7f6a7c100920 2024-03-18T20:34:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.135+0000 7f6a81a2d700 1 -- 172.21.15.46:0/1992154951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c1092c0 con 0x7f6a7c100920 2024-03-18T20:34:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.136+0000 7f6a80a2b700 1 -- 172.21.15.46:0/1992154951 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f6a70003680 con 0x7f6a7c100920 2024-03-18T20:34:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.137+0000 7f6a80a2b700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6a640a11b0 0x7f6a640a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.137+0000 7f6a7affd700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6a640a11b0 0x7f6a640a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.138+0000 7f6a80a2b700 1 -- 172.21.15.46:0/1992154951 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6a700cca90 con 0x7f6a7c100920 2024-03-18T20:34:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.138+0000 7f6a7affd700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6a640a11b0 0x7f6a640a3670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c005950 tx=0x7f6a6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:53.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.141+0000 7f6a80a2b700 1 -- 172.21.15.46:0/1992154951 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6a700961f0 con 0x7f6a7c100920 2024-03-18T20:34:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.354+0000 7f6a81a2d700 1 -- 172.21.15.46:0/1992154951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a7c04ea80 con 0x7f6a7c100920 2024-03-18T20:34:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.356+0000 7f6a80a2b700 1 -- 172.21.15.46:0/1992154951 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f6a70016090 con 0x7f6a7c100920 2024-03-18T20:34:53.360 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:53.360 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:34:39.786955+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:34:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.358+0000 7f6a81a2d700 1 -- 172.21.15.46:0/1992154951 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6a640a11b0 msgr2=0x7f6a640a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.358+0000 7f6a81a2d700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6a640a11b0 0x7f6a640a3670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c005950 tx=0x7f6a6c00b410 comp rx=0 tx=0).stop 2024-03-18T20:34:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.358+0000 7f6a81a2d700 1 -- 172.21.15.46:0/1992154951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c100920 msgr2=0x7f6a7c101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.358+0000 7f6a81a2d700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c100920 0x7f6a7c101e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000d950 tx=0x7f6a7000dc60 comp rx=0 tx=0).stop 2024-03-18T20:34:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.358+0000 7f6a81a2d700 1 -- 172.21.15.46:0/1992154951 shutdown_connections 2024-03-18T20:34:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.358+0000 7f6a81a2d700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6a640a11b0 0x7f6a640a3670 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.358+0000 7f6a81a2d700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a7c104f90 0x7f6a7c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.358+0000 7f6a81a2d700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a7c0fff70 0x7f6a7c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.358+0000 7f6a81a2d700 1 --2- 172.21.15.46:0/1992154951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c100920 0x7f6a7c101e90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.358+0000 7f6a81a2d700 1 -- 172.21.15.46:0/1992154951 >> 172.21.15.46:0/1992154951 conn(0x7f6a7c075d90 msgr2=0x7f6a7c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.358+0000 7f6a81a2d700 1 -- 172.21.15.46:0/1992154951 shutdown_connections 2024-03-18T20:34:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:53.359+0000 7f6a81a2d700 1 -- 172.21.15.46:0/1992154951 wait complete. 2024-03-18T20:34:53.737 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-18T20:34:53.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T20:34:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:54.455 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:54 smithi046 ceph-mon[43495]: pgmap v226: 97 pgs: 97 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.0 KiB/s wr, 26 op/s 2024-03-18T20:34:54.455 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:54 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1992154951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:54 smithi073 ceph-mon[47384]: pgmap v226: 97 pgs: 97 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.0 KiB/s wr, 26 op/s 2024-03-18T20:34:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:54 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1992154951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:54 smithi155 ceph-mon[48344]: pgmap v226: 97 pgs: 97 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 9.0 KiB/s wr, 26 op/s 2024-03-18T20:34:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:54 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1992154951' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.992+0000 7f982156a700 1 -- 172.21.15.46:0/87141176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c1017c0 msgr2=0x7f981c10dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.992+0000 7f982156a700 1 --2- 172.21.15.46:0/87141176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c1017c0 0x7f981c10dbf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f981400b0a0 tx=0x7f981400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:34:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.992+0000 7f982156a700 1 -- 172.21.15.46:0/87141176 shutdown_connections 2024-03-18T20:34:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.992+0000 7f982156a700 1 --2- 172.21.15.46:0/87141176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c1017c0 0x7f981c10dbf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.992+0000 7f982156a700 1 --2- 172.21.15.46:0/87141176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c106e40 0x7f981c107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.992+0000 7f982156a700 1 --2- 172.21.15.46:0/87141176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f981c100e00 0x7f981c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.992+0000 7f982156a700 1 -- 172.21.15.46:0/87141176 >> 172.21.15.46:0/87141176 conn(0x7f981c077f10 msgr2=0x7f981c078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:54.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.993+0000 7f982156a700 1 -- 172.21.15.46:0/87141176 shutdown_connections 2024-03-18T20:34:54.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.993+0000 7f982156a700 1 -- 172.21.15.46:0/87141176 wait complete. 2024-03-18T20:34:54.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.993+0000 7f982156a700 1 Processor -- start 2024-03-18T20:34:54.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.993+0000 7f982156a700 1 -- start start 2024-03-18T20:34:54.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f982156a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c100e00 0x7f981c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:54.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f982156a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f981c1017c0 0x7f981c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:54.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f982156a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c106e40 0x7f981c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:54.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f982156a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c19c0e0 con 0x7f981c100e00 2024-03-18T20:34:54.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f982156a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c19c250 con 0x7f981c106e40 2024-03-18T20:34:54.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f982156a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c19c410 con 0x7f981c1017c0 2024-03-18T20:34:54.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f981affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c100e00 0x7f981c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:54.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f981affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c100e00 0x7f981c1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42904/0 (socket says 172.21.15.46:42904) 2024-03-18T20:34:54.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f981affd700 1 -- 172.21.15.46:0/308435142 learned_addr learned my addr 172.21.15.46:0/308435142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:54.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f981b7fe700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c106e40 0x7f981c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:54.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f981affd700 1 -- 172.21.15.46:0/308435142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f981c1017c0 msgr2=0x7f981c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f98127fc700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f981c1017c0 0x7f981c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f981affd700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f981c1017c0 0x7f981c1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f981affd700 1 -- 172.21.15.46:0/308435142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c106e40 msgr2=0x7f981c19bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f981affd700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c106e40 0x7f981c19bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f981affd700 1 -- 172.21.15.46:0/308435142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9814009d00 con 0x7f981c100e00 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f981b7fe700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c106e40 0x7f981c19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f98127fc700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f981c1017c0 0x7f981c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f981affd700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c100e00 0x7f981c1972a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f980c00c920 tx=0x7f980c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.994+0000 7f9818ff9700 1 -- 172.21.15.46:0/308435142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c007b00 con 0x7f981c100e00 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.995+0000 7f982156a700 1 -- 172.21.15.46:0/308435142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c19c6f0 con 0x7f981c100e00 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.995+0000 7f9818ff9700 1 -- 172.21.15.46:0/308435142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f980c004cf0 con 0x7f981c100e00 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.995+0000 7f9818ff9700 1 -- 172.21.15.46:0/308435142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c005520 con 0x7f981c100e00 2024-03-18T20:34:55.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.995+0000 7f982156a700 1 -- 172.21.15.46:0/308435142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c0731b0 con 0x7f981c100e00 2024-03-18T20:34:55.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.996+0000 7f982156a700 1 -- 172.21.15.46:0/308435142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c1092d0 con 0x7f981c100e00 2024-03-18T20:34:55.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.997+0000 7f9818ff9700 1 -- 172.21.15.46:0/308435142 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f980c004750 con 0x7f981c100e00 2024-03-18T20:34:55.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.997+0000 7f9818ff9700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f98080a11b0 0x7f98080a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:55.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.997+0000 7f9818ff9700 1 -- 172.21.15.46:0/308435142 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f980c0cc5a0 con 0x7f981c100e00 2024-03-18T20:34:55.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.997+0000 7f98127fc700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f98080a11b0 0x7f98080a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:55.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:54.998+0000 7f98127fc700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f98080a11b0 0x7f98080a3670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9804005fd0 tx=0x7f9804005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:55.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.002+0000 7f9818ff9700 1 -- 172.21.15.46:0/308435142 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f980c095d80 con 0x7f981c100e00 2024-03-18T20:34:55.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.214+0000 7f982156a700 1 -- 172.21.15.46:0/308435142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f981c19c8e0 con 0x7f981c100e00 2024-03-18T20:34:55.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.216+0000 7f9818ff9700 1 -- 172.21.15.46:0/308435142 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f980c01d090 con 0x7f981c100e00 2024-03-18T20:34:55.220 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:34:55.220 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:34:39.786955+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:34:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.218+0000 7f982156a700 1 -- 172.21.15.46:0/308435142 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f98080a11b0 msgr2=0x7f98080a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.218+0000 7f982156a700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f98080a11b0 0x7f98080a3670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9804005fd0 tx=0x7f9804005dc0 comp rx=0 tx=0).stop 2024-03-18T20:34:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.218+0000 7f982156a700 1 -- 172.21.15.46:0/308435142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c100e00 msgr2=0x7f981c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.218+0000 7f982156a700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c100e00 0x7f981c1972a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f980c00c920 tx=0x7f980c00cce0 comp rx=0 tx=0).stop 2024-03-18T20:34:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.218+0000 7f982156a700 1 -- 172.21.15.46:0/308435142 shutdown_connections 2024-03-18T20:34:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.218+0000 7f982156a700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f98080a11b0 0x7f98080a3670 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.218+0000 7f982156a700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c100e00 0x7f981c1972a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.218+0000 7f982156a700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c106e40 0x7f981c19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.218+0000 7f982156a700 1 --2- 172.21.15.46:0/308435142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f981c1017c0 0x7f981c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.218+0000 7f982156a700 1 -- 172.21.15.46:0/308435142 >> 172.21.15.46:0/308435142 conn(0x7f981c077f10 msgr2=0x7f981c0ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.218+0000 7f982156a700 1 -- 172.21.15.46:0/308435142 shutdown_connections 2024-03-18T20:34:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:55.218+0000 7f982156a700 1 -- 172.21.15.46:0/308435142 wait complete. 2024-03-18T20:34:55.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-18T20:34:55.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-18T20:34:55.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-18T20:34:55.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-18T20:34:55.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-18T20:34:55.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-18T20:34:55.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-18T20:34:55.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-18T20:34:55.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-18T20:34:55.596 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-18T20:34:55.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-18T20:34:55.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-18T20:34:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:55 smithi046 ceph-mon[43495]: pgmap v227: 97 pgs: 97 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.0 KiB/s wr, 25 op/s 2024-03-18T20:34:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:55 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/308435142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:55 smithi073 ceph-mon[47384]: pgmap v227: 97 pgs: 97 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.0 KiB/s wr, 25 op/s 2024-03-18T20:34:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:55 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/308435142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:56.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:55 smithi155 ceph-mon[48344]: pgmap v227: 97 pgs: 97 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.0 KiB/s wr, 25 op/s 2024-03-18T20:34:56.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:55 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/308435142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:34:56.995 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:57.045 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:57.551 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:34:58.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:34:58 smithi046 ceph-mon[43495]: pgmap v228: 97 pgs: 97 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.5 KiB/s wr, 20 op/s 2024-03-18T20:34:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:34:58 smithi073 ceph-mon[47384]: pgmap v228: 97 pgs: 97 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.5 KiB/s wr, 20 op/s 2024-03-18T20:34:58.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:34:58 smithi155 ceph-mon[48344]: pgmap v228: 97 pgs: 97 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.5 KiB/s wr, 20 op/s 2024-03-18T20:34:59.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.975+0000 7ff8b5114700 1 -- 172.21.15.46:0/4154589454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b0068750 msgr2=0x7ff8b0068b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:59.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.975+0000 7ff8b5114700 1 --2- 172.21.15.46:0/4154589454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b0068750 0x7ff8b0068b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009990 tx=0x7ff8a4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:34:59.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.975+0000 7ff8b5114700 1 -- 172.21.15.46:0/4154589454 shutdown_connections 2024-03-18T20:34:59.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.975+0000 7ff8b5114700 1 --2- 172.21.15.46:0/4154589454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b0068750 0x7ff8b0068b30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:59.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.975+0000 7ff8b5114700 1 --2- 172.21.15.46:0/4154589454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b010d8c0 0x7ff8b010fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:59.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.975+0000 7ff8b5114700 1 --2- 172.21.15.46:0/4154589454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b0069070 0x7ff8b010d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:59.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.975+0000 7ff8b5114700 1 -- 172.21.15.46:0/4154589454 >> 172.21.15.46:0/4154589454 conn(0x7ff8b0075d50 msgr2=0x7ff8b0076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:34:59.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.975+0000 7ff8b5114700 1 -- 172.21.15.46:0/4154589454 shutdown_connections 2024-03-18T20:34:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.975+0000 7ff8b5114700 1 -- 172.21.15.46:0/4154589454 wait complete. 2024-03-18T20:34:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.976+0000 7ff8b5114700 1 Processor -- start 2024-03-18T20:34:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.976+0000 7ff8b5114700 1 -- start start 2024-03-18T20:34:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.976+0000 7ff8b5114700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b0068750 0x7ff8b01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.976+0000 7ff8b5114700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b0069070 0x7ff8b0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8b5114700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b010d8c0 0x7ff8b019dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8b5114700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b019e2d0 con 0x7ff8b010d8c0 2024-03-18T20:34:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8b5114700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b019e440 con 0x7ff8b0068750 2024-03-18T20:34:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8b5114700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b019e600 con 0x7ff8b0069070 2024-03-18T20:34:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8af59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b010d8c0 0x7ff8b019dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8af59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b010d8c0 0x7ff8b019dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39072/0 (socket says 172.21.15.46:39072) 2024-03-18T20:34:59.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8af59e700 1 -- 172.21.15.46:0/1907120154 learned_addr learned my addr 172.21.15.46:0/1907120154 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8aed9d700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b0068750 0x7ff8b01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8ae59c700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b0069070 0x7ff8b0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8af59e700 1 -- 172.21.15.46:0/1907120154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b0069070 msgr2=0x7ff8b0199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8af59e700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b0069070 0x7ff8b0199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8af59e700 1 -- 172.21.15.46:0/1907120154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b0068750 msgr2=0x7ff8b01993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8af59e700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8b0068750 0x7ff8b01993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8af59e700 1 -- 172.21.15.46:0/1907120154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a40095f0 con 0x7ff8b010d8c0 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff8af59e700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b010d8c0 0x7ff8b019dcc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80094f0 tx=0x7ff8a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.977+0000 7ff897fff700 1 -- 172.21.15.46:0/1907120154 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8004db0 con 0x7ff8b010d8c0 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.978+0000 7ff8b5114700 1 -- 172.21.15.46:0/1907120154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b019e8e0 con 0x7ff8b010d8c0 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.978+0000 7ff897fff700 1 -- 172.21.15.46:0/1907120154 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8a800f040 con 0x7ff8b010d8c0 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.978+0000 7ff897fff700 1 -- 172.21.15.46:0/1907120154 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8005980 con 0x7ff8b010d8c0 2024-03-18T20:34:59.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.978+0000 7ff8b5114700 1 -- 172.21.15.46:0/1907120154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b01a6140 con 0x7ff8b010d8c0 2024-03-18T20:34:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.979+0000 7ff8b5114700 1 -- 172.21.15.46:0/1907120154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8b01002f0 con 0x7ff8b010d8c0 2024-03-18T20:34:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.980+0000 7ff897fff700 1 -- 172.21.15.46:0/1907120154 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff8a8005020 con 0x7ff8b010d8c0 2024-03-18T20:34:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.980+0000 7ff897fff700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff8980a11b0 0x7ff8980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.980+0000 7ff8aed9d700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff8980a11b0 0x7ff8980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.981+0000 7ff897fff700 1 -- 172.21.15.46:0/1907120154 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff8a80cc140 con 0x7ff8b010d8c0 2024-03-18T20:34:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.981+0000 7ff897fff700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215] conn(0x7ff8980aa990 0x7ff8980acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:34:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.981+0000 7ff897fff700 1 -- 172.21.15.46:0/1907120154 --> [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8980ad460 con 0x7ff8980aa990 2024-03-18T20:34:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.981+0000 7ff897fff700 1 -- 172.21.15.46:0/1907120154 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8a80ce850 con 0x7ff8b010d8c0 2024-03-18T20:34:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.981+0000 7ff8aed9d700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff8980a11b0 0x7ff8980a3670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400b5c0 tx=0x7ff8a4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:59.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.981+0000 7ff8ae59c700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215] conn(0x7ff8980aa990 0x7ff8980acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:34:59.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.981+0000 7ff8ae59c700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215] conn(0x7ff8980aa990 0x7ff8980acdb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:34:59.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.982+0000 7ff897fff700 1 -- 172.21.15.46:0/1907120154 <== osd.5 v2:172.21.15.155:6808/2840490215 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ff8980ad460 con 0x7ff8980aa990 2024-03-18T20:35:00.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:34:59.999+0000 7ff8b5114700 1 -- 172.21.15.46:0/1907120154 --> [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff8b01131a0 con 0x7ff8980aa990 2024-03-18T20:35:00.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.000+0000 7ff897fff700 1 -- 172.21.15.46:0/1907120154 <== osd.5 v2:172.21.15.155:6808/2840490215 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff8b01131a0 con 0x7ff8980aa990 2024-03-18T20:35:00.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.000+0000 7ff8b5114700 1 -- 172.21.15.46:0/1907120154 >> [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215] conn(0x7ff8980aa990 msgr2=0x7ff8980acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:00.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.000+0000 7ff8b5114700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215] conn(0x7ff8980aa990 0x7ff8980acdb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:00.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.000+0000 7ff8b5114700 1 -- 172.21.15.46:0/1907120154 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff8980a11b0 msgr2=0x7ff8980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:00.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.000+0000 7ff8b5114700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff8980a11b0 0x7ff8980a3670 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400b5c0 tx=0x7ff8a4005960 comp rx=0 tx=0).stop 2024-03-18T20:35:00.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.000+0000 7ff8b5114700 1 -- 172.21.15.46:0/1907120154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b010d8c0 msgr2=0x7ff8b019dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:00.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.000+0000 7ff8b5114700 1 --2- 172.21.15.46:0/1907120154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8b010d8c0 0x7ff8b019dcc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80094f0 tx=0x7ff8a800d3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:00.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.000+0000 7ff8af59e700 1 -- 172.21.15.46:0/1907120154 reap_dead start 2024-03-18T20:35:00.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.000+0000 7ff8b5114700 1 -- 172.21.15.46:0/1907120154 shutdown_connections 2024-03-18T20:35:00.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.000+0000 7ff8b5114700 1 -- 172.21.15.46:0/1907120154 >> 172.21.15.46:0/1907120154 conn(0x7ff8b0075d50 msgr2=0x7ff8b010f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:00.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.000+0000 7ff8b5114700 1 -- 172.21.15.46:0/1907120154 shutdown_connections 2024-03-18T20:35:00.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.000+0000 7ff8b5114700 1 -- 172.21.15.46:0/1907120154 wait complete. 2024-03-18T20:35:00.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:00 smithi046 ceph-mon[43495]: pgmap v229: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 21 op/s 2024-03-18T20:35:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:00 smithi073 ceph-mon[47384]: pgmap v229: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 21 op/s 2024-03-18T20:35:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:00 smithi155 ceph-mon[48344]: pgmap v229: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 21 op/s 2024-03-18T20:35:00.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.552+0000 7f70d76f2700 1 -- 172.21.15.46:0/3317139085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d00ff750 msgr2=0x7f70d00ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.552+0000 7f70d76f2700 1 --2- 172.21.15.46:0/3317139085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d00ff750 0x7f70d00ffbb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009990 tx=0x7f70c0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.552+0000 7f70d76f2700 1 -- 172.21.15.46:0/3317139085 shutdown_connections 2024-03-18T20:35:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.552+0000 7f70d76f2700 1 --2- 172.21.15.46:0/3317139085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d01000f0 0x7f70d010db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.552+0000 7f70d76f2700 1 --2- 172.21.15.46:0/3317139085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70d0105e90 0x7f70d0106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.552+0000 7f70d76f2700 1 --2- 172.21.15.46:0/3317139085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d00ff750 0x7f70d00ffbb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.552+0000 7f70d76f2700 1 -- 172.21.15.46:0/3317139085 >> 172.21.15.46:0/3317139085 conn(0x7f70d0074fd0 msgr2=0x7f70d00753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.552+0000 7f70d76f2700 1 -- 172.21.15.46:0/3317139085 shutdown_connections 2024-03-18T20:35:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.552+0000 7f70d76f2700 1 -- 172.21.15.46:0/3317139085 wait complete. 2024-03-18T20:35:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d76f2700 1 Processor -- start 2024-03-18T20:35:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d76f2700 1 -- start start 2024-03-18T20:35:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d76f2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d00ff750 0x7f70d0100930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d76f2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70d01000f0 0x7f70d0100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d76f2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d0105e90 0x7f70d01013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d76f2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d01a5530 con 0x7f70d00ff750 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d76f2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d01a56a0 con 0x7f70d0105e90 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d76f2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d01a5860 con 0x7f70d01000f0 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d548e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d00ff750 0x7f70d0100930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d548e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d00ff750 0x7f70d0100930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39092/0 (socket says 172.21.15.46:39092) 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d548e700 1 -- 172.21.15.46:0/3764716823 learned_addr learned my addr 172.21.15.46:0/3764716823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d5c8f700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d0105e90 0x7f70d01013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d4c8d700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70d01000f0 0x7f70d0100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d548e700 1 -- 172.21.15.46:0/3764716823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70d01000f0 msgr2=0x7f70d0100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d548e700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70d01000f0 0x7f70d0100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d548e700 1 -- 172.21.15.46:0/3764716823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d0105e90 msgr2=0x7f70d01013b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d548e700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d0105e90 0x7f70d01013b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d548e700 1 -- 172.21.15.46:0/3764716823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c00095f0 con 0x7f70d00ff750 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d4c8d700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70d01000f0 0x7f70d0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.553+0000 7f70d5c8f700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d0105e90 0x7f70d01013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.554+0000 7f70d548e700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d00ff750 0x7f70d0100930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00e970 tx=0x7f70cc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.554+0000 7f70c67fc700 1 -- 172.21.15.46:0/3764716823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc009790 con 0x7f70d00ff750 2024-03-18T20:35:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.554+0000 7f70c67fc700 1 -- 172.21.15.46:0/3764716823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70cc0044e0 con 0x7f70d00ff750 2024-03-18T20:35:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.554+0000 7f70d76f2700 1 -- 172.21.15.46:0/3764716823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d01a5b40 con 0x7f70d00ff750 2024-03-18T20:35:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.554+0000 7f70c67fc700 1 -- 172.21.15.46:0/3764716823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc003e30 con 0x7f70d00ff750 2024-03-18T20:35:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.554+0000 7f70d76f2700 1 -- 172.21.15.46:0/3764716823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d01a6090 con 0x7f70d00ff750 2024-03-18T20:35:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.555+0000 7f70d76f2700 1 -- 172.21.15.46:0/3764716823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f70d00681f0 con 0x7f70d00ff750 2024-03-18T20:35:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.556+0000 7f70c67fc700 1 -- 172.21.15.46:0/3764716823 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f70cc004000 con 0x7f70d00ff750 2024-03-18T20:35:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.557+0000 7f70c67fc700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f70bc0a1230 0x7f70bc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.557+0000 7f70d4c8d700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f70bc0a1230 0x7f70bc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.557+0000 7f70c67fc700 1 -- 172.21.15.46:0/3764716823 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f70cc014070 con 0x7f70d00ff750 2024-03-18T20:35:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.557+0000 7f70c67fc700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025] conn(0x7f70bc0aaa10 0x7f70bc0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.557+0000 7f70c67fc700 1 -- 172.21.15.46:0/3764716823 --> [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70bc0ad4e0 con 0x7f70bc0aaa10 2024-03-18T20:35:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.557+0000 7f70c67fc700 1 -- 172.21.15.46:0/3764716823 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70cc0cd020 con 0x7f70d00ff750 2024-03-18T20:35:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.557+0000 7f70d4c8d700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f70bc0a1230 0x7f70bc0a36f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f70c000b2e0 tx=0x7f70c0005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.557+0000 7f70d5c8f700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025] conn(0x7f70bc0aaa10 0x7f70bc0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.558+0000 7f70d5c8f700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025] conn(0x7f70bc0aaa10 0x7f70bc0ace30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.559+0000 7f70c67fc700 1 -- 172.21.15.46:0/3764716823 <== osd.10 v2:172.21.15.73:6824/2313090025 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f70bc0ad4e0 con 0x7f70bc0aaa10 2024-03-18T20:35:00.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.577+0000 7f70d76f2700 1 -- 172.21.15.46:0/3764716823 --> [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f70d0110f20 con 0x7f70bc0aaa10 2024-03-18T20:35:00.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.577+0000 7f70c67fc700 1 -- 172.21.15.46:0/3764716823 <== osd.10 v2:172.21.15.73:6824/2313090025 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f70d0110f20 con 0x7f70bc0aaa10 2024-03-18T20:35:00.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.578+0000 7f70d76f2700 1 -- 172.21.15.46:0/3764716823 >> [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025] conn(0x7f70bc0aaa10 msgr2=0x7f70bc0ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:00.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.578+0000 7f70d76f2700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025] conn(0x7f70bc0aaa10 0x7f70bc0ace30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:00.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.578+0000 7f70d76f2700 1 -- 172.21.15.46:0/3764716823 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f70bc0a1230 msgr2=0x7f70bc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:00.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.578+0000 7f70d76f2700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f70bc0a1230 0x7f70bc0a36f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f70c000b2e0 tx=0x7f70c0005e80 comp rx=0 tx=0).stop 2024-03-18T20:35:00.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.578+0000 7f70d76f2700 1 -- 172.21.15.46:0/3764716823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d00ff750 msgr2=0x7f70d0100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:00.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.578+0000 7f70d76f2700 1 --2- 172.21.15.46:0/3764716823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d00ff750 0x7f70d0100930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00e970 tx=0x7f70cc00ed30 comp rx=0 tx=0).stop 2024-03-18T20:35:00.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.578+0000 7f70d5c8f700 1 -- 172.21.15.46:0/3764716823 reap_dead start 2024-03-18T20:35:00.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.578+0000 7f70d76f2700 1 -- 172.21.15.46:0/3764716823 shutdown_connections 2024-03-18T20:35:00.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.578+0000 7f70d76f2700 1 -- 172.21.15.46:0/3764716823 >> 172.21.15.46:0/3764716823 conn(0x7f70d0074fd0 msgr2=0x7f70d00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:00.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.578+0000 7f70d76f2700 1 -- 172.21.15.46:0/3764716823 shutdown_connections 2024-03-18T20:35:00.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.578+0000 7f70d76f2700 1 -- 172.21.15.46:0/3764716823 wait complete. 2024-03-18T20:35:00.621 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.841+0000 7fe57d94a700 1 -- 172.21.15.46:0/882220565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe578106ef0 msgr2=0x7fe5781072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.841+0000 7fe57d94a700 1 --2- 172.21.15.46:0/882220565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe578106ef0 0x7fe5781072d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe560009920 tx=0x7fe560009c30 comp rx=0 tx=0).stop 2024-03-18T20:35:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.841+0000 7fe57d94a700 1 -- 172.21.15.46:0/882220565 shutdown_connections 2024-03-18T20:35:01.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.841+0000 7fe57d94a700 1 --2- 172.21.15.46:0/882220565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe578106ef0 0x7fe5781072d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.841+0000 7fe57d94a700 1 --2- 172.21.15.46:0/882220565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe578101820 0x7fe57810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.841+0000 7fe57d94a700 1 --2- 172.21.15.46:0/882220565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe578100e60 0x7fe5781012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.841+0000 7fe57d94a700 1 -- 172.21.15.46:0/882220565 >> 172.21.15.46:0/882220565 conn(0x7fe578077fa0 msgr2=0x7fe5780783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.841+0000 7fe57d94a700 1 -- 172.21.15.46:0/882220565 shutdown_connections 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.841+0000 7fe57d94a700 1 -- 172.21.15.46:0/882220565 wait complete. 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.842+0000 7fe57d94a700 1 Processor -- start 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.842+0000 7fe57d94a700 1 -- start start 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe57d94a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe578100e60 0x7fe5780730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe57d94a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe578101820 0x7fe5780735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe57d94a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe578106ef0 0x7fe578073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe57d94a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5781a5630 con 0x7fe578101820 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe57d94a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5781a57a0 con 0x7fe578100e60 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe57d94a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5781a5960 con 0x7fe578106ef0 2024-03-18T20:35:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe5767fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe578101820 0x7fe5780735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe5767fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe578101820 0x7fe5780735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39108/0 (socket says 172.21.15.46:39108) 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe5767fc700 1 -- 172.21.15.46:0/1880295223 learned_addr learned my addr 172.21.15.46:0/1880295223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe576ffd700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe578100e60 0x7fe5780730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe5777fe700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe578106ef0 0x7fe578073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe5767fc700 1 -- 172.21.15.46:0/1880295223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe578106ef0 msgr2=0x7fe578073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe5767fc700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe578106ef0 0x7fe578073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe5767fc700 1 -- 172.21.15.46:0/1880295223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe578100e60 msgr2=0x7fe5780730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe5767fc700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe578100e60 0x7fe5780730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe5767fc700 1 -- 172.21.15.46:0/1880295223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe570009d00 con 0x7fe578101820 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe576ffd700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe578100e60 0x7fe5780730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe5777fe700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe578106ef0 0x7fe578073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe5767fc700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe578101820 0x7fe5780735f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe56800e9a0 tx=0x7fe56800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:01.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe57c948700 1 -- 172.21.15.46:0/1880295223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568009790 con 0x7fe578101820 2024-03-18T20:35:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe57c948700 1 -- 172.21.15.46:0/1880295223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe56800cbd0 con 0x7fe578101820 2024-03-18T20:35:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe57d94a700 1 -- 172.21.15.46:0/1880295223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe560009520 con 0x7fe578101820 2024-03-18T20:35:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.843+0000 7fe57c948700 1 -- 172.21.15.46:0/1880295223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568018680 con 0x7fe578101820 2024-03-18T20:35:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.844+0000 7fe57d94a700 1 -- 172.21.15.46:0/1880295223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5781a6000 con 0x7fe578101820 2024-03-18T20:35:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.844+0000 7fe57d94a700 1 -- 172.21.15.46:0/1880295223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe578110fc0 con 0x7fe578101820 2024-03-18T20:35:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.846+0000 7fe57c948700 1 -- 172.21.15.46:0/1880295223 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fe56800cd40 con 0x7fe578101820 2024-03-18T20:35:01.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.846+0000 7fe57c948700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe5640a1230 0x7fe5640a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.847+0000 7fe57c948700 1 -- 172.21.15.46:0/1880295223 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe568014070 con 0x7fe578101820 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.847+0000 7fe576ffd700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe5640a1230 0x7fe5640a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.847+0000 7fe57c948700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640] conn(0x7fe5640aaa10 0x7fe5640ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.847+0000 7fe57c948700 1 -- 172.21.15.46:0/1880295223 --> [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5640ad4e0 con 0x7fe5640aaa10 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.847+0000 7fe57c948700 1 -- 172.21.15.46:0/1880295223 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5680cd610 con 0x7fe578101820 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.847+0000 7fe5777fe700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640] conn(0x7fe5640aaa10 0x7fe5640ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.847+0000 7fe576ffd700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe5640a1230 0x7fe5640a36f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe56000c010 tx=0x7fe560019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.847+0000 7fe5777fe700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640] conn(0x7fe5640aaa10 0x7fe5640ace30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.848+0000 7fe57c948700 1 -- 172.21.15.46:0/1880295223 <== osd.6 v2:172.21.15.46:6818/2670405640 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fe5640ad4e0 con 0x7fe5640aaa10 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.872+0000 7fe57d94a700 1 -- 172.21.15.46:0/1880295223 --> [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe578061430 con 0x7fe5640aaa10 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.873+0000 7fe57c948700 1 -- 172.21.15.46:0/1880295223 <== osd.6 v2:172.21.15.46:6818/2670405640 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe578061430 con 0x7fe5640aaa10 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.873+0000 7fe57d94a700 1 -- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640] conn(0x7fe5640aaa10 msgr2=0x7fe5640ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.873+0000 7fe57d94a700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640] conn(0x7fe5640aaa10 0x7fe5640ace30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.873+0000 7fe57d94a700 1 -- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe5640a1230 msgr2=0x7fe5640a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:01.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.873+0000 7fe57d94a700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe5640a1230 0x7fe5640a36f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe56000c010 tx=0x7fe560019040 comp rx=0 tx=0).stop 2024-03-18T20:35:01.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.873+0000 7fe57d94a700 1 -- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe578101820 msgr2=0x7fe5780735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:01.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.873+0000 7fe57d94a700 1 --2- 172.21.15.46:0/1880295223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe578101820 0x7fe5780735f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe56800e9a0 tx=0x7fe56800ed60 comp rx=0 tx=0).stop 2024-03-18T20:35:01.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.873+0000 7fe5777fe700 1 -- 172.21.15.46:0/1880295223 reap_dead start 2024-03-18T20:35:01.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.873+0000 7fe57d94a700 1 -- 172.21.15.46:0/1880295223 shutdown_connections 2024-03-18T20:35:01.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.873+0000 7fe57d94a700 1 -- 172.21.15.46:0/1880295223 >> 172.21.15.46:0/1880295223 conn(0x7fe578077fa0 msgr2=0x7fe5780ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:01.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.874+0000 7fe57d94a700 1 -- 172.21.15.46:0/1880295223 shutdown_connections 2024-03-18T20:35:01.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:00.874+0000 7fe57d94a700 1 -- 172.21.15.46:0/1880295223 wait complete. 2024-03-18T20:35:01.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.954+0000 7fb0341ef700 1 -- 172.21.15.46:0/464055827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c1000f0 msgr2=0x7fb02c10db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.954+0000 7fb0341ef700 1 --2- 172.21.15.46:0/464055827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c1000f0 0x7fb02c10db80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b0a0 tx=0x7fb02400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.954+0000 7fb0341ef700 1 -- 172.21.15.46:0/464055827 shutdown_connections 2024-03-18T20:35:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.954+0000 7fb0341ef700 1 --2- 172.21.15.46:0/464055827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c1000f0 0x7fb02c10db80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.954+0000 7fb0341ef700 1 --2- 172.21.15.46:0/464055827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb02c105e90 0x7fb02c106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.954+0000 7fb0341ef700 1 --2- 172.21.15.46:0/464055827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb02c0ff750 0x7fb02c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.954+0000 7fb0341ef700 1 -- 172.21.15.46:0/464055827 >> 172.21.15.46:0/464055827 conn(0x7fb02c074fd0 msgr2=0x7fb02c0753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.954+0000 7fb0341ef700 1 -- 172.21.15.46:0/464055827 shutdown_connections 2024-03-18T20:35:01.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.954+0000 7fb0341ef700 1 -- 172.21.15.46:0/464055827 wait complete. 2024-03-18T20:35:01.960 INFO:teuthology.orchestra.run.smithi046.stdout:150323855395 2024-03-18T20:35:01.961 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-18T20:35:01.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.955+0000 7fb0341ef700 1 Processor -- start 2024-03-18T20:35:01.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.955+0000 7fb0341ef700 1 -- start start 2024-03-18T20:35:01.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb0341ef700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb02c0ff750 0x7fb02c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:01.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb0341ef700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb02c1000f0 0x7fb02c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:01.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb0341ef700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c105e90 0x7fb02c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb0341ef700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c19c190 con 0x7fb02c105e90 2024-03-18T20:35:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb0341ef700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c19c300 con 0x7fb02c1000f0 2024-03-18T20:35:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb0341ef700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c19c4c0 con 0x7fb02c0ff750 2024-03-18T20:35:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb03278c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c105e90 0x7fb02c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb03278c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c105e90 0x7fb02c19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39118/0 (socket says 172.21.15.46:39118) 2024-03-18T20:35:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb03278c700 1 -- 172.21.15.46:0/211190184 learned_addr learned my addr 172.21.15.46:0/211190184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb03178a700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb02c1000f0 0x7fb02c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb031f8b700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb02c0ff750 0x7fb02c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb03278c700 1 -- 172.21.15.46:0/211190184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb02c0ff750 msgr2=0x7fb02c1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb03278c700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb02c0ff750 0x7fb02c1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.956+0000 7fb03278c700 1 -- 172.21.15.46:0/211190184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb02c1000f0 msgr2=0x7fb02c197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.957+0000 7fb03278c700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb02c1000f0 0x7fb02c197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.957+0000 7fb03278c700 1 -- 172.21.15.46:0/211190184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb024009d00 con 0x7fb02c105e90 2024-03-18T20:35:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.957+0000 7fb03278c700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c105e90 0x7fb02c19bb80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb024006070 tx=0x7fb024007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.957+0000 7fb022ffd700 1 -- 172.21.15.46:0/211190184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02400e040 con 0x7fb02c105e90 2024-03-18T20:35:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.957+0000 7fb0341ef700 1 -- 172.21.15.46:0/211190184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c19c740 con 0x7fb02c105e90 2024-03-18T20:35:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.957+0000 7fb022ffd700 1 -- 172.21.15.46:0/211190184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb02401d470 con 0x7fb02c105e90 2024-03-18T20:35:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.957+0000 7fb0341ef700 1 -- 172.21.15.46:0/211190184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c1a4000 con 0x7fb02c105e90 2024-03-18T20:35:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.958+0000 7fb022ffd700 1 -- 172.21.15.46:0/211190184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02401b6c0 con 0x7fb02c105e90 2024-03-18T20:35:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.959+0000 7fb0341ef700 1 -- 172.21.15.46:0/211190184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb010000fc0 con 0x7fb02c105e90 2024-03-18T20:35:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.959+0000 7fb022ffd700 1 -- 172.21.15.46:0/211190184 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb02401b820 con 0x7fb02c105e90 2024-03-18T20:35:01.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.960+0000 7fb022ffd700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0180a11b0 0x7fb0180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:01.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.960+0000 7fb031f8b700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0180a11b0 0x7fb0180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:01.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.960+0000 7fb022ffd700 1 -- 172.21.15.46:0/211190184 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb0240cd630 con 0x7fb02c105e90 2024-03-18T20:35:01.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.960+0000 7fb022ffd700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456] conn(0x7fb0180aa990 0x7fb0180acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:01.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.960+0000 7fb022ffd700 1 -- 172.21.15.46:0/211190184 --> [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0180ad460 con 0x7fb0180aa990 2024-03-18T20:35:01.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.960+0000 7fb022ffd700 1 -- 172.21.15.46:0/211190184 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0240cfd40 con 0x7fb02c105e90 2024-03-18T20:35:01.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.961+0000 7fb03178a700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456] conn(0x7fb0180aa990 0x7fb0180acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:01.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.961+0000 7fb031f8b700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0180a11b0 0x7fb0180a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb02c068720 tx=0x7fb028007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:01.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.961+0000 7fb03178a700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456] conn(0x7fb0180aa990 0x7fb0180acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:01.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.962+0000 7fb022ffd700 1 -- 172.21.15.46:0/211190184 <== osd.3 v2:172.21.15.46:6810/1070659456 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb0180ad460 con 0x7fb0180aa990 2024-03-18T20:35:01.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.979+0000 7fb0341ef700 1 -- 172.21.15.46:0/211190184 --> [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb010002d40 con 0x7fb0180aa990 2024-03-18T20:35:01.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.980+0000 7fb022ffd700 1 -- 172.21.15.46:0/211190184 <== osd.3 v2:172.21.15.46:6810/1070659456 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb010002d40 con 0x7fb0180aa990 2024-03-18T20:35:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.981+0000 7fb0341ef700 1 -- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456] conn(0x7fb0180aa990 msgr2=0x7fb0180acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.981+0000 7fb0341ef700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456] conn(0x7fb0180aa990 0x7fb0180acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.981+0000 7fb0341ef700 1 -- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0180a11b0 msgr2=0x7fb0180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.981+0000 7fb0341ef700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0180a11b0 0x7fb0180a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb02c068720 tx=0x7fb028007400 comp rx=0 tx=0).stop 2024-03-18T20:35:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.981+0000 7fb0341ef700 1 -- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c105e90 msgr2=0x7fb02c19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.981+0000 7fb0341ef700 1 --2- 172.21.15.46:0/211190184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c105e90 0x7fb02c19bb80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb024006070 tx=0x7fb024007930 comp rx=0 tx=0).stop 2024-03-18T20:35:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.981+0000 7fb03278c700 1 -- 172.21.15.46:0/211190184 reap_dead start 2024-03-18T20:35:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.981+0000 7fb0341ef700 1 -- 172.21.15.46:0/211190184 shutdown_connections 2024-03-18T20:35:01.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.981+0000 7fb0341ef700 1 -- 172.21.15.46:0/211190184 >> 172.21.15.46:0/211190184 conn(0x7fb02c074fd0 msgr2=0x7fb02c0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:01.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.981+0000 7fb0341ef700 1 -- 172.21.15.46:0/211190184 shutdown_connections 2024-03-18T20:35:01.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:01.981+0000 7fb0341ef700 1 -- 172.21.15.46:0/211190184 wait complete. 2024-03-18T20:35:02.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:02 smithi046 ceph-mon[43495]: pgmap v230: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-03-18T20:35:02.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.447+0000 7f6a05be0700 1 -- 172.21.15.46:0/2524412431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a00100660 msgr2=0x7f6a0010cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:02.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.447+0000 7f6a05be0700 1 --2- 172.21.15.46:0/2524412431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a00100660 0x7f6a0010cb00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f69f800b0a0 tx=0x7f69f800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:02.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.447+0000 7f6a05be0700 1 -- 172.21.15.46:0/2524412431 shutdown_connections 2024-03-18T20:35:02.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.447+0000 7f6a05be0700 1 --2- 172.21.15.46:0/2524412431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a00100660 0x7f6a0010cb00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:02.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.447+0000 7f6a05be0700 1 --2- 172.21.15.46:0/2524412431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00105d50 0x7f6a00106130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:02.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.447+0000 7f6a05be0700 1 --2- 172.21.15.46:0/2524412431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000ffca0 0x7f6a00100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:02.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.447+0000 7f6a05be0700 1 -- 172.21.15.46:0/2524412431 >> 172.21.15.46:0/2524412431 conn(0x7f6a00076c80 msgr2=0x7f6a00077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:02.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.447+0000 7f6a05be0700 1 -- 172.21.15.46:0/2524412431 shutdown_connections 2024-03-18T20:35:02.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.447+0000 7f6a05be0700 1 -- 172.21.15.46:0/2524412431 wait complete. 2024-03-18T20:35:02.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f6a05be0700 1 Processor -- start 2024-03-18T20:35:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f6a05be0700 1 -- start start 2024-03-18T20:35:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f6a05be0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000ffca0 0x7f6a00101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f6a05be0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00100660 0x7f6a00101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f6a05be0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a00105d50 0x7f6a001023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f6a05be0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a001a5690 con 0x7f6a00105d50 2024-03-18T20:35:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f6a05be0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a001a5800 con 0x7f6a00100660 2024-03-18T20:35:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f6a05be0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a001a59c0 con 0x7f6a000ffca0 2024-03-18T20:35:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f69fffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a00105d50 0x7f6a001023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f69fffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a00105d50 0x7f6a001023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39138/0 (socket says 172.21.15.46:39138) 2024-03-18T20:35:02.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f69fffff700 1 -- 172.21.15.46:0/2028990346 learned_addr learned my addr 172.21.15.46:0/2028990346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f69f7fff700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00100660 0x7f6a00101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f69ff7fe700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000ffca0 0x7f6a00101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f69fffff700 1 -- 172.21.15.46:0/2028990346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000ffca0 msgr2=0x7f6a00101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f69fffff700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a000ffca0 0x7f6a00101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f69fffff700 1 -- 172.21.15.46:0/2028990346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00100660 msgr2=0x7f6a00101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f69fffff700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00100660 0x7f6a00101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.448+0000 7f69fffff700 1 -- 172.21.15.46:0/2028990346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f8009d00 con 0x7f6a00105d50 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.449+0000 7f69fffff700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a00105d50 0x7f6a001023d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f69f800b070 tx=0x7f69f80078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.449+0000 7f69fd7fa700 1 -- 172.21.15.46:0/2028990346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f800e040 con 0x7f6a00105d50 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.449+0000 7f6a05be0700 1 -- 172.21.15.46:0/2028990346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a001a5c40 con 0x7f6a00105d50 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.449+0000 7f69fd7fa700 1 -- 172.21.15.46:0/2028990346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69f801d470 con 0x7f6a00105d50 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.449+0000 7f69fd7fa700 1 -- 172.21.15.46:0/2028990346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f80052a0 con 0x7f6a00105d50 2024-03-18T20:35:02.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.449+0000 7f6a05be0700 1 -- 172.21.15.46:0/2028990346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a001a6100 con 0x7f6a00105d50 2024-03-18T20:35:02.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.450+0000 7f6a05be0700 1 -- 172.21.15.46:0/2028990346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a0010fe30 con 0x7f6a00105d50 2024-03-18T20:35:02.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.452+0000 7f69fd7fa700 1 -- 172.21.15.46:0/2028990346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f69f801d5e0 con 0x7f6a00105d50 2024-03-18T20:35:02.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.453+0000 7f69fd7fa700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f69ec0a11b0 0x7f69ec0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:02.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.453+0000 7f69fd7fa700 1 -- 172.21.15.46:0/2028990346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f69f80ce560 con 0x7f6a00105d50 2024-03-18T20:35:02.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.453+0000 7f69ff7fe700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f69ec0a11b0 0x7f69ec0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:02.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.453+0000 7f69fd7fa700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015] conn(0x7f69ec0aa990 0x7f69ec0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:02.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.453+0000 7f69fd7fa700 1 -- 172.21.15.46:0/2028990346 --> [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69ec0ad460 con 0x7f69ec0aa990 2024-03-18T20:35:02.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.453+0000 7f69fd7fa700 1 -- 172.21.15.46:0/2028990346 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69f80cea00 con 0x7f6a00105d50 2024-03-18T20:35:02.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.454+0000 7f69f7fff700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015] conn(0x7f69ec0aa990 0x7f69ec0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:02.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.454+0000 7f69ff7fe700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f69ec0a11b0 0x7f69ec0a3670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009bd0 tx=0x7f69f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:02.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.454+0000 7f69f7fff700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015] conn(0x7f69ec0aa990 0x7f69ec0acdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:02.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.455+0000 7f69fd7fa700 1 -- 172.21.15.46:0/2028990346 <== osd.9 v2:172.21.15.46:6826/167112015 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f69ec0ad460 con 0x7f69ec0aa990 2024-03-18T20:35:02.459 INFO:teuthology.orchestra.run.smithi046.stdout:283467841553 2024-03-18T20:35:02.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-18T20:35:02.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:02 smithi073 ceph-mon[47384]: pgmap v230: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-03-18T20:35:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.471+0000 7f6a05be0700 1 -- 172.21.15.46:0/2028990346 --> [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6a00101390 con 0x7f69ec0aa990 2024-03-18T20:35:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.472+0000 7f69fd7fa700 1 -- 172.21.15.46:0/2028990346 <== osd.9 v2:172.21.15.46:6826/167112015 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6a00101390 con 0x7f69ec0aa990 2024-03-18T20:35:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.472+0000 7f6a05be0700 1 -- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015] conn(0x7f69ec0aa990 msgr2=0x7f69ec0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.472+0000 7f6a05be0700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015] conn(0x7f69ec0aa990 0x7f69ec0acdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.472+0000 7f6a05be0700 1 -- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f69ec0a11b0 msgr2=0x7f69ec0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.472+0000 7f6a05be0700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f69ec0a11b0 0x7f69ec0a3670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f69f0009bd0 tx=0x7f69f0009260 comp rx=0 tx=0).stop 2024-03-18T20:35:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.472+0000 7f6a05be0700 1 -- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a00105d50 msgr2=0x7f6a001023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.472+0000 7f6a05be0700 1 --2- 172.21.15.46:0/2028990346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a00105d50 0x7f6a001023d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f69f800b070 tx=0x7f69f80078e0 comp rx=0 tx=0).stop 2024-03-18T20:35:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.472+0000 7f69fffff700 1 -- 172.21.15.46:0/2028990346 reap_dead start 2024-03-18T20:35:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.472+0000 7f6a05be0700 1 -- 172.21.15.46:0/2028990346 shutdown_connections 2024-03-18T20:35:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.472+0000 7f6a05be0700 1 -- 172.21.15.46:0/2028990346 >> 172.21.15.46:0/2028990346 conn(0x7f6a00076c80 msgr2=0x7f6a000fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.473+0000 7f6a05be0700 1 -- 172.21.15.46:0/2028990346 shutdown_connections 2024-03-18T20:35:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.473+0000 7f6a05be0700 1 -- 172.21.15.46:0/2028990346 wait complete. 2024-03-18T20:35:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:02 smithi155 ceph-mon[48344]: pgmap v230: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-03-18T20:35:02.633 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:02.922 INFO:teuthology.orchestra.run.smithi046.stdout:180388626463 2024-03-18T20:35:02.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-18T20:35:02.986 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:03.222 INFO:teuthology.orchestra.run.smithi046.stdout:103079215145 2024-03-18T20:35:03.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-18T20:35:03.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.935+0000 7f9ca34d0700 1 -- 172.21.15.46:0/3742413945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c9c069440 msgr2=0x7f9c9c069820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.935+0000 7f9ca34d0700 1 --2- 172.21.15.46:0/3742413945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c9c069440 0x7f9c9c069820 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009990 tx=0x7f9c98009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.935+0000 7f9ca34d0700 1 -- 172.21.15.46:0/3742413945 shutdown_connections 2024-03-18T20:35:03.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.935+0000 7f9ca34d0700 1 --2- 172.21.15.46:0/3742413945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c9c069440 0x7f9c9c069820 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.935+0000 7f9ca34d0700 1 --2- 172.21.15.46:0/3742413945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c9c10d8c0 0x7f9c9c10fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.935+0000 7f9ca34d0700 1 --2- 172.21.15.46:0/3742413945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c9c069d60 0x7f9c9c10d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.935+0000 7f9ca34d0700 1 -- 172.21.15.46:0/3742413945 >> 172.21.15.46:0/3742413945 conn(0x7f9c9c075d90 msgr2=0x7f9c9c0761a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.935+0000 7f9ca34d0700 1 -- 172.21.15.46:0/3742413945 shutdown_connections 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.935+0000 7f9ca34d0700 1 -- 172.21.15.46:0/3742413945 wait complete. 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca34d0700 1 Processor -- start 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca34d0700 1 -- start start 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca34d0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c9c069440 0x7f9c9c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca34d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c9c069d60 0x7f9c9c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca34d0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c9c10d8c0 0x7f9c9c19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca34d0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c19e2c0 con 0x7f9c9c10d8c0 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca34d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c19e430 con 0x7f9c9c069d60 2024-03-18T20:35:03.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca34d0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c19e5f0 con 0x7f9c9c069440 2024-03-18T20:35:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca1a6d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c9c10d8c0 0x7f9c9c19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca126c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c9c069440 0x7f9c9c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca1a6d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c9c10d8c0 0x7f9c9c19dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39168/0 (socket says 172.21.15.46:39168) 2024-03-18T20:35:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca1a6d700 1 -- 172.21.15.46:0/2979100597 learned_addr learned my addr 172.21.15.46:0/2979100597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.936+0000 7f9ca0a6b700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c9c069d60 0x7f9c9c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.937+0000 7f9ca1a6d700 1 -- 172.21.15.46:0/2979100597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c9c069440 msgr2=0x7f9c9c199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.937+0000 7f9ca1a6d700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c9c069440 0x7f9c9c199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.937+0000 7f9ca1a6d700 1 -- 172.21.15.46:0/2979100597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c9c069d60 msgr2=0x7f9c9c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.937+0000 7f9ca1a6d700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c9c069d60 0x7f9c9c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.937+0000 7f9ca1a6d700 1 -- 172.21.15.46:0/2979100597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c980095f0 con 0x7f9c9c10d8c0 2024-03-18T20:35:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.937+0000 7f9ca126c700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c9c069440 0x7f9c9c199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:03.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.937+0000 7f9ca1a6d700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c9c10d8c0 0x7f9c9c19dcb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9c940094f0 tx=0x7f9c9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.937+0000 7f9c927fc700 1 -- 172.21.15.46:0/2979100597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94004db0 con 0x7f9c9c10d8c0 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.937+0000 7f9c927fc700 1 -- 172.21.15.46:0/2979100597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c9400f040 con 0x7f9c9c10d8c0 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.937+0000 7f9ca34d0700 1 -- 172.21.15.46:0/2979100597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9c19e8d0 con 0x7f9c9c10d8c0 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.937+0000 7f9c927fc700 1 -- 172.21.15.46:0/2979100597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94005980 con 0x7f9c9c10d8c0 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.937+0000 7f9ca34d0700 1 -- 172.21.15.46:0/2979100597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9c1a6130 con 0x7f9c9c10d8c0 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.938+0000 7f9ca34d0700 1 -- 172.21.15.46:0/2979100597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c9c1002b0 con 0x7f9c9c10d8c0 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.939+0000 7f9c927fc700 1 -- 172.21.15.46:0/2979100597 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f9c94005000 con 0x7f9c9c10d8c0 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.940+0000 7f9c927fc700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9c880a1230 0x7f9c880a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.940+0000 7f9ca126c700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9c880a1230 0x7f9c880a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.940+0000 7f9c927fc700 1 -- 172.21.15.46:0/2979100597 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9c940ccc20 con 0x7f9c9c10d8c0 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.940+0000 7f9c927fc700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462] conn(0x7f9c880aaa10 0x7f9c880ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.941+0000 7f9c927fc700 1 -- 172.21.15.46:0/2979100597 --> [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c880ad4e0 con 0x7f9c880aaa10 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.941+0000 7f9c927fc700 1 -- 172.21.15.46:0/2979100597 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c940cd0c0 con 0x7f9c9c10d8c0 2024-03-18T20:35:03.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.941+0000 7f9ca0a6b700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462] conn(0x7f9c880aaa10 0x7f9c880ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.942+0000 7f9ca126c700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9c880a1230 0x7f9c880a36f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800b2e0 tx=0x7f9c98005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.942+0000 7f9ca0a6b700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462] conn(0x7f9c880aaa10 0x7f9c880ace30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.942+0000 7f9c927fc700 1 -- 172.21.15.46:0/2979100597 <== osd.8 v2:172.21.15.155:6816/962631462 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f9c880ad4e0 con 0x7f9c880aaa10 2024-03-18T20:35:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.960+0000 7f9ca34d0700 1 -- 172.21.15.46:0/2979100597 --> [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9c9c1131c0 con 0x7f9c880aaa10 2024-03-18T20:35:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.960+0000 7f9c927fc700 1 -- 172.21.15.46:0/2979100597 <== osd.8 v2:172.21.15.155:6816/962631462 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9c9c1131c0 con 0x7f9c880aaa10 2024-03-18T20:35:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.961+0000 7f9ca34d0700 1 -- 172.21.15.46:0/2979100597 >> [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462] conn(0x7f9c880aaa10 msgr2=0x7f9c880ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.961+0000 7f9ca34d0700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462] conn(0x7f9c880aaa10 0x7f9c880ace30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.961+0000 7f9ca34d0700 1 -- 172.21.15.46:0/2979100597 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9c880a1230 msgr2=0x7f9c880a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.961+0000 7f9ca34d0700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9c880a1230 0x7f9c880a36f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800b2e0 tx=0x7f9c98005fd0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.961+0000 7f9ca34d0700 1 -- 172.21.15.46:0/2979100597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c9c10d8c0 msgr2=0x7f9c9c19dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.961+0000 7f9ca34d0700 1 --2- 172.21.15.46:0/2979100597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c9c10d8c0 0x7f9c9c19dcb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9c940094f0 tx=0x7f9c9400d3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.961+0000 7f9ca1a6d700 1 -- 172.21.15.46:0/2979100597 reap_dead start 2024-03-18T20:35:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.961+0000 7f9ca34d0700 1 -- 172.21.15.46:0/2979100597 shutdown_connections 2024-03-18T20:35:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.961+0000 7f9ca34d0700 1 -- 172.21.15.46:0/2979100597 >> 172.21.15.46:0/2979100597 conn(0x7f9c9c075d90 msgr2=0x7f9c9c10ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.961+0000 7f9ca34d0700 1 -- 172.21.15.46:0/2979100597 shutdown_connections 2024-03-18T20:35:03.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:02.961+0000 7f9ca34d0700 1 -- 172.21.15.46:0/2979100597 wait complete. 2024-03-18T20:35:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:03.681 INFO:teuthology.orchestra.run.smithi046.stdout:231928234009 2024-03-18T20:35:03.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-18T20:35:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.387+0000 7f1a815ba700 1 -- 172.21.15.46:0/2141958491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a7c073900 msgr2=0x7f1a7c10d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.387+0000 7f1a815ba700 1 --2- 172.21.15.46:0/2141958491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a7c073900 0x7f1a7c10d300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009990 tx=0x7f1a70009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.387+0000 7f1a815ba700 1 -- 172.21.15.46:0/2141958491 shutdown_connections 2024-03-18T20:35:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.387+0000 7f1a815ba700 1 --2- 172.21.15.46:0/2141958491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a7c10d840 0x7f1a7c10fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.387+0000 7f1a815ba700 1 --2- 172.21.15.46:0/2141958491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a7c072fe0 0x7f1a7c0733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.387+0000 7f1a815ba700 1 --2- 172.21.15.46:0/2141958491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a7c073900 0x7f1a7c10d300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.387+0000 7f1a815ba700 1 -- 172.21.15.46:0/2141958491 >> 172.21.15.46:0/2141958491 conn(0x7f1a7c077ed0 msgr2=0x7f1a7c0782e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.387+0000 7f1a815ba700 1 -- 172.21.15.46:0/2141958491 shutdown_connections 2024-03-18T20:35:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.387+0000 7f1a815ba700 1 -- 172.21.15.46:0/2141958491 wait complete. 2024-03-18T20:35:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.388+0000 7f1a815ba700 1 Processor -- start 2024-03-18T20:35:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a815ba700 1 -- start start 2024-03-18T20:35:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a815ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a7c072fe0 0x7f1a7c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a815ba700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a7c073900 0x7f1a7c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a815ba700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a7c10d840 0x7f1a7c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a815ba700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c19e200 con 0x7f1a7c072fe0 2024-03-18T20:35:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a815ba700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c19e370 con 0x7f1a7c10d840 2024-03-18T20:35:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a815ba700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c19e530 con 0x7f1a7c073900 2024-03-18T20:35:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a7affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a7c072fe0 0x7f1a7c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a7affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a7c072fe0 0x7f1a7c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39184/0 (socket says 172.21.15.46:39184) 2024-03-18T20:35:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a7affd700 1 -- 172.21.15.46:0/2977190482 learned_addr learned my addr 172.21.15.46:0/2977190482 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a7a7fc700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a7c073900 0x7f1a7c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a7b7fe700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a7c10d840 0x7f1a7c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a7affd700 1 -- 172.21.15.46:0/2977190482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a7c073900 msgr2=0x7f1a7c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a7affd700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a7c073900 0x7f1a7c199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a7affd700 1 -- 172.21.15.46:0/2977190482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a7c10d840 msgr2=0x7f1a7c19dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a7affd700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a7c10d840 0x7f1a7c19dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.389+0000 7f1a7affd700 1 -- 172.21.15.46:0/2977190482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a700095f0 con 0x7f1a7c072fe0 2024-03-18T20:35:03.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.390+0000 7f1a7affd700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a7c072fe0 0x7f1a7c199300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00e9a0 tx=0x7f1a6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:03.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.390+0000 7f1a63fff700 1 -- 172.21.15.46:0/2977190482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c009790 con 0x7f1a7c072fe0 2024-03-18T20:35:03.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.390+0000 7f1a63fff700 1 -- 172.21.15.46:0/2977190482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a6c00cbd0 con 0x7f1a7c072fe0 2024-03-18T20:35:03.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.390+0000 7f1a815ba700 1 -- 172.21.15.46:0/2977190482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7c19e810 con 0x7f1a7c072fe0 2024-03-18T20:35:03.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.390+0000 7f1a63fff700 1 -- 172.21.15.46:0/2977190482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c018680 con 0x7f1a7c072fe0 2024-03-18T20:35:03.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.390+0000 7f1a815ba700 1 -- 172.21.15.46:0/2977190482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7c1a6070 con 0x7f1a7c072fe0 2024-03-18T20:35:03.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.391+0000 7f1a815ba700 1 -- 172.21.15.46:0/2977190482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a7c074600 con 0x7f1a7c072fe0 2024-03-18T20:35:03.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.393+0000 7f1a63fff700 1 -- 172.21.15.46:0/2977190482 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1a6c00cd40 con 0x7f1a7c072fe0 2024-03-18T20:35:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.394+0000 7f1a63fff700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1a640a11b0 0x7f1a640a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.394+0000 7f1a63fff700 1 -- 172.21.15.46:0/2977190482 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1a6c014070 con 0x7f1a7c072fe0 2024-03-18T20:35:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.394+0000 7f1a7a7fc700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1a640a11b0 0x7f1a640a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.394+0000 7f1a63fff700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822] conn(0x7f1a640aa990 0x7f1a640acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.394+0000 7f1a63fff700 1 -- 172.21.15.46:0/2977190482 --> [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a640ad460 con 0x7f1a640aa990 2024-03-18T20:35:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.394+0000 7f1a63fff700 1 -- 172.21.15.46:0/2977190482 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a6c0cd3a0 con 0x7f1a7c072fe0 2024-03-18T20:35:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.395+0000 7f1a7b7fe700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822] conn(0x7f1a640aa990 0x7f1a640acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.395+0000 7f1a7a7fc700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1a640a11b0 0x7f1a640a3670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000b2e0 tx=0x7f1a70005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.395+0000 7f1a7b7fe700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822] conn(0x7f1a640aa990 0x7f1a640acdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.396+0000 7f1a63fff700 1 -- 172.21.15.46:0/2977190482 <== osd.1 v2:172.21.15.73:6800/1334268822 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f1a640ad460 con 0x7f1a640aa990 2024-03-18T20:35:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.414+0000 7f1a815ba700 1 -- 172.21.15.46:0/2977190482 --> [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1a7c113110 con 0x7f1a640aa990 2024-03-18T20:35:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.414+0000 7f1a63fff700 1 -- 172.21.15.46:0/2977190482 <== osd.1 v2:172.21.15.73:6800/1334268822 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1a7c113110 con 0x7f1a640aa990 2024-03-18T20:35:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.415+0000 7f1a815ba700 1 -- 172.21.15.46:0/2977190482 >> [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822] conn(0x7f1a640aa990 msgr2=0x7f1a640acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.415+0000 7f1a815ba700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822] conn(0x7f1a640aa990 0x7f1a640acdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.415+0000 7f1a815ba700 1 -- 172.21.15.46:0/2977190482 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1a640a11b0 msgr2=0x7f1a640a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.415+0000 7f1a815ba700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1a640a11b0 0x7f1a640a3670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000b2e0 tx=0x7f1a70005960 comp rx=0 tx=0).stop 2024-03-18T20:35:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.415+0000 7f1a815ba700 1 -- 172.21.15.46:0/2977190482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a7c072fe0 msgr2=0x7f1a7c199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.415+0000 7f1a815ba700 1 --2- 172.21.15.46:0/2977190482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a7c072fe0 0x7f1a7c199300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00e9a0 tx=0x7f1a6c00ecb0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.415+0000 7f1a7b7fe700 1 -- 172.21.15.46:0/2977190482 reap_dead start 2024-03-18T20:35:03.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.415+0000 7f1a815ba700 1 -- 172.21.15.46:0/2977190482 shutdown_connections 2024-03-18T20:35:03.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.415+0000 7f1a815ba700 1 -- 172.21.15.46:0/2977190482 >> 172.21.15.46:0/2977190482 conn(0x7f1a7c077ed0 msgr2=0x7f1a7c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:03.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.417+0000 7f1a815ba700 1 -- 172.21.15.46:0/2977190482 shutdown_connections 2024-03-18T20:35:03.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.417+0000 7f1a815ba700 1 -- 172.21.15.46:0/2977190482 wait complete. 2024-03-18T20:35:03.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.794+0000 7efd93d6d700 1 -- 172.21.15.46:0/3464496420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8c106e40 msgr2=0x7efd8c107220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.794+0000 7efd93d6d700 1 --2- 172.21.15.46:0/3464496420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8c106e40 0x7efd8c107220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efd88009990 tx=0x7efd88009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.794+0000 7efd93d6d700 1 -- 172.21.15.46:0/3464496420 shutdown_connections 2024-03-18T20:35:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.794+0000 7efd93d6d700 1 --2- 172.21.15.46:0/3464496420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8c106e40 0x7efd8c107220 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.794+0000 7efd93d6d700 1 --2- 172.21.15.46:0/3464496420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c1017c0 0x7efd8c10dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.794+0000 7efd93d6d700 1 --2- 172.21.15.46:0/3464496420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd8c100e00 0x7efd8c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.794+0000 7efd93d6d700 1 -- 172.21.15.46:0/3464496420 >> 172.21.15.46:0/3464496420 conn(0x7efd8c077f10 msgr2=0x7efd8c078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.794+0000 7efd93d6d700 1 -- 172.21.15.46:0/3464496420 shutdown_connections 2024-03-18T20:35:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.794+0000 7efd93d6d700 1 -- 172.21.15.46:0/3464496420 wait complete. 2024-03-18T20:35:03.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd93d6d700 1 Processor -- start 2024-03-18T20:35:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd93d6d700 1 -- start start 2024-03-18T20:35:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd93d6d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd8c100e00 0x7efd8c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd93d6d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c1017c0 0x7efd8c197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd93d6d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8c106e40 0x7efd8c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd93d6d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c19c010 con 0x7efd8c106e40 2024-03-18T20:35:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd93d6d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c19c180 con 0x7efd8c100e00 2024-03-18T20:35:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd93d6d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c19c340 con 0x7efd8c1017c0 2024-03-18T20:35:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd9230a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8c106e40 0x7efd8c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd9230a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8c106e40 0x7efd8c19ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39224/0 (socket says 172.21.15.46:39224) 2024-03-18T20:35:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd9230a700 1 -- 172.21.15.46:0/1310770961 learned_addr learned my addr 172.21.15.46:0/1310770961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd91b09700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd8c100e00 0x7efd8c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd91308700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c1017c0 0x7efd8c197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd9230a700 1 -- 172.21.15.46:0/1310770961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c1017c0 msgr2=0x7efd8c197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd9230a700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c1017c0 0x7efd8c197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd9230a700 1 -- 172.21.15.46:0/1310770961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd8c100e00 msgr2=0x7efd8c197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd9230a700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd8c100e00 0x7efd8c197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.795+0000 7efd9230a700 1 -- 172.21.15.46:0/1310770961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd880095f0 con 0x7efd8c106e40 2024-03-18T20:35:03.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.796+0000 7efd9230a700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8c106e40 0x7efd8c19ba00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efd84007ee0 tx=0x7efd8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:03.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.796+0000 7efd82ffd700 1 -- 172.21.15.46:0/1310770961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84017070 con 0x7efd8c106e40 2024-03-18T20:35:03.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.796+0000 7efd82ffd700 1 -- 172.21.15.46:0/1310770961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd8400f040 con 0x7efd8c106e40 2024-03-18T20:35:03.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.796+0000 7efd93d6d700 1 -- 172.21.15.46:0/1310770961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c19c620 con 0x7efd8c106e40 2024-03-18T20:35:03.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.796+0000 7efd82ffd700 1 -- 172.21.15.46:0/1310770961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84005580 con 0x7efd8c106e40 2024-03-18T20:35:03.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.796+0000 7efd93d6d700 1 -- 172.21.15.46:0/1310770961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c1a3e80 con 0x7efd8c106e40 2024-03-18T20:35:03.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.797+0000 7efd93d6d700 1 -- 172.21.15.46:0/1310770961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd8c102480 con 0x7efd8c106e40 2024-03-18T20:35:03.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.799+0000 7efd82ffd700 1 -- 172.21.15.46:0/1310770961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7efd8400db20 con 0x7efd8c106e40 2024-03-18T20:35:03.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.799+0000 7efd82ffd700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efd780a11b0 0x7efd780a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.799+0000 7efd91b09700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efd780a11b0 0x7efd780a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.799+0000 7efd82ffd700 1 -- 172.21.15.46:0/1310770961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efd840cc6e0 con 0x7efd8c106e40 2024-03-18T20:35:03.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.800+0000 7efd82ffd700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749] conn(0x7efd780aa990 0x7efd780acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.800+0000 7efd82ffd700 1 -- 172.21.15.46:0/1310770961 --> [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd780ad460 con 0x7efd780aa990 2024-03-18T20:35:03.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.800+0000 7efd82ffd700 1 -- 172.21.15.46:0/1310770961 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd840ccb80 con 0x7efd8c106e40 2024-03-18T20:35:03.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.800+0000 7efd91308700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749] conn(0x7efd780aa990 0x7efd780acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.800+0000 7efd91b09700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efd780a11b0 0x7efd780a3670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efd880098e0 tx=0x7efd88005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:03.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.800+0000 7efd91308700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749] conn(0x7efd780aa990 0x7efd780acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:03.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.801+0000 7efd82ffd700 1 -- 172.21.15.46:0/1310770961 <== osd.4 v2:172.21.15.73:6808/1954145749 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7efd780ad460 con 0x7efd780aa990 2024-03-18T20:35:03.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.818+0000 7efd93d6d700 1 -- 172.21.15.46:0/1310770961 --> [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efd8c110f60 con 0x7efd780aa990 2024-03-18T20:35:03.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.818+0000 7efd82ffd700 1 -- 172.21.15.46:0/1310770961 <== osd.4 v2:172.21.15.73:6808/1954145749 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efd8c110f60 con 0x7efd780aa990 2024-03-18T20:35:03.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.818+0000 7efd93d6d700 1 -- 172.21.15.46:0/1310770961 >> [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749] conn(0x7efd780aa990 msgr2=0x7efd780acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.818+0000 7efd93d6d700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749] conn(0x7efd780aa990 0x7efd780acdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.819+0000 7efd93d6d700 1 -- 172.21.15.46:0/1310770961 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efd780a11b0 msgr2=0x7efd780a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.819+0000 7efd93d6d700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efd780a11b0 0x7efd780a3670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efd880098e0 tx=0x7efd88005960 comp rx=0 tx=0).stop 2024-03-18T20:35:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.819+0000 7efd93d6d700 1 -- 172.21.15.46:0/1310770961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8c106e40 msgr2=0x7efd8c19ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.819+0000 7efd93d6d700 1 --2- 172.21.15.46:0/1310770961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8c106e40 0x7efd8c19ba00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efd84007ee0 tx=0x7efd8400d3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.819+0000 7efd9230a700 1 -- 172.21.15.46:0/1310770961 reap_dead start 2024-03-18T20:35:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.819+0000 7efd93d6d700 1 -- 172.21.15.46:0/1310770961 shutdown_connections 2024-03-18T20:35:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.819+0000 7efd93d6d700 1 -- 172.21.15.46:0/1310770961 >> 172.21.15.46:0/1310770961 conn(0x7efd8c077f10 msgr2=0x7efd8c0ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.819+0000 7efd93d6d700 1 -- 172.21.15.46:0/1310770961 shutdown_connections 2024-03-18T20:35:03.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.819+0000 7efd93d6d700 1 -- 172.21.15.46:0/1310770961 wait complete. 2024-03-18T20:35:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.649+0000 7f2744d4b700 1 -- 172.21.15.46:0/979989257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2740106df0 msgr2=0x7f27401071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.649+0000 7f2744d4b700 1 --2- 172.21.15.46:0/979989257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2740106df0 0x7f27401071d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f27280099c0 tx=0x7f2728009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.649+0000 7f2744d4b700 1 -- 172.21.15.46:0/979989257 shutdown_connections 2024-03-18T20:35:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.649+0000 7f2744d4b700 1 --2- 172.21.15.46:0/979989257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2740106df0 0x7f27401071d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.649+0000 7f2744d4b700 1 --2- 172.21.15.46:0/979989257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27400699d0 0x7f274010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.649+0000 7f2744d4b700 1 --2- 172.21.15.46:0/979989257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2740069010 0x7f2740069490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.649+0000 7f2744d4b700 1 -- 172.21.15.46:0/979989257 >> 172.21.15.46:0/979989257 conn(0x7f2740076c80 msgr2=0x7f2740077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.649+0000 7f2744d4b700 1 -- 172.21.15.46:0/979989257 shutdown_connections 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.649+0000 7f2744d4b700 1 -- 172.21.15.46:0/979989257 wait complete. 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.650+0000 7f2744d4b700 1 Processor -- start 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f2744d4b700 1 -- start start 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f2744d4b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2740069010 0x7f274010b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f2744d4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27400699d0 0x7f274010bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f2744d4b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2740106df0 0x7f27401a3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f2744d4b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27401a3780 con 0x7f27400699d0 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f2744d4b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27401a38f0 con 0x7f2740106df0 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f2744d4b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27401a3ab0 con 0x7f2740069010 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f273dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27400699d0 0x7f274010bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f273dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27400699d0 0x7f274010bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39198/0 (socket says 172.21.15.46:39198) 2024-03-18T20:35:03.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f273dd9b700 1 -- 172.21.15.46:0/1785116807 learned_addr learned my addr 172.21.15.46:0/1785116807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f273e59c700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2740069010 0x7f274010b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f273ed9d700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2740106df0 0x7f27401a3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f273dd9b700 1 -- 172.21.15.46:0/1785116807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2740069010 msgr2=0x7f274010b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f273dd9b700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2740069010 0x7f274010b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f273dd9b700 1 -- 172.21.15.46:0/1785116807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2740106df0 msgr2=0x7f27401a3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.651+0000 7f273dd9b700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2740106df0 0x7f27401a3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.652+0000 7f273dd9b700 1 -- 172.21.15.46:0/1785116807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27280095f0 con 0x7f27400699d0 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.652+0000 7f273ed9d700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2740106df0 0x7f27401a3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.652+0000 7f273dd9b700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27400699d0 0x7f274010bc80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f273000d950 tx=0x7f273000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.652+0000 7f27377fe700 1 -- 172.21.15.46:0/1785116807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2730009940 con 0x7f27400699d0 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.652+0000 7f2744d4b700 1 -- 172.21.15.46:0/1785116807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27401a3d90 con 0x7f27400699d0 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.652+0000 7f27377fe700 1 -- 172.21.15.46:0/1785116807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2730010460 con 0x7f27400699d0 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.652+0000 7f2744d4b700 1 -- 172.21.15.46:0/1785116807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27401aa450 con 0x7f27400699d0 2024-03-18T20:35:03.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.652+0000 7f27377fe700 1 -- 172.21.15.46:0/1785116807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273000f620 con 0x7f27400699d0 2024-03-18T20:35:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.653+0000 7f2744d4b700 1 -- 172.21.15.46:0/1785116807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f274010edc0 con 0x7f27400699d0 2024-03-18T20:35:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.655+0000 7f27377fe700 1 -- 172.21.15.46:0/1785116807 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f27300108d0 con 0x7f27400699d0 2024-03-18T20:35:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.655+0000 7f27377fe700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f272c0a11e0 0x7f272c0a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.655+0000 7f273e59c700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f272c0a11e0 0x7f272c0a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.656+0000 7f27377fe700 1 -- 172.21.15.46:0/1785116807 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f27300ccac0 con 0x7f27400699d0 2024-03-18T20:35:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.656+0000 7f27377fe700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296] conn(0x7f272c0aa9c0 0x7f272c0acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.656+0000 7f27377fe700 1 -- 172.21.15.46:0/1785116807 --> [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f272c0ad490 con 0x7f272c0aa9c0 2024-03-18T20:35:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.656+0000 7f27377fe700 1 -- 172.21.15.46:0/1785116807 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27300ccf60 con 0x7f27400699d0 2024-03-18T20:35:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.656+0000 7f273e59c700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f272c0a11e0 0x7f272c0a36a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f272800b580 tx=0x7f2728009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.656+0000 7f273ed9d700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296] conn(0x7f272c0aa9c0 0x7f272c0acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:03.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.657+0000 7f273ed9d700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296] conn(0x7f272c0aa9c0 0x7f272c0acde0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.657+0000 7f27377fe700 1 -- 172.21.15.46:0/1785116807 <== osd.2 v2:172.21.15.155:6800/772117296 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f272c0ad490 con 0x7f272c0aa9c0 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.674+0000 7f2744d4b700 1 -- 172.21.15.46:0/1785116807 --> [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2740112010 con 0x7f272c0aa9c0 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.675+0000 7f27377fe700 1 -- 172.21.15.46:0/1785116807 <== osd.2 v2:172.21.15.155:6800/772117296 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2740112010 con 0x7f272c0aa9c0 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.675+0000 7f2744d4b700 1 -- 172.21.15.46:0/1785116807 >> [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296] conn(0x7f272c0aa9c0 msgr2=0x7f272c0acde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.676+0000 7f2744d4b700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296] conn(0x7f272c0aa9c0 0x7f272c0acde0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.676+0000 7f2744d4b700 1 -- 172.21.15.46:0/1785116807 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f272c0a11e0 msgr2=0x7f272c0a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.676+0000 7f2744d4b700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f272c0a11e0 0x7f272c0a36a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f272800b580 tx=0x7f2728009f10 comp rx=0 tx=0).stop 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.676+0000 7f2744d4b700 1 -- 172.21.15.46:0/1785116807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27400699d0 msgr2=0x7f274010bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.676+0000 7f2744d4b700 1 --2- 172.21.15.46:0/1785116807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27400699d0 0x7f274010bc80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f273000d950 tx=0x7f273000dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.676+0000 7f273ed9d700 1 -- 172.21.15.46:0/1785116807 reap_dead start 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.676+0000 7f2744d4b700 1 -- 172.21.15.46:0/1785116807 shutdown_connections 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.676+0000 7f2744d4b700 1 -- 172.21.15.46:0/1785116807 >> 172.21.15.46:0/1785116807 conn(0x7f2740076c80 msgr2=0x7f27401116c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:03.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.676+0000 7f2744d4b700 1 -- 172.21.15.46:0/1785116807 shutdown_connections 2024-03-18T20:35:03.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:03.676+0000 7f2744d4b700 1 -- 172.21.15.46:0/1785116807 wait complete. 2024-03-18T20:35:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:04.167 INFO:teuthology.orchestra.run.smithi046.stdout:261993005078 2024-03-18T20:35:04.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-18T20:35:04.347 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:04 smithi046 ceph-mon[43495]: pgmap v231: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-03-18T20:35:04.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:04 smithi073 ceph-mon[47384]: pgmap v231: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-03-18T20:35:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:04 smithi155 ceph-mon[48344]: pgmap v231: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-03-18T20:35:04.844 INFO:teuthology.orchestra.run.smithi046.stdout:55834574897 2024-03-18T20:35:04.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-18T20:35:05.025 INFO:teuthology.orchestra.run.smithi046.stdout:133143986216 2024-03-18T20:35:05.025 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-18T20:35:05.431 INFO:teuthology.orchestra.run.smithi046.stdout:77309411375 2024-03-18T20:35:05.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-18T20:35:05.794 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:05 smithi046 ceph-mon[43495]: pgmap v232: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-03-18T20:35:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:05 smithi073 ceph-mon[47384]: pgmap v232: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-03-18T20:35:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:05 smithi155 ceph-mon[48344]: pgmap v232: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-03-18T20:35:06.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.597+0000 7f1ac60d5700 1 -- 172.21.15.46:0/1365601720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac0068750 msgr2=0x7f1ac0068bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:06.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.597+0000 7f1ac60d5700 1 --2- 172.21.15.46:0/1365601720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac0068750 0x7f1ac0068bb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009990 tx=0x7f1aa8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.597+0000 7f1ac60d5700 1 -- 172.21.15.46:0/1365601720 shutdown_connections 2024-03-18T20:35:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.597+0000 7f1ac60d5700 1 --2- 172.21.15.46:0/1365601720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ac00690f0 0x7f1ac010dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.597+0000 7f1ac60d5700 1 --2- 172.21.15.46:0/1365601720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac0106df0 0x7f1ac01071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.597+0000 7f1ac60d5700 1 --2- 172.21.15.46:0/1365601720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac0068750 0x7f1ac0068bb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.597+0000 7f1ac60d5700 1 -- 172.21.15.46:0/1365601720 >> 172.21.15.46:0/1365601720 conn(0x7f1ac0075d50 msgr2=0x7f1ac0076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.597+0000 7f1ac60d5700 1 -- 172.21.15.46:0/1365601720 shutdown_connections 2024-03-18T20:35:06.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.597+0000 7f1ac60d5700 1 -- 172.21.15.46:0/1365601720 wait complete. 2024-03-18T20:35:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.598+0000 7f1ac60d5700 1 Processor -- start 2024-03-18T20:35:06.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.598+0000 7f1ac60d5700 1 -- start start 2024-03-18T20:35:06.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.598+0000 7f1ac60d5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac0068750 0x7f1ac00ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:06.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.598+0000 7f1ac60d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ac00690f0 0x7f1ac0100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:06.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.598+0000 7f1ac60d5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac0106df0 0x7f1ac0100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:06.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.598+0000 7f1ac60d5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac01a5460 con 0x7f1ac0068750 2024-03-18T20:35:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.598+0000 7f1ac60d5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac01a55d0 con 0x7f1ac0106df0 2024-03-18T20:35:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.598+0000 7f1ac60d5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac01a5790 con 0x7f1ac00690f0 2024-03-18T20:35:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac0068750 0x7f1ac00ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ac00690f0 0x7f1ac0100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac0068750 0x7f1ac00ffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49988/0 (socket says 172.21.15.46:49988) 2024-03-18T20:35:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abf7fe700 1 -- 172.21.15.46:0/2880751177 learned_addr learned my addr 172.21.15.46:0/2880751177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abffff700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac0106df0 0x7f1ac0100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abf7fe700 1 -- 172.21.15.46:0/2880751177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ac00690f0 msgr2=0x7f1ac0100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abf7fe700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ac00690f0 0x7f1ac0100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:06.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abf7fe700 1 -- 172.21.15.46:0/2880751177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac0106df0 msgr2=0x7f1ac0100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abf7fe700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac0106df0 0x7f1ac0100860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abf7fe700 1 -- 172.21.15.46:0/2880751177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa80095f0 con 0x7f1ac0068750 2024-03-18T20:35:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abf7fe700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac0068750 0x7f1ac00ffde0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0007e20 tx=0x7f1ab000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abcff9700 1 -- 172.21.15.46:0/2880751177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000ac90 con 0x7f1ac0068750 2024-03-18T20:35:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abcff9700 1 -- 172.21.15.46:0/2880751177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab000adf0 con 0x7f1ac0068750 2024-03-18T20:35:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1ac60d5700 1 -- 172.21.15.46:0/2880751177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac01a5a70 con 0x7f1ac0068750 2024-03-18T20:35:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.599+0000 7f1abcff9700 1 -- 172.21.15.46:0/2880751177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0011690 con 0x7f1ac0068750 2024-03-18T20:35:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.600+0000 7f1ac60d5700 1 -- 172.21.15.46:0/2880751177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac01a5f40 con 0x7f1ac0068750 2024-03-18T20:35:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.600+0000 7f1ac60d5700 1 -- 172.21.15.46:0/2880751177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1ac01024a0 con 0x7f1ac0068750 2024-03-18T20:35:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.602+0000 7f1abcff9700 1 -- 172.21.15.46:0/2880751177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1ab0012420 con 0x7f1ac0068750 2024-03-18T20:35:06.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.602+0000 7f1abcff9700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1aac0a11b0 0x7f1aac0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:06.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.602+0000 7f1abeffd700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1aac0a11b0 0x7f1aac0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:06.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.602+0000 7f1abcff9700 1 -- 172.21.15.46:0/2880751177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1ab00ccfc0 con 0x7f1ac0068750 2024-03-18T20:35:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.603+0000 7f1abcff9700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490] conn(0x7f1aac0aa990 0x7f1aac0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.603+0000 7f1abcff9700 1 -- 172.21.15.46:0/2880751177 --> [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1aac0ad460 con 0x7f1aac0aa990 2024-03-18T20:35:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.603+0000 7f1abcff9700 1 -- 172.21.15.46:0/2880751177 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ab00cd460 con 0x7f1ac0068750 2024-03-18T20:35:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.603+0000 7f1abffff700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490] conn(0x7f1aac0aa990 0x7f1aac0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.603+0000 7f1abeffd700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1aac0a11b0 0x7f1aac0a3670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800b5c0 tx=0x7f1aa8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.603+0000 7f1abffff700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490] conn(0x7f1aac0aa990 0x7f1aac0acdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:06.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.604+0000 7f1abcff9700 1 -- 172.21.15.46:0/2880751177 <== osd.0 v2:172.21.15.46:6802/2908504490 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f1aac0ad460 con 0x7f1aac0aa990 2024-03-18T20:35:06.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.620+0000 7f1ac60d5700 1 -- 172.21.15.46:0/2880751177 --> [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1ac0110ef0 con 0x7f1aac0aa990 2024-03-18T20:35:06.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.621+0000 7f1abcff9700 1 -- 172.21.15.46:0/2880751177 <== osd.0 v2:172.21.15.46:6802/2908504490 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1ac0110ef0 con 0x7f1aac0aa990 2024-03-18T20:35:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.621+0000 7f1ac60d5700 1 -- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490] conn(0x7f1aac0aa990 msgr2=0x7f1aac0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.621+0000 7f1ac60d5700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490] conn(0x7f1aac0aa990 0x7f1aac0acdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.621+0000 7f1ac60d5700 1 -- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1aac0a11b0 msgr2=0x7f1aac0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.621+0000 7f1ac60d5700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1aac0a11b0 0x7f1aac0a3670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800b5c0 tx=0x7f1aa8005960 comp rx=0 tx=0).stop 2024-03-18T20:35:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.621+0000 7f1ac60d5700 1 -- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac0068750 msgr2=0x7f1ac00ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.621+0000 7f1ac60d5700 1 --2- 172.21.15.46:0/2880751177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ac0068750 0x7f1ac00ffde0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0007e20 tx=0x7f1ab000a340 comp rx=0 tx=0).stop 2024-03-18T20:35:06.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.621+0000 7f1abffff700 1 -- 172.21.15.46:0/2880751177 reap_dead start 2024-03-18T20:35:06.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.621+0000 7f1ac60d5700 1 -- 172.21.15.46:0/2880751177 shutdown_connections 2024-03-18T20:35:06.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.621+0000 7f1ac60d5700 1 -- 172.21.15.46:0/2880751177 >> 172.21.15.46:0/2880751177 conn(0x7f1ac0075d50 msgr2=0x7f1ac00fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:06.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.621+0000 7f1ac60d5700 1 -- 172.21.15.46:0/2880751177 shutdown_connections 2024-03-18T20:35:06.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:06.621+0000 7f1ac60d5700 1 -- 172.21.15.46:0/2880751177 wait complete. 2024-03-18T20:35:07.076 INFO:teuthology.orchestra.run.smithi046.stdout:34359738420 2024-03-18T20:35:07.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-18T20:35:07.219 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:07.322 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:07.331 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.641+0000 7fb05ed4d700 1 -- 172.21.15.46:0/3330222345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb058111be0 msgr2=0x7fb058113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.641+0000 7fb05ed4d700 1 --2- 172.21.15.46:0/3330222345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb058111be0 0x7fb058113fd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb05000b0a0 tx=0x7fb05000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.642+0000 7fb05ed4d700 1 -- 172.21.15.46:0/3330222345 shutdown_connections 2024-03-18T20:35:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.642+0000 7fb05ed4d700 1 --2- 172.21.15.46:0/3330222345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb058111be0 0x7fb058113fd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.642+0000 7fb05ed4d700 1 --2- 172.21.15.46:0/3330222345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0580ff750 0x7fb0580ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.642+0000 7fb05ed4d700 1 --2- 172.21.15.46:0/3330222345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb058100070 0x7fb0581116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.642+0000 7fb05ed4d700 1 -- 172.21.15.46:0/3330222345 >> 172.21.15.46:0/3330222345 conn(0x7fb058074fd0 msgr2=0x7fb0580753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.643+0000 7fb05ed4d700 1 -- 172.21.15.46:0/3330222345 shutdown_connections 2024-03-18T20:35:07.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.643+0000 7fb05ed4d700 1 -- 172.21.15.46:0/3330222345 wait complete. 2024-03-18T20:35:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.643+0000 7fb05ed4d700 1 Processor -- start 2024-03-18T20:35:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.643+0000 7fb05ed4d700 1 -- start start 2024-03-18T20:35:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.644+0000 7fb05ed4d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0580ff750 0x7fb0581a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:07.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.644+0000 7fb05ed4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb058100070 0x7fb0581a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.644+0000 7fb05ed4d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb058111be0 0x7fb05819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.644+0000 7fb05ed4d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05819ddb0 con 0x7fb058100070 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.644+0000 7fb05ed4d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05819df20 con 0x7fb0580ff750 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.644+0000 7fb05ed4d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05819e0e0 con 0x7fb058111be0 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.644+0000 7fb057fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb058100070 0x7fb0581a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.644+0000 7fb057fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb058100070 0x7fb0581a3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49996/0 (socket says 172.21.15.46:49996) 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.644+0000 7fb057fff700 1 -- 172.21.15.46:0/1424137195 learned_addr learned my addr 172.21.15.46:0/1424137195 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.644+0000 7fb05cae9700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0580ff750 0x7fb0581a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.644+0000 7fb05d2ea700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb058111be0 0x7fb05819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.645+0000 7fb057fff700 1 -- 172.21.15.46:0/1424137195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb058111be0 msgr2=0x7fb05819d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.645+0000 7fb057fff700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb058111be0 0x7fb05819d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.645+0000 7fb057fff700 1 -- 172.21.15.46:0/1424137195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0580ff750 msgr2=0x7fb0581a35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.645+0000 7fb057fff700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0580ff750 0x7fb0581a35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.645+0000 7fb057fff700 1 -- 172.21.15.46:0/1424137195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb050009d00 con 0x7fb058100070 2024-03-18T20:35:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.645+0000 7fb057fff700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb058100070 0x7fb0581a3af0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb04400c9b0 tx=0x7fb04400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.645+0000 7fb055ffb700 1 -- 172.21.15.46:0/1424137195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044007b00 con 0x7fb058100070 2024-03-18T20:35:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.645+0000 7fb055ffb700 1 -- 172.21.15.46:0/1424137195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb04400ce50 con 0x7fb058100070 2024-03-18T20:35:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.645+0000 7fb055ffb700 1 -- 172.21.15.46:0/1424137195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044018660 con 0x7fb058100070 2024-03-18T20:35:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.645+0000 7fb05ed4d700 1 -- 172.21.15.46:0/1424137195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05819e3c0 con 0x7fb058100070 2024-03-18T20:35:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.646+0000 7fb05ed4d700 1 -- 172.21.15.46:0/1424137195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0581aa5a0 con 0x7fb058100070 2024-03-18T20:35:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.646+0000 7fb05ed4d700 1 -- 172.21.15.46:0/1424137195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb058117600 con 0x7fb058100070 2024-03-18T20:35:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.649+0000 7fb055ffb700 1 -- 172.21.15.46:0/1424137195 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb04400f450 con 0x7fb058100070 2024-03-18T20:35:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.650+0000 7fb055ffb700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0480a11b0 0x7fb0480a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.650+0000 7fb055ffb700 1 -- 172.21.15.46:0/1424137195 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb0440ccd10 con 0x7fb058100070 2024-03-18T20:35:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.650+0000 7fb055ffb700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764] conn(0x7fb0480aa990 0x7fb0480acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.650+0000 7fb055ffb700 1 -- 172.21.15.46:0/1424137195 --> [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0480ad460 con 0x7fb0480aa990 2024-03-18T20:35:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.650+0000 7fb055ffb700 1 -- 172.21.15.46:0/1424137195 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0440cd1b0 con 0x7fb058100070 2024-03-18T20:35:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.650+0000 7fb05cae9700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0480a11b0 0x7fb0480a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.650+0000 7fb05d2ea700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764] conn(0x7fb0480aa990 0x7fb0480acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:07.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.651+0000 7fb05cae9700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0480a11b0 0x7fb0480a3670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009c60 tx=0x7fb04c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:07.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.651+0000 7fb05d2ea700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764] conn(0x7fb0480aa990 0x7fb0480acdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:07.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.652+0000 7fb055ffb700 1 -- 172.21.15.46:0/1424137195 <== osd.11 v2:172.21.15.155:6824/2159697764 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb0480ad460 con 0x7fb0480aa990 2024-03-18T20:35:07.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.669+0000 7fb05ed4d700 1 -- 172.21.15.46:0/1424137195 --> [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb05810ffe0 con 0x7fb0480aa990 2024-03-18T20:35:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.670+0000 7fb055ffb700 1 -- 172.21.15.46:0/1424137195 <== osd.11 v2:172.21.15.155:6824/2159697764 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb05810ffe0 con 0x7fb0480aa990 2024-03-18T20:35:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.670+0000 7fb05ed4d700 1 -- 172.21.15.46:0/1424137195 >> [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764] conn(0x7fb0480aa990 msgr2=0x7fb0480acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.670+0000 7fb05ed4d700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764] conn(0x7fb0480aa990 0x7fb0480acdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.670+0000 7fb05ed4d700 1 -- 172.21.15.46:0/1424137195 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0480a11b0 msgr2=0x7fb0480a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.670+0000 7fb05ed4d700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0480a11b0 0x7fb0480a3670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009c60 tx=0x7fb04c009260 comp rx=0 tx=0).stop 2024-03-18T20:35:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.670+0000 7fb05ed4d700 1 -- 172.21.15.46:0/1424137195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb058100070 msgr2=0x7fb0581a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.670+0000 7fb05ed4d700 1 --2- 172.21.15.46:0/1424137195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb058100070 0x7fb0581a3af0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb04400c9b0 tx=0x7fb04400ccc0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.670+0000 7fb05d2ea700 1 -- 172.21.15.46:0/1424137195 reap_dead start 2024-03-18T20:35:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.670+0000 7fb05ed4d700 1 -- 172.21.15.46:0/1424137195 shutdown_connections 2024-03-18T20:35:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.670+0000 7fb05ed4d700 1 -- 172.21.15.46:0/1424137195 >> 172.21.15.46:0/1424137195 conn(0x7fb058074fd0 msgr2=0x7fb05810f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.670+0000 7fb05ed4d700 1 -- 172.21.15.46:0/1424137195 shutdown_connections 2024-03-18T20:35:07.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.670+0000 7fb05ed4d700 1 -- 172.21.15.46:0/1424137195 wait complete. 2024-03-18T20:35:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.705+0000 7f7596f82700 1 -- 172.21.15.46:0/1576373967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7590106e40 msgr2=0x7f7590107220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.705+0000 7f7596f82700 1 --2- 172.21.15.46:0/1576373967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7590106e40 0x7f7590107220 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7584009990 tx=0x7f7584009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.706+0000 7f7596f82700 1 -- 172.21.15.46:0/1576373967 shutdown_connections 2024-03-18T20:35:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.706+0000 7f7596f82700 1 --2- 172.21.15.46:0/1576373967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7590106e40 0x7f7590107220 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.706+0000 7f7596f82700 1 --2- 172.21.15.46:0/1576373967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7590101820 0x7f759010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.706+0000 7f7596f82700 1 --2- 172.21.15.46:0/1576373967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7590100e60 0x7f75901012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.706+0000 7f7596f82700 1 -- 172.21.15.46:0/1576373967 >> 172.21.15.46:0/1576373967 conn(0x7f7590077f10 msgr2=0x7f7590078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.706+0000 7f7596f82700 1 -- 172.21.15.46:0/1576373967 shutdown_connections 2024-03-18T20:35:07.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.706+0000 7f7596f82700 1 -- 172.21.15.46:0/1576373967 wait complete. 2024-03-18T20:35:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.706+0000 7f7596f82700 1 Processor -- start 2024-03-18T20:35:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.707+0000 7f7596f82700 1 -- start start 2024-03-18T20:35:07.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.707+0000 7f7596f82700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7590100e60 0x7f75901993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.707+0000 7f7596f82700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7590101820 0x7f75901998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.707+0000 7f7596f82700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7590106e40 0x7f759019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.707+0000 7f7596f82700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019e2b0 con 0x7f7590100e60 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.707+0000 7f7596f82700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019e420 con 0x7f7590106e40 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.707+0000 7f7596f82700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759019e5e0 con 0x7f7590101820 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.707+0000 7f7594d1e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7590100e60 0x7f75901993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.707+0000 7f7594d1e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7590100e60 0x7f75901993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50006/0 (socket says 172.21.15.46:50006) 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.707+0000 7f7594d1e700 1 -- 172.21.15.46:0/146696088 learned_addr learned my addr 172.21.15.46:0/146696088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.707+0000 7f758ffff700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7590101820 0x7f75901998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.708+0000 7f759551f700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7590106e40 0x7f759019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.708+0000 7f7594d1e700 1 -- 172.21.15.46:0/146696088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7590101820 msgr2=0x7f75901998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.708+0000 7f7594d1e700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7590101820 0x7f75901998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.708+0000 7f7594d1e700 1 -- 172.21.15.46:0/146696088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7590106e40 msgr2=0x7f759019dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.708+0000 7f7594d1e700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7590106e40 0x7f759019dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.708+0000 7f7594d1e700 1 -- 172.21.15.46:0/146696088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75840095f0 con 0x7f7590100e60 2024-03-18T20:35:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.708+0000 7f759551f700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7590106e40 0x7f759019dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.708+0000 7f7594d1e700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7590100e60 0x7f75901993b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7584000c00 tx=0x7f7584004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.708+0000 7f758dffb700 1 -- 172.21.15.46:0/146696088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758401d070 con 0x7f7590100e60 2024-03-18T20:35:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.708+0000 7f758dffb700 1 -- 172.21.15.46:0/146696088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7584022470 con 0x7f7590100e60 2024-03-18T20:35:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.708+0000 7f758dffb700 1 -- 172.21.15.46:0/146696088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584017650 con 0x7f7590100e60 2024-03-18T20:35:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.708+0000 7f7596f82700 1 -- 172.21.15.46:0/146696088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759019e860 con 0x7f7590100e60 2024-03-18T20:35:07.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.709+0000 7f7596f82700 1 -- 172.21.15.46:0/146696088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759006a7d0 con 0x7f7590100e60 2024-03-18T20:35:07.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.710+0000 7f7596f82700 1 -- 172.21.15.46:0/146696088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7590102480 con 0x7f7590100e60 2024-03-18T20:35:07.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.711+0000 7f758dffb700 1 -- 172.21.15.46:0/146696088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f75840225e0 con 0x7f7590100e60 2024-03-18T20:35:07.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.711+0000 7f758dffb700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f75800a11b0 0x7f75800a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:07.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.712+0000 7f758dffb700 1 -- 172.21.15.46:0/146696088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f75840cdcb0 con 0x7f7590100e60 2024-03-18T20:35:07.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.712+0000 7f758ffff700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f75800a11b0 0x7f75800a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:07.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.712+0000 7f758ffff700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f75800a11b0 0x7f75800a3670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f759019a910 tx=0x7f757c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:07.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.719+0000 7f758dffb700 1 -- 172.21.15.46:0/146696088 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7584097420 con 0x7f7590100e60 2024-03-18T20:35:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.950+0000 7f7596f82700 1 -- 172.21.15.46:0/146696088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f759004ea80 con 0x7f7590100e60 2024-03-18T20:35:07.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.950+0000 7f758dffb700 1 -- 172.21.15.46:0/146696088 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f758409aa40 con 0x7f7590100e60 2024-03-18T20:35:07.954 INFO:teuthology.orchestra.run.smithi046.stdout:150323855396 2024-03-18T20:35:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.953+0000 7f7596f82700 1 -- 172.21.15.46:0/146696088 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f75800a11b0 msgr2=0x7f75800a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.953+0000 7f7596f82700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f75800a11b0 0x7f75800a3670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f759019a910 tx=0x7f757c009260 comp rx=0 tx=0).stop 2024-03-18T20:35:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.953+0000 7f7596f82700 1 -- 172.21.15.46:0/146696088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7590100e60 msgr2=0x7f75901993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.953+0000 7f7596f82700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7590100e60 0x7f75901993b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7584000c00 tx=0x7f7584004730 comp rx=0 tx=0).stop 2024-03-18T20:35:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.953+0000 7f7596f82700 1 -- 172.21.15.46:0/146696088 shutdown_connections 2024-03-18T20:35:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.953+0000 7f7596f82700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f75800a11b0 0x7f75800a3670 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.953+0000 7f7596f82700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7590100e60 0x7f75901993b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.953+0000 7f7596f82700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7590106e40 0x7f759019dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.953+0000 7f7596f82700 1 --2- 172.21.15.46:0/146696088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7590101820 0x7f75901998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.953+0000 7f7596f82700 1 -- 172.21.15.46:0/146696088 >> 172.21.15.46:0/146696088 conn(0x7f7590077f10 msgr2=0x7f759010c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.953+0000 7f7596f82700 1 -- 172.21.15.46:0/146696088 shutdown_connections 2024-03-18T20:35:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:07.953+0000 7f7596f82700 1 -- 172.21.15.46:0/146696088 wait complete. 2024-03-18T20:35:08.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.291+0000 7f7f51e26700 1 -- 172.21.15.46:0/193969250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f4c100e60 msgr2=0x7f7f4c1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:08.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.291+0000 7f7f51e26700 1 --2- 172.21.15.46:0/193969250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f4c100e60 0x7f7f4c1012e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7f340099c0 tx=0x7f7f34009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:08.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.291+0000 7f7f51e26700 1 -- 172.21.15.46:0/193969250 shutdown_connections 2024-03-18T20:35:08.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.291+0000 7f7f51e26700 1 --2- 172.21.15.46:0/193969250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c101820 0x7f7f4c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:08.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.291+0000 7f7f51e26700 1 --2- 172.21.15.46:0/193969250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f4c106ef0 0x7f7f4c1072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.292+0000 7f7f51e26700 1 --2- 172.21.15.46:0/193969250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f4c100e60 0x7f7f4c1012e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.292+0000 7f7f51e26700 1 -- 172.21.15.46:0/193969250 >> 172.21.15.46:0/193969250 conn(0x7f7f4c077fa0 msgr2=0x7f7f4c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.292+0000 7f7f51e26700 1 -- 172.21.15.46:0/193969250 shutdown_connections 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.292+0000 7f7f51e26700 1 -- 172.21.15.46:0/193969250 wait complete. 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.292+0000 7f7f51e26700 1 Processor -- start 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.292+0000 7f7f51e26700 1 -- start start 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f51e26700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f4c100e60 0x7f7f4c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f51e26700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f4c101820 0x7f7f4c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f51e26700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c106ef0 0x7f7f4c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f51e26700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1a5550 con 0x7f7f4c100e60 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f51e26700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1a56c0 con 0x7f7f4c101820 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f51e26700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1a5880 con 0x7f7f4c106ef0 2024-03-18T20:35:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f4b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f4c100e60 0x7f7f4c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f4b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f4c100e60 0x7f7f4c0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50038/0 (socket says 172.21.15.46:50038) 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f4b7fe700 1 -- 172.21.15.46:0/2380586595 learned_addr learned my addr 172.21.15.46:0/2380586595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f4bfff700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c106ef0 0x7f7f4c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f42ffd700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f4c101820 0x7f7f4c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f4b7fe700 1 -- 172.21.15.46:0/2380586595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c106ef0 msgr2=0x7f7f4c073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f4b7fe700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c106ef0 0x7f7f4c073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f4b7fe700 1 -- 172.21.15.46:0/2380586595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f4c101820 msgr2=0x7f7f4c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f4b7fe700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f4c101820 0x7f7f4c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f4b7fe700 1 -- 172.21.15.46:0/2380586595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f340095f0 con 0x7f7f4c100e60 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f4b7fe700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f4c100e60 0x7f7f4c0730b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00d950 tx=0x7f7f3c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f497fa700 1 -- 172.21.15.46:0/2380586595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c009940 con 0x7f7f4c100e60 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f497fa700 1 -- 172.21.15.46:0/2380586595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f3c010460 con 0x7f7f4c100e60 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f51e26700 1 -- 172.21.15.46:0/2380586595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c1a5b60 con 0x7f7f4c100e60 2024-03-18T20:35:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.293+0000 7f7f497fa700 1 -- 172.21.15.46:0/2380586595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00f620 con 0x7f7f4c100e60 2024-03-18T20:35:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.294+0000 7f7f51e26700 1 -- 172.21.15.46:0/2380586595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4c1a6030 con 0x7f7f4c100e60 2024-03-18T20:35:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.295+0000 7f7f51e26700 1 -- 172.21.15.46:0/2380586595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4c1092f0 con 0x7f7f4c100e60 2024-03-18T20:35:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.296+0000 7f7f497fa700 1 -- 172.21.15.46:0/2380586595 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f7f3c0108d0 con 0x7f7f4c100e60 2024-03-18T20:35:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.297+0000 7f7f497fa700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7f380a11b0 0x7f7f380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.297+0000 7f7f42ffd700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7f380a11b0 0x7f7f380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.297+0000 7f7f497fa700 1 -- 172.21.15.46:0/2380586595 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7f3c0cc700 con 0x7f7f4c100e60 2024-03-18T20:35:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.297+0000 7f7f42ffd700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7f380a11b0 0x7f7f380a3670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009f50 tx=0x7f7f34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.301+0000 7f7f497fa700 1 -- 172.21.15.46:0/2380586595 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7f3c095ee0 con 0x7f7f4c100e60 2024-03-18T20:35:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:08.337 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:08 smithi046 ceph-mon[43495]: pgmap v233: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-18T20:35:08.337 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:08 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/146696088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T20:35:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:08 smithi073 ceph-mon[47384]: pgmap v233: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-18T20:35:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:08 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/146696088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T20:35:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.521+0000 7f7f51e26700 1 -- 172.21.15.46:0/2380586595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7f4c074590 con 0x7f7f4c100e60 2024-03-18T20:35:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.522+0000 7f7f497fa700 1 -- 172.21.15.46:0/2380586595 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f3c016090 con 0x7f7f4c100e60 2024-03-18T20:35:08.525 INFO:teuthology.orchestra.run.smithi046.stdout:180388626464 2024-03-18T20:35:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.524+0000 7f7f51e26700 1 -- 172.21.15.46:0/2380586595 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7f380a11b0 msgr2=0x7f7f380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.524+0000 7f7f51e26700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7f380a11b0 0x7f7f380a3670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009f50 tx=0x7f7f34005fb0 comp rx=0 tx=0).stop 2024-03-18T20:35:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.524+0000 7f7f51e26700 1 -- 172.21.15.46:0/2380586595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f4c100e60 msgr2=0x7f7f4c0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.524+0000 7f7f51e26700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f4c100e60 0x7f7f4c0730b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00d950 tx=0x7f7f3c00dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.525+0000 7f7f51e26700 1 -- 172.21.15.46:0/2380586595 shutdown_connections 2024-03-18T20:35:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.525+0000 7f7f51e26700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7f380a11b0 0x7f7f380a3670 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.525+0000 7f7f51e26700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f4c100e60 0x7f7f4c0730b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.525+0000 7f7f51e26700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f4c106ef0 0x7f7f4c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.525+0000 7f7f51e26700 1 --2- 172.21.15.46:0/2380586595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f4c101820 0x7f7f4c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:08.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.525+0000 7f7f51e26700 1 -- 172.21.15.46:0/2380586595 >> 172.21.15.46:0/2380586595 conn(0x7f7f4c077fa0 msgr2=0x7f7f4c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:08.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.525+0000 7f7f51e26700 1 -- 172.21.15.46:0/2380586595 shutdown_connections 2024-03-18T20:35:08.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:08.525+0000 7f7f51e26700 1 -- 172.21.15.46:0/2380586595 wait complete. 2024-03-18T20:35:08.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:08 smithi155 ceph-mon[48344]: pgmap v233: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-03-18T20:35:08.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:08 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/146696088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T20:35:08.560 INFO:teuthology.orchestra.run.smithi046.stdout:313532612623 2024-03-18T20:35:08.561 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-18T20:35:09.074 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:09.074 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:09.115 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:09 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2380586595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-18T20:35:09.116 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855395 got 150323855396 for osd.5 2024-03-18T20:35:09.116 DEBUG:teuthology.parallel:result is None 2024-03-18T20:35:09.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:09 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2380586595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-18T20:35:09.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:09 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2380586595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-18T20:35:10.234 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:10 smithi046 ceph-mon[43495]: pgmap v234: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-18T20:35:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:10 smithi073 ceph-mon[47384]: pgmap v234: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-18T20:35:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:10 smithi155 ceph-mon[48344]: pgmap v234: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-18T20:35:10.556 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626463 got 180388626464 for osd.6 2024-03-18T20:35:10.556 DEBUG:teuthology.parallel:result is None 2024-03-18T20:35:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.799+0000 7f5439a67700 1 -- 172.21.15.46:0/2287428856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54340690f0 msgr2=0x7f543410dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.799+0000 7f5439a67700 1 --2- 172.21.15.46:0/2287428856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54340690f0 0x7f543410dc30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f542c00b0a0 tx=0x7f542c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.799+0000 7f5439a67700 1 -- 172.21.15.46:0/2287428856 shutdown_connections 2024-03-18T20:35:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.799+0000 7f5439a67700 1 --2- 172.21.15.46:0/2287428856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54340690f0 0x7f543410dc30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.799+0000 7f5439a67700 1 --2- 172.21.15.46:0/2287428856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5434106df0 0x7f54341071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.799+0000 7f5439a67700 1 --2- 172.21.15.46:0/2287428856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5434068750 0x7f5434068bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.799+0000 7f5439a67700 1 -- 172.21.15.46:0/2287428856 >> 172.21.15.46:0/2287428856 conn(0x7f5434075d50 msgr2=0x7f5434076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.799+0000 7f5439a67700 1 -- 172.21.15.46:0/2287428856 shutdown_connections 2024-03-18T20:35:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.799+0000 7f5439a67700 1 -- 172.21.15.46:0/2287428856 wait complete. 2024-03-18T20:35:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.801+0000 7f5439a67700 1 Processor -- start 2024-03-18T20:35:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.801+0000 7f5439a67700 1 -- start start 2024-03-18T20:35:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.801+0000 7f5439a67700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5434068750 0x7f5434197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.801+0000 7f5439a67700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54340690f0 0x7f54341978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.801+0000 7f5439a67700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5434106df0 0x7f543419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.801+0000 7f5439a67700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543419c260 con 0x7f5434068750 2024-03-18T20:35:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.801+0000 7f5439a67700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543419c3d0 con 0x7f5434106df0 2024-03-18T20:35:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.801+0000 7f5439a67700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543419c590 con 0x7f54340690f0 2024-03-18T20:35:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.801+0000 7f54337fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5434068750 0x7f5434197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.801+0000 7f54337fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5434068750 0x7f5434197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50066/0 (socket says 172.21.15.46:50066) 2024-03-18T20:35:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.801+0000 7f54337fe700 1 -- 172.21.15.46:0/851455975 learned_addr learned my addr 172.21.15.46:0/851455975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f5432ffd700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54340690f0 0x7f54341978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f5433fff700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5434106df0 0x7f543419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f54337fe700 1 -- 172.21.15.46:0/851455975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54340690f0 msgr2=0x7f54341978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f54337fe700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54340690f0 0x7f54341978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f54337fe700 1 -- 172.21.15.46:0/851455975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5434106df0 msgr2=0x7f543419bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f54337fe700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5434106df0 0x7f543419bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f54337fe700 1 -- 172.21.15.46:0/851455975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c009d00 con 0x7f5434068750 2024-03-18T20:35:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f54337fe700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5434068750 0x7f5434197360 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f542400d950 tx=0x7f542400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f5430ff9700 1 -- 172.21.15.46:0/851455975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400f890 con 0x7f5434068750 2024-03-18T20:35:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f5430ff9700 1 -- 172.21.15.46:0/851455975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5424010460 con 0x7f5434068750 2024-03-18T20:35:10.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f5439a67700 1 -- 172.21.15.46:0/851455975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543419c870 con 0x7f5434068750 2024-03-18T20:35:10.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f5430ff9700 1 -- 172.21.15.46:0/851455975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400e610 con 0x7f5434068750 2024-03-18T20:35:10.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.802+0000 7f5439a67700 1 -- 172.21.15.46:0/851455975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54341a40d0 con 0x7f5434068750 2024-03-18T20:35:10.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.804+0000 7f5439a67700 1 -- 172.21.15.46:0/851455975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5434109310 con 0x7f5434068750 2024-03-18T20:35:10.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.805+0000 7f5430ff9700 1 -- 172.21.15.46:0/851455975 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f54240108d0 con 0x7f5434068750 2024-03-18T20:35:10.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.806+0000 7f5430ff9700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f541c0a11b0 0x7f541c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.806+0000 7f5430ff9700 1 -- 172.21.15.46:0/851455975 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f54240cc5e0 con 0x7f5434068750 2024-03-18T20:35:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.807+0000 7f5432ffd700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f541c0a11b0 0x7f541c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.807+0000 7f5432ffd700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f541c0a11b0 0x7f541c0a3670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f54341988c0 tx=0x7f5428009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.811+0000 7f5430ff9700 1 -- 172.21.15.46:0/851455975 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5424095dc0 con 0x7f5434068750 2024-03-18T20:35:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.876+0000 7f8c7f189700 1 -- 172.21.15.46:0/1645901009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c78101950 msgr2=0x7f8c781059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.876+0000 7f8c7f189700 1 --2- 172.21.15.46:0/1645901009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c78101950 0x7f8c781059a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009990 tx=0x7f8c60009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.877+0000 7f8c7f189700 1 -- 172.21.15.46:0/1645901009 shutdown_connections 2024-03-18T20:35:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.877+0000 7f8c7f189700 1 --2- 172.21.15.46:0/1645901009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c78106080 0x7f8c7810dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.877+0000 7f8c7f189700 1 --2- 172.21.15.46:0/1645901009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c78100fa0 0x7f8c78101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.877+0000 7f8c7f189700 1 --2- 172.21.15.46:0/1645901009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c78101950 0x7f8c781059a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.877+0000 7f8c7f189700 1 -- 172.21.15.46:0/1645901009 >> 172.21.15.46:0/1645901009 conn(0x7f8c78078210 msgr2=0x7f8c78078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:10.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.877+0000 7f8c7f189700 1 -- 172.21.15.46:0/1645901009 shutdown_connections 2024-03-18T20:35:10.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.877+0000 7f8c7f189700 1 -- 172.21.15.46:0/1645901009 wait complete. 2024-03-18T20:35:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.877+0000 7f8c7f189700 1 Processor -- start 2024-03-18T20:35:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.877+0000 7f8c7f189700 1 -- start start 2024-03-18T20:35:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7f189700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c78100fa0 0x7f8c780730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7f189700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c78101950 0x7f8c780735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7f189700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c78106080 0x7f8c78073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7f189700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c781a5500 con 0x7f8c78106080 2024-03-18T20:35:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7f189700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c781a5670 con 0x7f8c78101950 2024-03-18T20:35:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7f189700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c781a5830 con 0x7f8c78100fa0 2024-03-18T20:35:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7d726700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c78106080 0x7f8c78073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7d726700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c78106080 0x7f8c78073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50090/0 (socket says 172.21.15.46:50090) 2024-03-18T20:35:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7d726700 1 -- 172.21.15.46:0/1910103400 learned_addr learned my addr 172.21.15.46:0/1910103400 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7cf25700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c78100fa0 0x7f8c780730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c77fff700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c78101950 0x7f8c780735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7d726700 1 -- 172.21.15.46:0/1910103400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c78100fa0 msgr2=0x7f8c780730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7d726700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c78100fa0 0x7f8c780730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7d726700 1 -- 172.21.15.46:0/1910103400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c78101950 msgr2=0x7f8c780735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7d726700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c78101950 0x7f8c780735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7d726700 1 -- 172.21.15.46:0/1910103400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c600095f0 con 0x7f8c78106080 2024-03-18T20:35:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7cf25700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c78100fa0 0x7f8c780730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.878+0000 7f8c7d726700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c78106080 0x7f8c78073b30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000efc0 tx=0x7f8c7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.879+0000 7f8c75ffb700 1 -- 172.21.15.46:0/1910103400 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70018070 con 0x7f8c78106080 2024-03-18T20:35:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.879+0000 7f8c75ffb700 1 -- 172.21.15.46:0/1910103400 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c7000f040 con 0x7f8c78106080 2024-03-18T20:35:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.879+0000 7f8c7f189700 1 -- 172.21.15.46:0/1910103400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c781a5ab0 con 0x7f8c78106080 2024-03-18T20:35:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.879+0000 7f8c75ffb700 1 -- 172.21.15.46:0/1910103400 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7000cc10 con 0x7f8c78106080 2024-03-18T20:35:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.879+0000 7f8c7f189700 1 -- 172.21.15.46:0/1910103400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c781a5f80 con 0x7f8c78106080 2024-03-18T20:35:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.880+0000 7f8c7f189700 1 -- 172.21.15.46:0/1910103400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8c78102480 con 0x7f8c78106080 2024-03-18T20:35:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.881+0000 7f8c75ffb700 1 -- 172.21.15.46:0/1910103400 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8c7000cd70 con 0x7f8c78106080 2024-03-18T20:35:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.882+0000 7f8c75ffb700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8c640a1230 0x7f8c640a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.882+0000 7f8c75ffb700 1 -- 172.21.15.46:0/1910103400 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8c700ccb00 con 0x7f8c78106080 2024-03-18T20:35:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.882+0000 7f8c7cf25700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8c640a1230 0x7f8c640a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.882+0000 7f8c75ffb700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190] conn(0x7f8c640aaa10 0x7f8c640ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.882+0000 7f8c75ffb700 1 -- 172.21.15.46:0/1910103400 --> [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c640ad4e0 con 0x7f8c640aaa10 2024-03-18T20:35:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.882+0000 7f8c75ffb700 1 -- 172.21.15.46:0/1910103400 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c700ccfa0 con 0x7f8c78106080 2024-03-18T20:35:10.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.882+0000 7f8c7cf25700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8c640a1230 0x7f8c640a36f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c68011490 tx=0x7f8c68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.882+0000 7f8c77fff700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190] conn(0x7f8c640aaa10 0x7f8c640ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.883+0000 7f8c77fff700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190] conn(0x7f8c640aaa10 0x7f8c640ace30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:10.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.884+0000 7f8c75ffb700 1 -- 172.21.15.46:0/1910103400 <== osd.7 v2:172.21.15.73:6816/1220123190 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f8c640ad4e0 con 0x7f8c640aaa10 2024-03-18T20:35:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.900+0000 7f8c7f189700 1 -- 172.21.15.46:0/1910103400 --> [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8c78110f70 con 0x7f8c640aaa10 2024-03-18T20:35:10.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.901+0000 7f8c75ffb700 1 -- 172.21.15.46:0/1910103400 <== osd.7 v2:172.21.15.73:6816/1220123190 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8c6001e040 con 0x7f8c640aaa10 2024-03-18T20:35:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.901+0000 7f8c7f189700 1 -- 172.21.15.46:0/1910103400 >> [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190] conn(0x7f8c640aaa10 msgr2=0x7f8c640ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.901+0000 7f8c7f189700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190] conn(0x7f8c640aaa10 0x7f8c640ace30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.901+0000 7f8c7f189700 1 -- 172.21.15.46:0/1910103400 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8c640a1230 msgr2=0x7f8c640a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.901+0000 7f8c7f189700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8c640a1230 0x7f8c640a36f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c68011490 tx=0x7f8c68009260 comp rx=0 tx=0).stop 2024-03-18T20:35:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.901+0000 7f8c7f189700 1 -- 172.21.15.46:0/1910103400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c78106080 msgr2=0x7f8c78073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.901+0000 7f8c7f189700 1 --2- 172.21.15.46:0/1910103400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c78106080 0x7f8c78073b30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000efc0 tx=0x7f8c7000c3a0 comp rx=0 tx=0).stop 2024-03-18T20:35:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.902+0000 7f8c7d726700 1 -- 172.21.15.46:0/1910103400 reap_dead start 2024-03-18T20:35:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.902+0000 7f8c7f189700 1 -- 172.21.15.46:0/1910103400 shutdown_connections 2024-03-18T20:35:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.902+0000 7f8c7f189700 1 -- 172.21.15.46:0/1910103400 >> 172.21.15.46:0/1910103400 conn(0x7f8c78078210 msgr2=0x7f8c780fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.902+0000 7f8c7f189700 1 -- 172.21.15.46:0/1910103400 shutdown_connections 2024-03-18T20:35:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:10.902+0000 7f8c7f189700 1 -- 172.21.15.46:0/1910103400 wait complete. 2024-03-18T20:35:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.064+0000 7f5439a67700 1 -- 172.21.15.46:0/851455975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f543404ea80 con 0x7f5434068750 2024-03-18T20:35:11.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.065+0000 7f5430ff9700 1 -- 172.21.15.46:0/851455975 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5424016090 con 0x7f5434068750 2024-03-18T20:35:11.068 INFO:teuthology.orchestra.run.smithi046.stdout:55834574899 2024-03-18T20:35:11.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.067+0000 7f5439a67700 1 -- 172.21.15.46:0/851455975 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f541c0a11b0 msgr2=0x7f541c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:11.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.067+0000 7f5439a67700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f541c0a11b0 0x7f541c0a3670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f54341988c0 tx=0x7f5428009260 comp rx=0 tx=0).stop 2024-03-18T20:35:11.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.067+0000 7f5439a67700 1 -- 172.21.15.46:0/851455975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5434068750 msgr2=0x7f5434197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:11.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.067+0000 7f5439a67700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5434068750 0x7f5434197360 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f542400d950 tx=0x7f542400dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.067+0000 7f5439a67700 1 -- 172.21.15.46:0/851455975 shutdown_connections 2024-03-18T20:35:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.067+0000 7f5439a67700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f541c0a11b0 0x7f541c0a3670 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.067+0000 7f5439a67700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5434068750 0x7f5434197360 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.067+0000 7f5439a67700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5434106df0 0x7f543419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.067+0000 7f5439a67700 1 --2- 172.21.15.46:0/851455975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54340690f0 0x7f54341978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.067+0000 7f5439a67700 1 -- 172.21.15.46:0/851455975 >> 172.21.15.46:0/851455975 conn(0x7f5434075d50 msgr2=0x7f54340feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.067+0000 7f5439a67700 1 -- 172.21.15.46:0/851455975 shutdown_connections 2024-03-18T20:35:11.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.067+0000 7f5439a67700 1 -- 172.21.15.46:0/851455975 wait complete. 2024-03-18T20:35:11.270 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:11 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/851455975' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T20:35:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.461+0000 7efdaec79700 1 -- 172.21.15.46:0/23159811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda8069a10 msgr2=0x7efda810dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.461+0000 7efdaec79700 1 --2- 172.21.15.46:0/23159811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda8069a10 0x7efda810dc10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efda000b0d0 tx=0x7efda000b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.461+0000 7efdaec79700 1 -- 172.21.15.46:0/23159811 shutdown_connections 2024-03-18T20:35:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.461+0000 7efdaec79700 1 --2- 172.21.15.46:0/23159811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda8069a10 0x7efda810dc10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.461+0000 7efdaec79700 1 --2- 172.21.15.46:0/23159811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efda8106df0 0x7efda81071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.461+0000 7efdaec79700 1 --2- 172.21.15.46:0/23159811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8069050 0x7efda80694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.461+0000 7efdaec79700 1 -- 172.21.15.46:0/23159811 >> 172.21.15.46:0/23159811 conn(0x7efda8076c80 msgr2=0x7efda8077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.461+0000 7efdaec79700 1 -- 172.21.15.46:0/23159811 shutdown_connections 2024-03-18T20:35:11.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.461+0000 7efdaec79700 1 -- 172.21.15.46:0/23159811 wait complete. 2024-03-18T20:35:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.462+0000 7efdaec79700 1 Processor -- start 2024-03-18T20:35:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.462+0000 7efdaec79700 1 -- start start 2024-03-18T20:35:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.462+0000 7efdaec79700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8069050 0x7efda8197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.462+0000 7efdaec79700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda8069a10 0x7efda8197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.462+0000 7efdaec79700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efda8106df0 0x7efda819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.462+0000 7efdaec79700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda819c120 con 0x7efda8069a10 2024-03-18T20:35:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.462+0000 7efdaec79700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda819c290 con 0x7efda8106df0 2024-03-18T20:35:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.462+0000 7efdaec79700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda819c450 con 0x7efda8069050 2024-03-18T20:35:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efda7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda8069a10 0x7efda8197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efda7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda8069a10 0x7efda8197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50104/0 (socket says 172.21.15.46:50104) 2024-03-18T20:35:11.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efda7fff700 1 -- 172.21.15.46:0/4072869939 learned_addr learned my addr 172.21.15.46:0/4072869939 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efdaca15700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8069050 0x7efda8197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efdad216700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efda8106df0 0x7efda819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efda7fff700 1 -- 172.21.15.46:0/4072869939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8069050 msgr2=0x7efda8197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efda7fff700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8069050 0x7efda8197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efda7fff700 1 -- 172.21.15.46:0/4072869939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efda8106df0 msgr2=0x7efda819bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efda7fff700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efda8106df0 0x7efda819bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efda7fff700 1 -- 172.21.15.46:0/4072869939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda0009d00 con 0x7efda8069a10 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efdaca15700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8069050 0x7efda8197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efdad216700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efda8106df0 0x7efda819bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efda7fff700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda8069a10 0x7efda8197760 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efd9400e9a0 tx=0x7efd9400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efda5ffb700 1 -- 172.21.15.46:0/4072869939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94009790 con 0x7efda8069a10 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efdaec79700 1 -- 172.21.15.46:0/4072869939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda819c730 con 0x7efda8069a10 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efda5ffb700 1 -- 172.21.15.46:0/4072869939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd9400cbd0 con 0x7efda8069a10 2024-03-18T20:35:11.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.463+0000 7efda5ffb700 1 -- 172.21.15.46:0/4072869939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94018680 con 0x7efda8069a10 2024-03-18T20:35:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.464+0000 7efdaec79700 1 -- 172.21.15.46:0/4072869939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda80ffe70 con 0x7efda8069a10 2024-03-18T20:35:11.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.465+0000 7efdaec79700 1 -- 172.21.15.46:0/4072869939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda81092f0 con 0x7efda8069a10 2024-03-18T20:35:11.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:11 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/851455975' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T20:35:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.467+0000 7efda5ffb700 1 -- 172.21.15.46:0/4072869939 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7efd9400cd40 con 0x7efda8069a10 2024-03-18T20:35:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.468+0000 7efda5ffb700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efd980a11b0 0x7efd980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.468+0000 7efdaca15700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efd980a11b0 0x7efd980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.468+0000 7efda5ffb700 1 -- 172.21.15.46:0/4072869939 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efd94014070 con 0x7efda8069a10 2024-03-18T20:35:11.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.469+0000 7efdaca15700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efd980a11b0 0x7efd980a3670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005950 tx=0x7efd9c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:11.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.472+0000 7efda5ffb700 1 -- 172.21.15.46:0/4072869939 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efd94096880 con 0x7efda8069a10 2024-03-18T20:35:11.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:11 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/851455975' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T20:35:11.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.692+0000 7efdaec79700 1 -- 172.21.15.46:0/4072869939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7efda81024a0 con 0x7efda8069a10 2024-03-18T20:35:11.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.692+0000 7efda5ffb700 1 -- 172.21.15.46:0/4072869939 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7efd94099ea0 con 0x7efda8069a10 2024-03-18T20:35:11.696 INFO:teuthology.orchestra.run.smithi046.stdout:283467841555 2024-03-18T20:35:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.695+0000 7efdaec79700 1 -- 172.21.15.46:0/4072869939 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efd980a11b0 msgr2=0x7efd980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.695+0000 7efdaec79700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efd980a11b0 0x7efd980a3670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005950 tx=0x7efd9c0058e0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.695+0000 7efdaec79700 1 -- 172.21.15.46:0/4072869939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda8069a10 msgr2=0x7efda8197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.695+0000 7efdaec79700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda8069a10 0x7efda8197760 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efd9400e9a0 tx=0x7efd9400ed60 comp rx=0 tx=0).stop 2024-03-18T20:35:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.695+0000 7efdaec79700 1 -- 172.21.15.46:0/4072869939 shutdown_connections 2024-03-18T20:35:11.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.695+0000 7efdaec79700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efd980a11b0 0x7efd980a3670 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.695+0000 7efdaec79700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efda8106df0 0x7efda819bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.695+0000 7efdaec79700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efda8069050 0x7efda8197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.695+0000 7efdaec79700 1 --2- 172.21.15.46:0/4072869939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efda8069a10 0x7efda8197760 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.695+0000 7efdaec79700 1 -- 172.21.15.46:0/4072869939 >> 172.21.15.46:0/4072869939 conn(0x7efda8076c80 msgr2=0x7efda80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.695+0000 7efdaec79700 1 -- 172.21.15.46:0/4072869939 shutdown_connections 2024-03-18T20:35:11.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:11.695+0000 7efdaec79700 1 -- 172.21.15.46:0/4072869939 wait complete. 2024-03-18T20:35:11.737 INFO:teuthology.orchestra.run.smithi046.stdout:201863462942 2024-03-18T20:35:11.737 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-18T20:35:12.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.016+0000 7f463d82c700 1 -- 172.21.15.46:0/2421737096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46381000f0 msgr2=0x7f463810db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.016+0000 7f463d82c700 1 --2- 172.21.15.46:0/2421737096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46381000f0 0x7f463810db80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f463000b0a0 tx=0x7f463000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.016+0000 7f463d82c700 1 -- 172.21.15.46:0/2421737096 shutdown_connections 2024-03-18T20:35:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.016+0000 7f463d82c700 1 --2- 172.21.15.46:0/2421737096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46381000f0 0x7f463810db80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.016+0000 7f463d82c700 1 --2- 172.21.15.46:0/2421737096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4638105e90 0x7f4638106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.016+0000 7f463d82c700 1 --2- 172.21.15.46:0/2421737096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46380ff750 0x7f46380ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.016+0000 7f463d82c700 1 -- 172.21.15.46:0/2421737096 >> 172.21.15.46:0/2421737096 conn(0x7f4638074fd0 msgr2=0x7f46380753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.016+0000 7f463d82c700 1 -- 172.21.15.46:0/2421737096 shutdown_connections 2024-03-18T20:35:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.017+0000 7f463d82c700 1 -- 172.21.15.46:0/2421737096 wait complete. 2024-03-18T20:35:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.017+0000 7f463d82c700 1 Processor -- start 2024-03-18T20:35:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.017+0000 7f463d82c700 1 -- start start 2024-03-18T20:35:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f463d82c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46380ff750 0x7f4638197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f463d82c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46381000f0 0x7f4638197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:12.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f463d82c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4638105e90 0x7f463819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f463d82c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463819c150 con 0x7f46381000f0 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f463d82c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463819c2c0 con 0x7f4638105e90 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f463d82c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463819c480 con 0x7f46380ff750 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f46367fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46381000f0 0x7f4638197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f46367fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46381000f0 0x7f4638197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50116/0 (socket says 172.21.15.46:50116) 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f46367fc700 1 -- 172.21.15.46:0/870819360 learned_addr learned my addr 172.21.15.46:0/870819360 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f4636ffd700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46380ff750 0x7f4638197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f46377fe700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4638105e90 0x7f463819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f46367fc700 1 -- 172.21.15.46:0/870819360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46380ff750 msgr2=0x7f4638197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f46367fc700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46380ff750 0x7f4638197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f46367fc700 1 -- 172.21.15.46:0/870819360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4638105e90 msgr2=0x7f463819bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f46367fc700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4638105e90 0x7f463819bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f46367fc700 1 -- 172.21.15.46:0/870819360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4630009d00 con 0x7f46381000f0 2024-03-18T20:35:12.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f46367fc700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46381000f0 0x7f4638197790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f462000c920 tx=0x7f462000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:12.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.018+0000 7f463c82a700 1 -- 172.21.15.46:0/870819360 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620004a30 con 0x7f46381000f0 2024-03-18T20:35:12.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.019+0000 7f463d82c700 1 -- 172.21.15.46:0/870819360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463819c760 con 0x7f46381000f0 2024-03-18T20:35:12.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.019+0000 7f463c82a700 1 -- 172.21.15.46:0/870819360 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46200054e0 con 0x7f46381000f0 2024-03-18T20:35:12.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.019+0000 7f463c82a700 1 -- 172.21.15.46:0/870819360 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620007bd0 con 0x7f46381000f0 2024-03-18T20:35:12.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.019+0000 7f463d82c700 1 -- 172.21.15.46:0/870819360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46381a3fc0 con 0x7f46381000f0 2024-03-18T20:35:12.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.020+0000 7f463d82c700 1 -- 172.21.15.46:0/870819360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46381092f0 con 0x7f46381000f0 2024-03-18T20:35:12.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.021+0000 7f463c82a700 1 -- 172.21.15.46:0/870819360 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4620005000 con 0x7f46381000f0 2024-03-18T20:35:12.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.021+0000 7f463c82a700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f46240a11b0 0x7f46240a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:12.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.021+0000 7f4636ffd700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f46240a11b0 0x7f46240a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:12.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.022+0000 7f463c82a700 1 -- 172.21.15.46:0/870819360 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f46200cc410 con 0x7f46381000f0 2024-03-18T20:35:12.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.022+0000 7f4636ffd700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f46240a11b0 0x7f46240a3670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4638068720 tx=0x7f462800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:12.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.026+0000 7f463c82a700 1 -- 172.21.15.46:0/870819360 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4620095bf0 con 0x7f46381000f0 2024-03-18T20:35:12.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.249+0000 7f463d82c700 1 -- 172.21.15.46:0/870819360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f4638198560 con 0x7f46381000f0 2024-03-18T20:35:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.249+0000 7f463c82a700 1 -- 172.21.15.46:0/870819360 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4620099210 con 0x7f46381000f0 2024-03-18T20:35:12.253 INFO:teuthology.orchestra.run.smithi046.stdout:261993005079 2024-03-18T20:35:12.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.252+0000 7f463d82c700 1 -- 172.21.15.46:0/870819360 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f46240a11b0 msgr2=0x7f46240a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.252+0000 7f463d82c700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f46240a11b0 0x7f46240a3670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4638068720 tx=0x7f462800b410 comp rx=0 tx=0).stop 2024-03-18T20:35:12.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.252+0000 7f463d82c700 1 -- 172.21.15.46:0/870819360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46381000f0 msgr2=0x7f4638197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.252+0000 7f463d82c700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46381000f0 0x7f4638197790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f462000c920 tx=0x7f462000cc30 comp rx=0 tx=0).stop 2024-03-18T20:35:12.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.252+0000 7f463d82c700 1 -- 172.21.15.46:0/870819360 shutdown_connections 2024-03-18T20:35:12.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.252+0000 7f463d82c700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f46240a11b0 0x7f46240a3670 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.252+0000 7f463d82c700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4638105e90 0x7f463819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.252+0000 7f463d82c700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46380ff750 0x7f4638197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.252+0000 7f463d82c700 1 --2- 172.21.15.46:0/870819360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46381000f0 0x7f4638197790 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.252+0000 7f463d82c700 1 -- 172.21.15.46:0/870819360 >> 172.21.15.46:0/870819360 conn(0x7f4638074fd0 msgr2=0x7f46380fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.252+0000 7f463d82c700 1 -- 172.21.15.46:0/870819360 shutdown_connections 2024-03-18T20:35:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.252+0000 7f463d82c700 1 -- 172.21.15.46:0/870819360 wait complete. 2024-03-18T20:35:12.319 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841553 got 283467841555 for osd.10 2024-03-18T20:35:12.319 DEBUG:teuthology.parallel:result is None 2024-03-18T20:35:12.353 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:12 smithi046 ceph-mon[43495]: pgmap v235: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T20:35:12.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:12 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/4072869939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-18T20:35:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:12 smithi073 ceph-mon[47384]: pgmap v235: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T20:35:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:12 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/4072869939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-18T20:35:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:12 smithi155 ceph-mon[48344]: pgmap v235: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T20:35:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:12 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/4072869939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-18T20:35:12.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.557+0000 7f75b36a2700 1 -- 172.21.15.46:0/2089305450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ac101820 msgr2=0x7f75ac10dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.557+0000 7f75b36a2700 1 --2- 172.21.15.46:0/2089305450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ac101820 0x7f75ac10dc10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f75a400b0d0 tx=0x7f75a400b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.557+0000 7f75b36a2700 1 -- 172.21.15.46:0/2089305450 shutdown_connections 2024-03-18T20:35:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.557+0000 7f75b36a2700 1 --2- 172.21.15.46:0/2089305450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ac101820 0x7f75ac10dc10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.557+0000 7f75b36a2700 1 --2- 172.21.15.46:0/2089305450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75ac106ef0 0x7f75ac1072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.557+0000 7f75b36a2700 1 --2- 172.21.15.46:0/2089305450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ac100e60 0x7f75ac1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.557+0000 7f75b36a2700 1 -- 172.21.15.46:0/2089305450 >> 172.21.15.46:0/2089305450 conn(0x7f75ac077fa0 msgr2=0x7f75ac0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.557+0000 7f75b36a2700 1 -- 172.21.15.46:0/2089305450 shutdown_connections 2024-03-18T20:35:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.557+0000 7f75b36a2700 1 -- 172.21.15.46:0/2089305450 wait complete. 2024-03-18T20:35:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b36a2700 1 Processor -- start 2024-03-18T20:35:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b36a2700 1 -- start start 2024-03-18T20:35:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b36a2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ac100e60 0x7f75ac199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b36a2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ac101820 0x7f75ac199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b36a2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75ac106ef0 0x7f75ac19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b36a2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac19e410 con 0x7f75ac101820 2024-03-18T20:35:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b36a2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac19e580 con 0x7f75ac100e60 2024-03-18T20:35:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b36a2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac19e740 con 0x7f75ac106ef0 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b0c3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ac101820 0x7f75ac199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b0c3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ac101820 0x7f75ac199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50130/0 (socket says 172.21.15.46:50130) 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b0c3d700 1 -- 172.21.15.46:0/3683322052 learned_addr learned my addr 172.21.15.46:0/3683322052 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b143e700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ac100e60 0x7f75ac199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.558+0000 7f75b1c3f700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75ac106ef0 0x7f75ac19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.559+0000 7f75b0c3d700 1 -- 172.21.15.46:0/3683322052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75ac106ef0 msgr2=0x7f75ac19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.559+0000 7f75b0c3d700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75ac106ef0 0x7f75ac19de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.559+0000 7f75b0c3d700 1 -- 172.21.15.46:0/3683322052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ac100e60 msgr2=0x7f75ac199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.559+0000 7f75b0c3d700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ac100e60 0x7f75ac199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.559+0000 7f75b0c3d700 1 -- 172.21.15.46:0/3683322052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a4009d00 con 0x7f75ac101820 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.559+0000 7f75b0c3d700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ac101820 0x7f75ac199a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f759c00d950 tx=0x7f759c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.559+0000 7f75a27fc700 1 -- 172.21.15.46:0/3683322052 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00f890 con 0x7f75ac101820 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.559+0000 7f75b36a2700 1 -- 172.21.15.46:0/3683322052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ac19ea20 con 0x7f75ac101820 2024-03-18T20:35:12.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.559+0000 7f75a27fc700 1 -- 172.21.15.46:0/3683322052 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f759c010460 con 0x7f75ac101820 2024-03-18T20:35:12.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.559+0000 7f75a27fc700 1 -- 172.21.15.46:0/3683322052 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00e610 con 0x7f75ac101820 2024-03-18T20:35:12.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.559+0000 7f75b36a2700 1 -- 172.21.15.46:0/3683322052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ac1a6280 con 0x7f75ac101820 2024-03-18T20:35:12.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.560+0000 7f75b36a2700 1 -- 172.21.15.46:0/3683322052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ac1024f0 con 0x7f75ac101820 2024-03-18T20:35:12.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.562+0000 7f75a27fc700 1 -- 172.21.15.46:0/3683322052 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f759c0108d0 con 0x7f75ac101820 2024-03-18T20:35:12.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.562+0000 7f75a27fc700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f75980a11b0 0x7f75980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:12.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.562+0000 7f75b143e700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f75980a11b0 0x7f75980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:12.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.562+0000 7f75a27fc700 1 -- 172.21.15.46:0/3683322052 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f759c0cc670 con 0x7f75ac101820 2024-03-18T20:35:12.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.563+0000 7f75b143e700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f75980a11b0 0x7f75980a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009c60 tx=0x7f75a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:12.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.570+0000 7f75a27fc700 1 -- 172.21.15.46:0/3683322052 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f759c095e50 con 0x7f75ac101820 2024-03-18T20:35:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.609+0000 7fa77eebe700 1 -- 172.21.15.46:0/3396815981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa77810d810 msgr2=0x7fa77810fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.609+0000 7fa77eebe700 1 --2- 172.21.15.46:0/3396815981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa77810d810 0x7fa77810fc00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa77000b0a0 tx=0x7fa77000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.610+0000 7fa77eebe700 1 -- 172.21.15.46:0/3396815981 shutdown_connections 2024-03-18T20:35:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.610+0000 7fa77eebe700 1 --2- 172.21.15.46:0/3396815981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa77810d810 0x7fa77810fc00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.610+0000 7fa77eebe700 1 --2- 172.21.15.46:0/3396815981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778072fe0 0x7fa7780733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.610+0000 7fa77eebe700 1 --2- 172.21.15.46:0/3396815981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa778073900 0x7fa77810d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.610+0000 7fa77eebe700 1 -- 172.21.15.46:0/3396815981 >> 172.21.15.46:0/3396815981 conn(0x7fa778077fa0 msgr2=0x7fa7780783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.610+0000 7fa77eebe700 1 -- 172.21.15.46:0/3396815981 shutdown_connections 2024-03-18T20:35:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.610+0000 7fa77eebe700 1 -- 172.21.15.46:0/3396815981 wait complete. 2024-03-18T20:35:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77eebe700 1 Processor -- start 2024-03-18T20:35:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77eebe700 1 -- start start 2024-03-18T20:35:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77eebe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa778072fe0 0x7fa7781995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77eebe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778073900 0x7fa778199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77eebe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa77810d810 0x7fa77819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77eebe700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77819e460 con 0x7fa778072fe0 2024-03-18T20:35:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77eebe700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77819e5d0 con 0x7fa77810d810 2024-03-18T20:35:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77eebe700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77819e790 con 0x7fa778073900 2024-03-18T20:35:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77cc5a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa778072fe0 0x7fa7781995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77cc5a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa778072fe0 0x7fa7781995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50150/0 (socket says 172.21.15.46:50150) 2024-03-18T20:35:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77cc5a700 1 -- 172.21.15.46:0/2538901959 learned_addr learned my addr 172.21.15.46:0/2538901959 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa777fff700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778073900 0x7fa778199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77cc5a700 1 -- 172.21.15.46:0/2538901959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778073900 msgr2=0x7fa778199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77cc5a700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778073900 0x7fa778199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77d45b700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa77810d810 0x7fa77819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77cc5a700 1 -- 172.21.15.46:0/2538901959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa77810d810 msgr2=0x7fa77819de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77cc5a700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa77810d810 0x7fa77819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa77cc5a700 1 -- 172.21.15.46:0/2538901959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa770009d00 con 0x7fa778072fe0 2024-03-18T20:35:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.611+0000 7fa777fff700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778073900 0x7fa778199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.612+0000 7fa77cc5a700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa778072fe0 0x7fa7781995b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00d950 tx=0x7fa76c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.612+0000 7fa775ffb700 1 -- 172.21.15.46:0/2538901959 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c009b40 con 0x7fa778072fe0 2024-03-18T20:35:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.612+0000 7fa77eebe700 1 -- 172.21.15.46:0/2538901959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77819ea70 con 0x7fa778072fe0 2024-03-18T20:35:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.612+0000 7fa775ffb700 1 -- 172.21.15.46:0/2538901959 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa76c009ca0 con 0x7fa778072fe0 2024-03-18T20:35:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.612+0000 7fa775ffb700 1 -- 172.21.15.46:0/2538901959 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00f620 con 0x7fa778072fe0 2024-03-18T20:35:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.612+0000 7fa77eebe700 1 -- 172.21.15.46:0/2538901959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7781a62d0 con 0x7fa778072fe0 2024-03-18T20:35:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.613+0000 7fa77eebe700 1 -- 172.21.15.46:0/2538901959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa778108a20 con 0x7fa778072fe0 2024-03-18T20:35:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.614+0000 7fa775ffb700 1 -- 172.21.15.46:0/2538901959 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa76c010460 con 0x7fa778072fe0 2024-03-18T20:35:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.615+0000 7fa775ffb700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7680a11b0 0x7fa7680a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.615+0000 7fa777fff700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7680a11b0 0x7fa7680a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.615+0000 7fa775ffb700 1 -- 172.21.15.46:0/2538901959 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa76c0ccc50 con 0x7fa778072fe0 2024-03-18T20:35:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.616+0000 7fa777fff700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7680a11b0 0x7fa7680a3670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa764005fd0 tx=0x7fa764005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:12.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.620+0000 7fa775ffb700 1 -- 172.21.15.46:0/2538901959 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa76c096430 con 0x7fa778072fe0 2024-03-18T20:35:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.793+0000 7f75b36a2700 1 -- 172.21.15.46:0/3683322052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f75ac19a820 con 0x7f75ac101820 2024-03-18T20:35:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.793+0000 7f75a27fc700 1 -- 172.21.15.46:0/3683322052 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f759c016090 con 0x7f75ac101820 2024-03-18T20:35:12.797 INFO:teuthology.orchestra.run.smithi046.stdout:77309411376 2024-03-18T20:35:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.797+0000 7f75b36a2700 1 -- 172.21.15.46:0/3683322052 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f75980a11b0 msgr2=0x7f75980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.797+0000 7f75b36a2700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f75980a11b0 0x7f75980a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009c60 tx=0x7f75a8009260 comp rx=0 tx=0).stop 2024-03-18T20:35:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.797+0000 7f75b36a2700 1 -- 172.21.15.46:0/3683322052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ac101820 msgr2=0x7f75ac199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.797+0000 7f75b36a2700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ac101820 0x7f75ac199a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f759c00d950 tx=0x7f759c00dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.797+0000 7f75b36a2700 1 -- 172.21.15.46:0/3683322052 shutdown_connections 2024-03-18T20:35:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.797+0000 7f75b36a2700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f75980a11b0 0x7f75980a3670 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.797+0000 7f75b36a2700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75ac106ef0 0x7f75ac19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.797+0000 7f75b36a2700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ac100e60 0x7f75ac199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.797+0000 7f75b36a2700 1 --2- 172.21.15.46:0/3683322052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ac101820 0x7f75ac199a50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.797+0000 7f75b36a2700 1 -- 172.21.15.46:0/3683322052 >> 172.21.15.46:0/3683322052 conn(0x7f75ac077fa0 msgr2=0x7f75ac10c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.797+0000 7f75b36a2700 1 -- 172.21.15.46:0/3683322052 shutdown_connections 2024-03-18T20:35:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.797+0000 7f75b36a2700 1 -- 172.21.15.46:0/3683322052 wait complete. 2024-03-18T20:35:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.838+0000 7fa77eebe700 1 -- 172.21.15.46:0/2538901959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa77804eab0 con 0x7fa778072fe0 2024-03-18T20:35:12.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.838+0000 7fa775ffb700 1 -- 172.21.15.46:0/2538901959 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa76c020090 con 0x7fa778072fe0 2024-03-18T20:35:12.842 INFO:teuthology.orchestra.run.smithi046.stdout:103079215147 2024-03-18T20:35:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.841+0000 7fa77eebe700 1 -- 172.21.15.46:0/2538901959 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7680a11b0 msgr2=0x7fa7680a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.841+0000 7fa77eebe700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7680a11b0 0x7fa7680a3670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa764005fd0 tx=0x7fa764005ee0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.841+0000 7fa77eebe700 1 -- 172.21.15.46:0/2538901959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa778072fe0 msgr2=0x7fa7781995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.841+0000 7fa77eebe700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa778072fe0 0x7fa7781995b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00d950 tx=0x7fa76c00dd10 comp rx=0 tx=0).stop 2024-03-18T20:35:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.841+0000 7fa77eebe700 1 -- 172.21.15.46:0/2538901959 shutdown_connections 2024-03-18T20:35:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.841+0000 7fa77eebe700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7680a11b0 0x7fa7680a3670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.841+0000 7fa77eebe700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa778072fe0 0x7fa7781995b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.841+0000 7fa77eebe700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa77810d810 0x7fa77819de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.841+0000 7fa77eebe700 1 --2- 172.21.15.46:0/2538901959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa778073900 0x7fa778199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:12.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.841+0000 7fa77eebe700 1 -- 172.21.15.46:0/2538901959 >> 172.21.15.46:0/2538901959 conn(0x7fa778077fa0 msgr2=0x7fa7780fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:12.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.841+0000 7fa77eebe700 1 -- 172.21.15.46:0/2538901959 shutdown_connections 2024-03-18T20:35:12.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.841+0000 7fa77eebe700 1 -- 172.21.15.46:0/2538901959 wait complete. 2024-03-18T20:35:13.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.878+0000 7fcccdf10700 1 -- 172.21.15.46:0/2756558291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccc8106df0 msgr2=0x7fccc81071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:13.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.878+0000 7fcccdf10700 1 --2- 172.21.15.46:0/2756558291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccc8106df0 0x7fccc81071d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009990 tx=0x7fccb8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.878+0000 7fcccdf10700 1 -- 172.21.15.46:0/2756558291 shutdown_connections 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.878+0000 7fcccdf10700 1 --2- 172.21.15.46:0/2756558291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccc8106df0 0x7fccc81071d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.878+0000 7fcccdf10700 1 --2- 172.21.15.46:0/2756558291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc80690f0 0x7fccc810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.878+0000 7fcccdf10700 1 --2- 172.21.15.46:0/2756558291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccc8068750 0x7fccc8068bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.878+0000 7fcccdf10700 1 -- 172.21.15.46:0/2756558291 >> 172.21.15.46:0/2756558291 conn(0x7fccc8075d50 msgr2=0x7fccc8076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.878+0000 7fcccdf10700 1 -- 172.21.15.46:0/2756558291 shutdown_connections 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.878+0000 7fcccdf10700 1 -- 172.21.15.46:0/2756558291 wait complete. 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.879+0000 7fcccdf10700 1 Processor -- start 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.879+0000 7fcccdf10700 1 -- start start 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fcccdf10700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccc8068750 0x7fccc81972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fcccdf10700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc80690f0 0x7fccc8197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fcccdf10700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccc8106df0 0x7fccc819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fcccdf10700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc819c130 con 0x7fccc8068750 2024-03-18T20:35:13.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fcccdf10700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc819c2a0 con 0x7fccc80690f0 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fcccdf10700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc819c460 con 0x7fccc8106df0 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccc77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccc8068750 0x7fccc81972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccc77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccc8068750 0x7fccc81972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50166/0 (socket says 172.21.15.46:50166) 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccc77fe700 1 -- 172.21.15.46:0/2810597865 learned_addr learned my addr 172.21.15.46:0/2810597865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccbeffd700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc80690f0 0x7fccc8197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccc7fff700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccc8106df0 0x7fccc819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccc77fe700 1 -- 172.21.15.46:0/2810597865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccc8106df0 msgr2=0x7fccc819bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccc77fe700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccc8106df0 0x7fccc819bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccc77fe700 1 -- 172.21.15.46:0/2810597865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc80690f0 msgr2=0x7fccc8197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccc77fe700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc80690f0 0x7fccc8197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccc77fe700 1 -- 172.21.15.46:0/2810597865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb80095f0 con 0x7fccc8068750 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccc77fe700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccc8068750 0x7fccc81972f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fccb80095c0 tx=0x7fccb800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccc57fa700 1 -- 172.21.15.46:0/2810597865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb801d070 con 0x7fccc8068750 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fccc57fa700 1 -- 172.21.15.46:0/2810597865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccb80054f0 con 0x7fccc8068750 2024-03-18T20:35:13.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.880+0000 7fcccdf10700 1 -- 172.21.15.46:0/2810597865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc819c6e0 con 0x7fccc8068750 2024-03-18T20:35:13.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.881+0000 7fccc57fa700 1 -- 172.21.15.46:0/2810597865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8004bd0 con 0x7fccc8068750 2024-03-18T20:35:13.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.881+0000 7fcccdf10700 1 -- 172.21.15.46:0/2810597865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc81a3fc0 con 0x7fccc8068750 2024-03-18T20:35:13.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.882+0000 7fcccdf10700 1 -- 172.21.15.46:0/2810597865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc8109310 con 0x7fccc8068750 2024-03-18T20:35:13.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.883+0000 7fccc57fa700 1 -- 172.21.15.46:0/2810597865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fccb800fb80 con 0x7fccc8068750 2024-03-18T20:35:13.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.883+0000 7fccc57fa700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fccb40a11b0 0x7fccb40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:13.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.884+0000 7fccc57fa700 1 -- 172.21.15.46:0/2810597865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fccb80cda10 con 0x7fccc8068750 2024-03-18T20:35:13.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.885+0000 7fccbeffd700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fccb40a11b0 0x7fccb40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:13.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.886+0000 7fccbeffd700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fccb40a11b0 0x7fccb40a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fccc8076390 tx=0x7fccb000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:13.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:12.887+0000 7fccc57fa700 1 -- 172.21.15.46:0/2810597865 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fccb8097280 con 0x7fccc8068750 2024-03-18T20:35:13.030 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574897 got 55834574899 for osd.1 2024-03-18T20:35:13.030 DEBUG:teuthology.parallel:result is None 2024-03-18T20:35:13.103 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:13 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/870819360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-18T20:35:13.103 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:13 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3683322052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T20:35:13.103 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:13 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2538901959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T20:35:13.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.113+0000 7fcccdf10700 1 -- 172.21.15.46:0/2810597865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fccc804ea80 con 0x7fccc8068750 2024-03-18T20:35:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.113+0000 7fccc57fa700 1 -- 172.21.15.46:0/2810597865 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fccb809a8a0 con 0x7fccc8068750 2024-03-18T20:35:13.117 INFO:teuthology.orchestra.run.smithi046.stdout:231928234011 2024-03-18T20:35:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.116+0000 7fcccdf10700 1 -- 172.21.15.46:0/2810597865 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fccb40a11b0 msgr2=0x7fccb40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.116+0000 7fcccdf10700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fccb40a11b0 0x7fccb40a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fccc8076390 tx=0x7fccb000a300 comp rx=0 tx=0).stop 2024-03-18T20:35:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.116+0000 7fcccdf10700 1 -- 172.21.15.46:0/2810597865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccc8068750 msgr2=0x7fccc81972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:13.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.116+0000 7fcccdf10700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccc8068750 0x7fccc81972f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fccb80095c0 tx=0x7fccb800f690 comp rx=0 tx=0).stop 2024-03-18T20:35:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.116+0000 7fcccdf10700 1 -- 172.21.15.46:0/2810597865 shutdown_connections 2024-03-18T20:35:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.116+0000 7fcccdf10700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fccb40a11b0 0x7fccb40a3670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.116+0000 7fcccdf10700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccc8068750 0x7fccc81972f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.116+0000 7fcccdf10700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccc8106df0 0x7fccc819bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.116+0000 7fcccdf10700 1 --2- 172.21.15.46:0/2810597865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc80690f0 0x7fccc8197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.116+0000 7fcccdf10700 1 -- 172.21.15.46:0/2810597865 >> 172.21.15.46:0/2810597865 conn(0x7fccc8075d50 msgr2=0x7fccc80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:13.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.116+0000 7fcccdf10700 1 -- 172.21.15.46:0/2810597865 shutdown_connections 2024-03-18T20:35:13.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.116+0000 7fcccdf10700 1 -- 172.21.15.46:0/2810597865 wait complete. 2024-03-18T20:35:13.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.303+0000 7fc0bb357700 1 -- 172.21.15.46:0/3534321967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4100110 msgr2=0x7fc0b410c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:13.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.303+0000 7fc0bb357700 1 --2- 172.21.15.46:0/3534321967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4100110 0x7fc0b410c2a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00b0a0 tx=0x7fc0ac00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.303+0000 7fc0bb357700 1 -- 172.21.15.46:0/3534321967 shutdown_connections 2024-03-18T20:35:13.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.303+0000 7fc0bb357700 1 --2- 172.21.15.46:0/3534321967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4100110 0x7fc0b410c2a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.303+0000 7fc0bb357700 1 --2- 172.21.15.46:0/3534321967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0b4105560 0x7fc0b4105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.303+0000 7fc0bb357700 1 --2- 172.21.15.46:0/3534321967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0b40ff750 0x7fc0b40ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.304+0000 7fc0bb357700 1 -- 172.21.15.46:0/3534321967 >> 172.21.15.46:0/3534321967 conn(0x7fc0b4074fd0 msgr2=0x7fc0b40753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:13.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.304+0000 7fc0bb357700 1 -- 172.21.15.46:0/3534321967 shutdown_connections 2024-03-18T20:35:13.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.304+0000 7fc0bb357700 1 -- 172.21.15.46:0/3534321967 wait complete. 2024-03-18T20:35:13.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.304+0000 7fc0bb357700 1 Processor -- start 2024-03-18T20:35:13.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.304+0000 7fc0bb357700 1 -- start start 2024-03-18T20:35:13.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0bb357700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0b40ff750 0x7fc0b4072970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:13.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0bb357700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4100110 0x7fc0b406da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0bb357700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0b4105560 0x7fc0b406dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0bb357700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b406e780 con 0x7fc0b4100110 2024-03-18T20:35:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0bb357700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b406e8f0 con 0x7fc0b4105560 2024-03-18T20:35:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0bb357700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b41abf10 con 0x7fc0b40ff750 2024-03-18T20:35:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0b88f2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4100110 0x7fc0b406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0b88f2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4100110 0x7fc0b406da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50170/0 (socket says 172.21.15.46:50170) 2024-03-18T20:35:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0b88f2700 1 -- 172.21.15.46:0/1130292452 learned_addr learned my addr 172.21.15.46:0/1130292452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0b90f3700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0b40ff750 0x7fc0b4072970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0b98f4700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0b4105560 0x7fc0b406dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0b88f2700 1 -- 172.21.15.46:0/1130292452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0b40ff750 msgr2=0x7fc0b4072970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0b88f2700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0b40ff750 0x7fc0b4072970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0b88f2700 1 -- 172.21.15.46:0/1130292452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0b4105560 msgr2=0x7fc0b406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0b88f2700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0b4105560 0x7fc0b406dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0b88f2700 1 -- 172.21.15.46:0/1130292452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac009d00 con 0x7fc0b4100110 2024-03-18T20:35:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0b98f4700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0b4105560 0x7fc0b406dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0b88f2700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4100110 0x7fc0b406da30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400e9a0 tx=0x7fc0a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.305+0000 7fc0aa7fc700 1 -- 172.21.15.46:0/1130292452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4009790 con 0x7fc0b4100110 2024-03-18T20:35:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.306+0000 7fc0bb357700 1 -- 172.21.15.46:0/1130292452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b41ac1f0 con 0x7fc0b4100110 2024-03-18T20:35:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.306+0000 7fc0aa7fc700 1 -- 172.21.15.46:0/1130292452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0a400cbd0 con 0x7fc0b4100110 2024-03-18T20:35:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.306+0000 7fc0aa7fc700 1 -- 172.21.15.46:0/1130292452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4018680 con 0x7fc0b4100110 2024-03-18T20:35:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.306+0000 7fc0bb357700 1 -- 172.21.15.46:0/1130292452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b41ac740 con 0x7fc0b4100110 2024-03-18T20:35:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.307+0000 7fc0bb357700 1 -- 172.21.15.46:0/1130292452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b4100b70 con 0x7fc0b4100110 2024-03-18T20:35:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.308+0000 7fc0aa7fc700 1 -- 172.21.15.46:0/1130292452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc0a400cd40 con 0x7fc0b4100110 2024-03-18T20:35:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.309+0000 7fc0aa7fc700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc0a00a1230 0x7fc0a00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.309+0000 7fc0b90f3700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc0a00a1230 0x7fc0a00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.309+0000 7fc0aa7fc700 1 -- 172.21.15.46:0/1130292452 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc0a4014070 con 0x7fc0b4100110 2024-03-18T20:35:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.309+0000 7fc0b90f3700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc0a00a1230 0x7fc0a00a36f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0005950 tx=0x7fc0b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:13.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.313+0000 7fc0aa7fc700 1 -- 172.21.15.46:0/1130292452 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc0a40529f0 con 0x7fc0b4100110 2024-03-18T20:35:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:13 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/870819360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-18T20:35:13.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:13 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3683322052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T20:35:13.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:13 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2538901959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T20:35:13.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.531+0000 7fc0bb357700 1 -- 172.21.15.46:0/1130292452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc0b41ac960 con 0x7fc0b4100110 2024-03-18T20:35:13.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.532+0000 7fc0aa7fc700 1 -- 172.21.15.46:0/1130292452 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc0a4096280 con 0x7fc0b4100110 2024-03-18T20:35:13.536 INFO:teuthology.orchestra.run.smithi046.stdout:34359738422 2024-03-18T20:35:13.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.534+0000 7fc0bb357700 1 -- 172.21.15.46:0/1130292452 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc0a00a1230 msgr2=0x7fc0a00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:13.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.534+0000 7fc0bb357700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc0a00a1230 0x7fc0a00a36f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0005950 tx=0x7fc0b000b410 comp rx=0 tx=0).stop 2024-03-18T20:35:13.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.534+0000 7fc0bb357700 1 -- 172.21.15.46:0/1130292452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4100110 msgr2=0x7fc0b406da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:13.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.534+0000 7fc0bb357700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4100110 0x7fc0b406da30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400e9a0 tx=0x7fc0a400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.534+0000 7fc0bb357700 1 -- 172.21.15.46:0/1130292452 shutdown_connections 2024-03-18T20:35:13.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.534+0000 7fc0bb357700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc0a00a1230 0x7fc0a00a36f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.535+0000 7fc0bb357700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0b4105560 0x7fc0b406dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.535+0000 7fc0bb357700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0b40ff750 0x7fc0b4072970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.535+0000 7fc0bb357700 1 --2- 172.21.15.46:0/1130292452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4100110 0x7fc0b406da30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:13.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.535+0000 7fc0bb357700 1 -- 172.21.15.46:0/1130292452 >> 172.21.15.46:0/1130292452 conn(0x7fc0b4074fd0 msgr2=0x7fc0b40fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:13.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.535+0000 7fc0bb357700 1 -- 172.21.15.46:0/1130292452 shutdown_connections 2024-03-18T20:35:13.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:13.535+0000 7fc0bb357700 1 -- 172.21.15.46:0/1130292452 wait complete. 2024-03-18T20:35:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:13 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/870819360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-18T20:35:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:13 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3683322052' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T20:35:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:13 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2538901959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T20:35:13.789 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411375 got 77309411376 for osd.2 2024-03-18T20:35:13.789 DEBUG:teuthology.parallel:result is None 2024-03-18T20:35:14.206 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005078 got 261993005079 for osd.9 2024-03-18T20:35:14.206 DEBUG:teuthology.parallel:result is None 2024-03-18T20:35:14.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:14 smithi046 ceph-mon[43495]: pgmap v236: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-18T20:35:14.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:14 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2810597865' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-18T20:35:14.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:14 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1130292452' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T20:35:14.381 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234009 got 231928234011 for osd.8 2024-03-18T20:35:14.381 DEBUG:teuthology.parallel:result is None 2024-03-18T20:35:14.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215145 got 103079215147 for osd.3 2024-03-18T20:35:14.421 DEBUG:teuthology.parallel:result is None 2024-03-18T20:35:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:14 smithi073 ceph-mon[47384]: pgmap v236: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-18T20:35:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:14 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2810597865' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-18T20:35:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:14 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1130292452' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T20:35:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:14 smithi155 ceph-mon[48344]: pgmap v236: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-18T20:35:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:14 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2810597865' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-18T20:35:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:14 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1130292452' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T20:35:14.822 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738420 got 34359738422 for osd.0 2024-03-18T20:35:14.822 DEBUG:teuthology.parallel:result is None 2024-03-18T20:35:14.941 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.031+0000 7f8d97ab1700 1 -- 172.21.15.46:0/1183600454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90073900 msgr2=0x7f8d9010d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.031+0000 7f8d97ab1700 1 --2- 172.21.15.46:0/1183600454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90073900 0x7f8d9010d2d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009990 tx=0x7f8d80009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.031+0000 7f8d97ab1700 1 -- 172.21.15.46:0/1183600454 shutdown_connections 2024-03-18T20:35:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.031+0000 7f8d97ab1700 1 --2- 172.21.15.46:0/1183600454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9010d810 0x7f8d9010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.031+0000 7f8d97ab1700 1 --2- 172.21.15.46:0/1183600454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d90072fe0 0x7f8d900733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.031+0000 7f8d97ab1700 1 --2- 172.21.15.46:0/1183600454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90073900 0x7f8d9010d2d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.031+0000 7f8d97ab1700 1 -- 172.21.15.46:0/1183600454 >> 172.21.15.46:0/1183600454 conn(0x7f8d90077fa0 msgr2=0x7f8d900783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.031+0000 7f8d97ab1700 1 -- 172.21.15.46:0/1183600454 shutdown_connections 2024-03-18T20:35:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.031+0000 7f8d97ab1700 1 -- 172.21.15.46:0/1183600454 wait complete. 2024-03-18T20:35:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d97ab1700 1 Processor -- start 2024-03-18T20:35:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d97ab1700 1 -- start start 2024-03-18T20:35:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d97ab1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90072fe0 0x7f8d901994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d97ab1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d90073900 0x7f8d90199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d97ab1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9010d810 0x7f8d9019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d97ab1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019e3c0 con 0x7f8d90072fe0 2024-03-18T20:35:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d97ab1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019e530 con 0x7f8d9010d810 2024-03-18T20:35:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d97ab1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019e6f0 con 0x7f8d90073900 2024-03-18T20:35:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d9584d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90072fe0 0x7f8d901994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d9584d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90072fe0 0x7f8d901994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50194/0 (socket says 172.21.15.46:50194) 2024-03-18T20:35:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d9584d700 1 -- 172.21.15.46:0/4045101055 learned_addr learned my addr 172.21.15.46:0/4045101055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d9504c700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d90073900 0x7f8d90199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d9604e700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9010d810 0x7f8d9019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d9584d700 1 -- 172.21.15.46:0/4045101055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d90073900 msgr2=0x7f8d90199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d9584d700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d90073900 0x7f8d90199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d9584d700 1 -- 172.21.15.46:0/4045101055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9010d810 msgr2=0x7f8d9019ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d9584d700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9010d810 0x7f8d9019ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.032+0000 7f8d9584d700 1 -- 172.21.15.46:0/4045101055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d800095f0 con 0x7f8d90072fe0 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.033+0000 7f8d9584d700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90072fe0 0x7f8d901994c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00d950 tx=0x7f8d8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.033+0000 7f8d86ffd700 1 -- 172.21.15.46:0/4045101055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00f890 con 0x7f8d90072fe0 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.033+0000 7f8d86ffd700 1 -- 172.21.15.46:0/4045101055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d8c010460 con 0x7f8d90072fe0 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.033+0000 7f8d86ffd700 1 -- 172.21.15.46:0/4045101055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00e610 con 0x7f8d90072fe0 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.033+0000 7f8d97ab1700 1 -- 172.21.15.46:0/4045101055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9019e9d0 con 0x7f8d90072fe0 2024-03-18T20:35:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.033+0000 7f8d97ab1700 1 -- 172.21.15.46:0/4045101055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d901a6230 con 0x7f8d90072fe0 2024-03-18T20:35:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.034+0000 7f8d97ab1700 1 -- 172.21.15.46:0/4045101055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d90108a20 con 0x7f8d90072fe0 2024-03-18T20:35:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.035+0000 7f8d86ffd700 1 -- 172.21.15.46:0/4045101055 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8d8c0108d0 con 0x7f8d90072fe0 2024-03-18T20:35:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.036+0000 7f8d86ffd700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8d7c0a11b0 0x7f8d7c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.036+0000 7f8d9504c700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8d7c0a11b0 0x7f8d7c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.036+0000 7f8d86ffd700 1 -- 172.21.15.46:0/4045101055 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8d8c0cc670 con 0x7f8d90072fe0 2024-03-18T20:35:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.036+0000 7f8d9504c700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8d7c0a11b0 0x7f8d7c0a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d9019aa20 tx=0x7f8d80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.040+0000 7f8d86ffd700 1 -- 172.21.15.46:0/4045101055 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8d8c095e50 con 0x7f8d90072fe0 2024-03-18T20:35:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.124+0000 7f78d4c07700 1 -- 172.21.15.46:0/1595203000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d0100660 msgr2=0x7f78d010cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.125+0000 7f78d4c07700 1 --2- 172.21.15.46:0/1595203000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d0100660 0x7f78d010cb00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f78c800b0a0 tx=0x7f78c800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.125+0000 7f78d4c07700 1 -- 172.21.15.46:0/1595203000 shutdown_connections 2024-03-18T20:35:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.125+0000 7f78d4c07700 1 --2- 172.21.15.46:0/1595203000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d0105d50 0x7f78d0106130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.125+0000 7f78d4c07700 1 --2- 172.21.15.46:0/1595203000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d0100660 0x7f78d010cb00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.125+0000 7f78d4c07700 1 --2- 172.21.15.46:0/1595203000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78d00ffca0 0x7f78d0100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.125+0000 7f78d4c07700 1 -- 172.21.15.46:0/1595203000 >> 172.21.15.46:0/1595203000 conn(0x7f78d0076c80 msgr2=0x7f78d0077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.125+0000 7f78d4c07700 1 -- 172.21.15.46:0/1595203000 shutdown_connections 2024-03-18T20:35:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.125+0000 7f78d4c07700 1 -- 172.21.15.46:0/1595203000 wait complete. 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.126+0000 7f78d4c07700 1 Processor -- start 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.126+0000 7f78d4c07700 1 -- start start 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.126+0000 7f78d4c07700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00ffca0 0x7f78d01950b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.126+0000 7f78d4c07700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d0100660 0x7f78d01955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.126+0000 7f78d4c07700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78d0105d50 0x7f78d01999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.126+0000 7f78d4c07700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d0199fb0 con 0x7f78d00ffca0 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.126+0000 7f78d4c07700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d019a120 con 0x7f78d0100660 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.126+0000 7f78d4c07700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d019a2e0 con 0x7f78d0105d50 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.127+0000 7f78ce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00ffca0 0x7f78d01950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.127+0000 7f78ce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00ffca0 0x7f78d01950b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50214/0 (socket says 172.21.15.46:50214) 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.127+0000 7f78ce59c700 1 -- 172.21.15.46:0/3204574614 learned_addr learned my addr 172.21.15.46:0/3204574614 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.127+0000 7f78ced9d700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78d0105d50 0x7f78d01999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.127+0000 7f78cdd9b700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d0100660 0x7f78d01955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.127+0000 7f78ce59c700 1 -- 172.21.15.46:0/3204574614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78d0105d50 msgr2=0x7f78d01999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.127+0000 7f78ce59c700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78d0105d50 0x7f78d01999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.127+0000 7f78ce59c700 1 -- 172.21.15.46:0/3204574614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d0100660 msgr2=0x7f78d01955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.127+0000 7f78ce59c700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d0100660 0x7f78d01955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.127+0000 7f78ce59c700 1 -- 172.21.15.46:0/3204574614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c8009d00 con 0x7f78d00ffca0 2024-03-18T20:35:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.127+0000 7f78ce59c700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00ffca0 0x7f78d01950b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f78c000e9a0 tx=0x7f78c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.128+0000 7f78bf7fe700 1 -- 172.21.15.46:0/3204574614 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0009790 con 0x7f78d00ffca0 2024-03-18T20:35:15.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.128+0000 7f78bf7fe700 1 -- 172.21.15.46:0/3204574614 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78c000cbd0 con 0x7f78d00ffca0 2024-03-18T20:35:15.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.128+0000 7f78bf7fe700 1 -- 172.21.15.46:0/3204574614 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0018680 con 0x7f78d00ffca0 2024-03-18T20:35:15.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.128+0000 7f78d4c07700 1 -- 172.21.15.46:0/3204574614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d019a5c0 con 0x7f78d00ffca0 2024-03-18T20:35:15.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.128+0000 7f78d4c07700 1 -- 172.21.15.46:0/3204574614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d01a6230 con 0x7f78d00ffca0 2024-03-18T20:35:15.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.129+0000 7f78d4c07700 1 -- 172.21.15.46:0/3204574614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d01081c0 con 0x7f78d00ffca0 2024-03-18T20:35:15.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.131+0000 7f78bf7fe700 1 -- 172.21.15.46:0/3204574614 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f78c000cd40 con 0x7f78d00ffca0 2024-03-18T20:35:15.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.132+0000 7f78bf7fe700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f78b80a1230 0x7f78b80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:15.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.132+0000 7f78bf7fe700 1 -- 172.21.15.46:0/3204574614 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f78c0014070 con 0x7f78d00ffca0 2024-03-18T20:35:15.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.133+0000 7f78cdd9b700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f78b80a1230 0x7f78b80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:15.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.133+0000 7f78cdd9b700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f78b80a1230 0x7f78b80a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f78c400b3c0 tx=0x7f78c400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:15.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.135+0000 7f78bf7fe700 1 -- 172.21.15.46:0/3204574614 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f78c0096d50 con 0x7f78d00ffca0 2024-03-18T20:35:15.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.266+0000 7f8d97ab1700 1 -- 172.21.15.46:0/4045101055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8d90074670 con 0x7f8d90072fe0 2024-03-18T20:35:15.281 INFO:teuthology.orchestra.run.smithi046.stdout:133143986217 2024-03-18T20:35:15.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.267+0000 7f8d86ffd700 1 -- 172.21.15.46:0/4045101055 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8d8c099470 con 0x7f8d90072fe0 2024-03-18T20:35:15.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.269+0000 7f8d97ab1700 1 -- 172.21.15.46:0/4045101055 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8d7c0a11b0 msgr2=0x7f8d7c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:15.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.269+0000 7f8d97ab1700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8d7c0a11b0 0x7f8d7c0a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d9019aa20 tx=0x7f8d80005fd0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.269+0000 7f8d97ab1700 1 -- 172.21.15.46:0/4045101055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90072fe0 msgr2=0x7f8d901994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:15.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.269+0000 7f8d97ab1700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90072fe0 0x7f8d901994c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00d950 tx=0x7f8d8c00dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:15.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.269+0000 7f8d97ab1700 1 -- 172.21.15.46:0/4045101055 shutdown_connections 2024-03-18T20:35:15.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.269+0000 7f8d97ab1700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8d7c0a11b0 0x7f8d7c0a3670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.269+0000 7f8d97ab1700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d90072fe0 0x7f8d901994c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.269+0000 7f8d97ab1700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d9010d810 0x7f8d9019ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.269+0000 7f8d97ab1700 1 --2- 172.21.15.46:0/4045101055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d90073900 0x7f8d90199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.269+0000 7f8d97ab1700 1 -- 172.21.15.46:0/4045101055 >> 172.21.15.46:0/4045101055 conn(0x7f8d90077fa0 msgr2=0x7f8d900fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:15.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.269+0000 7f8d97ab1700 1 -- 172.21.15.46:0/4045101055 shutdown_connections 2024-03-18T20:35:15.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.269+0000 7f8d97ab1700 1 -- 172.21.15.46:0/4045101055 wait complete. 2024-03-18T20:35:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.359+0000 7f78d4c07700 1 -- 172.21.15.46:0/3204574614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f78d01963c0 con 0x7f78d00ffca0 2024-03-18T20:35:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.360+0000 7f78bf7fe700 1 -- 172.21.15.46:0/3204574614 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f78c009a370 con 0x7f78d00ffca0 2024-03-18T20:35:15.363 INFO:teuthology.orchestra.run.smithi046.stdout:313532612624 2024-03-18T20:35:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.362+0000 7f78d4c07700 1 -- 172.21.15.46:0/3204574614 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f78b80a1230 msgr2=0x7f78b80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.362+0000 7f78d4c07700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f78b80a1230 0x7f78b80a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f78c400b3c0 tx=0x7f78c400d040 comp rx=0 tx=0).stop 2024-03-18T20:35:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.362+0000 7f78d4c07700 1 -- 172.21.15.46:0/3204574614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00ffca0 msgr2=0x7f78d01950b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.362+0000 7f78d4c07700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00ffca0 0x7f78d01950b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f78c000e9a0 tx=0x7f78c000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.362+0000 7f78d4c07700 1 -- 172.21.15.46:0/3204574614 shutdown_connections 2024-03-18T20:35:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.362+0000 7f78d4c07700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f78b80a1230 0x7f78b80a36f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.362+0000 7f78d4c07700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d00ffca0 0x7f78d01950b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.362+0000 7f78d4c07700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78d0105d50 0x7f78d01999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.362+0000 7f78d4c07700 1 --2- 172.21.15.46:0/3204574614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d0100660 0x7f78d01955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.362+0000 7f78d4c07700 1 -- 172.21.15.46:0/3204574614 >> 172.21.15.46:0/3204574614 conn(0x7f78d0076c80 msgr2=0x7f78d00fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.362+0000 7f78d4c07700 1 -- 172.21.15.46:0/3204574614 shutdown_connections 2024-03-18T20:35:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:15.363+0000 7f78d4c07700 1 -- 172.21.15.46:0/3204574614 wait complete. 2024-03-18T20:35:15.873 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986216 got 133143986217 for osd.4 2024-03-18T20:35:15.873 DEBUG:teuthology.parallel:result is None 2024-03-18T20:35:15.982 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:15 smithi046 ceph-mon[43495]: pgmap v237: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T20:35:15.982 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:15 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/4045101055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T20:35:15.982 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:15 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3204574614' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-18T20:35:16.009 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612623 got 313532612624 for osd.11 2024-03-18T20:35:16.010 DEBUG:teuthology.parallel:result is None 2024-03-18T20:35:16.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:15 smithi073 ceph-mon[47384]: pgmap v237: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T20:35:16.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:15 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/4045101055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T20:35:16.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:15 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3204574614' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-18T20:35:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:15 smithi155 ceph-mon[48344]: pgmap v237: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T20:35:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:15 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/4045101055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T20:35:16.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:15 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3204574614' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-18T20:35:16.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.306+0000 7fdc8e13c700 1 -- 172.21.15.46:0/248749275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc88106df0 msgr2=0x7fdc881071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.306+0000 7fdc8e13c700 1 --2- 172.21.15.46:0/248749275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc88106df0 0x7fdc881071d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009990 tx=0x7fdc78009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.307+0000 7fdc8e13c700 1 -- 172.21.15.46:0/248749275 shutdown_connections 2024-03-18T20:35:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.307+0000 7fdc8e13c700 1 --2- 172.21.15.46:0/248749275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc88106df0 0x7fdc881071d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.307+0000 7fdc8e13c700 1 --2- 172.21.15.46:0/248749275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880690f0 0x7fdc8810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.307+0000 7fdc8e13c700 1 --2- 172.21.15.46:0/248749275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc88068750 0x7fdc88068bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.307+0000 7fdc8e13c700 1 -- 172.21.15.46:0/248749275 >> 172.21.15.46:0/248749275 conn(0x7fdc88075d50 msgr2=0x7fdc88076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.307+0000 7fdc8e13c700 1 -- 172.21.15.46:0/248749275 shutdown_connections 2024-03-18T20:35:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.307+0000 7fdc8e13c700 1 -- 172.21.15.46:0/248749275 wait complete. 2024-03-18T20:35:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.307+0000 7fdc8e13c700 1 Processor -- start 2024-03-18T20:35:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.308+0000 7fdc8e13c700 1 -- start start 2024-03-18T20:35:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.308+0000 7fdc8e13c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc88068750 0x7fdc88197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.308+0000 7fdc8e13c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880690f0 0x7fdc881976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.308+0000 7fdc8e13c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc88106df0 0x7fdc8819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.308+0000 7fdc8e13c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8819c010 con 0x7fdc88068750 2024-03-18T20:35:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.308+0000 7fdc8e13c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8819c180 con 0x7fdc88106df0 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.308+0000 7fdc8e13c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8819c340 con 0x7fdc880690f0 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.308+0000 7fdc877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc88068750 0x7fdc88197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.308+0000 7fdc877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc88068750 0x7fdc88197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52196/0 (socket says 172.21.15.46:52196) 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.308+0000 7fdc877fe700 1 -- 172.21.15.46:0/829344840 learned_addr learned my addr 172.21.15.46:0/829344840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.308+0000 7fdc86ffd700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880690f0 0x7fdc881976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc87fff700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc88106df0 0x7fdc8819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc877fe700 1 -- 172.21.15.46:0/829344840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880690f0 msgr2=0x7fdc881976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc877fe700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880690f0 0x7fdc881976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc877fe700 1 -- 172.21.15.46:0/829344840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc88106df0 msgr2=0x7fdc8819ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc877fe700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc88106df0 0x7fdc8819ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc877fe700 1 -- 172.21.15.46:0/829344840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc780095f0 con 0x7fdc88068750 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc87fff700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc88106df0 0x7fdc8819ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:16.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc877fe700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc88068750 0x7fdc88197160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdc78000c00 tx=0x7fdc78005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:16.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc84ff9700 1 -- 172.21.15.46:0/829344840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7801d070 con 0x7fdc88068750 2024-03-18T20:35:16.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc8e13c700 1 -- 172.21.15.46:0/829344840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8819c5c0 con 0x7fdc88068750 2024-03-18T20:35:16.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc84ff9700 1 -- 172.21.15.46:0/829344840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc78022470 con 0x7fdc88068750 2024-03-18T20:35:16.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc8e13c700 1 -- 172.21.15.46:0/829344840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc881a3e80 con 0x7fdc88068750 2024-03-18T20:35:16.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.309+0000 7fdc84ff9700 1 -- 172.21.15.46:0/829344840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc78017650 con 0x7fdc88068750 2024-03-18T20:35:16.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.310+0000 7fdc8e13c700 1 -- 172.21.15.46:0/829344840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc88109310 con 0x7fdc88068750 2024-03-18T20:35:16.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.311+0000 7fdc84ff9700 1 -- 172.21.15.46:0/829344840 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fdc780225e0 con 0x7fdc88068750 2024-03-18T20:35:16.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.312+0000 7fdc84ff9700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdc700a11b0 0x7fdc700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:16.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.312+0000 7fdc86ffd700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdc700a11b0 0x7fdc700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:16.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.312+0000 7fdc84ff9700 1 -- 172.21.15.46:0/829344840 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdc780cdd90 con 0x7fdc88068750 2024-03-18T20:35:16.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.313+0000 7fdc86ffd700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdc700a11b0 0x7fdc700a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c005ea0 tx=0x7fdc7c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.317+0000 7fdc84ff9700 1 -- 172.21.15.46:0/829344840 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdc78097600 con 0x7fdc88068750 2024-03-18T20:35:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.525+0000 7fdc8e13c700 1 -- 172.21.15.46:0/829344840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fdc8804ea80 con 0x7fdc88068750 2024-03-18T20:35:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.525+0000 7fdc84ff9700 1 -- 172.21.15.46:0/829344840 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc78027020 con 0x7fdc88068750 2024-03-18T20:35:16.529 INFO:teuthology.orchestra.run.smithi046.stdout:201863462942 2024-03-18T20:35:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.527+0000 7fdc8e13c700 1 -- 172.21.15.46:0/829344840 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdc700a11b0 msgr2=0x7fdc700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.527+0000 7fdc8e13c700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdc700a11b0 0x7fdc700a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c005ea0 tx=0x7fdc7c005e30 comp rx=0 tx=0).stop 2024-03-18T20:35:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.527+0000 7fdc8e13c700 1 -- 172.21.15.46:0/829344840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc88068750 msgr2=0x7fdc88197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.527+0000 7fdc8e13c700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc88068750 0x7fdc88197160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdc78000c00 tx=0x7fdc78005ef0 comp rx=0 tx=0).stop 2024-03-18T20:35:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.528+0000 7fdc8e13c700 1 -- 172.21.15.46:0/829344840 shutdown_connections 2024-03-18T20:35:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.528+0000 7fdc8e13c700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdc700a11b0 0x7fdc700a3670 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.528+0000 7fdc8e13c700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc88068750 0x7fdc88197160 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.528+0000 7fdc8e13c700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc88106df0 0x7fdc8819ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.528+0000 7fdc8e13c700 1 --2- 172.21.15.46:0/829344840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc880690f0 0x7fdc881976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.528+0000 7fdc8e13c700 1 -- 172.21.15.46:0/829344840 >> 172.21.15.46:0/829344840 conn(0x7fdc88075d50 msgr2=0x7fdc880fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:16.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.528+0000 7fdc8e13c700 1 -- 172.21.15.46:0/829344840 shutdown_connections 2024-03-18T20:35:16.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:16.528+0000 7fdc8e13c700 1 -- 172.21.15.46:0/829344840 wait complete. 2024-03-18T20:35:16.908 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:16 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/829344840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-18T20:35:16.910 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462942 got 201863462942 for osd.7 2024-03-18T20:35:16.910 DEBUG:teuthology.parallel:result is None 2024-03-18T20:35:16.910 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-18T20:35:16.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-18T20:35:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:17.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:16 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/829344840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-18T20:35:17.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:16 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/829344840' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-18T20:35:17.940 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:17 smithi046 ceph-mon[43495]: pgmap v238: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T20:35:18.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.131+0000 7f2b210fa700 1 -- 172.21.15.46:0/2868457390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1c073900 msgr2=0x7f2b1c10d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:18.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.131+0000 7f2b210fa700 1 --2- 172.21.15.46:0/2868457390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1c073900 0x7f2b1c10d2d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009990 tx=0x7f2b10009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:18.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.131+0000 7f2b210fa700 1 -- 172.21.15.46:0/2868457390 shutdown_connections 2024-03-18T20:35:18.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.131+0000 7f2b210fa700 1 --2- 172.21.15.46:0/2868457390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b1c10d810 0x7f2b1c10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:18.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.131+0000 7f2b210fa700 1 --2- 172.21.15.46:0/2868457390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b1c072fe0 0x7f2b1c0733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:18.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.131+0000 7f2b210fa700 1 --2- 172.21.15.46:0/2868457390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1c073900 0x7f2b1c10d2d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:18.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.131+0000 7f2b210fa700 1 -- 172.21.15.46:0/2868457390 >> 172.21.15.46:0/2868457390 conn(0x7f2b1c077f10 msgr2=0x7f2b1c078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:18.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.131+0000 7f2b210fa700 1 -- 172.21.15.46:0/2868457390 shutdown_connections 2024-03-18T20:35:18.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.131+0000 7f2b210fa700 1 -- 172.21.15.46:0/2868457390 wait complete. 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.132+0000 7f2b210fa700 1 Processor -- start 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b210fa700 1 -- start start 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b210fa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b1c072fe0 0x7f2b1c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b210fa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b1c073900 0x7f2b1c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b210fa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1c10d810 0x7f2b1c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b210fa700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c19e290 con 0x7f2b1c10d810 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b210fa700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c19e400 con 0x7f2b1c073900 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b210fa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c19e5c0 con 0x7f2b1c072fe0 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b1b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1c10d810 0x7f2b1c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b1b59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1c10d810 0x7f2b1c19dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52214/0 (socket says 172.21.15.46:52214) 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b1b59e700 1 -- 172.21.15.46:0/1755827436 learned_addr learned my addr 172.21.15.46:0/1755827436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b1ad9d700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b1c072fe0 0x7f2b1c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:18.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b1a59c700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b1c073900 0x7f2b1c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:18.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b1b59e700 1 -- 172.21.15.46:0/1755827436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b1c072fe0 msgr2=0x7f2b1c199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b1b59e700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b1c072fe0 0x7f2b1c199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b1b59e700 1 -- 172.21.15.46:0/1755827436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b1c073900 msgr2=0x7f2b1c1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b1b59e700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b1c073900 0x7f2b1c1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.133+0000 7f2b1b59e700 1 -- 172.21.15.46:0/1755827436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b100095f0 con 0x7f2b1c10d810 2024-03-18T20:35:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.134+0000 7f2b1b59e700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1c10d810 0x7f2b1c19dc80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400ef90 tx=0x7f2b1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.134+0000 7f2b03fff700 1 -- 172.21.15.46:0/1755827436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14018070 con 0x7f2b1c10d810 2024-03-18T20:35:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.134+0000 7f2b210fa700 1 -- 172.21.15.46:0/1755827436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1c19e840 con 0x7f2b1c10d810 2024-03-18T20:35:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.134+0000 7f2b03fff700 1 -- 172.21.15.46:0/1755827436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b1400f040 con 0x7f2b1c10d810 2024-03-18T20:35:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.134+0000 7f2b03fff700 1 -- 172.21.15.46:0/1755827436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1400cda0 con 0x7f2b1c10d810 2024-03-18T20:35:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.134+0000 7f2b210fa700 1 -- 172.21.15.46:0/1755827436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1c19eaf0 con 0x7f2b1c10d810 2024-03-18T20:35:18.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.135+0000 7f2b210fa700 1 -- 172.21.15.46:0/1755827436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b1c108a20 con 0x7f2b1c10d810 2024-03-18T20:35:18.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.136+0000 7f2b03fff700 1 -- 172.21.15.46:0/1755827436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f2b14003a00 con 0x7f2b1c10d810 2024-03-18T20:35:18.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.137+0000 7f2b03fff700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2b040a11b0 0x7f2b040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:18.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.137+0000 7f2b1ad9d700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2b040a11b0 0x7f2b040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:18.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.137+0000 7f2b03fff700 1 -- 172.21.15.46:0/1755827436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2b140ccbf0 con 0x7f2b1c10d810 2024-03-18T20:35:18.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.137+0000 7f2b1ad9d700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2b040a11b0 0x7f2b040a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009c60 tx=0x7f2b0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:18.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.140+0000 7f2b03fff700 1 -- 172.21.15.46:0/1755827436 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2b14096460 con 0x7f2b1c10d810 2024-03-18T20:35:18.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:17 smithi073 ceph-mon[47384]: pgmap v238: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T20:35:18.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:17 smithi155 ceph-mon[48344]: pgmap v238: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T20:35:18.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.347+0000 7f2b210fa700 1 -- 172.21.15.46:0/1755827436 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2b1c19ea60 con 0x7f2b040a11b0 2024-03-18T20:35:18.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.358+0000 7f2b03fff700 1 -- 172.21.15.46:0/1755827436 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340772 (secure 0 0 0) 0x7f2b1c19ea60 con 0x7f2b040a11b0 2024-03-18T20:35:18.363 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:18.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.362+0000 7f2b210fa700 1 -- 172.21.15.46:0/1755827436 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2b040a11b0 msgr2=0x7f2b040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:18.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.362+0000 7f2b210fa700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2b040a11b0 0x7f2b040a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009c60 tx=0x7f2b0c009260 comp rx=0 tx=0).stop 2024-03-18T20:35:18.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.362+0000 7f2b210fa700 1 -- 172.21.15.46:0/1755827436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1c10d810 msgr2=0x7f2b1c19dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:18.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.362+0000 7f2b210fa700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1c10d810 0x7f2b1c19dc80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400ef90 tx=0x7f2b1400c3a0 comp rx=0 tx=0).stop 2024-03-18T20:35:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.362+0000 7f2b210fa700 1 -- 172.21.15.46:0/1755827436 shutdown_connections 2024-03-18T20:35:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.362+0000 7f2b210fa700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2b040a11b0 0x7f2b040a3670 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.362+0000 7f2b210fa700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1c10d810 0x7f2b1c19dc80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.362+0000 7f2b210fa700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b1c072fe0 0x7f2b1c199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.362+0000 7f2b210fa700 1 --2- 172.21.15.46:0/1755827436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b1c073900 0x7f2b1c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.362+0000 7f2b210fa700 1 -- 172.21.15.46:0/1755827436 >> 172.21.15.46:0/1755827436 conn(0x7f2b1c077f10 msgr2=0x7f2b1c0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.362+0000 7f2b210fa700 1 -- 172.21.15.46:0/1755827436 shutdown_connections 2024-03-18T20:35:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:18.362+0000 7f2b210fa700 1 -- 172.21.15.46:0/1755827436 wait complete. 2024-03-18T20:35:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-18T20:35:18.741 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-03-18T20:35:17.027556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656642,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":460,"num_read_kb":353,"num_write":412,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336336,"kb_used_data":12144,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605488,"statfs":{"total":1151940427776,"available":1151596019712,"internally_reserved":0,"allocated":12435456,"data_stored":6823783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1138,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":56,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003354"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.902054+0000","last_change":"2024-03-18T20:34:38.817028+0000","last_active":"2024-03-18T20:34:39.902054+0000","last_peered":"2024-03-18T20:34:39.902054+0000","last_clean":"2024-03-18T20:34:39.902054+0000","last_became_active":"2024-03-18T20:34:38.816711+0000","last_became_peered":"2024-03-18T20:34:38.816711+0000","last_unstale":"2024-03-18T20:34:39.902054+0000","last_undegraded":"2024-03-18T20:34:39.902054+0000","last_fullsized":"2024-03-18T20:34:39.902054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:32:16.827141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901345+0000","last_change":"2024-03-18T20:34:38.807350+0000","last_active":"2024-03-18T20:34:39.901345+0000","last_peered":"2024-03-18T20:34:39.901345+0000","last_clean":"2024-03-18T20:34:39.901345+0000","last_became_active":"2024-03-18T20:34:38.807008+0000","last_became_peered":"2024-03-18T20:34:38.807008+0000","last_unstale":"2024-03-18T20:34:39.901345+0000","last_undegraded":"2024-03-18T20:34:39.901345+0000","last_fullsized":"2024-03-18T20:34:39.901345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:35.667384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901221+0000","last_change":"2024-03-18T20:34:38.817045+0000","last_active":"2024-03-18T20:34:39.901221+0000","last_peered":"2024-03-18T20:34:39.901221+0000","last_clean":"2024-03-18T20:34:39.901221+0000","last_became_active":"2024-03-18T20:34:38.816659+0000","last_became_peered":"2024-03-18T20:34:38.816659+0000","last_unstale":"2024-03-18T20:34:39.901221+0000","last_undegraded":"2024-03-18T20:34:39.901221+0000","last_fullsized":"2024-03-18T20:34:39.901221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:04:49.316127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998431+0000","last_change":"2024-03-18T20:34:38.804833+0000","last_active":"2024-03-18T20:34:39.998431+0000","last_peered":"2024-03-18T20:34:39.998431+0000","last_clean":"2024-03-18T20:34:39.998431+0000","last_became_active":"2024-03-18T20:34:38.804486+0000","last_became_peered":"2024-03-18T20:34:38.804486+0000","last_unstale":"2024-03-18T20:34:39.998431+0000","last_undegraded":"2024-03-18T20:34:39.998431+0000","last_fullsized":"2024-03-18T20:34:39.998431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:19:44.266207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996571+0000","last_change":"2024-03-18T20:34:38.797660+0000","last_active":"2024-03-18T20:34:39.996571+0000","last_peered":"2024-03-18T20:34:39.996571+0000","last_clean":"2024-03-18T20:34:39.996571+0000","last_became_active":"2024-03-18T20:34:38.797155+0000","last_became_peered":"2024-03-18T20:34:38.797155+0000","last_unstale":"2024-03-18T20:34:39.996571+0000","last_undegraded":"2024-03-18T20:34:39.996571+0000","last_fullsized":"2024-03-18T20:34:39.996571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:22:11.363453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997430+0000","last_change":"2024-03-18T20:34:38.822891+0000","last_active":"2024-03-18T20:34:39.997430+0000","last_peered":"2024-03-18T20:34:39.997430+0000","last_clean":"2024-03-18T20:34:39.997430+0000","last_became_active":"2024-03-18T20:34:38.822574+0000","last_became_peered":"2024-03-18T20:34:38.822574+0000","last_unstale":"2024-03-18T20:34:39.997430+0000","last_undegraded":"2024-03-18T20:34:39.997430+0000","last_fullsized":"2024-03-18T20:34:39.997430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:42:05.727053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997617+0000","last_change":"2024-03-18T20:34:38.805378+0000","last_active":"2024-03-18T20:34:39.997617+0000","last_peered":"2024-03-18T20:34:39.997617+0000","last_clean":"2024-03-18T20:34:39.997617+0000","last_became_active":"2024-03-18T20:34:38.805130+0000","last_became_peered":"2024-03-18T20:34:38.805130+0000","last_unstale":"2024-03-18T20:34:39.997617+0000","last_undegraded":"2024-03-18T20:34:39.997617+0000","last_fullsized":"2024-03-18T20:34:39.997617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:46:19.420715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198520+0000","last_change":"2024-03-18T20:34:38.801034+0000","last_active":"2024-03-18T20:34:39.198520+0000","last_peered":"2024-03-18T20:34:39.198520+0000","last_clean":"2024-03-18T20:34:39.198520+0000","last_became_active":"2024-03-18T20:34:38.800706+0000","last_became_peered":"2024-03-18T20:34:38.800706+0000","last_unstale":"2024-03-18T20:34:39.198520+0000","last_undegraded":"2024-03-18T20:34:39.198520+0000","last_fullsized":"2024-03-18T20:34:39.198520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:10:22.641105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.823531+0000","last_change":"2024-03-18T20:34:38.822841+0000","last_active":"2024-03-18T20:34:38.823531+0000","last_peered":"2024-03-18T20:34:38.823531+0000","last_clean":"2024-03-18T20:34:38.823531+0000","last_became_active":"2024-03-18T20:34:38.822537+0000","last_became_peered":"2024-03-18T20:34:38.822537+0000","last_unstale":"2024-03-18T20:34:38.823531+0000","last_undegraded":"2024-03-18T20:34:38.823531+0000","last_fullsized":"2024-03-18T20:34:38.823531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:38.741995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997436+0000","last_change":"2024-03-18T20:34:38.814728+0000","last_active":"2024-03-18T20:34:39.997436+0000","last_peered":"2024-03-18T20:34:39.997436+0000","last_clean":"2024-03-18T20:34:39.997436+0000","last_became_active":"2024-03-18T20:34:38.814200+0000","last_became_peered":"2024-03-18T20:34:38.814200+0000","last_unstale":"2024-03-18T20:34:39.997436+0000","last_undegraded":"2024-03-18T20:34:39.997436+0000","last_fullsized":"2024-03-18T20:34:39.997436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:24:54.064770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997025+0000","last_change":"2024-03-18T20:34:38.818054+0000","last_active":"2024-03-18T20:34:39.997025+0000","last_peered":"2024-03-18T20:34:39.997025+0000","last_clean":"2024-03-18T20:34:39.997025+0000","last_became_active":"2024-03-18T20:34:38.817752+0000","last_became_peered":"2024-03-18T20:34:38.817752+0000","last_unstale":"2024-03-18T20:34:39.997025+0000","last_undegraded":"2024-03-18T20:34:39.997025+0000","last_fullsized":"2024-03-18T20:34:39.997025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:01:13.728434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998381+0000","last_change":"2024-03-18T20:34:38.801356+0000","last_active":"2024-03-18T20:34:39.998381+0000","last_peered":"2024-03-18T20:34:39.998381+0000","last_clean":"2024-03-18T20:34:39.998381+0000","last_became_active":"2024-03-18T20:34:38.800978+0000","last_became_peered":"2024-03-18T20:34:38.800978+0000","last_unstale":"2024-03-18T20:34:39.998381+0000","last_undegraded":"2024-03-18T20:34:39.998381+0000","last_fullsized":"2024-03-18T20:34:39.998381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:07.634686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793686+0000","last_change":"2024-03-18T20:34:38.803006+0000","last_active":"2024-03-18T20:34:39.793686+0000","last_peered":"2024-03-18T20:34:39.793686+0000","last_clean":"2024-03-18T20:34:39.793686+0000","last_became_active":"2024-03-18T20:34:38.802454+0000","last_became_peered":"2024-03-18T20:34:38.802454+0000","last_unstale":"2024-03-18T20:34:39.793686+0000","last_undegraded":"2024-03-18T20:34:39.793686+0000","last_fullsized":"2024-03-18T20:34:39.793686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:11:02.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901028+0000","last_change":"2024-03-18T20:34:38.807397+0000","last_active":"2024-03-18T20:34:39.901028+0000","last_peered":"2024-03-18T20:34:39.901028+0000","last_clean":"2024-03-18T20:34:39.901028+0000","last_became_active":"2024-03-18T20:34:38.807072+0000","last_became_peered":"2024-03-18T20:34:38.807072+0000","last_unstale":"2024-03-18T20:34:39.901028+0000","last_undegraded":"2024-03-18T20:34:39.901028+0000","last_fullsized":"2024-03-18T20:34:39.901028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:55:53.832825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.900962+0000","last_change":"2024-03-18T20:34:38.812925+0000","last_active":"2024-03-18T20:34:39.900962+0000","last_peered":"2024-03-18T20:34:39.900962+0000","last_clean":"2024-03-18T20:34:39.900962+0000","last_became_active":"2024-03-18T20:34:38.812597+0000","last_became_peered":"2024-03-18T20:34:38.812597+0000","last_unstale":"2024-03-18T20:34:39.900962+0000","last_undegraded":"2024-03-18T20:34:39.900962+0000","last_fullsized":"2024-03-18T20:34:39.900962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:14:35.103852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.820572+0000","last_change":"2024-03-18T20:34:38.817892+0000","last_active":"2024-03-18T20:34:38.820572+0000","last_peered":"2024-03-18T20:34:38.820572+0000","last_clean":"2024-03-18T20:34:38.820572+0000","last_became_active":"2024-03-18T20:34:38.816261+0000","last_became_peered":"2024-03-18T20:34:38.816261+0000","last_unstale":"2024-03-18T20:34:38.820572+0000","last_undegraded":"2024-03-18T20:34:38.820572+0000","last_fullsized":"2024-03-18T20:34:38.820572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:32.481210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.902148+0000","last_change":"2024-03-18T20:34:38.816966+0000","last_active":"2024-03-18T20:34:39.902148+0000","last_peered":"2024-03-18T20:34:39.902148+0000","last_clean":"2024-03-18T20:34:39.902148+0000","last_became_active":"2024-03-18T20:34:38.816645+0000","last_became_peered":"2024-03-18T20:34:38.816645+0000","last_unstale":"2024-03-18T20:34:39.902148+0000","last_undegraded":"2024-03-18T20:34:39.902148+0000","last_fullsized":"2024-03-18T20:34:39.902148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:39.904353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997076+0000","last_change":"2024-03-18T20:34:38.817005+0000","last_active":"2024-03-18T20:34:39.997076+0000","last_peered":"2024-03-18T20:34:39.997076+0000","last_clean":"2024-03-18T20:34:39.997076+0000","last_became_active":"2024-03-18T20:34:38.816710+0000","last_became_peered":"2024-03-18T20:34:38.816710+0000","last_unstale":"2024-03-18T20:34:39.997076+0000","last_undegraded":"2024-03-18T20:34:39.997076+0000","last_fullsized":"2024-03-18T20:34:39.997076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:30:43.483977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997574+0000","last_change":"2024-03-18T20:34:38.810957+0000","last_active":"2024-03-18T20:34:39.997574+0000","last_peered":"2024-03-18T20:34:39.997574+0000","last_clean":"2024-03-18T20:34:39.997574+0000","last_became_active":"2024-03-18T20:34:38.810615+0000","last_became_peered":"2024-03-18T20:34:38.810615+0000","last_unstale":"2024-03-18T20:34:39.997574+0000","last_undegraded":"2024-03-18T20:34:39.997574+0000","last_fullsized":"2024-03-18T20:34:39.997574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:26:36.544034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:14.861215+0000","last_change":"2024-03-18T20:34:38.337863+0000","last_active":"2024-03-18T20:35:14.861215+0000","last_peered":"2024-03-18T20:35:14.861215+0000","last_clean":"2024-03-18T20:35:14.861215+0000","last_became_active":"2024-03-18T20:34:38.337548+0000","last_became_peered":"2024-03-18T20:34:38.337548+0000","last_unstale":"2024-03-18T20:35:14.861215+0000","last_undegraded":"2024-03-18T20:35:14.861215+0000","last_fullsized":"2024-03-18T20:35:14.861215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:41:40.237882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198626+0000","last_change":"2024-03-18T20:34:38.814321+0000","last_active":"2024-03-18T20:34:39.198626+0000","last_peered":"2024-03-18T20:34:39.198626+0000","last_clean":"2024-03-18T20:34:39.198626+0000","last_became_active":"2024-03-18T20:34:38.813939+0000","last_became_peered":"2024-03-18T20:34:38.813939+0000","last_unstale":"2024-03-18T20:34:39.198626+0000","last_undegraded":"2024-03-18T20:34:39.198626+0000","last_fullsized":"2024-03-18T20:34:39.198626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:05:42.441099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996941+0000","last_change":"2024-03-18T20:34:38.347790+0000","last_active":"2024-03-18T20:34:39.996941+0000","last_peered":"2024-03-18T20:34:39.996941+0000","last_clean":"2024-03-18T20:34:39.996941+0000","last_became_active":"2024-03-18T20:34:38.347317+0000","last_became_peered":"2024-03-18T20:34:38.347317+0000","last_unstale":"2024-03-18T20:34:39.996941+0000","last_undegraded":"2024-03-18T20:34:39.996941+0000","last_fullsized":"2024-03-18T20:34:39.996941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.096480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901274+0000","last_change":"2024-03-18T20:34:38.817116+0000","last_active":"2024-03-18T20:34:39.901274+0000","last_peered":"2024-03-18T20:34:39.901274+0000","last_clean":"2024-03-18T20:34:39.901274+0000","last_became_active":"2024-03-18T20:34:38.816761+0000","last_became_peered":"2024-03-18T20:34:38.816761+0000","last_unstale":"2024-03-18T20:34:39.901274+0000","last_undegraded":"2024-03-18T20:34:39.901274+0000","last_fullsized":"2024-03-18T20:34:39.901274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:44.353533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.796411+0000","last_change":"2024-03-18T20:34:38.339236+0000","last_active":"2024-03-18T20:34:44.796411+0000","last_peered":"2024-03-18T20:34:44.796411+0000","last_clean":"2024-03-18T20:34:44.796411+0000","last_became_active":"2024-03-18T20:34:38.338879+0000","last_became_peered":"2024-03-18T20:34:38.338879+0000","last_unstale":"2024-03-18T20:34:44.796411+0000","last_undegraded":"2024-03-18T20:34:44.796411+0000","last_fullsized":"2024-03-18T20:34:44.796411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:33.340151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.820775+0000","last_change":"2024-03-18T20:34:38.817861+0000","last_active":"2024-03-18T20:34:38.820775+0000","last_peered":"2024-03-18T20:34:38.820775+0000","last_clean":"2024-03-18T20:34:38.820775+0000","last_became_active":"2024-03-18T20:34:38.815976+0000","last_became_peered":"2024-03-18T20:34:38.815976+0000","last_unstale":"2024-03-18T20:34:38.820775+0000","last_undegraded":"2024-03-18T20:34:38.820775+0000","last_fullsized":"2024-03-18T20:34:38.820775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:54.070067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.795018+0000","last_change":"2024-03-18T20:34:38.345758+0000","last_active":"2024-03-18T20:34:44.795018+0000","last_peered":"2024-03-18T20:34:44.795018+0000","last_clean":"2024-03-18T20:34:44.795018+0000","last_became_active":"2024-03-18T20:34:38.345417+0000","last_became_peered":"2024-03-18T20:34:38.345417+0000","last_unstale":"2024-03-18T20:34:44.795018+0000","last_undegraded":"2024-03-18T20:34:44.795018+0000","last_fullsized":"2024-03-18T20:34:44.795018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:25.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759430+0000","last_change":"2024-03-18T20:34:38.818745+0000","last_active":"2024-03-18T20:34:39.759430+0000","last_peered":"2024-03-18T20:34:39.759430+0000","last_clean":"2024-03-18T20:34:39.759430+0000","last_became_active":"2024-03-18T20:34:38.818415+0000","last_became_peered":"2024-03-18T20:34:38.818415+0000","last_unstale":"2024-03-18T20:34:39.759430+0000","last_undegraded":"2024-03-18T20:34:39.759430+0000","last_fullsized":"2024-03-18T20:34:39.759430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:45:49.454498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:12.833432+0000","last_change":"2024-03-18T20:34:38.345804+0000","last_active":"2024-03-18T20:35:12.833432+0000","last_peered":"2024-03-18T20:35:12.833432+0000","last_clean":"2024-03-18T20:35:12.833432+0000","last_became_active":"2024-03-18T20:34:38.345409+0000","last_became_peered":"2024-03-18T20:34:38.345409+0000","last_unstale":"2024-03-18T20:35:12.833432+0000","last_undegraded":"2024-03-18T20:35:12.833432+0000","last_fullsized":"2024-03-18T20:35:12.833432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:00.085412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996703+0000","last_change":"2024-03-18T20:34:38.799821+0000","last_active":"2024-03-18T20:34:39.996703+0000","last_peered":"2024-03-18T20:34:39.996703+0000","last_clean":"2024-03-18T20:34:39.996703+0000","last_became_active":"2024-03-18T20:34:38.799457+0000","last_became_peered":"2024-03-18T20:34:38.799457+0000","last_unstale":"2024-03-18T20:34:39.996703+0000","last_undegraded":"2024-03-18T20:34:39.996703+0000","last_fullsized":"2024-03-18T20:34:39.996703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:39:35.912892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:43.830658+0000","last_change":"2024-03-18T20:34:38.341999+0000","last_active":"2024-03-18T20:34:43.830658+0000","last_peered":"2024-03-18T20:34:43.830658+0000","last_clean":"2024-03-18T20:34:43.830658+0000","last_became_active":"2024-03-18T20:34:38.341431+0000","last_became_peered":"2024-03-18T20:34:38.341431+0000","last_unstale":"2024-03-18T20:34:43.830658+0000","last_undegraded":"2024-03-18T20:34:43.830658+0000","last_fullsized":"2024-03-18T20:34:43.830658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:07:26.711113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759388+0000","last_change":"2024-03-18T20:34:38.804003+0000","last_active":"2024-03-18T20:34:39.759388+0000","last_peered":"2024-03-18T20:34:39.759388+0000","last_clean":"2024-03-18T20:34:39.759388+0000","last_became_active":"2024-03-18T20:34:38.803667+0000","last_became_peered":"2024-03-18T20:34:38.803667+0000","last_unstale":"2024-03-18T20:34:39.759388+0000","last_undegraded":"2024-03-18T20:34:39.759388+0000","last_fullsized":"2024-03-18T20:34:39.759388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:19:58.230790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.822152+0000","last_change":"2024-03-18T20:34:38.346493+0000","last_active":"2024-03-18T20:34:42.822152+0000","last_peered":"2024-03-18T20:34:42.822152+0000","last_clean":"2024-03-18T20:34:42.822152+0000","last_became_active":"2024-03-18T20:34:38.345925+0000","last_became_peered":"2024-03-18T20:34:38.345925+0000","last_unstale":"2024-03-18T20:34:42.822152+0000","last_undegraded":"2024-03-18T20:34:42.822152+0000","last_fullsized":"2024-03-18T20:34:42.822152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:52:47.773860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198488+0000","last_change":"2024-03-18T20:34:38.807284+0000","last_active":"2024-03-18T20:34:39.198488+0000","last_peered":"2024-03-18T20:34:39.198488+0000","last_clean":"2024-03-18T20:34:39.198488+0000","last_became_active":"2024-03-18T20:34:38.806844+0000","last_became_peered":"2024-03-18T20:34:38.806844+0000","last_unstale":"2024-03-18T20:34:39.198488+0000","last_undegraded":"2024-03-18T20:34:39.198488+0000","last_fullsized":"2024-03-18T20:34:39.198488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:44:35.469953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.793789+0000","last_change":"2024-03-18T20:34:38.343090+0000","last_active":"2024-03-18T20:34:44.793789+0000","last_peered":"2024-03-18T20:34:44.793789+0000","last_clean":"2024-03-18T20:34:44.793789+0000","last_became_active":"2024-03-18T20:34:38.342712+0000","last_became_peered":"2024-03-18T20:34:38.342712+0000","last_unstale":"2024-03-18T20:34:44.793789+0000","last_undegraded":"2024-03-18T20:34:44.793789+0000","last_fullsized":"2024-03-18T20:34:44.793789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:14:05.909604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996769+0000","last_change":"2024-03-18T20:34:38.800616+0000","last_active":"2024-03-18T20:34:39.996769+0000","last_peered":"2024-03-18T20:34:39.996769+0000","last_clean":"2024-03-18T20:34:39.996769+0000","last_became_active":"2024-03-18T20:34:38.800340+0000","last_became_peered":"2024-03-18T20:34:38.800340+0000","last_unstale":"2024-03-18T20:34:39.996769+0000","last_undegraded":"2024-03-18T20:34:39.996769+0000","last_fullsized":"2024-03-18T20:34:39.996769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:18.263179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:11.830931+0000","last_change":"2024-03-18T20:34:38.345634+0000","last_active":"2024-03-18T20:35:11.830931+0000","last_peered":"2024-03-18T20:35:11.830931+0000","last_clean":"2024-03-18T20:35:11.830931+0000","last_became_active":"2024-03-18T20:34:38.345275+0000","last_became_peered":"2024-03-18T20:34:38.345275+0000","last_unstale":"2024-03-18T20:35:11.830931+0000","last_undegraded":"2024-03-18T20:35:11.830931+0000","last_fullsized":"2024-03-18T20:35:11.830931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:29.939529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.823111+0000","last_change":"2024-03-18T20:34:38.820286+0000","last_active":"2024-03-18T20:34:38.823111+0000","last_peered":"2024-03-18T20:34:38.823111+0000","last_clean":"2024-03-18T20:34:38.823111+0000","last_became_active":"2024-03-18T20:34:38.819939+0000","last_became_peered":"2024-03-18T20:34:38.819939+0000","last_unstale":"2024-03-18T20:34:38.823111+0000","last_undegraded":"2024-03-18T20:34:38.823111+0000","last_fullsized":"2024-03-18T20:34:38.823111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:19.229339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.794842+0000","last_change":"2024-03-18T20:34:38.341699+0000","last_active":"2024-03-18T20:34:44.794842+0000","last_peered":"2024-03-18T20:34:44.794842+0000","last_clean":"2024-03-18T20:34:44.794842+0000","last_became_active":"2024-03-18T20:34:38.341390+0000","last_became_peered":"2024-03-18T20:34:38.341390+0000","last_unstale":"2024-03-18T20:34:44.794842+0000","last_undegraded":"2024-03-18T20:34:44.794842+0000","last_fullsized":"2024-03-18T20:34:44.794842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:16:02.239414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901314+0000","last_change":"2024-03-18T20:34:38.805316+0000","last_active":"2024-03-18T20:34:39.901314+0000","last_peered":"2024-03-18T20:34:39.901314+0000","last_clean":"2024-03-18T20:34:39.901314+0000","last_became_active":"2024-03-18T20:34:38.804967+0000","last_became_peered":"2024-03-18T20:34:38.804967+0000","last_unstale":"2024-03-18T20:34:39.901314+0000","last_undegraded":"2024-03-18T20:34:39.901314+0000","last_fullsized":"2024-03-18T20:34:39.901314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:54.591401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:43.829682+0000","last_change":"2024-03-18T20:34:38.346821+0000","last_active":"2024-03-18T20:34:43.829682+0000","last_peered":"2024-03-18T20:34:43.829682+0000","last_clean":"2024-03-18T20:34:43.829682+0000","last_became_active":"2024-03-18T20:34:38.346314+0000","last_became_peered":"2024-03-18T20:34:38.346314+0000","last_unstale":"2024-03-18T20:34:43.829682+0000","last_undegraded":"2024-03-18T20:34:43.829682+0000","last_fullsized":"2024-03-18T20:34:43.829682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:12:04.552971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.823287+0000","last_change":"2024-03-18T20:34:38.820479+0000","last_active":"2024-03-18T20:34:38.823287+0000","last_peered":"2024-03-18T20:34:38.823287+0000","last_clean":"2024-03-18T20:34:38.823287+0000","last_became_active":"2024-03-18T20:34:38.820186+0000","last_became_peered":"2024-03-18T20:34:38.820186+0000","last_unstale":"2024-03-18T20:34:38.823287+0000","last_undegraded":"2024-03-18T20:34:38.823287+0000","last_fullsized":"2024-03-18T20:34:38.823287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:15.953764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'29","reported_seq":103,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:11.830866+0000","last_change":"2024-03-18T20:34:38.344332+0000","last_active":"2024-03-18T20:35:11.830866+0000","last_peered":"2024-03-18T20:35:11.830866+0000","last_clean":"2024-03-18T20:35:11.830866+0000","last_became_active":"2024-03-18T20:34:38.343827+0000","last_became_peered":"2024-03-18T20:34:38.343827+0000","last_unstale":"2024-03-18T20:35:11.830866+0000","last_undegraded":"2024-03-18T20:35:11.830866+0000","last_fullsized":"2024-03-18T20:35:11.830866+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:13:21.725004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59418,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":62,"num_read_kb":89,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996827+0000","last_change":"2024-03-18T20:34:38.814794+0000","last_active":"2024-03-18T20:34:39.996827+0000","last_peered":"2024-03-18T20:34:39.996827+0000","last_clean":"2024-03-18T20:34:39.996827+0000","last_became_active":"2024-03-18T20:34:38.814434+0000","last_became_peered":"2024-03-18T20:34:38.814434+0000","last_unstale":"2024-03-18T20:34:39.996827+0000","last_undegraded":"2024-03-18T20:34:39.996827+0000","last_fullsized":"2024-03-18T20:34:39.996827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:43:19.083212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.794905+0000","last_change":"2024-03-18T20:34:38.343894+0000","last_active":"2024-03-18T20:34:44.794905+0000","last_peered":"2024-03-18T20:34:44.794905+0000","last_clean":"2024-03-18T20:34:44.794905+0000","last_became_active":"2024-03-18T20:34:38.343493+0000","last_became_peered":"2024-03-18T20:34:38.343493+0000","last_unstale":"2024-03-18T20:34:44.794905+0000","last_undegraded":"2024-03-18T20:34:44.794905+0000","last_fullsized":"2024-03-18T20:34:44.794905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:13:32.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793766+0000","last_change":"2024-03-18T20:34:38.809968+0000","last_active":"2024-03-18T20:34:39.793766+0000","last_peered":"2024-03-18T20:34:39.793766+0000","last_clean":"2024-03-18T20:34:39.793766+0000","last_became_active":"2024-03-18T20:34:38.809641+0000","last_became_peered":"2024-03-18T20:34:38.809641+0000","last_unstale":"2024-03-18T20:34:39.793766+0000","last_undegraded":"2024-03-18T20:34:39.793766+0000","last_fullsized":"2024-03-18T20:34:39.793766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:50:52.925683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.793952+0000","last_change":"2024-03-18T20:34:38.345621+0000","last_active":"2024-03-18T20:34:44.793952+0000","last_peered":"2024-03-18T20:34:44.793952+0000","last_clean":"2024-03-18T20:34:44.793952+0000","last_became_active":"2024-03-18T20:34:38.345255+0000","last_became_peered":"2024-03-18T20:34:38.345255+0000","last_unstale":"2024-03-18T20:34:44.793952+0000","last_undegraded":"2024-03-18T20:34:44.793952+0000","last_fullsized":"2024-03-18T20:34:44.793952+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:17:03.737973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997875+0000","last_change":"2024-03-18T20:34:38.804390+0000","last_active":"2024-03-18T20:34:39.997875+0000","last_peered":"2024-03-18T20:34:39.997875+0000","last_clean":"2024-03-18T20:34:39.997875+0000","last_became_active":"2024-03-18T20:34:38.804033+0000","last_became_peered":"2024-03-18T20:34:38.804033+0000","last_unstale":"2024-03-18T20:34:39.997875+0000","last_undegraded":"2024-03-18T20:34:39.997875+0000","last_fullsized":"2024-03-18T20:34:39.997875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:09:45.407921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:11.836908+0000","last_change":"2024-03-18T20:34:38.340239+0000","last_active":"2024-03-18T20:35:11.836908+0000","last_peered":"2024-03-18T20:35:11.836908+0000","last_clean":"2024-03-18T20:35:11.836908+0000","last_became_active":"2024-03-18T20:34:38.339880+0000","last_became_peered":"2024-03-18T20:34:38.339880+0000","last_unstale":"2024-03-18T20:35:11.836908+0000","last_undegraded":"2024-03-18T20:35:11.836908+0000","last_fullsized":"2024-03-18T20:35:11.836908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:55:46.864885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998003+0000","last_change":"2024-03-18T20:34:38.819223+0000","last_active":"2024-03-18T20:34:39.998003+0000","last_peered":"2024-03-18T20:34:39.998003+0000","last_clean":"2024-03-18T20:34:39.998003+0000","last_became_active":"2024-03-18T20:34:38.818879+0000","last_became_peered":"2024-03-18T20:34:38.818879+0000","last_unstale":"2024-03-18T20:34:39.998003+0000","last_undegraded":"2024-03-18T20:34:39.998003+0000","last_fullsized":"2024-03-18T20:34:39.998003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:45.645912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.785345+0000","last_change":"2024-03-18T20:34:38.336817+0000","last_active":"2024-03-18T20:34:44.785345+0000","last_peered":"2024-03-18T20:34:44.785345+0000","last_clean":"2024-03-18T20:34:44.785345+0000","last_became_active":"2024-03-18T20:34:38.336453+0000","last_became_peered":"2024-03-18T20:34:38.336453+0000","last_unstale":"2024-03-18T20:34:44.785345+0000","last_undegraded":"2024-03-18T20:34:44.785345+0000","last_fullsized":"2024-03-18T20:34:44.785345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:10.253435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759322+0000","last_change":"2024-03-18T20:34:38.818048+0000","last_active":"2024-03-18T20:34:39.759322+0000","last_peered":"2024-03-18T20:34:39.759322+0000","last_clean":"2024-03-18T20:34:39.759322+0000","last_became_active":"2024-03-18T20:34:38.817682+0000","last_became_peered":"2024-03-18T20:34:38.817682+0000","last_unstale":"2024-03-18T20:34:39.759322+0000","last_undegraded":"2024-03-18T20:34:39.759322+0000","last_fullsized":"2024-03-18T20:34:39.759322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:25:55.291815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.785348+0000","last_change":"2024-03-18T20:34:38.345773+0000","last_active":"2024-03-18T20:34:44.785348+0000","last_peered":"2024-03-18T20:34:44.785348+0000","last_clean":"2024-03-18T20:34:44.785348+0000","last_became_active":"2024-03-18T20:34:38.345432+0000","last_became_peered":"2024-03-18T20:34:38.345432+0000","last_unstale":"2024-03-18T20:34:44.785348+0000","last_undegraded":"2024-03-18T20:34:44.785348+0000","last_fullsized":"2024-03-18T20:34:44.785348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:56:57.657424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901903+0000","last_change":"2024-03-18T20:34:38.806138+0000","last_active":"2024-03-18T20:34:39.901903+0000","last_peered":"2024-03-18T20:34:39.901903+0000","last_clean":"2024-03-18T20:34:39.901903+0000","last_became_active":"2024-03-18T20:34:38.805821+0000","last_became_peered":"2024-03-18T20:34:38.805821+0000","last_unstale":"2024-03-18T20:34:39.901903+0000","last_undegraded":"2024-03-18T20:34:39.901903+0000","last_fullsized":"2024-03-18T20:34:39.901903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:25:57.887813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.173015+0000","last_change":"2024-03-18T20:34:38.338751+0000","last_active":"2024-03-18T20:34:41.173015+0000","last_peered":"2024-03-18T20:34:41.173015+0000","last_clean":"2024-03-18T20:34:41.173015+0000","last_became_active":"2024-03-18T20:34:38.337495+0000","last_became_peered":"2024-03-18T20:34:38.337495+0000","last_unstale":"2024-03-18T20:34:41.173015+0000","last_undegraded":"2024-03-18T20:34:41.173015+0000","last_fullsized":"2024-03-18T20:34:41.173015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:38:04.402380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.784913+0000","last_change":"2024-03-18T20:34:38.803937+0000","last_active":"2024-03-18T20:34:42.784913+0000","last_peered":"2024-03-18T20:34:42.784913+0000","last_clean":"2024-03-18T20:34:42.784913+0000","last_became_active":"2024-03-18T20:34:38.803520+0000","last_became_peered":"2024-03-18T20:34:38.803520+0000","last_unstale":"2024-03-18T20:34:42.784913+0000","last_undegraded":"2024-03-18T20:34:42.784913+0000","last_fullsized":"2024-03-18T20:34:42.784913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:20:37.537265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:13.860539+0000","last_change":"2024-03-18T20:34:38.337481+0000","last_active":"2024-03-18T20:35:13.860539+0000","last_peered":"2024-03-18T20:35:13.860539+0000","last_clean":"2024-03-18T20:35:13.860539+0000","last_became_active":"2024-03-18T20:34:38.337152+0000","last_became_peered":"2024-03-18T20:34:38.337152+0000","last_unstale":"2024-03-18T20:35:13.860539+0000","last_undegraded":"2024-03-18T20:35:13.860539+0000","last_fullsized":"2024-03-18T20:35:13.860539+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:07:59.593978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997194+0000","last_change":"2024-03-18T20:34:38.814240+0000","last_active":"2024-03-18T20:34:39.997194+0000","last_peered":"2024-03-18T20:34:39.997194+0000","last_clean":"2024-03-18T20:34:39.997194+0000","last_became_active":"2024-03-18T20:34:38.813177+0000","last_became_peered":"2024-03-18T20:34:38.813177+0000","last_unstale":"2024-03-18T20:34:39.997194+0000","last_undegraded":"2024-03-18T20:34:39.997194+0000","last_fullsized":"2024-03-18T20:34:39.997194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:22:30.076880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759286+0000","last_change":"2024-03-18T20:34:38.806133+0000","last_active":"2024-03-18T20:34:39.759286+0000","last_peered":"2024-03-18T20:34:39.759286+0000","last_clean":"2024-03-18T20:34:39.759286+0000","last_became_active":"2024-03-18T20:34:38.805257+0000","last_became_peered":"2024-03-18T20:34:38.805257+0000","last_unstale":"2024-03-18T20:34:39.759286+0000","last_undegraded":"2024-03-18T20:34:39.759286+0000","last_fullsized":"2024-03-18T20:34:39.759286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:05:15.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":46,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:10.834967+0000","last_change":"2024-03-18T20:34:38.346271+0000","last_active":"2024-03-18T20:35:10.834967+0000","last_peered":"2024-03-18T20:35:10.834967+0000","last_clean":"2024-03-18T20:35:10.834967+0000","last_became_active":"2024-03-18T20:34:38.345054+0000","last_became_peered":"2024-03-18T20:34:38.345054+0000","last_unstale":"2024-03-18T20:35:10.834967+0000","last_undegraded":"2024-03-18T20:35:10.834967+0000","last_fullsized":"2024-03-18T20:35:10.834967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:42:52.510294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198582+0000","last_change":"2024-03-18T20:34:38.816834+0000","last_active":"2024-03-18T20:34:39.198582+0000","last_peered":"2024-03-18T20:34:39.198582+0000","last_clean":"2024-03-18T20:34:39.198582+0000","last_became_active":"2024-03-18T20:34:38.816516+0000","last_became_peered":"2024-03-18T20:34:38.816516+0000","last_unstale":"2024-03-18T20:34:39.198582+0000","last_undegraded":"2024-03-18T20:34:39.198582+0000","last_fullsized":"2024-03-18T20:34:39.198582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:33:54.529206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.793259+0000","last_change":"2024-03-18T20:34:38.339407+0000","last_active":"2024-03-18T20:34:44.793259+0000","last_peered":"2024-03-18T20:34:44.793259+0000","last_clean":"2024-03-18T20:34:44.793259+0000","last_became_active":"2024-03-18T20:34:38.339086+0000","last_became_peered":"2024-03-18T20:34:38.339086+0000","last_unstale":"2024-03-18T20:34:44.793259+0000","last_undegraded":"2024-03-18T20:34:44.793259+0000","last_fullsized":"2024-03-18T20:34:44.793259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:18.479630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":331,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998119+0000","last_change":"2024-03-18T20:33:50.777651+0000","last_active":"2024-03-18T20:34:39.998119+0000","last_peered":"2024-03-18T20:34:39.998119+0000","last_clean":"2024-03-18T20:34:39.998119+0000","last_became_active":"2024-03-18T20:33:50.777281+0000","last_became_peered":"2024-03-18T20:33:50.777281+0000","last_unstale":"2024-03-18T20:34:39.998119+0000","last_undegraded":"2024-03-18T20:34:39.998119+0000","last_fullsized":"2024-03-18T20:34:39.998119+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_clean_scrub_stamp":"2024-03-18T20:31:27.488744+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:28.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793222+0000","last_change":"2024-03-18T20:34:38.812714+0000","last_active":"2024-03-18T20:34:39.793222+0000","last_peered":"2024-03-18T20:34:39.793222+0000","last_clean":"2024-03-18T20:34:39.793222+0000","last_became_active":"2024-03-18T20:34:38.812464+0000","last_became_peered":"2024-03-18T20:34:38.812464+0000","last_unstale":"2024-03-18T20:34:39.793222+0000","last_undegraded":"2024-03-18T20:34:39.793222+0000","last_fullsized":"2024-03-18T20:34:39.793222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:03:13.853740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.820233+0000","last_change":"2024-03-18T20:34:38.343173+0000","last_active":"2024-03-18T20:34:42.820233+0000","last_peered":"2024-03-18T20:34:42.820233+0000","last_clean":"2024-03-18T20:34:42.820233+0000","last_became_active":"2024-03-18T20:34:38.342774+0000","last_became_peered":"2024-03-18T20:34:38.342774+0000","last_unstale":"2024-03-18T20:34:42.820233+0000","last_undegraded":"2024-03-18T20:34:42.820233+0000","last_fullsized":"2024-03-18T20:34:42.820233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:52.283033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.785384+0000","last_change":"2024-03-18T20:34:38.344539+0000","last_active":"2024-03-18T20:34:44.785384+0000","last_peered":"2024-03-18T20:34:44.785384+0000","last_clean":"2024-03-18T20:34:44.785384+0000","last_became_active":"2024-03-18T20:34:38.344188+0000","last_became_peered":"2024-03-18T20:34:38.344188+0000","last_unstale":"2024-03-18T20:34:44.785384+0000","last_undegraded":"2024-03-18T20:34:44.785384+0000","last_fullsized":"2024-03-18T20:34:44.785384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:38.641928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793487+0000","last_change":"2024-03-18T20:34:38.806602+0000","last_active":"2024-03-18T20:34:39.793487+0000","last_peered":"2024-03-18T20:34:39.793487+0000","last_clean":"2024-03-18T20:34:39.793487+0000","last_became_active":"2024-03-18T20:34:38.806241+0000","last_became_peered":"2024-03-18T20:34:38.806241+0000","last_unstale":"2024-03-18T20:34:39.793487+0000","last_undegraded":"2024-03-18T20:34:39.793487+0000","last_fullsized":"2024-03-18T20:34:39.793487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:04.463311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'9","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:47.937001+0000","last_change":"2024-03-18T20:34:38.339695+0000","last_active":"2024-03-18T20:34:47.937001+0000","last_peered":"2024-03-18T20:34:47.937001+0000","last_clean":"2024-03-18T20:34:47.937001+0000","last_became_active":"2024-03-18T20:34:38.339331+0000","last_became_peered":"2024-03-18T20:34:38.339331+0000","last_unstale":"2024-03-18T20:34:47.937001+0000","last_undegraded":"2024-03-18T20:34:47.937001+0000","last_fullsized":"2024-03-18T20:34:47.937001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:57:06.434289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198695+0000","last_change":"2024-03-18T20:34:38.809754+0000","last_active":"2024-03-18T20:34:39.198695+0000","last_peered":"2024-03-18T20:34:39.198695+0000","last_clean":"2024-03-18T20:34:39.198695+0000","last_became_active":"2024-03-18T20:34:38.809415+0000","last_became_peered":"2024-03-18T20:34:38.809415+0000","last_unstale":"2024-03-18T20:34:39.198695+0000","last_undegraded":"2024-03-18T20:34:39.198695+0000","last_fullsized":"2024-03-18T20:34:39.198695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:34:30.761582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.793801+0000","last_change":"2024-03-18T20:34:38.342080+0000","last_active":"2024-03-18T20:34:44.793801+0000","last_peered":"2024-03-18T20:34:44.793801+0000","last_clean":"2024-03-18T20:34:44.793801+0000","last_became_active":"2024-03-18T20:34:38.340623+0000","last_became_peered":"2024-03-18T20:34:38.340623+0000","last_unstale":"2024-03-18T20:34:44.793801+0000","last_undegraded":"2024-03-18T20:34:44.793801+0000","last_fullsized":"2024-03-18T20:34:44.793801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:49:14.912391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901335+0000","last_change":"2024-03-18T20:34:38.806558+0000","last_active":"2024-03-18T20:34:39.901335+0000","last_peered":"2024-03-18T20:34:39.901335+0000","last_clean":"2024-03-18T20:34:39.901335+0000","last_became_active":"2024-03-18T20:34:38.806171+0000","last_became_peered":"2024-03-18T20:34:38.806171+0000","last_unstale":"2024-03-18T20:34:39.901335+0000","last_undegraded":"2024-03-18T20:34:39.901335+0000","last_fullsized":"2024-03-18T20:34:39.901335+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:46:26.530157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.813762+0000","last_change":"2024-03-18T20:34:38.336175+0000","last_active":"2024-03-18T20:34:41.813762+0000","last_peered":"2024-03-18T20:34:41.813762+0000","last_clean":"2024-03-18T20:34:41.813762+0000","last_became_active":"2024-03-18T20:34:38.335798+0000","last_became_peered":"2024-03-18T20:34:38.335798+0000","last_unstale":"2024-03-18T20:34:41.813762+0000","last_undegraded":"2024-03-18T20:34:41.813762+0000","last_fullsized":"2024-03-18T20:34:41.813762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:33:33.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996504+0000","last_change":"2024-03-18T20:34:38.807684+0000","last_active":"2024-03-18T20:34:39.996504+0000","last_peered":"2024-03-18T20:34:39.996504+0000","last_clean":"2024-03-18T20:34:39.996504+0000","last_became_active":"2024-03-18T20:34:38.807300+0000","last_became_peered":"2024-03-18T20:34:38.807300+0000","last_unstale":"2024-03-18T20:34:39.996504+0000","last_undegraded":"2024-03-18T20:34:39.996504+0000","last_fullsized":"2024-03-18T20:34:39.996504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:31:43.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.793201+0000","last_change":"2024-03-18T20:34:38.341802+0000","last_active":"2024-03-18T20:34:44.793201+0000","last_peered":"2024-03-18T20:34:44.793201+0000","last_clean":"2024-03-18T20:34:44.793201+0000","last_became_active":"2024-03-18T20:34:38.340594+0000","last_became_peered":"2024-03-18T20:34:38.340594+0000","last_unstale":"2024-03-18T20:34:44.793201+0000","last_undegraded":"2024-03-18T20:34:44.793201+0000","last_fullsized":"2024-03-18T20:34:44.793201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:01:06.830323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901738+0000","last_change":"2024-03-18T20:34:38.809790+0000","last_active":"2024-03-18T20:34:39.901738+0000","last_peered":"2024-03-18T20:34:39.901738+0000","last_clean":"2024-03-18T20:34:39.901738+0000","last_became_active":"2024-03-18T20:34:38.809470+0000","last_became_peered":"2024-03-18T20:34:38.809470+0000","last_unstale":"2024-03-18T20:34:39.901738+0000","last_undegraded":"2024-03-18T20:34:39.901738+0000","last_fullsized":"2024-03-18T20:34:39.901738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:48.225759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.155918+0000","last_change":"2024-03-18T20:34:38.343103+0000","last_active":"2024-03-18T20:34:44.155918+0000","last_peered":"2024-03-18T20:34:44.155918+0000","last_clean":"2024-03-18T20:34:44.155918+0000","last_became_active":"2024-03-18T20:34:38.342744+0000","last_became_peered":"2024-03-18T20:34:38.342744+0000","last_unstale":"2024-03-18T20:34:44.155918+0000","last_undegraded":"2024-03-18T20:34:44.155918+0000","last_fullsized":"2024-03-18T20:34:44.155918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:39.078219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198604+0000","last_change":"2024-03-18T20:34:38.809294+0000","last_active":"2024-03-18T20:34:39.198604+0000","last_peered":"2024-03-18T20:34:39.198604+0000","last_clean":"2024-03-18T20:34:39.198604+0000","last_became_active":"2024-03-18T20:34:38.808542+0000","last_became_peered":"2024-03-18T20:34:38.808542+0000","last_unstale":"2024-03-18T20:34:39.198604+0000","last_undegraded":"2024-03-18T20:34:39.198604+0000","last_fullsized":"2024-03-18T20:34:39.198604+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:11:30.669317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.813772+0000","last_change":"2024-03-18T20:34:38.339454+0000","last_active":"2024-03-18T20:34:41.813772+0000","last_peered":"2024-03-18T20:34:41.813772+0000","last_clean":"2024-03-18T20:34:41.813772+0000","last_became_active":"2024-03-18T20:34:38.339154+0000","last_became_peered":"2024-03-18T20:34:38.339154+0000","last_unstale":"2024-03-18T20:34:41.813772+0000","last_undegraded":"2024-03-18T20:34:41.813772+0000","last_fullsized":"2024-03-18T20:34:41.813772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:12:06.300600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998255+0000","last_change":"2024-03-18T20:34:38.818462+0000","last_active":"2024-03-18T20:34:39.998255+0000","last_peered":"2024-03-18T20:34:39.998255+0000","last_clean":"2024-03-18T20:34:39.998255+0000","last_became_active":"2024-03-18T20:34:38.817522+0000","last_became_peered":"2024-03-18T20:34:38.817522+0000","last_unstale":"2024-03-18T20:34:39.998255+0000","last_undegraded":"2024-03-18T20:34:39.998255+0000","last_fullsized":"2024-03-18T20:34:39.998255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:50:32.258583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'5","reported_seq":47,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:10.829994+0000","last_change":"2024-03-18T20:34:38.341054+0000","last_active":"2024-03-18T20:35:10.829994+0000","last_peered":"2024-03-18T20:35:10.829994+0000","last_clean":"2024-03-18T20:35:10.829994+0000","last_became_active":"2024-03-18T20:34:38.340352+0000","last_became_peered":"2024-03-18T20:34:38.340352+0000","last_unstale":"2024-03-18T20:35:10.829994+0000","last_undegraded":"2024-03-18T20:35:10.829994+0000","last_fullsized":"2024-03-18T20:35:10.829994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:44.850244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997144+0000","last_change":"2024-03-18T20:34:38.818176+0000","last_active":"2024-03-18T20:34:39.997144+0000","last_peered":"2024-03-18T20:34:39.997144+0000","last_clean":"2024-03-18T20:34:39.997144+0000","last_became_active":"2024-03-18T20:34:38.817820+0000","last_became_peered":"2024-03-18T20:34:38.817820+0000","last_unstale":"2024-03-18T20:34:39.997144+0000","last_undegraded":"2024-03-18T20:34:39.997144+0000","last_fullsized":"2024-03-18T20:34:39.997144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:19:29.815174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.785145+0000","last_change":"2024-03-18T20:34:38.346591+0000","last_active":"2024-03-18T20:34:44.785145+0000","last_peered":"2024-03-18T20:34:44.785145+0000","last_clean":"2024-03-18T20:34:44.785145+0000","last_became_active":"2024-03-18T20:34:38.345897+0000","last_became_peered":"2024-03-18T20:34:38.345897+0000","last_unstale":"2024-03-18T20:34:44.785145+0000","last_undegraded":"2024-03-18T20:34:44.785145+0000","last_fullsized":"2024-03-18T20:34:44.785145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:47:32.118905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:40.284498+0000","last_change":"2024-03-18T20:34:38.804814+0000","last_active":"2024-03-18T20:34:40.284498+0000","last_peered":"2024-03-18T20:34:40.284498+0000","last_clean":"2024-03-18T20:34:40.284498+0000","last_became_active":"2024-03-18T20:34:38.804585+0000","last_became_peered":"2024-03-18T20:34:38.804585+0000","last_unstale":"2024-03-18T20:34:40.284498+0000","last_undegraded":"2024-03-18T20:34:40.284498+0000","last_fullsized":"2024-03-18T20:34:40.284498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:40:47.273070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.793953+0000","last_change":"2024-03-18T20:34:38.344615+0000","last_active":"2024-03-18T20:34:44.793953+0000","last_peered":"2024-03-18T20:34:44.793953+0000","last_clean":"2024-03-18T20:34:44.793953+0000","last_became_active":"2024-03-18T20:34:38.344242+0000","last_became_peered":"2024-03-18T20:34:38.344242+0000","last_unstale":"2024-03-18T20:34:44.793953+0000","last_undegraded":"2024-03-18T20:34:44.793953+0000","last_fullsized":"2024-03-18T20:34:44.793953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.449808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997264+0000","last_change":"2024-03-18T20:34:38.820875+0000","last_active":"2024-03-18T20:34:39.997264+0000","last_peered":"2024-03-18T20:34:39.997264+0000","last_clean":"2024-03-18T20:34:39.997264+0000","last_became_active":"2024-03-18T20:34:38.820550+0000","last_became_peered":"2024-03-18T20:34:38.820550+0000","last_unstale":"2024-03-18T20:34:39.997264+0000","last_undegraded":"2024-03-18T20:34:39.997264+0000","last_fullsized":"2024-03-18T20:34:39.997264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:49:36.358734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901824+0000","last_change":"2024-03-18T20:34:38.812552+0000","last_active":"2024-03-18T20:34:39.901824+0000","last_peered":"2024-03-18T20:34:39.901824+0000","last_clean":"2024-03-18T20:34:39.901824+0000","last_became_active":"2024-03-18T20:34:38.811939+0000","last_became_peered":"2024-03-18T20:34:38.811939+0000","last_unstale":"2024-03-18T20:34:39.901824+0000","last_undegraded":"2024-03-18T20:34:39.901824+0000","last_fullsized":"2024-03-18T20:34:39.901824+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:41:52.096974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997308+0000","last_change":"2024-03-18T20:34:38.815691+0000","last_active":"2024-03-18T20:34:39.997308+0000","last_peered":"2024-03-18T20:34:39.997308+0000","last_clean":"2024-03-18T20:34:39.997308+0000","last_became_active":"2024-03-18T20:34:38.815370+0000","last_became_peered":"2024-03-18T20:34:38.815370+0000","last_unstale":"2024-03-18T20:34:39.997308+0000","last_undegraded":"2024-03-18T20:34:39.997308+0000","last_fullsized":"2024-03-18T20:34:39.997308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:59:41.260013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997260+0000","last_change":"2024-03-18T20:34:38.812651+0000","last_active":"2024-03-18T20:34:39.997260+0000","last_peered":"2024-03-18T20:34:39.997260+0000","last_clean":"2024-03-18T20:34:39.997260+0000","last_became_active":"2024-03-18T20:34:38.812322+0000","last_became_peered":"2024-03-18T20:34:38.812322+0000","last_unstale":"2024-03-18T20:34:39.997260+0000","last_undegraded":"2024-03-18T20:34:39.997260+0000","last_fullsized":"2024-03-18T20:34:39.997260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:34:55.658371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901247+0000","last_change":"2024-03-18T20:34:38.806475+0000","last_active":"2024-03-18T20:34:39.901247+0000","last_peered":"2024-03-18T20:34:39.901247+0000","last_clean":"2024-03-18T20:34:39.901247+0000","last_became_active":"2024-03-18T20:34:38.806027+0000","last_became_peered":"2024-03-18T20:34:38.806027+0000","last_unstale":"2024-03-18T20:34:39.901247+0000","last_undegraded":"2024-03-18T20:34:39.901247+0000","last_fullsized":"2024-03-18T20:34:39.901247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:57:17.140031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793496+0000","last_change":"2024-03-18T20:34:38.811057+0000","last_active":"2024-03-18T20:34:39.793496+0000","last_peered":"2024-03-18T20:34:39.793496+0000","last_clean":"2024-03-18T20:34:39.793496+0000","last_became_active":"2024-03-18T20:34:38.810316+0000","last_became_peered":"2024-03-18T20:34:38.810316+0000","last_unstale":"2024-03-18T20:34:39.793496+0000","last_undegraded":"2024-03-18T20:34:39.793496+0000","last_fullsized":"2024-03-18T20:34:39.793496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:29:22.426717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.902010+0000","last_change":"2024-03-18T20:34:38.812952+0000","last_active":"2024-03-18T20:34:39.902010+0000","last_peered":"2024-03-18T20:34:39.902010+0000","last_clean":"2024-03-18T20:34:39.902010+0000","last_became_active":"2024-03-18T20:34:38.812550+0000","last_became_peered":"2024-03-18T20:34:38.812550+0000","last_unstale":"2024-03-18T20:34:39.902010+0000","last_undegraded":"2024-03-18T20:34:39.902010+0000","last_fullsized":"2024-03-18T20:34:39.902010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:06:26.525078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997748+0000","last_change":"2024-03-18T20:34:38.817358+0000","last_active":"2024-03-18T20:34:39.997748+0000","last_peered":"2024-03-18T20:34:39.997748+0000","last_clean":"2024-03-18T20:34:39.997748+0000","last_became_active":"2024-03-18T20:34:38.816939+0000","last_became_peered":"2024-03-18T20:34:38.816939+0000","last_unstale":"2024-03-18T20:34:39.997748+0000","last_undegraded":"2024-03-18T20:34:39.997748+0000","last_fullsized":"2024-03-18T20:34:39.997748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:33.258162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198671+0000","last_change":"2024-03-18T20:34:38.807798+0000","last_active":"2024-03-18T20:34:39.198671+0000","last_peered":"2024-03-18T20:34:39.198671+0000","last_clean":"2024-03-18T20:34:39.198671+0000","last_became_active":"2024-03-18T20:34:38.807489+0000","last_became_peered":"2024-03-18T20:34:38.807489+0000","last_unstale":"2024-03-18T20:34:39.198671+0000","last_undegraded":"2024-03-18T20:34:39.198671+0000","last_fullsized":"2024-03-18T20:34:39.198671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:15.395763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759267+0000","last_change":"2024-03-18T20:34:38.806363+0000","last_active":"2024-03-18T20:34:39.759267+0000","last_peered":"2024-03-18T20:34:39.759267+0000","last_clean":"2024-03-18T20:34:39.759267+0000","last_became_active":"2024-03-18T20:34:38.805707+0000","last_became_peered":"2024-03-18T20:34:38.805707+0000","last_unstale":"2024-03-18T20:34:39.759267+0000","last_undegraded":"2024-03-18T20:34:39.759267+0000","last_fullsized":"2024-03-18T20:34:39.759267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:10.918979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198542+0000","last_change":"2024-03-18T20:34:38.814439+0000","last_active":"2024-03-18T20:34:39.198542+0000","last_peered":"2024-03-18T20:34:39.198542+0000","last_clean":"2024-03-18T20:34:39.198542+0000","last_became_active":"2024-03-18T20:34:38.813988+0000","last_became_peered":"2024-03-18T20:34:38.813988+0000","last_unstale":"2024-03-18T20:34:39.198542+0000","last_undegraded":"2024-03-18T20:34:39.198542+0000","last_fullsized":"2024-03-18T20:34:39.198542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:03:14.639044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.821809+0000","last_change":"2024-03-18T20:34:38.818550+0000","last_active":"2024-03-18T20:34:38.821809+0000","last_peered":"2024-03-18T20:34:38.821809+0000","last_clean":"2024-03-18T20:34:38.821809+0000","last_became_active":"2024-03-18T20:34:38.818209+0000","last_became_peered":"2024-03-18T20:34:38.818209+0000","last_unstale":"2024-03-18T20:34:38.821809+0000","last_undegraded":"2024-03-18T20:34:38.821809+0000","last_fullsized":"2024-03-18T20:34:38.821809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:40:48.395331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793586+0000","last_change":"2024-03-18T20:34:38.808439+0000","last_active":"2024-03-18T20:34:39.793586+0000","last_peered":"2024-03-18T20:34:39.793586+0000","last_clean":"2024-03-18T20:34:39.793586+0000","last_became_active":"2024-03-18T20:34:38.807995+0000","last_became_peered":"2024-03-18T20:34:38.807995+0000","last_unstale":"2024-03-18T20:34:39.793586+0000","last_undegraded":"2024-03-18T20:34:39.793586+0000","last_fullsized":"2024-03-18T20:34:39.793586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:54.414148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901907+0000","last_change":"2024-03-18T20:34:38.810793+0000","last_active":"2024-03-18T20:34:39.901907+0000","last_peered":"2024-03-18T20:34:39.901907+0000","last_clean":"2024-03-18T20:34:39.901907+0000","last_became_active":"2024-03-18T20:34:38.810496+0000","last_became_peered":"2024-03-18T20:34:38.810496+0000","last_unstale":"2024-03-18T20:34:39.901907+0000","last_undegraded":"2024-03-18T20:34:39.901907+0000","last_fullsized":"2024-03-18T20:34:39.901907+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:33.780675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66040,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":324,"num_read_kb":234,"num_write":229,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":198120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612625,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":851968,"data_stored":405743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.39400000000000002}]},{"osd":1,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.68700000000000006}]},{"osd":2,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57699999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.374},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.46000000000000002}]},{"osd":6,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.53600000000000003}]},{"osd":7,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.65200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":9,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.67200000000000004}]},{"osd":10,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.71799999999999997}]}]},{"osd":10,"up_from":66,"seq":283467841556,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1056251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.60699999999999998}]},{"osd":2,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57999999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.62}]},{"osd":6,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.69199999999999995}]},{"osd":9,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.71099999999999997}]},{"osd":11,"last update":"Mon Mar 18 20:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.745}]}]},{"osd":9,"up_from":61,"seq":261993005081,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":405721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.505},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.66800000000000004}]},{"osd":1,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.78000000000000003}]},{"osd":2,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.59099999999999997}]},{"osd":4,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.81799999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.75}]},{"osd":6,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69099999999999995}]},{"osd":8,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.77400000000000002}]},{"osd":10,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.72199999999999998}]}]},{"osd":8,"up_from":54,"seq":231928234012,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":406099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55300000000000005}]},{"osd":1,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.79700000000000004}]},{"osd":2,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.71899999999999997}]},{"osd":3,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.625}]},{"osd":4,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.81200000000000006}]},{"osd":5,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.68100000000000005}]},{"osd":6,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.85099999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.874}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.90400000000000003}]},{"osd":10,"last update":"Mon Mar 18 20:34:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.95299999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]},{"osd":7,"up_from":47,"seq":201863462943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1470464,"data_stored":998474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.439}]},{"osd":1,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.42199999999999999,"15min":0.41399999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.36799999999999999}]},{"osd":2,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73799999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.62}]},{"osd":3,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.72999999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.53100000000000003}]},{"osd":5,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.40500000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.55800000000000005}]},{"osd":6,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89700000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.57799999999999996}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63600000000000001}]},{"osd":9,"last update":"Mon Mar 18 20:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.69299999999999995}]},{"osd":10,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.72699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":6,"up_from":42,"seq":180388626466,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":405390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.46100000000000002}]},{"osd":1,"last update":"Mon Mar 18 20:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.91900000000000004}]},{"osd":2,"last update":"Mon Mar 18 20:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80100000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Mar 18 20:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79700000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.497},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.874}]},{"osd":4,"last update":"Mon Mar 18 20:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59899999999999998,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.621,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93500000000000005,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.92400000000000004,"5min":1.052,"15min":1.052},"last":0.65800000000000003}]},{"osd":7,"last update":"Mon Mar 18 20:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.89000000000000001,"5min":0.995,"15min":0.995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58699999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.97199999999999998,"5min":1.006,"15min":1.006},"last":1.0269999999999999}]},{"osd":8,"last update":"Mon Mar 18 20:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.98699999999999999}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.73599999999999999}]},{"osd":10,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":1.101},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.75700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":465721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.54300000000000004},"min":{"1min":0.32400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.81799999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":1.76}]},{"osd":2,"last update":"Mon Mar 18 20:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.745,"5min":0.754,"15min":0.754},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.83999999999999997}]},{"osd":3,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.67600000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":1.4710000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79400000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":1.478}]},{"osd":4,"last update":"Mon Mar 18 20:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.443,"15min":0.40999999999999998},"min":{"1min":0.34499999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.70099999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.45000000000000001,"15min":0.42099999999999999},"min":{"1min":0.32400000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.89500000000000002}]},{"osd":6,"last update":"Mon Mar 18 20:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0229999999999999,"5min":1.196,"15min":1.196},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.434,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79200000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":1.3759999999999999}]},{"osd":7,"last update":"Mon Mar 18 20:34:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84399999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.66600000000000004}]},{"osd":8,"last update":"Mon Mar 18 20:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.98899999999999999}]},{"osd":9,"last update":"Mon Mar 18 20:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.3320000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":1.04}]},{"osd":10,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.79500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":406729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.36699999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.52400000000000002}]},{"osd":2,"last update":"Mon Mar 18 20:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.44700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.627},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92600000000000005,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.68200000000000005}]},{"osd":3,"last update":"Mon Mar 18 20:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.32700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.38300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58199999999999996}]},{"osd":4,"last update":"Mon Mar 18 20:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89900000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.44900000000000001}]},{"osd":5,"last update":"Mon Mar 18 20:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.628},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59699999999999998}]},{"osd":6,"last update":"Mon Mar 18 20:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.72499999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.88700000000000001}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.77400000000000002}]},{"osd":9,"last update":"Mon Mar 18 20:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71199999999999997}]},{"osd":10,"last update":"Mon Mar 18 20:34:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.69499999999999995}]},{"osd":11,"last update":"Mon Mar 18 20:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.74299999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":465224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.48699999999999999,"15min":0.46400000000000002},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.75,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.48399999999999999,"15min":0.46300000000000002},"min":{"1min":0.318,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.50900000000000001}]},{"osd":1,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.56000000000000005,"15min":0.52900000000000003},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.6100000000000003,"5min":4.6100000000000003,"15min":4.6100000000000003},"last":0.746},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.62}]},{"osd":3,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.54700000000000004,"15min":0.52100000000000002},"min":{"1min":0.43099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.46000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79700000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68400000000000005}]},{"osd":4,"last update":"Mon Mar 18 20:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.436,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.85699999999999998}]},{"osd":5,"last update":"Mon Mar 18 20:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.44500000000000001,"15min":0.438},"min":{"1min":0.307,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35099999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.69999999999999996}]},{"osd":6,"last update":"Mon Mar 18 20:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84999999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.77100000000000002,"5min":1.083,"15min":1.083},"last":0.72799999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.439},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45800000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.63300000000000001}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.77400000000000002}]},{"osd":10,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.4669999999999996,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.878},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.84199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":3,"up_from":24,"seq":103079215148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":405263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.40899999999999997,"15min":0.40699999999999997},"min":{"1min":0.30199999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57799999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.622},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.38900000000000001}]},{"osd":1,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.87,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41299999999999998}]},{"osd":2,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.59799999999999998},"min":{"1min":0.47799999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56699999999999995}]},{"osd":4,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90700000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67800000000000005,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.876,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.52300000000000002}]},{"osd":5,"last update":"Mon Mar 18 20:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.499,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.47399999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Mon Mar 18 20:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83999999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.496}]},{"osd":7,"last update":"Mon Mar 18 20:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.90500000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.60699999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73099999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.68300000000000005}]},{"osd":10,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.74099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986218,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":405643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.38400000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.371,"15min":0.36099999999999999},"min":{"1min":0.31,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.374,"15min":0.35999999999999999},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.63,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.53300000000000003}]},{"osd":2,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55100000000000005,"15min":0.52600000000000002},"min":{"1min":0.45600000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72199999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67200000000000004}]},{"osd":3,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56299999999999994}]},{"osd":5,"last update":"Mon Mar 18 20:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77600000000000002,"5min":0.875,"15min":0.875},"last":0.78100000000000003}]},{"osd":6,"last update":"Mon Mar 18 20:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.48199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.746}]},{"osd":7,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.621,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.70699999999999996}]},{"osd":8,"last update":"Mon Mar 18 20:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.79400000000000004}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.84599999999999997}]},{"osd":10,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.83499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]}]},{"osd":5,"up_from":35,"seq":150323855398,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1478656,"data_stored":997525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.52500000000000002}]},{"osd":1,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69099999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.626},{"interface":"front","average":{"1min":0.625,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.42999999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56299999999999994}]},{"osd":2,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.27900000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.27900000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.373}]},{"osd":3,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.35499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76300000000000001,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.75800000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.68500000000000005}]},{"osd":6,"last update":"Mon Mar 18 20:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.439},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54200000000000004}]},{"osd":7,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67100000000000004}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65400000000000003}]},{"osd":9,"last update":"Mon Mar 18 20:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64100000000000001}]},{"osd":10,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T20:35:18.748 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-18T20:35:19.010 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:19.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:18 smithi046 ceph-mon[43495]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:19.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:18 smithi073 ceph-mon[47384]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:19.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:18 smithi155 ceph-mon[48344]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:19.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.964+0000 7f3dcd915700 1 -- 172.21.15.46:0/2444468844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8104fb0 msgr2=0x7f3dc810cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:19.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.964+0000 7f3dcd915700 1 --2- 172.21.15.46:0/2444468844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8104fb0 0x7f3dc810cb80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000b0a0 tx=0x7f3dc000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:19.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.964+0000 7f3dcd915700 1 -- 172.21.15.46:0/2444468844 shutdown_connections 2024-03-18T20:35:19.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.964+0000 7f3dcd915700 1 --2- 172.21.15.46:0/2444468844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8104fb0 0x7f3dc810cb80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.964+0000 7f3dcd915700 1 --2- 172.21.15.46:0/2444468844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dc80fff70 0x7f3dc8100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.964+0000 7f3dcd915700 1 --2- 172.21.15.46:0/2444468844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc8100920 0x7f3dc81048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.964+0000 7f3dcd915700 1 -- 172.21.15.46:0/2444468844 >> 172.21.15.46:0/2444468844 conn(0x7f3dc8075d50 msgr2=0x7f3dc8076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.964+0000 7f3dcd915700 1 -- 172.21.15.46:0/2444468844 shutdown_connections 2024-03-18T20:35:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.964+0000 7f3dcd915700 1 -- 172.21.15.46:0/2444468844 wait complete. 2024-03-18T20:35:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.965+0000 7f3dcd915700 1 Processor -- start 2024-03-18T20:35:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dcd915700 1 -- start start 2024-03-18T20:35:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dcd915700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dc80fff70 0x7f3dc8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dcd915700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc8100920 0x7f3dc8199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dcd915700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8104fb0 0x7f3dc819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dcd915700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc819e350 con 0x7f3dc8104fb0 2024-03-18T20:35:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dcd915700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc819e4c0 con 0x7f3dc80fff70 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dcd915700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc819e680 con 0x7f3dc8100920 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dc77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8104fb0 0x7f3dc819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dc77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8104fb0 0x7f3dc819dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52220/0 (socket says 172.21.15.46:52220) 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dc77fe700 1 -- 172.21.15.46:0/2860407119 learned_addr learned my addr 172.21.15.46:0/2860407119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dc6ffd700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dc80fff70 0x7f3dc8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dc67fc700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc8100920 0x7f3dc8199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dc77fe700 1 -- 172.21.15.46:0/2860407119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc8100920 msgr2=0x7f3dc8199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dc77fe700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc8100920 0x7f3dc8199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dc77fe700 1 -- 172.21.15.46:0/2860407119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dc80fff70 msgr2=0x7f3dc8199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.966+0000 7f3dc77fe700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dc80fff70 0x7f3dc8199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.967+0000 7f3dc77fe700 1 -- 172.21.15.46:0/2860407119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc0009d00 con 0x7f3dc8104fb0 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.967+0000 7f3dc67fc700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc8100920 0x7f3dc8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.967+0000 7f3dc77fe700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8104fb0 0x7f3dc819dd40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000b830 tx=0x7f3dc00078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.967+0000 7f3dcc913700 1 -- 172.21.15.46:0/2860407119 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc000e040 con 0x7f3dc8104fb0 2024-03-18T20:35:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.967+0000 7f3dcc913700 1 -- 172.21.15.46:0/2860407119 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dc0004e60 con 0x7f3dc8104fb0 2024-03-18T20:35:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.967+0000 7f3dcd915700 1 -- 172.21.15.46:0/2860407119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc819e900 con 0x7f3dc8104fb0 2024-03-18T20:35:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.967+0000 7f3dcc913700 1 -- 172.21.15.46:0/2860407119 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0005420 con 0x7f3dc8104fb0 2024-03-18T20:35:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.967+0000 7f3dcd915700 1 -- 172.21.15.46:0/2860407119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc81a61c0 con 0x7f3dc8104fb0 2024-03-18T20:35:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.968+0000 7f3dcd915700 1 -- 172.21.15.46:0/2860407119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc8103040 con 0x7f3dc8104fb0 2024-03-18T20:35:19.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.969+0000 7f3dcc913700 1 -- 172.21.15.46:0/2860407119 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3dc001bb30 con 0x7f3dc8104fb0 2024-03-18T20:35:19.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.970+0000 7f3dcc913700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3db40a1230 0x7f3db40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:19.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.970+0000 7f3dc6ffd700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3db40a1230 0x7f3db40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:19.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.970+0000 7f3dcc913700 1 -- 172.21.15.46:0/2860407119 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3dc00ce1f0 con 0x7f3dc8104fb0 2024-03-18T20:35:19.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.970+0000 7f3dc6ffd700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3db40a1230 0x7f3db40a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3db0005fd0 tx=0x7f3db000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:19.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:19.975+0000 7f3dcc913700 1 -- 172.21.15.46:0/2860407119 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3dc0097960 con 0x7f3dc8104fb0 2024-03-18T20:35:20.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:19 smithi046 ceph-mon[43495]: pgmap v239: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T20:35:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.182+0000 7f3dcd915700 1 -- 172.21.15.46:0/2860407119 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3dc819a760 con 0x7f3db40a1230 2024-03-18T20:35:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.193+0000 7f3dcc913700 1 -- 172.21.15.46:0/2860407119 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340869 (secure 0 0 0) 0x7f3dc819a760 con 0x7f3db40a1230 2024-03-18T20:35:20.201 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:20.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.197+0000 7f3dcd915700 1 -- 172.21.15.46:0/2860407119 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3db40a1230 msgr2=0x7f3db40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:20.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.197+0000 7f3dcd915700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3db40a1230 0x7f3db40a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3db0005fd0 tx=0x7f3db000a560 comp rx=0 tx=0).stop 2024-03-18T20:35:20.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.197+0000 7f3dcd915700 1 -- 172.21.15.46:0/2860407119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8104fb0 msgr2=0x7f3dc819dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:20.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.197+0000 7f3dcd915700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8104fb0 0x7f3dc819dd40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000b830 tx=0x7f3dc00078a0 comp rx=0 tx=0).stop 2024-03-18T20:35:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.197+0000 7f3dcd915700 1 -- 172.21.15.46:0/2860407119 shutdown_connections 2024-03-18T20:35:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.197+0000 7f3dcd915700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3db40a1230 0x7f3db40a36f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.197+0000 7f3dcd915700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dc8104fb0 0x7f3dc819dd40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.197+0000 7f3dcd915700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dc80fff70 0x7f3dc8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.197+0000 7f3dcd915700 1 --2- 172.21.15.46:0/2860407119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dc8100920 0x7f3dc8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.197+0000 7f3dcd915700 1 -- 172.21.15.46:0/2860407119 >> 172.21.15.46:0/2860407119 conn(0x7f3dc8075d50 msgr2=0x7f3dc810bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.197+0000 7f3dcd915700 1 -- 172.21.15.46:0/2860407119 shutdown_connections 2024-03-18T20:35:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:20.197+0000 7f3dcd915700 1 -- 172.21.15.46:0/2860407119 wait complete. 2024-03-18T20:35:20.205 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-18T20:35:20.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:19 smithi073 ceph-mon[47384]: pgmap v239: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T20:35:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:19 smithi155 ceph-mon[48344]: pgmap v239: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T20:35:20.607 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-03-18T20:35:19.028271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656642,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":495,"num_read_kb":373,"num_write":412,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336336,"kb_used_data":12144,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605488,"statfs":{"total":1151940427776,"available":1151596019712,"internally_reserved":0,"allocated":12435456,"data_stored":6823783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1138,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":72,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003543"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.902054+0000","last_change":"2024-03-18T20:34:38.817028+0000","last_active":"2024-03-18T20:34:39.902054+0000","last_peered":"2024-03-18T20:34:39.902054+0000","last_clean":"2024-03-18T20:34:39.902054+0000","last_became_active":"2024-03-18T20:34:38.816711+0000","last_became_peered":"2024-03-18T20:34:38.816711+0000","last_unstale":"2024-03-18T20:34:39.902054+0000","last_undegraded":"2024-03-18T20:34:39.902054+0000","last_fullsized":"2024-03-18T20:34:39.902054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:32:16.827141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901345+0000","last_change":"2024-03-18T20:34:38.807350+0000","last_active":"2024-03-18T20:34:39.901345+0000","last_peered":"2024-03-18T20:34:39.901345+0000","last_clean":"2024-03-18T20:34:39.901345+0000","last_became_active":"2024-03-18T20:34:38.807008+0000","last_became_peered":"2024-03-18T20:34:38.807008+0000","last_unstale":"2024-03-18T20:34:39.901345+0000","last_undegraded":"2024-03-18T20:34:39.901345+0000","last_fullsized":"2024-03-18T20:34:39.901345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:35.667384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901221+0000","last_change":"2024-03-18T20:34:38.817045+0000","last_active":"2024-03-18T20:34:39.901221+0000","last_peered":"2024-03-18T20:34:39.901221+0000","last_clean":"2024-03-18T20:34:39.901221+0000","last_became_active":"2024-03-18T20:34:38.816659+0000","last_became_peered":"2024-03-18T20:34:38.816659+0000","last_unstale":"2024-03-18T20:34:39.901221+0000","last_undegraded":"2024-03-18T20:34:39.901221+0000","last_fullsized":"2024-03-18T20:34:39.901221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:04:49.316127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998431+0000","last_change":"2024-03-18T20:34:38.804833+0000","last_active":"2024-03-18T20:34:39.998431+0000","last_peered":"2024-03-18T20:34:39.998431+0000","last_clean":"2024-03-18T20:34:39.998431+0000","last_became_active":"2024-03-18T20:34:38.804486+0000","last_became_peered":"2024-03-18T20:34:38.804486+0000","last_unstale":"2024-03-18T20:34:39.998431+0000","last_undegraded":"2024-03-18T20:34:39.998431+0000","last_fullsized":"2024-03-18T20:34:39.998431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:19:44.266207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996571+0000","last_change":"2024-03-18T20:34:38.797660+0000","last_active":"2024-03-18T20:34:39.996571+0000","last_peered":"2024-03-18T20:34:39.996571+0000","last_clean":"2024-03-18T20:34:39.996571+0000","last_became_active":"2024-03-18T20:34:38.797155+0000","last_became_peered":"2024-03-18T20:34:38.797155+0000","last_unstale":"2024-03-18T20:34:39.996571+0000","last_undegraded":"2024-03-18T20:34:39.996571+0000","last_fullsized":"2024-03-18T20:34:39.996571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:22:11.363453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997430+0000","last_change":"2024-03-18T20:34:38.822891+0000","last_active":"2024-03-18T20:34:39.997430+0000","last_peered":"2024-03-18T20:34:39.997430+0000","last_clean":"2024-03-18T20:34:39.997430+0000","last_became_active":"2024-03-18T20:34:38.822574+0000","last_became_peered":"2024-03-18T20:34:38.822574+0000","last_unstale":"2024-03-18T20:34:39.997430+0000","last_undegraded":"2024-03-18T20:34:39.997430+0000","last_fullsized":"2024-03-18T20:34:39.997430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:42:05.727053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997617+0000","last_change":"2024-03-18T20:34:38.805378+0000","last_active":"2024-03-18T20:34:39.997617+0000","last_peered":"2024-03-18T20:34:39.997617+0000","last_clean":"2024-03-18T20:34:39.997617+0000","last_became_active":"2024-03-18T20:34:38.805130+0000","last_became_peered":"2024-03-18T20:34:38.805130+0000","last_unstale":"2024-03-18T20:34:39.997617+0000","last_undegraded":"2024-03-18T20:34:39.997617+0000","last_fullsized":"2024-03-18T20:34:39.997617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:46:19.420715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198520+0000","last_change":"2024-03-18T20:34:38.801034+0000","last_active":"2024-03-18T20:34:39.198520+0000","last_peered":"2024-03-18T20:34:39.198520+0000","last_clean":"2024-03-18T20:34:39.198520+0000","last_became_active":"2024-03-18T20:34:38.800706+0000","last_became_peered":"2024-03-18T20:34:38.800706+0000","last_unstale":"2024-03-18T20:34:39.198520+0000","last_undegraded":"2024-03-18T20:34:39.198520+0000","last_fullsized":"2024-03-18T20:34:39.198520+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:10:22.641105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.823531+0000","last_change":"2024-03-18T20:34:38.822841+0000","last_active":"2024-03-18T20:34:38.823531+0000","last_peered":"2024-03-18T20:34:38.823531+0000","last_clean":"2024-03-18T20:34:38.823531+0000","last_became_active":"2024-03-18T20:34:38.822537+0000","last_became_peered":"2024-03-18T20:34:38.822537+0000","last_unstale":"2024-03-18T20:34:38.823531+0000","last_undegraded":"2024-03-18T20:34:38.823531+0000","last_fullsized":"2024-03-18T20:34:38.823531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:07:38.741995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997436+0000","last_change":"2024-03-18T20:34:38.814728+0000","last_active":"2024-03-18T20:34:39.997436+0000","last_peered":"2024-03-18T20:34:39.997436+0000","last_clean":"2024-03-18T20:34:39.997436+0000","last_became_active":"2024-03-18T20:34:38.814200+0000","last_became_peered":"2024-03-18T20:34:38.814200+0000","last_unstale":"2024-03-18T20:34:39.997436+0000","last_undegraded":"2024-03-18T20:34:39.997436+0000","last_fullsized":"2024-03-18T20:34:39.997436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:24:54.064770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997025+0000","last_change":"2024-03-18T20:34:38.818054+0000","last_active":"2024-03-18T20:34:39.997025+0000","last_peered":"2024-03-18T20:34:39.997025+0000","last_clean":"2024-03-18T20:34:39.997025+0000","last_became_active":"2024-03-18T20:34:38.817752+0000","last_became_peered":"2024-03-18T20:34:38.817752+0000","last_unstale":"2024-03-18T20:34:39.997025+0000","last_undegraded":"2024-03-18T20:34:39.997025+0000","last_fullsized":"2024-03-18T20:34:39.997025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:01:13.728434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998381+0000","last_change":"2024-03-18T20:34:38.801356+0000","last_active":"2024-03-18T20:34:39.998381+0000","last_peered":"2024-03-18T20:34:39.998381+0000","last_clean":"2024-03-18T20:34:39.998381+0000","last_became_active":"2024-03-18T20:34:38.800978+0000","last_became_peered":"2024-03-18T20:34:38.800978+0000","last_unstale":"2024-03-18T20:34:39.998381+0000","last_undegraded":"2024-03-18T20:34:39.998381+0000","last_fullsized":"2024-03-18T20:34:39.998381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:07.634686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793686+0000","last_change":"2024-03-18T20:34:38.803006+0000","last_active":"2024-03-18T20:34:39.793686+0000","last_peered":"2024-03-18T20:34:39.793686+0000","last_clean":"2024-03-18T20:34:39.793686+0000","last_became_active":"2024-03-18T20:34:38.802454+0000","last_became_peered":"2024-03-18T20:34:38.802454+0000","last_unstale":"2024-03-18T20:34:39.793686+0000","last_undegraded":"2024-03-18T20:34:39.793686+0000","last_fullsized":"2024-03-18T20:34:39.793686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:11:02.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901028+0000","last_change":"2024-03-18T20:34:38.807397+0000","last_active":"2024-03-18T20:34:39.901028+0000","last_peered":"2024-03-18T20:34:39.901028+0000","last_clean":"2024-03-18T20:34:39.901028+0000","last_became_active":"2024-03-18T20:34:38.807072+0000","last_became_peered":"2024-03-18T20:34:38.807072+0000","last_unstale":"2024-03-18T20:34:39.901028+0000","last_undegraded":"2024-03-18T20:34:39.901028+0000","last_fullsized":"2024-03-18T20:34:39.901028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:55:53.832825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.900962+0000","last_change":"2024-03-18T20:34:38.812925+0000","last_active":"2024-03-18T20:34:39.900962+0000","last_peered":"2024-03-18T20:34:39.900962+0000","last_clean":"2024-03-18T20:34:39.900962+0000","last_became_active":"2024-03-18T20:34:38.812597+0000","last_became_peered":"2024-03-18T20:34:38.812597+0000","last_unstale":"2024-03-18T20:34:39.900962+0000","last_undegraded":"2024-03-18T20:34:39.900962+0000","last_fullsized":"2024-03-18T20:34:39.900962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:14:35.103852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.820572+0000","last_change":"2024-03-18T20:34:38.817892+0000","last_active":"2024-03-18T20:34:38.820572+0000","last_peered":"2024-03-18T20:34:38.820572+0000","last_clean":"2024-03-18T20:34:38.820572+0000","last_became_active":"2024-03-18T20:34:38.816261+0000","last_became_peered":"2024-03-18T20:34:38.816261+0000","last_unstale":"2024-03-18T20:34:38.820572+0000","last_undegraded":"2024-03-18T20:34:38.820572+0000","last_fullsized":"2024-03-18T20:34:38.820572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:32.481210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.902148+0000","last_change":"2024-03-18T20:34:38.816966+0000","last_active":"2024-03-18T20:34:39.902148+0000","last_peered":"2024-03-18T20:34:39.902148+0000","last_clean":"2024-03-18T20:34:39.902148+0000","last_became_active":"2024-03-18T20:34:38.816645+0000","last_became_peered":"2024-03-18T20:34:38.816645+0000","last_unstale":"2024-03-18T20:34:39.902148+0000","last_undegraded":"2024-03-18T20:34:39.902148+0000","last_fullsized":"2024-03-18T20:34:39.902148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:39.904353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997076+0000","last_change":"2024-03-18T20:34:38.817005+0000","last_active":"2024-03-18T20:34:39.997076+0000","last_peered":"2024-03-18T20:34:39.997076+0000","last_clean":"2024-03-18T20:34:39.997076+0000","last_became_active":"2024-03-18T20:34:38.816710+0000","last_became_peered":"2024-03-18T20:34:38.816710+0000","last_unstale":"2024-03-18T20:34:39.997076+0000","last_undegraded":"2024-03-18T20:34:39.997076+0000","last_fullsized":"2024-03-18T20:34:39.997076+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:30:43.483977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997574+0000","last_change":"2024-03-18T20:34:38.810957+0000","last_active":"2024-03-18T20:34:39.997574+0000","last_peered":"2024-03-18T20:34:39.997574+0000","last_clean":"2024-03-18T20:34:39.997574+0000","last_became_active":"2024-03-18T20:34:38.810615+0000","last_became_peered":"2024-03-18T20:34:38.810615+0000","last_unstale":"2024-03-18T20:34:39.997574+0000","last_undegraded":"2024-03-18T20:34:39.997574+0000","last_fullsized":"2024-03-18T20:34:39.997574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:26:36.544034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:14.861215+0000","last_change":"2024-03-18T20:34:38.337863+0000","last_active":"2024-03-18T20:35:14.861215+0000","last_peered":"2024-03-18T20:35:14.861215+0000","last_clean":"2024-03-18T20:35:14.861215+0000","last_became_active":"2024-03-18T20:34:38.337548+0000","last_became_peered":"2024-03-18T20:34:38.337548+0000","last_unstale":"2024-03-18T20:35:14.861215+0000","last_undegraded":"2024-03-18T20:35:14.861215+0000","last_fullsized":"2024-03-18T20:35:14.861215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:41:40.237882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198626+0000","last_change":"2024-03-18T20:34:38.814321+0000","last_active":"2024-03-18T20:34:39.198626+0000","last_peered":"2024-03-18T20:34:39.198626+0000","last_clean":"2024-03-18T20:34:39.198626+0000","last_became_active":"2024-03-18T20:34:38.813939+0000","last_became_peered":"2024-03-18T20:34:38.813939+0000","last_unstale":"2024-03-18T20:34:39.198626+0000","last_undegraded":"2024-03-18T20:34:39.198626+0000","last_fullsized":"2024-03-18T20:34:39.198626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:05:42.441099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996941+0000","last_change":"2024-03-18T20:34:38.347790+0000","last_active":"2024-03-18T20:34:39.996941+0000","last_peered":"2024-03-18T20:34:39.996941+0000","last_clean":"2024-03-18T20:34:39.996941+0000","last_became_active":"2024-03-18T20:34:38.347317+0000","last_became_peered":"2024-03-18T20:34:38.347317+0000","last_unstale":"2024-03-18T20:34:39.996941+0000","last_undegraded":"2024-03-18T20:34:39.996941+0000","last_fullsized":"2024-03-18T20:34:39.996941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.096480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901274+0000","last_change":"2024-03-18T20:34:38.817116+0000","last_active":"2024-03-18T20:34:39.901274+0000","last_peered":"2024-03-18T20:34:39.901274+0000","last_clean":"2024-03-18T20:34:39.901274+0000","last_became_active":"2024-03-18T20:34:38.816761+0000","last_became_peered":"2024-03-18T20:34:38.816761+0000","last_unstale":"2024-03-18T20:34:39.901274+0000","last_undegraded":"2024-03-18T20:34:39.901274+0000","last_fullsized":"2024-03-18T20:34:39.901274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:44.353533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.796411+0000","last_change":"2024-03-18T20:34:38.339236+0000","last_active":"2024-03-18T20:34:44.796411+0000","last_peered":"2024-03-18T20:34:44.796411+0000","last_clean":"2024-03-18T20:34:44.796411+0000","last_became_active":"2024-03-18T20:34:38.338879+0000","last_became_peered":"2024-03-18T20:34:38.338879+0000","last_unstale":"2024-03-18T20:34:44.796411+0000","last_undegraded":"2024-03-18T20:34:44.796411+0000","last_fullsized":"2024-03-18T20:34:44.796411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:33.340151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.820775+0000","last_change":"2024-03-18T20:34:38.817861+0000","last_active":"2024-03-18T20:34:38.820775+0000","last_peered":"2024-03-18T20:34:38.820775+0000","last_clean":"2024-03-18T20:34:38.820775+0000","last_became_active":"2024-03-18T20:34:38.815976+0000","last_became_peered":"2024-03-18T20:34:38.815976+0000","last_unstale":"2024-03-18T20:34:38.820775+0000","last_undegraded":"2024-03-18T20:34:38.820775+0000","last_fullsized":"2024-03-18T20:34:38.820775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:54.070067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.795018+0000","last_change":"2024-03-18T20:34:38.345758+0000","last_active":"2024-03-18T20:34:44.795018+0000","last_peered":"2024-03-18T20:34:44.795018+0000","last_clean":"2024-03-18T20:34:44.795018+0000","last_became_active":"2024-03-18T20:34:38.345417+0000","last_became_peered":"2024-03-18T20:34:38.345417+0000","last_unstale":"2024-03-18T20:34:44.795018+0000","last_undegraded":"2024-03-18T20:34:44.795018+0000","last_fullsized":"2024-03-18T20:34:44.795018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:25.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759430+0000","last_change":"2024-03-18T20:34:38.818745+0000","last_active":"2024-03-18T20:34:39.759430+0000","last_peered":"2024-03-18T20:34:39.759430+0000","last_clean":"2024-03-18T20:34:39.759430+0000","last_became_active":"2024-03-18T20:34:38.818415+0000","last_became_peered":"2024-03-18T20:34:38.818415+0000","last_unstale":"2024-03-18T20:34:39.759430+0000","last_undegraded":"2024-03-18T20:34:39.759430+0000","last_fullsized":"2024-03-18T20:34:39.759430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:45:49.454498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'5","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:12.833432+0000","last_change":"2024-03-18T20:34:38.345804+0000","last_active":"2024-03-18T20:35:12.833432+0000","last_peered":"2024-03-18T20:35:12.833432+0000","last_clean":"2024-03-18T20:35:12.833432+0000","last_became_active":"2024-03-18T20:34:38.345409+0000","last_became_peered":"2024-03-18T20:34:38.345409+0000","last_unstale":"2024-03-18T20:35:12.833432+0000","last_undegraded":"2024-03-18T20:35:12.833432+0000","last_fullsized":"2024-03-18T20:35:12.833432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:00.085412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996703+0000","last_change":"2024-03-18T20:34:38.799821+0000","last_active":"2024-03-18T20:34:39.996703+0000","last_peered":"2024-03-18T20:34:39.996703+0000","last_clean":"2024-03-18T20:34:39.996703+0000","last_became_active":"2024-03-18T20:34:38.799457+0000","last_became_peered":"2024-03-18T20:34:38.799457+0000","last_unstale":"2024-03-18T20:34:39.996703+0000","last_undegraded":"2024-03-18T20:34:39.996703+0000","last_fullsized":"2024-03-18T20:34:39.996703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:39:35.912892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:43.830658+0000","last_change":"2024-03-18T20:34:38.341999+0000","last_active":"2024-03-18T20:34:43.830658+0000","last_peered":"2024-03-18T20:34:43.830658+0000","last_clean":"2024-03-18T20:34:43.830658+0000","last_became_active":"2024-03-18T20:34:38.341431+0000","last_became_peered":"2024-03-18T20:34:38.341431+0000","last_unstale":"2024-03-18T20:34:43.830658+0000","last_undegraded":"2024-03-18T20:34:43.830658+0000","last_fullsized":"2024-03-18T20:34:43.830658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:07:26.711113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759388+0000","last_change":"2024-03-18T20:34:38.804003+0000","last_active":"2024-03-18T20:34:39.759388+0000","last_peered":"2024-03-18T20:34:39.759388+0000","last_clean":"2024-03-18T20:34:39.759388+0000","last_became_active":"2024-03-18T20:34:38.803667+0000","last_became_peered":"2024-03-18T20:34:38.803667+0000","last_unstale":"2024-03-18T20:34:39.759388+0000","last_undegraded":"2024-03-18T20:34:39.759388+0000","last_fullsized":"2024-03-18T20:34:39.759388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:19:58.230790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.822152+0000","last_change":"2024-03-18T20:34:38.346493+0000","last_active":"2024-03-18T20:34:42.822152+0000","last_peered":"2024-03-18T20:34:42.822152+0000","last_clean":"2024-03-18T20:34:42.822152+0000","last_became_active":"2024-03-18T20:34:38.345925+0000","last_became_peered":"2024-03-18T20:34:38.345925+0000","last_unstale":"2024-03-18T20:34:42.822152+0000","last_undegraded":"2024-03-18T20:34:42.822152+0000","last_fullsized":"2024-03-18T20:34:42.822152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:52:47.773860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198488+0000","last_change":"2024-03-18T20:34:38.807284+0000","last_active":"2024-03-18T20:34:39.198488+0000","last_peered":"2024-03-18T20:34:39.198488+0000","last_clean":"2024-03-18T20:34:39.198488+0000","last_became_active":"2024-03-18T20:34:38.806844+0000","last_became_peered":"2024-03-18T20:34:38.806844+0000","last_unstale":"2024-03-18T20:34:39.198488+0000","last_undegraded":"2024-03-18T20:34:39.198488+0000","last_fullsized":"2024-03-18T20:34:39.198488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:44:35.469953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.793789+0000","last_change":"2024-03-18T20:34:38.343090+0000","last_active":"2024-03-18T20:34:44.793789+0000","last_peered":"2024-03-18T20:34:44.793789+0000","last_clean":"2024-03-18T20:34:44.793789+0000","last_became_active":"2024-03-18T20:34:38.342712+0000","last_became_peered":"2024-03-18T20:34:38.342712+0000","last_unstale":"2024-03-18T20:34:44.793789+0000","last_undegraded":"2024-03-18T20:34:44.793789+0000","last_fullsized":"2024-03-18T20:34:44.793789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:14:05.909604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996769+0000","last_change":"2024-03-18T20:34:38.800616+0000","last_active":"2024-03-18T20:34:39.996769+0000","last_peered":"2024-03-18T20:34:39.996769+0000","last_clean":"2024-03-18T20:34:39.996769+0000","last_became_active":"2024-03-18T20:34:38.800340+0000","last_became_peered":"2024-03-18T20:34:38.800340+0000","last_unstale":"2024-03-18T20:34:39.996769+0000","last_undegraded":"2024-03-18T20:34:39.996769+0000","last_fullsized":"2024-03-18T20:34:39.996769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:18.263179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'4","reported_seq":82,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:16.838852+0000","last_change":"2024-03-18T20:34:38.345634+0000","last_active":"2024-03-18T20:35:16.838852+0000","last_peered":"2024-03-18T20:35:16.838852+0000","last_clean":"2024-03-18T20:35:16.838852+0000","last_became_active":"2024-03-18T20:34:38.345275+0000","last_became_peered":"2024-03-18T20:34:38.345275+0000","last_unstale":"2024-03-18T20:35:16.838852+0000","last_undegraded":"2024-03-18T20:35:16.838852+0000","last_fullsized":"2024-03-18T20:35:16.838852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:29.939529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.823111+0000","last_change":"2024-03-18T20:34:38.820286+0000","last_active":"2024-03-18T20:34:38.823111+0000","last_peered":"2024-03-18T20:34:38.823111+0000","last_clean":"2024-03-18T20:34:38.823111+0000","last_became_active":"2024-03-18T20:34:38.819939+0000","last_became_peered":"2024-03-18T20:34:38.819939+0000","last_unstale":"2024-03-18T20:34:38.823111+0000","last_undegraded":"2024-03-18T20:34:38.823111+0000","last_fullsized":"2024-03-18T20:34:38.823111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:19.229339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.794842+0000","last_change":"2024-03-18T20:34:38.341699+0000","last_active":"2024-03-18T20:34:44.794842+0000","last_peered":"2024-03-18T20:34:44.794842+0000","last_clean":"2024-03-18T20:34:44.794842+0000","last_became_active":"2024-03-18T20:34:38.341390+0000","last_became_peered":"2024-03-18T20:34:38.341390+0000","last_unstale":"2024-03-18T20:34:44.794842+0000","last_undegraded":"2024-03-18T20:34:44.794842+0000","last_fullsized":"2024-03-18T20:34:44.794842+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:16:02.239414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901314+0000","last_change":"2024-03-18T20:34:38.805316+0000","last_active":"2024-03-18T20:34:39.901314+0000","last_peered":"2024-03-18T20:34:39.901314+0000","last_clean":"2024-03-18T20:34:39.901314+0000","last_became_active":"2024-03-18T20:34:38.804967+0000","last_became_peered":"2024-03-18T20:34:38.804967+0000","last_unstale":"2024-03-18T20:34:39.901314+0000","last_undegraded":"2024-03-18T20:34:39.901314+0000","last_fullsized":"2024-03-18T20:34:39.901314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:54.591401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:43.829682+0000","last_change":"2024-03-18T20:34:38.346821+0000","last_active":"2024-03-18T20:34:43.829682+0000","last_peered":"2024-03-18T20:34:43.829682+0000","last_clean":"2024-03-18T20:34:43.829682+0000","last_became_active":"2024-03-18T20:34:38.346314+0000","last_became_peered":"2024-03-18T20:34:38.346314+0000","last_unstale":"2024-03-18T20:34:43.829682+0000","last_undegraded":"2024-03-18T20:34:43.829682+0000","last_fullsized":"2024-03-18T20:34:43.829682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:12:04.552971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.823287+0000","last_change":"2024-03-18T20:34:38.820479+0000","last_active":"2024-03-18T20:34:38.823287+0000","last_peered":"2024-03-18T20:34:38.823287+0000","last_clean":"2024-03-18T20:34:38.823287+0000","last_became_active":"2024-03-18T20:34:38.820186+0000","last_became_peered":"2024-03-18T20:34:38.820186+0000","last_unstale":"2024-03-18T20:34:38.823287+0000","last_undegraded":"2024-03-18T20:34:38.823287+0000","last_fullsized":"2024-03-18T20:34:38.823287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:15.953764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'29","reported_seq":113,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:16.838853+0000","last_change":"2024-03-18T20:34:38.344332+0000","last_active":"2024-03-18T20:35:16.838853+0000","last_peered":"2024-03-18T20:35:16.838853+0000","last_clean":"2024-03-18T20:35:16.838853+0000","last_became_active":"2024-03-18T20:34:38.343827+0000","last_became_peered":"2024-03-18T20:34:38.343827+0000","last_unstale":"2024-03-18T20:35:16.838853+0000","last_undegraded":"2024-03-18T20:35:16.838853+0000","last_fullsized":"2024-03-18T20:35:16.838853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:13:21.725004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59418,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":72,"num_read_kb":94,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996827+0000","last_change":"2024-03-18T20:34:38.814794+0000","last_active":"2024-03-18T20:34:39.996827+0000","last_peered":"2024-03-18T20:34:39.996827+0000","last_clean":"2024-03-18T20:34:39.996827+0000","last_became_active":"2024-03-18T20:34:38.814434+0000","last_became_peered":"2024-03-18T20:34:38.814434+0000","last_unstale":"2024-03-18T20:34:39.996827+0000","last_undegraded":"2024-03-18T20:34:39.996827+0000","last_fullsized":"2024-03-18T20:34:39.996827+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:43:19.083212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.794905+0000","last_change":"2024-03-18T20:34:38.343894+0000","last_active":"2024-03-18T20:34:44.794905+0000","last_peered":"2024-03-18T20:34:44.794905+0000","last_clean":"2024-03-18T20:34:44.794905+0000","last_became_active":"2024-03-18T20:34:38.343493+0000","last_became_peered":"2024-03-18T20:34:38.343493+0000","last_unstale":"2024-03-18T20:34:44.794905+0000","last_undegraded":"2024-03-18T20:34:44.794905+0000","last_fullsized":"2024-03-18T20:34:44.794905+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:13:32.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793766+0000","last_change":"2024-03-18T20:34:38.809968+0000","last_active":"2024-03-18T20:34:39.793766+0000","last_peered":"2024-03-18T20:34:39.793766+0000","last_clean":"2024-03-18T20:34:39.793766+0000","last_became_active":"2024-03-18T20:34:38.809641+0000","last_became_peered":"2024-03-18T20:34:38.809641+0000","last_unstale":"2024-03-18T20:34:39.793766+0000","last_undegraded":"2024-03-18T20:34:39.793766+0000","last_fullsized":"2024-03-18T20:34:39.793766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:50:52.925683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.793952+0000","last_change":"2024-03-18T20:34:38.345621+0000","last_active":"2024-03-18T20:34:44.793952+0000","last_peered":"2024-03-18T20:34:44.793952+0000","last_clean":"2024-03-18T20:34:44.793952+0000","last_became_active":"2024-03-18T20:34:38.345255+0000","last_became_peered":"2024-03-18T20:34:38.345255+0000","last_unstale":"2024-03-18T20:34:44.793952+0000","last_undegraded":"2024-03-18T20:34:44.793952+0000","last_fullsized":"2024-03-18T20:34:44.793952+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:17:03.737973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997875+0000","last_change":"2024-03-18T20:34:38.804390+0000","last_active":"2024-03-18T20:34:39.997875+0000","last_peered":"2024-03-18T20:34:39.997875+0000","last_clean":"2024-03-18T20:34:39.997875+0000","last_became_active":"2024-03-18T20:34:38.804033+0000","last_became_peered":"2024-03-18T20:34:38.804033+0000","last_unstale":"2024-03-18T20:34:39.997875+0000","last_undegraded":"2024-03-18T20:34:39.997875+0000","last_fullsized":"2024-03-18T20:34:39.997875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:09:45.407921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:16.844019+0000","last_change":"2024-03-18T20:34:38.340239+0000","last_active":"2024-03-18T20:35:16.844019+0000","last_peered":"2024-03-18T20:35:16.844019+0000","last_clean":"2024-03-18T20:35:16.844019+0000","last_became_active":"2024-03-18T20:34:38.339880+0000","last_became_peered":"2024-03-18T20:34:38.339880+0000","last_unstale":"2024-03-18T20:35:16.844019+0000","last_undegraded":"2024-03-18T20:35:16.844019+0000","last_fullsized":"2024-03-18T20:35:16.844019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:55:46.864885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998003+0000","last_change":"2024-03-18T20:34:38.819223+0000","last_active":"2024-03-18T20:34:39.998003+0000","last_peered":"2024-03-18T20:34:39.998003+0000","last_clean":"2024-03-18T20:34:39.998003+0000","last_became_active":"2024-03-18T20:34:38.818879+0000","last_became_peered":"2024-03-18T20:34:38.818879+0000","last_unstale":"2024-03-18T20:34:39.998003+0000","last_undegraded":"2024-03-18T20:34:39.998003+0000","last_fullsized":"2024-03-18T20:34:39.998003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:45.645912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":30,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.785345+0000","last_change":"2024-03-18T20:34:38.336817+0000","last_active":"2024-03-18T20:34:44.785345+0000","last_peered":"2024-03-18T20:34:44.785345+0000","last_clean":"2024-03-18T20:34:44.785345+0000","last_became_active":"2024-03-18T20:34:38.336453+0000","last_became_peered":"2024-03-18T20:34:38.336453+0000","last_unstale":"2024-03-18T20:34:44.785345+0000","last_undegraded":"2024-03-18T20:34:44.785345+0000","last_fullsized":"2024-03-18T20:34:44.785345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:10.253435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759322+0000","last_change":"2024-03-18T20:34:38.818048+0000","last_active":"2024-03-18T20:34:39.759322+0000","last_peered":"2024-03-18T20:34:39.759322+0000","last_clean":"2024-03-18T20:34:39.759322+0000","last_became_active":"2024-03-18T20:34:38.817682+0000","last_became_peered":"2024-03-18T20:34:38.817682+0000","last_unstale":"2024-03-18T20:34:39.759322+0000","last_undegraded":"2024-03-18T20:34:39.759322+0000","last_fullsized":"2024-03-18T20:34:39.759322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:25:55.291815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.785348+0000","last_change":"2024-03-18T20:34:38.345773+0000","last_active":"2024-03-18T20:34:44.785348+0000","last_peered":"2024-03-18T20:34:44.785348+0000","last_clean":"2024-03-18T20:34:44.785348+0000","last_became_active":"2024-03-18T20:34:38.345432+0000","last_became_peered":"2024-03-18T20:34:38.345432+0000","last_unstale":"2024-03-18T20:34:44.785348+0000","last_undegraded":"2024-03-18T20:34:44.785348+0000","last_fullsized":"2024-03-18T20:34:44.785348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:56:57.657424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901903+0000","last_change":"2024-03-18T20:34:38.806138+0000","last_active":"2024-03-18T20:34:39.901903+0000","last_peered":"2024-03-18T20:34:39.901903+0000","last_clean":"2024-03-18T20:34:39.901903+0000","last_became_active":"2024-03-18T20:34:38.805821+0000","last_became_peered":"2024-03-18T20:34:38.805821+0000","last_unstale":"2024-03-18T20:34:39.901903+0000","last_undegraded":"2024-03-18T20:34:39.901903+0000","last_fullsized":"2024-03-18T20:34:39.901903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:25:57.887813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.173015+0000","last_change":"2024-03-18T20:34:38.338751+0000","last_active":"2024-03-18T20:34:41.173015+0000","last_peered":"2024-03-18T20:34:41.173015+0000","last_clean":"2024-03-18T20:34:41.173015+0000","last_became_active":"2024-03-18T20:34:38.337495+0000","last_became_peered":"2024-03-18T20:34:38.337495+0000","last_unstale":"2024-03-18T20:34:41.173015+0000","last_undegraded":"2024-03-18T20:34:41.173015+0000","last_fullsized":"2024-03-18T20:34:41.173015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:38:04.402380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.784913+0000","last_change":"2024-03-18T20:34:38.803937+0000","last_active":"2024-03-18T20:34:42.784913+0000","last_peered":"2024-03-18T20:34:42.784913+0000","last_clean":"2024-03-18T20:34:42.784913+0000","last_became_active":"2024-03-18T20:34:38.803520+0000","last_became_peered":"2024-03-18T20:34:38.803520+0000","last_unstale":"2024-03-18T20:34:42.784913+0000","last_undegraded":"2024-03-18T20:34:42.784913+0000","last_fullsized":"2024-03-18T20:34:42.784913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:20:37.537265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":48,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:13.860539+0000","last_change":"2024-03-18T20:34:38.337481+0000","last_active":"2024-03-18T20:35:13.860539+0000","last_peered":"2024-03-18T20:35:13.860539+0000","last_clean":"2024-03-18T20:35:13.860539+0000","last_became_active":"2024-03-18T20:34:38.337152+0000","last_became_peered":"2024-03-18T20:34:38.337152+0000","last_unstale":"2024-03-18T20:35:13.860539+0000","last_undegraded":"2024-03-18T20:35:13.860539+0000","last_fullsized":"2024-03-18T20:35:13.860539+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:07:59.593978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997194+0000","last_change":"2024-03-18T20:34:38.814240+0000","last_active":"2024-03-18T20:34:39.997194+0000","last_peered":"2024-03-18T20:34:39.997194+0000","last_clean":"2024-03-18T20:34:39.997194+0000","last_became_active":"2024-03-18T20:34:38.813177+0000","last_became_peered":"2024-03-18T20:34:38.813177+0000","last_unstale":"2024-03-18T20:34:39.997194+0000","last_undegraded":"2024-03-18T20:34:39.997194+0000","last_fullsized":"2024-03-18T20:34:39.997194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:22:30.076880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759286+0000","last_change":"2024-03-18T20:34:38.806133+0000","last_active":"2024-03-18T20:34:39.759286+0000","last_peered":"2024-03-18T20:34:39.759286+0000","last_clean":"2024-03-18T20:34:39.759286+0000","last_became_active":"2024-03-18T20:34:38.805257+0000","last_became_peered":"2024-03-18T20:34:38.805257+0000","last_unstale":"2024-03-18T20:34:39.759286+0000","last_undegraded":"2024-03-18T20:34:39.759286+0000","last_fullsized":"2024-03-18T20:34:39.759286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:05:15.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:15.841783+0000","last_change":"2024-03-18T20:34:38.346271+0000","last_active":"2024-03-18T20:35:15.841783+0000","last_peered":"2024-03-18T20:35:15.841783+0000","last_clean":"2024-03-18T20:35:15.841783+0000","last_became_active":"2024-03-18T20:34:38.345054+0000","last_became_peered":"2024-03-18T20:34:38.345054+0000","last_unstale":"2024-03-18T20:35:15.841783+0000","last_undegraded":"2024-03-18T20:35:15.841783+0000","last_fullsized":"2024-03-18T20:35:15.841783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:42:52.510294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198582+0000","last_change":"2024-03-18T20:34:38.816834+0000","last_active":"2024-03-18T20:34:39.198582+0000","last_peered":"2024-03-18T20:34:39.198582+0000","last_clean":"2024-03-18T20:34:39.198582+0000","last_became_active":"2024-03-18T20:34:38.816516+0000","last_became_peered":"2024-03-18T20:34:38.816516+0000","last_unstale":"2024-03-18T20:34:39.198582+0000","last_undegraded":"2024-03-18T20:34:39.198582+0000","last_fullsized":"2024-03-18T20:34:39.198582+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:33:54.529206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.793259+0000","last_change":"2024-03-18T20:34:38.339407+0000","last_active":"2024-03-18T20:34:44.793259+0000","last_peered":"2024-03-18T20:34:44.793259+0000","last_clean":"2024-03-18T20:34:44.793259+0000","last_became_active":"2024-03-18T20:34:38.339086+0000","last_became_peered":"2024-03-18T20:34:38.339086+0000","last_unstale":"2024-03-18T20:34:44.793259+0000","last_undegraded":"2024-03-18T20:34:44.793259+0000","last_fullsized":"2024-03-18T20:34:44.793259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:18.479630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":331,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998119+0000","last_change":"2024-03-18T20:33:50.777651+0000","last_active":"2024-03-18T20:34:39.998119+0000","last_peered":"2024-03-18T20:34:39.998119+0000","last_clean":"2024-03-18T20:34:39.998119+0000","last_became_active":"2024-03-18T20:33:50.777281+0000","last_became_peered":"2024-03-18T20:33:50.777281+0000","last_unstale":"2024-03-18T20:34:39.998119+0000","last_undegraded":"2024-03-18T20:34:39.998119+0000","last_fullsized":"2024-03-18T20:34:39.998119+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_clean_scrub_stamp":"2024-03-18T20:31:27.488744+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:28.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793222+0000","last_change":"2024-03-18T20:34:38.812714+0000","last_active":"2024-03-18T20:34:39.793222+0000","last_peered":"2024-03-18T20:34:39.793222+0000","last_clean":"2024-03-18T20:34:39.793222+0000","last_became_active":"2024-03-18T20:34:38.812464+0000","last_became_peered":"2024-03-18T20:34:38.812464+0000","last_unstale":"2024-03-18T20:34:39.793222+0000","last_undegraded":"2024-03-18T20:34:39.793222+0000","last_fullsized":"2024-03-18T20:34:39.793222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:03:13.853740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:42.820233+0000","last_change":"2024-03-18T20:34:38.343173+0000","last_active":"2024-03-18T20:34:42.820233+0000","last_peered":"2024-03-18T20:34:42.820233+0000","last_clean":"2024-03-18T20:34:42.820233+0000","last_became_active":"2024-03-18T20:34:38.342774+0000","last_became_peered":"2024-03-18T20:34:38.342774+0000","last_unstale":"2024-03-18T20:34:42.820233+0000","last_undegraded":"2024-03-18T20:34:42.820233+0000","last_fullsized":"2024-03-18T20:34:42.820233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:52.283033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.785384+0000","last_change":"2024-03-18T20:34:38.344539+0000","last_active":"2024-03-18T20:34:44.785384+0000","last_peered":"2024-03-18T20:34:44.785384+0000","last_clean":"2024-03-18T20:34:44.785384+0000","last_became_active":"2024-03-18T20:34:38.344188+0000","last_became_peered":"2024-03-18T20:34:38.344188+0000","last_unstale":"2024-03-18T20:34:44.785384+0000","last_undegraded":"2024-03-18T20:34:44.785384+0000","last_fullsized":"2024-03-18T20:34:44.785384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:38.641928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793487+0000","last_change":"2024-03-18T20:34:38.806602+0000","last_active":"2024-03-18T20:34:39.793487+0000","last_peered":"2024-03-18T20:34:39.793487+0000","last_clean":"2024-03-18T20:34:39.793487+0000","last_became_active":"2024-03-18T20:34:38.806241+0000","last_became_peered":"2024-03-18T20:34:38.806241+0000","last_unstale":"2024-03-18T20:34:39.793487+0000","last_undegraded":"2024-03-18T20:34:39.793487+0000","last_fullsized":"2024-03-18T20:34:39.793487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:04.463311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'9","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:47.937001+0000","last_change":"2024-03-18T20:34:38.339695+0000","last_active":"2024-03-18T20:34:47.937001+0000","last_peered":"2024-03-18T20:34:47.937001+0000","last_clean":"2024-03-18T20:34:47.937001+0000","last_became_active":"2024-03-18T20:34:38.339331+0000","last_became_peered":"2024-03-18T20:34:38.339331+0000","last_unstale":"2024-03-18T20:34:47.937001+0000","last_undegraded":"2024-03-18T20:34:47.937001+0000","last_fullsized":"2024-03-18T20:34:47.937001+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:57:06.434289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198695+0000","last_change":"2024-03-18T20:34:38.809754+0000","last_active":"2024-03-18T20:34:39.198695+0000","last_peered":"2024-03-18T20:34:39.198695+0000","last_clean":"2024-03-18T20:34:39.198695+0000","last_became_active":"2024-03-18T20:34:38.809415+0000","last_became_peered":"2024-03-18T20:34:38.809415+0000","last_unstale":"2024-03-18T20:34:39.198695+0000","last_undegraded":"2024-03-18T20:34:39.198695+0000","last_fullsized":"2024-03-18T20:34:39.198695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:34:30.761582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.793801+0000","last_change":"2024-03-18T20:34:38.342080+0000","last_active":"2024-03-18T20:34:44.793801+0000","last_peered":"2024-03-18T20:34:44.793801+0000","last_clean":"2024-03-18T20:34:44.793801+0000","last_became_active":"2024-03-18T20:34:38.340623+0000","last_became_peered":"2024-03-18T20:34:38.340623+0000","last_unstale":"2024-03-18T20:34:44.793801+0000","last_undegraded":"2024-03-18T20:34:44.793801+0000","last_fullsized":"2024-03-18T20:34:44.793801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:49:14.912391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901335+0000","last_change":"2024-03-18T20:34:38.806558+0000","last_active":"2024-03-18T20:34:39.901335+0000","last_peered":"2024-03-18T20:34:39.901335+0000","last_clean":"2024-03-18T20:34:39.901335+0000","last_became_active":"2024-03-18T20:34:38.806171+0000","last_became_peered":"2024-03-18T20:34:38.806171+0000","last_unstale":"2024-03-18T20:34:39.901335+0000","last_undegraded":"2024-03-18T20:34:39.901335+0000","last_fullsized":"2024-03-18T20:34:39.901335+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:46:26.530157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.813762+0000","last_change":"2024-03-18T20:34:38.336175+0000","last_active":"2024-03-18T20:34:41.813762+0000","last_peered":"2024-03-18T20:34:41.813762+0000","last_clean":"2024-03-18T20:34:41.813762+0000","last_became_active":"2024-03-18T20:34:38.335798+0000","last_became_peered":"2024-03-18T20:34:38.335798+0000","last_unstale":"2024-03-18T20:34:41.813762+0000","last_undegraded":"2024-03-18T20:34:41.813762+0000","last_fullsized":"2024-03-18T20:34:41.813762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:33:33.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.996504+0000","last_change":"2024-03-18T20:34:38.807684+0000","last_active":"2024-03-18T20:34:39.996504+0000","last_peered":"2024-03-18T20:34:39.996504+0000","last_clean":"2024-03-18T20:34:39.996504+0000","last_became_active":"2024-03-18T20:34:38.807300+0000","last_became_peered":"2024-03-18T20:34:38.807300+0000","last_unstale":"2024-03-18T20:34:39.996504+0000","last_undegraded":"2024-03-18T20:34:39.996504+0000","last_fullsized":"2024-03-18T20:34:39.996504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:31:43.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.793201+0000","last_change":"2024-03-18T20:34:38.341802+0000","last_active":"2024-03-18T20:34:44.793201+0000","last_peered":"2024-03-18T20:34:44.793201+0000","last_clean":"2024-03-18T20:34:44.793201+0000","last_became_active":"2024-03-18T20:34:38.340594+0000","last_became_peered":"2024-03-18T20:34:38.340594+0000","last_unstale":"2024-03-18T20:34:44.793201+0000","last_undegraded":"2024-03-18T20:34:44.793201+0000","last_fullsized":"2024-03-18T20:34:44.793201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:01:06.830323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901738+0000","last_change":"2024-03-18T20:34:38.809790+0000","last_active":"2024-03-18T20:34:39.901738+0000","last_peered":"2024-03-18T20:34:39.901738+0000","last_clean":"2024-03-18T20:34:39.901738+0000","last_became_active":"2024-03-18T20:34:38.809470+0000","last_became_peered":"2024-03-18T20:34:38.809470+0000","last_unstale":"2024-03-18T20:34:39.901738+0000","last_undegraded":"2024-03-18T20:34:39.901738+0000","last_fullsized":"2024-03-18T20:34:39.901738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:48.225759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.155918+0000","last_change":"2024-03-18T20:34:38.343103+0000","last_active":"2024-03-18T20:34:44.155918+0000","last_peered":"2024-03-18T20:34:44.155918+0000","last_clean":"2024-03-18T20:34:44.155918+0000","last_became_active":"2024-03-18T20:34:38.342744+0000","last_became_peered":"2024-03-18T20:34:38.342744+0000","last_unstale":"2024-03-18T20:34:44.155918+0000","last_undegraded":"2024-03-18T20:34:44.155918+0000","last_fullsized":"2024-03-18T20:34:44.155918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:39.078219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198604+0000","last_change":"2024-03-18T20:34:38.809294+0000","last_active":"2024-03-18T20:34:39.198604+0000","last_peered":"2024-03-18T20:34:39.198604+0000","last_clean":"2024-03-18T20:34:39.198604+0000","last_became_active":"2024-03-18T20:34:38.808542+0000","last_became_peered":"2024-03-18T20:34:38.808542+0000","last_unstale":"2024-03-18T20:34:39.198604+0000","last_undegraded":"2024-03-18T20:34:39.198604+0000","last_fullsized":"2024-03-18T20:34:39.198604+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:11:30.669317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:41.813772+0000","last_change":"2024-03-18T20:34:38.339454+0000","last_active":"2024-03-18T20:34:41.813772+0000","last_peered":"2024-03-18T20:34:41.813772+0000","last_clean":"2024-03-18T20:34:41.813772+0000","last_became_active":"2024-03-18T20:34:38.339154+0000","last_became_peered":"2024-03-18T20:34:38.339154+0000","last_unstale":"2024-03-18T20:34:41.813772+0000","last_undegraded":"2024-03-18T20:34:41.813772+0000","last_fullsized":"2024-03-18T20:34:41.813772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:12:06.300600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.998255+0000","last_change":"2024-03-18T20:34:38.818462+0000","last_active":"2024-03-18T20:34:39.998255+0000","last_peered":"2024-03-18T20:34:39.998255+0000","last_clean":"2024-03-18T20:34:39.998255+0000","last_became_active":"2024-03-18T20:34:38.817522+0000","last_became_peered":"2024-03-18T20:34:38.817522+0000","last_unstale":"2024-03-18T20:34:39.998255+0000","last_undegraded":"2024-03-18T20:34:39.998255+0000","last_fullsized":"2024-03-18T20:34:39.998255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:50:32.258583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'5","reported_seq":52,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:35:15.837985+0000","last_change":"2024-03-18T20:34:38.341054+0000","last_active":"2024-03-18T20:35:15.837985+0000","last_peered":"2024-03-18T20:35:15.837985+0000","last_clean":"2024-03-18T20:35:15.837985+0000","last_became_active":"2024-03-18T20:34:38.340352+0000","last_became_peered":"2024-03-18T20:34:38.340352+0000","last_unstale":"2024-03-18T20:35:15.837985+0000","last_undegraded":"2024-03-18T20:35:15.837985+0000","last_fullsized":"2024-03-18T20:35:15.837985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:44.850244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997144+0000","last_change":"2024-03-18T20:34:38.818176+0000","last_active":"2024-03-18T20:34:39.997144+0000","last_peered":"2024-03-18T20:34:39.997144+0000","last_clean":"2024-03-18T20:34:39.997144+0000","last_became_active":"2024-03-18T20:34:38.817820+0000","last_became_peered":"2024-03-18T20:34:38.817820+0000","last_unstale":"2024-03-18T20:34:39.997144+0000","last_undegraded":"2024-03-18T20:34:39.997144+0000","last_fullsized":"2024-03-18T20:34:39.997144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:19:29.815174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.785145+0000","last_change":"2024-03-18T20:34:38.346591+0000","last_active":"2024-03-18T20:34:44.785145+0000","last_peered":"2024-03-18T20:34:44.785145+0000","last_clean":"2024-03-18T20:34:44.785145+0000","last_became_active":"2024-03-18T20:34:38.345897+0000","last_became_peered":"2024-03-18T20:34:38.345897+0000","last_unstale":"2024-03-18T20:34:44.785145+0000","last_undegraded":"2024-03-18T20:34:44.785145+0000","last_fullsized":"2024-03-18T20:34:44.785145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:47:32.118905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:40.284498+0000","last_change":"2024-03-18T20:34:38.804814+0000","last_active":"2024-03-18T20:34:40.284498+0000","last_peered":"2024-03-18T20:34:40.284498+0000","last_clean":"2024-03-18T20:34:40.284498+0000","last_became_active":"2024-03-18T20:34:38.804585+0000","last_became_peered":"2024-03-18T20:34:38.804585+0000","last_unstale":"2024-03-18T20:34:40.284498+0000","last_undegraded":"2024-03-18T20:34:40.284498+0000","last_fullsized":"2024-03-18T20:34:40.284498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:40:47.273070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:44.793953+0000","last_change":"2024-03-18T20:34:38.344615+0000","last_active":"2024-03-18T20:34:44.793953+0000","last_peered":"2024-03-18T20:34:44.793953+0000","last_clean":"2024-03-18T20:34:44.793953+0000","last_became_active":"2024-03-18T20:34:38.344242+0000","last_became_peered":"2024-03-18T20:34:38.344242+0000","last_unstale":"2024-03-18T20:34:44.793953+0000","last_undegraded":"2024-03-18T20:34:44.793953+0000","last_fullsized":"2024-03-18T20:34:44.793953+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.449808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997264+0000","last_change":"2024-03-18T20:34:38.820875+0000","last_active":"2024-03-18T20:34:39.997264+0000","last_peered":"2024-03-18T20:34:39.997264+0000","last_clean":"2024-03-18T20:34:39.997264+0000","last_became_active":"2024-03-18T20:34:38.820550+0000","last_became_peered":"2024-03-18T20:34:38.820550+0000","last_unstale":"2024-03-18T20:34:39.997264+0000","last_undegraded":"2024-03-18T20:34:39.997264+0000","last_fullsized":"2024-03-18T20:34:39.997264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:49:36.358734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901824+0000","last_change":"2024-03-18T20:34:38.812552+0000","last_active":"2024-03-18T20:34:39.901824+0000","last_peered":"2024-03-18T20:34:39.901824+0000","last_clean":"2024-03-18T20:34:39.901824+0000","last_became_active":"2024-03-18T20:34:38.811939+0000","last_became_peered":"2024-03-18T20:34:38.811939+0000","last_unstale":"2024-03-18T20:34:39.901824+0000","last_undegraded":"2024-03-18T20:34:39.901824+0000","last_fullsized":"2024-03-18T20:34:39.901824+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:41:52.096974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997308+0000","last_change":"2024-03-18T20:34:38.815691+0000","last_active":"2024-03-18T20:34:39.997308+0000","last_peered":"2024-03-18T20:34:39.997308+0000","last_clean":"2024-03-18T20:34:39.997308+0000","last_became_active":"2024-03-18T20:34:38.815370+0000","last_became_peered":"2024-03-18T20:34:38.815370+0000","last_unstale":"2024-03-18T20:34:39.997308+0000","last_undegraded":"2024-03-18T20:34:39.997308+0000","last_fullsized":"2024-03-18T20:34:39.997308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:59:41.260013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997260+0000","last_change":"2024-03-18T20:34:38.812651+0000","last_active":"2024-03-18T20:34:39.997260+0000","last_peered":"2024-03-18T20:34:39.997260+0000","last_clean":"2024-03-18T20:34:39.997260+0000","last_became_active":"2024-03-18T20:34:38.812322+0000","last_became_peered":"2024-03-18T20:34:38.812322+0000","last_unstale":"2024-03-18T20:34:39.997260+0000","last_undegraded":"2024-03-18T20:34:39.997260+0000","last_fullsized":"2024-03-18T20:34:39.997260+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:34:55.658371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901247+0000","last_change":"2024-03-18T20:34:38.806475+0000","last_active":"2024-03-18T20:34:39.901247+0000","last_peered":"2024-03-18T20:34:39.901247+0000","last_clean":"2024-03-18T20:34:39.901247+0000","last_became_active":"2024-03-18T20:34:38.806027+0000","last_became_peered":"2024-03-18T20:34:38.806027+0000","last_unstale":"2024-03-18T20:34:39.901247+0000","last_undegraded":"2024-03-18T20:34:39.901247+0000","last_fullsized":"2024-03-18T20:34:39.901247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:57:17.140031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793496+0000","last_change":"2024-03-18T20:34:38.811057+0000","last_active":"2024-03-18T20:34:39.793496+0000","last_peered":"2024-03-18T20:34:39.793496+0000","last_clean":"2024-03-18T20:34:39.793496+0000","last_became_active":"2024-03-18T20:34:38.810316+0000","last_became_peered":"2024-03-18T20:34:38.810316+0000","last_unstale":"2024-03-18T20:34:39.793496+0000","last_undegraded":"2024-03-18T20:34:39.793496+0000","last_fullsized":"2024-03-18T20:34:39.793496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:29:22.426717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.902010+0000","last_change":"2024-03-18T20:34:38.812952+0000","last_active":"2024-03-18T20:34:39.902010+0000","last_peered":"2024-03-18T20:34:39.902010+0000","last_clean":"2024-03-18T20:34:39.902010+0000","last_became_active":"2024-03-18T20:34:38.812550+0000","last_became_peered":"2024-03-18T20:34:38.812550+0000","last_unstale":"2024-03-18T20:34:39.902010+0000","last_undegraded":"2024-03-18T20:34:39.902010+0000","last_fullsized":"2024-03-18T20:34:39.902010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:06:26.525078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.997748+0000","last_change":"2024-03-18T20:34:38.817358+0000","last_active":"2024-03-18T20:34:39.997748+0000","last_peered":"2024-03-18T20:34:39.997748+0000","last_clean":"2024-03-18T20:34:39.997748+0000","last_became_active":"2024-03-18T20:34:38.816939+0000","last_became_peered":"2024-03-18T20:34:38.816939+0000","last_unstale":"2024-03-18T20:34:39.997748+0000","last_undegraded":"2024-03-18T20:34:39.997748+0000","last_fullsized":"2024-03-18T20:34:39.997748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:33.258162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198671+0000","last_change":"2024-03-18T20:34:38.807798+0000","last_active":"2024-03-18T20:34:39.198671+0000","last_peered":"2024-03-18T20:34:39.198671+0000","last_clean":"2024-03-18T20:34:39.198671+0000","last_became_active":"2024-03-18T20:34:38.807489+0000","last_became_peered":"2024-03-18T20:34:38.807489+0000","last_unstale":"2024-03-18T20:34:39.198671+0000","last_undegraded":"2024-03-18T20:34:39.198671+0000","last_fullsized":"2024-03-18T20:34:39.198671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:15.395763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.759267+0000","last_change":"2024-03-18T20:34:38.806363+0000","last_active":"2024-03-18T20:34:39.759267+0000","last_peered":"2024-03-18T20:34:39.759267+0000","last_clean":"2024-03-18T20:34:39.759267+0000","last_became_active":"2024-03-18T20:34:38.805707+0000","last_became_peered":"2024-03-18T20:34:38.805707+0000","last_unstale":"2024-03-18T20:34:39.759267+0000","last_undegraded":"2024-03-18T20:34:39.759267+0000","last_fullsized":"2024-03-18T20:34:39.759267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:10.918979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.198542+0000","last_change":"2024-03-18T20:34:38.814439+0000","last_active":"2024-03-18T20:34:39.198542+0000","last_peered":"2024-03-18T20:34:39.198542+0000","last_clean":"2024-03-18T20:34:39.198542+0000","last_became_active":"2024-03-18T20:34:38.813988+0000","last_became_peered":"2024-03-18T20:34:38.813988+0000","last_unstale":"2024-03-18T20:34:39.198542+0000","last_undegraded":"2024-03-18T20:34:39.198542+0000","last_fullsized":"2024-03-18T20:34:39.198542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:03:14.639044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-18T20:34:38.821809+0000","last_change":"2024-03-18T20:34:38.818550+0000","last_active":"2024-03-18T20:34:38.821809+0000","last_peered":"2024-03-18T20:34:38.821809+0000","last_clean":"2024-03-18T20:34:38.821809+0000","last_became_active":"2024-03-18T20:34:38.818209+0000","last_became_peered":"2024-03-18T20:34:38.818209+0000","last_unstale":"2024-03-18T20:34:38.821809+0000","last_undegraded":"2024-03-18T20:34:38.821809+0000","last_fullsized":"2024-03-18T20:34:38.821809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:40:48.395331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.793586+0000","last_change":"2024-03-18T20:34:38.808439+0000","last_active":"2024-03-18T20:34:39.793586+0000","last_peered":"2024-03-18T20:34:39.793586+0000","last_clean":"2024-03-18T20:34:39.793586+0000","last_became_active":"2024-03-18T20:34:38.807995+0000","last_became_peered":"2024-03-18T20:34:38.807995+0000","last_unstale":"2024-03-18T20:34:39.793586+0000","last_undegraded":"2024-03-18T20:34:39.793586+0000","last_fullsized":"2024-03-18T20:34:39.793586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:54.414148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-03-18T20:34:39.901907+0000","last_change":"2024-03-18T20:34:38.810793+0000","last_active":"2024-03-18T20:34:39.901907+0000","last_peered":"2024-03-18T20:34:39.901907+0000","last_clean":"2024-03-18T20:34:39.901907+0000","last_became_active":"2024-03-18T20:34:38.810496+0000","last_became_peered":"2024-03-18T20:34:38.810496+0000","last_unstale":"2024-03-18T20:34:39.901907+0000","last_undegraded":"2024-03-18T20:34:39.901907+0000","last_fullsized":"2024-03-18T20:34:39.901907+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:33.780675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66040,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":359,"num_read_kb":254,"num_write":229,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":198120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612625,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":851968,"data_stored":405743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.39400000000000002}]},{"osd":1,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.68700000000000006}]},{"osd":2,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57699999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.374},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.46000000000000002}]},{"osd":6,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.53600000000000003}]},{"osd":7,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.65200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":9,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.67200000000000004}]},{"osd":10,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.71799999999999997}]}]},{"osd":10,"up_from":66,"seq":283467841557,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1056251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":1,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.42899999999999999}]},{"osd":2,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.55900000000000005}]},{"osd":3,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.63500000000000001}]},{"osd":4,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58399999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.61399999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.746}]},{"osd":9,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.72399999999999998}]},{"osd":11,"last update":"Mon Mar 18 20:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.79400000000000004}]}]},{"osd":9,"up_from":61,"seq":261993005081,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":405721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.505},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.66800000000000004}]},{"osd":1,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.78000000000000003}]},{"osd":2,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.59099999999999997}]},{"osd":4,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.81799999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.75}]},{"osd":6,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69099999999999995}]},{"osd":8,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.77400000000000002}]},{"osd":10,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.72199999999999998}]}]},{"osd":8,"up_from":54,"seq":231928234013,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":872448,"data_stored":406099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.38500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.622,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60199999999999998}]},{"osd":1,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73399999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63500000000000001}]},{"osd":2,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42299999999999999,"15min":0.41399999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.5},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.32600000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.47599999999999998}]},{"osd":3,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.73999999999999999}]},{"osd":4,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.749,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61599999999999999}]},{"osd":5,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.51100000000000001,"15min":0.498},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.34699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65000000000000002}]},{"osd":6,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.66200000000000003,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.90400000000000003}]},{"osd":7,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.874,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.71599999999999997}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.753},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.88400000000000001}]},{"osd":10,"last update":"Mon Mar 18 20:34:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.84599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]}]},{"osd":7,"up_from":47,"seq":201863462943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1470464,"data_stored":998474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.439}]},{"osd":1,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.42199999999999999,"15min":0.41399999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.36799999999999999}]},{"osd":2,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73799999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.62}]},{"osd":3,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.72999999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.53100000000000003}]},{"osd":5,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.40500000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.55800000000000005}]},{"osd":6,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89700000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.57799999999999996}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63600000000000001}]},{"osd":9,"last update":"Mon Mar 18 20:34:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.69299999999999995}]},{"osd":10,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.72699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]}]},{"osd":6,"up_from":42,"seq":180388626467,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":405390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.437},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.39100000000000001}]},{"osd":1,"last update":"Mon Mar 18 20:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61099999999999999}]},{"osd":2,"last update":"Mon Mar 18 20:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80100000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Mon Mar 18 20:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79700000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.54100000000000004}]},{"osd":4,"last update":"Mon Mar 18 20:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59899999999999998,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.621,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70999999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69999999999999996,"15min":0.69599999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93500000000000005,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.66900000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.92400000000000004,"5min":1.052,"15min":1.052},"last":0.79600000000000004}]},{"osd":7,"last update":"Mon Mar 18 20:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.89000000000000001,"5min":0.995,"15min":0.995},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58699999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.97199999999999998,"5min":1.006,"15min":1.006},"last":0.65200000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.748}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.58499999999999996}]},{"osd":10,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.77100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":465721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.59299999999999997,"15min":0.54800000000000004},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.58099999999999996}]},{"osd":2,"last update":"Mon Mar 18 20:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.745,"5min":0.754,"15min":0.754},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71099999999999997}]},{"osd":3,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.67600000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79400000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.91900000000000004}]},{"osd":4,"last update":"Mon Mar 18 20:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.443,"15min":0.40999999999999998},"min":{"1min":0.34499999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.70099999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.45000000000000001,"15min":0.42099999999999999},"min":{"1min":0.32400000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.83299999999999996}]},{"osd":6,"last update":"Mon Mar 18 20:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0229999999999999,"5min":1.196,"15min":1.196},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.434,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79200000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.79800000000000004}]},{"osd":7,"last update":"Mon Mar 18 20:34:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84399999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":1.014}]},{"osd":8,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63600000000000001,"15min":0.627},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.55300000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98899999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.97199999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.90300000000000002}]},{"osd":10,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.05}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":406729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.36699999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.55500000000000005}]},{"osd":2,"last update":"Mon Mar 18 20:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.44700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.627},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92600000000000005,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.432}]},{"osd":3,"last update":"Mon Mar 18 20:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.32700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.38300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.73799999999999999}]},{"osd":4,"last update":"Mon Mar 18 20:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.499},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89900000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Mar 18 20:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76300000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57799999999999996}]},{"osd":6,"last update":"Mon Mar 18 20:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51200000000000001,"15min":0.50900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.59899999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.88700000000000001}]},{"osd":8,"last update":"Mon Mar 18 20:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67400000000000004,"15min":0.67700000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.92200000000000004,"5min":1.002,"15min":1.002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.745,"15min":0.747},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93400000000000005,"5min":0.998,"15min":0.998},"last":0.63100000000000001}]},{"osd":9,"last update":"Mon Mar 18 20:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71399999999999997}]},{"osd":10,"last update":"Mon Mar 18 20:34:48 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.72399999999999998}]},{"osd":11,"last update":"Mon Mar 18 20:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68500000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":465224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.48699999999999999,"15min":0.46400000000000002},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.75,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.48399999999999999,"15min":0.46300000000000002},"min":{"1min":0.318,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57399999999999995}]},{"osd":1,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.56000000000000005,"15min":0.52900000000000003},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":4.6100000000000003,"5min":4.6100000000000003,"15min":4.6100000000000003},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.39100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.505}]},{"osd":3,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.54700000000000004,"15min":0.52100000000000002},"min":{"1min":0.43099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.753},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.46000000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79700000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65800000000000003}]},{"osd":4,"last update":"Mon Mar 18 20:34:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.436,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.45200000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72299999999999998}]},{"osd":5,"last update":"Mon Mar 18 20:34:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.44500000000000001,"15min":0.438},"min":{"1min":0.307,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.35099999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53000000000000003}]},{"osd":6,"last update":"Mon Mar 18 20:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.45600000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84999999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.77100000000000002,"5min":1.083,"15min":1.083},"last":0.77100000000000002}]},{"osd":7,"last update":"Mon Mar 18 20:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.439},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45800000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80800000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84699999999999998,"5min":1.038,"15min":1.038},"last":0.78400000000000003}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.85299999999999998}]},{"osd":10,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.4669999999999996,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.80000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":405263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.40899999999999997,"15min":0.40699999999999997},"min":{"1min":0.30199999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57799999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.622},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.38900000000000001}]},{"osd":1,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54700000000000004,"15min":0.52900000000000003},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.87,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41299999999999998}]},{"osd":2,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.59799999999999998},"min":{"1min":0.47799999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56699999999999995}]},{"osd":4,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90700000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67800000000000005,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.876,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.52300000000000002}]},{"osd":5,"last update":"Mon Mar 18 20:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.499,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.47399999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Mon Mar 18 20:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83999999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.496}]},{"osd":7,"last update":"Mon Mar 18 20:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.90500000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.60699999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73099999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.68300000000000005}]},{"osd":10,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.74099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986219,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":405643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.38400000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.371,"15min":0.36099999999999999},"min":{"1min":0.31,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.374,"15min":0.35999999999999999},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.63,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.40699999999999997}]},{"osd":2,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55100000000000005,"15min":0.52600000000000002},"min":{"1min":0.45600000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72199999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71299999999999997}]},{"osd":3,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.499}]},{"osd":5,"last update":"Mon Mar 18 20:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77600000000000002,"5min":0.875,"15min":0.875},"last":0.78900000000000003}]},{"osd":6,"last update":"Mon Mar 18 20:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.48199999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.63300000000000001}]},{"osd":7,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47899999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.55300000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.67500000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.80500000000000005}]},{"osd":9,"last update":"Mon Mar 18 20:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.745}]},{"osd":10,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]}]},{"osd":5,"up_from":35,"seq":150323855398,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1478656,"data_stored":997525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.52500000000000002}]},{"osd":1,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69099999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.626},{"interface":"front","average":{"1min":0.625,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.42999999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56299999999999994}]},{"osd":2,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.27900000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.38300000000000001,"15min":0.378},"min":{"1min":0.27900000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.373}]},{"osd":3,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.35499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76300000000000001,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Mon Mar 18 20:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.75800000000000001,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.68500000000000005}]},{"osd":6,"last update":"Mon Mar 18 20:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.439},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54200000000000004}]},{"osd":7,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67100000000000004}]},{"osd":8,"last update":"Mon Mar 18 20:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65400000000000003}]},{"osd":9,"last update":"Mon Mar 18 20:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64100000000000001}]},{"osd":10,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T20:35:20.612 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-18T20:35:20.613 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-18T20:35:20.613 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-18T20:35:20.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-18T20:35:20.871 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:21.145 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:20 smithi046 ceph-mon[43495]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:21.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:20 smithi073 ceph-mon[47384]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:21.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:20 smithi155 ceph-mon[48344]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.832+0000 7f76bdba1700 1 -- 172.21.15.46:0/2799361229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b8101f60 msgr2=0x7f76b810ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.832+0000 7f76bdba1700 1 --2- 172.21.15.46:0/2799361229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b8101f60 0x7f76b810ca70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f76b000b0a0 tx=0x7f76b000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.832+0000 7f76bdba1700 1 -- 172.21.15.46:0/2799361229 shutdown_connections 2024-03-18T20:35:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.832+0000 7f76bdba1700 1 --2- 172.21.15.46:0/2799361229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b8101f60 0x7f76b810ca70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.832+0000 7f76bdba1700 1 --2- 172.21.15.46:0/2799361229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b80ffe50 0x7f76b8100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.832+0000 7f76bdba1700 1 --2- 172.21.15.46:0/2799361229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b81015a0 0x7f76b8101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.832+0000 7f76bdba1700 1 -- 172.21.15.46:0/2799361229 >> 172.21.15.46:0/2799361229 conn(0x7f76b80fb5f0 msgr2=0x7f76b80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.832+0000 7f76bdba1700 1 -- 172.21.15.46:0/2799361229 shutdown_connections 2024-03-18T20:35:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.832+0000 7f76bdba1700 1 -- 172.21.15.46:0/2799361229 wait complete. 2024-03-18T20:35:21.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.833+0000 7f76bdba1700 1 Processor -- start 2024-03-18T20:35:21.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.833+0000 7f76bdba1700 1 -- start start 2024-03-18T20:35:21.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.833+0000 7f76bdba1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b80ffe50 0x7f76b8197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.833+0000 7f76bdba1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b81015a0 0x7f76b8197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.833+0000 7f76bdba1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b8101f60 0x7f76b819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.833+0000 7f76bdba1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b819c040 con 0x7f76b8101f60 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.833+0000 7f76bdba1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b819c1b0 con 0x7f76b80ffe50 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.833+0000 7f76bdba1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b819c370 con 0x7f76b81015a0 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.833+0000 7f76bd3a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b8101f60 0x7f76b819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.833+0000 7f76bd3a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b8101f60 0x7f76b819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52248/0 (socket says 172.21.15.46:52248) 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.833+0000 7f76bd3a0700 1 -- 172.21.15.46:0/1863196655 learned_addr learned my addr 172.21.15.46:0/1863196655 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76bcb9f700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b80ffe50 0x7f76b8197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76b7fff700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b81015a0 0x7f76b8197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76bd3a0700 1 -- 172.21.15.46:0/1863196655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b81015a0 msgr2=0x7f76b8197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76bd3a0700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b81015a0 0x7f76b8197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76bd3a0700 1 -- 172.21.15.46:0/1863196655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b80ffe50 msgr2=0x7f76b8197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76bd3a0700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b80ffe50 0x7f76b8197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76bd3a0700 1 -- 172.21.15.46:0/1863196655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0009d00 con 0x7f76b8101f60 2024-03-18T20:35:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76bd3a0700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b8101f60 0x7f76b819ba30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f76b000b070 tx=0x7f76b0007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76b5ffb700 1 -- 172.21.15.46:0/1863196655 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b000e040 con 0x7f76b8101f60 2024-03-18T20:35:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76bdba1700 1 -- 172.21.15.46:0/1863196655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b819c5f0 con 0x7f76b8101f60 2024-03-18T20:35:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76b5ffb700 1 -- 172.21.15.46:0/1863196655 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76b001d470 con 0x7f76b8101f60 2024-03-18T20:35:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76b5ffb700 1 -- 172.21.15.46:0/1863196655 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00052a0 con 0x7f76b8101f60 2024-03-18T20:35:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.834+0000 7f76bdba1700 1 -- 172.21.15.46:0/1863196655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b81a3eb0 con 0x7f76b8101f60 2024-03-18T20:35:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.836+0000 7f76bdba1700 1 -- 172.21.15.46:0/1863196655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b8109140 con 0x7f76b8101f60 2024-03-18T20:35:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.837+0000 7f76b5ffb700 1 -- 172.21.15.46:0/1863196655 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f76b001d870 con 0x7f76b8101f60 2024-03-18T20:35:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.837+0000 7f76b5ffb700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f76a80a1230 0x7f76a80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.837+0000 7f76bcb9f700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f76a80a1230 0x7f76a80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.837+0000 7f76b5ffb700 1 -- 172.21.15.46:0/1863196655 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f76b00cdb20 con 0x7f76b8101f60 2024-03-18T20:35:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.838+0000 7f76bcb9f700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f76a80a1230 0x7f76a80a36f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f76ac005950 tx=0x7f76ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:21.842+0000 7f76b5ffb700 1 -- 172.21.15.46:0/1863196655 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f76b0097390 con 0x7f76b8101f60 2024-03-18T20:35:22.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:21 smithi046 ceph-mon[43495]: pgmap v240: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T20:35:22.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.104+0000 7f76bdba1700 1 -- 172.21.15.46:0/1863196655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f76b8198450 con 0x7f76b8101f60 2024-03-18T20:35:22.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.104+0000 7f76b5ffb700 1 -- 172.21.15.46:0/1863196655 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f76b009a9b0 con 0x7f76b8101f60 2024-03-18T20:35:22.107 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:22.107 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-18T20:35:22.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.106+0000 7f76bdba1700 1 -- 172.21.15.46:0/1863196655 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f76a80a1230 msgr2=0x7f76a80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:22.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.106+0000 7f76bdba1700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f76a80a1230 0x7f76a80a36f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f76ac005950 tx=0x7f76ac00b410 comp rx=0 tx=0).stop 2024-03-18T20:35:22.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.106+0000 7f76bdba1700 1 -- 172.21.15.46:0/1863196655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b8101f60 msgr2=0x7f76b819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:22.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.106+0000 7f76bdba1700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b8101f60 0x7f76b819ba30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f76b000b070 tx=0x7f76b0007930 comp rx=0 tx=0).stop 2024-03-18T20:35:22.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.106+0000 7f76bdba1700 1 -- 172.21.15.46:0/1863196655 shutdown_connections 2024-03-18T20:35:22.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.106+0000 7f76bdba1700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f76a80a1230 0x7f76a80a36f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.106+0000 7f76bdba1700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76b8101f60 0x7f76b819ba30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.106+0000 7f76bdba1700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b80ffe50 0x7f76b8197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.106+0000 7f76bdba1700 1 --2- 172.21.15.46:0/1863196655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76b81015a0 0x7f76b8197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.106+0000 7f76bdba1700 1 -- 172.21.15.46:0/1863196655 >> 172.21.15.46:0/1863196655 conn(0x7f76b80fb5f0 msgr2=0x7f76b8068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:22.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.106+0000 7f76bdba1700 1 -- 172.21.15.46:0/1863196655 shutdown_connections 2024-03-18T20:35:22.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.106+0000 7f76bdba1700 1 -- 172.21.15.46:0/1863196655 wait complete. 2024-03-18T20:35:22.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:21 smithi073 ceph-mon[47384]: pgmap v240: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T20:35:22.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:21 smithi155 ceph-mon[48344]: pgmap v240: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T20:35:22.490 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-18T20:35:22.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:35:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.636+0000 7fb6931d6700 1 -- 172.21.15.46:0/346575446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68c101700 msgr2=0x7fb68c107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.636+0000 7fb6931d6700 1 --2- 172.21.15.46:0/346575446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68c101700 0x7fb68c107fb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb68400b0d0 tx=0x7fb68400b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.636+0000 7fb6931d6700 1 -- 172.21.15.46:0/346575446 shutdown_connections 2024-03-18T20:35:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.636+0000 7fb6931d6700 1 --2- 172.21.15.46:0/346575446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68c101700 0x7fb68c107fb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.636+0000 7fb6931d6700 1 --2- 172.21.15.46:0/346575446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c0ffb70 0x7fb68c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.636+0000 7fb6931d6700 1 --2- 172.21.15.46:0/346575446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb68c100d60 0x7fb68c1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.637+0000 7fb6931d6700 1 -- 172.21.15.46:0/346575446 >> 172.21.15.46:0/346575446 conn(0x7fb68c0fb100 msgr2=0x7fb68c0fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.637+0000 7fb6931d6700 1 -- 172.21.15.46:0/346575446 shutdown_connections 2024-03-18T20:35:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.637+0000 7fb6931d6700 1 -- 172.21.15.46:0/346575446 wait complete. 2024-03-18T20:35:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.637+0000 7fb6931d6700 1 Processor -- start 2024-03-18T20:35:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.637+0000 7fb6931d6700 1 -- start start 2024-03-18T20:35:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb6931d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb68c0ffb70 0x7fb68c069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb6931d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68c100d60 0x7fb68c19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb6931d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c101700 0x7fb68c19e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb6931d6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c19e850 con 0x7fb68c100d60 2024-03-18T20:35:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb6931d6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c19e9c0 con 0x7fb68c0ffb70 2024-03-18T20:35:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb6931d6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c19eb80 con 0x7fb68c101700 2024-03-18T20:35:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb68bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68c100d60 0x7fb68c19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb68bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68c100d60 0x7fb68c19ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52272/0 (socket says 172.21.15.46:52272) 2024-03-18T20:35:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb68bfff700 1 -- 172.21.15.46:0/1312566243 learned_addr learned my addr 172.21.15.46:0/1312566243 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb690f72700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb68c0ffb70 0x7fb68c069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb691773700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c101700 0x7fb68c19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb68bfff700 1 -- 172.21.15.46:0/1312566243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c101700 msgr2=0x7fb68c19e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb68bfff700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c101700 0x7fb68c19e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb68bfff700 1 -- 172.21.15.46:0/1312566243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb68c0ffb70 msgr2=0x7fb68c069dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb68bfff700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb68c0ffb70 0x7fb68c069dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb68bfff700 1 -- 172.21.15.46:0/1312566243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb684009d00 con 0x7fb68c100d60 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb691773700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c101700 0x7fb68c19e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.638+0000 7fb68bfff700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68c100d60 0x7fb68c19ba40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb67800d950 tx=0x7fb67800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.639+0000 7fb689ffb700 1 -- 172.21.15.46:0/1312566243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678009940 con 0x7fb68c100d60 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.639+0000 7fb689ffb700 1 -- 172.21.15.46:0/1312566243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb678010460 con 0x7fb68c100d60 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.639+0000 7fb6931d6700 1 -- 172.21.15.46:0/1312566243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68c19ee00 con 0x7fb68c100d60 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.639+0000 7fb689ffb700 1 -- 172.21.15.46:0/1312566243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800f620 con 0x7fb68c100d60 2024-03-18T20:35:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.639+0000 7fb6931d6700 1 -- 172.21.15.46:0/1312566243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c19f290 con 0x7fb68c100d60 2024-03-18T20:35:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.640+0000 7fb6931d6700 1 -- 172.21.15.46:0/1312566243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68c106f20 con 0x7fb68c100d60 2024-03-18T20:35:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.641+0000 7fb689ffb700 1 -- 172.21.15.46:0/1312566243 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb6780108d0 con 0x7fb68c100d60 2024-03-18T20:35:22.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.642+0000 7fb689ffb700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb67c0a1230 0x7fb67c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.642+0000 7fb689ffb700 1 -- 172.21.15.46:0/1312566243 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb6780cca10 con 0x7fb68c100d60 2024-03-18T20:35:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.642+0000 7fb690f72700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb67c0a1230 0x7fb67c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.642+0000 7fb690f72700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb67c0a1230 0x7fb67c0a36f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb68c100bc0 tx=0x7fb68000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.646+0000 7fb689ffb700 1 -- 172.21.15.46:0/1312566243 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb6780961f0 con 0x7fb68c100d60 2024-03-18T20:35:22.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.878+0000 7fb6931d6700 1 -- 172.21.15.46:0/1312566243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb68c02ccb0 con 0x7fb68c100d60 2024-03-18T20:35:22.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.880+0000 7fb689ffb700 1 -- 172.21.15.46:0/1312566243 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+22071 (secure 0 0 0) 0x7fb678020090 con 0x7fb68c100d60 2024-03-18T20:35:22.883 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:22.884 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:34:39.786955+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:35:22.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.883+0000 7fb6931d6700 1 -- 172.21.15.46:0/1312566243 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb67c0a1230 msgr2=0x7fb67c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:22.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.883+0000 7fb6931d6700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb67c0a1230 0x7fb67c0a36f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb68c100bc0 tx=0x7fb68000a380 comp rx=0 tx=0).stop 2024-03-18T20:35:22.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.883+0000 7fb6931d6700 1 -- 172.21.15.46:0/1312566243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68c100d60 msgr2=0x7fb68c19ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:22.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.883+0000 7fb6931d6700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68c100d60 0x7fb68c19ba40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb67800d950 tx=0x7fb67800dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:22.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.883+0000 7fb6931d6700 1 -- 172.21.15.46:0/1312566243 shutdown_connections 2024-03-18T20:35:22.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.883+0000 7fb6931d6700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb67c0a1230 0x7fb67c0a36f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.883+0000 7fb6931d6700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb68c101700 0x7fb68c19e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.883+0000 7fb6931d6700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb68c0ffb70 0x7fb68c069dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.883+0000 7fb6931d6700 1 --2- 172.21.15.46:0/1312566243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb68c100d60 0x7fb68c19ba40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:22.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.883+0000 7fb6931d6700 1 -- 172.21.15.46:0/1312566243 >> 172.21.15.46:0/1312566243 conn(0x7fb68c0fb100 msgr2=0x7fb68c104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:22.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.883+0000 7fb6931d6700 1 -- 172.21.15.46:0/1312566243 shutdown_connections 2024-03-18T20:35:22.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:22.883+0000 7fb6931d6700 1 -- 172.21.15.46:0/1312566243 wait complete. 2024-03-18T20:35:22.901 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T20:31:27.488318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T20:34:37.213560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T20:34:38.162692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T20:35:22.901 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T20:35:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.040+0000 7f71b4c9b700 1 -- 172.21.15.46:0/3187393326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b0100950 msgr2=0x7f71b0100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.040+0000 7f71b4c9b700 1 --2- 172.21.15.46:0/3187393326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b0100950 0x7f71b0100dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f71a800b0d0 tx=0x7f71a800b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.040+0000 7f71b4c9b700 1 -- 172.21.15.46:0/3187393326 shutdown_connections 2024-03-18T20:35:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.040+0000 7f71b4c9b700 1 --2- 172.21.15.46:0/3187393326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b0100950 0x7f71b0100dd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.040+0000 7f71b4c9b700 1 --2- 172.21.15.46:0/3187393326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b00ff630 0x7f71b00ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.040+0000 7f71b4c9b700 1 --2- 172.21.15.46:0/3187393326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b00fff90 0x7f71b0100410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.040+0000 7f71b4c9b700 1 -- 172.21.15.46:0/3187393326 >> 172.21.15.46:0/3187393326 conn(0x7f71b00fb1d0 msgr2=0x7f71b00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.040+0000 7f71b4c9b700 1 -- 172.21.15.46:0/3187393326 shutdown_connections 2024-03-18T20:35:23.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.040+0000 7f71b4c9b700 1 -- 172.21.15.46:0/3187393326 wait complete. 2024-03-18T20:35:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71b4c9b700 1 Processor -- start 2024-03-18T20:35:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71b4c9b700 1 -- start start 2024-03-18T20:35:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71b4c9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b00ff630 0x7f71b0199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71b4c9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b00fff90 0x7f71b0199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71b4c9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b0100950 0x7f71b01a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71b4c9b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b01a0b50 con 0x7f71b00ff630 2024-03-18T20:35:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71b4c9b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b01a0c90 con 0x7f71b00fff90 2024-03-18T20:35:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71b4c9b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b01a0e20 con 0x7f71b0100950 2024-03-18T20:35:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71ae59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b00ff630 0x7f71b0199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71ae59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b00ff630 0x7f71b0199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52298/0 (socket says 172.21.15.46:52298) 2024-03-18T20:35:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71ae59c700 1 -- 172.21.15.46:0/1547151914 learned_addr learned my addr 172.21.15.46:0/1547151914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71aed9d700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b0100950 0x7f71b01a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.041+0000 7f71add9b700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b00fff90 0x7f71b0199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.042+0000 7f71ae59c700 1 -- 172.21.15.46:0/1547151914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b0100950 msgr2=0x7f71b01a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.042+0000 7f71ae59c700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b0100950 0x7f71b01a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.042+0000 7f71ae59c700 1 -- 172.21.15.46:0/1547151914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b00fff90 msgr2=0x7f71b0199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.042+0000 7f71ae59c700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b00fff90 0x7f71b0199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.042+0000 7f71ae59c700 1 -- 172.21.15.46:0/1547151914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a8009d00 con 0x7f71b00ff630 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.042+0000 7f71add9b700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b00fff90 0x7f71b0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.042+0000 7f71ae59c700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b00ff630 0x7f71b0199020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f71a000d950 tx=0x7f71a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.042+0000 7f719f7fe700 1 -- 172.21.15.46:0/1547151914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0009940 con 0x7f71b00ff630 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.042+0000 7f719f7fe700 1 -- 172.21.15.46:0/1547151914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a0010460 con 0x7f71b00ff630 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.042+0000 7f71b4c9b700 1 -- 172.21.15.46:0/1547151914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b01a1100 con 0x7f71b00ff630 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.042+0000 7f719f7fe700 1 -- 172.21.15.46:0/1547151914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a000f620 con 0x7f71b00ff630 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.042+0000 7f71b4c9b700 1 -- 172.21.15.46:0/1547151914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b01a1610 con 0x7f71b00ff630 2024-03-18T20:35:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.043+0000 7f71b4c9b700 1 -- 172.21.15.46:0/1547151914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b010a670 con 0x7f71b00ff630 2024-03-18T20:35:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.044+0000 7f719f7fe700 1 -- 172.21.15.46:0/1547151914 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f71a00108d0 con 0x7f71b00ff630 2024-03-18T20:35:23.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.045+0000 7f719f7fe700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f71980a1230 0x7f71980a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:23.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.045+0000 7f719f7fe700 1 -- 172.21.15.46:0/1547151914 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f71a00cc850 con 0x7f71b00ff630 2024-03-18T20:35:23.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.045+0000 7f71add9b700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f71980a1230 0x7f71980a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:23.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.046+0000 7f71add9b700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f71980a1230 0x7f71980a36f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f71a400ac10 tx=0x7f71a4005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:23.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.049+0000 7f719f7fe700 1 -- 172.21.15.46:0/1547151914 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f71a0095f30 con 0x7f71b00ff630 2024-03-18T20:35:23.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:22 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1863196655' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T20:35:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.263+0000 7f71b4c9b700 1 -- 172.21.15.46:0/1547151914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71b002d0d0 con 0x7f71b00ff630 2024-03-18T20:35:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.263+0000 7f719f7fe700 1 -- 172.21.15.46:0/1547151914 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71a00160a0 con 0x7f71b00ff630 2024-03-18T20:35:23.267 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-18T20:35:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.266+0000 7f71b4c9b700 1 -- 172.21.15.46:0/1547151914 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f71980a1230 msgr2=0x7f71980a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.266+0000 7f71b4c9b700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f71980a1230 0x7f71980a36f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f71a400ac10 tx=0x7f71a4005f30 comp rx=0 tx=0).stop 2024-03-18T20:35:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.266+0000 7f71b4c9b700 1 -- 172.21.15.46:0/1547151914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b00ff630 msgr2=0x7f71b0199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.266+0000 7f71b4c9b700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b00ff630 0x7f71b0199020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f71a000d950 tx=0x7f71a000dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.266+0000 7f71b4c9b700 1 -- 172.21.15.46:0/1547151914 shutdown_connections 2024-03-18T20:35:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.266+0000 7f71b4c9b700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f71980a1230 0x7f71980a36f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.266+0000 7f71b4c9b700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b00ff630 0x7f71b0199020 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.266+0000 7f71b4c9b700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b0100950 0x7f71b01a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.266+0000 7f71b4c9b700 1 --2- 172.21.15.46:0/1547151914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b00fff90 0x7f71b0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.266+0000 7f71b4c9b700 1 -- 172.21.15.46:0/1547151914 >> 172.21.15.46:0/1547151914 conn(0x7f71b00fb1d0 msgr2=0x7f71b00fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.266+0000 7f71b4c9b700 1 -- 172.21.15.46:0/1547151914 shutdown_connections 2024-03-18T20:35:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.266+0000 7f71b4c9b700 1 -- 172.21.15.46:0/1547151914 wait complete. 2024-03-18T20:35:23.282 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:22 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1863196655' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T20:35:23.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T20:35:23.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:22 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1863196655' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T20:35:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.420+0000 7f4bef2e4700 1 -- 172.21.15.46:0/1726469456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4be8100e50 msgr2=0x7f4be81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.420+0000 7f4bef2e4700 1 --2- 172.21.15.46:0/1726469456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4be8100e50 0x7f4be81012b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80099c0 tx=0x7f4bd8009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.420+0000 7f4bef2e4700 1 -- 172.21.15.46:0/1726469456 shutdown_connections 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.420+0000 7f4bef2e4700 1 --2- 172.21.15.46:0/1726469456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4be81017f0 0x7f4be81080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.420+0000 7f4bef2e4700 1 --2- 172.21.15.46:0/1726469456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4be80ffc60 0x7f4be8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.420+0000 7f4bef2e4700 1 --2- 172.21.15.46:0/1726469456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4be8100e50 0x7f4be81012b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.420+0000 7f4bef2e4700 1 -- 172.21.15.46:0/1726469456 >> 172.21.15.46:0/1726469456 conn(0x7f4be80fb1d0 msgr2=0x7f4be80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.420+0000 7f4bef2e4700 1 -- 172.21.15.46:0/1726469456 shutdown_connections 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.420+0000 7f4bef2e4700 1 -- 172.21.15.46:0/1726469456 wait complete. 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.420+0000 7f4bef2e4700 1 Processor -- start 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.420+0000 7f4bef2e4700 1 -- start start 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bef2e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4be80ffc60 0x7f4be8068910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bef2e4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4be8100e50 0x7f4be819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bef2e4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4be81017f0 0x7f4be819e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bef2e4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be819e7d0 con 0x7f4be80ffc60 2024-03-18T20:35:23.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bef2e4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be819e940 con 0x7f4be81017f0 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bef2e4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be819eb00 con 0x7f4be8100e50 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bed080700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4be80ffc60 0x7f4be8068910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bed080700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4be80ffc60 0x7f4be8068910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52312/0 (socket says 172.21.15.46:52312) 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bed080700 1 -- 172.21.15.46:0/2730582342 learned_addr learned my addr 172.21.15.46:0/2730582342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bed881700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4be81017f0 0x7f4be819e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bec87f700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4be8100e50 0x7f4be819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bed080700 1 -- 172.21.15.46:0/2730582342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4be8100e50 msgr2=0x7f4be819ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bed080700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4be8100e50 0x7f4be819ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bed080700 1 -- 172.21.15.46:0/2730582342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4be81017f0 msgr2=0x7f4be819e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bed080700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4be81017f0 0x7f4be819e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bed080700 1 -- 172.21.15.46:0/2730582342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd80095f0 con 0x7f4be80ffc60 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bed080700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4be80ffc60 0x7f4be8068910 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4be400d980 tx=0x7f4be400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.421+0000 7f4bde7fc700 1 -- 172.21.15.46:0/2730582342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4004020 con 0x7f4be80ffc60 2024-03-18T20:35:23.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.422+0000 7f4bde7fc700 1 -- 172.21.15.46:0/2730582342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4be400f690 con 0x7f4be80ffc60 2024-03-18T20:35:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.422+0000 7f4bef2e4700 1 -- 172.21.15.46:0/2730582342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be819ede0 con 0x7f4be80ffc60 2024-03-18T20:35:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.422+0000 7f4bde7fc700 1 -- 172.21.15.46:0/2730582342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4003a60 con 0x7f4be80ffc60 2024-03-18T20:35:23.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.422+0000 7f4bef2e4700 1 -- 172.21.15.46:0/2730582342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be819f210 con 0x7f4be80ffc60 2024-03-18T20:35:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.422+0000 7f4bef2e4700 1 -- 172.21.15.46:0/2730582342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be8107100 con 0x7f4be80ffc60 2024-03-18T20:35:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.424+0000 7f4bde7fc700 1 -- 172.21.15.46:0/2730582342 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4be400fb00 con 0x7f4be80ffc60 2024-03-18T20:35:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.424+0000 7f4bde7fc700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4bd40a1230 0x7f4bd40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.425+0000 7f4bde7fc700 1 -- 172.21.15.46:0/2730582342 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4be40cc770 con 0x7f4be80ffc60 2024-03-18T20:35:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.425+0000 7f4bec87f700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4bd40a1230 0x7f4bd40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:23.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.425+0000 7f4bec87f700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4bd40a1230 0x7f4bd40a36f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009990 tx=0x7f4bd8005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.428+0000 7f4bde7fc700 1 -- 172.21.15.46:0/2730582342 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4be4095fd0 con 0x7f4be80ffc60 2024-03-18T20:35:23.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.645+0000 7f4bef2e4700 1 -- 172.21.15.46:0/2730582342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4be802cdc0 con 0x7f4be80ffc60 2024-03-18T20:35:23.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.645+0000 7f4bde7fc700 1 -- 172.21.15.46:0/2730582342 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4be40995f0 con 0x7f4be80ffc60 2024-03-18T20:35:23.649 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-18T20:35:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.648+0000 7f4bef2e4700 1 -- 172.21.15.46:0/2730582342 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4bd40a1230 msgr2=0x7f4bd40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.648+0000 7f4bef2e4700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4bd40a1230 0x7f4bd40a36f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009990 tx=0x7f4bd8005ad0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.648+0000 7f4bef2e4700 1 -- 172.21.15.46:0/2730582342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4be80ffc60 msgr2=0x7f4be8068910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.648+0000 7f4bef2e4700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4be80ffc60 0x7f4be8068910 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4be400d980 tx=0x7f4be400dc90 comp rx=0 tx=0).stop 2024-03-18T20:35:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.648+0000 7f4bef2e4700 1 -- 172.21.15.46:0/2730582342 shutdown_connections 2024-03-18T20:35:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.648+0000 7f4bef2e4700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4bd40a1230 0x7f4bd40a36f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.648+0000 7f4bef2e4700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4be80ffc60 0x7f4be8068910 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.648+0000 7f4bef2e4700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4be81017f0 0x7f4be819e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.648+0000 7f4bef2e4700 1 --2- 172.21.15.46:0/2730582342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4be8100e50 0x7f4be819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.648+0000 7f4bef2e4700 1 -- 172.21.15.46:0/2730582342 >> 172.21.15.46:0/2730582342 conn(0x7f4be80fb1d0 msgr2=0x7f4be8104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.648+0000 7f4bef2e4700 1 -- 172.21.15.46:0/2730582342 shutdown_connections 2024-03-18T20:35:23.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.648+0000 7f4bef2e4700 1 -- 172.21.15.46:0/2730582342 wait complete. 2024-03-18T20:35:23.665 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T20:35:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.802+0000 7f77dba84700 1 -- 172.21.15.46:0/2565020626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d41017f0 msgr2=0x7f77d41080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.802+0000 7f77dba84700 1 --2- 172.21.15.46:0/2565020626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d41017f0 0x7f77d41080a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00b0d0 tx=0x7f77cc00b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.803+0000 7f77dba84700 1 -- 172.21.15.46:0/2565020626 shutdown_connections 2024-03-18T20:35:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.803+0000 7f77dba84700 1 --2- 172.21.15.46:0/2565020626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d41017f0 0x7f77d41080a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.803+0000 7f77dba84700 1 --2- 172.21.15.46:0/2565020626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d40ffc60 0x7f77d4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.803+0000 7f77dba84700 1 --2- 172.21.15.46:0/2565020626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77d4100e50 0x7f77d41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.803+0000 7f77dba84700 1 -- 172.21.15.46:0/2565020626 >> 172.21.15.46:0/2565020626 conn(0x7f77d40fb1d0 msgr2=0x7f77d40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.803+0000 7f77dba84700 1 -- 172.21.15.46:0/2565020626 shutdown_connections 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.803+0000 7f77dba84700 1 -- 172.21.15.46:0/2565020626 wait complete. 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.803+0000 7f77dba84700 1 Processor -- start 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.803+0000 7f77dba84700 1 -- start start 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.803+0000 7f77dba84700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d40ffc60 0x7f77d4068910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.803+0000 7f77dba84700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77d4100e50 0x7f77d419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77dba84700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d41017f0 0x7f77d419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77dba84700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d419e860 con 0x7f77d41017f0 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77dba84700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d419e9d0 con 0x7f77d4100e50 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77dba84700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d419eb90 con 0x7f77d40ffc60 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77da021700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d41017f0 0x7f77d419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77d9820700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d40ffc60 0x7f77d4068910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:23.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77da021700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d41017f0 0x7f77d419e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52322/0 (socket says 172.21.15.46:52322) 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77d9820700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d40ffc60 0x7f77d4068910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58978/0 (socket says 172.21.15.46:58978) 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77da021700 1 -- 172.21.15.46:0/2221411161 learned_addr learned my addr 172.21.15.46:0/2221411161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77d901f700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77d4100e50 0x7f77d419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77da021700 1 -- 172.21.15.46:0/2221411161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d40ffc60 msgr2=0x7f77d4068910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77da021700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d40ffc60 0x7f77d4068910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77da021700 1 -- 172.21.15.46:0/2221411161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77d4100e50 msgr2=0x7f77d419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77da021700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77d4100e50 0x7f77d419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77da021700 1 -- 172.21.15.46:0/2221411161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc009d00 con 0x7f77d41017f0 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77da021700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d41017f0 0x7f77d419e0c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f77cc007bd0 tx=0x7f77cc009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77caffd700 1 -- 172.21.15.46:0/2221411161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00e040 con 0x7f77d41017f0 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77caffd700 1 -- 172.21.15.46:0/2221411161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77cc01dd50 con 0x7f77d41017f0 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77dba84700 1 -- 172.21.15.46:0/2221411161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d419ee10 con 0x7f77d41017f0 2024-03-18T20:35:23.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77caffd700 1 -- 172.21.15.46:0/2221411161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0123f0 con 0x7f77d41017f0 2024-03-18T20:35:23.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.804+0000 7f77dba84700 1 -- 172.21.15.46:0/2221411161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d419f280 con 0x7f77d41017f0 2024-03-18T20:35:23.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.805+0000 7f77dba84700 1 -- 172.21.15.46:0/2221411161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d4107010 con 0x7f77d41017f0 2024-03-18T20:35:23.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.807+0000 7f77caffd700 1 -- 172.21.15.46:0/2221411161 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f77cc01d890 con 0x7f77d41017f0 2024-03-18T20:35:23.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.808+0000 7f77caffd700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f77c00a1230 0x7f77c00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:23.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.808+0000 7f77d9820700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f77c00a1230 0x7f77c00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:23.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.808+0000 7f77caffd700 1 -- 172.21.15.46:0/2221411161 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f77cc0cedf0 con 0x7f77d41017f0 2024-03-18T20:35:23.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.808+0000 7f77d9820700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f77c00a1230 0x7f77c00a36f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f77d4100cb0 tx=0x7f77d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:23.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:23.811+0000 7f77caffd700 1 -- 172.21.15.46:0/2221411161 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f77cc098660 con 0x7f77d41017f0 2024-03-18T20:35:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.029+0000 7f77dba84700 1 -- 172.21.15.46:0/2221411161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77d402d0b0 con 0x7f77d41017f0 2024-03-18T20:35:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.029+0000 7f77caffd700 1 -- 172.21.15.46:0/2221411161 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77cc09bc80 con 0x7f77d41017f0 2024-03-18T20:35:24.033 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-18T20:35:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.031+0000 7f77dba84700 1 -- 172.21.15.46:0/2221411161 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f77c00a1230 msgr2=0x7f77c00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.031+0000 7f77dba84700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f77c00a1230 0x7f77c00a36f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f77d4100cb0 tx=0x7f77d000b410 comp rx=0 tx=0).stop 2024-03-18T20:35:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.031+0000 7f77dba84700 1 -- 172.21.15.46:0/2221411161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d41017f0 msgr2=0x7f77d419e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.031+0000 7f77dba84700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d41017f0 0x7f77d419e0c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f77cc007bd0 tx=0x7f77cc009440 comp rx=0 tx=0).stop 2024-03-18T20:35:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.031+0000 7f77dba84700 1 -- 172.21.15.46:0/2221411161 shutdown_connections 2024-03-18T20:35:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.031+0000 7f77dba84700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f77c00a1230 0x7f77c00a36f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.031+0000 7f77dba84700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77d41017f0 0x7f77d419e0c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.031+0000 7f77dba84700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d40ffc60 0x7f77d4068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.031+0000 7f77dba84700 1 --2- 172.21.15.46:0/2221411161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77d4100e50 0x7f77d419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.031+0000 7f77dba84700 1 -- 172.21.15.46:0/2221411161 >> 172.21.15.46:0/2221411161 conn(0x7f77d40fb1d0 msgr2=0x7f77d4104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.032+0000 7f77dba84700 1 -- 172.21.15.46:0/2221411161 shutdown_connections 2024-03-18T20:35:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.032+0000 7f77dba84700 1 -- 172.21.15.46:0/2221411161 wait complete. 2024-03-18T20:35:24.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:23 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1312566243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:24.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:23 smithi046 ceph-mon[43495]: pgmap v241: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-18T20:35:24.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:23 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1547151914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:24.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:23 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2730582342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:24.049 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:35:24.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.182+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1655090035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd94102270 msgr2=0x7fdd94102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:24.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.182+0000 7fdd9afcf700 1 --2- 172.21.15.46:0/1655090035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd94102270 0x7fdd94102690 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009990 tx=0x7fdd88009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.182+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1655090035 shutdown_connections 2024-03-18T20:35:24.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.182+0000 7fdd9afcf700 1 --2- 172.21.15.46:0/1655090035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd94102270 0x7fdd94102690 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.182+0000 7fdd9afcf700 1 --2- 172.21.15.46:0/1655090035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd940ff6a0 0x7fdd94109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.182+0000 7fdd9afcf700 1 --2- 172.21.15.46:0/1655090035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd940fece0 0x7fdd940ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.182+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1655090035 >> 172.21.15.46:0/1655090035 conn(0x7fdd940fa880 msgr2=0x7fdd940fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:24.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.182+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1655090035 shutdown_connections 2024-03-18T20:35:24.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.182+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1655090035 wait complete. 2024-03-18T20:35:24.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.183+0000 7fdd9afcf700 1 Processor -- start 2024-03-18T20:35:24.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.183+0000 7fdd9afcf700 1 -- start start 2024-03-18T20:35:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.183+0000 7fdd9afcf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd940fece0 0x7fdd94196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.183+0000 7fdd9afcf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd940ff6a0 0x7fdd94197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.183+0000 7fdd9afcf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd94102270 0x7fdd9419e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.183+0000 7fdd9afcf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9419e710 con 0x7fdd94102270 2024-03-18T20:35:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.183+0000 7fdd9afcf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9419e850 con 0x7fdd940ff6a0 2024-03-18T20:35:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.183+0000 7fdd9afcf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9419e9e0 con 0x7fdd940fece0 2024-03-18T20:35:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.183+0000 7fdd9956c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd94102270 0x7fdd9419e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.183+0000 7fdd9956c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd94102270 0x7fdd9419e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52332/0 (socket says 172.21.15.46:52332) 2024-03-18T20:35:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.183+0000 7fdd9956c700 1 -- 172.21.15.46:0/1244443820 learned_addr learned my addr 172.21.15.46:0/1244443820 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:24.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.183+0000 7fdd98d6b700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd940fece0 0x7fdd94196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd9956c700 1 -- 172.21.15.46:0/1244443820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd940fece0 msgr2=0x7fdd94196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd9956c700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd940fece0 0x7fdd94196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd93fff700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd940ff6a0 0x7fdd94197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd9956c700 1 -- 172.21.15.46:0/1244443820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd940ff6a0 msgr2=0x7fdd94197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd9956c700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd940ff6a0 0x7fdd94197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd9956c700 1 -- 172.21.15.46:0/1244443820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd880095f0 con 0x7fdd94102270 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd93fff700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd940ff6a0 0x7fdd94197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd9956c700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd94102270 0x7fdd9419e140 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00ef30 tx=0x7fdd8c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd91ffb700 1 -- 172.21.15.46:0/1244443820 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c018070 con 0x7fdd94102270 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd91ffb700 1 -- 172.21.15.46:0/1244443820 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd8c00f040 con 0x7fdd94102270 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1244443820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9419ec60 con 0x7fdd94102270 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd91ffb700 1 -- 172.21.15.46:0/1244443820 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c003dc0 con 0x7fdd94102270 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.184+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1244443820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9419f170 con 0x7fdd94102270 2024-03-18T20:35:24.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.185+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1244443820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd94108280 con 0x7fdd94102270 2024-03-18T20:35:24.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.186+0000 7fdd91ffb700 1 -- 172.21.15.46:0/1244443820 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fdd8c00caa0 con 0x7fdd94102270 2024-03-18T20:35:24.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.187+0000 7fdd91ffb700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdd840a1230 0x7fdd840a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:24.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.187+0000 7fdd98d6b700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdd840a1230 0x7fdd840a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:24.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.187+0000 7fdd91ffb700 1 -- 172.21.15.46:0/1244443820 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdd8c0cc600 con 0x7fdd94102270 2024-03-18T20:35:24.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.187+0000 7fdd98d6b700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdd840a1230 0x7fdd840a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800b2e0 tx=0x7fdd88005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:24.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.191+0000 7fdd91ffb700 1 -- 172.21.15.46:0/1244443820 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdd8c095d70 con 0x7fdd94102270 2024-03-18T20:35:24.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:23 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1312566243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:24.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:23 smithi073 ceph-mon[47384]: pgmap v241: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-18T20:35:24.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:23 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1547151914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:24.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:23 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2730582342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:24.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:23 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1312566243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:24.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:23 smithi155 ceph-mon[48344]: pgmap v241: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-18T20:35:24.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:23 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1547151914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:24.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:23 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2730582342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.460+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1244443820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd9402ccb0 con 0x7fdd94102270 2024-03-18T20:35:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.461+0000 7fdd91ffb700 1 -- 172.21.15.46:0/1244443820 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10909 (secure 0 0 0) 0x7fdd8c099390 con 0x7fdd94102270 2024-03-18T20:35:24.464 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:24.464 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:34:45.796438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:35:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.463+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1244443820 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdd840a1230 msgr2=0x7fdd840a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.463+0000 7fdd9afcf700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdd840a1230 0x7fdd840a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800b2e0 tx=0x7fdd88005fd0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.463+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1244443820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd94102270 msgr2=0x7fdd9419e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.463+0000 7fdd9afcf700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd94102270 0x7fdd9419e140 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00ef30 tx=0x7fdd8c00c450 comp rx=0 tx=0).stop 2024-03-18T20:35:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.463+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1244443820 shutdown_connections 2024-03-18T20:35:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.463+0000 7fdd9afcf700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fdd840a1230 0x7fdd840a36f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.463+0000 7fdd9afcf700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd94102270 0x7fdd9419e140 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.463+0000 7fdd9afcf700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd940fece0 0x7fdd94196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.463+0000 7fdd9afcf700 1 --2- 172.21.15.46:0/1244443820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd940ff6a0 0x7fdd94197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.463+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1244443820 >> 172.21.15.46:0/1244443820 conn(0x7fdd940fa880 msgr2=0x7fdd941053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.463+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1244443820 shutdown_connections 2024-03-18T20:35:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.463+0000 7fdd9afcf700 1 -- 172.21.15.46:0/1244443820 wait complete. 2024-03-18T20:35:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-03-18T20:35:24.481 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:35:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.614+0000 7f5504902700 1 -- 172.21.15.46:0/1502818080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5500100c80 msgr2=0x7f55001010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.614+0000 7f5504902700 1 --2- 172.21.15.46:0/1502818080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5500100c80 0x7f55001010a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f54f00099c0 tx=0x7f54f0009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.614+0000 7f5504902700 1 -- 172.21.15.46:0/1502818080 shutdown_connections 2024-03-18T20:35:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.614+0000 7f5504902700 1 --2- 172.21.15.46:0/1502818080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5500100c80 0x7f55001010a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.614+0000 7f5504902700 1 --2- 172.21.15.46:0/1502818080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5500102810 0x7f55001090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.614+0000 7f5504902700 1 --2- 172.21.15.46:0/1502818080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5500101e70 0x7f55001022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.614+0000 7f5504902700 1 -- 172.21.15.46:0/1502818080 >> 172.21.15.46:0/1502818080 conn(0x7f55000fc210 msgr2=0x7f55000fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.614+0000 7f5504902700 1 -- 172.21.15.46:0/1502818080 shutdown_connections 2024-03-18T20:35:24.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.615+0000 7f5504902700 1 -- 172.21.15.46:0/1502818080 wait complete. 2024-03-18T20:35:24.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.615+0000 7f5504902700 1 Processor -- start 2024-03-18T20:35:24.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.615+0000 7f5504902700 1 -- start start 2024-03-18T20:35:24.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.615+0000 7f5504902700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5500100c80 0x7f5500196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:24.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f5504902700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5500101e70 0x7f5500197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:24.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f5504902700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5500102810 0x7f550019e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f5504902700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550019e750 con 0x7f5500101e70 2024-03-18T20:35:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f5504902700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550019e890 con 0x7f5500102810 2024-03-18T20:35:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f5504902700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550019ea20 con 0x7f5500100c80 2024-03-18T20:35:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54fdd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5500101e70 0x7f5500197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54fdd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5500101e70 0x7f5500197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52348/0 (socket says 172.21.15.46:52348) 2024-03-18T20:35:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54fdd9b700 1 -- 172.21.15.46:0/2158842307 learned_addr learned my addr 172.21.15.46:0/2158842307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54fed9d700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5500102810 0x7f550019e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54fe59c700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5500100c80 0x7f5500196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54fdd9b700 1 -- 172.21.15.46:0/2158842307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5500100c80 msgr2=0x7f5500196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54fdd9b700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5500100c80 0x7f5500196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54fdd9b700 1 -- 172.21.15.46:0/2158842307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5500102810 msgr2=0x7f550019e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54fdd9b700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5500102810 0x7f550019e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54fdd9b700 1 -- 172.21.15.46:0/2158842307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f00095f0 con 0x7f5500101e70 2024-03-18T20:35:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54fdd9b700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5500101e70 0x7f5500197160 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f54f400d950 tx=0x7f54f400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54ef7fe700 1 -- 172.21.15.46:0/2158842307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4009940 con 0x7f5500101e70 2024-03-18T20:35:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f5504902700 1 -- 172.21.15.46:0/2158842307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550019ed00 con 0x7f5500101e70 2024-03-18T20:35:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54ef7fe700 1 -- 172.21.15.46:0/2158842307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54f4010460 con 0x7f5500101e70 2024-03-18T20:35:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.616+0000 7f54ef7fe700 1 -- 172.21.15.46:0/2158842307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f400f620 con 0x7f5500101e70 2024-03-18T20:35:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.617+0000 7f5504902700 1 -- 172.21.15.46:0/2158842307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550019f1c0 con 0x7f5500101e70 2024-03-18T20:35:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.617+0000 7f5504902700 1 -- 172.21.15.46:0/2158842307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5500108100 con 0x7f5500101e70 2024-03-18T20:35:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.619+0000 7f54ef7fe700 1 -- 172.21.15.46:0/2158842307 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f54f40108d0 con 0x7f5500101e70 2024-03-18T20:35:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.619+0000 7f54ef7fe700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f54e80a11b0 0x7f54e80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.620+0000 7f54fe59c700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f54e80a11b0 0x7f54e80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.620+0000 7f54ef7fe700 1 -- 172.21.15.46:0/2158842307 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f54f40cc700 con 0x7f5500101e70 2024-03-18T20:35:24.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.620+0000 7f54fe59c700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f54e80a11b0 0x7f54e80a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009990 tx=0x7f54f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:24.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.623+0000 7f54ef7fe700 1 -- 172.21.15.46:0/2158842307 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f54f4095ee0 con 0x7f5500101e70 2024-03-18T20:35:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.855+0000 7f5504902700 1 -- 172.21.15.46:0/2158842307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5500067180 con 0x7f5500101e70 2024-03-18T20:35:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.857+0000 7f54ef7fe700 1 -- 172.21.15.46:0/2158842307 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f54f4016090 con 0x7f5500101e70 2024-03-18T20:35:24.860 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:24.860 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:34:39.786955+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:35:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.859+0000 7f5504902700 1 -- 172.21.15.46:0/2158842307 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f54e80a11b0 msgr2=0x7f54e80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.859+0000 7f5504902700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f54e80a11b0 0x7f54e80a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009990 tx=0x7f54f0005fb0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.859+0000 7f5504902700 1 -- 172.21.15.46:0/2158842307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5500101e70 msgr2=0x7f5500197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.859+0000 7f5504902700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5500101e70 0x7f5500197160 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f54f400d950 tx=0x7f54f400dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.859+0000 7f5504902700 1 -- 172.21.15.46:0/2158842307 shutdown_connections 2024-03-18T20:35:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.859+0000 7f5504902700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f54e80a11b0 0x7f54e80a3670 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.859+0000 7f5504902700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5500102810 0x7f550019e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.859+0000 7f5504902700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5500100c80 0x7f5500196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.859+0000 7f5504902700 1 --2- 172.21.15.46:0/2158842307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5500101e70 0x7f5500197160 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.859+0000 7f5504902700 1 -- 172.21.15.46:0/2158842307 >> 172.21.15.46:0/2158842307 conn(0x7f55000fc210 msgr2=0x7f5500105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.859+0000 7f5504902700 1 -- 172.21.15.46:0/2158842307 shutdown_connections 2024-03-18T20:35:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:24.860+0000 7f5504902700 1 -- 172.21.15.46:0/2158842307 wait complete. 2024-03-18T20:35:24.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T20:31:27.488318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T20:34:37.213560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T20:34:38.162692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T20:35:24.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T20:35:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.015+0000 7fc970afe700 1 -- 172.21.15.46:0/1964067676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc96c1017f0 msgr2=0x7fc96c103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.015+0000 7fc970afe700 1 --2- 172.21.15.46:0/1964067676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc96c1017f0 0x7fc96c103be0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009990 tx=0x7fc95c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.015+0000 7fc970afe700 1 -- 172.21.15.46:0/1964067676 shutdown_connections 2024-03-18T20:35:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.015+0000 7fc970afe700 1 --2- 172.21.15.46:0/1964067676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc96c1017f0 0x7fc96c103be0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.015+0000 7fc970afe700 1 --2- 172.21.15.46:0/1964067676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c106a50 0x7fc96c108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.015+0000 7fc970afe700 1 --2- 172.21.15.46:0/1964067676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc96c104120 0x7fc96c106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.015+0000 7fc970afe700 1 -- 172.21.15.46:0/1964067676 >> 172.21.15.46:0/1964067676 conn(0x7fc96c0fb200 msgr2=0x7fc96c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.015+0000 7fc970afe700 1 -- 172.21.15.46:0/1964067676 shutdown_connections 2024-03-18T20:35:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.015+0000 7fc970afe700 1 -- 172.21.15.46:0/1964067676 wait complete. 2024-03-18T20:35:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc970afe700 1 Processor -- start 2024-03-18T20:35:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc970afe700 1 -- start start 2024-03-18T20:35:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc970afe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc96c1017f0 0x7fc96c101110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc970afe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc96c104120 0x7fc96c0ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc970afe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c106a50 0x7fc96c0ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc970afe700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c100480 con 0x7fc96c1017f0 2024-03-18T20:35:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc970afe700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c1005f0 con 0x7fc96c104120 2024-03-18T20:35:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc970afe700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c1a0c80 con 0x7fc96c106a50 2024-03-18T20:35:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc96a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc96c1017f0 0x7fc96c101110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc96a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc96c1017f0 0x7fc96c101110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52364/0 (socket says 172.21.15.46:52364) 2024-03-18T20:35:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc96a59c700 1 -- 172.21.15.46:0/1295358276 learned_addr learned my addr 172.21.15.46:0/1295358276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc969d9b700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc96c104120 0x7fc96c0ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc96a59c700 1 -- 172.21.15.46:0/1295358276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c106a50 msgr2=0x7fc96c0ffce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc96a59c700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c106a50 0x7fc96c0ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc96a59c700 1 -- 172.21.15.46:0/1295358276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc96c104120 msgr2=0x7fc96c0ff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc96a59c700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc96c104120 0x7fc96c0ff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc96a59c700 1 -- 172.21.15.46:0/1295358276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c0095f0 con 0x7fc96c1017f0 2024-03-18T20:35:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.016+0000 7fc969d9b700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc96c104120 0x7fc96c0ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.017+0000 7fc96a59c700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc96c1017f0 0x7fc96c101110 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc95c000c00 tx=0x7fc95c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.017+0000 7fc95b7fe700 1 -- 172.21.15.46:0/1295358276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c01d070 con 0x7fc96c1017f0 2024-03-18T20:35:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.017+0000 7fc95b7fe700 1 -- 172.21.15.46:0/1295358276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc95c004ca0 con 0x7fc96c1017f0 2024-03-18T20:35:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.017+0000 7fc970afe700 1 -- 172.21.15.46:0/1295358276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96c1a0eb0 con 0x7fc96c1017f0 2024-03-18T20:35:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.017+0000 7fc95b7fe700 1 -- 172.21.15.46:0/1295358276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c005570 con 0x7fc96c1017f0 2024-03-18T20:35:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.017+0000 7fc970afe700 1 -- 172.21.15.46:0/1295358276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c1a1220 con 0x7fc96c1017f0 2024-03-18T20:35:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.018+0000 7fc970afe700 1 -- 172.21.15.46:0/1295358276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96c0fcd80 con 0x7fc96c1017f0 2024-03-18T20:35:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.019+0000 7fc95b7fe700 1 -- 172.21.15.46:0/1295358276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc95c00fbc0 con 0x7fc96c1017f0 2024-03-18T20:35:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.020+0000 7fc95b7fe700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc9540a11b0 0x7fc9540a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.020+0000 7fc969d9b700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc9540a11b0 0x7fc9540a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.020+0000 7fc95b7fe700 1 -- 172.21.15.46:0/1295358276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc95c0cdba0 con 0x7fc96c1017f0 2024-03-18T20:35:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.020+0000 7fc969d9b700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc9540a11b0 0x7fc9540a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc960005fd0 tx=0x7fc960005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:25.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.025+0000 7fc95b7fe700 1 -- 172.21.15.46:0/1295358276 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc95c097410 con 0x7fc96c1017f0 2024-03-18T20:35:25.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:24 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2221411161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:25.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:24 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1244443820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:25.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.244+0000 7fc970afe700 1 -- 172.21.15.46:0/1295358276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc96c067f40 con 0x7fc96c1017f0 2024-03-18T20:35:25.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.244+0000 7fc95b7fe700 1 -- 172.21.15.46:0/1295358276 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc96c067f40 con 0x7fc96c1017f0 2024-03-18T20:35:25.248 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-18T20:35:25.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.247+0000 7fc970afe700 1 -- 172.21.15.46:0/1295358276 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc9540a11b0 msgr2=0x7fc9540a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.247+0000 7fc970afe700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc9540a11b0 0x7fc9540a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc960005fd0 tx=0x7fc960005e30 comp rx=0 tx=0).stop 2024-03-18T20:35:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.247+0000 7fc970afe700 1 -- 172.21.15.46:0/1295358276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc96c1017f0 msgr2=0x7fc96c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.247+0000 7fc970afe700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc96c1017f0 0x7fc96c101110 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc95c000c00 tx=0x7fc95c005ef0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.247+0000 7fc970afe700 1 -- 172.21.15.46:0/1295358276 shutdown_connections 2024-03-18T20:35:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.247+0000 7fc970afe700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc9540a11b0 0x7fc9540a3670 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.247+0000 7fc970afe700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc96c1017f0 0x7fc96c101110 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.247+0000 7fc970afe700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c106a50 0x7fc96c0ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.247+0000 7fc970afe700 1 --2- 172.21.15.46:0/1295358276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc96c104120 0x7fc96c0ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.247+0000 7fc970afe700 1 -- 172.21.15.46:0/1295358276 >> 172.21.15.46:0/1295358276 conn(0x7fc96c0fb200 msgr2=0x7fc96c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.247+0000 7fc970afe700 1 -- 172.21.15.46:0/1295358276 shutdown_connections 2024-03-18T20:35:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.247+0000 7fc970afe700 1 -- 172.21.15.46:0/1295358276 wait complete. 2024-03-18T20:35:25.263 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:24 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2221411161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:25.263 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:24 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1244443820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:25.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T20:35:25.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:24 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2221411161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:25.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:24 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1244443820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.402+0000 7f33c0db7700 1 -- 172.21.15.46:0/3709946552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33bc074d70 msgr2=0x7f33bc0731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.402+0000 7f33c0db7700 1 --2- 172.21.15.46:0/3709946552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33bc074d70 0x7f33bc0731c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009990 tx=0x7f33a4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.402+0000 7f33c0db7700 1 -- 172.21.15.46:0/3709946552 shutdown_connections 2024-03-18T20:35:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.402+0000 7f33c0db7700 1 --2- 172.21.15.46:0/3709946552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33bc074d70 0x7f33bc0731c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.402+0000 7f33c0db7700 1 --2- 172.21.15.46:0/3709946552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33bc108c70 0x7f33bc10b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.402+0000 7f33c0db7700 1 --2- 172.21.15.46:0/3709946552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33bc073790 0x7f33bc073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.402+0000 7f33c0db7700 1 -- 172.21.15.46:0/3709946552 >> 172.21.15.46:0/3709946552 conn(0x7f33bc0fc2f0 msgr2=0x7f33bc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.402+0000 7f33c0db7700 1 -- 172.21.15.46:0/3709946552 shutdown_connections 2024-03-18T20:35:25.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.402+0000 7f33c0db7700 1 -- 172.21.15.46:0/3709946552 wait complete. 2024-03-18T20:35:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.403+0000 7f33c0db7700 1 Processor -- start 2024-03-18T20:35:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.403+0000 7f33c0db7700 1 -- start start 2024-03-18T20:35:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.403+0000 7f33c0db7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33bc073790 0x7f33bc198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.403+0000 7f33c0db7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33bc074d70 0x7f33bc1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33c0db7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33bc108c70 0x7f33bc1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33c0db7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc1a09e0 con 0x7f33bc074d70 2024-03-18T20:35:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33c0db7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc1a0b20 con 0x7f33bc108c70 2024-03-18T20:35:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33c0db7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc1a0cb0 con 0x7f33bc073790 2024-03-18T20:35:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33b9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33bc074d70 0x7f33bc1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33b9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33bc074d70 0x7f33bc1993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43860/0 (socket says 172.21.15.46:43860) 2024-03-18T20:35:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33b9d9b700 1 -- 172.21.15.46:0/3989038694 learned_addr learned my addr 172.21.15.46:0/3989038694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33ba59c700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33bc073790 0x7f33bc198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33bad9d700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33bc108c70 0x7f33bc1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33b9d9b700 1 -- 172.21.15.46:0/3989038694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33bc073790 msgr2=0x7f33bc198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33b9d9b700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33bc073790 0x7f33bc198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33b9d9b700 1 -- 172.21.15.46:0/3989038694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33bc108c70 msgr2=0x7f33bc1a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33b9d9b700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33bc108c70 0x7f33bc1a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33b9d9b700 1 -- 172.21.15.46:0/3989038694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a40095f0 con 0x7f33bc074d70 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33ba59c700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33bc073790 0x7f33bc198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33bad9d700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33bc108c70 0x7f33bc1a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33b9d9b700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33bc074d70 0x7f33bc1993f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00d980 tx=0x7f33ac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33b37fe700 1 -- 172.21.15.46:0/3989038694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00bb60 con 0x7f33bc074d70 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33b37fe700 1 -- 172.21.15.46:0/3989038694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33ac007640 con 0x7f33bc074d70 2024-03-18T20:35:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33b37fe700 1 -- 172.21.15.46:0/3989038694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac004e90 con 0x7f33bc074d70 2024-03-18T20:35:25.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.404+0000 7f33c0db7700 1 -- 172.21.15.46:0/3989038694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33bc1a0f90 con 0x7f33bc074d70 2024-03-18T20:35:25.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.405+0000 7f33c0db7700 1 -- 172.21.15.46:0/3989038694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33bc1a13c0 con 0x7f33bc074d70 2024-03-18T20:35:25.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.405+0000 7f33c0db7700 1 -- 172.21.15.46:0/3989038694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33bc107a80 con 0x7f33bc074d70 2024-03-18T20:35:25.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.407+0000 7f33b37fe700 1 -- 172.21.15.46:0/3989038694 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f33ac00bcc0 con 0x7f33bc074d70 2024-03-18T20:35:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.408+0000 7f33b37fe700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f33a80a11b0 0x7f33a80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.408+0000 7f33ba59c700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f33a80a11b0 0x7f33a80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.408+0000 7f33b37fe700 1 -- 172.21.15.46:0/3989038694 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f33ac0ccd10 con 0x7f33bc074d70 2024-03-18T20:35:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.408+0000 7f33ba59c700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f33a80a11b0 0x7f33a80a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f33a400b5c0 tx=0x7f33a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:25.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.412+0000 7f33b37fe700 1 -- 172.21.15.46:0/3989038694 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f33ac099f00 con 0x7f33bc074d70 2024-03-18T20:35:25.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.625+0000 7f33c0db7700 1 -- 172.21.15.46:0/3989038694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33bc02ccb0 con 0x7f33bc074d70 2024-03-18T20:35:25.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.626+0000 7f33b37fe700 1 -- 172.21.15.46:0/3989038694 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f33ac016020 con 0x7f33bc074d70 2024-03-18T20:35:25.629 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-18T20:35:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.628+0000 7f33c0db7700 1 -- 172.21.15.46:0/3989038694 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f33a80a11b0 msgr2=0x7f33a80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.628+0000 7f33c0db7700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f33a80a11b0 0x7f33a80a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f33a400b5c0 tx=0x7f33a4005fb0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.628+0000 7f33c0db7700 1 -- 172.21.15.46:0/3989038694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33bc074d70 msgr2=0x7f33bc1993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.628+0000 7f33c0db7700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33bc074d70 0x7f33bc1993f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00d980 tx=0x7f33ac00dd40 comp rx=0 tx=0).stop 2024-03-18T20:35:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.629+0000 7f33c0db7700 1 -- 172.21.15.46:0/3989038694 shutdown_connections 2024-03-18T20:35:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.629+0000 7f33c0db7700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f33a80a11b0 0x7f33a80a3670 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.629+0000 7f33c0db7700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33bc108c70 0x7f33bc1a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.629+0000 7f33c0db7700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33bc073790 0x7f33bc198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.629+0000 7f33c0db7700 1 --2- 172.21.15.46:0/3989038694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33bc074d70 0x7f33bc1993f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.629+0000 7f33c0db7700 1 -- 172.21.15.46:0/3989038694 >> 172.21.15.46:0/3989038694 conn(0x7f33bc0fc2f0 msgr2=0x7f33bc104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:25.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.629+0000 7f33c0db7700 1 -- 172.21.15.46:0/3989038694 shutdown_connections 2024-03-18T20:35:25.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.629+0000 7f33c0db7700 1 -- 172.21.15.46:0/3989038694 wait complete. 2024-03-18T20:35:25.646 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T20:35:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.783+0000 7fb334fe3700 1 -- 172.21.15.46:0/1087519276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb330100d80 msgr2=0x7fb3301011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.783+0000 7fb334fe3700 1 --2- 172.21.15.46:0/1087519276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb330100d80 0x7fb3301011a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb3200099c0 tx=0x7fb320009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.783+0000 7fb334fe3700 1 -- 172.21.15.46:0/1087519276 shutdown_connections 2024-03-18T20:35:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.783+0000 7fb334fe3700 1 --2- 172.21.15.46:0/1087519276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb330100d80 0x7fb3301011a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.783+0000 7fb334fe3700 1 --2- 172.21.15.46:0/1087519276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb330102910 0x7fb3301091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.783+0000 7fb334fe3700 1 --2- 172.21.15.46:0/1087519276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330101f70 0x7fb3301023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.783+0000 7fb334fe3700 1 -- 172.21.15.46:0/1087519276 >> 172.21.15.46:0/1087519276 conn(0x7fb3300fc2f0 msgr2=0x7fb3300fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.783+0000 7fb334fe3700 1 -- 172.21.15.46:0/1087519276 shutdown_connections 2024-03-18T20:35:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.783+0000 7fb334fe3700 1 -- 172.21.15.46:0/1087519276 wait complete. 2024-03-18T20:35:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb334fe3700 1 Processor -- start 2024-03-18T20:35:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb334fe3700 1 -- start start 2024-03-18T20:35:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb334fe3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb330100d80 0x7fb330074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb334fe3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb330101f70 0x7fb3300731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb334fe3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330102910 0x7fb3300736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb334fe3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb330073e80 con 0x7fb330100d80 2024-03-18T20:35:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb334fe3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb330073ff0 con 0x7fb330102910 2024-03-18T20:35:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb334fe3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3301a0d60 con 0x7fb330101f70 2024-03-18T20:35:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb32e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb330100d80 0x7fb330074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb32e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb330100d80 0x7fb330074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43880/0 (socket says 172.21.15.46:43880) 2024-03-18T20:35:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb32e59c700 1 -- 172.21.15.46:0/2485202638 learned_addr learned my addr 172.21.15.46:0/2485202638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb32ed9d700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330102910 0x7fb3300736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.784+0000 7fb32dd9b700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb330101f70 0x7fb3300731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.785+0000 7fb32e59c700 1 -- 172.21.15.46:0/2485202638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb330101f70 msgr2=0x7fb3300731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.785+0000 7fb32e59c700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb330101f70 0x7fb3300731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.785+0000 7fb32e59c700 1 -- 172.21.15.46:0/2485202638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330102910 msgr2=0x7fb3300736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.785+0000 7fb32e59c700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330102910 0x7fb3300736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.785+0000 7fb32e59c700 1 -- 172.21.15.46:0/2485202638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3200095f0 con 0x7fb330100d80 2024-03-18T20:35:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.785+0000 7fb32e59c700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb330100d80 0x7fb330074b10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb320009fd0 tx=0x7fb320005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.785+0000 7fb31f7fe700 1 -- 172.21.15.46:0/2485202638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32001d070 con 0x7fb330100d80 2024-03-18T20:35:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.785+0000 7fb334fe3700 1 -- 172.21.15.46:0/2485202638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3301a0f90 con 0x7fb330100d80 2024-03-18T20:35:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.785+0000 7fb31f7fe700 1 -- 172.21.15.46:0/2485202638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3200059a0 con 0x7fb330100d80 2024-03-18T20:35:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.785+0000 7fb31f7fe700 1 -- 172.21.15.46:0/2485202638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3200173f0 con 0x7fb330100d80 2024-03-18T20:35:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.785+0000 7fb334fe3700 1 -- 172.21.15.46:0/2485202638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3301a1350 con 0x7fb330100d80 2024-03-18T20:35:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.786+0000 7fb334fe3700 1 -- 172.21.15.46:0/2485202638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb330108200 con 0x7fb330100d80 2024-03-18T20:35:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.787+0000 7fb31f7fe700 1 -- 172.21.15.46:0/2485202638 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb320022600 con 0x7fb330100d80 2024-03-18T20:35:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.788+0000 7fb31f7fe700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb3180a1230 0x7fb3180a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.788+0000 7fb32dd9b700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb3180a1230 0x7fb3180a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.788+0000 7fb31f7fe700 1 -- 172.21.15.46:0/2485202638 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb3200cdb50 con 0x7fb330100d80 2024-03-18T20:35:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.788+0000 7fb32dd9b700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb3180a1230 0x7fb3180a36f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb324009c60 tx=0x7fb324009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:25.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:25.792+0000 7fb31f7fe700 1 -- 172.21.15.46:0/2485202638 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb3200972c0 con 0x7fb330100d80 2024-03-18T20:35:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.010+0000 7fb334fe3700 1 -- 172.21.15.46:0/2485202638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb33002cdc0 con 0x7fb330100d80 2024-03-18T20:35:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.010+0000 7fb31f7fe700 1 -- 172.21.15.46:0/2485202638 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb32009a8e0 con 0x7fb330100d80 2024-03-18T20:35:26.014 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-18T20:35:26.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.012+0000 7fb334fe3700 1 -- 172.21.15.46:0/2485202638 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb3180a1230 msgr2=0x7fb3180a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:26.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.012+0000 7fb334fe3700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb3180a1230 0x7fb3180a36f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb324009c60 tx=0x7fb324009260 comp rx=0 tx=0).stop 2024-03-18T20:35:26.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.013+0000 7fb334fe3700 1 -- 172.21.15.46:0/2485202638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb330100d80 msgr2=0x7fb330074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:26.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.013+0000 7fb334fe3700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb330100d80 0x7fb330074b10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb320009fd0 tx=0x7fb320005f10 comp rx=0 tx=0).stop 2024-03-18T20:35:26.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.013+0000 7fb334fe3700 1 -- 172.21.15.46:0/2485202638 shutdown_connections 2024-03-18T20:35:26.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.013+0000 7fb334fe3700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb3180a1230 0x7fb3180a36f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.013+0000 7fb334fe3700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb330100d80 0x7fb330074b10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.013+0000 7fb334fe3700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb330102910 0x7fb3300736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.013+0000 7fb334fe3700 1 --2- 172.21.15.46:0/2485202638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb330101f70 0x7fb3300731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.013+0000 7fb334fe3700 1 -- 172.21.15.46:0/2485202638 >> 172.21.15.46:0/2485202638 conn(0x7fb3300fc2f0 msgr2=0x7fb330105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:26.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.013+0000 7fb334fe3700 1 -- 172.21.15.46:0/2485202638 shutdown_connections 2024-03-18T20:35:26.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.013+0000 7fb334fe3700 1 -- 172.21.15.46:0/2485202638 wait complete. 2024-03-18T20:35:26.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:25 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2158842307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:26.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:25 smithi046 ceph-mon[43495]: pgmap v242: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T20:35:26.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:25 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1295358276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:26.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:25 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3989038694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:26.030 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:35:26.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.164+0000 7f26ea6db700 1 -- 172.21.15.46:0/3880599561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e41016f0 msgr2=0x7f26e4107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:26.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.164+0000 7f26ea6db700 1 --2- 172.21.15.46:0/3880599561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e41016f0 0x7f26e4107fa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00b0d0 tx=0x7f26dc00b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.164+0000 7f26ea6db700 1 -- 172.21.15.46:0/3880599561 shutdown_connections 2024-03-18T20:35:26.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.164+0000 7f26ea6db700 1 --2- 172.21.15.46:0/3880599561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e41016f0 0x7f26e4107fa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.164+0000 7f26ea6db700 1 --2- 172.21.15.46:0/3880599561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26e40ffb60 0x7f26e40fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.164+0000 7f26ea6db700 1 --2- 172.21.15.46:0/3880599561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e4100d50 0x7f26e41011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.164+0000 7f26ea6db700 1 -- 172.21.15.46:0/3880599561 >> 172.21.15.46:0/3880599561 conn(0x7f26e40fb0d0 msgr2=0x7f26e40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:26.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.164+0000 7f26ea6db700 1 -- 172.21.15.46:0/3880599561 shutdown_connections 2024-03-18T20:35:26.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.164+0000 7f26ea6db700 1 -- 172.21.15.46:0/3880599561 wait complete. 2024-03-18T20:35:26.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.165+0000 7f26ea6db700 1 Processor -- start 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.165+0000 7f26ea6db700 1 -- start start 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.165+0000 7f26ea6db700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40ffb60 0x7f26e4196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.165+0000 7f26ea6db700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26e4100d50 0x7f26e4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.165+0000 7f26ea6db700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e41016f0 0x7f26e419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.165+0000 7f26ea6db700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e419e7e0 con 0x7f26e40ffb60 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.165+0000 7f26ea6db700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e419e950 con 0x7f26e41016f0 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.165+0000 7f26ea6db700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e419eb10 con 0x7f26e4100d50 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40ffb60 0x7f26e4196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40ffb60 0x7f26e4196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43904/0 (socket says 172.21.15.46:43904) 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e3fff700 1 -- 172.21.15.46:0/1142559308 learned_addr learned my addr 172.21.15.46:0/1142559308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e37fe700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26e4100d50 0x7f26e4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e8c78700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e41016f0 0x7f26e419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e3fff700 1 -- 172.21.15.46:0/1142559308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26e4100d50 msgr2=0x7f26e4197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:26.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e3fff700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26e4100d50 0x7f26e4197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e3fff700 1 -- 172.21.15.46:0/1142559308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e41016f0 msgr2=0x7f26e419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e3fff700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e41016f0 0x7f26e419e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e3fff700 1 -- 172.21.15.46:0/1142559308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d8009520 con 0x7f26e40ffb60 2024-03-18T20:35:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e8c78700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e41016f0 0x7f26e419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e3fff700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40ffb60 0x7f26e4196cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f26d400e970 tx=0x7f26d400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e17fa700 1 -- 172.21.15.46:0/1142559308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4009790 con 0x7f26e40ffb60 2024-03-18T20:35:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26ea6db700 1 -- 172.21.15.46:0/1142559308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26dc009d00 con 0x7f26e40ffb60 2024-03-18T20:35:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e17fa700 1 -- 172.21.15.46:0/1142559308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26d40044e0 con 0x7f26e40ffb60 2024-03-18T20:35:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26e17fa700 1 -- 172.21.15.46:0/1142559308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4003e30 con 0x7f26e40ffb60 2024-03-18T20:35:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.166+0000 7f26ea6db700 1 -- 172.21.15.46:0/1142559308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e419f170 con 0x7f26e40ffb60 2024-03-18T20:35:26.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.167+0000 7f26ea6db700 1 -- 172.21.15.46:0/1142559308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e4106f90 con 0x7f26e40ffb60 2024-03-18T20:35:26.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.168+0000 7f26e17fa700 1 -- 172.21.15.46:0/1142559308 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f26d4003f90 con 0x7f26e40ffb60 2024-03-18T20:35:26.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.169+0000 7f26e17fa700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f26cc0a11b0 0x7f26cc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:26.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.169+0000 7f26e37fe700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f26cc0a11b0 0x7f26cc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:26.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.169+0000 7f26e17fa700 1 -- 172.21.15.46:0/1142559308 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f26d4014070 con 0x7f26e40ffb60 2024-03-18T20:35:26.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.170+0000 7f26e37fe700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f26cc0a11b0 0x7f26cc0a3670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f26e4198170 tx=0x7f26d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:26.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.174+0000 7f26e17fa700 1 -- 172.21.15.46:0/1142559308 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f26d4096370 con 0x7f26e40ffb60 2024-03-18T20:35:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:25 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2158842307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:25 smithi073 ceph-mon[47384]: pgmap v242: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T20:35:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:25 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1295358276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:26.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:25 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3989038694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:25 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2158842307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:26.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:25 smithi155 ceph-mon[48344]: pgmap v242: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T20:35:26.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:25 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1295358276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:26.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:25 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3989038694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:26.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.448+0000 7f26ea6db700 1 -- 172.21.15.46:0/1142559308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26e4067a10 con 0x7f26e40ffb60 2024-03-18T20:35:26.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.449+0000 7f26e17fa700 1 -- 172.21.15.46:0/1142559308 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f26d4099990 con 0x7f26e40ffb60 2024-03-18T20:35:26.452 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:26.452 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:34:45.796438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:35:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.451+0000 7f26ea6db700 1 -- 172.21.15.46:0/1142559308 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f26cc0a11b0 msgr2=0x7f26cc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.451+0000 7f26ea6db700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f26cc0a11b0 0x7f26cc0a3670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f26e4198170 tx=0x7f26d8009260 comp rx=0 tx=0).stop 2024-03-18T20:35:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.451+0000 7f26ea6db700 1 -- 172.21.15.46:0/1142559308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40ffb60 msgr2=0x7f26e4196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.451+0000 7f26ea6db700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40ffb60 0x7f26e4196cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f26d400e970 tx=0x7f26d400ed30 comp rx=0 tx=0).stop 2024-03-18T20:35:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.451+0000 7f26ea6db700 1 -- 172.21.15.46:0/1142559308 shutdown_connections 2024-03-18T20:35:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.451+0000 7f26ea6db700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f26cc0a11b0 0x7f26cc0a3670 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.451+0000 7f26ea6db700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40ffb60 0x7f26e4196cc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.451+0000 7f26ea6db700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26e41016f0 0x7f26e419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.451+0000 7f26ea6db700 1 --2- 172.21.15.46:0/1142559308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26e4100d50 0x7f26e4197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.451+0000 7f26ea6db700 1 -- 172.21.15.46:0/1142559308 >> 172.21.15.46:0/1142559308 conn(0x7f26e40fb0d0 msgr2=0x7f26e41040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:26.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.452+0000 7f26ea6db700 1 -- 172.21.15.46:0/1142559308 shutdown_connections 2024-03-18T20:35:26.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.452+0000 7f26ea6db700 1 -- 172.21.15.46:0/1142559308 wait complete. 2024-03-18T20:35:26.456 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-03-18T20:35:26.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:35:26.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.602+0000 7f10d9d70700 1 -- 172.21.15.46:0/1361188638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4100c80 msgr2=0x7f10d41010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:26.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.602+0000 7f10d9d70700 1 --2- 172.21.15.46:0/1361188638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4100c80 0x7f10d41010a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0099c0 tx=0x7f10cc009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.602+0000 7f10d9d70700 1 -- 172.21.15.46:0/1361188638 shutdown_connections 2024-03-18T20:35:26.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.602+0000 7f10d9d70700 1 --2- 172.21.15.46:0/1361188638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4100c80 0x7f10d41010a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.602+0000 7f10d9d70700 1 --2- 172.21.15.46:0/1361188638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d4102810 0x7f10d41090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.602+0000 7f10d9d70700 1 --2- 172.21.15.46:0/1361188638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d4101e70 0x7f10d41022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.602+0000 7f10d9d70700 1 -- 172.21.15.46:0/1361188638 >> 172.21.15.46:0/1361188638 conn(0x7f10d40fc210 msgr2=0x7f10d40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:26.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.602+0000 7f10d9d70700 1 -- 172.21.15.46:0/1361188638 shutdown_connections 2024-03-18T20:35:26.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.602+0000 7f10d9d70700 1 -- 172.21.15.46:0/1361188638 wait complete. 2024-03-18T20:35:26.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.603+0000 7f10d9d70700 1 Processor -- start 2024-03-18T20:35:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.603+0000 7f10d9d70700 1 -- start start 2024-03-18T20:35:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.603+0000 7f10d9d70700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4100c80 0x7f10d4196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.603+0000 7f10d9d70700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d4101e70 0x7f10d4197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.603+0000 7f10d9d70700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d4102810 0x7f10d419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.603+0000 7f10d9d70700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d419e700 con 0x7f10d4100c80 2024-03-18T20:35:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.603+0000 7f10d9d70700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d419e840 con 0x7f10d4102810 2024-03-18T20:35:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.603+0000 7f10d9d70700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d419e9d0 con 0x7f10d4101e70 2024-03-18T20:35:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.603+0000 7f10d37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4100c80 0x7f10d4196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.603+0000 7f10d37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4100c80 0x7f10d4196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43924/0 (socket says 172.21.15.46:43924) 2024-03-18T20:35:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d37fe700 1 -- 172.21.15.46:0/426401771 learned_addr learned my addr 172.21.15.46:0/426401771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:26.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d3fff700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d4102810 0x7f10d419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d2ffd700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d4101e70 0x7f10d4197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d37fe700 1 -- 172.21.15.46:0/426401771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d4101e70 msgr2=0x7f10d4197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d37fe700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d4101e70 0x7f10d4197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d37fe700 1 -- 172.21.15.46:0/426401771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d4102810 msgr2=0x7f10d419e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d37fe700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d4102810 0x7f10d419e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d37fe700 1 -- 172.21.15.46:0/426401771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10cc0095f0 con 0x7f10d4100c80 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d37fe700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4100c80 0x7f10d4196bd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f10cc003820 tx=0x7f10cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d0ff9700 1 -- 172.21.15.46:0/426401771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc01d070 con 0x7f10d4100c80 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d0ff9700 1 -- 172.21.15.46:0/426401771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10cc004ca0 con 0x7f10d4100c80 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d9d70700 1 -- 172.21.15.46:0/426401771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d419ec50 con 0x7f10d4100c80 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d0ff9700 1 -- 172.21.15.46:0/426401771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc003d30 con 0x7f10d4100c80 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.604+0000 7f10d9d70700 1 -- 172.21.15.46:0/426401771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d419f150 con 0x7f10d4100c80 2024-03-18T20:35:26.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.605+0000 7f10d9d70700 1 -- 172.21.15.46:0/426401771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d4108170 con 0x7f10d4100c80 2024-03-18T20:35:26.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.606+0000 7f10d0ff9700 1 -- 172.21.15.46:0/426401771 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f10cc005030 con 0x7f10d4100c80 2024-03-18T20:35:26.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.607+0000 7f10d0ff9700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f10c00a1230 0x7f10c00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:26.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.607+0000 7f10d0ff9700 1 -- 172.21.15.46:0/426401771 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f10cc0cd980 con 0x7f10d4100c80 2024-03-18T20:35:26.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.607+0000 7f10d2ffd700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f10c00a1230 0x7f10c00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:26.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.608+0000 7f10d2ffd700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f10c00a1230 0x7f10c00a36f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10bc005950 tx=0x7f10bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:26.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.611+0000 7f10d0ff9700 1 -- 172.21.15.46:0/426401771 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f10cc0971f0 con 0x7f10d4100c80 2024-03-18T20:35:26.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.845+0000 7f10d9d70700 1 -- 172.21.15.46:0/426401771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10d4067180 con 0x7f10d4100c80 2024-03-18T20:35:26.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.847+0000 7f10d0ff9700 1 -- 172.21.15.46:0/426401771 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f10cc09a810 con 0x7f10d4100c80 2024-03-18T20:35:26.850 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:26.851 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:34:39.786955+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:35:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.849+0000 7f10d9d70700 1 -- 172.21.15.46:0/426401771 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f10c00a1230 msgr2=0x7f10c00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.849+0000 7f10d9d70700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f10c00a1230 0x7f10c00a36f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10bc005950 tx=0x7f10bc00b410 comp rx=0 tx=0).stop 2024-03-18T20:35:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.849+0000 7f10d9d70700 1 -- 172.21.15.46:0/426401771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4100c80 msgr2=0x7f10d4196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.849+0000 7f10d9d70700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4100c80 0x7f10d4196bd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f10cc003820 tx=0x7f10cc004730 comp rx=0 tx=0).stop 2024-03-18T20:35:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.849+0000 7f10d9d70700 1 -- 172.21.15.46:0/426401771 shutdown_connections 2024-03-18T20:35:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.849+0000 7f10d9d70700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f10c00a1230 0x7f10c00a36f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.849+0000 7f10d9d70700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10d4100c80 0x7f10d4196bd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.849+0000 7f10d9d70700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10d4102810 0x7f10d419e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.849+0000 7f10d9d70700 1 --2- 172.21.15.46:0/426401771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10d4101e70 0x7f10d4197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:26.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.849+0000 7f10d9d70700 1 -- 172.21.15.46:0/426401771 >> 172.21.15.46:0/426401771 conn(0x7f10d40fc210 msgr2=0x7f10d41052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:26.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.850+0000 7f10d9d70700 1 -- 172.21.15.46:0/426401771 shutdown_connections 2024-03-18T20:35:26.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:26.850+0000 7f10d9d70700 1 -- 172.21.15.46:0/426401771 wait complete. 2024-03-18T20:35:26.867 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:35:27.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.001+0000 7f8afe1f5700 1 -- 172.21.15.46:0/1771948350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af8106940 msgr2=0x7f8af8108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.001+0000 7f8afe1f5700 1 --2- 172.21.15.46:0/1771948350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af8106940 0x7f8af8108db0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8af000b0a0 tx=0x7f8af000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.001+0000 7f8afe1f5700 1 -- 172.21.15.46:0/1771948350 shutdown_connections 2024-03-18T20:35:27.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.001+0000 7f8afe1f5700 1 --2- 172.21.15.46:0/1771948350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af8106940 0x7f8af8108db0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.001+0000 7f8afe1f5700 1 --2- 172.21.15.46:0/1771948350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8af81016e0 0x7f8af8103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.001+0000 7f8afe1f5700 1 --2- 172.21.15.46:0/1771948350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af8104010 0x7f8af8106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.001+0000 7f8afe1f5700 1 -- 172.21.15.46:0/1771948350 >> 172.21.15.46:0/1771948350 conn(0x7f8af80fb0d0 msgr2=0x7f8af80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.001+0000 7f8afe1f5700 1 -- 172.21.15.46:0/1771948350 shutdown_connections 2024-03-18T20:35:27.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.001+0000 7f8afe1f5700 1 -- 172.21.15.46:0/1771948350 wait complete. 2024-03-18T20:35:27.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.002+0000 7f8afe1f5700 1 Processor -- start 2024-03-18T20:35:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.002+0000 7f8afe1f5700 1 -- start start 2024-03-18T20:35:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.002+0000 7f8afe1f5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af81016e0 0x7f8af8196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.002+0000 7f8afe1f5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af8104010 0x7f8af81971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.002+0000 7f8afe1f5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8af8106940 0x7f8af819e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.002+0000 7f8afe1f5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af819e790 con 0x7f8af8104010 2024-03-18T20:35:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.002+0000 7f8afe1f5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af819e900 con 0x7f8af8106940 2024-03-18T20:35:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.002+0000 7f8afe1f5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af819eac0 con 0x7f8af81016e0 2024-03-18T20:35:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af8104010 0x7f8af81971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af8104010 0x7f8af81971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43946/0 (socket says 172.21.15.46:43946) 2024-03-18T20:35:27.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af6ffd700 1 -- 172.21.15.46:0/722054868 learned_addr learned my addr 172.21.15.46:0/722054868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af77fe700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af81016e0 0x7f8af8196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af7fff700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8af8106940 0x7f8af819e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af6ffd700 1 -- 172.21.15.46:0/722054868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af81016e0 msgr2=0x7f8af8196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af6ffd700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af81016e0 0x7f8af8196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af6ffd700 1 -- 172.21.15.46:0/722054868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8af8106940 msgr2=0x7f8af819e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af6ffd700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8af8106940 0x7f8af819e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af6ffd700 1 -- 172.21.15.46:0/722054868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af0009d00 con 0x7f8af8104010 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af6ffd700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af8104010 0x7f8af81971e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00d950 tx=0x7f8aec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af4ff9700 1 -- 172.21.15.46:0/722054868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec00f890 con 0x7f8af8104010 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8afe1f5700 1 -- 172.21.15.46:0/722054868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af819eda0 con 0x7f8af8104010 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af4ff9700 1 -- 172.21.15.46:0/722054868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aec010460 con 0x7f8af8104010 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8af4ff9700 1 -- 172.21.15.46:0/722054868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec00e610 con 0x7f8af8104010 2024-03-18T20:35:27.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.003+0000 7f8afe1f5700 1 -- 172.21.15.46:0/722054868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af819f270 con 0x7f8af8104010 2024-03-18T20:35:27.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.004+0000 7f8afe1f5700 1 -- 172.21.15.46:0/722054868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af80fcc50 con 0x7f8af8104010 2024-03-18T20:35:27.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.006+0000 7f8af4ff9700 1 -- 172.21.15.46:0/722054868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8aec0108d0 con 0x7f8af8104010 2024-03-18T20:35:27.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.006+0000 7f8af4ff9700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8ae00a11b0 0x7f8ae00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.006+0000 7f8af4ff9700 1 -- 172.21.15.46:0/722054868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8aec0cc670 con 0x7f8af8104010 2024-03-18T20:35:27.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.006+0000 7f8af77fe700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8ae00a11b0 0x7f8ae00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.007+0000 7f8af77fe700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8ae00a11b0 0x7f8ae00a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8af8198120 tx=0x7f8ae800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.010+0000 7f8af4ff9700 1 -- 172.21.15.46:0/722054868 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8aec095e50 con 0x7f8af8104010 2024-03-18T20:35:27.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:26 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2485202638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:27.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:26 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1142559308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.275+0000 7f8afe1f5700 1 -- 172.21.15.46:0/722054868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8af8067a10 con 0x7f8af8104010 2024-03-18T20:35:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.275+0000 7f8af4ff9700 1 -- 172.21.15.46:0/722054868 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f8aec016090 con 0x7f8af8104010 2024-03-18T20:35:27.279 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:27.279 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:34:45.796438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:35:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.278+0000 7f8afe1f5700 1 -- 172.21.15.46:0/722054868 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8ae00a11b0 msgr2=0x7f8ae00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.278+0000 7f8afe1f5700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8ae00a11b0 0x7f8ae00a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8af8198120 tx=0x7f8ae800b410 comp rx=0 tx=0).stop 2024-03-18T20:35:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.278+0000 7f8afe1f5700 1 -- 172.21.15.46:0/722054868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af8104010 msgr2=0x7f8af81971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.278+0000 7f8afe1f5700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af8104010 0x7f8af81971e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00d950 tx=0x7f8aec00dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.278+0000 7f8afe1f5700 1 -- 172.21.15.46:0/722054868 shutdown_connections 2024-03-18T20:35:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.278+0000 7f8afe1f5700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f8ae00a11b0 0x7f8ae00a3670 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.278+0000 7f8afe1f5700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8af8106940 0x7f8af819e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.278+0000 7f8afe1f5700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af81016e0 0x7f8af8196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.278+0000 7f8afe1f5700 1 --2- 172.21.15.46:0/722054868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af8104010 0x7f8af81971e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.278+0000 7f8afe1f5700 1 -- 172.21.15.46:0/722054868 >> 172.21.15.46:0/722054868 conn(0x7f8af80fb0d0 msgr2=0x7f8af8101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.278+0000 7f8afe1f5700 1 -- 172.21.15.46:0/722054868 shutdown_connections 2024-03-18T20:35:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.278+0000 7f8afe1f5700 1 -- 172.21.15.46:0/722054868 wait complete. 2024-03-18T20:35:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-03-18T20:35:27.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:26 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2485202638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:27.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:26 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1142559308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:27.294 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:26 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2485202638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:27.294 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:26 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1142559308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:27.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-18T20:34:38.785094+0000', 'modified': '2024-03-18T20:34:45.796438+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24536, 'mds_1': 24521, 'mds_2': 24514, 'mds_3': 24506, 'mds_4': 24446}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.46:6841/3470614253', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6840', 'nonce': 3470614253}, {'type': 'v1', 'addr': '172.21.15.46:6841', 'nonce': 3470614253}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.73:6833/2440572005', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 2440572005}, {'type': 'v1', 'addr': '172.21.15.73:6833', 'nonce': 2440572005}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6833/4207499009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 4207499009}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 4207499009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6835/2049469125', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 2049469125}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 2049469125}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6835/2573112997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 2573112997}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 2573112997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/1686098166', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 1686098166}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 1686098166}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.73:6837/2773013996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 2773013996}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 2773013996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.155:6839/3116856560', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 3116856560}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 3116856560}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'g', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.46:6839/1267173621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6838', 'nonce': 1267173621}, {'type': 'v1', 'addr': '172.21.15.46:6839', 'nonce': 1267173621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6839/2377859422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 2377859422}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 2377859422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-18T20:35:27.296 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-18T20:35:27.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-18T20:35:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.430+0000 7fa471a8a700 1 -- 172.21.15.46:0/3609338011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa46c106aa0 msgr2=0x7fa46c108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.430+0000 7fa471a8a700 1 --2- 172.21.15.46:0/3609338011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa46c106aa0 0x7fa46c108eb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa46400b0a0 tx=0x7fa46400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.430+0000 7fa471a8a700 1 -- 172.21.15.46:0/3609338011 shutdown_connections 2024-03-18T20:35:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.430+0000 7fa471a8a700 1 --2- 172.21.15.46:0/3609338011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa46c106aa0 0x7fa46c108eb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.430+0000 7fa471a8a700 1 --2- 172.21.15.46:0/3609338011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa46c101840 0x7fa46c103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.430+0000 7fa471a8a700 1 --2- 172.21.15.46:0/3609338011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa46c104170 0x7fa46c106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.430+0000 7fa471a8a700 1 -- 172.21.15.46:0/3609338011 >> 172.21.15.46:0/3609338011 conn(0x7fa46c0fb1d0 msgr2=0x7fa46c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.430+0000 7fa471a8a700 1 -- 172.21.15.46:0/3609338011 shutdown_connections 2024-03-18T20:35:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.430+0000 7fa471a8a700 1 -- 172.21.15.46:0/3609338011 wait complete. 2024-03-18T20:35:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.431+0000 7fa471a8a700 1 Processor -- start 2024-03-18T20:35:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa471a8a700 1 -- start start 2024-03-18T20:35:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa471a8a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa46c101840 0x7fa46c196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa471a8a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa46c104170 0x7fa46c1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa471a8a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa46c106aa0 0x7fa46c19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa471a8a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c19e940 con 0x7fa46c104170 2024-03-18T20:35:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa471a8a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c19ea80 con 0x7fa46c101840 2024-03-18T20:35:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa471a8a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c19ec40 con 0x7fa46c106aa0 2024-03-18T20:35:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa46affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa46c104170 0x7fa46c1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa46affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa46c104170 0x7fa46c1973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43968/0 (socket says 172.21.15.46:43968) 2024-03-18T20:35:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa46affd700 1 -- 172.21.15.46:0/3484829582 learned_addr learned my addr 172.21.15.46:0/3484829582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa46b7fe700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa46c101840 0x7fa46c196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa46bfff700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa46c106aa0 0x7fa46c19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa46affd700 1 -- 172.21.15.46:0/3484829582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa46c106aa0 msgr2=0x7fa46c19e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa46affd700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa46c106aa0 0x7fa46c19e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa46affd700 1 -- 172.21.15.46:0/3484829582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa46c101840 msgr2=0x7fa46c196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa46affd700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa46c101840 0x7fa46c196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.432+0000 7fa46affd700 1 -- 172.21.15.46:0/3484829582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa464009d00 con 0x7fa46c104170 2024-03-18T20:35:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.433+0000 7fa46affd700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa46c104170 0x7fa46c1973c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa46000e9a0 tx=0x7fa46000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.433+0000 7fa468ff9700 1 -- 172.21.15.46:0/3484829582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460009790 con 0x7fa46c104170 2024-03-18T20:35:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.433+0000 7fa468ff9700 1 -- 172.21.15.46:0/3484829582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa46000cbd0 con 0x7fa46c104170 2024-03-18T20:35:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.433+0000 7fa471a8a700 1 -- 172.21.15.46:0/3484829582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46c19eec0 con 0x7fa46c104170 2024-03-18T20:35:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.433+0000 7fa468ff9700 1 -- 172.21.15.46:0/3484829582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460018680 con 0x7fa46c104170 2024-03-18T20:35:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.433+0000 7fa471a8a700 1 -- 172.21.15.46:0/3484829582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46c19f390 con 0x7fa46c104170 2024-03-18T20:35:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.435+0000 7fa468ff9700 1 -- 172.21.15.46:0/3484829582 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa46000cd40 con 0x7fa46c104170 2024-03-18T20:35:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.436+0000 7fa468ff9700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa4540a1230 0x7fa4540a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.436+0000 7fa468ff9700 1 -- 172.21.15.46:0/3484829582 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa460014070 con 0x7fa46c104170 2024-03-18T20:35:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.436+0000 7fa46b7fe700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa4540a1230 0x7fa4540a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.436+0000 7fa46b7fe700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa4540a1230 0x7fa4540a36f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa45c005950 tx=0x7fa45c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa47dc1fb80 1 Processor -- start 2024-03-18T20:35:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa47dc1fb80 1 -- start start 2024-03-18T20:35:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa47dc1fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d5bd0 0x5644527d5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa47dc1fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 0x5644527d6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa47dc1fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527d7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa47dc1fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445272e940 con 0x5644527d6530 2024-03-18T20:35:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa47dc1fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445272c300 con 0x5644527d5bd0 2024-03-18T20:35:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa47dc1fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564452727bf0 con 0x5644527d73d0 2024-03-18T20:35:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa46affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 0x5644527d6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa46affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 0x5644527d6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43982/0 (socket says 172.21.15.46:43982) 2024-03-18T20:35:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa46affd700 1 -- 172.21.15.46:0/613041886 learned_addr learned my addr 172.21.15.46:0/613041886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa46b7fe700 1 --2- 172.21.15.46:0/613041886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d5bd0 0x5644527d5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa46bfff700 1 --2- 172.21.15.46:0/613041886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527d7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa46affd700 1 -- 172.21.15.46:0/613041886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 msgr2=0x5644527d7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa46affd700 1 --2- 172.21.15.46:0/613041886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527d7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa46affd700 1 -- 172.21.15.46:0/613041886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d5bd0 msgr2=0x5644527d5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa46affd700 1 --2- 172.21.15.46:0/613041886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d5bd0 0x5644527d5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa46affd700 1 -- 172.21.15.46:0/613041886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644525a9360 con 0x5644527d6530 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.437+0000 7fa46bfff700 1 --2- 172.21.15.46:0/613041886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527d7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.438+0000 7fa46affd700 1 --2- 172.21.15.46:0/613041886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 0x5644527d6df0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa4600969a0 tx=0x7fa460099910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4c50ced16c6836 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.438+0000 7fa451ffb700 1 -- 172.21.15.46:0/613041886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460099f30 con 0x5644527d6530 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.438+0000 7fa451ffb700 1 -- 172.21.15.46:0/613041886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa46009dd60 con 0x5644527d6530 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.438+0000 7fa451ffb700 1 -- 172.21.15.46:0/613041886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460099640 con 0x5644527d6530 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.439+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/613041886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 msgr2=0x5644527d6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.439+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/613041886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 0x5644527d6df0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa4600969a0 tx=0x7fa460099910 comp rx=0 tx=0).stop 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.439+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/613041886 shutdown_connections 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.439+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/613041886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527d7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.439+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/613041886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d5bd0 0x5644527d5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.439+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/613041886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 0x5644527d6df0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.439+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/613041886 >> 172.21.15.46:0/613041886 conn(0x56445274b1d0 msgr2=0x5644527d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.439+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/613041886 shutdown_connections 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.439+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/613041886 wait complete. 2024-03-18T20:35:27.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.439+0000 7fa47dc1fb80 1 -- start start 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.439+0000 7fa47dc1fb80 1 Processor -- start 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa47dc1fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d5bd0 0x5644527dfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa47dc1fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d6530 0x5644527e0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa47dc1fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa47dc1fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644526c3d30 con 0x5644527d5bd0 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa47dc1fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644526c39d0 con 0x5644527d6530 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa47dc1fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644525ab650 con 0x5644527d73d0 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa47dc1fb80 10 client.0 _ll_register_callbacks cb 0x56445252f350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d5bd0 0x5644527dfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d5bd0 0x5644527dfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43988/0 (socket says 172.21.15.46:43988) 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46b7fe700 1 -- 172.21.15.46:0/25886246 learned_addr learned my addr 172.21.15.46:0/25886246 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46affd700 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d6530 0x5644527e0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46bfff700 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46b7fe700 1 -- 172.21.15.46:0/25886246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 msgr2=0x5644527e70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46b7fe700 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527e70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46b7fe700 1 -- 172.21.15.46:0/25886246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d6530 msgr2=0x5644527e0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46b7fe700 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d6530 0x5644527e0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46b7fe700 1 -- 172.21.15.46:0/25886246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46c19f390 con 0x5644527d5bd0 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46bfff700 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46affd700 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d6530 0x5644527e0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa46b7fe700 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d5bd0 0x5644527dfb30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0116f0 tx=0x7fa45c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0aecbd004d51c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa44affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa44affd700 1 -- 172.21.15.46:0/25886246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c010ee0 con 0x5644527d5bd0 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa44affd700 1 -- 172.21.15.46:0/25886246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa45c017070 con 0x5644527d5bd0 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.440+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/25886246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644527c8a80 con 0x5644527d5bd0 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.441+0000 7fa44affd700 1 -- 172.21.15.46:0/25886246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa45c028670 con 0x5644527d5bd0 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.441+0000 7fa47dc1fb80 10 client.15042 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.441+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/25886246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564452555b10 con 0x5644527d5bd0 2024-03-18T20:35:27.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.441+0000 7fa44affd700 1 -- 172.21.15.46:0/25886246 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5644527c8a80 con 0x5644527d5bd0 2024-03-18T20:35:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.441+0000 7fa44affd700 1 -- 172.21.15.46:0/25886246 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa45c01e550 con 0x5644527d5bd0 2024-03-18T20:35:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.441+0000 7fa47dc1fb80 10 client.15042 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.441+0000 7fa47dc1fb80 10 client.15042 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-18T20:35:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.441+0000 7fa47dc1fb80 20 client.15042 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.441+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5644527ead20 0x5644527f0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.441+0000 7fa47dc1fb80 4 client.15042 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:35:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.441+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/25886246 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644527206b0 con 0x5644527ead20 2024-03-18T20:35:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.442+0000 7fa46affd700 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5644527ead20 0x5644527f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.442+0000 7fa46affd700 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5644527ead20 0x5644527f0040 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4acc13878cfd4efd server_cookie=6ae9b1e2fd964a87 in_seq=0 out_seq=0 2024-03-18T20:35:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.442+0000 7fa44affd700 10 client.15042 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:35:27.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa44affd700 1 -- 172.21.15.46:0/25886246 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5644527206b0 con 0x5644527ead20 2024-03-18T20:35:27.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa44affd700 10 client.15042 handle_command_reply: tid=0 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa47dc1fb80 1 client.15042 shutdown 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/25886246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d5bd0 msgr2=0x5644527dfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d5bd0 0x5644527dfb30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0116f0 tx=0x7fa45c0047c0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/25886246 shutdown_connections 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5644527ead20 0x5644527f0040 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d5bd0 0x5644527dfb30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa44affd700 0 client.15042 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/25886246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d6530 0x5644527e0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/25886246 >> 172.21.15.46:0/25886246 conn(0x56445274b1d0 msgr2=0x5644527d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/25886246 shutdown_connections 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/25886246 wait complete. 2024-03-18T20:35:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.443+0000 7fa47dc1fb80 20 client.15042 trim_cache size 0 max 16384 2024-03-18T20:35:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa47dc1fb80 1 Processor -- start 2024-03-18T20:35:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa47dc1fb80 1 -- start start 2024-03-18T20:35:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa47dc1fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d5bd0 0x5644527d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa47dc1fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 0x5644527e7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa47dc1fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527da330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa47dc1fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445279b8f0 con 0x5644527d6530 2024-03-18T20:35:27.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa47dc1fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644527e8db0 con 0x5644527d5bd0 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa47dc1fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644527ea470 con 0x5644527d73d0 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 0x5644527e7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 0x5644527e7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43992/0 (socket says 172.21.15.46:43992) 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46affd700 1 -- 172.21.15.46:0/963794949 learned_addr learned my addr 172.21.15.46:0/963794949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46bfff700 1 --2- 172.21.15.46:0/963794949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527da330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46b7fe700 1 --2- 172.21.15.46:0/963794949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d5bd0 0x5644527d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46affd700 1 -- 172.21.15.46:0/963794949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 msgr2=0x5644527da330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46affd700 1 --2- 172.21.15.46:0/963794949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527da330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46affd700 1 -- 172.21.15.46:0/963794949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d5bd0 msgr2=0x5644527d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46affd700 1 --2- 172.21.15.46:0/963794949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d5bd0 0x5644527d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46affd700 1 -- 172.21.15.46:0/963794949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644526ede50 con 0x5644527d6530 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46bfff700 1 --2- 172.21.15.46:0/963794949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527da330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46b7fe700 1 --2- 172.21.15.46:0/963794949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d5bd0 0x5644527d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:27.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.462+0000 7fa46affd700 1 --2- 172.21.15.46:0/963794949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 0x5644527e7070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa46000e970 tx=0x7fa460096380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac001b3bc62253 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.463+0000 7fa4527fc700 1 -- 172.21.15.46:0/963794949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4600b82c0 con 0x5644527d6530 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.463+0000 7fa4527fc700 1 -- 172.21.15.46:0/963794949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa460099f30 con 0x5644527d6530 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.463+0000 7fa4527fc700 1 -- 172.21.15.46:0/963794949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460099420 con 0x5644527d6530 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.463+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/963794949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 msgr2=0x5644527e7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.463+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/963794949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 0x5644527e7070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa46000e970 tx=0x7fa460096380 comp rx=0 tx=0).stop 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.463+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/963794949 shutdown_connections 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.463+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/963794949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d73d0 0x5644527da330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.463+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/963794949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d5bd0 0x5644527d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.463+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/963794949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d6530 0x5644527e7070 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.463+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/963794949 >> 172.21.15.46:0/963794949 conn(0x56445274b1d0 msgr2=0x5644527dd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.463+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/963794949 shutdown_connections 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.463+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/963794949 wait complete. 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.464+0000 7fa47dc1fb80 1 -- start start 2024-03-18T20:35:27.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.464+0000 7fa47dc1fb80 1 Processor -- start 2024-03-18T20:35:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.464+0000 7fa47dc1fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d5bd0 0x5644527e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.464+0000 7fa47dc1fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d6530 0x5644527e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.464+0000 7fa47dc1fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d73d0 0x5644527e40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.464+0000 7fa47dc1fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644527e47b0 con 0x5644527d73d0 2024-03-18T20:35:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.464+0000 7fa47dc1fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445283f630 con 0x5644527d6530 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.464+0000 7fa47dc1fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56445283f7a0 con 0x5644527d5bd0 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.464+0000 7fa47dc1fb80 10 client.0 _ll_register_callbacks cb 0x7fa45c010d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.464+0000 7fa46bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d73d0 0x5644527e40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa46bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d73d0 0x5644527e40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44008/0 (socket says 172.21.15.46:44008) 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa46bfff700 1 -- 172.21.15.46:0/4191439697 learned_addr learned my addr 172.21.15.46:0/4191439697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa46b7fe700 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d5bd0 0x5644527e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa46affd700 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d6530 0x5644527e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa46bfff700 1 -- 172.21.15.46:0/4191439697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d5bd0 msgr2=0x5644527e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa46bfff700 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d5bd0 0x5644527e6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa46bfff700 1 -- 172.21.15.46:0/4191439697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d6530 msgr2=0x5644527e3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa46bfff700 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d6530 0x5644527e3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa46bfff700 1 -- 172.21.15.46:0/4191439697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644525a9360 con 0x5644527d73d0 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa46bfff700 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d73d0 0x5644527e40f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa46400b070 tx=0x7fa464007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e962eeb316d7a32 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa44affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa44affd700 1 -- 172.21.15.46:0/4191439697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46400e040 con 0x5644527d73d0 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa44affd700 1 -- 172.21.15.46:0/4191439697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa464004c50 con 0x5644527d73d0 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/4191439697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa46009dd60 con 0x5644527d73d0 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa44affd700 1 -- 172.21.15.46:0/4191439697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa464004e10 con 0x5644527d73d0 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa47dc1fb80 10 client.15048 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.465+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/4191439697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5644525adc80 con 0x5644527d73d0 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa44affd700 1 -- 172.21.15.46:0/4191439697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa46402e020 con 0x5644527d73d0 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa44affd700 1 -- 172.21.15.46:0/4191439697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa464019050 con 0x5644527d73d0 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa47dc1fb80 10 client.15048 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa47dc1fb80 10 client.15048 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa47dc1fb80 20 client.15048 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x564452844580 0x564452840ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa47dc1fb80 4 client.15048 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/4191439697 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564452576ca0 con 0x564452844580 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa46b7fe700 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x564452844580 0x564452840ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa44affd700 10 client.15048 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa46b7fe700 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x564452844580 0x564452840ed0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef29d80a8ba999fe server_cookie=b55e7f65c884ccb7 in_seq=0 out_seq=0 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa44affd700 1 -- 172.21.15.46:0/4191439697 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x564452576ca0 con 0x564452844580 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.466+0000 7fa44affd700 10 client.15048 handle_command_reply: tid=0 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa47dc1fb80 1 client.15048 shutdown 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/4191439697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d73d0 msgr2=0x5644527e40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d73d0 0x5644527e40f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa46400b070 tx=0x7fa464007900 comp rx=0 tx=0).stop 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/4191439697 shutdown_connections 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x564452844580 0x564452840ed0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644527d73d0 0x5644527e40f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5644527d5bd0 0x5644527e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa47dc1fb80 1 --2- 172.21.15.46:0/4191439697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644527d6530 0x5644527e3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/4191439697 >> 172.21.15.46:0/4191439697 conn(0x56445274b1d0 msgr2=0x5644527da330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa44affd700 0 client.15048 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:35:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/4191439697 shutdown_connections 2024-03-18T20:35:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa47dc1fb80 1 -- 172.21.15.46:0/4191439697 wait complete. 2024-03-18T20:35:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa47dc1fb80 20 client.15048 trim_cache size 0 max 16384 2024-03-18T20:35:27.477 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:35:27.477 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:35:27.477 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 4, 2024-03-18T20:35:27.477 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24446, 2024-03-18T20:35:27.477 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-18T20:35:27.478 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-18T20:35:27.478 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-18T20:35:27.478 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 42.683730838000002, 2024-03-18T20:35:27.478 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-03-18T20:35:27.478 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 78, 2024-03-18T20:35:27.478 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-03-18T20:35:27.478 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 72.640440079000001 2024-03-18T20:35:27.478 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:35:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa471a8a700 1 -- 172.21.15.46:0/3484829582 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa4540a1230 msgr2=0x7fa4540a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa471a8a700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa4540a1230 0x7fa4540a36f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa45c005950 tx=0x7fa45c00b410 comp rx=0 tx=0).stop 2024-03-18T20:35:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa471a8a700 1 -- 172.21.15.46:0/3484829582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa46c104170 msgr2=0x7fa46c1973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.467+0000 7fa471a8a700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa46c104170 0x7fa46c1973c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa46000e9a0 tx=0x7fa46000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.468+0000 7fa471a8a700 1 -- 172.21.15.46:0/3484829582 shutdown_connections 2024-03-18T20:35:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.468+0000 7fa471a8a700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa4540a1230 0x7fa4540a36f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.468+0000 7fa471a8a700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa46c106aa0 0x7fa46c19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.468+0000 7fa471a8a700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa46c101840 0x7fa46c196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.468+0000 7fa471a8a700 1 --2- 172.21.15.46:0/3484829582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa46c104170 0x7fa46c1973c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.468+0000 7fa471a8a700 1 -- 172.21.15.46:0/3484829582 >> 172.21.15.46:0/3484829582 conn(0x7fa46c0fb1d0 msgr2=0x7fa46c102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.468+0000 7fa471a8a700 1 -- 172.21.15.46:0/3484829582 shutdown_connections 2024-03-18T20:35:27.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.468+0000 7fa471a8a700 1 -- 172.21.15.46:0/3484829582 wait complete. 2024-03-18T20:35:27.485 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-18T20:35:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.620+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/2857105584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7d0100e50 msgr2=0x7ff7d01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.620+0000 7ff7d5c7d700 1 --2- 172.21.15.46:0/2857105584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7d0100e50 0x7ff7d01012b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40099c0 tx=0x7ff7c4009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.620+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/2857105584 shutdown_connections 2024-03-18T20:35:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.620+0000 7ff7d5c7d700 1 --2- 172.21.15.46:0/2857105584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7d01017f0 0x7ff7d01080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.620+0000 7ff7d5c7d700 1 --2- 172.21.15.46:0/2857105584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d00ffc60 0x7ff7d0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.620+0000 7ff7d5c7d700 1 --2- 172.21.15.46:0/2857105584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7d0100e50 0x7ff7d01012b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.620+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/2857105584 >> 172.21.15.46:0/2857105584 conn(0x7ff7d00fb1d0 msgr2=0x7ff7d00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.620+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/2857105584 shutdown_connections 2024-03-18T20:35:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.620+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/2857105584 wait complete. 2024-03-18T20:35:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.620+0000 7ff7d5c7d700 1 Processor -- start 2024-03-18T20:35:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7d5c7d700 1 -- start start 2024-03-18T20:35:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7d5c7d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d00ffc60 0x7ff7d0068910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7d5c7d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7d0100e50 0x7ff7d019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7d5c7d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7d01017f0 0x7ff7d01a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7d5c7d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d01a09d0 con 0x7ff7d01017f0 2024-03-18T20:35:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7d5c7d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d01a0b40 con 0x7ff7d0100e50 2024-03-18T20:35:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7d5c7d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d01a0d00 con 0x7ff7d00ffc60 2024-03-18T20:35:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7d01017f0 0x7ff7d01a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7d01017f0 0x7ff7d01a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44022/0 (socket says 172.21.15.46:44022) 2024-03-18T20:35:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7cffff700 1 -- 172.21.15.46:0/3934915746 learned_addr learned my addr 172.21.15.46:0/3934915746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7d0100e50 0x7ff7d019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d00ffc60 0x7ff7d0068910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7cffff700 1 -- 172.21.15.46:0/3934915746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d00ffc60 msgr2=0x7ff7d0068910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7cffff700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d00ffc60 0x7ff7d0068910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7cffff700 1 -- 172.21.15.46:0/3934915746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7d0100e50 msgr2=0x7ff7d019de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7cffff700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7d0100e50 0x7ff7d019de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.621+0000 7ff7cffff700 1 -- 172.21.15.46:0/3934915746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c40095f0 con 0x7ff7d01017f0 2024-03-18T20:35:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.622+0000 7ff7cffff700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7d01017f0 0x7ff7d01a0370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8007ee0 tx=0x7ff7c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.622+0000 7ff7ccff9700 1 -- 172.21.15.46:0/3934915746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8017070 con 0x7ff7d01017f0 2024-03-18T20:35:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.622+0000 7ff7ccff9700 1 -- 172.21.15.46:0/3934915746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c800f040 con 0x7ff7d01017f0 2024-03-18T20:35:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.622+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/3934915746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d01a0fe0 con 0x7ff7d01017f0 2024-03-18T20:35:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.622+0000 7ff7ccff9700 1 -- 172.21.15.46:0/3934915746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8005740 con 0x7ff7d01017f0 2024-03-18T20:35:27.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.622+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/3934915746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d01a1410 con 0x7ff7d01017f0 2024-03-18T20:35:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.624+0000 7ff7ccff9700 1 -- 172.21.15.46:0/3934915746 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff7c80058a0 con 0x7ff7d01017f0 2024-03-18T20:35:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.625+0000 7ff7ccff9700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff7b80a11b0 0x7ff7b80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.625+0000 7ff7ccff9700 1 -- 172.21.15.46:0/3934915746 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff7c80cc0d0 con 0x7ff7d01017f0 2024-03-18T20:35:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.625+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff7b80a11b0 0x7ff7b80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff7b80a11b0 0x7ff7b80a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0100cb0 tx=0x7ff7c000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7e1e12b80 1 Processor -- start 2024-03-18T20:35:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7e1e12b80 1 -- start start 2024-03-18T20:35:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7e1e12b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e334bd0 0x56349e334ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7e1e12b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e335530 0x56349e335df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7e1e12b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e336850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7e1e12b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349e28d940 con 0x56349e335530 2024-03-18T20:35:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7e1e12b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349e28b300 con 0x56349e334bd0 2024-03-18T20:35:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7e1e12b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349e286bf0 con 0x56349e3363d0 2024-03-18T20:35:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7ceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e335530 0x56349e335df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7ceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e335530 0x56349e335df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44038/0 (socket says 172.21.15.46:44038) 2024-03-18T20:35:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7ceffd700 1 -- 172.21.15.46:0/1890322719 learned_addr learned my addr 172.21.15.46:0/1890322719 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/1890322719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e334bd0 0x56349e334ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7cffff700 1 --2- 172.21.15.46:0/1890322719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e336850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.626+0000 7ff7ceffd700 1 -- 172.21.15.46:0/1890322719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 msgr2=0x56349e336850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.627+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/1890322719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e336850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.627+0000 7ff7ceffd700 1 -- 172.21.15.46:0/1890322719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e334bd0 msgr2=0x56349e334ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.627+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/1890322719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e334bd0 0x56349e334ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.627+0000 7ff7ceffd700 1 -- 172.21.15.46:0/1890322719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56349e108360 con 0x56349e335530 2024-03-18T20:35:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.627+0000 7ff7cffff700 1 --2- 172.21.15.46:0/1890322719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e336850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.627+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/1890322719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e335530 0x56349e335df0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40098e0 tx=0x7ff7c4009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58454c82fa308802 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.627+0000 7ff7b5ffb700 1 -- 172.21.15.46:0/1890322719 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4005240 con 0x56349e335530 2024-03-18T20:35:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.627+0000 7ff7b5ffb700 1 -- 172.21.15.46:0/1890322719 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c401e040 con 0x56349e335530 2024-03-18T20:35:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.627+0000 7ff7b5ffb700 1 -- 172.21.15.46:0/1890322719 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4003ac0 con 0x56349e335530 2024-03-18T20:35:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.628+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/1890322719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e335530 msgr2=0x56349e335df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.628+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/1890322719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e335530 0x56349e335df0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40098e0 tx=0x7ff7c4009f10 comp rx=0 tx=0).stop 2024-03-18T20:35:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.628+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/1890322719 shutdown_connections 2024-03-18T20:35:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.628+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/1890322719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e336850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.628+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/1890322719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e334bd0 0x56349e334ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.628+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/1890322719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e335530 0x56349e335df0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.628+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/1890322719 >> 172.21.15.46:0/1890322719 conn(0x56349e2aa1d0 msgr2=0x56349e32fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.628+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/1890322719 shutdown_connections 2024-03-18T20:35:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.628+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/1890322719 wait complete. 2024-03-18T20:35:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.628+0000 7ff7e1e12b80 1 -- start start 2024-03-18T20:35:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.628+0000 7ff7e1e12b80 1 Processor -- start 2024-03-18T20:35:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7e1e12b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e33ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7e1e12b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e33ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7e1e12b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e345fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7e1e12b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349e222d30 con 0x56349e334bd0 2024-03-18T20:35:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7e1e12b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349e2229d0 con 0x56349e335530 2024-03-18T20:35:27.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7e1e12b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349e10a650 con 0x56349e3363d0 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7e1e12b80 10 client.0 _ll_register_callbacks cb 0x56349e08e350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7cf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e33ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7cf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e33ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44050/0 (socket says 172.21.15.46:44050) 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7cf7fe700 1 -- 172.21.15.46:0/3980036683 learned_addr learned my addr 172.21.15.46:0/3980036683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e33ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7cf7fe700 1 -- 172.21.15.46:0/3980036683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 msgr2=0x56349e345fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7cffff700 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e345fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e345fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7cf7fe700 1 -- 172.21.15.46:0/3980036683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 msgr2=0x56349e33ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e33ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7cf7fe700 1 -- 172.21.15.46:0/3980036683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d01089f0 con 0x56349e334bd0 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e33ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:27.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e33ea40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000b9b0 tx=0x7ff7c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65544ff76f4f89 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.629+0000 7ff7aeffd700 1 -- 172.21.15.46:0/3980036683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0028670 con 0x56349e334bd0 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7aeffd700 1 -- 172.21.15.46:0/3980036683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff7c00287d0 con 0x56349e334bd0 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/3980036683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56349e327a80 con 0x56349e334bd0 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7aeffd700 1 -- 172.21.15.46:0/3980036683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7c0011e70 con 0x56349e334bd0 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7e1e12b80 10 client.15060 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/3980036683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56349e0b4b10 con 0x56349e334bd0 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7aeffd700 1 -- 172.21.15.46:0/3980036683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56349e327a80 con 0x56349e334bd0 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7aeffd700 1 -- 172.21.15.46:0/3980036683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff7c001e5f0 con 0x56349e334bd0 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7e1e12b80 10 client.15060 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7e1e12b80 10 client.15060 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7e1e12b80 20 client.15060 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:27.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56349e3496a0 0x56349e34ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7e1e12b80 4 client.15060 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/3980036683 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56349e27f6b0 con 0x56349e3496a0 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.630+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56349e3496a0 0x56349e34ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.631+0000 7ff7aeffd700 10 client.15060 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.631+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56349e3496a0 0x56349e34ef90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7ecdc61df40a42e server_cookie=a39525dd9b7fff83 in_seq=0 out_seq=0 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7aeffd700 1 -- 172.21.15.46:0/3980036683 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56349e27f6b0 con 0x56349e3496a0 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7aeffd700 10 client.15060 handle_command_reply: tid=0 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7e1e12b80 1 client.15060 shutdown 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/3980036683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 msgr2=0x56349e33ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e33ea40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000b9b0 tx=0x7ff7c0004750 comp rx=0 tx=0).stop 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/3980036683 shutdown_connections 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56349e3496a0 0x56349e34ef90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e33ea40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7aeffd700 0 client.15060 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:35:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e345fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/3980036683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e33ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/3980036683 >> 172.21.15.46:0/3980036683 conn(0x56349e2aa1d0 msgr2=0x56349e32f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/3980036683 shutdown_connections 2024-03-18T20:35:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/3980036683 wait complete. 2024-03-18T20:35:27.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.632+0000 7ff7e1e12b80 20 client.15060 trim_cache size 0 max 16384 2024-03-18T20:35:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7e1e12b80 1 Processor -- start 2024-03-18T20:35:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7e1e12b80 1 -- start start 2024-03-18T20:35:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7e1e12b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e352fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7e1e12b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e345f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7e1e12b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e3534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7e1e12b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349e32a1c0 con 0x56349e334bd0 2024-03-18T20:35:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7e1e12b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349e32c590 con 0x56349e335530 2024-03-18T20:35:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7e1e12b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349e335a00 con 0x56349e3363d0 2024-03-18T20:35:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7cf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e352fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7cf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e352fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44062/0 (socket says 172.21.15.46:44062) 2024-03-18T20:35:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7cf7fe700 1 -- 172.21.15.46:0/1969304944 learned_addr learned my addr 172.21.15.46:0/1969304944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/1969304944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e345f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7cffff700 1 --2- 172.21.15.46:0/1969304944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e3534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7cf7fe700 1 -- 172.21.15.46:0/1969304944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 msgr2=0x56349e3534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/1969304944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e3534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7cf7fe700 1 -- 172.21.15.46:0/1969304944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 msgr2=0x56349e345f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/1969304944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e345f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7cf7fe700 1 -- 172.21.15.46:0/1969304944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56349e24ce50 con 0x56349e334bd0 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7cffff700 1 --2- 172.21.15.46:0/1969304944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e3534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/1969304944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e345f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/1969304944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e352fb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000f770 tx=0x7ff7c000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ec2e5dd3ffd63c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.651+0000 7ff7b67fc700 1 -- 172.21.15.46:0/1969304944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c001e630 con 0x56349e334bd0 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.652+0000 7ff7b67fc700 1 -- 172.21.15.46:0/1969304944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c0011e70 con 0x56349e334bd0 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.652+0000 7ff7b67fc700 1 -- 172.21.15.46:0/1969304944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0028d60 con 0x56349e334bd0 2024-03-18T20:35:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.652+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/1969304944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 msgr2=0x56349e352fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.652+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/1969304944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e352fb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000f770 tx=0x7ff7c000b870 comp rx=0 tx=0).stop 2024-03-18T20:35:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.652+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/1969304944 shutdown_connections 2024-03-18T20:35:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.652+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/1969304944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e352fb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.652+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/1969304944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e3534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.652+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/1969304944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e345f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.652+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/1969304944 >> 172.21.15.46:0/1969304944 conn(0x56349e2aa1d0 msgr2=0x56349e33c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.652+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/1969304944 shutdown_connections 2024-03-18T20:35:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.652+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/1969304944 wait complete. 2024-03-18T20:35:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.652+0000 7ff7e1e12b80 1 -- start start 2024-03-18T20:35:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7e1e12b80 1 Processor -- start 2024-03-18T20:35:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7e1e12b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e340900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7e1e12b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e345d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7e1e12b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e340e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7e1e12b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349e341410 con 0x56349e334bd0 2024-03-18T20:35:27.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7e1e12b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349e341580 con 0x56349e335530 2024-03-18T20:35:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7e1e12b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349e39e600 con 0x56349e3363d0 2024-03-18T20:35:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7e1e12b80 10 client.0 _ll_register_callbacks cb 0x7ff7c000fb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7cf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e340900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7cf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e340900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44076/0 (socket says 172.21.15.46:44076) 2024-03-18T20:35:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7cf7fe700 1 -- 172.21.15.46:0/382821797 learned_addr learned my addr 172.21.15.46:0/382821797 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e345d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7cffff700 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e340e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7cf7fe700 1 -- 172.21.15.46:0/382821797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 msgr2=0x56349e340e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e340e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7cf7fe700 1 -- 172.21.15.46:0/382821797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 msgr2=0x56349e345d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e345d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.653+0000 7ff7cf7fe700 1 -- 172.21.15.46:0/382821797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56349e0b4b10 con 0x56349e334bd0 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7cf7fe700 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e340900 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000b9b0 tx=0x7ff7c0011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a585f6cd661bfc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7aeffd700 1 -- 172.21.15.46:0/382821797 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c001e720 con 0x56349e334bd0 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7aeffd700 1 -- 172.21.15.46:0/382821797 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff7c0005040 con 0x56349e334bd0 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/382821797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7c401e040 con 0x56349e334bd0 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7aeffd700 1 -- 172.21.15.46:0/382821797 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7c0005200 con 0x56349e334bd0 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7e1e12b80 10 client.15066 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/382821797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56349e10cc80 con 0x56349e334bd0 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7aeffd700 1 -- 172.21.15.46:0/382821797 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c401e040 con 0x56349e334bd0 2024-03-18T20:35:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7aeffd700 1 -- 172.21.15.46:0/382821797 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff7c0046440 con 0x56349e334bd0 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7e1e12b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7e1e12b80 10 client.15066 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7e1e12b80 20 client.15066 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56349e3a2e40 0x56349e39f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7e1e12b80 4 client.15066 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.654+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/382821797 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56349e0d5ca0 con 0x56349e3a2e40 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.655+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56349e3a2e40 0x56349e39f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.655+0000 7ff7aeffd700 10 client.15066 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.655+0000 7ff7ceffd700 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56349e3a2e40 0x56349e39f860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23beecc3885922fa server_cookie=beadb6ac02b3a27f in_seq=0 out_seq=0 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.655+0000 7ff7aeffd700 1 -- 172.21.15.46:0/382821797 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56349e0d5ca0 con 0x56349e3a2e40 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.655+0000 7ff7aeffd700 10 client.15066 handle_command_reply: tid=0 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.655+0000 7ff7e1e12b80 1 client.15066 shutdown 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.655+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/382821797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 msgr2=0x56349e340900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.655+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e340900 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000b9b0 tx=0x7ff7c0011be0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.655+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/382821797 shutdown_connections 2024-03-18T20:35:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.655+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56349e3a2e40 0x56349e39f860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56349e334bd0 0x56349e340900 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56349e3363d0 0x56349e340e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7e1e12b80 1 --2- 172.21.15.46:0/382821797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56349e335530 0x56349e345d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7aeffd700 0 client.15066 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:35:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/382821797 >> 172.21.15.46:0/382821797 conn(0x56349e2aa1d0 msgr2=0x56349e352fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/382821797 shutdown_connections 2024-03-18T20:35:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7e1e12b80 1 -- 172.21.15.46:0/382821797 wait complete. 2024-03-18T20:35:27.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7e1e12b80 20 client.15066 trim_cache size 0 max 16384 2024-03-18T20:35:27.674 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:35:27.674 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:35:27.674 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 3, 2024-03-18T20:35:27.674 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24506, 2024-03-18T20:35:27.674 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-18T20:35:27.674 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-18T20:35:27.674 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-18T20:35:27.674 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 43.835397145000002, 2024-03-18T20:35:27.674 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-03-18T20:35:27.674 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 78, 2024-03-18T20:35:27.674 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-03-18T20:35:27.674 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 62.363788853999999 2024-03-18T20:35:27.675 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:35:27.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/3934915746 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff7b80a11b0 msgr2=0x7ff7b80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7d5c7d700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff7b80a11b0 0x7ff7b80a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0100cb0 tx=0x7ff7c000b5b0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/3934915746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7d01017f0 msgr2=0x7ff7d01a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7d5c7d700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7d01017f0 0x7ff7d01a0370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8007ee0 tx=0x7ff7c800d3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/3934915746 shutdown_connections 2024-03-18T20:35:27.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7d5c7d700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff7b80a11b0 0x7ff7b80a3670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7d5c7d700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7d01017f0 0x7ff7d01a0370 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7d5c7d700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7d00ffc60 0x7ff7d0068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7d5c7d700 1 --2- 172.21.15.46:0/3934915746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7d0100e50 0x7ff7d019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/3934915746 >> 172.21.15.46:0/3934915746 conn(0x7ff7d00fb1d0 msgr2=0x7ff7d01041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/3934915746 shutdown_connections 2024-03-18T20:35:27.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.656+0000 7ff7d5c7d700 1 -- 172.21.15.46:0/3934915746 wait complete. 2024-03-18T20:35:27.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-18T20:35:27.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.816+0000 7f4453299700 1 -- 172.21.15.46:0/1245223326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c102910 msgr2=0x7f444c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.816+0000 7f4453299700 1 --2- 172.21.15.46:0/1245223326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c102910 0x7f444c1091c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f444400b0d0 tx=0x7f444400b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.816+0000 7f4453299700 1 -- 172.21.15.46:0/1245223326 shutdown_connections 2024-03-18T20:35:27.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.816+0000 7f4453299700 1 --2- 172.21.15.46:0/1245223326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c102910 0x7f444c1091c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.816+0000 7f4453299700 1 --2- 172.21.15.46:0/1245223326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f444c100d80 0x7f444c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.816+0000 7f4453299700 1 --2- 172.21.15.46:0/1245223326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c101f70 0x7f444c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.816+0000 7f4453299700 1 -- 172.21.15.46:0/1245223326 >> 172.21.15.46:0/1245223326 conn(0x7f444c0fc2f0 msgr2=0x7f444c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.816+0000 7f4453299700 1 -- 172.21.15.46:0/1245223326 shutdown_connections 2024-03-18T20:35:27.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.816+0000 7f4453299700 1 -- 172.21.15.46:0/1245223326 wait complete. 2024-03-18T20:35:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4453299700 1 Processor -- start 2024-03-18T20:35:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4453299700 1 -- start start 2024-03-18T20:35:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4453299700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c100d80 0x7f444c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4453299700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c101f70 0x7f444c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4453299700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f444c102910 0x7f444c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4453299700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c073e80 con 0x7f444c101f70 2024-03-18T20:35:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4453299700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c073ff0 con 0x7f444c102910 2024-03-18T20:35:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4453299700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c1a0e40 con 0x7f444c100d80 2024-03-18T20:35:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4450834700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c101f70 0x7f444c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4450834700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c101f70 0x7f444c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44082/0 (socket says 172.21.15.46:44082) 2024-03-18T20:35:27.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4450834700 1 -- 172.21.15.46:0/1865146224 learned_addr learned my addr 172.21.15.46:0/1865146224 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4451035700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c100d80 0x7f444c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4451836700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f444c102910 0x7f444c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4450834700 1 -- 172.21.15.46:0/1865146224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c100d80 msgr2=0x7f444c074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4450834700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c100d80 0x7f444c074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4450834700 1 -- 172.21.15.46:0/1865146224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f444c102910 msgr2=0x7f444c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.817+0000 7f4450834700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f444c102910 0x7f444c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.818+0000 7f4450834700 1 -- 172.21.15.46:0/1865146224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4444009d00 con 0x7f444c101f70 2024-03-18T20:35:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.818+0000 7f4451836700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f444c102910 0x7f444c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.818+0000 7f4450834700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c101f70 0x7f444c0731a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f444800e9a0 tx=0x7f444800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.818+0000 7f443e7fc700 1 -- 172.21.15.46:0/1865146224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448009790 con 0x7f444c101f70 2024-03-18T20:35:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.818+0000 7f443e7fc700 1 -- 172.21.15.46:0/1865146224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f444800cb70 con 0x7f444c101f70 2024-03-18T20:35:27.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.818+0000 7f4453299700 1 -- 172.21.15.46:0/1865146224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c1a10d0 con 0x7f444c101f70 2024-03-18T20:35:27.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.818+0000 7f443e7fc700 1 -- 172.21.15.46:0/1865146224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448010ea0 con 0x7f444c101f70 2024-03-18T20:35:27.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.818+0000 7f4453299700 1 -- 172.21.15.46:0/1865146224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c1a15e0 con 0x7f444c101f70 2024-03-18T20:35:27.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.820+0000 7f443e7fc700 1 -- 172.21.15.46:0/1865146224 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f444800cce0 con 0x7f444c101f70 2024-03-18T20:35:27.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.821+0000 7f443e7fc700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f44380a11b0 0x7f44380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.821+0000 7f4451035700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f44380a11b0 0x7f44380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.821+0000 7f443e7fc700 1 -- 172.21.15.46:0/1865146224 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4448014070 con 0x7f444c101f70 2024-03-18T20:35:27.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.821+0000 7f4451035700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f44380a11b0 0x7f44380a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f444c101dd0 tx=0x7f4440009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.822+0000 7f445f42eb80 1 Processor -- start 2024-03-18T20:35:27.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.822+0000 7f445f42eb80 1 -- start start 2024-03-18T20:35:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.822+0000 7f445f42eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22abd0 0x55775f22aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.822+0000 7f445f42eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f22bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.822+0000 7f445f42eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22c3d0 0x55775f22c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.822+0000 7f445f42eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775f183940 con 0x55775f22abd0 2024-03-18T20:35:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.822+0000 7f445f42eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775f181300 con 0x55775f22c3d0 2024-03-18T20:35:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.822+0000 7f445f42eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775f17cbf0 con 0x55775f22b530 2024-03-18T20:35:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.822+0000 7f4451035700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22abd0 0x55775f22aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.822+0000 7f4451035700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22abd0 0x55775f22aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44088/0 (socket says 172.21.15.46:44088) 2024-03-18T20:35:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.822+0000 7f4451035700 1 -- 172.21.15.46:0/2274873896 learned_addr learned my addr 172.21.15.46:0/2274873896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.823+0000 7f4450834700 1 --2- 172.21.15.46:0/2274873896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f22bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.823+0000 7f4451836700 1 --2- 172.21.15.46:0/2274873896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22c3d0 0x55775f22c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.823+0000 7f4451035700 1 -- 172.21.15.46:0/2274873896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 msgr2=0x55775f22bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.823+0000 7f4451035700 1 --2- 172.21.15.46:0/2274873896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f22bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.823+0000 7f4451035700 1 -- 172.21.15.46:0/2274873896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22c3d0 msgr2=0x55775f22c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.823+0000 7f4451035700 1 --2- 172.21.15.46:0/2274873896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22c3d0 0x55775f22c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.823+0000 7f4451035700 1 -- 172.21.15.46:0/2274873896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55775effe360 con 0x55775f22abd0 2024-03-18T20:35:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.823+0000 7f4451836700 1 --2- 172.21.15.46:0/2274873896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22c3d0 0x55775f22c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.823+0000 7f4451035700 1 --2- 172.21.15.46:0/2274873896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22abd0 0x55775f22aff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4440011d50 tx=0x7f4440014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec92c3cd7cb6875 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.823+0000 7f44377fe700 1 -- 172.21.15.46:0/2274873896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444000edd0 con 0x55775f22abd0 2024-03-18T20:35:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.823+0000 7f44377fe700 1 -- 172.21.15.46:0/2274873896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4440021070 con 0x55775f22abd0 2024-03-18T20:35:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.823+0000 7f44377fe700 1 -- 172.21.15.46:0/2274873896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444001d630 con 0x55775f22abd0 2024-03-18T20:35:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.824+0000 7f445f42eb80 1 -- 172.21.15.46:0/2274873896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22abd0 msgr2=0x55775f22aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.824+0000 7f445f42eb80 1 --2- 172.21.15.46:0/2274873896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22abd0 0x55775f22aff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4440011d50 tx=0x7f4440014000 comp rx=0 tx=0).stop 2024-03-18T20:35:27.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.824+0000 7f445f42eb80 1 -- 172.21.15.46:0/2274873896 shutdown_connections 2024-03-18T20:35:27.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.824+0000 7f445f42eb80 1 --2- 172.21.15.46:0/2274873896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22abd0 0x55775f22aff0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.824+0000 7f445f42eb80 1 --2- 172.21.15.46:0/2274873896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22c3d0 0x55775f22c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.824+0000 7f445f42eb80 1 --2- 172.21.15.46:0/2274873896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f22bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.824+0000 7f445f42eb80 1 -- 172.21.15.46:0/2274873896 >> 172.21.15.46:0/2274873896 conn(0x55775f1a01d0 msgr2=0x55775f225b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.824+0000 7f445f42eb80 1 -- 172.21.15.46:0/2274873896 shutdown_connections 2024-03-18T20:35:27.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.824+0000 7f445f42eb80 1 -- 172.21.15.46:0/2274873896 wait complete. 2024-03-18T20:35:27.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.824+0000 7f445f42eb80 1 -- start start 2024-03-18T20:35:27.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.824+0000 7f445f42eb80 1 Processor -- start 2024-03-18T20:35:27.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f445f42eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22abd0 0x55775f234a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f445f42eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22b530 0x55775f234fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f445f42eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f445f42eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775f118d30 con 0x55775f22c3d0 2024-03-18T20:35:27.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f445f42eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775f1189d0 con 0x55775f22b530 2024-03-18T20:35:27.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f445f42eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775f000650 con 0x55775f22abd0 2024-03-18T20:35:27.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f445f42eb80 10 client.0 _ll_register_callbacks cb 0x55775ef84350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:27.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f4451836700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f4451836700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44092/0 (socket says 172.21.15.46:44092) 2024-03-18T20:35:27.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f4451836700 1 -- 172.21.15.46:0/3498121733 learned_addr learned my addr 172.21.15.46:0/3498121733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f4450834700 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22b530 0x55775f234fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f4451035700 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22abd0 0x55775f234a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f4451836700 1 -- 172.21.15.46:0/3498121733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22abd0 msgr2=0x55775f234a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f4451836700 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22abd0 0x55775f234a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f4451836700 1 -- 172.21.15.46:0/3498121733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22b530 msgr2=0x55775f234fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f4451836700 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22b530 0x55775f234fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f4451836700 1 -- 172.21.15.46:0/3498121733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4440017070 con 0x55775f22c3d0 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.825+0000 7f4451035700 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22abd0 0x55775f234a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.826+0000 7f4451836700 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23bff0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f444400b0d0 tx=0x7f4444007840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c27d4d8256fc59f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.826+0000 7f4434ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.826+0000 7f4434ff9700 1 -- 172.21.15.46:0/3498121733 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444400e040 con 0x55775f22c3d0 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.826+0000 7f4434ff9700 1 -- 172.21.15.46:0/3498121733 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f444400be30 con 0x55775f22c3d0 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.826+0000 7f445f42eb80 1 -- 172.21.15.46:0/3498121733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55775f21da80 con 0x55775f22c3d0 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.826+0000 7f4434ff9700 1 -- 172.21.15.46:0/3498121733 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4444006000 con 0x55775f22c3d0 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.826+0000 7f445f42eb80 10 client.15078 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.826+0000 7f445f42eb80 1 -- 172.21.15.46:0/3498121733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55775efaab10 con 0x55775f22c3d0 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.826+0000 7f4434ff9700 1 -- 172.21.15.46:0/3498121733 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55775f21da80 con 0x55775f22c3d0 2024-03-18T20:35:27.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.826+0000 7f4434ff9700 1 -- 172.21.15.46:0/3498121733 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4444019070 con 0x55775f22c3d0 2024-03-18T20:35:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.826+0000 7f445f42eb80 10 client.15078 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.827+0000 7f445f42eb80 10 client.15078 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T20:35:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.827+0000 7f445f42eb80 20 client.15078 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.827+0000 7f445f42eb80 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55775f23f0b0 0x55775f244ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.827+0000 7f445f42eb80 4 client.15078 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:35:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.827+0000 7f445f42eb80 1 -- 172.21.15.46:0/3498121733 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55775f1756b0 con 0x55775f23f0b0 2024-03-18T20:35:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.827+0000 7f4451035700 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55775f23f0b0 0x55775f244ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.828+0000 7f4451035700 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55775f23f0b0 0x55775f244ed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae8ba32be8df6f0 server_cookie=2c15319e1684e230 in_seq=0 out_seq=0 2024-03-18T20:35:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.828+0000 7f4434ff9700 10 client.15078 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:35:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.828+0000 7f4434ff9700 1 -- 172.21.15.46:0/3498121733 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55775f1756b0 con 0x55775f23f0b0 2024-03-18T20:35:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.828+0000 7f4434ff9700 10 client.15078 handle_command_reply: tid=0 2024-03-18T20:35:27.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.828+0000 7f445f42eb80 1 client.15078 shutdown 2024-03-18T20:35:27.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.829+0000 7f445f42eb80 1 -- 172.21.15.46:0/3498121733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 msgr2=0x55775f23bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.829+0000 7f445f42eb80 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23bff0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f444400b0d0 tx=0x7f4444007840 comp rx=0 tx=0).stop 2024-03-18T20:35:27.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.829+0000 7f445f42eb80 1 -- 172.21.15.46:0/3498121733 shutdown_connections 2024-03-18T20:35:27.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.829+0000 7f445f42eb80 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55775f23f0b0 0x55775f244ed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.829+0000 7f445f42eb80 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23bff0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.829+0000 7f445f42eb80 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22abd0 0x55775f234a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.829+0000 7f445f42eb80 1 --2- 172.21.15.46:0/3498121733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22b530 0x55775f234fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.829+0000 7f445f42eb80 1 -- 172.21.15.46:0/3498121733 >> 172.21.15.46:0/3498121733 conn(0x55775f1a01d0 msgr2=0x55775f2251a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.829+0000 7f4434ff9700 0 client.15078 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:35:27.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.829+0000 7f445f42eb80 1 -- 172.21.15.46:0/3498121733 shutdown_connections 2024-03-18T20:35:27.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.829+0000 7f445f42eb80 1 -- 172.21.15.46:0/3498121733 wait complete. 2024-03-18T20:35:27.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.829+0000 7f445f42eb80 20 client.15078 trim_cache size 0 max 16384 2024-03-18T20:35:27.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f445f42eb80 1 Processor -- start 2024-03-18T20:35:27.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f445f42eb80 1 -- start start 2024-03-18T20:35:27.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f445f42eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22abd0 0x55775f233f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f445f42eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f22c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f445f42eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f445f42eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775f23c450 con 0x55775f22c3d0 2024-03-18T20:35:27.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f445f42eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775f179850 con 0x55775f22abd0 2024-03-18T20:35:27.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f445f42eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775f11bde0 con 0x55775f22b530 2024-03-18T20:35:27.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f4451836700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f4451836700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44102/0 (socket says 172.21.15.46:44102) 2024-03-18T20:35:27.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f4451836700 1 -- 172.21.15.46:0/4212160795 learned_addr learned my addr 172.21.15.46:0/4212160795 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f4451035700 1 --2- 172.21.15.46:0/4212160795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22abd0 0x55775f233f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f4450834700 1 --2- 172.21.15.46:0/4212160795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f22c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f4451836700 1 -- 172.21.15.46:0/4212160795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 msgr2=0x55775f22c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f4451836700 1 --2- 172.21.15.46:0/4212160795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f22c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f4451836700 1 -- 172.21.15.46:0/4212160795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22abd0 msgr2=0x55775f233f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f4451836700 1 --2- 172.21.15.46:0/4212160795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22abd0 0x55775f233f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f4451836700 1 -- 172.21.15.46:0/4212160795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55775f142e50 con 0x55775f22c3d0 2024-03-18T20:35:27.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f4451035700 1 --2- 172.21.15.46:0/4212160795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22abd0 0x55775f233f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:27.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.848+0000 7f4450834700 1 --2- 172.21.15.46:0/4212160795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f22c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:27.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.849+0000 7f4451836700 1 --2- 172.21.15.46:0/4212160795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23bf10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4444009fd0 tx=0x7f444400baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de1a9fdfd3a7a80 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.849+0000 7f4437fff700 1 -- 172.21.15.46:0/4212160795 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444400e020 con 0x55775f22c3d0 2024-03-18T20:35:27.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.849+0000 7f4437fff700 1 -- 172.21.15.46:0/4212160795 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4444005390 con 0x55775f22c3d0 2024-03-18T20:35:27.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.849+0000 7f4437fff700 1 -- 172.21.15.46:0/4212160795 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444005950 con 0x55775f22c3d0 2024-03-18T20:35:27.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.850+0000 7f445f42eb80 1 -- 172.21.15.46:0/4212160795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 msgr2=0x55775f23bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.850+0000 7f445f42eb80 1 --2- 172.21.15.46:0/4212160795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23bf10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4444009fd0 tx=0x7f444400baa0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.850+0000 7f445f42eb80 1 -- 172.21.15.46:0/4212160795 shutdown_connections 2024-03-18T20:35:27.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.850+0000 7f445f42eb80 1 --2- 172.21.15.46:0/4212160795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23bf10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.850+0000 7f445f42eb80 1 --2- 172.21.15.46:0/4212160795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22abd0 0x55775f233f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.850+0000 7f445f42eb80 1 --2- 172.21.15.46:0/4212160795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f22c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.850+0000 7f445f42eb80 1 -- 172.21.15.46:0/4212160795 >> 172.21.15.46:0/4212160795 conn(0x55775f1a01d0 msgr2=0x55775f2328c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.850+0000 7f445f42eb80 1 -- 172.21.15.46:0/4212160795 shutdown_connections 2024-03-18T20:35:27.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.850+0000 7f445f42eb80 1 -- 172.21.15.46:0/4212160795 wait complete. 2024-03-18T20:35:27.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.850+0000 7f445f42eb80 1 -- start start 2024-03-18T20:35:27.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.850+0000 7f445f42eb80 1 Processor -- start 2024-03-18T20:35:27.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f445f42eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22abd0 0x55775f234a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f445f42eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f234fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f445f42eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f445f42eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775f23b640 con 0x55775f22c3d0 2024-03-18T20:35:27.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f445f42eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775f23b7b0 con 0x55775f22abd0 2024-03-18T20:35:27.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f445f42eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55775f23b970 con 0x55775f22b530 2024-03-18T20:35:27.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f445f42eb80 10 client.0 _ll_register_callbacks cb 0x7f4440005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:27.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f4451836700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f4451836700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44114/0 (socket says 172.21.15.46:44114) 2024-03-18T20:35:27.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f4451836700 1 -- 172.21.15.46:0/1481108178 learned_addr learned my addr 172.21.15.46:0/1481108178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:27.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f4451035700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22abd0 0x55775f234a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f4450834700 1 --2- 172.21.15.46:0/1481108178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f234fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f4451836700 1 -- 172.21.15.46:0/1481108178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 msgr2=0x55775f234fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f4451836700 1 --2- 172.21.15.46:0/1481108178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f234fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f4451836700 1 -- 172.21.15.46:0/1481108178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22abd0 msgr2=0x55775f234a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f4451836700 1 --2- 172.21.15.46:0/1481108178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22abd0 0x55775f234a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f4451836700 1 -- 172.21.15.46:0/1481108178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55775efaab10 con 0x55775f22c3d0 2024-03-18T20:35:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f4451836700 1 --2- 172.21.15.46:0/1481108178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23af50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4444004cc0 tx=0x7f4444004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2989ba02fbfffbc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.851+0000 7f4434ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f4434ff9700 1 -- 172.21.15.46:0/1481108178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444005a90 con 0x55775f22c3d0 2024-03-18T20:35:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f4434ff9700 1 -- 172.21.15.46:0/1481108178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4444011710 con 0x55775f22c3d0 2024-03-18T20:35:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f445f42eb80 1 -- 172.21.15.46:0/1481108178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4440021070 con 0x55775f22c3d0 2024-03-18T20:35:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f4434ff9700 1 -- 172.21.15.46:0/1481108178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4444005120 con 0x55775f22c3d0 2024-03-18T20:35:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f445f42eb80 10 client.15084 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f445f42eb80 1 -- 172.21.15.46:0/1481108178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55775f002c80 con 0x55775f22c3d0 2024-03-18T20:35:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f4434ff9700 1 -- 172.21.15.46:0/1481108178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4444042020 con 0x55775f22c3d0 2024-03-18T20:35:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f4434ff9700 1 -- 172.21.15.46:0/1481108178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4444019030 con 0x55775f22c3d0 2024-03-18T20:35:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f445f42eb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f445f42eb80 10 client.15084 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T20:35:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f445f42eb80 20 client.15084 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f445f42eb80 1 --2- 172.21.15.46:0/1481108178 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55775f298370 0x55775f294d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f445f42eb80 4 client.15084 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T20:35:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f445f42eb80 1 -- 172.21.15.46:0/1481108178 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55775efcbca0 con 0x55775f298370 2024-03-18T20:35:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.852+0000 7f4451035700 1 --2- 172.21.15.46:0/1481108178 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55775f298370 0x55775f294d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f4434ff9700 10 client.15084 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:35:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f4451035700 1 --2- 172.21.15.46:0/1481108178 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55775f298370 0x55775f294d00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7f6641c2275fd92 server_cookie=dc273d63b7c55d38 in_seq=0 out_seq=0 2024-03-18T20:35:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f4434ff9700 1 -- 172.21.15.46:0/1481108178 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55775efcbca0 con 0x55775f298370 2024-03-18T20:35:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f4434ff9700 10 client.15084 handle_command_reply: tid=0 2024-03-18T20:35:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f445f42eb80 1 client.15084 shutdown 2024-03-18T20:35:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f445f42eb80 1 -- 172.21.15.46:0/1481108178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 msgr2=0x55775f23af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f445f42eb80 1 --2- 172.21.15.46:0/1481108178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23af50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4444004cc0 tx=0x7f4444004cf0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f445f42eb80 1 -- 172.21.15.46:0/1481108178 shutdown_connections 2024-03-18T20:35:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f445f42eb80 1 --2- 172.21.15.46:0/1481108178 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55775f298370 0x55775f294d00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f445f42eb80 1 --2- 172.21.15.46:0/1481108178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55775f22c3d0 0x55775f23af50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f445f42eb80 1 --2- 172.21.15.46:0/1481108178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55775f22abd0 0x55775f234a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f445f42eb80 1 --2- 172.21.15.46:0/1481108178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55775f22b530 0x55775f234fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f445f42eb80 1 -- 172.21.15.46:0/1481108178 >> 172.21.15.46:0/1481108178 conn(0x55775f1a01d0 msgr2=0x55775f225a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.853+0000 7f4434ff9700 0 client.15084 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:35:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f445f42eb80 1 -- 172.21.15.46:0/1481108178 shutdown_connections 2024-03-18T20:35:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f445f42eb80 1 -- 172.21.15.46:0/1481108178 wait complete. 2024-03-18T20:35:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f445f42eb80 20 client.15084 trim_cache size 0 max 16384 2024-03-18T20:35:27.865 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:35:27.865 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:35:27.865 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-03-18T20:35:27.865 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24514, 2024-03-18T20:35:27.866 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-18T20:35:27.866 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-18T20:35:27.866 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-18T20:35:27.866 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 45.04369784, 2024-03-18T20:35:27.866 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-03-18T20:35:27.866 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 78, 2024-03-18T20:35:27.866 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-03-18T20:35:27.866 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 60.840302293000001 2024-03-18T20:35:27.866 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:35:27.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f4453299700 1 -- 172.21.15.46:0/1865146224 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f44380a11b0 msgr2=0x7f44380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f4453299700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f44380a11b0 0x7f44380a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f444c101dd0 tx=0x7f4440009260 comp rx=0 tx=0).stop 2024-03-18T20:35:27.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f4453299700 1 -- 172.21.15.46:0/1865146224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c101f70 msgr2=0x7f444c0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f4453299700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c101f70 0x7f444c0731a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f444800e9a0 tx=0x7f444800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f4453299700 1 -- 172.21.15.46:0/1865146224 shutdown_connections 2024-03-18T20:35:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f4453299700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f44380a11b0 0x7f44380a3670 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f4453299700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f444c102910 0x7f444c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f4453299700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f444c100d80 0x7f444c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f4453299700 1 --2- 172.21.15.46:0/1865146224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f444c101f70 0x7f444c0731a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.854+0000 7f4453299700 1 -- 172.21.15.46:0/1865146224 >> 172.21.15.46:0/1865146224 conn(0x7f444c0fc2f0 msgr2=0x7f444c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.855+0000 7f4453299700 1 -- 172.21.15.46:0/1865146224 shutdown_connections 2024-03-18T20:35:27.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:27.855+0000 7f4453299700 1 -- 172.21.15.46:0/1865146224 wait complete. 2024-03-18T20:35:27.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-03-18T20:35:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.007+0000 7f3166176700 1 -- 172.21.15.46:0/3779489414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3160101840 msgr2=0x7f3160103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.007+0000 7f3166176700 1 --2- 172.21.15.46:0/3779489414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3160101840 0x7f3160103c30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f315000b0a0 tx=0x7f315000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.007+0000 7f3166176700 1 -- 172.21.15.46:0/3779489414 shutdown_connections 2024-03-18T20:35:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.007+0000 7f3166176700 1 --2- 172.21.15.46:0/3779489414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3160101840 0x7f3160103c30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.007+0000 7f3166176700 1 --2- 172.21.15.46:0/3779489414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3160106aa0 0x7f3160108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.007+0000 7f3166176700 1 --2- 172.21.15.46:0/3779489414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3160104170 0x7f3160106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.007+0000 7f3166176700 1 -- 172.21.15.46:0/3779489414 >> 172.21.15.46:0/3779489414 conn(0x7f31600fb1d0 msgr2=0x7f31600fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.007+0000 7f3166176700 1 -- 172.21.15.46:0/3779489414 shutdown_connections 2024-03-18T20:35:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.007+0000 7f3166176700 1 -- 172.21.15.46:0/3779489414 wait complete. 2024-03-18T20:35:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.008+0000 7f3166176700 1 Processor -- start 2024-03-18T20:35:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.008+0000 7f3166176700 1 -- start start 2024-03-18T20:35:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.008+0000 7f3166176700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3160101840 0x7f3160196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.008+0000 7f3166176700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3160104170 0x7f3160197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.008+0000 7f3166176700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3160106aa0 0x7f316019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.008+0000 7f3166176700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316019e900 con 0x7f3160106aa0 2024-03-18T20:35:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.008+0000 7f3166176700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316019ea70 con 0x7f3160101840 2024-03-18T20:35:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.008+0000 7f3166176700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316019ec30 con 0x7f3160104170 2024-03-18T20:35:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3160106aa0 0x7f316019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3160106aa0 0x7f316019e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44128/0 (socket says 172.21.15.46:44128) 2024-03-18T20:35:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315ffff700 1 -- 172.21.15.46:0/1904767924 learned_addr learned my addr 172.21.15.46:0/1904767924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315f7fe700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3160101840 0x7f3160196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315effd700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3160104170 0x7f3160197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315ffff700 1 -- 172.21.15.46:0/1904767924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3160104170 msgr2=0x7f3160197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315ffff700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3160104170 0x7f3160197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315ffff700 1 -- 172.21.15.46:0/1904767924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3160101840 msgr2=0x7f3160196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315ffff700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3160101840 0x7f3160196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315ffff700 1 -- 172.21.15.46:0/1904767924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3150009d00 con 0x7f3160106aa0 2024-03-18T20:35:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315ffff700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3160106aa0 0x7f316019e330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f315800d980 tx=0x7f315800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315cff9700 1 -- 172.21.15.46:0/1904767924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158004a30 con 0x7f3160106aa0 2024-03-18T20:35:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f3166176700 1 -- 172.21.15.46:0/1904767924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316019ef10 con 0x7f3160106aa0 2024-03-18T20:35:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315cff9700 1 -- 172.21.15.46:0/1904767924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f315800b5c0 con 0x7f3160106aa0 2024-03-18T20:35:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f315cff9700 1 -- 172.21.15.46:0/1904767924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158009b50 con 0x7f3160106aa0 2024-03-18T20:35:28.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.009+0000 7f3166176700 1 -- 172.21.15.46:0/1904767924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316019f330 con 0x7f3160106aa0 2024-03-18T20:35:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.012+0000 7f315cff9700 1 -- 172.21.15.46:0/1904767924 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3158009cb0 con 0x7f3160106aa0 2024-03-18T20:35:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.013+0000 7f315cff9700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f31480a11b0 0x7f31480a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.013+0000 7f315f7fe700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f31480a11b0 0x7f31480a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.013+0000 7f315cff9700 1 -- 172.21.15.46:0/1904767924 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f31580cc840 con 0x7f3160106aa0 2024-03-18T20:35:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.013+0000 7f315f7fe700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f31480a11b0 0x7f31480a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3150009fd0 tx=0x7f315000b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f317230bb80 1 Processor -- start 2024-03-18T20:35:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f317230bb80 1 -- start start 2024-03-18T20:35:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f317230bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc842ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f317230bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 0x5598cc843df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f317230bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc8443d0 0x5598cc844850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f317230bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cc79b940 con 0x5598cc843530 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f317230bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cc799300 con 0x5598cc8443d0 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f317230bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cc794bf0 con 0x5598cc842bd0 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f315effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 0x5598cc843df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f315effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 0x5598cc843df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44132/0 (socket says 172.21.15.46:44132) 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f315effd700 1 -- 172.21.15.46:0/2651291620 learned_addr learned my addr 172.21.15.46:0/2651291620 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f315f7fe700 1 --2- 172.21.15.46:0/2651291620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc842ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f315ffff700 1 --2- 172.21.15.46:0/2651291620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc8443d0 0x5598cc844850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f315effd700 1 -- 172.21.15.46:0/2651291620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 msgr2=0x5598cc842ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f315effd700 1 --2- 172.21.15.46:0/2651291620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc842ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f315effd700 1 -- 172.21.15.46:0/2651291620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc8443d0 msgr2=0x5598cc844850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f315effd700 1 --2- 172.21.15.46:0/2651291620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc8443d0 0x5598cc844850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.014+0000 7f315effd700 1 -- 172.21.15.46:0/2651291620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598cc616360 con 0x5598cc843530 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.015+0000 7f315ffff700 1 --2- 172.21.15.46:0/2651291620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc8443d0 0x5598cc844850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.015+0000 7f315effd700 1 --2- 172.21.15.46:0/2651291620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 0x5598cc843df0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3154009d10 tx=0x7f31540116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77f1855f434ca74 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.015+0000 7f3145ffb700 1 -- 172.21.15.46:0/2651291620 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31540097e0 con 0x5598cc843530 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.015+0000 7f3145ffb700 1 -- 172.21.15.46:0/2651291620 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3154009e60 con 0x5598cc843530 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.015+0000 7f3145ffb700 1 -- 172.21.15.46:0/2651291620 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315400f7a0 con 0x5598cc843530 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.016+0000 7f317230bb80 1 -- 172.21.15.46:0/2651291620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 msgr2=0x5598cc843df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.016+0000 7f317230bb80 1 --2- 172.21.15.46:0/2651291620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 0x5598cc843df0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3154009d10 tx=0x7f31540116c0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.016+0000 7f317230bb80 1 -- 172.21.15.46:0/2651291620 shutdown_connections 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.016+0000 7f317230bb80 1 --2- 172.21.15.46:0/2651291620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc8443d0 0x5598cc844850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.016+0000 7f317230bb80 1 --2- 172.21.15.46:0/2651291620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc842ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.016+0000 7f317230bb80 1 --2- 172.21.15.46:0/2651291620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 0x5598cc843df0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.016+0000 7f317230bb80 1 -- 172.21.15.46:0/2651291620 >> 172.21.15.46:0/2651291620 conn(0x5598cc7b81d0 msgr2=0x5598cc83db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.016+0000 7f317230bb80 1 -- 172.21.15.46:0/2651291620 shutdown_connections 2024-03-18T20:35:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.016+0000 7f317230bb80 1 -- 172.21.15.46:0/2651291620 wait complete. 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.016+0000 7f317230bb80 1 -- start start 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.016+0000 7f317230bb80 1 Processor -- start 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f317230bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc84ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f317230bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 0x5598cc84cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f317230bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc8443d0 0x5598cc853fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f317230bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cc730d30 con 0x5598cc843530 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f317230bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cc7309d0 con 0x5598cc8443d0 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f317230bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cc618650 con 0x5598cc842bd0 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f317230bb80 10 client.0 _ll_register_callbacks cb 0x5598cc59c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f315effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 0x5598cc84cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f315effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 0x5598cc84cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44146/0 (socket says 172.21.15.46:44146) 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f315effd700 1 -- 172.21.15.46:0/540910249 learned_addr learned my addr 172.21.15.46:0/540910249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f315f7fe700 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc84ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f315ffff700 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc8443d0 0x5598cc853fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f315effd700 1 -- 172.21.15.46:0/540910249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 msgr2=0x5598cc84ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f315effd700 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc84ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f315effd700 1 -- 172.21.15.46:0/540910249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc8443d0 msgr2=0x5598cc853fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f315effd700 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc8443d0 0x5598cc853fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f315effd700 1 -- 172.21.15.46:0/540910249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3154009940 con 0x5598cc843530 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f315effd700 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 0x5598cc84cf90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f31540095c0 tx=0x7f3154012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cde256f05610b89 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f313effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.017+0000 7f313effd700 1 -- 172.21.15.46:0/540910249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154012d10 con 0x5598cc843530 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f313effd700 1 -- 172.21.15.46:0/540910249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3154005040 con 0x5598cc843530 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f317230bb80 1 -- 172.21.15.46:0/540910249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598cc835a80 con 0x5598cc843530 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f313effd700 1 -- 172.21.15.46:0/540910249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3154005200 con 0x5598cc843530 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f317230bb80 10 client.15096 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f317230bb80 1 -- 172.21.15.46:0/540910249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5598cc5c2b10 con 0x5598cc843530 2024-03-18T20:35:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f313effd700 1 -- 172.21.15.46:0/540910249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5598cc835a80 con 0x5598cc843530 2024-03-18T20:35:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f313effd700 1 -- 172.21.15.46:0/540910249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3154022070 con 0x5598cc843530 2024-03-18T20:35:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f317230bb80 10 client.15096 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f317230bb80 10 client.15096 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-18T20:35:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f317230bb80 20 client.15096 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f317230bb80 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5598cc857ba0 0x5598cc85cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f317230bb80 4 client.15096 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:35:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f317230bb80 1 -- 172.21.15.46:0/540910249 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598cc78d6b0 con 0x5598cc857ba0 2024-03-18T20:35:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.018+0000 7f315f7fe700 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5598cc857ba0 0x5598cc85cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.019+0000 7f313effd700 10 client.15096 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:35:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.019+0000 7f315f7fe700 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5598cc857ba0 0x5598cc85cea0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=beed75f634040dca server_cookie=7d364bca7a7c50b7 in_seq=0 out_seq=0 2024-03-18T20:35:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.019+0000 7f313effd700 1 -- 172.21.15.46:0/540910249 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5598cc78d6b0 con 0x5598cc857ba0 2024-03-18T20:35:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.019+0000 7f313effd700 10 client.15096 handle_command_reply: tid=0 2024-03-18T20:35:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.019+0000 7f317230bb80 1 client.15096 shutdown 2024-03-18T20:35:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.020+0000 7f317230bb80 1 -- 172.21.15.46:0/540910249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 msgr2=0x5598cc84cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.020+0000 7f317230bb80 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 0x5598cc84cf90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f31540095c0 tx=0x7f3154012700 comp rx=0 tx=0).stop 2024-03-18T20:35:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.020+0000 7f317230bb80 1 -- 172.21.15.46:0/540910249 shutdown_connections 2024-03-18T20:35:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.020+0000 7f317230bb80 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5598cc857ba0 0x5598cc85cea0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.020+0000 7f317230bb80 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc8443d0 0x5598cc853fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.020+0000 7f317230bb80 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc84ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.020+0000 7f313effd700 0 client.15096 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:35:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.020+0000 7f317230bb80 1 --2- 172.21.15.46:0/540910249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc843530 0x5598cc84cf90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.020+0000 7f317230bb80 1 -- 172.21.15.46:0/540910249 >> 172.21.15.46:0/540910249 conn(0x5598cc7b81d0 msgr2=0x5598cc83d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.020+0000 7f317230bb80 1 -- 172.21.15.46:0/540910249 shutdown_connections 2024-03-18T20:35:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.020+0000 7f317230bb80 1 -- 172.21.15.46:0/540910249 wait complete. 2024-03-18T20:35:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.020+0000 7f317230bb80 20 client.15096 trim_cache size 0 max 16384 2024-03-18T20:35:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.038+0000 7f317230bb80 1 Processor -- start 2024-03-18T20:35:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.038+0000 7f317230bb80 1 -- start start 2024-03-18T20:35:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.038+0000 7f317230bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc83da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.038+0000 7f317230bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 0x5598cc84bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.038+0000 7f317230bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 0x5598cc847290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.038+0000 7f317230bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cc78d6b0 con 0x5598cc8443d0 2024-03-18T20:35:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f317230bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cc78d990 con 0x5598cc843530 2024-03-18T20:35:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f317230bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cc733de0 con 0x5598cc842bd0 2024-03-18T20:35:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f315ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 0x5598cc847290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f315ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 0x5598cc847290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44160/0 (socket says 172.21.15.46:44160) 2024-03-18T20:35:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f315ffff700 1 -- 172.21.15.46:0/2997384968 learned_addr learned my addr 172.21.15.46:0/2997384968 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f315f7fe700 1 --2- 172.21.15.46:0/2997384968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc83da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f315effd700 1 --2- 172.21.15.46:0/2997384968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 0x5598cc84bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f315ffff700 1 -- 172.21.15.46:0/2997384968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 msgr2=0x5598cc83da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f315ffff700 1 --2- 172.21.15.46:0/2997384968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc83da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f315ffff700 1 -- 172.21.15.46:0/2997384968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 msgr2=0x5598cc84bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f315ffff700 1 --2- 172.21.15.46:0/2997384968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 0x5598cc84bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f315ffff700 1 -- 172.21.15.46:0/2997384968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598cc75ae50 con 0x5598cc8443d0 2024-03-18T20:35:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f315effd700 1 --2- 172.21.15.46:0/2997384968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 0x5598cc84bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f315ffff700 1 --2- 172.21.15.46:0/2997384968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 0x5598cc847290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3158099b70 tx=0x7f3158095f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a029b999a618752 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f31467fc700 1 -- 172.21.15.46:0/2997384968 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158099be0 con 0x5598cc8443d0 2024-03-18T20:35:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f31467fc700 1 -- 172.21.15.46:0/2997384968 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f315809da10 con 0x5598cc8443d0 2024-03-18T20:35:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.039+0000 7f31467fc700 1 -- 172.21.15.46:0/2997384968 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315809a0a0 con 0x5598cc8443d0 2024-03-18T20:35:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.040+0000 7f317230bb80 1 -- 172.21.15.46:0/2997384968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 msgr2=0x5598cc847290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.040+0000 7f317230bb80 1 --2- 172.21.15.46:0/2997384968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 0x5598cc847290 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3158099b70 tx=0x7f3158095f20 comp rx=0 tx=0).stop 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.040+0000 7f317230bb80 1 -- 172.21.15.46:0/2997384968 shutdown_connections 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.040+0000 7f317230bb80 1 --2- 172.21.15.46:0/2997384968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 0x5598cc847290 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.040+0000 7f317230bb80 1 --2- 172.21.15.46:0/2997384968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc83da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.040+0000 7f317230bb80 1 --2- 172.21.15.46:0/2997384968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 0x5598cc84bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.040+0000 7f317230bb80 1 -- 172.21.15.46:0/2997384968 >> 172.21.15.46:0/2997384968 conn(0x5598cc7b81d0 msgr2=0x5598cc84a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.040+0000 7f317230bb80 1 -- 172.21.15.46:0/2997384968 shutdown_connections 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.040+0000 7f317230bb80 1 -- 172.21.15.46:0/2997384968 wait complete. 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.040+0000 7f317230bb80 1 -- start start 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.040+0000 7f317230bb80 1 Processor -- start 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f317230bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc8496d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f317230bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 0x5598cc849c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f317230bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 0x5598cc8b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f317230bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cc8b0c10 con 0x5598cc8443d0 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f317230bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cc8b0d80 con 0x5598cc843530 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f317230bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cc8b0f40 con 0x5598cc842bd0 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f317230bb80 10 client.0 _ll_register_callbacks cb 0x7f3150009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 0x5598cc8b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 0x5598cc849c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc8496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 0x5598cc849c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54716/0 (socket says 172.21.15.46:54716) 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 0x5598cc8b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44166/0 (socket says 172.21.15.46:44166) 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315effd700 1 -- 172.21.15.46:0/4149230771 learned_addr learned my addr 172.21.15.46:0/4149230771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315effd700 1 -- 172.21.15.46:0/4149230771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 msgr2=0x5598cc8496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315effd700 1 --2- 172.21.15.46:0/4149230771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc8496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315effd700 1 -- 172.21.15.46:0/4149230771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 msgr2=0x5598cc8b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315effd700 1 --2- 172.21.15.46:0/4149230771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 0x5598cc8b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315effd700 1 -- 172.21.15.46:0/4149230771 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316019f330 con 0x5598cc843530 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315ffff700 1 --2- 172.21.15.46:0/4149230771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 0x5598cc8b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.041+0000 7f315effd700 1 --2- 172.21.15.46:0/4149230771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 0x5598cc849c10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f31540092a0 tx=0x7f315402fd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8278c12a01c2a584 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f313effd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f313effd700 1 -- 172.21.15.46:0/4149230771 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315402f7f0 con 0x5598cc843530 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f313effd700 1 -- 172.21.15.46:0/4149230771 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3154009350 con 0x5598cc843530 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f317230bb80 1 -- 172.21.15.46:0/4149230771 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3154009e60 con 0x5598cc843530 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f313effd700 1 -- 172.21.15.46:0/4149230771 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f315402ff00 con 0x5598cc843530 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f317230bb80 10 client.24935 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f317230bb80 1 -- 172.21.15.46:0/4149230771 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5598cc61ac80 con 0x5598cc843530 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f313effd700 1 -- 172.21.15.46:0/4149230771 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3154009e60 con 0x5598cc843530 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f313effd700 1 -- 172.21.15.46:0/4149230771 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3154011730 con 0x5598cc843530 2024-03-18T20:35:28.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f317230bb80 10 client.24935 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f317230bb80 10 client.24935 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-18T20:35:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f317230bb80 20 client.24935 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f317230bb80 1 --2- 172.21.15.46:0/4149230771 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5598cc8b5a30 0x5598cc8b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f317230bb80 4 client.24935 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T20:35:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.042+0000 7f317230bb80 1 -- 172.21.15.46:0/4149230771 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5598cc5e3ca0 con 0x5598cc8b5a30 2024-03-18T20:35:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.043+0000 7f315f7fe700 1 --2- 172.21.15.46:0/4149230771 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5598cc8b5a30 0x5598cc8b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.043+0000 7f313effd700 10 client.24935 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:35:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.043+0000 7f315f7fe700 1 --2- 172.21.15.46:0/4149230771 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5598cc8b5a30 0x5598cc8b23c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a728c0b82c71d13 server_cookie=cc1b624e833bf80b in_seq=0 out_seq=0 2024-03-18T20:35:28.052 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:27 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/426401771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:28.052 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:27 smithi046 ceph-mon[43495]: pgmap v243: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T20:35:28.052 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:27 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/722054868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:28.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.043+0000 7f313effd700 1 -- 172.21.15.46:0/4149230771 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5598cc5e3ca0 con 0x5598cc8b5a30 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.043+0000 7f313effd700 10 client.24935 handle_command_reply: tid=0 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.043+0000 7f317230bb80 1 client.24935 shutdown 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.043+0000 7f317230bb80 1 -- 172.21.15.46:0/4149230771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 msgr2=0x5598cc849c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.043+0000 7f317230bb80 1 --2- 172.21.15.46:0/4149230771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 0x5598cc849c10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f31540092a0 tx=0x7f315402fd10 comp rx=0 tx=0).stop 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f317230bb80 1 -- 172.21.15.46:0/4149230771 shutdown_connections 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f317230bb80 1 --2- 172.21.15.46:0/4149230771 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5598cc8b5a30 0x5598cc8b23c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f317230bb80 1 --2- 172.21.15.46:0/4149230771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598cc8443d0 0x5598cc8b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f317230bb80 1 --2- 172.21.15.46:0/4149230771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cc842bd0 0x5598cc8496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f317230bb80 1 --2- 172.21.15.46:0/4149230771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598cc843530 0x5598cc849c10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f313effd700 0 client.24935 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f317230bb80 1 -- 172.21.15.46:0/4149230771 >> 172.21.15.46:0/4149230771 conn(0x5598cc7b81d0 msgr2=0x5598cc84bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f317230bb80 1 -- 172.21.15.46:0/4149230771 shutdown_connections 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f317230bb80 1 -- 172.21.15.46:0/4149230771 wait complete. 2024-03-18T20:35:28.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f317230bb80 20 client.24935 trim_cache size 0 max 16384 2024-03-18T20:35:28.055 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:35:28.055 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:35:28.055 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-03-18T20:35:28.055 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24521, 2024-03-18T20:35:28.055 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-18T20:35:28.055 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-18T20:35:28.055 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-18T20:35:28.055 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 46.239413892000002, 2024-03-18T20:35:28.055 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-03-18T20:35:28.055 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 78, 2024-03-18T20:35:28.055 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-03-18T20:35:28.056 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 58.892749215999999 2024-03-18T20:35:28.056 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:35:28.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f3166176700 1 -- 172.21.15.46:0/1904767924 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f31480a11b0 msgr2=0x7f31480a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f3166176700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f31480a11b0 0x7f31480a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3150009fd0 tx=0x7f315000b700 comp rx=0 tx=0).stop 2024-03-18T20:35:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f3166176700 1 -- 172.21.15.46:0/1904767924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3160106aa0 msgr2=0x7f316019e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f3166176700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3160106aa0 0x7f316019e330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f315800d980 tx=0x7f315800dc90 comp rx=0 tx=0).stop 2024-03-18T20:35:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f3166176700 1 -- 172.21.15.46:0/1904767924 shutdown_connections 2024-03-18T20:35:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f3166176700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f31480a11b0 0x7f31480a3670 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f3166176700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3160106aa0 0x7f316019e330 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f3166176700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3160101840 0x7f3160196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f3166176700 1 --2- 172.21.15.46:0/1904767924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3160104170 0x7f3160197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.044+0000 7f3166176700 1 -- 172.21.15.46:0/1904767924 >> 172.21.15.46:0/1904767924 conn(0x7f31600fb1d0 msgr2=0x7f31601036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.045+0000 7f3166176700 1 -- 172.21.15.46:0/1904767924 shutdown_connections 2024-03-18T20:35:28.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.045+0000 7f3166176700 1 -- 172.21.15.46:0/1904767924 wait complete. 2024-03-18T20:35:28.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-18T20:35:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.196+0000 7f6da0b1c700 1 -- 172.21.15.46:0/3546007886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9c1017e0 msgr2=0x7f6d9c103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.196+0000 7f6da0b1c700 1 --2- 172.21.15.46:0/3546007886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9c1017e0 0x7f6d9c103bd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00b0a0 tx=0x7f6d8c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.196+0000 7f6da0b1c700 1 -- 172.21.15.46:0/3546007886 shutdown_connections 2024-03-18T20:35:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.196+0000 7f6da0b1c700 1 --2- 172.21.15.46:0/3546007886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9c1017e0 0x7f6d9c103bd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.196+0000 7f6da0b1c700 1 --2- 172.21.15.46:0/3546007886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9c106a40 0x7f6d9c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.196+0000 7f6da0b1c700 1 --2- 172.21.15.46:0/3546007886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c104110 0x7f6d9c106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.196+0000 7f6da0b1c700 1 -- 172.21.15.46:0/3546007886 >> 172.21.15.46:0/3546007886 conn(0x7f6d9c0fb1d0 msgr2=0x7f6d9c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.196+0000 7f6da0b1c700 1 -- 172.21.15.46:0/3546007886 shutdown_connections 2024-03-18T20:35:28.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.196+0000 7f6da0b1c700 1 -- 172.21.15.46:0/3546007886 wait complete. 2024-03-18T20:35:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.197+0000 7f6da0b1c700 1 Processor -- start 2024-03-18T20:35:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.197+0000 7f6da0b1c700 1 -- start start 2024-03-18T20:35:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.197+0000 7f6da0b1c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c1017e0 0x7f6d9c196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.197+0000 7f6da0b1c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9c104110 0x7f6d9c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.197+0000 7f6da0b1c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9c106a40 0x7f6d9c19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6da0b1c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c19e860 con 0x7f6d9c106a40 2024-03-18T20:35:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6da0b1c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c19e9d0 con 0x7f6d9c104110 2024-03-18T20:35:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6da0b1c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c19eb90 con 0x7f6d9c1017e0 2024-03-18T20:35:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d9ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9c106a40 0x7f6d9c19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d9ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9c106a40 0x7f6d9c19e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44192/0 (socket says 172.21.15.46:44192) 2024-03-18T20:35:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d9ad9d700 1 -- 172.21.15.46:0/1971221194 learned_addr learned my addr 172.21.15.46:0/1971221194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d9a59c700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c1017e0 0x7f6d9c196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d99d9b700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9c104110 0x7f6d9c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d9ad9d700 1 -- 172.21.15.46:0/1971221194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c1017e0 msgr2=0x7f6d9c196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d9ad9d700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c1017e0 0x7f6d9c196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d9ad9d700 1 -- 172.21.15.46:0/1971221194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9c104110 msgr2=0x7f6d9c1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d9ad9d700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9c104110 0x7f6d9c1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d9ad9d700 1 -- 172.21.15.46:0/1971221194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c009d00 con 0x7f6d9c106a40 2024-03-18T20:35:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d9ad9d700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9c106a40 0x7f6d9c19e290 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400d980 tx=0x7f6d9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d8b7fe700 1 -- 172.21.15.46:0/1971221194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94004a30 con 0x7f6d9c106a40 2024-03-18T20:35:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d8b7fe700 1 -- 172.21.15.46:0/1971221194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d940054e0 con 0x7f6d9c106a40 2024-03-18T20:35:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6da0b1c700 1 -- 172.21.15.46:0/1971221194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9c19ee70 con 0x7f6d9c106a40 2024-03-18T20:35:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.198+0000 7f6d8b7fe700 1 -- 172.21.15.46:0/1971221194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9400fbc0 con 0x7f6d9c106a40 2024-03-18T20:35:28.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.199+0000 7f6da0b1c700 1 -- 172.21.15.46:0/1971221194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9c19f240 con 0x7f6d9c106a40 2024-03-18T20:35:28.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.201+0000 7f6d8b7fe700 1 -- 172.21.15.46:0/1971221194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f6d94004c20 con 0x7f6d9c106a40 2024-03-18T20:35:28.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.201+0000 7f6d8b7fe700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6d840a11b0 0x7f6d840a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.202+0000 7f6d8b7fe700 1 -- 172.21.15.46:0/1971221194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d940cc770 con 0x7f6d9c106a40 2024-03-18T20:35:28.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.202+0000 7f6d9a59c700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6d840a11b0 0x7f6d840a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.202+0000 7f6d9a59c700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6d840a11b0 0x7f6d840a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009fd0 tx=0x7f6d8c006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.202+0000 7f6daccb1b80 1 Processor -- start 2024-03-18T20:35:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6daccb1b80 1 -- start start 2024-03-18T20:35:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6daccb1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec6fe0 0x5608fbec7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6daccb1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 0x5608fbec8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6daccb1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec87e0 0x5608fbec8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6daccb1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbe1b940 con 0x5608fbec7940 2024-03-18T20:35:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6daccb1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbe19300 con 0x5608fbec87e0 2024-03-18T20:35:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6daccb1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbe14bf0 con 0x5608fbec6fe0 2024-03-18T20:35:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6d99d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 0x5608fbec8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6d99d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 0x5608fbec8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44206/0 (socket says 172.21.15.46:44206) 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6d99d9b700 1 -- 172.21.15.46:0/3527801220 learned_addr learned my addr 172.21.15.46:0/3527801220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6d9a59c700 1 --2- 172.21.15.46:0/3527801220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec6fe0 0x5608fbec7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6d9ad9d700 1 --2- 172.21.15.46:0/3527801220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec87e0 0x5608fbec8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6d99d9b700 1 -- 172.21.15.46:0/3527801220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec6fe0 msgr2=0x5608fbec7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6d99d9b700 1 --2- 172.21.15.46:0/3527801220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec6fe0 0x5608fbec7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6d99d9b700 1 -- 172.21.15.46:0/3527801220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec87e0 msgr2=0x5608fbec8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6d99d9b700 1 --2- 172.21.15.46:0/3527801220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec87e0 0x5608fbec8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6d99d9b700 1 -- 172.21.15.46:0/3527801220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608fbc96360 con 0x5608fbec7940 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6d9ad9d700 1 --2- 172.21.15.46:0/3527801220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec87e0 0x5608fbec8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.203+0000 7f6d99d9b700 1 --2- 172.21.15.46:0/3527801220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 0x5608fbec8200 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000ad70 tx=0x7f6d9000f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96839328d68bf2e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.204+0000 7f6d88ff9700 1 -- 172.21.15.46:0/3527801220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000a870 con 0x5608fbec7940 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.204+0000 7f6d88ff9700 1 -- 172.21.15.46:0/3527801220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d9000fec0 con 0x5608fbec7940 2024-03-18T20:35:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.204+0000 7f6d88ff9700 1 -- 172.21.15.46:0/3527801220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d900186a0 con 0x5608fbec7940 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.204+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3527801220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 msgr2=0x5608fbec8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.204+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/3527801220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 0x5608fbec8200 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000ad70 tx=0x7f6d9000f6a0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.204+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3527801220 shutdown_connections 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.204+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/3527801220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec87e0 0x5608fbec8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.204+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/3527801220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec6fe0 0x5608fbec7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.204+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/3527801220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 0x5608fbec8200 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.204+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3527801220 >> 172.21.15.46:0/3527801220 conn(0x5608fbe381d0 msgr2=0x5608fbebdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3527801220 shutdown_connections 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3527801220 wait complete. 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6daccb1b80 1 -- start start 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6daccb1b80 1 Processor -- start 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6daccb1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec6fe0 0x5608fbed0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6daccb1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 0x5608fbed1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6daccb1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec87e0 0x5608fbed8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6daccb1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbdb0d30 con 0x5608fbec7940 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6daccb1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbdb09d0 con 0x5608fbec6fe0 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6daccb1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbc98650 con 0x5608fbec87e0 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6daccb1b80 10 client.0 _ll_register_callbacks cb 0x5608fbc1c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6d99d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 0x5608fbed1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6d99d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 0x5608fbed1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44208/0 (socket says 172.21.15.46:44208) 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.205+0000 7f6d99d9b700 1 -- 172.21.15.46:0/930480454 learned_addr learned my addr 172.21.15.46:0/930480454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d9ad9d700 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec87e0 0x5608fbed8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d9a59c700 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec6fe0 0x5608fbed0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d99d9b700 1 -- 172.21.15.46:0/930480454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec87e0 msgr2=0x5608fbed8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d99d9b700 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec87e0 0x5608fbed8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d99d9b700 1 -- 172.21.15.46:0/930480454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec6fe0 msgr2=0x5608fbed0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d99d9b700 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec6fe0 0x5608fbed0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d99d9b700 1 -- 172.21.15.46:0/930480454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9000a9d0 con 0x5608fbec7940 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d9ad9d700 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec87e0 0x5608fbed8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d99d9b700 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 0x5608fbed1450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000faa0 tx=0x7f6d900038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8c8bfe2a4632ae server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d81ffb700 1 -- 172.21.15.46:0/930480454 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90005450 con 0x5608fbec7940 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d81ffb700 1 -- 172.21.15.46:0/930480454 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d90005df0 con 0x5608fbec7940 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6daccb1b80 1 -- 172.21.15.46:0/930480454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608fbe0d6b0 con 0x5608fbec7940 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d81ffb700 1 -- 172.21.15.46:0/930480454 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d9000a850 con 0x5608fbec7940 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6daccb1b80 10 client.15114 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6daccb1b80 1 -- 172.21.15.46:0/930480454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5608fbc42b10 con 0x5608fbec7940 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.206+0000 7f6d81ffb700 1 -- 172.21.15.46:0/930480454 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5608fbe0d6b0 con 0x5608fbec7940 2024-03-18T20:35:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.207+0000 7f6d81ffb700 1 -- 172.21.15.46:0/930480454 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d90020040 con 0x5608fbec7940 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.207+0000 7f6daccb1b80 10 client.15114 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.207+0000 7f6daccb1b80 10 client.15114 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.207+0000 7f6daccb1b80 20 client.15114 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.207+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608fbedbde0 0x5608fbee10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.207+0000 7f6daccb1b80 4 client.15114 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.207+0000 7f6daccb1b80 1 -- 172.21.15.46:0/930480454 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608fbc63ca0 con 0x5608fbedbde0 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.207+0000 7f6d9a59c700 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608fbedbde0 0x5608fbee10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.207+0000 7f6d81ffb700 10 client.15114 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.207+0000 7f6d9a59c700 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608fbedbde0 0x5608fbee10e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d184ebaf150a14fd server_cookie=315a94341e0628f4 in_seq=0 out_seq=0 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6d81ffb700 1 -- 172.21.15.46:0/930480454 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5608fbc63ca0 con 0x5608fbedbde0 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6d81ffb700 10 client.15114 handle_command_reply: tid=0 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6daccb1b80 1 client.15114 shutdown 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6daccb1b80 1 -- 172.21.15.46:0/930480454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 msgr2=0x5608fbed1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 0x5608fbed1450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000faa0 tx=0x7f6d900038b0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6daccb1b80 1 -- 172.21.15.46:0/930480454 shutdown_connections 2024-03-18T20:35:28.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608fbedbde0 0x5608fbee10e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec87e0 0x5608fbed8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec6fe0 0x5608fbed0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/930480454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec7940 0x5608fbed1450 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6d81ffb700 0 client.15114 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:28.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6daccb1b80 1 -- 172.21.15.46:0/930480454 >> 172.21.15.46:0/930480454 conn(0x5608fbe381d0 msgr2=0x5608fbebd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:28.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.208+0000 7f6daccb1b80 1 -- 172.21.15.46:0/930480454 shutdown_connections 2024-03-18T20:35:28.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.209+0000 7f6daccb1b80 1 -- 172.21.15.46:0/930480454 wait complete. 2024-03-18T20:35:28.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.209+0000 7f6daccb1b80 20 client.15114 trim_cache size 0 max 16384 2024-03-18T20:35:28.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:27 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/426401771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:28.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:27 smithi073 ceph-mon[47384]: pgmap v243: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T20:35:28.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:27 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/722054868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.227+0000 7f6daccb1b80 1 Processor -- start 2024-03-18T20:35:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6daccb1b80 1 -- start start 2024-03-18T20:35:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6daccb1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec6fe0 0x5608fbebdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6daccb1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec7940 0x5608fbed8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6daccb1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec87e0 0x5608fbed0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6daccb1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbdb3de0 con 0x5608fbec6fe0 2024-03-18T20:35:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6daccb1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbe888f0 con 0x5608fbec87e0 2024-03-18T20:35:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6daccb1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbeb81c0 con 0x5608fbec7940 2024-03-18T20:35:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d9a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec6fe0 0x5608fbebdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d9a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec6fe0 0x5608fbebdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44210/0 (socket says 172.21.15.46:44210) 2024-03-18T20:35:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d9a59c700 1 -- 172.21.15.46:0/4094759472 learned_addr learned my addr 172.21.15.46:0/4094759472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d9a59c700 1 -- 172.21.15.46:0/4094759472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec7940 msgr2=0x5608fbed8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d9a59c700 1 --2- 172.21.15.46:0/4094759472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec7940 0x5608fbed8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d9a59c700 1 -- 172.21.15.46:0/4094759472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec87e0 msgr2=0x5608fbed0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d9ad9d700 1 --2- 172.21.15.46:0/4094759472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec87e0 0x5608fbed0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d9a59c700 1 --2- 172.21.15.46:0/4094759472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec87e0 0x5608fbed0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d9a59c700 1 -- 172.21.15.46:0/4094759472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608fbddae50 con 0x5608fbec6fe0 2024-03-18T20:35:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d9ad9d700 1 --2- 172.21.15.46:0/4094759472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec87e0 0x5608fbed0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:28.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d9a59c700 1 --2- 172.21.15.46:0/4094759472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec6fe0 0x5608fbebdad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0060b0 tx=0x7f6d8c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6b6719ccd097eb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d897fa700 1 -- 172.21.15.46:0/4094759472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00e040 con 0x5608fbec6fe0 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d897fa700 1 -- 172.21.15.46:0/4094759472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8c030d80 con 0x5608fbec6fe0 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.228+0000 7f6d897fa700 1 -- 172.21.15.46:0/4094759472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c01cca0 con 0x5608fbec6fe0 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.229+0000 7f6daccb1b80 1 -- 172.21.15.46:0/4094759472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec6fe0 msgr2=0x5608fbebdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.229+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/4094759472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec6fe0 0x5608fbebdad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0060b0 tx=0x7f6d8c01f000 comp rx=0 tx=0).stop 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.229+0000 7f6daccb1b80 1 -- 172.21.15.46:0/4094759472 shutdown_connections 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.229+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/4094759472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec6fe0 0x5608fbebdad0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.229+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/4094759472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec87e0 0x5608fbed0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.229+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/4094759472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec7940 0x5608fbed8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.229+0000 7f6daccb1b80 1 -- 172.21.15.46:0/4094759472 >> 172.21.15.46:0/4094759472 conn(0x5608fbe381d0 msgr2=0x5608fbeb99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.229+0000 7f6daccb1b80 1 -- 172.21.15.46:0/4094759472 shutdown_connections 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.229+0000 7f6daccb1b80 1 -- 172.21.15.46:0/4094759472 wait complete. 2024-03-18T20:35:28.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.229+0000 7f6daccb1b80 1 -- start start 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6daccb1b80 1 Processor -- start 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6daccb1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec6fe0 0x5608fbed7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6daccb1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec7940 0x5608fbed4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6daccb1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec87e0 0x5608fbed51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6daccb1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbed8850 con 0x5608fbec87e0 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6daccb1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbed5710 con 0x5608fbec6fe0 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6daccb1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608fbed58d0 con 0x5608fbec7940 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6daccb1b80 10 client.0 _ll_register_callbacks cb 0x7f6d9000abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d9ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec87e0 0x5608fbed51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d9a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec6fe0 0x5608fbed7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d9ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec87e0 0x5608fbed51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44226/0 (socket says 172.21.15.46:44226) 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d9ad9d700 1 -- 172.21.15.46:0/3903361432 learned_addr learned my addr 172.21.15.46:0/3903361432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d99d9b700 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec7940 0x5608fbed4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d9ad9d700 1 -- 172.21.15.46:0/3903361432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec7940 msgr2=0x5608fbed4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d9ad9d700 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec7940 0x5608fbed4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d9ad9d700 1 -- 172.21.15.46:0/3903361432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec6fe0 msgr2=0x5608fbed7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d9ad9d700 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec6fe0 0x5608fbed7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d9ad9d700 1 -- 172.21.15.46:0/3903361432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9c109aa0 con 0x5608fbec87e0 2024-03-18T20:35:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d99d9b700 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec7940 0x5608fbed4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.230+0000 7f6d9ad9d700 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec87e0 0x5608fbed51a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400be00 tx=0x7f6d94099c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b3c6476270b1d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6d81ffb700 1 -- 172.21.15.46:0/3903361432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94022300 con 0x5608fbec87e0 2024-03-18T20:35:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6d81ffb700 1 -- 172.21.15.46:0/3903361432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d94022460 con 0x5608fbec87e0 2024-03-18T20:35:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3903361432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608fbeb5a80 con 0x5608fbec87e0 2024-03-18T20:35:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6d81ffb700 1 -- 172.21.15.46:0/3903361432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d94099d70 con 0x5608fbec87e0 2024-03-18T20:35:28.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6daccb1b80 10 client.15120 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3903361432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5608fbc9ac80 con 0x5608fbec87e0 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6d81ffb700 1 -- 172.21.15.46:0/3903361432 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5608fbeb5a80 con 0x5608fbec87e0 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6d81ffb700 1 -- 172.21.15.46:0/3903361432 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d940a1b00 con 0x5608fbec87e0 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6daccb1b80 10 client.15120 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6daccb1b80 10 client.15120 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6daccb1b80 20 client.15120 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608fbf35370 0x5608fbf31ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6daccb1b80 4 client.15120 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3903361432 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5608fbc98c90 con 0x5608fbf35370 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.231+0000 7f6d9a59c700 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608fbf35370 0x5608fbf31ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6d81ffb700 10 client.15120 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6d9a59c700 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608fbf35370 0x5608fbf31ce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34296856445d254f server_cookie=7fdd647b406d104 in_seq=0 out_seq=0 2024-03-18T20:35:28.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6d81ffb700 1 -- 172.21.15.46:0/3903361432 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5608fbc98c90 con 0x5608fbf35370 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6d81ffb700 10 client.15120 handle_command_reply: tid=0 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6daccb1b80 1 client.15120 shutdown 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3903361432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec87e0 msgr2=0x5608fbed51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec87e0 0x5608fbed51a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400be00 tx=0x7f6d94099c50 comp rx=0 tx=0).stop 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3903361432 shutdown_connections 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608fbf35370 0x5608fbf31ce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608fbec87e0 0x5608fbed51a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608fbec6fe0 0x5608fbed7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6daccb1b80 1 --2- 172.21.15.46:0/3903361432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608fbec7940 0x5608fbed4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6d81ffb700 0 client.15120 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3903361432 >> 172.21.15.46:0/3903361432 conn(0x5608fbe381d0 msgr2=0x5608fbed0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.232+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3903361432 shutdown_connections 2024-03-18T20:35:28.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6daccb1b80 1 -- 172.21.15.46:0/3903361432 wait complete. 2024-03-18T20:35:28.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6daccb1b80 20 client.15120 trim_cache size 0 max 16384 2024-03-18T20:35:28.243 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:35:28.243 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:35:28.243 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-18T20:35:28.243 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24536, 2024-03-18T20:35:28.243 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-18T20:35:28.243 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-18T20:35:28.243 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-18T20:35:28.243 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 49.444616023000002, 2024-03-18T20:35:28.243 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-03-18T20:35:28.243 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 78, 2024-03-18T20:35:28.244 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-03-18T20:35:28.244 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 55.300237736 2024-03-18T20:35:28.244 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:35:28.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6da0b1c700 1 -- 172.21.15.46:0/1971221194 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6d840a11b0 msgr2=0x7f6d840a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6da0b1c700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6d840a11b0 0x7f6d840a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009fd0 tx=0x7f6d8c006000 comp rx=0 tx=0).stop 2024-03-18T20:35:28.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6da0b1c700 1 -- 172.21.15.46:0/1971221194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9c106a40 msgr2=0x7f6d9c19e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:28.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6da0b1c700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9c106a40 0x7f6d9c19e290 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400d980 tx=0x7f6d9400dc90 comp rx=0 tx=0).stop 2024-03-18T20:35:28.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6da0b1c700 1 -- 172.21.15.46:0/1971221194 shutdown_connections 2024-03-18T20:35:28.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6da0b1c700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6d840a11b0 0x7f6d840a3670 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6da0b1c700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d9c106a40 0x7f6d9c19e290 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6da0b1c700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c1017e0 0x7f6d9c196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6da0b1c700 1 --2- 172.21.15.46:0/1971221194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d9c104110 0x7f6d9c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:28.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6da0b1c700 1 -- 172.21.15.46:0/1971221194 >> 172.21.15.46:0/1971221194 conn(0x7f6d9c0fb1d0 msgr2=0x7f6d9c102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:28.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6da0b1c700 1 -- 172.21.15.46:0/1971221194 shutdown_connections 2024-03-18T20:35:28.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:28.233+0000 7f6da0b1c700 1 -- 172.21.15.46:0/1971221194 wait complete. 2024-03-18T20:35:28.250 INFO:tasks.cephadm:Setup complete, yielding 2024-03-18T20:35:28.251 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T20:35:28.259 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-03-18T20:35:28.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-18T20:35:28.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:27 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/426401771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:28.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:27 smithi155 ceph-mon[48344]: pgmap v243: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T20:35:28.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:27 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/722054868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:28.508 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.463+0000 7fc8f9485700 1 -- 172.21.15.46:0/2257178818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4106ae0 msgr2=0x7fc8f4108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.463+0000 7fc8f9485700 1 --2- 172.21.15.46:0/2257178818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4106ae0 0x7fc8f4108f50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00b0a0 tx=0x7fc8ec00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.463+0000 7fc8f9485700 1 -- 172.21.15.46:0/2257178818 shutdown_connections 2024-03-18T20:35:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.463+0000 7fc8f9485700 1 --2- 172.21.15.46:0/2257178818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4106ae0 0x7fc8f4108f50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.463+0000 7fc8f9485700 1 --2- 172.21.15.46:0/2257178818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f4101880 0x7fc8f4103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.463+0000 7fc8f9485700 1 --2- 172.21.15.46:0/2257178818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8f41041b0 0x7fc8f41065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:29.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.463+0000 7fc8f9485700 1 -- 172.21.15.46:0/2257178818 >> 172.21.15.46:0/2257178818 conn(0x7fc8f40fb2b0 msgr2=0x7fc8f40fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.463+0000 7fc8f9485700 1 -- 172.21.15.46:0/2257178818 shutdown_connections 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.463+0000 7fc8f9485700 1 -- 172.21.15.46:0/2257178818 wait complete. 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.464+0000 7fc8f9485700 1 Processor -- start 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.464+0000 7fc8f9485700 1 -- start start 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.464+0000 7fc8f9485700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f4101880 0x7fc8f4196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.464+0000 7fc8f9485700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f41041b0 0x7fc8f4197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.464+0000 7fc8f9485700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8f4106ae0 0x7fc8f419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.464+0000 7fc8f9485700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f419e950 con 0x7fc8f41041b0 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.464+0000 7fc8f9485700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f419ea90 con 0x7fc8f4106ae0 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.464+0000 7fc8f9485700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f419ec20 con 0x7fc8f4101880 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f27fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f41041b0 0x7fc8f4197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f27fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f41041b0 0x7fc8f4197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44246/0 (socket says 172.21.15.46:44246) 2024-03-18T20:35:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f27fc700 1 -- 172.21.15.46:0/3683011169 learned_addr learned my addr 172.21.15.46:0/3683011169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f2ffd700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f4101880 0x7fc8f4196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f37fe700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8f4106ae0 0x7fc8f419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f27fc700 1 -- 172.21.15.46:0/3683011169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f4101880 msgr2=0x7fc8f4196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f27fc700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f4101880 0x7fc8f4196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f27fc700 1 -- 172.21.15.46:0/3683011169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8f4106ae0 msgr2=0x7fc8f419e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f27fc700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8f4106ae0 0x7fc8f419e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f27fc700 1 -- 172.21.15.46:0/3683011169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec009d00 con 0x7fc8f41041b0 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f27fc700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f41041b0 0x7fc8f4197350 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400d980 tx=0x7fc8e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8ebfff700 1 -- 172.21.15.46:0/3683011169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40098d0 con 0x7fc8f41041b0 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8ebfff700 1 -- 172.21.15.46:0/3683011169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8e4010460 con 0x7fc8f41041b0 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f9485700 1 -- 172.21.15.46:0/3683011169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f419ef00 con 0x7fc8f41041b0 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8ebfff700 1 -- 172.21.15.46:0/3683011169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e400f620 con 0x7fc8f41041b0 2024-03-18T20:35:29.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.465+0000 7fc8f9485700 1 -- 172.21.15.46:0/3683011169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f419f370 con 0x7fc8f41041b0 2024-03-18T20:35:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.466+0000 7fc8f9485700 1 -- 172.21.15.46:0/3683011169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f40fce30 con 0x7fc8f41041b0 2024-03-18T20:35:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.468+0000 7fc8ebfff700 1 -- 172.21.15.46:0/3683011169 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc8e4010860 con 0x7fc8f41041b0 2024-03-18T20:35:29.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.469+0000 7fc8ebfff700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc8e00a11b0 0x7fc8e00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.469+0000 7fc8f2ffd700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc8e00a11b0 0x7fc8e00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.469+0000 7fc8ebfff700 1 -- 172.21.15.46:0/3683011169 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc8e40cc700 con 0x7fc8f41041b0 2024-03-18T20:35:29.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.470+0000 7fc8f2ffd700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc8e00a11b0 0x7fc8e00a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005950 tx=0x7fc8dc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:29.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.473+0000 7fc8ebfff700 1 -- 172.21.15.46:0/3683011169 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc8e4095de0 con 0x7fc8f41041b0 2024-03-18T20:35:29.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.704+0000 7fc8f9485700 1 -- 172.21.15.46:0/3683011169 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8f40611f0 con 0x7fc8e00a11b0 2024-03-18T20:35:29.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.705+0000 7fc8ebfff700 1 -- 172.21.15.46:0/3683011169 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc8f40611f0 con 0x7fc8e00a11b0 2024-03-18T20:35:29.709 INFO:teuthology.orchestra.run.smithi046.stdout:Backend: cephadm 2024-03-18T20:35:29.709 INFO:teuthology.orchestra.run.smithi046.stdout:Available: Yes 2024-03-18T20:35:29.709 INFO:teuthology.orchestra.run.smithi046.stdout:Paused: No 2024-03-18T20:35:29.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.708+0000 7fc8f9485700 1 -- 172.21.15.46:0/3683011169 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc8e00a11b0 msgr2=0x7fc8e00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:29.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.708+0000 7fc8f9485700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc8e00a11b0 0x7fc8e00a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005950 tx=0x7fc8dc00a300 comp rx=0 tx=0).stop 2024-03-18T20:35:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.708+0000 7fc8f9485700 1 -- 172.21.15.46:0/3683011169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f41041b0 msgr2=0x7fc8f4197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.708+0000 7fc8f9485700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f41041b0 0x7fc8f4197350 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400d980 tx=0x7fc8e400dc90 comp rx=0 tx=0).stop 2024-03-18T20:35:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.708+0000 7fc8f9485700 1 -- 172.21.15.46:0/3683011169 shutdown_connections 2024-03-18T20:35:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.708+0000 7fc8f9485700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc8e00a11b0 0x7fc8e00a3670 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.708+0000 7fc8f9485700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8f4106ae0 0x7fc8f419e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.708+0000 7fc8f9485700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f4101880 0x7fc8f4196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.708+0000 7fc8f9485700 1 --2- 172.21.15.46:0/3683011169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f41041b0 0x7fc8f4197350 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.708+0000 7fc8f9485700 1 -- 172.21.15.46:0/3683011169 >> 172.21.15.46:0/3683011169 conn(0x7fc8f40fb2b0 msgr2=0x7fc8f4102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.708+0000 7fc8f9485700 1 -- 172.21.15.46:0/3683011169 shutdown_connections 2024-03-18T20:35:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:29.708+0000 7fc8f9485700 1 -- 172.21.15.46:0/3683011169 wait complete. 2024-03-18T20:35:30.064 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-18T20:35:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:30 smithi046 ceph-mon[43495]: pgmap v244: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T20:35:30.321 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:30 smithi073 ceph-mon[47384]: pgmap v244: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T20:35:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:30 smithi155 ceph-mon[48344]: pgmap v244: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T20:35:31.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:31 smithi046 ceph-mon[43495]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:31.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:31 smithi073 ceph-mon[47384]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:31.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.474+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3788839969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ab0101890 msgr2=0x7f4ab0108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:31.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.474+0000 7f4ab7fd5700 1 --2- 172.21.15.46:0/3788839969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ab0101890 0x7f4ab0108140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800b0d0 tx=0x7f4aa800b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:31.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.474+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3788839969 shutdown_connections 2024-03-18T20:35:31.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.474+0000 7f4ab7fd5700 1 --2- 172.21.15.46:0/3788839969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ab0101890 0x7f4ab0108140 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:31.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.474+0000 7f4ab7fd5700 1 --2- 172.21.15.46:0/3788839969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ab00ffcf0 0x7f4ab0100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:31.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.474+0000 7f4ab7fd5700 1 --2- 172.21.15.46:0/3788839969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ab0100ef0 0x7f4ab0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:31.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.474+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3788839969 >> 172.21.15.46:0/3788839969 conn(0x7f4ab00fb2b0 msgr2=0x7f4ab00fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:31.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.474+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3788839969 shutdown_connections 2024-03-18T20:35:31.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.474+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3788839969 wait complete. 2024-03-18T20:35:31.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.475+0000 7f4ab7fd5700 1 Processor -- start 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.475+0000 7f4ab7fd5700 1 -- start start 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.475+0000 7f4ab7fd5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ab00ffcf0 0x7f4ab0069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.475+0000 7f4ab7fd5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ab0100ef0 0x7f4ab019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.475+0000 7f4ab7fd5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ab0101890 0x7f4ab019e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.475+0000 7f4ab7fd5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab019e810 con 0x7f4ab0101890 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.475+0000 7f4ab7fd5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab019e980 con 0x7f4ab0100ef0 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.475+0000 7f4ab7fd5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab019eb40 con 0x7f4ab00ffcf0 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.475+0000 7f4ab6572700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ab0101890 0x7f4ab019e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.475+0000 7f4ab6572700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ab0101890 0x7f4ab019e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44258/0 (socket says 172.21.15.46:44258) 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.475+0000 7f4ab6572700 1 -- 172.21.15.46:0/3772405227 learned_addr learned my addr 172.21.15.46:0/3772405227 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.475+0000 7f4ab5d71700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ab00ffcf0 0x7f4ab0069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.476+0000 7f4ab5570700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ab0100ef0 0x7f4ab019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.476+0000 7f4ab6572700 1 -- 172.21.15.46:0/3772405227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ab00ffcf0 msgr2=0x7f4ab0069280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.476+0000 7f4ab6572700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ab00ffcf0 0x7f4ab0069280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.476+0000 7f4ab6572700 1 -- 172.21.15.46:0/3772405227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ab0100ef0 msgr2=0x7f4ab019ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.476+0000 7f4ab6572700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ab0100ef0 0x7f4ab019ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.476+0000 7f4ab6572700 1 -- 172.21.15.46:0/3772405227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa8009d00 con 0x7f4ab0101890 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.476+0000 7f4ab6572700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ab0101890 0x7f4ab019e070 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8007b60 tx=0x7f4aa8008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.476+0000 7f4aa6ffd700 1 -- 172.21.15.46:0/3772405227 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa800e040 con 0x7f4ab0101890 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.476+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3772405227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab019edc0 con 0x7f4ab0101890 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.476+0000 7f4aa6ffd700 1 -- 172.21.15.46:0/3772405227 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4aa800be30 con 0x7f4ab0101890 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.476+0000 7f4aa6ffd700 1 -- 172.21.15.46:0/3772405227 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa801bae0 con 0x7f4ab0101890 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.476+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3772405227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab019f220 con 0x7f4ab0101890 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.477+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3772405227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab01070b0 con 0x7f4ab0101890 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.478+0000 7f4aa6ffd700 1 -- 172.21.15.46:0/3772405227 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4aa801d670 con 0x7f4ab0101890 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.479+0000 7f4aa6ffd700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4a9c0a1230 0x7f4a9c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.479+0000 7f4aa6ffd700 1 -- 172.21.15.46:0/3772405227 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4aa80cdf40 con 0x7f4ab0101890 2024-03-18T20:35:31.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.479+0000 7f4ab5d71700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4a9c0a1230 0x7f4a9c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:31.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.480+0000 7f4ab5d71700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4a9c0a1230 0x7f4a9c0a36f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0100d50 tx=0x7f4aac00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:31.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.484+0000 7f4aa6ffd700 1 -- 172.21.15.46:0/3772405227 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4aa80977b0 con 0x7f4ab0101890 2024-03-18T20:35:31.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:31 smithi155 ceph-mon[48344]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.705+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3772405227 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ab00611f0 con 0x7f4a9c0a1230 2024-03-18T20:35:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.720+0000 7f4aa6ffd700 1 -- 172.21.15.46:0/3772405227 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4154 (secure 0 0 0) 0x7f4ab00611f0 con 0x7f4a9c0a1230 2024-03-18T20:35:31.723 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T20:35:31.723 INFO:teuthology.orchestra.run.smithi046.stdout:mds.a smithi046 running (60s) 55s ago 60s 14.2M - 18.2.2-914-g49bdc511 015bdba7dcef d0396efc47ef 2024-03-18T20:35:31.723 INFO:teuthology.orchestra.run.smithi046.stdout:mds.b smithi073 running (59s) 55s ago 58s 14.0M - 18.2.2-914-g49bdc511 015bdba7dcef 5a752c3a7d3a 2024-03-18T20:35:31.723 INFO:teuthology.orchestra.run.smithi046.stdout:mds.c smithi155 running (68s) 55s ago 68s 14.4M - 18.2.2-914-g49bdc511 015bdba7dcef f86207f0aa4a 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mds.d smithi046 running (73s) 55s ago 73s 13.3M - 18.2.2-914-g49bdc511 015bdba7dcef 49531c10947a 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mds.e smithi073 running (71s) 55s ago 71s 15.6M - 18.2.2-914-g49bdc511 015bdba7dcef 34b59532efea 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mds.f smithi155 running (70s) 55s ago 69s 14.0M - 18.2.2-914-g49bdc511 015bdba7dcef 436e410027f7 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mds.g smithi046 running (62s) 55s ago 62s 14.0M - 18.2.2-914-g49bdc511 015bdba7dcef ba4fdb9479a0 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mds.h smithi073 running (66s) 55s ago 66s 14.0M - 18.2.2-914-g49bdc511 015bdba7dcef ea7390eda069 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mds.i smithi155 running (64s) 55s ago 64s 16.4M - 18.2.2-914-g49bdc511 015bdba7dcef 00e981ff7207 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mds.j smithi046 running (75s) 55s ago 75s 16.5M - 18.2.2-914-g49bdc511 015bdba7dcef 96542963d0e3 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mds.k smithi073 running (77s) 55s ago 76s 13.3M - 18.2.2-914-g49bdc511 015bdba7dcef c5e8c5baedf0 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mds.l smithi155 running (78s) 55s ago 78s 15.7M - 18.2.2-914-g49bdc511 015bdba7dcef 348f0a5da15c 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.x smithi046 *:9283,8765 running (6m) 55s ago 6m 522M - 18.2.2-914-g49bdc511 015bdba7dcef a5e0c557f3b2 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.y smithi073 *:8443,8765 running (5m) 55s ago 5m 456M - 18.2.2-914-g49bdc511 015bdba7dcef 3f5f54dd80ec 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.z smithi155 *:8443,8765 running (5m) 55s ago 5m 455M - 18.2.2-914-g49bdc511 015bdba7dcef 9f876dfca0f2 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mon.a smithi046 running (6m) 55s ago 6m 79.3M 2048M 18.2.2-914-g49bdc511 015bdba7dcef 8f8069fe096e 2024-03-18T20:35:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:mon.b smithi073 running (5m) 55s ago 5m 54.1M 2048M 18.2.2-914-g49bdc511 015bdba7dcef bc37e67e0002 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:mon.c smithi155 running (5m) 55s ago 5m 54.8M 2048M 18.2.2-914-g49bdc511 015bdba7dcef e31a09e44936 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (4m) 55s ago 4m 54.0M 4279M 18.2.2-914-g49bdc511 015bdba7dcef 75f9dfbddb61 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi073 running (4m) 55s ago 4m 58.0M 4278M 18.2.2-914-g49bdc511 015bdba7dcef 7bacc9afe2ac 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi155 running (4m) 55s ago 4m 53.2M 4278M 18.2.2-914-g49bdc511 015bdba7dcef 72863442920f 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi046 running (3m) 55s ago 3m 49.4M 4279M 18.2.2-914-g49bdc511 015bdba7dcef 3dc11007e042 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi073 running (3m) 55s ago 3m 49.0M 4278M 18.2.2-914-g49bdc511 015bdba7dcef 77422b67a9ce 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi155 running (3m) 55s ago 3m 50.2M 4278M 18.2.2-914-g49bdc511 015bdba7dcef 91a6bfb33549 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 smithi046 running (3m) 55s ago 3m 45.8M 4279M 18.2.2-914-g49bdc511 015bdba7dcef 9343f1e7ba87 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 smithi073 running (2m) 55s ago 2m 53.2M 4278M 18.2.2-914-g49bdc511 015bdba7dcef 0ec96761b4a0 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 smithi155 running (2m) 55s ago 2m 48.6M 4278M 18.2.2-914-g49bdc511 015bdba7dcef c4434cf33336 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 smithi046 running (2m) 55s ago 2m 48.4M 4279M 18.2.2-914-g49bdc511 015bdba7dcef 5ab165b5664b 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 smithi073 running (110s) 55s ago 110s 48.3M 4278M 18.2.2-914-g49bdc511 015bdba7dcef 36f687bb57ac 2024-03-18T20:35:31.725 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 smithi155 running (93s) 55s ago 92s 44.3M 4278M 18.2.2-914-g49bdc511 015bdba7dcef 641428438bcf 2024-03-18T20:35:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.722+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3772405227 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4a9c0a1230 msgr2=0x7f4a9c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.722+0000 7f4ab7fd5700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4a9c0a1230 0x7f4a9c0a36f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0100d50 tx=0x7f4aac00a3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.722+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3772405227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ab0101890 msgr2=0x7f4ab019e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.722+0000 7f4ab7fd5700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ab0101890 0x7f4ab019e070 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8007b60 tx=0x7f4aa8008f10 comp rx=0 tx=0).stop 2024-03-18T20:35:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.722+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3772405227 shutdown_connections 2024-03-18T20:35:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.722+0000 7f4ab7fd5700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4a9c0a1230 0x7f4a9c0a36f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:31.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.722+0000 7f4ab7fd5700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ab0101890 0x7f4ab019e070 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.722+0000 7f4ab7fd5700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ab00ffcf0 0x7f4ab0069280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.722+0000 7f4ab7fd5700 1 --2- 172.21.15.46:0/3772405227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ab0100ef0 0x7f4ab019ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.722+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3772405227 >> 172.21.15.46:0/3772405227 conn(0x7f4ab00fb2b0 msgr2=0x7f4ab01041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.722+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3772405227 shutdown_connections 2024-03-18T20:35:31.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:31.722+0000 7f4ab7fd5700 1 -- 172.21.15.46:0/3772405227 wait complete. 2024-03-18T20:35:32.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-18T20:35:32.350 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:32.451 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:32 smithi046 ceph-mon[43495]: pgmap v245: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T20:35:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:32 smithi073 ceph-mon[47384]: pgmap v245: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T20:35:32.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:32 smithi155 ceph-mon[48344]: pgmap v245: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T20:35:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.317+0000 7f10ab1e5700 1 -- 172.21.15.46:0/1370930294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a4101f60 msgr2=0x7f10a41023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.317+0000 7f10ab1e5700 1 --2- 172.21.15.46:0/1370930294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a4101f60 0x7f10a41023e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1098009990 tx=0x7f1098009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.317+0000 7f10ab1e5700 1 -- 172.21.15.46:0/1370930294 shutdown_connections 2024-03-18T20:35:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.317+0000 7f10ab1e5700 1 --2- 172.21.15.46:0/1370930294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4102920 0x7f10a4109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.317+0000 7f10ab1e5700 1 --2- 172.21.15.46:0/1370930294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10a4100e00 0x7f10a4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.317+0000 7f10ab1e5700 1 --2- 172.21.15.46:0/1370930294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a4101f60 0x7f10a41023e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.317+0000 7f10ab1e5700 1 -- 172.21.15.46:0/1370930294 >> 172.21.15.46:0/1370930294 conn(0x7f10a40fc3c0 msgr2=0x7f10a40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.317+0000 7f10ab1e5700 1 -- 172.21.15.46:0/1370930294 shutdown_connections 2024-03-18T20:35:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.317+0000 7f10ab1e5700 1 -- 172.21.15.46:0/1370930294 wait complete. 2024-03-18T20:35:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.317+0000 7f10ab1e5700 1 Processor -- start 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.317+0000 7f10ab1e5700 1 -- start start 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10ab1e5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4100e00 0x7f10a4196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10ab1e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a4101f60 0x7f10a41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10ab1e5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10a4102920 0x7f10a419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10ab1e5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a419e7e0 con 0x7f10a4101f60 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10ab1e5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a419e920 con 0x7f10a4102920 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10ab1e5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a419eae0 con 0x7f10a4100e00 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a4101f60 0x7f10a41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a4101f60 0x7f10a41971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44270/0 (socket says 172.21.15.46:44270) 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a3fff700 1 -- 172.21.15.46:0/2037421696 learned_addr learned my addr 172.21.15.46:0/2037421696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a8f81700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4100e00 0x7f10a4196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a9782700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10a4102920 0x7f10a419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a3fff700 1 -- 172.21.15.46:0/2037421696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4100e00 msgr2=0x7f10a4196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a3fff700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4100e00 0x7f10a4196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a3fff700 1 -- 172.21.15.46:0/2037421696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10a4102920 msgr2=0x7f10a419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a3fff700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10a4102920 0x7f10a419e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a3fff700 1 -- 172.21.15.46:0/2037421696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10980095f0 con 0x7f10a4101f60 2024-03-18T20:35:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a9782700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10a4102920 0x7f10a419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a3fff700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a4101f60 0x7f10a41971f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f10980098e0 tx=0x7f1098005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.318+0000 7f10a1ffb700 1 -- 172.21.15.46:0/2037421696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109801d070 con 0x7f10a4101f60 2024-03-18T20:35:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.319+0000 7f10a1ffb700 1 -- 172.21.15.46:0/2037421696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1098003fe0 con 0x7f10a4101f60 2024-03-18T20:35:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.319+0000 7f10a1ffb700 1 -- 172.21.15.46:0/2037421696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098022e00 con 0x7f10a4101f60 2024-03-18T20:35:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.319+0000 7f10ab1e5700 1 -- 172.21.15.46:0/2037421696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a419ed60 con 0x7f10a4101f60 2024-03-18T20:35:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.319+0000 7f10ab1e5700 1 -- 172.21.15.46:0/2037421696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a419f120 con 0x7f10a4101f60 2024-03-18T20:35:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.320+0000 7f10ab1e5700 1 -- 172.21.15.46:0/2037421696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a4108280 con 0x7f10a4101f60 2024-03-18T20:35:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.321+0000 7f10a1ffb700 1 -- 172.21.15.46:0/2037421696 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1098022500 con 0x7f10a4101f60 2024-03-18T20:35:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.322+0000 7f10a1ffb700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f108c0a1230 0x7f108c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.322+0000 7f10a8f81700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f108c0a1230 0x7f108c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.322+0000 7f10a1ffb700 1 -- 172.21.15.46:0/2037421696 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f10980ce3d0 con 0x7f10a4101f60 2024-03-18T20:35:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.322+0000 7f10a8f81700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f108c0a1230 0x7f108c0a36f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1094005950 tx=0x7f109400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.325+0000 7f10a1ffb700 1 -- 172.21.15.46:0/2037421696 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1098097c40 con 0x7f10a4101f60 2024-03-18T20:35:33.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:33 smithi073 ceph-mon[47384]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:33 smithi155 ceph-mon[48344]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:33.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:33 smithi046 ceph-mon[43495]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:33.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.548+0000 7f10ab1e5700 1 -- 172.21.15.46:0/2037421696 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f10a40611f0 con 0x7f108c0a1230 2024-03-18T20:35:33.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.554+0000 7f10a1ffb700 1 -- 172.21.15.46:0/2037421696 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f10a40611f0 con 0x7f108c0a1230 2024-03-18T20:35:33.558 INFO:teuthology.orchestra.run.smithi046.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-18T20:35:33.558 INFO:teuthology.orchestra.run.smithi046.stdout:mds.all 12/12 57s ago 82s smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi155=c;smithi155=f;smithi155=i;smithi155=l;count:12 2024-03-18T20:35:33.558 INFO:teuthology.orchestra.run.smithi046.stdout:mgr 3/3 57s ago 5m smithi046=x;smithi073=y;smithi155=z;count:3 2024-03-18T20:35:33.558 INFO:teuthology.orchestra.run.smithi046.stdout:mon 3/3 57s ago 5m smithi046:172.21.15.46=a;smithi073:172.21.15.73=b;smithi155:172.21.15.155=c;count:3 2024-03-18T20:35:33.558 INFO:teuthology.orchestra.run.smithi046.stdout:osd 12 57s ago - 2024-03-18T20:35:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.557+0000 7f10ab1e5700 1 -- 172.21.15.46:0/2037421696 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f108c0a1230 msgr2=0x7f108c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.557+0000 7f10ab1e5700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f108c0a1230 0x7f108c0a36f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1094005950 tx=0x7f109400a300 comp rx=0 tx=0).stop 2024-03-18T20:35:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.557+0000 7f10ab1e5700 1 -- 172.21.15.46:0/2037421696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a4101f60 msgr2=0x7f10a41971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.557+0000 7f10ab1e5700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a4101f60 0x7f10a41971f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f10980098e0 tx=0x7f1098005f10 comp rx=0 tx=0).stop 2024-03-18T20:35:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.557+0000 7f10ab1e5700 1 -- 172.21.15.46:0/2037421696 shutdown_connections 2024-03-18T20:35:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.557+0000 7f10ab1e5700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f108c0a1230 0x7f108c0a36f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.557+0000 7f10ab1e5700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10a4102920 0x7f10a419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:33.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.557+0000 7f10ab1e5700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a4100e00 0x7f10a4196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.557+0000 7f10ab1e5700 1 --2- 172.21.15.46:0/2037421696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10a4101f60 0x7f10a41971f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.557+0000 7f10ab1e5700 1 -- 172.21.15.46:0/2037421696 >> 172.21.15.46:0/2037421696 conn(0x7f10a40fc3c0 msgr2=0x7f10a41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.557+0000 7f10ab1e5700 1 -- 172.21.15.46:0/2037421696 shutdown_connections 2024-03-18T20:35:33.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:33.557+0000 7f10ab1e5700 1 -- 172.21.15.46:0/2037421696 wait complete. 2024-03-18T20:35:33.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-18T20:35:34.197 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:34.292 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:34 smithi046 ceph-mon[43495]: pgmap v246: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-18T20:35:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:34 smithi073 ceph-mon[47384]: pgmap v246: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-18T20:35:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:34 smithi155 ceph-mon[48344]: pgmap v246: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-18T20:35:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.172+0000 7efdbd713700 1 -- 172.21.15.46:0/2830238324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdb81041d0 msgr2=0x7efdb81065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.172+0000 7efdbd713700 1 --2- 172.21.15.46:0/2830238324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdb81041d0 0x7efdb81065c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efda0009990 tx=0x7efda0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.172+0000 7efdbd713700 1 -- 172.21.15.46:0/2830238324 shutdown_connections 2024-03-18T20:35:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.172+0000 7efdbd713700 1 --2- 172.21.15.46:0/2830238324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdb8106b00 0x7efdb8108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.172+0000 7efdbd713700 1 --2- 172.21.15.46:0/2830238324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdb81018a0 0x7efdb8103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.172+0000 7efdbd713700 1 --2- 172.21.15.46:0/2830238324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdb81041d0 0x7efdb81065c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.172+0000 7efdbd713700 1 -- 172.21.15.46:0/2830238324 >> 172.21.15.46:0/2830238324 conn(0x7efdb80fb2d0 msgr2=0x7efdb80fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.172+0000 7efdbd713700 1 -- 172.21.15.46:0/2830238324 shutdown_connections 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.172+0000 7efdbd713700 1 -- 172.21.15.46:0/2830238324 wait complete. 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.173+0000 7efdbd713700 1 Processor -- start 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.173+0000 7efdbd713700 1 -- start start 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.173+0000 7efdbd713700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdb81018a0 0x7efdb81011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.173+0000 7efdbd713700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdb81041d0 0x7efdb80ff850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.173+0000 7efdbd713700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdb8106b00 0x7efdb80ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.173+0000 7efdbd713700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb8100530 con 0x7efdb8106b00 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.173+0000 7efdbd713700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb81006a0 con 0x7efdb81041d0 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.173+0000 7efdbd713700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb81a0d70 con 0x7efdb81018a0 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdb8106b00 0x7efdb80ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdb8106b00 0x7efdb80ffd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44300/0 (socket says 172.21.15.46:44300) 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb77fe700 1 -- 172.21.15.46:0/2922018756 learned_addr learned my addr 172.21.15.46:0/2922018756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:35.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb6ffd700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdb81018a0 0x7efdb81011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdae7fc700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdb81041d0 0x7efdb80ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb77fe700 1 -- 172.21.15.46:0/2922018756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdb81018a0 msgr2=0x7efdb81011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb77fe700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdb81018a0 0x7efdb81011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb77fe700 1 -- 172.21.15.46:0/2922018756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdb81041d0 msgr2=0x7efdb80ff850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb77fe700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdb81041d0 0x7efdb80ff850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb77fe700 1 -- 172.21.15.46:0/2922018756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda00095f0 con 0x7efdb8106b00 2024-03-18T20:35:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdae7fc700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdb81041d0 0x7efdb80ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb77fe700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdb8106b00 0x7efdb80ffd90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7efdb0007d40 tx=0x7efdb000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb4ff9700 1 -- 172.21.15.46:0/2922018756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb000ad40 con 0x7efdb8106b00 2024-03-18T20:35:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb4ff9700 1 -- 172.21.15.46:0/2922018756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb0007e90 con 0x7efdb8106b00 2024-03-18T20:35:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdb4ff9700 1 -- 172.21.15.46:0/2922018756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb00116f0 con 0x7efdb8106b00 2024-03-18T20:35:35.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.174+0000 7efdbd713700 1 -- 172.21.15.46:0/2922018756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb81a0fa0 con 0x7efdb8106b00 2024-03-18T20:35:35.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.175+0000 7efdbd713700 1 -- 172.21.15.46:0/2922018756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb81a1420 con 0x7efdb8106b00 2024-03-18T20:35:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.176+0000 7efdbd713700 1 -- 172.21.15.46:0/2922018756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb80fce50 con 0x7efdb8106b00 2024-03-18T20:35:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.177+0000 7efdb4ff9700 1 -- 172.21.15.46:0/2922018756 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7efdb0012420 con 0x7efdb8106b00 2024-03-18T20:35:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.177+0000 7efdb4ff9700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efda40a1230 0x7efda40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.178+0000 7efdb6ffd700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efda40a1230 0x7efda40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.178+0000 7efdb4ff9700 1 -- 172.21.15.46:0/2922018756 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efdb00cca60 con 0x7efdb8106b00 2024-03-18T20:35:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.178+0000 7efdb6ffd700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efda40a1230 0x7efda40a36f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efda8005950 tx=0x7efda800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.182+0000 7efdb4ff9700 1 -- 172.21.15.46:0/2922018756 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efdb0096250 con 0x7efdb8106b00 2024-03-18T20:35:35.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:35 smithi046 ceph-mon[43495]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:35.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.401+0000 7efdbd713700 1 -- 172.21.15.46:0/2922018756 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7efdb80611f0 con 0x7efda40a1230 2024-03-18T20:35:35.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.402+0000 7efdb4ff9700 1 -- 172.21.15.46:0/2922018756 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7efdb80611f0 con 0x7efda40a1230 2024-03-18T20:35:35.406 INFO:teuthology.orchestra.run.smithi046.stdout:HOST ADDR LABELS STATUS 2024-03-18T20:35:35.406 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 172.21.15.46 2024-03-18T20:35:35.406 INFO:teuthology.orchestra.run.smithi046.stdout:smithi073 172.21.15.73 2024-03-18T20:35:35.406 INFO:teuthology.orchestra.run.smithi046.stdout:smithi155 172.21.15.155 2024-03-18T20:35:35.406 INFO:teuthology.orchestra.run.smithi046.stdout:3 hosts in cluster 2024-03-18T20:35:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.404+0000 7efdbd713700 1 -- 172.21.15.46:0/2922018756 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efda40a1230 msgr2=0x7efda40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.404+0000 7efdbd713700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efda40a1230 0x7efda40a36f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efda8005950 tx=0x7efda800b410 comp rx=0 tx=0).stop 2024-03-18T20:35:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.404+0000 7efdbd713700 1 -- 172.21.15.46:0/2922018756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdb8106b00 msgr2=0x7efdb80ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.404+0000 7efdbd713700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdb8106b00 0x7efdb80ffd90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7efdb0007d40 tx=0x7efdb000a340 comp rx=0 tx=0).stop 2024-03-18T20:35:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.405+0000 7efdbd713700 1 -- 172.21.15.46:0/2922018756 shutdown_connections 2024-03-18T20:35:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.405+0000 7efdbd713700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7efda40a1230 0x7efda40a36f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:35.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.405+0000 7efdbd713700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdb8106b00 0x7efdb80ffd90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.405+0000 7efdbd713700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdb81018a0 0x7efdb81011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.405+0000 7efdbd713700 1 --2- 172.21.15.46:0/2922018756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdb81041d0 0x7efdb80ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.405+0000 7efdbd713700 1 -- 172.21.15.46:0/2922018756 >> 172.21.15.46:0/2922018756 conn(0x7efdb80fb2d0 msgr2=0x7efdb81021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.405+0000 7efdbd713700 1 -- 172.21.15.46:0/2922018756 shutdown_connections 2024-03-18T20:35:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:35.405+0000 7efdbd713700 1 -- 172.21.15.46:0/2922018756 wait complete. 2024-03-18T20:35:35.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:35 smithi073 ceph-mon[47384]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:35.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:35 smithi155 ceph-mon[48344]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:35.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-18T20:35:36.072 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:36.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:36 smithi073 ceph-mon[47384]: pgmap v247: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T20:35:36.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:36 smithi073 ceph-mon[47384]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:36.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:36 smithi046 ceph-mon[43495]: pgmap v247: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T20:35:36.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:36 smithi046 ceph-mon[43495]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:36.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:36 smithi155 ceph-mon[48344]: pgmap v247: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-18T20:35:36.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:36 smithi155 ceph-mon[48344]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:37.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.136+0000 7f339f93c700 1 -- 172.21.15.46:0/172336867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3398100ef0 msgr2=0x7f3398101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:37.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.136+0000 7f339f93c700 1 --2- 172.21.15.46:0/172336867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3398100ef0 0x7f3398101350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f33880099c0 tx=0x7f3388009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:37.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.136+0000 7f339f93c700 1 -- 172.21.15.46:0/172336867 shutdown_connections 2024-03-18T20:35:37.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.136+0000 7f339f93c700 1 --2- 172.21.15.46:0/172336867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3398101890 0x7f3398108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:37.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.136+0000 7f339f93c700 1 --2- 172.21.15.46:0/172336867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33980ffcf0 0x7f3398100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:37.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.136+0000 7f339f93c700 1 --2- 172.21.15.46:0/172336867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3398100ef0 0x7f3398101350 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:37.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.136+0000 7f339f93c700 1 -- 172.21.15.46:0/172336867 >> 172.21.15.46:0/172336867 conn(0x7f33980fb2b0 msgr2=0x7f33980fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:37.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.136+0000 7f339f93c700 1 -- 172.21.15.46:0/172336867 shutdown_connections 2024-03-18T20:35:37.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.136+0000 7f339f93c700 1 -- 172.21.15.46:0/172336867 wait complete. 2024-03-18T20:35:37.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339f93c700 1 Processor -- start 2024-03-18T20:35:37.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339f93c700 1 -- start start 2024-03-18T20:35:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339f93c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33980ffcf0 0x7f3398103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339f93c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3398100ef0 0x7f33981022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339f93c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3398101890 0x7f3398102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339f93c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3398102fb0 con 0x7f3398100ef0 2024-03-18T20:35:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339f93c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3398103120 con 0x7f3398101890 2024-03-18T20:35:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339f93c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33981a0d70 con 0x7f33980ffcf0 2024-03-18T20:35:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339ced7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3398100ef0 0x7f33981022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339ced7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3398100ef0 0x7f33981022d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48498/0 (socket says 172.21.15.46:48498) 2024-03-18T20:35:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339d6d8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33980ffcf0 0x7f3398103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339ced7700 1 -- 172.21.15.46:0/868457315 learned_addr learned my addr 172.21.15.46:0/868457315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.137+0000 7f339ded9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3398101890 0x7f3398102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:37.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.138+0000 7f339ced7700 1 -- 172.21.15.46:0/868457315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33980ffcf0 msgr2=0x7f3398103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:37.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.138+0000 7f339ced7700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33980ffcf0 0x7f3398103c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:37.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.138+0000 7f339ced7700 1 -- 172.21.15.46:0/868457315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3398101890 msgr2=0x7f3398102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:37.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.138+0000 7f339ced7700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3398101890 0x7f3398102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:37.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.138+0000 7f339ced7700 1 -- 172.21.15.46:0/868457315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33880095f0 con 0x7f3398100ef0 2024-03-18T20:35:37.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.138+0000 7f339ded9700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3398101890 0x7f3398102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:37.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.138+0000 7f339ced7700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3398100ef0 0x7f33981022d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3388003730 tx=0x7f338800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:37.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.138+0000 7f338e7fc700 1 -- 172.21.15.46:0/868457315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338801d070 con 0x7f3398100ef0 2024-03-18T20:35:37.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.138+0000 7f338e7fc700 1 -- 172.21.15.46:0/868457315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33880059a0 con 0x7f3398100ef0 2024-03-18T20:35:37.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.138+0000 7f339f93c700 1 -- 172.21.15.46:0/868457315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33981a0fa0 con 0x7f3398100ef0 2024-03-18T20:35:37.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.138+0000 7f338e7fc700 1 -- 172.21.15.46:0/868457315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388004a00 con 0x7f3398100ef0 2024-03-18T20:35:37.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.138+0000 7f339f93c700 1 -- 172.21.15.46:0/868457315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33981a13b0 con 0x7f3398100ef0 2024-03-18T20:35:37.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.139+0000 7f339f93c700 1 -- 172.21.15.46:0/868457315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339804ea80 con 0x7f3398100ef0 2024-03-18T20:35:37.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.141+0000 7f338e7fc700 1 -- 172.21.15.46:0/868457315 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3388005210 con 0x7f3398100ef0 2024-03-18T20:35:37.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.141+0000 7f338e7fc700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f33840a11b0 0x7f33840a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:37.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.141+0000 7f338e7fc700 1 -- 172.21.15.46:0/868457315 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f33880cd5f0 con 0x7f3398100ef0 2024-03-18T20:35:37.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.141+0000 7f339d6d8700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f33840a11b0 0x7f33840a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:37.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.142+0000 7f339d6d8700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f33840a11b0 0x7f33840a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3398100d50 tx=0x7f3394009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:37.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.146+0000 7f338e7fc700 1 -- 172.21.15.46:0/868457315 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3388096cc0 con 0x7f3398100ef0 2024-03-18T20:35:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:35:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-18T20:35:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-18T20:35:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-18T20:35:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-03-18T20:35:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-03-18T20:35:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-03-18T20:35:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-18T20:35:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:37 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-18T20:35:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.386+0000 7f339f93c700 1 -- 172.21.15.46:0/868457315 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3398068e90 con 0x7f33840a11b0 2024-03-18T20:35:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.388+0000 7f338e7fc700 1 -- 172.21.15.46:0/868457315 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f3398068e90 con 0x7f33840a11b0 2024-03-18T20:35:37.392 INFO:teuthology.orchestra.run.smithi046.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-18T20:35:37.392 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7512009A480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-03-18T20:35:37.392 INFO:teuthology.orchestra.run.smithi046.stdout:smithi073 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040115400BGN 372G No 111s ago Has a FileSystem, LVM detected 2024-03-18T20:35:37.392 INFO:teuthology.orchestra.run.smithi046.stdout:smithi155 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GW400BGN 372G No 93s ago Has a FileSystem, LVM detected 2024-03-18T20:35:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.390+0000 7f339f93c700 1 -- 172.21.15.46:0/868457315 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f33840a11b0 msgr2=0x7f33840a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.390+0000 7f339f93c700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f33840a11b0 0x7f33840a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3398100d50 tx=0x7f3394009260 comp rx=0 tx=0).stop 2024-03-18T20:35:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.390+0000 7f339f93c700 1 -- 172.21.15.46:0/868457315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3398100ef0 msgr2=0x7f33981022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.390+0000 7f339f93c700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3398100ef0 0x7f33981022d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3388003730 tx=0x7f338800f670 comp rx=0 tx=0).stop 2024-03-18T20:35:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.390+0000 7f339f93c700 1 -- 172.21.15.46:0/868457315 shutdown_connections 2024-03-18T20:35:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.390+0000 7f339f93c700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f33840a11b0 0x7f33840a3670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.390+0000 7f339f93c700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3398101890 0x7f3398102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.390+0000 7f339f93c700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33980ffcf0 0x7f3398103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.390+0000 7f339f93c700 1 --2- 172.21.15.46:0/868457315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3398100ef0 0x7f33981022d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.390+0000 7f339f93c700 1 -- 172.21.15.46:0/868457315 >> 172.21.15.46:0/868457315 conn(0x7f33980fb2b0 msgr2=0x7f3398068780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.390+0000 7f339f93c700 1 -- 172.21.15.46:0/868457315 shutdown_connections 2024-03-18T20:35:37.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:37.390+0000 7f339f93c700 1 -- 172.21.15.46:0/868457315 wait complete. 2024-03-18T20:35:37.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:37 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:35:37.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:37 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-18T20:35:37.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:37 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-18T20:35:37.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:37 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-18T20:35:37.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:37 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-03-18T20:35:37.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:37 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-03-18T20:35:37.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:37 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-03-18T20:35:37.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:37 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-18T20:35:37.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:37 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-18T20:35:37.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:37 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:35:37.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:37 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-18T20:35:37.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:37 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-18T20:35:37.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:37 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-03-18T20:35:37.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:37 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-03-18T20:35:37.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:37 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-03-18T20:35:37.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:37 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-03-18T20:35:37.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:37 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-18T20:35:37.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:37 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-18T20:35:37.761 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T20:35:37.771 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-03-18T20:35:37.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-18T20:35:38.035 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:38.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: pgmap v248: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T20:35:38.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-18T20:35:38.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-18T20:35:38.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-18T20:35:38.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-03-18T20:35:38.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-03-18T20:35:38.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-03-18T20:35:38.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-18T20:35:38.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-18T20:35:38.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: osdmap e79: 12 total, 12 up, 12 in 2024-03-18T20:35:38.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:38.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.434 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.435 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:35:38.435 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:35:38.435 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:38 smithi046 ceph-mon[43495]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: pgmap v248: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: osdmap e79: 12 total, 12 up, 12 in 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:35:38.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:35:38.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:38 smithi073 ceph-mon[47384]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: pgmap v248: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T20:35:38.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-18T20:35:38.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-18T20:35:38.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: osdmap e79: 12 total, 12 up, 12 in 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:35:38.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:35:38.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:38 smithi155 ceph-mon[48344]: from='mgr.14152 172.21.15.46:0/129926516' entity='mgr.x' 2024-03-18T20:35:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.980+0000 7f237d774700 1 -- 172.21.15.46:0/1133776476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378100e00 msgr2=0x7f2378101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.980+0000 7f237d774700 1 --2- 172.21.15.46:0/1133776476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378100e00 0x7f2378101220 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2368009990 tx=0x7f2368009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.980+0000 7f237d774700 1 -- 172.21.15.46:0/1133776476 shutdown_connections 2024-03-18T20:35:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.980+0000 7f237d774700 1 --2- 172.21.15.46:0/1133776476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378100e00 0x7f2378101220 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.980+0000 7f237d774700 1 --2- 172.21.15.46:0/1133776476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2378102920 0x7f2378109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.980+0000 7f237d774700 1 --2- 172.21.15.46:0/1133776476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2378101f60 0x7f23781023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.980+0000 7f237d774700 1 -- 172.21.15.46:0/1133776476 >> 172.21.15.46:0/1133776476 conn(0x7f23780fc3c0 msgr2=0x7f23780fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.980+0000 7f237d774700 1 -- 172.21.15.46:0/1133776476 shutdown_connections 2024-03-18T20:35:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.980+0000 7f237d774700 1 -- 172.21.15.46:0/1133776476 wait complete. 2024-03-18T20:35:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.981+0000 7f237d774700 1 Processor -- start 2024-03-18T20:35:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.981+0000 7f237d774700 1 -- start start 2024-03-18T20:35:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.981+0000 7f237d774700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2378100e00 0x7f2378074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.981+0000 7f237d774700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2378101f60 0x7f23780731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.981+0000 7f237d774700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378102920 0x7f23780736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.981+0000 7f237d774700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2378073e80 con 0x7f2378102920 2024-03-18T20:35:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.981+0000 7f237d774700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2378073ff0 con 0x7f2378101f60 2024-03-18T20:35:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.981+0000 7f237d774700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23781a0d60 con 0x7f2378100e00 2024-03-18T20:35:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f236e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2378101f60 0x7f23780731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f2376ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2378100e00 0x7f2378074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f23777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378102920 0x7f23780736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:38.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f236e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2378101f60 0x7f23780731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55834/0 (socket says 172.21.15.46:55834) 2024-03-18T20:35:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f2376ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2378100e00 0x7f2378074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43942/0 (socket says 172.21.15.46:43942) 2024-03-18T20:35:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f23777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378102920 0x7f23780736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48514/0 (socket says 172.21.15.46:48514) 2024-03-18T20:35:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f236e7fc700 1 -- 172.21.15.46:0/4071393010 learned_addr learned my addr 172.21.15.46:0/4071393010 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f236e7fc700 1 -- 172.21.15.46:0/4071393010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2378100e00 msgr2=0x7f2378074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f236e7fc700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2378100e00 0x7f2378074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f236e7fc700 1 -- 172.21.15.46:0/4071393010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378102920 msgr2=0x7f23780736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f236e7fc700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378102920 0x7f23780736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f236e7fc700 1 -- 172.21.15.46:0/4071393010 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23680095f0 con 0x7f2378101f60 2024-03-18T20:35:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f23777fe700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378102920 0x7f23780736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f2376ffd700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2378100e00 0x7f2378074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:38.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f236e7fc700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2378101f60 0x7f23780731a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f236000d950 tx=0x7f236000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f2374ff9700 1 -- 172.21.15.46:0/4071393010 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236000f890 con 0x7f2378101f60 2024-03-18T20:35:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f2374ff9700 1 -- 172.21.15.46:0/4071393010 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2360010460 con 0x7f2378101f60 2024-03-18T20:35:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f237d774700 1 -- 172.21.15.46:0/4071393010 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23781a0ff0 con 0x7f2378101f60 2024-03-18T20:35:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f2374ff9700 1 -- 172.21.15.46:0/4071393010 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236000e610 con 0x7f2378101f60 2024-03-18T20:35:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.982+0000 7f237d774700 1 -- 172.21.15.46:0/4071393010 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23781a1460 con 0x7f2378101f60 2024-03-18T20:35:38.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.984+0000 7f237d774700 1 -- 172.21.15.46:0/4071393010 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23781082d0 con 0x7f2378101f60 2024-03-18T20:35:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.985+0000 7f2374ff9700 1 -- 172.21.15.46:0/4071393010 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f23600108d0 con 0x7f2378101f60 2024-03-18T20:35:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.985+0000 7f2374ff9700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f23640a11b0 0x7f23640a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.986+0000 7f2376ffd700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f23640a11b0 0x7f23640a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.986+0000 7f2374ff9700 1 -- 172.21.15.46:0/4071393010 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23600cc7d0 con 0x7f2378101f60 2024-03-18T20:35:38.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.986+0000 7f2376ffd700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f23640a11b0 0x7f23640a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f236800b5c0 tx=0x7f2368003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:38.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:38.989+0000 7f2374ff9700 1 -- 172.21.15.46:0/4071393010 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2360095e10 con 0x7f2378101f60 2024-03-18T20:35:39.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.247+0000 7f237d774700 1 -- 172.21.15.46:0/4071393010 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f237804ea80 con 0x7f2378101f60 2024-03-18T20:35:39.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.247+0000 7f2374ff9700 1 -- 172.21.15.46:0/4071393010 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2921 (secure 0 0 0) 0x7f2360020b80 con 0x7f2378101f60 2024-03-18T20:35:39.251 INFO:teuthology.orchestra.run.smithi046.stdout:e26 2024-03-18T20:35:39.251 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T20:35:39.251 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T20:35:39.251 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-03-18T20:35:39.251 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:39.251 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-03-18T20:35:39.251 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-03-18T20:35:39.251 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 26 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-03-18T20:34:38.785094+0000 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-03-18T20:34:45.796438+0000 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 300 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-18T20:35:39.252 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 5 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1,2,3,4 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=24536,1=24521,2=24514,3=24506,4=24446} 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data disabled 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 0 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b{0:24536} state up:active seq 3 addr [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c{0:24499} state up:standby-replay seq 1 addr [v2:172.21.15.155:6836/1686098166,v1:172.21.15.155:6837/1686098166] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T20:35:39.253 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.g{1:24521} state up:active seq 5 addr [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.f{1:24487} state up:standby-replay seq 1 addr [v2:172.21.15.155:6834/2573112997,v1:172.21.15.155:6835/2573112997] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.i{2:24514} state up:active seq 5 addr [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.e{2:24479} state up:standby-replay seq 1 addr [v2:172.21.15.73:6834/2049469125,v1:172.21.15.73:6835/2049469125] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.h{3:24506} state up:active seq 6 addr [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.l{3:24448} state up:standby-replay seq 1 addr [v2:172.21.15.155:6832/4207499009,v1:172.21.15.155:6833/4207499009] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.k{4:24446} state up:active seq 9 addr [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a{4:14628} state up:standby-replay seq 1 addr [v2:172.21.15.46:6840/3470614253,v1:172.21.15.46:6841/3470614253] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.46:6834/231081000,v1:172.21.15.46:6835/231081000] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T20:35:39.254 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.46:6836/114990598,v1:172.21.15.46:6837/114990598] compat {c=[1],r=[1],i=[7ff]}] 2024-03-18T20:35:39.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.249+0000 7f237d774700 1 -- 172.21.15.46:0/4071393010 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f23640a11b0 msgr2=0x7f23640a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:39.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.250+0000 7f237d774700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f23640a11b0 0x7f23640a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f236800b5c0 tx=0x7f2368003790 comp rx=0 tx=0).stop 2024-03-18T20:35:39.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.250+0000 7f237d774700 1 -- 172.21.15.46:0/4071393010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2378101f60 msgr2=0x7f23780731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:39.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.250+0000 7f237d774700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2378101f60 0x7f23780731a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f236000d950 tx=0x7f236000dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:39.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.250+0000 7f237d774700 1 -- 172.21.15.46:0/4071393010 shutdown_connections 2024-03-18T20:35:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.250+0000 7f237d774700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f23640a11b0 0x7f23640a3670 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.250+0000 7f237d774700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378102920 0x7f23780736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.250+0000 7f237d774700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2378100e00 0x7f2378074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.250+0000 7f237d774700 1 --2- 172.21.15.46:0/4071393010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2378101f60 0x7f23780731a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.250+0000 7f237d774700 1 -- 172.21.15.46:0/4071393010 >> 172.21.15.46:0/4071393010 conn(0x7f23780fc3c0 msgr2=0x7f2378105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.250+0000 7f237d774700 1 -- 172.21.15.46:0/4071393010 shutdown_connections 2024-03-18T20:35:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:39.250+0000 7f237d774700 1 -- 172.21.15.46:0/4071393010 wait complete. 2024-03-18T20:35:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-03-18T20:35:39.424 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:39 smithi046 ceph-mon[43495]: osdmap e80: 12 total, 12 up, 12 in 2024-03-18T20:35:39.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:39 smithi073 ceph-mon[47384]: osdmap e80: 12 total, 12 up, 12 in 2024-03-18T20:35:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:39 smithi155 ceph-mon[48344]: osdmap e80: 12 total, 12 up, 12 in 2024-03-18T20:35:39.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-18T20:35:39.891 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:40.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:40 smithi046 ceph-mon[43495]: pgmap v251: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-18T20:35:40.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:40 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/4071393010' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T20:35:40.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:40 smithi073 ceph-mon[47384]: pgmap v251: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-18T20:35:40.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:40 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/4071393010' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T20:35:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:40 smithi155 ceph-mon[48344]: pgmap v251: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-18T20:35:40.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:40 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/4071393010' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T20:35:40.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.856+0000 7f70e4144700 1 -- 172.21.15.46:0/1528469562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc101f60 msgr2=0x7f70dc1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:40.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.856+0000 7f70e4144700 1 --2- 172.21.15.46:0/1528469562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc101f60 0x7f70dc1023e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009990 tx=0x7f70cc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:40.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.856+0000 7f70e4144700 1 -- 172.21.15.46:0/1528469562 shutdown_connections 2024-03-18T20:35:40.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.856+0000 7f70e4144700 1 --2- 172.21.15.46:0/1528469562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc102920 0x7f70dc109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:40.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.856+0000 7f70e4144700 1 --2- 172.21.15.46:0/1528469562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc100e00 0x7f70dc101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:40.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.856+0000 7f70e4144700 1 --2- 172.21.15.46:0/1528469562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc101f60 0x7f70dc1023e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:40.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.856+0000 7f70e4144700 1 -- 172.21.15.46:0/1528469562 >> 172.21.15.46:0/1528469562 conn(0x7f70dc0fc3c0 msgr2=0x7f70dc0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:40.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.856+0000 7f70e4144700 1 -- 172.21.15.46:0/1528469562 shutdown_connections 2024-03-18T20:35:40.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.856+0000 7f70e4144700 1 -- 172.21.15.46:0/1528469562 wait complete. 2024-03-18T20:35:40.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.856+0000 7f70e4144700 1 Processor -- start 2024-03-18T20:35:40.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e4144700 1 -- start start 2024-03-18T20:35:40.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e4144700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc100e00 0x7f70dc074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:40.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e4144700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc101f60 0x7f70dc0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:40.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e4144700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc102920 0x7f70dc0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:40.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e4144700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc073e80 con 0x7f70dc100e00 2024-03-18T20:35:40.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e4144700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc073ff0 con 0x7f70dc102920 2024-03-18T20:35:40.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e4144700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc1a0d40 con 0x7f70dc101f60 2024-03-18T20:35:40.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e1ee0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc100e00 0x7f70dc074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:40.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e1ee0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc100e00 0x7f70dc074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48534/0 (socket says 172.21.15.46:48534) 2024-03-18T20:35:40.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e26e1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc102920 0x7f70dc0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:40.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e16df700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc101f60 0x7f70dc0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:40.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e1ee0700 1 -- 172.21.15.46:0/1274521562 learned_addr learned my addr 172.21.15.46:0/1274521562 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:40.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e1ee0700 1 -- 172.21.15.46:0/1274521562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc101f60 msgr2=0x7f70dc0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:40.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e1ee0700 1 --2- 172.21.15.46:0/1274521562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc101f60 0x7f70dc0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:40.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e1ee0700 1 -- 172.21.15.46:0/1274521562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc102920 msgr2=0x7f70dc0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:40.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e1ee0700 1 --2- 172.21.15.46:0/1274521562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc102920 0x7f70dc0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:40.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.857+0000 7f70e1ee0700 1 -- 172.21.15.46:0/1274521562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc0095f0 con 0x7f70dc100e00 2024-03-18T20:35:40.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.858+0000 7f70e1ee0700 1 --2- 172.21.15.46:0/1274521562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc100e00 0x7f70dc074b00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f70d800d950 tx=0x7f70d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:40.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.858+0000 7f70d2ffd700 1 -- 172.21.15.46:0/1274521562 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d800f890 con 0x7f70dc100e00 2024-03-18T20:35:40.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.858+0000 7f70e4144700 1 -- 172.21.15.46:0/1274521562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc1a0fd0 con 0x7f70dc100e00 2024-03-18T20:35:40.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.858+0000 7f70d2ffd700 1 -- 172.21.15.46:0/1274521562 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70d8010460 con 0x7f70dc100e00 2024-03-18T20:35:40.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.858+0000 7f70d2ffd700 1 -- 172.21.15.46:0/1274521562 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d800e610 con 0x7f70dc100e00 2024-03-18T20:35:40.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.858+0000 7f70e4144700 1 -- 172.21.15.46:0/1274521562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc1a13f0 con 0x7f70dc100e00 2024-03-18T20:35:40.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.859+0000 7f70e4144700 1 -- 172.21.15.46:0/1274521562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc108230 con 0x7f70dc100e00 2024-03-18T20:35:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.860+0000 7f70d2ffd700 1 -- 172.21.15.46:0/1274521562 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f70d80108d0 con 0x7f70dc100e00 2024-03-18T20:35:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.861+0000 7f70d2ffd700 1 --2- 172.21.15.46:0/1274521562 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f70c80a11b0 0x7f70c80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.861+0000 7f70e16df700 1 --2- 172.21.15.46:0/1274521562 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f70c80a11b0 0x7f70c80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.861+0000 7f70d2ffd700 1 -- 172.21.15.46:0/1274521562 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f70d80ccb70 con 0x7f70dc100e00 2024-03-18T20:35:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.861+0000 7f70e16df700 1 --2- 172.21.15.46:0/1274521562 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f70c80a11b0 0x7f70c80a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00b5c0 tx=0x7f70cc003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:40.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:40.864+0000 7f70d2ffd700 1 -- 172.21.15.46:0/1274521562 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f70d80961b0 con 0x7f70dc100e00 2024-03-18T20:35:41.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.078+0000 7f70e4144700 1 -- 172.21.15.46:0/1274521562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f70dc04ea80 con 0x7f70dc100e00 2024-03-18T20:35:41.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.079+0000 7f70d2ffd700 1 -- 172.21.15.46:0/1274521562 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v80) v1 ==== 56+0+5265 (secure 0 0 0) 0x7f70d80997d0 con 0x7f70dc100e00 2024-03-18T20:35:41.082 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 80 2024-03-18T20:35:41.083 INFO:teuthology.orchestra.run.smithi046.stdout:fsid 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T20:35:41.083 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-03-18T20:28:46.137332+0000 2024-03-18T20:35:41.083 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-03-18T20:35:38.108050+0000 2024-03-18T20:35:41.083 INFO:teuthology.orchestra.run.smithi046.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-18T20:35:41.083 INFO:teuthology.orchestra.run.smithi046.stdout:crush_version 26 2024-03-18T20:35:41.083 INFO:teuthology.orchestra.run.smithi046.stdout:full_ratio 0.95 2024-03-18T20:35:41.083 INFO:teuthology.orchestra.run.smithi046.stdout:backfillfull_ratio 0.9 2024-03-18T20:35:41.083 INFO:teuthology.orchestra.run.smithi046.stdout:nearfull_ratio 0.85 2024-03-18T20:35:41.083 INFO:teuthology.orchestra.run.smithi046.stdout:require_min_compat_client luminous 2024-03-18T20:35:41.083 INFO:teuthology.orchestra.run.smithi046.stdout:min_compat_client luminous 2024-03-18T20:35:41.083 INFO:teuthology.orchestra.run.smithi046.stdout:require_osd_release reef 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:stretch_mode_enabled false 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:max_osd 12 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6802/2908504490,v1:172.21.15.46:6803/2908504490] [v2:172.21.15.46:6804/2908504490,v1:172.21.15.46:6805/2908504490] exists,up 303e69be-0b32-4cff-b91e-ce6665f3a087 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 up in weight 1 up_from 13 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6800/1334268822,v1:172.21.15.73:6801/1334268822] [v2:172.21.15.73:6802/1334268822,v1:172.21.15.73:6803/1334268822] exists,up e7b934a7-0679-4158-bc78-6f8cc7d107c9 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 up in weight 1 up_from 18 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6800/772117296,v1:172.21.15.155:6801/772117296] [v2:172.21.15.155:6802/772117296,v1:172.21.15.155:6803/772117296] exists,up 10f51beb-1c27-4b01-9648-f296eddd43af 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 up in weight 1 up_from 24 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6810/1070659456,v1:172.21.15.46:6811/1070659456] [v2:172.21.15.46:6812/1070659456,v1:172.21.15.46:6813/1070659456] exists,up e36074d6-5d67-4411-998e-42bfd7f85a98 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 up in weight 1 up_from 31 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6808/1954145749,v1:172.21.15.73:6809/1954145749] [v2:172.21.15.73:6810/1954145749,v1:172.21.15.73:6811/1954145749] exists,up 73dd25b5-ca0d-4ca4-b079-34487065b2c9 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 up in weight 1 up_from 35 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6808/2840490215,v1:172.21.15.155:6809/2840490215] [v2:172.21.15.155:6810/2840490215,v1:172.21.15.155:6811/2840490215] exists,up 83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 up in weight 1 up_from 42 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6818/2670405640,v1:172.21.15.46:6819/2670405640] [v2:172.21.15.46:6820/2670405640,v1:172.21.15.46:6821/2670405640] exists,up cb44585c-e16c-421c-981a-a7cb8b9ba58d 2024-03-18T20:35:41.084 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 up in weight 1 up_from 47 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6816/1220123190,v1:172.21.15.73:6817/1220123190] [v2:172.21.15.73:6818/1220123190,v1:172.21.15.73:6819/1220123190] exists,up 85b71a49-23d9-4769-b084-7066e7bb38a0 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 up in weight 1 up_from 54 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6816/962631462,v1:172.21.15.155:6817/962631462] [v2:172.21.15.155:6818/962631462,v1:172.21.15.155:6819/962631462] exists,up 45ae5d47-1129-43f7-a5b5-d7e3336ae755 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 up in weight 1 up_from 61 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6826/167112015,v1:172.21.15.46:6827/167112015] [v2:172.21.15.46:6828/167112015,v1:172.21.15.46:6829/167112015] exists,up a8f8c910-b6f1-496a-b5ca-ef6f68fef718 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 up in weight 1 up_from 66 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6824/2313090025,v1:172.21.15.73:6825/2313090025] [v2:172.21.15.73:6826/2313090025,v1:172.21.15.73:6827/2313090025] exists,up b9975d16-8806-494b-926e-2ba60d76ac1e 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 up in weight 1 up_from 73 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6824/2159697764,v1:172.21.15.155:6825/2159697764] [v2:172.21.15.155:6826/2159697764,v1:172.21.15.155:6827/2159697764] exists,up ef1f614e-b094-4a41-a132-8c4db3d34b3b 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.a [2,11] 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.f [8,11] 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.3 [4,9] 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.c [3,9] 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.17 [4,7] 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.37 [5,6] 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.38 [3,9] 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.3a [4,8] 2024-03-18T20:35:41.085 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/2471573219 expires 2024-03-19T20:29:36.953241+0000 2024-03-18T20:35:41.086 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/1997677402 expires 2024-03-19T20:29:36.953241+0000 2024-03-18T20:35:41.086 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/2753187643 expires 2024-03-19T20:29:36.953241+0000 2024-03-18T20:35:41.086 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/2753187643 expires 2024-03-19T20:29:36.953241+0000 2024-03-18T20:35:41.086 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/606003730 expires 2024-03-19T20:29:14.346231+0000 2024-03-18T20:35:41.086 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/699745793 expires 2024-03-19T20:29:14.346231+0000 2024-03-18T20:35:41.086 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/2440783396 expires 2024-03-19T20:29:14.346231+0000 2024-03-18T20:35:41.086 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/1385957034 expires 2024-03-19T20:29:14.346231+0000 2024-03-18T20:35:41.086 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/1385957034 expires 2024-03-19T20:29:14.346231+0000 2024-03-18T20:35:41.086 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/3646752058 expires 2024-03-19T20:29:36.953241+0000 2024-03-18T20:35:41.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.081+0000 7f70e4144700 1 -- 172.21.15.46:0/1274521562 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f70c80a11b0 msgr2=0x7f70c80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:41.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.081+0000 7f70e4144700 1 --2- 172.21.15.46:0/1274521562 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f70c80a11b0 0x7f70c80a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00b5c0 tx=0x7f70cc003790 comp rx=0 tx=0).stop 2024-03-18T20:35:41.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.081+0000 7f70e4144700 1 -- 172.21.15.46:0/1274521562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc100e00 msgr2=0x7f70dc074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:41.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.081+0000 7f70e4144700 1 --2- 172.21.15.46:0/1274521562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc100e00 0x7f70dc074b00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f70d800d950 tx=0x7f70d800dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:41.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.082+0000 7f70e4144700 1 -- 172.21.15.46:0/1274521562 shutdown_connections 2024-03-18T20:35:41.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.082+0000 7f70e4144700 1 --2- 172.21.15.46:0/1274521562 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f70c80a11b0 0x7f70c80a3670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.082+0000 7f70e4144700 1 --2- 172.21.15.46:0/1274521562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc100e00 0x7f70dc074b00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.082+0000 7f70e4144700 1 --2- 172.21.15.46:0/1274521562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc102920 0x7f70dc0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.082+0000 7f70e4144700 1 --2- 172.21.15.46:0/1274521562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70dc101f60 0x7f70dc0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.082+0000 7f70e4144700 1 -- 172.21.15.46:0/1274521562 >> 172.21.15.46:0/1274521562 conn(0x7f70dc0fc3c0 msgr2=0x7f70dc105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:41.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.082+0000 7f70e4144700 1 -- 172.21.15.46:0/1274521562 shutdown_connections 2024-03-18T20:35:41.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.082+0000 7f70e4144700 1 -- 172.21.15.46:0/1274521562 wait complete. 2024-03-18T20:35:41.371 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:41 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1274521562' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-18T20:35:41.462 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-18T20:35:41.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:35:41.474 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:41 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1274521562' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-18T20:35:41.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:41 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1274521562' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-18T20:35:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.620+0000 7f624d6dc700 1 -- 172.21.15.46:0/1280168619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62481016f0 msgr2=0x7f6248107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.620+0000 7f624d6dc700 1 --2- 172.21.15.46:0/1280168619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62481016f0 0x7f6248107fa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f624000b0d0 tx=0x7f624000b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.620+0000 7f624d6dc700 1 -- 172.21.15.46:0/1280168619 shutdown_connections 2024-03-18T20:35:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.620+0000 7f624d6dc700 1 --2- 172.21.15.46:0/1280168619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62481016f0 0x7f6248107fa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.620+0000 7f624d6dc700 1 --2- 172.21.15.46:0/1280168619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62480ffb60 0x7f62480fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.620+0000 7f624d6dc700 1 --2- 172.21.15.46:0/1280168619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6248100d50 0x7f62481011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.620+0000 7f624d6dc700 1 -- 172.21.15.46:0/1280168619 >> 172.21.15.46:0/1280168619 conn(0x7f62480fb0d0 msgr2=0x7f62480fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.620+0000 7f624d6dc700 1 -- 172.21.15.46:0/1280168619 shutdown_connections 2024-03-18T20:35:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.620+0000 7f624d6dc700 1 -- 172.21.15.46:0/1280168619 wait complete. 2024-03-18T20:35:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.620+0000 7f624d6dc700 1 Processor -- start 2024-03-18T20:35:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f624d6dc700 1 -- start start 2024-03-18T20:35:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f624d6dc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62480ffb60 0x7f6248196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f624d6dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6248100d50 0x7f6248197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f624d6dc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62481016f0 0x7f624819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f624d6dc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624819e7e0 con 0x7f6248100d50 2024-03-18T20:35:41.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f624d6dc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624819e950 con 0x7f62480ffb60 2024-03-18T20:35:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f624d6dc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624819eb10 con 0x7f62481016f0 2024-03-18T20:35:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f62467fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6248100d50 0x7f6248197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f62467fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6248100d50 0x7f6248197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48544/0 (socket says 172.21.15.46:48544) 2024-03-18T20:35:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f62467fc700 1 -- 172.21.15.46:0/2769897008 learned_addr learned my addr 172.21.15.46:0/2769897008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f6246ffd700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62480ffb60 0x7f6248196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f62477fe700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62481016f0 0x7f624819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f62467fc700 1 -- 172.21.15.46:0/2769897008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62481016f0 msgr2=0x7f624819e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f62467fc700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62481016f0 0x7f624819e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f62467fc700 1 -- 172.21.15.46:0/2769897008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62480ffb60 msgr2=0x7f6248196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f62467fc700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62480ffb60 0x7f6248196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f62467fc700 1 -- 172.21.15.46:0/2769897008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6240009d00 con 0x7f6248100d50 2024-03-18T20:35:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.621+0000 7f62467fc700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6248100d50 0x7f6248197200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f623c00d950 tx=0x7f623c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.622+0000 7f622ffff700 1 -- 172.21.15.46:0/2769897008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c00f890 con 0x7f6248100d50 2024-03-18T20:35:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.622+0000 7f624d6dc700 1 -- 172.21.15.46:0/2769897008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624819edf0 con 0x7f6248100d50 2024-03-18T20:35:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.622+0000 7f622ffff700 1 -- 172.21.15.46:0/2769897008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f623c010460 con 0x7f6248100d50 2024-03-18T20:35:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.622+0000 7f622ffff700 1 -- 172.21.15.46:0/2769897008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c00e610 con 0x7f6248100d50 2024-03-18T20:35:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.622+0000 7f624d6dc700 1 -- 172.21.15.46:0/2769897008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624819f270 con 0x7f6248100d50 2024-03-18T20:35:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.623+0000 7f624d6dc700 1 -- 172.21.15.46:0/2769897008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6248106f90 con 0x7f6248100d50 2024-03-18T20:35:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.624+0000 7f622ffff700 1 -- 172.21.15.46:0/2769897008 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f623c0108d0 con 0x7f6248100d50 2024-03-18T20:35:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.624+0000 7f622ffff700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f62300a11b0 0x7f62300a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.624+0000 7f6246ffd700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f62300a11b0 0x7f62300a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.625+0000 7f622ffff700 1 -- 172.21.15.46:0/2769897008 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f623c0ccad0 con 0x7f6248100d50 2024-03-18T20:35:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.625+0000 7f6246ffd700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f62300a11b0 0x7f62300a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f62480fcc50 tx=0x7f6238007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:41.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.628+0000 7f622ffff700 1 -- 172.21.15.46:0/2769897008 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f623c096110 con 0x7f6248100d50 2024-03-18T20:35:41.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.860+0000 7f624d6dc700 1 -- 172.21.15.46:0/2769897008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6248067a10 con 0x7f6248100d50 2024-03-18T20:35:41.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.862+0000 7f622ffff700 1 -- 172.21.15.46:0/2769897008 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22503 (secure 0 0 0) 0x7f623c020090 con 0x7f6248100d50 2024-03-18T20:35:41.866 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:41.866 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:35:38.108050+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:35:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.864+0000 7f624d6dc700 1 -- 172.21.15.46:0/2769897008 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f62300a11b0 msgr2=0x7f62300a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.864+0000 7f624d6dc700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f62300a11b0 0x7f62300a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f62480fcc50 tx=0x7f6238007400 comp rx=0 tx=0).stop 2024-03-18T20:35:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.864+0000 7f624d6dc700 1 -- 172.21.15.46:0/2769897008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6248100d50 msgr2=0x7f6248197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.864+0000 7f624d6dc700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6248100d50 0x7f6248197200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f623c00d950 tx=0x7f623c00dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.864+0000 7f624d6dc700 1 -- 172.21.15.46:0/2769897008 shutdown_connections 2024-03-18T20:35:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.864+0000 7f624d6dc700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f62300a11b0 0x7f62300a3670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.864+0000 7f624d6dc700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62481016f0 0x7f624819e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.864+0000 7f624d6dc700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62480ffb60 0x7f6248196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.864+0000 7f624d6dc700 1 --2- 172.21.15.46:0/2769897008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6248100d50 0x7f6248197200 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.864+0000 7f624d6dc700 1 -- 172.21.15.46:0/2769897008 >> 172.21.15.46:0/2769897008 conn(0x7f62480fb0d0 msgr2=0x7f62481040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.864+0000 7f624d6dc700 1 -- 172.21.15.46:0/2769897008 shutdown_connections 2024-03-18T20:35:41.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:41.864+0000 7f624d6dc700 1 -- 172.21.15.46:0/2769897008 wait complete. 2024-03-18T20:35:41.882 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T20:31:27.488318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T20:34:37.213560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T20:34:38.162692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T20:35:41.883 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T20:35:42.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.020+0000 7fb06bd9b700 1 -- 172.21.15.46:0/1690462251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064106aa0 msgr2=0x7fb064108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.020+0000 7fb06bd9b700 1 --2- 172.21.15.46:0/1690462251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064106aa0 0x7fb064108eb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00b0a0 tx=0x7fb05c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.020+0000 7fb06bd9b700 1 -- 172.21.15.46:0/1690462251 shutdown_connections 2024-03-18T20:35:42.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.020+0000 7fb06bd9b700 1 --2- 172.21.15.46:0/1690462251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064106aa0 0x7fb064108eb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.020+0000 7fb06bd9b700 1 --2- 172.21.15.46:0/1690462251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb064101840 0x7fb064103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.020+0000 7fb06bd9b700 1 --2- 172.21.15.46:0/1690462251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb064104170 0x7fb064106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.020+0000 7fb06bd9b700 1 -- 172.21.15.46:0/1690462251 >> 172.21.15.46:0/1690462251 conn(0x7fb0640fb1d0 msgr2=0x7fb0640fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.020+0000 7fb06bd9b700 1 -- 172.21.15.46:0/1690462251 shutdown_connections 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.020+0000 7fb06bd9b700 1 -- 172.21.15.46:0/1690462251 wait complete. 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb06bd9b700 1 Processor -- start 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb06bd9b700 1 -- start start 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb06bd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb064101840 0x7fb064196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb06bd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064104170 0x7fb064197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb06bd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb064106aa0 0x7fb06419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb06bd9b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06419e870 con 0x7fb064104170 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb06bd9b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06419e9e0 con 0x7fb064101840 2024-03-18T20:35:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb06bd9b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06419eba0 con 0x7fb064106aa0 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb069336700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064104170 0x7fb064197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb069336700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064104170 0x7fb064197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48572/0 (socket says 172.21.15.46:48572) 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb069336700 1 -- 172.21.15.46:0/3250726727 learned_addr learned my addr 172.21.15.46:0/3250726727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb069b37700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb064101840 0x7fb064196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb06a338700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb064106aa0 0x7fb06419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb069336700 1 -- 172.21.15.46:0/3250726727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb064106aa0 msgr2=0x7fb06419e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb069336700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb064106aa0 0x7fb06419e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb069336700 1 -- 172.21.15.46:0/3250726727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb064101840 msgr2=0x7fb064196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb069336700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb064101840 0x7fb064196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb069336700 1 -- 172.21.15.46:0/3250726727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c009d00 con 0x7fb064104170 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.021+0000 7fb06a338700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb064106aa0 0x7fb06419e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.022+0000 7fb069336700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064104170 0x7fb064197310 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb05400d980 tx=0x7fb05400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.022+0000 7fb05affd700 1 -- 172.21.15.46:0/3250726727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054004220 con 0x7fb064104170 2024-03-18T20:35:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.022+0000 7fb06bd9b700 1 -- 172.21.15.46:0/3250726727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06419ee80 con 0x7fb064104170 2024-03-18T20:35:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.022+0000 7fb05affd700 1 -- 172.21.15.46:0/3250726727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb054007640 con 0x7fb064104170 2024-03-18T20:35:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.022+0000 7fb05affd700 1 -- 172.21.15.46:0/3250726727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054005610 con 0x7fb064104170 2024-03-18T20:35:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.022+0000 7fb06bd9b700 1 -- 172.21.15.46:0/3250726727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06419f2a0 con 0x7fb064104170 2024-03-18T20:35:42.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.023+0000 7fb06bd9b700 1 -- 172.21.15.46:0/3250726727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0640fcd50 con 0x7fb064104170 2024-03-18T20:35:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.024+0000 7fb05affd700 1 -- 172.21.15.46:0/3250726727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb054004750 con 0x7fb064104170 2024-03-18T20:35:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.025+0000 7fb05affd700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0500a11b0 0x7fb0500a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.025+0000 7fb069b37700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0500a11b0 0x7fb0500a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.025+0000 7fb05affd700 1 -- 172.21.15.46:0/3250726727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb0540cc900 con 0x7fb064104170 2024-03-18T20:35:42.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.026+0000 7fb069b37700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0500a11b0 0x7fb0500a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb064198200 tx=0x7fb060009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.030+0000 7fb05affd700 1 -- 172.21.15.46:0/3250726727 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb054095ee0 con 0x7fb064104170 2024-03-18T20:35:42.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.246+0000 7fb06bd9b700 1 -- 172.21.15.46:0/3250726727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb06402d0f0 con 0x7fb064104170 2024-03-18T20:35:42.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.246+0000 7fb05affd700 1 -- 172.21.15.46:0/3250726727 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb054016020 con 0x7fb064104170 2024-03-18T20:35:42.250 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-18T20:35:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.249+0000 7fb06bd9b700 1 -- 172.21.15.46:0/3250726727 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0500a11b0 msgr2=0x7fb0500a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.249+0000 7fb06bd9b700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0500a11b0 0x7fb0500a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb064198200 tx=0x7fb060009260 comp rx=0 tx=0).stop 2024-03-18T20:35:42.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.249+0000 7fb06bd9b700 1 -- 172.21.15.46:0/3250726727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064104170 msgr2=0x7fb064197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.249+0000 7fb06bd9b700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064104170 0x7fb064197310 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb05400d980 tx=0x7fb05400dc90 comp rx=0 tx=0).stop 2024-03-18T20:35:42.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.249+0000 7fb06bd9b700 1 -- 172.21.15.46:0/3250726727 shutdown_connections 2024-03-18T20:35:42.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.249+0000 7fb06bd9b700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb0500a11b0 0x7fb0500a3670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.249+0000 7fb06bd9b700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb064106aa0 0x7fb06419e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.249+0000 7fb06bd9b700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb064101840 0x7fb064196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.249+0000 7fb06bd9b700 1 --2- 172.21.15.46:0/3250726727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb064104170 0x7fb064197310 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.249+0000 7fb06bd9b700 1 -- 172.21.15.46:0/3250726727 >> 172.21.15.46:0/3250726727 conn(0x7fb0640fb1d0 msgr2=0x7fb0641020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:42.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.249+0000 7fb06bd9b700 1 -- 172.21.15.46:0/3250726727 shutdown_connections 2024-03-18T20:35:42.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.249+0000 7fb06bd9b700 1 -- 172.21.15.46:0/3250726727 wait complete. 2024-03-18T20:35:42.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T20:35:42.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.406+0000 7f2f3a280700 1 -- 172.21.15.46:0/894710740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f34101890 msgr2=0x7f2f34103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.406+0000 7f2f3a280700 1 --2- 172.21.15.46:0/894710740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f34101890 0x7f2f34103c80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009990 tx=0x7f2f24009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.406+0000 7f2f3a280700 1 -- 172.21.15.46:0/894710740 shutdown_connections 2024-03-18T20:35:42.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.406+0000 7f2f3a280700 1 --2- 172.21.15.46:0/894710740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f34101890 0x7f2f34103c80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.406+0000 7f2f3a280700 1 --2- 172.21.15.46:0/894710740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f34106af0 0x7f2f34108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.406+0000 7f2f3a280700 1 --2- 172.21.15.46:0/894710740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f341041c0 0x7f2f341065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.406+0000 7f2f3a280700 1 -- 172.21.15.46:0/894710740 >> 172.21.15.46:0/894710740 conn(0x7f2f340fb200 msgr2=0x7f2f340fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:42.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.406+0000 7f2f3a280700 1 -- 172.21.15.46:0/894710740 shutdown_connections 2024-03-18T20:35:42.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.406+0000 7f2f3a280700 1 -- 172.21.15.46:0/894710740 wait complete. 2024-03-18T20:35:42.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.407+0000 7f2f3a280700 1 Processor -- start 2024-03-18T20:35:42.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.407+0000 7f2f3a280700 1 -- start start 2024-03-18T20:35:42.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.407+0000 7f2f3a280700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f34101890 0x7f2f34196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:42.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.407+0000 7f2f3a280700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f341041c0 0x7f2f34197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:42.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.407+0000 7f2f3a280700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f34106af0 0x7f2f3419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:42.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.407+0000 7f2f3a280700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3419e7e0 con 0x7f2f341041c0 2024-03-18T20:35:42.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.407+0000 7f2f3a280700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3419e950 con 0x7f2f34106af0 2024-03-18T20:35:42.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.407+0000 7f2f3a280700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3419eb10 con 0x7f2f34101890 2024-03-18T20:35:42.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.407+0000 7f2f337fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f341041c0 0x7f2f34197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:42.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.407+0000 7f2f337fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f341041c0 0x7f2f34197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48590/0 (socket says 172.21.15.46:48590) 2024-03-18T20:35:42.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.407+0000 7f2f337fe700 1 -- 172.21.15.46:0/2986252318 learned_addr learned my addr 172.21.15.46:0/2986252318 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:42.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.407+0000 7f2f33fff700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f34101890 0x7f2f34196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.408+0000 7f2f3881d700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f34106af0 0x7f2f3419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.408+0000 7f2f337fe700 1 -- 172.21.15.46:0/2986252318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f34101890 msgr2=0x7f2f34196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.408+0000 7f2f337fe700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f34101890 0x7f2f34196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.408+0000 7f2f337fe700 1 -- 172.21.15.46:0/2986252318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f34106af0 msgr2=0x7f2f3419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.408+0000 7f2f337fe700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f34106af0 0x7f2f3419e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.408+0000 7f2f337fe700 1 -- 172.21.15.46:0/2986252318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f240095f0 con 0x7f2f341041c0 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.408+0000 7f2f337fe700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f341041c0 0x7f2f34197200 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800d950 tx=0x7f2f2800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.408+0000 7f2f317fa700 1 -- 172.21.15.46:0/2986252318 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28009940 con 0x7f2f341041c0 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.408+0000 7f2f317fa700 1 -- 172.21.15.46:0/2986252318 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f28010460 con 0x7f2f341041c0 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.408+0000 7f2f3a280700 1 -- 172.21.15.46:0/2986252318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3419edf0 con 0x7f2f341041c0 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.408+0000 7f2f317fa700 1 -- 172.21.15.46:0/2986252318 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800f620 con 0x7f2f341041c0 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.408+0000 7f2f3a280700 1 -- 172.21.15.46:0/2986252318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3419f280 con 0x7f2f341041c0 2024-03-18T20:35:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.409+0000 7f2f3a280700 1 -- 172.21.15.46:0/2986252318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f340fcd80 con 0x7f2f341041c0 2024-03-18T20:35:42.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.411+0000 7f2f317fa700 1 -- 172.21.15.46:0/2986252318 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f2f280108d0 con 0x7f2f341041c0 2024-03-18T20:35:42.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.412+0000 7f2f317fa700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2f1c0a1230 0x7f2f1c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:42.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.412+0000 7f2f33fff700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2f1c0a1230 0x7f2f1c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:42.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.412+0000 7f2f317fa700 1 -- 172.21.15.46:0/2986252318 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f280ccc10 con 0x7f2f341041c0 2024-03-18T20:35:42.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.412+0000 7f2f33fff700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2f1c0a1230 0x7f2f1c0a36f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400b5c0 tx=0x7f2f24003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:42.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.415+0000 7f2f317fa700 1 -- 172.21.15.46:0/2986252318 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2f280961f0 con 0x7f2f341041c0 2024-03-18T20:35:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:42 smithi073 ceph-mon[47384]: pgmap v252: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s; 14 B/s, 0 objects/s recovering 2024-03-18T20:35:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:42 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2769897008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:42.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:42 smithi155 ceph-mon[48344]: pgmap v252: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s; 14 B/s, 0 objects/s recovering 2024-03-18T20:35:42.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:42 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2769897008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:42 smithi046 ceph-mon[43495]: pgmap v252: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s; 14 B/s, 0 objects/s recovering 2024-03-18T20:35:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:42 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2769897008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.632+0000 7f2f3a280700 1 -- 172.21.15.46:0/2986252318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f3402d0f0 con 0x7f2f341041c0 2024-03-18T20:35:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.632+0000 7f2f317fa700 1 -- 172.21.15.46:0/2986252318 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f280200a0 con 0x7f2f341041c0 2024-03-18T20:35:42.636 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-18T20:35:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.634+0000 7f2f3a280700 1 -- 172.21.15.46:0/2986252318 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2f1c0a1230 msgr2=0x7f2f1c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.634+0000 7f2f3a280700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2f1c0a1230 0x7f2f1c0a36f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400b5c0 tx=0x7f2f24003790 comp rx=0 tx=0).stop 2024-03-18T20:35:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.634+0000 7f2f3a280700 1 -- 172.21.15.46:0/2986252318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f341041c0 msgr2=0x7f2f34197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.634+0000 7f2f3a280700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f341041c0 0x7f2f34197200 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800d950 tx=0x7f2f2800dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.635+0000 7f2f3a280700 1 -- 172.21.15.46:0/2986252318 shutdown_connections 2024-03-18T20:35:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.635+0000 7f2f3a280700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2f1c0a1230 0x7f2f1c0a36f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.635+0000 7f2f3a280700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f34106af0 0x7f2f3419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.635+0000 7f2f3a280700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f34101890 0x7f2f34196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.635+0000 7f2f3a280700 1 --2- 172.21.15.46:0/2986252318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f341041c0 0x7f2f34197200 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.635+0000 7f2f3a280700 1 -- 172.21.15.46:0/2986252318 >> 172.21.15.46:0/2986252318 conn(0x7f2f340fb200 msgr2=0x7f2f34103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.635+0000 7f2f3a280700 1 -- 172.21.15.46:0/2986252318 shutdown_connections 2024-03-18T20:35:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.635+0000 7f2f3a280700 1 -- 172.21.15.46:0/2986252318 wait complete. 2024-03-18T20:35:42.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T20:35:42.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.789+0000 7f90983cf700 1 -- 172.21.15.46:0/2014913463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9090100e60 msgr2=0x7f90901012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.789+0000 7f90983cf700 1 --2- 172.21.15.46:0/2014913463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9090100e60 0x7f90901012c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f90800099c0 tx=0x7f9080009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.789+0000 7f90983cf700 1 -- 172.21.15.46:0/2014913463 shutdown_connections 2024-03-18T20:35:42.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.789+0000 7f90983cf700 1 --2- 172.21.15.46:0/2014913463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9090101800 0x7f90901080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.789+0000 7f90983cf700 1 --2- 172.21.15.46:0/2014913463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90900ffc70 0x7f9090100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.789+0000 7f90983cf700 1 --2- 172.21.15.46:0/2014913463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9090100e60 0x7f90901012c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.789+0000 7f90983cf700 1 -- 172.21.15.46:0/2014913463 >> 172.21.15.46:0/2014913463 conn(0x7f90900fb200 msgr2=0x7f90900fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:42.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.789+0000 7f90983cf700 1 -- 172.21.15.46:0/2014913463 shutdown_connections 2024-03-18T20:35:42.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.789+0000 7f90983cf700 1 -- 172.21.15.46:0/2014913463 wait complete. 2024-03-18T20:35:42.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.789+0000 7f90983cf700 1 Processor -- start 2024-03-18T20:35:42.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.789+0000 7f90983cf700 1 -- start start 2024-03-18T20:35:42.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f90983cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90900ffc70 0x7f9090196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:42.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f90983cf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9090100e60 0x7f9090197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:42.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f90983cf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9090101800 0x7f909019e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:42.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f90983cf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909019e780 con 0x7f90900ffc70 2024-03-18T20:35:42.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f90983cf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909019e8f0 con 0x7f9090101800 2024-03-18T20:35:42.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f90983cf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909019eab0 con 0x7f9090100e60 2024-03-18T20:35:42.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f909616b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90900ffc70 0x7f9090196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:42.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f909616b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90900ffc70 0x7f9090196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48598/0 (socket says 172.21.15.46:48598) 2024-03-18T20:35:42.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f909616b700 1 -- 172.21.15.46:0/2396565788 learned_addr learned my addr 172.21.15.46:0/2396565788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f909596a700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9090100e60 0x7f9090197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f909696c700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9090101800 0x7f909019e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f909616b700 1 -- 172.21.15.46:0/2396565788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9090100e60 msgr2=0x7f9090197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f909616b700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9090100e60 0x7f9090197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f909616b700 1 -- 172.21.15.46:0/2396565788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9090101800 msgr2=0x7f909019e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f909616b700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9090101800 0x7f909019e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f909616b700 1 -- 172.21.15.46:0/2396565788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90800095f0 con 0x7f90900ffc70 2024-03-18T20:35:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f909616b700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90900ffc70 0x7f9090196ce0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f908c00e950 tx=0x7f908c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f90877fe700 1 -- 172.21.15.46:0/2396565788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c009770 con 0x7f90900ffc70 2024-03-18T20:35:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f90877fe700 1 -- 172.21.15.46:0/2396565788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908c00c7e0 con 0x7f90900ffc70 2024-03-18T20:35:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.790+0000 7f90877fe700 1 -- 172.21.15.46:0/2396565788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c003cf0 con 0x7f90900ffc70 2024-03-18T20:35:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.791+0000 7f90983cf700 1 -- 172.21.15.46:0/2396565788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909019ed90 con 0x7f90900ffc70 2024-03-18T20:35:42.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.791+0000 7f90983cf700 1 -- 172.21.15.46:0/2396565788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909019f190 con 0x7f90900ffc70 2024-03-18T20:35:42.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.792+0000 7f90983cf700 1 -- 172.21.15.46:0/2396565788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9090067f40 con 0x7f90900ffc70 2024-03-18T20:35:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.793+0000 7f90877fe700 1 -- 172.21.15.46:0/2396565788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f908c00c950 con 0x7f90900ffc70 2024-03-18T20:35:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.794+0000 7f90877fe700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f907c0a1230 0x7f907c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.794+0000 7f909596a700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f907c0a1230 0x7f907c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.794+0000 7f90877fe700 1 -- 172.21.15.46:0/2396565788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f908c014070 con 0x7f90900ffc70 2024-03-18T20:35:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.794+0000 7f909596a700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f907c0a1230 0x7f907c0a36f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9080009990 tx=0x7f9080000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:42.799+0000 7f90877fe700 1 -- 172.21.15.46:0/2396565788 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f908c099cd0 con 0x7f90900ffc70 2024-03-18T20:35:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.017+0000 7f90983cf700 1 -- 172.21.15.46:0/2396565788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f909019f440 con 0x7f90900ffc70 2024-03-18T20:35:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.017+0000 7f90877fe700 1 -- 172.21.15.46:0/2396565788 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f908c099860 con 0x7f90900ffc70 2024-03-18T20:35:43.021 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-18T20:35:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.019+0000 7f90983cf700 1 -- 172.21.15.46:0/2396565788 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f907c0a1230 msgr2=0x7f907c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.019+0000 7f90983cf700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f907c0a1230 0x7f907c0a36f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9080009990 tx=0x7f9080000bc0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.019+0000 7f90983cf700 1 -- 172.21.15.46:0/2396565788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90900ffc70 msgr2=0x7f9090196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.019+0000 7f90983cf700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90900ffc70 0x7f9090196ce0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f908c00e950 tx=0x7f908c00ec60 comp rx=0 tx=0).stop 2024-03-18T20:35:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.019+0000 7f90983cf700 1 -- 172.21.15.46:0/2396565788 shutdown_connections 2024-03-18T20:35:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.019+0000 7f90983cf700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f907c0a1230 0x7f907c0a36f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.019+0000 7f90983cf700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90900ffc70 0x7f9090196ce0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.019+0000 7f90983cf700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9090101800 0x7f909019e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.019+0000 7f90983cf700 1 --2- 172.21.15.46:0/2396565788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9090100e60 0x7f9090197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.019+0000 7f90983cf700 1 -- 172.21.15.46:0/2396565788 >> 172.21.15.46:0/2396565788 conn(0x7f90900fb200 msgr2=0x7f90900fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.020+0000 7f90983cf700 1 -- 172.21.15.46:0/2396565788 shutdown_connections 2024-03-18T20:35:43.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.020+0000 7f90983cf700 1 -- 172.21.15.46:0/2396565788 wait complete. 2024-03-18T20:35:43.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:35:43.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.173+0000 7f501f29b700 1 -- 172.21.15.46:0/557554960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50180ffb60 msgr2=0x7f50180fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:43.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.173+0000 7f501f29b700 1 --2- 172.21.15.46:0/557554960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50180ffb60 0x7f50180fff80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f50140099c0 tx=0x7f5014009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.173+0000 7f501f29b700 1 -- 172.21.15.46:0/557554960 shutdown_connections 2024-03-18T20:35:43.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.173+0000 7f501f29b700 1 --2- 172.21.15.46:0/557554960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50180ffb60 0x7f50180fff80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.173+0000 7f501f29b700 1 --2- 172.21.15.46:0/557554960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50181016f0 0x7f5018107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.173+0000 7f501f29b700 1 --2- 172.21.15.46:0/557554960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5018100d50 0x7f50181011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.173+0000 7f501f29b700 1 -- 172.21.15.46:0/557554960 >> 172.21.15.46:0/557554960 conn(0x7f50180fb0d0 msgr2=0x7f50180fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.173+0000 7f501f29b700 1 -- 172.21.15.46:0/557554960 shutdown_connections 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.173+0000 7f501f29b700 1 -- 172.21.15.46:0/557554960 wait complete. 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.173+0000 7f501f29b700 1 Processor -- start 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501f29b700 1 -- start start 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501f29b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50180ffb60 0x7f5018068910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501f29b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5018100d50 0x7f501819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501f29b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50181016f0 0x7f501819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501f29b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501819e640 con 0x7f5018100d50 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501f29b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501819e7b0 con 0x7f50180ffb60 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501f29b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501819e970 con 0x7f50181016f0 2024-03-18T20:35:43.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501c836700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5018100d50 0x7f501819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501c836700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5018100d50 0x7f501819b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48612/0 (socket says 172.21.15.46:48612) 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501c836700 1 -- 172.21.15.46:0/1939181188 learned_addr learned my addr 172.21.15.46:0/1939181188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501d037700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50180ffb60 0x7f5018068910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501d838700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50181016f0 0x7f501819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501c836700 1 -- 172.21.15.46:0/1939181188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50181016f0 msgr2=0x7f501819dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501c836700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50181016f0 0x7f501819dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501c836700 1 -- 172.21.15.46:0/1939181188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50180ffb60 msgr2=0x7f5018068910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501c836700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50180ffb60 0x7f5018068910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501c836700 1 -- 172.21.15.46:0/1939181188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50140095f0 con 0x7f5018100d50 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.174+0000 7f501c836700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5018100d50 0x7f501819b8b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f500800c9b0 tx=0x7f500800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.175+0000 7f500e7fc700 1 -- 172.21.15.46:0/1939181188 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008007b00 con 0x7f5018100d50 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.175+0000 7f500e7fc700 1 -- 172.21.15.46:0/1939181188 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f500800ce50 con 0x7f5018100d50 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.175+0000 7f501f29b700 1 -- 172.21.15.46:0/1939181188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501819ec50 con 0x7f5018100d50 2024-03-18T20:35:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.175+0000 7f500e7fc700 1 -- 172.21.15.46:0/1939181188 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008018660 con 0x7f5018100d50 2024-03-18T20:35:43.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.175+0000 7f501f29b700 1 -- 172.21.15.46:0/1939181188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501819f1b0 con 0x7f5018100d50 2024-03-18T20:35:43.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.176+0000 7f501f29b700 1 -- 172.21.15.46:0/1939181188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5018106f30 con 0x7f5018100d50 2024-03-18T20:35:43.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.177+0000 7f500e7fc700 1 -- 172.21.15.46:0/1939181188 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f500800f450 con 0x7f5018100d50 2024-03-18T20:35:43.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.178+0000 7f500e7fc700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f50040a1230 0x7f50040a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:43.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.178+0000 7f501d037700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f50040a1230 0x7f50040a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:43.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.178+0000 7f500e7fc700 1 -- 172.21.15.46:0/1939181188 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f50080ccc10 con 0x7f5018100d50 2024-03-18T20:35:43.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.178+0000 7f501d037700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f50040a1230 0x7f50040a36f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5014003820 tx=0x7f5014005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:43.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.181+0000 7f500e7fc700 1 -- 172.21.15.46:0/1939181188 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5008096300 con 0x7f5018100d50 2024-03-18T20:35:43.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:43 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3250726727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:43.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:43 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2986252318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:43.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:43 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2396565788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:43.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.447+0000 7f501f29b700 1 -- 172.21.15.46:0/1939181188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5018067a10 con 0x7f5018100d50 2024-03-18T20:35:43.492 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:43 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3250726727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:43.492 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:43 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2986252318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:43.492 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:43 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2396565788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:43.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.447+0000 7f500e7fc700 1 -- 172.21.15.46:0/1939181188 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f5008007e60 con 0x7f5018100d50 2024-03-18T20:35:43.493 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:43.493 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:34:45.796438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:35:43.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.450+0000 7f501f29b700 1 -- 172.21.15.46:0/1939181188 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f50040a1230 msgr2=0x7f50040a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:43.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.450+0000 7f501f29b700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f50040a1230 0x7f50040a36f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5014003820 tx=0x7f5014005920 comp rx=0 tx=0).stop 2024-03-18T20:35:43.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.450+0000 7f501f29b700 1 -- 172.21.15.46:0/1939181188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5018100d50 msgr2=0x7f501819b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:43.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.450+0000 7f501f29b700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5018100d50 0x7f501819b8b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f500800c9b0 tx=0x7f500800ccc0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.450+0000 7f501f29b700 1 -- 172.21.15.46:0/1939181188 shutdown_connections 2024-03-18T20:35:43.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.450+0000 7f501f29b700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f50040a1230 0x7f50040a36f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.450+0000 7f501f29b700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f50181016f0 0x7f501819dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.450+0000 7f501f29b700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50180ffb60 0x7f5018068910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.450+0000 7f501f29b700 1 --2- 172.21.15.46:0/1939181188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5018100d50 0x7f501819b8b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.450+0000 7f501f29b700 1 -- 172.21.15.46:0/1939181188 >> 172.21.15.46:0/1939181188 conn(0x7f50180fb0d0 msgr2=0x7f5018104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:43.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.450+0000 7f501f29b700 1 -- 172.21.15.46:0/1939181188 shutdown_connections 2024-03-18T20:35:43.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.450+0000 7f501f29b700 1 -- 172.21.15.46:0/1939181188 wait complete. 2024-03-18T20:35:43.495 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-03-18T20:35:43.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:35:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:43 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3250726727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:43.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:43 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2986252318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:43.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:43 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2396565788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.631+0000 7f86e8de6700 1 -- 172.21.15.46:0/1993016774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e40ffb60 msgr2=0x7f86e40fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:43.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.631+0000 7f86e8de6700 1 --2- 172.21.15.46:0/1993016774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e40ffb60 0x7f86e40fff80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f86d40099c0 tx=0x7f86d4009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.631+0000 7f86e8de6700 1 -- 172.21.15.46:0/1993016774 shutdown_connections 2024-03-18T20:35:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.631+0000 7f86e8de6700 1 --2- 172.21.15.46:0/1993016774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e40ffb60 0x7f86e40fff80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.631+0000 7f86e8de6700 1 --2- 172.21.15.46:0/1993016774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e41016f0 0x7f86e4107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.631+0000 7f86e8de6700 1 --2- 172.21.15.46:0/1993016774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e4100d50 0x7f86e41011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.631+0000 7f86e8de6700 1 -- 172.21.15.46:0/1993016774 >> 172.21.15.46:0/1993016774 conn(0x7f86e40fb0d0 msgr2=0x7f86e40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.632+0000 7f86e8de6700 1 -- 172.21.15.46:0/1993016774 shutdown_connections 2024-03-18T20:35:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.632+0000 7f86e8de6700 1 -- 172.21.15.46:0/1993016774 wait complete. 2024-03-18T20:35:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.632+0000 7f86e8de6700 1 Processor -- start 2024-03-18T20:35:43.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.632+0000 7f86e8de6700 1 -- start start 2024-03-18T20:35:43.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e8de6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e40ffb60 0x7f86e4196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:43.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e8de6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4100d50 0x7f86e41970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:43.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e8de6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e41016f0 0x7f86e419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:43.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e8de6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e419e690 con 0x7f86e4100d50 2024-03-18T20:35:43.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e8de6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e419e7d0 con 0x7f86e40ffb60 2024-03-18T20:35:43.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e8de6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e419e960 con 0x7f86e41016f0 2024-03-18T20:35:43.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4100d50 0x7f86e41970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4100d50 0x7f86e41970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48632/0 (socket says 172.21.15.46:48632) 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e1d9b700 1 -- 172.21.15.46:0/316034606 learned_addr learned my addr 172.21.15.46:0/316034606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e259c700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e40ffb60 0x7f86e4196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e2d9d700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e41016f0 0x7f86e419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e1d9b700 1 -- 172.21.15.46:0/316034606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e41016f0 msgr2=0x7f86e419e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e41016f0 0x7f86e419e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e1d9b700 1 -- 172.21.15.46:0/316034606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e40ffb60 msgr2=0x7f86e4196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e40ffb60 0x7f86e4196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e1d9b700 1 -- 172.21.15.46:0/316034606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d40095f0 con 0x7f86e4100d50 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4100d50 0x7f86e41970a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f86d800c9b0 tx=0x7f86d800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86d37fe700 1 -- 172.21.15.46:0/316034606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8007b00 con 0x7f86e4100d50 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86d37fe700 1 -- 172.21.15.46:0/316034606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86d800ce50 con 0x7f86e4100d50 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.633+0000 7f86e8de6700 1 -- 172.21.15.46:0/316034606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e419ec40 con 0x7f86e4100d50 2024-03-18T20:35:43.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.634+0000 7f86d37fe700 1 -- 172.21.15.46:0/316034606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8018660 con 0x7f86e4100d50 2024-03-18T20:35:43.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.634+0000 7f86e8de6700 1 -- 172.21.15.46:0/316034606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e419f150 con 0x7f86e4100d50 2024-03-18T20:35:43.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.634+0000 7f86e8de6700 1 -- 172.21.15.46:0/316034606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e4067a10 con 0x7f86e4100d50 2024-03-18T20:35:43.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.636+0000 7f86d37fe700 1 -- 172.21.15.46:0/316034606 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f86d800f450 con 0x7f86e4100d50 2024-03-18T20:35:43.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.636+0000 7f86d37fe700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f86cc0a11b0 0x7f86cc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:43.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.637+0000 7f86e259c700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f86cc0a11b0 0x7f86cc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:43.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.637+0000 7f86d37fe700 1 -- 172.21.15.46:0/316034606 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f86d80cc930 con 0x7f86e4100d50 2024-03-18T20:35:43.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.637+0000 7f86e259c700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f86cc0a11b0 0x7f86cc0a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f86d4003730 tx=0x7f86d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:43.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.641+0000 7f86d37fe700 1 -- 172.21.15.46:0/316034606 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f86d8095f70 con 0x7f86e4100d50 2024-03-18T20:35:43.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.871+0000 7f86e8de6700 1 -- 172.21.15.46:0/316034606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86e419f340 con 0x7f86e4100d50 2024-03-18T20:35:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.872+0000 7f86d37fe700 1 -- 172.21.15.46:0/316034606 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22503 (secure 0 0 0) 0x7f86d8099590 con 0x7f86e4100d50 2024-03-18T20:35:43.876 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:35:38.108050+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:35:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.875+0000 7f86e8de6700 1 -- 172.21.15.46:0/316034606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f86cc0a11b0 msgr2=0x7f86cc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.875+0000 7f86e8de6700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f86cc0a11b0 0x7f86cc0a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f86d4003730 tx=0x7f86d4000bc0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.875+0000 7f86e8de6700 1 -- 172.21.15.46:0/316034606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4100d50 msgr2=0x7f86e41970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.875+0000 7f86e8de6700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4100d50 0x7f86e41970a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f86d800c9b0 tx=0x7f86d800ccc0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.875+0000 7f86e8de6700 1 -- 172.21.15.46:0/316034606 shutdown_connections 2024-03-18T20:35:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.875+0000 7f86e8de6700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f86cc0a11b0 0x7f86cc0a3670 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.875+0000 7f86e8de6700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e41016f0 0x7f86e419e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.875+0000 7f86e8de6700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e40ffb60 0x7f86e4196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.875+0000 7f86e8de6700 1 --2- 172.21.15.46:0/316034606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4100d50 0x7f86e41970a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.875+0000 7f86e8de6700 1 -- 172.21.15.46:0/316034606 >> 172.21.15.46:0/316034606 conn(0x7f86e40fb0d0 msgr2=0x7f86e40fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.875+0000 7f86e8de6700 1 -- 172.21.15.46:0/316034606 shutdown_connections 2024-03-18T20:35:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:43.875+0000 7f86e8de6700 1 -- 172.21.15.46:0/316034606 wait complete. 2024-03-18T20:35:43.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T20:31:27.488318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T20:34:37.213560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T20:34:38.162692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T20:35:43.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T20:35:44.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.030+0000 7faef96ff700 1 -- 172.21.15.46:0/981333749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef4106a70 msgr2=0x7faef4108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.030+0000 7faef96ff700 1 --2- 172.21.15.46:0/981333749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef4106a70 0x7faef4108ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faeec00b0a0 tx=0x7faeec00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.030+0000 7faef96ff700 1 -- 172.21.15.46:0/981333749 shutdown_connections 2024-03-18T20:35:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.030+0000 7faef96ff700 1 --2- 172.21.15.46:0/981333749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef4106a70 0x7faef4108ee0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.030+0000 7faef96ff700 1 --2- 172.21.15.46:0/981333749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef4101810 0x7faef4103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.030+0000 7faef96ff700 1 --2- 172.21.15.46:0/981333749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faef4104140 0x7faef4106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.030+0000 7faef96ff700 1 -- 172.21.15.46:0/981333749 >> 172.21.15.46:0/981333749 conn(0x7faef40fb200 msgr2=0x7faef40fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.030+0000 7faef96ff700 1 -- 172.21.15.46:0/981333749 shutdown_connections 2024-03-18T20:35:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.030+0000 7faef96ff700 1 -- 172.21.15.46:0/981333749 wait complete. 2024-03-18T20:35:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.030+0000 7faef96ff700 1 Processor -- start 2024-03-18T20:35:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef96ff700 1 -- start start 2024-03-18T20:35:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef96ff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef4101810 0x7faef4196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef96ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faef4104140 0x7faef4197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef96ff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef4106a70 0x7faef419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef96ff700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef419e950 con 0x7faef4101810 2024-03-18T20:35:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef96ff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef419ea90 con 0x7faef4106a70 2024-03-18T20:35:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef96ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef419ec20 con 0x7faef4104140 2024-03-18T20:35:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef4101810 0x7faef4196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef4101810 0x7faef4196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48644/0 (socket says 172.21.15.46:48644) 2024-03-18T20:35:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef2ffd700 1 -- 172.21.15.46:0/3801520421 learned_addr learned my addr 172.21.15.46:0/3801520421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef27fc700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faef4104140 0x7faef4197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef37fe700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef4106a70 0x7faef419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef2ffd700 1 -- 172.21.15.46:0/3801520421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faef4104140 msgr2=0x7faef4197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef2ffd700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faef4104140 0x7faef4197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef2ffd700 1 -- 172.21.15.46:0/3801520421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef4106a70 msgr2=0x7faef419e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef2ffd700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef4106a70 0x7faef419e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef2ffd700 1 -- 172.21.15.46:0/3801520421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec009d00 con 0x7faef4101810 2024-03-18T20:35:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.031+0000 7faef37fe700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef4106a70 0x7faef419e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.032+0000 7faef2ffd700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef4101810 0x7faef4196e20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faee400d950 tx=0x7faee400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.032+0000 7faedbfff700 1 -- 172.21.15.46:0/3801520421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4009940 con 0x7faef4101810 2024-03-18T20:35:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.032+0000 7faedbfff700 1 -- 172.21.15.46:0/3801520421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee4010460 con 0x7faef4101810 2024-03-18T20:35:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.032+0000 7faedbfff700 1 -- 172.21.15.46:0/3801520421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee400f620 con 0x7faef4101810 2024-03-18T20:35:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.032+0000 7faef96ff700 1 -- 172.21.15.46:0/3801520421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef419ef00 con 0x7faef4101810 2024-03-18T20:35:44.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.032+0000 7faef96ff700 1 -- 172.21.15.46:0/3801520421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef419f2c0 con 0x7faef4101810 2024-03-18T20:35:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.033+0000 7faef96ff700 1 -- 172.21.15.46:0/3801520421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef40fcd80 con 0x7faef4101810 2024-03-18T20:35:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.034+0000 7faedbfff700 1 -- 172.21.15.46:0/3801520421 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7faee40108d0 con 0x7faef4101810 2024-03-18T20:35:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.035+0000 7faedbfff700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faedc0a11b0 0x7faedc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.035+0000 7faef27fc700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faedc0a11b0 0x7faedc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.035+0000 7faedbfff700 1 -- 172.21.15.46:0/3801520421 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faee40cc920 con 0x7faef4101810 2024-03-18T20:35:44.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.035+0000 7faef27fc700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faedc0a11b0 0x7faedc0a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faee8009c60 tx=0x7faee8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:44.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.039+0000 7faedbfff700 1 -- 172.21.15.46:0/3801520421 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faee4095f00 con 0x7faef4101810 2024-03-18T20:35:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.254+0000 7faef96ff700 1 -- 172.21.15.46:0/3801520421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faef402cf60 con 0x7faef4101810 2024-03-18T20:35:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.255+0000 7faedbfff700 1 -- 172.21.15.46:0/3801520421 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7faee40160a0 con 0x7faef4101810 2024-03-18T20:35:44.258 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-18T20:35:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.257+0000 7faef96ff700 1 -- 172.21.15.46:0/3801520421 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faedc0a11b0 msgr2=0x7faedc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.257+0000 7faef96ff700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faedc0a11b0 0x7faedc0a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faee8009c60 tx=0x7faee8009260 comp rx=0 tx=0).stop 2024-03-18T20:35:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.257+0000 7faef96ff700 1 -- 172.21.15.46:0/3801520421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef4101810 msgr2=0x7faef4196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.257+0000 7faef96ff700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef4101810 0x7faef4196e20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faee400d950 tx=0x7faee400dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.257+0000 7faef96ff700 1 -- 172.21.15.46:0/3801520421 shutdown_connections 2024-03-18T20:35:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.257+0000 7faef96ff700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7faedc0a11b0 0x7faedc0a3670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.257+0000 7faef96ff700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef4101810 0x7faef4196e20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.257+0000 7faef96ff700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef4106a70 0x7faef419e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.257+0000 7faef96ff700 1 --2- 172.21.15.46:0/3801520421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faef4104140 0x7faef4197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.257+0000 7faef96ff700 1 -- 172.21.15.46:0/3801520421 >> 172.21.15.46:0/3801520421 conn(0x7faef40fb200 msgr2=0x7faef4102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:44.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.257+0000 7faef96ff700 1 -- 172.21.15.46:0/3801520421 shutdown_connections 2024-03-18T20:35:44.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.257+0000 7faef96ff700 1 -- 172.21.15.46:0/3801520421 wait complete. 2024-03-18T20:35:44.275 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T20:35:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.411+0000 7fef5930c700 1 -- 172.21.15.46:0/3159048284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef541008b0 msgr2=0x7fef54108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.411+0000 7fef5930c700 1 --2- 172.21.15.46:0/3159048284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef541008b0 0x7fef54108150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00b0a0 tx=0x7fef4c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.411+0000 7fef5930c700 1 -- 172.21.15.46:0/3159048284 shutdown_connections 2024-03-18T20:35:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.411+0000 7fef5930c700 1 --2- 172.21.15.46:0/3159048284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef541008b0 0x7fef54108150 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.411+0000 7fef5930c700 1 --2- 172.21.15.46:0/3159048284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef541012a0 0x7fef541016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.411+0000 7fef5930c700 1 --2- 172.21.15.46:0/3159048284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef540ffef0 0x7fef54100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.411+0000 7fef5930c700 1 -- 172.21.15.46:0/3159048284 >> 172.21.15.46:0/3159048284 conn(0x7fef540fa980 msgr2=0x7fef540fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.411+0000 7fef5930c700 1 -- 172.21.15.46:0/3159048284 shutdown_connections 2024-03-18T20:35:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.411+0000 7fef5930c700 1 -- 172.21.15.46:0/3159048284 wait complete. 2024-03-18T20:35:44.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.412+0000 7fef5930c700 1 Processor -- start 2024-03-18T20:35:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef5930c700 1 -- start start 2024-03-18T20:35:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef5930c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef540ffef0 0x7fef540ff960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef5930c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef541008b0 0x7fef5419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef5930c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef541012a0 0x7fef5419e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:44.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef5930c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5419e880 con 0x7fef540ffef0 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef5930c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5419e9f0 con 0x7fef541012a0 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef5930c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5419ebb0 con 0x7fef541008b0 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef52ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef540ffef0 0x7fef540ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef52ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef540ffef0 0x7fef540ff960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48662/0 (socket says 172.21.15.46:48662) 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef52ffd700 1 -- 172.21.15.46:0/2421088100 learned_addr learned my addr 172.21.15.46:0/2421088100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef4a7fc700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef541008b0 0x7fef5419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef537fe700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef541012a0 0x7fef5419e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef52ffd700 1 -- 172.21.15.46:0/2421088100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef541008b0 msgr2=0x7fef5419ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef52ffd700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef541008b0 0x7fef5419ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef52ffd700 1 -- 172.21.15.46:0/2421088100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef541012a0 msgr2=0x7fef5419e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef52ffd700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef541012a0 0x7fef5419e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.413+0000 7fef52ffd700 1 -- 172.21.15.46:0/2421088100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c009d00 con 0x7fef540ffef0 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.414+0000 7fef52ffd700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef540ffef0 0x7fef540ff960 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fef4400e970 tx=0x7fef4400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.414+0000 7fef50ff9700 1 -- 172.21.15.46:0/2421088100 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef44009790 con 0x7fef540ffef0 2024-03-18T20:35:44.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.414+0000 7fef50ff9700 1 -- 172.21.15.46:0/2421088100 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef4400c7e0 con 0x7fef540ffef0 2024-03-18T20:35:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.414+0000 7fef5930c700 1 -- 172.21.15.46:0/2421088100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5419ee90 con 0x7fef540ffef0 2024-03-18T20:35:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.414+0000 7fef50ff9700 1 -- 172.21.15.46:0/2421088100 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef44003c70 con 0x7fef540ffef0 2024-03-18T20:35:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.414+0000 7fef5930c700 1 -- 172.21.15.46:0/2421088100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5419f2c0 con 0x7fef540ffef0 2024-03-18T20:35:44.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.415+0000 7fef5930c700 1 -- 172.21.15.46:0/2421088100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef541070c0 con 0x7fef540ffef0 2024-03-18T20:35:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.416+0000 7fef50ff9700 1 -- 172.21.15.46:0/2421088100 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fef4400cc50 con 0x7fef540ffef0 2024-03-18T20:35:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.417+0000 7fef50ff9700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fef400a11b0 0x7fef400a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.417+0000 7fef4a7fc700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fef400a11b0 0x7fef400a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.417+0000 7fef50ff9700 1 -- 172.21.15.46:0/2421088100 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fef44014070 con 0x7fef540ffef0 2024-03-18T20:35:44.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.417+0000 7fef4a7fc700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fef400a11b0 0x7fef400a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fef540ffec0 tx=0x7fef3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:44.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.421+0000 7fef50ff9700 1 -- 172.21.15.46:0/2421088100 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fef440cee60 con 0x7fef540ffef0 2024-03-18T20:35:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:44 smithi073 ceph-mon[47384]: pgmap v253: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 17 B/s, 0 objects/s recovering 2024-03-18T20:35:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:44 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1939181188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:44 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/316034606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:44 smithi155 ceph-mon[48344]: pgmap v253: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 17 B/s, 0 objects/s recovering 2024-03-18T20:35:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:44 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1939181188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:44 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/316034606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:44 smithi046 ceph-mon[43495]: pgmap v253: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 17 B/s, 0 objects/s recovering 2024-03-18T20:35:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:44 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1939181188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:44 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/316034606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:44.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.637+0000 7fef5930c700 1 -- 172.21.15.46:0/2421088100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef5402ccb0 con 0x7fef540ffef0 2024-03-18T20:35:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.637+0000 7fef50ff9700 1 -- 172.21.15.46:0/2421088100 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef44099d10 con 0x7fef540ffef0 2024-03-18T20:35:44.641 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-18T20:35:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.640+0000 7fef5930c700 1 -- 172.21.15.46:0/2421088100 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fef400a11b0 msgr2=0x7fef400a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.640+0000 7fef5930c700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fef400a11b0 0x7fef400a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fef540ffec0 tx=0x7fef3c009260 comp rx=0 tx=0).stop 2024-03-18T20:35:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.640+0000 7fef5930c700 1 -- 172.21.15.46:0/2421088100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef540ffef0 msgr2=0x7fef540ff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.640+0000 7fef5930c700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef540ffef0 0x7fef540ff960 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fef4400e970 tx=0x7fef4400ec80 comp rx=0 tx=0).stop 2024-03-18T20:35:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.640+0000 7fef5930c700 1 -- 172.21.15.46:0/2421088100 shutdown_connections 2024-03-18T20:35:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.640+0000 7fef5930c700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fef400a11b0 0x7fef400a3670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.640+0000 7fef5930c700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef540ffef0 0x7fef540ff960 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.640+0000 7fef5930c700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef541012a0 0x7fef5419e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.640+0000 7fef5930c700 1 --2- 172.21.15.46:0/2421088100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef541008b0 0x7fef5419ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.640+0000 7fef5930c700 1 -- 172.21.15.46:0/2421088100 >> 172.21.15.46:0/2421088100 conn(0x7fef540fa980 msgr2=0x7fef54104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.640+0000 7fef5930c700 1 -- 172.21.15.46:0/2421088100 shutdown_connections 2024-03-18T20:35:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.640+0000 7fef5930c700 1 -- 172.21.15.46:0/2421088100 wait complete. 2024-03-18T20:35:44.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T20:35:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.794+0000 7fde4b8de700 1 -- 172.21.15.46:0/3103866447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde44100d80 msgr2=0x7fde441011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.794+0000 7fde4b8de700 1 --2- 172.21.15.46:0/3103866447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde44100d80 0x7fde441011a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fde380099c0 tx=0x7fde38009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.794+0000 7fde4b8de700 1 -- 172.21.15.46:0/3103866447 shutdown_connections 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.794+0000 7fde4b8de700 1 --2- 172.21.15.46:0/3103866447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde44100d80 0x7fde441011a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.794+0000 7fde4b8de700 1 --2- 172.21.15.46:0/3103866447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde44102910 0x7fde441091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.794+0000 7fde4b8de700 1 --2- 172.21.15.46:0/3103866447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde44101f70 0x7fde441023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.794+0000 7fde4b8de700 1 -- 172.21.15.46:0/3103866447 >> 172.21.15.46:0/3103866447 conn(0x7fde440fc2f0 msgr2=0x7fde440fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.794+0000 7fde4b8de700 1 -- 172.21.15.46:0/3103866447 shutdown_connections 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.794+0000 7fde4b8de700 1 -- 172.21.15.46:0/3103866447 wait complete. 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde4b8de700 1 Processor -- start 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde4b8de700 1 -- start start 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde4b8de700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde44100d80 0x7fde44074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde4b8de700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde44101f70 0x7fde440731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde4b8de700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde44102910 0x7fde440736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde4b8de700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde44073e80 con 0x7fde44101f70 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde4b8de700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde44073ff0 con 0x7fde44100d80 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde4b8de700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde441a0d30 con 0x7fde44102910 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde48e79700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde44101f70 0x7fde440731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde48e79700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde44101f70 0x7fde440731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48678/0 (socket says 172.21.15.46:48678) 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde48e79700 1 -- 172.21.15.46:0/2011854562 learned_addr learned my addr 172.21.15.46:0/2011854562 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde4967a700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde44100d80 0x7fde44074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.795+0000 7fde49e7b700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde44102910 0x7fde440736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.796+0000 7fde48e79700 1 -- 172.21.15.46:0/2011854562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde44102910 msgr2=0x7fde440736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.796+0000 7fde48e79700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde44102910 0x7fde440736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.796+0000 7fde48e79700 1 -- 172.21.15.46:0/2011854562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde44100d80 msgr2=0x7fde44074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.796+0000 7fde48e79700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde44100d80 0x7fde44074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.796+0000 7fde48e79700 1 -- 172.21.15.46:0/2011854562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde380095f0 con 0x7fde44101f70 2024-03-18T20:35:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.796+0000 7fde48e79700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde44101f70 0x7fde440731a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fde4000e9a0 tx=0x7fde4000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.796+0000 7fde367fc700 1 -- 172.21.15.46:0/2011854562 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40009790 con 0x7fde44101f70 2024-03-18T20:35:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.796+0000 7fde367fc700 1 -- 172.21.15.46:0/2011854562 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde4000ce70 con 0x7fde44101f70 2024-03-18T20:35:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.796+0000 7fde4b8de700 1 -- 172.21.15.46:0/2011854562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde441a0fc0 con 0x7fde44101f70 2024-03-18T20:35:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.796+0000 7fde367fc700 1 -- 172.21.15.46:0/2011854562 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40018420 con 0x7fde44101f70 2024-03-18T20:35:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.796+0000 7fde4b8de700 1 -- 172.21.15.46:0/2011854562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde441a1490 con 0x7fde44101f70 2024-03-18T20:35:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.797+0000 7fde4b8de700 1 -- 172.21.15.46:0/2011854562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde44108270 con 0x7fde44101f70 2024-03-18T20:35:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.799+0000 7fde367fc700 1 -- 172.21.15.46:0/2011854562 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fde400106f0 con 0x7fde44101f70 2024-03-18T20:35:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.800+0000 7fde367fc700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde300a1230 0x7fde300a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.800+0000 7fde4967a700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde300a1230 0x7fde300a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.800+0000 7fde367fc700 1 -- 172.21.15.46:0/2011854562 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fde40014070 con 0x7fde44101f70 2024-03-18T20:35:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.800+0000 7fde4967a700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde300a1230 0x7fde300a36f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fde38009990 tx=0x7fde38000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:44.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:44.804+0000 7fde367fc700 1 -- 172.21.15.46:0/2011854562 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fde40096590 con 0x7fde44101f70 2024-03-18T20:35:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.018+0000 7fde4b8de700 1 -- 172.21.15.46:0/2011854562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde4402d130 con 0x7fde44101f70 2024-03-18T20:35:45.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.018+0000 7fde367fc700 1 -- 172.21.15.46:0/2011854562 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde4402d130 con 0x7fde44101f70 2024-03-18T20:35:45.022 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-18T20:35:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.020+0000 7fde4b8de700 1 -- 172.21.15.46:0/2011854562 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde300a1230 msgr2=0x7fde300a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.020+0000 7fde4b8de700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde300a1230 0x7fde300a36f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fde38009990 tx=0x7fde38000bc0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.020+0000 7fde4b8de700 1 -- 172.21.15.46:0/2011854562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde44101f70 msgr2=0x7fde440731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.020+0000 7fde4b8de700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde44101f70 0x7fde440731a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fde4000e9a0 tx=0x7fde4000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.020+0000 7fde4b8de700 1 -- 172.21.15.46:0/2011854562 shutdown_connections 2024-03-18T20:35:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.020+0000 7fde4b8de700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fde300a1230 0x7fde300a36f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.020+0000 7fde4b8de700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde44102910 0x7fde440736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.020+0000 7fde4b8de700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde44100d80 0x7fde44074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.020+0000 7fde4b8de700 1 --2- 172.21.15.46:0/2011854562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde44101f70 0x7fde440731a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.020+0000 7fde4b8de700 1 -- 172.21.15.46:0/2011854562 >> 172.21.15.46:0/2011854562 conn(0x7fde440fc2f0 msgr2=0x7fde441053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:45.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.020+0000 7fde4b8de700 1 -- 172.21.15.46:0/2011854562 shutdown_connections 2024-03-18T20:35:45.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.020+0000 7fde4b8de700 1 -- 172.21.15.46:0/2011854562 wait complete. 2024-03-18T20:35:45.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:35:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.174+0000 7fad25070700 1 -- 172.21.15.46:0/1908345898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad201016f0 msgr2=0x7fad20107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.174+0000 7fad25070700 1 --2- 172.21.15.46:0/1908345898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad201016f0 0x7fad20107fa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fad1800b0d0 tx=0x7fad1800b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.174+0000 7fad25070700 1 -- 172.21.15.46:0/1908345898 shutdown_connections 2024-03-18T20:35:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.174+0000 7fad25070700 1 --2- 172.21.15.46:0/1908345898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad201016f0 0x7fad20107fa0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.174+0000 7fad25070700 1 --2- 172.21.15.46:0/1908345898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad200ffb60 0x7fad200fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.174+0000 7fad25070700 1 --2- 172.21.15.46:0/1908345898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad20100d50 0x7fad201011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.174+0000 7fad25070700 1 -- 172.21.15.46:0/1908345898 >> 172.21.15.46:0/1908345898 conn(0x7fad200fb0d0 msgr2=0x7fad200fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.174+0000 7fad25070700 1 -- 172.21.15.46:0/1908345898 shutdown_connections 2024-03-18T20:35:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.174+0000 7fad25070700 1 -- 172.21.15.46:0/1908345898 wait complete. 2024-03-18T20:35:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.174+0000 7fad25070700 1 Processor -- start 2024-03-18T20:35:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad25070700 1 -- start start 2024-03-18T20:35:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad25070700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad200ffb60 0x7fad20103a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad25070700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad20100d50 0x7fad20102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad25070700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad201016f0 0x7fad20102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad25070700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad20102d70 con 0x7fad20100d50 2024-03-18T20:35:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad25070700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad20102ee0 con 0x7fad200ffb60 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad25070700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad201a0db0 con 0x7fad201016f0 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad1e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad20100d50 0x7fad20102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad1e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad20100d50 0x7fad20102130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48698/0 (socket says 172.21.15.46:48698) 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad1e59c700 1 -- 172.21.15.46:0/1158321552 learned_addr learned my addr 172.21.15.46:0/1158321552 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad1f59e700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad201016f0 0x7fad20102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad1ed9d700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad200ffb60 0x7fad20103a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad1e59c700 1 -- 172.21.15.46:0/1158321552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad201016f0 msgr2=0x7fad20102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad1e59c700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad201016f0 0x7fad20102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad1e59c700 1 -- 172.21.15.46:0/1158321552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad200ffb60 msgr2=0x7fad20103a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad1e59c700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad200ffb60 0x7fad20103a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.175+0000 7fad1e59c700 1 -- 172.21.15.46:0/1158321552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad18009d00 con 0x7fad20100d50 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.176+0000 7fad1e59c700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad20100d50 0x7fad20102130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fad1400d950 tx=0x7fad1400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.176+0000 7fad07fff700 1 -- 172.21.15.46:0/1158321552 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1400f890 con 0x7fad20100d50 2024-03-18T20:35:45.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.176+0000 7fad25070700 1 -- 172.21.15.46:0/1158321552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad201a0fb0 con 0x7fad20100d50 2024-03-18T20:35:45.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.176+0000 7fad07fff700 1 -- 172.21.15.46:0/1158321552 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad14010460 con 0x7fad20100d50 2024-03-18T20:35:45.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.176+0000 7fad07fff700 1 -- 172.21.15.46:0/1158321552 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1400e610 con 0x7fad20100d50 2024-03-18T20:35:45.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.176+0000 7fad25070700 1 -- 172.21.15.46:0/1158321552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad201a1490 con 0x7fad20100d50 2024-03-18T20:35:45.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.177+0000 7fad25070700 1 -- 172.21.15.46:0/1158321552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad20067a10 con 0x7fad20100d50 2024-03-18T20:35:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.178+0000 7fad07fff700 1 -- 172.21.15.46:0/1158321552 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fad140108d0 con 0x7fad20100d50 2024-03-18T20:35:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.179+0000 7fad07fff700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fad080a11b0 0x7fad080a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.179+0000 7fad1ed9d700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fad080a11b0 0x7fad080a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.179+0000 7fad07fff700 1 -- 172.21.15.46:0/1158321552 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fad140cc7f0 con 0x7fad20100d50 2024-03-18T20:35:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.180+0000 7fad1ed9d700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fad080a11b0 0x7fad080a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fad20100bb0 tx=0x7fad10007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:45.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.182+0000 7fad07fff700 1 -- 172.21.15.46:0/1158321552 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fad14095dd0 con 0x7fad20100d50 2024-03-18T20:35:45.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:45 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3801520421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:45.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:45 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2421088100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:45.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:45 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2011854562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.448+0000 7fad25070700 1 -- 172.21.15.46:0/1158321552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad201a1a70 con 0x7fad20100d50 2024-03-18T20:35:45.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.449+0000 7fad07fff700 1 -- 172.21.15.46:0/1158321552 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10909 (secure 0 0 0) 0x7fad14016090 con 0x7fad20100d50 2024-03-18T20:35:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:45.452 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:34:45.796438+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:35:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.451+0000 7fad25070700 1 -- 172.21.15.46:0/1158321552 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fad080a11b0 msgr2=0x7fad080a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.451+0000 7fad25070700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fad080a11b0 0x7fad080a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fad20100bb0 tx=0x7fad10007400 comp rx=0 tx=0).stop 2024-03-18T20:35:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.451+0000 7fad25070700 1 -- 172.21.15.46:0/1158321552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad20100d50 msgr2=0x7fad20102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.451+0000 7fad25070700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad20100d50 0x7fad20102130 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fad1400d950 tx=0x7fad1400dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.451+0000 7fad25070700 1 -- 172.21.15.46:0/1158321552 shutdown_connections 2024-03-18T20:35:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.451+0000 7fad25070700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fad080a11b0 0x7fad080a3670 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.451+0000 7fad25070700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad201016f0 0x7fad20102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.451+0000 7fad25070700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad200ffb60 0x7fad20103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.451+0000 7fad25070700 1 --2- 172.21.15.46:0/1158321552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad20100d50 0x7fad20102130 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.451+0000 7fad25070700 1 -- 172.21.15.46:0/1158321552 >> 172.21.15.46:0/1158321552 conn(0x7fad200fb0d0 msgr2=0x7fad200fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.451+0000 7fad25070700 1 -- 172.21.15.46:0/1158321552 shutdown_connections 2024-03-18T20:35:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.451+0000 7fad25070700 1 -- 172.21.15.46:0/1158321552 wait complete. 2024-03-18T20:35:45.456 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-03-18T20:35:45.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:35:45.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:45 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3801520421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:45.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:45 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2421088100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:45.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:45 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2011854562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:45 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3801520421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:45 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2421088100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:45 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2011854562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.603+0000 7f1f1f314700 1 -- 172.21.15.46:0/820911414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f18100d50 msgr2=0x7f1f181011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.603+0000 7f1f1f314700 1 --2- 172.21.15.46:0/820911414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f18100d50 0x7f1f181011b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1f080099c0 tx=0x7f1f08009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.603+0000 7f1f1f314700 1 -- 172.21.15.46:0/820911414 shutdown_connections 2024-03-18T20:35:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.603+0000 7f1f1f314700 1 --2- 172.21.15.46:0/820911414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f181016f0 0x7f1f18107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.603+0000 7f1f1f314700 1 --2- 172.21.15.46:0/820911414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f180ffb60 0x7f1f180fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.603+0000 7f1f1f314700 1 --2- 172.21.15.46:0/820911414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f18100d50 0x7f1f181011b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.603+0000 7f1f1f314700 1 -- 172.21.15.46:0/820911414 >> 172.21.15.46:0/820911414 conn(0x7f1f180fb0d0 msgr2=0x7f1f180fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.603+0000 7f1f1f314700 1 -- 172.21.15.46:0/820911414 shutdown_connections 2024-03-18T20:35:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.603+0000 7f1f1f314700 1 -- 172.21.15.46:0/820911414 wait complete. 2024-03-18T20:35:45.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1f314700 1 Processor -- start 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1f314700 1 -- start start 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1f314700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f180ffb60 0x7f1f1810a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1f314700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f18100d50 0x7f1f18108ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1f314700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f181016f0 0x7f1f18108ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1f314700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f18109790 con 0x7f1f181016f0 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1f314700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f18109900 con 0x7f1f18100d50 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1f314700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f181a0c60 con 0x7f1f180ffb60 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1d8b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f181016f0 0x7f1f18108ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1d8b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f181016f0 0x7f1f18108ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34150/0 (socket says 172.21.15.46:34150) 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1d8b1700 1 -- 172.21.15.46:0/2429583678 learned_addr learned my addr 172.21.15.46:0/2429583678 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1d0b0700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f180ffb60 0x7f1f1810a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.604+0000 7f1f1c8af700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f18100d50 0x7f1f18108ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.605+0000 7f1f1d8b1700 1 -- 172.21.15.46:0/2429583678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f180ffb60 msgr2=0x7f1f1810a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.605+0000 7f1f1d8b1700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f180ffb60 0x7f1f1810a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.605+0000 7f1f1d8b1700 1 -- 172.21.15.46:0/2429583678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f18100d50 msgr2=0x7f1f18108ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.605+0000 7f1f1d8b1700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f18100d50 0x7f1f18108ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.605+0000 7f1f1d8b1700 1 -- 172.21.15.46:0/2429583678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f080095f0 con 0x7f1f181016f0 2024-03-18T20:35:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.605+0000 7f1f1d8b1700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f181016f0 0x7f1f18108ff0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1f10007ee0 tx=0x7f1f1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.605+0000 7f1f0e7fc700 1 -- 172.21.15.46:0/2429583678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10017070 con 0x7f1f181016f0 2024-03-18T20:35:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.605+0000 7f1f0e7fc700 1 -- 172.21.15.46:0/2429583678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f1000f040 con 0x7f1f181016f0 2024-03-18T20:35:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.605+0000 7f1f1f314700 1 -- 172.21.15.46:0/2429583678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f181a0ef0 con 0x7f1f181016f0 2024-03-18T20:35:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.605+0000 7f1f0e7fc700 1 -- 172.21.15.46:0/2429583678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f100054b0 con 0x7f1f181016f0 2024-03-18T20:35:45.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.605+0000 7f1f1f314700 1 -- 172.21.15.46:0/2429583678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f181a1320 con 0x7f1f181016f0 2024-03-18T20:35:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.606+0000 7f1f1f314700 1 -- 172.21.15.46:0/2429583678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f18106f90 con 0x7f1f181016f0 2024-03-18T20:35:45.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.607+0000 7f1f0e7fc700 1 -- 172.21.15.46:0/2429583678 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1f10004ad0 con 0x7f1f181016f0 2024-03-18T20:35:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.608+0000 7f1f0e7fc700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1f040a11b0 0x7f1f040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.608+0000 7f1f1d0b0700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1f040a11b0 0x7f1f040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.608+0000 7f1f0e7fc700 1 -- 172.21.15.46:0/2429583678 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1f100cc2a0 con 0x7f1f181016f0 2024-03-18T20:35:45.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.608+0000 7f1f1d0b0700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1f040a11b0 0x7f1f040a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f18100bb0 tx=0x7f1f1400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:45.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.612+0000 7f1f0e7fc700 1 -- 172.21.15.46:0/2429583678 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1f10095880 con 0x7f1f181016f0 2024-03-18T20:35:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.844+0000 7f1f1f314700 1 -- 172.21.15.46:0/2429583678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f1802ccb0 con 0x7f1f181016f0 2024-03-18T20:35:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.845+0000 7f1f0e7fc700 1 -- 172.21.15.46:0/2429583678 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22503 (secure 0 0 0) 0x7f1f10098ea0 con 0x7f1f181016f0 2024-03-18T20:35:45.849 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:45.849 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:35:38.108050+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:35:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.848+0000 7f1f1f314700 1 -- 172.21.15.46:0/2429583678 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1f040a11b0 msgr2=0x7f1f040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.848+0000 7f1f1f314700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1f040a11b0 0x7f1f040a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f18100bb0 tx=0x7f1f1400b410 comp rx=0 tx=0).stop 2024-03-18T20:35:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.848+0000 7f1f1f314700 1 -- 172.21.15.46:0/2429583678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f181016f0 msgr2=0x7f1f18108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.848+0000 7f1f1f314700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f181016f0 0x7f1f18108ff0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1f10007ee0 tx=0x7f1f1000d3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.848+0000 7f1f1f314700 1 -- 172.21.15.46:0/2429583678 shutdown_connections 2024-03-18T20:35:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.848+0000 7f1f1f314700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f1f040a11b0 0x7f1f040a3670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.848+0000 7f1f1f314700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f181016f0 0x7f1f18108ff0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.848+0000 7f1f1f314700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f180ffb60 0x7f1f1810a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.848+0000 7f1f1f314700 1 --2- 172.21.15.46:0/2429583678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f18100d50 0x7f1f18108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.848+0000 7f1f1f314700 1 -- 172.21.15.46:0/2429583678 >> 172.21.15.46:0/2429583678 conn(0x7f1f180fb0d0 msgr2=0x7f1f181040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.848+0000 7f1f1f314700 1 -- 172.21.15.46:0/2429583678 shutdown_connections 2024-03-18T20:35:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:45.848+0000 7f1f1f314700 1 -- 172.21.15.46:0/2429583678 wait complete. 2024-03-18T20:35:45.866 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-18T20:34:38.785094+0000', 'modified': '2024-03-18T20:34:45.796438+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24536, 'mds_1': 24521, 'mds_2': 24514, 'mds_3': 24506, 'mds_4': 24446}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.46:6841/3470614253', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6840', 'nonce': 3470614253}, {'type': 'v1', 'addr': '172.21.15.46:6841', 'nonce': 3470614253}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.73:6833/2440572005', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 2440572005}, {'type': 'v1', 'addr': '172.21.15.73:6833', 'nonce': 2440572005}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6833/4207499009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 4207499009}, {'type': 'v1', 'addr': '172.21.15.155:6833', 'nonce': 4207499009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6835/2049469125', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 2049469125}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 2049469125}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6835/2573112997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 2573112997}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 2573112997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/1686098166', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 1686098166}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 1686098166}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.73:6837/2773013996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 2773013996}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 2773013996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.155:6839/3116856560', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 3116856560}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 3116856560}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'g', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.46:6839/1267173621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6838', 'nonce': 1267173621}, {'type': 'v1', 'addr': '172.21.15.46:6839', 'nonce': 1267173621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6839/2377859422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 2377859422}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 2377859422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-18T20:35:45.866 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-18T20:35:45.867 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-18T20:35:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.002+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/809772140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c68101840 msgr2=0x7f4c68103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.002+0000 7f4c6cb9f700 1 --2- 172.21.15.46:0/809772140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c68101840 0x7f4c68103c30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009990 tx=0x7f4c58009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.002+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/809772140 shutdown_connections 2024-03-18T20:35:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.002+0000 7f4c6cb9f700 1 --2- 172.21.15.46:0/809772140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c68101840 0x7f4c68103c30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.002+0000 7f4c6cb9f700 1 --2- 172.21.15.46:0/809772140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c68106aa0 0x7f4c68108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.002+0000 7f4c6cb9f700 1 --2- 172.21.15.46:0/809772140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c68104170 0x7f4c68106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.002+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/809772140 >> 172.21.15.46:0/809772140 conn(0x7f4c680fb1d0 msgr2=0x7f4c680fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.002+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/809772140 shutdown_connections 2024-03-18T20:35:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.002+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/809772140 wait complete. 2024-03-18T20:35:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.003+0000 7f4c6cb9f700 1 Processor -- start 2024-03-18T20:35:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.003+0000 7f4c6cb9f700 1 -- start start 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.003+0000 7f4c6cb9f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c68101840 0x7f4c68196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.003+0000 7f4c6cb9f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c68104170 0x7f4c68197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.003+0000 7f4c6cb9f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c68106aa0 0x7f4c6819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.003+0000 7f4c6cb9f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6819e7c0 con 0x7f4c68104170 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.003+0000 7f4c6cb9f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6819e930 con 0x7f4c68101840 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.003+0000 7f4c6cb9f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6819eaf0 con 0x7f4c68106aa0 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.003+0000 7f4c65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c68104170 0x7f4c68197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.003+0000 7f4c65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c68104170 0x7f4c68197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34176/0 (socket says 172.21.15.46:34176) 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.003+0000 7f4c65d9b700 1 -- 172.21.15.46:0/3121139637 learned_addr learned my addr 172.21.15.46:0/3121139637 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c6659c700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c68101840 0x7f4c68196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c66d9d700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c68106aa0 0x7f4c6819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c65d9b700 1 -- 172.21.15.46:0/3121139637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c68106aa0 msgr2=0x7f4c6819e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c68106aa0 0x7f4c6819e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c65d9b700 1 -- 172.21.15.46:0/3121139637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c68101840 msgr2=0x7f4c68196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c68101840 0x7f4c68196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c65d9b700 1 -- 172.21.15.46:0/3121139637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c580095f0 con 0x7f4c68104170 2024-03-18T20:35:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c68104170 0x7f4c68197210 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00d980 tx=0x7f4c5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c577fe700 1 -- 172.21.15.46:0/3121139637 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c004220 con 0x7f4c68104170 2024-03-18T20:35:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/3121139637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6819edd0 con 0x7f4c68104170 2024-03-18T20:35:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c577fe700 1 -- 172.21.15.46:0/3121139637 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c5c007640 con 0x7f4c68104170 2024-03-18T20:35:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c577fe700 1 -- 172.21.15.46:0/3121139637 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c005610 con 0x7f4c68104170 2024-03-18T20:35:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.004+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/3121139637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6819f220 con 0x7f4c68104170 2024-03-18T20:35:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.006+0000 7f4c577fe700 1 -- 172.21.15.46:0/3121139637 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4c5c004750 con 0x7f4c68104170 2024-03-18T20:35:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.007+0000 7f4c577fe700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4c500a11b0 0x7f4c500a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.007+0000 7f4c6659c700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4c500a11b0 0x7f4c500a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.007+0000 7f4c577fe700 1 -- 172.21.15.46:0/3121139637 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c5c0cc9a0 con 0x7f4c68104170 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c78d34b80 1 Processor -- start 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c6659c700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4c500a11b0 0x7f4c500a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800b680 tx=0x7f4c58005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c78d34b80 1 -- start start 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c78d34b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d6bd0 0x557ebe0d6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c78d34b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 0x557ebe0d7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c78d34b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe0d8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c78d34b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe02f940 con 0x557ebe0d7530 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c78d34b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe02d300 con 0x557ebe0d83d0 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c78d34b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe028bf0 con 0x557ebe0d6bd0 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 0x557ebe0d7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 0x557ebe0d7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34180/0 (socket says 172.21.15.46:34180) 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c65d9b700 1 -- 172.21.15.46:0/1812110264 learned_addr learned my addr 172.21.15.46:0/1812110264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c6659c700 1 --2- 172.21.15.46:0/1812110264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d6bd0 0x557ebe0d6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.008+0000 7f4c66d9d700 1 --2- 172.21.15.46:0/1812110264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe0d8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.009+0000 7f4c65d9b700 1 -- 172.21.15.46:0/1812110264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d6bd0 msgr2=0x557ebe0d6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.009+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/1812110264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d6bd0 0x557ebe0d6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.009+0000 7f4c65d9b700 1 -- 172.21.15.46:0/1812110264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 msgr2=0x557ebe0d8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.009+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/1812110264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe0d8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.009+0000 7f4c65d9b700 1 -- 172.21.15.46:0/1812110264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ebdeaa360 con 0x557ebe0d7530 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.009+0000 7f4c66d9d700 1 --2- 172.21.15.46:0/1812110264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe0d8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.009+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/1812110264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 0x557ebe0d7df0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0a1b90 tx=0x7f4c5c09a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5349131f77ea01a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.009+0000 7f4c54ff9700 1 -- 172.21.15.46:0/1812110264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0a20c0 con 0x557ebe0d7530 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.009+0000 7f4c54ff9700 1 -- 172.21.15.46:0/1812110264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c5c097fe0 con 0x557ebe0d7530 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.009+0000 7f4c54ff9700 1 -- 172.21.15.46:0/1812110264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c099400 con 0x557ebe0d7530 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.010+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1812110264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 msgr2=0x557ebe0d7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.010+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1812110264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 0x557ebe0d7df0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c0a1b90 tx=0x7f4c5c09a410 comp rx=0 tx=0).stop 2024-03-18T20:35:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.010+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1812110264 shutdown_connections 2024-03-18T20:35:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.010+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1812110264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe0d8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.010+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1812110264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d6bd0 0x557ebe0d6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.010+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1812110264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 0x557ebe0d7df0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.010+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1812110264 >> 172.21.15.46:0/1812110264 conn(0x557ebe04c1d0 msgr2=0x557ebe0d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.010+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1812110264 shutdown_connections 2024-03-18T20:35:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.010+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1812110264 wait complete. 2024-03-18T20:35:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.010+0000 7f4c78d34b80 1 -- start start 2024-03-18T20:35:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.010+0000 7f4c78d34b80 1 Processor -- start 2024-03-18T20:35:46.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c78d34b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d6bd0 0x557ebe0e09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c78d34b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 0x557ebe0e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c78d34b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d83d0 0x557ebe0e7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c78d34b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebdfc4d30 con 0x557ebe0d7530 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c78d34b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebdfc49d0 con 0x557ebe0d6bd0 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c78d34b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebdeac650 con 0x557ebe0d83d0 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c78d34b80 10 client.0 _ll_register_callbacks cb 0x557ebde30350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 0x557ebe0e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 0x557ebe0e0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34192/0 (socket says 172.21.15.46:34192) 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c65d9b700 1 -- 172.21.15.46:0/1226187775 learned_addr learned my addr 172.21.15.46:0/1226187775 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c6659c700 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d6bd0 0x557ebe0e09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c66d9d700 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d83d0 0x557ebe0e7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c65d9b700 1 -- 172.21.15.46:0/1226187775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d83d0 msgr2=0x557ebe0e7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d83d0 0x557ebe0e7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c65d9b700 1 -- 172.21.15.46:0/1226187775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d6bd0 msgr2=0x557ebe0e09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d6bd0 0x557ebe0e09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c65d9b700 1 -- 172.21.15.46:0/1226187775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6819f220 con 0x557ebe0d7530 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 0x557ebe0e0ef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c099bd0 tx=0x7f4c5c099880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a66eb218c12db63 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c4dffb700 1 -- 172.21.15.46:0/1226187775 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0993a0 con 0x557ebe0d7530 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c4dffb700 1 -- 172.21.15.46:0/1226187775 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c5c0a20c0 con 0x557ebe0d7530 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.011+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1226187775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ebe0c9a80 con 0x557ebe0d7530 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.012+0000 7f4c4dffb700 1 -- 172.21.15.46:0/1226187775 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c5c0a1c30 con 0x557ebe0d7530 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.012+0000 7f4c78d34b80 10 client.15240 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.012+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1226187775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557ebde56b10 con 0x557ebe0d7530 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.012+0000 7f4c4dffb700 1 -- 172.21.15.46:0/1226187775 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c5c07c080 con 0x557ebe0d7530 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.012+0000 7f4c4dffb700 1 -- 172.21.15.46:0/1226187775 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4c5c095a30 con 0x557ebe0d7530 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.012+0000 7f4c78d34b80 10 client.15240 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.012+0000 7f4c78d34b80 10 client.15240 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-18T20:35:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.012+0000 7f4c78d34b80 20 client.15240 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.012+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x557ebe0eb780 0x557ebe0f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.012+0000 7f4c78d34b80 4 client.15240 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.012+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1226187775 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ebe0216b0 con 0x557ebe0eb780 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.012+0000 7f4c6659c700 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x557ebe0eb780 0x557ebe0f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.013+0000 7f4c4dffb700 10 client.15240 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.013+0000 7f4c6659c700 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x557ebe0eb780 0x557ebe0f0a20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=73f547c988a49bef server_cookie=98865a6a571d0a48 in_seq=0 out_seq=0 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.013+0000 7f4c4dffb700 1 -- 172.21.15.46:0/1226187775 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557ebe0216b0 con 0x557ebe0eb780 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.013+0000 7f4c4dffb700 10 client.15240 handle_command_reply: tid=0 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.013+0000 7f4c78d34b80 1 client.15240 shutdown 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.013+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1226187775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 msgr2=0x557ebe0e0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.013+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 0x557ebe0e0ef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c099bd0 tx=0x7f4c5c099880 comp rx=0 tx=0).stop 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.014+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1226187775 shutdown_connections 2024-03-18T20:35:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.014+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x557ebe0eb780 0x557ebe0f0a20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.014+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d83d0 0x557ebe0e7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.014+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d6bd0 0x557ebe0e09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.014+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1226187775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d7530 0x557ebe0e0ef0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.014+0000 7f4c4dffb700 0 client.15240 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:35:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.014+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1226187775 >> 172.21.15.46:0/1226187775 conn(0x557ebe04c1d0 msgr2=0x557ebe0d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.014+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1226187775 shutdown_connections 2024-03-18T20:35:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.014+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1226187775 wait complete. 2024-03-18T20:35:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.014+0000 7f4c78d34b80 20 client.15240 trim_cache size 0 max 16384 2024-03-18T20:35:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c78d34b80 1 Processor -- start 2024-03-18T20:35:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c78d34b80 1 -- start start 2024-03-18T20:35:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c78d34b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 0x557ebe0f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c78d34b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d7530 0x557ebe0e7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c78d34b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe0f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c78d34b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe021990 con 0x557ebe0d6bd0 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c78d34b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe025850 con 0x557ebe0d83d0 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c78d34b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe09c8f0 con 0x557ebe0d7530 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c6659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 0x557ebe0f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c6659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 0x557ebe0f4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34208/0 (socket says 172.21.15.46:34208) 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c6659c700 1 -- 172.21.15.46:0/1146047481 learned_addr learned my addr 172.21.15.46:0/1146047481 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c66d9d700 1 --2- 172.21.15.46:0/1146047481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe0f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/1146047481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d7530 0x557ebe0e7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c6659c700 1 -- 172.21.15.46:0/1146047481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d7530 msgr2=0x557ebe0e7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c6659c700 1 --2- 172.21.15.46:0/1146047481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d7530 0x557ebe0e7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c6659c700 1 -- 172.21.15.46:0/1146047481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 msgr2=0x557ebe0f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c6659c700 1 --2- 172.21.15.46:0/1146047481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe0f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c6659c700 1 -- 172.21.15.46:0/1146047481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ebdfeee50 con 0x557ebe0d6bd0 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/1146047481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d7530 0x557ebe0e7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c66d9d700 1 --2- 172.21.15.46:0/1146047481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe0f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.033+0000 7f4c6659c700 1 --2- 172.21.15.46:0/1146047481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 0x557ebe0f4a40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4c580098e0 tx=0x7f4c58020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eefacc276b1d5e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.034+0000 7f4c557fa700 1 -- 172.21.15.46:0/1146047481 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5800bd90 con 0x557ebe0d6bd0 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.034+0000 7f4c557fa700 1 -- 172.21.15.46:0/1146047481 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c5801e040 con 0x557ebe0d6bd0 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.034+0000 7f4c557fa700 1 -- 172.21.15.46:0/1146047481 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58005020 con 0x557ebe0d6bd0 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.034+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1146047481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 msgr2=0x557ebe0f4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.034+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1146047481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 0x557ebe0f4a40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4c580098e0 tx=0x7f4c58020000 comp rx=0 tx=0).stop 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.034+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1146047481 shutdown_connections 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.034+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1146047481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 0x557ebe0f4a40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.034+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1146047481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe0f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.034+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/1146047481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d7530 0x557ebe0e7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.034+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1146047481 >> 172.21.15.46:0/1146047481 conn(0x557ebe04c1d0 msgr2=0x557ebe0db3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.034+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1146047481 shutdown_connections 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.034+0000 7f4c78d34b80 1 -- 172.21.15.46:0/1146047481 wait complete. 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c78d34b80 1 -- start start 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c78d34b80 1 Processor -- start 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c78d34b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 0x557ebe0e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c78d34b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d7530 0x557ebe0e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c78d34b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe140010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c78d34b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe140550 con 0x557ebe0d6bd0 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c78d34b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe140690 con 0x557ebe0d83d0 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c78d34b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebe1407d0 con 0x557ebe0d7530 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c78d34b80 10 client.0 _ll_register_callbacks cb 0x7f4c5c099650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c6659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 0x557ebe0e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c6659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 0x557ebe0e4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34216/0 (socket says 172.21.15.46:34216) 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c6659c700 1 -- 172.21.15.46:0/3388538646 learned_addr learned my addr 172.21.15.46:0/3388538646 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d7530 0x557ebe0e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c6659c700 1 -- 172.21.15.46:0/3388538646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d7530 msgr2=0x557ebe0e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c6659c700 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d7530 0x557ebe0e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c66d9d700 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c6659c700 1 -- 172.21.15.46:0/3388538646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 msgr2=0x557ebe140010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c6659c700 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe140010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.035+0000 7f4c6659c700 1 -- 172.21.15.46:0/3388538646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5802c040 con 0x557ebe0d6bd0 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c6659c700 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 0x557ebe0e4f20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4c580045f0 tx=0x7f4c58024aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3c0a0e58821a58 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c4dffb700 1 -- 172.21.15.46:0/3388538646 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58035300 con 0x557ebe0d6bd0 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c4dffb700 1 -- 172.21.15.46:0/3388538646 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c58035460 con 0x557ebe0d6bd0 2024-03-18T20:35:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c78d34b80 1 -- 172.21.15.46:0/3388538646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c5c097fe0 con 0x557ebe0d6bd0 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c4dffb700 1 -- 172.21.15.46:0/3388538646 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c5800f460 con 0x557ebe0d6bd0 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c78d34b80 10 client.15246 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c78d34b80 1 -- 172.21.15.46:0/3388538646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557ebdeaec80 con 0x557ebe0d6bd0 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c4dffb700 1 -- 172.21.15.46:0/3388538646 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c5c097fe0 con 0x557ebe0d6bd0 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c4dffb700 1 -- 172.21.15.46:0/3388538646 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4c5803c020 con 0x557ebe0d6bd0 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c78d34b80 10 client.15246 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c78d34b80 10 client.15246 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c78d34b80 20 client.15246 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.036+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x557ebe145100 0x557ebe141a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.037+0000 7f4c78d34b80 4 client.15246 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.037+0000 7f4c78d34b80 1 -- 172.21.15.46:0/3388538646 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557ebde77ca0 con 0x557ebe145100 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.037+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x557ebe145100 0x557ebe141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.037+0000 7f4c4dffb700 10 client.15246 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:35:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.037+0000 7f4c65d9b700 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x557ebe145100 0x557ebe141a30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a3bb808246fe885 server_cookie=6686311acdec7291 in_seq=0 out_seq=0 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.037+0000 7f4c4dffb700 1 -- 172.21.15.46:0/3388538646 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557ebde77ca0 con 0x557ebe145100 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.037+0000 7f4c4dffb700 10 client.15246 handle_command_reply: tid=0 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.037+0000 7f4c78d34b80 1 client.15246 shutdown 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c78d34b80 1 -- 172.21.15.46:0/3388538646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 msgr2=0x557ebe0e4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 0x557ebe0e4f20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4c580045f0 tx=0x7f4c58024aa0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c78d34b80 1 -- 172.21.15.46:0/3388538646 shutdown_connections 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x557ebe145100 0x557ebe141a30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ebe0d6bd0 0x557ebe0e4f20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ebe0d83d0 0x557ebe140010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c4dffb700 0 client.15246 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c78d34b80 1 --2- 172.21.15.46:0/3388538646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ebe0d7530 0x557ebe0e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c78d34b80 1 -- 172.21.15.46:0/3388538646 >> 172.21.15.46:0/3388538646 conn(0x557ebe04c1d0 msgr2=0x557ebe0f4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c78d34b80 1 -- 172.21.15.46:0/3388538646 shutdown_connections 2024-03-18T20:35:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c78d34b80 1 -- 172.21.15.46:0/3388538646 wait complete. 2024-03-18T20:35:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c78d34b80 20 client.15246 trim_cache size 0 max 16384 2024-03-18T20:35:46.049 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:35:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:35:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 4, 2024-03-18T20:35:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24446, 2024-03-18T20:35:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-18T20:35:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-18T20:35:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-18T20:35:46.049 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 61.254641984999999, 2024-03-18T20:35:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-03-18T20:35:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-03-18T20:35:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-03-18T20:35:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 91.211353563000003 2024-03-18T20:35:46.050 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:35:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/3121139637 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4c500a11b0 msgr2=0x7f4c500a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c6cb9f700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4c500a11b0 0x7f4c500a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800b680 tx=0x7f4c58005fd0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/3121139637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c68104170 msgr2=0x7f4c68197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c6cb9f700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c68104170 0x7f4c68197210 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00d980 tx=0x7f4c5c00dc90 comp rx=0 tx=0).stop 2024-03-18T20:35:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/3121139637 shutdown_connections 2024-03-18T20:35:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c6cb9f700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4c500a11b0 0x7f4c500a3670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c6cb9f700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c68106aa0 0x7f4c6819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c6cb9f700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c68101840 0x7f4c68196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c6cb9f700 1 --2- 172.21.15.46:0/3121139637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c68104170 0x7f4c68197210 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/3121139637 >> 172.21.15.46:0/3121139637 conn(0x7f4c680fb1d0 msgr2=0x7f4c68102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.038+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/3121139637 shutdown_connections 2024-03-18T20:35:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.039+0000 7f4c6cb9f700 1 -- 172.21.15.46:0/3121139637 wait complete. 2024-03-18T20:35:46.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-18T20:35:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.191+0000 7f7e742f4700 1 -- 172.21.15.46:0/1241597178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6c100e50 msgr2=0x7f7e6c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.191+0000 7f7e742f4700 1 --2- 172.21.15.46:0/1241597178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6c100e50 0x7f7e6c1012b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0099c0 tx=0x7f7e5c009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 -- 172.21.15.46:0/1241597178 shutdown_connections 2024-03-18T20:35:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 --2- 172.21.15.46:0/1241597178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e6c1017f0 0x7f7e6c1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 --2- 172.21.15.46:0/1241597178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c0ffc60 0x7f7e6c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 --2- 172.21.15.46:0/1241597178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6c100e50 0x7f7e6c1012b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 -- 172.21.15.46:0/1241597178 >> 172.21.15.46:0/1241597178 conn(0x7f7e6c0fb1d0 msgr2=0x7f7e6c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 -- 172.21.15.46:0/1241597178 shutdown_connections 2024-03-18T20:35:46.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 -- 172.21.15.46:0/1241597178 wait complete. 2024-03-18T20:35:46.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 Processor -- start 2024-03-18T20:35:46.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 -- start start 2024-03-18T20:35:46.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e6c0ffc60 0x7f7e6c196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c100e50 0x7f7e6c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6c1017f0 0x7f7e6c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c19e830 con 0x7f7e6c1017f0 2024-03-18T20:35:46.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c19e9a0 con 0x7f7e6c0ffc60 2024-03-18T20:35:46.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.192+0000 7f7e742f4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c19eb60 con 0x7f7e6c100e50 2024-03-18T20:35:46.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e72891700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6c1017f0 0x7f7e6c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e72891700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6c1017f0 0x7f7e6c19e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34232/0 (socket says 172.21.15.46:34232) 2024-03-18T20:35:46.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e72891700 1 -- 172.21.15.46:0/2497697005 learned_addr learned my addr 172.21.15.46:0/2497697005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e72090700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e6c0ffc60 0x7f7e6c196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e7188f700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c100e50 0x7f7e6c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e72891700 1 -- 172.21.15.46:0/2497697005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c100e50 msgr2=0x7f7e6c197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e72891700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c100e50 0x7f7e6c197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e72891700 1 -- 172.21.15.46:0/2497697005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e6c0ffc60 msgr2=0x7f7e6c196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e72891700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e6c0ffc60 0x7f7e6c196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e72891700 1 -- 172.21.15.46:0/2497697005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c0095f0 con 0x7f7e6c1017f0 2024-03-18T20:35:46.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e72891700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6c1017f0 0x7f7e6c19e260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400ef30 tx=0x7f7e6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e637fe700 1 -- 172.21.15.46:0/2497697005 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64018070 con 0x7f7e6c1017f0 2024-03-18T20:35:46.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e742f4700 1 -- 172.21.15.46:0/2497697005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6c19ede0 con 0x7f7e6c1017f0 2024-03-18T20:35:46.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e637fe700 1 -- 172.21.15.46:0/2497697005 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e6400f040 con 0x7f7e6c1017f0 2024-03-18T20:35:46.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e637fe700 1 -- 172.21.15.46:0/2497697005 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400cd00 con 0x7f7e6c1017f0 2024-03-18T20:35:46.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.193+0000 7f7e742f4700 1 -- 172.21.15.46:0/2497697005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c19f200 con 0x7f7e6c1017f0 2024-03-18T20:35:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.196+0000 7f7e637fe700 1 -- 172.21.15.46:0/2497697005 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f7e64003a00 con 0x7f7e6c1017f0 2024-03-18T20:35:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.197+0000 7f7e637fe700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7e580a11b0 0x7f7e580a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.197+0000 7f7e637fe700 1 -- 172.21.15.46:0/2497697005 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e640cc5f0 con 0x7f7e6c1017f0 2024-03-18T20:35:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.197+0000 7f7e72090700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7e580a11b0 0x7f7e580a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.197+0000 7f7e72090700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7e580a11b0 0x7f7e580a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c1981c0 tx=0x7f7e6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e80489b80 1 Processor -- start 2024-03-18T20:35:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e80489b80 1 -- start start 2024-03-18T20:35:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e80489b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2efe0 0x5634d7e2f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e80489b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2f940 0x5634d7e30200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e80489b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e307e0 0x5634d7e30c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e80489b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634d7d83940 con 0x5634d7e2f940 2024-03-18T20:35:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e80489b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634d7d81300 con 0x5634d7e2efe0 2024-03-18T20:35:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e80489b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634d7d7cbf0 con 0x5634d7e307e0 2024-03-18T20:35:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e7188f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2f940 0x5634d7e30200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e7188f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2f940 0x5634d7e30200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34240/0 (socket says 172.21.15.46:34240) 2024-03-18T20:35:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e7188f700 1 -- 172.21.15.46:0/2506275198 learned_addr learned my addr 172.21.15.46:0/2506275198 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e72090700 1 --2- 172.21.15.46:0/2506275198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2efe0 0x5634d7e2f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e7188f700 1 -- 172.21.15.46:0/2506275198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e307e0 msgr2=0x5634d7e30c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e72891700 1 --2- 172.21.15.46:0/2506275198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e307e0 0x5634d7e30c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e7188f700 1 --2- 172.21.15.46:0/2506275198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e307e0 0x5634d7e30c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e7188f700 1 -- 172.21.15.46:0/2506275198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2efe0 msgr2=0x5634d7e2f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.198+0000 7f7e7188f700 1 --2- 172.21.15.46:0/2506275198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2efe0 0x5634d7e2f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.199+0000 7f7e7188f700 1 -- 172.21.15.46:0/2506275198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634d7bfe360 con 0x5634d7e2f940 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.199+0000 7f7e72090700 1 --2- 172.21.15.46:0/2506275198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2efe0 0x5634d7e2f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.199+0000 7f7e7188f700 1 --2- 172.21.15.46:0/2506275198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2f940 0x5634d7e30200 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0058e0 tx=0x7f7e5c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925d2b607c234d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.199+0000 7f7e60da8700 1 -- 172.21.15.46:0/2506275198 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c00bbc0 con 0x5634d7e2f940 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.199+0000 7f7e60da8700 1 -- 172.21.15.46:0/2506275198 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e5c01e040 con 0x5634d7e2f940 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.199+0000 7f7e60da8700 1 -- 172.21.15.46:0/2506275198 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c003b80 con 0x5634d7e2f940 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.200+0000 7f7e80489b80 1 -- 172.21.15.46:0/2506275198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2f940 msgr2=0x5634d7e30200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.200+0000 7f7e80489b80 1 --2- 172.21.15.46:0/2506275198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2f940 0x5634d7e30200 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0058e0 tx=0x7f7e5c005c70 comp rx=0 tx=0).stop 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.200+0000 7f7e80489b80 1 -- 172.21.15.46:0/2506275198 shutdown_connections 2024-03-18T20:35:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.200+0000 7f7e80489b80 1 --2- 172.21.15.46:0/2506275198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e307e0 0x5634d7e30c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.200+0000 7f7e80489b80 1 --2- 172.21.15.46:0/2506275198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2efe0 0x5634d7e2f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.200+0000 7f7e80489b80 1 --2- 172.21.15.46:0/2506275198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2f940 0x5634d7e30200 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.200+0000 7f7e80489b80 1 -- 172.21.15.46:0/2506275198 >> 172.21.15.46:0/2506275198 conn(0x5634d7da01d0 msgr2=0x5634d7e25b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.200+0000 7f7e80489b80 1 -- 172.21.15.46:0/2506275198 shutdown_connections 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.200+0000 7f7e80489b80 1 -- 172.21.15.46:0/2506275198 wait complete. 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.200+0000 7f7e80489b80 1 -- start start 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.200+0000 7f7e80489b80 1 Processor -- start 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e80489b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2efe0 0x5634d7e38e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e80489b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2f940 0x5634d7e39360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e80489b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 0x5634d7e403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e80489b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634d7d18d30 con 0x5634d7e307e0 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e80489b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634d7d189d0 con 0x5634d7e2f940 2024-03-18T20:35:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e80489b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634d7c00650 con 0x5634d7e2efe0 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e80489b80 10 client.0 _ll_register_callbacks cb 0x5634d7b84350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e72891700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 0x5634d7e403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e72090700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2efe0 0x5634d7e38e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e7188f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2f940 0x5634d7e39360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e72891700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 0x5634d7e403a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34246/0 (socket says 172.21.15.46:34246) 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e72891700 1 -- 172.21.15.46:0/2635810117 learned_addr learned my addr 172.21.15.46:0/2635810117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e72891700 1 -- 172.21.15.46:0/2635810117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2efe0 msgr2=0x5634d7e38e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e72891700 1 --2- 172.21.15.46:0/2635810117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2efe0 0x5634d7e38e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e72891700 1 -- 172.21.15.46:0/2635810117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2f940 msgr2=0x5634d7e39360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e72891700 1 --2- 172.21.15.46:0/2635810117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2f940 0x5634d7e39360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e72891700 1 -- 172.21.15.46:0/2635810117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c1089f0 con 0x5634d7e307e0 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e72891700 1 --2- 172.21.15.46:0/2635810117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 0x5634d7e403a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7e64095e20 tx=0x7f7e64004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b5fa81e5cd54b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e55ffb700 1 -- 172.21.15.46:0/2635810117 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e640b92d0 con 0x5634d7e307e0 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e55ffb700 1 -- 172.21.15.46:0/2635810117 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e6409a020 con 0x5634d7e307e0 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.201+0000 7f7e80489b80 1 -- 172.21.15.46:0/2635810117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634d7e1da80 con 0x5634d7e307e0 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.202+0000 7f7e55ffb700 1 -- 172.21.15.46:0/2635810117 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e6400ce60 con 0x5634d7e307e0 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.202+0000 7f7e80489b80 10 client.15258 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.202+0000 7f7e80489b80 1 -- 172.21.15.46:0/2635810117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5634d7baab10 con 0x5634d7e307e0 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.202+0000 7f7e55ffb700 1 -- 172.21.15.46:0/2635810117 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5634d7e1da80 con 0x5634d7e307e0 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.202+0000 7f7e55ffb700 1 -- 172.21.15.46:0/2635810117 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7e64098e10 con 0x5634d7e307e0 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.202+0000 7f7e80489b80 10 client.15258 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.202+0000 7f7e80489b80 10 client.15258 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.202+0000 7f7e80489b80 20 client.15258 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.202+0000 7f7e80489b80 1 --2- 172.21.15.46:0/2635810117 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5634d7e43f90 0x5634d7e492b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.202+0000 7f7e80489b80 4 client.15258 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:35:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.202+0000 7f7e80489b80 1 -- 172.21.15.46:0/2635810117 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634d7d756b0 con 0x5634d7e43f90 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.202+0000 7f7e72090700 1 --2- 172.21.15.46:0/2635810117 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5634d7e43f90 0x5634d7e492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.203+0000 7f7e72090700 1 --2- 172.21.15.46:0/2635810117 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5634d7e43f90 0x5634d7e492b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a6463fd102f91d29 server_cookie=ffe202051e613eb3 in_seq=0 out_seq=0 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.203+0000 7f7e55ffb700 10 client.15258 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.203+0000 7f7e55ffb700 1 -- 172.21.15.46:0/2635810117 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5634d7d756b0 con 0x5634d7e43f90 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.203+0000 7f7e55ffb700 10 client.15258 handle_command_reply: tid=0 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e80489b80 1 client.15258 shutdown 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e80489b80 1 -- 172.21.15.46:0/2635810117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 msgr2=0x5634d7e403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e80489b80 1 --2- 172.21.15.46:0/2635810117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 0x5634d7e403a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7e64095e20 tx=0x7f7e64004b50 comp rx=0 tx=0).stop 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e80489b80 1 -- 172.21.15.46:0/2635810117 shutdown_connections 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e80489b80 1 --2- 172.21.15.46:0/2635810117 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5634d7e43f90 0x5634d7e492b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e80489b80 1 --2- 172.21.15.46:0/2635810117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 0x5634d7e403a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e80489b80 1 --2- 172.21.15.46:0/2635810117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2efe0 0x5634d7e38e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e80489b80 1 --2- 172.21.15.46:0/2635810117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2f940 0x5634d7e39360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e55ffb700 0 client.15258 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:35:46.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e80489b80 1 -- 172.21.15.46:0/2635810117 >> 172.21.15.46:0/2635810117 conn(0x5634d7da01d0 msgr2=0x5634d7e251a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e80489b80 1 -- 172.21.15.46:0/2635810117 shutdown_connections 2024-03-18T20:35:46.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e80489b80 1 -- 172.21.15.46:0/2635810117 wait complete. 2024-03-18T20:35:46.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.204+0000 7f7e80489b80 20 client.15258 trim_cache size 0 max 16384 2024-03-18T20:35:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e80489b80 1 Processor -- start 2024-03-18T20:35:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e80489b80 1 -- start start 2024-03-18T20:35:46.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e80489b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2efe0 0x5634d7e36ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e80489b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2f940 0x5634d7e40360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e80489b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e307e0 0x5634d7e25ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e80489b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634d7d1be20 con 0x5634d7e2efe0 2024-03-18T20:35:46.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e80489b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634d7df0920 con 0x5634d7e307e0 2024-03-18T20:35:46.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e80489b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634d7d75990 con 0x5634d7e2f940 2024-03-18T20:35:46.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e72090700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2efe0 0x5634d7e36ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e72090700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2efe0 0x5634d7e36ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34250/0 (socket says 172.21.15.46:34250) 2024-03-18T20:35:46.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e72090700 1 -- 172.21.15.46:0/3538550271 learned_addr learned my addr 172.21.15.46:0/3538550271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e72891700 1 --2- 172.21.15.46:0/3538550271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e307e0 0x5634d7e25ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e72090700 1 -- 172.21.15.46:0/3538550271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2f940 msgr2=0x5634d7e40360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e7188f700 1 --2- 172.21.15.46:0/3538550271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2f940 0x5634d7e40360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e72090700 1 --2- 172.21.15.46:0/3538550271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2f940 0x5634d7e40360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e72090700 1 -- 172.21.15.46:0/3538550271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e307e0 msgr2=0x5634d7e25ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e72090700 1 --2- 172.21.15.46:0/3538550271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e307e0 0x5634d7e25ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e72090700 1 -- 172.21.15.46:0/3538550271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634d7d42e50 con 0x5634d7e2efe0 2024-03-18T20:35:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.223+0000 7f7e72891700 1 --2- 172.21.15.46:0/3538550271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e307e0 0x5634d7e25ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.224+0000 7f7e72090700 1 --2- 172.21.15.46:0/3538550271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2efe0 0x5634d7e36ce0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x5634d7e39cf0 tx=0x7f7e68013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b8be2570ba340 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.224+0000 7f7e615a9700 1 -- 172.21.15.46:0/3538550271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68024070 con 0x5634d7e2efe0 2024-03-18T20:35:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.224+0000 7f7e615a9700 1 -- 172.21.15.46:0/3538550271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e68015070 con 0x5634d7e2efe0 2024-03-18T20:35:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.224+0000 7f7e615a9700 1 -- 172.21.15.46:0/3538550271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e680267a0 con 0x5634d7e2efe0 2024-03-18T20:35:46.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.224+0000 7f7e80489b80 1 -- 172.21.15.46:0/3538550271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2efe0 msgr2=0x5634d7e36ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.224+0000 7f7e80489b80 1 --2- 172.21.15.46:0/3538550271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2efe0 0x5634d7e36ce0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x5634d7e39cf0 tx=0x7f7e68013000 comp rx=0 tx=0).stop 2024-03-18T20:35:46.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.224+0000 7f7e80489b80 1 -- 172.21.15.46:0/3538550271 shutdown_connections 2024-03-18T20:35:46.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.224+0000 7f7e80489b80 1 --2- 172.21.15.46:0/3538550271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e2efe0 0x5634d7e36ce0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.224+0000 7f7e80489b80 1 --2- 172.21.15.46:0/3538550271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e307e0 0x5634d7e25ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.224+0000 7f7e80489b80 1 --2- 172.21.15.46:0/3538550271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2f940 0x5634d7e40360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.224+0000 7f7e80489b80 1 -- 172.21.15.46:0/3538550271 >> 172.21.15.46:0/3538550271 conn(0x5634d7da01d0 msgr2=0x5634d7e38330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e80489b80 1 -- 172.21.15.46:0/3538550271 shutdown_connections 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e80489b80 1 -- 172.21.15.46:0/3538550271 wait complete. 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e80489b80 1 -- start start 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e80489b80 1 Processor -- start 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e80489b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2efe0 0x5634d7e3b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e80489b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2f940 0x5634d7e3bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e80489b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 0x5634d7e988a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e80489b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634d7e3c240 con 0x5634d7e307e0 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e80489b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634d7e98e10 con 0x5634d7e2f940 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e80489b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634d7e98fd0 con 0x5634d7e2efe0 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e80489b80 10 client.0 _ll_register_callbacks cb 0x7f7e64099460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e72891700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 0x5634d7e988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e72891700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 0x5634d7e988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34254/0 (socket says 172.21.15.46:34254) 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e72891700 1 -- 172.21.15.46:0/3857387979 learned_addr learned my addr 172.21.15.46:0/3857387979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e7188f700 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2f940 0x5634d7e3bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.225+0000 7f7e72090700 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2efe0 0x5634d7e3b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e72891700 1 -- 172.21.15.46:0/3857387979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2efe0 msgr2=0x5634d7e3b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e72891700 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2efe0 0x5634d7e3b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e72891700 1 -- 172.21.15.46:0/3857387979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2f940 msgr2=0x5634d7e3bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e72891700 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2f940 0x5634d7e3bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e72891700 1 -- 172.21.15.46:0/3857387979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6800bd90 con 0x5634d7e307e0 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e7188f700 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2f940 0x5634d7e3bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e72891700 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 0x5634d7e988a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7e64099140 tx=0x7f7e64099330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476a3a8280a70113 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e55ffb700 1 -- 172.21.15.46:0/3857387979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e640a1b40 con 0x5634d7e307e0 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e55ffb700 1 -- 172.21.15.46:0/3857387979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7e6409a020 con 0x5634d7e307e0 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e80489b80 1 -- 172.21.15.46:0/3857387979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e5c01e040 con 0x5634d7e307e0 2024-03-18T20:35:46.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e55ffb700 1 -- 172.21.15.46:0/3857387979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e640a1950 con 0x5634d7e307e0 2024-03-18T20:35:46.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e80489b80 10 client.15264 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:46.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e80489b80 1 -- 172.21.15.46:0/3857387979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5634d7c02c80 con 0x5634d7e307e0 2024-03-18T20:35:46.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.226+0000 7f7e55ffb700 1 -- 172.21.15.46:0/3857387979 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e6407c080 con 0x5634d7e307e0 2024-03-18T20:35:46.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e55ffb700 1 -- 172.21.15.46:0/3857387979 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7e640651d0 con 0x5634d7e307e0 2024-03-18T20:35:46.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e80489b80 10 client.15264 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:46.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e80489b80 10 client.15264 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-18T20:35:46.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e80489b80 20 client.15264 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:46.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e80489b80 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5634d7e9db10 0x5634d7e9a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e80489b80 4 client.15264 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T20:35:46.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e80489b80 1 -- 172.21.15.46:0/3857387979 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5634d7bcbca0 con 0x5634d7e9db10 2024-03-18T20:35:46.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e72090700 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5634d7e9db10 0x5634d7e9a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e55ffb700 10 client.15264 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:35:46.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e72090700 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5634d7e9db10 0x5634d7e9a4f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2a3b2beefc96aeb4 server_cookie=9728cf8093fd233d in_seq=0 out_seq=0 2024-03-18T20:35:46.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e55ffb700 1 -- 172.21.15.46:0/3857387979 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5634d7bcbca0 con 0x5634d7e9db10 2024-03-18T20:35:46.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e55ffb700 10 client.15264 handle_command_reply: tid=0 2024-03-18T20:35:46.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.227+0000 7f7e80489b80 1 client.15264 shutdown 2024-03-18T20:35:46.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e80489b80 1 -- 172.21.15.46:0/3857387979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 msgr2=0x5634d7e988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e80489b80 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 0x5634d7e988a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7e64099140 tx=0x7f7e64099330 comp rx=0 tx=0).stop 2024-03-18T20:35:46.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e80489b80 1 -- 172.21.15.46:0/3857387979 shutdown_connections 2024-03-18T20:35:46.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e80489b80 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5634d7e9db10 0x5634d7e9a4f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e80489b80 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634d7e307e0 0x5634d7e988a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e80489b80 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634d7e2efe0 0x5634d7e3b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e55ffb700 0 client.15264 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:35:46.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e80489b80 1 --2- 172.21.15.46:0/3857387979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634d7e2f940 0x5634d7e3bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e80489b80 1 -- 172.21.15.46:0/3857387979 >> 172.21.15.46:0/3857387979 conn(0x5634d7da01d0 msgr2=0x5634d7e36c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e80489b80 1 -- 172.21.15.46:0/3857387979 shutdown_connections 2024-03-18T20:35:46.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e80489b80 1 -- 172.21.15.46:0/3857387979 wait complete. 2024-03-18T20:35:46.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e80489b80 20 client.15264 trim_cache size 0 max 16384 2024-03-18T20:35:46.242 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:35:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:35:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 3, 2024-03-18T20:35:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24506, 2024-03-18T20:35:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-18T20:35:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-18T20:35:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-18T20:35:46.242 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 62.407610396000003, 2024-03-18T20:35:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-03-18T20:35:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-03-18T20:35:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-03-18T20:35:46.243 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 80.936004777999997 2024-03-18T20:35:46.243 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:35:46.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e742f4700 1 -- 172.21.15.46:0/2497697005 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7e580a11b0 msgr2=0x7f7e580a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e742f4700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7e580a11b0 0x7f7e580a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c1981c0 tx=0x7f7e6800b410 comp rx=0 tx=0).stop 2024-03-18T20:35:46.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e742f4700 1 -- 172.21.15.46:0/2497697005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6c1017f0 msgr2=0x7f7e6c19e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e742f4700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6c1017f0 0x7f7e6c19e260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400ef30 tx=0x7f7e6400c3a0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e742f4700 1 -- 172.21.15.46:0/2497697005 shutdown_connections 2024-03-18T20:35:46.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e742f4700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7e580a11b0 0x7f7e580a3670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e742f4700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e6c1017f0 0x7f7e6c19e260 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e742f4700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e6c0ffc60 0x7f7e6c196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e742f4700 1 --2- 172.21.15.46:0/2497697005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e6c100e50 0x7f7e6c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.228+0000 7f7e742f4700 1 -- 172.21.15.46:0/2497697005 >> 172.21.15.46:0/2497697005 conn(0x7f7e6c0fb1d0 msgr2=0x7f7e6c067ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.229+0000 7f7e742f4700 1 -- 172.21.15.46:0/2497697005 shutdown_connections 2024-03-18T20:35:46.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.229+0000 7f7e742f4700 1 -- 172.21.15.46:0/2497697005 wait complete. 2024-03-18T20:35:46.246 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-18T20:35:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.380+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2243382079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40fff60 msgr2=0x7f6cd41003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.380+0000 7f6cdbb02700 1 --2- 172.21.15.46:0/2243382079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40fff60 0x7f6cd41003e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009990 tx=0x7f6cc4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.380+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2243382079 shutdown_connections 2024-03-18T20:35:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.380+0000 7f6cdbb02700 1 --2- 172.21.15.46:0/2243382079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4100920 0x7f6cd4100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.380+0000 7f6cdbb02700 1 --2- 172.21.15.46:0/2243382079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd40ff600 0x7f6cd40ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.380+0000 7f6cdbb02700 1 --2- 172.21.15.46:0/2243382079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40fff60 0x7f6cd41003e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.380+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2243382079 >> 172.21.15.46:0/2243382079 conn(0x7f6cd40fb200 msgr2=0x7f6cd40fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.380+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2243382079 shutdown_connections 2024-03-18T20:35:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.380+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2243382079 wait complete. 2024-03-18T20:35:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cdbb02700 1 Processor -- start 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cdbb02700 1 -- start start 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cdbb02700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd40ff600 0x7f6cd4198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cdbb02700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40fff60 0x7f6cd4199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cdbb02700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4100920 0x7f6cd41a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cdbb02700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd41a0a10 con 0x7f6cd40fff60 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cdbb02700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd41a0b50 con 0x7f6cd40ff600 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cdbb02700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd41a0ce0 con 0x7f6cd4100920 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cd909d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40fff60 0x7f6cd4199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cd909d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40fff60 0x7f6cd4199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34276/0 (socket says 172.21.15.46:34276) 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cd909d700 1 -- 172.21.15.46:0/2786789411 learned_addr learned my addr 172.21.15.46:0/2786789411 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cda09f700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4100920 0x7f6cd41a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.381+0000 7f6cd989e700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd40ff600 0x7f6cd4198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.382+0000 7f6cd909d700 1 -- 172.21.15.46:0/2786789411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4100920 msgr2=0x7f6cd41a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.382+0000 7f6cd909d700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4100920 0x7f6cd41a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.382+0000 7f6cd909d700 1 -- 172.21.15.46:0/2786789411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd40ff600 msgr2=0x7f6cd4198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.382+0000 7f6cd909d700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd40ff600 0x7f6cd4198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.382+0000 7f6cd909d700 1 -- 172.21.15.46:0/2786789411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc40095f0 con 0x7f6cd40fff60 2024-03-18T20:35:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.382+0000 7f6cda09f700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4100920 0x7f6cd41a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.382+0000 7f6cd909d700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40fff60 0x7f6cd4199420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4006010 tx=0x7f6cc4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.382+0000 7f6ccaffd700 1 -- 172.21.15.46:0/2786789411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc401d070 con 0x7f6cd40fff60 2024-03-18T20:35:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.382+0000 7f6ccaffd700 1 -- 172.21.15.46:0/2786789411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc4004ca0 con 0x7f6cd40fff60 2024-03-18T20:35:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.382+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2786789411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd41a0f60 con 0x7f6cd40fff60 2024-03-18T20:35:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.382+0000 7f6ccaffd700 1 -- 172.21.15.46:0/2786789411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4005500 con 0x7f6cd40fff60 2024-03-18T20:35:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.382+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2786789411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd41a1320 con 0x7f6cd40fff60 2024-03-18T20:35:46.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.384+0000 7f6ccaffd700 1 -- 172.21.15.46:0/2786789411 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f6cc400fb80 con 0x7f6cd40fff60 2024-03-18T20:35:46.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.385+0000 7f6ccaffd700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6cc00a11b0 0x7f6cc00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.385+0000 7f6ccaffd700 1 -- 172.21.15.46:0/2786789411 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6cc40cdcf0 con 0x7f6cd40fff60 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.385+0000 7f6cd989e700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6cc00a11b0 0x7f6cc00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.385+0000 7f6cd989e700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6cc00a11b0 0x7f6cc00a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009bf0 tx=0x7f6cd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.386+0000 7f6ce7c97b80 1 Processor -- start 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.386+0000 7f6ce7c97b80 1 -- start start 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.386+0000 7f6ce7c97b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2adfbd0 0x55d5c2adfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.386+0000 7f6ce7c97b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 0x55d5c2ae0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.386+0000 7f6ce7c97b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2ae13d0 0x55d5c2ae1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.386+0000 7f6ce7c97b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2a38940 con 0x55d5c2ae0530 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.386+0000 7f6ce7c97b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2a36300 con 0x55d5c2adfbd0 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.386+0000 7f6ce7c97b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2a31bf0 con 0x55d5c2ae13d0 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.386+0000 7f6cd909d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 0x55d5c2ae0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.386+0000 7f6cd909d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 0x55d5c2ae0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34286/0 (socket says 172.21.15.46:34286) 2024-03-18T20:35:46.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.386+0000 7f6cd909d700 1 -- 172.21.15.46:0/3053805584 learned_addr learned my addr 172.21.15.46:0/3053805584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.387+0000 7f6cda09f700 1 --2- 172.21.15.46:0/3053805584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2ae13d0 0x55d5c2ae1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.387+0000 7f6cd989e700 1 --2- 172.21.15.46:0/3053805584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2adfbd0 0x55d5c2adfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.387+0000 7f6cd909d700 1 -- 172.21.15.46:0/3053805584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2ae13d0 msgr2=0x55d5c2ae1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.387+0000 7f6cd909d700 1 --2- 172.21.15.46:0/3053805584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2ae13d0 0x55d5c2ae1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.387+0000 7f6cd909d700 1 -- 172.21.15.46:0/3053805584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2adfbd0 msgr2=0x55d5c2adfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.387+0000 7f6cd909d700 1 --2- 172.21.15.46:0/3053805584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2adfbd0 0x55d5c2adfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.387+0000 7f6cd909d700 1 -- 172.21.15.46:0/3053805584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5c28b3360 con 0x55d5c2ae0530 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.387+0000 7f6cda09f700 1 --2- 172.21.15.46:0/3053805584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2ae13d0 0x55d5c2ae1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.387+0000 7f6cd909d700 1 --2- 172.21.15.46:0/3053805584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 0x55d5c2ae0df0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400b2e0 tx=0x7f6cc409ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c974d418845137b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.387+0000 7f6cbffff700 1 -- 172.21.15.46:0/3053805584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc409b720 con 0x55d5c2ae0530 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.387+0000 7f6cbffff700 1 -- 172.21.15.46:0/3053805584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc40056e0 con 0x55d5c2ae0530 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.387+0000 7f6cbffff700 1 -- 172.21.15.46:0/3053805584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc409a510 con 0x55d5c2ae0530 2024-03-18T20:35:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.388+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3053805584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 msgr2=0x55d5c2ae0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.388+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3053805584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 0x55d5c2ae0df0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400b2e0 tx=0x7f6cc409ae70 comp rx=0 tx=0).stop 2024-03-18T20:35:46.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.388+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3053805584 shutdown_connections 2024-03-18T20:35:46.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.388+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3053805584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2ae13d0 0x55d5c2ae1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.388+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3053805584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2adfbd0 0x55d5c2adfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.388+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3053805584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 0x55d5c2ae0df0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.388+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3053805584 >> 172.21.15.46:0/3053805584 conn(0x55d5c2a551d0 msgr2=0x55d5c2adab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.388+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3053805584 shutdown_connections 2024-03-18T20:35:46.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.388+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3053805584 wait complete. 2024-03-18T20:35:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.388+0000 7f6ce7c97b80 1 -- start start 2024-03-18T20:35:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.388+0000 7f6ce7c97b80 1 Processor -- start 2024-03-18T20:35:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6ce7c97b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2ae96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6ce7c97b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae0530 0x55d5c2ae9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6ce7c97b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 0x55d5c2af0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6ce7c97b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c29cdd30 con 0x55d5c2ae13d0 2024-03-18T20:35:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6ce7c97b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c29cd9d0 con 0x55d5c2ae0530 2024-03-18T20:35:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6ce7c97b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c28b5650 con 0x55d5c2adfbd0 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6ce7c97b80 10 client.0 _ll_register_callbacks cb 0x55d5c2839350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6cda09f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 0x55d5c2af0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6cda09f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 0x55d5c2af0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34294/0 (socket says 172.21.15.46:34294) 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6cda09f700 1 -- 172.21.15.46:0/3523670841 learned_addr learned my addr 172.21.15.46:0/3523670841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6cd909d700 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae0530 0x55d5c2ae9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6cd989e700 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2ae96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6cda09f700 1 -- 172.21.15.46:0/3523670841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 msgr2=0x55d5c2ae96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6cda09f700 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2ae96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6cda09f700 1 -- 172.21.15.46:0/3523670841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae0530 msgr2=0x55d5c2ae9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6cda09f700 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae0530 0x55d5c2ae9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6cda09f700 1 -- 172.21.15.46:0/3523670841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd41a1320 con 0x55d5c2ae13d0 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.389+0000 7f6cd909d700 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae0530 0x55d5c2ae9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.390+0000 7f6cda09f700 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 0x55d5c2af0c40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc007b70 tx=0x7f6ccc007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd8dcb745a2743b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.390+0000 7f6cbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.390+0000 7f6cbd7fa700 1 -- 172.21.15.46:0/3523670841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0119e0 con 0x55d5c2ae13d0 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.390+0000 7f6cbd7fa700 1 -- 172.21.15.46:0/3523670841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6ccc011b40 con 0x55d5c2ae13d0 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.390+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3523670841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5c2a2a6b0 con 0x55d5c2ae13d0 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.390+0000 7f6cbd7fa700 1 -- 172.21.15.46:0/3523670841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ccc011d00 con 0x55d5c2ae13d0 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.390+0000 7f6ce7c97b80 10 client.15276 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.390+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3523670841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d5c285fb10 con 0x55d5c2ae13d0 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.390+0000 7f6cbd7fa700 1 -- 172.21.15.46:0/3523670841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5c2a2a6b0 con 0x55d5c2ae13d0 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.390+0000 7f6cbd7fa700 1 -- 172.21.15.46:0/3523670841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6ccc014070 con 0x55d5c2ae13d0 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.390+0000 7f6ce7c97b80 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.390+0000 7f6ce7c97b80 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.391+0000 7f6ce7c97b80 20 client.15276 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.391+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55d5c2af4830 0x55d5c2af9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.391+0000 7f6ce7c97b80 4 client.15276 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.391+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3523670841 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5c2880ca0 con 0x55d5c2af4830 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.391+0000 7f6cd989e700 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55d5c2af4830 0x55d5c2af9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.391+0000 7f6cd989e700 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55d5c2af4830 0x55d5c2af9b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc846328dd060720 server_cookie=e6edf16d8356912a in_seq=0 out_seq=0 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.391+0000 7f6cbd7fa700 10 client.15276 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6cbd7fa700 1 -- 172.21.15.46:0/3523670841 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d5c2880ca0 con 0x55d5c2af4830 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6cbd7fa700 10 client.15276 handle_command_reply: tid=0 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6ce7c97b80 1 client.15276 shutdown 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3523670841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 msgr2=0x55d5c2af0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 0x55d5c2af0c40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc007b70 tx=0x7f6ccc007f30 comp rx=0 tx=0).stop 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3523670841 shutdown_connections 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55d5c2af4830 0x55d5c2af9b10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 0x55d5c2af0c40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2ae96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6cbd7fa700 0 client.15276 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:35:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3523670841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae0530 0x55d5c2ae9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3523670841 >> 172.21.15.46:0/3523670841 conn(0x55d5c2a551d0 msgr2=0x55d5c2ada1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3523670841 shutdown_connections 2024-03-18T20:35:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3523670841 wait complete. 2024-03-18T20:35:46.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.392+0000 7f6ce7c97b80 20 client.15276 trim_cache size 0 max 16384 2024-03-18T20:35:46.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.411+0000 7f6ce7c97b80 1 Processor -- start 2024-03-18T20:35:46.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.411+0000 7f6ce7c97b80 1 -- start start 2024-03-18T20:35:46.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.411+0000 7f6ce7c97b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2ae17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.411+0000 7f6ce7c97b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 0x55d5c2af0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.411+0000 7f6ce7c97b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae13d0 0x55d5c2ae8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.411+0000 7f6ce7c97b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2a36300 con 0x55d5c2ae0530 2024-03-18T20:35:46.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.411+0000 7f6ce7c97b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2ae0bb0 con 0x55d5c2ae13d0 2024-03-18T20:35:46.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.411+0000 7f6ce7c97b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2880ca0 con 0x55d5c2adfbd0 2024-03-18T20:35:46.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.411+0000 7f6cd909d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 0x55d5c2af0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.411+0000 7f6cd909d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 0x55d5c2af0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34302/0 (socket says 172.21.15.46:34302) 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.411+0000 7f6cd909d700 1 -- 172.21.15.46:0/4001899752 learned_addr learned my addr 172.21.15.46:0/4001899752 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cda09f700 1 --2- 172.21.15.46:0/4001899752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae13d0 0x55d5c2ae8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cd989e700 1 --2- 172.21.15.46:0/4001899752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2ae17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cd909d700 1 -- 172.21.15.46:0/4001899752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 msgr2=0x55d5c2ae17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cd909d700 1 --2- 172.21.15.46:0/4001899752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2ae17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cd909d700 1 -- 172.21.15.46:0/4001899752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae13d0 msgr2=0x55d5c2ae8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cd909d700 1 --2- 172.21.15.46:0/4001899752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae13d0 0x55d5c2ae8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cd909d700 1 -- 172.21.15.46:0/4001899752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5c29f7e50 con 0x55d5c2ae0530 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cda09f700 1 --2- 172.21.15.46:0/4001899752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae13d0 0x55d5c2ae8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cd989e700 1 --2- 172.21.15.46:0/4001899752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2ae17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cd909d700 1 --2- 172.21.15.46:0/4001899752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 0x55d5c2af0b60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4024040 tx=0x7f6cc4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb42e3ad3c16574 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cc8da8700 1 -- 172.21.15.46:0/4001899752 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc40056c0 con 0x55d5c2ae0530 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cc8da8700 1 -- 172.21.15.46:0/4001899752 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc40a3410 con 0x55d5c2ae0530 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.412+0000 7f6cc8da8700 1 -- 172.21.15.46:0/4001899752 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc409a510 con 0x55d5c2ae0530 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.413+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/4001899752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 msgr2=0x55d5c2af0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.413+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/4001899752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 0x55d5c2af0b60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4024040 tx=0x7f6cc4009310 comp rx=0 tx=0).stop 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.413+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/4001899752 shutdown_connections 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.413+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/4001899752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae13d0 0x55d5c2ae8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.413+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/4001899752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2ae17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.413+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/4001899752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae0530 0x55d5c2af0b60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.413+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/4001899752 >> 172.21.15.46:0/4001899752 conn(0x55d5c2a551d0 msgr2=0x55d5c2ae7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.413+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/4001899752 shutdown_connections 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.413+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/4001899752 wait complete. 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.413+0000 7f6ce7c97b80 1 -- start start 2024-03-18T20:35:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.413+0000 7f6ce7c97b80 1 Processor -- start 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6ce7c97b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2af0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6ce7c97b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae0530 0x55d5c2af0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6ce7c97b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 0x55d5c2aed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6ce7c97b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2aed5d0 con 0x55d5c2ae13d0 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6ce7c97b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2aed740 con 0x55d5c2ae0530 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6ce7c97b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5c2aed900 con 0x55d5c2adfbd0 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6ce7c97b80 10 client.0 _ll_register_callbacks cb 0x7f6ccc009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cda09f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 0x55d5c2aed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cda09f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 0x55d5c2aed090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34308/0 (socket says 172.21.15.46:34308) 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cda09f700 1 -- 172.21.15.46:0/3081466595 learned_addr learned my addr 172.21.15.46:0/3081466595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cd989e700 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2af0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cd909d700 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae0530 0x55d5c2af0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cda09f700 1 -- 172.21.15.46:0/3081466595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 msgr2=0x55d5c2af0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cda09f700 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2af0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cda09f700 1 -- 172.21.15.46:0/3081466595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae0530 msgr2=0x55d5c2af0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cda09f700 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae0530 0x55d5c2af0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cda09f700 1 -- 172.21.15.46:0/3081466595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5c28b3360 con 0x55d5c2ae13d0 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cda09f700 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 0x55d5c2aed090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0215a0 tx=0x7f6ccc011560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cf31e10a6ae509 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cbd7fa700 1 -- 172.21.15.46:0/3081466595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc011df0 con 0x55d5c2ae13d0 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cbd7fa700 1 -- 172.21.15.46:0/3081466595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6ccc021740 con 0x55d5c2ae13d0 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3081466595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5c2ad2a80 con 0x55d5c2ae13d0 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.414+0000 7f6cbd7fa700 1 -- 172.21.15.46:0/3081466595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ccc021900 con 0x55d5c2ae13d0 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.415+0000 7f6ce7c97b80 10 client.15282 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.415+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3081466595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d5c28b7c80 con 0x55d5c2ae13d0 2024-03-18T20:35:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.415+0000 7f6cbd7fa700 1 -- 172.21.15.46:0/3081466595 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ccc039020 con 0x55d5c2ae13d0 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.415+0000 7f6cbd7fa700 1 -- 172.21.15.46:0/3081466595 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6ccc031060 con 0x55d5c2ae13d0 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.415+0000 7f6ce7c97b80 10 client.15282 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.415+0000 7f6ce7c97b80 10 client.15282 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.415+0000 7f6ce7c97b80 20 client.15282 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.415+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55d5c2b4d860 0x55d5c2b4a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.415+0000 7f6ce7c97b80 4 client.15282 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.415+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3081466595 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d5c28b5c90 con 0x55d5c2b4d860 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.415+0000 7f6cd989e700 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55d5c2b4d860 0x55d5c2b4a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6cbd7fa700 10 client.15282 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6cd989e700 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55d5c2b4d860 0x55d5c2b4a1e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf45bdcd9a4c6b00 server_cookie=c0157ee28627ea7e in_seq=0 out_seq=0 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6cbd7fa700 1 -- 172.21.15.46:0/3081466595 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d5c28b5c90 con 0x55d5c2b4d860 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6cbd7fa700 10 client.15282 handle_command_reply: tid=0 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6ce7c97b80 1 client.15282 shutdown 2024-03-18T20:35:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3081466595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 msgr2=0x55d5c2aed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 0x55d5c2aed090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0215a0 tx=0x7f6ccc011560 comp rx=0 tx=0).stop 2024-03-18T20:35:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3081466595 shutdown_connections 2024-03-18T20:35:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55d5c2b4d860 0x55d5c2b4a1e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5c2ae13d0 0x55d5c2aed090 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d5c2adfbd0 0x55d5c2af0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6ce7c97b80 1 --2- 172.21.15.46:0/3081466595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d5c2ae0530 0x55d5c2af0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6cbd7fa700 0 client.15282 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:35:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3081466595 >> 172.21.15.46:0/3081466595 conn(0x55d5c2a551d0 msgr2=0x55d5c2ae8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3081466595 shutdown_connections 2024-03-18T20:35:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6ce7c97b80 1 -- 172.21.15.46:0/3081466595 wait complete. 2024-03-18T20:35:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.416+0000 7f6ce7c97b80 20 client.15282 trim_cache size 0 max 16384 2024-03-18T20:35:46.427 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:35:46.427 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:35:46.427 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-03-18T20:35:46.427 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24514, 2024-03-18T20:35:46.427 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-18T20:35:46.427 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-18T20:35:46.427 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-18T20:35:46.427 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 63.606310616999998, 2024-03-18T20:35:46.427 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-03-18T20:35:46.428 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-03-18T20:35:46.428 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-03-18T20:35:46.428 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 79.402915440000001 2024-03-18T20:35:46.428 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:35:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.417+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2786789411 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6cc00a11b0 msgr2=0x7f6cc00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.417+0000 7f6cdbb02700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6cc00a11b0 0x7f6cc00a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009bf0 tx=0x7f6cd0009260 comp rx=0 tx=0).stop 2024-03-18T20:35:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.417+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2786789411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40fff60 msgr2=0x7f6cd4199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.417+0000 7f6cdbb02700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40fff60 0x7f6cd4199420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4006010 tx=0x7f6cc4005ef0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.417+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2786789411 shutdown_connections 2024-03-18T20:35:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.417+0000 7f6cdbb02700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f6cc00a11b0 0x7f6cc00a3670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.417+0000 7f6cdbb02700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4100920 0x7f6cd41a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.417+0000 7f6cdbb02700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd40ff600 0x7f6cd4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.417+0000 7f6cdbb02700 1 --2- 172.21.15.46:0/2786789411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40fff60 0x7f6cd4199420 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.417+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2786789411 >> 172.21.15.46:0/2786789411 conn(0x7f6cd40fb200 msgr2=0x7f6cd40fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.417+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2786789411 shutdown_connections 2024-03-18T20:35:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.417+0000 7f6cdbb02700 1 -- 172.21.15.46:0/2786789411 wait complete. 2024-03-18T20:35:46.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:46 smithi046 ceph-mon[43495]: pgmap v254: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 17 B/s, 0 objects/s recovering 2024-03-18T20:35:46.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:46 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1158321552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:46.433 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:46 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2429583678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:46.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-03-18T20:35:46.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:46 smithi073 ceph-mon[47384]: pgmap v254: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 17 B/s, 0 objects/s recovering 2024-03-18T20:35:46.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:46 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1158321552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:46.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:46 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2429583678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:46.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:46 smithi155 ceph-mon[48344]: pgmap v254: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 17 B/s, 0 objects/s recovering 2024-03-18T20:35:46.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:46 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1158321552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:46.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:46 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2429583678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:46.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.568+0000 7f3d3217f700 1 -- 172.21.15.46:0/3334171372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d2c102910 msgr2=0x7f3d2c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.568+0000 7f3d3217f700 1 --2- 172.21.15.46:0/3334171372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d2c102910 0x7f3d2c1091c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400b0d0 tx=0x7f3d2400b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.568+0000 7f3d3217f700 1 -- 172.21.15.46:0/3334171372 shutdown_connections 2024-03-18T20:35:46.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.568+0000 7f3d3217f700 1 --2- 172.21.15.46:0/3334171372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d2c102910 0x7f3d2c1091c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.568+0000 7f3d3217f700 1 --2- 172.21.15.46:0/3334171372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d2c100d80 0x7f3d2c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.568+0000 7f3d3217f700 1 --2- 172.21.15.46:0/3334171372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d2c101f70 0x7f3d2c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.568+0000 7f3d3217f700 1 -- 172.21.15.46:0/3334171372 >> 172.21.15.46:0/3334171372 conn(0x7f3d2c0fc2f0 msgr2=0x7f3d2c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.568+0000 7f3d3217f700 1 -- 172.21.15.46:0/3334171372 shutdown_connections 2024-03-18T20:35:46.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.568+0000 7f3d3217f700 1 -- 172.21.15.46:0/3334171372 wait complete. 2024-03-18T20:35:46.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.569+0000 7f3d3217f700 1 Processor -- start 2024-03-18T20:35:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.569+0000 7f3d3217f700 1 -- start start 2024-03-18T20:35:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d3217f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d2c100d80 0x7f3d2c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d3217f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d2c101f70 0x7f3d2c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d3217f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d2c102910 0x7f3d2c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d3217f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c073e80 con 0x7f3d2c101f70 2024-03-18T20:35:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d3217f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c073ff0 con 0x7f3d2c100d80 2024-03-18T20:35:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d3217f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c1a0e60 con 0x7f3d2c102910 2024-03-18T20:35:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d2affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d2c101f70 0x7f3d2c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d2affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d2c101f70 0x7f3d2c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34320/0 (socket says 172.21.15.46:34320) 2024-03-18T20:35:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d2b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d2c100d80 0x7f3d2c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d2affd700 1 -- 172.21.15.46:0/3182264628 learned_addr learned my addr 172.21.15.46:0/3182264628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d2bfff700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d2c102910 0x7f3d2c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d2affd700 1 -- 172.21.15.46:0/3182264628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d2c102910 msgr2=0x7f3d2c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d2affd700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d2c102910 0x7f3d2c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d2affd700 1 -- 172.21.15.46:0/3182264628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d2c100d80 msgr2=0x7f3d2c074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d2affd700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d2c100d80 0x7f3d2c074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d2affd700 1 -- 172.21.15.46:0/3182264628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d24009d00 con 0x7f3d2c101f70 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d2b7fe700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d2c100d80 0x7f3d2c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d2affd700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d2c101f70 0x7f3d2c0731a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400e9a0 tx=0x7f3d1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d28ff9700 1 -- 172.21.15.46:0/3182264628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14009790 con 0x7f3d2c101f70 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d3217f700 1 -- 172.21.15.46:0/3182264628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2c1a10f0 con 0x7f3d2c101f70 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d28ff9700 1 -- 172.21.15.46:0/3182264628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d1400ce70 con 0x7f3d2c101f70 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.570+0000 7f3d28ff9700 1 -- 172.21.15.46:0/3182264628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14018420 con 0x7f3d2c101f70 2024-03-18T20:35:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.571+0000 7f3d3217f700 1 -- 172.21.15.46:0/3182264628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c1a1600 con 0x7f3d2c101f70 2024-03-18T20:35:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.573+0000 7f3d28ff9700 1 -- 172.21.15.46:0/3182264628 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3d140106f0 con 0x7f3d2c101f70 2024-03-18T20:35:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.574+0000 7f3d28ff9700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3d180a11b0 0x7f3d180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.574+0000 7f3d2b7fe700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3d180a11b0 0x7f3d180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.574+0000 7f3d28ff9700 1 -- 172.21.15.46:0/3182264628 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3d14014070 con 0x7f3d2c101f70 2024-03-18T20:35:46.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.574+0000 7f3d2b7fe700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3d180a11b0 0x7f3d180a3670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c101dd0 tx=0x7f3d1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d3e314b80 1 Processor -- start 2024-03-18T20:35:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d3e314b80 1 -- start start 2024-03-18T20:35:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d3e314b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e7bd0 0x56384d6e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d3e314b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 0x56384d6e8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d3e314b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e93d0 0x56384d6e9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d3e314b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384d640940 con 0x56384d6e8530 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d3e314b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384d63e300 con 0x56384d6e7bd0 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d3e314b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384d639bf0 con 0x56384d6e93d0 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d2affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 0x56384d6e8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d2affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 0x56384d6e8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34326/0 (socket says 172.21.15.46:34326) 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d2affd700 1 -- 172.21.15.46:0/2673619842 learned_addr learned my addr 172.21.15.46:0/2673619842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d2b7fe700 1 --2- 172.21.15.46:0/2673619842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e7bd0 0x56384d6e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d2bfff700 1 --2- 172.21.15.46:0/2673619842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e93d0 0x56384d6e9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d2affd700 1 -- 172.21.15.46:0/2673619842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e93d0 msgr2=0x56384d6e9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d2affd700 1 --2- 172.21.15.46:0/2673619842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e93d0 0x56384d6e9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d2affd700 1 -- 172.21.15.46:0/2673619842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e7bd0 msgr2=0x56384d6e7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d2affd700 1 --2- 172.21.15.46:0/2673619842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e7bd0 0x56384d6e7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d2affd700 1 -- 172.21.15.46:0/2673619842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56384d4bb360 con 0x56384d6e8530 2024-03-18T20:35:46.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d2b7fe700 1 --2- 172.21.15.46:0/2673619842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e7bd0 0x56384d6e7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.575+0000 7f3d2bfff700 1 --2- 172.21.15.46:0/2673619842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e93d0 0x56384d6e9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.576+0000 7f3d2affd700 1 --2- 172.21.15.46:0/2673619842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 0x56384d6e8df0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3d14099c10 tx=0x7f3d1409a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbc4f7d811ce2f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.576+0000 7f3d21ffb700 1 -- 172.21.15.46:0/2673619842 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d140a2390 con 0x56384d6e8530 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.576+0000 7f3d21ffb700 1 -- 172.21.15.46:0/2673619842 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d14010430 con 0x56384d6e8530 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.576+0000 7f3d21ffb700 1 -- 172.21.15.46:0/2673619842 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14099660 con 0x56384d6e8530 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.576+0000 7f3d3e314b80 1 -- 172.21.15.46:0/2673619842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 msgr2=0x56384d6e8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.576+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/2673619842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 0x56384d6e8df0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3d14099c10 tx=0x7f3d1409a690 comp rx=0 tx=0).stop 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.576+0000 7f3d3e314b80 1 -- 172.21.15.46:0/2673619842 shutdown_connections 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.576+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/2673619842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e93d0 0x56384d6e9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.576+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/2673619842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e7bd0 0x56384d6e7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.576+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/2673619842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 0x56384d6e8df0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.576+0000 7f3d3e314b80 1 -- 172.21.15.46:0/2673619842 >> 172.21.15.46:0/2673619842 conn(0x56384d65d1d0 msgr2=0x56384d6e2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.577+0000 7f3d3e314b80 1 -- 172.21.15.46:0/2673619842 shutdown_connections 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.577+0000 7f3d3e314b80 1 -- 172.21.15.46:0/2673619842 wait complete. 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.577+0000 7f3d3e314b80 1 -- start start 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.577+0000 7f3d3e314b80 1 Processor -- start 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d3e314b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e7bd0 0x56384d6f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d3e314b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e8530 0x56384d6f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d3e314b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e93d0 0x56384d6f9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d3e314b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384d5d5d30 con 0x56384d6e7bd0 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d3e314b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384d5d59d0 con 0x56384d6e93d0 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d3e314b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384d4bd650 con 0x56384d6e8530 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d3e314b80 10 client.0 _ll_register_callbacks cb 0x56384d441350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d2b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e7bd0 0x56384d6f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d2b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e7bd0 0x56384d6f1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34334/0 (socket says 172.21.15.46:34334) 2024-03-18T20:35:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d2b7fe700 1 -- 172.21.15.46:0/1208967412 learned_addr learned my addr 172.21.15.46:0/1208967412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d2bfff700 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e93d0 0x56384d6f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d2affd700 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e8530 0x56384d6f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d2b7fe700 1 -- 172.21.15.46:0/1208967412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e8530 msgr2=0x56384d6f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d2b7fe700 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e8530 0x56384d6f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d2b7fe700 1 -- 172.21.15.46:0/1208967412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e93d0 msgr2=0x56384d6f9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d2b7fe700 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e93d0 0x56384d6f9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d2b7fe700 1 -- 172.21.15.46:0/1208967412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c1a1600 con 0x56384d6e7bd0 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d2b7fe700 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e7bd0 0x56384d6f1b60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00edd0 tx=0x7f3d1c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473481eb73aa6036 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.578+0000 7f3d0affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d0affd700 1 -- 172.21.15.46:0/1208967412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c01eda0 con 0x56384d6e7bd0 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d0affd700 1 -- 172.21.15.46:0/1208967412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3d1c0263f0 con 0x56384d6e7bd0 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1208967412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56384d6daa80 con 0x56384d6e7bd0 2024-03-18T20:35:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d0affd700 1 -- 172.21.15.46:0/1208967412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d1c0265b0 con 0x56384d6e7bd0 2024-03-18T20:35:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d3e314b80 10 client.15294 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1208967412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56384d467b10 con 0x56384d6e7bd0 2024-03-18T20:35:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d0affd700 1 -- 172.21.15.46:0/1208967412 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56384d6daa80 con 0x56384d6e7bd0 2024-03-18T20:35:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d0affd700 1 -- 172.21.15.46:0/1208967412 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3d1c01c7a0 con 0x56384d6e7bd0 2024-03-18T20:35:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d3e314b80 10 client.15294 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d3e314b80 10 client.15294 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-18T20:35:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d3e314b80 20 client.15294 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56384d6fccd0 0x56384d701fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d3e314b80 4 client.15294 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:35:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1208967412 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56384d6326b0 con 0x56384d6fccd0 2024-03-18T20:35:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.579+0000 7f3d2affd700 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56384d6fccd0 0x56384d701fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.580+0000 7f3d0affd700 10 client.15294 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.580+0000 7f3d2affd700 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56384d6fccd0 0x56384d701fb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc7b99b6bb65e26d server_cookie=9d305f47b0bfcf98 in_seq=0 out_seq=0 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.580+0000 7f3d0affd700 1 -- 172.21.15.46:0/1208967412 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56384d6326b0 con 0x56384d6fccd0 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.580+0000 7f3d0affd700 10 client.15294 handle_command_reply: tid=0 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.580+0000 7f3d3e314b80 1 client.15294 shutdown 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.580+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1208967412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e7bd0 msgr2=0x56384d6f1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.580+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e7bd0 0x56384d6f1b60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00edd0 tx=0x7f3d1c0047c0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.581+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1208967412 shutdown_connections 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.581+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56384d6fccd0 0x56384d701fb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.581+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e7bd0 0x56384d6f1b60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.581+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e93d0 0x56384d6f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.581+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/1208967412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e8530 0x56384d6f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.581+0000 7f3d0affd700 0 client.15294 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:35:46.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.581+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1208967412 >> 172.21.15.46:0/1208967412 conn(0x56384d65d1d0 msgr2=0x56384d6e21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.581+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1208967412 shutdown_connections 2024-03-18T20:35:46.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.581+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1208967412 wait complete. 2024-03-18T20:35:46.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.581+0000 7f3d3e314b80 20 client.15294 trim_cache size 0 max 16384 2024-03-18T20:35:46.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.599+0000 7f3d3e314b80 1 Processor -- start 2024-03-18T20:35:46.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.599+0000 7f3d3e314b80 1 -- start start 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d3e314b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e7bd0 0x56384d6e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d3e314b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e8530 0x56384d6f9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d3e314b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e93d0 0x56384d6e2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d3e314b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384d5d8de0 con 0x56384d6e93d0 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d3e314b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384d6ad8f0 con 0x56384d6e8530 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d3e314b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384d6fad90 con 0x56384d6e7bd0 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d2bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e93d0 0x56384d6e2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d2bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e93d0 0x56384d6e2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34350/0 (socket says 172.21.15.46:34350) 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d2bfff700 1 -- 172.21.15.46:0/2339745431 learned_addr learned my addr 172.21.15.46:0/2339745431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d2affd700 1 --2- 172.21.15.46:0/2339745431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e8530 0x56384d6f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d2b7fe700 1 --2- 172.21.15.46:0/2339745431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e7bd0 0x56384d6e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d2bfff700 1 -- 172.21.15.46:0/2339745431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e7bd0 msgr2=0x56384d6e97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d2bfff700 1 --2- 172.21.15.46:0/2339745431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e7bd0 0x56384d6e97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d2bfff700 1 -- 172.21.15.46:0/2339745431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e8530 msgr2=0x56384d6f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d2bfff700 1 --2- 172.21.15.46:0/2339745431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e8530 0x56384d6f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d2bfff700 1 -- 172.21.15.46:0/2339745431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56384d5ffe50 con 0x56384d6e93d0 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d2affd700 1 --2- 172.21.15.46:0/2339745431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e8530 0x56384d6f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d2bfff700 1 --2- 172.21.15.46:0/2339745431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e93d0 0x56384d6e2a70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400b0d0 tx=0x7f3d2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3317600ebaea290b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d227fc700 1 -- 172.21.15.46:0/2339745431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400e040 con 0x56384d6e93d0 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d227fc700 1 -- 172.21.15.46:0/2339745431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d240076e0 con 0x56384d6e93d0 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.600+0000 7f3d227fc700 1 -- 172.21.15.46:0/2339745431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24005310 con 0x56384d6e93d0 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.601+0000 7f3d3e314b80 1 -- 172.21.15.46:0/2339745431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e93d0 msgr2=0x56384d6e2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.601+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/2339745431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e93d0 0x56384d6e2a70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400b0d0 tx=0x7f3d2400b040 comp rx=0 tx=0).stop 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.601+0000 7f3d3e314b80 1 -- 172.21.15.46:0/2339745431 shutdown_connections 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.601+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/2339745431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e93d0 0x56384d6e2a70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.601+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/2339745431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e7bd0 0x56384d6e97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.601+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/2339745431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e8530 0x56384d6f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.601+0000 7f3d3e314b80 1 -- 172.21.15.46:0/2339745431 >> 172.21.15.46:0/2339745431 conn(0x56384d65d1d0 msgr2=0x56384d6de9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.601+0000 7f3d3e314b80 1 -- 172.21.15.46:0/2339745431 shutdown_connections 2024-03-18T20:35:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.601+0000 7f3d3e314b80 1 -- 172.21.15.46:0/2339745431 wait complete. 2024-03-18T20:35:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.601+0000 7f3d3e314b80 1 -- start start 2024-03-18T20:35:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d3e314b80 1 Processor -- start 2024-03-18T20:35:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d3e314b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e7bd0 0x56384d6f8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d3e314b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 0x56384d6f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d3e314b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e93d0 0x56384d6f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d3e314b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384d6f92a0 con 0x56384d6e8530 2024-03-18T20:35:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d3e314b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384d6f9410 con 0x56384d6e93d0 2024-03-18T20:35:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d3e314b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384d7515a0 con 0x56384d6e7bd0 2024-03-18T20:35:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d3e314b80 10 client.0 _ll_register_callbacks cb 0x7f3d1c00e900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d2affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 0x56384d6f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d2affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 0x56384d6f85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34358/0 (socket says 172.21.15.46:34358) 2024-03-18T20:35:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d2affd700 1 -- 172.21.15.46:0/1576040914 learned_addr learned my addr 172.21.15.46:0/1576040914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d2bfff700 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e93d0 0x56384d6f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d2b7fe700 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e7bd0 0x56384d6f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d2affd700 1 -- 172.21.15.46:0/1576040914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e7bd0 msgr2=0x56384d6f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d2affd700 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e7bd0 0x56384d6f8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d2affd700 1 -- 172.21.15.46:0/1576040914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e93d0 msgr2=0x56384d6f8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d2affd700 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e93d0 0x56384d6f8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.602+0000 7f3d2affd700 1 -- 172.21.15.46:0/1576040914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c109b10 con 0x56384d6e8530 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d2affd700 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 0x56384d6f85d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400c2d0 tx=0x7f3d14099980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314569aac323c245 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d0affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d0affd700 1 -- 172.21.15.46:0/1576040914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1409a250 con 0x56384d6e8530 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d0affd700 1 -- 172.21.15.46:0/1576040914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3d140a2390 con 0x56384d6e8530 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1576040914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d14010430 con 0x56384d6e8530 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d0affd700 1 -- 172.21.15.46:0/1576040914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d140b81e0 con 0x56384d6e8530 2024-03-18T20:35:46.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d3e314b80 10 client.15300 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1576040914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56384d4bfc80 con 0x56384d6e8530 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d0affd700 1 -- 172.21.15.46:0/1576040914 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d14063080 con 0x56384d6e8530 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d0affd700 1 -- 172.21.15.46:0/1576040914 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3d140a1f90 con 0x56384d6e8530 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d3e314b80 10 client.15300 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d3e314b80 10 client.15300 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d3e314b80 20 client.15300 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56384d756040 0x56384d752970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d3e314b80 4 client.15300 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1576040914 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56384d488ca0 con 0x56384d756040 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.603+0000 7f3d2b7fe700 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56384d756040 0x56384d752970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d0affd700 10 client.15300 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d2b7fe700 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56384d756040 0x56384d752970 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c961c2f6bd09ddcf server_cookie=a3c5d7df94aa7a49 in_seq=0 out_seq=0 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d0affd700 1 -- 172.21.15.46:0/1576040914 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56384d488ca0 con 0x56384d756040 2024-03-18T20:35:46.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d0affd700 10 client.15300 handle_command_reply: tid=0 2024-03-18T20:35:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d3e314b80 1 client.15300 shutdown 2024-03-18T20:35:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1576040914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 msgr2=0x56384d6f85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 0x56384d6f85d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400c2d0 tx=0x7f3d14099980 comp rx=0 tx=0).stop 2024-03-18T20:35:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1576040914 shutdown_connections 2024-03-18T20:35:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56384d756040 0x56384d752970 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56384d6e93d0 0x56384d6f8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56384d6e7bd0 0x56384d6f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d0affd700 0 client.15300 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:35:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d3e314b80 1 --2- 172.21.15.46:0/1576040914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56384d6e8530 0x56384d6f85d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1576040914 >> 172.21.15.46:0/1576040914 conn(0x56384d65d1d0 msgr2=0x56384d6e2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1576040914 shutdown_connections 2024-03-18T20:35:46.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d3e314b80 1 -- 172.21.15.46:0/1576040914 wait complete. 2024-03-18T20:35:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.604+0000 7f3d3e314b80 20 client.15300 trim_cache size 0 max 16384 2024-03-18T20:35:46.615 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:35:46.615 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:35:46.615 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-03-18T20:35:46.615 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24521, 2024-03-18T20:35:46.615 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-18T20:35:46.615 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-18T20:35:46.615 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-18T20:35:46.615 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 64.800145361000006, 2024-03-18T20:35:46.615 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-03-18T20:35:46.616 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-03-18T20:35:46.616 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-03-18T20:35:46.616 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 77.453480811999995 2024-03-18T20:35:46.616 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:35:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.605+0000 7f3d3217f700 1 -- 172.21.15.46:0/3182264628 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3d180a11b0 msgr2=0x7f3d180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.605+0000 7f3d3217f700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3d180a11b0 0x7f3d180a3670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c101dd0 tx=0x7f3d1c009260 comp rx=0 tx=0).stop 2024-03-18T20:35:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.605+0000 7f3d3217f700 1 -- 172.21.15.46:0/3182264628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d2c101f70 msgr2=0x7f3d2c0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.605+0000 7f3d3217f700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d2c101f70 0x7f3d2c0731a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400e9a0 tx=0x7f3d1400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.605+0000 7f3d3217f700 1 -- 172.21.15.46:0/3182264628 shutdown_connections 2024-03-18T20:35:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.605+0000 7f3d3217f700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3d180a11b0 0x7f3d180a3670 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.605+0000 7f3d3217f700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d2c102910 0x7f3d2c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.605+0000 7f3d3217f700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d2c100d80 0x7f3d2c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.605+0000 7f3d3217f700 1 --2- 172.21.15.46:0/3182264628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d2c101f70 0x7f3d2c0731a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.605+0000 7f3d3217f700 1 -- 172.21.15.46:0/3182264628 >> 172.21.15.46:0/3182264628 conn(0x7f3d2c0fc2f0 msgr2=0x7f3d2c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.605+0000 7f3d3217f700 1 -- 172.21.15.46:0/3182264628 shutdown_connections 2024-03-18T20:35:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.605+0000 7f3d3217f700 1 -- 172.21.15.46:0/3182264628 wait complete. 2024-03-18T20:35:46.622 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-18T20:35:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.756+0000 7fac3d270700 1 -- 172.21.15.46:0/3216125704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac38100d80 msgr2=0x7fac381011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.756+0000 7fac3d270700 1 --2- 172.21.15.46:0/3216125704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac38100d80 0x7fac381011a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fac280099c0 tx=0x7fac28009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.756+0000 7fac3d270700 1 -- 172.21.15.46:0/3216125704 shutdown_connections 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.757+0000 7fac3d270700 1 --2- 172.21.15.46:0/3216125704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac38100d80 0x7fac381011a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.757+0000 7fac3d270700 1 --2- 172.21.15.46:0/3216125704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac38102910 0x7fac381091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.757+0000 7fac3d270700 1 --2- 172.21.15.46:0/3216125704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac38101f70 0x7fac381023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.757+0000 7fac3d270700 1 -- 172.21.15.46:0/3216125704 >> 172.21.15.46:0/3216125704 conn(0x7fac380fc2f0 msgr2=0x7fac380fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.757+0000 7fac3d270700 1 -- 172.21.15.46:0/3216125704 shutdown_connections 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.757+0000 7fac3d270700 1 -- 172.21.15.46:0/3216125704 wait complete. 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.757+0000 7fac3d270700 1 Processor -- start 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.757+0000 7fac3d270700 1 -- start start 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac3d270700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac38100d80 0x7fac38196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac3d270700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac38101f70 0x7fac38197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac3d270700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac38102910 0x7fac3819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac3d270700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3819e800 con 0x7fac38100d80 2024-03-18T20:35:46.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac3d270700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3819e970 con 0x7fac38102910 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac3d270700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3819eb30 con 0x7fac38101f70 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac36ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac38100d80 0x7fac38196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac36ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac38100d80 0x7fac38196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34382/0 (socket says 172.21.15.46:34382) 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac36ffd700 1 -- 172.21.15.46:0/2699421594 learned_addr learned my addr 172.21.15.46:0/2699421594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac377fe700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac38102910 0x7fac3819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac367fc700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac38101f70 0x7fac38197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac36ffd700 1 -- 172.21.15.46:0/2699421594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac38101f70 msgr2=0x7fac38197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac36ffd700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac38101f70 0x7fac38197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac36ffd700 1 -- 172.21.15.46:0/2699421594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac38102910 msgr2=0x7fac3819e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac36ffd700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac38102910 0x7fac3819e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac36ffd700 1 -- 172.21.15.46:0/2699421594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac280095f0 con 0x7fac38100d80 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac367fc700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac38101f70 0x7fac38197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:46.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac377fe700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac38102910 0x7fac3819e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac36ffd700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac38100d80 0x7fac38196ce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fac28003720 tx=0x7fac28004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.758+0000 7fac1ffff700 1 -- 172.21.15.46:0/2699421594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2801d070 con 0x7fac38100d80 2024-03-18T20:35:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.759+0000 7fac1ffff700 1 -- 172.21.15.46:0/2699421594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac28004cf0 con 0x7fac38100d80 2024-03-18T20:35:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.759+0000 7fac3d270700 1 -- 172.21.15.46:0/2699421594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3819edb0 con 0x7fac38100d80 2024-03-18T20:35:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.759+0000 7fac1ffff700 1 -- 172.21.15.46:0/2699421594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac28005280 con 0x7fac38100d80 2024-03-18T20:35:46.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.759+0000 7fac3d270700 1 -- 172.21.15.46:0/2699421594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3819f170 con 0x7fac38100d80 2024-03-18T20:35:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.761+0000 7fac1ffff700 1 -- 172.21.15.46:0/2699421594 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fac28003c30 con 0x7fac38100d80 2024-03-18T20:35:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.761+0000 7fac1ffff700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fac200a11b0 0x7fac200a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.762+0000 7fac1ffff700 1 -- 172.21.15.46:0/2699421594 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fac280cda20 con 0x7fac38100d80 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.762+0000 7fac367fc700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fac200a11b0 0x7fac200a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.762+0000 7fac367fc700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fac200a11b0 0x7fac200a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac2c005950 tx=0x7fac2c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.762+0000 7fac49405b80 1 Processor -- start 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.762+0000 7fac49405b80 1 -- start start 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac49405b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb60fe0 0x56529bb61400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac49405b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb61940 0x56529bb62200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac49405b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb627e0 0x56529bb62c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac49405b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529bab5940 con 0x56529bb627e0 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac49405b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529bab3300 con 0x56529bb60fe0 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac49405b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529baaebf0 con 0x56529bb61940 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac377fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb627e0 0x56529bb62c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac377fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb627e0 0x56529bb62c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34396/0 (socket says 172.21.15.46:34396) 2024-03-18T20:35:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac377fe700 1 -- 172.21.15.46:0/2117418936 learned_addr learned my addr 172.21.15.46:0/2117418936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac36ffd700 1 --2- 172.21.15.46:0/2117418936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb60fe0 0x56529bb61400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac367fc700 1 --2- 172.21.15.46:0/2117418936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb61940 0x56529bb62200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac377fe700 1 -- 172.21.15.46:0/2117418936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb61940 msgr2=0x56529bb62200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac377fe700 1 --2- 172.21.15.46:0/2117418936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb61940 0x56529bb62200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac377fe700 1 -- 172.21.15.46:0/2117418936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb60fe0 msgr2=0x56529bb61400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac377fe700 1 --2- 172.21.15.46:0/2117418936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb60fe0 0x56529bb61400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac377fe700 1 -- 172.21.15.46:0/2117418936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56529b930360 con 0x56529bb627e0 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac377fe700 1 --2- 172.21.15.46:0/2117418936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb627e0 0x56529bb62c60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fac300078a0 tx=0x7fac30007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10d0e727c2e1b1c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.763+0000 7fac1d7fa700 1 -- 172.21.15.46:0/2117418936 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30004bb0 con 0x56529bb627e0 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.764+0000 7fac1d7fa700 1 -- 172.21.15.46:0/2117418936 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac3000f040 con 0x56529bb627e0 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.764+0000 7fac1d7fa700 1 -- 172.21.15.46:0/2117418936 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac300053c0 con 0x56529bb627e0 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.764+0000 7fac49405b80 1 -- 172.21.15.46:0/2117418936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb627e0 msgr2=0x56529bb62c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.764+0000 7fac49405b80 1 --2- 172.21.15.46:0/2117418936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb627e0 0x56529bb62c60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fac300078a0 tx=0x7fac30007bb0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.764+0000 7fac49405b80 1 -- 172.21.15.46:0/2117418936 shutdown_connections 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.764+0000 7fac49405b80 1 --2- 172.21.15.46:0/2117418936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb627e0 0x56529bb62c60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.764+0000 7fac49405b80 1 --2- 172.21.15.46:0/2117418936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb60fe0 0x56529bb61400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.764+0000 7fac49405b80 1 --2- 172.21.15.46:0/2117418936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb61940 0x56529bb62200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.764+0000 7fac49405b80 1 -- 172.21.15.46:0/2117418936 >> 172.21.15.46:0/2117418936 conn(0x56529bad21d0 msgr2=0x56529bb57b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.764+0000 7fac49405b80 1 -- 172.21.15.46:0/2117418936 shutdown_connections 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.764+0000 7fac49405b80 1 -- 172.21.15.46:0/2117418936 wait complete. 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.765+0000 7fac49405b80 1 -- start start 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.765+0000 7fac49405b80 1 Processor -- start 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.765+0000 7fac49405b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb60fe0 0x56529bb6ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.765+0000 7fac49405b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 0x56529bb6b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.765+0000 7fac49405b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bb723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.765+0000 7fac49405b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529ba4ad30 con 0x56529bb61940 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.765+0000 7fac49405b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529ba4a9d0 con 0x56529bb627e0 2024-03-18T20:35:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.765+0000 7fac49405b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529b932650 con 0x56529bb60fe0 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.765+0000 7fac49405b80 10 client.0 _ll_register_callbacks cb 0x56529b8b6350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.765+0000 7fac367fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 0x56529bb6b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.765+0000 7fac367fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 0x56529bb6b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34408/0 (socket says 172.21.15.46:34408) 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.765+0000 7fac367fc700 1 -- 172.21.15.46:0/2579899527 learned_addr learned my addr 172.21.15.46:0/2579899527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac36ffd700 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb60fe0 0x56529bb6ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac377fe700 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bb723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac367fc700 1 -- 172.21.15.46:0/2579899527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb60fe0 msgr2=0x56529bb6ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac367fc700 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb60fe0 0x56529bb6ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac367fc700 1 -- 172.21.15.46:0/2579899527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 msgr2=0x56529bb723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac367fc700 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bb723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac367fc700 1 -- 172.21.15.46:0/2579899527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac30007500 con 0x56529bb61940 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac367fc700 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 0x56529bb6b3b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00ffd0 tx=0x7fac2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e448c21d3864887 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac1aff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac1aff5700 1 -- 172.21.15.46:0/2579899527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c017040 con 0x56529bb61940 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac1aff5700 1 -- 172.21.15.46:0/2579899527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fac2c003d10 con 0x56529bb61940 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac49405b80 1 -- 172.21.15.46:0/2579899527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56529bb4fa80 con 0x56529bb61940 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac1aff5700 1 -- 172.21.15.46:0/2579899527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac2c020440 con 0x56529bb61940 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac49405b80 10 client.15312 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac49405b80 1 -- 172.21.15.46:0/2579899527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56529b8dcb10 con 0x56529bb61940 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.766+0000 7fac1aff5700 1 -- 172.21.15.46:0/2579899527 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56529bb4fa80 con 0x56529bb61940 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.767+0000 7fac1aff5700 1 -- 172.21.15.46:0/2579899527 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fac2c01d050 con 0x56529bb61940 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.767+0000 7fac49405b80 10 client.15312 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.767+0000 7fac49405b80 10 client.15312 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.767+0000 7fac49405b80 20 client.15312 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.767+0000 7fac49405b80 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56529bb75590 0x56529bb7b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.767+0000 7fac49405b80 4 client.15312 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:35:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.767+0000 7fac49405b80 1 -- 172.21.15.46:0/2579899527 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56529baa76b0 con 0x56529bb75590 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.767+0000 7fac36ffd700 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56529bb75590 0x56529bb7b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.767+0000 7fac36ffd700 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56529bb75590 0x56529bb7b3d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6abce821858efbb server_cookie=c5a4db6a7df24782 in_seq=0 out_seq=0 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.767+0000 7fac1aff5700 10 client.15312 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.768+0000 7fac1aff5700 1 -- 172.21.15.46:0/2579899527 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56529baa76b0 con 0x56529bb75590 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.768+0000 7fac1aff5700 10 client.15312 handle_command_reply: tid=0 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.768+0000 7fac49405b80 1 client.15312 shutdown 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.768+0000 7fac49405b80 1 -- 172.21.15.46:0/2579899527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 msgr2=0x56529bb6b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.768+0000 7fac49405b80 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 0x56529bb6b3b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00ffd0 tx=0x7fac2c004750 comp rx=0 tx=0).stop 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.768+0000 7fac49405b80 1 -- 172.21.15.46:0/2579899527 shutdown_connections 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.768+0000 7fac49405b80 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56529bb75590 0x56529bb7b3d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.768+0000 7fac49405b80 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bb723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.768+0000 7fac49405b80 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb60fe0 0x56529bb6ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.768+0000 7fac1aff5700 0 client.15312 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:46.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.768+0000 7fac49405b80 1 --2- 172.21.15.46:0/2579899527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 0x56529bb6b3b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.768+0000 7fac49405b80 1 -- 172.21.15.46:0/2579899527 >> 172.21.15.46:0/2579899527 conn(0x56529bad21d0 msgr2=0x56529bb571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.769+0000 7fac49405b80 1 -- 172.21.15.46:0/2579899527 shutdown_connections 2024-03-18T20:35:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.769+0000 7fac49405b80 1 -- 172.21.15.46:0/2579899527 wait complete. 2024-03-18T20:35:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.769+0000 7fac49405b80 20 client.15312 trim_cache size 0 max 16384 2024-03-18T20:35:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.787+0000 7fac49405b80 1 Processor -- start 2024-03-18T20:35:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.787+0000 7fac49405b80 1 -- start start 2024-03-18T20:35:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.787+0000 7fac49405b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb60fe0 0x56529bb62bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.787+0000 7fac49405b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 0x56529bb723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.787+0000 7fac49405b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bb6a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.787+0000 7fac49405b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529ba4dde0 con 0x56529bb61940 2024-03-18T20:35:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.787+0000 7fac49405b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529bb51f30 con 0x56529bb627e0 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.787+0000 7fac49405b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529bb521c0 con 0x56529bb60fe0 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac367fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 0x56529bb723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac367fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 0x56529bb723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34412/0 (socket says 172.21.15.46:34412) 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac367fc700 1 -- 172.21.15.46:0/2282416196 learned_addr learned my addr 172.21.15.46:0/2282416196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac36ffd700 1 --2- 172.21.15.46:0/2282416196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb60fe0 0x56529bb62bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac377fe700 1 --2- 172.21.15.46:0/2282416196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bb6a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac367fc700 1 -- 172.21.15.46:0/2282416196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb60fe0 msgr2=0x56529bb62bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac367fc700 1 --2- 172.21.15.46:0/2282416196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb60fe0 0x56529bb62bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac367fc700 1 -- 172.21.15.46:0/2282416196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 msgr2=0x56529bb6a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac367fc700 1 --2- 172.21.15.46:0/2282416196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bb6a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac367fc700 1 -- 172.21.15.46:0/2282416196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56529ba74e50 con 0x56529bb61940 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac377fe700 1 --2- 172.21.15.46:0/2282416196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bb6a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac367fc700 1 --2- 172.21.15.46:0/2282416196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 0x56529bb723b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00b950 tx=0x7fac2c00b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da527987500ae81a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac1dffb700 1 -- 172.21.15.46:0/2282416196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c003b60 con 0x56529bb61940 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac1dffb700 1 -- 172.21.15.46:0/2282416196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac2c011d80 con 0x56529bb61940 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.788+0000 7fac1dffb700 1 -- 172.21.15.46:0/2282416196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c020bf0 con 0x56529bb61940 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.789+0000 7fac49405b80 1 -- 172.21.15.46:0/2282416196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 msgr2=0x56529bb723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.789+0000 7fac49405b80 1 --2- 172.21.15.46:0/2282416196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 0x56529bb723b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00b950 tx=0x7fac2c00b980 comp rx=0 tx=0).stop 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.789+0000 7fac49405b80 1 -- 172.21.15.46:0/2282416196 shutdown_connections 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.789+0000 7fac49405b80 1 --2- 172.21.15.46:0/2282416196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bb6a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.789+0000 7fac49405b80 1 --2- 172.21.15.46:0/2282416196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb60fe0 0x56529bb62bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.789+0000 7fac49405b80 1 --2- 172.21.15.46:0/2282416196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb61940 0x56529bb723b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.789+0000 7fac49405b80 1 -- 172.21.15.46:0/2282416196 >> 172.21.15.46:0/2282416196 conn(0x56529bad21d0 msgr2=0x56529bb57ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.789+0000 7fac49405b80 1 -- 172.21.15.46:0/2282416196 shutdown_connections 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.789+0000 7fac49405b80 1 -- 172.21.15.46:0/2282416196 wait complete. 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.789+0000 7fac49405b80 1 -- start start 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.789+0000 7fac49405b80 1 Processor -- start 2024-03-18T20:35:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac49405b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb60fe0 0x56529bb6d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac49405b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb61940 0x56529bb6db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac49405b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bbca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac49405b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529bbcaf00 con 0x56529bb60fe0 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac49405b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529bbcb040 con 0x56529bb627e0 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac49405b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529bbcb180 con 0x56529bb61940 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac49405b80 10 client.0 _ll_register_callbacks cb 0x7fac2c00fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac36ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb60fe0 0x56529bb6d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac36ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb60fe0 0x56529bb6d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34428/0 (socket says 172.21.15.46:34428) 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac36ffd700 1 -- 172.21.15.46:0/1252557643 learned_addr learned my addr 172.21.15.46:0/1252557643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac377fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bbca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac367fc700 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb61940 0x56529bb6db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac36ffd700 1 -- 172.21.15.46:0/1252557643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb61940 msgr2=0x56529bb6db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac36ffd700 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb61940 0x56529bb6db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac36ffd700 1 -- 172.21.15.46:0/1252557643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 msgr2=0x56529bbca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac36ffd700 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bbca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac36ffd700 1 -- 172.21.15.46:0/1252557643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56529b8dcb10 con 0x56529bb60fe0 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac377fe700 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bbca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac36ffd700 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb60fe0 0x56529bb6d620 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fac28003680 tx=0x7fac28003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84bbd83e7e20dfa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.790+0000 7fac1aff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac1aff5700 1 -- 172.21.15.46:0/1252557643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2809adb0 con 0x56529bb60fe0 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac1aff5700 1 -- 172.21.15.46:0/1252557643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fac2809ebe0 con 0x56529bb60fe0 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac49405b80 1 -- 172.21.15.46:0/1252557643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56529bb593e0 con 0x56529bb60fe0 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac1aff5700 1 -- 172.21.15.46:0/1252557643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac280a31a0 con 0x56529bb60fe0 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac49405b80 10 client.15318 fetch_fsmap learned FSMap version 26 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac49405b80 1 -- 172.21.15.46:0/1252557643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56529b934c80 con 0x56529bb60fe0 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac1aff5700 1 -- 172.21.15.46:0/1252557643 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56529bb593e0 con 0x56529bb60fe0 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac1aff5700 1 -- 172.21.15.46:0/1252557643 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 26) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fac280a2d80 con 0x56529bb60fe0 2024-03-18T20:35:46.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac49405b80 10 client.15318 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac49405b80 10 client.15318 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac49405b80 20 client.15318 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac49405b80 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56529bbcfab0 0x56529bbcc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac49405b80 4 client.15318 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac49405b80 1 -- 172.21.15.46:0/1252557643 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fac3000f040 con 0x56529bbcfab0 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.791+0000 7fac367fc700 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56529bbcfab0 0x56529bbcc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac1aff5700 10 client.15318 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac367fc700 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56529bbcfab0 0x56529bbcc570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ff1c24c7282ba2 server_cookie=ce38aca81cbf4495 in_seq=0 out_seq=0 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac1aff5700 1 -- 172.21.15.46:0/1252557643 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x7fac3000f040 con 0x56529bbcfab0 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac1aff5700 10 client.15318 handle_command_reply: tid=0 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac49405b80 1 client.15318 shutdown 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac49405b80 1 -- 172.21.15.46:0/1252557643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb60fe0 msgr2=0x56529bb6d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac49405b80 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb60fe0 0x56529bb6d620 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fac28003680 tx=0x7fac28003fd0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac49405b80 1 -- 172.21.15.46:0/1252557643 shutdown_connections 2024-03-18T20:35:46.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac49405b80 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56529bbcfab0 0x56529bbcc570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac49405b80 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56529bb60fe0 0x56529bb6d620 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac49405b80 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56529bb627e0 0x56529bbca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac1aff5700 0 client.15318 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac49405b80 1 --2- 172.21.15.46:0/1252557643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56529bb61940 0x56529bb6db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac49405b80 1 -- 172.21.15.46:0/1252557643 >> 172.21.15.46:0/1252557643 conn(0x56529bad21d0 msgr2=0x56529bb56700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac49405b80 1 -- 172.21.15.46:0/1252557643 shutdown_connections 2024-03-18T20:35:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.792+0000 7fac49405b80 1 -- 172.21.15.46:0/1252557643 wait complete. 2024-03-18T20:35:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac49405b80 20 client.15318 trim_cache size 0 max 16384 2024-03-18T20:35:46.802 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:35:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "04a8ae18-e566-11ee-95c9-87774f69a715", 2024-03-18T20:35:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-18T20:35:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24536, 2024-03-18T20:35:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-18T20:35:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-18T20:35:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-18T20:35:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 68.004558990999996, 2024-03-18T20:35:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 26, 2024-03-18T20:35:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-03-18T20:35:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-03-18T20:35:46.803 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 73.860182859000005 2024-03-18T20:35:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:35:46.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac3d270700 1 -- 172.21.15.46:0/2699421594 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fac200a11b0 msgr2=0x7fac200a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac3d270700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fac200a11b0 0x7fac200a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac2c005950 tx=0x7fac2c0058e0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac3d270700 1 -- 172.21.15.46:0/2699421594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac38100d80 msgr2=0x7fac38196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:46.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac3d270700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac38100d80 0x7fac38196ce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fac28003720 tx=0x7fac28004710 comp rx=0 tx=0).stop 2024-03-18T20:35:46.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac3d270700 1 -- 172.21.15.46:0/2699421594 shutdown_connections 2024-03-18T20:35:46.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac3d270700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fac200a11b0 0x7fac200a3670 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac3d270700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac38100d80 0x7fac38196ce0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac3d270700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac38102910 0x7fac3819e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac3d270700 1 --2- 172.21.15.46:0/2699421594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac38101f70 0x7fac38197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:46.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac3d270700 1 -- 172.21.15.46:0/2699421594 >> 172.21.15.46:0/2699421594 conn(0x7fac380fc2f0 msgr2=0x7fac381053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:46.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac3d270700 1 -- 172.21.15.46:0/2699421594 shutdown_connections 2024-03-18T20:35:46.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:46.793+0000 7fac3d270700 1 -- 172.21.15.46:0/2699421594 wait complete. 2024-03-18T20:35:46.810 INFO:teuthology.run_tasks:Running task kclient... 2024-03-18T20:35:46.838 INFO:tasks.kclient:Mounting kernel clients... 2024-03-18T20:35:46.839 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-18T20:35:46.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:46.839 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-18T20:35:46.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:46.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-03-18T20:35:46.936 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-03-18T20:35:46.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T20:35:46.938 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-03-18T20:35:46.939 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-18T20:35:46.939 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T20:35:46.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-18T20:35:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T20:35:48.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.187+0000 7f430cd52700 1 -- 172.21.15.46:0/787932280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43080ff750 msgr2=0x7f43080ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:48.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.187+0000 7f430cd52700 1 --2- 172.21.15.46:0/787932280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43080ff750 0x7f43080ffbb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009990 tx=0x7f42fc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:48.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.187+0000 7f430cd52700 1 -- 172.21.15.46:0/787932280 shutdown_connections 2024-03-18T20:35:48.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.187+0000 7f430cd52700 1 --2- 172.21.15.46:0/787932280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43081000f0 0x7f430810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:48.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.187+0000 7f430cd52700 1 --2- 172.21.15.46:0/787932280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4308105e90 0x7f4308106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:48.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.187+0000 7f430cd52700 1 --2- 172.21.15.46:0/787932280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43080ff750 0x7f43080ffbb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:48.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.187+0000 7f430cd52700 1 -- 172.21.15.46:0/787932280 >> 172.21.15.46:0/787932280 conn(0x7f4308074fd0 msgr2=0x7f43080753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:48.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.187+0000 7f430cd52700 1 -- 172.21.15.46:0/787932280 shutdown_connections 2024-03-18T20:35:48.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.187+0000 7f430cd52700 1 -- 172.21.15.46:0/787932280 wait complete. 2024-03-18T20:35:48.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.188+0000 7f430cd52700 1 Processor -- start 2024-03-18T20:35:48.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.188+0000 7f430cd52700 1 -- start start 2024-03-18T20:35:48.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.188+0000 7f430cd52700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43080ff750 0x7f4308197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:48.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.188+0000 7f430cd52700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43081000f0 0x7f43081976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:48.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.188+0000 7f430cd52700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4308105e90 0x7f430819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:48.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.188+0000 7f430cd52700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430819c090 con 0x7f43081000f0 2024-03-18T20:35:48.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.188+0000 7f430cd52700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430819c200 con 0x7f4308105e90 2024-03-18T20:35:48.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.188+0000 7f430cd52700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430819c3c0 con 0x7f43080ff750 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.188+0000 7f4305d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43081000f0 0x7f43081976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.188+0000 7f4305d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43081000f0 0x7f43081976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34438/0 (socket says 172.21.15.46:34438) 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.188+0000 7f4305d9b700 1 -- 172.21.15.46:0/3691079060 learned_addr learned my addr 172.21.15.46:0/3691079060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f430659c700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43080ff750 0x7f4308197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f4306d9d700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4308105e90 0x7f430819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f4305d9b700 1 -- 172.21.15.46:0/3691079060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43080ff750 msgr2=0x7f4308197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f4305d9b700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43080ff750 0x7f4308197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f4305d9b700 1 -- 172.21.15.46:0/3691079060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4308105e90 msgr2=0x7f430819ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f4305d9b700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4308105e90 0x7f430819ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f4305d9b700 1 -- 172.21.15.46:0/3691079060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc0095f0 con 0x7f43081000f0 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f4306d9d700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4308105e90 0x7f430819ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f4305d9b700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43081000f0 0x7f43081976d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0098e0 tx=0x7f42fc00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f42f77fe700 1 -- 172.21.15.46:0/3691079060 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc01d070 con 0x7f43081000f0 2024-03-18T20:35:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f430cd52700 1 -- 172.21.15.46:0/3691079060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430819c640 con 0x7f43081000f0 2024-03-18T20:35:48.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f42f77fe700 1 -- 172.21.15.46:0/3691079060 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42fc022470 con 0x7f43081000f0 2024-03-18T20:35:48.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f42f77fe700 1 -- 172.21.15.46:0/3691079060 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc017650 con 0x7f43081000f0 2024-03-18T20:35:48.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.189+0000 7f430cd52700 1 -- 172.21.15.46:0/3691079060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43081a3f00 con 0x7f43081000f0 2024-03-18T20:35:48.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.190+0000 7f430cd52700 1 -- 172.21.15.46:0/3691079060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43081092f0 con 0x7f43081000f0 2024-03-18T20:35:48.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.192+0000 7f42f77fe700 1 -- 172.21.15.46:0/3691079060 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f42fc0225e0 con 0x7f43081000f0 2024-03-18T20:35:48.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.192+0000 7f42f77fe700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f42f00a11b0 0x7f42f00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:48.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.192+0000 7f430659c700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f42f00a11b0 0x7f42f00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:48.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.192+0000 7f42f77fe700 1 -- 172.21.15.46:0/3691079060 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f42fc0cdac0 con 0x7f43081000f0 2024-03-18T20:35:48.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.193+0000 7f430659c700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f42f00a11b0 0x7f42f00a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4308068720 tx=0x7f42f800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:48.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.197+0000 7f42f77fe700 1 -- 172.21.15.46:0/3691079060 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f42fc097190 con 0x7f43081000f0 2024-03-18T20:35:48.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:48 smithi046 ceph-mon[43495]: pgmap v255: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 14 B/s, 0 objects/s recovering 2024-03-18T20:35:48.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.456+0000 7f430cd52700 1 -- 172.21.15.46:0/3691079060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f43081984a0 con 0x7f43081000f0 2024-03-18T20:35:48.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.457+0000 7f42f77fe700 1 -- 172.21.15.46:0/3691079060 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f42fc09a7b0 con 0x7f43081000f0 2024-03-18T20:35:48.460 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-18T20:35:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.459+0000 7f430cd52700 1 -- 172.21.15.46:0/3691079060 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f42f00a11b0 msgr2=0x7f42f00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.459+0000 7f430cd52700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f42f00a11b0 0x7f42f00a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4308068720 tx=0x7f42f800a3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.459+0000 7f430cd52700 1 -- 172.21.15.46:0/3691079060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43081000f0 msgr2=0x7f43081976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.459+0000 7f430cd52700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43081000f0 0x7f43081976d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0098e0 tx=0x7f42fc00f670 comp rx=0 tx=0).stop 2024-03-18T20:35:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.459+0000 7f430cd52700 1 -- 172.21.15.46:0/3691079060 shutdown_connections 2024-03-18T20:35:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.459+0000 7f430cd52700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f42f00a11b0 0x7f42f00a3670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.459+0000 7f430cd52700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4308105e90 0x7f430819ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.459+0000 7f430cd52700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43080ff750 0x7f4308197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.459+0000 7f430cd52700 1 --2- 172.21.15.46:0/3691079060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43081000f0 0x7f43081976d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.459+0000 7f430cd52700 1 -- 172.21.15.46:0/3691079060 >> 172.21.15.46:0/3691079060 conn(0x7f4308074fd0 msgr2=0x7f43080fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:48.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.460+0000 7f430cd52700 1 -- 172.21.15.46:0/3691079060 shutdown_connections 2024-03-18T20:35:48.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:48.460+0000 7f430cd52700 1 -- 172.21.15.46:0/3691079060 wait complete. 2024-03-18T20:35:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:48 smithi073 ceph-mon[47384]: pgmap v255: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 14 B/s, 0 objects/s recovering 2024-03-18T20:35:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:48 smithi155 ceph-mon[48344]: pgmap v255: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 14 B/s, 0 objects/s recovering 2024-03-18T20:35:48.861 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-18T20:35:48.861 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-18T20:35:48.861 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-18T20:35:48.861 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-18T20:35:48.861 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T20:35:48.862 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-18T20:35:48.862 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-18T20:35:48.862 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-18T20:35:48.862 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-18T20:35:48.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:48.862 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-18T20:35:48.885 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-18T20:35:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-18T20:35:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-18T20:35:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-18T20:35:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-18T20:35:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-18T20:35:48.885 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T20:35:48.885 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41846sec preferred_lft 41846sec 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T20:35:48.886 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-18T20:35:48.887 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-18T20:35:48.887 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-18T20:35:48.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:48.887 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-18T20:35:48.887 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-18T20:35:48.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-03-18T20:35:48.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-03-18T20:35:48.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-03-18T20:35:48.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-18T20:35:48.888 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-18T20:35:49.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:49.083 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-18T20:35:49.119 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-18T20:35:49.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:49.120 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-18T20:35:49.191 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-18T20:35:49.192 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-18T20:35:49.192 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-18T20:35:49.192 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-18T20:35:49.192 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-18T20:35:49.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:49.193 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-18T20:35:49.193 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-18T20:35:49.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-18T20:35:49.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-18T20:35:49.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-18T20:35:49.193 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-18T20:35:49.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:49 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3691079060' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T20:35:49.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:49.429 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-18T20:35:49.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:49.450 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-03-18T20:35:49.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:49 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3691079060' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T20:35:49.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:49.511 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-18T20:35:49.511 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-18T20:35:49.511 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T20:35:49.511 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-18T20:35:49.512 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-18T20:35:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:49 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3691079060' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T20:35:49.623 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-18T20:35:49.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:49.623 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-18T20:35:49.623 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-18T20:35:49.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-18T20:35:49.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-18T20:35:49.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-18T20:35:49.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-18T20:35:49.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-18T20:35:49.624 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-18T20:35:49.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:49.844 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-18T20:35:49.844 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-18T20:35:49.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-18T20:35:49.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-03-18T20:35:49.844 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-18T20:35:49.956 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T20:35:49.956 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-18T20:35:49.975 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-18T20:35:49.976 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T20:35:49.976 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-18T20:35:50.037 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-18T20:35:50.037 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-18T20:35:50.038 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410 2024-03-18T20:35:50.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:50.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-03-18T20:35:50.136 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-03-18T20:35:50.137 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-03-18T20:35:50.241 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-03-18T20:35:50.241 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-03-18T20:35:50.241 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.73:3300,172.21.15.155:3300" 2024-03-18T20:35:50.241 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with new device syntax: 0@04a8ae18-e566-11ee-95c9-87774f69a715.cephfs=/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410 2024-03-18T20:35:50.242 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.46:3300/172.21.15.73:3300/172.21.15.155:3300" will pass to kernel 2024-03-18T20:35:50.242 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: v2 mount support check returned 2 2024-03-18T20:35:50.242 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: kernel does not support v2 syntax 2024-03-18T20:35:50.242 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:3300,172.21.15.73:3300,172.21.15.155:3300:/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410 2024-03-18T20:35:50.242 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mds_namespace=cephfs,fsid=04a8ae18-e566-11ee-95c9-87774f69a715" will pass to kernel 2024-03-18T20:35:50.244 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-18T20:35:50.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:50.244 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-18T20:35:50.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:50.283 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T20:35:50.384 INFO:teuthology.orchestra.run.smithi046.stdout:client25108 2024-03-18T20:35:50.384 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:35:50.384 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T20:35:50.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:50.407 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T20:35:50.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:50 smithi073 ceph-mon[47384]: pgmap v256: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 12 B/s, 0 objects/s recovering 2024-03-18T20:35:50.510 INFO:teuthology.orchestra.run.smithi046.stdout:client25108 2024-03-18T20:35:50.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:50.510 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T20:35:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:50 smithi155 ceph-mon[48344]: pgmap v256: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 12 B/s, 0 objects/s recovering 2024-03-18T20:35:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:50 smithi046 ceph-mon[43495]: pgmap v256: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 12 B/s, 0 objects/s recovering 2024-03-18T20:35:50.581 INFO:teuthology.orchestra.run.smithi046.stdout:client25108 2024-03-18T20:35:50.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:50.581 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/04a8ae18-e566-11ee-95c9-87774f69a715.client25108/status') 2024-03-18T20:35:50.697 INFO:teuthology.orchestra.run.smithi046.stdout:instance: client.25108 (3)172.21.15.46:0/1584101617 2024-03-18T20:35:50.697 INFO:teuthology.orchestra.run.smithi046.stdout:blocklisted: false 2024-03-18T20:35:50.697 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records in 2024-03-18T20:35:50.698 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records out 2024-03-18T20:35:50.698 INFO:teuthology.orchestra.run.smithi046.stderr:71 bytes copied, 3.0531e-05 s, 2.3 MB/s 2024-03-18T20:35:50.699 INFO:teuthology.run_tasks:Running task exec... 2024-03-18T20:35:50.709 INFO:teuthology.task.exec:Executing custom commands... 2024-03-18T20:35:50.710 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-03-18T20:35:50.710 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-18T20:35:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.857+0000 7f6086aea700 1 -- 172.21.15.46:0/4176406653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60801069e0 msgr2=0x7f6080108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.857+0000 7f6086aea700 1 --2- 172.21.15.46:0/4176406653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60801069e0 0x7f6080108e50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f607800b0a0 tx=0x7f607800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:50.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.857+0000 7f6086aea700 1 -- 172.21.15.46:0/4176406653 shutdown_connections 2024-03-18T20:35:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.857+0000 7f6086aea700 1 --2- 172.21.15.46:0/4176406653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60801069e0 0x7f6080108e50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.857+0000 7f6086aea700 1 --2- 172.21.15.46:0/4176406653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6080101780 0x7f6080103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.857+0000 7f6086aea700 1 --2- 172.21.15.46:0/4176406653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60801040b0 0x7f60801064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.857+0000 7f6086aea700 1 -- 172.21.15.46:0/4176406653 >> 172.21.15.46:0/4176406653 conn(0x7f60800fb170 msgr2=0x7f60800fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.857+0000 7f6086aea700 1 -- 172.21.15.46:0/4176406653 shutdown_connections 2024-03-18T20:35:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.857+0000 7f6086aea700 1 -- 172.21.15.46:0/4176406653 wait complete. 2024-03-18T20:35:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f6086aea700 1 Processor -- start 2024-03-18T20:35:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f6086aea700 1 -- start start 2024-03-18T20:35:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f6086aea700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6080101780 0x7f6080196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f6086aea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60801040b0 0x7f6080197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f6086aea700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60801069e0 0x7f608019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:50.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f6086aea700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608019e830 con 0x7f60801040b0 2024-03-18T20:35:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f6086aea700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608019e9a0 con 0x7f60801069e0 2024-03-18T20:35:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f6086aea700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608019eb60 con 0x7f6080101780 2024-03-18T20:35:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f607ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60801040b0 0x7f6080197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f607ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60801040b0 0x7f6080197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34446/0 (socket says 172.21.15.46:34446) 2024-03-18T20:35:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f607ffff700 1 -- 172.21.15.46:0/2986568576 learned_addr learned my addr 172.21.15.46:0/2986568576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f6085087700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60801069e0 0x7f608019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f6084886700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6080101780 0x7f6080196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f607ffff700 1 -- 172.21.15.46:0/2986568576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6080101780 msgr2=0x7f6080196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f607ffff700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6080101780 0x7f6080196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f607ffff700 1 -- 172.21.15.46:0/2986568576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60801069e0 msgr2=0x7f608019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f607ffff700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60801069e0 0x7f608019e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f607ffff700 1 -- 172.21.15.46:0/2986568576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6078009d00 con 0x7f60801040b0 2024-03-18T20:35:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f6085087700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60801069e0 0x7f608019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.858+0000 7f6084886700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6080101780 0x7f6080196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.859+0000 7f607ffff700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60801040b0 0x7f6080197280 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f606800d950 tx=0x7f606800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.859+0000 7f607dffb700 1 -- 172.21.15.46:0/2986568576 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068009b40 con 0x7f60801040b0 2024-03-18T20:35:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.859+0000 7f607dffb700 1 -- 172.21.15.46:0/2986568576 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6068009ca0 con 0x7f60801040b0 2024-03-18T20:35:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.859+0000 7f607dffb700 1 -- 172.21.15.46:0/2986568576 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606800f620 con 0x7f60801040b0 2024-03-18T20:35:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.859+0000 7f6086aea700 1 -- 172.21.15.46:0/2986568576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608019ee40 con 0x7f60801040b0 2024-03-18T20:35:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.859+0000 7f6086aea700 1 -- 172.21.15.46:0/2986568576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608019f210 con 0x7f60801040b0 2024-03-18T20:35:50.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.860+0000 7f6086aea700 1 -- 172.21.15.46:0/2986568576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60800fccf0 con 0x7f60801040b0 2024-03-18T20:35:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.861+0000 7f607dffb700 1 -- 172.21.15.46:0/2986568576 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f6068010460 con 0x7f60801040b0 2024-03-18T20:35:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.862+0000 7f607dffb700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f606c0a1230 0x7f606c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.862+0000 7f6084886700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f606c0a1230 0x7f606c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:50.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.862+0000 7f607dffb700 1 -- 172.21.15.46:0/2986568576 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f60680ccd90 con 0x7f60801040b0 2024-03-18T20:35:50.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.863+0000 7f6084886700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f606c0a1230 0x7f606c0a36f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f60801981c0 tx=0x7f6070009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:50.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:50.867+0000 7f607dffb700 1 -- 172.21.15.46:0/2986568576 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f60680d0030 con 0x7f60801040b0 2024-03-18T20:35:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.133+0000 7f6086aea700 1 -- 172.21.15.46:0/2986568576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f6080067c30 con 0x7f60801040b0 2024-03-18T20:35:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.158+0000 7f607dffb700 1 -- 172.21.15.46:0/2986568576 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) v1 ==== 117+0+0 (secure 0 0 0) 0x7f60680963d0 con 0x7f60801040b0 2024-03-18T20:35:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.161+0000 7f6086aea700 1 -- 172.21.15.46:0/2986568576 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f606c0a1230 msgr2=0x7f606c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.161+0000 7f6086aea700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f606c0a1230 0x7f606c0a36f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f60801981c0 tx=0x7f6070009260 comp rx=0 tx=0).stop 2024-03-18T20:35:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.161+0000 7f6086aea700 1 -- 172.21.15.46:0/2986568576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60801040b0 msgr2=0x7f6080197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.161+0000 7f6086aea700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60801040b0 0x7f6080197280 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f606800d950 tx=0x7f606800dd10 comp rx=0 tx=0).stop 2024-03-18T20:35:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.161+0000 7f6086aea700 1 -- 172.21.15.46:0/2986568576 shutdown_connections 2024-03-18T20:35:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.161+0000 7f6086aea700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f606c0a1230 0x7f606c0a36f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.161+0000 7f6086aea700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60801069e0 0x7f608019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.161+0000 7f6086aea700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6080101780 0x7f6080196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.161+0000 7f6086aea700 1 --2- 172.21.15.46:0/2986568576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60801040b0 0x7f6080197280 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.161+0000 7f6086aea700 1 -- 172.21.15.46:0/2986568576 >> 172.21.15.46:0/2986568576 conn(0x7f60800fb170 msgr2=0x7f6080102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.161+0000 7f6086aea700 1 -- 172.21.15.46:0/2986568576 shutdown_connections 2024-03-18T20:35:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.161+0000 7f6086aea700 1 -- 172.21.15.46:0/2986568576 wait complete. 2024-03-18T20:35:51.178 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-18T20:35:51.190 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-18T20:35:51.200 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:35:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.338+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2694670147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce40ffb90 msgr2=0x7f4ce40fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.338+0000 7f4cec7fd700 1 --2- 172.21.15.46:0/2694670147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce40ffb90 0x7f4ce40fffb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00099c0 tx=0x7f4ce0009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.338+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2694670147 shutdown_connections 2024-03-18T20:35:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.338+0000 7f4cec7fd700 1 --2- 172.21.15.46:0/2694670147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce40ffb90 0x7f4ce40fffb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.338+0000 7f4cec7fd700 1 --2- 172.21.15.46:0/2694670147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ce4101720 0x7f4ce4107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.338+0000 7f4cec7fd700 1 --2- 172.21.15.46:0/2694670147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ce4100d80 0x7f4ce41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.338+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2694670147 >> 172.21.15.46:0/2694670147 conn(0x7f4ce40fb100 msgr2=0x7f4ce40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.338+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2694670147 shutdown_connections 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.338+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2694670147 wait complete. 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.339+0000 7f4cec7fd700 1 Processor -- start 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.339+0000 7f4cec7fd700 1 -- start start 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.339+0000 7f4cec7fd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ce40ffb90 0x7f4ce4069d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.339+0000 7f4cec7fd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ce4100d80 0x7f4ce419dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.339+0000 7f4cec7fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce4101720 0x7f4ce41a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.339+0000 7f4cec7fd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce41a0910 con 0x7f4ce4101720 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.339+0000 7f4cec7fd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce41a0a80 con 0x7f4ce40ffb90 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.339+0000 7f4cec7fd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce41a0c40 con 0x7f4ce4100d80 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.339+0000 7f4cead9a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce4101720 0x7f4ce41a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.339+0000 7f4cead9a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce4101720 0x7f4ce41a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34458/0 (socket says 172.21.15.46:34458) 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.339+0000 7f4cead9a700 1 -- 172.21.15.46:0/2409609594 learned_addr learned my addr 172.21.15.46:0/2409609594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4cea599700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ce40ffb90 0x7f4ce4069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4ce9d98700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ce4100d80 0x7f4ce419dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4cead9a700 1 -- 172.21.15.46:0/2409609594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ce4100d80 msgr2=0x7f4ce419dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4cead9a700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ce4100d80 0x7f4ce419dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4cead9a700 1 -- 172.21.15.46:0/2409609594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ce40ffb90 msgr2=0x7f4ce4069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4cead9a700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ce40ffb90 0x7f4ce4069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4cead9a700 1 -- 172.21.15.46:0/2409609594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce00095f0 con 0x7f4ce4101720 2024-03-18T20:35:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4cead9a700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce4101720 0x7f4ce41a02b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00ef30 tx=0x7f4cdc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4cdb7fe700 1 -- 172.21.15.46:0/2409609594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc018070 con 0x7f4ce4101720 2024-03-18T20:35:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4cdb7fe700 1 -- 172.21.15.46:0/2409609594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cdc00f040 con 0x7f4ce4101720 2024-03-18T20:35:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4cdb7fe700 1 -- 172.21.15.46:0/2409609594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc00cd00 con 0x7f4ce4101720 2024-03-18T20:35:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2409609594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce41a0ec0 con 0x7f4ce4101720 2024-03-18T20:35:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.340+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2409609594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce41a12f0 con 0x7f4ce4101720 2024-03-18T20:35:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.341+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2409609594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce4107010 con 0x7f4ce4101720 2024-03-18T20:35:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.343+0000 7f4cdb7fe700 1 -- 172.21.15.46:0/2409609594 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4cdc003a00 con 0x7f4ce4101720 2024-03-18T20:35:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.343+0000 7f4cdb7fe700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4cd00a11b0 0x7f4cd00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.344+0000 7f4cea599700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4cd00a11b0 0x7f4cd00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.344+0000 7f4cdb7fe700 1 -- 172.21.15.46:0/2409609594 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4cdc0ccf40 con 0x7f4ce4101720 2024-03-18T20:35:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.344+0000 7f4cea599700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4cd00a11b0 0x7f4cd00a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0003730 tx=0x7f4ce0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:51.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.347+0000 7f4cdb7fe700 1 -- 172.21.15.46:0/2409609594 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4cdc096610 con 0x7f4ce4101720 2024-03-18T20:35:51.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:51 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2986568576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-18T20:35:51.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:51 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2986568576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-18T20:35:51.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:51 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2986568576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-18T20:35:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.582+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2409609594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ce4067e60 con 0x7f4ce4101720 2024-03-18T20:35:51.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.584+0000 7f4cdb7fe700 1 -- 172.21.15.46:0/2409609594 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22503 (secure 0 0 0) 0x7f4cdc099c30 con 0x7f4ce4101720 2024-03-18T20:35:51.587 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:51.588 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:35:38.108050+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:35:51.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.586+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2409609594 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4cd00a11b0 msgr2=0x7f4cd00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:51.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.586+0000 7f4cec7fd700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4cd00a11b0 0x7f4cd00a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0003730 tx=0x7f4ce0000bc0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.586+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2409609594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce4101720 msgr2=0x7f4ce41a02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:51.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.586+0000 7f4cec7fd700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce4101720 0x7f4ce41a02b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00ef30 tx=0x7f4cdc00c3a0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.586+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2409609594 shutdown_connections 2024-03-18T20:35:51.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.586+0000 7f4cec7fd700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f4cd00a11b0 0x7f4cd00a3670 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.586+0000 7f4cec7fd700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce4101720 0x7f4ce41a02b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.586+0000 7f4cec7fd700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ce40ffb90 0x7f4ce4069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.586+0000 7f4cec7fd700 1 --2- 172.21.15.46:0/2409609594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ce4100d80 0x7f4ce419dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.586+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2409609594 >> 172.21.15.46:0/2409609594 conn(0x7f4ce40fb100 msgr2=0x7f4ce4104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:51.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.586+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2409609594 shutdown_connections 2024-03-18T20:35:51.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.586+0000 7f4cec7fd700 1 -- 172.21.15.46:0/2409609594 wait complete. 2024-03-18T20:35:51.603 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T20:31:27.488318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T20:34:37.213560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T20:34:38.162692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T20:35:51.604 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T20:35:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.740+0000 7f5782213700 1 -- 172.21.15.46:0/2487115990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577c100d80 msgr2=0x7f577c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.740+0000 7f5782213700 1 --2- 172.21.15.46:0/2487115990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577c100d80 0x7f577c1011a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f57640099c0 tx=0x7f5764009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.740+0000 7f5782213700 1 -- 172.21.15.46:0/2487115990 shutdown_connections 2024-03-18T20:35:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.740+0000 7f5782213700 1 --2- 172.21.15.46:0/2487115990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577c100d80 0x7f577c1011a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.740+0000 7f5782213700 1 --2- 172.21.15.46:0/2487115990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f577c102910 0x7f577c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.740+0000 7f5782213700 1 --2- 172.21.15.46:0/2487115990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f577c101f70 0x7f577c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.740+0000 7f5782213700 1 -- 172.21.15.46:0/2487115990 >> 172.21.15.46:0/2487115990 conn(0x7f577c0fc2f0 msgr2=0x7f577c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.740+0000 7f5782213700 1 -- 172.21.15.46:0/2487115990 shutdown_connections 2024-03-18T20:35:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.740+0000 7f5782213700 1 -- 172.21.15.46:0/2487115990 wait complete. 2024-03-18T20:35:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.741+0000 7f5782213700 1 Processor -- start 2024-03-18T20:35:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f5782213700 1 -- start start 2024-03-18T20:35:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f5782213700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f577c100d80 0x7f577c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f5782213700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f577c101f70 0x7f577c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f5782213700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577c102910 0x7f577c1a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f5782213700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c1a0a60 con 0x7f577c102910 2024-03-18T20:35:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f5782213700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c1a0ba0 con 0x7f577c100d80 2024-03-18T20:35:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f5782213700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c1a0d30 con 0x7f577c101f70 2024-03-18T20:35:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f577bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577c102910 0x7f577c1a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f577bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577c102910 0x7f577c1a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34476/0 (socket says 172.21.15.46:34476) 2024-03-18T20:35:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f577bfff700 1 -- 172.21.15.46:0/1489393893 learned_addr learned my addr 172.21.15.46:0/1489393893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f577b7fe700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f577c100d80 0x7f577c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f577affd700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f577c101f70 0x7f577c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f577bfff700 1 -- 172.21.15.46:0/1489393893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f577c101f70 msgr2=0x7f577c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f577bfff700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f577c101f70 0x7f577c199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f577bfff700 1 -- 172.21.15.46:0/1489393893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f577c100d80 msgr2=0x7f577c198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f577bfff700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f577c100d80 0x7f577c198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f577bfff700 1 -- 172.21.15.46:0/1489393893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57640095f0 con 0x7f577c102910 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.742+0000 7f577affd700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f577c101f70 0x7f577c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.743+0000 7f577bfff700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577c102910 0x7f577c1a0490 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5774007ee0 tx=0x7f577400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.743+0000 7f5778ff9700 1 -- 172.21.15.46:0/1489393893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774017070 con 0x7f577c102910 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.743+0000 7f5778ff9700 1 -- 172.21.15.46:0/1489393893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f577400f040 con 0x7f577c102910 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.743+0000 7f5782213700 1 -- 172.21.15.46:0/1489393893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c1a0fb0 con 0x7f577c102910 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.743+0000 7f5778ff9700 1 -- 172.21.15.46:0/1489393893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774005740 con 0x7f577c102910 2024-03-18T20:35:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.743+0000 7f5782213700 1 -- 172.21.15.46:0/1489393893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c1a1440 con 0x7f577c102910 2024-03-18T20:35:51.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.744+0000 7f5782213700 1 -- 172.21.15.46:0/1489393893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577c067260 con 0x7f577c102910 2024-03-18T20:35:51.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.745+0000 7f5778ff9700 1 -- 172.21.15.46:0/1489393893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f577400dc60 con 0x7f577c102910 2024-03-18T20:35:51.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.746+0000 7f5778ff9700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f57680a11b0 0x7f57680a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:51.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.746+0000 7f5778ff9700 1 -- 172.21.15.46:0/1489393893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f57740cc220 con 0x7f577c102910 2024-03-18T20:35:51.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.746+0000 7f577b7fe700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f57680a11b0 0x7f57680a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:51.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.746+0000 7f577b7fe700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f57680a11b0 0x7f57680a3670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5764003820 tx=0x7f5764005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:51.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.750+0000 7f5778ff9700 1 -- 172.21.15.46:0/1489393893 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5774095800 con 0x7f577c102910 2024-03-18T20:35:51.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.964+0000 7f5782213700 1 -- 172.21.15.46:0/1489393893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f577c1a1630 con 0x7f577c102910 2024-03-18T20:35:51.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.965+0000 7f5778ff9700 1 -- 172.21.15.46:0/1489393893 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5774098e20 con 0x7f577c102910 2024-03-18T20:35:51.968 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-18T20:35:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.967+0000 7f5782213700 1 -- 172.21.15.46:0/1489393893 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f57680a11b0 msgr2=0x7f57680a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.967+0000 7f5782213700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f57680a11b0 0x7f57680a3670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5764003820 tx=0x7f5764005920 comp rx=0 tx=0).stop 2024-03-18T20:35:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.967+0000 7f5782213700 1 -- 172.21.15.46:0/1489393893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577c102910 msgr2=0x7f577c1a0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.967+0000 7f5782213700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577c102910 0x7f577c1a0490 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5774007ee0 tx=0x7f577400d3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.967+0000 7f5782213700 1 -- 172.21.15.46:0/1489393893 shutdown_connections 2024-03-18T20:35:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.967+0000 7f5782213700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f57680a11b0 0x7f57680a3670 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.967+0000 7f5782213700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f577c102910 0x7f577c1a0490 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.967+0000 7f5782213700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f577c100d80 0x7f577c198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.967+0000 7f5782213700 1 --2- 172.21.15.46:0/1489393893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f577c101f70 0x7f577c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:51.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.967+0000 7f5782213700 1 -- 172.21.15.46:0/1489393893 >> 172.21.15.46:0/1489393893 conn(0x7f577c0fc2f0 msgr2=0x7f577c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:51.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.968+0000 7f5782213700 1 -- 172.21.15.46:0/1489393893 shutdown_connections 2024-03-18T20:35:51.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:51.968+0000 7f5782213700 1 -- 172.21.15.46:0/1489393893 wait complete. 2024-03-18T20:35:51.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T20:35:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.122+0000 7f313092a700 1 -- 172.21.15.46:0/3962131655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c104110 msgr2=0x7f312c106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.122+0000 7f313092a700 1 --2- 172.21.15.46:0/3962131655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c104110 0x7f312c106500 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3114009990 tx=0x7f3114009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.122+0000 7f313092a700 1 -- 172.21.15.46:0/3962131655 shutdown_connections 2024-03-18T20:35:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.122+0000 7f313092a700 1 --2- 172.21.15.46:0/3962131655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f312c106a40 0x7f312c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.122+0000 7f313092a700 1 --2- 172.21.15.46:0/3962131655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f312c1017e0 0x7f312c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.122+0000 7f313092a700 1 --2- 172.21.15.46:0/3962131655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c104110 0x7f312c106500 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.122+0000 7f313092a700 1 -- 172.21.15.46:0/3962131655 >> 172.21.15.46:0/3962131655 conn(0x7f312c0fb1d0 msgr2=0x7f312c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.122+0000 7f313092a700 1 -- 172.21.15.46:0/3962131655 shutdown_connections 2024-03-18T20:35:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.122+0000 7f313092a700 1 -- 172.21.15.46:0/3962131655 wait complete. 2024-03-18T20:35:52.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.123+0000 7f313092a700 1 Processor -- start 2024-03-18T20:35:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.123+0000 7f313092a700 1 -- start start 2024-03-18T20:35:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.123+0000 7f313092a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f312c1017e0 0x7f312c196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.123+0000 7f313092a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f312c104110 0x7f312c197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.123+0000 7f313092a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c106a40 0x7f312c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.123+0000 7f313092a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c19e760 con 0x7f312c106a40 2024-03-18T20:35:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.123+0000 7f313092a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c19e8a0 con 0x7f312c104110 2024-03-18T20:35:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.123+0000 7f313092a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c19ea30 con 0x7f312c1017e0 2024-03-18T20:35:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.123+0000 7f312ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c106a40 0x7f312c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.123+0000 7f312ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c106a40 0x7f312c19e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34482/0 (socket says 172.21.15.46:34482) 2024-03-18T20:35:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.123+0000 7f312ad9d700 1 -- 172.21.15.46:0/1049258938 learned_addr learned my addr 172.21.15.46:0/1049258938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.123+0000 7f3121d9b700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f312c104110 0x7f312c197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:52.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f312a59c700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f312c1017e0 0x7f312c196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f312ad9d700 1 -- 172.21.15.46:0/1049258938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f312c1017e0 msgr2=0x7f312c196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f312ad9d700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f312c1017e0 0x7f312c196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f312ad9d700 1 -- 172.21.15.46:0/1049258938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f312c104110 msgr2=0x7f312c197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f312ad9d700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f312c104110 0x7f312c197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f312ad9d700 1 -- 172.21.15.46:0/1049258938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31140095f0 con 0x7f312c106a40 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f3121d9b700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f312c104110 0x7f312c197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f312ad9d700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c106a40 0x7f312c19e190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3124007d70 tx=0x7f312400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f3123fff700 1 -- 172.21.15.46:0/1049258938 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312400ac90 con 0x7f312c106a40 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f3123fff700 1 -- 172.21.15.46:0/1049258938 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f312400adf0 con 0x7f312c106a40 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f313092a700 1 -- 172.21.15.46:0/1049258938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312c19ed10 con 0x7f312c106a40 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f3123fff700 1 -- 172.21.15.46:0/1049258938 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31240115e0 con 0x7f312c106a40 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.124+0000 7f313092a700 1 -- 172.21.15.46:0/1049258938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312c19f1d0 con 0x7f312c106a40 2024-03-18T20:35:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.125+0000 7f313092a700 1 -- 172.21.15.46:0/1049258938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312c0fcd50 con 0x7f312c106a40 2024-03-18T20:35:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.127+0000 7f3123fff700 1 -- 172.21.15.46:0/1049258938 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3124012420 con 0x7f312c106a40 2024-03-18T20:35:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.127+0000 7f3123fff700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f31180a11b0 0x7f31180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.127+0000 7f312a59c700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f31180a11b0 0x7f31180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.127+0000 7f3123fff700 1 -- 172.21.15.46:0/1049258938 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f31240ccb10 con 0x7f312c106a40 2024-03-18T20:35:52.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.128+0000 7f312a59c700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f31180a11b0 0x7f31180a3670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f311c005950 tx=0x7f311c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:52.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.132+0000 7f3123fff700 1 -- 172.21.15.46:0/1049258938 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3124096260 con 0x7f312c106a40 2024-03-18T20:35:52.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:52 smithi046 ceph-mon[43495]: pgmap v257: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-03-18T20:35:52.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:52 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2986568576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-18T20:35:52.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:52 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:35:52.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:52 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2409609594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:52.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:52 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1489393893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.349+0000 7f313092a700 1 -- 172.21.15.46:0/1049258938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f312c02d100 con 0x7f312c106a40 2024-03-18T20:35:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.349+0000 7f3123fff700 1 -- 172.21.15.46:0/1049258938 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3124099880 con 0x7f312c106a40 2024-03-18T20:35:52.353 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-18T20:35:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.351+0000 7f313092a700 1 -- 172.21.15.46:0/1049258938 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f31180a11b0 msgr2=0x7f31180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.351+0000 7f313092a700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f31180a11b0 0x7f31180a3670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f311c005950 tx=0x7f311c00b410 comp rx=0 tx=0).stop 2024-03-18T20:35:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.351+0000 7f313092a700 1 -- 172.21.15.46:0/1049258938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c106a40 msgr2=0x7f312c19e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.351+0000 7f313092a700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c106a40 0x7f312c19e190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3124007d70 tx=0x7f312400a340 comp rx=0 tx=0).stop 2024-03-18T20:35:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.351+0000 7f313092a700 1 -- 172.21.15.46:0/1049258938 shutdown_connections 2024-03-18T20:35:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.351+0000 7f313092a700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f31180a11b0 0x7f31180a3670 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.351+0000 7f313092a700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f312c106a40 0x7f312c19e190 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.351+0000 7f313092a700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f312c1017e0 0x7f312c196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.351+0000 7f313092a700 1 --2- 172.21.15.46:0/1049258938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f312c104110 0x7f312c197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.351+0000 7f313092a700 1 -- 172.21.15.46:0/1049258938 >> 172.21.15.46:0/1049258938 conn(0x7f312c0fb1d0 msgr2=0x7f312c102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:52.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.352+0000 7f313092a700 1 -- 172.21.15.46:0/1049258938 shutdown_connections 2024-03-18T20:35:52.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.352+0000 7f313092a700 1 -- 172.21.15.46:0/1049258938 wait complete. 2024-03-18T20:35:52.369 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T20:35:52.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:52 smithi073 ceph-mon[47384]: pgmap v257: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-03-18T20:35:52.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:52 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2986568576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-18T20:35:52.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:52 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:35:52.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:52 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2409609594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:52.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:52 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1489393893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.506+0000 7f979c7e0700 1 -- 172.21.15.46:0/1795336035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9794101820 msgr2=0x7f97941080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.506+0000 7f979c7e0700 1 --2- 172.21.15.46:0/1795336035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9794101820 0x7f97941080d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f978c00b0d0 tx=0x7f978c00b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.506+0000 7f979c7e0700 1 -- 172.21.15.46:0/1795336035 shutdown_connections 2024-03-18T20:35:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.506+0000 7f979c7e0700 1 --2- 172.21.15.46:0/1795336035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9794101820 0x7f97941080d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.506+0000 7f979c7e0700 1 --2- 172.21.15.46:0/1795336035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97940ffc90 0x7f97941000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.506+0000 7f979c7e0700 1 --2- 172.21.15.46:0/1795336035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9794100e80 0x7f97941012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.506+0000 7f979c7e0700 1 -- 172.21.15.46:0/1795336035 >> 172.21.15.46:0/1795336035 conn(0x7f97940fb200 msgr2=0x7f97940fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.506+0000 7f979c7e0700 1 -- 172.21.15.46:0/1795336035 shutdown_connections 2024-03-18T20:35:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.506+0000 7f979c7e0700 1 -- 172.21.15.46:0/1795336035 wait complete. 2024-03-18T20:35:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979c7e0700 1 Processor -- start 2024-03-18T20:35:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979c7e0700 1 -- start start 2024-03-18T20:35:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979c7e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97940ffc90 0x7f9794198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979c7e0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9794100e80 0x7f9794199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979c7e0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9794101820 0x7f97941a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979c7e0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97941a0b00 con 0x7f97940ffc90 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979c7e0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97941a0c40 con 0x7f9794100e80 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979c7e0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97941a0dd0 con 0x7f9794101820 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979a57c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97940ffc90 0x7f9794198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979a57c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97940ffc90 0x7f9794198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34490/0 (socket says 172.21.15.46:34490) 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979a57c700 1 -- 172.21.15.46:0/1949297727 learned_addr learned my addr 172.21.15.46:0/1949297727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979ad7d700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9794101820 0x7f97941a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f9799d7b700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9794100e80 0x7f9794199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979a57c700 1 -- 172.21.15.46:0/1949297727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9794101820 msgr2=0x7f97941a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979a57c700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9794101820 0x7f97941a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979a57c700 1 -- 172.21.15.46:0/1949297727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9794100e80 msgr2=0x7f9794199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979a57c700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9794100e80 0x7f9794199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.507+0000 7f979a57c700 1 -- 172.21.15.46:0/1949297727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c009d00 con 0x7f97940ffc90 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.508+0000 7f979a57c700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97940ffc90 0x7f9794198fd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f979000d980 tx=0x7f979000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.508+0000 7f978b7fe700 1 -- 172.21.15.46:0/1949297727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979000f890 con 0x7f97940ffc90 2024-03-18T20:35:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.508+0000 7f978b7fe700 1 -- 172.21.15.46:0/1949297727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9790010460 con 0x7f97940ffc90 2024-03-18T20:35:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.508+0000 7f979c7e0700 1 -- 172.21.15.46:0/1949297727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97941a10b0 con 0x7f97940ffc90 2024-03-18T20:35:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.508+0000 7f978b7fe700 1 -- 172.21.15.46:0/1949297727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97900195c0 con 0x7f97940ffc90 2024-03-18T20:35:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.508+0000 7f979c7e0700 1 -- 172.21.15.46:0/1949297727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97941a1610 con 0x7f97940ffc90 2024-03-18T20:35:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.509+0000 7f979c7e0700 1 -- 172.21.15.46:0/1949297727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9794067260 con 0x7f97940ffc90 2024-03-18T20:35:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.511+0000 7f978b7fe700 1 -- 172.21.15.46:0/1949297727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f97900105d0 con 0x7f97940ffc90 2024-03-18T20:35:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.511+0000 7f978b7fe700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f97800a1230 0x7f97800a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.512+0000 7f9799d7b700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f97800a1230 0x7f97800a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.512+0000 7f978b7fe700 1 -- 172.21.15.46:0/1949297727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f97900ccce0 con 0x7f97940ffc90 2024-03-18T20:35:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.512+0000 7f9799d7b700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f97800a1230 0x7f97800a36f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9784009be0 tx=0x7f9784009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:52.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.515+0000 7f978b7fe700 1 -- 172.21.15.46:0/1949297727 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f97900962c0 con 0x7f97940ffc90 2024-03-18T20:35:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:52 smithi155 ceph-mon[48344]: pgmap v257: 97 pgs: 97 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 11 B/s, 0 objects/s recovering 2024-03-18T20:35:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:52 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2986568576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-18T20:35:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:52 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:35:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:52 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2409609594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:52.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:52 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1489393893' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.734+0000 7f979c7e0700 1 -- 172.21.15.46:0/1949297727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97941a1890 con 0x7f97940ffc90 2024-03-18T20:35:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.734+0000 7f978b7fe700 1 -- 172.21.15.46:0/1949297727 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97900200a0 con 0x7f97940ffc90 2024-03-18T20:35:52.738 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-18T20:35:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.737+0000 7f979c7e0700 1 -- 172.21.15.46:0/1949297727 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f97800a1230 msgr2=0x7f97800a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.737+0000 7f979c7e0700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f97800a1230 0x7f97800a36f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9784009be0 tx=0x7f9784009260 comp rx=0 tx=0).stop 2024-03-18T20:35:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.737+0000 7f979c7e0700 1 -- 172.21.15.46:0/1949297727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97940ffc90 msgr2=0x7f9794198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.737+0000 7f979c7e0700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97940ffc90 0x7f9794198fd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f979000d980 tx=0x7f979000dc90 comp rx=0 tx=0).stop 2024-03-18T20:35:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.737+0000 7f979c7e0700 1 -- 172.21.15.46:0/1949297727 shutdown_connections 2024-03-18T20:35:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.737+0000 7f979c7e0700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f97800a1230 0x7f97800a36f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.737+0000 7f979c7e0700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97940ffc90 0x7f9794198fd0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.737+0000 7f979c7e0700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9794101820 0x7f97941a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.737+0000 7f979c7e0700 1 --2- 172.21.15.46:0/1949297727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9794100e80 0x7f9794199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.737+0000 7f979c7e0700 1 -- 172.21.15.46:0/1949297727 >> 172.21.15.46:0/1949297727 conn(0x7f97940fb200 msgr2=0x7f97940fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:52.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.737+0000 7f979c7e0700 1 -- 172.21.15.46:0/1949297727 shutdown_connections 2024-03-18T20:35:52.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.737+0000 7f979c7e0700 1 -- 172.21.15.46:0/1949297727 wait complete. 2024-03-18T20:35:52.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:35:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.889+0000 7f12c5470700 1 -- 172.21.15.46:0/1752167163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c0101e60 msgr2=0x7f12c01022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.889+0000 7f12c5470700 1 --2- 172.21.15.46:0/1752167163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c0101e60 0x7f12c01022c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12a80099c0 tx=0x7f12a8009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.889+0000 7f12c5470700 1 -- 172.21.15.46:0/1752167163 shutdown_connections 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.889+0000 7f12c5470700 1 --2- 172.21.15.46:0/1752167163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c0102800 0x7f12c01090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.889+0000 7f12c5470700 1 --2- 172.21.15.46:0/1752167163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c0100c70 0x7f12c0101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.889+0000 7f12c5470700 1 --2- 172.21.15.46:0/1752167163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c0101e60 0x7f12c01022c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.889+0000 7f12c5470700 1 -- 172.21.15.46:0/1752167163 >> 172.21.15.46:0/1752167163 conn(0x7f12c00fc1e0 msgr2=0x7f12c00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.889+0000 7f12c5470700 1 -- 172.21.15.46:0/1752167163 shutdown_connections 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.889+0000 7f12c5470700 1 -- 172.21.15.46:0/1752167163 wait complete. 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.889+0000 7f12c5470700 1 Processor -- start 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.889+0000 7f12c5470700 1 -- start start 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12c5470700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c0100c70 0x7f12c0196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12c5470700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c0101e60 0x7f12c0197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12c5470700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c0102800 0x7f12c019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12c5470700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c019e720 con 0x7f12c0102800 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12c5470700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c019e860 con 0x7f12c0100c70 2024-03-18T20:35:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12c5470700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c019e9f0 con 0x7f12c0101e60 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12bf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c0102800 0x7f12c019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12bf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c0102800 0x7f12c019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34518/0 (socket says 172.21.15.46:34518) 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12bf7fe700 1 -- 172.21.15.46:0/867781981 learned_addr learned my addr 172.21.15.46:0/867781981 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12beffd700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c0100c70 0x7f12c0196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12b7fff700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c0101e60 0x7f12c0197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12bf7fe700 1 -- 172.21.15.46:0/867781981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c0101e60 msgr2=0x7f12c0197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12bf7fe700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c0101e60 0x7f12c0197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12bf7fe700 1 -- 172.21.15.46:0/867781981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c0100c70 msgr2=0x7f12c0196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12bf7fe700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c0100c70 0x7f12c0196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12bf7fe700 1 -- 172.21.15.46:0/867781981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a80095f0 con 0x7f12c0102800 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12b7fff700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c0101e60 0x7f12c0197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12bf7fe700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c0102800 0x7f12c019e150 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f12b800ef30 tx=0x7f12b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.890+0000 7f12bcff9700 1 -- 172.21.15.46:0/867781981 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b8018070 con 0x7f12c0102800 2024-03-18T20:35:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.891+0000 7f12c5470700 1 -- 172.21.15.46:0/867781981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c019ec70 con 0x7f12c0102800 2024-03-18T20:35:52.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.891+0000 7f12bcff9700 1 -- 172.21.15.46:0/867781981 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12b800f040 con 0x7f12c0102800 2024-03-18T20:35:52.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.891+0000 7f12bcff9700 1 -- 172.21.15.46:0/867781981 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b800cdb0 con 0x7f12c0102800 2024-03-18T20:35:52.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.891+0000 7f12c5470700 1 -- 172.21.15.46:0/867781981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c019f180 con 0x7f12c0102800 2024-03-18T20:35:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.892+0000 7f12c5470700 1 -- 172.21.15.46:0/867781981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c01080f0 con 0x7f12c0102800 2024-03-18T20:35:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.893+0000 7f12bcff9700 1 -- 172.21.15.46:0/867781981 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f12b8003a00 con 0x7f12c0102800 2024-03-18T20:35:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.894+0000 7f12bcff9700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f12ac0a11b0 0x7f12ac0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.894+0000 7f12beffd700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f12ac0a11b0 0x7f12ac0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.894+0000 7f12bcff9700 1 -- 172.21.15.46:0/867781981 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f12b80ccff0 con 0x7f12c0102800 2024-03-18T20:35:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.894+0000 7f12beffd700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f12ac0a11b0 0x7f12ac0a3670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f12c0101cc0 tx=0x7f12b0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:52.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:52.897+0000 7f12bcff9700 1 -- 172.21.15.46:0/867781981 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f12b8096660 con 0x7f12c0102800 2024-03-18T20:35:53.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.126+0000 7f12c5470700 1 -- 172.21.15.46:0/867781981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12c002ccb0 con 0x7f12c0102800 2024-03-18T20:35:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.127+0000 7f12bcff9700 1 -- 172.21.15.46:0/867781981 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22503 (secure 0 0 0) 0x7f12b8099c80 con 0x7f12c0102800 2024-03-18T20:35:53.131 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:53.131 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:35:38.108050+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:35:53.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.129+0000 7f12c5470700 1 -- 172.21.15.46:0/867781981 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f12ac0a11b0 msgr2=0x7f12ac0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:53.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.129+0000 7f12c5470700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f12ac0a11b0 0x7f12ac0a3670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f12c0101cc0 tx=0x7f12b0007400 comp rx=0 tx=0).stop 2024-03-18T20:35:53.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.129+0000 7f12c5470700 1 -- 172.21.15.46:0/867781981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c0102800 msgr2=0x7f12c019e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:53.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.129+0000 7f12c5470700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c0102800 0x7f12c019e150 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f12b800ef30 tx=0x7f12b800c3a0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.130+0000 7f12c5470700 1 -- 172.21.15.46:0/867781981 shutdown_connections 2024-03-18T20:35:53.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.130+0000 7f12c5470700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f12ac0a11b0 0x7f12ac0a3670 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.130+0000 7f12c5470700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c0102800 0x7f12c019e150 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.130+0000 7f12c5470700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c0100c70 0x7f12c0196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.130+0000 7f12c5470700 1 --2- 172.21.15.46:0/867781981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12c0101e60 0x7f12c0197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.130+0000 7f12c5470700 1 -- 172.21.15.46:0/867781981 >> 172.21.15.46:0/867781981 conn(0x7f12c00fc1e0 msgr2=0x7f12c0105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:53.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.130+0000 7f12c5470700 1 -- 172.21.15.46:0/867781981 shutdown_connections 2024-03-18T20:35:53.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.130+0000 7f12c5470700 1 -- 172.21.15.46:0/867781981 wait complete. 2024-03-18T20:35:53.147 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T20:31:27.488318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T20:34:37.213560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T20:34:38.162692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T20:35:53.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T20:35:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.286+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2388427316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d8101810 msgr2=0x7ff0d8103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.286+0000 7ff0dd5ce700 1 --2- 172.21.15.46:0/2388427316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d8101810 0x7ff0d8103c00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009990 tx=0x7ff0c8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.286+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2388427316 shutdown_connections 2024-03-18T20:35:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.286+0000 7ff0dd5ce700 1 --2- 172.21.15.46:0/2388427316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d8101810 0x7ff0d8103c00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.286+0000 7ff0dd5ce700 1 --2- 172.21.15.46:0/2388427316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d8106a70 0x7ff0d8108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.286+0000 7ff0dd5ce700 1 --2- 172.21.15.46:0/2388427316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0d8104140 0x7ff0d8106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.286+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2388427316 >> 172.21.15.46:0/2388427316 conn(0x7ff0d80fb200 msgr2=0x7ff0d80fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.286+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2388427316 shutdown_connections 2024-03-18T20:35:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.286+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2388427316 wait complete. 2024-03-18T20:35:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0dd5ce700 1 Processor -- start 2024-03-18T20:35:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0dd5ce700 1 -- start start 2024-03-18T20:35:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0dd5ce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d8101810 0x7ff0d8196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0dd5ce700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d8104140 0x7ff0d8197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0dd5ce700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0d8106a70 0x7ff0d819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0dd5ce700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d819e7e0 con 0x7ff0d8101810 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0dd5ce700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d819e950 con 0x7ff0d8106a70 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0dd5ce700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d819eb10 con 0x7ff0d8104140 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0d6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d8101810 0x7ff0d8196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0d6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d8101810 0x7ff0d8196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34530/0 (socket says 172.21.15.46:34530) 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0d6ffd700 1 -- 172.21.15.46:0/2339780551 learned_addr learned my addr 172.21.15.46:0/2339780551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0d77fe700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0d8106a70 0x7ff0d819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0d67fc700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d8104140 0x7ff0d8197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0d6ffd700 1 -- 172.21.15.46:0/2339780551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d8104140 msgr2=0x7ff0d8197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0d6ffd700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d8104140 0x7ff0d8197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.287+0000 7ff0d6ffd700 1 -- 172.21.15.46:0/2339780551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0d8106a70 msgr2=0x7ff0d819e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.288+0000 7ff0d6ffd700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0d8106a70 0x7ff0d819e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.288+0000 7ff0d6ffd700 1 -- 172.21.15.46:0/2339780551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c80095f0 con 0x7ff0d8101810 2024-03-18T20:35:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.288+0000 7ff0d6ffd700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d8101810 0x7ff0d8196cc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80095c0 tx=0x7ff0c800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.288+0000 7ff0cffff700 1 -- 172.21.15.46:0/2339780551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c801d070 con 0x7ff0d8101810 2024-03-18T20:35:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.288+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2339780551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d819ed90 con 0x7ff0d8101810 2024-03-18T20:35:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.288+0000 7ff0cffff700 1 -- 172.21.15.46:0/2339780551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0c8005510 con 0x7ff0d8101810 2024-03-18T20:35:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.288+0000 7ff0cffff700 1 -- 172.21.15.46:0/2339780551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8004c20 con 0x7ff0d8101810 2024-03-18T20:35:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.288+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2339780551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d819f1b0 con 0x7ff0d8101810 2024-03-18T20:35:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.289+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2339780551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d80fcd80 con 0x7ff0d8101810 2024-03-18T20:35:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.290+0000 7ff0cffff700 1 -- 172.21.15.46:0/2339780551 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff0c800fb90 con 0x7ff0d8101810 2024-03-18T20:35:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.291+0000 7ff0cffff700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff0c40a11b0 0x7ff0c40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.291+0000 7ff0cffff700 1 -- 172.21.15.46:0/2339780551 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff0c80cd9e0 con 0x7ff0d8101810 2024-03-18T20:35:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.291+0000 7ff0d67fc700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff0c40a11b0 0x7ff0c40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.291+0000 7ff0d67fc700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff0c40a11b0 0x7ff0c40a3670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0005950 tx=0x7ff0c00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:53.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.295+0000 7ff0cffff700 1 -- 172.21.15.46:0/2339780551 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff0c80970b0 con 0x7ff0d8101810 2024-03-18T20:35:53.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:53 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1049258938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:53.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:53 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1949297727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:53.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:53 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/867781981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.509+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2339780551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0d802d0f0 con 0x7ff0d8101810 2024-03-18T20:35:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.510+0000 7ff0cffff700 1 -- 172.21.15.46:0/2339780551 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0c809a6d0 con 0x7ff0d8101810 2024-03-18T20:35:53.513 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-18T20:35:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.512+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2339780551 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff0c40a11b0 msgr2=0x7ff0c40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.512+0000 7ff0dd5ce700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff0c40a11b0 0x7ff0c40a3670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0005950 tx=0x7ff0c00058e0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.512+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2339780551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d8101810 msgr2=0x7ff0d8196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.512+0000 7ff0dd5ce700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d8101810 0x7ff0d8196cc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80095c0 tx=0x7ff0c800f690 comp rx=0 tx=0).stop 2024-03-18T20:35:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.512+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2339780551 shutdown_connections 2024-03-18T20:35:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.512+0000 7ff0dd5ce700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7ff0c40a11b0 0x7ff0c40a3670 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.512+0000 7ff0dd5ce700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d8101810 0x7ff0d8196cc0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.512+0000 7ff0dd5ce700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0d8106a70 0x7ff0d819e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.512+0000 7ff0dd5ce700 1 --2- 172.21.15.46:0/2339780551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d8104140 0x7ff0d8197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.512+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2339780551 >> 172.21.15.46:0/2339780551 conn(0x7ff0d80fb200 msgr2=0x7ff0d81036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.512+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2339780551 shutdown_connections 2024-03-18T20:35:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.512+0000 7ff0dd5ce700 1 -- 172.21.15.46:0/2339780551 wait complete. 2024-03-18T20:35:53.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:53 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1049258938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:53.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:53 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1949297727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:53.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:53 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/867781981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:53.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T20:35:53.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:53 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1049258938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:53.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:53 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1949297727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:53.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:53 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/867781981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:53.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.666+0000 7f3674de6700 1 -- 172.21.15.46:0/2741126313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3670100e50 msgr2=0x7f36701012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:53.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.666+0000 7f3674de6700 1 --2- 172.21.15.46:0/2741126313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3670100e50 0x7f36701012b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f36580099c0 tx=0x7f3658009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.666+0000 7f3674de6700 1 -- 172.21.15.46:0/2741126313 shutdown_connections 2024-03-18T20:35:53.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.666+0000 7f3674de6700 1 --2- 172.21.15.46:0/2741126313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36701017f0 0x7f36701080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.666+0000 7f3674de6700 1 --2- 172.21.15.46:0/2741126313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36700ffc60 0x7f3670100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.666+0000 7f3674de6700 1 --2- 172.21.15.46:0/2741126313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3670100e50 0x7f36701012b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.666+0000 7f3674de6700 1 -- 172.21.15.46:0/2741126313 >> 172.21.15.46:0/2741126313 conn(0x7f36700fb1d0 msgr2=0x7f36700fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.666+0000 7f3674de6700 1 -- 172.21.15.46:0/2741126313 shutdown_connections 2024-03-18T20:35:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.666+0000 7f3674de6700 1 -- 172.21.15.46:0/2741126313 wait complete. 2024-03-18T20:35:53.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.667+0000 7f3674de6700 1 Processor -- start 2024-03-18T20:35:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.667+0000 7f3674de6700 1 -- start start 2024-03-18T20:35:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f3674de6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ffc60 0x7f3670103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f3674de6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3670100e50 0x7f3670102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f3674de6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36701017f0 0x7f3670102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f3674de6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3670102f10 con 0x7f36700ffc60 2024-03-18T20:35:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f3674de6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3670103080 con 0x7f3670100e50 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f3674de6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36701a0d90 con 0x7f36701017f0 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f366e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ffc60 0x7f3670103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f366e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ffc60 0x7f3670103ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34548/0 (socket says 172.21.15.46:34548) 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f366e59c700 1 -- 172.21.15.46:0/3484101613 learned_addr learned my addr 172.21.15.46:0/3484101613 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f366ed9d700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36701017f0 0x7f3670102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f3665d9b700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3670100e50 0x7f3670102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f366e59c700 1 -- 172.21.15.46:0/3484101613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36701017f0 msgr2=0x7f3670102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f366e59c700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36701017f0 0x7f3670102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f366e59c700 1 -- 172.21.15.46:0/3484101613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3670100e50 msgr2=0x7f3670102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f366e59c700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3670100e50 0x7f3670102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f366e59c700 1 -- 172.21.15.46:0/3484101613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36580095f0 con 0x7f36700ffc60 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.668+0000 7f366e59c700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ffc60 0x7f3670103ba0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f366000e970 tx=0x7f366000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.669+0000 7f3667fff700 1 -- 172.21.15.46:0/3484101613 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660009790 con 0x7f36700ffc60 2024-03-18T20:35:53.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.669+0000 7f3667fff700 1 -- 172.21.15.46:0/3484101613 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f366000c780 con 0x7f36700ffc60 2024-03-18T20:35:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.669+0000 7f3674de6700 1 -- 172.21.15.46:0/3484101613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36701a0fc0 con 0x7f36700ffc60 2024-03-18T20:35:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.669+0000 7f3667fff700 1 -- 172.21.15.46:0/3484101613 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660003be0 con 0x7f36700ffc60 2024-03-18T20:35:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.669+0000 7f3674de6700 1 -- 172.21.15.46:0/3484101613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36701a1440 con 0x7f36700ffc60 2024-03-18T20:35:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.670+0000 7f3674de6700 1 -- 172.21.15.46:0/3484101613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3670067af0 con 0x7f36700ffc60 2024-03-18T20:35:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.671+0000 7f3667fff700 1 -- 172.21.15.46:0/3484101613 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3660003d60 con 0x7f36700ffc60 2024-03-18T20:35:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.672+0000 7f3667fff700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f365c0a11b0 0x7f365c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.672+0000 7f3665d9b700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f365c0a11b0 0x7f365c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.672+0000 7f3667fff700 1 -- 172.21.15.46:0/3484101613 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3660014070 con 0x7f36700ffc60 2024-03-18T20:35:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.672+0000 7f3665d9b700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f365c0a11b0 0x7f365c0a3670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3658003730 tx=0x7f365800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.675+0000 7f3667fff700 1 -- 172.21.15.46:0/3484101613 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3660096380 con 0x7f36700ffc60 2024-03-18T20:35:53.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.893+0000 7f3674de6700 1 -- 172.21.15.46:0/3484101613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36701a1690 con 0x7f36700ffc60 2024-03-18T20:35:53.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.894+0000 7f3667fff700 1 -- 172.21.15.46:0/3484101613 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36600999a0 con 0x7f36700ffc60 2024-03-18T20:35:53.897 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-18T20:35:53.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.896+0000 7f3674de6700 1 -- 172.21.15.46:0/3484101613 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f365c0a11b0 msgr2=0x7f365c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:53.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.896+0000 7f3674de6700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f365c0a11b0 0x7f365c0a3670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3658003730 tx=0x7f365800b540 comp rx=0 tx=0).stop 2024-03-18T20:35:53.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.896+0000 7f3674de6700 1 -- 172.21.15.46:0/3484101613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ffc60 msgr2=0x7f3670103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:53.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.896+0000 7f3674de6700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ffc60 0x7f3670103ba0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f366000e970 tx=0x7f366000ec80 comp rx=0 tx=0).stop 2024-03-18T20:35:53.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.896+0000 7f3674de6700 1 -- 172.21.15.46:0/3484101613 shutdown_connections 2024-03-18T20:35:53.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.897+0000 7f3674de6700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f365c0a11b0 0x7f365c0a3670 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.897+0000 7f3674de6700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36700ffc60 0x7f3670103ba0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.897+0000 7f3674de6700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36701017f0 0x7f3670102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.897+0000 7f3674de6700 1 --2- 172.21.15.46:0/3484101613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3670100e50 0x7f3670102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:53.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.897+0000 7f3674de6700 1 -- 172.21.15.46:0/3484101613 >> 172.21.15.46:0/3484101613 conn(0x7f36700fb1d0 msgr2=0x7f36700fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:53.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.897+0000 7f3674de6700 1 -- 172.21.15.46:0/3484101613 shutdown_connections 2024-03-18T20:35:53.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:53.897+0000 7f3674de6700 1 -- 172.21.15.46:0/3484101613 wait complete. 2024-03-18T20:35:53.914 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T20:35:54.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.052+0000 7f2b51aa6700 1 -- 172.21.15.46:0/569848262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b4c1041a0 msgr2=0x7f2b4c106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.052+0000 7f2b51aa6700 1 --2- 172.21.15.46:0/569848262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b4c1041a0 0x7f2b4c106590 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009990 tx=0x7f2b40009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.052+0000 7f2b51aa6700 1 -- 172.21.15.46:0/569848262 shutdown_connections 2024-03-18T20:35:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.052+0000 7f2b51aa6700 1 --2- 172.21.15.46:0/569848262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c106ad0 0x7f2b4c108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.052+0000 7f2b51aa6700 1 --2- 172.21.15.46:0/569848262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b4c101870 0x7f2b4c103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.052+0000 7f2b51aa6700 1 --2- 172.21.15.46:0/569848262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b4c1041a0 0x7f2b4c106590 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.052+0000 7f2b51aa6700 1 -- 172.21.15.46:0/569848262 >> 172.21.15.46:0/569848262 conn(0x7f2b4c0fb200 msgr2=0x7f2b4c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.052+0000 7f2b51aa6700 1 -- 172.21.15.46:0/569848262 shutdown_connections 2024-03-18T20:35:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.052+0000 7f2b51aa6700 1 -- 172.21.15.46:0/569848262 wait complete. 2024-03-18T20:35:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.052+0000 7f2b51aa6700 1 Processor -- start 2024-03-18T20:35:54.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b51aa6700 1 -- start start 2024-03-18T20:35:54.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b51aa6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b4c101870 0x7f2b4c196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:54.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b51aa6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b4c1041a0 0x7f2b4c197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:54.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b51aa6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c106ad0 0x7f2b4c19e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b51aa6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c19e7b0 con 0x7f2b4c1041a0 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b51aa6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c19e920 con 0x7f2b4c101870 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b51aa6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c19eae0 con 0x7f2b4c106ad0 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b4affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b4c1041a0 0x7f2b4c197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b4affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b4c1041a0 0x7f2b4c197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34568/0 (socket says 172.21.15.46:34568) 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b4affd700 1 -- 172.21.15.46:0/3983978137 learned_addr learned my addr 172.21.15.46:0/3983978137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b4b7fe700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b4c101870 0x7f2b4c196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b4bfff700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c106ad0 0x7f2b4c19e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b4affd700 1 -- 172.21.15.46:0/3983978137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c106ad0 msgr2=0x7f2b4c19e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b4affd700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c106ad0 0x7f2b4c19e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b4affd700 1 -- 172.21.15.46:0/3983978137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b4c101870 msgr2=0x7f2b4c196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b4affd700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b4c101870 0x7f2b4c196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.053+0000 7f2b4affd700 1 -- 172.21.15.46:0/3983978137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b400095f0 con 0x7f2b4c1041a0 2024-03-18T20:35:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.054+0000 7f2b4affd700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b4c1041a0 0x7f2b4c197200 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2b400098e0 tx=0x7f2b40005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.054+0000 7f2b48ff9700 1 -- 172.21.15.46:0/3983978137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4001d070 con 0x7f2b4c1041a0 2024-03-18T20:35:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.054+0000 7f2b51aa6700 1 -- 172.21.15.46:0/3983978137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4c19ed60 con 0x7f2b4c1041a0 2024-03-18T20:35:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.054+0000 7f2b48ff9700 1 -- 172.21.15.46:0/3983978137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b40022470 con 0x7f2b4c1041a0 2024-03-18T20:35:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.054+0000 7f2b48ff9700 1 -- 172.21.15.46:0/3983978137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40017650 con 0x7f2b4c1041a0 2024-03-18T20:35:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.054+0000 7f2b51aa6700 1 -- 172.21.15.46:0/3983978137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4c19f190 con 0x7f2b4c1041a0 2024-03-18T20:35:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.055+0000 7f2b51aa6700 1 -- 172.21.15.46:0/3983978137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b4c0fcd80 con 0x7f2b4c1041a0 2024-03-18T20:35:54.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.056+0000 7f2b48ff9700 1 -- 172.21.15.46:0/3983978137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f2b40022850 con 0x7f2b4c1041a0 2024-03-18T20:35:54.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.057+0000 7f2b48ff9700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2b340a11b0 0x7f2b340a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:54.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.057+0000 7f2b4b7fe700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2b340a11b0 0x7f2b340a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:54.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.057+0000 7f2b48ff9700 1 -- 172.21.15.46:0/3983978137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2b400ce660 con 0x7f2b4c1041a0 2024-03-18T20:35:54.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.057+0000 7f2b4b7fe700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2b340a11b0 0x7f2b340a3670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c198140 tx=0x7f2b3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.060+0000 7f2b48ff9700 1 -- 172.21.15.46:0/3983978137 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2b40097de0 con 0x7f2b4c1041a0 2024-03-18T20:35:54.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.278+0000 7f2b51aa6700 1 -- 172.21.15.46:0/3983978137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2b4c02d060 con 0x7f2b4c1041a0 2024-03-18T20:35:54.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.278+0000 7f2b48ff9700 1 -- 172.21.15.46:0/3983978137 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2b4009b400 con 0x7f2b4c1041a0 2024-03-18T20:35:54.282 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-18T20:35:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.281+0000 7f2b51aa6700 1 -- 172.21.15.46:0/3983978137 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2b340a11b0 msgr2=0x7f2b340a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.281+0000 7f2b51aa6700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2b340a11b0 0x7f2b340a3670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c198140 tx=0x7f2b3c009260 comp rx=0 tx=0).stop 2024-03-18T20:35:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.281+0000 7f2b51aa6700 1 -- 172.21.15.46:0/3983978137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b4c1041a0 msgr2=0x7f2b4c197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.281+0000 7f2b51aa6700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b4c1041a0 0x7f2b4c197200 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2b400098e0 tx=0x7f2b40005720 comp rx=0 tx=0).stop 2024-03-18T20:35:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.281+0000 7f2b51aa6700 1 -- 172.21.15.46:0/3983978137 shutdown_connections 2024-03-18T20:35:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.281+0000 7f2b51aa6700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2b340a11b0 0x7f2b340a3670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.281+0000 7f2b51aa6700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b4c106ad0 0x7f2b4c19e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.281+0000 7f2b51aa6700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b4c101870 0x7f2b4c196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.281+0000 7f2b51aa6700 1 --2- 172.21.15.46:0/3983978137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b4c1041a0 0x7f2b4c197200 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.281+0000 7f2b51aa6700 1 -- 172.21.15.46:0/3983978137 >> 172.21.15.46:0/3983978137 conn(0x7f2b4c0fb200 msgr2=0x7f2b4c102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.281+0000 7f2b51aa6700 1 -- 172.21.15.46:0/3983978137 shutdown_connections 2024-03-18T20:35:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.281+0000 7f2b51aa6700 1 -- 172.21.15.46:0/3983978137 wait complete. 2024-03-18T20:35:54.298 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-18T20:35:54.298 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:35:54.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.432+0000 7f9f4cd66700 1 -- 172.21.15.46:0/870024641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f48102810 msgr2=0x7f9f481090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.432+0000 7f9f4cd66700 1 --2- 172.21.15.46:0/870024641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f48102810 0x7f9f481090c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000b0d0 tx=0x7f9f4000b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.432+0000 7f9f4cd66700 1 -- 172.21.15.46:0/870024641 shutdown_connections 2024-03-18T20:35:54.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.432+0000 7f9f4cd66700 1 --2- 172.21.15.46:0/870024641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f48102810 0x7f9f481090c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.432+0000 7f9f4cd66700 1 --2- 172.21.15.46:0/870024641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48100c80 0x7f9f481010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.432+0000 7f9f4cd66700 1 --2- 172.21.15.46:0/870024641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f48101e70 0x7f9f481022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.432+0000 7f9f4cd66700 1 -- 172.21.15.46:0/870024641 >> 172.21.15.46:0/870024641 conn(0x7f9f480fc1f0 msgr2=0x7f9f480fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:54.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.432+0000 7f9f4cd66700 1 -- 172.21.15.46:0/870024641 shutdown_connections 2024-03-18T20:35:54.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.432+0000 7f9f4cd66700 1 -- 172.21.15.46:0/870024641 wait complete. 2024-03-18T20:35:54.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.433+0000 7f9f4cd66700 1 Processor -- start 2024-03-18T20:35:54.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.433+0000 7f9f4cd66700 1 -- start start 2024-03-18T20:35:54.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.433+0000 7f9f4cd66700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48100c80 0x7f9f48196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:54.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f4cd66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f48101e70 0x7f9f48197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:54.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f4cd66700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f48102810 0x7f9f4819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:54.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f4cd66700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4819e7e0 con 0x7f9f48101e70 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f4cd66700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4819e950 con 0x7f9f48102810 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f4cd66700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4819eb10 con 0x7f9f48100c80 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f45d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f48101e70 0x7f9f48197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f45d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f48101e70 0x7f9f48197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34584/0 (socket says 172.21.15.46:34584) 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f45d9b700 1 -- 172.21.15.46:0/1437122841 learned_addr learned my addr 172.21.15.46:0/1437122841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f46d9d700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f48102810 0x7f9f4819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f4659c700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48100c80 0x7f9f48196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f45d9b700 1 -- 172.21.15.46:0/1437122841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48100c80 msgr2=0x7f9f48196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f45d9b700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48100c80 0x7f9f48196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f45d9b700 1 -- 172.21.15.46:0/1437122841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f48102810 msgr2=0x7f9f4819e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f45d9b700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f48102810 0x7f9f4819e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f45d9b700 1 -- 172.21.15.46:0/1437122841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f40009d00 con 0x7f9f48101e70 2024-03-18T20:35:54.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f45d9b700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f48101e70 0x7f9f48197200 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000d950 tx=0x7f9f3000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f3f7fe700 1 -- 172.21.15.46:0/1437122841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000f890 con 0x7f9f48101e70 2024-03-18T20:35:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f3f7fe700 1 -- 172.21.15.46:0/1437122841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f30010460 con 0x7f9f48101e70 2024-03-18T20:35:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f4cd66700 1 -- 172.21.15.46:0/1437122841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4819edf0 con 0x7f9f48101e70 2024-03-18T20:35:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.434+0000 7f9f3f7fe700 1 -- 172.21.15.46:0/1437122841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000e610 con 0x7f9f48101e70 2024-03-18T20:35:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.435+0000 7f9f4cd66700 1 -- 172.21.15.46:0/1437122841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4819f280 con 0x7f9f48101e70 2024-03-18T20:35:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.436+0000 7f9f4cd66700 1 -- 172.21.15.46:0/1437122841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f481080d0 con 0x7f9f48101e70 2024-03-18T20:35:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.437+0000 7f9f3f7fe700 1 -- 172.21.15.46:0/1437122841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f9f300108d0 con 0x7f9f48101e70 2024-03-18T20:35:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.437+0000 7f9f3f7fe700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9f340a1230 0x7f9f340a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.437+0000 7f9f4659c700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9f340a1230 0x7f9f340a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.437+0000 7f9f3f7fe700 1 -- 172.21.15.46:0/1437122841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9f300ccb00 con 0x7f9f48101e70 2024-03-18T20:35:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.438+0000 7f9f4659c700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9f340a1230 0x7f9f340a36f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9f480fe5f0 tx=0x7f9f38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.442+0000 7f9f3f7fe700 1 -- 172.21.15.46:0/1437122841 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9f30096140 con 0x7f9f48101e70 2024-03-18T20:35:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:54 smithi073 ceph-mon[47384]: pgmap v258: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s; 1 B/s, 0 objects/s recovering 2024-03-18T20:35:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:54 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2339780551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:54 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3484101613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:54 smithi155 ceph-mon[48344]: pgmap v258: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s; 1 B/s, 0 objects/s recovering 2024-03-18T20:35:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:54 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2339780551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:54.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:54 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3484101613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:54 smithi046 ceph-mon[43495]: pgmap v258: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s; 1 B/s, 0 objects/s recovering 2024-03-18T20:35:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:54 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2339780551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:54 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3484101613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:54.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.708+0000 7f9f4cd66700 1 -- 172.21.15.46:0/1437122841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f48067a10 con 0x7f9f48101e70 2024-03-18T20:35:54.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.709+0000 7f9f3f7fe700 1 -- 172.21.15.46:0/1437122841 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f9f30020090 con 0x7f9f48101e70 2024-03-18T20:35:54.712 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:54.713 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:35:51.152306+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:35:54.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.711+0000 7f9f4cd66700 1 -- 172.21.15.46:0/1437122841 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9f340a1230 msgr2=0x7f9f340a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.711+0000 7f9f4cd66700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9f340a1230 0x7f9f340a36f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9f480fe5f0 tx=0x7f9f38009260 comp rx=0 tx=0).stop 2024-03-18T20:35:54.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.711+0000 7f9f4cd66700 1 -- 172.21.15.46:0/1437122841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f48101e70 msgr2=0x7f9f48197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.711+0000 7f9f4cd66700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f48101e70 0x7f9f48197200 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000d950 tx=0x7f9f3000dc60 comp rx=0 tx=0).stop 2024-03-18T20:35:54.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.711+0000 7f9f4cd66700 1 -- 172.21.15.46:0/1437122841 shutdown_connections 2024-03-18T20:35:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.711+0000 7f9f4cd66700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f9f340a1230 0x7f9f340a36f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.711+0000 7f9f4cd66700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f48102810 0x7f9f4819e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.711+0000 7f9f4cd66700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f48100c80 0x7f9f48196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.711+0000 7f9f4cd66700 1 --2- 172.21.15.46:0/1437122841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f48101e70 0x7f9f48197200 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.711+0000 7f9f4cd66700 1 -- 172.21.15.46:0/1437122841 >> 172.21.15.46:0/1437122841 conn(0x7f9f480fc1f0 msgr2=0x7f9f48105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.712+0000 7f9f4cd66700 1 -- 172.21.15.46:0/1437122841 shutdown_connections 2024-03-18T20:35:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.712+0000 7f9f4cd66700 1 -- 172.21.15.46:0/1437122841 wait complete. 2024-03-18T20:35:54.716 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-03-18T20:35:54.729 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-18T20:35:54.729 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-18T20:35:54.729 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T20:35:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.864+0000 7fd948086700 1 -- 172.21.15.46:0/3578591299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd940104090 msgr2=0x7fd940106480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.864+0000 7fd948086700 1 --2- 172.21.15.46:0/3578591299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd940104090 0x7fd940106480 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd930009990 tx=0x7fd930009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.864+0000 7fd948086700 1 -- 172.21.15.46:0/3578591299 shutdown_connections 2024-03-18T20:35:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.864+0000 7fd948086700 1 --2- 172.21.15.46:0/3578591299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9401069c0 0x7fd940108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.864+0000 7fd948086700 1 --2- 172.21.15.46:0/3578591299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd940101760 0x7fd940103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.864+0000 7fd948086700 1 --2- 172.21.15.46:0/3578591299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd940104090 0x7fd940106480 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.864+0000 7fd948086700 1 -- 172.21.15.46:0/3578591299 >> 172.21.15.46:0/3578591299 conn(0x7fd9400fb0d0 msgr2=0x7fd9400fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.864+0000 7fd948086700 1 -- 172.21.15.46:0/3578591299 shutdown_connections 2024-03-18T20:35:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.864+0000 7fd948086700 1 -- 172.21.15.46:0/3578591299 wait complete. 2024-03-18T20:35:54.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd948086700 1 Processor -- start 2024-03-18T20:35:54.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd948086700 1 -- start start 2024-03-18T20:35:54.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd948086700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd940101760 0x7fd940196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:54.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd948086700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd940104090 0x7fd940197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:54.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd948086700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9401069c0 0x7fd94019e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:54.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd948086700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94019e700 con 0x7fd940104090 2024-03-18T20:35:54.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd948086700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94019e840 con 0x7fd9401069c0 2024-03-18T20:35:54.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd948086700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94019e9d0 con 0x7fd940101760 2024-03-18T20:35:54.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd945621700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd940104090 0x7fd940197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:54.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd945621700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd940104090 0x7fd940197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34594/0 (socket says 172.21.15.46:34594) 2024-03-18T20:35:54.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd945621700 1 -- 172.21.15.46:0/469056658 learned_addr learned my addr 172.21.15.46:0/469056658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:54.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd946623700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9401069c0 0x7fd94019e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:54.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.865+0000 7fd945e22700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd940101760 0x7fd940196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:54.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.866+0000 7fd945621700 1 -- 172.21.15.46:0/469056658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd940101760 msgr2=0x7fd940196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.866+0000 7fd945621700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd940101760 0x7fd940196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.866+0000 7fd945621700 1 -- 172.21.15.46:0/469056658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9401069c0 msgr2=0x7fd94019e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:54.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.866+0000 7fd945621700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9401069c0 0x7fd94019e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:54.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.866+0000 7fd945621700 1 -- 172.21.15.46:0/469056658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9300095f0 con 0x7fd940104090 2024-03-18T20:35:54.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.866+0000 7fd945621700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd940104090 0x7fd940197110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd9300095c0 tx=0x7fd93000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:54.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.866+0000 7fd936ffd700 1 -- 172.21.15.46:0/469056658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93001d070 con 0x7fd940104090 2024-03-18T20:35:54.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.866+0000 7fd948086700 1 -- 172.21.15.46:0/469056658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94019ec50 con 0x7fd940104090 2024-03-18T20:35:54.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.866+0000 7fd936ffd700 1 -- 172.21.15.46:0/469056658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd930022470 con 0x7fd940104090 2024-03-18T20:35:54.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.866+0000 7fd936ffd700 1 -- 172.21.15.46:0/469056658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930017650 con 0x7fd940104090 2024-03-18T20:35:54.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.866+0000 7fd948086700 1 -- 172.21.15.46:0/469056658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94019f0d0 con 0x7fd940104090 2024-03-18T20:35:54.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.867+0000 7fd948086700 1 -- 172.21.15.46:0/469056658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9400fcc50 con 0x7fd940104090 2024-03-18T20:35:54.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.868+0000 7fd936ffd700 1 -- 172.21.15.46:0/469056658 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fd930022890 con 0x7fd940104090 2024-03-18T20:35:54.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.869+0000 7fd936ffd700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fd92c0a11b0 0x7fd92c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:54.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.870+0000 7fd945e22700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fd92c0a11b0 0x7fd92c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:54.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.870+0000 7fd936ffd700 1 -- 172.21.15.46:0/469056658 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd9300ce1f0 con 0x7fd940104090 2024-03-18T20:35:54.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.870+0000 7fd945e22700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fd92c0a11b0 0x7fd92c0a3670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00bbd0 tx=0x7fd93c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:54.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:54.873+0000 7fd936ffd700 1 -- 172.21.15.46:0/469056658 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd9300978c0 con 0x7fd940104090 2024-03-18T20:35:55.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.090+0000 7fd948086700 1 -- 172.21.15.46:0/469056658 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd940067a10 con 0x7fd92c0a11b0 2024-03-18T20:35:55.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.101+0000 7fd936ffd700 1 -- 172.21.15.46:0/469056658 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344356 (secure 0 0 0) 0x7fd940067a10 con 0x7fd92c0a11b0 2024-03-18T20:35:55.109 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.105+0000 7fd948086700 1 -- 172.21.15.46:0/469056658 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fd92c0a11b0 msgr2=0x7fd92c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.105+0000 7fd948086700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fd92c0a11b0 0x7fd92c0a3670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00bbd0 tx=0x7fd93c00b5b0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.105+0000 7fd948086700 1 -- 172.21.15.46:0/469056658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd940104090 msgr2=0x7fd940197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.105+0000 7fd948086700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd940104090 0x7fd940197110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd9300095c0 tx=0x7fd93000f690 comp rx=0 tx=0).stop 2024-03-18T20:35:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.105+0000 7fd948086700 1 -- 172.21.15.46:0/469056658 shutdown_connections 2024-03-18T20:35:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.105+0000 7fd948086700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fd92c0a11b0 0x7fd92c0a3670 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.105+0000 7fd948086700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9401069c0 0x7fd94019e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.105+0000 7fd948086700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd940101760 0x7fd940196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.105+0000 7fd948086700 1 --2- 172.21.15.46:0/469056658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd940104090 0x7fd940197110 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.105+0000 7fd948086700 1 -- 172.21.15.46:0/469056658 >> 172.21.15.46:0/469056658 conn(0x7fd9400fb0d0 msgr2=0x7fd9401022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.106+0000 7fd948086700 1 -- 172.21.15.46:0/469056658 shutdown_connections 2024-03-18T20:35:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.106+0000 7fd948086700 1 -- 172.21.15.46:0/469056658 wait complete. 2024-03-18T20:35:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-18T20:35:55.123 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-03-18T20:35:55.037736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657270,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":852,"num_read_kb":550,"num_write":416,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336680,"kb_used_data":12432,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605144,"statfs":{"total":1151940427776,"available":1151595667456,"internally_reserved":0,"allocated":12730368,"data_stored":7030577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":628,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":52,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003020"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116835+0000","last_change":"2024-03-18T20:34:38.817028+0000","last_active":"2024-03-18T20:35:37.116835+0000","last_peered":"2024-03-18T20:35:37.116835+0000","last_clean":"2024-03-18T20:35:37.116835+0000","last_became_active":"2024-03-18T20:34:38.816711+0000","last_became_peered":"2024-03-18T20:34:38.816711+0000","last_unstale":"2024-03-18T20:35:37.116835+0000","last_undegraded":"2024-03-18T20:35:37.116835+0000","last_fullsized":"2024-03-18T20:35:37.116835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:32:16.827141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117600+0000","last_change":"2024-03-18T20:34:38.807350+0000","last_active":"2024-03-18T20:35:37.117600+0000","last_peered":"2024-03-18T20:35:37.117600+0000","last_clean":"2024-03-18T20:35:37.117600+0000","last_became_active":"2024-03-18T20:34:38.807008+0000","last_became_peered":"2024-03-18T20:34:38.807008+0000","last_unstale":"2024-03-18T20:35:37.117600+0000","last_undegraded":"2024-03-18T20:35:37.117600+0000","last_fullsized":"2024-03-18T20:35:37.117600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:35.667384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117502+0000","last_change":"2024-03-18T20:34:38.817045+0000","last_active":"2024-03-18T20:35:37.117502+0000","last_peered":"2024-03-18T20:35:37.117502+0000","last_clean":"2024-03-18T20:35:37.117502+0000","last_became_active":"2024-03-18T20:34:38.816659+0000","last_became_peered":"2024-03-18T20:34:38.816659+0000","last_unstale":"2024-03-18T20:35:37.117502+0000","last_undegraded":"2024-03-18T20:35:37.117502+0000","last_fullsized":"2024-03-18T20:35:37.117502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:04:49.316127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116820+0000","last_change":"2024-03-18T20:34:38.804833+0000","last_active":"2024-03-18T20:35:37.116820+0000","last_peered":"2024-03-18T20:35:37.116820+0000","last_clean":"2024-03-18T20:35:37.116820+0000","last_became_active":"2024-03-18T20:34:38.804486+0000","last_became_peered":"2024-03-18T20:34:38.804486+0000","last_unstale":"2024-03-18T20:35:37.116820+0000","last_undegraded":"2024-03-18T20:35:37.116820+0000","last_fullsized":"2024-03-18T20:35:37.116820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:19:44.266207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110860+0000","last_change":"2024-03-18T20:34:38.797660+0000","last_active":"2024-03-18T20:35:37.110860+0000","last_peered":"2024-03-18T20:35:37.110860+0000","last_clean":"2024-03-18T20:35:37.110860+0000","last_became_active":"2024-03-18T20:34:38.797155+0000","last_became_peered":"2024-03-18T20:34:38.797155+0000","last_unstale":"2024-03-18T20:35:37.110860+0000","last_undegraded":"2024-03-18T20:35:37.110860+0000","last_fullsized":"2024-03-18T20:35:37.110860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:22:11.363453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.118992+0000","last_change":"2024-03-18T20:35:38.118992+0000","last_active":"2024-03-18T20:35:38.118992+0000","last_peered":"2024-03-18T20:35:38.118992+0000","last_clean":"2024-03-18T20:35:38.118992+0000","last_became_active":"2024-03-18T20:35:38.118632+0000","last_became_peered":"2024-03-18T20:35:38.118632+0000","last_unstale":"2024-03-18T20:35:38.118992+0000","last_undegraded":"2024-03-18T20:35:38.118992+0000","last_fullsized":"2024-03-18T20:35:38.118992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:00:49.799078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115609+0000","last_change":"2024-03-18T20:34:38.805378+0000","last_active":"2024-03-18T20:35:37.115609+0000","last_peered":"2024-03-18T20:35:37.115609+0000","last_clean":"2024-03-18T20:35:37.115609+0000","last_became_active":"2024-03-18T20:34:38.805130+0000","last_became_peered":"2024-03-18T20:34:38.805130+0000","last_unstale":"2024-03-18T20:35:37.115609+0000","last_undegraded":"2024-03-18T20:35:37.115609+0000","last_fullsized":"2024-03-18T20:35:37.115609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:46:19.420715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.118345+0000","last_change":"2024-03-18T20:35:38.117964+0000","last_active":"2024-03-18T20:35:38.118345+0000","last_peered":"2024-03-18T20:35:38.118345+0000","last_clean":"2024-03-18T20:35:38.118345+0000","last_became_active":"2024-03-18T20:35:38.117579+0000","last_became_peered":"2024-03-18T20:35:38.117579+0000","last_unstale":"2024-03-18T20:35:38.118345+0000","last_undegraded":"2024-03-18T20:35:38.118345+0000","last_fullsized":"2024-03-18T20:35:38.118345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:44:20.292995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.234734+0000","last_change":"2024-03-18T20:35:38.234734+0000","last_active":"2024-03-18T20:35:38.234734+0000","last_peered":"2024-03-18T20:35:38.234734+0000","last_clean":"2024-03-18T20:35:38.234734+0000","last_became_active":"2024-03-18T20:35:38.234403+0000","last_became_peered":"2024-03-18T20:35:38.234403+0000","last_unstale":"2024-03-18T20:35:38.234734+0000","last_undegraded":"2024-03-18T20:35:38.234734+0000","last_fullsized":"2024-03-18T20:35:38.234734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:13:54.696645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119706+0000","last_change":"2024-03-18T20:34:38.814728+0000","last_active":"2024-03-18T20:35:37.119706+0000","last_peered":"2024-03-18T20:35:37.119706+0000","last_clean":"2024-03-18T20:35:37.119706+0000","last_became_active":"2024-03-18T20:34:38.814200+0000","last_became_peered":"2024-03-18T20:34:38.814200+0000","last_unstale":"2024-03-18T20:35:37.119706+0000","last_undegraded":"2024-03-18T20:35:37.119706+0000","last_fullsized":"2024-03-18T20:35:37.119706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:24:54.064770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115901+0000","last_change":"2024-03-18T20:34:38.818054+0000","last_active":"2024-03-18T20:35:37.115901+0000","last_peered":"2024-03-18T20:35:37.115901+0000","last_clean":"2024-03-18T20:35:37.115901+0000","last_became_active":"2024-03-18T20:34:38.817752+0000","last_became_peered":"2024-03-18T20:34:38.817752+0000","last_unstale":"2024-03-18T20:35:37.115901+0000","last_undegraded":"2024-03-18T20:35:37.115901+0000","last_fullsized":"2024-03-18T20:35:37.115901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:01:13.728434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116729+0000","last_change":"2024-03-18T20:34:38.801356+0000","last_active":"2024-03-18T20:35:37.116729+0000","last_peered":"2024-03-18T20:35:37.116729+0000","last_clean":"2024-03-18T20:35:37.116729+0000","last_became_active":"2024-03-18T20:34:38.800978+0000","last_became_peered":"2024-03-18T20:34:38.800978+0000","last_unstale":"2024-03-18T20:35:37.116729+0000","last_undegraded":"2024-03-18T20:35:37.116729+0000","last_fullsized":"2024-03-18T20:35:37.116729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:07.634686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110534+0000","last_change":"2024-03-18T20:34:38.803006+0000","last_active":"2024-03-18T20:35:37.110534+0000","last_peered":"2024-03-18T20:35:37.110534+0000","last_clean":"2024-03-18T20:35:37.110534+0000","last_became_active":"2024-03-18T20:34:38.802454+0000","last_became_peered":"2024-03-18T20:34:38.802454+0000","last_unstale":"2024-03-18T20:35:37.110534+0000","last_undegraded":"2024-03-18T20:35:37.110534+0000","last_fullsized":"2024-03-18T20:35:37.110534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:11:02.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116047+0000","last_change":"2024-03-18T20:34:38.807397+0000","last_active":"2024-03-18T20:35:37.116047+0000","last_peered":"2024-03-18T20:35:37.116047+0000","last_clean":"2024-03-18T20:35:37.116047+0000","last_became_active":"2024-03-18T20:34:38.807072+0000","last_became_peered":"2024-03-18T20:34:38.807072+0000","last_unstale":"2024-03-18T20:35:37.116047+0000","last_undegraded":"2024-03-18T20:35:37.116047+0000","last_fullsized":"2024-03-18T20:35:37.116047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:55:53.832825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116047+0000","last_change":"2024-03-18T20:34:38.812925+0000","last_active":"2024-03-18T20:35:37.116047+0000","last_peered":"2024-03-18T20:35:37.116047+0000","last_clean":"2024-03-18T20:35:37.116047+0000","last_became_active":"2024-03-18T20:34:38.812597+0000","last_became_peered":"2024-03-18T20:34:38.812597+0000","last_unstale":"2024-03-18T20:35:37.116047+0000","last_undegraded":"2024-03-18T20:35:37.116047+0000","last_fullsized":"2024-03-18T20:35:37.116047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:14:35.103852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110532+0000","last_change":"2024-03-18T20:34:38.817892+0000","last_active":"2024-03-18T20:35:37.110532+0000","last_peered":"2024-03-18T20:35:37.110532+0000","last_clean":"2024-03-18T20:35:37.110532+0000","last_became_active":"2024-03-18T20:34:38.816261+0000","last_became_peered":"2024-03-18T20:34:38.816261+0000","last_unstale":"2024-03-18T20:35:37.110532+0000","last_undegraded":"2024-03-18T20:35:37.110532+0000","last_fullsized":"2024-03-18T20:35:37.110532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:32.481210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116723+0000","last_change":"2024-03-18T20:34:38.816966+0000","last_active":"2024-03-18T20:35:37.116723+0000","last_peered":"2024-03-18T20:35:37.116723+0000","last_clean":"2024-03-18T20:35:37.116723+0000","last_became_active":"2024-03-18T20:34:38.816645+0000","last_became_peered":"2024-03-18T20:34:38.816645+0000","last_unstale":"2024-03-18T20:35:37.116723+0000","last_undegraded":"2024-03-18T20:35:37.116723+0000","last_fullsized":"2024-03-18T20:35:37.116723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:39.904353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.118561+0000","last_change":"2024-03-18T20:34:38.817005+0000","last_active":"2024-03-18T20:35:37.118561+0000","last_peered":"2024-03-18T20:35:37.118561+0000","last_clean":"2024-03-18T20:35:37.118561+0000","last_became_active":"2024-03-18T20:34:38.816710+0000","last_became_peered":"2024-03-18T20:34:38.816710+0000","last_unstale":"2024-03-18T20:35:37.118561+0000","last_undegraded":"2024-03-18T20:35:37.118561+0000","last_fullsized":"2024-03-18T20:35:37.118561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:30:43.483977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117763+0000","last_change":"2024-03-18T20:34:38.810957+0000","last_active":"2024-03-18T20:35:37.117763+0000","last_peered":"2024-03-18T20:35:37.117763+0000","last_clean":"2024-03-18T20:35:37.117763+0000","last_became_active":"2024-03-18T20:34:38.810615+0000","last_became_peered":"2024-03-18T20:34:38.810615+0000","last_unstale":"2024-03-18T20:35:37.117763+0000","last_undegraded":"2024-03-18T20:35:37.117763+0000","last_fullsized":"2024-03-18T20:35:37.117763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:26:36.544034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":94,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:49.909225+0000","last_change":"2024-03-18T20:34:38.337863+0000","last_active":"2024-03-18T20:35:49.909225+0000","last_peered":"2024-03-18T20:35:49.909225+0000","last_clean":"2024-03-18T20:35:49.909225+0000","last_became_active":"2024-03-18T20:34:38.337548+0000","last_became_peered":"2024-03-18T20:34:38.337548+0000","last_unstale":"2024-03-18T20:35:49.909225+0000","last_undegraded":"2024-03-18T20:35:49.909225+0000","last_fullsized":"2024-03-18T20:35:49.909225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:41:40.237882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120594+0000","last_change":"2024-03-18T20:34:38.814321+0000","last_active":"2024-03-18T20:35:37.120594+0000","last_peered":"2024-03-18T20:35:37.120594+0000","last_clean":"2024-03-18T20:35:37.120594+0000","last_became_active":"2024-03-18T20:34:38.813939+0000","last_became_peered":"2024-03-18T20:34:38.813939+0000","last_unstale":"2024-03-18T20:35:37.120594+0000","last_undegraded":"2024-03-18T20:35:37.120594+0000","last_fullsized":"2024-03-18T20:35:37.120594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:05:42.441099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120794+0000","last_change":"2024-03-18T20:34:38.347790+0000","last_active":"2024-03-18T20:35:37.120794+0000","last_peered":"2024-03-18T20:35:37.120794+0000","last_clean":"2024-03-18T20:35:37.120794+0000","last_became_active":"2024-03-18T20:34:38.347317+0000","last_became_peered":"2024-03-18T20:34:38.347317+0000","last_unstale":"2024-03-18T20:35:37.120794+0000","last_undegraded":"2024-03-18T20:35:37.120794+0000","last_fullsized":"2024-03-18T20:35:37.120794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.096480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.118730+0000","last_change":"2024-03-18T20:34:38.817116+0000","last_active":"2024-03-18T20:35:37.118730+0000","last_peered":"2024-03-18T20:35:37.118730+0000","last_clean":"2024-03-18T20:35:37.118730+0000","last_became_active":"2024-03-18T20:34:38.816761+0000","last_became_peered":"2024-03-18T20:34:38.816761+0000","last_unstale":"2024-03-18T20:35:37.118730+0000","last_undegraded":"2024-03-18T20:35:37.118730+0000","last_fullsized":"2024-03-18T20:35:37.118730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:44.353533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111161+0000","last_change":"2024-03-18T20:34:38.339236+0000","last_active":"2024-03-18T20:35:37.111161+0000","last_peered":"2024-03-18T20:35:37.111161+0000","last_clean":"2024-03-18T20:35:37.111161+0000","last_became_active":"2024-03-18T20:34:38.338879+0000","last_became_peered":"2024-03-18T20:34:38.338879+0000","last_unstale":"2024-03-18T20:35:37.111161+0000","last_undegraded":"2024-03-18T20:35:37.111161+0000","last_fullsized":"2024-03-18T20:35:37.111161+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:33.340151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111163+0000","last_change":"2024-03-18T20:34:38.817861+0000","last_active":"2024-03-18T20:35:37.111163+0000","last_peered":"2024-03-18T20:35:37.111163+0000","last_clean":"2024-03-18T20:35:37.111163+0000","last_became_active":"2024-03-18T20:34:38.815976+0000","last_became_peered":"2024-03-18T20:34:38.815976+0000","last_unstale":"2024-03-18T20:35:37.111163+0000","last_undegraded":"2024-03-18T20:35:37.111163+0000","last_fullsized":"2024-03-18T20:35:37.111163+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:54.070067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111718+0000","last_change":"2024-03-18T20:34:38.345758+0000","last_active":"2024-03-18T20:35:37.111718+0000","last_peered":"2024-03-18T20:35:37.111718+0000","last_clean":"2024-03-18T20:35:37.111718+0000","last_became_active":"2024-03-18T20:34:38.345417+0000","last_became_peered":"2024-03-18T20:34:38.345417+0000","last_unstale":"2024-03-18T20:35:37.111718+0000","last_undegraded":"2024-03-18T20:35:37.111718+0000","last_fullsized":"2024-03-18T20:35:37.111718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:25.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113736+0000","last_change":"2024-03-18T20:34:38.818745+0000","last_active":"2024-03-18T20:35:37.113736+0000","last_peered":"2024-03-18T20:35:37.113736+0000","last_clean":"2024-03-18T20:35:37.113736+0000","last_became_active":"2024-03-18T20:34:38.818415+0000","last_became_peered":"2024-03-18T20:34:38.818415+0000","last_unstale":"2024-03-18T20:35:37.113736+0000","last_undegraded":"2024-03-18T20:35:37.113736+0000","last_fullsized":"2024-03-18T20:35:37.113736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:45:49.454498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":93,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:52.893926+0000","last_change":"2024-03-18T20:34:38.345804+0000","last_active":"2024-03-18T20:35:52.893926+0000","last_peered":"2024-03-18T20:35:52.893926+0000","last_clean":"2024-03-18T20:35:52.893926+0000","last_became_active":"2024-03-18T20:34:38.345409+0000","last_became_peered":"2024-03-18T20:34:38.345409+0000","last_unstale":"2024-03-18T20:35:52.893926+0000","last_undegraded":"2024-03-18T20:35:52.893926+0000","last_fullsized":"2024-03-18T20:35:52.893926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:00.085412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":888,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111822+0000","last_change":"2024-03-18T20:34:38.799821+0000","last_active":"2024-03-18T20:35:37.111822+0000","last_peered":"2024-03-18T20:35:37.111822+0000","last_clean":"2024-03-18T20:35:37.111822+0000","last_became_active":"2024-03-18T20:34:38.799457+0000","last_became_peered":"2024-03-18T20:34:38.799457+0000","last_unstale":"2024-03-18T20:35:37.111822+0000","last_undegraded":"2024-03-18T20:35:37.111822+0000","last_fullsized":"2024-03-18T20:35:37.111822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:39:35.912892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120102+0000","last_change":"2024-03-18T20:34:38.341999+0000","last_active":"2024-03-18T20:35:37.120102+0000","last_peered":"2024-03-18T20:35:37.120102+0000","last_clean":"2024-03-18T20:35:37.120102+0000","last_became_active":"2024-03-18T20:34:38.341431+0000","last_became_peered":"2024-03-18T20:34:38.341431+0000","last_unstale":"2024-03-18T20:35:37.120102+0000","last_undegraded":"2024-03-18T20:35:37.120102+0000","last_fullsized":"2024-03-18T20:35:37.120102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:07:26.711113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.114037+0000","last_change":"2024-03-18T20:34:38.804003+0000","last_active":"2024-03-18T20:35:37.114037+0000","last_peered":"2024-03-18T20:35:37.114037+0000","last_clean":"2024-03-18T20:35:37.114037+0000","last_became_active":"2024-03-18T20:34:38.803667+0000","last_became_peered":"2024-03-18T20:34:38.803667+0000","last_unstale":"2024-03-18T20:35:37.114037+0000","last_undegraded":"2024-03-18T20:35:37.114037+0000","last_fullsized":"2024-03-18T20:35:37.114037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:19:58.230790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110346+0000","last_change":"2024-03-18T20:34:38.346493+0000","last_active":"2024-03-18T20:35:37.110346+0000","last_peered":"2024-03-18T20:35:37.110346+0000","last_clean":"2024-03-18T20:35:37.110346+0000","last_became_active":"2024-03-18T20:34:38.345925+0000","last_became_peered":"2024-03-18T20:34:38.345925+0000","last_unstale":"2024-03-18T20:35:37.110346+0000","last_undegraded":"2024-03-18T20:35:37.110346+0000","last_fullsized":"2024-03-18T20:35:37.110346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:52:47.773860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119099+0000","last_change":"2024-03-18T20:34:38.807284+0000","last_active":"2024-03-18T20:35:37.119099+0000","last_peered":"2024-03-18T20:35:37.119099+0000","last_clean":"2024-03-18T20:35:37.119099+0000","last_became_active":"2024-03-18T20:34:38.806844+0000","last_became_peered":"2024-03-18T20:34:38.806844+0000","last_unstale":"2024-03-18T20:35:37.119099+0000","last_undegraded":"2024-03-18T20:35:37.119099+0000","last_fullsized":"2024-03-18T20:35:37.119099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:44:35.469953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.114113+0000","last_change":"2024-03-18T20:34:38.343090+0000","last_active":"2024-03-18T20:35:37.114113+0000","last_peered":"2024-03-18T20:35:37.114113+0000","last_clean":"2024-03-18T20:35:37.114113+0000","last_became_active":"2024-03-18T20:34:38.342712+0000","last_became_peered":"2024-03-18T20:34:38.342712+0000","last_unstale":"2024-03-18T20:35:37.114113+0000","last_undegraded":"2024-03-18T20:35:37.114113+0000","last_fullsized":"2024-03-18T20:35:37.114113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:14:05.909604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116701+0000","last_change":"2024-03-18T20:34:38.800616+0000","last_active":"2024-03-18T20:35:37.116701+0000","last_peered":"2024-03-18T20:35:37.116701+0000","last_clean":"2024-03-18T20:35:37.116701+0000","last_became_active":"2024-03-18T20:34:38.800340+0000","last_became_peered":"2024-03-18T20:34:38.800340+0000","last_unstale":"2024-03-18T20:35:37.116701+0000","last_undegraded":"2024-03-18T20:35:37.116701+0000","last_fullsized":"2024-03-18T20:35:37.116701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:18.263179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'5","reported_seq":155,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:51.891731+0000","last_change":"2024-03-18T20:34:38.345634+0000","last_active":"2024-03-18T20:35:51.891731+0000","last_peered":"2024-03-18T20:35:51.891731+0000","last_clean":"2024-03-18T20:35:51.891731+0000","last_became_active":"2024-03-18T20:34:38.345275+0000","last_became_peered":"2024-03-18T20:34:38.345275+0000","last_unstale":"2024-03-18T20:35:51.891731+0000","last_undegraded":"2024-03-18T20:35:51.891731+0000","last_fullsized":"2024-03-18T20:35:51.891731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:29.939529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110859+0000","last_change":"2024-03-18T20:34:38.820286+0000","last_active":"2024-03-18T20:35:37.110859+0000","last_peered":"2024-03-18T20:35:37.110859+0000","last_clean":"2024-03-18T20:35:37.110859+0000","last_became_active":"2024-03-18T20:34:38.819939+0000","last_became_peered":"2024-03-18T20:34:38.819939+0000","last_unstale":"2024-03-18T20:35:37.110859+0000","last_undegraded":"2024-03-18T20:35:37.110859+0000","last_fullsized":"2024-03-18T20:35:37.110859+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:19.229339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116083+0000","last_change":"2024-03-18T20:34:38.341699+0000","last_active":"2024-03-18T20:35:37.116083+0000","last_peered":"2024-03-18T20:35:37.116083+0000","last_clean":"2024-03-18T20:35:37.116083+0000","last_became_active":"2024-03-18T20:34:38.341390+0000","last_became_peered":"2024-03-18T20:34:38.341390+0000","last_unstale":"2024-03-18T20:35:37.116083+0000","last_undegraded":"2024-03-18T20:35:37.116083+0000","last_fullsized":"2024-03-18T20:35:37.116083+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:16:02.239414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.117141+0000","last_change":"2024-03-18T20:35:38.117141+0000","last_active":"2024-03-18T20:35:38.117141+0000","last_peered":"2024-03-18T20:35:38.117141+0000","last_clean":"2024-03-18T20:35:38.117141+0000","last_became_active":"2024-03-18T20:35:38.116809+0000","last_became_peered":"2024-03-18T20:35:38.116809+0000","last_unstale":"2024-03-18T20:35:38.117141+0000","last_undegraded":"2024-03-18T20:35:38.117141+0000","last_fullsized":"2024-03-18T20:35:38.117141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:10:22.545159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113855+0000","last_change":"2024-03-18T20:34:38.346821+0000","last_active":"2024-03-18T20:35:37.113855+0000","last_peered":"2024-03-18T20:35:37.113855+0000","last_clean":"2024-03-18T20:35:37.113855+0000","last_became_active":"2024-03-18T20:34:38.346314+0000","last_became_peered":"2024-03-18T20:34:38.346314+0000","last_unstale":"2024-03-18T20:35:37.113855+0000","last_undegraded":"2024-03-18T20:35:37.113855+0000","last_fullsized":"2024-03-18T20:35:37.113855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:12:04.552971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111241+0000","last_change":"2024-03-18T20:34:38.820479+0000","last_active":"2024-03-18T20:35:37.111241+0000","last_peered":"2024-03-18T20:35:37.111241+0000","last_clean":"2024-03-18T20:35:37.111241+0000","last_became_active":"2024-03-18T20:34:38.820186+0000","last_became_peered":"2024-03-18T20:34:38.820186+0000","last_unstale":"2024-03-18T20:35:37.111241+0000","last_undegraded":"2024-03-18T20:35:37.111241+0000","last_fullsized":"2024-03-18T20:35:37.111241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:15.953764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":188,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:51.891872+0000","last_change":"2024-03-18T20:34:38.344332+0000","last_active":"2024-03-18T20:35:51.891872+0000","last_peered":"2024-03-18T20:35:51.891872+0000","last_clean":"2024-03-18T20:35:51.891872+0000","last_became_active":"2024-03-18T20:34:38.343827+0000","last_became_peered":"2024-03-18T20:34:38.343827+0000","last_unstale":"2024-03-18T20:35:51.891872+0000","last_undegraded":"2024-03-18T20:35:51.891872+0000","last_fullsized":"2024-03-18T20:35:51.891872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:13:21.725004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59732,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":143,"num_read_kb":130,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111038+0000","last_change":"2024-03-18T20:34:38.814794+0000","last_active":"2024-03-18T20:35:37.111038+0000","last_peered":"2024-03-18T20:35:37.111038+0000","last_clean":"2024-03-18T20:35:37.111038+0000","last_became_active":"2024-03-18T20:34:38.814434+0000","last_became_peered":"2024-03-18T20:34:38.814434+0000","last_unstale":"2024-03-18T20:35:37.111038+0000","last_undegraded":"2024-03-18T20:35:37.111038+0000","last_fullsized":"2024-03-18T20:35:37.111038+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:43:19.083212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117588+0000","last_change":"2024-03-18T20:34:38.343894+0000","last_active":"2024-03-18T20:35:37.117588+0000","last_peered":"2024-03-18T20:35:37.117588+0000","last_clean":"2024-03-18T20:35:37.117588+0000","last_became_active":"2024-03-18T20:34:38.343493+0000","last_became_peered":"2024-03-18T20:34:38.343493+0000","last_unstale":"2024-03-18T20:35:37.117588+0000","last_undegraded":"2024-03-18T20:35:37.117588+0000","last_fullsized":"2024-03-18T20:35:37.117588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:13:32.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110231+0000","last_change":"2024-03-18T20:34:38.809968+0000","last_active":"2024-03-18T20:35:37.110231+0000","last_peered":"2024-03-18T20:35:37.110231+0000","last_clean":"2024-03-18T20:35:37.110231+0000","last_became_active":"2024-03-18T20:34:38.809641+0000","last_became_peered":"2024-03-18T20:34:38.809641+0000","last_unstale":"2024-03-18T20:35:37.110231+0000","last_undegraded":"2024-03-18T20:35:37.110231+0000","last_fullsized":"2024-03-18T20:35:37.110231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:50:52.925683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113815+0000","last_change":"2024-03-18T20:34:38.345621+0000","last_active":"2024-03-18T20:35:37.113815+0000","last_peered":"2024-03-18T20:35:37.113815+0000","last_clean":"2024-03-18T20:35:37.113815+0000","last_became_active":"2024-03-18T20:34:38.345255+0000","last_became_peered":"2024-03-18T20:34:38.345255+0000","last_unstale":"2024-03-18T20:35:37.113815+0000","last_undegraded":"2024-03-18T20:35:37.113815+0000","last_fullsized":"2024-03-18T20:35:37.113815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:17:03.737973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115794+0000","last_change":"2024-03-18T20:34:38.804390+0000","last_active":"2024-03-18T20:35:37.115794+0000","last_peered":"2024-03-18T20:35:37.115794+0000","last_clean":"2024-03-18T20:35:37.115794+0000","last_became_active":"2024-03-18T20:34:38.804033+0000","last_became_peered":"2024-03-18T20:34:38.804033+0000","last_unstale":"2024-03-18T20:35:37.115794+0000","last_undegraded":"2024-03-18T20:35:37.115794+0000","last_fullsized":"2024-03-18T20:35:37.115794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:09:45.407921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":91,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:51.891203+0000","last_change":"2024-03-18T20:34:38.340239+0000","last_active":"2024-03-18T20:35:51.891203+0000","last_peered":"2024-03-18T20:35:51.891203+0000","last_clean":"2024-03-18T20:35:51.891203+0000","last_became_active":"2024-03-18T20:34:38.339880+0000","last_became_peered":"2024-03-18T20:34:38.339880+0000","last_unstale":"2024-03-18T20:35:51.891203+0000","last_undegraded":"2024-03-18T20:35:51.891203+0000","last_fullsized":"2024-03-18T20:35:51.891203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:55:46.864885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115572+0000","last_change":"2024-03-18T20:34:38.819223+0000","last_active":"2024-03-18T20:35:37.115572+0000","last_peered":"2024-03-18T20:35:37.115572+0000","last_clean":"2024-03-18T20:35:37.115572+0000","last_became_active":"2024-03-18T20:34:38.818879+0000","last_became_peered":"2024-03-18T20:34:38.818879+0000","last_unstale":"2024-03-18T20:35:37.115572+0000","last_undegraded":"2024-03-18T20:35:37.115572+0000","last_fullsized":"2024-03-18T20:35:37.115572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:45.645912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110850+0000","last_change":"2024-03-18T20:34:38.336817+0000","last_active":"2024-03-18T20:35:37.110850+0000","last_peered":"2024-03-18T20:35:37.110850+0000","last_clean":"2024-03-18T20:35:37.110850+0000","last_became_active":"2024-03-18T20:34:38.336453+0000","last_became_peered":"2024-03-18T20:34:38.336453+0000","last_unstale":"2024-03-18T20:35:37.110850+0000","last_undegraded":"2024-03-18T20:35:37.110850+0000","last_fullsized":"2024-03-18T20:35:37.110850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:10.253435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113661+0000","last_change":"2024-03-18T20:34:38.818048+0000","last_active":"2024-03-18T20:35:37.113661+0000","last_peered":"2024-03-18T20:35:37.113661+0000","last_clean":"2024-03-18T20:35:37.113661+0000","last_became_active":"2024-03-18T20:34:38.817682+0000","last_became_peered":"2024-03-18T20:34:38.817682+0000","last_unstale":"2024-03-18T20:35:37.113661+0000","last_undegraded":"2024-03-18T20:35:37.113661+0000","last_fullsized":"2024-03-18T20:35:37.113661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:25:55.291815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.126651+0000","last_change":"2024-03-18T20:35:38.126271+0000","last_active":"2024-03-18T20:35:38.126651+0000","last_peered":"2024-03-18T20:35:38.126651+0000","last_clean":"2024-03-18T20:35:38.126651+0000","last_became_active":"2024-03-18T20:35:38.119078+0000","last_became_peered":"2024-03-18T20:35:38.119078+0000","last_unstale":"2024-03-18T20:35:38.126651+0000","last_undegraded":"2024-03-18T20:35:38.126651+0000","last_fullsized":"2024-03-18T20:35:38.126651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:46:20.914329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116043+0000","last_change":"2024-03-18T20:34:38.806138+0000","last_active":"2024-03-18T20:35:37.116043+0000","last_peered":"2024-03-18T20:35:37.116043+0000","last_clean":"2024-03-18T20:35:37.116043+0000","last_became_active":"2024-03-18T20:34:38.805821+0000","last_became_peered":"2024-03-18T20:34:38.805821+0000","last_unstale":"2024-03-18T20:35:37.116043+0000","last_undegraded":"2024-03-18T20:35:37.116043+0000","last_fullsized":"2024-03-18T20:35:37.116043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:25:57.887813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.118124+0000","last_change":"2024-03-18T20:34:38.338751+0000","last_active":"2024-03-18T20:35:37.118124+0000","last_peered":"2024-03-18T20:35:37.118124+0000","last_clean":"2024-03-18T20:35:37.118124+0000","last_became_active":"2024-03-18T20:34:38.337495+0000","last_became_peered":"2024-03-18T20:34:38.337495+0000","last_unstale":"2024-03-18T20:35:37.118124+0000","last_undegraded":"2024-03-18T20:35:37.118124+0000","last_fullsized":"2024-03-18T20:35:37.118124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:38:04.402380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119498+0000","last_change":"2024-03-18T20:34:38.803937+0000","last_active":"2024-03-18T20:35:37.119498+0000","last_peered":"2024-03-18T20:35:37.119498+0000","last_clean":"2024-03-18T20:35:37.119498+0000","last_became_active":"2024-03-18T20:34:38.803520+0000","last_became_peered":"2024-03-18T20:34:38.803520+0000","last_unstale":"2024-03-18T20:35:37.119498+0000","last_undegraded":"2024-03-18T20:35:37.119498+0000","last_fullsized":"2024-03-18T20:35:37.119498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:20:37.537265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":85,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:48.908311+0000","last_change":"2024-03-18T20:34:38.337481+0000","last_active":"2024-03-18T20:35:48.908311+0000","last_peered":"2024-03-18T20:35:48.908311+0000","last_clean":"2024-03-18T20:35:48.908311+0000","last_became_active":"2024-03-18T20:34:38.337152+0000","last_became_peered":"2024-03-18T20:34:38.337152+0000","last_unstale":"2024-03-18T20:35:48.908311+0000","last_undegraded":"2024-03-18T20:35:48.908311+0000","last_fullsized":"2024-03-18T20:35:48.908311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:07:59.593978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.119360+0000","last_change":"2024-03-18T20:35:38.119360+0000","last_active":"2024-03-18T20:35:38.119360+0000","last_peered":"2024-03-18T20:35:38.119360+0000","last_clean":"2024-03-18T20:35:38.119360+0000","last_became_active":"2024-03-18T20:35:38.118942+0000","last_became_peered":"2024-03-18T20:35:38.118942+0000","last_unstale":"2024-03-18T20:35:38.119360+0000","last_undegraded":"2024-03-18T20:35:38.119360+0000","last_fullsized":"2024-03-18T20:35:38.119360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:42.140321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.114312+0000","last_change":"2024-03-18T20:34:38.806133+0000","last_active":"2024-03-18T20:35:37.114312+0000","last_peered":"2024-03-18T20:35:37.114312+0000","last_clean":"2024-03-18T20:35:37.114312+0000","last_became_active":"2024-03-18T20:34:38.805257+0000","last_became_peered":"2024-03-18T20:34:38.805257+0000","last_unstale":"2024-03-18T20:35:37.114312+0000","last_undegraded":"2024-03-18T20:35:37.114312+0000","last_fullsized":"2024-03-18T20:35:37.114312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:05:15.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":88,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:50.889127+0000","last_change":"2024-03-18T20:34:38.346271+0000","last_active":"2024-03-18T20:35:50.889127+0000","last_peered":"2024-03-18T20:35:50.889127+0000","last_clean":"2024-03-18T20:35:50.889127+0000","last_became_active":"2024-03-18T20:34:38.345054+0000","last_became_peered":"2024-03-18T20:34:38.345054+0000","last_unstale":"2024-03-18T20:35:50.889127+0000","last_undegraded":"2024-03-18T20:35:50.889127+0000","last_fullsized":"2024-03-18T20:35:50.889127+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:42:52.510294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119964+0000","last_change":"2024-03-18T20:34:38.816834+0000","last_active":"2024-03-18T20:35:37.119964+0000","last_peered":"2024-03-18T20:35:37.119964+0000","last_clean":"2024-03-18T20:35:37.119964+0000","last_became_active":"2024-03-18T20:34:38.816516+0000","last_became_peered":"2024-03-18T20:34:38.816516+0000","last_unstale":"2024-03-18T20:35:37.119964+0000","last_undegraded":"2024-03-18T20:35:37.119964+0000","last_fullsized":"2024-03-18T20:35:37.119964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:33:54.529206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117428+0000","last_change":"2024-03-18T20:34:38.339407+0000","last_active":"2024-03-18T20:35:37.117428+0000","last_peered":"2024-03-18T20:35:37.117428+0000","last_clean":"2024-03-18T20:35:37.117428+0000","last_became_active":"2024-03-18T20:34:38.339086+0000","last_became_peered":"2024-03-18T20:34:38.339086+0000","last_unstale":"2024-03-18T20:35:37.117428+0000","last_undegraded":"2024-03-18T20:35:37.117428+0000","last_fullsized":"2024-03-18T20:35:37.117428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:18.479630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":333,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115643+0000","last_change":"2024-03-18T20:33:50.777651+0000","last_active":"2024-03-18T20:35:37.115643+0000","last_peered":"2024-03-18T20:35:37.115643+0000","last_clean":"2024-03-18T20:35:37.115643+0000","last_became_active":"2024-03-18T20:33:50.777281+0000","last_became_peered":"2024-03-18T20:33:50.777281+0000","last_unstale":"2024-03-18T20:35:37.115643+0000","last_undegraded":"2024-03-18T20:35:37.115643+0000","last_fullsized":"2024-03-18T20:35:37.115643+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_clean_scrub_stamp":"2024-03-18T20:31:27.488744+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:28.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110103+0000","last_change":"2024-03-18T20:34:38.812714+0000","last_active":"2024-03-18T20:35:37.110103+0000","last_peered":"2024-03-18T20:35:37.110103+0000","last_clean":"2024-03-18T20:35:37.110103+0000","last_became_active":"2024-03-18T20:34:38.812464+0000","last_became_peered":"2024-03-18T20:34:38.812464+0000","last_unstale":"2024-03-18T20:35:37.110103+0000","last_undegraded":"2024-03-18T20:35:37.110103+0000","last_fullsized":"2024-03-18T20:35:37.110103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:03:13.853740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115734+0000","last_change":"2024-03-18T20:34:38.343173+0000","last_active":"2024-03-18T20:35:37.115734+0000","last_peered":"2024-03-18T20:35:37.115734+0000","last_clean":"2024-03-18T20:35:37.115734+0000","last_became_active":"2024-03-18T20:34:38.342774+0000","last_became_peered":"2024-03-18T20:34:38.342774+0000","last_unstale":"2024-03-18T20:35:37.115734+0000","last_undegraded":"2024-03-18T20:35:37.115734+0000","last_fullsized":"2024-03-18T20:35:37.115734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:52.283033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111085+0000","last_change":"2024-03-18T20:34:38.344539+0000","last_active":"2024-03-18T20:35:37.111085+0000","last_peered":"2024-03-18T20:35:37.111085+0000","last_clean":"2024-03-18T20:35:37.111085+0000","last_became_active":"2024-03-18T20:34:38.344188+0000","last_became_peered":"2024-03-18T20:34:38.344188+0000","last_unstale":"2024-03-18T20:35:37.111085+0000","last_undegraded":"2024-03-18T20:35:37.111085+0000","last_fullsized":"2024-03-18T20:35:37.111085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:38.641928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110493+0000","last_change":"2024-03-18T20:34:38.806602+0000","last_active":"2024-03-18T20:35:37.110493+0000","last_peered":"2024-03-18T20:35:37.110493+0000","last_clean":"2024-03-18T20:35:37.110493+0000","last_became_active":"2024-03-18T20:34:38.806241+0000","last_became_peered":"2024-03-18T20:34:38.806241+0000","last_unstale":"2024-03-18T20:35:37.110493+0000","last_undegraded":"2024-03-18T20:35:37.110493+0000","last_fullsized":"2024-03-18T20:35:37.110493+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:04.463311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'9","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.118511+0000","last_change":"2024-03-18T20:34:38.339695+0000","last_active":"2024-03-18T20:35:37.118511+0000","last_peered":"2024-03-18T20:35:37.118511+0000","last_clean":"2024-03-18T20:35:37.118511+0000","last_became_active":"2024-03-18T20:34:38.339331+0000","last_became_peered":"2024-03-18T20:34:38.339331+0000","last_unstale":"2024-03-18T20:35:37.118511+0000","last_undegraded":"2024-03-18T20:35:37.118511+0000","last_fullsized":"2024-03-18T20:35:37.118511+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:57:06.434289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120777+0000","last_change":"2024-03-18T20:34:38.809754+0000","last_active":"2024-03-18T20:35:37.120777+0000","last_peered":"2024-03-18T20:35:37.120777+0000","last_clean":"2024-03-18T20:35:37.120777+0000","last_became_active":"2024-03-18T20:34:38.809415+0000","last_became_peered":"2024-03-18T20:34:38.809415+0000","last_unstale":"2024-03-18T20:35:37.120777+0000","last_undegraded":"2024-03-18T20:35:37.120777+0000","last_fullsized":"2024-03-18T20:35:37.120777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:34:30.761582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117134+0000","last_change":"2024-03-18T20:34:38.342080+0000","last_active":"2024-03-18T20:35:37.117134+0000","last_peered":"2024-03-18T20:35:37.117134+0000","last_clean":"2024-03-18T20:35:37.117134+0000","last_became_active":"2024-03-18T20:34:38.340623+0000","last_became_peered":"2024-03-18T20:34:38.340623+0000","last_unstale":"2024-03-18T20:35:37.117134+0000","last_undegraded":"2024-03-18T20:35:37.117134+0000","last_fullsized":"2024-03-18T20:35:37.117134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:49:14.912391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117644+0000","last_change":"2024-03-18T20:34:38.806558+0000","last_active":"2024-03-18T20:35:37.117644+0000","last_peered":"2024-03-18T20:35:37.117644+0000","last_clean":"2024-03-18T20:35:37.117644+0000","last_became_active":"2024-03-18T20:34:38.806171+0000","last_became_peered":"2024-03-18T20:34:38.806171+0000","last_unstale":"2024-03-18T20:35:37.117644+0000","last_undegraded":"2024-03-18T20:35:37.117644+0000","last_fullsized":"2024-03-18T20:35:37.117644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:46:26.530157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110601+0000","last_change":"2024-03-18T20:34:38.336175+0000","last_active":"2024-03-18T20:35:37.110601+0000","last_peered":"2024-03-18T20:35:37.110601+0000","last_clean":"2024-03-18T20:35:37.110601+0000","last_became_active":"2024-03-18T20:34:38.335798+0000","last_became_peered":"2024-03-18T20:34:38.335798+0000","last_unstale":"2024-03-18T20:35:37.110601+0000","last_undegraded":"2024-03-18T20:35:37.110601+0000","last_fullsized":"2024-03-18T20:35:37.110601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:33:33.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110847+0000","last_change":"2024-03-18T20:34:38.807684+0000","last_active":"2024-03-18T20:35:37.110847+0000","last_peered":"2024-03-18T20:35:37.110847+0000","last_clean":"2024-03-18T20:35:37.110847+0000","last_became_active":"2024-03-18T20:34:38.807300+0000","last_became_peered":"2024-03-18T20:34:38.807300+0000","last_unstale":"2024-03-18T20:35:37.110847+0000","last_undegraded":"2024-03-18T20:35:37.110847+0000","last_fullsized":"2024-03-18T20:35:37.110847+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:31:43.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120021+0000","last_change":"2024-03-18T20:34:38.341802+0000","last_active":"2024-03-18T20:35:37.120021+0000","last_peered":"2024-03-18T20:35:37.120021+0000","last_clean":"2024-03-18T20:35:37.120021+0000","last_became_active":"2024-03-18T20:34:38.340594+0000","last_became_peered":"2024-03-18T20:34:38.340594+0000","last_unstale":"2024-03-18T20:35:37.120021+0000","last_undegraded":"2024-03-18T20:35:37.120021+0000","last_fullsized":"2024-03-18T20:35:37.120021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:01:06.830323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116079+0000","last_change":"2024-03-18T20:34:38.809790+0000","last_active":"2024-03-18T20:35:37.116079+0000","last_peered":"2024-03-18T20:35:37.116079+0000","last_clean":"2024-03-18T20:35:37.116079+0000","last_became_active":"2024-03-18T20:34:38.809470+0000","last_became_peered":"2024-03-18T20:34:38.809470+0000","last_unstale":"2024-03-18T20:35:37.116079+0000","last_undegraded":"2024-03-18T20:35:37.116079+0000","last_fullsized":"2024-03-18T20:35:37.116079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:48.225759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.123225+0000","last_change":"2024-03-18T20:35:38.122793+0000","last_active":"2024-03-18T20:35:38.123225+0000","last_peered":"2024-03-18T20:35:38.123225+0000","last_clean":"2024-03-18T20:35:38.123225+0000","last_became_active":"2024-03-18T20:35:38.118680+0000","last_became_peered":"2024-03-18T20:35:38.118680+0000","last_unstale":"2024-03-18T20:35:38.123225+0000","last_undegraded":"2024-03-18T20:35:38.123225+0000","last_fullsized":"2024-03-18T20:35:38.123225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:04:06.361764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119251+0000","last_change":"2024-03-18T20:34:38.809294+0000","last_active":"2024-03-18T20:35:37.119251+0000","last_peered":"2024-03-18T20:35:37.119251+0000","last_clean":"2024-03-18T20:35:37.119251+0000","last_became_active":"2024-03-18T20:34:38.808542+0000","last_became_peered":"2024-03-18T20:34:38.808542+0000","last_unstale":"2024-03-18T20:35:37.119251+0000","last_undegraded":"2024-03-18T20:35:37.119251+0000","last_fullsized":"2024-03-18T20:35:37.119251+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:11:30.669317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110948+0000","last_change":"2024-03-18T20:34:38.339454+0000","last_active":"2024-03-18T20:35:37.110948+0000","last_peered":"2024-03-18T20:35:37.110948+0000","last_clean":"2024-03-18T20:35:37.110948+0000","last_became_active":"2024-03-18T20:34:38.339154+0000","last_became_peered":"2024-03-18T20:34:38.339154+0000","last_unstale":"2024-03-18T20:35:37.110948+0000","last_undegraded":"2024-03-18T20:35:37.110948+0000","last_fullsized":"2024-03-18T20:35:37.110948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:12:06.300600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116002+0000","last_change":"2024-03-18T20:34:38.818462+0000","last_active":"2024-03-18T20:35:37.116002+0000","last_peered":"2024-03-18T20:35:37.116002+0000","last_clean":"2024-03-18T20:35:37.116002+0000","last_became_active":"2024-03-18T20:34:38.817522+0000","last_became_peered":"2024-03-18T20:34:38.817522+0000","last_unstale":"2024-03-18T20:35:37.116002+0000","last_undegraded":"2024-03-18T20:35:37.116002+0000","last_fullsized":"2024-03-18T20:35:37.116002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:50:32.258583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'5","reported_seq":89,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:50.890976+0000","last_change":"2024-03-18T20:34:38.341054+0000","last_active":"2024-03-18T20:35:50.890976+0000","last_peered":"2024-03-18T20:35:50.890976+0000","last_clean":"2024-03-18T20:35:50.890976+0000","last_became_active":"2024-03-18T20:34:38.340352+0000","last_became_peered":"2024-03-18T20:34:38.340352+0000","last_unstale":"2024-03-18T20:35:50.890976+0000","last_undegraded":"2024-03-18T20:35:50.890976+0000","last_fullsized":"2024-03-18T20:35:50.890976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:44.850244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115977+0000","last_change":"2024-03-18T20:34:38.818176+0000","last_active":"2024-03-18T20:35:37.115977+0000","last_peered":"2024-03-18T20:35:37.115977+0000","last_clean":"2024-03-18T20:35:37.115977+0000","last_became_active":"2024-03-18T20:34:38.817820+0000","last_became_peered":"2024-03-18T20:34:38.817820+0000","last_unstale":"2024-03-18T20:35:37.115977+0000","last_undegraded":"2024-03-18T20:35:37.115977+0000","last_fullsized":"2024-03-18T20:35:37.115977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:19:29.815174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113678+0000","last_change":"2024-03-18T20:34:38.346591+0000","last_active":"2024-03-18T20:35:37.113678+0000","last_peered":"2024-03-18T20:35:37.113678+0000","last_clean":"2024-03-18T20:35:37.113678+0000","last_became_active":"2024-03-18T20:34:38.345897+0000","last_became_peered":"2024-03-18T20:34:38.345897+0000","last_unstale":"2024-03-18T20:35:37.113678+0000","last_undegraded":"2024-03-18T20:35:37.113678+0000","last_fullsized":"2024-03-18T20:35:37.113678+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:47:32.118905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.122016+0000","last_change":"2024-03-18T20:35:38.122016+0000","last_active":"2024-03-18T20:35:38.122016+0000","last_peered":"2024-03-18T20:35:38.122016+0000","last_clean":"2024-03-18T20:35:38.122016+0000","last_became_active":"2024-03-18T20:35:38.118053+0000","last_became_peered":"2024-03-18T20:35:38.118053+0000","last_unstale":"2024-03-18T20:35:38.122016+0000","last_undegraded":"2024-03-18T20:35:38.122016+0000","last_fullsized":"2024-03-18T20:35:38.122016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:25:28.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110701+0000","last_change":"2024-03-18T20:34:38.344615+0000","last_active":"2024-03-18T20:35:37.110701+0000","last_peered":"2024-03-18T20:35:37.110701+0000","last_clean":"2024-03-18T20:35:37.110701+0000","last_became_active":"2024-03-18T20:34:38.344242+0000","last_became_peered":"2024-03-18T20:34:38.344242+0000","last_unstale":"2024-03-18T20:35:37.110701+0000","last_undegraded":"2024-03-18T20:35:37.110701+0000","last_fullsized":"2024-03-18T20:35:37.110701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.449808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116268+0000","last_change":"2024-03-18T20:34:38.820875+0000","last_active":"2024-03-18T20:35:37.116268+0000","last_peered":"2024-03-18T20:35:37.116268+0000","last_clean":"2024-03-18T20:35:37.116268+0000","last_became_active":"2024-03-18T20:34:38.820550+0000","last_became_peered":"2024-03-18T20:34:38.820550+0000","last_unstale":"2024-03-18T20:35:37.116268+0000","last_undegraded":"2024-03-18T20:35:37.116268+0000","last_fullsized":"2024-03-18T20:35:37.116268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:49:36.358734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116141+0000","last_change":"2024-03-18T20:34:38.812552+0000","last_active":"2024-03-18T20:35:37.116141+0000","last_peered":"2024-03-18T20:35:37.116141+0000","last_clean":"2024-03-18T20:35:37.116141+0000","last_became_active":"2024-03-18T20:34:38.811939+0000","last_became_peered":"2024-03-18T20:34:38.811939+0000","last_unstale":"2024-03-18T20:35:37.116141+0000","last_undegraded":"2024-03-18T20:35:37.116141+0000","last_fullsized":"2024-03-18T20:35:37.116141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:41:52.096974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117526+0000","last_change":"2024-03-18T20:34:38.815691+0000","last_active":"2024-03-18T20:35:37.117526+0000","last_peered":"2024-03-18T20:35:37.117526+0000","last_clean":"2024-03-18T20:35:37.117526+0000","last_became_active":"2024-03-18T20:34:38.815370+0000","last_became_peered":"2024-03-18T20:34:38.815370+0000","last_unstale":"2024-03-18T20:35:37.117526+0000","last_undegraded":"2024-03-18T20:35:37.117526+0000","last_fullsized":"2024-03-18T20:35:37.117526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:59:41.260013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117452+0000","last_change":"2024-03-18T20:34:38.812651+0000","last_active":"2024-03-18T20:35:37.117452+0000","last_peered":"2024-03-18T20:35:37.117452+0000","last_clean":"2024-03-18T20:35:37.117452+0000","last_became_active":"2024-03-18T20:34:38.812322+0000","last_became_peered":"2024-03-18T20:34:38.812322+0000","last_unstale":"2024-03-18T20:35:37.117452+0000","last_undegraded":"2024-03-18T20:35:37.117452+0000","last_fullsized":"2024-03-18T20:35:37.117452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:34:55.658371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117275+0000","last_change":"2024-03-18T20:34:38.806475+0000","last_active":"2024-03-18T20:35:37.117275+0000","last_peered":"2024-03-18T20:35:37.117275+0000","last_clean":"2024-03-18T20:35:37.117275+0000","last_became_active":"2024-03-18T20:34:38.806027+0000","last_became_peered":"2024-03-18T20:34:38.806027+0000","last_unstale":"2024-03-18T20:35:37.117275+0000","last_undegraded":"2024-03-18T20:35:37.117275+0000","last_fullsized":"2024-03-18T20:35:37.117275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:57:17.140031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110573+0000","last_change":"2024-03-18T20:34:38.811057+0000","last_active":"2024-03-18T20:35:37.110573+0000","last_peered":"2024-03-18T20:35:37.110573+0000","last_clean":"2024-03-18T20:35:37.110573+0000","last_became_active":"2024-03-18T20:34:38.810316+0000","last_became_peered":"2024-03-18T20:34:38.810316+0000","last_unstale":"2024-03-18T20:35:37.110573+0000","last_undegraded":"2024-03-18T20:35:37.110573+0000","last_fullsized":"2024-03-18T20:35:37.110573+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:29:22.426717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116579+0000","last_change":"2024-03-18T20:34:38.812952+0000","last_active":"2024-03-18T20:35:37.116579+0000","last_peered":"2024-03-18T20:35:37.116579+0000","last_clean":"2024-03-18T20:35:37.116579+0000","last_became_active":"2024-03-18T20:34:38.812550+0000","last_became_peered":"2024-03-18T20:34:38.812550+0000","last_unstale":"2024-03-18T20:35:37.116579+0000","last_undegraded":"2024-03-18T20:35:37.116579+0000","last_fullsized":"2024-03-18T20:35:37.116579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:06:26.525078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117914+0000","last_change":"2024-03-18T20:34:38.817358+0000","last_active":"2024-03-18T20:35:37.117914+0000","last_peered":"2024-03-18T20:35:37.117914+0000","last_clean":"2024-03-18T20:35:37.117914+0000","last_became_active":"2024-03-18T20:34:38.816939+0000","last_became_peered":"2024-03-18T20:34:38.816939+0000","last_unstale":"2024-03-18T20:35:37.117914+0000","last_undegraded":"2024-03-18T20:35:37.117914+0000","last_fullsized":"2024-03-18T20:35:37.117914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:33.258162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120687+0000","last_change":"2024-03-18T20:34:38.807798+0000","last_active":"2024-03-18T20:35:37.120687+0000","last_peered":"2024-03-18T20:35:37.120687+0000","last_clean":"2024-03-18T20:35:37.120687+0000","last_became_active":"2024-03-18T20:34:38.807489+0000","last_became_peered":"2024-03-18T20:34:38.807489+0000","last_unstale":"2024-03-18T20:35:37.120687+0000","last_undegraded":"2024-03-18T20:35:37.120687+0000","last_fullsized":"2024-03-18T20:35:37.120687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:15.395763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.114222+0000","last_change":"2024-03-18T20:34:38.806363+0000","last_active":"2024-03-18T20:35:37.114222+0000","last_peered":"2024-03-18T20:35:37.114222+0000","last_clean":"2024-03-18T20:35:37.114222+0000","last_became_active":"2024-03-18T20:34:38.805707+0000","last_became_peered":"2024-03-18T20:34:38.805707+0000","last_unstale":"2024-03-18T20:35:37.114222+0000","last_undegraded":"2024-03-18T20:35:37.114222+0000","last_fullsized":"2024-03-18T20:35:37.114222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:10.918979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119927+0000","last_change":"2024-03-18T20:34:38.814439+0000","last_active":"2024-03-18T20:35:37.119927+0000","last_peered":"2024-03-18T20:35:37.119927+0000","last_clean":"2024-03-18T20:35:37.119927+0000","last_became_active":"2024-03-18T20:34:38.813988+0000","last_became_peered":"2024-03-18T20:34:38.813988+0000","last_unstale":"2024-03-18T20:35:37.119927+0000","last_undegraded":"2024-03-18T20:35:37.119927+0000","last_fullsized":"2024-03-18T20:35:37.119927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:03:14.639044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110698+0000","last_change":"2024-03-18T20:34:38.818550+0000","last_active":"2024-03-18T20:35:37.110698+0000","last_peered":"2024-03-18T20:35:37.110698+0000","last_clean":"2024-03-18T20:35:37.110698+0000","last_became_active":"2024-03-18T20:34:38.818209+0000","last_became_peered":"2024-03-18T20:34:38.818209+0000","last_unstale":"2024-03-18T20:35:37.110698+0000","last_undegraded":"2024-03-18T20:35:37.110698+0000","last_fullsized":"2024-03-18T20:35:37.110698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:40:48.395331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110529+0000","last_change":"2024-03-18T20:34:38.808439+0000","last_active":"2024-03-18T20:35:37.110529+0000","last_peered":"2024-03-18T20:35:37.110529+0000","last_clean":"2024-03-18T20:35:37.110529+0000","last_became_active":"2024-03-18T20:34:38.807995+0000","last_became_peered":"2024-03-18T20:34:38.807995+0000","last_unstale":"2024-03-18T20:35:37.110529+0000","last_undegraded":"2024-03-18T20:35:37.110529+0000","last_fullsized":"2024-03-18T20:35:37.110529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:54.414148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116358+0000","last_change":"2024-03-18T20:34:38.810793+0000","last_active":"2024-03-18T20:35:37.116358+0000","last_peered":"2024-03-18T20:35:37.116358+0000","last_clean":"2024-03-18T20:35:37.116358+0000","last_became_active":"2024-03-18T20:34:38.810496+0000","last_became_peered":"2024-03-18T20:34:38.810496+0000","last_unstale":"2024-03-18T20:35:37.116358+0000","last_undegraded":"2024-03-18T20:35:37.116358+0000","last_fullsized":"2024-03-18T20:35:37.116358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:33.780675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66668,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":716,"num_read_kb":431,"num_write":233,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":204318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612632,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":422867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.379},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67100000000000004}]},{"osd":2,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":4,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.95799999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.61299999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59899999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.0309999999999999}]},{"osd":8,"last update":"Mon Mar 18 20:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.47999999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.76100000000000001}]},{"osd":10,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.56999999999999995}]}]},{"osd":10,"up_from":66,"seq":283467841564,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1572864,"data_stored":1073667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.4699999999999998,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":6.6449999999999996,"5min":6.6449999999999996,"15min":6.6449999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.376}]},{"osd":2,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56999999999999995}]},{"osd":3,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.50600000000000001}]},{"osd":4,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57899999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.66000000000000003}]},{"osd":6,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.67400000000000004}]},{"osd":9,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.69799999999999995}]},{"osd":11,"last update":"Mon Mar 18 20:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.624},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.70799999999999996}]}]},{"osd":9,"up_from":61,"seq":261993005088,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":422940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55200000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57999999999999996,"15min":0.59199999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.878,"15min":0.878},"last":0.435}]},{"osd":1,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77000000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.67500000000000004,"15min":0.68400000000000005},"min":{"1min":0.443,"5min":0.442,"15min":0.442},"max":{"1min":0.88800000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.74399999999999999}]},{"osd":2,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.46100000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83099999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.46600000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.59999999999999998}]},{"osd":3,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.51600000000000001,"15min":0.52400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69299999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.34699999999999998}]},{"osd":4,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64700000000000002,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81100000000000005,"5min":1.016,"15min":1.016},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.73799999999999999,"15min":0.751},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93300000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.69399999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83199999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86299999999999999,"5min":1.008,"15min":1.008},"last":0.76500000000000001}]},{"osd":6,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73599999999999999,"5min":1.179,"15min":1.179},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.70599999999999996,"15min":0.73099999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77100000000000002,"5min":1.151,"15min":1.151},"last":0.46999999999999997}]},{"osd":7,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.80800000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0049999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.73599999999999999,"15min":0.748},"min":{"1min":0.51200000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.85799999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.65500000000000003}]},{"osd":8,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.80700000000000005,"15min":0.81699999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96899999999999997,"5min":1.353,"15min":1.353},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.78700000000000003,"15min":0.79900000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.90300000000000002,"5min":1.103,"15min":1.103},"last":0.61899999999999999}]},{"osd":10,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.49199999999999999}]}]},{"osd":8,"up_from":54,"seq":231928234020,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":423179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.38500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.622,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47099999999999997}]},{"osd":1,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73399999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42299999999999999,"15min":0.41399999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.32600000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.38600000000000001}]},{"osd":3,"last update":"Mon Mar 18 20:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.622}]},{"osd":4,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.749,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.51100000000000001,"15min":0.498},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.495},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.34699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40600000000000003}]},{"osd":6,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.66200000000000003,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.75600000000000001}]},{"osd":7,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.874,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66000000000000003}]},{"osd":9,"last update":"Mon Mar 18 20:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.51700000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.70899999999999996,"15min":0.69399999999999995},"min":{"1min":0.63900000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79900000000000004}]},{"osd":10,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.65900000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.72499999999999998,"15min":0.70699999999999996},"min":{"1min":0.76500000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95299999999999996,"5min":1.018,"15min":1.018},"last":0.82299999999999995}]},{"osd":11,"last update":"Mon Mar 18 20:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.78500000000000003}]}]},{"osd":7,"up_from":47,"seq":201863462950,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1495040,"data_stored":1015576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.439}]},{"osd":1,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.42199999999999999,"15min":0.41399999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.378}]},{"osd":2,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73799999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.65900000000000003}]},{"osd":3,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.72999999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.44500000000000001}]},{"osd":5,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.40500000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.497},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.61899999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89700000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60199999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.71299999999999997}]},{"osd":9,"last update":"Mon Mar 18 20:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.59299999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94099999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.61299999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.72799999999999998}]},{"osd":10,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.69599999999999995}]},{"osd":11,"last update":"Mon Mar 18 20:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.748}]}]},{"osd":6,"up_from":42,"seq":180388626474,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":422492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.437,"15min":0.42699999999999999},"min":{"1min":0.35399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.44900000000000001,"15min":0.44500000000000001},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.625,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.441}]},{"osd":1,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75800000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.505,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59799999999999998}]},{"osd":2,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.46800000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63}]},{"osd":3,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.874,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.54800000000000004}]},{"osd":4,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60599999999999998,"15min":0.58999999999999997},"min":{"1min":0.498,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77200000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.45600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.56399999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.54500000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95999999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.94699999999999995,"5min":1.052,"15min":1.052},"last":0.58099999999999996}]},{"osd":7,"last update":"Mon Mar 18 20:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.89000000000000001,"5min":0.995,"15min":0.995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58699999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.97199999999999998,"5min":1.006,"15min":1.006},"last":0.81399999999999995}]},{"osd":8,"last update":"Mon Mar 18 20:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.47399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92000000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76600000000000001,"15min":0.76500000000000001},"min":{"1min":0.60999999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.77000000000000002}]},{"osd":9,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.81799999999999995,"15min":0.83199999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.83499999999999996,"5min":1.028,"15min":1.028},"last":0.61899999999999999}]},{"osd":10,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84699999999999998,"15min":0.85299999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.89700000000000002,"5min":1.081,"15min":1.081},"last":0.80000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":483137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.59299999999999997,"15min":0.54800000000000004},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.46400000000000002}]},{"osd":2,"last update":"Mon Mar 18 20:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.56999999999999995,"15min":0.54400000000000004},"min":{"1min":0.497,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.38600000000000001}]},{"osd":3,"last update":"Mon Mar 18 20:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.497,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.497}]},{"osd":4,"last update":"Mon Mar 18 20:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.443,"15min":0.40999999999999998},"min":{"1min":0.34499999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.70099999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.45000000000000001,"15min":0.42099999999999999},"min":{"1min":0.32400000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.58399999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.64400000000000002,"15min":0.623},"min":{"1min":0.52700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.86099999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.67900000000000005,"15min":0.623},"min":{"1min":0.52200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":4.5679999999999996,"5min":4.5679999999999996,"15min":4.5679999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.64700000000000002,"15min":0.60299999999999998},"min":{"1min":0.44,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.5330000000000004,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.54100000000000004}]},{"osd":7,"last update":"Mon Mar 18 20:34:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84399999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42699999999999999}]},{"osd":8,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63600000000000001,"15min":0.627},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.55300000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98899999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.629}]},{"osd":9,"last update":"Mon Mar 18 20:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.745,"15min":0.71899999999999997},"min":{"1min":0.69099999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70199999999999996,"15min":0.68600000000000005},"min":{"1min":0.61199999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.65600000000000003}]},{"osd":10,"last update":"Mon Mar 18 20:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.58199999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.70599999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.70799999999999996}]},{"osd":11,"last update":"Mon Mar 18 20:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73199999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":423831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Mar 18 20:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53500000000000003,"15min":0.51100000000000001},"min":{"1min":0.34599999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53600000000000003,"15min":0.52300000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79700000000000004,"5min":1.024,"15min":1.024},"last":0.51300000000000001}]},{"osd":2,"last update":"Mon Mar 18 20:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57799999999999996,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63200000000000001,"15min":0.63100000000000001},"min":{"1min":0.42799999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1100000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.46400000000000002}]},{"osd":3,"last update":"Mon Mar 18 20:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63800000000000001}]},{"osd":4,"last update":"Mon Mar 18 20:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.629},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89900000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57399999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.48499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.623,"15min":0.60599999999999998},"min":{"1min":0.47599999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85099999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.34699999999999998}]},{"osd":6,"last update":"Mon Mar 18 20:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.311},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.55100000000000005}]},{"osd":7,"last update":"Mon Mar 18 20:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.65100000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67400000000000004,"15min":0.67700000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.92200000000000004,"5min":1.002,"15min":1.002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.745,"15min":0.747},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93400000000000005,"5min":0.998,"15min":0.998},"last":0.58799999999999997}]},{"osd":9,"last update":"Mon Mar 18 20:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.44600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.439},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90600000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67300000000000004}]},{"osd":10,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.004,"5min":1.105,"15min":1.105},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.61499999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.016,"5min":1.177,"15min":1.177},"last":0.62}]},{"osd":11,"last update":"Mon Mar 18 20:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":482954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.58599999999999997,"15min":0.497},"min":{"1min":0.379,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":7.8559999999999999,"5min":7.8559999999999999,"15min":7.8559999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.58299999999999996,"15min":0.496},"min":{"1min":0.42799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.7880000000000003,"5min":7.7880000000000003,"15min":7.7880000000000003},"last":0.39300000000000002}]},{"osd":1,"last update":"Mon Mar 18 20:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58399999999999996,"15min":0.53700000000000003},"min":{"1min":0.48399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81499999999999995,"5min":4.6100000000000003,"15min":4.6100000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.53900000000000003}]},{"osd":3,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57099999999999995,"15min":0.52900000000000003},"min":{"1min":0.40000000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.497},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57799999999999996,"15min":0.53800000000000003},"min":{"1min":0.46700000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46700000000000003}]},{"osd":4,"last update":"Mon Mar 18 20:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.63},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Mar 18 20:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.47099999999999997,"15min":0.44700000000000001},"min":{"1min":0.39000000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.31},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.54900000000000004,"15min":0.53700000000000003},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Mon Mar 18 20:35:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.66300000000000003,"15min":0.65600000000000003},"min":{"1min":0.54500000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":0.75,"15min":0.68500000000000005},"min":{"1min":0.45300000000000001,"5min":0.436,"15min":0.436},"max":{"1min":9.9969999999999999,"5min":9.9969999999999999,"15min":9.9969999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Mon Mar 18 20:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.439},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45800000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80800000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.438},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84699999999999998,"5min":1.038,"15min":1.038},"last":0.64000000000000001}]},{"osd":9,"last update":"Mon Mar 18 20:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.53900000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93100000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.274,"5min":0.83299999999999996,"15min":0.75900000000000001},"min":{"1min":0.59999999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":9.8780000000000001,"5min":9.8780000000000001,"15min":9.8780000000000001},"last":0.65000000000000002}]},{"osd":10,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.89800000000000002,"15min":0.92000000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0589999999999999,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.751,"15min":0.745},"min":{"1min":0.60799999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97799999999999998,"5min":1.044,"15min":1.044},"last":0.60799999999999998}]},{"osd":11,"last update":"Mon Mar 18 20:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66900000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215156,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":422248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.41099999999999998},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.307},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.442,"15min":0.41999999999999998},"min":{"1min":0.33400000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.40699999999999997}]},{"osd":1,"last update":"Mon Mar 18 20:35:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.36799999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.052,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.41299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Mon Mar 18 20:35:52 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.372,"15min":0.372},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.46100000000000002}]},{"osd":4,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90700000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.625},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67800000000000005,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.876,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.69299999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72899999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84399999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52500000000000002}]},{"osd":6,"last update":"Mon Mar 18 20:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.376,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.62},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.755,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59799999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.90500000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.61299999999999999}]},{"osd":8,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73099999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.87,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55900000000000005}]},{"osd":10,"last update":"Mon Mar 18 20:35:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.77700000000000002,"15min":0.78600000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.97899999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72099999999999997}]},{"osd":11,"last update":"Mon Mar 18 20:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57499999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":422745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.38400000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.371,"15min":0.36099999999999999},"min":{"1min":0.31,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.435,"5min":0.374,"15min":0.35999999999999999},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.63,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.432}]},{"osd":2,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55100000000000005,"15min":0.52600000000000002},"min":{"1min":0.45600000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72199999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.62}]},{"osd":3,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.45400000000000001}]},{"osd":5,"last update":"Mon Mar 18 20:35:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.46100000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82299999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81200000000000006,"5min":0.875,"15min":0.875},"last":0.59199999999999997}]},{"osd":6,"last update":"Mon Mar 18 20:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.63200000000000001,"15min":0.58399999999999996},"min":{"1min":0.48499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":5.0460000000000003,"5min":5.0460000000000003,"15min":5.0460000000000003},"last":0.85099999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.621,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.88900000000000001}]},{"osd":8,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.55300000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.67500000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.873}]},{"osd":9,"last update":"Mon Mar 18 20:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63900000000000001,"15min":0.63},"min":{"1min":0.57499999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.65400000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82099999999999995}]},{"osd":10,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.49399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90500000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66500000000000004,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.86099999999999999}]},{"osd":11,"last update":"Mon Mar 18 20:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.72999999999999998}]}]},{"osd":5,"up_from":35,"seq":150323855406,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1503232,"data_stored":1014941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53300000000000003,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.745,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.61299999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.53400000000000003},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57199999999999995}]},{"osd":2,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.39500000000000002,"15min":0.38},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.435},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.40200000000000002,"15min":0.38400000000000001},"min":{"1min":0.317,"5min":0.25,"15min":0.25},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38400000000000001}]},{"osd":3,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.746,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.372,"15min":0.372},"max":{"1min":0.69299999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53300000000000003}]},{"osd":4,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.53100000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.41799999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Mon Mar 18 20:35:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.496,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.68400000000000005}]},{"osd":7,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.72199999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80400000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.626}]},{"osd":9,"last update":"Mon Mar 18 20:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.60599999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.73699999999999999,"15min":0.72499999999999998},"min":{"1min":0.625,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.73799999999999999}]},{"osd":10,"last update":"Mon Mar 18 20:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.57899999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.97599999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.56899999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T20:35:55.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T20:35:55.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.300+0000 7f3e8533d700 1 -- 172.21.15.46:0/4148829276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e80104060 msgr2=0x7f3e80106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.300+0000 7f3e8533d700 1 --2- 172.21.15.46:0/4148829276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e80104060 0x7f3e80106450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3e740098f0 tx=0x7f3e74009c00 comp rx=0 tx=0).stop 2024-03-18T20:35:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.300+0000 7f3e8533d700 1 -- 172.21.15.46:0/4148829276 shutdown_connections 2024-03-18T20:35:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.300+0000 7f3e8533d700 1 --2- 172.21.15.46:0/4148829276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e80106990 0x7f3e80108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.300+0000 7f3e8533d700 1 --2- 172.21.15.46:0/4148829276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e80101730 0x7f3e80103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.300+0000 7f3e8533d700 1 --2- 172.21.15.46:0/4148829276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e80104060 0x7f3e80106450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.300+0000 7f3e8533d700 1 -- 172.21.15.46:0/4148829276 >> 172.21.15.46:0/4148829276 conn(0x7f3e800fb0e0 msgr2=0x7f3e800fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.300+0000 7f3e8533d700 1 -- 172.21.15.46:0/4148829276 shutdown_connections 2024-03-18T20:35:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.300+0000 7f3e8533d700 1 -- 172.21.15.46:0/4148829276 wait complete. 2024-03-18T20:35:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.300+0000 7f3e8533d700 1 Processor -- start 2024-03-18T20:35:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e8533d700 1 -- start start 2024-03-18T20:35:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e8533d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e80101730 0x7f3e80196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e8533d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e80104060 0x7f3e801971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e8533d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e80106990 0x7f3e8019e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e8533d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8019e770 con 0x7f3e80106990 2024-03-18T20:35:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e8533d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8019e8e0 con 0x7f3e80101730 2024-03-18T20:35:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e8533d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8019eaa0 con 0x7f3e80104060 2024-03-18T20:35:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e7f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e80106990 0x7f3e8019e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e7f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e80106990 0x7f3e8019e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38632/0 (socket says 172.21.15.46:38632) 2024-03-18T20:35:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e7f7fe700 1 -- 172.21.15.46:0/1721734596 learned_addr learned my addr 172.21.15.46:0/1721734596 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e7effd700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e80101730 0x7f3e80196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e7e7fc700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e80104060 0x7f3e801971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e7f7fe700 1 -- 172.21.15.46:0/1721734596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e80104060 msgr2=0x7f3e801971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e7f7fe700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e80104060 0x7f3e801971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e7f7fe700 1 -- 172.21.15.46:0/1721734596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e80101730 msgr2=0x7f3e80196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e7f7fe700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e80101730 0x7f3e80196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.301+0000 7f3e7f7fe700 1 -- 172.21.15.46:0/1721734596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e78009d00 con 0x7f3e80106990 2024-03-18T20:35:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.302+0000 7f3e7f7fe700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e80106990 0x7f3e8019e1a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e3c0 tx=0x7f3e7800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.302+0000 7f3e67fff700 1 -- 172.21.15.46:0/1721734596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780092d0 con 0x7f3e80106990 2024-03-18T20:35:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.302+0000 7f3e8533d700 1 -- 172.21.15.46:0/1721734596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e74009520 con 0x7f3e80106990 2024-03-18T20:35:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.302+0000 7f3e67fff700 1 -- 172.21.15.46:0/1721734596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e7800f040 con 0x7f3e80106990 2024-03-18T20:35:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.302+0000 7f3e67fff700 1 -- 172.21.15.46:0/1721734596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78007cc0 con 0x7f3e80106990 2024-03-18T20:35:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.302+0000 7f3e8533d700 1 -- 172.21.15.46:0/1721734596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8019f100 con 0x7f3e80106990 2024-03-18T20:35:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.303+0000 7f3e8533d700 1 -- 172.21.15.46:0/1721734596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e800fcc60 con 0x7f3e80106990 2024-03-18T20:35:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.304+0000 7f3e67fff700 1 -- 172.21.15.46:0/1721734596 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3e78004ad0 con 0x7f3e80106990 2024-03-18T20:35:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.305+0000 7f3e67fff700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3e680a11b0 0x7f3e680a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.305+0000 7f3e7effd700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3e680a11b0 0x7f3e680a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.305+0000 7f3e67fff700 1 -- 172.21.15.46:0/1721734596 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3e780ccb10 con 0x7f3e80106990 2024-03-18T20:35:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.306+0000 7f3e7effd700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3e680a11b0 0x7f3e680a3670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3e80198100 tx=0x7f3e70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.309+0000 7f3e67fff700 1 -- 172.21.15.46:0/1721734596 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3e780961e0 con 0x7f3e80106990 2024-03-18T20:35:55.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:55 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3983978137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:55.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:55 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1437122841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.520+0000 7f3e8533d700 1 -- 172.21.15.46:0/1721734596 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e800611f0 con 0x7f3e680a11b0 2024-03-18T20:35:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.531+0000 7f3e67fff700 1 -- 172.21.15.46:0/1721734596 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344356 (secure 0 0 0) 0x7f3e800611f0 con 0x7f3e680a11b0 2024-03-18T20:35:55.538 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:55 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3983978137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:55 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1437122841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.534+0000 7f3e8533d700 1 -- 172.21.15.46:0/1721734596 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3e680a11b0 msgr2=0x7f3e680a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.534+0000 7f3e8533d700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3e680a11b0 0x7f3e680a3670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3e80198100 tx=0x7f3e70009260 comp rx=0 tx=0).stop 2024-03-18T20:35:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.535+0000 7f3e8533d700 1 -- 172.21.15.46:0/1721734596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e80106990 msgr2=0x7f3e8019e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.535+0000 7f3e8533d700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e80106990 0x7f3e8019e1a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e3c0 tx=0x7f3e7800e6d0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.535+0000 7f3e8533d700 1 -- 172.21.15.46:0/1721734596 shutdown_connections 2024-03-18T20:35:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.535+0000 7f3e8533d700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3e680a11b0 0x7f3e680a3670 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.535+0000 7f3e8533d700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e80106990 0x7f3e8019e1a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.535+0000 7f3e8533d700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e80101730 0x7f3e80196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.535+0000 7f3e8533d700 1 --2- 172.21.15.46:0/1721734596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e80104060 0x7f3e801971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.535+0000 7f3e8533d700 1 -- 172.21.15.46:0/1721734596 >> 172.21.15.46:0/1721734596 conn(0x7f3e800fb0e0 msgr2=0x7f3e80101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.535+0000 7f3e8533d700 1 -- 172.21.15.46:0/1721734596 shutdown_connections 2024-03-18T20:35:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.535+0000 7f3e8533d700 1 -- 172.21.15.46:0/1721734596 wait complete. 2024-03-18T20:35:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-18T20:35:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:55 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3983978137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:55 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1437122841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:55.552 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-03-18T20:35:55.037736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657270,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":852,"num_read_kb":550,"num_write":416,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336680,"kb_used_data":12432,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605144,"statfs":{"total":1151940427776,"available":1151595667456,"internally_reserved":0,"allocated":12730368,"data_stored":7030577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":628,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":52,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003020"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116835+0000","last_change":"2024-03-18T20:34:38.817028+0000","last_active":"2024-03-18T20:35:37.116835+0000","last_peered":"2024-03-18T20:35:37.116835+0000","last_clean":"2024-03-18T20:35:37.116835+0000","last_became_active":"2024-03-18T20:34:38.816711+0000","last_became_peered":"2024-03-18T20:34:38.816711+0000","last_unstale":"2024-03-18T20:35:37.116835+0000","last_undegraded":"2024-03-18T20:35:37.116835+0000","last_fullsized":"2024-03-18T20:35:37.116835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:32:16.827141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117600+0000","last_change":"2024-03-18T20:34:38.807350+0000","last_active":"2024-03-18T20:35:37.117600+0000","last_peered":"2024-03-18T20:35:37.117600+0000","last_clean":"2024-03-18T20:35:37.117600+0000","last_became_active":"2024-03-18T20:34:38.807008+0000","last_became_peered":"2024-03-18T20:34:38.807008+0000","last_unstale":"2024-03-18T20:35:37.117600+0000","last_undegraded":"2024-03-18T20:35:37.117600+0000","last_fullsized":"2024-03-18T20:35:37.117600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:35.667384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117502+0000","last_change":"2024-03-18T20:34:38.817045+0000","last_active":"2024-03-18T20:35:37.117502+0000","last_peered":"2024-03-18T20:35:37.117502+0000","last_clean":"2024-03-18T20:35:37.117502+0000","last_became_active":"2024-03-18T20:34:38.816659+0000","last_became_peered":"2024-03-18T20:34:38.816659+0000","last_unstale":"2024-03-18T20:35:37.117502+0000","last_undegraded":"2024-03-18T20:35:37.117502+0000","last_fullsized":"2024-03-18T20:35:37.117502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:04:49.316127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116820+0000","last_change":"2024-03-18T20:34:38.804833+0000","last_active":"2024-03-18T20:35:37.116820+0000","last_peered":"2024-03-18T20:35:37.116820+0000","last_clean":"2024-03-18T20:35:37.116820+0000","last_became_active":"2024-03-18T20:34:38.804486+0000","last_became_peered":"2024-03-18T20:34:38.804486+0000","last_unstale":"2024-03-18T20:35:37.116820+0000","last_undegraded":"2024-03-18T20:35:37.116820+0000","last_fullsized":"2024-03-18T20:35:37.116820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:19:44.266207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110860+0000","last_change":"2024-03-18T20:34:38.797660+0000","last_active":"2024-03-18T20:35:37.110860+0000","last_peered":"2024-03-18T20:35:37.110860+0000","last_clean":"2024-03-18T20:35:37.110860+0000","last_became_active":"2024-03-18T20:34:38.797155+0000","last_became_peered":"2024-03-18T20:34:38.797155+0000","last_unstale":"2024-03-18T20:35:37.110860+0000","last_undegraded":"2024-03-18T20:35:37.110860+0000","last_fullsized":"2024-03-18T20:35:37.110860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:22:11.363453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.118992+0000","last_change":"2024-03-18T20:35:38.118992+0000","last_active":"2024-03-18T20:35:38.118992+0000","last_peered":"2024-03-18T20:35:38.118992+0000","last_clean":"2024-03-18T20:35:38.118992+0000","last_became_active":"2024-03-18T20:35:38.118632+0000","last_became_peered":"2024-03-18T20:35:38.118632+0000","last_unstale":"2024-03-18T20:35:38.118992+0000","last_undegraded":"2024-03-18T20:35:38.118992+0000","last_fullsized":"2024-03-18T20:35:38.118992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:00:49.799078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115609+0000","last_change":"2024-03-18T20:34:38.805378+0000","last_active":"2024-03-18T20:35:37.115609+0000","last_peered":"2024-03-18T20:35:37.115609+0000","last_clean":"2024-03-18T20:35:37.115609+0000","last_became_active":"2024-03-18T20:34:38.805130+0000","last_became_peered":"2024-03-18T20:34:38.805130+0000","last_unstale":"2024-03-18T20:35:37.115609+0000","last_undegraded":"2024-03-18T20:35:37.115609+0000","last_fullsized":"2024-03-18T20:35:37.115609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:46:19.420715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.118345+0000","last_change":"2024-03-18T20:35:38.117964+0000","last_active":"2024-03-18T20:35:38.118345+0000","last_peered":"2024-03-18T20:35:38.118345+0000","last_clean":"2024-03-18T20:35:38.118345+0000","last_became_active":"2024-03-18T20:35:38.117579+0000","last_became_peered":"2024-03-18T20:35:38.117579+0000","last_unstale":"2024-03-18T20:35:38.118345+0000","last_undegraded":"2024-03-18T20:35:38.118345+0000","last_fullsized":"2024-03-18T20:35:38.118345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:44:20.292995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.234734+0000","last_change":"2024-03-18T20:35:38.234734+0000","last_active":"2024-03-18T20:35:38.234734+0000","last_peered":"2024-03-18T20:35:38.234734+0000","last_clean":"2024-03-18T20:35:38.234734+0000","last_became_active":"2024-03-18T20:35:38.234403+0000","last_became_peered":"2024-03-18T20:35:38.234403+0000","last_unstale":"2024-03-18T20:35:38.234734+0000","last_undegraded":"2024-03-18T20:35:38.234734+0000","last_fullsized":"2024-03-18T20:35:38.234734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:13:54.696645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119706+0000","last_change":"2024-03-18T20:34:38.814728+0000","last_active":"2024-03-18T20:35:37.119706+0000","last_peered":"2024-03-18T20:35:37.119706+0000","last_clean":"2024-03-18T20:35:37.119706+0000","last_became_active":"2024-03-18T20:34:38.814200+0000","last_became_peered":"2024-03-18T20:34:38.814200+0000","last_unstale":"2024-03-18T20:35:37.119706+0000","last_undegraded":"2024-03-18T20:35:37.119706+0000","last_fullsized":"2024-03-18T20:35:37.119706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:24:54.064770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115901+0000","last_change":"2024-03-18T20:34:38.818054+0000","last_active":"2024-03-18T20:35:37.115901+0000","last_peered":"2024-03-18T20:35:37.115901+0000","last_clean":"2024-03-18T20:35:37.115901+0000","last_became_active":"2024-03-18T20:34:38.817752+0000","last_became_peered":"2024-03-18T20:34:38.817752+0000","last_unstale":"2024-03-18T20:35:37.115901+0000","last_undegraded":"2024-03-18T20:35:37.115901+0000","last_fullsized":"2024-03-18T20:35:37.115901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:01:13.728434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116729+0000","last_change":"2024-03-18T20:34:38.801356+0000","last_active":"2024-03-18T20:35:37.116729+0000","last_peered":"2024-03-18T20:35:37.116729+0000","last_clean":"2024-03-18T20:35:37.116729+0000","last_became_active":"2024-03-18T20:34:38.800978+0000","last_became_peered":"2024-03-18T20:34:38.800978+0000","last_unstale":"2024-03-18T20:35:37.116729+0000","last_undegraded":"2024-03-18T20:35:37.116729+0000","last_fullsized":"2024-03-18T20:35:37.116729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:07.634686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110534+0000","last_change":"2024-03-18T20:34:38.803006+0000","last_active":"2024-03-18T20:35:37.110534+0000","last_peered":"2024-03-18T20:35:37.110534+0000","last_clean":"2024-03-18T20:35:37.110534+0000","last_became_active":"2024-03-18T20:34:38.802454+0000","last_became_peered":"2024-03-18T20:34:38.802454+0000","last_unstale":"2024-03-18T20:35:37.110534+0000","last_undegraded":"2024-03-18T20:35:37.110534+0000","last_fullsized":"2024-03-18T20:35:37.110534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:11:02.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116047+0000","last_change":"2024-03-18T20:34:38.807397+0000","last_active":"2024-03-18T20:35:37.116047+0000","last_peered":"2024-03-18T20:35:37.116047+0000","last_clean":"2024-03-18T20:35:37.116047+0000","last_became_active":"2024-03-18T20:34:38.807072+0000","last_became_peered":"2024-03-18T20:34:38.807072+0000","last_unstale":"2024-03-18T20:35:37.116047+0000","last_undegraded":"2024-03-18T20:35:37.116047+0000","last_fullsized":"2024-03-18T20:35:37.116047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:55:53.832825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116047+0000","last_change":"2024-03-18T20:34:38.812925+0000","last_active":"2024-03-18T20:35:37.116047+0000","last_peered":"2024-03-18T20:35:37.116047+0000","last_clean":"2024-03-18T20:35:37.116047+0000","last_became_active":"2024-03-18T20:34:38.812597+0000","last_became_peered":"2024-03-18T20:34:38.812597+0000","last_unstale":"2024-03-18T20:35:37.116047+0000","last_undegraded":"2024-03-18T20:35:37.116047+0000","last_fullsized":"2024-03-18T20:35:37.116047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:14:35.103852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110532+0000","last_change":"2024-03-18T20:34:38.817892+0000","last_active":"2024-03-18T20:35:37.110532+0000","last_peered":"2024-03-18T20:35:37.110532+0000","last_clean":"2024-03-18T20:35:37.110532+0000","last_became_active":"2024-03-18T20:34:38.816261+0000","last_became_peered":"2024-03-18T20:34:38.816261+0000","last_unstale":"2024-03-18T20:35:37.110532+0000","last_undegraded":"2024-03-18T20:35:37.110532+0000","last_fullsized":"2024-03-18T20:35:37.110532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:32.481210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116723+0000","last_change":"2024-03-18T20:34:38.816966+0000","last_active":"2024-03-18T20:35:37.116723+0000","last_peered":"2024-03-18T20:35:37.116723+0000","last_clean":"2024-03-18T20:35:37.116723+0000","last_became_active":"2024-03-18T20:34:38.816645+0000","last_became_peered":"2024-03-18T20:34:38.816645+0000","last_unstale":"2024-03-18T20:35:37.116723+0000","last_undegraded":"2024-03-18T20:35:37.116723+0000","last_fullsized":"2024-03-18T20:35:37.116723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:39.904353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.118561+0000","last_change":"2024-03-18T20:34:38.817005+0000","last_active":"2024-03-18T20:35:37.118561+0000","last_peered":"2024-03-18T20:35:37.118561+0000","last_clean":"2024-03-18T20:35:37.118561+0000","last_became_active":"2024-03-18T20:34:38.816710+0000","last_became_peered":"2024-03-18T20:34:38.816710+0000","last_unstale":"2024-03-18T20:35:37.118561+0000","last_undegraded":"2024-03-18T20:35:37.118561+0000","last_fullsized":"2024-03-18T20:35:37.118561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:30:43.483977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117763+0000","last_change":"2024-03-18T20:34:38.810957+0000","last_active":"2024-03-18T20:35:37.117763+0000","last_peered":"2024-03-18T20:35:37.117763+0000","last_clean":"2024-03-18T20:35:37.117763+0000","last_became_active":"2024-03-18T20:34:38.810615+0000","last_became_peered":"2024-03-18T20:34:38.810615+0000","last_unstale":"2024-03-18T20:35:37.117763+0000","last_undegraded":"2024-03-18T20:35:37.117763+0000","last_fullsized":"2024-03-18T20:35:37.117763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:26:36.544034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":94,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:49.909225+0000","last_change":"2024-03-18T20:34:38.337863+0000","last_active":"2024-03-18T20:35:49.909225+0000","last_peered":"2024-03-18T20:35:49.909225+0000","last_clean":"2024-03-18T20:35:49.909225+0000","last_became_active":"2024-03-18T20:34:38.337548+0000","last_became_peered":"2024-03-18T20:34:38.337548+0000","last_unstale":"2024-03-18T20:35:49.909225+0000","last_undegraded":"2024-03-18T20:35:49.909225+0000","last_fullsized":"2024-03-18T20:35:49.909225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:41:40.237882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120594+0000","last_change":"2024-03-18T20:34:38.814321+0000","last_active":"2024-03-18T20:35:37.120594+0000","last_peered":"2024-03-18T20:35:37.120594+0000","last_clean":"2024-03-18T20:35:37.120594+0000","last_became_active":"2024-03-18T20:34:38.813939+0000","last_became_peered":"2024-03-18T20:34:38.813939+0000","last_unstale":"2024-03-18T20:35:37.120594+0000","last_undegraded":"2024-03-18T20:35:37.120594+0000","last_fullsized":"2024-03-18T20:35:37.120594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:05:42.441099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120794+0000","last_change":"2024-03-18T20:34:38.347790+0000","last_active":"2024-03-18T20:35:37.120794+0000","last_peered":"2024-03-18T20:35:37.120794+0000","last_clean":"2024-03-18T20:35:37.120794+0000","last_became_active":"2024-03-18T20:34:38.347317+0000","last_became_peered":"2024-03-18T20:34:38.347317+0000","last_unstale":"2024-03-18T20:35:37.120794+0000","last_undegraded":"2024-03-18T20:35:37.120794+0000","last_fullsized":"2024-03-18T20:35:37.120794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.096480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.118730+0000","last_change":"2024-03-18T20:34:38.817116+0000","last_active":"2024-03-18T20:35:37.118730+0000","last_peered":"2024-03-18T20:35:37.118730+0000","last_clean":"2024-03-18T20:35:37.118730+0000","last_became_active":"2024-03-18T20:34:38.816761+0000","last_became_peered":"2024-03-18T20:34:38.816761+0000","last_unstale":"2024-03-18T20:35:37.118730+0000","last_undegraded":"2024-03-18T20:35:37.118730+0000","last_fullsized":"2024-03-18T20:35:37.118730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:44.353533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111161+0000","last_change":"2024-03-18T20:34:38.339236+0000","last_active":"2024-03-18T20:35:37.111161+0000","last_peered":"2024-03-18T20:35:37.111161+0000","last_clean":"2024-03-18T20:35:37.111161+0000","last_became_active":"2024-03-18T20:34:38.338879+0000","last_became_peered":"2024-03-18T20:34:38.338879+0000","last_unstale":"2024-03-18T20:35:37.111161+0000","last_undegraded":"2024-03-18T20:35:37.111161+0000","last_fullsized":"2024-03-18T20:35:37.111161+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:33.340151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111163+0000","last_change":"2024-03-18T20:34:38.817861+0000","last_active":"2024-03-18T20:35:37.111163+0000","last_peered":"2024-03-18T20:35:37.111163+0000","last_clean":"2024-03-18T20:35:37.111163+0000","last_became_active":"2024-03-18T20:34:38.815976+0000","last_became_peered":"2024-03-18T20:34:38.815976+0000","last_unstale":"2024-03-18T20:35:37.111163+0000","last_undegraded":"2024-03-18T20:35:37.111163+0000","last_fullsized":"2024-03-18T20:35:37.111163+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:54.070067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111718+0000","last_change":"2024-03-18T20:34:38.345758+0000","last_active":"2024-03-18T20:35:37.111718+0000","last_peered":"2024-03-18T20:35:37.111718+0000","last_clean":"2024-03-18T20:35:37.111718+0000","last_became_active":"2024-03-18T20:34:38.345417+0000","last_became_peered":"2024-03-18T20:34:38.345417+0000","last_unstale":"2024-03-18T20:35:37.111718+0000","last_undegraded":"2024-03-18T20:35:37.111718+0000","last_fullsized":"2024-03-18T20:35:37.111718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:25.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113736+0000","last_change":"2024-03-18T20:34:38.818745+0000","last_active":"2024-03-18T20:35:37.113736+0000","last_peered":"2024-03-18T20:35:37.113736+0000","last_clean":"2024-03-18T20:35:37.113736+0000","last_became_active":"2024-03-18T20:34:38.818415+0000","last_became_peered":"2024-03-18T20:34:38.818415+0000","last_unstale":"2024-03-18T20:35:37.113736+0000","last_undegraded":"2024-03-18T20:35:37.113736+0000","last_fullsized":"2024-03-18T20:35:37.113736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:45:49.454498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":93,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:52.893926+0000","last_change":"2024-03-18T20:34:38.345804+0000","last_active":"2024-03-18T20:35:52.893926+0000","last_peered":"2024-03-18T20:35:52.893926+0000","last_clean":"2024-03-18T20:35:52.893926+0000","last_became_active":"2024-03-18T20:34:38.345409+0000","last_became_peered":"2024-03-18T20:34:38.345409+0000","last_unstale":"2024-03-18T20:35:52.893926+0000","last_undegraded":"2024-03-18T20:35:52.893926+0000","last_fullsized":"2024-03-18T20:35:52.893926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:00.085412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":888,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111822+0000","last_change":"2024-03-18T20:34:38.799821+0000","last_active":"2024-03-18T20:35:37.111822+0000","last_peered":"2024-03-18T20:35:37.111822+0000","last_clean":"2024-03-18T20:35:37.111822+0000","last_became_active":"2024-03-18T20:34:38.799457+0000","last_became_peered":"2024-03-18T20:34:38.799457+0000","last_unstale":"2024-03-18T20:35:37.111822+0000","last_undegraded":"2024-03-18T20:35:37.111822+0000","last_fullsized":"2024-03-18T20:35:37.111822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:39:35.912892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120102+0000","last_change":"2024-03-18T20:34:38.341999+0000","last_active":"2024-03-18T20:35:37.120102+0000","last_peered":"2024-03-18T20:35:37.120102+0000","last_clean":"2024-03-18T20:35:37.120102+0000","last_became_active":"2024-03-18T20:34:38.341431+0000","last_became_peered":"2024-03-18T20:34:38.341431+0000","last_unstale":"2024-03-18T20:35:37.120102+0000","last_undegraded":"2024-03-18T20:35:37.120102+0000","last_fullsized":"2024-03-18T20:35:37.120102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:07:26.711113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.114037+0000","last_change":"2024-03-18T20:34:38.804003+0000","last_active":"2024-03-18T20:35:37.114037+0000","last_peered":"2024-03-18T20:35:37.114037+0000","last_clean":"2024-03-18T20:35:37.114037+0000","last_became_active":"2024-03-18T20:34:38.803667+0000","last_became_peered":"2024-03-18T20:34:38.803667+0000","last_unstale":"2024-03-18T20:35:37.114037+0000","last_undegraded":"2024-03-18T20:35:37.114037+0000","last_fullsized":"2024-03-18T20:35:37.114037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:19:58.230790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110346+0000","last_change":"2024-03-18T20:34:38.346493+0000","last_active":"2024-03-18T20:35:37.110346+0000","last_peered":"2024-03-18T20:35:37.110346+0000","last_clean":"2024-03-18T20:35:37.110346+0000","last_became_active":"2024-03-18T20:34:38.345925+0000","last_became_peered":"2024-03-18T20:34:38.345925+0000","last_unstale":"2024-03-18T20:35:37.110346+0000","last_undegraded":"2024-03-18T20:35:37.110346+0000","last_fullsized":"2024-03-18T20:35:37.110346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:52:47.773860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119099+0000","last_change":"2024-03-18T20:34:38.807284+0000","last_active":"2024-03-18T20:35:37.119099+0000","last_peered":"2024-03-18T20:35:37.119099+0000","last_clean":"2024-03-18T20:35:37.119099+0000","last_became_active":"2024-03-18T20:34:38.806844+0000","last_became_peered":"2024-03-18T20:34:38.806844+0000","last_unstale":"2024-03-18T20:35:37.119099+0000","last_undegraded":"2024-03-18T20:35:37.119099+0000","last_fullsized":"2024-03-18T20:35:37.119099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:44:35.469953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.114113+0000","last_change":"2024-03-18T20:34:38.343090+0000","last_active":"2024-03-18T20:35:37.114113+0000","last_peered":"2024-03-18T20:35:37.114113+0000","last_clean":"2024-03-18T20:35:37.114113+0000","last_became_active":"2024-03-18T20:34:38.342712+0000","last_became_peered":"2024-03-18T20:34:38.342712+0000","last_unstale":"2024-03-18T20:35:37.114113+0000","last_undegraded":"2024-03-18T20:35:37.114113+0000","last_fullsized":"2024-03-18T20:35:37.114113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:14:05.909604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116701+0000","last_change":"2024-03-18T20:34:38.800616+0000","last_active":"2024-03-18T20:35:37.116701+0000","last_peered":"2024-03-18T20:35:37.116701+0000","last_clean":"2024-03-18T20:35:37.116701+0000","last_became_active":"2024-03-18T20:34:38.800340+0000","last_became_peered":"2024-03-18T20:34:38.800340+0000","last_unstale":"2024-03-18T20:35:37.116701+0000","last_undegraded":"2024-03-18T20:35:37.116701+0000","last_fullsized":"2024-03-18T20:35:37.116701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:18.263179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'5","reported_seq":155,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:51.891731+0000","last_change":"2024-03-18T20:34:38.345634+0000","last_active":"2024-03-18T20:35:51.891731+0000","last_peered":"2024-03-18T20:35:51.891731+0000","last_clean":"2024-03-18T20:35:51.891731+0000","last_became_active":"2024-03-18T20:34:38.345275+0000","last_became_peered":"2024-03-18T20:34:38.345275+0000","last_unstale":"2024-03-18T20:35:51.891731+0000","last_undegraded":"2024-03-18T20:35:51.891731+0000","last_fullsized":"2024-03-18T20:35:51.891731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:29.939529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110859+0000","last_change":"2024-03-18T20:34:38.820286+0000","last_active":"2024-03-18T20:35:37.110859+0000","last_peered":"2024-03-18T20:35:37.110859+0000","last_clean":"2024-03-18T20:35:37.110859+0000","last_became_active":"2024-03-18T20:34:38.819939+0000","last_became_peered":"2024-03-18T20:34:38.819939+0000","last_unstale":"2024-03-18T20:35:37.110859+0000","last_undegraded":"2024-03-18T20:35:37.110859+0000","last_fullsized":"2024-03-18T20:35:37.110859+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:19.229339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116083+0000","last_change":"2024-03-18T20:34:38.341699+0000","last_active":"2024-03-18T20:35:37.116083+0000","last_peered":"2024-03-18T20:35:37.116083+0000","last_clean":"2024-03-18T20:35:37.116083+0000","last_became_active":"2024-03-18T20:34:38.341390+0000","last_became_peered":"2024-03-18T20:34:38.341390+0000","last_unstale":"2024-03-18T20:35:37.116083+0000","last_undegraded":"2024-03-18T20:35:37.116083+0000","last_fullsized":"2024-03-18T20:35:37.116083+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:16:02.239414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.117141+0000","last_change":"2024-03-18T20:35:38.117141+0000","last_active":"2024-03-18T20:35:38.117141+0000","last_peered":"2024-03-18T20:35:38.117141+0000","last_clean":"2024-03-18T20:35:38.117141+0000","last_became_active":"2024-03-18T20:35:38.116809+0000","last_became_peered":"2024-03-18T20:35:38.116809+0000","last_unstale":"2024-03-18T20:35:38.117141+0000","last_undegraded":"2024-03-18T20:35:38.117141+0000","last_fullsized":"2024-03-18T20:35:38.117141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:10:22.545159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113855+0000","last_change":"2024-03-18T20:34:38.346821+0000","last_active":"2024-03-18T20:35:37.113855+0000","last_peered":"2024-03-18T20:35:37.113855+0000","last_clean":"2024-03-18T20:35:37.113855+0000","last_became_active":"2024-03-18T20:34:38.346314+0000","last_became_peered":"2024-03-18T20:34:38.346314+0000","last_unstale":"2024-03-18T20:35:37.113855+0000","last_undegraded":"2024-03-18T20:35:37.113855+0000","last_fullsized":"2024-03-18T20:35:37.113855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:12:04.552971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111241+0000","last_change":"2024-03-18T20:34:38.820479+0000","last_active":"2024-03-18T20:35:37.111241+0000","last_peered":"2024-03-18T20:35:37.111241+0000","last_clean":"2024-03-18T20:35:37.111241+0000","last_became_active":"2024-03-18T20:34:38.820186+0000","last_became_peered":"2024-03-18T20:34:38.820186+0000","last_unstale":"2024-03-18T20:35:37.111241+0000","last_undegraded":"2024-03-18T20:35:37.111241+0000","last_fullsized":"2024-03-18T20:35:37.111241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:15.953764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":188,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:51.891872+0000","last_change":"2024-03-18T20:34:38.344332+0000","last_active":"2024-03-18T20:35:51.891872+0000","last_peered":"2024-03-18T20:35:51.891872+0000","last_clean":"2024-03-18T20:35:51.891872+0000","last_became_active":"2024-03-18T20:34:38.343827+0000","last_became_peered":"2024-03-18T20:34:38.343827+0000","last_unstale":"2024-03-18T20:35:51.891872+0000","last_undegraded":"2024-03-18T20:35:51.891872+0000","last_fullsized":"2024-03-18T20:35:51.891872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:13:21.725004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59732,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":143,"num_read_kb":130,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111038+0000","last_change":"2024-03-18T20:34:38.814794+0000","last_active":"2024-03-18T20:35:37.111038+0000","last_peered":"2024-03-18T20:35:37.111038+0000","last_clean":"2024-03-18T20:35:37.111038+0000","last_became_active":"2024-03-18T20:34:38.814434+0000","last_became_peered":"2024-03-18T20:34:38.814434+0000","last_unstale":"2024-03-18T20:35:37.111038+0000","last_undegraded":"2024-03-18T20:35:37.111038+0000","last_fullsized":"2024-03-18T20:35:37.111038+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:43:19.083212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117588+0000","last_change":"2024-03-18T20:34:38.343894+0000","last_active":"2024-03-18T20:35:37.117588+0000","last_peered":"2024-03-18T20:35:37.117588+0000","last_clean":"2024-03-18T20:35:37.117588+0000","last_became_active":"2024-03-18T20:34:38.343493+0000","last_became_peered":"2024-03-18T20:34:38.343493+0000","last_unstale":"2024-03-18T20:35:37.117588+0000","last_undegraded":"2024-03-18T20:35:37.117588+0000","last_fullsized":"2024-03-18T20:35:37.117588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:13:32.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110231+0000","last_change":"2024-03-18T20:34:38.809968+0000","last_active":"2024-03-18T20:35:37.110231+0000","last_peered":"2024-03-18T20:35:37.110231+0000","last_clean":"2024-03-18T20:35:37.110231+0000","last_became_active":"2024-03-18T20:34:38.809641+0000","last_became_peered":"2024-03-18T20:34:38.809641+0000","last_unstale":"2024-03-18T20:35:37.110231+0000","last_undegraded":"2024-03-18T20:35:37.110231+0000","last_fullsized":"2024-03-18T20:35:37.110231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:50:52.925683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113815+0000","last_change":"2024-03-18T20:34:38.345621+0000","last_active":"2024-03-18T20:35:37.113815+0000","last_peered":"2024-03-18T20:35:37.113815+0000","last_clean":"2024-03-18T20:35:37.113815+0000","last_became_active":"2024-03-18T20:34:38.345255+0000","last_became_peered":"2024-03-18T20:34:38.345255+0000","last_unstale":"2024-03-18T20:35:37.113815+0000","last_undegraded":"2024-03-18T20:35:37.113815+0000","last_fullsized":"2024-03-18T20:35:37.113815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:17:03.737973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115794+0000","last_change":"2024-03-18T20:34:38.804390+0000","last_active":"2024-03-18T20:35:37.115794+0000","last_peered":"2024-03-18T20:35:37.115794+0000","last_clean":"2024-03-18T20:35:37.115794+0000","last_became_active":"2024-03-18T20:34:38.804033+0000","last_became_peered":"2024-03-18T20:34:38.804033+0000","last_unstale":"2024-03-18T20:35:37.115794+0000","last_undegraded":"2024-03-18T20:35:37.115794+0000","last_fullsized":"2024-03-18T20:35:37.115794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:09:45.407921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":91,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:51.891203+0000","last_change":"2024-03-18T20:34:38.340239+0000","last_active":"2024-03-18T20:35:51.891203+0000","last_peered":"2024-03-18T20:35:51.891203+0000","last_clean":"2024-03-18T20:35:51.891203+0000","last_became_active":"2024-03-18T20:34:38.339880+0000","last_became_peered":"2024-03-18T20:34:38.339880+0000","last_unstale":"2024-03-18T20:35:51.891203+0000","last_undegraded":"2024-03-18T20:35:51.891203+0000","last_fullsized":"2024-03-18T20:35:51.891203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:55:46.864885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115572+0000","last_change":"2024-03-18T20:34:38.819223+0000","last_active":"2024-03-18T20:35:37.115572+0000","last_peered":"2024-03-18T20:35:37.115572+0000","last_clean":"2024-03-18T20:35:37.115572+0000","last_became_active":"2024-03-18T20:34:38.818879+0000","last_became_peered":"2024-03-18T20:34:38.818879+0000","last_unstale":"2024-03-18T20:35:37.115572+0000","last_undegraded":"2024-03-18T20:35:37.115572+0000","last_fullsized":"2024-03-18T20:35:37.115572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:45.645912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110850+0000","last_change":"2024-03-18T20:34:38.336817+0000","last_active":"2024-03-18T20:35:37.110850+0000","last_peered":"2024-03-18T20:35:37.110850+0000","last_clean":"2024-03-18T20:35:37.110850+0000","last_became_active":"2024-03-18T20:34:38.336453+0000","last_became_peered":"2024-03-18T20:34:38.336453+0000","last_unstale":"2024-03-18T20:35:37.110850+0000","last_undegraded":"2024-03-18T20:35:37.110850+0000","last_fullsized":"2024-03-18T20:35:37.110850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:10.253435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113661+0000","last_change":"2024-03-18T20:34:38.818048+0000","last_active":"2024-03-18T20:35:37.113661+0000","last_peered":"2024-03-18T20:35:37.113661+0000","last_clean":"2024-03-18T20:35:37.113661+0000","last_became_active":"2024-03-18T20:34:38.817682+0000","last_became_peered":"2024-03-18T20:34:38.817682+0000","last_unstale":"2024-03-18T20:35:37.113661+0000","last_undegraded":"2024-03-18T20:35:37.113661+0000","last_fullsized":"2024-03-18T20:35:37.113661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:25:55.291815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.126651+0000","last_change":"2024-03-18T20:35:38.126271+0000","last_active":"2024-03-18T20:35:38.126651+0000","last_peered":"2024-03-18T20:35:38.126651+0000","last_clean":"2024-03-18T20:35:38.126651+0000","last_became_active":"2024-03-18T20:35:38.119078+0000","last_became_peered":"2024-03-18T20:35:38.119078+0000","last_unstale":"2024-03-18T20:35:38.126651+0000","last_undegraded":"2024-03-18T20:35:38.126651+0000","last_fullsized":"2024-03-18T20:35:38.126651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:46:20.914329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116043+0000","last_change":"2024-03-18T20:34:38.806138+0000","last_active":"2024-03-18T20:35:37.116043+0000","last_peered":"2024-03-18T20:35:37.116043+0000","last_clean":"2024-03-18T20:35:37.116043+0000","last_became_active":"2024-03-18T20:34:38.805821+0000","last_became_peered":"2024-03-18T20:34:38.805821+0000","last_unstale":"2024-03-18T20:35:37.116043+0000","last_undegraded":"2024-03-18T20:35:37.116043+0000","last_fullsized":"2024-03-18T20:35:37.116043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:25:57.887813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.118124+0000","last_change":"2024-03-18T20:34:38.338751+0000","last_active":"2024-03-18T20:35:37.118124+0000","last_peered":"2024-03-18T20:35:37.118124+0000","last_clean":"2024-03-18T20:35:37.118124+0000","last_became_active":"2024-03-18T20:34:38.337495+0000","last_became_peered":"2024-03-18T20:34:38.337495+0000","last_unstale":"2024-03-18T20:35:37.118124+0000","last_undegraded":"2024-03-18T20:35:37.118124+0000","last_fullsized":"2024-03-18T20:35:37.118124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:38:04.402380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119498+0000","last_change":"2024-03-18T20:34:38.803937+0000","last_active":"2024-03-18T20:35:37.119498+0000","last_peered":"2024-03-18T20:35:37.119498+0000","last_clean":"2024-03-18T20:35:37.119498+0000","last_became_active":"2024-03-18T20:34:38.803520+0000","last_became_peered":"2024-03-18T20:34:38.803520+0000","last_unstale":"2024-03-18T20:35:37.119498+0000","last_undegraded":"2024-03-18T20:35:37.119498+0000","last_fullsized":"2024-03-18T20:35:37.119498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:20:37.537265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":85,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:48.908311+0000","last_change":"2024-03-18T20:34:38.337481+0000","last_active":"2024-03-18T20:35:48.908311+0000","last_peered":"2024-03-18T20:35:48.908311+0000","last_clean":"2024-03-18T20:35:48.908311+0000","last_became_active":"2024-03-18T20:34:38.337152+0000","last_became_peered":"2024-03-18T20:34:38.337152+0000","last_unstale":"2024-03-18T20:35:48.908311+0000","last_undegraded":"2024-03-18T20:35:48.908311+0000","last_fullsized":"2024-03-18T20:35:48.908311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:07:59.593978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.119360+0000","last_change":"2024-03-18T20:35:38.119360+0000","last_active":"2024-03-18T20:35:38.119360+0000","last_peered":"2024-03-18T20:35:38.119360+0000","last_clean":"2024-03-18T20:35:38.119360+0000","last_became_active":"2024-03-18T20:35:38.118942+0000","last_became_peered":"2024-03-18T20:35:38.118942+0000","last_unstale":"2024-03-18T20:35:38.119360+0000","last_undegraded":"2024-03-18T20:35:38.119360+0000","last_fullsized":"2024-03-18T20:35:38.119360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:42.140321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.114312+0000","last_change":"2024-03-18T20:34:38.806133+0000","last_active":"2024-03-18T20:35:37.114312+0000","last_peered":"2024-03-18T20:35:37.114312+0000","last_clean":"2024-03-18T20:35:37.114312+0000","last_became_active":"2024-03-18T20:34:38.805257+0000","last_became_peered":"2024-03-18T20:34:38.805257+0000","last_unstale":"2024-03-18T20:35:37.114312+0000","last_undegraded":"2024-03-18T20:35:37.114312+0000","last_fullsized":"2024-03-18T20:35:37.114312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:05:15.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":88,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:50.889127+0000","last_change":"2024-03-18T20:34:38.346271+0000","last_active":"2024-03-18T20:35:50.889127+0000","last_peered":"2024-03-18T20:35:50.889127+0000","last_clean":"2024-03-18T20:35:50.889127+0000","last_became_active":"2024-03-18T20:34:38.345054+0000","last_became_peered":"2024-03-18T20:34:38.345054+0000","last_unstale":"2024-03-18T20:35:50.889127+0000","last_undegraded":"2024-03-18T20:35:50.889127+0000","last_fullsized":"2024-03-18T20:35:50.889127+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:42:52.510294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119964+0000","last_change":"2024-03-18T20:34:38.816834+0000","last_active":"2024-03-18T20:35:37.119964+0000","last_peered":"2024-03-18T20:35:37.119964+0000","last_clean":"2024-03-18T20:35:37.119964+0000","last_became_active":"2024-03-18T20:34:38.816516+0000","last_became_peered":"2024-03-18T20:34:38.816516+0000","last_unstale":"2024-03-18T20:35:37.119964+0000","last_undegraded":"2024-03-18T20:35:37.119964+0000","last_fullsized":"2024-03-18T20:35:37.119964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:33:54.529206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117428+0000","last_change":"2024-03-18T20:34:38.339407+0000","last_active":"2024-03-18T20:35:37.117428+0000","last_peered":"2024-03-18T20:35:37.117428+0000","last_clean":"2024-03-18T20:35:37.117428+0000","last_became_active":"2024-03-18T20:34:38.339086+0000","last_became_peered":"2024-03-18T20:34:38.339086+0000","last_unstale":"2024-03-18T20:35:37.117428+0000","last_undegraded":"2024-03-18T20:35:37.117428+0000","last_fullsized":"2024-03-18T20:35:37.117428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:18.479630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":333,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115643+0000","last_change":"2024-03-18T20:33:50.777651+0000","last_active":"2024-03-18T20:35:37.115643+0000","last_peered":"2024-03-18T20:35:37.115643+0000","last_clean":"2024-03-18T20:35:37.115643+0000","last_became_active":"2024-03-18T20:33:50.777281+0000","last_became_peered":"2024-03-18T20:33:50.777281+0000","last_unstale":"2024-03-18T20:35:37.115643+0000","last_undegraded":"2024-03-18T20:35:37.115643+0000","last_fullsized":"2024-03-18T20:35:37.115643+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_clean_scrub_stamp":"2024-03-18T20:31:27.488744+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:28.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110103+0000","last_change":"2024-03-18T20:34:38.812714+0000","last_active":"2024-03-18T20:35:37.110103+0000","last_peered":"2024-03-18T20:35:37.110103+0000","last_clean":"2024-03-18T20:35:37.110103+0000","last_became_active":"2024-03-18T20:34:38.812464+0000","last_became_peered":"2024-03-18T20:34:38.812464+0000","last_unstale":"2024-03-18T20:35:37.110103+0000","last_undegraded":"2024-03-18T20:35:37.110103+0000","last_fullsized":"2024-03-18T20:35:37.110103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:03:13.853740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115734+0000","last_change":"2024-03-18T20:34:38.343173+0000","last_active":"2024-03-18T20:35:37.115734+0000","last_peered":"2024-03-18T20:35:37.115734+0000","last_clean":"2024-03-18T20:35:37.115734+0000","last_became_active":"2024-03-18T20:34:38.342774+0000","last_became_peered":"2024-03-18T20:34:38.342774+0000","last_unstale":"2024-03-18T20:35:37.115734+0000","last_undegraded":"2024-03-18T20:35:37.115734+0000","last_fullsized":"2024-03-18T20:35:37.115734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:52.283033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111085+0000","last_change":"2024-03-18T20:34:38.344539+0000","last_active":"2024-03-18T20:35:37.111085+0000","last_peered":"2024-03-18T20:35:37.111085+0000","last_clean":"2024-03-18T20:35:37.111085+0000","last_became_active":"2024-03-18T20:34:38.344188+0000","last_became_peered":"2024-03-18T20:34:38.344188+0000","last_unstale":"2024-03-18T20:35:37.111085+0000","last_undegraded":"2024-03-18T20:35:37.111085+0000","last_fullsized":"2024-03-18T20:35:37.111085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:38.641928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110493+0000","last_change":"2024-03-18T20:34:38.806602+0000","last_active":"2024-03-18T20:35:37.110493+0000","last_peered":"2024-03-18T20:35:37.110493+0000","last_clean":"2024-03-18T20:35:37.110493+0000","last_became_active":"2024-03-18T20:34:38.806241+0000","last_became_peered":"2024-03-18T20:34:38.806241+0000","last_unstale":"2024-03-18T20:35:37.110493+0000","last_undegraded":"2024-03-18T20:35:37.110493+0000","last_fullsized":"2024-03-18T20:35:37.110493+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:04.463311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'9","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.118511+0000","last_change":"2024-03-18T20:34:38.339695+0000","last_active":"2024-03-18T20:35:37.118511+0000","last_peered":"2024-03-18T20:35:37.118511+0000","last_clean":"2024-03-18T20:35:37.118511+0000","last_became_active":"2024-03-18T20:34:38.339331+0000","last_became_peered":"2024-03-18T20:34:38.339331+0000","last_unstale":"2024-03-18T20:35:37.118511+0000","last_undegraded":"2024-03-18T20:35:37.118511+0000","last_fullsized":"2024-03-18T20:35:37.118511+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:57:06.434289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120777+0000","last_change":"2024-03-18T20:34:38.809754+0000","last_active":"2024-03-18T20:35:37.120777+0000","last_peered":"2024-03-18T20:35:37.120777+0000","last_clean":"2024-03-18T20:35:37.120777+0000","last_became_active":"2024-03-18T20:34:38.809415+0000","last_became_peered":"2024-03-18T20:34:38.809415+0000","last_unstale":"2024-03-18T20:35:37.120777+0000","last_undegraded":"2024-03-18T20:35:37.120777+0000","last_fullsized":"2024-03-18T20:35:37.120777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:34:30.761582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117134+0000","last_change":"2024-03-18T20:34:38.342080+0000","last_active":"2024-03-18T20:35:37.117134+0000","last_peered":"2024-03-18T20:35:37.117134+0000","last_clean":"2024-03-18T20:35:37.117134+0000","last_became_active":"2024-03-18T20:34:38.340623+0000","last_became_peered":"2024-03-18T20:34:38.340623+0000","last_unstale":"2024-03-18T20:35:37.117134+0000","last_undegraded":"2024-03-18T20:35:37.117134+0000","last_fullsized":"2024-03-18T20:35:37.117134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:49:14.912391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117644+0000","last_change":"2024-03-18T20:34:38.806558+0000","last_active":"2024-03-18T20:35:37.117644+0000","last_peered":"2024-03-18T20:35:37.117644+0000","last_clean":"2024-03-18T20:35:37.117644+0000","last_became_active":"2024-03-18T20:34:38.806171+0000","last_became_peered":"2024-03-18T20:34:38.806171+0000","last_unstale":"2024-03-18T20:35:37.117644+0000","last_undegraded":"2024-03-18T20:35:37.117644+0000","last_fullsized":"2024-03-18T20:35:37.117644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:46:26.530157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110601+0000","last_change":"2024-03-18T20:34:38.336175+0000","last_active":"2024-03-18T20:35:37.110601+0000","last_peered":"2024-03-18T20:35:37.110601+0000","last_clean":"2024-03-18T20:35:37.110601+0000","last_became_active":"2024-03-18T20:34:38.335798+0000","last_became_peered":"2024-03-18T20:34:38.335798+0000","last_unstale":"2024-03-18T20:35:37.110601+0000","last_undegraded":"2024-03-18T20:35:37.110601+0000","last_fullsized":"2024-03-18T20:35:37.110601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:33:33.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110847+0000","last_change":"2024-03-18T20:34:38.807684+0000","last_active":"2024-03-18T20:35:37.110847+0000","last_peered":"2024-03-18T20:35:37.110847+0000","last_clean":"2024-03-18T20:35:37.110847+0000","last_became_active":"2024-03-18T20:34:38.807300+0000","last_became_peered":"2024-03-18T20:34:38.807300+0000","last_unstale":"2024-03-18T20:35:37.110847+0000","last_undegraded":"2024-03-18T20:35:37.110847+0000","last_fullsized":"2024-03-18T20:35:37.110847+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:31:43.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120021+0000","last_change":"2024-03-18T20:34:38.341802+0000","last_active":"2024-03-18T20:35:37.120021+0000","last_peered":"2024-03-18T20:35:37.120021+0000","last_clean":"2024-03-18T20:35:37.120021+0000","last_became_active":"2024-03-18T20:34:38.340594+0000","last_became_peered":"2024-03-18T20:34:38.340594+0000","last_unstale":"2024-03-18T20:35:37.120021+0000","last_undegraded":"2024-03-18T20:35:37.120021+0000","last_fullsized":"2024-03-18T20:35:37.120021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:01:06.830323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116079+0000","last_change":"2024-03-18T20:34:38.809790+0000","last_active":"2024-03-18T20:35:37.116079+0000","last_peered":"2024-03-18T20:35:37.116079+0000","last_clean":"2024-03-18T20:35:37.116079+0000","last_became_active":"2024-03-18T20:34:38.809470+0000","last_became_peered":"2024-03-18T20:34:38.809470+0000","last_unstale":"2024-03-18T20:35:37.116079+0000","last_undegraded":"2024-03-18T20:35:37.116079+0000","last_fullsized":"2024-03-18T20:35:37.116079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:48.225759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.123225+0000","last_change":"2024-03-18T20:35:38.122793+0000","last_active":"2024-03-18T20:35:38.123225+0000","last_peered":"2024-03-18T20:35:38.123225+0000","last_clean":"2024-03-18T20:35:38.123225+0000","last_became_active":"2024-03-18T20:35:38.118680+0000","last_became_peered":"2024-03-18T20:35:38.118680+0000","last_unstale":"2024-03-18T20:35:38.123225+0000","last_undegraded":"2024-03-18T20:35:38.123225+0000","last_fullsized":"2024-03-18T20:35:38.123225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:04:06.361764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119251+0000","last_change":"2024-03-18T20:34:38.809294+0000","last_active":"2024-03-18T20:35:37.119251+0000","last_peered":"2024-03-18T20:35:37.119251+0000","last_clean":"2024-03-18T20:35:37.119251+0000","last_became_active":"2024-03-18T20:34:38.808542+0000","last_became_peered":"2024-03-18T20:34:38.808542+0000","last_unstale":"2024-03-18T20:35:37.119251+0000","last_undegraded":"2024-03-18T20:35:37.119251+0000","last_fullsized":"2024-03-18T20:35:37.119251+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:11:30.669317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110948+0000","last_change":"2024-03-18T20:34:38.339454+0000","last_active":"2024-03-18T20:35:37.110948+0000","last_peered":"2024-03-18T20:35:37.110948+0000","last_clean":"2024-03-18T20:35:37.110948+0000","last_became_active":"2024-03-18T20:34:38.339154+0000","last_became_peered":"2024-03-18T20:34:38.339154+0000","last_unstale":"2024-03-18T20:35:37.110948+0000","last_undegraded":"2024-03-18T20:35:37.110948+0000","last_fullsized":"2024-03-18T20:35:37.110948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:12:06.300600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116002+0000","last_change":"2024-03-18T20:34:38.818462+0000","last_active":"2024-03-18T20:35:37.116002+0000","last_peered":"2024-03-18T20:35:37.116002+0000","last_clean":"2024-03-18T20:35:37.116002+0000","last_became_active":"2024-03-18T20:34:38.817522+0000","last_became_peered":"2024-03-18T20:34:38.817522+0000","last_unstale":"2024-03-18T20:35:37.116002+0000","last_undegraded":"2024-03-18T20:35:37.116002+0000","last_fullsized":"2024-03-18T20:35:37.116002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:50:32.258583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'5","reported_seq":89,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:50.890976+0000","last_change":"2024-03-18T20:34:38.341054+0000","last_active":"2024-03-18T20:35:50.890976+0000","last_peered":"2024-03-18T20:35:50.890976+0000","last_clean":"2024-03-18T20:35:50.890976+0000","last_became_active":"2024-03-18T20:34:38.340352+0000","last_became_peered":"2024-03-18T20:34:38.340352+0000","last_unstale":"2024-03-18T20:35:50.890976+0000","last_undegraded":"2024-03-18T20:35:50.890976+0000","last_fullsized":"2024-03-18T20:35:50.890976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:44.850244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115977+0000","last_change":"2024-03-18T20:34:38.818176+0000","last_active":"2024-03-18T20:35:37.115977+0000","last_peered":"2024-03-18T20:35:37.115977+0000","last_clean":"2024-03-18T20:35:37.115977+0000","last_became_active":"2024-03-18T20:34:38.817820+0000","last_became_peered":"2024-03-18T20:34:38.817820+0000","last_unstale":"2024-03-18T20:35:37.115977+0000","last_undegraded":"2024-03-18T20:35:37.115977+0000","last_fullsized":"2024-03-18T20:35:37.115977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:19:29.815174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113678+0000","last_change":"2024-03-18T20:34:38.346591+0000","last_active":"2024-03-18T20:35:37.113678+0000","last_peered":"2024-03-18T20:35:37.113678+0000","last_clean":"2024-03-18T20:35:37.113678+0000","last_became_active":"2024-03-18T20:34:38.345897+0000","last_became_peered":"2024-03-18T20:34:38.345897+0000","last_unstale":"2024-03-18T20:35:37.113678+0000","last_undegraded":"2024-03-18T20:35:37.113678+0000","last_fullsized":"2024-03-18T20:35:37.113678+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:47:32.118905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.122016+0000","last_change":"2024-03-18T20:35:38.122016+0000","last_active":"2024-03-18T20:35:38.122016+0000","last_peered":"2024-03-18T20:35:38.122016+0000","last_clean":"2024-03-18T20:35:38.122016+0000","last_became_active":"2024-03-18T20:35:38.118053+0000","last_became_peered":"2024-03-18T20:35:38.118053+0000","last_unstale":"2024-03-18T20:35:38.122016+0000","last_undegraded":"2024-03-18T20:35:38.122016+0000","last_fullsized":"2024-03-18T20:35:38.122016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:25:28.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110701+0000","last_change":"2024-03-18T20:34:38.344615+0000","last_active":"2024-03-18T20:35:37.110701+0000","last_peered":"2024-03-18T20:35:37.110701+0000","last_clean":"2024-03-18T20:35:37.110701+0000","last_became_active":"2024-03-18T20:34:38.344242+0000","last_became_peered":"2024-03-18T20:34:38.344242+0000","last_unstale":"2024-03-18T20:35:37.110701+0000","last_undegraded":"2024-03-18T20:35:37.110701+0000","last_fullsized":"2024-03-18T20:35:37.110701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.449808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116268+0000","last_change":"2024-03-18T20:34:38.820875+0000","last_active":"2024-03-18T20:35:37.116268+0000","last_peered":"2024-03-18T20:35:37.116268+0000","last_clean":"2024-03-18T20:35:37.116268+0000","last_became_active":"2024-03-18T20:34:38.820550+0000","last_became_peered":"2024-03-18T20:34:38.820550+0000","last_unstale":"2024-03-18T20:35:37.116268+0000","last_undegraded":"2024-03-18T20:35:37.116268+0000","last_fullsized":"2024-03-18T20:35:37.116268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:49:36.358734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116141+0000","last_change":"2024-03-18T20:34:38.812552+0000","last_active":"2024-03-18T20:35:37.116141+0000","last_peered":"2024-03-18T20:35:37.116141+0000","last_clean":"2024-03-18T20:35:37.116141+0000","last_became_active":"2024-03-18T20:34:38.811939+0000","last_became_peered":"2024-03-18T20:34:38.811939+0000","last_unstale":"2024-03-18T20:35:37.116141+0000","last_undegraded":"2024-03-18T20:35:37.116141+0000","last_fullsized":"2024-03-18T20:35:37.116141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:41:52.096974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117526+0000","last_change":"2024-03-18T20:34:38.815691+0000","last_active":"2024-03-18T20:35:37.117526+0000","last_peered":"2024-03-18T20:35:37.117526+0000","last_clean":"2024-03-18T20:35:37.117526+0000","last_became_active":"2024-03-18T20:34:38.815370+0000","last_became_peered":"2024-03-18T20:34:38.815370+0000","last_unstale":"2024-03-18T20:35:37.117526+0000","last_undegraded":"2024-03-18T20:35:37.117526+0000","last_fullsized":"2024-03-18T20:35:37.117526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:59:41.260013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117452+0000","last_change":"2024-03-18T20:34:38.812651+0000","last_active":"2024-03-18T20:35:37.117452+0000","last_peered":"2024-03-18T20:35:37.117452+0000","last_clean":"2024-03-18T20:35:37.117452+0000","last_became_active":"2024-03-18T20:34:38.812322+0000","last_became_peered":"2024-03-18T20:34:38.812322+0000","last_unstale":"2024-03-18T20:35:37.117452+0000","last_undegraded":"2024-03-18T20:35:37.117452+0000","last_fullsized":"2024-03-18T20:35:37.117452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:34:55.658371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117275+0000","last_change":"2024-03-18T20:34:38.806475+0000","last_active":"2024-03-18T20:35:37.117275+0000","last_peered":"2024-03-18T20:35:37.117275+0000","last_clean":"2024-03-18T20:35:37.117275+0000","last_became_active":"2024-03-18T20:34:38.806027+0000","last_became_peered":"2024-03-18T20:34:38.806027+0000","last_unstale":"2024-03-18T20:35:37.117275+0000","last_undegraded":"2024-03-18T20:35:37.117275+0000","last_fullsized":"2024-03-18T20:35:37.117275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:57:17.140031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110573+0000","last_change":"2024-03-18T20:34:38.811057+0000","last_active":"2024-03-18T20:35:37.110573+0000","last_peered":"2024-03-18T20:35:37.110573+0000","last_clean":"2024-03-18T20:35:37.110573+0000","last_became_active":"2024-03-18T20:34:38.810316+0000","last_became_peered":"2024-03-18T20:34:38.810316+0000","last_unstale":"2024-03-18T20:35:37.110573+0000","last_undegraded":"2024-03-18T20:35:37.110573+0000","last_fullsized":"2024-03-18T20:35:37.110573+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:29:22.426717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116579+0000","last_change":"2024-03-18T20:34:38.812952+0000","last_active":"2024-03-18T20:35:37.116579+0000","last_peered":"2024-03-18T20:35:37.116579+0000","last_clean":"2024-03-18T20:35:37.116579+0000","last_became_active":"2024-03-18T20:34:38.812550+0000","last_became_peered":"2024-03-18T20:34:38.812550+0000","last_unstale":"2024-03-18T20:35:37.116579+0000","last_undegraded":"2024-03-18T20:35:37.116579+0000","last_fullsized":"2024-03-18T20:35:37.116579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:06:26.525078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117914+0000","last_change":"2024-03-18T20:34:38.817358+0000","last_active":"2024-03-18T20:35:37.117914+0000","last_peered":"2024-03-18T20:35:37.117914+0000","last_clean":"2024-03-18T20:35:37.117914+0000","last_became_active":"2024-03-18T20:34:38.816939+0000","last_became_peered":"2024-03-18T20:34:38.816939+0000","last_unstale":"2024-03-18T20:35:37.117914+0000","last_undegraded":"2024-03-18T20:35:37.117914+0000","last_fullsized":"2024-03-18T20:35:37.117914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:33.258162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120687+0000","last_change":"2024-03-18T20:34:38.807798+0000","last_active":"2024-03-18T20:35:37.120687+0000","last_peered":"2024-03-18T20:35:37.120687+0000","last_clean":"2024-03-18T20:35:37.120687+0000","last_became_active":"2024-03-18T20:34:38.807489+0000","last_became_peered":"2024-03-18T20:34:38.807489+0000","last_unstale":"2024-03-18T20:35:37.120687+0000","last_undegraded":"2024-03-18T20:35:37.120687+0000","last_fullsized":"2024-03-18T20:35:37.120687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:15.395763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.114222+0000","last_change":"2024-03-18T20:34:38.806363+0000","last_active":"2024-03-18T20:35:37.114222+0000","last_peered":"2024-03-18T20:35:37.114222+0000","last_clean":"2024-03-18T20:35:37.114222+0000","last_became_active":"2024-03-18T20:34:38.805707+0000","last_became_peered":"2024-03-18T20:34:38.805707+0000","last_unstale":"2024-03-18T20:35:37.114222+0000","last_undegraded":"2024-03-18T20:35:37.114222+0000","last_fullsized":"2024-03-18T20:35:37.114222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:10.918979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119927+0000","last_change":"2024-03-18T20:34:38.814439+0000","last_active":"2024-03-18T20:35:37.119927+0000","last_peered":"2024-03-18T20:35:37.119927+0000","last_clean":"2024-03-18T20:35:37.119927+0000","last_became_active":"2024-03-18T20:34:38.813988+0000","last_became_peered":"2024-03-18T20:34:38.813988+0000","last_unstale":"2024-03-18T20:35:37.119927+0000","last_undegraded":"2024-03-18T20:35:37.119927+0000","last_fullsized":"2024-03-18T20:35:37.119927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:03:14.639044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110698+0000","last_change":"2024-03-18T20:34:38.818550+0000","last_active":"2024-03-18T20:35:37.110698+0000","last_peered":"2024-03-18T20:35:37.110698+0000","last_clean":"2024-03-18T20:35:37.110698+0000","last_became_active":"2024-03-18T20:34:38.818209+0000","last_became_peered":"2024-03-18T20:34:38.818209+0000","last_unstale":"2024-03-18T20:35:37.110698+0000","last_undegraded":"2024-03-18T20:35:37.110698+0000","last_fullsized":"2024-03-18T20:35:37.110698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:40:48.395331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110529+0000","last_change":"2024-03-18T20:34:38.808439+0000","last_active":"2024-03-18T20:35:37.110529+0000","last_peered":"2024-03-18T20:35:37.110529+0000","last_clean":"2024-03-18T20:35:37.110529+0000","last_became_active":"2024-03-18T20:34:38.807995+0000","last_became_peered":"2024-03-18T20:34:38.807995+0000","last_unstale":"2024-03-18T20:35:37.110529+0000","last_undegraded":"2024-03-18T20:35:37.110529+0000","last_fullsized":"2024-03-18T20:35:37.110529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:54.414148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116358+0000","last_change":"2024-03-18T20:34:38.810793+0000","last_active":"2024-03-18T20:35:37.116358+0000","last_peered":"2024-03-18T20:35:37.116358+0000","last_clean":"2024-03-18T20:35:37.116358+0000","last_became_active":"2024-03-18T20:34:38.810496+0000","last_became_peered":"2024-03-18T20:34:38.810496+0000","last_unstale":"2024-03-18T20:35:37.116358+0000","last_undegraded":"2024-03-18T20:35:37.116358+0000","last_fullsized":"2024-03-18T20:35:37.116358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:33.780675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66668,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":716,"num_read_kb":431,"num_write":233,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":204318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612632,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":422867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.379},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67100000000000004}]},{"osd":2,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":4,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.95799999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.61299999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59899999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.0309999999999999}]},{"osd":8,"last update":"Mon Mar 18 20:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.47999999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.76100000000000001}]},{"osd":10,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.56999999999999995}]}]},{"osd":10,"up_from":66,"seq":283467841564,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1572864,"data_stored":1073667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.4699999999999998,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":6.6449999999999996,"5min":6.6449999999999996,"15min":6.6449999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.376}]},{"osd":2,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56999999999999995}]},{"osd":3,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.50600000000000001}]},{"osd":4,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57899999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.66000000000000003}]},{"osd":6,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.67400000000000004}]},{"osd":9,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.69799999999999995}]},{"osd":11,"last update":"Mon Mar 18 20:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.624},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.70799999999999996}]}]},{"osd":9,"up_from":61,"seq":261993005088,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":422940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55200000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57999999999999996,"15min":0.59199999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.878,"15min":0.878},"last":0.435}]},{"osd":1,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77000000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.67500000000000004,"15min":0.68400000000000005},"min":{"1min":0.443,"5min":0.442,"15min":0.442},"max":{"1min":0.88800000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.74399999999999999}]},{"osd":2,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.46100000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83099999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.46600000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.59999999999999998}]},{"osd":3,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.51600000000000001,"15min":0.52400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69299999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.34699999999999998}]},{"osd":4,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64700000000000002,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81100000000000005,"5min":1.016,"15min":1.016},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.73799999999999999,"15min":0.751},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93300000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.69399999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83199999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86299999999999999,"5min":1.008,"15min":1.008},"last":0.76500000000000001}]},{"osd":6,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73599999999999999,"5min":1.179,"15min":1.179},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.70599999999999996,"15min":0.73099999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77100000000000002,"5min":1.151,"15min":1.151},"last":0.46999999999999997}]},{"osd":7,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.80800000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0049999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.73599999999999999,"15min":0.748},"min":{"1min":0.51200000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.85799999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.65500000000000003}]},{"osd":8,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.80700000000000005,"15min":0.81699999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96899999999999997,"5min":1.353,"15min":1.353},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.78700000000000003,"15min":0.79900000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.90300000000000002,"5min":1.103,"15min":1.103},"last":0.61899999999999999}]},{"osd":10,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.49199999999999999}]}]},{"osd":8,"up_from":54,"seq":231928234020,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":423179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.38500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.622,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47099999999999997}]},{"osd":1,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73399999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42299999999999999,"15min":0.41399999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.32600000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.38600000000000001}]},{"osd":3,"last update":"Mon Mar 18 20:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.622}]},{"osd":4,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.749,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.51100000000000001,"15min":0.498},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.495},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.34699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40600000000000003}]},{"osd":6,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.66200000000000003,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.75600000000000001}]},{"osd":7,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.874,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66000000000000003}]},{"osd":9,"last update":"Mon Mar 18 20:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.51700000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.70899999999999996,"15min":0.69399999999999995},"min":{"1min":0.63900000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79900000000000004}]},{"osd":10,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.65900000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.72499999999999998,"15min":0.70699999999999996},"min":{"1min":0.76500000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95299999999999996,"5min":1.018,"15min":1.018},"last":0.82299999999999995}]},{"osd":11,"last update":"Mon Mar 18 20:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.78500000000000003}]}]},{"osd":7,"up_from":47,"seq":201863462950,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1495040,"data_stored":1015576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.439}]},{"osd":1,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.42199999999999999,"15min":0.41399999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.378}]},{"osd":2,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73799999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.65900000000000003}]},{"osd":3,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.72999999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.44500000000000001}]},{"osd":5,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.40500000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.497},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.61899999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89700000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60199999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.71299999999999997}]},{"osd":9,"last update":"Mon Mar 18 20:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.59299999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94099999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.61299999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.72799999999999998}]},{"osd":10,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.69599999999999995}]},{"osd":11,"last update":"Mon Mar 18 20:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.748}]}]},{"osd":6,"up_from":42,"seq":180388626474,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":422492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.437,"15min":0.42699999999999999},"min":{"1min":0.35399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.44900000000000001,"15min":0.44500000000000001},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.625,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.441}]},{"osd":1,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75800000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.505,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59799999999999998}]},{"osd":2,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.46800000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63}]},{"osd":3,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.874,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.54800000000000004}]},{"osd":4,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60599999999999998,"15min":0.58999999999999997},"min":{"1min":0.498,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77200000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.45600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.56399999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.54500000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95999999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.94699999999999995,"5min":1.052,"15min":1.052},"last":0.58099999999999996}]},{"osd":7,"last update":"Mon Mar 18 20:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.89000000000000001,"5min":0.995,"15min":0.995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58699999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.97199999999999998,"5min":1.006,"15min":1.006},"last":0.81399999999999995}]},{"osd":8,"last update":"Mon Mar 18 20:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.47399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92000000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76600000000000001,"15min":0.76500000000000001},"min":{"1min":0.60999999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.77000000000000002}]},{"osd":9,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.81799999999999995,"15min":0.83199999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.83499999999999996,"5min":1.028,"15min":1.028},"last":0.61899999999999999}]},{"osd":10,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84699999999999998,"15min":0.85299999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.89700000000000002,"5min":1.081,"15min":1.081},"last":0.80000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":483137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.59299999999999997,"15min":0.54800000000000004},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.46400000000000002}]},{"osd":2,"last update":"Mon Mar 18 20:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.56999999999999995,"15min":0.54400000000000004},"min":{"1min":0.497,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.38600000000000001}]},{"osd":3,"last update":"Mon Mar 18 20:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.497,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.497}]},{"osd":4,"last update":"Mon Mar 18 20:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.443,"15min":0.40999999999999998},"min":{"1min":0.34499999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.70099999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.45000000000000001,"15min":0.42099999999999999},"min":{"1min":0.32400000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.58399999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.64400000000000002,"15min":0.623},"min":{"1min":0.52700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.86099999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.67900000000000005,"15min":0.623},"min":{"1min":0.52200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":4.5679999999999996,"5min":4.5679999999999996,"15min":4.5679999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.64700000000000002,"15min":0.60299999999999998},"min":{"1min":0.44,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.5330000000000004,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.54100000000000004}]},{"osd":7,"last update":"Mon Mar 18 20:34:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84399999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42699999999999999}]},{"osd":8,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63600000000000001,"15min":0.627},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.55300000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98899999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.629}]},{"osd":9,"last update":"Mon Mar 18 20:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.745,"15min":0.71899999999999997},"min":{"1min":0.69099999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70199999999999996,"15min":0.68600000000000005},"min":{"1min":0.61199999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.65600000000000003}]},{"osd":10,"last update":"Mon Mar 18 20:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.58199999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.70599999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.70799999999999996}]},{"osd":11,"last update":"Mon Mar 18 20:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73199999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":423831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Mar 18 20:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53500000000000003,"15min":0.51100000000000001},"min":{"1min":0.34599999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53600000000000003,"15min":0.52300000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79700000000000004,"5min":1.024,"15min":1.024},"last":0.51300000000000001}]},{"osd":2,"last update":"Mon Mar 18 20:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57799999999999996,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63200000000000001,"15min":0.63100000000000001},"min":{"1min":0.42799999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1100000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.46400000000000002}]},{"osd":3,"last update":"Mon Mar 18 20:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63800000000000001}]},{"osd":4,"last update":"Mon Mar 18 20:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.629},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89900000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57399999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.48499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.623,"15min":0.60599999999999998},"min":{"1min":0.47599999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85099999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.34699999999999998}]},{"osd":6,"last update":"Mon Mar 18 20:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.311},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.55100000000000005}]},{"osd":7,"last update":"Mon Mar 18 20:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.65100000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67400000000000004,"15min":0.67700000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.92200000000000004,"5min":1.002,"15min":1.002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.745,"15min":0.747},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93400000000000005,"5min":0.998,"15min":0.998},"last":0.58799999999999997}]},{"osd":9,"last update":"Mon Mar 18 20:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.44600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.439},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90600000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67300000000000004}]},{"osd":10,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.004,"5min":1.105,"15min":1.105},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.61499999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.016,"5min":1.177,"15min":1.177},"last":0.62}]},{"osd":11,"last update":"Mon Mar 18 20:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":482954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.58599999999999997,"15min":0.497},"min":{"1min":0.379,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":7.8559999999999999,"5min":7.8559999999999999,"15min":7.8559999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.58299999999999996,"15min":0.496},"min":{"1min":0.42799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.7880000000000003,"5min":7.7880000000000003,"15min":7.7880000000000003},"last":0.39300000000000002}]},{"osd":1,"last update":"Mon Mar 18 20:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58399999999999996,"15min":0.53700000000000003},"min":{"1min":0.48399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81499999999999995,"5min":4.6100000000000003,"15min":4.6100000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.53900000000000003}]},{"osd":3,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57099999999999995,"15min":0.52900000000000003},"min":{"1min":0.40000000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.497},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57799999999999996,"15min":0.53800000000000003},"min":{"1min":0.46700000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46700000000000003}]},{"osd":4,"last update":"Mon Mar 18 20:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.63},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Mar 18 20:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.47099999999999997,"15min":0.44700000000000001},"min":{"1min":0.39000000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.31},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.54900000000000004,"15min":0.53700000000000003},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Mon Mar 18 20:35:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.66300000000000003,"15min":0.65600000000000003},"min":{"1min":0.54500000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":0.75,"15min":0.68500000000000005},"min":{"1min":0.45300000000000001,"5min":0.436,"15min":0.436},"max":{"1min":9.9969999999999999,"5min":9.9969999999999999,"15min":9.9969999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Mon Mar 18 20:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.439},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45800000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80800000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.438},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84699999999999998,"5min":1.038,"15min":1.038},"last":0.64000000000000001}]},{"osd":9,"last update":"Mon Mar 18 20:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.53900000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93100000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.274,"5min":0.83299999999999996,"15min":0.75900000000000001},"min":{"1min":0.59999999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":9.8780000000000001,"5min":9.8780000000000001,"15min":9.8780000000000001},"last":0.65000000000000002}]},{"osd":10,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.89800000000000002,"15min":0.92000000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0589999999999999,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.751,"15min":0.745},"min":{"1min":0.60799999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97799999999999998,"5min":1.044,"15min":1.044},"last":0.60799999999999998}]},{"osd":11,"last update":"Mon Mar 18 20:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66900000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215156,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":422248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.41099999999999998},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.307},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.442,"15min":0.41999999999999998},"min":{"1min":0.33400000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.40699999999999997}]},{"osd":1,"last update":"Mon Mar 18 20:35:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.36799999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.052,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.41299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Mon Mar 18 20:35:52 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.372,"15min":0.372},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.46100000000000002}]},{"osd":4,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90700000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.625},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67800000000000005,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.876,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.69299999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72899999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84399999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52500000000000002}]},{"osd":6,"last update":"Mon Mar 18 20:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.376,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.62},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.755,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59799999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.90500000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.61299999999999999}]},{"osd":8,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73099999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.87,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55900000000000005}]},{"osd":10,"last update":"Mon Mar 18 20:35:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.77700000000000002,"15min":0.78600000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.97899999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72099999999999997}]},{"osd":11,"last update":"Mon Mar 18 20:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57499999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":422745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.38400000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.371,"15min":0.36099999999999999},"min":{"1min":0.31,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.435,"5min":0.374,"15min":0.35999999999999999},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.63,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.432}]},{"osd":2,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55100000000000005,"15min":0.52600000000000002},"min":{"1min":0.45600000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72199999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.62}]},{"osd":3,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.45400000000000001}]},{"osd":5,"last update":"Mon Mar 18 20:35:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.46100000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82299999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81200000000000006,"5min":0.875,"15min":0.875},"last":0.59199999999999997}]},{"osd":6,"last update":"Mon Mar 18 20:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.63200000000000001,"15min":0.58399999999999996},"min":{"1min":0.48499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":5.0460000000000003,"5min":5.0460000000000003,"15min":5.0460000000000003},"last":0.85099999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.621,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.88900000000000001}]},{"osd":8,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.55300000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.67500000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.873}]},{"osd":9,"last update":"Mon Mar 18 20:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63900000000000001,"15min":0.63},"min":{"1min":0.57499999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.65400000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82099999999999995}]},{"osd":10,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.49399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90500000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66500000000000004,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.86099999999999999}]},{"osd":11,"last update":"Mon Mar 18 20:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.72999999999999998}]}]},{"osd":5,"up_from":35,"seq":150323855406,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1503232,"data_stored":1014941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53300000000000003,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.745,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.61299999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.53400000000000003},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57199999999999995}]},{"osd":2,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.39500000000000002,"15min":0.38},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.435},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.40200000000000002,"15min":0.38400000000000001},"min":{"1min":0.317,"5min":0.25,"15min":0.25},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38400000000000001}]},{"osd":3,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.746,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.372,"15min":0.372},"max":{"1min":0.69299999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53300000000000003}]},{"osd":4,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.53100000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.41799999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Mon Mar 18 20:35:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.496,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.68400000000000005}]},{"osd":7,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.72199999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80400000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.626}]},{"osd":9,"last update":"Mon Mar 18 20:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.60599999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.73699999999999999,"15min":0.72499999999999998},"min":{"1min":0.625,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.73799999999999999}]},{"osd":10,"last update":"Mon Mar 18 20:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.57899999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.97599999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.56899999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T20:35:55.558 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-18T20:35:55.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T20:35:55.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.728+0000 7f7a942b1700 1 -- 172.21.15.46:0/878991086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a8c0ff660 msgr2=0x7f7a8c109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:55.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.728+0000 7f7a942b1700 1 --2- 172.21.15.46:0/878991086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a8c0ff660 0x7f7a8c109230 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400b0d0 tx=0x7f7a8400b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.728+0000 7f7a942b1700 1 -- 172.21.15.46:0/878991086 shutdown_connections 2024-03-18T20:35:55.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.728+0000 7f7a942b1700 1 --2- 172.21.15.46:0/878991086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a8c0ff660 0x7f7a8c109230 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.728+0000 7f7a942b1700 1 --2- 172.21.15.46:0/878991086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a8c102250 0x7f7a8c102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.728+0000 7f7a942b1700 1 --2- 172.21.15.46:0/878991086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a8c0fecc0 0x7f7a8c0ff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.728+0000 7f7a942b1700 1 -- 172.21.15.46:0/878991086 >> 172.21.15.46:0/878991086 conn(0x7f7a8c0fa880 msgr2=0x7f7a8c0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:55.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.728+0000 7f7a942b1700 1 -- 172.21.15.46:0/878991086 shutdown_connections 2024-03-18T20:35:55.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.728+0000 7f7a942b1700 1 -- 172.21.15.46:0/878991086 wait complete. 2024-03-18T20:35:55.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a942b1700 1 Processor -- start 2024-03-18T20:35:55.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a942b1700 1 -- start start 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a942b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a8c0fecc0 0x7f7a8c196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a942b1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a8c0ff660 0x7f7a8c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a942b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a8c102250 0x7f7a8c19e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a942b1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c19e7c0 con 0x7f7a8c102250 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a942b1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c19e930 con 0x7f7a8c0fecc0 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a942b1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c19eaf0 con 0x7f7a8c0ff660 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a9284e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a8c102250 0x7f7a8c19e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a9284e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a8c102250 0x7f7a8c19e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38648/0 (socket says 172.21.15.46:38648) 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a9284e700 1 -- 172.21.15.46:0/354587465 learned_addr learned my addr 172.21.15.46:0/354587465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a9204d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a8c0fecc0 0x7f7a8c196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.729+0000 7f7a9184c700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a8c0ff660 0x7f7a8c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.730+0000 7f7a9284e700 1 -- 172.21.15.46:0/354587465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a8c0ff660 msgr2=0x7f7a8c197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.730+0000 7f7a9284e700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a8c0ff660 0x7f7a8c197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.730+0000 7f7a9284e700 1 -- 172.21.15.46:0/354587465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a8c0fecc0 msgr2=0x7f7a8c196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.730+0000 7f7a9284e700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a8c0fecc0 0x7f7a8c196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.730+0000 7f7a9284e700 1 -- 172.21.15.46:0/354587465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a84009d00 con 0x7f7a8c102250 2024-03-18T20:35:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.730+0000 7f7a9284e700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a8c102250 0x7f7a8c19e1f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007b60 tx=0x7f7a84008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.730+0000 7f7a837fe700 1 -- 172.21.15.46:0/354587465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8400e040 con 0x7f7a8c102250 2024-03-18T20:35:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.730+0000 7f7a837fe700 1 -- 172.21.15.46:0/354587465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a8401d930 con 0x7f7a8c102250 2024-03-18T20:35:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.730+0000 7f7a942b1700 1 -- 172.21.15.46:0/354587465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8c0ffdd0 con 0x7f7a8c102250 2024-03-18T20:35:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.730+0000 7f7a837fe700 1 -- 172.21.15.46:0/354587465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8401ba00 con 0x7f7a8c102250 2024-03-18T20:35:55.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.730+0000 7f7a942b1700 1 -- 172.21.15.46:0/354587465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8c1002e0 con 0x7f7a8c102250 2024-03-18T20:35:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.731+0000 7f7a942b1700 1 -- 172.21.15.46:0/354587465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8c108220 con 0x7f7a8c102250 2024-03-18T20:35:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.732+0000 7f7a837fe700 1 -- 172.21.15.46:0/354587465 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f7a8401daa0 con 0x7f7a8c102250 2024-03-18T20:35:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.733+0000 7f7a837fe700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7a780a11b0 0x7f7a780a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.733+0000 7f7a9204d700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7a780a11b0 0x7f7a780a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.733+0000 7f7a837fe700 1 -- 172.21.15.46:0/354587465 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a840ce3a0 con 0x7f7a8c102250 2024-03-18T20:35:55.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.733+0000 7f7a9204d700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7a780a11b0 0x7f7a780a3670 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c198150 tx=0x7f7a88007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:55.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.736+0000 7f7a837fe700 1 -- 172.21.15.46:0/354587465 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7a84097a10 con 0x7f7a8c102250 2024-03-18T20:35:55.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.947+0000 7f7a942b1700 1 -- 172.21.15.46:0/354587465 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a8c0611f0 con 0x7f7a780a11b0 2024-03-18T20:35:55.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.958+0000 7f7a837fe700 1 -- 172.21.15.46:0/354587465 <== mgr.14152 v2:172.21.15.46:6800/3400106473 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344356 (secure 0 0 0) 0x7f7a8c0611f0 con 0x7f7a780a11b0 2024-03-18T20:35:55.965 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.962+0000 7f7a942b1700 1 -- 172.21.15.46:0/354587465 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7a780a11b0 msgr2=0x7f7a780a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.962+0000 7f7a942b1700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7a780a11b0 0x7f7a780a3670 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c198150 tx=0x7f7a88007400 comp rx=0 tx=0).stop 2024-03-18T20:35:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.962+0000 7f7a942b1700 1 -- 172.21.15.46:0/354587465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a8c102250 msgr2=0x7f7a8c19e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.962+0000 7f7a942b1700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a8c102250 0x7f7a8c19e1f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007b60 tx=0x7f7a84008f10 comp rx=0 tx=0).stop 2024-03-18T20:35:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.962+0000 7f7a942b1700 1 -- 172.21.15.46:0/354587465 shutdown_connections 2024-03-18T20:35:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.962+0000 7f7a942b1700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f7a780a11b0 0x7f7a780a3670 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.962+0000 7f7a942b1700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a8c102250 0x7f7a8c19e1f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.962+0000 7f7a942b1700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a8c0fecc0 0x7f7a8c196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.962+0000 7f7a942b1700 1 --2- 172.21.15.46:0/354587465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a8c0ff660 0x7f7a8c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.962+0000 7f7a942b1700 1 -- 172.21.15.46:0/354587465 >> 172.21.15.46:0/354587465 conn(0x7f7a8c0fa880 msgr2=0x7f7a8c105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:55.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.962+0000 7f7a942b1700 1 -- 172.21.15.46:0/354587465 shutdown_connections 2024-03-18T20:35:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:55.962+0000 7f7a942b1700 1 -- 172.21.15.46:0/354587465 wait complete. 2024-03-18T20:35:55.969 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-18T20:35:55.979 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-03-18T20:35:55.037736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657270,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":852,"num_read_kb":550,"num_write":416,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336680,"kb_used_data":12432,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605144,"statfs":{"total":1151940427776,"available":1151595667456,"internally_reserved":0,"allocated":12730368,"data_stored":7030577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19131,"internal_metadata":331855173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":628,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":52,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003020"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116835+0000","last_change":"2024-03-18T20:34:38.817028+0000","last_active":"2024-03-18T20:35:37.116835+0000","last_peered":"2024-03-18T20:35:37.116835+0000","last_clean":"2024-03-18T20:35:37.116835+0000","last_became_active":"2024-03-18T20:34:38.816711+0000","last_became_peered":"2024-03-18T20:34:38.816711+0000","last_unstale":"2024-03-18T20:35:37.116835+0000","last_undegraded":"2024-03-18T20:35:37.116835+0000","last_fullsized":"2024-03-18T20:35:37.116835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:32:16.827141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117600+0000","last_change":"2024-03-18T20:34:38.807350+0000","last_active":"2024-03-18T20:35:37.117600+0000","last_peered":"2024-03-18T20:35:37.117600+0000","last_clean":"2024-03-18T20:35:37.117600+0000","last_became_active":"2024-03-18T20:34:38.807008+0000","last_became_peered":"2024-03-18T20:34:38.807008+0000","last_unstale":"2024-03-18T20:35:37.117600+0000","last_undegraded":"2024-03-18T20:35:37.117600+0000","last_fullsized":"2024-03-18T20:35:37.117600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:35.667384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117502+0000","last_change":"2024-03-18T20:34:38.817045+0000","last_active":"2024-03-18T20:35:37.117502+0000","last_peered":"2024-03-18T20:35:37.117502+0000","last_clean":"2024-03-18T20:35:37.117502+0000","last_became_active":"2024-03-18T20:34:38.816659+0000","last_became_peered":"2024-03-18T20:34:38.816659+0000","last_unstale":"2024-03-18T20:35:37.117502+0000","last_undegraded":"2024-03-18T20:35:37.117502+0000","last_fullsized":"2024-03-18T20:35:37.117502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:04:49.316127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116820+0000","last_change":"2024-03-18T20:34:38.804833+0000","last_active":"2024-03-18T20:35:37.116820+0000","last_peered":"2024-03-18T20:35:37.116820+0000","last_clean":"2024-03-18T20:35:37.116820+0000","last_became_active":"2024-03-18T20:34:38.804486+0000","last_became_peered":"2024-03-18T20:34:38.804486+0000","last_unstale":"2024-03-18T20:35:37.116820+0000","last_undegraded":"2024-03-18T20:35:37.116820+0000","last_fullsized":"2024-03-18T20:35:37.116820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:19:44.266207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110860+0000","last_change":"2024-03-18T20:34:38.797660+0000","last_active":"2024-03-18T20:35:37.110860+0000","last_peered":"2024-03-18T20:35:37.110860+0000","last_clean":"2024-03-18T20:35:37.110860+0000","last_became_active":"2024-03-18T20:34:38.797155+0000","last_became_peered":"2024-03-18T20:34:38.797155+0000","last_unstale":"2024-03-18T20:35:37.110860+0000","last_undegraded":"2024-03-18T20:35:37.110860+0000","last_fullsized":"2024-03-18T20:35:37.110860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:22:11.363453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.118992+0000","last_change":"2024-03-18T20:35:38.118992+0000","last_active":"2024-03-18T20:35:38.118992+0000","last_peered":"2024-03-18T20:35:38.118992+0000","last_clean":"2024-03-18T20:35:38.118992+0000","last_became_active":"2024-03-18T20:35:38.118632+0000","last_became_peered":"2024-03-18T20:35:38.118632+0000","last_unstale":"2024-03-18T20:35:38.118992+0000","last_undegraded":"2024-03-18T20:35:38.118992+0000","last_fullsized":"2024-03-18T20:35:38.118992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:00:49.799078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115609+0000","last_change":"2024-03-18T20:34:38.805378+0000","last_active":"2024-03-18T20:35:37.115609+0000","last_peered":"2024-03-18T20:35:37.115609+0000","last_clean":"2024-03-18T20:35:37.115609+0000","last_became_active":"2024-03-18T20:34:38.805130+0000","last_became_peered":"2024-03-18T20:34:38.805130+0000","last_unstale":"2024-03-18T20:35:37.115609+0000","last_undegraded":"2024-03-18T20:35:37.115609+0000","last_fullsized":"2024-03-18T20:35:37.115609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:46:19.420715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.118345+0000","last_change":"2024-03-18T20:35:38.117964+0000","last_active":"2024-03-18T20:35:38.118345+0000","last_peered":"2024-03-18T20:35:38.118345+0000","last_clean":"2024-03-18T20:35:38.118345+0000","last_became_active":"2024-03-18T20:35:38.117579+0000","last_became_peered":"2024-03-18T20:35:38.117579+0000","last_unstale":"2024-03-18T20:35:38.118345+0000","last_undegraded":"2024-03-18T20:35:38.118345+0000","last_fullsized":"2024-03-18T20:35:38.118345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:44:20.292995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.234734+0000","last_change":"2024-03-18T20:35:38.234734+0000","last_active":"2024-03-18T20:35:38.234734+0000","last_peered":"2024-03-18T20:35:38.234734+0000","last_clean":"2024-03-18T20:35:38.234734+0000","last_became_active":"2024-03-18T20:35:38.234403+0000","last_became_peered":"2024-03-18T20:35:38.234403+0000","last_unstale":"2024-03-18T20:35:38.234734+0000","last_undegraded":"2024-03-18T20:35:38.234734+0000","last_fullsized":"2024-03-18T20:35:38.234734+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:13:54.696645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119706+0000","last_change":"2024-03-18T20:34:38.814728+0000","last_active":"2024-03-18T20:35:37.119706+0000","last_peered":"2024-03-18T20:35:37.119706+0000","last_clean":"2024-03-18T20:35:37.119706+0000","last_became_active":"2024-03-18T20:34:38.814200+0000","last_became_peered":"2024-03-18T20:34:38.814200+0000","last_unstale":"2024-03-18T20:35:37.119706+0000","last_undegraded":"2024-03-18T20:35:37.119706+0000","last_fullsized":"2024-03-18T20:35:37.119706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:24:54.064770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115901+0000","last_change":"2024-03-18T20:34:38.818054+0000","last_active":"2024-03-18T20:35:37.115901+0000","last_peered":"2024-03-18T20:35:37.115901+0000","last_clean":"2024-03-18T20:35:37.115901+0000","last_became_active":"2024-03-18T20:34:38.817752+0000","last_became_peered":"2024-03-18T20:34:38.817752+0000","last_unstale":"2024-03-18T20:35:37.115901+0000","last_undegraded":"2024-03-18T20:35:37.115901+0000","last_fullsized":"2024-03-18T20:35:37.115901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:01:13.728434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116729+0000","last_change":"2024-03-18T20:34:38.801356+0000","last_active":"2024-03-18T20:35:37.116729+0000","last_peered":"2024-03-18T20:35:37.116729+0000","last_clean":"2024-03-18T20:35:37.116729+0000","last_became_active":"2024-03-18T20:34:38.800978+0000","last_became_peered":"2024-03-18T20:34:38.800978+0000","last_unstale":"2024-03-18T20:35:37.116729+0000","last_undegraded":"2024-03-18T20:35:37.116729+0000","last_fullsized":"2024-03-18T20:35:37.116729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:07.634686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110534+0000","last_change":"2024-03-18T20:34:38.803006+0000","last_active":"2024-03-18T20:35:37.110534+0000","last_peered":"2024-03-18T20:35:37.110534+0000","last_clean":"2024-03-18T20:35:37.110534+0000","last_became_active":"2024-03-18T20:34:38.802454+0000","last_became_peered":"2024-03-18T20:34:38.802454+0000","last_unstale":"2024-03-18T20:35:37.110534+0000","last_undegraded":"2024-03-18T20:35:37.110534+0000","last_fullsized":"2024-03-18T20:35:37.110534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:11:02.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116047+0000","last_change":"2024-03-18T20:34:38.807397+0000","last_active":"2024-03-18T20:35:37.116047+0000","last_peered":"2024-03-18T20:35:37.116047+0000","last_clean":"2024-03-18T20:35:37.116047+0000","last_became_active":"2024-03-18T20:34:38.807072+0000","last_became_peered":"2024-03-18T20:34:38.807072+0000","last_unstale":"2024-03-18T20:35:37.116047+0000","last_undegraded":"2024-03-18T20:35:37.116047+0000","last_fullsized":"2024-03-18T20:35:37.116047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:55:53.832825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116047+0000","last_change":"2024-03-18T20:34:38.812925+0000","last_active":"2024-03-18T20:35:37.116047+0000","last_peered":"2024-03-18T20:35:37.116047+0000","last_clean":"2024-03-18T20:35:37.116047+0000","last_became_active":"2024-03-18T20:34:38.812597+0000","last_became_peered":"2024-03-18T20:34:38.812597+0000","last_unstale":"2024-03-18T20:35:37.116047+0000","last_undegraded":"2024-03-18T20:35:37.116047+0000","last_fullsized":"2024-03-18T20:35:37.116047+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:14:35.103852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110532+0000","last_change":"2024-03-18T20:34:38.817892+0000","last_active":"2024-03-18T20:35:37.110532+0000","last_peered":"2024-03-18T20:35:37.110532+0000","last_clean":"2024-03-18T20:35:37.110532+0000","last_became_active":"2024-03-18T20:34:38.816261+0000","last_became_peered":"2024-03-18T20:34:38.816261+0000","last_unstale":"2024-03-18T20:35:37.110532+0000","last_undegraded":"2024-03-18T20:35:37.110532+0000","last_fullsized":"2024-03-18T20:35:37.110532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:43:32.481210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116723+0000","last_change":"2024-03-18T20:34:38.816966+0000","last_active":"2024-03-18T20:35:37.116723+0000","last_peered":"2024-03-18T20:35:37.116723+0000","last_clean":"2024-03-18T20:35:37.116723+0000","last_became_active":"2024-03-18T20:34:38.816645+0000","last_became_peered":"2024-03-18T20:34:38.816645+0000","last_unstale":"2024-03-18T20:35:37.116723+0000","last_undegraded":"2024-03-18T20:35:37.116723+0000","last_fullsized":"2024-03-18T20:35:37.116723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:36:39.904353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.118561+0000","last_change":"2024-03-18T20:34:38.817005+0000","last_active":"2024-03-18T20:35:37.118561+0000","last_peered":"2024-03-18T20:35:37.118561+0000","last_clean":"2024-03-18T20:35:37.118561+0000","last_became_active":"2024-03-18T20:34:38.816710+0000","last_became_peered":"2024-03-18T20:34:38.816710+0000","last_unstale":"2024-03-18T20:35:37.118561+0000","last_undegraded":"2024-03-18T20:35:37.118561+0000","last_fullsized":"2024-03-18T20:35:37.118561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:30:43.483977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117763+0000","last_change":"2024-03-18T20:34:38.810957+0000","last_active":"2024-03-18T20:35:37.117763+0000","last_peered":"2024-03-18T20:35:37.117763+0000","last_clean":"2024-03-18T20:35:37.117763+0000","last_became_active":"2024-03-18T20:34:38.810615+0000","last_became_peered":"2024-03-18T20:34:38.810615+0000","last_unstale":"2024-03-18T20:35:37.117763+0000","last_undegraded":"2024-03-18T20:35:37.117763+0000","last_fullsized":"2024-03-18T20:35:37.117763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:26:36.544034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'13","reported_seq":94,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:49.909225+0000","last_change":"2024-03-18T20:34:38.337863+0000","last_active":"2024-03-18T20:35:49.909225+0000","last_peered":"2024-03-18T20:35:49.909225+0000","last_clean":"2024-03-18T20:35:49.909225+0000","last_became_active":"2024-03-18T20:34:38.337548+0000","last_became_peered":"2024-03-18T20:34:38.337548+0000","last_unstale":"2024-03-18T20:35:49.909225+0000","last_undegraded":"2024-03-18T20:35:49.909225+0000","last_fullsized":"2024-03-18T20:35:49.909225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:41:40.237882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120594+0000","last_change":"2024-03-18T20:34:38.814321+0000","last_active":"2024-03-18T20:35:37.120594+0000","last_peered":"2024-03-18T20:35:37.120594+0000","last_clean":"2024-03-18T20:35:37.120594+0000","last_became_active":"2024-03-18T20:34:38.813939+0000","last_became_peered":"2024-03-18T20:34:38.813939+0000","last_unstale":"2024-03-18T20:35:37.120594+0000","last_undegraded":"2024-03-18T20:35:37.120594+0000","last_fullsized":"2024-03-18T20:35:37.120594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:05:42.441099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120794+0000","last_change":"2024-03-18T20:34:38.347790+0000","last_active":"2024-03-18T20:35:37.120794+0000","last_peered":"2024-03-18T20:35:37.120794+0000","last_clean":"2024-03-18T20:35:37.120794+0000","last_became_active":"2024-03-18T20:34:38.347317+0000","last_became_peered":"2024-03-18T20:34:38.347317+0000","last_unstale":"2024-03-18T20:35:37.120794+0000","last_undegraded":"2024-03-18T20:35:37.120794+0000","last_fullsized":"2024-03-18T20:35:37.120794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.096480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.118730+0000","last_change":"2024-03-18T20:34:38.817116+0000","last_active":"2024-03-18T20:35:37.118730+0000","last_peered":"2024-03-18T20:35:37.118730+0000","last_clean":"2024-03-18T20:35:37.118730+0000","last_became_active":"2024-03-18T20:34:38.816761+0000","last_became_peered":"2024-03-18T20:34:38.816761+0000","last_unstale":"2024-03-18T20:35:37.118730+0000","last_undegraded":"2024-03-18T20:35:37.118730+0000","last_fullsized":"2024-03-18T20:35:37.118730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:44.353533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'12","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111161+0000","last_change":"2024-03-18T20:34:38.339236+0000","last_active":"2024-03-18T20:35:37.111161+0000","last_peered":"2024-03-18T20:35:37.111161+0000","last_clean":"2024-03-18T20:35:37.111161+0000","last_became_active":"2024-03-18T20:34:38.338879+0000","last_became_peered":"2024-03-18T20:34:38.338879+0000","last_unstale":"2024-03-18T20:35:37.111161+0000","last_undegraded":"2024-03-18T20:35:37.111161+0000","last_fullsized":"2024-03-18T20:35:37.111161+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:33.340151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111163+0000","last_change":"2024-03-18T20:34:38.817861+0000","last_active":"2024-03-18T20:35:37.111163+0000","last_peered":"2024-03-18T20:35:37.111163+0000","last_clean":"2024-03-18T20:35:37.111163+0000","last_became_active":"2024-03-18T20:34:38.815976+0000","last_became_peered":"2024-03-18T20:34:38.815976+0000","last_unstale":"2024-03-18T20:35:37.111163+0000","last_undegraded":"2024-03-18T20:35:37.111163+0000","last_fullsized":"2024-03-18T20:35:37.111163+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:54.070067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111718+0000","last_change":"2024-03-18T20:34:38.345758+0000","last_active":"2024-03-18T20:35:37.111718+0000","last_peered":"2024-03-18T20:35:37.111718+0000","last_clean":"2024-03-18T20:35:37.111718+0000","last_became_active":"2024-03-18T20:34:38.345417+0000","last_became_peered":"2024-03-18T20:34:38.345417+0000","last_unstale":"2024-03-18T20:35:37.111718+0000","last_undegraded":"2024-03-18T20:35:37.111718+0000","last_fullsized":"2024-03-18T20:35:37.111718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:25.872927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113736+0000","last_change":"2024-03-18T20:34:38.818745+0000","last_active":"2024-03-18T20:35:37.113736+0000","last_peered":"2024-03-18T20:35:37.113736+0000","last_clean":"2024-03-18T20:35:37.113736+0000","last_became_active":"2024-03-18T20:34:38.818415+0000","last_became_peered":"2024-03-18T20:34:38.818415+0000","last_unstale":"2024-03-18T20:35:37.113736+0000","last_undegraded":"2024-03-18T20:35:37.113736+0000","last_fullsized":"2024-03-18T20:35:37.113736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:45:49.454498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":93,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:52.893926+0000","last_change":"2024-03-18T20:34:38.345804+0000","last_active":"2024-03-18T20:35:52.893926+0000","last_peered":"2024-03-18T20:35:52.893926+0000","last_clean":"2024-03-18T20:35:52.893926+0000","last_became_active":"2024-03-18T20:34:38.345409+0000","last_became_peered":"2024-03-18T20:34:38.345409+0000","last_unstale":"2024-03-18T20:35:52.893926+0000","last_undegraded":"2024-03-18T20:35:52.893926+0000","last_fullsized":"2024-03-18T20:35:52.893926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:00.085412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":888,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111822+0000","last_change":"2024-03-18T20:34:38.799821+0000","last_active":"2024-03-18T20:35:37.111822+0000","last_peered":"2024-03-18T20:35:37.111822+0000","last_clean":"2024-03-18T20:35:37.111822+0000","last_became_active":"2024-03-18T20:34:38.799457+0000","last_became_peered":"2024-03-18T20:34:38.799457+0000","last_unstale":"2024-03-18T20:35:37.111822+0000","last_undegraded":"2024-03-18T20:35:37.111822+0000","last_fullsized":"2024-03-18T20:35:37.111822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:39:35.912892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'7","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120102+0000","last_change":"2024-03-18T20:34:38.341999+0000","last_active":"2024-03-18T20:35:37.120102+0000","last_peered":"2024-03-18T20:35:37.120102+0000","last_clean":"2024-03-18T20:35:37.120102+0000","last_became_active":"2024-03-18T20:34:38.341431+0000","last_became_peered":"2024-03-18T20:34:38.341431+0000","last_unstale":"2024-03-18T20:35:37.120102+0000","last_undegraded":"2024-03-18T20:35:37.120102+0000","last_fullsized":"2024-03-18T20:35:37.120102+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:07:26.711113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.114037+0000","last_change":"2024-03-18T20:34:38.804003+0000","last_active":"2024-03-18T20:35:37.114037+0000","last_peered":"2024-03-18T20:35:37.114037+0000","last_clean":"2024-03-18T20:35:37.114037+0000","last_became_active":"2024-03-18T20:34:38.803667+0000","last_became_peered":"2024-03-18T20:34:38.803667+0000","last_unstale":"2024-03-18T20:35:37.114037+0000","last_undegraded":"2024-03-18T20:35:37.114037+0000","last_fullsized":"2024-03-18T20:35:37.114037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:19:58.230790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110346+0000","last_change":"2024-03-18T20:34:38.346493+0000","last_active":"2024-03-18T20:35:37.110346+0000","last_peered":"2024-03-18T20:35:37.110346+0000","last_clean":"2024-03-18T20:35:37.110346+0000","last_became_active":"2024-03-18T20:34:38.345925+0000","last_became_peered":"2024-03-18T20:34:38.345925+0000","last_unstale":"2024-03-18T20:35:37.110346+0000","last_undegraded":"2024-03-18T20:35:37.110346+0000","last_fullsized":"2024-03-18T20:35:37.110346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:52:47.773860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119099+0000","last_change":"2024-03-18T20:34:38.807284+0000","last_active":"2024-03-18T20:35:37.119099+0000","last_peered":"2024-03-18T20:35:37.119099+0000","last_clean":"2024-03-18T20:35:37.119099+0000","last_became_active":"2024-03-18T20:34:38.806844+0000","last_became_peered":"2024-03-18T20:34:38.806844+0000","last_unstale":"2024-03-18T20:35:37.119099+0000","last_undegraded":"2024-03-18T20:35:37.119099+0000","last_fullsized":"2024-03-18T20:35:37.119099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:44:35.469953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.114113+0000","last_change":"2024-03-18T20:34:38.343090+0000","last_active":"2024-03-18T20:35:37.114113+0000","last_peered":"2024-03-18T20:35:37.114113+0000","last_clean":"2024-03-18T20:35:37.114113+0000","last_became_active":"2024-03-18T20:34:38.342712+0000","last_became_peered":"2024-03-18T20:34:38.342712+0000","last_unstale":"2024-03-18T20:35:37.114113+0000","last_undegraded":"2024-03-18T20:35:37.114113+0000","last_fullsized":"2024-03-18T20:35:37.114113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:14:05.909604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116701+0000","last_change":"2024-03-18T20:34:38.800616+0000","last_active":"2024-03-18T20:35:37.116701+0000","last_peered":"2024-03-18T20:35:37.116701+0000","last_clean":"2024-03-18T20:35:37.116701+0000","last_became_active":"2024-03-18T20:34:38.800340+0000","last_became_peered":"2024-03-18T20:34:38.800340+0000","last_unstale":"2024-03-18T20:35:37.116701+0000","last_undegraded":"2024-03-18T20:35:37.116701+0000","last_fullsized":"2024-03-18T20:35:37.116701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:18.263179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'5","reported_seq":155,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:51.891731+0000","last_change":"2024-03-18T20:34:38.345634+0000","last_active":"2024-03-18T20:35:51.891731+0000","last_peered":"2024-03-18T20:35:51.891731+0000","last_clean":"2024-03-18T20:35:51.891731+0000","last_became_active":"2024-03-18T20:34:38.345275+0000","last_became_peered":"2024-03-18T20:34:38.345275+0000","last_unstale":"2024-03-18T20:35:51.891731+0000","last_undegraded":"2024-03-18T20:35:51.891731+0000","last_fullsized":"2024-03-18T20:35:51.891731+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:29.939529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110859+0000","last_change":"2024-03-18T20:34:38.820286+0000","last_active":"2024-03-18T20:35:37.110859+0000","last_peered":"2024-03-18T20:35:37.110859+0000","last_clean":"2024-03-18T20:35:37.110859+0000","last_became_active":"2024-03-18T20:34:38.819939+0000","last_became_peered":"2024-03-18T20:34:38.819939+0000","last_unstale":"2024-03-18T20:35:37.110859+0000","last_undegraded":"2024-03-18T20:35:37.110859+0000","last_fullsized":"2024-03-18T20:35:37.110859+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:25:19.229339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116083+0000","last_change":"2024-03-18T20:34:38.341699+0000","last_active":"2024-03-18T20:35:37.116083+0000","last_peered":"2024-03-18T20:35:37.116083+0000","last_clean":"2024-03-18T20:35:37.116083+0000","last_became_active":"2024-03-18T20:34:38.341390+0000","last_became_peered":"2024-03-18T20:34:38.341390+0000","last_unstale":"2024-03-18T20:35:37.116083+0000","last_undegraded":"2024-03-18T20:35:37.116083+0000","last_fullsized":"2024-03-18T20:35:37.116083+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:16:02.239414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.117141+0000","last_change":"2024-03-18T20:35:38.117141+0000","last_active":"2024-03-18T20:35:38.117141+0000","last_peered":"2024-03-18T20:35:38.117141+0000","last_clean":"2024-03-18T20:35:38.117141+0000","last_became_active":"2024-03-18T20:35:38.116809+0000","last_became_peered":"2024-03-18T20:35:38.116809+0000","last_unstale":"2024-03-18T20:35:38.117141+0000","last_undegraded":"2024-03-18T20:35:38.117141+0000","last_fullsized":"2024-03-18T20:35:38.117141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:10:22.545159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113855+0000","last_change":"2024-03-18T20:34:38.346821+0000","last_active":"2024-03-18T20:35:37.113855+0000","last_peered":"2024-03-18T20:35:37.113855+0000","last_clean":"2024-03-18T20:35:37.113855+0000","last_became_active":"2024-03-18T20:34:38.346314+0000","last_became_peered":"2024-03-18T20:34:38.346314+0000","last_unstale":"2024-03-18T20:35:37.113855+0000","last_undegraded":"2024-03-18T20:35:37.113855+0000","last_fullsized":"2024-03-18T20:35:37.113855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:12:04.552971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111241+0000","last_change":"2024-03-18T20:34:38.820479+0000","last_active":"2024-03-18T20:35:37.111241+0000","last_peered":"2024-03-18T20:35:37.111241+0000","last_clean":"2024-03-18T20:35:37.111241+0000","last_became_active":"2024-03-18T20:34:38.820186+0000","last_became_peered":"2024-03-18T20:34:38.820186+0000","last_unstale":"2024-03-18T20:35:37.111241+0000","last_undegraded":"2024-03-18T20:35:37.111241+0000","last_fullsized":"2024-03-18T20:35:37.111241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:15.953764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":188,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:51.891872+0000","last_change":"2024-03-18T20:34:38.344332+0000","last_active":"2024-03-18T20:35:51.891872+0000","last_peered":"2024-03-18T20:35:51.891872+0000","last_clean":"2024-03-18T20:35:51.891872+0000","last_became_active":"2024-03-18T20:34:38.343827+0000","last_became_peered":"2024-03-18T20:34:38.343827+0000","last_unstale":"2024-03-18T20:35:51.891872+0000","last_undegraded":"2024-03-18T20:35:51.891872+0000","last_fullsized":"2024-03-18T20:35:51.891872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:13:21.725004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59732,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":143,"num_read_kb":130,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111038+0000","last_change":"2024-03-18T20:34:38.814794+0000","last_active":"2024-03-18T20:35:37.111038+0000","last_peered":"2024-03-18T20:35:37.111038+0000","last_clean":"2024-03-18T20:35:37.111038+0000","last_became_active":"2024-03-18T20:34:38.814434+0000","last_became_peered":"2024-03-18T20:34:38.814434+0000","last_unstale":"2024-03-18T20:35:37.111038+0000","last_undegraded":"2024-03-18T20:35:37.111038+0000","last_fullsized":"2024-03-18T20:35:37.111038+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:43:19.083212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'9","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117588+0000","last_change":"2024-03-18T20:34:38.343894+0000","last_active":"2024-03-18T20:35:37.117588+0000","last_peered":"2024-03-18T20:35:37.117588+0000","last_clean":"2024-03-18T20:35:37.117588+0000","last_became_active":"2024-03-18T20:34:38.343493+0000","last_became_peered":"2024-03-18T20:34:38.343493+0000","last_unstale":"2024-03-18T20:35:37.117588+0000","last_undegraded":"2024-03-18T20:35:37.117588+0000","last_fullsized":"2024-03-18T20:35:37.117588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:13:32.876781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110231+0000","last_change":"2024-03-18T20:34:38.809968+0000","last_active":"2024-03-18T20:35:37.110231+0000","last_peered":"2024-03-18T20:35:37.110231+0000","last_clean":"2024-03-18T20:35:37.110231+0000","last_became_active":"2024-03-18T20:34:38.809641+0000","last_became_peered":"2024-03-18T20:34:38.809641+0000","last_unstale":"2024-03-18T20:35:37.110231+0000","last_undegraded":"2024-03-18T20:35:37.110231+0000","last_fullsized":"2024-03-18T20:35:37.110231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:50:52.925683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'5","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113815+0000","last_change":"2024-03-18T20:34:38.345621+0000","last_active":"2024-03-18T20:35:37.113815+0000","last_peered":"2024-03-18T20:35:37.113815+0000","last_clean":"2024-03-18T20:35:37.113815+0000","last_became_active":"2024-03-18T20:34:38.345255+0000","last_became_peered":"2024-03-18T20:34:38.345255+0000","last_unstale":"2024-03-18T20:35:37.113815+0000","last_undegraded":"2024-03-18T20:35:37.113815+0000","last_fullsized":"2024-03-18T20:35:37.113815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:17:03.737973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115794+0000","last_change":"2024-03-18T20:34:38.804390+0000","last_active":"2024-03-18T20:35:37.115794+0000","last_peered":"2024-03-18T20:35:37.115794+0000","last_clean":"2024-03-18T20:35:37.115794+0000","last_became_active":"2024-03-18T20:34:38.804033+0000","last_became_peered":"2024-03-18T20:34:38.804033+0000","last_unstale":"2024-03-18T20:35:37.115794+0000","last_undegraded":"2024-03-18T20:35:37.115794+0000","last_fullsized":"2024-03-18T20:35:37.115794+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:09:45.407921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'6","reported_seq":91,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:51.891203+0000","last_change":"2024-03-18T20:34:38.340239+0000","last_active":"2024-03-18T20:35:51.891203+0000","last_peered":"2024-03-18T20:35:51.891203+0000","last_clean":"2024-03-18T20:35:51.891203+0000","last_became_active":"2024-03-18T20:34:38.339880+0000","last_became_peered":"2024-03-18T20:34:38.339880+0000","last_unstale":"2024-03-18T20:35:51.891203+0000","last_undegraded":"2024-03-18T20:35:51.891203+0000","last_fullsized":"2024-03-18T20:35:51.891203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:55:46.864885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115572+0000","last_change":"2024-03-18T20:34:38.819223+0000","last_active":"2024-03-18T20:35:37.115572+0000","last_peered":"2024-03-18T20:35:37.115572+0000","last_clean":"2024-03-18T20:35:37.115572+0000","last_became_active":"2024-03-18T20:34:38.818879+0000","last_became_peered":"2024-03-18T20:34:38.818879+0000","last_unstale":"2024-03-18T20:35:37.115572+0000","last_undegraded":"2024-03-18T20:35:37.115572+0000","last_fullsized":"2024-03-18T20:35:37.115572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:11:45.645912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'17","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110850+0000","last_change":"2024-03-18T20:34:38.336817+0000","last_active":"2024-03-18T20:35:37.110850+0000","last_peered":"2024-03-18T20:35:37.110850+0000","last_clean":"2024-03-18T20:35:37.110850+0000","last_became_active":"2024-03-18T20:34:38.336453+0000","last_became_peered":"2024-03-18T20:34:38.336453+0000","last_unstale":"2024-03-18T20:35:37.110850+0000","last_undegraded":"2024-03-18T20:35:37.110850+0000","last_fullsized":"2024-03-18T20:35:37.110850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:10.253435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113661+0000","last_change":"2024-03-18T20:34:38.818048+0000","last_active":"2024-03-18T20:35:37.113661+0000","last_peered":"2024-03-18T20:35:37.113661+0000","last_clean":"2024-03-18T20:35:37.113661+0000","last_became_active":"2024-03-18T20:34:38.817682+0000","last_became_peered":"2024-03-18T20:34:38.817682+0000","last_unstale":"2024-03-18T20:35:37.113661+0000","last_undegraded":"2024-03-18T20:35:37.113661+0000","last_fullsized":"2024-03-18T20:35:37.113661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:25:55.291815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.126651+0000","last_change":"2024-03-18T20:35:38.126271+0000","last_active":"2024-03-18T20:35:38.126651+0000","last_peered":"2024-03-18T20:35:38.126651+0000","last_clean":"2024-03-18T20:35:38.126651+0000","last_became_active":"2024-03-18T20:35:38.119078+0000","last_became_peered":"2024-03-18T20:35:38.119078+0000","last_unstale":"2024-03-18T20:35:38.126651+0000","last_undegraded":"2024-03-18T20:35:38.126651+0000","last_fullsized":"2024-03-18T20:35:38.126651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:46:20.914329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116043+0000","last_change":"2024-03-18T20:34:38.806138+0000","last_active":"2024-03-18T20:35:37.116043+0000","last_peered":"2024-03-18T20:35:37.116043+0000","last_clean":"2024-03-18T20:35:37.116043+0000","last_became_active":"2024-03-18T20:34:38.805821+0000","last_became_peered":"2024-03-18T20:34:38.805821+0000","last_unstale":"2024-03-18T20:35:37.116043+0000","last_undegraded":"2024-03-18T20:35:37.116043+0000","last_fullsized":"2024-03-18T20:35:37.116043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:25:57.887813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.118124+0000","last_change":"2024-03-18T20:34:38.338751+0000","last_active":"2024-03-18T20:35:37.118124+0000","last_peered":"2024-03-18T20:35:37.118124+0000","last_clean":"2024-03-18T20:35:37.118124+0000","last_became_active":"2024-03-18T20:34:38.337495+0000","last_became_peered":"2024-03-18T20:34:38.337495+0000","last_unstale":"2024-03-18T20:35:37.118124+0000","last_undegraded":"2024-03-18T20:35:37.118124+0000","last_fullsized":"2024-03-18T20:35:37.118124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:38:04.402380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119498+0000","last_change":"2024-03-18T20:34:38.803937+0000","last_active":"2024-03-18T20:35:37.119498+0000","last_peered":"2024-03-18T20:35:37.119498+0000","last_clean":"2024-03-18T20:35:37.119498+0000","last_became_active":"2024-03-18T20:34:38.803520+0000","last_became_peered":"2024-03-18T20:34:38.803520+0000","last_unstale":"2024-03-18T20:35:37.119498+0000","last_undegraded":"2024-03-18T20:35:37.119498+0000","last_fullsized":"2024-03-18T20:35:37.119498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:20:37.537265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'5","reported_seq":85,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:48.908311+0000","last_change":"2024-03-18T20:34:38.337481+0000","last_active":"2024-03-18T20:35:48.908311+0000","last_peered":"2024-03-18T20:35:48.908311+0000","last_clean":"2024-03-18T20:35:48.908311+0000","last_became_active":"2024-03-18T20:34:38.337152+0000","last_became_peered":"2024-03-18T20:34:38.337152+0000","last_unstale":"2024-03-18T20:35:48.908311+0000","last_undegraded":"2024-03-18T20:35:48.908311+0000","last_fullsized":"2024-03-18T20:35:48.908311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:07:59.593978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.119360+0000","last_change":"2024-03-18T20:35:38.119360+0000","last_active":"2024-03-18T20:35:38.119360+0000","last_peered":"2024-03-18T20:35:38.119360+0000","last_clean":"2024-03-18T20:35:38.119360+0000","last_became_active":"2024-03-18T20:35:38.118942+0000","last_became_peered":"2024-03-18T20:35:38.118942+0000","last_unstale":"2024-03-18T20:35:38.119360+0000","last_undegraded":"2024-03-18T20:35:38.119360+0000","last_fullsized":"2024-03-18T20:35:38.119360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:16:42.140321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.114312+0000","last_change":"2024-03-18T20:34:38.806133+0000","last_active":"2024-03-18T20:35:37.114312+0000","last_peered":"2024-03-18T20:35:37.114312+0000","last_clean":"2024-03-18T20:35:37.114312+0000","last_became_active":"2024-03-18T20:34:38.805257+0000","last_became_peered":"2024-03-18T20:34:38.805257+0000","last_unstale":"2024-03-18T20:35:37.114312+0000","last_undegraded":"2024-03-18T20:35:37.114312+0000","last_fullsized":"2024-03-18T20:35:37.114312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:05:15.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'5","reported_seq":88,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:50.889127+0000","last_change":"2024-03-18T20:34:38.346271+0000","last_active":"2024-03-18T20:35:50.889127+0000","last_peered":"2024-03-18T20:35:50.889127+0000","last_clean":"2024-03-18T20:35:50.889127+0000","last_became_active":"2024-03-18T20:34:38.345054+0000","last_became_peered":"2024-03-18T20:34:38.345054+0000","last_unstale":"2024-03-18T20:35:50.889127+0000","last_undegraded":"2024-03-18T20:35:50.889127+0000","last_fullsized":"2024-03-18T20:35:50.889127+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:42:52.510294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119964+0000","last_change":"2024-03-18T20:34:38.816834+0000","last_active":"2024-03-18T20:35:37.119964+0000","last_peered":"2024-03-18T20:35:37.119964+0000","last_clean":"2024-03-18T20:35:37.119964+0000","last_became_active":"2024-03-18T20:34:38.816516+0000","last_became_peered":"2024-03-18T20:34:38.816516+0000","last_unstale":"2024-03-18T20:35:37.119964+0000","last_undegraded":"2024-03-18T20:35:37.119964+0000","last_fullsized":"2024-03-18T20:35:37.119964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:33:54.529206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'6","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117428+0000","last_change":"2024-03-18T20:34:38.339407+0000","last_active":"2024-03-18T20:35:37.117428+0000","last_peered":"2024-03-18T20:35:37.117428+0000","last_clean":"2024-03-18T20:35:37.117428+0000","last_became_active":"2024-03-18T20:34:38.339086+0000","last_became_peered":"2024-03-18T20:34:38.339086+0000","last_unstale":"2024-03-18T20:35:37.117428+0000","last_undegraded":"2024-03-18T20:35:37.117428+0000","last_fullsized":"2024-03-18T20:35:37.117428+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:51:18.479630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":333,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115643+0000","last_change":"2024-03-18T20:33:50.777651+0000","last_active":"2024-03-18T20:35:37.115643+0000","last_peered":"2024-03-18T20:35:37.115643+0000","last_clean":"2024-03-18T20:35:37.115643+0000","last_became_active":"2024-03-18T20:33:50.777281+0000","last_became_peered":"2024-03-18T20:33:50.777281+0000","last_unstale":"2024-03-18T20:35:37.115643+0000","last_undegraded":"2024-03-18T20:35:37.115643+0000","last_fullsized":"2024-03-18T20:35:37.115643+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:31:27.488744+0000","last_clean_scrub_stamp":"2024-03-18T20:31:27.488744+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:28.605049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110103+0000","last_change":"2024-03-18T20:34:38.812714+0000","last_active":"2024-03-18T20:35:37.110103+0000","last_peered":"2024-03-18T20:35:37.110103+0000","last_clean":"2024-03-18T20:35:37.110103+0000","last_became_active":"2024-03-18T20:34:38.812464+0000","last_became_peered":"2024-03-18T20:34:38.812464+0000","last_unstale":"2024-03-18T20:35:37.110103+0000","last_undegraded":"2024-03-18T20:35:37.110103+0000","last_fullsized":"2024-03-18T20:35:37.110103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:03:13.853740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115734+0000","last_change":"2024-03-18T20:34:38.343173+0000","last_active":"2024-03-18T20:35:37.115734+0000","last_peered":"2024-03-18T20:35:37.115734+0000","last_clean":"2024-03-18T20:35:37.115734+0000","last_became_active":"2024-03-18T20:34:38.342774+0000","last_became_peered":"2024-03-18T20:34:38.342774+0000","last_unstale":"2024-03-18T20:35:37.115734+0000","last_undegraded":"2024-03-18T20:35:37.115734+0000","last_fullsized":"2024-03-18T20:35:37.115734+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:23:52.283033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.111085+0000","last_change":"2024-03-18T20:34:38.344539+0000","last_active":"2024-03-18T20:35:37.111085+0000","last_peered":"2024-03-18T20:35:37.111085+0000","last_clean":"2024-03-18T20:35:37.111085+0000","last_became_active":"2024-03-18T20:34:38.344188+0000","last_became_peered":"2024-03-18T20:34:38.344188+0000","last_unstale":"2024-03-18T20:35:37.111085+0000","last_undegraded":"2024-03-18T20:35:37.111085+0000","last_fullsized":"2024-03-18T20:35:37.111085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:40:38.641928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110493+0000","last_change":"2024-03-18T20:34:38.806602+0000","last_active":"2024-03-18T20:35:37.110493+0000","last_peered":"2024-03-18T20:35:37.110493+0000","last_clean":"2024-03-18T20:35:37.110493+0000","last_became_active":"2024-03-18T20:34:38.806241+0000","last_became_peered":"2024-03-18T20:34:38.806241+0000","last_unstale":"2024-03-18T20:35:37.110493+0000","last_undegraded":"2024-03-18T20:35:37.110493+0000","last_fullsized":"2024-03-18T20:35:37.110493+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:48:04.463311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'9","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.118511+0000","last_change":"2024-03-18T20:34:38.339695+0000","last_active":"2024-03-18T20:35:37.118511+0000","last_peered":"2024-03-18T20:35:37.118511+0000","last_clean":"2024-03-18T20:35:37.118511+0000","last_became_active":"2024-03-18T20:34:38.339331+0000","last_became_peered":"2024-03-18T20:34:38.339331+0000","last_unstale":"2024-03-18T20:35:37.118511+0000","last_undegraded":"2024-03-18T20:35:37.118511+0000","last_fullsized":"2024-03-18T20:35:37.118511+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:57:06.434289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120777+0000","last_change":"2024-03-18T20:34:38.809754+0000","last_active":"2024-03-18T20:35:37.120777+0000","last_peered":"2024-03-18T20:35:37.120777+0000","last_clean":"2024-03-18T20:35:37.120777+0000","last_became_active":"2024-03-18T20:34:38.809415+0000","last_became_peered":"2024-03-18T20:34:38.809415+0000","last_unstale":"2024-03-18T20:35:37.120777+0000","last_undegraded":"2024-03-18T20:35:37.120777+0000","last_fullsized":"2024-03-18T20:35:37.120777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:34:30.761582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'7","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117134+0000","last_change":"2024-03-18T20:34:38.342080+0000","last_active":"2024-03-18T20:35:37.117134+0000","last_peered":"2024-03-18T20:35:37.117134+0000","last_clean":"2024-03-18T20:35:37.117134+0000","last_became_active":"2024-03-18T20:34:38.340623+0000","last_became_peered":"2024-03-18T20:34:38.340623+0000","last_unstale":"2024-03-18T20:35:37.117134+0000","last_undegraded":"2024-03-18T20:35:37.117134+0000","last_fullsized":"2024-03-18T20:35:37.117134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:49:14.912391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117644+0000","last_change":"2024-03-18T20:34:38.806558+0000","last_active":"2024-03-18T20:35:37.117644+0000","last_peered":"2024-03-18T20:35:37.117644+0000","last_clean":"2024-03-18T20:35:37.117644+0000","last_became_active":"2024-03-18T20:34:38.806171+0000","last_became_peered":"2024-03-18T20:34:38.806171+0000","last_unstale":"2024-03-18T20:35:37.117644+0000","last_undegraded":"2024-03-18T20:35:37.117644+0000","last_fullsized":"2024-03-18T20:35:37.117644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:46:26.530157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110601+0000","last_change":"2024-03-18T20:34:38.336175+0000","last_active":"2024-03-18T20:35:37.110601+0000","last_peered":"2024-03-18T20:35:37.110601+0000","last_clean":"2024-03-18T20:35:37.110601+0000","last_became_active":"2024-03-18T20:34:38.335798+0000","last_became_peered":"2024-03-18T20:34:38.335798+0000","last_unstale":"2024-03-18T20:35:37.110601+0000","last_undegraded":"2024-03-18T20:35:37.110601+0000","last_fullsized":"2024-03-18T20:35:37.110601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:33:33.222340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110847+0000","last_change":"2024-03-18T20:34:38.807684+0000","last_active":"2024-03-18T20:35:37.110847+0000","last_peered":"2024-03-18T20:35:37.110847+0000","last_clean":"2024-03-18T20:35:37.110847+0000","last_became_active":"2024-03-18T20:34:38.807300+0000","last_became_peered":"2024-03-18T20:34:38.807300+0000","last_unstale":"2024-03-18T20:35:37.110847+0000","last_undegraded":"2024-03-18T20:35:37.110847+0000","last_fullsized":"2024-03-18T20:35:37.110847+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:31:43.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'7","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120021+0000","last_change":"2024-03-18T20:34:38.341802+0000","last_active":"2024-03-18T20:35:37.120021+0000","last_peered":"2024-03-18T20:35:37.120021+0000","last_clean":"2024-03-18T20:35:37.120021+0000","last_became_active":"2024-03-18T20:34:38.340594+0000","last_became_peered":"2024-03-18T20:34:38.340594+0000","last_unstale":"2024-03-18T20:35:37.120021+0000","last_undegraded":"2024-03-18T20:35:37.120021+0000","last_fullsized":"2024-03-18T20:35:37.120021+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:01:06.830323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116079+0000","last_change":"2024-03-18T20:34:38.809790+0000","last_active":"2024-03-18T20:35:37.116079+0000","last_peered":"2024-03-18T20:35:37.116079+0000","last_clean":"2024-03-18T20:35:37.116079+0000","last_became_active":"2024-03-18T20:34:38.809470+0000","last_became_peered":"2024-03-18T20:34:38.809470+0000","last_unstale":"2024-03-18T20:35:37.116079+0000","last_undegraded":"2024-03-18T20:35:37.116079+0000","last_fullsized":"2024-03-18T20:35:37.116079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:32:48.225759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.123225+0000","last_change":"2024-03-18T20:35:38.122793+0000","last_active":"2024-03-18T20:35:38.123225+0000","last_peered":"2024-03-18T20:35:38.123225+0000","last_clean":"2024-03-18T20:35:38.123225+0000","last_became_active":"2024-03-18T20:35:38.118680+0000","last_became_peered":"2024-03-18T20:35:38.118680+0000","last_unstale":"2024-03-18T20:35:38.123225+0000","last_undegraded":"2024-03-18T20:35:38.123225+0000","last_fullsized":"2024-03-18T20:35:38.123225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:04:06.361764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119251+0000","last_change":"2024-03-18T20:34:38.809294+0000","last_active":"2024-03-18T20:35:37.119251+0000","last_peered":"2024-03-18T20:35:37.119251+0000","last_clean":"2024-03-18T20:35:37.119251+0000","last_became_active":"2024-03-18T20:34:38.808542+0000","last_became_peered":"2024-03-18T20:34:38.808542+0000","last_unstale":"2024-03-18T20:35:37.119251+0000","last_undegraded":"2024-03-18T20:35:37.119251+0000","last_fullsized":"2024-03-18T20:35:37.119251+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T08:11:30.669317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110948+0000","last_change":"2024-03-18T20:34:38.339454+0000","last_active":"2024-03-18T20:35:37.110948+0000","last_peered":"2024-03-18T20:35:37.110948+0000","last_clean":"2024-03-18T20:35:37.110948+0000","last_became_active":"2024-03-18T20:34:38.339154+0000","last_became_peered":"2024-03-18T20:34:38.339154+0000","last_unstale":"2024-03-18T20:35:37.110948+0000","last_undegraded":"2024-03-18T20:35:37.110948+0000","last_fullsized":"2024-03-18T20:35:37.110948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:12:06.300600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116002+0000","last_change":"2024-03-18T20:34:38.818462+0000","last_active":"2024-03-18T20:35:37.116002+0000","last_peered":"2024-03-18T20:35:37.116002+0000","last_clean":"2024-03-18T20:35:37.116002+0000","last_became_active":"2024-03-18T20:34:38.817522+0000","last_became_peered":"2024-03-18T20:34:38.817522+0000","last_unstale":"2024-03-18T20:35:37.116002+0000","last_undegraded":"2024-03-18T20:35:37.116002+0000","last_fullsized":"2024-03-18T20:35:37.116002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:50:32.258583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'5","reported_seq":89,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:50.890976+0000","last_change":"2024-03-18T20:34:38.341054+0000","last_active":"2024-03-18T20:35:50.890976+0000","last_peered":"2024-03-18T20:35:50.890976+0000","last_clean":"2024-03-18T20:35:50.890976+0000","last_became_active":"2024-03-18T20:34:38.340352+0000","last_became_peered":"2024-03-18T20:34:38.340352+0000","last_unstale":"2024-03-18T20:35:50.890976+0000","last_undegraded":"2024-03-18T20:35:50.890976+0000","last_fullsized":"2024-03-18T20:35:50.890976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:32:44.850244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.115977+0000","last_change":"2024-03-18T20:34:38.818176+0000","last_active":"2024-03-18T20:35:37.115977+0000","last_peered":"2024-03-18T20:35:37.115977+0000","last_clean":"2024-03-18T20:35:37.115977+0000","last_became_active":"2024-03-18T20:34:38.817820+0000","last_became_peered":"2024-03-18T20:34:38.817820+0000","last_unstale":"2024-03-18T20:35:37.115977+0000","last_undegraded":"2024-03-18T20:35:37.115977+0000","last_fullsized":"2024-03-18T20:35:37.115977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:19:29.815174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.113678+0000","last_change":"2024-03-18T20:34:38.346591+0000","last_active":"2024-03-18T20:35:37.113678+0000","last_peered":"2024-03-18T20:35:37.113678+0000","last_clean":"2024-03-18T20:35:37.113678+0000","last_became_active":"2024-03-18T20:34:38.345897+0000","last_became_peered":"2024-03-18T20:34:38.345897+0000","last_unstale":"2024-03-18T20:35:37.113678+0000","last_undegraded":"2024-03-18T20:35:37.113678+0000","last_fullsized":"2024-03-18T20:35:37.113678+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:47:32.118905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-03-18T20:35:38.122016+0000","last_change":"2024-03-18T20:35:38.122016+0000","last_active":"2024-03-18T20:35:38.122016+0000","last_peered":"2024-03-18T20:35:38.122016+0000","last_clean":"2024-03-18T20:35:38.122016+0000","last_became_active":"2024-03-18T20:35:38.118053+0000","last_became_peered":"2024-03-18T20:35:38.118053+0000","last_unstale":"2024-03-18T20:35:38.122016+0000","last_undegraded":"2024-03-18T20:35:38.122016+0000","last_fullsized":"2024-03-18T20:35:38.122016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:25:28.130422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'5","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110701+0000","last_change":"2024-03-18T20:34:38.344615+0000","last_active":"2024-03-18T20:35:37.110701+0000","last_peered":"2024-03-18T20:35:37.110701+0000","last_clean":"2024-03-18T20:35:37.110701+0000","last_became_active":"2024-03-18T20:34:38.344242+0000","last_became_peered":"2024-03-18T20:34:38.344242+0000","last_unstale":"2024-03-18T20:35:37.110701+0000","last_undegraded":"2024-03-18T20:35:37.110701+0000","last_fullsized":"2024-03-18T20:35:37.110701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:37.310661+0000","last_clean_scrub_stamp":"2024-03-18T20:34:37.310661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T05:36:57.449808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116268+0000","last_change":"2024-03-18T20:34:38.820875+0000","last_active":"2024-03-18T20:35:37.116268+0000","last_peered":"2024-03-18T20:35:37.116268+0000","last_clean":"2024-03-18T20:35:37.116268+0000","last_became_active":"2024-03-18T20:34:38.820550+0000","last_became_peered":"2024-03-18T20:34:38.820550+0000","last_unstale":"2024-03-18T20:35:37.116268+0000","last_undegraded":"2024-03-18T20:35:37.116268+0000","last_fullsized":"2024-03-18T20:35:37.116268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T06:49:36.358734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116141+0000","last_change":"2024-03-18T20:34:38.812552+0000","last_active":"2024-03-18T20:35:37.116141+0000","last_peered":"2024-03-18T20:35:37.116141+0000","last_clean":"2024-03-18T20:35:37.116141+0000","last_became_active":"2024-03-18T20:34:38.811939+0000","last_became_peered":"2024-03-18T20:34:38.811939+0000","last_unstale":"2024-03-18T20:35:37.116141+0000","last_undegraded":"2024-03-18T20:35:37.116141+0000","last_fullsized":"2024-03-18T20:35:37.116141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:41:52.096974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117526+0000","last_change":"2024-03-18T20:34:38.815691+0000","last_active":"2024-03-18T20:35:37.117526+0000","last_peered":"2024-03-18T20:35:37.117526+0000","last_clean":"2024-03-18T20:35:37.117526+0000","last_became_active":"2024-03-18T20:34:38.815370+0000","last_became_peered":"2024-03-18T20:34:38.815370+0000","last_unstale":"2024-03-18T20:35:37.117526+0000","last_undegraded":"2024-03-18T20:35:37.117526+0000","last_fullsized":"2024-03-18T20:35:37.117526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:59:41.260013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117452+0000","last_change":"2024-03-18T20:34:38.812651+0000","last_active":"2024-03-18T20:35:37.117452+0000","last_peered":"2024-03-18T20:35:37.117452+0000","last_clean":"2024-03-18T20:35:37.117452+0000","last_became_active":"2024-03-18T20:34:38.812322+0000","last_became_peered":"2024-03-18T20:34:38.812322+0000","last_unstale":"2024-03-18T20:35:37.117452+0000","last_undegraded":"2024-03-18T20:35:37.117452+0000","last_fullsized":"2024-03-18T20:35:37.117452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:34:55.658371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117275+0000","last_change":"2024-03-18T20:34:38.806475+0000","last_active":"2024-03-18T20:35:37.117275+0000","last_peered":"2024-03-18T20:35:37.117275+0000","last_clean":"2024-03-18T20:35:37.117275+0000","last_became_active":"2024-03-18T20:34:38.806027+0000","last_became_peered":"2024-03-18T20:34:38.806027+0000","last_unstale":"2024-03-18T20:35:37.117275+0000","last_undegraded":"2024-03-18T20:35:37.117275+0000","last_fullsized":"2024-03-18T20:35:37.117275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:57:17.140031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110573+0000","last_change":"2024-03-18T20:34:38.811057+0000","last_active":"2024-03-18T20:35:37.110573+0000","last_peered":"2024-03-18T20:35:37.110573+0000","last_clean":"2024-03-18T20:35:37.110573+0000","last_became_active":"2024-03-18T20:34:38.810316+0000","last_became_peered":"2024-03-18T20:34:38.810316+0000","last_unstale":"2024-03-18T20:35:37.110573+0000","last_undegraded":"2024-03-18T20:35:37.110573+0000","last_fullsized":"2024-03-18T20:35:37.110573+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T04:29:22.426717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116579+0000","last_change":"2024-03-18T20:34:38.812952+0000","last_active":"2024-03-18T20:35:37.116579+0000","last_peered":"2024-03-18T20:35:37.116579+0000","last_clean":"2024-03-18T20:35:37.116579+0000","last_became_active":"2024-03-18T20:34:38.812550+0000","last_became_peered":"2024-03-18T20:34:38.812550+0000","last_unstale":"2024-03-18T20:35:37.116579+0000","last_undegraded":"2024-03-18T20:35:37.116579+0000","last_fullsized":"2024-03-18T20:35:37.116579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:06:26.525078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.117914+0000","last_change":"2024-03-18T20:34:38.817358+0000","last_active":"2024-03-18T20:35:37.117914+0000","last_peered":"2024-03-18T20:35:37.117914+0000","last_clean":"2024-03-18T20:35:37.117914+0000","last_became_active":"2024-03-18T20:34:38.816939+0000","last_became_peered":"2024-03-18T20:34:38.816939+0000","last_unstale":"2024-03-18T20:35:37.117914+0000","last_undegraded":"2024-03-18T20:35:37.117914+0000","last_fullsized":"2024-03-18T20:35:37.117914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:33.258162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.120687+0000","last_change":"2024-03-18T20:34:38.807798+0000","last_active":"2024-03-18T20:35:37.120687+0000","last_peered":"2024-03-18T20:35:37.120687+0000","last_clean":"2024-03-18T20:35:37.120687+0000","last_became_active":"2024-03-18T20:34:38.807489+0000","last_became_peered":"2024-03-18T20:34:38.807489+0000","last_unstale":"2024-03-18T20:35:37.120687+0000","last_undegraded":"2024-03-18T20:35:37.120687+0000","last_fullsized":"2024-03-18T20:35:37.120687+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T07:00:15.395763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.114222+0000","last_change":"2024-03-18T20:34:38.806363+0000","last_active":"2024-03-18T20:35:37.114222+0000","last_peered":"2024-03-18T20:35:37.114222+0000","last_clean":"2024-03-18T20:35:37.114222+0000","last_became_active":"2024-03-18T20:34:38.805707+0000","last_became_peered":"2024-03-18T20:34:38.805707+0000","last_unstale":"2024-03-18T20:35:37.114222+0000","last_undegraded":"2024-03-18T20:35:37.114222+0000","last_fullsized":"2024-03-18T20:35:37.114222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:32:10.918979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.119927+0000","last_change":"2024-03-18T20:34:38.814439+0000","last_active":"2024-03-18T20:35:37.119927+0000","last_peered":"2024-03-18T20:35:37.119927+0000","last_clean":"2024-03-18T20:35:37.119927+0000","last_became_active":"2024-03-18T20:34:38.813988+0000","last_became_peered":"2024-03-18T20:34:38.813988+0000","last_unstale":"2024-03-18T20:35:37.119927+0000","last_undegraded":"2024-03-18T20:35:37.119927+0000","last_fullsized":"2024-03-18T20:35:37.119927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:03:14.639044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110698+0000","last_change":"2024-03-18T20:34:38.818550+0000","last_active":"2024-03-18T20:35:37.110698+0000","last_peered":"2024-03-18T20:35:37.110698+0000","last_clean":"2024-03-18T20:35:37.110698+0000","last_became_active":"2024-03-18T20:34:38.818209+0000","last_became_peered":"2024-03-18T20:34:38.818209+0000","last_unstale":"2024-03-18T20:35:37.110698+0000","last_undegraded":"2024-03-18T20:35:37.110698+0000","last_fullsized":"2024-03-18T20:35:37.110698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:40:48.395331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.110529+0000","last_change":"2024-03-18T20:34:38.808439+0000","last_active":"2024-03-18T20:35:37.110529+0000","last_peered":"2024-03-18T20:35:37.110529+0000","last_clean":"2024-03-18T20:35:37.110529+0000","last_became_active":"2024-03-18T20:34:38.807995+0000","last_became_peered":"2024-03-18T20:34:38.807995+0000","last_unstale":"2024-03-18T20:35:37.110529+0000","last_undegraded":"2024-03-18T20:35:37.110529+0000","last_fullsized":"2024-03-18T20:35:37.110529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:22:54.414148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-18T20:35:37.116358+0000","last_change":"2024-03-18T20:34:38.810793+0000","last_active":"2024-03-18T20:35:37.116358+0000","last_peered":"2024-03-18T20:35:37.116358+0000","last_clean":"2024-03-18T20:35:37.116358+0000","last_became_active":"2024-03-18T20:34:38.810496+0000","last_became_peered":"2024-03-18T20:34:38.810496+0000","last_unstale":"2024-03-18T20:35:37.116358+0000","last_undegraded":"2024-03-18T20:35:37.116358+0000","last_fullsized":"2024-03-18T20:35:37.116358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T20:34:38.315381+0000","last_clean_scrub_stamp":"2024-03-18T20:34:38.315381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:33.780675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66668,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":716,"num_read_kb":431,"num_write":233,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":204318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612632,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":422867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.379},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.67100000000000004}]},{"osd":2,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":4,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.95799999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.61299999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59899999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.0309999999999999}]},{"osd":8,"last update":"Mon Mar 18 20:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.47999999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.76100000000000001}]},{"osd":10,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.56999999999999995}]}]},{"osd":10,"up_from":66,"seq":283467841564,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1572864,"data_stored":1073667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.4699999999999998,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":6.6449999999999996,"5min":6.6449999999999996,"15min":6.6449999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.376}]},{"osd":2,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56999999999999995}]},{"osd":3,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.50600000000000001}]},{"osd":4,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57899999999999996}]},{"osd":5,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.66000000000000003}]},{"osd":6,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.67400000000000004}]},{"osd":9,"last update":"Mon Mar 18 20:34:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.69799999999999995}]},{"osd":11,"last update":"Mon Mar 18 20:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.624},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.70799999999999996}]}]},{"osd":9,"up_from":61,"seq":261993005088,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":422940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.46100000000000002,"15min":0.47099999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55200000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.57999999999999996,"15min":0.59199999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.878,"15min":0.878},"last":0.435}]},{"osd":1,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77000000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.67500000000000004,"15min":0.68400000000000005},"min":{"1min":0.443,"5min":0.442,"15min":0.442},"max":{"1min":0.88800000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.74399999999999999}]},{"osd":2,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.66900000000000004},"min":{"1min":0.46100000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83099999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.46600000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.59999999999999998}]},{"osd":3,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.51600000000000001,"15min":0.52400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69299999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.34699999999999998}]},{"osd":4,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64700000000000002,"15min":0.65900000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81100000000000005,"5min":1.016,"15min":1.016},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.73799999999999999,"15min":0.751},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93300000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.69399999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83199999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86299999999999999,"5min":1.008,"15min":1.008},"last":0.76500000000000001}]},{"osd":6,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.65500000000000003,"15min":0.67200000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73599999999999999,"5min":1.179,"15min":1.179},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.70599999999999996,"15min":0.73099999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77100000000000002,"5min":1.151,"15min":1.151},"last":0.46999999999999997}]},{"osd":7,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.80800000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0049999999999999,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.73599999999999999,"15min":0.748},"min":{"1min":0.51200000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.85799999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.65500000000000003}]},{"osd":8,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.80700000000000005,"15min":0.81699999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96899999999999997,"5min":1.353,"15min":1.353},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.78700000000000003,"15min":0.79900000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.90300000000000002,"5min":1.103,"15min":1.103},"last":0.61899999999999999}]},{"osd":10,"last update":"Mon Mar 18 20:34:50 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.49199999999999999}]}]},{"osd":8,"up_from":54,"seq":231928234020,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":423179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.38500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.622,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47099999999999997}]},{"osd":1,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73399999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42299999999999999,"15min":0.41399999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.32600000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.38600000000000001}]},{"osd":3,"last update":"Mon Mar 18 20:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.622}]},{"osd":4,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.749,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.51100000000000001,"15min":0.498},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.495},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.34699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40600000000000003}]},{"osd":6,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.66200000000000003,"15min":0.64900000000000002},"min":{"1min":0.44600000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.75600000000000001}]},{"osd":7,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.46899999999999997,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.874,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66000000000000003}]},{"osd":9,"last update":"Mon Mar 18 20:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.66000000000000003,"15min":0.64700000000000002},"min":{"1min":0.51700000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.70899999999999996,"15min":0.69399999999999995},"min":{"1min":0.63900000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.79900000000000004}]},{"osd":10,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.65900000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.72499999999999998,"15min":0.70699999999999996},"min":{"1min":0.76500000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.95299999999999996,"5min":1.018,"15min":1.018},"last":0.82299999999999995}]},{"osd":11,"last update":"Mon Mar 18 20:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.78500000000000003}]}]},{"osd":7,"up_from":47,"seq":201863462950,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1495040,"data_stored":1015576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.439}]},{"osd":1,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.42199999999999999,"15min":0.41399999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.28899999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.378}]},{"osd":2,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73799999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76800000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.65900000000000003}]},{"osd":3,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.72999999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51500000000000001,"15min":0.51600000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.44500000000000001}]},{"osd":5,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.40500000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.497},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63,"15min":0.624},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.61899999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89700000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60199999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.71299999999999997}]},{"osd":9,"last update":"Mon Mar 18 20:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.59299999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94099999999999995,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.61299999999999999,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.72799999999999998}]},{"osd":10,"last update":"Mon Mar 18 20:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.69599999999999995}]},{"osd":11,"last update":"Mon Mar 18 20:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.748}]}]},{"osd":6,"up_from":42,"seq":180388626474,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":422492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.437,"15min":0.42699999999999999},"min":{"1min":0.35399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.44900000000000001,"15min":0.44500000000000001},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.625,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.441}]},{"osd":1,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.39400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75800000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.60599999999999998,"15min":0.58399999999999996},"min":{"1min":0.505,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59799999999999998}]},{"osd":2,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58399999999999996,"15min":0.58199999999999996},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.46800000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63}]},{"osd":3,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.35099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.874,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.54800000000000004}]},{"osd":4,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60599999999999998,"15min":0.58999999999999997},"min":{"1min":0.498,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77200000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63,"15min":0.61599999999999999},"min":{"1min":0.45600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.56399999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.54500000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95999999999999996,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.94699999999999995,"5min":1.052,"15min":1.052},"last":0.58099999999999996}]},{"osd":7,"last update":"Mon Mar 18 20:35:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.89000000000000001,"5min":0.995,"15min":0.995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.58699999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.97199999999999998,"5min":1.006,"15min":1.006},"last":0.81399999999999995}]},{"osd":8,"last update":"Mon Mar 18 20:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74099999999999999,"15min":0.747},"min":{"1min":0.47399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92000000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76600000000000001,"15min":0.76500000000000001},"min":{"1min":0.60999999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.77000000000000002}]},{"osd":9,"last update":"Mon Mar 18 20:35:32 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.81799999999999995,"15min":0.83199999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.83499999999999996,"5min":1.028,"15min":1.028},"last":0.61899999999999999}]},{"osd":10,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84699999999999998,"15min":0.85299999999999998},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.89700000000000002,"5min":1.081,"15min":1.081},"last":0.80000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":483137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.59299999999999997,"15min":0.54800000000000004},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55100000000000005,"15min":0.54800000000000004},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.46400000000000002}]},{"osd":2,"last update":"Mon Mar 18 20:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.56999999999999995,"15min":0.54400000000000004},"min":{"1min":0.497,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.38600000000000001}]},{"osd":3,"last update":"Mon Mar 18 20:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.497,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.497}]},{"osd":4,"last update":"Mon Mar 18 20:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.443,"15min":0.40999999999999998},"min":{"1min":0.34499999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.70099999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.45000000000000001,"15min":0.42099999999999999},"min":{"1min":0.32400000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.58399999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.64400000000000002,"15min":0.623},"min":{"1min":0.52700000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.86099999999999999}]},{"osd":6,"last update":"Mon Mar 18 20:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.67900000000000005,"15min":0.623},"min":{"1min":0.52200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":4.5679999999999996,"5min":4.5679999999999996,"15min":4.5679999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.64700000000000002,"15min":0.60299999999999998},"min":{"1min":0.44,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":4.5330000000000004,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.54100000000000004}]},{"osd":7,"last update":"Mon Mar 18 20:34:55 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.84399999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.377,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42699999999999999}]},{"osd":8,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63600000000000001,"15min":0.627},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.55300000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98899999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.629}]},{"osd":9,"last update":"Mon Mar 18 20:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.745,"15min":0.71899999999999997},"min":{"1min":0.69099999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70199999999999996,"15min":0.68600000000000005},"min":{"1min":0.61199999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.65600000000000003}]},{"osd":10,"last update":"Mon Mar 18 20:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.58199999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77300000000000002,"15min":0.75900000000000001},"min":{"1min":0.70599999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.70799999999999996}]},{"osd":11,"last update":"Mon Mar 18 20:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73199999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":423831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Mar 18 20:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53500000000000003,"15min":0.51100000000000001},"min":{"1min":0.34599999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53600000000000003,"15min":0.52300000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79700000000000004,"5min":1.024,"15min":1.024},"last":0.51300000000000001}]},{"osd":2,"last update":"Mon Mar 18 20:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57799999999999996,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63200000000000001,"15min":0.63100000000000001},"min":{"1min":0.42799999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1100000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.46400000000000002}]},{"osd":3,"last update":"Mon Mar 18 20:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63800000000000001}]},{"osd":4,"last update":"Mon Mar 18 20:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.374,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.629},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89900000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.57399999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.48499999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.623,"15min":0.60599999999999998},"min":{"1min":0.47599999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85099999999999998,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.34699999999999998}]},{"osd":6,"last update":"Mon Mar 18 20:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.311},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.55100000000000005}]},{"osd":7,"last update":"Mon Mar 18 20:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.65100000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67400000000000004,"15min":0.67700000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.92200000000000004,"5min":1.002,"15min":1.002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.745,"15min":0.747},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93400000000000005,"5min":0.998,"15min":0.998},"last":0.58799999999999997}]},{"osd":9,"last update":"Mon Mar 18 20:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.44600000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.439},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90600000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67300000000000004}]},{"osd":10,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.004,"5min":1.105,"15min":1.105},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.61499999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.016,"5min":1.177,"15min":1.177},"last":0.62}]},{"osd":11,"last update":"Mon Mar 18 20:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":482954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.58599999999999997,"15min":0.497},"min":{"1min":0.379,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":7.8559999999999999,"5min":7.8559999999999999,"15min":7.8559999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.58299999999999996,"15min":0.496},"min":{"1min":0.42799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.7880000000000003,"5min":7.7880000000000003,"15min":7.7880000000000003},"last":0.39300000000000002}]},{"osd":1,"last update":"Mon Mar 18 20:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58399999999999996,"15min":0.53700000000000003},"min":{"1min":0.48399999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81499999999999995,"5min":4.6100000000000003,"15min":4.6100000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.53900000000000003}]},{"osd":3,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57099999999999995,"15min":0.52900000000000003},"min":{"1min":0.40000000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.497},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.57799999999999996,"15min":0.53800000000000003},"min":{"1min":0.46700000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46700000000000003}]},{"osd":4,"last update":"Mon Mar 18 20:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.63},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Mar 18 20:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.47099999999999997,"15min":0.44700000000000001},"min":{"1min":0.39000000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.31},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.54900000000000004,"15min":0.53700000000000003},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Mon Mar 18 20:35:45 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.66300000000000003,"15min":0.65600000000000003},"min":{"1min":0.54500000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":0.75,"15min":0.68500000000000005},"min":{"1min":0.45300000000000001,"5min":0.436,"15min":0.436},"max":{"1min":9.9969999999999999,"5min":9.9969999999999999,"15min":9.9969999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Mon Mar 18 20:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.439},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45800000000000002}]},{"osd":8,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80800000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.438},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84699999999999998,"5min":1.038,"15min":1.038},"last":0.64000000000000001}]},{"osd":9,"last update":"Mon Mar 18 20:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.70799999999999996,"15min":0.69799999999999995},"min":{"1min":0.53900000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93100000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.274,"5min":0.83299999999999996,"15min":0.75900000000000001},"min":{"1min":0.59999999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":9.8780000000000001,"5min":9.8780000000000001,"15min":9.8780000000000001},"last":0.65000000000000002}]},{"osd":10,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.89800000000000002,"15min":0.92000000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0589999999999999,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.751,"15min":0.745},"min":{"1min":0.60799999999999998,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97799999999999998,"5min":1.044,"15min":1.044},"last":0.60799999999999998}]},{"osd":11,"last update":"Mon Mar 18 20:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66900000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215156,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":422248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.41099999999999998},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.307},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.442,"15min":0.41999999999999998},"min":{"1min":0.33400000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.40699999999999997}]},{"osd":1,"last update":"Mon Mar 18 20:35:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.36799999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.052,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.41299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54100000000000004}]},{"osd":2,"last update":"Mon Mar 18 20:35:52 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.372,"15min":0.372},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.46100000000000002}]},{"osd":4,"last update":"Mon Mar 18 20:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90700000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.625},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.67800000000000005,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.876,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.69299999999999995}]},{"osd":5,"last update":"Mon Mar 18 20:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72899999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84399999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52500000000000002}]},{"osd":6,"last update":"Mon Mar 18 20:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56899999999999995,"15min":0.55600000000000005},"min":{"1min":0.376,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.62},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.755,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59799999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.90500000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0169999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.61299999999999999}]},{"osd":8,"last update":"Mon Mar 18 20:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73099999999999998}]},{"osd":9,"last update":"Mon Mar 18 20:35:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.87,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55900000000000005}]},{"osd":10,"last update":"Mon Mar 18 20:35:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.77700000000000002,"15min":0.78600000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.97899999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72099999999999997}]},{"osd":11,"last update":"Mon Mar 18 20:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57499999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":422745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.497,"15min":0.49199999999999999},"min":{"1min":0.38400000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.40500000000000003}]},{"osd":1,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.371,"15min":0.36099999999999999},"min":{"1min":0.31,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.435,"5min":0.374,"15min":0.35999999999999999},"min":{"1min":0.29399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.63,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.432}]},{"osd":2,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55100000000000005,"15min":0.52600000000000002},"min":{"1min":0.45600000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76700000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72199999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.62}]},{"osd":3,"last update":"Mon Mar 18 20:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.45400000000000001}]},{"osd":5,"last update":"Mon Mar 18 20:35:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.46100000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82299999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.41599999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81200000000000006,"5min":0.875,"15min":0.875},"last":0.59199999999999997}]},{"osd":6,"last update":"Mon Mar 18 20:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.63200000000000001,"15min":0.58399999999999996},"min":{"1min":0.48499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":5.0460000000000003,"5min":5.0460000000000003,"15min":5.0460000000000003},"last":0.85099999999999998}]},{"osd":7,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.35399999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.621,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.88900000000000001}]},{"osd":8,"last update":"Mon Mar 18 20:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67800000000000005,"15min":0.67200000000000004},"min":{"1min":0.55300000000000005,"5min":0.438,"15min":0.438},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68799999999999994,"15min":0.67600000000000005},"min":{"1min":0.67500000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90300000000000002,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.873}]},{"osd":9,"last update":"Mon Mar 18 20:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63900000000000001,"15min":0.63},"min":{"1min":0.57499999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.65400000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82099999999999995}]},{"osd":10,"last update":"Mon Mar 18 20:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.49399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90500000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66500000000000004,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.86099999999999999}]},{"osd":11,"last update":"Mon Mar 18 20:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.72999999999999998}]}]},{"osd":5,"up_from":35,"seq":150323855406,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1503232,"data_stored":1014941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53300000000000003,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.745,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.61299999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54300000000000004,"15min":0.53400000000000003},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.57699999999999996,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57199999999999995}]},{"osd":2,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.39500000000000002,"15min":0.38},"min":{"1min":0.29799999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.435},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.40200000000000002,"15min":0.38400000000000001},"min":{"1min":0.317,"5min":0.25,"15min":0.25},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38400000000000001}]},{"osd":3,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.746,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.372,"15min":0.372},"max":{"1min":0.69299999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53300000000000003}]},{"osd":4,"last update":"Mon Mar 18 20:35:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.53100000000000003,"5min":0.372,"15min":0.372},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.41799999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Mon Mar 18 20:35:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60099999999999998,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.63200000000000001,"15min":0.629},"min":{"1min":0.496,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.68400000000000005}]},{"osd":7,"last update":"Mon Mar 18 20:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.72199999999999998}]},{"osd":8,"last update":"Mon Mar 18 20:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80400000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.626}]},{"osd":9,"last update":"Mon Mar 18 20:35:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.60599999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.73699999999999999,"15min":0.72499999999999998},"min":{"1min":0.625,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.73799999999999999}]},{"osd":10,"last update":"Mon Mar 18 20:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.57899999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.97599999999999998,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.56899999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T20:35:55.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:35:56.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.156+0000 7f39c3f48700 1 -- 172.21.15.46:0/2690557819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc100c80 msgr2=0x7f39bc1010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.156+0000 7f39c3f48700 1 --2- 172.21.15.46:0/2690557819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc100c80 0x7f39bc1010a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f39b80099c0 tx=0x7f39b8009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.156+0000 7f39c3f48700 1 -- 172.21.15.46:0/2690557819 shutdown_connections 2024-03-18T20:35:56.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.156+0000 7f39c3f48700 1 --2- 172.21.15.46:0/2690557819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc100c80 0x7f39bc1010a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.156+0000 7f39c3f48700 1 --2- 172.21.15.46:0/2690557819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc102810 0x7f39bc1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.156+0000 7f39c3f48700 1 --2- 172.21.15.46:0/2690557819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39bc101e70 0x7f39bc1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.156+0000 7f39c3f48700 1 -- 172.21.15.46:0/2690557819 >> 172.21.15.46:0/2690557819 conn(0x7f39bc0fc1f0 msgr2=0x7f39bc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.156+0000 7f39c3f48700 1 -- 172.21.15.46:0/2690557819 shutdown_connections 2024-03-18T20:35:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.156+0000 7f39c3f48700 1 -- 172.21.15.46:0/2690557819 wait complete. 2024-03-18T20:35:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c3f48700 1 Processor -- start 2024-03-18T20:35:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c3f48700 1 -- start start 2024-03-18T20:35:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c3f48700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39bc100c80 0x7f39bc196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c3f48700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc101e70 0x7f39bc197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c3f48700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc102810 0x7f39bc19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c3f48700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc19e760 con 0x7f39bc102810 2024-03-18T20:35:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c3f48700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc19e8a0 con 0x7f39bc101e70 2024-03-18T20:35:56.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c3f48700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc19ea30 con 0x7f39bc100c80 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c24e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc102810 0x7f39bc19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c14e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc101e70 0x7f39bc197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c24e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc102810 0x7f39bc19e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38664/0 (socket says 172.21.15.46:38664) 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c14e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc101e70 0x7f39bc197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33604/0 (socket says 172.21.15.46:33604) 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c24e5700 1 -- 172.21.15.46:0/1476265318 learned_addr learned my addr 172.21.15.46:0/1476265318 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c1ce4700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39bc100c80 0x7f39bc196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c14e3700 1 -- 172.21.15.46:0/1476265318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39bc100c80 msgr2=0x7f39bc196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c14e3700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39bc100c80 0x7f39bc196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c14e3700 1 -- 172.21.15.46:0/1476265318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc102810 msgr2=0x7f39bc19e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c14e3700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc102810 0x7f39bc19e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c14e3700 1 -- 172.21.15.46:0/1476265318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b80095f0 con 0x7f39bc101e70 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.157+0000 7f39c24e5700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc102810 0x7f39bc19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.158+0000 7f39c14e3700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc101e70 0x7f39bc197170 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00e9f0 tx=0x7f39ac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:56.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.158+0000 7f39b2ffd700 1 -- 172.21.15.46:0/1476265318 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00c9c0 con 0x7f39bc101e70 2024-03-18T20:35:56.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.158+0000 7f39c3f48700 1 -- 172.21.15.46:0/1476265318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc19ed10 con 0x7f39bc101e70 2024-03-18T20:35:56.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.158+0000 7f39b2ffd700 1 -- 172.21.15.46:0/1476265318 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39ac010430 con 0x7f39bc101e70 2024-03-18T20:35:56.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.158+0000 7f39b2ffd700 1 -- 172.21.15.46:0/1476265318 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac018680 con 0x7f39bc101e70 2024-03-18T20:35:56.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.158+0000 7f39c3f48700 1 -- 172.21.15.46:0/1476265318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc19f1c0 con 0x7f39bc101e70 2024-03-18T20:35:56.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.159+0000 7f39c3f48700 1 -- 172.21.15.46:0/1476265318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc108150 con 0x7f39bc101e70 2024-03-18T20:35:56.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.160+0000 7f39b2ffd700 1 -- 172.21.15.46:0/1476265318 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f39ac010860 con 0x7f39bc101e70 2024-03-18T20:35:56.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.161+0000 7f39b2ffd700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f39a80a11b0 0x7f39a80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:56.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.161+0000 7f39c1ce4700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f39a80a11b0 0x7f39a80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:56.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.161+0000 7f39b2ffd700 1 -- 172.21.15.46:0/1476265318 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f39ac014070 con 0x7f39bc101e70 2024-03-18T20:35:56.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.161+0000 7f39c1ce4700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f39a80a11b0 0x7f39a80a3670 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009990 tx=0x7f39b8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:56.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.165+0000 7f39b2ffd700 1 -- 172.21.15.46:0/1476265318 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f39ac096140 con 0x7f39bc101e70 2024-03-18T20:35:56.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:56 smithi046 ceph-mon[43495]: pgmap v259: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-18T20:35:56.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:56 smithi046 ceph-mon[43495]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.403+0000 7f39c3f48700 1 -- 172.21.15.46:0/1476265318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39bc067a10 con 0x7f39bc101e70 2024-03-18T20:35:56.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.406+0000 7f39b2ffd700 1 -- 172.21.15.46:0/1476265318 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22503 (secure 0 0 0) 0x7f39ac099760 con 0x7f39bc101e70 2024-03-18T20:35:56.409 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:56.409 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:35:38.108050+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:35:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.408+0000 7f39c3f48700 1 -- 172.21.15.46:0/1476265318 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f39a80a11b0 msgr2=0x7f39a80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.408+0000 7f39c3f48700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f39a80a11b0 0x7f39a80a3670 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009990 tx=0x7f39b8000bc0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.408+0000 7f39c3f48700 1 -- 172.21.15.46:0/1476265318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc101e70 msgr2=0x7f39bc197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.408+0000 7f39c3f48700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc101e70 0x7f39bc197170 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00e9f0 tx=0x7f39ac00ed00 comp rx=0 tx=0).stop 2024-03-18T20:35:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.408+0000 7f39c3f48700 1 -- 172.21.15.46:0/1476265318 shutdown_connections 2024-03-18T20:35:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.408+0000 7f39c3f48700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f39a80a11b0 0x7f39a80a3670 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.408+0000 7f39c3f48700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39bc102810 0x7f39bc19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.408+0000 7f39c3f48700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39bc100c80 0x7f39bc196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.408+0000 7f39c3f48700 1 --2- 172.21.15.46:0/1476265318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39bc101e70 0x7f39bc197170 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.408+0000 7f39c3f48700 1 -- 172.21.15.46:0/1476265318 >> 172.21.15.46:0/1476265318 conn(0x7f39bc0fc1f0 msgr2=0x7f39bc105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.408+0000 7f39c3f48700 1 -- 172.21.15.46:0/1476265318 shutdown_connections 2024-03-18T20:35:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.408+0000 7f39c3f48700 1 -- 172.21.15.46:0/1476265318 wait complete. 2024-03-18T20:35:56.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T20:31:27.488318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T20:34:37.213560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T20:34:38.162692+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T20:35:56.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T20:35:56.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:56 smithi073 ceph-mon[47384]: pgmap v259: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-18T20:35:56.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:56 smithi073 ceph-mon[47384]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:56.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:56 smithi155 ceph-mon[48344]: pgmap v259: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-18T20:35:56.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:56 smithi155 ceph-mon[48344]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:56.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.563+0000 7fe3bba4d700 1 -- 172.21.15.46:0/3521398179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3b4101810 msgr2=0x7fe3b4103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.563+0000 7fe3bba4d700 1 --2- 172.21.15.46:0/3521398179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3b4101810 0x7fe3b4103c00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009990 tx=0x7fe3b0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.563+0000 7fe3bba4d700 1 -- 172.21.15.46:0/3521398179 shutdown_connections 2024-03-18T20:35:56.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.563+0000 7fe3bba4d700 1 --2- 172.21.15.46:0/3521398179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3b4101810 0x7fe3b4103c00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.563+0000 7fe3bba4d700 1 --2- 172.21.15.46:0/3521398179 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b4106a70 0x7fe3b4108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.563+0000 7fe3bba4d700 1 --2- 172.21.15.46:0/3521398179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3b4104140 0x7fe3b4106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.563+0000 7fe3bba4d700 1 -- 172.21.15.46:0/3521398179 >> 172.21.15.46:0/3521398179 conn(0x7fe3b40fb200 msgr2=0x7fe3b40fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:56.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.563+0000 7fe3bba4d700 1 -- 172.21.15.46:0/3521398179 shutdown_connections 2024-03-18T20:35:56.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.563+0000 7fe3bba4d700 1 -- 172.21.15.46:0/3521398179 wait complete. 2024-03-18T20:35:56.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3bba4d700 1 Processor -- start 2024-03-18T20:35:56.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3bba4d700 1 -- start start 2024-03-18T20:35:56.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3bba4d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b4101810 0x7fe3b4196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3bba4d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3b4104140 0x7fe3b4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3bba4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3b4106a70 0x7fe3b419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3bba4d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b419e7b0 con 0x7fe3b4106a70 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3bba4d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b419e920 con 0x7fe3b4104140 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3bba4d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b419eae0 con 0x7fe3b4101810 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3b9fea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3b4106a70 0x7fe3b419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3b9fea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3b4106a70 0x7fe3b419e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38690/0 (socket says 172.21.15.46:38690) 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3b9fea700 1 -- 172.21.15.46:0/89815885 learned_addr learned my addr 172.21.15.46:0/89815885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3b97e9700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b4101810 0x7fe3b4196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3b8fe8700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3b4104140 0x7fe3b4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3b9fea700 1 -- 172.21.15.46:0/89815885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b4101810 msgr2=0x7fe3b4196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3b9fea700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b4101810 0x7fe3b4196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3b9fea700 1 -- 172.21.15.46:0/89815885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3b4104140 msgr2=0x7fe3b4197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3b9fea700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3b4104140 0x7fe3b4197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.564+0000 7fe3b9fea700 1 -- 172.21.15.46:0/89815885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b00095f0 con 0x7fe3b4106a70 2024-03-18T20:35:56.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.565+0000 7fe3b9fea700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3b4106a70 0x7fe3b419e1e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00ef30 tx=0x7fe3ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:56.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.565+0000 7fe3a67fc700 1 -- 172.21.15.46:0/89815885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac018070 con 0x7fe3b4106a70 2024-03-18T20:35:56.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.565+0000 7fe3a67fc700 1 -- 172.21.15.46:0/89815885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3ac00f040 con 0x7fe3b4106a70 2024-03-18T20:35:56.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.565+0000 7fe3bba4d700 1 -- 172.21.15.46:0/89815885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b419ed60 con 0x7fe3b4106a70 2024-03-18T20:35:56.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.565+0000 7fe3a67fc700 1 -- 172.21.15.46:0/89815885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00cd00 con 0x7fe3b4106a70 2024-03-18T20:35:56.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.565+0000 7fe3bba4d700 1 -- 172.21.15.46:0/89815885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b419f200 con 0x7fe3b4106a70 2024-03-18T20:35:56.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.566+0000 7fe3bba4d700 1 -- 172.21.15.46:0/89815885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b40fcd80 con 0x7fe3b4106a70 2024-03-18T20:35:56.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.567+0000 7fe3a67fc700 1 -- 172.21.15.46:0/89815885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fe3ac003a00 con 0x7fe3b4106a70 2024-03-18T20:35:56.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.568+0000 7fe3a67fc700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe3a00a1230 0x7fe3a00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:56.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.568+0000 7fe3a67fc700 1 -- 172.21.15.46:0/89815885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe3ac0cced0 con 0x7fe3b4106a70 2024-03-18T20:35:56.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.568+0000 7fe3b97e9700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe3a00a1230 0x7fe3a00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:56.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.568+0000 7fe3b97e9700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe3a00a1230 0x7fe3a00a36f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000b680 tx=0x7fe3b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.571+0000 7fe3a67fc700 1 -- 172.21.15.46:0/89815885 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe3ac096540 con 0x7fe3b4106a70 2024-03-18T20:35:56.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.786+0000 7fe3bba4d700 1 -- 172.21.15.46:0/89815885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3b402cdc0 con 0x7fe3b4106a70 2024-03-18T20:35:56.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.786+0000 7fe3a67fc700 1 -- 172.21.15.46:0/89815885 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3ac025380 con 0x7fe3b4106a70 2024-03-18T20:35:56.789 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-18T20:35:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.788+0000 7fe3bba4d700 1 -- 172.21.15.46:0/89815885 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe3a00a1230 msgr2=0x7fe3a00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.788+0000 7fe3bba4d700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe3a00a1230 0x7fe3a00a36f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000b680 tx=0x7fe3b0005fd0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.788+0000 7fe3bba4d700 1 -- 172.21.15.46:0/89815885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3b4106a70 msgr2=0x7fe3b419e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.788+0000 7fe3bba4d700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3b4106a70 0x7fe3b419e1e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00ef30 tx=0x7fe3ac00c3a0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.789+0000 7fe3bba4d700 1 -- 172.21.15.46:0/89815885 shutdown_connections 2024-03-18T20:35:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.789+0000 7fe3bba4d700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fe3a00a1230 0x7fe3a00a36f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.789+0000 7fe3bba4d700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3b4106a70 0x7fe3b419e1e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.789+0000 7fe3bba4d700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b4101810 0x7fe3b4196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.789+0000 7fe3bba4d700 1 --2- 172.21.15.46:0/89815885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3b4104140 0x7fe3b4197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.789+0000 7fe3bba4d700 1 -- 172.21.15.46:0/89815885 >> 172.21.15.46:0/89815885 conn(0x7fe3b40fb200 msgr2=0x7fe3b41036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.789+0000 7fe3bba4d700 1 -- 172.21.15.46:0/89815885 shutdown_connections 2024-03-18T20:35:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.789+0000 7fe3bba4d700 1 -- 172.21.15.46:0/89815885 wait complete. 2024-03-18T20:35:56.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T20:35:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.943+0000 7f0363485700 1 -- 172.21.15.46:0/782954145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c1017f0 msgr2=0x7f035c1080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.943+0000 7f0363485700 1 --2- 172.21.15.46:0/782954145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c1017f0 0x7f035c1080a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f035400b0d0 tx=0x7f035400b3e0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.943+0000 7f0363485700 1 -- 172.21.15.46:0/782954145 shutdown_connections 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.943+0000 7f0363485700 1 --2- 172.21.15.46:0/782954145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c1017f0 0x7f035c1080a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.943+0000 7f0363485700 1 --2- 172.21.15.46:0/782954145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f035c0ffc60 0x7f035c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.943+0000 7f0363485700 1 --2- 172.21.15.46:0/782954145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c100e50 0x7f035c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.943+0000 7f0363485700 1 -- 172.21.15.46:0/782954145 >> 172.21.15.46:0/782954145 conn(0x7f035c0fb1d0 msgr2=0x7f035c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.943+0000 7f0363485700 1 -- 172.21.15.46:0/782954145 shutdown_connections 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.943+0000 7f0363485700 1 -- 172.21.15.46:0/782954145 wait complete. 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0363485700 1 Processor -- start 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0363485700 1 -- start start 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0363485700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0ffc60 0x7f035c196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0363485700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f035c100e50 0x7f035c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0363485700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c1017f0 0x7f035c19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0363485700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c19e8e0 con 0x7f035c0ffc60 2024-03-18T20:35:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0363485700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c19ea20 con 0x7f035c1017f0 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0363485700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c19ebb0 con 0x7f035c100e50 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0361221700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0ffc60 0x7f035c196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0361221700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0ffc60 0x7f035c196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38702/0 (socket says 172.21.15.46:38702) 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0361221700 1 -- 172.21.15.46:0/2568849721 learned_addr learned my addr 172.21.15.46:0/2568849721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0360a20700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f035c100e50 0x7f035c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0361a22700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c1017f0 0x7f035c19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0361221700 1 -- 172.21.15.46:0/2568849721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f035c100e50 msgr2=0x7f035c197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0361221700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f035c100e50 0x7f035c197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0361221700 1 -- 172.21.15.46:0/2568849721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c1017f0 msgr2=0x7f035c19e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0361221700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c1017f0 0x7f035c19e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.944+0000 7f0361221700 1 -- 172.21.15.46:0/2568849721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0354009d00 con 0x7f035c0ffc60 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.945+0000 7f0361221700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0ffc60 0x7f035c196e00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f035800e970 tx=0x7f035800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.945+0000 7f03527fc700 1 -- 172.21.15.46:0/2568849721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358009790 con 0x7f035c0ffc60 2024-03-18T20:35:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.945+0000 7f03527fc700 1 -- 172.21.15.46:0/2568849721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f035800c7e0 con 0x7f035c0ffc60 2024-03-18T20:35:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.945+0000 7f0363485700 1 -- 172.21.15.46:0/2568849721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035c19ee90 con 0x7f035c0ffc60 2024-03-18T20:35:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.945+0000 7f03527fc700 1 -- 172.21.15.46:0/2568849721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358003c70 con 0x7f035c0ffc60 2024-03-18T20:35:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.945+0000 7f0363485700 1 -- 172.21.15.46:0/2568849721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035c19f2b0 con 0x7f035c0ffc60 2024-03-18T20:35:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.946+0000 7f0363485700 1 -- 172.21.15.46:0/2568849721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035c067af0 con 0x7f035c0ffc60 2024-03-18T20:35:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.947+0000 7f03527fc700 1 -- 172.21.15.46:0/2568849721 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f035800cc50 con 0x7f035c0ffc60 2024-03-18T20:35:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.947+0000 7f03527fc700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f03480a11b0 0x7f03480a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.948+0000 7f03527fc700 1 -- 172.21.15.46:0/2568849721 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0358014070 con 0x7f035c0ffc60 2024-03-18T20:35:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.948+0000 7f0360a20700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f03480a11b0 0x7f03480a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.948+0000 7f0360a20700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f03480a11b0 0x7f03480a3670 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f034c009c60 tx=0x7f034c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:56.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:56.951+0000 7f03527fc700 1 -- 172.21.15.46:0/2568849721 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0358096340 con 0x7f035c0ffc60 2024-03-18T20:35:57.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.167+0000 7f0363485700 1 -- 172.21.15.46:0/2568849721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f035c19f500 con 0x7f035c0ffc60 2024-03-18T20:35:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.167+0000 7f03527fc700 1 -- 172.21.15.46:0/2568849721 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0358003dd0 con 0x7f035c0ffc60 2024-03-18T20:35:57.171 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-18T20:35:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.170+0000 7f0363485700 1 -- 172.21.15.46:0/2568849721 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f03480a11b0 msgr2=0x7f03480a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.170+0000 7f0363485700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f03480a11b0 0x7f03480a3670 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f034c009c60 tx=0x7f034c009260 comp rx=0 tx=0).stop 2024-03-18T20:35:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.170+0000 7f0363485700 1 -- 172.21.15.46:0/2568849721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0ffc60 msgr2=0x7f035c196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.170+0000 7f0363485700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0ffc60 0x7f035c196e00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f035800e970 tx=0x7f035800ec80 comp rx=0 tx=0).stop 2024-03-18T20:35:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.170+0000 7f0363485700 1 -- 172.21.15.46:0/2568849721 shutdown_connections 2024-03-18T20:35:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.170+0000 7f0363485700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f03480a11b0 0x7f03480a3670 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.170+0000 7f0363485700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0ffc60 0x7f035c196e00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.170+0000 7f0363485700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c1017f0 0x7f035c19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.170+0000 7f0363485700 1 --2- 172.21.15.46:0/2568849721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f035c100e50 0x7f035c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.170+0000 7f0363485700 1 -- 172.21.15.46:0/2568849721 >> 172.21.15.46:0/2568849721 conn(0x7f035c0fb1d0 msgr2=0x7f035c0fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.170+0000 7f0363485700 1 -- 172.21.15.46:0/2568849721 shutdown_connections 2024-03-18T20:35:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.170+0000 7f0363485700 1 -- 172.21.15.46:0/2568849721 wait complete. 2024-03-18T20:35:57.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T20:35:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.326+0000 7fc99e1a8700 1 -- 172.21.15.46:0/3844830888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc998101840 msgr2=0x7fc998103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.326+0000 7fc99e1a8700 1 --2- 172.21.15.46:0/3844830888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc998101840 0x7fc998103c30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc988009990 tx=0x7fc988009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.326+0000 7fc99e1a8700 1 -- 172.21.15.46:0/3844830888 shutdown_connections 2024-03-18T20:35:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.326+0000 7fc99e1a8700 1 --2- 172.21.15.46:0/3844830888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc998101840 0x7fc998103c30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.326+0000 7fc99e1a8700 1 --2- 172.21.15.46:0/3844830888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc998106aa0 0x7fc998108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.326+0000 7fc99e1a8700 1 --2- 172.21.15.46:0/3844830888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc998104170 0x7fc998106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.326+0000 7fc99e1a8700 1 -- 172.21.15.46:0/3844830888 >> 172.21.15.46:0/3844830888 conn(0x7fc9980fb1d0 msgr2=0x7fc9980fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:57.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.326+0000 7fc99e1a8700 1 -- 172.21.15.46:0/3844830888 shutdown_connections 2024-03-18T20:35:57.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.326+0000 7fc99e1a8700 1 -- 172.21.15.46:0/3844830888 wait complete. 2024-03-18T20:35:57.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.327+0000 7fc99e1a8700 1 Processor -- start 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.327+0000 7fc99e1a8700 1 -- start start 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.327+0000 7fc99e1a8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc998101840 0x7fc998196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.327+0000 7fc99e1a8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc998104170 0x7fc9981971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.327+0000 7fc99e1a8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc998106aa0 0x7fc99819e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.327+0000 7fc99e1a8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99819e7a0 con 0x7fc998101840 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.327+0000 7fc99e1a8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99819e910 con 0x7fc998104170 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.327+0000 7fc99e1a8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99819ead0 con 0x7fc998106aa0 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.327+0000 7fc9977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc998101840 0x7fc998196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.327+0000 7fc9977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc998101840 0x7fc998196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38722/0 (socket says 172.21.15.46:38722) 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc9977fe700 1 -- 172.21.15.46:0/1815465174 learned_addr learned my addr 172.21.15.46:0/1815465174 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc997fff700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc998106aa0 0x7fc99819e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc996ffd700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc998104170 0x7fc9981971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:57.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc9977fe700 1 -- 172.21.15.46:0/1815465174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc998106aa0 msgr2=0x7fc99819e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc9977fe700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc998106aa0 0x7fc99819e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc9977fe700 1 -- 172.21.15.46:0/1815465174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc998104170 msgr2=0x7fc9981971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc9977fe700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc998104170 0x7fc9981971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc9977fe700 1 -- 172.21.15.46:0/1815465174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9880095f0 con 0x7fc998101840 2024-03-18T20:35:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc9977fe700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc998101840 0x7fc998196cb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc9880095c0 tx=0x7fc98800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc994ff9700 1 -- 172.21.15.46:0/1815465174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98801d070 con 0x7fc998101840 2024-03-18T20:35:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc994ff9700 1 -- 172.21.15.46:0/1815465174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc988005030 con 0x7fc998101840 2024-03-18T20:35:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc99e1a8700 1 -- 172.21.15.46:0/1815465174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99819ed50 con 0x7fc998101840 2024-03-18T20:35:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc994ff9700 1 -- 172.21.15.46:0/1815465174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988004b00 con 0x7fc998101840 2024-03-18T20:35:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.328+0000 7fc99e1a8700 1 -- 172.21.15.46:0/1815465174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99819f170 con 0x7fc998101840 2024-03-18T20:35:57.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.329+0000 7fc99e1a8700 1 -- 172.21.15.46:0/1815465174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9980fcd50 con 0x7fc998101840 2024-03-18T20:35:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.331+0000 7fc994ff9700 1 -- 172.21.15.46:0/1815465174 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc988005410 con 0x7fc998101840 2024-03-18T20:35:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.331+0000 7fc994ff9700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc9840a11b0 0x7fc9840a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.331+0000 7fc994ff9700 1 -- 172.21.15.46:0/1815465174 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc9880cdd90 con 0x7fc998101840 2024-03-18T20:35:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.331+0000 7fc996ffd700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc9840a11b0 0x7fc9840a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:57.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.332+0000 7fc996ffd700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc9840a11b0 0x7fc9840a3670 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc980005950 tx=0x7fc98000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.335+0000 7fc994ff9700 1 -- 172.21.15.46:0/1815465174 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc988097460 con 0x7fc998101840 2024-03-18T20:35:57.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:57 smithi073 ceph-mon[47384]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:57.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:57 smithi073 ceph-mon[47384]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:57.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:57 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1476265318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:57.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:57 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/89815885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:57.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:57 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2568849721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:57.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:57 smithi155 ceph-mon[48344]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:57.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:57 smithi155 ceph-mon[48344]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:57.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:57 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1476265318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:57.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:57 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/89815885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:57.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:57 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2568849721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:57.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:57 smithi046 ceph-mon[43495]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:57.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:57 smithi046 ceph-mon[43495]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T20:35:57.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:57 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1476265318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:57.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:57 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/89815885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T20:35:57.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:57 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2568849721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T20:35:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.551+0000 7fc99e1a8700 1 -- 172.21.15.46:0/1815465174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc99802d0d0 con 0x7fc998101840 2024-03-18T20:35:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.551+0000 7fc994ff9700 1 -- 172.21.15.46:0/1815465174 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc98809aa80 con 0x7fc998101840 2024-03-18T20:35:57.554 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-18T20:35:57.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.553+0000 7fc99e1a8700 1 -- 172.21.15.46:0/1815465174 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc9840a11b0 msgr2=0x7fc9840a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:57.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.553+0000 7fc99e1a8700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc9840a11b0 0x7fc9840a3670 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc980005950 tx=0x7fc98000b410 comp rx=0 tx=0).stop 2024-03-18T20:35:57.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.553+0000 7fc99e1a8700 1 -- 172.21.15.46:0/1815465174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc998101840 msgr2=0x7fc998196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:57.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.553+0000 7fc99e1a8700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc998101840 0x7fc998196cb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc9880095c0 tx=0x7fc98800f690 comp rx=0 tx=0).stop 2024-03-18T20:35:57.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.553+0000 7fc99e1a8700 1 -- 172.21.15.46:0/1815465174 shutdown_connections 2024-03-18T20:35:57.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.553+0000 7fc99e1a8700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fc9840a11b0 0x7fc9840a3670 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.553+0000 7fc99e1a8700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc998101840 0x7fc998196cb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.553+0000 7fc99e1a8700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc998106aa0 0x7fc99819e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.553+0000 7fc99e1a8700 1 --2- 172.21.15.46:0/1815465174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc998104170 0x7fc9981971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.554+0000 7fc99e1a8700 1 -- 172.21.15.46:0/1815465174 >> 172.21.15.46:0/1815465174 conn(0x7fc9980fb1d0 msgr2=0x7fc9981036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:57.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.554+0000 7fc99e1a8700 1 -- 172.21.15.46:0/1815465174 shutdown_connections 2024-03-18T20:35:57.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.554+0000 7fc99e1a8700 1 -- 172.21.15.46:0/1815465174 wait complete. 2024-03-18T20:35:57.571 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:35:57.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.704+0000 7f463963c700 1 -- 172.21.15.46:0/3607691708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340ffb60 msgr2=0x7f46340fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.704+0000 7f463963c700 1 --2- 172.21.15.46:0/3607691708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340ffb60 0x7f46340fff80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f46240099c0 tx=0x7f4624009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.704+0000 7f463963c700 1 -- 172.21.15.46:0/3607691708 shutdown_connections 2024-03-18T20:35:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.704+0000 7f463963c700 1 --2- 172.21.15.46:0/3607691708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340ffb60 0x7f46340fff80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.704+0000 7f463963c700 1 --2- 172.21.15.46:0/3607691708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46341016f0 0x7f4634107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.704+0000 7f463963c700 1 --2- 172.21.15.46:0/3607691708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4634100d50 0x7f46341011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.704+0000 7f463963c700 1 -- 172.21.15.46:0/3607691708 >> 172.21.15.46:0/3607691708 conn(0x7f46340fb0d0 msgr2=0x7f46340fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.704+0000 7f463963c700 1 -- 172.21.15.46:0/3607691708 shutdown_connections 2024-03-18T20:35:57.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.704+0000 7f463963c700 1 -- 172.21.15.46:0/3607691708 wait complete. 2024-03-18T20:35:57.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.705+0000 7f463963c700 1 Processor -- start 2024-03-18T20:35:57.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.705+0000 7f463963c700 1 -- start start 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.705+0000 7f463963c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340ffb60 0x7f4634103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.705+0000 7f463963c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4634100d50 0x7f4634102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.705+0000 7f463963c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46341016f0 0x7f4634102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.705+0000 7f463963c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4634102e10 con 0x7f46340ffb60 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.705+0000 7f463963c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4634102f80 con 0x7f46341016f0 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.705+0000 7f463963c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46341a0c40 con 0x7f4634100d50 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.705+0000 7f4632ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340ffb60 0x7f4634103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.705+0000 7f4632ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340ffb60 0x7f4634103aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38740/0 (socket says 172.21.15.46:38740) 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.705+0000 7f4632ffd700 1 -- 172.21.15.46:0/2441417836 learned_addr learned my addr 172.21.15.46:0/2441417836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.705+0000 7f46327fc700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4634100d50 0x7f4634102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.706+0000 7f46337fe700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46341016f0 0x7f4634102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.706+0000 7f4632ffd700 1 -- 172.21.15.46:0/2441417836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4634100d50 msgr2=0x7f4634102130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.706+0000 7f4632ffd700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4634100d50 0x7f4634102130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.706+0000 7f4632ffd700 1 -- 172.21.15.46:0/2441417836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46341016f0 msgr2=0x7f4634102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.706+0000 7f4632ffd700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46341016f0 0x7f4634102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.706+0000 7f4632ffd700 1 -- 172.21.15.46:0/2441417836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46240095f0 con 0x7f46340ffb60 2024-03-18T20:35:57.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.706+0000 7f4632ffd700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340ffb60 0x7f4634103aa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4624006010 tx=0x7f4624005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:57.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.706+0000 7f461bfff700 1 -- 172.21.15.46:0/2441417836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462401d070 con 0x7f46340ffb60 2024-03-18T20:35:57.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.706+0000 7f463963c700 1 -- 172.21.15.46:0/2441417836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46341a0e70 con 0x7f46340ffb60 2024-03-18T20:35:57.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.706+0000 7f461bfff700 1 -- 172.21.15.46:0/2441417836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4624005940 con 0x7f46340ffb60 2024-03-18T20:35:57.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.706+0000 7f461bfff700 1 -- 172.21.15.46:0/2441417836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624022b60 con 0x7f46340ffb60 2024-03-18T20:35:57.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.706+0000 7f463963c700 1 -- 172.21.15.46:0/2441417836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46341a1230 con 0x7f46340ffb60 2024-03-18T20:35:57.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.707+0000 7f463963c700 1 -- 172.21.15.46:0/2441417836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4634067a10 con 0x7f46340ffb60 2024-03-18T20:35:57.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.708+0000 7f461bfff700 1 -- 172.21.15.46:0/2441417836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4624003ad0 con 0x7f46340ffb60 2024-03-18T20:35:57.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.709+0000 7f461bfff700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f461c0a11b0 0x7f461c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:57.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.709+0000 7f461bfff700 1 -- 172.21.15.46:0/2441417836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f46240ce070 con 0x7f46340ffb60 2024-03-18T20:35:57.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.709+0000 7f46327fc700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f461c0a11b0 0x7f461c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:57.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.710+0000 7f46327fc700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f461c0a11b0 0x7f461c0a3670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4628005950 tx=0x7f462800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.713+0000 7f461bfff700 1 -- 172.21.15.46:0/2441417836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f46240976e0 con 0x7f46340ffb60 2024-03-18T20:35:57.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.979+0000 7f463963c700 1 -- 172.21.15.46:0/2441417836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46341a1670 con 0x7f46340ffb60 2024-03-18T20:35:57.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.979+0000 7f461bfff700 1 -- 172.21.15.46:0/2441417836 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f462409ad00 con 0x7f46340ffb60 2024-03-18T20:35:57.983 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:57.983 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:35:51.152306+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:35:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.982+0000 7f463963c700 1 -- 172.21.15.46:0/2441417836 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f461c0a11b0 msgr2=0x7f461c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.982+0000 7f463963c700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f461c0a11b0 0x7f461c0a3670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4628005950 tx=0x7f462800b410 comp rx=0 tx=0).stop 2024-03-18T20:35:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.982+0000 7f463963c700 1 -- 172.21.15.46:0/2441417836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340ffb60 msgr2=0x7f4634103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.982+0000 7f463963c700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340ffb60 0x7f4634103aa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4624006010 tx=0x7f4624005740 comp rx=0 tx=0).stop 2024-03-18T20:35:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.982+0000 7f463963c700 1 -- 172.21.15.46:0/2441417836 shutdown_connections 2024-03-18T20:35:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.982+0000 7f463963c700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f461c0a11b0 0x7f461c0a3670 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.982+0000 7f463963c700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340ffb60 0x7f4634103aa0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.982+0000 7f463963c700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46341016f0 0x7f4634102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.982+0000 7f463963c700 1 --2- 172.21.15.46:0/2441417836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4634100d50 0x7f4634102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.982+0000 7f463963c700 1 -- 172.21.15.46:0/2441417836 >> 172.21.15.46:0/2441417836 conn(0x7f46340fb0d0 msgr2=0x7f4634067df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.982+0000 7f463963c700 1 -- 172.21.15.46:0/2441417836 shutdown_connections 2024-03-18T20:35:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:57.982+0000 7f463963c700 1 -- 172.21.15.46:0/2441417836 wait complete. 2024-03-18T20:35:57.987 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-03-18T20:35:58.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T20:35:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.133+0000 7fb1c1325700 1 -- 172.21.15.46:0/128421300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1bc100870 msgr2=0x7fb1bc100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.133+0000 7fb1c1325700 1 --2- 172.21.15.46:0/128421300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1bc100870 0x7fb1bc100cf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400b0a0 tx=0x7fb1b400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.133+0000 7fb1c1325700 1 -- 172.21.15.46:0/128421300 shutdown_connections 2024-03-18T20:35:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.133+0000 7fb1c1325700 1 --2- 172.21.15.46:0/128421300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1bc100870 0x7fb1bc100cf0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.133+0000 7fb1c1325700 1 --2- 172.21.15.46:0/128421300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc0ff550 0x7fb1bc0ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.133+0000 7fb1c1325700 1 --2- 172.21.15.46:0/128421300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1bc0ffeb0 0x7fb1bc100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.133+0000 7fb1c1325700 1 -- 172.21.15.46:0/128421300 >> 172.21.15.46:0/128421300 conn(0x7fb1bc0fb0d0 msgr2=0x7fb1bc0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.133+0000 7fb1c1325700 1 -- 172.21.15.46:0/128421300 shutdown_connections 2024-03-18T20:35:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.133+0000 7fb1c1325700 1 -- 172.21.15.46:0/128421300 wait complete. 2024-03-18T20:35:58.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.134+0000 7fb1c1325700 1 Processor -- start 2024-03-18T20:35:58.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.134+0000 7fb1c1325700 1 -- start start 2024-03-18T20:35:58.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.134+0000 7fb1c1325700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc0ff550 0x7fb1bc198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.134+0000 7fb1c1325700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1bc0ffeb0 0x7fb1bc1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.134+0000 7fb1c1325700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1bc100870 0x7fb1bc1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.134+0000 7fb1c1325700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1a09e0 con 0x7fb1bc100870 2024-03-18T20:35:58.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.134+0000 7fb1c1325700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1a0b20 con 0x7fb1bc0ffeb0 2024-03-18T20:35:58.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.134+0000 7fb1c1325700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1a0cb0 con 0x7fb1bc0ff550 2024-03-18T20:35:58.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1bb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1bc100870 0x7fb1bc1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1bb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1bc100870 0x7fb1bc1a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38750/0 (socket says 172.21.15.46:38750) 2024-03-18T20:35:58.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1bb7fe700 1 -- 172.21.15.46:0/3635930712 learned_addr learned my addr 172.21.15.46:0/3635930712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1baffd700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc0ff550 0x7fb1bc198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1ba7fc700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1bc0ffeb0 0x7fb1bc1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1bb7fe700 1 -- 172.21.15.46:0/3635930712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc0ff550 msgr2=0x7fb1bc198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1bb7fe700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc0ff550 0x7fb1bc198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1bb7fe700 1 -- 172.21.15.46:0/3635930712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1bc0ffeb0 msgr2=0x7fb1bc1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1bb7fe700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1bc0ffeb0 0x7fb1bc1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1bb7fe700 1 -- 172.21.15.46:0/3635930712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b4009d00 con 0x7fb1bc100870 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1bb7fe700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1bc100870 0x7fb1bc1a0410 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400b780 tx=0x7fb1b400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1a3fff700 1 -- 172.21.15.46:0/3635930712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b400e040 con 0x7fb1bc100870 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1a3fff700 1 -- 172.21.15.46:0/3635930712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1b4003e10 con 0x7fb1bc100870 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1c1325700 1 -- 172.21.15.46:0/3635930712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1bc1a0f30 con 0x7fb1bc100870 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1a3fff700 1 -- 172.21.15.46:0/3635930712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4005330 con 0x7fb1bc100870 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.135+0000 7fb1c1325700 1 -- 172.21.15.46:0/3635930712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc1a12f0 con 0x7fb1bc100870 2024-03-18T20:35:58.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.136+0000 7fb1c1325700 1 -- 172.21.15.46:0/3635930712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1bc10a510 con 0x7fb1bc100870 2024-03-18T20:35:58.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.138+0000 7fb1a3fff700 1 -- 172.21.15.46:0/3635930712 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb1b401d9d0 con 0x7fb1bc100870 2024-03-18T20:35:58.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.138+0000 7fb1a3fff700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb1a40a11b0 0x7fb1a40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.138+0000 7fb1baffd700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb1a40a11b0 0x7fb1a40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.138+0000 7fb1a3fff700 1 -- 172.21.15.46:0/3635930712 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb1b40cdc80 con 0x7fb1bc100870 2024-03-18T20:35:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.139+0000 7fb1baffd700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb1a40a11b0 0x7fb1a40a3670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009bf0 tx=0x7fb1ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.142+0000 7fb1a3fff700 1 -- 172.21.15.46:0/3635930712 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb1b4097350 con 0x7fb1bc100870 2024-03-18T20:35:58.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:58 smithi046 ceph-mon[43495]: pgmap v260: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-18T20:35:58.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:58 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1815465174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:58.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:58 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2441417836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:58.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.379+0000 7fb1c1325700 1 -- 172.21.15.46:0/3635930712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1bc067a10 con 0x7fb1bc100870 2024-03-18T20:35:58.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.380+0000 7fb1a3fff700 1 -- 172.21.15.46:0/3635930712 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22503 (secure 0 0 0) 0x7fb1b409a970 con 0x7fb1bc100870 2024-03-18T20:35:58.384 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:35:58.384 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T20:35:38.108050+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T20:35:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.383+0000 7fb1c1325700 1 -- 172.21.15.46:0/3635930712 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb1a40a11b0 msgr2=0x7fb1a40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.383+0000 7fb1c1325700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb1a40a11b0 0x7fb1a40a3670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009bf0 tx=0x7fb1ac009260 comp rx=0 tx=0).stop 2024-03-18T20:35:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.383+0000 7fb1c1325700 1 -- 172.21.15.46:0/3635930712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1bc100870 msgr2=0x7fb1bc1a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.383+0000 7fb1c1325700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1bc100870 0x7fb1bc1a0410 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400b780 tx=0x7fb1b400bd30 comp rx=0 tx=0).stop 2024-03-18T20:35:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.383+0000 7fb1c1325700 1 -- 172.21.15.46:0/3635930712 shutdown_connections 2024-03-18T20:35:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.383+0000 7fb1c1325700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fb1a40a11b0 0x7fb1a40a3670 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.383+0000 7fb1c1325700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1bc100870 0x7fb1bc1a0410 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.383+0000 7fb1c1325700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1bc0ff550 0x7fb1bc198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.383+0000 7fb1c1325700 1 --2- 172.21.15.46:0/3635930712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1bc0ffeb0 0x7fb1bc1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.383+0000 7fb1c1325700 1 -- 172.21.15.46:0/3635930712 >> 172.21.15.46:0/3635930712 conn(0x7fb1bc0fb0d0 msgr2=0x7fb1bc0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.383+0000 7fb1c1325700 1 -- 172.21.15.46:0/3635930712 shutdown_connections 2024-03-18T20:35:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.383+0000 7fb1c1325700 1 -- 172.21.15.46:0/3635930712 wait complete. 2024-03-18T20:35:58.400 DEBUG:tasks.fwd_scrub:Yielding 2024-03-18T20:35:58.401 INFO:teuthology.run_tasks:Running task exec... 2024-03-18T20:35:58.411 INFO:teuthology.task.exec:Executing custom commands... 2024-03-18T20:35:58.411 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-03-18T20:35:58.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-18T20:35:58.413 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-18T20:35:58.413 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:35:58.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:35:58.413 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:35:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:58 smithi073 ceph-mon[47384]: pgmap v260: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-18T20:35:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:58 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1815465174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:58 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2441417836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:58 smithi155 ceph-mon[48344]: pgmap v260: 97 pgs: 97 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-18T20:35:58.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:58 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1815465174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T20:35:58.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:58 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2441417836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:35:58.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.548+0000 7f9643272700 1 -- 172.21.15.46:0/1641555034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c106ac0 msgr2=0x7f963c108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.548+0000 7f9643272700 1 --2- 172.21.15.46:0/1641555034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c106ac0 0x7f963c108eb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9634009990 tx=0x7f9634009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.549+0000 7f9643272700 1 -- 172.21.15.46:0/1641555034 shutdown_connections 2024-03-18T20:35:58.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.549+0000 7f9643272700 1 --2- 172.21.15.46:0/1641555034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c106ac0 0x7f963c108eb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.549+0000 7f9643272700 1 --2- 172.21.15.46:0/1641555034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f963c101860 0x7f963c103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.549+0000 7f9643272700 1 --2- 172.21.15.46:0/1641555034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f963c104190 0x7f963c106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.549+0000 7f9643272700 1 -- 172.21.15.46:0/1641555034 >> 172.21.15.46:0/1641555034 conn(0x7f963c0fb1b0 msgr2=0x7f963c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.549+0000 7f9643272700 1 -- 172.21.15.46:0/1641555034 shutdown_connections 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.549+0000 7f9643272700 1 -- 172.21.15.46:0/1641555034 wait complete. 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.549+0000 7f9643272700 1 Processor -- start 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.549+0000 7f9643272700 1 -- start start 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.549+0000 7f9643272700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f963c101860 0x7f963c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f9643272700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f963c104190 0x7f963c1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f9643272700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c106ac0 0x7f963c19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f9643272700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c19e6e0 con 0x7f963c106ac0 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f9643272700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c19e820 con 0x7f963c104190 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f9643272700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c19e9b0 con 0x7f963c101860 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f964180f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c106ac0 0x7f963c19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f964180f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c106ac0 0x7f963c19e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38764/0 (socket says 172.21.15.46:38764) 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f964180f700 1 -- 172.21.15.46:0/1215477520 learned_addr learned my addr 172.21.15.46:0/1215477520 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f964100e700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f963c101860 0x7f963c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f964080d700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f963c104190 0x7f963c1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f964180f700 1 -- 172.21.15.46:0/1215477520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f963c101860 msgr2=0x7f963c196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f964180f700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f963c101860 0x7f963c196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f964180f700 1 -- 172.21.15.46:0/1215477520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f963c104190 msgr2=0x7f963c1970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f964180f700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f963c104190 0x7f963c1970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f964180f700 1 -- 172.21.15.46:0/1215477520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96340095f0 con 0x7f963c106ac0 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f964100e700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f963c101860 0x7f963c196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f964180f700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c106ac0 0x7f963c19e110 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f963400b7b0 tx=0x7f9634004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f96327fc700 1 -- 172.21.15.46:0/1215477520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963401d070 con 0x7f963c106ac0 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f9643272700 1 -- 172.21.15.46:0/1215477520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963c19ec30 con 0x7f963c106ac0 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f96327fc700 1 -- 172.21.15.46:0/1215477520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96340054f0 con 0x7f963c106ac0 2024-03-18T20:35:58.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.550+0000 7f96327fc700 1 -- 172.21.15.46:0/1215477520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634004cd0 con 0x7f963c106ac0 2024-03-18T20:35:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.551+0000 7f9643272700 1 -- 172.21.15.46:0/1215477520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c19f060 con 0x7f963c106ac0 2024-03-18T20:35:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.551+0000 7f9643272700 1 -- 172.21.15.46:0/1215477520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c0fcd30 con 0x7f963c106ac0 2024-03-18T20:35:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.553+0000 7f96327fc700 1 -- 172.21.15.46:0/1215477520 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f963400fbc0 con 0x7f963c106ac0 2024-03-18T20:35:58.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.554+0000 7f96327fc700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f96280a1230 0x7f96280a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.554+0000 7f96327fc700 1 -- 172.21.15.46:0/1215477520 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f96340cdea0 con 0x7f963c106ac0 2024-03-18T20:35:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.554+0000 7f964100e700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f96280a1230 0x7f96280a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.554+0000 7f964100e700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f96280a1230 0x7f96280a36f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9638009dd0 tx=0x7f9638009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.555+0000 7f7186372700 1 -- 172.21.15.46:0/2852551713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7180106a50 msgr2=0x7f7180108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.555+0000 7f7186372700 1 --2- 172.21.15.46:0/2852551713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7180106a50 0x7f7180108ec0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7178009f90 tx=0x7f717800b330 comp rx=0 tx=0).stop 2024-03-18T20:35:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.555+0000 7f7186372700 1 -- 172.21.15.46:0/2852551713 shutdown_connections 2024-03-18T20:35:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.555+0000 7f7186372700 1 --2- 172.21.15.46:0/2852551713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71801017f0 0x7f7180103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.555+0000 7f7186372700 1 --2- 172.21.15.46:0/2852551713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7180106a50 0x7f7180108ec0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.555+0000 7f7186372700 1 --2- 172.21.15.46:0/2852551713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7180104120 0x7f7180106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.555+0000 7f7186372700 1 -- 172.21.15.46:0/2852551713 >> 172.21.15.46:0/2852551713 conn(0x7f71800fb200 msgr2=0x7f71800fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.555+0000 7f7186372700 1 -- 172.21.15.46:0/2852551713 shutdown_connections 2024-03-18T20:35:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.555+0000 7f7186372700 1 -- 172.21.15.46:0/2852551713 wait complete. 2024-03-18T20:35:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f7186372700 1 Processor -- start 2024-03-18T20:35:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f7186372700 1 -- start start 2024-03-18T20:35:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f7186372700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71801017f0 0x7f7180196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f7186372700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7180104120 0x7f71801972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f7186372700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7180106a50 0x7f718019e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f7186372700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718019e890 con 0x7f71801017f0 2024-03-18T20:35:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f7186372700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718019ea00 con 0x7f7180104120 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f7186372700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718019ebc0 con 0x7f7180106a50 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f717ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71801017f0 0x7f7180196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f717ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71801017f0 0x7f7180196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38776/0 (socket says 172.21.15.46:38776) 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f717ffff700 1 -- 172.21.15.46:0/1338538868 learned_addr learned my addr 172.21.15.46:0/1338538868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f718490f700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7180106a50 0x7f718019e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.556+0000 7f717f7fe700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7180104120 0x7f71801972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.557+0000 7f717ffff700 1 -- 172.21.15.46:0/1338538868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7180106a50 msgr2=0x7f718019e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.557+0000 7f717ffff700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7180106a50 0x7f718019e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.557+0000 7f717ffff700 1 -- 172.21.15.46:0/1338538868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7180104120 msgr2=0x7f71801972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.557+0000 7f717ffff700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7180104120 0x7f71801972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.557+0000 7f717ffff700 1 -- 172.21.15.46:0/1338538868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7178009c40 con 0x7f71801017f0 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.557+0000 7f717f7fe700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7180104120 0x7f71801972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:35:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.557+0000 7f717ffff700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71801017f0 0x7f7180196da0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f717000e9c0 tx=0x7f717000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.558+0000 7f717d7fa700 1 -- 172.21.15.46:0/1338538868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717000cbd0 con 0x7f71801017f0 2024-03-18T20:35:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.558+0000 7f717d7fa700 1 -- 172.21.15.46:0/1338538868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f717000cd30 con 0x7f71801017f0 2024-03-18T20:35:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.558+0000 7f7186372700 1 -- 172.21.15.46:0/1338538868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718019eea0 con 0x7f71801017f0 2024-03-18T20:35:58.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.558+0000 7f717d7fa700 1 -- 172.21.15.46:0/1338538868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170005590 con 0x7f71801017f0 2024-03-18T20:35:58.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.558+0000 7f96327fc700 1 -- 172.21.15.46:0/1215477520 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9634097510 con 0x7f963c106ac0 2024-03-18T20:35:58.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.558+0000 7f7186372700 1 -- 172.21.15.46:0/1338538868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718019f2c0 con 0x7f71801017f0 2024-03-18T20:35:58.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.561+0000 7f717d7fa700 1 -- 172.21.15.46:0/1338538868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f7170021440 con 0x7f71801017f0 2024-03-18T20:35:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.561+0000 7f717d7fa700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f71680a11b0 0x7f71680a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.562+0000 7f717d7fa700 1 -- 172.21.15.46:0/1338538868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7170014070 con 0x7f71801017f0 2024-03-18T20:35:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.562+0000 7f717f7fe700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f71680a11b0 0x7f71680a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.562+0000 7f7192507b80 1 Processor -- start 2024-03-18T20:35:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.562+0000 7f7192507b80 1 -- start start 2024-03-18T20:35:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.562+0000 7f717f7fe700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f71680a11b0 0x7f71680a3670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7174009be0 tx=0x7f7174009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.562+0000 7f7192507b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa2a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.562+0000 7f7192507b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2a600 0x56519fa2aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.562+0000 7f7192507b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa2b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.562+0000 7f7192507b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519f982a10 con 0x56519fa29ca0 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.562+0000 7f7192507b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519f9803d0 con 0x56519fa2a600 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.562+0000 7f7192507b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519f97bcc0 con 0x56519fa2b4a0 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f717ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa2a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f717ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa2a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38782/0 (socket says 172.21.15.46:38782) 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f717ffff700 1 -- 172.21.15.46:0/821987649 learned_addr learned my addr 172.21.15.46:0/821987649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f717f7fe700 1 --2- 172.21.15.46:0/821987649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2a600 0x56519fa2aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f718490f700 1 --2- 172.21.15.46:0/821987649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa2b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f717ffff700 1 -- 172.21.15.46:0/821987649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 msgr2=0x56519fa2b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f717ffff700 1 --2- 172.21.15.46:0/821987649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa2b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f717ffff700 1 -- 172.21.15.46:0/821987649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2a600 msgr2=0x56519fa2aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f717ffff700 1 --2- 172.21.15.46:0/821987649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2a600 0x56519fa2aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f717ffff700 1 -- 172.21.15.46:0/821987649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56519f7fd430 con 0x56519fa29ca0 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f717ffff700 1 --2- 172.21.15.46:0/821987649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa2a0c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7170096500 tx=0x7f717009a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b61c9dbf4a92b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f71667fc700 1 -- 172.21.15.46:0/821987649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71700a2480 con 0x56519fa29ca0 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f71667fc700 1 -- 172.21.15.46:0/821987649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71700056f0 con 0x56519fa29ca0 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.563+0000 7f71667fc700 1 -- 172.21.15.46:0/821987649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717009a4c0 con 0x56519fa29ca0 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.564+0000 7f7192507b80 1 -- 172.21.15.46:0/821987649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 msgr2=0x56519fa2a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.564+0000 7f7192507b80 1 --2- 172.21.15.46:0/821987649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa2a0c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7170096500 tx=0x7f717009a780 comp rx=0 tx=0).stop 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.564+0000 7f7192507b80 1 -- 172.21.15.46:0/821987649 shutdown_connections 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.564+0000 7f7192507b80 1 --2- 172.21.15.46:0/821987649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa2a0c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.564+0000 7f7192507b80 1 --2- 172.21.15.46:0/821987649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa2b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.564+0000 7f7192507b80 1 --2- 172.21.15.46:0/821987649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2a600 0x56519fa2aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.564+0000 7f7192507b80 1 -- 172.21.15.46:0/821987649 >> 172.21.15.46:0/821987649 conn(0x56519f99f2a0 msgr2=0x56519fa24c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.564+0000 7f7192507b80 1 -- 172.21.15.46:0/821987649 shutdown_connections 2024-03-18T20:35:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.564+0000 7f7192507b80 1 -- 172.21.15.46:0/821987649 wait complete. 2024-03-18T20:35:58.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f7192507b80 1 -- start start 2024-03-18T20:35:58.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f7192507b80 1 Processor -- start 2024-03-18T20:35:58.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f7192507b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa29ca0 0x56519fa33ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f7192507b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa2a600 0x56519fa340e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f7192507b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa3b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f7192507b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519f917e00 con 0x56519fa2a600 2024-03-18T20:35:58.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f7192507b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519f917aa0 con 0x56519fa29ca0 2024-03-18T20:35:58.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f7192507b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519f7ff720 con 0x56519fa2b4a0 2024-03-18T20:35:58.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f7192507b80 10 client.0 _ll_register_callbacks cb 0x56519f783420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:58.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f717f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa2a600 0x56519fa340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f717f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa2a600 0x56519fa340e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38788/0 (socket says 172.21.15.46:38788) 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f717f7fe700 1 -- 172.21.15.46:0/40114436 learned_addr learned my addr 172.21.15.46:0/40114436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.565+0000 7f717ffff700 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa29ca0 0x56519fa33ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.566+0000 7f718490f700 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa3b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.566+0000 7f717f7fe700 1 -- 172.21.15.46:0/40114436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 msgr2=0x56519fa3b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.566+0000 7f717f7fe700 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa3b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.566+0000 7f717f7fe700 1 -- 172.21.15.46:0/40114436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa29ca0 msgr2=0x56519fa33ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.566+0000 7f717f7fe700 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa29ca0 0x56519fa33ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.566+0000 7f717f7fe700 1 -- 172.21.15.46:0/40114436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718019f2c0 con 0x56519fa2a600 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.566+0000 7f718490f700 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa3b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.566+0000 7f717f7fe700 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa2a600 0x56519fa340e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7174012480 tx=0x7f7174004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe10c65e0693d9e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.566+0000 7f715f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:58.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.566+0000 7f715f7fe700 1 -- 172.21.15.46:0/40114436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174029670 con 0x56519fa2a600 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.566+0000 7f7192507b80 1 -- 172.21.15.46:0/40114436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56519fa1cb50 con 0x56519fa2a600 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f715f7fe700 1 -- 172.21.15.46:0/40114436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7174020430 con 0x56519fa2a600 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f715f7fe700 1 -- 172.21.15.46:0/40114436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71740297d0 con 0x56519fa2a600 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f7192507b80 10 client.15456 fetch_fsmap learned FSMap version 27 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f7192507b80 1 -- 172.21.15.46:0/40114436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56519f7a9be0 con 0x56519fa2a600 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f715f7fe700 1 -- 172.21.15.46:0/40114436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56519fa1cb50 con 0x56519fa2a600 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f715f7fe700 1 -- 172.21.15.46:0/40114436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7174032070 con 0x56519fa2a600 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f7192507b80 10 client.15456 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f7192507b80 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f7192507b80 20 client.15456 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f7192507b80 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56519fa43db0 0x56519fa461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f7192507b80 4 client.15456 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f7192507b80 1 -- 172.21.15.46:0/40114436 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56519f974780 con 0x56519fa43db0 2024-03-18T20:35:58.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.567+0000 7f717ffff700 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56519fa43db0 0x56519fa461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.568+0000 7f717ffff700 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56519fa43db0 0x56519fa461d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d8c1c785d139cf2 server_cookie=50c757345b406c22 in_seq=0 out_seq=0 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.568+0000 7f715f7fe700 10 client.15456 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.568+0000 7f715f7fe700 1 -- 172.21.15.46:0/40114436 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56519f974780 con 0x56519fa43db0 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.568+0000 7f715f7fe700 10 client.15456 handle_command_reply: tid=0 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f7192507b80 1 client.15456 shutdown 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f7192507b80 1 -- 172.21.15.46:0/40114436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa2a600 msgr2=0x56519fa340e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f7192507b80 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa2a600 0x56519fa340e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7174012480 tx=0x7f7174004750 comp rx=0 tx=0).stop 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f7192507b80 1 -- 172.21.15.46:0/40114436 shutdown_connections 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f7192507b80 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56519fa43db0 0x56519fa461d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f7192507b80 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa3b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f7192507b80 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa29ca0 0x56519fa33ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f715f7fe700 0 client.15456 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f7192507b80 1 --2- 172.21.15.46:0/40114436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa2a600 0x56519fa340e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f7192507b80 1 -- 172.21.15.46:0/40114436 >> 172.21.15.46:0/40114436 conn(0x56519f99f2a0 msgr2=0x56519fa24270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f7192507b80 1 -- 172.21.15.46:0/40114436 shutdown_connections 2024-03-18T20:35:58.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f7192507b80 1 -- 172.21.15.46:0/40114436 wait complete. 2024-03-18T20:35:58.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.569+0000 7f7192507b80 20 client.15456 trim_cache size 0 max 16384 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.587+0000 7f7192507b80 1 Processor -- start 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.587+0000 7f7192507b80 1 -- start start 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f7192507b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa2e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f7192507b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2a600 0x56519fa3b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f7192507b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2b4a0 0x56519fa2e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f7192507b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519fa2aa10 con 0x56519fa29ca0 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f7192507b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519fa34a70 con 0x56519fa2b4a0 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f7192507b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519fa2a440 con 0x56519fa2a600 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f717ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa2e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f717ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa2e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38804/0 (socket says 172.21.15.46:38804) 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f717ffff700 1 -- 172.21.15.46:0/3827672321 learned_addr learned my addr 172.21.15.46:0/3827672321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f717f7fe700 1 --2- 172.21.15.46:0/3827672321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2a600 0x56519fa3b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f718490f700 1 --2- 172.21.15.46:0/3827672321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2b4a0 0x56519fa2e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f717ffff700 1 -- 172.21.15.46:0/3827672321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2a600 msgr2=0x56519fa3b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f717ffff700 1 --2- 172.21.15.46:0/3827672321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2a600 0x56519fa3b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f717ffff700 1 -- 172.21.15.46:0/3827672321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2b4a0 msgr2=0x56519fa2e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f717ffff700 1 --2- 172.21.15.46:0/3827672321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2b4a0 0x56519fa2e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f717ffff700 1 -- 172.21.15.46:0/3827672321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56519f941f20 con 0x56519fa29ca0 2024-03-18T20:35:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.588+0000 7f717ffff700 1 --2- 172.21.15.46:0/3827672321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa2e3f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7170099d00 tx=0x7f7170009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c95821d44cf2130 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.589+0000 7f7166ffd700 1 -- 172.21.15.46:0/3827672321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717007a070 con 0x56519fa29ca0 2024-03-18T20:35:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.589+0000 7f7166ffd700 1 -- 172.21.15.46:0/3827672321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71700a2480 con 0x56519fa29ca0 2024-03-18T20:35:58.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.589+0000 7f7166ffd700 1 -- 172.21.15.46:0/3827672321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717007e730 con 0x56519fa29ca0 2024-03-18T20:35:58.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.589+0000 7f7192507b80 1 -- 172.21.15.46:0/3827672321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 msgr2=0x56519fa2e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.589+0000 7f7192507b80 1 --2- 172.21.15.46:0/3827672321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa2e3f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7170099d00 tx=0x7f7170009740 comp rx=0 tx=0).stop 2024-03-18T20:35:58.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.589+0000 7f7192507b80 1 -- 172.21.15.46:0/3827672321 shutdown_connections 2024-03-18T20:35:58.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.589+0000 7f7192507b80 1 --2- 172.21.15.46:0/3827672321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa2e3f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.589+0000 7f7192507b80 1 --2- 172.21.15.46:0/3827672321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2b4a0 0x56519fa2e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.589+0000 7f7192507b80 1 --2- 172.21.15.46:0/3827672321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2a600 0x56519fa3b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.589+0000 7f7192507b80 1 -- 172.21.15.46:0/3827672321 >> 172.21.15.46:0/3827672321 conn(0x56519f99f2a0 msgr2=0x56519fa24ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.589+0000 7f7192507b80 1 -- 172.21.15.46:0/3827672321 shutdown_connections 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.590+0000 7f7192507b80 1 -- 172.21.15.46:0/3827672321 wait complete. 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.590+0000 7f7192507b80 1 -- start start 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.590+0000 7f7192507b80 1 Processor -- start 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.590+0000 7f7192507b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa3a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.590+0000 7f7192507b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2a600 0x56519fa3aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.590+0000 7f7192507b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa98040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.590+0000 7f7192507b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519fa98580 con 0x56519fa29ca0 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.590+0000 7f7192507b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519fa986c0 con 0x56519fa2a600 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.590+0000 7f7192507b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56519fa98850 con 0x56519fa2b4a0 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f7192507b80 10 client.0 _ll_register_callbacks cb 0x7f7174010c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f717ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa3a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f717ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa3a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38812/0 (socket says 172.21.15.46:38812) 2024-03-18T20:35:58.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f717ffff700 1 -- 172.21.15.46:0/3111328295 learned_addr learned my addr 172.21.15.46:0/3111328295 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f718490f700 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa98040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f717f7fe700 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2a600 0x56519fa3aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f717ffff700 1 -- 172.21.15.46:0/3111328295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 msgr2=0x56519fa98040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f717ffff700 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa98040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f717ffff700 1 -- 172.21.15.46:0/3111328295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2a600 msgr2=0x56519fa3aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f717ffff700 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2a600 0x56519fa3aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f717ffff700 1 -- 172.21.15.46:0/3111328295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56519f7fd430 con 0x56519fa29ca0 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f717f7fe700 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2a600 0x56519fa3aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f717ffff700 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa3a590 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f71700cc010 tx=0x7f717000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a4f3528232e7b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f715f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f715f7fe700 1 -- 172.21.15.46:0/3111328295 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717007a070 con 0x56519fa29ca0 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f715f7fe700 1 -- 172.21.15.46:0/3111328295 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f71700983a0 con 0x56519fa29ca0 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f7192507b80 1 -- 172.21.15.46:0/3111328295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71700056f0 con 0x56519fa29ca0 2024-03-18T20:35:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f715f7fe700 1 -- 172.21.15.46:0/3111328295 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71700ca910 con 0x56519fa29ca0 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.591+0000 7f7192507b80 10 client.15462 fetch_fsmap learned FSMap version 27 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.592+0000 7f7192507b80 1 -- 172.21.15.46:0/3111328295 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56519f801d50 con 0x56519fa29ca0 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.592+0000 7f715f7fe700 1 -- 172.21.15.46:0/3111328295 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71700056f0 con 0x56519fa29ca0 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.592+0000 7f715f7fe700 1 -- 172.21.15.46:0/3111328295 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f71700a1ff0 con 0x56519fa29ca0 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.592+0000 7f7192507b80 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.592+0000 7f7192507b80 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.592+0000 7f7192507b80 20 client.15462 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.592+0000 7f7192507b80 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56519fa99e00 0x56519fa9c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.592+0000 7f7192507b80 4 client.15462 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.592+0000 7f7192507b80 1 -- 172.21.15.46:0/3111328295 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56519f7cad70 con 0x56519fa99e00 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.592+0000 7f717f7fe700 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56519fa99e00 0x56519fa9c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.593+0000 7f715f7fe700 10 client.15462 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.593+0000 7f717f7fe700 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56519fa99e00 0x56519fa9c220 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bec09819b1a0cf0 server_cookie=6c9076abce7e0953 in_seq=0 out_seq=0 2024-03-18T20:35:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.594+0000 7f715f7fe700 1 -- 172.21.15.46:0/3111328295 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56519f7cad70 con 0x56519fa99e00 2024-03-18T20:35:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.594+0000 7f715f7fe700 10 client.15462 handle_command_reply: tid=0 2024-03-18T20:35:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.594+0000 7f7192507b80 1 client.15462 shutdown 2024-03-18T20:35:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.594+0000 7f7192507b80 1 -- 172.21.15.46:0/3111328295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 msgr2=0x56519fa3a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.594+0000 7f7192507b80 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa3a590 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f71700cc010 tx=0x7f717000c6e0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7192507b80 1 -- 172.21.15.46:0/3111328295 shutdown_connections 2024-03-18T20:35:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7192507b80 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56519fa99e00 0x56519fa9c220 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7192507b80 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56519fa29ca0 0x56519fa3a590 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7192507b80 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56519fa2b4a0 0x56519fa98040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7192507b80 1 --2- 172.21.15.46:0/3111328295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56519fa2a600 0x56519fa3aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7192507b80 1 -- 172.21.15.46:0/3111328295 >> 172.21.15.46:0/3111328295 conn(0x56519f99f2a0 msgr2=0x56519fa2e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f715f7fe700 0 client.15462 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7192507b80 1 -- 172.21.15.46:0/3111328295 shutdown_connections 2024-03-18T20:35:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7192507b80 1 -- 172.21.15.46:0/3111328295 wait complete. 2024-03-18T20:35:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7192507b80 20 client.15462 trim_cache size 0 max 16384 2024-03-18T20:35:58.603 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:35:58.603 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:35:58.603 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "16612a36-e40d-4eca-b947-db2bb37a4f13", 2024-03-18T20:35:58.603 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:35:58.603 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:35:58.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7186372700 1 -- 172.21.15.46:0/1338538868 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f71680a11b0 msgr2=0x7f71680a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7186372700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f71680a11b0 0x7f71680a3670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7174009be0 tx=0x7f7174009260 comp rx=0 tx=0).stop 2024-03-18T20:35:58.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7186372700 1 -- 172.21.15.46:0/1338538868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71801017f0 msgr2=0x7f7180196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.595+0000 7f7186372700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71801017f0 0x7f7180196da0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f717000e9c0 tx=0x7f717000ed80 comp rx=0 tx=0).stop 2024-03-18T20:35:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.596+0000 7f7186372700 1 -- 172.21.15.46:0/1338538868 shutdown_connections 2024-03-18T20:35:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.596+0000 7f7186372700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f71680a11b0 0x7f71680a3670 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.596+0000 7f7186372700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71801017f0 0x7f7180196da0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.596+0000 7f7186372700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7180106a50 0x7f718019e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.596+0000 7f7186372700 1 --2- 172.21.15.46:0/1338538868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7180104120 0x7f71801972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.596+0000 7f7186372700 1 -- 172.21.15.46:0/1338538868 >> 172.21.15.46:0/1338538868 conn(0x7f71800fb200 msgr2=0x7f71801021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.596+0000 7f7186372700 1 -- 172.21.15.46:0/1338538868 shutdown_connections 2024-03-18T20:35:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.596+0000 7f7186372700 1 -- 172.21.15.46:0/1338538868 wait complete. 2024-03-18T20:35:58.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:35:58.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:35:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.753+0000 7f400d4bf700 1 -- 172.21.15.46:0/4121657219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40081041c0 msgr2=0x7f40081065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.753+0000 7f400d4bf700 1 --2- 172.21.15.46:0/4121657219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40081041c0 0x7f40081065b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009990 tx=0x7f3ffc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.753+0000 7f400d4bf700 1 -- 172.21.15.46:0/4121657219 shutdown_connections 2024-03-18T20:35:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.753+0000 7f400d4bf700 1 --2- 172.21.15.46:0/4121657219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4008106af0 0x7f4008108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.753+0000 7f400d4bf700 1 --2- 172.21.15.46:0/4121657219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4008101890 0x7f4008103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.753+0000 7f400d4bf700 1 --2- 172.21.15.46:0/4121657219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40081041c0 0x7f40081065b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.753+0000 7f400d4bf700 1 -- 172.21.15.46:0/4121657219 >> 172.21.15.46:0/4121657219 conn(0x7f40080fb200 msgr2=0x7f40080fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.753+0000 7f400d4bf700 1 -- 172.21.15.46:0/4121657219 shutdown_connections 2024-03-18T20:35:58.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.753+0000 7f400d4bf700 1 -- 172.21.15.46:0/4121657219 wait complete. 2024-03-18T20:35:58.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.754+0000 7f400d4bf700 1 Processor -- start 2024-03-18T20:35:58.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.754+0000 7f400d4bf700 1 -- start start 2024-03-18T20:35:58.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.754+0000 7f400d4bf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4008101890 0x7f4008196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.754+0000 7f400d4bf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40081041c0 0x7f4008197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f400d4bf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4008106af0 0x7f400819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f400d4bf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400819e800 con 0x7f4008106af0 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f400d4bf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400819e970 con 0x7f4008101890 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f400d4bf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400819eb30 con 0x7f40081041c0 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f40077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4008106af0 0x7f400819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f40077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4008106af0 0x7f400819e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38840/0 (socket says 172.21.15.46:38840) 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f40077fe700 1 -- 172.21.15.46:0/4021626487 learned_addr learned my addr 172.21.15.46:0/4021626487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f4006ffd700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4008101890 0x7f4008196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f40067fc700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40081041c0 0x7f4008197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f40077fe700 1 -- 172.21.15.46:0/4021626487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40081041c0 msgr2=0x7f4008197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f40077fe700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40081041c0 0x7f4008197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f40077fe700 1 -- 172.21.15.46:0/4021626487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4008101890 msgr2=0x7f4008196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f40077fe700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4008101890 0x7f4008196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f40077fe700 1 -- 172.21.15.46:0/4021626487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc0095f0 con 0x7f4008106af0 2024-03-18T20:35:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f40077fe700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4008106af0 0x7f400819e230 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f400000ef30 tx=0x7f400000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f3feffff700 1 -- 172.21.15.46:0/4021626487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000018070 con 0x7f4008106af0 2024-03-18T20:35:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f3feffff700 1 -- 172.21.15.46:0/4021626487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400000f040 con 0x7f4008106af0 2024-03-18T20:35:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.755+0000 7f3feffff700 1 -- 172.21.15.46:0/4021626487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400000cd00 con 0x7f4008106af0 2024-03-18T20:35:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.756+0000 7f400d4bf700 1 -- 172.21.15.46:0/4021626487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400819edb0 con 0x7f4008106af0 2024-03-18T20:35:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.756+0000 7f400d4bf700 1 -- 172.21.15.46:0/4021626487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400819f1e0 con 0x7f4008106af0 2024-03-18T20:35:58.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.758+0000 7f3feffff700 1 -- 172.21.15.46:0/4021626487 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4000003a00 con 0x7f4008106af0 2024-03-18T20:35:58.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.758+0000 7f3feffff700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3ff00a1230 0x7f3ff00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.759+0000 7f4006ffd700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3ff00a1230 0x7f3ff00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.759+0000 7f3feffff700 1 -- 172.21.15.46:0/4021626487 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f40000cc560 con 0x7f4008106af0 2024-03-18T20:35:58.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.759+0000 7f4006ffd700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3ff00a1230 0x7f3ff00a36f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4008103080 tx=0x7f3ff8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.760+0000 7f4019654b80 1 Processor -- start 2024-03-18T20:35:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.760+0000 7f4019654b80 1 -- start start 2024-03-18T20:35:58.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.760+0000 7f4019654b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afdc20 0x5564a8afe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.760+0000 7f4019654b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8afe580 0x5564a8afee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.760+0000 7f4019654b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8aff420 0x5564a8aff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.760+0000 7f4019654b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a8a56990 con 0x5564a8aff420 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.760+0000 7f4019654b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a8a54350 con 0x5564a8afe580 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.760+0000 7f4019654b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a8a4fc40 con 0x5564a8afdc20 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f40077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8aff420 0x5564a8aff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f40077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8aff420 0x5564a8aff8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38848/0 (socket says 172.21.15.46:38848) 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f40077fe700 1 -- 172.21.15.46:0/734901308 learned_addr learned my addr 172.21.15.46:0/734901308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f4006ffd700 1 --2- 172.21.15.46:0/734901308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afdc20 0x5564a8afe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f40067fc700 1 --2- 172.21.15.46:0/734901308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8afe580 0x5564a8afee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f40077fe700 1 -- 172.21.15.46:0/734901308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afdc20 msgr2=0x5564a8afe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f40077fe700 1 --2- 172.21.15.46:0/734901308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afdc20 0x5564a8afe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f40077fe700 1 -- 172.21.15.46:0/734901308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8afe580 msgr2=0x5564a8afee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f40077fe700 1 --2- 172.21.15.46:0/734901308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8afe580 0x5564a8afee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f40077fe700 1 -- 172.21.15.46:0/734901308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564a88d13b0 con 0x5564a8aff420 2024-03-18T20:35:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f40067fc700 1 --2- 172.21.15.46:0/734901308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8afe580 0x5564a8afee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:35:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f40077fe700 1 --2- 172.21.15.46:0/734901308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8aff420 0x5564a8aff8a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f40000090a0 tx=0x7f40000994a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53373f4d538fcf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f3fed7fa700 1 -- 172.21.15.46:0/734901308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000004ad0 con 0x5564a8aff420 2024-03-18T20:35:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f3fed7fa700 1 -- 172.21.15.46:0/734901308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400001b320 con 0x5564a8aff420 2024-03-18T20:35:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.761+0000 7f3fed7fa700 1 -- 172.21.15.46:0/734901308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40000ca110 con 0x5564a8aff420 2024-03-18T20:35:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.762+0000 7f4019654b80 1 -- 172.21.15.46:0/734901308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8aff420 msgr2=0x5564a8aff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.762+0000 7f4019654b80 1 --2- 172.21.15.46:0/734901308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8aff420 0x5564a8aff8a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f40000090a0 tx=0x7f40000994a0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.762+0000 7f4019654b80 1 -- 172.21.15.46:0/734901308 shutdown_connections 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.762+0000 7f4019654b80 1 --2- 172.21.15.46:0/734901308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8aff420 0x5564a8aff8a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.762+0000 7f4019654b80 1 --2- 172.21.15.46:0/734901308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afdc20 0x5564a8afe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.762+0000 7f4019654b80 1 --2- 172.21.15.46:0/734901308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8afe580 0x5564a8afee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.762+0000 7f4019654b80 1 -- 172.21.15.46:0/734901308 >> 172.21.15.46:0/734901308 conn(0x5564a8a73220 msgr2=0x5564a8af8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.762+0000 7f4019654b80 1 -- 172.21.15.46:0/734901308 shutdown_connections 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.762+0000 7f4019654b80 1 -- 172.21.15.46:0/734901308 wait complete. 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f4019654b80 1 -- start start 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f4019654b80 1 Processor -- start 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f4019654b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 0x5564a8b07af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f4019654b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afe580 0x5564a8b08030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f4019654b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 0x5564a8b0f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f4019654b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a89ebd80 con 0x5564a8afdc20 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f4019654b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a89eba20 con 0x5564a8aff420 2024-03-18T20:35:58.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f4019654b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a88d36a0 con 0x5564a8afe580 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f4019654b80 10 client.0 _ll_register_callbacks cb 0x5564a88573a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f4006ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 0x5564a8b07af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f4006ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 0x5564a8b07af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38854/0 (socket says 172.21.15.46:38854) 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f4006ffd700 1 -- 172.21.15.46:0/3667420205 learned_addr learned my addr 172.21.15.46:0/3667420205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f40077fe700 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 0x5564a8b0f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.763+0000 7f40067fc700 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afe580 0x5564a8b08030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f4006ffd700 1 -- 172.21.15.46:0/3667420205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afe580 msgr2=0x5564a8b08030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f4006ffd700 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afe580 0x5564a8b08030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f4006ffd700 1 -- 172.21.15.46:0/3667420205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 msgr2=0x5564a8b0f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f4006ffd700 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 0x5564a8b0f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f4006ffd700 1 -- 172.21.15.46:0/3667420205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400819f1e0 con 0x5564a8afdc20 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f4006ffd700 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 0x5564a8b07af0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8004600 tx=0x7f3ff80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791ee362514ae675 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f3feaff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:35:58.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f3feaff5700 1 -- 172.21.15.46:0/3667420205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8028640 con 0x5564a8afdc20 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f3feaff5700 1 -- 172.21.15.46:0/3667420205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3ff80287a0 con 0x5564a8afdc20 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f4019654b80 1 -- 172.21.15.46:0/3667420205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564a8af0ad0 con 0x5564a8afdc20 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f3feaff5700 1 -- 172.21.15.46:0/3667420205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ff802e030 con 0x5564a8afdc20 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f4019654b80 10 client.15474 fetch_fsmap learned FSMap version 27 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f4019654b80 1 -- 172.21.15.46:0/3667420205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5564a887db60 con 0x5564a8afdc20 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.764+0000 7f3feaff5700 1 -- 172.21.15.46:0/3667420205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564a8af0ad0 con 0x5564a8afdc20 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.765+0000 7f3feaff5700 1 -- 172.21.15.46:0/3667420205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3ff801a3c0 con 0x5564a8afdc20 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.765+0000 7f4019654b80 10 client.15474 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.765+0000 7f4019654b80 10 client.15474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.765+0000 7f4019654b80 20 client.15474 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.765+0000 7f4019654b80 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564a8b13910 0x5564a8b15d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.765+0000 7f4019654b80 4 client.15474 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.765+0000 7f4019654b80 1 -- 172.21.15.46:0/3667420205 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564a8a48700 con 0x5564a8b13910 2024-03-18T20:35:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.765+0000 7f40067fc700 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564a8b13910 0x5564a8b15d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.765+0000 7f40067fc700 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564a8b13910 0x5564a8b15d30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23bb3f2fa5763a2e server_cookie=aef77a0cb7006dc6 in_seq=0 out_seq=0 2024-03-18T20:35:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.765+0000 7f3feaff5700 10 client.15474 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f3feaff5700 1 -- 172.21.15.46:0/3667420205 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5564a8a48700 con 0x5564a8b13910 2024-03-18T20:35:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f3feaff5700 10 client.15474 handle_command_reply: tid=0 2024-03-18T20:35:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f4019654b80 1 client.15474 shutdown 2024-03-18T20:35:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f4019654b80 1 -- 172.21.15.46:0/3667420205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 msgr2=0x5564a8b07af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f4019654b80 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 0x5564a8b07af0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8004600 tx=0x7f3ff80047c0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f4019654b80 1 -- 172.21.15.46:0/3667420205 shutdown_connections 2024-03-18T20:35:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f4019654b80 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564a8b13910 0x5564a8b15d30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f4019654b80 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 0x5564a8b07af0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f4019654b80 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 0x5564a8b0f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f4019654b80 1 --2- 172.21.15.46:0/3667420205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afe580 0x5564a8b08030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f3feaff5700 0 client.15474 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:58.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f4019654b80 1 -- 172.21.15.46:0/3667420205 >> 172.21.15.46:0/3667420205 conn(0x5564a8a73220 msgr2=0x5564a8af81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f4019654b80 1 -- 172.21.15.46:0/3667420205 shutdown_connections 2024-03-18T20:35:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f4019654b80 1 -- 172.21.15.46:0/3667420205 wait complete. 2024-03-18T20:35:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.766+0000 7f4019654b80 20 client.15474 trim_cache size 0 max 16384 2024-03-18T20:35:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f4019654b80 1 Processor -- start 2024-03-18T20:35:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f4019654b80 1 -- start start 2024-03-18T20:35:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f4019654b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 0x5564a8b15d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f4019654b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afe580 0x5564a8b0f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f4019654b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 0x5564a8b05950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f4019654b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a8b055d0 con 0x5564a8afdc20 2024-03-18T20:35:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f4019654b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a8b02ec0 con 0x5564a8aff420 2024-03-18T20:35:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f4019654b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a8aff050 con 0x5564a8afe580 2024-03-18T20:35:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f4006ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 0x5564a8b15d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f4006ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 0x5564a8b15d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38856/0 (socket says 172.21.15.46:38856) 2024-03-18T20:35:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f4006ffd700 1 -- 172.21.15.46:0/3342205987 learned_addr learned my addr 172.21.15.46:0/3342205987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f40067fc700 1 --2- 172.21.15.46:0/3342205987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afe580 0x5564a8b0f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.784+0000 7f40077fe700 1 --2- 172.21.15.46:0/3342205987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 0x5564a8b05950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.785+0000 7f4006ffd700 1 -- 172.21.15.46:0/3342205987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afe580 msgr2=0x5564a8b0f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.785+0000 7f4006ffd700 1 --2- 172.21.15.46:0/3342205987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afe580 0x5564a8b0f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.785+0000 7f4006ffd700 1 -- 172.21.15.46:0/3342205987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 msgr2=0x5564a8b05950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.785+0000 7f4006ffd700 1 --2- 172.21.15.46:0/3342205987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 0x5564a8b05950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.785+0000 7f4006ffd700 1 -- 172.21.15.46:0/3342205987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564a8a15ea0 con 0x5564a8afdc20 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.785+0000 7f4006ffd700 1 --2- 172.21.15.46:0/3342205987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 0x5564a8b15d00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3ff801a720 tx=0x7f3ff8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec36a6aebf1b3e5f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.785+0000 7f3fedffb700 1 -- 172.21.15.46:0/3342205987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff802e040 con 0x5564a8afdc20 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.785+0000 7f3fedffb700 1 -- 172.21.15.46:0/3342205987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ff801a940 con 0x5564a8afdc20 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.785+0000 7f3fedffb700 1 -- 172.21.15.46:0/3342205987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8028d00 con 0x5564a8afdc20 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 -- 172.21.15.46:0/3342205987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 msgr2=0x5564a8b15d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 --2- 172.21.15.46:0/3342205987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 0x5564a8b15d00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3ff801a720 tx=0x7f3ff8004750 comp rx=0 tx=0).stop 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 -- 172.21.15.46:0/3342205987 shutdown_connections 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 --2- 172.21.15.46:0/3342205987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afdc20 0x5564a8b15d00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 --2- 172.21.15.46:0/3342205987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 0x5564a8b05950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 --2- 172.21.15.46:0/3342205987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afe580 0x5564a8b0f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 -- 172.21.15.46:0/3342205987 >> 172.21.15.46:0/3342205987 conn(0x5564a8a73220 msgr2=0x5564a8af8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 -- 172.21.15.46:0/3342205987 shutdown_connections 2024-03-18T20:35:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 -- 172.21.15.46:0/3342205987 wait complete. 2024-03-18T20:35:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 -- start start 2024-03-18T20:35:58.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 Processor -- start 2024-03-18T20:35:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afdc20 0x5564a8b0e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afe580 0x5564a8b0ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 0x5564a8b0cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a8b0d4a0 con 0x5564a8afe580 2024-03-18T20:35:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a8b0d5e0 con 0x5564a8aff420 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564a8b0d7a0 con 0x5564a8afdc20 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.786+0000 7f4019654b80 10 client.0 _ll_register_callbacks cb 0x7f3ff80043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f40067fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afe580 0x5564a8b0ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f4006ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afdc20 0x5564a8b0e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f40067fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afe580 0x5564a8b0ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38860/0 (socket says 172.21.15.46:38860) 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f40067fc700 1 -- 172.21.15.46:0/4204989123 learned_addr learned my addr 172.21.15.46:0/4204989123 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f40077fe700 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 0x5564a8b0cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f4006ffd700 1 -- 172.21.15.46:0/4204989123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 msgr2=0x5564a8b0cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f4006ffd700 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 0x5564a8b0cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f4006ffd700 1 -- 172.21.15.46:0/4204989123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afe580 msgr2=0x5564a8b0ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f4006ffd700 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afe580 0x5564a8b0ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f4006ffd700 1 -- 172.21.15.46:0/4204989123 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564a887db60 con 0x5564a8afdc20 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f40067fc700 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afe580 0x5564a8b0ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f4006ffd700 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afdc20 0x5564a8b0e820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80117d0 tx=0x7f3ff801ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81751eccda9fdcf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f3feaff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f3feaff5700 1 -- 172.21.15.46:0/4204989123 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff802e040 con 0x5564a8afdc20 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f4019654b80 1 -- 172.21.15.46:0/4204989123 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f400001b320 con 0x5564a8afdc20 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.787+0000 7f3feaff5700 1 -- 172.21.15.46:0/4204989123 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3ff802f070 con 0x5564a8afdc20 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.788+0000 7f3feaff5700 1 -- 172.21.15.46:0/4204989123 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ff8028c80 con 0x5564a8afdc20 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.788+0000 7f4019654b80 10 client.25261 fetch_fsmap learned FSMap version 27 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.788+0000 7f4019654b80 1 -- 172.21.15.46:0/4204989123 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5564a88d5cd0 con 0x5564a8afdc20 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.788+0000 7f3feaff5700 1 -- 172.21.15.46:0/4204989123 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400001b320 con 0x5564a8afdc20 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.788+0000 7f3feaff5700 1 -- 172.21.15.46:0/4204989123 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3ff8004950 con 0x5564a8afdc20 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.788+0000 7f4019654b80 10 client.25261 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.788+0000 7f4019654b80 10 client.25261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.788+0000 7f4019654b80 20 client.25261 populate_metadata read hostname 'smithi046' 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.788+0000 7f4019654b80 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564a8b684b0 0x5564a8b6a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.788+0000 7f4019654b80 4 client.25261 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.788+0000 7f4019654b80 1 -- 172.21.15.46:0/4204989123 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5564a889ecf0 con 0x5564a8b684b0 2024-03-18T20:35:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f40067fc700 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564a8b684b0 0x5564a8b6a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f40067fc700 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564a8b684b0 0x5564a8b6a8a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8eaa24ac4e2b9ac server_cookie=cbc9bb992352e6c0 in_seq=0 out_seq=0 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f3feaff5700 10 client.25261 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f3feaff5700 1 -- 172.21.15.46:0/4204989123 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5564a889ecf0 con 0x5564a8b684b0 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f3feaff5700 10 client.25261 handle_command_reply: tid=0 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f4019654b80 1 client.25261 shutdown 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f4019654b80 1 -- 172.21.15.46:0/4204989123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afdc20 msgr2=0x5564a8b0e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f4019654b80 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afdc20 0x5564a8b0e820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80117d0 tx=0x7f3ff801ace0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f4019654b80 1 -- 172.21.15.46:0/4204989123 shutdown_connections 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f4019654b80 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564a8b684b0 0x5564a8b6a8a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f4019654b80 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564a8aff420 0x5564a8b0cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f4019654b80 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564a8afdc20 0x5564a8b0e820 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f4019654b80 1 --2- 172.21.15.46:0/4204989123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564a8afe580 0x5564a8b0ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f3feaff5700 0 client.25261 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:35:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f4019654b80 1 -- 172.21.15.46:0/4204989123 >> 172.21.15.46:0/4204989123 conn(0x5564a8a73220 msgr2=0x5564a8b057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f4019654b80 1 -- 172.21.15.46:0/4204989123 shutdown_connections 2024-03-18T20:35:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.789+0000 7f4019654b80 1 -- 172.21.15.46:0/4204989123 wait complete. 2024-03-18T20:35:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f4019654b80 20 client.25261 trim_cache size 0 max 16384 2024-03-18T20:35:58.799 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:35:58.799 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:35:58.799 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:35:58.799 INFO:teuthology.orchestra.run.smithi046.stdout: "16612a36-e40d-4eca-b947-db2bb37a4f13": { 2024-03-18T20:35:58.799 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:35:58.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "16612a36-e40d-4eca-b947-db2bb37a4f13", 2024-03-18T20:35:58.799 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:35:58.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:35:58.800 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:35:58.800 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:35:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f400d4bf700 1 -- 172.21.15.46:0/4021626487 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3ff00a1230 msgr2=0x7f3ff00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f400d4bf700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3ff00a1230 0x7f3ff00a36f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4008103080 tx=0x7f3ff8009260 comp rx=0 tx=0).stop 2024-03-18T20:35:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f400d4bf700 1 -- 172.21.15.46:0/4021626487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4008106af0 msgr2=0x7f400819e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f400d4bf700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4008106af0 0x7f400819e230 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f400000ef30 tx=0x7f400000c3a0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f400d4bf700 1 -- 172.21.15.46:0/4021626487 shutdown_connections 2024-03-18T20:35:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f400d4bf700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f3ff00a1230 0x7f3ff00a36f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f400d4bf700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4008106af0 0x7f400819e230 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f400d4bf700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4008101890 0x7f4008196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f400d4bf700 1 --2- 172.21.15.46:0/4021626487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40081041c0 0x7f4008197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f400d4bf700 1 -- 172.21.15.46:0/4021626487 >> 172.21.15.46:0/4021626487 conn(0x7f40080fb200 msgr2=0x7f4008103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f400d4bf700 1 -- 172.21.15.46:0/4021626487 shutdown_connections 2024-03-18T20:35:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.790+0000 7f400d4bf700 1 -- 172.21.15.46:0/4021626487 wait complete. 2024-03-18T20:35:58.807 INFO:tasks.cephfs.filesystem:scrub status for tag:16612a36-e40d-4eca-b947-db2bb37a4f13 - {'path': '/', 'tag': '16612a36-e40d-4eca-b947-db2bb37a4f13', 'options': 'recursive,force'} 2024-03-18T20:35:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:58.866+0000 7f9643272700 1 -- 172.21.15.46:0/1215477520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f963c02ccb0 con 0x7f963c106ac0 2024-03-18T20:35:59.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.196+0000 7f96327fc700 1 -- 172.21.15.46:0/1215477520 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 18) v1 ==== 135933+0+0 (secure 0 0 0) 0x7f96340978e0 con 0x7f963c106ac0 2024-03-18T20:35:59.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.198+0000 7f96327fc700 1 -- 172.21.15.46:0/1215477520 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7f963405e2c0 con 0x7f963c106ac0 2024-03-18T20:35:59.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.202+0000 7f9643272700 1 -- 172.21.15.46:0/1215477520 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f96280a1230 msgr2=0x7f96280a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:59.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.202+0000 7f9643272700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f96280a1230 0x7f96280a36f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9638009dd0 tx=0x7f9638009040 comp rx=0 tx=0).stop 2024-03-18T20:35:59.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.202+0000 7f9643272700 1 -- 172.21.15.46:0/1215477520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c106ac0 msgr2=0x7f963c19e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:59.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.202+0000 7f9643272700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c106ac0 0x7f963c19e110 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f963400b7b0 tx=0x7f9634004730 comp rx=0 tx=0).stop 2024-03-18T20:35:59.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.202+0000 7f9643272700 1 -- 172.21.15.46:0/1215477520 shutdown_connections 2024-03-18T20:35:59.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.202+0000 7f9643272700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f96280a1230 0x7f96280a36f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.202+0000 7f9643272700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c106ac0 0x7f963c19e110 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.202+0000 7f9643272700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f963c101860 0x7f963c196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.202+0000 7f9643272700 1 --2- 172.21.15.46:0/1215477520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f963c104190 0x7f963c1970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.202+0000 7f9643272700 1 -- 172.21.15.46:0/1215477520 >> 172.21.15.46:0/1215477520 conn(0x7f963c0fb1b0 msgr2=0x7f963c1021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:59.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.202+0000 7f9643272700 1 -- 172.21.15.46:0/1215477520 shutdown_connections 2024-03-18T20:35:59.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.202+0000 7f9643272700 1 -- 172.21.15.46:0/1215477520 wait complete. 2024-03-18T20:35:59.226 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-18T20:35:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.369+0000 7f37febd5700 1 -- 172.21.15.46:0/3285102547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f8100d10 msgr2=0x7f37f8101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.369+0000 7f37febd5700 1 --2- 172.21.15.46:0/3285102547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f8100d10 0x7f37f8101130 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0099c0 tx=0x7f37ec009cd0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.369+0000 7f37febd5700 1 -- 172.21.15.46:0/3285102547 shutdown_connections 2024-03-18T20:35:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.369+0000 7f37febd5700 1 --2- 172.21.15.46:0/3285102547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f81028a0 0x7f37f8109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.369+0000 7f37febd5700 1 --2- 172.21.15.46:0/3285102547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f8100d10 0x7f37f8101130 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.369+0000 7f37febd5700 1 --2- 172.21.15.46:0/3285102547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f8101f00 0x7f37f8102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.369+0000 7f37febd5700 1 -- 172.21.15.46:0/3285102547 >> 172.21.15.46:0/3285102547 conn(0x7f37f80fc280 msgr2=0x7f37f80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.369+0000 7f37febd5700 1 -- 172.21.15.46:0/3285102547 shutdown_connections 2024-03-18T20:35:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.369+0000 7f37febd5700 1 -- 172.21.15.46:0/3285102547 wait complete. 2024-03-18T20:35:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37febd5700 1 Processor -- start 2024-03-18T20:35:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37febd5700 1 -- start start 2024-03-18T20:35:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37febd5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f8100d10 0x7f37f8074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37febd5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f8101f00 0x7f37f80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37febd5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f81028a0 0x7f37f80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37febd5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f8073e80 con 0x7f37f8101f00 2024-03-18T20:35:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37febd5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f8073ff0 con 0x7f37f8100d10 2024-03-18T20:35:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37febd5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f81a0d00 con 0x7f37f81028a0 2024-03-18T20:35:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37fc971700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f8100d10 0x7f37f8074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f8101f00 0x7f37f80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37fc971700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f8100d10 0x7f37f8074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33878/0 (socket says 172.21.15.46:33878) 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37fc971700 1 -- 172.21.15.46:0/912286944 learned_addr learned my addr 172.21.15.46:0/912286944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.370+0000 7f37fd172700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f81028a0 0x7f37f80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.371+0000 7f37fc971700 1 -- 172.21.15.46:0/912286944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f81028a0 msgr2=0x7f37f80736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.371+0000 7f37fc971700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f81028a0 0x7f37f80736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.371+0000 7f37fc971700 1 -- 172.21.15.46:0/912286944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f8101f00 msgr2=0x7f37f80731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.371+0000 7f37fc971700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f8101f00 0x7f37f80731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.371+0000 7f37fc971700 1 -- 172.21.15.46:0/912286944 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec0095f0 con 0x7f37f8100d10 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.371+0000 7f37fc971700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f8100d10 0x7f37f8074b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f37ec003730 tx=0x7f37ec00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.371+0000 7f37f5ffb700 1 -- 172.21.15.46:0/912286944 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec01d070 con 0x7f37f8100d10 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.371+0000 7f37f5ffb700 1 -- 172.21.15.46:0/912286944 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37ec005050 con 0x7f37f8100d10 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.371+0000 7f37f5ffb700 1 -- 172.21.15.46:0/912286944 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec004970 con 0x7f37f8100d10 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.371+0000 7f37febd5700 1 -- 172.21.15.46:0/912286944 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f81a0f30 con 0x7f37f8100d10 2024-03-18T20:35:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.371+0000 7f37febd5700 1 -- 172.21.15.46:0/912286944 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f81a12a0 con 0x7f37f8100d10 2024-03-18T20:35:59.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.372+0000 7f37febd5700 1 -- 172.21.15.46:0/912286944 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f81081e0 con 0x7f37f8100d10 2024-03-18T20:35:59.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.374+0000 7f37f5ffb700 1 -- 172.21.15.46:0/912286944 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) v1 ==== 135933+0+0 (secure 0 0 0) 0x7f37ec005470 con 0x7f37f8100d10 2024-03-18T20:35:59.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.374+0000 7f37f5ffb700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f37e80a1280 0x7f37e80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:59.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.374+0000 7f37f7fff700 1 -- 172.21.15.46:0/912286944 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f37e80a1280 msgr2=0x7f37e80a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3400106473 2024-03-18T20:35:59.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.374+0000 7f37f5ffb700 1 -- 172.21.15.46:0/912286944 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f37ec0cd6d0 con 0x7f37f8100d10 2024-03-18T20:35:59.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.374+0000 7f37f7fff700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f37e80a1280 0x7f37e80a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T20:35:59.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.378+0000 7f37f5ffb700 1 -- 172.21.15.46:0/912286944 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f37ec096da0 con 0x7f37f8100d10 2024-03-18T20:35:59.470 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:35:59 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: ignoring --setuser ceph since I am not root 2024-03-18T20:35:59.470 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:35:59 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: ignoring --setgroup ceph since I am not root 2024-03-18T20:35:59.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:59 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3635930712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:59.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:59 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:35:59.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:59 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:35:59.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:59 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:35:59.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:59 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:35:59.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:35:59 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1215477520' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-18T20:35:59.540 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:35:59 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: ignoring --setuser ceph since I am not root 2024-03-18T20:35:59.541 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:35:59 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: ignoring --setgroup ceph since I am not root 2024-03-18T20:35:59.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:59 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3635930712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:59.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:59 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:35:59.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:59 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:35:59.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:59 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:35:59.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:59 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:35:59.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:35:59 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1215477520' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-18T20:35:59.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:59 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3635930712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T20:35:59.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:59 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:35:59.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:59 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:35:59.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:59 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:35:59.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:59 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:35:59.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:35:59 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1215477520' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-18T20:35:59.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:35:59 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: ignoring --setuser ceph since I am not root 2024-03-18T20:35:59.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:35:59 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: ignoring --setgroup ceph since I am not root 2024-03-18T20:35:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.575+0000 7f37f7fff700 1 -- 172.21.15.46:0/912286944 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f37e80a1280 msgr2=0x7f37e80a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3400106473 2024-03-18T20:35:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.575+0000 7f37f7fff700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f37e80a1280 0x7f37e80a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T20:35:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.609+0000 7f37febd5700 1 -- 172.21.15.46:0/912286944 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f37f802d0e0 con 0x7f37f8100d10 2024-03-18T20:35:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.615+0000 7f37f5ffb700 1 -- 172.21.15.46:0/912286944 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f37ec09a3c0 con 0x7f37f8100d10 2024-03-18T20:35:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.617+0000 7f37febd5700 1 -- 172.21.15.46:0/912286944 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f37e80a1280 msgr2=0x7f37e80a3740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:35:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.617+0000 7f37febd5700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f37e80a1280 0x7f37e80a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.617+0000 7f37febd5700 1 -- 172.21.15.46:0/912286944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f8100d10 msgr2=0x7f37f8074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.617+0000 7f37febd5700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f8100d10 0x7f37f8074b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f37ec003730 tx=0x7f37ec00f690 comp rx=0 tx=0).stop 2024-03-18T20:35:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.617+0000 7f37febd5700 1 -- 172.21.15.46:0/912286944 shutdown_connections 2024-03-18T20:35:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.617+0000 7f37febd5700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f37e80a1280 0x7f37e80a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.617+0000 7f37febd5700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f81028a0 0x7f37f80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.617+0000 7f37febd5700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f8100d10 0x7f37f8074b10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.617+0000 7f37febd5700 1 --2- 172.21.15.46:0/912286944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37f8101f00 0x7f37f80731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.617+0000 7f37febd5700 1 -- 172.21.15.46:0/912286944 >> 172.21.15.46:0/912286944 conn(0x7f37f80fc280 msgr2=0x7f37f8105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:59.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.617+0000 7f37febd5700 1 -- 172.21.15.46:0/912286944 shutdown_connections 2024-03-18T20:35:59.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.617+0000 7f37febd5700 1 -- 172.21.15.46:0/912286944 wait complete. 2024-03-18T20:35:59.636 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-18T20:35:59.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.772+0000 7f2a413a6700 1 -- 172.21.15.46:0/3444167803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c0ff5f0 msgr2=0x7f2a3c0ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:59.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.772+0000 7f2a413a6700 1 --2- 172.21.15.46:0/3444167803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c0ff5f0 0x7f2a3c0ffa10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00b0a0 tx=0x7f2a2c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.772+0000 7f2a413a6700 1 -- 172.21.15.46:0/3444167803 shutdown_connections 2024-03-18T20:35:59.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.772+0000 7f2a413a6700 1 --2- 172.21.15.46:0/3444167803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c0ff5f0 0x7f2a3c0ffa10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.772+0000 7f2a413a6700 1 --2- 172.21.15.46:0/3444167803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a3c100910 0x7f2a3c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.772+0000 7f2a413a6700 1 --2- 172.21.15.46:0/3444167803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a3c0fff50 0x7f2a3c1003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.772+0000 7f2a413a6700 1 -- 172.21.15.46:0/3444167803 >> 172.21.15.46:0/3444167803 conn(0x7f2a3c0fb170 msgr2=0x7f2a3c0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:35:59.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.772+0000 7f2a413a6700 1 -- 172.21.15.46:0/3444167803 shutdown_connections 2024-03-18T20:35:59.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.772+0000 7f2a413a6700 1 -- 172.21.15.46:0/3444167803 wait complete. 2024-03-18T20:35:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a413a6700 1 Processor -- start 2024-03-18T20:35:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a413a6700 1 -- start start 2024-03-18T20:35:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a413a6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a3c0ff5f0 0x7f2a3c198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a413a6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a3c0fff50 0x7f2a3c199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a413a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c100910 0x7f2a3c1a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a413a6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c1a0b10 con 0x7f2a3c100910 2024-03-18T20:35:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a413a6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c1a0c50 con 0x7f2a3c0fff50 2024-03-18T20:35:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a413a6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c1a0de0 con 0x7f2a3c0ff5f0 2024-03-18T20:35:59.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c100910 0x7f2a3c1a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a3a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a3c0fff50 0x7f2a3c199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c100910 0x7f2a3c1a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38920/0 (socket says 172.21.15.46:38920) 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a3b7fe700 1 -- 172.21.15.46:0/700320996 learned_addr learned my addr 172.21.15.46:0/700320996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.773+0000 7f2a3affd700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a3c0ff5f0 0x7f2a3c198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.774+0000 7f2a3a7fc700 1 -- 172.21.15.46:0/700320996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a3c0ff5f0 msgr2=0x7f2a3c198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.774+0000 7f2a3a7fc700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a3c0ff5f0 0x7f2a3c198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.774+0000 7f2a3a7fc700 1 -- 172.21.15.46:0/700320996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c100910 msgr2=0x7f2a3c1a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.774+0000 7f2a3a7fc700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c100910 0x7f2a3c1a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.774+0000 7f2a3a7fc700 1 -- 172.21.15.46:0/700320996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a34009520 con 0x7f2a3c0fff50 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.774+0000 7f2a3a7fc700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a3c0fff50 0x7f2a3c199520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000e9f0 tx=0x7f2a3000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.774+0000 7f2a23fff700 1 -- 172.21.15.46:0/700320996 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000c9c0 con 0x7f2a3c0fff50 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.774+0000 7f2a413a6700 1 -- 172.21.15.46:0/700320996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c009d00 con 0x7f2a3c0fff50 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.774+0000 7f2a23fff700 1 -- 172.21.15.46:0/700320996 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a30010430 con 0x7f2a3c0fff50 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.774+0000 7f2a23fff700 1 -- 172.21.15.46:0/700320996 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30018680 con 0x7f2a3c0fff50 2024-03-18T20:35:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.774+0000 7f2a413a6700 1 -- 172.21.15.46:0/700320996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c1a1440 con 0x7f2a3c0fff50 2024-03-18T20:35:59.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.775+0000 7f2a413a6700 1 -- 172.21.15.46:0/700320996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c10a3d0 con 0x7f2a3c0fff50 2024-03-18T20:35:59.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.776+0000 7f2a23fff700 1 -- 172.21.15.46:0/700320996 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) v1 ==== 135933+0+0 (secure 0 0 0) 0x7f2a3000cb20 con 0x7f2a3c0fff50 2024-03-18T20:35:59.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.777+0000 7f2a23fff700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2a240a1200 0x7f2a240a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:35:59.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.777+0000 7f2a3affd700 1 -- 172.21.15.46:0/700320996 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2a240a1200 msgr2=0x7f2a240a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3400106473 2024-03-18T20:35:59.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.777+0000 7f2a23fff700 1 -- 172.21.15.46:0/700320996 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2a30014070 con 0x7f2a3c0fff50 2024-03-18T20:35:59.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.777+0000 7f2a3affd700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2a240a1200 0x7f2a240a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T20:35:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.781+0000 7f2a23fff700 1 -- 172.21.15.46:0/700320996 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2a300529f0 con 0x7f2a3c0fff50 2024-03-18T20:35:59.902 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:35:59 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:35:59.633+0000 7f0523688200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T20:35:59.903 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:35:59 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:35:59.725+0000 7f0523688200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T20:35:59.903 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:35:59 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:35:59.814+0000 7f0523688200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T20:35:59.903 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:35:59 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:35:59.634+0000 7f0b9d7e6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T20:35:59.903 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:35:59 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:35:59.726+0000 7f0b9d7e6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T20:35:59.903 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:35:59 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:35:59.814+0000 7f0b9d7e6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T20:35:59.918 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:35:59 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:35:59.703+0000 7ff601711200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T20:35:59.918 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:35:59 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:35:59.814+0000 7ff601711200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T20:35:59.918 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:35:59 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:35:59.915+0000 7ff601711200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T20:35:59.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.978+0000 7f2a3affd700 1 -- 172.21.15.46:0/700320996 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2a240a1200 msgr2=0x7f2a240a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3400106473 2024-03-18T20:35:59.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:35:59.978+0000 7f2a3affd700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2a240a1200 0x7f2a240a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T20:36:00.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.013+0000 7f2a413a6700 1 -- 172.21.15.46:0/700320996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f2a3c02ccb0 con 0x7f2a3c0fff50 2024-03-18T20:36:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.019+0000 7f2a23fff700 1 -- 172.21.15.46:0/700320996 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2a30096250 con 0x7f2a3c0fff50 2024-03-18T20:36:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.021+0000 7f2a413a6700 1 -- 172.21.15.46:0/700320996 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2a240a1200 msgr2=0x7f2a240a36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:36:00.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.021+0000 7f2a413a6700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2a240a1200 0x7f2a240a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:00.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.021+0000 7f2a413a6700 1 -- 172.21.15.46:0/700320996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a3c0fff50 msgr2=0x7f2a3c199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:00.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.021+0000 7f2a413a6700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a3c0fff50 0x7f2a3c199520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000e9f0 tx=0x7f2a3000ed00 comp rx=0 tx=0).stop 2024-03-18T20:36:00.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.021+0000 7f2a413a6700 1 -- 172.21.15.46:0/700320996 shutdown_connections 2024-03-18T20:36:00.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.021+0000 7f2a413a6700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7f2a240a1200 0x7f2a240a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:00.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.021+0000 7f2a413a6700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3c100910 0x7f2a3c1a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:00.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.021+0000 7f2a413a6700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a3c0ff5f0 0x7f2a3c198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:00.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.021+0000 7f2a413a6700 1 --2- 172.21.15.46:0/700320996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a3c0fff50 0x7f2a3c199520 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:00.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.021+0000 7f2a413a6700 1 -- 172.21.15.46:0/700320996 >> 172.21.15.46:0/700320996 conn(0x7f2a3c0fb170 msgr2=0x7f2a3c0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:00.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.021+0000 7f2a413a6700 1 -- 172.21.15.46:0/700320996 shutdown_connections 2024-03-18T20:36:00.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.022+0000 7f2a413a6700 1 -- 172.21.15.46:0/700320996 wait complete. 2024-03-18T20:36:00.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-18T20:36:00.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.176+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3583771743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d8104150 msgr2=0x7fa7d8106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:00.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.176+0000 7fa7df3a3700 1 --2- 172.21.15.46:0/3583771743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d8104150 0x7fa7d8106540 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009990 tx=0x7fa7c8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:36:00.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.176+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3583771743 shutdown_connections 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.176+0000 7fa7df3a3700 1 --2- 172.21.15.46:0/3583771743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d8106a80 0x7fa7d8108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.176+0000 7fa7df3a3700 1 --2- 172.21.15.46:0/3583771743 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d8101820 0x7fa7d8103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.176+0000 7fa7df3a3700 1 --2- 172.21.15.46:0/3583771743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d8104150 0x7fa7d8106540 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.176+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3583771743 >> 172.21.15.46:0/3583771743 conn(0x7fa7d80fb1b0 msgr2=0x7fa7d80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.177+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3583771743 shutdown_connections 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.177+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3583771743 wait complete. 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.177+0000 7fa7df3a3700 1 Processor -- start 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.177+0000 7fa7df3a3700 1 -- start start 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.177+0000 7fa7df3a3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d8101820 0x7fa7d8196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7df3a3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d8104150 0x7fa7d8197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7df3a3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d8106a80 0x7fa7d819e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7df3a3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d819e730 con 0x7fa7d8106a80 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7df3a3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d819e870 con 0x7fa7d8104150 2024-03-18T20:36:00.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7df3a3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d819ea00 con 0x7fa7d8101820 2024-03-18T20:36:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7dd940700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d8106a80 0x7fa7d819e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7dd940700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d8106a80 0x7fa7d819e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38938/0 (socket says 172.21.15.46:38938) 2024-03-18T20:36:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7dd940700 1 -- 172.21.15.46:0/3120907198 learned_addr learned my addr 172.21.15.46:0/3120907198 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7dc93e700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d8104150 0x7fa7d8197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7dd13f700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d8101820 0x7fa7d8196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7dd940700 1 -- 172.21.15.46:0/3120907198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d8101820 msgr2=0x7fa7d8196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7dd940700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d8101820 0x7fa7d8196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7dd940700 1 -- 172.21.15.46:0/3120907198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d8104150 msgr2=0x7fa7d8197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7dd940700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d8104150 0x7fa7d8197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7dd940700 1 -- 172.21.15.46:0/3120907198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c80095f0 con 0x7fa7d8106a80 2024-03-18T20:36:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7dc93e700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7d8104150 0x7fa7d8197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:00.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7dd940700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d8106a80 0x7fa7d819e160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000ef30 tx=0x7fa7d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:00.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.178+0000 7fa7ce7fc700 1 -- 172.21.15.46:0/3120907198 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0018070 con 0x7fa7d8106a80 2024-03-18T20:36:00.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.179+0000 7fa7ce7fc700 1 -- 172.21.15.46:0/3120907198 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7d000f040 con 0x7fa7d8106a80 2024-03-18T20:36:00.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.179+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3120907198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d819ec80 con 0x7fa7d8106a80 2024-03-18T20:36:00.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.179+0000 7fa7ce7fc700 1 -- 172.21.15.46:0/3120907198 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d000cd00 con 0x7fa7d8106a80 2024-03-18T20:36:00.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.179+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3120907198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d819f140 con 0x7fa7d8106a80 2024-03-18T20:36:00.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.180+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3120907198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d80fcd30 con 0x7fa7d8106a80 2024-03-18T20:36:00.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.181+0000 7fa7ce7fc700 1 -- 172.21.15.46:0/3120907198 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) v1 ==== 135933+0+0 (secure 0 0 0) 0x7fa7d0003a00 con 0x7fa7d8106a80 2024-03-18T20:36:00.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.181+0000 7fa7ce7fc700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 0x7fa7c40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:00.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.181+0000 7fa7dd13f700 1 -- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 msgr2=0x7fa7c40a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3400106473 2024-03-18T20:36:00.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.182+0000 7fa7ce7fc700 1 -- 172.21.15.46:0/3120907198 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa7d00cc650 con 0x7fa7d8106a80 2024-03-18T20:36:00.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.182+0000 7fa7dd13f700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 0x7fa7c40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T20:36:00.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.186+0000 7fa7ce7fc700 1 -- 172.21.15.46:0/3120907198 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa7d0095cc0 con 0x7fa7d8106a80 2024-03-18T20:36:00.190 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:00.011+0000 7ff601711200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T20:36:00.193 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:35:59 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:35:59.900+0000 7f0b9d7e6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T20:36:00.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.383+0000 7fa7dd13f700 1 -- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 msgr2=0x7fa7c40a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3400106473 2024-03-18T20:36:00.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.383+0000 7fa7dd13f700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 0x7fa7c40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T20:36:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.449+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3120907198 --> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7d80611f0 con 0x7fa7c40a1200 2024-03-18T20:36:00.470 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:35:59 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:35:59.899+0000 7f0523688200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T20:36:00.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:00 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1215477520' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-18T20:36:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:00 smithi073 ceph-mon[47384]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-03-18T20:36:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:00 smithi073 ceph-mon[47384]: from='client.? ' entity='client.admin' 2024-03-18T20:36:00.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:00 smithi073 ceph-mon[47384]: from='client.? ' entity='client.admin' 2024-03-18T20:36:00.529 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:00 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1215477520' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-18T20:36:00.529 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:00 smithi155 ceph-mon[48344]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-03-18T20:36:00.529 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:00 smithi155 ceph-mon[48344]: from='client.? ' entity='client.admin' 2024-03-18T20:36:00.529 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:00 smithi155 ceph-mon[48344]: from='client.? ' entity='client.admin' 2024-03-18T20:36:00.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:00 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1215477520' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-18T20:36:00.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:00 smithi046 ceph-mon[43495]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-03-18T20:36:00.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:00 smithi046 ceph-mon[43495]: from='client.? ' entity='client.admin' 2024-03-18T20:36:00.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:00 smithi046 ceph-mon[43495]: from='client.? ' entity='client.admin' 2024-03-18T20:36:00.781 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:00.525+0000 7f0b9d7e6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:36:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.784+0000 7fa7dd13f700 1 -- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 msgr2=0x7fa7c40a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3400106473 2024-03-18T20:36:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:00.784+0000 7fa7dd13f700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 0x7fa7c40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T20:36:00.860 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:00 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:00.521+0000 7f0523688200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:36:00.860 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:00 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:00.771+0000 7f0523688200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T20:36:01.040 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:00.778+0000 7f0b9d7e6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T20:36:01.041 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:00.865+0000 7f0b9d7e6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T20:36:01.041 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:00.974+0000 7f0b9d7e6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T20:36:01.046 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:00.760+0000 7ff601711200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:36:01.126 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:00 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:00.856+0000 7f0523688200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T20:36:01.126 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:00 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:00.966+0000 7f0523688200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T20:36:01.457 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:01.047+0000 7ff601711200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T20:36:01.457 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:01.145+0000 7ff601711200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T20:36:01.458 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:01.270+0000 7ff601711200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T20:36:01.458 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:01.454+0000 7ff601711200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T20:36:01.470 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:01.122+0000 7f0523688200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T20:36:01.470 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:01.249+0000 7f0523688200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T20:36:01.470 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:01.334+0000 7f0523688200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T20:36:01.540 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:01.132+0000 7f0b9d7e6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T20:36:01.540 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:01.259+0000 7f0b9d7e6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T20:36:01.541 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:01.345+0000 7f0b9d7e6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T20:36:01.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:01.585+0000 7fa7dd13f700 1 -- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 msgr2=0x7fa7c40a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3400106473 2024-03-18T20:36:01.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:01.585+0000 7fa7dd13f700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 0x7fa7c40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T20:36:01.796 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:01.599+0000 7ff601711200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T20:36:01.797 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:01.697+0000 7ff601711200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T20:36:01.970 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:01.699+0000 7f0523688200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T20:36:02.042 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:01.715+0000 7f0b9d7e6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T20:36:02.374 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:02 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:02.085+0000 7f0523688200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T20:36:02.374 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:02 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:02.278+0000 7f0523688200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T20:36:02.392 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:02 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:02.103+0000 7f0b9d7e6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T20:36:02.392 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:02 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:02.297+0000 7f0b9d7e6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T20:36:02.546 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:02 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:02.123+0000 7ff601711200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T20:36:02.790 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:02 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:02.389+0000 7f0b9d7e6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T20:36:02.899 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:02 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:02.568+0000 7ff601711200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T20:36:02.899 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:02 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:02.789+0000 7ff601711200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T20:36:02.899 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:02 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:02.895+0000 7ff601711200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T20:36:03.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:03.186+0000 7fa7dd13f700 1 -- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 msgr2=0x7fa7c40a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3400106473 2024-03-18T20:36:03.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:03.186+0000 7fa7dd13f700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 0x7fa7c40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T20:36:03.252 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:03 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:03.153+0000 7f0b9d7e6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T20:36:03.252 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:02 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:36:03.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:02 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:36:03.372 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:02 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:36:03.372 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:02 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:02.370+0000 7f0523688200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T20:36:03.372 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:03 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:03.126+0000 7f0523688200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T20:36:03.372 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:03 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:03.222+0000 7f0523688200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T20:36:03.372 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:03 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:03.368+0000 7f0523688200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T20:36:03.540 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:03 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:03.249+0000 7f0b9d7e6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T20:36:03.541 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:03 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:03.396+0000 7f0b9d7e6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T20:36:04.046 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:03 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:03.775+0000 7ff601711200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T20:36:04.046 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:03 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:03.886+0000 7ff601711200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T20:36:04.290 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:04.022+0000 7f0b9d7e6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T20:36:04.290 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:04.203+0000 7f0b9d7e6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T20:36:04.364 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:03 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:03.986+0000 7f0523688200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T20:36:04.364 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:04 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:04.168+0000 7f0523688200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T20:36:04.364 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:04 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:04.360+0000 7f0523688200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T20:36:04.546 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:04 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:04.057+0000 7ff601711200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T20:36:04.790 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:04.397+0000 7f0b9d7e6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T20:36:05.046 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:04 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:04.784+0000 7ff601711200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T20:36:05.046 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:04 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:04.997+0000 7ff601711200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T20:36:05.546 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:05 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:05.222+0000 7ff601711200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T20:36:05.568 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:05.281+0000 7f0523688200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T20:36:05.614 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:05.324+0000 7f0b9d7e6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T20:36:05.846 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:05.565+0000 7f0523688200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:36:05.846 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:05.653+0000 7f0523688200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T20:36:05.893 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:05.609+0000 7f0b9d7e6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:36:05.893 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:05.698+0000 7f0b9d7e6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T20:36:05.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:05.918+0000 7fa7ce7fc700 1 -- 172.21.15.46:0/3120907198 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 19) v1 ==== 135933+0+0 (secure 0 0 0) 0x7fa7d0003cb0 con 0x7fa7d8106a80 2024-03-18T20:36:06.220 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y[48859]: 2024-03-18T20:36:05.842+0000 7f0523688200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T20:36:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-mon[47384]: Standby manager daemon y restarted 2024-03-18T20:36:06.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-mon[47384]: Standby manager daemon y started 2024-03-18T20:36:06.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.73:0/3748830484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T20:36:06.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.73:0/3748830484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T20:36:06.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.73:0/3748830484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T20:36:06.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.73:0/3748830484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T20:36:06.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-mon[47384]: Standby manager daemon z restarted 2024-03-18T20:36:06.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-mon[47384]: Standby manager daemon z started 2024-03-18T20:36:06.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.155:0/3001312378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T20:36:06.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:05 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.155:0/3001312378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T20:36:06.290 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z[50209]: 2024-03-18T20:36:05.889+0000 7f0b9d7e6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T20:36:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-mon[48344]: Standby manager daemon y restarted 2024-03-18T20:36:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-mon[48344]: Standby manager daemon y started 2024-03-18T20:36:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.73:0/3748830484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T20:36:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.73:0/3748830484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T20:36:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.73:0/3748830484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T20:36:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.73:0/3748830484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T20:36:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-mon[48344]: Standby manager daemon z restarted 2024-03-18T20:36:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-mon[48344]: Standby manager daemon z started 2024-03-18T20:36:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.155:0/3001312378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T20:36:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:05 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.155:0/3001312378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T20:36:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:05 smithi046 ceph-mon[43495]: Standby manager daemon y restarted 2024-03-18T20:36:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:05 smithi046 ceph-mon[43495]: Standby manager daemon y started 2024-03-18T20:36:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:05 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.73:0/3748830484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T20:36:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:05 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.73:0/3748830484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T20:36:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:05 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.73:0/3748830484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T20:36:06.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:05 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.73:0/3748830484' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T20:36:06.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:05 smithi046 ceph-mon[43495]: Standby manager daemon z restarted 2024-03-18T20:36:06.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:05 smithi046 ceph-mon[43495]: Standby manager daemon z started 2024-03-18T20:36:06.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:05 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.155:0/3001312378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T20:36:06.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:05 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.155:0/3001312378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T20:36:06.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:06.390+0000 7fa7dd13f700 1 -- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 msgr2=0x7fa7c40a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3400106473 2024-03-18T20:36:06.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:06.390+0000 7fa7dd13f700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 0x7fa7c40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-18T20:36:06.633 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:06 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:06.295+0000 7ff601711200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T20:36:06.926 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:06 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:06.630+0000 7ff601711200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T20:36:06.926 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:06 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:06.733+0000 7ff601711200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T20:36:06.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:06.985+0000 7fa7ce7fc700 1 -- 172.21.15.46:0/3120907198 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 20) v1 ==== 135550+0+0 (secure 0 0 0) 0x7fa7d0096090 con 0x7fa7d8106a80 2024-03-18T20:36:06.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:06.986+0000 7fa7ce7fc700 1 -- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 msgr2=0x7fa7c40a36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:36:06.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:06.986+0000 7fa7ce7fc700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/3400106473,v1:172.21.15.46:6801/3400106473] conn(0x7fa7c40a1200 0x7fa7c40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:07.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:06 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.155:0/3001312378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T20:36:07.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:06 smithi073 ceph-mon[47384]: from='mgr.? 172.21.15.155:0/3001312378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T20:36:07.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:06 smithi073 ceph-mon[47384]: mgrmap e19: x(active, since 6m), standbys: z, y 2024-03-18T20:36:07.270 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:06 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:06.959+0000 7ff601711200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T20:36:07.270 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:07.266+0000 7ff5612e1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:36:07.270 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:06 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.155:0/3001312378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T20:36:07.270 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:06 smithi046 ceph-mon[43495]: from='mgr.? 172.21.15.155:0/3001312378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T20:36:07.270 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:06 smithi046 ceph-mon[43495]: mgrmap e19: x(active, since 6m), standbys: z, y 2024-03-18T20:36:07.271 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:06 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.155:0/3001312378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T20:36:07.271 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:06 smithi155 ceph-mon[48344]: from='mgr.? 172.21.15.155:0/3001312378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T20:36:07.271 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:06 smithi155 ceph-mon[48344]: mgrmap e19: x(active, since 6m), standbys: z, y 2024-03-18T20:36:07.546 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:07.266+0000 7ff5612e1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:36:07.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:07.266+0000 7ff5612e1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:36:07.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:07.266+0000 7ff5612e1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:36:07.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:07.266+0000 7ff5612e1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:36:07.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:07.267+0000 7ff5642e7700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:36:07.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:07.267+0000 7ff5642e7700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:36:07.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:07.267+0000 7ff5642e7700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:36:07.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:07.267+0000 7ff5642e7700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:36:07.547 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:36:07.267+0000 7ff5642e7700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:36:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:07.991+0000 7fa7ce7fc700 1 -- 172.21.15.46:0/3120907198 <== mon.0 v2:172.21.15.46:3300/0 9 ==== mgrmap(e 21) v1 ==== 135677+0+0 (secure 0 0 0) 0x7fa7d00ca200 con 0x7fa7d8106a80 2024-03-18T20:36:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:07.991+0000 7fa7ce7fc700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa7c401bb90 0x7fa7c401bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:07.991+0000 7fa7ce7fc700 1 -- 172.21.15.46:0/3120907198 --> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7d80611f0 con 0x7fa7c401bb90 2024-03-18T20:36:07.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:07.992+0000 7fa7dd13f700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa7c401bb90 0x7fa7c401bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:07.995+0000 7fa7dd13f700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa7c401bb90 0x7fa7c401bf70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009880 tx=0x7fa7d4009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:08.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.065+0000 7fa7ce7fc700 1 -- 172.21.15.46:0/3120907198 <== mgr.15498 v2:172.21.15.46:6800/805586292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa7d80611f0 con 0x7fa7c401bb90 2024-03-18T20:36:08.069 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path / 2024-03-18T20:36:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.068+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa7c401bb90 msgr2=0x7fa7c401bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.068+0000 7fa7df3a3700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa7c401bb90 0x7fa7c401bf70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009880 tx=0x7fa7d4009260 comp rx=0 tx=0).stop 2024-03-18T20:36:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.068+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d8106a80 msgr2=0x7fa7d819e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.068+0000 7fa7df3a3700 1 --2- 172.21.15.46:0/3120907198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7d8106a80 0x7fa7d819e160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000ef30 tx=0x7fa7d000c3a0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.068+0000 7fa7dd940700 1 -- 172.21.15.46:0/3120907198 reap_dead start 2024-03-18T20:36:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.068+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3120907198 shutdown_connections 2024-03-18T20:36:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.068+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3120907198 >> 172.21.15.46:0/3120907198 conn(0x7fa7d80fb1b0 msgr2=0x7fa7d8102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:08.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.070+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3120907198 shutdown_connections 2024-03-18T20:36:08.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.070+0000 7fa7df3a3700 1 -- 172.21.15.46:0/3120907198 wait complete. 2024-03-18T20:36:08.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-18T20:36:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: Active manager daemon x restarted 2024-03-18T20:36:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: Activating manager daemon x 2024-03-18T20:36:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: osdmap e81: 12 total, 12 up, 12 in 2024-03-18T20:36:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: mgrmap e20: x(active, starting, since 0.0137745s), standbys: z, y 2024-03-18T20:36:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:36:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:36:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:36:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T20:36:08.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T20:36:08.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T20:36:08.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T20:36:08.224 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T20:36:08.224 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: Manager daemon x is now available 2024-03-18T20:36:08.224 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:36:08.224 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:36:08.224 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T20:36:08.224 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T20:36:08.224 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T20:36:08.224 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:07 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T20:36:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.241+0000 7f6ad414e700 1 -- 172.21.15.46:0/118293420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc0ffc20 msgr2=0x7f6acc100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.241+0000 7f6ad414e700 1 --2- 172.21.15.46:0/118293420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc0ffc20 0x7f6acc100040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80099c0 tx=0x7f6ac8009cd0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.241+0000 7f6ad414e700 1 -- 172.21.15.46:0/118293420 shutdown_connections 2024-03-18T20:36:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.241+0000 7f6ad414e700 1 --2- 172.21.15.46:0/118293420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc0ffc20 0x7f6acc100040 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.241+0000 7f6ad414e700 1 --2- 172.21.15.46:0/118293420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6acc1017b0 0x7f6acc108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.241+0000 7f6ad414e700 1 --2- 172.21.15.46:0/118293420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6acc100e10 0x7f6acc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.241+0000 7f6ad414e700 1 -- 172.21.15.46:0/118293420 >> 172.21.15.46:0/118293420 conn(0x7f6acc0fb190 msgr2=0x7f6acc0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:08.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.241+0000 7f6ad414e700 1 -- 172.21.15.46:0/118293420 shutdown_connections 2024-03-18T20:36:08.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.242+0000 7f6ad414e700 1 -- 172.21.15.46:0/118293420 wait complete. 2024-03-18T20:36:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.242+0000 7f6ad414e700 1 Processor -- start 2024-03-18T20:36:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.242+0000 7f6ad414e700 1 -- start start 2024-03-18T20:36:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.242+0000 7f6ad414e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6acc0ffc20 0x7f6acc198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.242+0000 7f6ad414e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc100e10 0x7f6acc199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.242+0000 7f6ad414e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6acc1017b0 0x7f6acc1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.242+0000 7f6ad414e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc1a09f0 con 0x7f6acc100e10 2024-03-18T20:36:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.242+0000 7f6ad414e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc1a0b30 con 0x7f6acc0ffc20 2024-03-18T20:36:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.242+0000 7f6ad414e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc1a0cc0 con 0x7f6acc1017b0 2024-03-18T20:36:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad16e9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc100e10 0x7f6acc199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad1eea700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6acc0ffc20 0x7f6acc198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad26eb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6acc1017b0 0x7f6acc1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:08.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad16e9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc100e10 0x7f6acc199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55062/0 (socket says 172.21.15.46:55062) 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad16e9700 1 -- 172.21.15.46:0/2098192601 learned_addr learned my addr 172.21.15.46:0/2098192601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad16e9700 1 -- 172.21.15.46:0/2098192601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6acc1017b0 msgr2=0x7f6acc1a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad16e9700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6acc1017b0 0x7f6acc1a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad16e9700 1 -- 172.21.15.46:0/2098192601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6acc0ffc20 msgr2=0x7f6acc198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad16e9700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6acc0ffc20 0x7f6acc198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad16e9700 1 -- 172.21.15.46:0/2098192601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac80095f0 con 0x7f6acc100e10 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad1eea700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6acc0ffc20 0x7f6acc198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad16e9700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc100e10 0x7f6acc199450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00e9a0 tx=0x7f6abc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ac2ffd700 1 -- 172.21.15.46:0/2098192601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc009790 con 0x7f6acc100e10 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ac2ffd700 1 -- 172.21.15.46:0/2098192601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6abc010430 con 0x7f6acc100e10 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.243+0000 7f6ad414e700 1 -- 172.21.15.46:0/2098192601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6acc1a0fa0 con 0x7f6acc100e10 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.244+0000 7f6ac2ffd700 1 -- 172.21.15.46:0/2098192601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc018680 con 0x7f6acc100e10 2024-03-18T20:36:08.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.244+0000 7f6ad414e700 1 -- 172.21.15.46:0/2098192601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc1a1420 con 0x7f6acc100e10 2024-03-18T20:36:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.245+0000 7f6ad414e700 1 -- 172.21.15.46:0/2098192601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc0673a0 con 0x7f6acc100e10 2024-03-18T20:36:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.246+0000 7f6ac2ffd700 1 -- 172.21.15.46:0/2098192601 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 21) v1 ==== 135677+0+0 (secure 0 0 0) 0x7f6abc0105a0 con 0x7f6acc100e10 2024-03-18T20:36:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.246+0000 7f6ac2ffd700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6ab80a0f20 0x7f6ab80a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.247+0000 7f6ac2ffd700 1 -- 172.21.15.46:0/2098192601 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6abc014070 con 0x7f6acc100e10 2024-03-18T20:36:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.247+0000 7f6ad1eea700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6ab80a0f20 0x7f6ab80a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.247+0000 7f6ad1eea700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6ab80a0f20 0x7f6ab80a33e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8003730 tx=0x7f6ac801a040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.252+0000 7f6ac2ffd700 1 -- 172.21.15.46:0/2098192601 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6abc0529f0 con 0x7f6acc100e10 2024-03-18T20:36:08.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: Active manager daemon x restarted 2024-03-18T20:36:08.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: Activating manager daemon x 2024-03-18T20:36:08.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: osdmap e81: 12 total, 12 up, 12 in 2024-03-18T20:36:08.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: mgrmap e20: x(active, starting, since 0.0137745s), standbys: z, y 2024-03-18T20:36:08.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:36:08.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T20:36:08.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T20:36:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T20:36:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T20:36:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T20:36:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:36:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:36:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:36:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:36:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:36:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:36:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:36:08.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: Manager daemon x is now available 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T20:36:08.294 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:07 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: Active manager daemon x restarted 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: Activating manager daemon x 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: osdmap e81: 12 total, 12 up, 12 in 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: mgrmap e20: x(active, starting, since 0.0137745s), standbys: z, y 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T20:36:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T20:36:08.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: Manager daemon x is now available 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T20:36:08.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T20:36:08.300 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T20:36:08.300 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:07 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T20:36:08.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.538+0000 7f6ad414e700 1 -- 172.21.15.46:0/2098192601 --> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f6acc069730 con 0x7f6ab80a0f20 2024-03-18T20:36:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.607+0000 7f6ac2ffd700 1 -- 172.21.15.46:0/2098192601 <== mgr.15498 v2:172.21.15.46:6800/805586292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f6acc069730 con 0x7f6ab80a0f20 2024-03-18T20:36:08.610 INFO:teuthology.orchestra.run.smithi046.stdout:Retention added to path / 2024-03-18T20:36:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.610+0000 7f6ad414e700 1 -- 172.21.15.46:0/2098192601 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6ab80a0f20 msgr2=0x7f6ab80a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.610+0000 7f6ad414e700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6ab80a0f20 0x7f6ab80a33e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8003730 tx=0x7f6ac801a040 comp rx=0 tx=0).stop 2024-03-18T20:36:08.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.610+0000 7f6ad414e700 1 -- 172.21.15.46:0/2098192601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc100e10 msgr2=0x7f6acc199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.610+0000 7f6ad414e700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc100e10 0x7f6acc199450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00e9a0 tx=0x7f6abc00ecb0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.610+0000 7f6ad414e700 1 -- 172.21.15.46:0/2098192601 shutdown_connections 2024-03-18T20:36:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.610+0000 7f6ad414e700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6ab80a0f20 0x7f6ab80a33e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.610+0000 7f6ad414e700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6acc1017b0 0x7f6acc1a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.610+0000 7f6ad414e700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6acc0ffc20 0x7f6acc198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.610+0000 7f6ad414e700 1 --2- 172.21.15.46:0/2098192601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6acc100e10 0x7f6acc199450 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.610+0000 7f6ad414e700 1 -- 172.21.15.46:0/2098192601 >> 172.21.15.46:0/2098192601 conn(0x7f6acc0fb190 msgr2=0x7f6acc069020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.610+0000 7f6ad414e700 1 -- 172.21.15.46:0/2098192601 shutdown_connections 2024-03-18T20:36:08.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.610+0000 7f6ad414e700 1 -- 172.21.15.46:0/2098192601 wait complete. 2024-03-18T20:36:08.628 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-18T20:36:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.768+0000 7fde066b2700 1 -- 172.21.15.46:0/414229057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001017d0 msgr2=0x7fde00108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.768+0000 7fde066b2700 1 --2- 172.21.15.46:0/414229057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001017d0 0x7fde00108080 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fddf800b0d0 tx=0x7fddf800b3e0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.768+0000 7fde066b2700 1 -- 172.21.15.46:0/414229057 shutdown_connections 2024-03-18T20:36:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.768+0000 7fde066b2700 1 --2- 172.21.15.46:0/414229057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001017d0 0x7fde00108080 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.768+0000 7fde066b2700 1 --2- 172.21.15.46:0/414229057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde000ffc40 0x7fde00100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.768+0000 7fde066b2700 1 --2- 172.21.15.46:0/414229057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde00100e30 0x7fde00101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.768+0000 7fde066b2700 1 -- 172.21.15.46:0/414229057 >> 172.21.15.46:0/414229057 conn(0x7fde000fb1b0 msgr2=0x7fde000fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.768+0000 7fde066b2700 1 -- 172.21.15.46:0/414229057 shutdown_connections 2024-03-18T20:36:08.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.768+0000 7fde066b2700 1 -- 172.21.15.46:0/414229057 wait complete. 2024-03-18T20:36:08.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.769+0000 7fde066b2700 1 Processor -- start 2024-03-18T20:36:08.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.769+0000 7fde066b2700 1 -- start start 2024-03-18T20:36:08.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.769+0000 7fde066b2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde000ffc40 0x7fde00195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:08.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde066b2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde00100e30 0x7fde00196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:08.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde066b2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001017d0 0x7fde0019d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde066b2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0019d800 con 0x7fde001017d0 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde066b2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0019d970 con 0x7fde00100e30 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde066b2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0019db30 con 0x7fde000ffc40 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde04c4f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001017d0 0x7fde0019d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fddff7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde00100e30 0x7fde00196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde04c4f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001017d0 0x7fde0019d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55082/0 (socket says 172.21.15.46:55082) 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fddff7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde00100e30 0x7fde00196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:41320/0 (socket says 172.21.15.46:41320) 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde04c4f700 1 -- 172.21.15.46:0/55975456 learned_addr learned my addr 172.21.15.46:0/55975456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fddfffff700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde000ffc40 0x7fde00195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde04c4f700 1 -- 172.21.15.46:0/55975456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde000ffc40 msgr2=0x7fde00195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde04c4f700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde000ffc40 0x7fde00195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde04c4f700 1 -- 172.21.15.46:0/55975456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde00100e30 msgr2=0x7fde00196220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde04c4f700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde00100e30 0x7fde00196220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:08.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde04c4f700 1 -- 172.21.15.46:0/55975456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf0009520 con 0x7fde001017d0 2024-03-18T20:36:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fddff7fe700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde00100e30 0x7fde00196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fde04c4f700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001017d0 0x7fde0019d230 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fddf8007b60 tx=0x7fddf8008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.770+0000 7fddfd7fa700 1 -- 172.21.15.46:0/55975456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf800e040 con 0x7fde001017d0 2024-03-18T20:36:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.771+0000 7fddfd7fa700 1 -- 172.21.15.46:0/55975456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddf801d930 con 0x7fde001017d0 2024-03-18T20:36:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.771+0000 7fddfd7fa700 1 -- 172.21.15.46:0/55975456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf801b9f0 con 0x7fde001017d0 2024-03-18T20:36:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.771+0000 7fde066b2700 1 -- 172.21.15.46:0/55975456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf8009d00 con 0x7fde001017d0 2024-03-18T20:36:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.771+0000 7fde066b2700 1 -- 172.21.15.46:0/55975456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0019e050 con 0x7fde001017d0 2024-03-18T20:36:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.772+0000 7fde066b2700 1 -- 172.21.15.46:0/55975456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde00107070 con 0x7fde001017d0 2024-03-18T20:36:08.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.773+0000 7fddfd7fa700 1 -- 172.21.15.46:0/55975456 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 21) v1 ==== 135677+0+0 (secure 0 0 0) 0x7fddf801daa0 con 0x7fde001017d0 2024-03-18T20:36:08.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.774+0000 7fddfd7fa700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdde80a0f20 0x7fdde80a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:08.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.774+0000 7fddfd7fa700 1 -- 172.21.15.46:0/55975456 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fddf80ce3d0 con 0x7fde001017d0 2024-03-18T20:36:08.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.774+0000 7fddfffff700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdde80a0f20 0x7fdde80a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:08.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.775+0000 7fddfffff700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdde80a0f20 0x7fdde80a33e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fde000fcd80 tx=0x7fddf0009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:08.778+0000 7fddfd7fa700 1 -- 172.21.15.46:0/55975456 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fddf8097db0 con 0x7fde001017d0 2024-03-18T20:36:09.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.028+0000 7fde066b2700 1 -- 172.21.15.46:0/55975456 --> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fde000611f0 con 0x7fdde80a0f20 2024-03-18T20:36:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.036+0000 7fddfd7fa700 1 -- 172.21.15.46:0/55975456 <== mgr.15498 v2:172.21.15.46:6800/805586292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7fde000611f0 con 0x7fdde80a0f20 2024-03-18T20:36:09.039 INFO:teuthology.orchestra.run.smithi046.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-18T00:00:00", "created": "2024-03-18T20:36:08", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-18T20:36:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.039+0000 7fde066b2700 1 -- 172.21.15.46:0/55975456 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdde80a0f20 msgr2=0x7fdde80a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.039+0000 7fde066b2700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdde80a0f20 0x7fdde80a33e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fde000fcd80 tx=0x7fddf0009260 comp rx=0 tx=0).stop 2024-03-18T20:36:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.039+0000 7fde066b2700 1 -- 172.21.15.46:0/55975456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001017d0 msgr2=0x7fde0019d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.039+0000 7fde066b2700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001017d0 0x7fde0019d230 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fddf8007b60 tx=0x7fddf8008e40 comp rx=0 tx=0).stop 2024-03-18T20:36:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.040+0000 7fde066b2700 1 -- 172.21.15.46:0/55975456 shutdown_connections 2024-03-18T20:36:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.040+0000 7fde066b2700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdde80a0f20 0x7fdde80a33e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.040+0000 7fde066b2700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001017d0 0x7fde0019d230 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.040+0000 7fde066b2700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde000ffc40 0x7fde00195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.040+0000 7fde066b2700 1 --2- 172.21.15.46:0/55975456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde00100e30 0x7fde00196220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.040+0000 7fde066b2700 1 -- 172.21.15.46:0/55975456 >> 172.21.15.46:0/55975456 conn(0x7fde000fb1b0 msgr2=0x7fde001041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.040+0000 7fde066b2700 1 -- 172.21.15.46:0/55975456 shutdown_connections 2024-03-18T20:36:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.040+0000 7fde066b2700 1 -- 172.21.15.46:0/55975456 wait complete. 2024-03-18T20:36:09.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-18T20:36:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.213+0000 7f4e30191700 1 -- 172.21.15.46:0/4022722292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e28101230 msgr2=0x7f4e28101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.213+0000 7f4e30191700 1 --2- 172.21.15.46:0/4022722292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e28101230 0x7f4e28101650 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009990 tx=0x7f4e24009ca0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.213+0000 7f4e30191700 1 -- 172.21.15.46:0/4022722292 shutdown_connections 2024-03-18T20:36:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.213+0000 7f4e30191700 1 --2- 172.21.15.46:0/4022722292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e28101230 0x7f4e28101650 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.213+0000 7f4e30191700 1 --2- 172.21.15.46:0/4022722292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e28100840 0x7f4e281080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.213+0000 7f4e30191700 1 --2- 172.21.15.46:0/4022722292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e280ffe80 0x7f4e28100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.213+0000 7f4e30191700 1 -- 172.21.15.46:0/4022722292 >> 172.21.15.46:0/4022722292 conn(0x7f4e280fa930 msgr2=0x7f4e280fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.213+0000 7f4e30191700 1 -- 172.21.15.46:0/4022722292 shutdown_connections 2024-03-18T20:36:09.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.213+0000 7f4e30191700 1 -- 172.21.15.46:0/4022722292 wait complete. 2024-03-18T20:36:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.214+0000 7f4e30191700 1 Processor -- start 2024-03-18T20:36:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.214+0000 7f4e30191700 1 -- start start 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.214+0000 7f4e30191700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e280ffe80 0x7f4e28196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.214+0000 7f4e30191700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e28100840 0x7f4e28197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.214+0000 7f4e30191700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e28101230 0x7f4e2819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.214+0000 7f4e30191700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819e7f0 con 0x7f4e280ffe80 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.214+0000 7f4e30191700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819e960 con 0x7f4e28100840 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.214+0000 7f4e30191700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819eb20 con 0x7f4e28101230 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.214+0000 7f4e2df2d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e280ffe80 0x7f4e28196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.214+0000 7f4e2df2d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e280ffe80 0x7f4e28196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55110/0 (socket says 172.21.15.46:55110) 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.214+0000 7f4e2df2d700 1 -- 172.21.15.46:0/2750897717 learned_addr learned my addr 172.21.15.46:0/2750897717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e2e72e700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e28101230 0x7f4e2819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e2d72c700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e28100840 0x7f4e28197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e2df2d700 1 -- 172.21.15.46:0/2750897717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e28101230 msgr2=0x7f4e2819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:09.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e2df2d700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e28101230 0x7f4e2819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e2df2d700 1 -- 172.21.15.46:0/2750897717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e28100840 msgr2=0x7f4e28197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e2df2d700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e28100840 0x7f4e28197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e2df2d700 1 -- 172.21.15.46:0/2750897717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e240095f0 con 0x7f4e280ffe80 2024-03-18T20:36:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e2d72c700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e28100840 0x7f4e28197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e2df2d700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e280ffe80 0x7f4e28196cd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4e24000c00 tx=0x7f4e24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e1effd700 1 -- 172.21.15.46:0/2750897717 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2401d070 con 0x7f4e280ffe80 2024-03-18T20:36:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e30191700 1 -- 172.21.15.46:0/2750897717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2819eda0 con 0x7f4e280ffe80 2024-03-18T20:36:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e1effd700 1 -- 172.21.15.46:0/2750897717 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e24021410 con 0x7f4e280ffe80 2024-03-18T20:36:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e1effd700 1 -- 172.21.15.46:0/2750897717 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e24017650 con 0x7f4e280ffe80 2024-03-18T20:36:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.215+0000 7f4e30191700 1 -- 172.21.15.46:0/2750897717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2819f1d0 con 0x7f4e280ffe80 2024-03-18T20:36:09.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.216+0000 7f4e30191700 1 -- 172.21.15.46:0/2750897717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e28068080 con 0x7f4e280ffe80 2024-03-18T20:36:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.218+0000 7f4e1effd700 1 -- 172.21.15.46:0/2750897717 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 21) v1 ==== 135677+0+0 (secure 0 0 0) 0x7f4e24021580 con 0x7f4e280ffe80 2024-03-18T20:36:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.218+0000 7f4e1effd700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e140a0f20 0x7f4e140a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.219+0000 7f4e1effd700 1 -- 172.21.15.46:0/2750897717 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f4e240ce100 con 0x7f4e280ffe80 2024-03-18T20:36:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.219+0000 7f4e2d72c700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e140a0f20 0x7f4e140a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:09.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.219+0000 7f4e2d72c700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e140a0f20 0x7f4e140a33e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e28198180 tx=0x7f4e18009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:09.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.222+0000 7f4e1effd700 1 -- 172.21.15.46:0/2750897717 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4e24097b00 con 0x7f4e280ffe80 2024-03-18T20:36:09.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: [18/Mar/2024:20:36:07] ENGINE Bus STARTING 2024-03-18T20:36:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: [18/Mar/2024:20:36:07] ENGINE Serving on https://172.21.15.46:7150 2024-03-18T20:36:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: [18/Mar/2024:20:36:07] ENGINE Serving on http://172.21.15.46:8765 2024-03-18T20:36:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: [18/Mar/2024:20:36:07] ENGINE Bus STARTED 2024-03-18T20:36:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: mgrmap e21: x(active, since 1.02044s), standbys: z, y 2024-03-18T20:36:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:36:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:36:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:08 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: [18/Mar/2024:20:36:07] ENGINE Bus STARTING 2024-03-18T20:36:09.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: [18/Mar/2024:20:36:07] ENGINE Serving on https://172.21.15.46:7150 2024-03-18T20:36:09.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: [18/Mar/2024:20:36:07] ENGINE Serving on http://172.21.15.46:8765 2024-03-18T20:36:09.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: [18/Mar/2024:20:36:07] ENGINE Bus STARTED 2024-03-18T20:36:09.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: mgrmap e21: x(active, since 1.02044s), standbys: z, y 2024-03-18T20:36:09.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:36:09.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:36:09.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:08 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: [18/Mar/2024:20:36:07] ENGINE Bus STARTING 2024-03-18T20:36:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: [18/Mar/2024:20:36:07] ENGINE Serving on https://172.21.15.46:7150 2024-03-18T20:36:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: [18/Mar/2024:20:36:07] ENGINE Serving on http://172.21.15.46:8765 2024-03-18T20:36:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: [18/Mar/2024:20:36:07] ENGINE Bus STARTED 2024-03-18T20:36:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: mgrmap e21: x(active, since 1.02044s), standbys: z, y 2024-03-18T20:36:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:36:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:36:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:08 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:09.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.495+0000 7f4e30191700 1 -- 172.21.15.46:0/2750897717 --> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4e280ff4f0 con 0x7f4e140a0f20 2024-03-18T20:36:09.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.503+0000 7f4e1effd700 1 -- 172.21.15.46:0/2750897717 <== mgr.15498 v2:172.21.15.46:6800/805586292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f4e280ff4f0 con 0x7f4e140a0f20 2024-03-18T20:36:09.507 INFO:teuthology.orchestra.run.smithi046.stdout:/ 1m 6m3h 2024-03-18T20:36:09.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.508+0000 7f4e30191700 1 -- 172.21.15.46:0/2750897717 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e140a0f20 msgr2=0x7f4e140a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:09.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.508+0000 7f4e30191700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e140a0f20 0x7f4e140a33e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e28198180 tx=0x7f4e18009260 comp rx=0 tx=0).stop 2024-03-18T20:36:09.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.508+0000 7f4e30191700 1 -- 172.21.15.46:0/2750897717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e280ffe80 msgr2=0x7f4e28196cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:09.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.508+0000 7f4e30191700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e280ffe80 0x7f4e28196cd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4e24000c00 tx=0x7f4e24004730 comp rx=0 tx=0).stop 2024-03-18T20:36:09.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.508+0000 7f4e30191700 1 -- 172.21.15.46:0/2750897717 shutdown_connections 2024-03-18T20:36:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.508+0000 7f4e30191700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e140a0f20 0x7f4e140a33e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.508+0000 7f4e30191700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e280ffe80 0x7f4e28196cd0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.508+0000 7f4e30191700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e28101230 0x7f4e2819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.508+0000 7f4e30191700 1 --2- 172.21.15.46:0/2750897717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e28100840 0x7f4e28197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.508+0000 7f4e30191700 1 -- 172.21.15.46:0/2750897717 >> 172.21.15.46:0/2750897717 conn(0x7f4e280fa930 msgr2=0x7f4e280fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.508+0000 7f4e30191700 1 -- 172.21.15.46:0/2750897717 shutdown_connections 2024-03-18T20:36:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:09.508+0000 7f4e30191700 1 -- 172.21.15.46:0/2750897717 wait complete. 2024-03-18T20:36:09.526 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-18T20:36:09.561 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-18T20:36:09.715 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-18T20:36:09.745 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-03-18T20:36:09.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-18T20:36:09.747 INFO:teuthology.run_tasks:Running task workunit... 2024-03-18T20:36:09.813 INFO:tasks.workunit:Pulling workunits from ref 49bdc51142759622a281d3e5327eebd07cbb8bb3 2024-03-18T20:36:09.813 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-18T20:36:09.814 INFO:tasks.workunit:timeout=3h 2024-03-18T20:36:09.814 INFO:tasks.workunit:cleanup=True 2024-03-18T20:36:09.814 DEBUG:teuthology.orchestra.run.smithi046:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-18T20:36:09.835 INFO:teuthology.orchestra.run.smithi046.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-18T20:36:09.835 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-18T20:36:09.835 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2024-03-18T20:36:09.835 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T20:36:09.835 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:cephfs_t:s0 2024-03-18T20:36:09.835 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-03-18 20:34:45.572328435 +0000 2024-03-18T20:36:09.835 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-03-18 20:34:45.572328435 +0000 2024-03-18T20:36:09.835 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-03-18 20:35:50.273236551 +0000 2024-03-18T20:36:09.836 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-03-18 20:34:45.572328435 +0000 2024-03-18T20:36:09.836 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-18T20:36:09.836 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-18T20:36:09.913 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 49bdc51142759622a281d3e5327eebd07cbb8bb3 2024-03-18T20:36:09.973 INFO:tasks.workunit.client.0.smithi046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-18T20:36:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:36:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:36:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='client.15561 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:36:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi073 to 182.4M 2024-03-18T20:36:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: Unable to set osd_memory_target on smithi073 to 191350476: error parsing value: Value '191350476' is below minimum 939524096 2024-03-18T20:36:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:36:10.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:36:10.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:10 smithi155 ceph-mon[48344]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-03-18T20:36:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='client.15561 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi073 to 182.4M 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: Unable to set osd_memory_target on smithi073 to 191350476: error parsing value: Value '191350476' is below minimum 939524096 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:36:10.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:36:10.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:10 smithi046 ceph-mon[43495]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-03-18T20:36:10.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:36:10.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:36:10.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='client.15561 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:36:10.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi073 to 182.4M 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: Unable to set osd_memory_target on smithi073 to 191350476: error parsing value: Value '191350476' is below minimum 939524096 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:10.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:36:10.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:36:10.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:36:10.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:10 smithi073 ceph-mon[47384]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-03-18T20:36:11.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi046 to 183.5M 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Unable to set osd_memory_target on smithi046 to 192483737: error parsing value: Value '192483737' is below minimum 939524096 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Updating smithi046:/etc/ceph/ceph.conf 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Updating smithi073:/etc/ceph/ceph.conf 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: from='client.15573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Updating smithi073:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Updating smithi046:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Updating smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:36:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:36:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: Updating smithi073:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:36:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:36:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:36:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:36:11.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:11 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-03-18T20:36:11.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi046 to 183.5M 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Unable to set osd_memory_target on smithi046 to 192483737: error parsing value: Value '192483737' is below minimum 939524096 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Updating smithi046:/etc/ceph/ceph.conf 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Updating smithi073:/etc/ceph/ceph.conf 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: from='client.15573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Updating smithi073:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Updating smithi046:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Updating smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: Updating smithi073:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:36:11.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:36:11.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:36:11.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:11 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-03-18T20:36:11.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-03-18T20:36:11.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi046 to 183.5M 2024-03-18T20:36:11.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Unable to set osd_memory_target on smithi046 to 192483737: error parsing value: Value '192483737' is below minimum 939524096 2024-03-18T20:36:11.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Updating smithi046:/etc/ceph/ceph.conf 2024-03-18T20:36:11.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Updating smithi073:/etc/ceph/ceph.conf 2024-03-18T20:36:11.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-18T20:36:11.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: from='client.15573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T20:36:11.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Updating smithi073:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:36:11.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Updating smithi046:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:36:11.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Updating smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T20:36:11.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:36:11.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:36:11.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-18T20:36:11.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: Updating smithi073:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:36:11.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.724 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.724 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:11.724 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:36:11.724 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:36:11.724 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:36:11.724 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:11 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:36:12.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:12 smithi155 ceph-mon[48344]: Updating smithi046:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:36:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:12 smithi155 ceph-mon[48344]: Updating smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:36:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:12 smithi155 ceph-mon[48344]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:36:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:12 smithi155 ceph-mon[48344]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-18T20:36:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:12 smithi046 ceph-mon[43495]: Updating smithi046:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:36:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:12 smithi046 ceph-mon[43495]: Updating smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:36:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:12 smithi046 ceph-mon[43495]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:36:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:12 smithi046 ceph-mon[43495]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-18T20:36:12.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:12 smithi073 ceph-mon[47384]: Updating smithi046:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:36:12.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:12 smithi073 ceph-mon[47384]: Updating smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T20:36:12.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:12 smithi073 ceph-mon[47384]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T20:36:12.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:12 smithi073 ceph-mon[47384]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-18T20:36:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:14 smithi155 ceph-mon[48344]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 184 KiB/s wr, 111 op/s 2024-03-18T20:36:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:14 smithi046 ceph-mon[43495]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 184 KiB/s wr, 111 op/s 2024-03-18T20:36:14.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:14 smithi073 ceph-mon[47384]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 184 KiB/s wr, 111 op/s 2024-03-18T20:36:16.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:15 smithi073 ceph-mon[47384]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-03-18T20:36:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:15 smithi155 ceph-mon[48344]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-03-18T20:36:16.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:15 smithi046 ceph-mon[43495]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 79 op/s 2024-03-18T20:36:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:18 smithi073 ceph-mon[47384]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-03-18T20:36:18.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:18 smithi155 ceph-mon[48344]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-03-18T20:36:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:18 smithi046 ceph-mon[43495]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-03-18T20:36:20.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:20 smithi073 ceph-mon[47384]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-18T20:36:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:20 smithi155 ceph-mon[48344]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-18T20:36:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:20 smithi046 ceph-mon[43495]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-18T20:36:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:22 smithi073 ceph-mon[47384]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-18T20:36:22.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:22 smithi155 ceph-mon[48344]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-18T20:36:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:22 smithi046 ceph-mon[43495]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-18T20:36:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.945+0000 7fd73feb6700 1 -- 172.21.15.46:0/2512891891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd738100570 msgr2=0x7fd73810ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.945+0000 7fd73feb6700 1 --2- 172.21.15.46:0/2512891891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd738100570 0x7fd73810ca00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd73000b0a0 tx=0x7fd73000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.945+0000 7fd73feb6700 1 -- 172.21.15.46:0/2512891891 shutdown_connections 2024-03-18T20:36:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.945+0000 7fd73feb6700 1 --2- 172.21.15.46:0/2512891891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd738100570 0x7fd73810ca00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.945+0000 7fd73feb6700 1 --2- 172.21.15.46:0/2512891891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd738105c60 0x7fd738106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.945+0000 7fd73feb6700 1 --2- 172.21.15.46:0/2512891891 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7380ffbb0 0x7fd738100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.945+0000 7fd73feb6700 1 -- 172.21.15.46:0/2512891891 >> 172.21.15.46:0/2512891891 conn(0x7fd738076d50 msgr2=0x7fd738077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.945+0000 7fd73feb6700 1 -- 172.21.15.46:0/2512891891 shutdown_connections 2024-03-18T20:36:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.945+0000 7fd73feb6700 1 -- 172.21.15.46:0/2512891891 wait complete. 2024-03-18T20:36:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.946+0000 7fd73feb6700 1 Processor -- start 2024-03-18T20:36:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.946+0000 7fd73feb6700 1 -- start start 2024-03-18T20:36:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.946+0000 7fd73feb6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7380ffbb0 0x7fd738198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.946+0000 7fd73feb6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd738100570 0x7fd7381985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.946+0000 7fd73feb6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd738105c60 0x7fd7381a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.946+0000 7fd73feb6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd738198e00 con 0x7fd7380ffbb0 2024-03-18T20:36:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.946+0000 7fd73feb6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7381a3630 con 0x7fd738100570 2024-03-18T20:36:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.946+0000 7fd73feb6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7381a3770 con 0x7fd738105c60 2024-03-18T20:36:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73dc52700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7380ffbb0 0x7fd738198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73dc52700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7380ffbb0 0x7fd738198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55114/0 (socket says 172.21.15.46:55114) 2024-03-18T20:36:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73dc52700 1 -- 172.21.15.46:0/3497569633 learned_addr learned my addr 172.21.15.46:0/3497569633 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73d451700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd738100570 0x7fd7381985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73e453700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd738105c60 0x7fd7381a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73dc52700 1 -- 172.21.15.46:0/3497569633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd738105c60 msgr2=0x7fd7381a30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73dc52700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd738105c60 0x7fd7381a30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73dc52700 1 -- 172.21.15.46:0/3497569633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd738100570 msgr2=0x7fd7381985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73dc52700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd738100570 0x7fd7381985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73dc52700 1 -- 172.21.15.46:0/3497569633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd730009d00 con 0x7fd7380ffbb0 2024-03-18T20:36:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73e453700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd738105c60 0x7fd7381a30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:36:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73dc52700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7380ffbb0 0x7fd738198070 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd73400e9d0 tx=0x7fd73400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd72effd700 1 -- 172.21.15.46:0/3497569633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734009790 con 0x7fd7380ffbb0 2024-03-18T20:36:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd72effd700 1 -- 172.21.15.46:0/3497569633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd73400cbd0 con 0x7fd7380ffbb0 2024-03-18T20:36:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73feb6700 1 -- 172.21.15.46:0/3497569633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7381a39d0 con 0x7fd7380ffbb0 2024-03-18T20:36:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd72effd700 1 -- 172.21.15.46:0/3497569633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734018680 con 0x7fd7380ffbb0 2024-03-18T20:36:22.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.947+0000 7fd73feb6700 1 -- 172.21.15.46:0/3497569633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7381a3f30 con 0x7fd7380ffbb0 2024-03-18T20:36:22.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.951+0000 7fd72effd700 1 -- 172.21.15.46:0/3497569633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 136041+0+0 (secure 0 0 0) 0x7fd734010430 con 0x7fd7380ffbb0 2024-03-18T20:36:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.951+0000 7fd74c04bb80 1 Processor -- start 2024-03-18T20:36:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd74c04bb80 1 -- start start 2024-03-18T20:36:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd74c04bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 0x559ebbbfd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd74c04bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbbfde90 0x559ebbc08870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd74c04bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbc08db0 0x559ebbc0b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd74c04bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebbb57930 con 0x559ebbbfde90 2024-03-18T20:36:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd74c04bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebbb552f0 con 0x559ebbbfd570 2024-03-18T20:36:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd74c04bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebbb50be0 con 0x559ebbc08db0 2024-03-18T20:36:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd72effd700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd7240a13a0 0x7fd7240a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd73e453700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 0x559ebbbfd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd73e453700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 0x559ebbbfd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:58422/0 (socket says 172.21.15.46:58422) 2024-03-18T20:36:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd73e453700 1 -- 172.21.15.46:0/3583648045 learned_addr learned my addr 172.21.15.46:0/3583648045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd72effd700 1 -- 172.21.15.46:0/3497569633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd734014070 con 0x7fd7380ffbb0 2024-03-18T20:36:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd73d451700 1 --2- 172.21.15.46:0/3583648045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbc08db0 0x559ebbc0b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.952+0000 7fd73e453700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd7240a13a0 0x7fd7240a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.953+0000 7fd73e453700 1 -- 172.21.15.46:0/3583648045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbc08db0 msgr2=0x559ebbc0b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.953+0000 7fd73e453700 1 --2- 172.21.15.46:0/3583648045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbc08db0 0x559ebbc0b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.953+0000 7fd73e453700 1 -- 172.21.15.46:0/3583648045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbbfde90 msgr2=0x559ebbc08870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.953+0000 7fd73e453700 1 --2- 172.21.15.46:0/3583648045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbbfde90 0x559ebbc08870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.953+0000 7fd73e453700 1 -- 172.21.15.46:0/3583648045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ebb9d2350 con 0x559ebbbfd570 2024-03-18T20:36:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.953+0000 7fd73e453700 1 --2- 172.21.15.46:0/3583648045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 0x559ebbbfd950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd73000b070 tx=0x7fd730007e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f856c5d0b052f249 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:22.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.953+0000 7fd73e453700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd7240a13a0 0x7fd7240a3860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd730023000 tx=0x7fd730026000 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.953+0000 7fd723fff700 1 -- 172.21.15.46:0/3583648045 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73000e070 con 0x559ebbbfd570 2024-03-18T20:36:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.953+0000 7fd723fff700 1 -- 172.21.15.46:0/3583648045 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd730004c50 con 0x559ebbbfd570 2024-03-18T20:36:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.953+0000 7fd723fff700 1 -- 172.21.15.46:0/3583648045 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73001f960 con 0x559ebbbfd570 2024-03-18T20:36:22.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.954+0000 7fd74c04bb80 1 -- 172.21.15.46:0/3583648045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 msgr2=0x559ebbbfd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:22.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.954+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/3583648045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 0x559ebbbfd950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd73000b070 tx=0x7fd730007e30 comp rx=0 tx=0).stop 2024-03-18T20:36:22.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.954+0000 7fd74c04bb80 1 -- 172.21.15.46:0/3583648045 shutdown_connections 2024-03-18T20:36:22.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.954+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/3583648045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbc08db0 0x559ebbc0b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.954+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/3583648045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 0x559ebbbfd950 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.954+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/3583648045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbbfde90 0x559ebbc08870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.954+0000 7fd74c04bb80 1 -- 172.21.15.46:0/3583648045 >> 172.21.15.46:0/3583648045 conn(0x559ebbb741c0 msgr2=0x559ebbbf9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.955+0000 7fd74c04bb80 1 -- 172.21.15.46:0/3583648045 shutdown_connections 2024-03-18T20:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.955+0000 7fd74c04bb80 1 -- 172.21.15.46:0/3583648045 wait complete. 2024-03-18T20:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.955+0000 7fd74c04bb80 1 -- start start 2024-03-18T20:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.955+0000 7fd74c04bb80 1 Processor -- start 2024-03-18T20:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd74c04bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfd570 0x559ebbc087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd74c04bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfde90 0x559ebbc05b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd74c04bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 0x559ebbc06260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd74c04bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebbaecd20 con 0x559ebbc08db0 2024-03-18T20:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd74c04bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebbaec9c0 con 0x559ebbbfde90 2024-03-18T20:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd74c04bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebb9d4640 con 0x559ebbbfd570 2024-03-18T20:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd74c04bb80 10 client.0 _ll_register_callbacks cb 0x559ebb958340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd73d451700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfde90 0x559ebbc05b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd73e453700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 0x559ebbc06260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd73d451700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfde90 0x559ebbc05b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:58434/0 (socket says 172.21.15.46:58434) 2024-03-18T20:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd73d451700 1 -- 172.21.15.46:0/2404693597 learned_addr learned my addr 172.21.15.46:0/2404693597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd73dc52700 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfd570 0x559ebbc087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd73e453700 1 -- 172.21.15.46:0/2404693597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfd570 msgr2=0x559ebbc087e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd73e453700 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfd570 0x559ebbc087e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd73e453700 1 -- 172.21.15.46:0/2404693597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfde90 msgr2=0x559ebbc05b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd73e453700 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfde90 0x559ebbc05b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd73e453700 1 -- 172.21.15.46:0/2404693597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73810d2f0 con 0x559ebbc08db0 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.956+0000 7fd73d451700 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfde90 0x559ebbc05b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.957+0000 7fd73e453700 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 0x559ebbc06260 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd730009fd0 tx=0x7fd730007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38400dc509b8882 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.957+0000 7fd7217fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.957+0000 7fd7217fa700 1 -- 172.21.15.46:0/2404693597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73003a070 con 0x559ebbc08db0 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.957+0000 7fd7217fa700 1 -- 172.21.15.46:0/2404693597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd730025070 con 0x559ebbc08db0 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.957+0000 7fd74c04bb80 1 -- 172.21.15.46:0/2404693597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ebbbf16c0 con 0x559ebbc08db0 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.957+0000 7fd7217fa700 1 -- 172.21.15.46:0/2404693597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd73000e050 con 0x559ebbc08db0 2024-03-18T20:36:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.957+0000 7fd74c04bb80 10 client.15582 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.957+0000 7fd74c04bb80 1 -- 172.21.15.46:0/2404693597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559ebb97eb00 con 0x559ebbc08db0 2024-03-18T20:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.957+0000 7fd7217fa700 1 -- 172.21.15.46:0/2404693597 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ebbbf16c0 con 0x559ebbc08db0 2024-03-18T20:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.958+0000 7fd7217fa700 1 -- 172.21.15.46:0/2404693597 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd73003d020 con 0x559ebbc08db0 2024-03-18T20:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.958+0000 7fd74c04bb80 10 client.15582 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.958+0000 7fd74c04bb80 10 client.15582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.958+0000 7fd74c04bb80 20 client.15582 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.958+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559ebbc18870 0x559ebbc1ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.958+0000 7fd74c04bb80 4 client.15582 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.958+0000 7fd74c04bb80 1 -- 172.21.15.46:0/2404693597 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ebbb496a0 con 0x559ebbc18870 2024-03-18T20:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.958+0000 7fd73dc52700 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559ebbc18870 0x559ebbc1ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.958+0000 7fd73dc52700 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559ebbc18870 0x559ebbc1ac90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf1e0aa23e235dc4 server_cookie=fb0d7e21f975af7b in_seq=0 out_seq=0 2024-03-18T20:36:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.958+0000 7fd7217fa700 10 client.15582 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd7217fa700 1 -- 172.21.15.46:0/2404693597 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559ebbb496a0 con 0x559ebbc18870 2024-03-18T20:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd7217fa700 10 client.15582 handle_command_reply: tid=0 2024-03-18T20:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd74c04bb80 1 client.15582 shutdown 2024-03-18T20:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd74c04bb80 1 -- 172.21.15.46:0/2404693597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 msgr2=0x559ebbc06260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 0x559ebbc06260 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd730009fd0 tx=0x7fd730007b30 comp rx=0 tx=0).stop 2024-03-18T20:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd74c04bb80 1 -- 172.21.15.46:0/2404693597 shutdown_connections 2024-03-18T20:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559ebbc18870 0x559ebbc1ac90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 0x559ebbc06260 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfd570 0x559ebbc087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/2404693597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfde90 0x559ebbc05b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd7217fa700 0 client.15582 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd74c04bb80 1 -- 172.21.15.46:0/2404693597 >> 172.21.15.46:0/2404693597 conn(0x559ebbb741c0 msgr2=0x559ebbbf9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd74c04bb80 1 -- 172.21.15.46:0/2404693597 shutdown_connections 2024-03-18T20:36:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.959+0000 7fd74c04bb80 1 -- 172.21.15.46:0/2404693597 wait complete. 2024-03-18T20:36:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.960+0000 7fd74c04bb80 20 client.15582 trim_cache size 0 max 16384 2024-03-18T20:36:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.975+0000 7fd74c04bb80 1 Processor -- start 2024-03-18T20:36:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.975+0000 7fd74c04bb80 1 -- start start 2024-03-18T20:36:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.975+0000 7fd74c04bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfd570 0x559ebbc1ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.975+0000 7fd74c04bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbbfde90 0x559ebbc004e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.975+0000 7fd74c04bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbc08db0 0x559ebbc0f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.975+0000 7fd74c04bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebbbfdd10 con 0x559ebbbfde90 2024-03-18T20:36:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.975+0000 7fd74c04bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebbbff0f0 con 0x559ebbc08db0 2024-03-18T20:36:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.975+0000 7fd74c04bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebbbf61d0 con 0x559ebbbfd570 2024-03-18T20:36:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd73d451700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbbfde90 0x559ebbc004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd73d451700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbbfde90 0x559ebbc004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55128/0 (socket says 172.21.15.46:55128) 2024-03-18T20:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd73d451700 1 -- 172.21.15.46:0/3513245812 learned_addr learned my addr 172.21.15.46:0/3513245812 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd73dc52700 1 --2- 172.21.15.46:0/3513245812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfd570 0x559ebbc1ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd73e453700 1 --2- 172.21.15.46:0/3513245812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbc08db0 0x559ebbc0f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd73d451700 1 -- 172.21.15.46:0/3513245812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfd570 msgr2=0x559ebbc1ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd73d451700 1 --2- 172.21.15.46:0/3513245812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfd570 0x559ebbc1ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd73d451700 1 -- 172.21.15.46:0/3513245812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbc08db0 msgr2=0x559ebbc0f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd73d451700 1 --2- 172.21.15.46:0/3513245812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbc08db0 0x559ebbc0f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd73d451700 1 -- 172.21.15.46:0/3513245812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ebbb16e40 con 0x559ebbbfde90 2024-03-18T20:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd73d451700 1 --2- 172.21.15.46:0/3513245812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbbfde90 0x559ebbc004e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd728004670 tx=0x7fd728013a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c16df95132ea23 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd72cff9700 1 -- 172.21.15.46:0/3513245812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728022070 con 0x559ebbbfde90 2024-03-18T20:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd72cff9700 1 -- 172.21.15.46:0/3513245812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd728019040 con 0x559ebbbfde90 2024-03-18T20:36:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.976+0000 7fd72cff9700 1 -- 172.21.15.46:0/3513245812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728018050 con 0x559ebbbfde90 2024-03-18T20:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.977+0000 7fd74c04bb80 1 -- 172.21.15.46:0/3513245812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbbfde90 msgr2=0x559ebbc004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.977+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/3513245812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbbfde90 0x559ebbc004e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd728004670 tx=0x7fd728013a90 comp rx=0 tx=0).stop 2024-03-18T20:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.977+0000 7fd74c04bb80 1 -- 172.21.15.46:0/3513245812 shutdown_connections 2024-03-18T20:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.977+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/3513245812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbc08db0 0x559ebbc0f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.977+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/3513245812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfd570 0x559ebbc1ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.977+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/3513245812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbbfde90 0x559ebbc004e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.977+0000 7fd74c04bb80 1 -- 172.21.15.46:0/3513245812 >> 172.21.15.46:0/3513245812 conn(0x559ebbb741c0 msgr2=0x559ebbbf8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.977+0000 7fd74c04bb80 1 -- 172.21.15.46:0/3513245812 shutdown_connections 2024-03-18T20:36:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.977+0000 7fd74c04bb80 1 -- 172.21.15.46:0/3513245812 wait complete. 2024-03-18T20:36:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.977+0000 7fd74c04bb80 1 -- start start 2024-03-18T20:36:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd74c04bb80 1 Processor -- start 2024-03-18T20:36:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd74c04bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 0x559ebbc0d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd74c04bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfde90 0x559ebbc0d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd74c04bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 0x559ebbc0de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd74c04bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebbc6b790 con 0x559ebbc08db0 2024-03-18T20:36:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd74c04bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebbc6b900 con 0x559ebbbfd570 2024-03-18T20:36:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd74c04bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebbc6bac0 con 0x559ebbbfde90 2024-03-18T20:36:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd74c04bb80 10 client.0 _ll_register_callbacks cb 0x7fd72800d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd73e453700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 0x559ebbc0de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd73dc52700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 0x559ebbc0d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd73e453700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 0x559ebbc0de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55132/0 (socket says 172.21.15.46:55132) 2024-03-18T20:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd73e453700 1 -- 172.21.15.46:0/891103909 learned_addr learned my addr 172.21.15.46:0/891103909 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd73d451700 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfde90 0x559ebbc0d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd73dc52700 1 -- 172.21.15.46:0/891103909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfde90 msgr2=0x559ebbc0d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd73dc52700 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfde90 0x559ebbc0d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.978+0000 7fd73dc52700 1 -- 172.21.15.46:0/891103909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 msgr2=0x559ebbc0de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd73dc52700 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 0x559ebbc0de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd73dc52700 1 -- 172.21.15.46:0/891103909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd728004320 con 0x559ebbbfd570 2024-03-18T20:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd73e453700 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 0x559ebbc0de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd73d451700 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfde90 0x559ebbc0d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd7217fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:36:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd73dc52700 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 0x559ebbc0d400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd7340ca760 tx=0x7fd73400cfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f40bf792ae83e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd7217fa700 1 -- 172.21.15.46:0/891103909 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73409a070 con 0x559ebbbfd570 2024-03-18T20:36:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd7217fa700 1 -- 172.21.15.46:0/891103909 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd73409dea0 con 0x559ebbbfd570 2024-03-18T20:36:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd74c04bb80 1 -- 172.21.15.46:0/891103909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd730004c50 con 0x559ebbbfd570 2024-03-18T20:36:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd7217fa700 1 -- 172.21.15.46:0/891103909 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7340a24f0 con 0x559ebbbfd570 2024-03-18T20:36:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd74c04bb80 10 client.25358 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd74c04bb80 1 -- 172.21.15.46:0/891103909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559ebb9d6c70 con 0x559ebbbfd570 2024-03-18T20:36:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.979+0000 7fd7217fa700 1 -- 172.21.15.46:0/891103909 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd730004c50 con 0x559ebbbfd570 2024-03-18T20:36:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.980+0000 7fd7217fa700 1 -- 172.21.15.46:0/891103909 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd7340b8020 con 0x559ebbbfd570 2024-03-18T20:36:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.980+0000 7fd74c04bb80 10 client.25358 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.980+0000 7fd74c04bb80 10 client.25358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.980+0000 7fd74c04bb80 20 client.25358 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.980+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559ebbc6d260 0x559ebbc6f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.980+0000 7fd74c04bb80 4 client.25358 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:36:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.980+0000 7fd74c04bb80 1 -- 172.21.15.46:0/891103909 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559ebb99fc90 con 0x559ebbc6d260 2024-03-18T20:36:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.980+0000 7fd73d451700 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559ebbc6d260 0x559ebbc6f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.980+0000 7fd7217fa700 10 client.25358 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.980+0000 7fd73d451700 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559ebbc6d260 0x559ebbc6f650 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4073459e1df34261 server_cookie=cb587aa50e5943c4 in_seq=0 out_seq=0 2024-03-18T20:36:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.991+0000 7fd7217fa700 1 -- 172.21.15.46:0/891103909 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559ebb99fc90 con 0x559ebbc6d260 2024-03-18T20:36:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.991+0000 7fd7217fa700 10 client.25358 handle_command_reply: tid=0 2024-03-18T20:36:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.991+0000 7fd74c04bb80 1 client.25358 shutdown 2024-03-18T20:36:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.992+0000 7fd74c04bb80 1 -- 172.21.15.46:0/891103909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 msgr2=0x559ebbc0d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.992+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 0x559ebbc0d400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd7340ca760 tx=0x7fd73400cfb0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.992+0000 7fd74c04bb80 1 -- 172.21.15.46:0/891103909 shutdown_connections 2024-03-18T20:36:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.992+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559ebbc6d260 0x559ebbc6f650 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.992+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ebbc08db0 0x559ebbc0de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.992+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ebbbfd570 0x559ebbc0d400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.992+0000 7fd7217fa700 0 client.25358 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.992+0000 7fd74c04bb80 1 --2- 172.21.15.46:0/891103909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559ebbbfde90 0x559ebbc0d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.992+0000 7fd74c04bb80 1 -- 172.21.15.46:0/891103909 >> 172.21.15.46:0/891103909 conn(0x559ebbb741c0 msgr2=0x559ebbc04300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.992+0000 7fd74c04bb80 1 -- 172.21.15.46:0/891103909 shutdown_connections 2024-03-18T20:36:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.992+0000 7fd74c04bb80 1 -- 172.21.15.46:0/891103909 wait complete. 2024-03-18T20:36:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.992+0000 7fd74c04bb80 20 client.25358 trim_cache size 0 max 16384 2024-03-18T20:36:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:36:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:36:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:36:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:36:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.993+0000 7fd73feb6700 1 -- 172.21.15.46:0/3497569633 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd7240a13a0 msgr2=0x7fd7240a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.993+0000 7fd73feb6700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd7240a13a0 0x7fd7240a3860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd730023000 tx=0x7fd730026000 comp rx=0 tx=0).stop 2024-03-18T20:36:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.993+0000 7fd73feb6700 1 -- 172.21.15.46:0/3497569633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7380ffbb0 msgr2=0x7fd738198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.993+0000 7fd73feb6700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7380ffbb0 0x7fd738198070 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd73400e9d0 tx=0x7fd73400ed90 comp rx=0 tx=0).stop 2024-03-18T20:36:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.993+0000 7fd73feb6700 1 -- 172.21.15.46:0/3497569633 shutdown_connections 2024-03-18T20:36:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.993+0000 7fd73feb6700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd7240a13a0 0x7fd7240a3860 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.993+0000 7fd73feb6700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7380ffbb0 0x7fd738198070 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.993+0000 7fd73feb6700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd738105c60 0x7fd7381a30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.993+0000 7fd73feb6700 1 --2- 172.21.15.46:0/3497569633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd738100570 0x7fd7381985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.993+0000 7fd73feb6700 1 -- 172.21.15.46:0/3497569633 >> 172.21.15.46:0/3497569633 conn(0x7fd738076d50 msgr2=0x7fd7380feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.993+0000 7fd73feb6700 1 -- 172.21.15.46:0/3497569633 shutdown_connections 2024-03-18T20:36:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:22.993+0000 7fd73feb6700 1 -- 172.21.15.46:0/3497569633 wait complete. 2024-03-18T20:36:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:24 smithi073 ceph-mon[47384]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-03-18T20:36:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:24 smithi155 ceph-mon[48344]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-03-18T20:36:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:24 smithi046 ceph-mon[43495]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-03-18T20:36:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:25 smithi073 ceph-mon[47384]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-18T20:36:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:25 smithi155 ceph-mon[48344]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-18T20:36:26.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:25 smithi046 ceph-mon[43495]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-18T20:36:28.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:28 smithi073 ceph-mon[47384]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-18T20:36:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:28 smithi155 ceph-mon[48344]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-18T20:36:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:28 smithi046 ceph-mon[43495]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-18T20:36:28.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:36:28.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:36:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.970+0000 7f510f838700 1 -- 172.21.15.46:0/2670397646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5108067e30 msgr2=0x7f51080682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.970+0000 7f510f838700 1 --2- 172.21.15.46:0/2670397646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5108067e30 0x7f51080682b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009990 tx=0x7f50f8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.970+0000 7f510f838700 1 -- 172.21.15.46:0/2670397646 shutdown_connections 2024-03-18T20:36:28.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.970+0000 7f510f838700 1 --2- 172.21.15.46:0/2670397646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51080687f0 0x7f510810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.970+0000 7f510f838700 1 --2- 172.21.15.46:0/2670397646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5108105ce0 0x7f51081060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.970+0000 7f510f838700 1 --2- 172.21.15.46:0/2670397646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5108067e30 0x7f51080682b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.970+0000 7f510f838700 1 -- 172.21.15.46:0/2670397646 >> 172.21.15.46:0/2670397646 conn(0x7f5108076d40 msgr2=0x7f5108077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.970+0000 7f510f838700 1 -- 172.21.15.46:0/2670397646 shutdown_connections 2024-03-18T20:36:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.970+0000 7f510f838700 1 -- 172.21.15.46:0/2670397646 wait complete. 2024-03-18T20:36:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.971+0000 7f510f838700 1 Processor -- start 2024-03-18T20:36:28.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.971+0000 7f510f838700 1 -- start start 2024-03-18T20:36:28.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.971+0000 7f510f838700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5108067e30 0x7f51080ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:28.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.971+0000 7f510f838700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51080687f0 0x7f51081001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:28.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.971+0000 7f510f838700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5108105ce0 0x7f51081006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:28.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.971+0000 7f510f838700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081a55b0 con 0x7f51080687f0 2024-03-18T20:36:28.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.971+0000 7f510f838700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081a5720 con 0x7f5108067e30 2024-03-18T20:36:28.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.971+0000 7f510f838700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081a58e0 con 0x7f5108105ce0 2024-03-18T20:36:28.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.971+0000 7f510cdd3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51080687f0 0x7f51081001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:28.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.971+0000 7f510cdd3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51080687f0 0x7f51081001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34310/0 (socket says 172.21.15.46:34310) 2024-03-18T20:36:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.971+0000 7f510cdd3700 1 -- 172.21.15.46:0/1698531163 learned_addr learned my addr 172.21.15.46:0/1698531163 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.971+0000 7f510ddd5700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5108105ce0 0x7f51081006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.972+0000 7f510d5d4700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5108067e30 0x7f51080ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.972+0000 7f510cdd3700 1 -- 172.21.15.46:0/1698531163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5108105ce0 msgr2=0x7f51081006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.972+0000 7f510cdd3700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5108105ce0 0x7f51081006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.972+0000 7f510cdd3700 1 -- 172.21.15.46:0/1698531163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5108067e30 msgr2=0x7f51080ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.972+0000 7f510cdd3700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5108067e30 0x7f51080ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.972+0000 7f510cdd3700 1 -- 172.21.15.46:0/1698531163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80095f0 con 0x7f51080687f0 2024-03-18T20:36:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.972+0000 7f510cdd3700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51080687f0 0x7f51081001b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f50f800b2e0 tx=0x7f50f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.972+0000 7f50fe7fc700 1 -- 172.21.15.46:0/1698531163 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f801d070 con 0x7f51080687f0 2024-03-18T20:36:28.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.972+0000 7f50fe7fc700 1 -- 172.21.15.46:0/1698531163 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50f8021410 con 0x7f51080687f0 2024-03-18T20:36:28.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.972+0000 7f510f838700 1 -- 172.21.15.46:0/1698531163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51081a5b60 con 0x7f51080687f0 2024-03-18T20:36:28.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.972+0000 7f50fe7fc700 1 -- 172.21.15.46:0/1698531163 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8017650 con 0x7f51080687f0 2024-03-18T20:36:28.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.972+0000 7f510f838700 1 -- 172.21.15.46:0/1698531163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51081a5fe0 con 0x7f51080687f0 2024-03-18T20:36:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.975+0000 7f50fe7fc700 1 -- 172.21.15.46:0/1698531163 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f50f8021580 con 0x7f51080687f0 2024-03-18T20:36:28.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.976+0000 7f50fe7fc700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f50f40a13a0 0x7f50f40a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:28.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.976+0000 7f50fe7fc700 1 -- 172.21.15.46:0/1698531163 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f50f80cdfd0 con 0x7f51080687f0 2024-03-18T20:36:28.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.976+0000 7f510d5d4700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f50f40a13a0 0x7f50f40a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:28.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.976+0000 7f510d5d4700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f50f40a13a0 0x7f50f40a3860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5104009be0 tx=0x7f5104009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:28.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.977+0000 7f511b9cdb80 1 Processor -- start 2024-03-18T20:36:28.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.977+0000 7f511b9cdb80 1 -- start start 2024-03-18T20:36:28.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.977+0000 7f511b9cdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e4a60 0x56182b1e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:28.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.977+0000 7f511b9cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1e5380 0x56182b1efce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:28.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.977+0000 7f511b9cdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1f0220 0x56182b1f2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:28.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.977+0000 7f511b9cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182b13e990 con 0x56182b1e5380 2024-03-18T20:36:28.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.977+0000 7f511b9cdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182b13c350 con 0x56182b1e4a60 2024-03-18T20:36:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.977+0000 7f511b9cdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182b137c40 con 0x56182b1f0220 2024-03-18T20:36:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.977+0000 7f510cdd3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1e5380 0x56182b1efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.977+0000 7f510cdd3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1e5380 0x56182b1efce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34316/0 (socket says 172.21.15.46:34316) 2024-03-18T20:36:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.977+0000 7f510cdd3700 1 -- 172.21.15.46:0/4083266345 learned_addr learned my addr 172.21.15.46:0/4083266345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.977+0000 7f510d5d4700 1 --2- 172.21.15.46:0/4083266345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e4a60 0x56182b1e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.978+0000 7f510ddd5700 1 --2- 172.21.15.46:0/4083266345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1f0220 0x56182b1f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.978+0000 7f510cdd3700 1 -- 172.21.15.46:0/4083266345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1f0220 msgr2=0x56182b1f2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.978+0000 7f510cdd3700 1 --2- 172.21.15.46:0/4083266345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1f0220 0x56182b1f2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.978+0000 7f510cdd3700 1 -- 172.21.15.46:0/4083266345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e4a60 msgr2=0x56182b1e4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.978+0000 7f510cdd3700 1 --2- 172.21.15.46:0/4083266345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e4a60 0x56182b1e4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.978+0000 7f510cdd3700 1 -- 172.21.15.46:0/4083266345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56182afb93b0 con 0x56182b1e5380 2024-03-18T20:36:28.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.978+0000 7f510cdd3700 1 --2- 172.21.15.46:0/4083266345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1e5380 0x56182b1efce0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f50f8097510 tx=0x7f50f8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee328cbbb8a4672a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.978+0000 7f50f37fe700 1 -- 172.21.15.46:0/4083266345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f809b710 con 0x56182b1e5380 2024-03-18T20:36:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.978+0000 7f50f37fe700 1 -- 172.21.15.46:0/4083266345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50f809b010 con 0x56182b1e5380 2024-03-18T20:36:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.978+0000 7f50f37fe700 1 -- 172.21.15.46:0/4083266345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80cbb60 con 0x56182b1e5380 2024-03-18T20:36:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.979+0000 7f511b9cdb80 1 -- 172.21.15.46:0/4083266345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1e5380 msgr2=0x56182b1efce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.979+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/4083266345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1e5380 0x56182b1efce0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f50f8097510 tx=0x7f50f8009310 comp rx=0 tx=0).stop 2024-03-18T20:36:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.979+0000 7f511b9cdb80 1 -- 172.21.15.46:0/4083266345 shutdown_connections 2024-03-18T20:36:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.979+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/4083266345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1f0220 0x56182b1f2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.979+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/4083266345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e4a60 0x56182b1e4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.979+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/4083266345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1e5380 0x56182b1efce0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.979+0000 7f511b9cdb80 1 -- 172.21.15.46:0/4083266345 >> 172.21.15.46:0/4083266345 conn(0x56182b15b220 msgr2=0x56182b1e1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.979+0000 7f511b9cdb80 1 -- 172.21.15.46:0/4083266345 shutdown_connections 2024-03-18T20:36:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.979+0000 7f511b9cdb80 1 -- 172.21.15.46:0/4083266345 wait complete. 2024-03-18T20:36:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.979+0000 7f511b9cdb80 1 -- start start 2024-03-18T20:36:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f511b9cdb80 1 Processor -- start 2024-03-18T20:36:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f511b9cdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e4a60 0x56182b1f0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f511b9cdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e5380 0x56182b1f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f511b9cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b1fb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f511b9cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182b0d3d80 con 0x56182b1f0220 2024-03-18T20:36:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f511b9cdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182b0d3a20 con 0x56182b1e5380 2024-03-18T20:36:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f511b9cdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182afbb6a0 con 0x56182b1e4a60 2024-03-18T20:36:28.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f511b9cdb80 10 client.0 _ll_register_callbacks cb 0x56182af3f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:28.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f510ddd5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b1fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:28.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f510d5d4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e4a60 0x56182b1f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:28.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f510d5d4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e4a60 0x56182b1f0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59350/0 (socket says 172.21.15.46:59350) 2024-03-18T20:36:28.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f510d5d4700 1 -- 172.21.15.46:0/2002793617 learned_addr learned my addr 172.21.15.46:0/2002793617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:28.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f510ddd5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b1fb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34322/0 (socket says 172.21.15.46:34322) 2024-03-18T20:36:28.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.980+0000 7f510cdd3700 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e5380 0x56182b1f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:28.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.981+0000 7f510d5d4700 1 -- 172.21.15.46:0/2002793617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e5380 msgr2=0x56182b1f0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:28.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.981+0000 7f510d5d4700 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e5380 0x56182b1f0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.981+0000 7f510d5d4700 1 -- 172.21.15.46:0/2002793617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 msgr2=0x56182b1fb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:28.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.981+0000 7f510d5d4700 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b1fb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.981+0000 7f510d5d4700 1 -- 172.21.15.46:0/2002793617 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51081a5fe0 con 0x56182b1e4a60 2024-03-18T20:36:28.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.981+0000 7f510ddd5700 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b1fb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:28.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.981+0000 7f510cdd3700 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e5380 0x56182b1f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:28.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.981+0000 7f510d5d4700 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e4a60 0x56182b1f0600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f51040046e0 tx=0x7f51040047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f92df16a89f5ed1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:28.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.981+0000 7f50f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:36:28.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.981+0000 7f50f0ff9700 1 -- 172.21.15.46:0/2002793617 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510401d700 con 0x56182b1e4a60 2024-03-18T20:36:28.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.981+0000 7f511b9cdb80 1 -- 172.21.15.46:0/2002793617 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56182b1d8ad0 con 0x56182b1e4a60 2024-03-18T20:36:28.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.981+0000 7f50f0ff9700 1 -- 172.21.15.46:0/2002793617 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f510401d860 con 0x56182b1e4a60 2024-03-18T20:36:28.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.982+0000 7f50f0ff9700 1 -- 172.21.15.46:0/2002793617 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f510401da20 con 0x56182b1e4a60 2024-03-18T20:36:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.982+0000 7f511b9cdb80 10 client.25375 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.982+0000 7f511b9cdb80 1 -- 172.21.15.46:0/2002793617 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56182af65b60 con 0x56182b1e4a60 2024-03-18T20:36:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.982+0000 7f50f0ff9700 1 -- 172.21.15.46:0/2002793617 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56182b1d8ad0 con 0x56182b1e4a60 2024-03-18T20:36:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.982+0000 7f50f0ff9700 1 -- 172.21.15.46:0/2002793617 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f510400ebc0 con 0x56182b1e4a60 2024-03-18T20:36:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.982+0000 7f511b9cdb80 10 client.25375 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.982+0000 7f511b9cdb80 10 client.25375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.982+0000 7f511b9cdb80 20 client.25375 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.982+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56182b1ffa90 0x56182b201eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.982+0000 7f511b9cdb80 4 client.25375 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:36:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.982+0000 7f511b9cdb80 1 -- 172.21.15.46:0/2002793617 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56182b130700 con 0x56182b1ffa90 2024-03-18T20:36:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.982+0000 7f510cdd3700 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56182b1ffa90 0x56182b201eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.983+0000 7f50f0ff9700 10 client.25375 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.983+0000 7f510cdd3700 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56182b1ffa90 0x56182b201eb0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=821132b468a2d server_cookie=917dcce06e367b2d in_seq=0 out_seq=0 2024-03-18T20:36:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.983+0000 7f50f0ff9700 1 -- 172.21.15.46:0/2002793617 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56182b130700 con 0x56182b1ffa90 2024-03-18T20:36:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.983+0000 7f50f0ff9700 10 client.25375 handle_command_reply: tid=0 2024-03-18T20:36:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.983+0000 7f511b9cdb80 1 client.25375 shutdown 2024-03-18T20:36:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.983+0000 7f511b9cdb80 1 -- 172.21.15.46:0/2002793617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e4a60 msgr2=0x56182b1f0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.983+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e4a60 0x56182b1f0600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f51040046e0 tx=0x7f51040047c0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.984+0000 7f511b9cdb80 1 -- 172.21.15.46:0/2002793617 shutdown_connections 2024-03-18T20:36:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.984+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56182b1ffa90 0x56182b201eb0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.984+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b1fb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.984+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e4a60 0x56182b1f0600 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.984+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/2002793617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e5380 0x56182b1f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.984+0000 7f50f0ff9700 0 client.25375 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.984+0000 7f511b9cdb80 1 -- 172.21.15.46:0/2002793617 >> 172.21.15.46:0/2002793617 conn(0x56182b15b220 msgr2=0x56182b1e0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.984+0000 7f511b9cdb80 1 -- 172.21.15.46:0/2002793617 shutdown_connections 2024-03-18T20:36:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.984+0000 7f511b9cdb80 1 -- 172.21.15.46:0/2002793617 wait complete. 2024-03-18T20:36:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:28.984+0000 7f511b9cdb80 20 client.25375 trim_cache size 0 max 16384 2024-03-18T20:36:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f511b9cdb80 1 Processor -- start 2024-03-18T20:36:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f511b9cdb80 1 -- start start 2024-03-18T20:36:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f511b9cdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e4a60 0x56182b1f5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f511b9cdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e5380 0x56182b1ef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f511b9cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b1edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f511b9cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182b1fc5e0 con 0x56182b1f0220 2024-03-18T20:36:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f511b9cdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182b1daf80 con 0x56182b1e5380 2024-03-18T20:36:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f511b9cdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182b1e5d30 con 0x56182b1e4a60 2024-03-18T20:36:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f510ddd5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b1edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f510ddd5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b1edd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34324/0 (socket says 172.21.15.46:34324) 2024-03-18T20:36:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f510ddd5700 1 -- 172.21.15.46:0/1995787869 learned_addr learned my addr 172.21.15.46:0/1995787869 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f510cdd3700 1 --2- 172.21.15.46:0/1995787869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e5380 0x56182b1ef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f510d5d4700 1 --2- 172.21.15.46:0/1995787869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e4a60 0x56182b1f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f510ddd5700 1 -- 172.21.15.46:0/1995787869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e4a60 msgr2=0x56182b1f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f510ddd5700 1 --2- 172.21.15.46:0/1995787869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e4a60 0x56182b1f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f510ddd5700 1 -- 172.21.15.46:0/1995787869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e5380 msgr2=0x56182b1ef0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f510ddd5700 1 --2- 172.21.15.46:0/1995787869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e5380 0x56182b1ef0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f510ddd5700 1 -- 172.21.15.46:0/1995787869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56182b0fdea0 con 0x56182b1f0220 2024-03-18T20:36:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.006+0000 7f510cdd3700 1 --2- 172.21.15.46:0/1995787869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e5380 0x56182b1ef0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.007+0000 7f510ddd5700 1 --2- 172.21.15.46:0/1995787869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b1edd50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5100009f30 tx=0x7f5100009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94b03287250750e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.007+0000 7f50f3fff700 1 -- 172.21.15.46:0/1995787869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100003ee0 con 0x56182b1f0220 2024-03-18T20:36:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.007+0000 7f50f3fff700 1 -- 172.21.15.46:0/1995787869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f510000f040 con 0x56182b1f0220 2024-03-18T20:36:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.007+0000 7f50f3fff700 1 -- 172.21.15.46:0/1995787869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51000179b0 con 0x56182b1f0220 2024-03-18T20:36:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.008+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1995787869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 msgr2=0x56182b1edd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.008+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/1995787869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b1edd50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5100009f30 tx=0x7f5100009f60 comp rx=0 tx=0).stop 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.008+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1995787869 shutdown_connections 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.008+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/1995787869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b1edd50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.008+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/1995787869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e4a60 0x56182b1f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.008+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/1995787869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e5380 0x56182b1ef0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.008+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1995787869 >> 172.21.15.46:0/1995787869 conn(0x56182b15b220 msgr2=0x56182b1ee9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.008+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1995787869 shutdown_connections 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.008+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1995787869 wait complete. 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.008+0000 7f511b9cdb80 1 -- start start 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.008+0000 7f511b9cdb80 1 Processor -- start 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.009+0000 7f511b9cdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e4a60 0x56182b24d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.009+0000 7f511b9cdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e5380 0x56182b24d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.009+0000 7f511b9cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b247810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.009+0000 7f511b9cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182b247e20 con 0x56182b1f0220 2024-03-18T20:36:29.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.009+0000 7f511b9cdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182b247f90 con 0x56182b1e4a60 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.009+0000 7f511b9cdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182b248150 con 0x56182b1e5380 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.009+0000 7f511b9cdb80 10 client.0 _ll_register_callbacks cb 0x7f510401f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.009+0000 7f510ddd5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b247810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.009+0000 7f510ddd5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b247810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34332/0 (socket says 172.21.15.46:34332) 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.009+0000 7f510ddd5700 1 -- 172.21.15.46:0/1024403867 learned_addr learned my addr 172.21.15.46:0/1024403867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.009+0000 7f510d5d4700 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e4a60 0x56182b24d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.009+0000 7f510cdd3700 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e5380 0x56182b24d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f510ddd5700 1 -- 172.21.15.46:0/1024403867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e5380 msgr2=0x56182b24d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f510ddd5700 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e5380 0x56182b24d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f510ddd5700 1 -- 172.21.15.46:0/1024403867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e4a60 msgr2=0x56182b24d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f510ddd5700 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e4a60 0x56182b24d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f510ddd5700 1 -- 172.21.15.46:0/1024403867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5100009c10 con 0x56182b1f0220 2024-03-18T20:36:29.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f510ddd5700 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b247810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5100000f80 tx=0x7f5100017d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f96a3c71776b38 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f50f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:36:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f50f0ff9700 1 -- 172.21.15.46:0/1024403867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100004cd0 con 0x56182b1f0220 2024-03-18T20:36:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f50f0ff9700 1 -- 172.21.15.46:0/1024403867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5100004e30 con 0x56182b1f0220 2024-03-18T20:36:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1024403867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50f809b010 con 0x56182b1f0220 2024-03-18T20:36:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f50f0ff9700 1 -- 172.21.15.46:0/1024403867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5100005050 con 0x56182b1f0220 2024-03-18T20:36:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f511b9cdb80 10 client.15606 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.010+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1024403867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56182afbdcd0 con 0x56182b1f0220 2024-03-18T20:36:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.011+0000 7f50f0ff9700 1 -- 172.21.15.46:0/1024403867 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5100034020 con 0x56182b1f0220 2024-03-18T20:36:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.011+0000 7f50f0ff9700 1 -- 172.21.15.46:0/1024403867 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f510001d020 con 0x56182b1f0220 2024-03-18T20:36:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.011+0000 7f511b9cdb80 10 client.15606 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:29.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.011+0000 7f511b9cdb80 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.011+0000 7f511b9cdb80 20 client.15606 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.011+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56182b254e60 0x56182b257250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.011+0000 7f511b9cdb80 4 client.15606 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.011+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1024403867 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56182af86cf0 con 0x56182b254e60 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.011+0000 7f510d5d4700 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56182b254e60 0x56182b257250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f510d5d4700 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56182b254e60 0x56182b257250 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec6416378c68b22 server_cookie=7e36fd3e9f7e3aec in_seq=0 out_seq=0 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f50f0ff9700 10 client.15606 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f50f0ff9700 1 -- 172.21.15.46:0/1024403867 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56182af86cf0 con 0x56182b254e60 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f50f0ff9700 10 client.15606 handle_command_reply: tid=0 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f511b9cdb80 1 client.15606 shutdown 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1024403867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 msgr2=0x56182b247810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b247810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5100000f80 tx=0x7f5100017d10 comp rx=0 tx=0).stop 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1024403867 shutdown_connections 2024-03-18T20:36:29.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56182b254e60 0x56182b257250 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f50f0ff9700 0 client.15606 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56182b1f0220 0x56182b247810 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56182b1e4a60 0x56182b24d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f511b9cdb80 1 --2- 172.21.15.46:0/1024403867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56182b1e5380 0x56182b24d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1024403867 >> 172.21.15.46:0/1024403867 conn(0x56182b15b220 msgr2=0x56182b1ee9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1024403867 shutdown_connections 2024-03-18T20:36:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f511b9cdb80 1 -- 172.21.15.46:0/1024403867 wait complete. 2024-03-18T20:36:29.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.012+0000 7f511b9cdb80 20 client.15606 trim_cache size 0 max 16384 2024-03-18T20:36:29.025 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:36:29.025 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:36:29.025 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:36:29.025 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:36:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.013+0000 7f510f838700 1 -- 172.21.15.46:0/1698531163 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f50f40a13a0 msgr2=0x7f50f40a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.013+0000 7f510f838700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f50f40a13a0 0x7f50f40a3860 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5104009be0 tx=0x7f5104009260 comp rx=0 tx=0).stop 2024-03-18T20:36:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.013+0000 7f510f838700 1 -- 172.21.15.46:0/1698531163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51080687f0 msgr2=0x7f51081001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.013+0000 7f510f838700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51080687f0 0x7f51081001b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f50f800b2e0 tx=0x7f50f8004730 comp rx=0 tx=0).stop 2024-03-18T20:36:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.013+0000 7f510f838700 1 -- 172.21.15.46:0/1698531163 shutdown_connections 2024-03-18T20:36:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.013+0000 7f510f838700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f50f40a13a0 0x7f50f40a3860 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.013+0000 7f510f838700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5108105ce0 0x7f51081006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.013+0000 7f510f838700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5108067e30 0x7f51080ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.013+0000 7f510f838700 1 --2- 172.21.15.46:0/1698531163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51080687f0 0x7f51081001b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.013+0000 7f510f838700 1 -- 172.21.15.46:0/1698531163 >> 172.21.15.46:0/1698531163 conn(0x7f5108076d40 msgr2=0x7f51080fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.013+0000 7f510f838700 1 -- 172.21.15.46:0/1698531163 shutdown_connections 2024-03-18T20:36:29.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.013+0000 7f510f838700 1 -- 172.21.15.46:0/1698531163 wait complete. 2024-03-18T20:36:29.030 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:36:29.030 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:36:29.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.168+0000 7f8938125700 1 -- 172.21.15.46:0/3740981003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8930102740 msgr2=0x7f893010ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.168+0000 7f8938125700 1 --2- 172.21.15.46:0/3740981003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8930102740 0x7f893010ac20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8920009990 tx=0x7f8920009ca0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.168+0000 7f8938125700 1 -- 172.21.15.46:0/3740981003 shutdown_connections 2024-03-18T20:36:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.168+0000 7f8938125700 1 --2- 172.21.15.46:0/3740981003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f893010b1f0 0x7f893010d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.169+0000 7f8938125700 1 --2- 172.21.15.46:0/3740981003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8930101e20 0x7f8930102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.169+0000 7f8938125700 1 --2- 172.21.15.46:0/3740981003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8930102740 0x7f893010ac20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.169+0000 7f8938125700 1 -- 172.21.15.46:0/3740981003 >> 172.21.15.46:0/3740981003 conn(0x7f89300fb620 msgr2=0x7f89300fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.169+0000 7f8938125700 1 -- 172.21.15.46:0/3740981003 shutdown_connections 2024-03-18T20:36:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.169+0000 7f8938125700 1 -- 172.21.15.46:0/3740981003 wait complete. 2024-03-18T20:36:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.169+0000 7f8938125700 1 Processor -- start 2024-03-18T20:36:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.169+0000 7f8938125700 1 -- start start 2024-03-18T20:36:29.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8938125700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8930101e20 0x7f8930197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8938125700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8930102740 0x7f8930197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8938125700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893010b1f0 0x7f893019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8938125700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893019c030 con 0x7f8930101e20 2024-03-18T20:36:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8938125700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893019c1a0 con 0x7f893010b1f0 2024-03-18T20:36:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8938125700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893019c360 con 0x7f8930102740 2024-03-18T20:36:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8935ec1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8930101e20 0x7f8930197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8935ec1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8930101e20 0x7f8930197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34342/0 (socket says 172.21.15.46:34342) 2024-03-18T20:36:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8935ec1700 1 -- 172.21.15.46:0/2308313593 learned_addr learned my addr 172.21.15.46:0/2308313593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f89356c0700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8930102740 0x7f8930197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f89366c2700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893010b1f0 0x7f893019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8935ec1700 1 -- 172.21.15.46:0/2308313593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8930102740 msgr2=0x7f8930197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8935ec1700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8930102740 0x7f8930197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8935ec1700 1 -- 172.21.15.46:0/2308313593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893010b1f0 msgr2=0x7f893019b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8935ec1700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893010b1f0 0x7f893019b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8935ec1700 1 -- 172.21.15.46:0/2308313593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89200095f0 con 0x7f8930101e20 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f89356c0700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8930102740 0x7f8930197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f89366c2700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893010b1f0 0x7f893019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.170+0000 7f8935ec1700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8930101e20 0x7f8930197110 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f892c00e9a0 tx=0x7f892c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.171+0000 7f8926ffd700 1 -- 172.21.15.46:0/2308313593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c009790 con 0x7f8930101e20 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.171+0000 7f8938125700 1 -- 172.21.15.46:0/2308313593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893019c640 con 0x7f8930101e20 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.171+0000 7f8926ffd700 1 -- 172.21.15.46:0/2308313593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f892c00cbd0 con 0x7f8930101e20 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.171+0000 7f8938125700 1 -- 172.21.15.46:0/2308313593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89300ffb80 con 0x7f8930101e20 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.171+0000 7f8926ffd700 1 -- 172.21.15.46:0/2308313593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c018620 con 0x7f8930101e20 2024-03-18T20:36:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.172+0000 7f8938125700 1 -- 172.21.15.46:0/2308313593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8930106310 con 0x7f8930101e20 2024-03-18T20:36:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.174+0000 7f8926ffd700 1 -- 172.21.15.46:0/2308313593 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f892c00cd40 con 0x7f8930101e20 2024-03-18T20:36:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.175+0000 7f8926ffd700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f891c0a1320 0x7f891c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.175+0000 7f8926ffd700 1 -- 172.21.15.46:0/2308313593 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f892c014070 con 0x7f8930101e20 2024-03-18T20:36:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.175+0000 7f89356c0700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f891c0a1320 0x7f891c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.176+0000 7f89356c0700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f891c0a1320 0x7f891c0a37e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f892000b2e0 tx=0x7f8920005fd0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.178+0000 7f8926ffd700 1 -- 172.21.15.46:0/2308313593 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f892c096660 con 0x7f8930101e20 2024-03-18T20:36:29.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.479+0000 7f8938125700 1 -- 172.21.15.46:0/2308313593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89300698d0 con 0x7f8930101e20 2024-03-18T20:36:29.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.480+0000 7f8926ffd700 1 -- 172.21.15.46:0/2308313593 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f892c099c80 con 0x7f8930101e20 2024-03-18T20:36:29.483 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:36:29.483 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:35:51.152306+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:36:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.482+0000 7f8938125700 1 -- 172.21.15.46:0/2308313593 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f891c0a1320 msgr2=0x7f891c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.482+0000 7f8938125700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f891c0a1320 0x7f891c0a37e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f892000b2e0 tx=0x7f8920005fd0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.482+0000 7f8938125700 1 -- 172.21.15.46:0/2308313593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8930101e20 msgr2=0x7f8930197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.482+0000 7f8938125700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8930101e20 0x7f8930197110 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f892c00e9a0 tx=0x7f892c00ed60 comp rx=0 tx=0).stop 2024-03-18T20:36:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.482+0000 7f8938125700 1 -- 172.21.15.46:0/2308313593 shutdown_connections 2024-03-18T20:36:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.482+0000 7f8938125700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f891c0a1320 0x7f891c0a37e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.482+0000 7f8938125700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8930101e20 0x7f8930197110 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.482+0000 7f8938125700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893010b1f0 0x7f893019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.482+0000 7f8938125700 1 --2- 172.21.15.46:0/2308313593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8930102740 0x7f8930197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.482+0000 7f8938125700 1 -- 172.21.15.46:0/2308313593 >> 172.21.15.46:0/2308313593 conn(0x7f89300fb620 msgr2=0x7f89300fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.482+0000 7f8938125700 1 -- 172.21.15.46:0/2308313593 shutdown_connections 2024-03-18T20:36:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.483+0000 7f8938125700 1 -- 172.21.15.46:0/2308313593 wait complete. 2024-03-18T20:36:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-03-18T20:36:29.500 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:36:29.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:36:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.656+0000 7f693806e700 1 -- 172.21.15.46:0/3966250693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69300687f0 msgr2=0x7f693010db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.656+0000 7f693806e700 1 --2- 172.21.15.46:0/3966250693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69300687f0 0x7f693010db50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f692800b0a0 tx=0x7f692800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.656+0000 7f693806e700 1 -- 172.21.15.46:0/3966250693 shutdown_connections 2024-03-18T20:36:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.656+0000 7f693806e700 1 --2- 172.21.15.46:0/3966250693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69300687f0 0x7f693010db50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.656+0000 7f693806e700 1 --2- 172.21.15.46:0/3966250693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6930105ce0 0x7f69301060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.656+0000 7f693806e700 1 --2- 172.21.15.46:0/3966250693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6930067e30 0x7f69300682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.656+0000 7f693806e700 1 -- 172.21.15.46:0/3966250693 >> 172.21.15.46:0/3966250693 conn(0x7f6930076d40 msgr2=0x7f6930077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.656+0000 7f693806e700 1 -- 172.21.15.46:0/3966250693 shutdown_connections 2024-03-18T20:36:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.656+0000 7f693806e700 1 -- 172.21.15.46:0/3966250693 wait complete. 2024-03-18T20:36:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f693806e700 1 Processor -- start 2024-03-18T20:36:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f693806e700 1 -- start start 2024-03-18T20:36:29.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f693806e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6930067e30 0x7f6930199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f693806e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69300687f0 0x7f6930199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f693806e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6930105ce0 0x7f693019de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f693806e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693019e440 con 0x7f6930105ce0 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f693806e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693019e5b0 con 0x7f6930067e30 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f693806e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693019e770 con 0x7f69300687f0 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f693660b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6930105ce0 0x7f693019de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f6935e0a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6930067e30 0x7f6930199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f693660b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6930105ce0 0x7f693019de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34362/0 (socket says 172.21.15.46:34362) 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f6935609700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69300687f0 0x7f6930199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.657+0000 7f693660b700 1 -- 172.21.15.46:0/1649519836 learned_addr learned my addr 172.21.15.46:0/1649519836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.658+0000 7f693660b700 1 -- 172.21.15.46:0/1649519836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69300687f0 msgr2=0x7f6930199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.658+0000 7f693660b700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69300687f0 0x7f6930199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.658+0000 7f693660b700 1 -- 172.21.15.46:0/1649519836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6930067e30 msgr2=0x7f6930199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.658+0000 7f693660b700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6930067e30 0x7f6930199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.658+0000 7f693660b700 1 -- 172.21.15.46:0/1649519836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c009520 con 0x7f6930105ce0 2024-03-18T20:36:29.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.658+0000 7f6935e0a700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6930067e30 0x7f6930199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:29.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.658+0000 7f693660b700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6930105ce0 0x7f693019de00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f692800b070 tx=0x7f692800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.658+0000 7f6926ffd700 1 -- 172.21.15.46:0/1649519836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692800e040 con 0x7f6930105ce0 2024-03-18T20:36:29.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.658+0000 7f693806e700 1 -- 172.21.15.46:0/1649519836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6928009d00 con 0x7f6930105ce0 2024-03-18T20:36:29.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.658+0000 7f6926ffd700 1 -- 172.21.15.46:0/1649519836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f692801d470 con 0x7f6930105ce0 2024-03-18T20:36:29.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.658+0000 7f6926ffd700 1 -- 172.21.15.46:0/1649519836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69280079d0 con 0x7f6930105ce0 2024-03-18T20:36:29.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.658+0000 7f693806e700 1 -- 172.21.15.46:0/1649519836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69301a60b0 con 0x7f6930105ce0 2024-03-18T20:36:29.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.660+0000 7f6926ffd700 1 -- 172.21.15.46:0/1649519836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f692801d5e0 con 0x7f6930105ce0 2024-03-18T20:36:29.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.661+0000 7f6926ffd700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f691c0a1320 0x7f691c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.661+0000 7f6935e0a700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f691c0a1320 0x7f691c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.661+0000 7f6926ffd700 1 -- 172.21.15.46:0/1649519836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f69280ce0f0 con 0x7f6930105ce0 2024-03-18T20:36:29.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.662+0000 7f6935e0a700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f691c0a1320 0x7f691c0a37e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f692c009fd0 tx=0x7f692c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.663+0000 7f6944203b80 1 Processor -- start 2024-03-18T20:36:29.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6944203b80 1 -- start start 2024-03-18T20:36:29.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6944203b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40caa60 0x55afa40cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6944203b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 0x55afa40d5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6944203b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40d6220 0x55afa40d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6944203b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa4024990 con 0x55afa40cb380 2024-03-18T20:36:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6944203b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa4022350 con 0x55afa40d6220 2024-03-18T20:36:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6944203b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa401dc40 con 0x55afa40caa60 2024-03-18T20:36:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6935609700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 0x55afa40d5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6935609700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 0x55afa40d5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34374/0 (socket says 172.21.15.46:34374) 2024-03-18T20:36:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6935609700 1 -- 172.21.15.46:0/585233528 learned_addr learned my addr 172.21.15.46:0/585233528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f693660b700 1 --2- 172.21.15.46:0/585233528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40d6220 0x55afa40d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6935e0a700 1 --2- 172.21.15.46:0/585233528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40caa60 0x55afa40cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6935609700 1 -- 172.21.15.46:0/585233528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40caa60 msgr2=0x55afa40cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6935609700 1 --2- 172.21.15.46:0/585233528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40caa60 0x55afa40cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6935609700 1 -- 172.21.15.46:0/585233528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40d6220 msgr2=0x55afa40d8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6935609700 1 --2- 172.21.15.46:0/585233528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40d6220 0x55afa40d8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.664+0000 7f6935609700 1 -- 172.21.15.46:0/585233528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afa3e9f3b0 con 0x55afa40cb380 2024-03-18T20:36:29.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.665+0000 7f6935609700 1 --2- 172.21.15.46:0/585233528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 0x55afa40d5ce0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6920009d20 tx=0x7f6920011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53845da0602d8e29 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.665+0000 7f691bfff700 1 -- 172.21.15.46:0/585233528 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920019630 con 0x55afa40cb380 2024-03-18T20:36:29.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.665+0000 7f691bfff700 1 -- 172.21.15.46:0/585233528 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6920019c50 con 0x55afa40cb380 2024-03-18T20:36:29.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.665+0000 7f691bfff700 1 -- 172.21.15.46:0/585233528 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692000eb90 con 0x55afa40cb380 2024-03-18T20:36:29.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.666+0000 7f6944203b80 1 -- 172.21.15.46:0/585233528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 msgr2=0x55afa40d5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.666+0000 7f6944203b80 1 --2- 172.21.15.46:0/585233528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 0x55afa40d5ce0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6920009d20 tx=0x7f6920011720 comp rx=0 tx=0).stop 2024-03-18T20:36:29.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.666+0000 7f6944203b80 1 -- 172.21.15.46:0/585233528 shutdown_connections 2024-03-18T20:36:29.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.666+0000 7f6944203b80 1 --2- 172.21.15.46:0/585233528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40d6220 0x55afa40d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.666+0000 7f6944203b80 1 --2- 172.21.15.46:0/585233528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40caa60 0x55afa40cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.666+0000 7f6944203b80 1 --2- 172.21.15.46:0/585233528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 0x55afa40d5ce0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.666+0000 7f6944203b80 1 -- 172.21.15.46:0/585233528 >> 172.21.15.46:0/585233528 conn(0x55afa4041220 msgr2=0x55afa40c7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.666+0000 7f6944203b80 1 -- 172.21.15.46:0/585233528 shutdown_connections 2024-03-18T20:36:29.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.666+0000 7f6944203b80 1 -- 172.21.15.46:0/585233528 wait complete. 2024-03-18T20:36:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.666+0000 7f6944203b80 1 -- start start 2024-03-18T20:36:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.667+0000 7f6944203b80 1 Processor -- start 2024-03-18T20:36:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.667+0000 7f6944203b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40caa60 0x55afa40d5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.667+0000 7f6944203b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40cb380 0x55afa40d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.667+0000 7f6944203b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40d6220 0x55afa40d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.667+0000 7f6944203b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa3fb9d80 con 0x55afa40d6220 2024-03-18T20:36:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.667+0000 7f6944203b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa3fb9a20 con 0x55afa40caa60 2024-03-18T20:36:29.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.667+0000 7f6944203b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa3ea16a0 con 0x55afa40cb380 2024-03-18T20:36:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f6944203b80 10 client.0 _ll_register_callbacks cb 0x55afa3e253a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f693660b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40d6220 0x55afa40d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f693660b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40d6220 0x55afa40d36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34386/0 (socket says 172.21.15.46:34386) 2024-03-18T20:36:29.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f693660b700 1 -- 172.21.15.46:0/369619431 learned_addr learned my addr 172.21.15.46:0/369619431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f6935609700 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40cb380 0x55afa40d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f6935e0a700 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40caa60 0x55afa40d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f693660b700 1 -- 172.21.15.46:0/369619431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40cb380 msgr2=0x55afa40d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f693660b700 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40cb380 0x55afa40d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f693660b700 1 -- 172.21.15.46:0/369619431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40caa60 msgr2=0x55afa40d5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f693660b700 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40caa60 0x55afa40d5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f693660b700 1 -- 172.21.15.46:0/369619431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6920009980 con 0x55afa40d6220 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f6935609700 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40cb380 0x55afa40d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f693660b700 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40d6220 0x55afa40d36b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f692801d890 tx=0x7f692809aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66a1a6de634640a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f69197fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f69197fa700 1 -- 172.21.15.46:0/369619431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69280cbc40 con 0x55afa40d6220 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f69197fa700 1 -- 172.21.15.46:0/369619431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f692809ef60 con 0x55afa40d6220 2024-03-18T20:36:29.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f6944203b80 1 -- 172.21.15.46:0/369619431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afa40bead0 con 0x55afa40d6220 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.668+0000 7f69197fa700 1 -- 172.21.15.46:0/369619431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f692801daf0 con 0x55afa40d6220 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.669+0000 7f6944203b80 10 client.15624 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.669+0000 7f6944203b80 1 -- 172.21.15.46:0/369619431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55afa3e4bb60 con 0x55afa40d6220 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.669+0000 7f69197fa700 1 -- 172.21.15.46:0/369619431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6928064080 con 0x55afa40d6220 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.669+0000 7f69197fa700 1 -- 172.21.15.46:0/369619431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f69280b9020 con 0x55afa40d6220 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.669+0000 7f6944203b80 10 client.15624 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.669+0000 7f6944203b80 10 client.15624 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.669+0000 7f6944203b80 20 client.15624 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.669+0000 7f6944203b80 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55afa40e5bd0 0x55afa40e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.669+0000 7f6944203b80 4 client.15624 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.669+0000 7f6944203b80 1 -- 172.21.15.46:0/369619431 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afa4016700 con 0x55afa40e5bd0 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.669+0000 7f6935e0a700 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55afa40e5bd0 0x55afa40e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.670+0000 7f6935e0a700 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55afa40e5bd0 0x55afa40e7ff0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72bdac623a0bf9ea server_cookie=9a4f1f2e17d638c5 in_seq=0 out_seq=0 2024-03-18T20:36:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.670+0000 7f69197fa700 10 client.15624 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.670+0000 7f69197fa700 1 -- 172.21.15.46:0/369619431 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55afa4016700 con 0x55afa40e5bd0 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.670+0000 7f69197fa700 10 client.15624 handle_command_reply: tid=0 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.670+0000 7f6944203b80 1 client.15624 shutdown 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.671+0000 7f6944203b80 1 -- 172.21.15.46:0/369619431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40d6220 msgr2=0x55afa40d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.671+0000 7f6944203b80 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40d6220 0x55afa40d36b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f692801d890 tx=0x7f692809aa90 comp rx=0 tx=0).stop 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.671+0000 7f6944203b80 1 -- 172.21.15.46:0/369619431 shutdown_connections 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.671+0000 7f6944203b80 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55afa40e5bd0 0x55afa40e7ff0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.671+0000 7f6944203b80 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40d6220 0x55afa40d36b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.671+0000 7f6944203b80 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40caa60 0x55afa40d5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.671+0000 7f6944203b80 1 --2- 172.21.15.46:0/369619431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40cb380 0x55afa40d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.671+0000 7f69197fa700 0 client.15624 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.671+0000 7f6944203b80 1 -- 172.21.15.46:0/369619431 >> 172.21.15.46:0/369619431 conn(0x55afa4041220 msgr2=0x55afa40c6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.671+0000 7f6944203b80 1 -- 172.21.15.46:0/369619431 shutdown_connections 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.671+0000 7f6944203b80 1 -- 172.21.15.46:0/369619431 wait complete. 2024-03-18T20:36:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.671+0000 7f6944203b80 20 client.15624 trim_cache size 0 max 16384 2024-03-18T20:36:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.685+0000 7f6944203b80 1 Processor -- start 2024-03-18T20:36:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.685+0000 7f6944203b80 1 -- start start 2024-03-18T20:36:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.685+0000 7f6944203b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40caa60 0x55afa40d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.685+0000 7f6944203b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40cb380 0x55afa40dce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.685+0000 7f6944203b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40d6220 0x55afa40c5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.685+0000 7f6944203b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa4091940 con 0x55afa40caa60 2024-03-18T20:36:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.685+0000 7f6944203b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa40c0f80 con 0x55afa40d6220 2024-03-18T20:36:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.685+0000 7f6944203b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa40c8260 con 0x55afa40cb380 2024-03-18T20:36:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f6935e0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40caa60 0x55afa40d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f6935609700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40cb380 0x55afa40dce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f6935e0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40caa60 0x55afa40d2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34402/0 (socket says 172.21.15.46:34402) 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f6935e0a700 1 -- 172.21.15.46:0/1761055840 learned_addr learned my addr 172.21.15.46:0/1761055840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f693660b700 1 --2- 172.21.15.46:0/1761055840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40d6220 0x55afa40c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f6935e0a700 1 -- 172.21.15.46:0/1761055840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40cb380 msgr2=0x55afa40dce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f6935e0a700 1 --2- 172.21.15.46:0/1761055840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40cb380 0x55afa40dce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f6935e0a700 1 -- 172.21.15.46:0/1761055840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40d6220 msgr2=0x55afa40c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f6935e0a700 1 --2- 172.21.15.46:0/1761055840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40d6220 0x55afa40c5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f6935e0a700 1 -- 172.21.15.46:0/1761055840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afa40e2300 con 0x55afa40caa60 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f693660b700 1 --2- 172.21.15.46:0/1761055840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40d6220 0x55afa40c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f6935e0a700 1 --2- 172.21.15.46:0/1761055840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40caa60 0x55afa40d2f30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55afa40e88d0 tx=0x7f692c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357e59549e3cca18 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.686+0000 7f6924ff9700 1 -- 172.21.15.46:0/1761055840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c021050 con 0x55afa40caa60 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.687+0000 7f6924ff9700 1 -- 172.21.15.46:0/1761055840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f692c022040 con 0x55afa40caa60 2024-03-18T20:36:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.687+0000 7f6924ff9700 1 -- 172.21.15.46:0/1761055840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c0299f0 con 0x55afa40caa60 2024-03-18T20:36:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.687+0000 7f6944203b80 1 -- 172.21.15.46:0/1761055840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40caa60 msgr2=0x55afa40d2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.687+0000 7f6944203b80 1 --2- 172.21.15.46:0/1761055840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40caa60 0x55afa40d2f30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55afa40e88d0 tx=0x7f692c014000 comp rx=0 tx=0).stop 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.687+0000 7f6944203b80 1 -- 172.21.15.46:0/1761055840 shutdown_connections 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.687+0000 7f6944203b80 1 --2- 172.21.15.46:0/1761055840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40caa60 0x55afa40d2f30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.687+0000 7f6944203b80 1 --2- 172.21.15.46:0/1761055840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40d6220 0x55afa40c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.687+0000 7f6944203b80 1 --2- 172.21.15.46:0/1761055840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40cb380 0x55afa40dce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.687+0000 7f6944203b80 1 -- 172.21.15.46:0/1761055840 >> 172.21.15.46:0/1761055840 conn(0x55afa4041220 msgr2=0x55afa40c2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.687+0000 7f6944203b80 1 -- 172.21.15.46:0/1761055840 shutdown_connections 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.687+0000 7f6944203b80 1 -- 172.21.15.46:0/1761055840 wait complete. 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.688+0000 7f6944203b80 1 -- start start 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.688+0000 7f6944203b80 1 Processor -- start 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.688+0000 7f6944203b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40caa60 0x55afa40dc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.688+0000 7f6944203b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 0x55afa40dcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.688+0000 7f6944203b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40d6220 0x55afa40daf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.688+0000 7f6944203b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa40db650 con 0x55afa40cb380 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.688+0000 7f6944203b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa40db7c0 con 0x55afa40caa60 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.688+0000 7f6944203b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa40db980 con 0x55afa40d6220 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.688+0000 7f6944203b80 10 client.0 _ll_register_callbacks cb 0x7f692809ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.688+0000 7f6935609700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 0x55afa40dcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.688+0000 7f6935609700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 0x55afa40dcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34404/0 (socket says 172.21.15.46:34404) 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.688+0000 7f6935609700 1 -- 172.21.15.46:0/1258848450 learned_addr learned my addr 172.21.15.46:0/1258848450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f6935e0a700 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40caa60 0x55afa40dc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f693660b700 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40d6220 0x55afa40daf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f6935609700 1 -- 172.21.15.46:0/1258848450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40d6220 msgr2=0x55afa40daf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f6935609700 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40d6220 0x55afa40daf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f6935609700 1 -- 172.21.15.46:0/1258848450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40caa60 msgr2=0x55afa40dc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f6935609700 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40caa60 0x55afa40dc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f6935609700 1 -- 172.21.15.46:0/1258848450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afa3e4bb60 con 0x55afa40cb380 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f6935609700 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 0x55afa40dcdb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f692001c040 tx=0x7f6920019ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d0720e209fc66b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f69197fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f69197fa700 1 -- 172.21.15.46:0/1258848450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920005450 con 0x55afa40cb380 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f69197fa700 1 -- 172.21.15.46:0/1258848450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6920005ce0 con 0x55afa40cb380 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f6944203b80 1 -- 172.21.15.46:0/1258848450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6920019c50 con 0x55afa40cb380 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f69197fa700 1 -- 172.21.15.46:0/1258848450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69200055b0 con 0x55afa40cb380 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f6944203b80 10 client.15630 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.689+0000 7f6944203b80 1 -- 172.21.15.46:0/1258848450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f692c017050 con 0x55afa40cb380 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.690+0000 7f69197fa700 1 -- 172.21.15.46:0/1258848450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6920019c50 con 0x55afa40cb380 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.690+0000 7f69197fa700 1 -- 172.21.15.46:0/1258848450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6920034070 con 0x55afa40cb380 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.690+0000 7f6944203b80 10 client.15630 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.690+0000 7f6944203b80 10 client.15630 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.690+0000 7f6944203b80 20 client.15630 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.690+0000 7f6944203b80 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55afa413acf0 0x55afa413d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.690+0000 7f6944203b80 4 client.15630 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.690+0000 7f6944203b80 1 -- 172.21.15.46:0/1258848450 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55afa3e6ccf0 con 0x55afa413acf0 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.690+0000 7f6935e0a700 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55afa413acf0 0x55afa413d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6935e0a700 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55afa413acf0 0x55afa413d110 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e0801aaaf2b0ad33 server_cookie=fe428d730da2d4d5 in_seq=0 out_seq=0 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f69197fa700 10 client.15630 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f69197fa700 1 -- 172.21.15.46:0/1258848450 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55afa3e6ccf0 con 0x55afa413acf0 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f69197fa700 10 client.15630 handle_command_reply: tid=0 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6944203b80 1 client.15630 shutdown 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6944203b80 1 -- 172.21.15.46:0/1258848450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 msgr2=0x55afa40dcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6944203b80 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 0x55afa40dcdb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f692001c040 tx=0x7f6920019ab0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6944203b80 1 -- 172.21.15.46:0/1258848450 shutdown_connections 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6944203b80 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55afa413acf0 0x55afa413d110 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6944203b80 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55afa40d6220 0x55afa40daf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6944203b80 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55afa40caa60 0x55afa40dc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6944203b80 1 --2- 172.21.15.46:0/1258848450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55afa40cb380 0x55afa40dcdb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f69197fa700 0 client.15630 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:36:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6944203b80 1 -- 172.21.15.46:0/1258848450 >> 172.21.15.46:0/1258848450 conn(0x55afa4041220 msgr2=0x55afa40c5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6944203b80 1 -- 172.21.15.46:0/1258848450 shutdown_connections 2024-03-18T20:36:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6944203b80 1 -- 172.21.15.46:0/1258848450 wait complete. 2024-03-18T20:36:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.691+0000 7f6944203b80 20 client.15630 trim_cache size 0 max 16384 2024-03-18T20:36:29.700 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:36:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.692+0000 7f693806e700 1 -- 172.21.15.46:0/1649519836 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f691c0a1320 msgr2=0x7f691c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.692+0000 7f693806e700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f691c0a1320 0x7f691c0a37e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f692c009fd0 tx=0x7f692c009260 comp rx=0 tx=0).stop 2024-03-18T20:36:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.692+0000 7f693806e700 1 -- 172.21.15.46:0/1649519836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6930105ce0 msgr2=0x7f693019de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.692+0000 7f693806e700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6930105ce0 0x7f693019de00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f692800b070 tx=0x7f692800bdd0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.692+0000 7f693806e700 1 -- 172.21.15.46:0/1649519836 shutdown_connections 2024-03-18T20:36:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.692+0000 7f693806e700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f691c0a1320 0x7f691c0a37e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.692+0000 7f693806e700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6930105ce0 0x7f693019de00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.692+0000 7f693806e700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6930067e30 0x7f6930199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.692+0000 7f693806e700 1 --2- 172.21.15.46:0/1649519836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69300687f0 0x7f6930199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.692+0000 7f693806e700 1 -- 172.21.15.46:0/1649519836 >> 172.21.15.46:0/1649519836 conn(0x7f6930076d40 msgr2=0x7f6930104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.692+0000 7f693806e700 1 -- 172.21.15.46:0/1649519836 shutdown_connections 2024-03-18T20:36:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.692+0000 7f693806e700 1 -- 172.21.15.46:0/1649519836 wait complete. 2024-03-18T20:36:29.711 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:36:29.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:36:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.854+0000 7fa9e18d2700 1 -- 172.21.15.46:0/347769077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc069c30 msgr2=0x7fa9dc105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.854+0000 7fa9e18d2700 1 --2- 172.21.15.46:0/347769077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc069c30 0x7fa9dc105980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009920 tx=0x7fa9d0009c30 comp rx=0 tx=0).stop 2024-03-18T20:36:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.854+0000 7fa9e18d2700 1 -- 172.21.15.46:0/347769077 shutdown_connections 2024-03-18T20:36:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.854+0000 7fa9e18d2700 1 --2- 172.21.15.46:0/347769077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060b0 0x7fa9dc10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.854+0000 7fa9e18d2700 1 --2- 172.21.15.46:0/347769077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069280 0x7fa9dc069660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.854+0000 7fa9e18d2700 1 --2- 172.21.15.46:0/347769077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc069c30 0x7fa9dc105980 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.854+0000 7fa9e18d2700 1 -- 172.21.15.46:0/347769077 >> 172.21.15.46:0/347769077 conn(0x7fa9dc076d80 msgr2=0x7fa9dc077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.854+0000 7fa9e18d2700 1 -- 172.21.15.46:0/347769077 shutdown_connections 2024-03-18T20:36:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.854+0000 7fa9e18d2700 1 -- 172.21.15.46:0/347769077 wait complete. 2024-03-18T20:36:29.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.855+0000 7fa9e18d2700 1 Processor -- start 2024-03-18T20:36:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.855+0000 7fa9e18d2700 1 -- start start 2024-03-18T20:36:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.855+0000 7fa9e18d2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069280 0x7fa9dc1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.855+0000 7fa9e18d2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060b0 0x7fa9dc1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9e18d2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc19bb40 0x7fa9dc19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9e18d2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc19c5c0 con 0x7fa9dc1060b0 2024-03-18T20:36:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9e18d2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc19c730 con 0x7fa9dc069280 2024-03-18T20:36:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9e18d2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc19c8f0 con 0x7fa9dc19bb40 2024-03-18T20:36:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060b0 0x7fa9dc1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060b0 0x7fa9dc1977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34424/0 (socket says 172.21.15.46:34424) 2024-03-18T20:36:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9da7fc700 1 -- 172.21.15.46:0/4171320313 learned_addr learned my addr 172.21.15.46:0/4171320313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc19bb40 0x7fa9dc19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9daffd700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069280 0x7fa9dc1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9da7fc700 1 -- 172.21.15.46:0/4171320313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc19bb40 msgr2=0x7fa9dc19bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc19bb40 0x7fa9dc19bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9da7fc700 1 -- 172.21.15.46:0/4171320313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069280 msgr2=0x7fa9dc1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069280 0x7fa9dc1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9da7fc700 1 -- 172.21.15.46:0/4171320313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d0009520 con 0x7fa9dc1060b0 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9daffd700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069280 0x7fa9dc1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc19bb40 0x7fa9dc19bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060b0 0x7fa9dc1977f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009840 tx=0x7fa9d000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.856+0000 7fa9e08d0700 1 -- 172.21.15.46:0/4171320313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d001c070 con 0x7fa9dc1060b0 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.857+0000 7fa9e08d0700 1 -- 172.21.15.46:0/4171320313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9d000bb20 con 0x7fa9dc1060b0 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.857+0000 7fa9e18d2700 1 -- 172.21.15.46:0/4171320313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9dc1a3df0 con 0x7fa9dc1060b0 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.857+0000 7fa9e08d0700 1 -- 172.21.15.46:0/4171320313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00038c0 con 0x7fa9dc1060b0 2024-03-18T20:36:29.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.857+0000 7fa9e18d2700 1 -- 172.21.15.46:0/4171320313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc1a41c0 con 0x7fa9dc1060b0 2024-03-18T20:36:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.859+0000 7fa9e08d0700 1 -- 172.21.15.46:0/4171320313 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 136041+0+0 (secure 0 0 0) 0x7fa9d0020030 con 0x7fa9dc1060b0 2024-03-18T20:36:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.860+0000 7fa9e08d0700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9c40a1320 0x7fa9c40a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.860+0000 7fa9e08d0700 1 -- 172.21.15.46:0/4171320313 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fa9d00cde00 con 0x7fa9dc1060b0 2024-03-18T20:36:29.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.860+0000 7fa9daffd700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9c40a1320 0x7fa9c40a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9daffd700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9c40a1320 0x7fa9c40a37e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009cb0 tx=0x7fa9cc009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9eda67b80 1 Processor -- start 2024-03-18T20:36:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9eda67b80 1 -- start start 2024-03-18T20:36:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9eda67b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a1a60 0x5611779a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9eda67b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a2380 0x5611779b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9eda67b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x5611779b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9eda67b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611778fb990 con 0x5611779a2380 2024-03-18T20:36:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9eda67b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611778f9350 con 0x5611779b1630 2024-03-18T20:36:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9eda67b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611778f4c40 con 0x5611779a1a60 2024-03-18T20:36:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a2380 0x5611779b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a2380 0x5611779b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34430/0 (socket says 172.21.15.46:34430) 2024-03-18T20:36:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9da7fc700 1 -- 172.21.15.46:0/1662070479 learned_addr learned my addr 172.21.15.46:0/1662070479 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.861+0000 7fa9daffd700 1 --2- 172.21.15.46:0/1662070479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a1a60 0x5611779a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.862+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/1662070479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x5611779b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.862+0000 7fa9da7fc700 1 -- 172.21.15.46:0/1662070479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a1a60 msgr2=0x5611779a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.862+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/1662070479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a1a60 0x5611779a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.862+0000 7fa9da7fc700 1 -- 172.21.15.46:0/1662070479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 msgr2=0x5611779b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.862+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/1662070479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x5611779b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.862+0000 7fa9da7fc700 1 -- 172.21.15.46:0/1662070479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611777763b0 con 0x5611779a2380 2024-03-18T20:36:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.862+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/1662070479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x5611779b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.862+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/1662070479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a2380 0x5611779b10f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa9d009ab00 tx=0x7fa9d000bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5b5ece1fefdf8a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.862+0000 7fa9c1ffb700 1 -- 172.21.15.46:0/1662070479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d009b500 con 0x5611779a2380 2024-03-18T20:36:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.862+0000 7fa9c1ffb700 1 -- 172.21.15.46:0/1662070479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9d009ae00 con 0x5611779a2380 2024-03-18T20:36:29.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.862+0000 7fa9c1ffb700 1 -- 172.21.15.46:0/1662070479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00cb710 con 0x5611779a2380 2024-03-18T20:36:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.863+0000 7fa9eda67b80 1 -- 172.21.15.46:0/1662070479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a2380 msgr2=0x5611779b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.863+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/1662070479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a2380 0x5611779b10f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa9d009ab00 tx=0x7fa9d000bc90 comp rx=0 tx=0).stop 2024-03-18T20:36:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.863+0000 7fa9eda67b80 1 -- 172.21.15.46:0/1662070479 shutdown_connections 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.863+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/1662070479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x5611779b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.863+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/1662070479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a1a60 0x5611779a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.863+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/1662070479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a2380 0x5611779b10f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.863+0000 7fa9eda67b80 1 -- 172.21.15.46:0/1662070479 >> 172.21.15.46:0/1662070479 conn(0x561177918220 msgr2=0x56117799e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.863+0000 7fa9eda67b80 1 -- 172.21.15.46:0/1662070479 shutdown_connections 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.863+0000 7fa9eda67b80 1 -- 172.21.15.46:0/1662070479 wait complete. 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9eda67b80 1 -- start start 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9eda67b80 1 Processor -- start 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9eda67b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x5611779b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9eda67b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779a2380 0x5611779b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9eda67b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779b1630 0x5611779bca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9eda67b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561177890d80 con 0x5611779a1a60 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9eda67b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561177890a20 con 0x5611779a2380 2024-03-18T20:36:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9eda67b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611777786a0 con 0x5611779b1630 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9eda67b80 10 client.0 _ll_register_callbacks cb 0x5611776fc3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x5611779b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x5611779b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34434/0 (socket says 172.21.15.46:34434) 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9daffd700 1 -- 172.21.15.46:0/329592452 learned_addr learned my addr 172.21.15.46:0/329592452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.864+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779a2380 0x5611779b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779b1630 0x5611779bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9daffd700 1 -- 172.21.15.46:0/329592452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779b1630 msgr2=0x5611779bca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9daffd700 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779b1630 0x5611779bca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9daffd700 1 -- 172.21.15.46:0/329592452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779a2380 msgr2=0x5611779b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9daffd700 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779a2380 0x5611779b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9daffd700 1 -- 172.21.15.46:0/329592452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc1a41c0 con 0x5611779a1a60 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779b1630 0x5611779bca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9daffd700 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x5611779b1a10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0045c0 tx=0x7fa9cc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cd09e14f565865 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9baffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9baffd700 1 -- 172.21.15.46:0/329592452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc003ea0 con 0x5611779a1a60 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9baffd700 1 -- 172.21.15.46:0/329592452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fa9cc018070 con 0x5611779a1a60 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9eda67b80 1 -- 172.21.15.46:0/329592452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561177995ad0 con 0x5611779a1a60 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9baffd700 1 -- 172.21.15.46:0/329592452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9cc012450 con 0x5611779a1a60 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9eda67b80 10 client.15642 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.865+0000 7fa9eda67b80 1 -- 172.21.15.46:0/329592452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561177722b60 con 0x5611779a1a60 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.866+0000 7fa9baffd700 1 -- 172.21.15.46:0/329592452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561177995ad0 con 0x5611779a1a60 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.866+0000 7fa9baffd700 1 -- 172.21.15.46:0/329592452 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa9cc0216b0 con 0x5611779a1a60 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.866+0000 7fa9eda67b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.866+0000 7fa9eda67b80 10 client.15642 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.866+0000 7fa9eda67b80 20 client.15642 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.866+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5611779c0ef0 0x5611779c3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.866+0000 7fa9eda67b80 4 client.15642 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:36:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.866+0000 7fa9eda67b80 1 -- 172.21.15.46:0/329592452 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611778ed700 con 0x5611779c0ef0 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.866+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5611779c0ef0 0x5611779c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.867+0000 7fa9baffd700 10 client.15642 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.867+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5611779c0ef0 0x5611779c3310 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cdde4cfb43aa1536 server_cookie=56819df3d54ce3dd in_seq=0 out_seq=0 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.867+0000 7fa9baffd700 1 -- 172.21.15.46:0/329592452 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5611778ed700 con 0x5611779c0ef0 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.867+0000 7fa9baffd700 10 client.15642 handle_command_reply: tid=0 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.867+0000 7fa9eda67b80 1 client.15642 shutdown 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.868+0000 7fa9eda67b80 1 -- 172.21.15.46:0/329592452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 msgr2=0x5611779b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.868+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x5611779b1a10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0045c0 tx=0x7fa9cc0047c0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.868+0000 7fa9eda67b80 1 -- 172.21.15.46:0/329592452 shutdown_connections 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.868+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5611779c0ef0 0x5611779c3310 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.868+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x5611779b1a10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.868+0000 7fa9baffd700 0 client.15642 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.868+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779b1630 0x5611779bca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.868+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/329592452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779a2380 0x5611779b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.868+0000 7fa9eda67b80 1 -- 172.21.15.46:0/329592452 >> 172.21.15.46:0/329592452 conn(0x561177918220 msgr2=0x56117799dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.868+0000 7fa9eda67b80 1 -- 172.21.15.46:0/329592452 shutdown_connections 2024-03-18T20:36:29.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.868+0000 7fa9eda67b80 1 -- 172.21.15.46:0/329592452 wait complete. 2024-03-18T20:36:29.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.868+0000 7fa9eda67b80 20 client.15642 trim_cache size 0 max 16384 2024-03-18T20:36:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.882+0000 7fa9eda67b80 1 Processor -- start 2024-03-18T20:36:29.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.882+0000 7fa9eda67b80 1 -- start start 2024-03-18T20:36:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9eda67b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x5611779b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9eda67b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a2380 0x5611779b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9eda67b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x5611779b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9eda67b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611779a35d0 con 0x5611779a1a60 2024-03-18T20:36:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9eda67b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117799a5e0 con 0x5611779b1630 2024-03-18T20:36:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9eda67b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117799f230 con 0x5611779a2380 2024-03-18T20:36:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x5611779b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x5611779b6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34440/0 (socket says 172.21.15.46:34440) 2024-03-18T20:36:29.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9daffd700 1 -- 172.21.15.46:0/2905285739 learned_addr learned my addr 172.21.15.46:0/2905285739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/2905285739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a2380 0x5611779b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/2905285739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x5611779b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9daffd700 1 -- 172.21.15.46:0/2905285739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a2380 msgr2=0x5611779b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9daffd700 1 --2- 172.21.15.46:0/2905285739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a2380 0x5611779b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9daffd700 1 -- 172.21.15.46:0/2905285739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 msgr2=0x5611779b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9daffd700 1 --2- 172.21.15.46:0/2905285739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x5611779b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9daffd700 1 -- 172.21.15.46:0/2905285739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611778baea0 con 0x5611779a1a60 2024-03-18T20:36:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/2905285739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x5611779b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.883+0000 7fa9daffd700 1 --2- 172.21.15.46:0/2905285739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x5611779b6a20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc011ad0 tx=0x7fa9cc009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd20d1053675af67 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.884+0000 7fa9c27fc700 1 -- 172.21.15.46:0/2905285739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc003ea0 con 0x5611779a1a60 2024-03-18T20:36:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.884+0000 7fa9c27fc700 1 -- 172.21.15.46:0/2905285739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9cc009ea0 con 0x5611779a1a60 2024-03-18T20:36:29.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.884+0000 7fa9c27fc700 1 -- 172.21.15.46:0/2905285739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0127b0 con 0x5611779a1a60 2024-03-18T20:36:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.884+0000 7fa9eda67b80 1 -- 172.21.15.46:0/2905285739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 msgr2=0x5611779b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.884+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/2905285739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x5611779b6a20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc011ad0 tx=0x7fa9cc009400 comp rx=0 tx=0).stop 2024-03-18T20:36:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.884+0000 7fa9eda67b80 1 -- 172.21.15.46:0/2905285739 shutdown_connections 2024-03-18T20:36:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.884+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/2905285739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x5611779b6a20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.884+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/2905285739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x5611779b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.884+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/2905285739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a2380 0x5611779b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.884+0000 7fa9eda67b80 1 -- 172.21.15.46:0/2905285739 >> 172.21.15.46:0/2905285739 conn(0x561177918220 msgr2=0x5611779b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.884+0000 7fa9eda67b80 1 -- 172.21.15.46:0/2905285739 shutdown_connections 2024-03-18T20:36:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.884+0000 7fa9eda67b80 1 -- 172.21.15.46:0/2905285739 wait complete. 2024-03-18T20:36:29.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.885+0000 7fa9eda67b80 1 -- start start 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.885+0000 7fa9eda67b80 1 Processor -- start 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.885+0000 7fa9eda67b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x561177a08e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.885+0000 7fa9eda67b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a2380 0x561177a093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.885+0000 7fa9eda67b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x561177a13cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.885+0000 7fa9eda67b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561177a14230 con 0x5611779a1a60 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.885+0000 7fa9eda67b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561177a14370 con 0x5611779b1630 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.885+0000 7fa9eda67b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561177a144b0 con 0x5611779a2380 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.885+0000 7fa9eda67b80 10 client.0 _ll_register_callbacks cb 0x7fa9cc0043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x561177a08e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x561177a08e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34444/0 (socket says 172.21.15.46:34444) 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9daffd700 1 -- 172.21.15.46:0/3297947992 learned_addr learned my addr 172.21.15.46:0/3297947992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x561177a13cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a2380 0x561177a093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9daffd700 1 -- 172.21.15.46:0/3297947992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a2380 msgr2=0x561177a093a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9daffd700 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a2380 0x561177a093a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9daffd700 1 -- 172.21.15.46:0/3297947992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 msgr2=0x561177a13cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9daffd700 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x561177a13cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9daffd700 1 -- 172.21.15.46:0/3297947992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561177722b60 con 0x5611779a1a60 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9daffd700 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x561177a08e60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0096d0 tx=0x7fa9cc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3050466c8b3e95b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9baffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9baffd700 1 -- 172.21.15.46:0/3297947992 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc021820 con 0x5611779a1a60 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9baffd700 1 -- 172.21.15.46:0/3297947992 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fa9cc018070 con 0x5611779a1a60 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.886+0000 7fa9eda67b80 1 -- 172.21.15.46:0/3297947992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9d009ae00 con 0x5611779a1a60 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.887+0000 7fa9baffd700 1 -- 172.21.15.46:0/3297947992 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9cc0099e0 con 0x5611779a1a60 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.887+0000 7fa9eda67b80 10 client.15648 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.887+0000 7fa9eda67b80 1 -- 172.21.15.46:0/3297947992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56117777acd0 con 0x5611779a1a60 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.887+0000 7fa9baffd700 1 -- 172.21.15.46:0/3297947992 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9d009ae00 con 0x5611779a1a60 2024-03-18T20:36:29.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.887+0000 7fa9baffd700 1 -- 172.21.15.46:0/3297947992 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa9cc04f070 con 0x5611779a1a60 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.887+0000 7fa9eda67b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.887+0000 7fa9eda67b80 10 client.15648 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.887+0000 7fa9eda67b80 20 client.15648 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.887+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x561177a157a0 0x561177a17b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.887+0000 7fa9eda67b80 4 client.15648 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.887+0000 7fa9eda67b80 1 -- 172.21.15.46:0/3297947992 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561177743cf0 con 0x561177a157a0 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.887+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x561177a157a0 0x561177a17b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x561177a157a0 0x561177a17b90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1429a2c62254b591 server_cookie=1637861ed5aefa67 in_seq=0 out_seq=0 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9baffd700 10 client.15648 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9baffd700 1 -- 172.21.15.46:0/3297947992 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561177743cf0 con 0x561177a157a0 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9baffd700 10 client.15648 handle_command_reply: tid=0 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9eda67b80 1 client.15648 shutdown 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9eda67b80 1 -- 172.21.15.46:0/3297947992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 msgr2=0x561177a08e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x561177a08e60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0096d0 tx=0x7fa9cc014000 comp rx=0 tx=0).stop 2024-03-18T20:36:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9eda67b80 1 -- 172.21.15.46:0/3297947992 shutdown_connections 2024-03-18T20:36:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x561177a157a0 0x561177a17b90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611779a1a60 0x561177a08e60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611779b1630 0x561177a13cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9baffd700 0 client.15648 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:36:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9eda67b80 1 --2- 172.21.15.46:0/3297947992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611779a2380 0x561177a093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9eda67b80 1 -- 172.21.15.46:0/3297947992 >> 172.21.15.46:0/3297947992 conn(0x561177918220 msgr2=0x5611779b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9eda67b80 1 -- 172.21.15.46:0/3297947992 shutdown_connections 2024-03-18T20:36:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.888+0000 7fa9eda67b80 1 -- 172.21.15.46:0/3297947992 wait complete. 2024-03-18T20:36:29.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9eda67b80 20 client.15648 trim_cache size 0 max 16384 2024-03-18T20:36:29.897 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:36:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9e18d2700 1 -- 172.21.15.46:0/4171320313 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9c40a1320 msgr2=0x7fa9c40a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9e18d2700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9c40a1320 0x7fa9c40a37e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009cb0 tx=0x7fa9cc009260 comp rx=0 tx=0).stop 2024-03-18T20:36:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9e18d2700 1 -- 172.21.15.46:0/4171320313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060b0 msgr2=0x7fa9dc1977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9e18d2700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060b0 0x7fa9dc1977f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009840 tx=0x7fa9d000fa00 comp rx=0 tx=0).stop 2024-03-18T20:36:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9e18d2700 1 -- 172.21.15.46:0/4171320313 shutdown_connections 2024-03-18T20:36:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9e18d2700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9c40a1320 0x7fa9c40a37e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9e18d2700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc19bb40 0x7fa9dc19bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9e18d2700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069280 0x7fa9dc1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9e18d2700 1 --2- 172.21.15.46:0/4171320313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060b0 0x7fa9dc1977f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9e18d2700 1 -- 172.21.15.46:0/4171320313 >> 172.21.15.46:0/4171320313 conn(0x7fa9dc076d80 msgr2=0x7fa9dc0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9e18d2700 1 -- 172.21.15.46:0/4171320313 shutdown_connections 2024-03-18T20:36:29.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:29.889+0000 7fa9e18d2700 1 -- 172.21.15.46:0/4171320313 wait complete. 2024-03-18T20:36:29.906 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:36:29.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:36:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.046+0000 7fad563cb700 1 -- 172.21.15.46:0/90686255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad50105ce0 msgr2=0x7fad501060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.046+0000 7fad563cb700 1 --2- 172.21.15.46:0/90686255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad50105ce0 0x7fad501060c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fad400099c0 tx=0x7fad40009cd0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.046+0000 7fad563cb700 1 -- 172.21.15.46:0/90686255 shutdown_connections 2024-03-18T20:36:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.046+0000 7fad563cb700 1 --2- 172.21.15.46:0/90686255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad500687f0 0x7fad5010db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.046+0000 7fad563cb700 1 --2- 172.21.15.46:0/90686255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad50105ce0 0x7fad501060c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.046+0000 7fad563cb700 1 --2- 172.21.15.46:0/90686255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad50067e30 0x7fad500682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.046+0000 7fad563cb700 1 -- 172.21.15.46:0/90686255 >> 172.21.15.46:0/90686255 conn(0x7fad50076d40 msgr2=0x7fad50077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.046+0000 7fad563cb700 1 -- 172.21.15.46:0/90686255 shutdown_connections 2024-03-18T20:36:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.046+0000 7fad563cb700 1 -- 172.21.15.46:0/90686255 wait complete. 2024-03-18T20:36:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad563cb700 1 Processor -- start 2024-03-18T20:36:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad563cb700 1 -- start start 2024-03-18T20:36:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad563cb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad50067e30 0x7fad50197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad563cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad500687f0 0x7fad50197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad563cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad50105ce0 0x7fad5019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad563cb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5019c0e0 con 0x7fad500687f0 2024-03-18T20:36:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad563cb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5019c250 con 0x7fad50105ce0 2024-03-18T20:36:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad563cb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5019c410 con 0x7fad50067e30 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad4f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad500687f0 0x7fad50197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad4ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad50067e30 0x7fad50197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad4f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad500687f0 0x7fad50197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34450/0 (socket says 172.21.15.46:34450) 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad4f7fe700 1 -- 172.21.15.46:0/2469411497 learned_addr learned my addr 172.21.15.46:0/2469411497 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.047+0000 7fad54968700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad50105ce0 0x7fad5019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad4f7fe700 1 -- 172.21.15.46:0/2469411497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad50067e30 msgr2=0x7fad50197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad4f7fe700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad50067e30 0x7fad50197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad4f7fe700 1 -- 172.21.15.46:0/2469411497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad50105ce0 msgr2=0x7fad5019baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad4f7fe700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad50105ce0 0x7fad5019baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad4f7fe700 1 -- 172.21.15.46:0/2469411497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad400095f0 con 0x7fad500687f0 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad4ffff700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad50067e30 0x7fad50197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad54968700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad50105ce0 0x7fad5019baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad4f7fe700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad500687f0 0x7fad50197750 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fad4400d950 tx=0x7fad4400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad4d7fa700 1 -- 172.21.15.46:0/2469411497 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400f890 con 0x7fad500687f0 2024-03-18T20:36:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad4d7fa700 1 -- 172.21.15.46:0/2469411497 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad44010460 con 0x7fad500687f0 2024-03-18T20:36:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad4d7fa700 1 -- 172.21.15.46:0/2469411497 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400e610 con 0x7fad500687f0 2024-03-18T20:36:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad563cb700 1 -- 172.21.15.46:0/2469411497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5019c6f0 con 0x7fad500687f0 2024-03-18T20:36:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.048+0000 7fad563cb700 1 -- 172.21.15.46:0/2469411497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad500ffd80 con 0x7fad500687f0 2024-03-18T20:36:30.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.051+0000 7fad4d7fa700 1 -- 172.21.15.46:0/2469411497 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 136041+0+0 (secure 0 0 0) 0x7fad4400e770 con 0x7fad500687f0 2024-03-18T20:36:30.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.051+0000 7fad4d7fa700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fad380a13a0 0x7fad380a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.051+0000 7fad4d7fa700 1 -- 172.21.15.46:0/2469411497 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fad440ccc10 con 0x7fad500687f0 2024-03-18T20:36:30.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.051+0000 7fad4ffff700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fad380a13a0 0x7fad380a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.052+0000 7fad4ffff700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fad380a13a0 0x7fad380a3860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fad40009fd0 tx=0x7fad40005fb0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.052+0000 7fad62560b80 1 Processor -- start 2024-03-18T20:36:30.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.052+0000 7fad62560b80 1 -- start start 2024-03-18T20:36:30.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad62560b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 0x56198d327e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad62560b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d328380 0x56198d332ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad62560b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 0x56198d335610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad62560b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198d281990 con 0x56198d333220 2024-03-18T20:36:30.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad62560b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198d27f350 con 0x56198d328380 2024-03-18T20:36:30.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad62560b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198d27ac40 con 0x56198d327a60 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad54968700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 0x56198d335610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad54968700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 0x56198d335610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34452/0 (socket says 172.21.15.46:34452) 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad54968700 1 -- 172.21.15.46:0/1391579694 learned_addr learned my addr 172.21.15.46:0/1391579694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad4ffff700 1 --2- 172.21.15.46:0/1391579694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 0x56198d327e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad4f7fe700 1 --2- 172.21.15.46:0/1391579694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d328380 0x56198d332ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad54968700 1 -- 172.21.15.46:0/1391579694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 msgr2=0x56198d327e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad54968700 1 --2- 172.21.15.46:0/1391579694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 0x56198d327e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad54968700 1 -- 172.21.15.46:0/1391579694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d328380 msgr2=0x56198d332ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad54968700 1 --2- 172.21.15.46:0/1391579694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d328380 0x56198d332ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad54968700 1 -- 172.21.15.46:0/1391579694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56198d0fc3b0 con 0x56198d333220 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.053+0000 7fad54968700 1 --2- 172.21.15.46:0/1391579694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 0x56198d335610 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fad48003eb0 tx=0x7fad48009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7907d9820095f59 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.054+0000 7fad367fc700 1 -- 172.21.15.46:0/1391579694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480092d0 con 0x56198d333220 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.054+0000 7fad367fc700 1 -- 172.21.15.46:0/1391579694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad4800f040 con 0x56198d333220 2024-03-18T20:36:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.054+0000 7fad367fc700 1 -- 172.21.15.46:0/1391579694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4800e800 con 0x56198d333220 2024-03-18T20:36:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.054+0000 7fad62560b80 1 -- 172.21.15.46:0/1391579694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 msgr2=0x56198d335610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.054+0000 7fad62560b80 1 --2- 172.21.15.46:0/1391579694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 0x56198d335610 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fad48003eb0 tx=0x7fad48009920 comp rx=0 tx=0).stop 2024-03-18T20:36:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.054+0000 7fad62560b80 1 -- 172.21.15.46:0/1391579694 shutdown_connections 2024-03-18T20:36:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.055+0000 7fad62560b80 1 --2- 172.21.15.46:0/1391579694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 0x56198d335610 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.055+0000 7fad62560b80 1 --2- 172.21.15.46:0/1391579694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 0x56198d327e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.055+0000 7fad62560b80 1 --2- 172.21.15.46:0/1391579694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d328380 0x56198d332ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.055+0000 7fad62560b80 1 -- 172.21.15.46:0/1391579694 >> 172.21.15.46:0/1391579694 conn(0x56198d29e220 msgr2=0x56198d324020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.055+0000 7fad62560b80 1 -- 172.21.15.46:0/1391579694 shutdown_connections 2024-03-18T20:36:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.055+0000 7fad62560b80 1 -- 172.21.15.46:0/1391579694 wait complete. 2024-03-18T20:36:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.055+0000 7fad62560b80 1 -- start start 2024-03-18T20:36:30.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.055+0000 7fad62560b80 1 Processor -- start 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad62560b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 0x56198d332c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad62560b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d328380 0x56198d32ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad62560b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d333220 0x56198d3305e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad62560b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198d216d80 con 0x56198d328380 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad62560b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198d216a20 con 0x56198d333220 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad62560b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198d0fe6a0 con 0x56198d327a60 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad62560b80 10 client.0 _ll_register_callbacks cb 0x56198d0823a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad4f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d328380 0x56198d32ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad4f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d328380 0x56198d32ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34462/0 (socket says 172.21.15.46:34462) 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad4f7fe700 1 -- 172.21.15.46:0/573183011 learned_addr learned my addr 172.21.15.46:0/573183011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad4ffff700 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 0x56198d332c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad54968700 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d333220 0x56198d3305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad4f7fe700 1 -- 172.21.15.46:0/573183011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 msgr2=0x56198d332c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad4f7fe700 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 0x56198d332c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad4f7fe700 1 -- 172.21.15.46:0/573183011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d333220 msgr2=0x56198d3305e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad4f7fe700 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d333220 0x56198d3305e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad4f7fe700 1 -- 172.21.15.46:0/573183011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad48003b90 con 0x56198d328380 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.056+0000 7fad54968700 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d333220 0x56198d3305e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.057+0000 7fad4f7fe700 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d328380 0x56198d32ff60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fad4400b2f0 tx=0x7fad44099bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5928edbdbb0f42 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.057+0000 7fad2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.057+0000 7fad2f7fe700 1 -- 172.21.15.46:0/573183011 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad440ca760 con 0x56198d328380 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.057+0000 7fad62560b80 1 -- 172.21.15.46:0/573183011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56198d31bad0 con 0x56198d328380 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.057+0000 7fad2f7fe700 1 -- 172.21.15.46:0/573183011 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fad440a2040 con 0x56198d328380 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.057+0000 7fad2f7fe700 1 -- 172.21.15.46:0/573183011 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad44099e20 con 0x56198d328380 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.057+0000 7fad62560b80 10 client.15660 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.057+0000 7fad62560b80 1 -- 172.21.15.46:0/573183011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56198d0a8b60 con 0x56198d328380 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.057+0000 7fad2f7fe700 1 -- 172.21.15.46:0/573183011 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad4407d020 con 0x56198d328380 2024-03-18T20:36:30.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.057+0000 7fad2f7fe700 1 -- 172.21.15.46:0/573183011 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fad4409abd0 con 0x56198d328380 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.057+0000 7fad62560b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.058+0000 7fad62560b80 10 client.15660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.058+0000 7fad62560b80 20 client.15660 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.058+0000 7fad62560b80 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56198d342cf0 0x56198d345110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.058+0000 7fad62560b80 4 client.15660 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.058+0000 7fad62560b80 1 -- 172.21.15.46:0/573183011 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56198d273700 con 0x56198d342cf0 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.058+0000 7fad4ffff700 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56198d342cf0 0x56198d345110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.058+0000 7fad2f7fe700 10 client.15660 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.058+0000 7fad4ffff700 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56198d342cf0 0x56198d345110 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e500997380cb3a2 server_cookie=b52698111cfb6eeb in_seq=0 out_seq=0 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad2f7fe700 1 -- 172.21.15.46:0/573183011 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56198d273700 con 0x56198d342cf0 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad2f7fe700 10 client.15660 handle_command_reply: tid=0 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad62560b80 1 client.15660 shutdown 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad62560b80 1 -- 172.21.15.46:0/573183011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d328380 msgr2=0x56198d32ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad62560b80 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d328380 0x56198d32ff60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fad4400b2f0 tx=0x7fad44099bc0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad62560b80 1 -- 172.21.15.46:0/573183011 shutdown_connections 2024-03-18T20:36:30.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad62560b80 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d333220 0x56198d3305e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad62560b80 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 0x56198d332c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad62560b80 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56198d342cf0 0x56198d345110 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad62560b80 1 --2- 172.21.15.46:0/573183011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d328380 0x56198d32ff60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad62560b80 1 -- 172.21.15.46:0/573183011 >> 172.21.15.46:0/573183011 conn(0x56198d29e220 msgr2=0x56198d323c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad2f7fe700 0 client.15660 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:36:30.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad62560b80 1 -- 172.21.15.46:0/573183011 shutdown_connections 2024-03-18T20:36:30.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.059+0000 7fad62560b80 1 -- 172.21.15.46:0/573183011 wait complete. 2024-03-18T20:36:30.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.060+0000 7fad62560b80 20 client.15660 trim_cache size 0 max 16384 2024-03-18T20:36:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad62560b80 1 Processor -- start 2024-03-18T20:36:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad62560b80 1 -- start start 2024-03-18T20:36:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad62560b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 0x56198d339e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad62560b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d328380 0x56198d32b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad62560b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 0x56198d32b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad62560b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198d2ee940 con 0x56198d333220 2024-03-18T20:36:30.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad62560b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198d3205e0 con 0x56198d328380 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad62560b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198d31e210 con 0x56198d327a60 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad54968700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 0x56198d32b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad54968700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 0x56198d32b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34470/0 (socket says 172.21.15.46:34470) 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad54968700 1 -- 172.21.15.46:0/889352481 learned_addr learned my addr 172.21.15.46:0/889352481 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad4f7fe700 1 --2- 172.21.15.46:0/889352481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d328380 0x56198d32b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad4ffff700 1 --2- 172.21.15.46:0/889352481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 0x56198d339e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad54968700 1 -- 172.21.15.46:0/889352481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 msgr2=0x56198d339e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad54968700 1 --2- 172.21.15.46:0/889352481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 0x56198d339e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad54968700 1 -- 172.21.15.46:0/889352481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d328380 msgr2=0x56198d32b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad54968700 1 --2- 172.21.15.46:0/889352481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d328380 0x56198d32b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.075+0000 7fad54968700 1 -- 172.21.15.46:0/889352481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56198d240ea0 con 0x56198d333220 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.076+0000 7fad54968700 1 --2- 172.21.15.46:0/889352481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 0x56198d32b7c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fad480060a0 tx=0x7fad48003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d510ff4a49905614 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.076+0000 7fad36ffd700 1 -- 172.21.15.46:0/889352481 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48023070 con 0x56198d333220 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.076+0000 7fad36ffd700 1 -- 172.21.15.46:0/889352481 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad48004e60 con 0x56198d333220 2024-03-18T20:36:30.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.076+0000 7fad36ffd700 1 -- 172.21.15.46:0/889352481 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4800ece0 con 0x56198d333220 2024-03-18T20:36:30.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.077+0000 7fad62560b80 1 -- 172.21.15.46:0/889352481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 msgr2=0x56198d32b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.077+0000 7fad62560b80 1 --2- 172.21.15.46:0/889352481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 0x56198d32b7c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fad480060a0 tx=0x7fad48003ab0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.077+0000 7fad62560b80 1 -- 172.21.15.46:0/889352481 shutdown_connections 2024-03-18T20:36:30.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.077+0000 7fad62560b80 1 --2- 172.21.15.46:0/889352481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d333220 0x56198d32b7c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.077+0000 7fad62560b80 1 --2- 172.21.15.46:0/889352481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d327a60 0x56198d339e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.077+0000 7fad62560b80 1 --2- 172.21.15.46:0/889352481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d328380 0x56198d32b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.077+0000 7fad62560b80 1 -- 172.21.15.46:0/889352481 >> 172.21.15.46:0/889352481 conn(0x56198d29e220 msgr2=0x56198d32f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.077+0000 7fad62560b80 1 -- 172.21.15.46:0/889352481 shutdown_connections 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.077+0000 7fad62560b80 1 -- 172.21.15.46:0/889352481 wait complete. 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.077+0000 7fad62560b80 1 -- start start 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad62560b80 1 Processor -- start 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad62560b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d327a60 0x56198d338140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad62560b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d328380 0x56198d338680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad62560b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d333220 0x56198d339560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad62560b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198d339b40 con 0x56198d327a60 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad62560b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198d339cb0 con 0x56198d333220 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad62560b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198d339e70 con 0x56198d328380 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad62560b80 10 client.0 _ll_register_callbacks cb 0x7fad440997d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad4ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d327a60 0x56198d338140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad4ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d327a60 0x56198d338140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34480/0 (socket says 172.21.15.46:34480) 2024-03-18T20:36:30.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad4ffff700 1 -- 172.21.15.46:0/2518849193 learned_addr learned my addr 172.21.15.46:0/2518849193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad4f7fe700 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d328380 0x56198d338680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad54968700 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d333220 0x56198d339560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad4ffff700 1 -- 172.21.15.46:0/2518849193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d328380 msgr2=0x56198d338680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad4ffff700 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d328380 0x56198d338680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad4ffff700 1 -- 172.21.15.46:0/2518849193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d333220 msgr2=0x56198d339560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad4ffff700 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d333220 0x56198d339560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad4ffff700 1 -- 172.21.15.46:0/2518849193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56198d0fc3b0 con 0x56198d327a60 2024-03-18T20:36:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.078+0000 7fad4f7fe700 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d328380 0x56198d338680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.079+0000 7fad4ffff700 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d327a60 0x56198d338140 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fad40005cd0 tx=0x7fad40009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b26e4b4807c0e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.079+0000 7fad2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:36:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad2f7fe700 1 -- 172.21.15.46:0/2518849193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4002e040 con 0x56198d327a60 2024-03-18T20:36:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad2f7fe700 1 -- 172.21.15.46:0/2518849193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fad40038c70 con 0x56198d327a60 2024-03-18T20:36:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad62560b80 1 -- 172.21.15.46:0/2518849193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad4800f040 con 0x56198d327a60 2024-03-18T20:36:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad2f7fe700 1 -- 172.21.15.46:0/2518849193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad40038e30 con 0x56198d327a60 2024-03-18T20:36:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad62560b80 10 client.15666 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad62560b80 1 -- 172.21.15.46:0/2518849193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56198d100cd0 con 0x56198d327a60 2024-03-18T20:36:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad2f7fe700 1 -- 172.21.15.46:0/2518849193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad4800f040 con 0x56198d327a60 2024-03-18T20:36:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad2f7fe700 1 -- 172.21.15.46:0/2518849193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fad40047070 con 0x56198d327a60 2024-03-18T20:36:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad62560b80 10 client.15666 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad62560b80 10 client.15666 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:36:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad62560b80 20 client.15666 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad62560b80 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56198d397cb0 0x56198d39a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad62560b80 4 client.15666 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.082+0000 7fad62560b80 1 -- 172.21.15.46:0/2518849193 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56198d0c9cf0 con 0x56198d397cb0 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.083+0000 7fad4f7fe700 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56198d397cb0 0x56198d39a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.083+0000 7fad4f7fe700 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56198d397cb0 0x56198d39a0d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=615b1536c030b15c server_cookie=5bc8e9a757a5a5aa in_seq=0 out_seq=0 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.083+0000 7fad2f7fe700 10 client.15666 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.083+0000 7fad2f7fe700 1 -- 172.21.15.46:0/2518849193 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56198d0c9cf0 con 0x56198d397cb0 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.083+0000 7fad2f7fe700 10 client.15666 handle_command_reply: tid=0 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.083+0000 7fad62560b80 1 client.15666 shutdown 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.083+0000 7fad62560b80 1 -- 172.21.15.46:0/2518849193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d327a60 msgr2=0x56198d338140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.083+0000 7fad62560b80 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d327a60 0x56198d338140 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fad40005cd0 tx=0x7fad40009cd0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.083+0000 7fad62560b80 1 -- 172.21.15.46:0/2518849193 shutdown_connections 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.083+0000 7fad62560b80 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56198d397cb0 0x56198d39a0d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad62560b80 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56198d327a60 0x56198d338140 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad62560b80 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56198d333220 0x56198d339560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad2f7fe700 0 client.15666 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:36:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad62560b80 1 --2- 172.21.15.46:0/2518849193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56198d328380 0x56198d338680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad62560b80 1 -- 172.21.15.46:0/2518849193 >> 172.21.15.46:0/2518849193 conn(0x56198d29e220 msgr2=0x56198d32b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad62560b80 1 -- 172.21.15.46:0/2518849193 shutdown_connections 2024-03-18T20:36:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad62560b80 1 -- 172.21.15.46:0/2518849193 wait complete. 2024-03-18T20:36:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad62560b80 20 client.15666 trim_cache size 0 max 16384 2024-03-18T20:36:30.092 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:36:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad563cb700 1 -- 172.21.15.46:0/2469411497 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fad380a13a0 msgr2=0x7fad380a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad563cb700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fad380a13a0 0x7fad380a3860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fad40009fd0 tx=0x7fad40005fb0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad563cb700 1 -- 172.21.15.46:0/2469411497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad500687f0 msgr2=0x7fad50197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad563cb700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad500687f0 0x7fad50197750 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fad4400d950 tx=0x7fad4400dc60 comp rx=0 tx=0).stop 2024-03-18T20:36:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad563cb700 1 -- 172.21.15.46:0/2469411497 shutdown_connections 2024-03-18T20:36:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad563cb700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fad380a13a0 0x7fad380a3860 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad563cb700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad50105ce0 0x7fad5019baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad563cb700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad50067e30 0x7fad50197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad563cb700 1 --2- 172.21.15.46:0/2469411497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad500687f0 0x7fad50197750 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.084+0000 7fad563cb700 1 -- 172.21.15.46:0/2469411497 >> 172.21.15.46:0/2469411497 conn(0x7fad50076d40 msgr2=0x7fad500fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.085+0000 7fad563cb700 1 -- 172.21.15.46:0/2469411497 shutdown_connections 2024-03-18T20:36:30.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.085+0000 7fad563cb700 1 -- 172.21.15.46:0/2469411497 wait complete. 2024-03-18T20:36:30.102 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:36:30.102 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:36:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.252+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/673559049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd580687f0 msgr2=0x7fbd5810db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.252+0000 7fbd5dcc9700 1 --2- 172.21.15.46:0/673559049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd580687f0 0x7fbd5810db50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000b0a0 tx=0x7fbd5000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.252+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/673559049 shutdown_connections 2024-03-18T20:36:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.252+0000 7fbd5dcc9700 1 --2- 172.21.15.46:0/673559049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd580687f0 0x7fbd5810db50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.252+0000 7fbd5dcc9700 1 --2- 172.21.15.46:0/673559049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd58105ce0 0x7fbd581060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.252+0000 7fbd5dcc9700 1 --2- 172.21.15.46:0/673559049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58067e30 0x7fbd580682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.252+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/673559049 >> 172.21.15.46:0/673559049 conn(0x7fbd58076d40 msgr2=0x7fbd58077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.252+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/673559049 shutdown_connections 2024-03-18T20:36:30.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.252+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/673559049 wait complete. 2024-03-18T20:36:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.253+0000 7fbd5dcc9700 1 Processor -- start 2024-03-18T20:36:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.253+0000 7fbd5dcc9700 1 -- start start 2024-03-18T20:36:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.253+0000 7fbd5dcc9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58067e30 0x7fbd58197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.253+0000 7fbd5dcc9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd580687f0 0x7fbd58197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.253+0000 7fbd5dcc9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd58105ce0 0x7fbd5819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.253+0000 7fbd5dcc9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5819c220 con 0x7fbd58105ce0 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.253+0000 7fbd5dcc9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5819c390 con 0x7fbd580687f0 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.253+0000 7fbd5dcc9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5819c550 con 0x7fbd58067e30 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.253+0000 7fbd57fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd58105ce0 0x7fbd5819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.253+0000 7fbd57fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd58105ce0 0x7fbd5819bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34496/0 (socket says 172.21.15.46:34496) 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.253+0000 7fbd57fff700 1 -- 172.21.15.46:0/2083508419 learned_addr learned my addr 172.21.15.46:0/2083508419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd56ffd700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd580687f0 0x7fbd58197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd577fe700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58067e30 0x7fbd58197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd57fff700 1 -- 172.21.15.46:0/2083508419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58067e30 msgr2=0x7fbd58197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd57fff700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58067e30 0x7fbd58197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd57fff700 1 -- 172.21.15.46:0/2083508419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd580687f0 msgr2=0x7fbd58197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd57fff700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd580687f0 0x7fbd58197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd57fff700 1 -- 172.21.15.46:0/2083508419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd50009d00 con 0x7fbd58105ce0 2024-03-18T20:36:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd577fe700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58067e30 0x7fbd58197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd57fff700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd58105ce0 0x7fbd5819bbe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000b070 tx=0x7fbd5000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd54ff9700 1 -- 172.21.15.46:0/2083508419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5000e040 con 0x7fbd58105ce0 2024-03-18T20:36:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/2083508419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5819c7d0 con 0x7fbd58105ce0 2024-03-18T20:36:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd54ff9700 1 -- 172.21.15.46:0/2083508419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd50007980 con 0x7fbd58105ce0 2024-03-18T20:36:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd54ff9700 1 -- 172.21.15.46:0/2083508419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd500052a0 con 0x7fbd58105ce0 2024-03-18T20:36:30.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.254+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/2083508419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd581a4070 con 0x7fbd58105ce0 2024-03-18T20:36:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.257+0000 7fbd54ff9700 1 -- 172.21.15.46:0/2083508419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 136041+0+0 (secure 0 0 0) 0x7fbd5001d470 con 0x7fbd58105ce0 2024-03-18T20:36:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.257+0000 7fbd54ff9700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbd400a1320 0x7fbd400a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.257+0000 7fbd577fe700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbd400a1320 0x7fbd400a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.257+0000 7fbd54ff9700 1 -- 172.21.15.46:0/2083508419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fbd500ce2c0 con 0x7fbd58105ce0 2024-03-18T20:36:30.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.258+0000 7fbd577fe700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbd400a1320 0x7fbd400a37e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009cb0 tx=0x7fbd48009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.259+0000 7fbd69e5eb80 1 Processor -- start 2024-03-18T20:36:30.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd69e5eb80 1 -- start start 2024-03-18T20:36:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd69e5eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 0x560587597e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd69e5eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 0x5605875a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd69e5eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd69e5eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605874f1990 con 0x560587598380 2024-03-18T20:36:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd69e5eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605874ef350 con 0x5605875a3220 2024-03-18T20:36:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd69e5eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605874eac40 con 0x560587597a60 2024-03-18T20:36:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd56ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 0x5605875a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd56ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 0x5605875a2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34500/0 (socket says 172.21.15.46:34500) 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd56ffd700 1 -- 172.21.15.46:0/1915126486 learned_addr learned my addr 172.21.15.46:0/1915126486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd57fff700 1 --2- 172.21.15.46:0/1915126486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd577fe700 1 --2- 172.21.15.46:0/1915126486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 0x560587597e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd56ffd700 1 -- 172.21.15.46:0/1915126486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 msgr2=0x560587597e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd56ffd700 1 --2- 172.21.15.46:0/1915126486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 0x560587597e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd56ffd700 1 -- 172.21.15.46:0/1915126486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 msgr2=0x5605875a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd56ffd700 1 --2- 172.21.15.46:0/1915126486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.260+0000 7fbd56ffd700 1 -- 172.21.15.46:0/1915126486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56058736c3b0 con 0x560587598380 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.261+0000 7fbd56ffd700 1 --2- 172.21.15.46:0/1915126486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 0x5605875a2ce0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00bd30 tx=0x7fbd4c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e59777f0e1e174 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.261+0000 7fbd3dffb700 1 -- 172.21.15.46:0/1915126486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c010e10 con 0x560587598380 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.261+0000 7fbd3dffb700 1 -- 172.21.15.46:0/1915126486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd4c015070 con 0x560587598380 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.261+0000 7fbd3dffb700 1 -- 172.21.15.46:0/1915126486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c003e00 con 0x560587598380 2024-03-18T20:36:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.262+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1915126486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 msgr2=0x5605875a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.262+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1915126486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 0x5605875a2ce0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00bd30 tx=0x7fbd4c0106b0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.262+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1915126486 shutdown_connections 2024-03-18T20:36:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.262+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1915126486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.262+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1915126486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 0x560587597e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.262+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1915126486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 0x5605875a2ce0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.262+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1915126486 >> 172.21.15.46:0/1915126486 conn(0x56058750e220 msgr2=0x560587594020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.262+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1915126486 shutdown_connections 2024-03-18T20:36:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.262+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1915126486 wait complete. 2024-03-18T20:36:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.262+0000 7fbd69e5eb80 1 -- start start 2024-03-18T20:36:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.262+0000 7fbd69e5eb80 1 Processor -- start 2024-03-18T20:36:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd69e5eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 0x5605875a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd69e5eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 0x56058759ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd69e5eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd69e5eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560587486d80 con 0x560587598380 2024-03-18T20:36:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd69e5eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560587486a20 con 0x5605875a3220 2024-03-18T20:36:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd69e5eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56058736e6a0 con 0x560587597a60 2024-03-18T20:36:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd69e5eb80 10 client.0 _ll_register_callbacks cb 0x5605872f23a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd56ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 0x56058759ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd56ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 0x56058759ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34506/0 (socket says 172.21.15.46:34506) 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd56ffd700 1 -- 172.21.15.46:0/4219031970 learned_addr learned my addr 172.21.15.46:0/4219031970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd57fff700 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd577fe700 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 0x5605875a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd57fff700 1 -- 172.21.15.46:0/4219031970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 msgr2=0x5605875a2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd57fff700 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 0x5605875a2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd57fff700 1 -- 172.21.15.46:0/4219031970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 msgr2=0x56058759ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd57fff700 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 0x56058759ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd57fff700 1 -- 172.21.15.46:0/4219031970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c00b960 con 0x5605875a3220 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.263+0000 7fbd56ffd700 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 0x56058759ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd57fff700 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a06b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd500cb960 tx=0x7fbd5000b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be07a7e95925d61c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd36ffd700 1 -- 172.21.15.46:0/4219031970 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5009ba00 con 0x5605875a3220 2024-03-18T20:36:30.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd36ffd700 1 -- 172.21.15.46:0/4219031970 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fbd5009b300 con 0x5605875a3220 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/4219031970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56058758bad0 con 0x5605875a3220 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd36ffd700 1 -- 172.21.15.46:0/4219031970 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd5009f130 con 0x5605875a3220 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd69e5eb80 10 client.25448 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/4219031970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560587318b60 con 0x5605875a3220 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd36ffd700 1 -- 172.21.15.46:0/4219031970 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56058758bad0 con 0x5605875a3220 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd36ffd700 1 -- 172.21.15.46:0/4219031970 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbd5009a8f0 con 0x5605875a3220 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd69e5eb80 10 client.25448 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd69e5eb80 10 client.25448 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd69e5eb80 20 client.25448 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5605875b2b50 0x5605875b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd69e5eb80 4 client.25448 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.264+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/4219031970 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605874e3700 con 0x5605875b2b50 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.265+0000 7fbd577fe700 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5605875b2b50 0x5605875b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.265+0000 7fbd36ffd700 10 client.25448 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.265+0000 7fbd577fe700 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5605875b2b50 0x5605875b4f70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9105f1347627f6fa server_cookie=e77ba03a2cbdbe58 in_seq=0 out_seq=0 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd36ffd700 1 -- 172.21.15.46:0/4219031970 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5605874e3700 con 0x5605875b2b50 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd36ffd700 10 client.25448 handle_command_reply: tid=0 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd69e5eb80 1 client.25448 shutdown 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/4219031970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 msgr2=0x5605875a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a06b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd500cb960 tx=0x7fbd5000b930 comp rx=0 tx=0).stop 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/4219031970 shutdown_connections 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5605875b2b50 0x5605875b4f70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a06b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 0x5605875a2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd36ffd700 0 client.25448 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/4219031970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587598380 0x56058759ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/4219031970 >> 172.21.15.46:0/4219031970 conn(0x56058750e220 msgr2=0x560587593c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/4219031970 shutdown_connections 2024-03-18T20:36:30.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/4219031970 wait complete. 2024-03-18T20:36:30.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.266+0000 7fbd69e5eb80 20 client.25448 trim_cache size 0 max 16384 2024-03-18T20:36:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd69e5eb80 1 Processor -- start 2024-03-18T20:36:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd69e5eb80 1 -- start start 2024-03-18T20:36:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd69e5eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 0x5605875b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd69e5eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560587598380 0x5605875a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd69e5eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605875a3220 0x5605875a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd69e5eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56058755e940 con 0x5605875a3220 2024-03-18T20:36:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd69e5eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56058758df80 con 0x560587598380 2024-03-18T20:36:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd69e5eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560587595230 con 0x560587597a60 2024-03-18T20:36:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd57fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605875a3220 0x5605875a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd57fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605875a3220 0x5605875a28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34520/0 (socket says 172.21.15.46:34520) 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd57fff700 1 -- 172.21.15.46:0/1314733084 learned_addr learned my addr 172.21.15.46:0/1314733084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd577fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 0x5605875b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd56ffd700 1 --2- 172.21.15.46:0/1314733084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560587598380 0x5605875a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd57fff700 1 -- 172.21.15.46:0/1314733084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 msgr2=0x5605875b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd57fff700 1 --2- 172.21.15.46:0/1314733084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 0x5605875b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd57fff700 1 -- 172.21.15.46:0/1314733084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560587598380 msgr2=0x5605875a9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd57fff700 1 --2- 172.21.15.46:0/1314733084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560587598380 0x5605875a9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd57fff700 1 -- 172.21.15.46:0/1314733084 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605875af300 con 0x5605875a3220 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.281+0000 7fbd56ffd700 1 --2- 172.21.15.46:0/1314733084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560587598380 0x5605875a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.282+0000 7fbd57fff700 1 --2- 172.21.15.46:0/1314733084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605875a3220 0x5605875a28b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbd5009b000 tx=0x7fbd500cb990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e284a6fb5bde97f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.282+0000 7fbd3e7fc700 1 -- 172.21.15.46:0/1314733084 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5009ba00 con 0x5605875a3220 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.282+0000 7fbd3e7fc700 1 -- 172.21.15.46:0/1314733084 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd500b0070 con 0x5605875a3220 2024-03-18T20:36:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.282+0000 7fbd3e7fc700 1 -- 172.21.15.46:0/1314733084 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5009aa50 con 0x5605875a3220 2024-03-18T20:36:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.283+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1314733084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605875a3220 msgr2=0x5605875a28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.283+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1314733084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605875a3220 0x5605875a28b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbd5009b000 tx=0x7fbd500cb990 comp rx=0 tx=0).stop 2024-03-18T20:36:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.283+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1314733084 shutdown_connections 2024-03-18T20:36:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.283+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1314733084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605875a3220 0x5605875a28b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.283+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1314733084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587597a60 0x5605875b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.283+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1314733084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560587598380 0x5605875a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.283+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1314733084 >> 172.21.15.46:0/1314733084 conn(0x56058750e220 msgr2=0x560587592750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.283+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1314733084 shutdown_connections 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.283+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1314733084 wait complete. 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.283+0000 7fbd69e5eb80 1 -- start start 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.283+0000 7fbd69e5eb80 1 Processor -- start 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd69e5eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587597a60 0x5605875a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd69e5eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587598380 0x5605875a8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd69e5eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd69e5eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605875aa1e0 con 0x560587597a60 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd69e5eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560587605980 con 0x5605875a3220 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd69e5eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560587605b10 con 0x560587598380 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd69e5eb80 10 client.0 _ll_register_callbacks cb 0x7fbd5009b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd577fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587597a60 0x5605875a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd577fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587597a60 0x5605875a7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34522/0 (socket says 172.21.15.46:34522) 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd577fe700 1 -- 172.21.15.46:0/1580208205 learned_addr learned my addr 172.21.15.46:0/1580208205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd57fff700 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd56ffd700 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587598380 0x5605875a8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd577fe700 1 -- 172.21.15.46:0/1580208205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587598380 msgr2=0x5605875a8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd577fe700 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587598380 0x5605875a8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd577fe700 1 -- 172.21.15.46:0/1580208205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 msgr2=0x5605875a88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd577fe700 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd577fe700 1 -- 172.21.15.46:0/1580208205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560587318b60 con 0x560587597a60 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd57fff700 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd577fe700 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587597a60 0x5605875a7e30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009810 tx=0x7fbd480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2da6e3979b02a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.284+0000 7fbd36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd36ffd700 1 -- 172.21.15.46:0/1580208205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd480289a0 con 0x560587597a60 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1580208205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd4c015070 con 0x560587597a60 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd36ffd700 1 -- 172.21.15.46:0/1580208205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fbd48018040 con 0x560587597a60 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd36ffd700 1 -- 172.21.15.46:0/1580208205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd48004590 con 0x560587597a60 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd69e5eb80 10 client.15684 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1580208205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5605874b0ea0 con 0x560587597a60 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd36ffd700 1 -- 172.21.15.46:0/1580208205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd4c015070 con 0x560587597a60 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd36ffd700 1 -- 172.21.15.46:0/1580208205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbd48012450 con 0x560587597a60 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd69e5eb80 10 client.15684 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd69e5eb80 10 client.15684 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd69e5eb80 20 client.15684 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x560587607360 0x560587609780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd69e5eb80 4 client.15684 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1580208205 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560587339cf0 con 0x560587607360 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.285+0000 7fbd56ffd700 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x560587607360 0x560587609780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd36ffd700 10 client.15684 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd56ffd700 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x560587607360 0x560587609780 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47657bd111b30d3f server_cookie=b1cfdb0d432ffdfc in_seq=0 out_seq=0 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd36ffd700 1 -- 172.21.15.46:0/1580208205 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560587339cf0 con 0x560587607360 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd36ffd700 10 client.15684 handle_command_reply: tid=0 2024-03-18T20:36:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd69e5eb80 1 client.15684 shutdown 2024-03-18T20:36:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1580208205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587597a60 msgr2=0x5605875a7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587597a60 0x5605875a7e30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009810 tx=0x7fbd480047c0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1580208205 shutdown_connections 2024-03-18T20:36:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x560587607360 0x560587609780 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560587597a60 0x5605875a7e30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605875a3220 0x5605875a88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd36ffd700 0 client.15684 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:36:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd69e5eb80 1 --2- 172.21.15.46:0/1580208205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560587598380 0x5605875a8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1580208205 >> 172.21.15.46:0/1580208205 conn(0x56058750e220 msgr2=0x56058759b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.286+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1580208205 shutdown_connections 2024-03-18T20:36:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd69e5eb80 1 -- 172.21.15.46:0/1580208205 wait complete. 2024-03-18T20:36:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd69e5eb80 20 client.15684 trim_cache size 0 max 16384 2024-03-18T20:36:30.296 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:36:30.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/2083508419 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbd400a1320 msgr2=0x7fbd400a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd5dcc9700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbd400a1320 0x7fbd400a37e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009cb0 tx=0x7fbd48009260 comp rx=0 tx=0).stop 2024-03-18T20:36:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/2083508419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd58105ce0 msgr2=0x7fbd5819bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd5dcc9700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd58105ce0 0x7fbd5819bbe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000b070 tx=0x7fbd5000bea0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/2083508419 shutdown_connections 2024-03-18T20:36:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd5dcc9700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbd400a1320 0x7fbd400a37e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd5dcc9700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd58105ce0 0x7fbd5819bbe0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd5dcc9700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd58067e30 0x7fbd58197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd5dcc9700 1 --2- 172.21.15.46:0/2083508419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd580687f0 0x7fbd58197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/2083508419 >> 172.21.15.46:0/2083508419 conn(0x7fbd58076d40 msgr2=0x7fbd580fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/2083508419 shutdown_connections 2024-03-18T20:36:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.287+0000 7fbd5dcc9700 1 -- 172.21.15.46:0/2083508419 wait complete. 2024-03-18T20:36:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:30 smithi046 ceph-mon[43495]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-03-18T20:36:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:30 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2308313593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:36:30.305 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:36:30.305 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:36:30.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.460+0000 7ff69b0a1700 1 -- 172.21.15.46:0/2308507784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6940687f0 msgr2=0x7ff69410db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.460+0000 7ff69b0a1700 1 --2- 172.21.15.46:0/2308507784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6940687f0 0x7ff69410db50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00b0a0 tx=0x7ff68c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.460+0000 7ff69b0a1700 1 -- 172.21.15.46:0/2308507784 shutdown_connections 2024-03-18T20:36:30.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.460+0000 7ff69b0a1700 1 --2- 172.21.15.46:0/2308507784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6940687f0 0x7ff69410db50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.460+0000 7ff69b0a1700 1 --2- 172.21.15.46:0/2308507784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff694105ce0 0x7ff6941060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.460+0000 7ff69b0a1700 1 --2- 172.21.15.46:0/2308507784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff694067e30 0x7ff6940682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.460+0000 7ff69b0a1700 1 -- 172.21.15.46:0/2308507784 >> 172.21.15.46:0/2308507784 conn(0x7ff694076d40 msgr2=0x7ff694077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.460+0000 7ff69b0a1700 1 -- 172.21.15.46:0/2308507784 shutdown_connections 2024-03-18T20:36:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.460+0000 7ff69b0a1700 1 -- 172.21.15.46:0/2308507784 wait complete. 2024-03-18T20:36:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff69b0a1700 1 Processor -- start 2024-03-18T20:36:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff69b0a1700 1 -- start start 2024-03-18T20:36:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff69b0a1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff694067e30 0x7ff694197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff69b0a1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6940687f0 0x7ff694197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff69b0a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff694105ce0 0x7ff69419bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff69b0a1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69419c230 con 0x7ff694067e30 2024-03-18T20:36:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff69b0a1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69419c3a0 con 0x7ff694105ce0 2024-03-18T20:36:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff69b0a1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69419c560 con 0x7ff6940687f0 2024-03-18T20:36:30.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff698e3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff694067e30 0x7ff694197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff698e3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff694067e30 0x7ff694197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34546/0 (socket says 172.21.15.46:34546) 2024-03-18T20:36:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff698e3d700 1 -- 172.21.15.46:0/852811979 learned_addr learned my addr 172.21.15.46:0/852811979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff693fff700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6940687f0 0x7ff694197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.461+0000 7ff69963e700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff694105ce0 0x7ff69419bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.462+0000 7ff698e3d700 1 -- 172.21.15.46:0/852811979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6940687f0 msgr2=0x7ff694197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.462+0000 7ff698e3d700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6940687f0 0x7ff694197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.462+0000 7ff698e3d700 1 -- 172.21.15.46:0/852811979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff694105ce0 msgr2=0x7ff69419bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.462+0000 7ff698e3d700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff694105ce0 0x7ff69419bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.462+0000 7ff698e3d700 1 -- 172.21.15.46:0/852811979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c009d00 con 0x7ff694067e30 2024-03-18T20:36:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.462+0000 7ff698e3d700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff694067e30 0x7ff694197310 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff68800e9a0 tx=0x7ff68800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.462+0000 7ff691ffb700 1 -- 172.21.15.46:0/852811979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688009790 con 0x7ff694067e30 2024-03-18T20:36:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.462+0000 7ff69b0a1700 1 -- 172.21.15.46:0/852811979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69419c840 con 0x7ff694067e30 2024-03-18T20:36:30.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.462+0000 7ff691ffb700 1 -- 172.21.15.46:0/852811979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff68800cc80 con 0x7ff694067e30 2024-03-18T20:36:30.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.462+0000 7ff69b0a1700 1 -- 172.21.15.46:0/852811979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6940ffd80 con 0x7ff694067e30 2024-03-18T20:36:30.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.463+0000 7ff691ffb700 1 -- 172.21.15.46:0/852811979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68801b980 con 0x7ff694067e30 2024-03-18T20:36:30.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.465+0000 7ff691ffb700 1 -- 172.21.15.46:0/852811979 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 136041+0+0 (secure 0 0 0) 0x7ff68800c7c0 con 0x7ff694067e30 2024-03-18T20:36:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.466+0000 7ff691ffb700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff6840a1320 0x7ff6840a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.466+0000 7ff693fff700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff6840a1320 0x7ff6840a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.466+0000 7ff691ffb700 1 -- 172.21.15.46:0/852811979 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7ff6880d0f40 con 0x7ff694067e30 2024-03-18T20:36:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.466+0000 7ff6a7236b80 1 Processor -- start 2024-03-18T20:36:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff6a7236b80 1 -- start start 2024-03-18T20:36:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff693fff700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff6840a1320 0x7ff6840a37e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff694198860 tx=0x7ff6800073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff6a7236b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0ca60 0x563203a0ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff6a7236b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a0d380 0x563203a17ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff6a7236b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 0x563203a1a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff6a7236b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563203966990 con 0x563203a18220 2024-03-18T20:36:30.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff6a7236b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563203964350 con 0x563203a0ca60 2024-03-18T20:36:30.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff6a7236b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56320395fc40 con 0x563203a0d380 2024-03-18T20:36:30.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff69963e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 0x563203a1a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff69963e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 0x563203a1a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34550/0 (socket says 172.21.15.46:34550) 2024-03-18T20:36:30.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff69963e700 1 -- 172.21.15.46:0/1562652694 learned_addr learned my addr 172.21.15.46:0/1562652694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff698e3d700 1 --2- 172.21.15.46:0/1562652694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0ca60 0x563203a0ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff693fff700 1 --2- 172.21.15.46:0/1562652694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a0d380 0x563203a17ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff69963e700 1 -- 172.21.15.46:0/1562652694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a0d380 msgr2=0x563203a17ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff69963e700 1 --2- 172.21.15.46:0/1562652694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a0d380 0x563203a17ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff69963e700 1 -- 172.21.15.46:0/1562652694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0ca60 msgr2=0x563203a0ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff69963e700 1 --2- 172.21.15.46:0/1562652694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0ca60 0x563203a0ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff69963e700 1 -- 172.21.15.46:0/1562652694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632037e13b0 con 0x563203a18220 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.467+0000 7ff698e3d700 1 --2- 172.21.15.46:0/1562652694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0ca60 0x563203a0ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.468+0000 7ff69963e700 1 --2- 172.21.15.46:0/1562652694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 0x563203a1a610 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009c60 tx=0x7ff68c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca54bdd3d9d6749 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.468+0000 7ff67effd700 1 -- 172.21.15.46:0/1562652694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c00e050 con 0x563203a18220 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.468+0000 7ff67effd700 1 -- 172.21.15.46:0/1562652694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff68c009490 con 0x563203a18220 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.468+0000 7ff67effd700 1 -- 172.21.15.46:0/1562652694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c007720 con 0x563203a18220 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.469+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1562652694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 msgr2=0x563203a1a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.469+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/1562652694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 0x563203a1a610 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009c60 tx=0x7ff68c009c90 comp rx=0 tx=0).stop 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.469+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1562652694 shutdown_connections 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.469+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/1562652694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 0x563203a1a610 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.469+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/1562652694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0ca60 0x563203a0ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.469+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/1562652694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a0d380 0x563203a17ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.469+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1562652694 >> 172.21.15.46:0/1562652694 conn(0x563203983220 msgr2=0x563203a09020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.469+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1562652694 shutdown_connections 2024-03-18T20:36:30.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.469+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1562652694 wait complete. 2024-03-18T20:36:30.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.469+0000 7ff6a7236b80 1 -- start start 2024-03-18T20:36:30.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.469+0000 7ff6a7236b80 1 Processor -- start 2024-03-18T20:36:30.477 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:30 smithi073 ceph-mon[47384]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-03-18T20:36:30.477 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:30 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2308313593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:36:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff6a7236b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 0x563203a17c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff6a7236b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0d380 0x563203a14f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff6a7236b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a18220 0x563203a15590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff6a7236b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632038fbd80 con 0x563203a0ca60 2024-03-18T20:36:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff6a7236b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632038fba20 con 0x563203a0d380 2024-03-18T20:36:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff6a7236b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632037e36a0 con 0x563203a18220 2024-03-18T20:36:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff6a7236b80 10 client.0 _ll_register_callbacks cb 0x5632037673a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff698e3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 0x563203a17c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff698e3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 0x563203a17c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34552/0 (socket says 172.21.15.46:34552) 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff698e3d700 1 -- 172.21.15.46:0/1707624750 learned_addr learned my addr 172.21.15.46:0/1707624750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff693fff700 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0d380 0x563203a14f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff698e3d700 1 -- 172.21.15.46:0/1707624750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a18220 msgr2=0x563203a15590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff698e3d700 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a18220 0x563203a15590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff698e3d700 1 -- 172.21.15.46:0/1707624750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0d380 msgr2=0x563203a14f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.470+0000 7ff698e3d700 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0d380 0x563203a14f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.471+0000 7ff698e3d700 1 -- 172.21.15.46:0/1707624750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6940ffd80 con 0x563203a0ca60 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.471+0000 7ff693fff700 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0d380 0x563203a14f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.471+0000 7ff698e3d700 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 0x563203a17c20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff68800f010 tx=0x7ff6880ceb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2256c640d488f6aa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.471+0000 7ff673fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.471+0000 7ff673fff700 1 -- 172.21.15.46:0/1707624750 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6880b5070 con 0x563203a0ca60 2024-03-18T20:36:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.471+0000 7ff673fff700 1 -- 172.21.15.46:0/1707624750 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7ff6880a1e30 con 0x563203a0ca60 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.471+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1707624750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563203a00ad0 con 0x563203a0ca60 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.471+0000 7ff673fff700 1 -- 172.21.15.46:0/1707624750 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6880a6480 con 0x563203a0ca60 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.471+0000 7ff6a7236b80 10 client.15696 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.471+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1707624750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ff69410e550 con 0x563203a0ca60 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.471+0000 7ff673fff700 1 -- 172.21.15.46:0/1707624750 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563203a00ad0 con 0x563203a0ca60 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.471+0000 7ff673fff700 1 -- 172.21.15.46:0/1707624750 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff6880bc200 con 0x563203a0ca60 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.472+0000 7ff6a7236b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.472+0000 7ff6a7236b80 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.472+0000 7ff6a7236b80 20 client.15696 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.472+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563203a27d30 0x563203a2a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.472+0000 7ff6a7236b80 4 client.15696 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.472+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1707624750 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563203958700 con 0x563203a27d30 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.472+0000 7ff693fff700 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563203a27d30 0x563203a2a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.472+0000 7ff673fff700 10 client.15696 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.472+0000 7ff693fff700 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563203a27d30 0x563203a2a150 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d085a2dea9163cb server_cookie=a7f589803258f879 in_seq=0 out_seq=0 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff673fff700 1 -- 172.21.15.46:0/1707624750 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563203958700 con 0x563203a27d30 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff673fff700 10 client.15696 handle_command_reply: tid=0 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff6a7236b80 1 client.15696 shutdown 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1707624750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 msgr2=0x563203a17c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 0x563203a17c20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff68800f010 tx=0x7ff6880ceb70 comp rx=0 tx=0).stop 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1707624750 shutdown_connections 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563203a27d30 0x563203a2a150 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 0x563203a17c20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a18220 0x563203a15590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/1707624750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0d380 0x563203a14f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff673fff700 0 client.15696 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1707624750 >> 172.21.15.46:0/1707624750 conn(0x563203983220 msgr2=0x563203a08c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1707624750 shutdown_connections 2024-03-18T20:36:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff6a7236b80 1 -- 172.21.15.46:0/1707624750 wait complete. 2024-03-18T20:36:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.473+0000 7ff6a7236b80 20 client.15696 trim_cache size 0 max 16384 2024-03-18T20:36:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.488+0000 7ff6a7236b80 1 Processor -- start 2024-03-18T20:36:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.488+0000 7ff6a7236b80 1 -- start start 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.488+0000 7ff6a7236b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 0x563203a15590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.488+0000 7ff6a7236b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0d380 0x563203a1ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.488+0000 7ff6a7236b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a18220 0x563203a137e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.488+0000 7ff6a7236b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563203a1f320 con 0x563203a0ca60 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.488+0000 7ff6a7236b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563203a10d50 con 0x563203a0d380 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.488+0000 7ff6a7236b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563203a00610 con 0x563203a18220 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.488+0000 7ff698e3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 0x563203a15590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.488+0000 7ff698e3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 0x563203a15590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34564/0 (socket says 172.21.15.46:34564) 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.488+0000 7ff698e3d700 1 -- 172.21.15.46:0/2314736622 learned_addr learned my addr 172.21.15.46:0/2314736622 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.489+0000 7ff693fff700 1 --2- 172.21.15.46:0/2314736622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0d380 0x563203a1ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.489+0000 7ff69963e700 1 --2- 172.21.15.46:0/2314736622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a18220 0x563203a137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.489+0000 7ff69963e700 1 -- 172.21.15.46:0/2314736622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0d380 msgr2=0x563203a1ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.489+0000 7ff69963e700 1 --2- 172.21.15.46:0/2314736622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0d380 0x563203a1ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.489+0000 7ff69963e700 1 -- 172.21.15.46:0/2314736622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 msgr2=0x563203a15590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.489+0000 7ff69963e700 1 --2- 172.21.15.46:0/2314736622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 0x563203a15590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.489+0000 7ff69963e700 1 -- 172.21.15.46:0/2314736622 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56320378db60 con 0x563203a18220 2024-03-18T20:36:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.489+0000 7ff693fff700 1 --2- 172.21.15.46:0/2314736622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0d380 0x563203a1ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.489+0000 7ff698e3d700 1 --2- 172.21.15.46:0/2314736622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 0x563203a15590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.489+0000 7ff69963e700 1 --2- 172.21.15.46:0/2314736622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a18220 0x563203a137e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009c60 tx=0x7ff68c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fdcdab019c09f64 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.489+0000 7ff67f7fe700 1 -- 172.21.15.46:0/2314736622 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c007d60 con 0x563203a18220 2024-03-18T20:36:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.490+0000 7ff67f7fe700 1 -- 172.21.15.46:0/2314736622 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff68c01f070 con 0x563203a18220 2024-03-18T20:36:30.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.490+0000 7ff67f7fe700 1 -- 172.21.15.46:0/2314736622 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0078a0 con 0x563203a18220 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.490+0000 7ff6a7236b80 1 -- 172.21.15.46:0/2314736622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a18220 msgr2=0x563203a137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.490+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/2314736622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a18220 0x563203a137e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009c60 tx=0x7ff68c0099e0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.490+0000 7ff6a7236b80 1 -- 172.21.15.46:0/2314736622 shutdown_connections 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.490+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/2314736622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a0ca60 0x563203a15590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.490+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/2314736622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a18220 0x563203a137e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.490+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/2314736622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0d380 0x563203a1ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.490+0000 7ff6a7236b80 1 -- 172.21.15.46:0/2314736622 >> 172.21.15.46:0/2314736622 conn(0x563203983220 msgr2=0x563203a101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.490+0000 7ff6a7236b80 1 -- 172.21.15.46:0/2314736622 shutdown_connections 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.490+0000 7ff6a7236b80 1 -- 172.21.15.46:0/2314736622 wait complete. 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.491+0000 7ff6a7236b80 1 -- start start 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.491+0000 7ff6a7236b80 1 Processor -- start 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.491+0000 7ff6a7236b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0ca60 0x563203a1adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.491+0000 7ff6a7236b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a0d380 0x563203a1b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.491+0000 7ff6a7236b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 0x563203a1b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.491+0000 7ff6a7236b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563203a1f340 con 0x563203a18220 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.491+0000 7ff6a7236b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563203a7ab80 con 0x563203a0ca60 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.491+0000 7ff6a7236b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563203a7ad40 con 0x563203a0d380 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.491+0000 7ff6a7236b80 10 client.0 _ll_register_callbacks cb 0x7ff68809d960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.491+0000 7ff69963e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 0x563203a1b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff693fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a0d380 0x563203a1b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff69963e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 0x563203a1b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34566/0 (socket says 172.21.15.46:34566) 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff693fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a0d380 0x563203a1b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59670/0 (socket says 172.21.15.46:59670) 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff69963e700 1 -- 172.21.15.46:0/3284191640 learned_addr learned my addr 172.21.15.46:0/3284191640 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff698e3d700 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0ca60 0x563203a1adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff693fff700 1 -- 172.21.15.46:0/3284191640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0ca60 msgr2=0x563203a1adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff693fff700 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0ca60 0x563203a1adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff693fff700 1 -- 172.21.15.46:0/3284191640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 msgr2=0x563203a1b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff693fff700 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 0x563203a1b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff693fff700 1 -- 172.21.15.46:0/3284191640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69410e550 con 0x563203a0d380 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff69963e700 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 0x563203a1b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff693fff700 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a0d380 0x563203a1b300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x563203a2aa00 tx=0x7ff680013070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb72ac891db0f7d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff673fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff673fff700 1 -- 172.21.15.46:0/3284191640 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680027040 con 0x563203a0d380 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff6a7236b80 1 -- 172.21.15.46:0/3284191640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff68c009490 con 0x563203a0d380 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff673fff700 1 -- 172.21.15.46:0/3284191640 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7ff680017070 con 0x563203a0d380 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.492+0000 7ff673fff700 1 -- 172.21.15.46:0/3284191640 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6800273a0 con 0x563203a0d380 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.493+0000 7ff6a7236b80 10 client.25474 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.493+0000 7ff6a7236b80 1 -- 172.21.15.46:0/3284191640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ff680010c90 con 0x563203a0d380 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.493+0000 7ff673fff700 1 -- 172.21.15.46:0/3284191640 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff68c009490 con 0x563203a0d380 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.493+0000 7ff673fff700 1 -- 172.21.15.46:0/3284191640 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff68002d070 con 0x563203a0d380 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.493+0000 7ff6a7236b80 10 client.25474 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:30.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.493+0000 7ff6a7236b80 10 client.25474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.493+0000 7ff6a7236b80 20 client.25474 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.493+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563203a7c540 0x563203a7e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.493+0000 7ff6a7236b80 4 client.25474 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.493+0000 7ff6a7236b80 1 -- 172.21.15.46:0/3284191640 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5632037aecf0 con 0x563203a7c540 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.493+0000 7ff698e3d700 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563203a7c540 0x563203a7e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff673fff700 10 client.25474 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff698e3d700 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563203a7c540 0x563203a7e960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739bf43f3912d7b0 server_cookie=23cb6bdff10176c4 in_seq=0 out_seq=0 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff673fff700 1 -- 172.21.15.46:0/3284191640 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5632037aecf0 con 0x563203a7c540 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff673fff700 10 client.25474 handle_command_reply: tid=0 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff6a7236b80 1 client.25474 shutdown 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff6a7236b80 1 -- 172.21.15.46:0/3284191640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a0d380 msgr2=0x563203a1b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a0d380 0x563203a1b300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x563203a2aa00 tx=0x7ff680013070 comp rx=0 tx=0).stop 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff6a7236b80 1 -- 172.21.15.46:0/3284191640 shutdown_connections 2024-03-18T20:36:30.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563203a7c540 0x563203a7e960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563203a18220 0x563203a1b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563203a0ca60 0x563203a1adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff673fff700 0 client.25474 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:30.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff6a7236b80 1 --2- 172.21.15.46:0/3284191640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563203a0d380 0x563203a1b300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff6a7236b80 1 -- 172.21.15.46:0/3284191640 >> 172.21.15.46:0/3284191640 conn(0x563203983220 msgr2=0x563203a13780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff6a7236b80 1 -- 172.21.15.46:0/3284191640 shutdown_connections 2024-03-18T20:36:30.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff6a7236b80 1 -- 172.21.15.46:0/3284191640 wait complete. 2024-03-18T20:36:30.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.494+0000 7ff6a7236b80 20 client.25474 trim_cache size 0 max 16384 2024-03-18T20:36:30.503 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:36:30.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.495+0000 7ff69b0a1700 1 -- 172.21.15.46:0/852811979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff6840a1320 msgr2=0x7ff6840a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.495+0000 7ff69b0a1700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff6840a1320 0x7ff6840a37e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff694198860 tx=0x7ff6800073d0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.495+0000 7ff69b0a1700 1 -- 172.21.15.46:0/852811979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff694067e30 msgr2=0x7ff694197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.495+0000 7ff69b0a1700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff694067e30 0x7ff694197310 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff68800e9a0 tx=0x7ff68800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.495+0000 7ff69b0a1700 1 -- 172.21.15.46:0/852811979 shutdown_connections 2024-03-18T20:36:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.495+0000 7ff69b0a1700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff6840a1320 0x7ff6840a37e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.495+0000 7ff69b0a1700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff694067e30 0x7ff694197310 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.495+0000 7ff69b0a1700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff694105ce0 0x7ff69419bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.495+0000 7ff69b0a1700 1 --2- 172.21.15.46:0/852811979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6940687f0 0x7ff694197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.495+0000 7ff69b0a1700 1 -- 172.21.15.46:0/852811979 >> 172.21.15.46:0/852811979 conn(0x7ff694076d40 msgr2=0x7ff6940fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.495+0000 7ff69b0a1700 1 -- 172.21.15.46:0/852811979 shutdown_connections 2024-03-18T20:36:30.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:30.495+0000 7ff69b0a1700 1 -- 172.21.15.46:0/852811979 wait complete. 2024-03-18T20:36:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:30 smithi155 ceph-mon[48344]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2024-03-18T20:36:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:30 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2308313593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:36:31.515 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:36:31.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:36:31.515 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:36:31.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.690+0000 7fd29098e700 1 -- 172.21.15.46:0/2315641457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd28c10d750 msgr2=0x7fd28c10fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.690+0000 7fd29098e700 1 --2- 172.21.15.46:0/2315641457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd28c10d750 0x7fd28c10fb40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd28400b0a0 tx=0x7fd28400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.690+0000 7fd29098e700 1 -- 172.21.15.46:0/2315641457 shutdown_connections 2024-03-18T20:36:31.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.690+0000 7fd29098e700 1 --2- 172.21.15.46:0/2315641457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd28c10d750 0x7fd28c10fb40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.690+0000 7fd29098e700 1 --2- 172.21.15.46:0/2315641457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd28c069030 0x7fd28c069410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.690+0000 7fd29098e700 1 --2- 172.21.15.46:0/2315641457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c069950 0x7fd28c10d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.690+0000 7fd29098e700 1 -- 172.21.15.46:0/2315641457 >> 172.21.15.46:0/2315641457 conn(0x7fd28c076d80 msgr2=0x7fd28c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:31.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.690+0000 7fd29098e700 1 -- 172.21.15.46:0/2315641457 shutdown_connections 2024-03-18T20:36:31.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.690+0000 7fd29098e700 1 -- 172.21.15.46:0/2315641457 wait complete. 2024-03-18T20:36:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.691+0000 7fd29098e700 1 Processor -- start 2024-03-18T20:36:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.691+0000 7fd29098e700 1 -- start start 2024-03-18T20:36:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.691+0000 7fd29098e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd28c069030 0x7fd28c199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.691+0000 7fd29098e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c069950 0x7fd28c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.691+0000 7fd29098e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd28c10d750 0x7fd28c19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd29098e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c19e3b0 con 0x7fd28c069030 2024-03-18T20:36:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd29098e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c19e520 con 0x7fd28c069950 2024-03-18T20:36:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd29098e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c19e6e0 con 0x7fd28c10d750 2024-03-18T20:36:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd28a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd28c069030 0x7fd28c199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd28a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd28c069030 0x7fd28c199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34590/0 (socket says 172.21.15.46:34590) 2024-03-18T20:36:31.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd28a59c700 1 -- 172.21.15.46:0/1965857924 learned_addr learned my addr 172.21.15.46:0/1965857924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd289d9b700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c069950 0x7fd28c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd28ad9d700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd28c10d750 0x7fd28c19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd28a59c700 1 -- 172.21.15.46:0/1965857924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd28c10d750 msgr2=0x7fd28c19dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd28a59c700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd28c10d750 0x7fd28c19dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd28a59c700 1 -- 172.21.15.46:0/1965857924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c069950 msgr2=0x7fd28c1999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd28a59c700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c069950 0x7fd28c1999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd28a59c700 1 -- 172.21.15.46:0/1965857924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd284009d00 con 0x7fd28c069030 2024-03-18T20:36:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd28ad9d700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd28c10d750 0x7fd28c19dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:36:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd289d9b700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c069950 0x7fd28c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd28a59c700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd28c069030 0x7fd28c199490 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00e9a0 tx=0x7fd27c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:31.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.692+0000 7fd27b7fe700 1 -- 172.21.15.46:0/1965857924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c009790 con 0x7fd28c069030 2024-03-18T20:36:31.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.693+0000 7fd27b7fe700 1 -- 172.21.15.46:0/1965857924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd27c00cbd0 con 0x7fd28c069030 2024-03-18T20:36:31.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.693+0000 7fd29098e700 1 -- 172.21.15.46:0/1965857924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28c19e9c0 con 0x7fd28c069030 2024-03-18T20:36:31.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.693+0000 7fd27b7fe700 1 -- 172.21.15.46:0/1965857924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c018680 con 0x7fd28c069030 2024-03-18T20:36:31.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.693+0000 7fd29098e700 1 -- 172.21.15.46:0/1965857924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28c1a6200 con 0x7fd28c069030 2024-03-18T20:36:31.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.695+0000 7fd27b7fe700 1 -- 172.21.15.46:0/1965857924 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 136041+0+0 (secure 0 0 0) 0x7fd27c00cd40 con 0x7fd28c069030 2024-03-18T20:36:31.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.696+0000 7fd27b7fe700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2740a13a0 0x7fd2740a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.696+0000 7fd27b7fe700 1 -- 172.21.15.46:0/1965857924 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd27c014070 con 0x7fd28c069030 2024-03-18T20:36:31.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.697+0000 7fd289d9b700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2740a13a0 0x7fd2740a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.697+0000 7fd289d9b700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2740a13a0 0x7fd2740a3860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd28c19a9e0 tx=0x7fd280009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:31.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.698+0000 7fd29cb23b80 1 Processor -- start 2024-03-18T20:36:31.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.698+0000 7fd29cb23b80 1 -- start start 2024-03-18T20:36:31.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.698+0000 7fd29cb23b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae948ae0 0x561cae948ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.698+0000 7fd29cb23b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae949400 0x561cae953d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.698+0000 7fd29cb23b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae956690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.698+0000 7fd29cb23b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cae8a2a10 con 0x561cae948ae0 2024-03-18T20:36:31.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.698+0000 7fd29cb23b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cae8a03d0 con 0x561cae9542a0 2024-03-18T20:36:31.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.698+0000 7fd29cb23b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cae89bcc0 con 0x561cae949400 2024-03-18T20:36:31.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd28a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae948ae0 0x561cae948ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd289d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae949400 0x561cae953d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd28a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae948ae0 0x561cae948ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34606/0 (socket says 172.21.15.46:34606) 2024-03-18T20:36:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd28a59c700 1 -- 172.21.15.46:0/2305404077 learned_addr learned my addr 172.21.15.46:0/2305404077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:31.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd28ad9d700 1 --2- 172.21.15.46:0/2305404077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae956690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd289d9b700 1 -- 172.21.15.46:0/2305404077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 msgr2=0x561cae956690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd289d9b700 1 --2- 172.21.15.46:0/2305404077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae956690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd289d9b700 1 -- 172.21.15.46:0/2305404077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae948ae0 msgr2=0x561cae948ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd289d9b700 1 --2- 172.21.15.46:0/2305404077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae948ae0 0x561cae948ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd289d9b700 1 -- 172.21.15.46:0/2305404077 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cae71d430 con 0x561cae949400 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd28ad9d700 1 --2- 172.21.15.46:0/2305404077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae956690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd289d9b700 1 --2- 172.21.15.46:0/2305404077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae949400 0x561cae953d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd280010a40 tx=0x7fd280013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea641ef6fccfed2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.699+0000 7fd278ff9700 1 -- 172.21.15.46:0/2305404077 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280011bb0 con 0x561cae949400 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.700+0000 7fd278ff9700 1 -- 172.21.15.46:0/2305404077 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd280015070 con 0x561cae949400 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.700+0000 7fd278ff9700 1 -- 172.21.15.46:0/2305404077 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28001c610 con 0x561cae949400 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.700+0000 7fd29cb23b80 1 -- 172.21.15.46:0/2305404077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae949400 msgr2=0x561cae953d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.700+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/2305404077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae949400 0x561cae953d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd280010a40 tx=0x7fd280013000 comp rx=0 tx=0).stop 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.700+0000 7fd29cb23b80 1 -- 172.21.15.46:0/2305404077 shutdown_connections 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.700+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/2305404077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae948ae0 0x561cae948ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.700+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/2305404077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae956690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.700+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/2305404077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae949400 0x561cae953d60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.700+0000 7fd29cb23b80 1 -- 172.21.15.46:0/2305404077 >> 172.21.15.46:0/2305404077 conn(0x561cae8bf2a0 msgr2=0x561cae9450a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.701+0000 7fd29cb23b80 1 -- 172.21.15.46:0/2305404077 shutdown_connections 2024-03-18T20:36:31.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.701+0000 7fd29cb23b80 1 -- 172.21.15.46:0/2305404077 wait complete. 2024-03-18T20:36:31.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.701+0000 7fd29cb23b80 1 -- start start 2024-03-18T20:36:31.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.701+0000 7fd29cb23b80 1 Processor -- start 2024-03-18T20:36:31.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.701+0000 7fd29cb23b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae948ae0 0x561cae953d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.701+0000 7fd29cb23b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae949400 0x561cae951030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.701+0000 7fd29cb23b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae951770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.701+0000 7fd29cb23b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cae837e00 con 0x561cae949400 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.701+0000 7fd29cb23b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cae837aa0 con 0x561cae9542a0 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.701+0000 7fd29cb23b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cae71f720 con 0x561cae948ae0 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.702+0000 7fd29cb23b80 10 client.0 _ll_register_callbacks cb 0x561cae6a3420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.702+0000 7fd28a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae948ae0 0x561cae953d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.702+0000 7fd28a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae948ae0 0x561cae953d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59688/0 (socket says 172.21.15.46:59688) 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.702+0000 7fd28a59c700 1 -- 172.21.15.46:0/3941222991 learned_addr learned my addr 172.21.15.46:0/3941222991 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.702+0000 7fd28ad9d700 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae951770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.702+0000 7fd289d9b700 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae949400 0x561cae951030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.702+0000 7fd28ad9d700 1 -- 172.21.15.46:0/3941222991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae948ae0 msgr2=0x561cae953d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.702+0000 7fd28ad9d700 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae948ae0 0x561cae953d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.702+0000 7fd28ad9d700 1 -- 172.21.15.46:0/3941222991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae949400 msgr2=0x561cae951030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.702+0000 7fd28ad9d700 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae949400 0x561cae951030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.702+0000 7fd28ad9d700 1 -- 172.21.15.46:0/3941222991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd280010c50 con 0x561cae9542a0 2024-03-18T20:36:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.703+0000 7fd28ad9d700 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae951770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd28400b070 tx=0x7fd284007dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e44a0bf06186ffad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.703+0000 7fd271ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:36:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.703+0000 7fd271ffb700 1 -- 172.21.15.46:0/3941222991 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28400e040 con 0x561cae9542a0 2024-03-18T20:36:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.703+0000 7fd271ffb700 1 -- 172.21.15.46:0/3941222991 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd284004a50 con 0x561cae9542a0 2024-03-18T20:36:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.703+0000 7fd29cb23b80 1 -- 172.21.15.46:0/3941222991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cae93cb50 con 0x561cae9542a0 2024-03-18T20:36:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.703+0000 7fd271ffb700 1 -- 172.21.15.46:0/3941222991 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd284004c10 con 0x561cae9542a0 2024-03-18T20:36:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.703+0000 7fd29cb23b80 10 client.25484 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.703+0000 7fd29cb23b80 1 -- 172.21.15.46:0/3941222991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561cae6c9be0 con 0x561cae9542a0 2024-03-18T20:36:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.704+0000 7fd271ffb700 1 -- 172.21.15.46:0/3941222991 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561cae93cb50 con 0x561cae9542a0 2024-03-18T20:36:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.705+0000 7fd271ffb700 1 -- 172.21.15.46:0/3941222991 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd284019050 con 0x561cae9542a0 2024-03-18T20:36:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.705+0000 7fd29cb23b80 10 client.25484 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.705+0000 7fd29cb23b80 10 client.25484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.705+0000 7fd29cb23b80 20 client.25484 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.705+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561cae974ef0 0x561cae9772e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.705+0000 7fd29cb23b80 4 client.25484 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:36:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.705+0000 7fd29cb23b80 1 -- 172.21.15.46:0/3941222991 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cae894780 con 0x561cae974ef0 2024-03-18T20:36:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.705+0000 7fd28a59c700 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561cae974ef0 0x561cae9772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.706+0000 7fd271ffb700 10 client.25484 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.706+0000 7fd28a59c700 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561cae974ef0 0x561cae9772e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d97988a60ea307 server_cookie=1ccc8bcb7ea9ea84 in_seq=0 out_seq=0 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.706+0000 7fd271ffb700 1 -- 172.21.15.46:0/3941222991 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561cae894780 con 0x561cae974ef0 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.706+0000 7fd271ffb700 10 client.25484 handle_command_reply: tid=0 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.706+0000 7fd29cb23b80 1 client.25484 shutdown 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.707+0000 7fd29cb23b80 1 -- 172.21.15.46:0/3941222991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 msgr2=0x561cae951770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.707+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae951770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd28400b070 tx=0x7fd284007dd0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.707+0000 7fd29cb23b80 1 -- 172.21.15.46:0/3941222991 shutdown_connections 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.707+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561cae974ef0 0x561cae9772e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.707+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae951770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.707+0000 7fd271ffb700 0 client.25484 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.707+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae948ae0 0x561cae953d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.707+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/3941222991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae949400 0x561cae951030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.707+0000 7fd29cb23b80 1 -- 172.21.15.46:0/3941222991 >> 172.21.15.46:0/3941222991 conn(0x561cae8bf2a0 msgr2=0x561cae944c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.707+0000 7fd29cb23b80 1 -- 172.21.15.46:0/3941222991 shutdown_connections 2024-03-18T20:36:31.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.707+0000 7fd29cb23b80 1 -- 172.21.15.46:0/3941222991 wait complete. 2024-03-18T20:36:31.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.707+0000 7fd29cb23b80 20 client.25484 trim_cache size 0 max 16384 2024-03-18T20:36:31.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.741+0000 7fd29cb23b80 1 Processor -- start 2024-03-18T20:36:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.741+0000 7fd29cb23b80 1 -- start start 2024-03-18T20:36:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.741+0000 7fd29cb23b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae948ae0 0x561cae96bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.741+0000 7fd29cb23b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae949400 0x561cae94c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.741+0000 7fd29cb23b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae9542a0 0x561cae94c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.741+0000 7fd29cb23b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cae9464c0 con 0x561cae9542a0 2024-03-18T20:36:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.741+0000 7fd29cb23b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cae952630 con 0x561cae948ae0 2024-03-18T20:36:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.741+0000 7fd29cb23b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cae949db0 con 0x561cae949400 2024-03-18T20:36:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.742+0000 7fd28ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae9542a0 0x561cae94c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.742+0000 7fd28a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae948ae0 0x561cae96bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.742+0000 7fd28ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae9542a0 0x561cae94c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34634/0 (socket says 172.21.15.46:34634) 2024-03-18T20:36:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.742+0000 7fd28ad9d700 1 -- 172.21.15.46:0/1434068724 learned_addr learned my addr 172.21.15.46:0/1434068724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.742+0000 7fd28a59c700 1 -- 172.21.15.46:0/1434068724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae949400 msgr2=0x561cae94c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.742+0000 7fd28a59c700 1 --2- 172.21.15.46:0/1434068724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae949400 0x561cae94c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.742+0000 7fd28a59c700 1 -- 172.21.15.46:0/1434068724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae9542a0 msgr2=0x561cae94c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.742+0000 7fd28a59c700 1 --2- 172.21.15.46:0/1434068724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae9542a0 0x561cae94c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.742+0000 7fd28a59c700 1 -- 172.21.15.46:0/1434068724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cae97bf10 con 0x561cae948ae0 2024-03-18T20:36:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.742+0000 7fd28ad9d700 1 --2- 172.21.15.46:0/1434068724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae9542a0 0x561cae94c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.742+0000 7fd28a59c700 1 --2- 172.21.15.46:0/1434068724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae948ae0 0x561cae96bee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd27c099e50 tx=0x7fd27c01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e3534c075612860 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.743+0000 7fd2797fa700 1 -- 172.21.15.46:0/1434068724 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c09a150 con 0x561cae948ae0 2024-03-18T20:36:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.743+0000 7fd2797fa700 1 -- 172.21.15.46:0/1434068724 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd27c09df80 con 0x561cae948ae0 2024-03-18T20:36:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.743+0000 7fd2797fa700 1 -- 172.21.15.46:0/1434068724 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0998a0 con 0x561cae948ae0 2024-03-18T20:36:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.744+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1434068724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae948ae0 msgr2=0x561cae96bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.744+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/1434068724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae948ae0 0x561cae96bee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd27c099e50 tx=0x7fd27c01a040 comp rx=0 tx=0).stop 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.744+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1434068724 shutdown_connections 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.744+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/1434068724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae9542a0 0x561cae94c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.744+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/1434068724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae948ae0 0x561cae96bee0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.744+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/1434068724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae949400 0x561cae94c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.744+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1434068724 >> 172.21.15.46:0/1434068724 conn(0x561cae8bf2a0 msgr2=0x561cae944bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.744+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1434068724 shutdown_connections 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.744+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1434068724 wait complete. 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.744+0000 7fd29cb23b80 1 -- start start 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.744+0000 7fd29cb23b80 1 Processor -- start 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd29cb23b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae948ae0 0x561cae96ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd29cb23b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae949400 0x561cae96b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd29cb23b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae9d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd29cb23b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cae96b9d0 con 0x561cae949400 2024-03-18T20:36:31.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd29cb23b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cae9d1a50 con 0x561cae9542a0 2024-03-18T20:36:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd29cb23b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cae9becc0 con 0x561cae948ae0 2024-03-18T20:36:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd29cb23b80 10 client.0 _ll_register_callbacks cb 0x7fd280005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd28ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae9d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd28ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae9d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37386/0 (socket says 172.21.15.46:37386) 2024-03-18T20:36:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd28ad9d700 1 -- 172.21.15.46:0/1533750536 learned_addr learned my addr 172.21.15.46:0/1533750536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd28a59c700 1 --2- 172.21.15.46:0/1533750536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae948ae0 0x561cae96ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd28ad9d700 1 -- 172.21.15.46:0/1533750536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae948ae0 msgr2=0x561cae96ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd28ad9d700 1 --2- 172.21.15.46:0/1533750536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae948ae0 0x561cae96ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd28ad9d700 1 -- 172.21.15.46:0/1533750536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae949400 msgr2=0x561cae96b2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:36:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd28ad9d700 1 --2- 172.21.15.46:0/1533750536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae949400 0x561cae96b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd28ad9d700 1 -- 172.21.15.46:0/1533750536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cae6c9be0 con 0x561cae9542a0 2024-03-18T20:36:31.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd28ad9d700 1 --2- 172.21.15.46:0/1533750536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae9d1510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd28401c580 tx=0x7fd284007780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85af2ca3c87ce66e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.745+0000 7fd271ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:36:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.746+0000 7fd271ffb700 1 -- 172.21.15.46:0/1533750536 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28400e040 con 0x561cae9542a0 2024-03-18T20:36:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.746+0000 7fd271ffb700 1 -- 172.21.15.46:0/1533750536 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd284007e40 con 0x561cae9542a0 2024-03-18T20:36:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.746+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1533750536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd280015070 con 0x561cae9542a0 2024-03-18T20:36:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.746+0000 7fd271ffb700 1 -- 172.21.15.46:0/1533750536 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd28400b700 con 0x561cae9542a0 2024-03-18T20:36:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.746+0000 7fd29cb23b80 10 client.25490 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.746+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1533750536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fd27c09c040 con 0x561cae9542a0 2024-03-18T20:36:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.746+0000 7fd271ffb700 1 -- 172.21.15.46:0/1533750536 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd280015070 con 0x561cae9542a0 2024-03-18T20:36:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.746+0000 7fd271ffb700 1 -- 172.21.15.46:0/1533750536 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd284019070 con 0x561cae9542a0 2024-03-18T20:36:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.746+0000 7fd29cb23b80 10 client.25490 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:31.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.746+0000 7fd29cb23b80 10 client.25490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.746+0000 7fd29cb23b80 20 client.25490 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.747+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/1533750536 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561cae9c00f0 0x561cae9c2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.747+0000 7fd29cb23b80 4 client.25490 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:36:31.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.747+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1533750536 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561cae6ead70 con 0x561cae9c00f0 2024-03-18T20:36:31.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.748+0000 7fd28a59c700 1 --2- 172.21.15.46:0/1533750536 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561cae9c00f0 0x561cae9c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.748+0000 7fd271ffb700 10 client.25490 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.748+0000 7fd28a59c700 1 --2- 172.21.15.46:0/1533750536 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561cae9c00f0 0x561cae9c2510 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c4fdb756c74d23 server_cookie=5be64c15e982a539 in_seq=0 out_seq=0 2024-03-18T20:36:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.749+0000 7fd271ffb700 1 -- 172.21.15.46:0/1533750536 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561cae6ead70 con 0x561cae9c00f0 2024-03-18T20:36:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.749+0000 7fd271ffb700 10 client.25490 handle_command_reply: tid=0 2024-03-18T20:36:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.749+0000 7fd29cb23b80 1 client.25490 shutdown 2024-03-18T20:36:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.750+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1533750536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 msgr2=0x561cae9d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.750+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/1533750536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae9d1510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd28401c580 tx=0x7fd284007780 comp rx=0 tx=0).stop 2024-03-18T20:36:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.750+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1533750536 shutdown_connections 2024-03-18T20:36:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.750+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/1533750536 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561cae9c00f0 0x561cae9c2510 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.750+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/1533750536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cae9542a0 0x561cae9d1510 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.750+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/1533750536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561cae948ae0 0x561cae96ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.750+0000 7fd29cb23b80 1 --2- 172.21.15.46:0/1533750536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cae949400 0x561cae96b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.750+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1533750536 >> 172.21.15.46:0/1533750536 conn(0x561cae8bf2a0 msgr2=0x561cae944b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.750+0000 7fd271ffb700 0 client.25490 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.751+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1533750536 shutdown_connections 2024-03-18T20:36:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.751+0000 7fd29cb23b80 1 -- 172.21.15.46:0/1533750536 wait complete. 2024-03-18T20:36:31.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.751+0000 7fd29cb23b80 20 client.25490 trim_cache size 0 max 16384 2024-03-18T20:36:31.757 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:36:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:36:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "9392ff0f-bd59-49f0-b019-b832f4f615c1", 2024-03-18T20:36:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:36:31.758 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:36:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.752+0000 7fd29098e700 1 -- 172.21.15.46:0/1965857924 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2740a13a0 msgr2=0x7fd2740a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.752+0000 7fd29098e700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2740a13a0 0x7fd2740a3860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd28c19a9e0 tx=0x7fd280009260 comp rx=0 tx=0).stop 2024-03-18T20:36:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.752+0000 7fd29098e700 1 -- 172.21.15.46:0/1965857924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd28c069030 msgr2=0x7fd28c199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.752+0000 7fd29098e700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd28c069030 0x7fd28c199490 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00e9a0 tx=0x7fd27c00ed60 comp rx=0 tx=0).stop 2024-03-18T20:36:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.752+0000 7fd29098e700 1 -- 172.21.15.46:0/1965857924 shutdown_connections 2024-03-18T20:36:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.752+0000 7fd29098e700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2740a13a0 0x7fd2740a3860 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.752+0000 7fd29098e700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd28c069030 0x7fd28c199490 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.752+0000 7fd29098e700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd28c10d750 0x7fd28c19dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.752+0000 7fd29098e700 1 --2- 172.21.15.46:0/1965857924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c069950 0x7fd28c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.752+0000 7fd29098e700 1 -- 172.21.15.46:0/1965857924 >> 172.21.15.46:0/1965857924 conn(0x7fd28c076d80 msgr2=0x7fd28c10ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.752+0000 7fd29098e700 1 -- 172.21.15.46:0/1965857924 shutdown_connections 2024-03-18T20:36:31.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.752+0000 7fd29098e700 1 -- 172.21.15.46:0/1965857924 wait complete. 2024-03-18T20:36:31.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:36:31.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:36:31.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.916+0000 7f71b9f71700 1 -- 172.21.15.46:0/3884645165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b4106090 msgr2=0x7f71b410dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.916+0000 7f71b9f71700 1 --2- 172.21.15.46:0/3884645165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b4106090 0x7f71b410dc10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00b0a0 tx=0x7f71ac00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.916+0000 7f71b9f71700 1 -- 172.21.15.46:0/3884645165 shutdown_connections 2024-03-18T20:36:31.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.916+0000 7f71b9f71700 1 --2- 172.21.15.46:0/3884645165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b4106090 0x7f71b410dc10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.916+0000 7f71b9f71700 1 --2- 172.21.15.46:0/3884645165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b4100f70 0x7f71b4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.916+0000 7f71b9f71700 1 --2- 172.21.15.46:0/3884645165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b4101920 0x7f71b4105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.916+0000 7f71b9f71700 1 -- 172.21.15.46:0/3884645165 >> 172.21.15.46:0/3884645165 conn(0x7f71b40fc7f0 msgr2=0x7f71b40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:31.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.917+0000 7f71b9f71700 1 -- 172.21.15.46:0/3884645165 shutdown_connections 2024-03-18T20:36:31.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.917+0000 7f71b9f71700 1 -- 172.21.15.46:0/3884645165 wait complete. 2024-03-18T20:36:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.917+0000 7f71b9f71700 1 Processor -- start 2024-03-18T20:36:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.918+0000 7f71b9f71700 1 -- start start 2024-03-18T20:36:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.918+0000 7f71b9f71700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b4100f70 0x7f71b4197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.918+0000 7f71b9f71700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b4106090 0x7f71b4197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.918+0000 7f71b9f71700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b419bc20 0x7f71b419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.918+0000 7f71b9f71700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b419c6a0 con 0x7f71b4106090 2024-03-18T20:36:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.918+0000 7f71b9f71700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b419c810 con 0x7f71b419bc20 2024-03-18T20:36:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.918+0000 7f71b9f71700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b419c9d0 con 0x7f71b4100f70 2024-03-18T20:36:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.918+0000 7f71b2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b4106090 0x7f71b4197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.918+0000 7f71b2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b4106090 0x7f71b4197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34652/0 (socket says 172.21.15.46:34652) 2024-03-18T20:36:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.918+0000 7f71b2ffd700 1 -- 172.21.15.46:0/3056981327 learned_addr learned my addr 172.21.15.46:0/3056981327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.918+0000 7f71b3fff700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b419bc20 0x7f71b419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b37fe700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b4100f70 0x7f71b4197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b3fff700 1 -- 172.21.15.46:0/3056981327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b4100f70 msgr2=0x7f71b4197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b3fff700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b4100f70 0x7f71b4197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b3fff700 1 -- 172.21.15.46:0/3056981327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b4106090 msgr2=0x7f71b4197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b3fff700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b4106090 0x7f71b4197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b3fff700 1 -- 172.21.15.46:0/3056981327 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a8009520 con 0x7f71b419bc20 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b2ffd700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b4106090 0x7f71b4197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b3fff700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b419bc20 0x7f71b419c0a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0062a0 tx=0x7f71ac00bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b0ff9700 1 -- 172.21.15.46:0/3056981327 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00e040 con 0x7f71b419bc20 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b0ff9700 1 -- 172.21.15.46:0/3056981327 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71ac01d480 con 0x7f71b419bc20 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b0ff9700 1 -- 172.21.15.46:0/3056981327 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac01b6d0 con 0x7f71b419bc20 2024-03-18T20:36:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b9f71700 1 -- 172.21.15.46:0/3056981327 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac009d00 con 0x7f71b419bc20 2024-03-18T20:36:31.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.919+0000 7f71b9f71700 1 -- 172.21.15.46:0/3056981327 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b4073280 con 0x7f71b419bc20 2024-03-18T20:36:31.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.923+0000 7f71b0ff9700 1 -- 172.21.15.46:0/3056981327 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f71ac01d5f0 con 0x7f71b419bc20 2024-03-18T20:36:31.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.923+0000 7f71b0ff9700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f719c0a1320 0x7f719c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71b0ff9700 1 -- 172.21.15.46:0/3056981327 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f71ac0ce4c0 con 0x7f71b419bc20 2024-03-18T20:36:31.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71b37fe700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f719c0a1320 0x7f719c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71c6106b80 1 Processor -- start 2024-03-18T20:36:31.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71c6106b80 1 -- start start 2024-03-18T20:36:31.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71b37fe700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f719c0a1320 0x7f719c0a37e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009bf0 tx=0x7f71a4009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71c6106b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451e8a60 0x555d451e8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71c6106b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e9380 0x555d451f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71c6106b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451f4220 0x555d451f6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71c6106b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d45142990 con 0x555d451e8a60 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71c6106b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d45140350 con 0x555d451f4220 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71c6106b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4513bc40 con 0x555d451e9380 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71b37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451e8a60 0x555d451e8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71b37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451e8a60 0x555d451e8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34666/0 (socket says 172.21.15.46:34666) 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.924+0000 7f71b37fe700 1 -- 172.21.15.46:0/53545286 learned_addr learned my addr 172.21.15.46:0/53545286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.925+0000 7f71b3fff700 1 --2- 172.21.15.46:0/53545286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451f4220 0x555d451f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.925+0000 7f71b37fe700 1 -- 172.21.15.46:0/53545286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e9380 msgr2=0x555d451f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.925+0000 7f71b37fe700 1 --2- 172.21.15.46:0/53545286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e9380 0x555d451f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.925+0000 7f71b37fe700 1 -- 172.21.15.46:0/53545286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451f4220 msgr2=0x555d451f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.925+0000 7f71b37fe700 1 --2- 172.21.15.46:0/53545286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451f4220 0x555d451f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.925+0000 7f71b37fe700 1 -- 172.21.15.46:0/53545286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d44fbd3b0 con 0x555d451e8a60 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.925+0000 7f71b37fe700 1 --2- 172.21.15.46:0/53545286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451e8a60 0x555d451e8e40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f71a4011e40 tx=0x7f71a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfb877d810d226e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.925+0000 7f7199ffb700 1 -- 172.21.15.46:0/53545286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4012dc0 con 0x555d451e8a60 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.925+0000 7f7199ffb700 1 -- 172.21.15.46:0/53545286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a4016070 con 0x555d451e8a60 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.925+0000 7f7199ffb700 1 -- 172.21.15.46:0/53545286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4003dd0 con 0x555d451e8a60 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.926+0000 7f71c6106b80 1 -- 172.21.15.46:0/53545286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451e8a60 msgr2=0x555d451e8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.926+0000 7f71c6106b80 1 --2- 172.21.15.46:0/53545286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451e8a60 0x555d451e8e40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f71a4011e40 tx=0x7f71a4014000 comp rx=0 tx=0).stop 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.926+0000 7f71c6106b80 1 -- 172.21.15.46:0/53545286 shutdown_connections 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.926+0000 7f71c6106b80 1 --2- 172.21.15.46:0/53545286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451e8a60 0x555d451e8e40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.926+0000 7f71c6106b80 1 --2- 172.21.15.46:0/53545286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451f4220 0x555d451f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.926+0000 7f71c6106b80 1 --2- 172.21.15.46:0/53545286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e9380 0x555d451f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.926+0000 7f71c6106b80 1 -- 172.21.15.46:0/53545286 >> 172.21.15.46:0/53545286 conn(0x555d4515f220 msgr2=0x555d451e5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:31.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.926+0000 7f71c6106b80 1 -- 172.21.15.46:0/53545286 shutdown_connections 2024-03-18T20:36:31.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.926+0000 7f71c6106b80 1 -- 172.21.15.46:0/53545286 wait complete. 2024-03-18T20:36:31.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.927+0000 7f71c6106b80 1 -- start start 2024-03-18T20:36:31.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.927+0000 7f71c6106b80 1 Processor -- start 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.927+0000 7f71c6106b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e8a60 0x555d451f0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.927+0000 7f71c6106b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451f1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.927+0000 7f71c6106b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451f4220 0x555d451f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.927+0000 7f71c6106b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d450d7d80 con 0x555d451f4220 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.927+0000 7f71c6106b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d450d7a20 con 0x555d451e9380 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.927+0000 7f71c6106b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d44fbf6a0 con 0x555d451e8a60 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.927+0000 7f71c6106b80 10 client.0 _ll_register_callbacks cb 0x555d44f433a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.928+0000 7f71b2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.928+0000 7f71b2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451f1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37424/0 (socket says 172.21.15.46:37424) 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.928+0000 7f71b2ffd700 1 -- 172.21.15.46:0/3369068047 learned_addr learned my addr 172.21.15.46:0/3369068047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.928+0000 7f71b2ffd700 1 -- 172.21.15.46:0/3369068047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e8a60 msgr2=0x555d451f0fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.928+0000 7f71b2ffd700 1 --2- 172.21.15.46:0/3369068047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e8a60 0x555d451f0fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.928+0000 7f71b2ffd700 1 -- 172.21.15.46:0/3369068047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451f4220 msgr2=0x555d451f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.928+0000 7f71b2ffd700 1 --2- 172.21.15.46:0/3369068047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451f4220 0x555d451f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.928+0000 7f71b2ffd700 1 -- 172.21.15.46:0/3369068047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a4011b20 con 0x555d451e9380 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.929+0000 7f71b2ffd700 1 --2- 172.21.15.46:0/3369068047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451f1510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f71a8011ce0 tx=0x7f71a800f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ea47bcaa6e621c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.929+0000 7f7192ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.929+0000 7f7192ffd700 1 -- 172.21.15.46:0/3369068047 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8019430 con 0x555d451e9380 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.929+0000 7f7192ffd700 1 -- 172.21.15.46:0/3369068047 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f71a8019590 con 0x555d451e9380 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.929+0000 7f71c6106b80 1 -- 172.21.15.46:0/3369068047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d451dcad0 con 0x555d451e9380 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.929+0000 7f7192ffd700 1 -- 172.21.15.46:0/3369068047 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a8019750 con 0x555d451e9380 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.929+0000 7f71c6106b80 10 client.25502 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.929+0000 7f71c6106b80 1 -- 172.21.15.46:0/3369068047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555d44f69b60 con 0x555d451e9380 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.930+0000 7f7192ffd700 1 -- 172.21.15.46:0/3369068047 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a8026020 con 0x555d451e9380 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.930+0000 7f7192ffd700 1 -- 172.21.15.46:0/3369068047 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f71a8021430 con 0x555d451e9380 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.930+0000 7f71c6106b80 10 client.25502 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:31.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.930+0000 7f71c6106b80 10 client.25502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.930+0000 7f71c6106b80 20 client.25502 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.930+0000 7f71c6106b80 1 --2- 172.21.15.46:0/3369068047 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d45203ed0 0x555d452062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.930+0000 7f71c6106b80 4 client.25502 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:36:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.930+0000 7f71c6106b80 1 -- 172.21.15.46:0/3369068047 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d45134700 con 0x555d45203ed0 2024-03-18T20:36:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.931+0000 7f71b37fe700 1 --2- 172.21.15.46:0/3369068047 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d45203ed0 0x555d452062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.931+0000 7f7192ffd700 10 client.25502 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:31.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.931+0000 7f71b37fe700 1 --2- 172.21.15.46:0/3369068047 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d45203ed0 0x555d452062f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a1315f7cdc29e16 server_cookie=4bef38c1dd330291 in_seq=0 out_seq=0 2024-03-18T20:36:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f7192ffd700 1 -- 172.21.15.46:0/3369068047 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555d45134700 con 0x555d45203ed0 2024-03-18T20:36:31.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f7192ffd700 10 client.25502 handle_command_reply: tid=0 2024-03-18T20:36:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f71c6106b80 1 client.25502 shutdown 2024-03-18T20:36:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f71c6106b80 1 -- 172.21.15.46:0/3369068047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 msgr2=0x555d451f1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f71c6106b80 1 --2- 172.21.15.46:0/3369068047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451f1510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f71a8011ce0 tx=0x7f71a800f6a0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f71c6106b80 1 -- 172.21.15.46:0/3369068047 shutdown_connections 2024-03-18T20:36:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f71c6106b80 1 --2- 172.21.15.46:0/3369068047 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d45203ed0 0x555d452062f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f71c6106b80 1 --2- 172.21.15.46:0/3369068047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451f4220 0x555d451f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f71c6106b80 1 --2- 172.21.15.46:0/3369068047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e8a60 0x555d451f0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f71c6106b80 1 --2- 172.21.15.46:0/3369068047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451f1510 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f71c6106b80 1 -- 172.21.15.46:0/3369068047 >> 172.21.15.46:0/3369068047 conn(0x555d4515f220 msgr2=0x555d451e4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:31.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f7192ffd700 0 client.25502 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:31.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f71c6106b80 1 -- 172.21.15.46:0/3369068047 shutdown_connections 2024-03-18T20:36:31.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f71c6106b80 1 -- 172.21.15.46:0/3369068047 wait complete. 2024-03-18T20:36:31.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.932+0000 7f71c6106b80 20 client.25502 trim_cache size 0 max 16384 2024-03-18T20:36:31.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.963+0000 7f71c6106b80 1 Processor -- start 2024-03-18T20:36:31.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71c6106b80 1 -- start start 2024-03-18T20:36:31.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71c6106b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451e8a60 0x555d451ee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71c6106b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451ee790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71c6106b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451f4220 0x555d451ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71c6106b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d451af940 con 0x555d451e8a60 2024-03-18T20:36:31.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71c6106b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d451df210 con 0x555d451e9380 2024-03-18T20:36:31.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71c6106b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d451e6470 con 0x555d451f4220 2024-03-18T20:36:31.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71b2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451ee790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71b2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451ee790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37436/0 (socket says 172.21.15.46:37436) 2024-03-18T20:36:31.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451f4220 0x555d451ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71b2ffd700 1 -- 172.21.15.46:0/1753090805 learned_addr learned my addr 172.21.15.46:0/1753090805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:31.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71b2ffd700 1 -- 172.21.15.46:0/1753090805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451f4220 msgr2=0x555d451ef8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71b2ffd700 1 --2- 172.21.15.46:0/1753090805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451f4220 0x555d451ef8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71b2ffd700 1 -- 172.21.15.46:0/1753090805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451e8a60 msgr2=0x555d451ee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71b2ffd700 1 --2- 172.21.15.46:0/1753090805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451e8a60 0x555d451ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.964+0000 7f71b2ffd700 1 -- 172.21.15.46:0/1753090805 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d45101ea0 con 0x555d451e9380 2024-03-18T20:36:31.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.965+0000 7f71b2ffd700 1 --2- 172.21.15.46:0/1753090805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451ee790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f71a80057a0 tx=0x7f71a80057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acf5320a777e9523 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:31.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.965+0000 7f719a7fc700 1 -- 172.21.15.46:0/1753090805 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8021c70 con 0x555d451e9380 2024-03-18T20:36:31.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.965+0000 7f719a7fc700 1 -- 172.21.15.46:0/1753090805 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a8026040 con 0x555d451e9380 2024-03-18T20:36:31.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.965+0000 7f719a7fc700 1 -- 172.21.15.46:0/1753090805 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a802bcf0 con 0x555d451e9380 2024-03-18T20:36:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.966+0000 7f71c6106b80 1 -- 172.21.15.46:0/1753090805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 msgr2=0x555d451ee790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.966+0000 7f71c6106b80 1 --2- 172.21.15.46:0/1753090805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451ee790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f71a80057a0 tx=0x7f71a80057d0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.966+0000 7f71c6106b80 1 -- 172.21.15.46:0/1753090805 shutdown_connections 2024-03-18T20:36:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.966+0000 7f71c6106b80 1 --2- 172.21.15.46:0/1753090805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451e8a60 0x555d451ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.966+0000 7f71c6106b80 1 --2- 172.21.15.46:0/1753090805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451f4220 0x555d451ef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.966+0000 7f71c6106b80 1 --2- 172.21.15.46:0/1753090805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451ee790 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.966+0000 7f71c6106b80 1 -- 172.21.15.46:0/1753090805 >> 172.21.15.46:0/1753090805 conn(0x555d4515f220 msgr2=0x555d451ec2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.967+0000 7f71c6106b80 1 -- 172.21.15.46:0/1753090805 shutdown_connections 2024-03-18T20:36:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.967+0000 7f71c6106b80 1 -- 172.21.15.46:0/1753090805 wait complete. 2024-03-18T20:36:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.967+0000 7f71c6106b80 1 -- start start 2024-03-18T20:36:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.967+0000 7f71c6106b80 1 Processor -- start 2024-03-18T20:36:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.968+0000 7f71c6106b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e8a60 0x555d451fab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.968+0000 7f71c6106b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.968+0000 7f71c6106b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451f4220 0x555d451f9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.968+0000 7f71c6106b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d451fb400 con 0x555d451f4220 2024-03-18T20:36:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.968+0000 7f71c6106b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d451f9a70 con 0x555d451e9380 2024-03-18T20:36:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.968+0000 7f71c6106b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d451f9c30 con 0x555d451e8a60 2024-03-18T20:36:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.968+0000 7f71c6106b80 10 client.0 _ll_register_callbacks cb 0x555d44f433a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.968+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451f4220 0x555d451f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.968+0000 7f71b3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451f4220 0x555d451f9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34690/0 (socket says 172.21.15.46:34690) 2024-03-18T20:36:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.968+0000 7f71b3fff700 1 -- 172.21.15.46:0/4232047444 learned_addr learned my addr 172.21.15.46:0/4232047444 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.968+0000 7f71b37fe700 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e8a60 0x555d451fab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.969+0000 7f71b37fe700 1 -- 172.21.15.46:0/4232047444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 msgr2=0x555d451f8ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.969+0000 7f71b37fe700 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451f8ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.969+0000 7f71b37fe700 1 -- 172.21.15.46:0/4232047444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451f4220 msgr2=0x555d451f9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.969+0000 7f71b37fe700 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451f4220 0x555d451f9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.969+0000 7f71b37fe700 1 -- 172.21.15.46:0/4232047444 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d44f69b60 con 0x555d451e8a60 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.969+0000 7f71b3fff700 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451f4220 0x555d451f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.969+0000 7f71b37fe700 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e8a60 0x555d451fab20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71a40097d0 tx=0x7f71a4005790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7275961d4206983 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.969+0000 7f7192ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.969+0000 7f7192ffd700 1 -- 172.21.15.46:0/4232047444 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a402d040 con 0x555d451e8a60 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.969+0000 7f71c6106b80 1 -- 172.21.15.46:0/4232047444 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71a4016070 con 0x555d451e8a60 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.969+0000 7f7192ffd700 1 -- 172.21.15.46:0/4232047444 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f71a4018070 con 0x555d451e8a60 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.970+0000 7f7192ffd700 1 -- 172.21.15.46:0/4232047444 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a4031730 con 0x555d451e8a60 2024-03-18T20:36:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.970+0000 7f71c6106b80 10 client.25498 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:31.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.970+0000 7f71c6106b80 1 -- 172.21.15.46:0/4232047444 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555d44fc1cd0 con 0x555d451e8a60 2024-03-18T20:36:31.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.970+0000 7f7192ffd700 1 -- 172.21.15.46:0/4232047444 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a403e020 con 0x555d451e8a60 2024-03-18T20:36:31.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.970+0000 7f7192ffd700 1 -- 172.21.15.46:0/4232047444 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f71a4030030 con 0x555d451e8a60 2024-03-18T20:36:31.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.970+0000 7f71c6106b80 10 client.25498 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:31.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.970+0000 7f71c6106b80 10 client.25498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.970+0000 7f71c6106b80 20 client.25498 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.970+0000 7f71c6106b80 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d45261930 0x555d45263d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.970+0000 7f71c6106b80 4 client.25498 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.970+0000 7f71c6106b80 1 -- 172.21.15.46:0/4232047444 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d44f8acf0 con 0x555d45261930 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.971+0000 7f71b2ffd700 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d45261930 0x555d45263d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.971+0000 7f7192ffd700 10 client.25498 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.971+0000 7f71b2ffd700 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d45261930 0x555d45263d50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a637b9438febcfb3 server_cookie=e5672f3d9d03ddb9 in_seq=0 out_seq=0 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.971+0000 7f7192ffd700 1 -- 172.21.15.46:0/4232047444 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555d44f8acf0 con 0x555d45261930 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.971+0000 7f7192ffd700 10 client.25498 handle_command_reply: tid=0 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.971+0000 7f71c6106b80 1 client.25498 shutdown 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.972+0000 7f71c6106b80 1 -- 172.21.15.46:0/4232047444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e8a60 msgr2=0x555d451fab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.972+0000 7f71c6106b80 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e8a60 0x555d451fab20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71a40097d0 tx=0x7f71a4005790 comp rx=0 tx=0).stop 2024-03-18T20:36:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.972+0000 7f71c6106b80 1 -- 172.21.15.46:0/4232047444 shutdown_connections 2024-03-18T20:36:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.972+0000 7f71c6106b80 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d45261930 0x555d45263d50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.972+0000 7f71c6106b80 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d451f4220 0x555d451f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.972+0000 7f71c6106b80 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d451e8a60 0x555d451fab20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.972+0000 7f7192ffd700 0 client.25498 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.972+0000 7f71c6106b80 1 --2- 172.21.15.46:0/4232047444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d451e9380 0x555d451f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.972+0000 7f71c6106b80 1 -- 172.21.15.46:0/4232047444 >> 172.21.15.46:0/4232047444 conn(0x555d4515f220 msgr2=0x555d451ef8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.972+0000 7f71c6106b80 1 -- 172.21.15.46:0/4232047444 shutdown_connections 2024-03-18T20:36:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.972+0000 7f71c6106b80 1 -- 172.21.15.46:0/4232047444 wait complete. 2024-03-18T20:36:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.972+0000 7f71c6106b80 20 client.25498 trim_cache size 0 max 16384 2024-03-18T20:36:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:36:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:36:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:36:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "9392ff0f-bd59-49f0-b019-b832f4f615c1": { 2024-03-18T20:36:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:36:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "9392ff0f-bd59-49f0-b019-b832f4f615c1", 2024-03-18T20:36:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:36:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:36:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:36:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:36:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.973+0000 7f71b9f71700 1 -- 172.21.15.46:0/3056981327 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f719c0a1320 msgr2=0x7f719c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.973+0000 7f71b9f71700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f719c0a1320 0x7f719c0a37e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009bf0 tx=0x7f71a4009260 comp rx=0 tx=0).stop 2024-03-18T20:36:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.973+0000 7f71b9f71700 1 -- 172.21.15.46:0/3056981327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b419bc20 msgr2=0x7f71b419c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.973+0000 7f71b9f71700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b419bc20 0x7f71b419c0a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0062a0 tx=0x7f71ac00bd70 comp rx=0 tx=0).stop 2024-03-18T20:36:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.973+0000 7f71b9f71700 1 -- 172.21.15.46:0/3056981327 shutdown_connections 2024-03-18T20:36:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.973+0000 7f71b9f71700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f719c0a1320 0x7f719c0a37e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.973+0000 7f71b9f71700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71b419bc20 0x7f71b419c0a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.973+0000 7f71b9f71700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b4100f70 0x7f71b4197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.973+0000 7f71b9f71700 1 --2- 172.21.15.46:0/3056981327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71b4106090 0x7f71b4197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:31.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.973+0000 7f71b9f71700 1 -- 172.21.15.46:0/3056981327 >> 172.21.15.46:0/3056981327 conn(0x7f71b40fc7f0 msgr2=0x7f71b40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:31.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.973+0000 7f71b9f71700 1 -- 172.21.15.46:0/3056981327 shutdown_connections 2024-03-18T20:36:31.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:36:31.973+0000 7f71b9f71700 1 -- 172.21.15.46:0/3056981327 wait complete. 2024-03-18T20:36:31.991 INFO:tasks.cephfs.filesystem:scrub status for tag:9392ff0f-bd59-49f0-b019-b832f4f615c1 - {'path': '/', 'tag': '9392ff0f-bd59-49f0-b019-b832f4f615c1', 'options': 'recursive,force'} 2024-03-18T20:36:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:32 smithi073 ceph-mon[47384]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-18T20:36:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:32 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:36:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:32 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:36:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:32 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:36:32.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:32 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:36:32.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:32 smithi155 ceph-mon[48344]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-18T20:36:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:32 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:36:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:32 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:36:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:32 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:36:32.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:32 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:36:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:32 smithi046 ceph-mon[43495]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-03-18T20:36:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:32 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:36:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:32 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:36:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:32 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:36:32.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:32 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:36:33.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:33 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:36:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:33 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:36:33.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:33 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:36:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:34 smithi073 ceph-mon[47384]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2024-03-18T20:36:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:34 smithi155 ceph-mon[48344]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2024-03-18T20:36:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:34 smithi046 ceph-mon[43495]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 17 op/s 2024-03-18T20:36:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.128+0000 7fee100e5700 1 -- 172.21.15.46:0/2263024508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee080680e0 msgr2=0x7fee08068560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.128+0000 7fee100e5700 1 --2- 172.21.15.46:0/2263024508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee080680e0 0x7fee08068560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fee0000b0a0 tx=0x7fee0000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.128+0000 7fee100e5700 1 -- 172.21.15.46:0/2263024508 shutdown_connections 2024-03-18T20:36:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.128+0000 7fee100e5700 1 --2- 172.21.15.46:0/2263024508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08105c40 0x7fee081036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.128+0000 7fee100e5700 1 --2- 172.21.15.46:0/2263024508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee080680e0 0x7fee08068560 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.128+0000 7fee100e5700 1 --2- 172.21.15.46:0/2263024508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee08103c00 0x7fee08104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.128+0000 7fee100e5700 1 -- 172.21.15.46:0/2263024508 >> 172.21.15.46:0/2263024508 conn(0x7fee08076d50 msgr2=0x7fee08077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.129+0000 7fee100e5700 1 -- 172.21.15.46:0/2263024508 shutdown_connections 2024-03-18T20:36:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.129+0000 7fee100e5700 1 -- 172.21.15.46:0/2263024508 wait complete. 2024-03-18T20:36:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.129+0000 7fee100e5700 1 Processor -- start 2024-03-18T20:36:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.129+0000 7fee100e5700 1 -- start start 2024-03-18T20:36:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.130+0000 7fee100e5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee080680e0 0x7fee081a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.130+0000 7fee100e5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee08103c00 0x7fee081a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.130+0000 7fee100e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08105c40 0x7fee0819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.130+0000 7fee100e5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0819de10 con 0x7fee08105c40 2024-03-18T20:36:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.130+0000 7fee100e5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0819df80 con 0x7fee08103c00 2024-03-18T20:36:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.130+0000 7fee100e5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0819e140 con 0x7fee080680e0 2024-03-18T20:36:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.130+0000 7fee0e682700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08105c40 0x7fee0819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.130+0000 7fee0e682700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08105c40 0x7fee0819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40378/0 (socket says 172.21.15.46:40378) 2024-03-18T20:36:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.130+0000 7fee0e682700 1 -- 172.21.15.46:0/4228319326 learned_addr learned my addr 172.21.15.46:0/4228319326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.130+0000 7fee0de81700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee080680e0 0x7fee081a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.131+0000 7fee0e682700 1 -- 172.21.15.46:0/4228319326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee080680e0 msgr2=0x7fee081a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.131+0000 7fee0e682700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee080680e0 0x7fee081a35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.131+0000 7fee0e682700 1 -- 172.21.15.46:0/4228319326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee08103c00 msgr2=0x7fee081a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.131+0000 7fee0d680700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee08103c00 0x7fee081a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.131+0000 7fee0e682700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee08103c00 0x7fee081a3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.131+0000 7fee0e682700 1 -- 172.21.15.46:0/4228319326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee00009d00 con 0x7fee08105c40 2024-03-18T20:36:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.131+0000 7fee0e682700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08105c40 0x7fee0819d640 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fee0000b070 tx=0x7fee00007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.131+0000 7fedfeffd700 1 -- 172.21.15.46:0/4228319326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0000e040 con 0x7fee08105c40 2024-03-18T20:36:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.131+0000 7fedfeffd700 1 -- 172.21.15.46:0/4228319326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee0000be40 con 0x7fee08105c40 2024-03-18T20:36:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.131+0000 7fedfeffd700 1 -- 172.21.15.46:0/4228319326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee000052a0 con 0x7fee08105c40 2024-03-18T20:36:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.131+0000 7fee100e5700 1 -- 172.21.15.46:0/4228319326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0819e3c0 con 0x7fee08105c40 2024-03-18T20:36:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.131+0000 7fee100e5700 1 -- 172.21.15.46:0/4228319326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee081aa520 con 0x7fee08105c40 2024-03-18T20:36:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.136+0000 7fedfeffd700 1 -- 172.21.15.46:0/4228319326 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 136041+0+0 (secure 0 0 0) 0x7fee0001d470 con 0x7fee08105c40 2024-03-18T20:36:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.136+0000 7fee1c27ab80 1 Processor -- start 2024-03-18T20:36:36.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.136+0000 7fee1c27ab80 1 -- start start 2024-03-18T20:36:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.136+0000 7fee1c27ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29f570 0x55a22c29f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.136+0000 7fee1c27ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29fe90 0x55a22c2aa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee1c27ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 0x55a22c2ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee1c27ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22c1f9930 con 0x55a22c29f570 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee1c27ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22c1f72f0 con 0x55a22c29fe90 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee1c27ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22c1f2be0 con 0x55a22c2aadb0 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee0d680700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29f570 0x55a22c29f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee0d680700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29f570 0x55a22c29f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40388/0 (socket says 172.21.15.46:40388) 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee0d680700 1 -- 172.21.15.46:0/1654649064 learned_addr learned my addr 172.21.15.46:0/1654649064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fedfeffd700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fedf40a13a0 0x7fedf40a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee0d680700 1 --2- 172.21.15.46:0/1654649064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 0x55a22c2ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee0d680700 1 -- 172.21.15.46:0/1654649064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 msgr2=0x55a22c2ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee0d680700 1 --2- 172.21.15.46:0/1654649064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 0x55a22c2ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee0d680700 1 -- 172.21.15.46:0/1654649064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29fe90 msgr2=0x55a22c2aa870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee0d680700 1 --2- 172.21.15.46:0/1654649064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29fe90 0x55a22c2aa870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.137+0000 7fee0d680700 1 -- 172.21.15.46:0/1654649064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a22c074350 con 0x55a22c29f570 2024-03-18T20:36:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.138+0000 7fedfeffd700 1 -- 172.21.15.46:0/4228319326 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fee000cdb70 con 0x7fee08105c40 2024-03-18T20:36:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.138+0000 7fee0d680700 1 --2- 172.21.15.46:0/1654649064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29f570 0x55a22c29f950 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fedf8014490 tx=0x7fedf80097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f69ff5ff252a5af server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.138+0000 7fee0e682700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fedf40a13a0 0x7fedf40a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.138+0000 7fedf3fff700 1 -- 172.21.15.46:0/1654649064 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8014da0 con 0x55a22c29f570 2024-03-18T20:36:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.138+0000 7fedf3fff700 1 -- 172.21.15.46:0/1654649064 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedf80044e0 con 0x55a22c29f570 2024-03-18T20:36:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.138+0000 7fedf3fff700 1 -- 172.21.15.46:0/1654649064 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8011470 con 0x55a22c29f570 2024-03-18T20:36:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.139+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1654649064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29f570 msgr2=0x55a22c29f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.139+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/1654649064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29f570 0x55a22c29f950 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fedf8014490 tx=0x7fedf80097f0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.139+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1654649064 shutdown_connections 2024-03-18T20:36:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.139+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/1654649064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29f570 0x55a22c29f950 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.139+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/1654649064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 0x55a22c2ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.139+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/1654649064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29fe90 0x55a22c2aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.139+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1654649064 >> 172.21.15.46:0/1654649064 conn(0x55a22c2161c0 msgr2=0x55a22c29ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.139+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1654649064 shutdown_connections 2024-03-18T20:36:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.139+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1654649064 wait complete. 2024-03-18T20:36:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.139+0000 7fee1c27ab80 1 -- start start 2024-03-18T20:36:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.139+0000 7fee1c27ab80 1 Processor -- start 2024-03-18T20:36:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.140+0000 7fee1c27ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 0x55a22c2aa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.140+0000 7fee1c27ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.140+0000 7fee1c27ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 0x55a22c2a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.140+0000 7fee1c27ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22c18ed20 con 0x55a22c29fe90 2024-03-18T20:36:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.140+0000 7fee1c27ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22c18e9c0 con 0x55a22c29f570 2024-03-18T20:36:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.140+0000 7fee1c27ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22c076640 con 0x55a22c2aadb0 2024-03-18T20:36:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.140+0000 7fee1c27ab80 10 client.0 _ll_register_callbacks cb 0x55a22bffa340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.140+0000 7fee0e682700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fedf40a13a0 0x7fedf40a3860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fee000cb410 tx=0x7fee0001dbf0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.140+0000 7fee0de81700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.140+0000 7fee0d680700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 0x55a22c2aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee0d680700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 0x55a22c2aa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:36716/0 (socket says 172.21.15.46:36716) 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee0de81700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2a79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40392/0 (socket says 172.21.15.46:40392) 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee0d680700 1 -- 172.21.15.46:0/1266582043 learned_addr learned my addr 172.21.15.46:0/1266582043 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee0d680700 1 --2- 172.21.15.46:0/1266582043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 0x55a22c2a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee0d680700 1 -- 172.21.15.46:0/1266582043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 msgr2=0x55a22c2a7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee0d680700 1 --2- 172.21.15.46:0/1266582043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 0x55a22c2a7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee0d680700 1 -- 172.21.15.46:0/1266582043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 msgr2=0x55a22c2a79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee0d680700 1 --2- 172.21.15.46:0/1266582043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2a79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee0d680700 1 -- 172.21.15.46:0/1266582043 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf8009cd0 con 0x55a22c29f570 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee0d680700 1 --2- 172.21.15.46:0/1266582043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 0x55a22c2aa700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fedf8014fd0 tx=0x7fedf8016040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c162c1816f9b775 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fedf17fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:36:36.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fedf17fa700 1 -- 172.21.15.46:0/1266582043 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8005050 con 0x55a22c29f570 2024-03-18T20:36:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fedf17fa700 1 -- 172.21.15.46:0/1266582043 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fedf80051b0 con 0x55a22c29f570 2024-03-18T20:36:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1266582043 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a22c2936c0 con 0x55a22c29f570 2024-03-18T20:36:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fedf17fa700 1 -- 172.21.15.46:0/1266582043 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedf8005370 con 0x55a22c29f570 2024-03-18T20:36:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee1c27ab80 10 client.25511 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.141+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1266582043 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a22c020b00 con 0x55a22c29f570 2024-03-18T20:36:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.142+0000 7fedf17fa700 1 -- 172.21.15.46:0/1266582043 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a22c2936c0 con 0x55a22c29f570 2024-03-18T20:36:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.142+0000 7fedf17fa700 1 -- 172.21.15.46:0/1266582043 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fedf8004750 con 0x55a22c29f570 2024-03-18T20:36:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.142+0000 7fee1c27ab80 10 client.25511 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.142+0000 7fee1c27ab80 10 client.25511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.142+0000 7fee1c27ab80 20 client.25511 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.143+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/1266582043 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a22c2ba7d0 0x55a22c2bcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.143+0000 7fee1c27ab80 4 client.25511 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.143+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1266582043 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a22c1eb6a0 con 0x55a22c2ba7d0 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.143+0000 7fee0de81700 1 --2- 172.21.15.46:0/1266582043 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a22c2ba7d0 0x55a22c2bcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.143+0000 7fee0de81700 1 --2- 172.21.15.46:0/1266582043 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a22c2ba7d0 0x55a22c2bcbc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9661be797cbba16b server_cookie=de73a684fba7a61d in_seq=0 out_seq=0 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.143+0000 7fedf17fa700 10 client.25511 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.144+0000 7fedf17fa700 1 -- 172.21.15.46:0/1266582043 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a22c1eb6a0 con 0x55a22c2ba7d0 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.144+0000 7fedf17fa700 10 client.25511 handle_command_reply: tid=0 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.144+0000 7fee1c27ab80 1 client.25511 shutdown 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.144+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1266582043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 msgr2=0x55a22c2aa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.144+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/1266582043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 0x55a22c2aa700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fedf8014fd0 tx=0x7fedf8016040 comp rx=0 tx=0).stop 2024-03-18T20:36:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.144+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1266582043 shutdown_connections 2024-03-18T20:36:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.144+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/1266582043 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a22c2ba7d0 0x55a22c2bcbc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.144+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/1266582043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 0x55a22c2a7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.144+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/1266582043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 0x55a22c2aa700 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.144+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/1266582043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.144+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1266582043 >> 172.21.15.46:0/1266582043 conn(0x55a22c2161c0 msgr2=0x55a22c29b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.144+0000 7fedf17fa700 0 client.25511 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.145+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1266582043 shutdown_connections 2024-03-18T20:36:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.145+0000 7fee1c27ab80 1 -- 172.21.15.46:0/1266582043 wait complete. 2024-03-18T20:36:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.145+0000 7fee1c27ab80 20 client.25511 trim_cache size 0 max 16384 2024-03-18T20:36:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee1c27ab80 1 Processor -- start 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee1c27ab80 1 -- start start 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee1c27ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c29f570 0x55a22c29bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee1c27ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee1c27ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c2aadb0 0x55a22c2b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee1c27ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22c2a0640 con 0x55a22c29fe90 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee1c27ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22c2a1140 con 0x55a22c2aadb0 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee1c27ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22c298200 con 0x55a22c29f570 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee0de81700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee0d680700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c2aadb0 0x55a22c2b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee0de81700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40404/0 (socket says 172.21.15.46:40404) 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee0d680700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c2aadb0 0x55a22c2b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:36730/0 (socket says 172.21.15.46:36730) 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.161+0000 7fee0de81700 1 -- 172.21.15.46:0/3839233587 learned_addr learned my addr 172.21.15.46:0/3839233587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.162+0000 7fee0d680700 1 --2- 172.21.15.46:0/3839233587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c29f570 0x55a22c29bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.162+0000 7fee0d680700 1 -- 172.21.15.46:0/3839233587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c29f570 msgr2=0x55a22c29bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.162+0000 7fee0d680700 1 --2- 172.21.15.46:0/3839233587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c29f570 0x55a22c29bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.162+0000 7fee0d680700 1 -- 172.21.15.46:0/3839233587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 msgr2=0x55a22c2ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.162+0000 7fee0d680700 1 --2- 172.21.15.46:0/3839233587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.162+0000 7fee0d680700 1 -- 172.21.15.46:0/3839233587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a22c1b8e40 con 0x55a22c2aadb0 2024-03-18T20:36:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.162+0000 7fee0d680700 1 --2- 172.21.15.46:0/3839233587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c2aadb0 0x55a22c2b1a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedf8014490 tx=0x7fedf8016040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c70e3c5b53035ff9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.162+0000 7fedfcff9700 1 -- 172.21.15.46:0/3839233587 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8009260 con 0x55a22c2aadb0 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.162+0000 7fedfcff9700 1 -- 172.21.15.46:0/3839233587 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedf8014b30 con 0x55a22c2aadb0 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.162+0000 7fedfcff9700 1 -- 172.21.15.46:0/3839233587 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf800db90 con 0x55a22c2aadb0 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.163+0000 7fee1c27ab80 1 -- 172.21.15.46:0/3839233587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c2aadb0 msgr2=0x55a22c2b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.163+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/3839233587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c2aadb0 0x55a22c2b1a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedf8014490 tx=0x7fedf8016040 comp rx=0 tx=0).stop 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.163+0000 7fee1c27ab80 1 -- 172.21.15.46:0/3839233587 shutdown_connections 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.163+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/3839233587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c2aadb0 0x55a22c2b1a30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.163+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/3839233587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c29f570 0x55a22c29bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.163+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/3839233587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.163+0000 7fee1c27ab80 1 -- 172.21.15.46:0/3839233587 >> 172.21.15.46:0/3839233587 conn(0x55a22c2161c0 msgr2=0x55a22c2a2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.164+0000 7fee1c27ab80 1 -- 172.21.15.46:0/3839233587 shutdown_connections 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.164+0000 7fee1c27ab80 1 -- 172.21.15.46:0/3839233587 wait complete. 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.164+0000 7fee1c27ab80 1 -- start start 2024-03-18T20:36:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.164+0000 7fee1c27ab80 1 Processor -- start 2024-03-18T20:36:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.165+0000 7fee1c27ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 0x55a22c2ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.165+0000 7fee1c27ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.165+0000 7fee1c27ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 0x55a22c2ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.165+0000 7fee1c27ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22c30d5c0 con 0x55a22c29fe90 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.165+0000 7fee1c27ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22c30d730 con 0x55a22c29f570 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.165+0000 7fee1c27ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22c30d8f0 con 0x55a22c2aadb0 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.165+0000 7fee1c27ab80 10 client.0 _ll_register_callbacks cb 0x55a22bffa340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.165+0000 7fee0de81700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.165+0000 7fee0de81700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40410/0 (socket says 172.21.15.46:40410) 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.165+0000 7fee0de81700 1 -- 172.21.15.46:0/2524301725 learned_addr learned my addr 172.21.15.46:0/2524301725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.166+0000 7fee0d680700 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 0x55a22c2ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.166+0000 7fee0d680700 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 0x55a22c2ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.166+0000 7fee0d680700 1 -- 172.21.15.46:0/2524301725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 msgr2=0x55a22c2ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.166+0000 7fee0d680700 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 0x55a22c2ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.166+0000 7fee0d680700 1 -- 172.21.15.46:0/2524301725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 msgr2=0x55a22c2adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.166+0000 7fee0d680700 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.166+0000 7fee0d680700 1 -- 172.21.15.46:0/2524301725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a22c020b00 con 0x55a22c29f570 2024-03-18T20:36:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.167+0000 7fee0d680700 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 0x55a22c2ada10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fedf80099b0 tx=0x7fedf8023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80e36fd8a3fc6dd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.167+0000 7fedf17fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:36:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.167+0000 7fedf17fa700 1 -- 172.21.15.46:0/2524301725 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf80058e0 con 0x55a22c29f570 2024-03-18T20:36:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.167+0000 7fedf17fa700 1 -- 172.21.15.46:0/2524301725 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fedf8000e40 con 0x55a22c29f570 2024-03-18T20:36:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.167+0000 7fee1c27ab80 1 -- 172.21.15.46:0/2524301725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedf80044e0 con 0x55a22c29f570 2024-03-18T20:36:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.167+0000 7fedf17fa700 1 -- 172.21.15.46:0/2524301725 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedf8014260 con 0x55a22c29f570 2024-03-18T20:36:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.167+0000 7fee1c27ab80 10 client.25517 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.167+0000 7fee1c27ab80 1 -- 172.21.15.46:0/2524301725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a22c078c70 con 0x55a22c29f570 2024-03-18T20:36:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.168+0000 7fedf17fa700 1 -- 172.21.15.46:0/2524301725 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedf80044e0 con 0x55a22c29f570 2024-03-18T20:36:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.169+0000 7fedf17fa700 1 -- 172.21.15.46:0/2524301725 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fedf8016d40 con 0x55a22c29f570 2024-03-18T20:36:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.169+0000 7fee1c27ab80 10 client.25517 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.169+0000 7fee1c27ab80 10 client.25517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.169+0000 7fee1c27ab80 20 client.25517 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.169+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a22c30eb60 0x55a22c310f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.169+0000 7fee1c27ab80 4 client.25517 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:36:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.169+0000 7fee1c27ab80 1 -- 172.21.15.46:0/2524301725 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a22c041c90 con 0x55a22c30eb60 2024-03-18T20:36:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.169+0000 7fee0de81700 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a22c30eb60 0x55a22c310f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.169+0000 7fee0de81700 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a22c30eb60 0x55a22c310f50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d88d4930e54ea9c8 server_cookie=78a8d59bc61e571b in_seq=0 out_seq=0 2024-03-18T20:36:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.169+0000 7fedf17fa700 10 client.25517 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.178+0000 7fedf17fa700 1 -- 172.21.15.46:0/2524301725 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a22c041c90 con 0x55a22c30eb60 2024-03-18T20:36:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.178+0000 7fedf17fa700 10 client.25517 handle_command_reply: tid=0 2024-03-18T20:36:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.178+0000 7fee1c27ab80 1 client.25517 shutdown 2024-03-18T20:36:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.178+0000 7fee1c27ab80 1 -- 172.21.15.46:0/2524301725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 msgr2=0x55a22c2ada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.178+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 0x55a22c2ada10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fedf80099b0 tx=0x7fedf8023040 comp rx=0 tx=0).stop 2024-03-18T20:36:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.178+0000 7fee1c27ab80 1 -- 172.21.15.46:0/2524301725 shutdown_connections 2024-03-18T20:36:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.178+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a22c30eb60 0x55a22c310f50 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.178+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a22c2aadb0 0x55a22c2ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.178+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a22c29f570 0x55a22c2ada10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.178+0000 7fee1c27ab80 1 --2- 172.21.15.46:0/2524301725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a22c29fe90 0x55a22c2adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.178+0000 7fee1c27ab80 1 -- 172.21.15.46:0/2524301725 >> 172.21.15.46:0/2524301725 conn(0x55a22c2161c0 msgr2=0x55a22c2a62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fedf17fa700 0 client.25517 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fee1c27ab80 1 -- 172.21.15.46:0/2524301725 shutdown_connections 2024-03-18T20:36:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fee1c27ab80 1 -- 172.21.15.46:0/2524301725 wait complete. 2024-03-18T20:36:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fee1c27ab80 20 client.25517 trim_cache size 0 max 16384 2024-03-18T20:36:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:36:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:36:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:36:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:36:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fee100e5700 1 -- 172.21.15.46:0/4228319326 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fedf40a13a0 msgr2=0x7fedf40a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fee100e5700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fedf40a13a0 0x7fedf40a3860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fee000cb410 tx=0x7fee0001dbf0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fee100e5700 1 -- 172.21.15.46:0/4228319326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08105c40 msgr2=0x7fee0819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fee100e5700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08105c40 0x7fee0819d640 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fee0000b070 tx=0x7fee00007930 comp rx=0 tx=0).stop 2024-03-18T20:36:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fee100e5700 1 -- 172.21.15.46:0/4228319326 shutdown_connections 2024-03-18T20:36:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fee100e5700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fedf40a13a0 0x7fedf40a3860 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fee100e5700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee08105c40 0x7fee0819d640 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fee100e5700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee080680e0 0x7fee081a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.179+0000 7fee100e5700 1 --2- 172.21.15.46:0/4228319326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee08103c00 0x7fee081a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.180+0000 7fee100e5700 1 -- 172.21.15.46:0/4228319326 >> 172.21.15.46:0/4228319326 conn(0x7fee08076d50 msgr2=0x7fee080fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.180+0000 7fee100e5700 1 -- 172.21.15.46:0/4228319326 shutdown_connections 2024-03-18T20:36:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:36.180+0000 7fee100e5700 1 -- 172.21.15.46:0/4228319326 wait complete. 2024-03-18T20:36:36.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:35 smithi073 ceph-mon[47384]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-03-18T20:36:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:35 smithi155 ceph-mon[48344]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-03-18T20:36:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:35 smithi046 ceph-mon[43495]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-03-18T20:36:38.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:38 smithi073 ceph-mon[47384]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-03-18T20:36:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:38 smithi155 ceph-mon[48344]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-03-18T20:36:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:38 smithi046 ceph-mon[43495]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s 2024-03-18T20:36:40.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:40 smithi073 ceph-mon[47384]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 426 B/s wr, 14 op/s 2024-03-18T20:36:40.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:40 smithi155 ceph-mon[48344]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 426 B/s wr, 14 op/s 2024-03-18T20:36:40.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:40 smithi046 ceph-mon[43495]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 426 B/s wr, 14 op/s 2024-03-18T20:36:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:42 smithi073 ceph-mon[47384]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-18T20:36:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:42 smithi155 ceph-mon[48344]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-18T20:36:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:42 smithi046 ceph-mon[43495]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-18T20:36:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:44 smithi073 ceph-mon[47384]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 426 B/s wr, 14 op/s 2024-03-18T20:36:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:44 smithi155 ceph-mon[48344]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 426 B/s wr, 14 op/s 2024-03-18T20:36:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:44 smithi046 ceph-mon[43495]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 426 B/s wr, 14 op/s 2024-03-18T20:36:46.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:45 smithi073 ceph-mon[47384]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-18T20:36:46.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:45 smithi155 ceph-mon[48344]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-18T20:36:46.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:45 smithi046 ceph-mon[43495]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-18T20:36:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:48 smithi073 ceph-mon[47384]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-18T20:36:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:48 smithi155 ceph-mon[48344]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-18T20:36:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:48 smithi046 ceph-mon[43495]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-18T20:36:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.344+0000 7fee10d55700 1 -- 172.21.15.46:0/3826152320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee0c0680e0 msgr2=0x7fee0c068560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.344+0000 7fee10d55700 1 --2- 172.21.15.46:0/3826152320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee0c0680e0 0x7fee0c068560 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fee00009990 tx=0x7fee00009ca0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.345+0000 7fee10d55700 1 -- 172.21.15.46:0/3826152320 shutdown_connections 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.345+0000 7fee10d55700 1 --2- 172.21.15.46:0/3826152320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee0c068aa0 0x7fee0c10dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.345+0000 7fee10d55700 1 --2- 172.21.15.46:0/3826152320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0c106e10 0x7fee0c1071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.345+0000 7fee10d55700 1 --2- 172.21.15.46:0/3826152320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee0c0680e0 0x7fee0c068560 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.345+0000 7fee10d55700 1 -- 172.21.15.46:0/3826152320 >> 172.21.15.46:0/3826152320 conn(0x7fee0c076d50 msgr2=0x7fee0c077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.345+0000 7fee10d55700 1 -- 172.21.15.46:0/3826152320 shutdown_connections 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.345+0000 7fee10d55700 1 -- 172.21.15.46:0/3826152320 wait complete. 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee10d55700 1 Processor -- start 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee10d55700 1 -- start start 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee10d55700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee0c0680e0 0x7fee0c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee10d55700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0c068aa0 0x7fee0c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee10d55700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee0c106e10 0x7fee0c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee10d55700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c19e1c0 con 0x7fee0c106e10 2024-03-18T20:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee10d55700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c19e330 con 0x7fee0c068aa0 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee10d55700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c19e4f0 con 0x7fee0c0680e0 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee0ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee0c106e10 0x7fee0c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee0ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee0c106e10 0x7fee0c19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50424/0 (socket says 172.21.15.46:50424) 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1022269435 learned_addr learned my addr 172.21.15.46:0/1022269435 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee09d9b700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0c068aa0 0x7fee0c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.346+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee0c0680e0 0x7fee0c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.347+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1022269435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee0c0680e0 msgr2=0x7fee0c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.347+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee0c0680e0 0x7fee0c199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.347+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1022269435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0c068aa0 msgr2=0x7fee0c199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.347+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0c068aa0 0x7fee0c199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.347+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1022269435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee000095f0 con 0x7fee0c106e10 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.347+0000 7fee09d9b700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0c068aa0 0x7fee0c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.347+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee0c0680e0 0x7fee0c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.347+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee0c106e10 0x7fee0c19db80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fee0400ef30 tx=0x7fee0400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.347+0000 7fedfb7fe700 1 -- 172.21.15.46:0/1022269435 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04018070 con 0x7fee0c106e10 2024-03-18T20:36:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.347+0000 7fee10d55700 1 -- 172.21.15.46:0/1022269435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0c19e770 con 0x7fee0c106e10 2024-03-18T20:36:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.347+0000 7fee10d55700 1 -- 172.21.15.46:0/1022269435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c1a6010 con 0x7fee0c106e10 2024-03-18T20:36:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.350+0000 7fedfb7fe700 1 -- 172.21.15.46:0/1022269435 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee0400f040 con 0x7fee0c106e10 2024-03-18T20:36:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.350+0000 7fedfb7fe700 1 -- 172.21.15.46:0/1022269435 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0400ccb0 con 0x7fee0c106e10 2024-03-18T20:36:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.351+0000 7fedfb7fe700 1 -- 172.21.15.46:0/1022269435 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) v1 ==== 136041+0+0 (secure 0 0 0) 0x7fee04004020 con 0x7fee0c106e10 2024-03-18T20:36:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.352+0000 7fedfb7fe700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fedf40a1420 0x7fedf40a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.352+0000 7fedfb7fe700 1 -- 172.21.15.46:0/1022269435 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fee040cd7e0 con 0x7fee0c106e10 2024-03-18T20:36:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.352+0000 7fee1ceeab80 1 Processor -- start 2024-03-18T20:36:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fedf40a1420 0x7fedf40a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee1ceeab80 1 -- start start 2024-03-18T20:36:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee1ceeab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693f570 0x560c5693f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee1ceeab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693fe90 0x560c5694a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee1ceeab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5694adb0 0x560c5694d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee1ceeab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c56899930 con 0x560c5693f570 2024-03-18T20:36:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee1ceeab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c568972f0 con 0x560c5694adb0 2024-03-18T20:36:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee1ceeab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c56892be0 con 0x560c5693fe90 2024-03-18T20:36:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee0a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693f570 0x560c5693f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee0a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693f570 0x560c5693f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50438/0 (socket says 172.21.15.46:50438) 2024-03-18T20:36:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee0a59c700 1 -- 172.21.15.46:0/2424423629 learned_addr learned my addr 172.21.15.46:0/2424423629 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/2424423629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5694adb0 0x560c5694d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.353+0000 7fee09d9b700 1 --2- 172.21.15.46:0/2424423629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693fe90 0x560c5694a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.354+0000 7fee0ad9d700 1 -- 172.21.15.46:0/2424423629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693fe90 msgr2=0x560c5694a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.354+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/2424423629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693fe90 0x560c5694a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.354+0000 7fee0ad9d700 1 -- 172.21.15.46:0/2424423629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693f570 msgr2=0x560c5693f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.354+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/2424423629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693f570 0x560c5693f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.354+0000 7fee0ad9d700 1 -- 172.21.15.46:0/2424423629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c56714350 con 0x560c5694adb0 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.354+0000 7fee0a59c700 1 --2- 172.21.15.46:0/2424423629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693f570 0x560c5693f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.354+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/2424423629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5694adb0 0x560c5694d1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee040cae40 tx=0x7fee0409a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3fb1c6dc625ebba server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.354+0000 7fedf8ff9700 1 -- 172.21.15.46:0/2424423629 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0409aee0 con 0x560c5694adb0 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.354+0000 7fedf8ff9700 1 -- 172.21.15.46:0/2424423629 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee0409a7e0 con 0x560c5694adb0 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.354+0000 7fedf8ff9700 1 -- 172.21.15.46:0/2424423629 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0409aca0 con 0x560c5694adb0 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.355+0000 7fee1ceeab80 1 -- 172.21.15.46:0/2424423629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5694adb0 msgr2=0x560c5694d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.355+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/2424423629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5694adb0 0x560c5694d1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee040cae40 tx=0x7fee0409a320 comp rx=0 tx=0).stop 2024-03-18T20:36:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.355+0000 7fee1ceeab80 1 -- 172.21.15.46:0/2424423629 shutdown_connections 2024-03-18T20:36:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.355+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/2424423629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693f570 0x560c5693f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.355+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/2424423629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5694adb0 0x560c5694d1a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.355+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/2424423629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693fe90 0x560c5694a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.355+0000 7fee1ceeab80 1 -- 172.21.15.46:0/2424423629 >> 172.21.15.46:0/2424423629 conn(0x560c568b61c0 msgr2=0x560c5693ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.355+0000 7fee1ceeab80 1 -- 172.21.15.46:0/2424423629 shutdown_connections 2024-03-18T20:36:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.355+0000 7fee1ceeab80 1 -- 172.21.15.46:0/2424423629 wait complete. 2024-03-18T20:36:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee1ceeab80 1 -- start start 2024-03-18T20:36:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee1ceeab80 1 Processor -- start 2024-03-18T20:36:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee1ceeab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693f570 0x560c5694a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee1ceeab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693fe90 0x560c56947ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee1ceeab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 0x560c56948200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee1ceeab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5682ed20 con 0x560c5694adb0 2024-03-18T20:36:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee1ceeab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c5682e9c0 con 0x560c5693fe90 2024-03-18T20:36:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee1ceeab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c56716640 con 0x560c5693f570 2024-03-18T20:36:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee1ceeab80 10 client.0 _ll_register_callbacks cb 0x560c5669a340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee0ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 0x560c56948200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee0ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 0x560c56948200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50448/0 (socket says 172.21.15.46:50448) 2024-03-18T20:36:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1657306507 learned_addr learned my addr 172.21.15.46:0/1657306507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.356+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693f570 0x560c5694a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fee09d9b700 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693fe90 0x560c56947ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1657306507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693f570 msgr2=0x560c5694a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693f570 0x560c5694a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1657306507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693fe90 msgr2=0x560c56947ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693fe90 0x560c56947ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1657306507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedfc013860 con 0x560c5694adb0 2024-03-18T20:36:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fee09d9b700 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693fe90 0x560c56947ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:36:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 0x560c56948200 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fee04004dd0 tx=0x7fee0409afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f847026e1adc8d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fedf1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fedf1ffb700 1 -- 172.21.15.46:0/1657306507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04099ed0 con 0x560c5694adb0 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fedf1ffb700 1 -- 172.21.15.46:0/1657306507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fee0409aa40 con 0x560c5694adb0 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1657306507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c569336c0 con 0x560c5694adb0 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fedf1ffb700 1 -- 172.21.15.46:0/1657306507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee0409ac00 con 0x560c5694adb0 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fee1ceeab80 10 client.15759 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.357+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1657306507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fee0c1a6010 con 0x560c5694adb0 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.358+0000 7fedf1ffb700 1 -- 172.21.15.46:0/1657306507 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c569336c0 con 0x560c5694adb0 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.358+0000 7fedf1ffb700 1 -- 172.21.15.46:0/1657306507 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fee0407f050 con 0x560c5694adb0 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.358+0000 7fee1ceeab80 10 client.15759 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.358+0000 7fee1ceeab80 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.358+0000 7fee1ceeab80 20 client.15759 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.358+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560c56962dd0 0x560c569651f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.358+0000 7fee1ceeab80 4 client.15759 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.358+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1657306507 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c5688b6a0 con 0x560c56962dd0 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.359+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560c56962dd0 0x560c569651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.359+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fedf40a1420 0x7fedf40a38e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fedfc013d80 tx=0x7fedfc009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.359+0000 7fedf1ffb700 10 client.15759 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.359+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560c56962dd0 0x560c569651f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5bcb2b9780453e9 server_cookie=b61cd5dd65c8f285 in_seq=0 out_seq=0 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fedf1ffb700 1 -- 172.21.15.46:0/1657306507 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c5688b6a0 con 0x560c56962dd0 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fedf1ffb700 10 client.15759 handle_command_reply: tid=0 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fee1ceeab80 1 client.15759 shutdown 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1657306507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 msgr2=0x560c56948200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 0x560c56948200 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fee04004dd0 tx=0x7fee0409afd0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1657306507 shutdown_connections 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560c56962dd0 0x560c569651f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 0x560c56948200 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693f570 0x560c5694a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1657306507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693fe90 0x560c56947ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1657306507 >> 172.21.15.46:0/1657306507 conn(0x560c568b61c0 msgr2=0x560c5693b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fedf1ffb700 0 client.15759 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1657306507 shutdown_connections 2024-03-18T20:36:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.360+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1657306507 wait complete. 2024-03-18T20:36:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.361+0000 7fee1ceeab80 20 client.15759 trim_cache size 0 max 16384 2024-03-18T20:36:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.376+0000 7fee1ceeab80 1 Processor -- start 2024-03-18T20:36:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.376+0000 7fee1ceeab80 1 -- start start 2024-03-18T20:36:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.376+0000 7fee1ceeab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693f570 0x560c5693f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.376+0000 7fee1ceeab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693fe90 0x560c569424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.376+0000 7fee1ceeab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5694adb0 0x560c56951a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.376+0000 7fee1ceeab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c56935ca0 con 0x560c5693fe90 2024-03-18T20:36:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.376+0000 7fee1ceeab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c56941140 con 0x560c5693f570 2024-03-18T20:36:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.376+0000 7fee1ceeab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c56951fb0 con 0x560c5694adb0 2024-03-18T20:36:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee0a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693f570 0x560c5693f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee09d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693fe90 0x560c569424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee0ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5694adb0 0x560c56951a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee0ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5694adb0 0x560c56951a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40560/0 (socket says 172.21.15.46:40560) 2024-03-18T20:36:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee0a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693f570 0x560c5693f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:35678/0 (socket says 172.21.15.46:35678) 2024-03-18T20:36:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1998972536 learned_addr learned my addr 172.21.15.46:0/1998972536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee09d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693fe90 0x560c569424e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50458/0 (socket says 172.21.15.46:50458) 2024-03-18T20:36:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee0a59c700 1 -- 172.21.15.46:0/1998972536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5694adb0 msgr2=0x560c56951a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1998972536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5694adb0 0x560c56951a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee0a59c700 1 -- 172.21.15.46:0/1998972536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693fe90 msgr2=0x560c569424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1998972536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693fe90 0x560c569424e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee0a59c700 1 -- 172.21.15.46:0/1998972536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c566c0b00 con 0x560c5693f570 2024-03-18T20:36:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/1998972536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5694adb0 0x560c56951a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:36:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.377+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1998972536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693f570 0x560c5693f950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x560c56965a90 tx=0x7fedfc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e52ee20f3d47bc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.378+0000 7fedf97fa700 1 -- 172.21.15.46:0/1998972536 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc026070 con 0x560c5693f570 2024-03-18T20:36:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.378+0000 7fedf97fa700 1 -- 172.21.15.46:0/1998972536 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedfc010db0 con 0x560c5693f570 2024-03-18T20:36:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.378+0000 7fedf97fa700 1 -- 172.21.15.46:0/1998972536 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc021790 con 0x560c5693f570 2024-03-18T20:36:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.378+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1998972536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693f570 msgr2=0x560c5693f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.378+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1998972536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693f570 0x560c5693f950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x560c56965a90 tx=0x7fedfc0047c0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.378+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1998972536 shutdown_connections 2024-03-18T20:36:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.378+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1998972536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5694adb0 0x560c56951a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.378+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1998972536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693f570 0x560c5693f950 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.378+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1998972536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5693fe90 0x560c569424e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.378+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1998972536 >> 172.21.15.46:0/1998972536 conn(0x560c568b61c0 msgr2=0x560c56947350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.378+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1998972536 shutdown_connections 2024-03-18T20:36:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.379+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1998972536 wait complete. 2024-03-18T20:36:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.379+0000 7fee1ceeab80 1 -- start start 2024-03-18T20:36:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.379+0000 7fee1ceeab80 1 Processor -- start 2024-03-18T20:36:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.379+0000 7fee1ceeab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693f570 0x560c5694fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.379+0000 7fee1ceeab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693fe90 0x560c5694ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.379+0000 7fee1ceeab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 0x560c569504a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.379+0000 7fee1ceeab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c569b5c00 con 0x560c5694adb0 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.379+0000 7fee1ceeab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c569b5d70 con 0x560c5693f570 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.379+0000 7fee1ceeab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c569b5f30 con 0x560c5693fe90 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.379+0000 7fee1ceeab80 10 client.0 _ll_register_callbacks cb 0x7fedfc0134e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fee0ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 0x560c569504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fee0ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 0x560c569504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50470/0 (socket says 172.21.15.46:50470) 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1314457593 learned_addr learned my addr 172.21.15.46:0/1314457593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693f570 0x560c5694fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fee09d9b700 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693fe90 0x560c5694ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1314457593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693fe90 msgr2=0x560c5694ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693fe90 0x560c5694ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1314457593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693f570 msgr2=0x560c5694fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693f570 0x560c5694fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fee0ad9d700 1 -- 172.21.15.46:0/1314457593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c1a6010 con 0x560c5694adb0 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fee0ad9d700 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 0x560c569504a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fee04004dd0 tx=0x7fee040a2970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5494595c5c9d8798 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fedf1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fedf1ffb700 1 -- 172.21.15.46:0/1314457593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee040cad80 con 0x560c5694adb0 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1314457593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee0409a7e0 con 0x560c5694adb0 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.380+0000 7fedf1ffb700 1 -- 172.21.15.46:0/1314457593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fee0409e610 con 0x560c5694adb0 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.381+0000 7fedf1ffb700 1 -- 172.21.15.46:0/1314457593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee0409a320 con 0x560c5694adb0 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.381+0000 7fee1ceeab80 10 client.15765 fetch_fsmap learned FSMap version 27 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.381+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1314457593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fedfc010840 con 0x560c5694adb0 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.381+0000 7fedf1ffb700 1 -- 172.21.15.46:0/1314457593 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee0409a7e0 con 0x560c5694adb0 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.381+0000 7fedf1ffb700 1 -- 172.21.15.46:0/1314457593 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fee0407f070 con 0x560c5694adb0 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.381+0000 7fee1ceeab80 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:36:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.381+0000 7fee1ceeab80 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:36:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.381+0000 7fee1ceeab80 20 client.15765 populate_metadata read hostname 'smithi046' 2024-03-18T20:36:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.381+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560c569b7210 0x560c569b9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:36:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.381+0000 7fee1ceeab80 4 client.15765 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:36:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.381+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1314457593 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c566e1c90 con 0x560c569b7210 2024-03-18T20:36:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.381+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560c569b7210 0x560c569b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:36:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.382+0000 7fee0a59c700 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560c569b7210 0x560c569b9630 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea047909ef3edd12 server_cookie=8eb85cbf18c2ea26 in_seq=0 out_seq=0 2024-03-18T20:36:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.382+0000 7fedf1ffb700 10 client.15765 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.388+0000 7fedf1ffb700 1 -- 172.21.15.46:0/1314457593 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c566e1c90 con 0x560c569b7210 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.388+0000 7fedf1ffb700 10 client.15765 handle_command_reply: tid=0 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.388+0000 7fee1ceeab80 1 client.15765 shutdown 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.388+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1314457593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 msgr2=0x560c569504a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.388+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 0x560c569504a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fee04004dd0 tx=0x7fee040a2970 comp rx=0 tx=0).stop 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1314457593 shutdown_connections 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560c569b7210 0x560c569b9630 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c5694adb0 0x560c569504a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c5693f570 0x560c5694fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee1ceeab80 1 --2- 172.21.15.46:0/1314457593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c5693fe90 0x560c5694ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1314457593 >> 172.21.15.46:0/1314457593 conn(0x560c568b61c0 msgr2=0x560c5693a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fedf1ffb700 0 client.15765 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1314457593 shutdown_connections 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee1ceeab80 1 -- 172.21.15.46:0/1314457593 wait complete. 2024-03-18T20:36:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee1ceeab80 20 client.15765 trim_cache size 0 max 16384 2024-03-18T20:36:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:36:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:36:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:36:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:36:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee10d55700 1 -- 172.21.15.46:0/1022269435 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fedf40a1420 msgr2=0x7fedf40a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee10d55700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fedf40a1420 0x7fedf40a38e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fedfc013d80 tx=0x7fedfc009260 comp rx=0 tx=0).stop 2024-03-18T20:36:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee10d55700 1 -- 172.21.15.46:0/1022269435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee0c106e10 msgr2=0x7fee0c19db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:36:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.389+0000 7fee10d55700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee0c106e10 0x7fee0c19db80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fee0400ef30 tx=0x7fee0400c3a0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.390+0000 7fee10d55700 1 -- 172.21.15.46:0/1022269435 shutdown_connections 2024-03-18T20:36:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.390+0000 7fee10d55700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fedf40a1420 0x7fedf40a38e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.390+0000 7fee10d55700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee0c106e10 0x7fee0c19db80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.390+0000 7fee10d55700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee0c0680e0 0x7fee0c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.390+0000 7fee10d55700 1 --2- 172.21.15.46:0/1022269435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee0c068aa0 0x7fee0c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:36:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.390+0000 7fee10d55700 1 -- 172.21.15.46:0/1022269435 >> 172.21.15.46:0/1022269435 conn(0x7fee0c076d50 msgr2=0x7fee0c10be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:36:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.390+0000 7fee10d55700 1 -- 172.21.15.46:0/1022269435 shutdown_connections 2024-03-18T20:36:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:36:49.390+0000 7fee10d55700 1 -- 172.21.15.46:0/1022269435 wait complete. 2024-03-18T20:36:50.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:50 smithi073 ceph-mon[47384]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-03-18T20:36:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:50 smithi155 ceph-mon[48344]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-03-18T20:36:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:50 smithi046 ceph-mon[43495]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-03-18T20:36:52.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:52 smithi073 ceph-mon[47384]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-18T20:36:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:52 smithi155 ceph-mon[48344]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-18T20:36:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:52 smithi046 ceph-mon[43495]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-18T20:36:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:54 smithi073 ceph-mon[47384]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-18T20:36:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:54 smithi155 ceph-mon[48344]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-18T20:36:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:54 smithi046 ceph-mon[43495]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-18T20:36:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:55 smithi073 ceph-mon[47384]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-18T20:36:56.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:55 smithi155 ceph-mon[48344]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-18T20:36:56.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:55 smithi046 ceph-mon[43495]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-18T20:36:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:36:58 smithi073 ceph-mon[47384]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-18T20:36:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:36:58 smithi155 ceph-mon[48344]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-18T20:36:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:36:58 smithi046 ceph-mon[43495]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-18T20:37:00.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:00 smithi073 ceph-mon[47384]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-18T20:37:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:00 smithi155 ceph-mon[48344]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-18T20:37:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:00 smithi046 ceph-mon[43495]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-03-18T20:37:01.046 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:37:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:37:00.619+0000 7ff541ea3700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:37:01.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:37:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:37:00.619+0000 7ff541ea3700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:37:01.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:37:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:37:00.619+0000 7ff541ea3700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:37:01.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:37:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:37:00.619+0000 7ff541ea3700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:37:01.047 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:37:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:37:00.619+0000 7ff541ea3700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T20:37:01.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:37:01.993 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:37:02.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.138+0000 7f87f09c6700 1 -- 172.21.15.46:0/234343249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec105ce0 msgr2=0x7f87ec1060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.138+0000 7f87f09c6700 1 --2- 172.21.15.46:0/234343249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec105ce0 0x7f87ec1060c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009990 tx=0x7f87dc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.138+0000 7f87f09c6700 1 -- 172.21.15.46:0/234343249 shutdown_connections 2024-03-18T20:37:02.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.138+0000 7f87f09c6700 1 --2- 172.21.15.46:0/234343249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec105ce0 0x7f87ec1060c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.138+0000 7f87f09c6700 1 --2- 172.21.15.46:0/234343249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87ec0687f0 0x7f87ec10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.138+0000 7f87f09c6700 1 --2- 172.21.15.46:0/234343249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87ec067e30 0x7f87ec0682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.138+0000 7f87f09c6700 1 -- 172.21.15.46:0/234343249 >> 172.21.15.46:0/234343249 conn(0x7f87ec076d40 msgr2=0x7f87ec077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.138+0000 7f87f09c6700 1 -- 172.21.15.46:0/234343249 shutdown_connections 2024-03-18T20:37:02.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.138+0000 7f87f09c6700 1 -- 172.21.15.46:0/234343249 wait complete. 2024-03-18T20:37:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.139+0000 7f87f09c6700 1 Processor -- start 2024-03-18T20:37:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.139+0000 7f87f09c6700 1 -- start start 2024-03-18T20:37:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.139+0000 7f87f09c6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87ec067e30 0x7f87ec0ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.139+0000 7f87f09c6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec0687f0 0x7f87ec1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.139+0000 7f87f09c6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87ec105ce0 0x7f87ec1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.139+0000 7f87f09c6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1a55b0 con 0x7f87ec0687f0 2024-03-18T20:37:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.139+0000 7f87f09c6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1a5720 con 0x7f87ec105ce0 2024-03-18T20:37:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.139+0000 7f87f09c6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1a58e0 con 0x7f87ec067e30 2024-03-18T20:37:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87e9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec0687f0 0x7f87ec1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87e9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec0687f0 0x7f87ec1001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60374/0 (socket says 172.21.15.46:60374) 2024-03-18T20:37:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87e9d9b700 1 -- 172.21.15.46:0/2930086820 learned_addr learned my addr 172.21.15.46:0/2930086820 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87ea59c700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87ec067e30 0x7f87ec0ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87ead9d700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87ec105ce0 0x7f87ec1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87e9d9b700 1 -- 172.21.15.46:0/2930086820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87ec067e30 msgr2=0x7f87ec0ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87e9d9b700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87ec067e30 0x7f87ec0ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87e9d9b700 1 -- 172.21.15.46:0/2930086820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87ec105ce0 msgr2=0x7f87ec1006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87e9d9b700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87ec105ce0 0x7f87ec1006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87e9d9b700 1 -- 172.21.15.46:0/2930086820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0095f0 con 0x7f87ec0687f0 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87ea59c700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87ec067e30 0x7f87ec0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87e9d9b700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec0687f0 0x7f87ec1001b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f87e000e9a0 tx=0x7f87e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87db7fe700 1 -- 172.21.15.46:0/2930086820 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0009790 con 0x7f87ec0687f0 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87db7fe700 1 -- 172.21.15.46:0/2930086820 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87e0010430 con 0x7f87ec0687f0 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87f09c6700 1 -- 172.21.15.46:0/2930086820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec1a5bc0 con 0x7f87ec0687f0 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.140+0000 7f87db7fe700 1 -- 172.21.15.46:0/2930086820 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0018680 con 0x7f87ec0687f0 2024-03-18T20:37:02.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.141+0000 7f87f09c6700 1 -- 172.21.15.46:0/2930086820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec1a60d0 con 0x7f87ec0687f0 2024-03-18T20:37:02.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.143+0000 7f87db7fe700 1 -- 172.21.15.46:0/2930086820 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f87e00105a0 con 0x7f87ec0687f0 2024-03-18T20:37:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.144+0000 7f87db7fe700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f87d40a13b0 0x7f87d40a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.144+0000 7f87db7fe700 1 -- 172.21.15.46:0/2930086820 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f87e0014070 con 0x7f87ec0687f0 2024-03-18T20:37:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.144+0000 7f87ea59c700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f87d40a13b0 0x7f87d40a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.144+0000 7f87ea59c700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f87d40a13b0 0x7f87d40a3870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0098e0 tx=0x7f87dc005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.144+0000 7f87fcb5bb80 1 Processor -- start 2024-03-18T20:37:02.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.145+0000 7f87fcb5bb80 1 -- start start 2024-03-18T20:37:02.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.145+0000 7f87fcb5bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a2a60 0x55b95d6a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.145+0000 7f87fcb5bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a3380 0x55b95d6adce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.145+0000 7f87fcb5bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6ae220 0x55b95d6b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.145+0000 7f87fcb5bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95d5fc990 con 0x55b95d6a3380 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.145+0000 7f87fcb5bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95d5fa350 con 0x55b95d6a2a60 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.145+0000 7f87fcb5bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95d5f5c40 con 0x55b95d6ae220 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.145+0000 7f87e9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a3380 0x55b95d6adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.145+0000 7f87e9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a3380 0x55b95d6adce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60376/0 (socket says 172.21.15.46:60376) 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.145+0000 7f87e9d9b700 1 -- 172.21.15.46:0/453590663 learned_addr learned my addr 172.21.15.46:0/453590663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.145+0000 7f87ea59c700 1 --2- 172.21.15.46:0/453590663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a2a60 0x55b95d6a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.145+0000 7f87ead9d700 1 --2- 172.21.15.46:0/453590663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6ae220 0x55b95d6b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.146+0000 7f87e9d9b700 1 -- 172.21.15.46:0/453590663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6ae220 msgr2=0x55b95d6b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.146+0000 7f87e9d9b700 1 --2- 172.21.15.46:0/453590663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6ae220 0x55b95d6b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.146+0000 7f87e9d9b700 1 -- 172.21.15.46:0/453590663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a2a60 msgr2=0x55b95d6a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.146+0000 7f87e9d9b700 1 --2- 172.21.15.46:0/453590663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a2a60 0x55b95d6a2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.146+0000 7f87e9d9b700 1 -- 172.21.15.46:0/453590663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b95d4773b0 con 0x55b95d6a3380 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.146+0000 7f87ead9d700 1 --2- 172.21.15.46:0/453590663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6ae220 0x55b95d6b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.146+0000 7f87e9d9b700 1 --2- 172.21.15.46:0/453590663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a3380 0x55b95d6adce0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f87e0099fc0 tx=0x7f87e0099a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f13127c3df96f04 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.146+0000 7f87d8ff9700 1 -- 172.21.15.46:0/453590663 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e009a030 con 0x55b95d6a3380 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.146+0000 7f87d8ff9700 1 -- 172.21.15.46:0/453590663 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87e009de60 con 0x55b95d6a3380 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.146+0000 7f87d8ff9700 1 -- 172.21.15.46:0/453590663 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e00a2020 con 0x55b95d6a3380 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.147+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/453590663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a3380 msgr2=0x55b95d6adce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.147+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/453590663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a3380 0x55b95d6adce0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f87e0099fc0 tx=0x7f87e0099a80 comp rx=0 tx=0).stop 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.147+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/453590663 shutdown_connections 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.147+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/453590663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6ae220 0x55b95d6b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.147+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/453590663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a2a60 0x55b95d6a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.147+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/453590663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a3380 0x55b95d6adce0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.147+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/453590663 >> 172.21.15.46:0/453590663 conn(0x55b95d619220 msgr2=0x55b95d69f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.147+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/453590663 shutdown_connections 2024-03-18T20:37:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.147+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/453590663 wait complete. 2024-03-18T20:37:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.147+0000 7f87fcb5bb80 1 -- start start 2024-03-18T20:37:02.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87fcb5bb80 1 Processor -- start 2024-03-18T20:37:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87fcb5bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6a2a60 0x55b95d6aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87fcb5bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d6ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87fcb5bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 0x55b95d6aba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87fcb5bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95d591d80 con 0x55b95d6ae220 2024-03-18T20:37:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87fcb5bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95d591a20 con 0x55b95d6a3380 2024-03-18T20:37:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87fcb5bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95d4796a0 con 0x55b95d6a2a60 2024-03-18T20:37:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87fcb5bb80 10 client.0 _ll_register_callbacks cb 0x55b95d3fd3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87ead9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 0x55b95d6aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87ead9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 0x55b95d6aba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60388/0 (socket says 172.21.15.46:60388) 2024-03-18T20:37:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87ead9d700 1 -- 172.21.15.46:0/3279512061 learned_addr learned my addr 172.21.15.46:0/3279512061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87ea59c700 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6a2a60 0x55b95d6aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.148+0000 7f87e9d9b700 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d6ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87ead9d700 1 -- 172.21.15.46:0/3279512061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6a2a60 msgr2=0x55b95d6aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87ead9d700 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6a2a60 0x55b95d6aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87ead9d700 1 -- 172.21.15.46:0/3279512061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 msgr2=0x55b95d6ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87ead9d700 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d6ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87ead9d700 1 -- 172.21.15.46:0/3279512061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec1a60d0 con 0x55b95d6ae220 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87ead9d700 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 0x55b95d6aba20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f87e4007b00 tx=0x7f87e4007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c53bedfaae34277 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87d1ffb700 1 -- 172.21.15.46:0/3279512061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e400e040 con 0x55b95d6ae220 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87d1ffb700 1 -- 172.21.15.46:0/3279512061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f87e4015920 con 0x55b95d6ae220 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/3279512061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b95d696ad0 con 0x55b95d6ae220 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87d1ffb700 1 -- 172.21.15.46:0/3279512061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87e4015ae0 con 0x55b95d6ae220 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87fcb5bb80 10 client.15783 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:02.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/3279512061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b95d423b60 con 0x55b95d6ae220 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.149+0000 7f87d1ffb700 1 -- 172.21.15.46:0/3279512061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b95d696ad0 con 0x55b95d6ae220 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.150+0000 7f87d1ffb700 1 -- 172.21.15.46:0/3279512061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f87e401e3b0 con 0x55b95d6ae220 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.150+0000 7f87fcb5bb80 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.150+0000 7f87fcb5bb80 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.150+0000 7f87fcb5bb80 20 client.15783 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.150+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b95d6bde80 0x55b95d6c02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.150+0000 7f87fcb5bb80 4 client.15783 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.150+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/3279512061 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b95d5ee700 con 0x55b95d6bde80 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.150+0000 7f87ea59c700 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b95d6bde80 0x55b95d6c02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.150+0000 7f87d1ffb700 10 client.15783 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.150+0000 7f87ea59c700 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b95d6bde80 0x55b95d6c02a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da40576303545bfa server_cookie=544e944314968171 in_seq=0 out_seq=0 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87d1ffb700 1 -- 172.21.15.46:0/3279512061 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b95d5ee700 con 0x55b95d6bde80 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87d1ffb700 10 client.15783 handle_command_reply: tid=0 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87fcb5bb80 1 client.15783 shutdown 2024-03-18T20:37:02.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/3279512061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 msgr2=0x55b95d6aba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 0x55b95d6aba20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f87e4007b00 tx=0x7f87e4007b30 comp rx=0 tx=0).stop 2024-03-18T20:37:02.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/3279512061 shutdown_connections 2024-03-18T20:37:02.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b95d6bde80 0x55b95d6c02a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 0x55b95d6aba20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6a2a60 0x55b95d6aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/3279512061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d6ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/3279512061 >> 172.21.15.46:0/3279512061 conn(0x55b95d619220 msgr2=0x55b95d69ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87d1ffb700 0 client.15783 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:02.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/3279512061 shutdown_connections 2024-03-18T20:37:02.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/3279512061 wait complete. 2024-03-18T20:37:02.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.151+0000 7f87fcb5bb80 20 client.15783 trim_cache size 0 max 16384 2024-03-18T20:37:02.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.180+0000 7f87fcb5bb80 1 Processor -- start 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.180+0000 7f87fcb5bb80 1 -- start start 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.180+0000 7f87fcb5bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a2a60 0x55b95d6b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.180+0000 7f87fcb5bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d69eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.180+0000 7f87fcb5bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6ae220 0x55b95d69f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.180+0000 7f87fcb5bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95d6a3d30 con 0x55b95d6a2a60 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.180+0000 7f87fcb5bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95d69b5e0 con 0x55b95d6a3380 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.180+0000 7f87fcb5bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95d6b5390 con 0x55b95d6ae220 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87e9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d69eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87ea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a2a60 0x55b95d6b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87ea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a2a60 0x55b95d6b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60392/0 (socket says 172.21.15.46:60392) 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87e9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d69eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:41868/0 (socket says 172.21.15.46:41868) 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87ea59c700 1 -- 172.21.15.46:0/2539923234 learned_addr learned my addr 172.21.15.46:0/2539923234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87ead9d700 1 --2- 172.21.15.46:0/2539923234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6ae220 0x55b95d69f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87e9d9b700 1 -- 172.21.15.46:0/2539923234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6ae220 msgr2=0x55b95d69f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87e9d9b700 1 --2- 172.21.15.46:0/2539923234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6ae220 0x55b95d69f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87e9d9b700 1 -- 172.21.15.46:0/2539923234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a2a60 msgr2=0x55b95d6b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87e9d9b700 1 --2- 172.21.15.46:0/2539923234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a2a60 0x55b95d6b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87e9d9b700 1 -- 172.21.15.46:0/2539923234 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b95d5bbea0 con 0x55b95d6a3380 2024-03-18T20:37:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87ead9d700 1 --2- 172.21.15.46:0/2539923234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6ae220 0x55b95d69f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87ea59c700 1 --2- 172.21.15.46:0/2539923234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a2a60 0x55b95d6b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87e9d9b700 1 --2- 172.21.15.46:0/2539923234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d69eb50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f87e0099fc0 tx=0x7f87e00092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28f64a065f1366b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87d97fa700 1 -- 172.21.15.46:0/2539923234 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e009a0b0 con 0x55b95d6a3380 2024-03-18T20:37:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87d97fa700 1 -- 172.21.15.46:0/2539923234 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87e0010880 con 0x55b95d6a3380 2024-03-18T20:37:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.181+0000 7f87d97fa700 1 -- 172.21.15.46:0/2539923234 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e00a2020 con 0x55b95d6a3380 2024-03-18T20:37:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.182+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2539923234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 msgr2=0x55b95d69eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.182+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/2539923234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d69eb50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f87e0099fc0 tx=0x7f87e00092d0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.182+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2539923234 shutdown_connections 2024-03-18T20:37:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.182+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/2539923234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6a2a60 0x55b95d6b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.182+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/2539923234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6ae220 0x55b95d69f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.182+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/2539923234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d69eb50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.182+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2539923234 >> 172.21.15.46:0/2539923234 conn(0x55b95d619220 msgr2=0x55b95d6b95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.182+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2539923234 shutdown_connections 2024-03-18T20:37:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.182+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2539923234 wait complete. 2024-03-18T20:37:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87fcb5bb80 1 -- start start 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87fcb5bb80 1 Processor -- start 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87fcb5bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6a2a60 0x55b95d6b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87fcb5bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d6b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87fcb5bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 0x55b95d6b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87fcb5bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95d710cb0 con 0x55b95d6ae220 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87fcb5bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95d710df0 con 0x55b95d6a3380 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87fcb5bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95d710f30 con 0x55b95d6a2a60 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87fcb5bb80 10 client.0 _ll_register_callbacks cb 0x7f87e40094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87ead9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 0x55b95d6b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87ead9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 0x55b95d6b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60400/0 (socket says 172.21.15.46:60400) 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87ead9d700 1 -- 172.21.15.46:0/2192940624 learned_addr learned my addr 172.21.15.46:0/2192940624 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87e9d9b700 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d6b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.183+0000 7f87ea59c700 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6a2a60 0x55b95d6b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87ead9d700 1 -- 172.21.15.46:0/2192940624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6a2a60 msgr2=0x55b95d6b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87ead9d700 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6a2a60 0x55b95d6b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87ead9d700 1 -- 172.21.15.46:0/2192940624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 msgr2=0x55b95d6b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87ead9d700 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d6b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87ead9d700 1 -- 172.21.15.46:0/2192940624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec10e550 con 0x55b95d6ae220 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87ea59c700 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6a2a60 0x55b95d6b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87ead9d700 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 0x55b95d6b38b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f87e4002230 tx=0x7f87e4009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28170bfb61f49bc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87d1ffb700 1 -- 172.21.15.46:0/2192940624 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e400e040 con 0x55b95d6ae220 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87d1ffb700 1 -- 172.21.15.46:0/2192940624 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f87e401ee30 con 0x55b95d6ae220 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2192940624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87e009de60 con 0x55b95d6ae220 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87d1ffb700 1 -- 172.21.15.46:0/2192940624 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87e4003c00 con 0x55b95d6ae220 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87fcb5bb80 10 client.15789 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2192940624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b95d4773b0 con 0x55b95d6ae220 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.184+0000 7f87d1ffb700 1 -- 172.21.15.46:0/2192940624 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87e4037080 con 0x55b95d6ae220 2024-03-18T20:37:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87d1ffb700 1 -- 172.21.15.46:0/2192940624 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f87e40050d0 con 0x55b95d6ae220 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87fcb5bb80 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87fcb5bb80 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87fcb5bb80 20 client.15789 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b95d7126a0 0x55b95d714ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87fcb5bb80 4 client.15789 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2192940624 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b95d444cf0 con 0x55b95d7126a0 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87ea59c700 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b95d7126a0 0x55b95d714ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87ea59c700 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b95d7126a0 0x55b95d714ac0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31b2410c94d231b server_cookie=9e041bbcc7642943 in_seq=0 out_seq=0 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87d1ffb700 10 client.15789 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87d1ffb700 1 -- 172.21.15.46:0/2192940624 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b95d444cf0 con 0x55b95d7126a0 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87d1ffb700 10 client.15789 handle_command_reply: tid=0 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.185+0000 7f87fcb5bb80 1 client.15789 shutdown 2024-03-18T20:37:02.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2192940624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 msgr2=0x55b95d6b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 0x55b95d6b38b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f87e4002230 tx=0x7f87e4009bc0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2192940624 shutdown_connections 2024-03-18T20:37:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b95d7126a0 0x55b95d714ac0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95d6ae220 0x55b95d6b38b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b95d6a2a60 0x55b95d6b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87fcb5bb80 1 --2- 172.21.15.46:0/2192940624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95d6a3380 0x55b95d6b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2192940624 >> 172.21.15.46:0/2192940624 conn(0x55b95d619220 msgr2=0x55b95d6a62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87d1ffb700 0 client.15789 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2192940624 shutdown_connections 2024-03-18T20:37:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87fcb5bb80 1 -- 172.21.15.46:0/2192940624 wait complete. 2024-03-18T20:37:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87fcb5bb80 20 client.15789 trim_cache size 0 max 16384 2024-03-18T20:37:02.196 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:37:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:37:02.196 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:37:02.196 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:37:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87f09c6700 1 -- 172.21.15.46:0/2930086820 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f87d40a13b0 msgr2=0x7f87d40a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87f09c6700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f87d40a13b0 0x7f87d40a3870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0098e0 tx=0x7f87dc005960 comp rx=0 tx=0).stop 2024-03-18T20:37:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87f09c6700 1 -- 172.21.15.46:0/2930086820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec0687f0 msgr2=0x7f87ec1001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.186+0000 7f87f09c6700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec0687f0 0x7f87ec1001b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f87e000e9a0 tx=0x7f87e000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.187+0000 7f87f09c6700 1 -- 172.21.15.46:0/2930086820 shutdown_connections 2024-03-18T20:37:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.187+0000 7f87f09c6700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f87d40a13b0 0x7f87d40a3870 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.187+0000 7f87f09c6700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87ec105ce0 0x7f87ec1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.187+0000 7f87f09c6700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87ec067e30 0x7f87ec0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.187+0000 7f87f09c6700 1 --2- 172.21.15.46:0/2930086820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87ec0687f0 0x7f87ec1001b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.187+0000 7f87f09c6700 1 -- 172.21.15.46:0/2930086820 >> 172.21.15.46:0/2930086820 conn(0x7f87ec076d40 msgr2=0x7f87ec0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.187+0000 7f87f09c6700 1 -- 172.21.15.46:0/2930086820 shutdown_connections 2024-03-18T20:37:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.187+0000 7f87f09c6700 1 -- 172.21.15.46:0/2930086820 wait complete. 2024-03-18T20:37:02.203 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:37:02.204 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:37:02.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:02 smithi046 ceph-mon[43495]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-18T20:37:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.340+0000 7f1046c5e700 1 -- 172.21.15.46:0/4162309333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1040101810 msgr2=0x7f1040105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.340+0000 7f1046c5e700 1 --2- 172.21.15.46:0/4162309333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1040101810 0x7f1040105850 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1030009990 tx=0x7f1030009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.340+0000 7f1046c5e700 1 -- 172.21.15.46:0/4162309333 shutdown_connections 2024-03-18T20:37:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.340+0000 7f1046c5e700 1 --2- 172.21.15.46:0/4162309333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1040105f80 0x7f104010db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.340+0000 7f1046c5e700 1 --2- 172.21.15.46:0/4162309333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1040100e60 0x7f1040101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.340+0000 7f1046c5e700 1 --2- 172.21.15.46:0/4162309333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1040101810 0x7f1040105850 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.340+0000 7f1046c5e700 1 -- 172.21.15.46:0/4162309333 >> 172.21.15.46:0/4162309333 conn(0x7f10400fc720 msgr2=0x7f10400feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.340+0000 7f1046c5e700 1 -- 172.21.15.46:0/4162309333 shutdown_connections 2024-03-18T20:37:02.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.340+0000 7f1046c5e700 1 -- 172.21.15.46:0/4162309333 wait complete. 2024-03-18T20:37:02.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.341+0000 7f1046c5e700 1 Processor -- start 2024-03-18T20:37:02.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.341+0000 7f1046c5e700 1 -- start start 2024-03-18T20:37:02.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f1046c5e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1040100e60 0x7f1040199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f1046c5e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1040105f80 0x7f10401999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f1046c5e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f104019dd50 0x7f104019e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f1046c5e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104019e7d0 con 0x7f1040100e60 2024-03-18T20:37:02.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f1046c5e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104019e940 con 0x7f1040105f80 2024-03-18T20:37:02.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f1046c5e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104019eb00 con 0x7f104019dd50 2024-03-18T20:37:02.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f10449fa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1040100e60 0x7f1040199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f10449fa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1040100e60 0x7f1040199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60426/0 (socket says 172.21.15.46:60426) 2024-03-18T20:37:02.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f10449fa700 1 -- 172.21.15.46:0/1223195923 learned_addr learned my addr 172.21.15.46:0/1223195923 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f103ffff700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1040105f80 0x7f10401999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f10451fb700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f104019dd50 0x7f104019e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f103ffff700 1 -- 172.21.15.46:0/1223195923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f104019dd50 msgr2=0x7f104019e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f103ffff700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f104019dd50 0x7f104019e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f103ffff700 1 -- 172.21.15.46:0/1223195923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1040100e60 msgr2=0x7f1040199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f103ffff700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1040100e60 0x7f1040199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f103ffff700 1 -- 172.21.15.46:0/1223195923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10300095f0 con 0x7f1040105f80 2024-03-18T20:37:02.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f10449fa700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1040100e60 0x7f1040199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:02.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.342+0000 7f103ffff700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1040105f80 0x7f10401999b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f10300098e0 tx=0x7f1030005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.343+0000 7f103dffb700 1 -- 172.21.15.46:0/1223195923 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103001d070 con 0x7f1040105f80 2024-03-18T20:37:02.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.343+0000 7f1046c5e700 1 -- 172.21.15.46:0/1223195923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10401a6000 con 0x7f1040105f80 2024-03-18T20:37:02.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.343+0000 7f103dffb700 1 -- 172.21.15.46:0/1223195923 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1030022470 con 0x7f1040105f80 2024-03-18T20:37:02.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.343+0000 7f103dffb700 1 -- 172.21.15.46:0/1223195923 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030017650 con 0x7f1040105f80 2024-03-18T20:37:02.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.343+0000 7f1046c5e700 1 -- 172.21.15.46:0/1223195923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10401a6420 con 0x7f1040105f80 2024-03-18T20:37:02.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.344+0000 7f1046c5e700 1 -- 172.21.15.46:0/1223195923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1040102330 con 0x7f1040105f80 2024-03-18T20:37:02.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.345+0000 7f103dffb700 1 -- 172.21.15.46:0/1223195923 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f1030022850 con 0x7f1040105f80 2024-03-18T20:37:02.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.346+0000 7f103dffb700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f102c0a13b0 0x7f102c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.346+0000 7f10449fa700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f102c0a13b0 0x7f102c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.346+0000 7f103dffb700 1 -- 172.21.15.46:0/1223195923 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f10300ce8d0 con 0x7f1040105f80 2024-03-18T20:37:02.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.347+0000 7f10449fa700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f102c0a13b0 0x7f102c0a3870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1028005950 tx=0x7f102800b500 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.350+0000 7f103dffb700 1 -- 172.21.15.46:0/1223195923 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1030097c80 con 0x7f1040105f80 2024-03-18T20:37:02.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:02 smithi073 ceph-mon[47384]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-18T20:37:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.512+0000 7f147efc7700 1 -- 172.21.15.46:0/3659215682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1478101880 msgr2=0x7f14781058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.512+0000 7f147efc7700 1 --2- 172.21.15.46:0/3659215682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1478101880 0x7f14781058c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f146c009990 tx=0x7f146c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.512+0000 7f147efc7700 1 -- 172.21.15.46:0/3659215682 shutdown_connections 2024-03-18T20:37:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.512+0000 7f147efc7700 1 --2- 172.21.15.46:0/3659215682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1478105ff0 0x7f147810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.512+0000 7f147efc7700 1 --2- 172.21.15.46:0/3659215682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1478100ed0 0x7f14781012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.512+0000 7f147efc7700 1 --2- 172.21.15.46:0/3659215682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1478101880 0x7f14781058c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.512+0000 7f147efc7700 1 -- 172.21.15.46:0/3659215682 >> 172.21.15.46:0/3659215682 conn(0x7f14780fc750 msgr2=0x7f14780feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.512+0000 7f147efc7700 1 -- 172.21.15.46:0/3659215682 shutdown_connections 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.512+0000 7f147efc7700 1 -- 172.21.15.46:0/3659215682 wait complete. 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.512+0000 7f147efc7700 1 Processor -- start 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.512+0000 7f147efc7700 1 -- start start 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147efc7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1478100ed0 0x7f1478197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147efc7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1478105ff0 0x7f14781976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147efc7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f147819ba20 0x7f147819bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147efc7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147819c4a0 con 0x7f1478100ed0 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147efc7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147819c610 con 0x7f1478105ff0 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147efc7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147819c7d0 con 0x7f147819ba20 2024-03-18T20:37:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147cd63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1478100ed0 0x7f1478197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147cd63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1478100ed0 0x7f1478197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60456/0 (socket says 172.21.15.46:60456) 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147cd63700 1 -- 172.21.15.46:0/144031093 learned_addr learned my addr 172.21.15.46:0/144031093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147d564700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f147819ba20 0x7f147819bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f1477fff700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1478105ff0 0x7f14781976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147d564700 1 -- 172.21.15.46:0/144031093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1478105ff0 msgr2=0x7f14781976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147d564700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1478105ff0 0x7f14781976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147d564700 1 -- 172.21.15.46:0/144031093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1478100ed0 msgr2=0x7f1478197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147d564700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1478100ed0 0x7f1478197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147d564700 1 -- 172.21.15.46:0/144031093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0095f0 con 0x7f147819ba20 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f1477fff700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1478105ff0 0x7f14781976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.513+0000 7f147cd63700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1478100ed0 0x7f1478197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.514+0000 7f147d564700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f147819ba20 0x7f147819bea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1470007f10 tx=0x7f147000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.514+0000 7f1475ffb700 1 -- 172.21.15.46:0/144031093 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147000ddd0 con 0x7f147819ba20 2024-03-18T20:37:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.514+0000 7f147efc7700 1 -- 172.21.15.46:0/144031093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14781a3d30 con 0x7f147819ba20 2024-03-18T20:37:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.514+0000 7f1475ffb700 1 -- 172.21.15.46:0/144031093 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f147000f040 con 0x7f147819ba20 2024-03-18T20:37:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.514+0000 7f1475ffb700 1 -- 172.21.15.46:0/144031093 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470005a90 con 0x7f147819ba20 2024-03-18T20:37:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.514+0000 7f147efc7700 1 -- 172.21.15.46:0/144031093 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14781a4210 con 0x7f147819ba20 2024-03-18T20:37:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.516+0000 7f1475ffb700 1 -- 172.21.15.46:0/144031093 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f1470004ad0 con 0x7f147819ba20 2024-03-18T20:37:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.517+0000 7f1475ffb700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14600a13b0 0x7f14600a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.517+0000 7f1475ffb700 1 -- 172.21.15.46:0/144031093 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f14700cc840 con 0x7f147819ba20 2024-03-18T20:37:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.517+0000 7f147cd63700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14600a13b0 0x7f14600a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.518+0000 7f147cd63700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14600a13b0 0x7f14600a3870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1468011550 tx=0x7f1468009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.518+0000 7f148b15cb80 1 Processor -- start 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.518+0000 7f148b15cb80 1 -- start start 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.518+0000 7f148b15cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381570 0x55a25e381950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.518+0000 7f148b15cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381e90 0x55a25e38c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.518+0000 7f148b15cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e38f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.518+0000 7f148b15cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e2db930 con 0x55a25e381570 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.518+0000 7f148b15cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e2d92f0 con 0x55a25e38cdb0 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.518+0000 7f148b15cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e2d4be0 con 0x55a25e381e90 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.518+0000 7f147cd63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381570 0x55a25e381950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.518+0000 7f147cd63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381570 0x55a25e381950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60464/0 (socket says 172.21.15.46:60464) 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.518+0000 7f147cd63700 1 -- 172.21.15.46:0/4284584003 learned_addr learned my addr 172.21.15.46:0/4284584003 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.519+0000 7f1477fff700 1 --2- 172.21.15.46:0/4284584003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381e90 0x55a25e38c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.519+0000 7f147d564700 1 --2- 172.21.15.46:0/4284584003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e38f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.519+0000 7f147cd63700 1 -- 172.21.15.46:0/4284584003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381e90 msgr2=0x55a25e38c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.519+0000 7f147cd63700 1 --2- 172.21.15.46:0/4284584003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381e90 0x55a25e38c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.519+0000 7f147cd63700 1 -- 172.21.15.46:0/4284584003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 msgr2=0x55a25e38f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.519+0000 7f147cd63700 1 --2- 172.21.15.46:0/4284584003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e38f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.519+0000 7f147cd63700 1 -- 172.21.15.46:0/4284584003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a25e156350 con 0x55a25e381570 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.519+0000 7f147d564700 1 --2- 172.21.15.46:0/4284584003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e38f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.519+0000 7f147cd63700 1 --2- 172.21.15.46:0/4284584003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381570 0x55a25e381950 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1468009880 tx=0x7f1468014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa10b9f65802b765 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.519+0000 7f145effd700 1 -- 172.21.15.46:0/4284584003 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468026630 con 0x55a25e381570 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.519+0000 7f145effd700 1 -- 172.21.15.46:0/4284584003 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1468021070 con 0x55a25e381570 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.519+0000 7f145effd700 1 -- 172.21.15.46:0/4284584003 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146801d700 con 0x55a25e381570 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.520+0000 7f148b15cb80 1 -- 172.21.15.46:0/4284584003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381570 msgr2=0x55a25e381950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.520+0000 7f148b15cb80 1 --2- 172.21.15.46:0/4284584003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381570 0x55a25e381950 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1468009880 tx=0x7f1468014000 comp rx=0 tx=0).stop 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.520+0000 7f148b15cb80 1 -- 172.21.15.46:0/4284584003 shutdown_connections 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.520+0000 7f148b15cb80 1 --2- 172.21.15.46:0/4284584003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381570 0x55a25e381950 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.520+0000 7f148b15cb80 1 --2- 172.21.15.46:0/4284584003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e38f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.520+0000 7f148b15cb80 1 --2- 172.21.15.46:0/4284584003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381e90 0x55a25e38c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.520+0000 7f148b15cb80 1 -- 172.21.15.46:0/4284584003 >> 172.21.15.46:0/4284584003 conn(0x55a25e2f81c0 msgr2=0x55a25e37da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.520+0000 7f148b15cb80 1 -- 172.21.15.46:0/4284584003 shutdown_connections 2024-03-18T20:37:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.520+0000 7f148b15cb80 1 -- 172.21.15.46:0/4284584003 wait complete. 2024-03-18T20:37:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f148b15cb80 1 -- start start 2024-03-18T20:37:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f148b15cb80 1 Processor -- start 2024-03-18T20:37:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f148b15cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e38c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f148b15cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e381e90 0x55a25e389a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f148b15cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e38cdb0 0x55a25e389fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f148b15cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e270d20 con 0x55a25e38cdb0 2024-03-18T20:37:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f148b15cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e2709c0 con 0x55a25e381e90 2024-03-18T20:37:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f148b15cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e158640 con 0x55a25e381570 2024-03-18T20:37:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f148b15cb80 10 client.0 _ll_register_callbacks cb 0x55a25e0dc340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f147d564700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e38cdb0 0x55a25e389fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f147cd63700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e38c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f147d564700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e38cdb0 0x55a25e389fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60470/0 (socket says 172.21.15.46:60470) 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f147d564700 1 -- 172.21.15.46:0/2128663843 learned_addr learned my addr 172.21.15.46:0/2128663843 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.521+0000 7f1477fff700 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e381e90 0x55a25e389a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.522+0000 7f1477fff700 1 -- 172.21.15.46:0/2128663843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 msgr2=0x55a25e38c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.522+0000 7f1477fff700 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e38c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.522+0000 7f1477fff700 1 -- 172.21.15.46:0/2128663843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e38cdb0 msgr2=0x55a25e389fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.522+0000 7f1477fff700 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e38cdb0 0x55a25e389fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.522+0000 7f1477fff700 1 -- 172.21.15.46:0/2128663843 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1468017070 con 0x55a25e381e90 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.522+0000 7f147d564700 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e38cdb0 0x55a25e389fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.522+0000 7f147cd63700 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e38c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.522+0000 7f1477fff700 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e381e90 0x55a25e389a60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1478198690 tx=0x7f146c00bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f75fb5217b5e5010 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.522+0000 7f1457fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.522+0000 7f1457fff700 1 -- 172.21.15.46:0/2128663843 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c00f870 con 0x55a25e381e90 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.522+0000 7f148b15cb80 1 -- 172.21.15.46:0/2128663843 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a25e3756c0 con 0x55a25e381e90 2024-03-18T20:37:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f1457fff700 1 -- 172.21.15.46:0/2128663843 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f146c00f9d0 con 0x55a25e381e90 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f1457fff700 1 -- 172.21.15.46:0/2128663843 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f146c00fb90 con 0x55a25e381e90 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f148b15cb80 10 client.25574 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f148b15cb80 1 -- 172.21.15.46:0/2128663843 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a25e102b00 con 0x55a25e381e90 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f1457fff700 1 -- 172.21.15.46:0/2128663843 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a25e3756c0 con 0x55a25e381e90 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f1457fff700 1 -- 172.21.15.46:0/2128663843 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f146c023ce0 con 0x55a25e381e90 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f148b15cb80 10 client.25574 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f148b15cb80 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f148b15cb80 20 client.25574 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f148b15cb80 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a25e39c930 0x55a25e39ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f148b15cb80 4 client.25574 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f148b15cb80 1 -- 172.21.15.46:0/2128663843 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a25e2cd6a0 con 0x55a25e39c930 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.523+0000 7f147cd63700 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a25e39c930 0x55a25e39ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.524+0000 7f147cd63700 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a25e39c930 0x55a25e39ed50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a351166bb3ec3249 server_cookie=b3f4c2174414e713 in_seq=0 out_seq=0 2024-03-18T20:37:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.524+0000 7f1457fff700 10 client.25574 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f1457fff700 1 -- 172.21.15.46:0/2128663843 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a25e2cd6a0 con 0x55a25e39c930 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f1457fff700 10 client.25574 handle_command_reply: tid=0 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f148b15cb80 1 client.25574 shutdown 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f148b15cb80 1 -- 172.21.15.46:0/2128663843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e381e90 msgr2=0x55a25e389a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f148b15cb80 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e381e90 0x55a25e389a60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1478198690 tx=0x7f146c00bae0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f148b15cb80 1 -- 172.21.15.46:0/2128663843 shutdown_connections 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f148b15cb80 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a25e39c930 0x55a25e39ed50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f148b15cb80 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e38cdb0 0x55a25e389fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f148b15cb80 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e38c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f1457fff700 0 client.25574 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f148b15cb80 1 --2- 172.21.15.46:0/2128663843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e381e90 0x55a25e389a60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f148b15cb80 1 -- 172.21.15.46:0/2128663843 >> 172.21.15.46:0/2128663843 conn(0x55a25e2f81c0 msgr2=0x55a25e37d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f148b15cb80 1 -- 172.21.15.46:0/2128663843 shutdown_connections 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f148b15cb80 1 -- 172.21.15.46:0/2128663843 wait complete. 2024-03-18T20:37:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.525+0000 7f148b15cb80 20 client.25574 trim_cache size 0 max 16384 2024-03-18T20:37:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:02 smithi155 ceph-mon[48344]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-03-18T20:37:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f148b15cb80 1 Processor -- start 2024-03-18T20:37:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f148b15cb80 1 -- start start 2024-03-18T20:37:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f148b15cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e37dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f148b15cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 0x55a25e3988d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f148b15cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e393a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f148b15cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e377c80 con 0x55a25e381e90 2024-03-18T20:37:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f148b15cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e37a200 con 0x55a25e38cdb0 2024-03-18T20:37:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f148b15cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e393fa0 con 0x55a25e381570 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f1477fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 0x55a25e3988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f1477fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 0x55a25e3988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60474/0 (socket says 172.21.15.46:60474) 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f1477fff700 1 -- 172.21.15.46:0/909936026 learned_addr learned my addr 172.21.15.46:0/909936026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f147d564700 1 --2- 172.21.15.46:0/909936026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e393a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f147cd63700 1 --2- 172.21.15.46:0/909936026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e37dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f1477fff700 1 -- 172.21.15.46:0/909936026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 msgr2=0x55a25e37dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f1477fff700 1 --2- 172.21.15.46:0/909936026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e37dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f1477fff700 1 -- 172.21.15.46:0/909936026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 msgr2=0x55a25e393a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f1477fff700 1 --2- 172.21.15.46:0/909936026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e393a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.541+0000 7f1477fff700 1 -- 172.21.15.46:0/909936026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a25e29ae40 con 0x55a25e381e90 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.542+0000 7f1477fff700 1 --2- 172.21.15.46:0/909936026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 0x55a25e3988d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f146c009ce0 tx=0x7f146c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1ceb632cd1bcd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.542+0000 7f145f7fe700 1 -- 172.21.15.46:0/909936026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c00be30 con 0x55a25e381e90 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.542+0000 7f145f7fe700 1 -- 172.21.15.46:0/909936026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f146c01e050 con 0x55a25e381e90 2024-03-18T20:37:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.542+0000 7f145f7fe700 1 -- 172.21.15.46:0/909936026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c00f9f0 con 0x55a25e381e90 2024-03-18T20:37:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.543+0000 7f148b15cb80 1 -- 172.21.15.46:0/909936026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 msgr2=0x55a25e3988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.543+0000 7f148b15cb80 1 --2- 172.21.15.46:0/909936026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 0x55a25e3988d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f146c009ce0 tx=0x7f146c00b630 comp rx=0 tx=0).stop 2024-03-18T20:37:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.543+0000 7f148b15cb80 1 -- 172.21.15.46:0/909936026 shutdown_connections 2024-03-18T20:37:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.543+0000 7f148b15cb80 1 --2- 172.21.15.46:0/909936026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e393a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.543+0000 7f148b15cb80 1 --2- 172.21.15.46:0/909936026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e37dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.543+0000 7f148b15cb80 1 --2- 172.21.15.46:0/909936026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 0x55a25e3988d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.543+0000 7f148b15cb80 1 -- 172.21.15.46:0/909936026 >> 172.21.15.46:0/909936026 conn(0x55a25e2f81c0 msgr2=0x55a25e3981c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.543+0000 7f148b15cb80 1 -- 172.21.15.46:0/909936026 shutdown_connections 2024-03-18T20:37:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.543+0000 7f148b15cb80 1 -- 172.21.15.46:0/909936026 wait complete. 2024-03-18T20:37:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.543+0000 7f148b15cb80 1 -- start start 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.543+0000 7f148b15cb80 1 Processor -- start 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.544+0000 7f148b15cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e38fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.544+0000 7f148b15cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 0x55a25e38ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.544+0000 7f148b15cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e3904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.544+0000 7f148b15cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e3ef710 con 0x55a25e381e90 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.544+0000 7f148b15cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e3ef850 con 0x55a25e38cdb0 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.544+0000 7f148b15cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a25e3ef990 con 0x55a25e381570 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.544+0000 7f148b15cb80 10 client.0 _ll_register_callbacks cb 0x7f1468005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.544+0000 7f1477fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 0x55a25e38ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.544+0000 7f1477fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 0x55a25e38ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60484/0 (socket says 172.21.15.46:60484) 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.544+0000 7f1477fff700 1 -- 172.21.15.46:0/3417478607 learned_addr learned my addr 172.21.15.46:0/3417478607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.544+0000 7f147cd63700 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e38fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.544+0000 7f147d564700 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e3904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f147cd63700 1 -- 172.21.15.46:0/3417478607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 msgr2=0x55a25e3904c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f147cd63700 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e3904c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f147cd63700 1 -- 172.21.15.46:0/3417478607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 msgr2=0x55a25e38ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f147cd63700 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 0x55a25e38ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f147cd63700 1 -- 172.21.15.46:0/3417478607 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a25e102b00 con 0x55a25e381570 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f147cd63700 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e38fa40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f146801ce60 tx=0x7f1468005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0d3f5f0b4227f4d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f1457fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f1457fff700 1 -- 172.21.15.46:0/3417478607 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468033a90 con 0x55a25e381570 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f1457fff700 1 -- 172.21.15.46:0/3417478607 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f1468033bf0 con 0x55a25e381570 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f148b15cb80 1 -- 172.21.15.46:0/3417478607 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1468021070 con 0x55a25e381570 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f1457fff700 1 -- 172.21.15.46:0/3417478607 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1468033db0 con 0x55a25e381570 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f148b15cb80 10 client.25579 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.545+0000 7f148b15cb80 1 -- 172.21.15.46:0/3417478607 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a25e15ac70 con 0x55a25e381570 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.546+0000 7f1457fff700 1 -- 172.21.15.46:0/3417478607 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1468021070 con 0x55a25e381570 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.546+0000 7f1457fff700 1 -- 172.21.15.46:0/3417478607 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f14680046b0 con 0x55a25e381570 2024-03-18T20:37:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.546+0000 7f148b15cb80 10 client.25579 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.546+0000 7f148b15cb80 10 client.25579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.546+0000 7f148b15cb80 20 client.25579 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.546+0000 7f148b15cb80 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a25e3f1030 0x55a25e3f3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.546+0000 7f148b15cb80 4 client.25579 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:37:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.546+0000 7f148b15cb80 1 -- 172.21.15.46:0/3417478607 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a25e123c90 con 0x55a25e3f1030 2024-03-18T20:37:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.546+0000 7f1477fff700 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a25e3f1030 0x55a25e3f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.546+0000 7f1457fff700 10 client.25579 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.546+0000 7f1477fff700 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a25e3f1030 0x55a25e3f3450 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9dbca79124eecf server_cookie=d6854fcdba62e815 in_seq=0 out_seq=0 2024-03-18T20:37:02.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.640+0000 7f1046c5e700 1 -- 172.21.15.46:0/1223195923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f104019a7a0 con 0x7f1040105f80 2024-03-18T20:37:02.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.641+0000 7f103dffb700 1 -- 172.21.15.46:0/1223195923 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f103009b2a0 con 0x7f1040105f80 2024-03-18T20:37:02.645 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:37:02.645 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:35:51.152306+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1686098166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1686098166},{"type":"v1","addr":"172.21.15.155:6837","nonce":1686098166}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:37:02.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.644+0000 7f1046c5e700 1 -- 172.21.15.46:0/1223195923 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f102c0a13b0 msgr2=0x7f102c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.644+0000 7f1046c5e700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f102c0a13b0 0x7f102c0a3870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1028005950 tx=0x7f102800b500 comp rx=0 tx=0).stop 2024-03-18T20:37:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.644+0000 7f1046c5e700 1 -- 172.21.15.46:0/1223195923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1040105f80 msgr2=0x7f10401999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.644+0000 7f1046c5e700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1040105f80 0x7f10401999b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f10300098e0 tx=0x7f1030005740 comp rx=0 tx=0).stop 2024-03-18T20:37:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.644+0000 7f1046c5e700 1 -- 172.21.15.46:0/1223195923 shutdown_connections 2024-03-18T20:37:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.644+0000 7f1046c5e700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f102c0a13b0 0x7f102c0a3870 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.644+0000 7f1046c5e700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1040100e60 0x7f1040199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.644+0000 7f1046c5e700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f104019dd50 0x7f104019e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.644+0000 7f1046c5e700 1 --2- 172.21.15.46:0/1223195923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1040105f80 0x7f10401999b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.644+0000 7f1046c5e700 1 -- 172.21.15.46:0/1223195923 >> 172.21.15.46:0/1223195923 conn(0x7f10400fc720 msgr2=0x7f10400feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.644+0000 7f1046c5e700 1 -- 172.21.15.46:0/1223195923 shutdown_connections 2024-03-18T20:37:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.644+0000 7f1046c5e700 1 -- 172.21.15.46:0/1223195923 wait complete. 2024-03-18T20:37:02.649 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-03-18T20:37:02.662 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:37:02.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:37:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.799+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/3414175846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e28100710 msgr2=0x7f6e28100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.799+0000 7f6e2fc8f700 1 --2- 172.21.15.46:0/3414175846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e28100710 0x7f6e28100af0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009990 tx=0x7f6e24009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.799+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/3414175846 shutdown_connections 2024-03-18T20:37:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.799+0000 7f6e2fc8f700 1 --2- 172.21.15.46:0/3414175846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e28100710 0x7f6e28100af0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.799+0000 7f6e2fc8f700 1 --2- 172.21.15.46:0/3414175846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e2810d750 0x7f6e2810fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.799+0000 7f6e2fc8f700 1 --2- 172.21.15.46:0/3414175846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e28101030 0x7f6e2810d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.799+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/3414175846 >> 172.21.15.46:0/3414175846 conn(0x7f6e28076d40 msgr2=0x7f6e28077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.799+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/3414175846 shutdown_connections 2024-03-18T20:37:02.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.799+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/3414175846 wait complete. 2024-03-18T20:37:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.800+0000 7f6e2fc8f700 1 Processor -- start 2024-03-18T20:37:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.800+0000 7f6e2fc8f700 1 -- start start 2024-03-18T20:37:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2fc8f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e28100710 0x7f6e28199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2fc8f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e28101030 0x7f6e28199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2fc8f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e2810d750 0x7f6e2819dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2fc8f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2819e330 con 0x7f6e28100710 2024-03-18T20:37:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2fc8f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2819e4a0 con 0x7f6e2810d750 2024-03-18T20:37:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2fc8f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2819e660 con 0x7f6e28101030 2024-03-18T20:37:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2da2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e28100710 0x7f6e28199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2da2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e28100710 0x7f6e28199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60502/0 (socket says 172.21.15.46:60502) 2024-03-18T20:37:02.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2d22a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e28101030 0x7f6e28199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2da2b700 1 -- 172.21.15.46:0/2625738219 learned_addr learned my addr 172.21.15.46:0/2625738219 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2e22c700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e2810d750 0x7f6e2819dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2da2b700 1 -- 172.21.15.46:0/2625738219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e28101030 msgr2=0x7f6e28199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e28101030 0x7f6e28199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2da2b700 1 -- 172.21.15.46:0/2625738219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e2810d750 msgr2=0x7f6e2819dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e2810d750 0x7f6e2819dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2da2b700 1 -- 172.21.15.46:0/2625738219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e240095f0 con 0x7f6e28100710 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.801+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e28100710 0x7f6e28199410 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400b5c0 tx=0x7f6e2400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.802+0000 7f6e1effd700 1 -- 172.21.15.46:0/2625738219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2401d070 con 0x7f6e28100710 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.802+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/2625738219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2819e8e0 con 0x7f6e28100710 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.802+0000 7f6e1effd700 1 -- 172.21.15.46:0/2625738219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e24022470 con 0x7f6e28100710 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.802+0000 7f6e1effd700 1 -- 172.21.15.46:0/2625738219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24017650 con 0x7f6e28100710 2024-03-18T20:37:02.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.802+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/2625738219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e281a6180 con 0x7f6e28100710 2024-03-18T20:37:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.804+0000 7f6e1effd700 1 -- 172.21.15.46:0/2625738219 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f6e240225e0 con 0x7f6e28100710 2024-03-18T20:37:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.805+0000 7f6e1effd700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6e140a13b0 0x7f6e140a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.805+0000 7f6e1effd700 1 -- 172.21.15.46:0/2625738219 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6e240ce020 con 0x7f6e28100710 2024-03-18T20:37:02.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.805+0000 7f6e2d22a700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6e140a13b0 0x7f6e140a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.806+0000 7f6e2d22a700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6e140a13b0 0x7f6e140a3870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e2819a960 tx=0x7f6e1800b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.806+0000 7f6e3be24b80 1 Processor -- start 2024-03-18T20:37:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.806+0000 7f6e3be24b80 1 -- start start 2024-03-18T20:37:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e3be24b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 0x55a0373c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e3be24b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373ca380 0x55a0373d4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e3be24b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373d5220 0x55a0373d7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e3be24b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037323990 con 0x55a0373c9a60 2024-03-18T20:37:02.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e3be24b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037321350 con 0x55a0373d5220 2024-03-18T20:37:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e3be24b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a03731cc40 con 0x55a0373ca380 2024-03-18T20:37:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e2da2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 0x55a0373c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e2d22a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373ca380 0x55a0373d4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e2da2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 0x55a0373c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60512/0 (socket says 172.21.15.46:60512) 2024-03-18T20:37:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e2da2b700 1 -- 172.21.15.46:0/4070401794 learned_addr learned my addr 172.21.15.46:0/4070401794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e2e22c700 1 --2- 172.21.15.46:0/4070401794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373d5220 0x55a0373d7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e2da2b700 1 -- 172.21.15.46:0/4070401794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373ca380 msgr2=0x55a0373d4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/4070401794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373ca380 0x55a0373d4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e2da2b700 1 -- 172.21.15.46:0/4070401794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373d5220 msgr2=0x55a0373d7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/4070401794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373d5220 0x55a0373d7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e2da2b700 1 -- 172.21.15.46:0/4070401794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a03719e3b0 con 0x55a0373c9a60 2024-03-18T20:37:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.807+0000 7f6e2d22a700 1 --2- 172.21.15.46:0/4070401794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373ca380 0x55a0373d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.808+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/4070401794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 0x55a0373c9e40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6e240a2e40 tx=0x7f6e2409abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad3fc443b5e5679 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.808+0000 7f6e13fff700 1 -- 172.21.15.46:0/4070401794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240a1070 con 0x55a0373c9a60 2024-03-18T20:37:02.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.808+0000 7f6e13fff700 1 -- 172.21.15.46:0/4070401794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e2409b680 con 0x55a0373c9a60 2024-03-18T20:37:02.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.808+0000 7f6e13fff700 1 -- 172.21.15.46:0/4070401794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2409a470 con 0x55a0373c9a60 2024-03-18T20:37:02.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.809+0000 7f6e3be24b80 1 -- 172.21.15.46:0/4070401794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 msgr2=0x55a0373c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.809+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/4070401794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 0x55a0373c9e40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6e240a2e40 tx=0x7f6e2409abc0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.809+0000 7f6e3be24b80 1 -- 172.21.15.46:0/4070401794 shutdown_connections 2024-03-18T20:37:02.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.809+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/4070401794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 0x55a0373c9e40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.809+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/4070401794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373d5220 0x55a0373d7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.809+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/4070401794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373ca380 0x55a0373d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.809+0000 7f6e3be24b80 1 -- 172.21.15.46:0/4070401794 >> 172.21.15.46:0/4070401794 conn(0x55a037340220 msgr2=0x55a0373c6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.809+0000 7f6e3be24b80 1 -- 172.21.15.46:0/4070401794 shutdown_connections 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.809+0000 7f6e3be24b80 1 -- 172.21.15.46:0/4070401794 wait complete. 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.810+0000 7f6e3be24b80 1 -- start start 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.810+0000 7f6e3be24b80 1 Processor -- start 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e3be24b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373c9a60 0x55a0373d4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e3be24b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 0x55a0373d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e3be24b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373d5220 0x55a0373d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e3be24b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0372b8d80 con 0x55a0373ca380 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e3be24b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0372b8a20 con 0x55a0373c9a60 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e3be24b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0371a06a0 con 0x55a0373d5220 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e3be24b80 10 client.0 _ll_register_callbacks cb 0x55a0371243a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e2da2b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373c9a60 0x55a0373d4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e2da2b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373c9a60 0x55a0373d4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:41980/0 (socket says 172.21.15.46:41980) 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e2da2b700 1 -- 172.21.15.46:0/3761951761 learned_addr learned my addr 172.21.15.46:0/3761951761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e2e22c700 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373d5220 0x55a0373d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e2da2b700 1 -- 172.21.15.46:0/3761951761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373d5220 msgr2=0x55a0373d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e2d22a700 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 0x55a0373d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373d5220 0x55a0373d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e2da2b700 1 -- 172.21.15.46:0/3761951761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 msgr2=0x55a0373d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 0x55a0373d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e2da2b700 1 -- 172.21.15.46:0/3761951761 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e18010c20 con 0x55a0373c9a60 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.811+0000 7f6e2d22a700 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 0x55a0373d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.812+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373c9a60 0x55a0373d4c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6e24022890 tx=0x7f6e2409a830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c22284c5c7f8c98 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.812+0000 7f6e117fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.812+0000 7f6e117fa700 1 -- 172.21.15.46:0/3761951761 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240a10c0 con 0x55a0373c9a60 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.812+0000 7f6e117fa700 1 -- 172.21.15.46:0/3761951761 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6e2409af80 con 0x55a0373c9a60 2024-03-18T20:37:02.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.812+0000 7f6e3be24b80 1 -- 172.21.15.46:0/3761951761 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a0373bdad0 con 0x55a0373c9a60 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.812+0000 7f6e117fa700 1 -- 172.21.15.46:0/3761951761 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e240a30f0 con 0x55a0373c9a60 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.812+0000 7f6e3be24b80 10 client.25592 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.812+0000 7f6e3be24b80 1 -- 172.21.15.46:0/3761951761 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f6e281a6180 con 0x55a0373c9a60 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.813+0000 7f6e117fa700 1 -- 172.21.15.46:0/3761951761 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e2407e080 con 0x55a0373c9a60 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.813+0000 7f6e117fa700 1 -- 172.21.15.46:0/3761951761 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6e24095c20 con 0x55a0373c9a60 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.813+0000 7f6e3be24b80 10 client.25592 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.813+0000 7f6e3be24b80 10 client.25592 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.813+0000 7f6e3be24b80 20 client.25592 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.813+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a0373e45f0 0x55a0373e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.813+0000 7f6e3be24b80 4 client.25592 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.813+0000 7f6e3be24b80 1 -- 172.21.15.46:0/3761951761 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a037315700 con 0x55a0373e45f0 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.813+0000 7f6e2d22a700 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a0373e45f0 0x55a0373e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.813+0000 7f6e117fa700 10 client.25592 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:37:02.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.813+0000 7f6e2d22a700 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a0373e45f0 0x55a0373e69e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e724126045d82efa server_cookie=bf14fc26791b67d2 in_seq=0 out_seq=0 2024-03-18T20:37:02.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.814+0000 7f6e117fa700 1 -- 172.21.15.46:0/3761951761 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a037315700 con 0x55a0373e45f0 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.814+0000 7f6e117fa700 10 client.25592 handle_command_reply: tid=0 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.814+0000 7f6e3be24b80 1 client.25592 shutdown 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.814+0000 7f6e3be24b80 1 -- 172.21.15.46:0/3761951761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373c9a60 msgr2=0x55a0373d4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.814+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373c9a60 0x55a0373d4c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6e24022890 tx=0x7f6e2409a830 comp rx=0 tx=0).stop 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.814+0000 7f6e3be24b80 1 -- 172.21.15.46:0/3761951761 shutdown_connections 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.814+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a0373e45f0 0x55a0373e69e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.814+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373d5220 0x55a0373d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.814+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373c9a60 0x55a0373d4c20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.814+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/3761951761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 0x55a0373d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.814+0000 7f6e117fa700 0 client.25592 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.814+0000 7f6e3be24b80 1 -- 172.21.15.46:0/3761951761 >> 172.21.15.46:0/3761951761 conn(0x55a037340220 msgr2=0x55a0373c5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.815+0000 7f6e3be24b80 1 -- 172.21.15.46:0/3761951761 shutdown_connections 2024-03-18T20:37:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.815+0000 7f6e3be24b80 1 -- 172.21.15.46:0/3761951761 wait complete. 2024-03-18T20:37:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.815+0000 7f6e3be24b80 20 client.25592 trim_cache size 0 max 16384 2024-03-18T20:37:02.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.830+0000 7f6e3be24b80 1 Processor -- start 2024-03-18T20:37:02.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.830+0000 7f6e3be24b80 1 -- start start 2024-03-18T20:37:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.830+0000 7f6e3be24b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 0x55a0373d9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.830+0000 7f6e3be24b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373ca380 0x55a0373cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.830+0000 7f6e3be24b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373d5220 0x55a0373cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.830+0000 7f6e3be24b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037390940 con 0x55a0373c9a60 2024-03-18T20:37:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.830+0000 7f6e3be24b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0373c25e0 con 0x55a0373d5220 2024-03-18T20:37:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.830+0000 7f6e3be24b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0373c0210 con 0x55a0373ca380 2024-03-18T20:37:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.830+0000 7f6e2da2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 0x55a0373d9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.830+0000 7f6e2da2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 0x55a0373d9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60526/0 (socket says 172.21.15.46:60526) 2024-03-18T20:37:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.830+0000 7f6e2da2b700 1 -- 172.21.15.46:0/832178634 learned_addr learned my addr 172.21.15.46:0/832178634 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.831+0000 7f6e2e22c700 1 --2- 172.21.15.46:0/832178634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373d5220 0x55a0373cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.831+0000 7f6e2d22a700 1 --2- 172.21.15.46:0/832178634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373ca380 0x55a0373cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.831+0000 7f6e2da2b700 1 -- 172.21.15.46:0/832178634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373ca380 msgr2=0x55a0373cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.831+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/832178634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373ca380 0x55a0373cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.831+0000 7f6e2da2b700 1 -- 172.21.15.46:0/832178634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373d5220 msgr2=0x55a0373cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.831+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/832178634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373d5220 0x55a0373cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.831+0000 7f6e2da2b700 1 -- 172.21.15.46:0/832178634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a03714ab60 con 0x55a0373c9a60 2024-03-18T20:37:02.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.831+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/832178634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 0x55a0373d9620 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6e240a3530 tx=0x7f6e24009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fd2cba7114b67f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.831+0000 7f6e1cda8700 1 -- 172.21.15.46:0/832178634 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24099d60 con 0x55a0373c9a60 2024-03-18T20:37:02.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.831+0000 7f6e1cda8700 1 -- 172.21.15.46:0/832178634 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a0373bdad0 con 0x55a0373c9a60 2024-03-18T20:37:02.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.831+0000 7f6e1cda8700 1 -- 172.21.15.46:0/832178634 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2409a170 con 0x55a0373c9a60 2024-03-18T20:37:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.832+0000 7f6e3be24b80 1 -- 172.21.15.46:0/832178634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 msgr2=0x55a0373d9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.832+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/832178634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 0x55a0373d9620 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6e240a3530 tx=0x7f6e24009310 comp rx=0 tx=0).stop 2024-03-18T20:37:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.832+0000 7f6e3be24b80 1 -- 172.21.15.46:0/832178634 shutdown_connections 2024-03-18T20:37:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.832+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/832178634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373c9a60 0x55a0373d9620 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.832+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/832178634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373d5220 0x55a0373cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.832+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/832178634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373ca380 0x55a0373cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.832+0000 7f6e3be24b80 1 -- 172.21.15.46:0/832178634 >> 172.21.15.46:0/832178634 conn(0x55a037340220 msgr2=0x55a0373d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.832+0000 7f6e3be24b80 1 -- 172.21.15.46:0/832178634 shutdown_connections 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.832+0000 7f6e3be24b80 1 -- 172.21.15.46:0/832178634 wait complete. 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.832+0000 7f6e3be24b80 1 -- start start 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e3be24b80 1 Processor -- start 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e3be24b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373c9a60 0x55a0373d7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e3be24b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 0x55a0373d7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e3be24b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373d5220 0x55a0373d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e3be24b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037437360 con 0x55a0373ca380 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e3be24b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0374374d0 con 0x55a0373c9a60 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e3be24b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037437690 con 0x55a0373d5220 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e3be24b80 10 client.0 _ll_register_callbacks cb 0x7f6e20003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e2d22a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 0x55a0373d7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e2d22a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 0x55a0373d7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60540/0 (socket says 172.21.15.46:60540) 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e2d22a700 1 -- 172.21.15.46:0/2291246377 learned_addr learned my addr 172.21.15.46:0/2291246377 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e2e22c700 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373d5220 0x55a0373d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.833+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373c9a60 0x55a0373d7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e2d22a700 1 -- 172.21.15.46:0/2291246377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373d5220 msgr2=0x55a0373d8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e2d22a700 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373d5220 0x55a0373d8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e2d22a700 1 -- 172.21.15.46:0/2291246377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373c9a60 msgr2=0x55a0373d7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e2d22a700 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373c9a60 0x55a0373d7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e2d22a700 1 -- 172.21.15.46:0/2291246377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e281a6180 con 0x55a0373ca380 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e2d22a700 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 0x55a0373d7b50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800ba80 tx=0x7f6e18013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d5340a2b45e3c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e117fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e117fa700 1 -- 172.21.15.46:0/2291246377 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18026c70 con 0x55a0373ca380 2024-03-18T20:37:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e117fa700 1 -- 172.21.15.46:0/2291246377 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6e18017070 con 0x55a0373ca380 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e3be24b80 1 -- 172.21.15.46:0/2291246377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e2409b680 con 0x55a0373ca380 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e117fa700 1 -- 172.21.15.46:0/2291246377 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e18026dd0 con 0x55a0373ca380 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e3be24b80 10 client.15828 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e3be24b80 1 -- 172.21.15.46:0/2291246377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a0372e2ea0 con 0x55a0373ca380 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.834+0000 7f6e117fa700 1 -- 172.21.15.46:0/2291246377 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e2409b680 con 0x55a0373ca380 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e117fa700 1 -- 172.21.15.46:0/2291246377 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6e18037740 con 0x55a0373ca380 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e3be24b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e3be24b80 10 client.15828 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e3be24b80 20 client.15828 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a037438ed0 0x55a03743b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e3be24b80 4 client.15828 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e3be24b80 1 -- 172.21.15.46:0/2291246377 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a03716bcf0 con 0x55a037438ed0 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a037438ed0 0x55a03743b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e117fa700 10 client.15828 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:37:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e2da2b700 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a037438ed0 0x55a03743b2f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b5c078e407826587 server_cookie=d4f9bd295c005572 in_seq=0 out_seq=0 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e117fa700 1 -- 172.21.15.46:0/2291246377 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a03716bcf0 con 0x55a037438ed0 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e117fa700 10 client.15828 handle_command_reply: tid=0 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.835+0000 7f6e3be24b80 1 client.15828 shutdown 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e3be24b80 1 -- 172.21.15.46:0/2291246377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 msgr2=0x55a0373d7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 0x55a0373d7b50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800ba80 tx=0x7f6e18013000 comp rx=0 tx=0).stop 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e3be24b80 1 -- 172.21.15.46:0/2291246377 shutdown_connections 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a037438ed0 0x55a03743b2f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a0373d5220 0x55a0373d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0373c9a60 0x55a0373d7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e117fa700 0 client.15828 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e3be24b80 1 --2- 172.21.15.46:0/2291246377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a0373ca380 0x55a0373d7b50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e3be24b80 1 -- 172.21.15.46:0/2291246377 >> 172.21.15.46:0/2291246377 conn(0x55a037340220 msgr2=0x55a0373d37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e3be24b80 1 -- 172.21.15.46:0/2291246377 shutdown_connections 2024-03-18T20:37:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e3be24b80 1 -- 172.21.15.46:0/2291246377 wait complete. 2024-03-18T20:37:02.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e3be24b80 20 client.15828 trim_cache size 0 max 16384 2024-03-18T20:37:02.845 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:37:02.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/2625738219 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6e140a13b0 msgr2=0x7f6e140a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e2fc8f700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6e140a13b0 0x7f6e140a3870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e2819a960 tx=0x7f6e1800b410 comp rx=0 tx=0).stop 2024-03-18T20:37:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/2625738219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e28100710 msgr2=0x7f6e28199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.836+0000 7f6e2fc8f700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e28100710 0x7f6e28199410 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400b5c0 tx=0x7f6e2400f670 comp rx=0 tx=0).stop 2024-03-18T20:37:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.837+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/2625738219 shutdown_connections 2024-03-18T20:37:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.837+0000 7f6e2fc8f700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6e140a13b0 0x7f6e140a3870 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.837+0000 7f6e2fc8f700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e28100710 0x7f6e28199410 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.837+0000 7f6e2fc8f700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e2810d750 0x7f6e2819dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.837+0000 7f6e2fc8f700 1 --2- 172.21.15.46:0/2625738219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e28101030 0x7f6e28199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.837+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/2625738219 >> 172.21.15.46:0/2625738219 conn(0x7f6e28076d40 msgr2=0x7f6e2810eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.837+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/2625738219 shutdown_connections 2024-03-18T20:37:02.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.837+0000 7f6e2fc8f700 1 -- 172.21.15.46:0/2625738219 wait complete. 2024-03-18T20:37:02.854 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:37:02.854 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:37:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f1457fff700 1 -- 172.21.15.46:0/3417478607 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a25e123c90 con 0x55a25e3f1030 2024-03-18T20:37:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f1457fff700 10 client.25579 handle_command_reply: tid=0 2024-03-18T20:37:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f148b15cb80 1 client.25579 shutdown 2024-03-18T20:37:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f148b15cb80 1 -- 172.21.15.46:0/3417478607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 msgr2=0x55a25e38fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f148b15cb80 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e38fa40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f146801ce60 tx=0x7f1468005c80 comp rx=0 tx=0).stop 2024-03-18T20:37:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f148b15cb80 1 -- 172.21.15.46:0/3417478607 shutdown_connections 2024-03-18T20:37:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f148b15cb80 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a25e3f1030 0x55a25e3f3450 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f148b15cb80 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a25e38cdb0 0x55a25e3904c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f1457fff700 0 client.25579 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f148b15cb80 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a25e381570 0x55a25e38fa40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f148b15cb80 1 --2- 172.21.15.46:0/3417478607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a25e381e90 0x55a25e38ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f148b15cb80 1 -- 172.21.15.46:0/3417478607 >> 172.21.15.46:0/3417478607 conn(0x55a25e2f81c0 msgr2=0x55a25e389250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f148b15cb80 1 -- 172.21.15.46:0/3417478607 shutdown_connections 2024-03-18T20:37:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f148b15cb80 1 -- 172.21.15.46:0/3417478607 wait complete. 2024-03-18T20:37:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.941+0000 7f148b15cb80 20 client.25579 trim_cache size 0 max 16384 2024-03-18T20:37:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:37:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:37:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:37:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:37:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.942+0000 7f147efc7700 1 -- 172.21.15.46:0/144031093 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14600a13b0 msgr2=0x7f14600a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.942+0000 7f147efc7700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14600a13b0 0x7f14600a3870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1468011550 tx=0x7f1468009260 comp rx=0 tx=0).stop 2024-03-18T20:37:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.942+0000 7f147efc7700 1 -- 172.21.15.46:0/144031093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f147819ba20 msgr2=0x7f147819bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.942+0000 7f147efc7700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f147819ba20 0x7f147819bea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1470007f10 tx=0x7f147000d3b0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.942+0000 7f147efc7700 1 -- 172.21.15.46:0/144031093 shutdown_connections 2024-03-18T20:37:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.942+0000 7f147efc7700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14600a13b0 0x7f14600a3870 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.942+0000 7f147efc7700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1478100ed0 0x7f1478197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.942+0000 7f147efc7700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f147819ba20 0x7f147819bea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.942+0000 7f147efc7700 1 --2- 172.21.15.46:0/144031093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1478105ff0 0x7f14781976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.942+0000 7f147efc7700 1 -- 172.21.15.46:0/144031093 >> 172.21.15.46:0/144031093 conn(0x7f14780fc750 msgr2=0x7f14780fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.942+0000 7f147efc7700 1 -- 172.21.15.46:0/144031093 shutdown_connections 2024-03-18T20:37:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:02.942+0000 7f147efc7700 1 -- 172.21.15.46:0/144031093 wait complete. 2024-03-18T20:37:02.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.993+0000 7fb777bfb700 1 -- 172.21.15.46:0/2773824775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7700687f0 msgr2=0x7fb77010db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:02.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.993+0000 7fb777bfb700 1 --2- 172.21.15.46:0/2773824775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7700687f0 0x7fb77010db50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb76800b0a0 tx=0x7fb76800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.993+0000 7fb777bfb700 1 -- 172.21.15.46:0/2773824775 shutdown_connections 2024-03-18T20:37:02.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.993+0000 7fb777bfb700 1 --2- 172.21.15.46:0/2773824775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7700687f0 0x7fb77010db50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.993+0000 7fb777bfb700 1 --2- 172.21.15.46:0/2773824775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb770105ce0 0x7fb7701060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.993+0000 7fb777bfb700 1 --2- 172.21.15.46:0/2773824775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb770067e30 0x7fb7700682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:02.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.993+0000 7fb777bfb700 1 -- 172.21.15.46:0/2773824775 >> 172.21.15.46:0/2773824775 conn(0x7fb770076d40 msgr2=0x7fb770077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:02.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.993+0000 7fb777bfb700 1 -- 172.21.15.46:0/2773824775 shutdown_connections 2024-03-18T20:37:02.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.993+0000 7fb777bfb700 1 -- 172.21.15.46:0/2773824775 wait complete. 2024-03-18T20:37:02.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.994+0000 7fb777bfb700 1 Processor -- start 2024-03-18T20:37:02.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.994+0000 7fb777bfb700 1 -- start start 2024-03-18T20:37:02.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.994+0000 7fb777bfb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb770067e30 0x7fb7700ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.994+0000 7fb777bfb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7700687f0 0x7fb7701001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.994+0000 7fb777bfb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb770105ce0 0x7fb7701006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:02.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.994+0000 7fb777bfb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7701a5720 con 0x7fb7700687f0 2024-03-18T20:37:02.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.994+0000 7fb777bfb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7701a5890 con 0x7fb770067e30 2024-03-18T20:37:02.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.994+0000 7fb777bfb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7701a5a50 con 0x7fb770105ce0 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.994+0000 7fb775196700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7700687f0 0x7fb7701001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.994+0000 7fb775196700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7700687f0 0x7fb7701001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60558/0 (socket says 172.21.15.46:60558) 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.994+0000 7fb775196700 1 -- 172.21.15.46:0/3204820233 learned_addr learned my addr 172.21.15.46:0/3204820233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.994+0000 7fb775997700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb770067e30 0x7fb7700ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb776198700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb770105ce0 0x7fb7701006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb775196700 1 -- 172.21.15.46:0/3204820233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb770105ce0 msgr2=0x7fb7701006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb775196700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb770105ce0 0x7fb7701006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb775196700 1 -- 172.21.15.46:0/3204820233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb770067e30 msgr2=0x7fb7700ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb775196700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb770067e30 0x7fb7700ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb775196700 1 -- 172.21.15.46:0/3204820233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb760009520 con 0x7fb7700687f0 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb775997700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb770067e30 0x7fb7700ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb775196700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7700687f0 0x7fb7701001b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb76000eb30 tx=0x7fb76000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb766ffd700 1 -- 172.21.15.46:0/3204820233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760009880 con 0x7fb7700687f0 2024-03-18T20:37:03.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb766ffd700 1 -- 172.21.15.46:0/3204820233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7600044e0 con 0x7fb7700687f0 2024-03-18T20:37:03.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb777bfb700 1 -- 172.21.15.46:0/3204820233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb768009d00 con 0x7fb7700687f0 2024-03-18T20:37:03.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb766ffd700 1 -- 172.21.15.46:0/3204820233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760003d80 con 0x7fb7700687f0 2024-03-18T20:37:03.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.995+0000 7fb777bfb700 1 -- 172.21.15.46:0/3204820233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7701a6100 con 0x7fb7700687f0 2024-03-18T20:37:03.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.998+0000 7fb766ffd700 1 -- 172.21.15.46:0/3204820233 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fb760003f20 con 0x7fb7700687f0 2024-03-18T20:37:03.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.999+0000 7fb766ffd700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb75c0a1430 0x7fb75c0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.999+0000 7fb775997700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb75c0a1430 0x7fb75c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.999+0000 7fb766ffd700 1 -- 172.21.15.46:0/3204820233 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb760014070 con 0x7fb7700687f0 2024-03-18T20:37:03.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:02.999+0000 7fb783d90b80 1 Processor -- start 2024-03-18T20:37:03.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb775997700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb75c0a1430 0x7fb75c0a38f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb76c011490 tx=0x7fb76c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb783d90b80 1 -- start start 2024-03-18T20:37:03.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb783d90b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbe8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb783d90b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbe9380 0x55b9abbf3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb783d90b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbf4220 0x55b9abbf6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb783d90b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9abb42990 con 0x55b9abbe9380 2024-03-18T20:37:03.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb783d90b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9abb40350 con 0x55b9abbf4220 2024-03-18T20:37:03.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb783d90b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9abb3bc40 con 0x55b9abbe8a60 2024-03-18T20:37:03.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb775997700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbe8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb775997700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbe8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:44194/0 (socket says 172.21.15.46:44194) 2024-03-18T20:37:03.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb775997700 1 -- 172.21.15.46:0/3824483674 learned_addr learned my addr 172.21.15.46:0/3824483674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb776198700 1 --2- 172.21.15.46:0/3824483674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbf4220 0x55b9abbf6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.000+0000 7fb775196700 1 --2- 172.21.15.46:0/3824483674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbe9380 0x55b9abbf3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.001+0000 7fb775196700 1 -- 172.21.15.46:0/3824483674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 msgr2=0x55b9abbe8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.001+0000 7fb775196700 1 --2- 172.21.15.46:0/3824483674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbe8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.001+0000 7fb775196700 1 -- 172.21.15.46:0/3824483674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbf4220 msgr2=0x55b9abbf6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.001+0000 7fb775196700 1 --2- 172.21.15.46:0/3824483674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbf4220 0x55b9abbf6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.001+0000 7fb775196700 1 -- 172.21.15.46:0/3824483674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ab9bd3b0 con 0x55b9abbe9380 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.001+0000 7fb775997700 1 --2- 172.21.15.46:0/3824483674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbe8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.001+0000 7fb775196700 1 --2- 172.21.15.46:0/3824483674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbe9380 0x55b9abbf3ce0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb7600ca950 tx=0x7fb760099f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc8fdd361fdfe62 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.001+0000 7fb75bfff700 1 -- 172.21.15.46:0/3824483674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76009aa10 con 0x55b9abbe9380 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.002+0000 7fb75bfff700 1 -- 172.21.15.46:0/3824483674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb76009a310 con 0x55b9abbe9380 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.002+0000 7fb75bfff700 1 -- 172.21.15.46:0/3824483674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76009a570 con 0x55b9abbe9380 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.002+0000 7fb783d90b80 1 -- 172.21.15.46:0/3824483674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbe9380 msgr2=0x55b9abbf3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.002+0000 7fb783d90b80 1 --2- 172.21.15.46:0/3824483674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbe9380 0x55b9abbf3ce0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb7600ca950 tx=0x7fb760099f20 comp rx=0 tx=0).stop 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.002+0000 7fb783d90b80 1 -- 172.21.15.46:0/3824483674 shutdown_connections 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.002+0000 7fb783d90b80 1 --2- 172.21.15.46:0/3824483674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbf4220 0x55b9abbf6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.002+0000 7fb783d90b80 1 --2- 172.21.15.46:0/3824483674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbe8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.002+0000 7fb783d90b80 1 --2- 172.21.15.46:0/3824483674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbe9380 0x55b9abbf3ce0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.002+0000 7fb783d90b80 1 -- 172.21.15.46:0/3824483674 >> 172.21.15.46:0/3824483674 conn(0x55b9abb5f220 msgr2=0x55b9abbe5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.003+0000 7fb783d90b80 1 -- 172.21.15.46:0/3824483674 shutdown_connections 2024-03-18T20:37:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.003+0000 7fb783d90b80 1 -- 172.21.15.46:0/3824483674 wait complete. 2024-03-18T20:37:03.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.003+0000 7fb783d90b80 1 -- start start 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.003+0000 7fb783d90b80 1 Processor -- start 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb783d90b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbf3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb783d90b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbf0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb783d90b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb783d90b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9abad7d80 con 0x55b9abbf4220 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb783d90b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9abad7a20 con 0x55b9abbe9380 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb783d90b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ab9bf6a0 con 0x55b9abbe8a60 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb783d90b80 10 client.0 _ll_register_callbacks cb 0x55b9ab9433a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb775196700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbf0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb775196700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbf0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:42032/0 (socket says 172.21.15.46:42032) 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb775196700 1 -- 172.21.15.46:0/978390237 learned_addr learned my addr 172.21.15.46:0/978390237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb775997700 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbf3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb776198700 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb775196700 1 -- 172.21.15.46:0/978390237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 msgr2=0x55b9abbf3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb775196700 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbf3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb775196700 1 -- 172.21.15.46:0/978390237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 msgr2=0x55b9abbf1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb775196700 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.004+0000 7fb775196700 1 -- 172.21.15.46:0/978390237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76c017070 con 0x55b9abbe9380 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.005+0000 7fb775196700 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbf0fa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb76009a2a0 tx=0x7fb76009aa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e47ff9ed89f0425 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.005+0000 7fb7597fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.005+0000 7fb7597fa700 1 -- 172.21.15.46:0/978390237 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7600a2790 con 0x55b9abbe9380 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.005+0000 7fb783d90b80 1 -- 172.21.15.46:0/978390237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9abbdcad0 con 0x55b9abbe9380 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.005+0000 7fb7597fa700 1 -- 172.21.15.46:0/978390237 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb76000cdd0 con 0x55b9abbe9380 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.005+0000 7fb7597fa700 1 -- 172.21.15.46:0/978390237 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb760099800 con 0x55b9abbe9380 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.005+0000 7fb783d90b80 10 client.25610 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.005+0000 7fb783d90b80 1 -- 172.21.15.46:0/978390237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fb7701a6100 con 0x55b9abbe9380 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.006+0000 7fb7597fa700 1 -- 172.21.15.46:0/978390237 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9abbdcad0 con 0x55b9abbe9380 2024-03-18T20:37:03.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.006+0000 7fb7597fa700 1 -- 172.21.15.46:0/978390237 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb7600a2300 con 0x55b9abbe9380 2024-03-18T20:37:03.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.006+0000 7fb783d90b80 10 client.25610 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:03.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.006+0000 7fb783d90b80 10 client.25610 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:37:03.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.006+0000 7fb783d90b80 20 client.25610 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:03.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.006+0000 7fb783d90b80 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b9abc03900 0x55b9abc05d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.006+0000 7fb783d90b80 4 client.25610 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:03.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.006+0000 7fb783d90b80 1 -- 172.21.15.46:0/978390237 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9abb34700 con 0x55b9abc03900 2024-03-18T20:37:03.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.006+0000 7fb775997700 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b9abc03900 0x55b9abc05d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.007+0000 7fb7597fa700 10 client.25610 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:37:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.007+0000 7fb775997700 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b9abc03900 0x55b9abc05d20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d542617a9d154826 server_cookie=b01966c6e6f24c59 in_seq=0 out_seq=0 2024-03-18T20:37:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.007+0000 7fb7597fa700 1 -- 172.21.15.46:0/978390237 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b9abb34700 con 0x55b9abc03900 2024-03-18T20:37:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.007+0000 7fb7597fa700 10 client.25610 handle_command_reply: tid=0 2024-03-18T20:37:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.007+0000 7fb783d90b80 1 client.25610 shutdown 2024-03-18T20:37:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.008+0000 7fb783d90b80 1 -- 172.21.15.46:0/978390237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 msgr2=0x55b9abbf0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.008+0000 7fb783d90b80 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbf0fa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb76009a2a0 tx=0x7fb76009aa10 comp rx=0 tx=0).stop 2024-03-18T20:37:03.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.008+0000 7fb783d90b80 1 -- 172.21.15.46:0/978390237 shutdown_connections 2024-03-18T20:37:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.008+0000 7fb783d90b80 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b9abc03900 0x55b9abc05d20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.008+0000 7fb783d90b80 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.008+0000 7fb783d90b80 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbf3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.008+0000 7fb783d90b80 1 --2- 172.21.15.46:0/978390237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbf0fa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.008+0000 7fb7597fa700 0 client.25610 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:37:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.008+0000 7fb783d90b80 1 -- 172.21.15.46:0/978390237 >> 172.21.15.46:0/978390237 conn(0x55b9abb5f220 msgr2=0x55b9abbe4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.008+0000 7fb783d90b80 1 -- 172.21.15.46:0/978390237 shutdown_connections 2024-03-18T20:37:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.008+0000 7fb783d90b80 1 -- 172.21.15.46:0/978390237 wait complete. 2024-03-18T20:37:03.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.008+0000 7fb783d90b80 20 client.25610 trim_cache size 0 max 16384 2024-03-18T20:37:03.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.022+0000 7fb783d90b80 1 Processor -- start 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.022+0000 7fb783d90b80 1 -- start start 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.022+0000 7fb783d90b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.022+0000 7fb783d90b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbf0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.022+0000 7fb783d90b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb783d90b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9abbe9d30 con 0x55b9abbf4220 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb783d90b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9abbdef80 con 0x55b9abbe9380 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb783d90b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9abc06380 con 0x55b9abbe8a60 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb776198700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb776198700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60580/0 (socket says 172.21.15.46:60580) 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb776198700 1 -- 172.21.15.46:0/3709193870 learned_addr learned my addr 172.21.15.46:0/3709193870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb775196700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbf0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb775997700 1 --2- 172.21.15.46:0/3709193870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb776198700 1 -- 172.21.15.46:0/3709193870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 msgr2=0x55b9abbffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb776198700 1 --2- 172.21.15.46:0/3709193870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb776198700 1 -- 172.21.15.46:0/3709193870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 msgr2=0x55b9abbf0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb776198700 1 --2- 172.21.15.46:0/3709193870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbf0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb776198700 1 -- 172.21.15.46:0/3709193870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ab969b60 con 0x55b9abbf4220 2024-03-18T20:37:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb775997700 1 --2- 172.21.15.46:0/3709193870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.023+0000 7fb776198700 1 --2- 172.21.15.46:0/3709193870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf0dc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb768009580 tx=0x7fb7680095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac83bc5147a7670 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.024+0000 7fb764da8700 1 -- 172.21.15.46:0/3709193870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76800e040 con 0x55b9abbf4220 2024-03-18T20:37:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.024+0000 7fb764da8700 1 -- 172.21.15.46:0/3709193870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb76800bcb0 con 0x55b9abbf4220 2024-03-18T20:37:03.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.024+0000 7fb764da8700 1 -- 172.21.15.46:0/3709193870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7680052a0 con 0x55b9abbf4220 2024-03-18T20:37:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.024+0000 7fb783d90b80 1 -- 172.21.15.46:0/3709193870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 msgr2=0x55b9abbf0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.024+0000 7fb783d90b80 1 --2- 172.21.15.46:0/3709193870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf0dc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb768009580 tx=0x7fb7680095b0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.024+0000 7fb783d90b80 1 -- 172.21.15.46:0/3709193870 shutdown_connections 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.024+0000 7fb783d90b80 1 --2- 172.21.15.46:0/3709193870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf0dc0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.024+0000 7fb783d90b80 1 --2- 172.21.15.46:0/3709193870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.024+0000 7fb783d90b80 1 --2- 172.21.15.46:0/3709193870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbf0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.024+0000 7fb783d90b80 1 -- 172.21.15.46:0/3709193870 >> 172.21.15.46:0/3709193870 conn(0x55b9abb5f220 msgr2=0x55b9abbff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.025+0000 7fb783d90b80 1 -- 172.21.15.46:0/3709193870 shutdown_connections 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.025+0000 7fb783d90b80 1 -- 172.21.15.46:0/3709193870 wait complete. 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.025+0000 7fb783d90b80 1 -- start start 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.025+0000 7fb783d90b80 1 Processor -- start 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.025+0000 7fb783d90b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbfaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.025+0000 7fb783d90b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbfafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.025+0000 7fb783d90b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.025+0000 7fb783d90b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9abbf9740 con 0x55b9abbf4220 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.025+0000 7fb783d90b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9abbf98b0 con 0x55b9abbe9380 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.025+0000 7fb783d90b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9abbf9a70 con 0x55b9abbe8a60 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb783d90b80 10 client.0 _ll_register_callbacks cb 0x7fb760005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb776198700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb775997700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbfaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb776198700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60584/0 (socket says 172.21.15.46:60584) 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb776198700 1 -- 172.21.15.46:0/1913649218 learned_addr learned my addr 172.21.15.46:0/1913649218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb775196700 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbfafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb775997700 1 -- 172.21.15.46:0/1913649218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 msgr2=0x55b9abbfafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb775997700 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbfafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb775997700 1 -- 172.21.15.46:0/1913649218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 msgr2=0x55b9abbf9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb775997700 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb775997700 1 -- 172.21.15.46:0/1913649218 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76c004560 con 0x55b9abbe8a60 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb776198700 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb775997700 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbfaa80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb7600091b0 tx=0x7fb76c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e38ba6a53ca8563c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb7597fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.026+0000 7fb7597fa700 1 -- 172.21.15.46:0/1913649218 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c024040 con 0x55b9abbe8a60 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb7597fa700 1 -- 172.21.15.46:0/1913649218 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb76c003e40 con 0x55b9abbe8a60 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb783d90b80 1 -- 172.21.15.46:0/1913649218 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb76009a310 con 0x55b9abbe8a60 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb7597fa700 1 -- 172.21.15.46:0/1913649218 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb76c029900 con 0x55b9abbe8a60 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb783d90b80 10 client.25615 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb783d90b80 1 -- 172.21.15.46:0/1913649218 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fb77010e550 con 0x55b9abbe8a60 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb7597fa700 1 -- 172.21.15.46:0/1913649218 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb76c03e080 con 0x55b9abbe8a60 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb7597fa700 1 -- 172.21.15.46:0/1913649218 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb76c032ba0 con 0x55b9abbe8a60 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb783d90b80 10 client.25615 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb783d90b80 10 client.25615 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb783d90b80 20 client.25615 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb783d90b80 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b9abc58a20 0x55b9abc5ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb783d90b80 4 client.25615 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.027+0000 7fb783d90b80 1 -- 172.21.15.46:0/1913649218 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b9ab98acf0 con 0x55b9abc58a20 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb775196700 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b9abc58a20 0x55b9abc5ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb7597fa700 10 client.25615 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb775196700 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b9abc58a20 0x55b9abc5ae10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3f4fa47a0e65912 server_cookie=4c0e34e0c12b7f01 in_seq=0 out_seq=0 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb7597fa700 1 -- 172.21.15.46:0/1913649218 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b9ab98acf0 con 0x55b9abc58a20 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb7597fa700 10 client.25615 handle_command_reply: tid=0 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb783d90b80 1 client.25615 shutdown 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb783d90b80 1 -- 172.21.15.46:0/1913649218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 msgr2=0x55b9abbfaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb783d90b80 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbfaa80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb7600091b0 tx=0x7fb76c0047c0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb783d90b80 1 -- 172.21.15.46:0/1913649218 shutdown_connections 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb783d90b80 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b9abc58a20 0x55b9abc5ae10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb783d90b80 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9abbf4220 0x55b9abbf9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb783d90b80 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9abbe8a60 0x55b9abbfaa80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb7597fa700 0 client.25615 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:37:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb783d90b80 1 --2- 172.21.15.46:0/1913649218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9abbe9380 0x55b9abbfafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb783d90b80 1 -- 172.21.15.46:0/1913649218 >> 172.21.15.46:0/1913649218 conn(0x55b9abb5f220 msgr2=0x55b9abbf3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.028+0000 7fb783d90b80 1 -- 172.21.15.46:0/1913649218 shutdown_connections 2024-03-18T20:37:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb783d90b80 1 -- 172.21.15.46:0/1913649218 wait complete. 2024-03-18T20:37:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb783d90b80 20 client.25615 trim_cache size 0 max 16384 2024-03-18T20:37:03.037 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:37:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb777bfb700 1 -- 172.21.15.46:0/3204820233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb75c0a1430 msgr2=0x7fb75c0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb777bfb700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb75c0a1430 0x7fb75c0a38f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb76c011490 tx=0x7fb76c009260 comp rx=0 tx=0).stop 2024-03-18T20:37:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb777bfb700 1 -- 172.21.15.46:0/3204820233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7700687f0 msgr2=0x7fb7701001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb777bfb700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7700687f0 0x7fb7701001b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb76000eb30 tx=0x7fb76000ee40 comp rx=0 tx=0).stop 2024-03-18T20:37:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb777bfb700 1 -- 172.21.15.46:0/3204820233 shutdown_connections 2024-03-18T20:37:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb777bfb700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb75c0a1430 0x7fb75c0a38f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb777bfb700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb770105ce0 0x7fb7701006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb777bfb700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb770067e30 0x7fb7700ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb777bfb700 1 --2- 172.21.15.46:0/3204820233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7700687f0 0x7fb7701001b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb777bfb700 1 -- 172.21.15.46:0/3204820233 >> 172.21.15.46:0/3204820233 conn(0x7fb770076d40 msgr2=0x7fb7700fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb777bfb700 1 -- 172.21.15.46:0/3204820233 shutdown_connections 2024-03-18T20:37:03.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.029+0000 7fb777bfb700 1 -- 172.21.15.46:0/3204820233 wait complete. 2024-03-18T20:37:03.047 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:37:03.047 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:37:03.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.186+0000 7fd410c8c700 1 -- 172.21.15.46:0/186818249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd40c100710 msgr2=0x7fd40c100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.186+0000 7fd410c8c700 1 --2- 172.21.15.46:0/186818249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd40c100710 0x7fd40c100af0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009990 tx=0x7fd3fc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.186+0000 7fd410c8c700 1 -- 172.21.15.46:0/186818249 shutdown_connections 2024-03-18T20:37:03.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.186+0000 7fd410c8c700 1 --2- 172.21.15.46:0/186818249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd40c100710 0x7fd40c100af0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.186+0000 7fd410c8c700 1 --2- 172.21.15.46:0/186818249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd40c10d750 0x7fd40c10fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.186+0000 7fd410c8c700 1 --2- 172.21.15.46:0/186818249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd40c101030 0x7fd40c10d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.186+0000 7fd410c8c700 1 -- 172.21.15.46:0/186818249 >> 172.21.15.46:0/186818249 conn(0x7fd40c076d40 msgr2=0x7fd40c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.186+0000 7fd410c8c700 1 -- 172.21.15.46:0/186818249 shutdown_connections 2024-03-18T20:37:03.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.186+0000 7fd410c8c700 1 -- 172.21.15.46:0/186818249 wait complete. 2024-03-18T20:37:03.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.187+0000 7fd410c8c700 1 Processor -- start 2024-03-18T20:37:03.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.187+0000 7fd410c8c700 1 -- start start 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd410c8c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd40c100710 0x7fd40c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd410c8c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd40c101030 0x7fd40c1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd410c8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd40c10d750 0x7fd40c19dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd410c8c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c19e340 con 0x7fd40c10d750 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd410c8c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c19e4b0 con 0x7fd40c100710 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd410c8c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40c19e670 con 0x7fd40c101030 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd40ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd40c10d750 0x7fd40c19dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd40a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd40c100710 0x7fd40c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd40ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd40c10d750 0x7fd40c19dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60596/0 (socket says 172.21.15.46:60596) 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd40ad9d700 1 -- 172.21.15.46:0/3160992040 learned_addr learned my addr 172.21.15.46:0/3160992040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd409d9b700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd40c101030 0x7fd40c1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd40a59c700 1 -- 172.21.15.46:0/3160992040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd40c101030 msgr2=0x7fd40c1999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd40a59c700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd40c101030 0x7fd40c1999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd40a59c700 1 -- 172.21.15.46:0/3160992040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd40c10d750 msgr2=0x7fd40c19dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd40a59c700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd40c10d750 0x7fd40c19dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd40a59c700 1 -- 172.21.15.46:0/3160992040 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc0095f0 con 0x7fd40c100710 2024-03-18T20:37:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd40ad9d700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd40c10d750 0x7fd40c19dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.188+0000 7fd409d9b700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd40c101030 0x7fd40c1999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.189+0000 7fd40a59c700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd40c100710 0x7fd40c199470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0059d0 tx=0x7fd3fc00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.189+0000 7fd3fb7fe700 1 -- 172.21.15.46:0/3160992040 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc01d070 con 0x7fd40c100710 2024-03-18T20:37:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.189+0000 7fd3fb7fe700 1 -- 172.21.15.46:0/3160992040 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3fc022470 con 0x7fd40c100710 2024-03-18T20:37:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.189+0000 7fd3fb7fe700 1 -- 172.21.15.46:0/3160992040 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc017650 con 0x7fd40c100710 2024-03-18T20:37:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.189+0000 7fd410c8c700 1 -- 172.21.15.46:0/3160992040 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c19e8f0 con 0x7fd40c100710 2024-03-18T20:37:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.189+0000 7fd410c8c700 1 -- 172.21.15.46:0/3160992040 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c19eb40 con 0x7fd40c100710 2024-03-18T20:37:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.191+0000 7fd3fb7fe700 1 -- 172.21.15.46:0/3160992040 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fd3fc022850 con 0x7fd40c100710 2024-03-18T20:37:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.192+0000 7fd3fb7fe700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd3f40a13b0 0x7fd3f40a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.192+0000 7fd3fb7fe700 1 -- 172.21.15.46:0/3160992040 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd3fc0ce290 con 0x7fd40c100710 2024-03-18T20:37:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.192+0000 7fd409d9b700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd3f40a13b0 0x7fd3f40a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.192+0000 7fd409d9b700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd3f40a13b0 0x7fd3f40a3870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd400005fd0 tx=0x7fd400005f60 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.193+0000 7fd41ce21b80 1 Processor -- start 2024-03-18T20:37:03.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.193+0000 7fd41ce21b80 1 -- start start 2024-03-18T20:37:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd41ce21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x559829316e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd41ce21b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x559829321ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd41ce21b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x559829324610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd41ce21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559829270990 con 0x559829316a60 2024-03-18T20:37:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd41ce21b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982926e350 con 0x559829322220 2024-03-18T20:37:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd41ce21b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559829269c40 con 0x559829317380 2024-03-18T20:37:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd40a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x559829316e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd40a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x559829316e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60606/0 (socket says 172.21.15.46:60606) 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd40a59c700 1 -- 172.21.15.46:0/411485127 learned_addr learned my addr 172.21.15.46:0/411485127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd40ad9d700 1 --2- 172.21.15.46:0/411485127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x559829324610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd409d9b700 1 --2- 172.21.15.46:0/411485127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x559829321ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd40a59c700 1 -- 172.21.15.46:0/411485127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 msgr2=0x559829321ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd40a59c700 1 --2- 172.21.15.46:0/411485127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x559829321ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd40a59c700 1 -- 172.21.15.46:0/411485127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 msgr2=0x559829324610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd40a59c700 1 --2- 172.21.15.46:0/411485127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x559829324610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd40a59c700 1 -- 172.21.15.46:0/411485127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598290eb3b0 con 0x559829316a60 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd40a59c700 1 --2- 172.21.15.46:0/411485127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x559829316e40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc09b2a0 tx=0x7fd3fc09ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc72f86cce5f6b98 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.194+0000 7fd3f8ff9700 1 -- 172.21.15.46:0/411485127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc09b310 con 0x559829316a60 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.195+0000 7fd3f8ff9700 1 -- 172.21.15.46:0/411485127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3fc0cd070 con 0x559829316a60 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.195+0000 7fd3f8ff9700 1 -- 172.21.15.46:0/411485127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc09aa60 con 0x559829316a60 2024-03-18T20:37:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.195+0000 7fd41ce21b80 1 -- 172.21.15.46:0/411485127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 msgr2=0x559829316e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.195+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/411485127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x559829316e40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc09b2a0 tx=0x7fd3fc09ac70 comp rx=0 tx=0).stop 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.195+0000 7fd41ce21b80 1 -- 172.21.15.46:0/411485127 shutdown_connections 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.195+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/411485127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x559829316e40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.195+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/411485127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x559829324610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.195+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/411485127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x559829321ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.195+0000 7fd41ce21b80 1 -- 172.21.15.46:0/411485127 >> 172.21.15.46:0/411485127 conn(0x55982928d220 msgr2=0x559829313020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.196+0000 7fd41ce21b80 1 -- 172.21.15.46:0/411485127 shutdown_connections 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.196+0000 7fd41ce21b80 1 -- 172.21.15.46:0/411485127 wait complete. 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.196+0000 7fd41ce21b80 1 -- start start 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.196+0000 7fd41ce21b80 1 Processor -- start 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.196+0000 7fd41ce21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x559829322600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.196+0000 7fd41ce21b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x559829322b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.196+0000 7fd41ce21b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x55982932d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.196+0000 7fd41ce21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559829205d80 con 0x559829316a60 2024-03-18T20:37:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.196+0000 7fd41ce21b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559829205a20 con 0x559829322220 2024-03-18T20:37:03.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.196+0000 7fd41ce21b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598290ed6a0 con 0x559829317380 2024-03-18T20:37:03.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.196+0000 7fd41ce21b80 10 client.0 _ll_register_callbacks cb 0x5598290713a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:03.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.196+0000 7fd40a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x559829322600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd40a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x559829322600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60620/0 (socket says 172.21.15.46:60620) 2024-03-18T20:37:03.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd40a59c700 1 -- 172.21.15.46:0/3026332630 learned_addr learned my addr 172.21.15.46:0/3026332630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd409d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x559829322b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd40ad9d700 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x55982932d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd409d9b700 1 -- 172.21.15.46:0/3026332630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 msgr2=0x55982932d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd409d9b700 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x55982932d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd409d9b700 1 -- 172.21.15.46:0/3026332630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 msgr2=0x559829322600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd409d9b700 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x559829322600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd409d9b700 1 -- 172.21.15.46:0/3026332630 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c19eb40 con 0x559829317380 2024-03-18T20:37:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd40ad9d700 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x55982932d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd40a59c700 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x559829322600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd409d9b700 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x559829322b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd40000baa0 tx=0x7fd400004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69376f6b480baa19 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd3f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.197+0000 7fd3f1ffb700 1 -- 172.21.15.46:0/3026332630 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400028670 con 0x559829317380 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3026332630 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55982930aad0 con 0x559829317380 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd3f1ffb700 1 -- 172.21.15.46:0/3026332630 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd4000287d0 con 0x559829317380 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd3f1ffb700 1 -- 172.21.15.46:0/3026332630 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd400011e90 con 0x559829317380 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd41ce21b80 10 client.25627 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3026332630 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559829097b60 con 0x559829317380 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd3f1ffb700 1 -- 172.21.15.46:0/3026332630 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55982930aad0 con 0x559829317380 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd3f1ffb700 1 -- 172.21.15.46:0/3026332630 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd400031b90 con 0x559829317380 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd41ce21b80 10 client.25627 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd41ce21b80 10 client.25627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd41ce21b80 20 client.25627 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559829331a80 0x559829333ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd41ce21b80 4 client.25627 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.198+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3026332630 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559829262700 con 0x559829331a80 2024-03-18T20:37:03.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.199+0000 7fd40a59c700 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559829331a80 0x559829333ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.199+0000 7fd40a59c700 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559829331a80 0x559829333ea0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b15945a9f276e43f server_cookie=a7d0d717e071ec9e in_seq=0 out_seq=0 2024-03-18T20:37:03.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.199+0000 7fd3f1ffb700 10 client.25627 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:37:03.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd3f1ffb700 1 -- 172.21.15.46:0/3026332630 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559829262700 con 0x559829331a80 2024-03-18T20:37:03.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd3f1ffb700 10 client.25627 handle_command_reply: tid=0 2024-03-18T20:37:03.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd41ce21b80 1 client.25627 shutdown 2024-03-18T20:37:03.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3026332630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 msgr2=0x559829322b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x559829322b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd40000baa0 tx=0x7fd400004750 comp rx=0 tx=0).stop 2024-03-18T20:37:03.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3026332630 shutdown_connections 2024-03-18T20:37:03.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559829331a80 0x559829333ea0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x559829322600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x55982932d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd3f1ffb700 0 client.25627 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:37:03.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/3026332630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x559829322b40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3026332630 >> 172.21.15.46:0/3026332630 conn(0x55982928d220 msgr2=0x559829312c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3026332630 shutdown_connections 2024-03-18T20:37:03.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3026332630 wait complete. 2024-03-18T20:37:03.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.200+0000 7fd41ce21b80 20 client.25627 trim_cache size 0 max 16384 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.214+0000 7fd41ce21b80 1 Processor -- start 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.214+0000 7fd41ce21b80 1 -- start start 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.214+0000 7fd41ce21b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829316a60 0x559829327610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.214+0000 7fd41ce21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829317380 0x559829320d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.214+0000 7fd41ce21b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x55982931fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.214+0000 7fd41ce21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559829317a30 con 0x559829317380 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.214+0000 7fd41ce21b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559829317d30 con 0x559829322220 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.214+0000 7fd41ce21b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559829322eb0 con 0x559829316a60 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd409d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829317380 0x559829320d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd409d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829317380 0x559829320d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60624/0 (socket says 172.21.15.46:60624) 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd409d9b700 1 -- 172.21.15.46:0/3108981014 learned_addr learned my addr 172.21.15.46:0/3108981014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd40a59c700 1 --2- 172.21.15.46:0/3108981014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829316a60 0x559829327610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd40ad9d700 1 --2- 172.21.15.46:0/3108981014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x55982931fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd409d9b700 1 -- 172.21.15.46:0/3108981014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829316a60 msgr2=0x559829327610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd409d9b700 1 --2- 172.21.15.46:0/3108981014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829316a60 0x559829327610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd409d9b700 1 -- 172.21.15.46:0/3108981014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 msgr2=0x55982931fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd409d9b700 1 --2- 172.21.15.46:0/3108981014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x55982931fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd409d9b700 1 -- 172.21.15.46:0/3108981014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55982922fea0 con 0x559829317380 2024-03-18T20:37:03.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd40ad9d700 1 --2- 172.21.15.46:0/3108981014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x55982931fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:03.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd409d9b700 1 --2- 172.21.15.46:0/3108981014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829317380 0x559829320d30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd40000b4e0 tx=0x7fd40000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cd0f553c229b03 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.215+0000 7fd3f97fa700 1 -- 172.21.15.46:0/3108981014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400004220 con 0x559829317380 2024-03-18T20:37:03.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.216+0000 7fd3f97fa700 1 -- 172.21.15.46:0/3108981014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd400011e90 con 0x559829317380 2024-03-18T20:37:03.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.216+0000 7fd3f97fa700 1 -- 172.21.15.46:0/3108981014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400003680 con 0x559829317380 2024-03-18T20:37:03.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.217+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3108981014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829317380 msgr2=0x559829320d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.217+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/3108981014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829317380 0x559829320d30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd40000b4e0 tx=0x7fd40000bf70 comp rx=0 tx=0).stop 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.217+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3108981014 shutdown_connections 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.217+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/3108981014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x55982931fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.217+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/3108981014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829316a60 0x559829327610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.217+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/3108981014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829317380 0x559829320d30 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.217+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3108981014 >> 172.21.15.46:0/3108981014 conn(0x55982928d220 msgr2=0x55982931bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.217+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3108981014 shutdown_connections 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.217+0000 7fd41ce21b80 1 -- 172.21.15.46:0/3108981014 wait complete. 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.217+0000 7fd41ce21b80 1 -- start start 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.217+0000 7fd41ce21b80 1 Processor -- start 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd41ce21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x55982937dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd41ce21b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x55982937e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd41ce21b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x559829379840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd41ce21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559829379dc0 con 0x559829316a60 2024-03-18T20:37:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd41ce21b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559829379f30 con 0x559829322220 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd41ce21b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982937a0f0 con 0x559829317380 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd41ce21b80 10 client.0 _ll_register_callbacks cb 0x5598290713a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd40a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x55982937dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd40a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x55982937dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60632/0 (socket says 172.21.15.46:60632) 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd40a59c700 1 -- 172.21.15.46:0/2472414588 learned_addr learned my addr 172.21.15.46:0/2472414588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd40ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x559829379840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd409d9b700 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x55982937e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd40a59c700 1 -- 172.21.15.46:0/2472414588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 msgr2=0x55982937e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd40a59c700 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x55982937e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd40a59c700 1 -- 172.21.15.46:0/2472414588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 msgr2=0x559829379840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd40a59c700 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x559829379840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd40a59c700 1 -- 172.21.15.46:0/2472414588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559829097b60 con 0x559829316a60 2024-03-18T20:37:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd40ad9d700 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x559829379840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.218+0000 7fd409d9b700 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x55982937e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd40a59c700 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x55982937dce0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc097d80 tx=0x7fd3fc09b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecb1a4cd52d6faf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd3f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd3f1ffb700 1 -- 172.21.15.46:0/2472414588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc09f1c0 con 0x559829316a60 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd3f1ffb700 1 -- 172.21.15.46:0/2472414588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd3fc080070 con 0x559829316a60 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd41ce21b80 1 -- 172.21.15.46:0/2472414588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3fc0cd070 con 0x559829316a60 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd3f1ffb700 1 -- 172.21.15.46:0/2472414588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3fc09ac70 con 0x559829316a60 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd41ce21b80 10 client.15864 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd41ce21b80 1 -- 172.21.15.46:0/2472414588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5598290efcd0 con 0x559829316a60 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd3f1ffb700 1 -- 172.21.15.46:0/2472414588 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3fc0cd070 con 0x559829316a60 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd3f1ffb700 1 -- 172.21.15.46:0/2472414588 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd3fc097100 con 0x559829316a60 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd41ce21b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd41ce21b80 10 client.15864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:37:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd41ce21b80 20 client.15864 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559829386920 0x559829388d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd41ce21b80 4 client.15864 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.219+0000 7fd41ce21b80 1 -- 172.21.15.46:0/2472414588 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5598290b8cf0 con 0x559829386920 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.220+0000 7fd409d9b700 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559829386920 0x559829388d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.220+0000 7fd409d9b700 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559829386920 0x559829388d10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58365e1951dfa0f4 server_cookie=e402c0ced1792335 in_seq=0 out_seq=0 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.220+0000 7fd3f1ffb700 10 client.15864 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.220+0000 7fd3f1ffb700 1 -- 172.21.15.46:0/2472414588 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5598290b8cf0 con 0x559829386920 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.220+0000 7fd3f1ffb700 10 client.15864 handle_command_reply: tid=0 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.220+0000 7fd41ce21b80 1 client.15864 shutdown 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.220+0000 7fd41ce21b80 1 -- 172.21.15.46:0/2472414588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 msgr2=0x55982937dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.220+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x55982937dce0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc097d80 tx=0x7fd3fc09b0e0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.220+0000 7fd41ce21b80 1 -- 172.21.15.46:0/2472414588 shutdown_connections 2024-03-18T20:37:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.220+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559829386920 0x559829388d10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559829316a60 0x55982937dce0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559829322220 0x559829379840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd41ce21b80 1 --2- 172.21.15.46:0/2472414588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559829317380 0x55982937e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd3f1ffb700 0 client.15864 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:37:03.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd41ce21b80 1 -- 172.21.15.46:0/2472414588 >> 172.21.15.46:0/2472414588 conn(0x55982928d220 msgr2=0x55982931c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd41ce21b80 1 -- 172.21.15.46:0/2472414588 shutdown_connections 2024-03-18T20:37:03.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd41ce21b80 1 -- 172.21.15.46:0/2472414588 wait complete. 2024-03-18T20:37:03.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd41ce21b80 20 client.15864 trim_cache size 0 max 16384 2024-03-18T20:37:03.229 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:37:03.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd410c8c700 1 -- 172.21.15.46:0/3160992040 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd3f40a13b0 msgr2=0x7fd3f40a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd410c8c700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd3f40a13b0 0x7fd3f40a3870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd400005fd0 tx=0x7fd400005f60 comp rx=0 tx=0).stop 2024-03-18T20:37:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd410c8c700 1 -- 172.21.15.46:0/3160992040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd40c100710 msgr2=0x7fd40c199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd410c8c700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd40c100710 0x7fd40c199470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0059d0 tx=0x7fd3fc00f690 comp rx=0 tx=0).stop 2024-03-18T20:37:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd410c8c700 1 -- 172.21.15.46:0/3160992040 shutdown_connections 2024-03-18T20:37:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd410c8c700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd3f40a13b0 0x7fd3f40a3870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd410c8c700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd40c10d750 0x7fd40c19dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd410c8c700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd40c100710 0x7fd40c199470 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd410c8c700 1 --2- 172.21.15.46:0/3160992040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd40c101030 0x7fd40c1999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd410c8c700 1 -- 172.21.15.46:0/3160992040 >> 172.21.15.46:0/3160992040 conn(0x7fd40c076d40 msgr2=0x7fd40c10ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.221+0000 7fd410c8c700 1 -- 172.21.15.46:0/3160992040 shutdown_connections 2024-03-18T20:37:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.222+0000 7fd410c8c700 1 -- 172.21.15.46:0/3160992040 wait complete. 2024-03-18T20:37:03.238 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:37:03.238 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:37:03.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:03 smithi046 ceph-mon[43495]: mgrmap e24: x(active, since 55s), standbys: z, y 2024-03-18T20:37:03.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:03 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1223195923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:37:03.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.379+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/3045118291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb4106070 msgr2=0x7f6eb410dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.379+0000 7f6eb9a5a700 1 --2- 172.21.15.46:0/3045118291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb4106070 0x7f6eb410dbf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00b0a0 tx=0x7f6eac00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.379+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/3045118291 shutdown_connections 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.379+0000 7f6eb9a5a700 1 --2- 172.21.15.46:0/3045118291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb4106070 0x7f6eb410dbf0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.379+0000 7f6eb9a5a700 1 --2- 172.21.15.46:0/3045118291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb4100f50 0x7f6eb4101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.379+0000 7f6eb9a5a700 1 --2- 172.21.15.46:0/3045118291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eb4101900 0x7f6eb4105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.379+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/3045118291 >> 172.21.15.46:0/3045118291 conn(0x7f6eb40fc7f0 msgr2=0x7f6eb40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.379+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/3045118291 shutdown_connections 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.379+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/3045118291 wait complete. 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.380+0000 7f6eb9a5a700 1 Processor -- start 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.380+0000 7f6eb9a5a700 1 -- start start 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.380+0000 7f6eb9a5a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb4100f50 0x7f6eb4197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.380+0000 7f6eb9a5a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eb4106070 0x7f6eb4197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.380+0000 7f6eb9a5a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb419bc00 0x7f6eb419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.380+0000 7f6eb9a5a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb419c680 con 0x7f6eb4100f50 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.380+0000 7f6eb9a5a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb419c7f0 con 0x7f6eb4106070 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.380+0000 7f6eb9a5a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb419c9b0 con 0x7f6eb419bc00 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.380+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb4100f50 0x7f6eb4197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.380+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb4100f50 0x7f6eb4197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60652/0 (socket says 172.21.15.46:60652) 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.380+0000 7f6eb2ffd700 1 -- 172.21.15.46:0/4041206252 learned_addr learned my addr 172.21.15.46:0/4041206252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb27fc700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eb4106070 0x7f6eb4197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb37fe700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb419bc00 0x7f6eb419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb2ffd700 1 -- 172.21.15.46:0/4041206252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb419bc00 msgr2=0x7f6eb419c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb2ffd700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb419bc00 0x7f6eb419c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb2ffd700 1 -- 172.21.15.46:0/4041206252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eb4106070 msgr2=0x7f6eb4197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb2ffd700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eb4106070 0x7f6eb4197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb2ffd700 1 -- 172.21.15.46:0/4041206252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac009d00 con 0x7f6eb4100f50 2024-03-18T20:37:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb2ffd700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb4100f50 0x7f6eb4197320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400e9a0 tx=0x7f6ea400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb8a58700 1 -- 172.21.15.46:0/4041206252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4009790 con 0x7f6eb4100f50 2024-03-18T20:37:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb8a58700 1 -- 172.21.15.46:0/4041206252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ea4010430 con 0x7f6eb4100f50 2024-03-18T20:37:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/4041206252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb41a3f10 con 0x7f6eb4100f50 2024-03-18T20:37:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb8a58700 1 -- 172.21.15.46:0/4041206252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4018680 con 0x7f6eb4100f50 2024-03-18T20:37:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.381+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/4041206252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb41a4390 con 0x7f6eb4100f50 2024-03-18T20:37:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.384+0000 7f6eb8a58700 1 -- 172.21.15.46:0/4041206252 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f6ea40105a0 con 0x7f6eb4100f50 2024-03-18T20:37:03.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.384+0000 7f6eb8a58700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6e9c0a13b0 0x7f6e9c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.384+0000 7f6eb8a58700 1 -- 172.21.15.46:0/4041206252 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6ea4014070 con 0x7f6eb4100f50 2024-03-18T20:37:03.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.384+0000 7f6eb27fc700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6e9c0a13b0 0x7f6e9c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.385+0000 7f6ec5befb80 1 Processor -- start 2024-03-18T20:37:03.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.385+0000 7f6eb27fc700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6e9c0a13b0 0x7f6e9c0a3870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4198870 tx=0x7f6ea8009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.385+0000 7f6ec5befb80 1 -- start start 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.385+0000 7f6ec5befb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x555636403e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.385+0000 7f6ec5befb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636404380 0x5556364130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.385+0000 7f6ec5befb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 0x555636415a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.385+0000 7f6ec5befb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55563635d990 con 0x555636413630 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.385+0000 7f6ec5befb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55563635b350 con 0x555636404380 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.385+0000 7f6ec5befb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555636356c40 con 0x555636403a60 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.385+0000 7f6eb37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 0x555636415a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x555636403e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6eb27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636404380 0x5556364130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6eb37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 0x555636415a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60662/0 (socket says 172.21.15.46:60662) 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6eb37fe700 1 -- 172.21.15.46:0/640236748 learned_addr learned my addr 172.21.15.46:0/640236748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6eb37fe700 1 -- 172.21.15.46:0/640236748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 msgr2=0x555636403e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6eb37fe700 1 --2- 172.21.15.46:0/640236748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x555636403e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6eb37fe700 1 -- 172.21.15.46:0/640236748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636404380 msgr2=0x5556364130f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6eb37fe700 1 --2- 172.21.15.46:0/640236748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636404380 0x5556364130f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6eb37fe700 1 -- 172.21.15.46:0/640236748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556361d83b0 con 0x555636413630 2024-03-18T20:37:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6eb27fc700 1 --2- 172.21.15.46:0/640236748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636404380 0x5556364130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6eb37fe700 1 --2- 172.21.15.46:0/640236748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 0x555636415a20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6eac015000 tx=0x7f6eac0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33442454b0821b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6e99ffb700 1 -- 172.21.15.46:0/640236748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00e040 con 0x555636413630 2024-03-18T20:37:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6e99ffb700 1 -- 172.21.15.46:0/640236748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eac006000 con 0x555636413630 2024-03-18T20:37:03.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.386+0000 7f6e99ffb700 1 -- 172.21.15.46:0/640236748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac007910 con 0x555636413630 2024-03-18T20:37:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.387+0000 7f6ec5befb80 1 -- 172.21.15.46:0/640236748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 msgr2=0x555636415a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.387+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/640236748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 0x555636415a20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6eac015000 tx=0x7f6eac0092e0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.387+0000 7f6ec5befb80 1 -- 172.21.15.46:0/640236748 shutdown_connections 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.387+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/640236748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 0x555636415a20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.387+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/640236748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x555636403e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.387+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/640236748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636404380 0x5556364130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.387+0000 7f6ec5befb80 1 -- 172.21.15.46:0/640236748 >> 172.21.15.46:0/640236748 conn(0x55563637a220 msgr2=0x555636400020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.387+0000 7f6ec5befb80 1 -- 172.21.15.46:0/640236748 shutdown_connections 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.388+0000 7f6ec5befb80 1 -- 172.21.15.46:0/640236748 wait complete. 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.388+0000 7f6ec5befb80 1 -- start start 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.388+0000 7f6ec5befb80 1 Processor -- start 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.388+0000 7f6ec5befb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x555636413090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.388+0000 7f6ec5befb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636404380 0x5556364103d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.388+0000 7f6ec5befb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636413630 0x555636410b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.388+0000 7f6ec5befb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556362f2d80 con 0x555636404380 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.388+0000 7f6ec5befb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556362f2a20 con 0x555636413630 2024-03-18T20:37:03.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6ec5befb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556361da6a0 con 0x555636403a60 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6ec5befb80 10 client.0 _ll_register_callbacks cb 0x55563615e3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb27fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636404380 0x5556364103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x555636413090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb27fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636404380 0x5556364103d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60676/0 (socket says 172.21.15.46:60676) 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb27fc700 1 -- 172.21.15.46:0/1449907487 learned_addr learned my addr 172.21.15.46:0/1449907487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x555636413090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:44314/0 (socket says 172.21.15.46:44314) 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb37fe700 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636413630 0x555636410b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb2ffd700 1 -- 172.21.15.46:0/1449907487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636413630 msgr2=0x555636410b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb2ffd700 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636413630 0x555636410b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb2ffd700 1 -- 172.21.15.46:0/1449907487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636404380 msgr2=0x5556364103d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb2ffd700 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636404380 0x5556364103d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb2ffd700 1 -- 172.21.15.46:0/1449907487 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea8017070 con 0x555636403a60 2024-03-18T20:37:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb27fc700 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636404380 0x5556364103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6eb2ffd700 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x555636413090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40ca730 tx=0x7f6ea409a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=773fd3fc21eefd68 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6e92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6e92ffd700 1 -- 172.21.15.46:0/1449907487 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40a2210 con 0x555636403a60 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6e92ffd700 1 -- 172.21.15.46:0/1449907487 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6ea409de50 con 0x555636403a60 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.389+0000 7f6ec5befb80 1 -- 172.21.15.46:0/1449907487 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556363f7ad0 con 0x555636403a60 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.390+0000 7f6e92ffd700 1 -- 172.21.15.46:0/1449907487 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ea40a24a0 con 0x555636403a60 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.390+0000 7f6ec5befb80 10 client.25645 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.390+0000 7f6ec5befb80 1 -- 172.21.15.46:0/1449907487 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f6eb410e460 con 0x555636403a60 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.390+0000 7f6e92ffd700 1 -- 172.21.15.46:0/1449907487 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5556363f7ad0 con 0x555636403a60 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.390+0000 7f6e92ffd700 1 -- 172.21.15.46:0/1449907487 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6ea40b4d20 con 0x555636403a60 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.390+0000 7f6ec5befb80 10 client.25645 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.390+0000 7f6ec5befb80 10 client.25645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.390+0000 7f6ec5befb80 20 client.25645 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.390+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x555636423090 0x5556364254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.390+0000 7f6ec5befb80 4 client.25645 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.390+0000 7f6ec5befb80 1 -- 172.21.15.46:0/1449907487 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55563634f700 con 0x555636423090 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.390+0000 7f6eb27fc700 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x555636423090 0x5556364254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.391+0000 7f6e92ffd700 10 client.25645 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.391+0000 7f6eb27fc700 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x555636423090 0x5556364254b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd99a34cecfc6aee server_cookie=1db807a1431a3952 in_seq=0 out_seq=0 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.391+0000 7f6e92ffd700 1 -- 172.21.15.46:0/1449907487 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55563634f700 con 0x555636423090 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.391+0000 7f6e92ffd700 10 client.25645 handle_command_reply: tid=0 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.391+0000 7f6ec5befb80 1 client.25645 shutdown 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.391+0000 7f6ec5befb80 1 -- 172.21.15.46:0/1449907487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 msgr2=0x555636413090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.391+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x555636413090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40ca730 tx=0x7f6ea409a560 comp rx=0 tx=0).stop 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.392+0000 7f6ec5befb80 1 -- 172.21.15.46:0/1449907487 shutdown_connections 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.392+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x555636423090 0x5556364254b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.392+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636413630 0x555636410b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.392+0000 7f6e92ffd700 0 client.25645 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:37:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.392+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x555636413090 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.392+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/1449907487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636404380 0x5556364103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.392+0000 7f6ec5befb80 1 -- 172.21.15.46:0/1449907487 >> 172.21.15.46:0/1449907487 conn(0x55563637a220 msgr2=0x5556363ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.392+0000 7f6ec5befb80 1 -- 172.21.15.46:0/1449907487 shutdown_connections 2024-03-18T20:37:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.392+0000 7f6ec5befb80 1 -- 172.21.15.46:0/1449907487 wait complete. 2024-03-18T20:37:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.392+0000 7f6ec5befb80 20 client.25645 trim_cache size 0 max 16384 2024-03-18T20:37:03.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.419+0000 7f6ec5befb80 1 Processor -- start 2024-03-18T20:37:03.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.419+0000 7f6ec5befb80 1 -- start start 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.419+0000 7f6ec5befb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x55563641a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.419+0000 7f6ec5befb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636404380 0x55563640b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.419+0000 7f6ec5befb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 0x55563640bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.419+0000 7f6ec5befb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556363fa210 con 0x555636413630 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.419+0000 7f6ec5befb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55563641fd40 con 0x555636404380 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.419+0000 7f6ec5befb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555636405550 con 0x555636403a60 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.419+0000 7f6eb37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 0x55563640bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.419+0000 7f6eb37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 0x55563640bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60682/0 (socket says 172.21.15.46:60682) 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.419+0000 7f6eb37fe700 1 -- 172.21.15.46:0/2864730352 learned_addr learned my addr 172.21.15.46:0/2864730352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.420+0000 7f6eb27fc700 1 --2- 172.21.15.46:0/2864730352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636404380 0x55563640b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.420+0000 7f6eb2ffd700 1 --2- 172.21.15.46:0/2864730352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x55563641a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.420+0000 7f6eb37fe700 1 -- 172.21.15.46:0/2864730352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 msgr2=0x55563641a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.420+0000 7f6eb37fe700 1 --2- 172.21.15.46:0/2864730352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x55563641a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.420+0000 7f6eb37fe700 1 -- 172.21.15.46:0/2864730352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636404380 msgr2=0x55563640b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.420+0000 7f6eb37fe700 1 --2- 172.21.15.46:0/2864730352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636404380 0x55563640b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.420+0000 7f6eb37fe700 1 -- 172.21.15.46:0/2864730352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555636184b60 con 0x555636413630 2024-03-18T20:37:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.420+0000 7f6eb2ffd700 1 --2- 172.21.15.46:0/2864730352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x55563641a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.420+0000 7f6eb37fe700 1 --2- 172.21.15.46:0/2864730352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 0x55563640bc70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009a20 tx=0x7f6eac009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2344f9257d6edcd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.420+0000 7f6e9a7fc700 1 -- 172.21.15.46:0/2864730352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00e040 con 0x555636413630 2024-03-18T20:37:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.420+0000 7f6e9a7fc700 1 -- 172.21.15.46:0/2864730352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eac007c70 con 0x555636413630 2024-03-18T20:37:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.420+0000 7f6e9a7fc700 1 -- 172.21.15.46:0/2864730352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac003c60 con 0x555636413630 2024-03-18T20:37:03.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.421+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2864730352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 msgr2=0x55563640bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.421+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/2864730352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 0x55563640bc70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009a20 tx=0x7f6eac009b80 comp rx=0 tx=0).stop 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.421+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2864730352 shutdown_connections 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.421+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/2864730352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636413630 0x55563640bc70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.421+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/2864730352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636403a60 0x55563641a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.421+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/2864730352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636404380 0x55563640b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.421+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2864730352 >> 172.21.15.46:0/2864730352 conn(0x55563637a220 msgr2=0x555636412ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.421+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2864730352 shutdown_connections 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.421+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2864730352 wait complete. 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.421+0000 7f6ec5befb80 1 -- start start 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.422+0000 7f6ec5befb80 1 Processor -- start 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.422+0000 7f6ec5befb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636403a60 0x555636419c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.422+0000 7f6ec5befb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636404380 0x55563641a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.422+0000 7f6ec5befb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636413630 0x555636418240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.422+0000 7f6ec5befb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555636418950 con 0x555636403a60 2024-03-18T20:37:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.422+0000 7f6ec5befb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555636418ac0 con 0x555636413630 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.422+0000 7f6ec5befb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555636418c80 con 0x555636404380 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.422+0000 7f6ec5befb80 10 client.0 _ll_register_callbacks cb 0x7f6ea4099b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.422+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636403a60 0x555636419c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.422+0000 7f6eb2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636403a60 0x555636419c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60692/0 (socket says 172.21.15.46:60692) 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.422+0000 7f6eb2ffd700 1 -- 172.21.15.46:0/2771909875 learned_addr learned my addr 172.21.15.46:0/2771909875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.422+0000 7f6eb37fe700 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636413630 0x555636418240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6eb27fc700 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636404380 0x55563641a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6eb2ffd700 1 -- 172.21.15.46:0/2771909875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636404380 msgr2=0x55563641a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6eb2ffd700 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636404380 0x55563641a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6eb2ffd700 1 -- 172.21.15.46:0/2771909875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636413630 msgr2=0x555636418240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6eb2ffd700 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636413630 0x555636418240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6eb2ffd700 1 -- 172.21.15.46:0/2771909875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556361d83b0 con 0x555636403a60 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6eb2ffd700 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636403a60 0x555636419c30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40cd220 tx=0x7f6ea400e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac77b0ff9a2ea674 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6e92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6e92ffd700 1 -- 172.21.15.46:0/2771909875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40992d0 con 0x555636403a60 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6e92ffd700 1 -- 172.21.15.46:0/2771909875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6ea409de50 con 0x555636403a60 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2771909875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eac006000 con 0x555636403a60 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6e92ffd700 1 -- 172.21.15.46:0/2771909875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ea40b83b0 con 0x555636403a60 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6ec5befb80 10 client.15882 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2771909875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55563631cea0 con 0x555636403a60 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.423+0000 7f6e92ffd700 1 -- 172.21.15.46:0/2771909875 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eac006000 con 0x555636403a60 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.424+0000 7f6e92ffd700 1 -- 172.21.15.46:0/2771909875 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6ea40a2010 con 0x555636403a60 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.424+0000 7f6ec5befb80 10 client.15882 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.424+0000 7f6ec5befb80 10 client.15882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.424+0000 7f6ec5befb80 20 client.15882 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.424+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x555636477d00 0x55563647a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.424+0000 7f6ec5befb80 4 client.15882 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:37:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.424+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2771909875 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5556361a5cf0 con 0x555636477d00 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.424+0000 7f6eb27fc700 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x555636477d00 0x55563647a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.424+0000 7f6e92ffd700 10 client.15882 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.424+0000 7f6eb27fc700 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x555636477d00 0x55563647a0f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65df2c68a63e0072 server_cookie=e161fb4149e89d1c in_seq=0 out_seq=0 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6e92ffd700 1 -- 172.21.15.46:0/2771909875 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5556361a5cf0 con 0x555636477d00 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6e92ffd700 10 client.15882 handle_command_reply: tid=0 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6ec5befb80 1 client.15882 shutdown 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2771909875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636403a60 msgr2=0x555636419c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636403a60 0x555636419c30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40cd220 tx=0x7f6ea400e930 comp rx=0 tx=0).stop 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2771909875 shutdown_connections 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x555636477d00 0x55563647a0f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555636403a60 0x555636419c30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555636413630 0x555636418240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6e92ffd700 0 client.15882 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6ec5befb80 1 --2- 172.21.15.46:0/2771909875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555636404380 0x55563641a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2771909875 >> 172.21.15.46:0/2771909875 conn(0x55563637a220 msgr2=0x55563640ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2771909875 shutdown_connections 2024-03-18T20:37:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6ec5befb80 1 -- 172.21.15.46:0/2771909875 wait complete. 2024-03-18T20:37:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.425+0000 7f6ec5befb80 20 client.15882 trim_cache size 0 max 16384 2024-03-18T20:37:03.434 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:37:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.426+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/4041206252 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6e9c0a13b0 msgr2=0x7f6e9c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.426+0000 7f6eb9a5a700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6e9c0a13b0 0x7f6e9c0a3870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4198870 tx=0x7f6ea8009260 comp rx=0 tx=0).stop 2024-03-18T20:37:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.426+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/4041206252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb4100f50 msgr2=0x7f6eb4197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.426+0000 7f6eb9a5a700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb4100f50 0x7f6eb4197320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400e9a0 tx=0x7f6ea400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.426+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/4041206252 shutdown_connections 2024-03-18T20:37:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.426+0000 7f6eb9a5a700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6e9c0a13b0 0x7f6e9c0a3870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.426+0000 7f6eb9a5a700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6eb4100f50 0x7f6eb4197320 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.426+0000 7f6eb9a5a700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6eb419bc00 0x7f6eb419c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.426+0000 7f6eb9a5a700 1 --2- 172.21.15.46:0/4041206252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6eb4106070 0x7f6eb4197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.426+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/4041206252 >> 172.21.15.46:0/4041206252 conn(0x7f6eb40fc7f0 msgr2=0x7f6eb40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.426+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/4041206252 shutdown_connections 2024-03-18T20:37:03.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.426+0000 7f6eb9a5a700 1 -- 172.21.15.46:0/4041206252 wait complete. 2024-03-18T20:37:03.442 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:37:03.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:37:03.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:03 smithi073 ceph-mon[47384]: mgrmap e24: x(active, since 55s), standbys: z, y 2024-03-18T20:37:03.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:03 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1223195923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:37:03.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:03 smithi155 ceph-mon[48344]: mgrmap e24: x(active, since 55s), standbys: z, y 2024-03-18T20:37:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:03 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1223195923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:37:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.581+0000 7f97853a0700 1 -- 172.21.15.46:0/3289167888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9780106070 msgr2=0x7f978010dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.581+0000 7f97853a0700 1 --2- 172.21.15.46:0/3289167888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9780106070 0x7f978010dbf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f977800b0a0 tx=0x7f977800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.581+0000 7f97853a0700 1 -- 172.21.15.46:0/3289167888 shutdown_connections 2024-03-18T20:37:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.581+0000 7f97853a0700 1 --2- 172.21.15.46:0/3289167888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9780106070 0x7f978010dbf0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.581+0000 7f97853a0700 1 --2- 172.21.15.46:0/3289167888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9780100f50 0x7f9780101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.581+0000 7f97853a0700 1 --2- 172.21.15.46:0/3289167888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9780101900 0x7f9780105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.581+0000 7f97853a0700 1 -- 172.21.15.46:0/3289167888 >> 172.21.15.46:0/3289167888 conn(0x7f97800fc7f0 msgr2=0x7f97800fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.581+0000 7f97853a0700 1 -- 172.21.15.46:0/3289167888 shutdown_connections 2024-03-18T20:37:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.581+0000 7f97853a0700 1 -- 172.21.15.46:0/3289167888 wait complete. 2024-03-18T20:37:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.582+0000 7f97853a0700 1 Processor -- start 2024-03-18T20:37:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.582+0000 7f97853a0700 1 -- start start 2024-03-18T20:37:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.582+0000 7f97853a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9780100f50 0x7f9780199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.582+0000 7f97853a0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9780106070 0x7f9780199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.582+0000 7f97853a0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f978019de00 0x7f978019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.582+0000 7f97853a0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978019e880 con 0x7f9780100f50 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.582+0000 7f97853a0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978019e9f0 con 0x7f978019de00 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.582+0000 7f97853a0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978019ebb0 con 0x7f9780106070 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f977effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9780100f50 0x7f9780199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f977effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9780100f50 0x7f9780199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60706/0 (socket says 172.21.15.46:60706) 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f977effd700 1 -- 172.21.15.46:0/2694203620 learned_addr learned my addr 172.21.15.46:0/2694203620 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f977e7fc700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9780106070 0x7f9780199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f977f7fe700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f978019de00 0x7f978019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f977effd700 1 -- 172.21.15.46:0/2694203620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9780106070 msgr2=0x7f9780199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f977effd700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9780106070 0x7f9780199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f977effd700 1 -- 172.21.15.46:0/2694203620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f978019de00 msgr2=0x7f978019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f977effd700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f978019de00 0x7f978019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f977effd700 1 -- 172.21.15.46:0/2694203620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9778009d00 con 0x7f9780100f50 2024-03-18T20:37:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f977effd700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9780100f50 0x7f9780199520 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f977000e9a0 tx=0x7f977000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f9767fff700 1 -- 172.21.15.46:0/2694203620 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770009790 con 0x7f9780100f50 2024-03-18T20:37:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f97853a0700 1 -- 172.21.15.46:0/2694203620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97801a6110 con 0x7f9780100f50 2024-03-18T20:37:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f9767fff700 1 -- 172.21.15.46:0/2694203620 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9770010430 con 0x7f9780100f50 2024-03-18T20:37:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f9767fff700 1 -- 172.21.15.46:0/2694203620 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770018680 con 0x7f9780100f50 2024-03-18T20:37:03.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.583+0000 7f97853a0700 1 -- 172.21.15.46:0/2694203620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97801a65f0 con 0x7f9780100f50 2024-03-18T20:37:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.586+0000 7f9767fff700 1 -- 172.21.15.46:0/2694203620 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f97700105a0 con 0x7f9780100f50 2024-03-18T20:37:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.586+0000 7f9767fff700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f97680a13b0 0x7f97680a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.586+0000 7f9767fff700 1 -- 172.21.15.46:0/2694203620 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9770014070 con 0x7f9780100f50 2024-03-18T20:37:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.586+0000 7f977e7fc700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f97680a13b0 0x7f97680a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.587+0000 7f977e7fc700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f97680a13b0 0x7f97680a3870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f978019aa70 tx=0x7f977400b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.587+0000 7f9791535b80 1 Processor -- start 2024-03-18T20:37:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f9791535b80 1 -- start start 2024-03-18T20:37:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f9791535b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116235ea60 0x56116235ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f9791535b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235f380 0x561162369ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f9791535b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116236a220 0x56116236c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f9791535b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611622b8990 con 0x56116236a220 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f9791535b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611622b6350 con 0x56116235f380 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f9791535b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611622b1c40 con 0x56116235ea60 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f977f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116236a220 0x56116236c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f977e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235f380 0x561162369ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f977f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116236a220 0x56116236c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60718/0 (socket says 172.21.15.46:60718) 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f977f7fe700 1 -- 172.21.15.46:0/2638388526 learned_addr learned my addr 172.21.15.46:0/2638388526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f977effd700 1 --2- 172.21.15.46:0/2638388526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116235ea60 0x56116235ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f977e7fc700 1 -- 172.21.15.46:0/2638388526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116235ea60 msgr2=0x56116235ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f977e7fc700 1 --2- 172.21.15.46:0/2638388526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116235ea60 0x56116235ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f977e7fc700 1 -- 172.21.15.46:0/2638388526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116236a220 msgr2=0x56116236c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f977e7fc700 1 --2- 172.21.15.46:0/2638388526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116236a220 0x56116236c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f977e7fc700 1 -- 172.21.15.46:0/2638388526 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611621333b0 con 0x56116235f380 2024-03-18T20:37:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.588+0000 7f977f7fe700 1 --2- 172.21.15.46:0/2638388526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116236a220 0x56116236c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.589+0000 7f977e7fc700 1 --2- 172.21.15.46:0/2638388526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235f380 0x561162369ce0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f978006a580 tx=0x7f9774013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8768727196f6463 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.589+0000 7f97657fa700 1 -- 172.21.15.46:0/2638388526 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774011ad0 con 0x56116235f380 2024-03-18T20:37:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.589+0000 7f97657fa700 1 -- 172.21.15.46:0/2638388526 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9774015070 con 0x56116235f380 2024-03-18T20:37:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.589+0000 7f97657fa700 1 -- 172.21.15.46:0/2638388526 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977401c3b0 con 0x56116235f380 2024-03-18T20:37:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.590+0000 7f9791535b80 1 -- 172.21.15.46:0/2638388526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235f380 msgr2=0x561162369ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.590+0000 7f9791535b80 1 --2- 172.21.15.46:0/2638388526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235f380 0x561162369ce0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f978006a580 tx=0x7f9774013000 comp rx=0 tx=0).stop 2024-03-18T20:37:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.590+0000 7f9791535b80 1 -- 172.21.15.46:0/2638388526 shutdown_connections 2024-03-18T20:37:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.590+0000 7f9791535b80 1 --2- 172.21.15.46:0/2638388526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116236a220 0x56116236c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.590+0000 7f9791535b80 1 --2- 172.21.15.46:0/2638388526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116235ea60 0x56116235ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.590+0000 7f9791535b80 1 --2- 172.21.15.46:0/2638388526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235f380 0x561162369ce0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.590+0000 7f9791535b80 1 -- 172.21.15.46:0/2638388526 >> 172.21.15.46:0/2638388526 conn(0x5611622d5220 msgr2=0x56116235b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.590+0000 7f9791535b80 1 -- 172.21.15.46:0/2638388526 shutdown_connections 2024-03-18T20:37:03.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.590+0000 7f9791535b80 1 -- 172.21.15.46:0/2638388526 wait complete. 2024-03-18T20:37:03.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.590+0000 7f9791535b80 1 -- start start 2024-03-18T20:37:03.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.590+0000 7f9791535b80 1 Processor -- start 2024-03-18T20:37:03.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f9791535b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235ea60 0x561162369cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f9791535b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 0x561162366fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f9791535b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116236a220 0x561162367700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f9791535b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116224dd80 con 0x56116235f380 2024-03-18T20:37:03.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f9791535b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116224da20 con 0x56116235ea60 2024-03-18T20:37:03.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f9791535b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611621356a0 con 0x56116236a220 2024-03-18T20:37:03.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f9791535b80 10 client.0 _ll_register_callbacks cb 0x5611620b93a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:03.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f977e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 0x561162366fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f977effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235ea60 0x561162369cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f977e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 0x561162366fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60720/0 (socket says 172.21.15.46:60720) 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f977e7fc700 1 -- 172.21.15.46:0/3775799991 learned_addr learned my addr 172.21.15.46:0/3775799991 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f977f7fe700 1 --2- 172.21.15.46:0/3775799991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116236a220 0x561162367700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f977effd700 1 -- 172.21.15.46:0/3775799991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116236a220 msgr2=0x561162367700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f977effd700 1 --2- 172.21.15.46:0/3775799991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116236a220 0x561162367700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f977effd700 1 -- 172.21.15.46:0/3775799991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 msgr2=0x561162366fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f977effd700 1 --2- 172.21.15.46:0/3775799991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 0x561162366fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.591+0000 7f977effd700 1 -- 172.21.15.46:0/3775799991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978010e460 con 0x56116235ea60 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f977effd700 1 --2- 172.21.15.46:0/3775799991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235ea60 0x561162369cd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f97700ca740 tx=0x7f977009a570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70fe519d4d48ec2d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9762ff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9762ff5700 1 -- 172.21.15.46:0/3775799991 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97700b1070 con 0x56116235ea60 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9791535b80 1 -- 172.21.15.46:0/3775799991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561162352ad0 con 0x56116235ea60 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9762ff5700 1 -- 172.21.15.46:0/3775799991 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f977009de60 con 0x56116235ea60 2024-03-18T20:37:03.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9762ff5700 1 -- 172.21.15.46:0/3775799991 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97700a24b0 con 0x56116235ea60 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9791535b80 10 client.25664 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9791535b80 1 -- 172.21.15.46:0/3775799991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f9774010a20 con 0x56116235ea60 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9762ff5700 1 -- 172.21.15.46:0/3775799991 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9770063080 con 0x56116235ea60 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9762ff5700 1 -- 172.21.15.46:0/3775799991 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f97700caa20 con 0x56116235ea60 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9791535b80 10 client.25664 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9791535b80 10 client.25664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9791535b80 20 client.25664 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9791535b80 1 --2- 172.21.15.46:0/3775799991 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561162379ea0 0x56116237c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9791535b80 4 client.25664 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.592+0000 7f9791535b80 1 -- 172.21.15.46:0/3775799991 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611622aa700 con 0x561162379ea0 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.593+0000 7f977e7fc700 1 --2- 172.21.15.46:0/3775799991 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561162379ea0 0x56116237c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.593+0000 7f977e7fc700 1 --2- 172.21.15.46:0/3775799991 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561162379ea0 0x56116237c2c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a1bc21e6067f10 server_cookie=fa8f3df9ba22ab6a in_seq=0 out_seq=0 2024-03-18T20:37:03.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.593+0000 7f9762ff5700 10 client.25664 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9762ff5700 1 -- 172.21.15.46:0/3775799991 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5611622aa700 con 0x561162379ea0 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9762ff5700 10 client.25664 handle_command_reply: tid=0 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9791535b80 1 client.25664 shutdown 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9791535b80 1 -- 172.21.15.46:0/3775799991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235ea60 msgr2=0x561162369cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9791535b80 1 --2- 172.21.15.46:0/3775799991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235ea60 0x561162369cd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f97700ca740 tx=0x7f977009a570 comp rx=0 tx=0).stop 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9791535b80 1 -- 172.21.15.46:0/3775799991 shutdown_connections 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9791535b80 1 --2- 172.21.15.46:0/3775799991 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561162379ea0 0x56116237c2c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9791535b80 1 --2- 172.21.15.46:0/3775799991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116236a220 0x561162367700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9762ff5700 0 client.25664 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9791535b80 1 --2- 172.21.15.46:0/3775799991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235ea60 0x561162369cd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9791535b80 1 --2- 172.21.15.46:0/3775799991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 0x561162366fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9791535b80 1 -- 172.21.15.46:0/3775799991 >> 172.21.15.46:0/3775799991 conn(0x5611622d5220 msgr2=0x56116235ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9791535b80 1 -- 172.21.15.46:0/3775799991 shutdown_connections 2024-03-18T20:37:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9791535b80 1 -- 172.21.15.46:0/3775799991 wait complete. 2024-03-18T20:37:03.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.594+0000 7f9791535b80 20 client.25664 trim_cache size 0 max 16384 2024-03-18T20:37:03.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.608+0000 7f9791535b80 1 Processor -- start 2024-03-18T20:37:03.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.608+0000 7f9791535b80 1 -- start start 2024-03-18T20:37:03.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.608+0000 7f9791535b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235ea60 0x56116237c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.608+0000 7f9791535b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235f380 0x561162370e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.608+0000 7f9791535b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116236a220 0x5611623698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.608+0000 7f9791535b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561162376360 con 0x56116235ea60 2024-03-18T20:37:03.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.608+0000 7f9791535b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561162360550 con 0x56116235f380 2024-03-18T20:37:03.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.608+0000 7f9791535b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561162371390 con 0x56116236a220 2024-03-18T20:37:03.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.608+0000 7f977effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235ea60 0x56116237c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.608+0000 7f977effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235ea60 0x56116237c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60730/0 (socket says 172.21.15.46:60730) 2024-03-18T20:37:03.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f977effd700 1 -- 172.21.15.46:0/402664657 learned_addr learned my addr 172.21.15.46:0/402664657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f977e7fc700 1 --2- 172.21.15.46:0/402664657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235f380 0x561162370e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f977f7fe700 1 --2- 172.21.15.46:0/402664657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116236a220 0x5611623698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f977e7fc700 1 -- 172.21.15.46:0/402664657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116236a220 msgr2=0x5611623698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f977e7fc700 1 --2- 172.21.15.46:0/402664657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116236a220 0x5611623698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f977e7fc700 1 -- 172.21.15.46:0/402664657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235ea60 msgr2=0x56116237c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f977e7fc700 1 --2- 172.21.15.46:0/402664657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235ea60 0x56116237c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f977e7fc700 1 -- 172.21.15.46:0/402664657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611620dfb60 con 0x56116235f380 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f977effd700 1 --2- 172.21.15.46:0/402664657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235ea60 0x56116237c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f977e7fc700 1 --2- 172.21.15.46:0/402664657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235f380 0x561162370e50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x561162368500 tx=0x7f9774016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbc4296967e866e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f9765ffb700 1 -- 172.21.15.46:0/402664657 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977402b070 con 0x56116235f380 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f9765ffb700 1 -- 172.21.15.46:0/402664657 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f977402c040 con 0x56116235f380 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.609+0000 7f9765ffb700 1 -- 172.21.15.46:0/402664657 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774011bc0 con 0x56116235f380 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.610+0000 7f9791535b80 1 -- 172.21.15.46:0/402664657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235f380 msgr2=0x561162370e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.610+0000 7f9791535b80 1 --2- 172.21.15.46:0/402664657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235f380 0x561162370e50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x561162368500 tx=0x7f9774016000 comp rx=0 tx=0).stop 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.610+0000 7f9791535b80 1 -- 172.21.15.46:0/402664657 shutdown_connections 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.610+0000 7f9791535b80 1 --2- 172.21.15.46:0/402664657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235ea60 0x56116237c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.610+0000 7f9791535b80 1 --2- 172.21.15.46:0/402664657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116236a220 0x5611623698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.610+0000 7f9791535b80 1 --2- 172.21.15.46:0/402664657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116235f380 0x561162370e50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.610+0000 7f9791535b80 1 -- 172.21.15.46:0/402664657 >> 172.21.15.46:0/402664657 conn(0x5611622d5220 msgr2=0x561162356a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.610+0000 7f9791535b80 1 -- 172.21.15.46:0/402664657 shutdown_connections 2024-03-18T20:37:03.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.610+0000 7f9791535b80 1 -- 172.21.15.46:0/402664657 wait complete. 2024-03-18T20:37:03.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.610+0000 7f9791535b80 1 -- start start 2024-03-18T20:37:03.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f9791535b80 1 Processor -- start 2024-03-18T20:37:03.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f9791535b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116235ea60 0x561162370640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f9791535b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 0x561162370b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f9791535b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116236a220 0x56116236ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f9791535b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116236f3c0 con 0x56116235f380 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f9791535b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116236f530 con 0x56116236a220 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f9791535b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116236f6f0 con 0x56116235ea60 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f9791535b80 10 client.0 _ll_register_callbacks cb 0x7f9770099c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f977e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 0x561162370b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f977effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116235ea60 0x561162370640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f977e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 0x561162370b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60746/0 (socket says 172.21.15.46:60746) 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f977e7fc700 1 -- 172.21.15.46:0/1239911969 learned_addr learned my addr 172.21.15.46:0/1239911969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f977f7fe700 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116236a220 0x56116236ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f977e7fc700 1 -- 172.21.15.46:0/1239911969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116235ea60 msgr2=0x561162370640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f977e7fc700 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116235ea60 0x561162370640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f977e7fc700 1 -- 172.21.15.46:0/1239911969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116236a220 msgr2=0x56116236ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.611+0000 7f977e7fc700 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116236a220 0x56116236ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.612+0000 7f977e7fc700 1 -- 172.21.15.46:0/1239911969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9774010a20 con 0x56116235f380 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.612+0000 7f977effd700 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116235ea60 0x561162370640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.612+0000 7f977f7fe700 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116236a220 0x56116236ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.612+0000 7f977e7fc700 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 0x561162370b80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9774034040 tx=0x7f9774032610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b3bfe7e1299963 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.612+0000 7f9762ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.612+0000 7f9762ff5700 1 -- 172.21.15.46:0/1239911969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977402b040 con 0x56116235f380 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.612+0000 7f9762ff5700 1 -- 172.21.15.46:0/1239911969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9774017070 con 0x56116235f380 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.612+0000 7f9791535b80 1 -- 172.21.15.46:0/1239911969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9774015070 con 0x56116235f380 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.612+0000 7f9762ff5700 1 -- 172.21.15.46:0/1239911969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f977401ea50 con 0x56116235f380 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.612+0000 7f9791535b80 10 client.15900 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.612+0000 7f9791535b80 1 -- 172.21.15.46:0/1239911969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5611621333b0 con 0x56116235f380 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.612+0000 7f9762ff5700 1 -- 172.21.15.46:0/1239911969 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9774015070 con 0x56116235f380 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.613+0000 7f9762ff5700 1 -- 172.21.15.46:0/1239911969 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9774041020 con 0x56116235f380 2024-03-18T20:37:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.613+0000 7f9791535b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.613+0000 7f9791535b80 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.613+0000 7f9791535b80 20 client.15900 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.613+0000 7f9791535b80 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5611623ceac0 0x5611623d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.613+0000 7f9791535b80 4 client.15900 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.613+0000 7f9791535b80 1 -- 172.21.15.46:0/1239911969 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561162100cf0 con 0x5611623ceac0 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.613+0000 7f977effd700 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5611623ceac0 0x5611623d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.613+0000 7f9762ff5700 10 client.15900 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.613+0000 7f977effd700 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5611623ceac0 0x5611623d0ee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b2d483ae70f6c5c server_cookie=ab1387bc723e1 in_seq=0 out_seq=0 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.613+0000 7f9762ff5700 1 -- 172.21.15.46:0/1239911969 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561162100cf0 con 0x5611623ceac0 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.613+0000 7f9762ff5700 10 client.15900 handle_command_reply: tid=0 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9791535b80 1 client.15900 shutdown 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9791535b80 1 -- 172.21.15.46:0/1239911969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 msgr2=0x561162370b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9791535b80 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 0x561162370b80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9774034040 tx=0x7f9774032610 comp rx=0 tx=0).stop 2024-03-18T20:37:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9791535b80 1 -- 172.21.15.46:0/1239911969 shutdown_connections 2024-03-18T20:37:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9791535b80 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5611623ceac0 0x5611623d0ee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9791535b80 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56116236a220 0x56116236ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9791535b80 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56116235ea60 0x561162370640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9791535b80 1 --2- 172.21.15.46:0/1239911969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56116235f380 0x561162370b80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9791535b80 1 -- 172.21.15.46:0/1239911969 >> 172.21.15.46:0/1239911969 conn(0x5611622d5220 msgr2=0x5611623755a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9762ff5700 0 client.15900 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9791535b80 1 -- 172.21.15.46:0/1239911969 shutdown_connections 2024-03-18T20:37:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9791535b80 1 -- 172.21.15.46:0/1239911969 wait complete. 2024-03-18T20:37:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f9791535b80 20 client.15900 trim_cache size 0 max 16384 2024-03-18T20:37:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:37:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f97853a0700 1 -- 172.21.15.46:0/2694203620 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f97680a13b0 msgr2=0x7f97680a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.614+0000 7f97853a0700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f97680a13b0 0x7f97680a3870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f978019aa70 tx=0x7f977400b410 comp rx=0 tx=0).stop 2024-03-18T20:37:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.615+0000 7f97853a0700 1 -- 172.21.15.46:0/2694203620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9780100f50 msgr2=0x7f9780199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.615+0000 7f97853a0700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9780100f50 0x7f9780199520 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f977000e9a0 tx=0x7f977000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.615+0000 7f97853a0700 1 -- 172.21.15.46:0/2694203620 shutdown_connections 2024-03-18T20:37:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.615+0000 7f97853a0700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f97680a13b0 0x7f97680a3870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.615+0000 7f97853a0700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9780100f50 0x7f9780199520 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.615+0000 7f97853a0700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f978019de00 0x7f978019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.615+0000 7f97853a0700 1 --2- 172.21.15.46:0/2694203620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9780106070 0x7f9780199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.615+0000 7f97853a0700 1 -- 172.21.15.46:0/2694203620 >> 172.21.15.46:0/2694203620 conn(0x7f97800fc7f0 msgr2=0x7f97800feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.615+0000 7f97853a0700 1 -- 172.21.15.46:0/2694203620 shutdown_connections 2024-03-18T20:37:03.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:03.615+0000 7f97853a0700 1 -- 172.21.15.46:0/2694203620 wait complete. 2024-03-18T20:37:03.919 INFO:tasks.workunit.client.0.smithi046.stderr:Updating files: 82% (10009/12088) Updating files: 83% (10034/12088) Updating files: 84% (10154/12088) Updating files: 85% (10275/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-03-18T20:37:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:04 smithi073 ceph-mon[47384]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-03-18T20:37:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:04 smithi155 ceph-mon[48344]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-03-18T20:37:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:04 smithi046 ceph-mon[43495]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-03-18T20:37:04.633 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:37:04.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:37:04.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:37:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.775+0000 7f3407872700 1 -- 172.21.15.46:0/2126362976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400100820 msgr2=0x7f3400104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.775+0000 7f3407872700 1 --2- 172.21.15.46:0/2126362976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400100820 0x7f3400104860 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009990 tx=0x7f33f0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.776+0000 7f3407872700 1 -- 172.21.15.46:0/2126362976 shutdown_connections 2024-03-18T20:37:04.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.776+0000 7f3407872700 1 --2- 172.21.15.46:0/2126362976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3400104f90 0x7f340010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.776+0000 7f3407872700 1 --2- 172.21.15.46:0/2126362976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34000ffe70 0x7f3400100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.776+0000 7f3407872700 1 --2- 172.21.15.46:0/2126362976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3400100820 0x7f3400104860 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.776+0000 7f3407872700 1 -- 172.21.15.46:0/2126362976 >> 172.21.15.46:0/2126362976 conn(0x7f3400076d80 msgr2=0x7f3400077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:04.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.776+0000 7f3407872700 1 -- 172.21.15.46:0/2126362976 shutdown_connections 2024-03-18T20:37:04.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.776+0000 7f3407872700 1 -- 172.21.15.46:0/2126362976 wait complete. 2024-03-18T20:37:04.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3407872700 1 Processor -- start 2024-03-18T20:37:04.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3407872700 1 -- start start 2024-03-18T20:37:04.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3407872700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34000ffe70 0x7f3400199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3407872700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400104f90 0x7f3400199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3407872700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f340019dce0 0x7f340019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3407872700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340019e760 con 0x7f340019dce0 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3407872700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340019e8d0 con 0x7f34000ffe70 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3407872700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340019ea90 con 0x7f3400104f90 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3405e0f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f340019dce0 0x7f340019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3405e0f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f340019dce0 0x7f340019e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60770/0 (socket says 172.21.15.46:60770) 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3405e0f700 1 -- 172.21.15.46:0/2429590660 learned_addr learned my addr 172.21.15.46:0/2429590660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f340560e700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34000ffe70 0x7f3400199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3404e0d700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400104f90 0x7f3400199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3405e0f700 1 -- 172.21.15.46:0/2429590660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400104f90 msgr2=0x7f3400199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3405e0f700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400104f90 0x7f3400199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3405e0f700 1 -- 172.21.15.46:0/2429590660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34000ffe70 msgr2=0x7f3400199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.777+0000 7f3405e0f700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34000ffe70 0x7f3400199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.778+0000 7f3405e0f700 1 -- 172.21.15.46:0/2429590660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f00095f0 con 0x7f340019dce0 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.778+0000 7f340560e700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34000ffe70 0x7f3400199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.778+0000 7f3405e0f700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f340019dce0 0x7f340019e160 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f33f800ef30 tx=0x7f33f800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:04.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.778+0000 7f33f67fc700 1 -- 172.21.15.46:0/2429590660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8018070 con 0x7f340019dce0 2024-03-18T20:37:04.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.778+0000 7f33f67fc700 1 -- 172.21.15.46:0/2429590660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33f800f040 con 0x7f340019dce0 2024-03-18T20:37:04.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.778+0000 7f3407872700 1 -- 172.21.15.46:0/2429590660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34001a5f90 con 0x7f340019dce0 2024-03-18T20:37:04.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.778+0000 7f33f67fc700 1 -- 172.21.15.46:0/2429590660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8003dc0 con 0x7f340019dce0 2024-03-18T20:37:04.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.778+0000 7f3407872700 1 -- 172.21.15.46:0/2429590660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34001a6420 con 0x7f340019dce0 2024-03-18T20:37:04.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.781+0000 7f33f67fc700 1 -- 172.21.15.46:0/2429590660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f33f800caa0 con 0x7f340019dce0 2024-03-18T20:37:04.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.782+0000 7f33f67fc700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f33ec0a13b0 0x7f33ec0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.782+0000 7f340560e700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f33ec0a13b0 0x7f33ec0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.782+0000 7f33f67fc700 1 -- 172.21.15.46:0/2429590660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f33f80ccc00 con 0x7f340019dce0 2024-03-18T20:37:04.786 INFO:tasks.workunit.client.0.smithi046.stderr:Note: switching to '49bdc51142759622a281d3e5327eebd07cbb8bb3'. 2024-03-18T20:37:04.786 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-03-18T20:37:04.786 INFO:tasks.workunit.client.0.smithi046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-18T20:37:04.786 INFO:tasks.workunit.client.0.smithi046.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-18T20:37:04.786 INFO:tasks.workunit.client.0.smithi046.stderr:state without impacting any branches by switching back to a branch. 2024-03-18T20:37:04.786 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-03-18T20:37:04.786 INFO:tasks.workunit.client.0.smithi046.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-18T20:37:04.786 INFO:tasks.workunit.client.0.smithi046.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-18T20:37:04.786 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-03-18T20:37:04.787 INFO:tasks.workunit.client.0.smithi046.stderr: git switch -c 2024-03-18T20:37:04.787 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-03-18T20:37:04.787 INFO:tasks.workunit.client.0.smithi046.stderr:Or undo this operation with: 2024-03-18T20:37:04.787 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-03-18T20:37:04.787 INFO:tasks.workunit.client.0.smithi046.stderr: git switch - 2024-03-18T20:37:04.787 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-03-18T20:37:04.787 INFO:tasks.workunit.client.0.smithi046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-18T20:37:04.787 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-03-18T20:37:04.787 INFO:tasks.workunit.client.0.smithi046.stderr:HEAD is now at 49bdc511427 Merge branch 'wip-63770-reef' of https://github.com/idryomov/ceph into wip-yuri10-testing-2024-03-15-1653-reef 2024-03-18T20:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3413a07b80 1 Processor -- start 2024-03-18T20:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3413a07b80 1 -- start start 2024-03-18T20:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f340560e700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f33ec0a13b0 0x7f33ec0a3870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009cb0 tx=0x7f33fc009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3413a07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b750ae0 0x55ea3b750ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3413a07b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 0x55ea3b75bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3413a07b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b75c2a0 0x55ea3b75e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3413a07b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3b6aaa10 con 0x55ea3b751400 2024-03-18T20:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3413a07b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3b6a83d0 con 0x55ea3b75c2a0 2024-03-18T20:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3413a07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3b6a3cc0 con 0x55ea3b750ae0 2024-03-18T20:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3404e0d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 0x55ea3b75bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3404e0d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 0x55ea3b75bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60782/0 (socket says 172.21.15.46:60782) 2024-03-18T20:37:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3404e0d700 1 -- 172.21.15.46:0/2615801992 learned_addr learned my addr 172.21.15.46:0/2615801992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3405e0f700 1 --2- 172.21.15.46:0/2615801992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b75c2a0 0x55ea3b75e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3404e0d700 1 -- 172.21.15.46:0/2615801992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b750ae0 msgr2=0x55ea3b750ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3404e0d700 1 --2- 172.21.15.46:0/2615801992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b750ae0 0x55ea3b750ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3404e0d700 1 -- 172.21.15.46:0/2615801992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b75c2a0 msgr2=0x55ea3b75e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.783+0000 7f3404e0d700 1 --2- 172.21.15.46:0/2615801992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b75c2a0 0x55ea3b75e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.784+0000 7f3404e0d700 1 -- 172.21.15.46:0/2615801992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea3b525430 con 0x55ea3b751400 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.784+0000 7f3405e0f700 1 --2- 172.21.15.46:0/2615801992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b75c2a0 0x55ea3b75e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.784+0000 7f3404e0d700 1 --2- 172.21.15.46:0/2615801992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 0x55ea3b75bd60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f33f0015000 tx=0x7f33f0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1282794c2154d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.784+0000 7f33eb7fe700 1 -- 172.21.15.46:0/2615801992 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f00056d0 con 0x55ea3b751400 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.784+0000 7f33eb7fe700 1 -- 172.21.15.46:0/2615801992 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33f001e040 con 0x55ea3b751400 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.784+0000 7f33eb7fe700 1 -- 172.21.15.46:0/2615801992 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f00049c0 con 0x55ea3b751400 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.785+0000 7f3413a07b80 1 -- 172.21.15.46:0/2615801992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 msgr2=0x55ea3b75bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.785+0000 7f3413a07b80 1 --2- 172.21.15.46:0/2615801992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 0x55ea3b75bd60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f33f0015000 tx=0x7f33f0005ef0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.785+0000 7f3413a07b80 1 -- 172.21.15.46:0/2615801992 shutdown_connections 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.785+0000 7f3413a07b80 1 --2- 172.21.15.46:0/2615801992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b75c2a0 0x55ea3b75e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.785+0000 7f3413a07b80 1 --2- 172.21.15.46:0/2615801992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b750ae0 0x55ea3b750ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.785+0000 7f3413a07b80 1 --2- 172.21.15.46:0/2615801992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 0x55ea3b75bd60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.785+0000 7f3413a07b80 1 -- 172.21.15.46:0/2615801992 >> 172.21.15.46:0/2615801992 conn(0x55ea3b6c72a0 msgr2=0x55ea3b74d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.785+0000 7f3413a07b80 1 -- 172.21.15.46:0/2615801992 shutdown_connections 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.785+0000 7f3413a07b80 1 -- 172.21.15.46:0/2615801992 wait complete. 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.785+0000 7f3413a07b80 1 -- start start 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.785+0000 7f3413a07b80 1 Processor -- start 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.786+0000 7f3413a07b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 0x55ea3b75bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.786+0000 7f3413a07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b751400 0x55ea3b759030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.786+0000 7f3413a07b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b75c2a0 0x55ea3b759770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.786+0000 7f3413a07b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3b63fe00 con 0x55ea3b750ae0 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.786+0000 7f3413a07b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3b63faa0 con 0x55ea3b75c2a0 2024-03-18T20:37:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.786+0000 7f3413a07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3b527720 con 0x55ea3b751400 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.786+0000 7f3413a07b80 10 client.0 _ll_register_callbacks cb 0x55ea3b4ab420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.786+0000 7f340560e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 0x55ea3b75bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.786+0000 7f340560e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 0x55ea3b75bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60790/0 (socket says 172.21.15.46:60790) 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.786+0000 7f340560e700 1 -- 172.21.15.46:0/1448117960 learned_addr learned my addr 172.21.15.46:0/1448117960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.786+0000 7f3404e0d700 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b751400 0x55ea3b759030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.786+0000 7f3405e0f700 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b75c2a0 0x55ea3b759770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f340560e700 1 -- 172.21.15.46:0/1448117960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b751400 msgr2=0x55ea3b759030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f340560e700 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b751400 0x55ea3b759030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f340560e700 1 -- 172.21.15.46:0/1448117960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b75c2a0 msgr2=0x55ea3b759770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f340560e700 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b75c2a0 0x55ea3b759770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f340560e700 1 -- 172.21.15.46:0/1448117960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340010d380 con 0x55ea3b750ae0 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f340560e700 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 0x55ea3b75bd40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f33fc004430 tx=0x7f33fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4226f7d1113ccf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f33e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f33e8ff9700 1 -- 172.21.15.46:0/1448117960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc003c90 con 0x55ea3b750ae0 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f33e8ff9700 1 -- 172.21.15.46:0/1448117960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f33fc018070 con 0x55ea3b750ae0 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f3413a07b80 1 -- 172.21.15.46:0/1448117960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea3b744b50 con 0x55ea3b750ae0 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f33e8ff9700 1 -- 172.21.15.46:0/1448117960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33fc003df0 con 0x55ea3b750ae0 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f3413a07b80 10 client.15912 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.787+0000 7f3413a07b80 1 -- 172.21.15.46:0/1448117960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ea3b4d1be0 con 0x55ea3b750ae0 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.788+0000 7f33e8ff9700 1 -- 172.21.15.46:0/1448117960 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea3b744b50 con 0x55ea3b750ae0 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.788+0000 7f33e8ff9700 1 -- 172.21.15.46:0/1448117960 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f33fc0204c0 con 0x55ea3b750ae0 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.788+0000 7f3413a07b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.788+0000 7f3413a07b80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.788+0000 7f3413a07b80 20 client.15912 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.788+0000 7f3413a07b80 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ea3b76bf70 0x55ea3b76e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.788+0000 7f3413a07b80 4 client.15912 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.788+0000 7f3413a07b80 1 -- 172.21.15.46:0/1448117960 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea3b69c780 con 0x55ea3b76bf70 2024-03-18T20:37:04.798 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-18T20:37:04.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.788+0000 7f3404e0d700 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ea3b76bf70 0x55ea3b76e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.789+0000 7f3404e0d700 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ea3b76bf70 0x55ea3b76e390 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e8473ae597c7f5e server_cookie=978dc321d9d734d4 in_seq=0 out_seq=0 2024-03-18T20:37:04.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.789+0000 7f33e8ff9700 10 client.15912 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:04.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.789+0000 7f33e8ff9700 1 -- 172.21.15.46:0/1448117960 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ea3b69c780 con 0x55ea3b76bf70 2024-03-18T20:37:04.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.789+0000 7f33e8ff9700 10 client.15912 handle_command_reply: tid=0 2024-03-18T20:37:04.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.789+0000 7f3413a07b80 1 client.15912 shutdown 2024-03-18T20:37:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.790+0000 7f3413a07b80 1 -- 172.21.15.46:0/1448117960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 msgr2=0x55ea3b75bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.790+0000 7f3413a07b80 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 0x55ea3b75bd40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f33fc004430 tx=0x7f33fc004750 comp rx=0 tx=0).stop 2024-03-18T20:37:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.790+0000 7f3413a07b80 1 -- 172.21.15.46:0/1448117960 shutdown_connections 2024-03-18T20:37:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.790+0000 7f3413a07b80 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ea3b76bf70 0x55ea3b76e390 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.790+0000 7f3413a07b80 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 0x55ea3b75bd40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.790+0000 7f3413a07b80 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b75c2a0 0x55ea3b759770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.790+0000 7f3413a07b80 1 --2- 172.21.15.46:0/1448117960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b751400 0x55ea3b759030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.790+0000 7f33e8ff9700 0 client.15912 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.790+0000 7f3413a07b80 1 -- 172.21.15.46:0/1448117960 >> 172.21.15.46:0/1448117960 conn(0x55ea3b6c72a0 msgr2=0x55ea3b74cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.790+0000 7f3413a07b80 1 -- 172.21.15.46:0/1448117960 shutdown_connections 2024-03-18T20:37:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.790+0000 7f3413a07b80 1 -- 172.21.15.46:0/1448117960 wait complete. 2024-03-18T20:37:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.790+0000 7f3413a07b80 20 client.15912 trim_cache size 0 max 16384 2024-03-18T20:37:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f3413a07b80 1 Processor -- start 2024-03-18T20:37:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f3413a07b80 1 -- start start 2024-03-18T20:37:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f3413a07b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 0x55ea3b762e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f3413a07b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b751400 0x55ea3b74cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f3413a07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b75c2a0 0x55ea3b754300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f3413a07b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3b751db0 con 0x55ea3b750ae0 2024-03-18T20:37:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f3413a07b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3b749660 con 0x55ea3b751400 2024-03-18T20:37:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f3413a07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3b7525d0 con 0x55ea3b75c2a0 2024-03-18T20:37:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f340560e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 0x55ea3b762e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f340560e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 0x55ea3b762e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60800/0 (socket says 172.21.15.46:60800) 2024-03-18T20:37:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f340560e700 1 -- 172.21.15.46:0/496076456 learned_addr learned my addr 172.21.15.46:0/496076456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f3404e0d700 1 --2- 172.21.15.46:0/496076456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b751400 0x55ea3b74cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.811+0000 7f3405e0f700 1 --2- 172.21.15.46:0/496076456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b75c2a0 0x55ea3b754300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.812+0000 7f340560e700 1 -- 172.21.15.46:0/496076456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b75c2a0 msgr2=0x55ea3b754300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.812+0000 7f340560e700 1 --2- 172.21.15.46:0/496076456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b75c2a0 0x55ea3b754300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.812+0000 7f340560e700 1 -- 172.21.15.46:0/496076456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b751400 msgr2=0x55ea3b74cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:04.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.812+0000 7f340560e700 1 --2- 172.21.15.46:0/496076456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b751400 0x55ea3b74cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.812+0000 7f340560e700 1 -- 172.21.15.46:0/496076456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea3b772f80 con 0x55ea3b750ae0 2024-03-18T20:37:04.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.812+0000 7f3404e0d700 1 --2- 172.21.15.46:0/496076456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b751400 0x55ea3b74cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:04.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.812+0000 7f340560e700 1 --2- 172.21.15.46:0/496076456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 0x55ea3b762e70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f33fc000c00 tx=0x7f33fc0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfa5c747db0c3bd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:04.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.812+0000 7f33ebfff700 1 -- 172.21.15.46:0/496076456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc020430 con 0x55ea3b750ae0 2024-03-18T20:37:04.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.812+0000 7f33ebfff700 1 -- 172.21.15.46:0/496076456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33fc003e20 con 0x55ea3b750ae0 2024-03-18T20:37:04.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.812+0000 7f33ebfff700 1 -- 172.21.15.46:0/496076456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc012c40 con 0x55ea3b750ae0 2024-03-18T20:37:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.813+0000 7f3413a07b80 1 -- 172.21.15.46:0/496076456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 msgr2=0x55ea3b762e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.813+0000 7f3413a07b80 1 --2- 172.21.15.46:0/496076456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 0x55ea3b762e70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f33fc000c00 tx=0x7f33fc0093d0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.813+0000 7f3413a07b80 1 -- 172.21.15.46:0/496076456 shutdown_connections 2024-03-18T20:37:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.813+0000 7f3413a07b80 1 --2- 172.21.15.46:0/496076456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b750ae0 0x55ea3b762e70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.813+0000 7f3413a07b80 1 --2- 172.21.15.46:0/496076456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b75c2a0 0x55ea3b754300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.813+0000 7f3413a07b80 1 --2- 172.21.15.46:0/496076456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b751400 0x55ea3b74cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.813+0000 7f3413a07b80 1 -- 172.21.15.46:0/496076456 >> 172.21.15.46:0/496076456 conn(0x55ea3b6c72a0 msgr2=0x55ea3b7675c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.813+0000 7f3413a07b80 1 -- 172.21.15.46:0/496076456 shutdown_connections 2024-03-18T20:37:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.813+0000 7f3413a07b80 1 -- 172.21.15.46:0/496076456 wait complete. 2024-03-18T20:37:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.813+0000 7f3413a07b80 1 -- start start 2024-03-18T20:37:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.813+0000 7f3413a07b80 1 Processor -- start 2024-03-18T20:37:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.814+0000 7f3413a07b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b750ae0 0x55ea3b761a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.814+0000 7f3413a07b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 0x55ea3b761f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.814+0000 7f3413a07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b75c2a0 0x55ea3b7bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.814+0000 7f3413a07b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3b762890 con 0x55ea3b751400 2024-03-18T20:37:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.814+0000 7f3413a07b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3b7c02b0 con 0x55ea3b750ae0 2024-03-18T20:37:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.814+0000 7f3413a07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3b7c03f0 con 0x55ea3b75c2a0 2024-03-18T20:37:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.814+0000 7f3413a07b80 10 client.0 _ll_register_callbacks cb 0x7f33fc004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.814+0000 7f3404e0d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 0x55ea3b761f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.814+0000 7f3404e0d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 0x55ea3b761f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60816/0 (socket says 172.21.15.46:60816) 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.814+0000 7f3404e0d700 1 -- 172.21.15.46:0/3845661572 learned_addr learned my addr 172.21.15.46:0/3845661572 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.814+0000 7f340560e700 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b750ae0 0x55ea3b761a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.814+0000 7f3405e0f700 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b75c2a0 0x55ea3b7bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f3404e0d700 1 -- 172.21.15.46:0/3845661572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b75c2a0 msgr2=0x55ea3b7bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f3404e0d700 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b75c2a0 0x55ea3b7bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f3404e0d700 1 -- 172.21.15.46:0/3845661572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b750ae0 msgr2=0x55ea3b761a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f3404e0d700 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b750ae0 0x55ea3b761a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f3404e0d700 1 -- 172.21.15.46:0/3845661572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea3b4d1be0 con 0x55ea3b751400 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f3405e0f700 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b75c2a0 0x55ea3b7bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f3404e0d700 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 0x55ea3b761f80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005320 tx=0x7f33f0005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cb2678556a28fc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f33e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f33e8ff9700 1 -- 172.21.15.46:0/3845661572 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f00300b0 con 0x55ea3b751400 2024-03-18T20:37:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f33e8ff9700 1 -- 172.21.15.46:0/3845661572 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f33f0030210 con 0x55ea3b751400 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f3413a07b80 1 -- 172.21.15.46:0/3845661572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33f001e040 con 0x55ea3b751400 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f33e8ff9700 1 -- 172.21.15.46:0/3845661572 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33f00303d0 con 0x55ea3b751400 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f3413a07b80 10 client.15918 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f3413a07b80 1 -- 172.21.15.46:0/3845661572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ea3b669f20 con 0x55ea3b751400 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.815+0000 7f33e8ff9700 1 -- 172.21.15.46:0/3845661572 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33f001e040 con 0x55ea3b751400 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.816+0000 7f33e8ff9700 1 -- 172.21.15.46:0/3845661572 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f33f003f020 con 0x55ea3b751400 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.816+0000 7f3413a07b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.816+0000 7f3413a07b80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.816+0000 7f3413a07b80 20 client.15918 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.816+0000 7f3413a07b80 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ea3b7c16d0 0x55ea3b7c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.816+0000 7f3413a07b80 4 client.15918 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.816+0000 7f3413a07b80 1 -- 172.21.15.46:0/3845661572 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ea3b4f2d70 con 0x55ea3b7c16d0 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.816+0000 7f340560e700 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ea3b7c16d0 0x55ea3b7c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.816+0000 7f33e8ff9700 10 client.15918 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.816+0000 7f340560e700 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ea3b7c16d0 0x55ea3b7c3ac0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43859a04626fdd03 server_cookie=7899e8771e741cff in_seq=0 out_seq=0 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f33e8ff9700 1 -- 172.21.15.46:0/3845661572 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ea3b4f2d70 con 0x55ea3b7c16d0 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f33e8ff9700 10 client.15918 handle_command_reply: tid=0 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f3413a07b80 1 client.15918 shutdown 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f3413a07b80 1 -- 172.21.15.46:0/3845661572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 msgr2=0x55ea3b761f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f3413a07b80 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 0x55ea3b761f80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005320 tx=0x7f33f0005510 comp rx=0 tx=0).stop 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f3413a07b80 1 -- 172.21.15.46:0/3845661572 shutdown_connections 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f3413a07b80 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ea3b7c16d0 0x55ea3b7c3ac0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f3413a07b80 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ea3b75c2a0 0x55ea3b7bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f3413a07b80 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea3b750ae0 0x55ea3b761a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f33e8ff9700 0 client.15918 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f3413a07b80 1 --2- 172.21.15.46:0/3845661572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea3b751400 0x55ea3b761f80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f3413a07b80 1 -- 172.21.15.46:0/3845661572 >> 172.21.15.46:0/3845661572 conn(0x55ea3b6c72a0 msgr2=0x55ea3b74cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f3413a07b80 1 -- 172.21.15.46:0/3845661572 shutdown_connections 2024-03-18T20:37:04.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f3413a07b80 1 -- 172.21.15.46:0/3845661572 wait complete. 2024-03-18T20:37:04.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.818+0000 7f3413a07b80 20 client.15918 trim_cache size 0 max 16384 2024-03-18T20:37:04.827 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:37:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:37:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "00632aa3-e83d-4808-a346-2663c4aa562e", 2024-03-18T20:37:04.827 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:37:04.827 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:37:04.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.819+0000 7f3407872700 1 -- 172.21.15.46:0/2429590660 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f33ec0a13b0 msgr2=0x7f33ec0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:04.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.819+0000 7f3407872700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f33ec0a13b0 0x7f33ec0a3870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009cb0 tx=0x7f33fc009260 comp rx=0 tx=0).stop 2024-03-18T20:37:04.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.819+0000 7f3407872700 1 -- 172.21.15.46:0/2429590660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f340019dce0 msgr2=0x7f340019e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.819+0000 7f3407872700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f340019dce0 0x7f340019e160 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f33f800ef30 tx=0x7f33f800c450 comp rx=0 tx=0).stop 2024-03-18T20:37:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.819+0000 7f3407872700 1 -- 172.21.15.46:0/2429590660 shutdown_connections 2024-03-18T20:37:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.819+0000 7f3407872700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f33ec0a13b0 0x7f33ec0a3870 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.819+0000 7f3407872700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f340019dce0 0x7f340019e160 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.819+0000 7f3407872700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34000ffe70 0x7f3400199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.819+0000 7f3407872700 1 --2- 172.21.15.46:0/2429590660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3400104f90 0x7f3400199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.819+0000 7f3407872700 1 -- 172.21.15.46:0/2429590660 >> 172.21.15.46:0/2429590660 conn(0x7f3400076d80 msgr2=0x7f34000ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.819+0000 7f3407872700 1 -- 172.21.15.46:0/2429590660 shutdown_connections 2024-03-18T20:37:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:04.819+0000 7f3407872700 1 -- 172.21.15.46:0/2429590660 wait complete. 2024-03-18T20:37:04.831 INFO:tasks.workunit.client.0.smithi046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-18T20:37:04.834 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-18T20:37:04.834 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-18T20:37:04.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:37:04.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:37:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.016+0000 7fac51e8c700 1 -- 172.21.15.46:0/2644526249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0ffe50 msgr2=0x7fac4c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.016+0000 7fac51e8c700 1 --2- 172.21.15.46:0/2644526249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0ffe50 0x7fac4c100230 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fac40009990 tx=0x7fac40009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.016+0000 7fac51e8c700 1 -- 172.21.15.46:0/2644526249 shutdown_connections 2024-03-18T20:37:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.016+0000 7fac51e8c700 1 --2- 172.21.15.46:0/2644526249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0ffe50 0x7fac4c100230 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.016+0000 7fac51e8c700 1 --2- 172.21.15.46:0/2644526249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac4c104f60 0x7fac4c10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.016+0000 7fac51e8c700 1 --2- 172.21.15.46:0/2644526249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac4c100800 0x7fac4c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.016+0000 7fac51e8c700 1 -- 172.21.15.46:0/2644526249 >> 172.21.15.46:0/2644526249 conn(0x7fac4c076d40 msgr2=0x7fac4c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.016+0000 7fac51e8c700 1 -- 172.21.15.46:0/2644526249 shutdown_connections 2024-03-18T20:37:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.016+0000 7fac51e8c700 1 -- 172.21.15.46:0/2644526249 wait complete. 2024-03-18T20:37:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.017+0000 7fac51e8c700 1 Processor -- start 2024-03-18T20:37:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.017+0000 7fac51e8c700 1 -- start start 2024-03-18T20:37:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.017+0000 7fac51e8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0ffe50 0x7fac4c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.017+0000 7fac51e8c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac4c104f60 0x7fac4c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac51e8c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac4c19bac0 0x7fac4c19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac51e8c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c19c540 con 0x7fac4c0ffe50 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac51e8c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c19c6b0 con 0x7fac4c19bac0 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac51e8c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c19c870 con 0x7fac4c104f60 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac4b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0ffe50 0x7fac4c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac4b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0ffe50 0x7fac4c1971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60846/0 (socket says 172.21.15.46:60846) 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac4b7fe700 1 -- 172.21.15.46:0/1922504108 learned_addr learned my addr 172.21.15.46:0/1922504108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac4affd700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac4c104f60 0x7fac4c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac4bfff700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac4c19bac0 0x7fac4c19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac4b7fe700 1 -- 172.21.15.46:0/1922504108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac4c104f60 msgr2=0x7fac4c197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac4c104f60 0x7fac4c197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac4b7fe700 1 -- 172.21.15.46:0/1922504108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac4c19bac0 msgr2=0x7fac4c19bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac4c19bac0 0x7fac4c19bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac4b7fe700 1 -- 172.21.15.46:0/1922504108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac400095f0 con 0x7fac4c0ffe50 2024-03-18T20:37:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac4bfff700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac4c19bac0 0x7fac4c19bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0ffe50 0x7fac4c1971e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fac40005770 tx=0x7fac40009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac48ff9700 1 -- 172.21.15.46:0/1922504108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4001d070 con 0x7fac4c0ffe50 2024-03-18T20:37:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac51e8c700 1 -- 172.21.15.46:0/1922504108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c1a3d70 con 0x7fac4c0ffe50 2024-03-18T20:37:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac48ff9700 1 -- 172.21.15.46:0/1922504108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac4000fbf0 con 0x7fac4c0ffe50 2024-03-18T20:37:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac48ff9700 1 -- 172.21.15.46:0/1922504108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40022bf0 con 0x7fac4c0ffe50 2024-03-18T20:37:05.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.018+0000 7fac51e8c700 1 -- 172.21.15.46:0/1922504108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c1a4190 con 0x7fac4c0ffe50 2024-03-18T20:37:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.021+0000 7fac48ff9700 1 -- 172.21.15.46:0/1922504108 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fac40022d50 con 0x7fac4c0ffe50 2024-03-18T20:37:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.021+0000 7fac48ff9700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fac340a13b0 0x7fac340a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.021+0000 7fac48ff9700 1 -- 172.21.15.46:0/1922504108 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fac400ce120 con 0x7fac4c0ffe50 2024-03-18T20:37:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.021+0000 7fac4affd700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fac340a13b0 0x7fac340a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.022+0000 7fac4affd700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fac340a13b0 0x7fac340a3870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac4c198730 tx=0x7fac3c00b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:05.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.022+0000 7fac5e021b80 1 Processor -- start 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.022+0000 7fac5e021b80 1 -- start start 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.022+0000 7fac5e021b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 0x55e882373e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac5e021b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e882374380 0x55e88237ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac5e021b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88237f220 0x55e882381610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac5e021b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8822cd990 con 0x55e882373a60 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac5e021b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8822cb350 con 0x55e882374380 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac5e021b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8822c6c40 con 0x55e88237f220 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac4b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 0x55e882373e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac4b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 0x55e882373e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60850/0 (socket says 172.21.15.46:60850) 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac4b7fe700 1 -- 172.21.15.46:0/2735542832 learned_addr learned my addr 172.21.15.46:0/2735542832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac4affd700 1 --2- 172.21.15.46:0/2735542832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e882374380 0x55e88237ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac4bfff700 1 --2- 172.21.15.46:0/2735542832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88237f220 0x55e882381610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac4b7fe700 1 -- 172.21.15.46:0/2735542832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88237f220 msgr2=0x55e882381610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/2735542832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88237f220 0x55e882381610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac4b7fe700 1 -- 172.21.15.46:0/2735542832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e882374380 msgr2=0x55e88237ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/2735542832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e882374380 0x55e88237ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac4b7fe700 1 -- 172.21.15.46:0/2735542832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8821483b0 con 0x55e882373a60 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac4bfff700 1 --2- 172.21.15.46:0/2735542832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88237f220 0x55e882381610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/2735542832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 0x55e882373e40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fac4009ad80 tx=0x7fac400226c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b379bd492d28f8f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac31ffb700 1 -- 172.21.15.46:0/2735542832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4009b080 con 0x55e882373a60 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac31ffb700 1 -- 172.21.15.46:0/2735542832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac4009eeb0 con 0x55e882373a60 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.023+0000 7fac31ffb700 1 -- 172.21.15.46:0/2735542832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4009a570 con 0x55e882373a60 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.024+0000 7fac5e021b80 1 -- 172.21.15.46:0/2735542832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 msgr2=0x55e882373e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.024+0000 7fac5e021b80 1 --2- 172.21.15.46:0/2735542832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 0x55e882373e40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fac4009ad80 tx=0x7fac400226c0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.024+0000 7fac5e021b80 1 -- 172.21.15.46:0/2735542832 shutdown_connections 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.024+0000 7fac5e021b80 1 --2- 172.21.15.46:0/2735542832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 0x55e882373e40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.024+0000 7fac5e021b80 1 --2- 172.21.15.46:0/2735542832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e88237f220 0x55e882381610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.024+0000 7fac5e021b80 1 --2- 172.21.15.46:0/2735542832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e882374380 0x55e88237ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.024+0000 7fac5e021b80 1 -- 172.21.15.46:0/2735542832 >> 172.21.15.46:0/2735542832 conn(0x55e8822ea220 msgr2=0x55e882370020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.024+0000 7fac5e021b80 1 -- 172.21.15.46:0/2735542832 shutdown_connections 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.024+0000 7fac5e021b80 1 -- 172.21.15.46:0/2735542832 wait complete. 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.025+0000 7fac5e021b80 1 -- start start 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.025+0000 7fac5e021b80 1 Processor -- start 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.025+0000 7fac5e021b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e882373a60 0x55e88237f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.025+0000 7fac5e021b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882374380 0x55e88237fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.025+0000 7fac5e021b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e88237f220 0x55e88238a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.025+0000 7fac5e021b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e882262d80 con 0x55e88237f220 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.025+0000 7fac5e021b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e882262a20 con 0x55e882373a60 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.025+0000 7fac5e021b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e88214a6a0 con 0x55e882374380 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.025+0000 7fac5e021b80 10 client.0 _ll_register_callbacks cb 0x55e8820ce3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.025+0000 7fac4bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e88237f220 0x55e88238a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.025+0000 7fac4bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e88237f220 0x55e88238a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60854/0 (socket says 172.21.15.46:60854) 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.025+0000 7fac4bfff700 1 -- 172.21.15.46:0/1374421976 learned_addr learned my addr 172.21.15.46:0/1374421976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e882373a60 0x55e88237f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac4affd700 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882374380 0x55e88237fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac4bfff700 1 -- 172.21.15.46:0/1374421976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882374380 msgr2=0x55e88237fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac4bfff700 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882374380 0x55e88237fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac4bfff700 1 -- 172.21.15.46:0/1374421976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e882373a60 msgr2=0x55e88237f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac4bfff700 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e882373a60 0x55e88237f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac4bfff700 1 -- 172.21.15.46:0/1374421976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c1a4190 con 0x55e88237f220 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac4bfff700 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e88237f220 0x55e88238a680 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fac44007b70 tx=0x7fac44007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1046978ad3f44187 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac2affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac2affd700 1 -- 172.21.15.46:0/1374421976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac440118d0 con 0x55e88237f220 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac2affd700 1 -- 172.21.15.46:0/1374421976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fac44011a30 con 0x55e88237f220 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac5e021b80 1 -- 172.21.15.46:0/1374421976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e882367ad0 con 0x55e88237f220 2024-03-18T20:37:05.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac2affd700 1 -- 172.21.15.46:0/1374421976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac44011bf0 con 0x55e88237f220 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac5e021b80 10 client.15930 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.026+0000 7fac5e021b80 1 -- 172.21.15.46:0/1374421976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e8820f4b60 con 0x55e88237f220 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.027+0000 7fac2affd700 1 -- 172.21.15.46:0/1374421976 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e882367ad0 con 0x55e88237f220 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.027+0000 7fac2affd700 1 -- 172.21.15.46:0/1374421976 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fac44014070 con 0x55e88237f220 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.027+0000 7fac5e021b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.027+0000 7fac5e021b80 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.027+0000 7fac5e021b80 20 client.15930 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.027+0000 7fac5e021b80 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e88238eb50 0x55e882390f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.027+0000 7fac5e021b80 4 client.15930 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.027+0000 7fac5e021b80 1 -- 172.21.15.46:0/1374421976 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8822bf700 con 0x55e88238eb50 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.027+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e88238eb50 0x55e882390f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.027+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e88238eb50 0x55e882390f70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=242b9944c40903b3 server_cookie=ecfe84a4bf764a48 in_seq=0 out_seq=0 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.027+0000 7fac2affd700 10 client.15930 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.028+0000 7fac2affd700 1 -- 172.21.15.46:0/1374421976 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e8822bf700 con 0x55e88238eb50 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.028+0000 7fac2affd700 10 client.15930 handle_command_reply: tid=0 2024-03-18T20:37:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.028+0000 7fac5e021b80 1 client.15930 shutdown 2024-03-18T20:37:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.028+0000 7fac5e021b80 1 -- 172.21.15.46:0/1374421976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e88237f220 msgr2=0x55e88238a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.028+0000 7fac5e021b80 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e88237f220 0x55e88238a680 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fac44007b70 tx=0x7fac44007e80 comp rx=0 tx=0).stop 2024-03-18T20:37:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.028+0000 7fac5e021b80 1 -- 172.21.15.46:0/1374421976 shutdown_connections 2024-03-18T20:37:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.028+0000 7fac5e021b80 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e88238eb50 0x55e882390f70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.028+0000 7fac5e021b80 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e88237f220 0x55e88238a680 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.028+0000 7fac5e021b80 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e882373a60 0x55e88237f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.028+0000 7fac2affd700 0 client.15930 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.028+0000 7fac5e021b80 1 --2- 172.21.15.46:0/1374421976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882374380 0x55e88237fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.028+0000 7fac5e021b80 1 -- 172.21.15.46:0/1374421976 >> 172.21.15.46:0/1374421976 conn(0x55e8822ea220 msgr2=0x55e88236fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.029+0000 7fac5e021b80 1 -- 172.21.15.46:0/1374421976 shutdown_connections 2024-03-18T20:37:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.029+0000 7fac5e021b80 1 -- 172.21.15.46:0/1374421976 wait complete. 2024-03-18T20:37:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.029+0000 7fac5e021b80 20 client.15930 trim_cache size 0 max 16384 2024-03-18T20:37:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.046+0000 7fac5e021b80 1 Processor -- start 2024-03-18T20:37:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.046+0000 7fac5e021b80 1 -- start start 2024-03-18T20:37:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.046+0000 7fac5e021b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882373a60 0x55e882384640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.046+0000 7fac5e021b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882374380 0x55e882378b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.046+0000 7fac5e021b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e88237f220 0x55e88237d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac5e021b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e88238c180 con 0x55e882374380 2024-03-18T20:37:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac5e021b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e882373e40 con 0x55e88237f220 2024-03-18T20:37:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac5e021b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e882384b80 con 0x55e882373a60 2024-03-18T20:37:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac4affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882374380 0x55e882378b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac4affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882374380 0x55e882378b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60858/0 (socket says 172.21.15.46:60858) 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac4affd700 1 -- 172.21.15.46:0/540755516 learned_addr learned my addr 172.21.15.46:0/540755516 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/540755516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882373a60 0x55e882384640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac4bfff700 1 --2- 172.21.15.46:0/540755516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e88237f220 0x55e88237d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac4affd700 1 -- 172.21.15.46:0/540755516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882373a60 msgr2=0x55e882384640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac4affd700 1 --2- 172.21.15.46:0/540755516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882373a60 0x55e882384640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac4affd700 1 -- 172.21.15.46:0/540755516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e88237f220 msgr2=0x55e88237d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac4affd700 1 --2- 172.21.15.46:0/540755516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e88237f220 0x55e88237d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac4affd700 1 -- 172.21.15.46:0/540755516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e88228cea0 con 0x55e882374380 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac4bfff700 1 --2- 172.21.15.46:0/540755516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e88237f220 0x55e88237d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac4affd700 1 --2- 172.21.15.46:0/540755516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882374380 0x55e882378b90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fac3c005950 tx=0x7fac3c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b70d8488d602e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac327fc700 1 -- 172.21.15.46:0/540755516 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c024070 con 0x55e882374380 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac327fc700 1 -- 172.21.15.46:0/540755516 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac3c015070 con 0x55e882374380 2024-03-18T20:37:05.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.047+0000 7fac327fc700 1 -- 172.21.15.46:0/540755516 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c005240 con 0x55e882374380 2024-03-18T20:37:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.048+0000 7fac5e021b80 1 -- 172.21.15.46:0/540755516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882374380 msgr2=0x55e882378b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.048+0000 7fac5e021b80 1 --2- 172.21.15.46:0/540755516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882374380 0x55e882378b90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fac3c005950 tx=0x7fac3c013000 comp rx=0 tx=0).stop 2024-03-18T20:37:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.048+0000 7fac5e021b80 1 -- 172.21.15.46:0/540755516 shutdown_connections 2024-03-18T20:37:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.048+0000 7fac5e021b80 1 --2- 172.21.15.46:0/540755516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e88237f220 0x55e88237d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.048+0000 7fac5e021b80 1 --2- 172.21.15.46:0/540755516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882373a60 0x55e882384640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.048+0000 7fac5e021b80 1 --2- 172.21.15.46:0/540755516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882374380 0x55e882378b90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.048+0000 7fac5e021b80 1 -- 172.21.15.46:0/540755516 >> 172.21.15.46:0/540755516 conn(0x55e8822ea220 msgr2=0x55e88237f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.048+0000 7fac5e021b80 1 -- 172.21.15.46:0/540755516 shutdown_connections 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.048+0000 7fac5e021b80 1 -- 172.21.15.46:0/540755516 wait complete. 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.048+0000 7fac5e021b80 1 -- start start 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac5e021b80 1 Processor -- start 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac5e021b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 0x55e8823daed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac5e021b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882374380 0x55e8823db410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac5e021b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e88237f220 0x55e8823d68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac5e021b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8823d6ea0 con 0x55e882373a60 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac5e021b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8823d6fe0 con 0x55e88237f220 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac5e021b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8823d71a0 con 0x55e882374380 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac5e021b80 10 client.0 _ll_register_callbacks cb 0x7fac44009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac4b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 0x55e8823daed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac4b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 0x55e8823daed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60862/0 (socket says 172.21.15.46:60862) 2024-03-18T20:37:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac4b7fe700 1 -- 172.21.15.46:0/2829195762 learned_addr learned my addr 172.21.15.46:0/2829195762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac4affd700 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882374380 0x55e8823db410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.049+0000 7fac4bfff700 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e88237f220 0x55e8823d68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac4b7fe700 1 -- 172.21.15.46:0/2829195762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882374380 msgr2=0x55e8823db410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882374380 0x55e8823db410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac4b7fe700 1 -- 172.21.15.46:0/2829195762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e88237f220 msgr2=0x55e8823d68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e88237f220 0x55e8823d68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac4b7fe700 1 -- 172.21.15.46:0/2829195762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3c010e10 con 0x55e882373a60 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac4bfff700 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e88237f220 0x55e8823d68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac4b7fe700 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 0x55e8823daed0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fac4009b010 tx=0x7fac40009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2166aa39345f13 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac2affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac2affd700 1 -- 172.21.15.46:0/2829195762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac400973c0 con 0x55e882373a60 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac2affd700 1 -- 172.21.15.46:0/2829195762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fac4009b080 con 0x55e882373a60 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac5e021b80 1 -- 172.21.15.46:0/2829195762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac4009eeb0 con 0x55e882373a60 2024-03-18T20:37:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac2affd700 1 -- 172.21.15.46:0/2829195762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac400a3470 con 0x55e882373a60 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac5e021b80 10 client.15936 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac5e021b80 1 -- 172.21.15.46:0/2829195762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e88214ccd0 con 0x55e882373a60 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac2affd700 1 -- 172.21.15.46:0/2829195762 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac4009eeb0 con 0x55e882373a60 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac2affd700 1 -- 172.21.15.46:0/2829195762 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fac4007d070 con 0x55e882373a60 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac5e021b80 10 client.15936 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac5e021b80 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac5e021b80 20 client.15936 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac5e021b80 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8823e39b0 0x55e8823e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac5e021b80 4 client.15936 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.050+0000 7fac5e021b80 1 -- 172.21.15.46:0/2829195762 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e882115cf0 con 0x55e8823e39b0 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.051+0000 7fac4affd700 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8823e39b0 0x55e8823e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.051+0000 7fac4affd700 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8823e39b0 0x55e8823e5da0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d40f97de473c2bd4 server_cookie=ed79a37ce3d5fedc in_seq=0 out_seq=0 2024-03-18T20:37:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.051+0000 7fac2affd700 10 client.15936 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.051+0000 7fac2affd700 1 -- 172.21.15.46:0/2829195762 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e882115cf0 con 0x55e8823e39b0 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.051+0000 7fac2affd700 10 client.15936 handle_command_reply: tid=0 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.051+0000 7fac5e021b80 1 client.15936 shutdown 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.051+0000 7fac5e021b80 1 -- 172.21.15.46:0/2829195762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 msgr2=0x55e8823daed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.051+0000 7fac5e021b80 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 0x55e8823daed0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fac4009b010 tx=0x7fac40009f60 comp rx=0 tx=0).stop 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac5e021b80 1 -- 172.21.15.46:0/2829195762 shutdown_connections 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac5e021b80 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8823e39b0 0x55e8823e5da0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac5e021b80 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e882373a60 0x55e8823daed0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac5e021b80 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e88237f220 0x55e8823d68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac2affd700 0 client.15936 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac5e021b80 1 --2- 172.21.15.46:0/2829195762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e882374380 0x55e8823db410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac5e021b80 1 -- 172.21.15.46:0/2829195762 >> 172.21.15.46:0/2829195762 conn(0x55e8822ea220 msgr2=0x55e88237d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac5e021b80 1 -- 172.21.15.46:0/2829195762 shutdown_connections 2024-03-18T20:37:05.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac5e021b80 1 -- 172.21.15.46:0/2829195762 wait complete. 2024-03-18T20:37:05.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac5e021b80 20 client.15936 trim_cache size 0 max 16384 2024-03-18T20:37:05.062 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:37:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:37:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:37:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "00632aa3-e83d-4808-a346-2663c4aa562e": { 2024-03-18T20:37:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:37:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "00632aa3-e83d-4808-a346-2663c4aa562e", 2024-03-18T20:37:05.062 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:37:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:37:05.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:37:05.063 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:37:05.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac51e8c700 1 -- 172.21.15.46:0/1922504108 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fac340a13b0 msgr2=0x7fac340a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac51e8c700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fac340a13b0 0x7fac340a3870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac4c198730 tx=0x7fac3c00b410 comp rx=0 tx=0).stop 2024-03-18T20:37:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac51e8c700 1 -- 172.21.15.46:0/1922504108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0ffe50 msgr2=0x7fac4c1971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac51e8c700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0ffe50 0x7fac4c1971e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fac40005770 tx=0x7fac40009580 comp rx=0 tx=0).stop 2024-03-18T20:37:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac51e8c700 1 -- 172.21.15.46:0/1922504108 shutdown_connections 2024-03-18T20:37:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac51e8c700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fac340a13b0 0x7fac340a3870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac51e8c700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c0ffe50 0x7fac4c1971e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac51e8c700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac4c19bac0 0x7fac4c19bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac51e8c700 1 --2- 172.21.15.46:0/1922504108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac4c104f60 0x7fac4c197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.052+0000 7fac51e8c700 1 -- 172.21.15.46:0/1922504108 >> 172.21.15.46:0/1922504108 conn(0x7fac4c076d40 msgr2=0x7fac4c103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.053+0000 7fac51e8c700 1 -- 172.21.15.46:0/1922504108 shutdown_connections 2024-03-18T20:37:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:05.053+0000 7fac51e8c700 1 -- 172.21.15.46:0/1922504108 wait complete. 2024-03-18T20:37:05.069 INFO:tasks.cephfs.filesystem:scrub status for tag:00632aa3-e83d-4808-a346-2663c4aa562e - {'path': '/', 'tag': '00632aa3-e83d-4808-a346-2663c4aa562e', 'options': 'recursive,force'} 2024-03-18T20:37:05.321 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-18T20:37:05.385 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-18T20:37:05.428 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-18T20:37:05.429 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-18T20:37:05.430 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-18T20:37:05.470 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-18T20:37:05.474 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T20:37:05.474 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-18T20:37:05.493 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-03-18T20:37:05.494 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-03-18T20:37:05.495 DEBUG:teuthology.orchestra.run.smithi046:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=49bdc51142759622a281d3e5327eebd07cbb8bb3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-03-18T20:37:05.566 INFO:tasks.workunit.client.0.smithi046.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-18T20:37:05.567 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-03-18T20:37:05.567 INFO:tasks.workunit.client.0.smithi046.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-18T20:37:05.567 INFO:tasks.workunit.client.0.smithi046.stdout:0 of 1 processes prepared for launch 0 sec 2024-03-18T20:37:05.667 INFO:tasks.workunit.client.0.smithi046.stdout:1 of 1 processes prepared for launch 0 sec 2024-03-18T20:37:05.668 INFO:tasks.workunit.client.0.smithi046.stdout:releasing clients 2024-03-18T20:37:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:05 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:37:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:05 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:05 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:37:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:05 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:05 smithi073 ceph-mon[47384]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-18T20:37:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:05 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:37:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:05 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:05 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:37:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:05 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:05 smithi155 ceph-mon[48344]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-18T20:37:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:05 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:37:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:05 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:05 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:37:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:05 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:05 smithi046 ceph-mon[43495]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-18T20:37:06.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 3768 39.76 MB/sec warmup 1 sec latency 9.734 ms 2024-03-18T20:37:07.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 7102 28.77 MB/sec warmup 2 sec latency 5.639 ms 2024-03-18T20:37:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:08 smithi073 ceph-mon[47384]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-18T20:37:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:08 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:37:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:08 smithi155 ceph-mon[48344]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-18T20:37:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:08 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:37:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:08 smithi046 ceph-mon[43495]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-18T20:37:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:08 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:37:08.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 10275 24.12 MB/sec warmup 3 sec latency 7.476 ms 2024-03-18T20:37:09.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 13259 21.59 MB/sec warmup 4 sec latency 7.765 ms 2024-03-18T20:37:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:10 smithi073 ceph-mon[47384]: pgmap v34: 97 pgs: 97 active+clean; 5.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 381 KiB/s wr, 74 op/s 2024-03-18T20:37:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:10 smithi155 ceph-mon[48344]: pgmap v34: 97 pgs: 97 active+clean; 5.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 381 KiB/s wr, 74 op/s 2024-03-18T20:37:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:10 smithi046 ceph-mon[43495]: pgmap v34: 97 pgs: 97 active+clean; 5.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 381 KiB/s wr, 74 op/s 2024-03-18T20:37:10.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 16174 20.59 MB/sec warmup 5 sec latency 16.308 ms 2024-03-18T20:37:11.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:11 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:37:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:11 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:37:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:11 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:37:11.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 19194 20.27 MB/sec warmup 6 sec latency 11.300 ms 2024-03-18T20:37:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:12 smithi073 ceph-mon[47384]: pgmap v35: 97 pgs: 97 active+clean; 5.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 381 KiB/s wr, 70 op/s 2024-03-18T20:37:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:12 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:37:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:12 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:37:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:12 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:37:12.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:12 smithi155 ceph-mon[48344]: pgmap v35: 97 pgs: 97 active+clean; 5.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 381 KiB/s wr, 70 op/s 2024-03-18T20:37:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:12 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:37:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:12 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:37:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:12 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:37:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:12 smithi046 ceph-mon[43495]: pgmap v35: 97 pgs: 97 active+clean; 5.0 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 381 KiB/s wr, 70 op/s 2024-03-18T20:37:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:12 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:37:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:12 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:37:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:12 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:37:12.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 22248 19.51 MB/sec warmup 7 sec latency 21.581 ms 2024-03-18T20:37:13.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 25328 19.36 MB/sec warmup 8 sec latency 9.350 ms 2024-03-18T20:37:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:14 smithi073 ceph-mon[47384]: pgmap v36: 97 pgs: 97 active+clean; 13 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-03-18T20:37:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:14 smithi155 ceph-mon[48344]: pgmap v36: 97 pgs: 97 active+clean; 13 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-03-18T20:37:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:14 smithi046 ceph-mon[43495]: pgmap v36: 97 pgs: 97 active+clean; 13 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.1 MiB/s wr, 169 op/s 2024-03-18T20:37:14.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 28656 19.30 MB/sec warmup 9 sec latency 7.127 ms 2024-03-18T20:37:15.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 31440 18.66 MB/sec warmup 10 sec latency 7.580 ms 2024-03-18T20:37:16.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.069+0000 7f3f39a35700 1 -- 172.21.15.46:0/1122604768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f34105c60 msgr2=0x7f3f34106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:16.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.069+0000 7f3f39a35700 1 --2- 172.21.15.46:0/1122604768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f34105c60 0x7f3f34106040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009990 tx=0x7f3f24009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.069+0000 7f3f39a35700 1 -- 172.21.15.46:0/1122604768 shutdown_connections 2024-03-18T20:37:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.069+0000 7f3f39a35700 1 --2- 172.21.15.46:0/1122604768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f34105c60 0x7f3f34106040 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.069+0000 7f3f39a35700 1 --2- 172.21.15.46:0/1122604768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f34100570 0x7f3f3410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.069+0000 7f3f39a35700 1 --2- 172.21.15.46:0/1122604768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340ffbb0 0x7f3f34100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.069+0000 7f3f39a35700 1 -- 172.21.15.46:0/1122604768 >> 172.21.15.46:0/1122604768 conn(0x7f3f34076d50 msgr2=0x7f3f34077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.069+0000 7f3f39a35700 1 -- 172.21.15.46:0/1122604768 shutdown_connections 2024-03-18T20:37:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.069+0000 7f3f39a35700 1 -- 172.21.15.46:0/1122604768 wait complete. 2024-03-18T20:37:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.069+0000 7f3f39a35700 1 Processor -- start 2024-03-18T20:37:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f39a35700 1 -- start start 2024-03-18T20:37:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f39a35700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340ffbb0 0x7f3f341970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f39a35700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f34100570 0x7f3f34197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f39a35700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f34105c60 0x7f3f3419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f39a35700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3419c010 con 0x7f3f34105c60 2024-03-18T20:37:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f39a35700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3419c180 con 0x7f3f34100570 2024-03-18T20:37:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f39a35700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3419c340 con 0x7f3f340ffbb0 2024-03-18T20:37:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f337fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f34105c60 0x7f3f3419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f32ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340ffbb0 0x7f3f341970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f327fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f34100570 0x7f3f34197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f337fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f34105c60 0x7f3f3419b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32812/0 (socket says 172.21.15.46:32812) 2024-03-18T20:37:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f337fe700 1 -- 172.21.15.46:0/4104077232 learned_addr learned my addr 172.21.15.46:0/4104077232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f337fe700 1 -- 172.21.15.46:0/4104077232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340ffbb0 msgr2=0x7f3f341970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f337fe700 1 --2- 172.21.15.46:0/4104077232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340ffbb0 0x7f3f341970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f337fe700 1 -- 172.21.15.46:0/4104077232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f34100570 msgr2=0x7f3f34197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f337fe700 1 --2- 172.21.15.46:0/4104077232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f34100570 0x7f3f34197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.070+0000 7f3f337fe700 1 -- 172.21.15.46:0/4104077232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c009c40 con 0x7f3f34105c60 2024-03-18T20:37:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.071+0000 7f3f337fe700 1 --2- 172.21.15.46:0/4104077232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f34105c60 0x7f3f3419b9d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c370 tx=0x7f3f2c00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.071+0000 7f3f38a33700 1 -- 172.21.15.46:0/4104077232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c017070 con 0x7f3f34105c60 2024-03-18T20:37:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.071+0000 7f3f38a33700 1 -- 172.21.15.46:0/4104077232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f2c00f040 con 0x7f3f34105c60 2024-03-18T20:37:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.071+0000 7f3f39a35700 1 -- 172.21.15.46:0/4104077232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f240095f0 con 0x7f3f34105c60 2024-03-18T20:37:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.071+0000 7f3f38a33700 1 -- 172.21.15.46:0/4104077232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c00e050 con 0x7f3f34105c60 2024-03-18T20:37:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.071+0000 7f3f39a35700 1 -- 172.21.15.46:0/4104077232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f341a3c80 con 0x7f3f34105c60 2024-03-18T20:37:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.073+0000 7f3f38a33700 1 -- 172.21.15.46:0/4104077232 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f3f2c00cb60 con 0x7f3f34105c60 2024-03-18T20:37:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.074+0000 7f3f38a33700 1 --2- 172.21.15.46:0/4104077232 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f1c0a13b0 0x7f3f1c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.074+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/4104077232 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f1c0a13b0 0x7f3f1c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.074+0000 7f3f38a33700 1 -- 172.21.15.46:0/4104077232 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f3f2c0cc8c0 con 0x7f3f34105c60 2024-03-18T20:37:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f45bcab80 1 Processor -- start 2024-03-18T20:37:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/4104077232 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f1c0a13b0 0x7f3f1c0a3870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400b2e0 tx=0x7f3f24005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f45bcab80 1 -- start start 2024-03-18T20:37:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f45bcab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de7155570 0x562de7155950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f45bcab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155e90 0x562de7164c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f45bcab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de71651c0 0x562de71675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f45bcab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de70af930 con 0x562de71651c0 2024-03-18T20:37:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f45bcab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de70ad2f0 con 0x562de7155570 2024-03-18T20:37:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f45bcab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de70a8be0 con 0x562de7155e90 2024-03-18T20:37:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f32ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de7155570 0x562de7155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f337fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de71651c0 0x562de71675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f32ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de7155570 0x562de7155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52672/0 (socket says 172.21.15.46:52672) 2024-03-18T20:37:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f32ffd700 1 -- 172.21.15.46:0/47774171 learned_addr learned my addr 172.21.15.46:0/47774171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.075+0000 7f3f327fc700 1 --2- 172.21.15.46:0/47774171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155e90 0x562de7164c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.076+0000 7f3f32ffd700 1 -- 172.21.15.46:0/47774171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155e90 msgr2=0x562de7164c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.076+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/47774171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155e90 0x562de7164c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.076+0000 7f3f32ffd700 1 -- 172.21.15.46:0/47774171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de71651c0 msgr2=0x562de71675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.076+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/47774171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de71651c0 0x562de71675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.076+0000 7f3f32ffd700 1 -- 172.21.15.46:0/47774171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562de6f2a350 con 0x562de7155570 2024-03-18T20:37:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.076+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/47774171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de7155570 0x562de7155950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009990 tx=0x7f3f24020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83d63592b984e8c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.076+0000 7f3f19ffb700 1 -- 172.21.15.46:0/47774171 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24004700 con 0x562de7155570 2024-03-18T20:37:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.076+0000 7f3f19ffb700 1 -- 172.21.15.46:0/47774171 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f2401e040 con 0x562de7155570 2024-03-18T20:37:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.076+0000 7f3f19ffb700 1 -- 172.21.15.46:0/47774171 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400fa10 con 0x562de7155570 2024-03-18T20:37:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.077+0000 7f3f45bcab80 1 -- 172.21.15.46:0/47774171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de7155570 msgr2=0x562de7155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.077+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/47774171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de7155570 0x562de7155950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009990 tx=0x7f3f24020000 comp rx=0 tx=0).stop 2024-03-18T20:37:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.077+0000 7f3f45bcab80 1 -- 172.21.15.46:0/47774171 shutdown_connections 2024-03-18T20:37:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.077+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/47774171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de71651c0 0x562de71675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.077+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/47774171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de7155570 0x562de7155950 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.077+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/47774171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155e90 0x562de7164c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.077+0000 7f3f45bcab80 1 -- 172.21.15.46:0/47774171 >> 172.21.15.46:0/47774171 conn(0x562de70cc1c0 msgr2=0x562de7151a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.077+0000 7f3f45bcab80 1 -- 172.21.15.46:0/47774171 shutdown_connections 2024-03-18T20:37:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.077+0000 7f3f45bcab80 1 -- 172.21.15.46:0/47774171 wait complete. 2024-03-18T20:37:16.085 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.077+0000 7f3f45bcab80 1 -- start start 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.077+0000 7f3f45bcab80 1 Processor -- start 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f45bcab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155570 0x562de7164c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f45bcab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155e90 0x562de7161f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f45bcab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de71651c0 0x562de7162560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f45bcab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de7044d20 con 0x562de7155e90 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f45bcab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de70449c0 con 0x562de71651c0 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f45bcab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de6f2c640 con 0x562de7155570 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f45bcab80 10 client.0 _ll_register_callbacks cb 0x562de6eb0340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f327fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155e90 0x562de7161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f327fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155e90 0x562de7161f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32830/0 (socket says 172.21.15.46:32830) 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f327fc700 1 -- 172.21.15.46:0/2347261922 learned_addr learned my addr 172.21.15.46:0/2347261922 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155570 0x562de7164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f337fe700 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de71651c0 0x562de7162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f327fc700 1 -- 172.21.15.46:0/2347261922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155570 msgr2=0x562de7164c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f327fc700 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155570 0x562de7164c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f327fc700 1 -- 172.21.15.46:0/2347261922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de71651c0 msgr2=0x562de7162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f327fc700 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de71651c0 0x562de7162560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.078+0000 7f3f327fc700 1 -- 172.21.15.46:0/2347261922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f34068240 con 0x562de7155e90 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f327fc700 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155e90 0x562de7161f00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800b810 tx=0x7f3f2800bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59a9c895ccf07d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f12ffd700 1 -- 172.21.15.46:0/2347261922 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28003c30 con 0x562de7155e90 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f12ffd700 1 -- 172.21.15.46:0/2347261922 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f3f28003d90 con 0x562de7155e90 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2347261922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562de71496c0 con 0x562de7155e90 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f12ffd700 1 -- 172.21.15.46:0/2347261922 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f28004830 con 0x562de7155e90 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f45bcab80 10 client.15948 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2347261922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562de6ed6b00 con 0x562de7155e90 2024-03-18T20:37:16.087 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f12ffd700 1 -- 172.21.15.46:0/2347261922 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562de71496c0 con 0x562de7155e90 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f12ffd700 1 -- 172.21.15.46:0/2347261922 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3f28023470 con 0x562de7155e90 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f45bcab80 10 client.15948 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f45bcab80 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f45bcab80 20 client.15948 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562de7174da0 0x562de71771c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f45bcab80 4 client.15948 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.079+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2347261922 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562de70a16a0 con 0x562de7174da0 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.080+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562de7174da0 0x562de71771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.080+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562de7174da0 0x562de71771c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed1b8b8aa261fae2 server_cookie=4bb0c6d4d6d9cccf in_seq=0 out_seq=0 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.080+0000 7f3f12ffd700 10 client.15948 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f12ffd700 1 -- 172.21.15.46:0/2347261922 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562de70a16a0 con 0x562de7174da0 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f12ffd700 10 client.15948 handle_command_reply: tid=0 2024-03-18T20:37:16.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f45bcab80 1 client.15948 shutdown 2024-03-18T20:37:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2347261922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155e90 msgr2=0x562de7161f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155e90 0x562de7161f00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800b810 tx=0x7f3f2800bb20 comp rx=0 tx=0).stop 2024-03-18T20:37:16.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2347261922 shutdown_connections 2024-03-18T20:37:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562de7174da0 0x562de71771c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de71651c0 0x562de7162560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155570 0x562de7164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f12ffd700 0 client.15948 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/2347261922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155e90 0x562de7161f00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2347261922 >> 172.21.15.46:0/2347261922 conn(0x562de70cc1c0 msgr2=0x562de7151650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2347261922 shutdown_connections 2024-03-18T20:37:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.082+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2347261922 wait complete. 2024-03-18T20:37:16.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.083+0000 7f3f45bcab80 20 client.15948 trim_cache size 0 max 16384 2024-03-18T20:37:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.098+0000 7f3f45bcab80 1 Processor -- start 2024-03-18T20:37:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.098+0000 7f3f45bcab80 1 -- start start 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.098+0000 7f3f45bcab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 0x562de716bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.098+0000 7f3f45bcab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de7155e90 0x562de7164740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.098+0000 7f3f45bcab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de71651c0 0x562de7161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.098+0000 7f3f45bcab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de714bc10 con 0x562de7155570 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.098+0000 7f3f45bcab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de71628c0 con 0x562de7155e90 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.098+0000 7f3f45bcab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de716c330 con 0x562de71651c0 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f32ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 0x562de716bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f32ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 0x562de716bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32838/0 (socket says 172.21.15.46:32838) 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f32ffd700 1 -- 172.21.15.46:0/1105660130 learned_addr learned my addr 172.21.15.46:0/1105660130 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f327fc700 1 --2- 172.21.15.46:0/1105660130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de7155e90 0x562de7164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f337fe700 1 --2- 172.21.15.46:0/1105660130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de71651c0 0x562de7161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f32ffd700 1 -- 172.21.15.46:0/1105660130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de71651c0 msgr2=0x562de7161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/1105660130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de71651c0 0x562de7161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f32ffd700 1 -- 172.21.15.46:0/1105660130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de7155e90 msgr2=0x562de7164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/1105660130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de7155e90 0x562de7164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f32ffd700 1 -- 172.21.15.46:0/1105660130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562de706ee40 con 0x562de7155570 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/1105660130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 0x562de716bdf0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3f24005850 tx=0x7f3f2401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7556a95043b2e360 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f1a7fc700 1 -- 172.21.15.46:0/1105660130 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2403dcd0 con 0x562de7155570 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f1a7fc700 1 -- 172.21.15.46:0/1105660130 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f2403de30 con 0x562de7155570 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.099+0000 7f3f1a7fc700 1 -- 172.21.15.46:0/1105660130 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24004e90 con 0x562de7155570 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.100+0000 7f3f45bcab80 1 -- 172.21.15.46:0/1105660130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 msgr2=0x562de716bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.100+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/1105660130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 0x562de716bdf0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3f24005850 tx=0x7f3f2401a040 comp rx=0 tx=0).stop 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.100+0000 7f3f45bcab80 1 -- 172.21.15.46:0/1105660130 shutdown_connections 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.100+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/1105660130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 0x562de716bdf0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.100+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/1105660130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de71651c0 0x562de7161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.100+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/1105660130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de7155e90 0x562de7164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.100+0000 7f3f45bcab80 1 -- 172.21.15.46:0/1105660130 >> 172.21.15.46:0/1105660130 conn(0x562de70cc1c0 msgr2=0x562de717bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.100+0000 7f3f45bcab80 1 -- 172.21.15.46:0/1105660130 shutdown_connections 2024-03-18T20:37:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.100+0000 7f3f45bcab80 1 -- 172.21.15.46:0/1105660130 wait complete. 2024-03-18T20:37:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.100+0000 7f3f45bcab80 1 -- start start 2024-03-18T20:37:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f45bcab80 1 Processor -- start 2024-03-18T20:37:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f45bcab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 0x562de7169dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f45bcab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155e90 0x562de716a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f45bcab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de71651c0 0x562de71c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f45bcab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de716aa80 con 0x562de7155570 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f45bcab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de716abf0 con 0x562de71651c0 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f45bcab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de71c8110 con 0x562de7155e90 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f45bcab80 10 client.0 _ll_register_callbacks cb 0x7f3f280041d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f32ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 0x562de7169dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f327fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155e90 0x562de716a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f337fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de71651c0 0x562de71c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f327fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155e90 0x562de716a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:49308/0 (socket says 172.21.15.46:49308) 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f337fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de71651c0 0x562de71c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52698/0 (socket says 172.21.15.46:52698) 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f32ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 0x562de7169dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32854/0 (socket says 172.21.15.46:32854) 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.101+0000 7f3f337fe700 1 -- 172.21.15.46:0/2696469978 learned_addr learned my addr 172.21.15.46:0/2696469978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:16.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f337fe700 1 -- 172.21.15.46:0/2696469978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155e90 msgr2=0x562de716a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f337fe700 1 --2- 172.21.15.46:0/2696469978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155e90 0x562de716a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f337fe700 1 -- 172.21.15.46:0/2696469978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 msgr2=0x562de7169dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f337fe700 1 --2- 172.21.15.46:0/2696469978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 0x562de7169dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f337fe700 1 -- 172.21.15.46:0/2696469978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562de6f2a350 con 0x562de71651c0 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f337fe700 1 --2- 172.21.15.46:0/2696469978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de71651c0 0x562de71c7bd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c095c40 tx=0x7f3f2c0b9330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eefd3ea9ac48a0c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f12ffd700 1 -- 172.21.15.46:0/2696469978 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c099790 con 0x562de71651c0 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f12ffd700 1 -- 172.21.15.46:0/2696469978 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f3f2c09d5c0 con 0x562de71651c0 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2696469978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f2401e040 con 0x562de71651c0 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f12ffd700 1 -- 172.21.15.46:0/2696469978 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f2c0a1c10 con 0x562de71651c0 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f45bcab80 10 client.25724 fetch_fsmap learned FSMap version 27 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2696469978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562de6f2ec70 con 0x562de71651c0 2024-03-18T20:37:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.102+0000 7f3f12ffd700 1 -- 172.21.15.46:0/2696469978 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f2401e040 con 0x562de71651c0 2024-03-18T20:37:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.103+0000 7f3f12ffd700 1 -- 172.21.15.46:0/2696469978 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3f2c098d10 con 0x562de71651c0 2024-03-18T20:37:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.103+0000 7f3f45bcab80 10 client.25724 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T20:37:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.103+0000 7f3f45bcab80 10 client.25724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.103+0000 7f3f45bcab80 20 client.25724 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.103+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/2696469978 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562de71c9880 0x562de71cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.103+0000 7f3f45bcab80 4 client.25724 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:37:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.103+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2696469978 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562de6ef7c90 con 0x562de71c9880 2024-03-18T20:37:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.103+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/2696469978 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562de71c9880 0x562de71cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.103+0000 7f3f12ffd700 10 client.25724 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.103+0000 7f3f32ffd700 1 --2- 172.21.15.46:0/2696469978 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562de71c9880 0x562de71cbca0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d71b4f1128fd6a server_cookie=2b53325198ccefcc in_seq=0 out_seq=0 2024-03-18T20:37:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.207+0000 7f3f12ffd700 1 -- 172.21.15.46:0/2696469978 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562de6ef7c90 con 0x562de71c9880 2024-03-18T20:37:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.207+0000 7f3f12ffd700 10 client.25724 handle_command_reply: tid=0 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.207+0000 7f3f45bcab80 1 client.25724 shutdown 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.208+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2696469978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de71651c0 msgr2=0x562de71c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.208+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/2696469978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de71651c0 0x562de71c7bd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c095c40 tx=0x7f3f2c0b9330 comp rx=0 tx=0).stop 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.208+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2696469978 shutdown_connections 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.208+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/2696469978 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562de71c9880 0x562de71cbca0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.208+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/2696469978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562de7155570 0x562de7169dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.208+0000 7f3f12ffd700 0 client.25724 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.208+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/2696469978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de71651c0 0x562de71c7bd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.208+0000 7f3f45bcab80 1 --2- 172.21.15.46:0/2696469978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562de7155e90 0x562de716a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.208+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2696469978 >> 172.21.15.46:0/2696469978 conn(0x562de70cc1c0 msgr2=0x562de7161790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.208+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2696469978 shutdown_connections 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.208+0000 7f3f45bcab80 1 -- 172.21.15.46:0/2696469978 wait complete. 2024-03-18T20:37:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.208+0000 7f3f45bcab80 20 client.25724 trim_cache size 0 max 16384 2024-03-18T20:37:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:37:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:37:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:37:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:37:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.209+0000 7f3f39a35700 1 -- 172.21.15.46:0/4104077232 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f1c0a13b0 msgr2=0x7f3f1c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.209+0000 7f3f39a35700 1 --2- 172.21.15.46:0/4104077232 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f1c0a13b0 0x7f3f1c0a3870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400b2e0 tx=0x7f3f24005960 comp rx=0 tx=0).stop 2024-03-18T20:37:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.209+0000 7f3f39a35700 1 -- 172.21.15.46:0/4104077232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f34105c60 msgr2=0x7f3f3419b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.209+0000 7f3f39a35700 1 --2- 172.21.15.46:0/4104077232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f34105c60 0x7f3f3419b9d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c370 tx=0x7f3f2c00c680 comp rx=0 tx=0).stop 2024-03-18T20:37:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.209+0000 7f3f39a35700 1 -- 172.21.15.46:0/4104077232 shutdown_connections 2024-03-18T20:37:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.209+0000 7f3f39a35700 1 --2- 172.21.15.46:0/4104077232 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f1c0a13b0 0x7f3f1c0a3870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.209+0000 7f3f39a35700 1 --2- 172.21.15.46:0/4104077232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f34105c60 0x7f3f3419b9d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.209+0000 7f3f39a35700 1 --2- 172.21.15.46:0/4104077232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f340ffbb0 0x7f3f341970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.209+0000 7f3f39a35700 1 --2- 172.21.15.46:0/4104077232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f34100570 0x7f3f34197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.209+0000 7f3f39a35700 1 -- 172.21.15.46:0/4104077232 >> 172.21.15.46:0/4104077232 conn(0x7f3f34076d50 msgr2=0x7f3f340fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.209+0000 7f3f39a35700 1 -- 172.21.15.46:0/4104077232 shutdown_connections 2024-03-18T20:37:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:16.209+0000 7f3f39a35700 1 -- 172.21.15.46:0/4104077232 wait complete. 2024-03-18T20:37:16.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:15 smithi073 ceph-mon[47384]: pgmap v37: 97 pgs: 97 active+clean; 13 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-03-18T20:37:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:15 smithi155 ceph-mon[48344]: pgmap v37: 97 pgs: 97 active+clean; 13 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-03-18T20:37:16.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:15 smithi046 ceph-mon[43495]: pgmap v37: 97 pgs: 97 active+clean; 13 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-03-18T20:37:16.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 33652 18.03 MB/sec warmup 11 sec latency 61.551 ms 2024-03-18T20:37:16.790 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:37:16 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: ignoring --setuser ceph since I am not root 2024-03-18T20:37:16.791 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:37:16 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: ignoring --setgroup ceph since I am not root 2024-03-18T20:37:16.791 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:37:16 smithi155 ceph-mds[84998]: -- 172.21.15.155:0/3395652626 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5616f86741a0 con 0x5616f864a800 2024-03-18T20:37:16.791 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:37:16 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: starting mds.c at 2024-03-18T20:37:17.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:16 smithi073 ceph-mon[47384]: daemon mds.c restarted 2024-03-18T20:37:17.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:16 smithi073 ceph-mon[47384]: osdmap e82: 12 total, 12 up, 12 in 2024-03-18T20:37:17.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:16 smithi155 ceph-mon[48344]: daemon mds.c restarted 2024-03-18T20:37:17.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:16 smithi155 ceph-mon[48344]: osdmap e82: 12 total, 12 up, 12 in 2024-03-18T20:37:17.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:16 smithi046 ceph-mon[43495]: daemon mds.c restarted 2024-03-18T20:37:17.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:16 smithi046 ceph-mon[43495]: osdmap e82: 12 total, 12 up, 12 in 2024-03-18T20:37:17.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 36663 17.85 MB/sec warmup 12 sec latency 8.750 ms 2024-03-18T20:37:18.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:17 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6836/3087491981,v1:172.21.15.155:6837/3087491981] up:boot 2024-03-18T20:37:18.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:17 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T20:37:18.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:17 smithi073 ceph-mon[47384]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T20:37:18.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:17 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:18.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:17 smithi073 ceph-mon[47384]: pgmap v39: 97 pgs: 97 active+clean; 13 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-03-18T20:37:18.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:17 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6836/3087491981,v1:172.21.15.155:6837/3087491981] up:boot 2024-03-18T20:37:18.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:17 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T20:37:18.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:17 smithi155 ceph-mon[48344]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T20:37:18.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:17 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:18.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:17 smithi155 ceph-mon[48344]: pgmap v39: 97 pgs: 97 active+clean; 13 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-03-18T20:37:18.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:17 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6836/3087491981,v1:172.21.15.155:6837/3087491981] up:boot 2024-03-18T20:37:18.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:17 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T20:37:18.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:17 smithi046 ceph-mon[43495]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T20:37:18.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:17 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:18.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:17 smithi046 ceph-mon[43495]: pgmap v39: 97 pgs: 97 active+clean; 13 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-03-18T20:37:18.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 39287 17.77 MB/sec warmup 13 sec latency 228.642 ms 2024-03-18T20:37:19.668 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42066 17.42 MB/sec warmup 14 sec latency 84.921 ms 2024-03-18T20:37:20.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:19 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:20.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:19 smithi073 ceph-mon[47384]: pgmap v40: 97 pgs: 97 active+clean; 11 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.7 MiB/s wr, 349 op/s 2024-03-18T20:37:20.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:19 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:19 smithi155 ceph-mon[48344]: pgmap v40: 97 pgs: 97 active+clean; 11 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.7 MiB/s wr, 349 op/s 2024-03-18T20:37:20.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:19 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:19 smithi046 ceph-mon[43495]: pgmap v40: 97 pgs: 97 active+clean; 11 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.7 MiB/s wr, 349 op/s 2024-03-18T20:37:20.669 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42607 16.59 MB/sec warmup 15 sec latency 854.811 ms 2024-03-18T20:37:21.669 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42607 15.55 MB/sec warmup 16 sec latency 1854.924 ms 2024-03-18T20:37:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:22 smithi073 ceph-mon[47384]: pgmap v41: 97 pgs: 97 active+clean; 11 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.7 MiB/s wr, 349 op/s 2024-03-18T20:37:22.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:22 smithi155 ceph-mon[48344]: pgmap v41: 97 pgs: 97 active+clean; 11 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.7 MiB/s wr, 349 op/s 2024-03-18T20:37:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:22 smithi046 ceph-mon[43495]: pgmap v41: 97 pgs: 97 active+clean; 11 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.7 MiB/s wr, 349 op/s 2024-03-18T20:37:22.669 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42607 14.63 MB/sec warmup 17 sec latency 2855.018 ms 2024-03-18T20:37:23.669 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42607 13.82 MB/sec warmup 18 sec latency 3855.104 ms 2024-03-18T20:37:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:24 smithi073 ceph-mon[47384]: pgmap v42: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.2 MiB/s wr, 275 op/s 2024-03-18T20:37:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:24 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:24 smithi155 ceph-mon[48344]: pgmap v42: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.2 MiB/s wr, 275 op/s 2024-03-18T20:37:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:24 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:24 smithi046 ceph-mon[43495]: pgmap v42: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.2 MiB/s wr, 275 op/s 2024-03-18T20:37:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:24 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:24.669 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42672 13.10 MB/sec warmup 19 sec latency 4371.531 ms 2024-03-18T20:37:25.669 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42672 12.45 MB/sec warmup 20 sec latency 1468.606 ms 2024-03-18T20:37:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:25 smithi073 ceph-mon[47384]: pgmap v43: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.2 MiB/s wr, 275 op/s 2024-03-18T20:37:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:25 smithi155 ceph-mon[48344]: pgmap v43: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.2 MiB/s wr, 275 op/s 2024-03-18T20:37:26.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:25 smithi046 ceph-mon[43495]: pgmap v43: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.2 MiB/s wr, 275 op/s 2024-03-18T20:37:26.669 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42672 11.86 MB/sec warmup 21 sec latency 2468.660 ms 2024-03-18T20:37:27.669 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42672 11.32 MB/sec warmup 22 sec latency 3468.771 ms 2024-03-18T20:37:28.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:28 smithi073 ceph-mon[47384]: pgmap v44: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-03-18T20:37:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:28 smithi155 ceph-mon[48344]: pgmap v44: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-03-18T20:37:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:28 smithi046 ceph-mon[43495]: pgmap v44: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-03-18T20:37:28.669 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42672 10.82 MB/sec warmup 23 sec latency 4468.887 ms 2024-03-18T20:37:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.338+0000 7f8e63a1f700 1 -- 172.21.15.46:0/3174157595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c101890 msgr2=0x7f8e5c1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.338+0000 7f8e63a1f700 1 --2- 172.21.15.46:0/3174157595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c101890 0x7f8e5c1058d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009990 tx=0x7f8e4c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.338+0000 7f8e63a1f700 1 -- 172.21.15.46:0/3174157595 shutdown_connections 2024-03-18T20:37:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.338+0000 7f8e63a1f700 1 --2- 172.21.15.46:0/3174157595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e5c106000 0x7f8e5c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.338+0000 7f8e63a1f700 1 --2- 172.21.15.46:0/3174157595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5c100ee0 0x7f8e5c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.338+0000 7f8e63a1f700 1 --2- 172.21.15.46:0/3174157595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c101890 0x7f8e5c1058d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.338+0000 7f8e63a1f700 1 -- 172.21.15.46:0/3174157595 >> 172.21.15.46:0/3174157595 conn(0x7f8e5c0fc780 msgr2=0x7f8e5c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.338+0000 7f8e63a1f700 1 -- 172.21.15.46:0/3174157595 shutdown_connections 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.338+0000 7f8e63a1f700 1 -- 172.21.15.46:0/3174157595 wait complete. 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e63a1f700 1 Processor -- start 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e63a1f700 1 -- start start 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e63a1f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5c100ee0 0x7f8e5c197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e63a1f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e5c106000 0x7f8e5c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e63a1f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c19ba90 0x7f8e5c19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e63a1f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c19c510 con 0x7f8e5c19ba90 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e63a1f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c19c680 con 0x7f8e5c100ee0 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e63a1f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c19c840 con 0x7f8e5c106000 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e61fbc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c19ba90 0x7f8e5c19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e61fbc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c19ba90 0x7f8e5c19bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32932/0 (socket says 172.21.15.46:32932) 2024-03-18T20:37:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e61fbc700 1 -- 172.21.15.46:0/2149568487 learned_addr learned my addr 172.21.15.46:0/2149568487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e617bb700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5c100ee0 0x7f8e5c197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.339+0000 7f8e60fba700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e5c106000 0x7f8e5c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.340+0000 7f8e617bb700 1 -- 172.21.15.46:0/2149568487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e5c106000 msgr2=0x7f8e5c197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.340+0000 7f8e617bb700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e5c106000 0x7f8e5c197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.340+0000 7f8e617bb700 1 -- 172.21.15.46:0/2149568487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c19ba90 msgr2=0x7f8e5c19bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.340+0000 7f8e617bb700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c19ba90 0x7f8e5c19bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.340+0000 7f8e617bb700 1 -- 172.21.15.46:0/2149568487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e58009520 con 0x7f8e5c100ee0 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.340+0000 7f8e61fbc700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c19ba90 0x7f8e5c19bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.340+0000 7f8e617bb700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5c100ee0 0x7f8e5c197200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800e9f0 tx=0x7f8e5800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.340+0000 7f8e527fc700 1 -- 172.21.15.46:0/2149568487 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800cb70 con 0x7f8e5c100ee0 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.340+0000 7f8e63a1f700 1 -- 172.21.15.46:0/2149568487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c0095f0 con 0x7f8e5c100ee0 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.340+0000 7f8e527fc700 1 -- 172.21.15.46:0/2149568487 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e5800ee90 con 0x7f8e5c100ee0 2024-03-18T20:37:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.340+0000 7f8e527fc700 1 -- 172.21.15.46:0/2149568487 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e58003cf0 con 0x7f8e5c100ee0 2024-03-18T20:37:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.340+0000 7f8e63a1f700 1 -- 172.21.15.46:0/2149568487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c1a40f0 con 0x7f8e5c100ee0 2024-03-18T20:37:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.342+0000 7f8e527fc700 1 -- 172.21.15.46:0/2149568487 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f8e5800ccd0 con 0x7f8e5c100ee0 2024-03-18T20:37:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.343+0000 7f8e527fc700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8e480a13b0 0x7f8e480a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.343+0000 7f8e60fba700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8e480a13b0 0x7f8e480a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.343+0000 7f8e527fc700 1 -- 172.21.15.46:0/2149568487 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8e58014070 con 0x7f8e5c100ee0 2024-03-18T20:37:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.344+0000 7f8e60fba700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8e480a13b0 0x7f8e480a3870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00b2e0 tx=0x7f8e4c005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.344+0000 7f8e6fbb4b80 1 Processor -- start 2024-03-18T20:37:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.344+0000 7f8e6fbb4b80 1 -- start start 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e6fbb4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c10f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e6fbb4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c10f5e90 0x55c6c1104c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e6fbb4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 0x55c6c11075b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e6fbb4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c104f930 con 0x55c6c11051c0 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e6fbb4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c104d2f0 con 0x55c6c10f5e90 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e6fbb4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c1048be0 con 0x55c6c10f5570 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e61fbc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 0x55c6c11075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e61fbc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 0x55c6c11075b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32934/0 (socket says 172.21.15.46:32934) 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e61fbc700 1 -- 172.21.15.46:0/3531608438 learned_addr learned my addr 172.21.15.46:0/3531608438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e617bb700 1 --2- 172.21.15.46:0/3531608438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c10f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e60fba700 1 --2- 172.21.15.46:0/3531608438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c10f5e90 0x55c6c1104c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e61fbc700 1 -- 172.21.15.46:0/3531608438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 msgr2=0x55c6c10f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e61fbc700 1 --2- 172.21.15.46:0/3531608438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c10f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e61fbc700 1 -- 172.21.15.46:0/3531608438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c10f5e90 msgr2=0x55c6c1104c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e61fbc700 1 --2- 172.21.15.46:0/3531608438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c10f5e90 0x55c6c1104c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e61fbc700 1 -- 172.21.15.46:0/3531608438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c0eca350 con 0x55c6c11051c0 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.345+0000 7f8e61fbc700 1 --2- 172.21.15.46:0/3531608438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 0x55c6c11075b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8e54009500 tx=0x7f8e54009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf72a9866421cbc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.346+0000 7f8e477fe700 1 -- 172.21.15.46:0/3531608438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5400e900 con 0x55c6c11051c0 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.346+0000 7f8e477fe700 1 -- 172.21.15.46:0/3531608438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e5400f040 con 0x55c6c11051c0 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.346+0000 7f8e477fe700 1 -- 172.21.15.46:0/3531608438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54017cc0 con 0x55c6c11051c0 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.346+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/3531608438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 msgr2=0x55c6c11075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.346+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/3531608438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 0x55c6c11075b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8e54009500 tx=0x7f8e54009920 comp rx=0 tx=0).stop 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.346+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/3531608438 shutdown_connections 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.346+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/3531608438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 0x55c6c11075b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.346+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/3531608438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c10f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.346+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/3531608438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c10f5e90 0x55c6c1104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.346+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/3531608438 >> 172.21.15.46:0/3531608438 conn(0x55c6c106c1c0 msgr2=0x55c6c10f1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.346+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/3531608438 shutdown_connections 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.346+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/3531608438 wait complete. 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.347+0000 7f8e6fbb4b80 1 -- start start 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.347+0000 7f8e6fbb4b80 1 Processor -- start 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e6fbb4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c1104bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e6fbb4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 0x55c6c1101e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e6fbb4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c11051c0 0x55c6c11025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e6fbb4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0fe4d20 con 0x55c6c10f5e90 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e6fbb4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0fe49c0 con 0x55c6c11051c0 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e6fbb4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c0ecc640 con 0x55c6c10f5570 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e6fbb4b80 10 client.0 _ll_register_callbacks cb 0x55c6c0e50340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e60fba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 0x55c6c1101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e60fba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 0x55c6c1101e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32950/0 (socket says 172.21.15.46:32950) 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e60fba700 1 -- 172.21.15.46:0/2899166252 learned_addr learned my addr 172.21.15.46:0/2899166252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e61fbc700 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c11051c0 0x55c6c11025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e617bb700 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c1104bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e60fba700 1 -- 172.21.15.46:0/2899166252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 msgr2=0x55c6c1104bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e60fba700 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c1104bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e60fba700 1 -- 172.21.15.46:0/2899166252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c11051c0 msgr2=0x55c6c11025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e60fba700 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c11051c0 0x55c6c11025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e60fba700 1 -- 172.21.15.46:0/2899166252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e540091e0 con 0x55c6c10f5e90 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e617bb700 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c1104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e60fba700 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 0x55c6c1101e70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009990 tx=0x7f8e4c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90148debaa92e7e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.348+0000 7f8e44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e44ff9700 1 -- 172.21.15.46:0/2899166252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c004cf0 con 0x55c6c10f5e90 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e44ff9700 1 -- 172.21.15.46:0/2899166252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8e4c004e50 con 0x55c6c10f5e90 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2899166252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6c10e96c0 con 0x55c6c10f5e90 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e44ff9700 1 -- 172.21.15.46:0/2899166252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e4c00f460 con 0x55c6c10f5e90 2024-03-18T20:37:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e6fbb4b80 10 client.15972 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2899166252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c6c0e76b00 con 0x55c6c10f5e90 2024-03-18T20:37:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e44ff9700 1 -- 172.21.15.46:0/2899166252 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6c10e96c0 con 0x55c6c10f5e90 2024-03-18T20:37:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e44ff9700 1 -- 172.21.15.46:0/2899166252 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8e4c02f070 con 0x55c6c10f5e90 2024-03-18T20:37:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e6fbb4b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e6fbb4b80 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e6fbb4b80 20 client.15972 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c6c1114ed0 0x55c6c11172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e6fbb4b80 4 client.15972 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.349+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2899166252 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6c10416a0 con 0x55c6c1114ed0 2024-03-18T20:37:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.350+0000 7f8e617bb700 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c6c1114ed0 0x55c6c11172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.350+0000 7f8e617bb700 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c6c1114ed0 0x55c6c11172f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e14fbefcefdfa11 server_cookie=a90b271a923e97bf in_seq=0 out_seq=0 2024-03-18T20:37:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.350+0000 7f8e44ff9700 10 client.15972 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e44ff9700 1 -- 172.21.15.46:0/2899166252 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6c10416a0 con 0x55c6c1114ed0 2024-03-18T20:37:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e44ff9700 10 client.15972 handle_command_reply: tid=0 2024-03-18T20:37:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e6fbb4b80 1 client.15972 shutdown 2024-03-18T20:37:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2899166252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 msgr2=0x55c6c1101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 0x55c6c1101e70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009990 tx=0x7f8e4c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2899166252 shutdown_connections 2024-03-18T20:37:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c6c1114ed0 0x55c6c11172f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c11051c0 0x55c6c11025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e44ff9700 0 client.15972 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c1104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/2899166252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 0x55c6c1101e70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2899166252 >> 172.21.15.46:0/2899166252 conn(0x55c6c106c1c0 msgr2=0x55c6c10f1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2899166252 shutdown_connections 2024-03-18T20:37:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2899166252 wait complete. 2024-03-18T20:37:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.351+0000 7f8e6fbb4b80 20 client.15972 trim_cache size 0 max 16384 2024-03-18T20:37:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.366+0000 7f8e6fbb4b80 1 Processor -- start 2024-03-18T20:37:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.366+0000 7f8e6fbb4b80 1 -- start start 2024-03-18T20:37:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e6fbb4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c11074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e6fbb4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c10f5e90 0x55c6c110be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e6fbb4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 0x55c6c10fd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e6fbb4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c10ee1d0 con 0x55c6c11051c0 2024-03-18T20:37:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e6fbb4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c10f6850 con 0x55c6c10f5e90 2024-03-18T20:37:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e6fbb4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c10f26a0 con 0x55c6c10f5570 2024-03-18T20:37:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e61fbc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 0x55c6c10fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e61fbc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 0x55c6c10fd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32956/0 (socket says 172.21.15.46:32956) 2024-03-18T20:37:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e61fbc700 1 -- 172.21.15.46:0/3447456578 learned_addr learned my addr 172.21.15.46:0/3447456578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e60fba700 1 --2- 172.21.15.46:0/3447456578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c10f5e90 0x55c6c110be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e617bb700 1 --2- 172.21.15.46:0/3447456578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c11074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e61fbc700 1 -- 172.21.15.46:0/3447456578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 msgr2=0x55c6c11074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e61fbc700 1 --2- 172.21.15.46:0/3447456578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c11074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e61fbc700 1 -- 172.21.15.46:0/3447456578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c10f5e90 msgr2=0x55c6c110be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e61fbc700 1 --2- 172.21.15.46:0/3447456578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c10f5e90 0x55c6c110be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e61fbc700 1 -- 172.21.15.46:0/3447456578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c100ee40 con 0x55c6c11051c0 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e61fbc700 1 --2- 172.21.15.46:0/3447456578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 0x55c6c10fd140 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8e54009500 tx=0x7f8e5400afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8719a1f5ab4bc796 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e47fff700 1 -- 172.21.15.46:0/3447456578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54017660 con 0x55c6c11051c0 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e47fff700 1 -- 172.21.15.46:0/3447456578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e540178d0 con 0x55c6c11051c0 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.367+0000 7f8e47fff700 1 -- 172.21.15.46:0/3447456578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5400eac0 con 0x55c6c11051c0 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.368+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/3447456578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 msgr2=0x55c6c10fd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.368+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/3447456578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 0x55c6c10fd140 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8e54009500 tx=0x7f8e5400afd0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.368+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/3447456578 shutdown_connections 2024-03-18T20:37:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.368+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/3447456578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c11051c0 0x55c6c10fd140 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.368+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/3447456578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c11074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.368+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/3447456578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c10f5e90 0x55c6c110be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.368+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/3447456578 >> 172.21.15.46:0/3447456578 conn(0x55c6c106c1c0 msgr2=0x55c6c10f0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.368+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/3447456578 shutdown_connections 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.368+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/3447456578 wait complete. 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.368+0000 7f8e6fbb4b80 1 -- start start 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e6fbb4b80 1 Processor -- start 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e6fbb4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c1108010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e6fbb4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 0x55c6c1108550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e6fbb4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c11051c0 0x55c6c1167cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e6fbb4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c1108c80 con 0x55c6c10f5e90 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e6fbb4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c11681f0 con 0x55c6c11051c0 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e6fbb4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c1168380 con 0x55c6c10f5570 2024-03-18T20:37:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e6fbb4b80 10 client.0 _ll_register_callbacks cb 0x55c6c0e50340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e60fba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 0x55c6c1108550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e60fba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 0x55c6c1108550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32964/0 (socket says 172.21.15.46:32964) 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e60fba700 1 -- 172.21.15.46:0/2756062358 learned_addr learned my addr 172.21.15.46:0/2756062358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e617bb700 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c1108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e61fbc700 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c11051c0 0x55c6c1167cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e60fba700 1 -- 172.21.15.46:0/2756062358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 msgr2=0x55c6c1108010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e60fba700 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c1108010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e60fba700 1 -- 172.21.15.46:0/2756062358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c11051c0 msgr2=0x55c6c1167cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.369+0000 7f8e60fba700 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c11051c0 0x55c6c1167cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e60fba700 1 -- 172.21.15.46:0/2756062358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6c0eca350 con 0x55c6c10f5e90 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e60fba700 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 0x55c6c1108550 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x55c6c11033b0 tx=0x7f8e4c009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac5360ae7ab065b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e44ff9700 1 -- 172.21.15.46:0/2756062358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c024410 con 0x55c6c10f5e90 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e44ff9700 1 -- 172.21.15.46:0/2756062358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8e4c024e40 con 0x55c6c10f5e90 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2756062358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e5400f040 con 0x55c6c10f5e90 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e44ff9700 1 -- 172.21.15.46:0/2756062358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e4c024570 con 0x55c6c10f5e90 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e6fbb4b80 10 client.15978 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2756062358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c6c0ecec70 con 0x55c6c10f5e90 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e44ff9700 1 -- 172.21.15.46:0/2756062358 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e4c04f020 con 0x55c6c10f5e90 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e44ff9700 1 -- 172.21.15.46:0/2756062358 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8e4c02f070 con 0x55c6c10f5e90 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e6fbb4b80 10 client.15978 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e6fbb4b80 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e6fbb4b80 20 client.15978 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c6c1169710 0x55c6c116bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e6fbb4b80 4 client.15978 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.370+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2756062358 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c6c0e97c90 con 0x55c6c1169710 2024-03-18T20:37:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.371+0000 7f8e617bb700 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c6c1169710 0x55c6c116bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.371+0000 7f8e617bb700 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c6c1169710 0x55c6c116bb00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fb5ae9c02dfa574 server_cookie=af4516648470a786 in_seq=0 out_seq=0 2024-03-18T20:37:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.371+0000 7f8e44ff9700 10 client.15978 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.420+0000 7f8e44ff9700 1 -- 172.21.15.46:0/2756062358 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c6c0e97c90 con 0x55c6c1169710 2024-03-18T20:37:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.420+0000 7f8e44ff9700 10 client.15978 handle_command_reply: tid=0 2024-03-18T20:37:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.420+0000 7f8e6fbb4b80 1 client.15978 shutdown 2024-03-18T20:37:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.421+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2756062358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 msgr2=0x55c6c1108550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.421+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 0x55c6c1108550 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x55c6c11033b0 tx=0x7f8e4c009390 comp rx=0 tx=0).stop 2024-03-18T20:37:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.421+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2756062358 shutdown_connections 2024-03-18T20:37:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.421+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c6c1169710 0x55c6c116bb00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.421+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6c11051c0 0x55c6c1167cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.421+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c6c10f5570 0x55c6c1108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.421+0000 7f8e6fbb4b80 1 --2- 172.21.15.46:0/2756062358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c6c10f5e90 0x55c6c1108550 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.421+0000 7f8e44ff9700 0 client.15978 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.421+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2756062358 >> 172.21.15.46:0/2756062358 conn(0x55c6c106c1c0 msgr2=0x55c6c10fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.421+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2756062358 shutdown_connections 2024-03-18T20:37:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.421+0000 7f8e6fbb4b80 1 -- 172.21.15.46:0/2756062358 wait complete. 2024-03-18T20:37:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.421+0000 7f8e6fbb4b80 20 client.15978 trim_cache size 0 max 16384 2024-03-18T20:37:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:37:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:37:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:37:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:37:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.422+0000 7f8e63a1f700 1 -- 172.21.15.46:0/2149568487 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8e480a13b0 msgr2=0x7f8e480a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.422+0000 7f8e63a1f700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8e480a13b0 0x7f8e480a3870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00b2e0 tx=0x7f8e4c005960 comp rx=0 tx=0).stop 2024-03-18T20:37:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.422+0000 7f8e63a1f700 1 -- 172.21.15.46:0/2149568487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5c100ee0 msgr2=0x7f8e5c197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.422+0000 7f8e63a1f700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5c100ee0 0x7f8e5c197200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800e9f0 tx=0x7f8e5800ed00 comp rx=0 tx=0).stop 2024-03-18T20:37:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.422+0000 7f8e63a1f700 1 -- 172.21.15.46:0/2149568487 shutdown_connections 2024-03-18T20:37:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.422+0000 7f8e63a1f700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8e480a13b0 0x7f8e480a3870 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.422+0000 7f8e63a1f700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c19ba90 0x7f8e5c19bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.422+0000 7f8e63a1f700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5c100ee0 0x7f8e5c197200 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.422+0000 7f8e63a1f700 1 --2- 172.21.15.46:0/2149568487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e5c106000 0x7f8e5c197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.422+0000 7f8e63a1f700 1 -- 172.21.15.46:0/2149568487 >> 172.21.15.46:0/2149568487 conn(0x7f8e5c0fc780 msgr2=0x7f8e5c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.422+0000 7f8e63a1f700 1 -- 172.21.15.46:0/2149568487 shutdown_connections 2024-03-18T20:37:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:29.422+0000 7f8e63a1f700 1 -- 172.21.15.46:0/2149568487 wait complete. 2024-03-18T20:37:29.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42758 10.38 MB/sec warmup 24 sec latency 4970.355 ms 2024-03-18T20:37:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:30 smithi073 ceph-mon[47384]: pgmap v45: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.0 MiB/s wr, 240 op/s 2024-03-18T20:37:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:30 smithi155 ceph-mon[48344]: pgmap v45: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.0 MiB/s wr, 240 op/s 2024-03-18T20:37:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:30 smithi046 ceph-mon[43495]: pgmap v45: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.0 MiB/s wr, 240 op/s 2024-03-18T20:37:30.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42758 9.97 MB/sec warmup 25 sec latency 1417.404 ms 2024-03-18T20:37:31.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42839 9.62 MB/sec warmup 26 sec latency 2326.125 ms 2024-03-18T20:37:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:32 smithi073 ceph-mon[47384]: pgmap v46: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 345 KiB/s wr, 48 op/s 2024-03-18T20:37:32.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:32 smithi155 ceph-mon[48344]: pgmap v46: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 345 KiB/s wr, 48 op/s 2024-03-18T20:37:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:32 smithi046 ceph-mon[43495]: pgmap v46: 97 pgs: 97 active+clean; 13 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 345 KiB/s wr, 48 op/s 2024-03-18T20:37:32.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42974 9.28 MB/sec warmup 27 sec latency 424.493 ms 2024-03-18T20:37:33.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42974 8.94 MB/sec warmup 28 sec latency 1424.595 ms 2024-03-18T20:37:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:34 smithi073 ceph-mon[47384]: pgmap v47: 97 pgs: 97 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 4.9 MiB/s wr, 196 op/s 2024-03-18T20:37:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:34 smithi155 ceph-mon[48344]: pgmap v47: 97 pgs: 97 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 4.9 MiB/s wr, 196 op/s 2024-03-18T20:37:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:34 smithi046 ceph-mon[43495]: pgmap v47: 97 pgs: 97 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 4.9 MiB/s wr, 196 op/s 2024-03-18T20:37:34.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42974 8.64 MB/sec warmup 29 sec latency 2424.682 ms 2024-03-18T20:37:35.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:37:35.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:37:35.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.216+0000 7fb2eab0f700 1 -- 172.21.15.46:0/687112412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e4101900 msgr2=0x7fb2e4105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.217+0000 7fb2eab0f700 1 --2- 172.21.15.46:0/687112412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e4101900 0x7fb2e4105940 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009990 tx=0x7fb2d0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.217+0000 7fb2eab0f700 1 -- 172.21.15.46:0/687112412 shutdown_connections 2024-03-18T20:37:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.217+0000 7fb2eab0f700 1 --2- 172.21.15.46:0/687112412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2e4106070 0x7fb2e410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.217+0000 7fb2eab0f700 1 --2- 172.21.15.46:0/687112412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e4100f50 0x7fb2e4101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.217+0000 7fb2eab0f700 1 --2- 172.21.15.46:0/687112412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e4101900 0x7fb2e4105940 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.217+0000 7fb2eab0f700 1 -- 172.21.15.46:0/687112412 >> 172.21.15.46:0/687112412 conn(0x7fb2e40fc7f0 msgr2=0x7fb2e40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.217+0000 7fb2eab0f700 1 -- 172.21.15.46:0/687112412 shutdown_connections 2024-03-18T20:37:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.217+0000 7fb2eab0f700 1 -- 172.21.15.46:0/687112412 wait complete. 2024-03-18T20:37:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.217+0000 7fb2eab0f700 1 Processor -- start 2024-03-18T20:37:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.217+0000 7fb2eab0f700 1 -- start start 2024-03-18T20:37:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2eab0f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2e4100f50 0x7fb2e40730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2eab0f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e4106070 0x7fb2e40735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2eab0f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e4073b30 0x7fb2e41a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2eab0f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e41a59d0 con 0x7fb2e4106070 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2eab0f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e41a5b40 con 0x7fb2e4100f50 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2eab0f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e41a5d00 con 0x7fb2e4073b30 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e4106070 0x7fb2e40735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e4106070 0x7fb2e40735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32982/0 (socket says 172.21.15.46:32982) 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e3fff700 1 -- 172.21.15.46:0/159151971 learned_addr learned my addr 172.21.15.46:0/159151971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e90ac700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e4073b30 0x7fb2e41a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e88ab700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2e4100f50 0x7fb2e40730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e3fff700 1 -- 172.21.15.46:0/159151971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e4073b30 msgr2=0x7fb2e41a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e3fff700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e4073b30 0x7fb2e41a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e3fff700 1 -- 172.21.15.46:0/159151971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2e4100f50 msgr2=0x7fb2e40730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e3fff700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2e4100f50 0x7fb2e40730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e3fff700 1 -- 172.21.15.46:0/159151971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d00095f0 con 0x7fb2e4106070 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e3fff700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e4106070 0x7fb2e40735f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0005770 tx=0x7fb2d0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e1ffb700 1 -- 172.21.15.46:0/159151971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d001d070 con 0x7fb2e4106070 2024-03-18T20:37:35.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2e1ffb700 1 -- 172.21.15.46:0/159151971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d0021410 con 0x7fb2e4106070 2024-03-18T20:37:35.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.218+0000 7fb2eab0f700 1 -- 172.21.15.46:0/159151971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e41a5f80 con 0x7fb2e4106070 2024-03-18T20:37:35.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.219+0000 7fb2e1ffb700 1 -- 172.21.15.46:0/159151971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0017650 con 0x7fb2e4106070 2024-03-18T20:37:35.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.219+0000 7fb2eab0f700 1 -- 172.21.15.46:0/159151971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e41a63b0 con 0x7fb2e4106070 2024-03-18T20:37:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.221+0000 7fb2e1ffb700 1 -- 172.21.15.46:0/159151971 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fb2d0021580 con 0x7fb2e4106070 2024-03-18T20:37:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.222+0000 7fb2e1ffb700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb2d40a1430 0x7fb2d40a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.222+0000 7fb2e1ffb700 1 -- 172.21.15.46:0/159151971 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb2d00ce290 con 0x7fb2e4106070 2024-03-18T20:37:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.222+0000 7fb2e88ab700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb2d40a1430 0x7fb2d40a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.222+0000 7fb2e88ab700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb2d40a1430 0x7fb2d40a38f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8005950 tx=0x7fb2d80058e0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2f6ca4b80 1 Processor -- start 2024-03-18T20:37:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2f6ca4b80 1 -- start start 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2f6ca4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d8ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2f6ca4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8d380 0x55c7b8d97ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2f6ca4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8d9a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2f6ca4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b8ce6990 con 0x55c7b8d98220 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2f6ca4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b8ce4350 con 0x55c7b8d8d380 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2f6ca4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b8cdfc40 con 0x55c7b8d8ca60 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2e90ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8d9a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2e90ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8d9a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32990/0 (socket says 172.21.15.46:32990) 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2e90ac700 1 -- 172.21.15.46:0/1812204069 learned_addr learned my addr 172.21.15.46:0/1812204069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2e88ab700 1 --2- 172.21.15.46:0/1812204069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d8ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2e3fff700 1 --2- 172.21.15.46:0/1812204069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8d380 0x55c7b8d97ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2e90ac700 1 -- 172.21.15.46:0/1812204069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8ca60 msgr2=0x55c7b8d8ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2e90ac700 1 --2- 172.21.15.46:0/1812204069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d8ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2e90ac700 1 -- 172.21.15.46:0/1812204069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8d380 msgr2=0x55c7b8d97ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2e90ac700 1 --2- 172.21.15.46:0/1812204069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8d380 0x55c7b8d97ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2e90ac700 1 -- 172.21.15.46:0/1812204069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7b8b613b0 con 0x55c7b8d98220 2024-03-18T20:37:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.223+0000 7fb2e88ab700 1 --2- 172.21.15.46:0/1812204069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d8ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.224+0000 7fb2e90ac700 1 --2- 172.21.15.46:0/1812204069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8d9a610 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009440 tx=0x7fb2dc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d18257b242b22d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.224+0000 7fb2ceffd700 1 -- 172.21.15.46:0/1812204069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00e070 con 0x55c7b8d98220 2024-03-18T20:37:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.224+0000 7fb2ceffd700 1 -- 172.21.15.46:0/1812204069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2dc00f040 con 0x55c7b8d98220 2024-03-18T20:37:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.224+0000 7fb2ceffd700 1 -- 172.21.15.46:0/1812204069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc015b00 con 0x55c7b8d98220 2024-03-18T20:37:35.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.225+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1812204069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 msgr2=0x55c7b8d9a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.225+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1812204069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8d9a610 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009440 tx=0x7fb2dc009920 comp rx=0 tx=0).stop 2024-03-18T20:37:35.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.225+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1812204069 shutdown_connections 2024-03-18T20:37:35.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.225+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1812204069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8d9a610 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.225+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1812204069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d8ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.225+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1812204069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8d380 0x55c7b8d97ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.225+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1812204069 >> 172.21.15.46:0/1812204069 conn(0x55c7b8d03220 msgr2=0x55c7b8d89020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.225+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1812204069 shutdown_connections 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.225+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1812204069 wait complete. 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.225+0000 7fb2f6ca4b80 1 -- start start 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.225+0000 7fb2f6ca4b80 1 Processor -- start 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2f6ca4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d97be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2f6ca4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8d380 0x55c7b8d94f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2f6ca4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8d955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2f6ca4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b8c7bd80 con 0x55c7b8d98220 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2f6ca4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b8c7ba20 con 0x55c7b8d8ca60 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2f6ca4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b8b636a0 con 0x55c7b8d8d380 2024-03-18T20:37:35.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2f6ca4b80 10 client.0 _ll_register_callbacks cb 0x55c7b8ae73a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2e90ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8d955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2e90ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8d955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33002/0 (socket says 172.21.15.46:33002) 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2e90ac700 1 -- 172.21.15.46:0/1994334955 learned_addr learned my addr 172.21.15.46:0/1994334955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2e88ab700 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d97be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2e3fff700 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8d380 0x55c7b8d94f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2e90ac700 1 -- 172.21.15.46:0/1994334955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8d380 msgr2=0x55c7b8d94f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2e90ac700 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8d380 0x55c7b8d94f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2e90ac700 1 -- 172.21.15.46:0/1994334955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8ca60 msgr2=0x55c7b8d97be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2e90ac700 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d97be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2e90ac700 1 -- 172.21.15.46:0/1994334955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc0090f0 con 0x55c7b8d98220 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2e90ac700 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8d955a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc016f00 tx=0x7fb2dc016f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c037d765236047da server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.226+0000 7fb2c3fff700 1 -- 172.21.15.46:0/1994334955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00e040 con 0x55c7b8d98220 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2c3fff700 1 -- 172.21.15.46:0/1994334955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb2dc004ce0 con 0x55c7b8d98220 2024-03-18T20:37:35.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1994334955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7b8d80ad0 con 0x55c7b8d98220 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2c3fff700 1 -- 172.21.15.46:0/1994334955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2dc004ea0 con 0x55c7b8d98220 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2f6ca4b80 10 client.15990 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1994334955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c7b8b0db60 con 0x55c7b8d98220 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2c3fff700 1 -- 172.21.15.46:0/1994334955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c7b8d80ad0 con 0x55c7b8d98220 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2c3fff700 1 -- 172.21.15.46:0/1994334955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb2dc01c070 con 0x55c7b8d98220 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2f6ca4b80 10 client.15990 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2f6ca4b80 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2f6ca4b80 20 client.15990 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c7b8da7ce0 0x55c7b8daa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2f6ca4b80 4 client.15990 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1994334955 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7b8cd8700 con 0x55c7b8da7ce0 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.227+0000 7fb2e88ab700 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c7b8da7ce0 0x55c7b8daa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.228+0000 7fb2c3fff700 10 client.15990 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:35.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.228+0000 7fb2e88ab700 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c7b8da7ce0 0x55c7b8daa100 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d3dfa922b24cc17 server_cookie=35c106d6504c77f1 in_seq=0 out_seq=0 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.228+0000 7fb2c3fff700 1 -- 172.21.15.46:0/1994334955 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c7b8cd8700 con 0x55c7b8da7ce0 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.228+0000 7fb2c3fff700 10 client.15990 handle_command_reply: tid=0 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.228+0000 7fb2f6ca4b80 1 client.15990 shutdown 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.229+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1994334955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 msgr2=0x55c7b8d955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.229+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8d955a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc016f00 tx=0x7fb2dc016f30 comp rx=0 tx=0).stop 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.229+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1994334955 shutdown_connections 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.229+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c7b8da7ce0 0x55c7b8daa100 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.229+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8d955a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.229+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d97be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.229+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1994334955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8d380 0x55c7b8d94f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.229+0000 7fb2c3fff700 0 client.15990 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.229+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1994334955 >> 172.21.15.46:0/1994334955 conn(0x55c7b8d03220 msgr2=0x55c7b8d88c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.229+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1994334955 shutdown_connections 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.229+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1994334955 wait complete. 2024-03-18T20:37:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.229+0000 7fb2f6ca4b80 20 client.15990 trim_cache size 0 max 16384 2024-03-18T20:37:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2f6ca4b80 1 Processor -- start 2024-03-18T20:37:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2f6ca4b80 1 -- start start 2024-03-18T20:37:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2f6ca4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d954b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2f6ca4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8d380 0x55c7b8da35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2f6ca4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8da3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2f6ca4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b8d95e70 con 0x55c7b8d98220 2024-03-18T20:37:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2f6ca4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b8d8dd30 con 0x55c7b8d8d380 2024-03-18T20:37:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2f6ca4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b8d82f80 con 0x55c7b8d8ca60 2024-03-18T20:37:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2e90ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8da3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2e90ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8da3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38132/0 (socket says 172.21.15.46:38132) 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2e90ac700 1 -- 172.21.15.46:0/3976218677 learned_addr learned my addr 172.21.15.46:0/3976218677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2e88ab700 1 --2- 172.21.15.46:0/3976218677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2e3fff700 1 --2- 172.21.15.46:0/3976218677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8d380 0x55c7b8da35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2e90ac700 1 -- 172.21.15.46:0/3976218677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8ca60 msgr2=0x55c7b8d954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2e90ac700 1 --2- 172.21.15.46:0/3976218677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2e90ac700 1 -- 172.21.15.46:0/3976218677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8d380 msgr2=0x55c7b8da35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2e90ac700 1 --2- 172.21.15.46:0/3976218677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8d380 0x55c7b8da35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.247+0000 7fb2e90ac700 1 -- 172.21.15.46:0/3976218677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7b8ca5ea0 con 0x55c7b8d98220 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.248+0000 7fb2e90ac700 1 --2- 172.21.15.46:0/3976218677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8da3ae0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc004a20 tx=0x7fb2dc0048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a31116a6e2c415 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.248+0000 7fb2cf7fe700 1 -- 172.21.15.46:0/3976218677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc013550 con 0x55c7b8d98220 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.248+0000 7fb2cf7fe700 1 -- 172.21.15.46:0/3976218677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2dc013760 con 0x55c7b8d98220 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.248+0000 7fb2cf7fe700 1 -- 172.21.15.46:0/3976218677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc015500 con 0x55c7b8d98220 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.248+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/3976218677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 msgr2=0x55c7b8da3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.248+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/3976218677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8da3ae0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc004a20 tx=0x7fb2dc0048d0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.248+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/3976218677 shutdown_connections 2024-03-18T20:37:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.248+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/3976218677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d98220 0x55c7b8da3ae0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.248+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/3976218677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.248+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/3976218677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d8d380 0x55c7b8da35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.248+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/3976218677 >> 172.21.15.46:0/3976218677 conn(0x55c7b8d03220 msgr2=0x55c7b8d93830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.248+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/3976218677 shutdown_connections 2024-03-18T20:37:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.249+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/3976218677 wait complete. 2024-03-18T20:37:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.249+0000 7fb2f6ca4b80 1 -- start start 2024-03-18T20:37:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.249+0000 7fb2f6ca4b80 1 Processor -- start 2024-03-18T20:37:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.249+0000 7fb2f6ca4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d9d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.249+0000 7fb2f6ca4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8d380 0x55c7b8d9d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.249+0000 7fb2f6ca4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d98220 0x55c7b8dfaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.249+0000 7fb2f6ca4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b8dfb010 con 0x55c7b8d8ca60 2024-03-18T20:37:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.249+0000 7fb2f6ca4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b8dfb150 con 0x55c7b8d98220 2024-03-18T20:37:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.249+0000 7fb2f6ca4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b8dfb290 con 0x55c7b8d8d380 2024-03-18T20:37:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2f6ca4b80 10 client.0 _ll_register_callbacks cb 0x7fb2d0005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:35.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2e88ab700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d9d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2e88ab700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d9d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38140/0 (socket says 172.21.15.46:38140) 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2e88ab700 1 -- 172.21.15.46:0/1634001965 learned_addr learned my addr 172.21.15.46:0/1634001965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2e3fff700 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8d380 0x55c7b8d9d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2e90ac700 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d98220 0x55c7b8dfaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2e88ab700 1 -- 172.21.15.46:0/1634001965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8d380 msgr2=0x55c7b8d9d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2e88ab700 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8d380 0x55c7b8d9d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2e88ab700 1 -- 172.21.15.46:0/1634001965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d98220 msgr2=0x55c7b8dfaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2e88ab700 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d98220 0x55c7b8dfaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2e88ab700 1 -- 172.21.15.46:0/1634001965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7b8b0db60 con 0x55c7b8d8ca60 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2e88ab700 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d9d020 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8005f90 tx=0x7fb2d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29a23e220ee5d6e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2c3fff700 1 -- 172.21.15.46:0/1634001965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8025040 con 0x55c7b8d8ca60 2024-03-18T20:37:35.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2c3fff700 1 -- 172.21.15.46:0/1634001965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb2d8011850 con 0x55c7b8d8ca60 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.250+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1634001965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2dc00f040 con 0x55c7b8d8ca60 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.251+0000 7fb2c3fff700 1 -- 172.21.15.46:0/1634001965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2d800fb00 con 0x55c7b8d8ca60 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.251+0000 7fb2f6ca4b80 10 client.15996 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.251+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1634001965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c7b8b65cd0 con 0x55c7b8d8ca60 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.251+0000 7fb2c3fff700 1 -- 172.21.15.46:0/1634001965 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d803c020 con 0x55c7b8d8ca60 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.251+0000 7fb2c3fff700 1 -- 172.21.15.46:0/1634001965 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb2d801fbd0 con 0x55c7b8d8ca60 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.251+0000 7fb2f6ca4b80 10 client.15996 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.251+0000 7fb2f6ca4b80 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.251+0000 7fb2f6ca4b80 20 client.15996 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.251+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c7b8dfcb30 0x55c7b8dfef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.251+0000 7fb2f6ca4b80 4 client.15996 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.251+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1634001965 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c7b8b2ecf0 con 0x55c7b8dfcb30 2024-03-18T20:37:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.251+0000 7fb2e3fff700 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c7b8dfcb30 0x55c7b8dfef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2e3fff700 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c7b8dfcb30 0x55c7b8dfef50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb57f7a4cb3ff04 server_cookie=5de1fe58c87c77a0 in_seq=0 out_seq=0 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2c3fff700 10 client.15996 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2c3fff700 1 -- 172.21.15.46:0/1634001965 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c7b8b2ecf0 con 0x55c7b8dfcb30 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2c3fff700 10 client.15996 handle_command_reply: tid=0 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2f6ca4b80 1 client.15996 shutdown 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1634001965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d8ca60 msgr2=0x55c7b8d9d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d9d020 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8005f90 tx=0x7fb2d80047c0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1634001965 shutdown_connections 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c7b8dfcb30 0x55c7b8dfef50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7b8d8ca60 0x55c7b8d9d020 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c7b8d98220 0x55c7b8dfaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2c3fff700 0 client.15996 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2f6ca4b80 1 --2- 172.21.15.46:0/1634001965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c7b8d8d380 0x55c7b8d9d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1634001965 >> 172.21.15.46:0/1634001965 conn(0x55c7b8d03220 msgr2=0x55c7b8da35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1634001965 shutdown_connections 2024-03-18T20:37:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2f6ca4b80 1 -- 172.21.15.46:0/1634001965 wait complete. 2024-03-18T20:37:35.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.252+0000 7fb2f6ca4b80 20 client.15996 trim_cache size 0 max 16384 2024-03-18T20:37:35.262 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:37:35.262 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:37:35.262 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:37:35.262 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:37:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.253+0000 7fb2eab0f700 1 -- 172.21.15.46:0/159151971 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb2d40a1430 msgr2=0x7fb2d40a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.253+0000 7fb2eab0f700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb2d40a1430 0x7fb2d40a38f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8005950 tx=0x7fb2d80058e0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.253+0000 7fb2eab0f700 1 -- 172.21.15.46:0/159151971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e4106070 msgr2=0x7fb2e40735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.253+0000 7fb2eab0f700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e4106070 0x7fb2e40735f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0005770 tx=0x7fb2d0005ef0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.253+0000 7fb2eab0f700 1 -- 172.21.15.46:0/159151971 shutdown_connections 2024-03-18T20:37:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.253+0000 7fb2eab0f700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb2d40a1430 0x7fb2d40a38f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.253+0000 7fb2eab0f700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e4073b30 0x7fb2e41a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.253+0000 7fb2eab0f700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2e4100f50 0x7fb2e40730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.253+0000 7fb2eab0f700 1 --2- 172.21.15.46:0/159151971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e4106070 0x7fb2e40735f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.253+0000 7fb2eab0f700 1 -- 172.21.15.46:0/159151971 >> 172.21.15.46:0/159151971 conn(0x7fb2e40fc7f0 msgr2=0x7fb2e40fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.253+0000 7fb2eab0f700 1 -- 172.21.15.46:0/159151971 shutdown_connections 2024-03-18T20:37:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.253+0000 7fb2eab0f700 1 -- 172.21.15.46:0/159151971 wait complete. 2024-03-18T20:37:35.270 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:37:35.270 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:37:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.404+0000 7efedab16700 1 -- 172.21.15.46:0/3846033996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed41004b0 msgr2=0x7efed410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.404+0000 7efedab16700 1 --2- 172.21.15.46:0/3846033996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed41004b0 0x7efed410c9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efecc00b0a0 tx=0x7efecc00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.404+0000 7efedab16700 1 -- 172.21.15.46:0/3846033996 shutdown_connections 2024-03-18T20:37:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.404+0000 7efedab16700 1 --2- 172.21.15.46:0/3846033996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed41004b0 0x7efed410c9c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.404+0000 7efedab16700 1 --2- 172.21.15.46:0/3846033996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efed4105b90 0x7efed4105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.404+0000 7efedab16700 1 --2- 172.21.15.46:0/3846033996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efed40ffaf0 0x7efed40fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.404+0000 7efedab16700 1 -- 172.21.15.46:0/3846033996 >> 172.21.15.46:0/3846033996 conn(0x7efed40fb620 msgr2=0x7efed40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.404+0000 7efedab16700 1 -- 172.21.15.46:0/3846033996 shutdown_connections 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.404+0000 7efedab16700 1 -- 172.21.15.46:0/3846033996 wait complete. 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efedab16700 1 Processor -- start 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efedab16700 1 -- start start 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efedab16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed40ffaf0 0x7efed4196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efedab16700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efed41004b0 0x7efed4197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efedab16700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efed4105b90 0x7efed419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efedab16700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed419bee0 con 0x7efed40ffaf0 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efedab16700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed419c050 con 0x7efed41004b0 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efedab16700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed419c210 con 0x7efed4105b90 2024-03-18T20:37:35.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efed88b2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed40ffaf0 0x7efed4196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efed88b2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed40ffaf0 0x7efed4196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38150/0 (socket says 172.21.15.46:38150) 2024-03-18T20:37:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efed88b2700 1 -- 172.21.15.46:0/3259730049 learned_addr learned my addr 172.21.15.46:0/3259730049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:35.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efed3fff700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efed41004b0 0x7efed4197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efed90b3700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efed4105b90 0x7efed419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efed88b2700 1 -- 172.21.15.46:0/3259730049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efed4105b90 msgr2=0x7efed419b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efed88b2700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efed4105b90 0x7efed419b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efed88b2700 1 -- 172.21.15.46:0/3259730049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efed41004b0 msgr2=0x7efed4197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efed88b2700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efed41004b0 0x7efed4197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.405+0000 7efed88b2700 1 -- 172.21.15.46:0/3259730049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc009d00 con 0x7efed40ffaf0 2024-03-18T20:37:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.406+0000 7efed88b2700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed40ffaf0 0x7efed4196fc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efec800e9a0 tx=0x7efec800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.406+0000 7efed1ffb700 1 -- 172.21.15.46:0/3259730049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8009790 con 0x7efed40ffaf0 2024-03-18T20:37:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.406+0000 7efed1ffb700 1 -- 172.21.15.46:0/3259730049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efec8010430 con 0x7efed40ffaf0 2024-03-18T20:37:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.406+0000 7efedab16700 1 -- 172.21.15.46:0/3259730049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed419c4f0 con 0x7efed40ffaf0 2024-03-18T20:37:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.406+0000 7efed1ffb700 1 -- 172.21.15.46:0/3259730049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8018680 con 0x7efed40ffaf0 2024-03-18T20:37:35.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.406+0000 7efedab16700 1 -- 172.21.15.46:0/3259730049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed41a3d30 con 0x7efed40ffaf0 2024-03-18T20:37:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.407+0000 7efedab16700 1 -- 172.21.15.46:0/3259730049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed4109120 con 0x7efed40ffaf0 2024-03-18T20:37:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.408+0000 7efed1ffb700 1 -- 172.21.15.46:0/3259730049 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7efec80105a0 con 0x7efed40ffaf0 2024-03-18T20:37:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.409+0000 7efed1ffb700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efec40a1430 0x7efec40a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.409+0000 7efed1ffb700 1 -- 172.21.15.46:0/3259730049 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7efec8014070 con 0x7efed40ffaf0 2024-03-18T20:37:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.409+0000 7efed3fff700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efec40a1430 0x7efec40a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.410+0000 7efed3fff700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efec40a1430 0x7efec40a38f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efed4198510 tx=0x7efec0009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.413+0000 7efed1ffb700 1 -- 172.21.15.46:0/3259730049 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efec80964f0 con 0x7efed40ffaf0 2024-03-18T20:37:35.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42974 8.35 MB/sec warmup 30 sec latency 3424.736 ms 2024-03-18T20:37:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.677+0000 7efedab16700 1 -- 172.21.15.46:0/3259730049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efed41982c0 con 0x7efed40ffaf0 2024-03-18T20:37:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.678+0000 7efed1ffb700 1 -- 172.21.15.46:0/3259730049 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10916 (secure 0 0 0) 0x7efec8026020 con 0x7efed40ffaf0 2024-03-18T20:37:35.681 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:37:35.682 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:37:23.092096+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:37:35.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.681+0000 7efedab16700 1 -- 172.21.15.46:0/3259730049 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efec40a1430 msgr2=0x7efec40a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.681+0000 7efedab16700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efec40a1430 0x7efec40a38f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efed4198510 tx=0x7efec0009260 comp rx=0 tx=0).stop 2024-03-18T20:37:35.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.681+0000 7efedab16700 1 -- 172.21.15.46:0/3259730049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed40ffaf0 msgr2=0x7efed4196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.681+0000 7efedab16700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed40ffaf0 0x7efed4196fc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7efec800e9a0 tx=0x7efec800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.681+0000 7efedab16700 1 -- 172.21.15.46:0/3259730049 shutdown_connections 2024-03-18T20:37:35.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.681+0000 7efedab16700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efec40a1430 0x7efec40a38f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.681+0000 7efedab16700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed40ffaf0 0x7efed4196fc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.681+0000 7efedab16700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efed4105b90 0x7efed419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.681+0000 7efedab16700 1 --2- 172.21.15.46:0/3259730049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efed41004b0 0x7efed4197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.681+0000 7efedab16700 1 -- 172.21.15.46:0/3259730049 >> 172.21.15.46:0/3259730049 conn(0x7efed40fb620 msgr2=0x7efed4069350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.681+0000 7efedab16700 1 -- 172.21.15.46:0/3259730049 shutdown_connections 2024-03-18T20:37:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.681+0000 7efedab16700 1 -- 172.21.15.46:0/3259730049 wait complete. 2024-03-18T20:37:35.685 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 31 2024-03-18T20:37:35.698 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:37:35.698 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:37:35.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.836+0000 7f87afa45700 1 -- 172.21.15.46:0/423272806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a81060b0 msgr2=0x7f87a810dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.836+0000 7f87afa45700 1 --2- 172.21.15.46:0/423272806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a81060b0 0x7f87a810dc30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f87a00099c0 tx=0x7f87a0009cd0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.836+0000 7f87afa45700 1 -- 172.21.15.46:0/423272806 shutdown_connections 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.836+0000 7f87afa45700 1 --2- 172.21.15.46:0/423272806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a81060b0 0x7f87a810dc30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.836+0000 7f87afa45700 1 --2- 172.21.15.46:0/423272806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a80692c0 0x7f87a80696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.836+0000 7f87afa45700 1 --2- 172.21.15.46:0/423272806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87a8069c70 0x7f87a8105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.836+0000 7f87afa45700 1 -- 172.21.15.46:0/423272806 >> 172.21.15.46:0/423272806 conn(0x7f87a8076d80 msgr2=0x7f87a8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.837+0000 7f87afa45700 1 -- 172.21.15.46:0/423272806 shutdown_connections 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.837+0000 7f87afa45700 1 -- 172.21.15.46:0/423272806 wait complete. 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.837+0000 7f87afa45700 1 Processor -- start 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.837+0000 7f87afa45700 1 -- start start 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.837+0000 7f87afa45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a80692c0 0x7f87a81993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.837+0000 7f87afa45700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a81060b0 0x7f87a81998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.837+0000 7f87afa45700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87a819dc80 0x7f87a819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.837+0000 7f87afa45700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a819e700 con 0x7f87a80692c0 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.837+0000 7f87afa45700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a819e870 con 0x7f87a81060b0 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.837+0000 7f87afa45700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a819ea30 con 0x7f87a819dc80 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.838+0000 7f87ad7e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a80692c0 0x7f87a81993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.838+0000 7f87ad7e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a80692c0 0x7f87a81993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38172/0 (socket says 172.21.15.46:38172) 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.838+0000 7f87ad7e1700 1 -- 172.21.15.46:0/1427622528 learned_addr learned my addr 172.21.15.46:0/1427622528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.838+0000 7f87acfe0700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a81060b0 0x7f87a81998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.838+0000 7f87adfe2700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87a819dc80 0x7f87a819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.838+0000 7f87ad7e1700 1 -- 172.21.15.46:0/1427622528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87a819dc80 msgr2=0x7f87a819e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.838+0000 7f87ad7e1700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87a819dc80 0x7f87a819e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.838+0000 7f87ad7e1700 1 -- 172.21.15.46:0/1427622528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a81060b0 msgr2=0x7f87a81998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.838+0000 7f87ad7e1700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a81060b0 0x7f87a81998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.838+0000 7f87ad7e1700 1 -- 172.21.15.46:0/1427622528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a00095f0 con 0x7f87a80692c0 2024-03-18T20:37:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.838+0000 7f87ad7e1700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a80692c0 0x7f87a81993a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f87a40060b0 tx=0x7f87a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.839+0000 7f879e7fc700 1 -- 172.21.15.46:0/1427622528 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a400dd20 con 0x7f87a80692c0 2024-03-18T20:37:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.839+0000 7f879e7fc700 1 -- 172.21.15.46:0/1427622528 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87a400f040 con 0x7f87a80692c0 2024-03-18T20:37:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.839+0000 7f879e7fc700 1 -- 172.21.15.46:0/1427622528 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4014600 con 0x7f87a80692c0 2024-03-18T20:37:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.839+0000 7f87afa45700 1 -- 172.21.15.46:0/1427622528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a81a5f90 con 0x7f87a80692c0 2024-03-18T20:37:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.839+0000 7f87afa45700 1 -- 172.21.15.46:0/1427622528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a81a63c0 con 0x7f87a80692c0 2024-03-18T20:37:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.841+0000 7f879e7fc700 1 -- 172.21.15.46:0/1427622528 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f87a4015400 con 0x7f87a80692c0 2024-03-18T20:37:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.842+0000 7f879e7fc700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f87940a1430 0x7f87940a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.842+0000 7f87acfe0700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f87940a1430 0x7f87940a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.842+0000 7f879e7fc700 1 -- 172.21.15.46:0/1427622528 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f87a40cc800 con 0x7f87a80692c0 2024-03-18T20:37:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.843+0000 7f87acfe0700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f87940a1430 0x7f87940a38f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87a819a8f0 tx=0x7f87980073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.843+0000 7f87bbbdab80 1 Processor -- start 2024-03-18T20:37:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87bbbdab80 1 -- start start 2024-03-18T20:37:35.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87bbbdab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3acfa60 0x55c8b3acfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87bbbdab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3ad0380 0x55c8b3adace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87bbbdab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3adb220 0x55c8b3add610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87bbbdab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b3a29990 con 0x55c8b3ad0380 2024-03-18T20:37:35.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87bbbdab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b3a27350 con 0x55c8b3adb220 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87bbbdab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b3a22c40 con 0x55c8b3acfa60 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87acfe0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3ad0380 0x55c8b3adace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87acfe0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3ad0380 0x55c8b3adace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38184/0 (socket says 172.21.15.46:38184) 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87acfe0700 1 -- 172.21.15.46:0/1795369610 learned_addr learned my addr 172.21.15.46:0/1795369610 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87ad7e1700 1 --2- 172.21.15.46:0/1795369610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3acfa60 0x55c8b3acfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87adfe2700 1 --2- 172.21.15.46:0/1795369610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3adb220 0x55c8b3add610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87acfe0700 1 -- 172.21.15.46:0/1795369610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3acfa60 msgr2=0x55c8b3acfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87acfe0700 1 --2- 172.21.15.46:0/1795369610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3acfa60 0x55c8b3acfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87acfe0700 1 -- 172.21.15.46:0/1795369610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3adb220 msgr2=0x55c8b3add610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87acfe0700 1 --2- 172.21.15.46:0/1795369610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3adb220 0x55c8b3add610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.844+0000 7f87acfe0700 1 -- 172.21.15.46:0/1795369610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b38a43b0 con 0x55c8b3ad0380 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.845+0000 7f87acfe0700 1 --2- 172.21.15.46:0/1795369610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3ad0380 0x55c8b3adace0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8798010d30 tx=0x7f8798013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d741f765faae22c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.845+0000 7f87937fe700 1 -- 172.21.15.46:0/1795369610 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798011ad0 con 0x55c8b3ad0380 2024-03-18T20:37:35.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.845+0000 7f87937fe700 1 -- 172.21.15.46:0/1795369610 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8798015070 con 0x55c8b3ad0380 2024-03-18T20:37:35.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.845+0000 7f87937fe700 1 -- 172.21.15.46:0/1795369610 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879801c3d0 con 0x55c8b3ad0380 2024-03-18T20:37:35.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.846+0000 7f87bbbdab80 1 -- 172.21.15.46:0/1795369610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3ad0380 msgr2=0x55c8b3adace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.846+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/1795369610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3ad0380 0x55c8b3adace0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8798010d30 tx=0x7f8798013000 comp rx=0 tx=0).stop 2024-03-18T20:37:35.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.846+0000 7f87bbbdab80 1 -- 172.21.15.46:0/1795369610 shutdown_connections 2024-03-18T20:37:35.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.846+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/1795369610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3adb220 0x55c8b3add610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.846+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/1795369610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3acfa60 0x55c8b3acfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.846+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/1795369610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3ad0380 0x55c8b3adace0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.846+0000 7f87bbbdab80 1 -- 172.21.15.46:0/1795369610 >> 172.21.15.46:0/1795369610 conn(0x55c8b3a46220 msgr2=0x55c8b3acc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.846+0000 7f87bbbdab80 1 -- 172.21.15.46:0/1795369610 shutdown_connections 2024-03-18T20:37:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.846+0000 7f87bbbdab80 1 -- 172.21.15.46:0/1795369610 wait complete. 2024-03-18T20:37:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.846+0000 7f87bbbdab80 1 -- start start 2024-03-18T20:37:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.846+0000 7f87bbbdab80 1 Processor -- start 2024-03-18T20:37:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87bbbdab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3acfa60 0x55c8b3adac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87bbbdab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3ad0380 0x55c8b3ad7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87bbbdab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 0x55c8b3ad8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87bbbdab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b39bed80 con 0x55c8b3adb220 2024-03-18T20:37:35.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87bbbdab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b39bea20 con 0x55c8b3ad0380 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87bbbdab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b38a66a0 con 0x55c8b3acfa60 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87bbbdab80 10 client.0 _ll_register_callbacks cb 0x55c8b382a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87adfe2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 0x55c8b3ad8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87adfe2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 0x55c8b3ad8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38198/0 (socket says 172.21.15.46:38198) 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87adfe2700 1 -- 172.21.15.46:0/717224957 learned_addr learned my addr 172.21.15.46:0/717224957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87acfe0700 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3ad0380 0x55c8b3ad7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87ad7e1700 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3acfa60 0x55c8b3adac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87adfe2700 1 -- 172.21.15.46:0/717224957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3acfa60 msgr2=0x55c8b3adac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87adfe2700 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3acfa60 0x55c8b3adac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87adfe2700 1 -- 172.21.15.46:0/717224957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3ad0380 msgr2=0x55c8b3ad7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87adfe2700 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3ad0380 0x55c8b3ad7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.847+0000 7f87adfe2700 1 -- 172.21.15.46:0/717224957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87980109e0 con 0x55c8b3adb220 2024-03-18T20:37:35.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f87adfe2700 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 0x55c8b3ad8620 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f87a0015000 tx=0x7f87a0005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b67e6f8173b51b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f8790ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f8790ff9700 1 -- 172.21.15.46:0/717224957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a001d070 con 0x55c8b3adb220 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f8790ff9700 1 -- 172.21.15.46:0/717224957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f87a000bba0 con 0x55c8b3adb220 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f87bbbdab80 1 -- 172.21.15.46:0/717224957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8b3ac3ad0 con 0x55c8b3adb220 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f8790ff9700 1 -- 172.21.15.46:0/717224957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87a000bd60 con 0x55c8b3adb220 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f87bbbdab80 10 client.16014 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f87bbbdab80 1 -- 172.21.15.46:0/717224957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c8b3850b60 con 0x55c8b3adb220 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f8790ff9700 1 -- 172.21.15.46:0/717224957 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8b3ac3ad0 con 0x55c8b3adb220 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f8790ff9700 1 -- 172.21.15.46:0/717224957 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f87a000f4f0 con 0x55c8b3adb220 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f87bbbdab80 10 client.16014 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f87bbbdab80 10 client.16014 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.848+0000 7f87bbbdab80 20 client.16014 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.849+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c8b3aeae30 0x55c8b3aed250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.849+0000 7f87bbbdab80 4 client.16014 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.849+0000 7f87bbbdab80 1 -- 172.21.15.46:0/717224957 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8b3a1b700 con 0x55c8b3aeae30 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.849+0000 7f87ad7e1700 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c8b3aeae30 0x55c8b3aed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.849+0000 7f87ad7e1700 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c8b3aeae30 0x55c8b3aed250 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=77f5175f97d06365 server_cookie=ef6d2cc21379fdce in_seq=0 out_seq=0 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.849+0000 7f8790ff9700 10 client.16014 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f8790ff9700 1 -- 172.21.15.46:0/717224957 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c8b3a1b700 con 0x55c8b3aeae30 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f8790ff9700 10 client.16014 handle_command_reply: tid=0 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f87bbbdab80 1 client.16014 shutdown 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f87bbbdab80 1 -- 172.21.15.46:0/717224957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 msgr2=0x55c8b3ad8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 0x55c8b3ad8620 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f87a0015000 tx=0x7f87a0005f00 comp rx=0 tx=0).stop 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f87bbbdab80 1 -- 172.21.15.46:0/717224957 shutdown_connections 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c8b3aeae30 0x55c8b3aed250 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 0x55c8b3ad8620 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3acfa60 0x55c8b3adac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/717224957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3ad0380 0x55c8b3ad7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f8790ff9700 0 client.16014 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:37:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f87bbbdab80 1 -- 172.21.15.46:0/717224957 >> 172.21.15.46:0/717224957 conn(0x55c8b3a46220 msgr2=0x55c8b3acbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f87bbbdab80 1 -- 172.21.15.46:0/717224957 shutdown_connections 2024-03-18T20:37:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f87bbbdab80 1 -- 172.21.15.46:0/717224957 wait complete. 2024-03-18T20:37:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.850+0000 7f87bbbdab80 20 client.16014 trim_cache size 0 max 16384 2024-03-18T20:37:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.864+0000 7f87bbbdab80 1 Processor -- start 2024-03-18T20:37:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.864+0000 7f87bbbdab80 1 -- start start 2024-03-18T20:37:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.864+0000 7f87bbbdab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3acfa60 0x55c8b3ad84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.864+0000 7f87bbbdab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3ad0380 0x55c8b3ae1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.864+0000 7f87bbbdab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 0x55c8b3acbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.864+0000 7f87bbbdab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b3a1b700 con 0x55c8b3adb220 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.864+0000 7f87bbbdab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b3acd460 con 0x55c8b3acfa60 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.864+0000 7f87bbbdab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b3aeac60 con 0x55c8b3ad0380 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f87adfe2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 0x55c8b3acbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f87acfe0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3ad0380 0x55c8b3ae1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f87adfe2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 0x55c8b3acbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38210/0 (socket says 172.21.15.46:38210) 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f87adfe2700 1 -- 172.21.15.46:0/2608293264 learned_addr learned my addr 172.21.15.46:0/2608293264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f87acfe0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3ad0380 0x55c8b3ae1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:56368/0 (socket says 172.21.15.46:56368) 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f87ad7e1700 1 --2- 172.21.15.46:0/2608293264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3acfa60 0x55c8b3ad84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f87adfe2700 1 -- 172.21.15.46:0/2608293264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3ad0380 msgr2=0x55c8b3ae1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f87adfe2700 1 --2- 172.21.15.46:0/2608293264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3ad0380 0x55c8b3ae1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f87adfe2700 1 -- 172.21.15.46:0/2608293264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3acfa60 msgr2=0x55c8b3ad84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f87adfe2700 1 --2- 172.21.15.46:0/2608293264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3acfa60 0x55c8b3ad84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f87adfe2700 1 -- 172.21.15.46:0/2608293264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8b3af1e80 con 0x55c8b3adb220 2024-03-18T20:37:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f87adfe2700 1 --2- 172.21.15.46:0/2608293264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 0x55c8b3acbb50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009f90 tx=0x7f87a0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c0645130f4c934 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f8793fff700 1 -- 172.21.15.46:0/2608293264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a001d070 con 0x55c8b3adb220 2024-03-18T20:37:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f8793fff700 1 -- 172.21.15.46:0/2608293264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87a001e040 con 0x55c8b3adb220 2024-03-18T20:37:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.865+0000 7f8793fff700 1 -- 172.21.15.46:0/2608293264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0004760 con 0x55c8b3adb220 2024-03-18T20:37:35.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.866+0000 7f87bbbdab80 1 -- 172.21.15.46:0/2608293264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 msgr2=0x55c8b3acbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.866+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/2608293264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 0x55c8b3acbb50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009f90 tx=0x7f87a0009cd0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.866+0000 7f87bbbdab80 1 -- 172.21.15.46:0/2608293264 shutdown_connections 2024-03-18T20:37:35.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.866+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/2608293264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3adb220 0x55c8b3acbb50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.866+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/2608293264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3acfa60 0x55c8b3ad84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.866+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/2608293264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3ad0380 0x55c8b3ae1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.866+0000 7f87bbbdab80 1 -- 172.21.15.46:0/2608293264 >> 172.21.15.46:0/2608293264 conn(0x55c8b3a46220 msgr2=0x55c8b3ad32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.866+0000 7f87bbbdab80 1 -- 172.21.15.46:0/2608293264 shutdown_connections 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.866+0000 7f87bbbdab80 1 -- 172.21.15.46:0/2608293264 wait complete. 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.866+0000 7f87bbbdab80 1 -- start start 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.866+0000 7f87bbbdab80 1 Processor -- start 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87bbbdab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3acfa60 0x55c8b3adfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87bbbdab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3ad0380 0x55c8b3ae0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87bbbdab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3adb220 0x55c8b3ae08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87bbbdab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b3b3dc60 con 0x55c8b3acfa60 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87bbbdab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b3b3dda0 con 0x55c8b3ad0380 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87bbbdab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8b3b3dee0 con 0x55c8b3adb220 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87bbbdab80 10 client.0 _ll_register_callbacks cb 0x55c8b382a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87ad7e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3acfa60 0x55c8b3adfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87ad7e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3acfa60 0x55c8b3adfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38224/0 (socket says 172.21.15.46:38224) 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87ad7e1700 1 -- 172.21.15.46:0/3463127087 learned_addr learned my addr 172.21.15.46:0/3463127087 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87acfe0700 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3ad0380 0x55c8b3ae0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87adfe2700 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3adb220 0x55c8b3ae08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87ad7e1700 1 -- 172.21.15.46:0/3463127087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3adb220 msgr2=0x55c8b3ae08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87ad7e1700 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3adb220 0x55c8b3ae08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87ad7e1700 1 -- 172.21.15.46:0/3463127087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3ad0380 msgr2=0x55c8b3ae0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87ad7e1700 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3ad0380 0x55c8b3ae0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87ad7e1700 1 -- 172.21.15.46:0/3463127087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8b3850b60 con 0x55c8b3acfa60 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87adfe2700 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3adb220 0x55c8b3ae08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f87ad7e1700 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3acfa60 0x55c8b3adfe30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f87a40c9e40 tx=0x7f87a40cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f2aff0f42f84c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.867+0000 7f8790ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f8790ff9700 1 -- 172.21.15.46:0/3463127087 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40a1b50 con 0x55c8b3acfa60 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f8790ff9700 1 -- 172.21.15.46:0/3463127087 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f87a4097a70 con 0x55c8b3acfa60 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f87bbbdab80 1 -- 172.21.15.46:0/3463127087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8798015070 con 0x55c8b3acfa60 2024-03-18T20:37:35.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f8790ff9700 1 -- 172.21.15.46:0/3463127087 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87a409d120 con 0x55c8b3acfa60 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f87bbbdab80 10 client.16020 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f87bbbdab80 1 -- 172.21.15.46:0/3463127087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c8b39e8ea0 con 0x55c8b3acfa60 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f8790ff9700 1 -- 172.21.15.46:0/3463127087 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8798015070 con 0x55c8b3acfa60 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f8790ff9700 1 -- 172.21.15.46:0/3463127087 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f87a407e020 con 0x55c8b3acfa60 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f87bbbdab80 10 client.16020 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f87bbbdab80 10 client.16020 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f87bbbdab80 20 client.16020 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c8b3b3f5a0 0x55c8b3b419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f87bbbdab80 4 client.16020 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.868+0000 7f87bbbdab80 1 -- 172.21.15.46:0/3463127087 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c8b3871cf0 con 0x55c8b3b3f5a0 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87acfe0700 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c8b3b3f5a0 0x55c8b3b419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f8790ff9700 10 client.16020 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:37:35.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87acfe0700 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c8b3b3f5a0 0x55c8b3b419c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a2fd06389768a5e server_cookie=d526f2128b24dbfa in_seq=0 out_seq=0 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f8790ff9700 1 -- 172.21.15.46:0/3463127087 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c8b3871cf0 con 0x55c8b3b3f5a0 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f8790ff9700 10 client.16020 handle_command_reply: tid=0 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87bbbdab80 1 client.16020 shutdown 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87bbbdab80 1 -- 172.21.15.46:0/3463127087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3acfa60 msgr2=0x55c8b3adfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3acfa60 0x55c8b3adfe30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f87a40c9e40 tx=0x7f87a40cc040 comp rx=0 tx=0).stop 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87bbbdab80 1 -- 172.21.15.46:0/3463127087 shutdown_connections 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c8b3b3f5a0 0x55c8b3b419c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8b3acfa60 0x55c8b3adfe30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c8b3adb220 0x55c8b3ae08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f8790ff9700 0 client.16020 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87bbbdab80 1 --2- 172.21.15.46:0/3463127087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8b3ad0380 0x55c8b3ae0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87bbbdab80 1 -- 172.21.15.46:0/3463127087 >> 172.21.15.46:0/3463127087 conn(0x55c8b3a46220 msgr2=0x55c8b3acbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87bbbdab80 1 -- 172.21.15.46:0/3463127087 shutdown_connections 2024-03-18T20:37:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.869+0000 7f87bbbdab80 1 -- 172.21.15.46:0/3463127087 wait complete. 2024-03-18T20:37:35.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87bbbdab80 20 client.16020 trim_cache size 0 max 16384 2024-03-18T20:37:35.879 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:37:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87afa45700 1 -- 172.21.15.46:0/1427622528 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f87940a1430 msgr2=0x7f87940a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87afa45700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f87940a1430 0x7f87940a38f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87a819a8f0 tx=0x7f87980073d0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87afa45700 1 -- 172.21.15.46:0/1427622528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a80692c0 msgr2=0x7f87a81993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87afa45700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a80692c0 0x7f87a81993a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f87a40060b0 tx=0x7f87a400d3b0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87afa45700 1 -- 172.21.15.46:0/1427622528 shutdown_connections 2024-03-18T20:37:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87afa45700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f87940a1430 0x7f87940a38f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87afa45700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a80692c0 0x7f87a81993a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87afa45700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87a819dc80 0x7f87a819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87afa45700 1 --2- 172.21.15.46:0/1427622528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a81060b0 0x7f87a81998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87afa45700 1 -- 172.21.15.46:0/1427622528 >> 172.21.15.46:0/1427622528 conn(0x7f87a8076d80 msgr2=0x7f87a80fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:35.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87afa45700 1 -- 172.21.15.46:0/1427622528 shutdown_connections 2024-03-18T20:37:35.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:35.870+0000 7f87afa45700 1 -- 172.21.15.46:0/1427622528 wait complete. 2024-03-18T20:37:35.886 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:37:35.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:37:36.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.025+0000 7f63d0ab7700 1 -- 172.21.15.46:0/1969058606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63cc0ffe40 msgr2=0x7f63cc100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.025+0000 7f63d0ab7700 1 --2- 172.21.15.46:0/1969058606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63cc0ffe40 0x7f63cc100220 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009990 tx=0x7f63bc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.025+0000 7f63d0ab7700 1 -- 172.21.15.46:0/1969058606 shutdown_connections 2024-03-18T20:37:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.025+0000 7f63d0ab7700 1 --2- 172.21.15.46:0/1969058606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63cc0ffe40 0x7f63cc100220 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.025+0000 7f63d0ab7700 1 --2- 172.21.15.46:0/1969058606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63cc104f60 0x7f63cc10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.025+0000 7f63d0ab7700 1 --2- 172.21.15.46:0/1969058606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63cc1007f0 0x7f63cc104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.025+0000 7f63d0ab7700 1 -- 172.21.15.46:0/1969058606 >> 172.21.15.46:0/1969058606 conn(0x7f63cc076de0 msgr2=0x7f63cc0771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.025+0000 7f63d0ab7700 1 -- 172.21.15.46:0/1969058606 shutdown_connections 2024-03-18T20:37:36.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.025+0000 7f63d0ab7700 1 -- 172.21.15.46:0/1969058606 wait complete. 2024-03-18T20:37:36.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63d0ab7700 1 Processor -- start 2024-03-18T20:37:36.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63d0ab7700 1 -- start start 2024-03-18T20:37:36.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63d0ab7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63cc0ffe40 0x7f63cc1018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63d0ab7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63cc104f60 0x7f63cc101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63d0ab7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63cc102350 0x7f63cc1a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63d0ab7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc1a59e0 con 0x7f63cc0ffe40 2024-03-18T20:37:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63d0ab7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc1a5b50 con 0x7f63cc104f60 2024-03-18T20:37:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63d0ab7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc1a5d10 con 0x7f63cc102350 2024-03-18T20:37:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63ca59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63cc0ffe40 0x7f63cc1018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63ca59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63cc0ffe40 0x7f63cc1018d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38238/0 (socket says 172.21.15.46:38238) 2024-03-18T20:37:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63ca59c700 1 -- 172.21.15.46:0/254038161 learned_addr learned my addr 172.21.15.46:0/254038161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63c9d9b700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63cc104f60 0x7f63cc101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.026+0000 7f63cad9d700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63cc102350 0x7f63cc1a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.027+0000 7f63ca59c700 1 -- 172.21.15.46:0/254038161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63cc102350 msgr2=0x7f63cc1a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.027+0000 7f63ca59c700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63cc102350 0x7f63cc1a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.027+0000 7f63ca59c700 1 -- 172.21.15.46:0/254038161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63cc104f60 msgr2=0x7f63cc101e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.027+0000 7f63ca59c700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63cc104f60 0x7f63cc101e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.027+0000 7f63ca59c700 1 -- 172.21.15.46:0/254038161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc0095f0 con 0x7f63cc0ffe40 2024-03-18T20:37:36.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.027+0000 7f63cad9d700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63cc102350 0x7f63cc1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:36.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.027+0000 7f63ca59c700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63cc0ffe40 0x7f63cc1018d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f63bc004760 tx=0x7f63bc004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.027+0000 7f63c37fe700 1 -- 172.21.15.46:0/254038161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc01d070 con 0x7f63cc0ffe40 2024-03-18T20:37:36.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.027+0000 7f63c37fe700 1 -- 172.21.15.46:0/254038161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63bc00fbf0 con 0x7f63cc0ffe40 2024-03-18T20:37:36.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.027+0000 7f63d0ab7700 1 -- 172.21.15.46:0/254038161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc1a5f90 con 0x7f63cc0ffe40 2024-03-18T20:37:36.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.027+0000 7f63c37fe700 1 -- 172.21.15.46:0/254038161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc022d50 con 0x7f63cc0ffe40 2024-03-18T20:37:36.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.027+0000 7f63d0ab7700 1 -- 172.21.15.46:0/254038161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc1a6350 con 0x7f63cc0ffe40 2024-03-18T20:37:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.029+0000 7f63c37fe700 1 -- 172.21.15.46:0/254038161 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f63bc022470 con 0x7f63cc0ffe40 2024-03-18T20:37:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.030+0000 7f63c37fe700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f63b80a13b0 0x7f63b80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.030+0000 7f63c37fe700 1 -- 172.21.15.46:0/254038161 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f63bc0ce090 con 0x7f63cc0ffe40 2024-03-18T20:37:36.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.030+0000 7f63c9d9b700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f63b80a13b0 0x7f63b80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.031+0000 7f63c9d9b700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f63b80a13b0 0x7f63b80a3870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63cc102fb0 tx=0x7f63b400b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.031+0000 7f63dcc4cb80 1 Processor -- start 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.031+0000 7f63dcc4cb80 1 -- start start 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.031+0000 7f63dcc4cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9526a60 0x5563c9526e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.031+0000 7f63dcc4cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c9531ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.031+0000 7f63dcc4cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c9534610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.031+0000 7f63dcc4cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9480990 con 0x5563c9532220 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.031+0000 7f63dcc4cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c947e350 con 0x5563c9527380 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.031+0000 7f63dcc4cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9479c40 con 0x5563c9526a60 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63cad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c9534610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63cad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c9534610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38250/0 (socket says 172.21.15.46:38250) 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63c9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c9531ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63cad9d700 1 -- 172.21.15.46:0/3992353257 learned_addr learned my addr 172.21.15.46:0/3992353257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63ca59c700 1 --2- 172.21.15.46:0/3992353257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9526a60 0x5563c9526e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63cad9d700 1 -- 172.21.15.46:0/3992353257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9526a60 msgr2=0x5563c9526e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63cad9d700 1 --2- 172.21.15.46:0/3992353257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9526a60 0x5563c9526e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63cad9d700 1 -- 172.21.15.46:0/3992353257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 msgr2=0x5563c9531ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63cad9d700 1 --2- 172.21.15.46:0/3992353257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c9531ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63cad9d700 1 -- 172.21.15.46:0/3992353257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563c92fb3b0 con 0x5563c9532220 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63c9d9b700 1 --2- 172.21.15.46:0/3992353257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c9531ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63cad9d700 1 --2- 172.21.15.46:0/3992353257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c9534610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009dd0 tx=0x7f63c40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2981d0d56d8c0ba server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63c0ff9700 1 -- 172.21.15.46:0/3992353257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4018070 con 0x5563c9532220 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63c0ff9700 1 -- 172.21.15.46:0/3992353257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63c400f050 con 0x5563c9532220 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.032+0000 7f63c0ff9700 1 -- 172.21.15.46:0/3992353257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4005a90 con 0x5563c9532220 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.033+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/3992353257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 msgr2=0x5563c9534610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.033+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/3992353257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c9534610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009dd0 tx=0x7f63c40077e0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.033+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/3992353257 shutdown_connections 2024-03-18T20:37:36.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.033+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/3992353257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c9534610 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.033+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/3992353257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9526a60 0x5563c9526e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.033+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/3992353257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c9531ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.033+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/3992353257 >> 172.21.15.46:0/3992353257 conn(0x5563c949d220 msgr2=0x5563c9523020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.033+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/3992353257 shutdown_connections 2024-03-18T20:37:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.033+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/3992353257 wait complete. 2024-03-18T20:37:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63dcc4cb80 1 -- start start 2024-03-18T20:37:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63dcc4cb80 1 Processor -- start 2024-03-18T20:37:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63dcc4cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9526a60 0x5563c9531c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63dcc4cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c952f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63dcc4cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c952f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63dcc4cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9415d80 con 0x5563c9532220 2024-03-18T20:37:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63dcc4cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9415a20 con 0x5563c9527380 2024-03-18T20:37:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63dcc4cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c92fd6a0 con 0x5563c9526a60 2024-03-18T20:37:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63dcc4cb80 10 client.0 _ll_register_callbacks cb 0x5563c92813a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63cad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c952f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63cad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c952f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38254/0 (socket says 172.21.15.46:38254) 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63cad9d700 1 -- 172.21.15.46:0/1584403698 learned_addr learned my addr 172.21.15.46:0/1584403698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63ca59c700 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9526a60 0x5563c9531c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63c9d9b700 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c952f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63cad9d700 1 -- 172.21.15.46:0/1584403698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9526a60 msgr2=0x5563c9531c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63cad9d700 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9526a60 0x5563c9531c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63cad9d700 1 -- 172.21.15.46:0/1584403698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 msgr2=0x5563c952f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63cad9d700 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c952f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63cad9d700 1 -- 172.21.15.46:0/1584403698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b4010c20 con 0x5563c9532220 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.034+0000 7f63ca59c700 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9526a60 0x5563c9531c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63cad9d700 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c952f740 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f63c40059e0 tx=0x7f63c4005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae01e19cc3010c6a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63b1ffb700 1 -- 172.21.15.46:0/1584403698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4018030 con 0x5563c9532220 2024-03-18T20:37:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63b1ffb700 1 -- 172.21.15.46:0/1584403698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f63c4005350 con 0x5563c9532220 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/1584403698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563c951aad0 con 0x5563c9532220 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63b1ffb700 1 -- 172.21.15.46:0/1584403698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63c401e450 con 0x5563c9532220 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63dcc4cb80 10 client.16032 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/1584403698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f63c4009a00 con 0x5563c9532220 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63b1ffb700 1 -- 172.21.15.46:0/1584403698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5563c951aad0 con 0x5563c9532220 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63b1ffb700 1 -- 172.21.15.46:0/1584403698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f63c40145b0 con 0x5563c9532220 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63dcc4cb80 10 client.16032 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63dcc4cb80 10 client.16032 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63dcc4cb80 20 client.16032 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5563c9541eb0 0x5563c95442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63dcc4cb80 4 client.16032 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.035+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/1584403698 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563c9472700 con 0x5563c9541eb0 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.036+0000 7f63ca59c700 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5563c9541eb0 0x5563c95442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.036+0000 7f63b1ffb700 10 client.16032 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.036+0000 7f63ca59c700 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5563c9541eb0 0x5563c95442d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1f0d66425533f83a server_cookie=6c226434db3a0f3 in_seq=0 out_seq=0 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63b1ffb700 1 -- 172.21.15.46:0/1584403698 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5563c9472700 con 0x5563c9541eb0 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63b1ffb700 10 client.16032 handle_command_reply: tid=0 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63dcc4cb80 1 client.16032 shutdown 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/1584403698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 msgr2=0x5563c952f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c952f740 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f63c40059e0 tx=0x7f63c4005ac0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/1584403698 shutdown_connections 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5563c9541eb0 0x5563c95442d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c952f740 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9526a60 0x5563c9531c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63b1ffb700 0 client.16032 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/1584403698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c952f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/1584403698 >> 172.21.15.46:0/1584403698 conn(0x5563c949d220 msgr2=0x5563c9522c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.037+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/1584403698 shutdown_connections 2024-03-18T20:37:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.038+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/1584403698 wait complete. 2024-03-18T20:37:36.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.038+0000 7f63dcc4cb80 20 client.16032 trim_cache size 0 max 16384 2024-03-18T20:37:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:35 smithi046 ceph-mon[43495]: pgmap v48: 97 pgs: 97 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.6 MiB/s wr, 158 op/s 2024-03-18T20:37:36.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:35 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3259730049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:37:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63dcc4cb80 1 Processor -- start 2024-03-18T20:37:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63dcc4cb80 1 -- start start 2024-03-18T20:37:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63dcc4cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9526a60 0x5563c9529a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63dcc4cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9527380 0x5563c952f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63dcc4cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c9538e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63dcc4cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c95303a0 con 0x5563c9532220 2024-03-18T20:37:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63dcc4cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9530580 con 0x5563c9526a60 2024-03-18T20:37:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63dcc4cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c951f5e0 con 0x5563c9527380 2024-03-18T20:37:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63cad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c9538e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63ca59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9526a60 0x5563c9529a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63cad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c9538e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38262/0 (socket says 172.21.15.46:38262) 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63cad9d700 1 -- 172.21.15.46:0/864649035 learned_addr learned my addr 172.21.15.46:0/864649035 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63c9d9b700 1 --2- 172.21.15.46:0/864649035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9527380 0x5563c952f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63cad9d700 1 -- 172.21.15.46:0/864649035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9527380 msgr2=0x5563c952f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63cad9d700 1 --2- 172.21.15.46:0/864649035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9527380 0x5563c952f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63cad9d700 1 -- 172.21.15.46:0/864649035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9526a60 msgr2=0x5563c9529a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63cad9d700 1 --2- 172.21.15.46:0/864649035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9526a60 0x5563c9529a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63cad9d700 1 -- 172.21.15.46:0/864649035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563c92a7b60 con 0x5563c9532220 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.052+0000 7f63ca59c700 1 --2- 172.21.15.46:0/864649035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9526a60 0x5563c9529a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.053+0000 7f63cad9d700 1 --2- 172.21.15.46:0/864649035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c9538e80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f63c40098f0 tx=0x7f63c4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7fa247acf136b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.053+0000 7f63c17fa700 1 -- 172.21.15.46:0/864649035 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4018070 con 0x5563c9532220 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.053+0000 7f63c17fa700 1 -- 172.21.15.46:0/864649035 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63c4019040 con 0x5563c9532220 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.053+0000 7f63c17fa700 1 -- 172.21.15.46:0/864649035 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40277a0 con 0x5563c9532220 2024-03-18T20:37:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.053+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/864649035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 msgr2=0x5563c9538e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.053+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/864649035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c9538e80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f63c40098f0 tx=0x7f63c4009920 comp rx=0 tx=0).stop 2024-03-18T20:37:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.053+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/864649035 shutdown_connections 2024-03-18T20:37:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.053+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/864649035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9532220 0x5563c9538e80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.053+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/864649035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9526a60 0x5563c9529a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.053+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/864649035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9527380 0x5563c952f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.053+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/864649035 >> 172.21.15.46:0/864649035 conn(0x5563c949d220 msgr2=0x5563c9521750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/864649035 shutdown_connections 2024-03-18T20:37:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/864649035 wait complete. 2024-03-18T20:37:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63dcc4cb80 1 -- start start 2024-03-18T20:37:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63dcc4cb80 1 Processor -- start 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63dcc4cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9526a60 0x5563c9539070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63dcc4cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c9536e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63dcc4cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9532220 0x5563c95375a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63dcc4cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9537c80 con 0x5563c9526a60 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63dcc4cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9594ca0 con 0x5563c9527380 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63dcc4cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c9594de0 con 0x5563c9532220 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63dcc4cb80 10 client.0 _ll_register_callbacks cb 0x7f63b400b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63ca59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9526a60 0x5563c9539070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63ca59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9526a60 0x5563c9539070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38276/0 (socket says 172.21.15.46:38276) 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.054+0000 7f63ca59c700 1 -- 172.21.15.46:0/2406784443 learned_addr learned my addr 172.21.15.46:0/2406784443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63cad9d700 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9532220 0x5563c95375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63c9d9b700 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c9536e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63ca59c700 1 -- 172.21.15.46:0/2406784443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9532220 msgr2=0x5563c95375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63ca59c700 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9532220 0x5563c95375a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63ca59c700 1 -- 172.21.15.46:0/2406784443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 msgr2=0x5563c9536e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63ca59c700 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c9536e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63ca59c700 1 -- 172.21.15.46:0/2406784443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc1a6350 con 0x5563c9526a60 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63c9d9b700 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c9536e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63ca59c700 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9526a60 0x5563c9539070 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0095c0 tx=0x7f63bc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fbf9bd61a334fd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63b1ffb700 1 -- 172.21.15.46:0/2406784443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc09a6a0 con 0x5563c9526a60 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63b1ffb700 1 -- 172.21.15.46:0/2406784443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f63bc09ede0 con 0x5563c9526a60 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/2406784443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63c400f050 con 0x5563c9526a60 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63b1ffb700 1 -- 172.21.15.46:0/2406784443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63bc0a33a0 con 0x5563c9526a60 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63dcc4cb80 10 client.16038 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/2406784443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f63c4009a00 con 0x5563c9526a60 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.055+0000 7f63b1ffb700 1 -- 172.21.15.46:0/2406784443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63c400f050 con 0x5563c9526a60 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63b1ffb700 1 -- 172.21.15.46:0/2406784443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f63bc095bc0 con 0x5563c9526a60 2024-03-18T20:37:36.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63dcc4cb80 10 client.16038 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63dcc4cb80 10 client.16038 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63dcc4cb80 20 client.16038 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5563c9596110 0x5563c9598500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63dcc4cb80 4 client.16038 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/2406784443 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5563c92c8cf0 con 0x5563c9596110 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63c9d9b700 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5563c9596110 0x5563c9598500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63b1ffb700 10 client.16038 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63c9d9b700 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5563c9596110 0x5563c9598500 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c977c6bd8755ca6c server_cookie=5ba3989a346290c2 in_seq=0 out_seq=0 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63b1ffb700 1 -- 172.21.15.46:0/2406784443 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5563c92c8cf0 con 0x5563c9596110 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63b1ffb700 10 client.16038 handle_command_reply: tid=0 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.056+0000 7f63dcc4cb80 1 client.16038 shutdown 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/2406784443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9526a60 msgr2=0x5563c9539070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9526a60 0x5563c9539070 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0095c0 tx=0x7f63bc009f60 comp rx=0 tx=0).stop 2024-03-18T20:37:36.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/2406784443 shutdown_connections 2024-03-18T20:37:36.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x5563c9596110 0x5563c9598500 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5563c9526a60 0x5563c9539070 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563c9532220 0x5563c95375a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63b1ffb700 0 client.16038 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:37:36.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63dcc4cb80 1 --2- 172.21.15.46:0/2406784443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563c9527380 0x5563c9536e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/2406784443 >> 172.21.15.46:0/2406784443 conn(0x5563c949d220 msgr2=0x5563c9522aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/2406784443 shutdown_connections 2024-03-18T20:37:36.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63dcc4cb80 1 -- 172.21.15.46:0/2406784443 wait complete. 2024-03-18T20:37:36.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63dcc4cb80 20 client.16038 trim_cache size 0 max 16384 2024-03-18T20:37:36.066 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:37:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63d0ab7700 1 -- 172.21.15.46:0/254038161 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f63b80a13b0 msgr2=0x7f63b80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63d0ab7700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f63b80a13b0 0x7f63b80a3870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63cc102fb0 tx=0x7f63b400b410 comp rx=0 tx=0).stop 2024-03-18T20:37:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63d0ab7700 1 -- 172.21.15.46:0/254038161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63cc0ffe40 msgr2=0x7f63cc1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63d0ab7700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63cc0ffe40 0x7f63cc1018d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f63bc004760 tx=0x7f63bc004790 comp rx=0 tx=0).stop 2024-03-18T20:37:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63d0ab7700 1 -- 172.21.15.46:0/254038161 shutdown_connections 2024-03-18T20:37:36.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63d0ab7700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f63b80a13b0 0x7f63b80a3870 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63d0ab7700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63cc0ffe40 0x7f63cc1018d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63d0ab7700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63cc102350 0x7f63cc1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63d0ab7700 1 --2- 172.21.15.46:0/254038161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63cc104f60 0x7f63cc101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.057+0000 7f63d0ab7700 1 -- 172.21.15.46:0/254038161 >> 172.21.15.46:0/254038161 conn(0x7f63cc076de0 msgr2=0x7f63cc0696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.058+0000 7f63d0ab7700 1 -- 172.21.15.46:0/254038161 shutdown_connections 2024-03-18T20:37:36.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.058+0000 7f63d0ab7700 1 -- 172.21.15.46:0/254038161 wait complete. 2024-03-18T20:37:36.074 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:37:36.074 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:37:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.211+0000 7f2391821700 1 -- 172.21.15.46:0/3369892955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238c100600 msgr2=0x7f238c10caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.211+0000 7f2391821700 1 --2- 172.21.15.46:0/3369892955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238c100600 0x7f238c10caa0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f238400b0a0 tx=0x7f238400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.211+0000 7f2391821700 1 -- 172.21.15.46:0/3369892955 shutdown_connections 2024-03-18T20:37:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.211+0000 7f2391821700 1 --2- 172.21.15.46:0/3369892955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238c100600 0x7f238c10caa0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.211+0000 7f2391821700 1 --2- 172.21.15.46:0/3369892955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f238c105d00 0x7f238c1060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.211+0000 7f2391821700 1 --2- 172.21.15.46:0/3369892955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f238c0ffc40 0x7f238c1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.211+0000 7f2391821700 1 -- 172.21.15.46:0/3369892955 >> 172.21.15.46:0/3369892955 conn(0x7f238c076de0 msgr2=0x7f238c0771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.211+0000 7f2391821700 1 -- 172.21.15.46:0/3369892955 shutdown_connections 2024-03-18T20:37:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.211+0000 7f2391821700 1 -- 172.21.15.46:0/3369892955 wait complete. 2024-03-18T20:37:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.211+0000 7f2391821700 1 Processor -- start 2024-03-18T20:37:36.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f2391821700 1 -- start start 2024-03-18T20:37:36.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f2391821700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f238c0ffc40 0x7f238c1018f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f2391821700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238c100600 0x7f238c101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f2391821700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f238c105d00 0x7f238c102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f2391821700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c1a5620 con 0x7f238c100600 2024-03-18T20:37:36.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f2391821700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c1a5790 con 0x7f238c105d00 2024-03-18T20:37:36.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f2391821700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c1a5950 con 0x7f238c0ffc40 2024-03-18T20:37:36.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f238a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238c100600 0x7f238c101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f238a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238c100600 0x7f238c101e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38290/0 (socket says 172.21.15.46:38290) 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f238a7fc700 1 -- 172.21.15.46:0/2468837139 learned_addr learned my addr 172.21.15.46:0/2468837139 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f238affd700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f238c0ffc40 0x7f238c1018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f238b7fe700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f238c105d00 0x7f238c102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.212+0000 7f238a7fc700 1 -- 172.21.15.46:0/2468837139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f238c0ffc40 msgr2=0x7f238c1018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.213+0000 7f238a7fc700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f238c0ffc40 0x7f238c1018f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.213+0000 7f238a7fc700 1 -- 172.21.15.46:0/2468837139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f238c105d00 msgr2=0x7f238c102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.213+0000 7f238a7fc700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f238c105d00 0x7f238c102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.213+0000 7f238a7fc700 1 -- 172.21.15.46:0/2468837139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2384009d00 con 0x7f238c100600 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.213+0000 7f238a7fc700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238c100600 0x7f238c101e30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f238000d950 tx=0x7f238000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.213+0000 7f239081f700 1 -- 172.21.15.46:0/2468837139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238000f890 con 0x7f238c100600 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.213+0000 7f239081f700 1 -- 172.21.15.46:0/2468837139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2380010460 con 0x7f238c100600 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.213+0000 7f239081f700 1 -- 172.21.15.46:0/2468837139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238000e610 con 0x7f238c100600 2024-03-18T20:37:36.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.213+0000 7f2391821700 1 -- 172.21.15.46:0/2468837139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238c1a5c30 con 0x7f238c100600 2024-03-18T20:37:36.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.213+0000 7f2391821700 1 -- 172.21.15.46:0/2468837139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238c1a60f0 con 0x7f238c100600 2024-03-18T20:37:36.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.216+0000 7f239081f700 1 -- 172.21.15.46:0/2468837139 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f23800108d0 con 0x7f238c100600 2024-03-18T20:37:36.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.217+0000 7f239081f700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f23740a1430 0x7f23740a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.217+0000 7f239081f700 1 -- 172.21.15.46:0/2468837139 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f23800ccd00 con 0x7f238c100600 2024-03-18T20:37:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.217+0000 7f238affd700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f23740a1430 0x7f23740a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:35 smithi073 ceph-mon[47384]: pgmap v48: 97 pgs: 97 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.6 MiB/s wr, 158 op/s 2024-03-18T20:37:36.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:35 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3259730049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:37:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.217+0000 7f238affd700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f23740a1430 0x7f23740a38f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f237c009be0 tx=0x7f237c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.217+0000 7f239d9b6b80 1 Processor -- start 2024-03-18T20:37:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.217+0000 7f239d9b6b80 1 -- start start 2024-03-18T20:37:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f239d9b6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a517be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f239d9b6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a5186ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f239d9b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5189610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f239d9b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a50d5990 con 0x5569a5187220 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f239d9b6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a50d3350 con 0x5569a517c380 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f239d9b6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a50cec40 con 0x5569a517ba60 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5189610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5189610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38298/0 (socket says 172.21.15.46:38298) 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238b7fe700 1 -- 172.21.15.46:0/17667320 learned_addr learned my addr 172.21.15.46:0/17667320 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238affd700 1 --2- 172.21.15.46:0/17667320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a517be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238a7fc700 1 --2- 172.21.15.46:0/17667320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a5186ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238a7fc700 1 -- 172.21.15.46:0/17667320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 msgr2=0x5569a517be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238a7fc700 1 --2- 172.21.15.46:0/17667320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a517be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238a7fc700 1 -- 172.21.15.46:0/17667320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 msgr2=0x5569a5189610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238a7fc700 1 --2- 172.21.15.46:0/17667320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5189610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238a7fc700 1 -- 172.21.15.46:0/17667320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569a4f503b0 con 0x5569a517c380 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238b7fe700 1 --2- 172.21.15.46:0/17667320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5189610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238affd700 1 --2- 172.21.15.46:0/17667320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a517be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.218+0000 7f238a7fc700 1 --2- 172.21.15.46:0/17667320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a5186ce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f23800ca2a0 tx=0x7f238009a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b05c109e1729a52 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.219+0000 7f2371ffb700 1 -- 172.21.15.46:0/17667320 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238009d9c0 con 0x5569a517c380 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.219+0000 7f2371ffb700 1 -- 172.21.15.46:0/17667320 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23800a2010 con 0x5569a517c380 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.219+0000 7f2371ffb700 1 -- 172.21.15.46:0/17667320 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23800a1b80 con 0x5569a517c380 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.219+0000 7f239d9b6b80 1 -- 172.21.15.46:0/17667320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 msgr2=0x5569a5186ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.219+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/17667320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a5186ce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f23800ca2a0 tx=0x7f238009a290 comp rx=0 tx=0).stop 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.219+0000 7f239d9b6b80 1 -- 172.21.15.46:0/17667320 shutdown_connections 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.219+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/17667320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5189610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.219+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/17667320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a517be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.219+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/17667320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a5186ce0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.219+0000 7f239d9b6b80 1 -- 172.21.15.46:0/17667320 >> 172.21.15.46:0/17667320 conn(0x5569a50f2220 msgr2=0x5569a5178020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.219+0000 7f239d9b6b80 1 -- 172.21.15.46:0/17667320 shutdown_connections 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.219+0000 7f239d9b6b80 1 -- 172.21.15.46:0/17667320 wait complete. 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.220+0000 7f239d9b6b80 1 -- start start 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.220+0000 7f239d9b6b80 1 Processor -- start 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.220+0000 7f239d9b6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a5186c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.220+0000 7f239d9b6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a5183fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.220+0000 7f239d9b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5184720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.220+0000 7f239d9b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a506ad80 con 0x5569a5187220 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.220+0000 7f239d9b6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a506aa20 con 0x5569a517c380 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.220+0000 7f239d9b6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a4f526a0 con 0x5569a517ba60 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.220+0000 7f239d9b6b80 10 client.0 _ll_register_callbacks cb 0x5569a4ed63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f238b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5184720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f238b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5184720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38308/0 (socket says 172.21.15.46:38308) 2024-03-18T20:37:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f238b7fe700 1 -- 172.21.15.46:0/3165329977 learned_addr learned my addr 172.21.15.46:0/3165329977 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f238affd700 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a5186c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f238a7fc700 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a5183fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f238b7fe700 1 -- 172.21.15.46:0/3165329977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 msgr2=0x5569a5186c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f238b7fe700 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a5186c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f238b7fe700 1 -- 172.21.15.46:0/3165329977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 msgr2=0x5569a5183fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f238b7fe700 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a5183fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f238b7fe700 1 -- 172.21.15.46:0/3165329977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238c10d390 con 0x5569a5187220 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f238b7fe700 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5184720 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f238400bab0 tx=0x7f238400bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb25c08bea7419a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f236affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f236affd700 1 -- 172.21.15.46:0/3165329977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238400e040 con 0x5569a5187220 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f239d9b6b80 1 -- 172.21.15.46:0/3165329977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569a516fad0 con 0x5569a5187220 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f236affd700 1 -- 172.21.15.46:0/3165329977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2384003c00 con 0x5569a5187220 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f236affd700 1 -- 172.21.15.46:0/3165329977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2384003dc0 con 0x5569a5187220 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.221+0000 7f239d9b6b80 10 client.16050 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.222+0000 7f239d9b6b80 1 -- 172.21.15.46:0/3165329977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f238c1a60f0 con 0x5569a5187220 2024-03-18T20:37:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.222+0000 7f236affd700 1 -- 172.21.15.46:0/3165329977 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5569a516fad0 con 0x5569a5187220 2024-03-18T20:37:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.222+0000 7f236affd700 1 -- 172.21.15.46:0/3165329977 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2384023030 con 0x5569a5187220 2024-03-18T20:37:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.222+0000 7f239d9b6b80 10 client.16050 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.222+0000 7f239d9b6b80 10 client.16050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:37:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.222+0000 7f239d9b6b80 20 client.16050 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.222+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5569a5196d00 0x5569a5199120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.222+0000 7f239d9b6b80 4 client.16050 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.222+0000 7f239d9b6b80 1 -- 172.21.15.46:0/3165329977 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569a50c7700 con 0x5569a5196d00 2024-03-18T20:37:36.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.222+0000 7f238affd700 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5569a5196d00 0x5569a5199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.223+0000 7f236affd700 10 client.16050 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:37:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.223+0000 7f238affd700 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5569a5196d00 0x5569a5199120 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94e1b4c688c23923 server_cookie=6305313247ae7c44 in_seq=0 out_seq=0 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.223+0000 7f236affd700 1 -- 172.21.15.46:0/3165329977 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5569a50c7700 con 0x5569a5196d00 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.223+0000 7f236affd700 10 client.16050 handle_command_reply: tid=0 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.223+0000 7f239d9b6b80 1 client.16050 shutdown 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.224+0000 7f239d9b6b80 1 -- 172.21.15.46:0/3165329977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 msgr2=0x5569a5184720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.224+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5184720 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f238400bab0 tx=0x7f238400bae0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.224+0000 7f239d9b6b80 1 -- 172.21.15.46:0/3165329977 shutdown_connections 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.224+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5569a5196d00 0x5569a5199120 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.224+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5184720 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.224+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a5186c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.224+0000 7f236affd700 0 client.16050 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.224+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/3165329977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a5183fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.224+0000 7f239d9b6b80 1 -- 172.21.15.46:0/3165329977 >> 172.21.15.46:0/3165329977 conn(0x5569a50f2220 msgr2=0x5569a5177c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.224+0000 7f239d9b6b80 1 -- 172.21.15.46:0/3165329977 shutdown_connections 2024-03-18T20:37:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.224+0000 7f239d9b6b80 1 -- 172.21.15.46:0/3165329977 wait complete. 2024-03-18T20:37:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.224+0000 7f239d9b6b80 20 client.16050 trim_cache size 0 max 16384 2024-03-18T20:37:36.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f239d9b6b80 1 Processor -- start 2024-03-18T20:37:36.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f239d9b6b80 1 -- start start 2024-03-18T20:37:36.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f239d9b6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a517e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f239d9b6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a518ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f239d9b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5182930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f239d9b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a5196b30 con 0x5569a5187220 2024-03-18T20:37:36.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f239d9b6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a5142940 con 0x5569a517c380 2024-03-18T20:37:36.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f239d9b6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a5172210 con 0x5569a517ba60 2024-03-18T20:37:36.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f238b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5182930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f238b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5182930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38312/0 (socket says 172.21.15.46:38312) 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f238b7fe700 1 -- 172.21.15.46:0/2323743178 learned_addr learned my addr 172.21.15.46:0/2323743178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f238a7fc700 1 --2- 172.21.15.46:0/2323743178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a518ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f238affd700 1 --2- 172.21.15.46:0/2323743178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a517e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f238b7fe700 1 -- 172.21.15.46:0/2323743178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 msgr2=0x5569a517e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f238b7fe700 1 --2- 172.21.15.46:0/2323743178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a517e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f238b7fe700 1 -- 172.21.15.46:0/2323743178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 msgr2=0x5569a518ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f238b7fe700 1 --2- 172.21.15.46:0/2323743178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a518ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f238b7fe700 1 -- 172.21.15.46:0/2323743178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569a4efcb60 con 0x5569a5187220 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.238+0000 7f238b7fe700 1 --2- 172.21.15.46:0/2323743178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5182930 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2384006270 tx=0x7f238400afe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d35844e9431ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.239+0000 7f23727fc700 1 -- 172.21.15.46:0/2323743178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238400e040 con 0x5569a5187220 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.239+0000 7f23727fc700 1 -- 172.21.15.46:0/2323743178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2384009b80 con 0x5569a5187220 2024-03-18T20:37:36.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.239+0000 7f23727fc700 1 -- 172.21.15.46:0/2323743178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384007790 con 0x5569a5187220 2024-03-18T20:37:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.239+0000 7f239d9b6b80 1 -- 172.21.15.46:0/2323743178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 msgr2=0x5569a5182930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.239+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/2323743178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5182930 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2384006270 tx=0x7f238400afe0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.239+0000 7f239d9b6b80 1 -- 172.21.15.46:0/2323743178 shutdown_connections 2024-03-18T20:37:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.239+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/2323743178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a5182930 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.239+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/2323743178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a517e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.239+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/2323743178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a518ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.239+0000 7f239d9b6b80 1 -- 172.21.15.46:0/2323743178 >> 172.21.15.46:0/2323743178 conn(0x5569a50f2220 msgr2=0x5569a517f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.239+0000 7f239d9b6b80 1 -- 172.21.15.46:0/2323743178 shutdown_connections 2024-03-18T20:37:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.239+0000 7f239d9b6b80 1 -- 172.21.15.46:0/2323743178 wait complete. 2024-03-18T20:37:36.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.240+0000 7f239d9b6b80 1 -- start start 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.240+0000 7f239d9b6b80 1 Processor -- start 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.240+0000 7f239d9b6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a518daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.240+0000 7f239d9b6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a518dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.240+0000 7f239d9b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a518c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.240+0000 7f239d9b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a518c870 con 0x5569a5187220 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.240+0000 7f239d9b6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a518c9e0 con 0x5569a517c380 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.240+0000 7f239d9b6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569a518cba0 con 0x5569a517ba60 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.240+0000 7f239d9b6b80 10 client.0 _ll_register_callbacks cb 0x7f2384007b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.240+0000 7f238b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a518c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.240+0000 7f238b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a518c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38320/0 (socket says 172.21.15.46:38320) 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f238b7fe700 1 -- 172.21.15.46:0/429474841 learned_addr learned my addr 172.21.15.46:0/429474841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f238affd700 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a518daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f238a7fc700 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a518dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f238b7fe700 1 -- 172.21.15.46:0/429474841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 msgr2=0x5569a518daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f238b7fe700 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a518daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f238b7fe700 1 -- 172.21.15.46:0/429474841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 msgr2=0x5569a518dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f238b7fe700 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a518dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f238b7fe700 1 -- 172.21.15.46:0/429474841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238c1a60f0 con 0x5569a5187220 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f238a7fc700 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a518dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f238b7fe700 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a518c160 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f238400b3f0 tx=0x7f2384009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39fcbdd51865e08 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f236affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f236affd700 1 -- 172.21.15.46:0/429474841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238400e040 con 0x5569a5187220 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f236affd700 1 -- 172.21.15.46:0/429474841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2384003a00 con 0x5569a5187220 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f239d9b6b80 1 -- 172.21.15.46:0/429474841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569a4f1dcf0 con 0x5569a5187220 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f236affd700 1 -- 172.21.15.46:0/429474841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2384003bc0 con 0x5569a5187220 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f239d9b6b80 10 client.16056 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.241+0000 7f239d9b6b80 1 -- 172.21.15.46:0/429474841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5569a5094ea0 con 0x5569a5187220 2024-03-18T20:37:36.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.242+0000 7f236affd700 1 -- 172.21.15.46:0/429474841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2384044080 con 0x5569a5187220 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.242+0000 7f236affd700 1 -- 172.21.15.46:0/429474841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2384023070 con 0x5569a5187220 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.242+0000 7f239d9b6b80 10 client.16056 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.242+0000 7f239d9b6b80 10 client.16056 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.242+0000 7f239d9b6b80 20 client.16056 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.242+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5569a51eb9f0 0x5569a51edde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.242+0000 7f239d9b6b80 4 client.16056 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.242+0000 7f239d9b6b80 1 -- 172.21.15.46:0/429474841 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5569a4f52ce0 con 0x5569a51eb9f0 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.242+0000 7f238affd700 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5569a51eb9f0 0x5569a51edde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.242+0000 7f236affd700 10 client.16056 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.242+0000 7f238affd700 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5569a51eb9f0 0x5569a51edde0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=399e9a6850dcda1c server_cookie=93cf7786e425052f in_seq=0 out_seq=0 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f236affd700 1 -- 172.21.15.46:0/429474841 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5569a4f52ce0 con 0x5569a51eb9f0 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f236affd700 10 client.16056 handle_command_reply: tid=0 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f239d9b6b80 1 client.16056 shutdown 2024-03-18T20:37:36.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f239d9b6b80 1 -- 172.21.15.46:0/429474841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 msgr2=0x5569a518c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a518c160 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f238400b3f0 tx=0x7f2384009040 comp rx=0 tx=0).stop 2024-03-18T20:37:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f239d9b6b80 1 -- 172.21.15.46:0/429474841 shutdown_connections 2024-03-18T20:37:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5569a51eb9f0 0x5569a51edde0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569a5187220 0x5569a518c160 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5569a517ba60 0x5569a518daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f236affd700 0 client.16056 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:37:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f239d9b6b80 1 --2- 172.21.15.46:0/429474841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5569a517c380 0x5569a518dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f239d9b6b80 1 -- 172.21.15.46:0/429474841 >> 172.21.15.46:0/429474841 conn(0x5569a50f2220 msgr2=0x5569a51835a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f239d9b6b80 1 -- 172.21.15.46:0/429474841 shutdown_connections 2024-03-18T20:37:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f239d9b6b80 1 -- 172.21.15.46:0/429474841 wait complete. 2024-03-18T20:37:36.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.243+0000 7f239d9b6b80 20 client.16056 trim_cache size 0 max 16384 2024-03-18T20:37:36.253 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:37:36.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.244+0000 7f2391821700 1 -- 172.21.15.46:0/2468837139 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f23740a1430 msgr2=0x7f23740a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.244+0000 7f2391821700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f23740a1430 0x7f23740a38f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f237c009be0 tx=0x7f237c009260 comp rx=0 tx=0).stop 2024-03-18T20:37:36.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.244+0000 7f2391821700 1 -- 172.21.15.46:0/2468837139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238c100600 msgr2=0x7f238c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.244+0000 7f2391821700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238c100600 0x7f238c101e30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f238000d950 tx=0x7f238000dc60 comp rx=0 tx=0).stop 2024-03-18T20:37:36.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.244+0000 7f2391821700 1 -- 172.21.15.46:0/2468837139 shutdown_connections 2024-03-18T20:37:36.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.244+0000 7f2391821700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f23740a1430 0x7f23740a38f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.244+0000 7f2391821700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f238c105d00 0x7f238c102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.244+0000 7f2391821700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f238c0ffc40 0x7f238c1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.244+0000 7f2391821700 1 --2- 172.21.15.46:0/2468837139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f238c100600 0x7f238c101e30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.244+0000 7f2391821700 1 -- 172.21.15.46:0/2468837139 >> 172.21.15.46:0/2468837139 conn(0x7f238c076de0 msgr2=0x7f238c069510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.244+0000 7f2391821700 1 -- 172.21.15.46:0/2468837139 shutdown_connections 2024-03-18T20:37:36.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.244+0000 7f2391821700 1 -- 172.21.15.46:0/2468837139 wait complete. 2024-03-18T20:37:36.260 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:37:36.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:37:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:35 smithi155 ceph-mon[48344]: pgmap v48: 97 pgs: 97 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.6 MiB/s wr, 158 op/s 2024-03-18T20:37:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:35 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3259730049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:37:36.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.397+0000 7f8834f9a700 1 -- 172.21.15.46:0/379565306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8830100f60 msgr2=0x7f8830101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.397+0000 7f8834f9a700 1 --2- 172.21.15.46:0/379565306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8830100f60 0x7f8830101340 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f882000b0a0 tx=0x7f882000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.397+0000 7f8834f9a700 1 -- 172.21.15.46:0/379565306 shutdown_connections 2024-03-18T20:37:36.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.397+0000 7f8834f9a700 1 --2- 172.21.15.46:0/379565306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8830100f60 0x7f8830101340 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.397+0000 7f8834f9a700 1 --2- 172.21.15.46:0/379565306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8830106080 0x7f883010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.397+0000 7f8834f9a700 1 --2- 172.21.15.46:0/379565306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8830101910 0x7f8830105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.397+0000 7f8834f9a700 1 -- 172.21.15.46:0/379565306 >> 172.21.15.46:0/379565306 conn(0x7f88300fc7c0 msgr2=0x7f88300febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.397+0000 7f8834f9a700 1 -- 172.21.15.46:0/379565306 shutdown_connections 2024-03-18T20:37:36.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.397+0000 7f8834f9a700 1 -- 172.21.15.46:0/379565306 wait complete. 2024-03-18T20:37:36.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f8834f9a700 1 Processor -- start 2024-03-18T20:37:36.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f8834f9a700 1 -- start start 2024-03-18T20:37:36.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f8834f9a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8830100f60 0x7f8830199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f8834f9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8830106080 0x7f8830199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f8834f9a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883019de60 0x7f883019e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f8834f9a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883019e8e0 con 0x7f883019de60 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f8834f9a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883019ea50 con 0x7f8830100f60 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f8834f9a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883019ec10 con 0x7f8830106080 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f882ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883019de60 0x7f883019e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f882dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8830106080 0x7f8830199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f882dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8830106080 0x7f8830199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:56510/0 (socket says 172.21.15.46:56510) 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f882ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883019de60 0x7f883019e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38332/0 (socket says 172.21.15.46:38332) 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f882dd9b700 1 -- 172.21.15.46:0/3070021555 learned_addr learned my addr 172.21.15.46:0/3070021555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.398+0000 7f882e59c700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8830100f60 0x7f8830199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.399+0000 7f882ed9d700 1 -- 172.21.15.46:0/3070021555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8830106080 msgr2=0x7f8830199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.399+0000 7f882ed9d700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8830106080 0x7f8830199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.399+0000 7f882ed9d700 1 -- 172.21.15.46:0/3070021555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8830100f60 msgr2=0x7f8830199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.399+0000 7f882ed9d700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8830100f60 0x7f8830199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.399+0000 7f882ed9d700 1 -- 172.21.15.46:0/3070021555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8824009520 con 0x7f883019de60 2024-03-18T20:37:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.399+0000 7f882ed9d700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883019de60 0x7f883019e2e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f882800e970 tx=0x7f882800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.399+0000 7f881f7fe700 1 -- 172.21.15.46:0/3070021555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828004220 con 0x7f883019de60 2024-03-18T20:37:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.399+0000 7f881f7fe700 1 -- 172.21.15.46:0/3070021555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8828004cf0 con 0x7f883019de60 2024-03-18T20:37:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.399+0000 7f8834f9a700 1 -- 172.21.15.46:0/3070021555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8820009d00 con 0x7f883019de60 2024-03-18T20:37:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.399+0000 7f881f7fe700 1 -- 172.21.15.46:0/3070021555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828005660 con 0x7f883019de60 2024-03-18T20:37:36.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.399+0000 7f8834f9a700 1 -- 172.21.15.46:0/3070021555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88301a6470 con 0x7f883019de60 2024-03-18T20:37:36.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.401+0000 7f881f7fe700 1 -- 172.21.15.46:0/3070021555 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f8828004750 con 0x7f883019de60 2024-03-18T20:37:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.402+0000 7f881f7fe700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88180a13b0 0x7f88180a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.402+0000 7f881f7fe700 1 -- 172.21.15.46:0/3070021555 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8828014070 con 0x7f883019de60 2024-03-18T20:37:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.402+0000 7f882e59c700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88180a13b0 0x7f88180a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.403+0000 7f882e59c700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88180a13b0 0x7f88180a3870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8820000f80 tx=0x7f882000b740 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.403+0000 7f884112fb80 1 Processor -- start 2024-03-18T20:37:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.403+0000 7f884112fb80 1 -- start start 2024-03-18T20:37:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.403+0000 7f884112fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537987e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.403+0000 7f884112fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x563537992ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.403+0000 7f884112fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x563537995610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.403+0000 7f884112fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635378e1990 con 0x563537993220 2024-03-18T20:37:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.403+0000 7f884112fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635378df350 con 0x563537987a60 2024-03-18T20:37:36.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.403+0000 7f884112fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635378dac40 con 0x563537988380 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.403+0000 7f882ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x563537995610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f882dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x563537992ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f882e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537987e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f882dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x563537992ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:56514/0 (socket says 172.21.15.46:56514) 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f882dd9b700 1 -- 172.21.15.46:0/1526233940 learned_addr learned my addr 172.21.15.46:0/1526233940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f882dd9b700 1 -- 172.21.15.46:0/1526233940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 msgr2=0x563537987e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f882dd9b700 1 --2- 172.21.15.46:0/1526233940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537987e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f882dd9b700 1 -- 172.21.15.46:0/1526233940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 msgr2=0x563537995610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f882dd9b700 1 --2- 172.21.15.46:0/1526233940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x563537995610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f882dd9b700 1 -- 172.21.15.46:0/1526233940 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56353775c3b0 con 0x563537988380 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f882ed9d700 1 --2- 172.21.15.46:0/1526233940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x563537995610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f882e59c700 1 --2- 172.21.15.46:0/1526233940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537987e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f882dd9b700 1 --2- 172.21.15.46:0/1526233940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x563537992ce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8824009f60 tx=0x7f8824009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4058644fd4e2f614 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.404+0000 7f881cff9700 1 -- 172.21.15.46:0/1526233940 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824019630 con 0x563537988380 2024-03-18T20:37:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.405+0000 7f881cff9700 1 -- 172.21.15.46:0/1526233940 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8824019c50 con 0x563537988380 2024-03-18T20:37:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.405+0000 7f881cff9700 1 -- 172.21.15.46:0/1526233940 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882400ecb0 con 0x563537988380 2024-03-18T20:37:36.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.405+0000 7f884112fb80 1 -- 172.21.15.46:0/1526233940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 msgr2=0x563537992ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.405+0000 7f884112fb80 1 --2- 172.21.15.46:0/1526233940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x563537992ce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8824009f60 tx=0x7f8824009f90 comp rx=0 tx=0).stop 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.405+0000 7f884112fb80 1 -- 172.21.15.46:0/1526233940 shutdown_connections 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.405+0000 7f884112fb80 1 --2- 172.21.15.46:0/1526233940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x563537995610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.405+0000 7f884112fb80 1 --2- 172.21.15.46:0/1526233940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537987e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.405+0000 7f884112fb80 1 --2- 172.21.15.46:0/1526233940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x563537992ce0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.405+0000 7f884112fb80 1 -- 172.21.15.46:0/1526233940 >> 172.21.15.46:0/1526233940 conn(0x5635378fe220 msgr2=0x563537984020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.405+0000 7f884112fb80 1 -- 172.21.15.46:0/1526233940 shutdown_connections 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.405+0000 7f884112fb80 1 -- 172.21.15.46:0/1526233940 wait complete. 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f884112fb80 1 -- start start 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f884112fb80 1 Processor -- start 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f884112fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537992ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f884112fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x56353798ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f884112fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x563537990680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f884112fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563537876d80 con 0x563537993220 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f884112fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563537876a20 con 0x563537987a60 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f884112fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353775e6a0 con 0x563537988380 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f884112fb80 10 client.0 _ll_register_callbacks cb 0x5635376e23a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f882ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x563537990680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f882ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x563537990680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38352/0 (socket says 172.21.15.46:38352) 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f882ed9d700 1 -- 172.21.15.46:0/1432449906 learned_addr learned my addr 172.21.15.46:0/1432449906 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.406+0000 7f882e59c700 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537992ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f882dd9b700 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x56353798ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f882ed9d700 1 -- 172.21.15.46:0/1432449906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 msgr2=0x56353798ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f882ed9d700 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x56353798ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f882ed9d700 1 -- 172.21.15.46:0/1432449906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 msgr2=0x563537992ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f882ed9d700 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537992ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f882ed9d700 1 -- 172.21.15.46:0/1432449906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88301a6470 con 0x563537993220 2024-03-18T20:37:36.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f882ed9d700 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x563537990680 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f882800e940 tx=0x7f882809a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3c6f48966beb50 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f8815ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f8815ffb700 1 -- 172.21.15.46:0/1432449906 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828099280 con 0x563537993220 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f8815ffb700 1 -- 172.21.15.46:0/1432449906 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8828099b90 con 0x563537993220 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f884112fb80 1 -- 172.21.15.46:0/1432449906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56353797bad0 con 0x563537993220 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f8815ffb700 1 -- 172.21.15.46:0/1432449906 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f882809d9c0 con 0x563537993220 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f884112fb80 10 client.16068 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f884112fb80 1 -- 172.21.15.46:0/1432449906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f8824009b90 con 0x563537993220 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f8815ffb700 1 -- 172.21.15.46:0/1432449906 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56353797bad0 con 0x563537993220 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.407+0000 7f8815ffb700 1 -- 172.21.15.46:0/1432449906 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f88280ca6f0 con 0x563537993220 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.408+0000 7f884112fb80 10 client.16068 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.408+0000 7f884112fb80 10 client.16068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.408+0000 7f884112fb80 20 client.16068 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:36.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.408+0000 7f884112fb80 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5635379a2e50 0x5635379a5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.408+0000 7f884112fb80 4 client.16068 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.408+0000 7f884112fb80 1 -- 172.21.15.46:0/1432449906 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635378d3700 con 0x5635379a2e50 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.408+0000 7f882e59c700 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5635379a2e50 0x5635379a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.408+0000 7f882e59c700 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5635379a2e50 0x5635379a5270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81039a9e232bdb8d server_cookie=5e480a3f1e12dc2a in_seq=0 out_seq=0 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.408+0000 7f8815ffb700 10 client.16068 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f8815ffb700 1 -- 172.21.15.46:0/1432449906 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5635378d3700 con 0x5635379a2e50 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f8815ffb700 10 client.16068 handle_command_reply: tid=0 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f884112fb80 1 client.16068 shutdown 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f884112fb80 1 -- 172.21.15.46:0/1432449906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 msgr2=0x563537990680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f884112fb80 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x563537990680 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f882800e940 tx=0x7f882809a310 comp rx=0 tx=0).stop 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f884112fb80 1 -- 172.21.15.46:0/1432449906 shutdown_connections 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f884112fb80 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5635379a2e50 0x5635379a5270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f884112fb80 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x563537990680 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f884112fb80 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537992ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f884112fb80 1 --2- 172.21.15.46:0/1432449906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x56353798ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f8815ffb700 0 client.16068 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:37:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f884112fb80 1 -- 172.21.15.46:0/1432449906 >> 172.21.15.46:0/1432449906 conn(0x5635378fe220 msgr2=0x563537983c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f884112fb80 1 -- 172.21.15.46:0/1432449906 shutdown_connections 2024-03-18T20:37:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f884112fb80 1 -- 172.21.15.46:0/1432449906 wait complete. 2024-03-18T20:37:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.409+0000 7f884112fb80 20 client.16068 trim_cache size 0 max 16384 2024-03-18T20:37:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.423+0000 7f884112fb80 1 Processor -- start 2024-03-18T20:37:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.423+0000 7f884112fb80 1 -- start start 2024-03-18T20:37:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.423+0000 7f884112fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537999e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.423+0000 7f884112fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x563537983b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.423+0000 7f884112fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x56353798b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.423+0000 7f884112fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563537985260 con 0x563537993220 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.423+0000 7f884112fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563537989550 con 0x563537987a60 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.423+0000 7f884112fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353799a390 con 0x563537988380 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.423+0000 7f882ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x56353798b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.423+0000 7f882ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x56353798b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38362/0 (socket says 172.21.15.46:38362) 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.423+0000 7f882ed9d700 1 -- 172.21.15.46:0/2663232804 learned_addr learned my addr 172.21.15.46:0/2663232804 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.424+0000 7f882dd9b700 1 --2- 172.21.15.46:0/2663232804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x563537983b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.424+0000 7f882e59c700 1 --2- 172.21.15.46:0/2663232804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537999e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.424+0000 7f882ed9d700 1 -- 172.21.15.46:0/2663232804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 msgr2=0x563537983b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.424+0000 7f882ed9d700 1 --2- 172.21.15.46:0/2663232804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x563537983b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.424+0000 7f882ed9d700 1 -- 172.21.15.46:0/2663232804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 msgr2=0x563537999e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.424+0000 7f882ed9d700 1 --2- 172.21.15.46:0/2663232804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537999e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.424+0000 7f882ed9d700 1 -- 172.21.15.46:0/2663232804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563537708b60 con 0x563537993220 2024-03-18T20:37:36.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.424+0000 7f882ed9d700 1 --2- 172.21.15.46:0/2663232804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x56353798b280 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8828099890 tx=0x7f88280057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2294b58217dea459 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.424+0000 7f881d7fa700 1 -- 172.21.15.46:0/2663232804 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280a1d80 con 0x563537993220 2024-03-18T20:37:36.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.424+0000 7f881d7fa700 1 -- 172.21.15.46:0/2663232804 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f882809acc0 con 0x563537993220 2024-03-18T20:37:36.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.424+0000 7f881d7fa700 1 -- 172.21.15.46:0/2663232804 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280992e0 con 0x563537993220 2024-03-18T20:37:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 -- 172.21.15.46:0/2663232804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 msgr2=0x56353798b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 --2- 172.21.15.46:0/2663232804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x56353798b280 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8828099890 tx=0x7f88280057e0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 -- 172.21.15.46:0/2663232804 shutdown_connections 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 --2- 172.21.15.46:0/2663232804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537993220 0x56353798b280 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 --2- 172.21.15.46:0/2663232804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537999e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 --2- 172.21.15.46:0/2663232804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537988380 0x563537983b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 -- 172.21.15.46:0/2663232804 >> 172.21.15.46:0/2663232804 conn(0x5635378fe220 msgr2=0x5635379928b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 -- 172.21.15.46:0/2663232804 shutdown_connections 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 -- 172.21.15.46:0/2663232804 wait complete. 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 -- start start 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 Processor -- start 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537997e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537988380 0x563537998370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.425+0000 7f884112fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537993220 0x5635379988d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f884112fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635379f5e30 con 0x563537988380 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f884112fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635379f5fa0 con 0x563537987a60 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f884112fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635379f6160 con 0x563537993220 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f884112fb80 10 client.0 _ll_register_callbacks cb 0x7f8828004e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f882dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537988380 0x563537998370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f882dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537988380 0x563537998370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38370/0 (socket says 172.21.15.46:38370) 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f882dd9b700 1 -- 172.21.15.46:0/3082908492 learned_addr learned my addr 172.21.15.46:0/3082908492 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f882ed9d700 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537993220 0x5635379988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f882e59c700 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537997e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f882dd9b700 1 -- 172.21.15.46:0/3082908492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537993220 msgr2=0x5635379988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f882dd9b700 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537993220 0x5635379988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f882dd9b700 1 -- 172.21.15.46:0/3082908492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 msgr2=0x563537997e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f882dd9b700 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537997e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f882dd9b700 1 -- 172.21.15.46:0/3082908492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8824009b90 con 0x563537988380 2024-03-18T20:37:36.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f882dd9b700 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537988380 0x563537998370 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8824009910 tx=0x7f882400ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7b8ebaf5c25884 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f8815ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f8815ffb700 1 -- 172.21.15.46:0/3082908492 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824019dc0 con 0x563537988380 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f8815ffb700 1 -- 172.21.15.46:0/3082908492 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8824005ce0 con 0x563537988380 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.426+0000 7f884112fb80 1 -- 172.21.15.46:0/3082908492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8824019c50 con 0x563537988380 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.427+0000 7f8815ffb700 1 -- 172.21.15.46:0/3082908492 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8824004020 con 0x563537988380 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.427+0000 7f884112fb80 10 client.16074 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.427+0000 7f884112fb80 1 -- 172.21.15.46:0/3082908492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5635378a0ea0 con 0x563537988380 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.427+0000 7f8815ffb700 1 -- 172.21.15.46:0/3082908492 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8824019c50 con 0x563537988380 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.427+0000 7f8815ffb700 1 -- 172.21.15.46:0/3082908492 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8824005040 con 0x563537988380 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.427+0000 7f884112fb80 10 client.16074 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.427+0000 7f884112fb80 10 client.16074 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.427+0000 7f884112fb80 20 client.16074 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.427+0000 7f884112fb80 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5635379f7810 0x5635379f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.427+0000 7f884112fb80 4 client.16074 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.427+0000 7f884112fb80 1 -- 172.21.15.46:0/3082908492 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563537729cf0 con 0x5635379f7810 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.427+0000 7f882e59c700 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5635379f7810 0x5635379f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f882e59c700 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5635379f7810 0x5635379f9c30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=828707aa53f68692 server_cookie=3e048584bda26a5c in_seq=0 out_seq=0 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f8815ffb700 10 client.16074 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f8815ffb700 1 -- 172.21.15.46:0/3082908492 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563537729cf0 con 0x5635379f7810 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f8815ffb700 10 client.16074 handle_command_reply: tid=0 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f884112fb80 1 client.16074 shutdown 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f884112fb80 1 -- 172.21.15.46:0/3082908492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537988380 msgr2=0x563537998370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f884112fb80 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537988380 0x563537998370 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8824009910 tx=0x7f882400ea50 comp rx=0 tx=0).stop 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f884112fb80 1 -- 172.21.15.46:0/3082908492 shutdown_connections 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f884112fb80 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5635379f7810 0x5635379f9c30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f884112fb80 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563537993220 0x5635379988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f884112fb80 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563537987a60 0x563537997e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f8815ffb700 0 client.16074 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:37:36.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f884112fb80 1 --2- 172.21.15.46:0/3082908492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563537988380 0x563537998370 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f884112fb80 1 -- 172.21.15.46:0/3082908492 >> 172.21.15.46:0/3082908492 conn(0x5635378fe220 msgr2=0x56353798b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f884112fb80 1 -- 172.21.15.46:0/3082908492 shutdown_connections 2024-03-18T20:37:36.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f884112fb80 1 -- 172.21.15.46:0/3082908492 wait complete. 2024-03-18T20:37:36.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.428+0000 7f884112fb80 20 client.16074 trim_cache size 0 max 16384 2024-03-18T20:37:36.438 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:37:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.429+0000 7f8834f9a700 1 -- 172.21.15.46:0/3070021555 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88180a13b0 msgr2=0x7f88180a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.429+0000 7f8834f9a700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88180a13b0 0x7f88180a3870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8820000f80 tx=0x7f882000b740 comp rx=0 tx=0).stop 2024-03-18T20:37:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.429+0000 7f8834f9a700 1 -- 172.21.15.46:0/3070021555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883019de60 msgr2=0x7f883019e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.429+0000 7f8834f9a700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883019de60 0x7f883019e2e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f882800e970 tx=0x7f882800ec80 comp rx=0 tx=0).stop 2024-03-18T20:37:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.429+0000 7f8834f9a700 1 -- 172.21.15.46:0/3070021555 shutdown_connections 2024-03-18T20:37:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.429+0000 7f8834f9a700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88180a13b0 0x7f88180a3870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.429+0000 7f8834f9a700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883019de60 0x7f883019e2e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.429+0000 7f8834f9a700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8830100f60 0x7f8830199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.429+0000 7f8834f9a700 1 --2- 172.21.15.46:0/3070021555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8830106080 0x7f8830199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.429+0000 7f8834f9a700 1 -- 172.21.15.46:0/3070021555 >> 172.21.15.46:0/3070021555 conn(0x7f88300fc7c0 msgr2=0x7f88300fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.429+0000 7f8834f9a700 1 -- 172.21.15.46:0/3070021555 shutdown_connections 2024-03-18T20:37:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.429+0000 7f8834f9a700 1 -- 172.21.15.46:0/3070021555 wait complete. 2024-03-18T20:37:36.445 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:37:36.446 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:37:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.581+0000 7fc4efbf4700 1 -- 172.21.15.46:0/2427603497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e8101920 msgr2=0x7fc4e8105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.581+0000 7fc4efbf4700 1 --2- 172.21.15.46:0/2427603497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e8101920 0x7fc4e8105960 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009990 tx=0x7fc4d8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.581+0000 7fc4efbf4700 1 -- 172.21.15.46:0/2427603497 shutdown_connections 2024-03-18T20:37:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.581+0000 7fc4efbf4700 1 --2- 172.21.15.46:0/2427603497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4e8106090 0x7fc4e810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.581+0000 7fc4efbf4700 1 --2- 172.21.15.46:0/2427603497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e8100f70 0x7fc4e8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.581+0000 7fc4efbf4700 1 --2- 172.21.15.46:0/2427603497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e8101920 0x7fc4e8105960 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.581+0000 7fc4efbf4700 1 -- 172.21.15.46:0/2427603497 >> 172.21.15.46:0/2427603497 conn(0x7fc4e80fc7f0 msgr2=0x7fc4e80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.581+0000 7fc4efbf4700 1 -- 172.21.15.46:0/2427603497 shutdown_connections 2024-03-18T20:37:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.581+0000 7fc4efbf4700 1 -- 172.21.15.46:0/2427603497 wait complete. 2024-03-18T20:37:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.581+0000 7fc4efbf4700 1 Processor -- start 2024-03-18T20:37:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.581+0000 7fc4efbf4700 1 -- start start 2024-03-18T20:37:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4efbf4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e8100f70 0x7fc4e81971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4efbf4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e8106090 0x7fc4e8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4efbf4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4e819baa0 0x7fc4e819bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4efbf4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e819c520 con 0x7fc4e8106090 2024-03-18T20:37:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4efbf4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e819c690 con 0x7fc4e819baa0 2024-03-18T20:37:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4efbf4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e819c850 con 0x7fc4e8100f70 2024-03-18T20:37:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4ed18f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e8106090 0x7fc4e8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4ed18f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e8106090 0x7fc4e8197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38384/0 (socket says 172.21.15.46:38384) 2024-03-18T20:37:36.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4ed18f700 1 -- 172.21.15.46:0/3357111271 learned_addr learned my addr 172.21.15.46:0/3357111271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4ed990700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e8100f70 0x7fc4e81971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4ee191700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4e819baa0 0x7fc4e819bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4ed18f700 1 -- 172.21.15.46:0/3357111271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e8100f70 msgr2=0x7fc4e81971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4ed18f700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e8100f70 0x7fc4e81971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4ed18f700 1 -- 172.21.15.46:0/3357111271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4e819baa0 msgr2=0x7fc4e819bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4ed18f700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4e819baa0 0x7fc4e819bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4ed18f700 1 -- 172.21.15.46:0/3357111271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d80095f0 con 0x7fc4e8106090 2024-03-18T20:37:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4ed990700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e8100f70 0x7fc4e81971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.582+0000 7fc4ed18f700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e8106090 0x7fc4e8197700 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80059d0 tx=0x7fc4d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.583+0000 7fc4deffd700 1 -- 172.21.15.46:0/3357111271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d801d070 con 0x7fc4e8106090 2024-03-18T20:37:36.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.583+0000 7fc4deffd700 1 -- 172.21.15.46:0/3357111271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d8022470 con 0x7fc4e8106090 2024-03-18T20:37:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.583+0000 7fc4efbf4700 1 -- 172.21.15.46:0/3357111271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e81a3d50 con 0x7fc4e8106090 2024-03-18T20:37:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.583+0000 7fc4deffd700 1 -- 172.21.15.46:0/3357111271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8017650 con 0x7fc4e8106090 2024-03-18T20:37:36.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.583+0000 7fc4efbf4700 1 -- 172.21.15.46:0/3357111271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e81a41d0 con 0x7fc4e8106090 2024-03-18T20:37:36.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.585+0000 7fc4deffd700 1 -- 172.21.15.46:0/3357111271 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fc4d80225e0 con 0x7fc4e8106090 2024-03-18T20:37:36.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.585+0000 7fc4deffd700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc4d40a1430 0x7fc4d40a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.586+0000 7fc4deffd700 1 -- 172.21.15.46:0/3357111271 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc4d80ce1f0 con 0x7fc4e8106090 2024-03-18T20:37:36.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.586+0000 7fc4ed990700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc4d40a1430 0x7fc4d40a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.586+0000 7fc4ed990700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc4d40a1430 0x7fc4d40a38f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4011490 tx=0x7fc4e4009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.587+0000 7fc4fbd89b80 1 Processor -- start 2024-03-18T20:37:36.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.587+0000 7fc4fbd89b80 1 -- start start 2024-03-18T20:37:36.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.587+0000 7fc4fbd89b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d38228ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.587+0000 7fc4fbd89b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d38228d380 0x55d382297ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.587+0000 7fc4fbd89b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d382298220 0x55d38229a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.587+0000 7fc4fbd89b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3821e6990 con 0x55d38228d380 2024-03-18T20:37:36.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.587+0000 7fc4fbd89b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3821e4350 con 0x55d382298220 2024-03-18T20:37:36.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.587+0000 7fc4fbd89b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3821dfc40 con 0x55d38228ca60 2024-03-18T20:37:36.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.587+0000 7fc4ed18f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d38228d380 0x55d382297ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.587+0000 7fc4ed18f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d38228d380 0x55d382297ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38392/0 (socket says 172.21.15.46:38392) 2024-03-18T20:37:36.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.587+0000 7fc4ed18f700 1 -- 172.21.15.46:0/4055325872 learned_addr learned my addr 172.21.15.46:0/4055325872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.588+0000 7fc4ed990700 1 --2- 172.21.15.46:0/4055325872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d38228ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.588+0000 7fc4ee191700 1 --2- 172.21.15.46:0/4055325872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d382298220 0x55d38229a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.588+0000 7fc4ed18f700 1 -- 172.21.15.46:0/4055325872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 msgr2=0x55d38228ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.588+0000 7fc4ed18f700 1 --2- 172.21.15.46:0/4055325872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d38228ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.588+0000 7fc4ed18f700 1 -- 172.21.15.46:0/4055325872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d382298220 msgr2=0x55d38229a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.588+0000 7fc4ed18f700 1 --2- 172.21.15.46:0/4055325872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d382298220 0x55d38229a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.588+0000 7fc4ed18f700 1 -- 172.21.15.46:0/4055325872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3820613b0 con 0x55d38228d380 2024-03-18T20:37:36.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.588+0000 7fc4ed18f700 1 --2- 172.21.15.46:0/4055325872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d38228d380 0x55d382297ce0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc4d809b0e0 tx=0x7fc4d8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7b2c4501458042 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.588+0000 7fc4d3fff700 1 -- 172.21.15.46:0/4055325872 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d809fea0 con 0x55d38228d380 2024-03-18T20:37:36.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.588+0000 7fc4d3fff700 1 -- 172.21.15.46:0/4055325872 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d809b850 con 0x55d38228d380 2024-03-18T20:37:36.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.588+0000 7fc4d3fff700 1 -- 172.21.15.46:0/4055325872 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80a10f0 con 0x55d38228d380 2024-03-18T20:37:36.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.589+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/4055325872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d38228d380 msgr2=0x55d382297ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.589+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/4055325872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d38228d380 0x55d382297ce0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc4d809b0e0 tx=0x7fc4d8009310 comp rx=0 tx=0).stop 2024-03-18T20:37:36.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.589+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/4055325872 shutdown_connections 2024-03-18T20:37:36.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.589+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/4055325872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d382298220 0x55d38229a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.589+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/4055325872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d38228ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.589+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/4055325872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d38228d380 0x55d382297ce0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.589+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/4055325872 >> 172.21.15.46:0/4055325872 conn(0x55d382203220 msgr2=0x55d382289020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.589+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/4055325872 shutdown_connections 2024-03-18T20:37:36.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.589+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/4055325872 wait complete. 2024-03-18T20:37:36.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.589+0000 7fc4fbd89b80 1 -- start start 2024-03-18T20:37:36.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.589+0000 7fc4fbd89b80 1 Processor -- start 2024-03-18T20:37:36.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4fbd89b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d382298600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4fbd89b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228d380 0x55d382298b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4fbd89b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d3822a3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4fbd89b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38217bd80 con 0x55d382298220 2024-03-18T20:37:36.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4fbd89b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38217ba20 con 0x55d38228d380 2024-03-18T20:37:36.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4fbd89b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3820636a0 con 0x55d38228ca60 2024-03-18T20:37:36.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4fbd89b80 10 client.0 _ll_register_callbacks cb 0x55d381fe73a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:36.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4ee191700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d3822a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4ee191700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d3822a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38406/0 (socket says 172.21.15.46:38406) 2024-03-18T20:37:36.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4ee191700 1 -- 172.21.15.46:0/230339890 learned_addr learned my addr 172.21.15.46:0/230339890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4ed18f700 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228d380 0x55d382298b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4ed990700 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d382298600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4ee191700 1 -- 172.21.15.46:0/230339890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 msgr2=0x55d382298600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4ee191700 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d382298600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4ee191700 1 -- 172.21.15.46:0/230339890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228d380 msgr2=0x55d382298b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4ee191700 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228d380 0x55d382298b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.590+0000 7fc4ee191700 1 -- 172.21.15.46:0/230339890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e81a41d0 con 0x55d382298220 2024-03-18T20:37:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.591+0000 7fc4ee191700 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d3822a3680 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009f50 tx=0x7fc4e0004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44995e844fe14d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.591+0000 7fc4d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.591+0000 7fc4d17fa700 1 -- 172.21.15.46:0/230339890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0003ab0 con 0x55d382298220 2024-03-18T20:37:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.591+0000 7fc4d17fa700 1 -- 172.21.15.46:0/230339890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc4e0007910 con 0x55d382298220 2024-03-18T20:37:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.591+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/230339890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d382280ad0 con 0x55d382298220 2024-03-18T20:37:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.591+0000 7fc4d17fa700 1 -- 172.21.15.46:0/230339890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4e0004920 con 0x55d382298220 2024-03-18T20:37:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.591+0000 7fc4fbd89b80 10 client.16086 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.591+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/230339890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d38200db60 con 0x55d382298220 2024-03-18T20:37:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.591+0000 7fc4d17fa700 1 -- 172.21.15.46:0/230339890 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d382280ad0 con 0x55d382298220 2024-03-18T20:37:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.592+0000 7fc4d17fa700 1 -- 172.21.15.46:0/230339890 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc4e0014040 con 0x55d382298220 2024-03-18T20:37:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.592+0000 7fc4fbd89b80 10 client.16086 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:36.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.592+0000 7fc4fbd89b80 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:36.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.592+0000 7fc4fbd89b80 20 client.16086 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:36.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.592+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d3822a7b80 0x55d3822a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.592+0000 7fc4fbd89b80 4 client.16086 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:36.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.592+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/230339890 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3821d8700 con 0x55d3822a7b80 2024-03-18T20:37:36.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.592+0000 7fc4ed990700 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d3822a7b80 0x55d3822a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.592+0000 7fc4d17fa700 10 client.16086 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:36.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.592+0000 7fc4ed990700 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d3822a7b80 0x55d3822a9fa0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c47239c33c9cfcf server_cookie=e572f5c127008a48 in_seq=0 out_seq=0 2024-03-18T20:37:36.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4d17fa700 1 -- 172.21.15.46:0/230339890 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d3821d8700 con 0x55d3822a7b80 2024-03-18T20:37:36.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4d17fa700 10 client.16086 handle_command_reply: tid=0 2024-03-18T20:37:36.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4fbd89b80 1 client.16086 shutdown 2024-03-18T20:37:36.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/230339890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 msgr2=0x55d3822a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d3822a3680 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009f50 tx=0x7fc4e0004320 comp rx=0 tx=0).stop 2024-03-18T20:37:36.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/230339890 shutdown_connections 2024-03-18T20:37:36.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d3822a7b80 0x55d3822a9fa0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d3822a3680 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4d17fa700 0 client.16086 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:36.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d382298600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/230339890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228d380 0x55d382298b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/230339890 >> 172.21.15.46:0/230339890 conn(0x55d382203220 msgr2=0x55d382288c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/230339890 shutdown_connections 2024-03-18T20:37:36.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.593+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/230339890 wait complete. 2024-03-18T20:37:36.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.594+0000 7fc4fbd89b80 20 client.16086 trim_cache size 0 max 16384 2024-03-18T20:37:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.607+0000 7fc4fbd89b80 1 Processor -- start 2024-03-18T20:37:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.607+0000 7fc4fbd89b80 1 -- start start 2024-03-18T20:37:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.607+0000 7fc4fbd89b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d38229d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.607+0000 7fc4fbd89b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228d380 0x55d3822919d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.607+0000 7fc4fbd89b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d382288b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.607+0000 7fc4fbd89b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d382283210 con 0x55d382298220 2024-03-18T20:37:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.607+0000 7fc4fbd89b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38228da30 con 0x55d38228d380 2024-03-18T20:37:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.607+0000 7fc4fbd89b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3822855e0 con 0x55d38228ca60 2024-03-18T20:37:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.607+0000 7fc4ee191700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d382288b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.607+0000 7fc4ee191700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d382288b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38418/0 (socket says 172.21.15.46:38418) 2024-03-18T20:37:36.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4ee191700 1 -- 172.21.15.46:0/705138513 learned_addr learned my addr 172.21.15.46:0/705138513 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4ed990700 1 --2- 172.21.15.46:0/705138513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d38229d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4ed18f700 1 --2- 172.21.15.46:0/705138513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228d380 0x55d3822919d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4ee191700 1 -- 172.21.15.46:0/705138513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 msgr2=0x55d38229d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4ee191700 1 --2- 172.21.15.46:0/705138513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d38229d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4ee191700 1 -- 172.21.15.46:0/705138513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228d380 msgr2=0x55d3822919d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4ee191700 1 --2- 172.21.15.46:0/705138513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228d380 0x55d3822919d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4ee191700 1 -- 172.21.15.46:0/705138513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3822aebd0 con 0x55d382298220 2024-03-18T20:37:36.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4ed990700 1 --2- 172.21.15.46:0/705138513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d38229d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:36.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4ed18f700 1 --2- 172.21.15.46:0/705138513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228d380 0x55d3822919d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:36.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4ee191700 1 --2- 172.21.15.46:0/705138513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d382288b50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00061e0 tx=0x7fc4e0015480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d4629affdd2a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4dcda8700 1 -- 172.21.15.46:0/705138513 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e002d070 con 0x55d382298220 2024-03-18T20:37:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4dcda8700 1 -- 172.21.15.46:0/705138513 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4e000f040 con 0x55d382298220 2024-03-18T20:37:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.608+0000 7fc4dcda8700 1 -- 172.21.15.46:0/705138513 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0015e80 con 0x55d382298220 2024-03-18T20:37:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.609+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/705138513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 msgr2=0x55d382288b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.609+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/705138513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d382288b50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00061e0 tx=0x7fc4e0015480 comp rx=0 tx=0).stop 2024-03-18T20:37:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.609+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/705138513 shutdown_connections 2024-03-18T20:37:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.609+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/705138513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d382288b50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.609+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/705138513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228ca60 0x55d38229d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.609+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/705138513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228d380 0x55d3822919d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.609+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/705138513 >> 172.21.15.46:0/705138513 conn(0x55d382203220 msgr2=0x55d3822926d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.609+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/705138513 shutdown_connections 2024-03-18T20:37:36.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.609+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/705138513 wait complete. 2024-03-18T20:37:36.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.609+0000 7fc4fbd89b80 1 -- start start 2024-03-18T20:37:36.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.609+0000 7fc4fbd89b80 1 Processor -- start 2024-03-18T20:37:36.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.610+0000 7fc4fbd89b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228ca60 0x55d3822f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.610+0000 7fc4fbd89b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228d380 0x55d3822f4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.610+0000 7fc4fbd89b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d3822ef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.610+0000 7fc4fbd89b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3822f0050 con 0x55d382298220 2024-03-18T20:37:36.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.610+0000 7fc4fbd89b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3822f01c0 con 0x55d38228ca60 2024-03-18T20:37:36.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.610+0000 7fc4fbd89b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3822f0380 con 0x55d38228d380 2024-03-18T20:37:36.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.610+0000 7fc4fbd89b80 10 client.0 _ll_register_callbacks cb 0x7fc4e0009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:36.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.610+0000 7fc4ee191700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d3822ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.610+0000 7fc4ee191700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d3822ef940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38430/0 (socket says 172.21.15.46:38430) 2024-03-18T20:37:36.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.610+0000 7fc4ee191700 1 -- 172.21.15.46:0/1045642771 learned_addr learned my addr 172.21.15.46:0/1045642771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:36.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.610+0000 7fc4ed18f700 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228d380 0x55d3822f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.610+0000 7fc4ed990700 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228ca60 0x55d3822f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4ee191700 1 -- 172.21.15.46:0/1045642771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228d380 msgr2=0x55d3822f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4ee191700 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228d380 0x55d3822f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4ee191700 1 -- 172.21.15.46:0/1045642771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228ca60 msgr2=0x55d3822f3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:36.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4ee191700 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228ca60 0x55d3822f3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4ee191700 1 -- 172.21.15.46:0/1045642771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d38200db60 con 0x55d382298220 2024-03-18T20:37:36.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4ed18f700 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228d380 0x55d3822f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:36.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4ee191700 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d3822ef940 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009f00 tx=0x7fc4e00158a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c5231239f1bfd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:36.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4d17fa700 1 -- 172.21.15.46:0/1045642771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e002d040 con 0x55d382298220 2024-03-18T20:37:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4d17fa700 1 -- 172.21.15.46:0/1045642771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc4e0004920 con 0x55d382298220 2024-03-18T20:37:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/1045642771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4d809b850 con 0x55d382298220 2024-03-18T20:37:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4d17fa700 1 -- 172.21.15.46:0/1045642771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4e0015f10 con 0x55d382298220 2024-03-18T20:37:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4fbd89b80 10 client.16092 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/1045642771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d3821a5ea0 con 0x55d382298220 2024-03-18T20:37:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4d17fa700 1 -- 172.21.15.46:0/1045642771 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4e003e020 con 0x55d382298220 2024-03-18T20:37:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.611+0000 7fc4d17fa700 1 -- 172.21.15.46:0/1045642771 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc4e0014070 con 0x55d382298220 2024-03-18T20:37:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.612+0000 7fc4fbd89b80 10 client.16092 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.612+0000 7fc4fbd89b80 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.612+0000 7fc4fbd89b80 20 client.16092 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.612+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d3822fca60 0x55d3822fee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.612+0000 7fc4fbd89b80 4 client.16092 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:37:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.612+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/1045642771 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d38202ecf0 con 0x55d3822fca60 2024-03-18T20:37:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.612+0000 7fc4ed990700 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d3822fca60 0x55d3822fee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.612+0000 7fc4ed990700 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d3822fca60 0x55d3822fee80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caea90442fa3e049 server_cookie=9d8abcb388ad56e in_seq=0 out_seq=0 2024-03-18T20:37:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.612+0000 7fc4d17fa700 10 client.16092 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.612+0000 7fc4d17fa700 1 -- 172.21.15.46:0/1045642771 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d38202ecf0 con 0x55d3822fca60 2024-03-18T20:37:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.612+0000 7fc4d17fa700 10 client.16092 handle_command_reply: tid=0 2024-03-18T20:37:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.612+0000 7fc4fbd89b80 1 client.16092 shutdown 2024-03-18T20:37:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/1045642771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 msgr2=0x55d3822ef940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d3822ef940 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009f00 tx=0x7fc4e00158a0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/1045642771 shutdown_connections 2024-03-18T20:37:36.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d3822fca60 0x55d3822fee80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d382298220 0x55d3822ef940 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d38228ca60 0x55d3822f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4fbd89b80 1 --2- 172.21.15.46:0/1045642771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d38228d380 0x55d3822f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4d17fa700 0 client.16092 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:36.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/1045642771 >> 172.21.15.46:0/1045642771 conn(0x55d382203220 msgr2=0x55d3822926d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/1045642771 shutdown_connections 2024-03-18T20:37:36.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4fbd89b80 1 -- 172.21.15.46:0/1045642771 wait complete. 2024-03-18T20:37:36.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4fbd89b80 20 client.16092 trim_cache size 0 max 16384 2024-03-18T20:37:36.624 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:37:36.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4efbf4700 1 -- 172.21.15.46:0/3357111271 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc4d40a1430 msgr2=0x7fc4d40a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4efbf4700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc4d40a1430 0x7fc4d40a38f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4011490 tx=0x7fc4e4009260 comp rx=0 tx=0).stop 2024-03-18T20:37:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4efbf4700 1 -- 172.21.15.46:0/3357111271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e8106090 msgr2=0x7fc4e8197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4efbf4700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e8106090 0x7fc4e8197700 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80059d0 tx=0x7fc4d8004730 comp rx=0 tx=0).stop 2024-03-18T20:37:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4efbf4700 1 -- 172.21.15.46:0/3357111271 shutdown_connections 2024-03-18T20:37:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4efbf4700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc4d40a1430 0x7fc4d40a38f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4efbf4700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4e819baa0 0x7fc4e819bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4efbf4700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e8100f70 0x7fc4e81971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4efbf4700 1 --2- 172.21.15.46:0/3357111271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4e8106090 0x7fc4e8197700 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.613+0000 7fc4efbf4700 1 -- 172.21.15.46:0/3357111271 >> 172.21.15.46:0/3357111271 conn(0x7fc4e80fc7f0 msgr2=0x7fc4e80fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.614+0000 7fc4efbf4700 1 -- 172.21.15.46:0/3357111271 shutdown_connections 2024-03-18T20:37:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:36.614+0000 7fc4efbf4700 1 -- 172.21.15.46:0/3357111271 wait complete. 2024-03-18T20:37:36.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 42974 8.08 MB/sec warmup 31 sec latency 4424.819 ms 2024-03-18T20:37:37.630 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:37:37.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:37:37.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:37:37.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 43264 7.83 MB/sec warmup 32 sec latency 4790.869 ms 2024-03-18T20:37:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.779+0000 7f796bfe5700 1 -- 172.21.15.46:0/1809009096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7964101920 msgr2=0x7f7964105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.779+0000 7f796bfe5700 1 --2- 172.21.15.46:0/1809009096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7964101920 0x7f7964105960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7960009990 tx=0x7f7960009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.779+0000 7f796bfe5700 1 -- 172.21.15.46:0/1809009096 shutdown_connections 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.779+0000 7f796bfe5700 1 --2- 172.21.15.46:0/1809009096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7964100f70 0x7f7964101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.779+0000 7f796bfe5700 1 --2- 172.21.15.46:0/1809009096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7964106090 0x7f796410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.779+0000 7f796bfe5700 1 --2- 172.21.15.46:0/1809009096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7964101920 0x7f7964105960 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.779+0000 7f796bfe5700 1 -- 172.21.15.46:0/1809009096 >> 172.21.15.46:0/1809009096 conn(0x7f79640fc7f0 msgr2=0x7f79640fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.779+0000 7f796bfe5700 1 -- 172.21.15.46:0/1809009096 shutdown_connections 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.779+0000 7f796bfe5700 1 -- 172.21.15.46:0/1809009096 wait complete. 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f796bfe5700 1 Processor -- start 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f796bfe5700 1 -- start start 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f796bfe5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7964100f70 0x7f79641993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f796bfe5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7964106090 0x7f79641998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f796bfe5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f796419dc90 0x7f796419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f796bfe5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796419e710 con 0x7f7964100f70 2024-03-18T20:37:37.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f796bfe5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796419e880 con 0x7f7964106090 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f796bfe5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796419ea40 con 0x7f796419dc90 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f7969d81700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7964100f70 0x7f79641993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f7969d81700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7964100f70 0x7f79641993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38456/0 (socket says 172.21.15.46:38456) 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f7969d81700 1 -- 172.21.15.46:0/554805170 learned_addr learned my addr 172.21.15.46:0/554805170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f7969580700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7964106090 0x7f79641998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.780+0000 7f796a582700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f796419dc90 0x7f796419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.781+0000 7f7969d81700 1 -- 172.21.15.46:0/554805170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f796419dc90 msgr2=0x7f796419e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.781+0000 7f7969d81700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f796419dc90 0x7f796419e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.781+0000 7f7969d81700 1 -- 172.21.15.46:0/554805170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7964106090 msgr2=0x7f79641998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.781+0000 7f7969d81700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7964106090 0x7f79641998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.781+0000 7f7969d81700 1 -- 172.21.15.46:0/554805170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7954009520 con 0x7f7964100f70 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.781+0000 7f7969d81700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7964100f70 0x7f79641993b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f795400eb30 tx=0x7f795400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.781+0000 7f795affd700 1 -- 172.21.15.46:0/554805170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954009880 con 0x7f7964100f70 2024-03-18T20:37:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.781+0000 7f796bfe5700 1 -- 172.21.15.46:0/554805170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79600095f0 con 0x7f7964100f70 2024-03-18T20:37:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.781+0000 7f795affd700 1 -- 172.21.15.46:0/554805170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79540044e0 con 0x7f7964100f70 2024-03-18T20:37:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.781+0000 7f795affd700 1 -- 172.21.15.46:0/554805170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954003d80 con 0x7f7964100f70 2024-03-18T20:37:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.781+0000 7f796bfe5700 1 -- 172.21.15.46:0/554805170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79641a6240 con 0x7f7964100f70 2024-03-18T20:37:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.784+0000 7f795affd700 1 -- 172.21.15.46:0/554805170 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f7954004000 con 0x7f7964100f70 2024-03-18T20:37:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.785+0000 7f795affd700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79500a13b0 0x7f79500a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.785+0000 7f7969580700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79500a13b0 0x7f79500a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.785+0000 7f795affd700 1 -- 172.21.15.46:0/554805170 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f7954014070 con 0x7f7964100f70 2024-03-18T20:37:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.785+0000 7f7969580700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79500a13b0 0x7f79500a3870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f796000b2e0 tx=0x7f7960005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f797817ab80 1 Processor -- start 2024-03-18T20:37:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f797817ab80 1 -- start start 2024-03-18T20:37:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f797817ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b26ae0 0x562ad6b26ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f797817ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b27400 0x562ad6b31d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f797817ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b322a0 0x562ad6b34690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f797817ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ad6a80a10 con 0x562ad6b322a0 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f797817ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ad6a7e3d0 con 0x562ad6b27400 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f797817ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ad6a79cc0 con 0x562ad6b26ae0 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f796a582700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b322a0 0x562ad6b34690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f796a582700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b322a0 0x562ad6b34690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38462/0 (socket says 172.21.15.46:38462) 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f796a582700 1 -- 172.21.15.46:0/2011234273 learned_addr learned my addr 172.21.15.46:0/2011234273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f7969d81700 1 --2- 172.21.15.46:0/2011234273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b26ae0 0x562ad6b26ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.786+0000 7f7969580700 1 --2- 172.21.15.46:0/2011234273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b27400 0x562ad6b31d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.787+0000 7f796a582700 1 -- 172.21.15.46:0/2011234273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b26ae0 msgr2=0x562ad6b26ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.787+0000 7f796a582700 1 --2- 172.21.15.46:0/2011234273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b26ae0 0x562ad6b26ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.787+0000 7f796a582700 1 -- 172.21.15.46:0/2011234273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b27400 msgr2=0x562ad6b31d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.787+0000 7f796a582700 1 --2- 172.21.15.46:0/2011234273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b27400 0x562ad6b31d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.787+0000 7f796a582700 1 -- 172.21.15.46:0/2011234273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ad68fb430 con 0x562ad6b322a0 2024-03-18T20:37:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.787+0000 7f796a582700 1 --2- 172.21.15.46:0/2011234273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b322a0 0x562ad6b34690 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f795c003fa0 tx=0x7f795c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27b6363fb8a4f4c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.787+0000 7f794ffff700 1 -- 172.21.15.46:0/2011234273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c019070 con 0x562ad6b322a0 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.787+0000 7f794ffff700 1 -- 172.21.15.46:0/2011234273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f795c00f040 con 0x562ad6b322a0 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.787+0000 7f794ffff700 1 -- 172.21.15.46:0/2011234273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795c007a00 con 0x562ad6b322a0 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.788+0000 7f797817ab80 1 -- 172.21.15.46:0/2011234273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b322a0 msgr2=0x562ad6b34690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.788+0000 7f797817ab80 1 --2- 172.21.15.46:0/2011234273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b322a0 0x562ad6b34690 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f795c003fa0 tx=0x7f795c009ba0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.788+0000 7f797817ab80 1 -- 172.21.15.46:0/2011234273 shutdown_connections 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.788+0000 7f797817ab80 1 --2- 172.21.15.46:0/2011234273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b322a0 0x562ad6b34690 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.788+0000 7f797817ab80 1 --2- 172.21.15.46:0/2011234273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b26ae0 0x562ad6b26ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.788+0000 7f797817ab80 1 --2- 172.21.15.46:0/2011234273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b27400 0x562ad6b31d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.788+0000 7f797817ab80 1 -- 172.21.15.46:0/2011234273 >> 172.21.15.46:0/2011234273 conn(0x562ad6a9d2a0 msgr2=0x562ad6b230a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.788+0000 7f797817ab80 1 -- 172.21.15.46:0/2011234273 shutdown_connections 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.788+0000 7f797817ab80 1 -- 172.21.15.46:0/2011234273 wait complete. 2024-03-18T20:37:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.788+0000 7f797817ab80 1 -- start start 2024-03-18T20:37:37.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.788+0000 7f797817ab80 1 Processor -- start 2024-03-18T20:37:37.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f797817ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b26ae0 0x562ad6b31d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f797817ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b2f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f797817ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b322a0 0x562ad6b2f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f797817ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ad6a15e00 con 0x562ad6b27400 2024-03-18T20:37:37.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f797817ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ad6a15aa0 con 0x562ad6b322a0 2024-03-18T20:37:37.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f797817ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ad68fd720 con 0x562ad6b26ae0 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f797817ab80 10 client.0 _ll_register_callbacks cb 0x562ad6881420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f7969580700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b2f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f7969580700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b2f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38466/0 (socket says 172.21.15.46:38466) 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f7969d81700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b26ae0 0x562ad6b31d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f7969580700 1 -- 172.21.15.46:0/2751750995 learned_addr learned my addr 172.21.15.46:0/2751750995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f796a582700 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b322a0 0x562ad6b2f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f7969d81700 1 -- 172.21.15.46:0/2751750995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b322a0 msgr2=0x562ad6b2f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f7969d81700 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b322a0 0x562ad6b2f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f7969d81700 1 -- 172.21.15.46:0/2751750995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 msgr2=0x562ad6b2f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f7969d81700 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b2f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f7969d81700 1 -- 172.21.15.46:0/2751750995 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795c003bd0 con 0x562ad6b26ae0 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.789+0000 7f7969580700 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b2f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f7969d81700 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b26ae0 0x562ad6b31d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f79540ca950 tx=0x7f7954099cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e0ceec14ded7cc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:37.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f794d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f794d7fa700 1 -- 172.21.15.46:0/2751750995 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795409aa10 con 0x562ad6b26ae0 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f794d7fa700 1 -- 172.21.15.46:0/2751750995 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f795409a310 con 0x562ad6b26ae0 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f797817ab80 1 -- 172.21.15.46:0/2751750995 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ad6b1ab50 con 0x562ad6b26ae0 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f794d7fa700 1 -- 172.21.15.46:0/2751750995 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f795409e140 con 0x562ad6b26ae0 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f797817ab80 10 client.25867 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f797817ab80 1 -- 172.21.15.46:0/2751750995 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562ad68a7be0 con 0x562ad6b26ae0 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f794d7fa700 1 -- 172.21.15.46:0/2751750995 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ad6b1ab50 con 0x562ad6b26ae0 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f794d7fa700 1 -- 172.21.15.46:0/2751750995 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f79540a2300 con 0x562ad6b26ae0 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f797817ab80 10 client.25867 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f797817ab80 10 client.25867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f797817ab80 20 client.25867 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f797817ab80 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ad6b41ef0 0x562ad6b44310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f797817ab80 4 client.25867 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.790+0000 7f797817ab80 1 -- 172.21.15.46:0/2751750995 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ad6a72780 con 0x562ad6b41ef0 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.791+0000 7f7969580700 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ad6b41ef0 0x562ad6b44310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.791+0000 7f7969580700 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ad6b41ef0 0x562ad6b44310 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6b80737bce72aed server_cookie=d1ff7b62f000bb83 in_seq=0 out_seq=0 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.791+0000 7f794d7fa700 10 client.25867 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f794d7fa700 1 -- 172.21.15.46:0/2751750995 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562ad6a72780 con 0x562ad6b41ef0 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f794d7fa700 10 client.25867 handle_command_reply: tid=0 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f797817ab80 1 client.25867 shutdown 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f797817ab80 1 -- 172.21.15.46:0/2751750995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b26ae0 msgr2=0x562ad6b31d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f797817ab80 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b26ae0 0x562ad6b31d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f79540ca950 tx=0x7f7954099cf0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f797817ab80 1 -- 172.21.15.46:0/2751750995 shutdown_connections 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f797817ab80 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ad6b41ef0 0x562ad6b44310 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f797817ab80 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b322a0 0x562ad6b2f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f794d7fa700 0 client.25867 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f797817ab80 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b26ae0 0x562ad6b31d00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f797817ab80 1 --2- 172.21.15.46:0/2751750995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b2f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f797817ab80 1 -- 172.21.15.46:0/2751750995 >> 172.21.15.46:0/2751750995 conn(0x562ad6a9d2a0 msgr2=0x562ad6b22c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f797817ab80 1 -- 172.21.15.46:0/2751750995 shutdown_connections 2024-03-18T20:37:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f797817ab80 1 -- 172.21.15.46:0/2751750995 wait complete. 2024-03-18T20:37:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.792+0000 7f797817ab80 20 client.25867 trim_cache size 0 max 16384 2024-03-18T20:37:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.810+0000 7f797817ab80 1 Processor -- start 2024-03-18T20:37:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.810+0000 7f797817ab80 1 -- start start 2024-03-18T20:37:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.810+0000 7f797817ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b26ae0 0x562ad6b38ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.810+0000 7f797817ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b2a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.810+0000 7f797817ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b322a0 0x562ad6b2a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.810+0000 7f797817ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ad6b285d0 con 0x562ad6b27400 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.810+0000 7f797817ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ad6b1d000 con 0x562ad6b26ae0 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.810+0000 7f797817ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ad6b3f5c0 con 0x562ad6b322a0 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.810+0000 7f7969580700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b2a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f7969580700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b2a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38480/0 (socket says 172.21.15.46:38480) 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f7969580700 1 -- 172.21.15.46:0/4006817358 learned_addr learned my addr 172.21.15.46:0/4006817358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f796a582700 1 --2- 172.21.15.46:0/4006817358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b322a0 0x562ad6b2a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f7969d81700 1 --2- 172.21.15.46:0/4006817358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b26ae0 0x562ad6b38ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f7969580700 1 -- 172.21.15.46:0/4006817358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b322a0 msgr2=0x562ad6b2a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f7969580700 1 --2- 172.21.15.46:0/4006817358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b322a0 0x562ad6b2a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f7969580700 1 -- 172.21.15.46:0/4006817358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b26ae0 msgr2=0x562ad6b38ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f7969580700 1 --2- 172.21.15.46:0/4006817358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b26ae0 0x562ad6b38ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f7969580700 1 -- 172.21.15.46:0/4006817358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ad6b48f40 con 0x562ad6b27400 2024-03-18T20:37:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f7969580700 1 --2- 172.21.15.46:0/4006817358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b2a3a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x562ad6b30530 tx=0x7f796001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc615ad945a45dcb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f7958ff9700 1 -- 172.21.15.46:0/4006817358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960035e70 con 0x562ad6b27400 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f7958ff9700 1 -- 172.21.15.46:0/4006817358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f796001e040 con 0x562ad6b27400 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.811+0000 7f7958ff9700 1 -- 172.21.15.46:0/4006817358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960004af0 con 0x562ad6b27400 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.812+0000 7f797817ab80 1 -- 172.21.15.46:0/4006817358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 msgr2=0x562ad6b2a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.812+0000 7f797817ab80 1 --2- 172.21.15.46:0/4006817358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b2a3a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x562ad6b30530 tx=0x7f796001a050 comp rx=0 tx=0).stop 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.812+0000 7f797817ab80 1 -- 172.21.15.46:0/4006817358 shutdown_connections 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.812+0000 7f797817ab80 1 --2- 172.21.15.46:0/4006817358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b322a0 0x562ad6b2a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.812+0000 7f797817ab80 1 --2- 172.21.15.46:0/4006817358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b26ae0 0x562ad6b38ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.812+0000 7f797817ab80 1 --2- 172.21.15.46:0/4006817358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b2a3a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.812+0000 7f797817ab80 1 -- 172.21.15.46:0/4006817358 >> 172.21.15.46:0/4006817358 conn(0x562ad6a9d2a0 msgr2=0x562ad6b3d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.812+0000 7f797817ab80 1 -- 172.21.15.46:0/4006817358 shutdown_connections 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.812+0000 7f797817ab80 1 -- 172.21.15.46:0/4006817358 wait complete. 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.812+0000 7f797817ab80 1 -- start start 2024-03-18T20:37:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.812+0000 7f797817ab80 1 Processor -- start 2024-03-18T20:37:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f797817ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b26ae0 0x562ad6b375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f797817ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b37af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f797817ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b322a0 0x562ad6b38030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f797817ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ad6b95d30 con 0x562ad6b27400 2024-03-18T20:37:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f797817ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ad6b95e70 con 0x562ad6b26ae0 2024-03-18T20:37:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f797817ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ad6b95fb0 con 0x562ad6b322a0 2024-03-18T20:37:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f797817ab80 10 client.0 _ll_register_callbacks cb 0x7f795409a190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f7969580700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b37af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f7969580700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b37af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38488/0 (socket says 172.21.15.46:38488) 2024-03-18T20:37:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f7969580700 1 -- 172.21.15.46:0/932355726 learned_addr learned my addr 172.21.15.46:0/932355726 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f7969d81700 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b26ae0 0x562ad6b375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f796a582700 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b322a0 0x562ad6b38030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f7969580700 1 -- 172.21.15.46:0/932355726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b322a0 msgr2=0x562ad6b38030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f7969580700 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b322a0 0x562ad6b38030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f7969580700 1 -- 172.21.15.46:0/932355726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b26ae0 msgr2=0x562ad6b375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f7969580700 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b26ae0 0x562ad6b375b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f7969580700 1 -- 172.21.15.46:0/932355726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796002c040 con 0x562ad6b27400 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.813+0000 7f7969d81700 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b26ae0 0x562ad6b375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f7969580700 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b37af0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7960009350 tx=0x7f796000bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dffe7485f79281 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f794d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f794d7fa700 1 -- 172.21.15.46:0/932355726 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79600351b0 con 0x562ad6b27400 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f797817ab80 1 -- 172.21.15.46:0/932355726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f795c00f040 con 0x562ad6b27400 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f794d7fa700 1 -- 172.21.15.46:0/932355726 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f7960035310 con 0x562ad6b27400 2024-03-18T20:37:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f794d7fa700 1 -- 172.21.15.46:0/932355726 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79600354d0 con 0x562ad6b27400 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f797817ab80 10 client.16110 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f797817ab80 1 -- 172.21.15.46:0/932355726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562ad6a3ff20 con 0x562ad6b27400 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f794d7fa700 1 -- 172.21.15.46:0/932355726 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f796004f020 con 0x562ad6b27400 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f794d7fa700 1 -- 172.21.15.46:0/932355726 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f796000f990 con 0x562ad6b27400 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f797817ab80 10 client.16110 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f797817ab80 10 client.16110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f797817ab80 20 client.16110 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f797817ab80 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ad6b973e0 0x562ad6b99800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f797817ab80 4 client.16110 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.814+0000 7f797817ab80 1 -- 172.21.15.46:0/932355726 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562ad68c8d70 con 0x562ad6b973e0 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.815+0000 7f7969d81700 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ad6b973e0 0x562ad6b99800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.815+0000 7f794d7fa700 10 client.16110 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.815+0000 7f7969d81700 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ad6b973e0 0x562ad6b99800 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=949d5796f94ffaa2 server_cookie=7e7e6e89431e1a3e in_seq=0 out_seq=0 2024-03-18T20:37:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f794d7fa700 1 -- 172.21.15.46:0/932355726 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562ad68c8d70 con 0x562ad6b973e0 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f794d7fa700 10 client.16110 handle_command_reply: tid=0 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f797817ab80 1 client.16110 shutdown 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f797817ab80 1 -- 172.21.15.46:0/932355726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 msgr2=0x562ad6b37af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f797817ab80 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b37af0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7960009350 tx=0x7f796000bad0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f797817ab80 1 -- 172.21.15.46:0/932355726 shutdown_connections 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f797817ab80 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ad6b973e0 0x562ad6b99800 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f797817ab80 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ad6b322a0 0x562ad6b38030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f797817ab80 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ad6b26ae0 0x562ad6b375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f794d7fa700 0 client.16110 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f797817ab80 1 --2- 172.21.15.46:0/932355726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ad6b27400 0x562ad6b37af0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f797817ab80 1 -- 172.21.15.46:0/932355726 >> 172.21.15.46:0/932355726 conn(0x562ad6a9d2a0 msgr2=0x562ad6b2a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f797817ab80 1 -- 172.21.15.46:0/932355726 shutdown_connections 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f797817ab80 1 -- 172.21.15.46:0/932355726 wait complete. 2024-03-18T20:37:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.816+0000 7f797817ab80 20 client.16110 trim_cache size 0 max 16384 2024-03-18T20:37:37.826 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:37:37.826 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:37:37.826 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "9619e554-3c82-42e0-9b0d-d584964bdef3", 2024-03-18T20:37:37.826 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:37:37.826 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:37:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.817+0000 7f796bfe5700 1 -- 172.21.15.46:0/554805170 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79500a13b0 msgr2=0x7f79500a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.817+0000 7f796bfe5700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79500a13b0 0x7f79500a3870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f796000b2e0 tx=0x7f7960005960 comp rx=0 tx=0).stop 2024-03-18T20:37:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.817+0000 7f796bfe5700 1 -- 172.21.15.46:0/554805170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7964100f70 msgr2=0x7f79641993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.817+0000 7f796bfe5700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7964100f70 0x7f79641993b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f795400eb30 tx=0x7f795400ee40 comp rx=0 tx=0).stop 2024-03-18T20:37:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.817+0000 7f796bfe5700 1 -- 172.21.15.46:0/554805170 shutdown_connections 2024-03-18T20:37:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.817+0000 7f796bfe5700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79500a13b0 0x7f79500a3870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.817+0000 7f796bfe5700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7964100f70 0x7f79641993b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.817+0000 7f796bfe5700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f796419dc90 0x7f796419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.817+0000 7f796bfe5700 1 --2- 172.21.15.46:0/554805170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7964106090 0x7f79641998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.817+0000 7f796bfe5700 1 -- 172.21.15.46:0/554805170 >> 172.21.15.46:0/554805170 conn(0x7f79640fc7f0 msgr2=0x7f79640feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.817+0000 7f796bfe5700 1 -- 172.21.15.46:0/554805170 shutdown_connections 2024-03-18T20:37:37.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:37.817+0000 7f796bfe5700 1 -- 172.21.15.46:0/554805170 wait complete. 2024-03-18T20:37:37.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:37:37.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:37:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.022+0000 7fcceca76700 1 -- 172.21.15.46:0/3639951818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcce8069540 msgr2=0x7fcce8069920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.022+0000 7fcceca76700 1 --2- 172.21.15.46:0/3639951818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcce8069540 0x7fcce8069920 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fccd80099c0 tx=0x7fccd8009cd0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.022+0000 7fcceca76700 1 -- 172.21.15.46:0/3639951818 shutdown_connections 2024-03-18T20:37:38.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.022+0000 7fcceca76700 1 --2- 172.21.15.46:0/3639951818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcce8069540 0x7fcce8069920 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.022+0000 7fcceca76700 1 --2- 172.21.15.46:0/3639951818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce81060a0 0x7fcce810dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.022+0000 7fcceca76700 1 --2- 172.21.15.46:0/3639951818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcce8069ef0 0x7fcce8105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.022+0000 7fcceca76700 1 -- 172.21.15.46:0/3639951818 >> 172.21.15.46:0/3639951818 conn(0x7fcce8076d50 msgr2=0x7fcce8077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:38.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.022+0000 7fcceca76700 1 -- 172.21.15.46:0/3639951818 shutdown_connections 2024-03-18T20:37:38.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.022+0000 7fcceca76700 1 -- 172.21.15.46:0/3639951818 wait complete. 2024-03-18T20:37:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fcceca76700 1 Processor -- start 2024-03-18T20:37:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fcceca76700 1 -- start start 2024-03-18T20:37:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fcceca76700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcce8069540 0x7fcce80ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fcceca76700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce81060a0 0x7fcce81001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fcceca76700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcce81006f0 0x7fcce81a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fcceca76700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce81a5920 con 0x7fcce81006f0 2024-03-18T20:37:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fcceca76700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce81a5a90 con 0x7fcce81060a0 2024-03-18T20:37:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fcceca76700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce81a5c50 con 0x7fcce8069540 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fcce6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcce81006f0 0x7fcce81a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fcce6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcce81006f0 0x7fcce81a5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38504/0 (socket says 172.21.15.46:38504) 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fcce6d9d700 1 -- 172.21.15.46:0/363879016 learned_addr learned my addr 172.21.15.46:0/363879016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fccddd9b700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce81060a0 0x7fcce81001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.023+0000 7fcce659c700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcce8069540 0x7fcce80ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.024+0000 7fcce6d9d700 1 -- 172.21.15.46:0/363879016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcce8069540 msgr2=0x7fcce80ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.024+0000 7fcce6d9d700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcce8069540 0x7fcce80ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.024+0000 7fcce6d9d700 1 -- 172.21.15.46:0/363879016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce81060a0 msgr2=0x7fcce81001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.024+0000 7fcce6d9d700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce81060a0 0x7fcce81001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.024+0000 7fcce6d9d700 1 -- 172.21.15.46:0/363879016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd80095f0 con 0x7fcce81006f0 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.024+0000 7fcce659c700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcce8069540 0x7fcce80ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.024+0000 7fcce6d9d700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcce81006f0 0x7fcce81a5350 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcce000ef30 tx=0x7fcce000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.024+0000 7fccdffff700 1 -- 172.21.15.46:0/363879016 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0018070 con 0x7fcce81006f0 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.024+0000 7fcceca76700 1 -- 172.21.15.46:0/363879016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce81a5ed0 con 0x7fcce81006f0 2024-03-18T20:37:38.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.024+0000 7fccdffff700 1 -- 172.21.15.46:0/363879016 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcce000f040 con 0x7fcce81006f0 2024-03-18T20:37:38.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.024+0000 7fccdffff700 1 -- 172.21.15.46:0/363879016 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce000cd00 con 0x7fcce81006f0 2024-03-18T20:37:38.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.024+0000 7fcceca76700 1 -- 172.21.15.46:0/363879016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce81a6360 con 0x7fcce81006f0 2024-03-18T20:37:38.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.027+0000 7fccdffff700 1 -- 172.21.15.46:0/363879016 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fcce0003a00 con 0x7fcce81006f0 2024-03-18T20:37:38.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.027+0000 7fccdffff700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fccd40a13b0 0x7fccd40a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.027+0000 7fccdffff700 1 -- 172.21.15.46:0/363879016 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fcce00cca70 con 0x7fcce81006f0 2024-03-18T20:37:38.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.027+0000 7fcce659c700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fccd40a13b0 0x7fccd40a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.028+0000 7fcce659c700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fccd40a13b0 0x7fccd40a3870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009fd0 tx=0x7fccd8005fb0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:38.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.028+0000 7fccf8c0bb80 1 Processor -- start 2024-03-18T20:37:38.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fccf8c0bb80 1 -- start start 2024-03-18T20:37:38.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fccf8c0bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb72e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fccf8c0bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb7dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fccf8c0bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb80610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fccf8c0bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dffacc990 con 0x555dffb72a60 2024-03-18T20:37:38.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fccf8c0bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dffaca350 con 0x555dffb7e220 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fccf8c0bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dffac5c40 con 0x555dffb73380 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fcce659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb72e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fcce659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb72e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38516/0 (socket says 172.21.15.46:38516) 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fcce659c700 1 -- 172.21.15.46:0/1101961632 learned_addr learned my addr 172.21.15.46:0/1101961632 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fcce6d9d700 1 --2- 172.21.15.46:0/1101961632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb80610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fccddd9b700 1 --2- 172.21.15.46:0/1101961632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb7dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fcce659c700 1 -- 172.21.15.46:0/1101961632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 msgr2=0x555dffb7dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fcce659c700 1 --2- 172.21.15.46:0/1101961632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb7dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fcce659c700 1 -- 172.21.15.46:0/1101961632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 msgr2=0x555dffb80610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fcce659c700 1 --2- 172.21.15.46:0/1101961632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb80610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.029+0000 7fcce659c700 1 -- 172.21.15.46:0/1101961632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dff9473b0 con 0x555dffb72a60 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.030+0000 7fcce659c700 1 --2- 172.21.15.46:0/1101961632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb72e40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fccd8015000 tx=0x7fccd801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c687281d8c9c0d28 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.030+0000 7fccdcd99700 1 -- 172.21.15.46:0/1101961632 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd802a070 con 0x555dffb72a60 2024-03-18T20:37:38.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.030+0000 7fccdcd99700 1 -- 172.21.15.46:0/1101961632 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccd801d040 con 0x555dffb72a60 2024-03-18T20:37:38.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.030+0000 7fccdcd99700 1 -- 172.21.15.46:0/1101961632 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd800f960 con 0x555dffb72a60 2024-03-18T20:37:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.031+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/1101961632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 msgr2=0x555dffb72e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.031+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/1101961632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb72e40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fccd8015000 tx=0x7fccd801f000 comp rx=0 tx=0).stop 2024-03-18T20:37:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.031+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/1101961632 shutdown_connections 2024-03-18T20:37:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.031+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/1101961632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb72e40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.031+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/1101961632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb80610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.031+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/1101961632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb7dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.031+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/1101961632 >> 172.21.15.46:0/1101961632 conn(0x555dffae9220 msgr2=0x555dffb6f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.031+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/1101961632 shutdown_connections 2024-03-18T20:37:38.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.031+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/1101961632 wait complete. 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.031+0000 7fccf8c0bb80 1 -- start start 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.031+0000 7fccf8c0bb80 1 Processor -- start 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fccf8c0bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb7dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fccf8c0bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb7aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fccf8c0bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb7b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fccf8c0bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dffa61d80 con 0x555dffb72a60 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fccf8c0bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dffa61a20 con 0x555dffb7e220 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fccf8c0bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dff9496a0 con 0x555dffb73380 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fccf8c0bb80 10 client.0 _ll_register_callbacks cb 0x555dff8cd3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcce659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb7dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcce659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb7dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38522/0 (socket says 172.21.15.46:38522) 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcce659c700 1 -- 172.21.15.46:0/3184104059 learned_addr learned my addr 172.21.15.46:0/3184104059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fccddd9b700 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb7aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcce6d9d700 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb7b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcce659c700 1 -- 172.21.15.46:0/3184104059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 msgr2=0x555dffb7aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcce659c700 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb7aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcce659c700 1 -- 172.21.15.46:0/3184104059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 msgr2=0x555dffb7b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcce659c700 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb7b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcce659c700 1 -- 172.21.15.46:0/3184104059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce810e460 con 0x555dffb72a60 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcce6d9d700 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb7b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcce659c700 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb7dbf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fccd80058e0 tx=0x7fccd800f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40bee5f6671301a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcccdffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcccdffb700 1 -- 172.21.15.46:0/3184104059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd802a040 con 0x555dffb72a60 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.032+0000 7fcccdffb700 1 -- 172.21.15.46:0/3184104059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fccd8003a90 con 0x555dffb72a60 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3184104059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555dffb66ad0 con 0x555dffb72a60 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fcccdffb700 1 -- 172.21.15.46:0/3184104059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccd8005be0 con 0x555dffb72a60 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fccf8c0bb80 10 client.16122 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3184104059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x555dff8f3b60 con 0x555dffb72a60 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fcccdffb700 1 -- 172.21.15.46:0/3184104059 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccd8043020 con 0x555dffb72a60 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fcccdffb700 1 -- 172.21.15.46:0/3184104059 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fccd8023410 con 0x555dffb72a60 2024-03-18T20:37:38.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fccf8c0bb80 10 client.16122 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fccf8c0bb80 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fccf8c0bb80 20 client.16122 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555dffb8d970 0x555dffb8fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fccf8c0bb80 4 client.16122 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3184104059 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555dffabe700 con 0x555dffb8d970 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.033+0000 7fccddd9b700 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555dffb8d970 0x555dffb8fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.034+0000 7fccddd9b700 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555dffb8d970 0x555dffb8fd90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ad16c216b0a19f server_cookie=396a83911a1df91b in_seq=0 out_seq=0 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.034+0000 7fcccdffb700 10 client.16122 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.034+0000 7fcccdffb700 1 -- 172.21.15.46:0/3184104059 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555dffabe700 con 0x555dffb8d970 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.034+0000 7fcccdffb700 10 client.16122 handle_command_reply: tid=0 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.034+0000 7fccf8c0bb80 1 client.16122 shutdown 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.035+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3184104059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 msgr2=0x555dffb7dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.035+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb7dbf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fccd80058e0 tx=0x7fccd800f8c0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.035+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3184104059 shutdown_connections 2024-03-18T20:37:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.035+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555dffb8d970 0x555dffb8fd90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.035+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb7dbf0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.035+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb7b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.035+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/3184104059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb7aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.035+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3184104059 >> 172.21.15.46:0/3184104059 conn(0x555dffae9220 msgr2=0x555dffb6ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.035+0000 7fcccdffb700 0 client.16122 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.035+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3184104059 shutdown_connections 2024-03-18T20:37:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.035+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3184104059 wait complete. 2024-03-18T20:37:38.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.035+0000 7fccf8c0bb80 20 client.16122 trim_cache size 0 max 16384 2024-03-18T20:37:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.053+0000 7fccf8c0bb80 1 Processor -- start 2024-03-18T20:37:38.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.053+0000 7fccf8c0bb80 1 -- start start 2024-03-18T20:37:38.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.053+0000 7fccf8c0bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb84d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.053+0000 7fccf8c0bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb8f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.053+0000 7fccf8c0bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb6eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.053+0000 7fccf8c0bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dffb74610 con 0x555dffb72a60 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.053+0000 7fccf8c0bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dffb74440 con 0x555dffb7e220 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.053+0000 7fccf8c0bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dffb7be40 con 0x555dffb73380 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fcce659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb84d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fccddd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb8f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fcce659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb84d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38526/0 (socket says 172.21.15.46:38526) 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fcce659c700 1 -- 172.21.15.46:0/4014884236 learned_addr learned my addr 172.21.15.46:0/4014884236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fcce6d9d700 1 --2- 172.21.15.46:0/4014884236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb6eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fccddd9b700 1 -- 172.21.15.46:0/4014884236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 msgr2=0x555dffb6eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fccddd9b700 1 --2- 172.21.15.46:0/4014884236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb6eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fccddd9b700 1 -- 172.21.15.46:0/4014884236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 msgr2=0x555dffb84d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fccddd9b700 1 --2- 172.21.15.46:0/4014884236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb84d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fccddd9b700 1 -- 172.21.15.46:0/4014884236 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dffa8bea0 con 0x555dffb73380 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fccddd9b700 1 --2- 172.21.15.46:0/4014884236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb8f980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x555dffb90660 tx=0x7fccd0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d09fe4c22061e0f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fccdd59a700 1 -- 172.21.15.46:0/4014884236 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0017030 con 0x555dffb73380 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fccdd59a700 1 -- 172.21.15.46:0/4014884236 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccd0018040 con 0x555dffb73380 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.054+0000 7fccdd59a700 1 -- 172.21.15.46:0/4014884236 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0004cd0 con 0x555dffb73380 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.055+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/4014884236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 msgr2=0x555dffb8f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.055+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/4014884236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb8f980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x555dffb90660 tx=0x7fccd0009260 comp rx=0 tx=0).stop 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.055+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/4014884236 shutdown_connections 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.055+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/4014884236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb72a60 0x555dffb84d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.055+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/4014884236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb7e220 0x555dffb6eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.055+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/4014884236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb73380 0x555dffb8f980 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.055+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/4014884236 >> 172.21.15.46:0/4014884236 conn(0x555dffae9220 msgr2=0x555dffb7d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.055+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/4014884236 shutdown_connections 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.055+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/4014884236 wait complete. 2024-03-18T20:37:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.055+0000 7fccf8c0bb80 1 -- start start 2024-03-18T20:37:38.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fccf8c0bb80 1 Processor -- start 2024-03-18T20:37:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fccf8c0bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb72a60 0x555dffb82d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fccf8c0bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb73380 0x555dffb832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fccf8c0bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb7e220 0x555dffb837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fccf8c0bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dffbe0840 con 0x555dffb73380 2024-03-18T20:37:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fccf8c0bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dffbe0980 con 0x555dffb72a60 2024-03-18T20:37:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fccf8c0bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dffbe0b40 con 0x555dffb7e220 2024-03-18T20:37:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fccf8c0bb80 10 client.0 _ll_register_callbacks cb 0x7fcce0009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fccddd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb73380 0x555dffb832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fcce659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb72a60 0x555dffb82d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fccddd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb73380 0x555dffb832b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38542/0 (socket says 172.21.15.46:38542) 2024-03-18T20:37:38.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fccddd9b700 1 -- 172.21.15.46:0/3047519476 learned_addr learned my addr 172.21.15.46:0/3047519476 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:38.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fcce6d9d700 1 --2- 172.21.15.46:0/3047519476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb7e220 0x555dffb837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fcce659c700 1 -- 172.21.15.46:0/3047519476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb7e220 msgr2=0x555dffb837f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:38.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fcce659c700 1 --2- 172.21.15.46:0/3047519476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb7e220 0x555dffb837f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fcce659c700 1 -- 172.21.15.46:0/3047519476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb73380 msgr2=0x555dffb832b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:38.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fcce659c700 1 --2- 172.21.15.46:0/3047519476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb73380 0x555dffb832b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.056+0000 7fcce659c700 1 -- 172.21.15.46:0/3047519476 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555dff94bcd0 con 0x555dffb72a60 2024-03-18T20:37:38.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fcce659c700 1 --2- 172.21.15.46:0/3047519476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb72a60 0x555dffb82d70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fccd8005a80 tx=0x7fccd8023c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71637bf6af6788f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:38.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fcccdffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:37:38.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fcccdffb700 1 -- 172.21.15.46:0/3047519476 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8027060 con 0x555dffb72a60 2024-03-18T20:37:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fcccdffb700 1 -- 172.21.15.46:0/3047519476 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fccd8044080 con 0x555dffb72a60 2024-03-18T20:37:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3047519476 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccd801d040 con 0x555dffb72a60 2024-03-18T20:37:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fcccdffb700 1 -- 172.21.15.46:0/3047519476 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccd802a070 con 0x555dffb72a60 2024-03-18T20:37:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fccf8c0bb80 10 client.25892 fetch_fsmap learned FSMap version 31 2024-03-18T20:37:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3047519476 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x555dff8f2030 con 0x555dffb72a60 2024-03-18T20:37:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fcccdffb700 1 -- 172.21.15.46:0/3047519476 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccd804b080 con 0x555dffb72a60 2024-03-18T20:37:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fcccdffb700 1 -- 172.21.15.46:0/3047519476 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fccd8036a20 con 0x555dffb72a60 2024-03-18T20:37:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fccf8c0bb80 10 client.25892 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T20:37:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fccf8c0bb80 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fccf8c0bb80 20 client.25892 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:38.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/3047519476 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555dffbe1f00 0x555dffbe4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fccf8c0bb80 4 client.25892 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3047519476 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555dff914cf0 con 0x555dffbe1f00 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.057+0000 7fccddd9b700 1 --2- 172.21.15.46:0/3047519476 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555dffbe1f00 0x555dffbe4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fcccdffb700 10 client.25892 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fccddd9b700 1 --2- 172.21.15.46:0/3047519476 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555dffbe1f00 0x555dffbe4320 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af19679f1d6111c2 server_cookie=746fe53ceaa83a50 in_seq=0 out_seq=0 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fcccdffb700 1 -- 172.21.15.46:0/3047519476 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555dff914cf0 con 0x555dffbe1f00 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fcccdffb700 10 client.25892 handle_command_reply: tid=0 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fccf8c0bb80 1 client.25892 shutdown 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3047519476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb72a60 msgr2=0x555dffb82d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/3047519476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb72a60 0x555dffb82d70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fccd8005a80 tx=0x7fccd8023c20 comp rx=0 tx=0).stop 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3047519476 shutdown_connections 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/3047519476 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555dffbe1f00 0x555dffbe4320 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/3047519476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dffb7e220 0x555dffb837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fcccdffb700 0 client.25892 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/3047519476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555dffb72a60 0x555dffb82d70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fccf8c0bb80 1 --2- 172.21.15.46:0/3047519476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555dffb73380 0x555dffb832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3047519476 >> 172.21.15.46:0/3047519476 conn(0x555dffae9220 msgr2=0x555dffb76240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3047519476 shutdown_connections 2024-03-18T20:37:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.058+0000 7fccf8c0bb80 1 -- 172.21.15.46:0/3047519476 wait complete. 2024-03-18T20:37:38.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fccf8c0bb80 20 client.25892 trim_cache size 0 max 16384 2024-03-18T20:37:38.068 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:37:38.068 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:37:38.068 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:37:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: "9619e554-3c82-42e0-9b0d-d584964bdef3": { 2024-03-18T20:37:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:37:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "9619e554-3c82-42e0-9b0d-d584964bdef3", 2024-03-18T20:37:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:37:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:37:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:37:38.069 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:37:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fcceca76700 1 -- 172.21.15.46:0/363879016 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fccd40a13b0 msgr2=0x7fccd40a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fcceca76700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fccd40a13b0 0x7fccd40a3870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009fd0 tx=0x7fccd8005fb0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fcceca76700 1 -- 172.21.15.46:0/363879016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcce81006f0 msgr2=0x7fcce81a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fcceca76700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcce81006f0 0x7fcce81a5350 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcce000ef30 tx=0x7fcce000c3a0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fcceca76700 1 -- 172.21.15.46:0/363879016 shutdown_connections 2024-03-18T20:37:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fcceca76700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fccd40a13b0 0x7fccd40a3870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fcceca76700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcce81006f0 0x7fcce81a5350 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fcceca76700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcce8069540 0x7fcce80ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fcceca76700 1 --2- 172.21.15.46:0/363879016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce81060a0 0x7fcce81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fcceca76700 1 -- 172.21.15.46:0/363879016 >> 172.21.15.46:0/363879016 conn(0x7fcce8076d50 msgr2=0x7fcce80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fcceca76700 1 -- 172.21.15.46:0/363879016 shutdown_connections 2024-03-18T20:37:38.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:37:38.059+0000 7fcceca76700 1 -- 172.21.15.46:0/363879016 wait complete. 2024-03-18T20:37:38.076 INFO:tasks.cephfs.filesystem:scrub status for tag:9619e554-3c82-42e0-9b0d-d584964bdef3 - {'path': '/', 'tag': '9619e554-3c82-42e0-9b0d-d584964bdef3', 'options': 'recursive,force'} 2024-03-18T20:37:38.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: pgmap v49: 97 pgs: 97 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.6 MiB/s wr, 158 op/s 2024-03-18T20:37:38.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:37:38.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:37:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:37:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:37:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:37:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:37:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:37:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:38 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: pgmap v49: 97 pgs: 97 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.6 MiB/s wr, 158 op/s 2024-03-18T20:37:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:37:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:37:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:37:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:37:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:37:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:37:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:37:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:38 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: pgmap v49: 97 pgs: 97 active+clean; 26 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 4.6 MiB/s wr, 158 op/s 2024-03-18T20:37:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:38 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:38.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 43388 7.60 MB/sec warmup 33 sec latency 741.627 ms 2024-03-18T20:37:39.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:39 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:37:39.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:39 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:37:39.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:39 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:39.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:39 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:39.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:39 smithi073 ceph-mon[47384]: mgrmap e25: x(active, since 91s), standbys: z, y 2024-03-18T20:37:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:39 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:37:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:39 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:37:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:39 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:39 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:39 smithi155 ceph-mon[48344]: mgrmap e25: x(active, since 91s), standbys: z, y 2024-03-18T20:37:39.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:39 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:37:39.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:39 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:37:39.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:39 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:39.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:39 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:37:39.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:39 smithi046 ceph-mon[43495]: mgrmap e25: x(active, since 91s), standbys: z, y 2024-03-18T20:37:39.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 43388 7.38 MB/sec warmup 34 sec latency 1741.718 ms 2024-03-18T20:37:40.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:40 smithi073 ceph-mon[47384]: pgmap v50: 97 pgs: 97 active+clean; 27 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 4.6 MiB/s wr, 183 op/s 2024-03-18T20:37:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:40 smithi155 ceph-mon[48344]: pgmap v50: 97 pgs: 97 active+clean; 27 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 4.6 MiB/s wr, 183 op/s 2024-03-18T20:37:40.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:40 smithi046 ceph-mon[43495]: pgmap v50: 97 pgs: 97 active+clean; 27 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 4.6 MiB/s wr, 183 op/s 2024-03-18T20:37:40.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 43388 7.17 MB/sec warmup 35 sec latency 2741.823 ms 2024-03-18T20:37:41.670 INFO:tasks.workunit.client.0.smithi046.stdout: 1 43388 6.97 MB/sec warmup 36 sec latency 3741.889 ms 2024-03-18T20:37:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:42 smithi073 ceph-mon[47384]: pgmap v51: 97 pgs: 97 active+clean; 27 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 4.6 MiB/s wr, 172 op/s 2024-03-18T20:37:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:42 smithi155 ceph-mon[48344]: pgmap v51: 97 pgs: 97 active+clean; 27 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 4.6 MiB/s wr, 172 op/s 2024-03-18T20:37:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:42 smithi046 ceph-mon[43495]: pgmap v51: 97 pgs: 97 active+clean; 27 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 4.6 MiB/s wr, 172 op/s 2024-03-18T20:37:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.548+0000 7f79fa11a700 1 -- 172.21.15.46:0/1305669069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f40ffbb0 msgr2=0x7f79f4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.548+0000 7f79fa11a700 1 --2- 172.21.15.46:0/1305669069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f40ffbb0 0x7f79f4100030 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009990 tx=0x7f79e8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.548+0000 7f79fa11a700 1 -- 172.21.15.46:0/1305669069 shutdown_connections 2024-03-18T20:37:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.548+0000 7f79fa11a700 1 --2- 172.21.15.46:0/1305669069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f4100570 0x7f79f410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.548+0000 7f79fa11a700 1 --2- 172.21.15.46:0/1305669069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f4105c60 0x7f79f4106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.549+0000 7f79fa11a700 1 --2- 172.21.15.46:0/1305669069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f40ffbb0 0x7f79f4100030 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.549+0000 7f79fa11a700 1 -- 172.21.15.46:0/1305669069 >> 172.21.15.46:0/1305669069 conn(0x7f79f4076d50 msgr2=0x7f79f4077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.549+0000 7f79fa11a700 1 -- 172.21.15.46:0/1305669069 shutdown_connections 2024-03-18T20:37:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.549+0000 7f79fa11a700 1 -- 172.21.15.46:0/1305669069 wait complete. 2024-03-18T20:37:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.549+0000 7f79fa11a700 1 Processor -- start 2024-03-18T20:37:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79fa11a700 1 -- start start 2024-03-18T20:37:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79fa11a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f40ffbb0 0x7f79f4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79fa11a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f4100570 0x7f79f4199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79fa11a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f4105c60 0x7f79f419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79fa11a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f419e260 con 0x7f79f4105c60 2024-03-18T20:37:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79fa11a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f419e3d0 con 0x7f79f40ffbb0 2024-03-18T20:37:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79fa11a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f419e590 con 0x7f79f4100570 2024-03-18T20:37:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f4105c60 0x7f79f419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f4105c60 0x7f79f419dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38570/0 (socket says 172.21.15.46:38570) 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79f3fff700 1 -- 172.21.15.46:0/537247861 learned_addr learned my addr 172.21.15.46:0/537247861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79f37fe700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f40ffbb0 0x7f79f4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79f2ffd700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f4100570 0x7f79f4199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79f3fff700 1 -- 172.21.15.46:0/537247861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f4100570 msgr2=0x7f79f4199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79f3fff700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f4100570 0x7f79f4199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79f3fff700 1 -- 172.21.15.46:0/537247861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f40ffbb0 msgr2=0x7f79f4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79f3fff700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f40ffbb0 0x7f79f4199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79f3fff700 1 -- 172.21.15.46:0/537247861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e80095f0 con 0x7f79f4105c60 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.550+0000 7f79f3fff700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f4105c60 0x7f79f419dc20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f79ec007ee0 tx=0x7f79ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.551+0000 7f79f0ff9700 1 -- 172.21.15.46:0/537247861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec017070 con 0x7f79f4105c60 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.551+0000 7f79fa11a700 1 -- 172.21.15.46:0/537247861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f419e870 con 0x7f79f4105c60 2024-03-18T20:37:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.551+0000 7f79f0ff9700 1 -- 172.21.15.46:0/537247861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79ec00f040 con 0x7f79f4105c60 2024-03-18T20:37:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.551+0000 7f79f0ff9700 1 -- 172.21.15.46:0/537247861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec005740 con 0x7f79f4105c60 2024-03-18T20:37:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.551+0000 7f79fa11a700 1 -- 172.21.15.46:0/537247861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f41a60b0 con 0x7f79f4105c60 2024-03-18T20:37:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.553+0000 7f79f0ff9700 1 -- 172.21.15.46:0/537247861 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f79ec00dc60 con 0x7f79f4105c60 2024-03-18T20:37:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.554+0000 7f79f0ff9700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79dc0a1290 0x7f79dc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.554+0000 7f79f37fe700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79dc0a1290 0x7f79dc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.554+0000 7f79f0ff9700 1 -- 172.21.15.46:0/537247861 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f79ec0cc520 con 0x7f79f4105c60 2024-03-18T20:37:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.554+0000 7f79f37fe700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79dc0a1290 0x7f79dc0a3750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f79e4005950 tx=0x7f79e400b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.554+0000 7f7a062afb80 1 Processor -- start 2024-03-18T20:37:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f7a062afb80 1 -- start start 2024-03-18T20:37:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f7a062afb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81b570 0x55f65d81b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f7a062afb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81be90 0x55f65d826870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f7a062afb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 0x55f65d8291a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f7a062afb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65d775930 con 0x55f65d826db0 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f7a062afb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65d7732f0 con 0x55f65d81b570 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f7a062afb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65d76ebe0 con 0x55f65d81be90 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f79f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 0x55f65d8291a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f79f37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81b570 0x55f65d81b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f79f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 0x55f65d8291a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38586/0 (socket says 172.21.15.46:38586) 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f79f3fff700 1 -- 172.21.15.46:0/167812420 learned_addr learned my addr 172.21.15.46:0/167812420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f79f2ffd700 1 --2- 172.21.15.46:0/167812420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81be90 0x55f65d826870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f79f3fff700 1 -- 172.21.15.46:0/167812420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81be90 msgr2=0x55f65d826870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f79f3fff700 1 --2- 172.21.15.46:0/167812420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81be90 0x55f65d826870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f79f3fff700 1 -- 172.21.15.46:0/167812420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81b570 msgr2=0x55f65d81b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f79f3fff700 1 --2- 172.21.15.46:0/167812420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81b570 0x55f65d81b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.555+0000 7f79f3fff700 1 -- 172.21.15.46:0/167812420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f65d5f0350 con 0x55f65d826db0 2024-03-18T20:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.556+0000 7f79f3fff700 1 --2- 172.21.15.46:0/167812420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 0x55f65d8291a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0a1550 tx=0x7f79ec004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c060b76b99d45d65 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.556+0000 7f79d9ffb700 1 -- 172.21.15.46:0/167812420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0b90d0 con 0x55f65d826db0 2024-03-18T20:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.556+0000 7f79d9ffb700 1 -- 172.21.15.46:0/167812420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79ec09d430 con 0x55f65d826db0 2024-03-18T20:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.556+0000 7f79d9ffb700 1 -- 172.21.15.46:0/167812420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0840c0 con 0x55f65d826db0 2024-03-18T20:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.556+0000 7f7a062afb80 1 -- 172.21.15.46:0/167812420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 msgr2=0x55f65d8291a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.556+0000 7f7a062afb80 1 --2- 172.21.15.46:0/167812420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 0x55f65d8291a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0a1550 tx=0x7f79ec004c40 comp rx=0 tx=0).stop 2024-03-18T20:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.556+0000 7f7a062afb80 1 -- 172.21.15.46:0/167812420 shutdown_connections 2024-03-18T20:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.556+0000 7f7a062afb80 1 --2- 172.21.15.46:0/167812420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 0x55f65d8291a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.556+0000 7f7a062afb80 1 --2- 172.21.15.46:0/167812420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81b570 0x55f65d81b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.556+0000 7f7a062afb80 1 --2- 172.21.15.46:0/167812420 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81be90 0x55f65d826870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.556+0000 7f7a062afb80 1 -- 172.21.15.46:0/167812420 >> 172.21.15.46:0/167812420 conn(0x55f65d7921c0 msgr2=0x55f65d817a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.557+0000 7f7a062afb80 1 -- 172.21.15.46:0/167812420 shutdown_connections 2024-03-18T20:37:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.557+0000 7f7a062afb80 1 -- 172.21.15.46:0/167812420 wait complete. 2024-03-18T20:37:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.557+0000 7f7a062afb80 1 -- start start 2024-03-18T20:37:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.557+0000 7f7a062afb80 1 Processor -- start 2024-03-18T20:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.557+0000 7f7a062afb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81b570 0x55f65d8267b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.557+0000 7f7a062afb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 0x55f65d823aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f7a062afb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d826db0 0x55f65d824120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f7a062afb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65d70ad20 con 0x55f65d81be90 2024-03-18T20:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f7a062afb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65d70a9c0 con 0x55f65d81b570 2024-03-18T20:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f7a062afb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65d5f2640 con 0x55f65d826db0 2024-03-18T20:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f7a062afb80 10 client.0 _ll_register_callbacks cb 0x55f65d576340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 0x55f65d823aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 0x55f65d823aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38590/0 (socket says 172.21.15.46:38590) 2024-03-18T20:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79f2ffd700 1 -- 172.21.15.46:0/1894038355 learned_addr learned my addr 172.21.15.46:0/1894038355 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79f37fe700 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81b570 0x55f65d8267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79f3fff700 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d826db0 0x55f65d824120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79f2ffd700 1 -- 172.21.15.46:0/1894038355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d826db0 msgr2=0x55f65d824120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79f2ffd700 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d826db0 0x55f65d824120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79f2ffd700 1 -- 172.21.15.46:0/1894038355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81b570 msgr2=0x55f65d8267b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79f2ffd700 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81b570 0x55f65d8267b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79f2ffd700 1 -- 172.21.15.46:0/1894038355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f41a60b0 con 0x55f65d81be90 2024-03-18T20:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79f2ffd700 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 0x55f65d823aa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009990 tx=0x7f79e8009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cb3406285d69ec server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79d2ffd700 1 -- 172.21.15.46:0/1894038355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e801d050 con 0x55f65d81be90 2024-03-18T20:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f7a062afb80 1 -- 172.21.15.46:0/1894038355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f65d80f6c0 con 0x55f65d81be90 2024-03-18T20:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.558+0000 7f79d2ffd700 1 -- 172.21.15.46:0/1894038355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f79e80055e0 con 0x55f65d81be90 2024-03-18T20:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.559+0000 7f79d2ffd700 1 -- 172.21.15.46:0/1894038355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79e8003790 con 0x55f65d81be90 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.559+0000 7f7a062afb80 10 client.16140 fetch_fsmap learned FSMap version 32 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.559+0000 7f7a062afb80 1 -- 172.21.15.46:0/1894038355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f65d59cb00 con 0x55f65d81be90 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.559+0000 7f79d2ffd700 1 -- 172.21.15.46:0/1894038355 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f65d80f6c0 con 0x55f65d81be90 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.559+0000 7f79d2ffd700 1 -- 172.21.15.46:0/1894038355 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f79e80239b0 con 0x55f65d81be90 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.559+0000 7f7a062afb80 10 client.16140 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.559+0000 7f7a062afb80 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.559+0000 7f7a062afb80 20 client.16140 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.559+0000 7f7a062afb80 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f65d836ac0 0x55f65d838ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.559+0000 7f7a062afb80 4 client.16140 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.559+0000 7f7a062afb80 1 -- 172.21.15.46:0/1894038355 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f65d7676a0 con 0x55f65d836ac0 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.559+0000 7f79f37fe700 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f65d836ac0 0x55f65d838ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.560+0000 7f79f37fe700 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f65d836ac0 0x55f65d838ee0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1867d3482f5c825 server_cookie=a93f7df3e9d0ff54 in_seq=0 out_seq=0 2024-03-18T20:37:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.560+0000 7f79d2ffd700 10 client.16140 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.560+0000 7f79d2ffd700 1 -- 172.21.15.46:0/1894038355 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f65d7676a0 con 0x55f65d836ac0 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.560+0000 7f79d2ffd700 10 client.16140 handle_command_reply: tid=0 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.560+0000 7f7a062afb80 1 client.16140 shutdown 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.561+0000 7f7a062afb80 1 -- 172.21.15.46:0/1894038355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 msgr2=0x55f65d823aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.561+0000 7f7a062afb80 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 0x55f65d823aa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009990 tx=0x7f79e8009ee0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.561+0000 7f7a062afb80 1 -- 172.21.15.46:0/1894038355 shutdown_connections 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.561+0000 7f7a062afb80 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f65d836ac0 0x55f65d838ee0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.561+0000 7f7a062afb80 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d826db0 0x55f65d824120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.561+0000 7f7a062afb80 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81b570 0x55f65d8267b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.561+0000 7f79d2ffd700 0 client.16140 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.561+0000 7f7a062afb80 1 --2- 172.21.15.46:0/1894038355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 0x55f65d823aa0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.561+0000 7f7a062afb80 1 -- 172.21.15.46:0/1894038355 >> 172.21.15.46:0/1894038355 conn(0x55f65d7921c0 msgr2=0x55f65d817650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.561+0000 7f7a062afb80 1 -- 172.21.15.46:0/1894038355 shutdown_connections 2024-03-18T20:37:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.561+0000 7f7a062afb80 1 -- 172.21.15.46:0/1894038355 wait complete. 2024-03-18T20:37:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.561+0000 7f7a062afb80 20 client.16140 trim_cache size 0 max 16384 2024-03-18T20:37:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.576+0000 7f7a062afb80 1 Processor -- start 2024-03-18T20:37:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.576+0000 7f7a062afb80 1 -- start start 2024-03-18T20:37:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f7a062afb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81b570 0x55f65d832190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f7a062afb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 0x55f65d8326d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f7a062afb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d826db0 0x55f65d820d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f7a062afb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65d81d070 con 0x55f65d81be90 2024-03-18T20:37:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f7a062afb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65d811c10 con 0x55f65d826db0 2024-03-18T20:37:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f7a062afb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65d818f70 con 0x55f65d81b570 2024-03-18T20:37:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 0x55f65d8326d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 0x55f65d8326d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38600/0 (socket says 172.21.15.46:38600) 2024-03-18T20:37:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79f2ffd700 1 -- 172.21.15.46:0/4038353684 learned_addr learned my addr 172.21.15.46:0/4038353684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79f37fe700 1 --2- 172.21.15.46:0/4038353684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81b570 0x55f65d832190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79f3fff700 1 --2- 172.21.15.46:0/4038353684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d826db0 0x55f65d820d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79f2ffd700 1 -- 172.21.15.46:0/4038353684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81b570 msgr2=0x55f65d832190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79f2ffd700 1 --2- 172.21.15.46:0/4038353684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81b570 0x55f65d832190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79f2ffd700 1 -- 172.21.15.46:0/4038353684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d826db0 msgr2=0x55f65d820d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79f2ffd700 1 --2- 172.21.15.46:0/4038353684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d826db0 0x55f65d820d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79f2ffd700 1 -- 172.21.15.46:0/4038353684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f65d734e40 con 0x55f65d81be90 2024-03-18T20:37:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79f37fe700 1 --2- 172.21.15.46:0/4038353684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81b570 0x55f65d832190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79f2ffd700 1 --2- 172.21.15.46:0/4038353684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 0x55f65d8326d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f79e800c010 tx=0x7f79e8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f887a66bc1633049 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.577+0000 7f79da7fc700 1 -- 172.21.15.46:0/4038353684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e801d040 con 0x55f65d81be90 2024-03-18T20:37:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.578+0000 7f79da7fc700 1 -- 172.21.15.46:0/4038353684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79e801e040 con 0x55f65d81be90 2024-03-18T20:37:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.578+0000 7f79da7fc700 1 -- 172.21.15.46:0/4038353684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80039f0 con 0x55f65d81be90 2024-03-18T20:37:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.578+0000 7f7a062afb80 1 -- 172.21.15.46:0/4038353684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 msgr2=0x55f65d8326d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.578+0000 7f7a062afb80 1 --2- 172.21.15.46:0/4038353684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 0x55f65d8326d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f79e800c010 tx=0x7f79e8005e80 comp rx=0 tx=0).stop 2024-03-18T20:37:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.578+0000 7f7a062afb80 1 -- 172.21.15.46:0/4038353684 shutdown_connections 2024-03-18T20:37:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.578+0000 7f7a062afb80 1 --2- 172.21.15.46:0/4038353684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d826db0 0x55f65d820d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.578+0000 7f7a062afb80 1 --2- 172.21.15.46:0/4038353684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81b570 0x55f65d832190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.578+0000 7f7a062afb80 1 --2- 172.21.15.46:0/4038353684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d81be90 0x55f65d8326d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.578+0000 7f7a062afb80 1 -- 172.21.15.46:0/4038353684 >> 172.21.15.46:0/4038353684 conn(0x55f65d7921c0 msgr2=0x55f65d8223b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.578+0000 7f7a062afb80 1 -- 172.21.15.46:0/4038353684 shutdown_connections 2024-03-18T20:37:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.578+0000 7f7a062afb80 1 -- 172.21.15.46:0/4038353684 wait complete. 2024-03-18T20:37:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.578+0000 7f7a062afb80 1 -- start start 2024-03-18T20:37:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f7a062afb80 1 Processor -- start 2024-03-18T20:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f7a062afb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81b570 0x55f65d82ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f7a062afb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81be90 0x55f65d82bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f7a062afb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 0x55f65d82c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f7a062afb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65d8898b0 con 0x55f65d826db0 2024-03-18T20:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f7a062afb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65d8899f0 con 0x55f65d81be90 2024-03-18T20:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f7a062afb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f65d889b80 con 0x55f65d81b570 2024-03-18T20:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f7a062afb80 10 client.0 _ll_register_callbacks cb 0x7f79e8009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f79f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 0x55f65d82c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f79f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 0x55f65d82c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38616/0 (socket says 172.21.15.46:38616) 2024-03-18T20:37:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f79f3fff700 1 -- 172.21.15.46:0/2678088521 learned_addr learned my addr 172.21.15.46:0/2678088521 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f79f2ffd700 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81be90 0x55f65d82bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.579+0000 7f79f37fe700 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81b570 0x55f65d82ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79f3fff700 1 -- 172.21.15.46:0/2678088521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81b570 msgr2=0x55f65d82ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79f3fff700 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81b570 0x55f65d82ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79f3fff700 1 -- 172.21.15.46:0/2678088521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81be90 msgr2=0x55f65d82bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79f3fff700 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81be90 0x55f65d82bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79f3fff700 1 -- 172.21.15.46:0/2678088521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f65d59cb00 con 0x55f65d826db0 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79f2ffd700 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81be90 0x55f65d82bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79f37fe700 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81b570 0x55f65d82ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79f3fff700 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 0x55f65d82c4a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009510 tx=0x7f79ec007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23575bc36b35f6ea server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79d2ffd700 1 -- 172.21.15.46:0/2678088521 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0b90d0 con 0x55f65d826db0 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79d2ffd700 1 -- 172.21.15.46:0/2678088521 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f79ec0a1a80 con 0x55f65d826db0 2024-03-18T20:37:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f7a062afb80 1 -- 172.21.15.46:0/2678088521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79ec09d430 con 0x55f65d826db0 2024-03-18T20:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79d2ffd700 1 -- 172.21.15.46:0/2678088521 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79ec0058a0 con 0x55f65d826db0 2024-03-18T20:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f7a062afb80 10 client.16146 fetch_fsmap learned FSMap version 32 2024-03-18T20:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f7a062afb80 1 -- 172.21.15.46:0/2678088521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f65d5f4c70 con 0x55f65d826db0 2024-03-18T20:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.580+0000 7f79d2ffd700 1 -- 172.21.15.46:0/2678088521 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79ec09d430 con 0x55f65d826db0 2024-03-18T20:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.581+0000 7f79d2ffd700 1 -- 172.21.15.46:0/2678088521 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f79ec0b5070 con 0x55f65d826db0 2024-03-18T20:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.581+0000 7f7a062afb80 10 client.16146 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T20:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.581+0000 7f7a062afb80 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.581+0000 7f7a062afb80 20 client.16146 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.581+0000 7f7a062afb80 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f65d88b050 0x55f65d88d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.581+0000 7f7a062afb80 4 client.16146 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.581+0000 7f7a062afb80 1 -- 172.21.15.46:0/2678088521 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f65d5bdc90 con 0x55f65d88b050 2024-03-18T20:37:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.581+0000 7f79f37fe700 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f65d88b050 0x55f65d88d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.581+0000 7f79f37fe700 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f65d88b050 0x55f65d88d470 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=821118c6d8a10dd6 server_cookie=ab9ee2ffea195b3b in_seq=0 out_seq=0 2024-03-18T20:37:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.581+0000 7f79d2ffd700 10 client.16146 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.608+0000 7f79d2ffd700 1 -- 172.21.15.46:0/2678088521 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f65d5bdc90 con 0x55f65d88b050 2024-03-18T20:37:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.608+0000 7f79d2ffd700 10 client.16146 handle_command_reply: tid=0 2024-03-18T20:37:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.608+0000 7f7a062afb80 1 client.16146 shutdown 2024-03-18T20:37:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.609+0000 7f7a062afb80 1 -- 172.21.15.46:0/2678088521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 msgr2=0x55f65d82c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.609+0000 7f7a062afb80 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 0x55f65d82c4a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009510 tx=0x7f79ec007f10 comp rx=0 tx=0).stop 2024-03-18T20:37:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.609+0000 7f7a062afb80 1 -- 172.21.15.46:0/2678088521 shutdown_connections 2024-03-18T20:37:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.609+0000 7f7a062afb80 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f65d88b050 0x55f65d88d470 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.609+0000 7f7a062afb80 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f65d826db0 0x55f65d82c4a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.609+0000 7f7a062afb80 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f65d81b570 0x55f65d82ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.609+0000 7f7a062afb80 1 --2- 172.21.15.46:0/2678088521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f65d81be90 0x55f65d82bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.609+0000 7f79d2ffd700 0 client.16146 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.609+0000 7f7a062afb80 1 -- 172.21.15.46:0/2678088521 >> 172.21.15.46:0/2678088521 conn(0x55f65d7921c0 msgr2=0x55f65d832a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.609+0000 7f7a062afb80 1 -- 172.21.15.46:0/2678088521 shutdown_connections 2024-03-18T20:37:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.609+0000 7f7a062afb80 1 -- 172.21.15.46:0/2678088521 wait complete. 2024-03-18T20:37:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.609+0000 7f7a062afb80 20 client.16146 trim_cache size 0 max 16384 2024-03-18T20:37:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:37:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:37:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:37:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:37:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.610+0000 7f79fa11a700 1 -- 172.21.15.46:0/537247861 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79dc0a1290 msgr2=0x7f79dc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.610+0000 7f79fa11a700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79dc0a1290 0x7f79dc0a3750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f79e4005950 tx=0x7f79e400b410 comp rx=0 tx=0).stop 2024-03-18T20:37:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.610+0000 7f79fa11a700 1 -- 172.21.15.46:0/537247861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f4105c60 msgr2=0x7f79f419dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.610+0000 7f79fa11a700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f4105c60 0x7f79f419dc20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f79ec007ee0 tx=0x7f79ec00d3b0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.610+0000 7f79fa11a700 1 -- 172.21.15.46:0/537247861 shutdown_connections 2024-03-18T20:37:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.610+0000 7f79fa11a700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79dc0a1290 0x7f79dc0a3750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.610+0000 7f79fa11a700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f4105c60 0x7f79f419dc20 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.610+0000 7f79fa11a700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f40ffbb0 0x7f79f4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.610+0000 7f79fa11a700 1 --2- 172.21.15.46:0/537247861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f4100570 0x7f79f4199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.610+0000 7f79fa11a700 1 -- 172.21.15.46:0/537247861 >> 172.21.15.46:0/537247861 conn(0x7f79f4076d50 msgr2=0x7f79f4068080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.610+0000 7f79fa11a700 1 -- 172.21.15.46:0/537247861 shutdown_connections 2024-03-18T20:37:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:42.610+0000 7f79fa11a700 1 -- 172.21.15.46:0/537247861 wait complete. 2024-03-18T20:37:42.671 INFO:tasks.workunit.client.0.smithi046.stdout: 1 43576 6.79 MB/sec warmup 37 sec latency 4104.008 ms 2024-03-18T20:37:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:43 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:43 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:37:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:43 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:37:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:43 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:43 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:37:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:43 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:37:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:43 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:43 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:37:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:43 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:37:43.671 INFO:tasks.workunit.client.0.smithi046.stdout: 1 43576 6.61 MB/sec warmup 38 sec latency 1507.499 ms 2024-03-18T20:37:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:44 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:37:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:44 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:37:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:44 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:37:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:44 smithi073 ceph-mon[47384]: pgmap v52: 97 pgs: 97 active+clean; 25 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 4.6 MiB/s wr, 195 op/s 2024-03-18T20:37:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:44 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:37:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:44 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:37:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:44 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:37:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:44 smithi155 ceph-mon[48344]: pgmap v52: 97 pgs: 97 active+clean; 25 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 4.6 MiB/s wr, 195 op/s 2024-03-18T20:37:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:44 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:37:44.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:44 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:37:44.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:44 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:37:44.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:44 smithi046 ceph-mon[43495]: pgmap v52: 97 pgs: 97 active+clean; 25 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 4.6 MiB/s wr, 195 op/s 2024-03-18T20:37:44.671 INFO:tasks.workunit.client.0.smithi046.stdout: 1 43576 6.44 MB/sec warmup 39 sec latency 2507.595 ms 2024-03-18T20:37:45.671 INFO:tasks.workunit.client.0.smithi046.stdout: 1 43576 6.28 MB/sec warmup 40 sec latency 3507.647 ms 2024-03-18T20:37:46.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:45 smithi073 ceph-mon[47384]: pgmap v53: 97 pgs: 97 active+clean; 25 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 46 op/s 2024-03-18T20:37:46.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:45 smithi155 ceph-mon[48344]: pgmap v53: 97 pgs: 97 active+clean; 25 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 46 op/s 2024-03-18T20:37:46.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:45 smithi046 ceph-mon[43495]: pgmap v53: 97 pgs: 97 active+clean; 25 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 46 op/s 2024-03-18T20:37:46.671 INFO:tasks.workunit.client.0.smithi046.stdout: 1 43576 6.13 MB/sec warmup 41 sec latency 4507.755 ms 2024-03-18T20:37:47.671 INFO:tasks.workunit.client.0.smithi046.stdout: 1 45475 6.23 MB/sec warmup 42 sec latency 4875.343 ms 2024-03-18T20:37:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:48 smithi073 ceph-mon[47384]: pgmap v54: 97 pgs: 97 active+clean; 25 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 46 op/s 2024-03-18T20:37:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:48 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:48 smithi155 ceph-mon[48344]: pgmap v54: 97 pgs: 97 active+clean; 25 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 46 op/s 2024-03-18T20:37:48.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:48 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:48 smithi046 ceph-mon[43495]: pgmap v54: 97 pgs: 97 active+clean; 25 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 46 op/s 2024-03-18T20:37:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:48 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:37:48.671 INFO:tasks.workunit.client.0.smithi046.stdout: 1 45526 6.12 MB/sec warmup 43 sec latency 954.065 ms 2024-03-18T20:37:49.671 INFO:tasks.workunit.client.0.smithi046.stdout: 1 45804 5.99 MB/sec warmup 44 sec latency 1458.320 ms 2024-03-18T20:37:50.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:50 smithi073 ceph-mon[47384]: pgmap v55: 97 pgs: 97 active+clean; 25 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 172 KiB/s wr, 68 op/s 2024-03-18T20:37:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:50 smithi155 ceph-mon[48344]: pgmap v55: 97 pgs: 97 active+clean; 25 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 172 KiB/s wr, 68 op/s 2024-03-18T20:37:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:50 smithi046 ceph-mon[43495]: pgmap v55: 97 pgs: 97 active+clean; 25 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 172 KiB/s wr, 68 op/s 2024-03-18T20:37:50.671 INFO:tasks.workunit.client.0.smithi046.stdout: 1 45804 5.86 MB/sec warmup 45 sec latency 1370.009 ms 2024-03-18T20:37:51.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 45804 5.73 MB/sec warmup 46 sec latency 2370.130 ms 2024-03-18T20:37:52.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:52 smithi073 ceph-mon[47384]: pgmap v56: 97 pgs: 97 active+clean; 25 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 119 KiB/s wr, 43 op/s 2024-03-18T20:37:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:52 smithi155 ceph-mon[48344]: pgmap v56: 97 pgs: 97 active+clean; 25 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 119 KiB/s wr, 43 op/s 2024-03-18T20:37:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:52 smithi046 ceph-mon[43495]: pgmap v56: 97 pgs: 97 active+clean; 25 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 119 KiB/s wr, 43 op/s 2024-03-18T20:37:52.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 45804 5.61 MB/sec warmup 47 sec latency 3370.234 ms 2024-03-18T20:37:53.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 45804 5.50 MB/sec warmup 48 sec latency 4370.347 ms 2024-03-18T20:37:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:54 smithi073 ceph-mon[47384]: pgmap v57: 97 pgs: 97 active+clean; 26 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 135 KiB/s wr, 53 op/s 2024-03-18T20:37:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:54 smithi155 ceph-mon[48344]: pgmap v57: 97 pgs: 97 active+clean; 26 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 135 KiB/s wr, 53 op/s 2024-03-18T20:37:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:54 smithi046 ceph-mon[43495]: pgmap v57: 97 pgs: 97 active+clean; 26 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 135 KiB/s wr, 53 op/s 2024-03-18T20:37:54.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 46516 5.50 MB/sec warmup 49 sec latency 4878.295 ms 2024-03-18T20:37:55.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 46938 5.40 MB/sec warmup 50 sec latency 620.272 ms 2024-03-18T20:37:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.735+0000 7f0a46d57700 1 -- 172.21.15.46:0/3097802008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a40105c60 msgr2=0x7f0a40106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.735+0000 7f0a46d57700 1 --2- 172.21.15.46:0/3097802008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a40105c60 0x7f0a40106040 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009990 tx=0x7f0a34009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.735+0000 7f0a46d57700 1 -- 172.21.15.46:0/3097802008 shutdown_connections 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.735+0000 7f0a46d57700 1 --2- 172.21.15.46:0/3097802008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a40105c60 0x7f0a40106040 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.735+0000 7f0a46d57700 1 --2- 172.21.15.46:0/3097802008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a40100570 0x7f0a4010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.735+0000 7f0a46d57700 1 --2- 172.21.15.46:0/3097802008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a400ffbb0 0x7f0a40100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.735+0000 7f0a46d57700 1 -- 172.21.15.46:0/3097802008 >> 172.21.15.46:0/3097802008 conn(0x7f0a40076d50 msgr2=0x7f0a40077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.735+0000 7f0a46d57700 1 -- 172.21.15.46:0/3097802008 shutdown_connections 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.735+0000 7f0a46d57700 1 -- 172.21.15.46:0/3097802008 wait complete. 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a46d57700 1 Processor -- start 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a46d57700 1 -- start start 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a46d57700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a400ffbb0 0x7f0a40197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a46d57700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a40100570 0x7f0a401976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a46d57700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a40105c60 0x7f0a4019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a46d57700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4019bfd0 con 0x7f0a40100570 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a46d57700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4019c140 con 0x7f0a400ffbb0 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a46d57700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4019c300 con 0x7f0a40105c60 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a3ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a40100570 0x7f0a401976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a3ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a40100570 0x7f0a401976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43914/0 (socket says 172.21.15.46:43914) 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a3ffff700 1 -- 172.21.15.46:0/85045419 learned_addr learned my addr 172.21.15.46:0/85045419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a44af3700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a400ffbb0 0x7f0a40197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.736+0000 7f0a452f4700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a40105c60 0x7f0a4019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a3ffff700 1 -- 172.21.15.46:0/85045419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a40105c60 msgr2=0x7f0a4019b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a40105c60 0x7f0a4019b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a3ffff700 1 -- 172.21.15.46:0/85045419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a400ffbb0 msgr2=0x7f0a40197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a400ffbb0 0x7f0a40197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a3ffff700 1 -- 172.21.15.46:0/85045419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a340095f0 con 0x7f0a40100570 2024-03-18T20:37:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a452f4700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a40105c60 0x7f0a4019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:37:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a44af3700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a400ffbb0 0x7f0a40197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a40100570 0x7f0a401976b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00d950 tx=0x7f0a2c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a3dffb700 1 -- 172.21.15.46:0/85045419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c009b40 con 0x7f0a40100570 2024-03-18T20:37:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a3dffb700 1 -- 172.21.15.46:0/85045419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a2c009ca0 con 0x7f0a40100570 2024-03-18T20:37:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a46d57700 1 -- 172.21.15.46:0/85045419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4019c5e0 con 0x7f0a40100570 2024-03-18T20:37:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a3dffb700 1 -- 172.21.15.46:0/85045419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c00f620 con 0x7f0a40100570 2024-03-18T20:37:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.737+0000 7f0a46d57700 1 -- 172.21.15.46:0/85045419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a401a3e40 con 0x7f0a40100570 2024-03-18T20:37:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.739+0000 7f0a3dffb700 1 -- 172.21.15.46:0/85045419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0a2c010460 con 0x7f0a40100570 2024-03-18T20:37:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.740+0000 7f0a3dffb700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0a300a1290 0x7f0a300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.740+0000 7f0a44af3700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0a300a1290 0x7f0a300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.740+0000 7f0a3dffb700 1 -- 172.21.15.46:0/85045419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0a2c0cd0b0 con 0x7f0a40100570 2024-03-18T20:37:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.741+0000 7f0a44af3700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0a300a1290 0x7f0a300a3750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a34006010 tx=0x7f0a3400b260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.741+0000 7f0a52eecb80 1 Processor -- start 2024-03-18T20:37:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.741+0000 7f0a52eecb80 1 -- start start 2024-03-18T20:37:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.741+0000 7f0a52eecb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c69570 0x563330c69950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.741+0000 7f0a52eecb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69e90 0x563330c74870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.741+0000 7f0a52eecb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c74db0 0x563330c771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.741+0000 7f0a52eecb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563330bc3930 con 0x563330c69e90 2024-03-18T20:37:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.741+0000 7f0a52eecb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563330bc12f0 con 0x563330c74db0 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.741+0000 7f0a52eecb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563330bbcbe0 con 0x563330c69570 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.741+0000 7f0a3ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69e90 0x563330c74870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.741+0000 7f0a3ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69e90 0x563330c74870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43928/0 (socket says 172.21.15.46:43928) 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.741+0000 7f0a3ffff700 1 -- 172.21.15.46:0/3252427591 learned_addr learned my addr 172.21.15.46:0/3252427591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.742+0000 7f0a44af3700 1 --2- 172.21.15.46:0/3252427591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c69570 0x563330c69950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.742+0000 7f0a452f4700 1 --2- 172.21.15.46:0/3252427591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c74db0 0x563330c771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.742+0000 7f0a3ffff700 1 -- 172.21.15.46:0/3252427591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c69570 msgr2=0x563330c69950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.742+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/3252427591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c69570 0x563330c69950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.742+0000 7f0a3ffff700 1 -- 172.21.15.46:0/3252427591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c74db0 msgr2=0x563330c771a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.742+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/3252427591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c74db0 0x563330c771a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.742+0000 7f0a3ffff700 1 -- 172.21.15.46:0/3252427591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563330a3e350 con 0x563330c69e90 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.742+0000 7f0a452f4700 1 --2- 172.21.15.46:0/3252427591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c74db0 0x563330c771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:37:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.742+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/3252427591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69e90 0x563330c74870 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0a20e0 tx=0x7f0a2c099dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53236c9da8d359e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.742+0000 7f0a2affd700 1 -- 172.21.15.46:0/3252427591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c09a190 con 0x563330c69e90 2024-03-18T20:37:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.742+0000 7f0a2affd700 1 -- 172.21.15.46:0/3252427591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a2c09dfc0 con 0x563330c69e90 2024-03-18T20:37:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.742+0000 7f0a2affd700 1 -- 172.21.15.46:0/3252427591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c09a3f0 con 0x563330c69e90 2024-03-18T20:37:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.743+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3252427591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69e90 msgr2=0x563330c74870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.743+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/3252427591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69e90 0x563330c74870 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0a20e0 tx=0x7f0a2c099dd0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.743+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3252427591 shutdown_connections 2024-03-18T20:37:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.743+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/3252427591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c74db0 0x563330c771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.743+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/3252427591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c69570 0x563330c69950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.743+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/3252427591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69e90 0x563330c74870 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.743+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3252427591 >> 172.21.15.46:0/3252427591 conn(0x563330be01c0 msgr2=0x563330c65a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.743+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3252427591 shutdown_connections 2024-03-18T20:37:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.743+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3252427591 wait complete. 2024-03-18T20:37:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.743+0000 7f0a52eecb80 1 -- start start 2024-03-18T20:37:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.743+0000 7f0a52eecb80 1 Processor -- start 2024-03-18T20:37:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a52eecb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 0x563330c71b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a52eecb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a52eecb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c74db0 0x563330c725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a52eecb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563330b58d20 con 0x563330c69570 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a52eecb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563330b589c0 con 0x563330c69e90 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a52eecb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563330a40640 con 0x563330c74db0 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a52eecb80 10 client.0 _ll_register_callbacks cb 0x5633309c4340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a44af3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 0x563330c71b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a44af3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 0x563330c71b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43940/0 (socket says 172.21.15.46:43940) 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a44af3700 1 -- 172.21.15.46:0/3273481259 learned_addr learned my addr 172.21.15.46:0/3273481259 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a452f4700 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c74db0 0x563330c725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a44af3700 1 -- 172.21.15.46:0/3273481259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c74db0 msgr2=0x563330c725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a44af3700 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c74db0 0x563330c725f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a44af3700 1 -- 172.21.15.46:0/3273481259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 msgr2=0x563330c720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a44af3700 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.744+0000 7f0a44af3700 1 -- 172.21.15.46:0/3273481259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a401a3e40 con 0x563330c69570 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a44af3700 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 0x563330c71b70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400be80 tx=0x7f0a34009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4b333092f15c77 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a1ffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a1ffff700 1 -- 172.21.15.46:0/3273481259 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34004a90 con 0x563330c69570 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a1ffff700 1 -- 172.21.15.46:0/3273481259 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0a34004bf0 con 0x563330c69570 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3273481259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563330c5d6c0 con 0x563330c69570 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a1ffff700 1 -- 172.21.15.46:0/3273481259 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a34004db0 con 0x563330c69570 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a52eecb80 10 client.16158 fetch_fsmap learned FSMap version 33 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3273481259 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5633309eab00 con 0x563330c69570 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a1ffff700 1 -- 172.21.15.46:0/3273481259 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a3403d020 con 0x563330c69570 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a1ffff700 1 -- 172.21.15.46:0/3273481259 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0a340244a0 con 0x563330c69570 2024-03-18T20:37:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a52eecb80 10 client.16158 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a52eecb80 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a52eecb80 20 client.16158 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563330c84a50 0x563330c86e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a52eecb80 4 client.16158 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.745+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3273481259 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563330bb56a0 con 0x563330c84a50 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.746+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563330c84a50 0x563330c86e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.746+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563330c84a50 0x563330c86e70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae1de31d1fde3d2c server_cookie=c3be8d8a978622cf in_seq=0 out_seq=0 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.746+0000 7f0a1ffff700 10 client.16158 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.746+0000 7f0a1ffff700 1 -- 172.21.15.46:0/3273481259 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563330bb56a0 con 0x563330c84a50 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.746+0000 7f0a1ffff700 10 client.16158 handle_command_reply: tid=0 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.746+0000 7f0a52eecb80 1 client.16158 shutdown 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.747+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3273481259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 msgr2=0x563330c71b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.747+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 0x563330c71b70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400be80 tx=0x7f0a34009ca0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.747+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3273481259 shutdown_connections 2024-03-18T20:37:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.747+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563330c84a50 0x563330c86e70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.747+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 0x563330c71b70 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.747+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c74db0 0x563330c725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.747+0000 7f0a1ffff700 0 client.16158 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.747+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/3273481259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.747+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3273481259 >> 172.21.15.46:0/3273481259 conn(0x563330be01c0 msgr2=0x563330c65650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.747+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3273481259 shutdown_connections 2024-03-18T20:37:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.747+0000 7f0a52eecb80 1 -- 172.21.15.46:0/3273481259 wait complete. 2024-03-18T20:37:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.747+0000 7f0a52eecb80 20 client.16158 trim_cache size 0 max 16384 2024-03-18T20:37:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.762+0000 7f0a52eecb80 1 Processor -- start 2024-03-18T20:37:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.762+0000 7f0a52eecb80 1 -- start start 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a52eecb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c69570 0x563330c771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a52eecb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c7b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a52eecb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c74db0 0x563330c6cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a52eecb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563330c6ca60 con 0x563330c74db0 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a52eecb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563330c66d60 con 0x563330c69e90 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a52eecb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563330c7bf20 con 0x563330c69570 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a452f4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c74db0 0x563330c6cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a452f4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c74db0 0x563330c6cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43948/0 (socket says 172.21.15.46:43948) 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a452f4700 1 -- 172.21.15.46:0/2033389252 learned_addr learned my addr 172.21.15.46:0/2033389252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/2033389252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c7b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a44af3700 1 --2- 172.21.15.46:0/2033389252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c69570 0x563330c771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a452f4700 1 -- 172.21.15.46:0/2033389252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c69570 msgr2=0x563330c771a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a452f4700 1 --2- 172.21.15.46:0/2033389252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c69570 0x563330c771a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a452f4700 1 -- 172.21.15.46:0/2033389252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 msgr2=0x563330c7b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a452f4700 1 --2- 172.21.15.46:0/2033389252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c7b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a452f4700 1 -- 172.21.15.46:0/2033389252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563330b82e40 con 0x563330c74db0 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/2033389252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c7b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a452f4700 1 --2- 172.21.15.46:0/2033389252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c74db0 0x563330c6cf10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0a380084d0 tx=0x7f0a3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81293b236f305125 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a2b7fe700 1 -- 172.21.15.46:0/2033389252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a38016660 con 0x563330c74db0 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a2b7fe700 1 -- 172.21.15.46:0/2033389252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a3800f070 con 0x563330c74db0 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.763+0000 7f0a2b7fe700 1 -- 172.21.15.46:0/2033389252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a38007b90 con 0x563330c74db0 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.764+0000 7f0a52eecb80 1 -- 172.21.15.46:0/2033389252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c74db0 msgr2=0x563330c6cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.764+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/2033389252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c74db0 0x563330c6cf10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0a380084d0 tx=0x7f0a3800b040 comp rx=0 tx=0).stop 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.764+0000 7f0a52eecb80 1 -- 172.21.15.46:0/2033389252 shutdown_connections 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.764+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/2033389252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c74db0 0x563330c6cf10 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.764+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/2033389252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c69570 0x563330c771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.764+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/2033389252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c7b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.764+0000 7f0a52eecb80 1 -- 172.21.15.46:0/2033389252 >> 172.21.15.46:0/2033389252 conn(0x563330be01c0 msgr2=0x563330c64bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.764+0000 7f0a52eecb80 1 -- 172.21.15.46:0/2033389252 shutdown_connections 2024-03-18T20:37:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.764+0000 7f0a52eecb80 1 -- 172.21.15.46:0/2033389252 wait complete. 2024-03-18T20:37:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.764+0000 7f0a52eecb80 1 -- start start 2024-03-18T20:37:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.764+0000 7f0a52eecb80 1 Processor -- start 2024-03-18T20:37:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a52eecb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 0x563330c79bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a52eecb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c7a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a52eecb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c74db0 0x563330cd1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a52eecb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563330cd1d90 con 0x563330c69570 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a52eecb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563330cd1ed0 con 0x563330c69e90 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a52eecb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563330cd2090 con 0x563330c74db0 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a52eecb80 10 client.0 _ll_register_callbacks cb 0x7f0a3400ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a44af3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 0x563330c79bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a44af3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 0x563330c79bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43962/0 (socket says 172.21.15.46:43962) 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a3ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c7a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a44af3700 1 -- 172.21.15.46:0/750168422 learned_addr learned my addr 172.21.15.46:0/750168422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a44af3700 1 -- 172.21.15.46:0/750168422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c74db0 msgr2=0x563330cd1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a44af3700 1 --2- 172.21.15.46:0/750168422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c74db0 0x563330cd1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a44af3700 1 -- 172.21.15.46:0/750168422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 msgr2=0x563330c7a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a44af3700 1 --2- 172.21.15.46:0/750168422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c7a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a44af3700 1 -- 172.21.15.46:0/750168422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563330a3e350 con 0x563330c69570 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.765+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/750168422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c7a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:37:55.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a44af3700 1 --2- 172.21.15.46:0/750168422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 0x563330c79bb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009570 tx=0x7f0a34009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838770ecd3f1fc52 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a1ffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a1ffff700 1 -- 172.21.15.46:0/750168422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34004d30 con 0x563330c69570 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a1ffff700 1 -- 172.21.15.46:0/750168422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0a3403e080 con 0x563330c69570 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a52eecb80 1 -- 172.21.15.46:0/750168422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a2c09dfc0 con 0x563330c69570 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a1ffff700 1 -- 172.21.15.46:0/750168422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a34004e90 con 0x563330c69570 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a52eecb80 10 client.16164 fetch_fsmap learned FSMap version 33 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a52eecb80 1 -- 172.21.15.46:0/750168422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f0a380092a0 con 0x563330c69570 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a1ffff700 1 -- 172.21.15.46:0/750168422 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a3404e080 con 0x563330c69570 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a1ffff700 1 -- 172.21.15.46:0/750168422 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0a34030530 con 0x563330c69570 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a52eecb80 10 client.16164 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a52eecb80 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a52eecb80 20 client.16164 populate_metadata read hostname 'smithi046' 2024-03-18T20:37:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/750168422 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563330cd96c0 0x563330cdbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:37:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a52eecb80 4 client.16164 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:37:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.766+0000 7f0a52eecb80 1 -- 172.21.15.46:0/750168422 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563330a0bc90 con 0x563330cd96c0 2024-03-18T20:37:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.767+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/750168422 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563330cd96c0 0x563330cdbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:37:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.767+0000 7f0a1ffff700 10 client.16164 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.767+0000 7f0a3ffff700 1 --2- 172.21.15.46:0/750168422 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563330cd96c0 0x563330cdbae0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cffce30f874df6d0 server_cookie=891eb58e59ca7d3d in_seq=0 out_seq=0 2024-03-18T20:37:55.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a1ffff700 1 -- 172.21.15.46:0/750168422 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563330a0bc90 con 0x563330cd96c0 2024-03-18T20:37:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a1ffff700 10 client.16164 handle_command_reply: tid=0 2024-03-18T20:37:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a52eecb80 1 client.16164 shutdown 2024-03-18T20:37:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a52eecb80 1 -- 172.21.15.46:0/750168422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 msgr2=0x563330c79bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/750168422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 0x563330c79bb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009570 tx=0x7f0a34009930 comp rx=0 tx=0).stop 2024-03-18T20:37:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a52eecb80 1 -- 172.21.15.46:0/750168422 shutdown_connections 2024-03-18T20:37:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/750168422 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563330cd96c0 0x563330cdbae0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/750168422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330c69570 0x563330c79bb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/750168422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563330c74db0 0x563330cd1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a1ffff700 0 client.16164 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:37:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a52eecb80 1 --2- 172.21.15.46:0/750168422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563330c69e90 0x563330c7a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a52eecb80 1 -- 172.21.15.46:0/750168422 >> 172.21.15.46:0/750168422 conn(0x563330be01c0 msgr2=0x563330c6cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a52eecb80 1 -- 172.21.15.46:0/750168422 shutdown_connections 2024-03-18T20:37:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a52eecb80 1 -- 172.21.15.46:0/750168422 wait complete. 2024-03-18T20:37:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.789+0000 7f0a52eecb80 20 client.16164 trim_cache size 0 max 16384 2024-03-18T20:37:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:37:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:37:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:37:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:37:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.790+0000 7f0a46d57700 1 -- 172.21.15.46:0/85045419 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0a300a1290 msgr2=0x7f0a300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.790+0000 7f0a46d57700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0a300a1290 0x7f0a300a3750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a34006010 tx=0x7f0a3400b260 comp rx=0 tx=0).stop 2024-03-18T20:37:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.790+0000 7f0a46d57700 1 -- 172.21.15.46:0/85045419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a40100570 msgr2=0x7f0a401976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:37:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.790+0000 7f0a46d57700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a40100570 0x7f0a401976b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00d950 tx=0x7f0a2c00dd10 comp rx=0 tx=0).stop 2024-03-18T20:37:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.790+0000 7f0a46d57700 1 -- 172.21.15.46:0/85045419 shutdown_connections 2024-03-18T20:37:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.790+0000 7f0a46d57700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0a300a1290 0x7f0a300a3750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.790+0000 7f0a46d57700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a40105c60 0x7f0a4019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.790+0000 7f0a46d57700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a400ffbb0 0x7f0a40197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.790+0000 7f0a46d57700 1 --2- 172.21.15.46:0/85045419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a40100570 0x7f0a401976b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:37:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.790+0000 7f0a46d57700 1 -- 172.21.15.46:0/85045419 >> 172.21.15.46:0/85045419 conn(0x7f0a40076d50 msgr2=0x7f0a400feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:37:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.790+0000 7f0a46d57700 1 -- 172.21.15.46:0/85045419 shutdown_connections 2024-03-18T20:37:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:37:55.790+0000 7f0a46d57700 1 -- 172.21.15.46:0/85045419 wait complete. 2024-03-18T20:37:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:55 smithi073 ceph-mon[47384]: pgmap v58: 97 pgs: 97 active+clean; 26 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 92 KiB/s wr, 30 op/s 2024-03-18T20:37:56.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:55 smithi155 ceph-mon[48344]: pgmap v58: 97 pgs: 97 active+clean; 26 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 92 KiB/s wr, 30 op/s 2024-03-18T20:37:56.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:55 smithi046 ceph-mon[43495]: pgmap v58: 97 pgs: 97 active+clean; 26 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 92 KiB/s wr, 30 op/s 2024-03-18T20:37:56.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 46938 5.29 MB/sec warmup 51 sec latency 1620.352 ms 2024-03-18T20:37:57.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 48723 5.34 MB/sec warmup 52 sec latency 1982.311 ms 2024-03-18T20:37:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:37:58 smithi073 ceph-mon[47384]: pgmap v59: 97 pgs: 97 active+clean; 26 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 92 KiB/s wr, 30 op/s 2024-03-18T20:37:58.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:37:58 smithi155 ceph-mon[48344]: pgmap v59: 97 pgs: 97 active+clean; 26 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 92 KiB/s wr, 30 op/s 2024-03-18T20:37:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:37:58 smithi046 ceph-mon[43495]: pgmap v59: 97 pgs: 97 active+clean; 26 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 92 KiB/s wr, 30 op/s 2024-03-18T20:37:58.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 49023 5.30 MB/sec warmup 53 sec latency 895.097 ms 2024-03-18T20:37:59.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 49074 5.22 MB/sec warmup 54 sec latency 1394.251 ms 2024-03-18T20:38:00.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:00 smithi073 ceph-mon[47384]: pgmap v60: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 268 KiB/s wr, 77 op/s 2024-03-18T20:38:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:00 smithi155 ceph-mon[48344]: pgmap v60: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 268 KiB/s wr, 77 op/s 2024-03-18T20:38:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:00 smithi046 ceph-mon[43495]: pgmap v60: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 268 KiB/s wr, 77 op/s 2024-03-18T20:38:00.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 49074 5.13 MB/sec warmup 55 sec latency 1455.712 ms 2024-03-18T20:38:01.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 50090 5.15 MB/sec warmup 56 sec latency 1521.717 ms 2024-03-18T20:38:02.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:02 smithi073 ceph-mon[47384]: pgmap v61: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 191 KiB/s wr, 56 op/s 2024-03-18T20:38:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:02 smithi155 ceph-mon[48344]: pgmap v61: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 191 KiB/s wr, 56 op/s 2024-03-18T20:38:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:02 smithi046 ceph-mon[43495]: pgmap v61: 97 pgs: 97 active+clean; 27 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 191 KiB/s wr, 56 op/s 2024-03-18T20:38:02.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 50479 5.07 MB/sec warmup 57 sec latency 568.903 ms 2024-03-18T20:38:03.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 50479 4.98 MB/sec warmup 58 sec latency 1568.969 ms 2024-03-18T20:38:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:04 smithi073 ceph-mon[47384]: pgmap v62: 97 pgs: 97 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 501 KiB/s wr, 119 op/s 2024-03-18T20:38:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:04 smithi155 ceph-mon[48344]: pgmap v62: 97 pgs: 97 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 501 KiB/s wr, 119 op/s 2024-03-18T20:38:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:04 smithi046 ceph-mon[43495]: pgmap v62: 97 pgs: 97 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 501 KiB/s wr, 119 op/s 2024-03-18T20:38:04.672 INFO:tasks.workunit.client.0.smithi046.stdout: 1 50479 4.90 MB/sec warmup 59 sec latency 2569.049 ms 2024-03-18T20:38:05.673 INFO:tasks.workunit.client.0.smithi046.stdout: 1 50479 4.82 MB/sec warmup 60 sec latency 3569.158 ms 2024-03-18T20:38:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:05 smithi073 ceph-mon[47384]: pgmap v63: 97 pgs: 97 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 485 KiB/s wr, 109 op/s 2024-03-18T20:38:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:05 smithi155 ceph-mon[48344]: pgmap v63: 97 pgs: 97 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 485 KiB/s wr, 109 op/s 2024-03-18T20:38:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:05 smithi046 ceph-mon[43495]: pgmap v63: 97 pgs: 97 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 485 KiB/s wr, 109 op/s 2024-03-18T20:38:06.673 INFO:tasks.workunit.client.0.smithi046.stdout: 1 50479 4.74 MB/sec warmup 61 sec latency 4569.271 ms 2024-03-18T20:38:07.673 INFO:tasks.workunit.client.0.smithi046.stdout: 1 52181 4.78 MB/sec warmup 62 sec latency 4930.292 ms 2024-03-18T20:38:08.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:38:08.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:38:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.246+0000 7f7e1c045700 1 -- 172.21.15.46:0/3354369730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e140687f0 msgr2=0x7f7e1410db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.246+0000 7f7e1c045700 1 --2- 172.21.15.46:0/3354369730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e140687f0 0x7f7e1410db50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00b0a0 tx=0x7f7e0c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.246+0000 7f7e1c045700 1 -- 172.21.15.46:0/3354369730 shutdown_connections 2024-03-18T20:38:08.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.246+0000 7f7e1c045700 1 --2- 172.21.15.46:0/3354369730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e140687f0 0x7f7e1410db50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.246+0000 7f7e1c045700 1 --2- 172.21.15.46:0/3354369730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e14105ce0 0x7f7e141060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.246+0000 7f7e1c045700 1 --2- 172.21.15.46:0/3354369730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e14067e30 0x7f7e140682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.246+0000 7f7e1c045700 1 -- 172.21.15.46:0/3354369730 >> 172.21.15.46:0/3354369730 conn(0x7f7e14076d40 msgr2=0x7f7e14077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.246+0000 7f7e1c045700 1 -- 172.21.15.46:0/3354369730 shutdown_connections 2024-03-18T20:38:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.246+0000 7f7e1c045700 1 -- 172.21.15.46:0/3354369730 wait complete. 2024-03-18T20:38:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.247+0000 7f7e1c045700 1 Processor -- start 2024-03-18T20:38:08.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.247+0000 7f7e1c045700 1 -- start start 2024-03-18T20:38:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.247+0000 7f7e1c045700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e14067e30 0x7f7e140ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.247+0000 7f7e1c045700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e140687f0 0x7f7e141001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.247+0000 7f7e1c045700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e14105ce0 0x7f7e141006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.247+0000 7f7e1c045700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e141a5720 con 0x7f7e14067e30 2024-03-18T20:38:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.247+0000 7f7e1c045700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e141a5890 con 0x7f7e14105ce0 2024-03-18T20:38:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.247+0000 7f7e1c045700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e141a5a50 con 0x7f7e140687f0 2024-03-18T20:38:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e195e0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e140687f0 0x7f7e141001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e195e0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e140687f0 0x7f7e141001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:35482/0 (socket says 172.21.15.46:35482) 2024-03-18T20:38:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e195e0700 1 -- 172.21.15.46:0/3396746422 learned_addr learned my addr 172.21.15.46:0/3396746422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e1a5e2700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e14105ce0 0x7f7e141006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e19de1700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e14067e30 0x7f7e140ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e195e0700 1 -- 172.21.15.46:0/3396746422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e14105ce0 msgr2=0x7f7e141006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e195e0700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e14105ce0 0x7f7e141006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e195e0700 1 -- 172.21.15.46:0/3396746422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e14067e30 msgr2=0x7f7e140ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e195e0700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e14067e30 0x7f7e140ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e195e0700 1 -- 172.21.15.46:0/3396746422 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e04009520 con 0x7f7e140687f0 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e19de1700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e14067e30 0x7f7e140ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e1a5e2700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e14105ce0 0x7f7e141006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e195e0700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e140687f0 0x7f7e141001b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400eb30 tx=0x7f7e0400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e0affd700 1 -- 172.21.15.46:0/3396746422 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04009880 con 0x7f7e140687f0 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e0affd700 1 -- 172.21.15.46:0/3396746422 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e040044e0 con 0x7f7e140687f0 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e0affd700 1 -- 172.21.15.46:0/3396746422 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04003e90 con 0x7f7e140687f0 2024-03-18T20:38:08.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.248+0000 7f7e1c045700 1 -- 172.21.15.46:0/3396746422 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c009d00 con 0x7f7e140687f0 2024-03-18T20:38:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.249+0000 7f7e1c045700 1 -- 172.21.15.46:0/3396746422 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e141a6060 con 0x7f7e140687f0 2024-03-18T20:38:08.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.251+0000 7f7e0affd700 1 -- 172.21.15.46:0/3396746422 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7e04004000 con 0x7f7e140687f0 2024-03-18T20:38:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.251+0000 7f7e0affd700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e000a1290 0x7f7e000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.252+0000 7f7e19de1700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e000a1290 0x7f7e000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.252+0000 7f7e0affd700 1 -- 172.21.15.46:0/3396746422 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f7e04014070 con 0x7f7e140687f0 2024-03-18T20:38:08.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.252+0000 7f7e19de1700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e000a1290 0x7f7e000a3750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009cb0 tx=0x7f7e10009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.253+0000 7f7e281dab80 1 Processor -- start 2024-03-18T20:38:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.253+0000 7f7e281dab80 1 -- start start 2024-03-18T20:38:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e281dab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a00a60 0x559f14a00e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e281dab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a0bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e281dab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a0c220 0x559f14a0e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e281dab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f1495a990 con 0x559f14a0c220 2024-03-18T20:38:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e281dab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f14958350 con 0x559f14a01380 2024-03-18T20:38:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e281dab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f14953c40 con 0x559f14a00a60 2024-03-18T20:38:08.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e1a5e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a0c220 0x559f14a0e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e195e0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a0bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e19de1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a00a60 0x559f14a00e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e195e0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a0bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33868/0 (socket says 172.21.15.46:33868) 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e1a5e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a0c220 0x559f14a0e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45948/0 (socket says 172.21.15.46:45948) 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e195e0700 1 -- 172.21.15.46:0/1311838189 learned_addr learned my addr 172.21.15.46:0/1311838189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e195e0700 1 -- 172.21.15.46:0/1311838189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a00a60 msgr2=0x559f14a00e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e195e0700 1 --2- 172.21.15.46:0/1311838189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a00a60 0x559f14a00e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e195e0700 1 -- 172.21.15.46:0/1311838189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a0c220 msgr2=0x559f14a0e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e195e0700 1 --2- 172.21.15.46:0/1311838189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a0c220 0x559f14a0e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e195e0700 1 -- 172.21.15.46:0/1311838189 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f147d53b0 con 0x559f14a01380 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7e195e0700 1 --2- 172.21.15.46:0/1311838189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a0bce0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e04096cd0 tx=0x7f7e04099e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b32f28536d4727a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.254+0000 7f7dfffff700 1 -- 172.21.15.46:0/1311838189 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e040af070 con 0x559f14a01380 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.255+0000 7f7dfffff700 1 -- 172.21.15.46:0/1311838189 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0400cc90 con 0x559f14a01380 2024-03-18T20:38:08.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.255+0000 7f7dfffff700 1 -- 172.21.15.46:0/1311838189 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04098600 con 0x559f14a01380 2024-03-18T20:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.255+0000 7f7e281dab80 1 -- 172.21.15.46:0/1311838189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 msgr2=0x559f14a0bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.255+0000 7f7e281dab80 1 --2- 172.21.15.46:0/1311838189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a0bce0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e04096cd0 tx=0x7f7e04099e70 comp rx=0 tx=0).stop 2024-03-18T20:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.255+0000 7f7e281dab80 1 -- 172.21.15.46:0/1311838189 shutdown_connections 2024-03-18T20:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.255+0000 7f7e281dab80 1 --2- 172.21.15.46:0/1311838189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a0c220 0x559f14a0e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.255+0000 7f7e281dab80 1 --2- 172.21.15.46:0/1311838189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a00a60 0x559f14a00e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.255+0000 7f7e281dab80 1 --2- 172.21.15.46:0/1311838189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a0bce0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.255+0000 7f7e281dab80 1 -- 172.21.15.46:0/1311838189 >> 172.21.15.46:0/1311838189 conn(0x559f14977220 msgr2=0x559f149fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.256+0000 7f7e281dab80 1 -- 172.21.15.46:0/1311838189 shutdown_connections 2024-03-18T20:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.256+0000 7f7e281dab80 1 -- 172.21.15.46:0/1311838189 wait complete. 2024-03-18T20:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.256+0000 7f7e281dab80 1 -- start start 2024-03-18T20:38:08.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.256+0000 7f7e281dab80 1 Processor -- start 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e281dab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a00a60 0x559f14a09140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e281dab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a09680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e281dab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a0c220 0x559f14a09bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e281dab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f148efd80 con 0x559f14a00a60 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e281dab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f148efa20 con 0x559f14a01380 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e281dab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f147d76a0 con 0x559f14a0c220 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e281dab80 10 client.0 _ll_register_callbacks cb 0x559f1475b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e19de1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a00a60 0x559f14a09140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e1a5e2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a0c220 0x559f14a09bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e19de1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a00a60 0x559f14a09140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45962/0 (socket says 172.21.15.46:45962) 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e19de1700 1 -- 172.21.15.46:0/285645533 learned_addr learned my addr 172.21.15.46:0/285645533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e195e0700 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a09680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e1a5e2700 1 -- 172.21.15.46:0/285645533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 msgr2=0x559f14a09680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e1a5e2700 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a09680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e1a5e2700 1 -- 172.21.15.46:0/285645533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a00a60 msgr2=0x559f14a09140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e1a5e2700 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a00a60 0x559f14a09140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e1a5e2700 1 -- 172.21.15.46:0/285645533 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e141a6060 con 0x559f14a0c220 2024-03-18T20:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e195e0700 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a09680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7e1a5e2700 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a0c220 0x559f14a09bc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009fd0 tx=0x7f7e0c009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=840e22176789f2a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7dfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.257+0000 7f7dfd7fa700 1 -- 172.21.15.46:0/285645533 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00e040 con 0x559f14a0c220 2024-03-18T20:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7dfd7fa700 1 -- 172.21.15.46:0/285645533 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f7e0c023070 con 0x559f14a0c220 2024-03-18T20:38:08.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7e281dab80 1 -- 172.21.15.46:0/285645533 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f149f4ad0 con 0x559f14a0c220 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7dfd7fa700 1 -- 172.21.15.46:0/285645533 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0c0074e0 con 0x559f14a0c220 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7e281dab80 10 client.25933 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7e281dab80 1 -- 172.21.15.46:0/285645533 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559f14781b60 con 0x559f14a0c220 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7dfd7fa700 1 -- 172.21.15.46:0/285645533 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f149f4ad0 con 0x559f14a0c220 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7dfd7fa700 1 -- 172.21.15.46:0/285645533 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7e0c00b0d0 con 0x559f14a0c220 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7e281dab80 10 client.25933 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7e281dab80 10 client.25933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7e281dab80 20 client.25933 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7e281dab80 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559f14a1be30 0x559f14a1e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7e281dab80 4 client.25933 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7e281dab80 1 -- 172.21.15.46:0/285645533 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f1494c700 con 0x559f14a1be30 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.258+0000 7f7e19de1700 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559f14a1be30 0x559f14a1e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.259+0000 7f7dfd7fa700 10 client.25933 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:08.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.259+0000 7f7e19de1700 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559f14a1be30 0x559f14a1e250 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2394d92b7451770 server_cookie=54a492f0aec7baa in_seq=0 out_seq=0 2024-03-18T20:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.259+0000 7f7dfd7fa700 1 -- 172.21.15.46:0/285645533 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559f1494c700 con 0x559f14a1be30 2024-03-18T20:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.259+0000 7f7dfd7fa700 10 client.25933 handle_command_reply: tid=0 2024-03-18T20:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7e281dab80 1 client.25933 shutdown 2024-03-18T20:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7e281dab80 1 -- 172.21.15.46:0/285645533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a0c220 msgr2=0x559f14a09bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7e281dab80 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a0c220 0x559f14a09bc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009fd0 tx=0x7f7e0c009570 comp rx=0 tx=0).stop 2024-03-18T20:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7e281dab80 1 -- 172.21.15.46:0/285645533 shutdown_connections 2024-03-18T20:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7e281dab80 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559f14a1be30 0x559f14a1e250 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7e281dab80 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a00a60 0x559f14a09140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7e281dab80 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a0c220 0x559f14a09bc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7e281dab80 1 --2- 172.21.15.46:0/285645533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a09680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7dfd7fa700 0 client.25933 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7e281dab80 1 -- 172.21.15.46:0/285645533 >> 172.21.15.46:0/285645533 conn(0x559f14977220 msgr2=0x559f149fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7e281dab80 1 -- 172.21.15.46:0/285645533 shutdown_connections 2024-03-18T20:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7e281dab80 1 -- 172.21.15.46:0/285645533 wait complete. 2024-03-18T20:38:08.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.260+0000 7f7e281dab80 20 client.25933 trim_cache size 0 max 16384 2024-03-18T20:38:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e281dab80 1 Processor -- start 2024-03-18T20:38:08.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e281dab80 1 -- start start 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e281dab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a00a60 0x559f14a12e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e281dab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a099e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e281dab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a0c220 0x559f14a04460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e281dab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f14a189b0 con 0x559f14a0c220 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e281dab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f14a0a600 con 0x559f14a01380 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e281dab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f14a19540 con 0x559f14a00a60 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e195e0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e195e0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a099e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33882/0 (socket says 172.21.15.46:33882) 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e195e0700 1 -- 172.21.15.46:0/371875041 learned_addr learned my addr 172.21.15.46:0/371875041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e1a5e2700 1 --2- 172.21.15.46:0/371875041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a0c220 0x559f14a04460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.279+0000 7f7e19de1700 1 --2- 172.21.15.46:0/371875041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a00a60 0x559f14a12e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.280+0000 7f7e195e0700 1 -- 172.21.15.46:0/371875041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a00a60 msgr2=0x559f14a12e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.280+0000 7f7e195e0700 1 --2- 172.21.15.46:0/371875041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a00a60 0x559f14a12e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.280+0000 7f7e195e0700 1 -- 172.21.15.46:0/371875041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a0c220 msgr2=0x559f14a04460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.280+0000 7f7e195e0700 1 --2- 172.21.15.46:0/371875041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a0c220 0x559f14a04460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.280+0000 7f7e195e0700 1 -- 172.21.15.46:0/371875041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f14919ea0 con 0x559f14a01380 2024-03-18T20:38:08.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.280+0000 7f7e19de1700 1 --2- 172.21.15.46:0/371875041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a00a60 0x559f14a12e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:08.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.280+0000 7f7e195e0700 1 --2- 172.21.15.46:0/371875041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a099e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7e040037d0 tx=0x7f7e0400ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=107af0a92cb05e68 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.280+0000 7f7e08ff9700 1 -- 172.21.15.46:0/371875041 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e040af070 con 0x559f14a01380 2024-03-18T20:38:08.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.280+0000 7f7e08ff9700 1 -- 172.21.15.46:0/371875041 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e040a26e0 con 0x559f14a01380 2024-03-18T20:38:08.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.280+0000 7f7e08ff9700 1 -- 172.21.15.46:0/371875041 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0401fed0 con 0x559f14a01380 2024-03-18T20:38:08.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.281+0000 7f7e281dab80 1 -- 172.21.15.46:0/371875041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 msgr2=0x559f14a099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.281+0000 7f7e281dab80 1 --2- 172.21.15.46:0/371875041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a099e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7e040037d0 tx=0x7f7e0400ce00 comp rx=0 tx=0).stop 2024-03-18T20:38:08.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.281+0000 7f7e281dab80 1 -- 172.21.15.46:0/371875041 shutdown_connections 2024-03-18T20:38:08.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.281+0000 7f7e281dab80 1 --2- 172.21.15.46:0/371875041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a0c220 0x559f14a04460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.281+0000 7f7e281dab80 1 --2- 172.21.15.46:0/371875041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a00a60 0x559f14a12e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.281+0000 7f7e281dab80 1 --2- 172.21.15.46:0/371875041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a01380 0x559f14a099e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.281+0000 7f7e281dab80 1 -- 172.21.15.46:0/371875041 >> 172.21.15.46:0/371875041 conn(0x559f14977220 msgr2=0x559f149fb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.281+0000 7f7e281dab80 1 -- 172.21.15.46:0/371875041 shutdown_connections 2024-03-18T20:38:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.281+0000 7f7e281dab80 1 -- 172.21.15.46:0/371875041 wait complete. 2024-03-18T20:38:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.281+0000 7f7e281dab80 1 -- start start 2024-03-18T20:38:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.281+0000 7f7e281dab80 1 Processor -- start 2024-03-18T20:38:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e281dab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a00a60 0x559f14a12c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e281dab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a01380 0x559f14a11260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e281dab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a0c220 0x559f14a117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e281dab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f14a11ce0 con 0x559f14a01380 2024-03-18T20:38:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e281dab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f14a11e50 con 0x559f14a00a60 2024-03-18T20:38:08.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e281dab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f14a12010 con 0x559f14a0c220 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e281dab80 10 client.0 _ll_register_callbacks cb 0x7f7e0c00bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e195e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a01380 0x559f14a11260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e195e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a01380 0x559f14a11260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45982/0 (socket says 172.21.15.46:45982) 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e195e0700 1 -- 172.21.15.46:0/3516427728 learned_addr learned my addr 172.21.15.46:0/3516427728 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e19de1700 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a00a60 0x559f14a12c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e1a5e2700 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a0c220 0x559f14a117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e195e0700 1 -- 172.21.15.46:0/3516427728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a0c220 msgr2=0x559f14a117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e195e0700 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a0c220 0x559f14a117a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e195e0700 1 -- 172.21.15.46:0/3516427728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a00a60 msgr2=0x559f14a12c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e195e0700 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a00a60 0x559f14a12c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e195e0700 1 -- 172.21.15.46:0/3516427728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e04094e70 con 0x559f14a01380 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e1a5e2700 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a0c220 0x559f14a117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:08.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7e195e0700 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a01380 0x559f14a11260 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7e04096cd0 tx=0x7f7e04099c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829121fb02599507 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.282+0000 7f7dfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.283+0000 7f7dfd7fa700 1 -- 172.21.15.46:0/3516427728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e040af070 con 0x559f14a01380 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.283+0000 7f7dfd7fa700 1 -- 172.21.15.46:0/3516427728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f7e0409e090 con 0x559f14a01380 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.283+0000 7f7e281dab80 1 -- 172.21.15.46:0/3516427728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e0400cc90 con 0x559f14a01380 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.283+0000 7f7dfd7fa700 1 -- 172.21.15.46:0/3516427728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0409a260 con 0x559f14a01380 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.283+0000 7f7e281dab80 10 client.16182 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.283+0000 7f7e281dab80 1 -- 172.21.15.46:0/3516427728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559f147d9cd0 con 0x559f14a01380 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.283+0000 7f7dfd7fa700 1 -- 172.21.15.46:0/3516427728 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0400cc90 con 0x559f14a01380 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.284+0000 7f7dfd7fa700 1 -- 172.21.15.46:0/3516427728 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7e040a2250 con 0x559f14a01380 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.284+0000 7f7e281dab80 10 client.16182 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.284+0000 7f7e281dab80 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.284+0000 7f7e281dab80 20 client.16182 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.284+0000 7f7e281dab80 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559f14a70d40 0x559f14a73160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.284+0000 7f7e281dab80 4 client.16182 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.284+0000 7f7e281dab80 1 -- 172.21.15.46:0/3516427728 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559f147a2cf0 con 0x559f14a70d40 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.284+0000 7f7e19de1700 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559f14a70d40 0x559f14a73160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7dfd7fa700 10 client.16182 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e19de1700 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559f14a70d40 0x559f14a73160 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33e2ff82a85e5bbe server_cookie=acf323806975c33d in_seq=0 out_seq=0 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7dfd7fa700 1 -- 172.21.15.46:0/3516427728 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559f147a2cf0 con 0x559f14a70d40 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7dfd7fa700 10 client.16182 handle_command_reply: tid=0 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e281dab80 1 client.16182 shutdown 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e281dab80 1 -- 172.21.15.46:0/3516427728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a01380 msgr2=0x559f14a11260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e281dab80 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a01380 0x559f14a11260 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7e04096cd0 tx=0x7f7e04099c30 comp rx=0 tx=0).stop 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e281dab80 1 -- 172.21.15.46:0/3516427728 shutdown_connections 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e281dab80 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559f14a70d40 0x559f14a73160 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e281dab80 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559f14a0c220 0x559f14a117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7dfd7fa700 0 client.16182 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:08.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e281dab80 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f14a00a60 0x559f14a12c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e281dab80 1 --2- 172.21.15.46:0/3516427728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f14a01380 0x559f14a11260 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e281dab80 1 -- 172.21.15.46:0/3516427728 >> 172.21.15.46:0/3516427728 conn(0x559f14977220 msgr2=0x559f14a04460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e281dab80 1 -- 172.21.15.46:0/3516427728 shutdown_connections 2024-03-18T20:38:08.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e281dab80 1 -- 172.21.15.46:0/3516427728 wait complete. 2024-03-18T20:38:08.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.285+0000 7f7e281dab80 20 client.16182 trim_cache size 0 max 16384 2024-03-18T20:38:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:38:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:38:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:38:08.294 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:38:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.286+0000 7f7e1c045700 1 -- 172.21.15.46:0/3396746422 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e000a1290 msgr2=0x7f7e000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.286+0000 7f7e1c045700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e000a1290 0x7f7e000a3750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009cb0 tx=0x7f7e10009260 comp rx=0 tx=0).stop 2024-03-18T20:38:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.286+0000 7f7e1c045700 1 -- 172.21.15.46:0/3396746422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e140687f0 msgr2=0x7f7e141001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.286+0000 7f7e1c045700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e140687f0 0x7f7e141001b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e0400eb30 tx=0x7f7e0400ee40 comp rx=0 tx=0).stop 2024-03-18T20:38:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.286+0000 7f7e1c045700 1 -- 172.21.15.46:0/3396746422 shutdown_connections 2024-03-18T20:38:08.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.286+0000 7f7e1c045700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e000a1290 0x7f7e000a3750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.286+0000 7f7e1c045700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e14067e30 0x7f7e140ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.286+0000 7f7e1c045700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e14105ce0 0x7f7e141006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.286+0000 7f7e1c045700 1 --2- 172.21.15.46:0/3396746422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e140687f0 0x7f7e141001b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.286+0000 7f7e1c045700 1 -- 172.21.15.46:0/3396746422 >> 172.21.15.46:0/3396746422 conn(0x7f7e14076d40 msgr2=0x7f7e140fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.286+0000 7f7e1c045700 1 -- 172.21.15.46:0/3396746422 shutdown_connections 2024-03-18T20:38:08.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.286+0000 7f7e1c045700 1 -- 172.21.15.46:0/3396746422 wait complete. 2024-03-18T20:38:08.303 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:38:08.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:38:08.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.443+0000 7f36c20f2700 1 -- 172.21.15.46:0/3109899344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc101810 msgr2=0x7f36bc105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.443+0000 7f36c20f2700 1 --2- 172.21.15.46:0/3109899344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc101810 0x7f36bc105850 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009990 tx=0x7f36b0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.443+0000 7f36c20f2700 1 -- 172.21.15.46:0/3109899344 shutdown_connections 2024-03-18T20:38:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.443+0000 7f36c20f2700 1 --2- 172.21.15.46:0/3109899344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc105f80 0x7f36bc10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.443+0000 7f36c20f2700 1 --2- 172.21.15.46:0/3109899344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36bc100e60 0x7f36bc101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.443+0000 7f36c20f2700 1 --2- 172.21.15.46:0/3109899344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc101810 0x7f36bc105850 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.443+0000 7f36c20f2700 1 -- 172.21.15.46:0/3109899344 >> 172.21.15.46:0/3109899344 conn(0x7f36bc0fc700 msgr2=0x7f36bc0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.443+0000 7f36c20f2700 1 -- 172.21.15.46:0/3109899344 shutdown_connections 2024-03-18T20:38:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.443+0000 7f36c20f2700 1 -- 172.21.15.46:0/3109899344 wait complete. 2024-03-18T20:38:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.443+0000 7f36c20f2700 1 Processor -- start 2024-03-18T20:38:08.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.444+0000 7f36c20f2700 1 -- start start 2024-03-18T20:38:08.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.444+0000 7f36c20f2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc100e60 0x7f36bc197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.444+0000 7f36c20f2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc105f80 0x7f36bc1976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.444+0000 7f36c20f2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36bc19ba20 0x7f36bc19bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.444+0000 7f36c20f2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc19c4a0 con 0x7f36bc105f80 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.444+0000 7f36c20f2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc19c610 con 0x7f36bc19ba20 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.444+0000 7f36c20f2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc19c7d0 con 0x7f36bc100e60 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.444+0000 7f36baffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc105f80 0x7f36bc1976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.444+0000 7f36baffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc105f80 0x7f36bc1976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45996/0 (socket says 172.21.15.46:45996) 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.444+0000 7f36baffd700 1 -- 172.21.15.46:0/1085733222 learned_addr learned my addr 172.21.15.46:0/1085733222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.444+0000 7f36bb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc100e60 0x7f36bc197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.444+0000 7f36bbfff700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36bc19ba20 0x7f36bc19bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36baffd700 1 -- 172.21.15.46:0/1085733222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc100e60 msgr2=0x7f36bc197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36baffd700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc100e60 0x7f36bc197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36baffd700 1 -- 172.21.15.46:0/1085733222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36bc19ba20 msgr2=0x7f36bc19bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36baffd700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36bc19ba20 0x7f36bc19bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36baffd700 1 -- 172.21.15.46:0/1085733222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac009520 con 0x7f36bc105f80 2024-03-18T20:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36bb7fe700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc100e60 0x7f36bc197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36bbfff700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36bc19ba20 0x7f36bc19bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36baffd700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc105f80 0x7f36bc1976d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f36b0000c00 tx=0x7f36b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36b8ff9700 1 -- 172.21.15.46:0/1085733222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b001d070 con 0x7f36bc105f80 2024-03-18T20:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36b8ff9700 1 -- 172.21.15.46:0/1085733222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36b0021410 con 0x7f36bc105f80 2024-03-18T20:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36c20f2700 1 -- 172.21.15.46:0/1085733222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b00095f0 con 0x7f36bc105f80 2024-03-18T20:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36b8ff9700 1 -- 172.21.15.46:0/1085733222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0017650 con 0x7f36bc105f80 2024-03-18T20:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.445+0000 7f36c20f2700 1 -- 172.21.15.46:0/1085733222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc1a3f60 con 0x7f36bc105f80 2024-03-18T20:38:08.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.446+0000 7f36c20f2700 1 -- 172.21.15.46:0/1085733222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36bc109250 con 0x7f36bc105f80 2024-03-18T20:38:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.447+0000 7f36b8ff9700 1 -- 172.21.15.46:0/1085733222 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f36b0021580 con 0x7f36bc105f80 2024-03-18T20:38:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.448+0000 7f36b8ff9700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36a40a1290 0x7f36a40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.448+0000 7f36bb7fe700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36a40a1290 0x7f36a40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.448+0000 7f36b8ff9700 1 -- 172.21.15.46:0/1085733222 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f36b00cde80 con 0x7f36bc105f80 2024-03-18T20:38:08.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.449+0000 7f36bb7fe700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36a40a1290 0x7f36a40a3750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009fd0 tx=0x7f36ac009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.453+0000 7f36b8ff9700 1 -- 172.21.15.46:0/1085733222 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f36b0097330 con 0x7f36bc105f80 2024-03-18T20:38:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:08 smithi073 ceph-mon[47384]: pgmap v64: 97 pgs: 97 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 485 KiB/s wr, 109 op/s 2024-03-18T20:38:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:08 smithi155 ceph-mon[48344]: pgmap v64: 97 pgs: 97 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 485 KiB/s wr, 109 op/s 2024-03-18T20:38:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:08 smithi046 ceph-mon[43495]: pgmap v64: 97 pgs: 97 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 485 KiB/s wr, 109 op/s 2024-03-18T20:38:08.673 INFO:tasks.workunit.client.0.smithi046.stdout: 1 53785 4.89 MB/sec warmup 63 sec latency 13.407 ms 2024-03-18T20:38:08.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.750+0000 7f36c20f2700 1 -- 172.21.15.46:0/1085733222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36bc067a10 con 0x7f36bc105f80 2024-03-18T20:38:08.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.750+0000 7f36b8ff9700 1 -- 172.21.15.46:0/1085733222 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10919 (secure 0 0 0) 0x7f36b009a950 con 0x7f36bc105f80 2024-03-18T20:38:08.754 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:38:08.754 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:37:47.071553+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:38:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.753+0000 7f36c20f2700 1 -- 172.21.15.46:0/1085733222 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36a40a1290 msgr2=0x7f36a40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.753+0000 7f36c20f2700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36a40a1290 0x7f36a40a3750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009fd0 tx=0x7f36ac009260 comp rx=0 tx=0).stop 2024-03-18T20:38:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.753+0000 7f36c20f2700 1 -- 172.21.15.46:0/1085733222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc105f80 msgr2=0x7f36bc1976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.753+0000 7f36c20f2700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc105f80 0x7f36bc1976d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f36b0000c00 tx=0x7f36b0004730 comp rx=0 tx=0).stop 2024-03-18T20:38:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.753+0000 7f36c20f2700 1 -- 172.21.15.46:0/1085733222 shutdown_connections 2024-03-18T20:38:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.753+0000 7f36c20f2700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36a40a1290 0x7f36a40a3750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.753+0000 7f36c20f2700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36bc19ba20 0x7f36bc19bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.753+0000 7f36c20f2700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36bc100e60 0x7f36bc197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.753+0000 7f36c20f2700 1 --2- 172.21.15.46:0/1085733222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36bc105f80 0x7f36bc1976d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.753+0000 7f36c20f2700 1 -- 172.21.15.46:0/1085733222 >> 172.21.15.46:0/1085733222 conn(0x7f36bc0fc700 msgr2=0x7f36bc0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.753+0000 7f36c20f2700 1 -- 172.21.15.46:0/1085733222 shutdown_connections 2024-03-18T20:38:08.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.753+0000 7f36c20f2700 1 -- 172.21.15.46:0/1085733222 wait complete. 2024-03-18T20:38:08.757 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 33 2024-03-18T20:38:08.770 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:38:08.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:38:08.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.909+0000 7f2141ab4700 1 -- 172.21.15.46:0/3102784944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213c0692a0 msgr2=0x7f213c069680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.909+0000 7f2141ab4700 1 --2- 172.21.15.46:0/3102784944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213c0692a0 0x7f213c069680 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f212c00b0d0 tx=0x7f212c00b3e0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.909+0000 7f2141ab4700 1 -- 172.21.15.46:0/3102784944 shutdown_connections 2024-03-18T20:38:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.909+0000 7f2141ab4700 1 --2- 172.21.15.46:0/3102784944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213c0692a0 0x7f213c069680 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.909+0000 7f2141ab4700 1 --2- 172.21.15.46:0/3102784944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f213c1060b0 0x7f213c10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.909+0000 7f2141ab4700 1 --2- 172.21.15.46:0/3102784944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f213c069c50 0x7f213c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.909+0000 7f2141ab4700 1 -- 172.21.15.46:0/3102784944 >> 172.21.15.46:0/3102784944 conn(0x7f213c076d80 msgr2=0x7f213c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.909+0000 7f2141ab4700 1 -- 172.21.15.46:0/3102784944 shutdown_connections 2024-03-18T20:38:08.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.909+0000 7f2141ab4700 1 -- 172.21.15.46:0/3102784944 wait complete. 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f2141ab4700 1 Processor -- start 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f2141ab4700 1 -- start start 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f2141ab4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213c0692a0 0x7f213c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f2141ab4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f213c1060b0 0x7f213c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f2141ab4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f213c19bc60 0x7f213c19c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f2141ab4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c19c6e0 con 0x7f213c0692a0 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f2141ab4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c19c850 con 0x7f213c1060b0 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f2141ab4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c19ca10 con 0x7f213c19bc60 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213c0692a0 0x7f213c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213c0692a0 0x7f213c197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46034/0 (socket says 172.21.15.46:46034) 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f213b7fe700 1 -- 172.21.15.46:0/51069042 learned_addr learned my addr 172.21.15.46:0/51069042 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f213bfff700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f213c19bc60 0x7f213c19c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.910+0000 7f213affd700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f213c1060b0 0x7f213c1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f213b7fe700 1 -- 172.21.15.46:0/51069042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f213c19bc60 msgr2=0x7f213c19c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f213b7fe700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f213c19bc60 0x7f213c19c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f213b7fe700 1 -- 172.21.15.46:0/51069042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f213c1060b0 msgr2=0x7f213c1978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f213b7fe700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f213c1060b0 0x7f213c1978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f213b7fe700 1 -- 172.21.15.46:0/51069042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c009d00 con 0x7f213c0692a0 2024-03-18T20:38:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f133b3dc700 1 -- 172.21.15.46:0/489352335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1334100f10 msgr2=0x7f13341012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f133b3dc700 1 --2- 172.21.15.46:0/489352335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1334100f10 0x7f13341012f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1330009990 tx=0x7f1330009ca0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f213affd700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f213c1060b0 0x7f213c1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f133b3dc700 1 -- 172.21.15.46:0/489352335 shutdown_connections 2024-03-18T20:38:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f133b3dc700 1 --2- 172.21.15.46:0/489352335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1334100f10 0x7f13341012f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f133b3dc700 1 --2- 172.21.15.46:0/489352335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1334106030 0x7f133410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f133b3dc700 1 --2- 172.21.15.46:0/489352335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13341018c0 0x7f1334105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f133b3dc700 1 -- 172.21.15.46:0/489352335 >> 172.21.15.46:0/489352335 conn(0x7f13340fc7b0 msgr2=0x7f13340febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f133b3dc700 1 -- 172.21.15.46:0/489352335 shutdown_connections 2024-03-18T20:38:08.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f213b7fe700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213c0692a0 0x7f213c197380 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f212c00bd80 tx=0x7f212c00bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f133b3dc700 1 -- 172.21.15.46:0/489352335 wait complete. 2024-03-18T20:38:08.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f2138ff9700 1 -- 172.21.15.46:0/51069042 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c00e040 con 0x7f213c0692a0 2024-03-18T20:38:08.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f2138ff9700 1 -- 172.21.15.46:0/51069042 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f212c01b8e0 con 0x7f213c0692a0 2024-03-18T20:38:08.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f2141ab4700 1 -- 172.21.15.46:0/51069042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c1a3f10 con 0x7f213c0692a0 2024-03-18T20:38:08.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.911+0000 7f2138ff9700 1 -- 172.21.15.46:0/51069042 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c012930 con 0x7f213c0692a0 2024-03-18T20:38:08.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.912+0000 7f2141ab4700 1 -- 172.21.15.46:0/51069042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c1a4330 con 0x7f213c0692a0 2024-03-18T20:38:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.912+0000 7f133b3dc700 1 Processor -- start 2024-03-18T20:38:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.912+0000 7f133b3dc700 1 -- start start 2024-03-18T20:38:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.912+0000 7f133b3dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1334100f10 0x7f1334197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.912+0000 7f133b3dc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1334106030 0x7f1334197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.912+0000 7f133b3dc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f133419ba90 0x7f133419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.912+0000 7f133b3dc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133419c510 con 0x7f1334100f10 2024-03-18T20:38:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.912+0000 7f133b3dc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133419c680 con 0x7f133419ba90 2024-03-18T20:38:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.912+0000 7f133b3dc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133419c840 con 0x7f1334106030 2024-03-18T20:38:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f1338977700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1334106030 0x7f1334197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f1338977700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1334106030 0x7f1334197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:35600/0 (socket says 172.21.15.46:35600) 2024-03-18T20:38:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f1338977700 1 -- 172.21.15.46:0/3679005166 learned_addr learned my addr 172.21.15.46:0/3679005166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f1339979700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f133419ba90 0x7f133419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f1338977700 1 -- 172.21.15.46:0/3679005166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f133419ba90 msgr2=0x7f133419bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f1338977700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f133419ba90 0x7f133419bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f1338977700 1 -- 172.21.15.46:0/3679005166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1334100f10 msgr2=0x7f1334197200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:38:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f1338977700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1334100f10 0x7f1334197200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f1338977700 1 -- 172.21.15.46:0/3679005166 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13300095f0 con 0x7f1334106030 2024-03-18T20:38:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f1338977700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1334106030 0x7f1334197740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f132400e970 tx=0x7f132400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f132a7fc700 1 -- 172.21.15.46:0/3679005166 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324009790 con 0x7f1334106030 2024-03-18T20:38:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f133b3dc700 1 -- 172.21.15.46:0/3679005166 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13341a3da0 con 0x7f1334106030 2024-03-18T20:38:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f132a7fc700 1 -- 172.21.15.46:0/3679005166 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f132400eec0 con 0x7f1334106030 2024-03-18T20:38:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f132a7fc700 1 -- 172.21.15.46:0/3679005166 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324005520 con 0x7f1334106030 2024-03-18T20:38:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.913+0000 7f133b3dc700 1 -- 172.21.15.46:0/3679005166 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13341a4270 con 0x7f1334106030 2024-03-18T20:38:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.914+0000 7f2138ff9700 1 -- 172.21.15.46:0/51069042 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f212c01b420 con 0x7f213c0692a0 2024-03-18T20:38:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.915+0000 7f2138ff9700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f21240a1290 0x7f21240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.915+0000 7f2138ff9700 1 -- 172.21.15.46:0/51069042 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f212c0cd6e0 con 0x7f213c0692a0 2024-03-18T20:38:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.915+0000 7f213affd700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f21240a1290 0x7f21240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.916+0000 7f213affd700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f21240a1290 0x7f21240a3750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f213c1988d0 tx=0x7f2130009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.916+0000 7f132a7fc700 1 -- 172.21.15.46:0/3679005166 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1324004750 con 0x7f1334106030 2024-03-18T20:38:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.917+0000 7f214dc49b80 1 Processor -- start 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.918+0000 7f214dc49b80 1 -- start start 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.918+0000 7f214dc49b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d29412d0 0x5589d29416b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.918+0000 7f214dc49b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d2931a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.918+0000 7f214dc49b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2931fa0 0x5589d29323e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.918+0000 7f214dc49b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d288b990 con 0x5589d2931fa0 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.918+0000 7f214dc49b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2889350 con 0x5589d29412d0 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.918+0000 7f214dc49b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2884c40 con 0x5589d2941bf0 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f213affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d2931a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f213affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d2931a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:35610/0 (socket says 172.21.15.46:35610) 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f213affd700 1 -- 172.21.15.46:0/1609325094 learned_addr learned my addr 172.21.15.46:0/1609325094 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f213b7fe700 1 --2- 172.21.15.46:0/1609325094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d29412d0 0x5589d29416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f213affd700 1 -- 172.21.15.46:0/1609325094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d29412d0 msgr2=0x5589d29416b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f213affd700 1 --2- 172.21.15.46:0/1609325094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d29412d0 0x5589d29416b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f213affd700 1 -- 172.21.15.46:0/1609325094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2931fa0 msgr2=0x5589d29323e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f213affd700 1 --2- 172.21.15.46:0/1609325094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2931fa0 0x5589d29323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f213affd700 1 -- 172.21.15.46:0/1609325094 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589d27063b0 con 0x5589d2941bf0 2024-03-18T20:38:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f1347571b80 1 Processor -- start 2024-03-18T20:38:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f1347571b80 1 -- start start 2024-03-18T20:38:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f1347571b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493cadd0 0x555d493cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f1347571b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 0x555d493bca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f1347571b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f1347571b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d49315930 con 0x555d493bcf40 2024-03-18T20:38:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f1347571b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d493132f0 con 0x555d493cadd0 2024-03-18T20:38:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f1347571b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4930ebe0 con 0x555d493bc580 2024-03-18T20:38:08.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.919+0000 7f213affd700 1 --2- 172.21.15.46:0/1609325094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d2931a60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f21300043c0 tx=0x7f2130014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=112e13498b308305 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f1339979700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493cadd0 0x555d493cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f1339178700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 0x555d493bca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f1339979700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493cadd0 0x555d493cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33954/0 (socket says 172.21.15.46:33954) 2024-03-18T20:38:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f1339979700 1 -- 172.21.15.46:0/2560147323 learned_addr learned my addr 172.21.15.46:0/2560147323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f1338977700 1 --2- 172.21.15.46:0/2560147323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f1339178700 1 -- 172.21.15.46:0/2560147323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493cadd0 msgr2=0x555d493cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f1339178700 1 --2- 172.21.15.46:0/2560147323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493cadd0 0x555d493cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f1339178700 1 -- 172.21.15.46:0/2560147323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 msgr2=0x555d493d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f1339178700 1 --2- 172.21.15.46:0/2560147323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493d1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f132a7fc700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f13200a1310 0x7f13200a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f1339178700 1 -- 172.21.15.46:0/2560147323 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d49190350 con 0x555d493bc580 2024-03-18T20:38:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f1338977700 1 --2- 172.21.15.46:0/2560147323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f1339178700 1 --2- 172.21.15.46:0/2560147323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 0x555d493bca00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f13300059d0 tx=0x7f1330005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f877eaf333488a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f132a7fc700 1 -- 172.21.15.46:0/3679005166 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f1324014070 con 0x7f1334106030 2024-03-18T20:38:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f131f7fe700 1 -- 172.21.15.46:0/2560147323 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133001d070 con 0x555d493bc580 2024-03-18T20:38:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f131f7fe700 1 -- 172.21.15.46:0/2560147323 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f133000bbb0 con 0x555d493bc580 2024-03-18T20:38:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f131f7fe700 1 -- 172.21.15.46:0/2560147323 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133000fbd0 con 0x555d493bc580 2024-03-18T20:38:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f2121ffb700 1 -- 172.21.15.46:0/1609325094 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130003dc0 con 0x5589d2941bf0 2024-03-18T20:38:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f2121ffb700 1 -- 172.21.15.46:0/1609325094 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21300268c0 con 0x5589d2941bf0 2024-03-18T20:38:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.920+0000 7f2121ffb700 1 -- 172.21.15.46:0/1609325094 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213001e920 con 0x5589d2941bf0 2024-03-18T20:38:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f214dc49b80 1 -- 172.21.15.46:0/1609325094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 msgr2=0x5589d2931a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1609325094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d2931a60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f21300043c0 tx=0x7f2130014000 comp rx=0 tx=0).stop 2024-03-18T20:38:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f214dc49b80 1 -- 172.21.15.46:0/1609325094 shutdown_connections 2024-03-18T20:38:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1609325094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2931fa0 0x5589d29323e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1609325094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d29412d0 0x5589d29416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1609325094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d2931a60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f214dc49b80 1 -- 172.21.15.46:0/1609325094 >> 172.21.15.46:0/1609325094 conn(0x5589d28a8220 msgr2=0x5589d2932f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f1339979700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f13200a1310 0x7f13200a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f1347571b80 1 -- 172.21.15.46:0/2560147323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 msgr2=0x555d493bca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f1347571b80 1 --2- 172.21.15.46:0/2560147323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 0x555d493bca00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f13300059d0 tx=0x7f1330005a90 comp rx=0 tx=0).stop 2024-03-18T20:38:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f214dc49b80 1 -- 172.21.15.46:0/1609325094 shutdown_connections 2024-03-18T20:38:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f214dc49b80 1 -- 172.21.15.46:0/1609325094 wait complete. 2024-03-18T20:38:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f1347571b80 1 -- 172.21.15.46:0/2560147323 shutdown_connections 2024-03-18T20:38:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f1347571b80 1 --2- 172.21.15.46:0/2560147323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f1347571b80 1 --2- 172.21.15.46:0/2560147323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493cadd0 0x555d493cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f1347571b80 1 --2- 172.21.15.46:0/2560147323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 0x555d493bca00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.921+0000 7f1347571b80 1 -- 172.21.15.46:0/2560147323 >> 172.21.15.46:0/2560147323 conn(0x555d493321c0 msgr2=0x555d493b7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f214dc49b80 1 -- start start 2024-03-18T20:38:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f1339979700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f13200a1310 0x7f13200a37d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f132c0118e0 tx=0x7f132c00b040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f1347571b80 1 -- 172.21.15.46:0/2560147323 shutdown_connections 2024-03-18T20:38:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f1347571b80 1 -- 172.21.15.46:0/2560147323 wait complete. 2024-03-18T20:38:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f214dc49b80 1 Processor -- start 2024-03-18T20:38:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f1347571b80 1 -- start start 2024-03-18T20:38:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f214dc49b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d2931fa0 0x5589d2944f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f214dc49b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d29412d0 0x5589d2945450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f214dc49b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2941bf0 0x5589d2945990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f214dc49b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2820d80 con 0x5589d2941bf0 2024-03-18T20:38:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f1347571b80 1 Processor -- start 2024-03-18T20:38:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f214dc49b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2820a20 con 0x5589d2931fa0 2024-03-18T20:38:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f214dc49b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d27086a0 con 0x5589d29412d0 2024-03-18T20:38:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f214dc49b80 10 client.0 _ll_register_callbacks cb 0x5589d268c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f213bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2941bf0 0x5589d2945990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f213bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2941bf0 0x5589d2945990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46052/0 (socket says 172.21.15.46:46052) 2024-03-18T20:38:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f213bfff700 1 -- 172.21.15.46:0/1839885811 learned_addr learned my addr 172.21.15.46:0/1839885811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.922+0000 7f213affd700 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d29412d0 0x5589d2945450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f213b7fe700 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d2931fa0 0x5589d2944f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f213bfff700 1 -- 172.21.15.46:0/1839885811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d29412d0 msgr2=0x5589d2945450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f213bfff700 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d29412d0 0x5589d2945450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f213bfff700 1 -- 172.21.15.46:0/1839885811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d2931fa0 msgr2=0x5589d2944f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f213bfff700 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d2931fa0 0x5589d2944f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f213bfff700 1 -- 172.21.15.46:0/1839885811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2130017070 con 0x5589d2941bf0 2024-03-18T20:38:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1347571b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bc580 0x555d493d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1347571b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bcf40 0x555d493cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1347571b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493cadd0 0x555d493cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1347571b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d492aad20 con 0x555d493bc580 2024-03-18T20:38:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1347571b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d492aa9c0 con 0x555d493bcf40 2024-03-18T20:38:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1347571b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d49192640 con 0x555d493cadd0 2024-03-18T20:38:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1347571b80 10 client.0 _ll_register_callbacks cb 0x555d49116340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1338977700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bcf40 0x555d493cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1339178700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bc580 0x555d493d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1338977700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bcf40 0x555d493cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33974/0 (socket says 172.21.15.46:33974) 2024-03-18T20:38:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1338977700 1 -- 172.21.15.46:0/1182207492 learned_addr learned my addr 172.21.15.46:0/1182207492 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1339979700 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493cadd0 0x555d493cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f213bfff700 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2941bf0 0x5589d2945990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2134003900 tx=0x7f2134003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75afb020ec26470 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f211affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f211affd700 1 -- 172.21.15.46:0/1839885811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213400bba0 con 0x5589d2941bf0 2024-03-18T20:38:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1338977700 1 -- 172.21.15.46:0/1182207492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493cadd0 msgr2=0x555d493cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1338977700 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493cadd0 0x555d493cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1338977700 1 -- 172.21.15.46:0/1182207492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bc580 msgr2=0x555d493d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1338977700 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bc580 0x555d493d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1338977700 1 -- 172.21.15.46:0/1182207492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133410e420 con 0x555d493bcf40 2024-03-18T20:38:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f1339178700 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bc580 0x555d493d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f211affd700 1 -- 172.21.15.46:0/1839885811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f213400bd00 con 0x5589d2941bf0 2024-03-18T20:38:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.923+0000 7f214dc49b80 1 -- 172.21.15.46:0/1839885811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589d2925ad0 con 0x5589d2941bf0 2024-03-18T20:38:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f1338977700 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bcf40 0x555d493cee50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1324005950 tx=0x7f1324099c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a542409cac4fc8e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f131cff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:38:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f211affd700 1 -- 172.21.15.46:0/1839885811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2134004830 con 0x5589d2941bf0 2024-03-18T20:38:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f214dc49b80 10 client.16203 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:08.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f214dc49b80 1 -- 172.21.15.46:0/1839885811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5589d26b2b60 con 0x5589d2941bf0 2024-03-18T20:38:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f131cff9700 1 -- 172.21.15.46:0/1182207492 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13240cab10 con 0x555d493bcf40 2024-03-18T20:38:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f131cff9700 1 -- 172.21.15.46:0/1182207492 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f132409a050 con 0x555d493bcf40 2024-03-18T20:38:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f1347571b80 1 -- 172.21.15.46:0/1182207492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d493af6c0 con 0x555d493bcf40 2024-03-18T20:38:08.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f211affd700 1 -- 172.21.15.46:0/1839885811 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5589d2925ad0 con 0x5589d2941bf0 2024-03-18T20:38:08.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f211affd700 1 -- 172.21.15.46:0/1839885811 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2134020070 con 0x5589d2941bf0 2024-03-18T20:38:08.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f214dc49b80 10 client.16203 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:08.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f214dc49b80 10 client.16203 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:38:08.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f214dc49b80 20 client.16203 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:08.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5589d2966790 0x5589d2968bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f214dc49b80 4 client.16203 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:08.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f214dc49b80 1 -- 172.21.15.46:0/1839885811 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589d287d700 con 0x5589d2966790 2024-03-18T20:38:08.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f213b7fe700 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5589d2966790 0x5589d2968bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.925+0000 7f211affd700 10 client.16203 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:38:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f131cff9700 1 -- 172.21.15.46:0/1182207492 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f132409de80 con 0x555d493bcf40 2024-03-18T20:38:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f1347571b80 10 client.25976 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f1347571b80 1 -- 172.21.15.46:0/1182207492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555d4913cb00 con 0x555d493bcf40 2024-03-18T20:38:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f131cff9700 1 -- 172.21.15.46:0/1182207492 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d493af6c0 con 0x555d493bcf40 2024-03-18T20:38:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.924+0000 7f131cff9700 1 -- 172.21.15.46:0/1182207492 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f132409a3d0 con 0x555d493bcf40 2024-03-18T20:38:08.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.925+0000 7f213b7fe700 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5589d2966790 0x5589d2968bb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c60f12aef706ed8 server_cookie=e8a4b43107d4e9ef in_seq=0 out_seq=0 2024-03-18T20:38:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.925+0000 7f1347571b80 10 client.25976 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.925+0000 7f1347571b80 10 client.25976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.925+0000 7f1347571b80 20 client.25976 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.925+0000 7f1347571b80 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d493c18c0 0x555d493c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.925+0000 7f1347571b80 4 client.25976 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.925+0000 7f1347571b80 1 -- 172.21.15.46:0/1182207492 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d493076a0 con 0x555d493c18c0 2024-03-18T20:38:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.925+0000 7f1339178700 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d493c18c0 0x555d493c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f211affd700 1 -- 172.21.15.46:0/1839885811 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5589d287d700 con 0x5589d2966790 2024-03-18T20:38:08.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f211affd700 10 client.16203 handle_command_reply: tid=0 2024-03-18T20:38:08.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f214dc49b80 1 client.16203 shutdown 2024-03-18T20:38:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f131cff9700 10 client.25976 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f1339178700 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d493c18c0 0x555d493c3ce0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba9232d6be4c144 server_cookie=9f92def2488cf8d3 in_seq=0 out_seq=0 2024-03-18T20:38:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f214dc49b80 1 -- 172.21.15.46:0/1839885811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2941bf0 msgr2=0x5589d2945990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2941bf0 0x5589d2945990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2134003900 tx=0x7f2134003c10 comp rx=0 tx=0).stop 2024-03-18T20:38:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f214dc49b80 1 -- 172.21.15.46:0/1839885811 shutdown_connections 2024-03-18T20:38:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5589d2966790 0x5589d2968bb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2941bf0 0x5589d2945990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d2931fa0 0x5589d2944f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f211affd700 0 client.16203 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:38:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1839885811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d29412d0 0x5589d2945450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f214dc49b80 1 -- 172.21.15.46:0/1839885811 >> 172.21.15.46:0/1839885811 conn(0x5589d28a8220 msgr2=0x5589d2932bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f214dc49b80 1 -- 172.21.15.46:0/1839885811 shutdown_connections 2024-03-18T20:38:08.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f214dc49b80 1 -- 172.21.15.46:0/1839885811 wait complete. 2024-03-18T20:38:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f131cff9700 1 -- 172.21.15.46:0/1182207492 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555d493076a0 con 0x555d493c18c0 2024-03-18T20:38:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f131cff9700 10 client.25976 handle_command_reply: tid=0 2024-03-18T20:38:08.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f214dc49b80 20 client.16203 trim_cache size 0 max 16384 2024-03-18T20:38:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f1347571b80 1 client.25976 shutdown 2024-03-18T20:38:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f1347571b80 1 -- 172.21.15.46:0/1182207492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bcf40 msgr2=0x555d493cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.926+0000 7f1347571b80 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bcf40 0x555d493cee50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1324005950 tx=0x7f1324099c10 comp rx=0 tx=0).stop 2024-03-18T20:38:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.927+0000 7f1347571b80 1 -- 172.21.15.46:0/1182207492 shutdown_connections 2024-03-18T20:38:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.927+0000 7f1347571b80 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d493c18c0 0x555d493c3ce0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.927+0000 7f1347571b80 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bc580 0x555d493d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.927+0000 7f1347571b80 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493cadd0 0x555d493cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.927+0000 7f1347571b80 1 --2- 172.21.15.46:0/1182207492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bcf40 0x555d493cee50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.927+0000 7f131cff9700 0 client.25976 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.927+0000 7f1347571b80 1 -- 172.21.15.46:0/1182207492 >> 172.21.15.46:0/1182207492 conn(0x555d493321c0 msgr2=0x555d493b7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.927+0000 7f1347571b80 1 -- 172.21.15.46:0/1182207492 shutdown_connections 2024-03-18T20:38:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.927+0000 7f1347571b80 1 -- 172.21.15.46:0/1182207492 wait complete. 2024-03-18T20:38:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.927+0000 7f1347571b80 20 client.25976 trim_cache size 0 max 16384 2024-03-18T20:38:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.940+0000 7f214dc49b80 1 Processor -- start 2024-03-18T20:38:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.940+0000 7f214dc49b80 1 -- start start 2024-03-18T20:38:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.940+0000 7f214dc49b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d2931fa0 0x5589d2931960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.940+0000 7f214dc49b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d29412d0 0x5589d295d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.940+0000 7f214dc49b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d292e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.940+0000 7f214dc49b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2947eb0 con 0x5589d29412d0 2024-03-18T20:38:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.940+0000 7f214dc49b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2959480 con 0x5589d2931fa0 2024-03-18T20:38:08.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.940+0000 7f214dc49b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d2933a40 con 0x5589d2941bf0 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f213affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d29412d0 0x5589d295d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f213affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d29412d0 0x5589d295d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46062/0 (socket says 172.21.15.46:46062) 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f213affd700 1 -- 172.21.15.46:0/1527003611 learned_addr learned my addr 172.21.15.46:0/1527003611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f213b7fe700 1 --2- 172.21.15.46:0/1527003611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d2931fa0 0x5589d2931960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f213bfff700 1 --2- 172.21.15.46:0/1527003611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d292e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f213affd700 1 -- 172.21.15.46:0/1527003611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 msgr2=0x5589d292e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f213affd700 1 --2- 172.21.15.46:0/1527003611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d292e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f213affd700 1 -- 172.21.15.46:0/1527003611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d2931fa0 msgr2=0x5589d2931960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f213affd700 1 --2- 172.21.15.46:0/1527003611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d2931fa0 0x5589d2931960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f213affd700 1 -- 172.21.15.46:0/1527003611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589d284aea0 con 0x5589d29412d0 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f213affd700 1 --2- 172.21.15.46:0/1527003611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d29412d0 0x5589d295d6a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2130005950 tx=0x7f2130003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b833b213b32b110 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f21227fc700 1 -- 172.21.15.46:0/1527003611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213002a050 con 0x5589d29412d0 2024-03-18T20:38:08.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f21227fc700 1 -- 172.21.15.46:0/1527003611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f213002b040 con 0x5589d29412d0 2024-03-18T20:38:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.941+0000 7f21227fc700 1 -- 172.21.15.46:0/1527003611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130005640 con 0x5589d29412d0 2024-03-18T20:38:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.942+0000 7f214dc49b80 1 -- 172.21.15.46:0/1527003611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d29412d0 msgr2=0x5589d295d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.942+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1527003611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d29412d0 0x5589d295d6a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2130005950 tx=0x7f2130003ce0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.942+0000 7f214dc49b80 1 -- 172.21.15.46:0/1527003611 shutdown_connections 2024-03-18T20:38:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.942+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1527003611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d292e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.942+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1527003611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d2931fa0 0x5589d2931960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.942+0000 7f214dc49b80 1 --2- 172.21.15.46:0/1527003611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d29412d0 0x5589d295d6a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.942+0000 7f214dc49b80 1 -- 172.21.15.46:0/1527003611 >> 172.21.15.46:0/1527003611 conn(0x5589d28a8220 msgr2=0x5589d29457f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.942+0000 7f214dc49b80 1 -- 172.21.15.46:0/1527003611 shutdown_connections 2024-03-18T20:38:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.942+0000 7f214dc49b80 1 -- 172.21.15.46:0/1527003611 wait complete. 2024-03-18T20:38:08.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.942+0000 7f214dc49b80 1 -- start start 2024-03-18T20:38:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1347571b80 1 Processor -- start 2024-03-18T20:38:08.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f214dc49b80 1 Processor -- start 2024-03-18T20:38:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1347571b80 1 -- start start 2024-03-18T20:38:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1347571b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bc580 0x555d493cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1347571b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493d6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1347571b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493cadd0 0x555d493b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1347571b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d493d6850 con 0x555d493bcf40 2024-03-18T20:38:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1347571b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d493cd3b0 con 0x555d493bc580 2024-03-18T20:38:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1347571b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d493c9b70 con 0x555d493cadd0 2024-03-18T20:38:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1338977700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493d6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f214dc49b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2931fa0 0x5589d29ae700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f214dc49b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d29412d0 0x5589d29aec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f214dc49b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d29b9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f214dc49b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d29b9ad0 con 0x5589d2931fa0 2024-03-18T20:38:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1338977700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493d6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46064/0 (socket says 172.21.15.46:46064) 2024-03-18T20:38:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1338977700 1 -- 172.21.15.46:0/2825160041 learned_addr learned my addr 172.21.15.46:0/2825160041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f214dc49b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d29b9c10 con 0x5589d29412d0 2024-03-18T20:38:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f214dc49b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589d29b9d50 con 0x5589d2941bf0 2024-03-18T20:38:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f214dc49b80 10 client.0 _ll_register_callbacks cb 0x7f2134004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2931fa0 0x5589d29ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2931fa0 0x5589d29ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46070/0 (socket says 172.21.15.46:46070) 2024-03-18T20:38:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f213b7fe700 1 -- 172.21.15.46:0/3660616895 learned_addr learned my addr 172.21.15.46:0/3660616895 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f213bfff700 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d29b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1339178700 1 --2- 172.21.15.46:0/2825160041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bc580 0x555d493cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f1339979700 1 --2- 172.21.15.46:0/2825160041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493cadd0 0x555d493b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.943+0000 7f213affd700 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d29412d0 0x5589d29aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f1339178700 1 -- 172.21.15.46:0/2825160041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493cadd0 msgr2=0x555d493b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f1339178700 1 --2- 172.21.15.46:0/2825160041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493cadd0 0x555d493b6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f1339178700 1 -- 172.21.15.46:0/2825160041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 msgr2=0x555d493d6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f1339178700 1 --2- 172.21.15.46:0/2825160041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493d6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f1339178700 1 -- 172.21.15.46:0/2825160041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d492d4e40 con 0x555d493bc580 2024-03-18T20:38:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f1338977700 1 --2- 172.21.15.46:0/2825160041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493d6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f213bfff700 1 -- 172.21.15.46:0/3660616895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d29412d0 msgr2=0x5589d29aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f213bfff700 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d29412d0 0x5589d29aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f213bfff700 1 -- 172.21.15.46:0/3660616895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2931fa0 msgr2=0x5589d29ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f213bfff700 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2931fa0 0x5589d29ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f213bfff700 1 -- 172.21.15.46:0/3660616895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589d27063b0 con 0x5589d2941bf0 2024-03-18T20:38:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f213affd700 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d29412d0 0x5589d29aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:08.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f213b7fe700 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2931fa0 0x5589d29ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f1339178700 1 --2- 172.21.15.46:0/2825160041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bc580 0x555d493cca60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1330000c00 tx=0x7f1330005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41ba477bd26e2637 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f213bfff700 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d29b9590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2134005780 tx=0x7f2134012b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19b991aee53105a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f211affd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:38:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f131ffff700 1 -- 172.21.15.46:0/2825160041 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330034070 con 0x555d493bc580 2024-03-18T20:38:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f131ffff700 1 -- 172.21.15.46:0/2825160041 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f133001e040 con 0x555d493bc580 2024-03-18T20:38:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f131ffff700 1 -- 172.21.15.46:0/2825160041 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133001d040 con 0x555d493bc580 2024-03-18T20:38:08.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f211affd700 1 -- 172.21.15.46:0/3660616895 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213400bea0 con 0x5589d2941bf0 2024-03-18T20:38:08.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f211affd700 1 -- 172.21.15.46:0/3660616895 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2134000e40 con 0x5589d2941bf0 2024-03-18T20:38:08.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f214dc49b80 1 -- 172.21.15.46:0/3660616895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21300268c0 con 0x5589d2941bf0 2024-03-18T20:38:08.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.944+0000 7f211affd700 1 -- 172.21.15.46:0/3660616895 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2134011560 con 0x5589d2941bf0 2024-03-18T20:38:08.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f214dc49b80 10 client.25975 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:08.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f214dc49b80 1 -- 172.21.15.46:0/3660616895 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5589d270acd0 con 0x5589d2941bf0 2024-03-18T20:38:08.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f211affd700 1 -- 172.21.15.46:0/3660616895 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2134037020 con 0x5589d2941bf0 2024-03-18T20:38:08.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f211affd700 1 -- 172.21.15.46:0/3660616895 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2134003930 con 0x5589d2941bf0 2024-03-18T20:38:08.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f214dc49b80 10 client.25975 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:08.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f214dc49b80 10 client.25975 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:38:08.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f214dc49b80 20 client.25975 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:08.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f214dc49b80 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5589d29bb400 0x5589d29bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f214dc49b80 4 client.25975 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:38:08.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f214dc49b80 1 -- 172.21.15.46:0/3660616895 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5589d26d3cf0 con 0x5589d29bb400 2024-03-18T20:38:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f1347571b80 1 -- 172.21.15.46:0/2825160041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bc580 msgr2=0x555d493cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f1347571b80 1 --2- 172.21.15.46:0/2825160041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bc580 0x555d493cca60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1330000c00 tx=0x7f1330005f00 comp rx=0 tx=0).stop 2024-03-18T20:38:08.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f213b7fe700 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5589d29bb400 0x5589d29bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f1347571b80 1 -- 172.21.15.46:0/2825160041 shutdown_connections 2024-03-18T20:38:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f1347571b80 1 --2- 172.21.15.46:0/2825160041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493cadd0 0x555d493b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f1347571b80 1 --2- 172.21.15.46:0/2825160041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493bc580 0x555d493cca60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f1347571b80 1 --2- 172.21.15.46:0/2825160041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493d6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f1347571b80 1 -- 172.21.15.46:0/2825160041 >> 172.21.15.46:0/2825160041 conn(0x555d493321c0 msgr2=0x555d493d18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f1347571b80 1 -- 172.21.15.46:0/2825160041 shutdown_connections 2024-03-18T20:38:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.945+0000 7f1347571b80 1 -- 172.21.15.46:0/2825160041 wait complete. 2024-03-18T20:38:08.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f211affd700 10 client.25975 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:38:08.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f213b7fe700 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5589d29bb400 0x5589d29bd820 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71e92d143f7a4284 server_cookie=f703cfd65a7ab44 in_seq=0 out_seq=0 2024-03-18T20:38:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1347571b80 1 -- start start 2024-03-18T20:38:08.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f211affd700 1 -- 172.21.15.46:0/3660616895 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5589d26d3cf0 con 0x5589d29bb400 2024-03-18T20:38:08.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f211affd700 10 client.25975 handle_command_reply: tid=0 2024-03-18T20:38:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1347571b80 1 Processor -- start 2024-03-18T20:38:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f214dc49b80 1 client.25975 shutdown 2024-03-18T20:38:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f214dc49b80 1 -- 172.21.15.46:0/3660616895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 msgr2=0x5589d29b9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f214dc49b80 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d29b9590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2134005780 tx=0x7f2134012b30 comp rx=0 tx=0).stop 2024-03-18T20:38:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f214dc49b80 1 -- 172.21.15.46:0/3660616895 shutdown_connections 2024-03-18T20:38:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f214dc49b80 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5589d29bb400 0x5589d29bd820 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f214dc49b80 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5589d2931fa0 0x5589d29ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f214dc49b80 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5589d2941bf0 0x5589d29b9590 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f211affd700 0 client.25975 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:38:08.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f214dc49b80 1 --2- 172.21.15.46:0/3660616895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589d29412d0 0x5589d29aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f214dc49b80 1 -- 172.21.15.46:0/3660616895 >> 172.21.15.46:0/3660616895 conn(0x5589d28a8220 msgr2=0x5589d293f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f214dc49b80 1 -- 172.21.15.46:0/3660616895 shutdown_connections 2024-03-18T20:38:08.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f214dc49b80 1 -- 172.21.15.46:0/3660616895 wait complete. 2024-03-18T20:38:08.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f214dc49b80 20 client.25975 trim_cache size 0 max 16384 2024-03-18T20:38:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1347571b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 0x555d493d6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1347571b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493c4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1347571b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493cadd0 0x555d493c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1347571b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d493c50f0 con 0x555d493bcf40 2024-03-18T20:38:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1347571b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d493c5260 con 0x555d493cadd0 2024-03-18T20:38:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1347571b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d493c5420 con 0x555d493bc580 2024-03-18T20:38:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1347571b80 10 client.0 _ll_register_callbacks cb 0x555d49116340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1339178700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 0x555d493d6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1339979700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493cadd0 0x555d493c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1338977700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.946+0000 7f1339178700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 0x555d493d6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:35682/0 (socket says 172.21.15.46:35682) 2024-03-18T20:38:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f1339178700 1 -- 172.21.15.46:0/974665036 learned_addr learned my addr 172.21.15.46:0/974665036 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:08.972 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:38:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f1339979700 1 -- 172.21.15.46:0/974665036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 msgr2=0x555d493d6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f1339979700 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 0x555d493d6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f1339979700 1 -- 172.21.15.46:0/974665036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 msgr2=0x555d493c4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f1339979700 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493c4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f1339979700 1 -- 172.21.15.46:0/974665036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d4913cb00 con 0x555d493cadd0 2024-03-18T20:38:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f1339178700 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 0x555d493d6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f1338977700 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:08.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f2141ab4700 1 -- 172.21.15.46:0/51069042 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f21240a1290 msgr2=0x7f21240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f2141ab4700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f21240a1290 0x7f21240a3750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f213c1988d0 tx=0x7f2130009260 comp rx=0 tx=0).stop 2024-03-18T20:38:08.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f2141ab4700 1 -- 172.21.15.46:0/51069042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213c0692a0 msgr2=0x7f213c197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f2141ab4700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213c0692a0 0x7f213c197380 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f212c00bd80 tx=0x7f212c00bdb0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f2141ab4700 1 -- 172.21.15.46:0/51069042 shutdown_connections 2024-03-18T20:38:08.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f2141ab4700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f21240a1290 0x7f21240a3750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f2141ab4700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213c0692a0 0x7f213c197380 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f2141ab4700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f213c19bc60 0x7f213c19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f2141ab4700 1 --2- 172.21.15.46:0/51069042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f213c1060b0 0x7f213c1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f2141ab4700 1 -- 172.21.15.46:0/51069042 >> 172.21.15.46:0/51069042 conn(0x7f213c076d80 msgr2=0x7f213c1071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f1339979700 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493cadd0 0x555d493c4ad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f132c008a90 tx=0x7f132c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ae3761f4b33b7d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f131cff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:38:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f131cff9700 1 -- 172.21.15.46:0/974665036 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c027070 con 0x555d493cadd0 2024-03-18T20:38:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f131cff9700 1 -- 172.21.15.46:0/974665036 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f132c0148a0 con 0x555d493cadd0 2024-03-18T20:38:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f1347571b80 1 -- 172.21.15.46:0/974665036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f133000bbb0 con 0x555d493cadd0 2024-03-18T20:38:08.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f2141ab4700 1 -- 172.21.15.46:0/51069042 shutdown_connections 2024-03-18T20:38:08.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:08.947+0000 7f2141ab4700 1 -- 172.21.15.46:0/51069042 wait complete. 2024-03-18T20:38:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f131cff9700 1 -- 172.21.15.46:0/974665036 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f132c00e040 con 0x555d493cadd0 2024-03-18T20:38:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f1347571b80 10 client.25988 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f1347571b80 1 -- 172.21.15.46:0/974665036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555d49190350 con 0x555d493cadd0 2024-03-18T20:38:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f131cff9700 1 -- 172.21.15.46:0/974665036 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f132c038020 con 0x555d493cadd0 2024-03-18T20:38:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f131cff9700 1 -- 172.21.15.46:0/974665036 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f132c01e050 con 0x555d493cadd0 2024-03-18T20:38:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f1347571b80 10 client.25988 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f1347571b80 10 client.25988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f1347571b80 20 client.25988 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f1347571b80 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d4944dbf0 0x555d49450010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f1347571b80 4 client.25988 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:38:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f1347571b80 1 -- 172.21.15.46:0/974665036 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555d4915dc90 con 0x555d4944dbf0 2024-03-18T20:38:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f1339178700 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d4944dbf0 0x555d49450010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.948+0000 7f131cff9700 10 client.25988 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.949+0000 7f1339178700 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d4944dbf0 0x555d49450010 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c389c6e806c1ed1 server_cookie=4bb96933c46f17a8 in_seq=0 out_seq=0 2024-03-18T20:38:08.979 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:38:08.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:38:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.972+0000 7f131cff9700 1 -- 172.21.15.46:0/974665036 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555d4915dc90 con 0x555d4944dbf0 2024-03-18T20:38:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.972+0000 7f131cff9700 10 client.25988 handle_command_reply: tid=0 2024-03-18T20:38:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.972+0000 7f1347571b80 1 client.25988 shutdown 2024-03-18T20:38:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.972+0000 7f1347571b80 1 -- 172.21.15.46:0/974665036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493cadd0 msgr2=0x555d493c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.972+0000 7f1347571b80 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493cadd0 0x555d493c4ad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f132c008a90 tx=0x7f132c004b40 comp rx=0 tx=0).stop 2024-03-18T20:38:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.972+0000 7f1347571b80 1 -- 172.21.15.46:0/974665036 shutdown_connections 2024-03-18T20:38:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.972+0000 7f1347571b80 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d4944dbf0 0x555d49450010 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f1347571b80 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d493cadd0 0x555d493c4ad0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f1347571b80 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d493bc580 0x555d493d6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f131cff9700 0 client.25988 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f1347571b80 1 --2- 172.21.15.46:0/974665036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d493bcf40 0x555d493c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f1347571b80 1 -- 172.21.15.46:0/974665036 >> 172.21.15.46:0/974665036 conn(0x555d493321c0 msgr2=0x555d493c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f1347571b80 1 -- 172.21.15.46:0/974665036 shutdown_connections 2024-03-18T20:38:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f1347571b80 1 -- 172.21.15.46:0/974665036 wait complete. 2024-03-18T20:38:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f1347571b80 20 client.25988 trim_cache size 0 max 16384 2024-03-18T20:38:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:38:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:38:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:38:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:38:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f133b3dc700 1 -- 172.21.15.46:0/3679005166 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f13200a1310 msgr2=0x7f13200a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f133b3dc700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f13200a1310 0x7f13200a37d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f132c0118e0 tx=0x7f132c00b040 comp rx=0 tx=0).stop 2024-03-18T20:38:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f133b3dc700 1 -- 172.21.15.46:0/3679005166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1334106030 msgr2=0x7f1334197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f133b3dc700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1334106030 0x7f1334197740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f132400e970 tx=0x7f132400ec80 comp rx=0 tx=0).stop 2024-03-18T20:38:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f133b3dc700 1 -- 172.21.15.46:0/3679005166 shutdown_connections 2024-03-18T20:38:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f133b3dc700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f13200a1310 0x7f13200a37d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f133b3dc700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1334100f10 0x7f1334197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f133b3dc700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f133419ba90 0x7f133419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f133b3dc700 1 --2- 172.21.15.46:0/3679005166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1334106030 0x7f1334197740 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.973+0000 7f133b3dc700 1 -- 172.21.15.46:0/3679005166 >> 172.21.15.46:0/3679005166 conn(0x7f13340fc7b0 msgr2=0x7f13340fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.974+0000 7f133b3dc700 1 -- 172.21.15.46:0/3679005166 shutdown_connections 2024-03-18T20:38:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:08.974+0000 7f133b3dc700 1 -- 172.21.15.46:0/3679005166 wait complete. 2024-03-18T20:38:09.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.133+0000 7f6aa73e6700 1 -- 172.21.15.46:0/4039281919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00687f0 msgr2=0x7f6aa010db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.133+0000 7f6aa73e6700 1 --2- 172.21.15.46:0/4039281919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00687f0 0x7f6aa010db50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800b0a0 tx=0x7f6a9800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.133+0000 7f6aa73e6700 1 -- 172.21.15.46:0/4039281919 shutdown_connections 2024-03-18T20:38:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.133+0000 7f6aa73e6700 1 --2- 172.21.15.46:0/4039281919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00687f0 0x7f6aa010db50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.133+0000 7f6aa73e6700 1 --2- 172.21.15.46:0/4039281919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa0105ce0 0x7f6aa01060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.133+0000 7f6aa73e6700 1 --2- 172.21.15.46:0/4039281919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa0067e30 0x7f6aa00682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.133+0000 7f6aa73e6700 1 -- 172.21.15.46:0/4039281919 >> 172.21.15.46:0/4039281919 conn(0x7f6aa0076d40 msgr2=0x7f6aa0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.133+0000 7f6aa73e6700 1 -- 172.21.15.46:0/4039281919 shutdown_connections 2024-03-18T20:38:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.133+0000 7f6aa73e6700 1 -- 172.21.15.46:0/4039281919 wait complete. 2024-03-18T20:38:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.133+0000 7f6aa73e6700 1 Processor -- start 2024-03-18T20:38:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa73e6700 1 -- start start 2024-03-18T20:38:09.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa73e6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa0067e30 0x7f6aa0197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa73e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00687f0 0x7f6aa0197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa73e6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa0105ce0 0x7f6aa019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa73e6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa019c230 con 0x7f6aa00687f0 2024-03-18T20:38:09.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa73e6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa019c3a0 con 0x7f6aa0105ce0 2024-03-18T20:38:09.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa73e6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa019c560 con 0x7f6aa0067e30 2024-03-18T20:38:09.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa4981700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00687f0 0x7f6aa0197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa4981700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00687f0 0x7f6aa0197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46084/0 (socket says 172.21.15.46:46084) 2024-03-18T20:38:09.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa4981700 1 -- 172.21.15.46:0/1140117465 learned_addr learned my addr 172.21.15.46:0/1140117465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa5182700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa0067e30 0x7f6aa0197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa5983700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa0105ce0 0x7f6aa019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa4981700 1 -- 172.21.15.46:0/1140117465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa0067e30 msgr2=0x7f6aa0197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa4981700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa0067e30 0x7f6aa0197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa4981700 1 -- 172.21.15.46:0/1140117465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa0105ce0 msgr2=0x7f6aa019bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa4981700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa0105ce0 0x7f6aa019bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa4981700 1 -- 172.21.15.46:0/1140117465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a98009d00 con 0x7f6aa00687f0 2024-03-18T20:38:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa5983700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa0105ce0 0x7f6aa019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.134+0000 7f6aa5182700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa0067e30 0x7f6aa0197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.135+0000 7f6aa4981700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00687f0 0x7f6aa0197850 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000e9a0 tx=0x7f6a9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.135+0000 7f6a967fc700 1 -- 172.21.15.46:0/1140117465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90009790 con 0x7f6aa00687f0 2024-03-18T20:38:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.135+0000 7f6a967fc700 1 -- 172.21.15.46:0/1140117465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a90010430 con 0x7f6aa00687f0 2024-03-18T20:38:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.135+0000 7f6aa73e6700 1 -- 172.21.15.46:0/1140117465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa019c840 con 0x7f6aa00687f0 2024-03-18T20:38:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.135+0000 7f6a967fc700 1 -- 172.21.15.46:0/1140117465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90018680 con 0x7f6aa00687f0 2024-03-18T20:38:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.135+0000 7f6aa73e6700 1 -- 172.21.15.46:0/1140117465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa01a4080 con 0x7f6aa00687f0 2024-03-18T20:38:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.137+0000 7f6a967fc700 1 -- 172.21.15.46:0/1140117465 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6a900105a0 con 0x7f6aa00687f0 2024-03-18T20:38:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.138+0000 7f6a967fc700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6a8c0a1290 0x7f6a8c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.138+0000 7f6aa5182700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6a8c0a1290 0x7f6a8c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.138+0000 7f6a967fc700 1 -- 172.21.15.46:0/1140117465 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6a90014070 con 0x7f6aa00687f0 2024-03-18T20:38:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.139+0000 7f6aa5182700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6a8c0a1290 0x7f6a8c0a3750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c011490 tx=0x7f6a9c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.139+0000 7f6ab357bb80 1 Processor -- start 2024-03-18T20:38:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.139+0000 7f6ab357bb80 1 -- start start 2024-03-18T20:38:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6ab357bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5499ffa60 0x55f5499ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6ab357bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a0ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6ab357bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a0b220 0x55f549a0d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6ab357bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f549959990 con 0x55f549a0b220 2024-03-18T20:38:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6ab357bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f549957350 con 0x55f5499ffa60 2024-03-18T20:38:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6ab357bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f549952c40 con 0x55f549a00380 2024-03-18T20:38:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6aa5983700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a0b220 0x55f549a0d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6aa5983700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a0b220 0x55f549a0d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46088/0 (socket says 172.21.15.46:46088) 2024-03-18T20:38:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6aa5983700 1 -- 172.21.15.46:0/3315190867 learned_addr learned my addr 172.21.15.46:0/3315190867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6aa4981700 1 --2- 172.21.15.46:0/3315190867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a0ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6aa5182700 1 --2- 172.21.15.46:0/3315190867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5499ffa60 0x55f5499ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6aa5983700 1 -- 172.21.15.46:0/3315190867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 msgr2=0x55f549a0ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6aa5983700 1 --2- 172.21.15.46:0/3315190867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a0ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6aa5983700 1 -- 172.21.15.46:0/3315190867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5499ffa60 msgr2=0x55f5499ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6aa5983700 1 --2- 172.21.15.46:0/3315190867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5499ffa60 0x55f5499ffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6aa5983700 1 -- 172.21.15.46:0/3315190867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5497d43b0 con 0x55f549a0b220 2024-03-18T20:38:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6aa4981700 1 --2- 172.21.15.46:0/3315190867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a0ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.140+0000 7f6aa5983700 1 --2- 172.21.15.46:0/3315190867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a0b220 0x55f549a0d610 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009b80 tx=0x7f6a98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02c01fdd1bffd43 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.141+0000 7f6a8b7fe700 1 -- 172.21.15.46:0/3315190867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9800e040 con 0x55f549a0b220 2024-03-18T20:38:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.141+0000 7f6a8b7fe700 1 -- 172.21.15.46:0/3315190867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a98006040 con 0x55f549a0b220 2024-03-18T20:38:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.141+0000 7f6a8b7fe700 1 -- 172.21.15.46:0/3315190867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98005040 con 0x55f549a0b220 2024-03-18T20:38:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.141+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3315190867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a0b220 msgr2=0x55f549a0d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.141+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/3315190867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a0b220 0x55f549a0d610 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009b80 tx=0x7f6a98009bb0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.141+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3315190867 shutdown_connections 2024-03-18T20:38:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.141+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/3315190867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a0b220 0x55f549a0d610 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.141+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/3315190867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5499ffa60 0x55f5499ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.141+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/3315190867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a0ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.141+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3315190867 >> 172.21.15.46:0/3315190867 conn(0x55f549976220 msgr2=0x55f5499fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.142+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3315190867 shutdown_connections 2024-03-18T20:38:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.142+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3315190867 wait complete. 2024-03-18T20:38:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.142+0000 7f6ab357bb80 1 -- start start 2024-03-18T20:38:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.142+0000 7f6ab357bb80 1 Processor -- start 2024-03-18T20:38:09.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.142+0000 7f6ab357bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5499ffa60 0x55f549a0ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.142+0000 7f6ab357bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a00380 0x55f549a07ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.142+0000 7f6ab357bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a08550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6ab357bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5498eed80 con 0x55f549a00380 2024-03-18T20:38:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6ab357bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5498eea20 con 0x55f549a0b220 2024-03-18T20:38:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6ab357bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5497d66a0 con 0x55f5499ffa60 2024-03-18T20:38:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6ab357bb80 10 client.0 _ll_register_callbacks cb 0x55f54975a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6aa4981700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a00380 0x55f549a07ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6aa4981700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a00380 0x55f549a07ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46094/0 (socket says 172.21.15.46:46094) 2024-03-18T20:38:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6aa4981700 1 -- 172.21.15.46:0/3240640207 learned_addr learned my addr 172.21.15.46:0/3240640207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6aa5983700 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a08550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6aa5182700 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5499ffa60 0x55f549a0ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6aa4981700 1 -- 172.21.15.46:0/3240640207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5499ffa60 msgr2=0x55f549a0ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6aa4981700 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5499ffa60 0x55f549a0ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6aa4981700 1 -- 172.21.15.46:0/3240640207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 msgr2=0x55f549a08550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6aa4981700 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a08550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6aa4981700 1 -- 172.21.15.46:0/3240640207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa01a4080 con 0x55f549a00380 2024-03-18T20:38:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6aa5182700 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5499ffa60 0x55f549a0ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6aa4981700 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a00380 0x55f549a07ef0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6a90096a20 tx=0x7f6a9009a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f707348a18a105 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6a88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6a88ff9700 1 -- 172.21.15.46:0/3240640207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a900996a0 con 0x55f549a00380 2024-03-18T20:38:09.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6a88ff9700 1 -- 172.21.15.46:0/3240640207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6a900a2430 con 0x55f549a00380 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.143+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3240640207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5499f3ad0 con 0x55f549a00380 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.144+0000 7f6a88ff9700 1 -- 172.21.15.46:0/3240640207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a900a1fa0 con 0x55f549a00380 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.144+0000 7f6ab357bb80 10 client.16230 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.144+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3240640207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f6aa010e550 con 0x55f549a00380 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.144+0000 7f6a88ff9700 1 -- 172.21.15.46:0/3240640207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5499f3ad0 con 0x55f549a00380 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.144+0000 7f6a88ff9700 1 -- 172.21.15.46:0/3240640207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6a90094c30 con 0x55f549a00380 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.144+0000 7f6ab357bb80 10 client.16230 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.144+0000 7f6ab357bb80 10 client.16230 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.144+0000 7f6ab357bb80 20 client.16230 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.144+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f549a1adb0 0x55f549a1d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.144+0000 7f6ab357bb80 4 client.16230 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.144+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3240640207 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f54994b700 con 0x55f549a1adb0 2024-03-18T20:38:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.144+0000 7f6aa5182700 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f549a1adb0 0x55f549a1d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6a88ff9700 10 client.16230 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:38:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6aa5182700 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f549a1adb0 0x55f549a1d1d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=80942793fb163a91 server_cookie=fb0db8d3978fbcf0 in_seq=0 out_seq=0 2024-03-18T20:38:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6a88ff9700 1 -- 172.21.15.46:0/3240640207 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f54994b700 con 0x55f549a1adb0 2024-03-18T20:38:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6a88ff9700 10 client.16230 handle_command_reply: tid=0 2024-03-18T20:38:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6ab357bb80 1 client.16230 shutdown 2024-03-18T20:38:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3240640207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a00380 msgr2=0x55f549a07ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a00380 0x55f549a07ef0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6a90096a20 tx=0x7f6a9009a730 comp rx=0 tx=0).stop 2024-03-18T20:38:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3240640207 shutdown_connections 2024-03-18T20:38:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a08550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5499ffa60 0x55f549a0ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f549a1adb0 0x55f549a1d1d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/3240640207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f549a00380 0x55f549a07ef0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6a88ff9700 0 client.16230 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:38:09.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3240640207 >> 172.21.15.46:0/3240640207 conn(0x55f549976220 msgr2=0x55f5499fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.145+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3240640207 shutdown_connections 2024-03-18T20:38:09.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.146+0000 7f6ab357bb80 1 -- 172.21.15.46:0/3240640207 wait complete. 2024-03-18T20:38:09.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.146+0000 7f6ab357bb80 20 client.16230 trim_cache size 0 max 16384 2024-03-18T20:38:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.160+0000 7f6ab357bb80 1 Processor -- start 2024-03-18T20:38:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.160+0000 7f6ab357bb80 1 -- start start 2024-03-18T20:38:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6ab357bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 0x55f549a051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6ab357bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a056f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6ab357bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a16540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6ab357bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f549a06480 con 0x55f5499ffa60 2024-03-18T20:38:09.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6ab357bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5499fadd0 con 0x55f549a0b220 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6ab357bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f549a03d90 con 0x55f549a00380 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6aa5182700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 0x55f549a051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6aa5182700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 0x55f549a051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46108/0 (socket says 172.21.15.46:46108) 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6aa5182700 1 -- 172.21.15.46:0/997840590 learned_addr learned my addr 172.21.15.46:0/997840590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6aa5983700 1 --2- 172.21.15.46:0/997840590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a16540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6aa4981700 1 --2- 172.21.15.46:0/997840590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6aa5182700 1 -- 172.21.15.46:0/997840590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 msgr2=0x55f549a056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6aa5182700 1 --2- 172.21.15.46:0/997840590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a056f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6aa5182700 1 -- 172.21.15.46:0/997840590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 msgr2=0x55f549a16540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6aa5182700 1 --2- 172.21.15.46:0/997840590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a16540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6aa5182700 1 -- 172.21.15.46:0/997840590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f549780b60 con 0x55f5499ffa60 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6aa5983700 1 --2- 172.21.15.46:0/997840590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a16540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:09.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.161+0000 7f6aa5182700 1 --2- 172.21.15.46:0/997840590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 0x55f549a051b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55f549a1da80 tx=0x7f6a9c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4865b9abfb6c0094 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.162+0000 7f6a8bfff700 1 -- 172.21.15.46:0/997840590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c024040 con 0x55f5499ffa60 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.162+0000 7f6a8bfff700 1 -- 172.21.15.46:0/997840590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a9c029dc0 con 0x55f5499ffa60 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.162+0000 7f6a8bfff700 1 -- 172.21.15.46:0/997840590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c01e950 con 0x55f5499ffa60 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.162+0000 7f6ab357bb80 1 -- 172.21.15.46:0/997840590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 msgr2=0x55f549a051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.162+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/997840590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 0x55f549a051b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55f549a1da80 tx=0x7f6a9c014000 comp rx=0 tx=0).stop 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.162+0000 7f6ab357bb80 1 -- 172.21.15.46:0/997840590 shutdown_connections 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.162+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/997840590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 0x55f549a051b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.162+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/997840590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a16540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.162+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/997840590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.162+0000 7f6ab357bb80 1 -- 172.21.15.46:0/997840590 >> 172.21.15.46:0/997840590 conn(0x55f549976220 msgr2=0x55f549a06800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.163+0000 7f6ab357bb80 1 -- 172.21.15.46:0/997840590 shutdown_connections 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.163+0000 7f6ab357bb80 1 -- 172.21.15.46:0/997840590 wait complete. 2024-03-18T20:38:09.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.163+0000 7f6ab357bb80 1 -- start start 2024-03-18T20:38:09.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.163+0000 7f6ab357bb80 1 Processor -- start 2024-03-18T20:38:09.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6ab357bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 0x55f549a11be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6ab357bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a0ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6ab357bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6ab357bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f549a10a30 con 0x55f5499ffa60 2024-03-18T20:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6ab357bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f549a10b70 con 0x55f549a0b220 2024-03-18T20:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6ab357bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f549a10d30 con 0x55f549a00380 2024-03-18T20:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6ab357bb80 10 client.0 _ll_register_callbacks cb 0x7f6a90099af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6aa5182700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 0x55f549a11be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6aa5182700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 0x55f549a11be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46120/0 (socket says 172.21.15.46:46120) 2024-03-18T20:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6aa5182700 1 -- 172.21.15.46:0/4109698077 learned_addr learned my addr 172.21.15.46:0/4109698077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6aa5983700 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6aa4981700 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a0ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6aa5182700 1 -- 172.21.15.46:0/4109698077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 msgr2=0x55f549a0ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6aa5182700 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a0ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6aa5182700 1 -- 172.21.15.46:0/4109698077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 msgr2=0x55f549a104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6aa5182700 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6aa5182700 1 -- 172.21.15.46:0/4109698077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5497d43b0 con 0x55f5499ffa60 2024-03-18T20:38:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.164+0000 7f6aa4981700 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a0ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.165+0000 7f6aa5182700 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 0x55f549a11be0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c005820 tx=0x7f6a9c01e3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1855d0c6f2cbc56 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.165+0000 7f6a88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.165+0000 7f6a88ff9700 1 -- 172.21.15.46:0/4109698077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c024050 con 0x55f5499ffa60 2024-03-18T20:38:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.165+0000 7f6a88ff9700 1 -- 172.21.15.46:0/4109698077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6a9c029900 con 0x55f5499ffa60 2024-03-18T20:38:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.165+0000 7f6ab357bb80 1 -- 172.21.15.46:0/4109698077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a98006040 con 0x55f5499ffa60 2024-03-18T20:38:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.165+0000 7f6a88ff9700 1 -- 172.21.15.46:0/4109698077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a9c0295c0 con 0x55f5499ffa60 2024-03-18T20:38:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.165+0000 7f6ab357bb80 10 client.16236 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:09.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.165+0000 7f6ab357bb80 1 -- 172.21.15.46:0/4109698077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f6a9c017070 con 0x55f5499ffa60 2024-03-18T20:38:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.165+0000 7f6a88ff9700 1 -- 172.21.15.46:0/4109698077 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a98006040 con 0x55f5499ffa60 2024-03-18T20:38:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.165+0000 7f6a88ff9700 1 -- 172.21.15.46:0/4109698077 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6a9c043470 con 0x55f5499ffa60 2024-03-18T20:38:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6ab357bb80 10 client.16236 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6ab357bb80 10 client.16236 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:38:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6ab357bb80 20 client.16236 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f549a6ff60 0x55f549a72350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6ab357bb80 4 client.16236 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:38:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6ab357bb80 1 -- 172.21.15.46:0/4109698077 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f5497a1cf0 con 0x55f549a6ff60 2024-03-18T20:38:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6aa4981700 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f549a6ff60 0x55f549a72350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6a88ff9700 10 client.16236 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:38:09.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6aa4981700 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f549a6ff60 0x55f549a72350 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=739279aadc9b6b39 server_cookie=f800dd9ca1ab36e9 in_seq=0 out_seq=0 2024-03-18T20:38:09.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6a88ff9700 1 -- 172.21.15.46:0/4109698077 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f5497a1cf0 con 0x55f549a6ff60 2024-03-18T20:38:09.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6a88ff9700 10 client.16236 handle_command_reply: tid=0 2024-03-18T20:38:09.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6ab357bb80 1 client.16236 shutdown 2024-03-18T20:38:09.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6ab357bb80 1 -- 172.21.15.46:0/4109698077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 msgr2=0x55f549a11be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.166+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 0x55f549a11be0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c005820 tx=0x7f6a9c01e3f0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6ab357bb80 1 -- 172.21.15.46:0/4109698077 shutdown_connections 2024-03-18T20:38:09.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f549a6ff60 0x55f549a72350 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5499ffa60 0x55f549a11be0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6a88ff9700 0 client.16236 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:38:09.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f549a0b220 0x55f549a104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6ab357bb80 1 --2- 172.21.15.46:0/4109698077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f549a00380 0x55f549a0ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6ab357bb80 1 -- 172.21.15.46:0/4109698077 >> 172.21.15.46:0/4109698077 conn(0x55f549976220 msgr2=0x55f549a16540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6ab357bb80 1 -- 172.21.15.46:0/4109698077 shutdown_connections 2024-03-18T20:38:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6ab357bb80 1 -- 172.21.15.46:0/4109698077 wait complete. 2024-03-18T20:38:09.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6ab357bb80 20 client.16236 trim_cache size 0 max 16384 2024-03-18T20:38:09.177 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:38:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6aa73e6700 1 -- 172.21.15.46:0/1140117465 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6a8c0a1290 msgr2=0x7f6a8c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6aa73e6700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6a8c0a1290 0x7f6a8c0a3750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c011490 tx=0x7f6a9c009260 comp rx=0 tx=0).stop 2024-03-18T20:38:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6aa73e6700 1 -- 172.21.15.46:0/1140117465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00687f0 msgr2=0x7f6aa0197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6aa73e6700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00687f0 0x7f6aa0197850 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000e9a0 tx=0x7f6a9000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6aa73e6700 1 -- 172.21.15.46:0/1140117465 shutdown_connections 2024-03-18T20:38:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6aa73e6700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6a8c0a1290 0x7f6a8c0a3750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6aa73e6700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aa0105ce0 0x7f6aa019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6aa73e6700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa0067e30 0x7f6aa0197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6aa73e6700 1 --2- 172.21.15.46:0/1140117465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6aa00687f0 0x7f6aa0197850 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.167+0000 7f6aa73e6700 1 -- 172.21.15.46:0/1140117465 >> 172.21.15.46:0/1140117465 conn(0x7f6aa0076d40 msgr2=0x7f6aa00fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.168+0000 7f6aa73e6700 1 -- 172.21.15.46:0/1140117465 shutdown_connections 2024-03-18T20:38:09.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.168+0000 7f6aa73e6700 1 -- 172.21.15.46:0/1140117465 wait complete. 2024-03-18T20:38:09.184 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:38:09.184 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:38:09.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:09 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1085733222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:38:09.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.320+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3311294388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae00ffe50 msgr2=0x7f5ae0100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.320+0000 7f5ae6c3c700 1 --2- 172.21.15.46:0/3311294388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae00ffe50 0x7f5ae0100230 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009990 tx=0x7f5ad4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.320+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3311294388 shutdown_connections 2024-03-18T20:38:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.320+0000 7f5ae6c3c700 1 --2- 172.21.15.46:0/3311294388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae00ffe50 0x7f5ae0100230 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.320+0000 7f5ae6c3c700 1 --2- 172.21.15.46:0/3311294388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae0104f60 0x7f5ae010cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.320+0000 7f5ae6c3c700 1 --2- 172.21.15.46:0/3311294388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ae0100800 0x7f5ae0104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.320+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3311294388 >> 172.21.15.46:0/3311294388 conn(0x7f5ae0076d40 msgr2=0x7f5ae0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.320+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3311294388 shutdown_connections 2024-03-18T20:38:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.320+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3311294388 wait complete. 2024-03-18T20:38:09.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.321+0000 7f5ae6c3c700 1 Processor -- start 2024-03-18T20:38:09.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.321+0000 7f5ae6c3c700 1 -- start start 2024-03-18T20:38:09.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.321+0000 7f5ae6c3c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae00ffe50 0x7f5ae0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.321+0000 7f5ae6c3c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae0104f60 0x7f5ae0197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.321+0000 7f5ae6c3c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ae019ba90 0x7f5ae019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.321+0000 7f5ae6c3c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae019c510 con 0x7f5ae00ffe50 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.321+0000 7f5ae6c3c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae019c680 con 0x7f5ae019ba90 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.321+0000 7f5ae6c3c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae019c840 con 0x7f5ae0104f60 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5ae49d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae00ffe50 0x7f5ae0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5ae49d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae00ffe50 0x7f5ae0197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46138/0 (socket says 172.21.15.46:46138) 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5ae49d8700 1 -- 172.21.15.46:0/3565959740 learned_addr learned my addr 172.21.15.46:0/3565959740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5adffff700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae0104f60 0x7f5ae0197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5ae51d9700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ae019ba90 0x7f5ae019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5ae49d8700 1 -- 172.21.15.46:0/3565959740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae0104f60 msgr2=0x7f5ae0197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5ae49d8700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae0104f60 0x7f5ae0197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5ae49d8700 1 -- 172.21.15.46:0/3565959740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ae019ba90 msgr2=0x7f5ae019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5ae49d8700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ae019ba90 0x7f5ae019bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5ae49d8700 1 -- 172.21.15.46:0/3565959740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad40095f0 con 0x7f5ae00ffe50 2024-03-18T20:38:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5adffff700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae0104f60 0x7f5ae0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5ae49d8700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae00ffe50 0x7f5ae0197200 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4004760 tx=0x7f5ad4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5addffb700 1 -- 172.21.15.46:0/3565959740 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad401d070 con 0x7f5ae00ffe50 2024-03-18T20:38:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5addffb700 1 -- 172.21.15.46:0/3565959740 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ad4022470 con 0x7f5ae00ffe50 2024-03-18T20:38:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3565959740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae01a3d40 con 0x7f5ae00ffe50 2024-03-18T20:38:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.322+0000 7f5addffb700 1 -- 172.21.15.46:0/3565959740 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4017650 con 0x7f5ae00ffe50 2024-03-18T20:38:09.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.323+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3565959740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae01a4160 con 0x7f5ae00ffe50 2024-03-18T20:38:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.325+0000 7f5addffb700 1 -- 172.21.15.46:0/3565959740 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5ad40225e0 con 0x7f5ae00ffe50 2024-03-18T20:38:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.325+0000 7f5addffb700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5ad00a1290 0x7f5ad00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.326+0000 7f5addffb700 1 -- 172.21.15.46:0/3565959740 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5ad40cdef0 con 0x7f5ae00ffe50 2024-03-18T20:38:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.326+0000 7f5adffff700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5ad00a1290 0x7f5ad00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.326+0000 7f5adffff700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5ad00a1290 0x7f5ad00a3750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0198700 tx=0x7f5acc0073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.326+0000 7f5af2dd1b80 1 Processor -- start 2024-03-18T20:38:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.326+0000 7f5af2dd1b80 1 -- start start 2024-03-18T20:38:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5af2dd1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5efa60 0x56077e5efe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5af2dd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5f0380 0x56077e5face0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5af2dd1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5fb220 0x56077e5fd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5af2dd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077e549990 con 0x56077e5f0380 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5af2dd1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077e547350 con 0x56077e5fb220 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5af2dd1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077e542c40 con 0x56077e5efa60 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5adffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5f0380 0x56077e5face0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5adffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5f0380 0x56077e5face0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46140/0 (socket says 172.21.15.46:46140) 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5adffff700 1 -- 172.21.15.46:0/2588919917 learned_addr learned my addr 172.21.15.46:0/2588919917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5ae49d8700 1 --2- 172.21.15.46:0/2588919917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5efa60 0x56077e5efe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5ae51d9700 1 --2- 172.21.15.46:0/2588919917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5fb220 0x56077e5fd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5adffff700 1 -- 172.21.15.46:0/2588919917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5efa60 msgr2=0x56077e5efe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5adffff700 1 --2- 172.21.15.46:0/2588919917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5efa60 0x56077e5efe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5adffff700 1 -- 172.21.15.46:0/2588919917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5fb220 msgr2=0x56077e5fd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5adffff700 1 --2- 172.21.15.46:0/2588919917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5fb220 0x56077e5fd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5adffff700 1 -- 172.21.15.46:0/2588919917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56077e3c43b0 con 0x56077e5f0380 2024-03-18T20:38:09.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5ae49d8700 1 --2- 172.21.15.46:0/2588919917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5efa60 0x56077e5efe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5ae51d9700 1 --2- 172.21.15.46:0/2588919917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5fb220 0x56077e5fd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5adffff700 1 --2- 172.21.15.46:0/2588919917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5f0380 0x56077e5face0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5acc004380 tx=0x7f5acc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802414c1010f11c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.327+0000 7f5acaffd700 1 -- 172.21.15.46:0/2588919917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc003900 con 0x56077e5f0380 2024-03-18T20:38:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.328+0000 7f5acaffd700 1 -- 172.21.15.46:0/2588919917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5acc011440 con 0x56077e5f0380 2024-03-18T20:38:09.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.328+0000 7f5acaffd700 1 -- 172.21.15.46:0/2588919917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc01f690 con 0x56077e5f0380 2024-03-18T20:38:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.328+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2588919917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5f0380 msgr2=0x56077e5face0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.328+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2588919917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5f0380 0x56077e5face0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5acc004380 tx=0x7f5acc013000 comp rx=0 tx=0).stop 2024-03-18T20:38:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.328+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2588919917 shutdown_connections 2024-03-18T20:38:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.328+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2588919917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5fb220 0x56077e5fd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.328+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2588919917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5efa60 0x56077e5efe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.328+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2588919917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5f0380 0x56077e5face0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.328+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2588919917 >> 172.21.15.46:0/2588919917 conn(0x56077e566220 msgr2=0x56077e5ec020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.328+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2588919917 shutdown_connections 2024-03-18T20:38:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.328+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2588919917 wait complete. 2024-03-18T20:38:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.329+0000 7f5af2dd1b80 1 -- start start 2024-03-18T20:38:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.329+0000 7f5af2dd1b80 1 Processor -- start 2024-03-18T20:38:09.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.329+0000 7f5af2dd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5efa60 0x56077e5f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.329+0000 7f5af2dd1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5f0380 0x56077e5f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.329+0000 7f5af2dd1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5fb220 0x56077e5f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.329+0000 7f5af2dd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077e4ded80 con 0x56077e5efa60 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.329+0000 7f5af2dd1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077e4dea20 con 0x56077e5f0380 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.329+0000 7f5af2dd1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077e3c66a0 con 0x56077e5fb220 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.329+0000 7f5af2dd1b80 10 client.0 _ll_register_callbacks cb 0x56077e34a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5ae49d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5efa60 0x56077e5f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5ae49d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5efa60 0x56077e5f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46146/0 (socket says 172.21.15.46:46146) 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5ae49d8700 1 -- 172.21.15.46:0/3244231555 learned_addr learned my addr 172.21.15.46:0/3244231555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5ae51d9700 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5fb220 0x56077e5f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5adffff700 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5f0380 0x56077e5f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5ae49d8700 1 -- 172.21.15.46:0/3244231555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5fb220 msgr2=0x56077e5f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5ae49d8700 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5fb220 0x56077e5f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5ae49d8700 1 -- 172.21.15.46:0/3244231555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5f0380 msgr2=0x56077e5f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5ae49d8700 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5f0380 0x56077e5f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5ae49d8700 1 -- 172.21.15.46:0/3244231555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae01a4160 con 0x56077e5efa60 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5ae49d8700 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5efa60 0x56077e5f7f90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40cb670 tx=0x7f5ad409aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738c4696aead29c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5abffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5abffff700 1 -- 172.21.15.46:0/3244231555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40aeea0 con 0x56077e5efa60 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5abffff700 1 -- 172.21.15.46:0/3244231555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5ad409ee90 con 0x56077e5efa60 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/3244231555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56077e5e3ad0 con 0x56077e5efa60 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5abffff700 1 -- 172.21.15.46:0/3244231555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ad40a3450 con 0x56077e5efa60 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5af2dd1b80 10 client.16248 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.330+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/3244231555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f5acc004030 con 0x56077e5efa60 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.331+0000 7f5abffff700 1 -- 172.21.15.46:0/3244231555 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ad407b080 con 0x56077e5efa60 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.331+0000 7f5abffff700 1 -- 172.21.15.46:0/3244231555 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5ad409a650 con 0x56077e5efa60 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.331+0000 7f5af2dd1b80 10 client.16248 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:09.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.331+0000 7f5af2dd1b80 10 client.16248 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.331+0000 7f5af2dd1b80 20 client.16248 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.331+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56077e60aaa0 0x56077e60cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.331+0000 7f5af2dd1b80 4 client.16248 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.331+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/3244231555 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56077e53b700 con 0x56077e60aaa0 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.331+0000 7f5adffff700 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56077e60aaa0 0x56077e60cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.332+0000 7f5adffff700 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56077e60aaa0 0x56077e60cec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c894fc2a3834d4c server_cookie=c049b809ce7f64f9 in_seq=0 out_seq=0 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.332+0000 7f5abffff700 10 client.16248 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.332+0000 7f5abffff700 1 -- 172.21.15.46:0/3244231555 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56077e53b700 con 0x56077e60aaa0 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.332+0000 7f5abffff700 10 client.16248 handle_command_reply: tid=0 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.332+0000 7f5af2dd1b80 1 client.16248 shutdown 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.333+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/3244231555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5efa60 msgr2=0x56077e5f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.333+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5efa60 0x56077e5f7f90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40cb670 tx=0x7f5ad409aa90 comp rx=0 tx=0).stop 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.333+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/3244231555 shutdown_connections 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.333+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56077e60aaa0 0x56077e60cec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.333+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5efa60 0x56077e5f7f90 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.333+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5fb220 0x56077e5f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.333+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/3244231555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5f0380 0x56077e5f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.333+0000 7f5abffff700 0 client.16248 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:38:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.333+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/3244231555 >> 172.21.15.46:0/3244231555 conn(0x56077e566220 msgr2=0x56077e5ebc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.333+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/3244231555 shutdown_connections 2024-03-18T20:38:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.333+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/3244231555 wait complete. 2024-03-18T20:38:09.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.333+0000 7f5af2dd1b80 20 client.16248 trim_cache size 0 max 16384 2024-03-18T20:38:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5af2dd1b80 1 Processor -- start 2024-03-18T20:38:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5af2dd1b80 1 -- start start 2024-03-18T20:38:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5af2dd1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5efa60 0x56077e5f8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5af2dd1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5f0380 0x56077e5f6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5af2dd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 0x56077e5f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5af2dd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077e5e6210 con 0x56077e5fb220 2024-03-18T20:38:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5af2dd1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077e5f9420 con 0x56077e5efa60 2024-03-18T20:38:09.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5af2dd1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077e5f9600 con 0x56077e5f0380 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5ae51d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 0x56077e5f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5ae51d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 0x56077e5f6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46152/0 (socket says 172.21.15.46:46152) 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5ae51d9700 1 -- 172.21.15.46:0/2269866231 learned_addr learned my addr 172.21.15.46:0/2269866231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5adffff700 1 --2- 172.21.15.46:0/2269866231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5f0380 0x56077e5f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5ae49d8700 1 --2- 172.21.15.46:0/2269866231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5efa60 0x56077e5f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5ae51d9700 1 -- 172.21.15.46:0/2269866231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5f0380 msgr2=0x56077e5f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5ae51d9700 1 --2- 172.21.15.46:0/2269866231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5f0380 0x56077e5f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5ae51d9700 1 -- 172.21.15.46:0/2269866231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5efa60 msgr2=0x56077e5f8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5ae51d9700 1 --2- 172.21.15.46:0/2269866231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5efa60 0x56077e5f8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5ae51d9700 1 -- 172.21.15.46:0/2269866231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56077e370b60 con 0x56077e5fb220 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.347+0000 7f5adffff700 1 --2- 172.21.15.46:0/2269866231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5f0380 0x56077e5f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5ae51d9700 1 --2- 172.21.15.46:0/2269866231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 0x56077e5f6da0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8007710 tx=0x7f5ad800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4a0d529e3bcd36 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5acb7fe700 1 -- 172.21.15.46:0/2269866231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad800e040 con 0x56077e5fb220 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5acb7fe700 1 -- 172.21.15.46:0/2269866231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ad800f040 con 0x56077e5fb220 2024-03-18T20:38:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5acb7fe700 1 -- 172.21.15.46:0/2269866231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8017700 con 0x56077e5fb220 2024-03-18T20:38:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2269866231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 msgr2=0x56077e5f6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2269866231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 0x56077e5f6da0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8007710 tx=0x7f5ad800b040 comp rx=0 tx=0).stop 2024-03-18T20:38:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2269866231 shutdown_connections 2024-03-18T20:38:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2269866231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 0x56077e5f6da0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2269866231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5efa60 0x56077e5f8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2269866231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5f0380 0x56077e5f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2269866231 >> 172.21.15.46:0/2269866231 conn(0x56077e566220 msgr2=0x56077e5f3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2269866231 shutdown_connections 2024-03-18T20:38:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.348+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2269866231 wait complete. 2024-03-18T20:38:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.349+0000 7f5af2dd1b80 1 -- start start 2024-03-18T20:38:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.349+0000 7f5af2dd1b80 1 Processor -- start 2024-03-18T20:38:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.349+0000 7f5af2dd1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5efa60 0x56077e601820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5af2dd1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5f0380 0x56077e601d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5af2dd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 0x56077e5fff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5af2dd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077e6004f0 con 0x56077e5fb220 2024-03-18T20:38:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5af2dd1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077e600660 con 0x56077e5efa60 2024-03-18T20:38:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5af2dd1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077e600820 con 0x56077e5f0380 2024-03-18T20:38:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5af2dd1b80 10 client.0 _ll_register_callbacks cb 0x56077e34a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5ae51d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 0x56077e5fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5ae49d8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5efa60 0x56077e601820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5ae51d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 0x56077e5fff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46160/0 (socket says 172.21.15.46:46160) 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5ae51d9700 1 -- 172.21.15.46:0/2702237728 learned_addr learned my addr 172.21.15.46:0/2702237728 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5adffff700 1 --2- 172.21.15.46:0/2702237728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5f0380 0x56077e601d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5ae51d9700 1 -- 172.21.15.46:0/2702237728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5f0380 msgr2=0x56077e601d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5ae51d9700 1 --2- 172.21.15.46:0/2702237728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5f0380 0x56077e601d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5ae51d9700 1 -- 172.21.15.46:0/2702237728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5efa60 msgr2=0x56077e601820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5ae51d9700 1 --2- 172.21.15.46:0/2702237728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5efa60 0x56077e601820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5ae51d9700 1 -- 172.21.15.46:0/2702237728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad8009300 con 0x56077e5fb220 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5ae51d9700 1 --2- 172.21.15.46:0/2702237728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 0x56077e5fff20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8000f80 tx=0x7f5ad8004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40906a038fffb856 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5abffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5abffff700 1 -- 172.21.15.46:0/2702237728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad800e070 con 0x56077e5fb220 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5abffff700 1 -- 172.21.15.46:0/2702237728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5ad8003e10 con 0x56077e5fb220 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.350+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2702237728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5acc011440 con 0x56077e5fb220 2024-03-18T20:38:09.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.351+0000 7f5abffff700 1 -- 172.21.15.46:0/2702237728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ad80046c0 con 0x56077e5fb220 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.351+0000 7f5af2dd1b80 10 client.16254 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.351+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2702237728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56077e508ea0 con 0x56077e5fb220 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.351+0000 7f5abffff700 1 -- 172.21.15.46:0/2702237728 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5acc011440 con 0x56077e5fb220 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.351+0000 7f5abffff700 1 -- 172.21.15.46:0/2702237728 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5ad8017410 con 0x56077e5fb220 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.351+0000 7f5af2dd1b80 10 client.16254 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.351+0000 7f5af2dd1b80 10 client.16254 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.351+0000 7f5af2dd1b80 20 client.16254 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.351+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2702237728 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56077e65fac0 0x56077e661ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.351+0000 7f5af2dd1b80 4 client.16254 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.351+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2702237728 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56077e391cf0 con 0x56077e65fac0 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.351+0000 7f5ae49d8700 1 --2- 172.21.15.46:0/2702237728 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56077e65fac0 0x56077e661ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5ae49d8700 1 --2- 172.21.15.46:0/2702237728 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56077e65fac0 0x56077e661ee0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76985108b980c97f server_cookie=59cf7961a1179f0d in_seq=0 out_seq=0 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5abffff700 10 client.16254 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5abffff700 1 -- 172.21.15.46:0/2702237728 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56077e391cf0 con 0x56077e65fac0 2024-03-18T20:38:09.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5abffff700 10 client.16254 handle_command_reply: tid=0 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5af2dd1b80 1 client.16254 shutdown 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2702237728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 msgr2=0x56077e5fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2702237728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 0x56077e5fff20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8000f80 tx=0x7f5ad8004300 comp rx=0 tx=0).stop 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2702237728 shutdown_connections 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2702237728 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56077e65fac0 0x56077e661ee0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2702237728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56077e5fb220 0x56077e5fff20 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2702237728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56077e5efa60 0x56077e601820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5abffff700 0 client.16254 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5af2dd1b80 1 --2- 172.21.15.46:0/2702237728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56077e5f0380 0x56077e601d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2702237728 >> 172.21.15.46:0/2702237728 conn(0x56077e566220 msgr2=0x56077e5f67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2702237728 shutdown_connections 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5af2dd1b80 1 -- 172.21.15.46:0/2702237728 wait complete. 2024-03-18T20:38:09.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.352+0000 7f5af2dd1b80 20 client.16254 trim_cache size 0 max 16384 2024-03-18T20:38:09.363 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:38:09.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.353+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3565959740 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5ad00a1290 msgr2=0x7f5ad00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.353+0000 7f5ae6c3c700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5ad00a1290 0x7f5ad00a3750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0198700 tx=0x7f5acc0073d0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.353+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3565959740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae00ffe50 msgr2=0x7f5ae0197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.353+0000 7f5ae6c3c700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae00ffe50 0x7f5ae0197200 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4004760 tx=0x7f5ad4004790 comp rx=0 tx=0).stop 2024-03-18T20:38:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.353+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3565959740 shutdown_connections 2024-03-18T20:38:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.353+0000 7f5ae6c3c700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5ad00a1290 0x7f5ad00a3750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.353+0000 7f5ae6c3c700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae00ffe50 0x7f5ae0197200 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.353+0000 7f5ae6c3c700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ae019ba90 0x7f5ae019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.353+0000 7f5ae6c3c700 1 --2- 172.21.15.46:0/3565959740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ae0104f60 0x7f5ae0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.353+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3565959740 >> 172.21.15.46:0/3565959740 conn(0x7f5ae0076d40 msgr2=0x7f5ae00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.353+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3565959740 shutdown_connections 2024-03-18T20:38:09.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.353+0000 7f5ae6c3c700 1 -- 172.21.15.46:0/3565959740 wait complete. 2024-03-18T20:38:09.369 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:38:09.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:38:09.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:09 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1085733222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:38:09.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.506+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4042920801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb810d750 msgr2=0x7f9bb810fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.506+0000 7f9bc00a9700 1 --2- 172.21.15.46:0/4042920801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb810d750 0x7f9bb810fb40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000b0a0 tx=0x7f9bb000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.506+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4042920801 shutdown_connections 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.506+0000 7f9bc00a9700 1 --2- 172.21.15.46:0/4042920801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb810d750 0x7f9bb810fb40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.506+0000 7f9bc00a9700 1 --2- 172.21.15.46:0/4042920801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bb8068ff0 0x7f9bb80693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.506+0000 7f9bc00a9700 1 --2- 172.21.15.46:0/4042920801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8069910 0x7f9bb810d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.506+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4042920801 >> 172.21.15.46:0/4042920801 conn(0x7f9bb8076d80 msgr2=0x7f9bb8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.506+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4042920801 shutdown_connections 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.506+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4042920801 wait complete. 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.506+0000 7f9bc00a9700 1 Processor -- start 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.506+0000 7f9bc00a9700 1 -- start start 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bc00a9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8068ff0 0x7f9bb8199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bc00a9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8069910 0x7f9bb8199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bc00a9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bb810d750 0x7f9bb819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bc00a9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb819e440 con 0x7f9bb8069910 2024-03-18T20:38:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bc00a9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb819e5b0 con 0x7f9bb8068ff0 2024-03-18T20:38:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bc00a9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb819e770 con 0x7f9bb810d750 2024-03-18T20:38:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bbd644700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8069910 0x7f9bb8199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bbd644700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8069910 0x7f9bb8199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46176/0 (socket says 172.21.15.46:46176) 2024-03-18T20:38:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bbd644700 1 -- 172.21.15.46:0/4112377247 learned_addr learned my addr 172.21.15.46:0/4112377247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bbde45700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8068ff0 0x7f9bb8199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bbe646700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bb810d750 0x7f9bb819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bbd644700 1 -- 172.21.15.46:0/4112377247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bb810d750 msgr2=0x7f9bb819de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bbd644700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bb810d750 0x7f9bb819de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bbd644700 1 -- 172.21.15.46:0/4112377247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8068ff0 msgr2=0x7f9bb8199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bbd644700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8068ff0 0x7f9bb8199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bbd644700 1 -- 172.21.15.46:0/4112377247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb0009d00 con 0x7f9bb8069910 2024-03-18T20:38:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.507+0000 7f9bbd644700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8069910 0x7f9bb8199a60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00d950 tx=0x7f9bac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.508+0000 7f9bb6ffd700 1 -- 172.21.15.46:0/4112377247 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac00f890 con 0x7f9bb8069910 2024-03-18T20:38:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.508+0000 7f9bb6ffd700 1 -- 172.21.15.46:0/4112377247 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bac010460 con 0x7f9bb8069910 2024-03-18T20:38:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.508+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4112377247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb819ea50 con 0x7f9bb8069910 2024-03-18T20:38:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.508+0000 7f9bb6ffd700 1 -- 172.21.15.46:0/4112377247 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac00e610 con 0x7f9bb8069910 2024-03-18T20:38:09.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.508+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4112377247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb81a6290 con 0x7f9bb8069910 2024-03-18T20:38:09.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.510+0000 7f9bb6ffd700 1 -- 172.21.15.46:0/4112377247 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9bac0108d0 con 0x7f9bb8069910 2024-03-18T20:38:09.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.511+0000 7f9bb6ffd700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9ba00a1290 0x7f9ba00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.511+0000 7f9bb6ffd700 1 -- 172.21.15.46:0/4112377247 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9bac0cca60 con 0x7f9bb8069910 2024-03-18T20:38:09.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.511+0000 7f9bbde45700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9ba00a1290 0x7f9ba00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.511+0000 7f9bbde45700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9ba00a1290 0x7f9ba00a3750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009c60 tx=0x7f9ba8009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.511+0000 7f9bcc23eb80 1 Processor -- start 2024-03-18T20:38:09.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bcc23eb80 1 -- start start 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bcc23eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bcc23eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 0x563afa5dace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bcc23eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5dd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bcc23eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afa529990 con 0x563afa5db220 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bcc23eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afa527350 con 0x563afa5d0380 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bcc23eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afa522c40 con 0x563afa5cfa60 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bbe646700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5dd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bbe646700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5dd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46192/0 (socket says 172.21.15.46:46192) 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bbe646700 1 -- 172.21.15.46:0/1502044370 learned_addr learned my addr 172.21.15.46:0/1502044370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bbde45700 1 --2- 172.21.15.46:0/1502044370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bbd644700 1 --2- 172.21.15.46:0/1502044370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 0x563afa5dace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bbe646700 1 -- 172.21.15.46:0/1502044370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 msgr2=0x563afa5cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bbe646700 1 --2- 172.21.15.46:0/1502044370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bbe646700 1 -- 172.21.15.46:0/1502044370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 msgr2=0x563afa5dace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bbe646700 1 --2- 172.21.15.46:0/1502044370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 0x563afa5dace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.512+0000 7f9bbe646700 1 -- 172.21.15.46:0/1502044370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563afa3a43b0 con 0x563afa5db220 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.513+0000 7f9bbe646700 1 --2- 172.21.15.46:0/1502044370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5dd610 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000b0a0 tx=0x7f9bb0009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aafc6d5a9916613 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.513+0000 7f9b9ffff700 1 -- 172.21.15.46:0/1502044370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb000e040 con 0x563afa5db220 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.513+0000 7f9b9ffff700 1 -- 172.21.15.46:0/1502044370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bb0007db0 con 0x563afa5db220 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.513+0000 7f9b9ffff700 1 -- 172.21.15.46:0/1502044370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0007760 con 0x563afa5db220 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.513+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1502044370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 msgr2=0x563afa5dd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.513+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1502044370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5dd610 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000b0a0 tx=0x7f9bb0009410 comp rx=0 tx=0).stop 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.514+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1502044370 shutdown_connections 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.514+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1502044370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5dd610 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.514+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1502044370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.514+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1502044370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 0x563afa5dace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.514+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1502044370 >> 172.21.15.46:0/1502044370 conn(0x563afa546220 msgr2=0x563afa5cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.514+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1502044370 shutdown_connections 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.514+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1502044370 wait complete. 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.514+0000 7f9bcc23eb80 1 -- start start 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.514+0000 7f9bcc23eb80 1 Processor -- start 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bcc23eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5cfa60 0x563afa5dac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bcc23eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5d0380 0x563afa5d7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bcc23eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5d8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bcc23eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afa4bed80 con 0x563afa5db220 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bcc23eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afa4bea20 con 0x563afa5cfa60 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bcc23eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afa3a66a0 con 0x563afa5d0380 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bcc23eb80 10 client.0 _ll_register_callbacks cb 0x563afa32a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bbe646700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5d8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bbe646700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5d8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46204/0 (socket says 172.21.15.46:46204) 2024-03-18T20:38:09.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bbe646700 1 -- 172.21.15.46:0/3889105489 learned_addr learned my addr 172.21.15.46:0/3889105489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bbde45700 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5cfa60 0x563afa5dac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bbd644700 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5d0380 0x563afa5d7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bbe646700 1 -- 172.21.15.46:0/3889105489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5d0380 msgr2=0x563afa5d7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bbe646700 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5d0380 0x563afa5d7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bbe646700 1 -- 172.21.15.46:0/3889105489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5cfa60 msgr2=0x563afa5dac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bbe646700 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5cfa60 0x563afa5dac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bbe646700 1 -- 172.21.15.46:0/3889105489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb8110670 con 0x563afa5db220 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9bbe646700 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5d8700 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009a20 tx=0x7f9bb0003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da04d0917c48e7a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9b9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.515+0000 7f9b9d7fa700 1 -- 172.21.15.46:0/3889105489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb000e040 con 0x563afa5db220 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9b9d7fa700 1 -- 172.21.15.46:0/3889105489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9bb00078d0 con 0x563afa5db220 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/3889105489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563afa5c3ad0 con 0x563afa5db220 2024-03-18T20:38:09.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9b9d7fa700 1 -- 172.21.15.46:0/3889105489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bb0003c20 con 0x563afa5db220 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9bcc23eb80 10 client.16266 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/3889105489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563afa350b60 con 0x563afa5db220 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9b9d7fa700 1 -- 172.21.15.46:0/3889105489 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bb0039080 con 0x563afa5db220 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9b9d7fa700 1 -- 172.21.15.46:0/3889105489 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9bb0019030 con 0x563afa5db220 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9bcc23eb80 10 client.16266 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9bcc23eb80 10 client.16266 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9bcc23eb80 20 client.16266 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563afa5ea950 0x563afa5ecd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9bcc23eb80 4 client.16266 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/3889105489 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563afa51b700 con 0x563afa5ea950 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.516+0000 7f9bbde45700 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563afa5ea950 0x563afa5ecd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.517+0000 7f9bbde45700 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563afa5ea950 0x563afa5ecd70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63d5268d6e58fa67 server_cookie=fd3198e9119985d9 in_seq=0 out_seq=0 2024-03-18T20:38:09.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.517+0000 7f9b9d7fa700 10 client.16266 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.517+0000 7f9b9d7fa700 1 -- 172.21.15.46:0/3889105489 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563afa51b700 con 0x563afa5ea950 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.517+0000 7f9b9d7fa700 10 client.16266 handle_command_reply: tid=0 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.517+0000 7f9bcc23eb80 1 client.16266 shutdown 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.517+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/3889105489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 msgr2=0x563afa5d8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.517+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5d8700 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009a20 tx=0x7f9bb0003a70 comp rx=0 tx=0).stop 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.518+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/3889105489 shutdown_connections 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.518+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563afa5ea950 0x563afa5ecd70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.518+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5d8700 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.518+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5cfa60 0x563afa5dac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.518+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/3889105489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5d0380 0x563afa5d7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.518+0000 7f9b9d7fa700 0 client.16266 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.518+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/3889105489 >> 172.21.15.46:0/3889105489 conn(0x563afa546220 msgr2=0x563afa5cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.518+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/3889105489 shutdown_connections 2024-03-18T20:38:09.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.518+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/3889105489 wait complete. 2024-03-18T20:38:09.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.518+0000 7f9bcc23eb80 20 client.16266 trim_cache size 0 max 16384 2024-03-18T20:38:09.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.531+0000 7f9bcc23eb80 1 Processor -- start 2024-03-18T20:38:09.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.531+0000 7f9bcc23eb80 1 -- start start 2024-03-18T20:38:09.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bcc23eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5e1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bcc23eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 0x563afa5e6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bcc23eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bcc23eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afa5c6210 con 0x563afa5db220 2024-03-18T20:38:09.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bcc23eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afa5e7930 con 0x563afa5d0380 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bcc23eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afa5d1580 con 0x563afa5cfa60 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bbe646700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bbe646700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5d3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46214/0 (socket says 172.21.15.46:46214) 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bbe646700 1 -- 172.21.15.46:0/1716915940 learned_addr learned my addr 172.21.15.46:0/1716915940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bbd644700 1 --2- 172.21.15.46:0/1716915940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 0x563afa5e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bbde45700 1 --2- 172.21.15.46:0/1716915940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5e1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bbe646700 1 -- 172.21.15.46:0/1716915940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 msgr2=0x563afa5e1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bbe646700 1 --2- 172.21.15.46:0/1716915940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5e1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bbe646700 1 -- 172.21.15.46:0/1716915940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 msgr2=0x563afa5e6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bbe646700 1 --2- 172.21.15.46:0/1716915940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 0x563afa5e6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bbe646700 1 -- 172.21.15.46:0/1716915940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563afa4e8ea0 con 0x563afa5db220 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bbe646700 1 --2- 172.21.15.46:0/1716915940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5d3320 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0003ae0 tx=0x7f9bb0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3aac5eb72a811c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.532+0000 7f9bb4ff9700 1 -- 172.21.15.46:0/1716915940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb000e040 con 0x563afa5db220 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.533+0000 7f9bb4ff9700 1 -- 172.21.15.46:0/1716915940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563afa5c3ad0 con 0x563afa5db220 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.533+0000 7f9bb4ff9700 1 -- 172.21.15.46:0/1716915940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0003bd0 con 0x563afa5db220 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.533+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1716915940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 msgr2=0x563afa5d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.533+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1716915940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5d3320 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0003ae0 tx=0x7f9bb0009780 comp rx=0 tx=0).stop 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.533+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1716915940 shutdown_connections 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.533+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1716915940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa5d3320 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.533+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1716915940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5e1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.533+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1716915940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 0x563afa5e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.533+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1716915940 >> 172.21.15.46:0/1716915940 conn(0x563afa546220 msgr2=0x563afa5c7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.533+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1716915940 shutdown_connections 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.533+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1716915940 wait complete. 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bcc23eb80 1 -- start start 2024-03-18T20:38:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bcc23eb80 1 Processor -- start 2024-03-18T20:38:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bcc23eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5dffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bcc23eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 0x563afa5e0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bcc23eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa63d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bcc23eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afa63dc80 con 0x563afa5db220 2024-03-18T20:38:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bcc23eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afa63ddc0 con 0x563afa5d0380 2024-03-18T20:38:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bcc23eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afa63df30 con 0x563afa5cfa60 2024-03-18T20:38:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bcc23eb80 10 client.0 _ll_register_callbacks cb 0x7f9bb0009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bbe646700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa63d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bbde45700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5dffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bbe646700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa63d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46224/0 (socket says 172.21.15.46:46224) 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bbe646700 1 -- 172.21.15.46:0/1629236514 learned_addr learned my addr 172.21.15.46:0/1629236514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.534+0000 7f9bbd644700 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 0x563afa5e0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9bbde45700 1 -- 172.21.15.46:0/1629236514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 msgr2=0x563afa5e0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9bbde45700 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 0x563afa5e0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9bbde45700 1 -- 172.21.15.46:0/1629236514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 msgr2=0x563afa63d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9bbde45700 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa63d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9bbde45700 1 -- 172.21.15.46:0/1629236514 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563afa350b60 con 0x563afa5cfa60 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9bbe646700 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa63d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9bbde45700 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5dffc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x563afa5ead30 tx=0x7f9ba80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30954c2af122ba90 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9b9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9b9d7fa700 1 -- 172.21.15.46:0/1629236514 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8023040 con 0x563afa5cfa60 2024-03-18T20:38:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9b9d7fa700 1 -- 172.21.15.46:0/1629236514 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9ba8028900 con 0x563afa5cfa60 2024-03-18T20:38:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1629236514 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bb0007db0 con 0x563afa5cfa60 2024-03-18T20:38:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9b9d7fa700 1 -- 172.21.15.46:0/1629236514 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ba801e310 con 0x563afa5cfa60 2024-03-18T20:38:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9bcc23eb80 10 client.26035 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1629236514 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563afa3a8cd0 con 0x563afa5cfa60 2024-03-18T20:38:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.535+0000 7f9b9d7fa700 1 -- 172.21.15.46:0/1629236514 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bb0007db0 con 0x563afa5cfa60 2024-03-18T20:38:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9b9d7fa700 1 -- 172.21.15.46:0/1629236514 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9ba801e5a0 con 0x563afa5cfa60 2024-03-18T20:38:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9bcc23eb80 10 client.26035 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9bcc23eb80 10 client.26035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:38:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9bcc23eb80 20 client.26035 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563afa63f6e0 0x563afa641b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9bcc23eb80 4 client.26035 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:38:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1629236514 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563afa371cf0 con 0x563afa63f6e0 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9bbd644700 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563afa63f6e0 0x563afa641b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9bbd644700 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563afa63f6e0 0x563afa641b00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7fca01f74f68035e server_cookie=23f4bc1b2100d8fc in_seq=0 out_seq=0 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9b9d7fa700 10 client.26035 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9b9d7fa700 1 -- 172.21.15.46:0/1629236514 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563afa371cf0 con 0x563afa63f6e0 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9b9d7fa700 10 client.26035 handle_command_reply: tid=0 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9bcc23eb80 1 client.26035 shutdown 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1629236514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 msgr2=0x563afa5dffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.536+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5dffc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x563afa5ead30 tx=0x7f9ba80047c0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1629236514 shutdown_connections 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563afa63f6e0 0x563afa641b00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563afa5db220 0x563afa63d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563afa5cfa60 0x563afa5dffc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bcc23eb80 1 --2- 172.21.15.46:0/1629236514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563afa5d0380 0x563afa5e0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1629236514 >> 172.21.15.46:0/1629236514 conn(0x563afa546220 msgr2=0x563afa5e6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9b9d7fa700 0 client.26035 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:38:09.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1629236514 shutdown_connections 2024-03-18T20:38:09.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bcc23eb80 1 -- 172.21.15.46:0/1629236514 wait complete. 2024-03-18T20:38:09.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bcc23eb80 20 client.26035 trim_cache size 0 max 16384 2024-03-18T20:38:09.547 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:38:09.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4112377247 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9ba00a1290 msgr2=0x7f9ba00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bc00a9700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9ba00a1290 0x7f9ba00a3750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009c60 tx=0x7f9ba8009260 comp rx=0 tx=0).stop 2024-03-18T20:38:09.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4112377247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8069910 msgr2=0x7f9bb8199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bc00a9700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8069910 0x7f9bb8199a60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00d950 tx=0x7f9bac00dc60 comp rx=0 tx=0).stop 2024-03-18T20:38:09.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4112377247 shutdown_connections 2024-03-18T20:38:09.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bc00a9700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9ba00a1290 0x7f9ba00a3750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bc00a9700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bb810d750 0x7f9bb819de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bc00a9700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bb8068ff0 0x7f9bb8199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bc00a9700 1 --2- 172.21.15.46:0/4112377247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bb8069910 0x7f9bb8199a60 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.537+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4112377247 >> 172.21.15.46:0/4112377247 conn(0x7f9bb8076d80 msgr2=0x7f9bb810ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.538+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4112377247 shutdown_connections 2024-03-18T20:38:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.538+0000 7f9bc00a9700 1 -- 172.21.15.46:0/4112377247 wait complete. 2024-03-18T20:38:09.549 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:09 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1085733222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:38:09.554 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:38:09.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:38:09.673 INFO:tasks.workunit.client.0.smithi046.stdout: 1 53903 4.81 MB/sec warmup 64 sec latency 927.460 ms 2024-03-18T20:38:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.690+0000 7fd09a6c6700 1 -- 172.21.15.46:0/3929285676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd094104f60 msgr2=0x7fd09410cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.690+0000 7fd09a6c6700 1 --2- 172.21.15.46:0/3929285676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd094104f60 0x7fd09410cae0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00b0a0 tx=0x7fd08c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.690+0000 7fd09a6c6700 1 -- 172.21.15.46:0/3929285676 shutdown_connections 2024-03-18T20:38:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.690+0000 7fd09a6c6700 1 --2- 172.21.15.46:0/3929285676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd094104f60 0x7fd09410cae0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.690+0000 7fd09a6c6700 1 --2- 172.21.15.46:0/3929285676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0940ffe50 0x7fd094100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.690+0000 7fd09a6c6700 1 --2- 172.21.15.46:0/3929285676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd094100800 0x7fd094104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.690+0000 7fd09a6c6700 1 -- 172.21.15.46:0/3929285676 >> 172.21.15.46:0/3929285676 conn(0x7fd094076d40 msgr2=0x7fd094077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.690+0000 7fd09a6c6700 1 -- 172.21.15.46:0/3929285676 shutdown_connections 2024-03-18T20:38:09.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.690+0000 7fd09a6c6700 1 -- 172.21.15.46:0/3929285676 wait complete. 2024-03-18T20:38:09.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.691+0000 7fd09a6c6700 1 Processor -- start 2024-03-18T20:38:09.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.691+0000 7fd09a6c6700 1 -- start start 2024-03-18T20:38:09.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.691+0000 7fd09a6c6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0940ffe50 0x7fd0941972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.691+0000 7fd09a6c6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd094104f60 0x7fd094197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.691+0000 7fd09a6c6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09419bba0 0x7fd09419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.691+0000 7fd09a6c6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09419c620 con 0x7fd0940ffe50 2024-03-18T20:38:09.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.691+0000 7fd09a6c6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09419c790 con 0x7fd09419bba0 2024-03-18T20:38:09.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.691+0000 7fd09a6c6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09419c950 con 0x7fd094104f60 2024-03-18T20:38:09.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd093fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0940ffe50 0x7fd0941972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd093fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0940ffe50 0x7fd0941972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46244/0 (socket says 172.21.15.46:46244) 2024-03-18T20:38:09.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd094104f60 0x7fd094197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd093fff700 1 -- 172.21.15.46:0/4125470026 learned_addr learned my addr 172.21.15.46:0/4125470026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd098c63700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09419bba0 0x7fd09419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd0937fe700 1 -- 172.21.15.46:0/4125470026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09419bba0 msgr2=0x7fd09419c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd0937fe700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09419bba0 0x7fd09419c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd0937fe700 1 -- 172.21.15.46:0/4125470026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0940ffe50 msgr2=0x7fd0941972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd0937fe700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0940ffe50 0x7fd0941972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd0937fe700 1 -- 172.21.15.46:0/4125470026 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c009520 con 0x7fd094104f60 2024-03-18T20:38:09.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd093fff700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0940ffe50 0x7fd0941972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:09.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd0937fe700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd094104f60 0x7fd094197800 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00e9c0 tx=0x7fd07c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.692+0000 7fd0917fa700 1 -- 172.21.15.46:0/4125470026 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00cb70 con 0x7fd094104f60 2024-03-18T20:38:09.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.693+0000 7fd09a6c6700 1 -- 172.21.15.46:0/4125470026 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08c009d00 con 0x7fd094104f60 2024-03-18T20:38:09.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.693+0000 7fd0917fa700 1 -- 172.21.15.46:0/4125470026 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd07c0044e0 con 0x7fd094104f60 2024-03-18T20:38:09.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.693+0000 7fd0917fa700 1 -- 172.21.15.46:0/4125470026 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c003d20 con 0x7fd094104f60 2024-03-18T20:38:09.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.693+0000 7fd09a6c6700 1 -- 172.21.15.46:0/4125470026 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0941a41f0 con 0x7fd094104f60 2024-03-18T20:38:09.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.695+0000 7fd0917fa700 1 -- 172.21.15.46:0/4125470026 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd07c003ec0 con 0x7fd094104f60 2024-03-18T20:38:09.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.696+0000 7fd0917fa700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0800a1290 0x7fd0800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.696+0000 7fd0917fa700 1 -- 172.21.15.46:0/4125470026 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd07c014070 con 0x7fd094104f60 2024-03-18T20:38:09.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.696+0000 7fd093fff700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0800a1290 0x7fd0800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.696+0000 7fd0a685bb80 1 Processor -- start 2024-03-18T20:38:09.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.696+0000 7fd093fff700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0800a1290 0x7fd0800a3750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd084011550 tx=0x7fd084009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.696+0000 7fd0a685bb80 1 -- start start 2024-03-18T20:38:09.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0a685bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f12fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0a685bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 0x56414f13ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0a685bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f13b220 0x56414f13d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0a685bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414f089990 con 0x56414f130380 2024-03-18T20:38:09.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0a685bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414f087350 con 0x56414f12fa60 2024-03-18T20:38:09.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0a685bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414f082c40 con 0x56414f13b220 2024-03-18T20:38:09.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 0x56414f13ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 0x56414f13ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46246/0 (socket says 172.21.15.46:46246) 2024-03-18T20:38:09.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0937fe700 1 -- 172.21.15.46:0/2824247550 learned_addr learned my addr 172.21.15.46:0/2824247550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd098c63700 1 --2- 172.21.15.46:0/2824247550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f13b220 0x56414f13d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd093fff700 1 --2- 172.21.15.46:0/2824247550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f12fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0937fe700 1 -- 172.21.15.46:0/2824247550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f13b220 msgr2=0x56414f13d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0937fe700 1 --2- 172.21.15.46:0/2824247550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f13b220 0x56414f13d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0937fe700 1 -- 172.21.15.46:0/2824247550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 msgr2=0x56414f12fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0937fe700 1 --2- 172.21.15.46:0/2824247550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f12fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0937fe700 1 -- 172.21.15.46:0/2824247550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414ef043b0 con 0x56414f130380 2024-03-18T20:38:09.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd0937fe700 1 --2- 172.21.15.46:0/2824247550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 0x56414f13ace0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0a20b0 tx=0x7fd07c099d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74af16895326a02e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.697+0000 7fd08a7fc700 1 -- 172.21.15.46:0/2824247550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0b1070 con 0x56414f130380 2024-03-18T20:38:09.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.698+0000 7fd08a7fc700 1 -- 172.21.15.46:0/2824247550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd07c09a160 con 0x56414f130380 2024-03-18T20:38:09.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.698+0000 7fd08a7fc700 1 -- 172.21.15.46:0/2824247550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c098620 con 0x56414f130380 2024-03-18T20:38:09.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.698+0000 7fd0a685bb80 1 -- 172.21.15.46:0/2824247550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 msgr2=0x56414f13ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.698+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/2824247550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 0x56414f13ace0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0a20b0 tx=0x7fd07c099d70 comp rx=0 tx=0).stop 2024-03-18T20:38:09.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.698+0000 7fd0a685bb80 1 -- 172.21.15.46:0/2824247550 shutdown_connections 2024-03-18T20:38:09.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.698+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/2824247550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f13b220 0x56414f13d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.698+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/2824247550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f12fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.698+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/2824247550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 0x56414f13ace0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.698+0000 7fd0a685bb80 1 -- 172.21.15.46:0/2824247550 >> 172.21.15.46:0/2824247550 conn(0x56414f0a6220 msgr2=0x56414f12c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.698+0000 7fd0a685bb80 1 -- 172.21.15.46:0/2824247550 shutdown_connections 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.698+0000 7fd0a685bb80 1 -- 172.21.15.46:0/2824247550 wait complete. 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd0a685bb80 1 -- start start 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd0a685bb80 1 Processor -- start 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd0a685bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f138010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd0a685bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f130380 0x56414f138550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd0a685bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 0x56414f138a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd0a685bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414f01ed80 con 0x56414f13b220 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd0a685bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414f01ea20 con 0x56414f12fa60 2024-03-18T20:38:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd0a685bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414ef066a0 con 0x56414f130380 2024-03-18T20:38:09.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd0a685bb80 10 client.0 _ll_register_callbacks cb 0x56414ee8a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:09.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd098c63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 0x56414f138a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd098c63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 0x56414f138a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46256/0 (socket says 172.21.15.46:46256) 2024-03-18T20:38:09.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd098c63700 1 -- 172.21.15.46:0/3622903623 learned_addr learned my addr 172.21.15.46:0/3622903623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.699+0000 7fd093fff700 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f138010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd0937fe700 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f130380 0x56414f138550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd098c63700 1 -- 172.21.15.46:0/3622903623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f130380 msgr2=0x56414f138550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd098c63700 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f130380 0x56414f138550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd098c63700 1 -- 172.21.15.46:0/3622903623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 msgr2=0x56414f138010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd098c63700 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f138010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd098c63700 1 -- 172.21.15.46:0/3622903623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0941a41f0 con 0x56414f13b220 2024-03-18T20:38:09.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd0937fe700 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f130380 0x56414f138550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd098c63700 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 0x56414f138a90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd08c007f10 tx=0x7fd08c007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926037918c41e63f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd0737fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd0737fe700 1 -- 172.21.15.46:0/3622903623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00e040 con 0x56414f13b220 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd0a685bb80 1 -- 172.21.15.46:0/3622903623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56414f123ad0 con 0x56414f13b220 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd0737fe700 1 -- 172.21.15.46:0/3622903623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd08c022070 con 0x56414f13b220 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd0737fe700 1 -- 172.21.15.46:0/3622903623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd08c007910 con 0x56414f13b220 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd0a685bb80 10 client.16284 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.700+0000 7fd0a685bb80 1 -- 172.21.15.46:0/3622903623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56414eeb0b60 con 0x56414f13b220 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.701+0000 7fd0737fe700 1 -- 172.21.15.46:0/3622903623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56414f123ad0 con 0x56414f13b220 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.701+0000 7fd0737fe700 1 -- 172.21.15.46:0/3622903623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd08c02d070 con 0x56414f13b220 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.701+0000 7fd0a685bb80 10 client.16284 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.701+0000 7fd0a685bb80 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.701+0000 7fd0a685bb80 20 client.16284 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.701+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56414f14af60 0x56414f14d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.701+0000 7fd0a685bb80 4 client.16284 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.701+0000 7fd0a685bb80 1 -- 172.21.15.46:0/3622903623 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56414f07b700 con 0x56414f14af60 2024-03-18T20:38:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.701+0000 7fd093fff700 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56414f14af60 0x56414f14d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.702+0000 7fd0737fe700 10 client.16284 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.702+0000 7fd093fff700 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56414f14af60 0x56414f14d380 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ef0ad6c7a6eb71 server_cookie=815058937e8f6553 in_seq=0 out_seq=0 2024-03-18T20:38:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.702+0000 7fd0737fe700 1 -- 172.21.15.46:0/3622903623 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56414f07b700 con 0x56414f14af60 2024-03-18T20:38:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.702+0000 7fd0737fe700 10 client.16284 handle_command_reply: tid=0 2024-03-18T20:38:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.702+0000 7fd0a685bb80 1 client.16284 shutdown 2024-03-18T20:38:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.703+0000 7fd0a685bb80 1 -- 172.21.15.46:0/3622903623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 msgr2=0x56414f138a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.703+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 0x56414f138a90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd08c007f10 tx=0x7fd08c007f40 comp rx=0 tx=0).stop 2024-03-18T20:38:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.703+0000 7fd0a685bb80 1 -- 172.21.15.46:0/3622903623 shutdown_connections 2024-03-18T20:38:09.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.703+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56414f14af60 0x56414f14d380 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.703+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 0x56414f138a90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.703+0000 7fd0737fe700 0 client.16284 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:09.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.703+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f138010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.703+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/3622903623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f130380 0x56414f138550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.703+0000 7fd0a685bb80 1 -- 172.21.15.46:0/3622903623 >> 172.21.15.46:0/3622903623 conn(0x56414f0a6220 msgr2=0x56414f12bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.703+0000 7fd0a685bb80 1 -- 172.21.15.46:0/3622903623 shutdown_connections 2024-03-18T20:38:09.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.703+0000 7fd0a685bb80 1 -- 172.21.15.46:0/3622903623 wait complete. 2024-03-18T20:38:09.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.703+0000 7fd0a685bb80 20 client.16284 trim_cache size 0 max 16384 2024-03-18T20:38:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd0a685bb80 1 Processor -- start 2024-03-18T20:38:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd0a685bb80 1 -- start start 2024-03-18T20:38:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd0a685bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f141df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd0a685bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f130380 0x56414f12a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd0a685bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 0x56414f146540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd0a685bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414f12d440 con 0x56414f13b220 2024-03-18T20:38:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd0a685bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414f139680 con 0x56414f12fa60 2024-03-18T20:38:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd0a685bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414f132e90 con 0x56414f130380 2024-03-18T20:38:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd098c63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 0x56414f146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd098c63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 0x56414f146540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46258/0 (socket says 172.21.15.46:46258) 2024-03-18T20:38:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd098c63700 1 -- 172.21.15.46:0/2156907585 learned_addr learned my addr 172.21.15.46:0/2156907585 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd0937fe700 1 --2- 172.21.15.46:0/2156907585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f130380 0x56414f12a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.717+0000 7fd093fff700 1 --2- 172.21.15.46:0/2156907585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.718+0000 7fd098c63700 1 -- 172.21.15.46:0/2156907585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f130380 msgr2=0x56414f12a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.718+0000 7fd098c63700 1 --2- 172.21.15.46:0/2156907585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f130380 0x56414f12a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.718+0000 7fd098c63700 1 -- 172.21.15.46:0/2156907585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 msgr2=0x56414f141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.718+0000 7fd098c63700 1 --2- 172.21.15.46:0/2156907585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.718+0000 7fd098c63700 1 -- 172.21.15.46:0/2156907585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414f048ea0 con 0x56414f13b220 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.718+0000 7fd093fff700 1 --2- 172.21.15.46:0/2156907585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.718+0000 7fd098c63700 1 --2- 172.21.15.46:0/2156907585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 0x56414f146540 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd08c007fd0 tx=0x7fd08c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840b313fe5720dd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.718+0000 7fd08affd700 1 -- 172.21.15.46:0/2156907585 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00e020 con 0x56414f13b220 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.718+0000 7fd08affd700 1 -- 172.21.15.46:0/2156907585 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd08c020070 con 0x56414f13b220 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.718+0000 7fd08affd700 1 -- 172.21.15.46:0/2156907585 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c004d40 con 0x56414f13b220 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.718+0000 7fd0a685bb80 1 -- 172.21.15.46:0/2156907585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 msgr2=0x56414f146540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.718+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/2156907585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 0x56414f146540 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd08c007fd0 tx=0x7fd08c009b80 comp rx=0 tx=0).stop 2024-03-18T20:38:09.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 -- 172.21.15.46:0/2156907585 shutdown_connections 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/2156907585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f13b220 0x56414f146540 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/2156907585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f12fa60 0x56414f141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/2156907585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f130380 0x56414f12a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 -- 172.21.15.46:0/2156907585 >> 172.21.15.46:0/2156907585 conn(0x56414f0a6220 msgr2=0x56414f138910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 -- 172.21.15.46:0/2156907585 shutdown_connections 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 -- 172.21.15.46:0/2156907585 wait complete. 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 -- start start 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 Processor -- start 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f12fa60 0x56414f192d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 0x56414f193260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f13b220 0x56414f1937a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414f19de80 con 0x56414f130380 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414f19dff0 con 0x56414f13b220 2024-03-18T20:38:09.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.719+0000 7fd0a685bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414f19e1b0 con 0x56414f12fa60 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0a685bb80 10 client.0 _ll_register_callbacks cb 0x7fd08c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 0x56414f193260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd093fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f12fa60 0x56414f192d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 0x56414f193260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46266/0 (socket says 172.21.15.46:46266) 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0937fe700 1 -- 172.21.15.46:0/328098556 learned_addr learned my addr 172.21.15.46:0/328098556 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd098c63700 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f13b220 0x56414f1937a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0937fe700 1 -- 172.21.15.46:0/328098556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f12fa60 msgr2=0x56414f192d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0937fe700 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f12fa60 0x56414f192d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0937fe700 1 -- 172.21.15.46:0/328098556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f13b220 msgr2=0x56414f1937a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0937fe700 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f13b220 0x56414f1937a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0937fe700 1 -- 172.21.15.46:0/328098556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56414eeb0b60 con 0x56414f130380 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd093fff700 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f12fa60 0x56414f192d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:09.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0937fe700 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 0x56414f193260 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd07c096660 tx=0x7fd07c0a2660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1ded3d5b10d1a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0737fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0737fe700 1 -- 172.21.15.46:0/328098556 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0b1030 con 0x56414f130380 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0737fe700 1 -- 172.21.15.46:0/328098556 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd07c0b2070 con 0x56414f130380 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0a685bb80 1 -- 172.21.15.46:0/328098556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd07c09a160 con 0x56414f130380 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0737fe700 1 -- 172.21.15.46:0/328098556 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd07c09e010 con 0x56414f130380 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0a685bb80 10 client.16290 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.720+0000 7fd0a685bb80 1 -- 172.21.15.46:0/328098556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56414ef08cd0 con 0x56414f130380 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.721+0000 7fd0737fe700 1 -- 172.21.15.46:0/328098556 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd07c082020 con 0x56414f130380 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.721+0000 7fd0737fe700 1 -- 172.21.15.46:0/328098556 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd07c09a3c0 con 0x56414f130380 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.721+0000 7fd0a685bb80 10 client.16290 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.721+0000 7fd0a685bb80 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.721+0000 7fd0a685bb80 20 client.16290 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.721+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56414f19f790 0x56414f1a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.721+0000 7fd0a685bb80 4 client.16290 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.721+0000 7fd0a685bb80 1 -- 172.21.15.46:0/328098556 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56414eed1cf0 con 0x56414f19f790 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.721+0000 7fd093fff700 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56414f19f790 0x56414f1a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0737fe700 10 client.16290 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd093fff700 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56414f19f790 0x56414f1a1bb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f3ff6458384cd02 server_cookie=6fecd6e18c6d3cbc in_seq=0 out_seq=0 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0737fe700 1 -- 172.21.15.46:0/328098556 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56414eed1cf0 con 0x56414f19f790 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0737fe700 10 client.16290 handle_command_reply: tid=0 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0a685bb80 1 client.16290 shutdown 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0a685bb80 1 -- 172.21.15.46:0/328098556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 msgr2=0x56414f193260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 0x56414f193260 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd07c096660 tx=0x7fd07c0a2660 comp rx=0 tx=0).stop 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0a685bb80 1 -- 172.21.15.46:0/328098556 shutdown_connections 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56414f19f790 0x56414f1a1bb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56414f13b220 0x56414f1937a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56414f12fa60 0x56414f192d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0737fe700 0 client.16290 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0a685bb80 1 --2- 172.21.15.46:0/328098556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56414f130380 0x56414f193260 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0a685bb80 1 -- 172.21.15.46:0/328098556 >> 172.21.15.46:0/328098556 conn(0x56414f0a6220 msgr2=0x56414f1319b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0a685bb80 1 -- 172.21.15.46:0/328098556 shutdown_connections 2024-03-18T20:38:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0a685bb80 1 -- 172.21.15.46:0/328098556 wait complete. 2024-03-18T20:38:09.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.722+0000 7fd0a685bb80 20 client.16290 trim_cache size 0 max 16384 2024-03-18T20:38:09.733 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:38:09.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.723+0000 7fd09a6c6700 1 -- 172.21.15.46:0/4125470026 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0800a1290 msgr2=0x7fd0800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.723+0000 7fd09a6c6700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0800a1290 0x7fd0800a3750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd084011550 tx=0x7fd084009260 comp rx=0 tx=0).stop 2024-03-18T20:38:09.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.723+0000 7fd09a6c6700 1 -- 172.21.15.46:0/4125470026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd094104f60 msgr2=0x7fd094197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:09.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.723+0000 7fd09a6c6700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd094104f60 0x7fd094197800 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00e9c0 tx=0x7fd07c00ecd0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.723+0000 7fd09a6c6700 1 -- 172.21.15.46:0/4125470026 shutdown_connections 2024-03-18T20:38:09.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.723+0000 7fd09a6c6700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0800a1290 0x7fd0800a3750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.723+0000 7fd09a6c6700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0940ffe50 0x7fd0941972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.723+0000 7fd09a6c6700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09419bba0 0x7fd09419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.723+0000 7fd09a6c6700 1 --2- 172.21.15.46:0/4125470026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd094104f60 0x7fd094197800 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:09.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.723+0000 7fd09a6c6700 1 -- 172.21.15.46:0/4125470026 >> 172.21.15.46:0/4125470026 conn(0x7fd094076d40 msgr2=0x7fd094068170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:09.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.723+0000 7fd09a6c6700 1 -- 172.21.15.46:0/4125470026 shutdown_connections 2024-03-18T20:38:09.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:09.723+0000 7fd09a6c6700 1 -- 172.21.15.46:0/4125470026 wait complete. 2024-03-18T20:38:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:10 smithi073 ceph-mon[47384]: pgmap v65: 97 pgs: 97 active+clean; 31 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 638 KiB/s wr, 137 op/s 2024-03-18T20:38:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:10 smithi155 ceph-mon[48344]: pgmap v65: 97 pgs: 97 active+clean; 31 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 638 KiB/s wr, 137 op/s 2024-03-18T20:38:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:10 smithi046 ceph-mon[43495]: pgmap v65: 97 pgs: 97 active+clean; 31 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 638 KiB/s wr, 137 op/s 2024-03-18T20:38:10.673 INFO:tasks.workunit.client.0.smithi046.stdout: 1 53903 4.74 MB/sec warmup 65 sec latency 1927.496 ms 2024-03-18T20:38:10.740 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:38:10.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:38:10.741 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:38:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.888+0000 7fe52e6fe700 1 -- 172.21.15.46:0/3619949468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe528104f60 msgr2=0x7fe52810cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.888+0000 7fe52e6fe700 1 --2- 172.21.15.46:0/3619949468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe528104f60 0x7fe52810cb30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe52000b0a0 tx=0x7fe52000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.888+0000 7fe52e6fe700 1 -- 172.21.15.46:0/3619949468 shutdown_connections 2024-03-18T20:38:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.888+0000 7fe52e6fe700 1 --2- 172.21.15.46:0/3619949468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe528104f60 0x7fe52810cb30 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.888+0000 7fe52e6fe700 1 --2- 172.21.15.46:0/3619949468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5281005c0 0x7fe5281009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.888+0000 7fe52e6fe700 1 --2- 172.21.15.46:0/3619949468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe528100f70 0x7fe528104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.888+0000 7fe52e6fe700 1 -- 172.21.15.46:0/3619949468 >> 172.21.15.46:0/3619949468 conn(0x7fe5280761e0 msgr2=0x7fe5280fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.889+0000 7fe52e6fe700 1 -- 172.21.15.46:0/3619949468 shutdown_connections 2024-03-18T20:38:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.889+0000 7fe52e6fe700 1 -- 172.21.15.46:0/3619949468 wait complete. 2024-03-18T20:38:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.889+0000 7fe52e6fe700 1 Processor -- start 2024-03-18T20:38:10.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.889+0000 7fe52e6fe700 1 -- start start 2024-03-18T20:38:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52e6fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5281005c0 0x7fe5280fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52e6fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe528104f60 0x7fe5280fbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52e6fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5280fc070 0x7fe5281a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52e6fe700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281a5b10 con 0x7fe5280fc070 2024-03-18T20:38:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52e6fe700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281a5c80 con 0x7fe528104f60 2024-03-18T20:38:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52e6fe700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281a5e40 con 0x7fe5281005c0 2024-03-18T20:38:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52cc9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5280fc070 0x7fe5281a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52cc9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5280fc070 0x7fe5281a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46288/0 (socket says 172.21.15.46:46288) 2024-03-18T20:38:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe527fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5281005c0 0x7fe5280fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe5277fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe528104f60 0x7fe5280fbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52cc9b700 1 -- 172.21.15.46:0/162044478 learned_addr learned my addr 172.21.15.46:0/162044478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52cc9b700 1 -- 172.21.15.46:0/162044478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5281005c0 msgr2=0x7fe5280fb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52cc9b700 1 --2- 172.21.15.46:0/162044478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5281005c0 0x7fe5280fb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52cc9b700 1 -- 172.21.15.46:0/162044478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe528104f60 msgr2=0x7fe5280fbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52cc9b700 1 --2- 172.21.15.46:0/162044478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe528104f60 0x7fe5280fbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52cc9b700 1 -- 172.21.15.46:0/162044478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe520009d00 con 0x7fe5280fc070 2024-03-18T20:38:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.890+0000 7fe52cc9b700 1 --2- 172.21.15.46:0/162044478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5280fc070 0x7fe5281a54d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe52000b070 tx=0x7fe520007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.891+0000 7fe5257fa700 1 -- 172.21.15.46:0/162044478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52000e040 con 0x7fe5280fc070 2024-03-18T20:38:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.891+0000 7fe5257fa700 1 -- 172.21.15.46:0/162044478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe52001d470 con 0x7fe5280fc070 2024-03-18T20:38:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.891+0000 7fe5257fa700 1 -- 172.21.15.46:0/162044478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5200052a0 con 0x7fe5280fc070 2024-03-18T20:38:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.891+0000 7fe52e6fe700 1 -- 172.21.15.46:0/162044478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5281a60c0 con 0x7fe5280fc070 2024-03-18T20:38:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.891+0000 7fe52e6fe700 1 -- 172.21.15.46:0/162044478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5281a64f0 con 0x7fe5280fc070 2024-03-18T20:38:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.893+0000 7fe5257fa700 1 -- 172.21.15.46:0/162044478 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe52001d870 con 0x7fe5280fc070 2024-03-18T20:38:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.894+0000 7fe5257fa700 1 --2- 172.21.15.46:0/162044478 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe5140a1290 0x7fe5140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.894+0000 7fe5257fa700 1 -- 172.21.15.46:0/162044478 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fe5200ce010 con 0x7fe5280fc070 2024-03-18T20:38:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.894+0000 7fe527fff700 1 --2- 172.21.15.46:0/162044478 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe5140a1290 0x7fe5140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.894+0000 7fe527fff700 1 --2- 172.21.15.46:0/162044478 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe5140a1290 0x7fe5140a3750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe518009c60 tx=0x7fe518009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.894+0000 7fe53a893b80 1 Processor -- start 2024-03-18T20:38:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe53a893b80 1 -- start start 2024-03-18T20:38:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe53a893b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeafae0 0x5604bbeafec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe53a893b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeb0400 0x5604bbebad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe53a893b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbebb2a0 0x5604bbebd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe53a893b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bbe09a10 con 0x5604bbeb0400 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe53a893b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bbe073d0 con 0x5604bbeafae0 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe53a893b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bbe02cc0 con 0x5604bbebb2a0 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe5277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeb0400 0x5604bbebad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe527fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeafae0 0x5604bbeafec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe5277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeb0400 0x5604bbebad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46298/0 (socket says 172.21.15.46:46298) 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe5277fe700 1 -- 172.21.15.46:0/2500840840 learned_addr learned my addr 172.21.15.46:0/2500840840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe527fff700 1 --2- 172.21.15.46:0/2500840840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeafae0 0x5604bbeafec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:34246/0 (socket says 172.21.15.46:34246) 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe52cc9b700 1 --2- 172.21.15.46:0/2500840840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbebb2a0 0x5604bbebd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe5277fe700 1 -- 172.21.15.46:0/2500840840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbebb2a0 msgr2=0x5604bbebd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe5277fe700 1 --2- 172.21.15.46:0/2500840840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbebb2a0 0x5604bbebd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe5277fe700 1 -- 172.21.15.46:0/2500840840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeafae0 msgr2=0x5604bbeafec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe5277fe700 1 --2- 172.21.15.46:0/2500840840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeafae0 0x5604bbeafec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe5277fe700 1 -- 172.21.15.46:0/2500840840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604bbc84430 con 0x5604bbeb0400 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.895+0000 7fe527fff700 1 --2- 172.21.15.46:0/2500840840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeafae0 0x5604bbeafec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.896+0000 7fe5277fe700 1 --2- 172.21.15.46:0/2500840840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeb0400 0x5604bbebad60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe510007b90 tx=0x7fe510007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e4f7683ecdcc12 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.896+0000 7fe51e7fc700 1 -- 172.21.15.46:0/2500840840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510004310 con 0x5604bbeb0400 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.896+0000 7fe51e7fc700 1 -- 172.21.15.46:0/2500840840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe510010e40 con 0x5604bbeb0400 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.896+0000 7fe51e7fc700 1 -- 172.21.15.46:0/2500840840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510005020 con 0x5604bbeb0400 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.896+0000 7fe53a893b80 1 -- 172.21.15.46:0/2500840840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeb0400 msgr2=0x5604bbebad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.896+0000 7fe53a893b80 1 --2- 172.21.15.46:0/2500840840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeb0400 0x5604bbebad60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe510007b90 tx=0x7fe510007ea0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.897+0000 7fe53a893b80 1 -- 172.21.15.46:0/2500840840 shutdown_connections 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.897+0000 7fe53a893b80 1 --2- 172.21.15.46:0/2500840840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbebb2a0 0x5604bbebd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.897+0000 7fe53a893b80 1 --2- 172.21.15.46:0/2500840840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeafae0 0x5604bbeafec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.897+0000 7fe53a893b80 1 --2- 172.21.15.46:0/2500840840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeb0400 0x5604bbebad60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.897+0000 7fe53a893b80 1 -- 172.21.15.46:0/2500840840 >> 172.21.15.46:0/2500840840 conn(0x5604bbe262a0 msgr2=0x5604bbeac0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.897+0000 7fe53a893b80 1 -- 172.21.15.46:0/2500840840 shutdown_connections 2024-03-18T20:38:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.897+0000 7fe53a893b80 1 -- 172.21.15.46:0/2500840840 wait complete. 2024-03-18T20:38:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.897+0000 7fe53a893b80 1 -- start start 2024-03-18T20:38:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.897+0000 7fe53a893b80 1 Processor -- start 2024-03-18T20:38:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe53a893b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeafae0 0x5604bbebac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe53a893b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeb0400 0x5604bbeb7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe53a893b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbebb2a0 0x5604bbeb8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe53a893b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bbd9ee00 con 0x5604bbebb2a0 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe53a893b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bbd9eaa0 con 0x5604bbeb0400 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe53a893b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bbc86720 con 0x5604bbeafae0 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe53a893b80 10 client.0 _ll_register_callbacks cb 0x5604bbc0a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe52cc9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbebb2a0 0x5604bbeb8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe52cc9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbebb2a0 0x5604bbeb8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46312/0 (socket says 172.21.15.46:46312) 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe52cc9b700 1 -- 172.21.15.46:0/2385090516 learned_addr learned my addr 172.21.15.46:0/2385090516 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe5277fe700 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeb0400 0x5604bbeb7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe527fff700 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeafae0 0x5604bbebac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe52cc9b700 1 -- 172.21.15.46:0/2385090516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeafae0 msgr2=0x5604bbebac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe52cc9b700 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeafae0 0x5604bbebac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe52cc9b700 1 -- 172.21.15.46:0/2385090516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeb0400 msgr2=0x5604bbeb7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe52cc9b700 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeb0400 0x5604bbeb7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe52cc9b700 1 -- 172.21.15.46:0/2385090516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe518017070 con 0x5604bbebb2a0 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.898+0000 7fe5277fe700 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeb0400 0x5604bbeb7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe52cc9b700 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbebb2a0 0x5604bbeb8600 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe520009b80 tx=0x7fe52000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716276a2a92d2f05 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe5077fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe5077fe700 1 -- 172.21.15.46:0/2385090516 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52009b1c0 con 0x5604bbebb2a0 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe5077fe700 1 -- 172.21.15.46:0/2385090516 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fe52009eff0 con 0x5604bbebb2a0 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe53a893b80 1 -- 172.21.15.46:0/2385090516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604bbea3b50 con 0x5604bbebb2a0 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe5077fe700 1 -- 172.21.15.46:0/2385090516 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5200a35b0 con 0x5604bbebb2a0 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe53a893b80 10 client.16302 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe53a893b80 1 -- 172.21.15.46:0/2385090516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fe5100077c0 con 0x5604bbebb2a0 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe5077fe700 1 -- 172.21.15.46:0/2385090516 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5604bbea3b50 con 0x5604bbebb2a0 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe5077fe700 1 -- 172.21.15.46:0/2385090516 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe52009b420 con 0x5604bbebb2a0 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe53a893b80 10 client.16302 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe53a893b80 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe53a893b80 20 client.16302 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe53a893b80 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5604bbecac40 0x5604bbecd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe53a893b80 4 client.16302 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.899+0000 7fe53a893b80 1 -- 172.21.15.46:0/2385090516 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604bbdfb780 con 0x5604bbecac40 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.900+0000 7fe527fff700 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5604bbecac40 0x5604bbecd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.900+0000 7fe527fff700 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5604bbecac40 0x5604bbecd060 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffca816110c22333 server_cookie=112767119f2212ce in_seq=0 out_seq=0 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.900+0000 7fe5077fe700 10 client.16302 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe5077fe700 1 -- 172.21.15.46:0/2385090516 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5604bbdfb780 con 0x5604bbecac40 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe5077fe700 10 client.16302 handle_command_reply: tid=0 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe53a893b80 1 client.16302 shutdown 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe53a893b80 1 -- 172.21.15.46:0/2385090516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbebb2a0 msgr2=0x5604bbeb8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe53a893b80 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbebb2a0 0x5604bbeb8600 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe520009b80 tx=0x7fe52000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe53a893b80 1 -- 172.21.15.46:0/2385090516 shutdown_connections 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe53a893b80 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5604bbecac40 0x5604bbecd060 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe53a893b80 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbebb2a0 0x5604bbeb8600 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe53a893b80 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeafae0 0x5604bbebac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe53a893b80 1 --2- 172.21.15.46:0/2385090516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbeb0400 0x5604bbeb7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe5077fe700 0 client.16302 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe53a893b80 1 -- 172.21.15.46:0/2385090516 >> 172.21.15.46:0/2385090516 conn(0x5604bbe262a0 msgr2=0x5604bbeabc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe53a893b80 1 -- 172.21.15.46:0/2385090516 shutdown_connections 2024-03-18T20:38:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe53a893b80 1 -- 172.21.15.46:0/2385090516 wait complete. 2024-03-18T20:38:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.901+0000 7fe53a893b80 20 client.16302 trim_cache size 0 max 16384 2024-03-18T20:38:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.919+0000 7fe53a893b80 1 Processor -- start 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.919+0000 7fe53a893b80 1 -- start start 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.919+0000 7fe53a893b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 0x5604bbeb6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.919+0000 7fe53a893b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeb0400 0x5604bbeb6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.919+0000 7fe53a893b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbebb2a0 0x5604bbec6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.919+0000 7fe53a893b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bbe769c0 con 0x5604bbeafae0 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.919+0000 7fe53a893b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bbead4c0 con 0x5604bbebb2a0 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.919+0000 7fe53a893b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bbec79b0 con 0x5604bbeb0400 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe527fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 0x5604bbeb6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe527fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 0x5604bbeb6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46314/0 (socket says 172.21.15.46:46314) 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe527fff700 1 -- 172.21.15.46:0/4274956543 learned_addr learned my addr 172.21.15.46:0/4274956543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe5277fe700 1 --2- 172.21.15.46:0/4274956543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeb0400 0x5604bbeb6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe527fff700 1 -- 172.21.15.46:0/4274956543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeb0400 msgr2=0x5604bbeb6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe527fff700 1 --2- 172.21.15.46:0/4274956543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeb0400 0x5604bbeb6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe527fff700 1 -- 172.21.15.46:0/4274956543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbebb2a0 msgr2=0x5604bbec6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe52cc9b700 1 --2- 172.21.15.46:0/4274956543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbebb2a0 0x5604bbec6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe527fff700 1 --2- 172.21.15.46:0/4274956543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbebb2a0 0x5604bbec6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe527fff700 1 -- 172.21.15.46:0/4274956543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604bbc30be0 con 0x5604bbeafae0 2024-03-18T20:38:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe527fff700 1 --2- 172.21.15.46:0/4274956543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 0x5604bbeb6850 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe5180097c0 tx=0x7fe518014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd2cd7714149ecc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe51effd700 1 -- 172.21.15.46:0/4274956543 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518022070 con 0x5604bbeafae0 2024-03-18T20:38:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe51effd700 1 -- 172.21.15.46:0/4274956543 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe518023040 con 0x5604bbeafae0 2024-03-18T20:38:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.920+0000 7fe51effd700 1 -- 172.21.15.46:0/4274956543 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51801d650 con 0x5604bbeafae0 2024-03-18T20:38:10.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.921+0000 7fe53a893b80 1 -- 172.21.15.46:0/4274956543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 msgr2=0x5604bbeb6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:10.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.921+0000 7fe53a893b80 1 --2- 172.21.15.46:0/4274956543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 0x5604bbeb6850 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe5180097c0 tx=0x7fe518014000 comp rx=0 tx=0).stop 2024-03-18T20:38:10.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.921+0000 7fe53a893b80 1 -- 172.21.15.46:0/4274956543 shutdown_connections 2024-03-18T20:38:10.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.921+0000 7fe53a893b80 1 --2- 172.21.15.46:0/4274956543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 0x5604bbeb6850 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.921+0000 7fe53a893b80 1 --2- 172.21.15.46:0/4274956543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbebb2a0 0x5604bbec6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.921+0000 7fe53a893b80 1 --2- 172.21.15.46:0/4274956543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeb0400 0x5604bbeb6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.921+0000 7fe53a893b80 1 -- 172.21.15.46:0/4274956543 >> 172.21.15.46:0/4274956543 conn(0x5604bbe262a0 msgr2=0x5604bbeb7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:10.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.921+0000 7fe53a893b80 1 -- 172.21.15.46:0/4274956543 shutdown_connections 2024-03-18T20:38:10.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.921+0000 7fe53a893b80 1 -- 172.21.15.46:0/4274956543 wait complete. 2024-03-18T20:38:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.921+0000 7fe53a893b80 1 -- start start 2024-03-18T20:38:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe53a893b80 1 Processor -- start 2024-03-18T20:38:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe53a893b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 0x5604bbeb49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe53a893b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeb0400 0x5604bbeb4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe53a893b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbebb2a0 0x5604bbeb5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe53a893b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bbf1eac0 con 0x5604bbeafae0 2024-03-18T20:38:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe53a893b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bbf1ec30 con 0x5604bbebb2a0 2024-03-18T20:38:10.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe53a893b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604bbf1edf0 con 0x5604bbeb0400 2024-03-18T20:38:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe53a893b80 10 client.0 _ll_register_callbacks cb 0x7fe52009ad00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe527fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 0x5604bbeb49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe527fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 0x5604bbeb49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46316/0 (socket says 172.21.15.46:46316) 2024-03-18T20:38:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe527fff700 1 -- 172.21.15.46:0/1667444712 learned_addr learned my addr 172.21.15.46:0/1667444712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe5277fe700 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeb0400 0x5604bbeb4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe52cc9b700 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbebb2a0 0x5604bbeb5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe527fff700 1 -- 172.21.15.46:0/1667444712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeb0400 msgr2=0x5604bbeb4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe527fff700 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeb0400 0x5604bbeb4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe527fff700 1 -- 172.21.15.46:0/1667444712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbebb2a0 msgr2=0x5604bbeb5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:10.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe527fff700 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbebb2a0 0x5604bbeb5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.922+0000 7fe527fff700 1 -- 172.21.15.46:0/1667444712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe518004400 con 0x5604bbeafae0 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe527fff700 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 0x5604bbeb49d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe51802b5d0 tx=0x7fe51802be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6551caa8245a3cbc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe5077fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe5077fe700 1 -- 172.21.15.46:0/1667444712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518022040 con 0x5604bbeafae0 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe5077fe700 1 -- 172.21.15.46:0/1667444712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fe518003e70 con 0x5604bbeafae0 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe53a893b80 1 -- 172.21.15.46:0/1667444712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe510010e40 con 0x5604bbeafae0 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe5077fe700 1 -- 172.21.15.46:0/1667444712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe51802b3b0 con 0x5604bbeafae0 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe53a893b80 10 client.16308 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe53a893b80 1 -- 172.21.15.46:0/1667444712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5604bbdc8f20 con 0x5604bbeafae0 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe5077fe700 1 -- 172.21.15.46:0/1667444712 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe518042020 con 0x5604bbeafae0 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe5077fe700 1 -- 172.21.15.46:0/1667444712 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe518004c30 con 0x5604bbeafae0 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe53a893b80 10 client.16308 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe53a893b80 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe53a893b80 20 client.16308 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.923+0000 7fe53a893b80 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5604bbf205b0 0x5604bbf229d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.924+0000 7fe53a893b80 4 client.16308 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.924+0000 7fe53a893b80 1 -- 172.21.15.46:0/1667444712 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5604bbc51d70 con 0x5604bbf205b0 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.924+0000 7fe5277fe700 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5604bbf205b0 0x5604bbf229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.924+0000 7fe5277fe700 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5604bbf205b0 0x5604bbf229d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=629d4883fd942f8e server_cookie=be7aa1094fdd39d2 in_seq=0 out_seq=0 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.924+0000 7fe5077fe700 10 client.16308 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.925+0000 7fe5077fe700 1 -- 172.21.15.46:0/1667444712 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5604bbc51d70 con 0x5604bbf205b0 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.925+0000 7fe5077fe700 10 client.16308 handle_command_reply: tid=0 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.925+0000 7fe53a893b80 1 client.16308 shutdown 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.925+0000 7fe53a893b80 1 -- 172.21.15.46:0/1667444712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 msgr2=0x5604bbeb49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.925+0000 7fe53a893b80 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 0x5604bbeb49d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe51802b5d0 tx=0x7fe51802be80 comp rx=0 tx=0).stop 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe53a893b80 1 -- 172.21.15.46:0/1667444712 shutdown_connections 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe53a893b80 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5604bbf205b0 0x5604bbf229d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe53a893b80 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604bbeafae0 0x5604bbeb49d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe53a893b80 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604bbebb2a0 0x5604bbeb5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe53a893b80 1 --2- 172.21.15.46:0/1667444712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5604bbeb0400 0x5604bbeb4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe53a893b80 1 -- 172.21.15.46:0/1667444712 >> 172.21.15.46:0/1667444712 conn(0x5604bbe262a0 msgr2=0x5604bbec6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe5077fe700 0 client.16308 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe53a893b80 1 -- 172.21.15.46:0/1667444712 shutdown_connections 2024-03-18T20:38:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe53a893b80 1 -- 172.21.15.46:0/1667444712 wait complete. 2024-03-18T20:38:10.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe53a893b80 20 client.16308 trim_cache size 0 max 16384 2024-03-18T20:38:10.934 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:38:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:38:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "4f1f4461-13ec-4cc6-baa6-8ee94e097255", 2024-03-18T20:38:10.934 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:38:10.934 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:38:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe52e6fe700 1 -- 172.21.15.46:0/162044478 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe5140a1290 msgr2=0x7fe5140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe52e6fe700 1 --2- 172.21.15.46:0/162044478 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe5140a1290 0x7fe5140a3750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe518009c60 tx=0x7fe518009260 comp rx=0 tx=0).stop 2024-03-18T20:38:10.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe52e6fe700 1 -- 172.21.15.46:0/162044478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5280fc070 msgr2=0x7fe5281a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe52e6fe700 1 --2- 172.21.15.46:0/162044478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5280fc070 0x7fe5281a54d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe52000b070 tx=0x7fe520007930 comp rx=0 tx=0).stop 2024-03-18T20:38:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe52e6fe700 1 -- 172.21.15.46:0/162044478 shutdown_connections 2024-03-18T20:38:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe52e6fe700 1 --2- 172.21.15.46:0/162044478 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe5140a1290 0x7fe5140a3750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe52e6fe700 1 --2- 172.21.15.46:0/162044478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5280fc070 0x7fe5281a54d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe52e6fe700 1 --2- 172.21.15.46:0/162044478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5281005c0 0x7fe5280fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe52e6fe700 1 --2- 172.21.15.46:0/162044478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe528104f60 0x7fe5280fbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.926+0000 7fe52e6fe700 1 -- 172.21.15.46:0/162044478 >> 172.21.15.46:0/162044478 conn(0x7fe5280761e0 msgr2=0x7fe5280ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.927+0000 7fe52e6fe700 1 -- 172.21.15.46:0/162044478 shutdown_connections 2024-03-18T20:38:10.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:10.927+0000 7fe52e6fe700 1 -- 172.21.15.46:0/162044478 wait complete. 2024-03-18T20:38:10.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:38:10.945 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:38:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.084+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/4029115854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f68104f60 msgr2=0x7f5f6810cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.084+0000 7f5f6e5cb700 1 --2- 172.21.15.46:0/4029115854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f68104f60 0x7f5f6810cae0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000b0a0 tx=0x7f5f6000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.084+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/4029115854 shutdown_connections 2024-03-18T20:38:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.084+0000 7f5f6e5cb700 1 --2- 172.21.15.46:0/4029115854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f68104f60 0x7f5f6810cae0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.084+0000 7f5f6e5cb700 1 --2- 172.21.15.46:0/4029115854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f680ffe50 0x7f5f68100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.084+0000 7f5f6e5cb700 1 --2- 172.21.15.46:0/4029115854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f68100800 0x7f5f68104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.084+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/4029115854 >> 172.21.15.46:0/4029115854 conn(0x7f5f68076d40 msgr2=0x7f5f68077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.084+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/4029115854 shutdown_connections 2024-03-18T20:38:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.084+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/4029115854 wait complete. 2024-03-18T20:38:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.085+0000 7f5f6e5cb700 1 Processor -- start 2024-03-18T20:38:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.085+0000 7f5f6e5cb700 1 -- start start 2024-03-18T20:38:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.085+0000 7f5f6e5cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f680ffe50 0x7f5f68198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.085+0000 7f5f6e5cb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f68104f60 0x7f5f68198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.085+0000 7f5f6e5cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f68198b90 0x7f5f681a3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.085+0000 7f5f6e5cb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681a3820 con 0x7f5f680ffe50 2024-03-18T20:38:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.085+0000 7f5f6e5cb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681a3990 con 0x7f5f68198b90 2024-03-18T20:38:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.085+0000 7f5f6e5cb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f681a3b50 con 0x7f5f68104f60 2024-03-18T20:38:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.085+0000 7f5f67fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f680ffe50 0x7f5f68198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.085+0000 7f5f67fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f680ffe50 0x7f5f68198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46340/0 (socket says 172.21.15.46:46340) 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.085+0000 7f5f67fff700 1 -- 172.21.15.46:0/3246084339 learned_addr learned my addr 172.21.15.46:0/3246084339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.085+0000 7f5f677fe700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f68104f60 0x7f5f68198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.086+0000 7f5f6cb68700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f68198b90 0x7f5f681a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.086+0000 7f5f67fff700 1 -- 172.21.15.46:0/3246084339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f68104f60 msgr2=0x7f5f68198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.086+0000 7f5f67fff700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f68104f60 0x7f5f68198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.086+0000 7f5f67fff700 1 -- 172.21.15.46:0/3246084339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f68198b90 msgr2=0x7f5f681a3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.086+0000 7f5f67fff700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f68198b90 0x7f5f681a3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.086+0000 7f5f67fff700 1 -- 172.21.15.46:0/3246084339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f60009d00 con 0x7f5f680ffe50 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.086+0000 7f5f67fff700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f680ffe50 0x7f5f68198110 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800d980 tx=0x7f5f5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.086+0000 7f5f657fa700 1 -- 172.21.15.46:0/3246084339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58004220 con 0x7f5f680ffe50 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.086+0000 7f5f657fa700 1 -- 172.21.15.46:0/3246084339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f58007640 con 0x7f5f680ffe50 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.086+0000 7f5f657fa700 1 -- 172.21.15.46:0/3246084339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58005610 con 0x7f5f680ffe50 2024-03-18T20:38:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.086+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/3246084339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f681a3e30 con 0x7f5f680ffe50 2024-03-18T20:38:11.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.086+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/3246084339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f681a4260 con 0x7f5f680ffe50 2024-03-18T20:38:11.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.089+0000 7f5f657fa700 1 -- 172.21.15.46:0/3246084339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5f58004750 con 0x7f5f680ffe50 2024-03-18T20:38:11.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.089+0000 7f5f657fa700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5f500a1290 0x7f5f500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.090+0000 7f5f657fa700 1 -- 172.21.15.46:0/3246084339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5f580ccb40 con 0x7f5f680ffe50 2024-03-18T20:38:11.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.090+0000 7f5f677fe700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5f500a1290 0x7f5f500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.090+0000 7f5f7a760b80 1 Processor -- start 2024-03-18T20:38:11.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.090+0000 7f5f677fe700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5f500a1290 0x7f5f500a3750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f680685b0 tx=0x7f5f5c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:11.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.090+0000 7f5f7a760b80 1 -- start start 2024-03-18T20:38:11.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.090+0000 7f5f7a760b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd043de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.090+0000 7f5f7a760b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd043e380 0x558cd0448ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.090+0000 7f5f7a760b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd0449220 0x558cd044b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.090+0000 7f5f7a760b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd0397990 con 0x558cd0449220 2024-03-18T20:38:11.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.090+0000 7f5f7a760b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd0395350 con 0x558cd043e380 2024-03-18T20:38:11.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.090+0000 7f5f7a760b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd0390c40 con 0x558cd043da60 2024-03-18T20:38:11.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f6cb68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd0449220 0x558cd044b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f67fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd043de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f6cb68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd0449220 0x558cd044b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46348/0 (socket says 172.21.15.46:46348) 2024-03-18T20:38:11.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f6cb68700 1 -- 172.21.15.46:0/3205845358 learned_addr learned my addr 172.21.15.46:0/3205845358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:11.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f677fe700 1 --2- 172.21.15.46:0/3205845358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd043e380 0x558cd0448ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f6cb68700 1 -- 172.21.15.46:0/3205845358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 msgr2=0x558cd043de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:11.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f6cb68700 1 --2- 172.21.15.46:0/3205845358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd043de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f6cb68700 1 -- 172.21.15.46:0/3205845358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd043e380 msgr2=0x558cd0448ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:11.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f6cb68700 1 --2- 172.21.15.46:0/3205845358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd043e380 0x558cd0448ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f6cb68700 1 -- 172.21.15.46:0/3205845358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cd02123b0 con 0x558cd0449220 2024-03-18T20:38:11.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f67fff700 1 --2- 172.21.15.46:0/3205845358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd043de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:11.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f6cb68700 1 --2- 172.21.15.46:0/3205845358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd0449220 0x558cd044b610 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009c80 tx=0x7f5f600092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8e568ccd709045 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:11.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f4e7fc700 1 -- 172.21.15.46:0/3205845358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000e050 con 0x558cd0449220 2024-03-18T20:38:11.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f4e7fc700 1 -- 172.21.15.46:0/3205845358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f60007cc0 con 0x558cd0449220 2024-03-18T20:38:11.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.091+0000 7f5f4e7fc700 1 -- 172.21.15.46:0/3205845358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60004c90 con 0x558cd0449220 2024-03-18T20:38:11.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.092+0000 7f5f7a760b80 1 -- 172.21.15.46:0/3205845358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd0449220 msgr2=0x558cd044b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:11.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.092+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/3205845358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd0449220 0x558cd044b610 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009c80 tx=0x7f5f600092e0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.092+0000 7f5f7a760b80 1 -- 172.21.15.46:0/3205845358 shutdown_connections 2024-03-18T20:38:11.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.092+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/3205845358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd0449220 0x558cd044b610 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.092+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/3205845358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd043de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.092+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/3205845358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd043e380 0x558cd0448ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.092+0000 7f5f7a760b80 1 -- 172.21.15.46:0/3205845358 >> 172.21.15.46:0/3205845358 conn(0x558cd03b4220 msgr2=0x558cd043a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:11.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.092+0000 7f5f7a760b80 1 -- 172.21.15.46:0/3205845358 shutdown_connections 2024-03-18T20:38:11.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.092+0000 7f5f7a760b80 1 -- 172.21.15.46:0/3205845358 wait complete. 2024-03-18T20:38:11.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.092+0000 7f5f7a760b80 1 -- start start 2024-03-18T20:38:11.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f7a760b80 1 Processor -- start 2024-03-18T20:38:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f7a760b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd043da60 0x558cd0448be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f7a760b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd0445f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f7a760b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd0449220 0x558cd04465a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f7a760b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd032cd80 con 0x558cd043e380 2024-03-18T20:38:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f7a760b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd032ca20 con 0x558cd043da60 2024-03-18T20:38:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f7a760b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd02146a0 con 0x558cd0449220 2024-03-18T20:38:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f7a760b80 10 client.0 _ll_register_callbacks cb 0x558cd01983a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd0445f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd0445f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46350/0 (socket says 172.21.15.46:46350) 2024-03-18T20:38:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f677fe700 1 -- 172.21.15.46:0/226365628 learned_addr learned my addr 172.21.15.46:0/226365628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f67fff700 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd043da60 0x558cd0448be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.093+0000 7f5f6cb68700 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd0449220 0x558cd04465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f677fe700 1 -- 172.21.15.46:0/226365628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd0449220 msgr2=0x558cd04465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:11.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f677fe700 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd0449220 0x558cd04465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f677fe700 1 -- 172.21.15.46:0/226365628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd043da60 msgr2=0x558cd0448be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:11.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f677fe700 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd043da60 0x558cd0448be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f677fe700 1 -- 172.21.15.46:0/226365628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f681a4260 con 0x558cd043e380 2024-03-18T20:38:11.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f677fe700 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd0445f20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c01e740 tx=0x7f5f5c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d641bf3f93fb965 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:11.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f477fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:11.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f477fe700 1 -- 172.21.15.46:0/226365628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c01ee60 con 0x558cd043e380 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f477fe700 1 -- 172.21.15.46:0/226365628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5f5c01f430 con 0x558cd043e380 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f7a760b80 1 -- 172.21.15.46:0/226365628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cd0431ad0 con 0x558cd043e380 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f477fe700 1 -- 172.21.15.46:0/226365628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f5c01f5f0 con 0x558cd043e380 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f7a760b80 10 client.16320 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f7a760b80 1 -- 172.21.15.46:0/226365628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f5f6810d350 con 0x558cd043e380 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.094+0000 7f5f477fe700 1 -- 172.21.15.46:0/226365628 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f5c034080 con 0x558cd043e380 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.095+0000 7f5f477fe700 1 -- 172.21.15.46:0/226365628 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5f5c0288e0 con 0x558cd043e380 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.095+0000 7f5f7a760b80 10 client.16320 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.095+0000 7f5f7a760b80 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.095+0000 7f5f7a760b80 20 client.16320 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.095+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558cd0458ec0 0x558cd045b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.095+0000 7f5f7a760b80 4 client.16320 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:11.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.095+0000 7f5f7a760b80 1 -- 172.21.15.46:0/226365628 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cd0389700 con 0x558cd0458ec0 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.095+0000 7f5f67fff700 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558cd0458ec0 0x558cd045b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.095+0000 7f5f67fff700 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558cd0458ec0 0x558cd045b2e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ca2499692be67b server_cookie=dabb63bf9e39865b in_seq=0 out_seq=0 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.095+0000 7f5f477fe700 10 client.16320 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f477fe700 1 -- 172.21.15.46:0/226365628 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558cd0389700 con 0x558cd0458ec0 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f477fe700 10 client.16320 handle_command_reply: tid=0 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f7a760b80 1 client.16320 shutdown 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f7a760b80 1 -- 172.21.15.46:0/226365628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 msgr2=0x558cd0445f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd0445f20 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c01e740 tx=0x7f5f5c0047c0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f7a760b80 1 -- 172.21.15.46:0/226365628 shutdown_connections 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558cd0458ec0 0x558cd045b2e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd0449220 0x558cd04465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd043da60 0x558cd0448be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f477fe700 0 client.16320 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:11.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/226365628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd0445f20 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f7a760b80 1 -- 172.21.15.46:0/226365628 >> 172.21.15.46:0/226365628 conn(0x558cd03b4220 msgr2=0x558cd0439c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:11.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f7a760b80 1 -- 172.21.15.46:0/226365628 shutdown_connections 2024-03-18T20:38:11.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f7a760b80 1 -- 172.21.15.46:0/226365628 wait complete. 2024-03-18T20:38:11.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.096+0000 7f5f7a760b80 20 client.16320 trim_cache size 0 max 16384 2024-03-18T20:38:11.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f7a760b80 1 Processor -- start 2024-03-18T20:38:11.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f7a760b80 1 -- start start 2024-03-18T20:38:11.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f7a760b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd044b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f7a760b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd0444830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f7a760b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd0449220 0x558cd0444d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f7a760b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd043ed30 con 0x558cd043e380 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f7a760b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd0433f80 con 0x558cd0449220 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f7a760b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd04452b0 con 0x558cd043da60 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd0444830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd0444830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46366/0 (socket says 172.21.15.46:46366) 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f677fe700 1 -- 172.21.15.46:0/2596514559 learned_addr learned my addr 172.21.15.46:0/2596514559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f6cb68700 1 --2- 172.21.15.46:0/2596514559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd0449220 0x558cd0444d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.115+0000 7f5f67fff700 1 --2- 172.21.15.46:0/2596514559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd044b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.116+0000 7f5f677fe700 1 -- 172.21.15.46:0/2596514559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 msgr2=0x558cd044b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.116+0000 7f5f677fe700 1 --2- 172.21.15.46:0/2596514559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd044b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.116+0000 7f5f677fe700 1 -- 172.21.15.46:0/2596514559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd0449220 msgr2=0x558cd0444d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.116+0000 7f5f677fe700 1 --2- 172.21.15.46:0/2596514559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd0449220 0x558cd0444d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.116+0000 7f5f677fe700 1 -- 172.21.15.46:0/2596514559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cd01beb60 con 0x558cd043e380 2024-03-18T20:38:11.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.116+0000 7f5f6cb68700 1 --2- 172.21.15.46:0/2596514559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd0449220 0x558cd0444d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:11.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.116+0000 7f5f677fe700 1 --2- 172.21.15.46:0/2596514559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd0444830 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c028ce0 tx=0x7f5f5c009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f415eb29c5c7176 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:11.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.116+0000 7f5f4effd700 1 -- 172.21.15.46:0/2596514559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c0090c0 con 0x558cd043e380 2024-03-18T20:38:11.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.116+0000 7f5f4effd700 1 -- 172.21.15.46:0/2596514559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558cd0431ad0 con 0x558cd043e380 2024-03-18T20:38:11.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.116+0000 7f5f4effd700 1 -- 172.21.15.46:0/2596514559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c01fa00 con 0x558cd043e380 2024-03-18T20:38:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.117+0000 7f5f7a760b80 1 -- 172.21.15.46:0/2596514559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 msgr2=0x558cd0444830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.117+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/2596514559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd0444830 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c028ce0 tx=0x7f5f5c009a60 comp rx=0 tx=0).stop 2024-03-18T20:38:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.117+0000 7f5f7a760b80 1 -- 172.21.15.46:0/2596514559 shutdown_connections 2024-03-18T20:38:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.117+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/2596514559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd0449220 0x558cd0444d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.117+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/2596514559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd044b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.117+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/2596514559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd0444830 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.117+0000 7f5f7a760b80 1 -- 172.21.15.46:0/2596514559 >> 172.21.15.46:0/2596514559 conn(0x558cd03b4220 msgr2=0x558cd0438750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:11.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.117+0000 7f5f7a760b80 1 -- 172.21.15.46:0/2596514559 shutdown_connections 2024-03-18T20:38:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.117+0000 7f5f7a760b80 1 -- 172.21.15.46:0/2596514559 wait complete. 2024-03-18T20:38:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.117+0000 7f5f7a760b80 1 -- start start 2024-03-18T20:38:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f7a760b80 1 Processor -- start 2024-03-18T20:38:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f7a760b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd044dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f7a760b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd044e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f7a760b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd0449220 0x558cd04abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f7a760b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd044ebd0 con 0x558cd043e380 2024-03-18T20:38:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f7a760b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd04ac230 con 0x558cd0449220 2024-03-18T20:38:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f7a760b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd04ac370 con 0x558cd043da60 2024-03-18T20:38:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f7a760b80 10 client.0 _ll_register_callbacks cb 0x558cd01983a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd044e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd044e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46370/0 (socket says 172.21.15.46:46370) 2024-03-18T20:38:11.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f677fe700 1 -- 172.21.15.46:0/1089629569 learned_addr learned my addr 172.21.15.46:0/1089629569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:11.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f67fff700 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd044dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f6cb68700 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd0449220 0x558cd04abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f677fe700 1 -- 172.21.15.46:0/1089629569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 msgr2=0x558cd044dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:11.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f677fe700 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd044dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f677fe700 1 -- 172.21.15.46:0/1089629569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd0449220 msgr2=0x558cd04abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:11.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f677fe700 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd0449220 0x558cd04abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.118+0000 7f5f677fe700 1 -- 172.21.15.46:0/1089629569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cd02123b0 con 0x558cd043e380 2024-03-18T20:38:11.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f677fe700 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd044e500 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c000c00 tx=0x7f5f5c009130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f252615233a0e266 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:11.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f477fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:11.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f477fe700 1 -- 172.21.15.46:0/1089629569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c0289c0 con 0x558cd043e380 2024-03-18T20:38:11.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f477fe700 1 -- 172.21.15.46:0/1089629569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5f5c005040 con 0x558cd043e380 2024-03-18T20:38:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f7a760b80 1 -- 172.21.15.46:0/1089629569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f60007cc0 con 0x558cd043e380 2024-03-18T20:38:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f477fe700 1 -- 172.21.15.46:0/1089629569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f5c005200 con 0x558cd043e380 2024-03-18T20:38:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f7a760b80 10 client.16326 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f7a760b80 1 -- 172.21.15.46:0/1089629569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f5f6810d350 con 0x558cd043e380 2024-03-18T20:38:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f477fe700 1 -- 172.21.15.46:0/1089629569 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f60007cc0 con 0x558cd043e380 2024-03-18T20:38:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f477fe700 1 -- 172.21.15.46:0/1089629569 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5f5c04c500 con 0x558cd043e380 2024-03-18T20:38:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f7a760b80 10 client.16326 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f7a760b80 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f7a760b80 20 client.16326 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558cd04ad7a0 0x558cd04afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f7a760b80 4 client.16326 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:38:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.119+0000 7f5f7a760b80 1 -- 172.21.15.46:0/1089629569 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558cd01dfcf0 con 0x558cd04ad7a0 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f67fff700 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558cd04ad7a0 0x558cd04afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f477fe700 10 client.16326 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f67fff700 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558cd04ad7a0 0x558cd04afbc0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f4264726299ff17 server_cookie=e6f958125c286a42 in_seq=0 out_seq=0 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f477fe700 1 -- 172.21.15.46:0/1089629569 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558cd01dfcf0 con 0x558cd04ad7a0 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f477fe700 10 client.16326 handle_command_reply: tid=0 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f7a760b80 1 client.16326 shutdown 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f7a760b80 1 -- 172.21.15.46:0/1089629569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 msgr2=0x558cd044e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd044e500 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c000c00 tx=0x7f5f5c009130 comp rx=0 tx=0).stop 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f7a760b80 1 -- 172.21.15.46:0/1089629569 shutdown_connections 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558cd04ad7a0 0x558cd04afbc0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558cd0449220 0x558cd04abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cd043da60 0x558cd044dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f477fe700 0 client.16326 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f7a760b80 1 --2- 172.21.15.46:0/1089629569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558cd043e380 0x558cd044e500 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f7a760b80 1 -- 172.21.15.46:0/1089629569 >> 172.21.15.46:0/1089629569 conn(0x558cd03b4220 msgr2=0x558cd0444770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f7a760b80 1 -- 172.21.15.46:0/1089629569 shutdown_connections 2024-03-18T20:38:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.120+0000 7f5f7a760b80 1 -- 172.21.15.46:0/1089629569 wait complete. 2024-03-18T20:38:11.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f7a760b80 20 client.16326 trim_cache size 0 max 16384 2024-03-18T20:38:11.130 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:38:11.130 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:38:11.130 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:38:11.130 INFO:teuthology.orchestra.run.smithi046.stdout: "4f1f4461-13ec-4cc6-baa6-8ee94e097255": { 2024-03-18T20:38:11.130 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:38:11.130 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "4f1f4461-13ec-4cc6-baa6-8ee94e097255", 2024-03-18T20:38:11.130 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:38:11.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:38:11.131 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:38:11.131 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:38:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/3246084339 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5f500a1290 msgr2=0x7f5f500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f6e5cb700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5f500a1290 0x7f5f500a3750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f680685b0 tx=0x7f5f5c009260 comp rx=0 tx=0).stop 2024-03-18T20:38:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/3246084339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f680ffe50 msgr2=0x7f5f68198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f6e5cb700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f680ffe50 0x7f5f68198110 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800d980 tx=0x7f5f5800dc90 comp rx=0 tx=0).stop 2024-03-18T20:38:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/3246084339 shutdown_connections 2024-03-18T20:38:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f6e5cb700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5f500a1290 0x7f5f500a3750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f6e5cb700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f680ffe50 0x7f5f68198110 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f6e5cb700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f68198b90 0x7f5f681a3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f6e5cb700 1 --2- 172.21.15.46:0/3246084339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f68104f60 0x7f5f68198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/3246084339 >> 172.21.15.46:0/3246084339 conn(0x7f5f68076d40 msgr2=0x7f5f680fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/3246084339 shutdown_connections 2024-03-18T20:38:11.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:11.121+0000 7f5f6e5cb700 1 -- 172.21.15.46:0/3246084339 wait complete. 2024-03-18T20:38:11.138 INFO:tasks.cephfs.filesystem:scrub status for tag:4f1f4461-13ec-4cc6-baa6-8ee94e097255 - {'path': '/', 'tag': '4f1f4461-13ec-4cc6-baa6-8ee94e097255', 'options': 'recursive,force'} 2024-03-18T20:38:11.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:11 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:38:11.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:11 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:11.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:11 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:11.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:11 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:11.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:11 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:11.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:11 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:11.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:11 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:11.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:11 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:11.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:11 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:38:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:11 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:11 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:11 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:11 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:11 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:11 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:11.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:11 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:11.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:11 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:38:11.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:11 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:11.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:11 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:11.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:11 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:11.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:11 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:11 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:11 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:11.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:11 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:11.673 INFO:tasks.workunit.client.0.smithi046.stdout: 1 53959 4.67 MB/sec warmup 66 sec latency 2216.479 ms 2024-03-18T20:38:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:12 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:12 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:12 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:12 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:12 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:12 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:12 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:12 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:12 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:12 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:12 smithi073 ceph-mon[47384]: pgmap v66: 97 pgs: 97 active+clean; 31 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 462 KiB/s wr, 90 op/s 2024-03-18T20:38:12.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:12 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:38:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:12 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:12 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:12 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:12 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:12 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:12 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:12 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:12 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:12 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:12.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:12 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:12 smithi155 ceph-mon[48344]: pgmap v66: 97 pgs: 97 active+clean; 31 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 462 KiB/s wr, 90 op/s 2024-03-18T20:38:12.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:12 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:38:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:12 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:12 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:12 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:12 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:12 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:12 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:12 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:12 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:12 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:12 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:12 smithi046 ceph-mon[43495]: pgmap v66: 97 pgs: 97 active+clean; 31 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 462 KiB/s wr, 90 op/s 2024-03-18T20:38:12.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:12 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:38:12.673 INFO:tasks.workunit.client.0.smithi046.stdout: 1 54027 4.60 MB/sec warmup 67 sec latency 1037.050 ms 2024-03-18T20:38:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:13 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:38:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:13 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:38:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:13 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:38:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:13 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:38:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:13 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:38:13.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:13 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:38:13.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:13 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:38:13.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:13 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:38:13.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:13 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:38:13.673 INFO:tasks.workunit.client.0.smithi046.stdout: 1 54027 4.53 MB/sec warmup 68 sec latency 1589.735 ms 2024-03-18T20:38:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:14 smithi073 ceph-mon[47384]: pgmap v67: 97 pgs: 97 active+clean; 32 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 577 KiB/s wr, 133 op/s 2024-03-18T20:38:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:14 smithi155 ceph-mon[48344]: pgmap v67: 97 pgs: 97 active+clean; 32 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 577 KiB/s wr, 133 op/s 2024-03-18T20:38:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:14 smithi046 ceph-mon[43495]: pgmap v67: 97 pgs: 97 active+clean; 32 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 577 KiB/s wr, 133 op/s 2024-03-18T20:38:14.673 INFO:tasks.workunit.client.0.smithi046.stdout: 1 54027 4.47 MB/sec warmup 69 sec latency 2589.841 ms 2024-03-18T20:38:15.673 INFO:tasks.workunit.client.0.smithi046.stdout: 1 54027 4.40 MB/sec warmup 70 sec latency 3589.912 ms 2024-03-18T20:38:16.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:15 smithi073 ceph-mon[47384]: pgmap v68: 97 pgs: 97 active+clean; 32 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 268 KiB/s wr, 71 op/s 2024-03-18T20:38:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:15 smithi155 ceph-mon[48344]: pgmap v68: 97 pgs: 97 active+clean; 32 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 268 KiB/s wr, 71 op/s 2024-03-18T20:38:16.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:15 smithi046 ceph-mon[43495]: pgmap v68: 97 pgs: 97 active+clean; 32 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 268 KiB/s wr, 71 op/s 2024-03-18T20:38:16.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 54027 4.34 MB/sec warmup 71 sec latency 4590.027 ms 2024-03-18T20:38:17.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 55789 4.39 MB/sec warmup 72 sec latency 4954.386 ms 2024-03-18T20:38:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:18 smithi073 ceph-mon[47384]: pgmap v69: 97 pgs: 97 active+clean; 32 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 268 KiB/s wr, 71 op/s 2024-03-18T20:38:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:18 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:38:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:18 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:38:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:18 smithi155 ceph-mon[48344]: pgmap v69: 97 pgs: 97 active+clean; 32 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 268 KiB/s wr, 71 op/s 2024-03-18T20:38:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:18 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:38:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:18 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:38:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:18 smithi046 ceph-mon[43495]: pgmap v69: 97 pgs: 97 active+clean; 32 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 268 KiB/s wr, 71 op/s 2024-03-18T20:38:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:18 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:38:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:18 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:38:18.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 57351 4.48 MB/sec warmup 73 sec latency 17.964 ms 2024-03-18T20:38:19.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:19 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:38:19.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:19 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:38:19.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:19 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:38:19.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:19 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:38:19.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:19 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:38:19.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:19 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:38:19.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:19 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:38:19.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:19 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:38:19.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:19 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:38:19.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 57447 4.42 MB/sec warmup 74 sec latency 948.102 ms 2024-03-18T20:38:20.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:20 smithi073 ceph-mon[47384]: pgmap v70: 97 pgs: 97 active+clean; 36 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 762 KiB/s wr, 111 op/s 2024-03-18T20:38:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:20 smithi155 ceph-mon[48344]: pgmap v70: 97 pgs: 97 active+clean; 36 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 762 KiB/s wr, 111 op/s 2024-03-18T20:38:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:20 smithi046 ceph-mon[43495]: pgmap v70: 97 pgs: 97 active+clean; 36 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 762 KiB/s wr, 111 op/s 2024-03-18T20:38:20.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 57447 4.36 MB/sec warmup 75 sec latency 1948.195 ms 2024-03-18T20:38:21.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 57447 4.30 MB/sec warmup 76 sec latency 2948.245 ms 2024-03-18T20:38:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.100+0000 7f0d7046e700 1 -- 172.21.15.46:0/1559055995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68100100 msgr2=0x7f0d681004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.100+0000 7f0d7046e700 1 --2- 172.21.15.46:0/1559055995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68100100 0x7f0d681004e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009990 tx=0x7f0d64009ca0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.100+0000 7f0d7046e700 1 -- 172.21.15.46:0/1559055995 shutdown_connections 2024-03-18T20:38:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.100+0000 7f0d7046e700 1 --2- 172.21.15.46:0/1559055995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68100100 0x7f0d681004e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.100+0000 7f0d7046e700 1 --2- 172.21.15.46:0/1559055995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d68104f40 0x7f0d6810cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.100+0000 7f0d7046e700 1 --2- 172.21.15.46:0/1559055995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d68100ab0 0x7f0d68104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.100+0000 7f0d7046e700 1 -- 172.21.15.46:0/1559055995 >> 172.21.15.46:0/1559055995 conn(0x7f0d68076a80 msgr2=0x7f0d68076e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.100+0000 7f0d7046e700 1 -- 172.21.15.46:0/1559055995 shutdown_connections 2024-03-18T20:38:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.100+0000 7f0d7046e700 1 -- 172.21.15.46:0/1559055995 wait complete. 2024-03-18T20:38:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.101+0000 7f0d7046e700 1 Processor -- start 2024-03-18T20:38:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.101+0000 7f0d7046e700 1 -- start start 2024-03-18T20:38:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.101+0000 7f0d7046e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d68100100 0x7f0d68197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.101+0000 7f0d7046e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d68104f40 0x7f0d681984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.101+0000 7f0d7046e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68198a20 0x7f0d681a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.101+0000 7f0d7046e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d681a3750 con 0x7f0d68198a20 2024-03-18T20:38:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.101+0000 7f0d7046e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d681a38c0 con 0x7f0d68104f40 2024-03-18T20:38:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.101+0000 7f0d7046e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d681a3a80 con 0x7f0d68100100 2024-03-18T20:38:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.101+0000 7f0d6ea0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68198a20 0x7f0d681a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.101+0000 7f0d6ea0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68198a20 0x7f0d681a3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43072/0 (socket says 172.21.15.46:43072) 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d6ea0b700 1 -- 172.21.15.46:0/1956062257 learned_addr learned my addr 172.21.15.46:0/1956062257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d6e20a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d68100100 0x7f0d68197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d6da09700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d68104f40 0x7f0d681984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d6e20a700 1 -- 172.21.15.46:0/1956062257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d68104f40 msgr2=0x7f0d681984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d6e20a700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d68104f40 0x7f0d681984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d6e20a700 1 -- 172.21.15.46:0/1956062257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68198a20 msgr2=0x7f0d681a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d6e20a700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68198a20 0x7f0d681a3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d6e20a700 1 -- 172.21.15.46:0/1956062257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d640095f0 con 0x7f0d68100100 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d6ea0b700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68198a20 0x7f0d681a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d6da09700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d68104f40 0x7f0d681984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d6e20a700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d68100100 0x7f0d68197fa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0d640059d0 tx=0x7f0d64004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d5f7fe700 1 -- 172.21.15.46:0/1956062257 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6401d070 con 0x7f0d68100100 2024-03-18T20:38:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d7046e700 1 -- 172.21.15.46:0/1956062257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d681a3ca0 con 0x7f0d68100100 2024-03-18T20:38:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d5f7fe700 1 -- 172.21.15.46:0/1956062257 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d64022470 con 0x7f0d68100100 2024-03-18T20:38:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d5f7fe700 1 -- 172.21.15.46:0/1956062257 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64017650 con 0x7f0d68100100 2024-03-18T20:38:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.102+0000 7f0d7046e700 1 -- 172.21.15.46:0/1956062257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d681a4120 con 0x7f0d68100100 2024-03-18T20:38:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.105+0000 7f0d5f7fe700 1 -- 172.21.15.46:0/1956062257 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0d640225e0 con 0x7f0d68100100 2024-03-18T20:38:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.105+0000 7f0d5f7fe700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0d540a1290 0x7f0d540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.105+0000 7f0d5f7fe700 1 -- 172.21.15.46:0/1956062257 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0d640cdd70 con 0x7f0d68100100 2024-03-18T20:38:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.105+0000 7f0d6da09700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0d540a1290 0x7f0d540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.106+0000 7f0d6da09700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0d540a1290 0x7f0d540a3750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0d68069c30 tx=0x7f0d58009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.106+0000 7f0d7c603b80 1 Processor -- start 2024-03-18T20:38:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.106+0000 7f0d7c603b80 1 -- start start 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.106+0000 7f0d7c603b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e206957950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.106+0000 7f0d7c603b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206957e90 0x55e206962870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.106+0000 7f0d7c603b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 0x55e2069651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.106+0000 7f0d7c603b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2068b1930 con 0x55e206962db0 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.106+0000 7f0d7c603b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2068af2f0 con 0x55e206957570 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.106+0000 7f0d7c603b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2068aabe0 con 0x55e206957e90 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d6ea0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 0x55e2069651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d6ea0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 0x55e2069651a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43082/0 (socket says 172.21.15.46:43082) 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d6ea0b700 1 -- 172.21.15.46:0/2347837042 learned_addr learned my addr 172.21.15.46:0/2347837042 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d6da09700 1 --2- 172.21.15.46:0/2347837042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206957e90 0x55e206962870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d6e20a700 1 --2- 172.21.15.46:0/2347837042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e206957950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d6ea0b700 1 -- 172.21.15.46:0/2347837042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206957e90 msgr2=0x55e206962870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d6ea0b700 1 --2- 172.21.15.46:0/2347837042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206957e90 0x55e206962870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d6ea0b700 1 -- 172.21.15.46:0/2347837042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 msgr2=0x55e206957950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d6ea0b700 1 --2- 172.21.15.46:0/2347837042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e206957950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d6ea0b700 1 -- 172.21.15.46:0/2347837042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e20672c350 con 0x55e206962db0 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d6e20a700 1 --2- 172.21.15.46:0/2347837042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e206957950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d6ea0b700 1 --2- 172.21.15.46:0/2347837042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 0x55e2069651a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009550 tx=0x7f0d60009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935eaf2c8d04cb12 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d5cda8700 1 -- 172.21.15.46:0/2347837042 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60007700 con 0x55e206962db0 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d5cda8700 1 -- 172.21.15.46:0/2347837042 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d6000f040 con 0x55e206962db0 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.107+0000 7f0d5cda8700 1 -- 172.21.15.46:0/2347837042 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60017c20 con 0x55e206962db0 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.108+0000 7f0d7c603b80 1 -- 172.21.15.46:0/2347837042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 msgr2=0x55e2069651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.108+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/2347837042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 0x55e2069651a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009550 tx=0x7f0d60009920 comp rx=0 tx=0).stop 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.108+0000 7f0d7c603b80 1 -- 172.21.15.46:0/2347837042 shutdown_connections 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.108+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/2347837042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 0x55e2069651a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.108+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/2347837042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e206957950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.108+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/2347837042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206957e90 0x55e206962870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.108+0000 7f0d7c603b80 1 -- 172.21.15.46:0/2347837042 >> 172.21.15.46:0/2347837042 conn(0x55e2068ce1c0 msgr2=0x55e206953a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.108+0000 7f0d7c603b80 1 -- 172.21.15.46:0/2347837042 shutdown_connections 2024-03-18T20:38:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.108+0000 7f0d7c603b80 1 -- 172.21.15.46:0/2347837042 wait complete. 2024-03-18T20:38:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d7c603b80 1 -- start start 2024-03-18T20:38:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d7c603b80 1 Processor -- start 2024-03-18T20:38:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d7c603b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957570 0x55e206962740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d7c603b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957e90 0x55e20695fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d7c603b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206962db0 0x55e20695ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d7c603b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e206846d20 con 0x55e206957570 2024-03-18T20:38:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d7c603b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2068469c0 con 0x55e206957e90 2024-03-18T20:38:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d7c603b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20672e640 con 0x55e206962db0 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d7c603b80 10 client.0 _ll_register_callbacks cb 0x55e2066b2340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d6e20a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957570 0x55e206962740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d6e20a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957570 0x55e206962740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43090/0 (socket says 172.21.15.46:43090) 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d6e20a700 1 -- 172.21.15.46:0/103534113 learned_addr learned my addr 172.21.15.46:0/103534113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d6ea0b700 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206962db0 0x55e20695ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.109+0000 7f0d6da09700 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957e90 0x55e20695fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d6e20a700 1 -- 172.21.15.46:0/103534113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206962db0 msgr2=0x55e20695ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d6e20a700 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206962db0 0x55e20695ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d6e20a700 1 -- 172.21.15.46:0/103534113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957e90 msgr2=0x55e20695fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d6e20a700 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957e90 0x55e20695fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d6e20a700 1 -- 172.21.15.46:0/103534113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d60009200 con 0x55e206957570 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d6e20a700 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957570 0x55e206962740 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f0d64022890 tx=0x7f0d6409ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1bd437b3a781d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d51ffb700 1 -- 172.21.15.46:0/103534113 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6409aee0 con 0x55e206957570 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d51ffb700 1 -- 172.21.15.46:0/103534113 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0d6409ed10 con 0x55e206957570 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d7c603b80 1 -- 172.21.15.46:0/103534113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e20694b6c0 con 0x55e206957570 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d51ffb700 1 -- 172.21.15.46:0/103534113 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d640a32d0 con 0x55e206957570 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d7c603b80 10 client.16338 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d7c603b80 1 -- 172.21.15.46:0/103534113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e2066d8b00 con 0x55e206957570 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.110+0000 7f0d51ffb700 1 -- 172.21.15.46:0/103534113 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d640b7080 con 0x55e206957570 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.111+0000 7f0d51ffb700 1 -- 172.21.15.46:0/103534113 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0d640aed20 con 0x55e206957570 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.111+0000 7f0d7c603b80 10 client.16338 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.111+0000 7f0d7c603b80 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.111+0000 7f0d7c603b80 20 client.16338 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.111+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e206972700 0x55e206974b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.111+0000 7f0d7c603b80 4 client.16338 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.111+0000 7f0d7c603b80 1 -- 172.21.15.46:0/103534113 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2068a36a0 con 0x55e206972700 2024-03-18T20:38:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.111+0000 7f0d6da09700 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e206972700 0x55e206974b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.111+0000 7f0d51ffb700 10 client.16338 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.111+0000 7f0d6da09700 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e206972700 0x55e206974b20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58fb6c6861283194 server_cookie=c7de685a95b3d72a in_seq=0 out_seq=0 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d51ffb700 1 -- 172.21.15.46:0/103534113 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e2068a36a0 con 0x55e206972700 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d51ffb700 10 client.16338 handle_command_reply: tid=0 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d7c603b80 1 client.16338 shutdown 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d7c603b80 1 -- 172.21.15.46:0/103534113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957570 msgr2=0x55e206962740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957570 0x55e206962740 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f0d64022890 tx=0x7f0d6409ada0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d7c603b80 1 -- 172.21.15.46:0/103534113 shutdown_connections 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e206972700 0x55e206974b20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957570 0x55e206962740 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206962db0 0x55e20695ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d51ffb700 0 client.16338 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/103534113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957e90 0x55e20695fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d7c603b80 1 -- 172.21.15.46:0/103534113 >> 172.21.15.46:0/103534113 conn(0x55e2068ce1c0 msgr2=0x55e206953650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d7c603b80 1 -- 172.21.15.46:0/103534113 shutdown_connections 2024-03-18T20:38:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d7c603b80 1 -- 172.21.15.46:0/103534113 wait complete. 2024-03-18T20:38:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.112+0000 7f0d7c603b80 20 client.16338 trim_cache size 0 max 16384 2024-03-18T20:38:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.128+0000 7f0d7c603b80 1 Processor -- start 2024-03-18T20:38:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.128+0000 7f0d7c603b80 1 -- start start 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.128+0000 7f0d7c603b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e206953bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.128+0000 7f0d7c603b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206957e90 0x55e206962270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.128+0000 7f0d7c603b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 0x55e20695f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.128+0000 7f0d7c603b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2069501d0 con 0x55e206962db0 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.128+0000 7f0d7c603b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20694b200 con 0x55e206957570 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.128+0000 7f0d7c603b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20695dfe0 con 0x55e206957e90 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d6ea0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 0x55e20695f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d6ea0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 0x55e20695f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43100/0 (socket says 172.21.15.46:43100) 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d6ea0b700 1 -- 172.21.15.46:0/2219457356 learned_addr learned my addr 172.21.15.46:0/2219457356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d6e20a700 1 --2- 172.21.15.46:0/2219457356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e206953bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d6da09700 1 --2- 172.21.15.46:0/2219457356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206957e90 0x55e206962270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d6ea0b700 1 -- 172.21.15.46:0/2219457356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206957e90 msgr2=0x55e206962270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d6ea0b700 1 --2- 172.21.15.46:0/2219457356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206957e90 0x55e206962270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d6ea0b700 1 -- 172.21.15.46:0/2219457356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 msgr2=0x55e206953bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d6ea0b700 1 --2- 172.21.15.46:0/2219457356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e206953bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d6ea0b700 1 -- 172.21.15.46:0/2219457356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e206870e40 con 0x55e206962db0 2024-03-18T20:38:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d6ea0b700 1 --2- 172.21.15.46:0/2219457356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 0x55e20695f2e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009e50 tx=0x7f0d600175d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee0c78113136edc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d5d5a9700 1 -- 172.21.15.46:0/2219457356 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60003c00 con 0x55e206962db0 2024-03-18T20:38:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d5d5a9700 1 -- 172.21.15.46:0/2219457356 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d60003d60 con 0x55e206962db0 2024-03-18T20:38:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.129+0000 7f0d5d5a9700 1 -- 172.21.15.46:0/2219457356 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60007c10 con 0x55e206962db0 2024-03-18T20:38:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.130+0000 7f0d7c603b80 1 -- 172.21.15.46:0/2219457356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 msgr2=0x55e20695f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.130+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/2219457356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 0x55e20695f2e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009e50 tx=0x7f0d600175d0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.130+0000 7f0d7c603b80 1 -- 172.21.15.46:0/2219457356 shutdown_connections 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.130+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/2219457356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206962db0 0x55e20695f2e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.130+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/2219457356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e206953bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.130+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/2219457356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206957e90 0x55e206962270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.130+0000 7f0d7c603b80 1 -- 172.21.15.46:0/2219457356 >> 172.21.15.46:0/2219457356 conn(0x55e2068ce1c0 msgr2=0x55e206952120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.130+0000 7f0d7c603b80 1 -- 172.21.15.46:0/2219457356 shutdown_connections 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.130+0000 7f0d7c603b80 1 -- 172.21.15.46:0/2219457356 wait complete. 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.130+0000 7f0d7c603b80 1 -- start start 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d7c603b80 1 Processor -- start 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d7c603b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e20695d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d7c603b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957e90 0x55e20695be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d7c603b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206962db0 0x55e20695c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d7c603b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20695c8c0 con 0x55e206957e90 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d7c603b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20695ca30 con 0x55e206957570 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d7c603b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20695cbf0 con 0x55e206962db0 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d7c603b80 10 client.0 _ll_register_callbacks cb 0x55e2066b2340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d6da09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957e90 0x55e20695be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d6da09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957e90 0x55e20695be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43112/0 (socket says 172.21.15.46:43112) 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d6da09700 1 -- 172.21.15.46:0/96389013 learned_addr learned my addr 172.21.15.46:0/96389013 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d6e20a700 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e20695d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d6ea0b700 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206962db0 0x55e20695c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d6da09700 1 -- 172.21.15.46:0/96389013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206962db0 msgr2=0x55e20695c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d6da09700 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206962db0 0x55e20695c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d6da09700 1 -- 172.21.15.46:0/96389013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 msgr2=0x55e20695d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d6da09700 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e20695d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.131+0000 7f0d6da09700 1 -- 172.21.15.46:0/96389013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e20672c350 con 0x55e206957e90 2024-03-18T20:38:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d6da09700 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957e90 0x55e20695be40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0d58013010 tx=0x7f0d580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ba27c6cc2cc6fc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d51ffb700 1 -- 172.21.15.46:0/96389013 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d58024070 con 0x55e206957e90 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d51ffb700 1 -- 172.21.15.46:0/96389013 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0d580282a0 con 0x55e206957e90 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d7c603b80 1 -- 172.21.15.46:0/96389013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d6000f040 con 0x55e206957e90 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d51ffb700 1 -- 172.21.15.46:0/96389013 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d58028730 con 0x55e206957e90 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d7c603b80 10 client.16344 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d7c603b80 1 -- 172.21.15.46:0/96389013 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e206730c70 con 0x55e206957e90 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d51ffb700 1 -- 172.21.15.46:0/96389013 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d6000f040 con 0x55e206957e90 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d51ffb700 1 -- 172.21.15.46:0/96389013 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0d58037070 con 0x55e206957e90 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d7c603b80 10 client.16344 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d7c603b80 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d7c603b80 20 client.16344 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:22.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e2069cbe40 0x55e2069ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d7c603b80 4 client.16344 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:38:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.132+0000 7f0d7c603b80 1 -- 172.21.15.46:0/96389013 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e2066f9c90 con 0x55e2069cbe40 2024-03-18T20:38:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.133+0000 7f0d6e20a700 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e2069cbe40 0x55e2069ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.133+0000 7f0d6e20a700 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e2069cbe40 0x55e2069ce260 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f26485a3830cd4 server_cookie=9cf0bf160f622c88 in_seq=0 out_seq=0 2024-03-18T20:38:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.133+0000 7f0d51ffb700 10 client.16344 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d51ffb700 1 -- 172.21.15.46:0/96389013 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e2066f9c90 con 0x55e2069cbe40 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d51ffb700 10 client.16344 handle_command_reply: tid=0 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d7c603b80 1 client.16344 shutdown 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d7c603b80 1 -- 172.21.15.46:0/96389013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957e90 msgr2=0x55e20695be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957e90 0x55e20695be40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0d58013010 tx=0x7f0d580047c0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d7c603b80 1 -- 172.21.15.46:0/96389013 shutdown_connections 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e2069cbe40 0x55e2069ce260 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e206962db0 0x55e20695c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d51ffb700 0 client.16344 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e206957570 0x55e20695d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d7c603b80 1 --2- 172.21.15.46:0/96389013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e206957e90 0x55e20695be40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d7c603b80 1 -- 172.21.15.46:0/96389013 >> 172.21.15.46:0/96389013 conn(0x55e2068ce1c0 msgr2=0x55e20695e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d7c603b80 1 -- 172.21.15.46:0/96389013 shutdown_connections 2024-03-18T20:38:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d7c603b80 1 -- 172.21.15.46:0/96389013 wait complete. 2024-03-18T20:38:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.161+0000 7f0d7c603b80 20 client.16344 trim_cache size 0 max 16384 2024-03-18T20:38:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:38:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:38:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:38:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:38:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.162+0000 7f0d7046e700 1 -- 172.21.15.46:0/1956062257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0d540a1290 msgr2=0x7f0d540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.162+0000 7f0d7046e700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0d540a1290 0x7f0d540a3750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0d68069c30 tx=0x7f0d58009260 comp rx=0 tx=0).stop 2024-03-18T20:38:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.162+0000 7f0d7046e700 1 -- 172.21.15.46:0/1956062257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d68100100 msgr2=0x7f0d68197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.162+0000 7f0d7046e700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d68100100 0x7f0d68197fa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0d640059d0 tx=0x7f0d64004730 comp rx=0 tx=0).stop 2024-03-18T20:38:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.162+0000 7f0d7046e700 1 -- 172.21.15.46:0/1956062257 shutdown_connections 2024-03-18T20:38:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.162+0000 7f0d7046e700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0d540a1290 0x7f0d540a3750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.162+0000 7f0d7046e700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68198a20 0x7f0d681a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.162+0000 7f0d7046e700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d68100100 0x7f0d68197fa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.162+0000 7f0d7046e700 1 --2- 172.21.15.46:0/1956062257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d68104f40 0x7f0d681984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.162+0000 7f0d7046e700 1 -- 172.21.15.46:0/1956062257 >> 172.21.15.46:0/1956062257 conn(0x7f0d68076a80 msgr2=0x7f0d680fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.162+0000 7f0d7046e700 1 -- 172.21.15.46:0/1956062257 shutdown_connections 2024-03-18T20:38:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:22.162+0000 7f0d7046e700 1 -- 172.21.15.46:0/1956062257 wait complete. 2024-03-18T20:38:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:22 smithi073 ceph-mon[47384]: pgmap v71: 97 pgs: 97 active+clean; 36 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 610 KiB/s wr, 83 op/s 2024-03-18T20:38:22.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:22 smithi155 ceph-mon[48344]: pgmap v71: 97 pgs: 97 active+clean; 36 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 610 KiB/s wr, 83 op/s 2024-03-18T20:38:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:22 smithi046 ceph-mon[43495]: pgmap v71: 97 pgs: 97 active+clean; 36 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 610 KiB/s wr, 83 op/s 2024-03-18T20:38:22.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 57503 4.25 MB/sec warmup 77 sec latency 3309.190 ms 2024-03-18T20:38:23.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 57503 4.19 MB/sec warmup 78 sec latency 1603.555 ms 2024-03-18T20:38:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:24 smithi073 ceph-mon[47384]: pgmap v72: 97 pgs: 97 active+clean; 32 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 724 KiB/s wr, 116 op/s 2024-03-18T20:38:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:24 smithi155 ceph-mon[48344]: pgmap v72: 97 pgs: 97 active+clean; 32 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 724 KiB/s wr, 116 op/s 2024-03-18T20:38:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:24 smithi046 ceph-mon[43495]: pgmap v72: 97 pgs: 97 active+clean; 32 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 724 KiB/s wr, 116 op/s 2024-03-18T20:38:24.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 57503 4.14 MB/sec warmup 79 sec latency 2603.659 ms 2024-03-18T20:38:25.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 57503 4.09 MB/sec warmup 80 sec latency 3603.747 ms 2024-03-18T20:38:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:25 smithi073 ceph-mon[47384]: pgmap v73: 97 pgs: 97 active+clean; 32 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 608 KiB/s wr, 72 op/s 2024-03-18T20:38:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:25 smithi155 ceph-mon[48344]: pgmap v73: 97 pgs: 97 active+clean; 32 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 608 KiB/s wr, 72 op/s 2024-03-18T20:38:26.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:25 smithi046 ceph-mon[43495]: pgmap v73: 97 pgs: 97 active+clean; 32 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 608 KiB/s wr, 72 op/s 2024-03-18T20:38:26.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 57503 4.04 MB/sec warmup 81 sec latency 4603.832 ms 2024-03-18T20:38:27.675 INFO:tasks.workunit.client.0.smithi046.stdout: 1 59258 4.08 MB/sec warmup 82 sec latency 4962.910 ms 2024-03-18T20:38:28.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:28 smithi073 ceph-mon[47384]: pgmap v74: 97 pgs: 97 active+clean; 32 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 608 KiB/s wr, 72 op/s 2024-03-18T20:38:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:28 smithi155 ceph-mon[48344]: pgmap v74: 97 pgs: 97 active+clean; 32 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 608 KiB/s wr, 72 op/s 2024-03-18T20:38:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:28 smithi046 ceph-mon[43495]: pgmap v74: 97 pgs: 97 active+clean; 32 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 608 KiB/s wr, 72 op/s 2024-03-18T20:38:28.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 60876 4.17 MB/sec warmup 83 sec latency 12.725 ms 2024-03-18T20:38:29.674 INFO:tasks.workunit.client.0.smithi046.stdout: 1 63310 4.27 MB/sec warmup 84 sec latency 26.644 ms 2024-03-18T20:38:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:30 smithi073 ceph-mon[47384]: pgmap v75: 97 pgs: 97 active+clean; 33 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 782 KiB/s wr, 106 op/s 2024-03-18T20:38:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:30 smithi155 ceph-mon[48344]: pgmap v75: 97 pgs: 97 active+clean; 33 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 782 KiB/s wr, 106 op/s 2024-03-18T20:38:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:30 smithi046 ceph-mon[43495]: pgmap v75: 97 pgs: 97 active+clean; 33 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 782 KiB/s wr, 106 op/s 2024-03-18T20:38:30.675 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64536 4.30 MB/sec warmup 85 sec latency 141.083 ms 2024-03-18T20:38:31.675 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64536 4.25 MB/sec warmup 86 sec latency 1141.197 ms 2024-03-18T20:38:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:32 smithi073 ceph-mon[47384]: pgmap v76: 97 pgs: 97 active+clean; 33 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 287 KiB/s wr, 66 op/s 2024-03-18T20:38:32.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:32 smithi155 ceph-mon[48344]: pgmap v76: 97 pgs: 97 active+clean; 33 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 287 KiB/s wr, 66 op/s 2024-03-18T20:38:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:32 smithi046 ceph-mon[43495]: pgmap v76: 97 pgs: 97 active+clean; 33 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 287 KiB/s wr, 66 op/s 2024-03-18T20:38:32.675 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64592 4.20 MB/sec warmup 87 sec latency 1500.287 ms 2024-03-18T20:38:33.675 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64592 4.15 MB/sec warmup 88 sec latency 1604.639 ms 2024-03-18T20:38:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:34 smithi073 ceph-mon[47384]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 628 KiB/s wr, 126 op/s 2024-03-18T20:38:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:34 smithi155 ceph-mon[48344]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 628 KiB/s wr, 126 op/s 2024-03-18T20:38:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:34 smithi046 ceph-mon[43495]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 628 KiB/s wr, 126 op/s 2024-03-18T20:38:34.675 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64592 4.11 MB/sec warmup 89 sec latency 2604.728 ms 2024-03-18T20:38:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.289+0000 7efe08f77700 1 -- 172.21.15.46:0/603011093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe04105c60 msgr2=0x7efe04106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.289+0000 7efe08f77700 1 --2- 172.21.15.46:0/603011093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe04105c60 0x7efe04106040 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efdec009990 tx=0x7efdec009ca0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.289+0000 7efe08f77700 1 -- 172.21.15.46:0/603011093 shutdown_connections 2024-03-18T20:38:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.289+0000 7efe08f77700 1 --2- 172.21.15.46:0/603011093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe04105c60 0x7efe04106040 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.289+0000 7efe08f77700 1 --2- 172.21.15.46:0/603011093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe04100570 0x7efe0410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.289+0000 7efe08f77700 1 --2- 172.21.15.46:0/603011093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe040ffbb0 0x7efe04100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.289+0000 7efe08f77700 1 -- 172.21.15.46:0/603011093 >> 172.21.15.46:0/603011093 conn(0x7efe04076d50 msgr2=0x7efe04077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.289+0000 7efe08f77700 1 -- 172.21.15.46:0/603011093 shutdown_connections 2024-03-18T20:38:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.289+0000 7efe08f77700 1 -- 172.21.15.46:0/603011093 wait complete. 2024-03-18T20:38:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.290+0000 7efe08f77700 1 Processor -- start 2024-03-18T20:38:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.290+0000 7efe08f77700 1 -- start start 2024-03-18T20:38:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.290+0000 7efe08f77700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe040ffbb0 0x7efe04195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.290+0000 7efe08f77700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe04100570 0x7efe041961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.290+0000 7efe08f77700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe04105c60 0x7efe04196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.290+0000 7efe08f77700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041a5420 con 0x7efe040ffbb0 2024-03-18T20:38:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.290+0000 7efe08f77700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041a5590 con 0x7efe04100570 2024-03-18T20:38:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.290+0000 7efe08f77700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041a5750 con 0x7efe04105c60 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.290+0000 7efe0259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe040ffbb0 0x7efe04195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.290+0000 7efe0259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe040ffbb0 0x7efe04195c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33916/0 (socket says 172.21.15.46:33916) 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.290+0000 7efe0259c700 1 -- 172.21.15.46:0/1036518357 learned_addr learned my addr 172.21.15.46:0/1036518357 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efe02d9d700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe04105c60 0x7efe04196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efe01d9b700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe04100570 0x7efe041961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efe0259c700 1 -- 172.21.15.46:0/1036518357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe04105c60 msgr2=0x7efe04196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efe0259c700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe04105c60 0x7efe04196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efe0259c700 1 -- 172.21.15.46:0/1036518357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe04100570 msgr2=0x7efe041961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efe0259c700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe04100570 0x7efe041961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efe0259c700 1 -- 172.21.15.46:0/1036518357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec0095f0 con 0x7efe040ffbb0 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efe0259c700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe040ffbb0 0x7efe04195c80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efdec009fd0 tx=0x7efdec00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efdfb7fe700 1 -- 172.21.15.46:0/1036518357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec01d070 con 0x7efe040ffbb0 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efdfb7fe700 1 -- 172.21.15.46:0/1036518357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdec005050 con 0x7efe040ffbb0 2024-03-18T20:38:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efdfb7fe700 1 -- 172.21.15.46:0/1036518357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec004a90 con 0x7efe040ffbb0 2024-03-18T20:38:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efe08f77700 1 -- 172.21.15.46:0/1036518357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe041a59d0 con 0x7efe040ffbb0 2024-03-18T20:38:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.291+0000 7efe08f77700 1 -- 172.21.15.46:0/1036518357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe041a5e30 con 0x7efe040ffbb0 2024-03-18T20:38:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.294+0000 7efdfb7fe700 1 -- 172.21.15.46:0/1036518357 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7efdec004c10 con 0x7efe040ffbb0 2024-03-18T20:38:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.295+0000 7efdfb7fe700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efdf00a1290 0x7efdf00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.295+0000 7efe01d9b700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efdf00a1290 0x7efdf00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.295+0000 7efdfb7fe700 1 -- 172.21.15.46:0/1036518357 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7efdec0cda90 con 0x7efe040ffbb0 2024-03-18T20:38:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.295+0000 7efe01d9b700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efdf00a1290 0x7efdf00a3750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe04068920 tx=0x7efdf400a380 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.295+0000 7efe1510cb80 1 Processor -- start 2024-03-18T20:38:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe1510cb80 1 -- start start 2024-03-18T20:38:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe1510cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f22198950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe1510cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe1510cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe1510cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f220f2930 con 0x563f22198570 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe1510cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f220f02f0 con 0x563f221a3db0 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe1510cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f220ebbe0 con 0x563f22198e90 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe0259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f22198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe0259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f22198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33930/0 (socket says 172.21.15.46:33930) 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe0259c700 1 -- 172.21.15.46:0/2359051249 learned_addr learned my addr 172.21.15.46:0/2359051249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe01d9b700 1 --2- 172.21.15.46:0/2359051249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe02d9d700 1 --2- 172.21.15.46:0/2359051249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe0259c700 1 -- 172.21.15.46:0/2359051249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 msgr2=0x563f221a3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe0259c700 1 --2- 172.21.15.46:0/2359051249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe0259c700 1 -- 172.21.15.46:0/2359051249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 msgr2=0x563f221a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe0259c700 1 --2- 172.21.15.46:0/2359051249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.296+0000 7efe0259c700 1 -- 172.21.15.46:0/2359051249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f21f6d350 con 0x563f22198570 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.297+0000 7efe02d9d700 1 --2- 172.21.15.46:0/2359051249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.297+0000 7efe0259c700 1 --2- 172.21.15.46:0/2359051249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f22198950 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efdec0976b0 tx=0x7efdec09a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426d46cf9ed3f137 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.297+0000 7efdf8ff9700 1 -- 172.21.15.46:0/2359051249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec09ea70 con 0x563f22198570 2024-03-18T20:38:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.297+0000 7efdf8ff9700 1 -- 172.21.15.46:0/2359051249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdec0a3030 con 0x563f22198570 2024-03-18T20:38:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.297+0000 7efdf8ff9700 1 -- 172.21.15.46:0/2359051249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec0a2ba0 con 0x563f22198570 2024-03-18T20:38:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.298+0000 7efe1510cb80 1 -- 172.21.15.46:0/2359051249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 msgr2=0x563f22198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.298+0000 7efe1510cb80 1 --2- 172.21.15.46:0/2359051249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f22198950 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efdec0976b0 tx=0x7efdec09a890 comp rx=0 tx=0).stop 2024-03-18T20:38:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.298+0000 7efe1510cb80 1 -- 172.21.15.46:0/2359051249 shutdown_connections 2024-03-18T20:38:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.298+0000 7efe1510cb80 1 --2- 172.21.15.46:0/2359051249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f22198950 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.298+0000 7efe1510cb80 1 --2- 172.21.15.46:0/2359051249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.298+0000 7efe1510cb80 1 --2- 172.21.15.46:0/2359051249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.298+0000 7efe1510cb80 1 -- 172.21.15.46:0/2359051249 >> 172.21.15.46:0/2359051249 conn(0x563f2210f1c0 msgr2=0x563f22194a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.298+0000 7efe1510cb80 1 -- 172.21.15.46:0/2359051249 shutdown_connections 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.298+0000 7efe1510cb80 1 -- 172.21.15.46:0/2359051249 wait complete. 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.298+0000 7efe1510cb80 1 -- start start 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.298+0000 7efe1510cb80 1 Processor -- start 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe1510cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f221a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe1510cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe1510cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe1510cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f22087d20 con 0x563f22198570 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe1510cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f220879c0 con 0x563f221a3db0 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe1510cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f21f6f640 con 0x563f22198e90 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe1510cb80 10 client.0 _ll_register_callbacks cb 0x563f21ef3340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe0259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f221a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe0259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f221a3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33944/0 (socket says 172.21.15.46:33944) 2024-03-18T20:38:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe0259c700 1 -- 172.21.15.46:0/1114946324 learned_addr learned my addr 172.21.15.46:0/1114946324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe02d9d700 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe01d9b700 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe0259c700 1 -- 172.21.15.46:0/1114946324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 msgr2=0x563f221a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe0259c700 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe0259c700 1 -- 172.21.15.46:0/1114946324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 msgr2=0x563f221a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe0259c700 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.299+0000 7efe0259c700 1 -- 172.21.15.46:0/1114946324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe041a5e30 con 0x563f22198570 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.300+0000 7efe0259c700 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f221a3730 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efdec0095c0 tx=0x7efdec09a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52230cc9ade2bb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.300+0000 7efde9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.300+0000 7efde9ffb700 1 -- 172.21.15.46:0/1114946324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec09a3d0 con 0x563f22198570 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.300+0000 7efde9ffb700 1 -- 172.21.15.46:0/1114946324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7efdec0a2db0 con 0x563f22198570 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.300+0000 7efe1510cb80 1 -- 172.21.15.46:0/1114946324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f2218c6c0 con 0x563f22198570 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.300+0000 7efde9ffb700 1 -- 172.21.15.46:0/1114946324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdec00fae0 con 0x563f22198570 2024-03-18T20:38:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.300+0000 7efe1510cb80 10 client.16356 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.300+0000 7efe1510cb80 1 -- 172.21.15.46:0/1114946324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563f21f19b00 con 0x563f22198570 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.300+0000 7efde9ffb700 1 -- 172.21.15.46:0/1114946324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f2218c6c0 con 0x563f22198570 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.301+0000 7efde9ffb700 1 -- 172.21.15.46:0/1114946324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efdec079d20 con 0x563f22198570 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.301+0000 7efe1510cb80 10 client.16356 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.301+0000 7efe1510cb80 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.301+0000 7efe1510cb80 20 client.16356 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.301+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563f221b3100 0x563f221b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.301+0000 7efe1510cb80 4 client.16356 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.301+0000 7efe1510cb80 1 -- 172.21.15.46:0/1114946324 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f220e46a0 con 0x563f221b3100 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.301+0000 7efe01d9b700 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563f221b3100 0x563f221b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.301+0000 7efe01d9b700 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563f221b3100 0x563f221b5520 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3538afaf668a8a server_cookie=5a737be03dc713c6 in_seq=0 out_seq=0 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.301+0000 7efde9ffb700 10 client.16356 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efde9ffb700 1 -- 172.21.15.46:0/1114946324 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563f220e46a0 con 0x563f221b3100 2024-03-18T20:38:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efde9ffb700 10 client.16356 handle_command_reply: tid=0 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efe1510cb80 1 client.16356 shutdown 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efe1510cb80 1 -- 172.21.15.46:0/1114946324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 msgr2=0x563f221a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f221a3730 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efdec0095c0 tx=0x7efdec09a360 comp rx=0 tx=0).stop 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efe1510cb80 1 -- 172.21.15.46:0/1114946324 shutdown_connections 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563f221b3100 0x563f221b5520 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f221a3730 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1114946324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efde9ffb700 0 client.16356 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efe1510cb80 1 -- 172.21.15.46:0/1114946324 >> 172.21.15.46:0/1114946324 conn(0x563f2210f1c0 msgr2=0x563f22194650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efe1510cb80 1 -- 172.21.15.46:0/1114946324 shutdown_connections 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.302+0000 7efe1510cb80 1 -- 172.21.15.46:0/1114946324 wait complete. 2024-03-18T20:38:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.303+0000 7efe1510cb80 20 client.16356 trim_cache size 0 max 16384 2024-03-18T20:38:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe1510cb80 1 Processor -- start 2024-03-18T20:38:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe1510cb80 1 -- start start 2024-03-18T20:38:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe1510cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f22198570 0x563f22194bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe1510cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe1510cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f221a3db0 0x563f2219dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe1510cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f2218ec10 con 0x563f221a3db0 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe1510cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f22195760 con 0x563f22198570 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe1510cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f221b5b80 con 0x563f22198e90 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe02d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f221a3db0 0x563f2219dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe02d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f221a3db0 0x563f2219dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33948/0 (socket says 172.21.15.46:33948) 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe02d9d700 1 -- 172.21.15.46:0/1713076478 learned_addr learned my addr 172.21.15.46:0/1713076478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe0259c700 1 --2- 172.21.15.46:0/1713076478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f22198570 0x563f22194bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe01d9b700 1 --2- 172.21.15.46:0/1713076478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.319+0000 7efe02d9d700 1 -- 172.21.15.46:0/1713076478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 msgr2=0x563f221a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.320+0000 7efe02d9d700 1 --2- 172.21.15.46:0/1713076478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.320+0000 7efe02d9d700 1 -- 172.21.15.46:0/1713076478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f22198570 msgr2=0x563f22194bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.320+0000 7efe02d9d700 1 --2- 172.21.15.46:0/1713076478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f22198570 0x563f22194bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.320+0000 7efe02d9d700 1 -- 172.21.15.46:0/1713076478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f220b1e40 con 0x563f221a3db0 2024-03-18T20:38:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.320+0000 7efe02d9d700 1 --2- 172.21.15.46:0/1713076478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f221a3db0 0x563f2219dd80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009f60 tx=0x7efdfc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5ae5e141ec5890 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.320+0000 7efdf97fa700 1 -- 172.21.15.46:0/1713076478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc009ab0 con 0x563f221a3db0 2024-03-18T20:38:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.320+0000 7efdf97fa700 1 -- 172.21.15.46:0/1713076478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdfc00f070 con 0x563f221a3db0 2024-03-18T20:38:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.320+0000 7efdf97fa700 1 -- 172.21.15.46:0/1713076478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc016980 con 0x563f221a3db0 2024-03-18T20:38:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 -- 172.21.15.46:0/1713076478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f221a3db0 msgr2=0x563f2219dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1713076478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f221a3db0 0x563f2219dd80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009f60 tx=0x7efdfc009f90 comp rx=0 tx=0).stop 2024-03-18T20:38:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 -- 172.21.15.46:0/1713076478 shutdown_connections 2024-03-18T20:38:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1713076478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f221a3db0 0x563f2219dd80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1713076478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f22198570 0x563f22194bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1713076478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 -- 172.21.15.46:0/1713076478 >> 172.21.15.46:0/1713076478 conn(0x563f2210f1c0 msgr2=0x563f221a2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 -- 172.21.15.46:0/1713076478 shutdown_connections 2024-03-18T20:38:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 -- 172.21.15.46:0/1713076478 wait complete. 2024-03-18T20:38:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 -- start start 2024-03-18T20:38:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 Processor -- start 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f221a7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f221a6da0 con 0x563f22198570 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f221a6f10 con 0x563f221a3db0 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.321+0000 7efe1510cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f221a70d0 con 0x563f22198e90 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe1510cb80 10 client.0 _ll_register_callbacks cb 0x7efdec09a5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe0259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f221a7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe0259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f221a7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33952/0 (socket says 172.21.15.46:33952) 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe0259c700 1 -- 172.21.15.46:0/1797673138 learned_addr learned my addr 172.21.15.46:0/1797673138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe01d9b700 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe02d9d700 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe0259c700 1 -- 172.21.15.46:0/1797673138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 msgr2=0x563f221a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe0259c700 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe0259c700 1 -- 172.21.15.46:0/1797673138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 msgr2=0x563f221a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe0259c700 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe0259c700 1 -- 172.21.15.46:0/1797673138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc009c10 con 0x563f22198570 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe0259c700 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f221a7ee0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efdec098f90 tx=0x7efdec09a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178da3bc49bb23cd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efde9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efde9ffb700 1 -- 172.21.15.46:0/1797673138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec060cf0 con 0x563f22198570 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efde9ffb700 1 -- 172.21.15.46:0/1797673138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7efdec099e30 con 0x563f22198570 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efe1510cb80 1 -- 172.21.15.46:0/1797673138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdec0a3030 con 0x563f22198570 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.322+0000 7efde9ffb700 1 -- 172.21.15.46:0/1797673138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdec09a3d0 con 0x563f22198570 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.323+0000 7efe1510cb80 10 client.16362 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.323+0000 7efe1510cb80 1 -- 172.21.15.46:0/1797673138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563f21f71c70 con 0x563f22198570 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.323+0000 7efde9ffb700 1 -- 172.21.15.46:0/1797673138 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdec0a3030 con 0x563f22198570 2024-03-18T20:38:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.323+0000 7efde9ffb700 1 -- 172.21.15.46:0/1797673138 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efdec07f050 con 0x563f22198570 2024-03-18T20:38:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.323+0000 7efe1510cb80 10 client.16362 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.323+0000 7efe1510cb80 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.323+0000 7efe1510cb80 20 client.16362 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.323+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563f22207ea0 0x563f2220a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.323+0000 7efe1510cb80 4 client.16362 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:38:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.323+0000 7efe1510cb80 1 -- 172.21.15.46:0/1797673138 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563f21f3ac90 con 0x563f22207ea0 2024-03-18T20:38:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.323+0000 7efe01d9b700 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563f22207ea0 0x563f2220a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.324+0000 7efde9ffb700 10 client.16362 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.324+0000 7efe01d9b700 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563f22207ea0 0x563f2220a290 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d251c4f2236e2c2d server_cookie=466219260b3b05c7 in_seq=0 out_seq=0 2024-03-18T20:38:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efde9ffb700 1 -- 172.21.15.46:0/1797673138 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563f21f3ac90 con 0x563f22207ea0 2024-03-18T20:38:35.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efde9ffb700 10 client.16362 handle_command_reply: tid=0 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efe1510cb80 1 client.16362 shutdown 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efe1510cb80 1 -- 172.21.15.46:0/1797673138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 msgr2=0x563f221a7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f221a7ee0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efdec098f90 tx=0x7efdec09a650 comp rx=0 tx=0).stop 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efe1510cb80 1 -- 172.21.15.46:0/1797673138 shutdown_connections 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563f22207ea0 0x563f2220a290 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f22198570 0x563f221a7ee0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f221a3db0 0x563f221a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efe1510cb80 1 --2- 172.21.15.46:0/1797673138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f22198e90 0x563f221a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efde9ffb700 0 client.16362 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efe1510cb80 1 -- 172.21.15.46:0/1797673138 >> 172.21.15.46:0/1797673138 conn(0x563f2210f1c0 msgr2=0x563f221a08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efe1510cb80 1 -- 172.21.15.46:0/1797673138 shutdown_connections 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.352+0000 7efe1510cb80 1 -- 172.21.15.46:0/1797673138 wait complete. 2024-03-18T20:38:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe1510cb80 20 client.16362 trim_cache size 0 max 16384 2024-03-18T20:38:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:38:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:38:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:38:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:38:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe08f77700 1 -- 172.21.15.46:0/1036518357 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efdf00a1290 msgr2=0x7efdf00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe08f77700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efdf00a1290 0x7efdf00a3750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe04068920 tx=0x7efdf400a380 comp rx=0 tx=0).stop 2024-03-18T20:38:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe08f77700 1 -- 172.21.15.46:0/1036518357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe040ffbb0 msgr2=0x7efe04195c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe08f77700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe040ffbb0 0x7efe04195c80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efdec009fd0 tx=0x7efdec00f670 comp rx=0 tx=0).stop 2024-03-18T20:38:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe08f77700 1 -- 172.21.15.46:0/1036518357 shutdown_connections 2024-03-18T20:38:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe08f77700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efdf00a1290 0x7efdf00a3750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe08f77700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe040ffbb0 0x7efe04195c80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe08f77700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe04105c60 0x7efe04196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe08f77700 1 --2- 172.21.15.46:0/1036518357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe04100570 0x7efe041961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe08f77700 1 -- 172.21.15.46:0/1036518357 >> 172.21.15.46:0/1036518357 conn(0x7efe04076d50 msgr2=0x7efe040fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe08f77700 1 -- 172.21.15.46:0/1036518357 shutdown_connections 2024-03-18T20:38:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:35.353+0000 7efe08f77700 1 -- 172.21.15.46:0/1036518357 wait complete. 2024-03-18T20:38:35.675 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64592 4.06 MB/sec warmup 90 sec latency 3604.836 ms 2024-03-18T20:38:36.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:35 smithi073 ceph-mon[47384]: pgmap v78: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 514 KiB/s wr, 93 op/s 2024-03-18T20:38:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:35 smithi155 ceph-mon[48344]: pgmap v78: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 514 KiB/s wr, 93 op/s 2024-03-18T20:38:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:35 smithi046 ceph-mon[43495]: pgmap v78: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 514 KiB/s wr, 93 op/s 2024-03-18T20:38:36.675 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64592 4.02 MB/sec warmup 91 sec latency 4604.914 ms 2024-03-18T20:38:37.675 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64660 3.97 MB/sec warmup 92 sec latency 4966.094 ms 2024-03-18T20:38:38.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:38 smithi073 ceph-mon[47384]: pgmap v79: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 514 KiB/s wr, 93 op/s 2024-03-18T20:38:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:38 smithi155 ceph-mon[48344]: pgmap v79: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 514 KiB/s wr, 93 op/s 2024-03-18T20:38:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:38 smithi046 ceph-mon[43495]: pgmap v79: 97 pgs: 97 active+clean; 35 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 514 KiB/s wr, 93 op/s 2024-03-18T20:38:38.675 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64660 3.93 MB/sec warmup 93 sec latency 1594.834 ms 2024-03-18T20:38:39.675 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64660 3.89 MB/sec warmup 94 sec latency 2594.946 ms 2024-03-18T20:38:40.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:40 smithi073 ceph-mon[47384]: pgmap v80: 97 pgs: 97 active+clean; 35 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 621 KiB/s wr, 119 op/s 2024-03-18T20:38:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:40 smithi155 ceph-mon[48344]: pgmap v80: 97 pgs: 97 active+clean; 35 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 621 KiB/s wr, 119 op/s 2024-03-18T20:38:40.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:40 smithi046 ceph-mon[43495]: pgmap v80: 97 pgs: 97 active+clean; 35 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 621 KiB/s wr, 119 op/s 2024-03-18T20:38:40.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64660 3.85 MB/sec warmup 95 sec latency 3595.060 ms 2024-03-18T20:38:41.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:38:41.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:38:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.285+0000 7f5cc8a10700 1 -- 172.21.15.46:0/1035222269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cc4069c30 msgr2=0x7f5cc4105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.285+0000 7f5cc8a10700 1 --2- 172.21.15.46:0/1035222269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cc4069c30 0x7f5cc4105980 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0099c0 tx=0x7f5cac009cd0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.285+0000 7f5cc8a10700 1 -- 172.21.15.46:0/1035222269 shutdown_connections 2024-03-18T20:38:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.285+0000 7f5cc8a10700 1 --2- 172.21.15.46:0/1035222269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cc41060b0 0x7f5cc410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.285+0000 7f5cc8a10700 1 --2- 172.21.15.46:0/1035222269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cc4069280 0x7f5cc4069660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.285+0000 7f5cc8a10700 1 --2- 172.21.15.46:0/1035222269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cc4069c30 0x7f5cc4105980 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.285+0000 7f5cc8a10700 1 -- 172.21.15.46:0/1035222269 >> 172.21.15.46:0/1035222269 conn(0x7f5cc4076d80 msgr2=0x7f5cc4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.285+0000 7f5cc8a10700 1 -- 172.21.15.46:0/1035222269 shutdown_connections 2024-03-18T20:38:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.285+0000 7f5cc8a10700 1 -- 172.21.15.46:0/1035222269 wait complete. 2024-03-18T20:38:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.286+0000 7f5cc8a10700 1 Processor -- start 2024-03-18T20:38:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.286+0000 7f5cc8a10700 1 -- start start 2024-03-18T20:38:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.286+0000 7f5cc8a10700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cc4069280 0x7f5cc41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.286+0000 7f5cc8a10700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cc41060b0 0x7f5cc4197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.286+0000 7f5cc8a10700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cc419ba80 0x7f5cc419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.286+0000 7f5cc8a10700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc419c500 con 0x7f5cc41060b0 2024-03-18T20:38:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.286+0000 7f5cc8a10700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc419c670 con 0x7f5cc4069280 2024-03-18T20:38:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.286+0000 7f5cc8a10700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc419c830 con 0x7f5cc419ba80 2024-03-18T20:38:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cc41060b0 0x7f5cc4197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cc41060b0 0x7f5cc4197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33984/0 (socket says 172.21.15.46:33984) 2024-03-18T20:38:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc1d9b700 1 -- 172.21.15.46:0/430915507 learned_addr learned my addr 172.21.15.46:0/430915507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:41.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc259c700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cc4069280 0x7f5cc41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc2d9d700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cc419ba80 0x7f5cc419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc1d9b700 1 -- 172.21.15.46:0/430915507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cc419ba80 msgr2=0x7f5cc419bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cc419ba80 0x7f5cc419bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc1d9b700 1 -- 172.21.15.46:0/430915507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cc4069280 msgr2=0x7f5cc41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cc4069280 0x7f5cc41971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc1d9b700 1 -- 172.21.15.46:0/430915507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac0095f0 con 0x7f5cc41060b0 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc259c700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cc4069280 0x7f5cc41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cc41060b0 0x7f5cc4197730 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5cac006010 tx=0x7f5cac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cbb7fe700 1 -- 172.21.15.46:0/430915507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac01d070 con 0x7f5cc41060b0 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.287+0000 7f5cc8a10700 1 -- 172.21.15.46:0/430915507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc41a3d30 con 0x7f5cc41060b0 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.288+0000 7f5cc8a10700 1 -- 172.21.15.46:0/430915507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc41a41b0 con 0x7f5cc41060b0 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.288+0000 7f5cbb7fe700 1 -- 172.21.15.46:0/430915507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cac004ca0 con 0x7f5cc41060b0 2024-03-18T20:38:41.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.288+0000 7f5cbb7fe700 1 -- 172.21.15.46:0/430915507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00f8d0 con 0x7f5cc41060b0 2024-03-18T20:38:41.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.291+0000 7f5cbb7fe700 1 -- 172.21.15.46:0/430915507 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5cac0047e0 con 0x7f5cc41060b0 2024-03-18T20:38:41.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.291+0000 7f5cbb7fe700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5cb00a1290 0x7f5cb00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.291+0000 7f5cbb7fe700 1 -- 172.21.15.46:0/430915507 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5cac0cd930 con 0x7f5cc41060b0 2024-03-18T20:38:41.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.291+0000 7f5cc259c700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5cb00a1290 0x7f5cb00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.292+0000 7f5cc259c700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5cb00a1290 0x7f5cb00a3750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4007fd0 tx=0x7f5cb40073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.292+0000 7f5cd4ba5b80 1 Processor -- start 2024-03-18T20:38:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.292+0000 7f5cd4ba5b80 1 -- start start 2024-03-18T20:38:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.292+0000 7f5cd4ba5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a443a60 0x55f26a443e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.292+0000 7f5cd4ba5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a444380 0x55f26a4530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.292+0000 7f5cd4ba5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a453630 0x55f26a455a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.292+0000 7f5cd4ba5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26a39d990 con 0x55f26a453630 2024-03-18T20:38:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.292+0000 7f5cd4ba5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26a39b350 con 0x55f26a443a60 2024-03-18T20:38:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.292+0000 7f5cd4ba5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26a396c40 con 0x55f26a444380 2024-03-18T20:38:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cc2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a453630 0x55f26a455a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cc2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a453630 0x55f26a455a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33994/0 (socket says 172.21.15.46:33994) 2024-03-18T20:38:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cc2d9d700 1 -- 172.21.15.46:0/662916385 learned_addr learned my addr 172.21.15.46:0/662916385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/662916385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a444380 0x55f26a4530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cc259c700 1 --2- 172.21.15.46:0/662916385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a443a60 0x55f26a443e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cc2d9d700 1 -- 172.21.15.46:0/662916385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a444380 msgr2=0x55f26a4530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cc2d9d700 1 --2- 172.21.15.46:0/662916385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a444380 0x55f26a4530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cc2d9d700 1 -- 172.21.15.46:0/662916385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a443a60 msgr2=0x55f26a443e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cc2d9d700 1 --2- 172.21.15.46:0/662916385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a443a60 0x55f26a443e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cc2d9d700 1 -- 172.21.15.46:0/662916385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f26a2183b0 con 0x55f26a453630 2024-03-18T20:38:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/662916385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a444380 0x55f26a4530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cc2d9d700 1 --2- 172.21.15.46:0/662916385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a453630 0x55f26a455a20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc003fa0 tx=0x7f5cbc009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f384c4bad5105afb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cb8ff9700 1 -- 172.21.15.46:0/662916385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc019070 con 0x55f26a453630 2024-03-18T20:38:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cb8ff9700 1 -- 172.21.15.46:0/662916385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cbc00f040 con 0x55f26a453630 2024-03-18T20:38:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.293+0000 7f5cb8ff9700 1 -- 172.21.15.46:0/662916385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0079c0 con 0x55f26a453630 2024-03-18T20:38:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.294+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/662916385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a453630 msgr2=0x55f26a455a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.294+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/662916385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a453630 0x55f26a455a20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc003fa0 tx=0x7f5cbc009ba0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.294+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/662916385 shutdown_connections 2024-03-18T20:38:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.294+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/662916385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a453630 0x55f26a455a20 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.294+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/662916385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a443a60 0x55f26a443e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.294+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/662916385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a444380 0x55f26a4530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.294+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/662916385 >> 172.21.15.46:0/662916385 conn(0x55f26a3ba220 msgr2=0x55f26a440020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.294+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/662916385 shutdown_connections 2024-03-18T20:38:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.294+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/662916385 wait complete. 2024-03-18T20:38:41.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cd4ba5b80 1 -- start start 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cd4ba5b80 1 Processor -- start 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cd4ba5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a443a60 0x55f26a453010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cd4ba5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a444380 0x55f26a450300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cd4ba5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a453630 0x55f26a450960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cd4ba5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26a332d80 con 0x55f26a444380 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cd4ba5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26a332a20 con 0x55f26a453630 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cd4ba5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26a21a6a0 con 0x55f26a443a60 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cd4ba5b80 10 client.0 _ll_register_callbacks cb 0x55f26a19e3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cc1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a444380 0x55f26a450300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cc1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a444380 0x55f26a450300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34004/0 (socket says 172.21.15.46:34004) 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cc1d9b700 1 -- 172.21.15.46:0/4028343880 learned_addr learned my addr 172.21.15.46:0/4028343880 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cc259c700 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a443a60 0x55f26a453010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cc2d9d700 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a453630 0x55f26a450960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cc1d9b700 1 -- 172.21.15.46:0/4028343880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a443a60 msgr2=0x55f26a453010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a443a60 0x55f26a453010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cc1d9b700 1 -- 172.21.15.46:0/4028343880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a453630 msgr2=0x55f26a450960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.295+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a453630 0x55f26a450960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5cc1d9b700 1 -- 172.21.15.46:0/4028343880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc003bd0 con 0x55f26a444380 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a444380 0x55f26a450300 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5cac096fa0 tx=0x7f5cac09b220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168920482f28c73a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5ca9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5ca9ffb700 1 -- 172.21.15.46:0/4028343880 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac09a190 con 0x55f26a444380 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5ca9ffb700 1 -- 172.21.15.46:0/4028343880 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5cac09e8d0 con 0x55f26a444380 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/4028343880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f26a437ad0 con 0x55f26a444380 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5ca9ffb700 1 -- 172.21.15.46:0/4028343880 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cac0a2e90 con 0x55f26a444380 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5cd4ba5b80 10 client.16374 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/4028343880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f26a1c4b60 con 0x55f26a444380 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5ca9ffb700 1 -- 172.21.15.46:0/4028343880 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f26a437ad0 con 0x55f26a444380 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5ca9ffb700 1 -- 172.21.15.46:0/4028343880 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5cac0a2b50 con 0x55f26a444380 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5cd4ba5b80 10 client.16374 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.296+0000 7f5cd4ba5b80 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.297+0000 7f5cd4ba5b80 20 client.16374 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.297+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f26a462d90 0x55f26a4651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.297+0000 7f5cd4ba5b80 4 client.16374 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.297+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/4028343880 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f26a38f700 con 0x55f26a462d90 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.297+0000 7f5cc259c700 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f26a462d90 0x55f26a4651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.297+0000 7f5cc259c700 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f26a462d90 0x55f26a4651b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4356e224ecd757b8 server_cookie=2d4c5113258ee78 in_seq=0 out_seq=0 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.297+0000 7f5ca9ffb700 10 client.16374 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5ca9ffb700 1 -- 172.21.15.46:0/4028343880 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f26a38f700 con 0x55f26a462d90 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5ca9ffb700 10 client.16374 handle_command_reply: tid=0 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5cd4ba5b80 1 client.16374 shutdown 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/4028343880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a444380 msgr2=0x55f26a450300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a444380 0x55f26a450300 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5cac096fa0 tx=0x7f5cac09b220 comp rx=0 tx=0).stop 2024-03-18T20:38:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/4028343880 shutdown_connections 2024-03-18T20:38:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f26a462d90 0x55f26a4651b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a453630 0x55f26a450960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a443a60 0x55f26a453010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/4028343880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a444380 0x55f26a450300 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5ca9ffb700 0 client.16374 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/4028343880 >> 172.21.15.46:0/4028343880 conn(0x55f26a3ba220 msgr2=0x55f26a43fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/4028343880 shutdown_connections 2024-03-18T20:38:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/4028343880 wait complete. 2024-03-18T20:38:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.298+0000 7f5cd4ba5b80 20 client.16374 trim_cache size 0 max 16384 2024-03-18T20:38:41.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.316+0000 7f5cd4ba5b80 1 Processor -- start 2024-03-18T20:38:41.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.316+0000 7f5cd4ba5b80 1 -- start start 2024-03-18T20:38:41.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.316+0000 7f5cd4ba5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 0x55f26a44fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.316+0000 7f5cd4ba5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a444380 0x55f26a45e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.316+0000 7f5cd4ba5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a453630 0x55f26a45eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.316+0000 7f5cd4ba5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26a4405a0 con 0x55f26a443a60 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.316+0000 7f5cd4ba5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26a43a210 con 0x55f26a444380 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.316+0000 7f5cd4ba5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26a444d30 con 0x55f26a453630 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.316+0000 7f5cc259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 0x55f26a44fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cc259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 0x55f26a44fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34020/0 (socket says 172.21.15.46:34020) 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cc259c700 1 -- 172.21.15.46:0/628195664 learned_addr learned my addr 172.21.15.46:0/628195664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cc259c700 1 -- 172.21.15.46:0/628195664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a453630 msgr2=0x55f26a45eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cc2d9d700 1 --2- 172.21.15.46:0/628195664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a453630 0x55f26a45eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/628195664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a444380 0x55f26a45e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cc259c700 1 --2- 172.21.15.46:0/628195664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a453630 0x55f26a45eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cc259c700 1 -- 172.21.15.46:0/628195664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a444380 msgr2=0x55f26a45e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cc259c700 1 --2- 172.21.15.46:0/628195664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a444380 0x55f26a45e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cc259c700 1 -- 172.21.15.46:0/628195664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f26a35cea0 con 0x55f26a443a60 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cc2d9d700 1 --2- 172.21.15.46:0/628195664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a453630 0x55f26a45eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:38:41.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/628195664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a444380 0x55f26a45e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cc259c700 1 --2- 172.21.15.46:0/628195664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 0x55f26a44fc20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x55f26a465ac0 tx=0x7f5cb4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474f1b5517dbb550 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cb97fa700 1 -- 172.21.15.46:0/628195664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4026040 con 0x55f26a443a60 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cb97fa700 1 -- 172.21.15.46:0/628195664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cb4003df0 con 0x55f26a443a60 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.317+0000 7f5cb97fa700 1 -- 172.21.15.46:0/628195664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40116a0 con 0x55f26a443a60 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.318+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/628195664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 msgr2=0x55f26a44fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.318+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/628195664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 0x55f26a44fc20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x55f26a465ac0 tx=0x7f5cb4016000 comp rx=0 tx=0).stop 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.318+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/628195664 shutdown_connections 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.318+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/628195664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 0x55f26a44fc20 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.318+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/628195664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a453630 0x55f26a45eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.318+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/628195664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a444380 0x55f26a45e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.318+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/628195664 >> 172.21.15.46:0/628195664 conn(0x55f26a3ba220 msgr2=0x55f26a44b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.318+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/628195664 shutdown_connections 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.318+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/628195664 wait complete. 2024-03-18T20:38:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.318+0000 7f5cd4ba5b80 1 -- start start 2024-03-18T20:38:41.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.318+0000 7f5cd4ba5b80 1 Processor -- start 2024-03-18T20:38:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cd4ba5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 0x55f26a45a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cd4ba5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a444380 0x55f26a458860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cd4ba5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a453630 0x55f26a458da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cd4ba5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26a459460 con 0x55f26a443a60 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cd4ba5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26a4595d0 con 0x55f26a444380 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cd4ba5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f26a4b6bd0 con 0x55f26a453630 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cd4ba5b80 10 client.0 _ll_register_callbacks cb 0x7f5cac09a5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cc259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 0x55f26a45a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cc259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 0x55f26a45a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34034/0 (socket says 172.21.15.46:34034) 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cc259c700 1 -- 172.21.15.46:0/3408125690 learned_addr learned my addr 172.21.15.46:0/3408125690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a444380 0x55f26a458860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cc259c700 1 -- 172.21.15.46:0/3408125690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a453630 msgr2=0x55f26a458da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cc2d9d700 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a453630 0x55f26a458da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cc259c700 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a453630 0x55f26a458da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cc259c700 1 -- 172.21.15.46:0/3408125690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a444380 msgr2=0x55f26a458860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cc259c700 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a444380 0x55f26a458860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cc259c700 1 -- 172.21.15.46:0/3408125690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb4004450 con 0x55f26a443a60 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a444380 0x55f26a458860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cc259c700 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 0x55f26a45a290 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4003ac0 tx=0x7f5cb4028300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc918a18fbfc8e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5ca9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5ca9ffb700 1 -- 172.21.15.46:0/3408125690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4026050 con 0x55f26a443a60 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5ca9ffb700 1 -- 172.21.15.46:0/3408125690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5cb4017070 con 0x55f26a443a60 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.319+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/3408125690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5cbc00f040 con 0x55f26a443a60 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.320+0000 7f5ca9ffb700 1 -- 172.21.15.46:0/3408125690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cb4028880 con 0x55f26a443a60 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.320+0000 7f5cd4ba5b80 10 client.16380 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.320+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/3408125690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f26a21ccd0 con 0x55f26a443a60 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.320+0000 7f5ca9ffb700 1 -- 172.21.15.46:0/3408125690 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cbc00f040 con 0x55f26a443a60 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.320+0000 7f5ca9ffb700 1 -- 172.21.15.46:0/3408125690 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5cb4011b40 con 0x55f26a443a60 2024-03-18T20:38:41.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.320+0000 7f5cd4ba5b80 10 client.16380 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.320+0000 7f5cd4ba5b80 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.320+0000 7f5cd4ba5b80 20 client.16380 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.320+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f26a4b82e0 0x55f26a4ba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.320+0000 7f5cd4ba5b80 4 client.16380 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.320+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/3408125690 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f26a1e5cf0 con 0x55f26a4b82e0 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.320+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f26a4b82e0 0x55f26a4ba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cc1d9b700 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f26a4b82e0 0x55f26a4ba700 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=412398900b5d130a server_cookie=eb4f94723dfb0275 in_seq=0 out_seq=0 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5ca9ffb700 10 client.16380 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5ca9ffb700 1 -- 172.21.15.46:0/3408125690 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f26a1e5cf0 con 0x55f26a4b82e0 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5ca9ffb700 10 client.16380 handle_command_reply: tid=0 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cd4ba5b80 1 client.16380 shutdown 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/3408125690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 msgr2=0x55f26a45a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 0x55f26a45a290 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4003ac0 tx=0x7f5cb4028300 comp rx=0 tx=0).stop 2024-03-18T20:38:41.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/3408125690 shutdown_connections 2024-03-18T20:38:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f26a4b82e0 0x55f26a4ba700 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f26a443a60 0x55f26a45a290 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f26a453630 0x55f26a458da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5ca9ffb700 0 client.16380 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cd4ba5b80 1 --2- 172.21.15.46:0/3408125690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f26a444380 0x55f26a458860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/3408125690 >> 172.21.15.46:0/3408125690 conn(0x55f26a3ba220 msgr2=0x55f26a45e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/3408125690 shutdown_connections 2024-03-18T20:38:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cd4ba5b80 1 -- 172.21.15.46:0/3408125690 wait complete. 2024-03-18T20:38:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.321+0000 7f5cd4ba5b80 20 client.16380 trim_cache size 0 max 16384 2024-03-18T20:38:41.331 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:38:41.331 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:38:41.332 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:38:41.332 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:38:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.322+0000 7f5cc8a10700 1 -- 172.21.15.46:0/430915507 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5cb00a1290 msgr2=0x7f5cb00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.322+0000 7f5cc8a10700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5cb00a1290 0x7f5cb00a3750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4007fd0 tx=0x7f5cb40073d0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.322+0000 7f5cc8a10700 1 -- 172.21.15.46:0/430915507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cc41060b0 msgr2=0x7f5cc4197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.322+0000 7f5cc8a10700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cc41060b0 0x7f5cc4197730 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5cac006010 tx=0x7f5cac004730 comp rx=0 tx=0).stop 2024-03-18T20:38:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.322+0000 7f5cc8a10700 1 -- 172.21.15.46:0/430915507 shutdown_connections 2024-03-18T20:38:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.322+0000 7f5cc8a10700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5cb00a1290 0x7f5cb00a3750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.322+0000 7f5cc8a10700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5cc419ba80 0x7f5cc419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.322+0000 7f5cc8a10700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5cc4069280 0x7f5cc41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.322+0000 7f5cc8a10700 1 --2- 172.21.15.46:0/430915507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cc41060b0 0x7f5cc4197730 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.322+0000 7f5cc8a10700 1 -- 172.21.15.46:0/430915507 >> 172.21.15.46:0/430915507 conn(0x7f5cc4076d80 msgr2=0x7f5cc40fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.322+0000 7f5cc8a10700 1 -- 172.21.15.46:0/430915507 shutdown_connections 2024-03-18T20:38:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.322+0000 7f5cc8a10700 1 -- 172.21.15.46:0/430915507 wait complete. 2024-03-18T20:38:41.338 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:38:41.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:38:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.472+0000 7fe55f43d700 1 -- 172.21.15.46:0/1332701071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5581004c0 msgr2=0x7fe55810c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.472+0000 7fe55f43d700 1 --2- 172.21.15.46:0/1332701071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5581004c0 0x7fe55810c9d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe55000b0a0 tx=0x7fe55000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.472+0000 7fe55f43d700 1 -- 172.21.15.46:0/1332701071 shutdown_connections 2024-03-18T20:38:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.472+0000 7fe55f43d700 1 --2- 172.21.15.46:0/1332701071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5581004c0 0x7fe55810c9d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.472+0000 7fe55f43d700 1 --2- 172.21.15.46:0/1332701071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe558105ba0 0x7fe558105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.472+0000 7fe55f43d700 1 --2- 172.21.15.46:0/1332701071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5580ffb00 0x7fe5580fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.472+0000 7fe55f43d700 1 -- 172.21.15.46:0/1332701071 >> 172.21.15.46:0/1332701071 conn(0x7fe5580fb630 msgr2=0x7fe5580fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.472+0000 7fe55f43d700 1 -- 172.21.15.46:0/1332701071 shutdown_connections 2024-03-18T20:38:41.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.472+0000 7fe55f43d700 1 -- 172.21.15.46:0/1332701071 wait complete. 2024-03-18T20:38:41.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.472+0000 7fe55f43d700 1 Processor -- start 2024-03-18T20:38:41.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.472+0000 7fe55f43d700 1 -- start start 2024-03-18T20:38:41.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55f43d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5580ffb00 0x7fe5581991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55f43d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5581004c0 0x7fe558199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55f43d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe558105ba0 0x7fe55819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55f43d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55819e100 con 0x7fe5581004c0 2024-03-18T20:38:41.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55f43d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55819e270 con 0x7fe5580ffb00 2024-03-18T20:38:41.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55f43d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55819e430 con 0x7fe558105ba0 2024-03-18T20:38:41.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55c9d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5581004c0 0x7fe558199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55c9d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5581004c0 0x7fe558199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34046/0 (socket says 172.21.15.46:34046) 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55d1d9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5580ffb00 0x7fe5581991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55c9d8700 1 -- 172.21.15.46:0/3407128046 learned_addr learned my addr 172.21.15.46:0/3407128046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55d9da700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe558105ba0 0x7fe55819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55c9d8700 1 -- 172.21.15.46:0/3407128046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe558105ba0 msgr2=0x7fe55819dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55c9d8700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe558105ba0 0x7fe55819dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55c9d8700 1 -- 172.21.15.46:0/3407128046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5580ffb00 msgr2=0x7fe5581991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55c9d8700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5580ffb00 0x7fe5581991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55c9d8700 1 -- 172.21.15.46:0/3407128046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe548009520 con 0x7fe5581004c0 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55d1d9700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5580ffb00 0x7fe5581991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe55c9d8700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5581004c0 0x7fe558199720 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe54800eb30 tx=0x7fe54800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.473+0000 7fe54e7fc700 1 -- 172.21.15.46:0/3407128046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548009880 con 0x7fe5581004c0 2024-03-18T20:38:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.474+0000 7fe54e7fc700 1 -- 172.21.15.46:0/3407128046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5480044e0 con 0x7fe5581004c0 2024-03-18T20:38:41.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.474+0000 7fe55f43d700 1 -- 172.21.15.46:0/3407128046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe550009d00 con 0x7fe5581004c0 2024-03-18T20:38:41.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.474+0000 7fe54e7fc700 1 -- 172.21.15.46:0/3407128046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548003d80 con 0x7fe5581004c0 2024-03-18T20:38:41.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.474+0000 7fe55f43d700 1 -- 172.21.15.46:0/3407128046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5581a5d70 con 0x7fe5581004c0 2024-03-18T20:38:41.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.475+0000 7fe55f43d700 1 -- 172.21.15.46:0/3407128046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe558101200 con 0x7fe5581004c0 2024-03-18T20:38:41.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.476+0000 7fe54e7fc700 1 -- 172.21.15.46:0/3407128046 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe548003f00 con 0x7fe5581004c0 2024-03-18T20:38:41.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.476+0000 7fe54e7fc700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe5440a1290 0x7fe5440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.477+0000 7fe54e7fc700 1 -- 172.21.15.46:0/3407128046 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fe548014070 con 0x7fe5581004c0 2024-03-18T20:38:41.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.477+0000 7fe55d1d9700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe5440a1290 0x7fe5440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.477+0000 7fe55d1d9700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe5440a1290 0x7fe5440a3750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe554009fd0 tx=0x7fe554009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.480+0000 7fe54e7fc700 1 -- 172.21.15.46:0/3407128046 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe5480967e0 con 0x7fe5581004c0 2024-03-18T20:38:41.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 64660 3.81 MB/sec warmup 96 sec latency 4595.117 ms 2024-03-18T20:38:41.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.746+0000 7fe55f43d700 1 -- 172.21.15.46:0/3407128046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe558067180 con 0x7fe5581004c0 2024-03-18T20:38:41.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.747+0000 7fe54e7fc700 1 -- 172.21.15.46:0/3407128046 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10919 (secure 0 0 0) 0x7fe548099e00 con 0x7fe5581004c0 2024-03-18T20:38:41.750 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:38:41.750 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:37:47.071553+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:38:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.749+0000 7fe55f43d700 1 -- 172.21.15.46:0/3407128046 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe5440a1290 msgr2=0x7fe5440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.749+0000 7fe55f43d700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe5440a1290 0x7fe5440a3750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe554009fd0 tx=0x7fe554009260 comp rx=0 tx=0).stop 2024-03-18T20:38:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.749+0000 7fe55f43d700 1 -- 172.21.15.46:0/3407128046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5581004c0 msgr2=0x7fe558199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.749+0000 7fe55f43d700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5581004c0 0x7fe558199720 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe54800eb30 tx=0x7fe54800ee40 comp rx=0 tx=0).stop 2024-03-18T20:38:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.749+0000 7fe55f43d700 1 -- 172.21.15.46:0/3407128046 shutdown_connections 2024-03-18T20:38:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.749+0000 7fe55f43d700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe5440a1290 0x7fe5440a3750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.749+0000 7fe55f43d700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe558105ba0 0x7fe55819dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.749+0000 7fe55f43d700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5580ffb00 0x7fe5581991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.749+0000 7fe55f43d700 1 --2- 172.21.15.46:0/3407128046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5581004c0 0x7fe558199720 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.749+0000 7fe55f43d700 1 -- 172.21.15.46:0/3407128046 >> 172.21.15.46:0/3407128046 conn(0x7fe5580fb630 msgr2=0x7fe558069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.749+0000 7fe55f43d700 1 -- 172.21.15.46:0/3407128046 shutdown_connections 2024-03-18T20:38:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.749+0000 7fe55f43d700 1 -- 172.21.15.46:0/3407128046 wait complete. 2024-03-18T20:38:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 33 2024-03-18T20:38:41.766 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:38:41.766 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:38:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.903+0000 7f2f63b3d700 1 -- 172.21.15.46:0/1713684832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f5c100f50 msgr2=0x7f2f5c101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.903+0000 7f2f63b3d700 1 --2- 172.21.15.46:0/1713684832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f5c100f50 0x7f2f5c101330 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009990 tx=0x7f2f58009ca0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.903+0000 7f2f63b3d700 1 -- 172.21.15.46:0/1713684832 shutdown_connections 2024-03-18T20:38:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.903+0000 7f2f63b3d700 1 --2- 172.21.15.46:0/1713684832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f5c100f50 0x7f2f5c101330 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.903+0000 7f2f63b3d700 1 --2- 172.21.15.46:0/1713684832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f5c106070 0x7f2f5c10dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.903+0000 7f2f63b3d700 1 --2- 172.21.15.46:0/1713684832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f5c101900 0x7f2f5c105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.903+0000 7f2f63b3d700 1 -- 172.21.15.46:0/1713684832 >> 172.21.15.46:0/1713684832 conn(0x7f2f5c0fc7f0 msgr2=0x7f2f5c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.903+0000 7f2f63b3d700 1 -- 172.21.15.46:0/1713684832 shutdown_connections 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.903+0000 7f2f63b3d700 1 -- 172.21.15.46:0/1713684832 wait complete. 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.904+0000 7f2f63b3d700 1 Processor -- start 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.904+0000 7f2f63b3d700 1 -- start start 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.904+0000 7f2f63b3d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f5c100f50 0x7f2f5c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.904+0000 7f2f63b3d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f5c106070 0x7f2f5c197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.904+0000 7f2f63b3d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f5c19bb10 0x7f2f5c19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.904+0000 7f2f63b3d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c19c590 con 0x7f2f5c106070 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.904+0000 7f2f63b3d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c19c700 con 0x7f2f5c100f50 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.904+0000 7f2f63b3d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c19c8c0 con 0x7f2f5c19bb10 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.904+0000 7f2f610d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f5c106070 0x7f2f5c197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.904+0000 7f2f610d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f5c106070 0x7f2f5c197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34058/0 (socket says 172.21.15.46:34058) 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.904+0000 7f2f610d8700 1 -- 172.21.15.46:0/3916266053 learned_addr learned my addr 172.21.15.46:0/3916266053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:41.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.905+0000 7f2f618d9700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f5c100f50 0x7f2f5c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.905+0000 7f2f610d8700 1 -- 172.21.15.46:0/3916266053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f5c19bb10 msgr2=0x7f2f5c19bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.905+0000 7f2f610d8700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f5c19bb10 0x7f2f5c19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.905+0000 7f2f610d8700 1 -- 172.21.15.46:0/3916266053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f5c100f50 msgr2=0x7f2f5c197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.905+0000 7f2f610d8700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f5c100f50 0x7f2f5c197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.905+0000 7f2f610d8700 1 -- 172.21.15.46:0/3916266053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f580095f0 con 0x7f2f5c106070 2024-03-18T20:38:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.905+0000 7f2f618d9700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f5c100f50 0x7f2f5c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.905+0000 7f2f610d8700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f5c106070 0x7f2f5c197770 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00e970 tx=0x7f2f4c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.905+0000 7f2f52ffd700 1 -- 172.21.15.46:0/3916266053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c009790 con 0x7f2f5c106070 2024-03-18T20:38:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.905+0000 7f2f63b3d700 1 -- 172.21.15.46:0/3916266053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5c1a3e20 con 0x7f2f5c106070 2024-03-18T20:38:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.905+0000 7f2f63b3d700 1 -- 172.21.15.46:0/3916266053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5c1a42f0 con 0x7f2f5c106070 2024-03-18T20:38:41.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.906+0000 7f2f52ffd700 1 -- 172.21.15.46:0/3916266053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f4c00cb50 con 0x7f2f5c106070 2024-03-18T20:38:41.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.906+0000 7f2f52ffd700 1 -- 172.21.15.46:0/3916266053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c010690 con 0x7f2f5c106070 2024-03-18T20:38:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.908+0000 7f2f52ffd700 1 -- 172.21.15.46:0/3916266053 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2f4c0107f0 con 0x7f2f5c106070 2024-03-18T20:38:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.909+0000 7f2f52ffd700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2f480a1290 0x7f2f480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.909+0000 7f2f618d9700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2f480a1290 0x7f2f480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.909+0000 7f2f52ffd700 1 -- 172.21.15.46:0/3916266053 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2f4c014070 con 0x7f2f5c106070 2024-03-18T20:38:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.909+0000 7f2f6fcd2b80 1 Processor -- start 2024-03-18T20:38:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.909+0000 7f2f6fcd2b80 1 -- start start 2024-03-18T20:38:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f618d9700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2f480a1290 0x7f2f480a3750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800b2e0 tx=0x7f2f58005940 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f6fcd2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 0x563f38faf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f6fcd2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faf790 0x563f38fba100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f6fcd2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 0x563f38fbca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f6fcd2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f38f04990 con 0x563f38faee70 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f6fcd2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f38f02350 con 0x563f38faf790 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f6fcd2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f38efdc40 con 0x563f38fba640 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f618d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 0x563f38faf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f618d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 0x563f38faf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34064/0 (socket says 172.21.15.46:34064) 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f618d9700 1 -- 172.21.15.46:0/3040847019 learned_addr learned my addr 172.21.15.46:0/3040847019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f610d8700 1 --2- 172.21.15.46:0/3040847019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faf790 0x563f38fba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f620da700 1 --2- 172.21.15.46:0/3040847019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 0x563f38fbca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f618d9700 1 -- 172.21.15.46:0/3040847019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 msgr2=0x563f38fbca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f618d9700 1 --2- 172.21.15.46:0/3040847019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 0x563f38fbca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f618d9700 1 -- 172.21.15.46:0/3040847019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faf790 msgr2=0x563f38fba100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f618d9700 1 --2- 172.21.15.46:0/3040847019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faf790 0x563f38fba100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f618d9700 1 -- 172.21.15.46:0/3040847019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f38d7f3b0 con 0x563f38faee70 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.910+0000 7f2f618d9700 1 --2- 172.21.15.46:0/3040847019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 0x563f38faf250 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2f58015000 tx=0x7f2f5801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5fc8fb578c8fcc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.911+0000 7f2f47fff700 1 -- 172.21.15.46:0/3040847019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5802b070 con 0x563f38faee70 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.911+0000 7f2f47fff700 1 -- 172.21.15.46:0/3040847019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f5801e040 con 0x563f38faee70 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.911+0000 7f2f47fff700 1 -- 172.21.15.46:0/3040847019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58004de0 con 0x563f38faee70 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.911+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/3040847019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 msgr2=0x563f38faf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.911+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/3040847019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 0x563f38faf250 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2f58015000 tx=0x7f2f5801f000 comp rx=0 tx=0).stop 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.911+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/3040847019 shutdown_connections 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.911+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/3040847019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 0x563f38faf250 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.911+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/3040847019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 0x563f38fbca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.911+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/3040847019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faf790 0x563f38fba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.911+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/3040847019 >> 172.21.15.46:0/3040847019 conn(0x563f38f21220 msgr2=0x563f38fa6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.912+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/3040847019 shutdown_connections 2024-03-18T20:38:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.912+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/3040847019 wait complete. 2024-03-18T20:38:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.912+0000 7f2f6fcd2b80 1 -- start start 2024-03-18T20:38:41.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.912+0000 7f2f6fcd2b80 1 Processor -- start 2024-03-18T20:38:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f6fcd2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38faee70 0x563f38fba010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f6fcd2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 0x563f38fb7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f6fcd2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38fba640 0x563f38fb7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f6fcd2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f38e99d80 con 0x563f38faf790 2024-03-18T20:38:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f6fcd2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f38e99a20 con 0x563f38fba640 2024-03-18T20:38:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f6fcd2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f38d816a0 con 0x563f38faee70 2024-03-18T20:38:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f6fcd2b80 10 client.0 _ll_register_callbacks cb 0x563f38d053a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f610d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 0x563f38fb7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f610d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 0x563f38fb7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34072/0 (socket says 172.21.15.46:34072) 2024-03-18T20:38:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f610d8700 1 -- 172.21.15.46:0/2632047833 learned_addr learned my addr 172.21.15.46:0/2632047833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f618d9700 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38faee70 0x563f38fba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f620da700 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38fba640 0x563f38fb7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f610d8700 1 -- 172.21.15.46:0/2632047833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38faee70 msgr2=0x563f38fba010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f610d8700 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38faee70 0x563f38fba010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f610d8700 1 -- 172.21.15.46:0/2632047833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38fba640 msgr2=0x563f38fb7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f610d8700 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38fba640 0x563f38fb7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f610d8700 1 -- 172.21.15.46:0/2632047833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5c10e460 con 0x563f38faf790 2024-03-18T20:38:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.913+0000 7f2f610d8700 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 0x563f38fb7300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00c5b0 tx=0x7f2f4c09a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45be37e97f99f422 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f457fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f457fa700 1 -- 172.21.15.46:0/2632047833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0adea0 con 0x563f38faf790 2024-03-18T20:38:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f457fa700 1 -- 172.21.15.46:0/2632047833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2f4c0a2450 con 0x563f38faf790 2024-03-18T20:38:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/2632047833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f38f9ead0 con 0x563f38faf790 2024-03-18T20:38:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f457fa700 1 -- 172.21.15.46:0/2632047833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f4c094de0 con 0x563f38faf790 2024-03-18T20:38:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f6fcd2b80 10 client.16398 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/2632047833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563f38d2bb60 con 0x563f38faf790 2024-03-18T20:38:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f457fa700 1 -- 172.21.15.46:0/2632047833 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f38f9ead0 con 0x563f38faf790 2024-03-18T20:38:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f457fa700 1 -- 172.21.15.46:0/2632047833 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2f4c0995e0 con 0x563f38faf790 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f6fcd2b80 10 client.16398 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f6fcd2b80 10 client.16398 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f6fcd2b80 20 client.16398 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563f38fca230 0x563f38fcc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f6fcd2b80 4 client.16398 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.914+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/2632047833 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f38ef6700 con 0x563f38fca230 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.915+0000 7f2f618d9700 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563f38fca230 0x563f38fcc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.915+0000 7f2f618d9700 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563f38fca230 0x563f38fcc650 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6976c8da451f5170 server_cookie=645ab059ea76758d in_seq=0 out_seq=0 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.915+0000 7f2f457fa700 10 client.16398 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f457fa700 1 -- 172.21.15.46:0/2632047833 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563f38ef6700 con 0x563f38fca230 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f457fa700 10 client.16398 handle_command_reply: tid=0 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f6fcd2b80 1 client.16398 shutdown 2024-03-18T20:38:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/2632047833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 msgr2=0x563f38fb7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 0x563f38fb7300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00c5b0 tx=0x7f2f4c09a7c0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/2632047833 shutdown_connections 2024-03-18T20:38:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563f38fca230 0x563f38fcc650 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38fba640 0x563f38fb7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38faee70 0x563f38fba010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/2632047833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 0x563f38fb7300 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f457fa700 0 client.16398 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:38:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/2632047833 >> 172.21.15.46:0/2632047833 conn(0x563f38f21220 msgr2=0x563f38fa6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/2632047833 shutdown_connections 2024-03-18T20:38:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/2632047833 wait complete. 2024-03-18T20:38:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.916+0000 7f2f6fcd2b80 20 client.16398 trim_cache size 0 max 16384 2024-03-18T20:38:41.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f6fcd2b80 1 Processor -- start 2024-03-18T20:38:41.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f6fcd2b80 1 -- start start 2024-03-18T20:38:41.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f6fcd2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 0x563f38fcc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f6fcd2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faf790 0x563f38fb5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f6fcd2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 0x563f38fb6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f6fcd2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f38fa0f80 con 0x563f38faee70 2024-03-18T20:38:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f6fcd2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f38fafe60 con 0x563f38faf790 2024-03-18T20:38:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f6fcd2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f38fbd090 con 0x563f38fba640 2024-03-18T20:38:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f618d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 0x563f38fcc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f618d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 0x563f38fcc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34088/0 (socket says 172.21.15.46:34088) 2024-03-18T20:38:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f618d9700 1 -- 172.21.15.46:0/270142305 learned_addr learned my addr 172.21.15.46:0/270142305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f620da700 1 --2- 172.21.15.46:0/270142305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 0x563f38fb6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.930+0000 7f2f610d8700 1 --2- 172.21.15.46:0/270142305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faf790 0x563f38fb5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.931+0000 7f2f618d9700 1 -- 172.21.15.46:0/270142305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 msgr2=0x563f38fb6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.931+0000 7f2f618d9700 1 --2- 172.21.15.46:0/270142305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 0x563f38fb6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.931+0000 7f2f618d9700 1 -- 172.21.15.46:0/270142305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faf790 msgr2=0x563f38fb5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.931+0000 7f2f618d9700 1 --2- 172.21.15.46:0/270142305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faf790 0x563f38fb5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.931+0000 7f2f618d9700 1 -- 172.21.15.46:0/270142305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f38ec3ea0 con 0x563f38faee70 2024-03-18T20:38:41.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.931+0000 7f2f618d9700 1 --2- 172.21.15.46:0/270142305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 0x563f38fcc620 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2f58000c00 tx=0x7f2f58028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3336ec57e90d0c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.931+0000 7f2f50da8700 1 -- 172.21.15.46:0/270142305 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5802b040 con 0x563f38faee70 2024-03-18T20:38:41.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.931+0000 7f2f50da8700 1 -- 172.21.15.46:0/270142305 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f5800f920 con 0x563f38faee70 2024-03-18T20:38:41.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.931+0000 7f2f50da8700 1 -- 172.21.15.46:0/270142305 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58030820 con 0x563f38faee70 2024-03-18T20:38:41.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.932+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/270142305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 msgr2=0x563f38fcc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.932+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/270142305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 0x563f38fcc620 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2f58000c00 tx=0x7f2f58028040 comp rx=0 tx=0).stop 2024-03-18T20:38:41.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.932+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/270142305 shutdown_connections 2024-03-18T20:38:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.932+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/270142305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faee70 0x563f38fcc620 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.932+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/270142305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 0x563f38fb6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.932+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/270142305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faf790 0x563f38fb5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.932+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/270142305 >> 172.21.15.46:0/270142305 conn(0x563f38f21220 msgr2=0x563f38fb2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.932+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/270142305 shutdown_connections 2024-03-18T20:38:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.932+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/270142305 wait complete. 2024-03-18T20:38:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.932+0000 7f2f6fcd2b80 1 -- start start 2024-03-18T20:38:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.932+0000 7f2f6fcd2b80 1 Processor -- start 2024-03-18T20:38:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f6fcd2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faee70 0x563f38fbe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f6fcd2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 0x563f38fbee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f6fcd2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 0x563f38fbf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f6fcd2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3901d060 con 0x563f38faf790 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f6fcd2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3901d1a0 con 0x563f38faee70 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f6fcd2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f3901d2e0 con 0x563f38fba640 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f6fcd2b80 10 client.0 _ll_register_callbacks cb 0x563f38d053a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f610d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 0x563f38fbee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f610d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 0x563f38fbee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34094/0 (socket says 172.21.15.46:34094) 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f618d9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faee70 0x563f38fbe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f610d8700 1 -- 172.21.15.46:0/4073397014 learned_addr learned my addr 172.21.15.46:0/4073397014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f620da700 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 0x563f38fbf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f610d8700 1 -- 172.21.15.46:0/4073397014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 msgr2=0x563f38fbf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f610d8700 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 0x563f38fbf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f610d8700 1 -- 172.21.15.46:0/4073397014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faee70 msgr2=0x563f38fbe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f610d8700 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faee70 0x563f38fbe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f610d8700 1 -- 172.21.15.46:0/4073397014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f38d7f3b0 con 0x563f38faf790 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.933+0000 7f2f618d9700 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faee70 0x563f38fbe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f610d8700 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 0x563f38fbee90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c021c60 tx=0x7f2f4c09df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1904935692b835d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f457fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f457fa700 1 -- 172.21.15.46:0/4073397014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0adea0 con 0x563f38faf790 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f457fa700 1 -- 172.21.15.46:0/4073397014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2f4c0a2450 con 0x563f38faf790 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/4073397014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f5801e040 con 0x563f38faf790 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f457fa700 1 -- 172.21.15.46:0/4073397014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f4c0a20b0 con 0x563f38faf790 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f6fcd2b80 10 client.16404 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/4073397014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563f38d83cd0 con 0x563f38faf790 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f457fa700 1 -- 172.21.15.46:0/4073397014 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f5801e040 con 0x563f38faf790 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f457fa700 1 -- 172.21.15.46:0/4073397014 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2f4c0b2b80 con 0x563f38faf790 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f6fcd2b80 10 client.16404 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f6fcd2b80 10 client.16404 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:38:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f6fcd2b80 20 client.16404 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563f3901e4f0 0x563f39020910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f6fcd2b80 4 client.16404 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.934+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/4073397014 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f38d4ccf0 con 0x563f3901e4f0 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f618d9700 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563f3901e4f0 0x563f39020910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f618d9700 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563f3901e4f0 0x563f39020910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3c12a952ea449d3 server_cookie=f1327af90d88c085 in_seq=0 out_seq=0 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f457fa700 10 client.16404 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f457fa700 1 -- 172.21.15.46:0/4073397014 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f38d4ccf0 con 0x563f3901e4f0 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f457fa700 10 client.16404 handle_command_reply: tid=0 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f6fcd2b80 1 client.16404 shutdown 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/4073397014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 msgr2=0x563f38fbee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 0x563f38fbee90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c021c60 tx=0x7f2f4c09df80 comp rx=0 tx=0).stop 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/4073397014 shutdown_connections 2024-03-18T20:38:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563f3901e4f0 0x563f39020910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563f38fba640 0x563f38fbf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f457fa700 0 client.16404 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:38:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563f38faee70 0x563f38fbe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f6fcd2b80 1 --2- 172.21.15.46:0/4073397014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f38faf790 0x563f38fbee90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.935+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/4073397014 >> 172.21.15.46:0/4073397014 conn(0x563f38f21220 msgr2=0x563f38fb5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/4073397014 shutdown_connections 2024-03-18T20:38:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f6fcd2b80 1 -- 172.21.15.46:0/4073397014 wait complete. 2024-03-18T20:38:41.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f6fcd2b80 20 client.16404 trim_cache size 0 max 16384 2024-03-18T20:38:41.945 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:38:41.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f63b3d700 1 -- 172.21.15.46:0/3916266053 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2f480a1290 msgr2=0x7f2f480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f63b3d700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2f480a1290 0x7f2f480a3750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800b2e0 tx=0x7f2f58005940 comp rx=0 tx=0).stop 2024-03-18T20:38:41.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f63b3d700 1 -- 172.21.15.46:0/3916266053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f5c106070 msgr2=0x7f2f5c197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:41.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f63b3d700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f5c106070 0x7f2f5c197770 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00e970 tx=0x7f2f4c00ed30 comp rx=0 tx=0).stop 2024-03-18T20:38:41.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f63b3d700 1 -- 172.21.15.46:0/3916266053 shutdown_connections 2024-03-18T20:38:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f63b3d700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2f480a1290 0x7f2f480a3750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f63b3d700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f5c19bb10 0x7f2f5c19bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f63b3d700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f5c100f50 0x7f2f5c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f63b3d700 1 --2- 172.21.15.46:0/3916266053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f5c106070 0x7f2f5c197770 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f63b3d700 1 -- 172.21.15.46:0/3916266053 >> 172.21.15.46:0/3916266053 conn(0x7f2f5c0fc7f0 msgr2=0x7f2f5c0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.936+0000 7f2f63b3d700 1 -- 172.21.15.46:0/3916266053 shutdown_connections 2024-03-18T20:38:41.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:41.937+0000 7f2f63b3d700 1 -- 172.21.15.46:0/3916266053 wait complete. 2024-03-18T20:38:41.954 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:38:41.954 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:38:42.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.095+0000 7f83b7bcb700 1 -- 172.21.15.46:0/3074235198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0105ce0 msgr2=0x7f83b01060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.095+0000 7f83b7bcb700 1 --2- 172.21.15.46:0/3074235198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0105ce0 0x7f83b01060c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009990 tx=0x7f83ac009ca0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.095+0000 7f83b7bcb700 1 -- 172.21.15.46:0/3074235198 shutdown_connections 2024-03-18T20:38:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.095+0000 7f83b7bcb700 1 --2- 172.21.15.46:0/3074235198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0105ce0 0x7f83b01060c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.095+0000 7f83b7bcb700 1 --2- 172.21.15.46:0/3074235198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b00687f0 0x7f83b010db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.095+0000 7f83b7bcb700 1 --2- 172.21.15.46:0/3074235198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83b0067e30 0x7f83b00682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.095+0000 7f83b7bcb700 1 -- 172.21.15.46:0/3074235198 >> 172.21.15.46:0/3074235198 conn(0x7f83b0076d40 msgr2=0x7f83b0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.095+0000 7f83b7bcb700 1 -- 172.21.15.46:0/3074235198 shutdown_connections 2024-03-18T20:38:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.095+0000 7f83b7bcb700 1 -- 172.21.15.46:0/3074235198 wait complete. 2024-03-18T20:38:42.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b7bcb700 1 Processor -- start 2024-03-18T20:38:42.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b7bcb700 1 -- start start 2024-03-18T20:38:42.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b7bcb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b0067e30 0x7f83b00ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b7bcb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83b00687f0 0x7f83b01001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b7bcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0105ce0 0x7f83b01006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b7bcb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01a55b0 con 0x7f83b0105ce0 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b7bcb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01a5720 con 0x7f83b0067e30 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b7bcb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01a58e0 con 0x7f83b00687f0 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b6168700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0105ce0 0x7f83b01006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b6168700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0105ce0 0x7f83b01006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34106/0 (socket says 172.21.15.46:34106) 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b6168700 1 -- 172.21.15.46:0/1523912935 learned_addr learned my addr 172.21.15.46:0/1523912935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b5166700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83b00687f0 0x7f83b01001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b5967700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b0067e30 0x7f83b00ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b6168700 1 -- 172.21.15.46:0/1523912935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83b00687f0 msgr2=0x7f83b01001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b6168700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83b00687f0 0x7f83b01001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b6168700 1 -- 172.21.15.46:0/1523912935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b0067e30 msgr2=0x7f83b00ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b6168700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b0067e30 0x7f83b00ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.096+0000 7f83b6168700 1 -- 172.21.15.46:0/1523912935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0009520 con 0x7f83b0105ce0 2024-03-18T20:38:42.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.097+0000 7f83b6168700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0105ce0 0x7f83b01006f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f83a800ef30 tx=0x7f83a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.097+0000 7f83a6ffd700 1 -- 172.21.15.46:0/1523912935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8018070 con 0x7f83b0105ce0 2024-03-18T20:38:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.097+0000 7f83b7bcb700 1 -- 172.21.15.46:0/1523912935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ac0095f0 con 0x7f83b0105ce0 2024-03-18T20:38:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.097+0000 7f83a6ffd700 1 -- 172.21.15.46:0/1523912935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a800f040 con 0x7f83b0105ce0 2024-03-18T20:38:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.097+0000 7f83a6ffd700 1 -- 172.21.15.46:0/1523912935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a800cd00 con 0x7f83b0105ce0 2024-03-18T20:38:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.097+0000 7f83b7bcb700 1 -- 172.21.15.46:0/1523912935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b01a5e90 con 0x7f83b0105ce0 2024-03-18T20:38:42.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.099+0000 7f83a6ffd700 1 -- 172.21.15.46:0/1523912935 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f83a8003a00 con 0x7f83b0105ce0 2024-03-18T20:38:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.100+0000 7f83a6ffd700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f839c0a1290 0x7f839c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.100+0000 7f83a6ffd700 1 -- 172.21.15.46:0/1523912935 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f83a80cc810 con 0x7f83b0105ce0 2024-03-18T20:38:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.100+0000 7f83b5967700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f839c0a1290 0x7f839c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83b5967700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f839c0a1290 0x7f839c0a3750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0098e0 tx=0x7f83ac005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83c3d60b80 1 Processor -- start 2024-03-18T20:38:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83c3d60b80 1 -- start start 2024-03-18T20:38:42.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83c3d60b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5aa60 0x564e1ac5ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83c3d60b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 0x564e1ac65ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83c3d60b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1ac68610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83c3d60b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e1abb4990 con 0x564e1ac5b380 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83c3d60b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e1abb2350 con 0x564e1ac66220 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83c3d60b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e1abadc40 con 0x564e1ac5aa60 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83b5166700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 0x564e1ac65ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83b6168700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1ac68610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83b5967700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5aa60 0x564e1ac5ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83b5166700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 0x564e1ac65ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34108/0 (socket says 172.21.15.46:34108) 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.101+0000 7f83b5166700 1 -- 172.21.15.46:0/965555343 learned_addr learned my addr 172.21.15.46:0/965555343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.102+0000 7f83b6168700 1 -- 172.21.15.46:0/965555343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5aa60 msgr2=0x564e1ac5ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.102+0000 7f83b6168700 1 --2- 172.21.15.46:0/965555343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5aa60 0x564e1ac5ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.102+0000 7f83b6168700 1 -- 172.21.15.46:0/965555343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 msgr2=0x564e1ac65ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.102+0000 7f83b6168700 1 --2- 172.21.15.46:0/965555343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 0x564e1ac65ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.102+0000 7f83b6168700 1 -- 172.21.15.46:0/965555343 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e1aa2f3b0 con 0x564e1ac66220 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.102+0000 7f83b6168700 1 --2- 172.21.15.46:0/965555343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1ac68610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f83a80c9f70 tx=0x7f83a8099580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3219dff239148a4a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.102+0000 7f839bfff700 1 -- 172.21.15.46:0/965555343 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8004ad0 con 0x564e1ac66220 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.102+0000 7f839bfff700 1 -- 172.21.15.46:0/965555343 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a801b320 con 0x564e1ac66220 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.102+0000 7f839bfff700 1 -- 172.21.15.46:0/965555343 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8099c20 con 0x564e1ac66220 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.103+0000 7f83c3d60b80 1 -- 172.21.15.46:0/965555343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 msgr2=0x564e1ac68610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.103+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/965555343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1ac68610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f83a80c9f70 tx=0x7f83a8099580 comp rx=0 tx=0).stop 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.103+0000 7f83c3d60b80 1 -- 172.21.15.46:0/965555343 shutdown_connections 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.103+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/965555343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1ac68610 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.103+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/965555343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5aa60 0x564e1ac5ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.103+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/965555343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 0x564e1ac65ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.103+0000 7f83c3d60b80 1 -- 172.21.15.46:0/965555343 >> 172.21.15.46:0/965555343 conn(0x564e1abd1220 msgr2=0x564e1ac57020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.103+0000 7f83c3d60b80 1 -- 172.21.15.46:0/965555343 shutdown_connections 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.103+0000 7f83c3d60b80 1 -- 172.21.15.46:0/965555343 wait complete. 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.103+0000 7f83c3d60b80 1 -- start start 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83c3d60b80 1 Processor -- start 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83c3d60b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5aa60 0x564e1ac63110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83c3d60b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 0x564e1ac63650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83c3d60b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1ac63b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83c3d60b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e1ab49d80 con 0x564e1ac5aa60 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83c3d60b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e1ab49a20 con 0x564e1ac66220 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83c3d60b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e1aa316a0 con 0x564e1ac5b380 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83c3d60b80 10 client.0 _ll_register_callbacks cb 0x564e1a9b53a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83b5967700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5aa60 0x564e1ac63110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83b5166700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 0x564e1ac63650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83b6168700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1ac63b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83b5166700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 0x564e1ac63650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:41360/0 (socket says 172.21.15.46:41360) 2024-03-18T20:38:42.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83b5967700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5aa60 0x564e1ac63110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34124/0 (socket says 172.21.15.46:34124) 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.104+0000 7f83b5166700 1 -- 172.21.15.46:0/3148589659 learned_addr learned my addr 172.21.15.46:0/3148589659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83b5166700 1 -- 172.21.15.46:0/3148589659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 msgr2=0x564e1ac63b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83b5166700 1 --2- 172.21.15.46:0/3148589659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1ac63b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83b5166700 1 -- 172.21.15.46:0/3148589659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5aa60 msgr2=0x564e1ac63110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83b5166700 1 --2- 172.21.15.46:0/3148589659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5aa60 0x564e1ac63110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83b5166700 1 -- 172.21.15.46:0/3148589659 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b01a5e90 con 0x564e1ac5b380 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83b5166700 1 --2- 172.21.15.46:0/3148589659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 0x564e1ac63650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83a0005c50 tx=0x7f83a0012cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4bfda50ec6884a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83997fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83997fa700 1 -- 172.21.15.46:0/3148589659 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0017070 con 0x564e1ac5b380 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83c3d60b80 1 -- 172.21.15.46:0/3148589659 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e1ac4ead0 con 0x564e1ac5b380 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83997fa700 1 -- 172.21.15.46:0/3148589659 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f83a0004050 con 0x564e1ac5b380 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83997fa700 1 -- 172.21.15.46:0/3148589659 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83a0004210 con 0x564e1ac5b380 2024-03-18T20:38:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83c3d60b80 10 client.26167 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.105+0000 7f83c3d60b80 1 -- 172.21.15.46:0/3148589659 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564e1a9dbb60 con 0x564e1ac5b380 2024-03-18T20:38:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.106+0000 7f83997fa700 1 -- 172.21.15.46:0/3148589659 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e1ac4ead0 con 0x564e1ac5b380 2024-03-18T20:38:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.106+0000 7f83997fa700 1 -- 172.21.15.46:0/3148589659 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f83a0003840 con 0x564e1ac5b380 2024-03-18T20:38:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.106+0000 7f83c3d60b80 10 client.26167 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.106+0000 7f83c3d60b80 10 client.26167 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:38:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.106+0000 7f83c3d60b80 20 client.26167 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.106+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/3148589659 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564e1ac75e60 0x564e1ac78280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.106+0000 7f83c3d60b80 4 client.26167 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.106+0000 7f83c3d60b80 1 -- 172.21.15.46:0/3148589659 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e1aba6700 con 0x564e1ac75e60 2024-03-18T20:38:42.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.106+0000 7f83b5967700 1 --2- 172.21.15.46:0/3148589659 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564e1ac75e60 0x564e1ac78280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.106+0000 7f83997fa700 10 client.26167 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:38:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.106+0000 7f83b5967700 1 --2- 172.21.15.46:0/3148589659 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564e1ac75e60 0x564e1ac78280 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3c110b2315a123ad server_cookie=7f31f182d307a395 in_seq=0 out_seq=0 2024-03-18T20:38:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83997fa700 1 -- 172.21.15.46:0/3148589659 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564e1aba6700 con 0x564e1ac75e60 2024-03-18T20:38:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83997fa700 10 client.26167 handle_command_reply: tid=0 2024-03-18T20:38:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83c3d60b80 1 client.26167 shutdown 2024-03-18T20:38:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83c3d60b80 1 -- 172.21.15.46:0/3148589659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 msgr2=0x564e1ac63650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/3148589659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 0x564e1ac63650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83a0005c50 tx=0x7f83a0012cc0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83c3d60b80 1 -- 172.21.15.46:0/3148589659 shutdown_connections 2024-03-18T20:38:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/3148589659 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564e1ac75e60 0x564e1ac78280 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/3148589659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5aa60 0x564e1ac63110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83997fa700 0 client.26167 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:38:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/3148589659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1ac63b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/3148589659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 0x564e1ac63650 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83c3d60b80 1 -- 172.21.15.46:0/3148589659 >> 172.21.15.46:0/3148589659 conn(0x564e1abd1220 msgr2=0x564e1ac56c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83c3d60b80 1 -- 172.21.15.46:0/3148589659 shutdown_connections 2024-03-18T20:38:42.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.107+0000 7f83c3d60b80 1 -- 172.21.15.46:0/3148589659 wait complete. 2024-03-18T20:38:42.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.108+0000 7f83c3d60b80 20 client.26167 trim_cache size 0 max 16384 2024-03-18T20:38:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.122+0000 7f83c3d60b80 1 Processor -- start 2024-03-18T20:38:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.122+0000 7f83c3d60b80 1 -- start start 2024-03-18T20:38:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.122+0000 7f83c3d60b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac5aa60 0x564e1ac71fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.122+0000 7f83c3d60b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 0x564e1ac6ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.122+0000 7f83c3d60b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac66220 0x564e1ac639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.122+0000 7f83c3d60b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e1ac583f0 con 0x564e1ac5b380 2024-03-18T20:38:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.122+0000 7f83c3d60b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e1ac66040 con 0x564e1ac5aa60 2024-03-18T20:38:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.122+0000 7f83c3d60b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e1ac6d390 con 0x564e1ac66220 2024-03-18T20:38:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.122+0000 7f83b5166700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 0x564e1ac6ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.122+0000 7f83b5166700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 0x564e1ac6ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34128/0 (socket says 172.21.15.46:34128) 2024-03-18T20:38:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.122+0000 7f83b5166700 1 -- 172.21.15.46:0/2561070922 learned_addr learned my addr 172.21.15.46:0/2561070922 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.122+0000 7f83b6168700 1 --2- 172.21.15.46:0/2561070922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac66220 0x564e1ac639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.123+0000 7f83b5166700 1 -- 172.21.15.46:0/2561070922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac66220 msgr2=0x564e1ac639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.123+0000 7f83b5166700 1 --2- 172.21.15.46:0/2561070922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac66220 0x564e1ac639b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.123+0000 7f83b5166700 1 -- 172.21.15.46:0/2561070922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac5aa60 msgr2=0x564e1ac71fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.123+0000 7f83b5166700 1 --2- 172.21.15.46:0/2561070922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac5aa60 0x564e1ac71fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.123+0000 7f83b5166700 1 -- 172.21.15.46:0/2561070922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e1ab73ea0 con 0x564e1ac5b380 2024-03-18T20:38:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.123+0000 7f83b5166700 1 --2- 172.21.15.46:0/2561070922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 0x564e1ac6ce50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009870 tx=0x7f83a0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c5ce3d43bae484 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.125+0000 7f83a4da8700 1 -- 172.21.15.46:0/2561070922 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0017040 con 0x564e1ac5b380 2024-03-18T20:38:42.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.125+0000 7f83a4da8700 1 -- 172.21.15.46:0/2561070922 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a0003e90 con 0x564e1ac5b380 2024-03-18T20:38:42.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.125+0000 7f83a4da8700 1 -- 172.21.15.46:0/2561070922 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0005320 con 0x564e1ac5b380 2024-03-18T20:38:42.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.126+0000 7f83c3d60b80 1 -- 172.21.15.46:0/2561070922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 msgr2=0x564e1ac6ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.126+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/2561070922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 0x564e1ac6ce50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009870 tx=0x7f83a0003d40 comp rx=0 tx=0).stop 2024-03-18T20:38:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.126+0000 7f83c3d60b80 1 -- 172.21.15.46:0/2561070922 shutdown_connections 2024-03-18T20:38:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.126+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/2561070922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac66220 0x564e1ac639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.126+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/2561070922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac5aa60 0x564e1ac71fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.126+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/2561070922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5b380 0x564e1ac6ce50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.126+0000 7f83c3d60b80 1 -- 172.21.15.46:0/2561070922 >> 172.21.15.46:0/2561070922 conn(0x564e1abd1220 msgr2=0x564e1ac56b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.126+0000 7f83c3d60b80 1 -- 172.21.15.46:0/2561070922 shutdown_connections 2024-03-18T20:38:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.126+0000 7f83c3d60b80 1 -- 172.21.15.46:0/2561070922 wait complete. 2024-03-18T20:38:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.126+0000 7f83c3d60b80 1 -- start start 2024-03-18T20:38:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.126+0000 7f83c3d60b80 1 Processor -- start 2024-03-18T20:38:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83c3d60b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5aa60 0x564e1ac6b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83c3d60b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 0x564e1ac6b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83c3d60b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1acc8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83c3d60b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e1ac6bce0 con 0x564e1ac5aa60 2024-03-18T20:38:42.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83c3d60b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e1acc9200 con 0x564e1ac66220 2024-03-18T20:38:42.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83c3d60b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e1acc93c0 con 0x564e1ac5b380 2024-03-18T20:38:42.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83c3d60b80 10 client.0 _ll_register_callbacks cb 0x7f83a00124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:42.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83b5967700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5aa60 0x564e1ac6b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83b6168700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1acc8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83b5166700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 0x564e1ac6b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83b6168700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1acc8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:57272/0 (socket says 172.21.15.46:57272) 2024-03-18T20:38:42.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83b6168700 1 -- 172.21.15.46:0/1344512866 learned_addr learned my addr 172.21.15.46:0/1344512866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83b6168700 1 -- 172.21.15.46:0/1344512866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 msgr2=0x564e1ac6b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83b6168700 1 --2- 172.21.15.46:0/1344512866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 0x564e1ac6b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83b6168700 1 -- 172.21.15.46:0/1344512866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5aa60 msgr2=0x564e1ac6b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83b6168700 1 --2- 172.21.15.46:0/1344512866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5aa60 0x564e1ac6b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83b6168700 1 -- 172.21.15.46:0/1344512866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e1a9dbb60 con 0x564e1ac66220 2024-03-18T20:38:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83b5166700 1 --2- 172.21.15.46:0/1344512866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 0x564e1ac6b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83b6168700 1 --2- 172.21.15.46:0/1344512866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1acc8c90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f83a800c6f0 tx=0x7f83a8099160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f51cb54e8b1a759 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.127+0000 7f83997fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:38:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83997fa700 1 -- 172.21.15.46:0/1344512866 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8004ad0 con 0x564e1ac66220 2024-03-18T20:38:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83997fa700 1 -- 172.21.15.46:0/1344512866 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f83a809a0c0 con 0x564e1ac66220 2024-03-18T20:38:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83c3d60b80 1 -- 172.21.15.46:0/1344512866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83a801b320 con 0x564e1ac66220 2024-03-18T20:38:42.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83997fa700 1 -- 172.21.15.46:0/1344512866 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83a80999c0 con 0x564e1ac66220 2024-03-18T20:38:42.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83c3d60b80 10 client.26177 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:42.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83c3d60b80 1 -- 172.21.15.46:0/1344512866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564e1aa33cd0 con 0x564e1ac66220 2024-03-18T20:38:42.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83997fa700 1 -- 172.21.15.46:0/1344512866 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83a807c080 con 0x564e1ac66220 2024-03-18T20:38:42.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83997fa700 1 -- 172.21.15.46:0/1344512866 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f83a80a1920 con 0x564e1ac66220 2024-03-18T20:38:42.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83c3d60b80 10 client.26177 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:42.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83c3d60b80 10 client.26177 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:38:42.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83c3d60b80 20 client.26177 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:42.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/1344512866 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564e1acca670 0x564e1accca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83c3d60b80 4 client.26177 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:38:42.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83c3d60b80 1 -- 172.21.15.46:0/1344512866 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564e1a9fccf0 con 0x564e1acca670 2024-03-18T20:38:42.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.128+0000 7f83b5967700 1 --2- 172.21.15.46:0/1344512866 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564e1acca670 0x564e1accca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83997fa700 10 client.26177 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83b5967700 1 --2- 172.21.15.46:0/1344512866 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564e1acca670 0x564e1accca90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=becfbb65f4f42e66 server_cookie=b625e6730be43e45 in_seq=0 out_seq=0 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83997fa700 1 -- 172.21.15.46:0/1344512866 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564e1a9fccf0 con 0x564e1acca670 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83997fa700 10 client.26177 handle_command_reply: tid=0 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83c3d60b80 1 client.26177 shutdown 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83c3d60b80 1 -- 172.21.15.46:0/1344512866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 msgr2=0x564e1acc8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/1344512866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1acc8c90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f83a800c6f0 tx=0x7f83a8099160 comp rx=0 tx=0).stop 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83c3d60b80 1 -- 172.21.15.46:0/1344512866 shutdown_connections 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/1344512866 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564e1acca670 0x564e1accca90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/1344512866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e1ac5aa60 0x564e1ac6b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/1344512866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e1ac66220 0x564e1acc8c90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83c3d60b80 1 --2- 172.21.15.46:0/1344512866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e1ac5b380 0x564e1ac6b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83997fa700 0 client.26177 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:38:42.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.129+0000 7f83c3d60b80 1 -- 172.21.15.46:0/1344512866 >> 172.21.15.46:0/1344512866 conn(0x564e1abd1220 msgr2=0x564e1ac715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83c3d60b80 1 -- 172.21.15.46:0/1344512866 shutdown_connections 2024-03-18T20:38:42.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83c3d60b80 1 -- 172.21.15.46:0/1344512866 wait complete. 2024-03-18T20:38:42.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83c3d60b80 20 client.26177 trim_cache size 0 max 16384 2024-03-18T20:38:42.138 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:38:42.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83b7bcb700 1 -- 172.21.15.46:0/1523912935 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f839c0a1290 msgr2=0x7f839c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83b7bcb700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f839c0a1290 0x7f839c0a3750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0098e0 tx=0x7f83ac005960 comp rx=0 tx=0).stop 2024-03-18T20:38:42.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83b7bcb700 1 -- 172.21.15.46:0/1523912935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0105ce0 msgr2=0x7f83b01006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83b7bcb700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0105ce0 0x7f83b01006f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f83a800ef30 tx=0x7f83a800c3a0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83b7bcb700 1 -- 172.21.15.46:0/1523912935 shutdown_connections 2024-03-18T20:38:42.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83b7bcb700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f839c0a1290 0x7f839c0a3750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83b7bcb700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0105ce0 0x7f83b01006f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83b7bcb700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83b0067e30 0x7f83b00ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83b7bcb700 1 --2- 172.21.15.46:0/1523912935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83b00687f0 0x7f83b01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83b7bcb700 1 -- 172.21.15.46:0/1523912935 >> 172.21.15.46:0/1523912935 conn(0x7f83b0076d40 msgr2=0x7f83b00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.130+0000 7f83b7bcb700 1 -- 172.21.15.46:0/1523912935 shutdown_connections 2024-03-18T20:38:42.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.131+0000 7f83b7bcb700 1 -- 172.21.15.46:0/1523912935 wait complete. 2024-03-18T20:38:42.147 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:38:42.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:38:42.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.287+0000 7f86dac50700 1 -- 172.21.15.46:0/1390003757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d4104f60 msgr2=0x7f86d410cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.287+0000 7f86dac50700 1 --2- 172.21.15.46:0/1390003757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d4104f60 0x7f86d410cae0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00b0a0 tx=0x7f86cc00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.287+0000 7f86dac50700 1 -- 172.21.15.46:0/1390003757 shutdown_connections 2024-03-18T20:38:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.287+0000 7f86dac50700 1 --2- 172.21.15.46:0/1390003757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d4104f60 0x7f86d410cae0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.287+0000 7f86dac50700 1 --2- 172.21.15.46:0/1390003757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d40ffe50 0x7f86d4100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.287+0000 7f86dac50700 1 --2- 172.21.15.46:0/1390003757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d4100800 0x7f86d4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.287+0000 7f86dac50700 1 -- 172.21.15.46:0/1390003757 >> 172.21.15.46:0/1390003757 conn(0x7f86d4076d40 msgr2=0x7f86d4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.288+0000 7f86dac50700 1 -- 172.21.15.46:0/1390003757 shutdown_connections 2024-03-18T20:38:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.288+0000 7f86dac50700 1 -- 172.21.15.46:0/1390003757 wait complete. 2024-03-18T20:38:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.288+0000 7f86dac50700 1 Processor -- start 2024-03-18T20:38:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.288+0000 7f86dac50700 1 -- start start 2024-03-18T20:38:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86dac50700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d40ffe50 0x7f86d4197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86dac50700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d4104f60 0x7f86d4197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86dac50700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d419bc30 0x7f86d419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86dac50700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d419c6b0 con 0x7f86d40ffe50 2024-03-18T20:38:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86dac50700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d419c820 con 0x7f86d419bc30 2024-03-18T20:38:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86dac50700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d419c9e0 con 0x7f86d4104f60 2024-03-18T20:38:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86d89ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d40ffe50 0x7f86d4197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86d89ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d40ffe50 0x7f86d4197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34164/0 (socket says 172.21.15.46:34164) 2024-03-18T20:38:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86d89ec700 1 -- 172.21.15.46:0/2419483368 learned_addr learned my addr 172.21.15.46:0/2419483368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86d3fff700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d4104f60 0x7f86d4197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86d91ed700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d419bc30 0x7f86d419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86d89ec700 1 -- 172.21.15.46:0/2419483368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d4104f60 msgr2=0x7f86d4197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86d89ec700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d4104f60 0x7f86d4197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86d89ec700 1 -- 172.21.15.46:0/2419483368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d419bc30 msgr2=0x7f86d419c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86d89ec700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d419bc30 0x7f86d419c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86d89ec700 1 -- 172.21.15.46:0/2419483368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c0009520 con 0x7f86d40ffe50 2024-03-18T20:38:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86d89ec700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d40ffe50 0x7f86d4197350 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f86c800e9a0 tx=0x7f86c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.289+0000 7f86d1ffb700 1 -- 172.21.15.46:0/2419483368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8009790 con 0x7f86d40ffe50 2024-03-18T20:38:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.290+0000 7f86d1ffb700 1 -- 172.21.15.46:0/2419483368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c8010430 con 0x7f86d40ffe50 2024-03-18T20:38:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.290+0000 7f86dac50700 1 -- 172.21.15.46:0/2419483368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc009d00 con 0x7f86d40ffe50 2024-03-18T20:38:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.290+0000 7f86d1ffb700 1 -- 172.21.15.46:0/2419483368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8018680 con 0x7f86d40ffe50 2024-03-18T20:38:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.290+0000 7f86dac50700 1 -- 172.21.15.46:0/2419483368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d41a4230 con 0x7f86d40ffe50 2024-03-18T20:38:42.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.292+0000 7f86d1ffb700 1 -- 172.21.15.46:0/2419483368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f86c80105a0 con 0x7f86d40ffe50 2024-03-18T20:38:42.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.293+0000 7f86d1ffb700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f86c40a1290 0x7f86c40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.293+0000 7f86d1ffb700 1 -- 172.21.15.46:0/2419483368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f86c8014070 con 0x7f86d40ffe50 2024-03-18T20:38:42.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.293+0000 7f86d3fff700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f86c40a1290 0x7f86c40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.293+0000 7f86d3fff700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f86c40a1290 0x7f86c40a3750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f86d41988a0 tx=0x7f86c0009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86e6de5b80 1 Processor -- start 2024-03-18T20:38:42.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86e6de5b80 1 -- start start 2024-03-18T20:38:42.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86e6de5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3bba60 0x55c5fd3bbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86e6de5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86e6de5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3cb630 0x55c5fd3cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86e6de5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fd315990 con 0x55c5fd3bc380 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86e6de5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fd313350 con 0x55c5fd3cb630 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86e6de5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fd30ec40 con 0x55c5fd3bba60 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34170/0 (socket says 172.21.15.46:34170) 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86d3fff700 1 -- 172.21.15.46:0/644221973 learned_addr learned my addr 172.21.15.46:0/644221973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86d89ec700 1 --2- 172.21.15.46:0/644221973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3bba60 0x55c5fd3bbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86d91ed700 1 --2- 172.21.15.46:0/644221973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3cb630 0x55c5fd3cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86d89ec700 1 -- 172.21.15.46:0/644221973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3cb630 msgr2=0x55c5fd3cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86d89ec700 1 --2- 172.21.15.46:0/644221973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3cb630 0x55c5fd3cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86d89ec700 1 -- 172.21.15.46:0/644221973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 msgr2=0x55c5fd3cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86d89ec700 1 --2- 172.21.15.46:0/644221973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.294+0000 7f86d89ec700 1 -- 172.21.15.46:0/644221973 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5fd1903b0 con 0x55c5fd3bba60 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.295+0000 7f86d89ec700 1 --2- 172.21.15.46:0/644221973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3bba60 0x55c5fd3bbe40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f86c80ca5f0 tx=0x7f86c8099c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf029faceea0e70 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.295+0000 7f86beffd700 1 -- 172.21.15.46:0/644221973 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c809dde0 con 0x55c5fd3bba60 2024-03-18T20:38:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.295+0000 7f86beffd700 1 -- 172.21.15.46:0/644221973 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c80a2430 con 0x55c5fd3bba60 2024-03-18T20:38:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.295+0000 7f86beffd700 1 -- 172.21.15.46:0/644221973 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c80996e0 con 0x55c5fd3bba60 2024-03-18T20:38:42.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.296+0000 7f86e6de5b80 1 -- 172.21.15.46:0/644221973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3bba60 msgr2=0x55c5fd3bbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.296+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/644221973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3bba60 0x55c5fd3bbe40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f86c80ca5f0 tx=0x7f86c8099c00 comp rx=0 tx=0).stop 2024-03-18T20:38:42.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.296+0000 7f86e6de5b80 1 -- 172.21.15.46:0/644221973 shutdown_connections 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.296+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/644221973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3cb630 0x55c5fd3cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.296+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/644221973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3bba60 0x55c5fd3bbe40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.296+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/644221973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.296+0000 7f86e6de5b80 1 -- 172.21.15.46:0/644221973 >> 172.21.15.46:0/644221973 conn(0x55c5fd332220 msgr2=0x55c5fd3b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.296+0000 7f86e6de5b80 1 -- 172.21.15.46:0/644221973 shutdown_connections 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.296+0000 7f86e6de5b80 1 -- 172.21.15.46:0/644221973 wait complete. 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.296+0000 7f86e6de5b80 1 -- start start 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.296+0000 7f86e6de5b80 1 Processor -- start 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.297+0000 7f86e6de5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bba60 0x55c5fd3caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.297+0000 7f86e6de5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3bc380 0x55c5fd3c8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.297+0000 7f86e6de5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3cb630 0x55c5fd3c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.297+0000 7f86e6de5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fd2aad80 con 0x55c5fd3bba60 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.297+0000 7f86e6de5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fd2aaa20 con 0x55c5fd3cb630 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.297+0000 7f86e6de5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fd1926a0 con 0x55c5fd3bc380 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.297+0000 7f86e6de5b80 10 client.0 _ll_register_callbacks cb 0x55c5fd1163a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.297+0000 7f86d89ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bba60 0x55c5fd3caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.297+0000 7f86d89ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bba60 0x55c5fd3caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34180/0 (socket says 172.21.15.46:34180) 2024-03-18T20:38:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.297+0000 7f86d89ec700 1 -- 172.21.15.46:0/4118297635 learned_addr learned my addr 172.21.15.46:0/4118297635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.297+0000 7f86d91ed700 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3cb630 0x55c5fd3c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.297+0000 7f86d3fff700 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3bc380 0x55c5fd3c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86d91ed700 1 -- 172.21.15.46:0/4118297635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3bc380 msgr2=0x55c5fd3c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86d91ed700 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3bc380 0x55c5fd3c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86d91ed700 1 -- 172.21.15.46:0/4118297635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bba60 msgr2=0x55c5fd3caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86d91ed700 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bba60 0x55c5fd3caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86d91ed700 1 -- 172.21.15.46:0/4118297635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d41a4230 con 0x55c5fd3cb630 2024-03-18T20:38:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86d3fff700 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3bc380 0x55c5fd3c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86d91ed700 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3cb630 0x55c5fd3c89b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0062a0 tx=0x7f86cc0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77672cd24fc46f12 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86b3fff700 1 -- 172.21.15.46:0/4118297635 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc00e040 con 0x55c5fd3cb630 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86b3fff700 1 -- 172.21.15.46:0/4118297635 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f86cc0076e0 con 0x55c5fd3cb630 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86e6de5b80 1 -- 172.21.15.46:0/4118297635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5fd3afad0 con 0x55c5fd3cb630 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86b3fff700 1 -- 172.21.15.46:0/4118297635 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86cc0078e0 con 0x55c5fd3cb630 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86e6de5b80 10 client.26189 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86e6de5b80 1 -- 172.21.15.46:0/4118297635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c5fd13cb60 con 0x55c5fd3cb630 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.298+0000 7f86b3fff700 1 -- 172.21.15.46:0/4118297635 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5fd3afad0 con 0x55c5fd3cb630 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.299+0000 7f86b3fff700 1 -- 172.21.15.46:0/4118297635 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f86cc023020 con 0x55c5fd3cb630 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.299+0000 7f86e6de5b80 10 client.26189 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.299+0000 7f86e6de5b80 10 client.26189 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.299+0000 7f86e6de5b80 20 client.26189 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.299+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c5fd3df6a0 0x55c5fd3e1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.299+0000 7f86e6de5b80 4 client.26189 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.299+0000 7f86e6de5b80 1 -- 172.21.15.46:0/4118297635 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5fd307700 con 0x55c5fd3df6a0 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.299+0000 7f86d89ec700 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c5fd3df6a0 0x55c5fd3e1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.299+0000 7f86d89ec700 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c5fd3df6a0 0x55c5fd3e1ac0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f77583d8aa74bcb server_cookie=50c590b3722907d6 in_seq=0 out_seq=0 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.299+0000 7f86b3fff700 10 client.26189 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86b3fff700 1 -- 172.21.15.46:0/4118297635 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c5fd307700 con 0x55c5fd3df6a0 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86b3fff700 10 client.26189 handle_command_reply: tid=0 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86e6de5b80 1 client.26189 shutdown 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86e6de5b80 1 -- 172.21.15.46:0/4118297635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3cb630 msgr2=0x55c5fd3c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3cb630 0x55c5fd3c89b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0062a0 tx=0x7f86cc0095c0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86e6de5b80 1 -- 172.21.15.46:0/4118297635 shutdown_connections 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c5fd3df6a0 0x55c5fd3e1ac0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bba60 0x55c5fd3caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3cb630 0x55c5fd3c89b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/4118297635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3bc380 0x55c5fd3c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86b3fff700 0 client.26189 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:38:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86e6de5b80 1 -- 172.21.15.46:0/4118297635 >> 172.21.15.46:0/4118297635 conn(0x55c5fd332220 msgr2=0x55c5fd3b7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86e6de5b80 1 -- 172.21.15.46:0/4118297635 shutdown_connections 2024-03-18T20:38:42.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.300+0000 7f86e6de5b80 1 -- 172.21.15.46:0/4118297635 wait complete. 2024-03-18T20:38:42.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.301+0000 7f86e6de5b80 20 client.26189 trim_cache size 0 max 16384 2024-03-18T20:38:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86e6de5b80 1 Processor -- start 2024-03-18T20:38:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86e6de5b80 1 -- start start 2024-03-18T20:38:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86e6de5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 0x55c5fd3e1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86e6de5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86e6de5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3cb630 0x55c5fd3c8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86e6de5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fd3b2210 con 0x55c5fd3bc380 2024-03-18T20:38:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86e6de5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fd3c23f0 con 0x55c5fd3bba60 2024-03-18T20:38:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86e6de5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fd3c9a70 con 0x55c5fd3cb630 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86d89ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 0x55c5fd3e1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86d89ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 0x55c5fd3e1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:57328/0 (socket says 172.21.15.46:57328) 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86d89ec700 1 -- 172.21.15.46:0/1789208487 learned_addr learned my addr 172.21.15.46:0/1789208487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.315+0000 7f86d91ed700 1 --2- 172.21.15.46:0/1789208487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3cb630 0x55c5fd3c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.316+0000 7f86d89ec700 1 -- 172.21.15.46:0/1789208487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3cb630 msgr2=0x55c5fd3c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.316+0000 7f86d89ec700 1 --2- 172.21.15.46:0/1789208487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3cb630 0x55c5fd3c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.316+0000 7f86d89ec700 1 -- 172.21.15.46:0/1789208487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 msgr2=0x55c5fd3c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.316+0000 7f86d89ec700 1 --2- 172.21.15.46:0/1789208487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.316+0000 7f86d89ec700 1 -- 172.21.15.46:0/1789208487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5fd2d4ea0 con 0x55c5fd3bba60 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.316+0000 7f86d3fff700 1 --2- 172.21.15.46:0/1789208487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.316+0000 7f86d89ec700 1 --2- 172.21.15.46:0/1789208487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 0x55c5fd3e1a90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f86c80a1f00 tx=0x7f86c8099eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d879977fcbafb98a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.316+0000 7f86bf7fe700 1 -- 172.21.15.46:0/1789208487 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c80962f0 con 0x55c5fd3bba60 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.316+0000 7f86bf7fe700 1 -- 172.21.15.46:0/1789208487 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c80994a0 con 0x55c5fd3bba60 2024-03-18T20:38:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.316+0000 7f86bf7fe700 1 -- 172.21.15.46:0/1789208487 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c80a2200 con 0x55c5fd3bba60 2024-03-18T20:38:42.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.317+0000 7f86e6de5b80 1 -- 172.21.15.46:0/1789208487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 msgr2=0x55c5fd3e1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.317+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/1789208487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 0x55c5fd3e1a90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f86c80a1f00 tx=0x7f86c8099eb0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.317+0000 7f86e6de5b80 1 -- 172.21.15.46:0/1789208487 shutdown_connections 2024-03-18T20:38:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.317+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/1789208487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3cb630 0x55c5fd3c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.317+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/1789208487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 0x55c5fd3e1a90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.317+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/1789208487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.317+0000 7f86e6de5b80 1 -- 172.21.15.46:0/1789208487 >> 172.21.15.46:0/1789208487 conn(0x55c5fd332220 msgr2=0x55c5fd3b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.317+0000 7f86e6de5b80 1 -- 172.21.15.46:0/1789208487 shutdown_connections 2024-03-18T20:38:42.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.317+0000 7f86e6de5b80 1 -- 172.21.15.46:0/1789208487 wait complete. 2024-03-18T20:38:42.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.317+0000 7f86e6de5b80 1 -- start start 2024-03-18T20:38:42.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.317+0000 7f86e6de5b80 1 Processor -- start 2024-03-18T20:38:42.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86e6de5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 0x55c5fd3d9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86e6de5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3da180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86e6de5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3cb630 0x55c5fd3da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86e6de5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fd432510 con 0x55c5fd3bc380 2024-03-18T20:38:42.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86e6de5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fd432680 con 0x55c5fd3bba60 2024-03-18T20:38:42.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86e6de5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5fd432840 con 0x55c5fd3cb630 2024-03-18T20:38:42.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86e6de5b80 10 client.0 _ll_register_callbacks cb 0x55c5fd1163a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:42.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3da180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d89ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 0x55c5fd3d9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3da180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34210/0 (socket says 172.21.15.46:34210) 2024-03-18T20:38:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d89ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 0x55c5fd3d9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:57334/0 (socket says 172.21.15.46:57334) 2024-03-18T20:38:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d3fff700 1 -- 172.21.15.46:0/57782247 learned_addr learned my addr 172.21.15.46:0/57782247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d91ed700 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3cb630 0x55c5fd3da6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d3fff700 1 -- 172.21.15.46:0/57782247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3cb630 msgr2=0x55c5fd3da6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d3fff700 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3cb630 0x55c5fd3da6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d3fff700 1 -- 172.21.15.46:0/57782247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 msgr2=0x55c5fd3d9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d3fff700 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 0x55c5fd3d9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d3fff700 1 -- 172.21.15.46:0/57782247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c0017070 con 0x55c5fd3bc380 2024-03-18T20:38:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d89ec700 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 0x55c5fd3d9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86d3fff700 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3da180 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009b50 tx=0x7f86c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1b475d49682c1f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.318+0000 7f86b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86b3fff700 1 -- 172.21.15.46:0/57782247 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c00293f0 con 0x55c5fd3bc380 2024-03-18T20:38:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86e6de5b80 1 -- 172.21.15.46:0/57782247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86c80a2430 con 0x55c5fd3bc380 2024-03-18T20:38:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86b3fff700 1 -- 172.21.15.46:0/57782247 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f86c0029550 con 0x55c5fd3bc380 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86b3fff700 1 -- 172.21.15.46:0/57782247 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c0029710 con 0x55c5fd3bc380 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86e6de5b80 10 client.16440 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86e6de5b80 1 -- 172.21.15.46:0/57782247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c5fd1903b0 con 0x55c5fd3bc380 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86b3fff700 1 -- 172.21.15.46:0/57782247 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86c0036020 con 0x55c5fd3bc380 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86b3fff700 1 -- 172.21.15.46:0/57782247 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f86c001a420 con 0x55c5fd3bc380 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86e6de5b80 10 client.16440 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86e6de5b80 10 client.16440 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86e6de5b80 20 client.16440 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c5fd434100 0x55c5fd436520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86e6de5b80 4 client.16440 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.319+0000 7f86e6de5b80 1 -- 172.21.15.46:0/57782247 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c5fd15dcf0 con 0x55c5fd434100 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86d89ec700 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c5fd434100 0x55c5fd436520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86b3fff700 10 client.16440 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:38:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86d89ec700 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c5fd434100 0x55c5fd436520 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b289d2bca2b02450 server_cookie=f3782fe1e1424934 in_seq=0 out_seq=0 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86b3fff700 1 -- 172.21.15.46:0/57782247 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c5fd15dcf0 con 0x55c5fd434100 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86b3fff700 10 client.16440 handle_command_reply: tid=0 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86e6de5b80 1 client.16440 shutdown 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86e6de5b80 1 -- 172.21.15.46:0/57782247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 msgr2=0x55c5fd3da180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3da180 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009b50 tx=0x7f86c00047c0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86e6de5b80 1 -- 172.21.15.46:0/57782247 shutdown_connections 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5fd3cb630 0x55c5fd3da6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5fd3bba60 0x55c5fd3d9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c5fd434100 0x55c5fd436520 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86e6de5b80 1 --2- 172.21.15.46:0/57782247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5fd3bc380 0x55c5fd3da180 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86b3fff700 0 client.16440 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86e6de5b80 1 -- 172.21.15.46:0/57782247 >> 172.21.15.46:0/57782247 conn(0x55c5fd332220 msgr2=0x55c5fd3c8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86e6de5b80 1 -- 172.21.15.46:0/57782247 shutdown_connections 2024-03-18T20:38:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.320+0000 7f86e6de5b80 1 -- 172.21.15.46:0/57782247 wait complete. 2024-03-18T20:38:42.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86e6de5b80 20 client.16440 trim_cache size 0 max 16384 2024-03-18T20:38:42.333 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:38:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86dac50700 1 -- 172.21.15.46:0/2419483368 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f86c40a1290 msgr2=0x7f86c40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86dac50700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f86c40a1290 0x7f86c40a3750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f86d41988a0 tx=0x7f86c0009260 comp rx=0 tx=0).stop 2024-03-18T20:38:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86dac50700 1 -- 172.21.15.46:0/2419483368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d40ffe50 msgr2=0x7f86d4197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86dac50700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d40ffe50 0x7f86d4197350 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f86c800e9a0 tx=0x7f86c800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86dac50700 1 -- 172.21.15.46:0/2419483368 shutdown_connections 2024-03-18T20:38:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86dac50700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f86c40a1290 0x7f86c40a3750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86dac50700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d40ffe50 0x7f86d4197350 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86dac50700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d419bc30 0x7f86d419c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86dac50700 1 --2- 172.21.15.46:0/2419483368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d4104f60 0x7f86d4197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86dac50700 1 -- 172.21.15.46:0/2419483368 >> 172.21.15.46:0/2419483368 conn(0x7f86d4076d40 msgr2=0x7f86d4068170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86dac50700 1 -- 172.21.15.46:0/2419483368 shutdown_connections 2024-03-18T20:38:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.321+0000 7f86dac50700 1 -- 172.21.15.46:0/2419483368 wait complete. 2024-03-18T20:38:42.338 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:38:42.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:38:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:42 smithi073 ceph-mon[47384]: pgmap v81: 97 pgs: 97 active+clean; 35 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 447 KiB/s wr, 85 op/s 2024-03-18T20:38:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:42 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3407128046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:38:42.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.480+0000 7fa27e951700 1 -- 172.21.15.46:0/1834407552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa278104f60 msgr2=0x7fa27810cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.480+0000 7fa27e951700 1 --2- 172.21.15.46:0/1834407552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa278104f60 0x7fa27810cae0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa27000b0a0 tx=0x7fa27000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.480+0000 7fa27e951700 1 -- 172.21.15.46:0/1834407552 shutdown_connections 2024-03-18T20:38:42.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.480+0000 7fa27e951700 1 --2- 172.21.15.46:0/1834407552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa278104f60 0x7fa27810cae0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.480+0000 7fa27e951700 1 --2- 172.21.15.46:0/1834407552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2780ffe50 0x7fa278100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.480+0000 7fa27e951700 1 --2- 172.21.15.46:0/1834407552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa278100800 0x7fa278104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.480+0000 7fa27e951700 1 -- 172.21.15.46:0/1834407552 >> 172.21.15.46:0/1834407552 conn(0x7fa278076d40 msgr2=0x7fa278077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.480+0000 7fa27e951700 1 -- 172.21.15.46:0/1834407552 shutdown_connections 2024-03-18T20:38:42.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.480+0000 7fa27e951700 1 -- 172.21.15.46:0/1834407552 wait complete. 2024-03-18T20:38:42.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa27e951700 1 Processor -- start 2024-03-18T20:38:42.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa27e951700 1 -- start start 2024-03-18T20:38:42.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa27e951700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2780ffe50 0x7fa2781972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa27e951700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa278104f60 0x7fa2781977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa27e951700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27819bb90 0x7fa27819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa27e951700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27819c610 con 0x7fa27819bb90 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa27e951700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27819c780 con 0x7fa2780ffe50 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa27e951700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27819c940 con 0x7fa278104f60 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa27ceee700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27819bb90 0x7fa27819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa27ceee700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27819bb90 0x7fa27819c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34236/0 (socket says 172.21.15.46:34236) 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa27ceee700 1 -- 172.21.15.46:0/3410065654 learned_addr learned my addr 172.21.15.46:0/3410065654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa2777fe700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa278104f60 0x7fa2781977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.481+0000 7fa277fff700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2780ffe50 0x7fa2781972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.482+0000 7fa27ceee700 1 -- 172.21.15.46:0/3410065654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa278104f60 msgr2=0x7fa2781977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.482+0000 7fa27ceee700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa278104f60 0x7fa2781977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.482+0000 7fa27ceee700 1 -- 172.21.15.46:0/3410065654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2780ffe50 msgr2=0x7fa2781972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.482+0000 7fa27ceee700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2780ffe50 0x7fa2781972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.482+0000 7fa27ceee700 1 -- 172.21.15.46:0/3410065654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa270009d00 con 0x7fa27819bb90 2024-03-18T20:38:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.482+0000 7fa27ceee700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27819bb90 0x7fa27819c010 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa27000b070 tx=0x7fa27000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.482+0000 7fa2757fa700 1 -- 172.21.15.46:0/3410065654 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000e040 con 0x7fa27819bb90 2024-03-18T20:38:42.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.482+0000 7fa2757fa700 1 -- 172.21.15.46:0/3410065654 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa27001d480 con 0x7fa27819bb90 2024-03-18T20:38:42.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.482+0000 7fa27e951700 1 -- 172.21.15.46:0/3410065654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2781a3e40 con 0x7fa27819bb90 2024-03-18T20:38:42.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.482+0000 7fa2757fa700 1 -- 172.21.15.46:0/3410065654 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2700052a0 con 0x7fa27819bb90 2024-03-18T20:38:42.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.482+0000 7fa27e951700 1 -- 172.21.15.46:0/3410065654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2781a42c0 con 0x7fa27819bb90 2024-03-18T20:38:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.485+0000 7fa2757fa700 1 -- 172.21.15.46:0/3410065654 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa27001d5f0 con 0x7fa27819bb90 2024-03-18T20:38:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.485+0000 7fa2757fa700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2640a1310 0x7fa2640a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.485+0000 7fa2757fa700 1 -- 172.21.15.46:0/3410065654 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fa2700cd9c0 con 0x7fa27819bb90 2024-03-18T20:38:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.485+0000 7fa277fff700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2640a1310 0x7fa2640a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.486+0000 7fa277fff700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2640a1310 0x7fa2640a37d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa268009c60 tx=0x7fa268009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.486+0000 7fa28aae6b80 1 Processor -- start 2024-03-18T20:38:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.486+0000 7fa28aae6b80 1 -- start start 2024-03-18T20:38:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa28aae6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 0x5634f99f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa28aae6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f6380 0x5634f9a00ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa28aae6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f9a01220 0x5634f9a03610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa28aae6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f994f990 con 0x5634f99f5a60 2024-03-18T20:38:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa28aae6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f994d350 con 0x5634f99f6380 2024-03-18T20:38:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa28aae6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f9948c40 con 0x5634f9a01220 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa277fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 0x5634f99f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa2777fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f6380 0x5634f9a00ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa27ceee700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f9a01220 0x5634f9a03610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa277fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 0x5634f99f5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34250/0 (socket says 172.21.15.46:34250) 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa277fff700 1 -- 172.21.15.46:0/2881639765 learned_addr learned my addr 172.21.15.46:0/2881639765 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa277fff700 1 -- 172.21.15.46:0/2881639765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f9a01220 msgr2=0x5634f9a03610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa277fff700 1 --2- 172.21.15.46:0/2881639765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f9a01220 0x5634f9a03610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa277fff700 1 -- 172.21.15.46:0/2881639765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f6380 msgr2=0x5634f9a00ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa277fff700 1 --2- 172.21.15.46:0/2881639765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f6380 0x5634f9a00ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa277fff700 1 -- 172.21.15.46:0/2881639765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634f97ca3b0 con 0x5634f99f5a60 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa2777fe700 1 --2- 172.21.15.46:0/2881639765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f6380 0x5634f9a00ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa27ceee700 1 --2- 172.21.15.46:0/2881639765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f9a01220 0x5634f9a03610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa277fff700 1 --2- 172.21.15.46:0/2881639765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 0x5634f99f5e40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa26800e4e0 tx=0x7fa268014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c282fa6c4004ff3a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.487+0000 7fa26e7fc700 1 -- 172.21.15.46:0/2881639765 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800ee00 con 0x5634f99f5a60 2024-03-18T20:38:42.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.488+0000 7fa26e7fc700 1 -- 172.21.15.46:0/2881639765 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa268021070 con 0x5634f99f5a60 2024-03-18T20:38:42.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.488+0000 7fa26e7fc700 1 -- 172.21.15.46:0/2881639765 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26801d810 con 0x5634f99f5a60 2024-03-18T20:38:42.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.488+0000 7fa28aae6b80 1 -- 172.21.15.46:0/2881639765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 msgr2=0x5634f99f5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.488+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/2881639765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 0x5634f99f5e40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa26800e4e0 tx=0x7fa268014000 comp rx=0 tx=0).stop 2024-03-18T20:38:42.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.488+0000 7fa28aae6b80 1 -- 172.21.15.46:0/2881639765 shutdown_connections 2024-03-18T20:38:42.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.488+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/2881639765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 0x5634f99f5e40 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.488+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/2881639765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f9a01220 0x5634f9a03610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.488+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/2881639765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f6380 0x5634f9a00ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.488+0000 7fa28aae6b80 1 -- 172.21.15.46:0/2881639765 >> 172.21.15.46:0/2881639765 conn(0x5634f996c220 msgr2=0x5634f99f2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.488+0000 7fa28aae6b80 1 -- 172.21.15.46:0/2881639765 shutdown_connections 2024-03-18T20:38:42.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.488+0000 7fa28aae6b80 1 -- 172.21.15.46:0/2881639765 wait complete. 2024-03-18T20:38:42.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.489+0000 7fa28aae6b80 1 -- start start 2024-03-18T20:38:42.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.489+0000 7fa28aae6b80 1 Processor -- start 2024-03-18T20:38:42.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.489+0000 7fa28aae6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f99f5a60 0x5634f9a00c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.489+0000 7fa28aae6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f6380 0x5634f99fdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.489+0000 7fa28aae6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f9a01220 0x5634f99fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.489+0000 7fa28aae6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f98e4d80 con 0x5634f9a01220 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.489+0000 7fa28aae6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f98e4a20 con 0x5634f99f6380 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.489+0000 7fa28aae6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f97cc6a0 con 0x5634f99f5a60 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.489+0000 7fa28aae6b80 10 client.0 _ll_register_callbacks cb 0x5634f97503a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa277fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f99f5a60 0x5634f9a00c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa2777fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f6380 0x5634f99fdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa277fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f99f5a60 0x5634f9a00c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:41468/0 (socket says 172.21.15.46:41468) 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa277fff700 1 -- 172.21.15.46:0/1048895756 learned_addr learned my addr 172.21.15.46:0/1048895756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa27ceee700 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f9a01220 0x5634f99fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa2777fe700 1 -- 172.21.15.46:0/1048895756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f99f5a60 msgr2=0x5634f9a00c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa2777fe700 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f99f5a60 0x5634f9a00c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa2777fe700 1 -- 172.21.15.46:0/1048895756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f9a01220 msgr2=0x5634f99fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa2777fe700 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f9a01220 0x5634f99fe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa2777fe700 1 -- 172.21.15.46:0/1048895756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2781a42c0 con 0x5634f99f6380 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa27ceee700 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f9a01220 0x5634f99fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:42.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa277fff700 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f99f5a60 0x5634f9a00c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa2777fe700 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f6380 0x5634f99fdf50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa27806a4d0 tx=0x7fa260011a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a7ea98199642e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa2577fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa2577fe700 1 -- 172.21.15.46:0/1048895756 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260016070 con 0x5634f99f6380 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa2577fe700 1 -- 172.21.15.46:0/1048895756 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fa260004300 con 0x5634f99f6380 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa28aae6b80 1 -- 172.21.15.46:0/1048895756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634f99e9ad0 con 0x5634f99f6380 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.490+0000 7fa2577fe700 1 -- 172.21.15.46:0/1048895756 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2600044c0 con 0x5634f99f6380 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.491+0000 7fa28aae6b80 10 client.26207 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.491+0000 7fa28aae6b80 1 -- 172.21.15.46:0/1048895756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fa268017070 con 0x5634f99f6380 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.491+0000 7fa2577fe700 1 -- 172.21.15.46:0/1048895756 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5634f99e9ad0 con 0x5634f99f6380 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.491+0000 7fa2577fe700 1 -- 172.21.15.46:0/1048895756 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa260005580 con 0x5634f99f6380 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.491+0000 7fa28aae6b80 10 client.26207 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.491+0000 7fa28aae6b80 10 client.26207 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.491+0000 7fa28aae6b80 20 client.26207 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:42.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.491+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5634f9a10e70 0x5634f9a13290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.491+0000 7fa28aae6b80 4 client.26207 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.491+0000 7fa28aae6b80 1 -- 172.21.15.46:0/1048895756 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634f9941700 con 0x5634f9a10e70 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.491+0000 7fa277fff700 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5634f9a10e70 0x5634f9a13290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.491+0000 7fa2577fe700 10 client.26207 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa277fff700 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5634f9a10e70 0x5634f9a13290 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24d4f3aa9e3f57dd server_cookie=7c0569401e6e359b in_seq=0 out_seq=0 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa2577fe700 1 -- 172.21.15.46:0/1048895756 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5634f9941700 con 0x5634f9a10e70 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa2577fe700 10 client.26207 handle_command_reply: tid=0 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa28aae6b80 1 client.26207 shutdown 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa28aae6b80 1 -- 172.21.15.46:0/1048895756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f6380 msgr2=0x5634f99fdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f6380 0x5634f99fdf50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa27806a4d0 tx=0x7fa260011a10 comp rx=0 tx=0).stop 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa28aae6b80 1 -- 172.21.15.46:0/1048895756 shutdown_connections 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5634f9a10e70 0x5634f9a13290 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f9a01220 0x5634f99fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f99f5a60 0x5634f9a00c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/1048895756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f6380 0x5634f99fdf50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa2577fe700 0 client.26207 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:38:42.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.492+0000 7fa28aae6b80 1 -- 172.21.15.46:0/1048895756 >> 172.21.15.46:0/1048895756 conn(0x5634f996c220 msgr2=0x5634f99f1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.493+0000 7fa28aae6b80 1 -- 172.21.15.46:0/1048895756 shutdown_connections 2024-03-18T20:38:42.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.493+0000 7fa28aae6b80 1 -- 172.21.15.46:0/1048895756 wait complete. 2024-03-18T20:38:42.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.493+0000 7fa28aae6b80 20 client.26207 trim_cache size 0 max 16384 2024-03-18T20:38:42.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa28aae6b80 1 Processor -- start 2024-03-18T20:38:42.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa28aae6b80 1 -- start start 2024-03-18T20:38:42.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa28aae6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f5a60 0x5634f9a07e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa28aae6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f6380 0x5634f99f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa28aae6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f9a01220 0x5634f99f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa28aae6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f99f33d0 con 0x5634f99f6380 2024-03-18T20:38:42.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa28aae6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f99ff550 con 0x5634f99f5a60 2024-03-18T20:38:42.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa28aae6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f99f6d30 con 0x5634f9a01220 2024-03-18T20:38:42.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa2777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f6380 0x5634f99f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa277fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f5a60 0x5634f9a07e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa2777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f6380 0x5634f99f1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34266/0 (socket says 172.21.15.46:34266) 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa277fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f5a60 0x5634f9a07e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:57372/0 (socket says 172.21.15.46:57372) 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa2777fe700 1 -- 172.21.15.46:0/847326173 learned_addr learned my addr 172.21.15.46:0/847326173 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa27ceee700 1 --2- 172.21.15.46:0/847326173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f9a01220 0x5634f99f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa277fff700 1 -- 172.21.15.46:0/847326173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f9a01220 msgr2=0x5634f99f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa277fff700 1 --2- 172.21.15.46:0/847326173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f9a01220 0x5634f99f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa277fff700 1 -- 172.21.15.46:0/847326173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f6380 msgr2=0x5634f99f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa277fff700 1 --2- 172.21.15.46:0/847326173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f6380 0x5634f99f1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa277fff700 1 -- 172.21.15.46:0/847326173 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634f9776b60 con 0x5634f99f5a60 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.508+0000 7fa2777fe700 1 --2- 172.21.15.46:0/847326173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f6380 0x5634f99f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.509+0000 7fa277fff700 1 --2- 172.21.15.46:0/847326173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f5a60 0x5634f9a07e80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5634f9a13ba0 tx=0x7fa26800ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b504a738b4ed4859 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.509+0000 7fa26effd700 1 -- 172.21.15.46:0/847326173 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800edb0 con 0x5634f99f5a60 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.509+0000 7fa26effd700 1 -- 172.21.15.46:0/847326173 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2680330f0 con 0x5634f99f5a60 2024-03-18T20:38:42.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.509+0000 7fa26effd700 1 -- 172.21.15.46:0/847326173 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268005630 con 0x5634f99f5a60 2024-03-18T20:38:42.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.510+0000 7fa28aae6b80 1 -- 172.21.15.46:0/847326173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f5a60 msgr2=0x5634f9a07e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.510+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/847326173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f5a60 0x5634f9a07e80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5634f9a13ba0 tx=0x7fa26800ed40 comp rx=0 tx=0).stop 2024-03-18T20:38:42.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.510+0000 7fa28aae6b80 1 -- 172.21.15.46:0/847326173 shutdown_connections 2024-03-18T20:38:42.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.510+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/847326173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f9a01220 0x5634f99f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.510+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/847326173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f99f5a60 0x5634f9a07e80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.510+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/847326173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f6380 0x5634f99f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.510+0000 7fa28aae6b80 1 -- 172.21.15.46:0/847326173 >> 172.21.15.46:0/847326173 conn(0x5634f996c220 msgr2=0x5634f9a0c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.510+0000 7fa28aae6b80 1 -- 172.21.15.46:0/847326173 shutdown_connections 2024-03-18T20:38:42.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.510+0000 7fa28aae6b80 1 -- 172.21.15.46:0/847326173 wait complete. 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.510+0000 7fa28aae6b80 1 -- start start 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.510+0000 7fa28aae6b80 1 Processor -- start 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa28aae6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 0x5634f9a05e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa28aae6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f99f6380 0x5634f9a063a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa28aae6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f9a01220 0x5634f9a63ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa28aae6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f9a06b10 con 0x5634f99f5a60 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa28aae6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f9a06c80 con 0x5634f9a01220 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa28aae6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f9a641e0 con 0x5634f99f6380 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa28aae6b80 10 client.0 _ll_register_callbacks cb 0x7fa260009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa277fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 0x5634f9a05e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa2777fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f99f6380 0x5634f9a063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa277fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 0x5634f9a05e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34272/0 (socket says 172.21.15.46:34272) 2024-03-18T20:38:42.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa277fff700 1 -- 172.21.15.46:0/826195394 learned_addr learned my addr 172.21.15.46:0/826195394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa27ceee700 1 --2- 172.21.15.46:0/826195394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f9a01220 0x5634f9a63ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa277fff700 1 -- 172.21.15.46:0/826195394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f99f6380 msgr2=0x5634f9a063a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa277fff700 1 --2- 172.21.15.46:0/826195394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f99f6380 0x5634f9a063a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa277fff700 1 -- 172.21.15.46:0/826195394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f9a01220 msgr2=0x5634f9a63ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa277fff700 1 --2- 172.21.15.46:0/826195394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f9a01220 0x5634f9a63ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.511+0000 7fa277fff700 1 -- 172.21.15.46:0/826195394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa268017070 con 0x5634f99f5a60 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.512+0000 7fa277fff700 1 --2- 172.21.15.46:0/826195394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 0x5634f9a05e60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa26800e480 tx=0x7fa268033260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611d4fd91256cb96 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.512+0000 7fa2577fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.512+0000 7fa2577fe700 1 -- 172.21.15.46:0/826195394 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268033410 con 0x5634f99f5a60 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.512+0000 7fa2577fe700 1 -- 172.21.15.46:0/826195394 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fa268033570 con 0x5634f99f5a60 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.512+0000 7fa28aae6b80 1 -- 172.21.15.46:0/826195394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa268021070 con 0x5634f99f5a60 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.512+0000 7fa2577fe700 1 -- 172.21.15.46:0/826195394 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa268033a90 con 0x5634f99f5a60 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.512+0000 7fa28aae6b80 10 client.16458 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.512+0000 7fa28aae6b80 1 -- 172.21.15.46:0/826195394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fa26802a040 con 0x5634f99f5a60 2024-03-18T20:38:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.512+0000 7fa2577fe700 1 -- 172.21.15.46:0/826195394 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa268021070 con 0x5634f99f5a60 2024-03-18T20:38:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa2577fe700 1 -- 172.21.15.46:0/826195394 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa268044070 con 0x5634f99f5a60 2024-03-18T20:38:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa28aae6b80 10 client.16458 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa28aae6b80 10 client.16458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:38:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa28aae6b80 20 client.16458 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/826195394 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5634f9a65660 0x5634f9a67a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa28aae6b80 4 client.16458 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:38:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa28aae6b80 1 -- 172.21.15.46:0/826195394 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5634f9797cf0 con 0x5634f9a65660 2024-03-18T20:38:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa2777fe700 1 --2- 172.21.15.46:0/826195394 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5634f9a65660 0x5634f9a67a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa2777fe700 1 --2- 172.21.15.46:0/826195394 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5634f9a65660 0x5634f9a67a80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a181bfb2149c324f server_cookie=cce464cb999ca7fb in_seq=0 out_seq=0 2024-03-18T20:38:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa2577fe700 10 client.16458 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:38:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa2577fe700 1 -- 172.21.15.46:0/826195394 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5634f9797cf0 con 0x5634f9a65660 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa2577fe700 10 client.16458 handle_command_reply: tid=0 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.513+0000 7fa28aae6b80 1 client.16458 shutdown 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.514+0000 7fa28aae6b80 1 -- 172.21.15.46:0/826195394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 msgr2=0x5634f9a05e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.514+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/826195394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 0x5634f9a05e60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa26800e480 tx=0x7fa268033260 comp rx=0 tx=0).stop 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.514+0000 7fa28aae6b80 1 -- 172.21.15.46:0/826195394 shutdown_connections 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.514+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/826195394 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5634f9a65660 0x5634f9a67a80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.514+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/826195394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5634f99f5a60 0x5634f9a05e60 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.514+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/826195394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5634f9a01220 0x5634f9a63ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.514+0000 7fa2577fe700 0 client.16458 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.514+0000 7fa28aae6b80 1 --2- 172.21.15.46:0/826195394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5634f99f6380 0x5634f9a063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.514+0000 7fa28aae6b80 1 -- 172.21.15.46:0/826195394 >> 172.21.15.46:0/826195394 conn(0x5634f996c220 msgr2=0x5634f99f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.514+0000 7fa28aae6b80 1 -- 172.21.15.46:0/826195394 shutdown_connections 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.514+0000 7fa28aae6b80 1 -- 172.21.15.46:0/826195394 wait complete. 2024-03-18T20:38:42.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.514+0000 7fa28aae6b80 20 client.16458 trim_cache size 0 max 16384 2024-03-18T20:38:42.523 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:38:42.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.515+0000 7fa27e951700 1 -- 172.21.15.46:0/3410065654 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2640a1310 msgr2=0x7fa2640a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.515+0000 7fa27e951700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2640a1310 0x7fa2640a37d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa268009c60 tx=0x7fa268009260 comp rx=0 tx=0).stop 2024-03-18T20:38:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.515+0000 7fa27e951700 1 -- 172.21.15.46:0/3410065654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27819bb90 msgr2=0x7fa27819c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.515+0000 7fa27e951700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27819bb90 0x7fa27819c010 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa27000b070 tx=0x7fa27000bda0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.515+0000 7fa27e951700 1 -- 172.21.15.46:0/3410065654 shutdown_connections 2024-03-18T20:38:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.515+0000 7fa27e951700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2640a1310 0x7fa2640a37d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.515+0000 7fa27e951700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27819bb90 0x7fa27819c010 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.515+0000 7fa27e951700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2780ffe50 0x7fa2781972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.515+0000 7fa27e951700 1 --2- 172.21.15.46:0/3410065654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa278104f60 0x7fa2781977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.515+0000 7fa27e951700 1 -- 172.21.15.46:0/3410065654 >> 172.21.15.46:0/3410065654 conn(0x7fa278076d40 msgr2=0x7fa2780680d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.515+0000 7fa27e951700 1 -- 172.21.15.46:0/3410065654 shutdown_connections 2024-03-18T20:38:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.515+0000 7fa27e951700 1 -- 172.21.15.46:0/3410065654 wait complete. 2024-03-18T20:38:42.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:42 smithi046 ceph-mon[43495]: pgmap v81: 97 pgs: 97 active+clean; 35 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 447 KiB/s wr, 85 op/s 2024-03-18T20:38:42.530 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:42 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3407128046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:38:42.532 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:38:42.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:38:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:42 smithi155 ceph-mon[48344]: pgmap v81: 97 pgs: 97 active+clean; 35 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 447 KiB/s wr, 85 op/s 2024-03-18T20:38:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:42 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3407128046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:38:42.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 66466 3.85 MB/sec warmup 97 sec latency 4954.024 ms 2024-03-18T20:38:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.673+0000 7f57d4db0700 1 -- 172.21.15.46:0/2114629288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57d0106080 msgr2=0x7f57d010dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.673+0000 7f57d4db0700 1 --2- 172.21.15.46:0/2114629288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57d0106080 0x7f57d010dc00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f57c800b0a0 tx=0x7f57c800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.673+0000 7f57d4db0700 1 -- 172.21.15.46:0/2114629288 shutdown_connections 2024-03-18T20:38:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.673+0000 7f57d4db0700 1 --2- 172.21.15.46:0/2114629288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57d0106080 0x7f57d010dc00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.673+0000 7f57d4db0700 1 --2- 172.21.15.46:0/2114629288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d0100f60 0x7f57d0101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.673+0000 7f57d4db0700 1 --2- 172.21.15.46:0/2114629288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0101910 0x7f57d0105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.673+0000 7f57d4db0700 1 -- 172.21.15.46:0/2114629288 >> 172.21.15.46:0/2114629288 conn(0x7f57d00fc7c0 msgr2=0x7f57d00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.673+0000 7f57d4db0700 1 -- 172.21.15.46:0/2114629288 shutdown_connections 2024-03-18T20:38:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.673+0000 7f57d4db0700 1 -- 172.21.15.46:0/2114629288 wait complete. 2024-03-18T20:38:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.674+0000 7f57d4db0700 1 Processor -- start 2024-03-18T20:38:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.674+0000 7f57d4db0700 1 -- start start 2024-03-18T20:38:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57d4db0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d0100f60 0x7f57d00730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57d4db0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57d0106080 0x7f57d00735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57d4db0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0073b30 0x7f57d01a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57d4db0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d01a5af0 con 0x7f57d0106080 2024-03-18T20:38:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57d4db0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d01a5c30 con 0x7f57d0100f60 2024-03-18T20:38:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57d4db0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d01a5df0 con 0x7f57d0073b30 2024-03-18T20:38:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57cdd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57d0106080 0x7f57d00735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57cdd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57d0106080 0x7f57d00735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34282/0 (socket says 172.21.15.46:34282) 2024-03-18T20:38:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57ce59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d0100f60 0x7f57d00730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57cdd9b700 1 -- 172.21.15.46:0/1831992300 learned_addr learned my addr 172.21.15.46:0/1831992300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57ced9d700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0073b30 0x7f57d01a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57ce59c700 1 -- 172.21.15.46:0/1831992300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0073b30 msgr2=0x7f57d01a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57ce59c700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0073b30 0x7f57d01a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57ce59c700 1 -- 172.21.15.46:0/1831992300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57d0106080 msgr2=0x7f57d00735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57ce59c700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57d0106080 0x7f57d00735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57ce59c700 1 -- 172.21.15.46:0/1831992300 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c8009d00 con 0x7f57d0100f60 2024-03-18T20:38:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57d0106080 0x7f57d00735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.675+0000 7f57ce59c700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d0100f60 0x7f57d00730b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f57b800e9f0 tx=0x7f57b800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.676+0000 7f57c77fe700 1 -- 172.21.15.46:0/1831992300 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b800c9c0 con 0x7f57d0100f60 2024-03-18T20:38:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.676+0000 7f57c77fe700 1 -- 172.21.15.46:0/1831992300 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57b8010430 con 0x7f57d0100f60 2024-03-18T20:38:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.676+0000 7f57d4db0700 1 -- 172.21.15.46:0/1831992300 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d01a60d0 con 0x7f57d0100f60 2024-03-18T20:38:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.676+0000 7f57c77fe700 1 -- 172.21.15.46:0/1831992300 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8018680 con 0x7f57d0100f60 2024-03-18T20:38:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.676+0000 7f57d4db0700 1 -- 172.21.15.46:0/1831992300 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d01a6630 con 0x7f57d0100f60 2024-03-18T20:38:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.678+0000 7f57c77fe700 1 -- 172.21.15.46:0/1831992300 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f57b8010860 con 0x7f57d0100f60 2024-03-18T20:38:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.678+0000 7f57c77fe700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57bc0a1390 0x7f57bc0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.679+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57bc0a1390 0x7f57bc0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.679+0000 7f57c77fe700 1 -- 172.21.15.46:0/1831992300 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f57b8014070 con 0x7f57d0100f60 2024-03-18T20:38:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.679+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57bc0a1390 0x7f57bc0a3850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f57d0074790 tx=0x7f57c0009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.679+0000 7f57e0f45b80 1 Processor -- start 2024-03-18T20:38:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57e0f45b80 1 -- start start 2024-03-18T20:38:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57e0f45b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8443e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57e0f45b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a844ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57e0f45b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a844f220 0x5585a8451610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57e0f45b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a839d990 con 0x5585a844f220 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57e0f45b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a839b350 con 0x5585a8444380 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57e0f45b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8396c40 con 0x5585a8443a60 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57cdd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a844ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57ced9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a844f220 0x5585a8451610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57cdd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a844ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:57388/0 (socket says 172.21.15.46:57388) 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57cdd9b700 1 -- 172.21.15.46:0/3947411976 learned_addr learned my addr 172.21.15.46:0/3947411976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57ce59c700 1 --2- 172.21.15.46:0/3947411976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8443e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57cdd9b700 1 -- 172.21.15.46:0/3947411976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 msgr2=0x5585a8443e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/3947411976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8443e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57cdd9b700 1 -- 172.21.15.46:0/3947411976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a844f220 msgr2=0x5585a8451610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/3947411976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a844f220 0x5585a8451610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.680+0000 7f57cdd9b700 1 -- 172.21.15.46:0/3947411976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585a82183b0 con 0x5585a8444380 2024-03-18T20:38:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.681+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/3947411976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a844ece0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f57c00043b0 tx=0x7f57c0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198259bf939b4bac server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.681+0000 7f57c4ff9700 1 -- 172.21.15.46:0/3947411976 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0003d10 con 0x5585a8444380 2024-03-18T20:38:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.681+0000 7f57c4ff9700 1 -- 172.21.15.46:0/3947411976 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c0003e70 con 0x5585a8444380 2024-03-18T20:38:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.681+0000 7f57c4ff9700 1 -- 172.21.15.46:0/3947411976 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c001e890 con 0x5585a8444380 2024-03-18T20:38:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.682+0000 7f57e0f45b80 1 -- 172.21.15.46:0/3947411976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 msgr2=0x5585a844ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.682+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/3947411976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a844ece0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f57c00043b0 tx=0x7f57c0014000 comp rx=0 tx=0).stop 2024-03-18T20:38:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.682+0000 7f57e0f45b80 1 -- 172.21.15.46:0/3947411976 shutdown_connections 2024-03-18T20:38:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.682+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/3947411976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a844f220 0x5585a8451610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.682+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/3947411976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8443e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.682+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/3947411976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a844ece0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.682+0000 7f57e0f45b80 1 -- 172.21.15.46:0/3947411976 >> 172.21.15.46:0/3947411976 conn(0x5585a83ba220 msgr2=0x5585a8440020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.682+0000 7f57e0f45b80 1 -- 172.21.15.46:0/3947411976 shutdown_connections 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.682+0000 7f57e0f45b80 1 -- 172.21.15.46:0/3947411976 wait complete. 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.682+0000 7f57e0f45b80 1 -- start start 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.682+0000 7f57e0f45b80 1 Processor -- start 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57e0f45b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8443a60 0x5585a844bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57e0f45b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a844c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57e0f45b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a844f220 0x5585a844ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57e0f45b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8332d80 con 0x5585a8443a60 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57e0f45b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8332a20 con 0x5585a8444380 2024-03-18T20:38:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57e0f45b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a821a6a0 con 0x5585a844f220 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57e0f45b80 10 client.0 _ll_register_callbacks cb 0x5585a819e3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57ce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8443a60 0x5585a844bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57ce59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8443a60 0x5585a844bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34298/0 (socket says 172.21.15.46:34298) 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57cdd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a844c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57ce59c700 1 -- 172.21.15.46:0/4165241001 learned_addr learned my addr 172.21.15.46:0/4165241001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57ced9d700 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a844f220 0x5585a844ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57ce59c700 1 -- 172.21.15.46:0/4165241001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a844f220 msgr2=0x5585a844ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57ce59c700 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a844f220 0x5585a844ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57ce59c700 1 -- 172.21.15.46:0/4165241001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 msgr2=0x5585a844c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57ce59c700 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a844c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57ce59c700 1 -- 172.21.15.46:0/4165241001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c0017070 con 0x5585a8443a60 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a844c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57ce59c700 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8443a60 0x5585a844bf90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f57b8096930 tx=0x7f57b80caa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67883ad7a37efb3c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.683+0000 7f57b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57b5ffb700 1 -- 172.21.15.46:0/4165241001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80995b0 con 0x5585a8443a60 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57b5ffb700 1 -- 172.21.15.46:0/4165241001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f57b809dcf0 con 0x5585a8443a60 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57e0f45b80 1 -- 172.21.15.46:0/4165241001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585a8437ad0 con 0x5585a8443a60 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57b5ffb700 1 -- 172.21.15.46:0/4165241001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57b80a2340 con 0x5585a8443a60 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57e0f45b80 10 client.16470 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57e0f45b80 1 -- 172.21.15.46:0/4165241001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5585a81c4b60 con 0x5585a8443a60 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57b5ffb700 1 -- 172.21.15.46:0/4165241001 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57b80b8030 con 0x5585a8443a60 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57b5ffb700 1 -- 172.21.15.46:0/4165241001 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f57b8095cb0 con 0x5585a8443a60 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57e0f45b80 10 client.16470 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57e0f45b80 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57e0f45b80 20 client.16470 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5585a845ef50 0x5585a8461370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57e0f45b80 4 client.16470 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.684+0000 7f57e0f45b80 1 -- 172.21.15.46:0/4165241001 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585a838f700 con 0x5585a845ef50 2024-03-18T20:38:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.685+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5585a845ef50 0x5585a8461370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.685+0000 7f57b5ffb700 10 client.16470 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.685+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5585a845ef50 0x5585a8461370 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e597f537208e632 server_cookie=116efc2dd520c2d5 in_seq=0 out_seq=0 2024-03-18T20:38:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.685+0000 7f57b5ffb700 1 -- 172.21.15.46:0/4165241001 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5585a838f700 con 0x5585a845ef50 2024-03-18T20:38:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.685+0000 7f57b5ffb700 10 client.16470 handle_command_reply: tid=0 2024-03-18T20:38:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.685+0000 7f57e0f45b80 1 client.16470 shutdown 2024-03-18T20:38:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.686+0000 7f57e0f45b80 1 -- 172.21.15.46:0/4165241001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8443a60 msgr2=0x5585a844bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.686+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8443a60 0x5585a844bf90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f57b8096930 tx=0x7f57b80caa60 comp rx=0 tx=0).stop 2024-03-18T20:38:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.686+0000 7f57e0f45b80 1 -- 172.21.15.46:0/4165241001 shutdown_connections 2024-03-18T20:38:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.686+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5585a845ef50 0x5585a8461370 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.686+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8443a60 0x5585a844bf90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.686+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a844f220 0x5585a844ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.686+0000 7f57b5ffb700 0 client.16470 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.686+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/4165241001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a844c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.686+0000 7f57e0f45b80 1 -- 172.21.15.46:0/4165241001 >> 172.21.15.46:0/4165241001 conn(0x5585a83ba220 msgr2=0x5585a843fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.686+0000 7f57e0f45b80 1 -- 172.21.15.46:0/4165241001 shutdown_connections 2024-03-18T20:38:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.686+0000 7f57e0f45b80 1 -- 172.21.15.46:0/4165241001 wait complete. 2024-03-18T20:38:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.686+0000 7f57e0f45b80 20 client.16470 trim_cache size 0 max 16384 2024-03-18T20:38:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.699+0000 7f57e0f45b80 1 Processor -- start 2024-03-18T20:38:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.699+0000 7f57e0f45b80 1 -- start start 2024-03-18T20:38:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57e0f45b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8461340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57e0f45b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a8455e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57e0f45b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a844f220 0x5585a843fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57e0f45b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a845b9b0 con 0x5585a844f220 2024-03-18T20:38:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57e0f45b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8441230 con 0x5585a8444380 2024-03-18T20:38:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57e0f45b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8445550 con 0x5585a8443a60 2024-03-18T20:38:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57cdd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a8455e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57ce59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8461340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57cdd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a8455e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:57404/0 (socket says 172.21.15.46:57404) 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57ce59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8461340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:41542/0 (socket says 172.21.15.46:41542) 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57cdd9b700 1 -- 172.21.15.46:0/3201645930 learned_addr learned my addr 172.21.15.46:0/3201645930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57ced9d700 1 --2- 172.21.15.46:0/3201645930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a844f220 0x5585a843fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57cdd9b700 1 -- 172.21.15.46:0/3201645930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 msgr2=0x5585a8461340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/3201645930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8461340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57cdd9b700 1 -- 172.21.15.46:0/3201645930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a844f220 msgr2=0x5585a843fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/3201645930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a844f220 0x5585a843fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57cdd9b700 1 -- 172.21.15.46:0/3201645930 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585a835cea0 con 0x5585a8444380 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.700+0000 7f57ce59c700 1 --2- 172.21.15.46:0/3201645930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8461340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.701+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/3201645930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a8455e50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f57c0005950 tx=0x7f57c0005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65bc4018bd38c12e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.701+0000 7f57c57fa700 1 -- 172.21.15.46:0/3201645930 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c002a040 con 0x5585a8444380 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.701+0000 7f57c57fa700 1 -- 172.21.15.46:0/3201645930 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c002b040 con 0x5585a8444380 2024-03-18T20:38:42.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.701+0000 7f57c57fa700 1 -- 172.21.15.46:0/3201645930 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0005450 con 0x5585a8444380 2024-03-18T20:38:42.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.702+0000 7f57e0f45b80 1 -- 172.21.15.46:0/3201645930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 msgr2=0x5585a8455e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.702+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/3201645930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a8455e50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f57c0005950 tx=0x7f57c0005e40 comp rx=0 tx=0).stop 2024-03-18T20:38:42.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.702+0000 7f57e0f45b80 1 -- 172.21.15.46:0/3201645930 shutdown_connections 2024-03-18T20:38:42.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.702+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/3201645930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a844f220 0x5585a843fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.702+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/3201645930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8461340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.702+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/3201645930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a8444380 0x5585a8455e50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.702+0000 7f57e0f45b80 1 -- 172.21.15.46:0/3201645930 >> 172.21.15.46:0/3201645930 conn(0x5585a83ba220 msgr2=0x5585a845a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.702+0000 7f57e0f45b80 1 -- 172.21.15.46:0/3201645930 shutdown_connections 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.702+0000 7f57e0f45b80 1 -- 172.21.15.46:0/3201645930 wait complete. 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.702+0000 7f57e0f45b80 1 -- start start 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.702+0000 7f57e0f45b80 1 Processor -- start 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57e0f45b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8454020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57e0f45b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8444380 0x5585a8454560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57e0f45b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a844f220 0x5585a84b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57e0f45b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8454c40 con 0x5585a8444380 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57e0f45b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a84b22c0 con 0x5585a844f220 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57e0f45b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a84b2400 con 0x5585a8443a60 2024-03-18T20:38:42.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57e0f45b80 10 client.0 _ll_register_callbacks cb 0x7f57b8099af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57cdd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8444380 0x5585a8454560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57ced9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a844f220 0x5585a84b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57ced9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a844f220 0x5585a84b1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:57418/0 (socket says 172.21.15.46:57418) 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57cdd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8444380 0x5585a8454560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34324/0 (socket says 172.21.15.46:34324) 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57ced9d700 1 -- 172.21.15.46:0/2010073859 learned_addr learned my addr 172.21.15.46:0/2010073859 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57ce59c700 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8454020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57ced9d700 1 -- 172.21.15.46:0/2010073859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 msgr2=0x5585a8454020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57ced9d700 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8454020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57ced9d700 1 -- 172.21.15.46:0/2010073859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8444380 msgr2=0x5585a8454560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57ced9d700 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8444380 0x5585a8454560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57ced9d700 1 -- 172.21.15.46:0/2010073859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585a81c4b60 con 0x5585a844f220 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57cdd9b700 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8444380 0x5585a8454560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57ced9d700 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a844f220 0x5585a84b1d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f57c800bc60 tx=0x7f57c800bc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c7c2b9ec66c6e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:42.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.703+0000 7f57b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57b5ffb700 1 -- 172.21.15.46:0/2010073859 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8022070 con 0x5585a844f220 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57b5ffb700 1 -- 172.21.15.46:0/2010073859 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f57c80074e0 con 0x5585a844f220 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57e0f45b80 1 -- 172.21.15.46:0/2010073859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57c0003e70 con 0x5585a844f220 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57b5ffb700 1 -- 172.21.15.46:0/2010073859 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57c800e040 con 0x5585a844f220 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57e0f45b80 10 client.26231 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57e0f45b80 1 -- 172.21.15.46:0/2010073859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5585a82183b0 con 0x5585a844f220 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57b5ffb700 1 -- 172.21.15.46:0/2010073859 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c0003e70 con 0x5585a844f220 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57b5ffb700 1 -- 172.21.15.46:0/2010073859 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f57c800b0d0 con 0x5585a844f220 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57e0f45b80 10 client.26231 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57e0f45b80 10 client.26231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57e0f45b80 20 client.26231 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5585a84b3bc0 0x5585a84b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:42.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57e0f45b80 4 client.26231 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:38:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57e0f45b80 1 -- 172.21.15.46:0/2010073859 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5585a81e5cf0 con 0x5585a84b3bc0 2024-03-18T20:38:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.704+0000 7f57ce59c700 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5585a84b3bc0 0x5585a84b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57b5ffb700 10 client.26231 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57ce59c700 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5585a84b3bc0 0x5585a84b5fe0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc44cec841b77fd4 server_cookie=1aecc2c3f01af5ed in_seq=0 out_seq=0 2024-03-18T20:38:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57b5ffb700 1 -- 172.21.15.46:0/2010073859 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5585a81e5cf0 con 0x5585a84b3bc0 2024-03-18T20:38:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57b5ffb700 10 client.26231 handle_command_reply: tid=0 2024-03-18T20:38:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57e0f45b80 1 client.26231 shutdown 2024-03-18T20:38:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57e0f45b80 1 -- 172.21.15.46:0/2010073859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a844f220 msgr2=0x5585a84b1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a844f220 0x5585a84b1d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f57c800bc60 tx=0x7f57c800bc90 comp rx=0 tx=0).stop 2024-03-18T20:38:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57e0f45b80 1 -- 172.21.15.46:0/2010073859 shutdown_connections 2024-03-18T20:38:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5585a84b3bc0 0x5585a84b5fe0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5585a844f220 0x5585a84b1d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5585a8443a60 0x5585a8454020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57b5ffb700 0 client.26231 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:42.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57e0f45b80 1 --2- 172.21.15.46:0/2010073859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5585a8444380 0x5585a8454560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57e0f45b80 1 -- 172.21.15.46:0/2010073859 >> 172.21.15.46:0/2010073859 conn(0x5585a83ba220 msgr2=0x5585a8447280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57e0f45b80 1 -- 172.21.15.46:0/2010073859 shutdown_connections 2024-03-18T20:38:42.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.705+0000 7f57e0f45b80 1 -- 172.21.15.46:0/2010073859 wait complete. 2024-03-18T20:38:42.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57e0f45b80 20 client.26231 trim_cache size 0 max 16384 2024-03-18T20:38:42.716 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:38:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57d4db0700 1 -- 172.21.15.46:0/1831992300 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57bc0a1390 msgr2=0x7f57bc0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57d4db0700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57bc0a1390 0x7f57bc0a3850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f57d0074790 tx=0x7f57c0009260 comp rx=0 tx=0).stop 2024-03-18T20:38:42.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57d4db0700 1 -- 172.21.15.46:0/1831992300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d0100f60 msgr2=0x7f57d00730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:42.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57d4db0700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d0100f60 0x7f57d00730b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f57b800e9f0 tx=0x7f57b800ed00 comp rx=0 tx=0).stop 2024-03-18T20:38:42.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57d4db0700 1 -- 172.21.15.46:0/1831992300 shutdown_connections 2024-03-18T20:38:42.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57d4db0700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57bc0a1390 0x7f57bc0a3850 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57d4db0700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d0073b30 0x7f57d01a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57d4db0700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d0100f60 0x7f57d00730b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57d4db0700 1 --2- 172.21.15.46:0/1831992300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57d0106080 0x7f57d00735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:42.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57d4db0700 1 -- 172.21.15.46:0/1831992300 >> 172.21.15.46:0/1831992300 conn(0x7f57d00fc7c0 msgr2=0x7f57d00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:42.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57d4db0700 1 -- 172.21.15.46:0/1831992300 shutdown_connections 2024-03-18T20:38:42.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:42.706+0000 7f57d4db0700 1 -- 172.21.15.46:0/1831992300 wait complete. 2024-03-18T20:38:43.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 68087 3.92 MB/sec warmup 98 sec latency 34.134 ms 2024-03-18T20:38:43.724 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:38:43.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:38:43.725 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:38:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.872+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/904626699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd34100800 msgr2=0x7fcd34104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:43.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.872+0000 7fcd3aa5a700 1 --2- 172.21.15.46:0/904626699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd34100800 0x7fcd34104830 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009990 tx=0x7fcd28009ca0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.872+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/904626699 shutdown_connections 2024-03-18T20:38:43.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.872+0000 7fcd3aa5a700 1 --2- 172.21.15.46:0/904626699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd34104f60 0x7fcd3410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.872+0000 7fcd3aa5a700 1 --2- 172.21.15.46:0/904626699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd340ffe50 0x7fcd34100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.872+0000 7fcd3aa5a700 1 --2- 172.21.15.46:0/904626699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd34100800 0x7fcd34104830 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.872+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/904626699 >> 172.21.15.46:0/904626699 conn(0x7fcd34076d40 msgr2=0x7fcd34077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:43.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.872+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/904626699 shutdown_connections 2024-03-18T20:38:43.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.872+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/904626699 wait complete. 2024-03-18T20:38:43.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.872+0000 7fcd3aa5a700 1 Processor -- start 2024-03-18T20:38:43.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.872+0000 7fcd3aa5a700 1 -- start start 2024-03-18T20:38:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd3aa5a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd340ffe50 0x7fcd34197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd3aa5a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd34104f60 0x7fcd341977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd3aa5a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd3419baf0 0x7fcd3419bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd3aa5a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3419c570 con 0x7fcd3419baf0 2024-03-18T20:38:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd3aa5a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3419c6e0 con 0x7fcd34104f60 2024-03-18T20:38:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd3aa5a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3419c8a0 con 0x7fcd340ffe50 2024-03-18T20:38:43.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd38ff7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd3419baf0 0x7fcd3419bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd38ff7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd3419baf0 0x7fcd3419bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34348/0 (socket says 172.21.15.46:34348) 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd38ff7700 1 -- 172.21.15.46:0/1162336606 learned_addr learned my addr 172.21.15.46:0/1162336606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd337fe700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd34104f60 0x7fcd341977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd33fff700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd340ffe50 0x7fcd34197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd38ff7700 1 -- 172.21.15.46:0/1162336606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd340ffe50 msgr2=0x7fcd34197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd38ff7700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd340ffe50 0x7fcd34197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd38ff7700 1 -- 172.21.15.46:0/1162336606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd34104f60 msgr2=0x7fcd341977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd38ff7700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd34104f60 0x7fcd341977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd38ff7700 1 -- 172.21.15.46:0/1162336606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd280095f0 con 0x7fcd3419baf0 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.873+0000 7fcd337fe700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd34104f60 0x7fcd341977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.874+0000 7fcd38ff7700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd3419baf0 0x7fcd3419bf70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c007ee0 tx=0x7fcd2c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.874+0000 7fcd317fa700 1 -- 172.21.15.46:0/1162336606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c017070 con 0x7fcd3419baf0 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.874+0000 7fcd317fa700 1 -- 172.21.15.46:0/1162336606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd2c00f040 con 0x7fcd3419baf0 2024-03-18T20:38:43.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.874+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/1162336606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd341a3e00 con 0x7fcd3419baf0 2024-03-18T20:38:43.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.874+0000 7fcd317fa700 1 -- 172.21.15.46:0/1162336606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c005580 con 0x7fcd3419baf0 2024-03-18T20:38:43.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.874+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/1162336606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd341a4290 con 0x7fcd3419baf0 2024-03-18T20:38:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.876+0000 7fcd317fa700 1 -- 172.21.15.46:0/1162336606 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fcd2c00db20 con 0x7fcd3419baf0 2024-03-18T20:38:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.877+0000 7fcd317fa700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcd1c0a1290 0x7fcd1c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.877+0000 7fcd317fa700 1 -- 172.21.15.46:0/1162336606 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fcd2c0cc3a0 con 0x7fcd3419baf0 2024-03-18T20:38:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.877+0000 7fcd33fff700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcd1c0a1290 0x7fcd1c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.877+0000 7fcd33fff700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcd1c0a1290 0x7fcd1c0a3750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcd24005950 tx=0x7fcd2400b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd46befb80 1 Processor -- start 2024-03-18T20:38:43.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd46befb80 1 -- start start 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd46befb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4efeae0 0x5632f4efeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd46befb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4eff400 0x5632f4f09d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd46befb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4f0a2a0 0x5632f4f0c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd46befb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f4e58a10 con 0x5632f4eff400 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd46befb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f4e563d0 con 0x5632f4efeae0 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd46befb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f4e51cc0 con 0x5632f4f0a2a0 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd337fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4eff400 0x5632f4f09d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd33fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4efeae0 0x5632f4efeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd337fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4eff400 0x5632f4f09d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34360/0 (socket says 172.21.15.46:34360) 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd38ff7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4f0a2a0 0x5632f4f0c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd337fe700 1 -- 172.21.15.46:0/3401132931 learned_addr learned my addr 172.21.15.46:0/3401132931 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd33fff700 1 -- 172.21.15.46:0/3401132931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4f0a2a0 msgr2=0x5632f4f0c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:43.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd33fff700 1 --2- 172.21.15.46:0/3401132931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4f0a2a0 0x5632f4f0c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd33fff700 1 -- 172.21.15.46:0/3401132931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4eff400 msgr2=0x5632f4f09d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd33fff700 1 --2- 172.21.15.46:0/3401132931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4eff400 0x5632f4f09d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.878+0000 7fcd33fff700 1 -- 172.21.15.46:0/3401132931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632f4cd3430 con 0x5632f4efeae0 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.879+0000 7fcd33fff700 1 --2- 172.21.15.46:0/3401132931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4efeae0 0x5632f4efeec0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd24005d90 tx=0x7fcd24013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c397df794144dd18 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.879+0000 7fcd1a7fc700 1 -- 172.21.15.46:0/3401132931 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24011b80 con 0x5632f4efeae0 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.879+0000 7fcd1a7fc700 1 -- 172.21.15.46:0/3401132931 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd24015070 con 0x5632f4efeae0 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.879+0000 7fcd1a7fc700 1 -- 172.21.15.46:0/3401132931 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24005600 con 0x5632f4efeae0 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.880+0000 7fcd46befb80 1 -- 172.21.15.46:0/3401132931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4efeae0 msgr2=0x5632f4efeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.880+0000 7fcd46befb80 1 --2- 172.21.15.46:0/3401132931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4efeae0 0x5632f4efeec0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd24005d90 tx=0x7fcd24013000 comp rx=0 tx=0).stop 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.880+0000 7fcd46befb80 1 -- 172.21.15.46:0/3401132931 shutdown_connections 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.880+0000 7fcd46befb80 1 --2- 172.21.15.46:0/3401132931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4f0a2a0 0x5632f4f0c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.880+0000 7fcd46befb80 1 --2- 172.21.15.46:0/3401132931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4efeae0 0x5632f4efeec0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.880+0000 7fcd46befb80 1 --2- 172.21.15.46:0/3401132931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4eff400 0x5632f4f09d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.880+0000 7fcd46befb80 1 -- 172.21.15.46:0/3401132931 >> 172.21.15.46:0/3401132931 conn(0x5632f4e752a0 msgr2=0x5632f4efb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.880+0000 7fcd46befb80 1 -- 172.21.15.46:0/3401132931 shutdown_connections 2024-03-18T20:38:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.880+0000 7fcd46befb80 1 -- 172.21.15.46:0/3401132931 wait complete. 2024-03-18T20:38:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.880+0000 7fcd46befb80 1 -- start start 2024-03-18T20:38:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.880+0000 7fcd46befb80 1 Processor -- start 2024-03-18T20:38:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd46befb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f09d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd46befb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4eff400 0x5632f4f06ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd46befb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4f0a2a0 0x5632f4f07730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd46befb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f4dede00 con 0x5632f4efeae0 2024-03-18T20:38:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd46befb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f4dedaa0 con 0x5632f4eff400 2024-03-18T20:38:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd46befb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f4cd5720 con 0x5632f4f0a2a0 2024-03-18T20:38:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd46befb80 10 client.0 _ll_register_callbacks cb 0x5632f4c59420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f09d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd337fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4eff400 0x5632f4f06ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f09d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34374/0 (socket says 172.21.15.46:34374) 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd33fff700 1 -- 172.21.15.46:0/2338886400 learned_addr learned my addr 172.21.15.46:0/2338886400 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd38ff7700 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4f0a2a0 0x5632f4f07730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd33fff700 1 -- 172.21.15.46:0/2338886400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4f0a2a0 msgr2=0x5632f4f07730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd33fff700 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4f0a2a0 0x5632f4f07730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd33fff700 1 -- 172.21.15.46:0/2338886400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4eff400 msgr2=0x5632f4f06ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd33fff700 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4eff400 0x5632f4f06ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd33fff700 1 -- 172.21.15.46:0/2338886400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd24010c50 con 0x5632f4efeae0 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd337fe700 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4eff400 0x5632f4f06ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd33fff700 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f09d00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fcd24005100 tx=0x7fcd24012dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12bf98c1f68dc2f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd137fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.881+0000 7fcd137fe700 1 -- 172.21.15.46:0/2338886400 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2402c070 con 0x5632f4efeae0 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd137fe700 1 -- 172.21.15.46:0/2338886400 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fcd24017070 con 0x5632f4efeae0 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd46befb80 1 -- 172.21.15.46:0/2338886400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632f4ef2b50 con 0x5632f4efeae0 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd137fe700 1 -- 172.21.15.46:0/2338886400 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd240054d0 con 0x5632f4efeae0 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd46befb80 10 client.16488 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd46befb80 1 -- 172.21.15.46:0/2338886400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5632f4c7fbe0 con 0x5632f4efeae0 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd137fe700 1 -- 172.21.15.46:0/2338886400 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd2403e020 con 0x5632f4efeae0 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd137fe700 1 -- 172.21.15.46:0/2338886400 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcd24024070 con 0x5632f4efeae0 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd46befb80 10 client.16488 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd46befb80 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd46befb80 20 client.16488 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd46befb80 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632f4f19e20 0x5632f4f1c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd46befb80 4 client.16488 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd46befb80 1 -- 172.21.15.46:0/2338886400 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632f4e4a780 con 0x5632f4f19e20 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.882+0000 7fcd337fe700 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632f4f19e20 0x5632f4f1c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.883+0000 7fcd137fe700 10 client.16488 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.883+0000 7fcd337fe700 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632f4f19e20 0x5632f4f1c240 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157830f6b5cce413 server_cookie=645cd37b030ec1bf in_seq=0 out_seq=0 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.883+0000 7fcd137fe700 1 -- 172.21.15.46:0/2338886400 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5632f4e4a780 con 0x5632f4f19e20 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.883+0000 7fcd137fe700 10 client.16488 handle_command_reply: tid=0 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.883+0000 7fcd46befb80 1 client.16488 shutdown 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.884+0000 7fcd46befb80 1 -- 172.21.15.46:0/2338886400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 msgr2=0x5632f4f09d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.884+0000 7fcd46befb80 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f09d00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fcd24005100 tx=0x7fcd24012dc0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.884+0000 7fcd46befb80 1 -- 172.21.15.46:0/2338886400 shutdown_connections 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.884+0000 7fcd46befb80 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632f4f19e20 0x5632f4f1c240 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.884+0000 7fcd46befb80 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f09d00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.884+0000 7fcd46befb80 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4f0a2a0 0x5632f4f07730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.884+0000 7fcd137fe700 0 client.16488 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.884+0000 7fcd46befb80 1 --2- 172.21.15.46:0/2338886400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4eff400 0x5632f4f06ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.884+0000 7fcd46befb80 1 -- 172.21.15.46:0/2338886400 >> 172.21.15.46:0/2338886400 conn(0x5632f4e752a0 msgr2=0x5632f4efac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.884+0000 7fcd46befb80 1 -- 172.21.15.46:0/2338886400 shutdown_connections 2024-03-18T20:38:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.884+0000 7fcd46befb80 1 -- 172.21.15.46:0/2338886400 wait complete. 2024-03-18T20:38:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.884+0000 7fcd46befb80 20 client.16488 trim_cache size 0 max 16384 2024-03-18T20:38:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.902+0000 7fcd46befb80 1 Processor -- start 2024-03-18T20:38:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.902+0000 7fcd46befb80 1 -- start start 2024-03-18T20:38:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.902+0000 7fcd46befb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f10f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.902+0000 7fcd46befb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4eff400 0x5632f4efabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.902+0000 7fcd46befb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4f0a2a0 0x5632f4efb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.902+0000 7fcd46befb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f4f006d0 con 0x5632f4efeae0 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.902+0000 7fcd46befb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f4effdb0 con 0x5632f4f0a2a0 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.902+0000 7fcd46befb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f4ef5030 con 0x5632f4eff400 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.902+0000 7fcd33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f10f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f10f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34376/0 (socket says 172.21.15.46:34376) 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd33fff700 1 -- 172.21.15.46:0/1155180463 learned_addr learned my addr 172.21.15.46:0/1155180463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd38ff7700 1 --2- 172.21.15.46:0/1155180463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4f0a2a0 0x5632f4efb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd337fe700 1 --2- 172.21.15.46:0/1155180463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4eff400 0x5632f4efabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd33fff700 1 -- 172.21.15.46:0/1155180463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4eff400 msgr2=0x5632f4efabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd33fff700 1 --2- 172.21.15.46:0/1155180463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4eff400 0x5632f4efabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd33fff700 1 -- 172.21.15.46:0/1155180463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4f0a2a0 msgr2=0x5632f4efb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:43.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd33fff700 1 --2- 172.21.15.46:0/1155180463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4f0a2a0 0x5632f4efb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd33fff700 1 -- 172.21.15.46:0/1155180463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632f4e17f20 con 0x5632f4efeae0 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd38ff7700 1 --2- 172.21.15.46:0/1155180463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4f0a2a0 0x5632f4efb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd33fff700 1 --2- 172.21.15.46:0/1155180463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f10f00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcd24010f40 tx=0x7fcd240318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6ee35e166a97d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd1affd700 1 -- 172.21.15.46:0/1155180463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2402c070 con 0x5632f4efeae0 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd1affd700 1 -- 172.21.15.46:0/1155180463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd2403e070 con 0x5632f4efeae0 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.903+0000 7fcd1affd700 1 -- 172.21.15.46:0/1155180463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24004bd0 con 0x5632f4efeae0 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.904+0000 7fcd46befb80 1 -- 172.21.15.46:0/1155180463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 msgr2=0x5632f4f10f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.904+0000 7fcd46befb80 1 --2- 172.21.15.46:0/1155180463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f10f00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcd24010f40 tx=0x7fcd240318e0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.904+0000 7fcd46befb80 1 -- 172.21.15.46:0/1155180463 shutdown_connections 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.904+0000 7fcd46befb80 1 --2- 172.21.15.46:0/1155180463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f10f00 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.904+0000 7fcd46befb80 1 --2- 172.21.15.46:0/1155180463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4f0a2a0 0x5632f4efb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.904+0000 7fcd46befb80 1 --2- 172.21.15.46:0/1155180463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4eff400 0x5632f4efabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.904+0000 7fcd46befb80 1 -- 172.21.15.46:0/1155180463 >> 172.21.15.46:0/1155180463 conn(0x5632f4e752a0 msgr2=0x5632f4f15650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:43.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.904+0000 7fcd46befb80 1 -- 172.21.15.46:0/1155180463 shutdown_connections 2024-03-18T20:38:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.904+0000 7fcd46befb80 1 -- 172.21.15.46:0/1155180463 wait complete. 2024-03-18T20:38:43.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.904+0000 7fcd46befb80 1 -- start start 2024-03-18T20:38:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd46befb80 1 Processor -- start 2024-03-18T20:38:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd46befb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f0f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd46befb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4eff400 0x5632f4f0fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd46befb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4f0a2a0 0x5632f4f10410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd46befb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f4f6dda0 con 0x5632f4efeae0 2024-03-18T20:38:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd46befb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f4f6df10 con 0x5632f4f0a2a0 2024-03-18T20:38:43.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd46befb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632f4f6e0d0 con 0x5632f4eff400 2024-03-18T20:38:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd46befb80 10 client.0 _ll_register_callbacks cb 0x7fcd28009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f0f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f0f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34392/0 (socket says 172.21.15.46:34392) 2024-03-18T20:38:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd337fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4eff400 0x5632f4f0fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd33fff700 1 -- 172.21.15.46:0/3541813450 learned_addr learned my addr 172.21.15.46:0/3541813450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:43.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.905+0000 7fcd38ff7700 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4f0a2a0 0x5632f4f10410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd33fff700 1 -- 172.21.15.46:0/3541813450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4eff400 msgr2=0x5632f4f0fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd33fff700 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4eff400 0x5632f4f0fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd33fff700 1 -- 172.21.15.46:0/3541813450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4f0a2a0 msgr2=0x5632f4f10410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd33fff700 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4f0a2a0 0x5632f4f10410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd33fff700 1 -- 172.21.15.46:0/3541813450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd341a4290 con 0x5632f4efeae0 2024-03-18T20:38:43.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd337fe700 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4eff400 0x5632f4f0fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd33fff700 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f0f990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fcd24031550 tx=0x7fcd24005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e4bca680ad86d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd137fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd137fe700 1 -- 172.21.15.46:0/3541813450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2402c070 con 0x5632f4efeae0 2024-03-18T20:38:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd137fe700 1 -- 172.21.15.46:0/3541813450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fcd24017040 con 0x5632f4efeae0 2024-03-18T20:38:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd46befb80 1 -- 172.21.15.46:0/3541813450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd24015070 con 0x5632f4efeae0 2024-03-18T20:38:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd137fe700 1 -- 172.21.15.46:0/3541813450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd2400be60 con 0x5632f4efeae0 2024-03-18T20:38:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd46befb80 10 client.16494 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.906+0000 7fcd46befb80 1 -- 172.21.15.46:0/3541813450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5632f4c7fbe0 con 0x5632f4efeae0 2024-03-18T20:38:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.907+0000 7fcd137fe700 1 -- 172.21.15.46:0/3541813450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd2404f080 con 0x5632f4efeae0 2024-03-18T20:38:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.907+0000 7fcd137fe700 1 -- 172.21.15.46:0/3541813450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcd24024070 con 0x5632f4efeae0 2024-03-18T20:38:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.907+0000 7fcd46befb80 10 client.16494 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.907+0000 7fcd46befb80 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.907+0000 7fcd46befb80 20 client.16494 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.907+0000 7fcd46befb80 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632f4f6f4d0 0x5632f4f718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.907+0000 7fcd46befb80 4 client.16494 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:38:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.907+0000 7fcd46befb80 1 -- 172.21.15.46:0/3541813450 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5632f4ca0d70 con 0x5632f4f6f4d0 2024-03-18T20:38:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.907+0000 7fcd337fe700 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632f4f6f4d0 0x5632f4f718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.907+0000 7fcd337fe700 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632f4f6f4d0 0x5632f4f718f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7970d3efd712d830 server_cookie=4d76571172c4759d in_seq=0 out_seq=0 2024-03-18T20:38:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.907+0000 7fcd137fe700 10 client.16494 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.908+0000 7fcd137fe700 1 -- 172.21.15.46:0/3541813450 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5632f4ca0d70 con 0x5632f4f6f4d0 2024-03-18T20:38:43.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.908+0000 7fcd137fe700 10 client.16494 handle_command_reply: tid=0 2024-03-18T20:38:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.908+0000 7fcd46befb80 1 client.16494 shutdown 2024-03-18T20:38:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.908+0000 7fcd46befb80 1 -- 172.21.15.46:0/3541813450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 msgr2=0x5632f4f0f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.908+0000 7fcd46befb80 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f0f990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fcd24031550 tx=0x7fcd24005780 comp rx=0 tx=0).stop 2024-03-18T20:38:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd46befb80 1 -- 172.21.15.46:0/3541813450 shutdown_connections 2024-03-18T20:38:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd46befb80 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632f4f6f4d0 0x5632f4f718f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd46befb80 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632f4efeae0 0x5632f4f0f990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd46befb80 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632f4f0a2a0 0x5632f4f10410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd46befb80 1 --2- 172.21.15.46:0/3541813450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632f4eff400 0x5632f4f0fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd137fe700 0 client.16494 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:43.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd46befb80 1 -- 172.21.15.46:0/3541813450 >> 172.21.15.46:0/3541813450 conn(0x5632f4e752a0 msgr2=0x5632f4efab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:43.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd46befb80 1 -- 172.21.15.46:0/3541813450 shutdown_connections 2024-03-18T20:38:43.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd46befb80 1 -- 172.21.15.46:0/3541813450 wait complete. 2024-03-18T20:38:43.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd46befb80 20 client.16494 trim_cache size 0 max 16384 2024-03-18T20:38:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:38:43.923 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:38:43.923 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "456c40d1-ad00-451f-bc31-8babe007bae4", 2024-03-18T20:38:43.923 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:38:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:38:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/1162336606 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcd1c0a1290 msgr2=0x7fcd1c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd3aa5a700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcd1c0a1290 0x7fcd1c0a3750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcd24005950 tx=0x7fcd2400b410 comp rx=0 tx=0).stop 2024-03-18T20:38:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/1162336606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd3419baf0 msgr2=0x7fcd3419bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.909+0000 7fcd3aa5a700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd3419baf0 0x7fcd3419bf70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c007ee0 tx=0x7fcd2c00d3b0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.910+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/1162336606 shutdown_connections 2024-03-18T20:38:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.910+0000 7fcd3aa5a700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcd1c0a1290 0x7fcd1c0a3750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.910+0000 7fcd3aa5a700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd3419baf0 0x7fcd3419bf70 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.910+0000 7fcd3aa5a700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd340ffe50 0x7fcd34197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.910+0000 7fcd3aa5a700 1 --2- 172.21.15.46:0/1162336606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd34104f60 0x7fcd341977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.910+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/1162336606 >> 172.21.15.46:0/1162336606 conn(0x7fcd34076d40 msgr2=0x7fcd340681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.910+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/1162336606 shutdown_connections 2024-03-18T20:38:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:43.910+0000 7fcd3aa5a700 1 -- 172.21.15.46:0/1162336606 wait complete. 2024-03-18T20:38:43.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:38:43.931 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:38:44.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.077+0000 7f5142295700 1 -- 172.21.15.46:0/1876509159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c100800 msgr2=0x7f513c104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.077+0000 7f5142295700 1 --2- 172.21.15.46:0/1876509159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c100800 0x7f513c104830 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5130009990 tx=0x7f5130009ca0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.077+0000 7f5142295700 1 -- 172.21.15.46:0/1876509159 shutdown_connections 2024-03-18T20:38:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.077+0000 7f5142295700 1 --2- 172.21.15.46:0/1876509159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c104f60 0x7f513c10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.077+0000 7f5142295700 1 --2- 172.21.15.46:0/1876509159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f513c0ffe50 0x7f513c100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.077+0000 7f5142295700 1 --2- 172.21.15.46:0/1876509159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c100800 0x7f513c104830 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.077+0000 7f5142295700 1 -- 172.21.15.46:0/1876509159 >> 172.21.15.46:0/1876509159 conn(0x7f513c076d40 msgr2=0x7f513c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.077+0000 7f5142295700 1 -- 172.21.15.46:0/1876509159 shutdown_connections 2024-03-18T20:38:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.077+0000 7f5142295700 1 -- 172.21.15.46:0/1876509159 wait complete. 2024-03-18T20:38:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.077+0000 7f5142295700 1 Processor -- start 2024-03-18T20:38:44.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f5142295700 1 -- start start 2024-03-18T20:38:44.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f5142295700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c0ffe50 0x7f513c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f5142295700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f513c104f60 0x7f513c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f5142295700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c19dcd0 0x7f513c19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f5142295700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c19e750 con 0x7f513c19dcd0 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f5142295700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c19e8c0 con 0x7f513c0ffe50 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f5142295700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c19ea80 con 0x7f513c104f60 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f513bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c0ffe50 0x7f513c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f5140832700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c19dcd0 0x7f513c19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f513bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c0ffe50 0x7f513c1993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:57484/0 (socket says 172.21.15.46:57484) 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f513b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f513c104f60 0x7f513c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f5140832700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c19dcd0 0x7f513c19e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34410/0 (socket says 172.21.15.46:34410) 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f513bfff700 1 -- 172.21.15.46:0/379922480 learned_addr learned my addr 172.21.15.46:0/379922480 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f513bfff700 1 -- 172.21.15.46:0/379922480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f513c104f60 msgr2=0x7f513c199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f513bfff700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f513c104f60 0x7f513c199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f513bfff700 1 -- 172.21.15.46:0/379922480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c19dcd0 msgr2=0x7f513c19e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f513bfff700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c19dcd0 0x7f513c19e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f513bfff700 1 -- 172.21.15.46:0/379922480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5134009c40 con 0x7f513c0ffe50 2024-03-18T20:38:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.078+0000 7f5140832700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c19dcd0 0x7f513c19e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.079+0000 7f513bfff700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c0ffe50 0x7f513c1993f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f512c00e9c0 tx=0x7f512c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.079+0000 7f51397fa700 1 -- 172.21.15.46:0/379922480 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0097b0 con 0x7f513c0ffe50 2024-03-18T20:38:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.079+0000 7f51397fa700 1 -- 172.21.15.46:0/379922480 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f512c00ee50 con 0x7f513c0ffe50 2024-03-18T20:38:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.079+0000 7f5142295700 1 -- 172.21.15.46:0/379922480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51300095f0 con 0x7f513c0ffe50 2024-03-18T20:38:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.079+0000 7f51397fa700 1 -- 172.21.15.46:0/379922480 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0054f0 con 0x7f513c0ffe50 2024-03-18T20:38:44.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.079+0000 7f5142295700 1 -- 172.21.15.46:0/379922480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c1a6270 con 0x7f513c0ffe50 2024-03-18T20:38:44.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.081+0000 7f51397fa700 1 -- 172.21.15.46:0/379922480 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f512c004750 con 0x7f513c0ffe50 2024-03-18T20:38:44.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.082+0000 7f51397fa700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f51240a1310 0x7f51240a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.082+0000 7f51397fa700 1 -- 172.21.15.46:0/379922480 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f512c014070 con 0x7f513c0ffe50 2024-03-18T20:38:44.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.082+0000 7f513b7fe700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f51240a1310 0x7f51240a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f513b7fe700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f51240a1310 0x7f51240a37d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f513000b2e0 tx=0x7f5130005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:44.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f514e42ab80 1 Processor -- start 2024-03-18T20:38:44.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f514e42ab80 1 -- start start 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f514e42ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fda60 0x5636756fde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f514e42ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x563675708ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f514e42ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 0x56367570b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f514e42ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563675657990 con 0x563675709220 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f514e42ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563675655350 con 0x5636756fda60 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f514e42ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563675650c40 con 0x5636756fe380 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f513b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x563675708ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f513bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fda60 0x5636756fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f5140832700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 0x56367570b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f513bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fda60 0x5636756fde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:57500/0 (socket says 172.21.15.46:57500) 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f513bfff700 1 -- 172.21.15.46:0/342075222 learned_addr learned my addr 172.21.15.46:0/342075222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f513b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x563675708ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:41622/0 (socket says 172.21.15.46:41622) 2024-03-18T20:38:44.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.083+0000 7f5140832700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 0x56367570b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34418/0 (socket says 172.21.15.46:34418) 2024-03-18T20:38:44.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.084+0000 7f513bfff700 1 -- 172.21.15.46:0/342075222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 msgr2=0x563675708ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:44.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.084+0000 7f513bfff700 1 --2- 172.21.15.46:0/342075222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x563675708ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.084+0000 7f513bfff700 1 -- 172.21.15.46:0/342075222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 msgr2=0x56367570b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:44.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.084+0000 7f513bfff700 1 --2- 172.21.15.46:0/342075222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 0x56367570b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.084+0000 7f513bfff700 1 -- 172.21.15.46:0/342075222 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636754d23b0 con 0x5636756fda60 2024-03-18T20:38:44.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.084+0000 7f513bfff700 1 --2- 172.21.15.46:0/342075222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fda60 0x5636756fde40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f512c096a10 tx=0x7f512c09a790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0464d25cdcf2eee server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:44.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.084+0000 7f51227fc700 1 -- 172.21.15.46:0/342075222 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0b1070 con 0x5636756fda60 2024-03-18T20:38:44.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.084+0000 7f51227fc700 1 -- 172.21.15.46:0/342075222 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f512c09de30 con 0x5636756fda60 2024-03-18T20:38:44.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.084+0000 7f51227fc700 1 -- 172.21.15.46:0/342075222 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c016330 con 0x5636756fda60 2024-03-18T20:38:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.085+0000 7f514e42ab80 1 -- 172.21.15.46:0/342075222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fda60 msgr2=0x5636756fde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.085+0000 7f514e42ab80 1 --2- 172.21.15.46:0/342075222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fda60 0x5636756fde40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f512c096a10 tx=0x7f512c09a790 comp rx=0 tx=0).stop 2024-03-18T20:38:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.085+0000 7f514e42ab80 1 -- 172.21.15.46:0/342075222 shutdown_connections 2024-03-18T20:38:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.085+0000 7f514e42ab80 1 --2- 172.21.15.46:0/342075222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 0x56367570b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.085+0000 7f514e42ab80 1 --2- 172.21.15.46:0/342075222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fda60 0x5636756fde40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.085+0000 7f514e42ab80 1 --2- 172.21.15.46:0/342075222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x563675708ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.085+0000 7f514e42ab80 1 -- 172.21.15.46:0/342075222 >> 172.21.15.46:0/342075222 conn(0x563675674220 msgr2=0x5636756fa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.085+0000 7f514e42ab80 1 -- 172.21.15.46:0/342075222 shutdown_connections 2024-03-18T20:38:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.085+0000 7f514e42ab80 1 -- 172.21.15.46:0/342075222 wait complete. 2024-03-18T20:38:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.085+0000 7f514e42ab80 1 -- start start 2024-03-18T20:38:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f514e42ab80 1 Processor -- start 2024-03-18T20:38:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f514e42ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 0x563675706090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f514e42ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x5636757065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f514e42ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 0x563675706b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f514e42ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636755ecd80 con 0x5636756fda60 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f514e42ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636755eca20 con 0x563675709220 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f514e42ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636754d46a0 con 0x5636756fe380 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f514e42ab80 10 client.0 _ll_register_callbacks cb 0x5636754583a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f513bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 0x563675706090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f5140832700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 0x563675706b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f513b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x5636757065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f513bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 0x563675706090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34426/0 (socket says 172.21.15.46:34426) 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.086+0000 7f513bfff700 1 -- 172.21.15.46:0/1146325955 learned_addr learned my addr 172.21.15.46:0/1146325955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f5140832700 1 -- 172.21.15.46:0/1146325955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 msgr2=0x5636757065d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f5140832700 1 --2- 172.21.15.46:0/1146325955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x5636757065d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f5140832700 1 -- 172.21.15.46:0/1146325955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 msgr2=0x563675706090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f5140832700 1 --2- 172.21.15.46:0/1146325955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 0x563675706090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f5140832700 1 -- 172.21.15.46:0/1146325955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c1a6270 con 0x563675709220 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f513bfff700 1 --2- 172.21.15.46:0/1146325955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 0x563675706090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f5140832700 1 --2- 172.21.15.46:0/1146325955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 0x563675706b10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5134007860 tx=0x7f5134003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b30c605517078d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f511b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f511b7fe700 1 -- 172.21.15.46:0/1146325955 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134004510 con 0x563675709220 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f514e42ab80 1 -- 172.21.15.46:0/1146325955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636756f1ad0 con 0x563675709220 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f511b7fe700 1 -- 172.21.15.46:0/1146325955 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5134004670 con 0x563675709220 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f511b7fe700 1 -- 172.21.15.46:0/1146325955 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5134004830 con 0x563675709220 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f514e42ab80 10 client.26261 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f514e42ab80 1 -- 172.21.15.46:0/1146325955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56367547eb60 con 0x563675709220 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.087+0000 7f511b7fe700 1 -- 172.21.15.46:0/1146325955 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5636756f1ad0 con 0x563675709220 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.088+0000 7f511b7fe700 1 -- 172.21.15.46:0/1146325955 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f513401c020 con 0x563675709220 2024-03-18T20:38:44.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.088+0000 7f514e42ab80 10 client.26261 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.088+0000 7f514e42ab80 10 client.26261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.088+0000 7f514e42ab80 20 client.26261 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.088+0000 7f514e42ab80 1 --2- 172.21.15.46:0/1146325955 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563675718e60 0x56367571b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.088+0000 7f514e42ab80 4 client.26261 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.088+0000 7f514e42ab80 1 -- 172.21.15.46:0/1146325955 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563675649700 con 0x563675718e60 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.088+0000 7f513bfff700 1 --2- 172.21.15.46:0/1146325955 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563675718e60 0x56367571b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.088+0000 7f511b7fe700 10 client.26261 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.088+0000 7f513bfff700 1 --2- 172.21.15.46:0/1146325955 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563675718e60 0x56367571b280 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b1f202dc4783b5d server_cookie=3549529fec7a7c35 in_seq=0 out_seq=0 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f511b7fe700 1 -- 172.21.15.46:0/1146325955 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563675649700 con 0x563675718e60 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f511b7fe700 10 client.26261 handle_command_reply: tid=0 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f514e42ab80 1 client.26261 shutdown 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f514e42ab80 1 -- 172.21.15.46:0/1146325955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 msgr2=0x563675706b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f514e42ab80 1 --2- 172.21.15.46:0/1146325955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 0x563675706b10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5134007860 tx=0x7f5134003f90 comp rx=0 tx=0).stop 2024-03-18T20:38:44.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f514e42ab80 1 -- 172.21.15.46:0/1146325955 shutdown_connections 2024-03-18T20:38:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f514e42ab80 1 --2- 172.21.15.46:0/1146325955 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563675718e60 0x56367571b280 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f514e42ab80 1 --2- 172.21.15.46:0/1146325955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 0x563675706090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f514e42ab80 1 --2- 172.21.15.46:0/1146325955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 0x563675706b10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f514e42ab80 1 --2- 172.21.15.46:0/1146325955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x5636757065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f511b7fe700 0 client.26261 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f514e42ab80 1 -- 172.21.15.46:0/1146325955 >> 172.21.15.46:0/1146325955 conn(0x563675674220 msgr2=0x5636756f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f514e42ab80 1 -- 172.21.15.46:0/1146325955 shutdown_connections 2024-03-18T20:38:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f514e42ab80 1 -- 172.21.15.46:0/1146325955 wait complete. 2024-03-18T20:38:44.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.089+0000 7f514e42ab80 20 client.26261 trim_cache size 0 max 16384 2024-03-18T20:38:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.107+0000 7f514e42ab80 1 Processor -- start 2024-03-18T20:38:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f514e42ab80 1 -- start start 2024-03-18T20:38:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f514e42ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fda60 0x56367570fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f514e42ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fe380 0x563675706950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f514e42ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 0x5636757013e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f514e42ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563675710390 con 0x563675709220 2024-03-18T20:38:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f514e42ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636756f1610 con 0x5636756fe380 2024-03-18T20:38:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f514e42ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563675704640 con 0x5636756fda60 2024-03-18T20:38:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f513bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fda60 0x56367570fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f5140832700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 0x5636757013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f513bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fda60 0x56367570fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:41640/0 (socket says 172.21.15.46:41640) 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f5140832700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 0x5636757013e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34432/0 (socket says 172.21.15.46:34432) 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f513bfff700 1 -- 172.21.15.46:0/647440975 learned_addr learned my addr 172.21.15.46:0/647440975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f513b7fe700 1 --2- 172.21.15.46:0/647440975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fe380 0x563675706950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f513bfff700 1 -- 172.21.15.46:0/647440975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fe380 msgr2=0x563675706950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f513bfff700 1 --2- 172.21.15.46:0/647440975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fe380 0x563675706950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f513bfff700 1 -- 172.21.15.46:0/647440975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 msgr2=0x5636757013e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f513bfff700 1 --2- 172.21.15.46:0/647440975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 0x5636757013e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f513bfff700 1 -- 172.21.15.46:0/647440975 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563675616ea0 con 0x5636756fda60 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f5140832700 1 --2- 172.21.15.46:0/647440975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 0x5636757013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.108+0000 7f513b7fe700 1 --2- 172.21.15.46:0/647440975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fe380 0x563675706950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.109+0000 7f513bfff700 1 --2- 172.21.15.46:0/647440975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fda60 0x56367570fe50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f512c099ca0 tx=0x7f512c09a120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86cdb5bb3b778edf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:44.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.109+0000 7f5122ffd700 1 -- 172.21.15.46:0/647440975 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0b1070 con 0x5636756fda60 2024-03-18T20:38:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.109+0000 7f5122ffd700 1 -- 172.21.15.46:0/647440975 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f512c0962e0 con 0x5636756fda60 2024-03-18T20:38:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.109+0000 7f5122ffd700 1 -- 172.21.15.46:0/647440975 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c016330 con 0x5636756fda60 2024-03-18T20:38:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.109+0000 7f514e42ab80 1 -- 172.21.15.46:0/647440975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fda60 msgr2=0x56367570fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.109+0000 7f514e42ab80 1 --2- 172.21.15.46:0/647440975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fda60 0x56367570fe50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f512c099ca0 tx=0x7f512c09a120 comp rx=0 tx=0).stop 2024-03-18T20:38:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.109+0000 7f514e42ab80 1 -- 172.21.15.46:0/647440975 shutdown_connections 2024-03-18T20:38:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.109+0000 7f514e42ab80 1 --2- 172.21.15.46:0/647440975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563675709220 0x5636757013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.109+0000 7f514e42ab80 1 --2- 172.21.15.46:0/647440975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fda60 0x56367570fe50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.109+0000 7f514e42ab80 1 --2- 172.21.15.46:0/647440975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5636756fe380 0x563675706950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.109+0000 7f514e42ab80 1 -- 172.21.15.46:0/647440975 >> 172.21.15.46:0/647440975 conn(0x563675674220 msgr2=0x5636756f8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.109+0000 7f514e42ab80 1 -- 172.21.15.46:0/647440975 shutdown_connections 2024-03-18T20:38:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.110+0000 7f514e42ab80 1 -- 172.21.15.46:0/647440975 wait complete. 2024-03-18T20:38:44.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.110+0000 7f514e42ab80 1 -- start start 2024-03-18T20:38:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.110+0000 7f514e42ab80 1 Processor -- start 2024-03-18T20:38:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.110+0000 7f514e42ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 0x56367570dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.110+0000 7f514e42ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x56367570e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.110+0000 7f514e42ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 0x56367576e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.110+0000 7f514e42ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367576e970 con 0x5636756fda60 2024-03-18T20:38:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.110+0000 7f514e42ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367576eab0 con 0x563675709220 2024-03-18T20:38:44.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.110+0000 7f514e42ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367576ec70 con 0x5636756fe380 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.110+0000 7f514e42ab80 10 client.0 _ll_register_callbacks cb 0x7f5134003da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f513bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 0x56367570dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f5140832700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 0x56367576e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f513b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x56367570e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f513bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 0x56367570dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34444/0 (socket says 172.21.15.46:34444) 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f5140832700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 0x56367576e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:57520/0 (socket says 172.21.15.46:57520) 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f513bfff700 1 -- 172.21.15.46:0/3958426936 learned_addr learned my addr 172.21.15.46:0/3958426936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f513b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x56367570e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:41656/0 (socket says 172.21.15.46:41656) 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f513bfff700 1 -- 172.21.15.46:0/3958426936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 msgr2=0x56367570e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f513bfff700 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x56367570e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f513bfff700 1 -- 172.21.15.46:0/3958426936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 msgr2=0x56367576e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f513bfff700 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 0x56367576e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f513bfff700 1 -- 172.21.15.46:0/3958426936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56367547eb60 con 0x5636756fda60 2024-03-18T20:38:44.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f513b7fe700 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x56367570e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f5140832700 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 0x56367576e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f513bfff700 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 0x56367570dca0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f512c0a1ef0 tx=0x7f512c09a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3756d9bf70c4ef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f511b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f511b7fe700 1 -- 172.21.15.46:0/3958426936 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0b1030 con 0x5636756fda60 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f511b7fe700 1 -- 172.21.15.46:0/3958426936 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f512c0b2070 con 0x5636756fda60 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f514e42ab80 1 -- 172.21.15.46:0/3958426936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f512c09de30 con 0x5636756fda60 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f511b7fe700 1 -- 172.21.15.46:0/3958426936 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f512c0983e0 con 0x5636756fda60 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f514e42ab80 10 client.16512 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.111+0000 7f514e42ab80 1 -- 172.21.15.46:0/3958426936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5636754d23b0 con 0x5636756fda60 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.112+0000 7f511b7fe700 1 -- 172.21.15.46:0/3958426936 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f512c0a6020 con 0x5636756fda60 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.112+0000 7f511b7fe700 1 -- 172.21.15.46:0/3958426936 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f512c099490 con 0x5636756fda60 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.112+0000 7f514e42ab80 10 client.16512 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.112+0000 7f514e42ab80 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.112+0000 7f514e42ab80 20 client.16512 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:44.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.112+0000 7f514e42ab80 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563675776040 0x563675778430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.112+0000 7f514e42ab80 4 client.16512 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.112+0000 7f514e42ab80 1 -- 172.21.15.46:0/3958426936 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56367549fcf0 con 0x563675776040 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.112+0000 7f513b7fe700 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563675776040 0x563675778430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.112+0000 7f511b7fe700 10 client.16512 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.112+0000 7f513b7fe700 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563675776040 0x563675778430 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2f6d4a72a2e45bf server_cookie=c805ab34b842b856 in_seq=0 out_seq=0 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f511b7fe700 1 -- 172.21.15.46:0/3958426936 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56367549fcf0 con 0x563675776040 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f511b7fe700 10 client.16512 handle_command_reply: tid=0 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f514e42ab80 1 client.16512 shutdown 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f514e42ab80 1 -- 172.21.15.46:0/3958426936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 msgr2=0x56367570dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f514e42ab80 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 0x56367570dca0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f512c0a1ef0 tx=0x7f512c09a0b0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f514e42ab80 1 -- 172.21.15.46:0/3958426936 shutdown_connections 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f514e42ab80 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563675776040 0x563675778430 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f514e42ab80 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5636756fda60 0x56367570dca0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f514e42ab80 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563675709220 0x56367576e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f511b7fe700 0 client.16512 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:44.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f514e42ab80 1 --2- 172.21.15.46:0/3958426936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5636756fe380 0x56367570e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f514e42ab80 1 -- 172.21.15.46:0/3958426936 >> 172.21.15.46:0/3958426936 conn(0x563675674220 msgr2=0x5636757069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:44.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f514e42ab80 1 -- 172.21.15.46:0/3958426936 shutdown_connections 2024-03-18T20:38:44.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f514e42ab80 1 -- 172.21.15.46:0/3958426936 wait complete. 2024-03-18T20:38:44.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.113+0000 7f514e42ab80 20 client.16512 trim_cache size 0 max 16384 2024-03-18T20:38:44.123 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:38:44.123 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:38:44.123 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:38:44.123 INFO:teuthology.orchestra.run.smithi046.stdout: "456c40d1-ad00-451f-bc31-8babe007bae4": { 2024-03-18T20:38:44.123 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:38:44.124 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "456c40d1-ad00-451f-bc31-8babe007bae4", 2024-03-18T20:38:44.124 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:38:44.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:38:44.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:38:44.124 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:38:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.114+0000 7f5142295700 1 -- 172.21.15.46:0/379922480 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f51240a1310 msgr2=0x7f51240a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.114+0000 7f5142295700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f51240a1310 0x7f51240a37d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f513000b2e0 tx=0x7f5130005960 comp rx=0 tx=0).stop 2024-03-18T20:38:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.114+0000 7f5142295700 1 -- 172.21.15.46:0/379922480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c0ffe50 msgr2=0x7f513c1993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.114+0000 7f5142295700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c0ffe50 0x7f513c1993f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f512c00e9c0 tx=0x7f512c00ecd0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.114+0000 7f5142295700 1 -- 172.21.15.46:0/379922480 shutdown_connections 2024-03-18T20:38:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.114+0000 7f5142295700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f51240a1310 0x7f51240a37d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.114+0000 7f5142295700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c19dcd0 0x7f513c19e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.114+0000 7f5142295700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c0ffe50 0x7f513c1993f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.114+0000 7f5142295700 1 --2- 172.21.15.46:0/379922480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f513c104f60 0x7f513c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.114+0000 7f5142295700 1 -- 172.21.15.46:0/379922480 >> 172.21.15.46:0/379922480 conn(0x7f513c076d40 msgr2=0x7f513c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.114+0000 7f5142295700 1 -- 172.21.15.46:0/379922480 shutdown_connections 2024-03-18T20:38:44.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:38:44.114+0000 7f5142295700 1 -- 172.21.15.46:0/379922480 wait complete. 2024-03-18T20:38:44.132 INFO:tasks.cephfs.filesystem:scrub status for tag:456c40d1-ad00-451f-bc31-8babe007bae4 - {'path': '/', 'tag': '456c40d1-ad00-451f-bc31-8babe007bae4', 'options': 'recursive,force'} 2024-03-18T20:38:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:44 smithi073 ceph-mon[47384]: pgmap v82: 97 pgs: 97 active+clean; 36 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 883 KiB/s wr, 123 op/s 2024-03-18T20:38:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:44 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:38:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:44 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:44 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:44 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:44 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:44 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:44 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:44.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:44 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:44 smithi155 ceph-mon[48344]: pgmap v82: 97 pgs: 97 active+clean; 36 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 883 KiB/s wr, 123 op/s 2024-03-18T20:38:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:44 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:38:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:44 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:44 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:44 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:44 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:44 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:44 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:44.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:44 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:44 smithi046 ceph-mon[43495]: pgmap v82: 97 pgs: 97 active+clean; 36 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 883 KiB/s wr, 123 op/s 2024-03-18T20:38:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:44 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:38:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:44 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:44 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:44 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:44 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:44.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:44 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:44.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:44 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:44.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:44 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:44.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 69870 3.95 MB/sec warmup 99 sec latency 300.280 ms 2024-03-18T20:38:45.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:45 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:45.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:45 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:45 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:45.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:45 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:45.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:45 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:45.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:45 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:45 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:45 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:45 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:38:45.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:45 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:45 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:45 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:45 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:45 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:45 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:45 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:45 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:45 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:45 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:38:45.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:45 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:45 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:45 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:45 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:45 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:45 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:45 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:45 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:45 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:45 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:38:45.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:45 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:38:45.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 71515 4.03 MB/sec warmup 100 sec latency 11.732 ms 2024-03-18T20:38:46.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:46 smithi073 ceph-mon[47384]: pgmap v83: 97 pgs: 97 active+clean; 36 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 543 KiB/s wr, 63 op/s 2024-03-18T20:38:46.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:46 smithi155 ceph-mon[48344]: pgmap v83: 97 pgs: 97 active+clean; 36 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 543 KiB/s wr, 63 op/s 2024-03-18T20:38:46.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:46 smithi046 ceph-mon[43495]: pgmap v83: 97 pgs: 97 active+clean; 36 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 543 KiB/s wr, 63 op/s 2024-03-18T20:38:46.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 71685 4.00 MB/sec warmup 101 sec latency 877.280 ms 2024-03-18T20:38:47.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 73387 4.02 MB/sec warmup 102 sec latency 1241.817 ms 2024-03-18T20:38:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:48 smithi073 ceph-mon[47384]: pgmap v84: 97 pgs: 97 active+clean; 36 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 543 KiB/s wr, 63 op/s 2024-03-18T20:38:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.495+0000 7f0333521700 1 -- 172.21.15.46:0/197446375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f032c1089d0 msgr2=0x7f032c10cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.495+0000 7f0333521700 1 --2- 172.21.15.46:0/197446375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f032c1089d0 0x7f032c10cee0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f031c0098f0 tx=0x7f031c009c00 comp rx=0 tx=0).stop 2024-03-18T20:38:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.495+0000 7f0333521700 1 -- 172.21.15.46:0/197446375 shutdown_connections 2024-03-18T20:38:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.495+0000 7f0333521700 1 --2- 172.21.15.46:0/197446375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c10d5c0 0x7f032c10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.495+0000 7f0333521700 1 --2- 172.21.15.46:0/197446375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f032c108020 0x7f032c108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.495+0000 7f0333521700 1 --2- 172.21.15.46:0/197446375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f032c1089d0 0x7f032c10cee0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.495+0000 7f0333521700 1 -- 172.21.15.46:0/197446375 >> 172.21.15.46:0/197446375 conn(0x7f032c06bbc0 msgr2=0x7f032c06bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.495+0000 7f0333521700 1 -- 172.21.15.46:0/197446375 shutdown_connections 2024-03-18T20:38:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.495+0000 7f0333521700 1 -- 172.21.15.46:0/197446375 wait complete. 2024-03-18T20:38:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.496+0000 7f0333521700 1 Processor -- start 2024-03-18T20:38:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.496+0000 7f0333521700 1 -- start start 2024-03-18T20:38:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.496+0000 7f0333521700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f032c108020 0x7f032c117250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.496+0000 7f0333521700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c10d5c0 0x7f032c1177d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.496+0000 7f0333521700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f032c110480 0x7f032c110930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.496+0000 7f0333521700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c111100 con 0x7f032c10d5c0 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.496+0000 7f0333521700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c114de0 con 0x7f032c108020 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.496+0000 7f0333521700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c114fa0 con 0x7f032c110480 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.496+0000 7f0331d1e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c10d5c0 0x7f032c1177d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.496+0000 7f0331d1e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c10d5c0 0x7f032c1177d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42466/0 (socket says 172.21.15.46:42466) 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.496+0000 7f0331d1e700 1 -- 172.21.15.46:0/616125325 learned_addr learned my addr 172.21.15.46:0/616125325 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.496+0000 7f0332d20700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f032c110480 0x7f032c110930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f033251f700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f032c108020 0x7f032c117250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f0331d1e700 1 -- 172.21.15.46:0/616125325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f032c110480 msgr2=0x7f032c110930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f0331d1e700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f032c110480 0x7f032c110930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f0331d1e700 1 -- 172.21.15.46:0/616125325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f032c108020 msgr2=0x7f032c117250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f0331d1e700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f032c108020 0x7f032c117250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f0331d1e700 1 -- 172.21.15.46:0/616125325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c009520 con 0x7f032c10d5c0 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f033251f700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f032c108020 0x7f032c117250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f0331d1e700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c10d5c0 0x7f032c1177d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f031c009fd0 tx=0x7f031c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f03237fe700 1 -- 172.21.15.46:0/616125325 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c01c070 con 0x7f032c10d5c0 2024-03-18T20:38:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f0333521700 1 -- 172.21.15.46:0/616125325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032c115220 con 0x7f032c10d5c0 2024-03-18T20:38:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f0333521700 1 -- 172.21.15.46:0/616125325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032c115720 con 0x7f032c10d5c0 2024-03-18T20:38:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f03237fe700 1 -- 172.21.15.46:0/616125325 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f031c00bb20 con 0x7f032c10d5c0 2024-03-18T20:38:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.497+0000 7f03237fe700 1 -- 172.21.15.46:0/616125325 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c0038e0 con 0x7f032c10d5c0 2024-03-18T20:38:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.500+0000 7f03237fe700 1 -- 172.21.15.46:0/616125325 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f031c020030 con 0x7f032c10d5c0 2024-03-18T20:38:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.501+0000 7f03237fe700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f03180a1290 0x7f03180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.501+0000 7f03237fe700 1 -- 172.21.15.46:0/616125325 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f031c0cdb60 con 0x7f032c10d5c0 2024-03-18T20:38:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.501+0000 7f033251f700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f03180a1290 0x7f03180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.501+0000 7f033251f700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f03180a1290 0x7f03180a3750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0328009cb0 tx=0x7f0328009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.502+0000 7f0340918b80 1 Processor -- start 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.502+0000 7f0340918b80 1 -- start start 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.502+0000 7f0340918b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c046251b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.502+0000 7f0340918b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0462520a0 0x55c04625ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.502+0000 7f0340918b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c04625bfc0 0x55c04625e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.502+0000 7f0340918b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0461aa930 con 0x55c0462520a0 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.502+0000 7f0340918b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0461a82f0 con 0x55c046251780 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.502+0000 7f0340918b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0461a3be0 con 0x55c04625bfc0 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f033251f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c046251b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f0331d1e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0462520a0 0x55c04625ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f0332d20700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c04625bfc0 0x55c04625e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f033251f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c046251b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47114/0 (socket says 172.21.15.46:47114) 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f033251f700 1 -- 172.21.15.46:0/758782472 learned_addr learned my addr 172.21.15.46:0/758782472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f0332d20700 1 -- 172.21.15.46:0/758782472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 msgr2=0x55c046251b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f0332d20700 1 --2- 172.21.15.46:0/758782472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c046251b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f0332d20700 1 -- 172.21.15.46:0/758782472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0462520a0 msgr2=0x55c04625ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f0332d20700 1 --2- 172.21.15.46:0/758782472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0462520a0 0x55c04625ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f0332d20700 1 -- 172.21.15.46:0/758782472 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c046025350 con 0x55c04625bfc0 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f033251f700 1 --2- 172.21.15.46:0/758782472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c046251b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f0331d1e700 1 --2- 172.21.15.46:0/758782472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0462520a0 0x55c04625ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f0332d20700 1 --2- 172.21.15.46:0/758782472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c04625bfc0 0x55c04625e3b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0324011580 tx=0x7f03240099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e03e52feb6892c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f0320ff9700 1 -- 172.21.15.46:0/758782472 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032400e040 con 0x55c04625bfc0 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f0320ff9700 1 -- 172.21.15.46:0/758782472 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f032400f040 con 0x55c04625bfc0 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.503+0000 7f0320ff9700 1 -- 172.21.15.46:0/758782472 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03240043b0 con 0x55c04625bfc0 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.504+0000 7f0340918b80 1 -- 172.21.15.46:0/758782472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c04625bfc0 msgr2=0x55c04625e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.504+0000 7f0340918b80 1 --2- 172.21.15.46:0/758782472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c04625bfc0 0x55c04625e3b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0324011580 tx=0x7f03240099e0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.504+0000 7f0340918b80 1 -- 172.21.15.46:0/758782472 shutdown_connections 2024-03-18T20:38:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.504+0000 7f0340918b80 1 --2- 172.21.15.46:0/758782472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c04625bfc0 0x55c04625e3b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.504+0000 7f0340918b80 1 --2- 172.21.15.46:0/758782472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c046251b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.504+0000 7f0340918b80 1 --2- 172.21.15.46:0/758782472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0462520a0 0x55c04625ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.504+0000 7f0340918b80 1 -- 172.21.15.46:0/758782472 >> 172.21.15.46:0/758782472 conn(0x55c0461c71c0 msgr2=0x55c04624cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.504+0000 7f0340918b80 1 -- 172.21.15.46:0/758782472 shutdown_connections 2024-03-18T20:38:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.504+0000 7f0340918b80 1 -- 172.21.15.46:0/758782472 wait complete. 2024-03-18T20:38:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.505+0000 7f0340918b80 1 -- start start 2024-03-18T20:38:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.505+0000 7f0340918b80 1 Processor -- start 2024-03-18T20:38:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.505+0000 7f0340918b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c04625b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.505+0000 7f0340918b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0462520a0 0x55c046258ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.505+0000 7f0340918b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046259260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.505+0000 7f0340918b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04613fd20 con 0x55c04625bfc0 2024-03-18T20:38:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.505+0000 7f0340918b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04613f9c0 con 0x55c046251780 2024-03-18T20:38:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.505+0000 7f0340918b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046027640 con 0x55c0462520a0 2024-03-18T20:38:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.505+0000 7f0340918b80 10 client.0 _ll_register_callbacks cb 0x55c045fab340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.505+0000 7f033251f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c04625b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.505+0000 7f033251f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c04625b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47124/0 (socket says 172.21.15.46:47124) 2024-03-18T20:38:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f033251f700 1 -- 172.21.15.46:0/1498668389 learned_addr learned my addr 172.21.15.46:0/1498668389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f0332d20700 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046259260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f033251f700 1 -- 172.21.15.46:0/1498668389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0462520a0 msgr2=0x55c046258ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f033251f700 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0462520a0 0x55c046258ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f033251f700 1 -- 172.21.15.46:0/1498668389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 msgr2=0x55c046259260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f033251f700 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046259260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f033251f700 1 -- 172.21.15.46:0/1498668389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032c115720 con 0x55c046251780 2024-03-18T20:38:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f0332d20700 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046259260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:38:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f033251f700 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c04625b960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0328003930 tx=0x7f03280047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40292cb61dc95705 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f0315ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:38:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f0315ffb700 1 -- 172.21.15.46:0/1498668389 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328012860 con 0x55c046251780 2024-03-18T20:38:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f0315ffb700 1 -- 172.21.15.46:0/1498668389 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0328018030 con 0x55c046251780 2024-03-18T20:38:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.506+0000 7f0340918b80 1 -- 172.21.15.46:0/1498668389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c046244820 con 0x55c046251780 2024-03-18T20:38:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0315ffb700 1 -- 172.21.15.46:0/1498668389 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0328004570 con 0x55c046251780 2024-03-18T20:38:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0340918b80 10 client.26279 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0340918b80 1 -- 172.21.15.46:0/1498668389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f0324011230 con 0x55c046251780 2024-03-18T20:38:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0315ffb700 1 -- 172.21.15.46:0/1498668389 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c046244820 con 0x55c046251780 2024-03-18T20:38:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0315ffb700 1 -- 172.21.15.46:0/1498668389 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f032802f070 con 0x55c046251780 2024-03-18T20:38:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0340918b80 10 client.26279 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0340918b80 10 client.26279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0340918b80 20 client.26279 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0340918b80 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c04626bd50 0x55c04626e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0340918b80 4 client.26279 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:38:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0340918b80 1 -- 172.21.15.46:0/1498668389 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c04619c6a0 con 0x55c04626bd50 2024-03-18T20:38:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0331d1e700 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c04626bd50 0x55c04626e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0315ffb700 10 client.26279 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.507+0000 7f0331d1e700 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c04626bd50 0x55c04626e170 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e546f96298253b4a server_cookie=e9168c57ea595aaa in_seq=0 out_seq=0 2024-03-18T20:38:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0315ffb700 1 -- 172.21.15.46:0/1498668389 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c04619c6a0 con 0x55c04626bd50 2024-03-18T20:38:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0315ffb700 10 client.26279 handle_command_reply: tid=0 2024-03-18T20:38:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0340918b80 1 client.26279 shutdown 2024-03-18T20:38:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0340918b80 1 -- 172.21.15.46:0/1498668389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 msgr2=0x55c04625b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0340918b80 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c04625b960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0328003930 tx=0x7f03280047c0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0340918b80 1 -- 172.21.15.46:0/1498668389 shutdown_connections 2024-03-18T20:38:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0340918b80 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c04626bd50 0x55c04626e170 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0340918b80 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046259260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0340918b80 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c04625b960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0315ffb700 0 client.26279 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0340918b80 1 --2- 172.21.15.46:0/1498668389 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0462520a0 0x55c046258ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0340918b80 1 -- 172.21.15.46:0/1498668389 >> 172.21.15.46:0/1498668389 conn(0x55c0461c71c0 msgr2=0x55c04624c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.508+0000 7f0340918b80 1 -- 172.21.15.46:0/1498668389 shutdown_connections 2024-03-18T20:38:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.509+0000 7f0340918b80 1 -- 172.21.15.46:0/1498668389 wait complete. 2024-03-18T20:38:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.509+0000 7f0340918b80 20 client.26279 trim_cache size 0 max 16384 2024-03-18T20:38:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f0340918b80 1 Processor -- start 2024-03-18T20:38:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f0340918b80 1 -- start start 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f0340918b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c046262cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f0340918b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0462520a0 0x55c04625b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f0340918b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046258530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f0340918b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046253b70 con 0x55c04625bfc0 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f0340918b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046246e30 con 0x55c046251780 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f0340918b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0462692e0 con 0x55c0462520a0 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f033251f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c046262cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f0332d20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046258530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f033251f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c046262cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47130/0 (socket says 172.21.15.46:47130) 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f0332d20700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046258530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42500/0 (socket says 172.21.15.46:42500) 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f0331d1e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0462520a0 0x55c04625b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.525+0000 7f0332d20700 1 -- 172.21.15.46:0/2910660437 learned_addr learned my addr 172.21.15.46:0/2910660437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.526+0000 7f0332d20700 1 -- 172.21.15.46:0/2910660437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0462520a0 msgr2=0x55c04625b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.526+0000 7f0332d20700 1 --2- 172.21.15.46:0/2910660437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0462520a0 0x55c04625b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.526+0000 7f0332d20700 1 -- 172.21.15.46:0/2910660437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 msgr2=0x55c046262cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.526+0000 7f0332d20700 1 --2- 172.21.15.46:0/2910660437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c046262cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.526+0000 7f0332d20700 1 -- 172.21.15.46:0/2910660437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c045fd1b00 con 0x55c04625bfc0 2024-03-18T20:38:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.526+0000 7f0331d1e700 1 --2- 172.21.15.46:0/2910660437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0462520a0 0x55c04625b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.526+0000 7f033251f700 1 --2- 172.21.15.46:0/2910660437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c046262cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:38:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.526+0000 7f0332d20700 1 --2- 172.21.15.46:0/2910660437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046258530 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0324000f80 tx=0x7f0324009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0f4988ad10c82 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.526+0000 7f03217fa700 1 -- 172.21.15.46:0/2910660437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032400e040 con 0x55c04625bfc0 2024-03-18T20:38:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.526+0000 7f03217fa700 1 -- 172.21.15.46:0/2910660437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03240075c0 con 0x55c04625bfc0 2024-03-18T20:38:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.526+0000 7f03217fa700 1 -- 172.21.15.46:0/2910660437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03240094e0 con 0x55c04625bfc0 2024-03-18T20:38:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.527+0000 7f0340918b80 1 -- 172.21.15.46:0/2910660437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 msgr2=0x55c046258530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.527+0000 7f0340918b80 1 --2- 172.21.15.46:0/2910660437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046258530 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0324000f80 tx=0x7f0324009710 comp rx=0 tx=0).stop 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.527+0000 7f0340918b80 1 -- 172.21.15.46:0/2910660437 shutdown_connections 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.527+0000 7f0340918b80 1 --2- 172.21.15.46:0/2910660437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046258530 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.527+0000 7f0340918b80 1 --2- 172.21.15.46:0/2910660437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c046251780 0x55c046262cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.527+0000 7f0340918b80 1 --2- 172.21.15.46:0/2910660437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c0462520a0 0x55c04625b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.527+0000 7f0340918b80 1 -- 172.21.15.46:0/2910660437 >> 172.21.15.46:0/2910660437 conn(0x55c0461c71c0 msgr2=0x55c04624b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.527+0000 7f0340918b80 1 -- 172.21.15.46:0/2910660437 shutdown_connections 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.527+0000 7f0340918b80 1 -- 172.21.15.46:0/2910660437 wait complete. 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.527+0000 7f0340918b80 1 -- start start 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f0340918b80 1 Processor -- start 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f0340918b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c046251780 0x55c046262890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f0340918b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0462520a0 0x55c046260d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f0340918b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046261290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f0340918b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0462617d0 con 0x55c04625bfc0 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f0340918b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046261940 con 0x55c0462520a0 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f0340918b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c046261b00 con 0x55c046251780 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f0340918b80 10 client.0 _ll_register_callbacks cb 0x55c045fab340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f033251f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c046251780 0x55c046262890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f033251f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c046251780 0x55c046262890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:46368/0 (socket says 172.21.15.46:46368) 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f033251f700 1 -- 172.21.15.46:0/4221182733 learned_addr learned my addr 172.21.15.46:0/4221182733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f0331d1e700 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0462520a0 0x55c046260d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.528+0000 7f0332d20700 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046261290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f033251f700 1 -- 172.21.15.46:0/4221182733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0462520a0 msgr2=0x55c046260d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f033251f700 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0462520a0 0x55c046260d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f033251f700 1 -- 172.21.15.46:0/4221182733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 msgr2=0x55c046261290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f033251f700 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046261290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f033251f700 1 -- 172.21.15.46:0/4221182733 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0324011230 con 0x55c046251780 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f033251f700 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c046251780 0x55c046262890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0328028a80 tx=0x7f0328009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=483a0c0d14f37371 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f0315ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f0315ffb700 1 -- 172.21.15.46:0/4221182733 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328028400 con 0x55c046251780 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f0315ffb700 1 -- 172.21.15.46:0/4221182733 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0328018040 con 0x55c046251780 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f0340918b80 1 -- 172.21.15.46:0/4221182733 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f032400f040 con 0x55c046251780 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f0315ffb700 1 -- 172.21.15.46:0/4221182733 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0328028560 con 0x55c046251780 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f0340918b80 10 client.26278 fetch_fsmap learned FSMap version 33 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.529+0000 7f0340918b80 1 -- 172.21.15.46:0/4221182733 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f032c115220 con 0x55c046251780 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.530+0000 7f0315ffb700 1 -- 172.21.15.46:0/4221182733 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f032400f040 con 0x55c046251780 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.530+0000 7f0315ffb700 1 -- 172.21.15.46:0/4221182733 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f032802f050 con 0x55c046251780 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.530+0000 7f0340918b80 10 client.26278 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.530+0000 7f0340918b80 10 client.26278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.530+0000 7f0340918b80 20 client.26278 populate_metadata read hostname 'smithi046' 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.530+0000 7f0340918b80 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c0462c0fd0 0x55c0462c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:38:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.530+0000 7f0340918b80 4 client.26278 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:38:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.530+0000 7f0340918b80 1 -- 172.21.15.46:0/4221182733 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c045ff2c90 con 0x55c0462c0fd0 2024-03-18T20:38:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.530+0000 7f0331d1e700 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c0462c0fd0 0x55c0462c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:38:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.530+0000 7f0315ffb700 10 client.26278 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.530+0000 7f0331d1e700 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c0462c0fd0 0x55c0462c33f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d4a72bc30638e0 server_cookie=5161af84830ee3e0 in_seq=0 out_seq=0 2024-03-18T20:38:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:48 smithi155 ceph-mon[48344]: pgmap v84: 97 pgs: 97 active+clean; 36 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 543 KiB/s wr, 63 op/s 2024-03-18T20:38:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:48 smithi046 ceph-mon[43495]: pgmap v84: 97 pgs: 97 active+clean; 36 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 543 KiB/s wr, 63 op/s 2024-03-18T20:38:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.560+0000 7f0315ffb700 1 -- 172.21.15.46:0/4221182733 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c045ff2c90 con 0x55c0462c0fd0 2024-03-18T20:38:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.560+0000 7f0315ffb700 10 client.26278 handle_command_reply: tid=0 2024-03-18T20:38:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.560+0000 7f0340918b80 1 client.26278 shutdown 2024-03-18T20:38:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0340918b80 1 -- 172.21.15.46:0/4221182733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c046251780 msgr2=0x55c046262890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0340918b80 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c046251780 0x55c046262890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0328028a80 tx=0x7f0328009f20 comp rx=0 tx=0).stop 2024-03-18T20:38:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0340918b80 1 -- 172.21.15.46:0/4221182733 shutdown_connections 2024-03-18T20:38:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0340918b80 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c0462c0fd0 0x55c0462c33f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0340918b80 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c04625bfc0 0x55c046261290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0340918b80 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c046251780 0x55c046262890 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0340918b80 1 --2- 172.21.15.46:0/4221182733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0462520a0 0x55c046260d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0315ffb700 0 client.26278 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:38:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0340918b80 1 -- 172.21.15.46:0/4221182733 >> 172.21.15.46:0/4221182733 conn(0x55c0461c71c0 msgr2=0x55c046258530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0340918b80 1 -- 172.21.15.46:0/4221182733 shutdown_connections 2024-03-18T20:38:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0340918b80 1 -- 172.21.15.46:0/4221182733 wait complete. 2024-03-18T20:38:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0340918b80 20 client.26278 trim_cache size 0 max 16384 2024-03-18T20:38:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:38:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:38:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:38:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:38:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0333521700 1 -- 172.21.15.46:0/616125325 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f03180a1290 msgr2=0x7f03180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0333521700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f03180a1290 0x7f03180a3750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0328009cb0 tx=0x7f0328009260 comp rx=0 tx=0).stop 2024-03-18T20:38:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0333521700 1 -- 172.21.15.46:0/616125325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c10d5c0 msgr2=0x7f032c1177d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:38:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0333521700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c10d5c0 0x7f032c1177d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f031c009fd0 tx=0x7f031c00fa00 comp rx=0 tx=0).stop 2024-03-18T20:38:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0333521700 1 -- 172.21.15.46:0/616125325 shutdown_connections 2024-03-18T20:38:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0333521700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f03180a1290 0x7f03180a3750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0333521700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f032c110480 0x7f032c110930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0333521700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f032c108020 0x7f032c117250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0333521700 1 --2- 172.21.15.46:0/616125325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f032c10d5c0 0x7f032c1177d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:38:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.561+0000 7f0333521700 1 -- 172.21.15.46:0/616125325 >> 172.21.15.46:0/616125325 conn(0x7f032c06bbc0 msgr2=0x7f032c10aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:38:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.562+0000 7f0333521700 1 -- 172.21.15.46:0/616125325 shutdown_connections 2024-03-18T20:38:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:38:48.562+0000 7f0333521700 1 -- 172.21.15.46:0/616125325 wait complete. 2024-03-18T20:38:48.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 74951 4.10 MB/sec warmup 103 sec latency 77.293 ms 2024-03-18T20:38:49.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 75266 4.07 MB/sec warmup 104 sec latency 730.493 ms 2024-03-18T20:38:50.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:50 smithi073 ceph-mon[47384]: pgmap v85: 97 pgs: 97 active+clean; 40 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 997 KiB/s wr, 148 op/s 2024-03-18T20:38:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:50 smithi155 ceph-mon[48344]: pgmap v85: 97 pgs: 97 active+clean; 40 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 997 KiB/s wr, 148 op/s 2024-03-18T20:38:50.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:50 smithi046 ceph-mon[43495]: pgmap v85: 97 pgs: 97 active+clean; 40 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 997 KiB/s wr, 148 op/s 2024-03-18T20:38:50.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 75266 4.03 MB/sec warmup 105 sec latency 1730.606 ms 2024-03-18T20:38:51.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 75266 3.99 MB/sec warmup 106 sec latency 2730.649 ms 2024-03-18T20:38:52.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:52 smithi073 ceph-mon[47384]: pgmap v86: 97 pgs: 97 active+clean; 40 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 891 KiB/s wr, 123 op/s 2024-03-18T20:38:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:52 smithi155 ceph-mon[48344]: pgmap v86: 97 pgs: 97 active+clean; 40 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 891 KiB/s wr, 123 op/s 2024-03-18T20:38:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:52 smithi046 ceph-mon[43495]: pgmap v86: 97 pgs: 97 active+clean; 40 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 891 KiB/s wr, 123 op/s 2024-03-18T20:38:52.676 INFO:tasks.workunit.client.0.smithi046.stdout: 1 75334 3.95 MB/sec warmup 107 sec latency 3090.734 ms 2024-03-18T20:38:53.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:53 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:38:53.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:53 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:38:53.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:53 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:38:53.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:53 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:38:53.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:53 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:38:53.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:53 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:38:53.677 INFO:tasks.workunit.client.0.smithi046.stdout: 1 75334 3.92 MB/sec warmup 108 sec latency 1592.679 ms 2024-03-18T20:38:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:54 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:38:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:54 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:38:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:54 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:38:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:54 smithi073 ceph-mon[47384]: pgmap v87: 97 pgs: 97 active+clean; 37 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-03-18T20:38:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:54 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:38:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:54 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:38:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:54 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:38:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:54 smithi155 ceph-mon[48344]: pgmap v87: 97 pgs: 97 active+clean; 37 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-03-18T20:38:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:54 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:38:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:54 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:38:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:54 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:38:54.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:54 smithi046 ceph-mon[43495]: pgmap v87: 97 pgs: 97 active+clean; 37 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-03-18T20:38:54.677 INFO:tasks.workunit.client.0.smithi046.stdout: 1 75334 3.88 MB/sec warmup 109 sec latency 2592.783 ms 2024-03-18T20:38:55.677 INFO:tasks.workunit.client.0.smithi046.stdout: 1 75334 3.85 MB/sec warmup 110 sec latency 3592.910 ms 2024-03-18T20:38:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:55 smithi073 ceph-mon[47384]: pgmap v88: 97 pgs: 97 active+clean; 37 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 818 KiB/s wr, 134 op/s 2024-03-18T20:38:56.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:55 smithi155 ceph-mon[48344]: pgmap v88: 97 pgs: 97 active+clean; 37 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 818 KiB/s wr, 134 op/s 2024-03-18T20:38:56.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:55 smithi046 ceph-mon[43495]: pgmap v88: 97 pgs: 97 active+clean; 37 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 818 KiB/s wr, 134 op/s 2024-03-18T20:38:56.677 INFO:tasks.workunit.client.0.smithi046.stdout: 1 75334 3.81 MB/sec warmup 111 sec latency 4593.024 ms 2024-03-18T20:38:57.677 INFO:tasks.workunit.client.0.smithi046.stdout: 1 77114 3.84 MB/sec warmup 112 sec latency 4949.734 ms 2024-03-18T20:38:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:38:58 smithi073 ceph-mon[47384]: pgmap v89: 97 pgs: 97 active+clean; 37 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 818 KiB/s wr, 134 op/s 2024-03-18T20:38:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:38:58 smithi155 ceph-mon[48344]: pgmap v89: 97 pgs: 97 active+clean; 37 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 818 KiB/s wr, 134 op/s 2024-03-18T20:38:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:38:58 smithi046 ceph-mon[43495]: pgmap v89: 97 pgs: 97 active+clean; 37 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 818 KiB/s wr, 134 op/s 2024-03-18T20:38:58.677 INFO:tasks.workunit.client.0.smithi046.stdout: 1 78675 3.91 MB/sec warmup 113 sec latency 15.546 ms 2024-03-18T20:38:59.677 INFO:tasks.workunit.client.0.smithi046.stdout: 1 78753 3.88 MB/sec warmup 114 sec latency 961.347 ms 2024-03-18T20:39:00.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:00 smithi073 ceph-mon[47384]: pgmap v90: 97 pgs: 97 active+clean; 38 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 961 KiB/s wr, 166 op/s 2024-03-18T20:39:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:00 smithi155 ceph-mon[48344]: pgmap v90: 97 pgs: 97 active+clean; 38 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 961 KiB/s wr, 166 op/s 2024-03-18T20:39:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:00 smithi046 ceph-mon[43495]: pgmap v90: 97 pgs: 97 active+clean; 38 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 961 KiB/s wr, 166 op/s 2024-03-18T20:39:00.677 INFO:tasks.workunit.client.0.smithi046.stdout: 1 78753 3.84 MB/sec warmup 115 sec latency 1961.449 ms 2024-03-18T20:39:01.677 INFO:tasks.workunit.client.0.smithi046.stdout: 1 78877 3.81 MB/sec warmup 116 sec latency 2100.839 ms 2024-03-18T20:39:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.689+0000 7f5d964d5700 1 -- 172.21.15.46:0/3013128867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d90068a80 msgr2=0x7f5d9010db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.689+0000 7f5d964d5700 1 --2- 172.21.15.46:0/3013128867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d90068a80 0x7f5d9010db40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800b0a0 tx=0x7f5d8800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.689+0000 7f5d964d5700 1 -- 172.21.15.46:0/3013128867 shutdown_connections 2024-03-18T20:39:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.689+0000 7f5d964d5700 1 --2- 172.21.15.46:0/3013128867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d90068a80 0x7f5d9010db40 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.689+0000 7f5d964d5700 1 --2- 172.21.15.46:0/3013128867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d90106d50 0x7f5d90107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.689+0000 7f5d964d5700 1 --2- 172.21.15.46:0/3013128867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d900680c0 0x7f5d90068540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.689+0000 7f5d964d5700 1 -- 172.21.15.46:0/3013128867 >> 172.21.15.46:0/3013128867 conn(0x7f5d90076d50 msgr2=0x7f5d90077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.690+0000 7f5d964d5700 1 -- 172.21.15.46:0/3013128867 shutdown_connections 2024-03-18T20:39:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.690+0000 7f5d964d5700 1 -- 172.21.15.46:0/3013128867 wait complete. 2024-03-18T20:39:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.690+0000 7f5d964d5700 1 Processor -- start 2024-03-18T20:39:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.690+0000 7f5d964d5700 1 -- start start 2024-03-18T20:39:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d964d5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d900680c0 0x7f5d90197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d964d5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d90068a80 0x7f5d90197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d964d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d90106d50 0x7f5d9019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d964d5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9019c160 con 0x7f5d90068a80 2024-03-18T20:39:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d964d5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9019c2d0 con 0x7f5d900680c0 2024-03-18T20:39:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d964d5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9019c490 con 0x7f5d90106d50 2024-03-18T20:39:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d8f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d90068a80 0x7f5d90197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d8f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d90068a80 0x7f5d90197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58190/0 (socket says 172.21.15.46:58190) 2024-03-18T20:39:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d8f7fe700 1 -- 172.21.15.46:0/2213219399 learned_addr learned my addr 172.21.15.46:0/2213219399 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d8ffff700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d900680c0 0x7f5d90197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d94a72700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d90106d50 0x7f5d9019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d8f7fe700 1 -- 172.21.15.46:0/2213219399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d90106d50 msgr2=0x7f5d9019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d90106d50 0x7f5d9019bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d8f7fe700 1 -- 172.21.15.46:0/2213219399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d900680c0 msgr2=0x7f5d90197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.691+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d900680c0 0x7f5d90197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.692+0000 7f5d8f7fe700 1 -- 172.21.15.46:0/2213219399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d88009d00 con 0x7f5d90068a80 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.692+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d90068a80 0x7f5d90197780 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400e9a0 tx=0x7f5d8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.692+0000 7f5d8d7fa700 1 -- 172.21.15.46:0/2213219399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84009790 con 0x7f5d90068a80 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.692+0000 7f5d8d7fa700 1 -- 172.21.15.46:0/2213219399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d8400cbd0 con 0x7f5d90068a80 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.692+0000 7f5d964d5700 1 -- 172.21.15.46:0/2213219399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9019c770 con 0x7f5d90068a80 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.692+0000 7f5d8d7fa700 1 -- 172.21.15.46:0/2213219399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84018680 con 0x7f5d90068a80 2024-03-18T20:39:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.692+0000 7f5d964d5700 1 -- 172.21.15.46:0/2213219399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d901a3fb0 con 0x7f5d90068a80 2024-03-18T20:39:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.695+0000 7f5d8d7fa700 1 -- 172.21.15.46:0/2213219399 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5d8400cd40 con 0x7f5d90068a80 2024-03-18T20:39:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.695+0000 7f5d8d7fa700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5d780a1290 0x7f5d780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.696+0000 7f5d8ffff700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5d780a1290 0x7f5d780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.696+0000 7f5d8d7fa700 1 -- 172.21.15.46:0/2213219399 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5d84014070 con 0x7f5d90068a80 2024-03-18T20:39:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.696+0000 7f5d8ffff700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5d780a1290 0x7f5d780a3750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5d80005950 tx=0x7f5d8000b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.696+0000 7f5da266ab80 1 Processor -- start 2024-03-18T20:39:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.696+0000 7f5da266ab80 1 -- start start 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.696+0000 7f5da266ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5570 0x555d4e5c5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.696+0000 7f5da266ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 0x555d4e5d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5da266ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5d0db0 0x555d4e5d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5da266ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4e51f930 con 0x555d4e5c5e90 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5da266ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4e51d2f0 con 0x555d4e5c5570 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5da266ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4e518be0 con 0x555d4e5d0db0 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d8f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 0x555d4e5d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d8f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 0x555d4e5d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58206/0 (socket says 172.21.15.46:58206) 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d8f7fe700 1 -- 172.21.15.46:0/2342189500 learned_addr learned my addr 172.21.15.46:0/2342189500 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d8ffff700 1 --2- 172.21.15.46:0/2342189500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5570 0x555d4e5c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d94a72700 1 --2- 172.21.15.46:0/2342189500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5d0db0 0x555d4e5d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d8f7fe700 1 -- 172.21.15.46:0/2342189500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5d0db0 msgr2=0x555d4e5d31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/2342189500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5d0db0 0x555d4e5d31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d8f7fe700 1 -- 172.21.15.46:0/2342189500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5570 msgr2=0x555d4e5c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/2342189500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5570 0x555d4e5c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d8f7fe700 1 -- 172.21.15.46:0/2342189500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d4e39a350 con 0x555d4e5c5e90 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d94a72700 1 --2- 172.21.15.46:0/2342189500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5d0db0 0x555d4e5d31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/2342189500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 0x555d4e5d0870 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5d84096ad0 tx=0x7f5d84099a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578e3772157cbd5c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d767fc700 1 -- 172.21.15.46:0/2342189500 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8409a060 con 0x555d4e5c5e90 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d767fc700 1 -- 172.21.15.46:0/2342189500 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d8409de90 con 0x555d4e5c5e90 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.697+0000 7f5d767fc700 1 -- 172.21.15.46:0/2342189500 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84099750 con 0x555d4e5c5e90 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.698+0000 7f5da266ab80 1 -- 172.21.15.46:0/2342189500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 msgr2=0x555d4e5d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.698+0000 7f5da266ab80 1 --2- 172.21.15.46:0/2342189500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 0x555d4e5d0870 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5d84096ad0 tx=0x7f5d84099a40 comp rx=0 tx=0).stop 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.698+0000 7f5da266ab80 1 -- 172.21.15.46:0/2342189500 shutdown_connections 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.698+0000 7f5da266ab80 1 --2- 172.21.15.46:0/2342189500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5d0db0 0x555d4e5d31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.698+0000 7f5da266ab80 1 --2- 172.21.15.46:0/2342189500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5570 0x555d4e5c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.698+0000 7f5da266ab80 1 --2- 172.21.15.46:0/2342189500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 0x555d4e5d0870 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.698+0000 7f5da266ab80 1 -- 172.21.15.46:0/2342189500 >> 172.21.15.46:0/2342189500 conn(0x555d4e53c1c0 msgr2=0x555d4e5c1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.698+0000 7f5da266ab80 1 -- 172.21.15.46:0/2342189500 shutdown_connections 2024-03-18T20:39:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.698+0000 7f5da266ab80 1 -- 172.21.15.46:0/2342189500 wait complete. 2024-03-18T20:39:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5da266ab80 1 -- start start 2024-03-18T20:39:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5da266ab80 1 Processor -- start 2024-03-18T20:39:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5da266ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5c5570 0x555d4e5cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5da266ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 0x555d4e5ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5da266ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5d0db0 0x555d4e5ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5da266ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4e4b4d20 con 0x555d4e5c5e90 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5da266ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4e4b49c0 con 0x555d4e5d0db0 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5da266ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4e39c640 con 0x555d4e5c5570 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5da266ab80 10 client.0 _ll_register_callbacks cb 0x555d4e320340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5d8f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 0x555d4e5ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5d8f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 0x555d4e5ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58216/0 (socket says 172.21.15.46:58216) 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5d8f7fe700 1 -- 172.21.15.46:0/3329351951 learned_addr learned my addr 172.21.15.46:0/3329351951 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5d94a72700 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5d0db0 0x555d4e5ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.699+0000 7f5d8ffff700 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5c5570 0x555d4e5cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d8f7fe700 1 -- 172.21.15.46:0/3329351951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5c5570 msgr2=0x555d4e5cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5c5570 0x555d4e5cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d8f7fe700 1 -- 172.21.15.46:0/3329351951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5d0db0 msgr2=0x555d4e5ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5d0db0 0x555d4e5ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d8f7fe700 1 -- 172.21.15.46:0/3329351951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d901a3fb0 con 0x555d4e5c5e90 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d94a72700 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5d0db0 0x555d4e5ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d8ffff700 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5c5570 0x555d4e5cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 0x555d4e5ce100 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009230 tx=0x7f5d8409a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d42519c04f02369 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d6f7fe700 1 -- 172.21.15.46:0/3329351951 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840a2050 con 0x555d4e5c5e90 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d6f7fe700 1 -- 172.21.15.46:0/3329351951 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5d840a21b0 con 0x555d4e5c5e90 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5da266ab80 1 -- 172.21.15.46:0/3329351951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d4e5b96c0 con 0x555d4e5c5e90 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d6f7fe700 1 -- 172.21.15.46:0/3329351951 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d840b8180 con 0x555d4e5c5e90 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5da266ab80 10 client.16542 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5da266ab80 1 -- 172.21.15.46:0/3329351951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555d4e346b00 con 0x555d4e5c5e90 2024-03-18T20:39:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d6f7fe700 1 -- 172.21.15.46:0/3329351951 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d4e5b96c0 con 0x555d4e5c5e90 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.700+0000 7f5d6f7fe700 1 -- 172.21.15.46:0/3329351951 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5d840991a0 con 0x555d4e5c5e90 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.701+0000 7f5da266ab80 10 client.16542 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.701+0000 7f5da266ab80 10 client.16542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.701+0000 7f5da266ab80 20 client.16542 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.701+0000 7f5da266ab80 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d4e5e0820 0x555d4e5e2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.701+0000 7f5da266ab80 4 client.16542 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.701+0000 7f5da266ab80 1 -- 172.21.15.46:0/3329351951 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d4e5116a0 con 0x555d4e5e0820 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.701+0000 7f5d8ffff700 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d4e5e0820 0x555d4e5e2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.701+0000 7f5d6f7fe700 10 client.16542 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.701+0000 7f5d8ffff700 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d4e5e0820 0x555d4e5e2c40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f5091eb482f83a server_cookie=c45fb22fbbdb2a26 in_seq=0 out_seq=0 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5d6f7fe700 1 -- 172.21.15.46:0/3329351951 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555d4e5116a0 con 0x555d4e5e0820 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5d6f7fe700 10 client.16542 handle_command_reply: tid=0 2024-03-18T20:39:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5da266ab80 1 client.16542 shutdown 2024-03-18T20:39:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5da266ab80 1 -- 172.21.15.46:0/3329351951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 msgr2=0x555d4e5ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5da266ab80 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 0x555d4e5ce100 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009230 tx=0x7f5d8409a660 comp rx=0 tx=0).stop 2024-03-18T20:39:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5da266ab80 1 -- 172.21.15.46:0/3329351951 shutdown_connections 2024-03-18T20:39:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5da266ab80 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d4e5e0820 0x555d4e5e2c40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5da266ab80 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5d0db0 0x555d4e5ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5da266ab80 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5c5570 0x555d4e5cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5d6f7fe700 0 client.16542 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5da266ab80 1 --2- 172.21.15.46:0/3329351951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5e90 0x555d4e5ce100 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5da266ab80 1 -- 172.21.15.46:0/3329351951 >> 172.21.15.46:0/3329351951 conn(0x555d4e53c1c0 msgr2=0x555d4e5c1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5da266ab80 1 -- 172.21.15.46:0/3329351951 shutdown_connections 2024-03-18T20:39:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.702+0000 7f5da266ab80 1 -- 172.21.15.46:0/3329351951 wait complete. 2024-03-18T20:39:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.703+0000 7f5da266ab80 20 client.16542 trim_cache size 0 max 16384 2024-03-18T20:39:01.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.718+0000 7f5da266ab80 1 Processor -- start 2024-03-18T20:39:01.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.718+0000 7f5da266ab80 1 -- start start 2024-03-18T20:39:01.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.718+0000 7f5da266ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5c5570 0x555d4e5e2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.718+0000 7f5da266ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5e90 0x555d4e5c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.718+0000 7f5da266ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5d0db0 0x555d4e5ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.718+0000 7f5da266ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4e5c6850 con 0x555d4e5d0db0 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.718+0000 7f5da266ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4e5c7110 con 0x555d4e5c5e90 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.718+0000 7f5da266ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4e5c5ba0 con 0x555d4e5c5570 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.718+0000 7f5d94a72700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5d0db0 0x555d4e5ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.718+0000 7f5d94a72700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5d0db0 0x555d4e5ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58218/0 (socket says 172.21.15.46:58218) 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.719+0000 7f5d94a72700 1 -- 172.21.15.46:0/648406167 learned_addr learned my addr 172.21.15.46:0/648406167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.719+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/648406167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5e90 0x555d4e5c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.719+0000 7f5d8ffff700 1 --2- 172.21.15.46:0/648406167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5c5570 0x555d4e5e2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.719+0000 7f5d94a72700 1 -- 172.21.15.46:0/648406167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5c5570 msgr2=0x555d4e5e2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.719+0000 7f5d94a72700 1 --2- 172.21.15.46:0/648406167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5c5570 0x555d4e5e2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.719+0000 7f5d94a72700 1 -- 172.21.15.46:0/648406167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5e90 msgr2=0x555d4e5c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.719+0000 7f5d94a72700 1 --2- 172.21.15.46:0/648406167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5e90 0x555d4e5c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.719+0000 7f5d94a72700 1 -- 172.21.15.46:0/648406167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d4e4dee40 con 0x555d4e5d0db0 2024-03-18T20:39:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.719+0000 7f5d94a72700 1 --2- 172.21.15.46:0/648406167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5d0db0 0x555d4e5ce460 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009370 tx=0x7f5d880093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107df0ccb6a0932e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.719+0000 7f5d76ffd700 1 -- 172.21.15.46:0/648406167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8800e040 con 0x555d4e5d0db0 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.719+0000 7f5d76ffd700 1 -- 172.21.15.46:0/648406167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d88007d30 con 0x555d4e5d0db0 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.719+0000 7f5d76ffd700 1 -- 172.21.15.46:0/648406167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88007720 con 0x555d4e5d0db0 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.720+0000 7f5da266ab80 1 -- 172.21.15.46:0/648406167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5d0db0 msgr2=0x555d4e5ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.720+0000 7f5da266ab80 1 --2- 172.21.15.46:0/648406167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5d0db0 0x555d4e5ce460 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009370 tx=0x7f5d880093a0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.720+0000 7f5da266ab80 1 -- 172.21.15.46:0/648406167 shutdown_connections 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.720+0000 7f5da266ab80 1 --2- 172.21.15.46:0/648406167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5d0db0 0x555d4e5ce460 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.720+0000 7f5da266ab80 1 --2- 172.21.15.46:0/648406167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5c5570 0x555d4e5e2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.720+0000 7f5da266ab80 1 --2- 172.21.15.46:0/648406167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5e90 0x555d4e5c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.720+0000 7f5da266ab80 1 -- 172.21.15.46:0/648406167 >> 172.21.15.46:0/648406167 conn(0x555d4e53c1c0 msgr2=0x555d4e5e7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.720+0000 7f5da266ab80 1 -- 172.21.15.46:0/648406167 shutdown_connections 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.720+0000 7f5da266ab80 1 -- 172.21.15.46:0/648406167 wait complete. 2024-03-18T20:39:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.720+0000 7f5da266ab80 1 -- start start 2024-03-18T20:39:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.720+0000 7f5da266ab80 1 Processor -- start 2024-03-18T20:39:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5da266ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5570 0x555d4e5d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5da266ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5e90 0x555d4e5d5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5da266ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5d0db0 0x555d4e5d64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5da266ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4e637a60 con 0x555d4e5c5570 2024-03-18T20:39:01.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5da266ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4e637bd0 con 0x555d4e5c5e90 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5da266ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d4e637d90 con 0x555d4e5d0db0 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5da266ab80 10 client.0 _ll_register_callbacks cb 0x7f5d8000b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5d8ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5570 0x555d4e5d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5d8ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5570 0x555d4e5d5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58224/0 (socket says 172.21.15.46:58224) 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5d8ffff700 1 -- 172.21.15.46:0/1946505669 learned_addr learned my addr 172.21.15.46:0/1946505669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5e90 0x555d4e5d5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5d94a72700 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5d0db0 0x555d4e5d64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5d8ffff700 1 -- 172.21.15.46:0/1946505669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5d0db0 msgr2=0x555d4e5d64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5d8ffff700 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5d0db0 0x555d4e5d64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5d8ffff700 1 -- 172.21.15.46:0/1946505669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5e90 msgr2=0x555d4e5d5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5d8ffff700 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5e90 0x555d4e5d5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5d8ffff700 1 -- 172.21.15.46:0/1946505669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9010e440 con 0x555d4e5c5570 2024-03-18T20:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5d8ffff700 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5570 0x555d4e5d5a20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x555d4e5e3510 tx=0x7f5d80013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827cc0050e5461d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.721+0000 7f5d6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5d6f7fe700 1 -- 172.21.15.46:0/1946505669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80027a60 con 0x555d4e5c5570 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5d6f7fe700 1 -- 172.21.15.46:0/1946505669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5d80017070 con 0x555d4e5c5570 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5da266ab80 1 -- 172.21.15.46:0/1946505669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d8409de90 con 0x555d4e5c5570 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5d6f7fe700 1 -- 172.21.15.46:0/1946505669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d80027bc0 con 0x555d4e5c5570 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5da266ab80 10 client.16548 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5da266ab80 1 -- 172.21.15.46:0/1946505669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555d4e39ec70 con 0x555d4e5c5570 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5d6f7fe700 1 -- 172.21.15.46:0/1946505669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d8409de90 con 0x555d4e5c5570 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5d6f7fe700 1 -- 172.21.15.46:0/1946505669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5d80036070 con 0x555d4e5c5570 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5da266ab80 10 client.16548 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5da266ab80 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5da266ab80 20 client.16548 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5da266ab80 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d4e639550 0x555d4e63b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5da266ab80 4 client.16548 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:39:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5da266ab80 1 -- 172.21.15.46:0/1946505669 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555d4e367c90 con 0x555d4e639550 2024-03-18T20:39:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.722+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d4e639550 0x555d4e63b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.723+0000 7f5d6f7fe700 10 client.16548 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.723+0000 7f5d8f7fe700 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d4e639550 0x555d4e63b970 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=803191066b8fa24 server_cookie=a8c264224171121f in_seq=0 out_seq=0 2024-03-18T20:39:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5d6f7fe700 1 -- 172.21.15.46:0/1946505669 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555d4e367c90 con 0x555d4e639550 2024-03-18T20:39:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5d6f7fe700 10 client.16548 handle_command_reply: tid=0 2024-03-18T20:39:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5da266ab80 1 client.16548 shutdown 2024-03-18T20:39:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5da266ab80 1 -- 172.21.15.46:0/1946505669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5570 msgr2=0x555d4e5d5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5da266ab80 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5570 0x555d4e5d5a20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x555d4e5e3510 tx=0x7f5d80013070 comp rx=0 tx=0).stop 2024-03-18T20:39:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5da266ab80 1 -- 172.21.15.46:0/1946505669 shutdown_connections 2024-03-18T20:39:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5da266ab80 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d4e639550 0x555d4e63b970 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5da266ab80 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d4e5c5570 0x555d4e5d5a20 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5d6f7fe700 0 client.16548 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5da266ab80 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d4e5d0db0 0x555d4e5d64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5da266ab80 1 --2- 172.21.15.46:0/1946505669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d4e5c5e90 0x555d4e5d5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5da266ab80 1 -- 172.21.15.46:0/1946505669 >> 172.21.15.46:0/1946505669 conn(0x555d4e53c1c0 msgr2=0x555d4e5cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5da266ab80 1 -- 172.21.15.46:0/1946505669 shutdown_connections 2024-03-18T20:39:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.751+0000 7f5da266ab80 1 -- 172.21.15.46:0/1946505669 wait complete. 2024-03-18T20:39:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5da266ab80 20 client.16548 trim_cache size 0 max 16384 2024-03-18T20:39:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:39:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:39:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:39:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:39:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5d964d5700 1 -- 172.21.15.46:0/2213219399 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5d780a1290 msgr2=0x7f5d780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5d964d5700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5d780a1290 0x7f5d780a3750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5d80005950 tx=0x7f5d8000b410 comp rx=0 tx=0).stop 2024-03-18T20:39:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5d964d5700 1 -- 172.21.15.46:0/2213219399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d90068a80 msgr2=0x7f5d90197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5d964d5700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d90068a80 0x7f5d90197780 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400e9a0 tx=0x7f5d8400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5d964d5700 1 -- 172.21.15.46:0/2213219399 shutdown_connections 2024-03-18T20:39:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5d964d5700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5d780a1290 0x7f5d780a3750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5d964d5700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d90106d50 0x7f5d9019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5d964d5700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d900680c0 0x7f5d90197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5d964d5700 1 --2- 172.21.15.46:0/2213219399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d90068a80 0x7f5d90197780 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5d964d5700 1 -- 172.21.15.46:0/2213219399 >> 172.21.15.46:0/2213219399 conn(0x7f5d90076d50 msgr2=0x7f5d900fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5d964d5700 1 -- 172.21.15.46:0/2213219399 shutdown_connections 2024-03-18T20:39:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:01.752+0000 7f5d964d5700 1 -- 172.21.15.46:0/2213219399 wait complete. 2024-03-18T20:39:02.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:02 smithi073 ceph-mon[47384]: pgmap v91: 97 pgs: 97 active+clean; 38 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 506 KiB/s wr, 81 op/s 2024-03-18T20:39:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:02 smithi155 ceph-mon[48344]: pgmap v91: 97 pgs: 97 active+clean; 38 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 506 KiB/s wr, 81 op/s 2024-03-18T20:39:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:02 smithi046 ceph-mon[43495]: pgmap v91: 97 pgs: 97 active+clean; 38 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 506 KiB/s wr, 81 op/s 2024-03-18T20:39:02.677 INFO:tasks.workunit.client.0.smithi046.stdout: 1 79695 3.82 MB/sec warmup 117 sec latency 1135.078 ms 2024-03-18T20:39:03.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 81934 3.90 MB/sec warmup 118 sec latency 37.381 ms 2024-03-18T20:39:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:04 smithi073 ceph-mon[47384]: pgmap v92: 97 pgs: 97 active+clean; 40 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.0 MiB/s wr, 172 op/s 2024-03-18T20:39:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:04 smithi155 ceph-mon[48344]: pgmap v92: 97 pgs: 97 active+clean; 40 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.0 MiB/s wr, 172 op/s 2024-03-18T20:39:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:04 smithi046 ceph-mon[43495]: pgmap v92: 97 pgs: 97 active+clean; 40 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.0 MiB/s wr, 172 op/s 2024-03-18T20:39:04.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82302 3.87 MB/sec warmup 119 sec latency 609.261 ms 2024-03-18T20:39:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:05 smithi073 ceph-mon[47384]: pgmap v93: 97 pgs: 97 active+clean; 40 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 697 KiB/s wr, 122 op/s 2024-03-18T20:39:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:05 smithi155 ceph-mon[48344]: pgmap v93: 97 pgs: 97 active+clean; 40 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 697 KiB/s wr, 122 op/s 2024-03-18T20:39:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:05 smithi046 ceph-mon[43495]: pgmap v93: 97 pgs: 97 active+clean; 40 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 697 KiB/s wr, 122 op/s 2024-03-18T20:39:06.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82302 0.00 MB/sec execute 1 sec latency 2609.361 ms 2024-03-18T20:39:07.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82358 0.07 MB/sec execute 2 sec latency 2964.682 ms 2024-03-18T20:39:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:08 smithi073 ceph-mon[47384]: pgmap v94: 97 pgs: 97 active+clean; 40 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 697 KiB/s wr, 122 op/s 2024-03-18T20:39:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:08 smithi155 ceph-mon[48344]: pgmap v94: 97 pgs: 97 active+clean; 40 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 697 KiB/s wr, 122 op/s 2024-03-18T20:39:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:08 smithi046 ceph-mon[43495]: pgmap v94: 97 pgs: 97 active+clean; 40 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 697 KiB/s wr, 122 op/s 2024-03-18T20:39:08.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82358 0.05 MB/sec execute 3 sec latency 1608.032 ms 2024-03-18T20:39:09.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82358 0.03 MB/sec execute 4 sec latency 2608.091 ms 2024-03-18T20:39:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:10 smithi073 ceph-mon[47384]: pgmap v95: 97 pgs: 97 active+clean; 40 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-03-18T20:39:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:10 smithi155 ceph-mon[48344]: pgmap v95: 97 pgs: 97 active+clean; 40 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-03-18T20:39:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:10 smithi046 ceph-mon[43495]: pgmap v95: 97 pgs: 97 active+clean; 40 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-03-18T20:39:10.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82358 0.03 MB/sec execute 5 sec latency 3608.188 ms 2024-03-18T20:39:11.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82358 0.02 MB/sec execute 6 sec latency 4608.278 ms 2024-03-18T20:39:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:12 smithi073 ceph-mon[47384]: pgmap v96: 97 pgs: 97 active+clean; 40 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 956 KiB/s wr, 129 op/s 2024-03-18T20:39:12.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:12 smithi155 ceph-mon[48344]: pgmap v96: 97 pgs: 97 active+clean; 40 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 956 KiB/s wr, 129 op/s 2024-03-18T20:39:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:12 smithi046 ceph-mon[43495]: pgmap v96: 97 pgs: 97 active+clean; 40 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 956 KiB/s wr, 129 op/s 2024-03-18T20:39:12.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82426 0.04 MB/sec execute 7 sec latency 4963.985 ms 2024-03-18T20:39:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:13 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:39:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:13 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:39:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:13 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:39:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:13 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:39:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:13 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:39:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:13 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:39:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:13 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:39:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:13 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:39:13.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:13 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:39:13.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:13 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:39:13.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:13 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:39:13.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:13 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:39:13.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82426 0.04 MB/sec execute 8 sec latency 1600.637 ms 2024-03-18T20:39:14.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:39:14.133 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:39:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.277+0000 7fd4fe235700 1 -- 172.21.15.46:0/4109724689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f8106090 msgr2=0x7fd4f810dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.277+0000 7fd4fe235700 1 --2- 172.21.15.46:0/4109724689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f8106090 0x7fd4f810dc10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000b0a0 tx=0x7fd4f000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.277+0000 7fd4fe235700 1 -- 172.21.15.46:0/4109724689 shutdown_connections 2024-03-18T20:39:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.277+0000 7fd4fe235700 1 --2- 172.21.15.46:0/4109724689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f8106090 0x7fd4f810dc10 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.277+0000 7fd4fe235700 1 --2- 172.21.15.46:0/4109724689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f8100f70 0x7fd4f8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.277+0000 7fd4fe235700 1 --2- 172.21.15.46:0/4109724689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f8101920 0x7fd4f8105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.277+0000 7fd4fe235700 1 -- 172.21.15.46:0/4109724689 >> 172.21.15.46:0/4109724689 conn(0x7fd4f80fc7f0 msgr2=0x7fd4f80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.277+0000 7fd4fe235700 1 -- 172.21.15.46:0/4109724689 shutdown_connections 2024-03-18T20:39:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.277+0000 7fd4fe235700 1 -- 172.21.15.46:0/4109724689 wait complete. 2024-03-18T20:39:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.278+0000 7fd4fe235700 1 Processor -- start 2024-03-18T20:39:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.278+0000 7fd4fe235700 1 -- start start 2024-03-18T20:39:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.278+0000 7fd4fe235700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f8100f70 0x7fd4f80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.278+0000 7fd4fe235700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f8106090 0x7fd4f80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.278+0000 7fd4fe235700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f8073b30 0x7fd4f81a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.278+0000 7fd4fe235700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f81a5ae0 con 0x7fd4f8100f70 2024-03-18T20:39:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.278+0000 7fd4fe235700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f81a5c50 con 0x7fd4f8073b30 2024-03-18T20:39:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.278+0000 7fd4fe235700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f81a5e10 con 0x7fd4f8106090 2024-03-18T20:39:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.278+0000 7fd4f77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f8100f70 0x7fd4f80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f8100f70 0x7fd4f80730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40648/0 (socket says 172.21.15.46:40648) 2024-03-18T20:39:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f77fe700 1 -- 172.21.15.46:0/972958670 learned_addr learned my addr 172.21.15.46:0/972958670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f7fff700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f8073b30 0x7fd4f81a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f6ffd700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f8106090 0x7fd4f80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f77fe700 1 -- 172.21.15.46:0/972958670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f8106090 msgr2=0x7fd4f80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f77fe700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f8106090 0x7fd4f80735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f77fe700 1 -- 172.21.15.46:0/972958670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f8073b30 msgr2=0x7fd4f81a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f77fe700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f8073b30 0x7fd4f81a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f77fe700 1 -- 172.21.15.46:0/972958670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f0009d00 con 0x7fd4f8100f70 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f77fe700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f8100f70 0x7fd4f80730b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00e9a0 tx=0x7fd4ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f4ff9700 1 -- 172.21.15.46:0/972958670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec009790 con 0x7fd4f8100f70 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f4ff9700 1 -- 172.21.15.46:0/972958670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4ec010430 con 0x7fd4f8100f70 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4fe235700 1 -- 172.21.15.46:0/972958670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f81a60f0 con 0x7fd4f8100f70 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4f4ff9700 1 -- 172.21.15.46:0/972958670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec018680 con 0x7fd4f8100f70 2024-03-18T20:39:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.279+0000 7fd4fe235700 1 -- 172.21.15.46:0/972958670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f81a6600 con 0x7fd4f8100f70 2024-03-18T20:39:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.282+0000 7fd4f4ff9700 1 -- 172.21.15.46:0/972958670 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd4ec0105a0 con 0x7fd4f8100f70 2024-03-18T20:39:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.282+0000 7fd4f4ff9700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd4e00a1290 0x7fd4e00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.283+0000 7fd4f6ffd700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd4e00a1290 0x7fd4e00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.283+0000 7fd4f4ff9700 1 -- 172.21.15.46:0/972958670 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd4ec014070 con 0x7fd4f8100f70 2024-03-18T20:39:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.283+0000 7fd4f6ffd700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd4e00a1290 0x7fd4e00a3750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80747e0 tx=0x7fd4e8009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.283+0000 7fd50a3cab80 1 Processor -- start 2024-03-18T20:39:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.283+0000 7fd50a3cab80 1 -- start start 2024-03-18T20:39:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.283+0000 7fd50a3cab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad8a60 0x55d4ebad8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.283+0000 7fd50a3cab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebad9380 0x55d4ebae3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.283+0000 7fd50a3cab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 0x55d4ebae6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd50a3cab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4eba32990 con 0x55d4ebae4220 2024-03-18T20:39:14.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd50a3cab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4eba30350 con 0x55d4ebad8a60 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd50a3cab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4eba2bc40 con 0x55d4ebad9380 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 0x55d4ebae6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 0x55d4ebae6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40658/0 (socket says 172.21.15.46:40658) 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4f7fff700 1 -- 172.21.15.46:0/2430893331 learned_addr learned my addr 172.21.15.46:0/2430893331 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4f77fe700 1 --2- 172.21.15.46:0/2430893331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad8a60 0x55d4ebad8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4f6ffd700 1 --2- 172.21.15.46:0/2430893331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebad9380 0x55d4ebae3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4f7fff700 1 -- 172.21.15.46:0/2430893331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebad9380 msgr2=0x55d4ebae3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4f7fff700 1 --2- 172.21.15.46:0/2430893331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebad9380 0x55d4ebae3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4f7fff700 1 -- 172.21.15.46:0/2430893331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad8a60 msgr2=0x55d4ebad8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4f7fff700 1 --2- 172.21.15.46:0/2430893331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad8a60 0x55d4ebad8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4f7fff700 1 -- 172.21.15.46:0/2430893331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4eb8ad3b0 con 0x55d4ebae4220 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4f77fe700 1 --2- 172.21.15.46:0/2430893331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad8a60 0x55d4ebad8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4f7fff700 1 --2- 172.21.15.46:0/2430893331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 0x55d4ebae6610 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0015000 tx=0x7fd4f00092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c991fdcf7917ebbb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4ddffb700 1 -- 172.21.15.46:0/2430893331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000e040 con 0x55d4ebae4220 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4ddffb700 1 -- 172.21.15.46:0/2430893331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4f0006000 con 0x55d4ebae4220 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.284+0000 7fd4ddffb700 1 -- 172.21.15.46:0/2430893331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0003a00 con 0x55d4ebae4220 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.285+0000 7fd50a3cab80 1 -- 172.21.15.46:0/2430893331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 msgr2=0x55d4ebae6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.285+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/2430893331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 0x55d4ebae6610 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0015000 tx=0x7fd4f00092e0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.285+0000 7fd50a3cab80 1 -- 172.21.15.46:0/2430893331 shutdown_connections 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.285+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/2430893331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 0x55d4ebae6610 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.285+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/2430893331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad8a60 0x55d4ebad8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.285+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/2430893331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebad9380 0x55d4ebae3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.285+0000 7fd50a3cab80 1 -- 172.21.15.46:0/2430893331 >> 172.21.15.46:0/2430893331 conn(0x55d4eba4f220 msgr2=0x55d4ebad5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.285+0000 7fd50a3cab80 1 -- 172.21.15.46:0/2430893331 shutdown_connections 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.285+0000 7fd50a3cab80 1 -- 172.21.15.46:0/2430893331 wait complete. 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd50a3cab80 1 -- start start 2024-03-18T20:39:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd50a3cab80 1 Processor -- start 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd50a3cab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 0x55d4ebae3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd50a3cab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad9380 0x55d4ebae0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd50a3cab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebae4220 0x55d4ebae1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd50a3cab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4eb9c7d80 con 0x55d4ebad8a60 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd50a3cab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4eb9c7a20 con 0x55d4ebad9380 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd50a3cab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4eb8af6a0 con 0x55d4ebae4220 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd50a3cab80 10 client.0 _ll_register_callbacks cb 0x55d4eb8333a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd4f77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 0x55d4ebae3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd4f77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 0x55d4ebae3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40674/0 (socket says 172.21.15.46:40674) 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd4f77fe700 1 -- 172.21.15.46:0/125371436 learned_addr learned my addr 172.21.15.46:0/125371436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.286+0000 7fd4f6ffd700 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad9380 0x55d4ebae0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4f7fff700 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebae4220 0x55d4ebae1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4f77fe700 1 -- 172.21.15.46:0/125371436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebae4220 msgr2=0x55d4ebae1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4f77fe700 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebae4220 0x55d4ebae1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4f77fe700 1 -- 172.21.15.46:0/125371436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad9380 msgr2=0x55d4ebae0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4f77fe700 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad9380 0x55d4ebae0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4f77fe700 1 -- 172.21.15.46:0/125371436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f810e480 con 0x55d4ebad8a60 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4f7fff700 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebae4220 0x55d4ebae1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4f6ffd700 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad9380 0x55d4ebae0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4f77fe700 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 0x55d4ebae3c20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00c2d0 tx=0x7fd4ec099fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2eb9393c4a02980 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4d6ffd700 1 -- 172.21.15.46:0/125371436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0996a0 con 0x55d4ebad8a60 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4d6ffd700 1 -- 172.21.15.46:0/125371436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd4ec0a2430 con 0x55d4ebad8a60 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd50a3cab80 1 -- 172.21.15.46:0/125371436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4ebaccad0 con 0x55d4ebad8a60 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd4d6ffd700 1 -- 172.21.15.46:0/125371436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4ec0b8170 con 0x55d4ebad8a60 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd50a3cab80 10 client.16560 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.287+0000 7fd50a3cab80 1 -- 172.21.15.46:0/125371436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d4eb859b60 con 0x55d4ebad8a60 2024-03-18T20:39:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.288+0000 7fd4d6ffd700 1 -- 172.21.15.46:0/125371436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4ec063080 con 0x55d4ebad8a60 2024-03-18T20:39:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.288+0000 7fd4d6ffd700 1 -- 172.21.15.46:0/125371436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd4ec094c50 con 0x55d4ebad8a60 2024-03-18T20:39:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.288+0000 7fd50a3cab80 10 client.16560 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.288+0000 7fd50a3cab80 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.288+0000 7fd50a3cab80 20 client.16560 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.288+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4ebaf3dd0 0x55d4ebaf61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.288+0000 7fd50a3cab80 4 client.16560 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.288+0000 7fd50a3cab80 1 -- 172.21.15.46:0/125371436 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4eba24700 con 0x55d4ebaf3dd0 2024-03-18T20:39:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.288+0000 7fd4f6ffd700 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4ebaf3dd0 0x55d4ebaf61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.288+0000 7fd4d6ffd700 10 client.16560 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.288+0000 7fd4f6ffd700 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4ebaf3dd0 0x55d4ebaf61f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1fc5b70d5c1b03 server_cookie=7ebe611e0ebc7892 in_seq=0 out_seq=0 2024-03-18T20:39:14.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd4d6ffd700 1 -- 172.21.15.46:0/125371436 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d4eba24700 con 0x55d4ebaf3dd0 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd4d6ffd700 10 client.16560 handle_command_reply: tid=0 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd50a3cab80 1 client.16560 shutdown 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd50a3cab80 1 -- 172.21.15.46:0/125371436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 msgr2=0x55d4ebae3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 0x55d4ebae3c20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00c2d0 tx=0x7fd4ec099fb0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd50a3cab80 1 -- 172.21.15.46:0/125371436 shutdown_connections 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4ebaf3dd0 0x55d4ebaf61f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 0x55d4ebae3c20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebae4220 0x55d4ebae1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd4d6ffd700 0 client.16560 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/125371436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad9380 0x55d4ebae0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd50a3cab80 1 -- 172.21.15.46:0/125371436 >> 172.21.15.46:0/125371436 conn(0x55d4eba4f220 msgr2=0x55d4ebad4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd50a3cab80 1 -- 172.21.15.46:0/125371436 shutdown_connections 2024-03-18T20:39:14.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.289+0000 7fd50a3cab80 1 -- 172.21.15.46:0/125371436 wait complete. 2024-03-18T20:39:14.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.290+0000 7fd50a3cab80 20 client.16560 trim_cache size 0 max 16384 2024-03-18T20:39:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.307+0000 7fd50a3cab80 1 Processor -- start 2024-03-18T20:39:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd50a3cab80 1 -- start start 2024-03-18T20:39:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd50a3cab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad8a60 0x55d4ebaeade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd50a3cab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebad9380 0x55d4ebadf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd50a3cab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 0x55d4ebadfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd50a3cab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ebae0290 con 0x55d4ebae4220 2024-03-18T20:39:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd50a3cab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ebadba20 con 0x55d4ebad8a60 2024-03-18T20:39:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd50a3cab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ebaeb320 con 0x55d4ebad9380 2024-03-18T20:39:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 0x55d4ebadfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd4f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 0x55d4ebadfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40690/0 (socket says 172.21.15.46:40690) 2024-03-18T20:39:14.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd4f7fff700 1 -- 172.21.15.46:0/3204874729 learned_addr learned my addr 172.21.15.46:0/3204874729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd4f77fe700 1 --2- 172.21.15.46:0/3204874729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad8a60 0x55d4ebaeade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd4f6ffd700 1 --2- 172.21.15.46:0/3204874729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebad9380 0x55d4ebadf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd4f7fff700 1 -- 172.21.15.46:0/3204874729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebad9380 msgr2=0x55d4ebadf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd4f7fff700 1 --2- 172.21.15.46:0/3204874729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebad9380 0x55d4ebadf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd4f7fff700 1 -- 172.21.15.46:0/3204874729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad8a60 msgr2=0x55d4ebaeade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd4f7fff700 1 --2- 172.21.15.46:0/3204874729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad8a60 0x55d4ebaeade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd4f7fff700 1 -- 172.21.15.46:0/3204874729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4eb9f1ea0 con 0x55d4ebae4220 2024-03-18T20:39:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.308+0000 7fd4f7fff700 1 --2- 172.21.15.46:0/3204874729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 0x55d4ebadfd20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000f80 tx=0x7fd4f0007ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e952fde2e9ec4e16 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.309+0000 7fd4de7fc700 1 -- 172.21.15.46:0/3204874729 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000e040 con 0x55d4ebae4220 2024-03-18T20:39:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.309+0000 7fd4de7fc700 1 -- 172.21.15.46:0/3204874729 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4f0007cb0 con 0x55d4ebae4220 2024-03-18T20:39:14.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.309+0000 7fd4de7fc700 1 -- 172.21.15.46:0/3204874729 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0003dc0 con 0x55d4ebae4220 2024-03-18T20:39:14.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.309+0000 7fd50a3cab80 1 -- 172.21.15.46:0/3204874729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 msgr2=0x55d4ebadfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.309+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/3204874729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 0x55d4ebadfd20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000f80 tx=0x7fd4f0007ac0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.309+0000 7fd50a3cab80 1 -- 172.21.15.46:0/3204874729 shutdown_connections 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.309+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/3204874729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebae4220 0x55d4ebadfd20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.309+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/3204874729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad8a60 0x55d4ebaeade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.309+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/3204874729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebad9380 0x55d4ebadf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.309+0000 7fd50a3cab80 1 -- 172.21.15.46:0/3204874729 >> 172.21.15.46:0/3204874729 conn(0x55d4eba4f220 msgr2=0x55d4ebad3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.309+0000 7fd50a3cab80 1 -- 172.21.15.46:0/3204874729 shutdown_connections 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.309+0000 7fd50a3cab80 1 -- 172.21.15.46:0/3204874729 wait complete. 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd50a3cab80 1 -- start start 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd50a3cab80 1 Processor -- start 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd50a3cab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 0x55d4ebaea510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd50a3cab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad9380 0x55d4ebaeaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd50a3cab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebae4220 0x55d4ebae6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd50a3cab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ebae7460 con 0x55d4ebad8a60 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd50a3cab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ebae75d0 con 0x55d4ebad9380 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd50a3cab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ebae7790 con 0x55d4ebae4220 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd50a3cab80 10 client.0 _ll_register_callbacks cb 0x7fd4ec099910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd4f77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 0x55d4ebaea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd4f77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 0x55d4ebaea510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40704/0 (socket says 172.21.15.46:40704) 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd4f77fe700 1 -- 172.21.15.46:0/1384507583 learned_addr learned my addr 172.21.15.46:0/1384507583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.310+0000 7fd4f6ffd700 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad9380 0x55d4ebaeaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4f7fff700 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebae4220 0x55d4ebae6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4f77fe700 1 -- 172.21.15.46:0/1384507583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebae4220 msgr2=0x55d4ebae6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4f77fe700 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebae4220 0x55d4ebae6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4f77fe700 1 -- 172.21.15.46:0/1384507583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad9380 msgr2=0x55d4ebaeaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4f77fe700 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad9380 0x55d4ebaeaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4f77fe700 1 -- 172.21.15.46:0/1384507583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4eb8ad3b0 con 0x55d4ebad8a60 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4f7fff700 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebae4220 0x55d4ebae6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4f77fe700 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 0x55d4ebaea510 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec000c00 tx=0x7fd4ec00c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b692bd37dd4122a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4d6ffd700 1 -- 172.21.15.46:0/1384507583 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0a21a0 con 0x55d4ebad8a60 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4d6ffd700 1 -- 172.21.15.46:0/1384507583 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd4ec099290 con 0x55d4ebad8a60 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd50a3cab80 1 -- 172.21.15.46:0/1384507583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4f0006000 con 0x55d4ebad8a60 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4d6ffd700 1 -- 172.21.15.46:0/1384507583 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4ec07b020 con 0x55d4ebad8a60 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd50a3cab80 10 client.16566 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd50a3cab80 1 -- 172.21.15.46:0/1384507583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55d4eb8b1cd0 con 0x55d4ebad8a60 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.311+0000 7fd4d6ffd700 1 -- 172.21.15.46:0/1384507583 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4f0006000 con 0x55d4ebad8a60 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd4d6ffd700 1 -- 172.21.15.46:0/1384507583 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd4ec09ed00 con 0x55d4ebad8a60 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd50a3cab80 10 client.16566 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd50a3cab80 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd50a3cab80 20 client.16566 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4ebb48ba0 0x55d4ebb4afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd50a3cab80 4 client.16566 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd50a3cab80 1 -- 172.21.15.46:0/1384507583 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d4eb87acf0 con 0x55d4ebb48ba0 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd4f6ffd700 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4ebb48ba0 0x55d4ebb4afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd4f6ffd700 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4ebb48ba0 0x55d4ebb4afc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d8620c0cbbd3d2 server_cookie=6fe2927948764be in_seq=0 out_seq=0 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd4d6ffd700 10 client.16566 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd4d6ffd700 1 -- 172.21.15.46:0/1384507583 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d4eb87acf0 con 0x55d4ebb48ba0 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd4d6ffd700 10 client.16566 handle_command_reply: tid=0 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.312+0000 7fd50a3cab80 1 client.16566 shutdown 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd50a3cab80 1 -- 172.21.15.46:0/1384507583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 msgr2=0x55d4ebaea510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 0x55d4ebaea510 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec000c00 tx=0x7fd4ec00c480 comp rx=0 tx=0).stop 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd50a3cab80 1 -- 172.21.15.46:0/1384507583 shutdown_connections 2024-03-18T20:39:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4ebb48ba0 0x55d4ebb4afc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4ebad8a60 0x55d4ebaea510 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd4d6ffd700 0 client.16566 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4ebae4220 0x55d4ebae6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd50a3cab80 1 --2- 172.21.15.46:0/1384507583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4ebad9380 0x55d4ebaeaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd50a3cab80 1 -- 172.21.15.46:0/1384507583 >> 172.21.15.46:0/1384507583 conn(0x55d4eba4f220 msgr2=0x55d4ebadc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd50a3cab80 1 -- 172.21.15.46:0/1384507583 shutdown_connections 2024-03-18T20:39:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd50a3cab80 1 -- 172.21.15.46:0/1384507583 wait complete. 2024-03-18T20:39:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd50a3cab80 20 client.16566 trim_cache size 0 max 16384 2024-03-18T20:39:14.322 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:39:14.322 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:39:14.323 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:39:14.323 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:39:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd4fe235700 1 -- 172.21.15.46:0/972958670 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd4e00a1290 msgr2=0x7fd4e00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd4fe235700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd4e00a1290 0x7fd4e00a3750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80747e0 tx=0x7fd4e8009260 comp rx=0 tx=0).stop 2024-03-18T20:39:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd4fe235700 1 -- 172.21.15.46:0/972958670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f8100f70 msgr2=0x7fd4f80730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd4fe235700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f8100f70 0x7fd4f80730b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00e9a0 tx=0x7fd4ec00ecb0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd4fe235700 1 -- 172.21.15.46:0/972958670 shutdown_connections 2024-03-18T20:39:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd4fe235700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd4e00a1290 0x7fd4e00a3750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd4fe235700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f8100f70 0x7fd4f80730b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd4fe235700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f8073b30 0x7fd4f81a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd4fe235700 1 --2- 172.21.15.46:0/972958670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f8106090 0x7fd4f80735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.313+0000 7fd4fe235700 1 -- 172.21.15.46:0/972958670 >> 172.21.15.46:0/972958670 conn(0x7fd4f80fc7f0 msgr2=0x7fd4f80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.314+0000 7fd4fe235700 1 -- 172.21.15.46:0/972958670 shutdown_connections 2024-03-18T20:39:14.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.314+0000 7fd4fe235700 1 -- 172.21.15.46:0/972958670 wait complete. 2024-03-18T20:39:14.330 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:39:14.330 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:39:14.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.464+0000 7f0e3182e700 1 -- 172.21.15.46:0/3854736628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c100e60 msgr2=0x7f0e2c101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.464+0000 7f0e3182e700 1 --2- 172.21.15.46:0/3854736628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c100e60 0x7f0e2c101240 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0098f0 tx=0x7f0e1c009c00 comp rx=0 tx=0).stop 2024-03-18T20:39:14.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.464+0000 7f0e3182e700 1 -- 172.21.15.46:0/3854736628 shutdown_connections 2024-03-18T20:39:14.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.464+0000 7f0e3182e700 1 --2- 172.21.15.46:0/3854736628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e2c105f80 0x7f0e2c10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.464+0000 7f0e3182e700 1 --2- 172.21.15.46:0/3854736628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c100e60 0x7f0e2c101240 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.464+0000 7f0e3182e700 1 --2- 172.21.15.46:0/3854736628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e2c101810 0x7f0e2c105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.464+0000 7f0e3182e700 1 -- 172.21.15.46:0/3854736628 >> 172.21.15.46:0/3854736628 conn(0x7f0e2c0fc720 msgr2=0x7f0e2c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.465+0000 7f0e3182e700 1 -- 172.21.15.46:0/3854736628 shutdown_connections 2024-03-18T20:39:14.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.465+0000 7f0e3182e700 1 -- 172.21.15.46:0/3854736628 wait complete. 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.465+0000 7f0e3182e700 1 Processor -- start 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.465+0000 7f0e3182e700 1 -- start start 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e3182e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c100e60 0x7f0e2c197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e3182e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e2c105f80 0x7f0e2c197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e3182e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e2c19b9e0 0x7f0e2c19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e3182e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c19c460 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e3182e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c19c5d0 con 0x7f0e2c100e60 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e3182e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c19c790 con 0x7f0e2c105f80 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e2b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e2c19b9e0 0x7f0e2c19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e2b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e2c19b9e0 0x7f0e2c19be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40720/0 (socket says 172.21.15.46:40720) 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e2b7fe700 1 -- 172.21.15.46:0/570640738 learned_addr learned my addr 172.21.15.46:0/570640738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e2affd700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c100e60 0x7f0e2c197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e2a7fc700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e2c105f80 0x7f0e2c197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e2b7fe700 1 -- 172.21.15.46:0/570640738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e2c105f80 msgr2=0x7f0e2c197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e2b7fe700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e2c105f80 0x7f0e2c197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e2b7fe700 1 -- 172.21.15.46:0/570640738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c100e60 msgr2=0x7f0e2c197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e2b7fe700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c100e60 0x7f0e2c197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e2b7fe700 1 -- 172.21.15.46:0/570640738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e24009d00 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.466+0000 7f0e2b7fe700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e2c19b9e0 0x7f0e2c19be60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400e3c0 tx=0x7f0e2400e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.467+0000 7f0e3082c700 1 -- 172.21.15.46:0/570640738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e240090d0 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.467+0000 7f0e3182e700 1 -- 172.21.15.46:0/570640738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c009520 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.467+0000 7f0e3082c700 1 -- 172.21.15.46:0/570640738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e2400f040 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.467+0000 7f0e3082c700 1 -- 172.21.15.46:0/570640738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24007c20 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.467+0000 7f0e3182e700 1 -- 172.21.15.46:0/570640738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c1a4040 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.468+0000 7f0e3182e700 1 -- 172.21.15.46:0/570640738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c1091a0 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.474 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:14 smithi073 ceph-mon[47384]: pgmap v97: 97 pgs: 97 active+clean; 41 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 988 KiB/s wr, 140 op/s 2024-03-18T20:39:14.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.469+0000 7f0e3082c700 1 -- 172.21.15.46:0/570640738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0e24004ad0 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.469+0000 7f0e3082c700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0e180a1290 0x7f0e180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.470+0000 7f0e3082c700 1 -- 172.21.15.46:0/570640738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0e240cce50 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.470+0000 7f0e2affd700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0e180a1290 0x7f0e180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.470+0000 7f0e2affd700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0e180a1290 0x7f0e180a3750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009fd0 tx=0x7f0e1c019040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.473+0000 7f0e3082c700 1 -- 172.21.15.46:0/570640738 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0e24096340 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:14 smithi155 ceph-mon[48344]: pgmap v97: 97 pgs: 97 active+clean; 41 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 988 KiB/s wr, 140 op/s 2024-03-18T20:39:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:14 smithi046 ceph-mon[43495]: pgmap v97: 97 pgs: 97 active+clean; 41 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 988 KiB/s wr, 140 op/s 2024-03-18T20:39:14.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82426 0.03 MB/sec execute 9 sec latency 2600.690 ms 2024-03-18T20:39:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.740+0000 7f0e3182e700 1 -- 172.21.15.46:0/570640738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e2c102330 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.741+0000 7f0e3082c700 1 -- 172.21.15.46:0/570640738 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10919 (secure 0 0 0) 0x7f0e24099960 con 0x7f0e2c19b9e0 2024-03-18T20:39:14.744 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:39:14.745 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:37:47.071553+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:39:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.744+0000 7f0e3182e700 1 -- 172.21.15.46:0/570640738 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0e180a1290 msgr2=0x7f0e180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.744+0000 7f0e3182e700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0e180a1290 0x7f0e180a3750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009fd0 tx=0x7f0e1c019040 comp rx=0 tx=0).stop 2024-03-18T20:39:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.744+0000 7f0e3182e700 1 -- 172.21.15.46:0/570640738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e2c19b9e0 msgr2=0x7f0e2c19be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.744+0000 7f0e3182e700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e2c19b9e0 0x7f0e2c19be60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400e3c0 tx=0x7f0e2400e6d0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.744+0000 7f0e3182e700 1 -- 172.21.15.46:0/570640738 shutdown_connections 2024-03-18T20:39:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.744+0000 7f0e3182e700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0e180a1290 0x7f0e180a3750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.744+0000 7f0e3182e700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e2c19b9e0 0x7f0e2c19be60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.744+0000 7f0e3182e700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2c100e60 0x7f0e2c197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.744+0000 7f0e3182e700 1 --2- 172.21.15.46:0/570640738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e2c105f80 0x7f0e2c197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.744+0000 7f0e3182e700 1 -- 172.21.15.46:0/570640738 >> 172.21.15.46:0/570640738 conn(0x7f0e2c0fc720 msgr2=0x7f0e2c0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.744+0000 7f0e3182e700 1 -- 172.21.15.46:0/570640738 shutdown_connections 2024-03-18T20:39:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.744+0000 7f0e3182e700 1 -- 172.21.15.46:0/570640738 wait complete. 2024-03-18T20:39:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 33 2024-03-18T20:39:14.761 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:39:14.761 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:39:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.875+0000 7f511f963700 1 -- 172.21.15.46:0/593330194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5118100570 msgr2=0x7f511810ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.875+0000 7f511f963700 1 --2- 172.21.15.46:0/593330194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5118100570 0x7f511810ca00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f511000b0a0 tx=0x7f511000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.875+0000 7f511f963700 1 -- 172.21.15.46:0/593330194 shutdown_connections 2024-03-18T20:39:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.875+0000 7f511f963700 1 --2- 172.21.15.46:0/593330194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5118100570 0x7f511810ca00 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.875+0000 7f511f963700 1 --2- 172.21.15.46:0/593330194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5118105c60 0x7f5118106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.875+0000 7f511f963700 1 --2- 172.21.15.46:0/593330194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51180ffbb0 0x7f5118100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.875+0000 7f511f963700 1 -- 172.21.15.46:0/593330194 >> 172.21.15.46:0/593330194 conn(0x7f5118076d50 msgr2=0x7f5118077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.875+0000 7f511f963700 1 -- 172.21.15.46:0/593330194 shutdown_connections 2024-03-18T20:39:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.875+0000 7f511f963700 1 -- 172.21.15.46:0/593330194 wait complete. 2024-03-18T20:39:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.876+0000 7f511f963700 1 Processor -- start 2024-03-18T20:39:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.876+0000 7f511f963700 1 -- start start 2024-03-18T20:39:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.876+0000 7f511f963700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51180ffbb0 0x7f5118198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511f963700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5118100570 0x7f5118198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511f963700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5118105c60 0x7f51181a3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511f963700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5118198e10 con 0x7f5118100570 2024-03-18T20:39:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511f963700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181a35d0 con 0x7f51180ffbb0 2024-03-18T20:39:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511f963700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51181a3710 con 0x7f5118105c60 2024-03-18T20:39:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511cefe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5118100570 0x7f5118198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511cefe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5118100570 0x7f5118198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40736/0 (socket says 172.21.15.46:40736) 2024-03-18T20:39:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511cefe700 1 -- 172.21.15.46:0/3971466826 learned_addr learned my addr 172.21.15.46:0/3971466826 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511d6ff700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51180ffbb0 0x7f5118198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511df00700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5118105c60 0x7f51181a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511cefe700 1 -- 172.21.15.46:0/3971466826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5118105c60 msgr2=0x7f51181a3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511cefe700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5118105c60 0x7f51181a3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511cefe700 1 -- 172.21.15.46:0/3971466826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51180ffbb0 msgr2=0x7f5118198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511cefe700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51180ffbb0 0x7f5118198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511cefe700 1 -- 172.21.15.46:0/3971466826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5110009d00 con 0x7f5118100570 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511d6ff700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51180ffbb0 0x7f5118198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f511cefe700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5118100570 0x7f5118198550 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f510800e9a0 tx=0x7f510800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.877+0000 7f510e7fc700 1 -- 172.21.15.46:0/3971466826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108009790 con 0x7f5118100570 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.878+0000 7f510e7fc700 1 -- 172.21.15.46:0/3971466826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f510800cbd0 con 0x7f5118100570 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.878+0000 7f510e7fc700 1 -- 172.21.15.46:0/3971466826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108018680 con 0x7f5118100570 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.878+0000 7f511f963700 1 -- 172.21.15.46:0/3971466826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51181a39f0 con 0x7f5118100570 2024-03-18T20:39:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.878+0000 7f511f963700 1 -- 172.21.15.46:0/3971466826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51181a3f00 con 0x7f5118100570 2024-03-18T20:39:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.880+0000 7f510e7fc700 1 -- 172.21.15.46:0/3971466826 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f510800cd40 con 0x7f5118100570 2024-03-18T20:39:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.881+0000 7f510e7fc700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f51040a1310 0x7f51040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.881+0000 7f510e7fc700 1 -- 172.21.15.46:0/3971466826 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5108014070 con 0x7f5118100570 2024-03-18T20:39:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.881+0000 7f511d6ff700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f51040a1310 0x7f51040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.881+0000 7f511d6ff700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f51040a1310 0x7f51040a37d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5114009fd0 tx=0x7f5114009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.882+0000 7f512baf8b80 1 Processor -- start 2024-03-18T20:39:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.882+0000 7f512baf8b80 1 -- start start 2024-03-18T20:39:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.882+0000 7f512baf8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da834d570 0x561da834d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.882+0000 7f512baf8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8358870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.882+0000 7f512baf8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da8358db0 0x561da835b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.882+0000 7f512baf8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da82a7930 con 0x561da834de90 2024-03-18T20:39:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.882+0000 7f512baf8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da82a52f0 con 0x561da8358db0 2024-03-18T20:39:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.882+0000 7f512baf8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da82a0be0 con 0x561da834d570 2024-03-18T20:39:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.882+0000 7f511cefe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8358870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.882+0000 7f511cefe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8358870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40738/0 (socket says 172.21.15.46:40738) 2024-03-18T20:39:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.882+0000 7f511cefe700 1 -- 172.21.15.46:0/2365833188 learned_addr learned my addr 172.21.15.46:0/2365833188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.882+0000 7f511d6ff700 1 --2- 172.21.15.46:0/2365833188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da834d570 0x561da834d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.883+0000 7f511df00700 1 --2- 172.21.15.46:0/2365833188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da8358db0 0x561da835b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.883+0000 7f511cefe700 1 -- 172.21.15.46:0/2365833188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da834d570 msgr2=0x561da834d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.883+0000 7f511cefe700 1 --2- 172.21.15.46:0/2365833188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da834d570 0x561da834d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.883+0000 7f511cefe700 1 -- 172.21.15.46:0/2365833188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da8358db0 msgr2=0x561da835b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.883+0000 7f511cefe700 1 --2- 172.21.15.46:0/2365833188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da8358db0 0x561da835b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.883+0000 7f511cefe700 1 -- 172.21.15.46:0/2365833188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561da8122350 con 0x561da834de90 2024-03-18T20:39:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.883+0000 7f511df00700 1 --2- 172.21.15.46:0/2365833188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da8358db0 0x561da835b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.883+0000 7f511cefe700 1 --2- 172.21.15.46:0/2365833188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8358870 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5108096a60 tx=0x7f51080999d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d418c5f7ff3286 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.883+0000 7f51037fe700 1 -- 172.21.15.46:0/2365833188 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510809de20 con 0x561da834de90 2024-03-18T20:39:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.883+0000 7f51037fe700 1 -- 172.21.15.46:0/2365833188 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51080a2470 con 0x561da834de90 2024-03-18T20:39:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.883+0000 7f51037fe700 1 -- 172.21.15.46:0/2365833188 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51080a1fe0 con 0x561da834de90 2024-03-18T20:39:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.884+0000 7f512baf8b80 1 -- 172.21.15.46:0/2365833188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 msgr2=0x561da8358870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.884+0000 7f512baf8b80 1 --2- 172.21.15.46:0/2365833188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8358870 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5108096a60 tx=0x7f51080999d0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.884+0000 7f512baf8b80 1 -- 172.21.15.46:0/2365833188 shutdown_connections 2024-03-18T20:39:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.884+0000 7f512baf8b80 1 --2- 172.21.15.46:0/2365833188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da8358db0 0x561da835b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.884+0000 7f512baf8b80 1 --2- 172.21.15.46:0/2365833188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da834d570 0x561da834d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.884+0000 7f512baf8b80 1 --2- 172.21.15.46:0/2365833188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8358870 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.884+0000 7f512baf8b80 1 -- 172.21.15.46:0/2365833188 >> 172.21.15.46:0/2365833188 conn(0x561da82c41c0 msgr2=0x561da8349a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.884+0000 7f512baf8b80 1 -- 172.21.15.46:0/2365833188 shutdown_connections 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.884+0000 7f512baf8b80 1 -- 172.21.15.46:0/2365833188 wait complete. 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.885+0000 7f512baf8b80 1 -- start start 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.885+0000 7f512baf8b80 1 Processor -- start 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f512baf8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da834d570 0x561da8358850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f512baf8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8355b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f512baf8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da8358db0 0x561da83562d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f512baf8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da823cd20 con 0x561da834de90 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f512baf8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da823c9c0 con 0x561da834d570 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f512baf8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da8124640 con 0x561da8358db0 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f512baf8b80 10 client.0 _ll_register_callbacks cb 0x561da80a8340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f511cefe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8355b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f511cefe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8355b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40754/0 (socket says 172.21.15.46:40754) 2024-03-18T20:39:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f511cefe700 1 -- 172.21.15.46:0/4271785990 learned_addr learned my addr 172.21.15.46:0/4271785990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f511df00700 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da8358db0 0x561da83562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f511d6ff700 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da834d570 0x561da8358850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f511cefe700 1 -- 172.21.15.46:0/4271785990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da8358db0 msgr2=0x561da83562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f511cefe700 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da8358db0 0x561da83562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f511cefe700 1 -- 172.21.15.46:0/4271785990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da834d570 msgr2=0x561da8358850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f511cefe700 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da834d570 0x561da8358850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f511cefe700 1 -- 172.21.15.46:0/4271785990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51181a3f00 con 0x561da834de90 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f511cefe700 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8355b90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f510800c5b0 tx=0x7f510800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdaa89a317d9373 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.886+0000 7f5100ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f5100ff9700 1 -- 172.21.15.46:0/4271785990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510809de20 con 0x561da834de90 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f5100ff9700 1 -- 172.21.15.46:0/4271785990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f51080a21f0 con 0x561da834de90 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f512baf8b80 1 -- 172.21.15.46:0/4271785990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561da83416c0 con 0x561da834de90 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f5100ff9700 1 -- 172.21.15.46:0/4271785990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51080b8220 con 0x561da834de90 2024-03-18T20:39:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f512baf8b80 10 client.16584 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f512baf8b80 1 -- 172.21.15.46:0/4271785990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561da80ceb00 con 0x561da834de90 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f5100ff9700 1 -- 172.21.15.46:0/4271785990 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561da83416c0 con 0x561da834de90 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f5100ff9700 1 -- 172.21.15.46:0/4271785990 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f510807e070 con 0x561da834de90 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f512baf8b80 10 client.16584 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f512baf8b80 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f512baf8b80 20 client.16584 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f512baf8b80 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561da83686d0 0x561da836aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f512baf8b80 4 client.16584 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.887+0000 7f512baf8b80 1 -- 172.21.15.46:0/4271785990 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561da82996a0 con 0x561da83686d0 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.888+0000 7f511d6ff700 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561da83686d0 0x561da836aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.888+0000 7f5100ff9700 10 client.16584 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.888+0000 7f511d6ff700 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561da83686d0 0x561da836aaf0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a647c1ceccd12a server_cookie=61f86a937059b43a in_seq=0 out_seq=0 2024-03-18T20:39:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f5100ff9700 1 -- 172.21.15.46:0/4271785990 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561da82996a0 con 0x561da83686d0 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f5100ff9700 10 client.16584 handle_command_reply: tid=0 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f512baf8b80 1 client.16584 shutdown 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f512baf8b80 1 -- 172.21.15.46:0/4271785990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 msgr2=0x561da8355b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f512baf8b80 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8355b90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f510800c5b0 tx=0x7f510800ef60 comp rx=0 tx=0).stop 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f512baf8b80 1 -- 172.21.15.46:0/4271785990 shutdown_connections 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f512baf8b80 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561da83686d0 0x561da836aaf0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f512baf8b80 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da8358db0 0x561da83562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f512baf8b80 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da834d570 0x561da8358850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f5100ff9700 0 client.16584 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f512baf8b80 1 --2- 172.21.15.46:0/4271785990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8355b90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f512baf8b80 1 -- 172.21.15.46:0/4271785990 >> 172.21.15.46:0/4271785990 conn(0x561da82c41c0 msgr2=0x561da8349650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f512baf8b80 1 -- 172.21.15.46:0/4271785990 shutdown_connections 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f512baf8b80 1 -- 172.21.15.46:0/4271785990 wait complete. 2024-03-18T20:39:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.889+0000 7f512baf8b80 20 client.16584 trim_cache size 0 max 16384 2024-03-18T20:39:14.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.903+0000 7f65b8cad700 1 -- 172.21.15.46:0/2683827096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b40687f0 msgr2=0x7f65b410db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.903+0000 7f65b8cad700 1 --2- 172.21.15.46:0/2683827096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b40687f0 0x7f65b410db50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00b0a0 tx=0x7f65ac00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.904+0000 7f65b8cad700 1 -- 172.21.15.46:0/2683827096 shutdown_connections 2024-03-18T20:39:14.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.904+0000 7f65b8cad700 1 --2- 172.21.15.46:0/2683827096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b40687f0 0x7f65b410db50 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.904+0000 7f65b8cad700 1 --2- 172.21.15.46:0/2683827096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65b4105ce0 0x7f65b41060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.904+0000 7f65b8cad700 1 --2- 172.21.15.46:0/2683827096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b4067e30 0x7f65b40682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.904+0000 7f65b8cad700 1 -- 172.21.15.46:0/2683827096 >> 172.21.15.46:0/2683827096 conn(0x7f65b4076d40 msgr2=0x7f65b4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.904+0000 7f65b8cad700 1 -- 172.21.15.46:0/2683827096 shutdown_connections 2024-03-18T20:39:14.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.904+0000 7f65b8cad700 1 -- 172.21.15.46:0/2683827096 wait complete. 2024-03-18T20:39:14.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.904+0000 7f65b8cad700 1 Processor -- start 2024-03-18T20:39:14.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.904+0000 7f65b8cad700 1 -- start start 2024-03-18T20:39:14.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b8cad700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b4067e30 0x7f65b40ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b8cad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b40687f0 0x7f65b41001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b8cad700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65b4105ce0 0x7f65b41006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b8cad700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b41a56c0 con 0x7f65b40687f0 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b8cad700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b41a5830 con 0x7f65b4105ce0 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b8cad700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b41a59f0 con 0x7f65b4067e30 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b40687f0 0x7f65b41001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b40687f0 0x7f65b41001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40780/0 (socket says 172.21.15.46:40780) 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b1d9b700 1 -- 172.21.15.46:0/1647989828 learned_addr learned my addr 172.21.15.46:0/1647989828 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b2d9d700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65b4105ce0 0x7f65b41006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b259c700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b4067e30 0x7f65b40ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b1d9b700 1 -- 172.21.15.46:0/1647989828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b4067e30 msgr2=0x7f65b40ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b4067e30 0x7f65b40ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b1d9b700 1 -- 172.21.15.46:0/1647989828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65b4105ce0 msgr2=0x7f65b41006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65b4105ce0 0x7f65b41006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b1d9b700 1 -- 172.21.15.46:0/1647989828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac009d00 con 0x7f65b40687f0 2024-03-18T20:39:14.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.905+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b40687f0 0x7f65b41001b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f65a800e9a0 tx=0x7f65a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.906+0000 7f65a37fe700 1 -- 172.21.15.46:0/1647989828 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8009790 con 0x7f65b40687f0 2024-03-18T20:39:14.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.906+0000 7f65a37fe700 1 -- 172.21.15.46:0/1647989828 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65a8010430 con 0x7f65b40687f0 2024-03-18T20:39:14.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.906+0000 7f65a37fe700 1 -- 172.21.15.46:0/1647989828 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8018680 con 0x7f65b40687f0 2024-03-18T20:39:14.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.906+0000 7f65b8cad700 1 -- 172.21.15.46:0/1647989828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b41a5cd0 con 0x7f65b40687f0 2024-03-18T20:39:14.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.906+0000 7f65b8cad700 1 -- 172.21.15.46:0/1647989828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b41a6190 con 0x7f65b40687f0 2024-03-18T20:39:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.906+0000 7f512baf8b80 1 Processor -- start 2024-03-18T20:39:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.906+0000 7f512baf8b80 1 -- start start 2024-03-18T20:39:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f512baf8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da834d570 0x561da835fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f512baf8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8358380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f512baf8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da8358db0 0x561da8349500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f512baf8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da8357140 con 0x561da834de90 2024-03-18T20:39:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f512baf8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da83461d0 con 0x561da8358db0 2024-03-18T20:39:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f512baf8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da834a6a0 con 0x561da834d570 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f511cefe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8358380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f511cefe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8358380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40794/0 (socket says 172.21.15.46:40794) 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f511cefe700 1 -- 172.21.15.46:0/643724769 learned_addr learned my addr 172.21.15.46:0/643724769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f511d6ff700 1 --2- 172.21.15.46:0/643724769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da834d570 0x561da835fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f511df00700 1 --2- 172.21.15.46:0/643724769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da8358db0 0x561da8349500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f511cefe700 1 -- 172.21.15.46:0/643724769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da834d570 msgr2=0x561da835fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f511cefe700 1 --2- 172.21.15.46:0/643724769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da834d570 0x561da835fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f511cefe700 1 -- 172.21.15.46:0/643724769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da8358db0 msgr2=0x561da8349500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f511cefe700 1 --2- 172.21.15.46:0/643724769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da8358db0 0x561da8349500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.907+0000 7f511cefe700 1 -- 172.21.15.46:0/643724769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561da8266e40 con 0x561da834de90 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.908+0000 7f511cefe700 1 --2- 172.21.15.46:0/643724769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8358380 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5108098cb0 tx=0x7f510800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396658eb7716f4c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.908+0000 7f5103fff700 1 -- 172.21.15.46:0/643724769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108099190 con 0x561da834de90 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.908+0000 7f5103fff700 1 -- 172.21.15.46:0/643724769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f510809de20 con 0x561da834de90 2024-03-18T20:39:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.908+0000 7f5103fff700 1 -- 172.21.15.46:0/643724769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51080b8220 con 0x561da834de90 2024-03-18T20:39:14.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f65a37fe700 1 -- 172.21.15.46:0/1647989828 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f65a80105a0 con 0x7f65b40687f0 2024-03-18T20:39:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f512baf8b80 1 -- 172.21.15.46:0/643724769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 msgr2=0x561da8358380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f512baf8b80 1 --2- 172.21.15.46:0/643724769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8358380 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5108098cb0 tx=0x7f510800efd0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f512baf8b80 1 -- 172.21.15.46:0/643724769 shutdown_connections 2024-03-18T20:39:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f512baf8b80 1 --2- 172.21.15.46:0/643724769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da8358db0 0x561da8349500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f512baf8b80 1 --2- 172.21.15.46:0/643724769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da834d570 0x561da835fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f512baf8b80 1 --2- 172.21.15.46:0/643724769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834de90 0x561da8358380 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f512baf8b80 1 -- 172.21.15.46:0/643724769 >> 172.21.15.46:0/643724769 conn(0x561da82c41c0 msgr2=0x561da8350eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f512baf8b80 1 -- 172.21.15.46:0/643724769 shutdown_connections 2024-03-18T20:39:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f512baf8b80 1 -- 172.21.15.46:0/643724769 wait complete. 2024-03-18T20:39:14.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f65a37fe700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f659c0a1310 0x7f659c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f512baf8b80 1 -- start start 2024-03-18T20:39:14.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f65b259c700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f659c0a1310 0x7f659c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.909+0000 7f65a37fe700 1 -- 172.21.15.46:0/1647989828 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f65a8014070 con 0x7f65b40687f0 2024-03-18T20:39:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.910+0000 7f512baf8b80 1 Processor -- start 2024-03-18T20:39:14.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.910+0000 7f65b259c700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f659c0a1310 0x7f659c0a37d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009c60 tx=0x7f65a4009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.910+0000 7f512baf8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834d570 0x561da835da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.910+0000 7f512baf8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da834de90 0x561da835df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.910+0000 7f512baf8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da8358db0 0x561da835e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.910+0000 7f512baf8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da83bb4c0 con 0x561da834d570 2024-03-18T20:39:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.910+0000 7f512baf8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da83bb600 con 0x561da834de90 2024-03-18T20:39:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.910+0000 7f512baf8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da83bb740 con 0x561da8358db0 2024-03-18T20:39:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.910+0000 7f512baf8b80 10 client.0 _ll_register_callbacks cb 0x7f511400e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.910+0000 7f511d6ff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834d570 0x561da835da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.910+0000 7f511d6ff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834d570 0x561da835da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40796/0 (socket says 172.21.15.46:40796) 2024-03-18T20:39:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.911+0000 7f511d6ff700 1 -- 172.21.15.46:0/1422276606 learned_addr learned my addr 172.21.15.46:0/1422276606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.911+0000 7f511df00700 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da8358db0 0x561da835e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.910+0000 7f511cefe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da834de90 0x561da835df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.911+0000 7f511d6ff700 1 -- 172.21.15.46:0/1422276606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da8358db0 msgr2=0x561da835e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.911+0000 7f511d6ff700 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da8358db0 0x561da835e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.911+0000 7f511d6ff700 1 -- 172.21.15.46:0/1422276606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da834de90 msgr2=0x561da835df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.911+0000 7f511d6ff700 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da834de90 0x561da835df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.911+0000 7f511d6ff700 1 -- 172.21.15.46:0/1422276606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561da80ceb00 con 0x561da834d570 2024-03-18T20:39:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.911+0000 7f511df00700 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da8358db0 0x561da835e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:14.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.911+0000 7f65c4e42b80 1 Processor -- start 2024-03-18T20:39:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.911+0000 7f511d6ff700 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834d570 0x561da835da30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x561da836b390 tx=0x7f51140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d0ea29f637e8a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.911+0000 7f5100ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:14.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.911+0000 7f65c4e42b80 1 -- start start 2024-03-18T20:39:14.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65c4e42b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbee270 0x55cc0bbee650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65c4e42b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65c4e42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbfbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65c4e42b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0bb3f990 con 0x55cc0bbee270 2024-03-18T20:39:14.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65c4e42b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0bb3d350 con 0x55cc0bbeeb90 2024-03-18T20:39:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65c4e42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0bb38c40 con 0x55cc0bbf9a40 2024-03-18T20:39:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65b1d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65b1d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:53862/0 (socket says 172.21.15.46:53862) 2024-03-18T20:39:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65b1d9b700 1 -- 172.21.15.46:0/3750560245 learned_addr learned my addr 172.21.15.46:0/3750560245 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65b2d9d700 1 --2- 172.21.15.46:0/3750560245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbfbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65b259c700 1 --2- 172.21.15.46:0/3750560245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbee270 0x55cc0bbee650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65b1d9b700 1 -- 172.21.15.46:0/3750560245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbf9a40 msgr2=0x55cc0bbfbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/3750560245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbfbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65b1d9b700 1 -- 172.21.15.46:0/3750560245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbee270 msgr2=0x55cc0bbee650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/3750560245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbee270 0x55cc0bbee650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f65b1d9b700 1 -- 172.21.15.46:0/3750560245 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc0b9ba3b0 con 0x55cc0bbeeb90 2024-03-18T20:39:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f5100ff9700 1 -- 172.21.15.46:0/1422276606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114024040 con 0x561da834d570 2024-03-18T20:39:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.912+0000 7f5100ff9700 1 -- 172.21.15.46:0/1422276606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5114029b00 con 0x561da834d570 2024-03-18T20:39:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.913+0000 7f512baf8b80 1 -- 172.21.15.46:0/1422276606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51080a2470 con 0x561da834d570 2024-03-18T20:39:14.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.913+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/3750560245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf9500 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f65a80969a0 tx=0x7f65a8099ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f58ccb0406a40cc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.913+0000 7f5100ff9700 1 -- 172.21.15.46:0/1422276606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5114029cc0 con 0x561da834d570 2024-03-18T20:39:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.913+0000 7f512baf8b80 10 client.16596 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.913+0000 7f512baf8b80 1 -- 172.21.15.46:0/1422276606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561da8126c70 con 0x561da834d570 2024-03-18T20:39:14.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.913+0000 7f65a0ff9700 1 -- 172.21.15.46:0/3750560245 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a809dd60 con 0x55cc0bbeeb90 2024-03-18T20:39:14.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.913+0000 7f65a0ff9700 1 -- 172.21.15.46:0/3750560245 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65a80a23b0 con 0x55cc0bbeeb90 2024-03-18T20:39:14.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.913+0000 7f65a0ff9700 1 -- 172.21.15.46:0/3750560245 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8099420 con 0x55cc0bbeeb90 2024-03-18T20:39:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.913+0000 7f5100ff9700 1 -- 172.21.15.46:0/1422276606 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51080a2470 con 0x561da834d570 2024-03-18T20:39:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f5100ff9700 1 -- 172.21.15.46:0/1422276606 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5114039070 con 0x561da834d570 2024-03-18T20:39:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f512baf8b80 10 client.16596 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f512baf8b80 10 client.16596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f512baf8b80 20 client.16596 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:14.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f65c4e42b80 1 -- 172.21.15.46:0/3750560245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 msgr2=0x55cc0bbf9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/3750560245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf9500 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f65a80969a0 tx=0x7f65a8099ba0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f512baf8b80 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561da83bce30 0x561da83bf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f512baf8b80 4 client.16596 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:39:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f512baf8b80 1 -- 172.21.15.46:0/1422276606 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561da80efc90 con 0x561da83bce30 2024-03-18T20:39:14.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f65c4e42b80 1 -- 172.21.15.46:0/3750560245 shutdown_connections 2024-03-18T20:39:14.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/3750560245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbee270 0x55cc0bbee650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/3750560245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbfbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/3750560245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf9500 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f65c4e42b80 1 -- 172.21.15.46:0/3750560245 >> 172.21.15.46:0/3750560245 conn(0x55cc0bb5c220 msgr2=0x55cc0bbe1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f65c4e42b80 1 -- 172.21.15.46:0/3750560245 shutdown_connections 2024-03-18T20:39:14.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f65c4e42b80 1 -- 172.21.15.46:0/3750560245 wait complete. 2024-03-18T20:39:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.914+0000 7f511cefe700 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561da83bce30 0x561da83bf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65c4e42b80 1 -- start start 2024-03-18T20:39:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f5100ff9700 10 client.16596 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f511cefe700 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561da83bce30 0x561da83bf250 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fcb541e7cd2b10d server_cookie=3e14301cc675825c in_seq=0 out_seq=0 2024-03-18T20:39:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65c4e42b80 1 Processor -- start 2024-03-18T20:39:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65c4e42b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbee270 0x55cc0bbf6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65c4e42b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65c4e42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbf72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65c4e42b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0bad4d80 con 0x55cc0bbee270 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65c4e42b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0bad4a20 con 0x55cc0bbeeb90 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65c4e42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0b9bc6a0 con 0x55cc0bbf9a40 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65c4e42b80 10 client.0 _ll_register_callbacks cb 0x55cc0b9403a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65b259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbee270 0x55cc0bbf6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65b259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbee270 0x55cc0bbf6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40814/0 (socket says 172.21.15.46:40814) 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65b259c700 1 -- 172.21.15.46:0/146655511 learned_addr learned my addr 172.21.15.46:0/146655511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.915+0000 7f65b2d9d700 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbf72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.916+0000 7f65b259c700 1 -- 172.21.15.46:0/146655511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbf9a40 msgr2=0x55cc0bbf72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.916+0000 7f65b259c700 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbf72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.916+0000 7f65b259c700 1 -- 172.21.15.46:0/146655511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 msgr2=0x55cc0bbf6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.916+0000 7f65b259c700 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.916+0000 7f65b259c700 1 -- 172.21.15.46:0/146655511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b41a6190 con 0x55cc0bbee270 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.916+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.916+0000 7f65b2d9d700 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbf72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.916+0000 7f65b259c700 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbee270 0x55cc0bbf6820 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f65a401ebe0 tx=0x7f65a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825ceaba3afb5907 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.916+0000 7f6599ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.917+0000 7f6599ffb700 1 -- 172.21.15.46:0/146655511 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a400ec50 con 0x55cc0bbee270 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.917+0000 7f6599ffb700 1 -- 172.21.15.46:0/146655511 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f65a400edb0 con 0x55cc0bbee270 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.917+0000 7f65c4e42b80 1 -- 172.21.15.46:0/146655511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc0bbd9ad0 con 0x55cc0bbee270 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.917+0000 7f6599ffb700 1 -- 172.21.15.46:0/146655511 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65a4009ea0 con 0x55cc0bbee270 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.917+0000 7f65c4e42b80 10 client.16602 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.917+0000 7f65c4e42b80 1 -- 172.21.15.46:0/146655511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55cc0b966b60 con 0x55cc0bbee270 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.917+0000 7f6599ffb700 1 -- 172.21.15.46:0/146655511 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc0bbd9ad0 con 0x55cc0bbee270 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.917+0000 7f6599ffb700 1 -- 172.21.15.46:0/146655511 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f65a401c670 con 0x55cc0bbee270 2024-03-18T20:39:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.918+0000 7f65c4e42b80 10 client.16602 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.918+0000 7f65c4e42b80 10 client.16602 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.918+0000 7f65c4e42b80 20 client.16602 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.918+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55cc0bc097e0 0x55cc0bc0bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.918+0000 7f65c4e42b80 4 client.16602 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.918+0000 7f65c4e42b80 1 -- 172.21.15.46:0/146655511 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc0bb31700 con 0x55cc0bc097e0 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.918+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55cc0bc097e0 0x55cc0bc0bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.918+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55cc0bc097e0 0x55cc0bc0bc00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a015d8eff455f520 server_cookie=709ab7d838ce682f in_seq=0 out_seq=0 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.918+0000 7f6599ffb700 10 client.16602 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f6599ffb700 1 -- 172.21.15.46:0/146655511 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cc0bb31700 con 0x55cc0bc097e0 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f6599ffb700 10 client.16602 handle_command_reply: tid=0 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f65c4e42b80 1 client.16602 shutdown 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f65c4e42b80 1 -- 172.21.15.46:0/146655511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbee270 msgr2=0x55cc0bbf6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbee270 0x55cc0bbf6820 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f65a401ebe0 tx=0x7f65a40047c0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f65c4e42b80 1 -- 172.21.15.46:0/146655511 shutdown_connections 2024-03-18T20:39:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55cc0bc097e0 0x55cc0bc0bc00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbee270 0x55cc0bbf6820 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbf72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/146655511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f65c4e42b80 1 -- 172.21.15.46:0/146655511 >> 172.21.15.46:0/146655511 conn(0x55cc0bb5c220 msgr2=0x55cc0bbe1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f6599ffb700 0 client.16602 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:39:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f65c4e42b80 1 -- 172.21.15.46:0/146655511 shutdown_connections 2024-03-18T20:39:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f65c4e42b80 1 -- 172.21.15.46:0/146655511 wait complete. 2024-03-18T20:39:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.919+0000 7f65c4e42b80 20 client.16602 trim_cache size 0 max 16384 2024-03-18T20:39:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.938+0000 7f65c4e42b80 1 Processor -- start 2024-03-18T20:39:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.938+0000 7f65c4e42b80 1 -- start start 2024-03-18T20:39:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65c4e42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbee270 0x55cc0bbf6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65c4e42b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65c4e42b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbf50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65c4e42b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0bbf77f0 con 0x55cc0bbeeb90 2024-03-18T20:39:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65c4e42b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0bbac940 con 0x55cc0bbf9a40 2024-03-18T20:39:14.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65c4e42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0bbe3280 con 0x55cc0bbee270 2024-03-18T20:39:14.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65b1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65b259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbee270 0x55cc0bbf6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65b1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40820/0 (socket says 172.21.15.46:40820) 2024-03-18T20:39:14.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65b1d9b700 1 -- 172.21.15.46:0/4000344632 learned_addr learned my addr 172.21.15.46:0/4000344632 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65b2d9d700 1 --2- 172.21.15.46:0/4000344632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbf50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65b259c700 1 -- 172.21.15.46:0/4000344632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbf9a40 msgr2=0x55cc0bbf50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65b259c700 1 --2- 172.21.15.46:0/4000344632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbf50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65b259c700 1 -- 172.21.15.46:0/4000344632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbeeb90 msgr2=0x55cc0bbf72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65b259c700 1 --2- 172.21.15.46:0/4000344632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.939+0000 7f65b259c700 1 -- 172.21.15.46:0/4000344632 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc0bafeea0 con 0x55cc0bbee270 2024-03-18T20:39:14.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.940+0000 7f65b259c700 1 --2- 172.21.15.46:0/4000344632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbee270 0x55cc0bbf6d70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f65a401ca00 tx=0x7f65a401cfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf79118becfd1bc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.940+0000 7f65a17fa700 1 -- 172.21.15.46:0/4000344632 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a401c630 con 0x55cc0bbee270 2024-03-18T20:39:14.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.940+0000 7f65a17fa700 1 -- 172.21.15.46:0/4000344632 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65a4009ea0 con 0x55cc0bbee270 2024-03-18T20:39:14.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.940+0000 7f65a17fa700 1 -- 172.21.15.46:0/4000344632 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a4026770 con 0x55cc0bbee270 2024-03-18T20:39:14.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.941+0000 7f65c4e42b80 1 -- 172.21.15.46:0/4000344632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbee270 msgr2=0x55cc0bbf6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.941+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/4000344632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbee270 0x55cc0bbf6d70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f65a401ca00 tx=0x7f65a401cfd0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.941+0000 7f65c4e42b80 1 -- 172.21.15.46:0/4000344632 shutdown_connections 2024-03-18T20:39:14.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.941+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/4000344632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbf50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.941+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/4000344632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbee270 0x55cc0bbf6d70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.941+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/4000344632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbf72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.941+0000 7f65c4e42b80 1 -- 172.21.15.46:0/4000344632 >> 172.21.15.46:0/4000344632 conn(0x55cc0bb5c220 msgr2=0x55cc0bbf1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.941+0000 7f65c4e42b80 1 -- 172.21.15.46:0/4000344632 shutdown_connections 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.941+0000 7f65c4e42b80 1 -- 172.21.15.46:0/4000344632 wait complete. 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65c4e42b80 1 -- start start 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65c4e42b80 1 Processor -- start 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65c4e42b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbee270 0x55cc0bbfe650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65c4e42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbfeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65c4e42b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65c4e42b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0bc5c650 con 0x55cc0bbf9a40 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65c4e42b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0bc5c7c0 con 0x55cc0bbee270 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65c4e42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc0bc5c980 con 0x55cc0bbeeb90 2024-03-18T20:39:14.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65c4e42b80 10 client.0 _ll_register_callbacks cb 0x7f65a401e7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65b1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbfeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65b1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbfeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:34656/0 (socket says 172.21.15.46:34656) 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65b1d9b700 1 -- 172.21.15.46:0/2715051503 learned_addr learned my addr 172.21.15.46:0/2715051503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.942+0000 7f65b2d9d700 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f65b259c700 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbee270 0x55cc0bbfe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f65b1d9b700 1 -- 172.21.15.46:0/2715051503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbee270 msgr2=0x55cc0bbfe650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbee270 0x55cc0bbfe650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f65b1d9b700 1 -- 172.21.15.46:0/2715051503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbf9a40 msgr2=0x55cc0bbff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f65b1d9b700 1 -- 172.21.15.46:0/2715051503 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc0b966b60 con 0x55cc0bbeeb90 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f65b1d9b700 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbfeb90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65a80969a0 tx=0x7f65a8099db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=916dce8fba73c658 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f6599ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:39:14.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f6599ffb700 1 -- 172.21.15.46:0/2715051503 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a809a190 con 0x55cc0bbeeb90 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f65c4e42b80 1 -- 172.21.15.46:0/2715051503 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65a80a23b0 con 0x55cc0bbeeb90 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f6599ffb700 1 -- 172.21.15.46:0/2715051503 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f65a809a2f0 con 0x55cc0bbeeb90 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f6599ffb700 1 -- 172.21.15.46:0/2715051503 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65a80a1f20 con 0x55cc0bbeeb90 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f65c4e42b80 10 client.26356 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f65c4e42b80 1 -- 172.21.15.46:0/2715051503 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55cc0b9becd0 con 0x55cc0bbeeb90 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.943+0000 7f6599ffb700 1 -- 172.21.15.46:0/2715051503 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65a80a23b0 con 0x55cc0bbeeb90 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.944+0000 7f6599ffb700 1 -- 172.21.15.46:0/2715051503 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f65a8076b70 con 0x55cc0bbeeb90 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.944+0000 7f65c4e42b80 10 client.26356 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.944+0000 7f65c4e42b80 10 client.26356 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.944+0000 7f65c4e42b80 20 client.26356 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.944+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55cc0bc5ded0 0x55cc0bc602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.944+0000 7f65c4e42b80 4 client.26356 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.944+0000 7f65c4e42b80 1 -- 172.21.15.46:0/2715051503 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cc0b987cf0 con 0x55cc0bc5ded0 2024-03-18T20:39:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.944+0000 7f65b259c700 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55cc0bc5ded0 0x55cc0bc602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.944+0000 7f65b259c700 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55cc0bc5ded0 0x55cc0bc602c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=40d8d4563356abf9 server_cookie=fecefcc39331342a in_seq=0 out_seq=0 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.944+0000 7f6599ffb700 10 client.26356 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f6599ffb700 1 -- 172.21.15.46:0/2715051503 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cc0b987cf0 con 0x55cc0bc5ded0 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f6599ffb700 10 client.26356 handle_command_reply: tid=0 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f65c4e42b80 1 client.26356 shutdown 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f65c4e42b80 1 -- 172.21.15.46:0/2715051503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbeeb90 msgr2=0x55cc0bbfeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbfeb90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65a80969a0 tx=0x7f65a8099db0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f65c4e42b80 1 -- 172.21.15.46:0/2715051503 shutdown_connections 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55cc0bc5ded0 0x55cc0bc602c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cc0bbf9a40 0x55cc0bbff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cc0bbee270 0x55cc0bbfe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f65c4e42b80 1 --2- 172.21.15.46:0/2715051503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cc0bbeeb90 0x55cc0bbfeb90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f65c4e42b80 1 -- 172.21.15.46:0/2715051503 >> 172.21.15.46:0/2715051503 conn(0x55cc0bb5c220 msgr2=0x55cc0bbf5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f6599ffb700 0 client.26356 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:39:14.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f65c4e42b80 1 -- 172.21.15.46:0/2715051503 shutdown_connections 2024-03-18T20:39:14.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f65c4e42b80 1 -- 172.21.15.46:0/2715051503 wait complete. 2024-03-18T20:39:14.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f65c4e42b80 20 client.26356 trim_cache size 0 max 16384 2024-03-18T20:39:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:39:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f5100ff9700 1 -- 172.21.15.46:0/1422276606 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561da80efc90 con 0x561da83bce30 2024-03-18T20:39:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f5100ff9700 10 client.16596 handle_command_reply: tid=0 2024-03-18T20:39:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.945+0000 7f512baf8b80 1 client.16596 shutdown 2024-03-18T20:39:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f65b8cad700 1 -- 172.21.15.46:0/1647989828 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f659c0a1310 msgr2=0x7f659c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f65b8cad700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f659c0a1310 0x7f659c0a37d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009c60 tx=0x7f65a4009260 comp rx=0 tx=0).stop 2024-03-18T20:39:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f65b8cad700 1 -- 172.21.15.46:0/1647989828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b40687f0 msgr2=0x7f65b41001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f65b8cad700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b40687f0 0x7f65b41001b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f65a800e9a0 tx=0x7f65a800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f65b8cad700 1 -- 172.21.15.46:0/1647989828 shutdown_connections 2024-03-18T20:39:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f65b8cad700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f659c0a1310 0x7f659c0a37d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f65b8cad700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65b4105ce0 0x7f65b41006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f65b8cad700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65b4067e30 0x7f65b40ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f65b8cad700 1 --2- 172.21.15.46:0/1647989828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b40687f0 0x7f65b41001b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f65b8cad700 1 -- 172.21.15.46:0/1647989828 >> 172.21.15.46:0/1647989828 conn(0x7f65b4076d40 msgr2=0x7f65b40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f65b8cad700 1 -- 172.21.15.46:0/1647989828 shutdown_connections 2024-03-18T20:39:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f512baf8b80 1 -- 172.21.15.46:0/1422276606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834d570 msgr2=0x561da835da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f512baf8b80 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834d570 0x561da835da30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x561da836b390 tx=0x7f51140047c0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f65b8cad700 1 -- 172.21.15.46:0/1647989828 wait complete. 2024-03-18T20:39:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f512baf8b80 1 -- 172.21.15.46:0/1422276606 shutdown_connections 2024-03-18T20:39:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f512baf8b80 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561da83bce30 0x561da83bf250 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f512baf8b80 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561da834d570 0x561da835da30 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f512baf8b80 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561da8358db0 0x561da835e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f512baf8b80 1 --2- 172.21.15.46:0/1422276606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561da834de90 0x561da835df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f5100ff9700 0 client.16596 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f512baf8b80 1 -- 172.21.15.46:0/1422276606 >> 172.21.15.46:0/1422276606 conn(0x561da82c41c0 msgr2=0x561da8349470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f512baf8b80 1 -- 172.21.15.46:0/1422276606 shutdown_connections 2024-03-18T20:39:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f512baf8b80 1 -- 172.21.15.46:0/1422276606 wait complete. 2024-03-18T20:39:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.946+0000 7f512baf8b80 20 client.16596 trim_cache size 0 max 16384 2024-03-18T20:39:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:39:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:39:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:39:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:39:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.947+0000 7f511f963700 1 -- 172.21.15.46:0/3971466826 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f51040a1310 msgr2=0x7f51040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.947+0000 7f511f963700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f51040a1310 0x7f51040a37d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5114009fd0 tx=0x7f5114009260 comp rx=0 tx=0).stop 2024-03-18T20:39:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.947+0000 7f511f963700 1 -- 172.21.15.46:0/3971466826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5118100570 msgr2=0x7f5118198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.947+0000 7f511f963700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5118100570 0x7f5118198550 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f510800e9a0 tx=0x7f510800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.947+0000 7f511f963700 1 -- 172.21.15.46:0/3971466826 shutdown_connections 2024-03-18T20:39:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.947+0000 7f511f963700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f51040a1310 0x7f51040a37d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.947+0000 7f511f963700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5118105c60 0x7f51181a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.947+0000 7f511f963700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51180ffbb0 0x7f5118198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.947+0000 7f511f963700 1 --2- 172.21.15.46:0/3971466826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5118100570 0x7f5118198550 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.947+0000 7f511f963700 1 -- 172.21.15.46:0/3971466826 >> 172.21.15.46:0/3971466826 conn(0x7f5118076d50 msgr2=0x7f51180feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.947+0000 7f511f963700 1 -- 172.21.15.46:0/3971466826 shutdown_connections 2024-03-18T20:39:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:14.947+0000 7f511f963700 1 -- 172.21.15.46:0/3971466826 wait complete. 2024-03-18T20:39:14.968 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:39:14.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:39:15.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.108+0000 7f5b87376700 1 -- 172.21.15.46:0/2187911741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b800ffe70 msgr2=0x7f5b80100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.108+0000 7f5b87376700 1 --2- 172.21.15.46:0/2187911741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b800ffe70 0x7f5b80100250 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009990 tx=0x7f5b7c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.108+0000 7f5b87376700 1 -- 172.21.15.46:0/2187911741 shutdown_connections 2024-03-18T20:39:15.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.108+0000 7f5b87376700 1 --2- 172.21.15.46:0/2187911741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b800ffe70 0x7f5b80100250 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.108+0000 7f5b87376700 1 --2- 172.21.15.46:0/2187911741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b80104f70 0x7f5b8010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.108+0000 7f5b87376700 1 --2- 172.21.15.46:0/2187911741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b80100820 0x7f5b80104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.108+0000 7f5b87376700 1 -- 172.21.15.46:0/2187911741 >> 172.21.15.46:0/2187911741 conn(0x7f5b80076d50 msgr2=0x7f5b80077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.108+0000 7f5b87376700 1 -- 172.21.15.46:0/2187911741 shutdown_connections 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.108+0000 7f5b87376700 1 -- 172.21.15.46:0/2187911741 wait complete. 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b87376700 1 Processor -- start 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b87376700 1 -- start start 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b87376700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b800ffe70 0x7f5b801971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b87376700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b80104f70 0x7f5b80197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b87376700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8019bad0 0x7f5b8019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b87376700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8019c550 con 0x7f5b8019bad0 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b87376700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8019c6c0 con 0x7f5b80104f70 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b87376700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8019c880 con 0x7f5b800ffe70 2024-03-18T20:39:15.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b85913700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8019bad0 0x7f5b8019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b85913700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8019bad0 0x7f5b8019bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40852/0 (socket says 172.21.15.46:40852) 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b85112700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b800ffe70 0x7f5b801971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b85913700 1 -- 172.21.15.46:0/764327623 learned_addr learned my addr 172.21.15.46:0/764327623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b84911700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b80104f70 0x7f5b80197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b85913700 1 -- 172.21.15.46:0/764327623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b800ffe70 msgr2=0x7f5b801971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b85913700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b800ffe70 0x7f5b801971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b85913700 1 -- 172.21.15.46:0/764327623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b80104f70 msgr2=0x7f5b80197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b85913700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b80104f70 0x7f5b80197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.109+0000 7f5b85913700 1 -- 172.21.15.46:0/764327623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c0095f0 con 0x7f5b8019bad0 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.110+0000 7f5b85913700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8019bad0 0x7f5b8019bf50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5b78007ee0 tx=0x7f5b7800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.110+0000 7f5b767fc700 1 -- 172.21.15.46:0/764327623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78017070 con 0x7f5b8019bad0 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.110+0000 7f5b767fc700 1 -- 172.21.15.46:0/764327623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b7800f040 con 0x7f5b8019bad0 2024-03-18T20:39:15.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.110+0000 7f5b87376700 1 -- 172.21.15.46:0/764327623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b801a3de0 con 0x7f5b8019bad0 2024-03-18T20:39:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.110+0000 7f5b767fc700 1 -- 172.21.15.46:0/764327623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78005580 con 0x7f5b8019bad0 2024-03-18T20:39:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.110+0000 7f5b87376700 1 -- 172.21.15.46:0/764327623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b801a4270 con 0x7f5b8019bad0 2024-03-18T20:39:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.113+0000 7f5b767fc700 1 -- 172.21.15.46:0/764327623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5b7800db20 con 0x7f5b8019bad0 2024-03-18T20:39:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.113+0000 7f5b767fc700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5b6c0a1290 0x7f5b6c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.114+0000 7f5b85112700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5b6c0a1290 0x7f5b6c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.114+0000 7f5b767fc700 1 -- 172.21.15.46:0/764327623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5b780cc3e0 con 0x7f5b8019bad0 2024-03-18T20:39:15.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.114+0000 7f5b85112700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5b6c0a1290 0x7f5b6c0a3750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00b5c0 tx=0x7f5b7c005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.114+0000 7f5b9350bb80 1 Processor -- start 2024-03-18T20:39:15.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.114+0000 7f5b9350bb80 1 -- start start 2024-03-18T20:39:15.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b9350bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 0x564398fe4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b9350bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 0x564398fefce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b9350bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398ff0220 0x564398ff2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b9350bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564398f3e990 con 0x564398fe4a60 2024-03-18T20:39:15.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b9350bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564398f3c350 con 0x564398ff0220 2024-03-18T20:39:15.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b9350bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564398f37c40 con 0x564398fe5380 2024-03-18T20:39:15.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b85112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 0x564398fe4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b85112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 0x564398fe4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40862/0 (socket says 172.21.15.46:40862) 2024-03-18T20:39:15.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b85112700 1 -- 172.21.15.46:0/3888518963 learned_addr learned my addr 172.21.15.46:0/3888518963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b85913700 1 --2- 172.21.15.46:0/3888518963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398ff0220 0x564398ff2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b84911700 1 --2- 172.21.15.46:0/3888518963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 0x564398fefce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b85112700 1 -- 172.21.15.46:0/3888518963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 msgr2=0x564398fefce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b85112700 1 --2- 172.21.15.46:0/3888518963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 0x564398fefce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b85112700 1 -- 172.21.15.46:0/3888518963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398ff0220 msgr2=0x564398ff2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b85112700 1 --2- 172.21.15.46:0/3888518963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398ff0220 0x564398ff2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b85112700 1 -- 172.21.15.46:0/3888518963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564398db93b0 con 0x564398fe4a60 2024-03-18T20:39:15.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b85913700 1 --2- 172.21.15.46:0/3888518963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398ff0220 0x564398ff2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:15.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.115+0000 7f5b85112700 1 --2- 172.21.15.46:0/3888518963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 0x564398fe4e40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c005370 tx=0x7f5b7c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a9033c4184f4e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.116+0000 7f5b6b7fe700 1 -- 172.21.15.46:0/3888518963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00be00 con 0x564398fe4a60 2024-03-18T20:39:15.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.116+0000 7f5b6b7fe700 1 -- 172.21.15.46:0/3888518963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b7c01e040 con 0x564398fe4a60 2024-03-18T20:39:15.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.116+0000 7f5b6b7fe700 1 -- 172.21.15.46:0/3888518963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00f6c0 con 0x564398fe4a60 2024-03-18T20:39:15.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.116+0000 7f5b9350bb80 1 -- 172.21.15.46:0/3888518963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 msgr2=0x564398fe4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.116+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/3888518963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 0x564398fe4e40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c005370 tx=0x7f5b7c020000 comp rx=0 tx=0).stop 2024-03-18T20:39:15.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.116+0000 7f5b9350bb80 1 -- 172.21.15.46:0/3888518963 shutdown_connections 2024-03-18T20:39:15.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.116+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/3888518963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 0x564398fe4e40 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.116+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/3888518963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398ff0220 0x564398ff2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.116+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/3888518963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 0x564398fefce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.116+0000 7f5b9350bb80 1 -- 172.21.15.46:0/3888518963 >> 172.21.15.46:0/3888518963 conn(0x564398f5b220 msgr2=0x564398fe1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.116+0000 7f5b9350bb80 1 -- 172.21.15.46:0/3888518963 shutdown_connections 2024-03-18T20:39:15.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.116+0000 7f5b9350bb80 1 -- 172.21.15.46:0/3888518963 wait complete. 2024-03-18T20:39:15.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.117+0000 7f5b9350bb80 1 -- start start 2024-03-18T20:39:15.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.117+0000 7f5b9350bb80 1 Processor -- start 2024-03-18T20:39:15.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.117+0000 7f5b9350bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 0x564398fefca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.117+0000 7f5b9350bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 0x564398fecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.117+0000 7f5b9350bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398ff0220 0x564398fed6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.117+0000 7f5b9350bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564398ed3d80 con 0x564398fe4a60 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.117+0000 7f5b9350bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564398ed3a20 con 0x564398ff0220 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.117+0000 7f5b9350bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564398dbb6a0 con 0x564398fe5380 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.117+0000 7f5b9350bb80 10 client.0 _ll_register_callbacks cb 0x564398d3f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b85112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 0x564398fefca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b85112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 0x564398fefca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40870/0 (socket says 172.21.15.46:40870) 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b85112700 1 -- 172.21.15.46:0/1357844307 learned_addr learned my addr 172.21.15.46:0/1357844307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b85913700 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398ff0220 0x564398fed6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b84911700 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 0x564398fecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b85112700 1 -- 172.21.15.46:0/1357844307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 msgr2=0x564398fecfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b85112700 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 0x564398fecfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b85112700 1 -- 172.21.15.46:0/1357844307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398ff0220 msgr2=0x564398fed6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b85112700 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398ff0220 0x564398fed6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b85112700 1 -- 172.21.15.46:0/1357844307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8010d360 con 0x564398fe4a60 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b85112700 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 0x564398fefca0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009ce0 tx=0x7f5b7c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e7948b17f7cc10 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b68ff9700 1 -- 172.21.15.46:0/1357844307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00fb80 con 0x564398fe4a60 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b68ff9700 1 -- 172.21.15.46:0/1357844307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5b7c00fce0 con 0x564398fe4a60 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1357844307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564398fd8ad0 con 0x564398fe4a60 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.118+0000 7f5b68ff9700 1 -- 172.21.15.46:0/1357844307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b7c00fea0 con 0x564398fe4a60 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.119+0000 7f5b9350bb80 10 client.16620 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.119+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1357844307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564398d65b60 con 0x564398fe4a60 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.119+0000 7f5b68ff9700 1 -- 172.21.15.46:0/1357844307 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564398fd8ad0 con 0x564398fe4a60 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.119+0000 7f5b68ff9700 1 -- 172.21.15.46:0/1357844307 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5b7c02fa40 con 0x564398fe4a60 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.119+0000 7f5b9350bb80 10 client.16620 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.119+0000 7f5b9350bb80 10 client.16620 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.119+0000 7f5b9350bb80 20 client.16620 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.119+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564398fff8d0 0x564399001cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.119+0000 7f5b9350bb80 4 client.16620 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.119+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1357844307 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564398f30700 con 0x564398fff8d0 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.119+0000 7f5b84911700 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564398fff8d0 0x564399001cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.120+0000 7f5b84911700 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564398fff8d0 0x564399001cf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=61053d883a020d13 server_cookie=b0cffb7e09d2eefa in_seq=0 out_seq=0 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.120+0000 7f5b68ff9700 10 client.16620 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.120+0000 7f5b68ff9700 1 -- 172.21.15.46:0/1357844307 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564398f30700 con 0x564398fff8d0 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.120+0000 7f5b68ff9700 10 client.16620 handle_command_reply: tid=0 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.120+0000 7f5b9350bb80 1 client.16620 shutdown 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.121+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1357844307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 msgr2=0x564398fefca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.121+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 0x564398fefca0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009ce0 tx=0x7f5b7c0057b0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.121+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1357844307 shutdown_connections 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.121+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564398fff8d0 0x564399001cf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.121+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398fe4a60 0x564398fefca0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.121+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398ff0220 0x564398fed6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.121+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/1357844307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 0x564398fecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.121+0000 7f5b68ff9700 0 client.16620 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:39:15.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.121+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1357844307 >> 172.21.15.46:0/1357844307 conn(0x564398f5b220 msgr2=0x564398fe0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.121+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1357844307 shutdown_connections 2024-03-18T20:39:15.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.121+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1357844307 wait complete. 2024-03-18T20:39:15.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.121+0000 7f5b9350bb80 20 client.16620 trim_cache size 0 max 16384 2024-03-18T20:39:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b9350bb80 1 Processor -- start 2024-03-18T20:39:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b9350bb80 1 -- start start 2024-03-18T20:39:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b9350bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe4a60 0x564398fe7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b9350bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398fe5380 0x564398ff6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b9350bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 0x564398fec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b9350bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564398fdaf80 con 0x564398ff0220 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b9350bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564398fee560 con 0x564398fe5380 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b9350bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564398ff7360 con 0x564398fe4a60 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b85913700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 0x564398fec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b85913700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 0x564398fec870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40884/0 (socket says 172.21.15.46:40884) 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b85913700 1 -- 172.21.15.46:0/3227565073 learned_addr learned my addr 172.21.15.46:0/3227565073 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b84911700 1 --2- 172.21.15.46:0/3227565073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398fe5380 0x564398ff6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.135+0000 7f5b85112700 1 --2- 172.21.15.46:0/3227565073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe4a60 0x564398fe7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.136+0000 7f5b85913700 1 -- 172.21.15.46:0/3227565073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe4a60 msgr2=0x564398fe7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.136+0000 7f5b85913700 1 --2- 172.21.15.46:0/3227565073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe4a60 0x564398fe7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.136+0000 7f5b85913700 1 -- 172.21.15.46:0/3227565073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398fe5380 msgr2=0x564398ff6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.136+0000 7f5b85913700 1 --2- 172.21.15.46:0/3227565073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398fe5380 0x564398ff6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.136+0000 7f5b85913700 1 -- 172.21.15.46:0/3227565073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564398efdea0 con 0x564398ff0220 2024-03-18T20:39:15.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.136+0000 7f5b85913700 1 --2- 172.21.15.46:0/3227565073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 0x564398fec870 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5b78095f30 tx=0x7f5b78084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5acc5bfcd3f072 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.136+0000 7f5b6bfff700 1 -- 172.21.15.46:0/3227565073 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78099bc0 con 0x564398ff0220 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.136+0000 7f5b6bfff700 1 -- 172.21.15.46:0/3227565073 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b780994c0 con 0x564398ff0220 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.136+0000 7f5b6bfff700 1 -- 172.21.15.46:0/3227565073 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b780978f0 con 0x564398ff0220 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.137+0000 7f5b9350bb80 1 -- 172.21.15.46:0/3227565073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 msgr2=0x564398fec870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.137+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/3227565073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 0x564398fec870 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5b78095f30 tx=0x7f5b78084040 comp rx=0 tx=0).stop 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.137+0000 7f5b9350bb80 1 -- 172.21.15.46:0/3227565073 shutdown_connections 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.137+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/3227565073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 0x564398fec870 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.137+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/3227565073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe4a60 0x564398fe7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.137+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/3227565073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398fe5380 0x564398ff6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.137+0000 7f5b9350bb80 1 -- 172.21.15.46:0/3227565073 >> 172.21.15.46:0/3227565073 conn(0x564398f5b220 msgr2=0x564398fe8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.137+0000 7f5b9350bb80 1 -- 172.21.15.46:0/3227565073 shutdown_connections 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.137+0000 7f5b9350bb80 1 -- 172.21.15.46:0/3227565073 wait complete. 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.137+0000 7f5b9350bb80 1 -- start start 2024-03-18T20:39:15.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.137+0000 7f5b9350bb80 1 Processor -- start 2024-03-18T20:39:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b9350bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398fe4a60 0x564398ff61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b9350bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 0x564398ff6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b9350bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 0x5643990526c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b9350bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564399052c00 con 0x564398ff0220 2024-03-18T20:39:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b9350bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564399052d40 con 0x564398fe4a60 2024-03-18T20:39:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b9350bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564399052e80 con 0x564398fe5380 2024-03-18T20:39:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b9350bb80 10 client.0 _ll_register_callbacks cb 0x7f5b7c005270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b85913700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 0x5643990526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b85112700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398fe4a60 0x564398ff61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b85112700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398fe4a60 0x564398ff61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:53956/0 (socket says 172.21.15.46:53956) 2024-03-18T20:39:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b85112700 1 -- 172.21.15.46:0/1192972500 learned_addr learned my addr 172.21.15.46:0/1192972500 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b85913700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 0x5643990526c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40886/0 (socket says 172.21.15.46:40886) 2024-03-18T20:39:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b84911700 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 0x564398ff6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b85112700 1 -- 172.21.15.46:0/1192972500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 msgr2=0x564398ff6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b85112700 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 0x564398ff6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b85112700 1 -- 172.21.15.46:0/1192972500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 msgr2=0x5643990526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b85112700 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 0x5643990526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b85112700 1 -- 172.21.15.46:0/1192972500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b801a4270 con 0x564398fe4a60 2024-03-18T20:39:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.138+0000 7f5b85913700 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 0x5643990526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b85112700 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398fe4a60 0x564398ff61c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009ce0 tx=0x7f5b7c00f9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=512e988ccf8999a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:39:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b68ff9700 1 -- 172.21.15.46:0/1192972500 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0050f0 con 0x564398fe4a60 2024-03-18T20:39:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b68ff9700 1 -- 172.21.15.46:0/1192972500 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5b7c0049e0 con 0x564398fe4a60 2024-03-18T20:39:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1192972500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b7c01e040 con 0x564398fe4a60 2024-03-18T20:39:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b68ff9700 1 -- 172.21.15.46:0/1192972500 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b7c004c60 con 0x564398fe4a60 2024-03-18T20:39:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b9350bb80 10 client.26381 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1192972500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564398dbdcd0 con 0x564398fe4a60 2024-03-18T20:39:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b68ff9700 1 -- 172.21.15.46:0/1192972500 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b7c01e040 con 0x564398fe4a60 2024-03-18T20:39:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b68ff9700 1 -- 172.21.15.46:0/1192972500 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5b7c0245f0 con 0x564398fe4a60 2024-03-18T20:39:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b9350bb80 10 client.26381 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b9350bb80 10 client.26381 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:39:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b9350bb80 20 client.26381 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564399054120 0x564399056510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b9350bb80 4 client.26381 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:39:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.139+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1192972500 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564398d86cf0 con 0x564399054120 2024-03-18T20:39:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b84911700 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564399054120 0x564399056510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b84911700 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564399054120 0x564399056510 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e0ae1de20ba343a server_cookie=2ccf3df0009a68a3 in_seq=0 out_seq=0 2024-03-18T20:39:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b68ff9700 10 client.26381 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:39:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b68ff9700 1 -- 172.21.15.46:0/1192972500 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564398d86cf0 con 0x564399054120 2024-03-18T20:39:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b68ff9700 10 client.26381 handle_command_reply: tid=0 2024-03-18T20:39:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b9350bb80 1 client.26381 shutdown 2024-03-18T20:39:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1192972500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398fe4a60 msgr2=0x564398ff61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398fe4a60 0x564398ff61c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009ce0 tx=0x7f5b7c00f9a0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1192972500 shutdown_connections 2024-03-18T20:39:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564399054120 0x564399056510 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564398ff0220 0x5643990526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564398fe4a60 0x564398ff61c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b9350bb80 1 --2- 172.21.15.46:0/1192972500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564398fe5380 0x564398ff6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b68ff9700 0 client.26381 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:39:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1192972500 >> 172.21.15.46:0/1192972500 conn(0x564398f5b220 msgr2=0x564398fec870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1192972500 shutdown_connections 2024-03-18T20:39:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.140+0000 7f5b9350bb80 1 -- 172.21.15.46:0/1192972500 wait complete. 2024-03-18T20:39:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b9350bb80 20 client.26381 trim_cache size 0 max 16384 2024-03-18T20:39:15.151 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:39:15.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b87376700 1 -- 172.21.15.46:0/764327623 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5b6c0a1290 msgr2=0x7f5b6c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b87376700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5b6c0a1290 0x7f5b6c0a3750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00b5c0 tx=0x7f5b7c005960 comp rx=0 tx=0).stop 2024-03-18T20:39:15.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b87376700 1 -- 172.21.15.46:0/764327623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8019bad0 msgr2=0x7f5b8019bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b87376700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8019bad0 0x7f5b8019bf50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5b78007ee0 tx=0x7f5b7800d3b0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b87376700 1 -- 172.21.15.46:0/764327623 shutdown_connections 2024-03-18T20:39:15.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b87376700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5b6c0a1290 0x7f5b6c0a3750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b87376700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8019bad0 0x7f5b8019bf50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b87376700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b800ffe70 0x7f5b801971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b87376700 1 --2- 172.21.15.46:0/764327623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b80104f70 0x7f5b80197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b87376700 1 -- 172.21.15.46:0/764327623 >> 172.21.15.46:0/764327623 conn(0x7f5b80076d50 msgr2=0x7f5b80069710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b87376700 1 -- 172.21.15.46:0/764327623 shutdown_connections 2024-03-18T20:39:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.141+0000 7f5b87376700 1 -- 172.21.15.46:0/764327623 wait complete. 2024-03-18T20:39:15.158 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:39:15.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:39:15.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.295+0000 7f146b532700 1 -- 172.21.15.46:0/2893621781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640692a0 msgr2=0x7f1464069680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.295+0000 7f146b532700 1 --2- 172.21.15.46:0/2893621781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640692a0 0x7f1464069680 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f14600099c0 tx=0x7f1460009cd0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.295+0000 7f146b532700 1 -- 172.21.15.46:0/2893621781 shutdown_connections 2024-03-18T20:39:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.295+0000 7f146b532700 1 --2- 172.21.15.46:0/2893621781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640692a0 0x7f1464069680 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.295+0000 7f146b532700 1 --2- 172.21.15.46:0/2893621781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14641060b0 0x7f146410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.295+0000 7f146b532700 1 --2- 172.21.15.46:0/2893621781 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1464069c50 0x7f1464105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.295+0000 7f146b532700 1 -- 172.21.15.46:0/2893621781 >> 172.21.15.46:0/2893621781 conn(0x7f1464076d80 msgr2=0x7f1464077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.295+0000 7f146b532700 1 -- 172.21.15.46:0/2893621781 shutdown_connections 2024-03-18T20:39:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.295+0000 7f146b532700 1 -- 172.21.15.46:0/2893621781 wait complete. 2024-03-18T20:39:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f146b532700 1 Processor -- start 2024-03-18T20:39:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f146b532700 1 -- start start 2024-03-18T20:39:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f146b532700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14640692a0 0x7f1464197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f146b532700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14641060b0 0x7f1464197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f146b532700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146419bae0 0x7f146419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f146b532700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146419c560 con 0x7f14641060b0 2024-03-18T20:39:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f146b532700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146419c6d0 con 0x7f146419bae0 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f146b532700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146419c890 con 0x7f14640692a0 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f1468acd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14641060b0 0x7f1464197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f1468acd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14641060b0 0x7f1464197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46116/0 (socket says 172.21.15.46:46116) 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f1468acd700 1 -- 172.21.15.46:0/3504393165 learned_addr learned my addr 172.21.15.46:0/3504393165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f14692ce700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14640692a0 0x7f1464197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f1469acf700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146419bae0 0x7f146419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f1468acd700 1 -- 172.21.15.46:0/3504393165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14640692a0 msgr2=0x7f1464197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f1468acd700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14640692a0 0x7f1464197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f1468acd700 1 -- 172.21.15.46:0/3504393165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146419bae0 msgr2=0x7f146419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f1468acd700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146419bae0 0x7f146419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f1468acd700 1 -- 172.21.15.46:0/3504393165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14600095f0 con 0x7f14641060b0 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.296+0000 7f1469acf700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146419bae0 0x7f146419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.297+0000 7f1468acd700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14641060b0 0x7f1464197740 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f145400d980 tx=0x7f145400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.297+0000 7f145a7fc700 1 -- 172.21.15.46:0/3504393165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454009930 con 0x7f14641060b0 2024-03-18T20:39:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.297+0000 7f145a7fc700 1 -- 172.21.15.46:0/3504393165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1454010460 con 0x7f14641060b0 2024-03-18T20:39:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.297+0000 7f145a7fc700 1 -- 172.21.15.46:0/3504393165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400f620 con 0x7f14641060b0 2024-03-18T20:39:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.297+0000 7f146b532700 1 -- 172.21.15.46:0/3504393165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14641a3df0 con 0x7f14641060b0 2024-03-18T20:39:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.297+0000 7f146b532700 1 -- 172.21.15.46:0/3504393165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14641a4220 con 0x7f14641060b0 2024-03-18T20:39:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.299+0000 7f145a7fc700 1 -- 172.21.15.46:0/3504393165 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f14540108d0 con 0x7f14641060b0 2024-03-18T20:39:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.300+0000 7f145a7fc700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14500a1290 0x7f14500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.300+0000 7f145a7fc700 1 -- 172.21.15.46:0/3504393165 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f14540cc790 con 0x7f14641060b0 2024-03-18T20:39:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.300+0000 7f14692ce700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14500a1290 0x7f14500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f14692ce700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14500a1290 0x7f14500a3750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f146000b580 tx=0x7f1460005a90 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f14776c7b80 1 Processor -- start 2024-03-18T20:39:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f14776c7b80 1 -- start start 2024-03-18T20:39:15.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f14776c7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d72fde70 0x5599d72fe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f14776c7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 0x5599d7309100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f14776c7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d7309640 0x5599d730ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f14776c7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d7253990 con 0x5599d72fe790 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f14776c7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d7251350 con 0x5599d7309640 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f14776c7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d724cc40 con 0x5599d72fde70 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f1468acd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 0x5599d7309100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f1468acd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 0x5599d7309100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46128/0 (socket says 172.21.15.46:46128) 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f1468acd700 1 -- 172.21.15.46:0/1347617778 learned_addr learned my addr 172.21.15.46:0/1347617778 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.301+0000 7f14692ce700 1 --2- 172.21.15.46:0/1347617778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d72fde70 0x5599d72fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.302+0000 7f1469acf700 1 --2- 172.21.15.46:0/1347617778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d7309640 0x5599d730ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.302+0000 7f1468acd700 1 -- 172.21.15.46:0/1347617778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d72fde70 msgr2=0x5599d72fe250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.302+0000 7f1468acd700 1 --2- 172.21.15.46:0/1347617778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d72fde70 0x5599d72fe250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.302+0000 7f1468acd700 1 -- 172.21.15.46:0/1347617778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d7309640 msgr2=0x5599d730ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.302+0000 7f1468acd700 1 --2- 172.21.15.46:0/1347617778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d7309640 0x5599d730ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.302+0000 7f1468acd700 1 -- 172.21.15.46:0/1347617778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599d70ce3b0 con 0x5599d72fe790 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.302+0000 7f1468acd700 1 --2- 172.21.15.46:0/1347617778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 0x5599d7309100 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f14540a17c0 tx=0x7f145409a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec959572eec5d7f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.302+0000 7f144f7fe700 1 -- 172.21.15.46:0/1347617778 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540a1cf0 con 0x5599d72fe790 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.302+0000 7f144f7fe700 1 -- 172.21.15.46:0/1347617778 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1454095f80 con 0x5599d72fe790 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.302+0000 7f144f7fe700 1 -- 172.21.15.46:0/1347617778 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454098fc0 con 0x5599d72fe790 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.303+0000 7f14776c7b80 1 -- 172.21.15.46:0/1347617778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 msgr2=0x5599d7309100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.303+0000 7f14776c7b80 1 --2- 172.21.15.46:0/1347617778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 0x5599d7309100 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f14540a17c0 tx=0x7f145409a040 comp rx=0 tx=0).stop 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.303+0000 7f14776c7b80 1 -- 172.21.15.46:0/1347617778 shutdown_connections 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.303+0000 7f14776c7b80 1 --2- 172.21.15.46:0/1347617778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d7309640 0x5599d730ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.303+0000 7f14776c7b80 1 --2- 172.21.15.46:0/1347617778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d72fde70 0x5599d72fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.303+0000 7f14776c7b80 1 --2- 172.21.15.46:0/1347617778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 0x5599d7309100 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.303+0000 7f14776c7b80 1 -- 172.21.15.46:0/1347617778 >> 172.21.15.46:0/1347617778 conn(0x5599d7270220 msgr2=0x5599d72f5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.303+0000 7f14776c7b80 1 -- 172.21.15.46:0/1347617778 shutdown_connections 2024-03-18T20:39:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.303+0000 7f14776c7b80 1 -- 172.21.15.46:0/1347617778 wait complete. 2024-03-18T20:39:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.303+0000 7f14776c7b80 1 -- start start 2024-03-18T20:39:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f14776c7b80 1 Processor -- start 2024-03-18T20:39:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f14776c7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fde70 0x5599d7306430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f14776c7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 0x5599d7306970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f14776c7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d7309640 0x5599d7306eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f14776c7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d71e8d80 con 0x5599d72fe790 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f14776c7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d71e8a20 con 0x5599d72fde70 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f14776c7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d70d06a0 con 0x5599d7309640 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f14776c7b80 10 client.0 _ll_register_callbacks cb 0x5599d70543a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f1468acd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 0x5599d7306970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f1468acd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 0x5599d7306970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46140/0 (socket says 172.21.15.46:46140) 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f1468acd700 1 -- 172.21.15.46:0/3405564271 learned_addr learned my addr 172.21.15.46:0/3405564271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f14692ce700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fde70 0x5599d7306430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.304+0000 7f1469acf700 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d7309640 0x5599d7306eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f1468acd700 1 -- 172.21.15.46:0/3405564271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d7309640 msgr2=0x5599d7306eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f1468acd700 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d7309640 0x5599d7306eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f1468acd700 1 -- 172.21.15.46:0/3405564271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fde70 msgr2=0x5599d7306430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f1468acd700 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fde70 0x5599d7306430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f1468acd700 1 -- 172.21.15.46:0/3405564271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14641a4220 con 0x5599d72fe790 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f14692ce700 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fde70 0x5599d7306430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f1468acd700 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 0x5599d7306970 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f14540ca430 tx=0x7f1454098ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d6e6c08a35e95e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f144cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f144cff9700 1 -- 172.21.15.46:0/3405564271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540b2220 con 0x5599d72fe790 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f144cff9700 1 -- 172.21.15.46:0/3405564271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f14540a1cf0 con 0x5599d72fe790 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f14776c7b80 1 -- 172.21.15.46:0/3405564271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599d72edad0 con 0x5599d72fe790 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f144cff9700 1 -- 172.21.15.46:0/3405564271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1454099ce0 con 0x5599d72fe790 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f14776c7b80 10 client.16638 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f14776c7b80 1 -- 172.21.15.46:0/3405564271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5599d707ab60 con 0x5599d72fe790 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f144cff9700 1 -- 172.21.15.46:0/3405564271 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1454063080 con 0x5599d72fe790 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.305+0000 7f144cff9700 1 -- 172.21.15.46:0/3405564271 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1454064f90 con 0x5599d72fe790 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.306+0000 7f14776c7b80 10 client.16638 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.306+0000 7f14776c7b80 10 client.16638 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.306+0000 7f14776c7b80 20 client.16638 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.306+0000 7f14776c7b80 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5599d7318da0 0x5599d731b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.306+0000 7f14776c7b80 4 client.16638 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.306+0000 7f14776c7b80 1 -- 172.21.15.46:0/3405564271 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599d7245700 con 0x5599d7318da0 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.306+0000 7f14692ce700 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5599d7318da0 0x5599d731b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.306+0000 7f144cff9700 10 client.16638 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.306+0000 7f14692ce700 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5599d7318da0 0x5599d731b190 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8896696cf36f9828 server_cookie=678427c20c8d7681 in_seq=0 out_seq=0 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f144cff9700 1 -- 172.21.15.46:0/3405564271 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5599d7245700 con 0x5599d7318da0 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f144cff9700 10 client.16638 handle_command_reply: tid=0 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f14776c7b80 1 client.16638 shutdown 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f14776c7b80 1 -- 172.21.15.46:0/3405564271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 msgr2=0x5599d7306970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f14776c7b80 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 0x5599d7306970 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f14540ca430 tx=0x7f1454098ff0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f14776c7b80 1 -- 172.21.15.46:0/3405564271 shutdown_connections 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f14776c7b80 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d7309640 0x5599d7306eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f14776c7b80 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fde70 0x5599d7306430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f14776c7b80 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5599d7318da0 0x5599d731b190 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f14776c7b80 1 --2- 172.21.15.46:0/3405564271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fe790 0x5599d7306970 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f144cff9700 0 client.16638 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:39:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f14776c7b80 1 -- 172.21.15.46:0/3405564271 >> 172.21.15.46:0/3405564271 conn(0x5599d7270220 msgr2=0x5599d72f5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.307+0000 7f14776c7b80 1 -- 172.21.15.46:0/3405564271 shutdown_connections 2024-03-18T20:39:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.308+0000 7f14776c7b80 1 -- 172.21.15.46:0/3405564271 wait complete. 2024-03-18T20:39:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.308+0000 7f14776c7b80 20 client.16638 trim_cache size 0 max 16384 2024-03-18T20:39:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.321+0000 7f14776c7b80 1 Processor -- start 2024-03-18T20:39:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.321+0000 7f14776c7b80 1 -- start start 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.321+0000 7f14776c7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fde70 0x5599d73101b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.321+0000 7f14776c7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fe790 0x5599d73069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.321+0000 7f14776c7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d7309640 0x5599d7306ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.321+0000 7f14776c7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d72c0940 con 0x5599d72fde70 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.321+0000 7f14776c7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d7308fa0 con 0x5599d72fe790 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.321+0000 7f14776c7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d72f6ab0 con 0x5599d7309640 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f14692ce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fde70 0x5599d73101b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f14692ce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fde70 0x5599d73101b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46152/0 (socket says 172.21.15.46:46152) 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f14692ce700 1 -- 172.21.15.46:0/4151493722 learned_addr learned my addr 172.21.15.46:0/4151493722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f1469acf700 1 --2- 172.21.15.46:0/4151493722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d7309640 0x5599d7306ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f1468acd700 1 --2- 172.21.15.46:0/4151493722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fe790 0x5599d73069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f14692ce700 1 -- 172.21.15.46:0/4151493722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d7309640 msgr2=0x5599d7306ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f14692ce700 1 --2- 172.21.15.46:0/4151493722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d7309640 0x5599d7306ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f14692ce700 1 -- 172.21.15.46:0/4151493722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fe790 msgr2=0x5599d73069a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f14692ce700 1 --2- 172.21.15.46:0/4151493722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fe790 0x5599d73069a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f14692ce700 1 -- 172.21.15.46:0/4151493722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599d7212ea0 con 0x5599d72fde70 2024-03-18T20:39:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f1469acf700 1 --2- 172.21.15.46:0/4151493722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d7309640 0x5599d7306ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f14692ce700 1 --2- 172.21.15.46:0/4151493722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fde70 0x5599d73101b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1460005b00 tx=0x7f1460023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884d88b513607aad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f144ffff700 1 -- 172.21.15.46:0/4151493722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146000bee0 con 0x5599d72fde70 2024-03-18T20:39:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f144ffff700 1 -- 172.21.15.46:0/4151493722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f146001e040 con 0x5599d72fde70 2024-03-18T20:39:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.322+0000 7f144ffff700 1 -- 172.21.15.46:0/4151493722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460004cc0 con 0x5599d72fde70 2024-03-18T20:39:15.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.323+0000 7f14776c7b80 1 -- 172.21.15.46:0/4151493722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fde70 msgr2=0x5599d73101b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.323+0000 7f14776c7b80 1 --2- 172.21.15.46:0/4151493722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fde70 0x5599d73101b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1460005b00 tx=0x7f1460023000 comp rx=0 tx=0).stop 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.323+0000 7f14776c7b80 1 -- 172.21.15.46:0/4151493722 shutdown_connections 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.323+0000 7f14776c7b80 1 --2- 172.21.15.46:0/4151493722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d72fde70 0x5599d73101b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.323+0000 7f14776c7b80 1 --2- 172.21.15.46:0/4151493722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d7309640 0x5599d7306ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.323+0000 7f14776c7b80 1 --2- 172.21.15.46:0/4151493722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fe790 0x5599d73069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.323+0000 7f14776c7b80 1 -- 172.21.15.46:0/4151493722 >> 172.21.15.46:0/4151493722 conn(0x5599d7270220 msgr2=0x5599d7301740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.323+0000 7f14776c7b80 1 -- 172.21.15.46:0/4151493722 shutdown_connections 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.323+0000 7f14776c7b80 1 -- 172.21.15.46:0/4151493722 wait complete. 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.323+0000 7f14776c7b80 1 -- start start 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f14776c7b80 1 Processor -- start 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f14776c7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fde70 0x5599d730e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f14776c7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d72fe790 0x5599d730e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f14776c7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d7309640 0x5599d730ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f14776c7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d736bca0 con 0x5599d7309640 2024-03-18T20:39:15.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f14776c7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d736be10 con 0x5599d72fde70 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f14776c7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d736bfd0 con 0x5599d72fe790 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f14776c7b80 10 client.0 _ll_register_callbacks cb 0x7f1460003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f1469acf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d7309640 0x5599d730ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f1468acd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d72fe790 0x5599d730e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f1469acf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d7309640 0x5599d730ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46162/0 (socket says 172.21.15.46:46162) 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f1468acd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d72fe790 0x5599d730e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:52218/0 (socket says 172.21.15.46:52218) 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f1469acf700 1 -- 172.21.15.46:0/2915477761 learned_addr learned my addr 172.21.15.46:0/2915477761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.324+0000 7f14692ce700 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fde70 0x5599d730e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f1468acd700 1 -- 172.21.15.46:0/2915477761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fde70 msgr2=0x5599d730e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f1468acd700 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fde70 0x5599d730e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f1468acd700 1 -- 172.21.15.46:0/2915477761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d7309640 msgr2=0x5599d730ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f1468acd700 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d7309640 0x5599d730ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f1468acd700 1 -- 172.21.15.46:0/2915477761 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146410e470 con 0x5599d72fe790 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f1469acf700 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d7309640 0x5599d730ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:15.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f1468acd700 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d72fe790 0x5599d730e6d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1454095d20 tx=0x7f14540c9f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aa596670fd4dde server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f144cff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f144cff9700 1 -- 172.21.15.46:0/2915477761 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540958b0 con 0x5599d72fe790 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f144cff9700 1 -- 172.21.15.46:0/2915477761 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f14540b92e0 con 0x5599d72fe790 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f14776c7b80 1 -- 172.21.15.46:0/2915477761 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1454095f80 con 0x5599d72fe790 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f144cff9700 1 -- 172.21.15.46:0/2915477761 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1454095a10 con 0x5599d72fe790 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f14776c7b80 10 client.26392 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.325+0000 7f14776c7b80 1 -- 172.21.15.46:0/2915477761 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5599d70d2cd0 con 0x5599d72fe790 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.326+0000 7f144cff9700 1 -- 172.21.15.46:0/2915477761 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1454063070 con 0x5599d72fe790 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.326+0000 7f144cff9700 1 -- 172.21.15.46:0/2915477761 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f14540987d0 con 0x5599d72fe790 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.326+0000 7f14776c7b80 10 client.26392 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.326+0000 7f14776c7b80 10 client.26392 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.326+0000 7f14776c7b80 20 client.26392 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.326+0000 7f14776c7b80 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5599d736d350 0x5599d736f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.326+0000 7f14776c7b80 4 client.26392 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.326+0000 7f14776c7b80 1 -- 172.21.15.46:0/2915477761 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5599d709bcf0 con 0x5599d736d350 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.326+0000 7f14692ce700 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5599d736d350 0x5599d736f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.326+0000 7f14692ce700 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5599d736d350 0x5599d736f770 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=832b2fa383dec1da server_cookie=cc45bd51bc0cd014 in_seq=0 out_seq=0 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.326+0000 7f144cff9700 10 client.26392 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f144cff9700 1 -- 172.21.15.46:0/2915477761 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5599d709bcf0 con 0x5599d736d350 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f144cff9700 10 client.26392 handle_command_reply: tid=0 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f14776c7b80 1 client.26392 shutdown 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f14776c7b80 1 -- 172.21.15.46:0/2915477761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d72fe790 msgr2=0x5599d730e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f14776c7b80 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d72fe790 0x5599d730e6d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1454095d20 tx=0x7f14540c9f90 comp rx=0 tx=0).stop 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f14776c7b80 1 -- 172.21.15.46:0/2915477761 shutdown_connections 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f14776c7b80 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5599d736d350 0x5599d736f770 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f14776c7b80 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5599d7309640 0x5599d730ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f14776c7b80 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5599d72fde70 0x5599d730e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f14776c7b80 1 --2- 172.21.15.46:0/2915477761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5599d72fe790 0x5599d730e6d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f144cff9700 0 client.26392 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:39:15.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f14776c7b80 1 -- 172.21.15.46:0/2915477761 >> 172.21.15.46:0/2915477761 conn(0x5599d7270220 msgr2=0x5599d72f59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f14776c7b80 1 -- 172.21.15.46:0/2915477761 shutdown_connections 2024-03-18T20:39:15.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f14776c7b80 1 -- 172.21.15.46:0/2915477761 wait complete. 2024-03-18T20:39:15.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.327+0000 7f14776c7b80 20 client.26392 trim_cache size 0 max 16384 2024-03-18T20:39:15.336 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:39:15.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.328+0000 7f146b532700 1 -- 172.21.15.46:0/3504393165 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14500a1290 msgr2=0x7f14500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.328+0000 7f146b532700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14500a1290 0x7f14500a3750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f146000b580 tx=0x7f1460005a90 comp rx=0 tx=0).stop 2024-03-18T20:39:15.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.328+0000 7f146b532700 1 -- 172.21.15.46:0/3504393165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14641060b0 msgr2=0x7f1464197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.328+0000 7f146b532700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14641060b0 0x7f1464197740 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f145400d980 tx=0x7f145400dc90 comp rx=0 tx=0).stop 2024-03-18T20:39:15.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.328+0000 7f146b532700 1 -- 172.21.15.46:0/3504393165 shutdown_connections 2024-03-18T20:39:15.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.328+0000 7f146b532700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14500a1290 0x7f14500a3750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.328+0000 7f146b532700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f146419bae0 0x7f146419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.328+0000 7f146b532700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14640692a0 0x7f1464197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.328+0000 7f146b532700 1 --2- 172.21.15.46:0/3504393165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14641060b0 0x7f1464197740 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.328+0000 7f146b532700 1 -- 172.21.15.46:0/3504393165 >> 172.21.15.46:0/3504393165 conn(0x7f1464076d80 msgr2=0x7f14641071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.328+0000 7f146b532700 1 -- 172.21.15.46:0/3504393165 shutdown_connections 2024-03-18T20:39:15.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.328+0000 7f146b532700 1 -- 172.21.15.46:0/3504393165 wait complete. 2024-03-18T20:39:15.344 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:39:15.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:39:15.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:15 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/570640738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:39:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.479+0000 7f4a05147700 1 -- 172.21.15.46:0/2342247742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a001060f0 msgr2=0x7f4a0010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.479+0000 7f4a05147700 1 --2- 172.21.15.46:0/2342247742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a001060f0 0x7f4a0010dc90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009f90 tx=0x7f49f800b330 comp rx=0 tx=0).stop 2024-03-18T20:39:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.479+0000 7f4a05147700 1 -- 172.21.15.46:0/2342247742 shutdown_connections 2024-03-18T20:39:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.479+0000 7f4a05147700 1 --2- 172.21.15.46:0/2342247742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a000ff8e0 0x7f4a000ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.479+0000 7f4a05147700 1 --2- 172.21.15.46:0/2342247742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a001060f0 0x7f4a0010dc90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.479+0000 7f4a05147700 1 --2- 172.21.15.46:0/2342247742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a00100290 0x7f4a001059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.479+0000 7f4a05147700 1 -- 172.21.15.46:0/2342247742 >> 172.21.15.46:0/2342247742 conn(0x7f4a00074f00 msgr2=0x7f4a00075310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.479+0000 7f4a05147700 1 -- 172.21.15.46:0/2342247742 shutdown_connections 2024-03-18T20:39:15.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.479+0000 7f4a05147700 1 -- 172.21.15.46:0/2342247742 wait complete. 2024-03-18T20:39:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.480+0000 7f4a05147700 1 Processor -- start 2024-03-18T20:39:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f4a05147700 1 -- start start 2024-03-18T20:39:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f4a05147700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a000ff8e0 0x7f4a00100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f4a05147700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a001060f0 0x7f4a00100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f4a05147700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001012d0 0x7f4a001a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f4a05147700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001a58e0 con 0x7f4a001012d0 2024-03-18T20:39:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f4a05147700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001a5a50 con 0x7f4a001060f0 2024-03-18T20:39:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f4a05147700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001a5c10 con 0x7f4a000ff8e0 2024-03-18T20:39:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f49ff59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001012d0 0x7f4a001a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f49ff59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001012d0 0x7f4a001a5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46182/0 (socket says 172.21.15.46:46182) 2024-03-18T20:39:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f49ff59e700 1 -- 172.21.15.46:0/238071367 learned_addr learned my addr 172.21.15.46:0/238071367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f49fed9d700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a000ff8e0 0x7f4a00100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f49fe59c700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a001060f0 0x7f4a00100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f49ff59e700 1 -- 172.21.15.46:0/238071367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a000ff8e0 msgr2=0x7f4a00100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f49ff59e700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a000ff8e0 0x7f4a00100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f49ff59e700 1 -- 172.21.15.46:0/238071367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a001060f0 msgr2=0x7f4a00100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f49ff59e700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a001060f0 0x7f4a00100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f49ff59e700 1 -- 172.21.15.46:0/238071367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f00095f0 con 0x7f4a001012d0 2024-03-18T20:39:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.481+0000 7f49ff59e700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001012d0 0x7f4a001a5310 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f49f800b6c0 tx=0x7f49f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.482+0000 7f49f7fff700 1 -- 172.21.15.46:0/238071367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f800e040 con 0x7f4a001012d0 2024-03-18T20:39:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.482+0000 7f4a05147700 1 -- 172.21.15.46:0/238071367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f8009c40 con 0x7f4a001012d0 2024-03-18T20:39:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.482+0000 7f49f7fff700 1 -- 172.21.15.46:0/238071367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f800bbf0 con 0x7f4a001012d0 2024-03-18T20:39:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.482+0000 7f49f7fff700 1 -- 172.21.15.46:0/238071367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f801b7c0 con 0x7f4a001012d0 2024-03-18T20:39:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.482+0000 7f4a05147700 1 -- 172.21.15.46:0/238071367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a001a61d0 con 0x7f4a001012d0 2024-03-18T20:39:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.484+0000 7f49f7fff700 1 -- 172.21.15.46:0/238071367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f49f801d480 con 0x7f4a001012d0 2024-03-18T20:39:15.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.485+0000 7f49f7fff700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f49ec0a1290 0x7f49ec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.485+0000 7f49f7fff700 1 -- 172.21.15.46:0/238071367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f49f80ce300 con 0x7f4a001012d0 2024-03-18T20:39:15.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.485+0000 7f49fed9d700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f49ec0a1290 0x7f49ec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.485+0000 7f49fed9d700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f49ec0a1290 0x7f49ec0a3750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49f0005fd0 tx=0x7f49f0009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.485+0000 7f4a112dcb80 1 Processor -- start 2024-03-18T20:39:15.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.486+0000 7f4a112dcb80 1 -- start start 2024-03-18T20:39:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.486+0000 7f4a112dcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x55576918fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.486+0000 7f4a112dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 0x55576919ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.486+0000 7f4a112dcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 0x55576919d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.486+0000 7f4a112dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557690e9990 con 0x555769190380 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.486+0000 7f4a112dcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557690e7350 con 0x55576918fa60 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.486+0000 7f4a112dcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557690e2c40 con 0x55576919b220 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.486+0000 7f49fe59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 0x55576919ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.486+0000 7f49fe59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 0x55576919ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46184/0 (socket says 172.21.15.46:46184) 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.486+0000 7f49fe59c700 1 -- 172.21.15.46:0/1547580764 learned_addr learned my addr 172.21.15.46:0/1547580764 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.486+0000 7f49fed9d700 1 --2- 172.21.15.46:0/1547580764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x55576918fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.486+0000 7f49ff59e700 1 --2- 172.21.15.46:0/1547580764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 0x55576919d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.487+0000 7f49fe59c700 1 -- 172.21.15.46:0/1547580764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 msgr2=0x55576919d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.487+0000 7f49fe59c700 1 --2- 172.21.15.46:0/1547580764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 0x55576919d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.487+0000 7f49fe59c700 1 -- 172.21.15.46:0/1547580764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 msgr2=0x55576918fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.487+0000 7f49fe59c700 1 --2- 172.21.15.46:0/1547580764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x55576918fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.487+0000 7f49fe59c700 1 -- 172.21.15.46:0/1547580764 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555768f643b0 con 0x555769190380 2024-03-18T20:39:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.487+0000 7f49fe59c700 1 --2- 172.21.15.46:0/1547580764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 0x55576919ace0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f49e800dba0 tx=0x7f49e800deb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81ec468776909c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.487+0000 7f49f57fa700 1 -- 172.21.15.46:0/1547580764 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e800fb10 con 0x555769190380 2024-03-18T20:39:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.487+0000 7f49f57fa700 1 -- 172.21.15.46:0/1547580764 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49e8004cf0 con 0x555769190380 2024-03-18T20:39:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.487+0000 7f49f57fa700 1 -- 172.21.15.46:0/1547580764 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8005660 con 0x555769190380 2024-03-18T20:39:15.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.488+0000 7f4a112dcb80 1 -- 172.21.15.46:0/1547580764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 msgr2=0x55576919ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.488+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/1547580764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 0x55576919ace0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f49e800dba0 tx=0x7f49e800deb0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.488+0000 7f4a112dcb80 1 -- 172.21.15.46:0/1547580764 shutdown_connections 2024-03-18T20:39:15.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.488+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/1547580764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 0x55576919d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.488+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/1547580764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x55576918fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.488+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/1547580764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 0x55576919ace0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.488+0000 7f4a112dcb80 1 -- 172.21.15.46:0/1547580764 >> 172.21.15.46:0/1547580764 conn(0x555769106220 msgr2=0x55576918c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.488+0000 7f4a112dcb80 1 -- 172.21.15.46:0/1547580764 shutdown_connections 2024-03-18T20:39:15.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.488+0000 7f4a112dcb80 1 -- 172.21.15.46:0/1547580764 wait complete. 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.488+0000 7f4a112dcb80 1 -- start start 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.488+0000 7f4a112dcb80 1 Processor -- start 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f4a112dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576918fa60 0x55576919abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f4a112dcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555769190380 0x555769197f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f4a112dcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 0x555769198590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f4a112dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55576907ed80 con 0x55576918fa60 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f4a112dcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55576907ea20 con 0x555769190380 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f4a112dcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555768f666a0 con 0x55576919b220 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f4a112dcb80 10 client.0 _ll_register_callbacks cb 0x555768eea3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49fed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576918fa60 0x55576919abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49fed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576918fa60 0x55576919abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46200/0 (socket says 172.21.15.46:46200) 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49fed9d700 1 -- 172.21.15.46:0/3255195280 learned_addr learned my addr 172.21.15.46:0/3255195280 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49fe59c700 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555769190380 0x555769197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49ff59e700 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 0x555769198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49fed9d700 1 -- 172.21.15.46:0/3255195280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 msgr2=0x555769198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49fed9d700 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 0x555769198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49fed9d700 1 -- 172.21.15.46:0/3255195280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555769190380 msgr2=0x555769197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49fed9d700 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555769190380 0x555769197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49fed9d700 1 -- 172.21.15.46:0/3255195280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e800d7d0 con 0x55576918fa60 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49fe59c700 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555769190380 0x555769197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49fed9d700 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576918fa60 0x55576919abd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f49f000ede0 tx=0x7f49f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b69d73415df6c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.489+0000 7f49e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f49e67fc700 1 -- 172.21.15.46:0/3255195280 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0022070 con 0x55576918fa60 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f49e67fc700 1 -- 172.21.15.46:0/3255195280 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f49f000ee10 con 0x55576918fa60 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f4a112dcb80 1 -- 172.21.15.46:0/3255195280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555769183ad0 con 0x55576918fa60 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f49e67fc700 1 -- 172.21.15.46:0/3255195280 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49f001da30 con 0x55576918fa60 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f4a112dcb80 10 client.16656 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:15.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f4a112dcb80 1 -- 172.21.15.46:0/3255195280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555768f10b60 con 0x55576918fa60 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f49e67fc700 1 -- 172.21.15.46:0/3255195280 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f0033020 con 0x55576918fa60 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f49e67fc700 1 -- 172.21.15.46:0/3255195280 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f49f001cca0 con 0x55576918fa60 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f4a112dcb80 10 client.16656 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f4a112dcb80 10 client.16656 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f4a112dcb80 20 client.16656 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5557691aadf0 0x5557691ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f4a112dcb80 4 client.16656 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f4a112dcb80 1 -- 172.21.15.46:0/3255195280 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557690db700 con 0x5557691aadf0 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.490+0000 7f49fe59c700 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5557691aadf0 0x5557691ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.491+0000 7f49e67fc700 10 client.16656 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.491+0000 7f49fe59c700 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5557691aadf0 0x5557691ad210 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c2970dff8bdf159 server_cookie=3d32598ef462d268 in_seq=0 out_seq=0 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.491+0000 7f49e67fc700 1 -- 172.21.15.46:0/3255195280 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5557690db700 con 0x5557691aadf0 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.491+0000 7f49e67fc700 10 client.16656 handle_command_reply: tid=0 2024-03-18T20:39:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.491+0000 7f4a112dcb80 1 client.16656 shutdown 2024-03-18T20:39:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.492+0000 7f4a112dcb80 1 -- 172.21.15.46:0/3255195280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576918fa60 msgr2=0x55576919abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.492+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576918fa60 0x55576919abd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f49f000ede0 tx=0x7f49f00047c0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.492+0000 7f4a112dcb80 1 -- 172.21.15.46:0/3255195280 shutdown_connections 2024-03-18T20:39:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.492+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5557691aadf0 0x5557691ad210 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.492+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576918fa60 0x55576919abd0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.492+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 0x555769198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.492+0000 7f49e67fc700 0 client.16656 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:39:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.492+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/3255195280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555769190380 0x555769197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.492+0000 7f4a112dcb80 1 -- 172.21.15.46:0/3255195280 >> 172.21.15.46:0/3255195280 conn(0x555769106220 msgr2=0x55576918bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.492+0000 7f4a112dcb80 1 -- 172.21.15.46:0/3255195280 shutdown_connections 2024-03-18T20:39:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.492+0000 7f4a112dcb80 1 -- 172.21.15.46:0/3255195280 wait complete. 2024-03-18T20:39:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.492+0000 7f4a112dcb80 20 client.16656 trim_cache size 0 max 16384 2024-03-18T20:39:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.506+0000 7f4a112dcb80 1 Processor -- start 2024-03-18T20:39:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.506+0000 7f4a112dcb80 1 -- start start 2024-03-18T20:39:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.506+0000 7f4a112dcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x555769192a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.506+0000 7f4a112dcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555769190380 0x5557691a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.506+0000 7f4a112dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576919b220 0x5557691a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.506+0000 7f4a112dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55576918d230 con 0x55576919b220 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.506+0000 7f4a112dcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555769190d30 con 0x55576918fa60 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.506+0000 7f4a112dcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557691901c0 con 0x555769190380 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49ff59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576919b220 0x5557691a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49fe59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555769190380 0x5557691a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49ff59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576919b220 0x5557691a65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46214/0 (socket says 172.21.15.46:46214) 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49fe59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555769190380 0x5557691a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:52262/0 (socket says 172.21.15.46:52262) 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49ff59e700 1 -- 172.21.15.46:0/708090025 learned_addr learned my addr 172.21.15.46:0/708090025 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49fed9d700 1 --2- 172.21.15.46:0/708090025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x555769192a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49ff59e700 1 -- 172.21.15.46:0/708090025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555769190380 msgr2=0x5557691a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49ff59e700 1 --2- 172.21.15.46:0/708090025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555769190380 0x5557691a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49ff59e700 1 -- 172.21.15.46:0/708090025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 msgr2=0x555769192a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49ff59e700 1 --2- 172.21.15.46:0/708090025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x555769192a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49ff59e700 1 -- 172.21.15.46:0/708090025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557690a8ea0 con 0x55576919b220 2024-03-18T20:39:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49fed9d700 1 --2- 172.21.15.46:0/708090025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x555769192a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49fe59c700 1 --2- 172.21.15.46:0/708090025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555769190380 0x5557691a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49ff59e700 1 --2- 172.21.15.46:0/708090025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576919b220 0x5557691a65a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f49f809b440 tx=0x7f49f8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada6fa63a4680a93 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49f5ffb700 1 -- 172.21.15.46:0/708090025 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f809f0d0 con 0x55576919b220 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49f5ffb700 1 -- 172.21.15.46:0/708090025 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f809bbb0 con 0x55576919b220 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.507+0000 7f49f5ffb700 1 -- 172.21.15.46:0/708090025 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f801d730 con 0x55576919b220 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.508+0000 7f4a112dcb80 1 -- 172.21.15.46:0/708090025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576919b220 msgr2=0x5557691a65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.508+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/708090025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576919b220 0x5557691a65a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f49f809b440 tx=0x7f49f8009040 comp rx=0 tx=0).stop 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.508+0000 7f4a112dcb80 1 -- 172.21.15.46:0/708090025 shutdown_connections 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.508+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/708090025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55576919b220 0x5557691a65a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.508+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/708090025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x555769192a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.508+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/708090025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555769190380 0x5557691a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.508+0000 7f4a112dcb80 1 -- 172.21.15.46:0/708090025 >> 172.21.15.46:0/708090025 conn(0x555769106220 msgr2=0x555769196830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.508+0000 7f4a112dcb80 1 -- 172.21.15.46:0/708090025 shutdown_connections 2024-03-18T20:39:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.508+0000 7f4a112dcb80 1 -- 172.21.15.46:0/708090025 wait complete. 2024-03-18T20:39:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.508+0000 7f4a112dcb80 1 -- start start 2024-03-18T20:39:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f4a112dcb80 1 Processor -- start 2024-03-18T20:39:15.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f4a112dcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x55576919fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f4a112dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 0x5557691a0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f4a112dcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 0x5557691a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f4a112dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557691a2440 con 0x555769190380 2024-03-18T20:39:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f4a112dcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557691fdc20 con 0x55576918fa60 2024-03-18T20:39:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f4a112dcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557691fddb0 con 0x55576919b220 2024-03-18T20:39:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f4a112dcb80 10 client.0 _ll_register_callbacks cb 0x7f49f000e9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f49fe59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 0x5557691a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f49fe59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 0x5557691a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46224/0 (socket says 172.21.15.46:46224) 2024-03-18T20:39:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f49fe59c700 1 -- 172.21.15.46:0/2087361611 learned_addr learned my addr 172.21.15.46:0/2087361611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f49fed9d700 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x55576919fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f49ff59e700 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 0x5557691a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f49fe59c700 1 -- 172.21.15.46:0/2087361611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 msgr2=0x5557691a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f49fe59c700 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 0x5557691a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f49fe59c700 1 -- 172.21.15.46:0/2087361611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 msgr2=0x55576919fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.509+0000 7f49fe59c700 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x55576919fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f49fe59c700 1 -- 172.21.15.46:0/2087361611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555768f643b0 con 0x555769190380 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f49fed9d700 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x55576919fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f49fe59c700 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 0x5557691a0370 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f49e8000c00 tx=0x7f49e800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a108fb03a50e0be2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f49e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f49e67fc700 1 -- 172.21.15.46:0/2087361611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8021040 con 0x555769190380 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f49e67fc700 1 -- 172.21.15.46:0/2087361611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f49e80054b0 con 0x555769190380 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f4a112dcb80 1 -- 172.21.15.46:0/2087361611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49e8004cf0 con 0x555769190380 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f49e67fc700 1 -- 172.21.15.46:0/2087361611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49e8005670 con 0x555769190380 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f4a112dcb80 10 client.16662 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f4a112dcb80 1 -- 172.21.15.46:0/2087361611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f4a001a61d0 con 0x555769190380 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f49e67fc700 1 -- 172.21.15.46:0/2087361611 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49e8004cf0 con 0x555769190380 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f49e67fc700 1 -- 172.21.15.46:0/2087361611 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f49e800f990 con 0x555769190380 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f4a112dcb80 10 client.16662 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:15.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.510+0000 7f4a112dcb80 10 client.16662 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 20 client.16662 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5557691ff5d0 0x5557692019c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 4 client.16662 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 1 -- 172.21.15.46:0/2087361611 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555768f31cf0 con 0x5557691ff5d0 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f49fed9d700 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5557691ff5d0 0x5557692019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f49e67fc700 10 client.16662 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f49fed9d700 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5557691ff5d0 0x5557692019c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=641a98820bbc4a81 server_cookie=e8083ed4ca8ca711 in_seq=0 out_seq=0 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f49e67fc700 1 -- 172.21.15.46:0/2087361611 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555768f31cf0 con 0x5557691ff5d0 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f49e67fc700 10 client.16662 handle_command_reply: tid=0 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 1 client.16662 shutdown 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 1 -- 172.21.15.46:0/2087361611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 msgr2=0x5557691a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 0x5557691a0370 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f49e8000c00 tx=0x7f49e800d4a0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 1 -- 172.21.15.46:0/2087361611 shutdown_connections 2024-03-18T20:39:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5557691ff5d0 0x5557692019c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55576919b220 0x5557691a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55576918fa60 0x55576919fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f49e67fc700 0 client.16662 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:39:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 1 --2- 172.21.15.46:0/2087361611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555769190380 0x5557691a0370 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.511+0000 7f4a112dcb80 1 -- 172.21.15.46:0/2087361611 >> 172.21.15.46:0/2087361611 conn(0x555769106220 msgr2=0x5557691a65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a112dcb80 1 -- 172.21.15.46:0/2087361611 shutdown_connections 2024-03-18T20:39:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a112dcb80 1 -- 172.21.15.46:0/2087361611 wait complete. 2024-03-18T20:39:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a112dcb80 20 client.16662 trim_cache size 0 max 16384 2024-03-18T20:39:15.521 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:39:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a05147700 1 -- 172.21.15.46:0/238071367 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f49ec0a1290 msgr2=0x7f49ec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a05147700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f49ec0a1290 0x7f49ec0a3750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49f0005fd0 tx=0x7f49f0009310 comp rx=0 tx=0).stop 2024-03-18T20:39:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a05147700 1 -- 172.21.15.46:0/238071367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001012d0 msgr2=0x7f4a001a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a05147700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001012d0 0x7f4a001a5310 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f49f800b6c0 tx=0x7f49f8004ad0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a05147700 1 -- 172.21.15.46:0/238071367 shutdown_connections 2024-03-18T20:39:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a05147700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f49ec0a1290 0x7f49ec0a3750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a05147700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a001012d0 0x7f4a001a5310 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a05147700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a000ff8e0 0x7f4a00100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a05147700 1 --2- 172.21.15.46:0/238071367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a001060f0 0x7f4a00100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a05147700 1 -- 172.21.15.46:0/238071367 >> 172.21.15.46:0/238071367 conn(0x7f4a00074f00 msgr2=0x7f4a000fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a05147700 1 -- 172.21.15.46:0/238071367 shutdown_connections 2024-03-18T20:39:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.512+0000 7f4a05147700 1 -- 172.21.15.46:0/238071367 wait complete. 2024-03-18T20:39:15.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:15 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/570640738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:39:15.529 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:39:15.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:39:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:15 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/570640738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:39:15.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.664+0000 7f9668158700 1 -- 172.21.15.46:0/3458918412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9660106e90 msgr2=0x7f9660107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.664+0000 7f9668158700 1 --2- 172.21.15.46:0/3458918412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9660106e90 0x7f9660107270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f965c009990 tx=0x7f965c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.665+0000 7f9668158700 1 -- 172.21.15.46:0/3458918412 shutdown_connections 2024-03-18T20:39:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.665+0000 7f9668158700 1 --2- 172.21.15.46:0/3458918412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9660106e90 0x7f9660107270 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.665+0000 7f9668158700 1 --2- 172.21.15.46:0/3458918412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96600ffc20 0x7f966010dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.665+0000 7f9668158700 1 --2- 172.21.15.46:0/3458918412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96600ff280 0x7f96600ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.665+0000 7f9668158700 1 -- 172.21.15.46:0/3458918412 >> 172.21.15.46:0/3458918412 conn(0x7f96600761e0 msgr2=0x7f96600fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.665+0000 7f9668158700 1 -- 172.21.15.46:0/3458918412 shutdown_connections 2024-03-18T20:39:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.665+0000 7f9668158700 1 -- 172.21.15.46:0/3458918412 wait complete. 2024-03-18T20:39:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.665+0000 7f9668158700 1 Processor -- start 2024-03-18T20:39:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.665+0000 7f9668158700 1 -- start start 2024-03-18T20:39:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9668158700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96600ff280 0x7f9660100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9668158700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96600ffc20 0x7f96601009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9668158700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9660106e90 0x7f9660100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9668158700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96601a56b0 con 0x7f96600ff280 2024-03-18T20:39:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9668158700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96601a5820 con 0x7f96600ffc20 2024-03-18T20:39:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9668158700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96601a59e0 con 0x7f9660106e90 2024-03-18T20:39:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9665ef4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96600ff280 0x7f9660100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9665ef4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96600ff280 0x7f9660100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46250/0 (socket says 172.21.15.46:46250) 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9665ef4700 1 -- 172.21.15.46:0/1217877010 learned_addr learned my addr 172.21.15.46:0/1217877010 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f96656f3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96600ffc20 0x7f96601009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f96666f5700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9660106e90 0x7f9660100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9665ef4700 1 -- 172.21.15.46:0/1217877010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9660106e90 msgr2=0x7f9660100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9665ef4700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9660106e90 0x7f9660100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9665ef4700 1 -- 172.21.15.46:0/1217877010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96600ffc20 msgr2=0x7f96601009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9665ef4700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96600ffc20 0x7f96601009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9665ef4700 1 -- 172.21.15.46:0/1217877010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c0095f0 con 0x7f96600ff280 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9665ef4700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96600ff280 0x7f9660100460 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f965c000c00 tx=0x7f965c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9656ffd700 1 -- 172.21.15.46:0/1217877010 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c01d070 con 0x7f96600ff280 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.666+0000 7f9656ffd700 1 -- 172.21.15.46:0/1217877010 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f965c022470 con 0x7f96600ff280 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.667+0000 7f9668158700 1 -- 172.21.15.46:0/1217877010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96601a5c60 con 0x7f96600ff280 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.667+0000 7f9656ffd700 1 -- 172.21.15.46:0/1217877010 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c017650 con 0x7f96600ff280 2024-03-18T20:39:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.667+0000 7f9668158700 1 -- 172.21.15.46:0/1217877010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96601a60e0 con 0x7f96600ff280 2024-03-18T20:39:15.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.669+0000 7f9656ffd700 1 -- 172.21.15.46:0/1217877010 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f965c0225e0 con 0x7f96600ff280 2024-03-18T20:39:15.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.670+0000 7f9656ffd700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f964c0a1310 0x7f964c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.670+0000 7f96656f3700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f964c0a1310 0x7f964c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.670+0000 7f9656ffd700 1 -- 172.21.15.46:0/1217877010 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f965c0cde40 con 0x7f96600ff280 2024-03-18T20:39:15.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.670+0000 7f96742edb80 1 Processor -- start 2024-03-18T20:39:15.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.670+0000 7f96656f3700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f964c0a1310 0x7f964c0a37d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9660101b90 tx=0x7f9650009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96742edb80 1 -- start start 2024-03-18T20:39:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96742edb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x562707453e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96742edb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707454380 0x56270745ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96742edb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56270745f220 0x562707461610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96742edb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627073ad990 con 0x562707454380 2024-03-18T20:39:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96742edb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627073ab350 con 0x56270745f220 2024-03-18T20:39:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96742edb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627073a6c40 con 0x562707453a60 2024-03-18T20:39:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96656f3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707454380 0x56270745ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96656f3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707454380 0x56270745ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46258/0 (socket says 172.21.15.46:46258) 2024-03-18T20:39:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96656f3700 1 -- 172.21.15.46:0/2378490310 learned_addr learned my addr 172.21.15.46:0/2378490310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f9665ef4700 1 --2- 172.21.15.46:0/2378490310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x562707453e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96666f5700 1 --2- 172.21.15.46:0/2378490310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56270745f220 0x562707461610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96656f3700 1 -- 172.21.15.46:0/2378490310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 msgr2=0x562707453e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96656f3700 1 --2- 172.21.15.46:0/2378490310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x562707453e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96656f3700 1 -- 172.21.15.46:0/2378490310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56270745f220 msgr2=0x562707461610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96656f3700 1 --2- 172.21.15.46:0/2378490310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56270745f220 0x562707461610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.671+0000 7f96656f3700 1 -- 172.21.15.46:0/2378490310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627072283b0 con 0x562707454380 2024-03-18T20:39:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.672+0000 7f96656f3700 1 --2- 172.21.15.46:0/2378490310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707454380 0x56270745ece0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9650011d50 tx=0x7f9650014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c88d0df627ab8a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.672+0000 7f964bfff700 1 -- 172.21.15.46:0/2378490310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965000eb20 con 0x562707454380 2024-03-18T20:39:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.672+0000 7f964bfff700 1 -- 172.21.15.46:0/2378490310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9650021070 con 0x562707454380 2024-03-18T20:39:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.672+0000 7f964bfff700 1 -- 172.21.15.46:0/2378490310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965001d670 con 0x562707454380 2024-03-18T20:39:15.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.673+0000 7f96742edb80 1 -- 172.21.15.46:0/2378490310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707454380 msgr2=0x56270745ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.673+0000 7f96742edb80 1 --2- 172.21.15.46:0/2378490310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707454380 0x56270745ece0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9650011d50 tx=0x7f9650014000 comp rx=0 tx=0).stop 2024-03-18T20:39:15.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.673+0000 7f96742edb80 1 -- 172.21.15.46:0/2378490310 shutdown_connections 2024-03-18T20:39:15.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.673+0000 7f96742edb80 1 --2- 172.21.15.46:0/2378490310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56270745f220 0x562707461610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.673+0000 7f96742edb80 1 --2- 172.21.15.46:0/2378490310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x562707453e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.673+0000 7f96742edb80 1 --2- 172.21.15.46:0/2378490310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707454380 0x56270745ece0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.673+0000 7f96742edb80 1 -- 172.21.15.46:0/2378490310 >> 172.21.15.46:0/2378490310 conn(0x5627073ca220 msgr2=0x562707450020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.673+0000 7f96742edb80 1 -- 172.21.15.46:0/2378490310 shutdown_connections 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.673+0000 7f96742edb80 1 -- 172.21.15.46:0/2378490310 wait complete. 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.673+0000 7f96742edb80 1 -- start start 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.673+0000 7f96742edb80 1 Processor -- start 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96742edb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x56270745ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96742edb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562707454380 0x56270745c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96742edb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 0x56270745c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96742edb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562707342d80 con 0x56270745f220 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96742edb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562707342a20 con 0x562707454380 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96742edb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56270722a6a0 con 0x562707453a60 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96742edb80 10 client.0 _ll_register_callbacks cb 0x5627071ae3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96666f5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 0x56270745c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f9665ef4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x56270745ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96666f5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 0x56270745c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46272/0 (socket says 172.21.15.46:46272) 2024-03-18T20:39:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96666f5700 1 -- 172.21.15.46:0/3776346763 learned_addr learned my addr 172.21.15.46:0/3776346763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96656f3700 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562707454380 0x56270745c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96666f5700 1 -- 172.21.15.46:0/3776346763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 msgr2=0x56270745ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96666f5700 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x56270745ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96666f5700 1 -- 172.21.15.46:0/3776346763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562707454380 msgr2=0x56270745c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96666f5700 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562707454380 0x56270745c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96666f5700 1 -- 172.21.15.46:0/3776346763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9650017070 con 0x56270745f220 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f9665ef4700 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x56270745ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96666f5700 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 0x56270745c7a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f96580042a0 tx=0x7f96580042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227dd79f5db757a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.674+0000 7f96497fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96497fa700 1 -- 172.21.15.46:0/3776346763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658004db0 con 0x56270745f220 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96497fa700 1 -- 172.21.15.46:0/3776346763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9658007500 con 0x56270745f220 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96742edb80 1 -- 172.21.15.46:0/3776346763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562707447ad0 con 0x56270745f220 2024-03-18T20:39:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96497fa700 1 -- 172.21.15.46:0/3776346763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96580076c0 con 0x56270745f220 2024-03-18T20:39:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96742edb80 10 client.16674 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96742edb80 1 -- 172.21.15.46:0/3776346763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5627071d4b60 con 0x56270745f220 2024-03-18T20:39:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96497fa700 1 -- 172.21.15.46:0/3776346763 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562707447ad0 con 0x56270745f220 2024-03-18T20:39:15.683 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82426 0.03 MB/sec execute 10 sec latency 3600.724 ms 2024-03-18T20:39:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96497fa700 1 -- 172.21.15.46:0/3776346763 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9658016870 con 0x56270745f220 2024-03-18T20:39:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96742edb80 10 client.16674 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96742edb80 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96742edb80 20 client.16674 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96742edb80 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56270746eed0 0x5627074712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96742edb80 4 client.16674 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.675+0000 7f96742edb80 1 -- 172.21.15.46:0/3776346763 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56270739f700 con 0x56270746eed0 2024-03-18T20:39:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.676+0000 7f9665ef4700 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56270746eed0 0x5627074712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.676+0000 7f9665ef4700 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56270746eed0 0x5627074712f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1ff16413c235b54 server_cookie=83cb2a318f53d98a in_seq=0 out_seq=0 2024-03-18T20:39:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.676+0000 7f96497fa700 10 client.16674 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96497fa700 1 -- 172.21.15.46:0/3776346763 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56270739f700 con 0x56270746eed0 2024-03-18T20:39:15.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96497fa700 10 client.16674 handle_command_reply: tid=0 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96742edb80 1 client.16674 shutdown 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96742edb80 1 -- 172.21.15.46:0/3776346763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 msgr2=0x56270745c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96742edb80 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 0x56270745c7a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f96580042a0 tx=0x7f96580042d0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96742edb80 1 -- 172.21.15.46:0/3776346763 shutdown_connections 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96742edb80 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56270746eed0 0x5627074712f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96742edb80 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 0x56270745c7a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96742edb80 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x56270745ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96742edb80 1 --2- 172.21.15.46:0/3776346763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562707454380 0x56270745c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96497fa700 0 client.16674 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96742edb80 1 -- 172.21.15.46:0/3776346763 >> 172.21.15.46:0/3776346763 conn(0x5627073ca220 msgr2=0x56270744fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96742edb80 1 -- 172.21.15.46:0/3776346763 shutdown_connections 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96742edb80 1 -- 172.21.15.46:0/3776346763 wait complete. 2024-03-18T20:39:15.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.677+0000 7f96742edb80 20 client.16674 trim_cache size 0 max 16384 2024-03-18T20:39:15.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f96742edb80 1 Processor -- start 2024-03-18T20:39:15.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f96742edb80 1 -- start start 2024-03-18T20:39:15.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f96742edb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707453a60 0x562707465e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f96742edb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707454380 0x56270746a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f96742edb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56270745f220 0x56270745e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f96742edb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562707466390 con 0x562707453a60 2024-03-18T20:39:15.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f96742edb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56270746c500 con 0x56270745f220 2024-03-18T20:39:15.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f96742edb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56270745a5b0 con 0x562707454380 2024-03-18T20:39:15.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f9665ef4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707453a60 0x562707465e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f9665ef4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707453a60 0x562707465e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46282/0 (socket says 172.21.15.46:46282) 2024-03-18T20:39:15.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f9665ef4700 1 -- 172.21.15.46:0/2813138759 learned_addr learned my addr 172.21.15.46:0/2813138759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f96656f3700 1 --2- 172.21.15.46:0/2813138759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707454380 0x56270746a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f96666f5700 1 --2- 172.21.15.46:0/2813138759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56270745f220 0x56270745e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f9665ef4700 1 -- 172.21.15.46:0/2813138759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707454380 msgr2=0x56270746a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f9665ef4700 1 --2- 172.21.15.46:0/2813138759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707454380 0x56270746a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f9665ef4700 1 -- 172.21.15.46:0/2813138759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56270745f220 msgr2=0x56270745e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f9665ef4700 1 --2- 172.21.15.46:0/2813138759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56270745f220 0x56270745e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f9665ef4700 1 -- 172.21.15.46:0/2813138759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56270736cea0 con 0x562707453a60 2024-03-18T20:39:15.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f96666f5700 1 --2- 172.21.15.46:0/2813138759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56270745f220 0x56270745e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:39:15.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.691+0000 7f96656f3700 1 --2- 172.21.15.46:0/2813138759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707454380 0x56270746a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:15.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.692+0000 7f9665ef4700 1 --2- 172.21.15.46:0/2813138759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707453a60 0x562707465e50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f965c005b40 tx=0x7f965c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e37bcfd4210461 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.692+0000 7f9654ff9700 1 -- 172.21.15.46:0/2813138759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c09afb0 con 0x562707453a60 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.692+0000 7f9654ff9700 1 -- 172.21.15.46:0/2813138759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f965c09ede0 con 0x562707453a60 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.692+0000 7f9654ff9700 1 -- 172.21.15.46:0/2813138759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c09b470 con 0x562707453a60 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.692+0000 7f96742edb80 1 -- 172.21.15.46:0/2813138759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707453a60 msgr2=0x562707465e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.692+0000 7f96742edb80 1 --2- 172.21.15.46:0/2813138759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707453a60 0x562707465e50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f965c005b40 tx=0x7f965c01a040 comp rx=0 tx=0).stop 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 -- 172.21.15.46:0/2813138759 shutdown_connections 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 --2- 172.21.15.46:0/2813138759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562707453a60 0x562707465e50 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 --2- 172.21.15.46:0/2813138759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56270745f220 0x56270745e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 --2- 172.21.15.46:0/2813138759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707454380 0x56270746a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 -- 172.21.15.46:0/2813138759 >> 172.21.15.46:0/2813138759 conn(0x5627073ca220 msgr2=0x56270744ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 -- 172.21.15.46:0/2813138759 shutdown_connections 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 -- 172.21.15.46:0/2813138759 wait complete. 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 -- start start 2024-03-18T20:39:15.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 Processor -- start 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x562707463e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562707454380 0x562707464370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 0x5627074648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627074c1d50 con 0x56270745f220 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627074c1ec0 con 0x562707454380 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.693+0000 7f96742edb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627074c2080 con 0x562707453a60 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f96742edb80 10 client.0 _ll_register_callbacks cb 0x5627071ae3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f96666f5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 0x5627074648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f96666f5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 0x5627074648b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46292/0 (socket says 172.21.15.46:46292) 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f9665ef4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x562707463e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f96666f5700 1 -- 172.21.15.46:0/1851818723 learned_addr learned my addr 172.21.15.46:0/1851818723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f96656f3700 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562707454380 0x562707464370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f9665ef4700 1 -- 172.21.15.46:0/1851818723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562707454380 msgr2=0x562707464370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f9665ef4700 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562707454380 0x562707464370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f9665ef4700 1 -- 172.21.15.46:0/1851818723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 msgr2=0x5627074648b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:15.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f9665ef4700 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 0x5627074648b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f9665ef4700 1 -- 172.21.15.46:0/1851818723 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96601a60e0 con 0x562707453a60 2024-03-18T20:39:15.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f96666f5700 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 0x5627074648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:15.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f9665ef4700 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x562707463e30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f965c097a20 tx=0x7f965c09a9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=659c3a1ed0e818c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:15.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f96497fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:39:15.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.694+0000 7f96497fa700 1 -- 172.21.15.46:0/1851818723 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c0b9300 con 0x562707453a60 2024-03-18T20:39:15.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96742edb80 1 -- 172.21.15.46:0/1851818723 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9650021070 con 0x562707453a60 2024-03-18T20:39:15.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96497fa700 1 -- 172.21.15.46:0/1851818723 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f965c09afb0 con 0x562707453a60 2024-03-18T20:39:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96497fa700 1 -- 172.21.15.46:0/1851818723 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f965c099350 con 0x562707453a60 2024-03-18T20:39:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96742edb80 10 client.26428 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96742edb80 1 -- 172.21.15.46:0/1851818723 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56270722ccd0 con 0x562707453a60 2024-03-18T20:39:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96497fa700 1 -- 172.21.15.46:0/1851818723 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9650021070 con 0x562707453a60 2024-03-18T20:39:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96497fa700 1 -- 172.21.15.46:0/1851818723 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f965c0a2f10 con 0x562707453a60 2024-03-18T20:39:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96742edb80 10 client.26428 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96742edb80 10 client.26428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96742edb80 20 client.26428 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96742edb80 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5627074c33c0 0x5627074c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96742edb80 4 client.26428 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:39:15.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96742edb80 1 -- 172.21.15.46:0/1851818723 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5627071f5cf0 con 0x5627074c33c0 2024-03-18T20:39:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.695+0000 7f96656f3700 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5627074c33c0 0x5627074c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96497fa700 10 client.26428 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96656f3700 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5627074c33c0 0x5627074c57e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cd67fa7d954a044 server_cookie=270ae1035a4a3f36 in_seq=0 out_seq=0 2024-03-18T20:39:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96497fa700 1 -- 172.21.15.46:0/1851818723 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5627071f5cf0 con 0x5627074c33c0 2024-03-18T20:39:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96497fa700 10 client.26428 handle_command_reply: tid=0 2024-03-18T20:39:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96742edb80 1 client.26428 shutdown 2024-03-18T20:39:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96742edb80 1 -- 172.21.15.46:0/1851818723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 msgr2=0x562707463e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96742edb80 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x562707463e30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f965c097a20 tx=0x7f965c09a9c0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96742edb80 1 -- 172.21.15.46:0/1851818723 shutdown_connections 2024-03-18T20:39:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96742edb80 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5627074c33c0 0x5627074c57e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96742edb80 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56270745f220 0x5627074648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96742edb80 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562707453a60 0x562707463e30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96742edb80 1 --2- 172.21.15.46:0/1851818723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562707454380 0x562707464370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96497fa700 0 client.26428 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:15.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96742edb80 1 -- 172.21.15.46:0/1851818723 >> 172.21.15.46:0/1851818723 conn(0x5627073ca220 msgr2=0x56270746a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96742edb80 1 -- 172.21.15.46:0/1851818723 shutdown_connections 2024-03-18T20:39:15.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.696+0000 7f96742edb80 1 -- 172.21.15.46:0/1851818723 wait complete. 2024-03-18T20:39:15.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f96742edb80 20 client.26428 trim_cache size 0 max 16384 2024-03-18T20:39:15.708 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:39:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f9668158700 1 -- 172.21.15.46:0/1217877010 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f964c0a1310 msgr2=0x7f964c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f9668158700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f964c0a1310 0x7f964c0a37d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9660101b90 tx=0x7f9650009260 comp rx=0 tx=0).stop 2024-03-18T20:39:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f9668158700 1 -- 172.21.15.46:0/1217877010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96600ff280 msgr2=0x7f9660100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f9668158700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96600ff280 0x7f9660100460 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f965c000c00 tx=0x7f965c004730 comp rx=0 tx=0).stop 2024-03-18T20:39:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f9668158700 1 -- 172.21.15.46:0/1217877010 shutdown_connections 2024-03-18T20:39:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f9668158700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f964c0a1310 0x7f964c0a37d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f9668158700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96600ff280 0x7f9660100460 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f9668158700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9660106e90 0x7f9660100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f9668158700 1 --2- 172.21.15.46:0/1217877010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96600ffc20 0x7f96601009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f9668158700 1 -- 172.21.15.46:0/1217877010 >> 172.21.15.46:0/1217877010 conn(0x7f96600761e0 msgr2=0x7f9660103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f9668158700 1 -- 172.21.15.46:0/1217877010 shutdown_connections 2024-03-18T20:39:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:15.697+0000 7f9668158700 1 -- 172.21.15.46:0/1217877010 wait complete. 2024-03-18T20:39:16.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:16 smithi073 ceph-mon[47384]: pgmap v98: 97 pgs: 97 active+clean; 41 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 434 KiB/s wr, 49 op/s 2024-03-18T20:39:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:16 smithi155 ceph-mon[48344]: pgmap v98: 97 pgs: 97 active+clean; 41 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 434 KiB/s wr, 49 op/s 2024-03-18T20:39:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:16 smithi046 ceph-mon[43495]: pgmap v98: 97 pgs: 97 active+clean; 41 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 434 KiB/s wr, 49 op/s 2024-03-18T20:39:16.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 82426 0.03 MB/sec execute 11 sec latency 4600.836 ms 2024-03-18T20:39:16.714 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:39:16.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:39:16.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:39:16.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.862+0000 7f90807ec700 1 -- 172.21.15.46:0/374291503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078100800 msgr2=0x7f9078104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:16.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.862+0000 7f90807ec700 1 --2- 172.21.15.46:0/374291503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078100800 0x7f9078104830 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9068009990 tx=0x7f9068009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.862+0000 7f90807ec700 1 -- 172.21.15.46:0/374291503 shutdown_connections 2024-03-18T20:39:16.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.862+0000 7f90807ec700 1 --2- 172.21.15.46:0/374291503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9078104f60 0x7f907810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.862+0000 7f90807ec700 1 --2- 172.21.15.46:0/374291503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90780ffe50 0x7f9078100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.862+0000 7f90807ec700 1 --2- 172.21.15.46:0/374291503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078100800 0x7f9078104830 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.862+0000 7f90807ec700 1 -- 172.21.15.46:0/374291503 >> 172.21.15.46:0/374291503 conn(0x7f9078076d40 msgr2=0x7f9078077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.863+0000 7f90807ec700 1 -- 172.21.15.46:0/374291503 shutdown_connections 2024-03-18T20:39:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.863+0000 7f90807ec700 1 -- 172.21.15.46:0/374291503 wait complete. 2024-03-18T20:39:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.863+0000 7f90807ec700 1 Processor -- start 2024-03-18T20:39:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.863+0000 7f90807ec700 1 -- start start 2024-03-18T20:39:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.863+0000 7f90807ec700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90780ffe50 0x7f9078198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.863+0000 7f90807ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104f60 0x7f90781985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.863+0000 7f90807ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9078198ae0 0x7f90781a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.863+0000 7f90807ec700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781a37c0 con 0x7f9078104f60 2024-03-18T20:39:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.863+0000 7f90807ec700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781a3930 con 0x7f9078198ae0 2024-03-18T20:39:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.863+0000 7f90807ec700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781a3af0 con 0x7f90780ffe50 2024-03-18T20:39:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f907dd87700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104f60 0x7f90781985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f907dd87700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104f60 0x7f90781985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46308/0 (socket says 172.21.15.46:46308) 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f907dd87700 1 -- 172.21.15.46:0/398457677 learned_addr learned my addr 172.21.15.46:0/398457677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f907e588700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90780ffe50 0x7f9078198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f907ed89700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9078198ae0 0x7f90781a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f907dd87700 1 -- 172.21.15.46:0/398457677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90780ffe50 msgr2=0x7f9078198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f907dd87700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90780ffe50 0x7f9078198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f907dd87700 1 -- 172.21.15.46:0/398457677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9078198ae0 msgr2=0x7f90781a30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f907dd87700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9078198ae0 0x7f90781a30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f907dd87700 1 -- 172.21.15.46:0/398457677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90680095f0 con 0x7f9078104f60 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f907dd87700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104f60 0x7f90781985a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9068000c00 tx=0x7f9068004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f906f7fe700 1 -- 172.21.15.46:0/398457677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906801d070 con 0x7f9078104f60 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f90807ec700 1 -- 172.21.15.46:0/398457677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90781a3d70 con 0x7f9078104f60 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f906f7fe700 1 -- 172.21.15.46:0/398457677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9068021410 con 0x7f9078104f60 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f906f7fe700 1 -- 172.21.15.46:0/398457677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068017650 con 0x7f9078104f60 2024-03-18T20:39:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.864+0000 7f90807ec700 1 -- 172.21.15.46:0/398457677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90781a4190 con 0x7f9078104f60 2024-03-18T20:39:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.867+0000 7f906f7fe700 1 -- 172.21.15.46:0/398457677 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9068021580 con 0x7f9078104f60 2024-03-18T20:39:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.867+0000 7f906f7fe700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90640a1290 0x7f90640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.867+0000 7f906f7fe700 1 -- 172.21.15.46:0/398457677 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f90680cdf10 con 0x7f9078104f60 2024-03-18T20:39:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.867+0000 7f907e588700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90640a1290 0x7f90640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.868+0000 7f907e588700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90640a1290 0x7f90640a3750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9074007be0 tx=0x7f90740073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.868+0000 7f908c981b80 1 Processor -- start 2024-03-18T20:39:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.868+0000 7f908c981b80 1 -- start start 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f908c981b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 0x555f4c0c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f908c981b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c3400 0x555f4c0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f908c981b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0ce2a0 0x555f4c0d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f908c981b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4c01ca10 con 0x555f4c0c2ae0 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f908c981b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4c01a3d0 con 0x555f4c0c3400 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f908c981b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4c015cc0 con 0x555f4c0ce2a0 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f907e588700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 0x555f4c0c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f907e588700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 0x555f4c0c2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46312/0 (socket says 172.21.15.46:46312) 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f907e588700 1 -- 172.21.15.46:0/1565548224 learned_addr learned my addr 172.21.15.46:0/1565548224 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f907dd87700 1 --2- 172.21.15.46:0/1565548224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c3400 0x555f4c0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f907ed89700 1 --2- 172.21.15.46:0/1565548224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0ce2a0 0x555f4c0d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f907e588700 1 -- 172.21.15.46:0/1565548224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0ce2a0 msgr2=0x555f4c0d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f907e588700 1 --2- 172.21.15.46:0/1565548224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0ce2a0 0x555f4c0d0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f907e588700 1 -- 172.21.15.46:0/1565548224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c3400 msgr2=0x555f4c0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f907e588700 1 --2- 172.21.15.46:0/1565548224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c3400 0x555f4c0cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f907e588700 1 -- 172.21.15.46:0/1565548224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f4be97430 con 0x555f4c0c2ae0 2024-03-18T20:39:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f907e588700 1 --2- 172.21.15.46:0/1565548224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 0x555f4c0c2ec0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9074010e80 tx=0x7f9074013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9572593ab8536d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.869+0000 7f906cff9700 1 -- 172.21.15.46:0/1565548224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074010eb0 con 0x555f4c0c2ae0 2024-03-18T20:39:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.870+0000 7f906cff9700 1 -- 172.21.15.46:0/1565548224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9074003b40 con 0x555f4c0c2ae0 2024-03-18T20:39:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.870+0000 7f906cff9700 1 -- 172.21.15.46:0/1565548224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074011ba0 con 0x555f4c0c2ae0 2024-03-18T20:39:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.870+0000 7f908c981b80 1 -- 172.21.15.46:0/1565548224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 msgr2=0x555f4c0c2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.870+0000 7f908c981b80 1 --2- 172.21.15.46:0/1565548224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 0x555f4c0c2ec0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9074010e80 tx=0x7f9074013000 comp rx=0 tx=0).stop 2024-03-18T20:39:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.870+0000 7f908c981b80 1 -- 172.21.15.46:0/1565548224 shutdown_connections 2024-03-18T20:39:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.870+0000 7f908c981b80 1 --2- 172.21.15.46:0/1565548224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 0x555f4c0c2ec0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.870+0000 7f908c981b80 1 --2- 172.21.15.46:0/1565548224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0ce2a0 0x555f4c0d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.870+0000 7f908c981b80 1 --2- 172.21.15.46:0/1565548224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c3400 0x555f4c0cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.870+0000 7f908c981b80 1 -- 172.21.15.46:0/1565548224 >> 172.21.15.46:0/1565548224 conn(0x555f4c0392a0 msgr2=0x555f4c0bf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.870+0000 7f908c981b80 1 -- 172.21.15.46:0/1565548224 shutdown_connections 2024-03-18T20:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.870+0000 7f908c981b80 1 -- 172.21.15.46:0/1565548224 wait complete. 2024-03-18T20:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.871+0000 7f908c981b80 1 -- start start 2024-03-18T20:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.871+0000 7f908c981b80 1 Processor -- start 2024-03-18T20:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.871+0000 7f908c981b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 0x555f4c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.871+0000 7f908c981b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c3400 0x555f4c0cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.871+0000 7f908c981b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0ce2a0 0x555f4c0cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.871+0000 7f908c981b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4bfb1e00 con 0x555f4c0c2ae0 2024-03-18T20:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.871+0000 7f908c981b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4bfb1aa0 con 0x555f4c0c3400 2024-03-18T20:39:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.871+0000 7f908c981b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4be99720 con 0x555f4c0ce2a0 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.871+0000 7f908c981b80 10 client.0 _ll_register_callbacks cb 0x555f4be1d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.871+0000 7f907e588700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 0x555f4c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.871+0000 7f907e588700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 0x555f4c0cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46324/0 (socket says 172.21.15.46:46324) 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.871+0000 7f907e588700 1 -- 172.21.15.46:0/873884436 learned_addr learned my addr 172.21.15.46:0/873884436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f907ed89700 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0ce2a0 0x555f4c0cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f907e588700 1 -- 172.21.15.46:0/873884436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0ce2a0 msgr2=0x555f4c0cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f907e588700 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0ce2a0 0x555f4c0cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f907e588700 1 -- 172.21.15.46:0/873884436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c3400 msgr2=0x555f4c0cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f907dd87700 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c3400 0x555f4c0cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f907e588700 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c3400 0x555f4c0cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f907e588700 1 -- 172.21.15.46:0/873884436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9074010b60 con 0x555f4c0c2ae0 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f907ed89700 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0ce2a0 0x555f4c0cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:39:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f907dd87700 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c3400 0x555f4c0cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f907e588700 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 0x555f4c0cdce0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f90740079c0 tx=0x7f9074011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b6ed3300457aff server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f9061ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f9061ffb700 1 -- 172.21.15.46:0/873884436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074011880 con 0x555f4c0c2ae0 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f9061ffb700 1 -- 172.21.15.46:0/873884436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9074017070 con 0x555f4c0c2ae0 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f908c981b80 1 -- 172.21.15.46:0/873884436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f4c0b6b50 con 0x555f4c0c2ae0 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f9061ffb700 1 -- 172.21.15.46:0/873884436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90740119e0 con 0x555f4c0c2ae0 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f908c981b80 10 client.16692 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f908c981b80 1 -- 172.21.15.46:0/873884436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555f4be43be0 con 0x555f4c0c2ae0 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.872+0000 7f9061ffb700 1 -- 172.21.15.46:0/873884436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f4c0b6b50 con 0x555f4c0c2ae0 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.873+0000 7f9061ffb700 1 -- 172.21.15.46:0/873884436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f90740044c0 con 0x555f4c0c2ae0 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.873+0000 7f908c981b80 10 client.16692 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.873+0000 7f908c981b80 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.873+0000 7f908c981b80 20 client.16692 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.873+0000 7f908c981b80 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555f4c0ddea0 0x555f4c0e02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.873+0000 7f908c981b80 4 client.16692 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.873+0000 7f908c981b80 1 -- 172.21.15.46:0/873884436 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f4c00e780 con 0x555f4c0ddea0 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.873+0000 7f907dd87700 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555f4c0ddea0 0x555f4c0e02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.873+0000 7f907dd87700 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555f4c0ddea0 0x555f4c0e02c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1f422a15e59fbb server_cookie=5ec39b6afd1415bb in_seq=0 out_seq=0 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.873+0000 7f9061ffb700 10 client.16692 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f9061ffb700 1 -- 172.21.15.46:0/873884436 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555f4c00e780 con 0x555f4c0ddea0 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f9061ffb700 10 client.16692 handle_command_reply: tid=0 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f908c981b80 1 client.16692 shutdown 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f908c981b80 1 -- 172.21.15.46:0/873884436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 msgr2=0x555f4c0cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f908c981b80 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 0x555f4c0cdce0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f90740079c0 tx=0x7f9074011440 comp rx=0 tx=0).stop 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f908c981b80 1 -- 172.21.15.46:0/873884436 shutdown_connections 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f908c981b80 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555f4c0ddea0 0x555f4c0e02c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f908c981b80 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c2ae0 0x555f4c0cdce0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f908c981b80 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0ce2a0 0x555f4c0cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f9061ffb700 0 client.16692 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f908c981b80 1 --2- 172.21.15.46:0/873884436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c3400 0x555f4c0cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f908c981b80 1 -- 172.21.15.46:0/873884436 >> 172.21.15.46:0/873884436 conn(0x555f4c0392a0 msgr2=0x555f4c0bec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f908c981b80 1 -- 172.21.15.46:0/873884436 shutdown_connections 2024-03-18T20:39:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f908c981b80 1 -- 172.21.15.46:0/873884436 wait complete. 2024-03-18T20:39:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.874+0000 7f908c981b80 20 client.16692 trim_cache size 0 max 16384 2024-03-18T20:39:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.892+0000 7f908c981b80 1 Processor -- start 2024-03-18T20:39:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.892+0000 7f908c981b80 1 -- start start 2024-03-18T20:39:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.892+0000 7f908c981b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c2ae0 0x555f4c0d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.892+0000 7f908c981b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0c3400 0x555f4c0bebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.892+0000 7f908c981b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0ce2a0 0x555f4c0bf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.892+0000 7f908c981b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4c0cbab0 con 0x555f4c0ce2a0 2024-03-18T20:39:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.892+0000 7f908c981b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4c0bde50 con 0x555f4c0c2ae0 2024-03-18T20:39:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.892+0000 7f908c981b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4c0bb660 con 0x555f4c0c3400 2024-03-18T20:39:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f907ed89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0ce2a0 0x555f4c0bf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f907ed89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0ce2a0 0x555f4c0bf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46340/0 (socket says 172.21.15.46:46340) 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f907ed89700 1 -- 172.21.15.46:0/673868029 learned_addr learned my addr 172.21.15.46:0/673868029 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f907dd87700 1 --2- 172.21.15.46:0/673868029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0c3400 0x555f4c0bebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f907e588700 1 --2- 172.21.15.46:0/673868029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c2ae0 0x555f4c0d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f907ed89700 1 -- 172.21.15.46:0/673868029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0c3400 msgr2=0x555f4c0bebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f907ed89700 1 --2- 172.21.15.46:0/673868029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0c3400 0x555f4c0bebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f907ed89700 1 -- 172.21.15.46:0/673868029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c2ae0 msgr2=0x555f4c0d4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f907ed89700 1 --2- 172.21.15.46:0/673868029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c2ae0 0x555f4c0d4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f907ed89700 1 -- 172.21.15.46:0/673868029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f4bfdbf20 con 0x555f4c0ce2a0 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f907e588700 1 --2- 172.21.15.46:0/673868029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c2ae0 0x555f4c0d4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f907ed89700 1 --2- 172.21.15.46:0/673868029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0ce2a0 0x555f4c0bf110 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9070007500 tx=0x7f907000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244b0685620699f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f906d7fa700 1 -- 172.21.15.46:0/673868029 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907000e040 con 0x555f4c0ce2a0 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f906d7fa700 1 -- 172.21.15.46:0/673868029 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f907000f040 con 0x555f4c0ce2a0 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.893+0000 7f906d7fa700 1 -- 172.21.15.46:0/673868029 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90700169d0 con 0x555f4c0ce2a0 2024-03-18T20:39:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.894+0000 7f908c981b80 1 -- 172.21.15.46:0/673868029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0ce2a0 msgr2=0x555f4c0bf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.894+0000 7f908c981b80 1 --2- 172.21.15.46:0/673868029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0ce2a0 0x555f4c0bf110 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9070007500 tx=0x7f907000b040 comp rx=0 tx=0).stop 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.894+0000 7f908c981b80 1 -- 172.21.15.46:0/673868029 shutdown_connections 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.894+0000 7f908c981b80 1 --2- 172.21.15.46:0/673868029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0ce2a0 0x555f4c0bf110 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.894+0000 7f908c981b80 1 --2- 172.21.15.46:0/673868029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0c2ae0 0x555f4c0d4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.894+0000 7f908c981b80 1 --2- 172.21.15.46:0/673868029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0c3400 0x555f4c0bebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.894+0000 7f908c981b80 1 -- 172.21.15.46:0/673868029 >> 172.21.15.46:0/673868029 conn(0x555f4c0392a0 msgr2=0x555f4c0cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.894+0000 7f908c981b80 1 -- 172.21.15.46:0/673868029 shutdown_connections 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.894+0000 7f908c981b80 1 -- 172.21.15.46:0/673868029 wait complete. 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.894+0000 7f908c981b80 1 -- start start 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.894+0000 7f908c981b80 1 Processor -- start 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f908c981b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0c2ae0 0x555f4c0d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f908c981b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c3400 0x555f4c0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f908c981b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0ce2a0 0x555f4c131ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f908c981b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4c1321e0 con 0x555f4c0c3400 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f908c981b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4c132320 con 0x555f4c0ce2a0 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f908c981b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f4c132460 con 0x555f4c0c2ae0 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f908c981b80 10 client.0 _ll_register_callbacks cb 0x7f9068021b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f907dd87700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c3400 0x555f4c0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f907dd87700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c3400 0x555f4c0d4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46350/0 (socket says 172.21.15.46:46350) 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f907dd87700 1 -- 172.21.15.46:0/1317317523 learned_addr learned my addr 172.21.15.46:0/1317317523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f907ed89700 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0ce2a0 0x555f4c131ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f907e588700 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0c2ae0 0x555f4c0d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f907dd87700 1 -- 172.21.15.46:0/1317317523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0c2ae0 msgr2=0x555f4c0d3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f907dd87700 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0c2ae0 0x555f4c0d3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f907dd87700 1 -- 172.21.15.46:0/1317317523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0ce2a0 msgr2=0x555f4c131ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f907dd87700 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0ce2a0 0x555f4c131ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f907dd87700 1 -- 172.21.15.46:0/1317317523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9070009240 con 0x555f4c0c3400 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f907dd87700 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c3400 0x555f4c0d4380 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f90680975c0 tx=0x7f906809aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c28501204b4164 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f9061ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f9061ffb700 1 -- 172.21.15.46:0/1317317523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906809b0c0 con 0x555f4c0c3400 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f9061ffb700 1 -- 172.21.15.46:0/1317317523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f906809eef0 con 0x555f4c0c3400 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.895+0000 7f908c981b80 1 -- 172.21.15.46:0/1317317523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9074003b40 con 0x555f4c0c3400 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.896+0000 7f9061ffb700 1 -- 172.21.15.46:0/1317317523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90680a34b0 con 0x555f4c0c3400 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.896+0000 7f908c981b80 10 client.16698 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.896+0000 7f908c981b80 1 -- 172.21.15.46:0/1317317523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555f4be9bd50 con 0x555f4c0c3400 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.896+0000 7f9061ffb700 1 -- 172.21.15.46:0/1317317523 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9074003b40 con 0x555f4c0c3400 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.896+0000 7f9061ffb700 1 -- 172.21.15.46:0/1317317523 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f90680b3be0 con 0x555f4c0c3400 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.896+0000 7f908c981b80 10 client.16698 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.896+0000 7f908c981b80 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.896+0000 7f908c981b80 20 client.16698 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.896+0000 7f908c981b80 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555f4c133950 0x555f4c135d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:16.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.896+0000 7f908c981b80 4 client.16698 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:39:16.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.896+0000 7f908c981b80 1 -- 172.21.15.46:0/1317317523 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555f4be64d70 con 0x555f4c133950 2024-03-18T20:39:16.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.896+0000 7f907e588700 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555f4c133950 0x555f4c135d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:16.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.897+0000 7f9061ffb700 10 client.16698 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:16.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.897+0000 7f907e588700 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555f4c133950 0x555f4c135d70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39c2a2314b42f55b server_cookie=42c1ab94e6c9e17f in_seq=0 out_seq=0 2024-03-18T20:39:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f9061ffb700 1 -- 172.21.15.46:0/1317317523 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555f4be64d70 con 0x555f4c133950 2024-03-18T20:39:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f9061ffb700 10 client.16698 handle_command_reply: tid=0 2024-03-18T20:39:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f908c981b80 1 client.16698 shutdown 2024-03-18T20:39:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f908c981b80 1 -- 172.21.15.46:0/1317317523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c3400 msgr2=0x555f4c0d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f908c981b80 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c3400 0x555f4c0d4380 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f90680975c0 tx=0x7f906809aa20 comp rx=0 tx=0).stop 2024-03-18T20:39:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f908c981b80 1 -- 172.21.15.46:0/1317317523 shutdown_connections 2024-03-18T20:39:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f908c981b80 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555f4c133950 0x555f4c135d70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f908c981b80 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555f4c0ce2a0 0x555f4c131ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f908c981b80 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555f4c0c2ae0 0x555f4c0d3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f908c981b80 1 --2- 172.21.15.46:0/1317317523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f4c0c3400 0x555f4c0d4380 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f908c981b80 1 -- 172.21.15.46:0/1317317523 >> 172.21.15.46:0/1317317523 conn(0x555f4c0392a0 msgr2=0x555f4c0c6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:16.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f9061ffb700 0 client.16698 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:16.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f908c981b80 1 -- 172.21.15.46:0/1317317523 shutdown_connections 2024-03-18T20:39:16.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f908c981b80 1 -- 172.21.15.46:0/1317317523 wait complete. 2024-03-18T20:39:16.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.898+0000 7f908c981b80 20 client.16698 trim_cache size 0 max 16384 2024-03-18T20:39:16.907 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:39:16.907 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:39:16.908 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "a1495486-0520-4750-a9e5-f540953b768b", 2024-03-18T20:39:16.908 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:39:16.908 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:39:16.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.899+0000 7f90807ec700 1 -- 172.21.15.46:0/398457677 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90640a1290 msgr2=0x7f90640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:16.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.899+0000 7f90807ec700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90640a1290 0x7f90640a3750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9074007be0 tx=0x7f90740073d0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.899+0000 7f90807ec700 1 -- 172.21.15.46:0/398457677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104f60 msgr2=0x7f90781985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.899+0000 7f90807ec700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104f60 0x7f90781985a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9068000c00 tx=0x7f9068004730 comp rx=0 tx=0).stop 2024-03-18T20:39:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.899+0000 7f90807ec700 1 -- 172.21.15.46:0/398457677 shutdown_connections 2024-03-18T20:39:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.899+0000 7f90807ec700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90640a1290 0x7f90640a3750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.899+0000 7f90807ec700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9078198ae0 0x7f90781a30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.899+0000 7f90807ec700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90780ffe50 0x7f9078198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.899+0000 7f90807ec700 1 --2- 172.21.15.46:0/398457677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9078104f60 0x7f90781985a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.899+0000 7f90807ec700 1 -- 172.21.15.46:0/398457677 >> 172.21.15.46:0/398457677 conn(0x7f9078076d40 msgr2=0x7f90780fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.899+0000 7f90807ec700 1 -- 172.21.15.46:0/398457677 shutdown_connections 2024-03-18T20:39:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:16.899+0000 7f90807ec700 1 -- 172.21.15.46:0/398457677 wait complete. 2024-03-18T20:39:16.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:39:16.919 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:39:17.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.071+0000 7fbd8e570700 1 -- 172.21.15.46:0/3624046713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd88067e30 msgr2=0x7fbd880682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:17.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.071+0000 7fbd8e570700 1 --2- 172.21.15.46:0/3624046713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd88067e30 0x7fbd880682b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009990 tx=0x7fbd7c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.071+0000 7fbd8e570700 1 -- 172.21.15.46:0/3624046713 shutdown_connections 2024-03-18T20:39:17.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.071+0000 7fbd8e570700 1 --2- 172.21.15.46:0/3624046713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd880687f0 0x7fbd8810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.071+0000 7fbd8e570700 1 --2- 172.21.15.46:0/3624046713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd88105ce0 0x7fbd881060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.071+0000 7fbd8e570700 1 --2- 172.21.15.46:0/3624046713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd88067e30 0x7fbd880682b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.071+0000 7fbd8e570700 1 -- 172.21.15.46:0/3624046713 >> 172.21.15.46:0/3624046713 conn(0x7fbd88076d40 msgr2=0x7fbd88077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.071+0000 7fbd8e570700 1 -- 172.21.15.46:0/3624046713 shutdown_connections 2024-03-18T20:39:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.071+0000 7fbd8e570700 1 -- 172.21.15.46:0/3624046713 wait complete. 2024-03-18T20:39:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.072+0000 7fbd8e570700 1 Processor -- start 2024-03-18T20:39:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.072+0000 7fbd8e570700 1 -- start start 2024-03-18T20:39:17.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.072+0000 7fbd8e570700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd88067e30 0x7fbd880ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.072+0000 7fbd8e570700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd880687f0 0x7fbd881001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.072+0000 7fbd8e570700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd88105ce0 0x7fbd881006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.072+0000 7fbd8e570700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd881a5550 con 0x7fbd880687f0 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.072+0000 7fbd8e570700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd881a56c0 con 0x7fbd88105ce0 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd8e570700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd881a5880 con 0x7fbd88067e30 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd880687f0 0x7fbd881001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd880687f0 0x7fbd881001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46358/0 (socket says 172.21.15.46:46358) 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd8cb0d700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd88105ce0 0x7fbd881006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd877fe700 1 -- 172.21.15.46:0/2256545119 learned_addr learned my addr 172.21.15.46:0/2256545119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd87fff700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd88067e30 0x7fbd880ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd877fe700 1 -- 172.21.15.46:0/2256545119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd88067e30 msgr2=0x7fbd880ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd877fe700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd88067e30 0x7fbd880ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd877fe700 1 -- 172.21.15.46:0/2256545119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd88105ce0 msgr2=0x7fbd881006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd877fe700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd88105ce0 0x7fbd881006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd877fe700 1 -- 172.21.15.46:0/2256545119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0095f0 con 0x7fbd880687f0 2024-03-18T20:39:17.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd8cb0d700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd88105ce0 0x7fbd881006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd87fff700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd88067e30 0x7fbd880ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd877fe700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd880687f0 0x7fbd881001b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005770 tx=0x7fbd7c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.073+0000 7fbd857fa700 1 -- 172.21.15.46:0/2256545119 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c01d070 con 0x7fbd880687f0 2024-03-18T20:39:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.074+0000 7fbd857fa700 1 -- 172.21.15.46:0/2256545119 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd7c021410 con 0x7fbd880687f0 2024-03-18T20:39:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.074+0000 7fbd857fa700 1 -- 172.21.15.46:0/2256545119 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c017650 con 0x7fbd880687f0 2024-03-18T20:39:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.074+0000 7fbd8e570700 1 -- 172.21.15.46:0/2256545119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd881a5b00 con 0x7fbd880687f0 2024-03-18T20:39:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.075+0000 7fbd8e570700 1 -- 172.21.15.46:0/2256545119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd881a5f30 con 0x7fbd880687f0 2024-03-18T20:39:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.077+0000 7fbd857fa700 1 -- 172.21.15.46:0/2256545119 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fbd7c0217f0 con 0x7fbd880687f0 2024-03-18T20:39:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.078+0000 7fbd857fa700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbd700a1310 0x7fbd700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.078+0000 7fbd87fff700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbd700a1310 0x7fbd700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.078+0000 7fbd857fa700 1 -- 172.21.15.46:0/2256545119 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fbd7c0cdae0 con 0x7fbd880687f0 2024-03-18T20:39:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.078+0000 7fbd9a705b80 1 Processor -- start 2024-03-18T20:39:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.078+0000 7fbd87fff700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbd700a1310 0x7fbd700a37d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbd78007fd0 tx=0x7fbd780073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.078+0000 7fbd9a705b80 1 -- start start 2024-03-18T20:39:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.078+0000 7fbd9a705b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 0x55569317de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd9a705b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 0x555693188ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd9a705b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x55569318b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd9a705b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556930d7990 con 0x555693189220 2024-03-18T20:39:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd9a705b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556930d5350 con 0x55569317da60 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd9a705b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556930d0c40 con 0x55569317e380 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd8cb0d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x55569318b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd87fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 0x55569317de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd8cb0d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x55569318b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46374/0 (socket says 172.21.15.46:46374) 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd8cb0d700 1 -- 172.21.15.46:0/2375408245 learned_addr learned my addr 172.21.15.46:0/2375408245 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd877fe700 1 --2- 172.21.15.46:0/2375408245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 0x555693188ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd8cb0d700 1 -- 172.21.15.46:0/2375408245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 msgr2=0x555693188ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd8cb0d700 1 --2- 172.21.15.46:0/2375408245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 0x555693188ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd8cb0d700 1 -- 172.21.15.46:0/2375408245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 msgr2=0x55569317de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd8cb0d700 1 --2- 172.21.15.46:0/2375408245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 0x55569317de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd8cb0d700 1 -- 172.21.15.46:0/2375408245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555692f523b0 con 0x555693189220 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd87fff700 1 --2- 172.21.15.46:0/2375408245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 0x55569317de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd8cb0d700 1 --2- 172.21.15.46:0/2375408245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x55569318b610 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbd80003eb0 tx=0x7fbd80009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d29411c6a45a37 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd6e7fc700 1 -- 172.21.15.46:0/2375408245 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd800092d0 con 0x555693189220 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd6e7fc700 1 -- 172.21.15.46:0/2375408245 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd8000f040 con 0x555693189220 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.079+0000 7fbd6e7fc700 1 -- 172.21.15.46:0/2375408245 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000e850 con 0x555693189220 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.080+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2375408245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 msgr2=0x55569318b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.080+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2375408245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x55569318b610 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbd80003eb0 tx=0x7fbd80009920 comp rx=0 tx=0).stop 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.080+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2375408245 shutdown_connections 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.080+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2375408245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x55569318b610 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.080+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2375408245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 0x55569317de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.080+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2375408245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 0x555693188ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.080+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2375408245 >> 172.21.15.46:0/2375408245 conn(0x5556930f4220 msgr2=0x55569317a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.080+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2375408245 shutdown_connections 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.080+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2375408245 wait complete. 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd9a705b80 1 -- start start 2024-03-18T20:39:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd9a705b80 1 Processor -- start 2024-03-18T20:39:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd9a705b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317da60 0x555693188c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd9a705b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317e380 0x555693185f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd9a705b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x555693186590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd9a705b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569306cd80 con 0x555693189220 2024-03-18T20:39:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd9a705b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569306ca20 con 0x55569317e380 2024-03-18T20:39:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd9a705b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555692f546a0 con 0x55569317da60 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd9a705b80 10 client.0 _ll_register_callbacks cb 0x555692ed83a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd8cb0d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x555693186590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd8cb0d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x555693186590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46388/0 (socket says 172.21.15.46:46388) 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd8cb0d700 1 -- 172.21.15.46:0/4238882741 learned_addr learned my addr 172.21.15.46:0/4238882741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd87fff700 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317da60 0x555693188c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.081+0000 7fbd877fe700 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317e380 0x555693185f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd8cb0d700 1 -- 172.21.15.46:0/4238882741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317da60 msgr2=0x555693188c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd8cb0d700 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317da60 0x555693188c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd8cb0d700 1 -- 172.21.15.46:0/4238882741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317e380 msgr2=0x555693185f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd8cb0d700 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317e380 0x555693185f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd8cb0d700 1 -- 172.21.15.46:0/4238882741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd78010e90 con 0x555693189220 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd87fff700 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317da60 0x555693188c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd877fe700 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317e380 0x555693185f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd8cb0d700 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x555693186590 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbd80000f80 tx=0x7fbd80004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9999d4781e67cdc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd677fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd677fe700 1 -- 172.21.15.46:0/4238882741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80004ea0 con 0x555693189220 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd677fe700 1 -- 172.21.15.46:0/4238882741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fbd80007910 con 0x555693189220 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd9a705b80 1 -- 172.21.15.46:0/4238882741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555693171ad0 con 0x555693189220 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd677fe700 1 -- 172.21.15.46:0/4238882741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd80007cc0 con 0x555693189220 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd9a705b80 10 client.16710 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.082+0000 7fbd9a705b80 1 -- 172.21.15.46:0/4238882741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fbd80003b90 con 0x555693189220 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.083+0000 7fbd677fe700 1 -- 172.21.15.46:0/4238882741 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555693171ad0 con 0x555693189220 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.083+0000 7fbd677fe700 1 -- 172.21.15.46:0/4238882741 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbd8002f070 con 0x555693189220 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.083+0000 7fbd9a705b80 10 client.16710 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.083+0000 7fbd9a705b80 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.083+0000 7fbd9a705b80 20 client.16710 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.083+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555693198d70 0x55569319b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.083+0000 7fbd9a705b80 4 client.16710 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.083+0000 7fbd9a705b80 1 -- 172.21.15.46:0/4238882741 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556930c9700 con 0x555693198d70 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.083+0000 7fbd87fff700 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555693198d70 0x55569319b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd677fe700 10 client.16710 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd87fff700 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555693198d70 0x55569319b190 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6ca3d3153c9d417 server_cookie=807a1c481fc75424 in_seq=0 out_seq=0 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd677fe700 1 -- 172.21.15.46:0/4238882741 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5556930c9700 con 0x555693198d70 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd677fe700 10 client.16710 handle_command_reply: tid=0 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd9a705b80 1 client.16710 shutdown 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd9a705b80 1 -- 172.21.15.46:0/4238882741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 msgr2=0x555693186590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x555693186590 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbd80000f80 tx=0x7fbd80004d00 comp rx=0 tx=0).stop 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd9a705b80 1 -- 172.21.15.46:0/4238882741 shutdown_connections 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555693198d70 0x55569319b190 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x555693186590 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd677fe700 0 client.16710 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317da60 0x555693188c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/4238882741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317e380 0x555693185f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd9a705b80 1 -- 172.21.15.46:0/4238882741 >> 172.21.15.46:0/4238882741 conn(0x5556930f4220 msgr2=0x555693179c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.084+0000 7fbd9a705b80 1 -- 172.21.15.46:0/4238882741 shutdown_connections 2024-03-18T20:39:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.085+0000 7fbd9a705b80 1 -- 172.21.15.46:0/4238882741 wait complete. 2024-03-18T20:39:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.085+0000 7fbd9a705b80 20 client.16710 trim_cache size 0 max 16384 2024-03-18T20:39:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.103+0000 7fbd9a705b80 1 Processor -- start 2024-03-18T20:39:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd9a705b80 1 -- start start 2024-03-18T20:39:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd9a705b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55569317da60 0x555693186460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd9a705b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 0x55569318fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd9a705b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555693189220 0x5556931945a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd9a705b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555693174250 con 0x55569317da60 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd9a705b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555693195960 con 0x555693189220 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd9a705b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569317f3d0 con 0x55569317e380 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd87fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55569317da60 0x555693186460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd8cb0d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555693189220 0x5556931945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd87fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55569317da60 0x555693186460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46400/0 (socket says 172.21.15.46:46400) 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd877fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 0x55569318fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd87fff700 1 -- 172.21.15.46:0/2658261127 learned_addr learned my addr 172.21.15.46:0/2658261127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd8cb0d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555693189220 0x5556931945a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:46590/0 (socket says 172.21.15.46:46590) 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd87fff700 1 -- 172.21.15.46:0/2658261127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 msgr2=0x55569318fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd87fff700 1 --2- 172.21.15.46:0/2658261127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 0x55569318fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd87fff700 1 -- 172.21.15.46:0/2658261127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555693189220 msgr2=0x5556931945a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd87fff700 1 --2- 172.21.15.46:0/2658261127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555693189220 0x5556931945a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd87fff700 1 -- 172.21.15.46:0/2658261127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555692efeb60 con 0x55569317da60 2024-03-18T20:39:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.104+0000 7fbd8cb0d700 1 --2- 172.21.15.46:0/2658261127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555693189220 0x5556931945a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.105+0000 7fbd87fff700 1 --2- 172.21.15.46:0/2658261127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55569317da60 0x555693186460 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x55569319ba70 tx=0x7fbd78016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8c25c2f7d92615 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.105+0000 7fbd6effd700 1 -- 172.21.15.46:0/2658261127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78026040 con 0x55569317da60 2024-03-18T20:39:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.105+0000 7fbd6effd700 1 -- 172.21.15.46:0/2658261127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd78003e20 con 0x55569317da60 2024-03-18T20:39:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.105+0000 7fbd6effd700 1 -- 172.21.15.46:0/2658261127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd780116a0 con 0x55569317da60 2024-03-18T20:39:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.106+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2658261127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55569317da60 msgr2=0x555693186460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.106+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2658261127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55569317da60 0x555693186460 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x55569319ba70 tx=0x7fbd78016000 comp rx=0 tx=0).stop 2024-03-18T20:39:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.106+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2658261127 shutdown_connections 2024-03-18T20:39:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.106+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2658261127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55569317da60 0x555693186460 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.106+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2658261127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555693189220 0x5556931945a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.106+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2658261127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 0x55569318fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.106+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2658261127 >> 172.21.15.46:0/2658261127 conn(0x5556930f4220 msgr2=0x5556931847e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.106+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2658261127 shutdown_connections 2024-03-18T20:39:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.106+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2658261127 wait complete. 2024-03-18T20:39:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.106+0000 7fbd9a705b80 1 -- start start 2024-03-18T20:39:17.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.106+0000 7fbd9a705b80 1 Processor -- start 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd9a705b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 0x5556931e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd9a705b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 0x5556931e63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd9a705b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x55569318be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd9a705b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569318c3f0 con 0x555693189220 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd9a705b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569318c560 con 0x55569317da60 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd9a705b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569318c720 con 0x55569317e380 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd9a705b80 10 client.0 _ll_register_callbacks cb 0x7fbd78010d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd87fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 0x5556931e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd8cb0d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x55569318be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd87fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 0x5556931e5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:46602/0 (socket says 172.21.15.46:46602) 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd87fff700 1 -- 172.21.15.46:0/2219872309 learned_addr learned my addr 172.21.15.46:0/2219872309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd877fe700 1 --2- 172.21.15.46:0/2219872309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 0x5556931e63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd87fff700 1 -- 172.21.15.46:0/2219872309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 msgr2=0x5556931e63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd87fff700 1 --2- 172.21.15.46:0/2219872309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 0x5556931e63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd87fff700 1 -- 172.21.15.46:0/2219872309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 msgr2=0x55569318be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd87fff700 1 --2- 172.21.15.46:0/2219872309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x55569318be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd87fff700 1 -- 172.21.15.46:0/2219872309 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd80003b90 con 0x55569317da60 2024-03-18T20:39:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd87fff700 1 --2- 172.21.15.46:0/2219872309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 0x5556931e5e70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbd780284a0 tx=0x7fbd780282c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=120251c5e484495c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.107+0000 7fbd677fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:39:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd677fe700 1 -- 172.21.15.46:0/2219872309 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78026040 con 0x55569317da60 2024-03-18T20:39:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd677fe700 1 -- 172.21.15.46:0/2219872309 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fbd78017070 con 0x55569317da60 2024-03-18T20:39:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2219872309 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd8000f040 con 0x55569317da60 2024-03-18T20:39:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd677fe700 1 -- 172.21.15.46:0/2219872309 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd780287b0 con 0x55569317da60 2024-03-18T20:39:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd9a705b80 10 client.26465 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2219872309 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fbd78003680 con 0x55569317da60 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd677fe700 1 -- 172.21.15.46:0/2219872309 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd8000f040 con 0x55569317da60 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd677fe700 1 -- 172.21.15.46:0/2219872309 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbd78011aa0 con 0x55569317da60 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd9a705b80 10 client.26465 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd9a705b80 10 client.26465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd9a705b80 20 client.26465 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2219872309 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5556931edd40 0x5556931f0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd9a705b80 4 client.26465 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.108+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2219872309 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555692f1fcf0 con 0x5556931edd40 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd877fe700 1 --2- 172.21.15.46:0/2219872309 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5556931edd40 0x5556931f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd677fe700 10 client.26465 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd877fe700 1 --2- 172.21.15.46:0/2219872309 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5556931edd40 0x5556931f0160 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=449dbffd8ed984c2 server_cookie=5d9127485310db08 in_seq=0 out_seq=0 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd677fe700 1 -- 172.21.15.46:0/2219872309 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555692f1fcf0 con 0x5556931edd40 2024-03-18T20:39:17.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd677fe700 10 client.26465 handle_command_reply: tid=0 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd9a705b80 1 client.26465 shutdown 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2219872309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 msgr2=0x5556931e5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2219872309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 0x5556931e5e70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbd780284a0 tx=0x7fbd780282c0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2219872309 shutdown_connections 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2219872309 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5556931edd40 0x5556931f0160 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2219872309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555693189220 0x55569318be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2219872309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55569317da60 0x5556931e5e70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd9a705b80 1 --2- 172.21.15.46:0/2219872309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55569317e380 0x5556931e63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2219872309 >> 172.21.15.46:0/2219872309 conn(0x5556930f4220 msgr2=0x5556931877b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd677fe700 0 client.26465 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2219872309 shutdown_connections 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.109+0000 7fbd9a705b80 1 -- 172.21.15.46:0/2219872309 wait complete. 2024-03-18T20:39:17.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd9a705b80 20 client.26465 trim_cache size 0 max 16384 2024-03-18T20:39:17.119 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:39:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:39:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:39:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "a1495486-0520-4750-a9e5-f540953b768b": { 2024-03-18T20:39:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:39:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "a1495486-0520-4750-a9e5-f540953b768b", 2024-03-18T20:39:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:39:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:39:17.119 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:39:17.119 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:39:17.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd8e570700 1 -- 172.21.15.46:0/2256545119 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbd700a1310 msgr2=0x7fbd700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:17.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd8e570700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbd700a1310 0x7fbd700a37d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbd78007fd0 tx=0x7fbd780073d0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd8e570700 1 -- 172.21.15.46:0/2256545119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd880687f0 msgr2=0x7fbd881001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:17.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd8e570700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd880687f0 0x7fbd881001b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005770 tx=0x7fbd7c0057a0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd8e570700 1 -- 172.21.15.46:0/2256545119 shutdown_connections 2024-03-18T20:39:17.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd8e570700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbd700a1310 0x7fbd700a37d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd8e570700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd88105ce0 0x7fbd881006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd8e570700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd88067e30 0x7fbd880ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd8e570700 1 --2- 172.21.15.46:0/2256545119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd880687f0 0x7fbd881001b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:17.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd8e570700 1 -- 172.21.15.46:0/2256545119 >> 172.21.15.46:0/2256545119 conn(0x7fbd88076d40 msgr2=0x7fbd880fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:17.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd8e570700 1 -- 172.21.15.46:0/2256545119 shutdown_connections 2024-03-18T20:39:17.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:17.110+0000 7fbd8e570700 1 -- 172.21.15.46:0/2256545119 wait complete. 2024-03-18T20:39:17.128 INFO:tasks.cephfs.filesystem:scrub status for tag:a1495486-0520-4750-a9e5-f540953b768b - {'path': '/', 'tag': 'a1495486-0520-4750-a9e5-f540953b768b', 'options': 'recursive,force'} 2024-03-18T20:39:17.220 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 20:39:16 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: 2024-03-18T20:39:16.907+0000 7f60b88e0700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [...5,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/EXCEL/ auth{3=1} v12852 ap=2 f(v254 m2024-03-18T20:39:03.978225+0000 2=2+0) n(v10 rc2024-03-18T20:39:03.978225+0000 b520192 3=2+1) old_inodes=2 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25108=pAsLsXsFs/pAsLsXsFsxcral@2144} mcw={3=AsLsXsFs} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55b6c361f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020c:[<0x10000000009/EXCEL v5933>,<0x10000000008/~dmtmp v53>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v42>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v52>,<0x1/volumes v48>]//[]","memoryvalue":"(2)0x1000000020c:[<0x10000000009/EXCEL v12852>,<0x10000000008/~dmtmp v3463>,<0x10000000007/client0 v225>,<0x10000000006/clients v210>,<0x10000000005/tmp v208>,<0x100000001fb/client.0 v209>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v224>,<0x10000000001/sv_1 v244>,<0x10000000000/qa v225>,<0x1/volumes v218>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-18T20:39:03.002236+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-03-18T20:39:03.978225+0000 b520192 3=2+1)","memory_value.dirstat":"f(v254 m2024-03-18T20:39:03.978225+0000 2=2+0)","memory_value.rstat":"n(v10 rc2024-03-18T20:39:03.978225+0000 b520192 3=2+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-18T20:39:17.220 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 20:39:16 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: 2024-03-18T20:39:16.911+0000 7f60b88e0700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...5,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{3=1} v12850 ap=2 f(v177 m2024-03-18T20:39:03.957226+0000 3=3+0) n(v10 rc2024-03-18T20:39:03.957226+0000 b780288 4=3+1) old_inodes=2 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25108=pAsLsXsFs/pAsLsXsFsxcral@1422} mcw={3=AsLsXsFs} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55b6c37d0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v5773>,<0x10000000008/~dmtmp v53>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v42>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v52>,<0x1/volumes v48>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v12850>,<0x10000000008/~dmtmp v3463>,<0x10000000007/client0 v225>,<0x10000000006/clients v210>,<0x10000000005/tmp v208>,<0x100000001fb/client.0 v209>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v224>,<0x10000000001/sv_1 v244>,<0x10000000000/qa v225>,<0x1/volumes v218>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-18T20:39:02.588240+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-03-18T20:39:03.957226+0000 b780288 4=3+1)","memory_value.dirstat":"f(v177 m2024-03-18T20:39:03.957226+0000 3=3+0)","memory_value.rstat":"n(v10 rc2024-03-18T20:39:03.957226+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-18T20:39:17.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:17 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:39:17.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:17 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:17.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:17 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:17.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:17 smithi073 ceph-mon[47384]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-03-18T20:39:17.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:17 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:17.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:17 smithi073 ceph-mon[47384]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-03-18T20:39:17.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:17 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:17.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:17 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:17.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:17 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:39:17.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:17 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:17.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:17 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:17.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:17 smithi155 ceph-mon[48344]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-03-18T20:39:17.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:17 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:17.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:17 smithi155 ceph-mon[48344]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-03-18T20:39:17.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:17 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:17.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:17 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:17.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:17 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:39:17.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:17 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:17.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:17 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:17.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:17 smithi046 ceph-mon[43495]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-03-18T20:39:17.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:17 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:17.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:17 smithi046 ceph-mon[43495]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-03-18T20:39:17.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:17 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:17.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:17 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:17.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 84403 0.91 MB/sec execute 12 sec latency 4850.447 ms 2024-03-18T20:39:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:18.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:18.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:18.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:18 smithi073 ceph-mon[47384]: pgmap v99: 97 pgs: 97 active+clean; 41 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 434 KiB/s wr, 49 op/s 2024-03-18T20:39:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:18.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:18.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:18.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:18.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:18.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:18 smithi155 ceph-mon[48344]: pgmap v99: 97 pgs: 97 active+clean; 41 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 434 KiB/s wr, 49 op/s 2024-03-18T20:39:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:18.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:18.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:18.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:18.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:18 smithi046 ceph-mon[43495]: pgmap v99: 97 pgs: 97 active+clean; 41 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 434 KiB/s wr, 49 op/s 2024-03-18T20:39:18.678 INFO:tasks.workunit.client.0.smithi046.stdout: 1 85899 1.48 MB/sec execute 13 sec latency 10.828 ms 2024-03-18T20:39:19.679 INFO:tasks.workunit.client.0.smithi046.stdout: 1 85899 1.38 MB/sec execute 14 sec latency 1004.799 ms 2024-03-18T20:39:20.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:20 smithi073 ceph-mon[47384]: pgmap v100: 97 pgs: 97 active+clean; 43 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 925 KiB/s wr, 106 op/s 2024-03-18T20:39:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:20 smithi155 ceph-mon[48344]: pgmap v100: 97 pgs: 97 active+clean; 43 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 925 KiB/s wr, 106 op/s 2024-03-18T20:39:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:20 smithi046 ceph-mon[43495]: pgmap v100: 97 pgs: 97 active+clean; 43 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 925 KiB/s wr, 106 op/s 2024-03-18T20:39:20.679 INFO:tasks.workunit.client.0.smithi046.stdout: 1 85899 1.28 MB/sec execute 15 sec latency 2004.947 ms 2024-03-18T20:39:21.679 INFO:tasks.workunit.client.0.smithi046.stdout: 1 85899 1.20 MB/sec execute 16 sec latency 3005.054 ms 2024-03-18T20:39:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:22 smithi073 ceph-mon[47384]: pgmap v101: 97 pgs: 97 active+clean; 43 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 523 KiB/s wr, 67 op/s 2024-03-18T20:39:22.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:22 smithi155 ceph-mon[48344]: pgmap v101: 97 pgs: 97 active+clean; 43 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 523 KiB/s wr, 67 op/s 2024-03-18T20:39:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:22 smithi046 ceph-mon[43495]: pgmap v101: 97 pgs: 97 active+clean; 43 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 523 KiB/s wr, 67 op/s 2024-03-18T20:39:22.679 INFO:tasks.workunit.client.0.smithi046.stdout: 1 85967 1.14 MB/sec execute 17 sec latency 3366.987 ms 2024-03-18T20:39:23.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:23 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:39:23.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:23 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:39:23.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:23 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:39:23.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:23 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:39:23.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:23 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:39:23.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:23 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:39:23.679 INFO:tasks.workunit.client.0.smithi046.stdout: 1 85967 1.08 MB/sec execute 18 sec latency 1593.954 ms 2024-03-18T20:39:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:24 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:39:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:24 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:39:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:24 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:39:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:24 smithi073 ceph-mon[47384]: pgmap v102: 97 pgs: 97 active+clean; 46 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-03-18T20:39:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:24 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:39:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:24 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:39:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:24 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:39:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:24 smithi155 ceph-mon[48344]: pgmap v102: 97 pgs: 97 active+clean; 46 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-03-18T20:39:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:24 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:39:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:24 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:39:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:24 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:39:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:24 smithi046 ceph-mon[43495]: pgmap v102: 97 pgs: 97 active+clean; 46 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-03-18T20:39:24.679 INFO:tasks.workunit.client.0.smithi046.stdout: 1 85967 1.02 MB/sec execute 19 sec latency 2594.005 ms 2024-03-18T20:39:25.679 INFO:tasks.workunit.client.0.smithi046.stdout: 1 85967 0.97 MB/sec execute 20 sec latency 3594.113 ms 2024-03-18T20:39:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:25 smithi073 ceph-mon[47384]: pgmap v103: 97 pgs: 97 active+clean; 46 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-03-18T20:39:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:25 smithi155 ceph-mon[48344]: pgmap v103: 97 pgs: 97 active+clean; 46 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-03-18T20:39:26.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:25 smithi046 ceph-mon[43495]: pgmap v103: 97 pgs: 97 active+clean; 46 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-03-18T20:39:26.679 INFO:tasks.workunit.client.0.smithi046.stdout: 1 85967 0.92 MB/sec execute 21 sec latency 4594.225 ms 2024-03-18T20:39:27.679 INFO:tasks.workunit.client.0.smithi046.stdout: 1 87758 1.23 MB/sec execute 22 sec latency 4948.795 ms 2024-03-18T20:39:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.092+0000 7f9aea38b700 1 -- 172.21.15.46:0/4103982028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae41018a0 msgr2=0x7f9ae41058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.092+0000 7f9aea38b700 1 --2- 172.21.15.46:0/4103982028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae41018a0 0x7f9ae41058e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009990 tx=0x7f9ad8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.092+0000 7f9aea38b700 1 -- 172.21.15.46:0/4103982028 shutdown_connections 2024-03-18T20:39:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.092+0000 7f9aea38b700 1 --2- 172.21.15.46:0/4103982028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ae4106010 0x7f9ae410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.092+0000 7f9aea38b700 1 --2- 172.21.15.46:0/4103982028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ae4100ef0 0x7f9ae41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.092+0000 7f9aea38b700 1 --2- 172.21.15.46:0/4103982028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae41018a0 0x7f9ae41058e0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.092+0000 7f9aea38b700 1 -- 172.21.15.46:0/4103982028 >> 172.21.15.46:0/4103982028 conn(0x7f9ae40fc750 msgr2=0x7f9ae40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.092+0000 7f9aea38b700 1 -- 172.21.15.46:0/4103982028 shutdown_connections 2024-03-18T20:39:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.092+0000 7f9aea38b700 1 -- 172.21.15.46:0/4103982028 wait complete. 2024-03-18T20:39:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9aea38b700 1 Processor -- start 2024-03-18T20:39:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9aea38b700 1 -- start start 2024-03-18T20:39:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9aea38b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae4100ef0 0x7f9ae40730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9aea38b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ae4106010 0x7f9ae40735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9aea38b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ae4073b30 0x7f9ae41a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9aea38b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae41a59c0 con 0x7f9ae4100ef0 2024-03-18T20:39:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9aea38b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae41a5b30 con 0x7f9ae4106010 2024-03-18T20:39:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9aea38b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae41a5cf0 con 0x7f9ae4073b30 2024-03-18T20:39:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9ae3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae4100ef0 0x7f9ae40730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9ae37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ae4106010 0x7f9ae40735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9ae3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae4100ef0 0x7f9ae40730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47956/0 (socket says 172.21.15.46:47956) 2024-03-18T20:39:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9ae3fff700 1 -- 172.21.15.46:0/2610236405 learned_addr learned my addr 172.21.15.46:0/2610236405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.093+0000 7f9ae8928700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ae4073b30 0x7f9ae41a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9ae37fe700 1 -- 172.21.15.46:0/2610236405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ae4073b30 msgr2=0x7f9ae41a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9ae37fe700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ae4073b30 0x7f9ae41a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9ae37fe700 1 -- 172.21.15.46:0/2610236405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae4100ef0 msgr2=0x7f9ae40730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9ae37fe700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae4100ef0 0x7f9ae40730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9ae37fe700 1 -- 172.21.15.46:0/2610236405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad80095f0 con 0x7f9ae4106010 2024-03-18T20:39:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9ae3fff700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae4100ef0 0x7f9ae40730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9ae8928700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ae4073b30 0x7f9ae41a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9ae37fe700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ae4106010 0x7f9ae40735f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8000c00 tx=0x7f9ad8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9ae17fa700 1 -- 172.21.15.46:0/2610236405 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad801d070 con 0x7f9ae4106010 2024-03-18T20:39:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9ae17fa700 1 -- 172.21.15.46:0/2610236405 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ad8021410 con 0x7f9ae4106010 2024-03-18T20:39:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9aea38b700 1 -- 172.21.15.46:0/2610236405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae41a5f70 con 0x7f9ae4106010 2024-03-18T20:39:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9ae17fa700 1 -- 172.21.15.46:0/2610236405 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8017650 con 0x7f9ae4106010 2024-03-18T20:39:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.094+0000 7f9aea38b700 1 -- 172.21.15.46:0/2610236405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae41a6390 con 0x7f9ae4106010 2024-03-18T20:39:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.096+0000 7f9ae17fa700 1 -- 172.21.15.46:0/2610236405 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9ad8021580 con 0x7f9ae4106010 2024-03-18T20:39:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.097+0000 7f9ae17fa700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9acc0a1310 0x7f9acc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.097+0000 7f9ae3fff700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9acc0a1310 0x7f9acc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.097+0000 7f9ae17fa700 1 -- 172.21.15.46:0/2610236405 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9ad80cde20 con 0x7f9ae4106010 2024-03-18T20:39:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.098+0000 7f9ae3fff700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9acc0a1310 0x7f9acc0a37d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4011550 tx=0x7f9ad4009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.098+0000 7f9af6520b80 1 Processor -- start 2024-03-18T20:39:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9af6520b80 1 -- start start 2024-03-18T20:39:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9af6520b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 0x564a35835950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9af6520b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835e90 0x564a35840870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9af6520b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a358431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9af6520b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3578f930 con 0x564a35840db0 2024-03-18T20:39:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9af6520b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3578d2f0 con 0x564a35835e90 2024-03-18T20:39:28.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9af6520b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a35788be0 con 0x564a35835570 2024-03-18T20:39:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9ae8928700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a358431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9ae3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 0x564a35835950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9ae8928700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a358431a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47968/0 (socket says 172.21.15.46:47968) 2024-03-18T20:39:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9ae8928700 1 -- 172.21.15.46:0/22035383 learned_addr learned my addr 172.21.15.46:0/22035383 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9ae3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 0x564a35835950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37422/0 (socket says 172.21.15.46:37422) 2024-03-18T20:39:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9ae37fe700 1 --2- 172.21.15.46:0/22035383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835e90 0x564a35840870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9ae8928700 1 -- 172.21.15.46:0/22035383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 msgr2=0x564a35835950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9ae8928700 1 --2- 172.21.15.46:0/22035383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 0x564a35835950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9ae8928700 1 -- 172.21.15.46:0/22035383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835e90 msgr2=0x564a35840870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9ae8928700 1 --2- 172.21.15.46:0/22035383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835e90 0x564a35840870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9ae8928700 1 -- 172.21.15.46:0/22035383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a3560a350 con 0x564a35840db0 2024-03-18T20:39:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.099+0000 7f9ae37fe700 1 --2- 172.21.15.46:0/22035383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835e90 0x564a35840870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.100+0000 7f9ae8928700 1 --2- 172.21.15.46:0/22035383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a358431a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9adc003eb0 tx=0x7f9adc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3362dad4708eb6ef server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.100+0000 7f9aca7fc700 1 -- 172.21.15.46:0/22035383 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0092d0 con 0x564a35840db0 2024-03-18T20:39:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.100+0000 7f9aca7fc700 1 -- 172.21.15.46:0/22035383 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9adc00f040 con 0x564a35840db0 2024-03-18T20:39:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.100+0000 7f9aca7fc700 1 -- 172.21.15.46:0/22035383 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc00e750 con 0x564a35840db0 2024-03-18T20:39:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.100+0000 7f9af6520b80 1 -- 172.21.15.46:0/22035383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 msgr2=0x564a358431a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.100+0000 7f9af6520b80 1 --2- 172.21.15.46:0/22035383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a358431a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9adc003eb0 tx=0x7f9adc009920 comp rx=0 tx=0).stop 2024-03-18T20:39:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.101+0000 7f9af6520b80 1 -- 172.21.15.46:0/22035383 shutdown_connections 2024-03-18T20:39:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.101+0000 7f9af6520b80 1 --2- 172.21.15.46:0/22035383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a358431a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.101+0000 7f9af6520b80 1 --2- 172.21.15.46:0/22035383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 0x564a35835950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.101+0000 7f9af6520b80 1 --2- 172.21.15.46:0/22035383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835e90 0x564a35840870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.101+0000 7f9af6520b80 1 -- 172.21.15.46:0/22035383 >> 172.21.15.46:0/22035383 conn(0x564a357ac1c0 msgr2=0x564a35831a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.101+0000 7f9af6520b80 1 -- 172.21.15.46:0/22035383 shutdown_connections 2024-03-18T20:39:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.101+0000 7f9af6520b80 1 -- 172.21.15.46:0/22035383 wait complete. 2024-03-18T20:39:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.101+0000 7f9af6520b80 1 -- start start 2024-03-18T20:39:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.101+0000 7f9af6520b80 1 Processor -- start 2024-03-18T20:39:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9af6520b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35835570 0x564a35840760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9af6520b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835e90 0x564a3583da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9af6520b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35840db0 0x564a3583e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9af6520b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a35724d20 con 0x564a35835570 2024-03-18T20:39:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9af6520b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a357249c0 con 0x564a35840db0 2024-03-18T20:39:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9af6520b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3560c640 con 0x564a35835e90 2024-03-18T20:39:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9af6520b80 10 client.0 _ll_register_callbacks cb 0x564a35590340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ae3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35835570 0x564a35840760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ae3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35835570 0x564a35840760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47970/0 (socket says 172.21.15.46:47970) 2024-03-18T20:39:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ae3fff700 1 -- 172.21.15.46:0/199568768 learned_addr learned my addr 172.21.15.46:0/199568768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ae8928700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35840db0 0x564a3583e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ae37fe700 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835e90 0x564a3583da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ae3fff700 1 -- 172.21.15.46:0/199568768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835e90 msgr2=0x564a3583da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ae3fff700 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835e90 0x564a3583da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ae3fff700 1 -- 172.21.15.46:0/199568768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35840db0 msgr2=0x564a3583e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ae3fff700 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35840db0 0x564a3583e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ae3fff700 1 -- 172.21.15.46:0/199568768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae41a6390 con 0x564a35835570 2024-03-18T20:39:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ae8928700 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35840db0 0x564a3583e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ae3fff700 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35835570 0x564a35840760 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400ee30 tx=0x7f9ad40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d2906426c4b317 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ac37fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.102+0000 7f9ac37fe700 1 -- 172.21.15.46:0/199568768 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4022070 con 0x564a35835570 2024-03-18T20:39:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9ac37fe700 1 -- 172.21.15.46:0/199568768 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9ad401dd10 con 0x564a35835570 2024-03-18T20:39:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9af6520b80 1 -- 172.21.15.46:0/199568768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a358296c0 con 0x564a35835570 2024-03-18T20:39:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9ac37fe700 1 -- 172.21.15.46:0/199568768 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ad400ee60 con 0x564a35835570 2024-03-18T20:39:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9af6520b80 10 client.16728 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9af6520b80 1 -- 172.21.15.46:0/199568768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f9adc003b90 con 0x564a35835570 2024-03-18T20:39:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9ac37fe700 1 -- 172.21.15.46:0/199568768 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ad4033080 con 0x564a35835570 2024-03-18T20:39:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9ac37fe700 1 -- 172.21.15.46:0/199568768 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9ad402f480 con 0x564a35835570 2024-03-18T20:39:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9af6520b80 10 client.16728 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9af6520b80 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9af6520b80 20 client.16728 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9af6520b80 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a35850aa0 0x564a35852ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9af6520b80 4 client.16728 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9af6520b80 1 -- 172.21.15.46:0/199568768 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a357816a0 con 0x564a35850aa0 2024-03-18T20:39:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.103+0000 7f9ae37fe700 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a35850aa0 0x564a35852ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.104+0000 7f9ae37fe700 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a35850aa0 0x564a35852ec0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d628ba3abff1fc server_cookie=2364fd7de8e92661 in_seq=0 out_seq=0 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.104+0000 7f9ac37fe700 10 client.16728 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.104+0000 7f9ac37fe700 1 -- 172.21.15.46:0/199568768 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564a357816a0 con 0x564a35850aa0 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.104+0000 7f9ac37fe700 10 client.16728 handle_command_reply: tid=0 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.104+0000 7f9af6520b80 1 client.16728 shutdown 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.105+0000 7f9af6520b80 1 -- 172.21.15.46:0/199568768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35835570 msgr2=0x564a35840760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.105+0000 7f9af6520b80 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35835570 0x564a35840760 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400ee30 tx=0x7f9ad40047c0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.105+0000 7f9af6520b80 1 -- 172.21.15.46:0/199568768 shutdown_connections 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.105+0000 7f9af6520b80 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a35850aa0 0x564a35852ec0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.105+0000 7f9af6520b80 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35835570 0x564a35840760 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.105+0000 7f9af6520b80 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35840db0 0x564a3583e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.105+0000 7f9af6520b80 1 --2- 172.21.15.46:0/199568768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835e90 0x564a3583da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.105+0000 7f9ac37fe700 0 client.16728 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.105+0000 7f9af6520b80 1 -- 172.21.15.46:0/199568768 >> 172.21.15.46:0/199568768 conn(0x564a357ac1c0 msgr2=0x564a35831650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.105+0000 7f9af6520b80 1 -- 172.21.15.46:0/199568768 shutdown_connections 2024-03-18T20:39:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.105+0000 7f9af6520b80 1 -- 172.21.15.46:0/199568768 wait complete. 2024-03-18T20:39:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.105+0000 7f9af6520b80 20 client.16728 trim_cache size 0 max 16384 2024-03-18T20:39:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.122+0000 7f9af6520b80 1 Processor -- start 2024-03-18T20:39:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.122+0000 7f9af6520b80 1 -- start start 2024-03-18T20:39:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.122+0000 7f9af6520b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 0x564a35840290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.122+0000 7f9af6520b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835e90 0x564a3583d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.122+0000 7f9af6520b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a3583d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.122+0000 7f9af6520b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3584e010 con 0x564a35840db0 2024-03-18T20:39:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.122+0000 7f9af6520b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3583bfe0 con 0x564a35835e90 2024-03-18T20:39:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.122+0000 7f9af6520b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a358398f0 con 0x564a35835570 2024-03-18T20:39:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9ae3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 0x564a35840290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9ae3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 0x564a35840290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37442/0 (socket says 172.21.15.46:37442) 2024-03-18T20:39:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9ae3fff700 1 -- 172.21.15.46:0/3149669664 learned_addr learned my addr 172.21.15.46:0/3149669664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9ae37fe700 1 --2- 172.21.15.46:0/3149669664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835e90 0x564a3583d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9ae8928700 1 --2- 172.21.15.46:0/3149669664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a3583d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9ae3fff700 1 -- 172.21.15.46:0/3149669664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835e90 msgr2=0x564a3583d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9ae3fff700 1 --2- 172.21.15.46:0/3149669664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835e90 0x564a3583d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9ae3fff700 1 -- 172.21.15.46:0/3149669664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 msgr2=0x564a3583d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9ae3fff700 1 --2- 172.21.15.46:0/3149669664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a3583d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9ae3fff700 1 -- 172.21.15.46:0/3149669664 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a355b6b00 con 0x564a35835570 2024-03-18T20:39:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9ae3fff700 1 --2- 172.21.15.46:0/3149669664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 0x564a35840290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4013010 tx=0x7f9ad40046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d988acc551ee7137 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9acaffd700 1 -- 172.21.15.46:0/3149669664 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4022050 con 0x564a35835570 2024-03-18T20:39:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9acaffd700 1 -- 172.21.15.46:0/3149669664 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a358296c0 con 0x564a35835570 2024-03-18T20:39:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.123+0000 7f9acaffd700 1 -- 172.21.15.46:0/3149669664 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4004d00 con 0x564a35835570 2024-03-18T20:39:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.124+0000 7f9af6520b80 1 -- 172.21.15.46:0/3149669664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 msgr2=0x564a35840290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.124+0000 7f9af6520b80 1 --2- 172.21.15.46:0/3149669664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 0x564a35840290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4013010 tx=0x7f9ad40046b0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.124+0000 7f9af6520b80 1 -- 172.21.15.46:0/3149669664 shutdown_connections 2024-03-18T20:39:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.124+0000 7f9af6520b80 1 --2- 172.21.15.46:0/3149669664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a3583d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.124+0000 7f9af6520b80 1 --2- 172.21.15.46:0/3149669664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835570 0x564a35840290 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.124+0000 7f9af6520b80 1 --2- 172.21.15.46:0/3149669664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835e90 0x564a3583d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.124+0000 7f9af6520b80 1 -- 172.21.15.46:0/3149669664 >> 172.21.15.46:0/3149669664 conn(0x564a357ac1c0 msgr2=0x564a35830120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.125+0000 7f9af6520b80 1 -- 172.21.15.46:0/3149669664 shutdown_connections 2024-03-18T20:39:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.125+0000 7f9af6520b80 1 -- 172.21.15.46:0/3149669664 wait complete. 2024-03-18T20:39:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.125+0000 7f9af6520b80 1 -- start start 2024-03-18T20:39:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.125+0000 7f9af6520b80 1 Processor -- start 2024-03-18T20:39:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.125+0000 7f9af6520b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835570 0x564a3583b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.125+0000 7f9af6520b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835e90 0x564a3583b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9af6520b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a35839d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9af6520b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3583a240 con 0x564a35840db0 2024-03-18T20:39:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9af6520b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3583a3b0 con 0x564a35835570 2024-03-18T20:39:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9af6520b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3583a570 con 0x564a35835e90 2024-03-18T20:39:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9af6520b80 10 client.0 _ll_register_callbacks cb 0x7f9ad400ea10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835570 0x564a3583b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae8928700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a35839d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835e90 0x564a3583b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835570 0x564a3583b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:60144/0 (socket says 172.21.15.46:60144) 2024-03-18T20:39:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae8928700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a35839d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47986/0 (socket says 172.21.15.46:47986) 2024-03-18T20:39:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae3fff700 1 -- 172.21.15.46:0/4215474979 learned_addr learned my addr 172.21.15.46:0/4215474979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae3fff700 1 -- 172.21.15.46:0/4215474979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835e90 msgr2=0x564a3583b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae3fff700 1 --2- 172.21.15.46:0/4215474979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835e90 0x564a3583b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae3fff700 1 -- 172.21.15.46:0/4215474979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 msgr2=0x564a35839d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae3fff700 1 --2- 172.21.15.46:0/4215474979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a35839d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae3fff700 1 -- 172.21.15.46:0/4215474979 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc003b90 con 0x564a35835570 2024-03-18T20:39:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae37fe700 1 --2- 172.21.15.46:0/4215474979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835e90 0x564a3583b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ae3fff700 1 --2- 172.21.15.46:0/4215474979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835570 0x564a3583b0b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4000c00 tx=0x7f9ad402fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8023040cf57cb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ac37fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:39:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.126+0000 7f9ac37fe700 1 -- 172.21.15.46:0/4215474979 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4022040 con 0x564a35835570 2024-03-18T20:39:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.127+0000 7f9ac37fe700 1 -- 172.21.15.46:0/4215474979 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9ad4009350 con 0x564a35835570 2024-03-18T20:39:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.127+0000 7f9af6520b80 1 -- 172.21.15.46:0/4215474979 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9adc00f040 con 0x564a35835570 2024-03-18T20:39:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.127+0000 7f9ac37fe700 1 -- 172.21.15.46:0/4215474979 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ad4005250 con 0x564a35835570 2024-03-18T20:39:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.127+0000 7f9af6520b80 10 client.26483 fetch_fsmap learned FSMap version 33 2024-03-18T20:39:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.127+0000 7f9af6520b80 1 -- 172.21.15.46:0/4215474979 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564a3574ee40 con 0x564a35835570 2024-03-18T20:39:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.128+0000 7f9ac37fe700 1 -- 172.21.15.46:0/4215474979 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9adc00f040 con 0x564a35835570 2024-03-18T20:39:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.128+0000 7f9ac37fe700 1 -- 172.21.15.46:0/4215474979 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9ad404c470 con 0x564a35835570 2024-03-18T20:39:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.128+0000 7f9af6520b80 10 client.26483 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T20:39:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.128+0000 7f9af6520b80 10 client.26483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.128+0000 7f9af6520b80 20 client.26483 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.128+0000 7f9af6520b80 1 --2- 172.21.15.46:0/4215474979 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a358a5550 0x564a358a7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.128+0000 7f9af6520b80 4 client.26483 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:39:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.128+0000 7f9af6520b80 1 -- 172.21.15.46:0/4215474979 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564a355d7c90 con 0x564a358a5550 2024-03-18T20:39:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.128+0000 7f9ae37fe700 1 --2- 172.21.15.46:0/4215474979 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a358a5550 0x564a358a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.128+0000 7f9ac37fe700 10 client.26483 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.129+0000 7f9ae37fe700 1 --2- 172.21.15.46:0/4215474979 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a358a5550 0x564a358a7970 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6f8e1e43f97095 server_cookie=1b17e6870fe01577 in_seq=0 out_seq=0 2024-03-18T20:39:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9ac37fe700 1 -- 172.21.15.46:0/4215474979 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564a355d7c90 con 0x564a358a5550 2024-03-18T20:39:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9ac37fe700 10 client.26483 handle_command_reply: tid=0 2024-03-18T20:39:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9af6520b80 1 client.26483 shutdown 2024-03-18T20:39:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9af6520b80 1 -- 172.21.15.46:0/4215474979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835570 msgr2=0x564a3583b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9af6520b80 1 --2- 172.21.15.46:0/4215474979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835570 0x564a3583b0b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4000c00 tx=0x7f9ad402fc30 comp rx=0 tx=0).stop 2024-03-18T20:39:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9af6520b80 1 -- 172.21.15.46:0/4215474979 shutdown_connections 2024-03-18T20:39:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9af6520b80 1 --2- 172.21.15.46:0/4215474979 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a358a5550 0x564a358a7970 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9af6520b80 1 --2- 172.21.15.46:0/4215474979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a35840db0 0x564a35839d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9af6520b80 1 --2- 172.21.15.46:0/4215474979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a35835570 0x564a3583b0b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9ac37fe700 0 client.26483 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9af6520b80 1 --2- 172.21.15.46:0/4215474979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a35835e90 0x564a3583b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9af6520b80 1 -- 172.21.15.46:0/4215474979 >> 172.21.15.46:0/4215474979 conn(0x564a357ac1c0 msgr2=0x564a3584c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9af6520b80 1 -- 172.21.15.46:0/4215474979 shutdown_connections 2024-03-18T20:39:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.158+0000 7f9af6520b80 1 -- 172.21.15.46:0/4215474979 wait complete. 2024-03-18T20:39:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.159+0000 7f9af6520b80 20 client.26483 trim_cache size 0 max 16384 2024-03-18T20:39:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:39:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:39:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:39:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:39:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.159+0000 7f9aea38b700 1 -- 172.21.15.46:0/2610236405 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9acc0a1310 msgr2=0x7f9acc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.159+0000 7f9aea38b700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9acc0a1310 0x7f9acc0a37d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4011550 tx=0x7f9ad4009260 comp rx=0 tx=0).stop 2024-03-18T20:39:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.159+0000 7f9aea38b700 1 -- 172.21.15.46:0/2610236405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ae4106010 msgr2=0x7f9ae40735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.159+0000 7f9aea38b700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ae4106010 0x7f9ae40735f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8000c00 tx=0x7f9ad8004730 comp rx=0 tx=0).stop 2024-03-18T20:39:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.159+0000 7f9aea38b700 1 -- 172.21.15.46:0/2610236405 shutdown_connections 2024-03-18T20:39:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.159+0000 7f9aea38b700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9acc0a1310 0x7f9acc0a37d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.159+0000 7f9aea38b700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ae4100ef0 0x7f9ae40730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.159+0000 7f9aea38b700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ae4073b30 0x7f9ae41a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.159+0000 7f9aea38b700 1 --2- 172.21.15.46:0/2610236405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ae4106010 0x7f9ae40735f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.159+0000 7f9aea38b700 1 -- 172.21.15.46:0/2610236405 >> 172.21.15.46:0/2610236405 conn(0x7f9ae40fc750 msgr2=0x7f9ae40fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.159+0000 7f9aea38b700 1 -- 172.21.15.46:0/2610236405 shutdown_connections 2024-03-18T20:39:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:28.160+0000 7f9aea38b700 1 -- 172.21.15.46:0/2610236405 wait complete. 2024-03-18T20:39:28.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:28 smithi073 ceph-mon[47384]: pgmap v104: 97 pgs: 97 active+clean; 46 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-03-18T20:39:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:28 smithi155 ceph-mon[48344]: pgmap v104: 97 pgs: 97 active+clean; 46 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-03-18T20:39:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:28 smithi046 ceph-mon[43495]: pgmap v104: 97 pgs: 97 active+clean; 46 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-03-18T20:39:28.679 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89314 1.66 MB/sec execute 23 sec latency 13.208 ms 2024-03-18T20:39:29.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:29 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:29.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:29 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:29.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:29 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:29.679 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89392 1.59 MB/sec execute 24 sec latency 958.430 ms 2024-03-18T20:39:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:30 smithi073 ceph-mon[47384]: pgmap v105: 97 pgs: 97 active+clean; 44 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-18T20:39:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:30 smithi155 ceph-mon[48344]: pgmap v105: 97 pgs: 97 active+clean; 44 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-18T20:39:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:30 smithi046 ceph-mon[43495]: pgmap v105: 97 pgs: 97 active+clean; 44 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-18T20:39:30.680 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89392 1.53 MB/sec execute 25 sec latency 1958.522 ms 2024-03-18T20:39:31.680 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89392 1.47 MB/sec execute 26 sec latency 2958.592 ms 2024-03-18T20:39:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:32 smithi073 ceph-mon[47384]: pgmap v106: 97 pgs: 97 active+clean; 44 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 752 KiB/s wr, 72 op/s 2024-03-18T20:39:32.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:32 smithi155 ceph-mon[48344]: pgmap v106: 97 pgs: 97 active+clean; 44 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 752 KiB/s wr, 72 op/s 2024-03-18T20:39:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:32 smithi046 ceph-mon[43495]: pgmap v106: 97 pgs: 97 active+clean; 44 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 752 KiB/s wr, 72 op/s 2024-03-18T20:39:32.680 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89448 1.42 MB/sec execute 27 sec latency 3320.461 ms 2024-03-18T20:39:33.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:33 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:33 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:33.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:33 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:33.680 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89448 1.37 MB/sec execute 28 sec latency 1600.679 ms 2024-03-18T20:39:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:34 smithi073 ceph-mon[47384]: pgmap v107: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-03-18T20:39:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:34 smithi155 ceph-mon[48344]: pgmap v107: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-03-18T20:39:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:34 smithi046 ceph-mon[43495]: pgmap v107: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-03-18T20:39:34.680 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89448 1.32 MB/sec execute 29 sec latency 2600.788 ms 2024-03-18T20:39:35.680 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89448 1.28 MB/sec execute 30 sec latency 3600.855 ms 2024-03-18T20:39:36.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:35 smithi073 ceph-mon[47384]: pgmap v108: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 499 KiB/s wr, 73 op/s 2024-03-18T20:39:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:35 smithi155 ceph-mon[48344]: pgmap v108: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 499 KiB/s wr, 73 op/s 2024-03-18T20:39:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:35 smithi046 ceph-mon[43495]: pgmap v108: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 499 KiB/s wr, 73 op/s 2024-03-18T20:39:36.680 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89448 1.24 MB/sec execute 31 sec latency 4600.945 ms 2024-03-18T20:39:37.680 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89516 1.20 MB/sec execute 32 sec latency 4955.321 ms 2024-03-18T20:39:38.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:38 smithi073 ceph-mon[47384]: pgmap v109: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 499 KiB/s wr, 73 op/s 2024-03-18T20:39:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:38 smithi155 ceph-mon[48344]: pgmap v109: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 499 KiB/s wr, 73 op/s 2024-03-18T20:39:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:38 smithi046 ceph-mon[43495]: pgmap v109: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 499 KiB/s wr, 73 op/s 2024-03-18T20:39:38.680 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89516 1.17 MB/sec execute 33 sec latency 1594.842 ms 2024-03-18T20:39:39.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:39 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:39 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:39.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:39 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:39.680 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89516 1.13 MB/sec execute 34 sec latency 2594.930 ms 2024-03-18T20:39:40.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:40 smithi073 ceph-mon[47384]: pgmap v110: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 783 KiB/s wr, 95 op/s 2024-03-18T20:39:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:40 smithi155 ceph-mon[48344]: pgmap v110: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 783 KiB/s wr, 95 op/s 2024-03-18T20:39:40.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:40 smithi046 ceph-mon[43495]: pgmap v110: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 783 KiB/s wr, 95 op/s 2024-03-18T20:39:40.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89516 1.10 MB/sec execute 35 sec latency 3595.005 ms 2024-03-18T20:39:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.288+0000 7f3246464700 1 -- 172.21.15.46:0/168063562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3240100690 msgr2=0x7f3240100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.288+0000 7f3246464700 1 --2- 172.21.15.46:0/168063562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3240100690 0x7f3240100a70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3230009990 tx=0x7f3230009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.288+0000 7f3246464700 1 -- 172.21.15.46:0/168063562 shutdown_connections 2024-03-18T20:39:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.288+0000 7f3246464700 1 --2- 172.21.15.46:0/168063562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3240100690 0x7f3240100a70 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.288+0000 7f3246464700 1 --2- 172.21.15.46:0/168063562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f324010d700 0x7f324010faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.288+0000 7f3246464700 1 --2- 172.21.15.46:0/168063562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3240100fb0 0x7f324010d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.288+0000 7f3246464700 1 -- 172.21.15.46:0/168063562 >> 172.21.15.46:0/168063562 conn(0x7f3240076d50 msgr2=0x7f3240077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.288+0000 7f3246464700 1 -- 172.21.15.46:0/168063562 shutdown_connections 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.288+0000 7f3246464700 1 -- 172.21.15.46:0/168063562 wait complete. 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.289+0000 7f3246464700 1 Processor -- start 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.289+0000 7f3246464700 1 -- start start 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.289+0000 7f3246464700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3240100690 0x7f3240101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.289+0000 7f3246464700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3240100fb0 0x7f3240102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.289+0000 7f3246464700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f324010d700 0x7f32401028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.289+0000 7f3246464700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32401a9960 con 0x7f3240100fb0 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.289+0000 7f3246464700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32401a9aa0 con 0x7f3240100690 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.289+0000 7f3246464700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32401a9c10 con 0x7f324010d700 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.289+0000 7f3237fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3240100fb0 0x7f3240102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f3237fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3240100fb0 0x7f3240102370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38788/0 (socket says 172.21.15.46:38788) 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f3237fff700 1 -- 172.21.15.46:0/865531041 learned_addr learned my addr 172.21.15.46:0/865531041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f3244a01700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f324010d700 0x7f32401028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f323ffff700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3240100690 0x7f3240101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f3237fff700 1 -- 172.21.15.46:0/865531041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f324010d700 msgr2=0x7f32401028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f3237fff700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f324010d700 0x7f32401028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f3237fff700 1 -- 172.21.15.46:0/865531041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3240100690 msgr2=0x7f3240101e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f3237fff700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3240100690 0x7f3240101e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f3237fff700 1 -- 172.21.15.46:0/865531041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32300095f0 con 0x7f3240100fb0 2024-03-18T20:39:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f323ffff700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3240100690 0x7f3240101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:39:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f3237fff700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3240100fb0 0x7f3240102370 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f322800e970 tx=0x7f322800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f323dffb700 1 -- 172.21.15.46:0/865531041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228009790 con 0x7f3240100fb0 2024-03-18T20:39:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f3246464700 1 -- 172.21.15.46:0/865531041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32401a9ef0 con 0x7f3240100fb0 2024-03-18T20:39:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f323dffb700 1 -- 172.21.15.46:0/865531041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f322800eec0 con 0x7f3240100fb0 2024-03-18T20:39:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f323dffb700 1 -- 172.21.15.46:0/865531041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228003cf0 con 0x7f3240100fb0 2024-03-18T20:39:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.290+0000 7f3246464700 1 -- 172.21.15.46:0/865531041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32401aa3c0 con 0x7f3240100fb0 2024-03-18T20:39:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.293+0000 7f323dffb700 1 -- 172.21.15.46:0/865531041 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f322800ccd0 con 0x7f3240100fb0 2024-03-18T20:39:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.294+0000 7f323dffb700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f322c0a1290 0x7f322c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.294+0000 7f323dffb700 1 -- 172.21.15.46:0/865531041 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f3228014070 con 0x7f3240100fb0 2024-03-18T20:39:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.294+0000 7f323ffff700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f322c0a1290 0x7f322c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.294+0000 7f323ffff700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f322c0a1290 0x7f322c0a3750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f323000b2e0 tx=0x7f3230005e80 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f32525f9b80 1 Processor -- start 2024-03-18T20:39:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f32525f9b80 1 -- start start 2024-03-18T20:39:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f32525f9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11baba950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f32525f9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 0x55d11bac5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f32525f9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bac81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f32525f9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ba14930 con 0x55d11baba570 2024-03-18T20:39:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f32525f9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ba122f0 con 0x55d11babae90 2024-03-18T20:39:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f32525f9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11ba0dbe0 con 0x55d11bac5db0 2024-03-18T20:39:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f323ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11baba950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f323ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11baba950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38804/0 (socket says 172.21.15.46:38804) 2024-03-18T20:39:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f323ffff700 1 -- 172.21.15.46:0/2582208703 learned_addr learned my addr 172.21.15.46:0/2582208703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f3244a01700 1 --2- 172.21.15.46:0/2582208703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bac81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f3237fff700 1 --2- 172.21.15.46:0/2582208703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 0x55d11bac5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f323ffff700 1 -- 172.21.15.46:0/2582208703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 msgr2=0x55d11bac81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f323ffff700 1 --2- 172.21.15.46:0/2582208703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bac81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f323ffff700 1 -- 172.21.15.46:0/2582208703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 msgr2=0x55d11bac5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f323ffff700 1 --2- 172.21.15.46:0/2582208703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 0x55d11bac5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f323ffff700 1 -- 172.21.15.46:0/2582208703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d11b88f350 con 0x55d11baba570 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.295+0000 7f3244a01700 1 --2- 172.21.15.46:0/2582208703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bac81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.296+0000 7f323ffff700 1 --2- 172.21.15.46:0/2582208703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11baba950 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f32300057d0 tx=0x7f323001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fcf3429f9b1c0b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.296+0000 7f32367fc700 1 -- 172.21.15.46:0/2582208703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323001e070 con 0x55d11baba570 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.296+0000 7f32367fc700 1 -- 172.21.15.46:0/2582208703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3230005510 con 0x55d11baba570 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.296+0000 7f32367fc700 1 -- 172.21.15.46:0/2582208703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230004880 con 0x55d11baba570 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.296+0000 7f32525f9b80 1 -- 172.21.15.46:0/2582208703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 msgr2=0x55d11baba950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.297+0000 7f32525f9b80 1 --2- 172.21.15.46:0/2582208703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11baba950 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f32300057d0 tx=0x7f323001e000 comp rx=0 tx=0).stop 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.297+0000 7f32525f9b80 1 -- 172.21.15.46:0/2582208703 shutdown_connections 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.297+0000 7f32525f9b80 1 --2- 172.21.15.46:0/2582208703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11baba950 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.297+0000 7f32525f9b80 1 --2- 172.21.15.46:0/2582208703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bac81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.297+0000 7f32525f9b80 1 --2- 172.21.15.46:0/2582208703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 0x55d11bac5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.297+0000 7f32525f9b80 1 -- 172.21.15.46:0/2582208703 >> 172.21.15.46:0/2582208703 conn(0x55d11ba311c0 msgr2=0x55d11bab6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.297+0000 7f32525f9b80 1 -- 172.21.15.46:0/2582208703 shutdown_connections 2024-03-18T20:39:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.297+0000 7f32525f9b80 1 -- 172.21.15.46:0/2582208703 wait complete. 2024-03-18T20:39:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.297+0000 7f32525f9b80 1 -- start start 2024-03-18T20:39:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.297+0000 7f32525f9b80 1 Processor -- start 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f32525f9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11bac5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f32525f9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 0x55d11bac2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f32525f9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bac3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f32525f9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11b9a9d20 con 0x55d11baba570 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f32525f9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11b9a99c0 con 0x55d11babae90 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f32525f9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11b891640 con 0x55d11bac5db0 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f32525f9b80 10 client.0 _ll_register_callbacks cb 0x55d11b815340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f323ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11bac5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f323ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11bac5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38810/0 (socket says 172.21.15.46:38810) 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f323ffff700 1 -- 172.21.15.46:0/3684859912 learned_addr learned my addr 172.21.15.46:0/3684859912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f3237fff700 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 0x55d11bac2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f3244a01700 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bac3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f323ffff700 1 -- 172.21.15.46:0/3684859912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 msgr2=0x55d11bac3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f323ffff700 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bac3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f323ffff700 1 -- 172.21.15.46:0/3684859912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 msgr2=0x55d11bac2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f323ffff700 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 0x55d11bac2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f323ffff700 1 -- 172.21.15.46:0/3684859912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32401105f0 con 0x55d11baba570 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f323ffff700 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11bac5790 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3230006010 tx=0x7f323000faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb472cb4ceab4d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f321f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f321f7fe700 1 -- 172.21.15.46:0/3684859912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230004830 con 0x55d11baba570 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f321f7fe700 1 -- 172.21.15.46:0/3684859912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f3230005050 con 0x55d11baba570 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.298+0000 7f32525f9b80 1 -- 172.21.15.46:0/3684859912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d11baae6c0 con 0x55d11baba570 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.299+0000 7f321f7fe700 1 -- 172.21.15.46:0/3684859912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3230005210 con 0x55d11baba570 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.299+0000 7f32525f9b80 10 client.16746 fetch_fsmap learned FSMap version 36 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.299+0000 7f32525f9b80 1 -- 172.21.15.46:0/3684859912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d11b83bb00 con 0x55d11baba570 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.299+0000 7f321f7fe700 1 -- 172.21.15.46:0/3684859912 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d11baae6c0 con 0x55d11baba570 2024-03-18T20:39:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.299+0000 7f321f7fe700 1 -- 172.21.15.46:0/3684859912 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f3230042a10 con 0x55d11baba570 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.299+0000 7f32525f9b80 10 client.16746 fetch_fsmap finished waiting for FSMap version 36 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.299+0000 7f32525f9b80 10 client.16746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.299+0000 7f32525f9b80 20 client.16746 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.299+0000 7f32525f9b80 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d11bad5670 0x55d11bad7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.299+0000 7f32525f9b80 4 client.16746 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.299+0000 7f32525f9b80 1 -- 172.21.15.46:0/3684859912 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d11ba066a0 con 0x55d11bad5670 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.299+0000 7f3237fff700 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d11bad5670 0x55d11bad7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.300+0000 7f321f7fe700 10 client.16746 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.300+0000 7f3237fff700 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d11bad5670 0x55d11bad7a90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c7c2a6ec5d8b9f server_cookie=b57ac07d45fea41f in_seq=0 out_seq=0 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.300+0000 7f321f7fe700 1 -- 172.21.15.46:0/3684859912 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d11ba066a0 con 0x55d11bad5670 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.300+0000 7f321f7fe700 10 client.16746 handle_command_reply: tid=0 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.300+0000 7f32525f9b80 1 client.16746 shutdown 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.301+0000 7f32525f9b80 1 -- 172.21.15.46:0/3684859912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 msgr2=0x55d11bac5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.301+0000 7f32525f9b80 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11bac5790 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3230006010 tx=0x7f323000faa0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.301+0000 7f32525f9b80 1 -- 172.21.15.46:0/3684859912 shutdown_connections 2024-03-18T20:39:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.301+0000 7f32525f9b80 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d11bad5670 0x55d11bad7a90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.301+0000 7f32525f9b80 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11bac5790 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.301+0000 7f32525f9b80 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bac3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.301+0000 7f321f7fe700 0 client.16746 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.301+0000 7f32525f9b80 1 --2- 172.21.15.46:0/3684859912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 0x55d11bac2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.301+0000 7f32525f9b80 1 -- 172.21.15.46:0/3684859912 >> 172.21.15.46:0/3684859912 conn(0x55d11ba311c0 msgr2=0x55d11bab6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.301+0000 7f32525f9b80 1 -- 172.21.15.46:0/3684859912 shutdown_connections 2024-03-18T20:39:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.301+0000 7f32525f9b80 1 -- 172.21.15.46:0/3684859912 wait complete. 2024-03-18T20:39:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.301+0000 7f32525f9b80 20 client.16746 trim_cache size 0 max 16384 2024-03-18T20:39:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.316+0000 7f32525f9b80 1 Processor -- start 2024-03-18T20:39:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.316+0000 7f32525f9b80 1 -- start start 2024-03-18T20:39:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f32525f9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11bad7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f32525f9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 0x55d11bacca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f32525f9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bab5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f32525f9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11baccf80 con 0x55d11baba570 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f32525f9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11babe8c0 con 0x55d11babae90 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f32525f9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11baae200 con 0x55d11bac5db0 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f323ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11bad7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f323ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11bad7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38822/0 (socket says 172.21.15.46:38822) 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f323ffff700 1 -- 172.21.15.46:0/11766521 learned_addr learned my addr 172.21.15.46:0/11766521 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f3237fff700 1 --2- 172.21.15.46:0/11766521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 0x55d11bacca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f3244a01700 1 --2- 172.21.15.46:0/11766521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bab5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f323ffff700 1 -- 172.21.15.46:0/11766521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 msgr2=0x55d11bab5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f323ffff700 1 --2- 172.21.15.46:0/11766521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bab5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f323ffff700 1 -- 172.21.15.46:0/11766521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 msgr2=0x55d11bacca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f323ffff700 1 --2- 172.21.15.46:0/11766521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 0x55d11bacca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f323ffff700 1 -- 172.21.15.46:0/11766521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d11b9d3e40 con 0x55d11baba570 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f323ffff700 1 --2- 172.21.15.46:0/11766521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11bad7a60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f323000c010 tx=0x7f323000fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3b1164a2fb7ddf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f3236ffd700 1 -- 172.21.15.46:0/11766521 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323001e040 con 0x55d11baba570 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f3236ffd700 1 -- 172.21.15.46:0/11766521 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f323000f740 con 0x55d11baba570 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.317+0000 7f3236ffd700 1 -- 172.21.15.46:0/11766521 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230030d90 con 0x55d11baba570 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.318+0000 7f32525f9b80 1 -- 172.21.15.46:0/11766521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 msgr2=0x55d11bad7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.318+0000 7f32525f9b80 1 --2- 172.21.15.46:0/11766521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11bad7a60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f323000c010 tx=0x7f323000fb90 comp rx=0 tx=0).stop 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.318+0000 7f32525f9b80 1 -- 172.21.15.46:0/11766521 shutdown_connections 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.318+0000 7f32525f9b80 1 --2- 172.21.15.46:0/11766521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11baba570 0x55d11bad7a60 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.318+0000 7f32525f9b80 1 --2- 172.21.15.46:0/11766521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11bac5db0 0x55d11bab5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.318+0000 7f32525f9b80 1 --2- 172.21.15.46:0/11766521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11babae90 0x55d11bacca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.318+0000 7f32525f9b80 1 -- 172.21.15.46:0/11766521 >> 172.21.15.46:0/11766521 conn(0x55d11ba311c0 msgr2=0x55d11badc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.318+0000 7f32525f9b80 1 -- 172.21.15.46:0/11766521 shutdown_connections 2024-03-18T20:39:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.318+0000 7f32525f9b80 1 -- 172.21.15.46:0/11766521 wait complete. 2024-03-18T20:39:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.318+0000 7f32525f9b80 1 -- start start 2024-03-18T20:39:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f32525f9b80 1 Processor -- start 2024-03-18T20:39:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f32525f9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11baba570 0x55d11bac8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f32525f9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11babae90 0x55d11bac8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f32525f9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11bac5db0 0x55d11bac9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f32525f9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11bb28450 con 0x55d11bac5db0 2024-03-18T20:39:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f32525f9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11bb28590 con 0x55d11baba570 2024-03-18T20:39:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f32525f9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11bb286d0 con 0x55d11babae90 2024-03-18T20:39:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f32525f9b80 10 client.0 _ll_register_callbacks cb 0x7f3228095d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f3244a01700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11bac5db0 0x55d11bac9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f3244a01700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11bac5db0 0x55d11bac9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38834/0 (socket says 172.21.15.46:38834) 2024-03-18T20:39:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f3244a01700 1 -- 172.21.15.46:0/2819313419 learned_addr learned my addr 172.21.15.46:0/2819313419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f323ffff700 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11baba570 0x55d11bac8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f3237fff700 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11babae90 0x55d11bac8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.319+0000 7f3244a01700 1 -- 172.21.15.46:0/2819313419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11babae90 msgr2=0x55d11bac8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f3244a01700 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11babae90 0x55d11bac8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f3244a01700 1 -- 172.21.15.46:0/2819313419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11baba570 msgr2=0x55d11bac8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f3244a01700 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11baba570 0x55d11bac8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f3244a01700 1 -- 172.21.15.46:0/2819313419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d11b83bb00 con 0x55d11bac5db0 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f323ffff700 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11baba570 0x55d11bac8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f3244a01700 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11bac5db0 0x55d11bac9490 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3238016f60 tx=0x7f3238016f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee24e941ba2d0e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f321f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f321f7fe700 1 -- 172.21.15.46:0/2819313419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238014d10 con 0x55d11bac5db0 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f321f7fe700 1 -- 172.21.15.46:0/2819313419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f3238014e70 con 0x55d11bac5db0 2024-03-18T20:39:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f32525f9b80 1 -- 172.21.15.46:0/2819313419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3230005510 con 0x55d11bac5db0 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f321f7fe700 1 -- 172.21.15.46:0/2819313419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3238003a00 con 0x55d11bac5db0 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f32525f9b80 10 client.16752 fetch_fsmap learned FSMap version 36 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f32525f9b80 1 -- 172.21.15.46:0/2819313419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d11b893c70 con 0x55d11bac5db0 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f321f7fe700 1 -- 172.21.15.46:0/2819313419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3230005510 con 0x55d11bac5db0 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f321f7fe700 1 -- 172.21.15.46:0/2819313419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f3238027070 con 0x55d11bac5db0 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.320+0000 7f32525f9b80 10 client.16752 fetch_fsmap finished waiting for FSMap version 36 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.321+0000 7f32525f9b80 10 client.16752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.321+0000 7f32525f9b80 20 client.16752 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.321+0000 7f32525f9b80 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d11bb29f40 0x55d11bb2c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.321+0000 7f32525f9b80 4 client.16752 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.321+0000 7f32525f9b80 1 -- 172.21.15.46:0/2819313419 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d11b85cc90 con 0x55d11bb29f40 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.321+0000 7f323ffff700 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d11bb29f40 0x55d11bb2c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.321+0000 7f321f7fe700 10 client.16752 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.321+0000 7f323ffff700 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d11bb29f40 0x55d11bb2c360 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9e5bf7ced9f07a server_cookie=4172ab7098e43bf in_seq=0 out_seq=0 2024-03-18T20:39:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.344+0000 7f321f7fe700 1 -- 172.21.15.46:0/2819313419 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d11b85cc90 con 0x55d11bb29f40 2024-03-18T20:39:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.344+0000 7f321f7fe700 10 client.16752 handle_command_reply: tid=0 2024-03-18T20:39:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.344+0000 7f32525f9b80 1 client.16752 shutdown 2024-03-18T20:39:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f32525f9b80 1 -- 172.21.15.46:0/2819313419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11bac5db0 msgr2=0x55d11bac9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f32525f9b80 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11bac5db0 0x55d11bac9490 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3238016f60 tx=0x7f3238016f90 comp rx=0 tx=0).stop 2024-03-18T20:39:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f32525f9b80 1 -- 172.21.15.46:0/2819313419 shutdown_connections 2024-03-18T20:39:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f32525f9b80 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d11bb29f40 0x55d11bb2c360 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f32525f9b80 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d11bac5db0 0x55d11bac9490 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f32525f9b80 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d11baba570 0x55d11bac8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f32525f9b80 1 --2- 172.21.15.46:0/2819313419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d11babae90 0x55d11bac8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f321f7fe700 0 client.16752 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f32525f9b80 1 -- 172.21.15.46:0/2819313419 >> 172.21.15.46:0/2819313419 conn(0x55d11ba311c0 msgr2=0x55d11bab5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f32525f9b80 1 -- 172.21.15.46:0/2819313419 shutdown_connections 2024-03-18T20:39:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f32525f9b80 1 -- 172.21.15.46:0/2819313419 wait complete. 2024-03-18T20:39:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f32525f9b80 20 client.16752 trim_cache size 0 max 16384 2024-03-18T20:39:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:39:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:39:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:39:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:39:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f3246464700 1 -- 172.21.15.46:0/865531041 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f322c0a1290 msgr2=0x7f322c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f3246464700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f322c0a1290 0x7f322c0a3750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f323000b2e0 tx=0x7f3230005e80 comp rx=0 tx=0).stop 2024-03-18T20:39:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f3246464700 1 -- 172.21.15.46:0/865531041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3240100fb0 msgr2=0x7f3240102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f3246464700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3240100fb0 0x7f3240102370 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f322800e970 tx=0x7f322800ec80 comp rx=0 tx=0).stop 2024-03-18T20:39:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f3246464700 1 -- 172.21.15.46:0/865531041 shutdown_connections 2024-03-18T20:39:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f3246464700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f322c0a1290 0x7f322c0a3750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f3246464700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f324010d700 0x7f32401028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f3246464700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3240100690 0x7f3240101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f3246464700 1 --2- 172.21.15.46:0/865531041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3240100fb0 0x7f3240102370 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.345+0000 7f3246464700 1 -- 172.21.15.46:0/865531041 >> 172.21.15.46:0/865531041 conn(0x7f3240076d50 msgr2=0x7f324010edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.346+0000 7f3246464700 1 -- 172.21.15.46:0/865531041 shutdown_connections 2024-03-18T20:39:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:41.346+0000 7f3246464700 1 -- 172.21.15.46:0/865531041 wait complete. 2024-03-18T20:39:41.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 89516 1.07 MB/sec execute 36 sec latency 4595.140 ms 2024-03-18T20:39:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:42 smithi073 ceph-mon[47384]: pgmap v111: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 620 KiB/s wr, 61 op/s 2024-03-18T20:39:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:42 smithi155 ceph-mon[48344]: pgmap v111: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 620 KiB/s wr, 61 op/s 2024-03-18T20:39:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:42 smithi046 ceph-mon[43495]: pgmap v111: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 620 KiB/s wr, 61 op/s 2024-03-18T20:39:42.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 91297 1.25 MB/sec execute 37 sec latency 4952.328 ms 2024-03-18T20:39:43.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 92871 1.51 MB/sec execute 38 sec latency 11.132 ms 2024-03-18T20:39:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:44 smithi073 ceph-mon[47384]: pgmap v112: 97 pgs: 97 active+clean; 42 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 776 KiB/s wr, 97 op/s 2024-03-18T20:39:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:44 smithi155 ceph-mon[48344]: pgmap v112: 97 pgs: 97 active+clean; 42 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 776 KiB/s wr, 97 op/s 2024-03-18T20:39:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:44 smithi046 ceph-mon[43495]: pgmap v112: 97 pgs: 97 active+clean; 42 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 776 KiB/s wr, 97 op/s 2024-03-18T20:39:44.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 92936 1.47 MB/sec execute 39 sec latency 973.131 ms 2024-03-18T20:39:45.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 92936 1.43 MB/sec execute 40 sec latency 1973.225 ms 2024-03-18T20:39:46.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:45 smithi073 ceph-mon[47384]: pgmap v113: 97 pgs: 97 active+clean; 42 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 441 KiB/s wr, 57 op/s 2024-03-18T20:39:46.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:45 smithi155 ceph-mon[48344]: pgmap v113: 97 pgs: 97 active+clean; 42 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 441 KiB/s wr, 57 op/s 2024-03-18T20:39:46.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:45 smithi046 ceph-mon[43495]: pgmap v113: 97 pgs: 97 active+clean; 42 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 441 KiB/s wr, 57 op/s 2024-03-18T20:39:46.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 92936 1.40 MB/sec execute 41 sec latency 2973.324 ms 2024-03-18T20:39:47.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:39:47.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:39:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.274+0000 7f88c457e700 1 -- 172.21.15.46:0/3993106552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88bc1060a0 msgr2=0x7f88bc10dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.274+0000 7f88c457e700 1 --2- 172.21.15.46:0/3993106552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88bc1060a0 0x7f88bc10dc20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f88b400b0d0 tx=0x7f88b400b3e0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.275+0000 7f88c457e700 1 -- 172.21.15.46:0/3993106552 shutdown_connections 2024-03-18T20:39:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.275+0000 7f88c457e700 1 --2- 172.21.15.46:0/3993106552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88bc1060a0 0x7f88bc10dc20 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.275+0000 7f88c457e700 1 --2- 172.21.15.46:0/3993106552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88bc069540 0x7f88bc069920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.275+0000 7f88c457e700 1 --2- 172.21.15.46:0/3993106552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88bc069ef0 0x7f88bc105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.275+0000 7f88c457e700 1 -- 172.21.15.46:0/3993106552 >> 172.21.15.46:0/3993106552 conn(0x7f88bc076d50 msgr2=0x7f88bc077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.275+0000 7f88c457e700 1 -- 172.21.15.46:0/3993106552 shutdown_connections 2024-03-18T20:39:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.275+0000 7f88c457e700 1 -- 172.21.15.46:0/3993106552 wait complete. 2024-03-18T20:39:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.275+0000 7f88c457e700 1 Processor -- start 2024-03-18T20:39:47.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.275+0000 7f88c457e700 1 -- start start 2024-03-18T20:39:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c457e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88bc069540 0x7f88bc197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c457e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88bc1060a0 0x7f88bc197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c457e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88bc19bbe0 0x7f88bc19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c457e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc19c660 con 0x7f88bc069540 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c457e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc19c7d0 con 0x7f88bc19bbe0 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c457e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc19c990 con 0x7f88bc1060a0 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c231a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88bc069540 0x7f88bc197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c231a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88bc069540 0x7f88bc197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51804/0 (socket says 172.21.15.46:51804) 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c231a700 1 -- 172.21.15.46:0/1585305995 learned_addr learned my addr 172.21.15.46:0/1585305995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c1b19700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88bc1060a0 0x7f88bc197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c2b1b700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88bc19bbe0 0x7f88bc19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c231a700 1 -- 172.21.15.46:0/1585305995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88bc1060a0 msgr2=0x7f88bc197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c231a700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88bc1060a0 0x7f88bc197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c231a700 1 -- 172.21.15.46:0/1585305995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88bc19bbe0 msgr2=0x7f88bc19c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c231a700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88bc19bbe0 0x7f88bc19c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c231a700 1 -- 172.21.15.46:0/1585305995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b4009d00 con 0x7f88bc069540 2024-03-18T20:39:47.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.276+0000 7f88c231a700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88bc069540 0x7f88bc197300 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f88b800e9a0 tx=0x7f88b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.277+0000 7f88b37fe700 1 -- 172.21.15.46:0/1585305995 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8009790 con 0x7f88bc069540 2024-03-18T20:39:47.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.277+0000 7f88b37fe700 1 -- 172.21.15.46:0/1585305995 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88b8010430 con 0x7f88bc069540 2024-03-18T20:39:47.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.277+0000 7f88c457e700 1 -- 172.21.15.46:0/1585305995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc1a3ef0 con 0x7f88bc069540 2024-03-18T20:39:47.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.277+0000 7f88b37fe700 1 -- 172.21.15.46:0/1585305995 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8018680 con 0x7f88bc069540 2024-03-18T20:39:47.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.277+0000 7f88c457e700 1 -- 172.21.15.46:0/1585305995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc1a4370 con 0x7f88bc069540 2024-03-18T20:39:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.279+0000 7f88b37fe700 1 -- 172.21.15.46:0/1585305995 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f88b80105a0 con 0x7f88bc069540 2024-03-18T20:39:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.279+0000 7f88b37fe700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88a80a1290 0x7f88a80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.280+0000 7f88b37fe700 1 -- 172.21.15.46:0/1585305995 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f88b8014070 con 0x7f88bc069540 2024-03-18T20:39:47.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.280+0000 7f88c1b19700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88a80a1290 0x7f88a80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.280+0000 7f88c1b19700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88a80a1290 0x7f88a80a3750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f88bc198850 tx=0x7f88ac009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.280+0000 7f88d0713b80 1 Processor -- start 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88d0713b80 1 -- start start 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88d0713b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334ba60 0x55cdf334be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88d0713b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334c380 0x55cdf3356ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88d0713b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf3359610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88d0713b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf32a5990 con 0x55cdf3357220 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88d0713b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf32a3350 con 0x55cdf334ba60 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88d0713b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf329ec40 con 0x55cdf334c380 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88c2b1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf3359610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88c2b1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf3359610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51818/0 (socket says 172.21.15.46:51818) 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88c2b1b700 1 -- 172.21.15.46:0/1152239950 learned_addr learned my addr 172.21.15.46:0/1152239950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88c231a700 1 --2- 172.21.15.46:0/1152239950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334ba60 0x55cdf334be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88c1b19700 1 --2- 172.21.15.46:0/1152239950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334c380 0x55cdf3356ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88c2b1b700 1 -- 172.21.15.46:0/1152239950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334c380 msgr2=0x55cdf3356ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.281+0000 7f88c2b1b700 1 --2- 172.21.15.46:0/1152239950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334c380 0x55cdf3356ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.282+0000 7f88c2b1b700 1 -- 172.21.15.46:0/1152239950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334ba60 msgr2=0x55cdf334be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.282+0000 7f88c2b1b700 1 --2- 172.21.15.46:0/1152239950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334ba60 0x55cdf334be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.282+0000 7f88c2b1b700 1 -- 172.21.15.46:0/1152239950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdf31203b0 con 0x55cdf3357220 2024-03-18T20:39:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.282+0000 7f88c2b1b700 1 --2- 172.21.15.46:0/1152239950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf3359610 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009cb0 tx=0x7f88b4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30043df20d22462 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.282+0000 7f88b0ff9700 1 -- 172.21.15.46:0/1152239950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400e040 con 0x55cdf3357220 2024-03-18T20:39:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.282+0000 7f88b0ff9700 1 -- 172.21.15.46:0/1152239950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88b40060c0 con 0x55cdf3357220 2024-03-18T20:39:47.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.282+0000 7f88b0ff9700 1 -- 172.21.15.46:0/1152239950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4003a00 con 0x55cdf3357220 2024-03-18T20:39:47.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.283+0000 7f88d0713b80 1 -- 172.21.15.46:0/1152239950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 msgr2=0x55cdf3359610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.283+0000 7f88d0713b80 1 --2- 172.21.15.46:0/1152239950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf3359610 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009cb0 tx=0x7f88b4005a20 comp rx=0 tx=0).stop 2024-03-18T20:39:47.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.283+0000 7f88d0713b80 1 -- 172.21.15.46:0/1152239950 shutdown_connections 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.283+0000 7f88d0713b80 1 --2- 172.21.15.46:0/1152239950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf3359610 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.283+0000 7f88d0713b80 1 --2- 172.21.15.46:0/1152239950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334ba60 0x55cdf334be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.283+0000 7f88d0713b80 1 --2- 172.21.15.46:0/1152239950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334c380 0x55cdf3356ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.283+0000 7f88d0713b80 1 -- 172.21.15.46:0/1152239950 >> 172.21.15.46:0/1152239950 conn(0x55cdf32c2220 msgr2=0x55cdf3348020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.283+0000 7f88d0713b80 1 -- 172.21.15.46:0/1152239950 shutdown_connections 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.283+0000 7f88d0713b80 1 -- 172.21.15.46:0/1152239950 wait complete. 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.283+0000 7f88d0713b80 1 -- start start 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.283+0000 7f88d0713b80 1 Processor -- start 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88d0713b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334ba60 0x55cdf3356c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88d0713b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf334c380 0x55cdf3353f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88d0713b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf3357220 0x55cdf3354590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88d0713b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf323ad80 con 0x55cdf334c380 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88d0713b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf323aa20 con 0x55cdf3357220 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88d0713b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf31226a0 con 0x55cdf334ba60 2024-03-18T20:39:47.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88d0713b80 10 client.0 _ll_register_callbacks cb 0x55cdf30a63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88c1b19700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf334c380 0x55cdf3353f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88c1b19700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf334c380 0x55cdf3353f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51826/0 (socket says 172.21.15.46:51826) 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88c1b19700 1 -- 172.21.15.46:0/1906347798 learned_addr learned my addr 172.21.15.46:0/1906347798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88c2b1b700 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf3357220 0x55cdf3354590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88c231a700 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334ba60 0x55cdf3356c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88c1b19700 1 -- 172.21.15.46:0/1906347798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334ba60 msgr2=0x55cdf3356c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88c1b19700 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334ba60 0x55cdf3356c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88c1b19700 1 -- 172.21.15.46:0/1906347798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf3357220 msgr2=0x55cdf3354590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88c1b19700 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf3357220 0x55cdf3354590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88c1b19700 1 -- 172.21.15.46:0/1906347798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc10e460 con 0x55cdf334c380 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88c1b19700 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf334c380 0x55cdf3353f10 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f88ac01e810 tx=0x7f88ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd796e2c287e1208 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:47.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.284+0000 7f88a5ffb700 1 -- 172.21.15.46:0/1906347798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac028430 con 0x55cdf334c380 2024-03-18T20:39:47.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88a5ffb700 1 -- 172.21.15.46:0/1906347798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f88ac01f430 con 0x55cdf334c380 2024-03-18T20:39:47.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88d0713b80 1 -- 172.21.15.46:0/1906347798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdf333fad0 con 0x55cdf334c380 2024-03-18T20:39:47.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88a5ffb700 1 -- 172.21.15.46:0/1906347798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88ac01f5f0 con 0x55cdf334c380 2024-03-18T20:39:47.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88d0713b80 10 client.16764 fetch_fsmap learned FSMap version 37 2024-03-18T20:39:47.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88d0713b80 1 -- 172.21.15.46:0/1906347798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55cdf30ccb60 con 0x55cdf334c380 2024-03-18T20:39:47.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88a5ffb700 1 -- 172.21.15.46:0/1906347798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88ac034080 con 0x55cdf334c380 2024-03-18T20:39:47.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88a5ffb700 1 -- 172.21.15.46:0/1906347798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 37) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f88ac028a50 con 0x55cdf334c380 2024-03-18T20:39:47.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88d0713b80 10 client.16764 fetch_fsmap finished waiting for FSMap version 37 2024-03-18T20:39:47.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88d0713b80 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:47.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88d0713b80 20 client.16764 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:47.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88d0713b80 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55cdf3366eb0 0x55cdf33692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88d0713b80 4 client.16764 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88d0713b80 1 -- 172.21.15.46:0/1906347798 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdf3297700 con 0x55cdf3366eb0 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.285+0000 7f88c231a700 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55cdf3366eb0 0x55cdf33692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.286+0000 7f88a5ffb700 10 client.16764 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.286+0000 7f88c231a700 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55cdf3366eb0 0x55cdf33692d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578d2f410dff4c31 server_cookie=249a3e39026dd052 in_seq=0 out_seq=0 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.286+0000 7f88a5ffb700 1 -- 172.21.15.46:0/1906347798 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cdf3297700 con 0x55cdf3366eb0 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.286+0000 7f88a5ffb700 10 client.16764 handle_command_reply: tid=0 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.286+0000 7f88d0713b80 1 client.16764 shutdown 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.287+0000 7f88d0713b80 1 -- 172.21.15.46:0/1906347798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf334c380 msgr2=0x55cdf3353f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.287+0000 7f88d0713b80 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf334c380 0x55cdf3353f10 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f88ac01e810 tx=0x7f88ac0047c0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.287+0000 7f88d0713b80 1 -- 172.21.15.46:0/1906347798 shutdown_connections 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.287+0000 7f88d0713b80 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55cdf3366eb0 0x55cdf33692d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.287+0000 7f88d0713b80 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf3357220 0x55cdf3354590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.287+0000 7f88d0713b80 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334ba60 0x55cdf3356c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.287+0000 7f88d0713b80 1 --2- 172.21.15.46:0/1906347798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf334c380 0x55cdf3353f10 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.287+0000 7f88a5ffb700 0 client.16764 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:47.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.287+0000 7f88d0713b80 1 -- 172.21.15.46:0/1906347798 >> 172.21.15.46:0/1906347798 conn(0x55cdf32c2220 msgr2=0x55cdf3347c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.287+0000 7f88d0713b80 1 -- 172.21.15.46:0/1906347798 shutdown_connections 2024-03-18T20:39:47.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.287+0000 7f88d0713b80 1 -- 172.21.15.46:0/1906347798 wait complete. 2024-03-18T20:39:47.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.287+0000 7f88d0713b80 20 client.16764 trim_cache size 0 max 16384 2024-03-18T20:39:47.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.305+0000 7f88d0713b80 1 Processor -- start 2024-03-18T20:39:47.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.305+0000 7f88d0713b80 1 -- start start 2024-03-18T20:39:47.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.305+0000 7f88d0713b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334ba60 0x55cdf33527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.305+0000 7f88d0713b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334c380 0x55cdf3352d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.305+0000 7f88d0713b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf3362540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.305+0000 7f88d0713b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf3342250 con 0x55cdf3357220 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.305+0000 7f88d0713b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf3363900 con 0x55cdf334c380 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.305+0000 7f88d0713b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf3355020 con 0x55cdf334ba60 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88c2b1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf3362540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88c2b1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf3362540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51840/0 (socket says 172.21.15.46:51840) 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88c2b1b700 1 -- 172.21.15.46:0/3891076258 learned_addr learned my addr 172.21.15.46:0/3891076258 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88c231a700 1 --2- 172.21.15.46:0/3891076258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334ba60 0x55cdf33527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88c1b19700 1 --2- 172.21.15.46:0/3891076258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334c380 0x55cdf3352d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88c2b1b700 1 -- 172.21.15.46:0/3891076258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334ba60 msgr2=0x55cdf33527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88c2b1b700 1 --2- 172.21.15.46:0/3891076258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334ba60 0x55cdf33527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88c2b1b700 1 -- 172.21.15.46:0/3891076258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334c380 msgr2=0x55cdf3352d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88c2b1b700 1 --2- 172.21.15.46:0/3891076258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334c380 0x55cdf3352d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88c2b1b700 1 -- 172.21.15.46:0/3891076258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdf3264ea0 con 0x55cdf3357220 2024-03-18T20:39:47.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88c1b19700 1 --2- 172.21.15.46:0/3891076258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334c380 0x55cdf3352d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88c2b1b700 1 --2- 172.21.15.46:0/3891076258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf3362540 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f88b400b420 tx=0x7f88b40093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c60d927231f0f37 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88b17fa700 1 -- 172.21.15.46:0/3891076258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400e040 con 0x55cdf3357220 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88b17fa700 1 -- 172.21.15.46:0/3891076258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88b4003a00 con 0x55cdf3357220 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.306+0000 7f88b17fa700 1 -- 172.21.15.46:0/3891076258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40050c0 con 0x55cdf3357220 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.307+0000 7f88d0713b80 1 -- 172.21.15.46:0/3891076258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 msgr2=0x55cdf3362540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.307+0000 7f88d0713b80 1 --2- 172.21.15.46:0/3891076258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf3362540 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f88b400b420 tx=0x7f88b40093e0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.307+0000 7f88d0713b80 1 -- 172.21.15.46:0/3891076258 shutdown_connections 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.307+0000 7f88d0713b80 1 --2- 172.21.15.46:0/3891076258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf3362540 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.307+0000 7f88d0713b80 1 --2- 172.21.15.46:0/3891076258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334ba60 0x55cdf33527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.307+0000 7f88d0713b80 1 --2- 172.21.15.46:0/3891076258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334c380 0x55cdf3352d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.307+0000 7f88d0713b80 1 -- 172.21.15.46:0/3891076258 >> 172.21.15.46:0/3891076258 conn(0x55cdf32c2220 msgr2=0x55cdf334f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.307+0000 7f88d0713b80 1 -- 172.21.15.46:0/3891076258 shutdown_connections 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.307+0000 7f88d0713b80 1 -- 172.21.15.46:0/3891076258 wait complete. 2024-03-18T20:39:47.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.307+0000 7f88d0713b80 1 -- start start 2024-03-18T20:39:47.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.307+0000 7f88d0713b80 1 Processor -- start 2024-03-18T20:39:47.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88d0713b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334ba60 0x55cdf335d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88d0713b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334c380 0x55cdf335dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88d0713b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf335bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88d0713b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf335c2d0 con 0x55cdf3357220 2024-03-18T20:39:47.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88d0713b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf335c440 con 0x55cdf334ba60 2024-03-18T20:39:47.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88d0713b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdf335c600 con 0x55cdf334c380 2024-03-18T20:39:47.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88d0713b80 10 client.0 _ll_register_callbacks cb 0x7f88ac01e3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:47.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88c2b1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf335bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88c2b1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf335bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51850/0 (socket says 172.21.15.46:51850) 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88c2b1b700 1 -- 172.21.15.46:0/643978150 learned_addr learned my addr 172.21.15.46:0/643978150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88c231a700 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334ba60 0x55cdf335d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88c1b19700 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334c380 0x55cdf335dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88c2b1b700 1 -- 172.21.15.46:0/643978150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334c380 msgr2=0x55cdf335dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88c2b1b700 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334c380 0x55cdf335dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88c2b1b700 1 -- 172.21.15.46:0/643978150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334ba60 msgr2=0x55cdf335d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88c2b1b700 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334ba60 0x55cdf335d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88c2b1b700 1 -- 172.21.15.46:0/643978150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdf31203b0 con 0x55cdf3357220 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88c2b1b700 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf335bd00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f88b40074e0 tx=0x7f88b4003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab40a4cb9506822d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.308+0000 7f88a5ffb700 1 -- 172.21.15.46:0/643978150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400e040 con 0x55cdf3357220 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88a5ffb700 1 -- 172.21.15.46:0/643978150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f88b400bd20 con 0x55cdf3357220 2024-03-18T20:39:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88d0713b80 1 -- 172.21.15.46:0/643978150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88b40060c0 con 0x55cdf3357220 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88a5ffb700 1 -- 172.21.15.46:0/643978150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88b4007b00 con 0x55cdf3357220 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88d0713b80 10 client.16770 fetch_fsmap learned FSMap version 37 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88d0713b80 1 -- 172.21.15.46:0/643978150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55cdf3124cd0 con 0x55cdf3357220 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88a5ffb700 1 -- 172.21.15.46:0/643978150 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88b40060c0 con 0x55cdf3357220 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88a5ffb700 1 -- 172.21.15.46:0/643978150 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 37) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f88b400b100 con 0x55cdf3357220 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88d0713b80 10 client.16770 fetch_fsmap finished waiting for FSMap version 37 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88d0713b80 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88d0713b80 20 client.16770 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88d0713b80 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55cdf33bbad0 0x55cdf33bdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88d0713b80 4 client.16770 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88d0713b80 1 -- 172.21.15.46:0/643978150 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cdf30edcf0 con 0x55cdf33bbad0 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.309+0000 7f88c231a700 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55cdf33bbad0 0x55cdf33bdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88a5ffb700 10 client.16770 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88c231a700 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55cdf33bbad0 0x55cdf33bdef0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3498ceb5662b834 server_cookie=7fabebc461597f81 in_seq=0 out_seq=0 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88a5ffb700 1 -- 172.21.15.46:0/643978150 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cdf30edcf0 con 0x55cdf33bbad0 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88a5ffb700 10 client.16770 handle_command_reply: tid=0 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88d0713b80 1 client.16770 shutdown 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88d0713b80 1 -- 172.21.15.46:0/643978150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 msgr2=0x55cdf335bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88d0713b80 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf335bd00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f88b40074e0 tx=0x7f88b4003da0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88d0713b80 1 -- 172.21.15.46:0/643978150 shutdown_connections 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88d0713b80 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55cdf33bbad0 0x55cdf33bdef0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88d0713b80 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdf3357220 0x55cdf335bd00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88d0713b80 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdf334ba60 0x55cdf335d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88d0713b80 1 --2- 172.21.15.46:0/643978150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdf334c380 0x55cdf335dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88a5ffb700 0 client.16770 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88d0713b80 1 -- 172.21.15.46:0/643978150 >> 172.21.15.46:0/643978150 conn(0x55cdf32c2220 msgr2=0x55cdf3352420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88d0713b80 1 -- 172.21.15.46:0/643978150 shutdown_connections 2024-03-18T20:39:47.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88d0713b80 1 -- 172.21.15.46:0/643978150 wait complete. 2024-03-18T20:39:47.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.310+0000 7f88d0713b80 20 client.16770 trim_cache size 0 max 16384 2024-03-18T20:39:47.320 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:39:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:39:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:39:47.320 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:39:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.311+0000 7f88c457e700 1 -- 172.21.15.46:0/1585305995 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88a80a1290 msgr2=0x7f88a80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.311+0000 7f88c457e700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88a80a1290 0x7f88a80a3750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f88bc198850 tx=0x7f88ac009260 comp rx=0 tx=0).stop 2024-03-18T20:39:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.311+0000 7f88c457e700 1 -- 172.21.15.46:0/1585305995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88bc069540 msgr2=0x7f88bc197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.311+0000 7f88c457e700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88bc069540 0x7f88bc197300 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f88b800e9a0 tx=0x7f88b800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.311+0000 7f88c457e700 1 -- 172.21.15.46:0/1585305995 shutdown_connections 2024-03-18T20:39:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.311+0000 7f88c457e700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88a80a1290 0x7f88a80a3750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.311+0000 7f88c457e700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88bc069540 0x7f88bc197300 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.311+0000 7f88c457e700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88bc19bbe0 0x7f88bc19c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.311+0000 7f88c457e700 1 --2- 172.21.15.46:0/1585305995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88bc1060a0 0x7f88bc197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.311+0000 7f88c457e700 1 -- 172.21.15.46:0/1585305995 >> 172.21.15.46:0/1585305995 conn(0x7f88bc076d50 msgr2=0x7f88bc0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.311+0000 7f88c457e700 1 -- 172.21.15.46:0/1585305995 shutdown_connections 2024-03-18T20:39:47.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.311+0000 7f88c457e700 1 -- 172.21.15.46:0/1585305995 wait complete. 2024-03-18T20:39:47.327 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:39:47.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:39:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.463+0000 7f807c6e8700 1 -- 172.21.15.46:0/4291568530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8074105c10 msgr2=0x7f8074105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.463+0000 7f807c6e8700 1 --2- 172.21.15.46:0/4291568530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8074105c10 0x7f8074105ff0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8070009990 tx=0x7f8070009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.463+0000 7f807c6e8700 1 -- 172.21.15.46:0/4291568530 shutdown_connections 2024-03-18T20:39:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.463+0000 7f807c6e8700 1 --2- 172.21.15.46:0/4291568530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8074105c10 0x7f8074105ff0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.463+0000 7f807c6e8700 1 --2- 172.21.15.46:0/4291568530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80740ffaf0 0x7f807410dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.463+0000 7f807c6e8700 1 --2- 172.21.15.46:0/4291568530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ff130 0x7f80740ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.463+0000 7f807c6e8700 1 -- 172.21.15.46:0/4291568530 >> 172.21.15.46:0/4291568530 conn(0x7f8074076050 msgr2=0x7f80740fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.463+0000 7f807c6e8700 1 -- 172.21.15.46:0/4291568530 shutdown_connections 2024-03-18T20:39:47.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.463+0000 7f807c6e8700 1 -- 172.21.15.46:0/4291568530 wait complete. 2024-03-18T20:39:47.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.464+0000 7f807c6e8700 1 Processor -- start 2024-03-18T20:39:47.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.464+0000 7f807c6e8700 1 -- start start 2024-03-18T20:39:47.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.464+0000 7f807c6e8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80740ff130 0x7f8074197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.464+0000 7f807c6e8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ffaf0 0x7f80741976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.464+0000 7f807c6e8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8074105c10 0x7f807419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.464+0000 7f807c6e8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807419c0b0 con 0x7f80740ff130 2024-03-18T20:39:47.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.464+0000 7f807c6e8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807419c220 con 0x7f8074105c10 2024-03-18T20:39:47.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.464+0000 7f807c6e8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807419c3e0 con 0x7f80740ffaf0 2024-03-18T20:39:47.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807a484700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80740ff130 0x7f8074197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807a484700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80740ff130 0x7f8074197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51856/0 (socket says 172.21.15.46:51856) 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807a484700 1 -- 172.21.15.46:0/1070811139 learned_addr learned my addr 172.21.15.46:0/1070811139 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f8079c83700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ffaf0 0x7f80741976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807ac85700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8074105c10 0x7f807419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807a484700 1 -- 172.21.15.46:0/1070811139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ffaf0 msgr2=0x7f80741976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807a484700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ffaf0 0x7f80741976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807a484700 1 -- 172.21.15.46:0/1070811139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8074105c10 msgr2=0x7f807419ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807a484700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8074105c10 0x7f807419ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807a484700 1 -- 172.21.15.46:0/1070811139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80700095f0 con 0x7f80740ff130 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807ac85700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8074105c10 0x7f807419ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807a484700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80740ff130 0x7f8074197190 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8070000c00 tx=0x7f8070005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f806b7fe700 1 -- 172.21.15.46:0/1070811139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807001d070 con 0x7f80740ff130 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f806b7fe700 1 -- 172.21.15.46:0/1070811139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8070022470 con 0x7f80740ff130 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807c6e8700 1 -- 172.21.15.46:0/1070811139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807419c660 con 0x7f80740ff130 2024-03-18T20:39:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f806b7fe700 1 -- 172.21.15.46:0/1070811139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070017650 con 0x7f80740ff130 2024-03-18T20:39:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.465+0000 7f807c6e8700 1 -- 172.21.15.46:0/1070811139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80741a3f00 con 0x7f80740ff130 2024-03-18T20:39:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.466+0000 7f807c6e8700 1 -- 172.21.15.46:0/1070811139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8074109180 con 0x7f80740ff130 2024-03-18T20:39:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.467+0000 7f806b7fe700 1 -- 172.21.15.46:0/1070811139 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f80700225e0 con 0x7f80740ff130 2024-03-18T20:39:47.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.468+0000 7f806b7fe700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f80600a1290 0x7f80600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.468+0000 7f8079c83700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f80600a1290 0x7f80600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.468+0000 7f806b7fe700 1 -- 172.21.15.46:0/1070811139 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f80700ce5f0 con 0x7f80740ff130 2024-03-18T20:39:47.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.469+0000 7f8079c83700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f80600a1290 0x7f80600a3750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f80741986e0 tx=0x7f8064009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.472+0000 7f806b7fe700 1 -- 172.21.15.46:0/1070811139 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8070097ae0 con 0x7f80740ff130 2024-03-18T20:39:47.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 92992 1.37 MB/sec execute 42 sec latency 3329.904 ms 2024-03-18T20:39:47.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.740+0000 7f807c6e8700 1 -- 172.21.15.46:0/1070811139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8074067e60 con 0x7f80740ff130 2024-03-18T20:39:47.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.741+0000 7f806b7fe700 1 -- 172.21.15.46:0/1070811139 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+10912 (secure 0 0 0) 0x7f807009b100 con 0x7f80740ff130 2024-03-18T20:39:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:39:47.744 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:39:47.074504+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:39:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.744+0000 7f807c6e8700 1 -- 172.21.15.46:0/1070811139 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f80600a1290 msgr2=0x7f80600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.744+0000 7f807c6e8700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f80600a1290 0x7f80600a3750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f80741986e0 tx=0x7f8064009260 comp rx=0 tx=0).stop 2024-03-18T20:39:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.744+0000 7f807c6e8700 1 -- 172.21.15.46:0/1070811139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80740ff130 msgr2=0x7f8074197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.744+0000 7f807c6e8700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80740ff130 0x7f8074197190 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8070000c00 tx=0x7f8070005ef0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.744+0000 7f807c6e8700 1 -- 172.21.15.46:0/1070811139 shutdown_connections 2024-03-18T20:39:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.744+0000 7f807c6e8700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f80600a1290 0x7f80600a3750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.744+0000 7f807c6e8700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80740ff130 0x7f8074197190 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.744+0000 7f807c6e8700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8074105c10 0x7f807419ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.744+0000 7f807c6e8700 1 --2- 172.21.15.46:0/1070811139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80740ffaf0 0x7f80741976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.744+0000 7f807c6e8700 1 -- 172.21.15.46:0/1070811139 >> 172.21.15.46:0/1070811139 conn(0x7f8074076050 msgr2=0x7f8074103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.744+0000 7f807c6e8700 1 -- 172.21.15.46:0/1070811139 shutdown_connections 2024-03-18T20:39:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.744+0000 7f807c6e8700 1 -- 172.21.15.46:0/1070811139 wait complete. 2024-03-18T20:39:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 37 2024-03-18T20:39:47.760 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:39:47.761 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:39:47.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.897+0000 7f03fc063700 1 -- 172.21.15.46:0/4180929045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03f4104f90 msgr2=0x7f03f410cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.897+0000 7f03fc063700 1 --2- 172.21.15.46:0/4180929045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03f4104f90 0x7f03f410cb10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00b0a0 tx=0x7f03ec00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.897+0000 7f03fc063700 1 -- 172.21.15.46:0/4180929045 shutdown_connections 2024-03-18T20:39:47.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.897+0000 7f03fc063700 1 --2- 172.21.15.46:0/4180929045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03f4104f90 0x7f03f410cb10 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.897+0000 7f03fc063700 1 --2- 172.21.15.46:0/4180929045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f40ffe70 0x7f03f4100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.897+0000 7f03fc063700 1 --2- 172.21.15.46:0/4180929045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03f4100820 0x7f03f4104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.897+0000 7f03fc063700 1 -- 172.21.15.46:0/4180929045 >> 172.21.15.46:0/4180929045 conn(0x7f03f4076d80 msgr2=0x7f03f4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.897+0000 7f03fc063700 1 -- 172.21.15.46:0/4180929045 shutdown_connections 2024-03-18T20:39:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.897+0000 7f03fc063700 1 -- 172.21.15.46:0/4180929045 wait complete. 2024-03-18T20:39:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03fc063700 1 Processor -- start 2024-03-18T20:39:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03fc063700 1 -- start start 2024-03-18T20:39:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03fc063700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03f40ffe70 0x7f03f4197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03fc063700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03f4104f90 0x7f03f41978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03fc063700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f419bc50 0x7f03f419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03fc063700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f419c6d0 con 0x7f03f4104f90 2024-03-18T20:39:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03fc063700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f419c840 con 0x7f03f419bc50 2024-03-18T20:39:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03fc063700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f419ca00 con 0x7f03f40ffe70 2024-03-18T20:39:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03f95fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03f4104f90 0x7f03f41978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03f95fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03f4104f90 0x7f03f41978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51880/0 (socket says 172.21.15.46:51880) 2024-03-18T20:39:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03f95fe700 1 -- 172.21.15.46:0/1691610969 learned_addr learned my addr 172.21.15.46:0/1691610969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03f9dff700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03f40ffe70 0x7f03f4197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.898+0000 7f03fa600700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f419bc50 0x7f03f419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.899+0000 7f03f95fe700 1 -- 172.21.15.46:0/1691610969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03f40ffe70 msgr2=0x7f03f4197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.899+0000 7f03f95fe700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03f40ffe70 0x7f03f4197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.899+0000 7f03f95fe700 1 -- 172.21.15.46:0/1691610969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f419bc50 msgr2=0x7f03f419c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.899+0000 7f03f95fe700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f419bc50 0x7f03f419c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.899+0000 7f03f95fe700 1 -- 172.21.15.46:0/1691610969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec009d00 con 0x7f03f4104f90 2024-03-18T20:39:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.899+0000 7f03fa600700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f419bc50 0x7f03f419c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.899+0000 7f03f95fe700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03f4104f90 0x7f03f41978b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f03e400d950 tx=0x7f03e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.899+0000 7f03eaffd700 1 -- 172.21.15.46:0/1691610969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e4004220 con 0x7f03f4104f90 2024-03-18T20:39:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.899+0000 7f03eaffd700 1 -- 172.21.15.46:0/1691610969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03e4007620 con 0x7f03f4104f90 2024-03-18T20:39:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.899+0000 7f03fc063700 1 -- 172.21.15.46:0/1691610969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f41a3f60 con 0x7f03f4104f90 2024-03-18T20:39:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.899+0000 7f03eaffd700 1 -- 172.21.15.46:0/1691610969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e4005690 con 0x7f03f4104f90 2024-03-18T20:39:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.899+0000 7f03fc063700 1 -- 172.21.15.46:0/1691610969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f41a43e0 con 0x7f03f4104f90 2024-03-18T20:39:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.902+0000 7f03eaffd700 1 -- 172.21.15.46:0/1691610969 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f03e4004750 con 0x7f03f4104f90 2024-03-18T20:39:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.903+0000 7f03eaffd700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f03e00a1290 0x7f03e00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.903+0000 7f03eaffd700 1 -- 172.21.15.46:0/1691610969 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f03e40cca20 con 0x7f03f4104f90 2024-03-18T20:39:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.903+0000 7f03f9dff700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f03e00a1290 0x7f03e00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.903+0000 7f03f9dff700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f03e00a1290 0x7f03e00a3750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009c60 tx=0x7f03f0009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.904+0000 7f04081f8b80 1 Processor -- start 2024-03-18T20:39:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.904+0000 7f04081f8b80 1 -- start start 2024-03-18T20:39:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.904+0000 7f04081f8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57ea60 0x564f1c57ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.904+0000 7f04081f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57f380 0x564f1c589ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.904+0000 7f04081f8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c58a220 0x564f1c58c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.904+0000 7f04081f8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1c4d8990 con 0x564f1c57ea60 2024-03-18T20:39:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.904+0000 7f04081f8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1c4d6350 con 0x564f1c58a220 2024-03-18T20:39:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.904+0000 7f04081f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1c4d1c40 con 0x564f1c57f380 2024-03-18T20:39:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.904+0000 7f03f9dff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57ea60 0x564f1c57ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.904+0000 7f03f9dff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57ea60 0x564f1c57ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51886/0 (socket says 172.21.15.46:51886) 2024-03-18T20:39:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.904+0000 7f03f9dff700 1 -- 172.21.15.46:0/3834374142 learned_addr learned my addr 172.21.15.46:0/3834374142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03f95fe700 1 --2- 172.21.15.46:0/3834374142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57f380 0x564f1c589ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03fa600700 1 --2- 172.21.15.46:0/3834374142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c58a220 0x564f1c58c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03f9dff700 1 -- 172.21.15.46:0/3834374142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57f380 msgr2=0x564f1c589ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03f9dff700 1 --2- 172.21.15.46:0/3834374142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57f380 0x564f1c589ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03f9dff700 1 -- 172.21.15.46:0/3834374142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c58a220 msgr2=0x564f1c58c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03f9dff700 1 --2- 172.21.15.46:0/3834374142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c58a220 0x564f1c58c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03f9dff700 1 -- 172.21.15.46:0/3834374142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f1c3533b0 con 0x564f1c57ea60 2024-03-18T20:39:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03f95fe700 1 --2- 172.21.15.46:0/3834374142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57f380 0x564f1c589ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03fa600700 1 --2- 172.21.15.46:0/3834374142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c58a220 0x564f1c58c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:39:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03f9dff700 1 --2- 172.21.15.46:0/3834374142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57ea60 0x564f1c57ee40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f03f000e4e0 tx=0x7f03f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b526dc01432efc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03dffff700 1 -- 172.21.15.46:0/3834374142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0026430 con 0x564f1c57ea60 2024-03-18T20:39:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03dffff700 1 -- 172.21.15.46:0/3834374142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03f0021070 con 0x564f1c57ea60 2024-03-18T20:39:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.905+0000 7f03dffff700 1 -- 172.21.15.46:0/3834374142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f001d740 con 0x564f1c57ea60 2024-03-18T20:39:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.906+0000 7f04081f8b80 1 -- 172.21.15.46:0/3834374142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57ea60 msgr2=0x564f1c57ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.906+0000 7f04081f8b80 1 --2- 172.21.15.46:0/3834374142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57ea60 0x564f1c57ee40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f03f000e4e0 tx=0x7f03f0014000 comp rx=0 tx=0).stop 2024-03-18T20:39:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.906+0000 7f04081f8b80 1 -- 172.21.15.46:0/3834374142 shutdown_connections 2024-03-18T20:39:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.906+0000 7f04081f8b80 1 --2- 172.21.15.46:0/3834374142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57ea60 0x564f1c57ee40 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.906+0000 7f04081f8b80 1 --2- 172.21.15.46:0/3834374142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c58a220 0x564f1c58c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.906+0000 7f04081f8b80 1 --2- 172.21.15.46:0/3834374142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57f380 0x564f1c589ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.906+0000 7f04081f8b80 1 -- 172.21.15.46:0/3834374142 >> 172.21.15.46:0/3834374142 conn(0x564f1c4f5220 msgr2=0x564f1c57b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.906+0000 7f04081f8b80 1 -- 172.21.15.46:0/3834374142 shutdown_connections 2024-03-18T20:39:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.906+0000 7f04081f8b80 1 -- 172.21.15.46:0/3834374142 wait complete. 2024-03-18T20:39:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f04081f8b80 1 -- start start 2024-03-18T20:39:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f04081f8b80 1 Processor -- start 2024-03-18T20:39:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f04081f8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57ea60 0x564f1c589ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f04081f8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57f380 0x564f1c586fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f04081f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c58a220 0x564f1c5876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f04081f8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1c46dd80 con 0x564f1c57f380 2024-03-18T20:39:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f04081f8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1c46da20 con 0x564f1c57ea60 2024-03-18T20:39:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f04081f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1c3556a0 con 0x564f1c58a220 2024-03-18T20:39:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f04081f8b80 10 client.0 _ll_register_callbacks cb 0x564f1c2d93a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f03f95fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57f380 0x564f1c586fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f03f95fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57f380 0x564f1c586fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51896/0 (socket says 172.21.15.46:51896) 2024-03-18T20:39:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f03f95fe700 1 -- 172.21.15.46:0/1440007077 learned_addr learned my addr 172.21.15.46:0/1440007077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.907+0000 7f03f9dff700 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57ea60 0x564f1c589ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f03fa600700 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c58a220 0x564f1c5876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f03f95fe700 1 -- 172.21.15.46:0/1440007077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c58a220 msgr2=0x564f1c5876d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f03f95fe700 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c58a220 0x564f1c5876d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f03f95fe700 1 -- 172.21.15.46:0/1440007077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57ea60 msgr2=0x564f1c589ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f03f95fe700 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57ea60 0x564f1c589ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f03f95fe700 1 -- 172.21.15.46:0/1440007077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f0017070 con 0x564f1c57f380 2024-03-18T20:39:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f03f9dff700 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57ea60 0x564f1c589ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f03f95fe700 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57f380 0x564f1c586fe0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f03e4096000 tx=0x7f03e409a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09eadea20dfb596 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f03dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f03dd7fa700 1 -- 172.21.15.46:0/1440007077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e40a1f80 con 0x564f1c57f380 2024-03-18T20:39:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f03dd7fa700 1 -- 172.21.15.46:0/1440007077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f03e4099d60 con 0x564f1c57f380 2024-03-18T20:39:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f04081f8b80 1 -- 172.21.15.46:0/1440007077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f1c572ad0 con 0x564f1c57f380 2024-03-18T20:39:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f03dd7fa700 1 -- 172.21.15.46:0/1440007077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03e4099f20 con 0x564f1c57f380 2024-03-18T20:39:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f04081f8b80 10 client.16788 fetch_fsmap learned FSMap version 37 2024-03-18T20:39:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.908+0000 7f04081f8b80 1 -- 172.21.15.46:0/1440007077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x564f1c2ffb60 con 0x564f1c57f380 2024-03-18T20:39:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.909+0000 7f03dd7fa700 1 -- 172.21.15.46:0/1440007077 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f1c572ad0 con 0x564f1c57f380 2024-03-18T20:39:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.909+0000 7f03dd7fa700 1 -- 172.21.15.46:0/1440007077 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 37) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f03e40a1bf0 con 0x564f1c57f380 2024-03-18T20:39:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.909+0000 7f04081f8b80 10 client.16788 fetch_fsmap finished waiting for FSMap version 37 2024-03-18T20:39:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.909+0000 7f04081f8b80 10 client.16788 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:39:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.909+0000 7f04081f8b80 20 client.16788 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.909+0000 7f04081f8b80 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x564f1c599d90 0x564f1c59c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.909+0000 7f04081f8b80 4 client.16788 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.909+0000 7f04081f8b80 1 -- 172.21.15.46:0/1440007077 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f1c4ca700 con 0x564f1c599d90 2024-03-18T20:39:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.909+0000 7f03f9dff700 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x564f1c599d90 0x564f1c59c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.910+0000 7f03dd7fa700 10 client.16788 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:39:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.910+0000 7f03f9dff700 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x564f1c599d90 0x564f1c59c1b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4a126788d44e1988 server_cookie=53df0a66e0cd1367 in_seq=0 out_seq=0 2024-03-18T20:39:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.910+0000 7f03dd7fa700 1 -- 172.21.15.46:0/1440007077 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564f1c4ca700 con 0x564f1c599d90 2024-03-18T20:39:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.910+0000 7f03dd7fa700 10 client.16788 handle_command_reply: tid=0 2024-03-18T20:39:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.910+0000 7f04081f8b80 1 client.16788 shutdown 2024-03-18T20:39:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.910+0000 7f04081f8b80 1 -- 172.21.15.46:0/1440007077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57f380 msgr2=0x564f1c586fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.910+0000 7f04081f8b80 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57f380 0x564f1c586fe0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f03e4096000 tx=0x7f03e409a280 comp rx=0 tx=0).stop 2024-03-18T20:39:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.910+0000 7f04081f8b80 1 -- 172.21.15.46:0/1440007077 shutdown_connections 2024-03-18T20:39:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.910+0000 7f04081f8b80 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x564f1c599d90 0x564f1c59c1b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.910+0000 7f04081f8b80 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c58a220 0x564f1c5876d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.910+0000 7f03dd7fa700 0 client.16788 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:39:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.910+0000 7f04081f8b80 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57ea60 0x564f1c589ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.911+0000 7f04081f8b80 1 --2- 172.21.15.46:0/1440007077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c57f380 0x564f1c586fe0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.911+0000 7f04081f8b80 1 -- 172.21.15.46:0/1440007077 >> 172.21.15.46:0/1440007077 conn(0x564f1c4f5220 msgr2=0x564f1c57ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.911+0000 7f04081f8b80 1 -- 172.21.15.46:0/1440007077 shutdown_connections 2024-03-18T20:39:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.911+0000 7f04081f8b80 1 -- 172.21.15.46:0/1440007077 wait complete. 2024-03-18T20:39:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.911+0000 7f04081f8b80 20 client.16788 trim_cache size 0 max 16384 2024-03-18T20:39:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.924+0000 7f04081f8b80 1 Processor -- start 2024-03-18T20:39:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.924+0000 7f04081f8b80 1 -- start start 2024-03-18T20:39:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f04081f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57ea60 0x564f1c581a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f04081f8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57f380 0x564f1c590e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f04081f8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 0x564f1c586870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f04081f8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1c57c440 con 0x564f1c58a220 2024-03-18T20:39:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f04081f8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1c588590 con 0x564f1c57f380 2024-03-18T20:39:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f04081f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1c57fd30 con 0x564f1c57ea60 2024-03-18T20:39:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f03fa600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 0x564f1c586870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f03fa600700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 0x564f1c586870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51904/0 (socket says 172.21.15.46:51904) 2024-03-18T20:39:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f03fa600700 1 -- 172.21.15.46:0/1736286749 learned_addr learned my addr 172.21.15.46:0/1736286749 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f03f9dff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57ea60 0x564f1c581a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f03f95fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57f380 0x564f1c590e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f03fa600700 1 -- 172.21.15.46:0/1736286749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57ea60 msgr2=0x564f1c581a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f03fa600700 1 --2- 172.21.15.46:0/1736286749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57ea60 0x564f1c581a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f03fa600700 1 -- 172.21.15.46:0/1736286749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57f380 msgr2=0x564f1c590e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f03fa600700 1 --2- 172.21.15.46:0/1736286749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57f380 0x564f1c590e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f03fa600700 1 -- 172.21.15.46:0/1736286749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f1c497ea0 con 0x564f1c58a220 2024-03-18T20:39:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f03fa600700 1 --2- 172.21.15.46:0/1736286749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 0x564f1c586870 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f03ec015000 tx=0x7f03ec009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc644206c2c8800d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.925+0000 7f03e8ff9700 1 -- 172.21.15.46:0/1736286749 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00e040 con 0x564f1c58a220 2024-03-18T20:39:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.926+0000 7f03e8ff9700 1 -- 172.21.15.46:0/1736286749 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03ec021070 con 0x564f1c58a220 2024-03-18T20:39:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.926+0000 7f03e8ff9700 1 -- 172.21.15.46:0/1736286749 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec004cb0 con 0x564f1c58a220 2024-03-18T20:39:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.926+0000 7f04081f8b80 1 -- 172.21.15.46:0/1736286749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 msgr2=0x564f1c586870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.926+0000 7f04081f8b80 1 --2- 172.21.15.46:0/1736286749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 0x564f1c586870 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f03ec015000 tx=0x7f03ec009390 comp rx=0 tx=0).stop 2024-03-18T20:39:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.926+0000 7f04081f8b80 1 -- 172.21.15.46:0/1736286749 shutdown_connections 2024-03-18T20:39:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.926+0000 7f04081f8b80 1 --2- 172.21.15.46:0/1736286749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 0x564f1c586870 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.926+0000 7f04081f8b80 1 --2- 172.21.15.46:0/1736286749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57ea60 0x564f1c581a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.926+0000 7f04081f8b80 1 --2- 172.21.15.46:0/1736286749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57f380 0x564f1c590e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.926+0000 7f04081f8b80 1 -- 172.21.15.46:0/1736286749 >> 172.21.15.46:0/1736286749 conn(0x564f1c4f5220 msgr2=0x564f1c582320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.926+0000 7f04081f8b80 1 -- 172.21.15.46:0/1736286749 shutdown_connections 2024-03-18T20:39:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.926+0000 7f04081f8b80 1 -- 172.21.15.46:0/1736286749 wait complete. 2024-03-18T20:39:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.926+0000 7f04081f8b80 1 -- start start 2024-03-18T20:39:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f04081f8b80 1 Processor -- start 2024-03-18T20:39:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f04081f8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57ea60 0x564f1c58ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f04081f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57f380 0x564f1c58f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f04081f8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 0x564f1c58f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f04081f8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1c5f0fd0 con 0x564f1c58a220 2024-03-18T20:39:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f04081f8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1c5f1140 con 0x564f1c57ea60 2024-03-18T20:39:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f04081f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1c5f1300 con 0x564f1c57f380 2024-03-18T20:39:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f04081f8b80 10 client.0 _ll_register_callbacks cb 0x7f03e4099660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f03f95fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57f380 0x564f1c58f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f03f9dff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57ea60 0x564f1c58ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f03f95fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57f380 0x564f1c58f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37352/0 (socket says 172.21.15.46:37352) 2024-03-18T20:39:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f03f95fe700 1 -- 172.21.15.46:0/4060281280 learned_addr learned my addr 172.21.15.46:0/4060281280 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.927+0000 7f03fa600700 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 0x564f1c58f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.928+0000 7f03f9dff700 1 -- 172.21.15.46:0/4060281280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57f380 msgr2=0x564f1c58f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.928+0000 7f03f9dff700 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57f380 0x564f1c58f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.928+0000 7f03f9dff700 1 -- 172.21.15.46:0/4060281280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 msgr2=0x564f1c58f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.928+0000 7f03f9dff700 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 0x564f1c58f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.928+0000 7f03f9dff700 1 -- 172.21.15.46:0/4060281280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f410d380 con 0x564f1c57ea60 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.928+0000 7f03fa600700 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 0x564f1c58f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.928+0000 7f03f9dff700 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57ea60 0x564f1c58ee60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f03f0011ab0 tx=0x7f03f000ebb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d36d555a1051ce4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.928+0000 7f03dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.928+0000 7f03dd7fa700 1 -- 172.21.15.46:0/4060281280 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0033a90 con 0x564f1c57ea60 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f04081f8b80 1 -- 172.21.15.46:0/4060281280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03f0021070 con 0x564f1c57ea60 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f03dd7fa700 1 -- 172.21.15.46:0/4060281280 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f03f0033bf0 con 0x564f1c57ea60 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f03dd7fa700 1 -- 172.21.15.46:0/4060281280 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03f0033db0 con 0x564f1c57ea60 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f04081f8b80 10 client.26525 fetch_fsmap learned FSMap version 37 2024-03-18T20:39:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f04081f8b80 1 -- 172.21.15.46:0/4060281280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x564f1c357cd0 con 0x564f1c57ea60 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f03dd7fa700 1 -- 172.21.15.46:0/4060281280 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03f0021070 con 0x564f1c57ea60 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f03dd7fa700 1 -- 172.21.15.46:0/4060281280 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 37) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f03f003a060 con 0x564f1c57ea60 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f04081f8b80 10 client.26525 fetch_fsmap finished waiting for FSMap version 37 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f04081f8b80 10 client.26525 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f04081f8b80 20 client.26525 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f04081f8b80 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x564f1c599d90 0x564f1c5f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f04081f8b80 4 client.26525 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f04081f8b80 1 -- 172.21.15.46:0/4060281280 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564f1c320cf0 con 0x564f1c599d90 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.929+0000 7f03f95fe700 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x564f1c599d90 0x564f1c5f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f03f95fe700 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x564f1c599d90 0x564f1c5f4ae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e57bfcac4826cb3f server_cookie=37d00f2799ea2d4 in_seq=0 out_seq=0 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f03dd7fa700 10 client.26525 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f03dd7fa700 1 -- 172.21.15.46:0/4060281280 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564f1c320cf0 con 0x564f1c599d90 2024-03-18T20:39:47.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f03dd7fa700 10 client.26525 handle_command_reply: tid=0 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f04081f8b80 1 client.26525 shutdown 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f04081f8b80 1 -- 172.21.15.46:0/4060281280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57ea60 msgr2=0x564f1c58ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f04081f8b80 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57ea60 0x564f1c58ee60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f03f0011ab0 tx=0x7f03f000ebb0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f04081f8b80 1 -- 172.21.15.46:0/4060281280 shutdown_connections 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f04081f8b80 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x564f1c599d90 0x564f1c5f4ae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f04081f8b80 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564f1c58a220 0x564f1c58f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f04081f8b80 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564f1c57ea60 0x564f1c58ee60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f03dd7fa700 0 client.26525 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f04081f8b80 1 --2- 172.21.15.46:0/4060281280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564f1c57f380 0x564f1c58f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f04081f8b80 1 -- 172.21.15.46:0/4060281280 >> 172.21.15.46:0/4060281280 conn(0x564f1c4f5220 msgr2=0x564f1c586870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.930+0000 7f04081f8b80 1 -- 172.21.15.46:0/4060281280 shutdown_connections 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f04081f8b80 1 -- 172.21.15.46:0/4060281280 wait complete. 2024-03-18T20:39:47.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f04081f8b80 20 client.26525 trim_cache size 0 max 16384 2024-03-18T20:39:47.939 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:39:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f03fc063700 1 -- 172.21.15.46:0/1691610969 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f03e00a1290 msgr2=0x7f03e00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f03fc063700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f03e00a1290 0x7f03e00a3750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009c60 tx=0x7f03f0009260 comp rx=0 tx=0).stop 2024-03-18T20:39:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f03fc063700 1 -- 172.21.15.46:0/1691610969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03f4104f90 msgr2=0x7f03f41978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f03fc063700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03f4104f90 0x7f03f41978b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f03e400d950 tx=0x7f03e400dc60 comp rx=0 tx=0).stop 2024-03-18T20:39:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f03fc063700 1 -- 172.21.15.46:0/1691610969 shutdown_connections 2024-03-18T20:39:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f03fc063700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f03e00a1290 0x7f03e00a3750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f03fc063700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f419bc50 0x7f03f419c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f03fc063700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03f40ffe70 0x7f03f4197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f03fc063700 1 --2- 172.21.15.46:0/1691610969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03f4104f90 0x7f03f41978b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f03fc063700 1 -- 172.21.15.46:0/1691610969 >> 172.21.15.46:0/1691610969 conn(0x7f03f4076d80 msgr2=0x7f03f4069350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f03fc063700 1 -- 172.21.15.46:0/1691610969 shutdown_connections 2024-03-18T20:39:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:47.931+0000 7f03fc063700 1 -- 172.21.15.46:0/1691610969 wait complete. 2024-03-18T20:39:47.948 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:39:47.948 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:39:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.093+0000 7fb55aff4700 1 -- 172.21.15.46:0/41568643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554100030 msgr2=0x7fb55410daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.093+0000 7fb55aff4700 1 --2- 172.21.15.46:0/41568643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554100030 0x7fb55410daa0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00b0a0 tx=0x7fb54c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.093+0000 7fb55aff4700 1 -- 172.21.15.46:0/41568643 shutdown_connections 2024-03-18T20:39:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.093+0000 7fb55aff4700 1 --2- 172.21.15.46:0/41568643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554100030 0x7fb55410daa0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.093+0000 7fb55aff4700 1 --2- 172.21.15.46:0/41568643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554105da0 0x7fb554106180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.093+0000 7fb55aff4700 1 --2- 172.21.15.46:0/41568643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5540ff670 0x7fb5540ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.093+0000 7fb55aff4700 1 -- 172.21.15.46:0/41568643 >> 172.21.15.46:0/41568643 conn(0x7fb554074ed0 msgr2=0x7fb5540752e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.093+0000 7fb55aff4700 1 -- 172.21.15.46:0/41568643 shutdown_connections 2024-03-18T20:39:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.093+0000 7fb55aff4700 1 -- 172.21.15.46:0/41568643 wait complete. 2024-03-18T20:39:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.093+0000 7fb55aff4700 1 Processor -- start 2024-03-18T20:39:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.093+0000 7fb55aff4700 1 -- start start 2024-03-18T20:39:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb55aff4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5540ff670 0x7fb554100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb55aff4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554100030 0x7fb554100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb55aff4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554105da0 0x7fb5541012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb55aff4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5541a5640 con 0x7fb554105da0 2024-03-18T20:39:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb55aff4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5541a57b0 con 0x7fb5540ff670 2024-03-18T20:39:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb55aff4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5541a5970 con 0x7fb554100030 2024-03-18T20:39:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb559591700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554105da0 0x7fb5541012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb558d90700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5540ff670 0x7fb554100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb559591700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554105da0 0x7fb5541012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51920/0 (socket says 172.21.15.46:51920) 2024-03-18T20:39:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb559591700 1 -- 172.21.15.46:0/923947371 learned_addr learned my addr 172.21.15.46:0/923947371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb558d90700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5540ff670 0x7fb554100850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40140/0 (socket says 172.21.15.46:40140) 2024-03-18T20:39:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb54bfff700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554100030 0x7fb554100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb559591700 1 -- 172.21.15.46:0/923947371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554100030 msgr2=0x7fb554100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb559591700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554100030 0x7fb554100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb559591700 1 -- 172.21.15.46:0/923947371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5540ff670 msgr2=0x7fb554100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb559591700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5540ff670 0x7fb554100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb559591700 1 -- 172.21.15.46:0/923947371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c009d00 con 0x7fb554105da0 2024-03-18T20:39:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb558d90700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5540ff670 0x7fb554100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb559591700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554105da0 0x7fb5541012d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00b070 tx=0x7fb54c0078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb549ffb700 1 -- 172.21.15.46:0/923947371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00e040 con 0x7fb554105da0 2024-03-18T20:39:48.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.094+0000 7fb549ffb700 1 -- 172.21.15.46:0/923947371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54c01d470 con 0x7fb554105da0 2024-03-18T20:39:48.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.095+0000 7fb55aff4700 1 -- 172.21.15.46:0/923947371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5541a5bf0 con 0x7fb554105da0 2024-03-18T20:39:48.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.095+0000 7fb549ffb700 1 -- 172.21.15.46:0/923947371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0052a0 con 0x7fb554105da0 2024-03-18T20:39:48.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.095+0000 7fb55aff4700 1 -- 172.21.15.46:0/923947371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5541a60c0 con 0x7fb554105da0 2024-03-18T20:39:48.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.098+0000 7fb549ffb700 1 -- 172.21.15.46:0/923947371 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fb54c01d5e0 con 0x7fb554105da0 2024-03-18T20:39:48.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.098+0000 7fb549ffb700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb53c0a1310 0x7fb53c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.099+0000 7fb549ffb700 1 -- 172.21.15.46:0/923947371 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb54c0cdf30 con 0x7fb554105da0 2024-03-18T20:39:48.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.099+0000 7fb558d90700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb53c0a1310 0x7fb53c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.099+0000 7fb558d90700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb53c0a1310 0x7fb53c0a37d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb554076450 tx=0x7fb550009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.099+0000 7fb567189b80 1 Processor -- start 2024-03-18T20:39:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.099+0000 7fb567189b80 1 -- start start 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.099+0000 7fb567189b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 0x55b293708e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.099+0000 7fb567189b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293709380 0x55b293713ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.099+0000 7fb567189b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293716610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.099+0000 7fb567189b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b293662990 con 0x55b293708a60 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.099+0000 7fb567189b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b293660350 con 0x55b293709380 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.099+0000 7fb567189b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29365bc40 con 0x55b293714220 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb558d90700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 0x55b293708e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb558d90700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 0x55b293708e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51928/0 (socket says 172.21.15.46:51928) 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb558d90700 1 -- 172.21.15.46:0/1780782110 learned_addr learned my addr 172.21.15.46:0/1780782110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb559591700 1 --2- 172.21.15.46:0/1780782110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293716610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb54bfff700 1 --2- 172.21.15.46:0/1780782110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293709380 0x55b293713ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb558d90700 1 -- 172.21.15.46:0/1780782110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 msgr2=0x55b293716610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb558d90700 1 --2- 172.21.15.46:0/1780782110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293716610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb558d90700 1 -- 172.21.15.46:0/1780782110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293709380 msgr2=0x55b293713ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb558d90700 1 --2- 172.21.15.46:0/1780782110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293709380 0x55b293713ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb558d90700 1 -- 172.21.15.46:0/1780782110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2934dd3b0 con 0x55b293708a60 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb558d90700 1 --2- 172.21.15.46:0/1780782110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 0x55b293708e40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb55000e720 tx=0x7fb550014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361df2a5f7d3d0cb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb53affd700 1 -- 172.21.15.46:0/1780782110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550026600 con 0x55b293708a60 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb53affd700 1 -- 172.21.15.46:0/1780782110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb550026c20 con 0x55b293708a60 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.100+0000 7fb53affd700 1 -- 172.21.15.46:0/1780782110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55001ec10 con 0x55b293708a60 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.101+0000 7fb567189b80 1 -- 172.21.15.46:0/1780782110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 msgr2=0x55b293708e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.101+0000 7fb567189b80 1 --2- 172.21.15.46:0/1780782110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 0x55b293708e40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb55000e720 tx=0x7fb550014000 comp rx=0 tx=0).stop 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.101+0000 7fb567189b80 1 -- 172.21.15.46:0/1780782110 shutdown_connections 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.101+0000 7fb567189b80 1 --2- 172.21.15.46:0/1780782110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 0x55b293708e40 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.101+0000 7fb567189b80 1 --2- 172.21.15.46:0/1780782110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293716610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.101+0000 7fb567189b80 1 --2- 172.21.15.46:0/1780782110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293709380 0x55b293713ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.101+0000 7fb567189b80 1 -- 172.21.15.46:0/1780782110 >> 172.21.15.46:0/1780782110 conn(0x55b29367f220 msgr2=0x55b293705020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.101+0000 7fb567189b80 1 -- 172.21.15.46:0/1780782110 shutdown_connections 2024-03-18T20:39:48.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.101+0000 7fb567189b80 1 -- 172.21.15.46:0/1780782110 wait complete. 2024-03-18T20:39:48.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb567189b80 1 -- start start 2024-03-18T20:39:48.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb567189b80 1 Processor -- start 2024-03-18T20:39:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb567189b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 0x55b293713c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb567189b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293709380 0x55b293710f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb567189b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293711690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb567189b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2935f7d80 con 0x55b293708a60 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb567189b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2935f7a20 con 0x55b293709380 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb567189b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2934df6a0 con 0x55b293714220 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb567189b80 10 client.0 _ll_register_callbacks cb 0x55b2934633a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb558d90700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 0x55b293713c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb558d90700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 0x55b293713c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51934/0 (socket says 172.21.15.46:51934) 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb558d90700 1 -- 172.21.15.46:0/2388386593 learned_addr learned my addr 172.21.15.46:0/2388386593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb54bfff700 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293709380 0x55b293710f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.102+0000 7fb559591700 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293711690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb54bfff700 1 -- 172.21.15.46:0/2388386593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 msgr2=0x55b293711690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb54bfff700 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293711690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb54bfff700 1 -- 172.21.15.46:0/2388386593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 msgr2=0x55b293713c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb54bfff700 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 0x55b293713c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb54bfff700 1 -- 172.21.15.46:0/2388386593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb550017070 con 0x55b293709380 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb558d90700 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 0x55b293713c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:48.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb54bfff700 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293709380 0x55b293710f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb544012740 tx=0x7fb544012a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b18a43fb12cfa6b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb533fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb533fff700 1 -- 172.21.15.46:0/2388386593 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544017070 con 0x55b293709380 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb533fff700 1 -- 172.21.15.46:0/2388386593 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb544010430 con 0x55b293709380 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb567189b80 1 -- 172.21.15.46:0/2388386593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2936fcad0 con 0x55b293709380 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb533fff700 1 -- 172.21.15.46:0/2388386593 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5440105f0 con 0x55b293709380 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb567189b80 10 client.26537 fetch_fsmap learned FSMap version 38 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb567189b80 1 -- 172.21.15.46:0/2388386593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55b293489b60 con 0x55b293709380 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.103+0000 7fb533fff700 1 -- 172.21.15.46:0/2388386593 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb544028080 con 0x55b293709380 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.104+0000 7fb533fff700 1 -- 172.21.15.46:0/2388386593 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 38) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb54401c820 con 0x55b293709380 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.104+0000 7fb567189b80 10 client.26537 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.104+0000 7fb567189b80 10 client.26537 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.104+0000 7fb567189b80 20 client.26537 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.104+0000 7fb567189b80 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b293723eb0 0x55b2937262d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.104+0000 7fb567189b80 4 client.26537 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.104+0000 7fb567189b80 1 -- 172.21.15.46:0/2388386593 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b293654700 con 0x55b293723eb0 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.104+0000 7fb558d90700 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b293723eb0 0x55b2937262d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.104+0000 7fb533fff700 10 client.26537 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.104+0000 7fb558d90700 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b293723eb0 0x55b2937262d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=47cb3b71402b51e7 server_cookie=26c288a8c02dbac in_seq=0 out_seq=0 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb533fff700 1 -- 172.21.15.46:0/2388386593 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b293654700 con 0x55b293723eb0 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb533fff700 10 client.26537 handle_command_reply: tid=0 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb567189b80 1 client.26537 shutdown 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb567189b80 1 -- 172.21.15.46:0/2388386593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293709380 msgr2=0x55b293710f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb567189b80 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293709380 0x55b293710f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb544012740 tx=0x7fb544012a50 comp rx=0 tx=0).stop 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb567189b80 1 -- 172.21.15.46:0/2388386593 shutdown_connections 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb567189b80 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b293723eb0 0x55b2937262d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb567189b80 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293708a60 0x55b293713c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb567189b80 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293711690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb567189b80 1 --2- 172.21.15.46:0/2388386593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293709380 0x55b293710f50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb533fff700 0 client.26537 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:39:48.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb567189b80 1 -- 172.21.15.46:0/2388386593 >> 172.21.15.46:0/2388386593 conn(0x55b29367f220 msgr2=0x55b293704c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb567189b80 1 -- 172.21.15.46:0/2388386593 shutdown_connections 2024-03-18T20:39:48.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb567189b80 1 -- 172.21.15.46:0/2388386593 wait complete. 2024-03-18T20:39:48.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.105+0000 7fb567189b80 20 client.26537 trim_cache size 0 max 16384 2024-03-18T20:39:48.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.119+0000 7fb567189b80 1 Processor -- start 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.119+0000 7fb567189b80 1 -- start start 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.119+0000 7fb567189b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293708a60 0x55b29371ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.119+0000 7fb567189b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 0x55b293713870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.119+0000 7fb567189b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293714220 0x55b293704b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.119+0000 7fb567189b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2936ff240 con 0x55b293709380 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.119+0000 7fb567189b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b293709d60 con 0x55b293714220 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.119+0000 7fb567189b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29371b390 con 0x55b293708a60 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb54bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 0x55b293713870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb558d90700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293708a60 0x55b29371ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb54bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 0x55b293713870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51948/0 (socket says 172.21.15.46:51948) 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb558d90700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293708a60 0x55b29371ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37412/0 (socket says 172.21.15.46:37412) 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb54bfff700 1 -- 172.21.15.46:0/3363242140 learned_addr learned my addr 172.21.15.46:0/3363242140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb559591700 1 --2- 172.21.15.46:0/3363242140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293714220 0x55b293704b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb558d90700 1 -- 172.21.15.46:0/3363242140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293714220 msgr2=0x55b293704b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb558d90700 1 --2- 172.21.15.46:0/3363242140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293714220 0x55b293704b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb558d90700 1 -- 172.21.15.46:0/3363242140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 msgr2=0x55b293713870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb558d90700 1 --2- 172.21.15.46:0/3363242140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 0x55b293713870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb558d90700 1 -- 172.21.15.46:0/3363242140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b293621ea0 con 0x55b293708a60 2024-03-18T20:39:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb54bfff700 1 --2- 172.21.15.46:0/3363242140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 0x55b293713870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb559591700 1 --2- 172.21.15.46:0/3363242140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293714220 0x55b293704b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb558d90700 1 --2- 172.21.15.46:0/3363242140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293708a60 0x55b29371ae50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55b2937268b0 tx=0x7fb550017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fd0ad131ae47b6e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb53b7fe700 1 -- 172.21.15.46:0/3363242140 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55002c030 con 0x55b293708a60 2024-03-18T20:39:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb53b7fe700 1 -- 172.21.15.46:0/3363242140 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb55002d040 con 0x55b293708a60 2024-03-18T20:39:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.120+0000 7fb53b7fe700 1 -- 172.21.15.46:0/3363242140 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550005630 con 0x55b293708a60 2024-03-18T20:39:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.121+0000 7fb567189b80 1 -- 172.21.15.46:0/3363242140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293708a60 msgr2=0x55b29371ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.121+0000 7fb567189b80 1 --2- 172.21.15.46:0/3363242140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293708a60 0x55b29371ae50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55b2937268b0 tx=0x7fb550017000 comp rx=0 tx=0).stop 2024-03-18T20:39:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.121+0000 7fb567189b80 1 -- 172.21.15.46:0/3363242140 shutdown_connections 2024-03-18T20:39:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.121+0000 7fb567189b80 1 --2- 172.21.15.46:0/3363242140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293714220 0x55b293704b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.121+0000 7fb567189b80 1 --2- 172.21.15.46:0/3363242140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293708a60 0x55b29371ae50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.121+0000 7fb567189b80 1 --2- 172.21.15.46:0/3363242140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 0x55b293713870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.121+0000 7fb567189b80 1 -- 172.21.15.46:0/3363242140 >> 172.21.15.46:0/3363242140 conn(0x55b29367f220 msgr2=0x55b293703750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.121+0000 7fb567189b80 1 -- 172.21.15.46:0/3363242140 shutdown_connections 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.121+0000 7fb567189b80 1 -- 172.21.15.46:0/3363242140 wait complete. 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.121+0000 7fb567189b80 1 -- start start 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.121+0000 7fb567189b80 1 Processor -- start 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb567189b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293708a60 0x55b29371a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb567189b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 0x55b29371a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb567189b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293776ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb567189b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b293777220 con 0x55b293709380 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb567189b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b293777360 con 0x55b293708a60 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb567189b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2937774a0 con 0x55b293714220 2024-03-18T20:39:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb567189b80 10 client.0 _ll_register_callbacks cb 0x7fb544009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb54bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 0x55b29371a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb54bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 0x55b29371a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51962/0 (socket says 172.21.15.46:51962) 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb54bfff700 1 -- 172.21.15.46:0/3704254455 learned_addr learned my addr 172.21.15.46:0/3704254455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb559591700 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293776ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb558d90700 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293708a60 0x55b29371a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb559591700 1 -- 172.21.15.46:0/3704254455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293708a60 msgr2=0x55b29371a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb559591700 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293708a60 0x55b29371a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb559591700 1 -- 172.21.15.46:0/3704254455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 msgr2=0x55b29371a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb559591700 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 0x55b29371a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb559591700 1 -- 172.21.15.46:0/3704254455 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b293489b60 con 0x55b293714220 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb54bfff700 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 0x55b29371a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.122+0000 7fb558d90700 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293708a60 0x55b29371a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb559591700 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293776ce0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0cb650 tx=0x7fb54c09aa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49f9e143c081fa7d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb533fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb533fff700 1 -- 172.21.15.46:0/3704254455 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c09b0a0 con 0x55b293714220 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb567189b80 1 -- 172.21.15.46:0/3704254455 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb550026c20 con 0x55b293714220 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb533fff700 1 -- 172.21.15.46:0/3704254455 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb54c09eed0 con 0x55b293714220 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb533fff700 1 -- 172.21.15.46:0/3704254455 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb54c0a3490 con 0x55b293714220 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb567189b80 10 client.26557 fetch_fsmap learned FSMap version 38 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb567189b80 1 -- 172.21.15.46:0/3704254455 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55b2934dd3b0 con 0x55b293714220 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb533fff700 1 -- 172.21.15.46:0/3704254455 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54c0b9020 con 0x55b293714220 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb533fff700 1 -- 172.21.15.46:0/3704254455 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb54c09b300 con 0x55b293714220 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb567189b80 10 client.26557 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb567189b80 10 client.26557 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb567189b80 20 client.26557 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb567189b80 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b293778a70 0x55b29377ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb567189b80 4 client.26557 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.123+0000 7fb567189b80 1 -- 172.21.15.46:0/3704254455 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b2934aacf0 con 0x55b293778a70 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb558d90700 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b293778a70 0x55b29377ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb533fff700 10 client.26557 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb558d90700 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b293778a70 0x55b29377ae90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f682a8ec1e38a25 server_cookie=961d5a1de99bcc32 in_seq=0 out_seq=0 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb533fff700 1 -- 172.21.15.46:0/3704254455 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b2934aacf0 con 0x55b293778a70 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb533fff700 10 client.26557 handle_command_reply: tid=0 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb567189b80 1 client.26557 shutdown 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb567189b80 1 -- 172.21.15.46:0/3704254455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 msgr2=0x55b293776ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb567189b80 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293776ce0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0cb650 tx=0x7fb54c09aa50 comp rx=0 tx=0).stop 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb567189b80 1 -- 172.21.15.46:0/3704254455 shutdown_connections 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb567189b80 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b293714220 0x55b293776ce0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb567189b80 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b293708a60 0x55b29371a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb567189b80 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55b293778a70 0x55b29377ae90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb567189b80 1 --2- 172.21.15.46:0/3704254455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b293709380 0x55b29371a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb533fff700 0 client.26557 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:39:48.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.124+0000 7fb567189b80 1 -- 172.21.15.46:0/3704254455 >> 172.21.15.46:0/3704254455 conn(0x55b29367f220 msgr2=0x55b29370c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb567189b80 1 -- 172.21.15.46:0/3704254455 shutdown_connections 2024-03-18T20:39:48.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb567189b80 1 -- 172.21.15.46:0/3704254455 wait complete. 2024-03-18T20:39:48.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb567189b80 20 client.26557 trim_cache size 0 max 16384 2024-03-18T20:39:48.134 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:39:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb55aff4700 1 -- 172.21.15.46:0/923947371 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb53c0a1310 msgr2=0x7fb53c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb55aff4700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb53c0a1310 0x7fb53c0a37d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb554076450 tx=0x7fb550009260 comp rx=0 tx=0).stop 2024-03-18T20:39:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb55aff4700 1 -- 172.21.15.46:0/923947371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554105da0 msgr2=0x7fb5541012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb55aff4700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554105da0 0x7fb5541012d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00b070 tx=0x7fb54c0078e0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb55aff4700 1 -- 172.21.15.46:0/923947371 shutdown_connections 2024-03-18T20:39:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb55aff4700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb53c0a1310 0x7fb53c0a37d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb55aff4700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554105da0 0x7fb5541012d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb55aff4700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5540ff670 0x7fb554100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb55aff4700 1 --2- 172.21.15.46:0/923947371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb554100030 0x7fb554100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb55aff4700 1 -- 172.21.15.46:0/923947371 >> 172.21.15.46:0/923947371 conn(0x7fb554074ed0 msgr2=0x7fb5540fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb55aff4700 1 -- 172.21.15.46:0/923947371 shutdown_connections 2024-03-18T20:39:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.125+0000 7fb55aff4700 1 -- 172.21.15.46:0/923947371 wait complete. 2024-03-18T20:39:48.141 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:39:48.142 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:39:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.278+0000 7f22dabe4700 1 -- 172.21.15.46:0/1930102427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40ffe50 msgr2=0x7f22d4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.278+0000 7f22dabe4700 1 --2- 172.21.15.46:0/1930102427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40ffe50 0x7f22d4100230 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009990 tx=0x7f22d0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.278+0000 7f22dabe4700 1 -- 172.21.15.46:0/1930102427 shutdown_connections 2024-03-18T20:39:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.278+0000 7f22dabe4700 1 --2- 172.21.15.46:0/1930102427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40ffe50 0x7f22d4100230 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.278+0000 7f22dabe4700 1 --2- 172.21.15.46:0/1930102427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d4104f60 0x7f22d410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.278+0000 7f22dabe4700 1 --2- 172.21.15.46:0/1930102427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d4100800 0x7f22d4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.278+0000 7f22dabe4700 1 -- 172.21.15.46:0/1930102427 >> 172.21.15.46:0/1930102427 conn(0x7f22d4076d40 msgr2=0x7f22d4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.278+0000 7f22dabe4700 1 -- 172.21.15.46:0/1930102427 shutdown_connections 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.278+0000 7f22dabe4700 1 -- 172.21.15.46:0/1930102427 wait complete. 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.278+0000 7f22dabe4700 1 Processor -- start 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.278+0000 7f22dabe4700 1 -- start start 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22dabe4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40ffe50 0x7f22d4199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22dabe4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d4104f60 0x7f22d41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22dabe4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d419dc40 0x7f22d419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22dabe4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d419e6c0 con 0x7f22d40ffe50 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22dabe4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d419e830 con 0x7f22d419dc40 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22dabe4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d419e9f0 con 0x7f22d4104f60 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22d8980700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40ffe50 0x7f22d4199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22d8980700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40ffe50 0x7f22d4199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51978/0 (socket says 172.21.15.46:51978) 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22d8980700 1 -- 172.21.15.46:0/3627354721 learned_addr learned my addr 172.21.15.46:0/3627354721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22d9181700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d419dc40 0x7f22d419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22cbfff700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d4104f60 0x7f22d41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22d9181700 1 -- 172.21.15.46:0/3627354721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d4104f60 msgr2=0x7f22d41998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22d9181700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d4104f60 0x7f22d41998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22d9181700 1 -- 172.21.15.46:0/3627354721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40ffe50 msgr2=0x7f22d4199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22d9181700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40ffe50 0x7f22d4199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22d9181700 1 -- 172.21.15.46:0/3627354721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d00095f0 con 0x7f22d419dc40 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22d8980700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40ffe50 0x7f22d4199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.279+0000 7f22d9181700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d419dc40 0x7f22d419e0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00cc50 tx=0x7f22cc00cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.280+0000 7f22c9ffb700 1 -- 172.21.15.46:0/3627354721 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00aa90 con 0x7f22d419dc40 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.280+0000 7f22c9ffb700 1 -- 172.21.15.46:0/3627354721 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22cc00abf0 con 0x7f22d419dc40 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.280+0000 7f22c9ffb700 1 -- 172.21.15.46:0/3627354721 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc0115e0 con 0x7f22d419dc40 2024-03-18T20:39:48.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.280+0000 7f22dabe4700 1 -- 172.21.15.46:0/3627354721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d41a5f50 con 0x7f22d419dc40 2024-03-18T20:39:48.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.280+0000 7f22dabe4700 1 -- 172.21.15.46:0/3627354721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d41a6380 con 0x7f22d419dc40 2024-03-18T20:39:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.283+0000 7f22c9ffb700 1 -- 172.21.15.46:0/3627354721 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f22cc012420 con 0x7f22d419dc40 2024-03-18T20:39:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.283+0000 7f22c9ffb700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f22bc0a1290 0x7f22bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.283+0000 7f22d8980700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f22bc0a1290 0x7f22bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.283+0000 7f22c9ffb700 1 -- 172.21.15.46:0/3627354721 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f22cc0ccc50 con 0x7f22d419dc40 2024-03-18T20:39:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.284+0000 7f22d8980700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f22bc0a1290 0x7f22bc0a3750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f22d0006010 tx=0x7f22d000b540 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.284+0000 7f22e6d79b80 1 Processor -- start 2024-03-18T20:39:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.284+0000 7f22e6d79b80 1 -- start start 2024-03-18T20:39:48.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.284+0000 7f22e6d79b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22ca60 0x55be5a22ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.284+0000 7f22e6d79b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 0x55be5a237ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.284+0000 7f22e6d79b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a238220 0x55be5a23a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.284+0000 7f22e6d79b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5a186990 con 0x55be5a22d380 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.284+0000 7f22e6d79b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5a184350 con 0x55be5a22ca60 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.284+0000 7f22e6d79b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5a17fc40 con 0x55be5a238220 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.284+0000 7f22cbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 0x55be5a237ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22cbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 0x55be5a237ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51990/0 (socket says 172.21.15.46:51990) 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22cbfff700 1 -- 172.21.15.46:0/1980718691 learned_addr learned my addr 172.21.15.46:0/1980718691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22d9181700 1 --2- 172.21.15.46:0/1980718691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a238220 0x55be5a23a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22d8980700 1 --2- 172.21.15.46:0/1980718691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22ca60 0x55be5a22ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22cbfff700 1 -- 172.21.15.46:0/1980718691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a238220 msgr2=0x55be5a23a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22cbfff700 1 --2- 172.21.15.46:0/1980718691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a238220 0x55be5a23a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22cbfff700 1 -- 172.21.15.46:0/1980718691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22ca60 msgr2=0x55be5a22ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22cbfff700 1 --2- 172.21.15.46:0/1980718691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22ca60 0x55be5a22ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22cbfff700 1 -- 172.21.15.46:0/1980718691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be5a0013b0 con 0x55be5a22d380 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22cbfff700 1 --2- 172.21.15.46:0/1980718691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 0x55be5a237ce0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f22c400bcf0 tx=0x7f22c40103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bdd849968e7dcd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22baffd700 1 -- 172.21.15.46:0/1980718691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4010d10 con 0x55be5a22d380 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22baffd700 1 -- 172.21.15.46:0/1980718691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22c4015070 con 0x55be5a22d380 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.285+0000 7f22baffd700 1 -- 172.21.15.46:0/1980718691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4005020 con 0x55be5a22d380 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.286+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1980718691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 msgr2=0x55be5a237ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.286+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/1980718691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 0x55be5a237ce0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f22c400bcf0 tx=0x7f22c40103d0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.286+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1980718691 shutdown_connections 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.286+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/1980718691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a238220 0x55be5a23a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.286+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/1980718691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22ca60 0x55be5a22ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.286+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/1980718691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 0x55be5a237ce0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.286+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1980718691 >> 172.21.15.46:0/1980718691 conn(0x55be5a1a3220 msgr2=0x55be5a229020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.286+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1980718691 shutdown_connections 2024-03-18T20:39:48.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.286+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1980718691 wait complete. 2024-03-18T20:39:48.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.286+0000 7f22e6d79b80 1 -- start start 2024-03-18T20:39:48.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22e6d79b80 1 Processor -- start 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22e6d79b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22ca60 0x55be5a237bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22e6d79b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 0x55be5a234f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22e6d79b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a238220 0x55be5a235590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22e6d79b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5a11bd80 con 0x55be5a22d380 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22e6d79b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5a11ba20 con 0x55be5a22ca60 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22e6d79b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5a0036a0 con 0x55be5a238220 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22e6d79b80 10 client.0 _ll_register_callbacks cb 0x55be59f873a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22cbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 0x55be5a234f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22cbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 0x55be5a234f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51996/0 (socket says 172.21.15.46:51996) 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22cbfff700 1 -- 172.21.15.46:0/4076170118 learned_addr learned my addr 172.21.15.46:0/4076170118 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22d8980700 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22ca60 0x55be5a237bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.287+0000 7f22d9181700 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a238220 0x55be5a235590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22d8980700 1 -- 172.21.15.46:0/4076170118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a238220 msgr2=0x55be5a235590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22d8980700 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a238220 0x55be5a235590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22d8980700 1 -- 172.21.15.46:0/4076170118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 msgr2=0x55be5a234f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22d8980700 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 0x55be5a234f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22d8980700 1 -- 172.21.15.46:0/4076170118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c400b920 con 0x55be5a22ca60 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22d8980700 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22ca60 0x55be5a237bd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f22d000bcb0 tx=0x7f22d0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b470e849ef83c9ab server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22b3fff700 1 -- 172.21.15.46:0/4076170118 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000f460 con 0x55be5a22ca60 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22b3fff700 1 -- 172.21.15.46:0/4076170118 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f22d000f5c0 con 0x55be5a22ca60 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22e6d79b80 1 -- 172.21.15.46:0/4076170118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be5a220ad0 con 0x55be5a22ca60 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22b3fff700 1 -- 172.21.15.46:0/4076170118 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d000f780 con 0x55be5a22ca60 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22e6d79b80 10 client.26555 fetch_fsmap learned FSMap version 38 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22e6d79b80 1 -- 172.21.15.46:0/4076170118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55be59fadb60 con 0x55be5a22ca60 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.288+0000 7f22b3fff700 1 -- 172.21.15.46:0/4076170118 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d003d020 con 0x55be5a22ca60 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.289+0000 7f22b3fff700 1 -- 172.21.15.46:0/4076170118 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 38) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f22d0024950 con 0x55be5a22ca60 2024-03-18T20:39:48.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.289+0000 7f22e6d79b80 10 client.26555 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.289+0000 7f22e6d79b80 10 client.26555 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.289+0000 7f22e6d79b80 20 client.26555 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.289+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55be5a247e20 0x55be5a24a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.289+0000 7f22e6d79b80 4 client.26555 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.289+0000 7f22e6d79b80 1 -- 172.21.15.46:0/4076170118 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be5a178700 con 0x55be5a247e20 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.289+0000 7f22cbfff700 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55be5a247e20 0x55be5a24a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.289+0000 7f22b3fff700 10 client.26555 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.289+0000 7f22cbfff700 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55be5a247e20 0x55be5a24a240 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5dee600a2238563 server_cookie=a58289dc244a248e in_seq=0 out_seq=0 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.290+0000 7f22b3fff700 1 -- 172.21.15.46:0/4076170118 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55be5a178700 con 0x55be5a247e20 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.290+0000 7f22b3fff700 10 client.26555 handle_command_reply: tid=0 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.290+0000 7f22e6d79b80 1 client.26555 shutdown 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.290+0000 7f22e6d79b80 1 -- 172.21.15.46:0/4076170118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22ca60 msgr2=0x55be5a237bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.290+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22ca60 0x55be5a237bd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f22d000bcb0 tx=0x7f22d0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.290+0000 7f22e6d79b80 1 -- 172.21.15.46:0/4076170118 shutdown_connections 2024-03-18T20:39:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.290+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a238220 0x55be5a235590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.290+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22ca60 0x55be5a237bd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.290+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55be5a247e20 0x55be5a24a240 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.290+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/4076170118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22d380 0x55be5a234f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.290+0000 7f22b3fff700 0 client.26555 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:39:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.290+0000 7f22e6d79b80 1 -- 172.21.15.46:0/4076170118 >> 172.21.15.46:0/4076170118 conn(0x55be5a1a3220 msgr2=0x55be5a228c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.291+0000 7f22e6d79b80 1 -- 172.21.15.46:0/4076170118 shutdown_connections 2024-03-18T20:39:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.291+0000 7f22e6d79b80 1 -- 172.21.15.46:0/4076170118 wait complete. 2024-03-18T20:39:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.291+0000 7f22e6d79b80 20 client.26555 trim_cache size 0 max 16384 2024-03-18T20:39:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.304+0000 7f22e6d79b80 1 Processor -- start 2024-03-18T20:39:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.304+0000 7f22e6d79b80 1 -- start start 2024-03-18T20:39:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22e6d79b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a22ca60 0x55be5a22fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22e6d79b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22d380 0x55be5a23ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22e6d79b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a238220 0x55be5a2435a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22e6d79b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5a22d1c0 con 0x55be5a238220 2024-03-18T20:39:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22e6d79b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5a22e550 con 0x55be5a22d380 2024-03-18T20:39:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22e6d79b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5a222f80 con 0x55be5a22ca60 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22d8980700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a22ca60 0x55be5a22fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22d8980700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a22ca60 0x55be5a22fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37466/0 (socket says 172.21.15.46:37466) 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22d8980700 1 -- 172.21.15.46:0/619149672 learned_addr learned my addr 172.21.15.46:0/619149672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22cbfff700 1 --2- 172.21.15.46:0/619149672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22d380 0x55be5a23ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22d8980700 1 -- 172.21.15.46:0/619149672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22d380 msgr2=0x55be5a23ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22d8980700 1 --2- 172.21.15.46:0/619149672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22d380 0x55be5a23ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22d8980700 1 -- 172.21.15.46:0/619149672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a238220 msgr2=0x55be5a2435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22d8980700 1 --2- 172.21.15.46:0/619149672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a238220 0x55be5a2435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22d8980700 1 -- 172.21.15.46:0/619149672 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be5a145ea0 con 0x55be5a22ca60 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.305+0000 7f22cbfff700 1 --2- 172.21.15.46:0/619149672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22d380 0x55be5a23ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.306+0000 7f22d8980700 1 --2- 172.21.15.46:0/619149672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a22ca60 0x55be5a22fa20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009350 tx=0x7f22d0004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff5e456e8ecbb72b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.306+0000 7f22bb7fe700 1 -- 172.21.15.46:0/619149672 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0024410 con 0x55be5a22ca60 2024-03-18T20:39:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.306+0000 7f22bb7fe700 1 -- 172.21.15.46:0/619149672 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22d003d030 con 0x55be5a22ca60 2024-03-18T20:39:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.306+0000 7f22bb7fe700 1 -- 172.21.15.46:0/619149672 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0018e10 con 0x55be5a22ca60 2024-03-18T20:39:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.306+0000 7f22e6d79b80 1 -- 172.21.15.46:0/619149672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a22ca60 msgr2=0x55be5a22fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.306+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/619149672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a22ca60 0x55be5a22fa20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009350 tx=0x7f22d0004b50 comp rx=0 tx=0).stop 2024-03-18T20:39:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.306+0000 7f22e6d79b80 1 -- 172.21.15.46:0/619149672 shutdown_connections 2024-03-18T20:39:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.306+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/619149672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a238220 0x55be5a2435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.306+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/619149672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a22ca60 0x55be5a22fa20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.306+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/619149672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a22d380 0x55be5a23ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.306+0000 7f22e6d79b80 1 -- 172.21.15.46:0/619149672 >> 172.21.15.46:0/619149672 conn(0x55be5a1a3220 msgr2=0x55be5a233830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.306+0000 7f22e6d79b80 1 -- 172.21.15.46:0/619149672 shutdown_connections 2024-03-18T20:39:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.307+0000 7f22e6d79b80 1 -- 172.21.15.46:0/619149672 wait complete. 2024-03-18T20:39:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.307+0000 7f22e6d79b80 1 -- start start 2024-03-18T20:39:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.307+0000 7f22e6d79b80 1 Processor -- start 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.307+0000 7f22e6d79b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22ca60 0x55be5a23d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.307+0000 7f22e6d79b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a22d380 0x55be5a23d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.307+0000 7f22e6d79b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a238220 0x55be5a29ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.307+0000 7f22e6d79b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5a29b150 con 0x55be5a22ca60 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.307+0000 7f22e6d79b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5a29b290 con 0x55be5a238220 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.307+0000 7f22e6d79b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5a29b420 con 0x55be5a22d380 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.307+0000 7f22e6d79b80 10 client.0 _ll_register_callbacks cb 0x7f22d0005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22d8980700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22ca60 0x55be5a23d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22d8980700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22ca60 0x55be5a23d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52006/0 (socket says 172.21.15.46:52006) 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22d8980700 1 -- 172.21.15.46:0/1332476101 learned_addr learned my addr 172.21.15.46:0/1332476101 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22cbfff700 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a22d380 0x55be5a23d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22d9181700 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a238220 0x55be5a29ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22d8980700 1 -- 172.21.15.46:0/1332476101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a22d380 msgr2=0x55be5a23d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22d8980700 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a22d380 0x55be5a23d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22d8980700 1 -- 172.21.15.46:0/1332476101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a238220 msgr2=0x55be5a29ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22d8980700 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a238220 0x55be5a29ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22d8980700 1 -- 172.21.15.46:0/1332476101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be5a0013b0 con 0x55be5a22ca60 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22d8980700 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22ca60 0x55be5a23d020 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009ce0 tx=0x7f22d0018db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18ccb553ec1f878 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22b3fff700 1 -- 172.21.15.46:0/1332476101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0024820 con 0x55be5a22ca60 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22b3fff700 1 -- 172.21.15.46:0/1332476101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f22d003e080 con 0x55be5a22ca60 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1332476101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22c4015070 con 0x55be5a22ca60 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22b3fff700 1 -- 172.21.15.46:0/1332476101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d0024980 con 0x55be5a22ca60 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22e6d79b80 10 client.16827 fetch_fsmap learned FSMap version 38 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.308+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1332476101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55be59fadb60 con 0x55be5a22ca60 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.309+0000 7f22b3fff700 1 -- 172.21.15.46:0/1332476101 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22c4015070 con 0x55be5a22ca60 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.309+0000 7f22b3fff700 1 -- 172.21.15.46:0/1332476101 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 38) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f22d000fcb0 con 0x55be5a22ca60 2024-03-18T20:39:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.309+0000 7f22e6d79b80 10 client.16827 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.309+0000 7f22e6d79b80 10 client.16827 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.309+0000 7f22e6d79b80 20 client.16827 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.309+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55be5a29c6f0 0x55be5a29eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.309+0000 7f22e6d79b80 4 client.16827 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.309+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1332476101 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be59fcecf0 con 0x55be5a29c6f0 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.309+0000 7f22cbfff700 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55be5a29c6f0 0x55be5a29eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22cbfff700 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55be5a29c6f0 0x55be5a29eae0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a61d1882e7fc7ac server_cookie=58e6c63e4a28ab5b in_seq=0 out_seq=0 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22b3fff700 10 client.16827 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22b3fff700 1 -- 172.21.15.46:0/1332476101 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be59fcecf0 con 0x55be5a29c6f0 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22b3fff700 10 client.16827 handle_command_reply: tid=0 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22e6d79b80 1 client.16827 shutdown 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1332476101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22ca60 msgr2=0x55be5a23d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22ca60 0x55be5a23d020 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009ce0 tx=0x7f22d0018db0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1332476101 shutdown_connections 2024-03-18T20:39:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55be5a29c6f0 0x55be5a29eae0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be5a22ca60 0x55be5a23d020 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22b3fff700 0 client.16827 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:39:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be5a238220 0x55be5a29ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22e6d79b80 1 --2- 172.21.15.46:0/1332476101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be5a22d380 0x55be5a23d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1332476101 >> 172.21.15.46:0/1332476101 conn(0x55be5a1a3220 msgr2=0x55be5a243e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1332476101 shutdown_connections 2024-03-18T20:39:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22e6d79b80 1 -- 172.21.15.46:0/1332476101 wait complete. 2024-03-18T20:39:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.310+0000 7f22e6d79b80 20 client.16827 trim_cache size 0 max 16384 2024-03-18T20:39:48.319 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:39:48.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.311+0000 7f22dabe4700 1 -- 172.21.15.46:0/3627354721 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f22bc0a1290 msgr2=0x7f22bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.311+0000 7f22dabe4700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f22bc0a1290 0x7f22bc0a3750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f22d0006010 tx=0x7f22d000b540 comp rx=0 tx=0).stop 2024-03-18T20:39:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.311+0000 7f22dabe4700 1 -- 172.21.15.46:0/3627354721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d419dc40 msgr2=0x7f22d419e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.311+0000 7f22dabe4700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d419dc40 0x7f22d419e0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00cc50 tx=0x7f22cc00cf60 comp rx=0 tx=0).stop 2024-03-18T20:39:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.311+0000 7f22dabe4700 1 -- 172.21.15.46:0/3627354721 shutdown_connections 2024-03-18T20:39:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.311+0000 7f22dabe4700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f22bc0a1290 0x7f22bc0a3750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.311+0000 7f22dabe4700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40ffe50 0x7f22d4199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.311+0000 7f22dabe4700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d419dc40 0x7f22d419e0c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.311+0000 7f22dabe4700 1 --2- 172.21.15.46:0/3627354721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d4104f60 0x7f22d41998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.311+0000 7f22dabe4700 1 -- 172.21.15.46:0/3627354721 >> 172.21.15.46:0/3627354721 conn(0x7f22d4076d40 msgr2=0x7f22d40fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.311+0000 7f22dabe4700 1 -- 172.21.15.46:0/3627354721 shutdown_connections 2024-03-18T20:39:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.311+0000 7f22dabe4700 1 -- 172.21.15.46:0/3627354721 wait complete. 2024-03-18T20:39:48.327 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:39:48.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:39:48.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:48 smithi046 ceph-mon[43495]: pgmap v114: 97 pgs: 97 active+clean; 42 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 441 KiB/s wr, 57 op/s 2024-03-18T20:39:48.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:48 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:48.354 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:48 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1070811139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:39:48.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.464+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2235665077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc3c105ce0 msgr2=0x7fdc3c1060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.464+0000 7fdc41d2d700 1 --2- 172.21.15.46:0/2235665077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc3c105ce0 0x7fdc3c1060c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009990 tx=0x7fdc2c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.464+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2235665077 shutdown_connections 2024-03-18T20:39:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.464+0000 7fdc41d2d700 1 --2- 172.21.15.46:0/2235665077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc3c105ce0 0x7fdc3c1060c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.464+0000 7fdc41d2d700 1 --2- 172.21.15.46:0/2235665077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc3c0687f0 0x7fdc3c10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.464+0000 7fdc41d2d700 1 --2- 172.21.15.46:0/2235665077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3c067e30 0x7fdc3c0682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.464+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2235665077 >> 172.21.15.46:0/2235665077 conn(0x7fdc3c076d40 msgr2=0x7fdc3c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.464+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2235665077 shutdown_connections 2024-03-18T20:39:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.464+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2235665077 wait complete. 2024-03-18T20:39:48.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.465+0000 7fdc41d2d700 1 Processor -- start 2024-03-18T20:39:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.465+0000 7fdc41d2d700 1 -- start start 2024-03-18T20:39:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.465+0000 7fdc41d2d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3c067e30 0x7fdc3c1971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.465+0000 7fdc41d2d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc3c0687f0 0x7fdc3c197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.465+0000 7fdc41d2d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc3c105ce0 0x7fdc3c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.465+0000 7fdc41d2d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c19c110 con 0x7fdc3c105ce0 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.465+0000 7fdc41d2d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c19c280 con 0x7fdc3c067e30 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.465+0000 7fdc41d2d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c19c440 con 0x7fdc3c0687f0 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.465+0000 7fdc3bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc3c105ce0 0x7fdc3c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.465+0000 7fdc3bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc3c105ce0 0x7fdc3c19bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52028/0 (socket says 172.21.15.46:52028) 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.465+0000 7fdc3bfff700 1 -- 172.21.15.46:0/2825139561 learned_addr learned my addr 172.21.15.46:0/2825139561 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3c067e30 0x7fdc3c1971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc3affd700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc3c0687f0 0x7fdc3c197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc3bfff700 1 -- 172.21.15.46:0/2825139561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc3c0687f0 msgr2=0x7fdc3c197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc3bfff700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc3c0687f0 0x7fdc3c197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc3bfff700 1 -- 172.21.15.46:0/2825139561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3c067e30 msgr2=0x7fdc3c1971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc3bfff700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3c067e30 0x7fdc3c1971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc3bfff700 1 -- 172.21.15.46:0/2825139561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c0095f0 con 0x7fdc3c105ce0 2024-03-18T20:39:48.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc3bfff700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc3c105ce0 0x7fdc3c19bad0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdc34007ee0 tx=0x7fdc3400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc38ff9700 1 -- 172.21.15.46:0/2825139561 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34017070 con 0x7fdc3c105ce0 2024-03-18T20:39:48.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc38ff9700 1 -- 172.21.15.46:0/2825139561 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc3400f040 con 0x7fdc3c105ce0 2024-03-18T20:39:48.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2825139561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c19c720 con 0x7fdc3c105ce0 2024-03-18T20:39:48.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc38ff9700 1 -- 172.21.15.46:0/2825139561 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34005740 con 0x7fdc3c105ce0 2024-03-18T20:39:48.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.466+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2825139561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c1a3f60 con 0x7fdc3c105ce0 2024-03-18T20:39:48.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:48 smithi073 ceph-mon[47384]: pgmap v114: 97 pgs: 97 active+clean; 42 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 441 KiB/s wr, 57 op/s 2024-03-18T20:39:48.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:48 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:48.473 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:48 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1070811139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:39:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.468+0000 7fdc38ff9700 1 -- 172.21.15.46:0/2825139561 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fdc340058a0 con 0x7fdc3c105ce0 2024-03-18T20:39:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.469+0000 7fdc38ff9700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdc280a1290 0x7fdc280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.469+0000 7fdc38ff9700 1 -- 172.21.15.46:0/2825139561 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fdc340cc500 con 0x7fdc3c105ce0 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.469+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdc280a1290 0x7fdc280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdc280a1290 0x7fdc280a3750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c006010 tx=0x7fdc2c01a040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc4dec2b80 1 Processor -- start 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc4dec2b80 1 -- start start 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc4dec2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489c5a60 0x562b489c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc4dec2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc4dec2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489d5630 0x562b489d7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc4dec2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4891f990 con 0x562b489d5630 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc4dec2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4891d350 con 0x562b489c5a60 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc4dec2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b48918c40 con 0x562b489c6380 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc3bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489d5630 0x562b489d7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc3bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489d5630 0x562b489d7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52032/0 (socket says 172.21.15.46:52032) 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc3bfff700 1 -- 172.21.15.46:0/142992275 learned_addr learned my addr 172.21.15.46:0/142992275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.470+0000 7fdc3affd700 1 --2- 172.21.15.46:0/142992275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.471+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/142992275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489c5a60 0x562b489c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.471+0000 7fdc3bfff700 1 -- 172.21.15.46:0/142992275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 msgr2=0x562b489d50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.471+0000 7fdc3bfff700 1 --2- 172.21.15.46:0/142992275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.471+0000 7fdc3bfff700 1 -- 172.21.15.46:0/142992275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489c5a60 msgr2=0x562b489c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.471+0000 7fdc3bfff700 1 --2- 172.21.15.46:0/142992275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489c5a60 0x562b489c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.471+0000 7fdc3bfff700 1 -- 172.21.15.46:0/142992275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b4879a3b0 con 0x562b489d5630 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.471+0000 7fdc3bfff700 1 --2- 172.21.15.46:0/142992275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489d5630 0x562b489d7a20 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdc34096050 tx=0x7fdc34095a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a91b458bd69103a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.471+0000 7fdc31ffb700 1 -- 172.21.15.46:0/142992275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340b90d0 con 0x562b489d5630 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.471+0000 7fdc31ffb700 1 -- 172.21.15.46:0/142992275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc3409d410 con 0x562b489d5630 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.471+0000 7fdc31ffb700 1 -- 172.21.15.46:0/142992275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340840c0 con 0x562b489d5630 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.472+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/142992275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489d5630 msgr2=0x562b489d7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.472+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/142992275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489d5630 0x562b489d7a20 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdc34096050 tx=0x7fdc34095a50 comp rx=0 tx=0).stop 2024-03-18T20:39:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.472+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/142992275 shutdown_connections 2024-03-18T20:39:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.472+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/142992275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489d5630 0x562b489d7a20 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.472+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/142992275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489c5a60 0x562b489c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.472+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/142992275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.472+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/142992275 >> 172.21.15.46:0/142992275 conn(0x562b4893c220 msgr2=0x562b489c2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.472+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/142992275 shutdown_connections 2024-03-18T20:39:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.472+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/142992275 wait complete. 2024-03-18T20:39:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.472+0000 7fdc4dec2b80 1 -- start start 2024-03-18T20:39:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc4dec2b80 1 Processor -- start 2024-03-18T20:39:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc4dec2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc4dec2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc4dec2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489d5630 0x562b489d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc4dec2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b488b4d80 con 0x562b489c5a60 2024-03-18T20:39:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc4dec2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b488b4a20 con 0x562b489d5630 2024-03-18T20:39:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc4dec2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4879c6a0 con 0x562b489c6380 2024-03-18T20:39:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc4dec2b80 10 client.0 _ll_register_callbacks cb 0x562b487203a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489d2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52042/0 (socket says 172.21.15.46:52042) 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc3b7fe700 1 -- 172.21.15.46:0/2624929177 learned_addr learned my addr 172.21.15.46:0/2624929177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc3bfff700 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489d5630 0x562b489d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.473+0000 7fdc3affd700 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc3b7fe700 1 -- 172.21.15.46:0/2624929177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 msgr2=0x562b489d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc3b7fe700 1 -- 172.21.15.46:0/2624929177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489d5630 msgr2=0x562b489d2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489d5630 0x562b489d2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc3b7fe700 1 -- 172.21.15.46:0/2624929177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c1a3f60 con 0x562b489c5a60 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489d2500 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00f730 tx=0x7fdc2c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5129e9c568befd71 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc1affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc1affd700 1 -- 172.21.15.46:0/2624929177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c01d040 con 0x562b489c5a60 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc1affd700 1 -- 172.21.15.46:0/2624929177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fdc2c023610 con 0x562b489c5a60 2024-03-18T20:39:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2624929177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b489b9ad0 con 0x562b489c5a60 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc1affd700 1 -- 172.21.15.46:0/2624929177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc2c0237d0 con 0x562b489c5a60 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc4dec2b80 10 client.16839 fetch_fsmap learned FSMap version 38 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2624929177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x562b48746b60 con 0x562b489c5a60 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.474+0000 7fdc1affd700 1 -- 172.21.15.46:0/2624929177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc2c03c020 con 0x562b489c5a60 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.475+0000 7fdc1affd700 1 -- 172.21.15.46:0/2624929177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 38) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fdc2c02e070 con 0x562b489c5a60 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.475+0000 7fdc4dec2b80 10 client.16839 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.475+0000 7fdc4dec2b80 10 client.16839 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.475+0000 7fdc4dec2b80 20 client.16839 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.475+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x562b489e5350 0x562b489e7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.475+0000 7fdc4dec2b80 4 client.16839 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.475+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2624929177 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b48911700 con 0x562b489e5350 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.475+0000 7fdc3affd700 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x562b489e5350 0x562b489e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.475+0000 7fdc1affd700 10 client.16839 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:39:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.475+0000 7fdc3affd700 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x562b489e5350 0x562b489e7770 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=563069a5b00fdafe server_cookie=5660e30a8c315fbc in_seq=0 out_seq=0 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc1affd700 1 -- 172.21.15.46:0/2624929177 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562b48911700 con 0x562b489e5350 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc1affd700 10 client.16839 handle_command_reply: tid=0 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc4dec2b80 1 client.16839 shutdown 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2624929177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 msgr2=0x562b489d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489d2500 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00f730 tx=0x7fdc2c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2624929177 shutdown_connections 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x562b489e5350 0x562b489e7770 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489d2500 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489d5630 0x562b489d2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc1affd700 0 client.16839 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/2624929177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2624929177 >> 172.21.15.46:0/2624929177 conn(0x562b4893c220 msgr2=0x562b489c1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2624929177 shutdown_connections 2024-03-18T20:39:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2624929177 wait complete. 2024-03-18T20:39:48.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.476+0000 7fdc4dec2b80 20 client.16839 trim_cache size 0 max 16384 2024-03-18T20:39:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc4dec2b80 1 Processor -- start 2024-03-18T20:39:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc4dec2b80 1 -- start start 2024-03-18T20:39:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc4dec2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489dc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc4dec2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc4dec2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489d5630 0x562b489c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc4dec2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b489c7440 con 0x562b489c5a60 2024-03-18T20:39:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc4dec2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b489d3a60 con 0x562b489d5630 2024-03-18T20:39:48.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc4dec2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b489be610 con 0x562b489c6380 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489dc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52058/0 (socket says 172.21.15.46:52058) 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc3b7fe700 1 -- 172.21.15.46:0/460785267 learned_addr learned my addr 172.21.15.46:0/460785267 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc3affd700 1 --2- 172.21.15.46:0/460785267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc3b7fe700 1 -- 172.21.15.46:0/460785267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 msgr2=0x562b489d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc3bfff700 1 --2- 172.21.15.46:0/460785267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489d5630 0x562b489c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/460785267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc3b7fe700 1 -- 172.21.15.46:0/460785267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489d5630 msgr2=0x562b489c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/460785267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489d5630 0x562b489c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.490+0000 7fdc3b7fe700 1 -- 172.21.15.46:0/460785267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b488deea0 con 0x562b489c5a60 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.491+0000 7fdc3affd700 1 --2- 172.21.15.46:0/460785267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.491+0000 7fdc3bfff700 1 --2- 172.21.15.46:0/460785267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489d5630 0x562b489c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.491+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/460785267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489dc260 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c021020 tx=0x7fdc2c0098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78450aad986da243 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.491+0000 7fdc327fc700 1 -- 172.21.15.46:0/460785267 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c01d040 con 0x562b489c5a60 2024-03-18T20:39:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.491+0000 7fdc327fc700 1 -- 172.21.15.46:0/460785267 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc2c03c070 con 0x562b489c5a60 2024-03-18T20:39:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.491+0000 7fdc327fc700 1 -- 172.21.15.46:0/460785267 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c023a80 con 0x562b489c5a60 2024-03-18T20:39:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.492+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/460785267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 msgr2=0x562b489dc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.492+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/460785267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489dc260 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c021020 tx=0x7fdc2c0098c0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.492+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/460785267 shutdown_connections 2024-03-18T20:39:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.492+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/460785267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489dc260 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.492+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/460785267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489d5630 0x562b489c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.492+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/460785267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489c6380 0x562b489d2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.492+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/460785267 >> 172.21.15.46:0/460785267 conn(0x562b4893c220 msgr2=0x562b489cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.492+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/460785267 shutdown_connections 2024-03-18T20:39:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.492+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/460785267 wait complete. 2024-03-18T20:39:48.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.492+0000 7fdc4dec2b80 1 -- start start 2024-03-18T20:39:48.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.492+0000 7fdc4dec2b80 1 Processor -- start 2024-03-18T20:39:48.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc4dec2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489da430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc4dec2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489c6380 0x562b489da970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc4dec2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489d5630 0x562b48a38140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc4dec2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b489db0c0 con 0x562b489c5a60 2024-03-18T20:39:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc4dec2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b48a38680 con 0x562b489c6380 2024-03-18T20:39:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc4dec2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b48a38840 con 0x562b489d5630 2024-03-18T20:39:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc4dec2b80 10 client.0 _ll_register_callbacks cb 0x562b487203a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc3b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489da430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52070/0 (socket says 172.21.15.46:52070) 2024-03-18T20:39:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc3b7fe700 1 -- 172.21.15.46:0/2912487769 learned_addr learned my addr 172.21.15.46:0/2912487769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc3affd700 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489c6380 0x562b489da970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc3bfff700 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489d5630 0x562b48a38140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc3affd700 1 -- 172.21.15.46:0/2912487769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489d5630 msgr2=0x562b48a38140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc3affd700 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489d5630 0x562b48a38140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc3affd700 1 -- 172.21.15.46:0/2912487769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 msgr2=0x562b489da430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc3affd700 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489da430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc3affd700 1 -- 172.21.15.46:0/2912487769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b48746b60 con 0x562b489c6380 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489da430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc3affd700 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489c6380 0x562b489da970 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x562b489e7fe0 tx=0x7fdc24011880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=658a7d01b8019755 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc1affd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.493+0000 7fdc1affd700 1 -- 172.21.15.46:0/2912487769 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24017040 con 0x562b489c6380 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc1affd700 1 -- 172.21.15.46:0/2912487769 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fdc24004230 con 0x562b489c6380 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2912487769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc3409d410 con 0x562b489c6380 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc1affd700 1 -- 172.21.15.46:0/2912487769 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc2401c5b0 con 0x562b489c6380 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc4dec2b80 10 client.26576 fetch_fsmap learned FSMap version 38 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2912487769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x562b4879ecd0 con 0x562b489c6380 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc1affd700 1 -- 172.21.15.46:0/2912487769 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc3409d410 con 0x562b489c6380 2024-03-18T20:39:48.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc1affd700 1 -- 172.21.15.46:0/2912487769 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 38) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fdc240038b0 con 0x562b489c6380 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc4dec2b80 10 client.26576 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc4dec2b80 10 client.26576 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc4dec2b80 20 client.26576 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x562b48a39f80 0x562b48a3c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc4dec2b80 4 client.26576 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2912487769 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562b48767cf0 con 0x562b48a39f80 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.494+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x562b48a39f80 0x562b48a3c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc3b7fe700 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x562b48a39f80 0x562b48a3c3a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82f662b36fbc729e server_cookie=8fe0f9073678f026 in_seq=0 out_seq=0 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc1affd700 10 client.26576 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc1affd700 1 -- 172.21.15.46:0/2912487769 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562b48767cf0 con 0x562b48a39f80 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc1affd700 10 client.26576 handle_command_reply: tid=0 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc4dec2b80 1 client.26576 shutdown 2024-03-18T20:39:48.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2912487769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489c6380 msgr2=0x562b489da970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489c6380 0x562b489da970 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x562b489e7fe0 tx=0x7fdc24011880 comp rx=0 tx=0).stop 2024-03-18T20:39:48.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2912487769 shutdown_connections 2024-03-18T20:39:48.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x562b48a39f80 0x562b48a3c3a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b489c5a60 0x562b489da430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc1affd700 0 client.26576 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:39:48.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b489d5630 0x562b48a38140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc4dec2b80 1 --2- 172.21.15.46:0/2912487769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b489c6380 0x562b489da970 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2912487769 >> 172.21.15.46:0/2912487769 conn(0x562b4893c220 msgr2=0x562b489c1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2912487769 shutdown_connections 2024-03-18T20:39:48.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc4dec2b80 1 -- 172.21.15.46:0/2912487769 wait complete. 2024-03-18T20:39:48.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.495+0000 7fdc4dec2b80 20 client.26576 trim_cache size 0 max 16384 2024-03-18T20:39:48.506 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:39:48.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.496+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2825139561 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdc280a1290 msgr2=0x7fdc280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.496+0000 7fdc41d2d700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdc280a1290 0x7fdc280a3750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c006010 tx=0x7fdc2c01a040 comp rx=0 tx=0).stop 2024-03-18T20:39:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.496+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2825139561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc3c105ce0 msgr2=0x7fdc3c19bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.496+0000 7fdc41d2d700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc3c105ce0 0x7fdc3c19bad0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdc34007ee0 tx=0x7fdc3400d3b0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.496+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2825139561 shutdown_connections 2024-03-18T20:39:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.496+0000 7fdc41d2d700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdc280a1290 0x7fdc280a3750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.496+0000 7fdc41d2d700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc3c105ce0 0x7fdc3c19bad0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.496+0000 7fdc41d2d700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc3c067e30 0x7fdc3c1971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.496+0000 7fdc41d2d700 1 --2- 172.21.15.46:0/2825139561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc3c0687f0 0x7fdc3c197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.496+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2825139561 >> 172.21.15.46:0/2825139561 conn(0x7fdc3c076d40 msgr2=0x7fdc3c0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.496+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2825139561 shutdown_connections 2024-03-18T20:39:48.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.496+0000 7fdc41d2d700 1 -- 172.21.15.46:0/2825139561 wait complete. 2024-03-18T20:39:48.512 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:39:48.513 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:39:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:48 smithi155 ceph-mon[48344]: pgmap v114: 97 pgs: 97 active+clean; 42 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 441 KiB/s wr, 57 op/s 2024-03-18T20:39:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:48 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:48 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1070811139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:39:48.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 92992 1.34 MB/sec execute 43 sec latency 1603.536 ms 2024-03-18T20:39:48.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.691+0000 7f2bfb649700 1 -- 172.21.15.46:0/639378289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf4104f90 msgr2=0x7f2bf410cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.691+0000 7f2bfb649700 1 --2- 172.21.15.46:0/639378289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf4104f90 0x7f2bf410cb10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00b0a0 tx=0x7f2bec00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.691+0000 7f2bfb649700 1 -- 172.21.15.46:0/639378289 shutdown_connections 2024-03-18T20:39:48.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.691+0000 7f2bfb649700 1 --2- 172.21.15.46:0/639378289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf4104f90 0x7f2bf410cb10 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.691+0000 7f2bfb649700 1 --2- 172.21.15.46:0/639378289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bf40ffe70 0x7f2bf4100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.691+0000 7f2bfb649700 1 --2- 172.21.15.46:0/639378289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf4100820 0x7f2bf4104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.691+0000 7f2bfb649700 1 -- 172.21.15.46:0/639378289 >> 172.21.15.46:0/639378289 conn(0x7f2bf4076d80 msgr2=0x7f2bf4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.691+0000 7f2bfb649700 1 -- 172.21.15.46:0/639378289 shutdown_connections 2024-03-18T20:39:48.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.691+0000 7f2bfb649700 1 -- 172.21.15.46:0/639378289 wait complete. 2024-03-18T20:39:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bfb649700 1 Processor -- start 2024-03-18T20:39:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bfb649700 1 -- start start 2024-03-18T20:39:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bfb649700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf40ffe70 0x7f2bf41994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bfb649700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bf4104f90 0x7f2bf4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bfb649700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf419dda0 0x7f2bf419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bfb649700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf419e820 con 0x7f2bf419dda0 2024-03-18T20:39:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bfb649700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf419e990 con 0x7f2bf40ffe70 2024-03-18T20:39:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bfb649700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf419eb50 con 0x7f2bf4104f90 2024-03-18T20:39:48.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bf9be6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf419dda0 0x7f2bf419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bf9be6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf419dda0 0x7f2bf419e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52082/0 (socket says 172.21.15.46:52082) 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bf9be6700 1 -- 172.21.15.46:0/3335350110 learned_addr learned my addr 172.21.15.46:0/3335350110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf40ffe70 0x7f2bf41994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bf8be4700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bf4104f90 0x7f2bf4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bf9be6700 1 -- 172.21.15.46:0/3335350110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bf4104f90 msgr2=0x7f2bf4199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bf9be6700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bf4104f90 0x7f2bf4199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bf9be6700 1 -- 172.21.15.46:0/3335350110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf40ffe70 msgr2=0x7f2bf41994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bf9be6700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf40ffe70 0x7f2bf41994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.692+0000 7f2bf9be6700 1 -- 172.21.15.46:0/3335350110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec009d00 con 0x7f2bf419dda0 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.693+0000 7f2bf9be6700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf419dda0 0x7f2bf419e220 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2bec004c60 tx=0x7f2bec004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.693+0000 7f2bea7fc700 1 -- 172.21.15.46:0/3335350110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec0079d0 con 0x7f2bf419dda0 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.693+0000 7f2bea7fc700 1 -- 172.21.15.46:0/3335350110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bec004e70 con 0x7f2bf419dda0 2024-03-18T20:39:48.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.693+0000 7f2bfb649700 1 -- 172.21.15.46:0/3335350110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf41a6050 con 0x7f2bf419dda0 2024-03-18T20:39:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.693+0000 7f2bea7fc700 1 -- 172.21.15.46:0/3335350110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec0116c0 con 0x7f2bf419dda0 2024-03-18T20:39:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.693+0000 7f2bfb649700 1 -- 172.21.15.46:0/3335350110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf41a64d0 con 0x7f2bf419dda0 2024-03-18T20:39:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.695+0000 7f2bea7fc700 1 -- 172.21.15.46:0/3335350110 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2bec020070 con 0x7f2bf419dda0 2024-03-18T20:39:48.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.696+0000 7f2bea7fc700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2be00a1290 0x7f2be00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.696+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2be00a1290 0x7f2be00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.696+0000 7f2bea7fc700 1 -- 172.21.15.46:0/3335350110 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2bec0cde10 con 0x7f2bf419dda0 2024-03-18T20:39:48.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.697+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2be00a1290 0x7f2be00a3750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0005950 tx=0x7f2bf000b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.697+0000 7f2c077deb80 1 Processor -- start 2024-03-18T20:39:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.697+0000 7f2c077deb80 1 -- start start 2024-03-18T20:39:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.697+0000 7f2c077deb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e069a60 0x55ff2e069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.697+0000 7f2c077deb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e06a380 0x55ff2e074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2c077deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e075220 0x55ff2e077610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2c077deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2dfc3990 con 0x55ff2e075220 2024-03-18T20:39:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2c077deb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2dfc1350 con 0x55ff2e069a60 2024-03-18T20:39:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2c077deb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2dfbcc40 con 0x55ff2e06a380 2024-03-18T20:39:48.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf9be6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e075220 0x55ff2e077610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf9be6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e075220 0x55ff2e077610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52088/0 (socket says 172.21.15.46:52088) 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf9be6700 1 -- 172.21.15.46:0/555999100 learned_addr learned my addr 172.21.15.46:0/555999100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/555999100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e069a60 0x55ff2e069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf8be4700 1 --2- 172.21.15.46:0/555999100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e06a380 0x55ff2e074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf9be6700 1 -- 172.21.15.46:0/555999100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e06a380 msgr2=0x55ff2e074ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf9be6700 1 --2- 172.21.15.46:0/555999100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e06a380 0x55ff2e074ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf9be6700 1 -- 172.21.15.46:0/555999100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e069a60 msgr2=0x55ff2e069e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf9be6700 1 --2- 172.21.15.46:0/555999100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e069a60 0x55ff2e069e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf9be6700 1 -- 172.21.15.46:0/555999100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff2de3e3b0 con 0x55ff2e075220 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf8be4700 1 --2- 172.21.15.46:0/555999100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e06a380 0x55ff2e074ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/555999100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e069a60 0x55ff2e069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bf9be6700 1 --2- 172.21.15.46:0/555999100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e075220 0x55ff2e077610 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0975d0 tx=0x7f2bec0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb62263de4292de server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bdf7fe700 1 -- 172.21.15.46:0/555999100 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec09b7d0 con 0x55ff2e075220 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bdf7fe700 1 -- 172.21.15.46:0/555999100 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bec09e070 con 0x55ff2e075220 2024-03-18T20:39:48.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.698+0000 7f2bdf7fe700 1 -- 172.21.15.46:0/555999100 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec09b330 con 0x55ff2e075220 2024-03-18T20:39:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.699+0000 7f2c077deb80 1 -- 172.21.15.46:0/555999100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e075220 msgr2=0x55ff2e077610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.699+0000 7f2c077deb80 1 --2- 172.21.15.46:0/555999100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e075220 0x55ff2e077610 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0975d0 tx=0x7f2bec0099e0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.699+0000 7f2c077deb80 1 -- 172.21.15.46:0/555999100 shutdown_connections 2024-03-18T20:39:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.699+0000 7f2c077deb80 1 --2- 172.21.15.46:0/555999100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e075220 0x55ff2e077610 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.699+0000 7f2c077deb80 1 --2- 172.21.15.46:0/555999100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e069a60 0x55ff2e069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.699+0000 7f2c077deb80 1 --2- 172.21.15.46:0/555999100 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e06a380 0x55ff2e074ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.699+0000 7f2c077deb80 1 -- 172.21.15.46:0/555999100 >> 172.21.15.46:0/555999100 conn(0x55ff2dfe0220 msgr2=0x55ff2e066020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.699+0000 7f2c077deb80 1 -- 172.21.15.46:0/555999100 shutdown_connections 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.699+0000 7f2c077deb80 1 -- 172.21.15.46:0/555999100 wait complete. 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2c077deb80 1 -- start start 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2c077deb80 1 Processor -- start 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2c077deb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e069a60 0x55ff2e075600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2c077deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e06a380 0x55ff2e075b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2c077deb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e080680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2c077deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2df58d80 con 0x55ff2e06a380 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2c077deb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2df58a20 con 0x55ff2e069a60 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2c077deb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2de406a0 con 0x55ff2e075220 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2c077deb80 10 client.0 _ll_register_callbacks cb 0x55ff2ddc43a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2bf8be4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e06a380 0x55ff2e075b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2bf8be4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e06a380 0x55ff2e075b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52092/0 (socket says 172.21.15.46:52092) 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2bf8be4700 1 -- 172.21.15.46:0/3258106341 learned_addr learned my addr 172.21.15.46:0/3258106341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.700+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e069a60 0x55ff2e075600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2bf9be6700 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e080680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2bf8be4700 1 -- 172.21.15.46:0/3258106341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 msgr2=0x55ff2e080680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2bf8be4700 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e080680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2bf8be4700 1 -- 172.21.15.46:0/3258106341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e069a60 msgr2=0x55ff2e075600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2bf8be4700 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e069a60 0x55ff2e075600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2bf8be4700 1 -- 172.21.15.46:0/3258106341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf0010c50 con 0x55ff2e06a380 2024-03-18T20:39:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2bf9be6700 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e080680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2bf8be4700 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e06a380 0x55ff2e075b40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2be40116f0 tx=0x7f2be4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c3e1ba71256e41 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2bdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2bdcff9700 1 -- 172.21.15.46:0/3258106341 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4016070 con 0x55ff2e06a380 2024-03-18T20:39:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2bdcff9700 1 -- 172.21.15.46:0/3258106341 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2be4004320 con 0x55ff2e06a380 2024-03-18T20:39:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2c077deb80 1 -- 172.21.15.46:0/3258106341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff2e05dad0 con 0x55ff2e06a380 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2bdcff9700 1 -- 172.21.15.46:0/3258106341 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2be4004520 con 0x55ff2e06a380 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2c077deb80 10 client.16857 fetch_fsmap learned FSMap version 38 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.701+0000 7f2c077deb80 1 -- 172.21.15.46:0/3258106341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f2bf41a64d0 con 0x55ff2e06a380 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.702+0000 7f2bdcff9700 1 -- 172.21.15.46:0/3258106341 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2be4027020 con 0x55ff2e06a380 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.702+0000 7f2bdcff9700 1 -- 172.21.15.46:0/3258106341 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 38) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2be400f8a0 con 0x55ff2e06a380 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.702+0000 7f2c077deb80 10 client.16857 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.702+0000 7f2c077deb80 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.702+0000 7f2c077deb80 20 client.16857 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.702+0000 7f2c077deb80 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ff2e084b90 0x55ff2e086fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.702+0000 7f2c077deb80 4 client.16857 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.702+0000 7f2c077deb80 1 -- 172.21.15.46:0/3258106341 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff2dfb5700 con 0x55ff2e084b90 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.702+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ff2e084b90 0x55ff2e086fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.703+0000 7f2bdcff9700 10 client.16857 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.703+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ff2e084b90 0x55ff2e086fb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61ad0c9bc9a2e51 server_cookie=3bf4e82827989602 in_seq=0 out_seq=0 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.703+0000 7f2bdcff9700 1 -- 172.21.15.46:0/3258106341 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ff2dfb5700 con 0x55ff2e084b90 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.703+0000 7f2bdcff9700 10 client.16857 handle_command_reply: tid=0 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.703+0000 7f2c077deb80 1 client.16857 shutdown 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.704+0000 7f2c077deb80 1 -- 172.21.15.46:0/3258106341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e06a380 msgr2=0x55ff2e075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.704+0000 7f2c077deb80 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e06a380 0x55ff2e075b40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2be40116f0 tx=0x7f2be4011a00 comp rx=0 tx=0).stop 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.704+0000 7f2c077deb80 1 -- 172.21.15.46:0/3258106341 shutdown_connections 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.704+0000 7f2c077deb80 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ff2e084b90 0x55ff2e086fb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.704+0000 7f2c077deb80 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e080680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.704+0000 7f2c077deb80 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e069a60 0x55ff2e075600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.704+0000 7f2bdcff9700 0 client.16857 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.704+0000 7f2c077deb80 1 --2- 172.21.15.46:0/3258106341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e06a380 0x55ff2e075b40 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.704+0000 7f2c077deb80 1 -- 172.21.15.46:0/3258106341 >> 172.21.15.46:0/3258106341 conn(0x55ff2dfe0220 msgr2=0x55ff2e065c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.704+0000 7f2c077deb80 1 -- 172.21.15.46:0/3258106341 shutdown_connections 2024-03-18T20:39:48.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.704+0000 7f2c077deb80 1 -- 172.21.15.46:0/3258106341 wait complete. 2024-03-18T20:39:48.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.704+0000 7f2c077deb80 20 client.16857 trim_cache size 0 max 16384 2024-03-18T20:39:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2c077deb80 1 Processor -- start 2024-03-18T20:39:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2c077deb80 1 -- start start 2024-03-18T20:39:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2c077deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 0x55ff2e07a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2c077deb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e06a380 0x55ff2e072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2c077deb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2c077deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2e06ad30 con 0x55ff2e069a60 2024-03-18T20:39:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2c077deb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2e0738d0 con 0x55ff2e06a380 2024-03-18T20:39:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2c077deb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2e067230 con 0x55ff2e075220 2024-03-18T20:39:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2bf93e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 0x55ff2e07a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2bf93e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 0x55ff2e07a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52100/0 (socket says 172.21.15.46:52100) 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2bf93e5700 1 -- 172.21.15.46:0/1008720177 learned_addr learned my addr 172.21.15.46:0/1008720177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2bf8be4700 1 --2- 172.21.15.46:0/1008720177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e06a380 0x55ff2e072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2bf9be6700 1 --2- 172.21.15.46:0/1008720177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2bf93e5700 1 -- 172.21.15.46:0/1008720177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 msgr2=0x55ff2e073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/1008720177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2bf93e5700 1 -- 172.21.15.46:0/1008720177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e06a380 msgr2=0x55ff2e072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/1008720177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e06a380 0x55ff2e072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2bf93e5700 1 -- 172.21.15.46:0/1008720177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff2ddeab60 con 0x55ff2e069a60 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2bf9be6700 1 --2- 172.21.15.46:0/1008720177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.718+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/1008720177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 0x55ff2e07a580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0010a40 tx=0x7f2bf0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf1e25d6ec4bbad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.719+0000 7f2bdffff700 1 -- 172.21.15.46:0/1008720177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0024040 con 0x55ff2e069a60 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.719+0000 7f2bdffff700 1 -- 172.21.15.46:0/1008720177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bf0015070 con 0x55ff2e069a60 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.719+0000 7f2bdffff700 1 -- 172.21.15.46:0/1008720177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf001f830 con 0x55ff2e069a60 2024-03-18T20:39:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.719+0000 7f2c077deb80 1 -- 172.21.15.46:0/1008720177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 msgr2=0x55ff2e07a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.719+0000 7f2c077deb80 1 --2- 172.21.15.46:0/1008720177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 0x55ff2e07a580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0010a40 tx=0x7f2bf0013000 comp rx=0 tx=0).stop 2024-03-18T20:39:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.719+0000 7f2c077deb80 1 -- 172.21.15.46:0/1008720177 shutdown_connections 2024-03-18T20:39:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.719+0000 7f2c077deb80 1 --2- 172.21.15.46:0/1008720177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 0x55ff2e07a580 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.719+0000 7f2c077deb80 1 --2- 172.21.15.46:0/1008720177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.719+0000 7f2c077deb80 1 --2- 172.21.15.46:0/1008720177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e06a380 0x55ff2e072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.719+0000 7f2c077deb80 1 -- 172.21.15.46:0/1008720177 >> 172.21.15.46:0/1008720177 conn(0x55ff2dfe0220 msgr2=0x55ff2e06f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.719+0000 7f2c077deb80 1 -- 172.21.15.46:0/1008720177 shutdown_connections 2024-03-18T20:39:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.719+0000 7f2c077deb80 1 -- 172.21.15.46:0/1008720177 wait complete. 2024-03-18T20:39:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.720+0000 7f2c077deb80 1 -- start start 2024-03-18T20:39:48.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.720+0000 7f2c077deb80 1 Processor -- start 2024-03-18T20:39:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.720+0000 7f2c077deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 0x55ff2e0d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.720+0000 7f2c077deb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e06a380 0x55ff2e0d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.720+0000 7f2c077deb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e0784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.720+0000 7f2c077deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2e078bf0 con 0x55ff2e069a60 2024-03-18T20:39:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.720+0000 7f2c077deb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2e078d60 con 0x55ff2e06a380 2024-03-18T20:39:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.720+0000 7f2c077deb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff2e078f20 con 0x55ff2e075220 2024-03-18T20:39:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.720+0000 7f2c077deb80 10 client.0 _ll_register_callbacks cb 0x7f2be4009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.720+0000 7f2bf93e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 0x55ff2e0d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.720+0000 7f2bf93e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 0x55ff2e0d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52106/0 (socket says 172.21.15.46:52106) 2024-03-18T20:39:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.720+0000 7f2bf93e5700 1 -- 172.21.15.46:0/1210027658 learned_addr learned my addr 172.21.15.46:0/1210027658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2bf9be6700 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e0784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2bf8be4700 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e06a380 0x55ff2e0d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2bf93e5700 1 -- 172.21.15.46:0/1210027658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 msgr2=0x55ff2e0784e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e0784e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2bf93e5700 1 -- 172.21.15.46:0/1210027658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e06a380 msgr2=0x55ff2e0d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e06a380 0x55ff2e0d21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2bf93e5700 1 -- 172.21.15.46:0/1210027658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf00115e0 con 0x55ff2e069a60 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2bf93e5700 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 0x55ff2e0d1c80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0029460 tx=0x7f2bf0029490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a88a28576c170e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2bdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2bdcff9700 1 -- 172.21.15.46:0/1210027658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0024070 con 0x55ff2e069a60 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2bdcff9700 1 -- 172.21.15.46:0/1210027658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2bf0017070 con 0x55ff2e069a60 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2c077deb80 1 -- 172.21.15.46:0/1210027658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bec09e070 con 0x55ff2e069a60 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2bdcff9700 1 -- 172.21.15.46:0/1210027658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bf0029d10 con 0x55ff2e069a60 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2c077deb80 10 client.16863 fetch_fsmap learned FSMap version 38 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.721+0000 7f2c077deb80 1 -- 172.21.15.46:0/1210027658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55ff2df82ea0 con 0x55ff2e069a60 2024-03-18T20:39:48.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.722+0000 7f2bdcff9700 1 -- 172.21.15.46:0/1210027658 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bec09e070 con 0x55ff2e069a60 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.722+0000 7f2bdcff9700 1 -- 172.21.15.46:0/1210027658 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 38) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2bf0043070 con 0x55ff2e069a60 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.722+0000 7f2c077deb80 10 client.16863 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.722+0000 7f2c077deb80 10 client.16863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.722+0000 7f2c077deb80 20 client.16863 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.722+0000 7f2c077deb80 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ff2e0d9a70 0x55ff2e0dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.722+0000 7f2c077deb80 4 client.16863 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.722+0000 7f2c077deb80 1 -- 172.21.15.46:0/1210027658 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ff2de0bcf0 con 0x55ff2e0d9a70 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.722+0000 7f2bf8be4700 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ff2e0d9a70 0x55ff2e0dbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.722+0000 7f2bdcff9700 10 client.16863 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.722+0000 7f2bf8be4700 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ff2e0d9a70 0x55ff2e0dbe90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b244a068a7a97b server_cookie=918ce4632b9b0e75 in_seq=0 out_seq=0 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2bdcff9700 1 -- 172.21.15.46:0/1210027658 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ff2de0bcf0 con 0x55ff2e0d9a70 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2bdcff9700 10 client.16863 handle_command_reply: tid=0 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2c077deb80 1 client.16863 shutdown 2024-03-18T20:39:48.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2c077deb80 1 -- 172.21.15.46:0/1210027658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 msgr2=0x55ff2e0d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2c077deb80 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 0x55ff2e0d1c80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0029460 tx=0x7f2bf0029490 comp rx=0 tx=0).stop 2024-03-18T20:39:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2c077deb80 1 -- 172.21.15.46:0/1210027658 shutdown_connections 2024-03-18T20:39:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2c077deb80 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ff2e0d9a70 0x55ff2e0dbe90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2c077deb80 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff2e069a60 0x55ff2e0d1c80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2c077deb80 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff2e075220 0x55ff2e0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2bdcff9700 0 client.16863 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2c077deb80 1 --2- 172.21.15.46:0/1210027658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2e06a380 0x55ff2e0d21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2c077deb80 1 -- 172.21.15.46:0/1210027658 >> 172.21.15.46:0/1210027658 conn(0x55ff2dfe0220 msgr2=0x55ff2e072dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2c077deb80 1 -- 172.21.15.46:0/1210027658 shutdown_connections 2024-03-18T20:39:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2c077deb80 1 -- 172.21.15.46:0/1210027658 wait complete. 2024-03-18T20:39:48.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.723+0000 7f2c077deb80 20 client.16863 trim_cache size 0 max 16384 2024-03-18T20:39:48.733 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:39:48.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.724+0000 7f2bfb649700 1 -- 172.21.15.46:0/3335350110 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2be00a1290 msgr2=0x7f2be00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.724+0000 7f2bfb649700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2be00a1290 0x7f2be00a3750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0005950 tx=0x7f2bf000b410 comp rx=0 tx=0).stop 2024-03-18T20:39:48.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.724+0000 7f2bfb649700 1 -- 172.21.15.46:0/3335350110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf419dda0 msgr2=0x7f2bf419e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.724+0000 7f2bfb649700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf419dda0 0x7f2bf419e220 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2bec004c60 tx=0x7f2bec004c90 comp rx=0 tx=0).stop 2024-03-18T20:39:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.724+0000 7f2bfb649700 1 -- 172.21.15.46:0/3335350110 shutdown_connections 2024-03-18T20:39:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.724+0000 7f2bfb649700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2be00a1290 0x7f2be00a3750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.724+0000 7f2bfb649700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf419dda0 0x7f2bf419e220 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.724+0000 7f2bfb649700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bf40ffe70 0x7f2bf41994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.724+0000 7f2bfb649700 1 --2- 172.21.15.46:0/3335350110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bf4104f90 0x7f2bf4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.724+0000 7f2bfb649700 1 -- 172.21.15.46:0/3335350110 >> 172.21.15.46:0/3335350110 conn(0x7f2bf4076d80 msgr2=0x7f2bf40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.724+0000 7f2bfb649700 1 -- 172.21.15.46:0/3335350110 shutdown_connections 2024-03-18T20:39:48.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:48.724+0000 7f2bfb649700 1 -- 172.21.15.46:0/3335350110 wait complete. 2024-03-18T20:39:49.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:49 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:49 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:49 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:49.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 92992 1.31 MB/sec execute 44 sec latency 2603.638 ms 2024-03-18T20:39:49.741 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:39:49.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:39:49.741 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:39:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.890+0000 7f813e3d1700 1 -- 172.21.15.46:0/451806424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81381005c0 msgr2=0x7f81381009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.890+0000 7f813e3d1700 1 --2- 172.21.15.46:0/451806424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81381005c0 0x7f81381009a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8128009990 tx=0x7f8128009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.890+0000 7f813e3d1700 1 -- 172.21.15.46:0/451806424 shutdown_connections 2024-03-18T20:39:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.890+0000 7f813e3d1700 1 --2- 172.21.15.46:0/451806424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81381005c0 0x7f81381009a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.890+0000 7f813e3d1700 1 --2- 172.21.15.46:0/451806424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8138104f60 0x7f813810cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.890+0000 7f813e3d1700 1 --2- 172.21.15.46:0/451806424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8138100f70 0x7f8138104880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.890+0000 7f813e3d1700 1 -- 172.21.15.46:0/451806424 >> 172.21.15.46:0/451806424 conn(0x7f81380761e0 msgr2=0x7f81380fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:49.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.890+0000 7f813e3d1700 1 -- 172.21.15.46:0/451806424 shutdown_connections 2024-03-18T20:39:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.890+0000 7f813e3d1700 1 -- 172.21.15.46:0/451806424 wait complete. 2024-03-18T20:39:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.891+0000 7f813e3d1700 1 Processor -- start 2024-03-18T20:39:49.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.891+0000 7f813e3d1700 1 -- start start 2024-03-18T20:39:49.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.891+0000 7f813e3d1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381005c0 0x7f8138197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.891+0000 7f813e3d1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8138104f60 0x7f8138197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.891+0000 7f813e3d1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f813819bb30 0x7f813819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.891+0000 7f813e3d1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813819c5b0 con 0x7f813819bb30 2024-03-18T20:39:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.891+0000 7f813e3d1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813819c720 con 0x7f81381005c0 2024-03-18T20:39:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.891+0000 7f813e3d1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813819c8e0 con 0x7f8138104f60 2024-03-18T20:39:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f813c96e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f813819bb30 0x7f813819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f813c96e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f813819bb30 0x7f813819bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52118/0 (socket says 172.21.15.46:52118) 2024-03-18T20:39:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f813c96e700 1 -- 172.21.15.46:0/1666142148 learned_addr learned my addr 172.21.15.46:0/1666142148 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f81377fe700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8138104f60 0x7f8138197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f8137fff700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381005c0 0x7f8138197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f813c96e700 1 -- 172.21.15.46:0/1666142148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8138104f60 msgr2=0x7f8138197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:49.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f813c96e700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8138104f60 0x7f8138197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f813c96e700 1 -- 172.21.15.46:0/1666142148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381005c0 msgr2=0x7f8138197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f813c96e700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381005c0 0x7f8138197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f813c96e700 1 -- 172.21.15.46:0/1666142148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81280095f0 con 0x7f813819bb30 2024-03-18T20:39:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f813c96e700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f813819bb30 0x7f813819bfb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f813000ef30 tx=0x7f813000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f81357fa700 1 -- 172.21.15.46:0/1666142148 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130018070 con 0x7f813819bb30 2024-03-18T20:39:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f813e3d1700 1 -- 172.21.15.46:0/1666142148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81381a3de0 con 0x7f813819bb30 2024-03-18T20:39:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f81357fa700 1 -- 172.21.15.46:0/1666142148 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f813000f040 con 0x7f813819bb30 2024-03-18T20:39:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f81357fa700 1 -- 172.21.15.46:0/1666142148 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813000cd00 con 0x7f813819bb30 2024-03-18T20:39:49.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.892+0000 7f813e3d1700 1 -- 172.21.15.46:0/1666142148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81381a4270 con 0x7f813819bb30 2024-03-18T20:39:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.895+0000 7f81357fa700 1 -- 172.21.15.46:0/1666142148 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8130003a00 con 0x7f813819bb30 2024-03-18T20:39:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.895+0000 7f81357fa700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f81200a1290 0x7f81200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.895+0000 7f8137fff700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f81200a1290 0x7f81200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.895+0000 7f81357fa700 1 -- 172.21.15.46:0/1666142148 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f81300cc7b0 con 0x7f813819bb30 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.896+0000 7f8137fff700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f81200a1290 0x7f81200a3750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f812800b2e0 tx=0x7f8128005fd0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.896+0000 7f814a566b80 1 Processor -- start 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.896+0000 7f814a566b80 1 -- start start 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.896+0000 7f814a566b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572eba8cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.896+0000 7f814a566b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572eba97d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.896+0000 7f814a566b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba9a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.896+0000 7f814a566b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572eb9e6a10 con 0x5572eba8d400 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.896+0000 7f814a566b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572eb9e43d0 con 0x5572eba8cae0 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.896+0000 7f814a566b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572eb9dfcc0 con 0x5572eba982a0 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.896+0000 7f81377fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572eba97d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.896+0000 7f81377fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572eba97d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52134/0 (socket says 172.21.15.46:52134) 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.896+0000 7f81377fe700 1 -- 172.21.15.46:0/3317530867 learned_addr learned my addr 172.21.15.46:0/3317530867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f813c96e700 1 --2- 172.21.15.46:0/3317530867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba9a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f8137fff700 1 --2- 172.21.15.46:0/3317530867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572eba8cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f81377fe700 1 -- 172.21.15.46:0/3317530867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 msgr2=0x5572eba9a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f81377fe700 1 --2- 172.21.15.46:0/3317530867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba9a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f81377fe700 1 -- 172.21.15.46:0/3317530867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 msgr2=0x5572eba8cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f81377fe700 1 --2- 172.21.15.46:0/3317530867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572eba8cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f81377fe700 1 -- 172.21.15.46:0/3317530867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572eb861430 con 0x5572eba8d400 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f8137fff700 1 --2- 172.21.15.46:0/3317530867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572eba8cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f813c96e700 1 --2- 172.21.15.46:0/3317530867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba9a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f81377fe700 1 --2- 172.21.15.46:0/3317530867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572eba97d60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f812c009cb0 tx=0x7f812c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4a220c7adcac0c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f811e7fc700 1 -- 172.21.15.46:0/3317530867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c011e30 con 0x5572eba8d400 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f811e7fc700 1 -- 172.21.15.46:0/3317530867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f812c004510 con 0x5572eba8d400 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.897+0000 7f811e7fc700 1 -- 172.21.15.46:0/3317530867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c005280 con 0x5572eba8d400 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.898+0000 7f814a566b80 1 -- 172.21.15.46:0/3317530867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 msgr2=0x5572eba97d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.898+0000 7f814a566b80 1 --2- 172.21.15.46:0/3317530867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572eba97d60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f812c009cb0 tx=0x7f812c011440 comp rx=0 tx=0).stop 2024-03-18T20:39:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.898+0000 7f814a566b80 1 -- 172.21.15.46:0/3317530867 shutdown_connections 2024-03-18T20:39:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.898+0000 7f814a566b80 1 --2- 172.21.15.46:0/3317530867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba9a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.898+0000 7f814a566b80 1 --2- 172.21.15.46:0/3317530867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572eba8cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.898+0000 7f814a566b80 1 --2- 172.21.15.46:0/3317530867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572eba97d60 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.898+0000 7f814a566b80 1 -- 172.21.15.46:0/3317530867 >> 172.21.15.46:0/3317530867 conn(0x5572eba032a0 msgr2=0x5572eba890a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.898+0000 7f814a566b80 1 -- 172.21.15.46:0/3317530867 shutdown_connections 2024-03-18T20:39:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.898+0000 7f814a566b80 1 -- 172.21.15.46:0/3317530867 wait complete. 2024-03-18T20:39:49.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.898+0000 7f814a566b80 1 -- start start 2024-03-18T20:39:49.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f814a566b80 1 Processor -- start 2024-03-18T20:39:49.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f814a566b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572eba97c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f814a566b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572eba94fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f814a566b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba95610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f814a566b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572eb97be00 con 0x5572eba8d400 2024-03-18T20:39:49.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f814a566b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572eb97baa0 con 0x5572eba8cae0 2024-03-18T20:39:49.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f814a566b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572eb863720 con 0x5572eba982a0 2024-03-18T20:39:49.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f814a566b80 10 client.0 _ll_register_callbacks cb 0x5572eb7e7420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:49.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f81377fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572eba94fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f81377fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572eba94fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52150/0 (socket says 172.21.15.46:52150) 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f81377fe700 1 -- 172.21.15.46:0/593978029 learned_addr learned my addr 172.21.15.46:0/593978029 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f813c96e700 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba95610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f8137fff700 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572eba97c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f81377fe700 1 -- 172.21.15.46:0/593978029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 msgr2=0x5572eba95610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f81377fe700 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba95610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f81377fe700 1 -- 172.21.15.46:0/593978029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 msgr2=0x5572eba97c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.899+0000 7f81377fe700 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572eba97c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.900+0000 7f81377fe700 1 -- 172.21.15.46:0/593978029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c0098e0 con 0x5572eba8d400 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.900+0000 7f81377fe700 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572eba94fe0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f812c005d60 tx=0x7f812c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3331d0af4bcdf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.900+0000 7f81177fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.900+0000 7f81177fe700 1 -- 172.21.15.46:0/593978029 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c003680 con 0x5572eba8d400 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.900+0000 7f81177fe700 1 -- 172.21.15.46:0/593978029 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f812c0037e0 con 0x5572eba8d400 2024-03-18T20:39:49.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.900+0000 7f814a566b80 1 -- 172.21.15.46:0/593978029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572eba80b50 con 0x5572eba8d400 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.900+0000 7f81177fe700 1 -- 172.21.15.46:0/593978029 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f812c0039a0 con 0x5572eba8d400 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.900+0000 7f814a566b80 10 client.16875 fetch_fsmap learned FSMap version 38 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.900+0000 7f814a566b80 1 -- 172.21.15.46:0/593978029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5572eb80dbe0 con 0x5572eba8d400 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.900+0000 7f81177fe700 1 -- 172.21.15.46:0/593978029 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5572eba80b50 con 0x5572eba8d400 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.900+0000 7f81177fe700 1 -- 172.21.15.46:0/593978029 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 38) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f812c022400 con 0x5572eba8d400 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.900+0000 7f814a566b80 10 client.16875 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.901+0000 7f814a566b80 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.901+0000 7f814a566b80 20 client.16875 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.901+0000 7f814a566b80 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5572ebaabfd0 0x5572ebaae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.901+0000 7f814a566b80 4 client.16875 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.901+0000 7f814a566b80 1 -- 172.21.15.46:0/593978029 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572eb9d8780 con 0x5572ebaabfd0 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.901+0000 7f8137fff700 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5572ebaabfd0 0x5572ebaae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.901+0000 7f81177fe700 10 client.16875 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.901+0000 7f8137fff700 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5572ebaabfd0 0x5572ebaae3f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f222a066b61a9ba2 server_cookie=ad228025b38efa1f in_seq=0 out_seq=0 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f81177fe700 1 -- 172.21.15.46:0/593978029 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5572eb9d8780 con 0x5572ebaabfd0 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f81177fe700 10 client.16875 handle_command_reply: tid=0 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f814a566b80 1 client.16875 shutdown 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f814a566b80 1 -- 172.21.15.46:0/593978029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 msgr2=0x5572eba94fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f814a566b80 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572eba94fe0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f812c005d60 tx=0x7f812c005d90 comp rx=0 tx=0).stop 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f814a566b80 1 -- 172.21.15.46:0/593978029 shutdown_connections 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f814a566b80 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5572ebaabfd0 0x5572ebaae3f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f814a566b80 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba95610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f814a566b80 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572eba97c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f81177fe700 0 client.16875 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f814a566b80 1 --2- 172.21.15.46:0/593978029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572eba94fe0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f814a566b80 1 -- 172.21.15.46:0/593978029 >> 172.21.15.46:0/593978029 conn(0x5572eba032a0 msgr2=0x5572eba88c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f814a566b80 1 -- 172.21.15.46:0/593978029 shutdown_connections 2024-03-18T20:39:49.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f814a566b80 1 -- 172.21.15.46:0/593978029 wait complete. 2024-03-18T20:39:49.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.902+0000 7f814a566b80 20 client.16875 trim_cache size 0 max 16384 2024-03-18T20:39:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.921+0000 7f814a566b80 1 Processor -- start 2024-03-18T20:39:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.926+0000 7f814a566b80 1 -- start start 2024-03-18T20:39:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.926+0000 7f814a566b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8cae0 0x5572eba8fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.927+0000 7f814a566b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8d400 0x5572ebaa81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.927+0000 7f814a566b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba92260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.927+0000 7f814a566b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572eba85660 con 0x5572eba8cae0 2024-03-18T20:39:49.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.927+0000 7f814a566b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572eba8ddb0 con 0x5572eba8d400 2024-03-18T20:39:49.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.927+0000 7f814a566b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ebaa8d80 con 0x5572eba982a0 2024-03-18T20:39:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.927+0000 7f813c96e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba92260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.927+0000 7f813c96e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba92260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37582/0 (socket says 172.21.15.46:37582) 2024-03-18T20:39:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.927+0000 7f813c96e700 1 -- 172.21.15.46:0/3668917125 learned_addr learned my addr 172.21.15.46:0/3668917125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.928+0000 7f8137fff700 1 --2- 172.21.15.46:0/3668917125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8cae0 0x5572eba8fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.928+0000 7f8137fff700 1 -- 172.21.15.46:0/3668917125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 msgr2=0x5572eba92260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.928+0000 7f8137fff700 1 --2- 172.21.15.46:0/3668917125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba92260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.928+0000 7f8137fff700 1 -- 172.21.15.46:0/3668917125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8d400 msgr2=0x5572ebaa81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.928+0000 7f8137fff700 1 --2- 172.21.15.46:0/3668917125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8d400 0x5572ebaa81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.929+0000 7f8137fff700 1 -- 172.21.15.46:0/3668917125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572ebab3020 con 0x5572eba8cae0 2024-03-18T20:39:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.931+0000 7f8137fff700 1 --2- 172.21.15.46:0/3668917125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8cae0 0x5572eba8fa40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f812800b6f0 tx=0x7f8128020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af35d6efb28f70a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.931+0000 7f811effd700 1 -- 172.21.15.46:0/3668917125 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128005530 con 0x5572eba8cae0 2024-03-18T20:39:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.931+0000 7f811effd700 1 -- 172.21.15.46:0/3668917125 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f812802a040 con 0x5572eba8cae0 2024-03-18T20:39:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.931+0000 7f811effd700 1 -- 172.21.15.46:0/3668917125 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812800f460 con 0x5572eba8cae0 2024-03-18T20:39:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.934+0000 7f814a566b80 1 -- 172.21.15.46:0/3668917125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8cae0 msgr2=0x5572eba8fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.934+0000 7f814a566b80 1 --2- 172.21.15.46:0/3668917125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8cae0 0x5572eba8fa40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f812800b6f0 tx=0x7f8128020000 comp rx=0 tx=0).stop 2024-03-18T20:39:49.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.935+0000 7f814a566b80 1 -- 172.21.15.46:0/3668917125 shutdown_connections 2024-03-18T20:39:49.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.935+0000 7f814a566b80 1 --2- 172.21.15.46:0/3668917125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8cae0 0x5572eba8fa40 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.935+0000 7f814a566b80 1 --2- 172.21.15.46:0/3668917125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572eba92260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.935+0000 7f814a566b80 1 --2- 172.21.15.46:0/3668917125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8d400 0x5572ebaa81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.935+0000 7f814a566b80 1 -- 172.21.15.46:0/3668917125 >> 172.21.15.46:0/3668917125 conn(0x5572eba032a0 msgr2=0x5572ebaa7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:49.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.937+0000 7f814a566b80 1 -- 172.21.15.46:0/3668917125 shutdown_connections 2024-03-18T20:39:49.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.938+0000 7f814a566b80 1 -- 172.21.15.46:0/3668917125 wait complete. 2024-03-18T20:39:49.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.938+0000 7f814a566b80 1 -- start start 2024-03-18T20:39:49.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.939+0000 7f814a566b80 1 Processor -- start 2024-03-18T20:39:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.939+0000 7f814a566b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572ebaa34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.939+0000 7f814a566b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572ebaa1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.939+0000 7f814a566b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572ebaa21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.939+0000 7f814a566b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ebaa28c0 con 0x5572eba8d400 2024-03-18T20:39:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.939+0000 7f814a566b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ebaa2a30 con 0x5572eba8cae0 2024-03-18T20:39:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.939+0000 7f814a566b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572ebb06db0 con 0x5572eba982a0 2024-03-18T20:39:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.939+0000 7f814a566b80 10 client.0 _ll_register_callbacks cb 0x5572eb7e7420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:49.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.940+0000 7f813c96e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572ebaa21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.940+0000 7f813c96e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572ebaa21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37588/0 (socket says 172.21.15.46:37588) 2024-03-18T20:39:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.940+0000 7f813c96e700 1 -- 172.21.15.46:0/2213715165 learned_addr learned my addr 172.21.15.46:0/2213715165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:49.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.940+0000 7f8137fff700 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572ebaa34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.943+0000 7f81377fe700 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572ebaa1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.943+0000 7f813c96e700 1 -- 172.21.15.46:0/2213715165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 msgr2=0x5572ebaa34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.943+0000 7f813c96e700 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572ebaa34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.943+0000 7f813c96e700 1 -- 172.21.15.46:0/2213715165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 msgr2=0x5572ebaa1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.943+0000 7f813c96e700 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572ebaa1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.943+0000 7f813c96e700 1 -- 172.21.15.46:0/2213715165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813810d370 con 0x5572eba982a0 2024-03-18T20:39:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.946+0000 7f813c96e700 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572ebaa21b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8130004d70 tx=0x7f8130004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=998c2e441e34d659 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:49.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.946+0000 7f81177fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:39:49.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.946+0000 7f81177fe700 1 -- 172.21.15.46:0/2213715165 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813000ce60 con 0x5572eba982a0 2024-03-18T20:39:49.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.946+0000 7f81177fe700 1 -- 172.21.15.46:0/2213715165 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8130099960 con 0x5572eba982a0 2024-03-18T20:39:49.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.946+0000 7f814a566b80 1 -- 172.21.15.46:0/2213715165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f812c004510 con 0x5572eba982a0 2024-03-18T20:39:49.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.947+0000 7f81177fe700 1 -- 172.21.15.46:0/2213715165 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f813009d790 con 0x5572eba982a0 2024-03-18T20:39:49.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.948+0000 7f814a566b80 10 client.26620 fetch_fsmap learned FSMap version 38 2024-03-18T20:39:49.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.948+0000 7f814a566b80 1 -- 172.21.15.46:0/2213715165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5572eb9a5f20 con 0x5572eba982a0 2024-03-18T20:39:49.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.948+0000 7f81177fe700 1 -- 172.21.15.46:0/2213715165 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f812c004510 con 0x5572eba982a0 2024-03-18T20:39:49.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.948+0000 7f81177fe700 1 -- 172.21.15.46:0/2213715165 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8130098f70 con 0x5572eba982a0 2024-03-18T20:39:49.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.948+0000 7f814a566b80 10 client.26620 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T20:39:49.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.948+0000 7f814a566b80 10 client.26620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:49.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.948+0000 7f814a566b80 20 client.26620 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:49.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.948+0000 7f814a566b80 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5572ebb08470 0x5572ebb0a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:49.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.948+0000 7f814a566b80 4 client.26620 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:39:49.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.948+0000 7f814a566b80 1 -- 172.21.15.46:0/2213715165 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5572eb82ed70 con 0x5572ebb08470 2024-03-18T20:39:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.951+0000 7f8137fff700 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5572ebb08470 0x5572ebb0a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.951+0000 7f81177fe700 10 client.26620 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:49.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.951+0000 7f8137fff700 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5572ebb08470 0x5572ebb0a890 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d2d723c909a5758 server_cookie=a0d75667af78a297 in_seq=0 out_seq=0 2024-03-18T20:39:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f81177fe700 1 -- 172.21.15.46:0/2213715165 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5572eb82ed70 con 0x5572ebb08470 2024-03-18T20:39:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f81177fe700 10 client.26620 handle_command_reply: tid=0 2024-03-18T20:39:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f814a566b80 1 client.26620 shutdown 2024-03-18T20:39:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f814a566b80 1 -- 172.21.15.46:0/2213715165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 msgr2=0x5572ebaa21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f814a566b80 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572ebaa21b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8130004d70 tx=0x7f8130004ad0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f814a566b80 1 -- 172.21.15.46:0/2213715165 shutdown_connections 2024-03-18T20:39:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f814a566b80 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5572ebb08470 0x5572ebb0a890 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f814a566b80 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5572eba982a0 0x5572ebaa21b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f814a566b80 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572eba8cae0 0x5572ebaa34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f81177fe700 0 client.26620 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:49.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f814a566b80 1 --2- 172.21.15.46:0/2213715165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572eba8d400 0x5572ebaa1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f814a566b80 1 -- 172.21.15.46:0/2213715165 >> 172.21.15.46:0/2213715165 conn(0x5572eba032a0 msgr2=0x5572eba94520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:49.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f814a566b80 1 -- 172.21.15.46:0/2213715165 shutdown_connections 2024-03-18T20:39:49.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f814a566b80 1 -- 172.21.15.46:0/2213715165 wait complete. 2024-03-18T20:39:49.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.954+0000 7f814a566b80 20 client.26620 trim_cache size 0 max 16384 2024-03-18T20:39:49.959 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:39:49.959 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:39:49.959 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "4b0223fd-c8ed-4b3a-a17e-bef8059da527", 2024-03-18T20:39:49.959 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:39:49.960 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:39:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.955+0000 7f813e3d1700 1 -- 172.21.15.46:0/1666142148 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f81200a1290 msgr2=0x7f81200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.955+0000 7f813e3d1700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f81200a1290 0x7f81200a3750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f812800b2e0 tx=0x7f8128005fd0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.955+0000 7f813e3d1700 1 -- 172.21.15.46:0/1666142148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f813819bb30 msgr2=0x7f813819bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.955+0000 7f813e3d1700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f813819bb30 0x7f813819bfb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f813000ef30 tx=0x7f813000c3a0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.956+0000 7f813e3d1700 1 -- 172.21.15.46:0/1666142148 shutdown_connections 2024-03-18T20:39:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.956+0000 7f813e3d1700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f81200a1290 0x7f81200a3750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.956+0000 7f813e3d1700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f813819bb30 0x7f813819bfb0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.956+0000 7f813e3d1700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81381005c0 0x7f8138197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.956+0000 7f813e3d1700 1 --2- 172.21.15.46:0/1666142148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8138104f60 0x7f8138197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.956+0000 7f813e3d1700 1 -- 172.21.15.46:0/1666142148 >> 172.21.15.46:0/1666142148 conn(0x7f81380761e0 msgr2=0x7f81380ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.958+0000 7f813e3d1700 1 -- 172.21.15.46:0/1666142148 shutdown_connections 2024-03-18T20:39:49.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:49.958+0000 7f813e3d1700 1 -- 172.21.15.46:0/1666142148 wait complete. 2024-03-18T20:39:49.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:39:49.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:39:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.134+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/1982632997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf0100800 msgr2=0x7f3cf0104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.134+0000 7f3cf5f9b700 1 --2- 172.21.15.46:0/1982632997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf0100800 0x7f3cf0104830 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009990 tx=0x7f3cd8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.134+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/1982632997 shutdown_connections 2024-03-18T20:39:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.134+0000 7f3cf5f9b700 1 --2- 172.21.15.46:0/1982632997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf0104f60 0x7f3cf010cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.134+0000 7f3cf5f9b700 1 --2- 172.21.15.46:0/1982632997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf00ffe50 0x7f3cf0100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.134+0000 7f3cf5f9b700 1 --2- 172.21.15.46:0/1982632997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf0100800 0x7f3cf0104830 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.134+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/1982632997 >> 172.21.15.46:0/1982632997 conn(0x7f3cf0076d40 msgr2=0x7f3cf0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.134+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/1982632997 shutdown_connections 2024-03-18T20:39:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.134+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/1982632997 wait complete. 2024-03-18T20:39:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3cf5f9b700 1 Processor -- start 2024-03-18T20:39:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3cf5f9b700 1 -- start start 2024-03-18T20:39:50.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3cf5f9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf00ffe50 0x7f3cf0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3cf5f9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf0104f60 0x7f3cf0197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3cf5f9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf019bae0 0x7f3cf019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3cf5f9b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf019c560 con 0x7f3cf019bae0 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3cf5f9b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf019c6d0 con 0x7f3cf00ffe50 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3cf5f9b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf019c890 con 0x7f3cf0104f60 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3ceffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf019bae0 0x7f3cf019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3ceffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf019bae0 0x7f3cf019bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52196/0 (socket says 172.21.15.46:52196) 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3ceffff700 1 -- 172.21.15.46:0/2480683481 learned_addr learned my addr 172.21.15.46:0/2480683481 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3ceeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf0104f60 0x7f3cf0197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.135+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf00ffe50 0x7f3cf0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.136+0000 7f3ceffff700 1 -- 172.21.15.46:0/2480683481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf0104f60 msgr2=0x7f3cf0197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.136+0000 7f3ceffff700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf0104f60 0x7f3cf0197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.136+0000 7f3ceffff700 1 -- 172.21.15.46:0/2480683481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf00ffe50 msgr2=0x7f3cf0197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.136+0000 7f3ceffff700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf00ffe50 0x7f3cf0197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.136+0000 7f3ceffff700 1 -- 172.21.15.46:0/2480683481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd80095f0 con 0x7f3cf019bae0 2024-03-18T20:39:50.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.136+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf00ffe50 0x7f3cf0197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.136+0000 7f3ceffff700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf019bae0 0x7f3cf019bf60 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000d980 tx=0x7f3ce000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.136+0000 7f3cecff9700 1 -- 172.21.15.46:0/2480683481 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0004220 con 0x7f3cf019bae0 2024-03-18T20:39:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.136+0000 7f3cecff9700 1 -- 172.21.15.46:0/2480683481 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ce00075e0 con 0x7f3cf019bae0 2024-03-18T20:39:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.136+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/2480683481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf01a3df0 con 0x7f3cf019bae0 2024-03-18T20:39:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.136+0000 7f3cecff9700 1 -- 172.21.15.46:0/2480683481 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0005650 con 0x7f3cf019bae0 2024-03-18T20:39:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.136+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/2480683481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf01a42d0 con 0x7f3cf019bae0 2024-03-18T20:39:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.139+0000 7f3cecff9700 1 -- 172.21.15.46:0/2480683481 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3ce0004750 con 0x7f3cf019bae0 2024-03-18T20:39:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.139+0000 7f3cecff9700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3cdc0a1310 0x7f3cdc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.140+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3cdc0a1310 0x7f3cdc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.140+0000 7f3cecff9700 1 -- 172.21.15.46:0/2480683481 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f3ce00cca50 con 0x7f3cf019bae0 2024-03-18T20:39:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.140+0000 7f3d02130b80 1 Processor -- start 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.140+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3cdc0a1310 0x7f3cdc0a37d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80093a0 tx=0x7f3ce800b040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.140+0000 7f3d02130b80 1 -- start start 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.140+0000 7f3d02130b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 0x561d23e5ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.140+0000 7f3d02130b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e69ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.140+0000 7f3d02130b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e6a220 0x561d23e6c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.140+0000 7f3d02130b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d23db8990 con 0x561d23e5ea60 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.140+0000 7f3d02130b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d23db6350 con 0x561d23e6a220 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3d02130b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d23db1c40 con 0x561d23e5f380 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3cef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 0x561d23e5ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3cef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 0x561d23e5ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52212/0 (socket says 172.21.15.46:52212) 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3cef7fe700 1 -- 172.21.15.46:0/2814590111 learned_addr learned my addr 172.21.15.46:0/2814590111 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3ceeffd700 1 --2- 172.21.15.46:0/2814590111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e69ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3ceffff700 1 --2- 172.21.15.46:0/2814590111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e6a220 0x561d23e6c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3cef7fe700 1 -- 172.21.15.46:0/2814590111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 msgr2=0x561d23e69ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/2814590111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e69ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3cef7fe700 1 -- 172.21.15.46:0/2814590111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e6a220 msgr2=0x561d23e6c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:50.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/2814590111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e6a220 0x561d23e6c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3cef7fe700 1 -- 172.21.15.46:0/2814590111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d23c333b0 con 0x561d23e5ea60 2024-03-18T20:39:50.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/2814590111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 0x561d23e5ee40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8003c90 tx=0x7f3ce8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedb4cec03c464e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:50.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3ce5ffb700 1 -- 172.21.15.46:0/2814590111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800e050 con 0x561d23e5ea60 2024-03-18T20:39:50.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3ce5ffb700 1 -- 172.21.15.46:0/2814590111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ce800f040 con 0x561d23e5ea60 2024-03-18T20:39:50.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.141+0000 7f3ce5ffb700 1 -- 172.21.15.46:0/2814590111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8022640 con 0x561d23e5ea60 2024-03-18T20:39:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.142+0000 7f3d02130b80 1 -- 172.21.15.46:0/2814590111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 msgr2=0x561d23e5ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.142+0000 7f3d02130b80 1 --2- 172.21.15.46:0/2814590111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 0x561d23e5ee40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8003c90 tx=0x7f3ce8013000 comp rx=0 tx=0).stop 2024-03-18T20:39:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.142+0000 7f3d02130b80 1 -- 172.21.15.46:0/2814590111 shutdown_connections 2024-03-18T20:39:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.142+0000 7f3d02130b80 1 --2- 172.21.15.46:0/2814590111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 0x561d23e5ee40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.142+0000 7f3d02130b80 1 --2- 172.21.15.46:0/2814590111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e6a220 0x561d23e6c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.142+0000 7f3d02130b80 1 --2- 172.21.15.46:0/2814590111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e69ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.142+0000 7f3d02130b80 1 -- 172.21.15.46:0/2814590111 >> 172.21.15.46:0/2814590111 conn(0x561d23dd5220 msgr2=0x561d23e5b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.142+0000 7f3d02130b80 1 -- 172.21.15.46:0/2814590111 shutdown_connections 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.142+0000 7f3d02130b80 1 -- 172.21.15.46:0/2814590111 wait complete. 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.143+0000 7f3d02130b80 1 -- start start 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.143+0000 7f3d02130b80 1 Processor -- start 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.143+0000 7f3d02130b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 0x561d23e69c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.143+0000 7f3d02130b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e66f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.143+0000 7f3d02130b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e6a220 0x561d23e676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3d02130b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d23d4dd80 con 0x561d23e6a220 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3d02130b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d23d4da20 con 0x561d23e5ea60 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3d02130b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d23c356a0 con 0x561d23e5f380 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3d02130b80 10 client.0 _ll_register_callbacks cb 0x561d23bb93a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3cef7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 0x561d23e69c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3ceffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e6a220 0x561d23e676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3ceffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e6a220 0x561d23e676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52224/0 (socket says 172.21.15.46:52224) 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3cef7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 0x561d23e69c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40428/0 (socket says 172.21.15.46:40428) 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3ceeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e66f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3ceffff700 1 -- 172.21.15.46:0/3439936958 learned_addr learned my addr 172.21.15.46:0/3439936958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3cef7fe700 1 -- 172.21.15.46:0/3439936958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 msgr2=0x561d23e66f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/3439936958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e66f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3cef7fe700 1 -- 172.21.15.46:0/3439936958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e6a220 msgr2=0x561d23e676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/3439936958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e6a220 0x561d23e676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3cef7fe700 1 -- 172.21.15.46:0/3439936958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce8003a00 con 0x561d23e5ea60 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/3439936958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 0x561d23e69c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8025040 tx=0x7f3ce8005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894d4c4a3e6443c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3cceffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3cceffd700 1 -- 172.21.15.46:0/3439936958 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800e070 con 0x561d23e5ea60 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3cceffd700 1 -- 172.21.15.46:0/3439936958 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f3ce802c640 con 0x561d23e5ea60 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.144+0000 7f3d02130b80 1 -- 172.21.15.46:0/3439936958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d23e52ad0 con 0x561d23e5ea60 2024-03-18T20:39:50.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.145+0000 7f3cceffd700 1 -- 172.21.15.46:0/3439936958 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ce802c800 con 0x561d23e5ea60 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.145+0000 7f3d02130b80 10 client.26618 fetch_fsmap learned FSMap version 38 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.145+0000 7f3d02130b80 1 -- 172.21.15.46:0/3439936958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561d23bdfb60 con 0x561d23e5ea60 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.145+0000 7f3cceffd700 1 -- 172.21.15.46:0/3439936958 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561d23e52ad0 con 0x561d23e5ea60 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.145+0000 7f3cceffd700 1 -- 172.21.15.46:0/3439936958 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 38) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3ce8022400 con 0x561d23e5ea60 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.145+0000 7f3d02130b80 10 client.26618 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.145+0000 7f3d02130b80 10 client.26618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.145+0000 7f3d02130b80 20 client.26618 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.145+0000 7f3d02130b80 1 --2- 172.21.15.46:0/3439936958 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561d23e79e70 0x561d23e7c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.145+0000 7f3d02130b80 4 client.26618 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.145+0000 7f3d02130b80 1 -- 172.21.15.46:0/3439936958 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d23daa700 con 0x561d23e79e70 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.145+0000 7f3ceeffd700 1 --2- 172.21.15.46:0/3439936958 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561d23e79e70 0x561d23e7c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3cceffd700 10 client.26618 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3ceeffd700 1 --2- 172.21.15.46:0/3439936958 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561d23e79e70 0x561d23e7c290 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5bb25ffecb0b44 server_cookie=33861ce3858d7c69 in_seq=0 out_seq=0 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3cceffd700 1 -- 172.21.15.46:0/3439936958 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561d23daa700 con 0x561d23e79e70 2024-03-18T20:39:50.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3cceffd700 10 client.26618 handle_command_reply: tid=0 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3d02130b80 1 client.26618 shutdown 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3d02130b80 1 -- 172.21.15.46:0/3439936958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 msgr2=0x561d23e69c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3d02130b80 1 --2- 172.21.15.46:0/3439936958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 0x561d23e69c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8025040 tx=0x7f3ce8005700 comp rx=0 tx=0).stop 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3d02130b80 1 -- 172.21.15.46:0/3439936958 shutdown_connections 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3d02130b80 1 --2- 172.21.15.46:0/3439936958 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561d23e79e70 0x561d23e7c290 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3d02130b80 1 --2- 172.21.15.46:0/3439936958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e6a220 0x561d23e676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3cceffd700 0 client.26618 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3d02130b80 1 --2- 172.21.15.46:0/3439936958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 0x561d23e69c80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3d02130b80 1 --2- 172.21.15.46:0/3439936958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e66f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.146+0000 7f3d02130b80 1 -- 172.21.15.46:0/3439936958 >> 172.21.15.46:0/3439936958 conn(0x561d23dd5220 msgr2=0x561d23e5ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.147+0000 7f3d02130b80 1 -- 172.21.15.46:0/3439936958 shutdown_connections 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.147+0000 7f3d02130b80 1 -- 172.21.15.46:0/3439936958 wait complete. 2024-03-18T20:39:50.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.147+0000 7f3d02130b80 20 client.26618 trim_cache size 0 max 16384 2024-03-18T20:39:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.164+0000 7f3d02130b80 1 Processor -- start 2024-03-18T20:39:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.164+0000 7f3d02130b80 1 -- start start 2024-03-18T20:39:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3d02130b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 0x561d23e7c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3d02130b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5f380 0x561d23e70e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3d02130b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e6a220 0x561d23e62280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3d02130b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d23e76990 con 0x561d23e5f380 2024-03-18T20:39:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3d02130b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d23e5fd30 con 0x561d23e5ea60 2024-03-18T20:39:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3d02130b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d23e54f80 con 0x561d23e6a220 2024-03-18T20:39:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3ceeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5f380 0x561d23e70e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3cef7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 0x561d23e7c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3ceeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5f380 0x561d23e70e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52232/0 (socket says 172.21.15.46:52232) 2024-03-18T20:39:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3ceeffd700 1 -- 172.21.15.46:0/1984573959 learned_addr learned my addr 172.21.15.46:0/1984573959 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3ceffff700 1 --2- 172.21.15.46:0/1984573959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e6a220 0x561d23e62280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3cef7fe700 1 -- 172.21.15.46:0/1984573959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e6a220 msgr2=0x561d23e62280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/1984573959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e6a220 0x561d23e62280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3cef7fe700 1 -- 172.21.15.46:0/1984573959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5f380 msgr2=0x561d23e70e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:50.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/1984573959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5f380 0x561d23e70e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3cef7fe700 1 -- 172.21.15.46:0/1984573959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d23d77ea0 con 0x561d23e5ea60 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3ceffff700 1 --2- 172.21.15.46:0/1984573959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e6a220 0x561d23e62280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/1984573959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 0x561d23e7c260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8022810 tx=0x7f3ce8022930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59e98ca9807ec852 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3ce67fc700 1 -- 172.21.15.46:0/1984573959 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800e070 con 0x561d23e5ea60 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3ce67fc700 1 -- 172.21.15.46:0/1984573959 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ce8009ac0 con 0x561d23e5ea60 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.165+0000 7f3ce67fc700 1 -- 172.21.15.46:0/1984573959 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8015a00 con 0x561d23e5ea60 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.166+0000 7f3d02130b80 1 -- 172.21.15.46:0/1984573959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 msgr2=0x561d23e7c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.166+0000 7f3d02130b80 1 --2- 172.21.15.46:0/1984573959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 0x561d23e7c260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8022810 tx=0x7f3ce8022930 comp rx=0 tx=0).stop 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.166+0000 7f3d02130b80 1 -- 172.21.15.46:0/1984573959 shutdown_connections 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.166+0000 7f3d02130b80 1 --2- 172.21.15.46:0/1984573959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e6a220 0x561d23e62280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.166+0000 7f3d02130b80 1 --2- 172.21.15.46:0/1984573959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e5ea60 0x561d23e7c260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.166+0000 7f3d02130b80 1 --2- 172.21.15.46:0/1984573959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5f380 0x561d23e70e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.166+0000 7f3d02130b80 1 -- 172.21.15.46:0/1984573959 >> 172.21.15.46:0/1984573959 conn(0x561d23dd5220 msgr2=0x561d23e59750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.166+0000 7f3d02130b80 1 -- 172.21.15.46:0/1984573959 shutdown_connections 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.166+0000 7f3d02130b80 1 -- 172.21.15.46:0/1984573959 wait complete. 2024-03-18T20:39:50.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.166+0000 7f3d02130b80 1 -- start start 2024-03-18T20:39:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3d02130b80 1 Processor -- start 2024-03-18T20:39:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3d02130b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 0x561d23e6ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3d02130b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e6f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3d02130b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e6a220 0x561d23e6f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3d02130b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d23eccd50 con 0x561d23e5ea60 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3d02130b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d23eccec0 con 0x561d23e6a220 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3d02130b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d23ecd080 con 0x561d23e5f380 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3d02130b80 10 client.0 _ll_register_callbacks cb 0x7f3ce0095f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3cef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 0x561d23e6ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3cef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 0x561d23e6ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52236/0 (socket says 172.21.15.46:52236) 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3cef7fe700 1 -- 172.21.15.46:0/330743496 learned_addr learned my addr 172.21.15.46:0/330743496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3ceeffd700 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e6f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3ceffff700 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e6a220 0x561d23e6f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3ceeffd700 1 -- 172.21.15.46:0/330743496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e6a220 msgr2=0x561d23e6f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.167+0000 7f3ceeffd700 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e6a220 0x561d23e6f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.168+0000 7f3ceeffd700 1 -- 172.21.15.46:0/330743496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 msgr2=0x561d23e6ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.168+0000 7f3ceeffd700 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 0x561d23e6ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.168+0000 7f3ceeffd700 1 -- 172.21.15.46:0/330743496 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d23bdfb60 con 0x561d23e5f380 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.168+0000 7f3ceeffd700 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e6f370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b2e0 tx=0x7f3cd80055a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b28aba48996fb6c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.168+0000 7f3cceffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.168+0000 7f3cceffd700 1 -- 172.21.15.46:0/330743496 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd801d040 con 0x561d23e5f380 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.168+0000 7f3cceffd700 1 -- 172.21.15.46:0/330743496 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f3cd8005690 con 0x561d23e5f380 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.168+0000 7f3d02130b80 1 -- 172.21.15.46:0/330743496 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d23c00cf0 con 0x561d23e5f380 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.168+0000 7f3cceffd700 1 -- 172.21.15.46:0/330743496 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cd800bb00 con 0x561d23e5f380 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.168+0000 7f3d02130b80 10 client.26638 fetch_fsmap learned FSMap version 38 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.168+0000 7f3d02130b80 1 -- 172.21.15.46:0/330743496 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561d23c37cd0 con 0x561d23e5f380 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3cceffd700 1 -- 172.21.15.46:0/330743496 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cd8036080 con 0x561d23e5f380 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3cceffd700 1 -- 172.21.15.46:0/330743496 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 38) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3cd800f460 con 0x561d23e5f380 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3d02130b80 10 client.26638 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3d02130b80 10 client.26638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3d02130b80 20 client.26638 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3d02130b80 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561d23ece910 0x561d23ed0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3d02130b80 4 client.26638 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:39:50.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3d02130b80 1 -- 172.21.15.46:0/330743496 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561d23c35ce0 con 0x561d23ece910 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561d23ece910 0x561d23ed0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3cef7fe700 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561d23ece910 0x561d23ed0d30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4d8625b80f5fc server_cookie=465e08aa759f0f90 in_seq=0 out_seq=0 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3cceffd700 10 client.26638 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3cceffd700 1 -- 172.21.15.46:0/330743496 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561d23c35ce0 con 0x561d23ece910 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3cceffd700 10 client.26638 handle_command_reply: tid=0 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.169+0000 7f3d02130b80 1 client.26638 shutdown 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3d02130b80 1 -- 172.21.15.46:0/330743496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 msgr2=0x561d23e6f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3d02130b80 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e6f370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b2e0 tx=0x7f3cd80055a0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3d02130b80 1 -- 172.21.15.46:0/330743496 shutdown_connections 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3d02130b80 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561d23ece910 0x561d23ed0d30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3d02130b80 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561d23e5ea60 0x561d23e6ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3d02130b80 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561d23e6a220 0x561d23e6f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3cceffd700 0 client.26638 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:50.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3d02130b80 1 --2- 172.21.15.46:0/330743496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561d23e5f380 0x561d23e6f370 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3d02130b80 1 -- 172.21.15.46:0/330743496 >> 172.21.15.46:0/330743496 conn(0x561d23dd5220 msgr2=0x561d23e657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3d02130b80 1 -- 172.21.15.46:0/330743496 shutdown_connections 2024-03-18T20:39:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3d02130b80 1 -- 172.21.15.46:0/330743496 wait complete. 2024-03-18T20:39:50.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3d02130b80 20 client.26638 trim_cache size 0 max 16384 2024-03-18T20:39:50.179 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:39:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:39:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:39:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "4b0223fd-c8ed-4b3a-a17e-bef8059da527": { 2024-03-18T20:39:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:39:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "4b0223fd-c8ed-4b3a-a17e-bef8059da527", 2024-03-18T20:39:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:39:50.179 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:39:50.180 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:39:50.180 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:39:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/2480683481 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3cdc0a1310 msgr2=0x7f3cdc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:50.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3cf5f9b700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3cdc0a1310 0x7f3cdc0a37d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80093a0 tx=0x7f3ce800b040 comp rx=0 tx=0).stop 2024-03-18T20:39:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/2480683481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf019bae0 msgr2=0x7f3cf019bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3cf5f9b700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf019bae0 0x7f3cf019bf60 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000d980 tx=0x7f3ce000dc90 comp rx=0 tx=0).stop 2024-03-18T20:39:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/2480683481 shutdown_connections 2024-03-18T20:39:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3cf5f9b700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3cdc0a1310 0x7f3cdc0a37d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3cf5f9b700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf019bae0 0x7f3cf019bf60 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3cf5f9b700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf00ffe50 0x7f3cf0197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3cf5f9b700 1 --2- 172.21.15.46:0/2480683481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf0104f60 0x7f3cf0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.170+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/2480683481 >> 172.21.15.46:0/2480683481 conn(0x7f3cf0076d40 msgr2=0x7f3cf00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.171+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/2480683481 shutdown_connections 2024-03-18T20:39:50.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:39:50.171+0000 7f3cf5f9b700 1 -- 172.21.15.46:0/2480683481 wait complete. 2024-03-18T20:39:50.187 INFO:tasks.cephfs.filesystem:scrub status for tag:4b0223fd-c8ed-4b3a-a17e-bef8059da527 - {'path': '/', 'tag': '4b0223fd-c8ed-4b3a-a17e-bef8059da527', 'options': 'recursive,force'} 2024-03-18T20:39:50.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:50 smithi073 ceph-mon[47384]: pgmap v115: 97 pgs: 97 active+clean; 43 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 531 KiB/s wr, 79 op/s 2024-03-18T20:39:50.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:50 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:39:50.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:50 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:50.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:50 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:50.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:50 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:50.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:50 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:50.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:50 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:50.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:50 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:50 smithi155 ceph-mon[48344]: pgmap v115: 97 pgs: 97 active+clean; 43 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 531 KiB/s wr, 79 op/s 2024-03-18T20:39:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:50 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:39:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:50 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:50 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:50 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:50 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:50 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:50.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:50 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:50 smithi046 ceph-mon[43495]: pgmap v115: 97 pgs: 97 active+clean; 43 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 531 KiB/s wr, 79 op/s 2024-03-18T20:39:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:50 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:39:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:50 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:50 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:50 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:50 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:50 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:50.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:50 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:50.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 93060 1.28 MB/sec execute 45 sec latency 2938.061 ms 2024-03-18T20:39:51.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:51.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:51.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:51.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:51.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:51.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:39:51.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:51 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:51.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:39:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:51 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:51.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:39:51.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:51 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:39:51.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 93060 1.25 MB/sec execute 46 sec latency 1591.843 ms 2024-03-18T20:39:52.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:52 smithi073 ceph-mon[47384]: pgmap v116: 97 pgs: 97 active+clean; 43 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 246 KiB/s wr, 57 op/s 2024-03-18T20:39:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:52 smithi155 ceph-mon[48344]: pgmap v116: 97 pgs: 97 active+clean; 43 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 246 KiB/s wr, 57 op/s 2024-03-18T20:39:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:52 smithi046 ceph-mon[43495]: pgmap v116: 97 pgs: 97 active+clean; 43 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 246 KiB/s wr, 57 op/s 2024-03-18T20:39:52.681 INFO:tasks.workunit.client.0.smithi046.stdout: 1 94761 1.38 MB/sec execute 47 sec latency 1946.070 ms 2024-03-18T20:39:53.682 INFO:tasks.workunit.client.0.smithi046.stdout: 1 96276 1.59 MB/sec execute 48 sec latency 12.731 ms 2024-03-18T20:39:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:54 smithi073 ceph-mon[47384]: pgmap v117: 97 pgs: 97 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-03-18T20:39:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:54 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.472+0000 7fe96adbd700 1 -- 172.21.15.46:0/2848124712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe964104f40 msgr2=0x7fe96410cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.472+0000 7fe96adbd700 1 --2- 172.21.15.46:0/2848124712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe964104f40 0x7fe96410cac0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00b0a0 tx=0x7fe95c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.472+0000 7fe96adbd700 1 -- 172.21.15.46:0/2848124712 shutdown_connections 2024-03-18T20:39:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.472+0000 7fe96adbd700 1 --2- 172.21.15.46:0/2848124712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe964104f40 0x7fe96410cac0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.472+0000 7fe96adbd700 1 --2- 172.21.15.46:0/2848124712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9640ffe20 0x7fe964100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.472+0000 7fe96adbd700 1 --2- 172.21.15.46:0/2848124712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9641007d0 0x7fe964104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.472+0000 7fe96adbd700 1 -- 172.21.15.46:0/2848124712 >> 172.21.15.46:0/2848124712 conn(0x7fe9640fb680 msgr2=0x7fe9640fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.473+0000 7fe96adbd700 1 -- 172.21.15.46:0/2848124712 shutdown_connections 2024-03-18T20:39:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.473+0000 7fe96adbd700 1 -- 172.21.15.46:0/2848124712 wait complete. 2024-03-18T20:39:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.473+0000 7fe96adbd700 1 Processor -- start 2024-03-18T20:39:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.473+0000 7fe96adbd700 1 -- start start 2024-03-18T20:39:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.473+0000 7fe96adbd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640ffe20 0x7fe9641972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.473+0000 7fe96adbd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe964104f40 0x7fe964197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.473+0000 7fe96adbd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe96419bba0 0x7fe96419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.473+0000 7fe96adbd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96419c620 con 0x7fe9640ffe20 2024-03-18T20:39:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.473+0000 7fe96adbd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96419c790 con 0x7fe96419bba0 2024-03-18T20:39:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.473+0000 7fe96adbd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96419c950 con 0x7fe964104f40 2024-03-18T20:39:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe968b59700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640ffe20 0x7fe9641972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe968b59700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640ffe20 0x7fe9641972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52252/0 (socket says 172.21.15.46:52252) 2024-03-18T20:39:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe968b59700 1 -- 172.21.15.46:0/3051502672 learned_addr learned my addr 172.21.15.46:0/3051502672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe963fff700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe964104f40 0x7fe964197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe96935a700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe96419bba0 0x7fe96419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe968b59700 1 -- 172.21.15.46:0/3051502672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe964104f40 msgr2=0x7fe964197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe968b59700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe964104f40 0x7fe964197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe968b59700 1 -- 172.21.15.46:0/3051502672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe96419bba0 msgr2=0x7fe96419c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe968b59700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe96419bba0 0x7fe96419c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe968b59700 1 -- 172.21.15.46:0/3051502672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe954009520 con 0x7fe9640ffe20 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe96935a700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe96419bba0 0x7fe96419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe968b59700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640ffe20 0x7fe9641972c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe95400eb30 tx=0x7fe95400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe961ffb700 1 -- 172.21.15.46:0/3051502672 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954009880 con 0x7fe9640ffe20 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe961ffb700 1 -- 172.21.15.46:0/3051502672 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9540044e0 con 0x7fe9640ffe20 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe96adbd700 1 -- 172.21.15.46:0/3051502672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c009d00 con 0x7fe9640ffe20 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.474+0000 7fe961ffb700 1 -- 172.21.15.46:0/3051502672 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954003e90 con 0x7fe9640ffe20 2024-03-18T20:39:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.475+0000 7fe96adbd700 1 -- 172.21.15.46:0/3051502672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9641a41f0 con 0x7fe9640ffe20 2024-03-18T20:39:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.476+0000 7fe961ffb700 1 -- 172.21.15.46:0/3051502672 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe954004000 con 0x7fe9640ffe20 2024-03-18T20:39:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.477+0000 7fe961ffb700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe9500a1310 0x7fe9500a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.478+0000 7fe963fff700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe9500a1310 0x7fe9500a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.478+0000 7fe961ffb700 1 -- 172.21.15.46:0/3051502672 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fe954014070 con 0x7fe9640ffe20 2024-03-18T20:39:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.478+0000 7fe963fff700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe9500a1310 0x7fe9500a37d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe964198810 tx=0x7fe94c00b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe976f52b80 1 Processor -- start 2024-03-18T20:39:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe976f52b80 1 -- start start 2024-03-18T20:39:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe976f52b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7392b570 0x55ca7392b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe976f52b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 0x55ca7393ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe976f52b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7393b1c0 0x55ca7393d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe976f52b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca73885930 con 0x55ca7392be90 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe976f52b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca738832f0 con 0x55ca7392b570 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe976f52b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7387ebe0 con 0x55ca7393b1c0 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe963fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 0x55ca7393ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe963fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 0x55ca7393ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52262/0 (socket says 172.21.15.46:52262) 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe963fff700 1 -- 172.21.15.46:0/953200586 learned_addr learned my addr 172.21.15.46:0/953200586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe968b59700 1 --2- 172.21.15.46:0/953200586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7392b570 0x55ca7392b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe96935a700 1 --2- 172.21.15.46:0/953200586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7393b1c0 0x55ca7393d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe963fff700 1 -- 172.21.15.46:0/953200586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7393b1c0 msgr2=0x55ca7393d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe963fff700 1 --2- 172.21.15.46:0/953200586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7393b1c0 0x55ca7393d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe963fff700 1 -- 172.21.15.46:0/953200586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7392b570 msgr2=0x55ca7392b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe963fff700 1 --2- 172.21.15.46:0/953200586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7392b570 0x55ca7392b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.479+0000 7fe963fff700 1 -- 172.21.15.46:0/953200586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca73700350 con 0x55ca7392be90 2024-03-18T20:39:54.485 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.480+0000 7fe963fff700 1 --2- 172.21.15.46:0/953200586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 0x55ca7393ac80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe94c010dc0 tx=0x7fe94c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9c5a466765e388 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.480+0000 7fe95affd700 1 -- 172.21.15.46:0/953200586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c01e420 con 0x55ca7392be90 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.480+0000 7fe95affd700 1 -- 172.21.15.46:0/953200586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe94c015070 con 0x55ca7392be90 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.480+0000 7fe95affd700 1 -- 172.21.15.46:0/953200586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c01c930 con 0x55ca7392be90 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.481+0000 7fe976f52b80 1 -- 172.21.15.46:0/953200586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 msgr2=0x55ca7393ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.481+0000 7fe976f52b80 1 --2- 172.21.15.46:0/953200586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 0x55ca7393ac80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe94c010dc0 tx=0x7fe94c013000 comp rx=0 tx=0).stop 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.481+0000 7fe976f52b80 1 -- 172.21.15.46:0/953200586 shutdown_connections 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.481+0000 7fe976f52b80 1 --2- 172.21.15.46:0/953200586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7393b1c0 0x55ca7393d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.481+0000 7fe976f52b80 1 --2- 172.21.15.46:0/953200586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7392b570 0x55ca7392b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.481+0000 7fe976f52b80 1 --2- 172.21.15.46:0/953200586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 0x55ca7393ac80 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.481+0000 7fe976f52b80 1 -- 172.21.15.46:0/953200586 >> 172.21.15.46:0/953200586 conn(0x55ca738a21c0 msgr2=0x55ca73927a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.481+0000 7fe976f52b80 1 -- 172.21.15.46:0/953200586 shutdown_connections 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.481+0000 7fe976f52b80 1 -- 172.21.15.46:0/953200586 wait complete. 2024-03-18T20:39:54.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.481+0000 7fe976f52b80 1 -- start start 2024-03-18T20:39:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.481+0000 7fe976f52b80 1 Processor -- start 2024-03-18T20:39:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe976f52b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7392b570 0x55ca7393ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe976f52b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 0x55ca73937f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe976f52b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7393b1c0 0x55ca73938680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe976f52b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7381ad20 con 0x55ca7392be90 2024-03-18T20:39:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe976f52b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7381a9c0 con 0x55ca7393b1c0 2024-03-18T20:39:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe976f52b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca73702640 con 0x55ca7392b570 2024-03-18T20:39:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe976f52b80 10 client.0 _ll_register_callbacks cb 0x55ca73686340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe963fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 0x55ca73937f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe963fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 0x55ca73937f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52266/0 (socket says 172.21.15.46:52266) 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe968b59700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7392b570 0x55ca7393ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe963fff700 1 -- 172.21.15.46:0/167837213 learned_addr learned my addr 172.21.15.46:0/167837213 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe96935a700 1 --2- 172.21.15.46:0/167837213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7393b1c0 0x55ca73938680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe968b59700 1 -- 172.21.15.46:0/167837213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7393b1c0 msgr2=0x55ca73938680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe968b59700 1 --2- 172.21.15.46:0/167837213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7393b1c0 0x55ca73938680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe968b59700 1 -- 172.21.15.46:0/167837213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 msgr2=0x55ca73937f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe968b59700 1 --2- 172.21.15.46:0/167837213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 0x55ca73937f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe968b59700 1 -- 172.21.15.46:0/167837213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c010aa0 con 0x55ca7392b570 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe968b59700 1 --2- 172.21.15.46:0/167837213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7392b570 0x55ca7393ac00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe9540ca810 tx=0x7fe954099bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=208ff9d196750fc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe943fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe943fff700 1 -- 172.21.15.46:0/167837213 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95409a1d0 con 0x55ca7392b570 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe943fff700 1 -- 172.21.15.46:0/167837213 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fe95409e000 con 0x55ca7392b570 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.482+0000 7fe976f52b80 1 -- 172.21.15.46:0/167837213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca7391f6c0 con 0x55ca7392b570 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.483+0000 7fe943fff700 1 -- 172.21.15.46:0/167837213 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9540a2650 con 0x55ca7392b570 2024-03-18T20:39:54.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.483+0000 7fe976f52b80 10 client.26650 fetch_fsmap learned FSMap version 39 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.483+0000 7fe976f52b80 1 -- 172.21.15.46:0/167837213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ca736acb00 con 0x55ca7392b570 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.483+0000 7fe943fff700 1 -- 172.21.15.46:0/167837213 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca7391f6c0 con 0x55ca7392b570 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.483+0000 7fe943fff700 1 -- 172.21.15.46:0/167837213 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe95407a020 con 0x55ca7392b570 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.483+0000 7fe976f52b80 10 client.26650 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.483+0000 7fe976f52b80 10 client.26650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.483+0000 7fe976f52b80 20 client.26650 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.483+0000 7fe976f52b80 1 --2- 172.21.15.46:0/167837213 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ca7394ad80 0x55ca7394d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.483+0000 7fe976f52b80 4 client.26650 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.483+0000 7fe976f52b80 1 -- 172.21.15.46:0/167837213 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca738776a0 con 0x55ca7394ad80 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.483+0000 7fe963fff700 1 --2- 172.21.15.46:0/167837213 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ca7394ad80 0x55ca7394d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.484+0000 7fe963fff700 1 --2- 172.21.15.46:0/167837213 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ca7394ad80 0x55ca7394d1a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7086b7bcf6f88f4d server_cookie=ac04402ae6134b11 in_seq=0 out_seq=0 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.484+0000 7fe943fff700 10 client.26650 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.484+0000 7fe943fff700 1 -- 172.21.15.46:0/167837213 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ca738776a0 con 0x55ca7394ad80 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.484+0000 7fe943fff700 10 client.26650 handle_command_reply: tid=0 2024-03-18T20:39:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.484+0000 7fe976f52b80 1 client.26650 shutdown 2024-03-18T20:39:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.484+0000 7fe976f52b80 1 -- 172.21.15.46:0/167837213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7392b570 msgr2=0x55ca7393ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.484+0000 7fe976f52b80 1 --2- 172.21.15.46:0/167837213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7392b570 0x55ca7393ac00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe9540ca810 tx=0x7fe954099bb0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.485+0000 7fe976f52b80 1 -- 172.21.15.46:0/167837213 shutdown_connections 2024-03-18T20:39:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.485+0000 7fe976f52b80 1 --2- 172.21.15.46:0/167837213 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ca7394ad80 0x55ca7394d1a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.485+0000 7fe976f52b80 1 --2- 172.21.15.46:0/167837213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7393b1c0 0x55ca73938680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.485+0000 7fe976f52b80 1 --2- 172.21.15.46:0/167837213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7392b570 0x55ca7393ac00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.485+0000 7fe976f52b80 1 --2- 172.21.15.46:0/167837213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392be90 0x55ca73937f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.485+0000 7fe976f52b80 1 -- 172.21.15.46:0/167837213 >> 172.21.15.46:0/167837213 conn(0x55ca738a21c0 msgr2=0x55ca73927650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.485+0000 7fe943fff700 0 client.26650 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.485+0000 7fe976f52b80 1 -- 172.21.15.46:0/167837213 shutdown_connections 2024-03-18T20:39:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.485+0000 7fe976f52b80 1 -- 172.21.15.46:0/167837213 wait complete. 2024-03-18T20:39:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.485+0000 7fe976f52b80 20 client.26650 trim_cache size 0 max 16384 2024-03-18T20:39:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.500+0000 7fe976f52b80 1 Processor -- start 2024-03-18T20:39:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.500+0000 7fe976f52b80 1 -- start start 2024-03-18T20:39:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.500+0000 7fe976f52b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 0x55ca73941e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.500+0000 7fe976f52b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7392be90 0x55ca739332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.500+0000 7fe976f52b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7393b1c0 0x55ca73933800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.500+0000 7fe976f52b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7392d070 con 0x55ca7392b570 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.500+0000 7fe976f52b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7392e650 con 0x55ca7393b1c0 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.500+0000 7fe976f52b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca73928f70 con 0x55ca7392be90 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe968b59700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 0x55ca73941e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe968b59700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 0x55ca73941e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52280/0 (socket says 172.21.15.46:52280) 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe968b59700 1 -- 172.21.15.46:0/2380194409 learned_addr learned my addr 172.21.15.46:0/2380194409 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe963fff700 1 --2- 172.21.15.46:0/2380194409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7392be90 0x55ca739332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe96935a700 1 --2- 172.21.15.46:0/2380194409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7393b1c0 0x55ca73933800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe968b59700 1 -- 172.21.15.46:0/2380194409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7392be90 msgr2=0x55ca739332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe968b59700 1 --2- 172.21.15.46:0/2380194409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7392be90 0x55ca739332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe968b59700 1 -- 172.21.15.46:0/2380194409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7393b1c0 msgr2=0x55ca73933800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe968b59700 1 --2- 172.21.15.46:0/2380194409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7393b1c0 0x55ca73933800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe968b59700 1 -- 172.21.15.46:0/2380194409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca73844e40 con 0x55ca7392b570 2024-03-18T20:39:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe968b59700 1 --2- 172.21.15.46:0/2380194409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 0x55ca73941e50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe954098e40 tx=0x7fe954003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3348aa808241c28c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe95b7fe700 1 -- 172.21.15.46:0/2380194409 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954099970 con 0x55ca7392b570 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe95b7fe700 1 -- 172.21.15.46:0/2380194409 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9540b0070 con 0x55ca7392b570 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.501+0000 7fe95b7fe700 1 -- 172.21.15.46:0/2380194409 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9540a2210 con 0x55ca7392b570 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.502+0000 7fe976f52b80 1 -- 172.21.15.46:0/2380194409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 msgr2=0x55ca73941e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.502+0000 7fe976f52b80 1 --2- 172.21.15.46:0/2380194409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 0x55ca73941e50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe954098e40 tx=0x7fe954003930 comp rx=0 tx=0).stop 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.502+0000 7fe976f52b80 1 -- 172.21.15.46:0/2380194409 shutdown_connections 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.502+0000 7fe976f52b80 1 --2- 172.21.15.46:0/2380194409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 0x55ca73941e50 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.502+0000 7fe976f52b80 1 --2- 172.21.15.46:0/2380194409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7393b1c0 0x55ca73933800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.502+0000 7fe976f52b80 1 --2- 172.21.15.46:0/2380194409 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7392be90 0x55ca739332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.502+0000 7fe976f52b80 1 -- 172.21.15.46:0/2380194409 >> 172.21.15.46:0/2380194409 conn(0x55ca738a21c0 msgr2=0x55ca73926120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.502+0000 7fe976f52b80 1 -- 172.21.15.46:0/2380194409 shutdown_connections 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.502+0000 7fe976f52b80 1 -- 172.21.15.46:0/2380194409 wait complete. 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.502+0000 7fe976f52b80 1 -- start start 2024-03-18T20:39:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.502+0000 7fe976f52b80 1 Processor -- start 2024-03-18T20:39:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe976f52b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 0x55ca73940020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe976f52b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7392be90 0x55ca73940560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe976f52b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7393b1c0 0x55ca7399dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe976f52b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca73940c40 con 0x55ca7392b570 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe976f52b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7399e0f0 con 0x55ca7392be90 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe976f52b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca7399e230 con 0x55ca7393b1c0 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe976f52b80 10 client.0 _ll_register_callbacks cb 0x7fe954099fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe968b59700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 0x55ca73940020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe968b59700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 0x55ca73940020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52288/0 (socket says 172.21.15.46:52288) 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe968b59700 1 -- 172.21.15.46:0/2169010974 learned_addr learned my addr 172.21.15.46:0/2169010974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe963fff700 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7392be90 0x55ca73940560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe96935a700 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7393b1c0 0x55ca7399dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe968b59700 1 -- 172.21.15.46:0/2169010974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7393b1c0 msgr2=0x55ca7399dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe968b59700 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7393b1c0 0x55ca7399dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe968b59700 1 -- 172.21.15.46:0/2169010974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7392be90 msgr2=0x55ca73940560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:39:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe968b59700 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7392be90 0x55ca73940560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe968b59700 1 -- 172.21.15.46:0/2169010974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca736acb00 con 0x55ca7392b570 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe968b59700 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 0x55ca73940020 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe954098520 tx=0x7fe954099dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63b20834ebd2691 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.503+0000 7fe943fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe943fff700 1 -- 172.21.15.46:0/2169010974 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95409e000 con 0x55ca7392b570 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe943fff700 1 -- 172.21.15.46:0/2169010974 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fe95409a1d0 con 0x55ca7392b570 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe976f52b80 1 -- 172.21.15.46:0/2169010974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe94c015070 con 0x55ca7392b570 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe943fff700 1 -- 172.21.15.46:0/2169010974 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9540a2650 con 0x55ca7392b570 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe976f52b80 10 client.16917 fetch_fsmap learned FSMap version 39 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe976f52b80 1 -- 172.21.15.46:0/2169010974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ca73704c70 con 0x55ca7392b570 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe943fff700 1 -- 172.21.15.46:0/2169010974 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe94c015070 con 0x55ca7392b570 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe943fff700 1 -- 172.21.15.46:0/2169010974 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe95407a040 con 0x55ca7392b570 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe976f52b80 10 client.16917 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:39:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe976f52b80 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:39:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe976f52b80 20 client.16917 populate_metadata read hostname 'smithi046' 2024-03-18T20:39:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe976f52b80 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ca7399f530 0x55ca739a1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:39:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe976f52b80 4 client.16917 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:39:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.504+0000 7fe976f52b80 1 -- 172.21.15.46:0/2169010974 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca736cdc90 con 0x55ca7399f530 2024-03-18T20:39:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.505+0000 7fe963fff700 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ca7399f530 0x55ca739a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:39:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.505+0000 7fe943fff700 10 client.16917 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.505+0000 7fe963fff700 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ca7399f530 0x55ca739a1920 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a333d556b343065 server_cookie=6e4d0254ca75e4f5 in_seq=0 out_seq=0 2024-03-18T20:39:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.528+0000 7fe943fff700 1 -- 172.21.15.46:0/2169010974 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca736cdc90 con 0x55ca7399f530 2024-03-18T20:39:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe943fff700 10 client.16917 handle_command_reply: tid=0 2024-03-18T20:39:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe976f52b80 1 client.16917 shutdown 2024-03-18T20:39:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe976f52b80 1 -- 172.21.15.46:0/2169010974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 msgr2=0x55ca73940020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe976f52b80 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 0x55ca73940020 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe954098520 tx=0x7fe954099dc0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe976f52b80 1 -- 172.21.15.46:0/2169010974 shutdown_connections 2024-03-18T20:39:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe976f52b80 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ca7399f530 0x55ca739a1920 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe976f52b80 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ca7392b570 0x55ca73940020 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe976f52b80 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ca7393b1c0 0x55ca7399dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe976f52b80 1 --2- 172.21.15.46:0/2169010974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ca7392be90 0x55ca73940560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe943fff700 0 client.16917 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:39:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe976f52b80 1 -- 172.21.15.46:0/2169010974 >> 172.21.15.46:0/2169010974 conn(0x55ca738a21c0 msgr2=0x55ca739332c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe976f52b80 1 -- 172.21.15.46:0/2169010974 shutdown_connections 2024-03-18T20:39:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe976f52b80 1 -- 172.21.15.46:0/2169010974 wait complete. 2024-03-18T20:39:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.529+0000 7fe976f52b80 20 client.16917 trim_cache size 0 max 16384 2024-03-18T20:39:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:39:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:39:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:39:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:39:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.530+0000 7fe96adbd700 1 -- 172.21.15.46:0/3051502672 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe9500a1310 msgr2=0x7fe9500a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.530+0000 7fe96adbd700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe9500a1310 0x7fe9500a37d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe964198810 tx=0x7fe94c00b410 comp rx=0 tx=0).stop 2024-03-18T20:39:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.530+0000 7fe96adbd700 1 -- 172.21.15.46:0/3051502672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640ffe20 msgr2=0x7fe9641972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:39:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.530+0000 7fe96adbd700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640ffe20 0x7fe9641972c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe95400eb30 tx=0x7fe95400ee40 comp rx=0 tx=0).stop 2024-03-18T20:39:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.530+0000 7fe96adbd700 1 -- 172.21.15.46:0/3051502672 shutdown_connections 2024-03-18T20:39:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.530+0000 7fe96adbd700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe9500a1310 0x7fe9500a37d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.530+0000 7fe96adbd700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9640ffe20 0x7fe9641972c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.530+0000 7fe96adbd700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe96419bba0 0x7fe96419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.530+0000 7fe96adbd700 1 --2- 172.21.15.46:0/3051502672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe964104f40 0x7fe964197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:39:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.530+0000 7fe96adbd700 1 -- 172.21.15.46:0/3051502672 >> 172.21.15.46:0/3051502672 conn(0x7fe9640fb680 msgr2=0x7fe964069310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:39:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.530+0000 7fe96adbd700 1 -- 172.21.15.46:0/3051502672 shutdown_connections 2024-03-18T20:39:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:39:54.530+0000 7fe96adbd700 1 -- 172.21.15.46:0/3051502672 wait complete. 2024-03-18T20:39:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:54 smithi155 ceph-mon[48344]: pgmap v117: 97 pgs: 97 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-03-18T20:39:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:54 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:54 smithi046 ceph-mon[43495]: pgmap v117: 97 pgs: 97 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-03-18T20:39:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:54 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:39:54.682 INFO:tasks.workunit.client.0.smithi046.stdout: 1 96539 1.56 MB/sec execute 49 sec latency 787.250 ms 2024-03-18T20:39:55.682 INFO:tasks.workunit.client.0.smithi046.stdout: 1 96539 1.53 MB/sec execute 50 sec latency 1787.306 ms 2024-03-18T20:39:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:55 smithi073 ceph-mon[47384]: pgmap v118: 97 pgs: 97 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-03-18T20:39:56.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:55 smithi155 ceph-mon[48344]: pgmap v118: 97 pgs: 97 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-03-18T20:39:56.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:55 smithi046 ceph-mon[43495]: pgmap v118: 97 pgs: 97 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-03-18T20:39:56.682 INFO:tasks.workunit.client.0.smithi046.stdout: 1 96539 1.50 MB/sec execute 51 sec latency 2787.431 ms 2024-03-18T20:39:57.682 INFO:tasks.workunit.client.0.smithi046.stdout: 1 96607 1.48 MB/sec execute 52 sec latency 3145.467 ms 2024-03-18T20:39:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:58 smithi073 ceph-mon[47384]: pgmap v119: 97 pgs: 97 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-03-18T20:39:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:58 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:39:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:58 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:39:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:58 smithi155 ceph-mon[48344]: pgmap v119: 97 pgs: 97 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-03-18T20:39:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:58 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:39:58.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:58 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:39:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:58 smithi046 ceph-mon[43495]: pgmap v119: 97 pgs: 97 active+clean; 43 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-03-18T20:39:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:58 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:39:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:58 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:39:58.682 INFO:tasks.workunit.client.0.smithi046.stdout: 1 96607 1.45 MB/sec execute 53 sec latency 1594.845 ms 2024-03-18T20:39:59.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:59 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:39:59.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:59 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:39:59.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:39:59 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:39:59.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:59 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:39:59.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:59 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:39:59.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:39:59 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:39:59.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:59 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:39:59.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:59 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:39:59.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:39:59 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:39:59.682 INFO:tasks.workunit.client.0.smithi046.stdout: 1 96607 1.42 MB/sec execute 54 sec latency 2594.967 ms 2024-03-18T20:40:00.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:00 smithi073 ceph-mon[47384]: pgmap v120: 97 pgs: 97 active+clean; 44 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:40:00.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:00 smithi073 ceph-mon[47384]: overall HEALTH_OK 2024-03-18T20:40:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:00 smithi155 ceph-mon[48344]: pgmap v120: 97 pgs: 97 active+clean; 44 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:40:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:00 smithi155 ceph-mon[48344]: overall HEALTH_OK 2024-03-18T20:40:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:00 smithi046 ceph-mon[43495]: pgmap v120: 97 pgs: 97 active+clean; 44 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:40:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:00 smithi046 ceph-mon[43495]: overall HEALTH_OK 2024-03-18T20:40:00.682 INFO:tasks.workunit.client.0.smithi046.stdout: 1 96607 1.40 MB/sec execute 55 sec latency 3595.066 ms 2024-03-18T20:40:01.682 INFO:tasks.workunit.client.0.smithi046.stdout: 1 98145 1.49 MB/sec execute 56 sec latency 3814.770 ms 2024-03-18T20:40:02.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:02 smithi073 ceph-mon[47384]: pgmap v121: 97 pgs: 97 active+clean; 44 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-03-18T20:40:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:02 smithi155 ceph-mon[48344]: pgmap v121: 97 pgs: 97 active+clean; 44 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-03-18T20:40:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:02 smithi046 ceph-mon[43495]: pgmap v121: 97 pgs: 97 active+clean; 44 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-03-18T20:40:02.682 INFO:tasks.workunit.client.0.smithi046.stdout: 1 99790 1.68 MB/sec execute 57 sec latency 10.871 ms 2024-03-18T20:40:03.682 INFO:tasks.workunit.client.0.smithi046.stdout: 1 100028 1.65 MB/sec execute 58 sec latency 798.344 ms 2024-03-18T20:40:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:04 smithi073 ceph-mon[47384]: pgmap v122: 97 pgs: 97 active+clean; 46 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-03-18T20:40:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:04 smithi155 ceph-mon[48344]: pgmap v122: 97 pgs: 97 active+clean; 46 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-03-18T20:40:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:04 smithi046 ceph-mon[43495]: pgmap v122: 97 pgs: 97 active+clean; 46 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-03-18T20:40:04.683 INFO:tasks.workunit.client.0.smithi046.stdout: 1 100028 1.62 MB/sec execute 59 sec latency 1798.437 ms 2024-03-18T20:40:05.683 INFO:tasks.workunit.client.0.smithi046.stdout: 1 100028 1.59 MB/sec execute 60 sec latency 2798.556 ms 2024-03-18T20:40:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:05 smithi073 ceph-mon[47384]: pgmap v123: 97 pgs: 97 active+clean; 46 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 440 KiB/s wr, 106 op/s 2024-03-18T20:40:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:05 smithi155 ceph-mon[48344]: pgmap v123: 97 pgs: 97 active+clean; 46 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 440 KiB/s wr, 106 op/s 2024-03-18T20:40:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:05 smithi046 ceph-mon[43495]: pgmap v123: 97 pgs: 97 active+clean; 46 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 440 KiB/s wr, 106 op/s 2024-03-18T20:40:06.683 INFO:tasks.workunit.client.0.smithi046.stdout: 1 100028 1.57 MB/sec execute 61 sec latency 3798.613 ms 2024-03-18T20:40:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.658+0000 7fc056202700 1 -- 172.21.15.46:0/725026752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0501018c0 msgr2=0x7fc050105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.658+0000 7fc056202700 1 --2- 172.21.15.46:0/725026752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0501018c0 0x7fc050105900 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc040009990 tx=0x7fc040009ca0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.658+0000 7fc056202700 1 -- 172.21.15.46:0/725026752 shutdown_connections 2024-03-18T20:40:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.658+0000 7fc056202700 1 --2- 172.21.15.46:0/725026752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc050106030 0x7fc05010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.658+0000 7fc056202700 1 --2- 172.21.15.46:0/725026752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc050100f10 0x7fc0501012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.658+0000 7fc056202700 1 --2- 172.21.15.46:0/725026752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0501018c0 0x7fc050105900 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.658+0000 7fc056202700 1 -- 172.21.15.46:0/725026752 >> 172.21.15.46:0/725026752 conn(0x7fc0500fc770 msgr2=0x7fc0500feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.658+0000 7fc056202700 1 -- 172.21.15.46:0/725026752 shutdown_connections 2024-03-18T20:40:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.658+0000 7fc056202700 1 -- 172.21.15.46:0/725026752 wait complete. 2024-03-18T20:40:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.659+0000 7fc056202700 1 Processor -- start 2024-03-18T20:40:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc056202700 1 -- start start 2024-03-18T20:40:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc056202700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050100f10 0x7fc050197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc056202700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc050106030 0x7fc050197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc056202700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc05019bac0 0x7fc05019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc056202700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05019c540 con 0x7fc050100f10 2024-03-18T20:40:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc056202700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05019c6b0 con 0x7fc050106030 2024-03-18T20:40:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc056202700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05019c870 con 0x7fc05019bac0 2024-03-18T20:40:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050100f10 0x7fc050197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050100f10 0x7fc050197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54234/0 (socket says 172.21.15.46:54234) 2024-03-18T20:40:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2171850124 learned_addr learned my addr 172.21.15.46:0/2171850124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc04effd700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc050106030 0x7fc050197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc04ffff700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc05019bac0 0x7fc05019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2171850124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc05019bac0 msgr2=0x7fc05019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc05019bac0 0x7fc05019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2171850124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc050106030 msgr2=0x7fc050197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc050106030 0x7fc050197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc04f7fe700 1 -- 172.21.15.46:0/2171850124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0400095f0 con 0x7fc050100f10 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc04ffff700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc05019bac0 0x7fc05019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.660+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050100f10 0x7fc050197230 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc04400e9a0 tx=0x7fc04400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.661+0000 7fc04cff9700 1 -- 172.21.15.46:0/2171850124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc044009790 con 0x7fc050100f10 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.661+0000 7fc04cff9700 1 -- 172.21.15.46:0/2171850124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc04400cbd0 con 0x7fc050100f10 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.661+0000 7fc056202700 1 -- 172.21.15.46:0/2171850124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0501a3dd0 con 0x7fc050100f10 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.661+0000 7fc04cff9700 1 -- 172.21.15.46:0/2171850124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc044018680 con 0x7fc050100f10 2024-03-18T20:40:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.661+0000 7fc056202700 1 -- 172.21.15.46:0/2171850124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0501a4250 con 0x7fc050100f10 2024-03-18T20:40:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.663+0000 7fc04cff9700 1 -- 172.21.15.46:0/2171850124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc04400cd40 con 0x7fc050100f10 2024-03-18T20:40:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.663+0000 7fc04cff9700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0380a1290 0x7fc0380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.664+0000 7fc04cff9700 1 -- 172.21.15.46:0/2171850124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc044014070 con 0x7fc050100f10 2024-03-18T20:40:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.664+0000 7fc04effd700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0380a1290 0x7fc0380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.664+0000 7fc04effd700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0380a1290 0x7fc0380a3750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc04000b2e0 tx=0x7fc040005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc062397b80 1 Processor -- start 2024-03-18T20:40:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc062397b80 1 -- start start 2024-03-18T20:40:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc062397b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66570 0x5647aca66950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc062397b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66e90 0x5647aca71870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc062397b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 0x5647aca741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc062397b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ac9c0930 con 0x5647aca71db0 2024-03-18T20:40:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc062397b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ac9be2f0 con 0x5647aca66570 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc062397b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ac9b9be0 con 0x5647aca66e90 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 0x5647aca741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 0x5647aca741a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54244/0 (socket says 172.21.15.46:54244) 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc04ffff700 1 -- 172.21.15.46:0/2457687814 learned_addr learned my addr 172.21.15.46:0/2457687814 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/2457687814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66570 0x5647aca66950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.665+0000 7fc04effd700 1 --2- 172.21.15.46:0/2457687814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66e90 0x5647aca71870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.666+0000 7fc04ffff700 1 -- 172.21.15.46:0/2457687814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66e90 msgr2=0x5647aca71870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.666+0000 7fc04ffff700 1 --2- 172.21.15.46:0/2457687814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66e90 0x5647aca71870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.666+0000 7fc04ffff700 1 -- 172.21.15.46:0/2457687814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66570 msgr2=0x5647aca66950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.666+0000 7fc04ffff700 1 --2- 172.21.15.46:0/2457687814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66570 0x5647aca66950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.666+0000 7fc04ffff700 1 -- 172.21.15.46:0/2457687814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647ac83b350 con 0x5647aca71db0 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.666+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/2457687814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66570 0x5647aca66950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.666+0000 7fc04ffff700 1 --2- 172.21.15.46:0/2457687814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 0x5647aca741a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc048009440 tx=0x7fc048009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f815dce9e6616a40 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.666+0000 7fc035ffb700 1 -- 172.21.15.46:0/2457687814 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800e070 con 0x5647aca71db0 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.666+0000 7fc035ffb700 1 -- 172.21.15.46:0/2457687814 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc04800f040 con 0x5647aca71db0 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.666+0000 7fc035ffb700 1 -- 172.21.15.46:0/2457687814 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048015c30 con 0x5647aca71db0 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.667+0000 7fc062397b80 1 -- 172.21.15.46:0/2457687814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 msgr2=0x5647aca741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.667+0000 7fc062397b80 1 --2- 172.21.15.46:0/2457687814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 0x5647aca741a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc048009440 tx=0x7fc048009920 comp rx=0 tx=0).stop 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.667+0000 7fc062397b80 1 -- 172.21.15.46:0/2457687814 shutdown_connections 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.667+0000 7fc062397b80 1 --2- 172.21.15.46:0/2457687814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 0x5647aca741a0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.667+0000 7fc062397b80 1 --2- 172.21.15.46:0/2457687814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66570 0x5647aca66950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.667+0000 7fc062397b80 1 --2- 172.21.15.46:0/2457687814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66e90 0x5647aca71870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.667+0000 7fc062397b80 1 -- 172.21.15.46:0/2457687814 >> 172.21.15.46:0/2457687814 conn(0x5647ac9dd1c0 msgr2=0x5647aca62a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.667+0000 7fc062397b80 1 -- 172.21.15.46:0/2457687814 shutdown_connections 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.667+0000 7fc062397b80 1 -- 172.21.15.46:0/2457687814 wait complete. 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.667+0000 7fc062397b80 1 -- start start 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc062397b80 1 Processor -- start 2024-03-18T20:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc062397b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66570 0x5647aca717a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc062397b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66e90 0x5647aca6ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc062397b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca71db0 0x5647aca6f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc062397b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ac955d20 con 0x5647aca66570 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc062397b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ac9559c0 con 0x5647aca71db0 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc062397b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ac83d640 con 0x5647aca66e90 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc062397b80 10 client.0 _ll_register_callbacks cb 0x5647ac7c1340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66570 0x5647aca717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66570 0x5647aca717a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54248/0 (socket says 172.21.15.46:54248) 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc04f7fe700 1 -- 172.21.15.46:0/4286857703 learned_addr learned my addr 172.21.15.46:0/4286857703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc04effd700 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66e90 0x5647aca6ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc04ffff700 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca71db0 0x5647aca6f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc04f7fe700 1 -- 172.21.15.46:0/4286857703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66e90 msgr2=0x5647aca6ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66e90 0x5647aca6ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc04f7fe700 1 -- 172.21.15.46:0/4286857703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca71db0 msgr2=0x5647aca6f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca71db0 0x5647aca6f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.668+0000 7fc04f7fe700 1 -- 172.21.15.46:0/4286857703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0501a4250 con 0x5647aca66570 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66570 0x5647aca717a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc044096ac0 tx=0x7fc04409a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b1054ddce37be8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc02effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc02effd700 1 -- 172.21.15.46:0/4286857703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0440a24d0 con 0x5647aca66570 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc062397b80 1 -- 172.21.15.46:0/4286857703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647aca5a6c0 con 0x5647aca66570 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc02effd700 1 -- 172.21.15.46:0/4286857703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc044099750 con 0x5647aca66570 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc02effd700 1 -- 172.21.15.46:0/4286857703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0440b82b0 con 0x5647aca66570 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc062397b80 10 client.16929 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc062397b80 1 -- 172.21.15.46:0/4286857703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fc0480090f0 con 0x5647aca66570 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc02effd700 1 -- 172.21.15.46:0/4286857703 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc044063070 con 0x5647aca66570 2024-03-18T20:40:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc02effd700 1 -- 172.21.15.46:0/4286857703 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc044099120 con 0x5647aca66570 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc062397b80 10 client.16929 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc062397b80 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc062397b80 20 client.16929 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc062397b80 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647aca819c0 0x5647aca83db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc062397b80 4 client.16929 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.669+0000 7fc062397b80 1 -- 172.21.15.46:0/4286857703 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647ac9b26a0 con 0x5647aca819c0 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.670+0000 7fc04effd700 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647aca819c0 0x5647aca83db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.670+0000 7fc02effd700 10 client.16929 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.670+0000 7fc04effd700 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647aca819c0 0x5647aca83db0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab9c79a0ad8400d server_cookie=cbe4ef9116c8db3 in_seq=0 out_seq=0 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc02effd700 1 -- 172.21.15.46:0/4286857703 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5647ac9b26a0 con 0x5647aca819c0 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc02effd700 10 client.16929 handle_command_reply: tid=0 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc062397b80 1 client.16929 shutdown 2024-03-18T20:40:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc062397b80 1 -- 172.21.15.46:0/4286857703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66570 msgr2=0x5647aca717a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc062397b80 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66570 0x5647aca717a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc044096ac0 tx=0x7fc04409a050 comp rx=0 tx=0).stop 2024-03-18T20:40:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc062397b80 1 -- 172.21.15.46:0/4286857703 shutdown_connections 2024-03-18T20:40:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc062397b80 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647aca819c0 0x5647aca83db0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc062397b80 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66570 0x5647aca717a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc062397b80 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca71db0 0x5647aca6f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc062397b80 1 --2- 172.21.15.46:0/4286857703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66e90 0x5647aca6ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc02effd700 0 client.16929 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc062397b80 1 -- 172.21.15.46:0/4286857703 >> 172.21.15.46:0/4286857703 conn(0x5647ac9dd1c0 msgr2=0x5647aca62650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc062397b80 1 -- 172.21.15.46:0/4286857703 shutdown_connections 2024-03-18T20:40:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc062397b80 1 -- 172.21.15.46:0/4286857703 wait complete. 2024-03-18T20:40:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.671+0000 7fc062397b80 20 client.16929 trim_cache size 0 max 16384 2024-03-18T20:40:07.682 INFO:tasks.workunit.client.0.smithi046.stdout: 1 100084 1.54 MB/sec execute 62 sec latency 4156.088 ms 2024-03-18T20:40:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.686+0000 7fc062397b80 1 Processor -- start 2024-03-18T20:40:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.686+0000 7fc062397b80 1 -- start start 2024-03-18T20:40:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc062397b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66570 0x5647aca74130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc062397b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66e90 0x5647aca61120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc062397b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca71db0 0x5647aca712d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc062397b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647aca6f680 con 0x5647aca66e90 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc062397b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647aca6d040 con 0x5647aca66570 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc062397b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647aca6a950 con 0x5647aca71db0 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc04effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66e90 0x5647aca61120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc04effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66e90 0x5647aca61120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54264/0 (socket says 172.21.15.46:54264) 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc04effd700 1 -- 172.21.15.46:0/1744425007 learned_addr learned my addr 172.21.15.46:0/1744425007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/1744425007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66570 0x5647aca74130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc04ffff700 1 --2- 172.21.15.46:0/1744425007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca71db0 0x5647aca712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc04effd700 1 -- 172.21.15.46:0/1744425007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca71db0 msgr2=0x5647aca712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc04effd700 1 --2- 172.21.15.46:0/1744425007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca71db0 0x5647aca712d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc04effd700 1 -- 172.21.15.46:0/1744425007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66570 msgr2=0x5647aca74130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc04effd700 1 --2- 172.21.15.46:0/1744425007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66570 0x5647aca74130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc04effd700 1 -- 172.21.15.46:0/1744425007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647ac7e7b00 con 0x5647aca66e90 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc04effd700 1 --2- 172.21.15.46:0/1744425007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66e90 0x5647aca61120 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc040009990 tx=0x7fc040023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93dd65d6891e96b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc0367fc700 1 -- 172.21.15.46:0/1744425007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040035e20 con 0x5647aca66e90 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc0367fc700 1 -- 172.21.15.46:0/1744425007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc04001e040 con 0x5647aca66e90 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.687+0000 7fc0367fc700 1 -- 172.21.15.46:0/1744425007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040004c00 con 0x5647aca66e90 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.688+0000 7fc062397b80 1 -- 172.21.15.46:0/1744425007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66e90 msgr2=0x5647aca61120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.688+0000 7fc062397b80 1 --2- 172.21.15.46:0/1744425007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66e90 0x5647aca61120 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc040009990 tx=0x7fc040023000 comp rx=0 tx=0).stop 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.688+0000 7fc062397b80 1 -- 172.21.15.46:0/1744425007 shutdown_connections 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.688+0000 7fc062397b80 1 --2- 172.21.15.46:0/1744425007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca71db0 0x5647aca712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.688+0000 7fc062397b80 1 --2- 172.21.15.46:0/1744425007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66570 0x5647aca74130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.688+0000 7fc062397b80 1 --2- 172.21.15.46:0/1744425007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca66e90 0x5647aca61120 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.688+0000 7fc062397b80 1 -- 172.21.15.46:0/1744425007 >> 172.21.15.46:0/1744425007 conn(0x5647ac9dd1c0 msgr2=0x5647aca5e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.688+0000 7fc062397b80 1 -- 172.21.15.46:0/1744425007 shutdown_connections 2024-03-18T20:40:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.688+0000 7fc062397b80 1 -- 172.21.15.46:0/1744425007 wait complete. 2024-03-18T20:40:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.688+0000 7fc062397b80 1 -- start start 2024-03-18T20:40:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc062397b80 1 Processor -- start 2024-03-18T20:40:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc062397b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66570 0x5647acacda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc062397b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66e90 0x5647acacdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc062397b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 0x5647aca74a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc062397b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647aca74fe0 con 0x5647aca71db0 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc062397b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647aca75150 con 0x5647aca66e90 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc062397b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647aca75310 con 0x5647aca66570 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc062397b80 10 client.0 _ll_register_callbacks cb 0x7fc044099c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 0x5647aca74a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 0x5647aca74a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54268/0 (socket says 172.21.15.46:54268) 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc04ffff700 1 -- 172.21.15.46:0/1475320370 learned_addr learned my addr 172.21.15.46:0/1475320370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc04effd700 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66e90 0x5647acacdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66570 0x5647acacda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc04effd700 1 -- 172.21.15.46:0/1475320370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66570 msgr2=0x5647acacda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc04effd700 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66570 0x5647acacda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc04effd700 1 -- 172.21.15.46:0/1475320370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 msgr2=0x5647aca74a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc04effd700 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 0x5647aca74a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc04effd700 1 -- 172.21.15.46:0/1475320370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05010e420 con 0x5647aca66e90 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.689+0000 7fc04ffff700 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 0x5647aca74a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc04effd700 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66e90 0x5647acacdf80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc040005f50 tx=0x7fc04000be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faa30e56e37c7d60 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc02effd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc02effd700 1 -- 172.21.15.46:0/1475320370 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040035580 con 0x5647aca66e90 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc02effd700 1 -- 172.21.15.46:0/1475320370 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc0400047e0 con 0x5647aca66e90 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc062397b80 1 -- 172.21.15.46:0/1475320370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc04800f040 con 0x5647aca66e90 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc02effd700 1 -- 172.21.15.46:0/1475320370 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc040005540 con 0x5647aca66e90 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc062397b80 10 client.26657 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc062397b80 1 -- 172.21.15.46:0/1475320370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5647ac97fe40 con 0x5647aca66e90 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc02effd700 1 -- 172.21.15.46:0/1475320370 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc04800f040 con 0x5647aca66e90 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc02effd700 1 -- 172.21.15.46:0/1475320370 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc04000fad0 con 0x5647aca66e90 2024-03-18T20:40:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc062397b80 10 client.26657 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc062397b80 10 client.26657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc062397b80 20 client.26657 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.690+0000 7fc062397b80 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647acad6590 0x5647acad8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.691+0000 7fc062397b80 4 client.26657 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:40:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.691+0000 7fc062397b80 1 -- 172.21.15.46:0/1475320370 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5647ac808c90 con 0x5647acad6590 2024-03-18T20:40:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.691+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647acad6590 0x5647acad8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.691+0000 7fc04f7fe700 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647acad6590 0x5647acad8980 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2112208f55e73b8e server_cookie=ada402419e92eabc in_seq=0 out_seq=0 2024-03-18T20:40:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.691+0000 7fc02effd700 10 client.26657 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc02effd700 1 -- 172.21.15.46:0/1475320370 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5647ac808c90 con 0x5647acad6590 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc02effd700 10 client.26657 handle_command_reply: tid=0 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc062397b80 1 client.26657 shutdown 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc062397b80 1 -- 172.21.15.46:0/1475320370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66e90 msgr2=0x5647acacdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc062397b80 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66e90 0x5647acacdf80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc040005f50 tx=0x7fc04000be10 comp rx=0 tx=0).stop 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc062397b80 1 -- 172.21.15.46:0/1475320370 shutdown_connections 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc062397b80 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647acad6590 0x5647acad8980 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc062397b80 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647aca71db0 0x5647aca74a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc062397b80 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647aca66570 0x5647acacda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc02effd700 0 client.26657 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc062397b80 1 --2- 172.21.15.46:0/1475320370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647aca66e90 0x5647acacdf80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc062397b80 1 -- 172.21.15.46:0/1475320370 >> 172.21.15.46:0/1475320370 conn(0x5647ac9dd1c0 msgr2=0x5647aca6e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc062397b80 1 -- 172.21.15.46:0/1475320370 shutdown_connections 2024-03-18T20:40:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc062397b80 1 -- 172.21.15.46:0/1475320370 wait complete. 2024-03-18T20:40:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.718+0000 7fc062397b80 20 client.26657 trim_cache size 0 max 16384 2024-03-18T20:40:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:40:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:40:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:40:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:40:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.719+0000 7fc056202700 1 -- 172.21.15.46:0/2171850124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0380a1290 msgr2=0x7fc0380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.719+0000 7fc056202700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0380a1290 0x7fc0380a3750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc04000b2e0 tx=0x7fc040005960 comp rx=0 tx=0).stop 2024-03-18T20:40:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.719+0000 7fc056202700 1 -- 172.21.15.46:0/2171850124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050100f10 msgr2=0x7fc050197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.719+0000 7fc056202700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050100f10 0x7fc050197230 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc04400e9a0 tx=0x7fc04400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.719+0000 7fc056202700 1 -- 172.21.15.46:0/2171850124 shutdown_connections 2024-03-18T20:40:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.719+0000 7fc056202700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0380a1290 0x7fc0380a3750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.719+0000 7fc056202700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc050100f10 0x7fc050197230 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.719+0000 7fc056202700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc05019bac0 0x7fc05019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.719+0000 7fc056202700 1 --2- 172.21.15.46:0/2171850124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc050106030 0x7fc050197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.719+0000 7fc056202700 1 -- 172.21.15.46:0/2171850124 >> 172.21.15.46:0/2171850124 conn(0x7fc0500fc770 msgr2=0x7fc0500fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.719+0000 7fc056202700 1 -- 172.21.15.46:0/2171850124 shutdown_connections 2024-03-18T20:40:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:07.719+0000 7fc056202700 1 -- 172.21.15.46:0/2171850124 wait complete. 2024-03-18T20:40:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:08 smithi073 ceph-mon[47384]: pgmap v124: 97 pgs: 97 active+clean; 46 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 440 KiB/s wr, 106 op/s 2024-03-18T20:40:08.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:08 smithi155 ceph-mon[48344]: pgmap v124: 97 pgs: 97 active+clean; 46 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 440 KiB/s wr, 106 op/s 2024-03-18T20:40:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:08 smithi046 ceph-mon[43495]: pgmap v124: 97 pgs: 97 active+clean; 46 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 440 KiB/s wr, 106 op/s 2024-03-18T20:40:08.683 INFO:tasks.workunit.client.0.smithi046.stdout: 1 100084 1.52 MB/sec execute 63 sec latency 1602.509 ms 2024-03-18T20:40:09.683 INFO:tasks.workunit.client.0.smithi046.stdout: 1 100084 1.50 MB/sec execute 64 sec latency 2602.626 ms 2024-03-18T20:40:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:10 smithi073 ceph-mon[47384]: pgmap v125: 97 pgs: 97 active+clean; 43 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 493 KiB/s wr, 125 op/s 2024-03-18T20:40:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:10 smithi155 ceph-mon[48344]: pgmap v125: 97 pgs: 97 active+clean; 43 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 493 KiB/s wr, 125 op/s 2024-03-18T20:40:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:10 smithi046 ceph-mon[43495]: pgmap v125: 97 pgs: 97 active+clean; 43 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 493 KiB/s wr, 125 op/s 2024-03-18T20:40:10.683 INFO:tasks.workunit.client.0.smithi046.stdout: 1 100084 1.47 MB/sec execute 65 sec latency 3602.708 ms 2024-03-18T20:40:11.683 INFO:tasks.workunit.client.0.smithi046.stdout: 1 100084 1.45 MB/sec execute 66 sec latency 4602.809 ms 2024-03-18T20:40:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:12 smithi073 ceph-mon[47384]: pgmap v126: 97 pgs: 97 active+clean; 43 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 394 KiB/s wr, 96 op/s 2024-03-18T20:40:12.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:12 smithi155 ceph-mon[48344]: pgmap v126: 97 pgs: 97 active+clean; 43 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 394 KiB/s wr, 96 op/s 2024-03-18T20:40:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:12 smithi046 ceph-mon[43495]: pgmap v126: 97 pgs: 97 active+clean; 43 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 394 KiB/s wr, 96 op/s 2024-03-18T20:40:12.683 INFO:tasks.workunit.client.0.smithi046.stdout: 1 101811 1.53 MB/sec execute 67 sec latency 4955.124 ms 2024-03-18T20:40:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:13 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:40:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:13 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:40:13.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:13 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:40:13.683 INFO:tasks.workunit.client.0.smithi046.stdout: 1 103354 1.69 MB/sec execute 68 sec latency 25.803 ms 2024-03-18T20:40:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:14 smithi073 ceph-mon[47384]: pgmap v127: 97 pgs: 97 active+clean; 44 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 556 KiB/s wr, 124 op/s 2024-03-18T20:40:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:14 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:14 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:14 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:14 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:14 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:14 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:14 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:40:14.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:14 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:40:14.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:14 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:14 smithi155 ceph-mon[48344]: pgmap v127: 97 pgs: 97 active+clean; 44 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 556 KiB/s wr, 124 op/s 2024-03-18T20:40:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:14 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:14 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:14 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:14 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:14 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:14 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:14 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:40:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:14 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:40:14.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:14 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:14 smithi046 ceph-mon[43495]: pgmap v127: 97 pgs: 97 active+clean; 44 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 556 KiB/s wr, 124 op/s 2024-03-18T20:40:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:14 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:14 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:14 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:14 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:14 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:14 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:14 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:40:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:14 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:40:14.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:14 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:40:14.683 INFO:tasks.workunit.client.0.smithi046.stdout: 1 103700 1.67 MB/sec execute 69 sec latency 741.058 ms 2024-03-18T20:40:15.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 103700 1.64 MB/sec execute 70 sec latency 1741.162 ms 2024-03-18T20:40:16.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:15 smithi073 ceph-mon[47384]: pgmap v128: 97 pgs: 97 active+clean; 44 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 215 KiB/s wr, 47 op/s 2024-03-18T20:40:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:15 smithi155 ceph-mon[48344]: pgmap v128: 97 pgs: 97 active+clean; 44 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 215 KiB/s wr, 47 op/s 2024-03-18T20:40:16.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:15 smithi046 ceph-mon[43495]: pgmap v128: 97 pgs: 97 active+clean; 44 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 215 KiB/s wr, 47 op/s 2024-03-18T20:40:16.684 INFO:tasks.workunit.client.0.smithi046.stdout: 1 103700 1.62 MB/sec execute 71 sec latency 2741.284 ms 2024-03-18T20:40:17.684 INFO:tasks.workunit.client.0.smithi046.stdout: 1 105481 1.70 MB/sec execute 72 sec latency 3097.295 ms 2024-03-18T20:40:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:18 smithi073 ceph-mon[47384]: pgmap v129: 97 pgs: 97 active+clean; 44 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 215 KiB/s wr, 47 op/s 2024-03-18T20:40:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:18 smithi155 ceph-mon[48344]: pgmap v129: 97 pgs: 97 active+clean; 44 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 215 KiB/s wr, 47 op/s 2024-03-18T20:40:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:18 smithi046 ceph-mon[43495]: pgmap v129: 97 pgs: 97 active+clean; 44 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 215 KiB/s wr, 47 op/s 2024-03-18T20:40:18.684 INFO:tasks.workunit.client.0.smithi046.stdout: 1 106986 1.83 MB/sec execute 73 sec latency 12.737 ms 2024-03-18T20:40:19.684 INFO:tasks.workunit.client.0.smithi046.stdout: 1 107173 1.81 MB/sec execute 74 sec latency 851.407 ms 2024-03-18T20:40:20.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:40:20.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:40:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.336+0000 7f2ad5b04700 1 -- 172.21.15.46:0/2547989776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ad0106080 msgr2=0x7f2ad010dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.336+0000 7f2ad5b04700 1 --2- 172.21.15.46:0/2547989776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ad0106080 0x7f2ad010dc00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800b0a0 tx=0x7f2ac800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.336+0000 7f2ad5b04700 1 -- 172.21.15.46:0/2547989776 shutdown_connections 2024-03-18T20:40:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.336+0000 7f2ad5b04700 1 --2- 172.21.15.46:0/2547989776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ad0106080 0x7f2ad010dc00 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.336+0000 7f2ad5b04700 1 --2- 172.21.15.46:0/2547989776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad0100f60 0x7f2ad0101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.336+0000 7f2ad5b04700 1 --2- 172.21.15.46:0/2547989776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad0101910 0x7f2ad0105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.336+0000 7f2ad5b04700 1 -- 172.21.15.46:0/2547989776 >> 172.21.15.46:0/2547989776 conn(0x7f2ad00fc7c0 msgr2=0x7f2ad00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.336+0000 7f2ad5b04700 1 -- 172.21.15.46:0/2547989776 shutdown_connections 2024-03-18T20:40:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.336+0000 7f2ad5b04700 1 -- 172.21.15.46:0/2547989776 wait complete. 2024-03-18T20:40:20.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.337+0000 7f2ad5b04700 1 Processor -- start 2024-03-18T20:40:20.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.337+0000 7f2ad5b04700 1 -- start start 2024-03-18T20:40:20.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.337+0000 7f2ad5b04700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad0100f60 0x7f2ad0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.337+0000 7f2ad5b04700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad0106080 0x7f2ad0197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.337+0000 7f2ad5b04700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ad019bbe0 0x7f2ad019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.337+0000 7f2ad5b04700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad019c660 con 0x7f2ad019bbe0 2024-03-18T20:40:20.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.337+0000 7f2ad5b04700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad019c7d0 con 0x7f2ad0100f60 2024-03-18T20:40:20.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.337+0000 7f2ad5b04700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad019c990 con 0x7f2ad0106080 2024-03-18T20:40:20.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2acffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ad019bbe0 0x7f2ad019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2aceffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad0106080 0x7f2ad0197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2aceffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad0106080 0x7f2ad0197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58644/0 (socket says 172.21.15.46:58644) 2024-03-18T20:40:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2acffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ad019bbe0 0x7f2ad019c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34244/0 (socket says 172.21.15.46:34244) 2024-03-18T20:40:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2aceffd700 1 -- 172.21.15.46:0/237293955 learned_addr learned my addr 172.21.15.46:0/237293955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad0100f60 0x7f2ad0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2acffff700 1 -- 172.21.15.46:0/237293955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad0106080 msgr2=0x7f2ad0197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2acffff700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad0106080 0x7f2ad0197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2acffff700 1 -- 172.21.15.46:0/237293955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad0100f60 msgr2=0x7f2ad0197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2acffff700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad0100f60 0x7f2ad0197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2acffff700 1 -- 172.21.15.46:0/237293955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac8009d00 con 0x7f2ad019bbe0 2024-03-18T20:40:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2acffff700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ad019bbe0 0x7f2ad019c060 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80062a0 tx=0x7f2ac8009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2accff9700 1 -- 172.21.15.46:0/237293955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac800e040 con 0x7f2ad019bbe0 2024-03-18T20:40:20.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2accff9700 1 -- 172.21.15.46:0/237293955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac800bde0 con 0x7f2ad019bbe0 2024-03-18T20:40:20.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2ad5b04700 1 -- 172.21.15.46:0/237293955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad01a3e90 con 0x7f2ad019bbe0 2024-03-18T20:40:20.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2accff9700 1 -- 172.21.15.46:0/237293955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac801b6b0 con 0x7f2ad019bbe0 2024-03-18T20:40:20.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.338+0000 7f2ad5b04700 1 -- 172.21.15.46:0/237293955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad01a4310 con 0x7f2ad019bbe0 2024-03-18T20:40:20.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.341+0000 7f2accff9700 1 -- 172.21.15.46:0/237293955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2ac801d480 con 0x7f2ad019bbe0 2024-03-18T20:40:20.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.341+0000 7f2accff9700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ab80a1290 0x7f2ab80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.342+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ab80a1290 0x7f2ab80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.342+0000 7f2accff9700 1 -- 172.21.15.46:0/237293955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2ac80ce4b0 con 0x7f2ad019bbe0 2024-03-18T20:40:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.342+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ab80a1290 0x7f2ab80a3750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0007be0 tx=0x7f2ac00073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.342+0000 7f2ae1c99b80 1 Processor -- start 2024-03-18T20:40:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.342+0000 7f2ae1c99b80 1 -- start start 2024-03-18T20:40:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2ae1c99b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099589a60 0x558099589e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2ae1c99b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 0x558099594ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2ae1c99b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099595220 0x558099597610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2ae1c99b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580994e3990 con 0x55809958a380 2024-03-18T20:40:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2ae1c99b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580994e1350 con 0x558099589a60 2024-03-18T20:40:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2ae1c99b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580994dcc40 con 0x558099595220 2024-03-18T20:40:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2aceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 0x558099594ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2aceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 0x558099594ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34250/0 (socket says 172.21.15.46:34250) 2024-03-18T20:40:20.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2aceffd700 1 -- 172.21.15.46:0/2920530782 learned_addr learned my addr 172.21.15.46:0/2920530782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/2920530782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099589a60 0x558099589e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2acffff700 1 --2- 172.21.15.46:0/2920530782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099595220 0x558099597610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2aceffd700 1 -- 172.21.15.46:0/2920530782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099595220 msgr2=0x558099597610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2aceffd700 1 --2- 172.21.15.46:0/2920530782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099595220 0x558099597610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2aceffd700 1 -- 172.21.15.46:0/2920530782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099589a60 msgr2=0x558099589e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2aceffd700 1 --2- 172.21.15.46:0/2920530782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099589a60 0x558099589e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2aceffd700 1 -- 172.21.15.46:0/2920530782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55809935e3b0 con 0x55809958a380 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2acffff700 1 --2- 172.21.15.46:0/2920530782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099595220 0x558099597610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.343+0000 7f2aceffd700 1 --2- 172.21.15.46:0/2920530782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 0x558099594ce0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400bd70 tx=0x7f2ac40106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1d474127ee90e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.344+0000 7f2ab5ffb700 1 -- 172.21.15.46:0/2920530782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4010cc0 con 0x55809958a380 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.344+0000 7f2ab5ffb700 1 -- 172.21.15.46:0/2920530782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac4015070 con 0x55809958a380 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.344+0000 7f2ab5ffb700 1 -- 172.21.15.46:0/2920530782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac401b690 con 0x55809958a380 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.344+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2920530782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 msgr2=0x558099594ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.344+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2920530782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 0x558099594ce0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400bd70 tx=0x7f2ac40106b0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.344+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2920530782 shutdown_connections 2024-03-18T20:40:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.344+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2920530782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099595220 0x558099597610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.344+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2920530782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099589a60 0x558099589e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.344+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2920530782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 0x558099594ce0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.344+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2920530782 >> 172.21.15.46:0/2920530782 conn(0x558099500220 msgr2=0x558099586020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.344+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2920530782 shutdown_connections 2024-03-18T20:40:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.344+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2920530782 wait complete. 2024-03-18T20:40:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.345+0000 7f2ae1c99b80 1 -- start start 2024-03-18T20:40:20.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.345+0000 7f2ae1c99b80 1 Processor -- start 2024-03-18T20:40:20.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.345+0000 7f2ae1c99b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099589a60 0x558099594cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.345+0000 7f2ae1c99b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 0x558099591f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.345+0000 7f2ae1c99b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099595220 0x558099592680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.345+0000 7f2ae1c99b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558099478d80 con 0x55809958a380 2024-03-18T20:40:20.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.345+0000 7f2ae1c99b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558099478a20 con 0x558099595220 2024-03-18T20:40:20.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.345+0000 7f2ae1c99b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580993606a0 con 0x558099589a60 2024-03-18T20:40:20.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2ae1c99b80 10 client.0 _ll_register_callbacks cb 0x5580992e43a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:20.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2aceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 0x558099591f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2aceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 0x558099591f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34260/0 (socket says 172.21.15.46:34260) 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2aceffd700 1 -- 172.21.15.46:0/2533048276 learned_addr learned my addr 172.21.15.46:0/2533048276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099589a60 0x558099594cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2acffff700 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099595220 0x558099592680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2aceffd700 1 -- 172.21.15.46:0/2533048276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099589a60 msgr2=0x558099594cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2aceffd700 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099589a60 0x558099594cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2aceffd700 1 -- 172.21.15.46:0/2533048276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099595220 msgr2=0x558099592680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2aceffd700 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099595220 0x558099592680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2aceffd700 1 -- 172.21.15.46:0/2533048276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac400b9a0 con 0x55809958a380 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099589a60 0x558099594cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2acffff700 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099595220 0x558099592680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2aceffd700 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 0x558099591f40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4000c00 tx=0x7f2ac4011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba603e0c1f95217b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2aaeffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2aaeffd700 1 -- 172.21.15.46:0/2533048276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4020070 con 0x55809958a380 2024-03-18T20:40:20.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2aaeffd700 1 -- 172.21.15.46:0/2533048276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2ac4017070 con 0x55809958a380 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.346+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2533048276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55809957dad0 con 0x55809958a380 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.347+0000 7f2aaeffd700 1 -- 172.21.15.46:0/2533048276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ac4004050 con 0x55809958a380 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.347+0000 7f2ae1c99b80 10 client.16947 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.347+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2533048276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55809930ab60 con 0x55809958a380 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.347+0000 7f2aaeffd700 1 -- 172.21.15.46:0/2533048276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac4031080 con 0x55809958a380 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.347+0000 7f2aaeffd700 1 -- 172.21.15.46:0/2533048276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2ac40050d0 con 0x55809958a380 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.347+0000 7f2ae1c99b80 10 client.16947 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.347+0000 7f2ae1c99b80 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.347+0000 7f2ae1c99b80 20 client.16947 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.347+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5580995a9180 0x5580995ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.347+0000 7f2ae1c99b80 4 client.16947 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.347+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2533048276 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580994d5700 con 0x5580995a9180 2024-03-18T20:40:20.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.347+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5580995a9180 0x5580995ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.348+0000 7f2aaeffd700 10 client.16947 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:20.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.348+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5580995a9180 0x5580995ab5a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c803ea69d2252a2 server_cookie=6cb8e8b70c8aa1aa in_seq=0 out_seq=0 2024-03-18T20:40:20.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.348+0000 7f2aaeffd700 1 -- 172.21.15.46:0/2533048276 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5580994d5700 con 0x5580995a9180 2024-03-18T20:40:20.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.348+0000 7f2aaeffd700 10 client.16947 handle_command_reply: tid=0 2024-03-18T20:40:20.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.348+0000 7f2ae1c99b80 1 client.16947 shutdown 2024-03-18T20:40:20.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.349+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2533048276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 msgr2=0x558099591f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.349+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 0x558099591f40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4000c00 tx=0x7f2ac4011bd0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.349+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2533048276 shutdown_connections 2024-03-18T20:40:20.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.349+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5580995a9180 0x5580995ab5a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.349+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099595220 0x558099592680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.349+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099589a60 0x558099594cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.349+0000 7f2aaeffd700 0 client.16947 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:20.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.349+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2533048276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55809958a380 0x558099591f40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.349+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2533048276 >> 172.21.15.46:0/2533048276 conn(0x558099500220 msgr2=0x558099585c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.349+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2533048276 shutdown_connections 2024-03-18T20:40:20.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.349+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2533048276 wait complete. 2024-03-18T20:40:20.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.349+0000 7f2ae1c99b80 20 client.16947 trim_cache size 0 max 16384 2024-03-18T20:40:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.367+0000 7f2ae1c99b80 1 Processor -- start 2024-03-18T20:40:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.367+0000 7f2ae1c99b80 1 -- start start 2024-03-18T20:40:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.367+0000 7f2ae1c99b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 0x5580995a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.367+0000 7f2ae1c99b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55809958a380 0x5580995947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.367+0000 7f2ae1c99b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099595220 0x5580995917d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.367+0000 7f2ae1c99b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580995933d0 con 0x558099589a60 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.367+0000 7f2ae1c99b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55809958ca20 con 0x55809958a380 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.367+0000 7f2ae1c99b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558099592a00 con 0x558099595220 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.367+0000 7f2acf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 0x5580995a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.367+0000 7f2acf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 0x5580995a02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34268/0 (socket says 172.21.15.46:34268) 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.367+0000 7f2acf7fe700 1 -- 172.21.15.46:0/2981724406 learned_addr learned my addr 172.21.15.46:0/2981724406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.367+0000 7f2aceffd700 1 --2- 172.21.15.46:0/2981724406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55809958a380 0x5580995947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.368+0000 7f2acffff700 1 --2- 172.21.15.46:0/2981724406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099595220 0x5580995917d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.368+0000 7f2acf7fe700 1 -- 172.21.15.46:0/2981724406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099595220 msgr2=0x5580995917d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.368+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/2981724406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099595220 0x5580995917d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.368+0000 7f2acf7fe700 1 -- 172.21.15.46:0/2981724406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55809958a380 msgr2=0x5580995947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.368+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/2981724406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55809958a380 0x5580995947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.368+0000 7f2acf7fe700 1 -- 172.21.15.46:0/2981724406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580994a2ea0 con 0x558099589a60 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.368+0000 7f2acffff700 1 --2- 172.21.15.46:0/2981724406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099595220 0x5580995917d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.368+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/2981724406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 0x5580995a02e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x5580995abeb0 tx=0x7f2ac0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebb4e5bbb4028f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.368+0000 7f2ab67fc700 1 -- 172.21.15.46:0/2981724406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0026040 con 0x558099589a60 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.368+0000 7f2ab67fc700 1 -- 172.21.15.46:0/2981724406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac00284c0 con 0x558099589a60 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.368+0000 7f2ab67fc700 1 -- 172.21.15.46:0/2981724406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0003be0 con 0x558099589a60 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.369+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2981724406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 msgr2=0x5580995a02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.369+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2981724406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 0x5580995a02e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x5580995abeb0 tx=0x7f2ac0016000 comp rx=0 tx=0).stop 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.369+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2981724406 shutdown_connections 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.369+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2981724406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 0x5580995a02e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.369+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2981724406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558099595220 0x5580995917d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.369+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2981724406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55809958a380 0x5580995947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.369+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2981724406 >> 172.21.15.46:0/2981724406 conn(0x558099500220 msgr2=0x55809958d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.369+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2981724406 shutdown_connections 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.369+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2981724406 wait complete. 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.369+0000 7f2ae1c99b80 1 -- start start 2024-03-18T20:40:20.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.369+0000 7f2ae1c99b80 1 Processor -- start 2024-03-18T20:40:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2ae1c99b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 0x55809959e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2ae1c99b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55809958a380 0x55809959e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2ae1c99b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099595220 0x5580995f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2ae1c99b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580995f64c0 con 0x558099589a60 2024-03-18T20:40:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2ae1c99b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580995f6630 con 0x558099595220 2024-03-18T20:40:20.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2ae1c99b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580995f67f0 con 0x55809958a380 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2ae1c99b80 10 client.0 _ll_register_callbacks cb 0x5580992e43a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2acf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 0x55809959e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2acf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 0x55809959e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34280/0 (socket says 172.21.15.46:34280) 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2acf7fe700 1 -- 172.21.15.46:0/2487525312 learned_addr learned my addr 172.21.15.46:0/2487525312 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2acffff700 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099595220 0x5580995f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2aceffd700 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55809958a380 0x55809959e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2acf7fe700 1 -- 172.21.15.46:0/2487525312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55809958a380 msgr2=0x55809959e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55809958a380 0x55809959e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2acf7fe700 1 -- 172.21.15.46:0/2487525312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099595220 msgr2=0x5580995f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099595220 0x5580995f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2acf7fe700 1 -- 172.21.15.46:0/2487525312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac0010df0 con 0x558099589a60 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2acf7fe700 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 0x55809959e170 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00108e0 tx=0x7f2ac00282c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1106d0876c701a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2aaeffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:20.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2aaeffd700 1 -- 172.21.15.46:0/2487525312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0026070 con 0x558099589a60 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2aaeffd700 1 -- 172.21.15.46:0/2487525312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2ac0017070 con 0x558099589a60 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.370+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2487525312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ac4015070 con 0x558099589a60 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.371+0000 7f2aaeffd700 1 -- 172.21.15.46:0/2487525312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ac0003d30 con 0x558099589a60 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.371+0000 7f2ae1c99b80 10 client.16953 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.371+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2487525312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x558099362cd0 con 0x558099589a60 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.371+0000 7f2aaeffd700 1 -- 172.21.15.46:0/2487525312 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac4015070 con 0x558099589a60 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.371+0000 7f2aaeffd700 1 -- 172.21.15.46:0/2487525312 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2ac00115e0 con 0x558099589a60 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.371+0000 7f2ae1c99b80 10 client.16953 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.371+0000 7f2ae1c99b80 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.371+0000 7f2ae1c99b80 20 client.16953 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.371+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5580995fe060 0x558099600480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.371+0000 7f2ae1c99b80 4 client.16953 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.371+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2487525312 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55809932bcf0 con 0x5580995fe060 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.371+0000 7f2aceffd700 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5580995fe060 0x558099600480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2aaeffd700 10 client.16953 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2aceffd700 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5580995fe060 0x558099600480 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3c92861e7ef509 server_cookie=e3db18ef7b1b7f97 in_seq=0 out_seq=0 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2aaeffd700 1 -- 172.21.15.46:0/2487525312 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55809932bcf0 con 0x5580995fe060 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2aaeffd700 10 client.16953 handle_command_reply: tid=0 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2ae1c99b80 1 client.16953 shutdown 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2487525312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 msgr2=0x55809959e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 0x55809959e170 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00108e0 tx=0x7f2ac00282c0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2487525312 shutdown_connections 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5580995fe060 0x558099600480 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558099589a60 0x55809959e170 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558099595220 0x5580995f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2ae1c99b80 1 --2- 172.21.15.46:0/2487525312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55809958a380 0x55809959e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2487525312 >> 172.21.15.46:0/2487525312 conn(0x558099500220 msgr2=0x5580995917d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2aaeffd700 0 client.16953 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2487525312 shutdown_connections 2024-03-18T20:40:20.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2ae1c99b80 1 -- 172.21.15.46:0/2487525312 wait complete. 2024-03-18T20:40:20.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.372+0000 7f2ae1c99b80 20 client.16953 trim_cache size 0 max 16384 2024-03-18T20:40:20.382 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:40:20.382 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:40:20.382 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:40:20.382 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:40:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.373+0000 7f2ad5b04700 1 -- 172.21.15.46:0/237293955 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ab80a1290 msgr2=0x7f2ab80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.373+0000 7f2ad5b04700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ab80a1290 0x7f2ab80a3750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0007be0 tx=0x7f2ac00073d0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.373+0000 7f2ad5b04700 1 -- 172.21.15.46:0/237293955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ad019bbe0 msgr2=0x7f2ad019c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.373+0000 7f2ad5b04700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ad019bbe0 0x7f2ad019c060 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80062a0 tx=0x7f2ac8009380 comp rx=0 tx=0).stop 2024-03-18T20:40:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.373+0000 7f2ad5b04700 1 -- 172.21.15.46:0/237293955 shutdown_connections 2024-03-18T20:40:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.373+0000 7f2ad5b04700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ab80a1290 0x7f2ab80a3750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.373+0000 7f2ad5b04700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ad019bbe0 0x7f2ad019c060 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.373+0000 7f2ad5b04700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad0100f60 0x7f2ad0197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.373+0000 7f2ad5b04700 1 --2- 172.21.15.46:0/237293955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ad0106080 0x7f2ad0197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.373+0000 7f2ad5b04700 1 -- 172.21.15.46:0/237293955 >> 172.21.15.46:0/237293955 conn(0x7f2ad00fc7c0 msgr2=0x7f2ad00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.373+0000 7f2ad5b04700 1 -- 172.21.15.46:0/237293955 shutdown_connections 2024-03-18T20:40:20.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.373+0000 7f2ad5b04700 1 -- 172.21.15.46:0/237293955 wait complete. 2024-03-18T20:40:20.389 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:40:20.390 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:40:20.416 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:20 smithi046 ceph-mon[43495]: pgmap v130: 97 pgs: 97 active+clean; 47 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 470 KiB/s wr, 90 op/s 2024-03-18T20:40:20.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:20 smithi073 ceph-mon[47384]: pgmap v130: 97 pgs: 97 active+clean; 47 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 470 KiB/s wr, 90 op/s 2024-03-18T20:40:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.524+0000 7f5a18a17700 1 -- 172.21.15.46:0/375008683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a14100500 msgr2=0x7f5a1410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.524+0000 7f5a18a17700 1 --2- 172.21.15.46:0/375008683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a14100500 0x7f5a1410c9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00b0d0 tx=0x7f5a0c00b3e0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.524+0000 7f5a18a17700 1 -- 172.21.15.46:0/375008683 shutdown_connections 2024-03-18T20:40:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.524+0000 7f5a18a17700 1 --2- 172.21.15.46:0/375008683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a14100500 0x7f5a1410c9c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.524+0000 7f5a18a17700 1 --2- 172.21.15.46:0/375008683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a14105c00 0x7f5a14105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.524+0000 7f5a18a17700 1 --2- 172.21.15.46:0/375008683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a140ffb40 0x7f5a140fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.524+0000 7f5a18a17700 1 -- 172.21.15.46:0/375008683 >> 172.21.15.46:0/375008683 conn(0x7f5a140fb5f0 msgr2=0x7f5a140fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.524+0000 7f5a18a17700 1 -- 172.21.15.46:0/375008683 shutdown_connections 2024-03-18T20:40:20.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.524+0000 7f5a18a17700 1 -- 172.21.15.46:0/375008683 wait complete. 2024-03-18T20:40:20.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.525+0000 7f5a18a17700 1 Processor -- start 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.525+0000 7f5a18a17700 1 -- start start 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a18a17700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a140ffb40 0x7f5a14197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a18a17700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a14100500 0x7f5a141984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a18a17700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a14105c00 0x7f5a141a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a18a17700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14198d30 con 0x7f5a14105c00 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a18a17700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a141a3560 con 0x7f5a140ffb40 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a18a17700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a141a36a0 con 0x7f5a14100500 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a12d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a14105c00 0x7f5a141a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a12d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a14105c00 0x7f5a141a3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34294/0 (socket says 172.21.15.46:34294) 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a12d9d700 1 -- 172.21.15.46:0/403934935 learned_addr learned my addr 172.21.15.46:0/403934935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a1259c700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a140ffb40 0x7f5a14197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a11d9b700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a14100500 0x7f5a141984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a12d9d700 1 -- 172.21.15.46:0/403934935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a14100500 msgr2=0x7f5a141984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a12d9d700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a14100500 0x7f5a141984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a12d9d700 1 -- 172.21.15.46:0/403934935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a140ffb40 msgr2=0x7f5a14197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a12d9d700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a140ffb40 0x7f5a14197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a12d9d700 1 -- 172.21.15.46:0/403934935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c009d00 con 0x7f5a14105c00 2024-03-18T20:40:20.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a11d9b700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a14100500 0x7f5a141984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:20.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a12d9d700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a14105c00 0x7f5a141a3020 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0094f0 tx=0x7f5a0c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.526+0000 7f5a037fe700 1 -- 172.21.15.46:0/403934935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c00e040 con 0x7f5a14105c00 2024-03-18T20:40:20.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.527+0000 7f5a037fe700 1 -- 172.21.15.46:0/403934935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a0c01d940 con 0x7f5a14105c00 2024-03-18T20:40:20.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.527+0000 7f5a18a17700 1 -- 172.21.15.46:0/403934935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a141a38a0 con 0x7f5a14105c00 2024-03-18T20:40:20.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.527+0000 7f5a037fe700 1 -- 172.21.15.46:0/403934935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c01b9f0 con 0x7f5a14105c00 2024-03-18T20:40:20.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.527+0000 7f5a18a17700 1 -- 172.21.15.46:0/403934935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a141a3d20 con 0x7f5a14105c00 2024-03-18T20:40:20.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.528+0000 7f5a017fa700 1 -- 172.21.15.46:0/403934935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f80052f0 con 0x7f5a14105c00 2024-03-18T20:40:20.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.529+0000 7f5a037fe700 1 -- 172.21.15.46:0/403934935 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5a0c01bb50 con 0x7f5a14105c00 2024-03-18T20:40:20.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.529+0000 7f5a037fe700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f59fc0a1290 0x7f59fc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.530+0000 7f5a037fe700 1 -- 172.21.15.46:0/403934935 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5a0c0ced00 con 0x7f5a14105c00 2024-03-18T20:40:20.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.530+0000 7f5a1259c700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f59fc0a1290 0x7f59fc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.530+0000 7f5a1259c700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f59fc0a1290 0x7f59fc0a3750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5a04005950 tx=0x7f5a0400b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.533+0000 7f5a037fe700 1 -- 172.21.15.46:0/403934935 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5a0c0981b0 con 0x7f5a14105c00 2024-03-18T20:40:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:20 smithi155 ceph-mon[48344]: pgmap v130: 97 pgs: 97 active+clean; 47 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 470 KiB/s wr, 90 op/s 2024-03-18T20:40:20.684 INFO:tasks.workunit.client.0.smithi046.stdout: 1 107173 1.79 MB/sec execute 75 sec latency 1851.461 ms 2024-03-18T20:40:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.802+0000 7f5a017fa700 1 -- 172.21.15.46:0/403934935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59f8005160 con 0x7f5a14105c00 2024-03-18T20:40:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.803+0000 7f5a037fe700 1 -- 172.21.15.46:0/403934935 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f5a0c09b7d0 con 0x7f5a14105c00 2024-03-18T20:40:20.806 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:40:20.806 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:39:53.094005+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:40:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.805+0000 7f5a017fa700 1 -- 172.21.15.46:0/403934935 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f59fc0a1290 msgr2=0x7f59fc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.805+0000 7f5a017fa700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f59fc0a1290 0x7f59fc0a3750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5a04005950 tx=0x7f5a0400b410 comp rx=0 tx=0).stop 2024-03-18T20:40:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.805+0000 7f5a017fa700 1 -- 172.21.15.46:0/403934935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a14105c00 msgr2=0x7f5a141a3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.805+0000 7f5a017fa700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a14105c00 0x7f5a141a3020 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0094f0 tx=0x7f5a0c009520 comp rx=0 tx=0).stop 2024-03-18T20:40:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.805+0000 7f5a017fa700 1 -- 172.21.15.46:0/403934935 shutdown_connections 2024-03-18T20:40:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.805+0000 7f5a017fa700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f59fc0a1290 0x7f59fc0a3750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.805+0000 7f5a017fa700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a14105c00 0x7f5a141a3020 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.805+0000 7f5a017fa700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a140ffb40 0x7f5a14197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.805+0000 7f5a017fa700 1 --2- 172.21.15.46:0/403934935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a14100500 0x7f5a141984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.805+0000 7f5a017fa700 1 -- 172.21.15.46:0/403934935 >> 172.21.15.46:0/403934935 conn(0x7f5a140fb5f0 msgr2=0x7f5a140fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.805+0000 7f5a017fa700 1 -- 172.21.15.46:0/403934935 shutdown_connections 2024-03-18T20:40:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.806+0000 7f5a017fa700 1 -- 172.21.15.46:0/403934935 wait complete. 2024-03-18T20:40:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 39 2024-03-18T20:40:20.822 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:40:20.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:40:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.841+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/544929430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c54106030 msgr2=0x7f6c5410dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.841+0000 7f6c5c3e6700 1 --2- 172.21.15.46:0/544929430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c54106030 0x7f6c5410dbb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00b0a0 tx=0x7f6c4c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.841+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/544929430 shutdown_connections 2024-03-18T20:40:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.841+0000 7f6c5c3e6700 1 --2- 172.21.15.46:0/544929430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c54106030 0x7f6c5410dbb0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.841+0000 7f6c5c3e6700 1 --2- 172.21.15.46:0/544929430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c54100f10 0x7f6c541012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.841+0000 7f6c5c3e6700 1 --2- 172.21.15.46:0/544929430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c541018c0 0x7f6c54105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.841+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/544929430 >> 172.21.15.46:0/544929430 conn(0x7f6c540fc7b0 msgr2=0x7f6c540febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.841+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/544929430 shutdown_connections 2024-03-18T20:40:20.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.841+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/544929430 wait complete. 2024-03-18T20:40:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.842+0000 7f6c5c3e6700 1 Processor -- start 2024-03-18T20:40:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.842+0000 7f6c5c3e6700 1 -- start start 2024-03-18T20:40:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.842+0000 7f6c5c3e6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c54100f10 0x7f6c54197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.842+0000 7f6c5c3e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c54106030 0x7f6c54197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c5c3e6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c5419bb20 0x7f6c5419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c5c3e6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419c5a0 con 0x7f6c54106030 2024-03-18T20:40:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c5c3e6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419c710 con 0x7f6c5419bb20 2024-03-18T20:40:20.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c5c3e6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5419c8d0 con 0x7f6c54100f10 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c59981700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c54106030 0x7f6c54197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c59981700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c54106030 0x7f6c54197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34310/0 (socket says 172.21.15.46:34310) 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c59981700 1 -- 172.21.15.46:0/2613589094 learned_addr learned my addr 172.21.15.46:0/2613589094 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c5a182700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c54100f10 0x7f6c54197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c5a983700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c5419bb20 0x7f6c5419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c59981700 1 -- 172.21.15.46:0/2613589094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c54100f10 msgr2=0x7f6c54197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c59981700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c54100f10 0x7f6c54197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c59981700 1 -- 172.21.15.46:0/2613589094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c5419bb20 msgr2=0x7f6c5419bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c59981700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c5419bb20 0x7f6c5419bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c59981700 1 -- 172.21.15.46:0/2613589094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c009d00 con 0x7f6c54106030 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c59981700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c54106030 0x7f6c54197780 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000d980 tx=0x7f6c5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c477fe700 1 -- 172.21.15.46:0/2613589094 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50009930 con 0x7f6c54106030 2024-03-18T20:40:20.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/2613589094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c541a3e30 con 0x7f6c54106030 2024-03-18T20:40:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c477fe700 1 -- 172.21.15.46:0/2613589094 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c50010460 con 0x7f6c54106030 2024-03-18T20:40:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.843+0000 7f6c477fe700 1 -- 172.21.15.46:0/2613589094 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000f620 con 0x7f6c54106030 2024-03-18T20:40:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.844+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/2613589094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c541a42b0 con 0x7f6c54106030 2024-03-18T20:40:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.846+0000 7f6c477fe700 1 -- 172.21.15.46:0/2613589094 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6c500108d0 con 0x7f6c54106030 2024-03-18T20:40:20.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.846+0000 7f6c477fe700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6c400a1290 0x7f6c400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.846+0000 7f6c477fe700 1 -- 172.21.15.46:0/2613589094 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6c500cc6d0 con 0x7f6c54106030 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.847+0000 7f6c5a182700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6c400a1290 0x7f6c400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.847+0000 7f6c5a182700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6c400a1290 0x7f6c400a3750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009c60 tx=0x7f6c48009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.848+0000 7f6c6857bb80 1 Processor -- start 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.848+0000 7f6c6857bb80 1 -- start start 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.848+0000 7f6c6857bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45570 0x55a44bf45950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.848+0000 7f6c6857bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 0x55a44bf50870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.848+0000 7f6c6857bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf50db0 0x55a44bf531a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.848+0000 7f6c6857bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44be9f930 con 0x55a44bf45e90 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.848+0000 7f6c6857bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44be9d2f0 con 0x55a44bf50db0 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.848+0000 7f6c6857bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44be98be0 con 0x55a44bf45570 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.848+0000 7f6c59981700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 0x55a44bf50870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c59981700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 0x55a44bf50870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34318/0 (socket says 172.21.15.46:34318) 2024-03-18T20:40:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c59981700 1 -- 172.21.15.46:0/3433932919 learned_addr learned my addr 172.21.15.46:0/3433932919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c5a182700 1 --2- 172.21.15.46:0/3433932919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45570 0x55a44bf45950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c5a983700 1 --2- 172.21.15.46:0/3433932919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf50db0 0x55a44bf531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c59981700 1 -- 172.21.15.46:0/3433932919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45570 msgr2=0x55a44bf45950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c59981700 1 --2- 172.21.15.46:0/3433932919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45570 0x55a44bf45950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c59981700 1 -- 172.21.15.46:0/3433932919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf50db0 msgr2=0x55a44bf531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c59981700 1 --2- 172.21.15.46:0/3433932919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf50db0 0x55a44bf531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c59981700 1 -- 172.21.15.46:0/3433932919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a44bd1a350 con 0x55a44bf45e90 2024-03-18T20:40:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c5a182700 1 --2- 172.21.15.46:0/3433932919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45570 0x55a44bf45950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c59981700 1 --2- 172.21.15.46:0/3433932919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 0x55a44bf50870 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f6c500a1740 tx=0x7f6c50099460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2498d37ccae3d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c44da8700 1 -- 172.21.15.46:0/3433932919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c500a1c70 con 0x55a44bf45e90 2024-03-18T20:40:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c44da8700 1 -- 172.21.15.46:0/3433932919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c50095f00 con 0x55a44bf45e90 2024-03-18T20:40:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.849+0000 7f6c44da8700 1 -- 172.21.15.46:0/3433932919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50099cb0 con 0x55a44bf45e90 2024-03-18T20:40:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.850+0000 7f6c6857bb80 1 -- 172.21.15.46:0/3433932919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 msgr2=0x55a44bf50870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.850+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/3433932919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 0x55a44bf50870 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f6c500a1740 tx=0x7f6c50099460 comp rx=0 tx=0).stop 2024-03-18T20:40:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.850+0000 7f6c6857bb80 1 -- 172.21.15.46:0/3433932919 shutdown_connections 2024-03-18T20:40:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.850+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/3433932919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf50db0 0x55a44bf531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.850+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/3433932919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45570 0x55a44bf45950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.850+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/3433932919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 0x55a44bf50870 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.850+0000 7f6c6857bb80 1 -- 172.21.15.46:0/3433932919 >> 172.21.15.46:0/3433932919 conn(0x55a44bebc1c0 msgr2=0x55a44bf41a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.850+0000 7f6c6857bb80 1 -- 172.21.15.46:0/3433932919 shutdown_connections 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.850+0000 7f6c6857bb80 1 -- 172.21.15.46:0/3433932919 wait complete. 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c6857bb80 1 -- start start 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c6857bb80 1 Processor -- start 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c6857bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45570 0x55a44bf4dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c6857bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 0x55a44bf4e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c6857bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf50db0 0x55a44bf4e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c6857bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44be34d20 con 0x55a44bf45e90 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c6857bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44be349c0 con 0x55a44bf50db0 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c6857bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44bd1c640 con 0x55a44bf45570 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c6857bb80 10 client.0 _ll_register_callbacks cb 0x55a44bca0340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c59981700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 0x55a44bf4e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c59981700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 0x55a44bf4e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34328/0 (socket says 172.21.15.46:34328) 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c59981700 1 -- 172.21.15.46:0/1218569803 learned_addr learned my addr 172.21.15.46:0/1218569803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c5a182700 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45570 0x55a44bf4dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.851+0000 7f6c5a983700 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf50db0 0x55a44bf4e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c59981700 1 -- 172.21.15.46:0/1218569803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45570 msgr2=0x55a44bf4dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c59981700 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45570 0x55a44bf4dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c59981700 1 -- 172.21.15.46:0/1218569803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf50db0 msgr2=0x55a44bf4e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c59981700 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf50db0 0x55a44bf4e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c59981700 1 -- 172.21.15.46:0/1218569803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c48017070 con 0x55a44bf45e90 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c59981700 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 0x55a44bf4e160 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6c50099780 tx=0x7f6c500b9310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cded1961dd4f30 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c3dffb700 1 -- 172.21.15.46:0/1218569803 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c500a1c70 con 0x55a44bf45e90 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c3dffb700 1 -- 172.21.15.46:0/1218569803 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6c50098ef0 con 0x55a44bf45e90 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c6857bb80 1 -- 172.21.15.46:0/1218569803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a44bf396c0 con 0x55a44bf45e90 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c3dffb700 1 -- 172.21.15.46:0/1218569803 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c50099a50 con 0x55a44bf45e90 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c6857bb80 10 client.16971 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c6857bb80 1 -- 172.21.15.46:0/1218569803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f6c541a42b0 con 0x55a44bf45e90 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.852+0000 7f6c3dffb700 1 -- 172.21.15.46:0/1218569803 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c50063080 con 0x55a44bf45e90 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.853+0000 7f6c3dffb700 1 -- 172.21.15.46:0/1218569803 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6c50064f80 con 0x55a44bf45e90 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.853+0000 7f6c6857bb80 10 client.16971 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.853+0000 7f6c6857bb80 10 client.16971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.853+0000 7f6c6857bb80 20 client.16971 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.853+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a44bf60530 0x55a44bf62950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.853+0000 7f6c6857bb80 4 client.16971 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.853+0000 7f6c6857bb80 1 -- 172.21.15.46:0/1218569803 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a44be916a0 con 0x55a44bf60530 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.853+0000 7f6c5a182700 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a44bf60530 0x55a44bf62950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.853+0000 7f6c3dffb700 10 client.16971 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c5a182700 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a44bf60530 0x55a44bf62950 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf2e5a5f1ba82177 server_cookie=fccb0d4cfa8414b2 in_seq=0 out_seq=0 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c3dffb700 1 -- 172.21.15.46:0/1218569803 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a44be916a0 con 0x55a44bf60530 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c3dffb700 10 client.16971 handle_command_reply: tid=0 2024-03-18T20:40:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c6857bb80 1 client.16971 shutdown 2024-03-18T20:40:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c6857bb80 1 -- 172.21.15.46:0/1218569803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 msgr2=0x55a44bf4e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 0x55a44bf4e160 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6c50099780 tx=0x7f6c500b9310 comp rx=0 tx=0).stop 2024-03-18T20:40:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c6857bb80 1 -- 172.21.15.46:0/1218569803 shutdown_connections 2024-03-18T20:40:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a44bf60530 0x55a44bf62950 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf50db0 0x55a44bf4e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45570 0x55a44bf4dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c3dffb700 0 client.16971 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/1218569803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45e90 0x55a44bf4e160 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c6857bb80 1 -- 172.21.15.46:0/1218569803 >> 172.21.15.46:0/1218569803 conn(0x55a44bebc1c0 msgr2=0x55a44bf41650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.854+0000 7f6c6857bb80 1 -- 172.21.15.46:0/1218569803 shutdown_connections 2024-03-18T20:40:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.855+0000 7f6c6857bb80 1 -- 172.21.15.46:0/1218569803 wait complete. 2024-03-18T20:40:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.855+0000 7f6c6857bb80 20 client.16971 trim_cache size 0 max 16384 2024-03-18T20:40:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.871+0000 7f6c6857bb80 1 Processor -- start 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.871+0000 7f6c6857bb80 1 -- start start 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.871+0000 7f6c6857bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45570 0x55a44bf4cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.871+0000 7f6c6857bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf45e90 0x55a44bf57980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.871+0000 7f6c6857bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf50db0 0x55a44bf4e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.871+0000 7f6c6857bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44bf3e1d0 con 0x55a44bf45570 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.871+0000 7f6c6857bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44bf48a60 con 0x55a44bf45e90 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.871+0000 7f6c6857bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44bf4d2c0 con 0x55a44bf50db0 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.872+0000 7f6c5a182700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45570 0x55a44bf4cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.872+0000 7f6c5a182700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45570 0x55a44bf4cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34332/0 (socket says 172.21.15.46:34332) 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.872+0000 7f6c5a182700 1 -- 172.21.15.46:0/743323987 learned_addr learned my addr 172.21.15.46:0/743323987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.872+0000 7f6c59981700 1 --2- 172.21.15.46:0/743323987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf45e90 0x55a44bf57980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.872+0000 7f6c5a983700 1 --2- 172.21.15.46:0/743323987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf50db0 0x55a44bf4e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.872+0000 7f6c5a182700 1 -- 172.21.15.46:0/743323987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf50db0 msgr2=0x55a44bf4e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.872+0000 7f6c5a182700 1 --2- 172.21.15.46:0/743323987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf50db0 0x55a44bf4e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.872+0000 7f6c5a182700 1 -- 172.21.15.46:0/743323987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf45e90 msgr2=0x55a44bf57980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.872+0000 7f6c5a182700 1 --2- 172.21.15.46:0/743323987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf45e90 0x55a44bf57980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.872+0000 7f6c5a182700 1 -- 172.21.15.46:0/743323987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a44bcc6b00 con 0x55a44bf45570 2024-03-18T20:40:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.872+0000 7f6c5a182700 1 --2- 172.21.15.46:0/743323987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45570 0x55a44bf4cd80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x55a44bf63220 tx=0x7f6c48014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f145110715b14aa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.872+0000 7f6c455a9700 1 -- 172.21.15.46:0/743323987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48022030 con 0x55a44bf45570 2024-03-18T20:40:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.873+0000 7f6c455a9700 1 -- 172.21.15.46:0/743323987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c48023040 con 0x55a44bf45570 2024-03-18T20:40:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.873+0000 7f6c455a9700 1 -- 172.21.15.46:0/743323987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4801d670 con 0x55a44bf45570 2024-03-18T20:40:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.873+0000 7f6c6857bb80 1 -- 172.21.15.46:0/743323987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45570 msgr2=0x55a44bf4cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.873+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/743323987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45570 0x55a44bf4cd80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x55a44bf63220 tx=0x7f6c48014000 comp rx=0 tx=0).stop 2024-03-18T20:40:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.873+0000 7f6c6857bb80 1 -- 172.21.15.46:0/743323987 shutdown_connections 2024-03-18T20:40:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.873+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/743323987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf45570 0x55a44bf4cd80 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.873+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/743323987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf50db0 0x55a44bf4e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.873+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/743323987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf45e90 0x55a44bf57980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.873+0000 7f6c6857bb80 1 -- 172.21.15.46:0/743323987 >> 172.21.15.46:0/743323987 conn(0x55a44bebc1c0 msgr2=0x55a44bf47640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.873+0000 7f6c6857bb80 1 -- 172.21.15.46:0/743323987 shutdown_connections 2024-03-18T20:40:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.874+0000 7f6c6857bb80 1 -- 172.21.15.46:0/743323987 wait complete. 2024-03-18T20:40:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.874+0000 7f6c6857bb80 1 -- start start 2024-03-18T20:40:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.874+0000 7f6c6857bb80 1 Processor -- start 2024-03-18T20:40:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.874+0000 7f6c6857bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf45570 0x55a44bf575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.874+0000 7f6c6857bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45e90 0x55a44bf55960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.874+0000 7f6c6857bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf50db0 0x55a44bf55ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.874+0000 7f6c6857bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44bf563e0 con 0x55a44bf50db0 2024-03-18T20:40:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.874+0000 7f6c6857bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44bf56550 con 0x55a44bf45570 2024-03-18T20:40:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.874+0000 7f6c6857bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44bf56710 con 0x55a44bf45e90 2024-03-18T20:40:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.874+0000 7f6c6857bb80 10 client.0 _ll_register_callbacks cb 0x7f6c4c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.874+0000 7f6c5a983700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf50db0 0x55a44bf55ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.874+0000 7f6c5a983700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf50db0 0x55a44bf55ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34340/0 (socket says 172.21.15.46:34340) 2024-03-18T20:40:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c5a983700 1 -- 172.21.15.46:0/4063004265 learned_addr learned my addr 172.21.15.46:0/4063004265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c59981700 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45e90 0x55a44bf55960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c5a182700 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf45570 0x55a44bf575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c5a983700 1 -- 172.21.15.46:0/4063004265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45e90 msgr2=0x55a44bf55960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c5a983700 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45e90 0x55a44bf55960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c5a983700 1 -- 172.21.15.46:0/4063004265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf45570 msgr2=0x55a44bf575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c5a983700 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf45570 0x55a44bf575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c5a983700 1 -- 172.21.15.46:0/4063004265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c48004400 con 0x55a44bf50db0 2024-03-18T20:40:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c59981700 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45e90 0x55a44bf55960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c5a983700 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf50db0 0x55a44bf55ea0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00b0a0 tx=0x7f6c4c007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e70ae6695d19010 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c3dffb700 1 -- 172.21.15.46:0/4063004265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c021040 con 0x55a44bf50db0 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c3dffb700 1 -- 172.21.15.46:0/4063004265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6c4c00bc50 con 0x55a44bf50db0 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.875+0000 7f6c6857bb80 1 -- 172.21.15.46:0/4063004265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c50095f00 con 0x55a44bf50db0 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.876+0000 7f6c3dffb700 1 -- 172.21.15.46:0/4063004265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c4c00e040 con 0x55a44bf50db0 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.876+0000 7f6c6857bb80 10 client.16977 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.876+0000 7f6c6857bb80 1 -- 172.21.15.46:0/4063004265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a44be5ee40 con 0x55a44bf50db0 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.876+0000 7f6c3dffb700 1 -- 172.21.15.46:0/4063004265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c4c034020 con 0x55a44bf50db0 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.876+0000 7f6c3dffb700 1 -- 172.21.15.46:0/4063004265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6c4c00b0d0 con 0x55a44bf50db0 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.876+0000 7f6c6857bb80 10 client.16977 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.876+0000 7f6c6857bb80 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.876+0000 7f6c6857bb80 20 client.16977 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.876+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a44bfb9d20 0x55a44bfbc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.876+0000 7f6c6857bb80 4 client.16977 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:40:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.876+0000 7f6c6857bb80 1 -- 172.21.15.46:0/4063004265 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a44bce7c90 con 0x55a44bfb9d20 2024-03-18T20:40:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.876+0000 7f6c5a182700 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a44bfb9d20 0x55a44bfbc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.877+0000 7f6c5a182700 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a44bfb9d20 0x55a44bfbc140 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6718fabf958a2bda server_cookie=ea0e4290f4333322 in_seq=0 out_seq=0 2024-03-18T20:40:20.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.877+0000 7f6c3dffb700 10 client.16977 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c3dffb700 1 -- 172.21.15.46:0/4063004265 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a44bce7c90 con 0x55a44bfb9d20 2024-03-18T20:40:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c3dffb700 10 client.16977 handle_command_reply: tid=0 2024-03-18T20:40:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c6857bb80 1 client.16977 shutdown 2024-03-18T20:40:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c6857bb80 1 -- 172.21.15.46:0/4063004265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf50db0 msgr2=0x55a44bf55ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf50db0 0x55a44bf55ea0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00b0a0 tx=0x7f6c4c007710 comp rx=0 tx=0).stop 2024-03-18T20:40:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c6857bb80 1 -- 172.21.15.46:0/4063004265 shutdown_connections 2024-03-18T20:40:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a44bfb9d20 0x55a44bfbc140 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a44bf50db0 0x55a44bf55ea0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a44bf45570 0x55a44bf575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c6857bb80 1 --2- 172.21.15.46:0/4063004265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a44bf45e90 0x55a44bf55960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c6857bb80 1 -- 172.21.15.46:0/4063004265 >> 172.21.15.46:0/4063004265 conn(0x55a44bebc1c0 msgr2=0x55a44bf47ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c3dffb700 0 client.16977 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c6857bb80 1 -- 172.21.15.46:0/4063004265 shutdown_connections 2024-03-18T20:40:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c6857bb80 1 -- 172.21.15.46:0/4063004265 wait complete. 2024-03-18T20:40:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.912+0000 7f6c6857bb80 20 client.16977 trim_cache size 0 max 16384 2024-03-18T20:40:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:40:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:40:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:40:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:40:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.913+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/2613589094 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6c400a1290 msgr2=0x7f6c400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.913+0000 7f6c5c3e6700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6c400a1290 0x7f6c400a3750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009c60 tx=0x7f6c48009260 comp rx=0 tx=0).stop 2024-03-18T20:40:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.913+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/2613589094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c54106030 msgr2=0x7f6c54197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.913+0000 7f6c5c3e6700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c54106030 0x7f6c54197780 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000d980 tx=0x7f6c5000dc90 comp rx=0 tx=0).stop 2024-03-18T20:40:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.913+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/2613589094 shutdown_connections 2024-03-18T20:40:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.913+0000 7f6c5c3e6700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6c400a1290 0x7f6c400a3750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.913+0000 7f6c5c3e6700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c5419bb20 0x7f6c5419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.913+0000 7f6c5c3e6700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c54100f10 0x7f6c54197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.913+0000 7f6c5c3e6700 1 --2- 172.21.15.46:0/2613589094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c54106030 0x7f6c54197780 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.913+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/2613589094 >> 172.21.15.46:0/2613589094 conn(0x7f6c540fc7b0 msgr2=0x7f6c540feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.913+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/2613589094 shutdown_connections 2024-03-18T20:40:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:20.913+0000 7f6c5c3e6700 1 -- 172.21.15.46:0/2613589094 wait complete. 2024-03-18T20:40:20.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.990+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/939342876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f14106140 msgr2=0x7f7f1410dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.990+0000 7f7f1a0c0700 1 --2- 172.21.15.46:0/939342876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f14106140 0x7f7f1410dcc0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00b0a0 tx=0x7f7f0c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.990+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/939342876 shutdown_connections 2024-03-18T20:40:20.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.990+0000 7f7f1a0c0700 1 --2- 172.21.15.46:0/939342876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f14106140 0x7f7f1410dcc0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.990+0000 7f7f1a0c0700 1 --2- 172.21.15.46:0/939342876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f140ff7b0 0x7f7f140ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.990+0000 7f7f1a0c0700 1 --2- 172.21.15.46:0/939342876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f14100160 0x7f7f14105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.990+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/939342876 >> 172.21.15.46:0/939342876 conn(0x7f7f14074e90 msgr2=0x7f7f140752a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:20.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.990+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/939342876 shutdown_connections 2024-03-18T20:40:20.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.990+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/939342876 wait complete. 2024-03-18T20:40:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.991+0000 7f7f1a0c0700 1 Processor -- start 2024-03-18T20:40:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.991+0000 7f7f1a0c0700 1 -- start start 2024-03-18T20:40:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f1a0c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140ff7b0 0x7f7f14100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f1a0c0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f14106140 0x7f7f14100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f1a0c0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f141011a0 0x7f7f141a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f1a0c0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f141a5a50 con 0x7f7f140ff7b0 2024-03-18T20:40:20.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f1a0c0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f141a5bc0 con 0x7f7f141011a0 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f1a0c0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f141a5d80 con 0x7f7f14106140 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140ff7b0 0x7f7f14100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140ff7b0 0x7f7f14100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34358/0 (socket says 172.21.15.46:34358) 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f137fe700 1 -- 172.21.15.46:0/343448362 learned_addr learned my addr 172.21.15.46:0/343448362 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f12ffd700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f14106140 0x7f7f14100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f13fff700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f141011a0 0x7f7f141a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f137fe700 1 -- 172.21.15.46:0/343448362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f14106140 msgr2=0x7f7f14100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f137fe700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f14106140 0x7f7f14100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f137fe700 1 -- 172.21.15.46:0/343448362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f141011a0 msgr2=0x7f7f141a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f137fe700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f141011a0 0x7f7f141a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f137fe700 1 -- 172.21.15.46:0/343448362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f04009520 con 0x7f7f140ff7b0 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.992+0000 7f7f13fff700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f141011a0 0x7f7f141a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.993+0000 7f7f137fe700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140ff7b0 0x7f7f14100720 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400eb30 tx=0x7f7f0400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:20.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.993+0000 7f7f10ff9700 1 -- 172.21.15.46:0/343448362 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04009880 con 0x7f7f140ff7b0 2024-03-18T20:40:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.993+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/343448362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0c009d00 con 0x7f7f140ff7b0 2024-03-18T20:40:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.993+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/343448362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f141a6430 con 0x7f7f140ff7b0 2024-03-18T20:40:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.993+0000 7f7f10ff9700 1 -- 172.21.15.46:0/343448362 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f040044e0 con 0x7f7f140ff7b0 2024-03-18T20:40:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.993+0000 7f7f10ff9700 1 -- 172.21.15.46:0/343448362 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04003ce0 con 0x7f7f140ff7b0 2024-03-18T20:40:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.996+0000 7f7f10ff9700 1 -- 172.21.15.46:0/343448362 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7f04018430 con 0x7f7f140ff7b0 2024-03-18T20:40:21.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.997+0000 7f7f10ff9700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7efc0a1300 0x7f7efc0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.997+0000 7f7f12ffd700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7efc0a1300 0x7f7efc0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.997+0000 7f7f10ff9700 1 -- 172.21.15.46:0/343448362 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f7f04014070 con 0x7f7f140ff7b0 2024-03-18T20:40:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f26255b80 1 Processor -- start 2024-03-18T20:40:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f26255b80 1 -- start start 2024-03-18T20:40:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f12ffd700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7efc0a1300 0x7f7efc0a37c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7f14101e50 tx=0x7f7f08009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f26255b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c821e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f26255b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 0x55707c83dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f26255b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c83e230 0x55707c840620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f26255b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707c77b990 con 0x55707c822380 2024-03-18T20:40:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f26255b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707c779350 con 0x55707c83e230 2024-03-18T20:40:21.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f26255b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707c774c40 con 0x55707c821a60 2024-03-18T20:40:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 0x55707c83dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 0x55707c83dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34362/0 (socket says 172.21.15.46:34362) 2024-03-18T20:40:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f12ffd700 1 -- 172.21.15.46:0/2648834586 learned_addr learned my addr 172.21.15.46:0/2648834586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.998+0000 7f7f13fff700 1 --2- 172.21.15.46:0/2648834586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c83e230 0x55707c840620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.999+0000 7f7f137fe700 1 --2- 172.21.15.46:0/2648834586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c821e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.999+0000 7f7f13fff700 1 -- 172.21.15.46:0/2648834586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 msgr2=0x55707c821e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.999+0000 7f7f13fff700 1 --2- 172.21.15.46:0/2648834586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c821e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.999+0000 7f7f13fff700 1 -- 172.21.15.46:0/2648834586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 msgr2=0x55707c83dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.999+0000 7f7f13fff700 1 --2- 172.21.15.46:0/2648834586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 0x55707c83dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:20.999+0000 7f7f13fff700 1 -- 172.21.15.46:0/2648834586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55707c5f63b0 con 0x55707c83e230 2024-03-18T20:40:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.000+0000 7f7f13fff700 1 --2- 172.21.15.46:0/2648834586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c83e230 0x55707c840620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00b780 tx=0x7f7f0c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0857c53d538b522 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.000+0000 7f7ef9ffb700 1 -- 172.21.15.46:0/2648834586 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00e040 con 0x55707c83e230 2024-03-18T20:40:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.000+0000 7f7ef9ffb700 1 -- 172.21.15.46:0/2648834586 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f0c006080 con 0x55707c83e230 2024-03-18T20:40:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.000+0000 7f7ef9ffb700 1 -- 172.21.15.46:0/2648834586 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c003c60 con 0x55707c83e230 2024-03-18T20:40:21.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.002+0000 7f7f26255b80 1 -- 172.21.15.46:0/2648834586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c83e230 msgr2=0x55707c840620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.002+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2648834586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c83e230 0x55707c840620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00b780 tx=0x7f7f0c009b80 comp rx=0 tx=0).stop 2024-03-18T20:40:21.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.002+0000 7f7f26255b80 1 -- 172.21.15.46:0/2648834586 shutdown_connections 2024-03-18T20:40:21.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.002+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2648834586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c83e230 0x55707c840620 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.002+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2648834586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c821e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.002+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2648834586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 0x55707c83dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.002+0000 7f7f26255b80 1 -- 172.21.15.46:0/2648834586 >> 172.21.15.46:0/2648834586 conn(0x55707c798220 msgr2=0x55707c81e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.002+0000 7f7f26255b80 1 -- 172.21.15.46:0/2648834586 shutdown_connections 2024-03-18T20:40:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.002+0000 7f7f26255b80 1 -- 172.21.15.46:0/2648834586 wait complete. 2024-03-18T20:40:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.002+0000 7f7f26255b80 1 -- start start 2024-03-18T20:40:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.003+0000 7f7f26255b80 1 Processor -- start 2024-03-18T20:40:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.003+0000 7f7f26255b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c8314b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.003+0000 7f7f26255b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 0x55707c8319f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.003+0000 7f7f26255b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 0x55707c82c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.003+0000 7f7f26255b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707c710d80 con 0x55707c83e230 2024-03-18T20:40:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.003+0000 7f7f26255b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707c710a20 con 0x55707c822380 2024-03-18T20:40:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.003+0000 7f7f26255b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707c5f86a0 con 0x55707c821a60 2024-03-18T20:40:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.003+0000 7f7f26255b80 10 client.0 _ll_register_callbacks cb 0x55707c57c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.003+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 0x55707c8319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.003+0000 7f7f137fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c8314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 0x55707c8319f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:42054/0 (socket says 172.21.15.46:42054) 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7f12ffd700 1 -- 172.21.15.46:0/3235485193 learned_addr learned my addr 172.21.15.46:0/3235485193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7f13fff700 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 0x55707c82c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7f137fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c8314b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58766/0 (socket says 172.21.15.46:58766) 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7f12ffd700 1 -- 172.21.15.46:0/3235485193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 msgr2=0x55707c8314b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7f12ffd700 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c8314b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7f12ffd700 1 -- 172.21.15.46:0/3235485193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 msgr2=0x55707c82c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7f12ffd700 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 0x55707c82c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7f12ffd700 1 -- 172.21.15.46:0/3235485193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1410e4b0 con 0x55707c822380 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7f13fff700 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 0x55707c82c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7f12ffd700 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 0x55707c8319f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7f08011fd0 tx=0x7f7f080047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82a592547dbfdb7b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7ef2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7ef2ffd700 1 -- 172.21.15.46:0/3235485193 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08044070 con 0x55707c822380 2024-03-18T20:40:21.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7ef2ffd700 1 -- 172.21.15.46:0/3235485193 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f7f08003d70 con 0x55707c822380 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.004+0000 7f7f26255b80 1 -- 172.21.15.46:0/3235485193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55707c815ad0 con 0x55707c822380 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.005+0000 7f7ef2ffd700 1 -- 172.21.15.46:0/3235485193 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f08049420 con 0x55707c822380 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.005+0000 7f7f26255b80 10 client.26708 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.005+0000 7f7f26255b80 1 -- 172.21.15.46:0/3235485193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55707c5a2b60 con 0x55707c822380 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.005+0000 7f7ef2ffd700 1 -- 172.21.15.46:0/3235485193 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f08055080 con 0x55707c822380 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.006+0000 7f7ef2ffd700 1 -- 172.21.15.46:0/3235485193 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7f0803f980 con 0x55707c822380 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.006+0000 7f7f26255b80 10 client.26708 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.006+0000 7f7f26255b80 10 client.26708 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.006+0000 7f7f26255b80 20 client.26708 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.006+0000 7f7f26255b80 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55707c83cbb0 0x55707c83cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.006+0000 7f7f26255b80 4 client.26708 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.006+0000 7f7f26255b80 1 -- 172.21.15.46:0/3235485193 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55707c76d700 con 0x55707c83cbb0 2024-03-18T20:40:21.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.006+0000 7f7f137fe700 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55707c83cbb0 0x55707c83cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.007+0000 7f7ef2ffd700 10 client.26708 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:40:21.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.007+0000 7f7f137fe700 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55707c83cbb0 0x55707c83cfc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2251e0f4d1c8caad server_cookie=41f0d41c4c28ee54 in_seq=0 out_seq=0 2024-03-18T20:40:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.007+0000 7f7ef2ffd700 1 -- 172.21.15.46:0/3235485193 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55707c76d700 con 0x55707c83cbb0 2024-03-18T20:40:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.007+0000 7f7ef2ffd700 10 client.26708 handle_command_reply: tid=0 2024-03-18T20:40:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.007+0000 7f7f26255b80 1 client.26708 shutdown 2024-03-18T20:40:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.007+0000 7f7f26255b80 1 -- 172.21.15.46:0/3235485193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 msgr2=0x55707c8319f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.007+0000 7f7f26255b80 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 0x55707c8319f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7f08011fd0 tx=0x7f7f080047c0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.008+0000 7f7f26255b80 1 -- 172.21.15.46:0/3235485193 shutdown_connections 2024-03-18T20:40:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.008+0000 7f7f26255b80 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55707c83cbb0 0x55707c83cfc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.008+0000 7f7f26255b80 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 0x55707c82c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.008+0000 7f7f26255b80 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c8314b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.008+0000 7f7f26255b80 1 --2- 172.21.15.46:0/3235485193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 0x55707c8319f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.008+0000 7f7f26255b80 1 -- 172.21.15.46:0/3235485193 >> 172.21.15.46:0/3235485193 conn(0x55707c798220 msgr2=0x55707c81dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.008+0000 7f7ef2ffd700 0 client.26708 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:40:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.008+0000 7f7f26255b80 1 -- 172.21.15.46:0/3235485193 shutdown_connections 2024-03-18T20:40:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.008+0000 7f7f26255b80 1 -- 172.21.15.46:0/3235485193 wait complete. 2024-03-18T20:40:21.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.008+0000 7f7f26255b80 20 client.26708 trim_cache size 0 max 16384 2024-03-18T20:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.027+0000 7f7f26255b80 1 Processor -- start 2024-03-18T20:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.027+0000 7f7f26255b80 1 -- start start 2024-03-18T20:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.027+0000 7f7f26255b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c821a60 0x55707c827a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.027+0000 7f7f26255b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 0x55707c827fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.027+0000 7f7f26255b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c83e230 0x55707c844640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.027+0000 7f7f26255b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707c821910 con 0x55707c822380 2024-03-18T20:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.027+0000 7f7f26255b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707c76d700 con 0x55707c821a60 2024-03-18T20:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.027+0000 7f7f26255b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707c7e8940 con 0x55707c83e230 2024-03-18T20:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.027+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 0x55707c827fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.027+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 0x55707c827fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34388/0 (socket says 172.21.15.46:34388) 2024-03-18T20:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7f12ffd700 1 -- 172.21.15.46:0/2438574537 learned_addr learned my addr 172.21.15.46:0/2438574537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7f137fe700 1 --2- 172.21.15.46:0/2438574537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c821a60 0x55707c827a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7f13fff700 1 --2- 172.21.15.46:0/2438574537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c83e230 0x55707c844640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7f12ffd700 1 -- 172.21.15.46:0/2438574537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c83e230 msgr2=0x55707c844640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7f12ffd700 1 --2- 172.21.15.46:0/2438574537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c83e230 0x55707c844640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7f12ffd700 1 -- 172.21.15.46:0/2438574537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c821a60 msgr2=0x55707c827a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7f12ffd700 1 --2- 172.21.15.46:0/2438574537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c821a60 0x55707c827a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7f12ffd700 1 -- 172.21.15.46:0/2438574537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55707c73aea0 con 0x55707c822380 2024-03-18T20:40:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7f13fff700 1 --2- 172.21.15.46:0/2438574537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c83e230 0x55707c844640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7f12ffd700 1 --2- 172.21.15.46:0/2438574537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 0x55707c827fc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7f08005c50 tx=0x7f7f080092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa47bce9ee9837a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7efa7fc700 1 -- 172.21.15.46:0/2438574537 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08044050 con 0x55707c822380 2024-03-18T20:40:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7efa7fc700 1 -- 172.21.15.46:0/2438574537 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55707c815ad0 con 0x55707c822380 2024-03-18T20:40:21.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.028+0000 7f7efa7fc700 1 -- 172.21.15.46:0/2438574537 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0803f8c0 con 0x55707c822380 2024-03-18T20:40:21.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.029+0000 7f7f26255b80 1 -- 172.21.15.46:0/2438574537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 msgr2=0x55707c827fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.029+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2438574537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 0x55707c827fc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7f08005c50 tx=0x7f7f080092d0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.029+0000 7f7f26255b80 1 -- 172.21.15.46:0/2438574537 shutdown_connections 2024-03-18T20:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.029+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2438574537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c83e230 0x55707c844640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.029+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2438574537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c821a60 0x55707c827a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.029+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2438574537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c822380 0x55707c827fc0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.029+0000 7f7f26255b80 1 -- 172.21.15.46:0/2438574537 >> 172.21.15.46:0/2438574537 conn(0x55707c798220 msgr2=0x55707c826f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.029+0000 7f7f26255b80 1 -- 172.21.15.46:0/2438574537 shutdown_connections 2024-03-18T20:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.029+0000 7f7f26255b80 1 -- 172.21.15.46:0/2438574537 wait complete. 2024-03-18T20:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.029+0000 7f7f26255b80 1 -- start start 2024-03-18T20:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f26255b80 1 Processor -- start 2024-03-18T20:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f26255b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c83e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f26255b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 0x55707c83eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f26255b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 0x55707c83f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f26255b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707c843660 con 0x55707c83e230 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f26255b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707c8437d0 con 0x55707c822380 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f26255b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55707c843990 con 0x55707c821a60 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f26255b80 10 client.0 _ll_register_callbacks cb 0x7f7f080040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 0x55707c83f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 0x55707c83f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34392/0 (socket says 172.21.15.46:34392) 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f13fff700 1 -- 172.21.15.46:0/2513491656 learned_addr learned my addr 172.21.15.46:0/2513491656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f12ffd700 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 0x55707c83eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.030+0000 7f7f137fe700 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c83e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f13fff700 1 -- 172.21.15.46:0/2513491656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 msgr2=0x55707c83e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f13fff700 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c83e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f13fff700 1 -- 172.21.15.46:0/2513491656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 msgr2=0x55707c83eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f13fff700 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 0x55707c83eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f13fff700 1 -- 172.21.15.46:0/2513491656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55707c5a2b60 con 0x55707c83e230 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f12ffd700 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 0x55707c83eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f13fff700 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 0x55707c83f090 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0062a0 tx=0x7f7f0c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321b1fb15d0fb481 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7ef2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7ef2ffd700 1 -- 172.21.15.46:0/2513491656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00e040 con 0x55707c83e230 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7ef2ffd700 1 -- 172.21.15.46:0/2513491656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f7f0c025460 con 0x55707c83e230 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f26255b80 1 -- 172.21.15.46:0/2513491656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f0c006080 con 0x55707c83e230 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7ef2ffd700 1 -- 172.21.15.46:0/2513491656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f0c025620 con 0x55707c83e230 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f26255b80 10 client.16995 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f26255b80 1 -- 172.21.15.46:0/2513491656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55707c5facd0 con 0x55707c83e230 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7ef2ffd700 1 -- 172.21.15.46:0/2513491656 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f0c006080 con 0x55707c83e230 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7ef2ffd700 1 -- 172.21.15.46:0/2513491656 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7f0c034070 con 0x55707c83e230 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f26255b80 10 client.16995 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:21.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f26255b80 10 client.16995 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.031+0000 7f7f26255b80 20 client.16995 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.032+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55707c8916e0 0x55707c893ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.032+0000 7f7f26255b80 4 client.16995 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.032+0000 7f7f26255b80 1 -- 172.21.15.46:0/2513491656 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55707c5c3cf0 con 0x55707c8916e0 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.032+0000 7f7f137fe700 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55707c8916e0 0x55707c893ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.032+0000 7f7ef2ffd700 10 client.16995 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.032+0000 7f7f137fe700 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55707c8916e0 0x55707c893ad0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c34c8bfd9fc68898 server_cookie=86ed33db621c643f in_seq=0 out_seq=0 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.032+0000 7f7ef2ffd700 1 -- 172.21.15.46:0/2513491656 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55707c5c3cf0 con 0x55707c8916e0 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.032+0000 7f7ef2ffd700 10 client.16995 handle_command_reply: tid=0 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.032+0000 7f7f26255b80 1 client.16995 shutdown 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f26255b80 1 -- 172.21.15.46:0/2513491656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 msgr2=0x55707c83f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 0x55707c83f090 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0062a0 tx=0x7f7f0c00bc80 comp rx=0 tx=0).stop 2024-03-18T20:40:21.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f26255b80 1 -- 172.21.15.46:0/2513491656 shutdown_connections 2024-03-18T20:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55707c8916e0 0x55707c893ad0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55707c83e230 0x55707c83f090 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55707c821a60 0x55707c83e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7ef2ffd700 0 client.16995 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f26255b80 1 --2- 172.21.15.46:0/2513491656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55707c822380 0x55707c83eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f26255b80 1 -- 172.21.15.46:0/2513491656 >> 172.21.15.46:0/2513491656 conn(0x55707c798220 msgr2=0x55707c82ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f26255b80 1 -- 172.21.15.46:0/2513491656 shutdown_connections 2024-03-18T20:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f26255b80 1 -- 172.21.15.46:0/2513491656 wait complete. 2024-03-18T20:40:21.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f26255b80 20 client.16995 trim_cache size 0 max 16384 2024-03-18T20:40:21.042 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:40:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/343448362 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7efc0a1300 msgr2=0x7f7efc0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f1a0c0700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7efc0a1300 0x7f7efc0a37c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7f14101e50 tx=0x7f7f08009260 comp rx=0 tx=0).stop 2024-03-18T20:40:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/343448362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140ff7b0 msgr2=0x7f7f14100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.033+0000 7f7f1a0c0700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140ff7b0 0x7f7f14100720 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400eb30 tx=0x7f7f0400ee40 comp rx=0 tx=0).stop 2024-03-18T20:40:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.034+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/343448362 shutdown_connections 2024-03-18T20:40:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.034+0000 7f7f1a0c0700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7efc0a1300 0x7f7efc0a37c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.034+0000 7f7f1a0c0700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f140ff7b0 0x7f7f14100720 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.034+0000 7f7f1a0c0700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f141011a0 0x7f7f141a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.034+0000 7f7f1a0c0700 1 --2- 172.21.15.46:0/343448362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f14106140 0x7f7f14100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.034+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/343448362 >> 172.21.15.46:0/343448362 conn(0x7f7f14074e90 msgr2=0x7f7f140fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.034+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/343448362 shutdown_connections 2024-03-18T20:40:21.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.034+0000 7f7f1a0c0700 1 -- 172.21.15.46:0/343448362 wait complete. 2024-03-18T20:40:21.050 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:40:21.050 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:40:21.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.229+0000 7f54fa0c7700 1 -- 172.21.15.46:0/4081070608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f41060b0 msgr2=0x7f54f410dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.229+0000 7f54fa0c7700 1 --2- 172.21.15.46:0/4081070608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f41060b0 0x7f54f410dc30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00b0d0 tx=0x7f54ec00b3e0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.229+0000 7f54fa0c7700 1 -- 172.21.15.46:0/4081070608 shutdown_connections 2024-03-18T20:40:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.229+0000 7f54fa0c7700 1 --2- 172.21.15.46:0/4081070608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f41060b0 0x7f54f410dc30 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.229+0000 7f54fa0c7700 1 --2- 172.21.15.46:0/4081070608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54f4069280 0x7f54f4069660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.229+0000 7f54fa0c7700 1 --2- 172.21.15.46:0/4081070608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54f4069c30 0x7f54f4105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.229+0000 7f54fa0c7700 1 -- 172.21.15.46:0/4081070608 >> 172.21.15.46:0/4081070608 conn(0x7f54f4076d80 msgr2=0x7f54f4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.229+0000 7f54fa0c7700 1 -- 172.21.15.46:0/4081070608 shutdown_connections 2024-03-18T20:40:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.229+0000 7f54fa0c7700 1 -- 172.21.15.46:0/4081070608 wait complete. 2024-03-18T20:40:21.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.230+0000 7f54fa0c7700 1 Processor -- start 2024-03-18T20:40:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.230+0000 7f54fa0c7700 1 -- start start 2024-03-18T20:40:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.230+0000 7f54fa0c7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f4069280 0x7f54f41972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.230+0000 7f54fa0c7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54f41060b0 0x7f54f41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.230+0000 7f54fa0c7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54f419bb90 0x7f54f419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.230+0000 7f54fa0c7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f419c610 con 0x7f54f4069280 2024-03-18T20:40:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.230+0000 7f54fa0c7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f419c780 con 0x7f54f41060b0 2024-03-18T20:40:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.230+0000 7f54fa0c7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f419c940 con 0x7f54f419bb90 2024-03-18T20:40:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f4069280 0x7f54f41972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f4069280 0x7f54f41972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34400/0 (socket says 172.21.15.46:34400) 2024-03-18T20:40:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f37fe700 1 -- 172.21.15.46:0/3369047504 learned_addr learned my addr 172.21.15.46:0/3369047504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54f41060b0 0x7f54f41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f3fff700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54f419bb90 0x7f54f419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f37fe700 1 -- 172.21.15.46:0/3369047504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54f419bb90 msgr2=0x7f54f419c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f37fe700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54f419bb90 0x7f54f419c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f37fe700 1 -- 172.21.15.46:0/3369047504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54f41060b0 msgr2=0x7f54f41977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f37fe700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54f41060b0 0x7f54f41977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f37fe700 1 -- 172.21.15.46:0/3369047504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec009d00 con 0x7f54f4069280 2024-03-18T20:40:21.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f37fe700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f4069280 0x7f54f41972b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00d980 tx=0x7f54dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f0ff9700 1 -- 172.21.15.46:0/3369047504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc004220 con 0x7f54f4069280 2024-03-18T20:40:21.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f0ff9700 1 -- 172.21.15.46:0/3369047504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54dc007640 con 0x7f54f4069280 2024-03-18T20:40:21.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54fa0c7700 1 -- 172.21.15.46:0/3369047504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f41a3ea0 con 0x7f54f4069280 2024-03-18T20:40:21.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54f0ff9700 1 -- 172.21.15.46:0/3369047504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc005610 con 0x7f54f4069280 2024-03-18T20:40:21.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.231+0000 7f54fa0c7700 1 -- 172.21.15.46:0/3369047504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f41a4320 con 0x7f54f4069280 2024-03-18T20:40:21.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.234+0000 7f54f0ff9700 1 -- 172.21.15.46:0/3369047504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f54dc004750 con 0x7f54f4069280 2024-03-18T20:40:21.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.234+0000 7f54f0ff9700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f54e00a1290 0x7f54e00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.234+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f54e00a1290 0x7f54e00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.234+0000 7f54f0ff9700 1 -- 172.21.15.46:0/3369047504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f54dc0ccb00 con 0x7f54f4069280 2024-03-18T20:40:21.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.235+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f54e00a1290 0x7f54e00a3750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f54f4198800 tx=0x7f54e4009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.235+0000 7f550625cb80 1 Processor -- start 2024-03-18T20:40:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.235+0000 7f550625cb80 1 -- start start 2024-03-18T20:40:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f550625cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfbba60 0x555e9bfbbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f550625cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 0x555e9bfc6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f550625cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfc7220 0x555e9bfc9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f550625cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9bf15990 con 0x555e9bfbc380 2024-03-18T20:40:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f550625cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9bf13350 con 0x555e9bfc7220 2024-03-18T20:40:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f550625cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9bf0ec40 con 0x555e9bfbba60 2024-03-18T20:40:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 0x555e9bfc6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 0x555e9bfc6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34416/0 (socket says 172.21.15.46:34416) 2024-03-18T20:40:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54f2ffd700 1 -- 172.21.15.46:0/1732552937 learned_addr learned my addr 172.21.15.46:0/1732552937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54f37fe700 1 --2- 172.21.15.46:0/1732552937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfbba60 0x555e9bfbbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54f3fff700 1 --2- 172.21.15.46:0/1732552937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfc7220 0x555e9bfc9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54f2ffd700 1 -- 172.21.15.46:0/1732552937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfbba60 msgr2=0x555e9bfbbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/1732552937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfbba60 0x555e9bfbbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54f2ffd700 1 -- 172.21.15.46:0/1732552937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfc7220 msgr2=0x555e9bfc9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/1732552937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfc7220 0x555e9bfc9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54f2ffd700 1 -- 172.21.15.46:0/1732552937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e9bd903b0 con 0x555e9bfbc380 2024-03-18T20:40:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/1732552937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 0x555e9bfc6ce0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f54e4011d10 tx=0x7f54e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c8dd91b3738658 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54e9ffb700 1 -- 172.21.15.46:0/1732552937 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4011d40 con 0x555e9bfbc380 2024-03-18T20:40:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54e9ffb700 1 -- 172.21.15.46:0/1732552937 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54e4004020 con 0x555e9bfbc380 2024-03-18T20:40:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.236+0000 7f54e9ffb700 1 -- 172.21.15.46:0/1732552937 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4003b20 con 0x555e9bfbc380 2024-03-18T20:40:21.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.237+0000 7f550625cb80 1 -- 172.21.15.46:0/1732552937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 msgr2=0x555e9bfc6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.237+0000 7f550625cb80 1 --2- 172.21.15.46:0/1732552937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 0x555e9bfc6ce0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f54e4011d10 tx=0x7f54e4014000 comp rx=0 tx=0).stop 2024-03-18T20:40:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.237+0000 7f550625cb80 1 -- 172.21.15.46:0/1732552937 shutdown_connections 2024-03-18T20:40:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.237+0000 7f550625cb80 1 --2- 172.21.15.46:0/1732552937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfc7220 0x555e9bfc9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.237+0000 7f550625cb80 1 --2- 172.21.15.46:0/1732552937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfbba60 0x555e9bfbbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.237+0000 7f550625cb80 1 --2- 172.21.15.46:0/1732552937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 0x555e9bfc6ce0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.237+0000 7f550625cb80 1 -- 172.21.15.46:0/1732552937 >> 172.21.15.46:0/1732552937 conn(0x555e9bf32220 msgr2=0x555e9bfb8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.237+0000 7f550625cb80 1 -- 172.21.15.46:0/1732552937 shutdown_connections 2024-03-18T20:40:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.237+0000 7f550625cb80 1 -- 172.21.15.46:0/1732552937 wait complete. 2024-03-18T20:40:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.238+0000 7f550625cb80 1 -- start start 2024-03-18T20:40:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.238+0000 7f550625cb80 1 Processor -- start 2024-03-18T20:40:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.238+0000 7f550625cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbba60 0x555e9bfc6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.238+0000 7f550625cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbc380 0x555e9bfc3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.238+0000 7f550625cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfc7220 0x555e9bfc4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.238+0000 7f550625cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9beaad80 con 0x555e9bfbba60 2024-03-18T20:40:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.238+0000 7f550625cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9beaaa20 con 0x555e9bfbc380 2024-03-18T20:40:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.238+0000 7f550625cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9bd926a0 con 0x555e9bfc7220 2024-03-18T20:40:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.238+0000 7f550625cb80 10 client.0 _ll_register_callbacks cb 0x555e9bd163a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.238+0000 7f54f37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbba60 0x555e9bfc6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.238+0000 7f54f37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbba60 0x555e9bfc6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34418/0 (socket says 172.21.15.46:34418) 2024-03-18T20:40:21.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.238+0000 7f54f37fe700 1 -- 172.21.15.46:0/1392007868 learned_addr learned my addr 172.21.15.46:0/1392007868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbc380 0x555e9bfc3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54f3fff700 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfc7220 0x555e9bfc4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54f37fe700 1 -- 172.21.15.46:0/1392007868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfc7220 msgr2=0x555e9bfc4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54f37fe700 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfc7220 0x555e9bfc4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54f37fe700 1 -- 172.21.15.46:0/1392007868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbc380 msgr2=0x555e9bfc3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54f37fe700 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbc380 0x555e9bfc3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54f37fe700 1 -- 172.21.15.46:0/1392007868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e4017070 con 0x555e9bfbba60 2024-03-18T20:40:21.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54f3fff700 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfc7220 0x555e9bfc4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:21.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54f37fe700 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbba60 0x555e9bfc6c20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0ca2c0 tx=0x7f54dc099ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf1c5a72c0af8f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:21.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54d2ffd700 1 -- 172.21.15.46:0/1392007868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc099e80 con 0x555e9bfbba60 2024-03-18T20:40:21.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54d2ffd700 1 -- 172.21.15.46:0/1392007868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f54dc099fe0 con 0x555e9bfbba60 2024-03-18T20:40:21.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f550625cb80 1 -- 172.21.15.46:0/1392007868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e9bfafad0 con 0x555e9bfbba60 2024-03-18T20:40:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54d2ffd700 1 -- 172.21.15.46:0/1392007868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54dc099110 con 0x555e9bfbba60 2024-03-18T20:40:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f550625cb80 10 client.17007 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f550625cb80 1 -- 172.21.15.46:0/1392007868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x555e9bd3cb60 con 0x555e9bfbba60 2024-03-18T20:40:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.239+0000 7f54d2ffd700 1 -- 172.21.15.46:0/1392007868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54dc063080 con 0x555e9bfbba60 2024-03-18T20:40:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.240+0000 7f54d2ffd700 1 -- 172.21.15.46:0/1392007868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f54dc05fad0 con 0x555e9bfbba60 2024-03-18T20:40:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.240+0000 7f550625cb80 10 client.17007 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.240+0000 7f550625cb80 10 client.17007 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:40:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.240+0000 7f550625cb80 20 client.17007 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.240+0000 7f550625cb80 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555e9bfd6d50 0x555e9bfd9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.240+0000 7f550625cb80 4 client.17007 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.240+0000 7f550625cb80 1 -- 172.21.15.46:0/1392007868 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e9bf07700 con 0x555e9bfd6d50 2024-03-18T20:40:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.240+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555e9bfd6d50 0x555e9bfd9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.240+0000 7f54d2ffd700 10 client.17007 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:40:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.240+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555e9bfd6d50 0x555e9bfd9140 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=338af95883c51540 server_cookie=6cb91dd527c85656 in_seq=0 out_seq=0 2024-03-18T20:40:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f54d2ffd700 1 -- 172.21.15.46:0/1392007868 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555e9bf07700 con 0x555e9bfd6d50 2024-03-18T20:40:21.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f54d2ffd700 10 client.17007 handle_command_reply: tid=0 2024-03-18T20:40:21.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f550625cb80 1 client.17007 shutdown 2024-03-18T20:40:21.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f550625cb80 1 -- 172.21.15.46:0/1392007868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbba60 msgr2=0x555e9bfc6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f550625cb80 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbba60 0x555e9bfc6c20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0ca2c0 tx=0x7f54dc099ce0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f550625cb80 1 -- 172.21.15.46:0/1392007868 shutdown_connections 2024-03-18T20:40:21.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f550625cb80 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555e9bfd6d50 0x555e9bfd9140 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f550625cb80 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbba60 0x555e9bfc6c20 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f550625cb80 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfc7220 0x555e9bfc4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f550625cb80 1 --2- 172.21.15.46:0/1392007868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbc380 0x555e9bfc3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f54d2ffd700 0 client.17007 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:40:21.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f550625cb80 1 -- 172.21.15.46:0/1392007868 >> 172.21.15.46:0/1392007868 conn(0x555e9bf32220 msgr2=0x555e9bfb7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f550625cb80 1 -- 172.21.15.46:0/1392007868 shutdown_connections 2024-03-18T20:40:21.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.241+0000 7f550625cb80 1 -- 172.21.15.46:0/1392007868 wait complete. 2024-03-18T20:40:21.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.242+0000 7f550625cb80 20 client.17007 trim_cache size 0 max 16384 2024-03-18T20:40:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.255+0000 7f550625cb80 1 Processor -- start 2024-03-18T20:40:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.255+0000 7f550625cb80 1 -- start start 2024-03-18T20:40:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f550625cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbba60 0x555e9bfc44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f550625cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 0x555e9bfbe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f550625cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfc7220 0x555e9bfcde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f550625cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9bfbcd80 con 0x555e9bfbc380 2024-03-18T20:40:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f550625cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9bfb9230 con 0x555e9bfbba60 2024-03-18T20:40:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f550625cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9bfce3c0 con 0x555e9bfc7220 2024-03-18T20:40:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 0x555e9bfbe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 0x555e9bfbe9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34422/0 (socket says 172.21.15.46:34422) 2024-03-18T20:40:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54f2ffd700 1 -- 172.21.15.46:0/90426377 learned_addr learned my addr 172.21.15.46:0/90426377 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54f37fe700 1 --2- 172.21.15.46:0/90426377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbba60 0x555e9bfc44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54f3fff700 1 --2- 172.21.15.46:0/90426377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfc7220 0x555e9bfcde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54f2ffd700 1 -- 172.21.15.46:0/90426377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfc7220 msgr2=0x555e9bfcde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/90426377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfc7220 0x555e9bfcde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54f2ffd700 1 -- 172.21.15.46:0/90426377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbba60 msgr2=0x555e9bfc44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/90426377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbba60 0x555e9bfc44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54f2ffd700 1 -- 172.21.15.46:0/90426377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e9bed4ea0 con 0x555e9bfbc380 2024-03-18T20:40:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54f37fe700 1 --2- 172.21.15.46:0/90426377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbba60 0x555e9bfc44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/90426377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 0x555e9bfbe9c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009fd0 tx=0x7f54e4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4b2e0ff24fda06 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54ea7fc700 1 -- 172.21.15.46:0/90426377 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e402b040 con 0x555e9bfbc380 2024-03-18T20:40:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54ea7fc700 1 -- 172.21.15.46:0/90426377 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54e40054e0 con 0x555e9bfbc380 2024-03-18T20:40:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.256+0000 7f54ea7fc700 1 -- 172.21.15.46:0/90426377 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4034330 con 0x555e9bfbc380 2024-03-18T20:40:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.257+0000 7f550625cb80 1 -- 172.21.15.46:0/90426377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 msgr2=0x555e9bfbe9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.257+0000 7f550625cb80 1 --2- 172.21.15.46:0/90426377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 0x555e9bfbe9c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009fd0 tx=0x7f54e4011ad0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.257+0000 7f550625cb80 1 -- 172.21.15.46:0/90426377 shutdown_connections 2024-03-18T20:40:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.257+0000 7f550625cb80 1 --2- 172.21.15.46:0/90426377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfc7220 0x555e9bfcde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.257+0000 7f550625cb80 1 --2- 172.21.15.46:0/90426377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbba60 0x555e9bfc44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.257+0000 7f550625cb80 1 --2- 172.21.15.46:0/90426377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfbc380 0x555e9bfbe9c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.257+0000 7f550625cb80 1 -- 172.21.15.46:0/90426377 >> 172.21.15.46:0/90426377 conn(0x555e9bf32220 msgr2=0x555e9bfbf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.257+0000 7f550625cb80 1 -- 172.21.15.46:0/90426377 shutdown_connections 2024-03-18T20:40:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.257+0000 7f550625cb80 1 -- 172.21.15.46:0/90426377 wait complete. 2024-03-18T20:40:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.257+0000 7f550625cb80 1 -- start start 2024-03-18T20:40:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f550625cb80 1 Processor -- start 2024-03-18T20:40:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f550625cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfbba60 0x555e9c027260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f550625cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbc380 0x555e9c0277a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f550625cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfc7220 0x555e9bfc9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f550625cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9bfca530 con 0x555e9bfc7220 2024-03-18T20:40:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f550625cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9bfca6a0 con 0x555e9bfbc380 2024-03-18T20:40:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f550625cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e9bfca860 con 0x555e9bfbba60 2024-03-18T20:40:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f550625cb80 10 client.0 _ll_register_callbacks cb 0x7f54dc09a320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f54f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfc7220 0x555e9bfc9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f54f3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfc7220 0x555e9bfc9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34434/0 (socket says 172.21.15.46:34434) 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f54f3fff700 1 -- 172.21.15.46:0/2040083561 learned_addr learned my addr 172.21.15.46:0/2040083561 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbc380 0x555e9c0277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f54f37fe700 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfbba60 0x555e9c027260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f54f3fff700 1 -- 172.21.15.46:0/2040083561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfbba60 msgr2=0x555e9c027260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f54f3fff700 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfbba60 0x555e9c027260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f54f3fff700 1 -- 172.21.15.46:0/2040083561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbc380 msgr2=0x555e9c0277a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f54f3fff700 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbc380 0x555e9c0277a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f54f3fff700 1 -- 172.21.15.46:0/2040083561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e9bd903b0 con 0x555e9bfc7220 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f54f37fe700 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfbba60 0x555e9c027260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.258+0000 7f54f2ffd700 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbc380 0x555e9c0277a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f54f3fff700 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfc7220 0x555e9bfc9e50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00baa0 tx=0x7f54ec00bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bc529726b48330 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f54d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f54d2ffd700 1 -- 172.21.15.46:0/2040083561 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00e040 con 0x555e9bfc7220 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f54d2ffd700 1 -- 172.21.15.46:0/2040083561 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f54ec004c50 con 0x555e9bfc7220 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f550625cb80 1 -- 172.21.15.46:0/2040083561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54e4004020 con 0x555e9bfc7220 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f54d2ffd700 1 -- 172.21.15.46:0/2040083561 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54ec004e50 con 0x555e9bfc7220 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f550625cb80 10 client.17013 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f550625cb80 1 -- 172.21.15.46:0/2040083561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x555e9bd94cd0 con 0x555e9bfc7220 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f54d2ffd700 1 -- 172.21.15.46:0/2040083561 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54e4004020 con 0x555e9bfc7220 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f54d2ffd700 1 -- 172.21.15.46:0/2040083561 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f54ec02e070 con 0x555e9bfc7220 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f550625cb80 10 client.17013 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f550625cb80 10 client.17013 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f550625cb80 20 client.17013 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f550625cb80 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555e9c0301d0 0x555e9c0325f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f550625cb80 4 client.17013 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:40:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.259+0000 7f550625cb80 1 -- 172.21.15.46:0/2040083561 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555e9bd5dcf0 con 0x555e9c0301d0 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.260+0000 7f54f37fe700 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555e9c0301d0 0x555e9c0325f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.260+0000 7f54f37fe700 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555e9c0301d0 0x555e9c0325f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b8ce1a17e2f09db1 server_cookie=2a6f87435693d8 in_seq=0 out_seq=0 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.260+0000 7f54d2ffd700 10 client.17013 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.260+0000 7f54d2ffd700 1 -- 172.21.15.46:0/2040083561 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555e9bd5dcf0 con 0x555e9c0301d0 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.260+0000 7f54d2ffd700 10 client.17013 handle_command_reply: tid=0 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.260+0000 7f550625cb80 1 client.17013 shutdown 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.260+0000 7f550625cb80 1 -- 172.21.15.46:0/2040083561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfc7220 msgr2=0x555e9bfc9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.260+0000 7f550625cb80 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfc7220 0x555e9bfc9e50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00baa0 tx=0x7f54ec00bad0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f550625cb80 1 -- 172.21.15.46:0/2040083561 shutdown_connections 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f550625cb80 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555e9c0301d0 0x555e9c0325f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f550625cb80 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e9bfc7220 0x555e9bfc9e50 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f550625cb80 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555e9bfbba60 0x555e9c027260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f550625cb80 1 --2- 172.21.15.46:0/2040083561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555e9bfbc380 0x555e9c0277a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f54d2ffd700 0 client.17013 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:40:21.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f550625cb80 1 -- 172.21.15.46:0/2040083561 >> 172.21.15.46:0/2040083561 conn(0x555e9bf32220 msgr2=0x555e9bfb7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f550625cb80 1 -- 172.21.15.46:0/2040083561 shutdown_connections 2024-03-18T20:40:21.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f550625cb80 1 -- 172.21.15.46:0/2040083561 wait complete. 2024-03-18T20:40:21.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f550625cb80 20 client.17013 trim_cache size 0 max 16384 2024-03-18T20:40:21.286 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:40:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f54fa0c7700 1 -- 172.21.15.46:0/3369047504 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f54e00a1290 msgr2=0x7f54e00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f54fa0c7700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f54e00a1290 0x7f54e00a3750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f54f4198800 tx=0x7f54e4009260 comp rx=0 tx=0).stop 2024-03-18T20:40:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f54fa0c7700 1 -- 172.21.15.46:0/3369047504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f4069280 msgr2=0x7f54f41972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f54fa0c7700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f4069280 0x7f54f41972b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00d980 tx=0x7f54dc00dc90 comp rx=0 tx=0).stop 2024-03-18T20:40:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f54fa0c7700 1 -- 172.21.15.46:0/3369047504 shutdown_connections 2024-03-18T20:40:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f54fa0c7700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f54e00a1290 0x7f54e00a3750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f54fa0c7700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54f4069280 0x7f54f41972b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f54fa0c7700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54f419bb90 0x7f54f419c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f54fa0c7700 1 --2- 172.21.15.46:0/3369047504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54f41060b0 0x7f54f41977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f54fa0c7700 1 -- 172.21.15.46:0/3369047504 >> 172.21.15.46:0/3369047504 conn(0x7f54f4076d80 msgr2=0x7f54f40ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.261+0000 7f54fa0c7700 1 -- 172.21.15.46:0/3369047504 shutdown_connections 2024-03-18T20:40:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.262+0000 7f54fa0c7700 1 -- 172.21.15.46:0/3369047504 wait complete. 2024-03-18T20:40:21.289 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:40:21.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:40:21.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.426+0000 7f8332549700 1 -- 172.21.15.46:0/1323267101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105ce0 msgr2=0x7f832c1060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.426+0000 7f8332549700 1 --2- 172.21.15.46:0/1323267101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105ce0 0x7f832c1060c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f831c009990 tx=0x7f831c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.426+0000 7f8332549700 1 -- 172.21.15.46:0/1323267101 shutdown_connections 2024-03-18T20:40:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.426+0000 7f8332549700 1 --2- 172.21.15.46:0/1323267101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105ce0 0x7f832c1060c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.426+0000 7f8332549700 1 --2- 172.21.15.46:0/1323267101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f832c0687f0 0x7f832c10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.426+0000 7f8332549700 1 --2- 172.21.15.46:0/1323267101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f832c067e30 0x7f832c0682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.426+0000 7f8332549700 1 -- 172.21.15.46:0/1323267101 >> 172.21.15.46:0/1323267101 conn(0x7f832c076d40 msgr2=0x7f832c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.426+0000 7f8332549700 1 -- 172.21.15.46:0/1323267101 shutdown_connections 2024-03-18T20:40:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.426+0000 7f8332549700 1 -- 172.21.15.46:0/1323267101 wait complete. 2024-03-18T20:40:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f8332549700 1 Processor -- start 2024-03-18T20:40:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f8332549700 1 -- start start 2024-03-18T20:40:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f8332549700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f832c067e30 0x7f832c1971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f8332549700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f832c0687f0 0x7f832c197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f8332549700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105ce0 0x7f832c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f8332549700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c19c110 con 0x7f832c105ce0 2024-03-18T20:40:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f8332549700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c19c280 con 0x7f832c0687f0 2024-03-18T20:40:21.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f8332549700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c19c440 con 0x7f832c067e30 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f8330ae6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105ce0 0x7f832c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f832b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f832c0687f0 0x7f832c197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f8330ae6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105ce0 0x7f832c19bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34458/0 (socket says 172.21.15.46:34458) 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f8330ae6700 1 -- 172.21.15.46:0/812443292 learned_addr learned my addr 172.21.15.46:0/812443292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.427+0000 7f832bfff700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f832c067e30 0x7f832c1971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.428+0000 7f8330ae6700 1 -- 172.21.15.46:0/812443292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f832c067e30 msgr2=0x7f832c1971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.428+0000 7f8330ae6700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f832c067e30 0x7f832c1971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.428+0000 7f8330ae6700 1 -- 172.21.15.46:0/812443292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f832c0687f0 msgr2=0x7f832c197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.428+0000 7f8330ae6700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f832c0687f0 0x7f832c197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.428+0000 7f8330ae6700 1 -- 172.21.15.46:0/812443292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8314009520 con 0x7f832c105ce0 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.428+0000 7f8330ae6700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105ce0 0x7f832c19bad0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f832400ef30 tx=0x7f832400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.428+0000 7f83297fa700 1 -- 172.21.15.46:0/812443292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324018070 con 0x7f832c105ce0 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.428+0000 7f8332549700 1 -- 172.21.15.46:0/812443292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831c0095f0 con 0x7f832c105ce0 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.428+0000 7f83297fa700 1 -- 172.21.15.46:0/812443292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f832400f040 con 0x7f832c105ce0 2024-03-18T20:40:21.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.428+0000 7f83297fa700 1 -- 172.21.15.46:0/812443292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832400cd00 con 0x7f832c105ce0 2024-03-18T20:40:21.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.428+0000 7f8332549700 1 -- 172.21.15.46:0/812443292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c1a3d80 con 0x7f832c105ce0 2024-03-18T20:40:21.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.430+0000 7f83297fa700 1 -- 172.21.15.46:0/812443292 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8324004000 con 0x7f832c105ce0 2024-03-18T20:40:21.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.431+0000 7f83297fa700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f83180a1290 0x7f83180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.431+0000 7f832bfff700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f83180a1290 0x7f83180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.431+0000 7f83297fa700 1 -- 172.21.15.46:0/812443292 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f83240cc8c0 con 0x7f832c105ce0 2024-03-18T20:40:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f833e6deb80 1 Processor -- start 2024-03-18T20:40:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f832bfff700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f83180a1290 0x7f83180a3750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f831c0098e0 tx=0x7f831c005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f833e6deb80 1 -- start start 2024-03-18T20:40:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f833e6deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a507a60 0x56325a507e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f833e6deb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a508380 0x56325a512ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f833e6deb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a513220 0x56325a515610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f833e6deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325a461990 con 0x56325a507a60 2024-03-18T20:40:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f833e6deb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325a45f350 con 0x56325a508380 2024-03-18T20:40:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f833e6deb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325a45ac40 con 0x56325a513220 2024-03-18T20:40:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f832bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a507a60 0x56325a507e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f832bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a507a60 0x56325a507e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34466/0 (socket says 172.21.15.46:34466) 2024-03-18T20:40:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f832bfff700 1 -- 172.21.15.46:0/3293957733 learned_addr learned my addr 172.21.15.46:0/3293957733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f832b7fe700 1 --2- 172.21.15.46:0/3293957733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a508380 0x56325a512ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.432+0000 7f8330ae6700 1 --2- 172.21.15.46:0/3293957733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a513220 0x56325a515610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.433+0000 7f832bfff700 1 -- 172.21.15.46:0/3293957733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a513220 msgr2=0x56325a515610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.433+0000 7f832bfff700 1 --2- 172.21.15.46:0/3293957733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a513220 0x56325a515610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.433+0000 7f832bfff700 1 -- 172.21.15.46:0/3293957733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a508380 msgr2=0x56325a512ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.433+0000 7f832bfff700 1 --2- 172.21.15.46:0/3293957733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a508380 0x56325a512ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.433+0000 7f832bfff700 1 -- 172.21.15.46:0/3293957733 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56325a2dc3b0 con 0x56325a507a60 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.433+0000 7f832bfff700 1 --2- 172.21.15.46:0/3293957733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a507a60 0x56325a507e40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f831c009990 tx=0x7f831c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dde09e1413fad29 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.433+0000 7f83227fc700 1 -- 172.21.15.46:0/3293957733 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c004700 con 0x56325a507a60 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.433+0000 7f83227fc700 1 -- 172.21.15.46:0/3293957733 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f831c01e040 con 0x56325a507a60 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.433+0000 7f83227fc700 1 -- 172.21.15.46:0/3293957733 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00f890 con 0x56325a507a60 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.434+0000 7f833e6deb80 1 -- 172.21.15.46:0/3293957733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a507a60 msgr2=0x56325a507e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.434+0000 7f833e6deb80 1 --2- 172.21.15.46:0/3293957733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a507a60 0x56325a507e40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f831c009990 tx=0x7f831c020000 comp rx=0 tx=0).stop 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.434+0000 7f833e6deb80 1 -- 172.21.15.46:0/3293957733 shutdown_connections 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.434+0000 7f833e6deb80 1 --2- 172.21.15.46:0/3293957733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a507a60 0x56325a507e40 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.434+0000 7f833e6deb80 1 --2- 172.21.15.46:0/3293957733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a513220 0x56325a515610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.434+0000 7f833e6deb80 1 --2- 172.21.15.46:0/3293957733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a508380 0x56325a512ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.434+0000 7f833e6deb80 1 -- 172.21.15.46:0/3293957733 >> 172.21.15.46:0/3293957733 conn(0x56325a47e220 msgr2=0x56325a504020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.434+0000 7f833e6deb80 1 -- 172.21.15.46:0/3293957733 shutdown_connections 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.434+0000 7f833e6deb80 1 -- 172.21.15.46:0/3293957733 wait complete. 2024-03-18T20:40:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.434+0000 7f833e6deb80 1 -- start start 2024-03-18T20:40:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f833e6deb80 1 Processor -- start 2024-03-18T20:40:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f833e6deb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a507a60 0x56325a512c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f833e6deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a508380 0x56325a50ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f833e6deb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a513220 0x56325a510690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f833e6deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325a3f6d80 con 0x56325a508380 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f833e6deb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325a3f6a20 con 0x56325a507a60 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f833e6deb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325a2de6a0 con 0x56325a513220 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f833e6deb80 10 client.0 _ll_register_callbacks cb 0x56325a2623a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f832b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a508380 0x56325a50ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f832b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a508380 0x56325a50ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34472/0 (socket says 172.21.15.46:34472) 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f832bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a507a60 0x56325a512c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f832b7fe700 1 -- 172.21.15.46:0/1779464418 learned_addr learned my addr 172.21.15.46:0/1779464418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f8330ae6700 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a513220 0x56325a510690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f832b7fe700 1 -- 172.21.15.46:0/1779464418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a513220 msgr2=0x56325a510690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f832b7fe700 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a513220 0x56325a510690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f832b7fe700 1 -- 172.21.15.46:0/1779464418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a507a60 msgr2=0x56325a512c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f832b7fe700 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a507a60 0x56325a512c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f832b7fe700 1 -- 172.21.15.46:0/1779464418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c10e550 con 0x56325a508380 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.435+0000 7f832bfff700 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a507a60 0x56325a512c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f832b7fe700 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a508380 0x56325a50ff50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8314009870 tx=0x7f8314012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e823a561d7f272ef server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f830b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f830b7fe700 1 -- 172.21.15.46:0/1779464418 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314017070 con 0x56325a508380 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f830b7fe700 1 -- 172.21.15.46:0/1779464418 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f831400f5c0 con 0x56325a508380 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f833e6deb80 1 -- 172.21.15.46:0/1779464418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56325a4fbad0 con 0x56325a508380 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f830b7fe700 1 -- 172.21.15.46:0/1779464418 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83140122e0 con 0x56325a508380 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f833e6deb80 10 client.17025 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f833e6deb80 1 -- 172.21.15.46:0/1779464418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56325a288b60 con 0x56325a508380 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f830b7fe700 1 -- 172.21.15.46:0/1779464418 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8314028080 con 0x56325a508380 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f830b7fe700 1 -- 172.21.15.46:0/1779464418 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8314010970 con 0x56325a508380 2024-03-18T20:40:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f833e6deb80 10 client.17025 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f833e6deb80 10 client.17025 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f833e6deb80 20 client.17025 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f833e6deb80 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56325a522e50 0x56325a525270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f833e6deb80 4 client.17025 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.436+0000 7f833e6deb80 1 -- 172.21.15.46:0/1779464418 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56325a453700 con 0x56325a522e50 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.437+0000 7f832bfff700 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56325a522e50 0x56325a525270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.437+0000 7f832bfff700 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56325a522e50 0x56325a525270 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3d3c349182837df server_cookie=14198958e0aa94ff in_seq=0 out_seq=0 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.437+0000 7f830b7fe700 10 client.17025 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.438+0000 7f830b7fe700 1 -- 172.21.15.46:0/1779464418 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56325a453700 con 0x56325a522e50 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.438+0000 7f830b7fe700 10 client.17025 handle_command_reply: tid=0 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.438+0000 7f833e6deb80 1 client.17025 shutdown 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.438+0000 7f833e6deb80 1 -- 172.21.15.46:0/1779464418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a508380 msgr2=0x56325a50ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.438+0000 7f833e6deb80 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a508380 0x56325a50ff50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8314009870 tx=0x7f8314012c30 comp rx=0 tx=0).stop 2024-03-18T20:40:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.438+0000 7f833e6deb80 1 -- 172.21.15.46:0/1779464418 shutdown_connections 2024-03-18T20:40:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.438+0000 7f833e6deb80 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a513220 0x56325a510690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.438+0000 7f833e6deb80 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a507a60 0x56325a512c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.438+0000 7f833e6deb80 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56325a522e50 0x56325a525270 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.439+0000 7f833e6deb80 1 --2- 172.21.15.46:0/1779464418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a508380 0x56325a50ff50 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.439+0000 7f833e6deb80 1 -- 172.21.15.46:0/1779464418 >> 172.21.15.46:0/1779464418 conn(0x56325a47e220 msgr2=0x56325a503c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.439+0000 7f830b7fe700 0 client.17025 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:40:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.439+0000 7f833e6deb80 1 -- 172.21.15.46:0/1779464418 shutdown_connections 2024-03-18T20:40:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.439+0000 7f833e6deb80 1 -- 172.21.15.46:0/1779464418 wait complete. 2024-03-18T20:40:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.439+0000 7f833e6deb80 20 client.17025 trim_cache size 0 max 16384 2024-03-18T20:40:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.452+0000 7f833e6deb80 1 Processor -- start 2024-03-18T20:40:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f833e6deb80 1 -- start start 2024-03-18T20:40:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f833e6deb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a507a60 0x56325a510690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f833e6deb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a508380 0x56325a519df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f833e6deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 0x56325a51e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f833e6deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325a5005e0 con 0x56325a513220 2024-03-18T20:40:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f833e6deb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325a508d30 con 0x56325a508380 2024-03-18T20:40:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f833e6deb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325a5089d0 con 0x56325a507a60 2024-03-18T20:40:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f8330ae6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 0x56325a51e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f8330ae6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 0x56325a51e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34482/0 (socket says 172.21.15.46:34482) 2024-03-18T20:40:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f8330ae6700 1 -- 172.21.15.46:0/1306206996 learned_addr learned my addr 172.21.15.46:0/1306206996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f832b7fe700 1 --2- 172.21.15.46:0/1306206996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a508380 0x56325a519df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f832bfff700 1 --2- 172.21.15.46:0/1306206996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a507a60 0x56325a510690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f8330ae6700 1 -- 172.21.15.46:0/1306206996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a507a60 msgr2=0x56325a510690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f8330ae6700 1 --2- 172.21.15.46:0/1306206996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a507a60 0x56325a510690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f8330ae6700 1 -- 172.21.15.46:0/1306206996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a508380 msgr2=0x56325a519df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f8330ae6700 1 --2- 172.21.15.46:0/1306206996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a508380 0x56325a519df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f8330ae6700 1 -- 172.21.15.46:0/1306206996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56325a420ea0 con 0x56325a513220 2024-03-18T20:40:21.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.453+0000 7f8330ae6700 1 --2- 172.21.15.46:0/1306206996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 0x56325a51e540 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f832400ef00 tx=0x7f83240b7f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3a0a69bc1d5a60 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.454+0000 7f8322ffd700 1 -- 172.21.15.46:0/1306206996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324004ad0 con 0x56325a513220 2024-03-18T20:40:21.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.454+0000 7f8322ffd700 1 -- 172.21.15.46:0/1306206996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f832409a130 con 0x56325a513220 2024-03-18T20:40:21.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.454+0000 7f8322ffd700 1 -- 172.21.15.46:0/1306206996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324099c90 con 0x56325a513220 2024-03-18T20:40:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.454+0000 7f833e6deb80 1 -- 172.21.15.46:0/1306206996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 msgr2=0x56325a51e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.454+0000 7f833e6deb80 1 --2- 172.21.15.46:0/1306206996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 0x56325a51e540 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f832400ef00 tx=0x7f83240b7f90 comp rx=0 tx=0).stop 2024-03-18T20:40:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.454+0000 7f833e6deb80 1 -- 172.21.15.46:0/1306206996 shutdown_connections 2024-03-18T20:40:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.454+0000 7f833e6deb80 1 --2- 172.21.15.46:0/1306206996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 0x56325a51e540 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.454+0000 7f833e6deb80 1 --2- 172.21.15.46:0/1306206996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a507a60 0x56325a510690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.454+0000 7f833e6deb80 1 --2- 172.21.15.46:0/1306206996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a508380 0x56325a519df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.454+0000 7f833e6deb80 1 -- 172.21.15.46:0/1306206996 >> 172.21.15.46:0/1306206996 conn(0x56325a47e220 msgr2=0x56325a502750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.454+0000 7f833e6deb80 1 -- 172.21.15.46:0/1306206996 shutdown_connections 2024-03-18T20:40:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.454+0000 7f833e6deb80 1 -- 172.21.15.46:0/1306206996 wait complete. 2024-03-18T20:40:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.455+0000 7f833e6deb80 1 -- start start 2024-03-18T20:40:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.455+0000 7f833e6deb80 1 Processor -- start 2024-03-18T20:40:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.455+0000 7f833e6deb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a507a60 0x56325a50feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.455+0000 7f833e6deb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a508380 0x56325a517dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.455+0000 7f833e6deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 0x56325a518350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.455+0000 7f833e6deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325a518a60 con 0x56325a513220 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.455+0000 7f833e6deb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325a518bd0 con 0x56325a507a60 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.455+0000 7f833e6deb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325a575c80 con 0x56325a508380 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f833e6deb80 10 client.0 _ll_register_callbacks cb 0x7f8314012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f8330ae6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 0x56325a518350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f8330ae6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 0x56325a518350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34496/0 (socket says 172.21.15.46:34496) 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f8330ae6700 1 -- 172.21.15.46:0/3290706341 learned_addr learned my addr 172.21.15.46:0/3290706341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f832bfff700 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a507a60 0x56325a50feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f832b7fe700 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a508380 0x56325a517dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f8330ae6700 1 -- 172.21.15.46:0/3290706341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a508380 msgr2=0x56325a517dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f8330ae6700 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a508380 0x56325a517dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f8330ae6700 1 -- 172.21.15.46:0/3290706341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a507a60 msgr2=0x56325a50feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f8330ae6700 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a507a60 0x56325a50feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f8330ae6700 1 -- 172.21.15.46:0/3290706341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c1a3d80 con 0x56325a513220 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f832b7fe700 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a508380 0x56325a517dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f8330ae6700 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 0x56325a518350 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f83240090a0 tx=0x7f8324095ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37b2bd009097321 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f830b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f830b7fe700 1 -- 172.21.15.46:0/3290706341 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832400ce60 con 0x56325a513220 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f830b7fe700 1 -- 172.21.15.46:0/3290706341 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f83240a1e20 con 0x56325a513220 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.456+0000 7f833e6deb80 1 -- 172.21.15.46:0/3290706341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f831c01e040 con 0x56325a513220 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.457+0000 7f830b7fe700 1 -- 172.21.15.46:0/3290706341 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8324099c90 con 0x56325a513220 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.457+0000 7f833e6deb80 10 client.17031 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.457+0000 7f833e6deb80 1 -- 172.21.15.46:0/3290706341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56325a2e0cd0 con 0x56325a513220 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.457+0000 7f830b7fe700 1 -- 172.21.15.46:0/3290706341 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f831c01e040 con 0x56325a513220 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.457+0000 7f830b7fe700 1 -- 172.21.15.46:0/3290706341 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8324082e10 con 0x56325a513220 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.457+0000 7f833e6deb80 10 client.17031 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.457+0000 7f833e6deb80 10 client.17031 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.457+0000 7f833e6deb80 20 client.17031 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.457+0000 7f833e6deb80 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56325a576fc0 0x56325a5793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.457+0000 7f833e6deb80 4 client.17031 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.457+0000 7f833e6deb80 1 -- 172.21.15.46:0/3290706341 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56325a2a9cf0 con 0x56325a576fc0 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.457+0000 7f832bfff700 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56325a576fc0 0x56325a5793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f832bfff700 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56325a576fc0 0x56325a5793b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=968cd5fd9638f9bf server_cookie=80180d8355e03fcd in_seq=0 out_seq=0 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f830b7fe700 10 client.17031 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f830b7fe700 1 -- 172.21.15.46:0/3290706341 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56325a2a9cf0 con 0x56325a576fc0 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f830b7fe700 10 client.17031 handle_command_reply: tid=0 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f833e6deb80 1 client.17031 shutdown 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f833e6deb80 1 -- 172.21.15.46:0/3290706341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 msgr2=0x56325a518350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f833e6deb80 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 0x56325a518350 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f83240090a0 tx=0x7f8324095ea0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f833e6deb80 1 -- 172.21.15.46:0/3290706341 shutdown_connections 2024-03-18T20:40:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f833e6deb80 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56325a576fc0 0x56325a5793b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f833e6deb80 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56325a513220 0x56325a518350 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f830b7fe700 0 client.17031 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:40:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f833e6deb80 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56325a507a60 0x56325a50feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f833e6deb80 1 --2- 172.21.15.46:0/3290706341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56325a508380 0x56325a517dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f833e6deb80 1 -- 172.21.15.46:0/3290706341 >> 172.21.15.46:0/3290706341 conn(0x56325a47e220 msgr2=0x56325a51e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f833e6deb80 1 -- 172.21.15.46:0/3290706341 shutdown_connections 2024-03-18T20:40:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f833e6deb80 1 -- 172.21.15.46:0/3290706341 wait complete. 2024-03-18T20:40:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.458+0000 7f833e6deb80 20 client.17031 trim_cache size 0 max 16384 2024-03-18T20:40:21.467 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:40:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.459+0000 7f8332549700 1 -- 172.21.15.46:0/812443292 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f83180a1290 msgr2=0x7f83180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.459+0000 7f8332549700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f83180a1290 0x7f83180a3750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f831c0098e0 tx=0x7f831c005960 comp rx=0 tx=0).stop 2024-03-18T20:40:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.459+0000 7f8332549700 1 -- 172.21.15.46:0/812443292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105ce0 msgr2=0x7f832c19bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.459+0000 7f8332549700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105ce0 0x7f832c19bad0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f832400ef30 tx=0x7f832400c3a0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.459+0000 7f8332549700 1 -- 172.21.15.46:0/812443292 shutdown_connections 2024-03-18T20:40:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.459+0000 7f8332549700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f83180a1290 0x7f83180a3750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.459+0000 7f8332549700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f832c105ce0 0x7f832c19bad0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.459+0000 7f8332549700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f832c067e30 0x7f832c1971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.459+0000 7f8332549700 1 --2- 172.21.15.46:0/812443292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f832c0687f0 0x7f832c197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.459+0000 7f8332549700 1 -- 172.21.15.46:0/812443292 >> 172.21.15.46:0/812443292 conn(0x7f832c076d40 msgr2=0x7f832c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.459+0000 7f8332549700 1 -- 172.21.15.46:0/812443292 shutdown_connections 2024-03-18T20:40:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.459+0000 7f8332549700 1 -- 172.21.15.46:0/812443292 wait complete. 2024-03-18T20:40:21.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:21 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/403934935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:40:21.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:21 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/403934935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:40:21.475 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:40:21.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:40:21.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:21 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/403934935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:40:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.611+0000 7fc62f372700 1 -- 172.21.15.46:0/3424349274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc628100390 msgr2=0x7fc628100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.611+0000 7fc62f372700 1 --2- 172.21.15.46:0/3424349274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc628100390 0x7fc628100810 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc624009990 tx=0x7fc624009ca0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.611+0000 7fc62f372700 1 -- 172.21.15.46:0/3424349274 shutdown_connections 2024-03-18T20:40:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.611+0000 7fc62f372700 1 --2- 172.21.15.46:0/3424349274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc628100d50 0x7fc62810ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.611+0000 7fc62f372700 1 --2- 172.21.15.46:0/3424349274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc628104bf0 0x7fc628104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.611+0000 7fc62f372700 1 --2- 172.21.15.46:0/3424349274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc628100390 0x7fc628100810 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.611+0000 7fc62f372700 1 -- 172.21.15.46:0/3424349274 >> 172.21.15.46:0/3424349274 conn(0x7fc6280761b0 msgr2=0x7fc6280fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.611+0000 7fc62f372700 1 -- 172.21.15.46:0/3424349274 shutdown_connections 2024-03-18T20:40:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.611+0000 7fc62f372700 1 -- 172.21.15.46:0/3424349274 wait complete. 2024-03-18T20:40:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.612+0000 7fc62f372700 1 Processor -- start 2024-03-18T20:40:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.612+0000 7fc62f372700 1 -- start start 2024-03-18T20:40:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.612+0000 7fc62f372700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc628100390 0x7fc628199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.612+0000 7fc62f372700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc628100d50 0x7fc628199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.612+0000 7fc62f372700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc628104bf0 0x7fc62819dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.612+0000 7fc62f372700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62819e330 con 0x7fc628104bf0 2024-03-18T20:40:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.612+0000 7fc62f372700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62819e4a0 con 0x7fc628100d50 2024-03-18T20:40:21.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.612+0000 7fc62f372700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62819e660 con 0x7fc628100390 2024-03-18T20:40:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.612+0000 7fc62d90f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc628104bf0 0x7fc62819dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62d10e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc628100390 0x7fc628199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62d90f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc628104bf0 0x7fc62819dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34520/0 (socket says 172.21.15.46:34520) 2024-03-18T20:40:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62c90d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc628100d50 0x7fc628199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62d90f700 1 -- 172.21.15.46:0/2480382881 learned_addr learned my addr 172.21.15.46:0/2480382881 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62d90f700 1 -- 172.21.15.46:0/2480382881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc628100390 msgr2=0x7fc628199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62d90f700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc628100390 0x7fc628199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62d90f700 1 -- 172.21.15.46:0/2480382881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc628100d50 msgr2=0x7fc628199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62d90f700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc628100d50 0x7fc628199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62d90f700 1 -- 172.21.15.46:0/2480382881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6240095f0 con 0x7fc628104bf0 2024-03-18T20:40:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62d10e700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc628100390 0x7fc628199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62c90d700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc628100d50 0x7fc628199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62d90f700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc628104bf0 0x7fc62819dcf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc62000ef30 tx=0x7fc62000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc61e7fc700 1 -- 172.21.15.46:0/2480382881 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620018070 con 0x7fc628104bf0 2024-03-18T20:40:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc61e7fc700 1 -- 172.21.15.46:0/2480382881 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc62000f040 con 0x7fc628104bf0 2024-03-18T20:40:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62f372700 1 -- 172.21.15.46:0/2480382881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62819e8e0 con 0x7fc628104bf0 2024-03-18T20:40:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc61e7fc700 1 -- 172.21.15.46:0/2480382881 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62000cd00 con 0x7fc628104bf0 2024-03-18T20:40:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.613+0000 7fc62f372700 1 -- 172.21.15.46:0/2480382881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6281a6180 con 0x7fc628104bf0 2024-03-18T20:40:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.616+0000 7fc61e7fc700 1 -- 172.21.15.46:0/2480382881 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc620003a00 con 0x7fc628104bf0 2024-03-18T20:40:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.616+0000 7fc61e7fc700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc6140a1290 0x7fc6140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.616+0000 7fc61e7fc700 1 -- 172.21.15.46:0/2480382881 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc6200cc810 con 0x7fc628104bf0 2024-03-18T20:40:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.616+0000 7fc62d10e700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc6140a1290 0x7fc6140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.617+0000 7fc62d10e700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc6140a1290 0x7fc6140a3750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc618011490 tx=0x7fc618009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.617+0000 7fc63b507b80 1 Processor -- start 2024-03-18T20:40:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.617+0000 7fc63b507b80 1 -- start start 2024-03-18T20:40:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc63b507b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defeca60 0x55d8defece40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc63b507b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defed380 0x55d8deff7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc63b507b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8deff8220 0x55d8deffa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc63b507b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8def46990 con 0x55d8defed380 2024-03-18T20:40:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc63b507b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8def44350 con 0x55d8deff8220 2024-03-18T20:40:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc63b507b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8def3fc40 con 0x55d8defeca60 2024-03-18T20:40:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc62c90d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defed380 0x55d8deff7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc62c90d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defed380 0x55d8deff7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34524/0 (socket says 172.21.15.46:34524) 2024-03-18T20:40:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc62c90d700 1 -- 172.21.15.46:0/500363768 learned_addr learned my addr 172.21.15.46:0/500363768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc62d10e700 1 --2- 172.21.15.46:0/500363768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defeca60 0x55d8defece40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc62d90f700 1 --2- 172.21.15.46:0/500363768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8deff8220 0x55d8deffa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc62c90d700 1 -- 172.21.15.46:0/500363768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defeca60 msgr2=0x55d8defece40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc62c90d700 1 --2- 172.21.15.46:0/500363768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defeca60 0x55d8defece40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc62c90d700 1 -- 172.21.15.46:0/500363768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8deff8220 msgr2=0x55d8deffa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc62c90d700 1 --2- 172.21.15.46:0/500363768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8deff8220 0x55d8deffa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc62c90d700 1 -- 172.21.15.46:0/500363768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8dedc13b0 con 0x55d8defed380 2024-03-18T20:40:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc62c90d700 1 --2- 172.21.15.46:0/500363768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defed380 0x55d8deff7ce0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc624009990 tx=0x7fc62400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52678e13cfa4a8e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.618+0000 7fc6137fe700 1 -- 172.21.15.46:0/500363768 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62401d050 con 0x55d8defed380 2024-03-18T20:40:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.619+0000 7fc6137fe700 1 -- 172.21.15.46:0/500363768 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc62401e040 con 0x55d8defed380 2024-03-18T20:40:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.619+0000 7fc6137fe700 1 -- 172.21.15.46:0/500363768 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6240053b0 con 0x55d8defed380 2024-03-18T20:40:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.619+0000 7fc63b507b80 1 -- 172.21.15.46:0/500363768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defed380 msgr2=0x55d8deff7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.619+0000 7fc63b507b80 1 --2- 172.21.15.46:0/500363768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defed380 0x55d8deff7ce0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc624009990 tx=0x7fc62400b610 comp rx=0 tx=0).stop 2024-03-18T20:40:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.619+0000 7fc63b507b80 1 -- 172.21.15.46:0/500363768 shutdown_connections 2024-03-18T20:40:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.619+0000 7fc63b507b80 1 --2- 172.21.15.46:0/500363768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8deff8220 0x55d8deffa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.619+0000 7fc63b507b80 1 --2- 172.21.15.46:0/500363768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defeca60 0x55d8defece40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.619+0000 7fc63b507b80 1 --2- 172.21.15.46:0/500363768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defed380 0x55d8deff7ce0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.619+0000 7fc63b507b80 1 -- 172.21.15.46:0/500363768 >> 172.21.15.46:0/500363768 conn(0x55d8def63220 msgr2=0x55d8defe9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.619+0000 7fc63b507b80 1 -- 172.21.15.46:0/500363768 shutdown_connections 2024-03-18T20:40:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.619+0000 7fc63b507b80 1 -- 172.21.15.46:0/500363768 wait complete. 2024-03-18T20:40:21.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.620+0000 7fc63b507b80 1 -- start start 2024-03-18T20:40:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.620+0000 7fc63b507b80 1 Processor -- start 2024-03-18T20:40:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.620+0000 7fc63b507b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8defeca60 0x55d8deff7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.620+0000 7fc63b507b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defed380 0x55d8deff4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.620+0000 7fc63b507b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8deff8220 0x55d8deff56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.620+0000 7fc63b507b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8deedbd80 con 0x55d8deff8220 2024-03-18T20:40:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.620+0000 7fc63b507b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8deedba20 con 0x55d8defeca60 2024-03-18T20:40:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.620+0000 7fc63b507b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8dedc36a0 con 0x55d8defed380 2024-03-18T20:40:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.620+0000 7fc63b507b80 10 client.0 _ll_register_callbacks cb 0x55d8ded473a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.620+0000 7fc62d90f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8deff8220 0x55d8deff56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc62d90f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8deff8220 0x55d8deff56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34536/0 (socket says 172.21.15.46:34536) 2024-03-18T20:40:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc62d90f700 1 -- 172.21.15.46:0/1766739773 learned_addr learned my addr 172.21.15.46:0/1766739773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc62c90d700 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defed380 0x55d8deff4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc62d10e700 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8defeca60 0x55d8deff7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc62d90f700 1 -- 172.21.15.46:0/1766739773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defed380 msgr2=0x55d8deff4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc62d90f700 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defed380 0x55d8deff4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc62d90f700 1 -- 172.21.15.46:0/1766739773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8defeca60 msgr2=0x55d8deff7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc62d90f700 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8defeca60 0x55d8deff7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc62d90f700 1 -- 172.21.15.46:0/1766739773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62810d460 con 0x55d8deff8220 2024-03-18T20:40:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc62d90f700 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8deff8220 0x55d8deff56b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc620099950 tx=0x7fc6200993a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3f3117aa669a50 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc610ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc610ff9700 1 -- 172.21.15.46:0/1766739773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62009a0c0 con 0x55d8deff8220 2024-03-18T20:40:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc63b507b80 1 -- 172.21.15.46:0/1766739773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8defe0ad0 con 0x55d8deff8220 2024-03-18T20:40:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc610ff9700 1 -- 172.21.15.46:0/1766739773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc62000ce60 con 0x55d8deff8220 2024-03-18T20:40:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc610ff9700 1 -- 172.21.15.46:0/1766739773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6200999c0 con 0x55d8deff8220 2024-03-18T20:40:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc63b507b80 10 client.17043 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.621+0000 7fc63b507b80 1 -- 172.21.15.46:0/1766739773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d8ded6db60 con 0x55d8deff8220 2024-03-18T20:40:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.622+0000 7fc610ff9700 1 -- 172.21.15.46:0/1766739773 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8defe0ad0 con 0x55d8deff8220 2024-03-18T20:40:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.622+0000 7fc610ff9700 1 -- 172.21.15.46:0/1766739773 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc620098f40 con 0x55d8deff8220 2024-03-18T20:40:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.622+0000 7fc63b507b80 10 client.17043 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.622+0000 7fc63b507b80 10 client.17043 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:40:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.622+0000 7fc63b507b80 20 client.17043 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.622+0000 7fc63b507b80 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55d8df0104a0 0x55d8df0128c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.622+0000 7fc63b507b80 4 client.17043 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.622+0000 7fc63b507b80 1 -- 172.21.15.46:0/1766739773 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8def38700 con 0x55d8df0104a0 2024-03-18T20:40:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.622+0000 7fc62d10e700 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55d8df0104a0 0x55d8df0128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.622+0000 7fc62d10e700 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55d8df0104a0 0x55d8df0128c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=462a111aceeacfba server_cookie=403ed89b3b03364d in_seq=0 out_seq=0 2024-03-18T20:40:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.622+0000 7fc610ff9700 10 client.17043 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:40:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc610ff9700 1 -- 172.21.15.46:0/1766739773 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d8def38700 con 0x55d8df0104a0 2024-03-18T20:40:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc610ff9700 10 client.17043 handle_command_reply: tid=0 2024-03-18T20:40:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc63b507b80 1 client.17043 shutdown 2024-03-18T20:40:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc63b507b80 1 -- 172.21.15.46:0/1766739773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8deff8220 msgr2=0x55d8deff56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc63b507b80 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8deff8220 0x55d8deff56b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc620099950 tx=0x7fc6200993a0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc63b507b80 1 -- 172.21.15.46:0/1766739773 shutdown_connections 2024-03-18T20:40:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc63b507b80 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55d8df0104a0 0x55d8df0128c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc63b507b80 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8deff8220 0x55d8deff56b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc63b507b80 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8defeca60 0x55d8deff7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc610ff9700 0 client.17043 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:40:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc63b507b80 1 --2- 172.21.15.46:0/1766739773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defed380 0x55d8deff4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc63b507b80 1 -- 172.21.15.46:0/1766739773 >> 172.21.15.46:0/1766739773 conn(0x55d8def63220 msgr2=0x55d8defe8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc63b507b80 1 -- 172.21.15.46:0/1766739773 shutdown_connections 2024-03-18T20:40:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.623+0000 7fc63b507b80 1 -- 172.21.15.46:0/1766739773 wait complete. 2024-03-18T20:40:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.624+0000 7fc63b507b80 20 client.17043 trim_cache size 0 max 16384 2024-03-18T20:40:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.637+0000 7fc63b507b80 1 Processor -- start 2024-03-18T20:40:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.637+0000 7fc63b507b80 1 -- start start 2024-03-18T20:40:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc63b507b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 0x55d8df007680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc63b507b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8defed380 0x55d8deff7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc63b507b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8deff8220 0x55d8defe8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc63b507b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8defea230 con 0x55d8defeca60 2024-03-18T20:40:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc63b507b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8defee550 con 0x55d8defed380 2024-03-18T20:40:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc63b507b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8df007bc0 con 0x55d8deff8220 2024-03-18T20:40:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62d10e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 0x55d8df007680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62d10e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 0x55d8df007680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34544/0 (socket says 172.21.15.46:34544) 2024-03-18T20:40:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62d10e700 1 -- 172.21.15.46:0/2363920638 learned_addr learned my addr 172.21.15.46:0/2363920638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62d90f700 1 --2- 172.21.15.46:0/2363920638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8deff8220 0x55d8defe8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62c90d700 1 --2- 172.21.15.46:0/2363920638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8defed380 0x55d8deff7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62d10e700 1 -- 172.21.15.46:0/2363920638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8deff8220 msgr2=0x55d8defe8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62d10e700 1 --2- 172.21.15.46:0/2363920638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8deff8220 0x55d8defe8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62d10e700 1 -- 172.21.15.46:0/2363920638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8defed380 msgr2=0x55d8deff7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62d10e700 1 --2- 172.21.15.46:0/2363920638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8defed380 0x55d8deff7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62d10e700 1 -- 172.21.15.46:0/2363920638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc628109bd0 con 0x55d8defeca60 2024-03-18T20:40:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62d90f700 1 --2- 172.21.15.46:0/2363920638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8deff8220 0x55d8defe8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62c90d700 1 --2- 172.21.15.46:0/2363920638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8defed380 0x55d8deff7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc62d10e700 1 --2- 172.21.15.46:0/2363920638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 0x55d8df007680 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc618011be0 tx=0x7fc618014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69774b0007de626d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc613fff700 1 -- 172.21.15.46:0/2363920638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618024040 con 0x55d8defeca60 2024-03-18T20:40:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc613fff700 1 -- 172.21.15.46:0/2363920638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc618029dc0 con 0x55d8defeca60 2024-03-18T20:40:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.638+0000 7fc613fff700 1 -- 172.21.15.46:0/2363920638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61801e8f0 con 0x55d8defeca60 2024-03-18T20:40:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.639+0000 7fc63b507b80 1 -- 172.21.15.46:0/2363920638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 msgr2=0x55d8df007680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.639+0000 7fc63b507b80 1 --2- 172.21.15.46:0/2363920638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 0x55d8df007680 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc618011be0 tx=0x7fc618014000 comp rx=0 tx=0).stop 2024-03-18T20:40:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.639+0000 7fc63b507b80 1 -- 172.21.15.46:0/2363920638 shutdown_connections 2024-03-18T20:40:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.639+0000 7fc63b507b80 1 --2- 172.21.15.46:0/2363920638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 0x55d8df007680 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.639+0000 7fc63b507b80 1 --2- 172.21.15.46:0/2363920638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8deff8220 0x55d8defe8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.639+0000 7fc63b507b80 1 --2- 172.21.15.46:0/2363920638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8defed380 0x55d8deff7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.639+0000 7fc63b507b80 1 -- 172.21.15.46:0/2363920638 >> 172.21.15.46:0/2363920638 conn(0x55d8def63220 msgr2=0x55d8defe7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.639+0000 7fc63b507b80 1 -- 172.21.15.46:0/2363920638 shutdown_connections 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.639+0000 7fc63b507b80 1 -- 172.21.15.46:0/2363920638 wait complete. 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.639+0000 7fc63b507b80 1 -- start start 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc63b507b80 1 Processor -- start 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc63b507b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 0x55d8df005c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc63b507b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defed380 0x55d8df0061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc63b507b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8deff8220 0x55d8df0066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc63b507b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8df007a10 con 0x55d8defeca60 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc63b507b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8df0642e0 con 0x55d8deff8220 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc63b507b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8df064470 con 0x55d8defed380 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc63b507b80 10 client.0 _ll_register_callbacks cb 0x7fc620099500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc62d10e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 0x55d8df005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc62d10e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 0x55d8df005c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34558/0 (socket says 172.21.15.46:34558) 2024-03-18T20:40:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc62d10e700 1 -- 172.21.15.46:0/3738777622 learned_addr learned my addr 172.21.15.46:0/3738777622 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc62c90d700 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defed380 0x55d8df0061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc62d90f700 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8deff8220 0x55d8df0066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc62d10e700 1 -- 172.21.15.46:0/3738777622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defed380 msgr2=0x55d8df0061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc62d10e700 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defed380 0x55d8df0061a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc62d10e700 1 -- 172.21.15.46:0/3738777622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8deff8220 msgr2=0x55d8df0066e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.640+0000 7fc62d10e700 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8deff8220 0x55d8df0066e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc62d10e700 1 -- 172.21.15.46:0/3738777622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc618017070 con 0x55d8defeca60 2024-03-18T20:40:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc62d10e700 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 0x55d8df005c60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc618005cb0 tx=0x7fc618003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbba5dd52200562 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc610ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc610ff9700 1 -- 172.21.15.46:0/3738777622 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618024070 con 0x55d8defeca60 2024-03-18T20:40:21.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc610ff9700 1 -- 172.21.15.46:0/3738777622 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc618029570 con 0x55d8defeca60 2024-03-18T20:40:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc63b507b80 1 -- 172.21.15.46:0/3738777622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc62401e040 con 0x55d8defeca60 2024-03-18T20:40:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc610ff9700 1 -- 172.21.15.46:0/3738777622 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc618005050 con 0x55d8defeca60 2024-03-18T20:40:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc63b507b80 10 client.17049 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc63b507b80 1 -- 172.21.15.46:0/3738777622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d8def05ea0 con 0x55d8defeca60 2024-03-18T20:40:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc610ff9700 1 -- 172.21.15.46:0/3738777622 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc62401e040 con 0x55d8defeca60 2024-03-18T20:40:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc610ff9700 1 -- 172.21.15.46:0/3738777622 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc61801e530 con 0x55d8defeca60 2024-03-18T20:40:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc63b507b80 10 client.17049 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc63b507b80 10 client.17049 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:40:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc63b507b80 20 client.17049 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.641+0000 7fc63b507b80 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55d8df065c80 0x55d8df0680a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc63b507b80 4 client.17049 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:40:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc63b507b80 1 -- 172.21.15.46:0/3738777622 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d8ded8ecf0 con 0x55d8df065c80 2024-03-18T20:40:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc62c90d700 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55d8df065c80 0x55d8df0680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc610ff9700 10 client.17049 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:40:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc62c90d700 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55d8df065c80 0x55d8df0680a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1716ecdfe7d77216 server_cookie=97956c214788cd37 in_seq=0 out_seq=0 2024-03-18T20:40:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc610ff9700 1 -- 172.21.15.46:0/3738777622 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d8ded8ecf0 con 0x55d8df065c80 2024-03-18T20:40:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc610ff9700 10 client.17049 handle_command_reply: tid=0 2024-03-18T20:40:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc63b507b80 1 client.17049 shutdown 2024-03-18T20:40:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc63b507b80 1 -- 172.21.15.46:0/3738777622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 msgr2=0x55d8df005c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc63b507b80 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 0x55d8df005c60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc618005cb0 tx=0x7fc618003eb0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc63b507b80 1 -- 172.21.15.46:0/3738777622 shutdown_connections 2024-03-18T20:40:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc63b507b80 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55d8df065c80 0x55d8df0680a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc63b507b80 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d8defeca60 0x55d8df005c60 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc610ff9700 0 client.17049 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:40:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc63b507b80 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d8deff8220 0x55d8df0066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc63b507b80 1 --2- 172.21.15.46:0/3738777622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d8defed380 0x55d8df0061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.642+0000 7fc63b507b80 1 -- 172.21.15.46:0/3738777622 >> 172.21.15.46:0/3738777622 conn(0x55d8def63220 msgr2=0x55d8deff36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc63b507b80 1 -- 172.21.15.46:0/3738777622 shutdown_connections 2024-03-18T20:40:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc63b507b80 1 -- 172.21.15.46:0/3738777622 wait complete. 2024-03-18T20:40:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc63b507b80 20 client.17049 trim_cache size 0 max 16384 2024-03-18T20:40:21.657 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:40:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc62f372700 1 -- 172.21.15.46:0/2480382881 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc6140a1290 msgr2=0x7fc6140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc62f372700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc6140a1290 0x7fc6140a3750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc618011490 tx=0x7fc618009260 comp rx=0 tx=0).stop 2024-03-18T20:40:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc62f372700 1 -- 172.21.15.46:0/2480382881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc628104bf0 msgr2=0x7fc62819dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc62f372700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc628104bf0 0x7fc62819dcf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc62000ef30 tx=0x7fc62000c3a0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc62f372700 1 -- 172.21.15.46:0/2480382881 shutdown_connections 2024-03-18T20:40:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc62f372700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc6140a1290 0x7fc6140a3750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc62f372700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc628104bf0 0x7fc62819dcf0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc62f372700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc628100390 0x7fc628199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc62f372700 1 --2- 172.21.15.46:0/2480382881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc628100d50 0x7fc628199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc62f372700 1 -- 172.21.15.46:0/2480382881 >> 172.21.15.46:0/2480382881 conn(0x7fc6280761b0 msgr2=0x7fc6280ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc62f372700 1 -- 172.21.15.46:0/2480382881 shutdown_connections 2024-03-18T20:40:21.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.643+0000 7fc62f372700 1 -- 172.21.15.46:0/2480382881 wait complete. 2024-03-18T20:40:21.660 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:40:21.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:40:21.684 INFO:tasks.workunit.client.0.smithi046.stdout: 1 107173 1.76 MB/sec execute 76 sec latency 2851.511 ms 2024-03-18T20:40:21.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.796+0000 7f57b8c18700 1 -- 172.21.15.46:0/3425335875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b4105cf0 msgr2=0x7f57b41060d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.796+0000 7f57b8c18700 1 --2- 172.21.15.46:0/3425335875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b4105cf0 0x7f57b41060d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009990 tx=0x7f57a4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.796+0000 7f57b8c18700 1 -- 172.21.15.46:0/3425335875 shutdown_connections 2024-03-18T20:40:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.796+0000 7f57b8c18700 1 --2- 172.21.15.46:0/3425335875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b4105cf0 0x7f57b41060d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.796+0000 7f57b8c18700 1 --2- 172.21.15.46:0/3425335875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57b4069cd0 0x7f57b410db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.796+0000 7f57b8c18700 1 --2- 172.21.15.46:0/3425335875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57b4069310 0x7f57b4069790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.796+0000 7f57b8c18700 1 -- 172.21.15.46:0/3425335875 >> 172.21.15.46:0/3425335875 conn(0x7f57b4076d20 msgr2=0x7f57b4077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.796+0000 7f57b8c18700 1 -- 172.21.15.46:0/3425335875 shutdown_connections 2024-03-18T20:40:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.796+0000 7f57b8c18700 1 -- 172.21.15.46:0/3425335875 wait complete. 2024-03-18T20:40:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.797+0000 7f57b8c18700 1 Processor -- start 2024-03-18T20:40:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.797+0000 7f57b8c18700 1 -- start start 2024-03-18T20:40:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.797+0000 7f57b8c18700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b4069310 0x7f57b4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b8c18700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57b4069cd0 0x7f57b4197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b8c18700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57b4105cf0 0x7f57b419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b8c18700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419c0d0 con 0x7f57b4069310 2024-03-18T20:40:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b8c18700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419c240 con 0x7f57b4069cd0 2024-03-18T20:40:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b8c18700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419c400 con 0x7f57b4105cf0 2024-03-18T20:40:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b4069310 0x7f57b4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b4069310 0x7f57b4197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34580/0 (socket says 172.21.15.46:34580) 2024-03-18T20:40:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b259c700 1 -- 172.21.15.46:0/2981388472 learned_addr learned my addr 172.21.15.46:0/2981388472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57b4069cd0 0x7f57b4197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b2d9d700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57b4105cf0 0x7f57b419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b259c700 1 -- 172.21.15.46:0/2981388472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57b4105cf0 msgr2=0x7f57b419ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b259c700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57b4105cf0 0x7f57b419ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b259c700 1 -- 172.21.15.46:0/2981388472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57b4069cd0 msgr2=0x7f57b4197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b259c700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57b4069cd0 0x7f57b4197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b259c700 1 -- 172.21.15.46:0/2981388472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a40095f0 con 0x7f57b4069310 2024-03-18T20:40:21.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57b259c700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b4069310 0x7f57b4197200 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f57a4000c00 tx=0x7f57a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.798+0000 7f57abfff700 1 -- 172.21.15.46:0/2981388472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a401d070 con 0x7f57b4069310 2024-03-18T20:40:21.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.799+0000 7f57b8c18700 1 -- 172.21.15.46:0/2981388472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b419c680 con 0x7f57b4069310 2024-03-18T20:40:21.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.799+0000 7f57abfff700 1 -- 172.21.15.46:0/2981388472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57a4022470 con 0x7f57b4069310 2024-03-18T20:40:21.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.799+0000 7f57abfff700 1 -- 172.21.15.46:0/2981388472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4017650 con 0x7f57b4069310 2024-03-18T20:40:21.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.799+0000 7f57b8c18700 1 -- 172.21.15.46:0/2981388472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b41a3f20 con 0x7f57b4069310 2024-03-18T20:40:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.801+0000 7f57abfff700 1 -- 172.21.15.46:0/2981388472 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f57a40225e0 con 0x7f57b4069310 2024-03-18T20:40:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.801+0000 7f57abfff700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57a00a1290 0x7f57a00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.802+0000 7f57abfff700 1 -- 172.21.15.46:0/2981388472 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f57a40cdf30 con 0x7f57b4069310 2024-03-18T20:40:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.802+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57a00a1290 0x7f57a00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.802+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57a00a1290 0x7f57a00a3750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f57b4198700 tx=0x7f579c00b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.802+0000 7f57c4dadb80 1 Processor -- start 2024-03-18T20:40:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57c4dadb80 1 -- start start 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57c4dadb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada4a60 0x556a0ada4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57c4dadb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada5380 0x556a0adafce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57c4dadb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0adb0220 0x556a0adb2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57c4dadb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a0acfe990 con 0x556a0ada5380 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57c4dadb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a0acfc350 con 0x556a0ada4a60 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57c4dadb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a0acf7c40 con 0x556a0adb0220 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57a9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada5380 0x556a0adafce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57b259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada4a60 0x556a0ada4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57a9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada5380 0x556a0adafce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34596/0 (socket says 172.21.15.46:34596) 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57a9d9b700 1 -- 172.21.15.46:0/3112184510 learned_addr learned my addr 172.21.15.46:0/3112184510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57b2d9d700 1 --2- 172.21.15.46:0/3112184510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0adb0220 0x556a0adb2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57a9d9b700 1 -- 172.21.15.46:0/3112184510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0adb0220 msgr2=0x556a0adb2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/3112184510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0adb0220 0x556a0adb2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57a9d9b700 1 -- 172.21.15.46:0/3112184510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada4a60 msgr2=0x556a0ada4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/3112184510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada4a60 0x556a0ada4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57a9d9b700 1 -- 172.21.15.46:0/3112184510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a0ab793b0 con 0x556a0ada5380 2024-03-18T20:40:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57b259c700 1 --2- 172.21.15.46:0/3112184510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada4a60 0x556a0ada4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.803+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/3112184510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada5380 0x556a0adafce0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f579c010f70 tx=0x7f579c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f668fac072accc09 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.804+0000 7f57a8d99700 1 -- 172.21.15.46:0/3112184510 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c011db0 con 0x556a0ada5380 2024-03-18T20:40:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.804+0000 7f57a8d99700 1 -- 172.21.15.46:0/3112184510 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f579c015070 con 0x556a0ada5380 2024-03-18T20:40:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.804+0000 7f57a8d99700 1 -- 172.21.15.46:0/3112184510 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c01c610 con 0x556a0ada5380 2024-03-18T20:40:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.804+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3112184510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada5380 msgr2=0x556a0adafce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.804+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3112184510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada5380 0x556a0adafce0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f579c010f70 tx=0x7f579c013000 comp rx=0 tx=0).stop 2024-03-18T20:40:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3112184510 shutdown_connections 2024-03-18T20:40:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3112184510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0adb0220 0x556a0adb2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3112184510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada4a60 0x556a0ada4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3112184510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada5380 0x556a0adafce0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3112184510 >> 172.21.15.46:0/3112184510 conn(0x556a0ad1b220 msgr2=0x556a0ada1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3112184510 shutdown_connections 2024-03-18T20:40:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3112184510 wait complete. 2024-03-18T20:40:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 -- start start 2024-03-18T20:40:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 Processor -- start 2024-03-18T20:40:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 0x556a0adafbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada5380 0x556a0adacf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0adb0220 0x556a0adad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a0ac93d80 con 0x556a0ada4a60 2024-03-18T20:40:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a0ac93a20 con 0x556a0ada5380 2024-03-18T20:40:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a0ab7b6a0 con 0x556a0adb0220 2024-03-18T20:40:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.805+0000 7f57c4dadb80 10 client.0 _ll_register_callbacks cb 0x556a0aaff3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57b259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 0x556a0adafbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57b259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 0x556a0adafbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34604/0 (socket says 172.21.15.46:34604) 2024-03-18T20:40:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57b259c700 1 -- 172.21.15.46:0/160764449 learned_addr learned my addr 172.21.15.46:0/160764449 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada5380 0x556a0adacf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57b2d9d700 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0adb0220 0x556a0adad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57b259c700 1 -- 172.21.15.46:0/160764449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0adb0220 msgr2=0x556a0adad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57b259c700 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0adb0220 0x556a0adad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57b259c700 1 -- 172.21.15.46:0/160764449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada5380 msgr2=0x556a0adacf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57b259c700 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada5380 0x556a0adacf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57b259c700 1 -- 172.21.15.46:0/160764449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b41a3f20 con 0x556a0ada4a60 2024-03-18T20:40:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada5380 0x556a0adacf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57b259c700 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 0x556a0adafbf0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f57a40975a0 tx=0x7f57a409b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4575104d7b2facbf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f5799ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f5799ffb700 1 -- 172.21.15.46:0/160764449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a409eed0 con 0x556a0ada4a60 2024-03-18T20:40:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57c4dadb80 1 -- 172.21.15.46:0/160764449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a0ad98ad0 con 0x556a0ada4a60 2024-03-18T20:40:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f5799ffb700 1 -- 172.21.15.46:0/160764449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f57a40a3490 con 0x556a0ada4a60 2024-03-18T20:40:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f5799ffb700 1 -- 172.21.15.46:0/160764449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57a40b9170 con 0x556a0ada4a60 2024-03-18T20:40:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57c4dadb80 10 client.17061 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.806+0000 7f57c4dadb80 1 -- 172.21.15.46:0/160764449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f579c010c20 con 0x556a0ada4a60 2024-03-18T20:40:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.807+0000 7f5799ffb700 1 -- 172.21.15.46:0/160764449 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a0ad98ad0 con 0x556a0ada4a60 2024-03-18T20:40:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.807+0000 7f5799ffb700 1 -- 172.21.15.46:0/160764449 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f57a409b390 con 0x556a0ada4a60 2024-03-18T20:40:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.807+0000 7f57c4dadb80 10 client.17061 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.807+0000 7f57c4dadb80 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.807+0000 7f57c4dadb80 20 client.17061 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.807+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a0adbf970 0x556a0adc1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.807+0000 7f57c4dadb80 4 client.17061 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.807+0000 7f57c4dadb80 1 -- 172.21.15.46:0/160764449 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a0acf0700 con 0x556a0adbf970 2024-03-18T20:40:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.807+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a0adbf970 0x556a0adc1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.808+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a0adbf970 0x556a0adc1d60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a533ecbc532958fd server_cookie=35439df67f6186b0 in_seq=0 out_seq=0 2024-03-18T20:40:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.808+0000 7f5799ffb700 10 client.17061 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.808+0000 7f5799ffb700 1 -- 172.21.15.46:0/160764449 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556a0acf0700 con 0x556a0adbf970 2024-03-18T20:40:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.808+0000 7f5799ffb700 10 client.17061 handle_command_reply: tid=0 2024-03-18T20:40:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.808+0000 7f57c4dadb80 1 client.17061 shutdown 2024-03-18T20:40:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.809+0000 7f57c4dadb80 1 -- 172.21.15.46:0/160764449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 msgr2=0x556a0adafbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.809+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 0x556a0adafbf0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f57a40975a0 tx=0x7f57a409b7a0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.809+0000 7f57c4dadb80 1 -- 172.21.15.46:0/160764449 shutdown_connections 2024-03-18T20:40:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.809+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a0adbf970 0x556a0adc1d60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.809+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 0x556a0adafbf0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.809+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0adb0220 0x556a0adad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.809+0000 7f5799ffb700 0 client.17061 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.809+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/160764449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada5380 0x556a0adacf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.809+0000 7f57c4dadb80 1 -- 172.21.15.46:0/160764449 >> 172.21.15.46:0/160764449 conn(0x556a0ad1b220 msgr2=0x556a0ada0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.809+0000 7f57c4dadb80 1 -- 172.21.15.46:0/160764449 shutdown_connections 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.809+0000 7f57c4dadb80 1 -- 172.21.15.46:0/160764449 wait complete. 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.809+0000 7f57c4dadb80 20 client.17061 trim_cache size 0 max 16384 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57c4dadb80 1 Processor -- start 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57c4dadb80 1 -- start start 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57c4dadb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0ada4a60 0x556a0adc1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57c4dadb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada5380 0x556a0ada79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57c4dadb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0adb0220 0x556a0adb6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57c4dadb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a0ad9d5e0 con 0x556a0adb0220 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57c4dadb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a0ada5d30 con 0x556a0ada5380 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57c4dadb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a0ada51c0 con 0x556a0ada4a60 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57b2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0adb0220 0x556a0adb6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57b2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0adb0220 0x556a0adb6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34616/0 (socket says 172.21.15.46:34616) 2024-03-18T20:40:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57b2d9d700 1 -- 172.21.15.46:0/3711063855 learned_addr learned my addr 172.21.15.46:0/3711063855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57b259c700 1 --2- 172.21.15.46:0/3711063855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0ada4a60 0x556a0adc1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57b2d9d700 1 -- 172.21.15.46:0/3711063855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0ada4a60 msgr2=0x556a0adc1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57b2d9d700 1 --2- 172.21.15.46:0/3711063855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0ada4a60 0x556a0adc1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/3711063855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada5380 0x556a0ada79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.823+0000 7f57b2d9d700 1 -- 172.21.15.46:0/3711063855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada5380 msgr2=0x556a0ada79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.824+0000 7f57b2d9d700 1 --2- 172.21.15.46:0/3711063855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada5380 0x556a0ada79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.824+0000 7f57b2d9d700 1 -- 172.21.15.46:0/3711063855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a0ab25b60 con 0x556a0adb0220 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.824+0000 7f57b259c700 1 --2- 172.21.15.46:0/3711063855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0ada4a60 0x556a0adc1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.824+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/3711063855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada5380 0x556a0ada79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.824+0000 7f57b2d9d700 1 --2- 172.21.15.46:0/3711063855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0adb0220 0x556a0adb6e50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009790 tx=0x7f57ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f52f29ce039266 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.824+0000 7f57a959a700 1 -- 172.21.15.46:0/3711063855 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac00e040 con 0x556a0adb0220 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.824+0000 7f57a959a700 1 -- 172.21.15.46:0/3711063855 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57ac00f040 con 0x556a0adb0220 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.824+0000 7f57a959a700 1 -- 172.21.15.46:0/3711063855 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac01ea10 con 0x556a0adb0220 2024-03-18T20:40:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.824+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3711063855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0adb0220 msgr2=0x556a0adb6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.824+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3711063855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0adb0220 0x556a0adb6e50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009790 tx=0x7f57ac00b040 comp rx=0 tx=0).stop 2024-03-18T20:40:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3711063855 shutdown_connections 2024-03-18T20:40:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3711063855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0adb0220 0x556a0adb6e50 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3711063855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0ada4a60 0x556a0adc1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3711063855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0ada5380 0x556a0ada79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3711063855 >> 172.21.15.46:0/3711063855 conn(0x556a0ad1b220 msgr2=0x556a0adac4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3711063855 shutdown_connections 2024-03-18T20:40:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3711063855 wait complete. 2024-03-18T20:40:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 -- start start 2024-03-18T20:40:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 Processor -- start 2024-03-18T20:40:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 0x556a0adb4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0ada5380 0x556a0adb4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0adb0220 0x556a0ae16b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a0adb53b0 con 0x556a0ada4a60 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a0adb5520 con 0x556a0adb0220 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a0ae17060 con 0x556a0ada5380 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57c4dadb80 10 client.0 _ll_register_callbacks cb 0x556a0aaff3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.825+0000 7f57b259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 0x556a0adb4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57b259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 0x556a0adb4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34622/0 (socket says 172.21.15.46:34622) 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57b259c700 1 -- 172.21.15.46:0/3766540117 learned_addr learned my addr 172.21.15.46:0/3766540117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57b2d9d700 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0adb0220 0x556a0ae16b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0ada5380 0x556a0adb4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57b259c700 1 -- 172.21.15.46:0/3766540117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0ada5380 msgr2=0x556a0adb4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57b259c700 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0ada5380 0x556a0adb4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57b259c700 1 -- 172.21.15.46:0/3766540117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0adb0220 msgr2=0x556a0ae16b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57b259c700 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0adb0220 0x556a0ae16b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57b259c700 1 -- 172.21.15.46:0/3766540117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ac0092a0 con 0x556a0ada4a60 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57b259c700 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 0x556a0adb4700 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f57a4006010 tx=0x7f57a40a12e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566ac4df9d9ca658 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f5799ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f5799ffb700 1 -- 172.21.15.46:0/3766540117 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a409a840 con 0x556a0ada4a60 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f5799ffb700 1 -- 172.21.15.46:0/3766540117 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f57a40a3490 con 0x556a0ada4a60 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3766540117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f579c015070 con 0x556a0ada4a60 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f5799ffb700 1 -- 172.21.15.46:0/3766540117 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57a409a630 con 0x556a0ada4a60 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57c4dadb80 10 client.17067 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.826+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3766540117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x556a0acbdea0 con 0x556a0ada4a60 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.827+0000 7f5799ffb700 1 -- 172.21.15.46:0/3766540117 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57a4083070 con 0x556a0ada4a60 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.827+0000 7f5799ffb700 1 -- 172.21.15.46:0/3766540117 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f57a40a3070 con 0x556a0ada4a60 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.827+0000 7f57c4dadb80 10 client.17067 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.827+0000 7f57c4dadb80 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.827+0000 7f57c4dadb80 20 client.17067 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.827+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a0ae18400 0x556a0ae1a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.827+0000 7f57c4dadb80 4 client.17067 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.827+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3766540117 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a0ab46cf0 con 0x556a0ae18400 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.827+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a0ae18400 0x556a0ae1a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.827+0000 7f5799ffb700 10 client.17067 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.827+0000 7f57a9d9b700 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a0ae18400 0x556a0ae1a820 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d8aa904ded10a9 server_cookie=389ed5dca55aa643 in_seq=0 out_seq=0 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f5799ffb700 1 -- 172.21.15.46:0/3766540117 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a0ab46cf0 con 0x556a0ae18400 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f5799ffb700 10 client.17067 handle_command_reply: tid=0 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57c4dadb80 1 client.17067 shutdown 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3766540117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 msgr2=0x556a0adb4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 0x556a0adb4700 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f57a4006010 tx=0x7f57a40a12e0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3766540117 shutdown_connections 2024-03-18T20:40:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a0ae18400 0x556a0ae1a820 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a0ada4a60 0x556a0adb4700 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a0adb0220 0x556a0ae16b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f5799ffb700 0 client.17067 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57c4dadb80 1 --2- 172.21.15.46:0/3766540117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a0ada5380 0x556a0adb4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3766540117 >> 172.21.15.46:0/3766540117 conn(0x556a0ad1b220 msgr2=0x556a0adac4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3766540117 shutdown_connections 2024-03-18T20:40:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57c4dadb80 1 -- 172.21.15.46:0/3766540117 wait complete. 2024-03-18T20:40:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57c4dadb80 20 client.17067 trim_cache size 0 max 16384 2024-03-18T20:40:21.858 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:40:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57b8c18700 1 -- 172.21.15.46:0/2981388472 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57a00a1290 msgr2=0x7f57a00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57b8c18700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57a00a1290 0x7f57a00a3750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f57b4198700 tx=0x7f579c00b410 comp rx=0 tx=0).stop 2024-03-18T20:40:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57b8c18700 1 -- 172.21.15.46:0/2981388472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b4069310 msgr2=0x7f57b4197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.828+0000 7f57b8c18700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b4069310 0x7f57b4197200 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f57a4000c00 tx=0x7f57a4004730 comp rx=0 tx=0).stop 2024-03-18T20:40:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.829+0000 7f57b8c18700 1 -- 172.21.15.46:0/2981388472 shutdown_connections 2024-03-18T20:40:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.829+0000 7f57b8c18700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57a00a1290 0x7f57a00a3750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.829+0000 7f57b8c18700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57b4069310 0x7f57b4197200 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.829+0000 7f57b8c18700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57b4105cf0 0x7f57b419ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.829+0000 7f57b8c18700 1 --2- 172.21.15.46:0/2981388472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57b4069cd0 0x7f57b4197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.829+0000 7f57b8c18700 1 -- 172.21.15.46:0/2981388472 >> 172.21.15.46:0/2981388472 conn(0x7f57b4076d20 msgr2=0x7f57b40fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.829+0000 7f57b8c18700 1 -- 172.21.15.46:0/2981388472 shutdown_connections 2024-03-18T20:40:21.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:21.829+0000 7f57b8c18700 1 -- 172.21.15.46:0/2981388472 wait complete. 2024-03-18T20:40:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:22 smithi073 ceph-mon[47384]: pgmap v131: 97 pgs: 97 active+clean; 47 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 417 KiB/s wr, 70 op/s 2024-03-18T20:40:22.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:22 smithi155 ceph-mon[48344]: pgmap v131: 97 pgs: 97 active+clean; 47 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 417 KiB/s wr, 70 op/s 2024-03-18T20:40:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:22 smithi046 ceph-mon[43495]: pgmap v131: 97 pgs: 97 active+clean; 47 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 417 KiB/s wr, 70 op/s 2024-03-18T20:40:22.684 INFO:tasks.workunit.client.0.smithi046.stdout: 1 107241 1.74 MB/sec execute 77 sec latency 3207.667 ms 2024-03-18T20:40:22.861 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:40:22.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:40:22.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:40:23.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.010+0000 7f92f2260700 1 -- 172.21.15.46:0/3190810582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec106090 msgr2=0x7f92ec10dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.010+0000 7f92f2260700 1 --2- 172.21.15.46:0/3190810582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec106090 0x7f92ec10dc10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f92e400b0a0 tx=0x7f92e400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.010+0000 7f92f2260700 1 -- 172.21.15.46:0/3190810582 shutdown_connections 2024-03-18T20:40:23.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.010+0000 7f92f2260700 1 --2- 172.21.15.46:0/3190810582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec106090 0x7f92ec10dc10 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.010+0000 7f92f2260700 1 --2- 172.21.15.46:0/3190810582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92ec100f70 0x7f92ec101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.010+0000 7f92f2260700 1 --2- 172.21.15.46:0/3190810582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92ec101920 0x7f92ec105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.010+0000 7f92f2260700 1 -- 172.21.15.46:0/3190810582 >> 172.21.15.46:0/3190810582 conn(0x7f92ec0fc7f0 msgr2=0x7f92ec0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:23.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.010+0000 7f92f2260700 1 -- 172.21.15.46:0/3190810582 shutdown_connections 2024-03-18T20:40:23.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.010+0000 7f92f2260700 1 -- 172.21.15.46:0/3190810582 wait complete. 2024-03-18T20:40:23.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.011+0000 7f92f2260700 1 Processor -- start 2024-03-18T20:40:23.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.011+0000 7f92f2260700 1 -- start start 2024-03-18T20:40:23.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92f2260700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92ec100f70 0x7f92ec0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92f2260700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92ec106090 0x7f92ec0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92f2260700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec073b30 0x7f92ec1a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92f2260700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec1a5ac0 con 0x7f92ec073b30 2024-03-18T20:40:23.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92f2260700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec1a5c30 con 0x7f92ec100f70 2024-03-18T20:40:23.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92f2260700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec1a5df0 con 0x7f92ec106090 2024-03-18T20:40:23.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec073b30 0x7f92ec1a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92ec106090 0x7f92ec0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec073b30 0x7f92ec1a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34642/0 (socket says 172.21.15.46:34642) 2024-03-18T20:40:23.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92ec106090 0x7f92ec0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59020/0 (socket says 172.21.15.46:59020) 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92ebfff700 1 -- 172.21.15.46:0/1735526769 learned_addr learned my addr 172.21.15.46:0/1735526769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92ec100f70 0x7f92ec0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92ebfff700 1 -- 172.21.15.46:0/1735526769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92ec106090 msgr2=0x7f92ec0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92ebfff700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92ec106090 0x7f92ec0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92ebfff700 1 -- 172.21.15.46:0/1735526769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92ec100f70 msgr2=0x7f92ec0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92ebfff700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92ec100f70 0x7f92ec0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92ebfff700 1 -- 172.21.15.46:0/1735526769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e4009d00 con 0x7f92ec073b30 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92ec100f70 0x7f92ec0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.012+0000 7f92ebfff700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec073b30 0x7f92ec1a54f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f92e400b070 tx=0x7f92e400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.013+0000 7f92e8ff9700 1 -- 172.21.15.46:0/1735526769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e400e040 con 0x7f92ec073b30 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.013+0000 7f92f2260700 1 -- 172.21.15.46:0/1735526769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ec1a6070 con 0x7f92ec073b30 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.013+0000 7f92e8ff9700 1 -- 172.21.15.46:0/1735526769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92e401d480 con 0x7f92ec073b30 2024-03-18T20:40:23.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.013+0000 7f92e8ff9700 1 -- 172.21.15.46:0/1735526769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40052a0 con 0x7f92ec073b30 2024-03-18T20:40:23.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.013+0000 7f92f2260700 1 -- 172.21.15.46:0/1735526769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec1a64f0 con 0x7f92ec073b30 2024-03-18T20:40:23.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.015+0000 7f92e8ff9700 1 -- 172.21.15.46:0/1735526769 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f92e401d5f0 con 0x7f92ec073b30 2024-03-18T20:40:23.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.016+0000 7f92e8ff9700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f92d40a1290 0x7f92d40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.016+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f92d40a1290 0x7f92d40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.016+0000 7f92e8ff9700 1 -- 172.21.15.46:0/1735526769 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f92e40cda60 con 0x7f92ec073b30 2024-03-18T20:40:23.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.016+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f92d40a1290 0x7f92d40a3750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f92dc011490 tx=0x7f92dc009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:23.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92fe3f5b80 1 Processor -- start 2024-03-18T20:40:23.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92fe3f5b80 1 -- start start 2024-03-18T20:40:23.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92fe3f5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecb6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92fe3f5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb7400 0x55ae1ecc1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92fe3f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecc4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92fe3f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1ec10a10 con 0x55ae1ecc22a0 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92fe3f5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1ec0e3d0 con 0x55ae1ecb6ae0 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92fe3f5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1ec09cc0 con 0x55ae1ecb7400 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecc4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecc4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34656/0 (socket says 172.21.15.46:34656) 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92eb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecb6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb7400 0x55ae1ecc1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.017+0000 7f92ebfff700 1 -- 172.21.15.46:0/2608797578 learned_addr learned my addr 172.21.15.46:0/2608797578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.018+0000 7f92eaffd700 1 -- 172.21.15.46:0/2608797578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb6ae0 msgr2=0x55ae1ecb6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.018+0000 7f92eaffd700 1 --2- 172.21.15.46:0/2608797578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecb6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.018+0000 7f92eaffd700 1 -- 172.21.15.46:0/2608797578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 msgr2=0x55ae1ecc4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.018+0000 7f92eaffd700 1 --2- 172.21.15.46:0/2608797578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecc4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.018+0000 7f92eaffd700 1 -- 172.21.15.46:0/2608797578 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae1ea8b430 con 0x55ae1ecb7400 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.018+0000 7f92ebfff700 1 --2- 172.21.15.46:0/2608797578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecc4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.018+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/2608797578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecb6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.018+0000 7f92eaffd700 1 --2- 172.21.15.46:0/2608797578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb7400 0x55ae1ecc1d60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f92ec06a3d0 tx=0x7f92e0009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb675493ec2e4e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.018+0000 7f92d1ffb700 1 -- 172.21.15.46:0/2608797578 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0011c80 con 0x55ae1ecb7400 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.018+0000 7f92d1ffb700 1 -- 172.21.15.46:0/2608797578 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92e0004510 con 0x55ae1ecb7400 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.018+0000 7f92d1ffb700 1 -- 172.21.15.46:0/2608797578 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0005280 con 0x55ae1ecb7400 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.019+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/2608797578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb7400 msgr2=0x55ae1ecc1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.019+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/2608797578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb7400 0x55ae1ecc1d60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f92ec06a3d0 tx=0x7f92e0009f60 comp rx=0 tx=0).stop 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.019+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/2608797578 shutdown_connections 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.019+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/2608797578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecc4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.019+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/2608797578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecb6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.019+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/2608797578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb7400 0x55ae1ecc1d60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.019+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/2608797578 >> 172.21.15.46:0/2608797578 conn(0x55ae1ec2d2a0 msgr2=0x55ae1ecb30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:23.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.019+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/2608797578 shutdown_connections 2024-03-18T20:40:23.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.019+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/2608797578 wait complete. 2024-03-18T20:40:23.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.019+0000 7f92fe3f5b80 1 -- start start 2024-03-18T20:40:23.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92fe3f5b80 1 Processor -- start 2024-03-18T20:40:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92fe3f5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92fe3f5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb7400 0x55ae1ecbf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92fe3f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecbf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92fe3f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1eba5e00 con 0x55ae1ecc22a0 2024-03-18T20:40:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92fe3f5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1eba5aa0 con 0x55ae1ecb7400 2024-03-18T20:40:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92fe3f5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1ea8d720 con 0x55ae1ecb6ae0 2024-03-18T20:40:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92fe3f5b80 10 client.0 _ll_register_callbacks cb 0x55ae1ea11420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecbf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecbf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34666/0 (socket says 172.21.15.46:34666) 2024-03-18T20:40:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92ebfff700 1 -- 172.21.15.46:0/3052583056 learned_addr learned my addr 172.21.15.46:0/3052583056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:23.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.020+0000 7f92eaffd700 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb7400 0x55ae1ecbf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92ebfff700 1 -- 172.21.15.46:0/3052583056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 msgr2=0x55ae1ecc1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92ebfff700 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92ebfff700 1 -- 172.21.15.46:0/3052583056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb7400 msgr2=0x55ae1ecbf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92ebfff700 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb7400 0x55ae1ecbf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92ebfff700 1 -- 172.21.15.46:0/3052583056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec1a64f0 con 0x55ae1ecc22a0 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92ebfff700 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecbf770 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f92e409a8d0 tx=0x7f92e409a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3df4167dd0c2af server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92caffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92caffd700 1 -- 172.21.15.46:0/3052583056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40ba040 con 0x55ae1ecc22a0 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92caffd700 1 -- 172.21.15.46:0/3052583056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f92e409abd0 con 0x55ae1ecc22a0 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3052583056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae1ecaab50 con 0x55ae1ecc22a0 2024-03-18T20:40:23.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92caffd700 1 -- 172.21.15.46:0/3052583056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92e401dae0 con 0x55ae1ecc22a0 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92fe3f5b80 10 client.17079 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3052583056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f92e00098e0 con 0x55ae1ecc22a0 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.021+0000 7f92caffd700 1 -- 172.21.15.46:0/3052583056 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92e4064020 con 0x55ae1ecc22a0 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.022+0000 7f92caffd700 1 -- 172.21.15.46:0/3052583056 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f92e409a0c0 con 0x55ae1ecc22a0 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.022+0000 7f92fe3f5b80 10 client.17079 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.022+0000 7f92fe3f5b80 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.022+0000 7f92fe3f5b80 20 client.17079 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.022+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae1ecd1be0 0x55ae1ecd4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.022+0000 7f92fe3f5b80 4 client.17079 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.022+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3052583056 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae1ec02780 con 0x55ae1ecd1be0 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.022+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae1ecd1be0 0x55ae1ecd4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.022+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae1ecd1be0 0x55ae1ecd4000 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa811504776016 server_cookie=5132ac51e6d51f97 in_seq=0 out_seq=0 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.022+0000 7f92caffd700 10 client.17079 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:23.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92caffd700 1 -- 172.21.15.46:0/3052583056 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ae1ec02780 con 0x55ae1ecd1be0 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92caffd700 10 client.17079 handle_command_reply: tid=0 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92fe3f5b80 1 client.17079 shutdown 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3052583056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 msgr2=0x55ae1ecbf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecbf770 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f92e409a8d0 tx=0x7f92e409a5b0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3052583056 shutdown_connections 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae1ecd1be0 0x55ae1ecd4000 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecbf770 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92caffd700 0 client.17079 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/3052583056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb7400 0x55ae1ecbf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3052583056 >> 172.21.15.46:0/3052583056 conn(0x55ae1ec2d2a0 msgr2=0x55ae1ecb2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3052583056 shutdown_connections 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.023+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3052583056 wait complete. 2024-03-18T20:40:23.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.024+0000 7f92fe3f5b80 20 client.17079 trim_cache size 0 max 16384 2024-03-18T20:40:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.042+0000 7f92fe3f5b80 1 Processor -- start 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.042+0000 7f92fe3f5b80 1 -- start start 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.042+0000 7f92fe3f5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.042+0000 7f92fe3f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecb7400 0x55ae1ecbc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.042+0000 7f92fe3f5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecbc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.042+0000 7f92fe3f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1ecad000 con 0x55ae1ecb7400 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.042+0000 7f92fe3f5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1ecb7db0 con 0x55ae1ecc22a0 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.042+0000 7f92fe3f5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1ecb42b0 con 0x55ae1ecb6ae0 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecb7400 0x55ae1ecbc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92eb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecb7400 0x55ae1ecbc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34668/0 (socket says 172.21.15.46:34668) 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92eb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59040/0 (socket says 172.21.15.46:59040) 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92eaffd700 1 -- 172.21.15.46:0/3493298699 learned_addr learned my addr 172.21.15.46:0/3493298699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92ebfff700 1 --2- 172.21.15.46:0/3493298699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecbc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92eb7fe700 1 -- 172.21.15.46:0/3493298699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecc22a0 msgr2=0x55ae1ecbc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/3493298699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecbc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92eb7fe700 1 -- 172.21.15.46:0/3493298699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecb7400 msgr2=0x55ae1ecbc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/3493298699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecb7400 0x55ae1ecbc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92eb7fe700 1 -- 172.21.15.46:0/3493298699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae1ecbeb90 con 0x55ae1ecb6ae0 2024-03-18T20:40:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92ebfff700 1 --2- 172.21.15.46:0/3493298699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecbc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/3493298699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc1820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55ae1ecd1fc0 tx=0x7f92dc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a969c15b968671ea server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.043+0000 7f92d27fc700 1 -- 172.21.15.46:0/3493298699 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc024040 con 0x55ae1ecb6ae0 2024-03-18T20:40:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.044+0000 7f92d27fc700 1 -- 172.21.15.46:0/3493298699 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92dc0295b0 con 0x55ae1ecb6ae0 2024-03-18T20:40:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.044+0000 7f92d27fc700 1 -- 172.21.15.46:0/3493298699 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc01ed90 con 0x55ae1ecb6ae0 2024-03-18T20:40:23.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.044+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3493298699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 msgr2=0x55ae1ecc1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.044+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/3493298699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc1820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55ae1ecd1fc0 tx=0x7f92dc014000 comp rx=0 tx=0).stop 2024-03-18T20:40:23.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.044+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3493298699 shutdown_connections 2024-03-18T20:40:23.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.044+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/3493298699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecc22a0 0x55ae1ecbc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.044+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/3493298699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc1820 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.044+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/3493298699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecb7400 0x55ae1ecbc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.044+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3493298699 >> 172.21.15.46:0/3493298699 conn(0x55ae1ec2d2a0 msgr2=0x55ae1ecb17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.044+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3493298699 shutdown_connections 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.044+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/3493298699 wait complete. 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92fe3f5b80 1 -- start start 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92fe3f5b80 1 Processor -- start 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92fe3f5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92fe3f5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb7400 0x55ae1ecc8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92fe3f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ed25a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92fe3f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1ed25f60 con 0x55ae1ecc22a0 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92fe3f5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1ed260a0 con 0x55ae1ecb7400 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92fe3f5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1ed26260 con 0x55ae1ecb6ae0 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92fe3f5b80 10 client.0 _ll_register_callbacks cb 0x7f92e409aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ed25a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ed25a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34672/0 (socket says 172.21.15.46:34672) 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92ebfff700 1 -- 172.21.15.46:0/1486376822 learned_addr learned my addr 172.21.15.46:0/1486376822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.045+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92eaffd700 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb7400 0x55ae1ecc8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92ebfff700 1 -- 172.21.15.46:0/1486376822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 msgr2=0x55ae1ecc7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92ebfff700 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92ebfff700 1 -- 172.21.15.46:0/1486376822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb7400 msgr2=0x55ae1ecc8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92ebfff700 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb7400 0x55ae1ecc8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92ebfff700 1 -- 172.21.15.46:0/1486376822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92dc0038e0 con 0x55ae1ecc22a0 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92eaffd700 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb7400 0x55ae1ecc8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92ebfff700 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ed25a20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f92e40a2a90 tx=0x7f92e40074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1b4d1a96598cc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92caffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92caffd700 1 -- 172.21.15.46:0/1486376822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4096f10 con 0x55ae1ecc22a0 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92caffd700 1 -- 172.21.15.46:0/1486376822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f92e4065080 con 0x55ae1ecc22a0 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/1486376822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae1ea58d70 con 0x55ae1ecc22a0 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92caffd700 1 -- 172.21.15.46:0/1486376822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92e409abd0 con 0x55ae1ecc22a0 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92fe3f5b80 10 client.17085 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/1486376822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ae1ecd8c30 con 0x55ae1ecc22a0 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.046+0000 7f92caffd700 1 -- 172.21.15.46:0/1486376822 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae1ea58d70 con 0x55ae1ecc22a0 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.047+0000 7f92caffd700 1 -- 172.21.15.46:0/1486376822 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f92e40b3200 con 0x55ae1ecc22a0 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.047+0000 7f92fe3f5b80 10 client.17085 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.047+0000 7f92fe3f5b80 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.047+0000 7f92fe3f5b80 20 client.17085 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.047+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae1ed27570 0x55ae1ed29960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.047+0000 7f92fe3f5b80 4 client.17085 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:40:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.047+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/1486376822 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ae1ea8dd60 con 0x55ae1ed27570 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.047+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae1ed27570 0x55ae1ed29960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.047+0000 7f92caffd700 10 client.17085 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.047+0000 7f92eb7fe700 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae1ed27570 0x55ae1ed29960 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=889f9a6aaa2979f8 server_cookie=d23088b397fe84e in_seq=0 out_seq=0 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.048+0000 7f92caffd700 1 -- 172.21.15.46:0/1486376822 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ae1ea8dd60 con 0x55ae1ed27570 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.048+0000 7f92caffd700 10 client.17085 handle_command_reply: tid=0 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.048+0000 7f92fe3f5b80 1 client.17085 shutdown 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/1486376822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 msgr2=0x55ae1ed25a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ed25a20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f92e40a2a90 tx=0x7f92e40074e0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/1486376822 shutdown_connections 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae1ed27570 0x55ae1ed29960 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1ecc22a0 0x55ae1ed25a20 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1ecb6ae0 0x55ae1ecc7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92fe3f5b80 1 --2- 172.21.15.46:0/1486376822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1ecb7400 0x55ae1ecc8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/1486376822 >> 172.21.15.46:0/1486376822 conn(0x55ae1ec2d2a0 msgr2=0x55ae1ecbd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92caffd700 0 client.17085 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/1486376822 shutdown_connections 2024-03-18T20:40:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92fe3f5b80 1 -- 172.21.15.46:0/1486376822 wait complete. 2024-03-18T20:40:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92fe3f5b80 20 client.17085 trim_cache size 0 max 16384 2024-03-18T20:40:23.057 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:40:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:40:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "ca397ece-415a-46aa-a8b9-2fe36acf0a1e", 2024-03-18T20:40:23.057 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:40:23.057 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:40:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92f2260700 1 -- 172.21.15.46:0/1735526769 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f92d40a1290 msgr2=0x7f92d40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92f2260700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f92d40a1290 0x7f92d40a3750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f92dc011490 tx=0x7f92dc009260 comp rx=0 tx=0).stop 2024-03-18T20:40:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92f2260700 1 -- 172.21.15.46:0/1735526769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec073b30 msgr2=0x7f92ec1a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92f2260700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec073b30 0x7f92ec1a54f0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f92e400b070 tx=0x7f92e400bda0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92f2260700 1 -- 172.21.15.46:0/1735526769 shutdown_connections 2024-03-18T20:40:23.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92f2260700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f92d40a1290 0x7f92d40a3750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92f2260700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92ec073b30 0x7f92ec1a54f0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92f2260700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92ec100f70 0x7f92ec0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92f2260700 1 --2- 172.21.15.46:0/1735526769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92ec106090 0x7f92ec0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.049+0000 7f92f2260700 1 -- 172.21.15.46:0/1735526769 >> 172.21.15.46:0/1735526769 conn(0x7f92ec0fc7f0 msgr2=0x7f92ec0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:23.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.050+0000 7f92f2260700 1 -- 172.21.15.46:0/1735526769 shutdown_connections 2024-03-18T20:40:23.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.050+0000 7f92f2260700 1 -- 172.21.15.46:0/1735526769 wait complete. 2024-03-18T20:40:23.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:40:23.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:40:23.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.218+0000 7f7103749700 1 -- 172.21.15.46:0/4118293723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70fc100f70 msgr2=0x7f70fc101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.218+0000 7f7103749700 1 --2- 172.21.15.46:0/4118293723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70fc100f70 0x7f70fc101350 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009990 tx=0x7f70f8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.218+0000 7f7103749700 1 -- 172.21.15.46:0/4118293723 shutdown_connections 2024-03-18T20:40:23.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.218+0000 7f7103749700 1 --2- 172.21.15.46:0/4118293723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70fc100f70 0x7f70fc101350 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.218+0000 7f7103749700 1 --2- 172.21.15.46:0/4118293723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc106090 0x7f70fc10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.218+0000 7f7103749700 1 --2- 172.21.15.46:0/4118293723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70fc101920 0x7f70fc105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.218+0000 7f7103749700 1 -- 172.21.15.46:0/4118293723 >> 172.21.15.46:0/4118293723 conn(0x7f70fc0fc7f0 msgr2=0x7f70fc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:23.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.219+0000 7f7103749700 1 -- 172.21.15.46:0/4118293723 shutdown_connections 2024-03-18T20:40:23.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.219+0000 7f7103749700 1 -- 172.21.15.46:0/4118293723 wait complete. 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.219+0000 7f7103749700 1 Processor -- start 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.219+0000 7f7103749700 1 -- start start 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f7103749700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70fc100f70 0x7f70fc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f7103749700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70fc106090 0x7f70fc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f7103749700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc074d20 0x7f70fc073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f7103749700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc1a9e10 con 0x7f70fc100f70 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f7103749700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc1a9f80 con 0x7f70fc106090 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f7103749700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc1aa140 con 0x7f70fc074d20 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f71014e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70fc100f70 0x7f70fc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f71014e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70fc100f70 0x7f70fc0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34688/0 (socket says 172.21.15.46:34688) 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f71014e5700 1 -- 172.21.15.46:0/942265656 learned_addr learned my addr 172.21.15.46:0/942265656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f7100ce4700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70fc106090 0x7f70fc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f7101ce6700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc074d20 0x7f70fc073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f71014e5700 1 -- 172.21.15.46:0/942265656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc074d20 msgr2=0x7f70fc073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f71014e5700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc074d20 0x7f70fc073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f71014e5700 1 -- 172.21.15.46:0/942265656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70fc106090 msgr2=0x7f70fc0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f71014e5700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70fc106090 0x7f70fc0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f71014e5700 1 -- 172.21.15.46:0/942265656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f80095f0 con 0x7f70fc100f70 2024-03-18T20:40:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f7101ce6700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc074d20 0x7f70fc073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f71014e5700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70fc100f70 0x7f70fc0730b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f70f8004760 tx=0x7f70f8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.220+0000 7f70ee7fc700 1 -- 172.21.15.46:0/942265656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f801d070 con 0x7f70fc100f70 2024-03-18T20:40:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.221+0000 7f7103749700 1 -- 172.21.15.46:0/942265656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc1aa3c0 con 0x7f70fc100f70 2024-03-18T20:40:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.221+0000 7f70ee7fc700 1 -- 172.21.15.46:0/942265656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f8022470 con 0x7f70fc100f70 2024-03-18T20:40:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.221+0000 7f70ee7fc700 1 -- 172.21.15.46:0/942265656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8017650 con 0x7f70fc100f70 2024-03-18T20:40:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.221+0000 7f7103749700 1 -- 172.21.15.46:0/942265656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc1aa840 con 0x7f70fc100f70 2024-03-18T20:40:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.223+0000 7f70ee7fc700 1 -- 172.21.15.46:0/942265656 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f70f80225e0 con 0x7f70fc100f70 2024-03-18T20:40:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.223+0000 7f70ee7fc700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f70e80a1290 0x7f70e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.224+0000 7f70ee7fc700 1 -- 172.21.15.46:0/942265656 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f70f80cdeb0 con 0x7f70fc100f70 2024-03-18T20:40:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.224+0000 7f7100ce4700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f70e80a1290 0x7f70e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.224+0000 7f7100ce4700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f70e80a1290 0x7f70e80a3750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70fc074790 tx=0x7f70f000b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f710f8deb80 1 Processor -- start 2024-03-18T20:40:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f710f8deb80 1 -- start start 2024-03-18T20:40:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f710f8deb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 0x564889873e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f710f8deb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889874380 0x56488987ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f710f8deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x564889881610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f710f8deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648897cd990 con 0x56488987f220 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f710f8deb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648897cb350 con 0x564889873a60 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f710f8deb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648897c6c40 con 0x564889874380 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f7101ce6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x564889881610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f71014e5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 0x564889873e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f7100ce4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889874380 0x56488987ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f7101ce6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x564889881610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34690/0 (socket says 172.21.15.46:34690) 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.225+0000 7f7101ce6700 1 -- 172.21.15.46:0/1005077017 learned_addr learned my addr 172.21.15.46:0/1005077017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.226+0000 7f71014e5700 1 -- 172.21.15.46:0/1005077017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889874380 msgr2=0x56488987ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.226+0000 7f71014e5700 1 --2- 172.21.15.46:0/1005077017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889874380 0x56488987ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.226+0000 7f71014e5700 1 -- 172.21.15.46:0/1005077017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 msgr2=0x564889881610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.226+0000 7f71014e5700 1 --2- 172.21.15.46:0/1005077017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x564889881610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.226+0000 7f71014e5700 1 -- 172.21.15.46:0/1005077017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648896483b0 con 0x564889873a60 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.226+0000 7f7100ce4700 1 --2- 172.21.15.46:0/1005077017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889874380 0x56488987ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.226+0000 7f71014e5700 1 --2- 172.21.15.46:0/1005077017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 0x564889873e40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f70f80cb610 tx=0x7f70f80974d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09cd30a4d5622e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.226+0000 7f70e77fe700 1 -- 172.21.15.46:0/1005077017 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f80aeea0 con 0x564889873a60 2024-03-18T20:40:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.226+0000 7f70e77fe700 1 -- 172.21.15.46:0/1005077017 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f809b760 con 0x564889873a60 2024-03-18T20:40:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.226+0000 7f70e77fe700 1 -- 172.21.15.46:0/1005077017 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f809b2c0 con 0x564889873a60 2024-03-18T20:40:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.227+0000 7f710f8deb80 1 -- 172.21.15.46:0/1005077017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 msgr2=0x564889873e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.227+0000 7f710f8deb80 1 --2- 172.21.15.46:0/1005077017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 0x564889873e40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f70f80cb610 tx=0x7f70f80974d0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.227+0000 7f710f8deb80 1 -- 172.21.15.46:0/1005077017 shutdown_connections 2024-03-18T20:40:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.227+0000 7f710f8deb80 1 --2- 172.21.15.46:0/1005077017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x564889881610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.227+0000 7f710f8deb80 1 --2- 172.21.15.46:0/1005077017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 0x564889873e40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.227+0000 7f710f8deb80 1 --2- 172.21.15.46:0/1005077017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889874380 0x56488987ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.227+0000 7f710f8deb80 1 -- 172.21.15.46:0/1005077017 >> 172.21.15.46:0/1005077017 conn(0x5648897ea220 msgr2=0x564889870020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.227+0000 7f710f8deb80 1 -- 172.21.15.46:0/1005077017 shutdown_connections 2024-03-18T20:40:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.227+0000 7f710f8deb80 1 -- 172.21.15.46:0/1005077017 wait complete. 2024-03-18T20:40:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.227+0000 7f710f8deb80 1 -- start start 2024-03-18T20:40:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.227+0000 7f710f8deb80 1 Processor -- start 2024-03-18T20:40:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f710f8deb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889873a60 0x56488987f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f710f8deb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 0x56488987fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f710f8deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x56488988a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f710f8deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564889762d80 con 0x56488987f220 2024-03-18T20:40:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f710f8deb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564889762a20 con 0x564889874380 2024-03-18T20:40:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f710f8deb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56488964a6a0 con 0x564889873a60 2024-03-18T20:40:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f710f8deb80 10 client.0 _ll_register_callbacks cb 0x5648895ce3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f71014e5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889873a60 0x56488987f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f7101ce6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x56488988a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f71014e5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889873a60 0x56488987f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59070/0 (socket says 172.21.15.46:59070) 2024-03-18T20:40:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f71014e5700 1 -- 172.21.15.46:0/2737639033 learned_addr learned my addr 172.21.15.46:0/2737639033 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f7100ce4700 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 0x56488987fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f7101ce6700 1 -- 172.21.15.46:0/2737639033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889873a60 msgr2=0x56488987f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f7101ce6700 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889873a60 0x56488987f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f7101ce6700 1 -- 172.21.15.46:0/2737639033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 msgr2=0x56488987fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f7101ce6700 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 0x56488987fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f7101ce6700 1 -- 172.21.15.46:0/2737639033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f0010c20 con 0x56488987f220 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f71014e5700 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889873a60 0x56488987f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.228+0000 7f7100ce4700 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 0x56488987fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.229+0000 7f7101ce6700 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x56488988a680 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f70f4004770 tx=0x7f70f4003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61db4abb00a2f7ad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.229+0000 7f70e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.229+0000 7f70e4ff9700 1 -- 172.21.15.46:0/2737639033 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f400ed30 con 0x56488987f220 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.229+0000 7f70e4ff9700 1 -- 172.21.15.46:0/2737639033 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f70f40093a0 con 0x56488987f220 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.229+0000 7f710f8deb80 1 -- 172.21.15.46:0/2737639033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564889867ad0 con 0x56488987f220 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.229+0000 7f70e4ff9700 1 -- 172.21.15.46:0/2737639033 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70f400ee90 con 0x56488987f220 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.229+0000 7f710f8deb80 10 client.17097 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.229+0000 7f710f8deb80 1 -- 172.21.15.46:0/2737639033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f70fc1aa840 con 0x56488987f220 2024-03-18T20:40:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.229+0000 7f70e4ff9700 1 -- 172.21.15.46:0/2737639033 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564889867ad0 con 0x56488987f220 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.229+0000 7f70e4ff9700 1 -- 172.21.15.46:0/2737639033 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f70f4025070 con 0x56488987f220 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.230+0000 7f710f8deb80 10 client.17097 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.230+0000 7f710f8deb80 10 client.17097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.230+0000 7f710f8deb80 20 client.17097 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.230+0000 7f710f8deb80 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56488988ec00 0x564889891020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.230+0000 7f710f8deb80 4 client.17097 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.230+0000 7f710f8deb80 1 -- 172.21.15.46:0/2737639033 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648897bf700 con 0x56488988ec00 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.230+0000 7f71014e5700 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56488988ec00 0x564889891020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.230+0000 7f70e4ff9700 10 client.17097 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.230+0000 7f71014e5700 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56488988ec00 0x564889891020 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea55d06ba6790414 server_cookie=e2159874e7456937 in_seq=0 out_seq=0 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f70e4ff9700 1 -- 172.21.15.46:0/2737639033 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5648897bf700 con 0x56488988ec00 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f70e4ff9700 10 client.17097 handle_command_reply: tid=0 2024-03-18T20:40:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f710f8deb80 1 client.17097 shutdown 2024-03-18T20:40:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f710f8deb80 1 -- 172.21.15.46:0/2737639033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 msgr2=0x56488988a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f710f8deb80 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x56488988a680 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f70f4004770 tx=0x7f70f4003f80 comp rx=0 tx=0).stop 2024-03-18T20:40:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f710f8deb80 1 -- 172.21.15.46:0/2737639033 shutdown_connections 2024-03-18T20:40:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f710f8deb80 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56488988ec00 0x564889891020 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f710f8deb80 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x56488988a680 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f710f8deb80 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889873a60 0x56488987f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f710f8deb80 1 --2- 172.21.15.46:0/2737639033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 0x56488987fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f70e4ff9700 0 client.17097 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f710f8deb80 1 -- 172.21.15.46:0/2737639033 >> 172.21.15.46:0/2737639033 conn(0x5648897ea220 msgr2=0x56488986fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f710f8deb80 1 -- 172.21.15.46:0/2737639033 shutdown_connections 2024-03-18T20:40:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.231+0000 7f710f8deb80 1 -- 172.21.15.46:0/2737639033 wait complete. 2024-03-18T20:40:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.232+0000 7f710f8deb80 20 client.17097 trim_cache size 0 max 16384 2024-03-18T20:40:23.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f710f8deb80 1 Processor -- start 2024-03-18T20:40:23.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f710f8deb80 1 -- start start 2024-03-18T20:40:23.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f710f8deb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 0x564889884620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f710f8deb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889874380 0x56488987db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f710f8deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x56488987e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f710f8deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56488988b670 con 0x56488987f220 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f710f8deb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56488988afd0 con 0x564889873a60 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f710f8deb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564889875620 con 0x564889874380 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f71014e5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 0x564889884620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f7101ce6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x56488987e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f71014e5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 0x564889884620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:42356/0 (socket says 172.21.15.46:42356) 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f71014e5700 1 -- 172.21.15.46:0/1240382030 learned_addr learned my addr 172.21.15.46:0/1240382030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.251+0000 7f7100ce4700 1 --2- 172.21.15.46:0/1240382030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889874380 0x56488987db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.252+0000 7f71014e5700 1 -- 172.21.15.46:0/1240382030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889874380 msgr2=0x56488987db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.252+0000 7f71014e5700 1 --2- 172.21.15.46:0/1240382030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889874380 0x56488987db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.252+0000 7f71014e5700 1 -- 172.21.15.46:0/1240382030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 msgr2=0x56488987e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.252+0000 7f71014e5700 1 --2- 172.21.15.46:0/1240382030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x56488987e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.252+0000 7f71014e5700 1 -- 172.21.15.46:0/1240382030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648895f4b60 con 0x564889873a60 2024-03-18T20:40:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.252+0000 7f7101ce6700 1 --2- 172.21.15.46:0/1240382030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x56488987e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.252+0000 7f71014e5700 1 --2- 172.21.15.46:0/1240382030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 0x564889884620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f70f8022c60 tx=0x7f70f8022cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fa2c0fa4dcd5032 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.252+0000 7f70e7fff700 1 -- 172.21.15.46:0/1240382030 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f80b2070 con 0x564889873a60 2024-03-18T20:40:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.252+0000 7f70e7fff700 1 -- 172.21.15.46:0/1240382030 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f809ee90 con 0x564889873a60 2024-03-18T20:40:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.252+0000 7f70e7fff700 1 -- 172.21.15.46:0/1240382030 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f80aeea0 con 0x564889873a60 2024-03-18T20:40:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.253+0000 7f710f8deb80 1 -- 172.21.15.46:0/1240382030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 msgr2=0x564889884620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.253+0000 7f710f8deb80 1 --2- 172.21.15.46:0/1240382030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 0x564889884620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f70f8022c60 tx=0x7f70f8022cb0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.253+0000 7f710f8deb80 1 -- 172.21.15.46:0/1240382030 shutdown_connections 2024-03-18T20:40:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.253+0000 7f710f8deb80 1 --2- 172.21.15.46:0/1240382030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56488987f220 0x56488987e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.253+0000 7f710f8deb80 1 --2- 172.21.15.46:0/1240382030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889873a60 0x564889884620 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.253+0000 7f710f8deb80 1 --2- 172.21.15.46:0/1240382030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564889874380 0x56488987db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.253+0000 7f710f8deb80 1 -- 172.21.15.46:0/1240382030 >> 172.21.15.46:0/1240382030 conn(0x5648897ea220 msgr2=0x5648898798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.253+0000 7f710f8deb80 1 -- 172.21.15.46:0/1240382030 shutdown_connections 2024-03-18T20:40:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.253+0000 7f710f8deb80 1 -- 172.21.15.46:0/1240382030 wait complete. 2024-03-18T20:40:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.253+0000 7f710f8deb80 1 -- start start 2024-03-18T20:40:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.253+0000 7f710f8deb80 1 Processor -- start 2024-03-18T20:40:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f710f8deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564889873a60 0x5648898db300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f710f8deb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 0x5648898db840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f710f8deb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56488987f220 0x564889883b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f710f8deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564889884150 con 0x564889873a60 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f710f8deb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648898842c0 con 0x564889874380 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f710f8deb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564889884480 con 0x56488987f220 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f710f8deb80 10 client.0 _ll_register_callbacks cb 0x5648895ce3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f71014e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564889873a60 0x5648898db300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f7100ce4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 0x5648898db840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f71014e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564889873a60 0x5648898db300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34720/0 (socket says 172.21.15.46:34720) 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f71014e5700 1 -- 172.21.15.46:0/3580873253 learned_addr learned my addr 172.21.15.46:0/3580873253 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f7100ce4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 0x5648898db840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:42364/0 (socket says 172.21.15.46:42364) 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f7100ce4700 1 -- 172.21.15.46:0/3580873253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56488987f220 msgr2=0x564889883b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f7100ce4700 1 --2- 172.21.15.46:0/3580873253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56488987f220 0x564889883b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f7100ce4700 1 -- 172.21.15.46:0/3580873253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564889873a60 msgr2=0x5648898db300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f7100ce4700 1 --2- 172.21.15.46:0/3580873253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564889873a60 0x5648898db300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f7100ce4700 1 -- 172.21.15.46:0/3580873253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc1aa840 con 0x564889874380 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f7100ce4700 1 --2- 172.21.15.46:0/3580873253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 0x5648898db840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f70f000ba80 tx=0x7f70f00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=940b87cab1d48515 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.254+0000 7f70e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f70e4ff9700 1 -- 172.21.15.46:0/3580873253 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0023070 con 0x564889874380 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f70e4ff9700 1 -- 172.21.15.46:0/3580873253 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f70f0017070 con 0x564889874380 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f710f8deb80 1 -- 172.21.15.46:0/3580873253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70f809b760 con 0x564889874380 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f70e4ff9700 1 -- 172.21.15.46:0/3580873253 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70f0004830 con 0x564889874380 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f710f8deb80 10 client.26816 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f710f8deb80 1 -- 172.21.15.46:0/3580873253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5648896483b0 con 0x564889874380 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f70e4ff9700 1 -- 172.21.15.46:0/3580873253 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f809b760 con 0x564889874380 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f70e4ff9700 1 -- 172.21.15.46:0/3580873253 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f70f00316d0 con 0x564889874380 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f710f8deb80 10 client.26816 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f710f8deb80 10 client.26816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f710f8deb80 20 client.26816 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f710f8deb80 1 --2- 172.21.15.46:0/3580873253 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5648898e3180 0x5648898e5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f710f8deb80 4 client.26816 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.255+0000 7f710f8deb80 1 -- 172.21.15.46:0/3580873253 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564889615cf0 con 0x5648898e3180 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f71014e5700 1 --2- 172.21.15.46:0/3580873253 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5648898e3180 0x5648898e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f70e4ff9700 10 client.26816 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f71014e5700 1 --2- 172.21.15.46:0/3580873253 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5648898e3180 0x5648898e5570 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caa9588bbd465b31 server_cookie=d0cad5eca61d4285 in_seq=0 out_seq=0 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f70e4ff9700 1 -- 172.21.15.46:0/3580873253 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564889615cf0 con 0x5648898e3180 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f70e4ff9700 10 client.26816 handle_command_reply: tid=0 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f710f8deb80 1 client.26816 shutdown 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f710f8deb80 1 -- 172.21.15.46:0/3580873253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 msgr2=0x5648898db840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f710f8deb80 1 --2- 172.21.15.46:0/3580873253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 0x5648898db840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f70f000ba80 tx=0x7f70f00047c0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f710f8deb80 1 -- 172.21.15.46:0/3580873253 shutdown_connections 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f710f8deb80 1 --2- 172.21.15.46:0/3580873253 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5648898e3180 0x5648898e5570 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f710f8deb80 1 --2- 172.21.15.46:0/3580873253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564889873a60 0x5648898db300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f710f8deb80 1 --2- 172.21.15.46:0/3580873253 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56488987f220 0x564889883b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f710f8deb80 1 --2- 172.21.15.46:0/3580873253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564889874380 0x5648898db840 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f70e4ff9700 0 client.26816 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f710f8deb80 1 -- 172.21.15.46:0/3580873253 >> 172.21.15.46:0/3580873253 conn(0x5648897ea220 msgr2=0x56488987db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f710f8deb80 1 -- 172.21.15.46:0/3580873253 shutdown_connections 2024-03-18T20:40:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.256+0000 7f710f8deb80 1 -- 172.21.15.46:0/3580873253 wait complete. 2024-03-18T20:40:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f710f8deb80 20 client.26816 trim_cache size 0 max 16384 2024-03-18T20:40:23.266 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:40:23.266 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:40:23.266 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:40:23.267 INFO:teuthology.orchestra.run.smithi046.stdout: "ca397ece-415a-46aa-a8b9-2fe36acf0a1e": { 2024-03-18T20:40:23.267 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:40:23.267 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "ca397ece-415a-46aa-a8b9-2fe36acf0a1e", 2024-03-18T20:40:23.267 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:40:23.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:40:23.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:40:23.267 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:40:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f7103749700 1 -- 172.21.15.46:0/942265656 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f70e80a1290 msgr2=0x7f70e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f7103749700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f70e80a1290 0x7f70e80a3750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f70fc074790 tx=0x7f70f000b410 comp rx=0 tx=0).stop 2024-03-18T20:40:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f7103749700 1 -- 172.21.15.46:0/942265656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70fc100f70 msgr2=0x7f70fc0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f7103749700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70fc100f70 0x7f70fc0730b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f70f8004760 tx=0x7f70f8004790 comp rx=0 tx=0).stop 2024-03-18T20:40:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f7103749700 1 -- 172.21.15.46:0/942265656 shutdown_connections 2024-03-18T20:40:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f7103749700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f70e80a1290 0x7f70e80a3750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f7103749700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70fc100f70 0x7f70fc0730b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f7103749700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70fc074d20 0x7f70fc073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f7103749700 1 --2- 172.21.15.46:0/942265656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70fc106090 0x7f70fc0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f7103749700 1 -- 172.21.15.46:0/942265656 >> 172.21.15.46:0/942265656 conn(0x7f70fc0fc7f0 msgr2=0x7f70fc0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f7103749700 1 -- 172.21.15.46:0/942265656 shutdown_connections 2024-03-18T20:40:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:23.257+0000 7f7103749700 1 -- 172.21.15.46:0/942265656 wait complete. 2024-03-18T20:40:23.274 INFO:tasks.cephfs.filesystem:scrub status for tag:ca397ece-415a-46aa-a8b9-2fe36acf0a1e - {'path': '/', 'tag': 'ca397ece-415a-46aa-a8b9-2fe36acf0a1e', 'options': 'recursive,force'} 2024-03-18T20:40:23.684 INFO:tasks.workunit.client.0.smithi046.stdout: 1 108513 1.79 MB/sec execute 78 sec latency 991.914 ms 2024-03-18T20:40:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:40:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:24 smithi073 ceph-mon[47384]: pgmap v132: 97 pgs: 97 active+clean; 48 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 548 KiB/s wr, 105 op/s 2024-03-18T20:40:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:40:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:24 smithi155 ceph-mon[48344]: pgmap v132: 97 pgs: 97 active+clean; 48 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 548 KiB/s wr, 105 op/s 2024-03-18T20:40:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:40:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:24.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:24 smithi046 ceph-mon[43495]: pgmap v132: 97 pgs: 97 active+clean; 48 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 548 KiB/s wr, 105 op/s 2024-03-18T20:40:24.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 110373 1.93 MB/sec execute 79 sec latency 11.901 ms 2024-03-18T20:40:25.684 INFO:tasks.workunit.client.0.smithi046.stdout: 1 110667 1.91 MB/sec execute 80 sec latency 693.144 ms 2024-03-18T20:40:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:25 smithi073 ceph-mon[47384]: pgmap v133: 97 pgs: 97 active+clean; 48 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 385 KiB/s wr, 77 op/s 2024-03-18T20:40:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:25 smithi155 ceph-mon[48344]: pgmap v133: 97 pgs: 97 active+clean; 48 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 385 KiB/s wr, 77 op/s 2024-03-18T20:40:26.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:25 smithi046 ceph-mon[43495]: pgmap v133: 97 pgs: 97 active+clean; 48 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 385 KiB/s wr, 77 op/s 2024-03-18T20:40:26.684 INFO:tasks.workunit.client.0.smithi046.stdout: 1 110667 1.89 MB/sec execute 81 sec latency 1693.246 ms 2024-03-18T20:40:27.685 INFO:tasks.workunit.client.0.smithi046.stdout: 1 110791 1.87 MB/sec execute 82 sec latency 2054.259 ms 2024-03-18T20:40:28.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:28 smithi073 ceph-mon[47384]: pgmap v134: 97 pgs: 97 active+clean; 48 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 385 KiB/s wr, 77 op/s 2024-03-18T20:40:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:28 smithi155 ceph-mon[48344]: pgmap v134: 97 pgs: 97 active+clean; 48 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 385 KiB/s wr, 77 op/s 2024-03-18T20:40:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:28 smithi046 ceph-mon[43495]: pgmap v134: 97 pgs: 97 active+clean; 48 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 385 KiB/s wr, 77 op/s 2024-03-18T20:40:28.685 INFO:tasks.workunit.client.0.smithi046.stdout: 1 110791 1.85 MB/sec execute 83 sec latency 1552.493 ms 2024-03-18T20:40:29.685 INFO:tasks.workunit.client.0.smithi046.stdout: 1 110791 1.82 MB/sec execute 84 sec latency 2552.607 ms 2024-03-18T20:40:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:30 smithi073 ceph-mon[47384]: pgmap v135: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 968 KiB/s wr, 136 op/s 2024-03-18T20:40:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:30 smithi155 ceph-mon[48344]: pgmap v135: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 968 KiB/s wr, 136 op/s 2024-03-18T20:40:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:30 smithi046 ceph-mon[43495]: pgmap v135: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 968 KiB/s wr, 136 op/s 2024-03-18T20:40:30.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 110791 1.80 MB/sec execute 85 sec latency 3552.649 ms 2024-03-18T20:40:31.685 INFO:tasks.workunit.client.0.smithi046.stdout: 1 110791 1.78 MB/sec execute 86 sec latency 4552.710 ms 2024-03-18T20:40:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:32 smithi073 ceph-mon[47384]: pgmap v136: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 713 KiB/s wr, 93 op/s 2024-03-18T20:40:32.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:32 smithi155 ceph-mon[48344]: pgmap v136: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 713 KiB/s wr, 93 op/s 2024-03-18T20:40:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:32 smithi046 ceph-mon[43495]: pgmap v136: 97 pgs: 97 active+clean; 54 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 713 KiB/s wr, 93 op/s 2024-03-18T20:40:32.685 INFO:tasks.workunit.client.0.smithi046.stdout: 1 112595 1.85 MB/sec execute 87 sec latency 4905.190 ms 2024-03-18T20:40:33.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:33 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:40:33.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:33 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:40:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:33 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:40:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:33 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:40:33.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:33 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:40:33.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:33 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:40:33.685 INFO:tasks.workunit.client.0.smithi046.stdout: 1 114150 1.95 MB/sec execute 88 sec latency 10.975 ms 2024-03-18T20:40:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.042+0000 7fd632f2b700 1 -- 172.21.15.46:0/119317679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd62c100570 msgr2=0x7fd62c10ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.042+0000 7fd632f2b700 1 --2- 172.21.15.46:0/119317679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd62c100570 0x7fd62c10ca00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd62400b0a0 tx=0x7fd62400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.042+0000 7fd632f2b700 1 -- 172.21.15.46:0/119317679 shutdown_connections 2024-03-18T20:40:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.042+0000 7fd632f2b700 1 --2- 172.21.15.46:0/119317679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd62c100570 0x7fd62c10ca00 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.042+0000 7fd632f2b700 1 --2- 172.21.15.46:0/119317679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd62c105c60 0x7fd62c106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.042+0000 7fd632f2b700 1 --2- 172.21.15.46:0/119317679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd62c0ffbb0 0x7fd62c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.042+0000 7fd632f2b700 1 -- 172.21.15.46:0/119317679 >> 172.21.15.46:0/119317679 conn(0x7fd62c076d50 msgr2=0x7fd62c077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.042+0000 7fd632f2b700 1 -- 172.21.15.46:0/119317679 shutdown_connections 2024-03-18T20:40:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.042+0000 7fd632f2b700 1 -- 172.21.15.46:0/119317679 wait complete. 2024-03-18T20:40:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd632f2b700 1 Processor -- start 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd632f2b700 1 -- start start 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd632f2b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd62c0ffbb0 0x7fd62c101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd632f2b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd62c100570 0x7fd62c101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd632f2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd62c105c60 0x7fd62c1022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd632f2b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c1a55b0 con 0x7fd62c105c60 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd632f2b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c1a5720 con 0x7fd62c0ffbb0 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd632f2b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c1a58e0 con 0x7fd62c100570 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd6314c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd62c105c60 0x7fd62c1022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd630cc7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd62c0ffbb0 0x7fd62c101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd62bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd62c100570 0x7fd62c101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd6314c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd62c105c60 0x7fd62c1022d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48854/0 (socket says 172.21.15.46:48854) 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.043+0000 7fd6314c8700 1 -- 172.21.15.46:0/4053567863 learned_addr learned my addr 172.21.15.46:0/4053567863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.044+0000 7fd6314c8700 1 -- 172.21.15.46:0/4053567863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd62c100570 msgr2=0x7fd62c101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.044+0000 7fd6314c8700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd62c100570 0x7fd62c101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.044+0000 7fd6314c8700 1 -- 172.21.15.46:0/4053567863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd62c0ffbb0 msgr2=0x7fd62c101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.044+0000 7fd6314c8700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd62c0ffbb0 0x7fd62c101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.044+0000 7fd6314c8700 1 -- 172.21.15.46:0/4053567863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd624009d00 con 0x7fd62c105c60 2024-03-18T20:40:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.044+0000 7fd630cc7700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd62c0ffbb0 0x7fd62c101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.044+0000 7fd6314c8700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd62c105c60 0x7fd62c1022d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd62400b070 tx=0x7fd624007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.044+0000 7fd629ffb700 1 -- 172.21.15.46:0/4053567863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62400e040 con 0x7fd62c105c60 2024-03-18T20:40:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.044+0000 7fd629ffb700 1 -- 172.21.15.46:0/4053567863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62401d470 con 0x7fd62c105c60 2024-03-18T20:40:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.044+0000 7fd632f2b700 1 -- 172.21.15.46:0/4053567863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c1a5b60 con 0x7fd62c105c60 2024-03-18T20:40:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.044+0000 7fd629ffb700 1 -- 172.21.15.46:0/4053567863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6240052a0 con 0x7fd62c105c60 2024-03-18T20:40:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.044+0000 7fd632f2b700 1 -- 172.21.15.46:0/4053567863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62c1a6070 con 0x7fd62c105c60 2024-03-18T20:40:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.046+0000 7fd629ffb700 1 -- 172.21.15.46:0/4053567863 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd62401d850 con 0x7fd62c105c60 2024-03-18T20:40:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.047+0000 7fd629ffb700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd6140a1290 0x7fd6140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.047+0000 7fd630cc7700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd6140a1290 0x7fd6140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.047+0000 7fd629ffb700 1 -- 172.21.15.46:0/4053567863 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd6240cdda0 con 0x7fd62c105c60 2024-03-18T20:40:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.048+0000 7fd630cc7700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd6140a1290 0x7fd6140a3750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd61c011490 tx=0x7fd61c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.048+0000 7fd63f0c0b80 1 Processor -- start 2024-03-18T20:40:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.048+0000 7fd63f0c0b80 1 -- start start 2024-03-18T20:40:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.048+0000 7fd63f0c0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.048+0000 7fd63f0c0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 0x5650e49db870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.048+0000 7fd63f0c0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49de1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd63f0c0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e492a930 con 0x5650e49dbdb0 2024-03-18T20:40:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd63f0c0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e49282f0 con 0x5650e49d0e90 2024-03-18T20:40:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd63f0c0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e4923be0 con 0x5650e49d0570 2024-03-18T20:40:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd630cc7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd630cc7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49d0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:34482/0 (socket says 172.21.15.46:34482) 2024-03-18T20:40:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd630cc7700 1 -- 172.21.15.46:0/2520853517 learned_addr learned my addr 172.21.15.46:0/2520853517 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd6314c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49de1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd62bfff700 1 --2- 172.21.15.46:0/2520853517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 0x5650e49db870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd62bfff700 1 -- 172.21.15.46:0/2520853517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 msgr2=0x5650e49d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd62bfff700 1 --2- 172.21.15.46:0/2520853517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd62bfff700 1 -- 172.21.15.46:0/2520853517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 msgr2=0x5650e49de1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd62bfff700 1 --2- 172.21.15.46:0/2520853517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49de1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd62bfff700 1 -- 172.21.15.46:0/2520853517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650e47a5350 con 0x5650e49d0e90 2024-03-18T20:40:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd62bfff700 1 --2- 172.21.15.46:0/2520853517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 0x5650e49db870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd620005950 tx=0x7fd62000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2175e77501bed72 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd612ffd700 1 -- 172.21.15.46:0/2520853517 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620010c70 con 0x5650e49d0e90 2024-03-18T20:40:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd612ffd700 1 -- 172.21.15.46:0/2520853517 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd620015070 con 0x5650e49d0e90 2024-03-18T20:40:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.049+0000 7fd612ffd700 1 -- 172.21.15.46:0/2520853517 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620005020 con 0x5650e49d0e90 2024-03-18T20:40:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.050+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/2520853517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 msgr2=0x5650e49db870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.050+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/2520853517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 0x5650e49db870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd620005950 tx=0x7fd62000bf60 comp rx=0 tx=0).stop 2024-03-18T20:40:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.050+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/2520853517 shutdown_connections 2024-03-18T20:40:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.050+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/2520853517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49de1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.050+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/2520853517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.050+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/2520853517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 0x5650e49db870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.050+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/2520853517 >> 172.21.15.46:0/2520853517 conn(0x5650e49471c0 msgr2=0x5650e49cca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.050+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/2520853517 shutdown_connections 2024-03-18T20:40:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.050+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/2520853517 wait complete. 2024-03-18T20:40:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd63f0c0b80 1 -- start start 2024-03-18T20:40:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd63f0c0b80 1 Processor -- start 2024-03-18T20:40:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd63f0c0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49db730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd63f0c0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49d0e90 0x5650e49d8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd63f0c0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49dbdb0 0x5650e49d9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd63f0c0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e48bfd20 con 0x5650e49d0e90 2024-03-18T20:40:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd63f0c0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e48bf9c0 con 0x5650e49dbdb0 2024-03-18T20:40:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd63f0c0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e47a7640 con 0x5650e49d0570 2024-03-18T20:40:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd63f0c0b80 10 client.0 _ll_register_callbacks cb 0x5650e472b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd630cc7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49db730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd62bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49d0e90 0x5650e49d8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd630cc7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49db730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:34496/0 (socket says 172.21.15.46:34496) 2024-03-18T20:40:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd630cc7700 1 -- 172.21.15.46:0/1233813217 learned_addr learned my addr 172.21.15.46:0/1233813217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.051+0000 7fd6314c8700 1 --2- 172.21.15.46:0/1233813217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49dbdb0 0x5650e49d9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd630cc7700 1 -- 172.21.15.46:0/1233813217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49dbdb0 msgr2=0x5650e49d9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd630cc7700 1 --2- 172.21.15.46:0/1233813217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49dbdb0 0x5650e49d9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd630cc7700 1 -- 172.21.15.46:0/1233813217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49d0e90 msgr2=0x5650e49d8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd630cc7700 1 --2- 172.21.15.46:0/1233813217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49d0e90 0x5650e49d8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd630cc7700 1 -- 172.21.15.46:0/1233813217 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62000b8e0 con 0x5650e49d0570 2024-03-18T20:40:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd630cc7700 1 --2- 172.21.15.46:0/1233813217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49db730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd61c011be0 tx=0x7fd61c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a2dd72446966a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd60bfff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:40:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd60bfff700 1 -- 172.21.15.46:0/1233813217 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c01fc20 con 0x5650e49d0570 2024-03-18T20:40:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd60bfff700 1 -- 172.21.15.46:0/1233813217 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd61c003e10 con 0x5650e49d0570 2024-03-18T20:40:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/1233813217 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650e49c46c0 con 0x5650e49d0570 2024-03-18T20:40:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd60bfff700 1 -- 172.21.15.46:0/1233813217 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd61c01fd80 con 0x5650e49d0570 2024-03-18T20:40:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd63f0c0b80 10 client.26842 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.052+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/1233813217 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5650e4751b00 con 0x5650e49d0570 2024-03-18T20:40:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.053+0000 7fd60bfff700 1 -- 172.21.15.46:0/1233813217 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd61c034080 con 0x5650e49d0570 2024-03-18T20:40:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.053+0000 7fd60bfff700 1 -- 172.21.15.46:0/1233813217 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd61c030510 con 0x5650e49d0570 2024-03-18T20:40:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.053+0000 7fd63f0c0b80 10 client.26842 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.053+0000 7fd63f0c0b80 10 client.26842 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.053+0000 7fd63f0c0b80 20 client.26842 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.053+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/1233813217 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650e49eba70 0x5650e49ede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.053+0000 7fd63f0c0b80 4 client.26842 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.053+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/1233813217 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650e491c6a0 con 0x5650e49eba70 2024-03-18T20:40:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.053+0000 7fd62bfff700 1 --2- 172.21.15.46:0/1233813217 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650e49eba70 0x5650e49ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd60bfff700 10 client.26842 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd62bfff700 1 --2- 172.21.15.46:0/1233813217 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650e49eba70 0x5650e49ede90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f240e7453d4298c server_cookie=6113d8193752ee0c in_seq=0 out_seq=0 2024-03-18T20:40:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd60bfff700 1 -- 172.21.15.46:0/1233813217 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5650e491c6a0 con 0x5650e49eba70 2024-03-18T20:40:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd60bfff700 10 client.26842 handle_command_reply: tid=0 2024-03-18T20:40:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd63f0c0b80 1 client.26842 shutdown 2024-03-18T20:40:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/1233813217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 msgr2=0x5650e49db730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/1233813217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49db730 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd61c011be0 tx=0x7fd61c0047c0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/1233813217 shutdown_connections 2024-03-18T20:40:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/1233813217 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650e49eba70 0x5650e49ede90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/1233813217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49dbdb0 0x5650e49d9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/1233813217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49db730 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/1233813217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49d0e90 0x5650e49d8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/1233813217 >> 172.21.15.46:0/1233813217 conn(0x5650e49471c0 msgr2=0x5650e49cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.054+0000 7fd60bfff700 0 client.26842 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.055+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/1233813217 shutdown_connections 2024-03-18T20:40:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.055+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/1233813217 wait complete. 2024-03-18T20:40:34.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.055+0000 7fd63f0c0b80 20 client.26842 trim_cache size 0 max 16384 2024-03-18T20:40:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.071+0000 7fd63f0c0b80 1 Processor -- start 2024-03-18T20:40:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.071+0000 7fd63f0c0b80 1 -- start start 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.071+0000 7fd63f0c0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49d72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.071+0000 7fd63f0c0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 0x5650e49d7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.071+0000 7fd63f0c0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49e7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.071+0000 7fd63f0c0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e49d7d70 con 0x5650e49dbdb0 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.071+0000 7fd63f0c0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e49cdd60 con 0x5650e49d0e90 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.071+0000 7fd63f0c0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e49cd760 con 0x5650e49d0570 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd62bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 0x5650e49d7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd6314c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd630cc7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49d72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd6314c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49e7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48880/0 (socket says 172.21.15.46:48880) 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd6314c8700 1 -- 172.21.15.46:0/2250813925 learned_addr learned my addr 172.21.15.46:0/2250813925 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd62bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 0x5650e49d7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33870/0 (socket says 172.21.15.46:33870) 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd630cc7700 1 --2- 172.21.15.46:0/2250813925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49d72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:34500/0 (socket says 172.21.15.46:34500) 2024-03-18T20:40:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd62bfff700 1 -- 172.21.15.46:0/2250813925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 msgr2=0x5650e49d72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd62bfff700 1 --2- 172.21.15.46:0/2250813925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49d72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd62bfff700 1 -- 172.21.15.46:0/2250813925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 msgr2=0x5650e49e7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd62bfff700 1 --2- 172.21.15.46:0/2250813925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49e7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd62bfff700 1 -- 172.21.15.46:0/2250813925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650e48e9e40 con 0x5650e49d0e90 2024-03-18T20:40:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd630cc7700 1 --2- 172.21.15.46:0/2250813925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49d72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd62bfff700 1 --2- 172.21.15.46:0/2250813925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 0x5650e49d7830 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd620004ab0 tx=0x7fd620005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea50b3d50792d25 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd6137fe700 1 -- 172.21.15.46:0/2250813925 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620020030 con 0x5650e49d0e90 2024-03-18T20:40:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd6137fe700 1 -- 172.21.15.46:0/2250813925 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd620021040 con 0x5650e49d0e90 2024-03-18T20:40:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.072+0000 7fd6137fe700 1 -- 172.21.15.46:0/2250813925 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6200038e0 con 0x5650e49d0e90 2024-03-18T20:40:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.073+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/2250813925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 msgr2=0x5650e49d7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.073+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/2250813925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 0x5650e49d7830 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd620004ab0 tx=0x7fd620005d50 comp rx=0 tx=0).stop 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.073+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/2250813925 shutdown_connections 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.073+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/2250813925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49e7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.073+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/2250813925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0570 0x5650e49d72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.073+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/2250813925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0e90 0x5650e49d7830 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.073+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/2250813925 >> 172.21.15.46:0/2250813925 conn(0x5650e49471c0 msgr2=0x5650e49d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.073+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/2250813925 shutdown_connections 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.073+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/2250813925 wait complete. 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd63f0c0b80 1 -- start start 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd63f0c0b80 1 Processor -- start 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd63f0c0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0570 0x5650e49e0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd63f0c0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0e90 0x5650e49e0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd63f0c0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49e0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd63f0c0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e49e2fb0 con 0x5650e49dbdb0 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd63f0c0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e4a3e860 con 0x5650e49d0570 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd63f0c0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650e4a3e9a0 con 0x5650e49d0e90 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd63f0c0b80 10 client.0 _ll_register_callbacks cb 0x7fd61c003710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd630cc7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0570 0x5650e49e0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd630cc7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0570 0x5650e49e0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33886/0 (socket says 172.21.15.46:33886) 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd630cc7700 1 -- 172.21.15.46:0/3363339326 learned_addr learned my addr 172.21.15.46:0/3363339326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd62bfff700 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0e90 0x5650e49e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.074+0000 7fd6314c8700 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49e0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd630cc7700 1 -- 172.21.15.46:0/3363339326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0e90 msgr2=0x5650e49e0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd630cc7700 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0e90 0x5650e49e0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd630cc7700 1 -- 172.21.15.46:0/3363339326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 msgr2=0x5650e49e0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd630cc7700 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49e0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd630cc7700 1 -- 172.21.15.46:0/3363339326 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650e4751b00 con 0x5650e49d0570 2024-03-18T20:40:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd630cc7700 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0570 0x5650e49e0140 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd61c013010 tx=0x7fd61c0306a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b14c5add52fe49 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd60bfff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd60bfff700 1 -- 172.21.15.46:0/3363339326 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c003e90 con 0x5650e49d0570 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd60bfff700 1 -- 172.21.15.46:0/3363339326 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd61c01fa20 con 0x5650e49d0570 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/3363339326 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd620015070 con 0x5650e49d0570 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd60bfff700 1 -- 172.21.15.46:0/3363339326 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd61c01fbe0 con 0x5650e49d0570 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd63f0c0b80 10 client.26834 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/3363339326 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5650e47a5350 con 0x5650e49d0570 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd60bfff700 1 -- 172.21.15.46:0/3363339326 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd620015070 con 0x5650e49d0570 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.075+0000 7fd60bfff700 1 -- 172.21.15.46:0/3363339326 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd61c03c450 con 0x5650e49d0570 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.076+0000 7fd63f0c0b80 10 client.26834 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.076+0000 7fd63f0c0b80 10 client.26834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.076+0000 7fd63f0c0b80 20 client.26834 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.076+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650e4a40170 0x5650e4a42590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.076+0000 7fd63f0c0b80 4 client.26834 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:40:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.076+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/3363339326 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5650e4772c90 con 0x5650e4a40170 2024-03-18T20:40:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.076+0000 7fd62bfff700 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650e4a40170 0x5650e4a42590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.076+0000 7fd60bfff700 10 client.26834 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.076+0000 7fd62bfff700 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650e4a40170 0x5650e4a42590 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76d673b11732c64e server_cookie=740758d74c97b40e in_seq=0 out_seq=0 2024-03-18T20:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd60bfff700 1 -- 172.21.15.46:0/3363339326 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5650e4772c90 con 0x5650e4a40170 2024-03-18T20:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd60bfff700 10 client.26834 handle_command_reply: tid=0 2024-03-18T20:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd63f0c0b80 1 client.26834 shutdown 2024-03-18T20:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/3363339326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0570 msgr2=0x5650e49e0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0570 0x5650e49e0140 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd61c013010 tx=0x7fd61c0306a0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/3363339326 shutdown_connections 2024-03-18T20:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650e4a40170 0x5650e4a42590 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650e49dbdb0 0x5650e49e0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650e49d0570 0x5650e49e0140 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd60bfff700 0 client.26834 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd63f0c0b80 1 --2- 172.21.15.46:0/3363339326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650e49d0e90 0x5650e49e0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/3363339326 >> 172.21.15.46:0/3363339326 conn(0x5650e49471c0 msgr2=0x5650e49d3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/3363339326 shutdown_connections 2024-03-18T20:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.105+0000 7fd63f0c0b80 1 -- 172.21.15.46:0/3363339326 wait complete. 2024-03-18T20:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd63f0c0b80 20 client.26834 trim_cache size 0 max 16384 2024-03-18T20:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:40:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd632f2b700 1 -- 172.21.15.46:0/4053567863 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd6140a1290 msgr2=0x7fd6140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd632f2b700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd6140a1290 0x7fd6140a3750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd61c011490 tx=0x7fd61c009260 comp rx=0 tx=0).stop 2024-03-18T20:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd632f2b700 1 -- 172.21.15.46:0/4053567863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd62c105c60 msgr2=0x7fd62c1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd632f2b700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd62c105c60 0x7fd62c1022d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd62400b070 tx=0x7fd624007930 comp rx=0 tx=0).stop 2024-03-18T20:40:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd632f2b700 1 -- 172.21.15.46:0/4053567863 shutdown_connections 2024-03-18T20:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd632f2b700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd6140a1290 0x7fd6140a3750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd632f2b700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd62c105c60 0x7fd62c1022d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd632f2b700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd62c0ffbb0 0x7fd62c101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd632f2b700 1 --2- 172.21.15.46:0/4053567863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd62c100570 0x7fd62c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd632f2b700 1 -- 172.21.15.46:0/4053567863 >> 172.21.15.46:0/4053567863 conn(0x7fd62c076d50 msgr2=0x7fd62c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd632f2b700 1 -- 172.21.15.46:0/4053567863 shutdown_connections 2024-03-18T20:40:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:34.106+0000 7fd632f2b700 1 -- 172.21.15.46:0/4053567863 wait complete. 2024-03-18T20:40:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:34 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:40:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:34 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:40:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:34 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:40:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:34 smithi073 ceph-mon[47384]: pgmap v137: 97 pgs: 97 active+clean; 53 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 888 KiB/s wr, 130 op/s 2024-03-18T20:40:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:34 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:40:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:34 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:40:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:34 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:40:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:34 smithi155 ceph-mon[48344]: pgmap v137: 97 pgs: 97 active+clean; 53 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 888 KiB/s wr, 130 op/s 2024-03-18T20:40:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:34 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:40:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:34 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:40:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:34 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:40:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:34 smithi046 ceph-mon[43495]: pgmap v137: 97 pgs: 97 active+clean; 53 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 888 KiB/s wr, 130 op/s 2024-03-18T20:40:34.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 116533 2.07 MB/sec execute 89 sec latency 12.518 ms 2024-03-18T20:40:35.685 INFO:tasks.workunit.client.0.smithi046.stdout: 1 117812 2.13 MB/sec execute 90 sec latency 55.744 ms 2024-03-18T20:40:36.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:35 smithi073 ceph-mon[47384]: pgmap v138: 97 pgs: 97 active+clean; 53 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 757 KiB/s wr, 96 op/s 2024-03-18T20:40:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:35 smithi155 ceph-mon[48344]: pgmap v138: 97 pgs: 97 active+clean; 53 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 757 KiB/s wr, 96 op/s 2024-03-18T20:40:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:35 smithi046 ceph-mon[43495]: pgmap v138: 97 pgs: 97 active+clean; 53 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 757 KiB/s wr, 96 op/s 2024-03-18T20:40:36.685 INFO:tasks.workunit.client.0.smithi046.stdout: 1 117812 2.10 MB/sec execute 91 sec latency 1055.868 ms 2024-03-18T20:40:37.685 INFO:tasks.workunit.client.0.smithi046.stdout: 1 119532 2.15 MB/sec execute 92 sec latency 1425.765 ms 2024-03-18T20:40:38.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:38 smithi073 ceph-mon[47384]: pgmap v139: 97 pgs: 97 active+clean; 53 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 757 KiB/s wr, 96 op/s 2024-03-18T20:40:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:38 smithi155 ceph-mon[48344]: pgmap v139: 97 pgs: 97 active+clean; 53 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 757 KiB/s wr, 96 op/s 2024-03-18T20:40:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:38 smithi046 ceph-mon[43495]: pgmap v139: 97 pgs: 97 active+clean; 53 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 757 KiB/s wr, 96 op/s 2024-03-18T20:40:38.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 121087 2.26 MB/sec execute 93 sec latency 14.134 ms 2024-03-18T20:40:39.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 121300 2.24 MB/sec execute 94 sec latency 839.424 ms 2024-03-18T20:40:40.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:40 smithi073 ceph-mon[47384]: pgmap v140: 97 pgs: 97 active+clean; 53 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.5 MiB/s wr, 192 op/s 2024-03-18T20:40:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:40 smithi155 ceph-mon[48344]: pgmap v140: 97 pgs: 97 active+clean; 53 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.5 MiB/s wr, 192 op/s 2024-03-18T20:40:40.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:40 smithi046 ceph-mon[43495]: pgmap v140: 97 pgs: 97 active+clean; 53 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.5 MiB/s wr, 192 op/s 2024-03-18T20:40:40.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 121300 2.21 MB/sec execute 95 sec latency 1839.567 ms 2024-03-18T20:40:41.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 121300 2.19 MB/sec execute 96 sec latency 2839.692 ms 2024-03-18T20:40:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:42 smithi073 ceph-mon[47384]: pgmap v141: 97 pgs: 97 active+clean; 53 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 924 KiB/s wr, 132 op/s 2024-03-18T20:40:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:42 smithi155 ceph-mon[48344]: pgmap v141: 97 pgs: 97 active+clean; 53 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 924 KiB/s wr, 132 op/s 2024-03-18T20:40:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:42 smithi046 ceph-mon[43495]: pgmap v141: 97 pgs: 97 active+clean; 53 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 924 KiB/s wr, 132 op/s 2024-03-18T20:40:42.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 121424 2.17 MB/sec execute 97 sec latency 3198.636 ms 2024-03-18T20:40:43.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 121424 2.15 MB/sec execute 98 sec latency 1551.245 ms 2024-03-18T20:40:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:44 smithi073 ceph-mon[47384]: pgmap v142: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.0 MiB/s wr, 162 op/s 2024-03-18T20:40:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:44 smithi155 ceph-mon[48344]: pgmap v142: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.0 MiB/s wr, 162 op/s 2024-03-18T20:40:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:44 smithi046 ceph-mon[43495]: pgmap v142: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 1.0 MiB/s wr, 162 op/s 2024-03-18T20:40:44.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 121424 2.13 MB/sec execute 99 sec latency 2551.309 ms 2024-03-18T20:40:45.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 121424 2.11 MB/sec execute 100 sec latency 3551.417 ms 2024-03-18T20:40:46.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:45 smithi073 ceph-mon[47384]: pgmap v143: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 877 KiB/s wr, 125 op/s 2024-03-18T20:40:46.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:45 smithi155 ceph-mon[48344]: pgmap v143: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 877 KiB/s wr, 125 op/s 2024-03-18T20:40:46.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:45 smithi046 ceph-mon[43495]: pgmap v143: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 877 KiB/s wr, 125 op/s 2024-03-18T20:40:46.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 121424 2.09 MB/sec execute 101 sec latency 4551.524 ms 2024-03-18T20:40:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.242+0000 7f8a23694700 1 -- 172.21.15.46:0/919978561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1c106e10 msgr2=0x7f8a1c1071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.242+0000 7f8a23694700 1 --2- 172.21.15.46:0/919978561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1c106e10 0x7f8a1c1071f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009990 tx=0x7f8a18009ca0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.242+0000 7f8a23694700 1 -- 172.21.15.46:0/919978561 shutdown_connections 2024-03-18T20:40:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.242+0000 7f8a23694700 1 --2- 172.21.15.46:0/919978561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1c106e10 0x7f8a1c1071f0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.242+0000 7f8a23694700 1 --2- 172.21.15.46:0/919978561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a1c068aa0 0x7f8a1c10dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.242+0000 7f8a23694700 1 --2- 172.21.15.46:0/919978561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a1c0680e0 0x7f8a1c068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.242+0000 7f8a23694700 1 -- 172.21.15.46:0/919978561 >> 172.21.15.46:0/919978561 conn(0x7f8a1c076d50 msgr2=0x7f8a1c077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.242+0000 7f8a23694700 1 -- 172.21.15.46:0/919978561 shutdown_connections 2024-03-18T20:40:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.242+0000 7f8a23694700 1 -- 172.21.15.46:0/919978561 wait complete. 2024-03-18T20:40:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a23694700 1 Processor -- start 2024-03-18T20:40:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a23694700 1 -- start start 2024-03-18T20:40:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a23694700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a1c0680e0 0x7f8a1c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a23694700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1c068aa0 0x7f8a1c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a23694700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a1c106e10 0x7f8a1c19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a23694700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c19e1f0 con 0x7f8a1c068aa0 2024-03-18T20:40:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a23694700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c19e360 con 0x7f8a1c106e10 2024-03-18T20:40:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a23694700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c19e520 con 0x7f8a1c0680e0 2024-03-18T20:40:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a20c2f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1c068aa0 0x7f8a1c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a20c2f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1c068aa0 0x7f8a1c199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33762/0 (socket says 172.21.15.46:33762) 2024-03-18T20:40:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a20c2f700 1 -- 172.21.15.46:0/898622091 learned_addr learned my addr 172.21.15.46:0/898622091 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a21c31700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a1c106e10 0x7f8a1c19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a21430700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a1c0680e0 0x7f8a1c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a20c2f700 1 -- 172.21.15.46:0/898622091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a1c0680e0 msgr2=0x7f8a1c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a1c0680e0 0x7f8a1c1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a20c2f700 1 -- 172.21.15.46:0/898622091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a1c106e10 msgr2=0x7f8a1c19dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a1c106e10 0x7f8a1c19dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a20c2f700 1 -- 172.21.15.46:0/898622091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a14009c40 con 0x7f8a1c068aa0 2024-03-18T20:40:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.243+0000 7f8a21c31700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a1c106e10 0x7f8a1c19dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.244+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1c068aa0 0x7f8a1c199810 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00d950 tx=0x7f8a0c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.244+0000 7f8a127fc700 1 -- 172.21.15.46:0/898622091 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c009930 con 0x7f8a1c068aa0 2024-03-18T20:40:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.244+0000 7f8a23694700 1 -- 172.21.15.46:0/898622091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a180095f0 con 0x7f8a1c068aa0 2024-03-18T20:40:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.244+0000 7f8a127fc700 1 -- 172.21.15.46:0/898622091 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a0c010460 con 0x7f8a1c068aa0 2024-03-18T20:40:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.244+0000 7f8a127fc700 1 -- 172.21.15.46:0/898622091 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c00f620 con 0x7f8a1c068aa0 2024-03-18T20:40:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.244+0000 7f8a23694700 1 -- 172.21.15.46:0/898622091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1c19e9f0 con 0x7f8a1c068aa0 2024-03-18T20:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.246+0000 7f8a127fc700 1 -- 172.21.15.46:0/898622091 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8a0c0108d0 con 0x7f8a1c068aa0 2024-03-18T20:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.247+0000 7f8a127fc700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8a080a1310 0x7f8a080a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.247+0000 7f8a127fc700 1 -- 172.21.15.46:0/898622091 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8a0c0cca20 con 0x7f8a1c068aa0 2024-03-18T20:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.247+0000 7f8a21430700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8a080a1310 0x7f8a080a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.248+0000 7f8a21430700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8a080a1310 0x7f8a080a37d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800b6f0 tx=0x7f8a1800b280 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.248+0000 7f8a2f829b80 1 Processor -- start 2024-03-18T20:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.248+0000 7f8a2f829b80 1 -- start start 2024-03-18T20:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.248+0000 7f8a2f829b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 0x557e4b8c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.248+0000 7f8a2f829b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 0x557e4b8d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a2f829b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 0x557e4b8d61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a2f829b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4b822930 con 0x557e4b8c8570 2024-03-18T20:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a2f829b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4b8202f0 con 0x557e4b8c8e90 2024-03-18T20:40:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a2f829b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4b81bbe0 con 0x557e4b8d3db0 2024-03-18T20:40:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a21430700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 0x557e4b8c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a20c2f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 0x557e4b8d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a20c2f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 0x557e4b8d3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37210/0 (socket says 172.21.15.46:37210) 2024-03-18T20:40:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a20c2f700 1 -- 172.21.15.46:0/3072001571 learned_addr learned my addr 172.21.15.46:0/3072001571 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a21430700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 0x557e4b8c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33764/0 (socket says 172.21.15.46:33764) 2024-03-18T20:40:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a21c31700 1 --2- 172.21.15.46:0/3072001571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 0x557e4b8d61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a20c2f700 1 -- 172.21.15.46:0/3072001571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 msgr2=0x557e4b8d61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/3072001571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 0x557e4b8d61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a20c2f700 1 -- 172.21.15.46:0/3072001571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 msgr2=0x557e4b8c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/3072001571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 0x557e4b8c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a20c2f700 1 -- 172.21.15.46:0/3072001571 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e4b69d350 con 0x557e4b8c8e90 2024-03-18T20:40:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/3072001571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 0x557e4b8d3870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0965b0 tx=0x7f8a0c099570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb15e1bce6e49a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a077fe700 1 -- 172.21.15.46:0/3072001571 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0b9050 con 0x557e4b8c8e90 2024-03-18T20:40:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a077fe700 1 -- 172.21.15.46:0/3072001571 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a0c09d970 con 0x557e4b8c8e90 2024-03-18T20:40:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.249+0000 7f8a077fe700 1 -- 172.21.15.46:0/3072001571 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c099030 con 0x557e4b8c8e90 2024-03-18T20:40:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.250+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3072001571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 msgr2=0x557e4b8d3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.250+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3072001571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 0x557e4b8d3870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0965b0 tx=0x7f8a0c099570 comp rx=0 tx=0).stop 2024-03-18T20:40:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.250+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3072001571 shutdown_connections 2024-03-18T20:40:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.250+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3072001571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 0x557e4b8c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.250+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3072001571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 0x557e4b8d61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.250+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3072001571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 0x557e4b8d3870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.250+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3072001571 >> 172.21.15.46:0/3072001571 conn(0x557e4b83f1c0 msgr2=0x557e4b8c4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.250+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3072001571 shutdown_connections 2024-03-18T20:40:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.250+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3072001571 wait complete. 2024-03-18T20:40:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.251+0000 7f8a2f829b80 1 -- start start 2024-03-18T20:40:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.251+0000 7f8a2f829b80 1 Processor -- start 2024-03-18T20:40:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.251+0000 7f8a2f829b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 0x557e4b8d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.251+0000 7f8a2f829b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 0x557e4b8d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.251+0000 7f8a2f829b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 0x557e4b8d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.251+0000 7f8a2f829b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4b7b7d20 con 0x557e4b8c8570 2024-03-18T20:40:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.251+0000 7f8a2f829b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4b7b79c0 con 0x557e4b8c8e90 2024-03-18T20:40:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.251+0000 7f8a2f829b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4b69f640 con 0x557e4b8d3db0 2024-03-18T20:40:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.251+0000 7f8a2f829b80 10 client.0 _ll_register_callbacks cb 0x557e4b623340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a21430700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 0x557e4b8d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a20c2f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 0x557e4b8d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a21430700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 0x557e4b8d3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33768/0 (socket says 172.21.15.46:33768) 2024-03-18T20:40:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a21430700 1 -- 172.21.15.46:0/3328311869 learned_addr learned my addr 172.21.15.46:0/3328311869 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a21c31700 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 0x557e4b8d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a20c2f700 1 -- 172.21.15.46:0/3328311869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 msgr2=0x557e4b8d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 0x557e4b8d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a20c2f700 1 -- 172.21.15.46:0/3328311869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 msgr2=0x557e4b8d3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 0x557e4b8d3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a20c2f700 1 -- 172.21.15.46:0/3328311869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1c19e9f0 con 0x557e4b8c8e90 2024-03-18T20:40:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a21430700 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 0x557e4b8d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 0x557e4b8d0b60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c012040 tx=0x7f8a0c0980f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6f2b0cb12c31ae server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a04ff9700 1 -- 172.21.15.46:0/3328311869 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c099230 con 0x557e4b8c8e90 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a04ff9700 1 -- 172.21.15.46:0/3328311869 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8a0c099da0 con 0x557e4b8c8e90 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3328311869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e4b8bc6c0 con 0x557e4b8c8e90 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a04ff9700 1 -- 172.21.15.46:0/3328311869 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a0c099f60 con 0x557e4b8c8e90 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a2f829b80 10 client.26846 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.252+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3328311869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557e4b649b00 con 0x557e4b8c8e90 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.253+0000 7f8a04ff9700 1 -- 172.21.15.46:0/3328311869 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e4b8bc6c0 con 0x557e4b8c8e90 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.253+0000 7f8a04ff9700 1 -- 172.21.15.46:0/3328311869 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8a0c07e020 con 0x557e4b8c8e90 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.253+0000 7f8a2f829b80 10 client.26846 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.253+0000 7f8a2f829b80 10 client.26846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.253+0000 7f8a2f829b80 20 client.26846 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.253+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e4b8e34a0 0x557e4b8e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.253+0000 7f8a2f829b80 4 client.26846 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.253+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3328311869 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e4b8146a0 con 0x557e4b8e34a0 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.253+0000 7f8a21430700 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e4b8e34a0 0x557e4b8e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.253+0000 7f8a04ff9700 10 client.26846 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.253+0000 7f8a21430700 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e4b8e34a0 0x557e4b8e58c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=235746d9607c2d46 server_cookie=a56eca9883b55077 in_seq=0 out_seq=0 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a04ff9700 1 -- 172.21.15.46:0/3328311869 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557e4b8146a0 con 0x557e4b8e34a0 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a04ff9700 10 client.26846 handle_command_reply: tid=0 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a2f829b80 1 client.26846 shutdown 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3328311869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 msgr2=0x557e4b8d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 0x557e4b8d0b60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c012040 tx=0x7f8a0c0980f0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3328311869 shutdown_connections 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e4b8e34a0 0x557e4b8e58c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8570 0x557e4b8d3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 0x557e4b8d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a04ff9700 0 client.26846 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3328311869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8e90 0x557e4b8d0b60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3328311869 >> 172.21.15.46:0/3328311869 conn(0x557e4b83f1c0 msgr2=0x557e4b8c4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3328311869 shutdown_connections 2024-03-18T20:40:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.254+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3328311869 wait complete. 2024-03-18T20:40:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.255+0000 7f8a2f829b80 20 client.26846 trim_cache size 0 max 16384 2024-03-18T20:40:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a2f829b80 1 Processor -- start 2024-03-18T20:40:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a2f829b80 1 -- start start 2024-03-18T20:40:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a2f829b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8570 0x557e4b8da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a2f829b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 0x557e4b8cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a2f829b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 0x557e4b8cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a2f829b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4b8ca070 con 0x557e4b8c8e90 2024-03-18T20:40:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a2f829b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4b8bec10 con 0x557e4b8c8570 2024-03-18T20:40:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a2f829b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4b8cb980 con 0x557e4b8d3db0 2024-03-18T20:40:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a21430700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8570 0x557e4b8da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a20c2f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 0x557e4b8cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a20c2f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 0x557e4b8cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33774/0 (socket says 172.21.15.46:33774) 2024-03-18T20:40:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a21430700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8570 0x557e4b8da9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37230/0 (socket says 172.21.15.46:37230) 2024-03-18T20:40:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a20c2f700 1 -- 172.21.15.46:0/3977270648 learned_addr learned my addr 172.21.15.46:0/3977270648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.271+0000 7f8a21c31700 1 --2- 172.21.15.46:0/3977270648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 0x557e4b8cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.272+0000 7f8a20c2f700 1 -- 172.21.15.46:0/3977270648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 msgr2=0x557e4b8cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.272+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/3977270648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 0x557e4b8cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.272+0000 7f8a20c2f700 1 -- 172.21.15.46:0/3977270648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8570 msgr2=0x557e4b8da9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.272+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/3977270648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8570 0x557e4b8da9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.272+0000 7f8a20c2f700 1 -- 172.21.15.46:0/3977270648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e4b7e1e40 con 0x557e4b8c8e90 2024-03-18T20:40:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.272+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/3977270648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 0x557e4b8cbe50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c099840 tx=0x7f8a0c010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6d0d713168bd22 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.272+0000 7f8a07fff700 1 -- 172.21.15.46:0/3977270648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0b9070 con 0x557e4b8c8e90 2024-03-18T20:40:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.272+0000 7f8a07fff700 1 -- 172.21.15.46:0/3977270648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a0c0830f0 con 0x557e4b8c8e90 2024-03-18T20:40:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.272+0000 7f8a07fff700 1 -- 172.21.15.46:0/3977270648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0a1b30 con 0x557e4b8c8e90 2024-03-18T20:40:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.273+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3977270648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 msgr2=0x557e4b8cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.273+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3977270648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 0x557e4b8cbe50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c099840 tx=0x7f8a0c010bd0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.273+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3977270648 shutdown_connections 2024-03-18T20:40:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.273+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3977270648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8d3db0 0x557e4b8cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.273+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3977270648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8c8570 0x557e4b8da9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.273+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/3977270648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 0x557e4b8cbe50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.273+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3977270648 >> 172.21.15.46:0/3977270648 conn(0x557e4b83f1c0 msgr2=0x557e4b8c3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.273+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3977270648 shutdown_connections 2024-03-18T20:40:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.273+0000 7f8a2f829b80 1 -- 172.21.15.46:0/3977270648 wait complete. 2024-03-18T20:40:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.273+0000 7f8a2f829b80 1 -- start start 2024-03-18T20:40:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.273+0000 7f8a2f829b80 1 Processor -- start 2024-03-18T20:40:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a2f829b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8c8570 0x557e4b8d89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a2f829b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 0x557e4b8d8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a2f829b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8d3db0 0x557e4b8d9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a2f829b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4b936230 con 0x557e4b8c8e90 2024-03-18T20:40:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a2f829b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4b936370 con 0x557e4b8d3db0 2024-03-18T20:40:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a2f829b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4b936500 con 0x557e4b8c8570 2024-03-18T20:40:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a2f829b80 10 client.0 _ll_register_callbacks cb 0x7f8a18005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a20c2f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 0x557e4b8d8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a21430700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8c8570 0x557e4b8d89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a21430700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8c8570 0x557e4b8d89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:49584/0 (socket says 172.21.15.46:49584) 2024-03-18T20:40:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a21c31700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8d3db0 0x557e4b8d9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a21430700 1 -- 172.21.15.46:0/2468857447 learned_addr learned my addr 172.21.15.46:0/2468857447 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a20c2f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 0x557e4b8d8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33780/0 (socket says 172.21.15.46:33780) 2024-03-18T20:40:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a21c31700 1 -- 172.21.15.46:0/2468857447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8c8570 msgr2=0x557e4b8d89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a21c31700 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8c8570 0x557e4b8d89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a21c31700 1 -- 172.21.15.46:0/2468857447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 msgr2=0x557e4b8d8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a21c31700 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 0x557e4b8d8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a21c31700 1 -- 172.21.15.46:0/2468857447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e4b649b00 con 0x557e4b8d3db0 2024-03-18T20:40:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a21430700 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8c8570 0x557e4b8d89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.274+0000 7f8a20c2f700 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 0x557e4b8d8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a21c31700 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8d3db0 0x557e4b8d9440 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a14007bd0 tx=0x7f8a14007ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7224add2c38ef552 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:40:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a04ff9700 1 -- 172.21.15.46:0/2468857447 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400e4f0 con 0x557e4b8d3db0 2024-03-18T20:40:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a2f829b80 1 -- 172.21.15.46:0/2468857447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a0c09d970 con 0x557e4b8d3db0 2024-03-18T20:40:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a04ff9700 1 -- 172.21.15.46:0/2468857447 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8a1400e650 con 0x557e4b8d3db0 2024-03-18T20:40:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a04ff9700 1 -- 172.21.15.46:0/2468857447 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a1400e810 con 0x557e4b8d3db0 2024-03-18T20:40:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a2f829b80 10 client.26852 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a2f829b80 1 -- 172.21.15.46:0/2468857447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557e4b6a1c70 con 0x557e4b8d3db0 2024-03-18T20:40:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a04ff9700 1 -- 172.21.15.46:0/2468857447 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a0c09d970 con 0x557e4b8d3db0 2024-03-18T20:40:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a04ff9700 1 -- 172.21.15.46:0/2468857447 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8a14028070 con 0x557e4b8d3db0 2024-03-18T20:40:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a2f829b80 10 client.26852 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a2f829b80 10 client.26852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a2f829b80 20 client.26852 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e4b937d20 0x557e4b93a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a2f829b80 4 client.26852 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:40:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.275+0000 7f8a2f829b80 1 -- 172.21.15.46:0/2468857447 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557e4b66ac90 con 0x557e4b937d20 2024-03-18T20:40:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.276+0000 7f8a21430700 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e4b937d20 0x557e4b93a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.276+0000 7f8a04ff9700 10 client.26852 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.276+0000 7f8a21430700 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e4b937d20 0x557e4b93a140 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac95962c6f2e357 server_cookie=6deae06676bb0a1f in_seq=0 out_seq=0 2024-03-18T20:40:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a04ff9700 1 -- 172.21.15.46:0/2468857447 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557e4b66ac90 con 0x557e4b937d20 2024-03-18T20:40:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a04ff9700 10 client.26852 handle_command_reply: tid=0 2024-03-18T20:40:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a2f829b80 1 client.26852 shutdown 2024-03-18T20:40:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a2f829b80 1 -- 172.21.15.46:0/2468857447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8d3db0 msgr2=0x557e4b8d9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8d3db0 0x557e4b8d9440 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a14007bd0 tx=0x7f8a14007ee0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a2f829b80 1 -- 172.21.15.46:0/2468857447 shutdown_connections 2024-03-18T20:40:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e4b937d20 0x557e4b93a140 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4b8d3db0 0x557e4b8d9440 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e4b8c8570 0x557e4b8d89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a04ff9700 0 client.26852 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a2f829b80 1 --2- 172.21.15.46:0/2468857447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e4b8c8e90 0x557e4b8d8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a2f829b80 1 -- 172.21.15.46:0/2468857447 >> 172.21.15.46:0/2468857447 conn(0x557e4b83f1c0 msgr2=0x557e4b8cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.302+0000 7f8a2f829b80 1 -- 172.21.15.46:0/2468857447 shutdown_connections 2024-03-18T20:40:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a2f829b80 1 -- 172.21.15.46:0/2468857447 wait complete. 2024-03-18T20:40:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a2f829b80 20 client.26852 trim_cache size 0 max 16384 2024-03-18T20:40:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:40:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:40:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:40:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:40:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a23694700 1 -- 172.21.15.46:0/898622091 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8a080a1310 msgr2=0x7f8a080a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a23694700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8a080a1310 0x7f8a080a37d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800b6f0 tx=0x7f8a1800b280 comp rx=0 tx=0).stop 2024-03-18T20:40:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a23694700 1 -- 172.21.15.46:0/898622091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1c068aa0 msgr2=0x7f8a1c199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a23694700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1c068aa0 0x7f8a1c199810 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00d950 tx=0x7f8a0c00dc60 comp rx=0 tx=0).stop 2024-03-18T20:40:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a23694700 1 -- 172.21.15.46:0/898622091 shutdown_connections 2024-03-18T20:40:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a23694700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8a080a1310 0x7f8a080a37d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a23694700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a1c106e10 0x7f8a1c19dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a23694700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a1c0680e0 0x7f8a1c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a23694700 1 --2- 172.21.15.46:0/898622091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1c068aa0 0x7f8a1c199810 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a23694700 1 -- 172.21.15.46:0/898622091 >> 172.21.15.46:0/898622091 conn(0x7f8a1c076d50 msgr2=0x7f8a1c10be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a23694700 1 -- 172.21.15.46:0/898622091 shutdown_connections 2024-03-18T20:40:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:40:47.303+0000 7f8a23694700 1 -- 172.21.15.46:0/898622091 wait complete. 2024-03-18T20:40:47.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 123087 2.13 MB/sec execute 102 sec latency 4901.695 ms 2024-03-18T20:40:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:48 smithi073 ceph-mon[47384]: pgmap v144: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 877 KiB/s wr, 125 op/s 2024-03-18T20:40:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:48 smithi155 ceph-mon[48344]: pgmap v144: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 877 KiB/s wr, 125 op/s 2024-03-18T20:40:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:48 smithi046 ceph-mon[43495]: pgmap v144: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 877 KiB/s wr, 125 op/s 2024-03-18T20:40:48.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 124715 2.23 MB/sec execute 103 sec latency 13.775 ms 2024-03-18T20:40:49.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 124973 2.21 MB/sec execute 104 sec latency 819.861 ms 2024-03-18T20:40:50.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:50 smithi073 ceph-mon[47384]: pgmap v145: 97 pgs: 97 active+clean; 52 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-03-18T20:40:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:50 smithi155 ceph-mon[48344]: pgmap v145: 97 pgs: 97 active+clean; 52 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-03-18T20:40:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:50 smithi046 ceph-mon[43495]: pgmap v145: 97 pgs: 97 active+clean; 52 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.3 MiB/s wr, 171 op/s 2024-03-18T20:40:50.686 INFO:tasks.workunit.client.0.smithi046.stdout: 1 124973 2.19 MB/sec execute 105 sec latency 1819.934 ms 2024-03-18T20:40:51.687 INFO:tasks.workunit.client.0.smithi046.stdout: 1 124973 2.17 MB/sec execute 106 sec latency 2820.050 ms 2024-03-18T20:40:52.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:52 smithi073 ceph-mon[47384]: pgmap v146: 97 pgs: 97 active+clean; 52 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 543 KiB/s wr, 75 op/s 2024-03-18T20:40:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:52 smithi155 ceph-mon[48344]: pgmap v146: 97 pgs: 97 active+clean; 52 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 543 KiB/s wr, 75 op/s 2024-03-18T20:40:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:52 smithi046 ceph-mon[43495]: pgmap v146: 97 pgs: 97 active+clean; 52 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 543 KiB/s wr, 75 op/s 2024-03-18T20:40:52.687 INFO:tasks.workunit.client.0.smithi046.stdout: 1 126745 2.22 MB/sec execute 107 sec latency 3176.732 ms 2024-03-18T20:40:53.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:40:53.276 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:40:53.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.425+0000 7efc9deef700 1 -- 172.21.15.46:0/3369585984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9810d750 msgr2=0x7efc9810fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.425+0000 7efc9deef700 1 --2- 172.21.15.46:0/3369585984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9810d750 0x7efc9810fb40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efc90009f90 tx=0x7efc9000b330 comp rx=0 tx=0).stop 2024-03-18T20:40:53.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.425+0000 7efc9deef700 1 -- 172.21.15.46:0/3369585984 shutdown_connections 2024-03-18T20:40:53.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.425+0000 7efc9deef700 1 --2- 172.21.15.46:0/3369585984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc98068fd0 0x7efc980693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.425+0000 7efc9deef700 1 --2- 172.21.15.46:0/3369585984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9810d750 0x7efc9810fb40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.425+0000 7efc9deef700 1 --2- 172.21.15.46:0/3369585984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc980698f0 0x7efc9810d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.425+0000 7efc9deef700 1 -- 172.21.15.46:0/3369585984 >> 172.21.15.46:0/3369585984 conn(0x7efc98076d80 msgr2=0x7efc98077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:53.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.425+0000 7efc9deef700 1 -- 172.21.15.46:0/3369585984 shutdown_connections 2024-03-18T20:40:53.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.425+0000 7efc9deef700 1 -- 172.21.15.46:0/3369585984 wait complete. 2024-03-18T20:40:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.425+0000 7efc9deef700 1 Processor -- start 2024-03-18T20:40:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.426+0000 7efc9deef700 1 -- start start 2024-03-18T20:40:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.426+0000 7efc9deef700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98068fd0 0x7efc981995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.426+0000 7efc9deef700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc980698f0 0x7efc98199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.426+0000 7efc9deef700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc9810d750 0x7efc9819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.426+0000 7efc9deef700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9819e4e0 con 0x7efc9810d750 2024-03-18T20:40:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.426+0000 7efc9deef700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9819e650 con 0x7efc98068fd0 2024-03-18T20:40:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.426+0000 7efc9deef700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9819e810 con 0x7efc980698f0 2024-03-18T20:40:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.426+0000 7efc977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98068fd0 0x7efc981995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.426+0000 7efc97fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc9810d750 0x7efc9819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.426+0000 7efc977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98068fd0 0x7efc981995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37260/0 (socket says 172.21.15.46:37260) 2024-03-18T20:40:53.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.426+0000 7efc977fe700 1 -- 172.21.15.46:0/2178628444 learned_addr learned my addr 172.21.15.46:0/2178628444 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.426+0000 7efc96ffd700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc980698f0 0x7efc98199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.427+0000 7efc977fe700 1 -- 172.21.15.46:0/2178628444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc980698f0 msgr2=0x7efc98199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.427+0000 7efc977fe700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc980698f0 0x7efc98199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.427+0000 7efc977fe700 1 -- 172.21.15.46:0/2178628444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc9810d750 msgr2=0x7efc9819dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.427+0000 7efc977fe700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc9810d750 0x7efc9819dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.427+0000 7efc977fe700 1 -- 172.21.15.46:0/2178628444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc90009c40 con 0x7efc98068fd0 2024-03-18T20:40:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.427+0000 7efc977fe700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98068fd0 0x7efc981995c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00e9c0 tx=0x7efc8c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.427+0000 7efc94ff9700 1 -- 172.21.15.46:0/2178628444 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c004a30 con 0x7efc98068fd0 2024-03-18T20:40:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.427+0000 7efc9deef700 1 -- 172.21.15.46:0/2178628444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9819eaf0 con 0x7efc98068fd0 2024-03-18T20:40:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.427+0000 7efc94ff9700 1 -- 172.21.15.46:0/2178628444 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc8c00cc80 con 0x7efc98068fd0 2024-03-18T20:40:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.427+0000 7efc94ff9700 1 -- 172.21.15.46:0/2178628444 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0054e0 con 0x7efc98068fd0 2024-03-18T20:40:53.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.427+0000 7efc9deef700 1 -- 172.21.15.46:0/2178628444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc981a6330 con 0x7efc98068fd0 2024-03-18T20:40:53.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.429+0000 7efc94ff9700 1 -- 172.21.15.46:0/2178628444 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7efc8c004b90 con 0x7efc98068fd0 2024-03-18T20:40:53.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.430+0000 7efc94ff9700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efc800a1290 0x7efc800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.430+0000 7efc94ff9700 1 -- 172.21.15.46:0/2178628444 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7efc8c014070 con 0x7efc98068fd0 2024-03-18T20:40:53.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.430+0000 7efc96ffd700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efc800a1290 0x7efc800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.431+0000 7efc96ffd700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efc800a1290 0x7efc800a3750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc9819ab10 tx=0x7efc88009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:53.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.431+0000 7efcaa084b80 1 Processor -- start 2024-03-18T20:40:53.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.431+0000 7efcaa084b80 1 -- start start 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.431+0000 7efcaa084b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 0x56193c18be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.431+0000 7efcaa084b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18c380 0x56193c196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.431+0000 7efcaa084b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c199610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.431+0000 7efcaa084b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193c0e5990 con 0x56193c18ba60 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.431+0000 7efcaa084b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193c0e3350 con 0x56193c18c380 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.431+0000 7efcaa084b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193c0dec40 con 0x56193c197220 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 0x56193c18be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 0x56193c18be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33800/0 (socket says 172.21.15.46:33800) 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc977fe700 1 -- 172.21.15.46:0/2706158642 learned_addr learned my addr 172.21.15.46:0/2706158642 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc97fff700 1 --2- 172.21.15.46:0/2706158642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc96ffd700 1 --2- 172.21.15.46:0/2706158642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18c380 0x56193c196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc977fe700 1 -- 172.21.15.46:0/2706158642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 msgr2=0x56193c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc977fe700 1 --2- 172.21.15.46:0/2706158642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc977fe700 1 -- 172.21.15.46:0/2706158642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18c380 msgr2=0x56193c196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc977fe700 1 --2- 172.21.15.46:0/2706158642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18c380 0x56193c196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc977fe700 1 -- 172.21.15.46:0/2706158642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56193bf603b0 con 0x56193c18ba60 2024-03-18T20:40:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc96ffd700 1 --2- 172.21.15.46:0/2706158642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18c380 0x56193c196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc97fff700 1 --2- 172.21.15.46:0/2706158642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc977fe700 1 --2- 172.21.15.46:0/2706158642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 0x56193c18be40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7efc8c096390 tx=0x7efc8c0998c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9380bed084de3c9d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc7dffb700 1 -- 172.21.15.46:0/2706158642 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0b1070 con 0x56193c18ba60 2024-03-18T20:40:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc7dffb700 1 -- 172.21.15.46:0/2706158642 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc8c09dcc0 con 0x56193c18ba60 2024-03-18T20:40:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.432+0000 7efc7dffb700 1 -- 172.21.15.46:0/2706158642 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c09aea0 con 0x56193c18ba60 2024-03-18T20:40:53.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.433+0000 7efcaa084b80 1 -- 172.21.15.46:0/2706158642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 msgr2=0x56193c18be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.433+0000 7efcaa084b80 1 --2- 172.21.15.46:0/2706158642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 0x56193c18be40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7efc8c096390 tx=0x7efc8c0998c0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.433+0000 7efcaa084b80 1 -- 172.21.15.46:0/2706158642 shutdown_connections 2024-03-18T20:40:53.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.433+0000 7efcaa084b80 1 --2- 172.21.15.46:0/2706158642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 0x56193c18be40 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.433+0000 7efcaa084b80 1 --2- 172.21.15.46:0/2706158642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.433+0000 7efcaa084b80 1 --2- 172.21.15.46:0/2706158642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18c380 0x56193c196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.433+0000 7efcaa084b80 1 -- 172.21.15.46:0/2706158642 >> 172.21.15.46:0/2706158642 conn(0x56193c102220 msgr2=0x56193c188020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.433+0000 7efcaa084b80 1 -- 172.21.15.46:0/2706158642 shutdown_connections 2024-03-18T20:40:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.433+0000 7efcaa084b80 1 -- 172.21.15.46:0/2706158642 wait complete. 2024-03-18T20:40:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efcaa084b80 1 -- start start 2024-03-18T20:40:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efcaa084b80 1 Processor -- start 2024-03-18T20:40:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efcaa084b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 0x56193c1940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efcaa084b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18c380 0x56193c1945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efcaa084b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c194b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efcaa084b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193c07ad80 con 0x56193c18c380 2024-03-18T20:40:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efcaa084b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193c07aa20 con 0x56193c18ba60 2024-03-18T20:40:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efcaa084b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193bf626a0 con 0x56193c197220 2024-03-18T20:40:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efcaa084b80 10 client.0 _ll_register_callbacks cb 0x56193bee63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efc96ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18c380 0x56193c1945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efc977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 0x56193c1940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efc96ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18c380 0x56193c1945f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33812/0 (socket says 172.21.15.46:33812) 2024-03-18T20:40:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efc977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 0x56193c1940b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37284/0 (socket says 172.21.15.46:37284) 2024-03-18T20:40:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efc96ffd700 1 -- 172.21.15.46:0/3363686328 learned_addr learned my addr 172.21.15.46:0/3363686328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.434+0000 7efc97fff700 1 --2- 172.21.15.46:0/3363686328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c194b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efc977fe700 1 -- 172.21.15.46:0/3363686328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 msgr2=0x56193c194b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efc977fe700 1 --2- 172.21.15.46:0/3363686328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c194b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efc977fe700 1 -- 172.21.15.46:0/3363686328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18c380 msgr2=0x56193c1945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efc977fe700 1 --2- 172.21.15.46:0/3363686328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18c380 0x56193c1945f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efc977fe700 1 -- 172.21.15.46:0/3363686328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc98110670 con 0x56193c18ba60 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efc977fe700 1 --2- 172.21.15.46:0/3363686328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 0x56193c1940b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc8c096390 tx=0x7efc8c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6859ad820db760dc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efc76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efc76ffd700 1 -- 172.21.15.46:0/3363686328 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0b1070 con 0x56193c18ba60 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efc76ffd700 1 -- 172.21.15.46:0/3363686328 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7efc8c09a0f0 con 0x56193c18ba60 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efcaa084b80 1 -- 172.21.15.46:0/3363686328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56193c17fad0 con 0x56193c18ba60 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efc76ffd700 1 -- 172.21.15.46:0/3363686328 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc8c09aea0 con 0x56193c18ba60 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efcaa084b80 10 client.26864 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efcaa084b80 1 -- 172.21.15.46:0/3363686328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7efc981a6330 con 0x56193c18ba60 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efc76ffd700 1 -- 172.21.15.46:0/3363686328 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc8c065030 con 0x56193c18ba60 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.435+0000 7efc76ffd700 1 -- 172.21.15.46:0/3363686328 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efc8c0a1e80 con 0x56193c18ba60 2024-03-18T20:40:53.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.436+0000 7efcaa084b80 10 client.26864 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.436+0000 7efcaa084b80 10 client.26864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.436+0000 7efcaa084b80 20 client.26864 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.436+0000 7efcaa084b80 1 --2- 172.21.15.46:0/3363686328 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56193c1a6ab0 0x56193c1a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.436+0000 7efcaa084b80 4 client.26864 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.436+0000 7efcaa084b80 1 -- 172.21.15.46:0/3363686328 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56193c0d7700 con 0x56193c1a6ab0 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.436+0000 7efc96ffd700 1 --2- 172.21.15.46:0/3363686328 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56193c1a6ab0 0x56193c1a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.436+0000 7efc76ffd700 10 client.26864 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.436+0000 7efc96ffd700 1 --2- 172.21.15.46:0/3363686328 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56193c1a6ab0 0x56193c1a8ea0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62adc27370be040 server_cookie=d8fba7847f7aa8dc in_seq=0 out_seq=0 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efc76ffd700 1 -- 172.21.15.46:0/3363686328 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56193c0d7700 con 0x56193c1a6ab0 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efc76ffd700 10 client.26864 handle_command_reply: tid=0 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efcaa084b80 1 client.26864 shutdown 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efcaa084b80 1 -- 172.21.15.46:0/3363686328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 msgr2=0x56193c1940b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efcaa084b80 1 --2- 172.21.15.46:0/3363686328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 0x56193c1940b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc8c096390 tx=0x7efc8c005c00 comp rx=0 tx=0).stop 2024-03-18T20:40:53.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efcaa084b80 1 -- 172.21.15.46:0/3363686328 shutdown_connections 2024-03-18T20:40:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efcaa084b80 1 --2- 172.21.15.46:0/3363686328 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56193c1a6ab0 0x56193c1a8ea0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efcaa084b80 1 --2- 172.21.15.46:0/3363686328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c194b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efcaa084b80 1 --2- 172.21.15.46:0/3363686328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 0x56193c1940b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efcaa084b80 1 --2- 172.21.15.46:0/3363686328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18c380 0x56193c1945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efcaa084b80 1 -- 172.21.15.46:0/3363686328 >> 172.21.15.46:0/3363686328 conn(0x56193c102220 msgr2=0x56193c187c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efc76ffd700 0 client.26864 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efcaa084b80 1 -- 172.21.15.46:0/3363686328 shutdown_connections 2024-03-18T20:40:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efcaa084b80 1 -- 172.21.15.46:0/3363686328 wait complete. 2024-03-18T20:40:53.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.437+0000 7efcaa084b80 20 client.26864 trim_cache size 0 max 16384 2024-03-18T20:40:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.456+0000 7efcaa084b80 1 Processor -- start 2024-03-18T20:40:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.456+0000 7efcaa084b80 1 -- start start 2024-03-18T20:40:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.456+0000 7efcaa084b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 0x56193c19de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.456+0000 7efcaa084b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18c380 0x56193c194950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.456+0000 7efcaa084b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c18f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.456+0000 7efcaa084b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193c18cd30 con 0x56193c18c380 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.456+0000 7efcaa084b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193c1845e0 con 0x56193c18ba60 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.456+0000 7efcaa084b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193c18d440 con 0x56193c197220 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.456+0000 7efc96ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18c380 0x56193c194950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 0x56193c19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc97fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c18f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc96ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18c380 0x56193c194950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33814/0 (socket says 172.21.15.46:33814) 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc96ffd700 1 -- 172.21.15.46:0/1598200733 learned_addr learned my addr 172.21.15.46:0/1598200733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc977fe700 1 -- 172.21.15.46:0/1598200733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 msgr2=0x56193c18f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc977fe700 1 --2- 172.21.15.46:0/1598200733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c18f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc977fe700 1 -- 172.21.15.46:0/1598200733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18c380 msgr2=0x56193c194950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc977fe700 1 --2- 172.21.15.46:0/1598200733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18c380 0x56193c194950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc977fe700 1 -- 172.21.15.46:0/1598200733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56193bf0cb60 con 0x56193c18ba60 2024-03-18T20:40:53.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc977fe700 1 --2- 172.21.15.46:0/1598200733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 0x56193c19de50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc8c099e20 tx=0x7efc8c005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71714b1ec4325092 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:53.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc7e7fc700 1 -- 172.21.15.46:0/1598200733 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c09aea0 con 0x56193c18ba60 2024-03-18T20:40:53.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc7e7fc700 1 -- 172.21.15.46:0/1598200733 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc8c065070 con 0x56193c18ba60 2024-03-18T20:40:53.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.457+0000 7efc7e7fc700 1 -- 172.21.15.46:0/1598200733 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c083910 con 0x56193c18ba60 2024-03-18T20:40:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.458+0000 7efcaa084b80 1 -- 172.21.15.46:0/1598200733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 msgr2=0x56193c19de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.458+0000 7efcaa084b80 1 --2- 172.21.15.46:0/1598200733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 0x56193c19de50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc8c099e20 tx=0x7efc8c005c70 comp rx=0 tx=0).stop 2024-03-18T20:40:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.458+0000 7efcaa084b80 1 -- 172.21.15.46:0/1598200733 shutdown_connections 2024-03-18T20:40:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.458+0000 7efcaa084b80 1 --2- 172.21.15.46:0/1598200733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c197220 0x56193c18f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.458+0000 7efcaa084b80 1 --2- 172.21.15.46:0/1598200733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c18ba60 0x56193c19de50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.458+0000 7efcaa084b80 1 --2- 172.21.15.46:0/1598200733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18c380 0x56193c194950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.458+0000 7efcaa084b80 1 -- 172.21.15.46:0/1598200733 >> 172.21.15.46:0/1598200733 conn(0x56193c102220 msgr2=0x56193c1a25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.458+0000 7efcaa084b80 1 -- 172.21.15.46:0/1598200733 shutdown_connections 2024-03-18T20:40:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.458+0000 7efcaa084b80 1 -- 172.21.15.46:0/1598200733 wait complete. 2024-03-18T20:40:53.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.458+0000 7efcaa084b80 1 -- start start 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.459+0000 7efcaa084b80 1 Processor -- start 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.459+0000 7efcaa084b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 0x56193c19c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.459+0000 7efcaa084b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c18c380 0x56193c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.459+0000 7efcaa084b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c197220 0x56193c1f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.459+0000 7efcaa084b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193c19cc70 con 0x56193c18ba60 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.459+0000 7efcaa084b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193c1f9df0 con 0x56193c197220 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.459+0000 7efcaa084b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193c1f9f30 con 0x56193c18c380 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.459+0000 7efcaa084b80 10 client.0 _ll_register_callbacks cb 0x7efc880119e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.459+0000 7efc977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 0x56193c19c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.459+0000 7efc97fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c197220 0x56193c1f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.459+0000 7efc977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 0x56193c19c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33828/0 (socket says 172.21.15.46:33828) 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.459+0000 7efc977fe700 1 -- 172.21.15.46:0/2651605552 learned_addr learned my addr 172.21.15.46:0/2651605552 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efc96ffd700 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c18c380 0x56193c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efc97fff700 1 -- 172.21.15.46:0/2651605552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c18c380 msgr2=0x56193c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:53.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efc97fff700 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c18c380 0x56193c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efc97fff700 1 -- 172.21.15.46:0/2651605552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 msgr2=0x56193c19c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efc97fff700 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 0x56193c19c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efc97fff700 1 -- 172.21.15.46:0/2651605552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc981a6330 con 0x56193c197220 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efc977fe700 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 0x56193c19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efc97fff700 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c197220 0x56193c1f98b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc90009790 tx=0x7efc90003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0cd2a2277e045f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efc76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efc76ffd700 1 -- 172.21.15.46:0/2651605552 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9000e040 con 0x56193c197220 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efc76ffd700 1 -- 172.21.15.46:0/2651605552 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7efc9000be00 con 0x56193c197220 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efcaa084b80 1 -- 172.21.15.46:0/2651605552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc8c09dcc0 con 0x56193c197220 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efc76ffd700 1 -- 172.21.15.46:0/2651605552 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc900079d0 con 0x56193c197220 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efcaa084b80 10 client.26870 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.460+0000 7efcaa084b80 1 -- 172.21.15.46:0/2651605552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56193c0a4ea0 con 0x56193c197220 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.461+0000 7efc76ffd700 1 -- 172.21.15.46:0/2651605552 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc8c09dcc0 con 0x56193c197220 2024-03-18T20:40:53.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.461+0000 7efc76ffd700 1 -- 172.21.15.46:0/2651605552 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efc9000b050 con 0x56193c197220 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.461+0000 7efcaa084b80 10 client.26870 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.461+0000 7efcaa084b80 10 client.26870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.461+0000 7efcaa084b80 20 client.26870 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.461+0000 7efcaa084b80 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56193c1fb650 0x56193c1fda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.461+0000 7efcaa084b80 4 client.26870 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.461+0000 7efcaa084b80 1 -- 172.21.15.46:0/2651605552 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56193bf2dcf0 con 0x56193c1fb650 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.461+0000 7efc977fe700 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56193c1fb650 0x56193c1fda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.461+0000 7efc76ffd700 10 client.26870 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.461+0000 7efc977fe700 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56193c1fb650 0x56193c1fda70 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a01d14cc2a0f1865 server_cookie=391fffffe340bde5 in_seq=0 out_seq=0 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efc76ffd700 1 -- 172.21.15.46:0/2651605552 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56193bf2dcf0 con 0x56193c1fb650 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efc76ffd700 10 client.26870 handle_command_reply: tid=0 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efcaa084b80 1 client.26870 shutdown 2024-03-18T20:40:53.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efcaa084b80 1 -- 172.21.15.46:0/2651605552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c197220 msgr2=0x56193c1f98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efcaa084b80 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c197220 0x56193c1f98b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc90009790 tx=0x7efc90003d70 comp rx=0 tx=0).stop 2024-03-18T20:40:53.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efcaa084b80 1 -- 172.21.15.46:0/2651605552 shutdown_connections 2024-03-18T20:40:53.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efcaa084b80 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56193c1fb650 0x56193c1fda70 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efcaa084b80 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56193c18ba60 0x56193c19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efcaa084b80 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56193c197220 0x56193c1f98b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efc76ffd700 0 client.26870 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:53.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efcaa084b80 1 --2- 172.21.15.46:0/2651605552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56193c18c380 0x56193c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efcaa084b80 1 -- 172.21.15.46:0/2651605552 >> 172.21.15.46:0/2651605552 conn(0x56193c102220 msgr2=0x56193c18f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:53.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efcaa084b80 1 -- 172.21.15.46:0/2651605552 shutdown_connections 2024-03-18T20:40:53.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efcaa084b80 1 -- 172.21.15.46:0/2651605552 wait complete. 2024-03-18T20:40:53.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.462+0000 7efcaa084b80 20 client.26870 trim_cache size 0 max 16384 2024-03-18T20:40:53.471 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:40:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:40:53.471 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:40:53.471 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:40:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.463+0000 7efc9deef700 1 -- 172.21.15.46:0/2178628444 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efc800a1290 msgr2=0x7efc800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.463+0000 7efc9deef700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efc800a1290 0x7efc800a3750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc9819ab10 tx=0x7efc88009260 comp rx=0 tx=0).stop 2024-03-18T20:40:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.463+0000 7efc9deef700 1 -- 172.21.15.46:0/2178628444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98068fd0 msgr2=0x7efc981995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.463+0000 7efc9deef700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98068fd0 0x7efc981995c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00e9c0 tx=0x7efc8c00ecd0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.463+0000 7efc9deef700 1 -- 172.21.15.46:0/2178628444 shutdown_connections 2024-03-18T20:40:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.463+0000 7efc9deef700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efc800a1290 0x7efc800a3750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.463+0000 7efc9deef700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc9810d750 0x7efc9819dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.463+0000 7efc9deef700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc98068fd0 0x7efc981995c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.463+0000 7efc9deef700 1 --2- 172.21.15.46:0/2178628444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc980698f0 0x7efc98199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.463+0000 7efc9deef700 1 -- 172.21.15.46:0/2178628444 >> 172.21.15.46:0/2178628444 conn(0x7efc98076d80 msgr2=0x7efc9810ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.463+0000 7efc9deef700 1 -- 172.21.15.46:0/2178628444 shutdown_connections 2024-03-18T20:40:53.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.463+0000 7efc9deef700 1 -- 172.21.15.46:0/2178628444 wait complete. 2024-03-18T20:40:53.479 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:40:53.480 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:40:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.618+0000 7efe12dbb700 1 -- 172.21.15.46:0/653304289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c1004a0 msgr2=0x7efe0c10c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.618+0000 7efe12dbb700 1 --2- 172.21.15.46:0/653304289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c1004a0 0x7efe0c10c9b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efe04009f90 tx=0x7efe0400b330 comp rx=0 tx=0).stop 2024-03-18T20:40:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.618+0000 7efe12dbb700 1 -- 172.21.15.46:0/653304289 shutdown_connections 2024-03-18T20:40:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.618+0000 7efe12dbb700 1 --2- 172.21.15.46:0/653304289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c1004a0 0x7efe0c10c9b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.618+0000 7efe12dbb700 1 --2- 172.21.15.46:0/653304289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe0c105b80 0x7efe0c105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.618+0000 7efe12dbb700 1 --2- 172.21.15.46:0/653304289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0c0ffae0 0x7efe0c0fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.618+0000 7efe12dbb700 1 -- 172.21.15.46:0/653304289 >> 172.21.15.46:0/653304289 conn(0x7efe0c0fb5f0 msgr2=0x7efe0c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.618+0000 7efe12dbb700 1 -- 172.21.15.46:0/653304289 shutdown_connections 2024-03-18T20:40:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.618+0000 7efe12dbb700 1 -- 172.21.15.46:0/653304289 wait complete. 2024-03-18T20:40:53.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.619+0000 7efe12dbb700 1 Processor -- start 2024-03-18T20:40:53.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.619+0000 7efe12dbb700 1 -- start start 2024-03-18T20:40:53.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.619+0000 7efe12dbb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c0ffae0 0x7efe0c0fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.619+0000 7efe12dbb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe0c1004a0 0x7efe0c0fc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.619+0000 7efe12dbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0c105b80 0x7efe0c0fc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.619+0000 7efe12dbb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c1a53f0 con 0x7efe0c105b80 2024-03-18T20:40:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.619+0000 7efe12dbb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c1a5560 con 0x7efe0c0ffae0 2024-03-18T20:40:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.619+0000 7efe12dbb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c1a5720 con 0x7efe0c1004a0 2024-03-18T20:40:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.619+0000 7efe10b57700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c0ffae0 0x7efe0c0fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.619+0000 7efe11358700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0c105b80 0x7efe0c0fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.619+0000 7efe10b57700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c0ffae0 0x7efe0c0fbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37328/0 (socket says 172.21.15.46:37328) 2024-03-18T20:40:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.619+0000 7efe10b57700 1 -- 172.21.15.46:0/2582923748 learned_addr learned my addr 172.21.15.46:0/2582923748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.620+0000 7efe0bfff700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe0c1004a0 0x7efe0c0fc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.620+0000 7efe10b57700 1 -- 172.21.15.46:0/2582923748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe0c1004a0 msgr2=0x7efe0c0fc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.620+0000 7efe10b57700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe0c1004a0 0x7efe0c0fc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.620+0000 7efe10b57700 1 -- 172.21.15.46:0/2582923748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0c105b80 msgr2=0x7efe0c0fc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.620+0000 7efe10b57700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0c105b80 0x7efe0c0fc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.620+0000 7efe10b57700 1 -- 172.21.15.46:0/2582923748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf80095f0 con 0x7efe0c0ffae0 2024-03-18T20:40:53.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.620+0000 7efe11358700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0c105b80 0x7efe0c0fc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:53.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.620+0000 7efe10b57700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c0ffae0 0x7efe0c0fbaf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe0000eb30 tx=0x7efe0000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:53.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.620+0000 7efe09ffb700 1 -- 172.21.15.46:0/2582923748 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00009880 con 0x7efe0c0ffae0 2024-03-18T20:40:53.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.620+0000 7efe12dbb700 1 -- 172.21.15.46:0/2582923748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe00009520 con 0x7efe0c0ffae0 2024-03-18T20:40:53.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.620+0000 7efe09ffb700 1 -- 172.21.15.46:0/2582923748 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe000044e0 con 0x7efe0c0ffae0 2024-03-18T20:40:53.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.620+0000 7efe09ffb700 1 -- 172.21.15.46:0/2582923748 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00003d80 con 0x7efe0c0ffae0 2024-03-18T20:40:53.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.621+0000 7efe12dbb700 1 -- 172.21.15.46:0/2582923748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe04009c40 con 0x7efe0c0ffae0 2024-03-18T20:40:53.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.621+0000 7efe12dbb700 1 -- 172.21.15.46:0/2582923748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0c109110 con 0x7efe0c0ffae0 2024-03-18T20:40:53.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.622+0000 7efe09ffb700 1 -- 172.21.15.46:0/2582923748 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7efe00003f20 con 0x7efe0c0ffae0 2024-03-18T20:40:53.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.623+0000 7efe09ffb700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efdfc0a1290 0x7efdfc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:53.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.623+0000 7efe0bfff700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efdfc0a1290 0x7efdfc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:53.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.623+0000 7efe09ffb700 1 -- 172.21.15.46:0/2582923748 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7efe00014070 con 0x7efe0c0ffae0 2024-03-18T20:40:53.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.624+0000 7efe0bfff700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efdfc0a1290 0x7efdfc0a3750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0fd220 tx=0x7efdf8009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:53.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.628+0000 7efe09ffb700 1 -- 172.21.15.46:0/2582923748 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efe000965f0 con 0x7efe0c0ffae0 2024-03-18T20:40:53.687 INFO:tasks.workunit.client.0.smithi046.stdout: 1 128329 2.30 MB/sec execute 108 sec latency 15.007 ms 2024-03-18T20:40:53.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.909+0000 7efe12dbb700 1 -- 172.21.15.46:0/2582923748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe0c1011e0 con 0x7efe0c0ffae0 2024-03-18T20:40:53.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.910+0000 7efe09ffb700 1 -- 172.21.15.46:0/2582923748 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10909 (secure 0 0 0) 0x7efe00099c10 con 0x7efe0c0ffae0 2024-03-18T20:40:53.913 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:40:53.913 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:39:53.094005+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:40:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.912+0000 7efe12dbb700 1 -- 172.21.15.46:0/2582923748 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efdfc0a1290 msgr2=0x7efdfc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.912+0000 7efe12dbb700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efdfc0a1290 0x7efdfc0a3750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0fd220 tx=0x7efdf8009310 comp rx=0 tx=0).stop 2024-03-18T20:40:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.912+0000 7efe12dbb700 1 -- 172.21.15.46:0/2582923748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c0ffae0 msgr2=0x7efe0c0fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.912+0000 7efe12dbb700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c0ffae0 0x7efe0c0fbaf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe0000eb30 tx=0x7efe0000ee40 comp rx=0 tx=0).stop 2024-03-18T20:40:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.912+0000 7efe12dbb700 1 -- 172.21.15.46:0/2582923748 shutdown_connections 2024-03-18T20:40:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.912+0000 7efe12dbb700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efdfc0a1290 0x7efdfc0a3750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.912+0000 7efe12dbb700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe0c105b80 0x7efe0c0fc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.912+0000 7efe12dbb700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe0c0ffae0 0x7efe0c0fbaf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.912+0000 7efe12dbb700 1 --2- 172.21.15.46:0/2582923748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe0c1004a0 0x7efe0c0fc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.912+0000 7efe12dbb700 1 -- 172.21.15.46:0/2582923748 >> 172.21.15.46:0/2582923748 conn(0x7efe0c0fb5f0 msgr2=0x7efe0c068150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.913+0000 7efe12dbb700 1 -- 172.21.15.46:0/2582923748 shutdown_connections 2024-03-18T20:40:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:53.913+0000 7efe12dbb700 1 -- 172.21.15.46:0/2582923748 wait complete. 2024-03-18T20:40:53.917 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 39 2024-03-18T20:40:53.929 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:40:53.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:40:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.074+0000 7fdbeec32700 1 -- 172.21.15.46:0/3602481136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe81060b0 msgr2=0x7fdbe810dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.074+0000 7fdbeec32700 1 --2- 172.21.15.46:0/3602481136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe81060b0 0x7fdbe810dc30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000b0d0 tx=0x7fdbe000b3e0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.074+0000 7fdbeec32700 1 -- 172.21.15.46:0/3602481136 shutdown_connections 2024-03-18T20:40:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.074+0000 7fdbeec32700 1 --2- 172.21.15.46:0/3602481136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe81060b0 0x7fdbe810dc30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.074+0000 7fdbeec32700 1 --2- 172.21.15.46:0/3602481136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbe80692a0 0x7fdbe8069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.074+0000 7fdbeec32700 1 --2- 172.21.15.46:0/3602481136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbe8069c50 0x7fdbe8105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.074+0000 7fdbeec32700 1 -- 172.21.15.46:0/3602481136 >> 172.21.15.46:0/3602481136 conn(0x7fdbe8076d80 msgr2=0x7fdbe8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.074+0000 7fdbeec32700 1 -- 172.21.15.46:0/3602481136 shutdown_connections 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.074+0000 7fdbeec32700 1 -- 172.21.15.46:0/3602481136 wait complete. 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.074+0000 7fdbeec32700 1 Processor -- start 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.074+0000 7fdbeec32700 1 -- start start 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbeec32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe80692a0 0x7fdbe81972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbeec32700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbe81060b0 0x7fdbe81977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbeec32700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbe819bb90 0x7fdbe819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbeec32700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe819c610 con 0x7fdbe80692a0 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbeec32700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe819c780 con 0x7fdbe81060b0 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbeec32700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe819c940 con 0x7fdbe819bb90 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbec9ce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe80692a0 0x7fdbe81972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbec9ce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe80692a0 0x7fdbe81972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33866/0 (socket says 172.21.15.46:33866) 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbec9ce700 1 -- 172.21.15.46:0/2961646589 learned_addr learned my addr 172.21.15.46:0/2961646589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbdffff700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbe81060b0 0x7fdbe81977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbed1cf700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbe819bb90 0x7fdbe819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbec9ce700 1 -- 172.21.15.46:0/2961646589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbe819bb90 msgr2=0x7fdbe819c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbec9ce700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbe819bb90 0x7fdbe819c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbec9ce700 1 -- 172.21.15.46:0/2961646589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbe81060b0 msgr2=0x7fdbe81977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbec9ce700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbe81060b0 0x7fdbe81977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbec9ce700 1 -- 172.21.15.46:0/2961646589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe0009d00 con 0x7fdbe80692a0 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbed1cf700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbe819bb90 0x7fdbe819c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbec9ce700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe80692a0 0x7fdbe81972b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400e9a0 tx=0x7fdbe400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbddffb700 1 -- 172.21.15.46:0/2961646589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4009790 con 0x7fdbe80692a0 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.075+0000 7fdbddffb700 1 -- 172.21.15.46:0/2961646589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbe400cbd0 con 0x7fdbe80692a0 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.076+0000 7fdbeec32700 1 -- 172.21.15.46:0/2961646589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe81a3ea0 con 0x7fdbe80692a0 2024-03-18T20:40:54.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.076+0000 7fdbddffb700 1 -- 172.21.15.46:0/2961646589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4018680 con 0x7fdbe80692a0 2024-03-18T20:40:54.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.076+0000 7fdbeec32700 1 -- 172.21.15.46:0/2961646589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe81a4320 con 0x7fdbe80692a0 2024-03-18T20:40:54.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.078+0000 7fdbddffb700 1 -- 172.21.15.46:0/2961646589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fdbe400cd40 con 0x7fdbe80692a0 2024-03-18T20:40:54.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.079+0000 7fdbddffb700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdbd00a1290 0x7fdbd00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.079+0000 7fdbdffff700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdbd00a1290 0x7fdbd00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.079+0000 7fdbddffb700 1 -- 172.21.15.46:0/2961646589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fdbe4014070 con 0x7fdbe80692a0 2024-03-18T20:40:54.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.080+0000 7fdbdffff700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdbd00a1290 0x7fdbd00a3750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8198800 tx=0x7fdbd8009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.080+0000 7fdbfadc7b80 1 Processor -- start 2024-03-18T20:40:54.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.080+0000 7fdbfadc7b80 1 -- start start 2024-03-18T20:40:54.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.080+0000 7fdbfadc7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e240a60 0x55fd4e240e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.080+0000 7fdbfadc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 0x55fd4e24bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.080+0000 7fdbfadc7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e24c220 0x55fd4e24e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.080+0000 7fdbfadc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4e19a990 con 0x55fd4e241380 2024-03-18T20:40:54.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.080+0000 7fdbfadc7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4e198350 con 0x55fd4e240a60 2024-03-18T20:40:54.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.080+0000 7fdbfadc7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4e193c40 con 0x55fd4e24c220 2024-03-18T20:40:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbdffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 0x55fd4e24bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbdffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 0x55fd4e24bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33872/0 (socket says 172.21.15.46:33872) 2024-03-18T20:40:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbdffff700 1 -- 172.21.15.46:0/1326035132 learned_addr learned my addr 172.21.15.46:0/1326035132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbed1cf700 1 --2- 172.21.15.46:0/1326035132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e24c220 0x55fd4e24e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbec9ce700 1 --2- 172.21.15.46:0/1326035132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e240a60 0x55fd4e240e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbed1cf700 1 -- 172.21.15.46:0/1326035132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e240a60 msgr2=0x55fd4e240e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbed1cf700 1 --2- 172.21.15.46:0/1326035132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e240a60 0x55fd4e240e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbed1cf700 1 -- 172.21.15.46:0/1326035132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 msgr2=0x55fd4e24bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbed1cf700 1 --2- 172.21.15.46:0/1326035132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 0x55fd4e24bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbed1cf700 1 -- 172.21.15.46:0/1326035132 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd4e0153b0 con 0x55fd4e24c220 2024-03-18T20:40:54.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbec9ce700 1 --2- 172.21.15.46:0/1326035132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e240a60 0x55fd4e240e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbdffff700 1 --2- 172.21.15.46:0/1326035132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 0x55fd4e24bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbed1cf700 1 --2- 172.21.15.46:0/1326035132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e24c220 0x55fd4e24e610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000b770 tx=0x7fdbe00092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad110fa9c1f50235 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.081+0000 7fdbceffd700 1 -- 172.21.15.46:0/1326035132 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000e040 con 0x55fd4e24c220 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.082+0000 7fdbceffd700 1 -- 172.21.15.46:0/1326035132 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbe00094d0 con 0x55fd4e24c220 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.082+0000 7fdbceffd700 1 -- 172.21.15.46:0/1326035132 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0007740 con 0x55fd4e24c220 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.082+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1326035132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e24c220 msgr2=0x55fd4e24e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.082+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/1326035132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e24c220 0x55fd4e24e610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000b770 tx=0x7fdbe00092e0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.082+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1326035132 shutdown_connections 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.082+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/1326035132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e24c220 0x55fd4e24e610 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.082+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/1326035132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e240a60 0x55fd4e240e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.082+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/1326035132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 0x55fd4e24bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.082+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1326035132 >> 172.21.15.46:0/1326035132 conn(0x55fd4e1b7220 msgr2=0x55fd4e23d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.082+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1326035132 shutdown_connections 2024-03-18T20:40:54.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.082+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1326035132 wait complete. 2024-03-18T20:40:54.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.083+0000 7fdbfadc7b80 1 -- start start 2024-03-18T20:40:54.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.083+0000 7fdbfadc7b80 1 Processor -- start 2024-03-18T20:40:54.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.083+0000 7fdbfadc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e240a60 0x55fd4e24bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.083+0000 7fdbfadc7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e241380 0x55fd4e248f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.083+0000 7fdbfadc7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e24c220 0x55fd4e249590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.083+0000 7fdbfadc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4e12fd80 con 0x55fd4e240a60 2024-03-18T20:40:54.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbfadc7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4e12fa20 con 0x55fd4e24c220 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbfadc7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4e0176a0 con 0x55fd4e241380 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbfadc7b80 10 client.0 _ll_register_callbacks cb 0x55fd4df9b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbec9ce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e240a60 0x55fd4e24bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbec9ce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e240a60 0x55fd4e24bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33880/0 (socket says 172.21.15.46:33880) 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbec9ce700 1 -- 172.21.15.46:0/1290079755 learned_addr learned my addr 172.21.15.46:0/1290079755 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbdffff700 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e241380 0x55fd4e248f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbed1cf700 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e24c220 0x55fd4e249590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbec9ce700 1 -- 172.21.15.46:0/1290079755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e241380 msgr2=0x55fd4e248f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbec9ce700 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e241380 0x55fd4e248f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbec9ce700 1 -- 172.21.15.46:0/1290079755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e24c220 msgr2=0x55fd4e249590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbec9ce700 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e24c220 0x55fd4e249590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbec9ce700 1 -- 172.21.15.46:0/1290079755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd8017070 con 0x55fd4e240a60 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbed1cf700 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e24c220 0x55fd4e249590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:54.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbec9ce700 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e240a60 0x55fd4e24bc20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4096ac0 tx=0x7fdbe409a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3821231eae8dc0d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbc7fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbc7fff700 1 -- 172.21.15.46:0/1290079755 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40a24d0 con 0x55fd4e240a60 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbc7fff700 1 -- 172.21.15.46:0/1290079755 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fdbe4099750 con 0x55fd4e240a60 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.084+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1290079755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd4e234ad0 con 0x55fd4e240a60 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.085+0000 7fdbc7fff700 1 -- 172.21.15.46:0/1290079755 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbe40b82b0 con 0x55fd4e240a60 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.085+0000 7fdbfadc7b80 10 client.17175 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.085+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1290079755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fdbe810e470 con 0x55fd4e240a60 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.085+0000 7fdbc7fff700 1 -- 172.21.15.46:0/1290079755 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbe4063070 con 0x55fd4e240a60 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.085+0000 7fdbc7fff700 1 -- 172.21.15.46:0/1290079755 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdbe4099040 con 0x55fd4e240a60 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.085+0000 7fdbfadc7b80 10 client.17175 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.085+0000 7fdbfadc7b80 10 client.17175 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.085+0000 7fdbfadc7b80 20 client.17175 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.085+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55fd4e260210 0x55fd4e262630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.085+0000 7fdbfadc7b80 4 client.17175 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.085+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1290079755 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd4e18c700 con 0x55fd4e260210 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.085+0000 7fdbdffff700 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55fd4e260210 0x55fd4e262630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbc7fff700 10 client.17175 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbdffff700 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55fd4e260210 0x55fd4e262630 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9054bcb915139811 server_cookie=67eb79d28969a130 in_seq=0 out_seq=0 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbc7fff700 1 -- 172.21.15.46:0/1290079755 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fd4e18c700 con 0x55fd4e260210 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbc7fff700 10 client.17175 handle_command_reply: tid=0 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbfadc7b80 1 client.17175 shutdown 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1290079755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e240a60 msgr2=0x55fd4e24bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e240a60 0x55fd4e24bc20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4096ac0 tx=0x7fdbe409a850 comp rx=0 tx=0).stop 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1290079755 shutdown_connections 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55fd4e260210 0x55fd4e262630 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e240a60 0x55fd4e24bc20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e24c220 0x55fd4e249590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/1290079755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e241380 0x55fd4e248f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbc7fff700 0 client.17175 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:40:54.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.086+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1290079755 >> 172.21.15.46:0/1290079755 conn(0x55fd4e1b7220 msgr2=0x55fd4e23cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.087+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1290079755 shutdown_connections 2024-03-18T20:40:54.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.087+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/1290079755 wait complete. 2024-03-18T20:40:54.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.087+0000 7fdbfadc7b80 20 client.17175 trim_cache size 0 max 16384 2024-03-18T20:40:54.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.100+0000 7fdbfadc7b80 1 Processor -- start 2024-03-18T20:40:54.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.100+0000 7fdbfadc7b80 1 -- start start 2024-03-18T20:40:54.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.100+0000 7fdbfadc7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e240a60 0x55fd4e249530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.100+0000 7fdbfadc7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e241380 0x55fd4e257280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.100+0000 7fdbfadc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e24c220 0x55fd4e23cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.100+0000 7fdbfadc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4e241d30 con 0x55fd4e24c220 2024-03-18T20:40:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbfadc7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4e242550 con 0x55fd4e241380 2024-03-18T20:40:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbfadc7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4e236f80 con 0x55fd4e240a60 2024-03-18T20:40:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbed1cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e24c220 0x55fd4e23cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbed1cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e24c220 0x55fd4e23cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33884/0 (socket says 172.21.15.46:33884) 2024-03-18T20:40:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbed1cf700 1 -- 172.21.15.46:0/763623749 learned_addr learned my addr 172.21.15.46:0/763623749 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbec9ce700 1 --2- 172.21.15.46:0/763623749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e240a60 0x55fd4e249530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbdffff700 1 --2- 172.21.15.46:0/763623749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e241380 0x55fd4e257280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbed1cf700 1 -- 172.21.15.46:0/763623749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e240a60 msgr2=0x55fd4e249530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbed1cf700 1 --2- 172.21.15.46:0/763623749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e240a60 0x55fd4e249530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbed1cf700 1 -- 172.21.15.46:0/763623749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e241380 msgr2=0x55fd4e257280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbed1cf700 1 --2- 172.21.15.46:0/763623749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e241380 0x55fd4e257280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbed1cf700 1 -- 172.21.15.46:0/763623749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd4dfc1b60 con 0x55fd4e24c220 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbed1cf700 1 --2- 172.21.15.46:0/763623749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e24c220 0x55fd4e23cb50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009fd0 tx=0x7fdbe0007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa3fe94deae9585 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbcf7fe700 1 -- 172.21.15.46:0/763623749 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000e030 con 0x55fd4e24c220 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbcf7fe700 1 -- 172.21.15.46:0/763623749 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbe0004a50 con 0x55fd4e24c220 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.101+0000 7fdbcf7fe700 1 -- 172.21.15.46:0/763623749 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0025450 con 0x55fd4e24c220 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.102+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/763623749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e24c220 msgr2=0x55fd4e23cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.102+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/763623749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e24c220 0x55fd4e23cb50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009fd0 tx=0x7fdbe0007b00 comp rx=0 tx=0).stop 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.102+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/763623749 shutdown_connections 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.102+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/763623749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e24c220 0x55fd4e23cb50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.102+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/763623749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e240a60 0x55fd4e249530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.102+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/763623749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e241380 0x55fd4e257280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.102+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/763623749 >> 172.21.15.46:0/763623749 conn(0x55fd4e1b7220 msgr2=0x55fd4e24b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.102+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/763623749 shutdown_connections 2024-03-18T20:40:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.102+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/763623749 wait complete. 2024-03-18T20:40:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.102+0000 7fdbfadc7b80 1 -- start start 2024-03-18T20:40:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.102+0000 7fdbfadc7b80 1 Processor -- start 2024-03-18T20:40:54.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbfadc7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e240a60 0x55fd4e255260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbfadc7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 0x55fd4e2557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbfadc7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e24c220 0x55fd4e255ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbfadc7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4e2b3040 con 0x55fd4e241380 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbfadc7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4e2b3180 con 0x55fd4e240a60 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbfadc7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4e2b32c0 con 0x55fd4e24c220 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbfadc7b80 10 client.0 _ll_register_callbacks cb 0x7fdbe4099c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbdffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 0x55fd4e2557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbdffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 0x55fd4e2557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33890/0 (socket says 172.21.15.46:33890) 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbed1cf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e24c220 0x55fd4e255ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbdffff700 1 -- 172.21.15.46:0/239485757 learned_addr learned my addr 172.21.15.46:0/239485757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbec9ce700 1 --2- 172.21.15.46:0/239485757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e240a60 0x55fd4e255260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbdffff700 1 -- 172.21.15.46:0/239485757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e24c220 msgr2=0x55fd4e255ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbdffff700 1 --2- 172.21.15.46:0/239485757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e24c220 0x55fd4e255ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbdffff700 1 -- 172.21.15.46:0/239485757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e240a60 msgr2=0x55fd4e255260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbdffff700 1 --2- 172.21.15.46:0/239485757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e240a60 0x55fd4e255260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.103+0000 7fdbdffff700 1 -- 172.21.15.46:0/239485757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd4e0153b0 con 0x55fd4e241380 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbdffff700 1 --2- 172.21.15.46:0/239485757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 0x55fd4e2557a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009fd0 tx=0x7fdbd80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f13c337f8c25b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbc7fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbc7fff700 1 -- 172.21.15.46:0/239485757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8024040 con 0x55fd4e241380 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/239485757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbe00094d0 con 0x55fd4e241380 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbc7fff700 1 -- 172.21.15.46:0/239485757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fdbd8029900 con 0x55fd4e241380 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbc7fff700 1 -- 172.21.15.46:0/239485757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbd8029ac0 con 0x55fd4e241380 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbfadc7b80 10 client.17181 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/239485757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55fd4e159ea0 con 0x55fd4e241380 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbc7fff700 1 -- 172.21.15.46:0/239485757 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbe00094d0 con 0x55fd4e241380 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbc7fff700 1 -- 172.21.15.46:0/239485757 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdbd801e420 con 0x55fd4e241380 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbfadc7b80 10 client.17181 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbfadc7b80 10 client.17181 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:40:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbfadc7b80 20 client.17181 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/239485757 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55fd4e2b4b50 0x55fd4e2b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbfadc7b80 4 client.17181 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.104+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/239485757 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd4dfe2cf0 con 0x55fd4e2b4b50 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbec9ce700 1 --2- 172.21.15.46:0/239485757 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55fd4e2b4b50 0x55fd4e2b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbc7fff700 10 client.17181 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbec9ce700 1 --2- 172.21.15.46:0/239485757 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55fd4e2b4b50 0x55fd4e2b6f70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=37dbb6cb140d7446 server_cookie=923e1be58e3ce264 in_seq=0 out_seq=0 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbc7fff700 1 -- 172.21.15.46:0/239485757 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd4dfe2cf0 con 0x55fd4e2b4b50 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbc7fff700 10 client.17181 handle_command_reply: tid=0 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbfadc7b80 1 client.17181 shutdown 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/239485757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 msgr2=0x55fd4e2557a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/239485757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 0x55fd4e2557a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009fd0 tx=0x7fdbd80047c0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/239485757 shutdown_connections 2024-03-18T20:40:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/239485757 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55fd4e2b4b50 0x55fd4e2b6f70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/239485757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fd4e24c220 0x55fd4e255ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/239485757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd4e240a60 0x55fd4e255260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbc7fff700 0 client.17181 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:40:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbfadc7b80 1 --2- 172.21.15.46:0/239485757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd4e241380 0x55fd4e2557a0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/239485757 >> 172.21.15.46:0/239485757 conn(0x55fd4e1b7220 msgr2=0x55fd4e244220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/239485757 shutdown_connections 2024-03-18T20:40:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.105+0000 7fdbfadc7b80 1 -- 172.21.15.46:0/239485757 wait complete. 2024-03-18T20:40:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbfadc7b80 20 client.17181 trim_cache size 0 max 16384 2024-03-18T20:40:54.115 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:40:54.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbeec32700 1 -- 172.21.15.46:0/2961646589 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdbd00a1290 msgr2=0x7fdbd00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbeec32700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdbd00a1290 0x7fdbd00a3750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8198800 tx=0x7fdbd8009260 comp rx=0 tx=0).stop 2024-03-18T20:40:54.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbeec32700 1 -- 172.21.15.46:0/2961646589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe80692a0 msgr2=0x7fdbe81972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbeec32700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe80692a0 0x7fdbe81972b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400e9a0 tx=0x7fdbe400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbeec32700 1 -- 172.21.15.46:0/2961646589 shutdown_connections 2024-03-18T20:40:54.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbeec32700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdbd00a1290 0x7fdbd00a3750 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbeec32700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe80692a0 0x7fdbe81972b0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbeec32700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbe819bb90 0x7fdbe819c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbeec32700 1 --2- 172.21.15.46:0/2961646589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbe81060b0 0x7fdbe81977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbeec32700 1 -- 172.21.15.46:0/2961646589 >> 172.21.15.46:0/2961646589 conn(0x7fdbe8076d80 msgr2=0x7fdbe81071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbeec32700 1 -- 172.21.15.46:0/2961646589 shutdown_connections 2024-03-18T20:40:54.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.106+0000 7fdbeec32700 1 -- 172.21.15.46:0/2961646589 wait complete. 2024-03-18T20:40:54.123 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:40:54.123 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:40:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.266+0000 7ff4fb68a700 1 -- 172.21.15.46:0/985816432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f40ffe50 msgr2=0x7ff4f4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.266+0000 7ff4fb68a700 1 --2- 172.21.15.46:0/985816432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f40ffe50 0x7ff4f4100230 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009990 tx=0x7ff4f0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.266+0000 7ff4fb68a700 1 -- 172.21.15.46:0/985816432 shutdown_connections 2024-03-18T20:40:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.266+0000 7ff4fb68a700 1 --2- 172.21.15.46:0/985816432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f40ffe50 0x7ff4f4100230 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.266+0000 7ff4fb68a700 1 --2- 172.21.15.46:0/985816432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f4104f60 0x7ff4f410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.266+0000 7ff4fb68a700 1 --2- 172.21.15.46:0/985816432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4f4100800 0x7ff4f4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.266+0000 7ff4fb68a700 1 -- 172.21.15.46:0/985816432 >> 172.21.15.46:0/985816432 conn(0x7ff4f4076d40 msgr2=0x7ff4f4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.266+0000 7ff4fb68a700 1 -- 172.21.15.46:0/985816432 shutdown_connections 2024-03-18T20:40:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.266+0000 7ff4fb68a700 1 -- 172.21.15.46:0/985816432 wait complete. 2024-03-18T20:40:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4fb68a700 1 Processor -- start 2024-03-18T20:40:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4fb68a700 1 -- start start 2024-03-18T20:40:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4fb68a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f40ffe50 0x7ff4f4197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4fb68a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4f4104f60 0x7ff4f4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4fb68a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f419baf0 0x7ff4f419bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4fb68a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f419c570 con 0x7ff4f419baf0 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4fb68a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f419c6e0 con 0x7ff4f4104f60 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4fb68a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f419c8a0 con 0x7ff4f40ffe50 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4f9c27700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f419baf0 0x7ff4f419bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4f9426700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f40ffe50 0x7ff4f4197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4f9426700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f40ffe50 0x7ff4f4197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:49674/0 (socket says 172.21.15.46:49674) 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4f9426700 1 -- 172.21.15.46:0/2085019238 learned_addr learned my addr 172.21.15.46:0/2085019238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4f8c25700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4f4104f60 0x7ff4f4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4f9c27700 1 -- 172.21.15.46:0/2085019238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f40ffe50 msgr2=0x7ff4f4197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.267+0000 7ff4f9c27700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f40ffe50 0x7ff4f4197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.268+0000 7ff4f9c27700 1 -- 172.21.15.46:0/2085019238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4f4104f60 msgr2=0x7ff4f4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.268+0000 7ff4f9c27700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4f4104f60 0x7ff4f4197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.268+0000 7ff4f9c27700 1 -- 172.21.15.46:0/2085019238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f00095f0 con 0x7ff4f419baf0 2024-03-18T20:40:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.268+0000 7ff4f9c27700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f419baf0 0x7ff4f419bf70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00efc0 tx=0x7ff4ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.268+0000 7ff4ea7fc700 1 -- 172.21.15.46:0/2085019238 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec018070 con 0x7ff4f419baf0 2024-03-18T20:40:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.268+0000 7ff4ea7fc700 1 -- 172.21.15.46:0/2085019238 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ec00f040 con 0x7ff4f419baf0 2024-03-18T20:40:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.268+0000 7ff4fb68a700 1 -- 172.21.15.46:0/2085019238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f41a3da0 con 0x7ff4f419baf0 2024-03-18T20:40:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.268+0000 7ff4ea7fc700 1 -- 172.21.15.46:0/2085019238 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec00cd40 con 0x7ff4f419baf0 2024-03-18T20:40:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.268+0000 7ff4fb68a700 1 -- 172.21.15.46:0/2085019238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f41a4280 con 0x7ff4f419baf0 2024-03-18T20:40:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.271+0000 7ff4ea7fc700 1 -- 172.21.15.46:0/2085019238 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff4ec003a00 con 0x7ff4f419baf0 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.272+0000 7ff4ea7fc700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff4e00a1310 0x7ff4e00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.272+0000 7ff50781fb80 1 Processor -- start 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.272+0000 7ff50781fb80 1 -- start start 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.272+0000 7ff4ea7fc700 1 -- 172.21.15.46:0/2085019238 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7ff4ec0cc7a0 con 0x7ff4f419baf0 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.272+0000 7ff4f9426700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff4e00a1310 0x7ff4e00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff50781fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f3a60 0x5649661f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff50781fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661f4380 0x5649661fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff50781fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661ff220 0x564966201610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff50781fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496614d990 con 0x5649661ff220 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff50781fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496614b350 con 0x5649661f4380 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff50781fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564966146c40 con 0x5649661f3a60 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f9c27700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661ff220 0x564966201610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f9426700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f3a60 0x5649661f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f9426700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f3a60 0x5649661f3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:49682/0 (socket says 172.21.15.46:49682) 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f9426700 1 -- 172.21.15.46:0/252355599 learned_addr learned my addr 172.21.15.46:0/252355599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f9c27700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661ff220 0x564966201610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33914/0 (socket says 172.21.15.46:33914) 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f8c25700 1 --2- 172.21.15.46:0/252355599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661f4380 0x5649661fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f9c27700 1 -- 172.21.15.46:0/252355599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f3a60 msgr2=0x5649661f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f9c27700 1 --2- 172.21.15.46:0/252355599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f3a60 0x5649661f3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f9426700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff4e00a1310 0x7ff4e00a37d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0005420 tx=0x7ff4f0005450 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f9c27700 1 -- 172.21.15.46:0/252355599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661f4380 msgr2=0x5649661fece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f9c27700 1 --2- 172.21.15.46:0/252355599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661f4380 0x5649661fece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f9c27700 1 -- 172.21.15.46:0/252355599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564965fc83b0 con 0x5649661ff220 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f8c25700 1 --2- 172.21.15.46:0/252355599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661f4380 0x5649661fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4f9c27700 1 --2- 172.21.15.46:0/252355599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661ff220 0x564966201610 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0c9f00 tx=0x7ff4ec095660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d085453e79124c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.273+0000 7ff4df7fe700 1 -- 172.21.15.46:0/252355599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec09a760 con 0x5649661ff220 2024-03-18T20:40:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.274+0000 7ff4df7fe700 1 -- 172.21.15.46:0/252355599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ec01b320 con 0x5649661ff220 2024-03-18T20:40:54.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.274+0000 7ff4df7fe700 1 -- 172.21.15.46:0/252355599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec099df0 con 0x5649661ff220 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.274+0000 7ff50781fb80 1 -- 172.21.15.46:0/252355599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661ff220 msgr2=0x564966201610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.274+0000 7ff50781fb80 1 --2- 172.21.15.46:0/252355599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661ff220 0x564966201610 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0c9f00 tx=0x7ff4ec095660 comp rx=0 tx=0).stop 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.274+0000 7ff50781fb80 1 -- 172.21.15.46:0/252355599 shutdown_connections 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.274+0000 7ff50781fb80 1 --2- 172.21.15.46:0/252355599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661ff220 0x564966201610 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.274+0000 7ff50781fb80 1 --2- 172.21.15.46:0/252355599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f3a60 0x5649661f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.274+0000 7ff50781fb80 1 --2- 172.21.15.46:0/252355599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661f4380 0x5649661fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.274+0000 7ff50781fb80 1 -- 172.21.15.46:0/252355599 >> 172.21.15.46:0/252355599 conn(0x56496616a220 msgr2=0x5649661f0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.274+0000 7ff50781fb80 1 -- 172.21.15.46:0/252355599 shutdown_connections 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.274+0000 7ff50781fb80 1 -- 172.21.15.46:0/252355599 wait complete. 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.275+0000 7ff50781fb80 1 -- start start 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.275+0000 7ff50781fb80 1 Processor -- start 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff50781fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 0x5649661febc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff50781fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f4380 0x5649661fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff50781fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x5649661fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff50781fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649660e2d80 con 0x5649661f3a60 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff50781fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649660e2a20 con 0x5649661ff220 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff50781fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564965fca6a0 con 0x5649661f4380 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4f9426700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 0x5649661febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff50781fb80 10 client.0 _ll_register_callbacks cb 0x564965f4e3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4f9426700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 0x5649661febc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33916/0 (socket says 172.21.15.46:33916) 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4f9426700 1 -- 172.21.15.46:0/384195486 learned_addr learned my addr 172.21.15.46:0/384195486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4f8c25700 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f4380 0x5649661fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4f9c27700 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x5649661fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4f9426700 1 -- 172.21.15.46:0/384195486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f4380 msgr2=0x5649661fbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4f9426700 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f4380 0x5649661fbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4f9426700 1 -- 172.21.15.46:0/384195486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 msgr2=0x5649661fc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4f9426700 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x5649661fc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4f9426700 1 -- 172.21.15.46:0/384195486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f41a4280 con 0x5649661f3a60 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4f9426700 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 0x5649661febc0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0006010 tx=0x7ff4f0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7183374b753b5b01 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4dcff9700 1 -- 172.21.15.46:0/384195486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0004cf0 con 0x5649661f3a60 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff4dcff9700 1 -- 172.21.15.46:0/384195486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7ff4f000be50 con 0x5649661f3a60 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.276+0000 7ff50781fb80 1 -- 172.21.15.46:0/384195486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649661e7ad0 con 0x5649661f3a60 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.277+0000 7ff4dcff9700 1 -- 172.21.15.46:0/384195486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4f0004e50 con 0x5649661f3a60 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.277+0000 7ff50781fb80 10 client.17193 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.277+0000 7ff50781fb80 1 -- 172.21.15.46:0/384195486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564965f74b60 con 0x5649661f3a60 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.277+0000 7ff4dcff9700 1 -- 172.21.15.46:0/384195486 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649661e7ad0 con 0x5649661f3a60 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.277+0000 7ff4dcff9700 1 -- 172.21.15.46:0/384195486 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff4f0018660 con 0x5649661f3a60 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.277+0000 7ff50781fb80 10 client.17193 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.277+0000 7ff50781fb80 10 client.17193 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.277+0000 7ff50781fb80 20 client.17193 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.277+0000 7ff50781fb80 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56496620ed20 0x564966211140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.277+0000 7ff50781fb80 4 client.17193 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.277+0000 7ff50781fb80 1 -- 172.21.15.46:0/384195486 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56496613f700 con 0x56496620ed20 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.277+0000 7ff4f8c25700 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56496620ed20 0x564966211140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.278+0000 7ff4dcff9700 10 client.17193 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.278+0000 7ff4f8c25700 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56496620ed20 0x564966211140 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=24742576f8bf6217 server_cookie=424c9b085d7b32ea in_seq=0 out_seq=0 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.278+0000 7ff4dcff9700 1 -- 172.21.15.46:0/384195486 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56496613f700 con 0x56496620ed20 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.278+0000 7ff4dcff9700 10 client.17193 handle_command_reply: tid=0 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.278+0000 7ff50781fb80 1 client.17193 shutdown 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.278+0000 7ff50781fb80 1 -- 172.21.15.46:0/384195486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 msgr2=0x5649661febc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.278+0000 7ff50781fb80 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 0x5649661febc0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0006010 tx=0x7ff4f0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.278+0000 7ff50781fb80 1 -- 172.21.15.46:0/384195486 shutdown_connections 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.278+0000 7ff50781fb80 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56496620ed20 0x564966211140 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.278+0000 7ff50781fb80 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 0x5649661febc0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.278+0000 7ff50781fb80 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x5649661fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.279+0000 7ff50781fb80 1 --2- 172.21.15.46:0/384195486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f4380 0x5649661fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.279+0000 7ff4dcff9700 0 client.17193 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:40:54.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.279+0000 7ff50781fb80 1 -- 172.21.15.46:0/384195486 >> 172.21.15.46:0/384195486 conn(0x56496616a220 msgr2=0x5649661efc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.279+0000 7ff50781fb80 1 -- 172.21.15.46:0/384195486 shutdown_connections 2024-03-18T20:40:54.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.279+0000 7ff50781fb80 1 -- 172.21.15.46:0/384195486 wait complete. 2024-03-18T20:40:54.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.279+0000 7ff50781fb80 20 client.17193 trim_cache size 0 max 16384 2024-03-18T20:40:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.292+0000 7ff50781fb80 1 Processor -- start 2024-03-18T20:40:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff50781fb80 1 -- start start 2024-03-18T20:40:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff50781fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 0x564966205e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff50781fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f4380 0x5649661fb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff50781fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x5649661efb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff50781fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496620b8f0 con 0x5649661f3a60 2024-03-18T20:40:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff50781fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649661f4d30 con 0x5649661ff220 2024-03-18T20:40:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff50781fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649661ec5e0 con 0x5649661f4380 2024-03-18T20:40:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff4f9426700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 0x564966205e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff4f9426700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 0x564966205e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33932/0 (socket says 172.21.15.46:33932) 2024-03-18T20:40:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff4f9426700 1 -- 172.21.15.46:0/2543607374 learned_addr learned my addr 172.21.15.46:0/2543607374 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff4f8c25700 1 --2- 172.21.15.46:0/2543607374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f4380 0x5649661fb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff4f9c27700 1 --2- 172.21.15.46:0/2543607374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x5649661efb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:54 smithi046 ceph-mon[43495]: pgmap v147: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-03-18T20:40:54.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:54 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2582923748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:40:54.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff4f9426700 1 -- 172.21.15.46:0/2543607374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f4380 msgr2=0x5649661fb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff4f9426700 1 --2- 172.21.15.46:0/2543607374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f4380 0x5649661fb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff4f9426700 1 -- 172.21.15.46:0/2543607374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 msgr2=0x5649661efb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff4f9426700 1 --2- 172.21.15.46:0/2543607374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x5649661efb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff4f9426700 1 -- 172.21.15.46:0/2543607374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56496610cea0 con 0x5649661f3a60 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.293+0000 7ff4f9c27700 1 --2- 172.21.15.46:0/2543607374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x5649661efb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.294+0000 7ff4f9426700 1 --2- 172.21.15.46:0/2543607374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 0x564966205e50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0018990 tx=0x7ff4f000b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b664bd0245da778 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.294+0000 7ff4dffff700 1 -- 172.21.15.46:0/2543607374 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f000be50 con 0x5649661f3a60 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.294+0000 7ff4dffff700 1 -- 172.21.15.46:0/2543607374 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4f001e040 con 0x5649661f3a60 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.294+0000 7ff4dffff700 1 -- 172.21.15.46:0/2543607374 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f000f6c0 con 0x5649661f3a60 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.294+0000 7ff50781fb80 1 -- 172.21.15.46:0/2543607374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 msgr2=0x564966205e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.294+0000 7ff50781fb80 1 --2- 172.21.15.46:0/2543607374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 0x564966205e50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0018990 tx=0x7ff4f000b2f0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.294+0000 7ff50781fb80 1 -- 172.21.15.46:0/2543607374 shutdown_connections 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.294+0000 7ff50781fb80 1 --2- 172.21.15.46:0/2543607374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f3a60 0x564966205e50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.294+0000 7ff50781fb80 1 --2- 172.21.15.46:0/2543607374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x5649661efb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.294+0000 7ff50781fb80 1 --2- 172.21.15.46:0/2543607374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f4380 0x5649661fb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.294+0000 7ff50781fb80 1 -- 172.21.15.46:0/2543607374 >> 172.21.15.46:0/2543607374 conn(0x56496616a220 msgr2=0x5649661fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.295+0000 7ff50781fb80 1 -- 172.21.15.46:0/2543607374 shutdown_connections 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.295+0000 7ff50781fb80 1 -- 172.21.15.46:0/2543607374 wait complete. 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.295+0000 7ff50781fb80 1 -- start start 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.295+0000 7ff50781fb80 1 Processor -- start 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.295+0000 7ff50781fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f3a60 0x564966204020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.295+0000 7ff50781fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f4380 0x564966204560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.295+0000 7ff50781fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x564966261b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.295+0000 7ff50781fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564966204cb0 con 0x5649661f4380 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.295+0000 7ff50781fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564966262050 con 0x5649661ff220 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.295+0000 7ff50781fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564966262210 con 0x5649661f3a60 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.295+0000 7ff50781fb80 10 client.0 _ll_register_callbacks cb 0x7ff4f00050a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f8c25700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f4380 0x564966204560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f9426700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f3a60 0x564966204020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f8c25700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f4380 0x564966204560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33944/0 (socket says 172.21.15.46:33944) 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f8c25700 1 -- 172.21.15.46:0/2222328942 learned_addr learned my addr 172.21.15.46:0/2222328942 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f9c27700 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x564966261b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f8c25700 1 -- 172.21.15.46:0/2222328942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f3a60 msgr2=0x564966204020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f8c25700 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f3a60 0x564966204020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f8c25700 1 -- 172.21.15.46:0/2222328942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 msgr2=0x564966261b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f8c25700 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x564966261b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f8c25700 1 -- 172.21.15.46:0/2222328942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564965fc83b0 con 0x5649661f4380 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f9c27700 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x564966261b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f9426700 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f3a60 0x564966204020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4f8c25700 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f4380 0x564966204560 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4009c60 tx=0x7ff4e401fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe596f774a3295 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4dcff9700 1 -- 172.21.15.46:0/2222328942 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4004460 con 0x5649661f4380 2024-03-18T20:40:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4dcff9700 1 -- 172.21.15.46:0/2222328942 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7ff4e4024070 con 0x5649661f4380 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff50781fb80 1 -- 172.21.15.46:0/2222328942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4ec01b320 con 0x5649661f4380 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff4dcff9700 1 -- 172.21.15.46:0/2222328942 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4e401f540 con 0x5649661f4380 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff50781fb80 10 client.17199 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.296+0000 7ff50781fb80 1 -- 172.21.15.46:0/2222328942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564965f74b60 con 0x5649661f4380 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.297+0000 7ff4dcff9700 1 -- 172.21.15.46:0/2222328942 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4e4034080 con 0x5649661f4380 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.297+0000 7ff4dcff9700 1 -- 172.21.15.46:0/2222328942 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff4e4003820 con 0x5649661f4380 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.297+0000 7ff50781fb80 10 client.17199 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.297+0000 7ff50781fb80 10 client.17199 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.297+0000 7ff50781fb80 20 client.17199 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.297+0000 7ff50781fb80 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564966263a60 0x564966265e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.297+0000 7ff50781fb80 4 client.17199 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.297+0000 7ff50781fb80 1 -- 172.21.15.46:0/2222328942 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564965f95cf0 con 0x564966263a60 2024-03-18T20:40:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.297+0000 7ff4f9426700 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564966263a60 0x564966265e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.297+0000 7ff4dcff9700 10 client.17199 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:40:54.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.297+0000 7ff4f9426700 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564966263a60 0x564966265e80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5bdcc4be867fbf64 server_cookie=3158bacdab3dddd1 in_seq=0 out_seq=0 2024-03-18T20:40:54.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff4dcff9700 1 -- 172.21.15.46:0/2222328942 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564965f95cf0 con 0x564966263a60 2024-03-18T20:40:54.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff4dcff9700 10 client.17199 handle_command_reply: tid=0 2024-03-18T20:40:54.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff50781fb80 1 client.17199 shutdown 2024-03-18T20:40:54.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff50781fb80 1 -- 172.21.15.46:0/2222328942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f4380 msgr2=0x564966204560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff50781fb80 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f4380 0x564966204560 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4009c60 tx=0x7ff4e401fdc0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff50781fb80 1 -- 172.21.15.46:0/2222328942 shutdown_connections 2024-03-18T20:40:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff50781fb80 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5649661ff220 0x564966261b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff50781fb80 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5649661f3a60 0x564966204020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff50781fb80 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564966263a60 0x564966265e80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff50781fb80 1 --2- 172.21.15.46:0/2222328942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5649661f4380 0x564966204560 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff4dcff9700 0 client.17199 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:40:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff50781fb80 1 -- 172.21.15.46:0/2222328942 >> 172.21.15.46:0/2222328942 conn(0x56496616a220 msgr2=0x5649661f7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff50781fb80 1 -- 172.21.15.46:0/2222328942 shutdown_connections 2024-03-18T20:40:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff50781fb80 1 -- 172.21.15.46:0/2222328942 wait complete. 2024-03-18T20:40:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.298+0000 7ff50781fb80 20 client.17199 trim_cache size 0 max 16384 2024-03-18T20:40:54.309 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:40:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.299+0000 7ff4fb68a700 1 -- 172.21.15.46:0/2085019238 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff4e00a1310 msgr2=0x7ff4e00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.299+0000 7ff4fb68a700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff4e00a1310 0x7ff4e00a37d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0005420 tx=0x7ff4f0005450 comp rx=0 tx=0).stop 2024-03-18T20:40:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.299+0000 7ff4fb68a700 1 -- 172.21.15.46:0/2085019238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f419baf0 msgr2=0x7ff4f419bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.299+0000 7ff4fb68a700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f419baf0 0x7ff4f419bf70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00efc0 tx=0x7ff4ec00c3a0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.299+0000 7ff4fb68a700 1 -- 172.21.15.46:0/2085019238 shutdown_connections 2024-03-18T20:40:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.299+0000 7ff4fb68a700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff4e00a1310 0x7ff4e00a37d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.299+0000 7ff4fb68a700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4f419baf0 0x7ff4f419bf70 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.299+0000 7ff4fb68a700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4f40ffe50 0x7ff4f4197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.299+0000 7ff4fb68a700 1 --2- 172.21.15.46:0/2085019238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4f4104f60 0x7ff4f4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.299+0000 7ff4fb68a700 1 -- 172.21.15.46:0/2085019238 >> 172.21.15.46:0/2085019238 conn(0x7ff4f4076d40 msgr2=0x7ff4f4068270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.299+0000 7ff4fb68a700 1 -- 172.21.15.46:0/2085019238 shutdown_connections 2024-03-18T20:40:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.299+0000 7ff4fb68a700 1 -- 172.21.15.46:0/2085019238 wait complete. 2024-03-18T20:40:54.315 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:40:54.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:40:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.452+0000 7fd9df8f6700 1 -- 172.21.15.46:0/3597674685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9d80fffe0 msgr2=0x7fd9d810c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.452+0000 7fd9df8f6700 1 --2- 172.21.15.46:0/3597674685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9d80fffe0 0x7fd9d810c220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009f90 tx=0x7fd9d000b330 comp rx=0 tx=0).stop 2024-03-18T20:40:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.452+0000 7fd9df8f6700 1 -- 172.21.15.46:0/3597674685 shutdown_connections 2024-03-18T20:40:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.452+0000 7fd9df8f6700 1 --2- 172.21.15.46:0/3597674685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9d8104350 0x7fd9d8104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.452+0000 7fd9df8f6700 1 --2- 172.21.15.46:0/3597674685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9d80fffe0 0x7fd9d810c220 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.452+0000 7fd9df8f6700 1 --2- 172.21.15.46:0/3597674685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9d80ff620 0x7fd9d80ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.452+0000 7fd9df8f6700 1 -- 172.21.15.46:0/3597674685 >> 172.21.15.46:0/3597674685 conn(0x7fd9d80761e0 msgr2=0x7fd9d80765f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.452+0000 7fd9df8f6700 1 -- 172.21.15.46:0/3597674685 shutdown_connections 2024-03-18T20:40:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.452+0000 7fd9df8f6700 1 -- 172.21.15.46:0/3597674685 wait complete. 2024-03-18T20:40:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9df8f6700 1 Processor -- start 2024-03-18T20:40:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9df8f6700 1 -- start start 2024-03-18T20:40:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9df8f6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9d80ff620 0x7fd9d81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9df8f6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9d80fffe0 0x7fd9d81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9df8f6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9d8104350 0x7fd9d8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9df8f6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81a34c0 con 0x7fd9d80ff620 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9df8f6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81a3630 con 0x7fd9d8104350 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9df8f6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d81a37f0 con 0x7fd9d80fffe0 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9dd692700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9d80ff620 0x7fd9d81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9dd692700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9d80ff620 0x7fd9d81981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33968/0 (socket says 172.21.15.46:33968) 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9dd692700 1 -- 172.21.15.46:0/382810099 learned_addr learned my addr 172.21.15.46:0/382810099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9dce91700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9d80fffe0 0x7fd9d81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9dd692700 1 -- 172.21.15.46:0/382810099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9d80fffe0 msgr2=0x7fd9d81986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9dde93700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9d8104350 0x7fd9d8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9dd692700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9d80fffe0 0x7fd9d81986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9dd692700 1 -- 172.21.15.46:0/382810099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9d8104350 msgr2=0x7fd9d8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9dd692700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9d8104350 0x7fd9d8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9dd692700 1 -- 172.21.15.46:0/382810099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d0009c40 con 0x7fd9d80ff620 2024-03-18T20:40:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9dce91700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9d80fffe0 0x7fd9d81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:40:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.453+0000 7fd9dde93700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9d8104350 0x7fd9d8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:40:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.454+0000 7fd9dd692700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9d80ff620 0x7fd9d81981b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400e9c0 tx=0x7fd9d400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.454+0000 7fd9ce7fc700 1 -- 172.21.15.46:0/382810099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400cbd0 con 0x7fd9d80ff620 2024-03-18T20:40:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.454+0000 7fd9ce7fc700 1 -- 172.21.15.46:0/382810099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9d400cd30 con 0x7fd9d80ff620 2024-03-18T20:40:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.454+0000 7fd9df8f6700 1 -- 172.21.15.46:0/382810099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d81a3ad0 con 0x7fd9d80ff620 2024-03-18T20:40:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.454+0000 7fd9ce7fc700 1 -- 172.21.15.46:0/382810099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4005580 con 0x7fd9d80ff620 2024-03-18T20:40:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.454+0000 7fd9df8f6700 1 -- 172.21.15.46:0/382810099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d81a3fe0 con 0x7fd9d80ff620 2024-03-18T20:40:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.457+0000 7fd9ce7fc700 1 -- 172.21.15.46:0/382810099 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd9d4004750 con 0x7fd9d80ff620 2024-03-18T20:40:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.457+0000 7fd9ce7fc700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd9c40a1290 0x7fd9c40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.458+0000 7fd9dce91700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd9c40a1290 0x7fd9c40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.458+0000 7fd9ce7fc700 1 -- 172.21.15.46:0/382810099 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd9d4014070 con 0x7fd9d80ff620 2024-03-18T20:40:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.458+0000 7fd9dce91700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd9c40a1290 0x7fd9c40a3750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8005fd0 tx=0x7fd9c8005ee0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.458+0000 7fd9eba8bb80 1 Processor -- start 2024-03-18T20:40:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.458+0000 7fd9eba8bb80 1 -- start start 2024-03-18T20:40:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9eba8bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766ba60 0x55a85766be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9eba8bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766c380 0x55a857676ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9eba8bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 0x55a857679610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9eba8bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8575c5990 con 0x55a857677220 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9eba8bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8575c3350 con 0x55a85766c380 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9eba8bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8575bec40 con 0x55a85766ba60 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9dde93700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 0x55a857679610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9dce91700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766c380 0x55a857676ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9dde93700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 0x55a857679610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33978/0 (socket says 172.21.15.46:33978) 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9dde93700 1 -- 172.21.15.46:0/1478300950 learned_addr learned my addr 172.21.15.46:0/1478300950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9dd692700 1 --2- 172.21.15.46:0/1478300950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766ba60 0x55a85766be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9dde93700 1 -- 172.21.15.46:0/1478300950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766ba60 msgr2=0x55a85766be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9dde93700 1 --2- 172.21.15.46:0/1478300950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766ba60 0x55a85766be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9dde93700 1 -- 172.21.15.46:0/1478300950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766c380 msgr2=0x55a857676ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9dde93700 1 --2- 172.21.15.46:0/1478300950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766c380 0x55a857676ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9dde93700 1 -- 172.21.15.46:0/1478300950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8574403b0 con 0x55a857677220 2024-03-18T20:40:54.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9dce91700 1 --2- 172.21.15.46:0/1478300950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766c380 0x55a857676ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.459+0000 7fd9dde93700 1 --2- 172.21.15.46:0/1478300950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 0x55a857679610 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009360 tx=0x7fd9d0009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606d622058d66867 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.460+0000 7fd9c37fe700 1 -- 172.21.15.46:0/1478300950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000e070 con 0x55a857677220 2024-03-18T20:40:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.460+0000 7fd9c37fe700 1 -- 172.21.15.46:0/1478300950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9d0004a50 con 0x55a857677220 2024-03-18T20:40:54.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.460+0000 7fd9c37fe700 1 -- 172.21.15.46:0/1478300950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0007c10 con 0x55a857677220 2024-03-18T20:40:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.460+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1478300950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 msgr2=0x55a857679610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.460+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1478300950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 0x55a857679610 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009360 tx=0x7fd9d0009390 comp rx=0 tx=0).stop 2024-03-18T20:40:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.460+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1478300950 shutdown_connections 2024-03-18T20:40:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.460+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1478300950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 0x55a857679610 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.460+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1478300950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766ba60 0x55a85766be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.460+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1478300950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766c380 0x55a857676ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.460+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1478300950 >> 172.21.15.46:0/1478300950 conn(0x55a8575e2220 msgr2=0x55a857668020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.460+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1478300950 shutdown_connections 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.460+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1478300950 wait complete. 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.461+0000 7fd9eba8bb80 1 -- start start 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.461+0000 7fd9eba8bb80 1 Processor -- start 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9eba8bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766ba60 0x55a857676c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9eba8bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766c380 0x55a857673fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9eba8bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 0x55a857674700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9eba8bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85755ad80 con 0x55a857677220 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9eba8bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85755aa20 con 0x55a85766ba60 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9eba8bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8574426a0 con 0x55a85766c380 2024-03-18T20:40:54.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9eba8bb80 10 client.0 _ll_register_callbacks cb 0x55a8573c63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9dde93700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 0x55a857674700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9dde93700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 0x55a857674700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33980/0 (socket says 172.21.15.46:33980) 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9dde93700 1 -- 172.21.15.46:0/437607216 learned_addr learned my addr 172.21.15.46:0/437607216 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9dd692700 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766ba60 0x55a857676c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9dce91700 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766c380 0x55a857673fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9dde93700 1 -- 172.21.15.46:0/437607216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766c380 msgr2=0x55a857673fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9dde93700 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766c380 0x55a857673fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9dde93700 1 -- 172.21.15.46:0/437607216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766ba60 msgr2=0x55a857676c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9dde93700 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766ba60 0x55a857676c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9dde93700 1 -- 172.21.15.46:0/437607216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c800e9f0 con 0x55a857677220 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9dce91700 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766c380 0x55a857673fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9dde93700 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 0x55a857674700 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009f90 tx=0x7fd9d000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f2235a0216f3c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.462+0000 7fd9c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9c0ff9700 1 -- 172.21.15.46:0/437607216 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d000e050 con 0x55a857677220 2024-03-18T20:40:54.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9c0ff9700 1 -- 172.21.15.46:0/437607216 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd9d00079d0 con 0x55a857677220 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/437607216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a85765fad0 con 0x55a857677220 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9c0ff9700 1 -- 172.21.15.46:0/437607216 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9d0007b90 con 0x55a857677220 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9eba8bb80 10 client.17211 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/437607216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fd9d8068a20 con 0x55a857677220 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9c0ff9700 1 -- 172.21.15.46:0/437607216 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a85765fad0 con 0x55a857677220 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9c0ff9700 1 -- 172.21.15.46:0/437607216 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd9d000b050 con 0x55a857677220 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9eba8bb80 10 client.17211 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9eba8bb80 10 client.17211 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9eba8bb80 20 client.17211 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a857686ad0 0x55a857688ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9eba8bb80 4 client.17211 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/437607216 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8575b7700 con 0x55a857686ad0 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.463+0000 7fd9dd692700 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a857686ad0 0x55a857688ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.464+0000 7fd9dd692700 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a857686ad0 0x55a857688ef0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3bbff4a1ffac5d6a server_cookie=79692e816b357ab7 in_seq=0 out_seq=0 2024-03-18T20:40:54.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.464+0000 7fd9c0ff9700 10 client.17211 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.464+0000 7fd9c0ff9700 1 -- 172.21.15.46:0/437607216 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a8575b7700 con 0x55a857686ad0 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.464+0000 7fd9c0ff9700 10 client.17211 handle_command_reply: tid=0 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9eba8bb80 1 client.17211 shutdown 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/437607216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 msgr2=0x55a857674700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 0x55a857674700 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009f90 tx=0x7fd9d000bbb0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/437607216 shutdown_connections 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a857686ad0 0x55a857688ef0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a857677220 0x55a857674700 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9c0ff9700 0 client.17211 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766ba60 0x55a857676c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/437607216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766c380 0x55a857673fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/437607216 >> 172.21.15.46:0/437607216 conn(0x55a8575e2220 msgr2=0x55a857667c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/437607216 shutdown_connections 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/437607216 wait complete. 2024-03-18T20:40:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.465+0000 7fd9eba8bb80 20 client.17211 trim_cache size 0 max 16384 2024-03-18T20:40:54.474 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:54 smithi073 ceph-mon[47384]: pgmap v147: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-03-18T20:40:54.474 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:54 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2582923748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:40:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9eba8bb80 1 Processor -- start 2024-03-18T20:40:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9eba8bb80 1 -- start start 2024-03-18T20:40:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9eba8bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 0x55a857679610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9eba8bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766c380 0x55a85767ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9eba8bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a857677220 0x55a857682540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9eba8bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a857686900 con 0x55a85766ba60 2024-03-18T20:40:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9eba8bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a857662210 con 0x55a857677220 2024-03-18T20:40:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9eba8bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a85766d550 con 0x55a85766c380 2024-03-18T20:40:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9dd692700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 0x55a857679610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9dd692700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 0x55a857679610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33984/0 (socket says 172.21.15.46:33984) 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9dd692700 1 -- 172.21.15.46:0/1470910721 learned_addr learned my addr 172.21.15.46:0/1470910721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9dce91700 1 --2- 172.21.15.46:0/1470910721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766c380 0x55a85767ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9dd692700 1 -- 172.21.15.46:0/1470910721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766c380 msgr2=0x55a85767ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9dd692700 1 --2- 172.21.15.46:0/1470910721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766c380 0x55a85767ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9dde93700 1 --2- 172.21.15.46:0/1470910721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a857677220 0x55a857682540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9dd692700 1 -- 172.21.15.46:0/1470910721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a857677220 msgr2=0x55a857682540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9dd692700 1 --2- 172.21.15.46:0/1470910721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a857677220 0x55a857682540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.479+0000 7fd9dd692700 1 -- 172.21.15.46:0/1470910721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8573ecb60 con 0x55a85766ba60 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.480+0000 7fd9dd692700 1 --2- 172.21.15.46:0/1470910721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 0x55a857679610 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4099dc0 tx=0x7fd9d40adf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd42668cefb87a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.480+0000 7fd9c3fff700 1 -- 172.21.15.46:0/1470910721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d409def0 con 0x55a85766ba60 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.480+0000 7fd9c3fff700 1 -- 172.21.15.46:0/1470910721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9d409d040 con 0x55a85766ba60 2024-03-18T20:40:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.480+0000 7fd9c3fff700 1 -- 172.21.15.46:0/1470910721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d409a580 con 0x55a85766ba60 2024-03-18T20:40:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.480+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1470910721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 msgr2=0x55a857679610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.480+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1470910721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 0x55a857679610 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4099dc0 tx=0x7fd9d40adf90 comp rx=0 tx=0).stop 2024-03-18T20:40:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1470910721 shutdown_connections 2024-03-18T20:40:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1470910721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 0x55a857679610 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1470910721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a857677220 0x55a857682540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1470910721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a85766c380 0x55a85767ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1470910721 >> 172.21.15.46:0/1470910721 conn(0x55a8575e2220 msgr2=0x55a857663a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1470910721 shutdown_connections 2024-03-18T20:40:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1470910721 wait complete. 2024-03-18T20:40:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 -- start start 2024-03-18T20:40:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 Processor -- start 2024-03-18T20:40:54.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 0x55a85767bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766c380 0x55a85767c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a857677220 0x55a8576d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8576d3e80 con 0x55a85766ba60 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8576d3ff0 con 0x55a85766c380 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.481+0000 7fd9eba8bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8576d41b0 con 0x55a857677220 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9eba8bb80 10 client.0 _ll_register_callbacks cb 0x7fd9c80098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9dd692700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 0x55a85767bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9dd692700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 0x55a85767bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33996/0 (socket says 172.21.15.46:33996) 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9dd692700 1 -- 172.21.15.46:0/1562727497 learned_addr learned my addr 172.21.15.46:0/1562727497 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9dce91700 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766c380 0x55a85767c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9dde93700 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a857677220 0x55a8576d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9dd692700 1 -- 172.21.15.46:0/1562727497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a857677220 msgr2=0x55a8576d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9dd692700 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a857677220 0x55a8576d38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9dd692700 1 -- 172.21.15.46:0/1562727497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766c380 msgr2=0x55a85767c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9dd692700 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766c380 0x55a85767c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9dd692700 1 -- 172.21.15.46:0/1562727497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d81a3fe0 con 0x55a85766ba60 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9dce91700 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766c380 0x55a85767c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9dd692700 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 0x55a85767bfc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40b4040 tx=0x7fd9d40a2540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7797781c944110a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9c0ff9700 1 -- 172.21.15.46:0/1562727497 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d40b6040 con 0x55a85766ba60 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9c0ff9700 1 -- 172.21.15.46:0/1562727497 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd9d40995b0 con 0x55a85766ba60 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1562727497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9d0004a50 con 0x55a85766ba60 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.482+0000 7fd9c0ff9700 1 -- 172.21.15.46:0/1562727497 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9d4099790 con 0x55a85766ba60 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9eba8bb80 10 client.17217 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1562727497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a857584ea0 con 0x55a85766ba60 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9c0ff9700 1 -- 172.21.15.46:0/1562727497 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9d0004a50 con 0x55a85766ba60 2024-03-18T20:40:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9c0ff9700 1 -- 172.21.15.46:0/1562727497 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd9d4004a00 con 0x55a85766ba60 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9eba8bb80 10 client.17217 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9eba8bb80 10 client.17217 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9eba8bb80 20 client.17217 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a8576db7e0 0x55a8576ddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9eba8bb80 4 client.17217 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1562727497 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a85740dcf0 con 0x55a8576db7e0 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9dce91700 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a8576db7e0 0x55a8576ddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9c0ff9700 10 client.17217 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.483+0000 7fd9dce91700 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a8576db7e0 0x55a8576ddbd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b430110caea6ff1 server_cookie=71823917f2dd859d in_seq=0 out_seq=0 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9c0ff9700 1 -- 172.21.15.46:0/1562727497 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a85740dcf0 con 0x55a8576db7e0 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9c0ff9700 10 client.17217 handle_command_reply: tid=0 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9eba8bb80 1 client.17217 shutdown 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1562727497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 msgr2=0x55a85767bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 0x55a85767bfc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40b4040 tx=0x7fd9d40a2540 comp rx=0 tx=0).stop 2024-03-18T20:40:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1562727497 shutdown_connections 2024-03-18T20:40:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a8576db7e0 0x55a8576ddbd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a85766ba60 0x55a85767bfc0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9c0ff9700 0 client.17217 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:40:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a857677220 0x55a8576d38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9eba8bb80 1 --2- 172.21.15.46:0/1562727497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a85766c380 0x55a85767c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1562727497 >> 172.21.15.46:0/1562727497 conn(0x55a8575e2220 msgr2=0x55a857682540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1562727497 shutdown_connections 2024-03-18T20:40:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9eba8bb80 1 -- 172.21.15.46:0/1562727497 wait complete. 2024-03-18T20:40:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.484+0000 7fd9eba8bb80 20 client.17217 trim_cache size 0 max 16384 2024-03-18T20:40:54.493 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:40:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.485+0000 7fd9df8f6700 1 -- 172.21.15.46:0/382810099 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd9c40a1290 msgr2=0x7fd9c40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.485+0000 7fd9df8f6700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd9c40a1290 0x7fd9c40a3750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8005fd0 tx=0x7fd9c8005ee0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.485+0000 7fd9df8f6700 1 -- 172.21.15.46:0/382810099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9d80ff620 msgr2=0x7fd9d81981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.485+0000 7fd9df8f6700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9d80ff620 0x7fd9d81981b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400e9c0 tx=0x7fd9d400ed80 comp rx=0 tx=0).stop 2024-03-18T20:40:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.485+0000 7fd9df8f6700 1 -- 172.21.15.46:0/382810099 shutdown_connections 2024-03-18T20:40:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.485+0000 7fd9df8f6700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd9c40a1290 0x7fd9c40a3750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.485+0000 7fd9df8f6700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9d80ff620 0x7fd9d81981b0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.485+0000 7fd9df8f6700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9d8104350 0x7fd9d8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.485+0000 7fd9df8f6700 1 --2- 172.21.15.46:0/382810099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9d80fffe0 0x7fd9d81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.485+0000 7fd9df8f6700 1 -- 172.21.15.46:0/382810099 >> 172.21.15.46:0/382810099 conn(0x7fd9d80761e0 msgr2=0x7fd9d80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.485+0000 7fd9df8f6700 1 -- 172.21.15.46:0/382810099 shutdown_connections 2024-03-18T20:40:54.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.485+0000 7fd9df8f6700 1 -- 172.21.15.46:0/382810099 wait complete. 2024-03-18T20:40:54.501 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:40:54.501 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:40:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:54 smithi155 ceph-mon[48344]: pgmap v147: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-03-18T20:40:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:54 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2582923748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:40:54.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.637+0000 7f2c1fa61700 1 -- 172.21.15.46:0/1520063871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c18104f80 msgr2=0x7f2c1810cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.637+0000 7f2c1fa61700 1 --2- 172.21.15.46:0/1520063871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c18104f80 0x7f2c1810cb00 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000b0a0 tx=0x7f2c1000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.637+0000 7f2c1fa61700 1 -- 172.21.15.46:0/1520063871 shutdown_connections 2024-03-18T20:40:54.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.637+0000 7f2c1fa61700 1 --2- 172.21.15.46:0/1520063871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c18104f80 0x7f2c1810cb00 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.637+0000 7f2c1fa61700 1 --2- 172.21.15.46:0/1520063871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c180ffe60 0x7f2c18100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.637+0000 7f2c1fa61700 1 --2- 172.21.15.46:0/1520063871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c18100810 0x7f2c18104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.637+0000 7f2c1fa61700 1 -- 172.21.15.46:0/1520063871 >> 172.21.15.46:0/1520063871 conn(0x7f2c180fb6c0 msgr2=0x7f2c180fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.637+0000 7f2c1fa61700 1 -- 172.21.15.46:0/1520063871 shutdown_connections 2024-03-18T20:40:54.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.637+0000 7f2c1fa61700 1 -- 172.21.15.46:0/1520063871 wait complete. 2024-03-18T20:40:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.638+0000 7f2c1fa61700 1 Processor -- start 2024-03-18T20:40:54.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.638+0000 7f2c1fa61700 1 -- start start 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.638+0000 7f2c1fa61700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c180ffe60 0x7f2c181018f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.638+0000 7f2c1fa61700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c18104f80 0x7f2c18101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.638+0000 7f2c1fa61700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c18102370 0x7f2c181a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.638+0000 7f2c1fa61700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181a5a80 con 0x7f2c18102370 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.638+0000 7f2c1fa61700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181a5bf0 con 0x7f2c180ffe60 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.638+0000 7f2c1fa61700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181a5db0 con 0x7f2c18104f80 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.638+0000 7f2c1dffe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c18102370 0x7f2c181a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.638+0000 7f2c1dffe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c18102370 0x7f2c181a54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34014/0 (socket says 172.21.15.46:34014) 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.638+0000 7f2c1dffe700 1 -- 172.21.15.46:0/3205707292 learned_addr learned my addr 172.21.15.46:0/3205707292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.638+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c180ffe60 0x7f2c181018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.639+0000 7f2c1cffc700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c18104f80 0x7f2c18101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.639+0000 7f2c1dffe700 1 -- 172.21.15.46:0/3205707292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c18104f80 msgr2=0x7f2c18101e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.639+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c18104f80 0x7f2c18101e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.639+0000 7f2c1dffe700 1 -- 172.21.15.46:0/3205707292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c180ffe60 msgr2=0x7f2c181018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.639+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c180ffe60 0x7f2c181018f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.639+0000 7f2c1dffe700 1 -- 172.21.15.46:0/3205707292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c10009d00 con 0x7f2c18102370 2024-03-18T20:40:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.639+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c18102370 0x7f2c181a54b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2c10006130 tx=0x7f2c10009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.639+0000 7f2c0e7fc700 1 -- 172.21.15.46:0/3205707292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1000e040 con 0x7f2c18102370 2024-03-18T20:40:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.639+0000 7f2c0e7fc700 1 -- 172.21.15.46:0/3205707292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c10004e40 con 0x7f2c18102370 2024-03-18T20:40:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.639+0000 7f2c1fa61700 1 -- 172.21.15.46:0/3205707292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c181a6030 con 0x7f2c18102370 2024-03-18T20:40:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.639+0000 7f2c0e7fc700 1 -- 172.21.15.46:0/3205707292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1001b6b0 con 0x7f2c18102370 2024-03-18T20:40:54.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.639+0000 7f2c1fa61700 1 -- 172.21.15.46:0/3205707292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c181a64b0 con 0x7f2c18102370 2024-03-18T20:40:54.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.641+0000 7f2c0e7fc700 1 -- 172.21.15.46:0/3205707292 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2c1001b810 con 0x7f2c18102370 2024-03-18T20:40:54.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.642+0000 7f2c0e7fc700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2c040a1290 0x7f2c040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.642+0000 7f2c0e7fc700 1 -- 172.21.15.46:0/3205707292 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2c100ce6a0 con 0x7f2c18102370 2024-03-18T20:40:54.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.642+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2c040a1290 0x7f2c040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2c040a1290 0x7f2c040a3750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c08007be0 tx=0x7f2c080073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c2bbf6b80 1 Processor -- start 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c2bbf6b80 1 -- start start 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c2bbf6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4da60 0x55a2aef4de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c2bbf6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4e380 0x55a2aef58ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c2bbf6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef59220 0x55a2aef5b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c2bbf6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aeea7990 con 0x55a2aef4da60 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c2bbf6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aeea5350 con 0x55a2aef4e380 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c2bbf6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aeea0c40 con 0x55a2aef59220 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c1d7fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4da60 0x55a2aef4de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c1d7fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4da60 0x55a2aef4de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34024/0 (socket says 172.21.15.46:34024) 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c1d7fd700 1 -- 172.21.15.46:0/1727194572 learned_addr learned my addr 172.21.15.46:0/1727194572 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.643+0000 7f2c1cffc700 1 --2- 172.21.15.46:0/1727194572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4e380 0x55a2aef58ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.644+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/1727194572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef59220 0x55a2aef5b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.644+0000 7f2c1d7fd700 1 -- 172.21.15.46:0/1727194572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef59220 msgr2=0x55a2aef5b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.644+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/1727194572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef59220 0x55a2aef5b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.644+0000 7f2c1d7fd700 1 -- 172.21.15.46:0/1727194572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4e380 msgr2=0x55a2aef58ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.644+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/1727194572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4e380 0x55a2aef58ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.644+0000 7f2c1d7fd700 1 -- 172.21.15.46:0/1727194572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2aed223b0 con 0x55a2aef4da60 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.644+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/1727194572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef59220 0x55a2aef5b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.644+0000 7f2c1cffc700 1 --2- 172.21.15.46:0/1727194572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4e380 0x55a2aef58ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.644+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/1727194572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4da60 0x55a2aef4de40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2c08010e80 tx=0x7f2c08013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2720a2c706affa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.644+0000 7f2c037fe700 1 -- 172.21.15.46:0/1727194572 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08010eb0 con 0x55a2aef4da60 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.644+0000 7f2c037fe700 1 -- 172.21.15.46:0/1727194572 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c08003b40 con 0x55a2aef4da60 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.644+0000 7f2c037fe700 1 -- 172.21.15.46:0/1727194572 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08011ba0 con 0x55a2aef4da60 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.645+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1727194572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4da60 msgr2=0x55a2aef4de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.645+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1727194572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4da60 0x55a2aef4de40 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2c08010e80 tx=0x7f2c08013000 comp rx=0 tx=0).stop 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.645+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1727194572 shutdown_connections 2024-03-18T20:40:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.645+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1727194572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4da60 0x55a2aef4de40 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.645+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1727194572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef59220 0x55a2aef5b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.645+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1727194572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4e380 0x55a2aef58ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.645+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1727194572 >> 172.21.15.46:0/1727194572 conn(0x55a2aeec4220 msgr2=0x55a2aef4a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.645+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1727194572 shutdown_connections 2024-03-18T20:40:54.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.645+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1727194572 wait complete. 2024-03-18T20:40:54.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.645+0000 7f2c2bbf6b80 1 -- start start 2024-03-18T20:40:54.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.645+0000 7f2c2bbf6b80 1 Processor -- start 2024-03-18T20:40:54.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c2bbf6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef58c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c2bbf6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef4e380 0x55a2aef55f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c2bbf6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 0x55a2aef56690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c2bbf6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aee3cd80 con 0x55a2aef59220 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c2bbf6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aee3ca20 con 0x55a2aef4da60 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c2bbf6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aed246a0 con 0x55a2aef4e380 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c2bbf6b80 10 client.0 _ll_register_callbacks cb 0x55a2aeca83a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c1dffe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 0x55a2aef56690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c1d7fd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef58c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c1dffe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 0x55a2aef56690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34030/0 (socket says 172.21.15.46:34030) 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c1dffe700 1 -- 172.21.15.46:0/1102774287 learned_addr learned my addr 172.21.15.46:0/1102774287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c1cffc700 1 --2- 172.21.15.46:0/1102774287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef4e380 0x55a2aef55f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c1dffe700 1 -- 172.21.15.46:0/1102774287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef4e380 msgr2=0x55a2aef55f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/1102774287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef4e380 0x55a2aef55f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c1dffe700 1 -- 172.21.15.46:0/1102774287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 msgr2=0x55a2aef58c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/1102774287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef58c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.646+0000 7f2c1dffe700 1 -- 172.21.15.46:0/1102774287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1400d760 con 0x55a2aef59220 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/1102774287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 0x55a2aef56690 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2c10097d10 tx=0x7f2c1009b1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a56eb13576bad47 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c00ff9700 1 -- 172.21.15.46:0/1102774287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1009bc70 con 0x55a2aef59220 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c00ff9700 1 -- 172.21.15.46:0/1102774287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2c1001ce80 con 0x55a2aef59220 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1102774287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2aef41ad0 con 0x55a2aef59220 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c00ff9700 1 -- 172.21.15.46:0/1102774287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c1009b810 con 0x55a2aef59220 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c2bbf6b80 10 client.17229 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1102774287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f2c08010b60 con 0x55a2aef59220 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c00ff9700 1 -- 172.21.15.46:0/1102774287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a2aef41ad0 con 0x55a2aef59220 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c00ff9700 1 -- 172.21.15.46:0/1102774287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2c100a3770 con 0x55a2aef59220 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c2bbf6b80 10 client.17229 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c2bbf6b80 10 client.17229 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:40:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c2bbf6b80 20 client.17229 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1102774287 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a2aef68c20 0x55a2aef6b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c2bbf6b80 4 client.17229 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.647+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1102774287 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2aee99700 con 0x55a2aef68c20 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.648+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/1102774287 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a2aef68c20 0x55a2aef6b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.648+0000 7f2c00ff9700 10 client.17229 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.648+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/1102774287 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a2aef68c20 0x55a2aef6b040 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=523239b9760ff708 server_cookie=434e1cc9ed50ba61 in_seq=0 out_seq=0 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.648+0000 7f2c00ff9700 1 -- 172.21.15.46:0/1102774287 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a2aee99700 con 0x55a2aef68c20 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.648+0000 7f2c00ff9700 10 client.17229 handle_command_reply: tid=0 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c2bbf6b80 1 client.17229 shutdown 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1102774287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 msgr2=0x55a2aef56690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1102774287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 0x55a2aef56690 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2c10097d10 tx=0x7f2c1009b1f0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1102774287 shutdown_connections 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1102774287 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a2aef68c20 0x55a2aef6b040 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1102774287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 0x55a2aef56690 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1102774287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef58c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c00ff9700 0 client.17229 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:40:54.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1102774287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef4e380 0x55a2aef55f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1102774287 >> 172.21.15.46:0/1102774287 conn(0x55a2aeec4220 msgr2=0x55a2aef49c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1102774287 shutdown_connections 2024-03-18T20:40:54.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1102774287 wait complete. 2024-03-18T20:40:54.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.649+0000 7f2c2bbf6b80 20 client.17229 trim_cache size 0 max 16384 2024-03-18T20:40:54.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c2bbf6b80 1 Processor -- start 2024-03-18T20:40:54.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c2bbf6b80 1 -- start start 2024-03-18T20:40:54.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c2bbf6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef5fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c2bbf6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4e380 0x55a2aef58870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c2bbf6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef59220 0x55a2aef49b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c2bbf6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aef4f610 con 0x55a2aef4e380 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c2bbf6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aef60390 con 0x55a2aef4da60 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c2bbf6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aef51df0 con 0x55a2aef59220 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c1cffc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4e380 0x55a2aef58870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c1cffc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4e380 0x55a2aef58870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34038/0 (socket says 172.21.15.46:34038) 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c1cffc700 1 -- 172.21.15.46:0/1988188469 learned_addr learned my addr 172.21.15.46:0/1988188469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/1988188469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef5fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c1cffc700 1 -- 172.21.15.46:0/1988188469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef59220 msgr2=0x55a2aef49b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/1988188469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef59220 0x55a2aef49b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c1cffc700 1 --2- 172.21.15.46:0/1988188469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef59220 0x55a2aef49b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c1cffc700 1 -- 172.21.15.46:0/1988188469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 msgr2=0x55a2aef5fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c1cffc700 1 --2- 172.21.15.46:0/1988188469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef5fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c1cffc700 1 -- 172.21.15.46:0/1988188469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2aecceb60 con 0x55a2aef4e380 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/1988188469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef59220 0x55a2aef49b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:40:54.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.663+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/1988188469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef5fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c1cffc700 1 --2- 172.21.15.46:0/1988188469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4e380 0x55a2aef58870 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2c14004640 tx=0x7f2c1400dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d538f1a3c63537d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c03fff700 1 -- 172.21.15.46:0/1988188469 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14017070 con 0x55a2aef4e380 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c03fff700 1 -- 172.21.15.46:0/1988188469 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c14004cf0 con 0x55a2aef4e380 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c03fff700 1 -- 172.21.15.46:0/1988188469 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14005280 con 0x55a2aef4e380 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1988188469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4e380 msgr2=0x55a2aef58870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1988188469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4e380 0x55a2aef58870 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2c14004640 tx=0x7f2c1400dbd0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1988188469 shutdown_connections 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1988188469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef59220 0x55a2aef49b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1988188469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef5fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/1988188469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef4e380 0x55a2aef58870 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1988188469 >> 172.21.15.46:0/1988188469 conn(0x55a2aeec4220 msgr2=0x55a2aef48750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1988188469 shutdown_connections 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.664+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/1988188469 wait complete. 2024-03-18T20:40:54.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.665+0000 7f2c2bbf6b80 1 -- start start 2024-03-18T20:40:54.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.665+0000 7f2c2bbf6b80 1 Processor -- start 2024-03-18T20:40:54.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c2bbf6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef5fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c2bbf6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef4e380 0x55a2aef5e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c2bbf6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 0x55a2aef5e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c2bbf6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aef5ee80 con 0x55a2aef59220 2024-03-18T20:40:54.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c2bbf6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aef5eff0 con 0x55a2aef4da60 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c2bbf6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2aefbca20 con 0x55a2aef4e380 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c2bbf6b80 10 client.0 _ll_register_callbacks cb 0x7f2c1009b5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c1dffe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 0x55a2aef5e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c1dffe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 0x55a2aef5e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34048/0 (socket says 172.21.15.46:34048) 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c1dffe700 1 -- 172.21.15.46:0/2463467857 learned_addr learned my addr 172.21.15.46:0/2463467857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c1cffc700 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef4e380 0x55a2aef5e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef5fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c1dffe700 1 -- 172.21.15.46:0/2463467857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef4e380 msgr2=0x55a2aef5e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef4e380 0x55a2aef5e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c1dffe700 1 -- 172.21.15.46:0/2463467857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 msgr2=0x55a2aef5fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef5fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c1dffe700 1 -- 172.21.15.46:0/2463467857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2aed223b0 con 0x55a2aef59220 2024-03-18T20:40:54.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c1dffe700 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 0x55a2aef5e7a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2c1009b7a0 tx=0x7f2c100a39a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655840377460a6be server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c00ff9700 1 -- 172.21.15.46:0/2463467857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1009bc70 con 0x55a2aef59220 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c00ff9700 1 -- 172.21.15.46:0/2463467857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f2c1001ce80 con 0x55a2aef59220 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.666+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/2463467857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c08003b40 con 0x55a2aef59220 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c00ff9700 1 -- 172.21.15.46:0/2463467857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c10009b80 con 0x55a2aef59220 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c2bbf6b80 10 client.17235 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/2463467857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f2c14004320 con 0x55a2aef59220 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c00ff9700 1 -- 172.21.15.46:0/2463467857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c1007e080 con 0x55a2aef59220 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c00ff9700 1 -- 172.21.15.46:0/2463467857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2c10015120 con 0x55a2aef59220 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c2bbf6b80 10 client.17235 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c2bbf6b80 10 client.17235 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c2bbf6b80 20 client.17235 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a2aefbdc50 0x55a2aefc0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c2bbf6b80 4 client.17235 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/2463467857 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a2aecefcf0 con 0x55a2aefbdc50 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a2aefbdc50 0x55a2aefc0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c1d7fd700 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a2aefbdc50 0x55a2aefc0040 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd8ef10870acdc2e server_cookie=a0e2cca3904983fb in_seq=0 out_seq=0 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.667+0000 7f2c00ff9700 10 client.17235 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c00ff9700 1 -- 172.21.15.46:0/2463467857 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a2aecefcf0 con 0x55a2aefbdc50 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c00ff9700 10 client.17235 handle_command_reply: tid=0 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c2bbf6b80 1 client.17235 shutdown 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/2463467857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 msgr2=0x55a2aef5e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 0x55a2aef5e7a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2c1009b7a0 tx=0x7f2c100a39a0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/2463467857 shutdown_connections 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a2aefbdc50 0x55a2aefc0040 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a2aef59220 0x55a2aef5e7a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a2aef4da60 0x55a2aef5fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c00ff9700 0 client.17235 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:40:54.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c2bbf6b80 1 --2- 172.21.15.46:0/2463467857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a2aef4e380 0x55a2aef5e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/2463467857 >> 172.21.15.46:0/2463467857 conn(0x55a2aeec4220 msgr2=0x55a2aef51280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/2463467857 shutdown_connections 2024-03-18T20:40:54.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c2bbf6b80 1 -- 172.21.15.46:0/2463467857 wait complete. 2024-03-18T20:40:54.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.668+0000 7f2c2bbf6b80 20 client.17235 trim_cache size 0 max 16384 2024-03-18T20:40:54.678 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:40:54.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.669+0000 7f2c1fa61700 1 -- 172.21.15.46:0/3205707292 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2c040a1290 msgr2=0x7f2c040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.669+0000 7f2c1fa61700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2c040a1290 0x7f2c040a3750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c08007be0 tx=0x7f2c080073d0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.669+0000 7f2c1fa61700 1 -- 172.21.15.46:0/3205707292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c18102370 msgr2=0x7f2c181a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.669+0000 7f2c1fa61700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c18102370 0x7f2c181a54b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2c10006130 tx=0x7f2c10009300 comp rx=0 tx=0).stop 2024-03-18T20:40:54.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.669+0000 7f2c1fa61700 1 -- 172.21.15.46:0/3205707292 shutdown_connections 2024-03-18T20:40:54.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.669+0000 7f2c1fa61700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2c040a1290 0x7f2c040a3750 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.669+0000 7f2c1fa61700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c18102370 0x7f2c181a54b0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.669+0000 7f2c1fa61700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c180ffe60 0x7f2c181018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.669+0000 7f2c1fa61700 1 --2- 172.21.15.46:0/3205707292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c18104f80 0x7f2c18101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.669+0000 7f2c1fa61700 1 -- 172.21.15.46:0/3205707292 >> 172.21.15.46:0/3205707292 conn(0x7f2c180fb6c0 msgr2=0x7f2c180695f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.669+0000 7f2c1fa61700 1 -- 172.21.15.46:0/3205707292 shutdown_connections 2024-03-18T20:40:54.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.669+0000 7f2c1fa61700 1 -- 172.21.15.46:0/3205707292 wait complete. 2024-03-18T20:40:54.685 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:40:54.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:40:54.688 INFO:tasks.workunit.client.0.smithi046.stdout: 1 128393 2.28 MB/sec execute 109 sec latency 956.019 ms 2024-03-18T20:40:54.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.823+0000 7f550d815700 1 -- 172.21.15.46:0/2525922089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5508100820 msgr2=0x7f5508104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.823+0000 7f550d815700 1 --2- 172.21.15.46:0/2525922089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5508100820 0x7f5508104860 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009990 tx=0x7f54f8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.823+0000 7f550d815700 1 -- 172.21.15.46:0/2525922089 shutdown_connections 2024-03-18T20:40:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.823+0000 7f550d815700 1 --2- 172.21.15.46:0/2525922089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5508104f90 0x7f550810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.823+0000 7f550d815700 1 --2- 172.21.15.46:0/2525922089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55080ffe70 0x7f5508100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.823+0000 7f550d815700 1 --2- 172.21.15.46:0/2525922089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5508100820 0x7f5508104860 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.823+0000 7f550d815700 1 -- 172.21.15.46:0/2525922089 >> 172.21.15.46:0/2525922089 conn(0x7f5508076d80 msgr2=0x7f5508077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.823+0000 7f550d815700 1 -- 172.21.15.46:0/2525922089 shutdown_connections 2024-03-18T20:40:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.823+0000 7f550d815700 1 -- 172.21.15.46:0/2525922089 wait complete. 2024-03-18T20:40:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f550d815700 1 Processor -- start 2024-03-18T20:40:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f550d815700 1 -- start start 2024-03-18T20:40:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f550d815700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55080ffe70 0x7f55081a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f550d815700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5508104f90 0x7f55081a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f550d815700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f550819d9c0 0x7f550819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f550d815700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550819e480 con 0x7f550819d9c0 2024-03-18T20:40:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f550d815700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550819e5f0 con 0x7f55080ffe70 2024-03-18T20:40:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f550d815700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550819e7b0 con 0x7f5508104f90 2024-03-18T20:40:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f55077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f550819d9c0 0x7f550819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f5506ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55080ffe70 0x7f55081a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f5506ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55080ffe70 0x7f55081a3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37542/0 (socket says 172.21.15.46:37542) 2024-03-18T20:40:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f5506ffd700 1 -- 172.21.15.46:0/1260009946 learned_addr learned my addr 172.21.15.46:0/1260009946 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.824+0000 7f55067fc700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5508104f90 0x7f55081a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.825+0000 7f5506ffd700 1 -- 172.21.15.46:0/1260009946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5508104f90 msgr2=0x7f55081a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.825+0000 7f5506ffd700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5508104f90 0x7f55081a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.825+0000 7f5506ffd700 1 -- 172.21.15.46:0/1260009946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f550819d9c0 msgr2=0x7f550819de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.825+0000 7f5506ffd700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f550819d9c0 0x7f550819de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.825+0000 7f5506ffd700 1 -- 172.21.15.46:0/1260009946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f80095f0 con 0x7f55080ffe70 2024-03-18T20:40:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.825+0000 7f5506ffd700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55080ffe70 0x7f55081a3550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f54f000e9a0 tx=0x7f54f000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.825+0000 7f550c813700 1 -- 172.21.15.46:0/1260009946 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0009790 con 0x7f55080ffe70 2024-03-18T20:40:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.825+0000 7f550d815700 1 -- 172.21.15.46:0/1260009946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55081aa3d0 con 0x7f55080ffe70 2024-03-18T20:40:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.825+0000 7f550c813700 1 -- 172.21.15.46:0/1260009946 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54f0010430 con 0x7f55080ffe70 2024-03-18T20:40:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.825+0000 7f550c813700 1 -- 172.21.15.46:0/1260009946 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0018680 con 0x7f55080ffe70 2024-03-18T20:40:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.825+0000 7f550d815700 1 -- 172.21.15.46:0/1260009946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55081aa8b0 con 0x7f55080ffe70 2024-03-18T20:40:54.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.828+0000 7f550c813700 1 -- 172.21.15.46:0/1260009946 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f54f00105a0 con 0x7f55080ffe70 2024-03-18T20:40:54.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.828+0000 7f550c813700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f54f40a1290 0x7f54f40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.829+0000 7f550c813700 1 -- 172.21.15.46:0/1260009946 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f54f0014070 con 0x7f55080ffe70 2024-03-18T20:40:54.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.829+0000 7f55067fc700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f54f40a1290 0x7f54f40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.829+0000 7f55199aab80 1 Processor -- start 2024-03-18T20:40:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.829+0000 7f55067fc700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f54f40a1290 0x7f54f40a3750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54f800b2e0 tx=0x7f54f8005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.829+0000 7f55199aab80 1 -- start start 2024-03-18T20:40:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.829+0000 7f55199aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b12e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.829+0000 7f55199aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b13380 0x55b549b1dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.829+0000 7f55199aab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b1e220 0x55b549b20610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.829+0000 7f55199aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b549a6c990 con 0x55b549b12a60 2024-03-18T20:40:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.829+0000 7f55199aab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b549a6a350 con 0x55b549b1e220 2024-03-18T20:40:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.829+0000 7f55199aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b549a65c40 con 0x55b549b13380 2024-03-18T20:40:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f5506ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b12e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f5506ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b12e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34078/0 (socket says 172.21.15.46:34078) 2024-03-18T20:40:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f5506ffd700 1 -- 172.21.15.46:0/3363384558 learned_addr learned my addr 172.21.15.46:0/3363384558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f55077fe700 1 --2- 172.21.15.46:0/3363384558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b1e220 0x55b549b20610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f55067fc700 1 --2- 172.21.15.46:0/3363384558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b13380 0x55b549b1dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f5506ffd700 1 -- 172.21.15.46:0/3363384558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b13380 msgr2=0x55b549b1dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f5506ffd700 1 --2- 172.21.15.46:0/3363384558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b13380 0x55b549b1dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f5506ffd700 1 -- 172.21.15.46:0/3363384558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b1e220 msgr2=0x55b549b20610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f5506ffd700 1 --2- 172.21.15.46:0/3363384558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b1e220 0x55b549b20610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f5506ffd700 1 -- 172.21.15.46:0/3363384558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5498e73b0 con 0x55b549b12a60 2024-03-18T20:40:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f5506ffd700 1 --2- 172.21.15.46:0/3363384558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b12e40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f54f0096a20 tx=0x7f54f0099c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73b6de0006abf9a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f54fdffb700 1 -- 172.21.15.46:0/3363384558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f009dde0 con 0x55b549b12a60 2024-03-18T20:40:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f54fdffb700 1 -- 172.21.15.46:0/3363384558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54f00a2430 con 0x55b549b12a60 2024-03-18T20:40:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.830+0000 7f54fdffb700 1 -- 172.21.15.46:0/3363384558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0099700 con 0x55b549b12a60 2024-03-18T20:40:54.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.831+0000 7f55199aab80 1 -- 172.21.15.46:0/3363384558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 msgr2=0x55b549b12e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.831+0000 7f55199aab80 1 --2- 172.21.15.46:0/3363384558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b12e40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f54f0096a20 tx=0x7f54f0099c20 comp rx=0 tx=0).stop 2024-03-18T20:40:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.831+0000 7f55199aab80 1 -- 172.21.15.46:0/3363384558 shutdown_connections 2024-03-18T20:40:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.831+0000 7f55199aab80 1 --2- 172.21.15.46:0/3363384558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b12e40 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.831+0000 7f55199aab80 1 --2- 172.21.15.46:0/3363384558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b1e220 0x55b549b20610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.831+0000 7f55199aab80 1 --2- 172.21.15.46:0/3363384558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b13380 0x55b549b1dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.831+0000 7f55199aab80 1 -- 172.21.15.46:0/3363384558 >> 172.21.15.46:0/3363384558 conn(0x55b549a89220 msgr2=0x55b549b0f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.831+0000 7f55199aab80 1 -- 172.21.15.46:0/3363384558 shutdown_connections 2024-03-18T20:40:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.831+0000 7f55199aab80 1 -- 172.21.15.46:0/3363384558 wait complete. 2024-03-18T20:40:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.832+0000 7f55199aab80 1 -- start start 2024-03-18T20:40:54.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.832+0000 7f55199aab80 1 Processor -- start 2024-03-18T20:40:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.832+0000 7f55199aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b12a60 0x55b549b1b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.832+0000 7f55199aab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b1b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.832+0000 7f55199aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b1e220 0x55b549b1bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.832+0000 7f55199aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b549a01d80 con 0x55b549b1e220 2024-03-18T20:40:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.832+0000 7f55199aab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b549a01a20 con 0x55b549b13380 2024-03-18T20:40:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.832+0000 7f55199aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5498e96a0 con 0x55b549b12a60 2024-03-18T20:40:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.832+0000 7f55199aab80 10 client.0 _ll_register_callbacks cb 0x55b54986d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f55077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b1e220 0x55b549b1bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f55067fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b1b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f5506ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b12a60 0x55b549b1b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f55077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b1e220 0x55b549b1bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34084/0 (socket says 172.21.15.46:34084) 2024-03-18T20:40:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f55077fe700 1 -- 172.21.15.46:0/2366910908 learned_addr learned my addr 172.21.15.46:0/2366910908 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f55067fc700 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b1b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37560/0 (socket says 172.21.15.46:37560) 2024-03-18T20:40:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f5506ffd700 1 -- 172.21.15.46:0/2366910908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 msgr2=0x55b549b1b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f5506ffd700 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b1b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f5506ffd700 1 -- 172.21.15.46:0/2366910908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b1e220 msgr2=0x55b549b1bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f5506ffd700 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b1e220 0x55b549b1bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f5506ffd700 1 -- 172.21.15.46:0/2366910908 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55081aa8b0 con 0x55b549b12a60 2024-03-18T20:40:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f55077fe700 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b1e220 0x55b549b1bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f5506ffd700 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b12a60 0x55b549b1b0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f54f0099f40 tx=0x7f54f0010ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd00883d87e23d0c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f54e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:40:54.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f54e6ffd700 1 -- 172.21.15.46:0/2366910908 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00a1fa0 con 0x55b549b12a60 2024-03-18T20:40:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.833+0000 7f55199aab80 1 -- 172.21.15.46:0/2366910908 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b549b06ad0 con 0x55b549b12a60 2024-03-18T20:40:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f54e6ffd700 1 -- 172.21.15.46:0/2366910908 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f54f00a2100 con 0x55b549b12a60 2024-03-18T20:40:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f54e6ffd700 1 -- 172.21.15.46:0/2366910908 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54f00b82f0 con 0x55b549b12a60 2024-03-18T20:40:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f55199aab80 10 client.26968 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f55199aab80 1 -- 172.21.15.46:0/2366910908 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55b549893b60 con 0x55b549b12a60 2024-03-18T20:40:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f54e6ffd700 1 -- 172.21.15.46:0/2366910908 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b549b06ad0 con 0x55b549b12a60 2024-03-18T20:40:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f54e6ffd700 1 -- 172.21.15.46:0/2366910908 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f54f00990a0 con 0x55b549b12a60 2024-03-18T20:40:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f55199aab80 10 client.26968 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:54.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f55199aab80 10 client.26968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f55199aab80 20 client.26968 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f55199aab80 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b549b2db20 0x55b549b2ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f55199aab80 4 client.26968 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f55199aab80 1 -- 172.21.15.46:0/2366910908 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b549a5e700 con 0x55b549b2db20 2024-03-18T20:40:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.834+0000 7f55067fc700 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b549b2db20 0x55b549b2ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.835+0000 7f55067fc700 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b549b2db20 0x55b549b2ff10 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=561a431ad3ef95b9 server_cookie=b801a5b8884a1ac4 in_seq=0 out_seq=0 2024-03-18T20:40:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.835+0000 7f54e6ffd700 10 client.26968 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.835+0000 7f54e6ffd700 1 -- 172.21.15.46:0/2366910908 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b549a5e700 con 0x55b549b2db20 2024-03-18T20:40:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.835+0000 7f54e6ffd700 10 client.26968 handle_command_reply: tid=0 2024-03-18T20:40:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.835+0000 7f55199aab80 1 client.26968 shutdown 2024-03-18T20:40:54.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.836+0000 7f55199aab80 1 -- 172.21.15.46:0/2366910908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b12a60 msgr2=0x55b549b1b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.836+0000 7f55199aab80 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b12a60 0x55b549b1b0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f54f0099f40 tx=0x7f54f0010ab0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.836+0000 7f55199aab80 1 -- 172.21.15.46:0/2366910908 shutdown_connections 2024-03-18T20:40:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.836+0000 7f55199aab80 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b549b2db20 0x55b549b2ff10 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.836+0000 7f55199aab80 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b1e220 0x55b549b1bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.836+0000 7f54e6ffd700 0 client.26968 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.836+0000 7f55199aab80 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b12a60 0x55b549b1b0b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.836+0000 7f55199aab80 1 --2- 172.21.15.46:0/2366910908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b1b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.836+0000 7f55199aab80 1 -- 172.21.15.46:0/2366910908 >> 172.21.15.46:0/2366910908 conn(0x55b549a89220 msgr2=0x55b549b0ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.836+0000 7f55199aab80 1 -- 172.21.15.46:0/2366910908 shutdown_connections 2024-03-18T20:40:54.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.836+0000 7f55199aab80 1 -- 172.21.15.46:0/2366910908 wait complete. 2024-03-18T20:40:54.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.836+0000 7f55199aab80 20 client.26968 trim_cache size 0 max 16384 2024-03-18T20:40:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f55199aab80 1 Processor -- start 2024-03-18T20:40:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f55199aab80 1 -- start start 2024-03-18T20:40:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f55199aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b24e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f55199aab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b16380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f55199aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b1e220 0x55b549b168c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f55199aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b549b10440 con 0x55b549b12a60 2024-03-18T20:40:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f55199aab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b549b13d30 con 0x55b549b13380 2024-03-18T20:40:54.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f55199aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b549b0b5e0 con 0x55b549b1e220 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f5506ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b24e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f5506ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b24e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34088/0 (socket says 172.21.15.46:34088) 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f5506ffd700 1 -- 172.21.15.46:0/1482247819 learned_addr learned my addr 172.21.15.46:0/1482247819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f55077fe700 1 --2- 172.21.15.46:0/1482247819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b1e220 0x55b549b168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f55067fc700 1 --2- 172.21.15.46:0/1482247819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b16380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f5506ffd700 1 -- 172.21.15.46:0/1482247819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b1e220 msgr2=0x55b549b168c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f5506ffd700 1 --2- 172.21.15.46:0/1482247819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b1e220 0x55b549b168c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f5506ffd700 1 -- 172.21.15.46:0/1482247819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 msgr2=0x55b549b16380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f5506ffd700 1 --2- 172.21.15.46:0/1482247819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b16380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f5506ffd700 1 -- 172.21.15.46:0/1482247819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b549a2bea0 con 0x55b549b12a60 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f55077fe700 1 --2- 172.21.15.46:0/1482247819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b1e220 0x55b549b168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f55067fc700 1 --2- 172.21.15.46:0/1482247819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b16380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.851+0000 7f5506ffd700 1 --2- 172.21.15.46:0/1482247819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b24e50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f54f0010cc0 tx=0x7f54f0099c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e52d28085ddbea server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.852+0000 7f54fe7fc700 1 -- 172.21.15.46:0/1482247819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f009a6b0 con 0x55b549b12a60 2024-03-18T20:40:54.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.852+0000 7f54fe7fc700 1 -- 172.21.15.46:0/1482247819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54f0098410 con 0x55b549b12a60 2024-03-18T20:40:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.852+0000 7f54fe7fc700 1 -- 172.21.15.46:0/1482247819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00a21e0 con 0x55b549b12a60 2024-03-18T20:40:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.852+0000 7f55199aab80 1 -- 172.21.15.46:0/1482247819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 msgr2=0x55b549b24e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.852+0000 7f55199aab80 1 --2- 172.21.15.46:0/1482247819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b24e50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f54f0010cc0 tx=0x7f54f0099c20 comp rx=0 tx=0).stop 2024-03-18T20:40:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.852+0000 7f55199aab80 1 -- 172.21.15.46:0/1482247819 shutdown_connections 2024-03-18T20:40:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.852+0000 7f55199aab80 1 --2- 172.21.15.46:0/1482247819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b24e50 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.852+0000 7f55199aab80 1 --2- 172.21.15.46:0/1482247819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b1e220 0x55b549b168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.852+0000 7f55199aab80 1 --2- 172.21.15.46:0/1482247819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b16380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.852+0000 7f55199aab80 1 -- 172.21.15.46:0/1482247819 >> 172.21.15.46:0/1482247819 conn(0x55b549a89220 msgr2=0x55b549b1b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.853+0000 7f55199aab80 1 -- 172.21.15.46:0/1482247819 shutdown_connections 2024-03-18T20:40:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.853+0000 7f55199aab80 1 -- 172.21.15.46:0/1482247819 wait complete. 2024-03-18T20:40:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.853+0000 7f55199aab80 1 -- start start 2024-03-18T20:40:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.853+0000 7f55199aab80 1 Processor -- start 2024-03-18T20:40:54.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.853+0000 7f55199aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b24600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.853+0000 7f55199aab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b24b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.853+0000 7f55199aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b1e220 0x55b549b78e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.853+0000 7f55199aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b549b793f0 con 0x55b549b12a60 2024-03-18T20:40:54.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.853+0000 7f55199aab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b549b79560 con 0x55b549b13380 2024-03-18T20:40:54.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.853+0000 7f55199aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b549b79720 con 0x55b549b1e220 2024-03-18T20:40:54.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f55199aab80 10 client.0 _ll_register_callbacks cb 0x7f5500009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f5506ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b24600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f5506ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b24600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34096/0 (socket says 172.21.15.46:34096) 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f5506ffd700 1 -- 172.21.15.46:0/2871549970 learned_addr learned my addr 172.21.15.46:0/2871549970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f55077fe700 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b1e220 0x55b549b78e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f55067fc700 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b24b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f5506ffd700 1 -- 172.21.15.46:0/2871549970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b1e220 msgr2=0x55b549b78e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f5506ffd700 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b1e220 0x55b549b78e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f5506ffd700 1 -- 172.21.15.46:0/2871549970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 msgr2=0x55b549b24b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f5506ffd700 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b24b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f5506ffd700 1 -- 172.21.15.46:0/2871549970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b549893b60 con 0x55b549b12a60 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f5506ffd700 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b24600 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f54f00998a0 tx=0x7f54f0010ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce137baf8a16ad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f54e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f54e6ffd700 1 -- 172.21.15.46:0/2871549970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00b83b0 con 0x55b549b12a60 2024-03-18T20:40:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f54e6ffd700 1 -- 172.21.15.46:0/2871549970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f54f009ee00 con 0x55b549b12a60 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.854+0000 7f55199aab80 1 -- 172.21.15.46:0/2871549970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54f00a2430 con 0x55b549b12a60 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.855+0000 7f54e6ffd700 1 -- 172.21.15.46:0/2871549970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54f009efc0 con 0x55b549b12a60 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.855+0000 7f55199aab80 10 client.17253 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.855+0000 7f55199aab80 1 -- 172.21.15.46:0/2871549970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55b5498ebcd0 con 0x55b549b12a60 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.855+0000 7f54e6ffd700 1 -- 172.21.15.46:0/2871549970 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54f00a2430 con 0x55b549b12a60 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.855+0000 7f54e6ffd700 1 -- 172.21.15.46:0/2871549970 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f54f000ca10 con 0x55b549b12a60 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.855+0000 7f55199aab80 10 client.17253 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.855+0000 7f55199aab80 10 client.17253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.855+0000 7f55199aab80 20 client.17253 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.855+0000 7f55199aab80 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b549b81c50 0x55b549b84040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.855+0000 7f55199aab80 4 client.17253 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.855+0000 7f55199aab80 1 -- 172.21.15.46:0/2871549970 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b5498b4cf0 con 0x55b549b81c50 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.855+0000 7f55067fc700 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b549b81c50 0x55b549b84040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:54.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55067fc700 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b549b81c50 0x55b549b84040 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=190162a86fe31982 server_cookie=988e9ebd1476c01 in_seq=0 out_seq=0 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f54e6ffd700 10 client.17253 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f54e6ffd700 1 -- 172.21.15.46:0/2871549970 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b5498b4cf0 con 0x55b549b81c50 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f54e6ffd700 10 client.17253 handle_command_reply: tid=0 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55199aab80 1 client.17253 shutdown 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55199aab80 1 -- 172.21.15.46:0/2871549970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 msgr2=0x55b549b24600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55199aab80 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b24600 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f54f00998a0 tx=0x7f54f0010ab0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55199aab80 1 -- 172.21.15.46:0/2871549970 shutdown_connections 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55199aab80 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b549b81c50 0x55b549b84040 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55199aab80 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b549b12a60 0x55b549b24600 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55199aab80 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b549b1e220 0x55b549b78e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55199aab80 1 --2- 172.21.15.46:0/2871549970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b549b13380 0x55b549b24b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f54e6ffd700 0 client.17253 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55199aab80 1 -- 172.21.15.46:0/2871549970 >> 172.21.15.46:0/2871549970 conn(0x55b549a89220 msgr2=0x55b549b16380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55199aab80 1 -- 172.21.15.46:0/2871549970 shutdown_connections 2024-03-18T20:40:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55199aab80 1 -- 172.21.15.46:0/2871549970 wait complete. 2024-03-18T20:40:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.856+0000 7f55199aab80 20 client.17253 trim_cache size 0 max 16384 2024-03-18T20:40:54.866 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:40:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.857+0000 7f550d815700 1 -- 172.21.15.46:0/1260009946 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f54f40a1290 msgr2=0x7f54f40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.857+0000 7f550d815700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f54f40a1290 0x7f54f40a3750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54f800b2e0 tx=0x7f54f8005960 comp rx=0 tx=0).stop 2024-03-18T20:40:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.857+0000 7f550d815700 1 -- 172.21.15.46:0/1260009946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55080ffe70 msgr2=0x7f55081a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.857+0000 7f550d815700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55080ffe70 0x7f55081a3550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f54f000e9a0 tx=0x7f54f000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.857+0000 7f550d815700 1 -- 172.21.15.46:0/1260009946 shutdown_connections 2024-03-18T20:40:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.857+0000 7f550d815700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f54f40a1290 0x7f54f40a3750 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.857+0000 7f550d815700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f550819d9c0 0x7f550819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.857+0000 7f550d815700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55080ffe70 0x7f55081a3550 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.857+0000 7f550d815700 1 --2- 172.21.15.46:0/1260009946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5508104f90 0x7f55081a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.857+0000 7f550d815700 1 -- 172.21.15.46:0/1260009946 >> 172.21.15.46:0/1260009946 conn(0x7f5508076d80 msgr2=0x7f5508069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.857+0000 7f550d815700 1 -- 172.21.15.46:0/1260009946 shutdown_connections 2024-03-18T20:40:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:54.857+0000 7f550d815700 1 -- 172.21.15.46:0/1260009946 wait complete. 2024-03-18T20:40:55.687 INFO:tasks.workunit.client.0.smithi046.stdout: 1 128393 2.26 MB/sec execute 110 sec latency 1956.137 ms 2024-03-18T20:40:55.874 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:40:55.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:40:55.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:40:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.022+0000 7f0e23b8c700 1 -- 172.21.15.46:0/158453367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c106090 msgr2=0x7f0e1c10dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.022+0000 7f0e23b8c700 1 --2- 172.21.15.46:0/158453367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c106090 0x7f0e1c10dc10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400b0a0 tx=0x7f0e1400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.022+0000 7f0e23b8c700 1 -- 172.21.15.46:0/158453367 shutdown_connections 2024-03-18T20:40:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.022+0000 7f0e23b8c700 1 --2- 172.21.15.46:0/158453367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c106090 0x7f0e1c10dc10 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.022+0000 7f0e23b8c700 1 --2- 172.21.15.46:0/158453367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e1c100f70 0x7f0e1c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.022+0000 7f0e23b8c700 1 --2- 172.21.15.46:0/158453367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c101920 0x7f0e1c105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.022+0000 7f0e23b8c700 1 -- 172.21.15.46:0/158453367 >> 172.21.15.46:0/158453367 conn(0x7f0e1c0fc7f0 msgr2=0x7f0e1c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.022+0000 7f0e23b8c700 1 -- 172.21.15.46:0/158453367 shutdown_connections 2024-03-18T20:40:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.023+0000 7f0e23b8c700 1 -- 172.21.15.46:0/158453367 wait complete. 2024-03-18T20:40:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.023+0000 7f0e23b8c700 1 Processor -- start 2024-03-18T20:40:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.023+0000 7f0e23b8c700 1 -- start start 2024-03-18T20:40:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.023+0000 7f0e23b8c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c100f70 0x7f0e1c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.023+0000 7f0e23b8c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c106090 0x7f0e1c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.023+0000 7f0e23b8c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e1c073b30 0x7f0e1c1a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.023+0000 7f0e23b8c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c1a5ae0 con 0x7f0e1c106090 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.023+0000 7f0e23b8c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c1a5c20 con 0x7f0e1c100f70 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.023+0000 7f0e23b8c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c1a5de0 con 0x7f0e1c073b30 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e21127700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c106090 0x7f0e1c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e21127700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c106090 0x7f0e1c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39116/0 (socket says 172.21.15.46:39116) 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e21928700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c100f70 0x7f0e1c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e21127700 1 -- 172.21.15.46:0/3965217611 learned_addr learned my addr 172.21.15.46:0/3965217611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e22129700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e1c073b30 0x7f0e1c1a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e21127700 1 -- 172.21.15.46:0/3965217611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e1c073b30 msgr2=0x7f0e1c1a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e21127700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e1c073b30 0x7f0e1c1a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e21127700 1 -- 172.21.15.46:0/3965217611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c100f70 msgr2=0x7f0e1c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e21127700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c100f70 0x7f0e1c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e21127700 1 -- 172.21.15.46:0/3965217611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e14009d00 con 0x7f0e1c106090 2024-03-18T20:40:56.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e21127700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c106090 0x7f0e1c0735f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000e970 tx=0x7f0e1000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:56.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e0effd700 1 -- 172.21.15.46:0/3965217611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10009790 con 0x7f0e1c106090 2024-03-18T20:40:56.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e0effd700 1 -- 172.21.15.46:0/3965217611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e1000c7e0 con 0x7f0e1c106090 2024-03-18T20:40:56.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e23b8c700 1 -- 172.21.15.46:0/3965217611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c1a60c0 con 0x7f0e1c106090 2024-03-18T20:40:56.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e0effd700 1 -- 172.21.15.46:0/3965217611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e100054e0 con 0x7f0e1c106090 2024-03-18T20:40:56.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.024+0000 7f0e23b8c700 1 -- 172.21.15.46:0/3965217611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c1a65d0 con 0x7f0e1c106090 2024-03-18T20:40:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.027+0000 7f0e0effd700 1 -- 172.21.15.46:0/3965217611 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0e1000c950 con 0x7f0e1c106090 2024-03-18T20:40:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.028+0000 7f0e0effd700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0e080a1290 0x7f0e080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.028+0000 7f0e21928700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0e080a1290 0x7f0e080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.028+0000 7f0e0effd700 1 -- 172.21.15.46:0/3965217611 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0e10014070 con 0x7f0e1c106090 2024-03-18T20:40:56.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.028+0000 7f0e21928700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0e080a1290 0x7f0e080a3750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009c60 tx=0x7f0e18009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.028+0000 7f0e2fd21b80 1 Processor -- start 2024-03-18T20:40:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e2fd21b80 1 -- start start 2024-03-18T20:40:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e2fd21b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565eeae0 0x5615565eeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e2fd21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 0x5615565f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e2fd21b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 0x5615565fc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e2fd21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561556548a10 con 0x5615565ef400 2024-03-18T20:40:56.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e2fd21b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615565463d0 con 0x5615565eeae0 2024-03-18T20:40:56.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e2fd21b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561556541cc0 con 0x5615565fa2a0 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e21127700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 0x5615565f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e21127700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 0x5615565f9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39126/0 (socket says 172.21.15.46:39126) 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e21127700 1 -- 172.21.15.46:0/3360220242 learned_addr learned my addr 172.21.15.46:0/3360220242 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e22129700 1 --2- 172.21.15.46:0/3360220242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 0x5615565fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e21928700 1 --2- 172.21.15.46:0/3360220242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565eeae0 0x5615565eeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e21127700 1 -- 172.21.15.46:0/3360220242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 msgr2=0x5615565fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e21127700 1 --2- 172.21.15.46:0/3360220242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 0x5615565fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e21127700 1 -- 172.21.15.46:0/3360220242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565eeae0 msgr2=0x5615565eeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e21127700 1 --2- 172.21.15.46:0/3360220242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565eeae0 0x5615565eeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.029+0000 7f0e21127700 1 -- 172.21.15.46:0/3360220242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615563c3430 con 0x5615565ef400 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.030+0000 7f0e21127700 1 --2- 172.21.15.46:0/3360220242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 0x5615565f9d60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0e100a1f00 tx=0x7f0e1000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef20a9a2d674347 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.030+0000 7f0e07fff700 1 -- 172.21.15.46:0/3360220242 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1009dde0 con 0x5615565ef400 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.030+0000 7f0e07fff700 1 -- 172.21.15.46:0/3360220242 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e100a2430 con 0x5615565ef400 2024-03-18T20:40:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.030+0000 7f0e07fff700 1 -- 172.21.15.46:0/3360220242 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10016590 con 0x5615565ef400 2024-03-18T20:40:56.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.031+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/3360220242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 msgr2=0x5615565f9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.031+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/3360220242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 0x5615565f9d60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0e100a1f00 tx=0x7f0e1000ccd0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.031+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/3360220242 shutdown_connections 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.031+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/3360220242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 0x5615565fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.031+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/3360220242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565eeae0 0x5615565eeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.031+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/3360220242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 0x5615565f9d60 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.031+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/3360220242 >> 172.21.15.46:0/3360220242 conn(0x5615565652a0 msgr2=0x5615565eb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.031+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/3360220242 shutdown_connections 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.031+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/3360220242 wait complete. 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.031+0000 7f0e2fd21b80 1 -- start start 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.031+0000 7f0e2fd21b80 1 Processor -- start 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e2fd21b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565eeae0 0x5615565f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e2fd21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 0x5615565f7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e2fd21b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 0x5615565f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e2fd21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615564dde00 con 0x5615565ef400 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e2fd21b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615564ddaa0 con 0x5615565eeae0 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e2fd21b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615563c5720 con 0x5615565fa2a0 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e2fd21b80 10 client.0 _ll_register_callbacks cb 0x561556349420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e21127700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 0x5615565f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e21127700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 0x5615565f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39130/0 (socket says 172.21.15.46:39130) 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e21127700 1 -- 172.21.15.46:0/4085619677 learned_addr learned my addr 172.21.15.46:0/4085619677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e21928700 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565eeae0 0x5615565f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e22129700 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 0x5615565f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e21127700 1 -- 172.21.15.46:0/4085619677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 msgr2=0x5615565f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e21127700 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 0x5615565f7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e21127700 1 -- 172.21.15.46:0/4085619677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565eeae0 msgr2=0x5615565f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e21127700 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565eeae0 0x5615565f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e21127700 1 -- 172.21.15.46:0/4085619677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c1a65d0 con 0x5615565ef400 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e21127700 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 0x5615565f7620 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000cf90 tx=0x7f0e1009dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2db998413213b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e057fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e057fa700 1 -- 172.21.15.46:0/4085619677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e10016530 con 0x5615565ef400 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e057fa700 1 -- 172.21.15.46:0/4085619677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0e100994a0 con 0x5615565ef400 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.032+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/4085619677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615565e2b50 con 0x5615565ef400 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.033+0000 7f0e057fa700 1 -- 172.21.15.46:0/4085619677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e10099660 con 0x5615565ef400 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.033+0000 7f0e2fd21b80 10 client.17265 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.033+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/4085619677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56155636fbe0 con 0x5615565ef400 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.033+0000 7f0e057fa700 1 -- 172.21.15.46:0/4085619677 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615565e2b50 con 0x5615565ef400 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.033+0000 7f0e057fa700 1 -- 172.21.15.46:0/4085619677 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0e10094bc0 con 0x5615565ef400 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.033+0000 7f0e2fd21b80 10 client.17265 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.033+0000 7f0e2fd21b80 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.033+0000 7f0e2fd21b80 20 client.17265 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:56.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.033+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561556609c70 0x56155660c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.033+0000 7f0e2fd21b80 4 client.17265 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.033+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/4085619677 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56155653a780 con 0x561556609c70 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.033+0000 7f0e21928700 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561556609c70 0x56155660c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.034+0000 7f0e21928700 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561556609c70 0x56155660c090 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70610cd6a5a13505 server_cookie=1121371af2cd1207 in_seq=0 out_seq=0 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.034+0000 7f0e057fa700 10 client.17265 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.034+0000 7f0e057fa700 1 -- 172.21.15.46:0/4085619677 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56155653a780 con 0x561556609c70 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.034+0000 7f0e057fa700 10 client.17265 handle_command_reply: tid=0 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.034+0000 7f0e2fd21b80 1 client.17265 shutdown 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.034+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/4085619677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 msgr2=0x5615565f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.034+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 0x5615565f7620 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000cf90 tx=0x7f0e1009dde0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.035+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/4085619677 shutdown_connections 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.035+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561556609c70 0x56155660c090 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.035+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 0x5615565f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.035+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565eeae0 0x5615565f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.035+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/4085619677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565ef400 0x5615565f7620 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.035+0000 7f0e057fa700 0 client.17265 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.035+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/4085619677 >> 172.21.15.46:0/4085619677 conn(0x5615565652a0 msgr2=0x5615565eac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.035+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/4085619677 shutdown_connections 2024-03-18T20:40:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.035+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/4085619677 wait complete. 2024-03-18T20:40:56.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.035+0000 7f0e2fd21b80 20 client.17265 trim_cache size 0 max 16384 2024-03-18T20:40:56.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e2fd21b80 1 Processor -- start 2024-03-18T20:40:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e2fd21b80 1 -- start start 2024-03-18T20:40:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e2fd21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 0x5615565f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e2fd21b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565ef400 0x5615565f7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e2fd21b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 0x5615565f4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e2fd21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615565e5290 con 0x5615565eeae0 2024-03-18T20:40:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e2fd21b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615565ec4c0 con 0x5615565ef400 2024-03-18T20:40:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e2fd21b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615565f86e0 con 0x5615565fa2a0 2024-03-18T20:40:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e21928700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 0x5615565f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e21928700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 0x5615565f1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39142/0 (socket says 172.21.15.46:39142) 2024-03-18T20:40:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e21928700 1 -- 172.21.15.46:0/2230982705 learned_addr learned my addr 172.21.15.46:0/2230982705 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e21127700 1 --2- 172.21.15.46:0/2230982705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565ef400 0x5615565f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e21928700 1 -- 172.21.15.46:0/2230982705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 msgr2=0x5615565f4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e21928700 1 --2- 172.21.15.46:0/2230982705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 0x5615565f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e21928700 1 -- 172.21.15.46:0/2230982705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565ef400 msgr2=0x5615565f7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e21928700 1 --2- 172.21.15.46:0/2230982705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565ef400 0x5615565f7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e21928700 1 -- 172.21.15.46:0/2230982705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561556507f20 con 0x5615565eeae0 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e21127700 1 --2- 172.21.15.46:0/2230982705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565ef400 0x5615565f7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e21928700 1 --2- 172.21.15.46:0/2230982705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 0x5615565f1b40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0e18013010 tx=0x7f0e18014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127412f83b42f9ea server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.053+0000 7f0e0cda8700 1 -- 172.21.15.46:0/2230982705 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18023040 con 0x5615565eeae0 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.054+0000 7f0e0cda8700 1 -- 172.21.15.46:0/2230982705 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e18028c60 con 0x5615565eeae0 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.054+0000 7f0e0cda8700 1 -- 172.21.15.46:0/2230982705 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1801d670 con 0x5615565eeae0 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.054+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2230982705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 msgr2=0x5615565f1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.054+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/2230982705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 0x5615565f1b40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0e18013010 tx=0x7f0e18014000 comp rx=0 tx=0).stop 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.054+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2230982705 shutdown_connections 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.054+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/2230982705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 0x5615565f1b40 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.054+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/2230982705 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565fa2a0 0x5615565f4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.054+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/2230982705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565ef400 0x5615565f7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.054+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2230982705 >> 172.21.15.46:0/2230982705 conn(0x5615565652a0 msgr2=0x5615565f6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:56.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.054+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2230982705 shutdown_connections 2024-03-18T20:40:56.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.054+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2230982705 wait complete. 2024-03-18T20:40:56.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.055+0000 7f0e2fd21b80 1 -- start start 2024-03-18T20:40:56.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.055+0000 7f0e2fd21b80 1 Processor -- start 2024-03-18T20:40:56.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.055+0000 7f0e2fd21b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 0x5615565ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.055+0000 7f0e2fd21b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565ef400 0x561556600250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.055+0000 7f0e2fd21b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565fa2a0 0x56155665da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.055+0000 7f0e2fd21b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561556600910 con 0x5615565eeae0 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.055+0000 7f0e2fd21b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155665dfb0 con 0x5615565fa2a0 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.055+0000 7f0e2fd21b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56155665e0f0 con 0x5615565ef400 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.055+0000 7f0e2fd21b80 10 client.0 _ll_register_callbacks cb 0x7f0e10099930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.055+0000 7f0e21928700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 0x5615565ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.055+0000 7f0e21928700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 0x5615565ffd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39148/0 (socket says 172.21.15.46:39148) 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.055+0000 7f0e21928700 1 -- 172.21.15.46:0/2988532767 learned_addr learned my addr 172.21.15.46:0/2988532767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e21127700 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565ef400 0x561556600250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e22129700 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565fa2a0 0x56155665da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e21928700 1 -- 172.21.15.46:0/2988532767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565ef400 msgr2=0x561556600250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e21928700 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565ef400 0x561556600250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e21928700 1 -- 172.21.15.46:0/2988532767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565fa2a0 msgr2=0x56155665da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e21928700 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565fa2a0 0x56155665da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e21928700 1 -- 172.21.15.46:0/2988532767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e18017070 con 0x5615565eeae0 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e21127700 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565ef400 0x561556600250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e22129700 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565fa2a0 0x56155665da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e21928700 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 0x5615565ffd10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0e18004090 tx=0x7f0e180284d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b73da28185c5a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e057fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e057fa700 1 -- 172.21.15.46:0/2988532767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18023050 con 0x5615565eeae0 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e057fa700 1 -- 172.21.15.46:0/2988532767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0e18005040 con 0x5615565eeae0 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2988532767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e100a2430 con 0x5615565eeae0 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e057fa700 1 -- 172.21.15.46:0/2988532767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e18021070 con 0x5615565eeae0 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e2fd21b80 10 client.17271 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.056+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2988532767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5615563c7d50 con 0x5615565eeae0 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.057+0000 7f0e057fa700 1 -- 172.21.15.46:0/2988532767 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e100a2430 con 0x5615565eeae0 2024-03-18T20:40:56.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.057+0000 7f0e057fa700 1 -- 172.21.15.46:0/2988532767 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0e18004cf0 con 0x5615565eeae0 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.057+0000 7f0e2fd21b80 10 client.17271 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.057+0000 7f0e2fd21b80 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.057+0000 7f0e2fd21b80 20 client.17271 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.057+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56155665f7b0 0x561556661bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.057+0000 7f0e2fd21b80 4 client.17271 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.057+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2988532767 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561556390d70 con 0x56155665f7b0 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.057+0000 7f0e21127700 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56155665f7b0 0x561556661bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.057+0000 7f0e057fa700 10 client.17271 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.057+0000 7f0e21127700 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56155665f7b0 0x561556661bd0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ccc3a76b50b2e9 server_cookie=d7c82e3bcf897f2c in_seq=0 out_seq=0 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.058+0000 7f0e057fa700 1 -- 172.21.15.46:0/2988532767 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561556390d70 con 0x56155665f7b0 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.058+0000 7f0e057fa700 10 client.17271 handle_command_reply: tid=0 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.058+0000 7f0e2fd21b80 1 client.17271 shutdown 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2988532767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 msgr2=0x5615565ffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 0x5615565ffd10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0e18004090 tx=0x7f0e180284d0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2988532767 shutdown_connections 2024-03-18T20:40:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56155665f7b0 0x561556661bd0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615565eeae0 0x5615565ffd10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5615565fa2a0 0x56155665da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e057fa700 0 client.17271 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e2fd21b80 1 --2- 172.21.15.46:0/2988532767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5615565ef400 0x561556600250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2988532767 >> 172.21.15.46:0/2988532767 conn(0x5615565652a0 msgr2=0x5615565f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2988532767 shutdown_connections 2024-03-18T20:40:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e2fd21b80 1 -- 172.21.15.46:0/2988532767 wait complete. 2024-03-18T20:40:56.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e2fd21b80 20 client.17271 trim_cache size 0 max 16384 2024-03-18T20:40:56.067 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:40:56.068 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:40:56.068 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "5ba3037f-ca91-401b-9bf7-d455a4425bc3", 2024-03-18T20:40:56.068 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:40:56.068 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:40:56.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e23b8c700 1 -- 172.21.15.46:0/3965217611 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0e080a1290 msgr2=0x7f0e080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e23b8c700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0e080a1290 0x7f0e080a3750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009c60 tx=0x7f0e18009260 comp rx=0 tx=0).stop 2024-03-18T20:40:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e23b8c700 1 -- 172.21.15.46:0/3965217611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c106090 msgr2=0x7f0e1c0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.059+0000 7f0e23b8c700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c106090 0x7f0e1c0735f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000e970 tx=0x7f0e1000ec80 comp rx=0 tx=0).stop 2024-03-18T20:40:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.060+0000 7f0e23b8c700 1 -- 172.21.15.46:0/3965217611 shutdown_connections 2024-03-18T20:40:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.060+0000 7f0e23b8c700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0e080a1290 0x7f0e080a3750 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.060+0000 7f0e23b8c700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e1c073b30 0x7f0e1c1a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.060+0000 7f0e23b8c700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c100f70 0x7f0e1c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.060+0000 7f0e23b8c700 1 --2- 172.21.15.46:0/3965217611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c106090 0x7f0e1c0735f0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.060+0000 7f0e23b8c700 1 -- 172.21.15.46:0/3965217611 >> 172.21.15.46:0/3965217611 conn(0x7f0e1c0fc7f0 msgr2=0x7f0e1c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.060+0000 7f0e23b8c700 1 -- 172.21.15.46:0/3965217611 shutdown_connections 2024-03-18T20:40:56.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.060+0000 7f0e23b8c700 1 -- 172.21.15.46:0/3965217611 wait complete. 2024-03-18T20:40:56.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:40:56.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:40:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:55 smithi073 ceph-mon[47384]: pgmap v148: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 924 KiB/s wr, 102 op/s 2024-03-18T20:40:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.222+0000 7f39633c9700 1 -- 172.21.15.46:0/3698486048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c105c50 msgr2=0x7f395c069360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.222+0000 7f39633c9700 1 --2- 172.21.15.46:0/3698486048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c105c50 0x7f395c069360 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f394c00b0a0 tx=0x7f394c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.222+0000 7f39633c9700 1 -- 172.21.15.46:0/3698486048 shutdown_connections 2024-03-18T20:40:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.222+0000 7f39633c9700 1 --2- 172.21.15.46:0/3698486048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c105c50 0x7f395c069360 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.222+0000 7f39633c9700 1 --2- 172.21.15.46:0/3698486048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f395c103750 0x7f395c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.222+0000 7f39633c9700 1 --2- 172.21.15.46:0/3698486048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f395c0698a0 0x7f395c069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.222+0000 7f39633c9700 1 -- 172.21.15.46:0/3698486048 >> 172.21.15.46:0/3698486048 conn(0x7f395c076db0 msgr2=0x7f395c0771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.222+0000 7f39633c9700 1 -- 172.21.15.46:0/3698486048 shutdown_connections 2024-03-18T20:40:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.222+0000 7f39633c9700 1 -- 172.21.15.46:0/3698486048 wait complete. 2024-03-18T20:40:56.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.223+0000 7f39633c9700 1 Processor -- start 2024-03-18T20:40:56.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.223+0000 7f39633c9700 1 -- start start 2024-03-18T20:40:56.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.223+0000 7f39633c9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f395c0698a0 0x7f395c1a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f39633c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c103750 0x7f395c1a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f39633c9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f395c105c50 0x7f395c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f39633c9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c19de40 con 0x7f395c103750 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f39633c9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c19dfb0 con 0x7f395c0698a0 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f39633c9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c19e170 con 0x7f395c105c50 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f3961165700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f395c0698a0 0x7f395c1a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f3961165700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f395c0698a0 0x7f395c1a36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47280/0 (socket says 172.21.15.46:47280) 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f3961165700 1 -- 172.21.15.46:0/1430351163 learned_addr learned my addr 172.21.15.46:0/1430351163 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f3960964700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c103750 0x7f395c1a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f3961966700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f395c105c50 0x7f395c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f3961165700 1 -- 172.21.15.46:0/1430351163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f395c105c50 msgr2=0x7f395c19d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f3961165700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f395c105c50 0x7f395c19d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f3961165700 1 -- 172.21.15.46:0/1430351163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c103750 msgr2=0x7f395c1a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f3961165700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c103750 0x7f395c1a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f3961165700 1 -- 172.21.15.46:0/1430351163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3954009520 con 0x7f395c0698a0 2024-03-18T20:40:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f3961165700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f395c0698a0 0x7f395c1a36a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f394c00b780 tx=0x7f394c007940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f395a7fc700 1 -- 172.21.15.46:0/1430351163 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c00e040 con 0x7f395c0698a0 2024-03-18T20:40:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f39633c9700 1 -- 172.21.15.46:0/1430351163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c009d00 con 0x7f395c0698a0 2024-03-18T20:40:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f395a7fc700 1 -- 172.21.15.46:0/1430351163 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f394c01d470 con 0x7f395c0698a0 2024-03-18T20:40:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f395a7fc700 1 -- 172.21.15.46:0/1430351163 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c0052a0 con 0x7f395c0698a0 2024-03-18T20:40:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.224+0000 7f39633c9700 1 -- 172.21.15.46:0/1430351163 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395c1aa4b0 con 0x7f395c0698a0 2024-03-18T20:40:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.227+0000 7f395a7fc700 1 -- 172.21.15.46:0/1430351163 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f394c005400 con 0x7f395c0698a0 2024-03-18T20:40:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.228+0000 7f395a7fc700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f39440a1290 0x7f39440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.228+0000 7f3960964700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f39440a1290 0x7f39440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.228+0000 7f395a7fc700 1 -- 172.21.15.46:0/1430351163 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f394c0cdd60 con 0x7f395c0698a0 2024-03-18T20:40:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.229+0000 7f3960964700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f39440a1290 0x7f39440a3750 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f395c19ebf0 tx=0x7f3950009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.229+0000 7f396f55eb80 1 Processor -- start 2024-03-18T20:40:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.229+0000 7f396f55eb80 1 -- start start 2024-03-18T20:40:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.229+0000 7f396f55eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8105a60 0x5598c8105e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.229+0000 7f396f55eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8106380 0x5598c8110ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.229+0000 7f396f55eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8111220 0x5598c8113610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.229+0000 7f396f55eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c805f990 con 0x5598c8111220 2024-03-18T20:40:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.229+0000 7f396f55eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c805d350 con 0x5598c8105a60 2024-03-18T20:40:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.229+0000 7f396f55eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c8058c40 con 0x5598c8106380 2024-03-18T20:40:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f3961165700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8105a60 0x5598c8105e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f3960964700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8106380 0x5598c8110ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f3960964700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8106380 0x5598c8110ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:48182/0 (socket says 172.21.15.46:48182) 2024-03-18T20:40:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f3960964700 1 -- 172.21.15.46:0/68898628 learned_addr learned my addr 172.21.15.46:0/68898628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f3961966700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8111220 0x5598c8113610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f3961165700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8105a60 0x5598c8105e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47290/0 (socket says 172.21.15.46:47290) 2024-03-18T20:40:56.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f3960964700 1 -- 172.21.15.46:0/68898628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8105a60 msgr2=0x5598c8105e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f3960964700 1 --2- 172.21.15.46:0/68898628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8105a60 0x5598c8105e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f3960964700 1 -- 172.21.15.46:0/68898628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8111220 msgr2=0x5598c8113610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f3960964700 1 --2- 172.21.15.46:0/68898628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8111220 0x5598c8113610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f3960964700 1 -- 172.21.15.46:0/68898628 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598c7eda3b0 con 0x5598c8106380 2024-03-18T20:40:56.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f3960964700 1 --2- 172.21.15.46:0/68898628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8106380 0x5598c8110ce0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3950011da0 tx=0x7f3950014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce1aea65d9bd5763 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:56.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f39437fe700 1 -- 172.21.15.46:0/68898628 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950004a30 con 0x5598c8106380 2024-03-18T20:40:56.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f39437fe700 1 -- 172.21.15.46:0/68898628 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39500054e0 con 0x5598c8106380 2024-03-18T20:40:56.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.230+0000 7f39437fe700 1 -- 172.21.15.46:0/68898628 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950003a20 con 0x5598c8106380 2024-03-18T20:40:56.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.231+0000 7f396f55eb80 1 -- 172.21.15.46:0/68898628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8106380 msgr2=0x5598c8110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.231+0000 7f396f55eb80 1 --2- 172.21.15.46:0/68898628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8106380 0x5598c8110ce0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3950011da0 tx=0x7f3950014000 comp rx=0 tx=0).stop 2024-03-18T20:40:56.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.231+0000 7f396f55eb80 1 -- 172.21.15.46:0/68898628 shutdown_connections 2024-03-18T20:40:56.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.231+0000 7f396f55eb80 1 --2- 172.21.15.46:0/68898628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8111220 0x5598c8113610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.231+0000 7f396f55eb80 1 --2- 172.21.15.46:0/68898628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8105a60 0x5598c8105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.231+0000 7f396f55eb80 1 --2- 172.21.15.46:0/68898628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8106380 0x5598c8110ce0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.231+0000 7f396f55eb80 1 -- 172.21.15.46:0/68898628 >> 172.21.15.46:0/68898628 conn(0x5598c807c220 msgr2=0x5598c8102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:56.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.231+0000 7f396f55eb80 1 -- 172.21.15.46:0/68898628 shutdown_connections 2024-03-18T20:40:56.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.231+0000 7f396f55eb80 1 -- 172.21.15.46:0/68898628 wait complete. 2024-03-18T20:40:56.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.232+0000 7f396f55eb80 1 -- start start 2024-03-18T20:40:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.232+0000 7f396f55eb80 1 Processor -- start 2024-03-18T20:40:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.232+0000 7f396f55eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c8110ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f396f55eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 0x5598c810dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f396f55eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8111220 0x5598c810e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f396f55eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c7ff4d80 con 0x5598c8106380 2024-03-18T20:40:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f396f55eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c7ff4a20 con 0x5598c8111220 2024-03-18T20:40:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f396f55eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c7edc6a0 con 0x5598c8105a60 2024-03-18T20:40:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f396f55eb80 10 client.0 _ll_register_callbacks cb 0x5598c7e603a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3960964700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 0x5598c810dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3961165700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c8110ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3960964700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 0x5598c810dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39182/0 (socket says 172.21.15.46:39182) 2024-03-18T20:40:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3960964700 1 -- 172.21.15.46:0/543593437 learned_addr learned my addr 172.21.15.46:0/543593437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3961966700 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8111220 0x5598c810e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3961165700 1 -- 172.21.15.46:0/543593437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8111220 msgr2=0x5598c810e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3961165700 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8111220 0x5598c810e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3961165700 1 -- 172.21.15.46:0/543593437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 msgr2=0x5598c810dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3961165700 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 0x5598c810dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3961165700 1 -- 172.21.15.46:0/543593437 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3950017070 con 0x5598c8105a60 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3961966700 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8111220 0x5598c810e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3960964700 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 0x5598c810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3961165700 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c8110ca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f394c0cb4c0 tx=0x7f394c09a870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=420d57f16bdd0903 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.233+0000 7f3940ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f3940ff9700 1 -- 172.21.15.46:0/543593437 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c09af10 con 0x5598c8105a60 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f3940ff9700 1 -- 172.21.15.46:0/543593437 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f394c09ed40 con 0x5598c8105a60 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f396f55eb80 1 -- 172.21.15.46:0/543593437 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598c80f9ad0 con 0x5598c8105a60 2024-03-18T20:40:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f3940ff9700 1 -- 172.21.15.46:0/543593437 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f394c0a3300 con 0x5598c8105a60 2024-03-18T20:40:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f396f55eb80 10 client.26998 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f396f55eb80 1 -- 172.21.15.46:0/543593437 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5598c7e86b60 con 0x5598c8105a60 2024-03-18T20:40:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f3940ff9700 1 -- 172.21.15.46:0/543593437 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f394c0b9030 con 0x5598c8105a60 2024-03-18T20:40:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f3940ff9700 1 -- 172.21.15.46:0/543593437 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f394c09b200 con 0x5598c8105a60 2024-03-18T20:40:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f396f55eb80 10 client.26998 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f396f55eb80 10 client.26998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f396f55eb80 20 client.26998 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f396f55eb80 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5598c8120900 0x5598c8122cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f396f55eb80 4 client.26998 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:40:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f396f55eb80 1 -- 172.21.15.46:0/543593437 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598c8051700 con 0x5598c8120900 2024-03-18T20:40:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.234+0000 7f3960964700 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5598c8120900 0x5598c8122cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.235+0000 7f3940ff9700 10 client.26998 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.235+0000 7f3960964700 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5598c8120900 0x5598c8122cf0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90492e6627e7fb07 server_cookie=7b45a75c46c87cee in_seq=0 out_seq=0 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.235+0000 7f3940ff9700 1 -- 172.21.15.46:0/543593437 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5598c8051700 con 0x5598c8120900 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.235+0000 7f3940ff9700 10 client.26998 handle_command_reply: tid=0 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.235+0000 7f396f55eb80 1 client.26998 shutdown 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.235+0000 7f396f55eb80 1 -- 172.21.15.46:0/543593437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 msgr2=0x5598c8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.235+0000 7f396f55eb80 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c8110ca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f394c0cb4c0 tx=0x7f394c09a870 comp rx=0 tx=0).stop 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.236+0000 7f396f55eb80 1 -- 172.21.15.46:0/543593437 shutdown_connections 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.236+0000 7f396f55eb80 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5598c8120900 0x5598c8122cf0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.236+0000 7f396f55eb80 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8111220 0x5598c810e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.236+0000 7f396f55eb80 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c8110ca0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.236+0000 7f396f55eb80 1 --2- 172.21.15.46:0/543593437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 0x5598c810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.236+0000 7f3940ff9700 0 client.26998 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.236+0000 7f396f55eb80 1 -- 172.21.15.46:0/543593437 >> 172.21.15.46:0/543593437 conn(0x5598c807c220 msgr2=0x5598c8101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.236+0000 7f396f55eb80 1 -- 172.21.15.46:0/543593437 shutdown_connections 2024-03-18T20:40:56.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.236+0000 7f396f55eb80 1 -- 172.21.15.46:0/543593437 wait complete. 2024-03-18T20:40:56.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.236+0000 7f396f55eb80 20 client.26998 trim_cache size 0 max 16384 2024-03-18T20:40:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.254+0000 7f396f55eb80 1 Processor -- start 2024-03-18T20:40:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.254+0000 7f396f55eb80 1 -- start start 2024-03-18T20:40:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.254+0000 7f396f55eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c811ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.254+0000 7f396f55eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8106380 0x5598c8117e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.254+0000 7f396f55eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8111220 0x5598c81107d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.254+0000 7f396f55eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c810ed10 con 0x5598c8111220 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.254+0000 7f396f55eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c8103470 con 0x5598c8106380 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.254+0000 7f396f55eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c810f590 con 0x5598c8105a60 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3961165700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c811ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3961966700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8111220 0x5598c81107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3961165700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c811ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:48194/0 (socket says 172.21.15.46:48194) 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3961165700 1 -- 172.21.15.46:0/2806852609 learned_addr learned my addr 172.21.15.46:0/2806852609 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3960964700 1 --2- 172.21.15.46:0/2806852609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8106380 0x5598c8117e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3961165700 1 -- 172.21.15.46:0/2806852609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8106380 msgr2=0x5598c8117e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3961165700 1 --2- 172.21.15.46:0/2806852609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8106380 0x5598c8117e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3961165700 1 -- 172.21.15.46:0/2806852609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8111220 msgr2=0x5598c81107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3961165700 1 --2- 172.21.15.46:0/2806852609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8111220 0x5598c81107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3961165700 1 -- 172.21.15.46:0/2806852609 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598c801eea0 con 0x5598c8105a60 2024-03-18T20:40:56.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3961966700 1 --2- 172.21.15.46:0/2806852609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8111220 0x5598c81107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:56.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3961165700 1 --2- 172.21.15.46:0/2806852609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c811ce30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f394c097410 tx=0x7f394c097380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=697a1897f7e29860 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:56.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3943fff700 1 -- 172.21.15.46:0/2806852609 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c09ed40 con 0x5598c8105a60 2024-03-18T20:40:56.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3943fff700 1 -- 172.21.15.46:0/2806852609 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f394c0b9040 con 0x5598c8105a60 2024-03-18T20:40:56.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.255+0000 7f3943fff700 1 -- 172.21.15.46:0/2806852609 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c0977e0 con 0x5598c8105a60 2024-03-18T20:40:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.256+0000 7f396f55eb80 1 -- 172.21.15.46:0/2806852609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 msgr2=0x5598c811ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.256+0000 7f396f55eb80 1 --2- 172.21.15.46:0/2806852609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c811ce30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f394c097410 tx=0x7f394c097380 comp rx=0 tx=0).stop 2024-03-18T20:40:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.256+0000 7f396f55eb80 1 -- 172.21.15.46:0/2806852609 shutdown_connections 2024-03-18T20:40:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.256+0000 7f396f55eb80 1 --2- 172.21.15.46:0/2806852609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8111220 0x5598c81107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.256+0000 7f396f55eb80 1 --2- 172.21.15.46:0/2806852609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c811ce30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.256+0000 7f396f55eb80 1 --2- 172.21.15.46:0/2806852609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8106380 0x5598c8117e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.256+0000 7f396f55eb80 1 -- 172.21.15.46:0/2806852609 >> 172.21.15.46:0/2806852609 conn(0x5598c807c220 msgr2=0x5598c8100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.256+0000 7f396f55eb80 1 -- 172.21.15.46:0/2806852609 shutdown_connections 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.256+0000 7f396f55eb80 1 -- 172.21.15.46:0/2806852609 wait complete. 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f396f55eb80 1 -- start start 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f396f55eb80 1 Processor -- start 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f396f55eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c816d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f396f55eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 0x5598c816de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f396f55eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8111220 0x5598c8115e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f396f55eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c8116370 con 0x5598c8106380 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f396f55eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c81164e0 con 0x5598c8111220 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f396f55eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c81166a0 con 0x5598c8105a60 2024-03-18T20:40:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f396f55eb80 10 client.0 _ll_register_callbacks cb 0x7f394c09adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f3960964700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 0x5598c816de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f3961165700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c816d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f3961165700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c816d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:48202/0 (socket says 172.21.15.46:48202) 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f3960964700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 0x5598c816de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39204/0 (socket says 172.21.15.46:39204) 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f3961165700 1 -- 172.21.15.46:0/927115453 learned_addr learned my addr 172.21.15.46:0/927115453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.257+0000 7f3961966700 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8111220 0x5598c8115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f3961165700 1 -- 172.21.15.46:0/927115453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8111220 msgr2=0x5598c8115e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f3961165700 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8111220 0x5598c8115e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f3961165700 1 -- 172.21.15.46:0/927115453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 msgr2=0x5598c816de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f3961165700 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 0x5598c816de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f3961165700 1 -- 172.21.15.46:0/927115453 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3954011440 con 0x5598c8105a60 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f3961966700 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8111220 0x5598c8115e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f3961165700 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c816d8e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f394c09ab60 tx=0x7f394c09b020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d11cfc12490273 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:40:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f3940ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f3940ff9700 1 -- 172.21.15.46:0/927115453 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c09b370 con 0x5598c8105a60 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f3940ff9700 1 -- 172.21.15.46:0/927115453 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f394c09ed40 con 0x5598c8105a60 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f396f55eb80 1 -- 172.21.15.46:0/927115453 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39500054e0 con 0x5598c8105a60 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f3940ff9700 1 -- 172.21.15.46:0/927115453 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f394c09a870 con 0x5598c8105a60 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f396f55eb80 10 client.27004 fetch_fsmap learned FSMap version 39 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f396f55eb80 1 -- 172.21.15.46:0/927115453 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f3950017070 con 0x5598c8105a60 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.258+0000 7f3940ff9700 1 -- 172.21.15.46:0/927115453 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f394c083020 con 0x5598c8105a60 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.259+0000 7f3940ff9700 1 -- 172.21.15.46:0/927115453 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f394c096d00 con 0x5598c8105a60 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.259+0000 7f396f55eb80 10 client.27004 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.259+0000 7f396f55eb80 10 client.27004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.259+0000 7f396f55eb80 20 client.27004 populate_metadata read hostname 'smithi046' 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.259+0000 7f396f55eb80 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5598c8175230 0x5598c8177620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:40:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.259+0000 7f396f55eb80 4 client.27004 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.259+0000 7f396f55eb80 1 -- 172.21.15.46:0/927115453 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5598c7ea7cf0 con 0x5598c8175230 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.259+0000 7f3960964700 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5598c8175230 0x5598c8177620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f3960964700 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5598c8175230 0x5598c8177620 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d44f7de00dd102 server_cookie=6adfc98a68d3d768 in_seq=0 out_seq=0 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f3940ff9700 10 client.27004 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f3940ff9700 1 -- 172.21.15.46:0/927115453 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5598c7ea7cf0 con 0x5598c8175230 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f3940ff9700 10 client.27004 handle_command_reply: tid=0 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f396f55eb80 1 client.27004 shutdown 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f396f55eb80 1 -- 172.21.15.46:0/927115453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 msgr2=0x5598c816d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f396f55eb80 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c816d8e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f394c09ab60 tx=0x7f394c09b020 comp rx=0 tx=0).stop 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f396f55eb80 1 -- 172.21.15.46:0/927115453 shutdown_connections 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f396f55eb80 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5598c8175230 0x5598c8177620 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f396f55eb80 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598c8111220 0x5598c8115e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f3940ff9700 0 client.27004 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:40:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f396f55eb80 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598c8105a60 0x5598c816d8e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f396f55eb80 1 --2- 172.21.15.46:0/927115453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598c8106380 0x5598c816de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f396f55eb80 1 -- 172.21.15.46:0/927115453 >> 172.21.15.46:0/927115453 conn(0x5598c807c220 msgr2=0x5598c811ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f396f55eb80 1 -- 172.21.15.46:0/927115453 shutdown_connections 2024-03-18T20:40:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f396f55eb80 1 -- 172.21.15.46:0/927115453 wait complete. 2024-03-18T20:40:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.260+0000 7f396f55eb80 20 client.27004 trim_cache size 0 max 16384 2024-03-18T20:40:56.268 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:40:56.268 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:40:56.268 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:40:56.268 INFO:teuthology.orchestra.run.smithi046.stdout: "5ba3037f-ca91-401b-9bf7-d455a4425bc3": { 2024-03-18T20:40:56.268 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:40:56.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "5ba3037f-ca91-401b-9bf7-d455a4425bc3", 2024-03-18T20:40:56.269 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:40:56.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:40:56.269 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:40:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:40:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.261+0000 7f39633c9700 1 -- 172.21.15.46:0/1430351163 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f39440a1290 msgr2=0x7f39440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.261+0000 7f39633c9700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f39440a1290 0x7f39440a3750 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f395c19ebf0 tx=0x7f3950009260 comp rx=0 tx=0).stop 2024-03-18T20:40:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.261+0000 7f39633c9700 1 -- 172.21.15.46:0/1430351163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f395c0698a0 msgr2=0x7f395c1a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:40:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.261+0000 7f39633c9700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f395c0698a0 0x7f395c1a36a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f394c00b780 tx=0x7f394c007940 comp rx=0 tx=0).stop 2024-03-18T20:40:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.261+0000 7f39633c9700 1 -- 172.21.15.46:0/1430351163 shutdown_connections 2024-03-18T20:40:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.261+0000 7f39633c9700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f39440a1290 0x7f39440a3750 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.261+0000 7f39633c9700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f395c105c50 0x7f395c19d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.261+0000 7f39633c9700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f395c0698a0 0x7f395c1a36a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.261+0000 7f39633c9700 1 --2- 172.21.15.46:0/1430351163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f395c103750 0x7f395c1a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:40:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.261+0000 7f39633c9700 1 -- 172.21.15.46:0/1430351163 >> 172.21.15.46:0/1430351163 conn(0x7f395c076db0 msgr2=0x7f395c102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:40:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.261+0000 7f39633c9700 1 -- 172.21.15.46:0/1430351163 shutdown_connections 2024-03-18T20:40:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:40:56.261+0000 7f39633c9700 1 -- 172.21.15.46:0/1430351163 wait complete. 2024-03-18T20:40:56.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:55 smithi046 ceph-mon[43495]: pgmap v148: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 924 KiB/s wr, 102 op/s 2024-03-18T20:40:56.278 INFO:tasks.cephfs.filesystem:scrub status for tag:5ba3037f-ca91-401b-9bf7-d455a4425bc3 - {'path': '/', 'tag': '5ba3037f-ca91-401b-9bf7-d455a4425bc3', 'options': 'recursive,force'} 2024-03-18T20:40:56.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:55 smithi155 ceph-mon[48344]: pgmap v148: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 924 KiB/s wr, 102 op/s 2024-03-18T20:40:56.687 INFO:tasks.workunit.client.0.smithi046.stdout: 1 130063 2.30 MB/sec execute 111 sec latency 2358.878 ms 2024-03-18T20:40:57.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:56 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:57.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:56 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:56 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:57.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:56 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:57.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:56 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:57.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:56 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:56 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:56 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:56 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:57.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:56 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:56 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:57.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:56 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:56 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:57.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:56 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:57.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:56 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:57.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:56 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:56 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:56 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:56 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:57.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:56 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:56 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:57.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:56 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:56 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:57.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:56 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:57.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:56 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:57.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:56 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:56 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:56 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:56 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:57.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:56 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:57.687 INFO:tasks.workunit.client.0.smithi046.stdout: 1 131672 2.39 MB/sec execute 112 sec latency 14.846 ms 2024-03-18T20:40:58.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:57 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:40:58.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:57 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:58.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:57 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:58.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:57 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:58.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:57 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:58.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:57 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:40:58.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:57 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:58.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:57 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:58.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:40:57 smithi073 ceph-mon[47384]: pgmap v149: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 924 KiB/s wr, 102 op/s 2024-03-18T20:40:58.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:57 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:40:58.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:57 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:58.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:57 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:58.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:57 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:58.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:57 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:58.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:57 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:40:58.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:57 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:58.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:57 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:58.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:40:57 smithi155 ceph-mon[48344]: pgmap v149: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 924 KiB/s wr, 102 op/s 2024-03-18T20:40:58.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:57 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:40:58.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:57 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:58.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:57 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:58.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:57 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:58.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:57 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:58.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:57 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:40:58.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:57 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:58.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:57 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:40:58.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:40:57 smithi046 ceph-mon[43495]: pgmap v149: 97 pgs: 97 active+clean; 54 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 924 KiB/s wr, 102 op/s 2024-03-18T20:40:58.687 INFO:tasks.workunit.client.0.smithi046.stdout: 1 131943 2.37 MB/sec execute 113 sec latency 743.768 ms 2024-03-18T20:40:59.687 INFO:tasks.workunit.client.0.smithi046.stdout: 1 131943 2.35 MB/sec execute 114 sec latency 1743.879 ms 2024-03-18T20:41:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.430+0000 7f9ab1506700 1 -- 172.21.15.46:0/2973343913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac1018a0 msgr2=0x7f9aac1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.430+0000 7f9ab1506700 1 --2- 172.21.15.46:0/2973343913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac1018a0 0x7f9aac1058e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009990 tx=0x7f9aa0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.430+0000 7f9ab1506700 1 -- 172.21.15.46:0/2973343913 shutdown_connections 2024-03-18T20:41:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.430+0000 7f9ab1506700 1 --2- 172.21.15.46:0/2973343913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9aac106010 0x7f9aac10db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.430+0000 7f9ab1506700 1 --2- 172.21.15.46:0/2973343913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aac100ef0 0x7f9aac1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.430+0000 7f9ab1506700 1 --2- 172.21.15.46:0/2973343913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac1018a0 0x7f9aac1058e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.430+0000 7f9ab1506700 1 -- 172.21.15.46:0/2973343913 >> 172.21.15.46:0/2973343913 conn(0x7f9aac0fc770 msgr2=0x7f9aac0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.430+0000 7f9ab1506700 1 -- 172.21.15.46:0/2973343913 shutdown_connections 2024-03-18T20:41:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.430+0000 7f9ab1506700 1 -- 172.21.15.46:0/2973343913 wait complete. 2024-03-18T20:41:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.430+0000 7f9ab1506700 1 Processor -- start 2024-03-18T20:41:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.431+0000 7f9ab1506700 1 -- start start 2024-03-18T20:41:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.431+0000 7f9ab1506700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9aac100ef0 0x7f9aac1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.431+0000 7f9ab1506700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac106010 0x7f9aac197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.431+0000 7f9ab1506700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aac19ba60 0x7f9aac19bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.431+0000 7f9ab1506700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac19c4e0 con 0x7f9aac106010 2024-03-18T20:41:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.431+0000 7f9ab1506700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac19c650 con 0x7f9aac100ef0 2024-03-18T20:41:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.431+0000 7f9ab1506700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac19c810 con 0x7f9aac19ba60 2024-03-18T20:41:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.431+0000 7f9aaa7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac106010 0x7f9aac197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.431+0000 7f9aaa7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac106010 0x7f9aac197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39222/0 (socket says 172.21.15.46:39222) 2024-03-18T20:41:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.431+0000 7f9aaa7fc700 1 -- 172.21.15.46:0/2877122897 learned_addr learned my addr 172.21.15.46:0/2877122897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.431+0000 7f9aab7fe700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aac19ba60 0x7f9aac19bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.431+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9aac100ef0 0x7f9aac1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.432+0000 7f9aaa7fc700 1 -- 172.21.15.46:0/2877122897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aac19ba60 msgr2=0x7f9aac19bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.432+0000 7f9aaa7fc700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aac19ba60 0x7f9aac19bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.432+0000 7f9aaa7fc700 1 -- 172.21.15.46:0/2877122897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9aac100ef0 msgr2=0x7f9aac1971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.432+0000 7f9aaa7fc700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9aac100ef0 0x7f9aac1971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.432+0000 7f9aaa7fc700 1 -- 172.21.15.46:0/2877122897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa00095f0 con 0x7f9aac106010 2024-03-18T20:41:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.432+0000 7f9aaa7fc700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac106010 0x7f9aac197710 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00059d0 tx=0x7f9aa0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.432+0000 7f9a93fff700 1 -- 172.21.15.46:0/2877122897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa001d070 con 0x7f9aac106010 2024-03-18T20:41:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.432+0000 7f9ab1506700 1 -- 172.21.15.46:0/2877122897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aac1a3d10 con 0x7f9aac106010 2024-03-18T20:41:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.432+0000 7f9a93fff700 1 -- 172.21.15.46:0/2877122897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9aa0022470 con 0x7f9aac106010 2024-03-18T20:41:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.432+0000 7f9a93fff700 1 -- 172.21.15.46:0/2877122897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0017650 con 0x7f9aac106010 2024-03-18T20:41:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.432+0000 7f9ab1506700 1 -- 172.21.15.46:0/2877122897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aac1a4190 con 0x7f9aac106010 2024-03-18T20:41:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.435+0000 7f9a93fff700 1 -- 172.21.15.46:0/2877122897 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9aa00225e0 con 0x7f9aac106010 2024-03-18T20:41:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.435+0000 7f9a93fff700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9a940a1290 0x7f9a940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.435+0000 7f9a93fff700 1 -- 172.21.15.46:0/2877122897 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9aa00ce0d0 con 0x7f9aac106010 2024-03-18T20:41:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.435+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9a940a1290 0x7f9a940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.436+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9a940a1290 0x7f9a940a3750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009bf0 tx=0x7f9a9c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.436+0000 7f9abd69bb80 1 Processor -- start 2024-03-18T20:41:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.436+0000 7f9abd69bb80 1 -- start start 2024-03-18T20:41:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.436+0000 7f9abd69bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 0x55c299848950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.436+0000 7f9abd69bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c299848e90 0x55c299857c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.436+0000 7f9abd69bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2998581c0 0x55c29985a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.436+0000 7f9abd69bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2997a2930 con 0x55c299848570 2024-03-18T20:41:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.436+0000 7f9abd69bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2997a02f0 con 0x55c2998581c0 2024-03-18T20:41:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.436+0000 7f9abd69bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29979bbe0 con 0x55c299848e90 2024-03-18T20:41:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.436+0000 7f9aaaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 0x55c299848950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9aaaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 0x55c299848950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39234/0 (socket says 172.21.15.46:39234) 2024-03-18T20:41:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9aaaffd700 1 -- 172.21.15.46:0/3108688046 learned_addr learned my addr 172.21.15.46:0/3108688046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9aaa7fc700 1 --2- 172.21.15.46:0/3108688046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c299848e90 0x55c299857c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9aab7fe700 1 --2- 172.21.15.46:0/3108688046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2998581c0 0x55c29985a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9aaaffd700 1 -- 172.21.15.46:0/3108688046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c299848e90 msgr2=0x55c299857c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/3108688046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c299848e90 0x55c299857c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9aaaffd700 1 -- 172.21.15.46:0/3108688046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2998581c0 msgr2=0x55c29985a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/3108688046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2998581c0 0x55c29985a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9aaaffd700 1 -- 172.21.15.46:0/3108688046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c29961d350 con 0x55c299848570 2024-03-18T20:41:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9aab7fe700 1 --2- 172.21.15.46:0/3108688046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2998581c0 0x55c29985a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:41:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/3108688046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 0x55c299848950 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c011e40 tx=0x7f9a9c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1c89ec58d6b516 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9a917fa700 1 -- 172.21.15.46:0/3108688046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c012dc0 con 0x55c299848570 2024-03-18T20:41:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9a917fa700 1 -- 172.21.15.46:0/3108688046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a9c016070 con 0x55c299848570 2024-03-18T20:41:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.437+0000 7f9a917fa700 1 -- 172.21.15.46:0/3108688046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c003d70 con 0x55c299848570 2024-03-18T20:41:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.438+0000 7f9abd69bb80 1 -- 172.21.15.46:0/3108688046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 msgr2=0x55c299848950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.438+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/3108688046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 0x55c299848950 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c011e40 tx=0x7f9a9c014000 comp rx=0 tx=0).stop 2024-03-18T20:41:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.438+0000 7f9abd69bb80 1 -- 172.21.15.46:0/3108688046 shutdown_connections 2024-03-18T20:41:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.438+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/3108688046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 0x55c299848950 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.438+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/3108688046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c2998581c0 0x55c29985a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.438+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/3108688046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c299848e90 0x55c299857c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.438+0000 7f9abd69bb80 1 -- 172.21.15.46:0/3108688046 >> 172.21.15.46:0/3108688046 conn(0x55c2997bf1c0 msgr2=0x55c299844a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.438+0000 7f9abd69bb80 1 -- 172.21.15.46:0/3108688046 shutdown_connections 2024-03-18T20:41:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.438+0000 7f9abd69bb80 1 -- 172.21.15.46:0/3108688046 wait complete. 2024-03-18T20:41:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.438+0000 7f9abd69bb80 1 -- start start 2024-03-18T20:41:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9abd69bb80 1 Processor -- start 2024-03-18T20:41:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9abd69bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848570 0x55c299857c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9abd69bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 0x55c299854f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9abd69bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c299855620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9abd69bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c299737d20 con 0x55c299848e90 2024-03-18T20:41:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9abd69bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2997379c0 con 0x55c299848570 2024-03-18T20:41:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9abd69bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29961f640 con 0x55c2998581c0 2024-03-18T20:41:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9abd69bb80 10 client.0 _ll_register_callbacks cb 0x55c2995a3340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9aaa7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 0x55c299854f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9aaa7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 0x55c299854f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39246/0 (socket says 172.21.15.46:39246) 2024-03-18T20:41:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9aaa7fc700 1 -- 172.21.15.46:0/2818445631 learned_addr learned my addr 172.21.15.46:0/2818445631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848570 0x55c299857c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9aab7fe700 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c299855620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9aaa7fc700 1 -- 172.21.15.46:0/2818445631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 msgr2=0x55c299855620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9aaa7fc700 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c299855620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.439+0000 7f9aaa7fc700 1 -- 172.21.15.46:0/2818445631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848570 msgr2=0x55c299857c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9aaa7fc700 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848570 0x55c299857c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9aaa7fc700 1 -- 172.21.15.46:0/2818445631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c011b20 con 0x55c299848e90 2024-03-18T20:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9aaa7fc700 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 0x55c299854f30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00cb890 tx=0x7f9aa009ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ae4d754c08ff86 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9a8eff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9a8eff5700 1 -- 172.21.15.46:0/2818445631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa009a970 con 0x55c299848e90 2024-03-18T20:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9a8eff5700 1 -- 172.21.15.46:0/2818445631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9aa00a3670 con 0x55c299848e90 2024-03-18T20:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9abd69bb80 1 -- 172.21.15.46:0/2818445631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c29983c6c0 con 0x55c299848e90 2024-03-18T20:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9a8eff5700 1 -- 172.21.15.46:0/2818445631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9aa00a31e0 con 0x55c299848e90 2024-03-18T20:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9abd69bb80 10 client.17301 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9abd69bb80 1 -- 172.21.15.46:0/2818445631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c2995c9b00 con 0x55c299848e90 2024-03-18T20:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9a8eff5700 1 -- 172.21.15.46:0/2818445631 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c29983c6c0 con 0x55c299848e90 2024-03-18T20:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9a8eff5700 1 -- 172.21.15.46:0/2818445631 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9aa00b3050 con 0x55c299848e90 2024-03-18T20:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9abd69bb80 10 client.17301 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9abd69bb80 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9abd69bb80 20 client.17301 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c299867990 0x55c299869db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9abd69bb80 4 client.17301 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.440+0000 7f9abd69bb80 1 -- 172.21.15.46:0/2818445631 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2997946a0 con 0x55c299867990 2024-03-18T20:41:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.441+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c299867990 0x55c299869db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.441+0000 7f9a8eff5700 10 client.17301 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.441+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c299867990 0x55c299869db0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3e0b0a2797ad5d server_cookie=f7a8c7ebf51b5e64 in_seq=0 out_seq=0 2024-03-18T20:41:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9a8eff5700 1 -- 172.21.15.46:0/2818445631 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c2997946a0 con 0x55c299867990 2024-03-18T20:41:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9a8eff5700 10 client.17301 handle_command_reply: tid=0 2024-03-18T20:41:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9abd69bb80 1 client.17301 shutdown 2024-03-18T20:41:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9abd69bb80 1 -- 172.21.15.46:0/2818445631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 msgr2=0x55c299854f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 0x55c299854f30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00cb890 tx=0x7f9aa009ba80 comp rx=0 tx=0).stop 2024-03-18T20:41:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9abd69bb80 1 -- 172.21.15.46:0/2818445631 shutdown_connections 2024-03-18T20:41:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c299867990 0x55c299869db0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c299855620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848570 0x55c299857c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/2818445631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 0x55c299854f30 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9a8eff5700 0 client.17301 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9abd69bb80 1 -- 172.21.15.46:0/2818445631 >> 172.21.15.46:0/2818445631 conn(0x55c2997bf1c0 msgr2=0x55c299844650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9abd69bb80 1 -- 172.21.15.46:0/2818445631 shutdown_connections 2024-03-18T20:41:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9abd69bb80 1 -- 172.21.15.46:0/2818445631 wait complete. 2024-03-18T20:41:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.442+0000 7f9abd69bb80 20 client.17301 trim_cache size 0 max 16384 2024-03-18T20:41:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.458+0000 7f9abd69bb80 1 Processor -- start 2024-03-18T20:41:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.458+0000 7f9abd69bb80 1 -- start start 2024-03-18T20:41:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.458+0000 7f9abd69bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 0x55c2998555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.458+0000 7f9abd69bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848e90 0x55c299863e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.458+0000 7f9abd69bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c29985ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.458+0000 7f9abd69bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29984b620 con 0x55c299848570 2024-03-18T20:41:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.458+0000 7f9abd69bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c299841200 con 0x55c299848e90 2024-03-18T20:41:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.458+0000 7f9abd69bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2998457e0 con 0x55c2998581c0 2024-03-18T20:41:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.458+0000 7f9aaaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 0x55c2998555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.458+0000 7f9aaaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 0x55c2998555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39252/0 (socket says 172.21.15.46:39252) 2024-03-18T20:41:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.458+0000 7f9aaaffd700 1 -- 172.21.15.46:0/1218282082 learned_addr learned my addr 172.21.15.46:0/1218282082 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.458+0000 7f9aab7fe700 1 --2- 172.21.15.46:0/1218282082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c29985ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.459+0000 7f9aaa7fc700 1 --2- 172.21.15.46:0/1218282082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848e90 0x55c299863e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.459+0000 7f9aaaffd700 1 -- 172.21.15.46:0/1218282082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 msgr2=0x55c29985ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.459+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/1218282082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c29985ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.459+0000 7f9aaaffd700 1 -- 172.21.15.46:0/1218282082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848e90 msgr2=0x55c299863e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.459+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/1218282082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848e90 0x55c299863e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.459+0000 7f9aaaffd700 1 -- 172.21.15.46:0/1218282082 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c299761e40 con 0x55c299848570 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.459+0000 7f9aab7fe700 1 --2- 172.21.15.46:0/1218282082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c29985ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.459+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/1218282082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 0x55c2998555c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c011e10 tx=0x7f9a9c012ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59905fb244b56391 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.459+0000 7f9a91ffb700 1 -- 172.21.15.46:0/1218282082 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c038070 con 0x55c299848570 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.459+0000 7f9a91ffb700 1 -- 172.21.15.46:0/1218282082 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a9c02e040 con 0x55c299848570 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.459+0000 7f9a91ffb700 1 -- 172.21.15.46:0/1218282082 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c02d040 con 0x55c299848570 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.460+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1218282082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 msgr2=0x55c2998555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.460+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/1218282082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 0x55c2998555c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c011e10 tx=0x7f9a9c012ac0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.460+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1218282082 shutdown_connections 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.460+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/1218282082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848570 0x55c2998555c0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.460+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/1218282082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c29985ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.460+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/1218282082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848e90 0x55c299863e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.460+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1218282082 >> 172.21.15.46:0/1218282082 conn(0x55c2997bf1c0 msgr2=0x55c2998547c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.460+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1218282082 shutdown_connections 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.460+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1218282082 wait complete. 2024-03-18T20:41:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.460+0000 7f9abd69bb80 1 -- start start 2024-03-18T20:41:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.460+0000 7f9abd69bb80 1 Processor -- start 2024-03-18T20:41:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9abd69bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848570 0x55c29985eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9abd69bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 0x55c29985cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9abd69bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c29985d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9abd69bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29985da00 con 0x55c299848e90 2024-03-18T20:41:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9abd69bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29985db40 con 0x55c299848570 2024-03-18T20:41:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9abd69bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c29985dd00 con 0x55c2998581c0 2024-03-18T20:41:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9abd69bb80 10 client.0 _ll_register_callbacks cb 0x55c2995a3340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9aaa7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 0x55c29985cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9aaa7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 0x55c29985cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39256/0 (socket says 172.21.15.46:39256) 2024-03-18T20:41:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9aaa7fc700 1 -- 172.21.15.46:0/1632014693 learned_addr learned my addr 172.21.15.46:0/1632014693 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848570 0x55c29985eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9aab7fe700 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c29985d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9aaa7fc700 1 -- 172.21.15.46:0/1632014693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 msgr2=0x55c29985d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9aaa7fc700 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c29985d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9aaa7fc700 1 -- 172.21.15.46:0/1632014693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848570 msgr2=0x55c29985eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9aaa7fc700 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848570 0x55c29985eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9aaa7fc700 1 -- 172.21.15.46:0/1632014693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa4009a00 con 0x55c299848e90 2024-03-18T20:41:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9aab7fe700 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c29985d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:41:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9aaa7fc700 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 0x55c29985cf40 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0097cf0 tx=0x7f9aa0022c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6776d7f2b8dd35f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9a8eff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.461+0000 7f9a8eff5700 1 -- 172.21.15.46:0/1632014693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0062020 con 0x55c299848e90 2024-03-18T20:41:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1632014693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a9c016070 con 0x55c299848e90 2024-03-18T20:41:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9a8eff5700 1 -- 172.21.15.46:0/1632014693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9aa00a3670 con 0x55c299848e90 2024-03-18T20:41:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9a8eff5700 1 -- 172.21.15.46:0/1632014693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9aa00b1ef0 con 0x55c299848e90 2024-03-18T20:41:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9abd69bb80 10 client.17307 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1632014693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c29961d350 con 0x55c299848e90 2024-03-18T20:41:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9a8eff5700 1 -- 172.21.15.46:0/1632014693 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9aa0080020 con 0x55c299848e90 2024-03-18T20:41:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9a8eff5700 1 -- 172.21.15.46:0/1632014693 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9aa00b3040 con 0x55c299848e90 2024-03-18T20:41:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9abd69bb80 10 client.17307 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9abd69bb80 10 client.17307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9abd69bb80 20 client.17307 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c2998bc840 0x55c2998bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9abd69bb80 4 client.17307 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:41:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1632014693 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2995eac90 con 0x55c2998bc840 2024-03-18T20:41:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.462+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c2998bc840 0x55c2998bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.463+0000 7f9aaaffd700 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c2998bc840 0x55c2998bec30 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba21dac820c0a393 server_cookie=59ce8704a4cbf92a in_seq=0 out_seq=0 2024-03-18T20:41:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.463+0000 7f9a8eff5700 10 client.17307 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:00.476 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:00 smithi073 ceph-mon[47384]: pgmap v150: 97 pgs: 97 active+clean; 56 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-03-18T20:41:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.498+0000 7f9a8eff5700 1 -- 172.21.15.46:0/1632014693 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2995eac90 con 0x55c2998bc840 2024-03-18T20:41:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.498+0000 7f9a8eff5700 10 client.17307 handle_command_reply: tid=0 2024-03-18T20:41:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.498+0000 7f9abd69bb80 1 client.17307 shutdown 2024-03-18T20:41:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.498+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1632014693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 msgr2=0x55c29985cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.498+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 0x55c29985cf40 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0097cf0 tx=0x7f9aa0022c90 comp rx=0 tx=0).stop 2024-03-18T20:41:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.498+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1632014693 shutdown_connections 2024-03-18T20:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.498+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c2998bc840 0x55c2998bec30 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.498+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2998581c0 0x55c29985d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.498+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c299848570 0x55c29985eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.498+0000 7f9abd69bb80 1 --2- 172.21.15.46:0/1632014693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c299848e90 0x55c29985cf40 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.498+0000 7f9a8eff5700 0 client.17307 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.498+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1632014693 >> 172.21.15.46:0/1632014693 conn(0x55c2997bf1c0 msgr2=0x55c2998635a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1632014693 shutdown_connections 2024-03-18T20:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9abd69bb80 1 -- 172.21.15.46:0/1632014693 wait complete. 2024-03-18T20:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9abd69bb80 20 client.17307 trim_cache size 0 max 16384 2024-03-18T20:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:41:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:41:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:41:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9ab1506700 1 -- 172.21.15.46:0/2877122897 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9a940a1290 msgr2=0x7f9a940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9ab1506700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9a940a1290 0x7f9a940a3750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009bf0 tx=0x7f9a9c009260 comp rx=0 tx=0).stop 2024-03-18T20:41:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9ab1506700 1 -- 172.21.15.46:0/2877122897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac106010 msgr2=0x7f9aac197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9ab1506700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac106010 0x7f9aac197710 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00059d0 tx=0x7f9aa0004730 comp rx=0 tx=0).stop 2024-03-18T20:41:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9ab1506700 1 -- 172.21.15.46:0/2877122897 shutdown_connections 2024-03-18T20:41:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9ab1506700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9a940a1290 0x7f9a940a3750 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9ab1506700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9aac19ba60 0x7f9aac19bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9ab1506700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9aac100ef0 0x7f9aac1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9ab1506700 1 --2- 172.21.15.46:0/2877122897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9aac106010 0x7f9aac197710 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9ab1506700 1 -- 172.21.15.46:0/2877122897 >> 172.21.15.46:0/2877122897 conn(0x7f9aac0fc770 msgr2=0x7f9aac0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9ab1506700 1 -- 172.21.15.46:0/2877122897 shutdown_connections 2024-03-18T20:41:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:00.499+0000 7f9ab1506700 1 -- 172.21.15.46:0/2877122897 wait complete. 2024-03-18T20:41:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:00 smithi155 ceph-mon[48344]: pgmap v150: 97 pgs: 97 active+clean; 56 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-03-18T20:41:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:00 smithi046 ceph-mon[43495]: pgmap v150: 97 pgs: 97 active+clean; 56 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-03-18T20:41:00.687 INFO:tasks.workunit.client.0.smithi046.stdout: 1 131943 2.33 MB/sec execute 115 sec latency 2743.968 ms 2024-03-18T20:41:01.687 INFO:tasks.workunit.client.0.smithi046.stdout: 1 131999 2.31 MB/sec execute 116 sec latency 3043.028 ms 2024-03-18T20:41:02.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:02 smithi073 ceph-mon[47384]: pgmap v151: 97 pgs: 97 active+clean; 56 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-03-18T20:41:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:02 smithi155 ceph-mon[48344]: pgmap v151: 97 pgs: 97 active+clean; 56 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-03-18T20:41:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:02 smithi046 ceph-mon[43495]: pgmap v151: 97 pgs: 97 active+clean; 56 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-03-18T20:41:02.687 INFO:tasks.workunit.client.0.smithi046.stdout: 1 132067 2.29 MB/sec execute 117 sec latency 1011.920 ms 2024-03-18T20:41:03.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:03 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:41:03.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:03 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:41:03.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:03 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:41:03.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:03 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:41:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:03 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:41:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:03 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:41:03.688 INFO:tasks.workunit.client.0.smithi046.stdout: 1 132067 2.27 MB/sec execute 118 sec latency 1606.268 ms 2024-03-18T20:41:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:04 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:41:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:04 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:41:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:04 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:41:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:04 smithi073 ceph-mon[47384]: pgmap v152: 97 pgs: 97 active+clean; 56 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-03-18T20:41:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:04 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:41:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:04 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:41:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:04 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:41:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:04 smithi155 ceph-mon[48344]: pgmap v152: 97 pgs: 97 active+clean; 56 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-03-18T20:41:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:04 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:41:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:04 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:41:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:04 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:41:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:04 smithi046 ceph-mon[43495]: pgmap v152: 97 pgs: 97 active+clean; 56 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-03-18T20:41:04.688 INFO:tasks.workunit.client.0.smithi046.stdout: 1 132067 2.25 MB/sec execute 119 sec latency 2606.318 ms 2024-03-18T20:41:05.688 INFO:tasks.workunit.client.0.smithi046.stdout: 1 132067 2.23 MB/sec execute 120 sec latency 3606.427 ms 2024-03-18T20:41:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:05 smithi073 ceph-mon[47384]: pgmap v153: 97 pgs: 97 active+clean; 56 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 665 KiB/s wr, 116 op/s 2024-03-18T20:41:06.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:05 smithi155 ceph-mon[48344]: pgmap v153: 97 pgs: 97 active+clean; 56 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 665 KiB/s wr, 116 op/s 2024-03-18T20:41:06.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:05 smithi046 ceph-mon[43495]: pgmap v153: 97 pgs: 97 active+clean; 56 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 665 KiB/s wr, 116 op/s 2024-03-18T20:41:06.688 INFO:tasks.workunit.client.0.smithi046.stdout: 1 132067 2.21 MB/sec execute 121 sec latency 4606.529 ms 2024-03-18T20:41:07.688 INFO:tasks.workunit.client.0.smithi046.stdout: 1 133786 2.26 MB/sec execute 122 sec latency 4957.035 ms 2024-03-18T20:41:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:08 smithi073 ceph-mon[47384]: pgmap v154: 97 pgs: 97 active+clean; 56 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 665 KiB/s wr, 116 op/s 2024-03-18T20:41:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:08 smithi155 ceph-mon[48344]: pgmap v154: 97 pgs: 97 active+clean; 56 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 665 KiB/s wr, 116 op/s 2024-03-18T20:41:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:08 smithi046 ceph-mon[43495]: pgmap v154: 97 pgs: 97 active+clean; 56 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 665 KiB/s wr, 116 op/s 2024-03-18T20:41:08.688 INFO:tasks.workunit.client.0.smithi046.stdout: 1 135377 2.33 MB/sec execute 123 sec latency 11.573 ms 2024-03-18T20:41:09.688 INFO:tasks.workunit.client.0.smithi046.stdout: 1 137748 2.41 MB/sec execute 124 sec latency 11.388 ms 2024-03-18T20:41:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:10 smithi073 ceph-mon[47384]: pgmap v155: 97 pgs: 97 active+clean; 59 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-03-18T20:41:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:10 smithi155 ceph-mon[48344]: pgmap v155: 97 pgs: 97 active+clean; 59 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-03-18T20:41:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:10 smithi046 ceph-mon[43495]: pgmap v155: 97 pgs: 97 active+clean; 59 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-03-18T20:41:10.688 INFO:tasks.workunit.client.0.smithi046.stdout: 1 139090 2.45 MB/sec execute 125 sec latency 33.669 ms 2024-03-18T20:41:11.688 INFO:tasks.workunit.client.0.smithi046.stdout: 1 139090 2.43 MB/sec execute 126 sec latency 1033.772 ms 2024-03-18T20:41:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:12 smithi073 ceph-mon[47384]: pgmap v156: 97 pgs: 97 active+clean; 59 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 857 KiB/s wr, 91 op/s 2024-03-18T20:41:12.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:12 smithi155 ceph-mon[48344]: pgmap v156: 97 pgs: 97 active+clean; 59 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 857 KiB/s wr, 91 op/s 2024-03-18T20:41:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:12 smithi046 ceph-mon[43495]: pgmap v156: 97 pgs: 97 active+clean; 59 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 857 KiB/s wr, 91 op/s 2024-03-18T20:41:12.688 INFO:tasks.workunit.client.0.smithi046.stdout: 1 140808 2.46 MB/sec execute 127 sec latency 1387.219 ms 2024-03-18T20:41:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.632+0000 7fc80dd30700 1 -- 172.21.15.46:0/2120116218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808105c60 msgr2=0x7fc808106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.632+0000 7fc80dd30700 1 --2- 172.21.15.46:0/2120116218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808105c60 0x7fc808106040 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009990 tx=0x7fc7f8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.632+0000 7fc80dd30700 1 -- 172.21.15.46:0/2120116218 shutdown_connections 2024-03-18T20:41:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.632+0000 7fc80dd30700 1 --2- 172.21.15.46:0/2120116218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808105c60 0x7fc808106040 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.632+0000 7fc80dd30700 1 --2- 172.21.15.46:0/2120116218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc808100570 0x7fc80810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.633+0000 7fc80dd30700 1 --2- 172.21.15.46:0/2120116218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8080ffbb0 0x7fc808100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.633+0000 7fc80dd30700 1 -- 172.21.15.46:0/2120116218 >> 172.21.15.46:0/2120116218 conn(0x7fc808076d50 msgr2=0x7fc808077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.633+0000 7fc80dd30700 1 -- 172.21.15.46:0/2120116218 shutdown_connections 2024-03-18T20:41:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.633+0000 7fc80dd30700 1 -- 172.21.15.46:0/2120116218 wait complete. 2024-03-18T20:41:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.633+0000 7fc80dd30700 1 Processor -- start 2024-03-18T20:41:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.633+0000 7fc80dd30700 1 -- start start 2024-03-18T20:41:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc80dd30700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8080ffbb0 0x7fc8081970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc80dd30700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc808100570 0x7fc808197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc80dd30700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc808105c60 0x7fc80819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc80dd30700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80819c000 con 0x7fc8080ffbb0 2024-03-18T20:41:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc80dd30700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80819c170 con 0x7fc808105c60 2024-03-18T20:41:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc80dd30700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80819c330 con 0x7fc808100570 2024-03-18T20:41:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8080ffbb0 0x7fc8081970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8080ffbb0 0x7fc8081970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35140/0 (socket says 172.21.15.46:35140) 2024-03-18T20:41:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc8077fe700 1 -- 172.21.15.46:0/3181700319 learned_addr learned my addr 172.21.15.46:0/3181700319 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc807fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc808105c60 0x7fc80819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc806ffd700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc808100570 0x7fc808197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc8077fe700 1 -- 172.21.15.46:0/3181700319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc808100570 msgr2=0x7fc808197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc8077fe700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc808100570 0x7fc808197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc8077fe700 1 -- 172.21.15.46:0/3181700319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc808105c60 msgr2=0x7fc80819b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc8077fe700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc808105c60 0x7fc80819b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc8077fe700 1 -- 172.21.15.46:0/3181700319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f80095f0 con 0x7fc8080ffbb0 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc807fff700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc808105c60 0x7fc80819b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.634+0000 7fc8077fe700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8080ffbb0 0x7fc8081970e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8000c00 tx=0x7fc7f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.635+0000 7fc804ff9700 1 -- 172.21.15.46:0/3181700319 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f801d070 con 0x7fc8080ffbb0 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.635+0000 7fc80dd30700 1 -- 172.21.15.46:0/3181700319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80819c5b0 con 0x7fc8080ffbb0 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.635+0000 7fc804ff9700 1 -- 172.21.15.46:0/3181700319 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7f8022470 con 0x7fc8080ffbb0 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.635+0000 7fc804ff9700 1 -- 172.21.15.46:0/3181700319 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8017650 con 0x7fc8080ffbb0 2024-03-18T20:41:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.635+0000 7fc80dd30700 1 -- 172.21.15.46:0/3181700319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8081a3e50 con 0x7fc8080ffbb0 2024-03-18T20:41:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.638+0000 7fc804ff9700 1 -- 172.21.15.46:0/3181700319 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc7f80225e0 con 0x7fc8080ffbb0 2024-03-18T20:41:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.638+0000 7fc804ff9700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7f00a1290 0x7fc7f00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.639+0000 7fc806ffd700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7f00a1290 0x7fc7f00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.639+0000 7fc804ff9700 1 -- 172.21.15.46:0/3181700319 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc7f80cdef0 con 0x7fc8080ffbb0 2024-03-18T20:41:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.639+0000 7fc806ffd700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7f00a1290 0x7fc7f00a3750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc808198630 tx=0x7fc7fc00b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc819ec5b80 1 Processor -- start 2024-03-18T20:41:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc819ec5b80 1 -- start start 2024-03-18T20:41:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc819ec5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e2e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc819ec5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e39870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc819ec5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e3c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc819ec5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8d88930 con 0x55a9f8e39db0 2024-03-18T20:41:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc819ec5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8d862f0 con 0x55a9f8e2e570 2024-03-18T20:41:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc819ec5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8d81be0 con 0x55a9f8e2ee90 2024-03-18T20:41:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e2e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc807fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e3c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc806ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e39870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e2e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54770/0 (socket says 172.21.15.46:54770) 2024-03-18T20:41:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc807fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e3c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35144/0 (socket says 172.21.15.46:35144) 2024-03-18T20:41:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc8077fe700 1 -- 172.21.15.46:0/3842781864 learned_addr learned my addr 172.21.15.46:0/3842781864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc806ffd700 1 -- 172.21.15.46:0/3842781864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 msgr2=0x55a9f8e2e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc806ffd700 1 --2- 172.21.15.46:0/3842781864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e2e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc806ffd700 1 -- 172.21.15.46:0/3842781864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 msgr2=0x55a9f8e3c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc806ffd700 1 --2- 172.21.15.46:0/3842781864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e3c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.640+0000 7fc806ffd700 1 -- 172.21.15.46:0/3842781864 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f8c03350 con 0x55a9f8e2ee90 2024-03-18T20:41:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.641+0000 7fc806ffd700 1 --2- 172.21.15.46:0/3842781864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e39870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc005d90 tx=0x7fc7fc013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed2b1cb39e29ebcc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.641+0000 7fc7edffb700 1 -- 172.21.15.46:0/3842781864 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc011b00 con 0x55a9f8e2ee90 2024-03-18T20:41:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.641+0000 7fc7edffb700 1 -- 172.21.15.46:0/3842781864 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7fc015070 con 0x55a9f8e2ee90 2024-03-18T20:41:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.641+0000 7fc7edffb700 1 -- 172.21.15.46:0/3842781864 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc0054e0 con 0x55a9f8e2ee90 2024-03-18T20:41:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.642+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3842781864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2ee90 msgr2=0x55a9f8e39870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.642+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/3842781864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e39870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc005d90 tx=0x7fc7fc013000 comp rx=0 tx=0).stop 2024-03-18T20:41:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.642+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3842781864 shutdown_connections 2024-03-18T20:41:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.642+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/3842781864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e3c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.642+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/3842781864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e2e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.642+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/3842781864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e39870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.642+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3842781864 >> 172.21.15.46:0/3842781864 conn(0x55a9f8da51c0 msgr2=0x55a9f8e2aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.642+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3842781864 shutdown_connections 2024-03-18T20:41:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.642+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3842781864 wait complete. 2024-03-18T20:41:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.642+0000 7fc819ec5b80 1 -- start start 2024-03-18T20:41:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.642+0000 7fc819ec5b80 1 Processor -- start 2024-03-18T20:41:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc819ec5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2e570 0x55a9f8e397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc819ec5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e36b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc819ec5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e37280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc819ec5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8d1dd20 con 0x55a9f8e39db0 2024-03-18T20:41:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc819ec5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8d1d9c0 con 0x55a9f8e2ee90 2024-03-18T20:41:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc819ec5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8c05640 con 0x55a9f8e2e570 2024-03-18T20:41:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc819ec5b80 10 client.0 _ll_register_callbacks cb 0x55a9f8b89340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2e570 0x55a9f8e397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc807fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e37280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2e570 0x55a9f8e397e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43730/0 (socket says 172.21.15.46:43730) 2024-03-18T20:41:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc8077fe700 1 -- 172.21.15.46:0/3114987463 learned_addr learned my addr 172.21.15.46:0/3114987463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc806ffd700 1 --2- 172.21.15.46:0/3114987463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e36b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc8077fe700 1 -- 172.21.15.46:0/3114987463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2ee90 msgr2=0x55a9f8e36b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc8077fe700 1 --2- 172.21.15.46:0/3114987463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e36b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc8077fe700 1 -- 172.21.15.46:0/3114987463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 msgr2=0x55a9f8e37280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc8077fe700 1 --2- 172.21.15.46:0/3114987463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e37280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc8077fe700 1 -- 172.21.15.46:0/3114987463 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7fc010c20 con 0x55a9f8e2e570 2024-03-18T20:41:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc8077fe700 1 --2- 172.21.15.46:0/3114987463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2e570 0x55a9f8e397e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc7f809ada0 tx=0x7fc7f809aa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f9c8186398629d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc7e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:41:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.643+0000 7fc7e6ffd700 1 -- 172.21.15.46:0/3114987463 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f809ae90 con 0x55a9f8e2e570 2024-03-18T20:41:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc7e6ffd700 1 -- 172.21.15.46:0/3114987463 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc7f809b0a0 con 0x55a9f8e2e570 2024-03-18T20:41:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3114987463 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9f8e226c0 con 0x55a9f8e2e570 2024-03-18T20:41:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc7e6ffd700 1 -- 172.21.15.46:0/3114987463 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7f809eed0 con 0x55a9f8e2e570 2024-03-18T20:41:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc819ec5b80 10 client.27034 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3114987463 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a9f8bafb00 con 0x55a9f8e2e570 2024-03-18T20:41:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc7e6ffd700 1 -- 172.21.15.46:0/3114987463 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9f8e226c0 con 0x55a9f8e2e570 2024-03-18T20:41:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc7e6ffd700 1 -- 172.21.15.46:0/3114987463 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc7f809b300 con 0x55a9f8e2e570 2024-03-18T20:41:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc819ec5b80 10 client.27034 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc819ec5b80 10 client.27034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc819ec5b80 20 client.27034 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/3114987463 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a9f8e49650 0x55a9f8e4ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc819ec5b80 4 client.27034 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3114987463 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9f8d7a6a0 con 0x55a9f8e49650 2024-03-18T20:41:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.644+0000 7fc806ffd700 1 --2- 172.21.15.46:0/3114987463 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a9f8e49650 0x55a9f8e4ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.645+0000 7fc806ffd700 1 --2- 172.21.15.46:0/3114987463 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a9f8e49650 0x55a9f8e4ba40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c838f36907bbe server_cookie=cca5ecb9101c7ae2 in_seq=0 out_seq=0 2024-03-18T20:41:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.645+0000 7fc7e6ffd700 10 client.27034 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.645+0000 7fc7e6ffd700 1 -- 172.21.15.46:0/3114987463 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a9f8d7a6a0 con 0x55a9f8e49650 2024-03-18T20:41:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.645+0000 7fc7e6ffd700 10 client.27034 handle_command_reply: tid=0 2024-03-18T20:41:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc819ec5b80 1 client.27034 shutdown 2024-03-18T20:41:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3114987463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2e570 msgr2=0x55a9f8e397e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/3114987463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2e570 0x55a9f8e397e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc7f809ada0 tx=0x7fc7f809aa50 comp rx=0 tx=0).stop 2024-03-18T20:41:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3114987463 shutdown_connections 2024-03-18T20:41:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/3114987463 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a9f8e49650 0x55a9f8e4ba40 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/3114987463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e37280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc7e6ffd700 0 client.27034 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/3114987463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2e570 0x55a9f8e397e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/3114987463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e36b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3114987463 >> 172.21.15.46:0/3114987463 conn(0x55a9f8da51c0 msgr2=0x55a9f8e2a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3114987463 shutdown_connections 2024-03-18T20:41:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc819ec5b80 1 -- 172.21.15.46:0/3114987463 wait complete. 2024-03-18T20:41:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.646+0000 7fc819ec5b80 20 client.27034 trim_cache size 0 max 16384 2024-03-18T20:41:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc819ec5b80 1 Processor -- start 2024-03-18T20:41:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc819ec5b80 1 -- start start 2024-03-18T20:41:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc819ec5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e40a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc819ec5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc819ec5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e39db0 0x55a9f8e36080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc819ec5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8e36ec0 con 0x55a9f8e2ee90 2024-03-18T20:41:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc819ec5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8e24c10 con 0x55a9f8e2e570 2024-03-18T20:41:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc819ec5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8e40f80 con 0x55a9f8e39db0 2024-03-18T20:41:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc806ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc807fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e39db0 0x55a9f8e36080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc806ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e393f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35162/0 (socket says 172.21.15.46:35162) 2024-03-18T20:41:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc806ffd700 1 -- 172.21.15.46:0/648423244 learned_addr learned my addr 172.21.15.46:0/648423244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc8077fe700 1 --2- 172.21.15.46:0/648423244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e40a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc806ffd700 1 -- 172.21.15.46:0/648423244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e39db0 msgr2=0x55a9f8e36080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc806ffd700 1 --2- 172.21.15.46:0/648423244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e39db0 0x55a9f8e36080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc806ffd700 1 -- 172.21.15.46:0/648423244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 msgr2=0x55a9f8e40a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc806ffd700 1 --2- 172.21.15.46:0/648423244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e40a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.662+0000 7fc806ffd700 1 -- 172.21.15.46:0/648423244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f8d47e40 con 0x55a9f8e2ee90 2024-03-18T20:41:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.663+0000 7fc806ffd700 1 --2- 172.21.15.46:0/648423244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e393f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00b960 tx=0x7fc7fc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e684cfac9949fd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.663+0000 7fc7ee7fc700 1 -- 172.21.15.46:0/648423244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc02c070 con 0x55a9f8e2ee90 2024-03-18T20:41:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.663+0000 7fc7ee7fc700 1 -- 172.21.15.46:0/648423244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7fc02d040 con 0x55a9f8e2ee90 2024-03-18T20:41:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.663+0000 7fc7ee7fc700 1 -- 172.21.15.46:0/648423244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc005390 con 0x55a9f8e2ee90 2024-03-18T20:41:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.664+0000 7fc819ec5b80 1 -- 172.21.15.46:0/648423244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e2ee90 msgr2=0x55a9f8e393f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.664+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/648423244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e393f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00b960 tx=0x7fc7fc00bf10 comp rx=0 tx=0).stop 2024-03-18T20:41:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.664+0000 7fc819ec5b80 1 -- 172.21.15.46:0/648423244 shutdown_connections 2024-03-18T20:41:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.664+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/648423244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e39db0 0x55a9f8e36080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.664+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/648423244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e40a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.664+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/648423244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e393f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.664+0000 7fc819ec5b80 1 -- 172.21.15.46:0/648423244 >> 172.21.15.46:0/648423244 conn(0x55a9f8da51c0 msgr2=0x55a9f8e29120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.664+0000 7fc819ec5b80 1 -- 172.21.15.46:0/648423244 shutdown_connections 2024-03-18T20:41:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.664+0000 7fc819ec5b80 1 -- 172.21.15.46:0/648423244 wait complete. 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.664+0000 7fc819ec5b80 1 -- start start 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.664+0000 7fc819ec5b80 1 Processor -- start 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc819ec5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e40580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc819ec5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e3e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc819ec5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e3ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc819ec5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8e3f380 con 0x55a9f8e39db0 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc819ec5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8e3f4f0 con 0x55a9f8e2e570 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc819ec5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f8e3f6b0 con 0x55a9f8e2ee90 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc819ec5b80 10 client.0 _ll_register_callbacks cb 0x7fc7f809abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e40580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc807fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e3ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e40580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54796/0 (socket says 172.21.15.46:54796) 2024-03-18T20:41:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc806ffd700 1 --2- 172.21.15.46:0/1958787248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e3e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc8077fe700 1 -- 172.21.15.46:0/1958787248 learned_addr learned my addr 172.21.15.46:0/1958787248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc8077fe700 1 -- 172.21.15.46:0/1958787248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2ee90 msgr2=0x55a9f8e3e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc8077fe700 1 --2- 172.21.15.46:0/1958787248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e3e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc8077fe700 1 -- 172.21.15.46:0/1958787248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 msgr2=0x55a9f8e3ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc8077fe700 1 --2- 172.21.15.46:0/1958787248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e3ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc8077fe700 1 -- 172.21.15.46:0/1958787248 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9f8c03350 con 0x55a9f8e2e570 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc8077fe700 1 --2- 172.21.15.46:0/1958787248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e40580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc7f809b660 tx=0x7fc7f80973e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e7a526be78a7eba server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc7e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc7e6ffd700 1 -- 172.21.15.46:0/1958787248 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800fd70 con 0x55a9f8e2e570 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc7e6ffd700 1 -- 172.21.15.46:0/1958787248 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc7f809b0a0 con 0x55a9f8e2e570 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.665+0000 7fc819ec5b80 1 -- 172.21.15.46:0/1958787248 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7fc015070 con 0x55a9f8e2e570 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.666+0000 7fc7e6ffd700 1 -- 172.21.15.46:0/1958787248 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7f809ae70 con 0x55a9f8e2e570 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.666+0000 7fc819ec5b80 10 client.27020 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.666+0000 7fc819ec5b80 1 -- 172.21.15.46:0/1958787248 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a9f8c07c70 con 0x55a9f8e2e570 2024-03-18T20:41:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.666+0000 7fc7e6ffd700 1 -- 172.21.15.46:0/1958787248 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7fc015070 con 0x55a9f8e2e570 2024-03-18T20:41:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.666+0000 7fc7e6ffd700 1 -- 172.21.15.46:0/1958787248 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc7f80b9170 con 0x55a9f8e2e570 2024-03-18T20:41:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.666+0000 7fc819ec5b80 10 client.27020 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.666+0000 7fc819ec5b80 10 client.27020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.666+0000 7fc819ec5b80 20 client.27020 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.666+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/1958787248 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a9f8e9e410 0x55a9f8ea0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.666+0000 7fc819ec5b80 4 client.27020 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:41:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.666+0000 7fc819ec5b80 1 -- 172.21.15.46:0/1958787248 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a9f8bd0c90 con 0x55a9f8e9e410 2024-03-18T20:41:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.666+0000 7fc806ffd700 1 --2- 172.21.15.46:0/1958787248 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a9f8e9e410 0x55a9f8ea0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.667+0000 7fc806ffd700 1 --2- 172.21.15.46:0/1958787248 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a9f8e9e410 0x55a9f8ea0800 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=577390fb6a99fb80 server_cookie=8a29702ba4630856 in_seq=0 out_seq=0 2024-03-18T20:41:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.667+0000 7fc7e6ffd700 10 client.27020 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:13.688 INFO:tasks.workunit.client.0.smithi046.stdout: 1 142337 2.54 MB/sec execute 128 sec latency 18.448 ms 2024-03-18T20:41:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.697+0000 7fc7e6ffd700 1 -- 172.21.15.46:0/1958787248 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a9f8bd0c90 con 0x55a9f8e9e410 2024-03-18T20:41:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.697+0000 7fc7e6ffd700 10 client.27020 handle_command_reply: tid=0 2024-03-18T20:41:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.697+0000 7fc819ec5b80 1 client.27020 shutdown 2024-03-18T20:41:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.697+0000 7fc819ec5b80 1 -- 172.21.15.46:0/1958787248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 msgr2=0x55a9f8e40580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.697+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/1958787248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e40580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc7f809b660 tx=0x7fc7f80973e0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.697+0000 7fc819ec5b80 1 -- 172.21.15.46:0/1958787248 shutdown_connections 2024-03-18T20:41:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.697+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/1958787248 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a9f8e9e410 0x55a9f8ea0800 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.697+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/1958787248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a9f8e39db0 0x55a9f8e3ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.697+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/1958787248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a9f8e2e570 0x55a9f8e40580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.697+0000 7fc819ec5b80 1 --2- 172.21.15.46:0/1958787248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a9f8e2ee90 0x55a9f8e3e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.697+0000 7fc7e6ffd700 0 client.27020 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.697+0000 7fc819ec5b80 1 -- 172.21.15.46:0/1958787248 >> 172.21.15.46:0/1958787248 conn(0x55a9f8da51c0 msgr2=0x55a9f8e36080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc819ec5b80 1 -- 172.21.15.46:0/1958787248 shutdown_connections 2024-03-18T20:41:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc819ec5b80 1 -- 172.21.15.46:0/1958787248 wait complete. 2024-03-18T20:41:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc819ec5b80 20 client.27020 trim_cache size 0 max 16384 2024-03-18T20:41:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:41:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:41:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:41:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:41:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc80dd30700 1 -- 172.21.15.46:0/3181700319 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7f00a1290 msgr2=0x7fc7f00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc80dd30700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7f00a1290 0x7fc7f00a3750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc808198630 tx=0x7fc7fc00b410 comp rx=0 tx=0).stop 2024-03-18T20:41:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc80dd30700 1 -- 172.21.15.46:0/3181700319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8080ffbb0 msgr2=0x7fc8081970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:13.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc80dd30700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8080ffbb0 0x7fc8081970e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8000c00 tx=0x7fc7f8004730 comp rx=0 tx=0).stop 2024-03-18T20:41:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc80dd30700 1 -- 172.21.15.46:0/3181700319 shutdown_connections 2024-03-18T20:41:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc80dd30700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7f00a1290 0x7fc7f00a3750 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc80dd30700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8080ffbb0 0x7fc8081970e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc80dd30700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc808105c60 0x7fc80819b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc80dd30700 1 --2- 172.21.15.46:0/3181700319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc808100570 0x7fc808197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.698+0000 7fc80dd30700 1 -- 172.21.15.46:0/3181700319 >> 172.21.15.46:0/3181700319 conn(0x7fc808076d50 msgr2=0x7fc8080682c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.699+0000 7fc80dd30700 1 -- 172.21.15.46:0/3181700319 shutdown_connections 2024-03-18T20:41:13.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:13.699+0000 7fc80dd30700 1 -- 172.21.15.46:0/3181700319 wait complete. 2024-03-18T20:41:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:14 smithi073 ceph-mon[47384]: pgmap v157: 97 pgs: 97 active+clean; 64 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-03-18T20:41:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:14 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:41:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:14 smithi155 ceph-mon[48344]: pgmap v157: 97 pgs: 97 active+clean; 64 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-03-18T20:41:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:14 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:41:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:14 smithi046 ceph-mon[43495]: pgmap v157: 97 pgs: 97 active+clean; 64 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.6 MiB/s wr, 178 op/s 2024-03-18T20:41:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:14 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:41:14.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 144002 2.56 MB/sec execute 129 sec latency 26.463 ms 2024-03-18T20:41:15.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:15 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:41:15.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:15 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:41:15.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:15 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:41:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:15 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:41:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:15 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:41:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:15 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:41:15.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:15 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:41:15.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:15 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:41:15.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:15 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:41:15.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 145813 2.65 MB/sec execute 130 sec latency 18.380 ms 2024-03-18T20:41:16.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:16 smithi073 ceph-mon[47384]: pgmap v158: 97 pgs: 97 active+clean; 64 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-03-18T20:41:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:16 smithi155 ceph-mon[48344]: pgmap v158: 97 pgs: 97 active+clean; 64 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-03-18T20:41:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:16 smithi046 ceph-mon[43495]: pgmap v158: 97 pgs: 97 active+clean; 64 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-03-18T20:41:16.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 146126 2.63 MB/sec execute 131 sec latency 665.841 ms 2024-03-18T20:41:17.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 147286 2.65 MB/sec execute 132 sec latency 1016.185 ms 2024-03-18T20:41:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:18 smithi073 ceph-mon[47384]: pgmap v159: 97 pgs: 97 active+clean; 64 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-03-18T20:41:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:18 smithi155 ceph-mon[48344]: pgmap v159: 97 pgs: 97 active+clean; 64 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-03-18T20:41:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:18 smithi046 ceph-mon[43495]: pgmap v159: 97 pgs: 97 active+clean; 64 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-03-18T20:41:18.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 149324 2.73 MB/sec execute 133 sec latency 19.232 ms 2024-03-18T20:41:19.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 149667 2.71 MB/sec execute 134 sec latency 623.204 ms 2024-03-18T20:41:20.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:20 smithi073 ceph-mon[47384]: pgmap v160: 97 pgs: 97 active+clean; 63 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-03-18T20:41:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:20 smithi155 ceph-mon[48344]: pgmap v160: 97 pgs: 97 active+clean; 63 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-03-18T20:41:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:20 smithi046 ceph-mon[43495]: pgmap v160: 97 pgs: 97 active+clean; 63 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-03-18T20:41:20.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 149667 2.69 MB/sec execute 135 sec latency 1623.313 ms 2024-03-18T20:41:21.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 149667 2.67 MB/sec execute 136 sec latency 2623.352 ms 2024-03-18T20:41:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:22 smithi073 ceph-mon[47384]: pgmap v161: 97 pgs: 97 active+clean; 63 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-03-18T20:41:22.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:22 smithi155 ceph-mon[48344]: pgmap v161: 97 pgs: 97 active+clean; 63 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-03-18T20:41:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:22 smithi046 ceph-mon[43495]: pgmap v161: 97 pgs: 97 active+clean; 63 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-03-18T20:41:22.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 149723 2.65 MB/sec execute 137 sec latency 2981.115 ms 2024-03-18T20:41:23.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 149723 2.63 MB/sec execute 138 sec latency 1582.370 ms 2024-03-18T20:41:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:24 smithi073 ceph-mon[47384]: pgmap v162: 97 pgs: 97 active+clean; 67 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 212 op/s 2024-03-18T20:41:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:24 smithi155 ceph-mon[48344]: pgmap v162: 97 pgs: 97 active+clean; 67 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 212 op/s 2024-03-18T20:41:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:24 smithi046 ceph-mon[43495]: pgmap v162: 97 pgs: 97 active+clean; 67 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 212 op/s 2024-03-18T20:41:24.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 149723 2.61 MB/sec execute 139 sec latency 2582.500 ms 2024-03-18T20:41:25.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 149723 2.60 MB/sec execute 140 sec latency 3582.617 ms 2024-03-18T20:41:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:25 smithi073 ceph-mon[47384]: pgmap v163: 97 pgs: 97 active+clean; 67 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 919 KiB/s wr, 125 op/s 2024-03-18T20:41:26.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:41:26.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:41:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:25 smithi155 ceph-mon[48344]: pgmap v163: 97 pgs: 97 active+clean; 67 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 919 KiB/s wr, 125 op/s 2024-03-18T20:41:26.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:25 smithi046 ceph-mon[43495]: pgmap v163: 97 pgs: 97 active+clean; 67 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 919 KiB/s wr, 125 op/s 2024-03-18T20:41:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.426+0000 7f4f8748a700 1 -- 172.21.15.46:0/1415979220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f80104f60 msgr2=0x7f4f8010cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.426+0000 7f4f8748a700 1 --2- 172.21.15.46:0/1415979220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f80104f60 0x7f4f8010cae0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800b0a0 tx=0x7f4f7800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.426+0000 7f4f8748a700 1 -- 172.21.15.46:0/1415979220 shutdown_connections 2024-03-18T20:41:26.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.426+0000 7f4f8748a700 1 --2- 172.21.15.46:0/1415979220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f80104f60 0x7f4f8010cae0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.426+0000 7f4f8748a700 1 --2- 172.21.15.46:0/1415979220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f800ffe50 0x7f4f80100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.426+0000 7f4f8748a700 1 --2- 172.21.15.46:0/1415979220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f80100800 0x7f4f80104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.426+0000 7f4f8748a700 1 -- 172.21.15.46:0/1415979220 >> 172.21.15.46:0/1415979220 conn(0x7f4f80076d40 msgr2=0x7f4f80077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.426+0000 7f4f8748a700 1 -- 172.21.15.46:0/1415979220 shutdown_connections 2024-03-18T20:41:26.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.426+0000 7f4f8748a700 1 -- 172.21.15.46:0/1415979220 wait complete. 2024-03-18T20:41:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f8748a700 1 Processor -- start 2024-03-18T20:41:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f8748a700 1 -- start start 2024-03-18T20:41:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f8748a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f800ffe50 0x7f4f801994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f8748a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f80104f60 0x7f4f80199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f8748a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f8019dda0 0x7f4f8019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f8748a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8019e820 con 0x7f4f800ffe50 2024-03-18T20:41:26.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f8748a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8019e990 con 0x7f4f8019dda0 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f8748a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8019eb50 con 0x7f4f80104f60 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f85226700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f800ffe50 0x7f4f801994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f85226700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f800ffe50 0x7f4f801994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35718/0 (socket says 172.21.15.46:35718) 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f85226700 1 -- 172.21.15.46:0/3184193994 learned_addr learned my addr 172.21.15.46:0/3184193994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f85a27700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f8019dda0 0x7f4f8019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f84a25700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f80104f60 0x7f4f80199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f85226700 1 -- 172.21.15.46:0/3184193994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f80104f60 msgr2=0x7f4f80199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f85226700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f80104f60 0x7f4f80199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f85226700 1 -- 172.21.15.46:0/3184193994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f8019dda0 msgr2=0x7f4f8019e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f85226700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f8019dda0 0x7f4f8019e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.427+0000 7f4f85226700 1 -- 172.21.15.46:0/3184193994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f78009d00 con 0x7f4f800ffe50 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.428+0000 7f4f85226700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f800ffe50 0x7f4f801994c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000d950 tx=0x7f4f7000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.428+0000 7f4f767fc700 1 -- 172.21.15.46:0/3184193994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70009b40 con 0x7f4f800ffe50 2024-03-18T20:41:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.428+0000 7f4f8748a700 1 -- 172.21.15.46:0/3184193994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f801a60b0 con 0x7f4f800ffe50 2024-03-18T20:41:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.428+0000 7f4f767fc700 1 -- 172.21.15.46:0/3184193994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f70009ca0 con 0x7f4f800ffe50 2024-03-18T20:41:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.428+0000 7f4f767fc700 1 -- 172.21.15.46:0/3184193994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7000f620 con 0x7f4f800ffe50 2024-03-18T20:41:26.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.428+0000 7f4f8748a700 1 -- 172.21.15.46:0/3184193994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f801a65e0 con 0x7f4f800ffe50 2024-03-18T20:41:26.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.430+0000 7f4f767fc700 1 -- 172.21.15.46:0/3184193994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f4f70010460 con 0x7f4f800ffe50 2024-03-18T20:41:26.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.431+0000 7f4f767fc700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4f6c0a1290 0x7f4f6c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.431+0000 7f4f84a25700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4f6c0a1290 0x7f4f6c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.431+0000 7f4f767fc700 1 -- 172.21.15.46:0/3184193994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f4f700cd090 con 0x7f4f800ffe50 2024-03-18T20:41:26.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.432+0000 7f4f84a25700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4f6c0a1290 0x7f4f6c0a3750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4f8019aa10 tx=0x7f4f7c00d4b0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.432+0000 7f4f9361fb80 1 Processor -- start 2024-03-18T20:41:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.432+0000 7f4f9361fb80 1 -- start start 2024-03-18T20:41:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f9361fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db7a60 0x564074db7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f9361fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db8380 0x564074dc2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f9361fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074dc3220 0x564074dc5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f9361fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564074d11990 con 0x564074db7a60 2024-03-18T20:41:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f9361fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564074d0f350 con 0x564074db8380 2024-03-18T20:41:26.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f9361fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564074d0ac40 con 0x564074dc3220 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f85226700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db7a60 0x564074db7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f85226700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db7a60 0x564074db7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35724/0 (socket says 172.21.15.46:35724) 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f85226700 1 -- 172.21.15.46:0/2674536757 learned_addr learned my addr 172.21.15.46:0/2674536757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f84a25700 1 --2- 172.21.15.46:0/2674536757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db8380 0x564074dc2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f85a27700 1 --2- 172.21.15.46:0/2674536757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074dc3220 0x564074dc5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f85226700 1 -- 172.21.15.46:0/2674536757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074dc3220 msgr2=0x564074dc5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f85226700 1 --2- 172.21.15.46:0/2674536757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074dc3220 0x564074dc5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f85226700 1 -- 172.21.15.46:0/2674536757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db8380 msgr2=0x564074dc2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f85226700 1 --2- 172.21.15.46:0/2674536757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db8380 0x564074dc2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f85226700 1 -- 172.21.15.46:0/2674536757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564074b8c3b0 con 0x564074db7a60 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f85a27700 1 --2- 172.21.15.46:0/2674536757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074dc3220 0x564074dc5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.433+0000 7f4f85226700 1 --2- 172.21.15.46:0/2674536757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db7a60 0x564074db7e40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4f700a20c0 tx=0x7f4f70099db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfa71759276fe2c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.434+0000 7f4f6b7fe700 1 -- 172.21.15.46:0/2674536757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7009a170 con 0x564074db7a60 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.434+0000 7f4f6b7fe700 1 -- 172.21.15.46:0/2674536757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f7009dfa0 con 0x564074db7a60 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.434+0000 7f4f6b7fe700 1 -- 172.21.15.46:0/2674536757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7009a3d0 con 0x564074db7a60 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.434+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2674536757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db7a60 msgr2=0x564074db7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.434+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/2674536757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db7a60 0x564074db7e40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4f700a20c0 tx=0x7f4f70099db0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.434+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2674536757 shutdown_connections 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.434+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/2674536757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db7a60 0x564074db7e40 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.434+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/2674536757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074dc3220 0x564074dc5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.434+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/2674536757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db8380 0x564074dc2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.434+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2674536757 >> 172.21.15.46:0/2674536757 conn(0x564074d2e220 msgr2=0x564074db4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.435+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2674536757 shutdown_connections 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.435+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2674536757 wait complete. 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.435+0000 7f4f9361fb80 1 -- start start 2024-03-18T20:41:26.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.435+0000 7f4f9361fb80 1 Processor -- start 2024-03-18T20:41:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f9361fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db7a60 0x564074dc2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f9361fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db8380 0x564074dbff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f9361fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074dc3220 0x564074dc0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f9361fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564074ca6d80 con 0x564074db8380 2024-03-18T20:41:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f9361fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564074ca6a20 con 0x564074dc3220 2024-03-18T20:41:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f9361fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564074b8e6a0 con 0x564074db7a60 2024-03-18T20:41:26.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f9361fb80 10 client.0 _ll_register_callbacks cb 0x564074b123a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f84a25700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db8380 0x564074dbff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f84a25700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db8380 0x564074dbff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35738/0 (socket says 172.21.15.46:35738) 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f84a25700 1 -- 172.21.15.46:0/1420269364 learned_addr learned my addr 172.21.15.46:0/1420269364 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f85a27700 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074dc3220 0x564074dc0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f85226700 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db7a60 0x564074dc2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f84a25700 1 -- 172.21.15.46:0/1420269364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db7a60 msgr2=0x564074dc2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f84a25700 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db7a60 0x564074dc2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f84a25700 1 -- 172.21.15.46:0/1420269364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074dc3220 msgr2=0x564074dc0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f84a25700 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074dc3220 0x564074dc0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f84a25700 1 -- 172.21.15.46:0/1420269364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f801a65e0 con 0x564074db8380 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f85a27700 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074dc3220 0x564074dc0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f84a25700 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db8380 0x564074dbff30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00ff20 tx=0x7f4f7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78981de73347df5d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f68ff9700 1 -- 172.21.15.46:0/1420269364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c028670 con 0x564074db8380 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f68ff9700 1 -- 172.21.15.46:0/1420269364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f4f7c0287d0 con 0x564074db8380 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.436+0000 7f4f9361fb80 1 -- 172.21.15.46:0/1420269364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564074dabad0 con 0x564074db8380 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.437+0000 7f4f68ff9700 1 -- 172.21.15.46:0/1420269364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f7c028990 con 0x564074db8380 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.437+0000 7f4f9361fb80 10 client.17337 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.437+0000 7f4f9361fb80 1 -- 172.21.15.46:0/1420269364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564074b38b60 con 0x564074db8380 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.437+0000 7f4f68ff9700 1 -- 172.21.15.46:0/1420269364 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564074dabad0 con 0x564074db8380 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.437+0000 7f4f68ff9700 1 -- 172.21.15.46:0/1420269364 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4f7c031d10 con 0x564074db8380 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.437+0000 7f4f9361fb80 10 client.17337 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.437+0000 7f4f9361fb80 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.437+0000 7f4f9361fb80 20 client.17337 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.437+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564074dd2d40 0x564074dd5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.437+0000 7f4f9361fb80 4 client.17337 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.437+0000 7f4f9361fb80 1 -- 172.21.15.46:0/1420269364 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564074d03700 con 0x564074dd2d40 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.437+0000 7f4f85226700 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564074dd2d40 0x564074dd5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.438+0000 7f4f85226700 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564074dd2d40 0x564074dd5160 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb4a9a17d06ea6d6 server_cookie=46d73e659f27491a in_seq=0 out_seq=0 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.438+0000 7f4f68ff9700 10 client.17337 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.438+0000 7f4f68ff9700 1 -- 172.21.15.46:0/1420269364 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564074d03700 con 0x564074dd2d40 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.438+0000 7f4f68ff9700 10 client.17337 handle_command_reply: tid=0 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.438+0000 7f4f9361fb80 1 client.17337 shutdown 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.439+0000 7f4f9361fb80 1 -- 172.21.15.46:0/1420269364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db8380 msgr2=0x564074dbff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.439+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db8380 0x564074dbff30 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00ff20 tx=0x7f4f7c004750 comp rx=0 tx=0).stop 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.439+0000 7f4f9361fb80 1 -- 172.21.15.46:0/1420269364 shutdown_connections 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.439+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564074dd2d40 0x564074dd5160 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.439+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074dc3220 0x564074dc0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.439+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db7a60 0x564074dc2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.439+0000 7f4f68ff9700 0 client.17337 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.439+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/1420269364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074db8380 0x564074dbff30 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.439+0000 7f4f9361fb80 1 -- 172.21.15.46:0/1420269364 >> 172.21.15.46:0/1420269364 conn(0x564074d2e220 msgr2=0x564074db3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.439+0000 7f4f9361fb80 1 -- 172.21.15.46:0/1420269364 shutdown_connections 2024-03-18T20:41:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.439+0000 7f4f9361fb80 1 -- 172.21.15.46:0/1420269364 wait complete. 2024-03-18T20:41:26.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.439+0000 7f4f9361fb80 20 client.17337 trim_cache size 0 max 16384 2024-03-18T20:41:26.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f9361fb80 1 Processor -- start 2024-03-18T20:41:26.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f9361fb80 1 -- start start 2024-03-18T20:41:26.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f9361fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db7a60 0x564074dc9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f9361fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db8380 0x564074dcec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f9361fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 0x564074dc27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f9361fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564074db9610 con 0x564074dc3220 2024-03-18T20:41:26.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f9361fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564074dca330 con 0x564074db8380 2024-03-18T20:41:26.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f9361fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564074dbe480 con 0x564074db7a60 2024-03-18T20:41:26.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f85a27700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 0x564074dc27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f85a27700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 0x564074dc27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35744/0 (socket says 172.21.15.46:35744) 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f85a27700 1 -- 172.21.15.46:0/3936126451 learned_addr learned my addr 172.21.15.46:0/3936126451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f85226700 1 --2- 172.21.15.46:0/3936126451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db7a60 0x564074dc9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f84a25700 1 --2- 172.21.15.46:0/3936126451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db8380 0x564074dcec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f85a27700 1 -- 172.21.15.46:0/3936126451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db7a60 msgr2=0x564074dc9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f85a27700 1 --2- 172.21.15.46:0/3936126451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db7a60 0x564074dc9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.457+0000 7f4f85a27700 1 -- 172.21.15.46:0/3936126451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db8380 msgr2=0x564074dcec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.458+0000 7f4f85a27700 1 --2- 172.21.15.46:0/3936126451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db8380 0x564074dcec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.458+0000 7f4f85a27700 1 -- 172.21.15.46:0/3936126451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564074cd0ea0 con 0x564074dc3220 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.458+0000 7f4f84a25700 1 --2- 172.21.15.46:0/3936126451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db8380 0x564074dcec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.458+0000 7f4f85a27700 1 --2- 172.21.15.46:0/3936126451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 0x564074dc27e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4f78015000 tx=0x7f4f78009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667fa6e7f350a872 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.458+0000 7f4f6bfff700 1 -- 172.21.15.46:0/3936126451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7800e040 con 0x564074dc3220 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.458+0000 7f4f6bfff700 1 -- 172.21.15.46:0/3936126451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f78020070 con 0x564074dc3220 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.458+0000 7f4f6bfff700 1 -- 172.21.15.46:0/3936126451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78005040 con 0x564074dc3220 2024-03-18T20:41:26.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.458+0000 7f4f9361fb80 1 -- 172.21.15.46:0/3936126451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 msgr2=0x564074dc27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.458+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/3936126451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 0x564074dc27e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4f78015000 tx=0x7f4f78009c40 comp rx=0 tx=0).stop 2024-03-18T20:41:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 -- 172.21.15.46:0/3936126451 shutdown_connections 2024-03-18T20:41:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/3936126451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 0x564074dc27e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/3936126451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db7a60 0x564074dc9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/3936126451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db8380 0x564074dcec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 -- 172.21.15.46:0/3936126451 >> 172.21.15.46:0/3936126451 conn(0x564074d2e220 msgr2=0x564074dce540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 -- 172.21.15.46:0/3936126451 shutdown_connections 2024-03-18T20:41:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 -- 172.21.15.46:0/3936126451 wait complete. 2024-03-18T20:41:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 -- start start 2024-03-18T20:41:26.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 Processor -- start 2024-03-18T20:41:26.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db7a60 0x564074dc9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db8380 0x564074dc7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 0x564074dc8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564074dc88b0 con 0x564074dc3220 2024-03-18T20:41:26.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564074dc8a20 con 0x564074db7a60 2024-03-18T20:41:26.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564074dc8be0 con 0x564074db8380 2024-03-18T20:41:26.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.459+0000 7f4f9361fb80 10 client.0 _ll_register_callbacks cb 0x7f4f7c00fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:26.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f85a27700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 0x564074dc8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f85226700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db7a60 0x564074dc9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f85a27700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 0x564074dc8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35746/0 (socket says 172.21.15.46:35746) 2024-03-18T20:41:26.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f85a27700 1 -- 172.21.15.46:0/2402094713 learned_addr learned my addr 172.21.15.46:0/2402094713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:26.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f84a25700 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db8380 0x564074dc7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f85a27700 1 -- 172.21.15.46:0/2402094713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db8380 msgr2=0x564074dc7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f85a27700 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db8380 0x564074dc7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f85a27700 1 -- 172.21.15.46:0/2402094713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db7a60 msgr2=0x564074dc9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f85a27700 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db7a60 0x564074dc9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f85a27700 1 -- 172.21.15.46:0/2402094713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8010d350 con 0x564074dc3220 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f85226700 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db7a60 0x564074dc9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f85a27700 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 0x564074dc8370 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800b830 tx=0x7f4f78004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dce81e36a29a88c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f68ff9700 1 -- 172.21.15.46:0/2402094713 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7800e030 con 0x564074dc3220 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f68ff9700 1 -- 172.21.15.46:0/2402094713 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f4f78022070 con 0x564074dc3220 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2402094713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f7009dfa0 con 0x564074dc3220 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f68ff9700 1 -- 172.21.15.46:0/2402094713 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f78007970 con 0x564074dc3220 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f9361fb80 10 client.17343 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.460+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2402094713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564074b90cd0 con 0x564074dc3220 2024-03-18T20:41:26.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.461+0000 7f4f68ff9700 1 -- 172.21.15.46:0/2402094713 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f7009dfa0 con 0x564074dc3220 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.461+0000 7f4f68ff9700 1 -- 172.21.15.46:0/2402094713 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4f7800b0d0 con 0x564074dc3220 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.461+0000 7f4f9361fb80 10 client.17343 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.461+0000 7f4f9361fb80 10 client.17343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.461+0000 7f4f9361fb80 20 client.17343 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.461+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564074e27d10 0x564074e2a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.461+0000 7f4f9361fb80 4 client.17343 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.461+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2402094713 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564074b59cf0 con 0x564074e27d10 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.461+0000 7f4f85226700 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564074e27d10 0x564074e2a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.461+0000 7f4f68ff9700 10 client.17343 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.461+0000 7f4f85226700 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564074e27d10 0x564074e2a130 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9485cc2a662927ad server_cookie=8be505e9a23c8d09 in_seq=0 out_seq=0 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f68ff9700 1 -- 172.21.15.46:0/2402094713 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564074b59cf0 con 0x564074e27d10 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f68ff9700 10 client.17343 handle_command_reply: tid=0 2024-03-18T20:41:26.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f9361fb80 1 client.17343 shutdown 2024-03-18T20:41:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2402094713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 msgr2=0x564074dc8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 0x564074dc8370 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800b830 tx=0x7f4f78004d00 comp rx=0 tx=0).stop 2024-03-18T20:41:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2402094713 shutdown_connections 2024-03-18T20:41:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564074e27d10 0x564074e2a130 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564074dc3220 0x564074dc8370 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564074db7a60 0x564074dc9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f9361fb80 1 --2- 172.21.15.46:0/2402094713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564074db8380 0x564074dc7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f68ff9700 0 client.17343 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2402094713 >> 172.21.15.46:0/2402094713 conn(0x564074d2e220 msgr2=0x564074dce540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2402094713 shutdown_connections 2024-03-18T20:41:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f9361fb80 1 -- 172.21.15.46:0/2402094713 wait complete. 2024-03-18T20:41:26.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.462+0000 7f4f9361fb80 20 client.17343 trim_cache size 0 max 16384 2024-03-18T20:41:26.473 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:41:26.473 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:41:26.473 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:41:26.473 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:41:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.463+0000 7f4f8748a700 1 -- 172.21.15.46:0/3184193994 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4f6c0a1290 msgr2=0x7f4f6c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.463+0000 7f4f8748a700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4f6c0a1290 0x7f4f6c0a3750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4f8019aa10 tx=0x7f4f7c00d4b0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.463+0000 7f4f8748a700 1 -- 172.21.15.46:0/3184193994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f800ffe50 msgr2=0x7f4f801994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.463+0000 7f4f8748a700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f800ffe50 0x7f4f801994c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000d950 tx=0x7f4f7000dd10 comp rx=0 tx=0).stop 2024-03-18T20:41:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.463+0000 7f4f8748a700 1 -- 172.21.15.46:0/3184193994 shutdown_connections 2024-03-18T20:41:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.463+0000 7f4f8748a700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4f6c0a1290 0x7f4f6c0a3750 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.463+0000 7f4f8748a700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f800ffe50 0x7f4f801994c0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.463+0000 7f4f8748a700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f8019dda0 0x7f4f8019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.463+0000 7f4f8748a700 1 --2- 172.21.15.46:0/3184193994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f80104f60 0x7f4f80199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.463+0000 7f4f8748a700 1 -- 172.21.15.46:0/3184193994 >> 172.21.15.46:0/3184193994 conn(0x7f4f80076d40 msgr2=0x7f4f800683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.463+0000 7f4f8748a700 1 -- 172.21.15.46:0/3184193994 shutdown_connections 2024-03-18T20:41:26.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.463+0000 7f4f8748a700 1 -- 172.21.15.46:0/3184193994 wait complete. 2024-03-18T20:41:26.479 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:41:26.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:41:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.655+0000 7fa8c80d4700 1 -- 172.21.15.46:0/194896404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c01004a0 msgr2=0x7fa8c010c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.655+0000 7fa8c80d4700 1 --2- 172.21.15.46:0/194896404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c01004a0 0x7fa8c010c9b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800b0a0 tx=0x7fa8b800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.655+0000 7fa8c80d4700 1 -- 172.21.15.46:0/194896404 shutdown_connections 2024-03-18T20:41:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.655+0000 7fa8c80d4700 1 --2- 172.21.15.46:0/194896404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c01004a0 0x7fa8c010c9b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.655+0000 7fa8c80d4700 1 --2- 172.21.15.46:0/194896404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c0105b80 0x7fa8c0105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.655+0000 7fa8c80d4700 1 --2- 172.21.15.46:0/194896404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c00ffae0 0x7fa8c00fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.655+0000 7fa8c80d4700 1 -- 172.21.15.46:0/194896404 >> 172.21.15.46:0/194896404 conn(0x7fa8c00fb5f0 msgr2=0x7fa8c00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.656+0000 7fa8c80d4700 1 -- 172.21.15.46:0/194896404 shutdown_connections 2024-03-18T20:41:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.656+0000 7fa8c80d4700 1 -- 172.21.15.46:0/194896404 wait complete. 2024-03-18T20:41:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.656+0000 7fa8c80d4700 1 Processor -- start 2024-03-18T20:41:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.656+0000 7fa8c80d4700 1 -- start start 2024-03-18T20:41:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.656+0000 7fa8c80d4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c00ffae0 0x7fa8c0197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.656+0000 7fa8c80d4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c01004a0 0x7fa8c0197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.656+0000 7fa8c80d4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c0105b80 0x7fa8c019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.656+0000 7fa8c80d4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c019bef0 con 0x7fa8c0105b80 2024-03-18T20:41:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.656+0000 7fa8c80d4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c019c060 con 0x7fa8c01004a0 2024-03-18T20:41:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.656+0000 7fa8c80d4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c019c220 con 0x7fa8c00ffae0 2024-03-18T20:41:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c6671700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c0105b80 0x7fa8c019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c6671700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c0105b80 0x7fa8c019b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35760/0 (socket says 172.21.15.46:35760) 2024-03-18T20:41:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c6671700 1 -- 172.21.15.46:0/1252026233 learned_addr learned my addr 172.21.15.46:0/1252026233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c5e70700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c00ffae0 0x7fa8c0197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c566f700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c01004a0 0x7fa8c0197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c6671700 1 -- 172.21.15.46:0/1252026233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c00ffae0 msgr2=0x7fa8c0197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c6671700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c00ffae0 0x7fa8c0197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c6671700 1 -- 172.21.15.46:0/1252026233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c01004a0 msgr2=0x7fa8c0197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c6671700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c01004a0 0x7fa8c0197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c6671700 1 -- 172.21.15.46:0/1252026233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b8009d00 con 0x7fa8c0105b80 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c5e70700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c00ffae0 0x7fa8c0197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c6671700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c0105b80 0x7fa8c019b8b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800b070 tx=0x7fa8b8007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8b2ffd700 1 -- 172.21.15.46:0/1252026233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b800e040 con 0x7fa8c0105b80 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8b2ffd700 1 -- 172.21.15.46:0/1252026233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8b8004e60 con 0x7fa8c0105b80 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8c80d4700 1 -- 172.21.15.46:0/1252026233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c019c4a0 con 0x7fa8c0105b80 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.657+0000 7fa8b2ffd700 1 -- 172.21.15.46:0/1252026233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80052a0 con 0x7fa8c0105b80 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.658+0000 7fa8c80d4700 1 -- 172.21.15.46:0/1252026233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c01a3d40 con 0x7fa8c0105b80 2024-03-18T20:41:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.658+0000 7fa8c80d4700 1 -- 172.21.15.46:0/1252026233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c01090a0 con 0x7fa8c0105b80 2024-03-18T20:41:26.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.660+0000 7fa8b2ffd700 1 -- 172.21.15.46:0/1252026233 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa8b801d470 con 0x7fa8c0105b80 2024-03-18T20:41:26.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.661+0000 7fa8b2ffd700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa8ac0a1290 0x7fa8ac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.661+0000 7fa8b2ffd700 1 -- 172.21.15.46:0/1252026233 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fa8b80cec50 con 0x7fa8c0105b80 2024-03-18T20:41:26.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.661+0000 7fa8c5e70700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa8ac0a1290 0x7fa8ac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.661+0000 7fa8c5e70700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa8ac0a1290 0x7fa8ac0a3750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009fd0 tx=0x7fa8b4009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.665+0000 7fa8b2ffd700 1 -- 172.21.15.46:0/1252026233 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa8b8098140 con 0x7fa8c0105b80 2024-03-18T20:41:26.689 INFO:tasks.workunit.client.0.smithi046.stdout: 1 149723 2.58 MB/sec execute 141 sec latency 4582.667 ms 2024-03-18T20:41:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.822+0000 7f0a89d37700 1 -- 172.21.15.46:0/772151673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a840680c0 msgr2=0x7f0a84068540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.822+0000 7f0a89d37700 1 --2- 172.21.15.46:0/772151673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a840680c0 0x7f0a84068540 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009990 tx=0x7f0a78009ca0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.822+0000 7f0a89d37700 1 -- 172.21.15.46:0/772151673 shutdown_connections 2024-03-18T20:41:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.822+0000 7f0a89d37700 1 --2- 172.21.15.46:0/772151673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a84106d50 0x7f0a84107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.822+0000 7f0a89d37700 1 --2- 172.21.15.46:0/772151673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a84068a80 0x7f0a8410db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.822+0000 7f0a89d37700 1 --2- 172.21.15.46:0/772151673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a840680c0 0x7f0a84068540 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.822+0000 7f0a89d37700 1 -- 172.21.15.46:0/772151673 >> 172.21.15.46:0/772151673 conn(0x7f0a84076d50 msgr2=0x7f0a84077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.822+0000 7f0a89d37700 1 -- 172.21.15.46:0/772151673 shutdown_connections 2024-03-18T20:41:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.822+0000 7f0a89d37700 1 -- 172.21.15.46:0/772151673 wait complete. 2024-03-18T20:41:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.823+0000 7f0a89d37700 1 Processor -- start 2024-03-18T20:41:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.823+0000 7f0a89d37700 1 -- start start 2024-03-18T20:41:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.823+0000 7f0a89d37700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a840680c0 0x7f0a84197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.823+0000 7f0a89d37700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a84068a80 0x7f0a84197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a89d37700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a84106d50 0x7f0a8419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a89d37700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8419c020 con 0x7f0a840680c0 2024-03-18T20:41:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a89d37700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8419c190 con 0x7f0a84068a80 2024-03-18T20:41:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a89d37700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8419c350 con 0x7f0a84106d50 2024-03-18T20:41:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a840680c0 0x7f0a84197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a840680c0 0x7f0a84197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35772/0 (socket says 172.21.15.46:35772) 2024-03-18T20:41:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a837fe700 1 -- 172.21.15.46:0/3994643945 learned_addr learned my addr 172.21.15.46:0/3994643945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a82ffd700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a84068a80 0x7f0a84197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a83fff700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a84106d50 0x7f0a8419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a837fe700 1 -- 172.21.15.46:0/3994643945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a84106d50 msgr2=0x7f0a8419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a837fe700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a84106d50 0x7f0a8419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a837fe700 1 -- 172.21.15.46:0/3994643945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a84068a80 msgr2=0x7f0a84197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a837fe700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a84068a80 0x7f0a84197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a837fe700 1 -- 172.21.15.46:0/3994643945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a780095f0 con 0x7f0a840680c0 2024-03-18T20:41:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.824+0000 7f0a837fe700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a840680c0 0x7f0a84197100 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400e9a0 tx=0x7f0a7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.825+0000 7f0a80ff9700 1 -- 172.21.15.46:0/3994643945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74009790 con 0x7f0a840680c0 2024-03-18T20:41:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.825+0000 7f0a89d37700 1 -- 172.21.15.46:0/3994643945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8419c630 con 0x7f0a840680c0 2024-03-18T20:41:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.825+0000 7f0a80ff9700 1 -- 172.21.15.46:0/3994643945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a7400cbd0 con 0x7f0a840680c0 2024-03-18T20:41:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.825+0000 7f0a80ff9700 1 -- 172.21.15.46:0/3994643945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74018680 con 0x7f0a840680c0 2024-03-18T20:41:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.825+0000 7f0a89d37700 1 -- 172.21.15.46:0/3994643945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a841a3e70 con 0x7f0a840680c0 2024-03-18T20:41:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.828+0000 7f0a80ff9700 1 -- 172.21.15.46:0/3994643945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0a7400cd40 con 0x7f0a840680c0 2024-03-18T20:41:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.828+0000 7f0a80ff9700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0a6c0a1290 0x7f0a6c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.829+0000 7f0a82ffd700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0a6c0a1290 0x7f0a6c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.829+0000 7f0a80ff9700 1 -- 172.21.15.46:0/3994643945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0a74014070 con 0x7f0a840680c0 2024-03-18T20:41:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.829+0000 7f0a95eccb80 1 Processor -- start 2024-03-18T20:41:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.829+0000 7f0a95eccb80 1 -- start start 2024-03-18T20:41:26.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.829+0000 7f0a95eccb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 0x562cdf2f1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.829+0000 7f0a82ffd700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0a6c0a1290 0x7f0a6c0a3750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a78006010 tx=0x7f0a7800b260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.829+0000 7f0a95eccb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf300c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.829+0000 7f0a95eccb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf3011c0 0x562cdf3035b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.829+0000 7f0a95eccb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf24b930 con 0x562cdf2f1570 2024-03-18T20:41:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.829+0000 7f0a95eccb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf2492f0 con 0x562cdf3011c0 2024-03-18T20:41:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.829+0000 7f0a95eccb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf244be0 con 0x562cdf2f1e90 2024-03-18T20:41:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.829+0000 7f0a837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 0x562cdf2f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 0x562cdf2f1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35780/0 (socket says 172.21.15.46:35780) 2024-03-18T20:41:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a837fe700 1 -- 172.21.15.46:0/3527930016 learned_addr learned my addr 172.21.15.46:0/3527930016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a82ffd700 1 --2- 172.21.15.46:0/3527930016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf300c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a83fff700 1 --2- 172.21.15.46:0/3527930016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf3011c0 0x562cdf3035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a837fe700 1 -- 172.21.15.46:0/3527930016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 msgr2=0x562cdf300c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a837fe700 1 --2- 172.21.15.46:0/3527930016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf300c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a837fe700 1 -- 172.21.15.46:0/3527930016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf3011c0 msgr2=0x562cdf3035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a837fe700 1 --2- 172.21.15.46:0/3527930016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf3011c0 0x562cdf3035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a837fe700 1 -- 172.21.15.46:0/3527930016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cdf0c6350 con 0x562cdf2f1570 2024-03-18T20:41:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a837fe700 1 --2- 172.21.15.46:0/3527930016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 0x562cdf2f1950 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0a74096a20 tx=0x7f0a74099990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad991ca78811e01 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a69ffb700 1 -- 172.21.15.46:0/3527930016 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7409dde0 con 0x562cdf2f1570 2024-03-18T20:41:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a69ffb700 1 -- 172.21.15.46:0/3527930016 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a740a2430 con 0x562cdf2f1570 2024-03-18T20:41:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.830+0000 7f0a69ffb700 1 -- 172.21.15.46:0/3527930016 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a740a1fa0 con 0x562cdf2f1570 2024-03-18T20:41:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.831+0000 7f0a95eccb80 1 -- 172.21.15.46:0/3527930016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 msgr2=0x562cdf2f1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.831+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/3527930016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 0x562cdf2f1950 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0a74096a20 tx=0x7f0a74099990 comp rx=0 tx=0).stop 2024-03-18T20:41:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.831+0000 7f0a95eccb80 1 -- 172.21.15.46:0/3527930016 shutdown_connections 2024-03-18T20:41:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.831+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/3527930016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 0x562cdf2f1950 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.831+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/3527930016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf3011c0 0x562cdf3035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.831+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/3527930016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf300c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.831+0000 7f0a95eccb80 1 -- 172.21.15.46:0/3527930016 >> 172.21.15.46:0/3527930016 conn(0x562cdf2681c0 msgr2=0x562cdf2eda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.831+0000 7f0a95eccb80 1 -- 172.21.15.46:0/3527930016 shutdown_connections 2024-03-18T20:41:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.831+0000 7f0a95eccb80 1 -- 172.21.15.46:0/3527930016 wait complete. 2024-03-18T20:41:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a95eccb80 1 -- start start 2024-03-18T20:41:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a95eccb80 1 Processor -- start 2024-03-18T20:41:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a95eccb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1570 0x562cdf300c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a95eccb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf2f1e90 0x562cdf2fded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a95eccb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 0x562cdf2fe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a95eccb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf1e0d20 con 0x562cdf3011c0 2024-03-18T20:41:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a95eccb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf1e09c0 con 0x562cdf2f1e90 2024-03-18T20:41:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a95eccb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf0c8640 con 0x562cdf2f1570 2024-03-18T20:41:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a95eccb80 10 client.0 _ll_register_callbacks cb 0x562cdf04c340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a83fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 0x562cdf2fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a83fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 0x562cdf2fe610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35796/0 (socket says 172.21.15.46:35796) 2024-03-18T20:41:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a83fff700 1 -- 172.21.15.46:0/2430890241 learned_addr learned my addr 172.21.15.46:0/2430890241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.832+0000 7f0a837fe700 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1570 0x562cdf300c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a82ffd700 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf2f1e90 0x562cdf2fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a83fff700 1 -- 172.21.15.46:0/2430890241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1570 msgr2=0x562cdf300c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a83fff700 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1570 0x562cdf300c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a83fff700 1 -- 172.21.15.46:0/2430890241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf2f1e90 msgr2=0x562cdf2fded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a83fff700 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf2f1e90 0x562cdf2fded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a83fff700 1 -- 172.21.15.46:0/2430890241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a841a3e70 con 0x562cdf3011c0 2024-03-18T20:41:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a83fff700 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 0x562cdf2fe610 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0042a0 tx=0x7f0a7c0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b655d4db20be6086 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a62ffd700 1 -- 172.21.15.46:0/2430890241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c007700 con 0x562cdf3011c0 2024-03-18T20:41:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2430890241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cdf2e56c0 con 0x562cdf3011c0 2024-03-18T20:41:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a62ffd700 1 -- 172.21.15.46:0/2430890241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0a7c007860 con 0x562cdf3011c0 2024-03-18T20:41:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a62ffd700 1 -- 172.21.15.46:0/2430890241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a7c007a60 con 0x562cdf3011c0 2024-03-18T20:41:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a95eccb80 10 client.17361 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.833+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2430890241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x562cdf072b00 con 0x562cdf3011c0 2024-03-18T20:41:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.834+0000 7f0a62ffd700 1 -- 172.21.15.46:0/2430890241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562cdf2e56c0 con 0x562cdf3011c0 2024-03-18T20:41:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.834+0000 7f0a62ffd700 1 -- 172.21.15.46:0/2430890241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0a7c016c90 con 0x562cdf3011c0 2024-03-18T20:41:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.834+0000 7f0a95eccb80 10 client.17361 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.834+0000 7f0a95eccb80 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.834+0000 7f0a95eccb80 20 client.17361 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.834+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562cdf310e10 0x562cdf313230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.834+0000 7f0a95eccb80 4 client.17361 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.834+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2430890241 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cdf23d6a0 con 0x562cdf310e10 2024-03-18T20:41:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.834+0000 7f0a837fe700 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562cdf310e10 0x562cdf313230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.834+0000 7f0a837fe700 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562cdf310e10 0x562cdf313230 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29878a6f5d05c0a1 server_cookie=fb3cc4a7383662c3 in_seq=0 out_seq=0 2024-03-18T20:41:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a62ffd700 10 client.17361 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a62ffd700 1 -- 172.21.15.46:0/2430890241 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562cdf23d6a0 con 0x562cdf310e10 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a62ffd700 10 client.17361 handle_command_reply: tid=0 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a95eccb80 1 client.17361 shutdown 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2430890241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 msgr2=0x562cdf2fe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 0x562cdf2fe610 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0042a0 tx=0x7f0a7c0042d0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2430890241 shutdown_connections 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562cdf310e10 0x562cdf313230 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 0x562cdf2fe610 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1570 0x562cdf300c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a62ffd700 0 client.17361 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/2430890241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf2f1e90 0x562cdf2fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2430890241 >> 172.21.15.46:0/2430890241 conn(0x562cdf2681c0 msgr2=0x562cdf2ed650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.835+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2430890241 shutdown_connections 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.836+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2430890241 wait complete. 2024-03-18T20:41:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.836+0000 7f0a95eccb80 20 client.17361 trim_cache size 0 max 16384 2024-03-18T20:41:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.851+0000 7f0a95eccb80 1 Processor -- start 2024-03-18T20:41:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.851+0000 7f0a95eccb80 1 -- start start 2024-03-18T20:41:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.851+0000 7f0a95eccb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 0x562cdf3035b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.851+0000 7f0a95eccb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf300780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.851+0000 7f0a95eccb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf3011c0 0x562cdf2fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.851+0000 7f0a95eccb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf2fc420 con 0x562cdf2f1570 2024-03-18T20:41:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.851+0000 7f0a95eccb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf2f9d30 con 0x562cdf3011c0 2024-03-18T20:41:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.851+0000 7f0a95eccb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf30d760 con 0x562cdf2f1e90 2024-03-18T20:41:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 0x562cdf3035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 0x562cdf3035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35808/0 (socket says 172.21.15.46:35808) 2024-03-18T20:41:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a837fe700 1 -- 172.21.15.46:0/1483977627 learned_addr learned my addr 172.21.15.46:0/1483977627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a82ffd700 1 --2- 172.21.15.46:0/1483977627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf300780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a83fff700 1 --2- 172.21.15.46:0/1483977627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf3011c0 0x562cdf2fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a837fe700 1 -- 172.21.15.46:0/1483977627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 msgr2=0x562cdf300780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a837fe700 1 --2- 172.21.15.46:0/1483977627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf300780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a837fe700 1 -- 172.21.15.46:0/1483977627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf3011c0 msgr2=0x562cdf2fd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a837fe700 1 --2- 172.21.15.46:0/1483977627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf3011c0 0x562cdf2fd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a837fe700 1 -- 172.21.15.46:0/1483977627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cdf20ae40 con 0x562cdf2f1570 2024-03-18T20:41:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a837fe700 1 --2- 172.21.15.46:0/1483977627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 0x562cdf3035b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0a74098300 tx=0x7f0a74099e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f382d47b56906a1e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a6a7fc700 1 -- 172.21.15.46:0/1483977627 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a740b8130 con 0x562cdf2f1570 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a6a7fc700 1 -- 172.21.15.46:0/1483977627 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a740b8290 con 0x562cdf2f1570 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.852+0000 7f0a6a7fc700 1 -- 172.21.15.46:0/1483977627 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7409a450 con 0x562cdf2f1570 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.853+0000 7f0a95eccb80 1 -- 172.21.15.46:0/1483977627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 msgr2=0x562cdf3035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.853+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/1483977627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 0x562cdf3035b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0a74098300 tx=0x7f0a74099e10 comp rx=0 tx=0).stop 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.853+0000 7f0a95eccb80 1 -- 172.21.15.46:0/1483977627 shutdown_connections 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.853+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/1483977627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf2f1570 0x562cdf3035b0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.853+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/1483977627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf3011c0 0x562cdf2fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.853+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/1483977627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf300780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.853+0000 7f0a95eccb80 1 -- 172.21.15.46:0/1483977627 >> 172.21.15.46:0/1483977627 conn(0x562cdf2681c0 msgr2=0x562cdf317b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.853+0000 7f0a95eccb80 1 -- 172.21.15.46:0/1483977627 shutdown_connections 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.853+0000 7f0a95eccb80 1 -- 172.21.15.46:0/1483977627 wait complete. 2024-03-18T20:41:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.853+0000 7f0a95eccb80 1 -- start start 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.854+0000 7f0a95eccb80 1 Processor -- start 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.854+0000 7f0a95eccb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf2f1570 0x562cdf305dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.854+0000 7f0a95eccb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf306310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.854+0000 7f0a95eccb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 0x562cdf306850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.854+0000 7f0a95eccb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf363c00 con 0x562cdf3011c0 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.854+0000 7f0a95eccb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf363d70 con 0x562cdf2f1570 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.854+0000 7f0a95eccb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cdf363f30 con 0x562cdf2f1e90 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.854+0000 7f0a95eccb80 10 client.0 _ll_register_callbacks cb 0x7f0a7c009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a83fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 0x562cdf306850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a82ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf306310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a82ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf306310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:45654/0 (socket says 172.21.15.46:45654) 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a83fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 0x562cdf306850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35822/0 (socket says 172.21.15.46:35822) 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a82ffd700 1 -- 172.21.15.46:0/2123616171 learned_addr learned my addr 172.21.15.46:0/2123616171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a837fe700 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf2f1570 0x562cdf305dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a82ffd700 1 -- 172.21.15.46:0/2123616171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf2f1570 msgr2=0x562cdf305dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a82ffd700 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf2f1570 0x562cdf305dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a82ffd700 1 -- 172.21.15.46:0/2123616171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 msgr2=0x562cdf306850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a82ffd700 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 0x562cdf306850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a82ffd700 1 -- 172.21.15.46:0/2123616171 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562cdf0c6350 con 0x562cdf2f1e90 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a83fff700 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 0x562cdf306850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a82ffd700 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf306310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009990 tx=0x7f0a78009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee6fc7569f54a937 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a62ffd700 1 -- 172.21.15.46:0/2123616171 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7802d030 con 0x562cdf2f1e90 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2123616171 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a740a2430 con 0x562cdf2f1e90 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.855+0000 7f0a62ffd700 1 -- 172.21.15.46:0/2123616171 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f0a78004cf0 con 0x562cdf2f1e90 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.856+0000 7f0a62ffd700 1 -- 172.21.15.46:0/2123616171 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a78034480 con 0x562cdf2f1e90 2024-03-18T20:41:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.856+0000 7f0a95eccb80 10 client.27079 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.856+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2123616171 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x562cdf0cac70 con 0x562cdf2f1e90 2024-03-18T20:41:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.856+0000 7f0a62ffd700 1 -- 172.21.15.46:0/2123616171 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a740a2430 con 0x562cdf2f1e90 2024-03-18T20:41:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.856+0000 7f0a62ffd700 1 -- 172.21.15.46:0/2123616171 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0a7800f5f0 con 0x562cdf2f1e90 2024-03-18T20:41:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.856+0000 7f0a95eccb80 10 client.27079 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.856+0000 7f0a95eccb80 10 client.27079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.856+0000 7f0a95eccb80 20 client.27079 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.856+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562cdf3657f0 0x562cdf367c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.856+0000 7f0a95eccb80 4 client.27079 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:41:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.856+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2123616171 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562cdf093c90 con 0x562cdf3657f0 2024-03-18T20:41:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.856+0000 7f0a837fe700 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562cdf3657f0 0x562cdf367c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.857+0000 7f0a62ffd700 10 client.27079 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.857+0000 7f0a837fe700 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562cdf3657f0 0x562cdf367c10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733699e9be8cfa11 server_cookie=80280a4888d86175 in_seq=0 out_seq=0 2024-03-18T20:41:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.886+0000 7f0a62ffd700 1 -- 172.21.15.46:0/2123616171 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562cdf093c90 con 0x562cdf3657f0 2024-03-18T20:41:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.886+0000 7f0a62ffd700 10 client.27079 handle_command_reply: tid=0 2024-03-18T20:41:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.886+0000 7f0a95eccb80 1 client.27079 shutdown 2024-03-18T20:41:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.887+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2123616171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 msgr2=0x562cdf306310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.887+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf306310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009990 tx=0x7f0a78009ca0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.887+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2123616171 shutdown_connections 2024-03-18T20:41:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.887+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562cdf3657f0 0x562cdf367c10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.887+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562cdf3011c0 0x562cdf306850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.887+0000 7f0a62ffd700 0 client.27079 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.887+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562cdf2f1570 0x562cdf305dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.887+0000 7f0a95eccb80 1 --2- 172.21.15.46:0/2123616171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562cdf2f1e90 0x562cdf306310 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.887+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2123616171 >> 172.21.15.46:0/2123616171 conn(0x562cdf2681c0 msgr2=0x562cdf30c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.887+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2123616171 shutdown_connections 2024-03-18T20:41:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.887+0000 7f0a95eccb80 1 -- 172.21.15.46:0/2123616171 wait complete. 2024-03-18T20:41:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.887+0000 7f0a95eccb80 20 client.27079 trim_cache size 0 max 16384 2024-03-18T20:41:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:41:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:41:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:41:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:41:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.888+0000 7f0a89d37700 1 -- 172.21.15.46:0/3994643945 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0a6c0a1290 msgr2=0x7f0a6c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.888+0000 7f0a89d37700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0a6c0a1290 0x7f0a6c0a3750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a78006010 tx=0x7f0a7800b260 comp rx=0 tx=0).stop 2024-03-18T20:41:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.888+0000 7f0a89d37700 1 -- 172.21.15.46:0/3994643945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a840680c0 msgr2=0x7f0a84197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.888+0000 7f0a89d37700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a840680c0 0x7f0a84197100 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400e9a0 tx=0x7f0a7400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.888+0000 7f0a89d37700 1 -- 172.21.15.46:0/3994643945 shutdown_connections 2024-03-18T20:41:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.888+0000 7f0a89d37700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0a6c0a1290 0x7f0a6c0a3750 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.888+0000 7f0a89d37700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a840680c0 0x7f0a84197100 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.888+0000 7f0a89d37700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a84106d50 0x7f0a8419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.888+0000 7f0a89d37700 1 --2- 172.21.15.46:0/3994643945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a84068a80 0x7f0a84197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.888+0000 7f0a89d37700 1 -- 172.21.15.46:0/3994643945 >> 172.21.15.46:0/3994643945 conn(0x7f0a84076d50 msgr2=0x7f0a840fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.888+0000 7f0a89d37700 1 -- 172.21.15.46:0/3994643945 shutdown_connections 2024-03-18T20:41:26.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:26.888+0000 7f0a89d37700 1 -- 172.21.15.46:0/3994643945 wait complete. 2024-03-18T20:41:26.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.954+0000 7fa8c80d4700 1 -- 172.21.15.46:0/1252026233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8c01982d0 con 0x7fa8c0105b80 2024-03-18T20:41:26.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.954+0000 7fa8b2ffd700 1 -- 172.21.15.46:0/1252026233 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10909 (secure 0 0 0) 0x7fa8b809b760 con 0x7fa8c0105b80 2024-03-18T20:41:26.963 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:41:26.963 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:39:53.094005+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:41:26.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.957+0000 7fa8c80d4700 1 -- 172.21.15.46:0/1252026233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa8ac0a1290 msgr2=0x7fa8ac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.957+0000 7fa8c80d4700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa8ac0a1290 0x7fa8ac0a3750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009fd0 tx=0x7fa8b4009260 comp rx=0 tx=0).stop 2024-03-18T20:41:26.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.957+0000 7fa8c80d4700 1 -- 172.21.15.46:0/1252026233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c0105b80 msgr2=0x7fa8c019b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:26.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.957+0000 7fa8c80d4700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c0105b80 0x7fa8c019b8b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800b070 tx=0x7fa8b8007870 comp rx=0 tx=0).stop 2024-03-18T20:41:26.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.957+0000 7fa8c80d4700 1 -- 172.21.15.46:0/1252026233 shutdown_connections 2024-03-18T20:41:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.957+0000 7fa8c80d4700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa8ac0a1290 0x7fa8ac0a3750 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.957+0000 7fa8c80d4700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c0105b80 0x7fa8c019b8b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.957+0000 7fa8c80d4700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c00ffae0 0x7fa8c0197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.957+0000 7fa8c80d4700 1 --2- 172.21.15.46:0/1252026233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8c01004a0 0x7fa8c0197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.957+0000 7fa8c80d4700 1 -- 172.21.15.46:0/1252026233 >> 172.21.15.46:0/1252026233 conn(0x7fa8c00fb5f0 msgr2=0x7fa8c00681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.957+0000 7fa8c80d4700 1 -- 172.21.15.46:0/1252026233 shutdown_connections 2024-03-18T20:41:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:26.957+0000 7fa8c80d4700 1 -- 172.21.15.46:0/1252026233 wait complete. 2024-03-18T20:41:26.965 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 39 2024-03-18T20:41:26.975 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:41:26.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:41:27.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.169+0000 7fdaeaedd700 1 -- 172.21.15.46:0/3548082868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae40ffe50 msgr2=0x7fdae4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.169+0000 7fdaeaedd700 1 --2- 172.21.15.46:0/3548082868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae40ffe50 0x7fdae4100230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009990 tx=0x7fdad8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.169+0000 7fdaeaedd700 1 -- 172.21.15.46:0/3548082868 shutdown_connections 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.169+0000 7fdaeaedd700 1 --2- 172.21.15.46:0/3548082868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae4104f60 0x7fdae410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.169+0000 7fdaeaedd700 1 --2- 172.21.15.46:0/3548082868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae40ffe50 0x7fdae4100230 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.169+0000 7fdaeaedd700 1 --2- 172.21.15.46:0/3548082868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae4100800 0x7fdae4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.169+0000 7fdaeaedd700 1 -- 172.21.15.46:0/3548082868 >> 172.21.15.46:0/3548082868 conn(0x7fdae4076d40 msgr2=0x7fdae4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.169+0000 7fdaeaedd700 1 -- 172.21.15.46:0/3548082868 shutdown_connections 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.169+0000 7fdaeaedd700 1 -- 172.21.15.46:0/3548082868 wait complete. 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.170+0000 7fdaeaedd700 1 Processor -- start 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.170+0000 7fdaeaedd700 1 -- start start 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.170+0000 7fdaeaedd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae40ffe50 0x7fdae4197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.170+0000 7fdaeaedd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae4104f60 0x7fdae41976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.170+0000 7fdaeaedd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae419ba60 0x7fdae419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.170+0000 7fdaeaedd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae419c4e0 con 0x7fdae40ffe50 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.170+0000 7fdaeaedd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae419c650 con 0x7fdae4104f60 2024-03-18T20:41:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.170+0000 7fdaeaedd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae419c810 con 0x7fdae419ba60 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.170+0000 7fdae8c79700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae40ffe50 0x7fdae4197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae8c79700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae40ffe50 0x7fdae4197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35852/0 (socket says 172.21.15.46:35852) 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae8c79700 1 -- 172.21.15.46:0/726190933 learned_addr learned my addr 172.21.15.46:0/726190933 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae3fff700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae4104f60 0x7fdae41976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae947a700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae419ba60 0x7fdae419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae8c79700 1 -- 172.21.15.46:0/726190933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae419ba60 msgr2=0x7fdae419bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae8c79700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae419ba60 0x7fdae419bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae8c79700 1 -- 172.21.15.46:0/726190933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae4104f60 msgr2=0x7fdae41976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae8c79700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae4104f60 0x7fdae41976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae8c79700 1 -- 172.21.15.46:0/726190933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad80095f0 con 0x7fdae40ffe50 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae947a700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae419ba60 0x7fdae419bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae3fff700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae4104f60 0x7fdae41976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae8c79700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae40ffe50 0x7fdae4197180 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fdad8005910 tx=0x7fdad8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae1ffb700 1 -- 172.21.15.46:0/726190933 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad801d070 con 0x7fdae40ffe50 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdaeaedd700 1 -- 172.21.15.46:0/726190933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae41a3d10 con 0x7fdae40ffe50 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae1ffb700 1 -- 172.21.15.46:0/726190933 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdad8022470 con 0x7fdae40ffe50 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.171+0000 7fdae1ffb700 1 -- 172.21.15.46:0/726190933 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad8017650 con 0x7fdae40ffe50 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.172+0000 7fdaeaedd700 1 -- 172.21.15.46:0/726190933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae41a4190 con 0x7fdae40ffe50 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.174+0000 7fdae1ffb700 1 -- 172.21.15.46:0/726190933 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fdad80225e0 con 0x7fdae40ffe50 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.175+0000 7fdae1ffb700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdad40a1290 0x7fdad40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.175+0000 7fdae1ffb700 1 -- 172.21.15.46:0/726190933 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fdad80cdfe0 con 0x7fdae40ffe50 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.175+0000 7fdae3fff700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdad40a1290 0x7fdad40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.176+0000 7fdae3fff700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdad40a1290 0x7fdad40a3750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdae41986d0 tx=0x7fdad0009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.176+0000 7fdaf7072b80 1 Processor -- start 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.176+0000 7fdaf7072b80 1 -- start start 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.176+0000 7fdaf7072b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 0x56086fbe6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.176+0000 7fdaf7072b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 0x56086fbf60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.176+0000 7fdaf7072b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbf6630 0x56086fbf8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.176+0000 7fdaf7072b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086fb40990 con 0x56086fbe7380 2024-03-18T20:41:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.176+0000 7fdaf7072b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086fb3e350 con 0x56086fbe6a60 2024-03-18T20:41:27.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.176+0000 7fdaf7072b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086fb39c40 con 0x56086fbf6630 2024-03-18T20:41:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdae3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 0x56086fbf60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdae8c79700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 0x56086fbe6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdae947a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbf6630 0x56086fbf8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdae8c79700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 0x56086fbe6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40350/0 (socket says 172.21.15.46:40350) 2024-03-18T20:41:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdae3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 0x56086fbf60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35866/0 (socket says 172.21.15.46:35866) 2024-03-18T20:41:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdae8c79700 1 -- 172.21.15.46:0/3481541296 learned_addr learned my addr 172.21.15.46:0/3481541296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdae8c79700 1 -- 172.21.15.46:0/3481541296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbf6630 msgr2=0x56086fbf8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdae8c79700 1 --2- 172.21.15.46:0/3481541296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbf6630 0x56086fbf8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdae8c79700 1 -- 172.21.15.46:0/3481541296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 msgr2=0x56086fbf60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdae8c79700 1 --2- 172.21.15.46:0/3481541296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 0x56086fbf60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdae8c79700 1 -- 172.21.15.46:0/3481541296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56086f9bb3b0 con 0x56086fbe6a60 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdae8c79700 1 --2- 172.21.15.46:0/3481541296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 0x56086fbe6e40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdad8097bc0 tx=0x7fdad8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f949a8c540abab6f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdaceffd700 1 -- 172.21.15.46:0/3481541296 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad809fea0 con 0x56086fbe6a60 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdaceffd700 1 -- 172.21.15.46:0/3481541296 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdad809b150 con 0x56086fbe6a60 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.177+0000 7fdaceffd700 1 -- 172.21.15.46:0/3481541296 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad80a1070 con 0x56086fbe6a60 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.178+0000 7fdaf7072b80 1 -- 172.21.15.46:0/3481541296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 msgr2=0x56086fbe6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.178+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/3481541296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 0x56086fbe6e40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdad8097bc0 tx=0x7fdad8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.178+0000 7fdaf7072b80 1 -- 172.21.15.46:0/3481541296 shutdown_connections 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.178+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/3481541296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbf6630 0x56086fbf8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.178+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/3481541296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 0x56086fbe6e40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.178+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/3481541296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 0x56086fbf60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.178+0000 7fdaf7072b80 1 -- 172.21.15.46:0/3481541296 >> 172.21.15.46:0/3481541296 conn(0x56086fb5d220 msgr2=0x56086fbe3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.178+0000 7fdaf7072b80 1 -- 172.21.15.46:0/3481541296 shutdown_connections 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.178+0000 7fdaf7072b80 1 -- 172.21.15.46:0/3481541296 wait complete. 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.179+0000 7fdaf7072b80 1 -- start start 2024-03-18T20:41:27.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.179+0000 7fdaf7072b80 1 Processor -- start 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.179+0000 7fdaf7072b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe6a60 0x56086fbf6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.179+0000 7fdaf7072b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe7380 0x56086fbf6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.179+0000 7fdaf7072b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 0x56086fc01a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.179+0000 7fdaf7072b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086fad5d80 con 0x56086fbf6630 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.179+0000 7fdaf7072b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086fad5a20 con 0x56086fbe7380 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.179+0000 7fdaf7072b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086f9bd6a0 con 0x56086fbe6a60 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.179+0000 7fdaf7072b80 10 client.0 _ll_register_callbacks cb 0x56086f9413a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdae3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe7380 0x56086fbf6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdae3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe7380 0x56086fbf6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40362/0 (socket says 172.21.15.46:40362) 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdae947a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 0x56086fc01a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdae8c79700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe6a60 0x56086fbf6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdae3fff700 1 -- 172.21.15.46:0/2027579803 learned_addr learned my addr 172.21.15.46:0/2027579803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdae3fff700 1 -- 172.21.15.46:0/2027579803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe6a60 msgr2=0x56086fbf6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdae3fff700 1 --2- 172.21.15.46:0/2027579803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe6a60 0x56086fbf6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdae3fff700 1 -- 172.21.15.46:0/2027579803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 msgr2=0x56086fc01a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdae3fff700 1 --2- 172.21.15.46:0/2027579803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 0x56086fc01a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdae3fff700 1 -- 172.21.15.46:0/2027579803 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae41a4190 con 0x56086fbe7380 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdae947a700 1 --2- 172.21.15.46:0/2027579803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 0x56086fc01a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdae3fff700 1 --2- 172.21.15.46:0/2027579803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe7380 0x56086fbf6f50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdad00037e0 tx=0x7fdad00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70844bee391a7874 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdac3fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdac3fff700 1 -- 172.21.15.46:0/2027579803 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad001e800 con 0x56086fbe7380 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdaf7072b80 1 -- 172.21.15.46:0/2027579803 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56086fbdaad0 con 0x56086fbe7380 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdac3fff700 1 -- 172.21.15.46:0/2027579803 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fdad001e960 con 0x56086fbe7380 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdac3fff700 1 -- 172.21.15.46:0/2027579803 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdad001eb20 con 0x56086fbe7380 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdaf7072b80 10 client.27071 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.180+0000 7fdaf7072b80 1 -- 172.21.15.46:0/2027579803 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56086f967b60 con 0x56086fbe7380 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.181+0000 7fdac3fff700 1 -- 172.21.15.46:0/2027579803 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56086fbdaad0 con 0x56086fbe7380 2024-03-18T20:41:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.181+0000 7fdac3fff700 1 -- 172.21.15.46:0/2027579803 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdad0028440 con 0x56086fbe7380 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.181+0000 7fdaf7072b80 10 client.27071 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.181+0000 7fdaf7072b80 10 client.27071 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.181+0000 7fdaf7072b80 20 client.27071 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.181+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/2027579803 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56086fc05ea0 0x56086fc082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.181+0000 7fdaf7072b80 4 client.27071 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.181+0000 7fdaf7072b80 1 -- 172.21.15.46:0/2027579803 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56086fb32700 con 0x56086fc05ea0 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.181+0000 7fdae8c79700 1 --2- 172.21.15.46:0/2027579803 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56086fc05ea0 0x56086fc082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.181+0000 7fdae8c79700 1 --2- 172.21.15.46:0/2027579803 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56086fc05ea0 0x56086fc082c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6d72b11153c6e95 server_cookie=f300878f038927bb in_seq=0 out_seq=0 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.181+0000 7fdac3fff700 10 client.27071 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdac3fff700 1 -- 172.21.15.46:0/2027579803 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56086fb32700 con 0x56086fc05ea0 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdac3fff700 10 client.27071 handle_command_reply: tid=0 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdaf7072b80 1 client.27071 shutdown 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdaf7072b80 1 -- 172.21.15.46:0/2027579803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe7380 msgr2=0x56086fbf6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/2027579803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe7380 0x56086fbf6f50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdad00037e0 tx=0x7fdad00047c0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdaf7072b80 1 -- 172.21.15.46:0/2027579803 shutdown_connections 2024-03-18T20:41:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/2027579803 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56086fc05ea0 0x56086fc082c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/2027579803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 0x56086fc01a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdac3fff700 0 client.27071 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:41:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/2027579803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe6a60 0x56086fbf6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/2027579803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe7380 0x56086fbf6f50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdaf7072b80 1 -- 172.21.15.46:0/2027579803 >> 172.21.15.46:0/2027579803 conn(0x56086fb5d220 msgr2=0x56086fbe2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdaf7072b80 1 -- 172.21.15.46:0/2027579803 shutdown_connections 2024-03-18T20:41:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.182+0000 7fdaf7072b80 1 -- 172.21.15.46:0/2027579803 wait complete. 2024-03-18T20:41:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.183+0000 7fdaf7072b80 20 client.27071 trim_cache size 0 max 16384 2024-03-18T20:41:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.197+0000 7fdaf7072b80 1 Processor -- start 2024-03-18T20:41:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdaf7072b80 1 -- start start 2024-03-18T20:41:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdaf7072b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe6a60 0x56086fbf5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdaf7072b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 0x56086fbfba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdaf7072b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbf6630 0x56086fbf2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdaf7072b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086fbf7750 con 0x56086fbe7380 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdaf7072b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086fbdd210 con 0x56086fbf6630 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdaf7072b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086fbe4440 con 0x56086fbe6a60 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdae3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 0x56086fbfba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdae8c79700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe6a60 0x56086fbf5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdae3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 0x56086fbfba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35894/0 (socket says 172.21.15.46:35894) 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdae8c79700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe6a60 0x56086fbf5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:45684/0 (socket says 172.21.15.46:45684) 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdae3fff700 1 -- 172.21.15.46:0/3904724379 learned_addr learned my addr 172.21.15.46:0/3904724379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdae947a700 1 --2- 172.21.15.46:0/3904724379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbf6630 0x56086fbf2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdae8c79700 1 -- 172.21.15.46:0/3904724379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbf6630 msgr2=0x56086fbf2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdae8c79700 1 --2- 172.21.15.46:0/3904724379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbf6630 0x56086fbf2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdae8c79700 1 -- 172.21.15.46:0/3904724379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 msgr2=0x56086fbfba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdae8c79700 1 --2- 172.21.15.46:0/3904724379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 0x56086fbfba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdae8c79700 1 -- 172.21.15.46:0/3904724379 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56086faffea0 con 0x56086fbe6a60 2024-03-18T20:41:27.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.198+0000 7fdae3fff700 1 --2- 172.21.15.46:0/3904724379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 0x56086fbfba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.199+0000 7fdae947a700 1 --2- 172.21.15.46:0/3904724379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbf6630 0x56086fbf2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.215+0000 7fdae8c79700 1 --2- 172.21.15.46:0/3904724379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe6a60 0x56086fbf5220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdad802e000 tx=0x7fdad80aef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0aefbd8bd77ec37 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.215+0000 7fdacf7fe700 1 -- 172.21.15.46:0/3904724379 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad80a3540 con 0x56086fbe6a60 2024-03-18T20:41:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.215+0000 7fdacf7fe700 1 -- 172.21.15.46:0/3904724379 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdad809ef80 con 0x56086fbe6a60 2024-03-18T20:41:27.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.215+0000 7fdacf7fe700 1 -- 172.21.15.46:0/3904724379 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad80a1260 con 0x56086fbe6a60 2024-03-18T20:41:27.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.218+0000 7fdaf7072b80 1 -- 172.21.15.46:0/3904724379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe6a60 msgr2=0x56086fbf5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.218+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/3904724379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe6a60 0x56086fbf5220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdad802e000 tx=0x7fdad80aef90 comp rx=0 tx=0).stop 2024-03-18T20:41:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.219+0000 7fdaf7072b80 1 -- 172.21.15.46:0/3904724379 shutdown_connections 2024-03-18T20:41:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.219+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/3904724379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbf6630 0x56086fbf2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.219+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/3904724379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe6a60 0x56086fbf5220 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.219+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/3904724379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbe7380 0x56086fbfba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.219+0000 7fdaf7072b80 1 -- 172.21.15.46:0/3904724379 >> 172.21.15.46:0/3904724379 conn(0x56086fb5d220 msgr2=0x56086fbf00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.219+0000 7fdaf7072b80 1 -- 172.21.15.46:0/3904724379 shutdown_connections 2024-03-18T20:41:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.219+0000 7fdaf7072b80 1 -- 172.21.15.46:0/3904724379 wait complete. 2024-03-18T20:41:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.220+0000 7fdaf7072b80 1 -- start start 2024-03-18T20:41:27.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.220+0000 7fdaf7072b80 1 Processor -- start 2024-03-18T20:41:27.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.220+0000 7fdaf7072b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 0x56086fc4dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.220+0000 7fdaf7072b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe7380 0x56086fc4e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.220+0000 7fdaf7072b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 0x56086fc4e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.220+0000 7fdaf7072b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086fc58ce0 con 0x56086fbf6630 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.220+0000 7fdaf7072b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086fc58e20 con 0x56086fbe6a60 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.220+0000 7fdaf7072b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086fc58f60 con 0x56086fbe7380 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.220+0000 7fdaf7072b80 10 client.0 _ll_register_callbacks cb 0x7fdad0004660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.221+0000 7fdae8c79700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 0x56086fc4dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.221+0000 7fdae947a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 0x56086fc4e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.221+0000 7fdae947a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 0x56086fc4e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35896/0 (socket says 172.21.15.46:35896) 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.221+0000 7fdae947a700 1 -- 172.21.15.46:0/831958530 learned_addr learned my addr 172.21.15.46:0/831958530 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.221+0000 7fdae947a700 1 -- 172.21.15.46:0/831958530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe7380 msgr2=0x56086fc4e1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.221+0000 7fdae947a700 1 --2- 172.21.15.46:0/831958530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe7380 0x56086fc4e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.221+0000 7fdae947a700 1 -- 172.21.15.46:0/831958530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 msgr2=0x56086fc4dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.221+0000 7fdae947a700 1 --2- 172.21.15.46:0/831958530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 0x56086fc4dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.221+0000 7fdae947a700 1 -- 172.21.15.46:0/831958530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56086f967b60 con 0x56086fbf6630 2024-03-18T20:41:27.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.221+0000 7fdae8c79700 1 --2- 172.21.15.46:0/831958530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 0x56086fc4dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.222+0000 7fdae947a700 1 --2- 172.21.15.46:0/831958530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 0x56086fc4e6e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0186a0 tx=0x7fdadc0186d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d489d667fa0987 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.222+0000 7fdac3fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.222+0000 7fdac3fff700 1 -- 172.21.15.46:0/831958530 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc00e040 con 0x56086fbf6630 2024-03-18T20:41:27.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.222+0000 7fdac3fff700 1 -- 172.21.15.46:0/831958530 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fdadc003c00 con 0x56086fbf6630 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.222+0000 7fdaf7072b80 1 -- 172.21.15.46:0/831958530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdad809b150 con 0x56086fbf6630 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.222+0000 7fdac3fff700 1 -- 172.21.15.46:0/831958530 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdadc003dc0 con 0x56086fbf6630 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.222+0000 7fdaf7072b80 10 client.17385 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.222+0000 7fdaf7072b80 1 -- 172.21.15.46:0/831958530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fdadc007690 con 0x56086fbf6630 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.223+0000 7fdac3fff700 1 -- 172.21.15.46:0/831958530 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdadc02c080 con 0x56086fbf6630 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.223+0000 7fdac3fff700 1 -- 172.21.15.46:0/831958530 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdadc004340 con 0x56086fbf6630 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.223+0000 7fdaf7072b80 10 client.17385 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.223+0000 7fdaf7072b80 10 client.17385 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.223+0000 7fdaf7072b80 20 client.17385 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.223+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/831958530 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56086fbeb350 0x56086fbed770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.223+0000 7fdaf7072b80 4 client.17385 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.223+0000 7fdaf7072b80 1 -- 172.21.15.46:0/831958530 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56086f988cf0 con 0x56086fbeb350 2024-03-18T20:41:27.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.223+0000 7fdae8c79700 1 --2- 172.21.15.46:0/831958530 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56086fbeb350 0x56086fbed770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.223+0000 7fdac3fff700 10 client.17385 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdae8c79700 1 --2- 172.21.15.46:0/831958530 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56086fbeb350 0x56086fbed770 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9690fcaab736f06b server_cookie=a94f4b72848a2175 in_seq=0 out_seq=0 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdac3fff700 1 -- 172.21.15.46:0/831958530 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56086f988cf0 con 0x56086fbeb350 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdac3fff700 10 client.17385 handle_command_reply: tid=0 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdaf7072b80 1 client.17385 shutdown 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdaf7072b80 1 -- 172.21.15.46:0/831958530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 msgr2=0x56086fc4e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/831958530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 0x56086fc4e6e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0186a0 tx=0x7fdadc0186d0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdaf7072b80 1 -- 172.21.15.46:0/831958530 shutdown_connections 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/831958530 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56086fbeb350 0x56086fbed770 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/831958530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56086fbf6630 0x56086fc4e6e0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/831958530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56086fbe6a60 0x56086fc4dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdac3fff700 0 client.17385 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:41:27.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdaf7072b80 1 --2- 172.21.15.46:0/831958530 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56086fbe7380 0x56086fc4e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdaf7072b80 1 -- 172.21.15.46:0/831958530 >> 172.21.15.46:0/831958530 conn(0x56086fb5d220 msgr2=0x56086fbe8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.224+0000 7fdaf7072b80 1 -- 172.21.15.46:0/831958530 shutdown_connections 2024-03-18T20:41:27.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaf7072b80 1 -- 172.21.15.46:0/831958530 wait complete. 2024-03-18T20:41:27.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaf7072b80 20 client.17385 trim_cache size 0 max 16384 2024-03-18T20:41:27.231 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:41:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaeaedd700 1 -- 172.21.15.46:0/726190933 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdad40a1290 msgr2=0x7fdad40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaeaedd700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdad40a1290 0x7fdad40a3750 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdae41986d0 tx=0x7fdad0009260 comp rx=0 tx=0).stop 2024-03-18T20:41:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaeaedd700 1 -- 172.21.15.46:0/726190933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae40ffe50 msgr2=0x7fdae4197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaeaedd700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae40ffe50 0x7fdae4197180 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fdad8005910 tx=0x7fdad8004730 comp rx=0 tx=0).stop 2024-03-18T20:41:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaeaedd700 1 -- 172.21.15.46:0/726190933 shutdown_connections 2024-03-18T20:41:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaeaedd700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdad40a1290 0x7fdad40a3750 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaeaedd700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdae40ffe50 0x7fdae4197180 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaeaedd700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdae419ba60 0x7fdae419bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaeaedd700 1 --2- 172.21.15.46:0/726190933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae4104f60 0x7fdae41976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaeaedd700 1 -- 172.21.15.46:0/726190933 >> 172.21.15.46:0/726190933 conn(0x7fdae4076d40 msgr2=0x7fdae4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.225+0000 7fdaeaedd700 1 -- 172.21.15.46:0/726190933 shutdown_connections 2024-03-18T20:41:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.226+0000 7fdaeaedd700 1 -- 172.21.15.46:0/726190933 wait complete. 2024-03-18T20:41:27.242 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:27 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1252026233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:41:27.243 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:41:27.243 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:41:27.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:27 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1252026233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:41:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.386+0000 7fb34b22f700 1 -- 172.21.15.46:0/3280522632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3440ffe70 msgr2=0x7fb344100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.386+0000 7fb34b22f700 1 --2- 172.21.15.46:0/3280522632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3440ffe70 0x7fb344100250 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb334009990 tx=0x7fb334009ca0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.386+0000 7fb34b22f700 1 -- 172.21.15.46:0/3280522632 shutdown_connections 2024-03-18T20:41:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.386+0000 7fb34b22f700 1 --2- 172.21.15.46:0/3280522632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3440ffe70 0x7fb344100250 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.386+0000 7fb34b22f700 1 --2- 172.21.15.46:0/3280522632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb344104f90 0x7fb34410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.386+0000 7fb34b22f700 1 --2- 172.21.15.46:0/3280522632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb344100820 0x7fb344104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.386+0000 7fb34b22f700 1 -- 172.21.15.46:0/3280522632 >> 172.21.15.46:0/3280522632 conn(0x7fb344076d80 msgr2=0x7fb344077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.386+0000 7fb34b22f700 1 -- 172.21.15.46:0/3280522632 shutdown_connections 2024-03-18T20:41:27.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.386+0000 7fb34b22f700 1 -- 172.21.15.46:0/3280522632 wait complete. 2024-03-18T20:41:27.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.387+0000 7fb34b22f700 1 Processor -- start 2024-03-18T20:41:27.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.387+0000 7fb34b22f700 1 -- start start 2024-03-18T20:41:27.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.387+0000 7fb34b22f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3440ffe70 0x7fb344101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.387+0000 7fb34b22f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb344104f90 0x7fb344101e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.387+0000 7fb34b22f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb344102380 0x7fb3441a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.387+0000 7fb34b22f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3441a5990 con 0x7fb344104f90 2024-03-18T20:41:27.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.387+0000 7fb34b22f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3441a5b00 con 0x7fb3440ffe70 2024-03-18T20:41:27.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.387+0000 7fb34b22f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3441a5cc0 con 0x7fb344102380 2024-03-18T20:41:27.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb343fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb344104f90 0x7fb344101e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb348fcb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3440ffe70 0x7fb344101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb348fcb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3440ffe70 0x7fb344101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40396/0 (socket says 172.21.15.46:40396) 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb343fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb344104f90 0x7fb344101e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35908/0 (socket says 172.21.15.46:35908) 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb348fcb700 1 -- 172.21.15.46:0/2643065463 learned_addr learned my addr 172.21.15.46:0/2643065463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb3497cc700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb344102380 0x7fb3441a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb348fcb700 1 -- 172.21.15.46:0/2643065463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb344102380 msgr2=0x7fb3441a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb348fcb700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb344102380 0x7fb3441a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb348fcb700 1 -- 172.21.15.46:0/2643065463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb344104f90 msgr2=0x7fb344101e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb348fcb700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb344104f90 0x7fb344101e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb348fcb700 1 -- 172.21.15.46:0/2643065463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3340095f0 con 0x7fb3440ffe70 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb343fff700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb344104f90 0x7fb344101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb348fcb700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3440ffe70 0x7fb344101900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb334004760 tx=0x7fb334004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb341ffb700 1 -- 172.21.15.46:0/2643065463 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33401d070 con 0x7fb3440ffe70 2024-03-18T20:41:27.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb34b22f700 1 -- 172.21.15.46:0/2643065463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3441a5f40 con 0x7fb3440ffe70 2024-03-18T20:41:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb341ffb700 1 -- 172.21.15.46:0/2643065463 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb334022470 con 0x7fb3440ffe70 2024-03-18T20:41:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.388+0000 7fb341ffb700 1 -- 172.21.15.46:0/2643065463 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334017650 con 0x7fb3440ffe70 2024-03-18T20:41:27.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.389+0000 7fb34b22f700 1 -- 172.21.15.46:0/2643065463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3441a6360 con 0x7fb3440ffe70 2024-03-18T20:41:27.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.391+0000 7fb341ffb700 1 -- 172.21.15.46:0/2643065463 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fb3340225e0 con 0x7fb3440ffe70 2024-03-18T20:41:27.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.392+0000 7fb341ffb700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb32c0a1290 0x7fb32c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.392+0000 7fb343fff700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb32c0a1290 0x7fb32c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.392+0000 7fb341ffb700 1 -- 172.21.15.46:0/2643065463 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb3340cde10 con 0x7fb3440ffe70 2024-03-18T20:41:27.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.392+0000 7fb343fff700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb32c0a1290 0x7fb32c0a3750 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb344103030 tx=0x7fb33800b040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb3573c4b80 1 Processor -- start 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb3573c4b80 1 -- start start 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb3573c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 0x55aa3d0f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb3573c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d0f9380 0x55aa3d103ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb3573c4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d104220 0x55aa3d106610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb3573c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d052990 con 0x55aa3d0f8a60 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb3573c4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d050350 con 0x55aa3d104220 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb3573c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d04bc40 con 0x55aa3d0f9380 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb348fcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 0x55aa3d0f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb348fcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 0x55aa3d0f8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35912/0 (socket says 172.21.15.46:35912) 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb348fcb700 1 -- 172.21.15.46:0/2468519293 learned_addr learned my addr 172.21.15.46:0/2468519293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb3497cc700 1 --2- 172.21.15.46:0/2468519293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d104220 0x55aa3d106610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb343fff700 1 --2- 172.21.15.46:0/2468519293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d0f9380 0x55aa3d103ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb348fcb700 1 -- 172.21.15.46:0/2468519293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d0f9380 msgr2=0x55aa3d103ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb348fcb700 1 --2- 172.21.15.46:0/2468519293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d0f9380 0x55aa3d103ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb348fcb700 1 -- 172.21.15.46:0/2468519293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d104220 msgr2=0x55aa3d106610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb348fcb700 1 --2- 172.21.15.46:0/2468519293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d104220 0x55aa3d106610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb348fcb700 1 -- 172.21.15.46:0/2468519293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa3cecd3b0 con 0x55aa3d0f8a60 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.393+0000 7fb3497cc700 1 --2- 172.21.15.46:0/2468519293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d104220 0x55aa3d106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.394+0000 7fb348fcb700 1 --2- 172.21.15.46:0/2468519293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 0x55aa3d0f8e40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb334006010 tx=0x7fb33409aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc3c54b9c8db0ad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.394+0000 7fb32affd700 1 -- 172.21.15.46:0/2468519293 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3340a1070 con 0x55aa3d0f8a60 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.394+0000 7fb32affd700 1 -- 172.21.15.46:0/2468519293 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb33409b680 con 0x55aa3d0f8a60 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.394+0000 7fb32affd700 1 -- 172.21.15.46:0/2468519293 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33409b400 con 0x55aa3d0f8a60 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.394+0000 7fb3573c4b80 1 -- 172.21.15.46:0/2468519293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 msgr2=0x55aa3d0f8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.394+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/2468519293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 0x55aa3d0f8e40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb334006010 tx=0x7fb33409aac0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.395+0000 7fb3573c4b80 1 -- 172.21.15.46:0/2468519293 shutdown_connections 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.395+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/2468519293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 0x55aa3d0f8e40 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.395+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/2468519293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d104220 0x55aa3d106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.395+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/2468519293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d0f9380 0x55aa3d103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.395+0000 7fb3573c4b80 1 -- 172.21.15.46:0/2468519293 >> 172.21.15.46:0/2468519293 conn(0x55aa3d06f220 msgr2=0x55aa3d0f5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.395+0000 7fb3573c4b80 1 -- 172.21.15.46:0/2468519293 shutdown_connections 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.395+0000 7fb3573c4b80 1 -- 172.21.15.46:0/2468519293 wait complete. 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.395+0000 7fb3573c4b80 1 -- start start 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.395+0000 7fb3573c4b80 1 Processor -- start 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3573c4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f8a60 0x55aa3d104600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3573c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d0f9380 0x55aa3d104b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3573c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d104220 0x55aa3d10f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3573c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3cfe7d80 con 0x55aa3d104220 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3573c4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3cfe7a20 con 0x55aa3d0f8a60 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3573c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3cecf6a0 con 0x55aa3d0f9380 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3573c4b80 10 client.0 _ll_register_callbacks cb 0x55aa3ce533a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3497cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d104220 0x55aa3d10f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb343fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d0f9380 0x55aa3d104b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3497cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d104220 0x55aa3d10f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35924/0 (socket says 172.21.15.46:35924) 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3497cc700 1 -- 172.21.15.46:0/902505959 learned_addr learned my addr 172.21.15.46:0/902505959 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb348fcb700 1 --2- 172.21.15.46:0/902505959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f8a60 0x55aa3d104600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3497cc700 1 -- 172.21.15.46:0/902505959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d0f9380 msgr2=0x55aa3d104b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3497cc700 1 --2- 172.21.15.46:0/902505959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d0f9380 0x55aa3d104b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3497cc700 1 -- 172.21.15.46:0/902505959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f8a60 msgr2=0x55aa3d104600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3497cc700 1 --2- 172.21.15.46:0/902505959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f8a60 0x55aa3d104600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3497cc700 1 -- 172.21.15.46:0/902505959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c0098c0 con 0x55aa3d104220 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb3497cc700 1 --2- 172.21.15.46:0/902505959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d104220 0x55aa3d10f680 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb33c012990 tx=0x7fb33c012ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485ddedaf42dcd22 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.396+0000 7fb323fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb323fff700 1 -- 172.21.15.46:0/902505959 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c017070 con 0x55aa3d104220 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb3573c4b80 1 -- 172.21.15.46:0/902505959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa3d0ecad0 con 0x55aa3d104220 2024-03-18T20:41:27.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb323fff700 1 -- 172.21.15.46:0/902505959 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb33c00f5f0 con 0x55aa3d104220 2024-03-18T20:41:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb323fff700 1 -- 172.21.15.46:0/902505959 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb33c00f7b0 con 0x55aa3d104220 2024-03-18T20:41:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb3573c4b80 10 client.17397 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb3573c4b80 1 -- 172.21.15.46:0/902505959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fb3441a6360 con 0x55aa3d104220 2024-03-18T20:41:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb323fff700 1 -- 172.21.15.46:0/902505959 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb33c028080 con 0x55aa3d104220 2024-03-18T20:41:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb323fff700 1 -- 172.21.15.46:0/902505959 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb33c010920 con 0x55aa3d104220 2024-03-18T20:41:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb3573c4b80 10 client.17397 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb3573c4b80 10 client.17397 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:41:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb3573c4b80 20 client.17397 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/902505959 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55aa3d113b90 0x55aa3d115fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb3573c4b80 4 client.17397 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb3573c4b80 1 -- 172.21.15.46:0/902505959 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa3d044700 con 0x55aa3d113b90 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.397+0000 7fb348fcb700 1 --2- 172.21.15.46:0/902505959 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55aa3d113b90 0x55aa3d115fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.398+0000 7fb348fcb700 1 --2- 172.21.15.46:0/902505959 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55aa3d113b90 0x55aa3d115fb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e0b9f3741d58742e server_cookie=909ff29f2765deda in_seq=0 out_seq=0 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.398+0000 7fb323fff700 10 client.17397 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.398+0000 7fb323fff700 1 -- 172.21.15.46:0/902505959 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aa3d044700 con 0x55aa3d113b90 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.398+0000 7fb323fff700 10 client.17397 handle_command_reply: tid=0 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.398+0000 7fb3573c4b80 1 client.17397 shutdown 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.398+0000 7fb3573c4b80 1 -- 172.21.15.46:0/902505959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d104220 msgr2=0x55aa3d10f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.398+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/902505959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d104220 0x55aa3d10f680 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb33c012990 tx=0x7fb33c012ca0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.399+0000 7fb3573c4b80 1 -- 172.21.15.46:0/902505959 shutdown_connections 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.399+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/902505959 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55aa3d113b90 0x55aa3d115fb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.399+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/902505959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d104220 0x55aa3d10f680 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.399+0000 7fb323fff700 0 client.17397 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.399+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/902505959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f8a60 0x55aa3d104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.399+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/902505959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d0f9380 0x55aa3d104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.399+0000 7fb3573c4b80 1 -- 172.21.15.46:0/902505959 >> 172.21.15.46:0/902505959 conn(0x55aa3d06f220 msgr2=0x55aa3d0f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.399+0000 7fb3573c4b80 1 -- 172.21.15.46:0/902505959 shutdown_connections 2024-03-18T20:41:27.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.399+0000 7fb3573c4b80 1 -- 172.21.15.46:0/902505959 wait complete. 2024-03-18T20:41:27.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.399+0000 7fb3573c4b80 20 client.17397 trim_cache size 0 max 16384 2024-03-18T20:41:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb3573c4b80 1 Processor -- start 2024-03-18T20:41:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb3573c4b80 1 -- start start 2024-03-18T20:41:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb3573c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 0x55aa3d1095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb3573c4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f9380 0x55aa3d0fe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb3573c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d104220 0x55aa3d0fed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb3573c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d0f9d30 con 0x55aa3d0f8a60 2024-03-18T20:41:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb3573c4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d0f15e0 con 0x55aa3d0f9380 2024-03-18T20:41:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb3573c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d1032e0 con 0x55aa3d104220 2024-03-18T20:41:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb348fcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 0x55aa3d1095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb348fcb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 0x55aa3d1095e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35928/0 (socket says 172.21.15.46:35928) 2024-03-18T20:41:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb343fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f9380 0x55aa3d0fe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb348fcb700 1 -- 172.21.15.46:0/3053553831 learned_addr learned my addr 172.21.15.46:0/3053553831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb3497cc700 1 --2- 172.21.15.46:0/3053553831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d104220 0x55aa3d0fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb348fcb700 1 -- 172.21.15.46:0/3053553831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d104220 msgr2=0x55aa3d0fed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb348fcb700 1 --2- 172.21.15.46:0/3053553831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d104220 0x55aa3d0fed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb348fcb700 1 -- 172.21.15.46:0/3053553831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f9380 msgr2=0x55aa3d0fe810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb348fcb700 1 --2- 172.21.15.46:0/3053553831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f9380 0x55aa3d0fe810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.413+0000 7fb348fcb700 1 -- 172.21.15.46:0/3053553831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa3ce79b60 con 0x55aa3d0f8a60 2024-03-18T20:41:27.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.414+0000 7fb348fcb700 1 --2- 172.21.15.46:0/3053553831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 0x55aa3d1095e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb334022c60 tx=0x7fb33409b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa963fd632409f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.414+0000 7fb32b7fe700 1 -- 172.21.15.46:0/3053553831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33409edb0 con 0x55aa3d0f8a60 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.414+0000 7fb32b7fe700 1 -- 172.21.15.46:0/3053553831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3340228e0 con 0x55aa3d0f8a60 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.414+0000 7fb32b7fe700 1 -- 172.21.15.46:0/3053553831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33409b2d0 con 0x55aa3d0f8a60 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.415+0000 7fb3573c4b80 1 -- 172.21.15.46:0/3053553831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 msgr2=0x55aa3d1095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.415+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/3053553831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 0x55aa3d1095e0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb334022c60 tx=0x7fb33409b7f0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.415+0000 7fb3573c4b80 1 -- 172.21.15.46:0/3053553831 shutdown_connections 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.415+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/3053553831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f8a60 0x55aa3d1095e0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.415+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/3053553831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d104220 0x55aa3d0fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.415+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/3053553831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f9380 0x55aa3d0fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.415+0000 7fb3573c4b80 1 -- 172.21.15.46:0/3053553831 >> 172.21.15.46:0/3053553831 conn(0x55aa3d06f220 msgr2=0x55aa3d104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.415+0000 7fb3573c4b80 1 -- 172.21.15.46:0/3053553831 shutdown_connections 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.415+0000 7fb3573c4b80 1 -- 172.21.15.46:0/3053553831 wait complete. 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.415+0000 7fb3573c4b80 1 -- start start 2024-03-18T20:41:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.415+0000 7fb3573c4b80 1 Processor -- start 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb3573c4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f8a60 0x55aa3d15bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb3573c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f9380 0x55aa3d15c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb3573c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d104220 0x55aa3d1609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb3573c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d160f40 con 0x55aa3d0f9380 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb3573c4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d1610b0 con 0x55aa3d0f8a60 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb3573c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3d161270 con 0x55aa3d104220 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb3573c4b80 10 client.0 _ll_register_callbacks cb 0x7fb33c0124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb348fcb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f8a60 0x55aa3d15bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb343fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f9380 0x55aa3d15c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb348fcb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f8a60 0x55aa3d15bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40442/0 (socket says 172.21.15.46:40442) 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb343fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f9380 0x55aa3d15c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35938/0 (socket says 172.21.15.46:35938) 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb348fcb700 1 -- 172.21.15.46:0/4232209754 learned_addr learned my addr 172.21.15.46:0/4232209754 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb3497cc700 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d104220 0x55aa3d1609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb348fcb700 1 -- 172.21.15.46:0/4232209754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d104220 msgr2=0x55aa3d1609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb348fcb700 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d104220 0x55aa3d1609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb348fcb700 1 -- 172.21.15.46:0/4232209754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f9380 msgr2=0x55aa3d15c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb348fcb700 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f9380 0x55aa3d15c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb348fcb700 1 -- 172.21.15.46:0/4232209754 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa3cecd3b0 con 0x55aa3d0f8a60 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb343fff700 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f9380 0x55aa3d15c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb348fcb700 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f8a60 0x55aa3d15bb00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb334024000 tx=0x7fb334022cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d07fe4deb7196a2c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb323fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb323fff700 1 -- 172.21.15.46:0/4232209754 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33409a670 con 0x55aa3d0f8a60 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb323fff700 1 -- 172.21.15.46:0/4232209754 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb33409b1e0 con 0x55aa3d0f8a60 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.416+0000 7fb3573c4b80 1 -- 172.21.15.46:0/4232209754 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb33409b680 con 0x55aa3d0f8a60 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.417+0000 7fb323fff700 1 -- 172.21.15.46:0/4232209754 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3340a10d0 con 0x55aa3d0f8a60 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.417+0000 7fb3573c4b80 10 client.27095 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.417+0000 7fb3573c4b80 1 -- 172.21.15.46:0/4232209754 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55aa3d011ea0 con 0x55aa3d0f8a60 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.417+0000 7fb323fff700 1 -- 172.21.15.46:0/4232209754 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb33409b680 con 0x55aa3d0f8a60 2024-03-18T20:41:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.417+0000 7fb323fff700 1 -- 172.21.15.46:0/4232209754 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb334095b90 con 0x55aa3d0f8a60 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.417+0000 7fb3573c4b80 10 client.27095 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.417+0000 7fb3573c4b80 10 client.27095 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.417+0000 7fb3573c4b80 20 client.27095 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.417+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55aa3d168630 0x55aa3d16aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.417+0000 7fb3573c4b80 4 client.27095 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.417+0000 7fb3573c4b80 1 -- 172.21.15.46:0/4232209754 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aa3ce9acf0 con 0x55aa3d168630 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.417+0000 7fb343fff700 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55aa3d168630 0x55aa3d16aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb323fff700 10 client.27095 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb343fff700 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55aa3d168630 0x55aa3d16aa20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5bbdf42938db3533 server_cookie=de0ad7ab3b828e53 in_seq=0 out_seq=0 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb323fff700 1 -- 172.21.15.46:0/4232209754 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aa3ce9acf0 con 0x55aa3d168630 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb323fff700 10 client.27095 handle_command_reply: tid=0 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb3573c4b80 1 client.27095 shutdown 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb3573c4b80 1 -- 172.21.15.46:0/4232209754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f8a60 msgr2=0x55aa3d15bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f8a60 0x55aa3d15bb00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb334024000 tx=0x7fb334022cb0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb3573c4b80 1 -- 172.21.15.46:0/4232209754 shutdown_connections 2024-03-18T20:41:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa3d104220 0x55aa3d1609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa3d0f8a60 0x55aa3d15bb00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55aa3d168630 0x55aa3d16aa20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb3573c4b80 1 --2- 172.21.15.46:0/4232209754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa3d0f9380 0x55aa3d15c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb323fff700 0 client.27095 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:41:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb3573c4b80 1 -- 172.21.15.46:0/4232209754 >> 172.21.15.46:0/4232209754 conn(0x55aa3d06f220 msgr2=0x55aa3d0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb3573c4b80 1 -- 172.21.15.46:0/4232209754 shutdown_connections 2024-03-18T20:41:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb3573c4b80 1 -- 172.21.15.46:0/4232209754 wait complete. 2024-03-18T20:41:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.418+0000 7fb3573c4b80 20 client.27095 trim_cache size 0 max 16384 2024-03-18T20:41:27.427 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:41:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.419+0000 7fb34b22f700 1 -- 172.21.15.46:0/2643065463 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb32c0a1290 msgr2=0x7fb32c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.419+0000 7fb34b22f700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb32c0a1290 0x7fb32c0a3750 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb344103030 tx=0x7fb33800b040 comp rx=0 tx=0).stop 2024-03-18T20:41:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.419+0000 7fb34b22f700 1 -- 172.21.15.46:0/2643065463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3440ffe70 msgr2=0x7fb344101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.419+0000 7fb34b22f700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3440ffe70 0x7fb344101900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb334004760 tx=0x7fb334004790 comp rx=0 tx=0).stop 2024-03-18T20:41:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.419+0000 7fb34b22f700 1 -- 172.21.15.46:0/2643065463 shutdown_connections 2024-03-18T20:41:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.419+0000 7fb34b22f700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb32c0a1290 0x7fb32c0a3750 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.419+0000 7fb34b22f700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb344102380 0x7fb3441a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.419+0000 7fb34b22f700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3440ffe70 0x7fb344101900 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.419+0000 7fb34b22f700 1 --2- 172.21.15.46:0/2643065463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb344104f90 0x7fb344101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.419+0000 7fb34b22f700 1 -- 172.21.15.46:0/2643065463 >> 172.21.15.46:0/2643065463 conn(0x7fb344076d80 msgr2=0x7fb344103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.419+0000 7fb34b22f700 1 -- 172.21.15.46:0/2643065463 shutdown_connections 2024-03-18T20:41:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.419+0000 7fb34b22f700 1 -- 172.21.15.46:0/2643065463 wait complete. 2024-03-18T20:41:27.435 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:41:27.436 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:41:27.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:27 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1252026233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:41:27.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.575+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1522582108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48810d700 msgr2=0x7fd48810faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.575+0000 7fd48d6cb700 1 --2- 172.21.15.46:0/1522582108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48810d700 0x7fd48810faf0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd48000b0a0 tx=0x7fd48000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.576+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1522582108 shutdown_connections 2024-03-18T20:41:27.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.576+0000 7fd48d6cb700 1 --2- 172.21.15.46:0/1522582108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48810d700 0x7fd48810faf0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.576+0000 7fd48d6cb700 1 --2- 172.21.15.46:0/1522582108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd488069380 0x7fd488069760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.576+0000 7fd48d6cb700 1 --2- 172.21.15.46:0/1522582108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd488069ca0 0x7fd48810d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.576+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1522582108 >> 172.21.15.46:0/1522582108 conn(0x7fd488076db0 msgr2=0x7fd4880771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.576+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1522582108 shutdown_connections 2024-03-18T20:41:27.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.576+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1522582108 wait complete. 2024-03-18T20:41:27.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.576+0000 7fd48d6cb700 1 Processor -- start 2024-03-18T20:41:27.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd48d6cb700 1 -- start start 2024-03-18T20:41:27.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd48d6cb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd488069380 0x7fd4881994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd48d6cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd488069ca0 0x7fd488199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd48d6cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48810d700 0x7fd48819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd48d6cb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48819e410 con 0x7fd48810d700 2024-03-18T20:41:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd48d6cb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48819e580 con 0x7fd488069ca0 2024-03-18T20:41:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd48d6cb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48819e740 con 0x7fd488069380 2024-03-18T20:41:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd4877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48810d700 0x7fd48819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd4867fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd488069ca0 0x7fd488199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd486ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd488069380 0x7fd4881994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd4867fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd488069ca0 0x7fd488199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40460/0 (socket says 172.21.15.46:40460) 2024-03-18T20:41:27.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd4877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48810d700 0x7fd48819ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35958/0 (socket says 172.21.15.46:35958) 2024-03-18T20:41:27.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd4867fc700 1 -- 172.21.15.46:0/1085995953 learned_addr learned my addr 172.21.15.46:0/1085995953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd486ffd700 1 -- 172.21.15.46:0/1085995953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd488069ca0 msgr2=0x7fd488199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd486ffd700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd488069ca0 0x7fd488199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd486ffd700 1 -- 172.21.15.46:0/1085995953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48810d700 msgr2=0x7fd48819ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd486ffd700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48810d700 0x7fd48819ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd486ffd700 1 -- 172.21.15.46:0/1085995953 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c009520 con 0x7fd488069380 2024-03-18T20:41:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd4877fe700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48810d700 0x7fd48819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.577+0000 7fd4867fc700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd488069ca0 0x7fd488199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.578+0000 7fd486ffd700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd488069380 0x7fd4881994f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd47800e9f0 tx=0x7fd47800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.578+0000 7fd46ffff700 1 -- 172.21.15.46:0/1085995953 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47800c9c0 con 0x7fd488069380 2024-03-18T20:41:27.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.578+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1085995953 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd480009d00 con 0x7fd488069380 2024-03-18T20:41:27.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.578+0000 7fd46ffff700 1 -- 172.21.15.46:0/1085995953 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd478010430 con 0x7fd488069380 2024-03-18T20:41:27.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.578+0000 7fd46ffff700 1 -- 172.21.15.46:0/1085995953 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478018680 con 0x7fd488069380 2024-03-18T20:41:27.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.578+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1085995953 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4881a6080 con 0x7fd488069380 2024-03-18T20:41:27.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.580+0000 7fd46ffff700 1 -- 172.21.15.46:0/1085995953 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd478010860 con 0x7fd488069380 2024-03-18T20:41:27.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.581+0000 7fd46ffff700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd4700a1290 0x7fd4700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.581+0000 7fd4867fc700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd4700a1290 0x7fd4700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.581+0000 7fd46ffff700 1 -- 172.21.15.46:0/1085995953 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd478014070 con 0x7fd488069380 2024-03-18T20:41:27.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd4867fc700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd4700a1290 0x7fd4700a3750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd48819aa40 tx=0x7fd47c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd499860b80 1 Processor -- start 2024-03-18T20:41:27.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd499860b80 1 -- start start 2024-03-18T20:41:27.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd499860b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc6a60 0x557cd6dc6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd499860b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dd1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd499860b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd499860b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd6d20990 con 0x557cd6dc6a60 2024-03-18T20:41:27.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd499860b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd6d1e350 con 0x557cd6dc7380 2024-03-18T20:41:27.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd499860b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd6d19c40 con 0x557cd6dd2220 2024-03-18T20:41:27.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd486ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc6a60 0x557cd6dc6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd4867fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dd1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd4877fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd4867fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dd1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40466/0 (socket says 172.21.15.46:40466) 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.582+0000 7fd4867fc700 1 -- 172.21.15.46:0/2613534828 learned_addr learned my addr 172.21.15.46:0/2613534828 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.583+0000 7fd4867fc700 1 -- 172.21.15.46:0/2613534828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 msgr2=0x557cd6dd4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.583+0000 7fd4867fc700 1 --2- 172.21.15.46:0/2613534828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.583+0000 7fd4867fc700 1 -- 172.21.15.46:0/2613534828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc6a60 msgr2=0x557cd6dc6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.583+0000 7fd4867fc700 1 --2- 172.21.15.46:0/2613534828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc6a60 0x557cd6dc6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.583+0000 7fd4867fc700 1 -- 172.21.15.46:0/2613534828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cd6b9b3b0 con 0x557cd6dc7380 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.583+0000 7fd4867fc700 1 --2- 172.21.15.46:0/2613534828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dd1ce0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0097a0 tx=0x7fd47c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9deeb7884eee5480 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.583+0000 7fd46d7fa700 1 -- 172.21.15.46:0/2613534828 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c00f4d0 con 0x557cd6dc7380 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.583+0000 7fd46d7fa700 1 -- 172.21.15.46:0/2613534828 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd47c021070 con 0x557cd6dc7380 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.583+0000 7fd46d7fa700 1 -- 172.21.15.46:0/2613534828 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c01d7b0 con 0x557cd6dc7380 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.584+0000 7fd499860b80 1 -- 172.21.15.46:0/2613534828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 msgr2=0x557cd6dd1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.584+0000 7fd499860b80 1 --2- 172.21.15.46:0/2613534828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dd1ce0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0097a0 tx=0x7fd47c014000 comp rx=0 tx=0).stop 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.584+0000 7fd499860b80 1 -- 172.21.15.46:0/2613534828 shutdown_connections 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.584+0000 7fd499860b80 1 --2- 172.21.15.46:0/2613534828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc6a60 0x557cd6dc6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.584+0000 7fd499860b80 1 --2- 172.21.15.46:0/2613534828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.584+0000 7fd499860b80 1 --2- 172.21.15.46:0/2613534828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dd1ce0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.584+0000 7fd499860b80 1 -- 172.21.15.46:0/2613534828 >> 172.21.15.46:0/2613534828 conn(0x557cd6d3d220 msgr2=0x557cd6dc3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.584+0000 7fd499860b80 1 -- 172.21.15.46:0/2613534828 shutdown_connections 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.584+0000 7fd499860b80 1 -- 172.21.15.46:0/2613534828 wait complete. 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.584+0000 7fd499860b80 1 -- start start 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.584+0000 7fd499860b80 1 Processor -- start 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd499860b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dc6a60 0x557cd6dd1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd499860b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dcf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd499860b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dd2220 0x557cd6dcf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd499860b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd6cb5d80 con 0x557cd6dd2220 2024-03-18T20:41:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd499860b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd6cb5a20 con 0x557cd6dc7380 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd499860b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd6b9d6a0 con 0x557cd6dc6a60 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd499860b80 10 client.0 _ll_register_callbacks cb 0x557cd6b213a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd4877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dd2220 0x557cd6dcf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd4877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dd2220 0x557cd6dcf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35972/0 (socket says 172.21.15.46:35972) 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd486ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dc6a60 0x557cd6dd1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd4877fe700 1 -- 172.21.15.46:0/517079204 learned_addr learned my addr 172.21.15.46:0/517079204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd4867fc700 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dcf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd4877fe700 1 -- 172.21.15.46:0/517079204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dc6a60 msgr2=0x557cd6dd1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd4877fe700 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dc6a60 0x557cd6dd1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd4877fe700 1 -- 172.21.15.46:0/517079204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 msgr2=0x557cd6dcf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd4877fe700 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dcf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd4877fe700 1 -- 172.21.15.46:0/517079204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47c017050 con 0x557cd6dd2220 2024-03-18T20:41:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd4867fc700 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dcf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd486ffd700 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dc6a60 0x557cd6dd1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd4877fe700 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dd2220 0x557cd6dcf710 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd48000be50 tx=0x7fd48000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0803be243b94b46 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.585+0000 7fd46aff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd46aff5700 1 -- 172.21.15.46:0/517079204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48000e040 con 0x557cd6dd2220 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd46aff5700 1 -- 172.21.15.46:0/517079204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd480023070 con 0x557cd6dd2220 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd499860b80 1 -- 172.21.15.46:0/517079204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cd6dbaad0 con 0x557cd6dd2220 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd46aff5700 1 -- 172.21.15.46:0/517079204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4800074e0 con 0x557cd6dd2220 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd499860b80 10 client.17415 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd499860b80 1 -- 172.21.15.46:0/517079204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557cd6b47b60 con 0x557cd6dd2220 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd46aff5700 1 -- 172.21.15.46:0/517079204 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557cd6dbaad0 con 0x557cd6dd2220 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd46aff5700 1 -- 172.21.15.46:0/517079204 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd48000b0d0 con 0x557cd6dd2220 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd499860b80 10 client.17415 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd499860b80 10 client.17415 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:41:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd499860b80 20 client.17415 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd499860b80 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557cd6de1de0 0x557cd6de4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd499860b80 4 client.17415 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.586+0000 7fd499860b80 1 -- 172.21.15.46:0/517079204 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cd6d12700 con 0x557cd6de1de0 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.587+0000 7fd486ffd700 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557cd6de1de0 0x557cd6de4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.587+0000 7fd46aff5700 10 client.17415 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.587+0000 7fd486ffd700 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557cd6de1de0 0x557cd6de4200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b754cfff650a669 server_cookie=4d7641bd67d44f55 in_seq=0 out_seq=0 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd46aff5700 1 -- 172.21.15.46:0/517079204 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557cd6d12700 con 0x557cd6de1de0 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd46aff5700 10 client.17415 handle_command_reply: tid=0 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd499860b80 1 client.17415 shutdown 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd499860b80 1 -- 172.21.15.46:0/517079204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dd2220 msgr2=0x557cd6dcf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd499860b80 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dd2220 0x557cd6dcf710 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd48000be50 tx=0x7fd48000be80 comp rx=0 tx=0).stop 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd499860b80 1 -- 172.21.15.46:0/517079204 shutdown_connections 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd499860b80 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557cd6de1de0 0x557cd6de4200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd499860b80 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dd2220 0x557cd6dcf710 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd499860b80 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dc6a60 0x557cd6dd1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd499860b80 1 --2- 172.21.15.46:0/517079204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dcf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd46aff5700 0 client.17415 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:41:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd499860b80 1 -- 172.21.15.46:0/517079204 >> 172.21.15.46:0/517079204 conn(0x557cd6d3d220 msgr2=0x557cd6dc2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd499860b80 1 -- 172.21.15.46:0/517079204 shutdown_connections 2024-03-18T20:41:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd499860b80 1 -- 172.21.15.46:0/517079204 wait complete. 2024-03-18T20:41:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.588+0000 7fd499860b80 20 client.17415 trim_cache size 0 max 16384 2024-03-18T20:41:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd499860b80 1 Processor -- start 2024-03-18T20:41:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd499860b80 1 -- start start 2024-03-18T20:41:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd499860b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc6a60 0x557cd6dcf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd499860b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc7380 0x557cd6ddde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd499860b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd499860b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd6dd0580 con 0x557cd6dc7380 2024-03-18T20:41:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd499860b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd6dbcf80 con 0x557cd6dc6a60 2024-03-18T20:41:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd499860b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd6dc83d0 con 0x557cd6dd2220 2024-03-18T20:41:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd4867fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc7380 0x557cd6ddde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd4867fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc7380 0x557cd6ddde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35984/0 (socket says 172.21.15.46:35984) 2024-03-18T20:41:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd4867fc700 1 -- 172.21.15.46:0/2158520398 learned_addr learned my addr 172.21.15.46:0/2158520398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd4877fe700 1 --2- 172.21.15.46:0/2158520398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd486ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc6a60 0x557cd6dcf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd4867fc700 1 -- 172.21.15.46:0/2158520398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 msgr2=0x557cd6dd17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.603+0000 7fd4867fc700 1 --2- 172.21.15.46:0/2158520398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.604+0000 7fd4867fc700 1 -- 172.21.15.46:0/2158520398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc6a60 msgr2=0x557cd6dcf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.604+0000 7fd4867fc700 1 --2- 172.21.15.46:0/2158520398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc6a60 0x557cd6dcf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.604+0000 7fd4867fc700 1 -- 172.21.15.46:0/2158520398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cd6cdfea0 con 0x557cd6dc7380 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.604+0000 7fd4877fe700 1 --2- 172.21.15.46:0/2158520398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.604+0000 7fd4867fc700 1 --2- 172.21.15.46:0/2158520398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc7380 0x557cd6ddde60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0097a0 tx=0x7fd47c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14efc4e63f113542 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.604+0000 7fd46dffb700 1 -- 172.21.15.46:0/2158520398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c02c040 con 0x557cd6dc7380 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.604+0000 7fd46dffb700 1 -- 172.21.15.46:0/2158520398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd47c0046d0 con 0x557cd6dc7380 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.604+0000 7fd46dffb700 1 -- 172.21.15.46:0/2158520398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c01c920 con 0x557cd6dc7380 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 -- 172.21.15.46:0/2158520398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc7380 msgr2=0x557cd6ddde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 --2- 172.21.15.46:0/2158520398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc7380 0x557cd6ddde60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0097a0 tx=0x7fd47c009900 comp rx=0 tx=0).stop 2024-03-18T20:41:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 -- 172.21.15.46:0/2158520398 shutdown_connections 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 --2- 172.21.15.46:0/2158520398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 --2- 172.21.15.46:0/2158520398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc6a60 0x557cd6dcf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 --2- 172.21.15.46:0/2158520398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc7380 0x557cd6ddde60 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 -- 172.21.15.46:0/2158520398 >> 172.21.15.46:0/2158520398 conn(0x557cd6d3d220 msgr2=0x557cd6dc1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 -- 172.21.15.46:0/2158520398 shutdown_connections 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 -- 172.21.15.46:0/2158520398 wait complete. 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 -- start start 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 Processor -- start 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc6a60 0x557cd6dd6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dd6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd6e34d40 con 0x557cd6dc6a60 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd6e34eb0 con 0x557cd6dc7380 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cd6e35070 con 0x557cd6dd2220 2024-03-18T20:41:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.605+0000 7fd499860b80 10 client.0 _ll_register_callbacks cb 0x7fd480008ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd486ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc6a60 0x557cd6dd6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd486ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc6a60 0x557cd6dd6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35990/0 (socket says 172.21.15.46:35990) 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd486ffd700 1 -- 172.21.15.46:0/990551589 learned_addr learned my addr 172.21.15.46:0/990551589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd4877fe700 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd4867fc700 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dd6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd486ffd700 1 -- 172.21.15.46:0/990551589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 msgr2=0x557cd6dd71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd486ffd700 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd486ffd700 1 -- 172.21.15.46:0/990551589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 msgr2=0x557cd6dd6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd486ffd700 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dd6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd486ffd700 1 -- 172.21.15.46:0/990551589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cd6b47b60 con 0x557cd6dc6a60 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd486ffd700 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc6a60 0x557cd6dd6740 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd478099c00 tx=0x7fd4780cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69df5d55e78bcd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd46aff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd46aff5700 1 -- 172.21.15.46:0/990551589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47809dd30 con 0x557cd6dc6a60 2024-03-18T20:41:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd46aff5700 1 -- 172.21.15.46:0/990551589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd4780a2380 con 0x557cd6dc6a60 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd499860b80 1 -- 172.21.15.46:0/990551589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd47c021070 con 0x557cd6dc6a60 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd46aff5700 1 -- 172.21.15.46:0/990551589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4780993f0 con 0x557cd6dc6a60 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd499860b80 10 client.17421 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.606+0000 7fd499860b80 1 -- 172.21.15.46:0/990551589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557cd6b9b3b0 con 0x557cd6dc6a60 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.607+0000 7fd46aff5700 1 -- 172.21.15.46:0/990551589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd47c021070 con 0x557cd6dc6a60 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.607+0000 7fd46aff5700 1 -- 172.21.15.46:0/990551589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd478094b10 con 0x557cd6dc6a60 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.607+0000 7fd499860b80 10 client.17421 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.607+0000 7fd499860b80 10 client.17421 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.607+0000 7fd499860b80 20 client.17421 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.607+0000 7fd499860b80 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557cd6e36790 0x557cd6e38bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.607+0000 7fd499860b80 4 client.17421 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.607+0000 7fd499860b80 1 -- 172.21.15.46:0/990551589 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557cd6b68cf0 con 0x557cd6e36790 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.607+0000 7fd4867fc700 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557cd6e36790 0x557cd6e38bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.607+0000 7fd46aff5700 10 client.17421 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.607+0000 7fd4867fc700 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557cd6e36790 0x557cd6e38bb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aefe1c8b9e126383 server_cookie=e7d6be486775da67 in_seq=0 out_seq=0 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd46aff5700 1 -- 172.21.15.46:0/990551589 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557cd6b68cf0 con 0x557cd6e36790 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd46aff5700 10 client.17421 handle_command_reply: tid=0 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd499860b80 1 client.17421 shutdown 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd499860b80 1 -- 172.21.15.46:0/990551589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc6a60 msgr2=0x557cd6dd6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd499860b80 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc6a60 0x557cd6dd6740 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd478099c00 tx=0x7fd4780cc040 comp rx=0 tx=0).stop 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd499860b80 1 -- 172.21.15.46:0/990551589 shutdown_connections 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd499860b80 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557cd6e36790 0x557cd6e38bb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd499860b80 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557cd6dc6a60 0x557cd6dd6740 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd499860b80 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557cd6dd2220 0x557cd6dd71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd499860b80 1 --2- 172.21.15.46:0/990551589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557cd6dc7380 0x557cd6dd6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd46aff5700 0 client.17421 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd499860b80 1 -- 172.21.15.46:0/990551589 >> 172.21.15.46:0/990551589 conn(0x557cd6d3d220 msgr2=0x557cd6dc21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd499860b80 1 -- 172.21.15.46:0/990551589 shutdown_connections 2024-03-18T20:41:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd499860b80 1 -- 172.21.15.46:0/990551589 wait complete. 2024-03-18T20:41:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.608+0000 7fd499860b80 20 client.17421 trim_cache size 0 max 16384 2024-03-18T20:41:27.621 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:41:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.609+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1085995953 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd4700a1290 msgr2=0x7fd4700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.609+0000 7fd48d6cb700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd4700a1290 0x7fd4700a3750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd48819aa40 tx=0x7fd47c009260 comp rx=0 tx=0).stop 2024-03-18T20:41:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.609+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1085995953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd488069380 msgr2=0x7fd4881994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.609+0000 7fd48d6cb700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd488069380 0x7fd4881994f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd47800e9f0 tx=0x7fd47800ed00 comp rx=0 tx=0).stop 2024-03-18T20:41:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.609+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1085995953 shutdown_connections 2024-03-18T20:41:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.609+0000 7fd48d6cb700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd4700a1290 0x7fd4700a3750 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.609+0000 7fd48d6cb700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd48810d700 0x7fd48819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.609+0000 7fd48d6cb700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd488069380 0x7fd4881994f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.609+0000 7fd48d6cb700 1 --2- 172.21.15.46:0/1085995953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd488069ca0 0x7fd488199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.609+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1085995953 >> 172.21.15.46:0/1085995953 conn(0x7fd488076db0 msgr2=0x7fd48810ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.609+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1085995953 shutdown_connections 2024-03-18T20:41:27.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.609+0000 7fd48d6cb700 1 -- 172.21.15.46:0/1085995953 wait complete. 2024-03-18T20:41:27.626 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:41:27.626 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:41:27.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 151299 2.61 MB/sec execute 142 sec latency 4929.725 ms 2024-03-18T20:41:27.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.766+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3170761053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468101900 msgr2=0x7fc468105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.766+0000 7fc46e9d3700 1 --2- 172.21.15.46:0/3170761053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468101900 0x7fc468105940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009990 tx=0x7fc45c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.766+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3170761053 shutdown_connections 2024-03-18T20:41:27.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.766+0000 7fc46e9d3700 1 --2- 172.21.15.46:0/3170761053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc468106070 0x7fc46810dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.766+0000 7fc46e9d3700 1 --2- 172.21.15.46:0/3170761053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc468100f50 0x7fc468101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.766+0000 7fc46e9d3700 1 --2- 172.21.15.46:0/3170761053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468101900 0x7fc468105940 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.766+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3170761053 >> 172.21.15.46:0/3170761053 conn(0x7fc4680fc7f0 msgr2=0x7fc4680fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.766+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3170761053 shutdown_connections 2024-03-18T20:41:27.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.766+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3170761053 wait complete. 2024-03-18T20:41:27.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.767+0000 7fc46e9d3700 1 Processor -- start 2024-03-18T20:41:27.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.767+0000 7fc46e9d3700 1 -- start start 2024-03-18T20:41:27.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc46e9d3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468100f50 0x7fc468197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc46e9d3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc468106070 0x7fc468197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc46e9d3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc46819bad0 0x7fc46819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc46e9d3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46819c550 con 0x7fc46819bad0 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc46e9d3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46819c6c0 con 0x7fc468106070 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc46e9d3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46819c880 con 0x7fc468100f50 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc4677fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc468106070 0x7fc468197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc46cf70700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc46819bad0 0x7fc46819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc4677fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc468106070 0x7fc468197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40510/0 (socket says 172.21.15.46:40510) 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc4677fe700 1 -- 172.21.15.46:0/3793080122 learned_addr learned my addr 172.21.15.46:0/3793080122 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc467fff700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468100f50 0x7fc468197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc4677fe700 1 -- 172.21.15.46:0/3793080122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468100f50 msgr2=0x7fc468197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc4677fe700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468100f50 0x7fc468197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc4677fe700 1 -- 172.21.15.46:0/3793080122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc46819bad0 msgr2=0x7fc46819bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc4677fe700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc46819bad0 0x7fc46819bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc4677fe700 1 -- 172.21.15.46:0/3793080122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc460009c40 con 0x7fc468106070 2024-03-18T20:41:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.768+0000 7fc4677fe700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc468106070 0x7fc468197780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005770 tx=0x7fc45c0057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.769+0000 7fc4657fa700 1 -- 172.21.15.46:0/3793080122 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c01d070 con 0x7fc468106070 2024-03-18T20:41:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.769+0000 7fc4657fa700 1 -- 172.21.15.46:0/3793080122 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc45c021410 con 0x7fc468106070 2024-03-18T20:41:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.769+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3793080122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45c0095f0 con 0x7fc468106070 2024-03-18T20:41:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.769+0000 7fc4657fa700 1 -- 172.21.15.46:0/3793080122 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c017650 con 0x7fc468106070 2024-03-18T20:41:27.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.769+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3793080122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4681a4070 con 0x7fc468106070 2024-03-18T20:41:27.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.771+0000 7fc4657fa700 1 -- 172.21.15.46:0/3793080122 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc45c00fb40 con 0x7fc468106070 2024-03-18T20:41:27.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.772+0000 7fc4657fa700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc4500a1290 0x7fc4500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.772+0000 7fc467fff700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc4500a1290 0x7fc4500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.772+0000 7fc4657fa700 1 -- 172.21.15.46:0/3793080122 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc45c0cde60 con 0x7fc468106070 2024-03-18T20:41:27.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.772+0000 7fc467fff700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc4500a1290 0x7fc4500a3750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc458009c60 tx=0x7fc458009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.773+0000 7fc47ab68b80 1 Processor -- start 2024-03-18T20:41:27.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.773+0000 7fc47ab68b80 1 -- start start 2024-03-18T20:41:27.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.773+0000 7fc47ab68b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f1a60 0x55ae1e7f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.773+0000 7fc47ab68b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.773+0000 7fc47ab68b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e7ff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.773+0000 7fc47ab68b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1e74b990 con 0x55ae1e7f2380 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.773+0000 7fc47ab68b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1e749350 con 0x55ae1e7fd220 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.773+0000 7fc47ab68b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1e744c40 con 0x55ae1e7f1a60 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.773+0000 7fc4677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.773+0000 7fc467fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f1a60 0x55ae1e7f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc467fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f1a60 0x55ae1e7f1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:45810/0 (socket says 172.21.15.46:45810) 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc4677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36026/0 (socket says 172.21.15.46:36026) 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc467fff700 1 -- 172.21.15.46:0/1415194739 learned_addr learned my addr 172.21.15.46:0/1415194739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc46cf70700 1 --2- 172.21.15.46:0/1415194739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e7ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc4677fe700 1 -- 172.21.15.46:0/1415194739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f1a60 msgr2=0x55ae1e7f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc4677fe700 1 --2- 172.21.15.46:0/1415194739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f1a60 0x55ae1e7f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc4677fe700 1 -- 172.21.15.46:0/1415194739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 msgr2=0x55ae1e7ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc4677fe700 1 --2- 172.21.15.46:0/1415194739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e7ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc4677fe700 1 -- 172.21.15.46:0/1415194739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae1e5c63b0 con 0x55ae1e7f2380 2024-03-18T20:41:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc46cf70700 1 --2- 172.21.15.46:0/1415194739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e7ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc467fff700 1 --2- 172.21.15.46:0/1415194739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f1a60 0x55ae1e7f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc4677fe700 1 --2- 172.21.15.46:0/1415194739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fcce0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0cb580 tx=0x7fc45c09ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f45a202ec0b2f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc44e7fc700 1 -- 172.21.15.46:0/1415194739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0a10c0 con 0x55ae1e7f2380 2024-03-18T20:41:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc44e7fc700 1 -- 172.21.15.46:0/1415194739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc45c09b6d0 con 0x55ae1e7f2380 2024-03-18T20:41:27.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.774+0000 7fc44e7fc700 1 -- 172.21.15.46:0/1415194739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c021810 con 0x55ae1e7f2380 2024-03-18T20:41:27.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.775+0000 7fc47ab68b80 1 -- 172.21.15.46:0/1415194739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 msgr2=0x55ae1e7fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.775+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/1415194739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fcce0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0cb580 tx=0x7fc45c09ae20 comp rx=0 tx=0).stop 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.775+0000 7fc47ab68b80 1 -- 172.21.15.46:0/1415194739 shutdown_connections 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.775+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/1415194739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e7ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.775+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/1415194739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f1a60 0x55ae1e7f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.775+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/1415194739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fcce0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.775+0000 7fc47ab68b80 1 -- 172.21.15.46:0/1415194739 >> 172.21.15.46:0/1415194739 conn(0x55ae1e768220 msgr2=0x55ae1e7ee020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.775+0000 7fc47ab68b80 1 -- 172.21.15.46:0/1415194739 shutdown_connections 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.775+0000 7fc47ab68b80 1 -- 172.21.15.46:0/1415194739 wait complete. 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc47ab68b80 1 -- start start 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc47ab68b80 1 Processor -- start 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc47ab68b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f1a60 0x55ae1e7fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc47ab68b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc47ab68b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e808680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc47ab68b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1e6e0d80 con 0x55ae1e7f2380 2024-03-18T20:41:27.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc47ab68b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1e6e0a20 con 0x55ae1e7fd220 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc47ab68b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1e5c86a0 con 0x55ae1e7f1a60 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc47ab68b80 10 client.0 _ll_register_callbacks cb 0x55ae1e54c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc4677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc46cf70700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e808680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc46cf70700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e808680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40528/0 (socket says 172.21.15.46:40528) 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc4677fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36032/0 (socket says 172.21.15.46:36032) 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc467fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f1a60 0x55ae1e7fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.776+0000 7fc46cf70700 1 -- 172.21.15.46:0/423438731 learned_addr learned my addr 172.21.15.46:0/423438731 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc46cf70700 1 -- 172.21.15.46:0/423438731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f1a60 msgr2=0x55ae1e7fd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc46cf70700 1 --2- 172.21.15.46:0/423438731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f1a60 0x55ae1e7fd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc46cf70700 1 -- 172.21.15.46:0/423438731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 msgr2=0x55ae1e7fdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc46cf70700 1 --2- 172.21.15.46:0/423438731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc46cf70700 1 -- 172.21.15.46:0/423438731 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46000e330 con 0x55ae1e7fd220 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc4677fe700 1 --2- 172.21.15.46:0/423438731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc46cf70700 1 --2- 172.21.15.46:0/423438731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e808680 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc4600145d0 tx=0x7fc4600148e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81c08b1308d53d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc4477fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc4477fe700 1 -- 172.21.15.46:0/423438731 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46000e040 con 0x55ae1e7fd220 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc4477fe700 1 -- 172.21.15.46:0/423438731 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc46001a070 con 0x55ae1e7fd220 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc47ab68b80 1 -- 172.21.15.46:0/423438731 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae1e7e5ad0 con 0x55ae1e7fd220 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc4477fe700 1 -- 172.21.15.46:0/423438731 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4600092d0 con 0x55ae1e7fd220 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc47ab68b80 10 client.27125 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc47ab68b80 1 -- 172.21.15.46:0/423438731 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fc4681a4070 con 0x55ae1e7fd220 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.777+0000 7fc4477fe700 1 -- 172.21.15.46:0/423438731 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae1e7e5ad0 con 0x55ae1e7fd220 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.778+0000 7fc4477fe700 1 -- 172.21.15.46:0/423438731 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc460026070 con 0x55ae1e7fd220 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.778+0000 7fc47ab68b80 10 client.27125 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.778+0000 7fc47ab68b80 10 client.27125 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.778+0000 7fc47ab68b80 20 client.27125 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:27.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.778+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/423438731 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ae1e80cc00 0x55ae1e80f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.778+0000 7fc47ab68b80 4 client.27125 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.778+0000 7fc47ab68b80 1 -- 172.21.15.46:0/423438731 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae1e73d700 con 0x55ae1e80cc00 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.778+0000 7fc467fff700 1 --2- 172.21.15.46:0/423438731 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ae1e80cc00 0x55ae1e80f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.778+0000 7fc467fff700 1 --2- 172.21.15.46:0/423438731 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ae1e80cc00 0x55ae1e80f020 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a190d9cdd36a966 server_cookie=40b8b450b9aa99c1 in_seq=0 out_seq=0 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.778+0000 7fc4477fe700 10 client.27125 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc4477fe700 1 -- 172.21.15.46:0/423438731 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ae1e73d700 con 0x55ae1e80cc00 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc4477fe700 10 client.27125 handle_command_reply: tid=0 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc47ab68b80 1 client.27125 shutdown 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc47ab68b80 1 -- 172.21.15.46:0/423438731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 msgr2=0x55ae1e808680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/423438731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e808680 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc4600145d0 tx=0x7fc4600148e0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc47ab68b80 1 -- 172.21.15.46:0/423438731 shutdown_connections 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/423438731 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ae1e80cc00 0x55ae1e80f020 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/423438731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e808680 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/423438731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f1a60 0x55ae1e7fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/423438731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc4477fe700 0 client.27125 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:41:27.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc47ab68b80 1 -- 172.21.15.46:0/423438731 >> 172.21.15.46:0/423438731 conn(0x55ae1e768220 msgr2=0x55ae1e7edc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc47ab68b80 1 -- 172.21.15.46:0/423438731 shutdown_connections 2024-03-18T20:41:27.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc47ab68b80 1 -- 172.21.15.46:0/423438731 wait complete. 2024-03-18T20:41:27.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.779+0000 7fc47ab68b80 20 client.27125 trim_cache size 0 max 16384 2024-03-18T20:41:27.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc47ab68b80 1 Processor -- start 2024-03-18T20:41:27.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc47ab68b80 1 -- start start 2024-03-18T20:41:27.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc47ab68b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 0x55ae1e802380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc47ab68b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc47ab68b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e7fb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc47ab68b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1e809700 con 0x55ae1e7f1a60 2024-03-18T20:41:27.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc47ab68b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1e7fb870 con 0x55ae1e7fd220 2024-03-18T20:41:27.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc47ab68b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1e7ef2b0 con 0x55ae1e7f2380 2024-03-18T20:41:27.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc467fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 0x55ae1e802380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc467fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 0x55ae1e802380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36044/0 (socket says 172.21.15.46:36044) 2024-03-18T20:41:27.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc467fff700 1 -- 172.21.15.46:0/501392489 learned_addr learned my addr 172.21.15.46:0/501392489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc4677fe700 1 --2- 172.21.15.46:0/501392489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.793+0000 7fc46cf70700 1 --2- 172.21.15.46:0/501392489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e7fb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.794+0000 7fc467fff700 1 -- 172.21.15.46:0/501392489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f2380 msgr2=0x55ae1e7fadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.794+0000 7fc467fff700 1 --2- 172.21.15.46:0/501392489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.794+0000 7fc467fff700 1 -- 172.21.15.46:0/501392489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 msgr2=0x55ae1e7fb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.794+0000 7fc467fff700 1 --2- 172.21.15.46:0/501392489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e7fb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.794+0000 7fc467fff700 1 -- 172.21.15.46:0/501392489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae1e572b60 con 0x55ae1e7f1a60 2024-03-18T20:41:27.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.794+0000 7fc467fff700 1 --2- 172.21.15.46:0/501392489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 0x55ae1e802380 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x55ae1e80f900 tx=0x7fc458014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a05858cf922b35 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.794+0000 7fc44effd700 1 -- 172.21.15.46:0/501392489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458022070 con 0x55ae1e7f1a60 2024-03-18T20:41:27.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.794+0000 7fc44effd700 1 -- 172.21.15.46:0/501392489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc458023040 con 0x55ae1e7f1a60 2024-03-18T20:41:27.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.794+0000 7fc44effd700 1 -- 172.21.15.46:0/501392489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45801d670 con 0x55ae1e7f1a60 2024-03-18T20:41:27.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.795+0000 7fc47ab68b80 1 -- 172.21.15.46:0/501392489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 msgr2=0x55ae1e802380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.795+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/501392489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 0x55ae1e802380 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x55ae1e80f900 tx=0x7fc458014000 comp rx=0 tx=0).stop 2024-03-18T20:41:27.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.795+0000 7fc47ab68b80 1 -- 172.21.15.46:0/501392489 shutdown_connections 2024-03-18T20:41:27.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.795+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/501392489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 0x55ae1e802380 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.795+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/501392489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e7fb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.795+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/501392489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f2380 0x55ae1e7fadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.795+0000 7fc47ab68b80 1 -- 172.21.15.46:0/501392489 >> 172.21.15.46:0/501392489 conn(0x55ae1e768220 msgr2=0x55ae1e7f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.795+0000 7fc47ab68b80 1 -- 172.21.15.46:0/501392489 shutdown_connections 2024-03-18T20:41:27.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.795+0000 7fc47ab68b80 1 -- 172.21.15.46:0/501392489 wait complete. 2024-03-18T20:41:27.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.795+0000 7fc47ab68b80 1 -- start start 2024-03-18T20:41:27.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc47ab68b80 1 Processor -- start 2024-03-18T20:41:27.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc47ab68b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 0x55ae1e859d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc47ab68b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f2380 0x55ae1e85a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc47ab68b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e8004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc47ab68b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1e800a20 con 0x55ae1e7f1a60 2024-03-18T20:41:27.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc47ab68b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1e800b90 con 0x55ae1e7fd220 2024-03-18T20:41:27.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc47ab68b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae1e800d50 con 0x55ae1e7f2380 2024-03-18T20:41:27.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc47ab68b80 10 client.0 _ll_register_callbacks cb 0x7fc46000eab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:27.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc467fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 0x55ae1e859d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc467fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 0x55ae1e859d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36050/0 (socket says 172.21.15.46:36050) 2024-03-18T20:41:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc467fff700 1 -- 172.21.15.46:0/3620236553 learned_addr learned my addr 172.21.15.46:0/3620236553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc4677fe700 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f2380 0x55ae1e85a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc46cf70700 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e8004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc467fff700 1 -- 172.21.15.46:0/3620236553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f2380 msgr2=0x55ae1e85a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc467fff700 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f2380 0x55ae1e85a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc467fff700 1 -- 172.21.15.46:0/3620236553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 msgr2=0x55ae1e8004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc467fff700 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e8004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.796+0000 7fc467fff700 1 -- 172.21.15.46:0/3620236553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae1e70aea0 con 0x55ae1e7f1a60 2024-03-18T20:41:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc467fff700 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 0x55ae1e859d30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc458009f40 tx=0x7fc45802b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5327897b4728ce server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc4477fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc4477fe700 1 -- 172.21.15.46:0/3620236553 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458022040 con 0x55ae1e7f1a60 2024-03-18T20:41:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc4477fe700 1 -- 172.21.15.46:0/3620236553 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc458003a60 con 0x55ae1e7f1a60 2024-03-18T20:41:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc47ab68b80 1 -- 172.21.15.46:0/3620236553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc45c09b6d0 con 0x55ae1e7f1a60 2024-03-18T20:41:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc4477fe700 1 -- 172.21.15.46:0/3620236553 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc458003c20 con 0x55ae1e7f1a60 2024-03-18T20:41:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc47ab68b80 10 client.17439 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc47ab68b80 1 -- 172.21.15.46:0/3620236553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ae1e5cacd0 con 0x55ae1e7f1a60 2024-03-18T20:41:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc4477fe700 1 -- 172.21.15.46:0/3620236553 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc45c09b6d0 con 0x55ae1e7f1a60 2024-03-18T20:41:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc4477fe700 1 -- 172.21.15.46:0/3620236553 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc45801d410 con 0x55ae1e7f1a60 2024-03-18T20:41:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc47ab68b80 10 client.17439 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc47ab68b80 10 client.17439 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:41:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc47ab68b80 20 client.17439 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ae1e861bf0 0x55ae1e864010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc47ab68b80 4 client.17439 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.797+0000 7fc47ab68b80 1 -- 172.21.15.46:0/3620236553 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ae1e593cf0 con 0x55ae1e861bf0 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.798+0000 7fc4677fe700 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ae1e861bf0 0x55ae1e864010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.798+0000 7fc4477fe700 10 client.17439 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.798+0000 7fc4677fe700 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ae1e861bf0 0x55ae1e864010 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4eaa5df66b55a40b server_cookie=76f84eac740ac5a4 in_seq=0 out_seq=0 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.798+0000 7fc4477fe700 1 -- 172.21.15.46:0/3620236553 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ae1e593cf0 con 0x55ae1e861bf0 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.798+0000 7fc4477fe700 10 client.17439 handle_command_reply: tid=0 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.798+0000 7fc47ab68b80 1 client.17439 shutdown 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.798+0000 7fc47ab68b80 1 -- 172.21.15.46:0/3620236553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 msgr2=0x55ae1e859d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.798+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 0x55ae1e859d30 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc458009f40 tx=0x7fc45802b5a0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc47ab68b80 1 -- 172.21.15.46:0/3620236553 shutdown_connections 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ae1e861bf0 0x55ae1e864010 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae1e7f1a60 0x55ae1e859d30 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae1e7fd220 0x55ae1e8004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc47ab68b80 1 --2- 172.21.15.46:0/3620236553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae1e7f2380 0x55ae1e85a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc47ab68b80 1 -- 172.21.15.46:0/3620236553 >> 172.21.15.46:0/3620236553 conn(0x55ae1e768220 msgr2=0x55ae1e7facd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc4477fe700 0 client.17439 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:41:27.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc47ab68b80 1 -- 172.21.15.46:0/3620236553 shutdown_connections 2024-03-18T20:41:27.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc47ab68b80 1 -- 172.21.15.46:0/3620236553 wait complete. 2024-03-18T20:41:27.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc47ab68b80 20 client.17439 trim_cache size 0 max 16384 2024-03-18T20:41:27.809 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:41:27.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3793080122 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc4500a1290 msgr2=0x7fc4500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc46e9d3700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc4500a1290 0x7fc4500a3750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc458009c60 tx=0x7fc458009260 comp rx=0 tx=0).stop 2024-03-18T20:41:27.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3793080122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc468106070 msgr2=0x7fc468197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.799+0000 7fc46e9d3700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc468106070 0x7fc468197780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005770 tx=0x7fc45c0057a0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.800+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3793080122 shutdown_connections 2024-03-18T20:41:27.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.800+0000 7fc46e9d3700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc4500a1290 0x7fc4500a3750 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.800+0000 7fc46e9d3700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc46819bad0 0x7fc46819bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.800+0000 7fc46e9d3700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc468100f50 0x7fc468197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.800+0000 7fc46e9d3700 1 --2- 172.21.15.46:0/3793080122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc468106070 0x7fc468197780 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.800+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3793080122 >> 172.21.15.46:0/3793080122 conn(0x7fc4680fc7f0 msgr2=0x7fc4680fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.800+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3793080122 shutdown_connections 2024-03-18T20:41:27.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.800+0000 7fc46e9d3700 1 -- 172.21.15.46:0/3793080122 wait complete. 2024-03-18T20:41:27.816 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:41:27.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:41:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.963+0000 7f176811a700 1 -- 172.21.15.46:0/557713590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f176010d540 msgr2=0x7f176010f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.963+0000 7f176811a700 1 --2- 172.21.15.46:0/557713590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f176010d540 0x7f176010f9b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f175800b0a0 tx=0x7f175800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.963+0000 7f176811a700 1 -- 172.21.15.46:0/557713590 shutdown_connections 2024-03-18T20:41:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.963+0000 7f176811a700 1 --2- 172.21.15.46:0/557713590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17600731e0 0x7f17600735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.963+0000 7f176811a700 1 --2- 172.21.15.46:0/557713590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f176010d540 0x7f176010f9b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.963+0000 7f176811a700 1 --2- 172.21.15.46:0/557713590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1760073b00 0x7f176010d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.963+0000 7f176811a700 1 -- 172.21.15.46:0/557713590 >> 172.21.15.46:0/557713590 conn(0x7f17600fc7f0 msgr2=0x7f17600fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.963+0000 7f176811a700 1 -- 172.21.15.46:0/557713590 shutdown_connections 2024-03-18T20:41:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.963+0000 7f176811a700 1 -- 172.21.15.46:0/557713590 wait complete. 2024-03-18T20:41:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.963+0000 7f176811a700 1 Processor -- start 2024-03-18T20:41:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.963+0000 7f176811a700 1 -- start start 2024-03-18T20:41:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f176811a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17600731e0 0x7f17601994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f176811a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760073b00 0x7f17601999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f176811a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f176010d540 0x7f176019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f176811a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176019e3d0 con 0x7f1760073b00 2024-03-18T20:41:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f176811a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176019e540 con 0x7f176010d540 2024-03-18T20:41:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f176811a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176019e700 con 0x7f17600731e0 2024-03-18T20:41:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f17656b5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760073b00 0x7f17601999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f17656b5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760073b00 0x7f17601999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36062/0 (socket says 172.21.15.46:36062) 2024-03-18T20:41:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f17656b5700 1 -- 172.21.15.46:0/1283513818 learned_addr learned my addr 172.21.15.46:0/1283513818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f1765eb6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17600731e0 0x7f17601994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f17666b7700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f176010d540 0x7f176019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f17656b5700 1 -- 172.21.15.46:0/1283513818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17600731e0 msgr2=0x7f17601994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f17656b5700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17600731e0 0x7f17601994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f17656b5700 1 -- 172.21.15.46:0/1283513818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f176010d540 msgr2=0x7f176019dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f17656b5700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f176010d540 0x7f176019dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f17656b5700 1 -- 172.21.15.46:0/1283513818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c009520 con 0x7f1760073b00 2024-03-18T20:41:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f1765eb6700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17600731e0 0x7f17601994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f17666b7700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f176010d540 0x7f176019dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.964+0000 7f17656b5700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760073b00 0x7f17601999f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f175c00eb30 tx=0x7f175c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.965+0000 7f1756ffd700 1 -- 172.21.15.46:0/1283513818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c009880 con 0x7f1760073b00 2024-03-18T20:41:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.965+0000 7f176811a700 1 -- 172.21.15.46:0/1283513818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1758009d00 con 0x7f1760073b00 2024-03-18T20:41:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.965+0000 7f1756ffd700 1 -- 172.21.15.46:0/1283513818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f175c0044e0 con 0x7f1760073b00 2024-03-18T20:41:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.965+0000 7f1756ffd700 1 -- 172.21.15.46:0/1283513818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c003d80 con 0x7f1760073b00 2024-03-18T20:41:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.965+0000 7f176811a700 1 -- 172.21.15.46:0/1283513818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176019ebd0 con 0x7f1760073b00 2024-03-18T20:41:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.968+0000 7f1756ffd700 1 -- 172.21.15.46:0/1283513818 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f175c004000 con 0x7f1760073b00 2024-03-18T20:41:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.969+0000 7f1756ffd700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f174c0a1290 0x7f174c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.969+0000 7f17742afb80 1 Processor -- start 2024-03-18T20:41:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.969+0000 7f1756ffd700 1 -- 172.21.15.46:0/1283513818 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f175c014070 con 0x7f1760073b00 2024-03-18T20:41:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.969+0000 7f1765eb6700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f174c0a1290 0x7f174c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.969+0000 7f17742afb80 1 -- start start 2024-03-18T20:41:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.969+0000 7f17742afb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a7752a60 0x55f3a7752e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.969+0000 7f17742afb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7753380 0x55f3a775dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.969+0000 7f17742afb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a775e220 0x55f3a7760610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.969+0000 7f17742afb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a76ac990 con 0x55f3a7752a60 2024-03-18T20:41:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.969+0000 7f17742afb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a76aa350 con 0x55f3a775e220 2024-03-18T20:41:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.969+0000 7f17742afb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a76a5c40 con 0x55f3a7753380 2024-03-18T20:41:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f1765eb6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a7752a60 0x55f3a7752e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f17656b5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7753380 0x55f3a775dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f1765eb6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a7752a60 0x55f3a7752e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36068/0 (socket says 172.21.15.46:36068) 2024-03-18T20:41:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f1765eb6700 1 -- 172.21.15.46:0/1196523947 learned_addr learned my addr 172.21.15.46:0/1196523947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f1765eb6700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f174c0a1290 0x7f174c0a3750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1750011490 tx=0x7f1750009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f17656b5700 1 -- 172.21.15.46:0/1196523947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a775e220 msgr2=0x55f3a7760610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:41:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f17656b5700 1 --2- 172.21.15.46:0/1196523947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a775e220 0x55f3a7760610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f17656b5700 1 -- 172.21.15.46:0/1196523947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a7752a60 msgr2=0x55f3a7752e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f17656b5700 1 --2- 172.21.15.46:0/1196523947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a7752a60 0x55f3a7752e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f17656b5700 1 -- 172.21.15.46:0/1196523947 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3a75273b0 con 0x55f3a7753380 2024-03-18T20:41:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f17656b5700 1 --2- 172.21.15.46:0/1196523947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7753380 0x55f3a775dce0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f175c0ca900 tx=0x7f175c099c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0e8b46ff79cdf27 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f174bfff700 1 -- 172.21.15.46:0/1196523947 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c09a2c0 con 0x55f3a7753380 2024-03-18T20:41:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f174bfff700 1 -- 172.21.15.46:0/1196523947 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f175c09e0f0 con 0x55f3a7753380 2024-03-18T20:41:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.970+0000 7f174bfff700 1 -- 172.21.15.46:0/1196523947 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c09a740 con 0x55f3a7753380 2024-03-18T20:41:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.971+0000 7f17742afb80 1 -- 172.21.15.46:0/1196523947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7753380 msgr2=0x55f3a775dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.971+0000 7f17742afb80 1 --2- 172.21.15.46:0/1196523947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7753380 0x55f3a775dce0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f175c0ca900 tx=0x7f175c099c20 comp rx=0 tx=0).stop 2024-03-18T20:41:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.971+0000 7f17742afb80 1 -- 172.21.15.46:0/1196523947 shutdown_connections 2024-03-18T20:41:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.971+0000 7f17742afb80 1 --2- 172.21.15.46:0/1196523947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a7752a60 0x55f3a7752e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.971+0000 7f17742afb80 1 --2- 172.21.15.46:0/1196523947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a775e220 0x55f3a7760610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.971+0000 7f17742afb80 1 --2- 172.21.15.46:0/1196523947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7753380 0x55f3a775dce0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.971+0000 7f17742afb80 1 -- 172.21.15.46:0/1196523947 >> 172.21.15.46:0/1196523947 conn(0x55f3a76c9220 msgr2=0x55f3a774f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.971+0000 7f17742afb80 1 -- 172.21.15.46:0/1196523947 shutdown_connections 2024-03-18T20:41:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.971+0000 7f17742afb80 1 -- 172.21.15.46:0/1196523947 wait complete. 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f17742afb80 1 -- start start 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f17742afb80 1 Processor -- start 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f17742afb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a775b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f17742afb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a775b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f17742afb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a775ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f17742afb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a7641d80 con 0x55f3a775e220 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f17742afb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a7641a20 con 0x55f3a7753380 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f17742afb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a75296a0 con 0x55f3a7752a60 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f17742afb80 10 client.0 _ll_register_callbacks cb 0x55f3a74ad3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f1765eb6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a775b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f1765eb6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a775b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:45872/0 (socket says 172.21.15.46:45872) 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f1765eb6700 1 -- 172.21.15.46:0/2524043452 learned_addr learned my addr 172.21.15.46:0/2524043452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.972+0000 7f17656b5700 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a775b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17666b7700 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a775ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17656b5700 1 -- 172.21.15.46:0/2524043452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 msgr2=0x55f3a775b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17656b5700 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a775b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17656b5700 1 -- 172.21.15.46:0/2524043452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 msgr2=0x55f3a775ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17656b5700 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a775ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17656b5700 1 -- 172.21.15.46:0/2524043452 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176019ebd0 con 0x55f3a7753380 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f1765eb6700 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a775b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17656b5700 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a775b540 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f175c0a2900 tx=0x7f175c003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdb652efa3289fc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17497fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17497fa700 1 -- 172.21.15.46:0/2524043452 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0999b0 con 0x55f3a7753380 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17497fa700 1 -- 172.21.15.46:0/2524043452 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f175c0af030 con 0x55f3a7753380 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17742afb80 1 -- 172.21.15.46:0/2524043452 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3a7746ad0 con 0x55f3a7753380 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17497fa700 1 -- 172.21.15.46:0/2524043452 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f175c09a2c0 con 0x55f3a7753380 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17742afb80 10 client.27140 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.973+0000 7f17742afb80 1 -- 172.21.15.46:0/2524043452 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f3a74d3b60 con 0x55f3a7753380 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.974+0000 7f17497fa700 1 -- 172.21.15.46:0/2524043452 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f3a7746ad0 con 0x55f3a7753380 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.974+0000 7f17497fa700 1 -- 172.21.15.46:0/2524043452 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f175c064070 con 0x55f3a7753380 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.974+0000 7f17742afb80 10 client.27140 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.974+0000 7f17742afb80 10 client.27140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.974+0000 7f17742afb80 20 client.27140 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.974+0000 7f17742afb80 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f3a776dac0 0x55f3a776fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.974+0000 7f17742afb80 4 client.27140 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.974+0000 7f17742afb80 1 -- 172.21.15.46:0/2524043452 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3a769e700 con 0x55f3a776dac0 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.974+0000 7f1765eb6700 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f3a776dac0 0x55f3a776fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.974+0000 7f17497fa700 10 client.27140 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.974+0000 7f1765eb6700 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f3a776dac0 0x55f3a776fee0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692b7c23a719c0e2 server_cookie=8605b86f2b0a721a in_seq=0 out_seq=0 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.975+0000 7f17497fa700 1 -- 172.21.15.46:0/2524043452 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f3a769e700 con 0x55f3a776dac0 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.975+0000 7f17497fa700 10 client.27140 handle_command_reply: tid=0 2024-03-18T20:41:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.975+0000 7f17742afb80 1 client.27140 shutdown 2024-03-18T20:41:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.975+0000 7f17742afb80 1 -- 172.21.15.46:0/2524043452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 msgr2=0x55f3a775b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.975+0000 7f17742afb80 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a775b540 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f175c0a2900 tx=0x7f175c003680 comp rx=0 tx=0).stop 2024-03-18T20:41:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.975+0000 7f17742afb80 1 -- 172.21.15.46:0/2524043452 shutdown_connections 2024-03-18T20:41:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.975+0000 7f17742afb80 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f3a776dac0 0x55f3a776fee0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.975+0000 7f17742afb80 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a775ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.975+0000 7f17742afb80 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a775b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.975+0000 7f17497fa700 0 client.27140 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.975+0000 7f17742afb80 1 --2- 172.21.15.46:0/2524043452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a775b540 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.976+0000 7f17742afb80 1 -- 172.21.15.46:0/2524043452 >> 172.21.15.46:0/2524043452 conn(0x55f3a76c9220 msgr2=0x55f3a774ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.976+0000 7f17742afb80 1 -- 172.21.15.46:0/2524043452 shutdown_connections 2024-03-18T20:41:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.976+0000 7f17742afb80 1 -- 172.21.15.46:0/2524043452 wait complete. 2024-03-18T20:41:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.976+0000 7f17742afb80 20 client.27140 trim_cache size 0 max 16384 2024-03-18T20:41:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17742afb80 1 Processor -- start 2024-03-18T20:41:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17742afb80 1 -- start start 2024-03-18T20:41:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17742afb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a775b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17742afb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a7758280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17742afb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a77587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:28.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17742afb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a7753090 con 0x55f3a775e220 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17742afb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a7749210 con 0x55f3a7753380 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17742afb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a7750440 con 0x55f3a7752a60 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17656b5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a7758280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17666b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a77587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f1765eb6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a775b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17656b5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a7758280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40608/0 (socket says 172.21.15.46:40608) 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17656b5700 1 -- 172.21.15.46:0/2230820164 learned_addr learned my addr 172.21.15.46:0/2230820164 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.992+0000 7f17666b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a77587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36092/0 (socket says 172.21.15.46:36092) 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.993+0000 7f17656b5700 1 -- 172.21.15.46:0/2230820164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 msgr2=0x55f3a775b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.993+0000 7f17656b5700 1 --2- 172.21.15.46:0/2230820164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a775b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.993+0000 7f17656b5700 1 -- 172.21.15.46:0/2230820164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 msgr2=0x55f3a77587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.993+0000 7f17656b5700 1 --2- 172.21.15.46:0/2230820164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a77587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.993+0000 7f17656b5700 1 -- 172.21.15.46:0/2230820164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3a766bea0 con 0x55f3a7753380 2024-03-18T20:41:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.993+0000 7f17656b5700 1 --2- 172.21.15.46:0/2230820164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a7758280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f175c00c5b0 tx=0x7f175c0b8000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bea8eccc0e03e960 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.993+0000 7f1754ff9700 1 -- 172.21.15.46:0/2230820164 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c09a720 con 0x55f3a7753380 2024-03-18T20:41:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.993+0000 7f1754ff9700 1 -- 172.21.15.46:0/2230820164 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f175c00cdd0 con 0x55f3a7753380 2024-03-18T20:41:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.993+0000 7f1754ff9700 1 -- 172.21.15.46:0/2230820164 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c099a10 con 0x55f3a7753380 2024-03-18T20:41:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.994+0000 7f17742afb80 1 -- 172.21.15.46:0/2230820164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 msgr2=0x55f3a7758280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.994+0000 7f17742afb80 1 --2- 172.21.15.46:0/2230820164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a7758280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f175c00c5b0 tx=0x7f175c0b8000 comp rx=0 tx=0).stop 2024-03-18T20:41:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.994+0000 7f17742afb80 1 -- 172.21.15.46:0/2230820164 shutdown_connections 2024-03-18T20:41:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.994+0000 7f17742afb80 1 --2- 172.21.15.46:0/2230820164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a77587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.994+0000 7f17742afb80 1 --2- 172.21.15.46:0/2230820164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a775b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.994+0000 7f17742afb80 1 --2- 172.21.15.46:0/2230820164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a7758280 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.994+0000 7f17742afb80 1 -- 172.21.15.46:0/2230820164 >> 172.21.15.46:0/2230820164 conn(0x55f3a76c9220 msgr2=0x55f3a77598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.994+0000 7f17742afb80 1 -- 172.21.15.46:0/2230820164 shutdown_connections 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.994+0000 7f17742afb80 1 -- 172.21.15.46:0/2230820164 wait complete. 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.994+0000 7f17742afb80 1 -- start start 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.994+0000 7f17742afb80 1 Processor -- start 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17742afb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a7762e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17742afb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a7763370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17742afb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a77638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17742afb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a77c08b0 con 0x55f3a775e220 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17742afb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a77c09f0 con 0x55f3a7753380 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17742afb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a77c0b30 con 0x55f3a7752a60 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17742afb80 10 client.0 _ll_register_callbacks cb 0x7f1750004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17666b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a77638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f1765eb6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a7762e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17656b5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a7763370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17666b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a77638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36102/0 (socket says 172.21.15.46:36102) 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f1765eb6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a7762e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:45896/0 (socket says 172.21.15.46:45896) 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17666b7700 1 -- 172.21.15.46:0/477479336 learned_addr learned my addr 172.21.15.46:0/477479336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17656b5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a7763370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40610/0 (socket says 172.21.15.46:40610) 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17656b5700 1 -- 172.21.15.46:0/477479336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 msgr2=0x55f3a7762e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17656b5700 1 --2- 172.21.15.46:0/477479336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a7762e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17656b5700 1 -- 172.21.15.46:0/477479336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 msgr2=0x55f3a77638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17656b5700 1 --2- 172.21.15.46:0/477479336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a77638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17656b5700 1 -- 172.21.15.46:0/477479336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3a74d3b60 con 0x55f3a7753380 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17666b7700 1 --2- 172.21.15.46:0/477479336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a77638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17656b5700 1 --2- 172.21.15.46:0/477479336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a7763370 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f175c00cd80 tx=0x7f175c098870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d244b977f2c96cdc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.995+0000 7f17497fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17497fa700 1 -- 172.21.15.46:0/477479336 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c099c20 con 0x55f3a7753380 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17497fa700 1 -- 172.21.15.46:0/477479336 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f175c0af070 con 0x55f3a7753380 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17742afb80 1 -- 172.21.15.46:0/477479336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f175c09e0f0 con 0x55f3a7753380 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17497fa700 1 -- 172.21.15.46:0/477479336 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f175c0a23e0 con 0x55f3a7753380 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17742afb80 10 client.27146 fetch_fsmap learned FSMap version 39 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17742afb80 1 -- 172.21.15.46:0/477479336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f3a752bcd0 con 0x55f3a7753380 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17497fa700 1 -- 172.21.15.46:0/477479336 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f175c09e0f0 con 0x55f3a7753380 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17497fa700 1 -- 172.21.15.46:0/477479336 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 39) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f175c064070 con 0x55f3a7753380 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17742afb80 10 client.27146 fetch_fsmap finished waiting for FSMap version 39 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17742afb80 10 client.27146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17742afb80 20 client.27146 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17742afb80 1 --2- 172.21.15.46:0/477479336 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f3a77c1df0 0x55f3a77c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:28.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17742afb80 4 client.27146 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f17742afb80 1 -- 172.21.15.46:0/477479336 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f3a74f4cf0 con 0x55f3a77c1df0 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.996+0000 7f1765eb6700 1 --2- 172.21.15.46:0/477479336 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f3a77c1df0 0x55f3a77c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f1765eb6700 1 --2- 172.21.15.46:0/477479336 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f3a77c1df0 0x55f3a77c41e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d217ceb8e9d4cf1 server_cookie=a1e46288aa730835 in_seq=0 out_seq=0 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17497fa700 10 client.27146 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17497fa700 1 -- 172.21.15.46:0/477479336 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f3a74f4cf0 con 0x55f3a77c1df0 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17497fa700 10 client.27146 handle_command_reply: tid=0 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17742afb80 1 client.27146 shutdown 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17742afb80 1 -- 172.21.15.46:0/477479336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 msgr2=0x55f3a7763370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17742afb80 1 --2- 172.21.15.46:0/477479336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a7763370 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f175c00cd80 tx=0x7f175c098870 comp rx=0 tx=0).stop 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17742afb80 1 -- 172.21.15.46:0/477479336 shutdown_connections 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17742afb80 1 --2- 172.21.15.46:0/477479336 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f3a77c1df0 0x55f3a77c41e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17742afb80 1 --2- 172.21.15.46:0/477479336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f3a775e220 0x55f3a77638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17742afb80 1 --2- 172.21.15.46:0/477479336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f3a7752a60 0x55f3a7762e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17742afb80 1 --2- 172.21.15.46:0/477479336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3a7753380 0x55f3a7763370 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17497fa700 0 client.27146 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17742afb80 1 -- 172.21.15.46:0/477479336 >> 172.21.15.46:0/477479336 conn(0x55f3a76c9220 msgr2=0x55f3a7759da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17742afb80 1 -- 172.21.15.46:0/477479336 shutdown_connections 2024-03-18T20:41:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17742afb80 1 -- 172.21.15.46:0/477479336 wait complete. 2024-03-18T20:41:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.997+0000 7f17742afb80 20 client.27146 trim_cache size 0 max 16384 2024-03-18T20:41:28.008 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:41:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.998+0000 7f176811a700 1 -- 172.21.15.46:0/1283513818 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f174c0a1290 msgr2=0x7f174c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.998+0000 7f176811a700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f174c0a1290 0x7f174c0a3750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1750011490 tx=0x7f1750009260 comp rx=0 tx=0).stop 2024-03-18T20:41:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.998+0000 7f176811a700 1 -- 172.21.15.46:0/1283513818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760073b00 msgr2=0x7f17601999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.998+0000 7f176811a700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760073b00 0x7f17601999f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f175c00eb30 tx=0x7f175c00ee40 comp rx=0 tx=0).stop 2024-03-18T20:41:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.998+0000 7f176811a700 1 -- 172.21.15.46:0/1283513818 shutdown_connections 2024-03-18T20:41:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.998+0000 7f176811a700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f174c0a1290 0x7f174c0a3750 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.998+0000 7f176811a700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f176010d540 0x7f176019dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.998+0000 7f176811a700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17600731e0 0x7f17601994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.998+0000 7f176811a700 1 --2- 172.21.15.46:0/1283513818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760073b00 0x7f17601999f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.998+0000 7f176811a700 1 -- 172.21.15.46:0/1283513818 >> 172.21.15.46:0/1283513818 conn(0x7f17600fc7f0 msgr2=0x7f17600fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.998+0000 7f176811a700 1 -- 172.21.15.46:0/1283513818 shutdown_connections 2024-03-18T20:41:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:27.998+0000 7f176811a700 1 -- 172.21.15.46:0/1283513818 wait complete. 2024-03-18T20:41:28.015 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:28 smithi046 ceph-mon[43495]: pgmap v164: 97 pgs: 97 active+clean; 67 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 919 KiB/s wr, 125 op/s 2024-03-18T20:41:28.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:28 smithi155 ceph-mon[48344]: pgmap v164: 97 pgs: 97 active+clean; 67 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 919 KiB/s wr, 125 op/s 2024-03-18T20:41:28.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:28 smithi073 ceph-mon[47384]: pgmap v164: 97 pgs: 97 active+clean; 67 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 919 KiB/s wr, 125 op/s 2024-03-18T20:41:28.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 152954 2.67 MB/sec execute 143 sec latency 13.408 ms 2024-03-18T20:41:29.018 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:41:29.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:41:29.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:41:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.167+0000 7f6bf1cce700 1 -- 172.21.15.46:0/2663079785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bec106070 msgr2=0x7f6bec10dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.167+0000 7f6bf1cce700 1 --2- 172.21.15.46:0/2663079785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bec106070 0x7f6bec10dbf0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009990 tx=0x7f6be4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.167+0000 7f6bf1cce700 1 -- 172.21.15.46:0/2663079785 shutdown_connections 2024-03-18T20:41:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.167+0000 7f6bf1cce700 1 --2- 172.21.15.46:0/2663079785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bec106070 0x7f6bec10dbf0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.167+0000 7f6bf1cce700 1 --2- 172.21.15.46:0/2663079785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec100f50 0x7f6bec101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.167+0000 7f6bf1cce700 1 --2- 172.21.15.46:0/2663079785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bec101900 0x7f6bec105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.167+0000 7f6bf1cce700 1 -- 172.21.15.46:0/2663079785 >> 172.21.15.46:0/2663079785 conn(0x7f6bec0fc7f0 msgr2=0x7f6bec0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.167+0000 7f6bf1cce700 1 -- 172.21.15.46:0/2663079785 shutdown_connections 2024-03-18T20:41:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.167+0000 7f6bf1cce700 1 -- 172.21.15.46:0/2663079785 wait complete. 2024-03-18T20:41:29.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6bf1cce700 1 Processor -- start 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6bf1cce700 1 -- start start 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6bf1cce700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec100f50 0x7f6bec0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6bf1cce700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bec106070 0x7f6bec0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6bf1cce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bec073b30 0x7f6bec1a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6bf1cce700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec1a59b0 con 0x7f6bec073b30 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6bf1cce700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec1a5b20 con 0x7f6bec106070 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6bf1cce700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec1a5ce0 con 0x7f6bec100f50 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bec073b30 0x7f6bec1a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6beb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec100f50 0x7f6bec0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bec073b30 0x7f6bec1a5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36116/0 (socket says 172.21.15.46:36116) 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6bebfff700 1 -- 172.21.15.46:0/1474822109 learned_addr learned my addr 172.21.15.46:0/1474822109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.168+0000 7f6beaffd700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bec106070 0x7f6bec0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.169+0000 7f6beb7fe700 1 -- 172.21.15.46:0/1474822109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bec106070 msgr2=0x7f6bec0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.169+0000 7f6beb7fe700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bec106070 0x7f6bec0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.169+0000 7f6beb7fe700 1 -- 172.21.15.46:0/1474822109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bec073b30 msgr2=0x7f6bec1a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.169+0000 7f6beb7fe700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bec073b30 0x7f6bec1a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.169+0000 7f6beb7fe700 1 -- 172.21.15.46:0/1474822109 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be40095f0 con 0x7f6bec100f50 2024-03-18T20:41:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.169+0000 7f6bebfff700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bec073b30 0x7f6bec1a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.169+0000 7f6beb7fe700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec100f50 0x7f6bec0730b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00ef10 tx=0x7f6bdc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.169+0000 7f6be8ff9700 1 -- 172.21.15.46:0/1474822109 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00c9c0 con 0x7f6bec100f50 2024-03-18T20:41:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.169+0000 7f6bf1cce700 1 -- 172.21.15.46:0/1474822109 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec1a5f60 con 0x7f6bec100f50 2024-03-18T20:41:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.169+0000 7f6be8ff9700 1 -- 172.21.15.46:0/1474822109 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bdc00f040 con 0x7f6bec100f50 2024-03-18T20:41:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.169+0000 7f6be8ff9700 1 -- 172.21.15.46:0/1474822109 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc014480 con 0x7f6bec100f50 2024-03-18T20:41:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.169+0000 7f6bf1cce700 1 -- 172.21.15.46:0/1474822109 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec1a63e0 con 0x7f6bec100f50 2024-03-18T20:41:29.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.172+0000 7f6be8ff9700 1 -- 172.21.15.46:0/1474822109 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6bdc00cb20 con 0x7f6bec100f50 2024-03-18T20:41:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.173+0000 7f6be8ff9700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6bd40a1290 0x7f6bd40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.173+0000 7f6beaffd700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6bd40a1290 0x7f6bd40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.173+0000 7f6bfde63b80 1 Processor -- start 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.173+0000 7f6be8ff9700 1 -- 172.21.15.46:0/1474822109 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6bdc0ccfb0 con 0x7f6bec100f50 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.173+0000 7f6bfde63b80 1 -- start start 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.173+0000 7f6bfde63b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4dfaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.173+0000 7f6bfde63b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e05d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.173+0000 7f6bfde63b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e08690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.173+0000 7f6bfde63b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a4d54a10 con 0x55b3a4e062a0 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.173+0000 7f6bfde63b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a4d523d0 con 0x55b3a4dfb400 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.173+0000 7f6bfde63b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a4d4dcc0 con 0x55b3a4dfaae0 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.173+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e08690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6beaffd700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6bd40a1290 0x7f6bd40a3750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0747e0 tx=0x7f6be00073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e08690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36128/0 (socket says 172.21.15.46:36128) 2024-03-18T20:41:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6bebfff700 1 -- 172.21.15.46:0/1523742592 learned_addr learned my addr 172.21.15.46:0/1523742592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6beb7fe700 1 --2- 172.21.15.46:0/1523742592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4dfaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6beaffd700 1 --2- 172.21.15.46:0/1523742592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e05d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6bebfff700 1 -- 172.21.15.46:0/1523742592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 msgr2=0x55b3a4dfaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6bebfff700 1 --2- 172.21.15.46:0/1523742592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4dfaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6bebfff700 1 -- 172.21.15.46:0/1523742592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 msgr2=0x55b3a4e05d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6bebfff700 1 --2- 172.21.15.46:0/1523742592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e05d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6bebfff700 1 -- 172.21.15.46:0/1523742592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3a4bcf430 con 0x55b3a4e062a0 2024-03-18T20:41:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6bebfff700 1 --2- 172.21.15.46:0/1523742592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e08690 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005cb0 tx=0x7f6be400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d0222779a56527 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6bd1ffb700 1 -- 172.21.15.46:0/1523742592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4027070 con 0x55b3a4e062a0 2024-03-18T20:41:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6bd1ffb700 1 -- 172.21.15.46:0/1523742592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6be401e040 con 0x55b3a4e062a0 2024-03-18T20:41:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.174+0000 7f6bd1ffb700 1 -- 172.21.15.46:0/1523742592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be401d050 con 0x55b3a4e062a0 2024-03-18T20:41:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.175+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1523742592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 msgr2=0x55b3a4e08690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.175+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1523742592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e08690 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005cb0 tx=0x7f6be400b5b0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.175+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1523742592 shutdown_connections 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.175+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1523742592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e08690 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.175+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1523742592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4dfaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.175+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1523742592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e05d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.175+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1523742592 >> 172.21.15.46:0/1523742592 conn(0x55b3a4d712a0 msgr2=0x55b3a4df70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.175+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1523742592 shutdown_connections 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.175+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1523742592 wait complete. 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.175+0000 7f6bfde63b80 1 -- start start 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6bfde63b80 1 Processor -- start 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6bfde63b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e05d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6bfde63b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e03040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6bfde63b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4e062a0 0x55b3a4e03780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6bfde63b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a4ce9e00 con 0x55b3a4dfaae0 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6bfde63b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a4ce9aa0 con 0x55b3a4dfb400 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6bfde63b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a4bd1720 con 0x55b3a4e062a0 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6bfde63b80 10 client.0 _ll_register_callbacks cb 0x55b3a4b55420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6beb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e05d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6beaffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e03040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6beaffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e03040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40630/0 (socket says 172.21.15.46:40630) 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6beaffd700 1 -- 172.21.15.46:0/1415716609 learned_addr learned my addr 172.21.15.46:0/1415716609 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.176+0000 7f6bebfff700 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4e062a0 0x55b3a4e03780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6beaffd700 1 -- 172.21.15.46:0/1415716609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4e062a0 msgr2=0x55b3a4e03780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6beaffd700 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4e062a0 0x55b3a4e03780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6beaffd700 1 -- 172.21.15.46:0/1415716609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4dfaae0 msgr2=0x55b3a4e05d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6beaffd700 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e05d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6beaffd700 1 -- 172.21.15.46:0/1415716609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec10e460 con 0x55b3a4dfb400 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6bebfff700 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4e062a0 0x55b3a4e03780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6beaffd700 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e03040 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6be0010fe0 tx=0x7f6be00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c4f7c137b7f757 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6bcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6bcaffd700 1 -- 172.21.15.46:0/1415716609 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be0028470 con 0x55b3a4dfb400 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1415716609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3a4deeb50 con 0x55b3a4dfb400 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6bcaffd700 1 -- 172.21.15.46:0/1415716609 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6be0017070 con 0x55b3a4dfb400 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6bcaffd700 1 -- 172.21.15.46:0/1415716609 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6be00285d0 con 0x55b3a4dfb400 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6bfde63b80 10 client.27158 fetch_fsmap learned FSMap version 40 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1415716609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b3a4b7bbe0 con 0x55b3a4dfb400 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.177+0000 7f6bcaffd700 1 -- 172.21.15.46:0/1415716609 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b3a4deeb50 con 0x55b3a4dfb400 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.178+0000 7f6bcaffd700 1 -- 172.21.15.46:0/1415716609 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6be002f070 con 0x55b3a4dfb400 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.178+0000 7f6bfde63b80 10 client.27158 fetch_fsmap finished waiting for FSMap version 40 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.178+0000 7f6bfde63b80 10 client.27158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.178+0000 7f6bfde63b80 20 client.27158 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:29.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.178+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b3a4e15f60 0x55b3a4e18380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.178+0000 7f6bfde63b80 4 client.27158 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.178+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1415716609 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3a4d46780 con 0x55b3a4e15f60 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.178+0000 7f6beb7fe700 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b3a4e15f60 0x55b3a4e18380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.178+0000 7f6bcaffd700 10 client.27158 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.178+0000 7f6beb7fe700 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b3a4e15f60 0x55b3a4e18380 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa591ff0d946cf1 server_cookie=899108207b17538c in_seq=0 out_seq=0 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bcaffd700 1 -- 172.21.15.46:0/1415716609 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b3a4d46780 con 0x55b3a4e15f60 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bcaffd700 10 client.27158 handle_command_reply: tid=0 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bfde63b80 1 client.27158 shutdown 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1415716609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 msgr2=0x55b3a4e03040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e03040 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6be0010fe0 tx=0x7f6be00047c0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1415716609 shutdown_connections 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b3a4e15f60 0x55b3a4e18380 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e05d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bcaffd700 0 client.27158 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4e062a0 0x55b3a4e03780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1415716609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e03040 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1415716609 >> 172.21.15.46:0/1415716609 conn(0x55b3a4d712a0 msgr2=0x55b3a4df6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1415716609 shutdown_connections 2024-03-18T20:41:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1415716609 wait complete. 2024-03-18T20:41:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.179+0000 7f6bfde63b80 20 client.27158 trim_cache size 0 max 16384 2024-03-18T20:41:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.197+0000 7f6bfde63b80 1 Processor -- start 2024-03-18T20:41:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.197+0000 7f6bfde63b80 1 -- start start 2024-03-18T20:41:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.197+0000 7f6bfde63b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e0ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.197+0000 7f6bfde63b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e01950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.197+0000 7f6bfde63b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e01e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.197+0000 7f6bfde63b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a4e129b0 con 0x55b3a4e062a0 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.197+0000 7f6bfde63b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a4e13590 con 0x55b3a4dfb400 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.197+0000 7f6bfde63b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a4e023d0 con 0x55b3a4dfaae0 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e01e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e01e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36158/0 (socket says 172.21.15.46:36158) 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6bebfff700 1 -- 172.21.15.46:0/3607465403 learned_addr learned my addr 172.21.15.46:0/3607465403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6beb7fe700 1 --2- 172.21.15.46:0/3607465403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e0ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6beaffd700 1 --2- 172.21.15.46:0/3607465403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e01950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6bebfff700 1 -- 172.21.15.46:0/3607465403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 msgr2=0x55b3a4e0ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6bebfff700 1 --2- 172.21.15.46:0/3607465403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e0ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6bebfff700 1 -- 172.21.15.46:0/3607465403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 msgr2=0x55b3a4e01950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6bebfff700 1 --2- 172.21.15.46:0/3607465403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e01950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6bebfff700 1 -- 172.21.15.46:0/3607465403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3a4e1cfb0 con 0x55b3a4e062a0 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6beb7fe700 1 --2- 172.21.15.46:0/3607465403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e0ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:41:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6bebfff700 1 --2- 172.21.15.46:0/3607465403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e01e90 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005cb0 tx=0x7f6be40059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef24f3710944f91a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6bd27fc700 1 -- 172.21.15.46:0/3607465403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4027040 con 0x55b3a4e062a0 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6bd27fc700 1 -- 172.21.15.46:0/3607465403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6be4028040 con 0x55b3a4e062a0 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.198+0000 7f6bd27fc700 1 -- 172.21.15.46:0/3607465403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be400f7a0 con 0x55b3a4e062a0 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.199+0000 7f6bfde63b80 1 -- 172.21.15.46:0/3607465403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 msgr2=0x55b3a4e01e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.199+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/3607465403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e01e90 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005cb0 tx=0x7f6be40059f0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.199+0000 7f6bfde63b80 1 -- 172.21.15.46:0/3607465403 shutdown_connections 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.199+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/3607465403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e01e90 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.199+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/3607465403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e0ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.199+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/3607465403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e01950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.199+0000 7f6bfde63b80 1 -- 172.21.15.46:0/3607465403 >> 172.21.15.46:0/3607465403 conn(0x55b3a4d712a0 msgr2=0x55b3a4dfe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.199+0000 7f6bfde63b80 1 -- 172.21.15.46:0/3607465403 shutdown_connections 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.199+0000 7f6bfde63b80 1 -- 172.21.15.46:0/3607465403 wait complete. 2024-03-18T20:41:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.199+0000 7f6bfde63b80 1 -- start start 2024-03-18T20:41:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6bfde63b80 1 Processor -- start 2024-03-18T20:41:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6bfde63b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e0ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6bfde63b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e0b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6bfde63b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e0b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6bfde63b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a4e0be70 con 0x55b3a4e062a0 2024-03-18T20:41:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6bfde63b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a4e0bfe0 con 0x55b3a4dfb400 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6bfde63b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3a4e69da0 con 0x55b3a4dfaae0 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6bfde63b80 10 client.0 _ll_register_callbacks cb 0x7f6be0010b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e0b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6bebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e0b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36164/0 (socket says 172.21.15.46:36164) 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6bebfff700 1 -- 172.21.15.46:0/1733385564 learned_addr learned my addr 172.21.15.46:0/1733385564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6beaffd700 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e0b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.200+0000 7f6beb7fe700 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e0ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bebfff700 1 -- 172.21.15.46:0/1733385564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 msgr2=0x55b3a4e0ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bebfff700 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e0ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bebfff700 1 -- 172.21.15.46:0/1733385564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 msgr2=0x55b3a4e0b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bebfff700 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e0b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bebfff700 1 -- 172.21.15.46:0/1733385564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3a4bcf430 con 0x55b3a4e062a0 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bebfff700 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e0b7a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6be4004800 tx=0x7f6be4023e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb55014b063f8435 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bcaffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bcaffd700 1 -- 172.21.15.46:0/1733385564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be401d070 con 0x55b3a4e062a0 2024-03-18T20:41:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bcaffd700 1 -- 172.21.15.46:0/1733385564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6be400fc10 con 0x55b3a4e062a0 2024-03-18T20:41:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1733385564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6be401e040 con 0x55b3a4e062a0 2024-03-18T20:41:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bcaffd700 1 -- 172.21.15.46:0/1733385564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6be4027040 con 0x55b3a4e062a0 2024-03-18T20:41:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bfde63b80 10 client.17475 fetch_fsmap learned FSMap version 40 2024-03-18T20:41:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1733385564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b3a4d13f20 con 0x55b3a4e062a0 2024-03-18T20:41:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.201+0000 7f6bcaffd700 1 -- 172.21.15.46:0/1733385564 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6be401e040 con 0x55b3a4e062a0 2024-03-18T20:41:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.202+0000 7f6bcaffd700 1 -- 172.21.15.46:0/1733385564 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 40) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6be4046070 con 0x55b3a4e062a0 2024-03-18T20:41:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.202+0000 7f6bfde63b80 10 client.17475 fetch_fsmap finished waiting for FSMap version 40 2024-03-18T20:41:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.202+0000 7f6bfde63b80 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.202+0000 7f6bfde63b80 20 client.17475 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.202+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b3a4e6b050 0x55b3a4e6d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.202+0000 7f6bfde63b80 4 client.17475 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.202+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1733385564 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b3a4b9cd70 con 0x55b3a4e6b050 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.202+0000 7f6beb7fe700 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b3a4e6b050 0x55b3a4e6d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.202+0000 7f6beb7fe700 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b3a4e6b050 0x55b3a4e6d440 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f615651994ef146d server_cookie=30a65958afced248 in_seq=0 out_seq=0 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.202+0000 7f6bcaffd700 10 client.17475 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.203+0000 7f6bcaffd700 1 -- 172.21.15.46:0/1733385564 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b3a4b9cd70 con 0x55b3a4e6b050 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.203+0000 7f6bcaffd700 10 client.17475 handle_command_reply: tid=0 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.203+0000 7f6bfde63b80 1 client.17475 shutdown 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.203+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1733385564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 msgr2=0x55b3a4e0b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.203+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e0b7a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6be4004800 tx=0x7f6be4023e90 comp rx=0 tx=0).stop 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.203+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1733385564 shutdown_connections 2024-03-18T20:41:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.203+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b3a4e6b050 0x55b3a4e6d440 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.203+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3a4e062a0 0x55b3a4e0b7a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.203+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3a4dfaae0 0x55b3a4e0ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.203+0000 7f6bfde63b80 1 --2- 172.21.15.46:0/1733385564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a4dfb400 0x55b3a4e0b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.203+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1733385564 >> 172.21.15.46:0/1733385564 conn(0x55b3a4d712a0 msgr2=0x55b3a4e025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.203+0000 7f6bcaffd700 0 client.17475 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1733385564 shutdown_connections 2024-03-18T20:41:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bfde63b80 1 -- 172.21.15.46:0/1733385564 wait complete. 2024-03-18T20:41:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bfde63b80 20 client.17475 trim_cache size 0 max 16384 2024-03-18T20:41:29.212 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:41:29.212 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:41:29.212 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "3c1823b7-7cdf-4c0c-91fa-c6f2364a6e73", 2024-03-18T20:41:29.213 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:41:29.213 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:41:29.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bf1cce700 1 -- 172.21.15.46:0/1474822109 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6bd40a1290 msgr2=0x7f6bd40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bf1cce700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6bd40a1290 0x7f6bd40a3750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0747e0 tx=0x7f6be00073d0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bf1cce700 1 -- 172.21.15.46:0/1474822109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec100f50 msgr2=0x7f6bec0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bf1cce700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec100f50 0x7f6bec0730b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00ef10 tx=0x7f6bdc00c3a0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bf1cce700 1 -- 172.21.15.46:0/1474822109 shutdown_connections 2024-03-18T20:41:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bf1cce700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6bd40a1290 0x7f6bd40a3750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bf1cce700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bec073b30 0x7f6bec1a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bf1cce700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bec100f50 0x7f6bec0730b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bf1cce700 1 --2- 172.21.15.46:0/1474822109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bec106070 0x7f6bec0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bf1cce700 1 -- 172.21.15.46:0/1474822109 >> 172.21.15.46:0/1474822109 conn(0x7f6bec0fc7f0 msgr2=0x7f6bec0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bf1cce700 1 -- 172.21.15.46:0/1474822109 shutdown_connections 2024-03-18T20:41:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.204+0000 7f6bf1cce700 1 -- 172.21.15.46:0/1474822109 wait complete. 2024-03-18T20:41:29.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:41:29.223 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:41:29.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:29 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:41:29.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:29 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:41:29.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.365+0000 7f909f310700 1 -- 172.21.15.46:0/4151813493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90981060b0 msgr2=0x7f909810dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.365+0000 7f909f310700 1 --2- 172.21.15.46:0/4151813493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90981060b0 0x7f909810dc30 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f909000b0d0 tx=0x7f909000b3e0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.366+0000 7f909f310700 1 -- 172.21.15.46:0/4151813493 shutdown_connections 2024-03-18T20:41:29.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.366+0000 7f909f310700 1 --2- 172.21.15.46:0/4151813493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90981060b0 0x7f909810dc30 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.366+0000 7f909f310700 1 --2- 172.21.15.46:0/4151813493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9098069280 0x7f9098069660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.366+0000 7f909f310700 1 --2- 172.21.15.46:0/4151813493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9098069c30 0x7f9098105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.366+0000 7f909f310700 1 -- 172.21.15.46:0/4151813493 >> 172.21.15.46:0/4151813493 conn(0x7f9098076d80 msgr2=0x7f9098077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:29.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.366+0000 7f909f310700 1 -- 172.21.15.46:0/4151813493 shutdown_connections 2024-03-18T20:41:29.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.366+0000 7f909f310700 1 -- 172.21.15.46:0/4151813493 wait complete. 2024-03-18T20:41:29.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.366+0000 7f909f310700 1 Processor -- start 2024-03-18T20:41:29.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.366+0000 7f909f310700 1 -- start start 2024-03-18T20:41:29.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909f310700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9098069280 0x7f90980ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909f310700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90981060b0 0x7f90981001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909f310700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90981006f0 0x7f90981a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909f310700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90981a5b00 con 0x7f9098069280 2024-03-18T20:41:29.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909f310700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90981a5c40 con 0x7f90981006f0 2024-03-18T20:41:29.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909f310700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90981a5e00 con 0x7f90981060b0 2024-03-18T20:41:29.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909d0ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9098069280 0x7f90980ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909d8ad700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90981006f0 0x7f90981a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909d0ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9098069280 0x7f90980ffc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36184/0 (socket says 172.21.15.46:36184) 2024-03-18T20:41:29.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909d0ac700 1 -- 172.21.15.46:0/2138793118 learned_addr learned my addr 172.21.15.46:0/2138793118 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909c8ab700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90981060b0 0x7f90981001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909d0ac700 1 -- 172.21.15.46:0/2138793118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90981060b0 msgr2=0x7f90981001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909d0ac700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90981060b0 0x7f90981001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909d0ac700 1 -- 172.21.15.46:0/2138793118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90981006f0 msgr2=0x7f90981a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909d0ac700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90981006f0 0x7f90981a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909d0ac700 1 -- 172.21.15.46:0/2138793118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9090009d00 con 0x7f9098069280 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909d8ad700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90981006f0 0x7f90981a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.367+0000 7f909d0ac700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9098069280 0x7f90980ffc70 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f909400e9a0 tx=0x7f909400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.368+0000 7f908e7fc700 1 -- 172.21.15.46:0/2138793118 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094009790 con 0x7f9098069280 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.368+0000 7f908e7fc700 1 -- 172.21.15.46:0/2138793118 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9094010430 con 0x7f9098069280 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.368+0000 7f908e7fc700 1 -- 172.21.15.46:0/2138793118 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094018680 con 0x7f9098069280 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.368+0000 7f909f310700 1 -- 172.21.15.46:0/2138793118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90981a60e0 con 0x7f9098069280 2024-03-18T20:41:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.368+0000 7f909f310700 1 -- 172.21.15.46:0/2138793118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90981a65a0 con 0x7f9098069280 2024-03-18T20:41:29.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.371+0000 7f908e7fc700 1 -- 172.21.15.46:0/2138793118 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f90940105a0 con 0x7f9098069280 2024-03-18T20:41:29.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.371+0000 7f908e7fc700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90840a1290 0x7f90840a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.371+0000 7f908e7fc700 1 -- 172.21.15.46:0/2138793118 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9094014070 con 0x7f9098069280 2024-03-18T20:41:29.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.371+0000 7f909c8ab700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90840a1290 0x7f90840a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.372+0000 7f909c8ab700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90840a1290 0x7f90840a3750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9098101350 tx=0x7f9088009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:29.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.372+0000 7f90ab4a5b80 1 Processor -- start 2024-03-18T20:41:29.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.372+0000 7f90ab4a5b80 1 -- start start 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.372+0000 7f90ab4a5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6aba60 0x55de4e6abe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.372+0000 7f90ab4a5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6ac380 0x55de4e6b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.372+0000 7f90ab4a5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6b7220 0x55de4e6b9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.372+0000 7f90ab4a5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4e605990 con 0x55de4e6aba60 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.372+0000 7f90ab4a5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4e603350 con 0x55de4e6b7220 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.372+0000 7f90ab4a5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4e5fec40 con 0x55de4e6ac380 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f909d0ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6aba60 0x55de4e6abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f909d0ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6aba60 0x55de4e6abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36188/0 (socket says 172.21.15.46:36188) 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f909d0ac700 1 -- 172.21.15.46:0/2513631007 learned_addr learned my addr 172.21.15.46:0/2513631007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f909c8ab700 1 --2- 172.21.15.46:0/2513631007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6ac380 0x55de4e6b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f909d8ad700 1 --2- 172.21.15.46:0/2513631007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6b7220 0x55de4e6b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f909d0ac700 1 -- 172.21.15.46:0/2513631007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6ac380 msgr2=0x55de4e6b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f909d0ac700 1 --2- 172.21.15.46:0/2513631007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6ac380 0x55de4e6b6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f909d0ac700 1 -- 172.21.15.46:0/2513631007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6b7220 msgr2=0x55de4e6b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f909d0ac700 1 --2- 172.21.15.46:0/2513631007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6b7220 0x55de4e6b9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f909d0ac700 1 -- 172.21.15.46:0/2513631007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de4e4803b0 con 0x55de4e6aba60 2024-03-18T20:41:29.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f909d8ad700 1 --2- 172.21.15.46:0/2513631007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6b7220 0x55de4e6b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:41:29.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f909d0ac700 1 --2- 172.21.15.46:0/2513631007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6aba60 0x55de4e6abe40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9094096a20 tx=0x7f9094099c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574a3a55374ffd5a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:29.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f90837fe700 1 -- 172.21.15.46:0/2513631007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909409dde0 con 0x55de4e6aba60 2024-03-18T20:41:29.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f90837fe700 1 -- 172.21.15.46:0/2513631007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90940a2430 con 0x55de4e6aba60 2024-03-18T20:41:29.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.373+0000 7f90837fe700 1 -- 172.21.15.46:0/2513631007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094099700 con 0x55de4e6aba60 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.374+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/2513631007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6aba60 msgr2=0x55de4e6abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.374+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/2513631007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6aba60 0x55de4e6abe40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9094096a20 tx=0x7f9094099c20 comp rx=0 tx=0).stop 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.374+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/2513631007 shutdown_connections 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.374+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/2513631007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6aba60 0x55de4e6abe40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.374+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/2513631007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6b7220 0x55de4e6b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.374+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/2513631007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6ac380 0x55de4e6b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.374+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/2513631007 >> 172.21.15.46:0/2513631007 conn(0x55de4e622220 msgr2=0x55de4e6a8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.374+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/2513631007 shutdown_connections 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.374+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/2513631007 wait complete. 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.374+0000 7f90ab4a5b80 1 -- start start 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f90ab4a5b80 1 Processor -- start 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f90ab4a5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6aba60 0x55de4e6b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f90ab4a5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6ac380 0x55de4e6b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f90ab4a5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6b7220 0x55de4e6b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f90ab4a5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4e59ad80 con 0x55de4e6b7220 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f90ab4a5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4e59aa20 con 0x55de4e6aba60 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f90ab4a5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4e4826a0 con 0x55de4e6ac380 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f90ab4a5b80 10 client.0 _ll_register_callbacks cb 0x55de4e4063a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f909d8ad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6b7220 0x55de4e6b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f909d8ad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6b7220 0x55de4e6b4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36202/0 (socket says 172.21.15.46:36202) 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f909d8ad700 1 -- 172.21.15.46:0/3033213663 learned_addr learned my addr 172.21.15.46:0/3033213663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f909c8ab700 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6ac380 0x55de4e6b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.375+0000 7f909d0ac700 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6aba60 0x55de4e6b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f909d8ad700 1 -- 172.21.15.46:0/3033213663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6ac380 msgr2=0x55de4e6b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f909d8ad700 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6ac380 0x55de4e6b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f909d8ad700 1 -- 172.21.15.46:0/3033213663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6aba60 msgr2=0x55de4e6b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f909d8ad700 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6aba60 0x55de4e6b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f909d8ad700 1 -- 172.21.15.46:0/3033213663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90981a65a0 con 0x55de4e6b7220 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f909c8ab700 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6ac380 0x55de4e6b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f909d0ac700 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6aba60 0x55de4e6b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f909d8ad700 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6b7220 0x55de4e6b4b30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9090007d60 tx=0x7f9090007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5ce7ae116a0736 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f9080ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f9080ff9700 1 -- 172.21.15.46:0/3033213663 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909000e040 con 0x55de4e6b7220 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f9080ff9700 1 -- 172.21.15.46:0/3033213663 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9090022070 con 0x55de4e6b7220 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3033213663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de4e69fad0 con 0x55de4e6b7220 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f9080ff9700 1 -- 172.21.15.46:0/3033213663 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9090006160 con 0x55de4e6b7220 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f90ab4a5b80 10 client.17487 fetch_fsmap learned FSMap version 40 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.376+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3033213663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55de4e42cb60 con 0x55de4e6b7220 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.377+0000 7f9080ff9700 1 -- 172.21.15.46:0/3033213663 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de4e69fad0 con 0x55de4e6b7220 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.377+0000 7f9080ff9700 1 -- 172.21.15.46:0/3033213663 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 40) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f909000b100 con 0x55de4e6b7220 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.377+0000 7f90ab4a5b80 10 client.17487 fetch_fsmap finished waiting for FSMap version 40 2024-03-18T20:41:29.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.377+0000 7f90ab4a5b80 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.377+0000 7f90ab4a5b80 20 client.17487 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.377+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4e6c6f60 0x55de4e6c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.377+0000 7f90ab4a5b80 4 client.17487 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.377+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3033213663 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de4e5f7700 con 0x55de4e6c6f60 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.377+0000 7f909d0ac700 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4e6c6f60 0x55de4e6c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.377+0000 7f9080ff9700 10 client.17487 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.377+0000 7f909d0ac700 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4e6c6f60 0x55de4e6c9380 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=239fabeb8121dc6a server_cookie=14c6f165cf095073 in_seq=0 out_seq=0 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f9080ff9700 1 -- 172.21.15.46:0/3033213663 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55de4e5f7700 con 0x55de4e6c6f60 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f9080ff9700 10 client.17487 handle_command_reply: tid=0 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f90ab4a5b80 1 client.17487 shutdown 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3033213663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6b7220 msgr2=0x55de4e6b4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6b7220 0x55de4e6b4b30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9090007d60 tx=0x7f9090007d90 comp rx=0 tx=0).stop 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3033213663 shutdown_connections 2024-03-18T20:41:29.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4e6c6f60 0x55de4e6c9380 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6b7220 0x55de4e6b4b30 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6aba60 0x55de4e6b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f9080ff9700 0 client.17487 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/3033213663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6ac380 0x55de4e6b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3033213663 >> 172.21.15.46:0/3033213663 conn(0x55de4e622220 msgr2=0x55de4e6a7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3033213663 shutdown_connections 2024-03-18T20:41:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.378+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3033213663 wait complete. 2024-03-18T20:41:29.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.379+0000 7f90ab4a5b80 20 client.17487 trim_cache size 0 max 16384 2024-03-18T20:41:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.396+0000 7f90ab4a5b80 1 Processor -- start 2024-03-18T20:41:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.396+0000 7f90ab4a5b80 1 -- start start 2024-03-18T20:41:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.396+0000 7f90ab4a5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6aba60 0x55de4e6bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.396+0000 7f90ab4a5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 0x55de4e6af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.396+0000 7f90ab4a5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6b7220 0x55de4e6af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.396+0000 7f90ab4a5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4e6b5660 con 0x55de4e6ac380 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.396+0000 7f90ab4a5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4e6b5010 con 0x55de4e6aba60 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.396+0000 7f90ab4a5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4e6ad5d0 con 0x55de4e6b7220 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f909c8ab700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 0x55de4e6af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f909c8ab700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 0x55de4e6af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36212/0 (socket says 172.21.15.46:36212) 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f909c8ab700 1 -- 172.21.15.46:0/1118308596 learned_addr learned my addr 172.21.15.46:0/1118308596 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f909d8ad700 1 --2- 172.21.15.46:0/1118308596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6b7220 0x55de4e6af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f909d0ac700 1 --2- 172.21.15.46:0/1118308596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6aba60 0x55de4e6bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f909c8ab700 1 -- 172.21.15.46:0/1118308596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6b7220 msgr2=0x55de4e6af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f909c8ab700 1 --2- 172.21.15.46:0/1118308596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6b7220 0x55de4e6af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f909c8ab700 1 -- 172.21.15.46:0/1118308596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6aba60 msgr2=0x55de4e6bde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f909c8ab700 1 --2- 172.21.15.46:0/1118308596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6aba60 0x55de4e6bde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f909c8ab700 1 -- 172.21.15.46:0/1118308596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de4e5c4ea0 con 0x55de4e6ac380 2024-03-18T20:41:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f909c8ab700 1 --2- 172.21.15.46:0/1118308596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 0x55de4e6af380 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9098067bd0 tx=0x7f9088014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fe1f18e75fd1e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:29.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f9083fff700 1 -- 172.21.15.46:0/1118308596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088024070 con 0x55de4e6ac380 2024-03-18T20:41:29.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f9083fff700 1 -- 172.21.15.46:0/1118308596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9088004510 con 0x55de4e6ac380 2024-03-18T20:41:29.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.397+0000 7f9083fff700 1 -- 172.21.15.46:0/1118308596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90880283c0 con 0x55de4e6ac380 2024-03-18T20:41:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.398+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/1118308596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 msgr2=0x55de4e6af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.398+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/1118308596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 0x55de4e6af380 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9098067bd0 tx=0x7f9088014000 comp rx=0 tx=0).stop 2024-03-18T20:41:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.398+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/1118308596 shutdown_connections 2024-03-18T20:41:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.398+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/1118308596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6b7220 0x55de4e6af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.398+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/1118308596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6aba60 0x55de4e6bde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.398+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/1118308596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 0x55de4e6af380 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.398+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/1118308596 >> 172.21.15.46:0/1118308596 conn(0x55de4e622220 msgr2=0x55de4e6c25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.398+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/1118308596 shutdown_connections 2024-03-18T20:41:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.398+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/1118308596 wait complete. 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.398+0000 7f90ab4a5b80 1 -- start start 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.398+0000 7f90ab4a5b80 1 Processor -- start 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f90ab4a5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6aba60 0x55de4e70ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f90ab4a5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 0x55de4e70f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f90ab4a5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6b7220 0x55de4e719da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f90ab4a5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4e70fb40 con 0x55de4e6ac380 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f90ab4a5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4e71a2e0 con 0x55de4e6b7220 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f90ab4a5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4e71a420 con 0x55de4e6aba60 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f90ab4a5b80 10 client.0 _ll_register_callbacks cb 0x7f9090009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f909c8ab700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 0x55de4e70f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f909c8ab700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 0x55de4e70f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36218/0 (socket says 172.21.15.46:36218) 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f909d8ad700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6b7220 0x55de4e719da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f909c8ab700 1 -- 172.21.15.46:0/3136235194 learned_addr learned my addr 172.21.15.46:0/3136235194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f909d0ac700 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6aba60 0x55de4e70ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f909c8ab700 1 -- 172.21.15.46:0/3136235194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6aba60 msgr2=0x55de4e70ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f909c8ab700 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6aba60 0x55de4e70ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f909c8ab700 1 -- 172.21.15.46:0/3136235194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6b7220 msgr2=0x55de4e719da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f909c8ab700 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6b7220 0x55de4e719da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f909c8ab700 1 -- 172.21.15.46:0/3136235194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9088017070 con 0x55de4e6ac380 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f909d8ad700 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6b7220 0x55de4e719da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f909c8ab700 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 0x55de4e70f450 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9088003b30 tx=0x7f9088003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4f0ae9c1705b56 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.399+0000 7f9080ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f9080ff9700 1 -- 172.21.15.46:0/3136235194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088024070 con 0x55de4e6ac380 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f9080ff9700 1 -- 172.21.15.46:0/3136235194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f9088004260 con 0x55de4e6ac380 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3136235194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90940a2430 con 0x55de4e6ac380 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f9080ff9700 1 -- 172.21.15.46:0/3136235194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9088003ed0 con 0x55de4e6ac380 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f90ab4a5b80 10 client.17493 fetch_fsmap learned FSMap version 40 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3136235194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55de4e484cd0 con 0x55de4e6ac380 2024-03-18T20:41:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f9080ff9700 1 -- 172.21.15.46:0/3136235194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f908803e020 con 0x55de4e6ac380 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f9080ff9700 1 -- 172.21.15.46:0/3136235194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 40) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9088004cc0 con 0x55de4e6ac380 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f90ab4a5b80 10 client.17493 fetch_fsmap finished waiting for FSMap version 40 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f90ab4a5b80 10 client.17493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f90ab4a5b80 20 client.17493 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4e71bc20 0x55de4e71e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f90ab4a5b80 4 client.17493 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3136235194 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55de4e44dcf0 con 0x55de4e71bc20 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.400+0000 7f909d0ac700 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4e71bc20 0x55de4e71e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f909d0ac700 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4e71bc20 0x55de4e71e040 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b356b1b134013f4 server_cookie=56cc03687cb8d556 in_seq=0 out_seq=0 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f9080ff9700 10 client.17493 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f9080ff9700 1 -- 172.21.15.46:0/3136235194 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55de4e44dcf0 con 0x55de4e71bc20 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f9080ff9700 10 client.17493 handle_command_reply: tid=0 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f90ab4a5b80 1 client.17493 shutdown 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3136235194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 msgr2=0x55de4e70f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 0x55de4e70f450 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9088003b30 tx=0x7f9088003b60 comp rx=0 tx=0).stop 2024-03-18T20:41:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3136235194 shutdown_connections 2024-03-18T20:41:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4e71bc20 0x55de4e71e040 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4e6b7220 0x55de4e719da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4e6aba60 0x55de4e70ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f9080ff9700 0 client.17493 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f90ab4a5b80 1 --2- 172.21.15.46:0/3136235194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4e6ac380 0x55de4e70f450 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3136235194 >> 172.21.15.46:0/3136235194 conn(0x55de4e622220 msgr2=0x55de4e6af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3136235194 shutdown_connections 2024-03-18T20:41:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.401+0000 7f90ab4a5b80 1 -- 172.21.15.46:0/3136235194 wait complete. 2024-03-18T20:41:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f90ab4a5b80 20 client.17493 trim_cache size 0 max 16384 2024-03-18T20:41:29.410 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:41:29.411 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:41:29.411 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:41:29.411 INFO:teuthology.orchestra.run.smithi046.stdout: "3c1823b7-7cdf-4c0c-91fa-c6f2364a6e73": { 2024-03-18T20:41:29.411 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:41:29.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "3c1823b7-7cdf-4c0c-91fa-c6f2364a6e73", 2024-03-18T20:41:29.411 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:41:29.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:41:29.411 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:41:29.411 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:41:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f909f310700 1 -- 172.21.15.46:0/2138793118 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90840a1290 msgr2=0x7f90840a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f909f310700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90840a1290 0x7f90840a3750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9098101350 tx=0x7f9088009260 comp rx=0 tx=0).stop 2024-03-18T20:41:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f909f310700 1 -- 172.21.15.46:0/2138793118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9098069280 msgr2=0x7f90980ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f909f310700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9098069280 0x7f90980ffc70 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f909400e9a0 tx=0x7f909400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f909f310700 1 -- 172.21.15.46:0/2138793118 shutdown_connections 2024-03-18T20:41:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f909f310700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90840a1290 0x7f90840a3750 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f909f310700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9098069280 0x7f90980ffc70 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f909f310700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90981006f0 0x7f90981a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f909f310700 1 --2- 172.21.15.46:0/2138793118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90981060b0 0x7f90981001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f909f310700 1 -- 172.21.15.46:0/2138793118 >> 172.21.15.46:0/2138793118 conn(0x7f9098076d80 msgr2=0x7f90980ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f909f310700 1 -- 172.21.15.46:0/2138793118 shutdown_connections 2024-03-18T20:41:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:29.402+0000 7f909f310700 1 -- 172.21.15.46:0/2138793118 wait complete. 2024-03-18T20:41:29.418 INFO:tasks.cephfs.filesystem:scrub status for tag:3c1823b7-7cdf-4c0c-91fa-c6f2364a6e73 - {'path': '/', 'tag': '3c1823b7-7cdf-4c0c-91fa-c6f2364a6e73', 'options': 'recursive,force'} 2024-03-18T20:41:29.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:29 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:41:29.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 153341 2.66 MB/sec execute 144 sec latency 360.184 ms 2024-03-18T20:41:30.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: pgmap v165: 97 pgs: 97 active+clean; 67 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-03-18T20:41:30.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:41:30.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:41:30.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:41:30.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:41:30.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:41:30.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:41:30.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:41:30.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:30 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: pgmap v165: 97 pgs: 97 active+clean; 67 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-03-18T20:41:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:41:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:41:30.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:41:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:41:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:41:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:41:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:41:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:30 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: pgmap v165: 97 pgs: 97 active+clean; 67 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-03-18T20:41:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:41:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:41:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:41:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:41:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:41:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:41:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:41:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:30 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:30.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 153341 2.64 MB/sec execute 145 sec latency 1360.290 ms 2024-03-18T20:41:31.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:31 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:41:31.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:31 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:31.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:31 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:41:31.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:31 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:31.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:31 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:41:31.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:31 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:31.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:31 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:41:31.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:31 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:31.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:31 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:41:31.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:31 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:31.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:31 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:41:31.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:31 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:31.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:31 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:41:31.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:31 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:31.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:31 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:41:31.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:31 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:31.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:31 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:41:31.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:31 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:41:31.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 153341 2.62 MB/sec execute 146 sec latency 2360.385 ms 2024-03-18T20:41:32.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:32 smithi155 ceph-mon[48344]: pgmap v166: 97 pgs: 97 active+clean; 67 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.0 MiB/s wr, 91 op/s 2024-03-18T20:41:32.302 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:32 smithi046 ceph-mon[43495]: pgmap v166: 97 pgs: 97 active+clean; 67 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.0 MiB/s wr, 91 op/s 2024-03-18T20:41:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:32 smithi073 ceph-mon[47384]: pgmap v166: 97 pgs: 97 active+clean; 67 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.0 MiB/s wr, 91 op/s 2024-03-18T20:41:32.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 154585 2.64 MB/sec execute 147 sec latency 2710.985 ms 2024-03-18T20:41:33.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 156454 2.71 MB/sec execute 148 sec latency 16.093 ms 2024-03-18T20:41:34.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:34 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:41:34.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:34 smithi155 ceph-mon[48344]: pgmap v167: 97 pgs: 97 active+clean; 65 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-03-18T20:41:34.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:34 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:41:34.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:34 smithi046 ceph-mon[43495]: pgmap v167: 97 pgs: 97 active+clean; 65 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-03-18T20:41:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:34 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:41:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:34 smithi073 ceph-mon[47384]: pgmap v167: 97 pgs: 97 active+clean; 65 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-03-18T20:41:34.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 157627 2.73 MB/sec execute 149 sec latency 18.502 ms 2024-03-18T20:41:35.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 159849 2.80 MB/sec execute 150 sec latency 14.212 ms 2024-03-18T20:41:36.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:35 smithi073 ceph-mon[47384]: pgmap v168: 97 pgs: 97 active+clean; 65 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 982 KiB/s wr, 118 op/s 2024-03-18T20:41:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:35 smithi155 ceph-mon[48344]: pgmap v168: 97 pgs: 97 active+clean; 65 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 982 KiB/s wr, 118 op/s 2024-03-18T20:41:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:35 smithi046 ceph-mon[43495]: pgmap v168: 97 pgs: 97 active+clean; 65 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 982 KiB/s wr, 118 op/s 2024-03-18T20:41:36.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 161075 2.82 MB/sec execute 151 sec latency 14.261 ms 2024-03-18T20:41:37.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 163010 2.86 MB/sec execute 152 sec latency 25.233 ms 2024-03-18T20:41:38.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:38 smithi073 ceph-mon[47384]: pgmap v169: 97 pgs: 97 active+clean; 65 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 982 KiB/s wr, 118 op/s 2024-03-18T20:41:38.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:38 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:41:38.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:38 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:41:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:38 smithi155 ceph-mon[48344]: pgmap v169: 97 pgs: 97 active+clean; 65 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 982 KiB/s wr, 118 op/s 2024-03-18T20:41:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:38 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:41:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:38 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:41:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:38 smithi046 ceph-mon[43495]: pgmap v169: 97 pgs: 97 active+clean; 65 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 982 KiB/s wr, 118 op/s 2024-03-18T20:41:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:38 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:41:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:38 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:41:38.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 164186 2.88 MB/sec execute 153 sec latency 13.401 ms 2024-03-18T20:41:39.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:39 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:41:39.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:39 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:41:39.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:39 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:41:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:39 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:41:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:39 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:41:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:39 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:41:39.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:39 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:41:39.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:39 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:41:39.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:39 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:41:39.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 166110 2.94 MB/sec execute 154 sec latency 19.608 ms 2024-03-18T20:41:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.027+0000 7fd2813cf700 1 -- 172.21.15.46:0/2245091533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c106e30 msgr2=0x7fd27c107210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.027+0000 7fd2813cf700 1 --2- 172.21.15.46:0/2245091533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c106e30 0x7fd27c107210 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0099c0 tx=0x7fd26c009cd0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.028+0000 7fd2813cf700 1 -- 172.21.15.46:0/2245091533 shutdown_connections 2024-03-18T20:41:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.028+0000 7fd2813cf700 1 --2- 172.21.15.46:0/2245091533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c106e30 0x7fd27c107210 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.028+0000 7fd2813cf700 1 --2- 172.21.15.46:0/2245091533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd27c0ffbe0 0x7fd27c10dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.028+0000 7fd2813cf700 1 --2- 172.21.15.46:0/2245091533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd27c0ff220 0x7fd27c0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.028+0000 7fd2813cf700 1 -- 172.21.15.46:0/2245091533 >> 172.21.15.46:0/2245091533 conn(0x7fd27c0761b0 msgr2=0x7fd27c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.028+0000 7fd2813cf700 1 -- 172.21.15.46:0/2245091533 shutdown_connections 2024-03-18T20:41:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.028+0000 7fd2813cf700 1 -- 172.21.15.46:0/2245091533 wait complete. 2024-03-18T20:41:40.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.028+0000 7fd2813cf700 1 Processor -- start 2024-03-18T20:41:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd2813cf700 1 -- start start 2024-03-18T20:41:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd2813cf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd27c0ff220 0x7fd27c100400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd2813cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c0ffbe0 0x7fd27c100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd2813cf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd27c106e30 0x7fd27c100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd2813cf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c1a5600 con 0x7fd27c0ffbe0 2024-03-18T20:41:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd2813cf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c1a5770 con 0x7fd27c0ff220 2024-03-18T20:41:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd2813cf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c1a5930 con 0x7fd27c106e30 2024-03-18T20:41:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd27a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c0ffbe0 0x7fd27c100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd27affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd27c0ff220 0x7fd27c100400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd27a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c0ffbe0 0x7fd27c100940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49150/0 (socket says 172.21.15.46:49150) 2024-03-18T20:41:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd27affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd27c0ff220 0x7fd27c100400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:43782/0 (socket says 172.21.15.46:43782) 2024-03-18T20:41:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd27a7fc700 1 -- 172.21.15.46:0/3422558200 learned_addr learned my addr 172.21.15.46:0/3422558200 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd27b7fe700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd27c106e30 0x7fd27c100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd27a7fc700 1 -- 172.21.15.46:0/3422558200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd27c106e30 msgr2=0x7fd27c100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd27c106e30 0x7fd27c100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd27a7fc700 1 -- 172.21.15.46:0/3422558200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd27c0ff220 msgr2=0x7fd27c100400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd27c0ff220 0x7fd27c100400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.029+0000 7fd27a7fc700 1 -- 172.21.15.46:0/3422558200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274009c40 con 0x7fd27c0ffbe0 2024-03-18T20:41:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.030+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c0ffbe0 0x7fd27c100940 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd27000e9a0 tx=0x7fd27000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.030+0000 7fd263fff700 1 -- 172.21.15.46:0/3422558200 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270009790 con 0x7fd27c0ffbe0 2024-03-18T20:41:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.030+0000 7fd2813cf700 1 -- 172.21.15.46:0/3422558200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26c0095f0 con 0x7fd27c0ffbe0 2024-03-18T20:41:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.030+0000 7fd2813cf700 1 -- 172.21.15.46:0/3422558200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c1a5fe0 con 0x7fd27c0ffbe0 2024-03-18T20:41:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.030+0000 7fd263fff700 1 -- 172.21.15.46:0/3422558200 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd270010430 con 0x7fd27c0ffbe0 2024-03-18T20:41:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.030+0000 7fd263fff700 1 -- 172.21.15.46:0/3422558200 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270018680 con 0x7fd27c0ffbe0 2024-03-18T20:41:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.033+0000 7fd263fff700 1 -- 172.21.15.46:0/3422558200 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd2700187e0 con 0x7fd27c0ffbe0 2024-03-18T20:41:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.034+0000 7fd263fff700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2640a1290 0x7fd2640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.034+0000 7fd27affd700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2640a1290 0x7fd2640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.034+0000 7fd263fff700 1 -- 172.21.15.46:0/3422558200 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd270014070 con 0x7fd27c0ffbe0 2024-03-18T20:41:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.034+0000 7fd28d564b80 1 Processor -- start 2024-03-18T20:41:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.034+0000 7fd28d564b80 1 -- start start 2024-03-18T20:41:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.034+0000 7fd27affd700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2640a1290 0x7fd2640a3750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009fd0 tx=0x7fd26c019040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd28d564b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9536570 0x5638b9536950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd28d564b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536e90 0x5638b9541870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd28d564b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9541db0 0x5638b95441a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd28d564b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b9490930 con 0x5638b9536e90 2024-03-18T20:41:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd28d564b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b948e2f0 con 0x5638b9541db0 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd28d564b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b9489be0 con 0x5638b9536570 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd27affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9536570 0x5638b9536950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd27affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9536570 0x5638b9536950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59320/0 (socket says 172.21.15.46:59320) 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd27affd700 1 -- 172.21.15.46:0/994514473 learned_addr learned my addr 172.21.15.46:0/994514473 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd27b7fe700 1 --2- 172.21.15.46:0/994514473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9541db0 0x5638b95441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/994514473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536e90 0x5638b9541870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd27affd700 1 -- 172.21.15.46:0/994514473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9541db0 msgr2=0x5638b95441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd27affd700 1 --2- 172.21.15.46:0/994514473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9541db0 0x5638b95441a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd27affd700 1 -- 172.21.15.46:0/994514473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536e90 msgr2=0x5638b9541870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd27affd700 1 --2- 172.21.15.46:0/994514473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536e90 0x5638b9541870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd27affd700 1 -- 172.21.15.46:0/994514473 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638b930b350 con 0x5638b9536570 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd27b7fe700 1 --2- 172.21.15.46:0/994514473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9541db0 0x5638b95441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.035+0000 7fd27affd700 1 --2- 172.21.15.46:0/994514473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9536570 0x5638b9536950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0099c0 tx=0x7fd26c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9369d4174421b80c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.036+0000 7fd2617fa700 1 -- 172.21.15.46:0/994514473 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00f8c0 con 0x5638b9536570 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.036+0000 7fd2617fa700 1 -- 172.21.15.46:0/994514473 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd26c01d040 con 0x5638b9536570 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.036+0000 7fd2617fa700 1 -- 172.21.15.46:0/994514473 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c023df0 con 0x5638b9536570 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.036+0000 7fd28d564b80 1 -- 172.21.15.46:0/994514473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9536570 msgr2=0x5638b9536950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.036+0000 7fd28d564b80 1 --2- 172.21.15.46:0/994514473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9536570 0x5638b9536950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0099c0 tx=0x7fd26c01f000 comp rx=0 tx=0).stop 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.036+0000 7fd28d564b80 1 -- 172.21.15.46:0/994514473 shutdown_connections 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.036+0000 7fd28d564b80 1 --2- 172.21.15.46:0/994514473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9541db0 0x5638b95441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.036+0000 7fd28d564b80 1 --2- 172.21.15.46:0/994514473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9536570 0x5638b9536950 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.036+0000 7fd28d564b80 1 --2- 172.21.15.46:0/994514473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536e90 0x5638b9541870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.036+0000 7fd28d564b80 1 -- 172.21.15.46:0/994514473 >> 172.21.15.46:0/994514473 conn(0x5638b94ad1c0 msgr2=0x5638b9532a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.036+0000 7fd28d564b80 1 -- 172.21.15.46:0/994514473 shutdown_connections 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd28d564b80 1 -- 172.21.15.46:0/994514473 wait complete. 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd28d564b80 1 -- start start 2024-03-18T20:41:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd28d564b80 1 Processor -- start 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd28d564b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536570 0x5638b9541820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd28d564b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9536e90 0x5638b953eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd28d564b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9541db0 0x5638b953f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd28d564b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b9425d20 con 0x5638b9541db0 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd28d564b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b94259c0 con 0x5638b9536570 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd28d564b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b930d640 con 0x5638b9536e90 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd28d564b80 10 client.0 _ll_register_callbacks cb 0x5638b9291340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd27affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536570 0x5638b9541820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd27b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9541db0 0x5638b953f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd27affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536570 0x5638b9541820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:43806/0 (socket says 172.21.15.46:43806) 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.037+0000 7fd27affd700 1 -- 172.21.15.46:0/569419772 learned_addr learned my addr 172.21.15.46:0/569419772 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/569419772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9536e90 0x5638b953eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd27affd700 1 -- 172.21.15.46:0/569419772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9536e90 msgr2=0x5638b953eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd27affd700 1 --2- 172.21.15.46:0/569419772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9536e90 0x5638b953eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd27affd700 1 -- 172.21.15.46:0/569419772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9541db0 msgr2=0x5638b953f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd27affd700 1 --2- 172.21.15.46:0/569419772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9541db0 0x5638b953f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd27affd700 1 -- 172.21.15.46:0/569419772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27400e330 con 0x5638b9536570 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd27affd700 1 --2- 172.21.15.46:0/569419772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536570 0x5638b9541820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0239d0 tx=0x7fd26c005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5635be67ebf9a117 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd25eff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd25eff5700 1 -- 172.21.15.46:0/569419772 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c023cb0 con 0x5638b9536570 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd25eff5700 1 -- 172.21.15.46:0/569419772 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd26c023e10 con 0x5638b9536570 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd28d564b80 1 -- 172.21.15.46:0/569419772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638b952a6c0 con 0x5638b9536570 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd25eff5700 1 -- 172.21.15.46:0/569419772 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd26c00f8c0 con 0x5638b9536570 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd28d564b80 10 client.27194 fetch_fsmap learned FSMap version 41 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.038+0000 7fd28d564b80 1 -- 172.21.15.46:0/569419772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fd27c10e4f0 con 0x5638b9536570 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.039+0000 7fd25eff5700 1 -- 172.21.15.46:0/569419772 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638b952a6c0 con 0x5638b9536570 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.039+0000 7fd25eff5700 1 -- 172.21.15.46:0/569419772 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd26c046070 con 0x5638b9536570 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.039+0000 7fd28d564b80 10 client.27194 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T20:41:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.039+0000 7fd28d564b80 10 client.27194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.039+0000 7fd28d564b80 20 client.27194 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.039+0000 7fd28d564b80 1 --2- 172.21.15.46:0/569419772 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5638b95515c0 0x5638b95539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.039+0000 7fd28d564b80 4 client.27194 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.039+0000 7fd28d564b80 1 -- 172.21.15.46:0/569419772 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638b94826a0 con 0x5638b95515c0 2024-03-18T20:41:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.039+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/569419772 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5638b95515c0 0x5638b95539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd25eff5700 10 client.27194 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/569419772 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5638b95515c0 0x5638b95539e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a096e31da1ed368 server_cookie=9ab75ee88e96e03 in_seq=0 out_seq=0 2024-03-18T20:41:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd25eff5700 1 -- 172.21.15.46:0/569419772 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5638b94826a0 con 0x5638b95515c0 2024-03-18T20:41:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd25eff5700 10 client.27194 handle_command_reply: tid=0 2024-03-18T20:41:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd28d564b80 1 client.27194 shutdown 2024-03-18T20:41:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd28d564b80 1 -- 172.21.15.46:0/569419772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536570 msgr2=0x5638b9541820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd28d564b80 1 --2- 172.21.15.46:0/569419772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536570 0x5638b9541820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0239d0 tx=0x7fd26c005c70 comp rx=0 tx=0).stop 2024-03-18T20:41:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd28d564b80 1 -- 172.21.15.46:0/569419772 shutdown_connections 2024-03-18T20:41:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd28d564b80 1 --2- 172.21.15.46:0/569419772 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5638b95515c0 0x5638b95539e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd28d564b80 1 --2- 172.21.15.46:0/569419772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9541db0 0x5638b953f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd28d564b80 1 --2- 172.21.15.46:0/569419772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536570 0x5638b9541820 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd28d564b80 1 --2- 172.21.15.46:0/569419772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9536e90 0x5638b953eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.040+0000 7fd28d564b80 1 -- 172.21.15.46:0/569419772 >> 172.21.15.46:0/569419772 conn(0x5638b94ad1c0 msgr2=0x5638b9532650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.041+0000 7fd25eff5700 0 client.27194 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.041+0000 7fd28d564b80 1 -- 172.21.15.46:0/569419772 shutdown_connections 2024-03-18T20:41:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.041+0000 7fd28d564b80 1 -- 172.21.15.46:0/569419772 wait complete. 2024-03-18T20:41:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.041+0000 7fd28d564b80 20 client.27194 trim_cache size 0 max 16384 2024-03-18T20:41:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.059+0000 7fd28d564b80 1 Processor -- start 2024-03-18T20:41:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.059+0000 7fd28d564b80 1 -- start start 2024-03-18T20:41:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd28d564b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536570 0x5638b95441a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd28d564b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536e90 0x5638b9548a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd28d564b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9541db0 0x5638b954d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd28d564b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b952f1d0 con 0x5638b9536570 2024-03-18T20:41:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd28d564b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b9537850 con 0x5638b9536e90 2024-03-18T20:41:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd28d564b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b9538130 con 0x5638b9541db0 2024-03-18T20:41:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd27a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536e90 0x5638b9548a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd27a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536e90 0x5638b9548a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:43814/0 (socket says 172.21.15.46:43814) 2024-03-18T20:41:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd27a7fc700 1 -- 172.21.15.46:0/1043321559 learned_addr learned my addr 172.21.15.46:0/1043321559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd27affd700 1 --2- 172.21.15.46:0/1043321559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536570 0x5638b95441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd27b7fe700 1 --2- 172.21.15.46:0/1043321559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9541db0 0x5638b954d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd27a7fc700 1 -- 172.21.15.46:0/1043321559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9541db0 msgr2=0x5638b954d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/1043321559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9541db0 0x5638b954d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd27a7fc700 1 -- 172.21.15.46:0/1043321559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536570 msgr2=0x5638b95441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/1043321559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536570 0x5638b95441a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd27a7fc700 1 -- 172.21.15.46:0/1043321559 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638b95585d0 con 0x5638b9536e90 2024-03-18T20:41:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.060+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/1043321559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536e90 0x5638b9548a40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd270096a00 tx=0x7fd2700cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7825430dcc174f88 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.061+0000 7fd261ffb700 1 -- 172.21.15.46:0/1043321559 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2700a2410 con 0x5638b9536e90 2024-03-18T20:41:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.061+0000 7fd261ffb700 1 -- 172.21.15.46:0/1043321559 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd270010e90 con 0x5638b9536e90 2024-03-18T20:41:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.061+0000 7fd261ffb700 1 -- 172.21.15.46:0/1043321559 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27009a4a0 con 0x5638b9536e90 2024-03-18T20:41:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.061+0000 7fd28d564b80 1 -- 172.21.15.46:0/1043321559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536e90 msgr2=0x5638b9548a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.061+0000 7fd28d564b80 1 --2- 172.21.15.46:0/1043321559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536e90 0x5638b9548a40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd270096a00 tx=0x7fd2700cc040 comp rx=0 tx=0).stop 2024-03-18T20:41:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.062+0000 7fd28d564b80 1 -- 172.21.15.46:0/1043321559 shutdown_connections 2024-03-18T20:41:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.062+0000 7fd28d564b80 1 --2- 172.21.15.46:0/1043321559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536570 0x5638b95441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.062+0000 7fd28d564b80 1 --2- 172.21.15.46:0/1043321559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9541db0 0x5638b954d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.062+0000 7fd28d564b80 1 --2- 172.21.15.46:0/1043321559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536e90 0x5638b9548a40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.062+0000 7fd28d564b80 1 -- 172.21.15.46:0/1043321559 >> 172.21.15.46:0/1043321559 conn(0x5638b94ad1c0 msgr2=0x5638b954d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.062+0000 7fd28d564b80 1 -- 172.21.15.46:0/1043321559 shutdown_connections 2024-03-18T20:41:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.062+0000 7fd28d564b80 1 -- 172.21.15.46:0/1043321559 wait complete. 2024-03-18T20:41:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.062+0000 7fd28d564b80 1 -- start start 2024-03-18T20:41:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.062+0000 7fd28d564b80 1 Processor -- start 2024-03-18T20:41:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.062+0000 7fd28d564b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536570 0x5638b95481a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd28d564b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536e90 0x5638b95486e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd28d564b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9541db0 0x5638b9546a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd28d564b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b9547130 con 0x5638b9536570 2024-03-18T20:41:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd28d564b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b95472a0 con 0x5638b9536e90 2024-03-18T20:41:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd28d564b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b9547460 con 0x5638b9541db0 2024-03-18T20:41:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd28d564b80 10 client.0 _ll_register_callbacks cb 0x7fd2700962d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd27affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536570 0x5638b95481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd27affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536570 0x5638b95481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49196/0 (socket says 172.21.15.46:49196) 2024-03-18T20:41:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd27affd700 1 -- 172.21.15.46:0/4264073702 learned_addr learned my addr 172.21.15.46:0/4264073702 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd27b7fe700 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9541db0 0x5638b9546a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536e90 0x5638b95486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd27affd700 1 -- 172.21.15.46:0/4264073702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9541db0 msgr2=0x5638b9546a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd27affd700 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9541db0 0x5638b9546a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd27affd700 1 -- 172.21.15.46:0/4264073702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536e90 msgr2=0x5638b95486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd27affd700 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536e90 0x5638b95486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd27affd700 1 -- 172.21.15.46:0/4264073702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c1a5fe0 con 0x5638b9536570 2024-03-18T20:41:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd27affd700 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536570 0x5638b95481a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00b9d0 tx=0x7fd26c03f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6123c9d4e4a1c0cf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.063+0000 7fd25eff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd25eff5700 1 -- 172.21.15.46:0/4264073702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00f940 con 0x5638b9536570 2024-03-18T20:41:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd28d564b80 1 -- 172.21.15.46:0/4264073702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd26c01d040 con 0x5638b9536570 2024-03-18T20:41:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd25eff5700 1 -- 172.21.15.46:0/4264073702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fd26c023800 con 0x5638b9536570 2024-03-18T20:41:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd25eff5700 1 -- 172.21.15.46:0/4264073702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd26c00be90 con 0x5638b9536570 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd28d564b80 10 client.17511 fetch_fsmap learned FSMap version 41 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd28d564b80 1 -- 172.21.15.46:0/4264073702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5638b92b7b00 con 0x5638b9536570 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd25eff5700 1 -- 172.21.15.46:0/4264073702 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd26c01d040 con 0x5638b9536570 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd25eff5700 1 -- 172.21.15.46:0/4264073702 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 41) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd26c046070 con 0x5638b9536570 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd28d564b80 10 client.17511 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd28d564b80 10 client.17511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd28d564b80 20 client.17511 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd28d564b80 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5638b95a61c0 0x5638b95a85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd28d564b80 4 client.17511 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.064+0000 7fd28d564b80 1 -- 172.21.15.46:0/4264073702 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5638b92d8c90 con 0x5638b95a61c0 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.065+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5638b95a61c0 0x5638b95a85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.065+0000 7fd25eff5700 10 client.17511 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.065+0000 7fd27a7fc700 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5638b95a61c0 0x5638b95a85e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ea6a3f8a2143212 server_cookie=ce45947ebb1d0c23 in_seq=0 out_seq=0 2024-03-18T20:41:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.095+0000 7fd25eff5700 1 -- 172.21.15.46:0/4264073702 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5638b92d8c90 con 0x5638b95a61c0 2024-03-18T20:41:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.095+0000 7fd25eff5700 10 client.17511 handle_command_reply: tid=0 2024-03-18T20:41:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.095+0000 7fd28d564b80 1 client.17511 shutdown 2024-03-18T20:41:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.095+0000 7fd28d564b80 1 -- 172.21.15.46:0/4264073702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536570 msgr2=0x5638b95481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.095+0000 7fd28d564b80 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536570 0x5638b95481a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00b9d0 tx=0x7fd26c03f6d0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.096+0000 7fd28d564b80 1 -- 172.21.15.46:0/4264073702 shutdown_connections 2024-03-18T20:41:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.096+0000 7fd28d564b80 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5638b95a61c0 0x5638b95a85e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.096+0000 7fd28d564b80 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5638b9536570 0x5638b95481a0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.096+0000 7fd28d564b80 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638b9541db0 0x5638b9546a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.096+0000 7fd25eff5700 0 client.17511 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.096+0000 7fd28d564b80 1 --2- 172.21.15.46:0/4264073702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5638b9536e90 0x5638b95486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.096+0000 7fd28d564b80 1 -- 172.21.15.46:0/4264073702 >> 172.21.15.46:0/4264073702 conn(0x5638b94ad1c0 msgr2=0x5638b954d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.096+0000 7fd28d564b80 1 -- 172.21.15.46:0/4264073702 shutdown_connections 2024-03-18T20:41:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.096+0000 7fd28d564b80 1 -- 172.21.15.46:0/4264073702 wait complete. 2024-03-18T20:41:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.096+0000 7fd28d564b80 20 client.17511 trim_cache size 0 max 16384 2024-03-18T20:41:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:41:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:41:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:41:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:41:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.097+0000 7fd2813cf700 1 -- 172.21.15.46:0/3422558200 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2640a1290 msgr2=0x7fd2640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.097+0000 7fd2813cf700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2640a1290 0x7fd2640a3750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009fd0 tx=0x7fd26c019040 comp rx=0 tx=0).stop 2024-03-18T20:41:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.097+0000 7fd2813cf700 1 -- 172.21.15.46:0/3422558200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c0ffbe0 msgr2=0x7fd27c100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.097+0000 7fd2813cf700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c0ffbe0 0x7fd27c100940 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd27000e9a0 tx=0x7fd27000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.097+0000 7fd2813cf700 1 -- 172.21.15.46:0/3422558200 shutdown_connections 2024-03-18T20:41:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.097+0000 7fd2813cf700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2640a1290 0x7fd2640a3750 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.097+0000 7fd2813cf700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd27c106e30 0x7fd27c100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.097+0000 7fd2813cf700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd27c0ff220 0x7fd27c100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.097+0000 7fd2813cf700 1 --2- 172.21.15.46:0/3422558200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27c0ffbe0 0x7fd27c100940 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.097+0000 7fd2813cf700 1 -- 172.21.15.46:0/3422558200 >> 172.21.15.46:0/3422558200 conn(0x7fd27c0761b0 msgr2=0x7fd27c103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.097+0000 7fd2813cf700 1 -- 172.21.15.46:0/3422558200 shutdown_connections 2024-03-18T20:41:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:40.097+0000 7fd2813cf700 1 -- 172.21.15.46:0/3422558200 wait complete. 2024-03-18T20:41:40.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:40 smithi073 ceph-mon[47384]: pgmap v170: 97 pgs: 97 active+clean; 71 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-03-18T20:41:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:40 smithi155 ceph-mon[48344]: pgmap v170: 97 pgs: 97 active+clean; 71 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-03-18T20:41:40.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:40 smithi046 ceph-mon[43495]: pgmap v170: 97 pgs: 97 active+clean; 71 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-03-18T20:41:40.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 167470 2.96 MB/sec execute 155 sec latency 17.382 ms 2024-03-18T20:41:41.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 169348 2.99 MB/sec execute 156 sec latency 17.224 ms 2024-03-18T20:41:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:42 smithi073 ceph-mon[47384]: pgmap v171: 97 pgs: 97 active+clean; 71 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1009 KiB/s wr, 156 op/s 2024-03-18T20:41:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:42 smithi155 ceph-mon[48344]: pgmap v171: 97 pgs: 97 active+clean; 71 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1009 KiB/s wr, 156 op/s 2024-03-18T20:41:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:42 smithi046 ceph-mon[43495]: pgmap v171: 97 pgs: 97 active+clean; 71 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1009 KiB/s wr, 156 op/s 2024-03-18T20:41:42.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 170809 3.04 MB/sec execute 157 sec latency 30.376 ms 2024-03-18T20:41:43.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 172764 3.07 MB/sec execute 158 sec latency 15.472 ms 2024-03-18T20:41:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:44 smithi073 ceph-mon[47384]: pgmap v172: 97 pgs: 97 active+clean; 75 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 280 op/s 2024-03-18T20:41:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:44 smithi155 ceph-mon[48344]: pgmap v172: 97 pgs: 97 active+clean; 75 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 280 op/s 2024-03-18T20:41:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:44 smithi046 ceph-mon[43495]: pgmap v172: 97 pgs: 97 active+clean; 75 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 280 op/s 2024-03-18T20:41:44.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 174285 3.13 MB/sec execute 159 sec latency 50.737 ms 2024-03-18T20:41:45.690 INFO:tasks.workunit.client.0.smithi046.stdout: 1 176022 3.14 MB/sec execute 160 sec latency 14.685 ms 2024-03-18T20:41:46.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:45 smithi073 ceph-mon[47384]: pgmap v173: 97 pgs: 97 active+clean; 75 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-03-18T20:41:46.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:45 smithi155 ceph-mon[48344]: pgmap v173: 97 pgs: 97 active+clean; 75 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-03-18T20:41:46.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:45 smithi046 ceph-mon[43495]: pgmap v173: 97 pgs: 97 active+clean; 75 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-03-18T20:41:46.691 INFO:tasks.workunit.client.0.smithi046.stdout: 1 177739 3.21 MB/sec execute 161 sec latency 13.895 ms 2024-03-18T20:41:47.693 INFO:tasks.workunit.client.0.smithi046.stdout: 1 179142 3.22 MB/sec execute 162 sec latency 14.391 ms 2024-03-18T20:41:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:48 smithi073 ceph-mon[47384]: pgmap v174: 97 pgs: 97 active+clean; 75 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-03-18T20:41:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:48 smithi155 ceph-mon[48344]: pgmap v174: 97 pgs: 97 active+clean; 75 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-03-18T20:41:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:48 smithi046 ceph-mon[43495]: pgmap v174: 97 pgs: 97 active+clean; 75 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-03-18T20:41:48.693 INFO:tasks.workunit.client.0.smithi046.stdout: 1 181206 3.28 MB/sec execute 163 sec latency 30.396 ms 2024-03-18T20:41:49.693 INFO:tasks.workunit.client.0.smithi046.stdout: 1 182604 3.30 MB/sec execute 164 sec latency 32.168 ms 2024-03-18T20:41:50.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:50 smithi073 ceph-mon[47384]: pgmap v175: 97 pgs: 97 active+clean; 79 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-03-18T20:41:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:50 smithi155 ceph-mon[48344]: pgmap v175: 97 pgs: 97 active+clean; 79 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-03-18T20:41:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:50 smithi046 ceph-mon[43495]: pgmap v175: 97 pgs: 97 active+clean; 79 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-03-18T20:41:50.693 INFO:tasks.workunit.client.0.smithi046.stdout: 1 184659 3.36 MB/sec execute 165 sec latency 187.553 ms 2024-03-18T20:41:51.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:51 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:41:51.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:51 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:41:51.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:51 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:41:51.693 INFO:tasks.workunit.client.0.smithi046.stdout: 1 185866 3.37 MB/sec execute 166 sec latency 12.227 ms 2024-03-18T20:41:52.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:52 smithi073 ceph-mon[47384]: pgmap v176: 97 pgs: 97 active+clean; 79 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-03-18T20:41:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:52 smithi155 ceph-mon[48344]: pgmap v176: 97 pgs: 97 active+clean; 79 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-03-18T20:41:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:52 smithi046 ceph-mon[43495]: pgmap v176: 97 pgs: 97 active+clean; 79 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-03-18T20:41:52.693 INFO:tasks.workunit.client.0.smithi046.stdout: 1 186148 3.35 MB/sec execute 167 sec latency 722.646 ms 2024-03-18T20:41:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.225+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/2050372181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c141000b0 msgr2=0x7f8c14100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.225+0000 7f8c1c0a1700 1 --2- 172.21.15.46:0/2050372181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c141000b0 0x7f8c14100530 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009990 tx=0x7f8c04009ca0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.225+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/2050372181 shutdown_connections 2024-03-18T20:41:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.225+0000 7f8c1c0a1700 1 --2- 172.21.15.46:0/2050372181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c14100a70 0x7f8c1410c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.225+0000 7f8c1c0a1700 1 --2- 172.21.15.46:0/2050372181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c14105d50 0x7f8c14106130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.225+0000 7f8c1c0a1700 1 --2- 172.21.15.46:0/2050372181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c141000b0 0x7f8c14100530 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.225+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/2050372181 >> 172.21.15.46:0/2050372181 conn(0x7f8c14076a20 msgr2=0x7f8c14076e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.225+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/2050372181 shutdown_connections 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.225+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/2050372181 wait complete. 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1c0a1700 1 Processor -- start 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1c0a1700 1 -- start start 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1c0a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c141000b0 0x7f8c14101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1c0a1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c14100a70 0x7f8c141021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1c0a1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c14105d50 0x7f8c14102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1c0a1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c141a5560 con 0x7f8c14105d50 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1c0a1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c141a56d0 con 0x7f8c141000b0 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1c0a1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c141a5890 con 0x7f8c14100a70 2024-03-18T20:41:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1a63e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c14105d50 0x7f8c14102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1a63e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c14105d50 0x7f8c14102720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52920/0 (socket says 172.21.15.46:52920) 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1a63e700 1 -- 172.21.15.46:0/1955511612 learned_addr learned my addr 172.21.15.46:0/1955511612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1963c700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c14100a70 0x7f8c141021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c19e3d700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c141000b0 0x7f8c14101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1a63e700 1 -- 172.21.15.46:0/1955511612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c14100a70 msgr2=0x7f8c141021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c14100a70 0x7f8c141021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1a63e700 1 -- 172.21.15.46:0/1955511612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c141000b0 msgr2=0x7f8c14101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c141000b0 0x7f8c14101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.226+0000 7f8c1a63e700 1 -- 172.21.15.46:0/1955511612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c040095f0 con 0x7f8c14105d50 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.227+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c14105d50 0x7f8c14102720 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00ef30 tx=0x7f8c0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.227+0000 7f8c0affd700 1 -- 172.21.15.46:0/1955511612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c018070 con 0x7f8c14105d50 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.227+0000 7f8c0affd700 1 -- 172.21.15.46:0/1955511612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0c00f040 con 0x7f8c14105d50 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.227+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/1955511612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c141a5b10 con 0x7f8c14105d50 2024-03-18T20:41:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.227+0000 7f8c0affd700 1 -- 172.21.15.46:0/1955511612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00cd00 con 0x7f8c14105d50 2024-03-18T20:41:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.227+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/1955511612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c141a6020 con 0x7f8c14105d50 2024-03-18T20:41:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.230+0000 7f8c0affd700 1 -- 172.21.15.46:0/1955511612 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8c0c003a00 con 0x7f8c14105d50 2024-03-18T20:41:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.231+0000 7f8c0affd700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8c000a1310 0x7f8c000a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.231+0000 7f8c0affd700 1 -- 172.21.15.46:0/1955511612 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8c0c0cc7c0 con 0x7f8c14105d50 2024-03-18T20:41:53.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.231+0000 7f8c19e3d700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8c000a1310 0x7f8c000a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.231+0000 7f8c19e3d700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8c000a1310 0x7f8c000a37d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009be0 tx=0x7f8c10009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.231+0000 7f8c28236b80 1 Processor -- start 2024-03-18T20:41:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c28236b80 1 -- start start 2024-03-18T20:41:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c28236b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95570 0x558a38e95950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c28236b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 0x558a38ea0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c28236b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38ea0db0 0x558a38ea31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c28236b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a38def930 con 0x558a38e95e90 2024-03-18T20:41:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c28236b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a38ded2f0 con 0x558a38ea0db0 2024-03-18T20:41:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c28236b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a38de8be0 con 0x558a38e95570 2024-03-18T20:41:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c1963c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 0x558a38ea0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c1963c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 0x558a38ea0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52922/0 (socket says 172.21.15.46:52922) 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c1963c700 1 -- 172.21.15.46:0/1283447792 learned_addr learned my addr 172.21.15.46:0/1283447792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/1283447792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38ea0db0 0x558a38ea31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c19e3d700 1 --2- 172.21.15.46:0/1283447792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95570 0x558a38e95950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c1a63e700 1 -- 172.21.15.46:0/1283447792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95570 msgr2=0x558a38e95950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/1283447792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95570 0x558a38e95950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c1a63e700 1 -- 172.21.15.46:0/1283447792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 msgr2=0x558a38ea0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/1283447792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 0x558a38ea0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c1a63e700 1 -- 172.21.15.46:0/1283447792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a38c6a350 con 0x558a38ea0db0 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.232+0000 7f8c1963c700 1 --2- 172.21.15.46:0/1283447792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 0x558a38ea0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.233+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/1283447792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38ea0db0 0x558a38ea31a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0c9ee0 tx=0x7f8c0c0994f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da1876858fc4fd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.233+0000 7f8bfffff700 1 -- 172.21.15.46:0/1283447792 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c084040 con 0x558a38ea0db0 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.233+0000 7f8bfffff700 1 -- 172.21.15.46:0/1283447792 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0c0b7e90 con 0x558a38ea0db0 2024-03-18T20:41:53.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.233+0000 7f8bfffff700 1 -- 172.21.15.46:0/1283447792 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c09d760 con 0x558a38ea0db0 2024-03-18T20:41:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.233+0000 7f8c28236b80 1 -- 172.21.15.46:0/1283447792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38ea0db0 msgr2=0x558a38ea31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.233+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1283447792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38ea0db0 0x558a38ea31a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0c9ee0 tx=0x7f8c0c0994f0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.234+0000 7f8c28236b80 1 -- 172.21.15.46:0/1283447792 shutdown_connections 2024-03-18T20:41:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.234+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1283447792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38ea0db0 0x558a38ea31a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.234+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1283447792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95570 0x558a38e95950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.234+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1283447792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 0x558a38ea0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.234+0000 7f8c28236b80 1 -- 172.21.15.46:0/1283447792 >> 172.21.15.46:0/1283447792 conn(0x558a38e0c1c0 msgr2=0x558a38e91a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.234+0000 7f8c28236b80 1 -- 172.21.15.46:0/1283447792 shutdown_connections 2024-03-18T20:41:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.234+0000 7f8c28236b80 1 -- 172.21.15.46:0/1283447792 wait complete. 2024-03-18T20:41:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.234+0000 7f8c28236b80 1 -- start start 2024-03-18T20:41:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.234+0000 7f8c28236b80 1 Processor -- start 2024-03-18T20:41:53.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c28236b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38e95570 0x558a38e9db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c28236b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95e90 0x558a38e9e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c28236b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 0x558a38e9e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c28236b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a38d84d20 con 0x558a38ea0db0 2024-03-18T20:41:53.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c28236b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a38d849c0 con 0x558a38e95570 2024-03-18T20:41:53.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c28236b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a38c6c640 con 0x558a38e95e90 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c28236b80 10 client.0 _ll_register_callbacks cb 0x558a38bf0340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c1a63e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 0x558a38e9e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c1a63e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 0x558a38e9e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52926/0 (socket says 172.21.15.46:52926) 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c1a63e700 1 -- 172.21.15.46:0/1669062346 learned_addr learned my addr 172.21.15.46:0/1669062346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c19e3d700 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38e95570 0x558a38e9db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c1963c700 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95e90 0x558a38e9e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c1a63e700 1 -- 172.21.15.46:0/1669062346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95e90 msgr2=0x558a38e9e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95e90 0x558a38e9e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c1a63e700 1 -- 172.21.15.46:0/1669062346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38e95570 msgr2=0x558a38e9db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38e95570 0x558a38e9db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c1a63e700 1 -- 172.21.15.46:0/1669062346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1410d320 con 0x558a38ea0db0 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c1963c700 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95e90 0x558a38e9e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:41:53.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 0x558a38e9e610 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c099870 tx=0x7f8c0c099dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f86894a35604d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8bfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8bfd7fa700 1 -- 172.21.15.46:0/1669062346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c084040 con 0x558a38ea0db0 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8bfd7fa700 1 -- 172.21.15.46:0/1669062346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8c0c01b320 con 0x558a38ea0db0 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.235+0000 7f8c28236b80 1 -- 172.21.15.46:0/1669062346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a38e896c0 con 0x558a38ea0db0 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.236+0000 7f8bfd7fa700 1 -- 172.21.15.46:0/1669062346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c0c0b2260 con 0x558a38ea0db0 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.236+0000 7f8c28236b80 10 client.17523 fetch_fsmap learned FSMap version 42 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.236+0000 7f8c28236b80 1 -- 172.21.15.46:0/1669062346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f8c141a6020 con 0x558a38ea0db0 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.236+0000 7f8bfd7fa700 1 -- 172.21.15.46:0/1669062346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a38e896c0 con 0x558a38ea0db0 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.236+0000 7f8bfd7fa700 1 -- 172.21.15.46:0/1669062346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f8c0c0a1890 con 0x558a38ea0db0 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.236+0000 7f8c28236b80 10 client.17523 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.236+0000 7f8c28236b80 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.236+0000 7f8c28236b80 20 client.17523 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.236+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a38eb06d0 0x558a38eb2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.236+0000 7f8c28236b80 4 client.17523 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.236+0000 7f8c28236b80 1 -- 172.21.15.46:0/1669062346 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a38de16a0 con 0x558a38eb06d0 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.236+0000 7f8c19e3d700 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a38eb06d0 0x558a38eb2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.237+0000 7f8c19e3d700 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a38eb06d0 0x558a38eb2af0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b2754e499d0271d server_cookie=ee9fead94f6af647 in_seq=0 out_seq=0 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.237+0000 7f8bfd7fa700 10 client.17523 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.237+0000 7f8bfd7fa700 1 -- 172.21.15.46:0/1669062346 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558a38de16a0 con 0x558a38eb06d0 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.237+0000 7f8bfd7fa700 10 client.17523 handle_command_reply: tid=0 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.237+0000 7f8c28236b80 1 client.17523 shutdown 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.237+0000 7f8c28236b80 1 -- 172.21.15.46:0/1669062346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 msgr2=0x558a38e9e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.237+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 0x558a38e9e610 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c099870 tx=0x7f8c0c099dc0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.238+0000 7f8c28236b80 1 -- 172.21.15.46:0/1669062346 shutdown_connections 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.238+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a38eb06d0 0x558a38eb2af0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.238+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 0x558a38e9e610 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.238+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38e95570 0x558a38e9db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.238+0000 7f8bfd7fa700 0 client.17523 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.238+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1669062346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95e90 0x558a38e9e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.238+0000 7f8c28236b80 1 -- 172.21.15.46:0/1669062346 >> 172.21.15.46:0/1669062346 conn(0x558a38e0c1c0 msgr2=0x558a38e91650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.238+0000 7f8c28236b80 1 -- 172.21.15.46:0/1669062346 shutdown_connections 2024-03-18T20:41:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.238+0000 7f8c28236b80 1 -- 172.21.15.46:0/1669062346 wait complete. 2024-03-18T20:41:53.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.238+0000 7f8c28236b80 20 client.17523 trim_cache size 0 max 16384 2024-03-18T20:41:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.253+0000 7f8c28236b80 1 Processor -- start 2024-03-18T20:41:53.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.253+0000 7f8c28236b80 1 -- start start 2024-03-18T20:41:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.253+0000 7f8c28236b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38e95570 0x558a38e95950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.253+0000 7f8c28236b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95e90 0x558a38e984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.253+0000 7f8c28236b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 0x558a38e9e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.253+0000 7f8c28236b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a38e96850 con 0x558a38ea0db0 2024-03-18T20:41:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.253+0000 7f8c28236b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a38e8bc10 con 0x558a38e95570 2024-03-18T20:41:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.253+0000 7f8c28236b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a38e999f0 con 0x558a38e95e90 2024-03-18T20:41:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c1a63e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 0x558a38e9e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c1a63e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 0x558a38e9e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52936/0 (socket says 172.21.15.46:52936) 2024-03-18T20:41:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c1a63e700 1 -- 172.21.15.46:0/385035436 learned_addr learned my addr 172.21.15.46:0/385035436 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c1963c700 1 --2- 172.21.15.46:0/385035436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95e90 0x558a38e984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c19e3d700 1 --2- 172.21.15.46:0/385035436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38e95570 0x558a38e95950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c1a63e700 1 -- 172.21.15.46:0/385035436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95e90 msgr2=0x558a38e984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/385035436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95e90 0x558a38e984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c1a63e700 1 -- 172.21.15.46:0/385035436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38e95570 msgr2=0x558a38e95950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/385035436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38e95570 0x558a38e95950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c1a63e700 1 -- 172.21.15.46:0/385035436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a38c16b00 con 0x558a38ea0db0 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/385035436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 0x558a38e9e430 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0963a0 tx=0x7f8c0c095da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42c43d21b82c64b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c08ff9700 1 -- 172.21.15.46:0/385035436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c084040 con 0x558a38ea0db0 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c08ff9700 1 -- 172.21.15.46:0/385035436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0c0b2260 con 0x558a38ea0db0 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.254+0000 7f8c08ff9700 1 -- 172.21.15.46:0/385035436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c09d760 con 0x558a38ea0db0 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.255+0000 7f8c28236b80 1 -- 172.21.15.46:0/385035436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 msgr2=0x558a38e9e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.255+0000 7f8c28236b80 1 --2- 172.21.15.46:0/385035436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 0x558a38e9e430 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0963a0 tx=0x7f8c0c095da0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.255+0000 7f8c28236b80 1 -- 172.21.15.46:0/385035436 shutdown_connections 2024-03-18T20:41:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.255+0000 7f8c28236b80 1 --2- 172.21.15.46:0/385035436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38ea0db0 0x558a38e9e430 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.255+0000 7f8c28236b80 1 --2- 172.21.15.46:0/385035436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38e95570 0x558a38e95950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.255+0000 7f8c28236b80 1 --2- 172.21.15.46:0/385035436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95e90 0x558a38e984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.255+0000 7f8c28236b80 1 -- 172.21.15.46:0/385035436 >> 172.21.15.46:0/385035436 conn(0x558a38e0c1c0 msgr2=0x558a38e90120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.255+0000 7f8c28236b80 1 -- 172.21.15.46:0/385035436 shutdown_connections 2024-03-18T20:41:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.255+0000 7f8c28236b80 1 -- 172.21.15.46:0/385035436 wait complete. 2024-03-18T20:41:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.255+0000 7f8c28236b80 1 -- start start 2024-03-18T20:41:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.255+0000 7f8c28236b80 1 Processor -- start 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c28236b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95570 0x558a38ea5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c28236b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 0x558a38ea5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c28236b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38ea0db0 0x558a38ea64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c28236b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a38f03500 con 0x558a38e95e90 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c28236b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a38f03640 con 0x558a38ea0db0 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c28236b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a38f03780 con 0x558a38e95570 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c28236b80 10 client.0 _ll_register_callbacks cb 0x7f8c0c0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c1963c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 0x558a38ea5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c1963c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 0x558a38ea5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52946/0 (socket says 172.21.15.46:52946) 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c1963c700 1 -- 172.21.15.46:0/1926847926 learned_addr learned my addr 172.21.15.46:0/1926847926 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c19e3d700 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95570 0x558a38ea5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c1a63e700 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38ea0db0 0x558a38ea64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c1963c700 1 -- 172.21.15.46:0/1926847926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95570 msgr2=0x558a38ea5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c1963c700 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95570 0x558a38ea5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c1963c700 1 -- 172.21.15.46:0/1926847926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38ea0db0 msgr2=0x558a38ea64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c1963c700 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38ea0db0 0x558a38ea64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.256+0000 7f8c1963c700 1 -- 172.21.15.46:0/1926847926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c141a6020 con 0x558a38e95e90 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8c1963c700 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 0x558a38ea5f60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009990 tx=0x7f8c04005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b546c2b81c1b3852 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8bfd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8bfd7fa700 1 -- 172.21.15.46:0/1926847926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0401d070 con 0x558a38e95e90 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8bfd7fa700 1 -- 172.21.15.46:0/1926847926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f8c04004500 con 0x558a38e95e90 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8c28236b80 1 -- 172.21.15.46:0/1926847926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c0c0b7e90 con 0x558a38e95e90 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8bfd7fa700 1 -- 172.21.15.46:0/1926847926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c0400bee0 con 0x558a38e95e90 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8c28236b80 10 client.17529 fetch_fsmap learned FSMap version 42 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8c28236b80 1 -- 172.21.15.46:0/1926847926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558a38daee40 con 0x558a38e95e90 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8bfd7fa700 1 -- 172.21.15.46:0/1926847926 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c0c0b7e90 con 0x558a38e95e90 2024-03-18T20:41:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8bfd7fa700 1 -- 172.21.15.46:0/1926847926 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 42) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f8c0400f760 con 0x558a38e95e90 2024-03-18T20:41:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8c28236b80 10 client.17529 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T20:41:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8c28236b80 10 client.17529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8c28236b80 20 client.17529 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a38f04f10 0x558a38f07330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8c28236b80 4 client.17529 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:41:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.257+0000 7f8c28236b80 1 -- 172.21.15.46:0/1926847926 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558a38c37c90 con 0x558a38f04f10 2024-03-18T20:41:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.258+0000 7f8c19e3d700 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a38f04f10 0x558a38f07330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.258+0000 7f8bfd7fa700 10 client.17529 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.258+0000 7f8c19e3d700 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a38f04f10 0x558a38f07330 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d9ac75b42d9ac5 server_cookie=3113460b8c6ad776 in_seq=0 out_seq=0 2024-03-18T20:41:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.290+0000 7f8bfd7fa700 1 -- 172.21.15.46:0/1926847926 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558a38c37c90 con 0x558a38f04f10 2024-03-18T20:41:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.290+0000 7f8bfd7fa700 10 client.17529 handle_command_reply: tid=0 2024-03-18T20:41:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8c28236b80 1 client.17529 shutdown 2024-03-18T20:41:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8c28236b80 1 -- 172.21.15.46:0/1926847926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 msgr2=0x558a38ea5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 0x558a38ea5f60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009990 tx=0x7f8c04005800 comp rx=0 tx=0).stop 2024-03-18T20:41:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8c28236b80 1 -- 172.21.15.46:0/1926847926 shutdown_connections 2024-03-18T20:41:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a38f04f10 0x558a38f07330 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a38ea0db0 0x558a38ea64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a38e95570 0x558a38ea5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8bfd7fa700 0 client.17529 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8c28236b80 1 --2- 172.21.15.46:0/1926847926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a38e95e90 0x558a38ea5f60 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8c28236b80 1 -- 172.21.15.46:0/1926847926 >> 172.21.15.46:0/1926847926 conn(0x558a38e0c1c0 msgr2=0x558a38e9c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8c28236b80 1 -- 172.21.15.46:0/1926847926 shutdown_connections 2024-03-18T20:41:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8c28236b80 1 -- 172.21.15.46:0/1926847926 wait complete. 2024-03-18T20:41:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.291+0000 7f8c28236b80 20 client.17529 trim_cache size 0 max 16384 2024-03-18T20:41:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:41:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:41:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:41:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:41:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.292+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/1955511612 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8c000a1310 msgr2=0x7f8c000a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.292+0000 7f8c1c0a1700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8c000a1310 0x7f8c000a37d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009be0 tx=0x7f8c10009260 comp rx=0 tx=0).stop 2024-03-18T20:41:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.292+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/1955511612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c14105d50 msgr2=0x7f8c14102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.292+0000 7f8c1c0a1700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c14105d50 0x7f8c14102720 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00ef30 tx=0x7f8c0c00c3a0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.292+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/1955511612 shutdown_connections 2024-03-18T20:41:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.292+0000 7f8c1c0a1700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8c000a1310 0x7f8c000a37d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.292+0000 7f8c1c0a1700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c14105d50 0x7f8c14102720 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.292+0000 7f8c1c0a1700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c141000b0 0x7f8c14101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.292+0000 7f8c1c0a1700 1 --2- 172.21.15.46:0/1955511612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8c14100a70 0x7f8c141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.292+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/1955511612 >> 172.21.15.46:0/1955511612 conn(0x7f8c14076a20 msgr2=0x7f8c14103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.292+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/1955511612 shutdown_connections 2024-03-18T20:41:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:41:53.292+0000 7f8c1c0a1700 1 -- 172.21.15.46:0/1955511612 wait complete. 2024-03-18T20:41:53.693 INFO:tasks.workunit.client.0.smithi046.stdout: 1 186148 3.33 MB/sec execute 168 sec latency 1722.774 ms 2024-03-18T20:41:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:54 smithi073 ceph-mon[47384]: pgmap v177: 97 pgs: 97 active+clean; 81 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-03-18T20:41:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:54 smithi155 ceph-mon[48344]: pgmap v177: 97 pgs: 97 active+clean; 81 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-03-18T20:41:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:54 smithi046 ceph-mon[43495]: pgmap v177: 97 pgs: 97 active+clean; 81 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-03-18T20:41:54.693 INFO:tasks.workunit.client.0.smithi046.stdout: 1 186148 3.31 MB/sec execute 169 sec latency 2722.867 ms 2024-03-18T20:41:55.693 INFO:tasks.workunit.client.0.smithi046.stdout: 1 187051 3.31 MB/sec execute 170 sec latency 3357.275 ms 2024-03-18T20:41:56.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:56 smithi073 ceph-mon[47384]: pgmap v178: 97 pgs: 97 active+clean; 81 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-03-18T20:41:56.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:56 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:41:56.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:56 smithi155 ceph-mon[48344]: pgmap v178: 97 pgs: 97 active+clean; 81 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-03-18T20:41:56.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:56 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:41:56.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:56 smithi046 ceph-mon[43495]: pgmap v178: 97 pgs: 97 active+clean; 81 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-03-18T20:41:56.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:56 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:41:56.693 INFO:tasks.workunit.client.0.smithi046.stdout: 1 188529 3.35 MB/sec execute 171 sec latency 17.112 ms 2024-03-18T20:41:57.693 INFO:tasks.workunit.client.0.smithi046.stdout: 1 189698 3.36 MB/sec execute 172 sec latency 208.810 ms 2024-03-18T20:41:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:41:58 smithi073 ceph-mon[47384]: pgmap v179: 97 pgs: 97 active+clean; 81 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-03-18T20:41:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:41:58 smithi155 ceph-mon[48344]: pgmap v179: 97 pgs: 97 active+clean; 81 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-03-18T20:41:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:41:58 smithi046 ceph-mon[43495]: pgmap v179: 97 pgs: 97 active+clean; 81 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-03-18T20:41:58.694 INFO:tasks.workunit.client.0.smithi046.stdout: 1 189698 3.35 MB/sec execute 173 sec latency 1208.941 ms 2024-03-18T20:41:59.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:41:59.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:41:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.570+0000 7f2408d5d700 1 -- 172.21.15.46:0/131598477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2404069c50 msgr2=0x7f2404105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.570+0000 7f2408d5d700 1 --2- 172.21.15.46:0/131598477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2404069c50 0x7f2404105980 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f23f80099c0 tx=0x7f23f8009cd0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.570+0000 7f2408d5d700 1 -- 172.21.15.46:0/131598477 shutdown_connections 2024-03-18T20:41:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.570+0000 7f2408d5d700 1 --2- 172.21.15.46:0/131598477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24041060b0 0x7f240410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.570+0000 7f2408d5d700 1 --2- 172.21.15.46:0/131598477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24040692a0 0x7f2404069680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.570+0000 7f2408d5d700 1 --2- 172.21.15.46:0/131598477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2404069c50 0x7f2404105980 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.570+0000 7f2408d5d700 1 -- 172.21.15.46:0/131598477 >> 172.21.15.46:0/131598477 conn(0x7f2404076d80 msgr2=0x7f2404077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.570+0000 7f2408d5d700 1 -- 172.21.15.46:0/131598477 shutdown_connections 2024-03-18T20:41:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.570+0000 7f2408d5d700 1 -- 172.21.15.46:0/131598477 wait complete. 2024-03-18T20:41:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.571+0000 7f2408d5d700 1 Processor -- start 2024-03-18T20:41:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.571+0000 7f2408d5d700 1 -- start start 2024-03-18T20:41:59.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.571+0000 7f2408d5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040692a0 0x7f24041971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.571+0000 7f2408d5d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24041060b0 0x7f24041976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.571+0000 7f2408d5d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f240419ba90 0x7f240419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.571+0000 7f2408d5d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240419c510 con 0x7f24040692a0 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.571+0000 7f2408d5d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240419c680 con 0x7f240419ba90 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.571+0000 7f2408d5d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240419c840 con 0x7f24041060b0 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.571+0000 7f240259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040692a0 0x7f24041971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.571+0000 7f240259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040692a0 0x7f24041971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36378/0 (socket says 172.21.15.46:36378) 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.571+0000 7f240259c700 1 -- 172.21.15.46:0/172504649 learned_addr learned my addr 172.21.15.46:0/172504649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f2401d9b700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24041060b0 0x7f24041976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f2402d9d700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f240419ba90 0x7f240419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f240259c700 1 -- 172.21.15.46:0/172504649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24041060b0 msgr2=0x7f24041976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f240259c700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24041060b0 0x7f24041976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f240259c700 1 -- 172.21.15.46:0/172504649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f240419ba90 msgr2=0x7f240419bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:59.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f240259c700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f240419ba90 0x7f240419bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f240259c700 1 -- 172.21.15.46:0/172504649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f80095f0 con 0x7f24040692a0 2024-03-18T20:41:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f2402d9d700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f240419ba90 0x7f240419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:41:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f240259c700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040692a0 0x7f24041971b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f23f400e9a0 tx=0x7f23f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f23f37fe700 1 -- 172.21.15.46:0/172504649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4009790 con 0x7f24040692a0 2024-03-18T20:41:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f2408d5d700 1 -- 172.21.15.46:0/172504649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24041a3da0 con 0x7f24040692a0 2024-03-18T20:41:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f23f37fe700 1 -- 172.21.15.46:0/172504649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f400cbd0 con 0x7f24040692a0 2024-03-18T20:41:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f23f37fe700 1 -- 172.21.15.46:0/172504649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4018680 con 0x7f24040692a0 2024-03-18T20:41:59.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.572+0000 7f2408d5d700 1 -- 172.21.15.46:0/172504649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24041a4270 con 0x7f24040692a0 2024-03-18T20:41:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.574+0000 7f23f37fe700 1 -- 172.21.15.46:0/172504649 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f23f400cd40 con 0x7f24040692a0 2024-03-18T20:41:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.575+0000 7f23f37fe700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f23ec0a1290 0x7f23ec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.575+0000 7f23f37fe700 1 -- 172.21.15.46:0/172504649 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f23f4014070 con 0x7f24040692a0 2024-03-18T20:41:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.575+0000 7f2401d9b700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f23ec0a1290 0x7f23ec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.576+0000 7f2401d9b700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f23ec0a1290 0x7f23ec0a3750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009990 tx=0x7f23f8019040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.576+0000 7f2414ef2b80 1 Processor -- start 2024-03-18T20:41:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.576+0000 7f2414ef2b80 1 -- start start 2024-03-18T20:41:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.576+0000 7f2414ef2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387aa60 0x5555f387ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.576+0000 7f2414ef2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f388a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.576+0000 7f2414ef2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f388a630 0x5555f388ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.576+0000 7f2414ef2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f37d4990 con 0x5555f387aa60 2024-03-18T20:41:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.576+0000 7f2414ef2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f37d2350 con 0x5555f388a630 2024-03-18T20:41:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.576+0000 7f2414ef2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f37cdc40 con 0x5555f387b380 2024-03-18T20:41:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f240259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387aa60 0x5555f387ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f240259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387aa60 0x5555f387ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36394/0 (socket says 172.21.15.46:36394) 2024-03-18T20:41:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f240259c700 1 -- 172.21.15.46:0/3585993590 learned_addr learned my addr 172.21.15.46:0/3585993590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f2401d9b700 1 --2- 172.21.15.46:0/3585993590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f388a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f2402d9d700 1 --2- 172.21.15.46:0/3585993590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f388a630 0x5555f388ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f240259c700 1 -- 172.21.15.46:0/3585993590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 msgr2=0x5555f388a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f240259c700 1 --2- 172.21.15.46:0/3585993590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f388a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f240259c700 1 -- 172.21.15.46:0/3585993590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f388a630 msgr2=0x5555f388ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f240259c700 1 --2- 172.21.15.46:0/3585993590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f388a630 0x5555f388ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f240259c700 1 -- 172.21.15.46:0/3585993590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555f364f3b0 con 0x5555f387aa60 2024-03-18T20:41:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f240259c700 1 --2- 172.21.15.46:0/3585993590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387aa60 0x5555f387ae40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f23f4096ac0 tx=0x7f23f4099a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3b0c1cd446b4d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f23f0ff9700 1 -- 172.21.15.46:0/3585993590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f409de80 con 0x5555f387aa60 2024-03-18T20:41:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f23f0ff9700 1 -- 172.21.15.46:0/3585993590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f40a24d0 con 0x5555f387aa60 2024-03-18T20:41:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.577+0000 7f23f0ff9700 1 -- 172.21.15.46:0/3585993590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f40a2040 con 0x5555f387aa60 2024-03-18T20:41:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.578+0000 7f2414ef2b80 1 -- 172.21.15.46:0/3585993590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387aa60 msgr2=0x5555f387ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.578+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/3585993590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387aa60 0x5555f387ae40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f23f4096ac0 tx=0x7f23f4099a30 comp rx=0 tx=0).stop 2024-03-18T20:41:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.578+0000 7f2414ef2b80 1 -- 172.21.15.46:0/3585993590 shutdown_connections 2024-03-18T20:41:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.578+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/3585993590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387aa60 0x5555f387ae40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.578+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/3585993590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f388a630 0x5555f388ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.578+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/3585993590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f388a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.578+0000 7f2414ef2b80 1 -- 172.21.15.46:0/3585993590 >> 172.21.15.46:0/3585993590 conn(0x5555f37f1220 msgr2=0x5555f3877020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.578+0000 7f2414ef2b80 1 -- 172.21.15.46:0/3585993590 shutdown_connections 2024-03-18T20:41:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.578+0000 7f2414ef2b80 1 -- 172.21.15.46:0/3585993590 wait complete. 2024-03-18T20:41:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2414ef2b80 1 -- start start 2024-03-18T20:41:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2414ef2b80 1 Processor -- start 2024-03-18T20:41:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2414ef2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f387aa60 0x5555f38874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2414ef2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f3887a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2414ef2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 0x5555f3887f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2414ef2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f3769d80 con 0x5555f388a630 2024-03-18T20:41:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2414ef2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f3769a20 con 0x5555f387aa60 2024-03-18T20:41:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2414ef2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f36516a0 con 0x5555f387b380 2024-03-18T20:41:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2414ef2b80 10 client.0 _ll_register_callbacks cb 0x5555f35d53a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2402d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 0x5555f3887f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2402d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 0x5555f3887f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36402/0 (socket says 172.21.15.46:36402) 2024-03-18T20:41:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2402d9d700 1 -- 172.21.15.46:0/1871722549 learned_addr learned my addr 172.21.15.46:0/1871722549 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f2401d9b700 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f3887a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.579+0000 7f240259c700 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f387aa60 0x5555f38874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f2401d9b700 1 -- 172.21.15.46:0/1871722549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f387aa60 msgr2=0x5555f38874c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f2401d9b700 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f387aa60 0x5555f38874c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f2401d9b700 1 -- 172.21.15.46:0/1871722549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 msgr2=0x5555f3887f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f2401d9b700 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 0x5555f3887f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f2401d9b700 1 -- 172.21.15.46:0/1871722549 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24041a4270 con 0x5555f387b380 2024-03-18T20:41:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f240259c700 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f387aa60 0x5555f38874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f2402d9d700 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 0x5555f3887f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f2401d9b700 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f3887a00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f23f80098e0 tx=0x7f23f8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aee1cc8818f177e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f23e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f23e9ffb700 1 -- 172.21.15.46:0/1871722549 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8004d00 con 0x5555f387b380 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f2414ef2b80 1 -- 172.21.15.46:0/1871722549 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555f386ead0 con 0x5555f387b380 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f23e9ffb700 1 -- 172.21.15.46:0/1871722549 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f23f8004e60 con 0x5555f387b380 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f23e9ffb700 1 -- 172.21.15.46:0/1871722549 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23f800f460 con 0x5555f387b380 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f2414ef2b80 10 client.27247 fetch_fsmap learned FSMap version 43 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.580+0000 7f2414ef2b80 1 -- 172.21.15.46:0/1871722549 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5555f35fbb60 con 0x5555f387b380 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.581+0000 7f23e9ffb700 1 -- 172.21.15.46:0/1871722549 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555f386ead0 con 0x5555f387b380 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.581+0000 7f23e9ffb700 1 -- 172.21.15.46:0/1871722549 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f23f802e070 con 0x5555f387b380 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.581+0000 7f2414ef2b80 10 client.27247 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.581+0000 7f2414ef2b80 10 client.27247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.581+0000 7f2414ef2b80 20 client.27247 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.581+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5555f389a340 0x5555f389c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.581+0000 7f2414ef2b80 4 client.27247 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.581+0000 7f2414ef2b80 1 -- 172.21.15.46:0/1871722549 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555f37c6700 con 0x5555f389a340 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.581+0000 7f240259c700 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5555f389a340 0x5555f389c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.582+0000 7f240259c700 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5555f389a340 0x5555f389c760 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc5caee3a04cac6f server_cookie=f46b26edfbe8d7a5 in_seq=0 out_seq=0 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.582+0000 7f23e9ffb700 10 client.27247 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.582+0000 7f23e9ffb700 1 -- 172.21.15.46:0/1871722549 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5555f37c6700 con 0x5555f389a340 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.582+0000 7f23e9ffb700 10 client.27247 handle_command_reply: tid=0 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.582+0000 7f2414ef2b80 1 client.27247 shutdown 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.583+0000 7f2414ef2b80 1 -- 172.21.15.46:0/1871722549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 msgr2=0x5555f3887a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.583+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f3887a00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f23f80098e0 tx=0x7f23f8009310 comp rx=0 tx=0).stop 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.583+0000 7f2414ef2b80 1 -- 172.21.15.46:0/1871722549 shutdown_connections 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.583+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5555f389a340 0x5555f389c760 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.583+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 0x5555f3887f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.583+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f387aa60 0x5555f38874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.583+0000 7f23e9ffb700 0 client.27247 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:59.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.583+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/1871722549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f3887a00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.583+0000 7f2414ef2b80 1 -- 172.21.15.46:0/1871722549 >> 172.21.15.46:0/1871722549 conn(0x5555f37f1220 msgr2=0x5555f3876c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:59.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.583+0000 7f2414ef2b80 1 -- 172.21.15.46:0/1871722549 shutdown_connections 2024-03-18T20:41:59.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.583+0000 7f2414ef2b80 1 -- 172.21.15.46:0/1871722549 wait complete. 2024-03-18T20:41:59.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.583+0000 7f2414ef2b80 20 client.27247 trim_cache size 0 max 16384 2024-03-18T20:41:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.601+0000 7f2414ef2b80 1 Processor -- start 2024-03-18T20:41:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.601+0000 7f2414ef2b80 1 -- start start 2024-03-18T20:41:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.601+0000 7f2414ef2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f387aa60 0x5555f389c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.601+0000 7f2414ef2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f3891260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.601+0000 7f2414ef2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 0x5555f3887d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.601+0000 7f2414ef2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f387ba30 con 0x5555f388a630 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.601+0000 7f2414ef2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f387c9b0 con 0x5555f387aa60 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.601+0000 7f2414ef2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f3888ac0 con 0x5555f387b380 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f2402d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 0x5555f3887d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f2401d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f3891260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f2402d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 0x5555f3887d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36406/0 (socket says 172.21.15.46:36406) 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f2402d9d700 1 -- 172.21.15.46:0/302367086 learned_addr learned my addr 172.21.15.46:0/302367086 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f2402d9d700 1 -- 172.21.15.46:0/302367086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 msgr2=0x5555f3891260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f2402d9d700 1 --2- 172.21.15.46:0/302367086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f3891260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f240259c700 1 --2- 172.21.15.46:0/302367086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f387aa60 0x5555f389c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f2402d9d700 1 -- 172.21.15.46:0/302367086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f387aa60 msgr2=0x5555f389c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f2402d9d700 1 --2- 172.21.15.46:0/302367086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f387aa60 0x5555f389c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f2402d9d700 1 -- 172.21.15.46:0/302367086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555f3793ea0 con 0x5555f388a630 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f240259c700 1 --2- 172.21.15.46:0/302367086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f387aa60 0x5555f389c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f2402d9d700 1 --2- 172.21.15.46:0/302367086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 0x5555f3887d60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f23fc007b60 tx=0x7f23fc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f136cfe0c63c039a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f23f17fa700 1 -- 172.21.15.46:0/302367086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc004db0 con 0x5555f388a630 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f23f17fa700 1 -- 172.21.15.46:0/302367086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23fc011040 con 0x5555f388a630 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.602+0000 7f23f17fa700 1 -- 172.21.15.46:0/302367086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc005650 con 0x5555f388a630 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.603+0000 7f2414ef2b80 1 -- 172.21.15.46:0/302367086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 msgr2=0x5555f3887d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.603+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/302367086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 0x5555f3887d60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f23fc007b60 tx=0x7f23fc009040 comp rx=0 tx=0).stop 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.603+0000 7f2414ef2b80 1 -- 172.21.15.46:0/302367086 shutdown_connections 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.603+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/302367086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f388a630 0x5555f3887d60 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.603+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/302367086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f387aa60 0x5555f389c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.603+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/302367086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387b380 0x5555f3891260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.603+0000 7f2414ef2b80 1 -- 172.21.15.46:0/302367086 >> 172.21.15.46:0/302367086 conn(0x5555f37f1220 msgr2=0x5555f3872a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.603+0000 7f2414ef2b80 1 -- 172.21.15.46:0/302367086 shutdown_connections 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.603+0000 7f2414ef2b80 1 -- 172.21.15.46:0/302367086 wait complete. 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.603+0000 7f2414ef2b80 1 -- start start 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.603+0000 7f2414ef2b80 1 Processor -- start 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2414ef2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387aa60 0x5555f388e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2414ef2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387b380 0x5555f388ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2414ef2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f388a630 0x5555f388f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2414ef2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f38ed200 con 0x5555f387b380 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2414ef2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f38ed370 con 0x5555f388a630 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2414ef2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555f38ed530 con 0x5555f387aa60 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2414ef2b80 10 client.0 _ll_register_callbacks cb 0x7f23f8005360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2401d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387b380 0x5555f388ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2401d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387b380 0x5555f388ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36408/0 (socket says 172.21.15.46:36408) 2024-03-18T20:41:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2401d9b700 1 -- 172.21.15.46:0/2515535202 learned_addr learned my addr 172.21.15.46:0/2515535202 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2402d9d700 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f388a630 0x5555f388f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f240259c700 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387aa60 0x5555f388e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2401d9b700 1 -- 172.21.15.46:0/2515535202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387aa60 msgr2=0x5555f388e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2401d9b700 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387aa60 0x5555f388e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2401d9b700 1 -- 172.21.15.46:0/2515535202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f388a630 msgr2=0x5555f388f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2401d9b700 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f388a630 0x5555f388f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2401d9b700 1 -- 172.21.15.46:0/2515535202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23fc007810 con 0x5555f387b380 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2401d9b700 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387b380 0x5555f388ee40 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x5555f3871050 tx=0x7f23f8023af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1563348bf2ca5bb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f23e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f23e9ffb700 1 -- 172.21.15.46:0/2515535202 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8047070 con 0x5555f387b380 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f23e9ffb700 1 -- 172.21.15.46:0/2515535202 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f23f8023410 con 0x5555f387b380 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.604+0000 7f2414ef2b80 1 -- 172.21.15.46:0/2515535202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23f40a24d0 con 0x5555f387b380 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.605+0000 7f23e9ffb700 1 -- 172.21.15.46:0/2515535202 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23f800b9c0 con 0x5555f387b380 2024-03-18T20:41:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.605+0000 7f2414ef2b80 10 client.17547 fetch_fsmap learned FSMap version 43 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.605+0000 7f2414ef2b80 1 -- 172.21.15.46:0/2515535202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5555f3653cd0 con 0x5555f387b380 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.605+0000 7f23e9ffb700 1 -- 172.21.15.46:0/2515535202 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f40a24d0 con 0x5555f387b380 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.605+0000 7f23e9ffb700 1 -- 172.21.15.46:0/2515535202 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f23f804a020 con 0x5555f387b380 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.605+0000 7f2414ef2b80 10 client.17547 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.605+0000 7f2414ef2b80 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.605+0000 7f2414ef2b80 20 client.17547 populate_metadata read hostname 'smithi046' 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.605+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5555f38ee880 0x5555f38f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.605+0000 7f2414ef2b80 4 client.17547 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.605+0000 7f2414ef2b80 1 -- 172.21.15.46:0/2515535202 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5555f361ccf0 con 0x5555f38ee880 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.605+0000 7f240259c700 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5555f38ee880 0x5555f38f0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f23e9ffb700 10 client.17547 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f240259c700 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5555f38ee880 0x5555f38f0ca0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=382150b6c5efb6c6 server_cookie=dbf36ded25a4937 in_seq=0 out_seq=0 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f23e9ffb700 1 -- 172.21.15.46:0/2515535202 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5555f361ccf0 con 0x5555f38ee880 2024-03-18T20:41:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f23e9ffb700 10 client.17547 handle_command_reply: tid=0 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f2414ef2b80 1 client.17547 shutdown 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f2414ef2b80 1 -- 172.21.15.46:0/2515535202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387b380 msgr2=0x5555f388ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387b380 0x5555f388ee40 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x5555f3871050 tx=0x7f23f8023af0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f2414ef2b80 1 -- 172.21.15.46:0/2515535202 shutdown_connections 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5555f38ee880 0x5555f38f0ca0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5555f388a630 0x5555f388f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5555f387aa60 0x5555f388e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f2414ef2b80 1 --2- 172.21.15.46:0/2515535202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555f387b380 0x5555f388ee40 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f23e9ffb700 0 client.17547 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f2414ef2b80 1 -- 172.21.15.46:0/2515535202 >> 172.21.15.46:0/2515535202 conn(0x5555f37f1220 msgr2=0x5555f3882790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f2414ef2b80 1 -- 172.21.15.46:0/2515535202 shutdown_connections 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f2414ef2b80 1 -- 172.21.15.46:0/2515535202 wait complete. 2024-03-18T20:41:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.606+0000 7f2414ef2b80 20 client.17547 trim_cache size 0 max 16384 2024-03-18T20:41:59.616 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:41:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:41:59.616 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:41:59.616 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:41:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.607+0000 7f2408d5d700 1 -- 172.21.15.46:0/172504649 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f23ec0a1290 msgr2=0x7f23ec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.607+0000 7f2408d5d700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f23ec0a1290 0x7f23ec0a3750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009990 tx=0x7f23f8019040 comp rx=0 tx=0).stop 2024-03-18T20:41:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.607+0000 7f2408d5d700 1 -- 172.21.15.46:0/172504649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040692a0 msgr2=0x7f24041971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.607+0000 7f2408d5d700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040692a0 0x7f24041971b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f23f400e9a0 tx=0x7f23f400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.607+0000 7f2408d5d700 1 -- 172.21.15.46:0/172504649 shutdown_connections 2024-03-18T20:41:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.607+0000 7f2408d5d700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f23ec0a1290 0x7f23ec0a3750 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.607+0000 7f2408d5d700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24040692a0 0x7f24041971b0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.607+0000 7f2408d5d700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f240419ba90 0x7f240419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.607+0000 7f2408d5d700 1 --2- 172.21.15.46:0/172504649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24041060b0 0x7f24041976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.607+0000 7f2408d5d700 1 -- 172.21.15.46:0/172504649 >> 172.21.15.46:0/172504649 conn(0x7f2404076d80 msgr2=0x7f24040fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.607+0000 7f2408d5d700 1 -- 172.21.15.46:0/172504649 shutdown_connections 2024-03-18T20:41:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.607+0000 7f2408d5d700 1 -- 172.21.15.46:0/172504649 wait complete. 2024-03-18T20:41:59.627 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:41:59.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:41:59.694 INFO:tasks.workunit.client.0.smithi046.stdout: 1 189698 3.33 MB/sec execute 174 sec latency 2209.007 ms 2024-03-18T20:41:59.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.761+0000 7fa490973700 1 -- 172.21.15.46:0/2895642372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa48c101e10 msgr2=0x7fa48c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:59.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.761+0000 7fa490973700 1 --2- 172.21.15.46:0/2895642372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa48c101e10 0x7fa48c1021f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009990 tx=0x7fa47c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.761+0000 7fa490973700 1 -- 172.21.15.46:0/2895642372 shutdown_connections 2024-03-18T20:41:59.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.761+0000 7fa490973700 1 --2- 172.21.15.46:0/2895642372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa48c101e10 0x7fa48c1021f0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.761+0000 7fa490973700 1 --2- 172.21.15.46:0/2895642372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa48c10b1e0 0x7fa48c10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.761+0000 7fa490973700 1 --2- 172.21.15.46:0/2895642372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa48c102730 0x7fa48c10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.761+0000 7fa490973700 1 -- 172.21.15.46:0/2895642372 >> 172.21.15.46:0/2895642372 conn(0x7fa48c0fb5f0 msgr2=0x7fa48c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:41:59.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.761+0000 7fa490973700 1 -- 172.21.15.46:0/2895642372 shutdown_connections 2024-03-18T20:41:59.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.761+0000 7fa490973700 1 -- 172.21.15.46:0/2895642372 wait complete. 2024-03-18T20:41:59.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.762+0000 7fa490973700 1 Processor -- start 2024-03-18T20:41:59.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.762+0000 7fa490973700 1 -- start start 2024-03-18T20:41:59.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.762+0000 7fa490973700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa48c101e10 0x7fa48c0ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.762+0000 7fa490973700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa48c102730 0x7fa48c0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.762+0000 7fa490973700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa48c10b1e0 0x7fa48c1004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa490973700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1a5430 con 0x7fa48c101e10 2024-03-18T20:41:59.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa490973700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1a55a0 con 0x7fa48c102730 2024-03-18T20:41:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa490973700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1a5760 con 0x7fa48c10b1e0 2024-03-18T20:41:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa48a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa48c101e10 0x7fa48c0ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa48a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa48c101e10 0x7fa48c0ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36430/0 (socket says 172.21.15.46:36430) 2024-03-18T20:41:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa48a59c700 1 -- 172.21.15.46:0/2303288455 learned_addr learned my addr 172.21.15.46:0/2303288455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:41:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa48ad9d700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa48c10b1e0 0x7fa48c1004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa489d9b700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa48c102730 0x7fa48c0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa48a59c700 1 -- 172.21.15.46:0/2303288455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa48c10b1e0 msgr2=0x7fa48c1004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa48a59c700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa48c10b1e0 0x7fa48c1004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa48a59c700 1 -- 172.21.15.46:0/2303288455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa48c102730 msgr2=0x7fa48c0fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:41:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa48a59c700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa48c102730 0x7fa48c0fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:41:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa48a59c700 1 -- 172.21.15.46:0/2303288455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0095f0 con 0x7fa48c101e10 2024-03-18T20:41:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa48a59c700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa48c101e10 0x7fa48c0ffa60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa47c000c00 tx=0x7fa47c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa47b7fe700 1 -- 172.21.15.46:0/2303288455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c01d070 con 0x7fa48c101e10 2024-03-18T20:41:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa47b7fe700 1 -- 172.21.15.46:0/2303288455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa47c022470 con 0x7fa48c101e10 2024-03-18T20:41:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa490973700 1 -- 172.21.15.46:0/2303288455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c1a59e0 con 0x7fa48c101e10 2024-03-18T20:41:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.763+0000 7fa47b7fe700 1 -- 172.21.15.46:0/2303288455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c017650 con 0x7fa48c101e10 2024-03-18T20:41:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.764+0000 7fa490973700 1 -- 172.21.15.46:0/2303288455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c1a5ee0 con 0x7fa48c101e10 2024-03-18T20:41:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.765+0000 7fa490973700 1 -- 172.21.15.46:0/2303288455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c106360 con 0x7fa48c101e10 2024-03-18T20:41:59.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.766+0000 7fa47b7fe700 1 -- 172.21.15.46:0/2303288455 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa47c0225e0 con 0x7fa48c101e10 2024-03-18T20:41:59.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.767+0000 7fa47b7fe700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa4740a1310 0x7fa4740a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:41:59.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.767+0000 7fa489d9b700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa4740a1310 0x7fa4740a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:41:59.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.767+0000 7fa47b7fe700 1 -- 172.21.15.46:0/2303288455 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fa47c0ce350 con 0x7fa48c101e10 2024-03-18T20:41:59.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.767+0000 7fa489d9b700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa4740a1310 0x7fa4740a37d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa48c101190 tx=0x7fa480009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:41:59.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:41:59.771+0000 7fa47b7fe700 1 -- 172.21.15.46:0/2303288455 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa47c097880 con 0x7fa48c101e10 2024-03-18T20:42:00.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.039+0000 7fa490973700 1 -- 172.21.15.46:0/2303288455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa48c068670 con 0x7fa48c101e10 2024-03-18T20:42:00.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.040+0000 7fa47b7fe700 1 -- 172.21.15.46:0/2303288455 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+10919 (secure 0 0 0) 0x7fa47c09aea0 con 0x7fa48c101e10 2024-03-18T20:42:00.043 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:42:00.043 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:41:55.183335+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:42:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.042+0000 7fa490973700 1 -- 172.21.15.46:0/2303288455 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa4740a1310 msgr2=0x7fa4740a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.042+0000 7fa490973700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa4740a1310 0x7fa4740a37d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa48c101190 tx=0x7fa480009260 comp rx=0 tx=0).stop 2024-03-18T20:42:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.042+0000 7fa490973700 1 -- 172.21.15.46:0/2303288455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa48c101e10 msgr2=0x7fa48c0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.042+0000 7fa490973700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa48c101e10 0x7fa48c0ffa60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa47c000c00 tx=0x7fa47c004730 comp rx=0 tx=0).stop 2024-03-18T20:42:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.042+0000 7fa490973700 1 -- 172.21.15.46:0/2303288455 shutdown_connections 2024-03-18T20:42:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.042+0000 7fa490973700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa4740a1310 0x7fa4740a37d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.042+0000 7fa490973700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa48c101e10 0x7fa48c0ffa60 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.042+0000 7fa490973700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa48c10b1e0 0x7fa48c1004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.042+0000 7fa490973700 1 --2- 172.21.15.46:0/2303288455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa48c102730 0x7fa48c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.042+0000 7fa490973700 1 -- 172.21.15.46:0/2303288455 >> 172.21.15.46:0/2303288455 conn(0x7fa48c0fb5f0 msgr2=0x7fa48c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.043+0000 7fa490973700 1 -- 172.21.15.46:0/2303288455 shutdown_connections 2024-03-18T20:42:00.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.043+0000 7fa490973700 1 -- 172.21.15.46:0/2303288455 wait complete. 2024-03-18T20:42:00.047 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 43 2024-03-18T20:42:00.060 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:42:00.060 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:42:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.207+0000 7f486c7ed700 1 -- 172.21.15.46:0/609563364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48641060a0 msgr2=0x7f486410dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.207+0000 7f486c7ed700 1 --2- 172.21.15.46:0/609563364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48641060a0 0x7f486410dc20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f485c00b0d0 tx=0x7f485c00b3e0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.207+0000 7f486c7ed700 1 -- 172.21.15.46:0/609563364 shutdown_connections 2024-03-18T20:42:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.207+0000 7f486c7ed700 1 --2- 172.21.15.46:0/609563364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48641060a0 0x7f486410dc20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.207+0000 7f486c7ed700 1 --2- 172.21.15.46:0/609563364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4864069540 0x7f4864069920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.207+0000 7f486c7ed700 1 --2- 172.21.15.46:0/609563364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4864069ef0 0x7f4864105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.207+0000 7f486c7ed700 1 -- 172.21.15.46:0/609563364 >> 172.21.15.46:0/609563364 conn(0x7f4864076d50 msgr2=0x7f4864077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.207+0000 7f486c7ed700 1 -- 172.21.15.46:0/609563364 shutdown_connections 2024-03-18T20:42:00.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.207+0000 7f486c7ed700 1 -- 172.21.15.46:0/609563364 wait complete. 2024-03-18T20:42:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.207+0000 7f486c7ed700 1 Processor -- start 2024-03-18T20:42:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486c7ed700 1 -- start start 2024-03-18T20:42:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486c7ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864069540 0x7f4864197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486c7ed700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48641060a0 0x7f48641977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486c7ed700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486419bb60 0x7f486419bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486c7ed700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486419c5e0 con 0x7f4864069540 2024-03-18T20:42:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486c7ed700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486419c750 con 0x7f48641060a0 2024-03-18T20:42:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486c7ed700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486419c910 con 0x7f486419bb60 2024-03-18T20:42:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486a589700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864069540 0x7f4864197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486a589700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864069540 0x7f4864197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36458/0 (socket says 172.21.15.46:36458) 2024-03-18T20:42:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486a589700 1 -- 172.21.15.46:0/3263909694 learned_addr learned my addr 172.21.15.46:0/3263909694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f4869d88700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48641060a0 0x7f48641977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486ad8a700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486419bb60 0x7f486419bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486a589700 1 -- 172.21.15.46:0/3263909694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486419bb60 msgr2=0x7f486419bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486a589700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486419bb60 0x7f486419bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486a589700 1 -- 172.21.15.46:0/3263909694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48641060a0 msgr2=0x7f48641977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486a589700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48641060a0 0x7f48641977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486a589700 1 -- 172.21.15.46:0/3263909694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c009d00 con 0x7f4864069540 2024-03-18T20:42:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.208+0000 7f486a589700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864069540 0x7f4864197280 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f486000d980 tx=0x7f486000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.209+0000 7f485b7fe700 1 -- 172.21.15.46:0/3263909694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860004220 con 0x7f4864069540 2024-03-18T20:42:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.209+0000 7f485b7fe700 1 -- 172.21.15.46:0/3263909694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4860007640 con 0x7f4864069540 2024-03-18T20:42:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.209+0000 7f486c7ed700 1 -- 172.21.15.46:0/3263909694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48641a3e70 con 0x7f4864069540 2024-03-18T20:42:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.209+0000 7f485b7fe700 1 -- 172.21.15.46:0/3263909694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860005610 con 0x7f4864069540 2024-03-18T20:42:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.209+0000 7f486c7ed700 1 -- 172.21.15.46:0/3263909694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48641a4340 con 0x7f4864069540 2024-03-18T20:42:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.212+0000 7f485b7fe700 1 -- 172.21.15.46:0/3263909694 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f4860004750 con 0x7f4864069540 2024-03-18T20:42:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.212+0000 7f485b7fe700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48500a1310 0x7f48500a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.212+0000 7f485b7fe700 1 -- 172.21.15.46:0/3263909694 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f48600cca80 con 0x7f4864069540 2024-03-18T20:42:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.212+0000 7f4869d88700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48500a1310 0x7f48500a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.213+0000 7f4869d88700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48500a1310 0x7f48500a37d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f48641987d0 tx=0x7f4854009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.213+0000 7f4878982b80 1 Processor -- start 2024-03-18T20:42:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.213+0000 7f4878982b80 1 -- start start 2024-03-18T20:42:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.213+0000 7f4878982b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c16816fa60 0x55c16816fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.213+0000 7f4878982b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c168170380 0x55c16817ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.213+0000 7f4878982b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c16817d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.213+0000 7f4878982b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1680c9990 con 0x55c168170380 2024-03-18T20:42:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.213+0000 7f4878982b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1680c7350 con 0x55c16816fa60 2024-03-18T20:42:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.213+0000 7f4878982b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1680c2c40 con 0x55c16817b220 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f4869d88700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c168170380 0x55c16817ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f4869d88700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c168170380 0x55c16817ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36472/0 (socket says 172.21.15.46:36472) 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f4869d88700 1 -- 172.21.15.46:0/3256519926 learned_addr learned my addr 172.21.15.46:0/3256519926 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f486ad8a700 1 --2- 172.21.15.46:0/3256519926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c16817d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f486a589700 1 --2- 172.21.15.46:0/3256519926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c16816fa60 0x55c16816fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f4869d88700 1 -- 172.21.15.46:0/3256519926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 msgr2=0x55c16817d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f4869d88700 1 --2- 172.21.15.46:0/3256519926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c16817d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f4869d88700 1 -- 172.21.15.46:0/3256519926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c16816fa60 msgr2=0x55c16816fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f4869d88700 1 --2- 172.21.15.46:0/3256519926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c16816fa60 0x55c16816fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f4869d88700 1 -- 172.21.15.46:0/3256519926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c167f443b0 con 0x55c168170380 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f4869d88700 1 --2- 172.21.15.46:0/3256519926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c168170380 0x55c16817ace0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4854011d10 tx=0x7f4854014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3a15115793d234 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f4858ff9700 1 -- 172.21.15.46:0/3256519926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854011d40 con 0x55c168170380 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f4858ff9700 1 -- 172.21.15.46:0/3256519926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48540044e0 con 0x55c168170380 2024-03-18T20:42:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.214+0000 7f4858ff9700 1 -- 172.21.15.46:0/3256519926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854003d80 con 0x55c168170380 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.215+0000 7f4878982b80 1 -- 172.21.15.46:0/3256519926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c168170380 msgr2=0x55c16817ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.215+0000 7f4878982b80 1 --2- 172.21.15.46:0/3256519926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c168170380 0x55c16817ace0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4854011d10 tx=0x7f4854014000 comp rx=0 tx=0).stop 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.215+0000 7f4878982b80 1 -- 172.21.15.46:0/3256519926 shutdown_connections 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.215+0000 7f4878982b80 1 --2- 172.21.15.46:0/3256519926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c16817d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.215+0000 7f4878982b80 1 --2- 172.21.15.46:0/3256519926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c16816fa60 0x55c16816fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.215+0000 7f4878982b80 1 --2- 172.21.15.46:0/3256519926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c168170380 0x55c16817ace0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.215+0000 7f4878982b80 1 -- 172.21.15.46:0/3256519926 >> 172.21.15.46:0/3256519926 conn(0x55c1680e6220 msgr2=0x55c16816c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.215+0000 7f4878982b80 1 -- 172.21.15.46:0/3256519926 shutdown_connections 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.215+0000 7f4878982b80 1 -- 172.21.15.46:0/3256519926 wait complete. 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f4878982b80 1 -- start start 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f4878982b80 1 Processor -- start 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f4878982b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16816fa60 0x55c16817abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f4878982b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c168177f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f4878982b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16817b220 0x55c168178600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f4878982b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16805ed80 con 0x55c16817b220 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f4878982b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16805ea20 con 0x55c168170380 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f4878982b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c167f466a0 con 0x55c16816fa60 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f4878982b80 10 client.0 _ll_register_callbacks cb 0x55c167eca3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f486ad8a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16817b220 0x55c168178600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f486ad8a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16817b220 0x55c168178600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36480/0 (socket says 172.21.15.46:36480) 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f486ad8a700 1 -- 172.21.15.46:0/2349262118 learned_addr learned my addr 172.21.15.46:0/2349262118 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f4869d88700 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c168177f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f486a589700 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16816fa60 0x55c16817abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.216+0000 7f486ad8a700 1 -- 172.21.15.46:0/2349262118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16816fa60 msgr2=0x55c16817abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f486ad8a700 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16816fa60 0x55c16817abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f486ad8a700 1 -- 172.21.15.46:0/2349262118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 msgr2=0x55c168177f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f486ad8a700 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c168177f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f486ad8a700 1 -- 172.21.15.46:0/2349262118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4854017070 con 0x55c16817b220 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f486ad8a700 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16817b220 0x55c168178600 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f485c007e70 tx=0x7f485c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa05fbe035a12a57 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f484dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f484dffb700 1 -- 172.21.15.46:0/2349262118 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c00e040 con 0x55c16817b220 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f484dffb700 1 -- 172.21.15.46:0/2349262118 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f485c004a50 con 0x55c16817b220 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f4878982b80 1 -- 172.21.15.46:0/2349262118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c168163ad0 con 0x55c16817b220 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f484dffb700 1 -- 172.21.15.46:0/2349262118 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f485c004c10 con 0x55c16817b220 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f4878982b80 10 client.17565 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f4878982b80 1 -- 172.21.15.46:0/2349262118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c167ef0b60 con 0x55c16817b220 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.217+0000 7f484dffb700 1 -- 172.21.15.46:0/2349262118 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c168163ad0 con 0x55c16817b220 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.218+0000 7f484dffb700 1 -- 172.21.15.46:0/2349262118 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f485c00b100 con 0x55c16817b220 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.218+0000 7f4878982b80 10 client.17565 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.218+0000 7f4878982b80 10 client.17565 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.218+0000 7f4878982b80 20 client.17565 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:00.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.218+0000 7f4878982b80 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c16818ad60 0x55c16818d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.218+0000 7f4878982b80 4 client.17565 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.218+0000 7f4878982b80 1 -- 172.21.15.46:0/2349262118 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1680bb700 con 0x55c16818ad60 2024-03-18T20:42:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.218+0000 7f486a589700 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c16818ad60 0x55c16818d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.218+0000 7f486a589700 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c16818ad60 0x55c16818d180 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd80177d530fd8f1 server_cookie=ec8e8204cdbda795 in_seq=0 out_seq=0 2024-03-18T20:42:00.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.218+0000 7f484dffb700 10 client.17565 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:42:00.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f484dffb700 1 -- 172.21.15.46:0/2349262118 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c1680bb700 con 0x55c16818ad60 2024-03-18T20:42:00.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f484dffb700 10 client.17565 handle_command_reply: tid=0 2024-03-18T20:42:00.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f4878982b80 1 client.17565 shutdown 2024-03-18T20:42:00.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f4878982b80 1 -- 172.21.15.46:0/2349262118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16817b220 msgr2=0x55c168178600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f4878982b80 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16817b220 0x55c168178600 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f485c007e70 tx=0x7f485c007ea0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f4878982b80 1 -- 172.21.15.46:0/2349262118 shutdown_connections 2024-03-18T20:42:00.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f4878982b80 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c16818ad60 0x55c16818d180 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f4878982b80 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16817b220 0x55c168178600 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f4878982b80 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16816fa60 0x55c16817abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f4878982b80 1 --2- 172.21.15.46:0/2349262118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c168177f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f4878982b80 1 -- 172.21.15.46:0/2349262118 >> 172.21.15.46:0/2349262118 conn(0x55c1680e6220 msgr2=0x55c16816bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f484dffb700 0 client.17565 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:42:00.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f4878982b80 1 -- 172.21.15.46:0/2349262118 shutdown_connections 2024-03-18T20:42:00.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f4878982b80 1 -- 172.21.15.46:0/2349262118 wait complete. 2024-03-18T20:42:00.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.219+0000 7f4878982b80 20 client.17565 trim_cache size 0 max 16384 2024-03-18T20:42:00.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.233+0000 7f4878982b80 1 Processor -- start 2024-03-18T20:42:00.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.233+0000 7f4878982b80 1 -- start start 2024-03-18T20:42:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.233+0000 7f4878982b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 0x55c1681729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.233+0000 7f4878982b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c1681784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.233+0000 7f4878982b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c1681865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.233+0000 7f4878982b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c168187900 con 0x55c16816fa60 2024-03-18T20:42:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.233+0000 7f4878982b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c168188490 con 0x55c168170380 2024-03-18T20:42:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.233+0000 7f4878982b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c168170d60 con 0x55c16817b220 2024-03-18T20:42:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.233+0000 7f486a589700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 0x55c1681729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.233+0000 7f486a589700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 0x55c1681729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36492/0 (socket says 172.21.15.46:36492) 2024-03-18T20:42:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.233+0000 7f486a589700 1 -- 172.21.15.46:0/2654824788 learned_addr learned my addr 172.21.15.46:0/2654824788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.234+0000 7f4869d88700 1 --2- 172.21.15.46:0/2654824788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c1681784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.234+0000 7f486ad8a700 1 --2- 172.21.15.46:0/2654824788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c1681865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.234+0000 7f486a589700 1 -- 172.21.15.46:0/2654824788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 msgr2=0x55c1681865a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.234+0000 7f486a589700 1 --2- 172.21.15.46:0/2654824788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c1681865a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.234+0000 7f486a589700 1 -- 172.21.15.46:0/2654824788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 msgr2=0x55c1681784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.234+0000 7f486a589700 1 --2- 172.21.15.46:0/2654824788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c1681784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.234+0000 7f486a589700 1 -- 172.21.15.46:0/2654824788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c168088ea0 con 0x55c16816fa60 2024-03-18T20:42:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.234+0000 7f4869d88700 1 --2- 172.21.15.46:0/2654824788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c1681784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.234+0000 7f486a589700 1 --2- 172.21.15.46:0/2654824788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 0x55c1681729c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f486000b540 tx=0x7f48600cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ab84c2a686f3fc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.234+0000 7f48597fa700 1 -- 172.21.15.46:0/2654824788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48600a2020 con 0x55c16816fa60 2024-03-18T20:42:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.234+0000 7f48597fa700 1 -- 172.21.15.46:0/2654824788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4860022e10 con 0x55c16816fa60 2024-03-18T20:42:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.234+0000 7f48597fa700 1 -- 172.21.15.46:0/2654824788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486009a060 con 0x55c16816fa60 2024-03-18T20:42:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.235+0000 7f4878982b80 1 -- 172.21.15.46:0/2654824788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 msgr2=0x55c1681729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.235+0000 7f4878982b80 1 --2- 172.21.15.46:0/2654824788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 0x55c1681729c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f486000b540 tx=0x7f48600cc040 comp rx=0 tx=0).stop 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.235+0000 7f4878982b80 1 -- 172.21.15.46:0/2654824788 shutdown_connections 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.235+0000 7f4878982b80 1 --2- 172.21.15.46:0/2654824788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 0x55c1681729c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.235+0000 7f4878982b80 1 --2- 172.21.15.46:0/2654824788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c1681865a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.235+0000 7f4878982b80 1 --2- 172.21.15.46:0/2654824788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c1681784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.235+0000 7f4878982b80 1 -- 172.21.15.46:0/2654824788 >> 172.21.15.46:0/2654824788 conn(0x55c1680e6220 msgr2=0x55c168176850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.235+0000 7f4878982b80 1 -- 172.21.15.46:0/2654824788 shutdown_connections 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.235+0000 7f4878982b80 1 -- 172.21.15.46:0/2654824788 wait complete. 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.235+0000 7f4878982b80 1 -- start start 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.235+0000 7f4878982b80 1 Processor -- start 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f4878982b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 0x55c16817f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f4878982b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c16817fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f4878982b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c1681802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f4878982b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1681e2150 con 0x55c16816fa60 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f4878982b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1681e22c0 con 0x55c168170380 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f4878982b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1681e2480 con 0x55c16817b220 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f4878982b80 10 client.0 _ll_register_callbacks cb 0x7f4854005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f486a589700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 0x55c16817f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f486a589700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 0x55c16817f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36508/0 (socket says 172.21.15.46:36508) 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f486a589700 1 -- 172.21.15.46:0/3858792197 learned_addr learned my addr 172.21.15.46:0/3858792197 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f486ad8a700 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c1681802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f4869d88700 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c16817fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f486a589700 1 -- 172.21.15.46:0/3858792197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 msgr2=0x55c1681802d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f486a589700 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c1681802d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f486a589700 1 -- 172.21.15.46:0/3858792197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 msgr2=0x55c16817fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f486a589700 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c16817fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f486a589700 1 -- 172.21.15.46:0/3858792197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c167f443b0 con 0x55c16816fa60 2024-03-18T20:42:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f4869d88700 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c16817fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f486ad8a700 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c1681802d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f486a589700 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 0x55c16817f830 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4860099ae0 tx=0x7f486009db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c2233c2c29f43d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.236+0000 7f484dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f484dffb700 1 -- 172.21.15.46:0/3858792197 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860099090 con 0x55c16816fa60 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f484dffb700 1 -- 172.21.15.46:0/3858792197 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f48600b9040 con 0x55c16816fa60 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f4878982b80 1 -- 172.21.15.46:0/3858792197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48540044e0 con 0x55c16816fa60 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f484dffb700 1 -- 172.21.15.46:0/3858792197 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48600991f0 con 0x55c16816fa60 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f4878982b80 10 client.17571 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f4878982b80 1 -- 172.21.15.46:0/3858792197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f48641a4340 con 0x55c16816fa60 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f484dffb700 1 -- 172.21.15.46:0/3858792197 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48600a9020 con 0x55c16816fa60 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f484dffb700 1 -- 172.21.15.46:0/3858792197 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f48600a1cd0 con 0x55c16816fa60 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f4878982b80 10 client.17571 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f4878982b80 10 client.17571 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:42:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f4878982b80 20 client.17571 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f4878982b80 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c1681e3840 0x55c1681e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f4878982b80 4 client.17571 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.237+0000 7f4878982b80 1 -- 172.21.15.46:0/3858792197 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c167f11cf0 con 0x55c1681e3840 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4869d88700 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c1681e3840 0x55c1681e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f484dffb700 10 client.17571 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4869d88700 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c1681e3840 0x55c1681e5c30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8fe8a26f2460203e server_cookie=ec0f472a0dd8d46a in_seq=0 out_seq=0 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f484dffb700 1 -- 172.21.15.46:0/3858792197 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c167f11cf0 con 0x55c1681e3840 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f484dffb700 10 client.17571 handle_command_reply: tid=0 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4878982b80 1 client.17571 shutdown 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4878982b80 1 -- 172.21.15.46:0/3858792197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 msgr2=0x55c16817f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4878982b80 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 0x55c16817f830 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4860099ae0 tx=0x7f486009db10 comp rx=0 tx=0).stop 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4878982b80 1 -- 172.21.15.46:0/3858792197 shutdown_connections 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4878982b80 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55c1681e3840 0x55c1681e5c30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4878982b80 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c16816fa60 0x55c16817f830 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f484dffb700 0 client.17571 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:42:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4878982b80 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c16817b220 0x55c1681802d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4878982b80 1 --2- 172.21.15.46:0/3858792197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c168170380 0x55c16817fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4878982b80 1 -- 172.21.15.46:0/3858792197 >> 172.21.15.46:0/3858792197 conn(0x55c1680e6220 msgr2=0x55c168179780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4878982b80 1 -- 172.21.15.46:0/3858792197 shutdown_connections 2024-03-18T20:42:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.238+0000 7f4878982b80 1 -- 172.21.15.46:0/3858792197 wait complete. 2024-03-18T20:42:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f4878982b80 20 client.17571 trim_cache size 0 max 16384 2024-03-18T20:42:00.252 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:42:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f486c7ed700 1 -- 172.21.15.46:0/3263909694 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48500a1310 msgr2=0x7f48500a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f486c7ed700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48500a1310 0x7f48500a37d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f48641987d0 tx=0x7f4854009260 comp rx=0 tx=0).stop 2024-03-18T20:42:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f486c7ed700 1 -- 172.21.15.46:0/3263909694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864069540 msgr2=0x7f4864197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f486c7ed700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864069540 0x7f4864197280 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f486000d980 tx=0x7f486000dc90 comp rx=0 tx=0).stop 2024-03-18T20:42:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f486c7ed700 1 -- 172.21.15.46:0/3263909694 shutdown_connections 2024-03-18T20:42:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f486c7ed700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48500a1310 0x7f48500a37d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f486c7ed700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864069540 0x7f4864197280 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f486c7ed700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f486419bb60 0x7f486419bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f486c7ed700 1 --2- 172.21.15.46:0/3263909694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48641060a0 0x7f48641977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f486c7ed700 1 -- 172.21.15.46:0/3263909694 >> 172.21.15.46:0/3263909694 conn(0x7f4864076d50 msgr2=0x7f48640fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f486c7ed700 1 -- 172.21.15.46:0/3263909694 shutdown_connections 2024-03-18T20:42:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.239+0000 7f486c7ed700 1 -- 172.21.15.46:0/3263909694 wait complete. 2024-03-18T20:42:00.255 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:42:00.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:42:00.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.407+0000 7ff009269700 1 -- 172.21.15.46:0/4134797199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0040ffe70 msgr2=0x7ff004100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.407+0000 7ff009269700 1 --2- 172.21.15.46:0/4134797199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0040ffe70 0x7ff004100250 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7feff4009990 tx=0x7feff4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.407+0000 7ff009269700 1 -- 172.21.15.46:0/4134797199 shutdown_connections 2024-03-18T20:42:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.407+0000 7ff009269700 1 --2- 172.21.15.46:0/4134797199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0040ffe70 0x7ff004100250 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.407+0000 7ff009269700 1 --2- 172.21.15.46:0/4134797199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff004104f70 0x7ff00410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.407+0000 7ff009269700 1 --2- 172.21.15.46:0/4134797199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff004100820 0x7ff004104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.407+0000 7ff009269700 1 -- 172.21.15.46:0/4134797199 >> 172.21.15.46:0/4134797199 conn(0x7ff004076d50 msgr2=0x7ff004077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.407+0000 7ff009269700 1 -- 172.21.15.46:0/4134797199 shutdown_connections 2024-03-18T20:42:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.407+0000 7ff009269700 1 -- 172.21.15.46:0/4134797199 wait complete. 2024-03-18T20:42:00.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.408+0000 7ff009269700 1 Processor -- start 2024-03-18T20:42:00.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.408+0000 7ff009269700 1 -- start start 2024-03-18T20:42:00.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.408+0000 7ff009269700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0040ffe70 0x7ff0041018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.408+0000 7ff009269700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff004104f70 0x7ff004101e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.408+0000 7ff009269700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff004102360 0x7ff0041a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.408+0000 7ff009269700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0041a5a70 con 0x7ff004102360 2024-03-18T20:42:00.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.408+0000 7ff009269700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0041a5be0 con 0x7ff0040ffe70 2024-03-18T20:42:00.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.408+0000 7ff009269700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0041a5da0 con 0x7ff004104f70 2024-03-18T20:42:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.408+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0040ffe70 0x7ff0041018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.408+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0040ffe70 0x7ff0041018e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:38400/0 (socket says 172.21.15.46:38400) 2024-03-18T20:42:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.408+0000 7ff002ffd700 1 -- 172.21.15.46:0/1014210722 learned_addr learned my addr 172.21.15.46:0/1014210722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.408+0000 7ff0027fc700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff004104f70 0x7ff004101e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7ff0037fe700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff004102360 0x7ff0041a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7ff002ffd700 1 -- 172.21.15.46:0/1014210722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff004104f70 msgr2=0x7ff004101e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7ff002ffd700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff004104f70 0x7ff004101e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7ff002ffd700 1 -- 172.21.15.46:0/1014210722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff004102360 msgr2=0x7ff0041a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7ff002ffd700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff004102360 0x7ff0041a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7ff002ffd700 1 -- 172.21.15.46:0/1014210722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff40095f0 con 0x7ff0040ffe70 2024-03-18T20:42:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7ff0027fc700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff004104f70 0x7ff004101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7ff002ffd700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0040ffe70 0x7ff0041018e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feff4004730 tx=0x7feff4004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7feffbfff700 1 -- 172.21.15.46:0/1014210722 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff401d070 con 0x7ff0040ffe70 2024-03-18T20:42:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7ff009269700 1 -- 172.21.15.46:0/1014210722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0041a6020 con 0x7ff0040ffe70 2024-03-18T20:42:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7feffbfff700 1 -- 172.21.15.46:0/1014210722 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff4022470 con 0x7ff0040ffe70 2024-03-18T20:42:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7feffbfff700 1 -- 172.21.15.46:0/1014210722 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4017650 con 0x7ff0040ffe70 2024-03-18T20:42:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.409+0000 7ff009269700 1 -- 172.21.15.46:0/1014210722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0041a6440 con 0x7ff0040ffe70 2024-03-18T20:42:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.412+0000 7feffbfff700 1 -- 172.21.15.46:0/1014210722 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7feff40225e0 con 0x7ff0040ffe70 2024-03-18T20:42:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.413+0000 7feffbfff700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7feff00a1290 0x7feff00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.413+0000 7ff0027fc700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7feff00a1290 0x7feff00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.413+0000 7feffbfff700 1 -- 172.21.15.46:0/1014210722 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7feff40cddd0 con 0x7ff0040ffe70 2024-03-18T20:42:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.413+0000 7ff0153feb80 1 Processor -- start 2024-03-18T20:42:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.413+0000 7ff0153feb80 1 -- start start 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.413+0000 7ff0027fc700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7feff00a1290 0x7feff00a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff004103010 tx=0x7fefec009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.413+0000 7ff0153feb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83e4a60 0x55f8d83e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.413+0000 7ff0153feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d83efce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.413+0000 7ff0153feb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83f0220 0x55f8d83f2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.413+0000 7ff0153feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d833e990 con 0x55f8d83e5380 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.413+0000 7ff0153feb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d833c350 con 0x55f8d83f0220 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.413+0000 7ff0153feb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d8337c40 con 0x55f8d83e4a60 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff0027fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d83efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83e4a60 0x55f8d83e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff0027fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d83efce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36518/0 (socket says 172.21.15.46:36518) 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff0037fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83f0220 0x55f8d83f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83e4a60 0x55f8d83e4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:53184/0 (socket says 172.21.15.46:53184) 2024-03-18T20:42:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff0027fc700 1 -- 172.21.15.46:0/2648581479 learned_addr learned my addr 172.21.15.46:0/2648581479 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff002ffd700 1 -- 172.21.15.46:0/2648581479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83f0220 msgr2=0x55f8d83f2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff002ffd700 1 --2- 172.21.15.46:0/2648581479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83f0220 0x55f8d83f2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff002ffd700 1 -- 172.21.15.46:0/2648581479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 msgr2=0x55f8d83efce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff002ffd700 1 --2- 172.21.15.46:0/2648581479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d83efce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff002ffd700 1 -- 172.21.15.46:0/2648581479 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8d81b93b0 con 0x55f8d83e4a60 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff0027fc700 1 --2- 172.21.15.46:0/2648581479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d83efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7ff002ffd700 1 --2- 172.21.15.46:0/2648581479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83e4a60 0x55f8d83e4e40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feff40cb530 tx=0x7feff409add0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849a0281a3cf517e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7feff97fa700 1 -- 172.21.15.46:0/2648581479 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff40a10d0 con 0x55f8d83e4a60 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7feff97fa700 1 -- 172.21.15.46:0/2648581479 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff409b680 con 0x55f8d83e4a60 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.414+0000 7feff97fa700 1 -- 172.21.15.46:0/2648581479 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff409b440 con 0x55f8d83e4a60 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.415+0000 7ff0153feb80 1 -- 172.21.15.46:0/2648581479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83e4a60 msgr2=0x55f8d83e4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.415+0000 7ff0153feb80 1 --2- 172.21.15.46:0/2648581479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83e4a60 0x55f8d83e4e40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feff40cb530 tx=0x7feff409add0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.415+0000 7ff0153feb80 1 -- 172.21.15.46:0/2648581479 shutdown_connections 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.415+0000 7ff0153feb80 1 --2- 172.21.15.46:0/2648581479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83f0220 0x55f8d83f2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.415+0000 7ff0153feb80 1 --2- 172.21.15.46:0/2648581479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83e4a60 0x55f8d83e4e40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.415+0000 7ff0153feb80 1 --2- 172.21.15.46:0/2648581479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d83efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.415+0000 7ff0153feb80 1 -- 172.21.15.46:0/2648581479 >> 172.21.15.46:0/2648581479 conn(0x55f8d835b220 msgr2=0x55f8d83e1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.415+0000 7ff0153feb80 1 -- 172.21.15.46:0/2648581479 shutdown_connections 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.415+0000 7ff0153feb80 1 -- 172.21.15.46:0/2648581479 wait complete. 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff0153feb80 1 -- start start 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff0153feb80 1 Processor -- start 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff0153feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e4a60 0x55f8d83f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff0153feb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e5380 0x55f8d83f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff0153feb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83f0220 0x55f8d83fb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff0153feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d82d3d80 con 0x55f8d83e4a60 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff0153feb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d82d3a20 con 0x55f8d83e5380 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff0153feb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d81bb6a0 con 0x55f8d83f0220 2024-03-18T20:42:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff0153feb80 10 client.0 _ll_register_callbacks cb 0x55f8d813f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e4a60 0x55f8d83f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e4a60 0x55f8d83f07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36524/0 (socket says 172.21.15.46:36524) 2024-03-18T20:42:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff002ffd700 1 -- 172.21.15.46:0/1348223849 learned_addr learned my addr 172.21.15.46:0/1348223849 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff0027fc700 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e5380 0x55f8d83f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.416+0000 7ff0037fe700 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83f0220 0x55f8d83fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7ff0027fc700 1 -- 172.21.15.46:0/1348223849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83f0220 msgr2=0x55f8d83fb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7ff0027fc700 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83f0220 0x55f8d83fb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7ff0027fc700 1 -- 172.21.15.46:0/1348223849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e4a60 msgr2=0x55f8d83f07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7ff0027fc700 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e4a60 0x55f8d83f07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7ff0027fc700 1 -- 172.21.15.46:0/1348223849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec017070 con 0x55f8d83e5380 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7ff002ffd700 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e4a60 0x55f8d83f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7ff0027fc700 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e5380 0x55f8d83f0d30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fefec0037c0 tx=0x7fefec0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff6d204bad16121c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7fefea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7fefea7fc700 1 -- 172.21.15.46:0/1348223849 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec01e6b0 con 0x55f8d83e5380 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7fefea7fc700 1 -- 172.21.15.46:0/1348223849 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fefec01e810 con 0x55f8d83e5380 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7ff0153feb80 1 -- 172.21.15.46:0/1348223849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8d83d8ad0 con 0x55f8d83e5380 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7fefea7fc700 1 -- 172.21.15.46:0/1348223849 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefec01e9d0 con 0x55f8d83e5380 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7ff0153feb80 10 client.27266 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.417+0000 7ff0153feb80 1 -- 172.21.15.46:0/1348223849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7ff0041a6440 con 0x55f8d83e5380 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.418+0000 7fefea7fc700 1 -- 172.21.15.46:0/1348223849 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8d83d8ad0 con 0x55f8d83e5380 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.418+0000 7fefea7fc700 1 -- 172.21.15.46:0/1348223849 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fefec031c70 con 0x55f8d83e5380 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.418+0000 7ff0153feb80 10 client.27266 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.418+0000 7ff0153feb80 10 client.27266 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.418+0000 7ff0153feb80 20 client.27266 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.418+0000 7ff0153feb80 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f8d83ffb80 0x55f8d8401fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.418+0000 7ff0153feb80 4 client.27266 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.418+0000 7ff0153feb80 1 -- 172.21.15.46:0/1348223849 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8d8330700 con 0x55f8d83ffb80 2024-03-18T20:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.418+0000 7ff002ffd700 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f8d83ffb80 0x55f8d8401fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.418+0000 7fefea7fc700 10 client.27266 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:42:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.418+0000 7ff002ffd700 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f8d83ffb80 0x55f8d8401fa0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3e2ec8a3ea34387 server_cookie=c70dabb308d39769 in_seq=0 out_seq=0 2024-03-18T20:42:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.419+0000 7fefea7fc700 1 -- 172.21.15.46:0/1348223849 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f8d8330700 con 0x55f8d83ffb80 2024-03-18T20:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.419+0000 7fefea7fc700 10 client.27266 handle_command_reply: tid=0 2024-03-18T20:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.419+0000 7ff0153feb80 1 client.27266 shutdown 2024-03-18T20:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.419+0000 7ff0153feb80 1 -- 172.21.15.46:0/1348223849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e5380 msgr2=0x55f8d83f0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.419+0000 7ff0153feb80 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e5380 0x55f8d83f0d30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fefec0037c0 tx=0x7fefec0047c0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.419+0000 7ff0153feb80 1 -- 172.21.15.46:0/1348223849 shutdown_connections 2024-03-18T20:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.419+0000 7ff0153feb80 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f8d83ffb80 0x55f8d8401fa0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.419+0000 7ff0153feb80 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e4a60 0x55f8d83f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.419+0000 7ff0153feb80 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83f0220 0x55f8d83fb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.419+0000 7fefea7fc700 0 client.27266 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.419+0000 7ff0153feb80 1 --2- 172.21.15.46:0/1348223849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e5380 0x55f8d83f0d30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.420+0000 7ff0153feb80 1 -- 172.21.15.46:0/1348223849 >> 172.21.15.46:0/1348223849 conn(0x55f8d835b220 msgr2=0x55f8d83e0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.420+0000 7ff0153feb80 1 -- 172.21.15.46:0/1348223849 shutdown_connections 2024-03-18T20:42:00.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.420+0000 7ff0153feb80 1 -- 172.21.15.46:0/1348223849 wait complete. 2024-03-18T20:42:00.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.420+0000 7ff0153feb80 20 client.27266 trim_cache size 0 max 16384 2024-03-18T20:42:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff0153feb80 1 Processor -- start 2024-03-18T20:42:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff0153feb80 1 -- start start 2024-03-18T20:42:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff0153feb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e4a60 0x55f8d83f5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff0153feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d83e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff0153feb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83f0220 0x55f8d83ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff0153feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d83f1310 con 0x55f8d83e5380 2024-03-18T20:42:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff0153feb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d83dd5e0 con 0x55f8d83e4a60 2024-03-18T20:42:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff0153feb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d83e6610 con 0x55f8d83f0220 2024-03-18T20:42:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff0027fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d83e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e4a60 0x55f8d83f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff0027fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d83e9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36530/0 (socket says 172.21.15.46:36530) 2024-03-18T20:42:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff002ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e4a60 0x55f8d83f5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:38440/0 (socket says 172.21.15.46:38440) 2024-03-18T20:42:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff0027fc700 1 -- 172.21.15.46:0/3275847958 learned_addr learned my addr 172.21.15.46:0/3275847958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.436+0000 7ff0037fe700 1 --2- 172.21.15.46:0/3275847958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83f0220 0x55f8d83ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.437+0000 7ff0027fc700 1 -- 172.21.15.46:0/3275847958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83f0220 msgr2=0x55f8d83ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.437+0000 7ff0027fc700 1 --2- 172.21.15.46:0/3275847958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83f0220 0x55f8d83ea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.437+0000 7ff0027fc700 1 -- 172.21.15.46:0/3275847958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e4a60 msgr2=0x55f8d83f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.437+0000 7ff0027fc700 1 --2- 172.21.15.46:0/3275847958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e4a60 0x55f8d83f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.437+0000 7ff0027fc700 1 -- 172.21.15.46:0/3275847958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8d8165b60 con 0x55f8d83e5380 2024-03-18T20:42:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.437+0000 7ff0037fe700 1 --2- 172.21.15.46:0/3275847958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83f0220 0x55f8d83ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.437+0000 7ff002ffd700 1 --2- 172.21.15.46:0/3275847958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e4a60 0x55f8d83f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.437+0000 7ff0027fc700 1 --2- 172.21.15.46:0/3275847958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d83e9b90 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fefec031fd0 tx=0x7fefec003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9d336dcac38f72 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.437+0000 7feff9ffb700 1 -- 172.21.15.46:0/3275847958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec028970 con 0x55f8d83e5380 2024-03-18T20:42:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.437+0000 7feff9ffb700 1 -- 172.21.15.46:0/3275847958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefec031470 con 0x55f8d83e5380 2024-03-18T20:42:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.437+0000 7feff9ffb700 1 -- 172.21.15.46:0/3275847958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec01e710 con 0x55f8d83e5380 2024-03-18T20:42:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.438+0000 7ff0153feb80 1 -- 172.21.15.46:0/3275847958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 msgr2=0x55f8d83e9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.438+0000 7ff0153feb80 1 --2- 172.21.15.46:0/3275847958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d83e9b90 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fefec031fd0 tx=0x7fefec003db0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.438+0000 7ff0153feb80 1 -- 172.21.15.46:0/3275847958 shutdown_connections 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.438+0000 7ff0153feb80 1 --2- 172.21.15.46:0/3275847958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83f0220 0x55f8d83ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.438+0000 7ff0153feb80 1 --2- 172.21.15.46:0/3275847958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83e4a60 0x55f8d83f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.438+0000 7ff0153feb80 1 --2- 172.21.15.46:0/3275847958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d83e9b90 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.438+0000 7ff0153feb80 1 -- 172.21.15.46:0/3275847958 >> 172.21.15.46:0/3275847958 conn(0x55f8d835b220 msgr2=0x55f8d83e01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.438+0000 7ff0153feb80 1 -- 172.21.15.46:0/3275847958 shutdown_connections 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.438+0000 7ff0153feb80 1 -- 172.21.15.46:0/3275847958 wait complete. 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.438+0000 7ff0153feb80 1 -- start start 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.438+0000 7ff0153feb80 1 Processor -- start 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0153feb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83e4a60 0x55f8d8447900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0153feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d8447e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0153feb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83f0220 0x55f8d8448380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0153feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d84529c0 con 0x55f8d83e5380 2024-03-18T20:42:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0153feb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d8452b00 con 0x55f8d83f0220 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0153feb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d8452c90 con 0x55f8d83e4a60 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0153feb80 10 client.0 _ll_register_callbacks cb 0x7fefec004640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0027fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d8447e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0027fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d8447e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36532/0 (socket says 172.21.15.46:36532) 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0027fc700 1 -- 172.21.15.46:0/928683269 learned_addr learned my addr 172.21.15.46:0/928683269 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0037fe700 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83f0220 0x55f8d8448380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff002ffd700 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83e4a60 0x55f8d8447900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0027fc700 1 -- 172.21.15.46:0/928683269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83e4a60 msgr2=0x55f8d8447900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0027fc700 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83e4a60 0x55f8d8447900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0027fc700 1 -- 172.21.15.46:0/928683269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83f0220 msgr2=0x55f8d8448380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0027fc700 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83f0220 0x55f8d8448380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0027fc700 1 -- 172.21.15.46:0/928683269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8d81b93b0 con 0x55f8d83e5380 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7ff0027fc700 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d8447e40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fefec009eb0 tx=0x7fefec031bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca32e449fd8b3b49 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.439+0000 7fefea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7fefea7fc700 1 -- 172.21.15.46:0/928683269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec028ab0 con 0x55f8d83e5380 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7ff0153feb80 1 -- 172.21.15.46:0/928683269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feff409b680 con 0x55f8d83e5380 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7fefea7fc700 1 -- 172.21.15.46:0/928683269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fefec036070 con 0x55f8d83e5380 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7fefea7fc700 1 -- 172.21.15.46:0/928683269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefec031df0 con 0x55f8d83e5380 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7ff0153feb80 10 client.17589 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7ff0153feb80 1 -- 172.21.15.46:0/928683269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7ff0041a6440 con 0x55f8d83e5380 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7fefea7fc700 1 -- 172.21.15.46:0/928683269 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff409b680 con 0x55f8d83e5380 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7fefea7fc700 1 -- 172.21.15.46:0/928683269 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fefec03b480 con 0x55f8d83e5380 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7ff0153feb80 10 client.17589 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7ff0153feb80 10 client.17589 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7ff0153feb80 20 client.17589 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7ff0153feb80 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f8d8454020 0x55f8d8456410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7ff0153feb80 4 client.17589 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.440+0000 7ff0153feb80 1 -- 172.21.15.46:0/928683269 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f8d8186cf0 con 0x55f8d8454020 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff002ffd700 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f8d8454020 0x55f8d8456410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff002ffd700 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f8d8454020 0x55f8d8456410 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=36ff7b1fa1db7151 server_cookie=80e61215fdc4cc1f in_seq=0 out_seq=0 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7fefea7fc700 10 client.17589 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7fefea7fc700 1 -- 172.21.15.46:0/928683269 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f8d8186cf0 con 0x55f8d8454020 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7fefea7fc700 10 client.17589 handle_command_reply: tid=0 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff0153feb80 1 client.17589 shutdown 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff0153feb80 1 -- 172.21.15.46:0/928683269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 msgr2=0x55f8d8447e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff0153feb80 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d8447e40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fefec009eb0 tx=0x7fefec031bf0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff0153feb80 1 -- 172.21.15.46:0/928683269 shutdown_connections 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff0153feb80 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8d83f0220 0x55f8d8448380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff0153feb80 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8d83e4a60 0x55f8d8447900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff0153feb80 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55f8d8454020 0x55f8d8456410 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff0153feb80 1 --2- 172.21.15.46:0/928683269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8d83e5380 0x55f8d8447e40 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff0153feb80 1 -- 172.21.15.46:0/928683269 >> 172.21.15.46:0/928683269 conn(0x55f8d835b220 msgr2=0x55f8d83ea810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7fefea7fc700 0 client.17589 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:42:00.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff0153feb80 1 -- 172.21.15.46:0/928683269 shutdown_connections 2024-03-18T20:42:00.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.441+0000 7ff0153feb80 1 -- 172.21.15.46:0/928683269 wait complete. 2024-03-18T20:42:00.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff0153feb80 20 client.17589 trim_cache size 0 max 16384 2024-03-18T20:42:00.454 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:42:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff009269700 1 -- 172.21.15.46:0/1014210722 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7feff00a1290 msgr2=0x7feff00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff009269700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7feff00a1290 0x7feff00a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff004103010 tx=0x7fefec009260 comp rx=0 tx=0).stop 2024-03-18T20:42:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff009269700 1 -- 172.21.15.46:0/1014210722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0040ffe70 msgr2=0x7ff0041018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff009269700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0040ffe70 0x7ff0041018e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feff4004730 tx=0x7feff4004760 comp rx=0 tx=0).stop 2024-03-18T20:42:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff009269700 1 -- 172.21.15.46:0/1014210722 shutdown_connections 2024-03-18T20:42:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff009269700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7feff00a1290 0x7feff00a3750 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff009269700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff004102360 0x7ff0041a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff009269700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0040ffe70 0x7ff0041018e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff009269700 1 --2- 172.21.15.46:0/1014210722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff004104f70 0x7ff004101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff009269700 1 -- 172.21.15.46:0/1014210722 >> 172.21.15.46:0/1014210722 conn(0x7ff004076d50 msgr2=0x7ff0040696d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff009269700 1 -- 172.21.15.46:0/1014210722 shutdown_connections 2024-03-18T20:42:00.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.442+0000 7ff009269700 1 -- 172.21.15.46:0/1014210722 wait complete. 2024-03-18T20:42:00.458 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:00 smithi046 ceph-mon[43495]: pgmap v180: 97 pgs: 97 active+clean; 78 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-03-18T20:42:00.458 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:00 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2303288455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:42:00.459 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:42:00.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:42:00.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:00 smithi073 ceph-mon[47384]: pgmap v180: 97 pgs: 97 active+clean; 78 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-03-18T20:42:00.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:00 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2303288455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:42:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:00 smithi155 ceph-mon[48344]: pgmap v180: 97 pgs: 97 active+clean; 78 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-03-18T20:42:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:00 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2303288455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:42:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.611+0000 7fb4032d4700 1 -- 172.21.15.46:0/3888338992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3fc106080 msgr2=0x7fb3fc10dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.611+0000 7fb4032d4700 1 --2- 172.21.15.46:0/3888338992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3fc106080 0x7fb3fc10dc00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400b0a0 tx=0x7fb3f400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.611+0000 7fb4032d4700 1 -- 172.21.15.46:0/3888338992 shutdown_connections 2024-03-18T20:42:00.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.611+0000 7fb4032d4700 1 --2- 172.21.15.46:0/3888338992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3fc106080 0x7fb3fc10dc00 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.611+0000 7fb4032d4700 1 --2- 172.21.15.46:0/3888338992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3fc100f60 0x7fb3fc101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.611+0000 7fb4032d4700 1 --2- 172.21.15.46:0/3888338992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3fc101910 0x7fb3fc105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.611+0000 7fb4032d4700 1 -- 172.21.15.46:0/3888338992 >> 172.21.15.46:0/3888338992 conn(0x7fb3fc0fc7c0 msgr2=0x7fb3fc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.611+0000 7fb4032d4700 1 -- 172.21.15.46:0/3888338992 shutdown_connections 2024-03-18T20:42:00.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.611+0000 7fb4032d4700 1 -- 172.21.15.46:0/3888338992 wait complete. 2024-03-18T20:42:00.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb4032d4700 1 Processor -- start 2024-03-18T20:42:00.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb4032d4700 1 -- start start 2024-03-18T20:42:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb4032d4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3fc100f60 0x7fb3fc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb4032d4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3fc106080 0x7fb3fc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb4032d4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3fc073b30 0x7fb3fc1a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb4032d4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc1a5c40 con 0x7fb3fc073b30 2024-03-18T20:42:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb4032d4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc1a5db0 con 0x7fb3fc100f60 2024-03-18T20:42:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb4032d4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc1a5f70 con 0x7fb3fc106080 2024-03-18T20:42:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb401070700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3fc100f60 0x7fb3fc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb401070700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3fc100f60 0x7fb3fc0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:38460/0 (socket says 172.21.15.46:38460) 2024-03-18T20:42:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb401070700 1 -- 172.21.15.46:0/1640646522 learned_addr learned my addr 172.21.15.46:0/1640646522 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb401871700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3fc073b30 0x7fb3fc1a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.612+0000 7fb40086f700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3fc106080 0x7fb3fc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.613+0000 7fb401070700 1 -- 172.21.15.46:0/1640646522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3fc106080 msgr2=0x7fb3fc0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.613+0000 7fb401070700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3fc106080 0x7fb3fc0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.613+0000 7fb401070700 1 -- 172.21.15.46:0/1640646522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3fc073b30 msgr2=0x7fb3fc1a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.613+0000 7fb401070700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3fc073b30 0x7fb3fc1a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.613+0000 7fb401070700 1 -- 172.21.15.46:0/1640646522 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f4009d00 con 0x7fb3fc100f60 2024-03-18T20:42:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.613+0000 7fb401070700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3fc100f60 0x7fb3fc0730b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800b530 tx=0x7fb3f800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.613+0000 7fb3f27fc700 1 -- 172.21.15.46:0/1640646522 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8011830 con 0x7fb3fc100f60 2024-03-18T20:42:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.613+0000 7fb4032d4700 1 -- 172.21.15.46:0/1640646522 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3fc1a6250 con 0x7fb3fc100f60 2024-03-18T20:42:00.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.613+0000 7fb3f27fc700 1 -- 172.21.15.46:0/1640646522 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3f8011e50 con 0x7fb3fc100f60 2024-03-18T20:42:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.613+0000 7fb3f27fc700 1 -- 172.21.15.46:0/1640646522 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f800f2e0 con 0x7fb3fc100f60 2024-03-18T20:42:00.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.613+0000 7fb4032d4700 1 -- 172.21.15.46:0/1640646522 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3fc1a67b0 con 0x7fb3fc100f60 2024-03-18T20:42:00.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.615+0000 7fb3f27fc700 1 -- 172.21.15.46:0/1640646522 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fb3f80105a0 con 0x7fb3fc100f60 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.616+0000 7fb3f27fc700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb3e80a1290 0x7fb3e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.616+0000 7fb40086f700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb3e80a1290 0x7fb3e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.616+0000 7fb3f27fc700 1 -- 172.21.15.46:0/1640646522 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb3f80cc610 con 0x7fb3fc100f60 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.617+0000 7fb40086f700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb3e80a1290 0x7fb3e80a3750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0747e0 tx=0x7fb3ec009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.617+0000 7fb40f469b80 1 Processor -- start 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.617+0000 7fb40f469b80 1 -- start start 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40f469b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 0x55f55cdd6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40f469b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 0x55f55cde1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40f469b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cde2220 0x55f55cde4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40f469b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55cd30990 con 0x55f55cdd7380 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40f469b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55cd2e350 con 0x55f55cde2220 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40f469b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55cd29c40 con 0x55f55cdd6a60 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40086f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 0x55f55cde1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40086f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 0x55f55cde1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36574/0 (socket says 172.21.15.46:36574) 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40086f700 1 -- 172.21.15.46:0/4272782551 learned_addr learned my addr 172.21.15.46:0/4272782551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb401871700 1 --2- 172.21.15.46:0/4272782551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cde2220 0x55f55cde4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb401070700 1 --2- 172.21.15.46:0/4272782551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 0x55f55cdd6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40086f700 1 -- 172.21.15.46:0/4272782551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 msgr2=0x55f55cdd6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40086f700 1 --2- 172.21.15.46:0/4272782551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 0x55f55cdd6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40086f700 1 -- 172.21.15.46:0/4272782551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cde2220 msgr2=0x55f55cde4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40086f700 1 --2- 172.21.15.46:0/4272782551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cde2220 0x55f55cde4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb40086f700 1 -- 172.21.15.46:0/4272782551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f55cbab3b0 con 0x55f55cdd7380 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.618+0000 7fb401070700 1 --2- 172.21.15.46:0/4272782551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 0x55f55cdd6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.619+0000 7fb40086f700 1 --2- 172.21.15.46:0/4272782551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 0x55f55cde1ce0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec011d40 tx=0x7fb3ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf78916aab6e29d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.619+0000 7fb3e77fe700 1 -- 172.21.15.46:0/4272782551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec004a30 con 0x55f55cdd7380 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.619+0000 7fb3e77fe700 1 -- 172.21.15.46:0/4272782551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3ec011e70 con 0x55f55cdd7380 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.619+0000 7fb3e77fe700 1 -- 172.21.15.46:0/4272782551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0039f0 con 0x55f55cdd7380 2024-03-18T20:42:00.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.620+0000 7fb40f469b80 1 -- 172.21.15.46:0/4272782551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 msgr2=0x55f55cde1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.620+0000 7fb40f469b80 1 --2- 172.21.15.46:0/4272782551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 0x55f55cde1ce0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec011d40 tx=0x7fb3ec014000 comp rx=0 tx=0).stop 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.620+0000 7fb40f469b80 1 -- 172.21.15.46:0/4272782551 shutdown_connections 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.620+0000 7fb40f469b80 1 --2- 172.21.15.46:0/4272782551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cde2220 0x55f55cde4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.620+0000 7fb40f469b80 1 --2- 172.21.15.46:0/4272782551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 0x55f55cdd6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.620+0000 7fb40f469b80 1 --2- 172.21.15.46:0/4272782551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 0x55f55cde1ce0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.620+0000 7fb40f469b80 1 -- 172.21.15.46:0/4272782551 >> 172.21.15.46:0/4272782551 conn(0x55f55cd4d220 msgr2=0x55f55cdd3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.620+0000 7fb40f469b80 1 -- 172.21.15.46:0/4272782551 shutdown_connections 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.620+0000 7fb40f469b80 1 -- 172.21.15.46:0/4272782551 wait complete. 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.620+0000 7fb40f469b80 1 -- start start 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.620+0000 7fb40f469b80 1 Processor -- start 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.621+0000 7fb40f469b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd6a60 0x55f55cddeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.621+0000 7fb40f469b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd7380 0x55f55cddf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.621+0000 7fb40f469b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cde2220 0x55f55cddfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.621+0000 7fb40f469b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55ccc5d80 con 0x55f55cdd6a60 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.621+0000 7fb40f469b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55ccc5a20 con 0x55f55cde2220 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.621+0000 7fb40f469b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55cbad6a0 con 0x55f55cdd7380 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.621+0000 7fb40f469b80 10 client.0 _ll_register_callbacks cb 0x55f55cb313a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.621+0000 7fb401070700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd6a60 0x55f55cddeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.621+0000 7fb401070700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd6a60 0x55f55cddeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36576/0 (socket says 172.21.15.46:36576) 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.621+0000 7fb401070700 1 -- 172.21.15.46:0/1157915633 learned_addr learned my addr 172.21.15.46:0/1157915633 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.621+0000 7fb401871700 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cde2220 0x55f55cddfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.621+0000 7fb40086f700 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd7380 0x55f55cddf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb401070700 1 -- 172.21.15.46:0/1157915633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd7380 msgr2=0x55f55cddf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb401070700 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd7380 0x55f55cddf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb401070700 1 -- 172.21.15.46:0/1157915633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cde2220 msgr2=0x55f55cddfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb401070700 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cde2220 0x55f55cddfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb401070700 1 -- 172.21.15.46:0/1157915633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ec017070 con 0x55f55cdd6a60 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb401070700 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd6a60 0x55f55cddeff0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8095d00 tx=0x7fb3f8099f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45497946a04d6f8c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb3e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb3e4ff9700 1 -- 172.21.15.46:0/1157915633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80a1c80 con 0x55f55cdd6a60 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb3e4ff9700 1 -- 172.21.15.46:0/1157915633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb3f80b90c0 con 0x55f55cdd6a60 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb40f469b80 1 -- 172.21.15.46:0/1157915633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f55cdcaad0 con 0x55f55cdd6a60 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb3e4ff9700 1 -- 172.21.15.46:0/1157915633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3f80b9280 con 0x55f55cdd6a60 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb40f469b80 10 client.17601 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.622+0000 7fb40f469b80 1 -- 172.21.15.46:0/1157915633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f55cb57b60 con 0x55f55cdd6a60 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.623+0000 7fb3e4ff9700 1 -- 172.21.15.46:0/1157915633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f55cdcaad0 con 0x55f55cdd6a60 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.623+0000 7fb3e4ff9700 1 -- 172.21.15.46:0/1157915633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb3f8098cf0 con 0x55f55cdd6a60 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.623+0000 7fb40f469b80 10 client.17601 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.623+0000 7fb40f469b80 10 client.17601 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.623+0000 7fb40f469b80 20 client.17601 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.623+0000 7fb40f469b80 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55f55cdf1eb0 0x55f55cdf42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.623+0000 7fb40f469b80 4 client.17601 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.623+0000 7fb40f469b80 1 -- 172.21.15.46:0/1157915633 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f55cd22700 con 0x55f55cdf1eb0 2024-03-18T20:42:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.623+0000 7fb40086f700 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55f55cdf1eb0 0x55f55cdf42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb40086f700 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55f55cdf1eb0 0x55f55cdf42d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eea216e8ec97a144 server_cookie=d12df7d1fd1d7fc2 in_seq=0 out_seq=0 2024-03-18T20:42:00.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb3e4ff9700 10 client.17601 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:42:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb3e4ff9700 1 -- 172.21.15.46:0/1157915633 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f55cd22700 con 0x55f55cdf1eb0 2024-03-18T20:42:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb3e4ff9700 10 client.17601 handle_command_reply: tid=0 2024-03-18T20:42:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb40f469b80 1 client.17601 shutdown 2024-03-18T20:42:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb40f469b80 1 -- 172.21.15.46:0/1157915633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd6a60 msgr2=0x55f55cddeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb40f469b80 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd6a60 0x55f55cddeff0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8095d00 tx=0x7fb3f8099f00 comp rx=0 tx=0).stop 2024-03-18T20:42:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb40f469b80 1 -- 172.21.15.46:0/1157915633 shutdown_connections 2024-03-18T20:42:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb40f469b80 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55f55cdf1eb0 0x55f55cdf42d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb40f469b80 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd6a60 0x55f55cddeff0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb40f469b80 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cde2220 0x55f55cddfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb40f469b80 1 --2- 172.21.15.46:0/1157915633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd7380 0x55f55cddf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb3e4ff9700 0 client.17601 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:42:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.624+0000 7fb40f469b80 1 -- 172.21.15.46:0/1157915633 >> 172.21.15.46:0/1157915633 conn(0x55f55cd4d220 msgr2=0x55f55cdd2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.625+0000 7fb40f469b80 1 -- 172.21.15.46:0/1157915633 shutdown_connections 2024-03-18T20:42:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.625+0000 7fb40f469b80 1 -- 172.21.15.46:0/1157915633 wait complete. 2024-03-18T20:42:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.625+0000 7fb40f469b80 20 client.17601 trim_cache size 0 max 16384 2024-03-18T20:42:00.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.641+0000 7fb40f469b80 1 Processor -- start 2024-03-18T20:42:00.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.641+0000 7fb40f469b80 1 -- start start 2024-03-18T20:42:00.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.641+0000 7fb40f469b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cdd6a60 0x55f55cde8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.641+0000 7fb40f469b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 0x55f55cdda320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.641+0000 7fb40f469b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cde2220 0x55f55cdda860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.641+0000 7fb40f469b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55cdd7a30 con 0x55f55cdd7380 2024-03-18T20:42:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.641+0000 7fb40f469b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55cdd9eb0 con 0x55f55cdd6a60 2024-03-18T20:42:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.641+0000 7fb40f469b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55cdef5a0 con 0x55f55cde2220 2024-03-18T20:42:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb40086f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 0x55f55cdda320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb40086f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 0x55f55cdda320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36584/0 (socket says 172.21.15.46:36584) 2024-03-18T20:42:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb401871700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cde2220 0x55f55cdda860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb40086f700 1 -- 172.21.15.46:0/1413889490 learned_addr learned my addr 172.21.15.46:0/1413889490 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb401070700 1 --2- 172.21.15.46:0/1413889490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cdd6a60 0x55f55cde8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb401070700 1 -- 172.21.15.46:0/1413889490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cde2220 msgr2=0x55f55cdda860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb401070700 1 --2- 172.21.15.46:0/1413889490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cde2220 0x55f55cdda860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb401070700 1 -- 172.21.15.46:0/1413889490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 msgr2=0x55f55cdda320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb401070700 1 --2- 172.21.15.46:0/1413889490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 0x55f55cdda320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb401070700 1 -- 172.21.15.46:0/1413889490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f55ccefea0 con 0x55f55cdd6a60 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb40086f700 1 --2- 172.21.15.46:0/1413889490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 0x55f55cdda320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb401070700 1 --2- 172.21.15.46:0/1413889490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cdd6a60 0x55f55cde8e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8097b50 tx=0x7fb3f8004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=478d460879ade7f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb3e7fff700 1 -- 172.21.15.46:0/1413889490 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8095b40 con 0x55f55cdd6a60 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb3e7fff700 1 -- 172.21.15.46:0/1413889490 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3f80a1c80 con 0x55f55cdd6a60 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.642+0000 7fb3e7fff700 1 -- 172.21.15.46:0/1413889490 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80b90b0 con 0x55f55cdd6a60 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.643+0000 7fb40f469b80 1 -- 172.21.15.46:0/1413889490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cdd6a60 msgr2=0x55f55cde8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.643+0000 7fb40f469b80 1 --2- 172.21.15.46:0/1413889490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cdd6a60 0x55f55cde8e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8097b50 tx=0x7fb3f8004860 comp rx=0 tx=0).stop 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.643+0000 7fb40f469b80 1 -- 172.21.15.46:0/1413889490 shutdown_connections 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.643+0000 7fb40f469b80 1 --2- 172.21.15.46:0/1413889490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cde2220 0x55f55cdda860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.643+0000 7fb40f469b80 1 --2- 172.21.15.46:0/1413889490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cdd6a60 0x55f55cde8e50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.643+0000 7fb40f469b80 1 --2- 172.21.15.46:0/1413889490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cdd7380 0x55f55cdda320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.643+0000 7fb40f469b80 1 -- 172.21.15.46:0/1413889490 >> 172.21.15.46:0/1413889490 conn(0x55f55cd4d220 msgr2=0x55f55cddf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.643+0000 7fb40f469b80 1 -- 172.21.15.46:0/1413889490 shutdown_connections 2024-03-18T20:42:00.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.643+0000 7fb40f469b80 1 -- 172.21.15.46:0/1413889490 wait complete. 2024-03-18T20:42:00.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.643+0000 7fb40f469b80 1 -- start start 2024-03-18T20:42:00.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.644+0000 7fb40f469b80 1 Processor -- start 2024-03-18T20:42:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.644+0000 7fb40f469b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 0x55f55ce39c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.644+0000 7fb40f469b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cdd7380 0x55f55ce3a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.644+0000 7fb40f469b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cde2220 0x55f55ce44cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.644+0000 7fb40f469b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55ce3aa80 con 0x55f55cde2220 2024-03-18T20:42:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.644+0000 7fb40f469b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55ce451f0 con 0x55f55cdd7380 2024-03-18T20:42:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.644+0000 7fb40f469b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55ce45360 con 0x55f55cdd6a60 2024-03-18T20:42:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.644+0000 7fb40f469b80 10 client.0 _ll_register_callbacks cb 0x7fb3f8099340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.644+0000 7fb401070700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 0x55f55ce39c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.644+0000 7fb401070700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 0x55f55ce39c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:53266/0 (socket says 172.21.15.46:53266) 2024-03-18T20:42:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.644+0000 7fb401070700 1 -- 172.21.15.46:0/656145112 learned_addr learned my addr 172.21.15.46:0/656145112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.644+0000 7fb40086f700 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cdd7380 0x55f55ce3a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb401871700 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cde2220 0x55f55ce44cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb401070700 1 -- 172.21.15.46:0/656145112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cdd7380 msgr2=0x55f55ce3a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb401070700 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cdd7380 0x55f55ce3a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb401070700 1 -- 172.21.15.46:0/656145112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cde2220 msgr2=0x55f55ce44cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb401070700 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cde2220 0x55f55ce44cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb401070700 1 -- 172.21.15.46:0/656145112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f55cbab3b0 con 0x55f55cdd6a60 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb401871700 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cde2220 0x55f55ce44cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb401070700 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 0x55f55ce39c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80a1750 tx=0x7fb3f8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cded656b8188b74b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb3e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb3e4ff9700 1 -- 172.21.15.46:0/656145112 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8098ea0 con 0x55f55cdd6a60 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb3e4ff9700 1 -- 172.21.15.46:0/656145112 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb3f8099800 con 0x55f55cdd6a60 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb40f469b80 1 -- 172.21.15.46:0/656145112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3ec011e70 con 0x55f55cdd6a60 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb3e4ff9700 1 -- 172.21.15.46:0/656145112 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3f8099c70 con 0x55f55cdd6a60 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb40f469b80 10 client.27307 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.645+0000 7fb40f469b80 1 -- 172.21.15.46:0/656145112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f55cb57b60 con 0x55f55cdd6a60 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.646+0000 7fb3e4ff9700 1 -- 172.21.15.46:0/656145112 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3ec011e70 con 0x55f55cdd6a60 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.646+0000 7fb3e4ff9700 1 -- 172.21.15.46:0/656145112 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb3f80a7020 con 0x55f55cdd6a60 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.646+0000 7fb40f469b80 10 client.27307 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.646+0000 7fb40f469b80 10 client.27307 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.646+0000 7fb40f469b80 20 client.27307 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.646+0000 7fb40f469b80 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55f55ce466a0 0x55f55ce48a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.646+0000 7fb40f469b80 4 client.27307 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.646+0000 7fb40f469b80 1 -- 172.21.15.46:0/656145112 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f55cb78cf0 con 0x55f55ce466a0 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.646+0000 7fb40086f700 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55f55ce466a0 0x55f55ce48a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40086f700 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55f55ce466a0 0x55f55ce48a90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42318ec2fe6257dd server_cookie=5b3b2d4f274c2ba9 in_seq=0 out_seq=0 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb3e4ff9700 10 client.27307 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb3e4ff9700 1 -- 172.21.15.46:0/656145112 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f55cb78cf0 con 0x55f55ce466a0 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb3e4ff9700 10 client.27307 handle_command_reply: tid=0 2024-03-18T20:42:00.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40f469b80 1 client.27307 shutdown 2024-03-18T20:42:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40f469b80 1 -- 172.21.15.46:0/656145112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 msgr2=0x55f55ce39c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40f469b80 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 0x55f55ce39c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80a1750 tx=0x7fb3f8000bc0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40f469b80 1 -- 172.21.15.46:0/656145112 shutdown_connections 2024-03-18T20:42:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40f469b80 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55f55ce466a0 0x55f55ce48a90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40f469b80 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f55cde2220 0x55f55ce44cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40f469b80 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f55cdd6a60 0x55f55ce39c30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40f469b80 1 --2- 172.21.15.46:0/656145112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f55cdd7380 0x55f55ce3a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb3e4ff9700 0 client.27307 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:42:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40f469b80 1 -- 172.21.15.46:0/656145112 >> 172.21.15.46:0/656145112 conn(0x55f55cd4d220 msgr2=0x55f55cdda320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40f469b80 1 -- 172.21.15.46:0/656145112 shutdown_connections 2024-03-18T20:42:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40f469b80 1 -- 172.21.15.46:0/656145112 wait complete. 2024-03-18T20:42:00.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.647+0000 7fb40f469b80 20 client.27307 trim_cache size 0 max 16384 2024-03-18T20:42:00.660 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:42:00.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.648+0000 7fb4032d4700 1 -- 172.21.15.46:0/1640646522 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb3e80a1290 msgr2=0x7fb3e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.648+0000 7fb4032d4700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb3e80a1290 0x7fb3e80a3750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0747e0 tx=0x7fb3ec009260 comp rx=0 tx=0).stop 2024-03-18T20:42:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.648+0000 7fb4032d4700 1 -- 172.21.15.46:0/1640646522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3fc100f60 msgr2=0x7fb3fc0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.648+0000 7fb4032d4700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3fc100f60 0x7fb3fc0730b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800b530 tx=0x7fb3f800b840 comp rx=0 tx=0).stop 2024-03-18T20:42:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.648+0000 7fb4032d4700 1 -- 172.21.15.46:0/1640646522 shutdown_connections 2024-03-18T20:42:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.648+0000 7fb4032d4700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb3e80a1290 0x7fb3e80a3750 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.648+0000 7fb4032d4700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3fc073b30 0x7fb3fc1a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.648+0000 7fb4032d4700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3fc100f60 0x7fb3fc0730b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.648+0000 7fb4032d4700 1 --2- 172.21.15.46:0/1640646522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb3fc106080 0x7fb3fc0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.648+0000 7fb4032d4700 1 -- 172.21.15.46:0/1640646522 >> 172.21.15.46:0/1640646522 conn(0x7fb3fc0fc7c0 msgr2=0x7fb3fc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.648+0000 7fb4032d4700 1 -- 172.21.15.46:0/1640646522 shutdown_connections 2024-03-18T20:42:00.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.648+0000 7fb4032d4700 1 -- 172.21.15.46:0/1640646522 wait complete. 2024-03-18T20:42:00.664 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:42:00.665 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:42:00.693 INFO:tasks.workunit.client.0.smithi046.stdout: 1 190293 3.31 MB/sec execute 175 sec latency 2838.961 ms 2024-03-18T20:42:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.801+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/40818969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f440ffe50 msgr2=0x7f3f44100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.801+0000 7f3f4ac9b700 1 --2- 172.21.15.46:0/40818969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f440ffe50 0x7f3f44100230 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009990 tx=0x7f3f34009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.801+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/40818969 shutdown_connections 2024-03-18T20:42:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.801+0000 7f3f4ac9b700 1 --2- 172.21.15.46:0/40818969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f440ffe50 0x7f3f44100230 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.801+0000 7f3f4ac9b700 1 --2- 172.21.15.46:0/40818969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f44104f60 0x7f3f4410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.801+0000 7f3f4ac9b700 1 --2- 172.21.15.46:0/40818969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f44100800 0x7f3f44104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.802+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/40818969 >> 172.21.15.46:0/40818969 conn(0x7f3f44076d40 msgr2=0x7f3f44077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.802+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/40818969 shutdown_connections 2024-03-18T20:42:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.802+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/40818969 wait complete. 2024-03-18T20:42:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.802+0000 7f3f4ac9b700 1 Processor -- start 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.802+0000 7f3f4ac9b700 1 -- start start 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.802+0000 7f3f4ac9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f440ffe50 0x7f3f44197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.802+0000 7f3f4ac9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f44104f60 0x7f3f44197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.802+0000 7f3f4ac9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f4419bb20 0x7f3f4419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.802+0000 7f3f4ac9b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4419c5a0 con 0x7f3f4419bb20 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.802+0000 7f3f4ac9b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4419c710 con 0x7f3f44104f60 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.802+0000 7f3f4ac9b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4419c8d0 con 0x7f3f440ffe50 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f49238700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f4419bb20 0x7f3f4419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f48a37700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f440ffe50 0x7f3f44197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f49238700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f4419bb20 0x7f3f4419bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36612/0 (socket says 172.21.15.46:36612) 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f49238700 1 -- 172.21.15.46:0/3102872208 learned_addr learned my addr 172.21.15.46:0/3102872208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f48a37700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f440ffe50 0x7f3f44197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:53280/0 (socket says 172.21.15.46:53280) 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f43fff700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f44104f60 0x7f3f44197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f49238700 1 -- 172.21.15.46:0/3102872208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f440ffe50 msgr2=0x7f3f44197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f49238700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f440ffe50 0x7f3f44197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f49238700 1 -- 172.21.15.46:0/3102872208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f44104f60 msgr2=0x7f3f44197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f49238700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f44104f60 0x7f3f44197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f49238700 1 -- 172.21.15.46:0/3102872208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f340095f0 con 0x7f3f4419bb20 2024-03-18T20:42:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f48a37700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f440ffe50 0x7f3f44197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f43fff700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f44104f60 0x7f3f44197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f49238700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f4419bb20 0x7f3f4419bfa0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00ef30 tx=0x7f3f3c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f41ffb700 1 -- 172.21.15.46:0/3102872208 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c018070 con 0x7f3f4419bb20 2024-03-18T20:42:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f41ffb700 1 -- 172.21.15.46:0/3102872208 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f3c00f040 con 0x7f3f4419bb20 2024-03-18T20:42:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/3102872208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f441a3dd0 con 0x7f3f4419bb20 2024-03-18T20:42:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.803+0000 7f3f41ffb700 1 -- 172.21.15.46:0/3102872208 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c00cd00 con 0x7f3f4419bb20 2024-03-18T20:42:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.804+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/3102872208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f441a4260 con 0x7f3f4419bb20 2024-03-18T20:42:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.806+0000 7f3f41ffb700 1 -- 172.21.15.46:0/3102872208 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3f3c003a00 con 0x7f3f4419bb20 2024-03-18T20:42:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.806+0000 7f3f41ffb700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f300a1290 0x7f3f300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.807+0000 7f3f41ffb700 1 -- 172.21.15.46:0/3102872208 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f3f3c0cc7f0 con 0x7f3f4419bb20 2024-03-18T20:42:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.807+0000 7f3f48a37700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f300a1290 0x7f3f300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.807+0000 7f3f48a37700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f300a1290 0x7f3f300a3750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400b2e0 tx=0x7f3f34005fd0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f56e30b80 1 Processor -- start 2024-03-18T20:42:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f56e30b80 1 -- start start 2024-03-18T20:42:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f56e30b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558aa60 0x55887558ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f56e30b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558b380 0x558875595ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f56e30b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 0x558875598610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f56e30b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588754e4990 con 0x558875596220 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f56e30b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588754e2350 con 0x55887558b380 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f56e30b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588754ddc40 con 0x55887558aa60 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f49238700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 0x558875598610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f48a37700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558aa60 0x55887558ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f49238700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 0x558875598610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36628/0 (socket says 172.21.15.46:36628) 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f49238700 1 -- 172.21.15.46:0/1154911527 learned_addr learned my addr 172.21.15.46:0/1154911527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f43fff700 1 --2- 172.21.15.46:0/1154911527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558b380 0x558875595ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f49238700 1 -- 172.21.15.46:0/1154911527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558aa60 msgr2=0x55887558ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f49238700 1 --2- 172.21.15.46:0/1154911527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558aa60 0x55887558ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f49238700 1 -- 172.21.15.46:0/1154911527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558b380 msgr2=0x558875595ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f49238700 1 --2- 172.21.15.46:0/1154911527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558b380 0x558875595ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f49238700 1 -- 172.21.15.46:0/1154911527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55887535f3b0 con 0x558875596220 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.808+0000 7f3f48a37700 1 --2- 172.21.15.46:0/1154911527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558aa60 0x55887558ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.809+0000 7f3f49238700 1 --2- 172.21.15.46:0/1154911527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 0x558875598610 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0a1820 tx=0x7f3f3c0994a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23d13bc86c49221 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.809+0000 7f3f3affd700 1 -- 172.21.15.46:0/1154911527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c004ad0 con 0x558875596220 2024-03-18T20:42:00.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.809+0000 7f3f3affd700 1 -- 172.21.15.46:0/1154911527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f3c01b320 con 0x558875596220 2024-03-18T20:42:00.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.809+0000 7f3f3affd700 1 -- 172.21.15.46:0/1154911527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c099bc0 con 0x558875596220 2024-03-18T20:42:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.810+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1154911527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 msgr2=0x558875598610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.810+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/1154911527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 0x558875598610 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0a1820 tx=0x7f3f3c0994a0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.810+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1154911527 shutdown_connections 2024-03-18T20:42:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.810+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/1154911527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 0x558875598610 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.810+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/1154911527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558aa60 0x55887558ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.810+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/1154911527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558b380 0x558875595ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.810+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1154911527 >> 172.21.15.46:0/1154911527 conn(0x558875501220 msgr2=0x558875587020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.810+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1154911527 shutdown_connections 2024-03-18T20:42:00.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.810+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1154911527 wait complete. 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.810+0000 7f3f56e30b80 1 -- start start 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.810+0000 7f3f56e30b80 1 Processor -- start 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f56e30b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 0x558875595ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f56e30b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558b380 0x558875592f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f56e30b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558875596220 0x558875593680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f56e30b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558875479d80 con 0x55887558aa60 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f56e30b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558875479a20 con 0x558875596220 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f56e30b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588753616a0 con 0x55887558b380 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f56e30b80 10 client.0 _ll_register_callbacks cb 0x5588752e53a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f48a37700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 0x558875595ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f48a37700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 0x558875595ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36634/0 (socket says 172.21.15.46:36634) 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f43fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558b380 0x558875592f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f48a37700 1 -- 172.21.15.46:0/1380238489 learned_addr learned my addr 172.21.15.46:0/1380238489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f49238700 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558875596220 0x558875593680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f48a37700 1 -- 172.21.15.46:0/1380238489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558b380 msgr2=0x558875592f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f48a37700 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558b380 0x558875592f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f48a37700 1 -- 172.21.15.46:0/1380238489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558875596220 msgr2=0x558875593680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f48a37700 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558875596220 0x558875593680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f48a37700 1 -- 172.21.15.46:0/1380238489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f44067df0 con 0x55887558aa60 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f49238700 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558875596220 0x558875593680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f43fff700 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558b380 0x558875592f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f48a37700 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 0x558875595ca0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3f34015000 tx=0x7f3f34020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191417b807f25e90 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.811+0000 7f3f23fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f23fff700 1 -- 172.21.15.46:0/1380238489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3402d040 con 0x55887558aa60 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f23fff700 1 -- 172.21.15.46:0/1380238489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f3f34026230 con 0x55887558aa60 2024-03-18T20:42:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1380238489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55887557ead0 con 0x55887558aa60 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f23fff700 1 -- 172.21.15.46:0/1380238489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f34026840 con 0x55887558aa60 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f56e30b80 10 client.17619 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1380238489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f3f3c084040 con 0x55887558aa60 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f23fff700 1 -- 172.21.15.46:0/1380238489 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f3403e080 con 0x55887558aa60 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f23fff700 1 -- 172.21.15.46:0/1380238489 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3f34032af0 con 0x55887558aa60 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f56e30b80 10 client.17619 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f56e30b80 10 client.17619 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f56e30b80 20 client.17619 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5588755a5d90 0x5588755a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f56e30b80 4 client.17619 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1380238489 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588754d6700 con 0x5588755a5d90 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.812+0000 7f3f43fff700 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5588755a5d90 0x5588755a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.813+0000 7f3f23fff700 10 client.17619 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.813+0000 7f3f43fff700 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5588755a5d90 0x5588755a81b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=131eae0839ecfd39 server_cookie=26563901b418e98e in_seq=0 out_seq=0 2024-03-18T20:42:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.813+0000 7f3f23fff700 1 -- 172.21.15.46:0/1380238489 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5588754d6700 con 0x5588755a5d90 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.813+0000 7f3f23fff700 10 client.17619 handle_command_reply: tid=0 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.813+0000 7f3f56e30b80 1 client.17619 shutdown 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.813+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1380238489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 msgr2=0x558875595ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.813+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 0x558875595ca0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3f34015000 tx=0x7f3f34020000 comp rx=0 tx=0).stop 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.814+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1380238489 shutdown_connections 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.814+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5588755a5d90 0x5588755a81b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.814+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 0x558875595ca0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.814+0000 7f3f23fff700 0 client.17619 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.814+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558875596220 0x558875593680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.814+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/1380238489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558b380 0x558875592f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.814+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1380238489 >> 172.21.15.46:0/1380238489 conn(0x558875501220 msgr2=0x558875586c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.814+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1380238489 shutdown_connections 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.814+0000 7f3f56e30b80 1 -- 172.21.15.46:0/1380238489 wait complete. 2024-03-18T20:42:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.814+0000 7f3f56e30b80 20 client.17619 trim_cache size 0 max 16384 2024-03-18T20:42:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.828+0000 7f3f56e30b80 1 Processor -- start 2024-03-18T20:42:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.828+0000 7f3f56e30b80 1 -- start start 2024-03-18T20:42:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.828+0000 7f3f56e30b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558aa60 0x558875598610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.828+0000 7f3f56e30b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558b380 0x55887559cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.828+0000 7f3f56e30b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 0x558875586b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.828+0000 7f3f56e30b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558875588430 con 0x558875596220 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.828+0000 7f3f56e30b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887559d270 con 0x55887558aa60 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.828+0000 7f3f56e30b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887557e610 con 0x55887558b380 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.828+0000 7f3f49238700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 0x558875586b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f48a37700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558aa60 0x558875598610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f48a37700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558aa60 0x558875598610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:38544/0 (socket says 172.21.15.46:38544) 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f48a37700 1 -- 172.21.15.46:0/2977234674 learned_addr learned my addr 172.21.15.46:0/2977234674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f49238700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 0x558875586b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36636/0 (socket says 172.21.15.46:36636) 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f43fff700 1 --2- 172.21.15.46:0/2977234674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558b380 0x55887559cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f48a37700 1 -- 172.21.15.46:0/2977234674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558b380 msgr2=0x55887559cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f48a37700 1 --2- 172.21.15.46:0/2977234674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558b380 0x55887559cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f48a37700 1 -- 172.21.15.46:0/2977234674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 msgr2=0x558875586b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f48a37700 1 --2- 172.21.15.46:0/2977234674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 0x558875586b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f48a37700 1 -- 172.21.15.46:0/2977234674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55887530bb60 con 0x55887558aa60 2024-03-18T20:42:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f43fff700 1 --2- 172.21.15.46:0/2977234674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558b380 0x55887559cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f49238700 1 --2- 172.21.15.46:0/2977234674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 0x558875586b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f48a37700 1 --2- 172.21.15.46:0/2977234674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558aa60 0x558875598610 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400b690 tx=0x7f3f340057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5b6621b5af03ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f3b7fe700 1 -- 172.21.15.46:0/2977234674 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3402d040 con 0x55887558aa60 2024-03-18T20:42:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f3b7fe700 1 -- 172.21.15.46:0/2977234674 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55887557ead0 con 0x55887558aa60 2024-03-18T20:42:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.829+0000 7f3f3b7fe700 1 -- 172.21.15.46:0/2977234674 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f34024c10 con 0x55887558aa60 2024-03-18T20:42:00.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.830+0000 7f3f56e30b80 1 -- 172.21.15.46:0/2977234674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558aa60 msgr2=0x558875598610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.830+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/2977234674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558aa60 0x558875598610 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400b690 tx=0x7f3f340057c0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.830+0000 7f3f56e30b80 1 -- 172.21.15.46:0/2977234674 shutdown_connections 2024-03-18T20:42:00.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.830+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/2977234674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558875596220 0x558875586b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.830+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/2977234674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558aa60 0x558875598610 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.830+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/2977234674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55887558b380 0x55887559cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.830+0000 7f3f56e30b80 1 -- 172.21.15.46:0/2977234674 >> 172.21.15.46:0/2977234674 conn(0x558875501220 msgr2=0x5588755958b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.830+0000 7f3f56e30b80 1 -- 172.21.15.46:0/2977234674 shutdown_connections 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.830+0000 7f3f56e30b80 1 -- 172.21.15.46:0/2977234674 wait complete. 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f56e30b80 1 -- start start 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f56e30b80 1 Processor -- start 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f56e30b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 0x55887559ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f56e30b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558b380 0x55887559b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f56e30b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558875596220 0x55887559b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f56e30b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588755f8ca0 con 0x55887558aa60 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f56e30b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588755f8de0 con 0x55887558b380 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f56e30b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588755f8fa0 con 0x558875596220 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f56e30b80 10 client.0 _ll_register_callbacks cb 0x7f3f3400f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f48a37700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 0x55887559ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f48a37700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 0x55887559ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36644/0 (socket says 172.21.15.46:36644) 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f48a37700 1 -- 172.21.15.46:0/3263144804 learned_addr learned my addr 172.21.15.46:0/3263144804 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f49238700 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558875596220 0x55887559b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.831+0000 7f3f43fff700 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558b380 0x55887559b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f48a37700 1 -- 172.21.15.46:0/3263144804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558875596220 msgr2=0x55887559b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f48a37700 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558875596220 0x55887559b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f48a37700 1 -- 172.21.15.46:0/3263144804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558b380 msgr2=0x55887559b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f48a37700 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558b380 0x55887559b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f48a37700 1 -- 172.21.15.46:0/3263144804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55887535f3b0 con 0x55887558aa60 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f43fff700 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558b380 0x55887559b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f48a37700 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 0x55887559ad10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3f340095c0 tx=0x7f3f34026280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db327111ac745689 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f23fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f23fff700 1 -- 172.21.15.46:0/3263144804 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3402d070 con 0x55887558aa60 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f23fff700 1 -- 172.21.15.46:0/3263144804 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f3f340245b0 con 0x55887558aa60 2024-03-18T20:42:00.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f56e30b80 1 -- 172.21.15.46:0/3263144804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f3c01b320 con 0x55887558aa60 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f23fff700 1 -- 172.21.15.46:0/3263144804 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f340223a0 con 0x55887558aa60 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f56e30b80 10 client.17625 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.832+0000 7f3f56e30b80 1 -- 172.21.15.46:0/3263144804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f3f3c084040 con 0x55887558aa60 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.833+0000 7f3f23fff700 1 -- 172.21.15.46:0/3263144804 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f34052020 con 0x55887558aa60 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.833+0000 7f3f23fff700 1 -- 172.21.15.46:0/3263144804 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3f34026a50 con 0x55887558aa60 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.833+0000 7f3f56e30b80 10 client.17625 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.833+0000 7f3f56e30b80 10 client.17625 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.833+0000 7f3f56e30b80 20 client.17625 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.833+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5588755fa290 0x5588755fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.833+0000 7f3f56e30b80 4 client.17625 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.833+0000 7f3f56e30b80 1 -- 172.21.15.46:0/3263144804 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55887532ccf0 con 0x5588755fa290 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.833+0000 7f3f43fff700 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5588755fa290 0x5588755fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.833+0000 7f3f23fff700 10 client.17625 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.833+0000 7f3f43fff700 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5588755fa290 0x5588755fc680 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4915d32f4b8bfbd server_cookie=270ef05fef56de4 in_seq=0 out_seq=0 2024-03-18T20:42:00.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f23fff700 1 -- 172.21.15.46:0/3263144804 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55887532ccf0 con 0x5588755fa290 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f23fff700 10 client.17625 handle_command_reply: tid=0 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f56e30b80 1 client.17625 shutdown 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f56e30b80 1 -- 172.21.15.46:0/3263144804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 msgr2=0x55887559ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 0x55887559ad10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3f340095c0 tx=0x7f3f34026280 comp rx=0 tx=0).stop 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f56e30b80 1 -- 172.21.15.46:0/3263144804 shutdown_connections 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x5588755fa290 0x5588755fc680 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55887558aa60 0x55887559ad10 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558875596220 0x55887559b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f23fff700 0 client.17625 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f56e30b80 1 --2- 172.21.15.46:0/3263144804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55887558b380 0x55887559b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f56e30b80 1 -- 172.21.15.46:0/3263144804 >> 172.21.15.46:0/3263144804 conn(0x558875501220 msgr2=0x558875586aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f56e30b80 1 -- 172.21.15.46:0/3263144804 shutdown_connections 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f56e30b80 1 -- 172.21.15.46:0/3263144804 wait complete. 2024-03-18T20:42:00.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.834+0000 7f3f56e30b80 20 client.17625 trim_cache size 0 max 16384 2024-03-18T20:42:00.843 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:42:00.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.835+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/3102872208 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f300a1290 msgr2=0x7f3f300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.835+0000 7f3f4ac9b700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f300a1290 0x7f3f300a3750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400b2e0 tx=0x7f3f34005fd0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.835+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/3102872208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f4419bb20 msgr2=0x7f3f4419bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.835+0000 7f3f4ac9b700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f4419bb20 0x7f3f4419bfa0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00ef30 tx=0x7f3f3c00c3a0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.835+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/3102872208 shutdown_connections 2024-03-18T20:42:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.835+0000 7f3f4ac9b700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f300a1290 0x7f3f300a3750 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.835+0000 7f3f4ac9b700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f4419bb20 0x7f3f4419bfa0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.835+0000 7f3f4ac9b700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f440ffe50 0x7f3f44197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.835+0000 7f3f4ac9b700 1 --2- 172.21.15.46:0/3102872208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f44104f60 0x7f3f44197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.835+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/3102872208 >> 172.21.15.46:0/3102872208 conn(0x7f3f44076d40 msgr2=0x7f3f440fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.835+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/3102872208 shutdown_connections 2024-03-18T20:42:00.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:00.835+0000 7f3f4ac9b700 1 -- 172.21.15.46:0/3102872208 wait complete. 2024-03-18T20:42:00.852 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:42:00.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:42:01.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.040+0000 7f344ab4f700 1 -- 172.21.15.46:0/930837139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3444101900 msgr2=0x7f3444105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:01.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.040+0000 7f344ab4f700 1 --2- 172.21.15.46:0/930837139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3444101900 0x7f3444105940 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3430009990 tx=0x7f3430009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.040+0000 7f344ab4f700 1 -- 172.21.15.46:0/930837139 shutdown_connections 2024-03-18T20:42:01.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.040+0000 7f344ab4f700 1 --2- 172.21.15.46:0/930837139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3444106070 0x7f344410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.040+0000 7f344ab4f700 1 --2- 172.21.15.46:0/930837139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444100f50 0x7f3444101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.040+0000 7f344ab4f700 1 --2- 172.21.15.46:0/930837139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3444101900 0x7f3444105940 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.040+0000 7f344ab4f700 1 -- 172.21.15.46:0/930837139 >> 172.21.15.46:0/930837139 conn(0x7f34440fc7f0 msgr2=0x7f34440fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.040+0000 7f344ab4f700 1 -- 172.21.15.46:0/930837139 shutdown_connections 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.040+0000 7f344ab4f700 1 -- 172.21.15.46:0/930837139 wait complete. 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f344ab4f700 1 Processor -- start 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f344ab4f700 1 -- start start 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f344ab4f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3444100f50 0x7f34440730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f344ab4f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3444106070 0x7f34440735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f344ab4f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444074d20 0x7f3444073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f344ab4f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34441a9e70 con 0x7f3444106070 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f344ab4f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34441a9fe0 con 0x7f3444074d20 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f344ab4f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34441aa1a0 con 0x7f3444100f50 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f3443fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3444106070 0x7f34440735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f3443fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3444106070 0x7f34440735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36658/0 (socket says 172.21.15.46:36658) 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f3443fff700 1 -- 172.21.15.46:0/3370674471 learned_addr learned my addr 172.21.15.46:0/3370674471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f34488eb700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3444100f50 0x7f34440730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f34490ec700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444074d20 0x7f3444073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f3443fff700 1 -- 172.21.15.46:0/3370674471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3444100f50 msgr2=0x7f34440730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f3443fff700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3444100f50 0x7f34440730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f3443fff700 1 -- 172.21.15.46:0/3370674471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444074d20 msgr2=0x7f3444073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.041+0000 7f3443fff700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444074d20 0x7f3444073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.042+0000 7f3443fff700 1 -- 172.21.15.46:0/3370674471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3438009520 con 0x7f3444106070 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.042+0000 7f34490ec700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444074d20 0x7f3444073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.042+0000 7f3443fff700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3444106070 0x7f34440735f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3430000c00 tx=0x7f3430005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.042+0000 7f3441ffb700 1 -- 172.21.15.46:0/3370674471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343001d070 con 0x7f3444106070 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.042+0000 7f3441ffb700 1 -- 172.21.15.46:0/3370674471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3430022470 con 0x7f3444106070 2024-03-18T20:42:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.042+0000 7f344ab4f700 1 -- 172.21.15.46:0/3370674471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34300095f0 con 0x7f3444106070 2024-03-18T20:42:01.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.042+0000 7f3441ffb700 1 -- 172.21.15.46:0/3370674471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430017650 con 0x7f3444106070 2024-03-18T20:42:01.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.042+0000 7f344ab4f700 1 -- 172.21.15.46:0/3370674471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34441aa6b0 con 0x7f3444106070 2024-03-18T20:42:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.045+0000 7f3441ffb700 1 -- 172.21.15.46:0/3370674471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f343000fad0 con 0x7f3444106070 2024-03-18T20:42:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.045+0000 7f3441ffb700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f34340a1290 0x7f34340a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.046+0000 7f34488eb700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f34340a1290 0x7f34340a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.046+0000 7f3441ffb700 1 -- 172.21.15.46:0/3370674471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f34300cdfe0 con 0x7f3444106070 2024-03-18T20:42:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.046+0000 7f34488eb700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f34340a1290 0x7f34340a3750 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3438009df0 tx=0x7f3438009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.046+0000 7f3456ce4b80 1 Processor -- start 2024-03-18T20:42:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.046+0000 7f3456ce4b80 1 -- start start 2024-03-18T20:42:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f3456ce4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e26902be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f3456ce4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902c380 0x55e269036ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f3456ce4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e269039610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f3456ce4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268f85990 con 0x55e26902c380 2024-03-18T20:42:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f3456ce4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268f83350 con 0x55e269037220 2024-03-18T20:42:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f3456ce4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268f7ec40 con 0x55e26902ba60 2024-03-18T20:42:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f3443fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902c380 0x55e269036ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f3443fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902c380 0x55e269036ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36672/0 (socket says 172.21.15.46:36672) 2024-03-18T20:42:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f34490ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e269039610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f3443fff700 1 -- 172.21.15.46:0/1135471498 learned_addr learned my addr 172.21.15.46:0/1135471498 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f34488eb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e26902be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f34490ec700 1 -- 172.21.15.46:0/1135471498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 msgr2=0x55e26902be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f34490ec700 1 --2- 172.21.15.46:0/1135471498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e26902be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f34490ec700 1 -- 172.21.15.46:0/1135471498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902c380 msgr2=0x55e269036ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f34490ec700 1 --2- 172.21.15.46:0/1135471498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902c380 0x55e269036ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f34490ec700 1 -- 172.21.15.46:0/1135471498 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e268e003b0 con 0x55e269037220 2024-03-18T20:42:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f3443fff700 1 --2- 172.21.15.46:0/1135471498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902c380 0x55e269036ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.047+0000 7f34490ec700 1 --2- 172.21.15.46:0/1135471498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e269039610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f343c003db0 tx=0x7f343c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=286bd4b35d03c180 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.048+0000 7f342effd700 1 -- 172.21.15.46:0/1135471498 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c004bb0 con 0x55e269037220 2024-03-18T20:42:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.048+0000 7f342effd700 1 -- 172.21.15.46:0/1135471498 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f343c00f040 con 0x55e269037220 2024-03-18T20:42:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.048+0000 7f342effd700 1 -- 172.21.15.46:0/1135471498 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c007cb0 con 0x55e269037220 2024-03-18T20:42:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.048+0000 7f3456ce4b80 1 -- 172.21.15.46:0/1135471498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 msgr2=0x55e269039610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:01.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.048+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/1135471498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e269039610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f343c003db0 tx=0x7f343c009920 comp rx=0 tx=0).stop 2024-03-18T20:42:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.049+0000 7f3456ce4b80 1 -- 172.21.15.46:0/1135471498 shutdown_connections 2024-03-18T20:42:01.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.049+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/1135471498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e269039610 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.049+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/1135471498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e26902be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.049+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/1135471498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902c380 0x55e269036ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.049+0000 7f3456ce4b80 1 -- 172.21.15.46:0/1135471498 >> 172.21.15.46:0/1135471498 conn(0x55e268fa2220 msgr2=0x55e269028020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.049+0000 7f3456ce4b80 1 -- 172.21.15.46:0/1135471498 shutdown_connections 2024-03-18T20:42:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.049+0000 7f3456ce4b80 1 -- 172.21.15.46:0/1135471498 wait complete. 2024-03-18T20:42:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.049+0000 7f3456ce4b80 1 -- start start 2024-03-18T20:42:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.049+0000 7f3456ce4b80 1 Processor -- start 2024-03-18T20:42:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f3456ce4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e269036cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f3456ce4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902c380 0x55e269033fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f3456ce4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e269034700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f3456ce4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268f1ad80 con 0x55e26902c380 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f3456ce4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268f1aa20 con 0x55e269037220 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f3456ce4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268e026a0 con 0x55e26902ba60 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f3456ce4b80 10 client.0 _ll_register_callbacks cb 0x55e268d863a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f34488eb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e269036cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f34490ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e269034700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f34488eb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e269036cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:53348/0 (socket says 172.21.15.46:53348) 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f34488eb700 1 -- 172.21.15.46:0/620878611 learned_addr learned my addr 172.21.15.46:0/620878611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f3443fff700 1 --2- 172.21.15.46:0/620878611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902c380 0x55e269033fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f34490ec700 1 -- 172.21.15.46:0/620878611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 msgr2=0x55e269036cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f34490ec700 1 --2- 172.21.15.46:0/620878611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e269036cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f34490ec700 1 -- 172.21.15.46:0/620878611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902c380 msgr2=0x55e269033fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f34490ec700 1 --2- 172.21.15.46:0/620878611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902c380 0x55e269033fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.050+0000 7f34490ec700 1 -- 172.21.15.46:0/620878611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34441aa6b0 con 0x55e269037220 2024-03-18T20:42:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.051+0000 7f34490ec700 1 --2- 172.21.15.46:0/620878611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e269034700 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f343c004600 tx=0x7f343c004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dec2591bd7d5505 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.051+0000 7f3423fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:42:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.051+0000 7f3423fff700 1 -- 172.21.15.46:0/620878611 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c007710 con 0x55e269037220 2024-03-18T20:42:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.051+0000 7f3423fff700 1 -- 172.21.15.46:0/620878611 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f343c007870 con 0x55e269037220 2024-03-18T20:42:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.051+0000 7f3456ce4b80 1 -- 172.21.15.46:0/620878611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e26901fad0 con 0x55e269037220 2024-03-18T20:42:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.051+0000 7f3423fff700 1 -- 172.21.15.46:0/620878611 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f343c007a50 con 0x55e269037220 2024-03-18T20:42:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.051+0000 7f3456ce4b80 10 client.27320 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:01.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.051+0000 7f3456ce4b80 1 -- 172.21.15.46:0/620878611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f343c003a60 con 0x55e269037220 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.051+0000 7f3423fff700 1 -- 172.21.15.46:0/620878611 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e26901fad0 con 0x55e269037220 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.051+0000 7f3423fff700 1 -- 172.21.15.46:0/620878611 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f343c02e070 con 0x55e269037220 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.052+0000 7f3456ce4b80 10 client.27320 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.052+0000 7f3456ce4b80 10 client.27320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.052+0000 7f3456ce4b80 20 client.27320 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.052+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/620878611 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e269057e90 0x55e26905a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.052+0000 7f3456ce4b80 4 client.27320 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.052+0000 7f3456ce4b80 1 -- 172.21.15.46:0/620878611 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e268f77700 con 0x55e269057e90 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.052+0000 7f34488eb700 1 --2- 172.21.15.46:0/620878611 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e269057e90 0x55e26905a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.053+0000 7f3423fff700 10 client.27320 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.053+0000 7f34488eb700 1 --2- 172.21.15.46:0/620878611 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e269057e90 0x55e26905a2b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd167b04a975ed98 server_cookie=b41b50868a1ce050 in_seq=0 out_seq=0 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.053+0000 7f3423fff700 1 -- 172.21.15.46:0/620878611 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e268f77700 con 0x55e269057e90 2024-03-18T20:42:01.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.053+0000 7f3423fff700 10 client.27320 handle_command_reply: tid=0 2024-03-18T20:42:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.053+0000 7f3456ce4b80 1 client.27320 shutdown 2024-03-18T20:42:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.054+0000 7f3456ce4b80 1 -- 172.21.15.46:0/620878611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 msgr2=0x55e269034700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.054+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/620878611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e269034700 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f343c004600 tx=0x7f343c004d00 comp rx=0 tx=0).stop 2024-03-18T20:42:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.054+0000 7f3456ce4b80 1 -- 172.21.15.46:0/620878611 shutdown_connections 2024-03-18T20:42:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.054+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/620878611 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e269057e90 0x55e26905a2b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.054+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/620878611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e269034700 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.054+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/620878611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e269036cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.054+0000 7f3423fff700 0 client.27320 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.054+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/620878611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902c380 0x55e269033fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.054+0000 7f3456ce4b80 1 -- 172.21.15.46:0/620878611 >> 172.21.15.46:0/620878611 conn(0x55e268fa2220 msgr2=0x55e269027c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.054+0000 7f3456ce4b80 1 -- 172.21.15.46:0/620878611 shutdown_connections 2024-03-18T20:42:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.055+0000 7f3456ce4b80 1 -- 172.21.15.46:0/620878611 wait complete. 2024-03-18T20:42:01.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.055+0000 7f3456ce4b80 20 client.27320 trim_cache size 0 max 16384 2024-03-18T20:42:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f3456ce4b80 1 Processor -- start 2024-03-18T20:42:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f3456ce4b80 1 -- start start 2024-03-18T20:42:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f3456ce4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902ba60 0x55e269033f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f3456ce4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902c380 0x55e2690338a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f3456ce4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e2690425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f3456ce4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268ff2940 con 0x55e26902ba60 2024-03-18T20:42:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f3456ce4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2690555b0 con 0x55e269037220 2024-03-18T20:42:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f3456ce4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269033de0 con 0x55e26902c380 2024-03-18T20:42:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f34490ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e2690425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f34488eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902ba60 0x55e269033f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f3443fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902c380 0x55e2690338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f3443fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902c380 0x55e2690338a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:53358/0 (socket says 172.21.15.46:53358) 2024-03-18T20:42:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f34490ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e2690425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:38602/0 (socket says 172.21.15.46:38602) 2024-03-18T20:42:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.071+0000 7f3443fff700 1 -- 172.21.15.46:0/1915627434 learned_addr learned my addr 172.21.15.46:0/1915627434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.072+0000 7f3443fff700 1 -- 172.21.15.46:0/1915627434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 msgr2=0x55e2690425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.072+0000 7f3443fff700 1 --2- 172.21.15.46:0/1915627434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e2690425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.072+0000 7f3443fff700 1 -- 172.21.15.46:0/1915627434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902ba60 msgr2=0x55e269033f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:01.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.072+0000 7f3443fff700 1 --2- 172.21.15.46:0/1915627434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902ba60 0x55e269033f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.072+0000 7f3443fff700 1 -- 172.21.15.46:0/1915627434 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e268dacb60 con 0x55e26902c380 2024-03-18T20:42:01.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.072+0000 7f3443fff700 1 --2- 172.21.15.46:0/1915627434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902c380 0x55e2690338a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f343009af70 tx=0x7f34300a1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0f4da13d261b66d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:01.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.072+0000 7f342f7fe700 1 -- 172.21.15.46:0/1915627434 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300b1070 con 0x55e26902c380 2024-03-18T20:42:01.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.072+0000 7f342f7fe700 1 -- 172.21.15.46:0/1915627434 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f343009adb0 con 0x55e26902c380 2024-03-18T20:42:01.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.072+0000 7f342f7fe700 1 -- 172.21.15.46:0/1915627434 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343009f0a0 con 0x55e26902c380 2024-03-18T20:42:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.073+0000 7f3456ce4b80 1 -- 172.21.15.46:0/1915627434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902c380 msgr2=0x55e2690338a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.073+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/1915627434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902c380 0x55e2690338a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f343009af70 tx=0x7f34300a1040 comp rx=0 tx=0).stop 2024-03-18T20:42:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.073+0000 7f3456ce4b80 1 -- 172.21.15.46:0/1915627434 shutdown_connections 2024-03-18T20:42:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.073+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/1915627434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e26902ba60 0x55e269033f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.073+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/1915627434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e269037220 0x55e2690425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.073+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/1915627434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902c380 0x55e2690338a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.073+0000 7f3456ce4b80 1 -- 172.21.15.46:0/1915627434 >> 172.21.15.46:0/1915627434 conn(0x55e268fa2220 msgr2=0x55e269023a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.073+0000 7f3456ce4b80 1 -- 172.21.15.46:0/1915627434 shutdown_connections 2024-03-18T20:42:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.073+0000 7f3456ce4b80 1 -- 172.21.15.46:0/1915627434 wait complete. 2024-03-18T20:42:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.073+0000 7f3456ce4b80 1 -- start start 2024-03-18T20:42:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.074+0000 7f3456ce4b80 1 Processor -- start 2024-03-18T20:42:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.074+0000 7f3456ce4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e26903b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.074+0000 7f3456ce4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e26902c380 0x55e26903bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.074+0000 7f3456ce4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e269037220 0x55e2690aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.074+0000 7f3456ce4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26903c820 con 0x55e269037220 2024-03-18T20:42:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.074+0000 7f3456ce4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2690ab1c0 con 0x55e26902c380 2024-03-18T20:42:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.074+0000 7f3456ce4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2690ab300 con 0x55e26902ba60 2024-03-18T20:42:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.074+0000 7f3456ce4b80 10 client.0 _ll_register_callbacks cb 0x7f3430005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.074+0000 7f3443fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e26902c380 0x55e26903bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.074+0000 7f34488eb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e26903b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.074+0000 7f3443fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e26902c380 0x55e26903bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:38612/0 (socket says 172.21.15.46:38612) 2024-03-18T20:42:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.074+0000 7f3443fff700 1 -- 172.21.15.46:0/2065888085 learned_addr learned my addr 172.21.15.46:0/2065888085 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f34490ec700 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e269037220 0x55e2690aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3443fff700 1 -- 172.21.15.46:0/2065888085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 msgr2=0x55e26903b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3443fff700 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e26903b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3443fff700 1 -- 172.21.15.46:0/2065888085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e269037220 msgr2=0x55e2690aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3443fff700 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e269037220 0x55e2690aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3443fff700 1 -- 172.21.15.46:0/2065888085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34300095f0 con 0x55e26902c380 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f34490ec700 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e269037220 0x55e2690aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3443fff700 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e26902c380 0x55e26903bf10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3430097ce0 tx=0x7f343000b3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52cd93a5a1fbb9df server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3423fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3423fff700 1 -- 172.21.15.46:0/2065888085 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343007e070 con 0x55e26902c380 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3423fff700 1 -- 172.21.15.46:0/2065888085 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f343009b970 con 0x55e26902c380 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3456ce4b80 1 -- 172.21.15.46:0/2065888085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f343c00f040 con 0x55e26902c380 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3423fff700 1 -- 172.21.15.46:0/2065888085 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34300b1040 con 0x55e26902c380 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3456ce4b80 10 client.27326 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3456ce4b80 1 -- 172.21.15.46:0/2065888085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55e268f44ea0 con 0x55e26902c380 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.075+0000 7f3423fff700 1 -- 172.21.15.46:0/2065888085 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34300a7020 con 0x55e26902c380 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.076+0000 7f3423fff700 1 -- 172.21.15.46:0/2065888085 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3430081030 con 0x55e26902c380 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.076+0000 7f3456ce4b80 10 client.27326 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.076+0000 7f3456ce4b80 10 client.27326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.076+0000 7f3456ce4b80 20 client.27326 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.076+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e2690ac680 0x55e2690aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.076+0000 7f3456ce4b80 4 client.27326 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.076+0000 7f3456ce4b80 1 -- 172.21.15.46:0/2065888085 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e268dcdcf0 con 0x55e2690ac680 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.076+0000 7f34488eb700 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e2690ac680 0x55e2690aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.076+0000 7f3423fff700 10 client.27326 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.076+0000 7f34488eb700 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e2690ac680 0x55e2690aeaa0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b061b2e8fea1d1 server_cookie=505cc210321fa1a7 in_seq=0 out_seq=0 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3423fff700 1 -- 172.21.15.46:0/2065888085 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e268dcdcf0 con 0x55e2690ac680 2024-03-18T20:42:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3423fff700 10 client.27326 handle_command_reply: tid=0 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3456ce4b80 1 client.27326 shutdown 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3456ce4b80 1 -- 172.21.15.46:0/2065888085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e26902c380 msgr2=0x55e26903bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e26902c380 0x55e26903bf10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3430097ce0 tx=0x7f343000b3a0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3456ce4b80 1 -- 172.21.15.46:0/2065888085 shutdown_connections 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e2690ac680 0x55e2690aeaa0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e269037220 0x55e2690aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e26902ba60 0x55e26903b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3456ce4b80 1 --2- 172.21.15.46:0/2065888085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e26902c380 0x55e26903bf10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3423fff700 0 client.27326 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3456ce4b80 1 -- 172.21.15.46:0/2065888085 >> 172.21.15.46:0/2065888085 conn(0x55e268fa2220 msgr2=0x55e2690425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3456ce4b80 1 -- 172.21.15.46:0/2065888085 shutdown_connections 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3456ce4b80 1 -- 172.21.15.46:0/2065888085 wait complete. 2024-03-18T20:42:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.077+0000 7f3456ce4b80 20 client.27326 trim_cache size 0 max 16384 2024-03-18T20:42:01.087 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:42:01.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.078+0000 7f344ab4f700 1 -- 172.21.15.46:0/3370674471 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f34340a1290 msgr2=0x7f34340a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.078+0000 7f344ab4f700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f34340a1290 0x7f34340a3750 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3438009df0 tx=0x7f3438009260 comp rx=0 tx=0).stop 2024-03-18T20:42:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.078+0000 7f344ab4f700 1 -- 172.21.15.46:0/3370674471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3444106070 msgr2=0x7f34440735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.078+0000 7f344ab4f700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3444106070 0x7f34440735f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3430000c00 tx=0x7f3430005740 comp rx=0 tx=0).stop 2024-03-18T20:42:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.078+0000 7f344ab4f700 1 -- 172.21.15.46:0/3370674471 shutdown_connections 2024-03-18T20:42:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.078+0000 7f344ab4f700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f34340a1290 0x7f34340a3750 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.078+0000 7f344ab4f700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3444074d20 0x7f3444073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.078+0000 7f344ab4f700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3444100f50 0x7f34440730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.078+0000 7f344ab4f700 1 --2- 172.21.15.46:0/3370674471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3444106070 0x7f34440735f0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.078+0000 7f344ab4f700 1 -- 172.21.15.46:0/3370674471 >> 172.21.15.46:0/3370674471 conn(0x7f34440fc7f0 msgr2=0x7f34440fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.078+0000 7f344ab4f700 1 -- 172.21.15.46:0/3370674471 shutdown_connections 2024-03-18T20:42:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:01.078+0000 7f344ab4f700 1 -- 172.21.15.46:0/3370674471 wait complete. 2024-03-18T20:42:01.694 INFO:tasks.workunit.client.0.smithi046.stdout: 1 191860 3.37 MB/sec execute 176 sec latency 27.904 ms 2024-03-18T20:42:02.095 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:42:02.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:42:02.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:42:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.253+0000 7f27cbda3700 1 -- 172.21.15.46:0/2030422847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c4100800 msgr2=0x7f27c4104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.253+0000 7f27cbda3700 1 --2- 172.21.15.46:0/2030422847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c4100800 0x7f27c4104830 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009990 tx=0x7f27c0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.253+0000 7f27cbda3700 1 -- 172.21.15.46:0/2030422847 shutdown_connections 2024-03-18T20:42:02.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.253+0000 7f27cbda3700 1 --2- 172.21.15.46:0/2030422847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27c4104f60 0x7f27c410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.253+0000 7f27cbda3700 1 --2- 172.21.15.46:0/2030422847 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c40ffe50 0x7f27c4100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.253+0000 7f27cbda3700 1 --2- 172.21.15.46:0/2030422847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c4100800 0x7f27c4104830 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.253+0000 7f27cbda3700 1 -- 172.21.15.46:0/2030422847 >> 172.21.15.46:0/2030422847 conn(0x7f27c4076d40 msgr2=0x7f27c4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:02.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.253+0000 7f27cbda3700 1 -- 172.21.15.46:0/2030422847 shutdown_connections 2024-03-18T20:42:02.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.253+0000 7f27cbda3700 1 -- 172.21.15.46:0/2030422847 wait complete. 2024-03-18T20:42:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27cbda3700 1 Processor -- start 2024-03-18T20:42:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27cbda3700 1 -- start start 2024-03-18T20:42:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27cbda3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c40ffe50 0x7f27c41018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27cbda3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c4104f60 0x7f27c4101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27cbda3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27c4102350 0x7f27c41a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27cbda3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c41a59a0 con 0x7f27c4104f60 2024-03-18T20:42:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27cbda3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c41a5b10 con 0x7f27c4102350 2024-03-18T20:42:02.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27cbda3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c41a5cd0 con 0x7f27c40ffe50 2024-03-18T20:42:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27c9b3f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c40ffe50 0x7f27c41018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27c933e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c4104f60 0x7f27c4101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27c9b3f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c40ffe50 0x7f27c41018d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:53386/0 (socket says 172.21.15.46:53386) 2024-03-18T20:42:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27c9b3f700 1 -- 172.21.15.46:0/2122532063 learned_addr learned my addr 172.21.15.46:0/2122532063 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.254+0000 7f27ca340700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27c4102350 0x7f27c41a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.255+0000 7f27c9b3f700 1 -- 172.21.15.46:0/2122532063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27c4102350 msgr2=0x7f27c41a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.255+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27c4102350 0x7f27c41a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.255+0000 7f27c9b3f700 1 -- 172.21.15.46:0/2122532063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c4104f60 msgr2=0x7f27c4101e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.255+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c4104f60 0x7f27c4101e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.255+0000 7f27c9b3f700 1 -- 172.21.15.46:0/2122532063 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c00095f0 con 0x7f27c40ffe50 2024-03-18T20:42:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.255+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c40ffe50 0x7f27c41018d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f27b800e9a0 tx=0x7f27b800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:02.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.255+0000 7f27b6ffd700 1 -- 172.21.15.46:0/2122532063 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8009790 con 0x7f27c40ffe50 2024-03-18T20:42:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.255+0000 7f27cbda3700 1 -- 172.21.15.46:0/2122532063 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c41a5fb0 con 0x7f27c40ffe50 2024-03-18T20:42:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.255+0000 7f27b6ffd700 1 -- 172.21.15.46:0/2122532063 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27b800cbd0 con 0x7f27c40ffe50 2024-03-18T20:42:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.255+0000 7f27b6ffd700 1 -- 172.21.15.46:0/2122532063 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8018680 con 0x7f27c40ffe50 2024-03-18T20:42:02.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.255+0000 7f27cbda3700 1 -- 172.21.15.46:0/2122532063 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c41a63e0 con 0x7f27c40ffe50 2024-03-18T20:42:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.258+0000 7f27b6ffd700 1 -- 172.21.15.46:0/2122532063 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f27b800cd40 con 0x7f27c40ffe50 2024-03-18T20:42:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.259+0000 7f27b6ffd700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f27b00a1310 0x7f27b00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.259+0000 7f27b6ffd700 1 -- 172.21.15.46:0/2122532063 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f27b8014070 con 0x7f27c40ffe50 2024-03-18T20:42:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.259+0000 7f27c933e700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f27b00a1310 0x7f27b00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.259+0000 7f27c933e700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f27b00a1310 0x7f27b00a37d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f27c000b2e0 tx=0x7f27c0005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27d7f38b80 1 Processor -- start 2024-03-18T20:42:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27d7f38b80 1 -- start start 2024-03-18T20:42:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27d7f38b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b85fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27d7f38b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 0x55948b86ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27d7f38b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b86b2a0 0x55948b86d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27d7f38b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948b7b9a10 con 0x55948b860400 2024-03-18T20:42:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27d7f38b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948b7b73d0 con 0x55948b85fae0 2024-03-18T20:42:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27d7f38b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948b7b2cc0 con 0x55948b86b2a0 2024-03-18T20:42:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27c933e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 0x55948b86ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27ca340700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b86b2a0 0x55948b86d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27c9b3f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b85fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27c933e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 0x55948b86ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36712/0 (socket says 172.21.15.46:36712) 2024-03-18T20:42:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.260+0000 7f27c933e700 1 -- 172.21.15.46:0/1192099866 learned_addr learned my addr 172.21.15.46:0/1192099866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.261+0000 7f27ca340700 1 -- 172.21.15.46:0/1192099866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 msgr2=0x55948b85fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.261+0000 7f27ca340700 1 --2- 172.21.15.46:0/1192099866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b85fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.261+0000 7f27ca340700 1 -- 172.21.15.46:0/1192099866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 msgr2=0x55948b86ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.261+0000 7f27ca340700 1 --2- 172.21.15.46:0/1192099866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 0x55948b86ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.261+0000 7f27ca340700 1 -- 172.21.15.46:0/1192099866 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55948b634430 con 0x55948b86b2a0 2024-03-18T20:42:02.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.261+0000 7f27c933e700 1 --2- 172.21.15.46:0/1192099866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 0x55948b86ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:02.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.261+0000 7f27ca340700 1 --2- 172.21.15.46:0/1192099866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b86b2a0 0x55948b86d690 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009310 tx=0x7f27bc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7361e134f08af6eb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:02.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.261+0000 7f27affff700 1 -- 172.21.15.46:0/1192099866 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc00e070 con 0x55948b86b2a0 2024-03-18T20:42:02.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.261+0000 7f27affff700 1 -- 172.21.15.46:0/1192099866 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27bc00f040 con 0x55948b86b2a0 2024-03-18T20:42:02.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.261+0000 7f27affff700 1 -- 172.21.15.46:0/1192099866 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc014840 con 0x55948b86b2a0 2024-03-18T20:42:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.262+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1192099866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b86b2a0 msgr2=0x55948b86d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.262+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/1192099866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b86b2a0 0x55948b86d690 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009310 tx=0x7f27bc009920 comp rx=0 tx=0).stop 2024-03-18T20:42:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.262+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1192099866 shutdown_connections 2024-03-18T20:42:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.262+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/1192099866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b86b2a0 0x55948b86d690 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.262+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/1192099866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b85fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.262+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/1192099866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 0x55948b86ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.262+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1192099866 >> 172.21.15.46:0/1192099866 conn(0x55948b7d62a0 msgr2=0x55948b85c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.262+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1192099866 shutdown_connections 2024-03-18T20:42:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.262+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1192099866 wait complete. 2024-03-18T20:42:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.262+0000 7f27d7f38b80 1 -- start start 2024-03-18T20:42:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.263+0000 7f27d7f38b80 1 Processor -- start 2024-03-18T20:42:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.263+0000 7f27d7f38b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b86ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.263+0000 7f27d7f38b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 0x55948b867f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.263+0000 7f27d7f38b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b86b2a0 0x55948b868560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.263+0000 7f27d7f38b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948b74ee00 con 0x55948b860400 2024-03-18T20:42:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.263+0000 7f27d7f38b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948b74eaa0 con 0x55948b85fae0 2024-03-18T20:42:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.263+0000 7f27d7f38b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948b636720 con 0x55948b86b2a0 2024-03-18T20:42:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.263+0000 7f27d7f38b80 10 client.0 _ll_register_callbacks cb 0x55948b5ba420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.263+0000 7f27c933e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 0x55948b867f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.263+0000 7f27c933e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 0x55948b867f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36718/0 (socket says 172.21.15.46:36718) 2024-03-18T20:42:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27c933e700 1 -- 172.21.15.46:0/1726478088 learned_addr learned my addr 172.21.15.46:0/1726478088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27ca340700 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b86b2a0 0x55948b868560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b86ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27c933e700 1 -- 172.21.15.46:0/1726478088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b86b2a0 msgr2=0x55948b868560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27c933e700 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b86b2a0 0x55948b868560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27c933e700 1 -- 172.21.15.46:0/1726478088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 msgr2=0x55948b86ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27c933e700 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b86ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27c933e700 1 -- 172.21.15.46:0/1726478088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c410d350 con 0x55948b860400 2024-03-18T20:42:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27ca340700 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b86b2a0 0x55948b868560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27c933e700 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 0x55948b867f30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f27c0000c00 tx=0x7f27c0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38db9c89d0969b83 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27ad7fa700 1 -- 172.21.15.46:0/1726478088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c000beb0 con 0x55948b860400 2024-03-18T20:42:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27ad7fa700 1 -- 172.21.15.46:0/1726478088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f27c0004e80 con 0x55948b860400 2024-03-18T20:42:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1726478088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55948b853b50 con 0x55948b860400 2024-03-18T20:42:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.264+0000 7f27ad7fa700 1 -- 172.21.15.46:0/1726478088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c000f460 con 0x55948b860400 2024-03-18T20:42:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.265+0000 7f27d7f38b80 10 client.17655 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.265+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1726478088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f27bc009050 con 0x55948b860400 2024-03-18T20:42:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.265+0000 7f27ad7fa700 1 -- 172.21.15.46:0/1726478088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55948b853b50 con 0x55948b860400 2024-03-18T20:42:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.265+0000 7f27ad7fa700 1 -- 172.21.15.46:0/1726478088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f27c002f070 con 0x55948b860400 2024-03-18T20:42:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.265+0000 7f27d7f38b80 10 client.17655 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.265+0000 7f27d7f38b80 10 client.17655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.265+0000 7f27d7f38b80 20 client.17655 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.265+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55948b87afa0 0x55948b87d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.265+0000 7f27d7f38b80 4 client.17655 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.265+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1726478088 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55948b7ab780 con 0x55948b87afa0 2024-03-18T20:42:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.265+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55948b87afa0 0x55948b87d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.266+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55948b87afa0 0x55948b87d3c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b28fdac53e2268 server_cookie=689b7bef4d5ed3a1 in_seq=0 out_seq=0 2024-03-18T20:42:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.266+0000 7f27ad7fa700 10 client.17655 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27ad7fa700 1 -- 172.21.15.46:0/1726478088 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55948b7ab780 con 0x55948b87afa0 2024-03-18T20:42:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27ad7fa700 10 client.17655 handle_command_reply: tid=0 2024-03-18T20:42:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27d7f38b80 1 client.17655 shutdown 2024-03-18T20:42:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1726478088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 msgr2=0x55948b867f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 0x55948b867f30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f27c0000c00 tx=0x7f27c0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1726478088 shutdown_connections 2024-03-18T20:42:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55948b87afa0 0x55948b87d3c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b86b2a0 0x55948b868560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b86ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/1726478088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b860400 0x55948b867f30 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27ad7fa700 0 client.17655 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1726478088 >> 172.21.15.46:0/1726478088 conn(0x55948b7d62a0 msgr2=0x55948b85bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1726478088 shutdown_connections 2024-03-18T20:42:02.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27d7f38b80 1 -- 172.21.15.46:0/1726478088 wait complete. 2024-03-18T20:42:02.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.267+0000 7f27d7f38b80 20 client.17655 trim_cache size 0 max 16384 2024-03-18T20:42:02.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.286+0000 7f27d7f38b80 1 Processor -- start 2024-03-18T20:42:02.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.286+0000 7f27d7f38b80 1 -- start start 2024-03-18T20:42:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.286+0000 7f27d7f38b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b85fae0 0x55948b866800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.286+0000 7f27d7f38b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b860400 0x55948b866d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.286+0000 7f27d7f38b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b86b2a0 0x55948b876650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.286+0000 7f27d7f38b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948b8779e0 con 0x55948b85fae0 2024-03-18T20:42:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.286+0000 7f27d7f38b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948b860de0 con 0x55948b86b2a0 2024-03-18T20:42:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.286+0000 7f27d7f38b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948b8615d0 con 0x55948b860400 2024-03-18T20:42:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27c9b3f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b85fae0 0x55948b866800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27c9b3f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b85fae0 0x55948b866800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36724/0 (socket says 172.21.15.46:36724) 2024-03-18T20:42:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27c9b3f700 1 -- 172.21.15.46:0/789050829 learned_addr learned my addr 172.21.15.46:0/789050829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27c933e700 1 --2- 172.21.15.46:0/789050829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b860400 0x55948b866d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27ca340700 1 --2- 172.21.15.46:0/789050829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b86b2a0 0x55948b876650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27c9b3f700 1 -- 172.21.15.46:0/789050829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b860400 msgr2=0x55948b866d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/789050829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b860400 0x55948b866d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27c9b3f700 1 -- 172.21.15.46:0/789050829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b86b2a0 msgr2=0x55948b876650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/789050829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b86b2a0 0x55948b876650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27c9b3f700 1 -- 172.21.15.46:0/789050829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55948b881ff0 con 0x55948b85fae0 2024-03-18T20:42:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/789050829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b85fae0 0x55948b866800 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f27b80ca5e0 tx=0x7f27b8023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539083709b09ec55 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27b4ff9700 1 -- 172.21.15.46:0/789050829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b809dd70 con 0x55948b85fae0 2024-03-18T20:42:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27b4ff9700 1 -- 172.21.15.46:0/789050829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27b80a23c0 con 0x55948b85fae0 2024-03-18T20:42:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.287+0000 7f27b4ff9700 1 -- 172.21.15.46:0/789050829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b80a1f30 con 0x55948b85fae0 2024-03-18T20:42:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.288+0000 7f27d7f38b80 1 -- 172.21.15.46:0/789050829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b85fae0 msgr2=0x55948b866800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.288+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/789050829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b85fae0 0x55948b866800 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f27b80ca5e0 tx=0x7f27b8023040 comp rx=0 tx=0).stop 2024-03-18T20:42:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.288+0000 7f27d7f38b80 1 -- 172.21.15.46:0/789050829 shutdown_connections 2024-03-18T20:42:02.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.288+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/789050829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b85fae0 0x55948b866800 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.288+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/789050829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b86b2a0 0x55948b876650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.288+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/789050829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b860400 0x55948b866d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.288+0000 7f27d7f38b80 1 -- 172.21.15.46:0/789050829 >> 172.21.15.46:0/789050829 conn(0x55948b7d62a0 msgr2=0x55948b85bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:02.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.288+0000 7f27d7f38b80 1 -- 172.21.15.46:0/789050829 shutdown_connections 2024-03-18T20:42:02.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.288+0000 7f27d7f38b80 1 -- 172.21.15.46:0/789050829 wait complete. 2024-03-18T20:42:02.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27d7f38b80 1 -- start start 2024-03-18T20:42:02.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27d7f38b80 1 Processor -- start 2024-03-18T20:42:02.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27d7f38b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b86e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27d7f38b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b860400 0x55948b86ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27d7f38b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b86b2a0 0x55948b86f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27d7f38b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948b8cedd0 con 0x55948b86b2a0 2024-03-18T20:42:02.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27d7f38b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948b8cef10 con 0x55948b85fae0 2024-03-18T20:42:02.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27d7f38b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55948b8cf0a0 con 0x55948b860400 2024-03-18T20:42:02.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27d7f38b80 10 client.0 _ll_register_callbacks cb 0x55948b5ba420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:02.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27ca340700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b86b2a0 0x55948b86f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27c933e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b860400 0x55948b86ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27ca340700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b86b2a0 0x55948b86f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36734/0 (socket says 172.21.15.46:36734) 2024-03-18T20:42:02.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27ca340700 1 -- 172.21.15.46:0/3791172987 learned_addr learned my addr 172.21.15.46:0/3791172987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.289+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b86e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27c933e700 1 -- 172.21.15.46:0/3791172987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 msgr2=0x55948b86e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27c933e700 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b86e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27c933e700 1 -- 172.21.15.46:0/3791172987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b86b2a0 msgr2=0x55948b86f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27c933e700 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b86b2a0 0x55948b86f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27c933e700 1 -- 172.21.15.46:0/3791172987 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c41a63e0 con 0x55948b860400 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b86e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27ca340700 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b86b2a0 0x55948b86f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27c933e700 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b860400 0x55948b86ecb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005f50 tx=0x7f27c00246c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23c8107738424f66 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27ad7fa700 1 -- 172.21.15.46:0/3791172987 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c000f660 con 0x55948b860400 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27d7f38b80 1 -- 172.21.15.46:0/3791172987 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27bc00f040 con 0x55948b860400 2024-03-18T20:42:02.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27ad7fa700 1 -- 172.21.15.46:0/3791172987 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f27c0004e80 con 0x55948b860400 2024-03-18T20:42:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27ad7fa700 1 -- 172.21.15.46:0/3791172987 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c0024a00 con 0x55948b860400 2024-03-18T20:42:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27d7f38b80 10 client.27361 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.290+0000 7f27d7f38b80 1 -- 172.21.15.46:0/3791172987 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55948b5e0be0 con 0x55948b860400 2024-03-18T20:42:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.291+0000 7f27ad7fa700 1 -- 172.21.15.46:0/3791172987 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c004f020 con 0x55948b860400 2024-03-18T20:42:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.291+0000 7f27ad7fa700 1 -- 172.21.15.46:0/3791172987 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f27c0004500 con 0x55948b860400 2024-03-18T20:42:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.291+0000 7f27d7f38b80 10 client.27361 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.291+0000 7f27d7f38b80 10 client.27361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.291+0000 7f27d7f38b80 20 client.27361 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.291+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55948b8d0540 0x55948b8d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.291+0000 7f27d7f38b80 4 client.27361 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:42:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.291+0000 7f27d7f38b80 1 -- 172.21.15.46:0/3791172987 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55948b601d70 con 0x55948b8d0540 2024-03-18T20:42:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.291+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55948b8d0540 0x55948b8d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.291+0000 7f27ad7fa700 10 client.27361 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.291+0000 7f27c9b3f700 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55948b8d0540 0x55948b8d2960 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c32891772c2d4a12 server_cookie=1b93dd622307c3ea in_seq=0 out_seq=0 2024-03-18T20:42:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.292+0000 7f27ad7fa700 1 -- 172.21.15.46:0/3791172987 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55948b601d70 con 0x55948b8d0540 2024-03-18T20:42:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.292+0000 7f27ad7fa700 10 client.27361 handle_command_reply: tid=0 2024-03-18T20:42:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.292+0000 7f27d7f38b80 1 client.27361 shutdown 2024-03-18T20:42:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.293+0000 7f27d7f38b80 1 -- 172.21.15.46:0/3791172987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b860400 msgr2=0x55948b86ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.293+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b860400 0x55948b86ecb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005f50 tx=0x7f27c00246c0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.293+0000 7f27d7f38b80 1 -- 172.21.15.46:0/3791172987 shutdown_connections 2024-03-18T20:42:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.293+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55948b8d0540 0x55948b8d2960 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.293+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55948b86b2a0 0x55948b86f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.293+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55948b85fae0 0x55948b86e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.293+0000 7f27ad7fa700 0 client.27361 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:02.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.293+0000 7f27d7f38b80 1 --2- 172.21.15.46:0/3791172987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55948b860400 0x55948b86ecb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.293+0000 7f27d7f38b80 1 -- 172.21.15.46:0/3791172987 >> 172.21.15.46:0/3791172987 conn(0x55948b7d62a0 msgr2=0x55948b85bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:02.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.293+0000 7f27d7f38b80 1 -- 172.21.15.46:0/3791172987 shutdown_connections 2024-03-18T20:42:02.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.293+0000 7f27d7f38b80 1 -- 172.21.15.46:0/3791172987 wait complete. 2024-03-18T20:42:02.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.293+0000 7f27d7f38b80 20 client.27361 trim_cache size 0 max 16384 2024-03-18T20:42:02.302 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:42:02.302 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:42:02.302 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "ca476425-06fb-42fb-b0a0-5ea31e5b1818", 2024-03-18T20:42:02.302 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:42:02.303 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:42:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.294+0000 7f27cbda3700 1 -- 172.21.15.46:0/2122532063 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f27b00a1310 msgr2=0x7f27b00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.294+0000 7f27cbda3700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f27b00a1310 0x7f27b00a37d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f27c000b2e0 tx=0x7f27c0005960 comp rx=0 tx=0).stop 2024-03-18T20:42:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.294+0000 7f27cbda3700 1 -- 172.21.15.46:0/2122532063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c40ffe50 msgr2=0x7f27c41018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.294+0000 7f27cbda3700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c40ffe50 0x7f27c41018d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f27b800e9a0 tx=0x7f27b800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.294+0000 7f27cbda3700 1 -- 172.21.15.46:0/2122532063 shutdown_connections 2024-03-18T20:42:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.294+0000 7f27cbda3700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f27b00a1310 0x7f27b00a37d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.294+0000 7f27cbda3700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27c4102350 0x7f27c41a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.294+0000 7f27cbda3700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27c40ffe50 0x7f27c41018d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.294+0000 7f27cbda3700 1 --2- 172.21.15.46:0/2122532063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c4104f60 0x7f27c4101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.294+0000 7f27cbda3700 1 -- 172.21.15.46:0/2122532063 >> 172.21.15.46:0/2122532063 conn(0x7f27c4076d40 msgr2=0x7f27c4068230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.294+0000 7f27cbda3700 1 -- 172.21.15.46:0/2122532063 shutdown_connections 2024-03-18T20:42:02.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.294+0000 7f27cbda3700 1 -- 172.21.15.46:0/2122532063 wait complete. 2024-03-18T20:42:02.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:42:02.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:42:02.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:02 smithi073 ceph-mon[47384]: pgmap v181: 97 pgs: 97 active+clean; 78 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-03-18T20:42:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.468+0000 7eff98e9b700 1 -- 172.21.15.46:0/2135536095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff94100800 msgr2=0x7eff94104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.468+0000 7eff98e9b700 1 --2- 172.21.15.46:0/2135536095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff94100800 0x7eff94104830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff88009990 tx=0x7eff88009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.468+0000 7eff98e9b700 1 -- 172.21.15.46:0/2135536095 shutdown_connections 2024-03-18T20:42:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.468+0000 7eff98e9b700 1 --2- 172.21.15.46:0/2135536095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff940ffe50 0x7eff94100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.468+0000 7eff98e9b700 1 --2- 172.21.15.46:0/2135536095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff94104f60 0x7eff9410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.468+0000 7eff98e9b700 1 --2- 172.21.15.46:0/2135536095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff94100800 0x7eff94104830 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.468+0000 7eff98e9b700 1 -- 172.21.15.46:0/2135536095 >> 172.21.15.46:0/2135536095 conn(0x7eff94076d40 msgr2=0x7eff94077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.468+0000 7eff98e9b700 1 -- 172.21.15.46:0/2135536095 shutdown_connections 2024-03-18T20:42:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.468+0000 7eff98e9b700 1 -- 172.21.15.46:0/2135536095 wait complete. 2024-03-18T20:42:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.469+0000 7eff98e9b700 1 Processor -- start 2024-03-18T20:42:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.469+0000 7eff98e9b700 1 -- start start 2024-03-18T20:42:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.469+0000 7eff98e9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff940ffe50 0x7eff94197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.469+0000 7eff98e9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff94104f60 0x7eff94197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.469+0000 7eff98e9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff9419baa0 0x7eff9419bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff98e9b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9419c520 con 0x7eff940ffe50 2024-03-18T20:42:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff98e9b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9419c690 con 0x7eff9419baa0 2024-03-18T20:42:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff98e9b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9419c850 con 0x7eff94104f60 2024-03-18T20:42:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff9259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff940ffe50 0x7eff94197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff9259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff940ffe50 0x7eff94197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36762/0 (socket says 172.21.15.46:36762) 2024-03-18T20:42:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff9259c700 1 -- 172.21.15.46:0/3568231536 learned_addr learned my addr 172.21.15.46:0/3568231536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff92d9d700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff9419baa0 0x7eff9419bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff91d9b700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff94104f60 0x7eff94197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff9259c700 1 -- 172.21.15.46:0/3568231536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff94104f60 msgr2=0x7eff94197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff9259c700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff94104f60 0x7eff94197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff9259c700 1 -- 172.21.15.46:0/3568231536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff9419baa0 msgr2=0x7eff9419bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff9259c700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff9419baa0 0x7eff9419bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff9259c700 1 -- 172.21.15.46:0/3568231536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff880095f0 con 0x7eff940ffe50 2024-03-18T20:42:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff92d9d700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff9419baa0 0x7eff9419bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff9259c700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff940ffe50 0x7eff94197210 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7eff8400e9a0 tx=0x7eff8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff837fe700 1 -- 172.21.15.46:0/3568231536 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84009790 con 0x7eff940ffe50 2024-03-18T20:42:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff98e9b700 1 -- 172.21.15.46:0/3568231536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff941a3db0 con 0x7eff940ffe50 2024-03-18T20:42:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff98e9b700 1 -- 172.21.15.46:0/3568231536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff941a4280 con 0x7eff940ffe50 2024-03-18T20:42:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff837fe700 1 -- 172.21.15.46:0/3568231536 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff84010430 con 0x7eff940ffe50 2024-03-18T20:42:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.470+0000 7eff837fe700 1 -- 172.21.15.46:0/3568231536 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84018680 con 0x7eff940ffe50 2024-03-18T20:42:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.473+0000 7eff837fe700 1 -- 172.21.15.46:0/3568231536 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7eff8400ca10 con 0x7eff940ffe50 2024-03-18T20:42:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.474+0000 7eff837fe700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff7c0a1290 0x7eff7c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.474+0000 7eff837fe700 1 -- 172.21.15.46:0/3568231536 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7eff84014070 con 0x7eff940ffe50 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.474+0000 7eff91d9b700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff7c0a1290 0x7eff7c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.474+0000 7eff91d9b700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff7c0a1290 0x7eff7c0a3750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7eff8800b2e0 tx=0x7eff88005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.474+0000 7effa5030b80 1 Processor -- start 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7effa5030b80 1 -- start start 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7effa5030b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efca60 0x556f48efce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7effa5030b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 0x556f48f07ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7effa5030b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f0a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7effa5030b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48e56990 con 0x556f48f08220 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7effa5030b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48e54350 con 0x556f48efd380 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7effa5030b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48e4fc40 con 0x556f48efca60 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7eff91d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 0x556f48f07ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7eff92d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f0a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7eff92d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f0a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36764/0 (socket says 172.21.15.46:36764) 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7eff91d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 0x556f48f07ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:38726/0 (socket says 172.21.15.46:38726) 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7eff92d9d700 1 -- 172.21.15.46:0/739516895 learned_addr learned my addr 172.21.15.46:0/739516895 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.475+0000 7eff9259c700 1 --2- 172.21.15.46:0/739516895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efca60 0x556f48efce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.476+0000 7eff92d9d700 1 -- 172.21.15.46:0/739516895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efca60 msgr2=0x556f48efce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.476+0000 7eff92d9d700 1 --2- 172.21.15.46:0/739516895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efca60 0x556f48efce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.476+0000 7eff92d9d700 1 -- 172.21.15.46:0/739516895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 msgr2=0x556f48f07ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.476+0000 7eff92d9d700 1 --2- 172.21.15.46:0/739516895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 0x556f48f07ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.476+0000 7eff92d9d700 1 -- 172.21.15.46:0/739516895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f48cd13b0 con 0x556f48f08220 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.476+0000 7eff91d9b700 1 --2- 172.21.15.46:0/739516895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 0x556f48f07ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.476+0000 7eff92d9d700 1 --2- 172.21.15.46:0/739516895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f0a610 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7eff8c003eb0 tx=0x7eff8c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4b591f3a6353c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.476+0000 7eff80ff9700 1 -- 172.21.15.46:0/739516895 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c0092d0 con 0x556f48f08220 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.476+0000 7eff80ff9700 1 -- 172.21.15.46:0/739516895 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff8c00f040 con 0x556f48f08220 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.476+0000 7eff80ff9700 1 -- 172.21.15.46:0/739516895 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00e8b0 con 0x556f48f08220 2024-03-18T20:42:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.477+0000 7effa5030b80 1 -- 172.21.15.46:0/739516895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 msgr2=0x556f48f0a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.477+0000 7effa5030b80 1 --2- 172.21.15.46:0/739516895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f0a610 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7eff8c003eb0 tx=0x7eff8c009920 comp rx=0 tx=0).stop 2024-03-18T20:42:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.477+0000 7effa5030b80 1 -- 172.21.15.46:0/739516895 shutdown_connections 2024-03-18T20:42:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.477+0000 7effa5030b80 1 --2- 172.21.15.46:0/739516895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f0a610 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.477+0000 7effa5030b80 1 --2- 172.21.15.46:0/739516895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efca60 0x556f48efce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.477+0000 7effa5030b80 1 --2- 172.21.15.46:0/739516895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 0x556f48f07ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.477+0000 7effa5030b80 1 -- 172.21.15.46:0/739516895 >> 172.21.15.46:0/739516895 conn(0x556f48e73220 msgr2=0x556f48ef9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.477+0000 7effa5030b80 1 -- 172.21.15.46:0/739516895 shutdown_connections 2024-03-18T20:42:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.477+0000 7effa5030b80 1 -- 172.21.15.46:0/739516895 wait complete. 2024-03-18T20:42:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.477+0000 7effa5030b80 1 -- start start 2024-03-18T20:42:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.477+0000 7effa5030b80 1 Processor -- start 2024-03-18T20:42:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7effa5030b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efca60 0x556f48f07bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7effa5030b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efd380 0x556f48f04f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7effa5030b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7effa5030b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48debd80 con 0x556f48f08220 2024-03-18T20:42:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7effa5030b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48deba20 con 0x556f48efca60 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7effa5030b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48cd36a0 con 0x556f48efd380 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7effa5030b80 10 client.0 _ll_register_callbacks cb 0x556f48c573a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7eff9259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efca60 0x556f48f07bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7eff9259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efca60 0x556f48f07bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:38738/0 (socket says 172.21.15.46:38738) 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7eff9259c700 1 -- 172.21.15.46:0/293498846 learned_addr learned my addr 172.21.15.46:0/293498846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7eff92d9d700 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7eff91d9b700 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efd380 0x556f48f04f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7eff9259c700 1 -- 172.21.15.46:0/293498846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efd380 msgr2=0x556f48f04f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7eff9259c700 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efd380 0x556f48f04f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7eff9259c700 1 -- 172.21.15.46:0/293498846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 msgr2=0x556f48f055a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7eff9259c700 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f055a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.478+0000 7eff9259c700 1 -- 172.21.15.46:0/293498846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9410d350 con 0x556f48efca60 2024-03-18T20:42:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7eff9259c700 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efca60 0x556f48f07bc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7eff84010c20 tx=0x7eff8409a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60608a4942e6b22 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7eff79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7eff79ffb700 1 -- 172.21.15.46:0/293498846 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff840b1070 con 0x556f48efca60 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7eff79ffb700 1 -- 172.21.15.46:0/293498846 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7eff840a2390 con 0x556f48efca60 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7effa5030b80 1 -- 172.21.15.46:0/293498846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f48ef0ad0 con 0x556f48efca60 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7eff79ffb700 1 -- 172.21.15.46:0/293498846 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff840b8170 con 0x556f48efca60 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7effa5030b80 10 client.27356 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7effa5030b80 1 -- 172.21.15.46:0/293498846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7eff8c003b90 con 0x556f48efca60 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7eff79ffb700 1 -- 172.21.15.46:0/293498846 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff84063080 con 0x556f48efca60 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7eff79ffb700 1 -- 172.21.15.46:0/293498846 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7eff84095d00 con 0x556f48efca60 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7effa5030b80 10 client.27356 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7effa5030b80 10 client.27356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7effa5030b80 20 client.27356 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7effa5030b80 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556f48f17e60 0x556f48f1a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7effa5030b80 4 client.27356 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.479+0000 7effa5030b80 1 -- 172.21.15.46:0/293498846 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f48e48700 con 0x556f48f17e60 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.480+0000 7eff91d9b700 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556f48f17e60 0x556f48f1a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.480+0000 7eff91d9b700 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556f48f17e60 0x556f48f1a280 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688013e1bf33d71e server_cookie=2fcad6749c604f7f in_seq=0 out_seq=0 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.480+0000 7eff79ffb700 10 client.27356 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7eff79ffb700 1 -- 172.21.15.46:0/293498846 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556f48e48700 con 0x556f48f17e60 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7eff79ffb700 10 client.27356 handle_command_reply: tid=0 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7effa5030b80 1 client.27356 shutdown 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7effa5030b80 1 -- 172.21.15.46:0/293498846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efca60 msgr2=0x556f48f07bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7effa5030b80 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efca60 0x556f48f07bc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7eff84010c20 tx=0x7eff8409a760 comp rx=0 tx=0).stop 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7effa5030b80 1 -- 172.21.15.46:0/293498846 shutdown_connections 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7effa5030b80 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556f48f17e60 0x556f48f1a280 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7effa5030b80 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7effa5030b80 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efca60 0x556f48f07bc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7eff79ffb700 0 client.27356 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7effa5030b80 1 --2- 172.21.15.46:0/293498846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efd380 0x556f48f04f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7effa5030b80 1 -- 172.21.15.46:0/293498846 >> 172.21.15.46:0/293498846 conn(0x556f48e73220 msgr2=0x556f48ef8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7effa5030b80 1 -- 172.21.15.46:0/293498846 shutdown_connections 2024-03-18T20:42:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7effa5030b80 1 -- 172.21.15.46:0/293498846 wait complete. 2024-03-18T20:42:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.481+0000 7effa5030b80 20 client.27356 trim_cache size 0 max 16384 2024-03-18T20:42:02.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.499+0000 7effa5030b80 1 Processor -- start 2024-03-18T20:42:02.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.499+0000 7effa5030b80 1 -- start start 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.499+0000 7effa5030b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efca60 0x556f48eff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.499+0000 7effa5030b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48efd380 0x556f48f0eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.499+0000 7effa5030b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48f08220 0x556f48ef8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.499+0000 7effa5030b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48f0ac70 con 0x556f48efd380 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.499+0000 7effa5030b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48efdd30 con 0x556f48efca60 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.499+0000 7effa5030b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48efe550 con 0x556f48f08220 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.499+0000 7eff91d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48efd380 0x556f48f0eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.499+0000 7eff91d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48efd380 0x556f48f0eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36788/0 (socket says 172.21.15.46:36788) 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff91d9b700 1 -- 172.21.15.46:0/1158883577 learned_addr learned my addr 172.21.15.46:0/1158883577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff9259c700 1 --2- 172.21.15.46:0/1158883577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efca60 0x556f48eff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff92d9d700 1 --2- 172.21.15.46:0/1158883577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48f08220 0x556f48ef8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff91d9b700 1 -- 172.21.15.46:0/1158883577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48f08220 msgr2=0x556f48ef8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff91d9b700 1 --2- 172.21.15.46:0/1158883577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48f08220 0x556f48ef8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff91d9b700 1 -- 172.21.15.46:0/1158883577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efca60 msgr2=0x556f48eff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff91d9b700 1 --2- 172.21.15.46:0/1158883577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efca60 0x556f48eff9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff91d9b700 1 -- 172.21.15.46:0/1158883577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f48c7db60 con 0x556f48efd380 2024-03-18T20:42:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff92d9d700 1 --2- 172.21.15.46:0/1158883577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48f08220 0x556f48ef8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff91d9b700 1 --2- 172.21.15.46:0/1158883577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48efd380 0x556f48f0eee0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7eff88009990 tx=0x7eff88023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9285aa88c33c78 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff817fa700 1 -- 172.21.15.46:0/1158883577 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8800bd70 con 0x556f48efd380 2024-03-18T20:42:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff817fa700 1 -- 172.21.15.46:0/1158883577 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff8801e040 con 0x556f48efd380 2024-03-18T20:42:02.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.500+0000 7eff817fa700 1 -- 172.21.15.46:0/1158883577 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff88004ce0 con 0x556f48efd380 2024-03-18T20:42:02.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.501+0000 7effa5030b80 1 -- 172.21.15.46:0/1158883577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48efd380 msgr2=0x556f48f0eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.501+0000 7effa5030b80 1 --2- 172.21.15.46:0/1158883577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48efd380 0x556f48f0eee0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7eff88009990 tx=0x7eff88023000 comp rx=0 tx=0).stop 2024-03-18T20:42:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.501+0000 7effa5030b80 1 -- 172.21.15.46:0/1158883577 shutdown_connections 2024-03-18T20:42:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.501+0000 7effa5030b80 1 --2- 172.21.15.46:0/1158883577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48f08220 0x556f48ef8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.501+0000 7effa5030b80 1 --2- 172.21.15.46:0/1158883577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efca60 0x556f48eff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.501+0000 7effa5030b80 1 --2- 172.21.15.46:0/1158883577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48efd380 0x556f48f0eee0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.501+0000 7effa5030b80 1 -- 172.21.15.46:0/1158883577 >> 172.21.15.46:0/1158883577 conn(0x556f48e73220 msgr2=0x556f48f077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.501+0000 7effa5030b80 1 -- 172.21.15.46:0/1158883577 shutdown_connections 2024-03-18T20:42:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.501+0000 7effa5030b80 1 -- 172.21.15.46:0/1158883577 wait complete. 2024-03-18T20:42:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.501+0000 7effa5030b80 1 -- start start 2024-03-18T20:42:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.501+0000 7effa5030b80 1 Processor -- start 2024-03-18T20:42:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7effa5030b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efca60 0x556f48f0ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7effa5030b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 0x556f48f0d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7effa5030b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f0d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7effa5030b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48f0de40 con 0x556f48f08220 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7effa5030b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48f0dfb0 con 0x556f48efd380 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7effa5030b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f48f70070 con 0x556f48efca60 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7effa5030b80 10 client.0 _ll_register_callbacks cb 0x7eff840998b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff92d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f0d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff91d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 0x556f48f0d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff92d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f0d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36790/0 (socket says 172.21.15.46:36790) 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff92d9d700 1 -- 172.21.15.46:0/684621296 learned_addr learned my addr 172.21.15.46:0/684621296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff9259c700 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efca60 0x556f48f0ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff91d9b700 1 -- 172.21.15.46:0/684621296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efca60 msgr2=0x556f48f0ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff91d9b700 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efca60 0x556f48f0ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff91d9b700 1 -- 172.21.15.46:0/684621296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 msgr2=0x556f48f0d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:02.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff91d9b700 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f0d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff91d9b700 1 -- 172.21.15.46:0/684621296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff88035a90 con 0x556f48efd380 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff92d9d700 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f0d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff91d9b700 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 0x556f48f0d1f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x556f48f037b0 tx=0x7eff88004880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cacc2763972eadc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.502+0000 7eff79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7eff79ffb700 1 -- 172.21.15.46:0/684621296 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff88035370 con 0x556f48efd380 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7effa5030b80 1 -- 172.21.15.46:0/684621296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff8c00f040 con 0x556f48efd380 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7eff79ffb700 1 -- 172.21.15.46:0/684621296 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7eff880354d0 con 0x556f48efd380 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7eff79ffb700 1 -- 172.21.15.46:0/684621296 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff88004940 con 0x556f48efd380 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7effa5030b80 10 client.27362 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7effa5030b80 1 -- 172.21.15.46:0/684621296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x556f48e15ea0 con 0x556f48efd380 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7eff79ffb700 1 -- 172.21.15.46:0/684621296 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff8804d080 con 0x556f48efd380 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7eff79ffb700 1 -- 172.21.15.46:0/684621296 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7eff8800fa00 con 0x556f48efd380 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7effa5030b80 10 client.27362 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7effa5030b80 10 client.27362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7effa5030b80 20 client.27362 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7effa5030b80 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556f48f71410 0x556f48f73830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7effa5030b80 4 client.27362 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.503+0000 7effa5030b80 1 -- 172.21.15.46:0/684621296 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556f48c9ecf0 con 0x556f48f71410 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.504+0000 7eff9259c700 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556f48f71410 0x556f48f73830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.504+0000 7eff9259c700 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556f48f71410 0x556f48f73830 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ca8cae608d0a8a server_cookie=4d21f0f7aa1fa3c2 in_seq=0 out_seq=0 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.504+0000 7eff79ffb700 10 client.27362 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7eff79ffb700 1 -- 172.21.15.46:0/684621296 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556f48c9ecf0 con 0x556f48f71410 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7eff79ffb700 10 client.27362 handle_command_reply: tid=0 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7effa5030b80 1 client.27362 shutdown 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7effa5030b80 1 -- 172.21.15.46:0/684621296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 msgr2=0x556f48f0d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7effa5030b80 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 0x556f48f0d1f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x556f48f037b0 tx=0x7eff88004880 comp rx=0 tx=0).stop 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7effa5030b80 1 -- 172.21.15.46:0/684621296 shutdown_connections 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7effa5030b80 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556f48f71410 0x556f48f73830 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7effa5030b80 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f48f08220 0x556f48f0d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7effa5030b80 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f48efca60 0x556f48f0ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7eff79ffb700 0 client.27362 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7effa5030b80 1 --2- 172.21.15.46:0/684621296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f48efd380 0x556f48f0d1f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7effa5030b80 1 -- 172.21.15.46:0/684621296 >> 172.21.15.46:0/684621296 conn(0x556f48e73220 msgr2=0x556f48ef8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7effa5030b80 1 -- 172.21.15.46:0/684621296 shutdown_connections 2024-03-18T20:42:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7effa5030b80 1 -- 172.21.15.46:0/684621296 wait complete. 2024-03-18T20:42:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.505+0000 7effa5030b80 20 client.27362 trim_cache size 0 max 16384 2024-03-18T20:42:02.514 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:42:02.514 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:42:02.515 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:42:02.515 INFO:teuthology.orchestra.run.smithi046.stdout: "ca476425-06fb-42fb-b0a0-5ea31e5b1818": { 2024-03-18T20:42:02.515 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:42:02.515 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "ca476425-06fb-42fb-b0a0-5ea31e5b1818", 2024-03-18T20:42:02.515 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:42:02.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:42:02.515 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:42:02.515 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:42:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.506+0000 7eff98e9b700 1 -- 172.21.15.46:0/3568231536 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff7c0a1290 msgr2=0x7eff7c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.506+0000 7eff98e9b700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff7c0a1290 0x7eff7c0a3750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7eff8800b2e0 tx=0x7eff88005960 comp rx=0 tx=0).stop 2024-03-18T20:42:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.506+0000 7eff98e9b700 1 -- 172.21.15.46:0/3568231536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff940ffe50 msgr2=0x7eff94197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.506+0000 7eff98e9b700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff940ffe50 0x7eff94197210 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7eff8400e9a0 tx=0x7eff8400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.506+0000 7eff98e9b700 1 -- 172.21.15.46:0/3568231536 shutdown_connections 2024-03-18T20:42:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.506+0000 7eff98e9b700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff7c0a1290 0x7eff7c0a3750 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.506+0000 7eff98e9b700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff940ffe50 0x7eff94197210 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.506+0000 7eff98e9b700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff9419baa0 0x7eff9419bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.506+0000 7eff98e9b700 1 --2- 172.21.15.46:0/3568231536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff94104f60 0x7eff94197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.506+0000 7eff98e9b700 1 -- 172.21.15.46:0/3568231536 >> 172.21.15.46:0/3568231536 conn(0x7eff94076d40 msgr2=0x7eff940681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.506+0000 7eff98e9b700 1 -- 172.21.15.46:0/3568231536 shutdown_connections 2024-03-18T20:42:02.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:02.506+0000 7eff98e9b700 1 -- 172.21.15.46:0/3568231536 wait complete. 2024-03-18T20:42:02.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:02 smithi046 ceph-mon[43495]: pgmap v181: 97 pgs: 97 active+clean; 78 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-03-18T20:42:02.522 INFO:tasks.cephfs.filesystem:scrub status for tag:ca476425-06fb-42fb-b0a0-5ea31e5b1818 - {'path': '/', 'tag': 'ca476425-06fb-42fb-b0a0-5ea31e5b1818', 'options': 'recursive,force'} 2024-03-18T20:42:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:02 smithi155 ceph-mon[48344]: pgmap v181: 97 pgs: 97 active+clean; 78 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-03-18T20:42:02.694 INFO:tasks.workunit.client.0.smithi046.stdout: 1 193077 3.38 MB/sec execute 177 sec latency 19.706 ms 2024-03-18T20:42:03.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:42:03.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:03.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.472 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:03 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:42:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:03.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:03.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:03 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:42:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:03.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:03.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:03.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:03 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:03.694 INFO:tasks.workunit.client.0.smithi046.stdout: 1 193239 3.36 MB/sec execute 178 sec latency 886.721 ms 2024-03-18T20:42:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:04 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:04 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:04 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:04 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:04 smithi073 ceph-mon[47384]: pgmap v182: 97 pgs: 97 active+clean; 80 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2024-03-18T20:42:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:04 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:04 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:04 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:04 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:04.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:04 smithi155 ceph-mon[48344]: pgmap v182: 97 pgs: 97 active+clean; 80 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2024-03-18T20:42:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:04 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:04 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:04 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:04 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:04 smithi046 ceph-mon[43495]: pgmap v182: 97 pgs: 97 active+clean; 80 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2024-03-18T20:42:04.694 INFO:tasks.workunit.client.0.smithi046.stdout: 1 193239 3.34 MB/sec execute 179 sec latency 1886.834 ms 2024-03-18T20:42:05.694 INFO:tasks.workunit.client.0.smithi046.stdout: 1 194138 3.34 MB/sec execute 180 sec latency 2524.925 ms 2024-03-18T20:42:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:05 smithi073 ceph-mon[47384]: pgmap v183: 97 pgs: 97 active+clean; 80 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 837 KiB/s wr, 164 op/s 2024-03-18T20:42:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:05 smithi155 ceph-mon[48344]: pgmap v183: 97 pgs: 97 active+clean; 80 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 837 KiB/s wr, 164 op/s 2024-03-18T20:42:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:05 smithi046 ceph-mon[43495]: pgmap v183: 97 pgs: 97 active+clean; 80 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 837 KiB/s wr, 164 op/s 2024-03-18T20:42:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.425+0000 7f1e05b4a700 1 -- 172.21.15.46:0/336913842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e000680e0 msgr2=0x7f1e00068560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.425+0000 7f1e05b4a700 1 --2- 172.21.15.46:0/336913842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e000680e0 0x7f1e00068560 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009990 tx=0x7f1df4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.425+0000 7f1e05b4a700 1 -- 172.21.15.46:0/336913842 shutdown_connections 2024-03-18T20:42:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.425+0000 7f1e05b4a700 1 --2- 172.21.15.46:0/336913842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e00068aa0 0x7f1e0010dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.425+0000 7f1e05b4a700 1 --2- 172.21.15.46:0/336913842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e00106e10 0x7f1e001071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.425+0000 7f1e05b4a700 1 --2- 172.21.15.46:0/336913842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e000680e0 0x7f1e00068560 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.425+0000 7f1e05b4a700 1 -- 172.21.15.46:0/336913842 >> 172.21.15.46:0/336913842 conn(0x7f1e00076d50 msgr2=0x7f1e00077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:06.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.425+0000 7f1e05b4a700 1 -- 172.21.15.46:0/336913842 shutdown_connections 2024-03-18T20:42:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.425+0000 7f1e05b4a700 1 -- 172.21.15.46:0/336913842 wait complete. 2024-03-18T20:42:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1e05b4a700 1 Processor -- start 2024-03-18T20:42:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1e05b4a700 1 -- start start 2024-03-18T20:42:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1e05b4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e000680e0 0x7f1e000ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1e05b4a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e00068aa0 0x7f1e00100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1e05b4a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e00106e10 0x7f1e001006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1e05b4a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e001a5440 con 0x7f1e000680e0 2024-03-18T20:42:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1e05b4a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e001a55b0 con 0x7f1e00106e10 2024-03-18T20:42:06.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1e05b4a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e001a5770 con 0x7f1e00068aa0 2024-03-18T20:42:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1dff7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e000680e0 0x7f1e000ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1dfffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e00106e10 0x7f1e001006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e00068aa0 0x7f1e00100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1dff7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e000680e0 0x7f1e000ffc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33918/0 (socket says 172.21.15.46:33918) 2024-03-18T20:42:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.426+0000 7f1dff7fe700 1 -- 172.21.15.46:0/2917034214 learned_addr learned my addr 172.21.15.46:0/2917034214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1dfffff700 1 -- 172.21.15.46:0/2917034214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e00068aa0 msgr2=0x7f1e00100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1dfffff700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e00068aa0 0x7f1e00100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1dfffff700 1 -- 172.21.15.46:0/2917034214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e000680e0 msgr2=0x7f1e000ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1dfffff700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e000680e0 0x7f1e000ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1dfffff700 1 -- 172.21.15.46:0/2917034214 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df40095f0 con 0x7f1e00106e10 2024-03-18T20:42:06.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e00068aa0 0x7f1e00100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1dff7fe700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e000680e0 0x7f1e000ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1dfffff700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e00106e10 0x7f1e001006a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1df8007f10 tx=0x7f1df800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1dfcff9700 1 -- 172.21.15.46:0/2917034214 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8009ea0 con 0x7f1e00106e10 2024-03-18T20:42:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1e05b4a700 1 -- 172.21.15.46:0/2917034214 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e001a5a50 con 0x7f1e00106e10 2024-03-18T20:42:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1dfcff9700 1 -- 172.21.15.46:0/2917034214 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df800f040 con 0x7f1e00106e10 2024-03-18T20:42:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1dfcff9700 1 -- 172.21.15.46:0/2917034214 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8004350 con 0x7f1e00106e10 2024-03-18T20:42:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.427+0000 7f1e05b4a700 1 -- 172.21.15.46:0/2917034214 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e001a5ee0 con 0x7f1e00106e10 2024-03-18T20:42:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.430+0000 7f1dfcff9700 1 -- 172.21.15.46:0/2917034214 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1df800db20 con 0x7f1e00106e10 2024-03-18T20:42:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.430+0000 7f1dfcff9700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1de80a1290 0x7f1de80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.430+0000 7f1dff7fe700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1de80a1290 0x7f1de80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.430+0000 7f1dfcff9700 1 -- 172.21.15.46:0/2917034214 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f1df80cc600 con 0x7f1e00106e10 2024-03-18T20:42:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.431+0000 7f1dff7fe700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1de80a1290 0x7f1de80a3750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1df0011550 tx=0x7f1df0009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.431+0000 7f1e11cdfb80 1 Processor -- start 2024-03-18T20:42:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.431+0000 7f1e11cdfb80 1 -- start start 2024-03-18T20:42:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1e11cdfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc31252950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1e11cdfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc3125d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1e11cdfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc3125ddb0 0x55dc312601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1e11cdfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc311ac930 con 0x55dc3125ddb0 2024-03-18T20:42:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1e11cdfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc311aa2f0 con 0x55dc31252570 2024-03-18T20:42:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1e11cdfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc311a5be0 con 0x55dc31252e90 2024-03-18T20:42:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc3125d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1dfffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc3125ddb0 0x55dc312601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc3125d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58310/0 (socket says 172.21.15.46:58310) 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1dfeffd700 1 -- 172.21.15.46:0/979508024 learned_addr learned my addr 172.21.15.46:0/979508024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1dff7fe700 1 --2- 172.21.15.46:0/979508024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc31252950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1dfeffd700 1 -- 172.21.15.46:0/979508024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 msgr2=0x55dc31252950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/979508024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc31252950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1dfeffd700 1 -- 172.21.15.46:0/979508024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc3125ddb0 msgr2=0x55dc312601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/979508024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc3125ddb0 0x55dc312601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1dfeffd700 1 -- 172.21.15.46:0/979508024 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc31027350 con 0x55dc31252e90 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1dfffff700 1 --2- 172.21.15.46:0/979508024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc3125ddb0 0x55dc312601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/979508024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc3125d870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009990 tx=0x7f1df4005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd3b438626b279bd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.432+0000 7f1de5ffb700 1 -- 172.21.15.46:0/979508024 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df401d050 con 0x55dc31252e90 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.433+0000 7f1de5ffb700 1 -- 172.21.15.46:0/979508024 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df401e040 con 0x55dc31252e90 2024-03-18T20:42:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.433+0000 7f1de5ffb700 1 -- 172.21.15.46:0/979508024 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4005630 con 0x55dc31252e90 2024-03-18T20:42:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.433+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/979508024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 msgr2=0x55dc3125d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.433+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/979508024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc3125d870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009990 tx=0x7f1df4005bc0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.433+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/979508024 shutdown_connections 2024-03-18T20:42:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.433+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/979508024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc3125ddb0 0x55dc312601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.433+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/979508024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc31252950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.433+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/979508024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc3125d870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.433+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/979508024 >> 172.21.15.46:0/979508024 conn(0x55dc311c91c0 msgr2=0x55dc3124ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.433+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/979508024 shutdown_connections 2024-03-18T20:42:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.433+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/979508024 wait complete. 2024-03-18T20:42:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.434+0000 7f1e11cdfb80 1 -- start start 2024-03-18T20:42:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.434+0000 7f1e11cdfb80 1 Processor -- start 2024-03-18T20:42:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.434+0000 7f1e11cdfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252570 0x55dc3125d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.434+0000 7f1e11cdfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc3125aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.434+0000 7f1e11cdfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc3125ddb0 0x55dc3125b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.434+0000 7f1e11cdfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc31141d20 con 0x55dc31252570 2024-03-18T20:42:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.434+0000 7f1e11cdfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc311419c0 con 0x55dc3125ddb0 2024-03-18T20:42:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.434+0000 7f1e11cdfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc31029640 con 0x55dc31252e90 2024-03-18T20:42:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.434+0000 7f1e11cdfb80 10 client.0 _ll_register_callbacks cb 0x55dc30fad340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dff7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252570 0x55dc3125d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc3125aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dfffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc3125ddb0 0x55dc3125b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc3125aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58326/0 (socket says 172.21.15.46:58326) 2024-03-18T20:42:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dff7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252570 0x55dc3125d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33936/0 (socket says 172.21.15.46:33936) 2024-03-18T20:42:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dfeffd700 1 -- 172.21.15.46:0/1145700586 learned_addr learned my addr 172.21.15.46:0/1145700586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dfeffd700 1 -- 172.21.15.46:0/1145700586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc3125ddb0 msgr2=0x55dc3125b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc3125ddb0 0x55dc3125b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dfeffd700 1 -- 172.21.15.46:0/1145700586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252570 msgr2=0x55dc3125d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252570 0x55dc3125d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dfeffd700 1 -- 172.21.15.46:0/1145700586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e001a5ee0 con 0x55dc31252e90 2024-03-18T20:42:06.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dfffff700 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc3125ddb0 0x55dc3125b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dff7fe700 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252570 0x55dc3125d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc3125aac0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1df4000c00 tx=0x7f1df40055a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=840316eb3da133e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1ddeffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:42:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1ddeffd700 1 -- 172.21.15.46:0/1145700586 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4004bf0 con 0x55dc31252e90 2024-03-18T20:42:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1ddeffd700 1 -- 172.21.15.46:0/1145700586 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f1df4004fa0 con 0x55dc31252e90 2024-03-18T20:42:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.435+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1145700586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc312466c0 con 0x55dc31252e90 2024-03-18T20:42:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.436+0000 7f1ddeffd700 1 -- 172.21.15.46:0/1145700586 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df400f460 con 0x55dc31252e90 2024-03-18T20:42:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.436+0000 7f1e11cdfb80 10 client.27391 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.436+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1145700586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f1e0010e440 con 0x55dc31252e90 2024-03-18T20:42:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.436+0000 7f1ddeffd700 1 -- 172.21.15.46:0/1145700586 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc312466c0 con 0x55dc31252e90 2024-03-18T20:42:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.436+0000 7f1ddeffd700 1 -- 172.21.15.46:0/1145700586 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1df4023460 con 0x55dc31252e90 2024-03-18T20:42:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.436+0000 7f1e11cdfb80 10 client.27391 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.436+0000 7f1e11cdfb80 10 client.27391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.436+0000 7f1e11cdfb80 20 client.27391 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.436+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dc3126d580 0x55dc3126f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.436+0000 7f1e11cdfb80 4 client.27391 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.436+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1145700586 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc3119e6a0 con 0x55dc3126d580 2024-03-18T20:42:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.436+0000 7f1dff7fe700 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dc3126d580 0x55dc3126f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.437+0000 7f1ddeffd700 10 client.27391 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.437+0000 7f1dff7fe700 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dc3126d580 0x55dc3126f9a0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a096086d352393 server_cookie=bc4f88e2220c4753 in_seq=0 out_seq=0 2024-03-18T20:42:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.437+0000 7f1ddeffd700 1 -- 172.21.15.46:0/1145700586 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dc3119e6a0 con 0x55dc3126d580 2024-03-18T20:42:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.437+0000 7f1ddeffd700 10 client.27391 handle_command_reply: tid=0 2024-03-18T20:42:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.437+0000 7f1e11cdfb80 1 client.27391 shutdown 2024-03-18T20:42:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.438+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1145700586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 msgr2=0x55dc3125aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.438+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc3125aac0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1df4000c00 tx=0x7f1df40055a0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.438+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1145700586 shutdown_connections 2024-03-18T20:42:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.438+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dc3126d580 0x55dc3126f9a0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.438+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252570 0x55dc3125d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.438+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc3125ddb0 0x55dc3125b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.438+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/1145700586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc3125aac0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.438+0000 7f1ddeffd700 0 client.27391 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.438+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1145700586 >> 172.21.15.46:0/1145700586 conn(0x55dc311c91c0 msgr2=0x55dc3124e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.438+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1145700586 shutdown_connections 2024-03-18T20:42:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.438+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1145700586 wait complete. 2024-03-18T20:42:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.438+0000 7f1e11cdfb80 20 client.27391 trim_cache size 0 max 16384 2024-03-18T20:42:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1e11cdfb80 1 Processor -- start 2024-03-18T20:42:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1e11cdfb80 1 -- start start 2024-03-18T20:42:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1e11cdfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc3125b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1e11cdfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252e90 0x55dc312649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1e11cdfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc3125ddb0 0x55dc3124d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1e11cdfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc3125b6f0 con 0x55dc31252e90 2024-03-18T20:42:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1e11cdfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc31259010 con 0x55dc31252570 2024-03-18T20:42:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1e11cdfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc31256900 con 0x55dc3125ddb0 2024-03-18T20:42:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252e90 0x55dc312649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252e90 0x55dc312649e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33942/0 (socket says 172.21.15.46:33942) 2024-03-18T20:42:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1dfeffd700 1 -- 172.21.15.46:0/1338141031 learned_addr learned my addr 172.21.15.46:0/1338141031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1dfffff700 1 --2- 172.21.15.46:0/1338141031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc3125ddb0 0x55dc3124d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.454+0000 7f1dff7fe700 1 --2- 172.21.15.46:0/1338141031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc3125b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.455+0000 7f1dfeffd700 1 -- 172.21.15.46:0/1338141031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc3125ddb0 msgr2=0x55dc3124d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.455+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/1338141031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc3125ddb0 0x55dc3124d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.455+0000 7f1dfeffd700 1 -- 172.21.15.46:0/1338141031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 msgr2=0x55dc3125b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.455+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/1338141031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc3125b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.455+0000 7f1dfeffd700 1 -- 172.21.15.46:0/1338141031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc30fd3b00 con 0x55dc31252e90 2024-03-18T20:42:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.455+0000 7f1dff7fe700 1 --2- 172.21.15.46:0/1338141031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc3125b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.455+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/1338141031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252e90 0x55dc312649e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009990 tx=0x7f1df4023a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc220d0a500fe server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.455+0000 7f1de67fc700 1 -- 172.21.15.46:0/1338141031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df401d040 con 0x55dc31252e90 2024-03-18T20:42:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.455+0000 7f1de67fc700 1 -- 172.21.15.46:0/1338141031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df4023ae0 con 0x55dc31252e90 2024-03-18T20:42:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.455+0000 7f1de67fc700 1 -- 172.21.15.46:0/1338141031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40049f0 con 0x55dc31252e90 2024-03-18T20:42:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.456+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1338141031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252e90 msgr2=0x55dc312649e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.456+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/1338141031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252e90 0x55dc312649e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009990 tx=0x7f1df4023a10 comp rx=0 tx=0).stop 2024-03-18T20:42:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.456+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1338141031 shutdown_connections 2024-03-18T20:42:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.456+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/1338141031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc3125ddb0 0x55dc3124d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.456+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/1338141031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc3125b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.456+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/1338141031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc31252e90 0x55dc312649e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.456+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1338141031 >> 172.21.15.46:0/1338141031 conn(0x55dc311c91c0 msgr2=0x55dc31274300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.456+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1338141031 shutdown_connections 2024-03-18T20:42:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.456+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/1338141031 wait complete. 2024-03-18T20:42:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.456+0000 7f1e11cdfb80 1 -- start start 2024-03-18T20:42:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.456+0000 7f1e11cdfb80 1 Processor -- start 2024-03-18T20:42:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1e11cdfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc3125fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1e11cdfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc312609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1e11cdfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc3125ddb0 0x55dc31260ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1e11cdfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc31261560 con 0x55dc3125ddb0 2024-03-18T20:42:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1e11cdfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc312616d0 con 0x55dc31252570 2024-03-18T20:42:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1e11cdfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc312c03a0 con 0x55dc31252e90 2024-03-18T20:42:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1e11cdfb80 10 client.0 _ll_register_callbacks cb 0x7f1df80953a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dff7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc3125fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc312609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc312609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58348/0 (socket says 172.21.15.46:58348) 2024-03-18T20:42:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dff7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc3125fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:45358/0 (socket says 172.21.15.46:45358) 2024-03-18T20:42:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dfeffd700 1 -- 172.21.15.46:0/248128056 learned_addr learned my addr 172.21.15.46:0/248128056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dfffff700 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc3125ddb0 0x55dc31260ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dfeffd700 1 -- 172.21.15.46:0/248128056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 msgr2=0x55dc3125fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc3125fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dfeffd700 1 -- 172.21.15.46:0/248128056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc3125ddb0 msgr2=0x55dc31260ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc3125ddb0 0x55dc31260ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dfeffd700 1 -- 172.21.15.46:0/248128056 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df0017070 con 0x55dc31252e90 2024-03-18T20:42:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dff7fe700 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc3125fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1dfeffd700 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc312609b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1df4006010 tx=0x7f1df4005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b7e3c3a2b4fc6ad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.457+0000 7f1ddeffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:42:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1ddeffd700 1 -- 172.21.15.46:0/248128056 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df401d040 con 0x55dc31252e90 2024-03-18T20:42:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1ddeffd700 1 -- 172.21.15.46:0/248128056 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f1df40049f0 con 0x55dc31252e90 2024-03-18T20:42:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/248128056 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1df401e040 con 0x55dc31252e90 2024-03-18T20:42:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1ddeffd700 1 -- 172.21.15.46:0/248128056 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df4023610 con 0x55dc31252e90 2024-03-18T20:42:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1e11cdfb80 10 client.27397 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/248128056 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f1e0010e440 con 0x55dc31252e90 2024-03-18T20:42:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1ddeffd700 1 -- 172.21.15.46:0/248128056 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df4055080 con 0x55dc31252e90 2024-03-18T20:42:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1ddeffd700 1 -- 172.21.15.46:0/248128056 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1df400fc70 con 0x55dc31252e90 2024-03-18T20:42:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1e11cdfb80 10 client.27397 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1e11cdfb80 10 client.27397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1e11cdfb80 20 client.27397 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dc312c1720 0x55dc312c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1e11cdfb80 4 client.27397 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:42:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.458+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/248128056 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dc30ff4c90 con 0x55dc312c1720 2024-03-18T20:42:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.459+0000 7f1dff7fe700 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dc312c1720 0x55dc312c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.459+0000 7f1ddeffd700 10 client.27397 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.459+0000 7f1dff7fe700 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dc312c1720 0x55dc312c3b40 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed8202912ee8b6ac server_cookie=dd67c52119e293c8 in_seq=0 out_seq=0 2024-03-18T20:42:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.484+0000 7f1ddeffd700 1 -- 172.21.15.46:0/248128056 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dc30ff4c90 con 0x55dc312c1720 2024-03-18T20:42:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.484+0000 7f1ddeffd700 10 client.27397 handle_command_reply: tid=0 2024-03-18T20:42:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.484+0000 7f1e11cdfb80 1 client.27397 shutdown 2024-03-18T20:42:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.485+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/248128056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 msgr2=0x55dc312609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.485+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc312609b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1df4006010 tx=0x7f1df4005bc0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.485+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/248128056 shutdown_connections 2024-03-18T20:42:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.485+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dc312c1720 0x55dc312c3b40 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.485+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc3125ddb0 0x55dc31260ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.485+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc31252570 0x55dc3125fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.485+0000 7f1e11cdfb80 1 --2- 172.21.15.46:0/248128056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc31252e90 0x55dc312609b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.485+0000 7f1ddeffd700 0 client.27397 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.485+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/248128056 >> 172.21.15.46:0/248128056 conn(0x55dc311c91c0 msgr2=0x55dc31255d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.485+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/248128056 shutdown_connections 2024-03-18T20:42:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.485+0000 7f1e11cdfb80 1 -- 172.21.15.46:0/248128056 wait complete. 2024-03-18T20:42:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.485+0000 7f1e11cdfb80 20 client.27397 trim_cache size 0 max 16384 2024-03-18T20:42:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:42:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:42:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:42:06.541 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:42:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.486+0000 7f1e05b4a700 1 -- 172.21.15.46:0/2917034214 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1de80a1290 msgr2=0x7f1de80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.486+0000 7f1e05b4a700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1de80a1290 0x7f1de80a3750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1df0011550 tx=0x7f1df0009260 comp rx=0 tx=0).stop 2024-03-18T20:42:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.486+0000 7f1e05b4a700 1 -- 172.21.15.46:0/2917034214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e00106e10 msgr2=0x7f1e001006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.486+0000 7f1e05b4a700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e00106e10 0x7f1e001006a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1df8007f10 tx=0x7f1df800d3b0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.486+0000 7f1e05b4a700 1 -- 172.21.15.46:0/2917034214 shutdown_connections 2024-03-18T20:42:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.486+0000 7f1e05b4a700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1de80a1290 0x7f1de80a3750 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.486+0000 7f1e05b4a700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e000680e0 0x7f1e000ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.486+0000 7f1e05b4a700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e00106e10 0x7f1e001006a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.486+0000 7f1e05b4a700 1 --2- 172.21.15.46:0/2917034214 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e00068aa0 0x7f1e00100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.486+0000 7f1e05b4a700 1 -- 172.21.15.46:0/2917034214 >> 172.21.15.46:0/2917034214 conn(0x7f1e00076d50 msgr2=0x7f1e000fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.486+0000 7f1e05b4a700 1 -- 172.21.15.46:0/2917034214 shutdown_connections 2024-03-18T20:42:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:06.486+0000 7f1e05b4a700 1 -- 172.21.15.46:0/2917034214 wait complete. 2024-03-18T20:42:06.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 195565 3.38 MB/sec execute 181 sec latency 16.077 ms 2024-03-18T20:42:07.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 196788 3.39 MB/sec execute 182 sec latency 136.837 ms 2024-03-18T20:42:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:08 smithi073 ceph-mon[47384]: pgmap v184: 97 pgs: 97 active+clean; 80 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 837 KiB/s wr, 164 op/s 2024-03-18T20:42:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:08 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:42:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:08 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:42:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:08 smithi155 ceph-mon[48344]: pgmap v184: 97 pgs: 97 active+clean; 80 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 837 KiB/s wr, 164 op/s 2024-03-18T20:42:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:08 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:42:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:08 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:42:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:08 smithi046 ceph-mon[43495]: pgmap v184: 97 pgs: 97 active+clean; 80 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 837 KiB/s wr, 164 op/s 2024-03-18T20:42:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:08 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:42:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:08 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:42:08.694 INFO:tasks.workunit.client.0.smithi046.stdout: 1 196788 3.37 MB/sec execute 183 sec latency 1136.887 ms 2024-03-18T20:42:09.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:09 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:42:09.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:09 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:42:09.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:09 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:42:09.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:09 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:42:09.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:09 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:42:09.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:09 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:42:09.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:09 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:42:09.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:09 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:42:09.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:09 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:42:09.694 INFO:tasks.workunit.client.0.smithi046.stdout: 1 196788 3.35 MB/sec execute 184 sec latency 2136.948 ms 2024-03-18T20:42:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:10 smithi073 ceph-mon[47384]: pgmap v185: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-03-18T20:42:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:10 smithi155 ceph-mon[48344]: pgmap v185: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-03-18T20:42:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:10 smithi046 ceph-mon[43495]: pgmap v185: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-03-18T20:42:10.694 INFO:tasks.workunit.client.0.smithi046.stdout: 1 198728 3.40 MB/sec execute 185 sec latency 2298.196 ms 2024-03-18T20:42:11.694 INFO:tasks.workunit.client.0.smithi046.stdout: 1 199890 3.40 MB/sec execute 186 sec latency 16.202 ms 2024-03-18T20:42:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:12 smithi073 ceph-mon[47384]: pgmap v186: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.0 MiB/s wr, 160 op/s 2024-03-18T20:42:12.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:12 smithi155 ceph-mon[48344]: pgmap v186: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.0 MiB/s wr, 160 op/s 2024-03-18T20:42:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:12 smithi046 ceph-mon[43495]: pgmap v186: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.0 MiB/s wr, 160 op/s 2024-03-18T20:42:12.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 200331 3.40 MB/sec execute 187 sec latency 781.668 ms 2024-03-18T20:42:13.694 INFO:tasks.workunit.client.0.smithi046.stdout: 1 200331 3.38 MB/sec execute 188 sec latency 1781.718 ms 2024-03-18T20:42:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:14 smithi073 ceph-mon[47384]: pgmap v187: 97 pgs: 97 active+clean; 82 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-03-18T20:42:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:14 smithi155 ceph-mon[48344]: pgmap v187: 97 pgs: 97 active+clean; 82 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-03-18T20:42:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:14 smithi046 ceph-mon[43495]: pgmap v187: 97 pgs: 97 active+clean; 82 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-03-18T20:42:14.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 200331 3.37 MB/sec execute 189 sec latency 2781.769 ms 2024-03-18T20:42:15.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:15 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:42:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:15 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:42:15.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:15 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:42:15.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 201198 3.36 MB/sec execute 190 sec latency 3412.364 ms 2024-03-18T20:42:16.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:16 smithi073 ceph-mon[47384]: pgmap v188: 97 pgs: 97 active+clean; 82 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-03-18T20:42:16.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:16 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:42:16.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:16 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:42:16.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:16 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:42:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:16 smithi155 ceph-mon[48344]: pgmap v188: 97 pgs: 97 active+clean; 82 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-03-18T20:42:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:16 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:42:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:16 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:42:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:16 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:42:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:16 smithi046 ceph-mon[43495]: pgmap v188: 97 pgs: 97 active+clean; 82 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-03-18T20:42:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:16 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:42:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:16 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:42:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:16 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:42:16.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 202641 3.40 MB/sec execute 191 sec latency 41.325 ms 2024-03-18T20:42:17.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 203878 3.41 MB/sec execute 192 sec latency 240.339 ms 2024-03-18T20:42:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:18 smithi073 ceph-mon[47384]: pgmap v189: 97 pgs: 97 active+clean; 82 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-03-18T20:42:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:18 smithi155 ceph-mon[48344]: pgmap v189: 97 pgs: 97 active+clean; 82 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-03-18T20:42:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:18 smithi046 ceph-mon[43495]: pgmap v189: 97 pgs: 97 active+clean; 82 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.6 MiB/s wr, 134 op/s 2024-03-18T20:42:18.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 203878 3.39 MB/sec execute 193 sec latency 1240.407 ms 2024-03-18T20:42:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.613+0000 7ffb6b3be700 1 -- 172.21.15.46:0/1783024243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb64100f10 msgr2=0x7ffb641012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.613+0000 7ffb6b3be700 1 --2- 172.21.15.46:0/1783024243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb64100f10 0x7ffb641012f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009990 tx=0x7ffb60009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.613+0000 7ffb6b3be700 1 -- 172.21.15.46:0/1783024243 shutdown_connections 2024-03-18T20:42:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.613+0000 7ffb6b3be700 1 --2- 172.21.15.46:0/1783024243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb64100f10 0x7ffb641012f0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.613+0000 7ffb6b3be700 1 --2- 172.21.15.46:0/1783024243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb64106030 0x7ffb6410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.613+0000 7ffb6b3be700 1 --2- 172.21.15.46:0/1783024243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb641018c0 0x7ffb64105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.613+0000 7ffb6b3be700 1 -- 172.21.15.46:0/1783024243 >> 172.21.15.46:0/1783024243 conn(0x7ffb640fc770 msgr2=0x7ffb640feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.614+0000 7ffb6b3be700 1 -- 172.21.15.46:0/1783024243 shutdown_connections 2024-03-18T20:42:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.614+0000 7ffb6b3be700 1 -- 172.21.15.46:0/1783024243 wait complete. 2024-03-18T20:42:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.614+0000 7ffb6b3be700 1 Processor -- start 2024-03-18T20:42:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.614+0000 7ffb6b3be700 1 -- start start 2024-03-18T20:42:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6b3be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb64100f10 0x7ffb641971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6b3be700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb64106030 0x7ffb64197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6b3be700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6419bac0 0x7ffb6419bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6b3be700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6419c540 con 0x7ffb64100f10 2024-03-18T20:42:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6b3be700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6419c6b0 con 0x7ffb64106030 2024-03-18T20:42:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6b3be700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6419c870 con 0x7ffb6419bac0 2024-03-18T20:42:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6915a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb64100f10 0x7ffb641971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6915a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb64100f10 0x7ffb641971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52524/0 (socket says 172.21.15.46:52524) 2024-03-18T20:42:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6915a700 1 -- 172.21.15.46:0/2367299160 learned_addr learned my addr 172.21.15.46:0/2367299160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6995b700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6419bac0 0x7ffb6419bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb68959700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb64106030 0x7ffb64197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6915a700 1 -- 172.21.15.46:0/2367299160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6419bac0 msgr2=0x7ffb6419bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6915a700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6419bac0 0x7ffb6419bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6915a700 1 -- 172.21.15.46:0/2367299160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb64106030 msgr2=0x7ffb64197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6915a700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb64106030 0x7ffb64197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6915a700 1 -- 172.21.15.46:0/2367299160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb600095f0 con 0x7ffb64100f10 2024-03-18T20:42:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.615+0000 7ffb6915a700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb64100f10 0x7ffb641971e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ffb60005770 tx=0x7ffb60009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.616+0000 7ffb5a7fc700 1 -- 172.21.15.46:0/2367299160 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6001d070 con 0x7ffb64100f10 2024-03-18T20:42:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.616+0000 7ffb5a7fc700 1 -- 172.21.15.46:0/2367299160 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb6000fbf0 con 0x7ffb64100f10 2024-03-18T20:42:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.616+0000 7ffb6b3be700 1 -- 172.21.15.46:0/2367299160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb641a3d70 con 0x7ffb64100f10 2024-03-18T20:42:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.616+0000 7ffb5a7fc700 1 -- 172.21.15.46:0/2367299160 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60022bf0 con 0x7ffb64100f10 2024-03-18T20:42:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.616+0000 7ffb6b3be700 1 -- 172.21.15.46:0/2367299160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb641a41a0 con 0x7ffb64100f10 2024-03-18T20:42:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.618+0000 7ffb5a7fc700 1 -- 172.21.15.46:0/2367299160 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ffb60022d50 con 0x7ffb64100f10 2024-03-18T20:42:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.619+0000 7ffb5a7fc700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ffb500a1310 0x7ffb500a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.619+0000 7ffb68959700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ffb500a1310 0x7ffb500a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.619+0000 7ffb5a7fc700 1 -- 172.21.15.46:0/2367299160 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7ffb600cdde0 con 0x7ffb64100f10 2024-03-18T20:42:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.620+0000 7ffb68959700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ffb500a1310 0x7ffb500a37d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005950 tx=0x7ffb540058e0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.620+0000 7ffb77553b80 1 Processor -- start 2024-03-18T20:42:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.620+0000 7ffb77553b80 1 -- start start 2024-03-18T20:42:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb77553b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697db4570 0x55d697db4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb77553b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 0x55d697dc3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb77553b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697dc41c0 0x55d697dc65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb77553b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d697d0e930 con 0x55d697db4e90 2024-03-18T20:42:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb77553b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d697d0c2f0 con 0x55d697db4570 2024-03-18T20:42:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb77553b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d697d07be0 con 0x55d697dc41c0 2024-03-18T20:42:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb68959700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 0x55d697dc3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb68959700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 0x55d697dc3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52526/0 (socket says 172.21.15.46:52526) 2024-03-18T20:42:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb68959700 1 -- 172.21.15.46:0/1830962372 learned_addr learned my addr 172.21.15.46:0/1830962372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb6915a700 1 --2- 172.21.15.46:0/1830962372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697db4570 0x55d697db4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb6995b700 1 --2- 172.21.15.46:0/1830962372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697dc41c0 0x55d697dc65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb68959700 1 -- 172.21.15.46:0/1830962372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697dc41c0 msgr2=0x55d697dc65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb68959700 1 --2- 172.21.15.46:0/1830962372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697dc41c0 0x55d697dc65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb68959700 1 -- 172.21.15.46:0/1830962372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697db4570 msgr2=0x55d697db4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb68959700 1 --2- 172.21.15.46:0/1830962372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697db4570 0x55d697db4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb68959700 1 -- 172.21.15.46:0/1830962372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d697b89350 con 0x55d697db4e90 2024-03-18T20:42:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb68959700 1 --2- 172.21.15.46:0/1830962372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 0x55d697dc3c80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400fd10 tx=0x7ffb54012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f109f19dc3b6ed server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.621+0000 7ffb4f7fe700 1 -- 172.21.15.46:0/1830962372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54011ad0 con 0x55d697db4e90 2024-03-18T20:42:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.622+0000 7ffb4f7fe700 1 -- 172.21.15.46:0/1830962372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb54011c30 con 0x55d697db4e90 2024-03-18T20:42:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.622+0000 7ffb4f7fe700 1 -- 172.21.15.46:0/1830962372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5401d5f0 con 0x55d697db4e90 2024-03-18T20:42:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.622+0000 7ffb77553b80 1 -- 172.21.15.46:0/1830962372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 msgr2=0x55d697dc3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.622+0000 7ffb77553b80 1 --2- 172.21.15.46:0/1830962372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 0x55d697dc3c80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400fd10 tx=0x7ffb54012000 comp rx=0 tx=0).stop 2024-03-18T20:42:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.622+0000 7ffb77553b80 1 -- 172.21.15.46:0/1830962372 shutdown_connections 2024-03-18T20:42:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.622+0000 7ffb77553b80 1 --2- 172.21.15.46:0/1830962372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697dc41c0 0x55d697dc65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.622+0000 7ffb77553b80 1 --2- 172.21.15.46:0/1830962372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697db4570 0x55d697db4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.622+0000 7ffb77553b80 1 --2- 172.21.15.46:0/1830962372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 0x55d697dc3c80 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.622+0000 7ffb77553b80 1 -- 172.21.15.46:0/1830962372 >> 172.21.15.46:0/1830962372 conn(0x55d697d2b1c0 msgr2=0x55d697db0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.622+0000 7ffb77553b80 1 -- 172.21.15.46:0/1830962372 shutdown_connections 2024-03-18T20:42:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.622+0000 7ffb77553b80 1 -- 172.21.15.46:0/1830962372 wait complete. 2024-03-18T20:42:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.623+0000 7ffb77553b80 1 -- start start 2024-03-18T20:42:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.623+0000 7ffb77553b80 1 Processor -- start 2024-03-18T20:42:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.623+0000 7ffb77553b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697db4570 0x55d697dc3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.623+0000 7ffb77553b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 0x55d697dc0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.623+0000 7ffb77553b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697dc41c0 0x55d697dc1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.623+0000 7ffb77553b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d697ca3d20 con 0x55d697db4e90 2024-03-18T20:42:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.623+0000 7ffb77553b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d697ca39c0 con 0x55d697dc41c0 2024-03-18T20:42:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.623+0000 7ffb77553b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d697b8b640 con 0x55d697db4570 2024-03-18T20:42:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.623+0000 7ffb77553b80 10 client.0 _ll_register_callbacks cb 0x55d697b0f340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb68959700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 0x55d697dc0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb68959700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 0x55d697dc0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52530/0 (socket says 172.21.15.46:52530) 2024-03-18T20:42:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb68959700 1 -- 172.21.15.46:0/1000703901 learned_addr learned my addr 172.21.15.46:0/1000703901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb6915a700 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697db4570 0x55d697dc3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb6995b700 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697dc41c0 0x55d697dc1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb68959700 1 -- 172.21.15.46:0/1000703901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697db4570 msgr2=0x55d697dc3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb68959700 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697db4570 0x55d697dc3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb68959700 1 -- 172.21.15.46:0/1000703901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697dc41c0 msgr2=0x55d697dc1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb68959700 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697dc41c0 0x55d697dc1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb68959700 1 -- 172.21.15.46:0/1000703901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5400f9f0 con 0x55d697db4e90 2024-03-18T20:42:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb68959700 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 0x55d697dc0e40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ffb54011ae0 tx=0x7ffb54011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9f021490098192 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb4cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb4cff9700 1 -- 172.21.15.46:0/1000703901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5402d070 con 0x55d697db4e90 2024-03-18T20:42:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb4cff9700 1 -- 172.21.15.46:0/1000703901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7ffb5401e660 con 0x55d697db4e90 2024-03-18T20:42:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb77553b80 1 -- 172.21.15.46:0/1000703901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d697da86c0 con 0x55d697db4e90 2024-03-18T20:42:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb4cff9700 1 -- 172.21.15.46:0/1000703901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb54011e90 con 0x55d697db4e90 2024-03-18T20:42:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb77553b80 10 client.17709 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.624+0000 7ffb77553b80 1 -- 172.21.15.46:0/1000703901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55d697b35b00 con 0x55d697db4e90 2024-03-18T20:42:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.625+0000 7ffb4cff9700 1 -- 172.21.15.46:0/1000703901 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb5403f020 con 0x55d697db4e90 2024-03-18T20:42:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.625+0000 7ffb4cff9700 1 -- 172.21.15.46:0/1000703901 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ffb54027450 con 0x55d697db4e90 2024-03-18T20:42:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.625+0000 7ffb77553b80 10 client.17709 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.625+0000 7ffb77553b80 10 client.17709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.625+0000 7ffb77553b80 20 client.17709 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.625+0000 7ffb77553b80 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d697dd3db0 0x55d697dd61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.625+0000 7ffb77553b80 4 client.17709 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.625+0000 7ffb77553b80 1 -- 172.21.15.46:0/1000703901 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d697d006a0 con 0x55d697dd3db0 2024-03-18T20:42:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.625+0000 7ffb6915a700 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d697dd3db0 0x55d697dd61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.626+0000 7ffb6915a700 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d697dd3db0 0x55d697dd61d0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a59009c990a7a7c3 server_cookie=2b3d04d6391d9782 in_seq=0 out_seq=0 2024-03-18T20:42:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.626+0000 7ffb4cff9700 10 client.17709 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.626+0000 7ffb4cff9700 1 -- 172.21.15.46:0/1000703901 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d697d006a0 con 0x55d697dd3db0 2024-03-18T20:42:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.626+0000 7ffb4cff9700 10 client.17709 handle_command_reply: tid=0 2024-03-18T20:42:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.626+0000 7ffb77553b80 1 client.17709 shutdown 2024-03-18T20:42:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.627+0000 7ffb77553b80 1 -- 172.21.15.46:0/1000703901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 msgr2=0x55d697dc0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.627+0000 7ffb77553b80 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 0x55d697dc0e40 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ffb54011ae0 tx=0x7ffb54011b10 comp rx=0 tx=0).stop 2024-03-18T20:42:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.627+0000 7ffb77553b80 1 -- 172.21.15.46:0/1000703901 shutdown_connections 2024-03-18T20:42:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.627+0000 7ffb77553b80 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d697dd3db0 0x55d697dd61d0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.627+0000 7ffb77553b80 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697dc41c0 0x55d697dc1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.627+0000 7ffb77553b80 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697db4570 0x55d697dc3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.627+0000 7ffb77553b80 1 --2- 172.21.15.46:0/1000703901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4e90 0x55d697dc0e40 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.627+0000 7ffb4cff9700 0 client.17709 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.627+0000 7ffb77553b80 1 -- 172.21.15.46:0/1000703901 >> 172.21.15.46:0/1000703901 conn(0x55d697d2b1c0 msgr2=0x55d697db0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.627+0000 7ffb77553b80 1 -- 172.21.15.46:0/1000703901 shutdown_connections 2024-03-18T20:42:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.627+0000 7ffb77553b80 1 -- 172.21.15.46:0/1000703901 wait complete. 2024-03-18T20:42:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.627+0000 7ffb77553b80 20 client.17709 trim_cache size 0 max 16384 2024-03-18T20:42:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.642+0000 7ffb77553b80 1 Processor -- start 2024-03-18T20:42:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.642+0000 7ffb77553b80 1 -- start start 2024-03-18T20:42:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb77553b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 0x55d697dc0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb77553b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697db4e90 0x55d697dc36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb77553b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697dc41c0 0x55d697dcf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb77553b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d697dbf410 con 0x55d697db4570 2024-03-18T20:42:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb77553b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d697dbcd20 con 0x55d697dc41c0 2024-03-18T20:42:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb77553b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d697da8200 con 0x55d697db4e90 2024-03-18T20:42:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb6915a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 0x55d697dc0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb6915a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 0x55d697dc0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52532/0 (socket says 172.21.15.46:52532) 2024-03-18T20:42:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb6915a700 1 -- 172.21.15.46:0/2106420351 learned_addr learned my addr 172.21.15.46:0/2106420351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb68959700 1 --2- 172.21.15.46:0/2106420351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697db4e90 0x55d697dc36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb6995b700 1 --2- 172.21.15.46:0/2106420351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697dc41c0 0x55d697dcf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb6915a700 1 -- 172.21.15.46:0/2106420351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697db4e90 msgr2=0x55d697dc36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb6915a700 1 --2- 172.21.15.46:0/2106420351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697db4e90 0x55d697dc36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb6915a700 1 -- 172.21.15.46:0/2106420351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697dc41c0 msgr2=0x55d697dcf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb6915a700 1 --2- 172.21.15.46:0/2106420351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697dc41c0 0x55d697dcf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb6915a700 1 -- 172.21.15.46:0/2106420351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d697ccde40 con 0x55d697db4570 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb6995b700 1 --2- 172.21.15.46:0/2106420351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697dc41c0 0x55d697dcf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb6915a700 1 --2- 172.21.15.46:0/2106420351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 0x55d697dc0e20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ffb6009adb0 tx=0x7ffb6002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f2076ad9c348ca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb4ffff700 1 -- 172.21.15.46:0/2106420351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6009e040 con 0x55d697db4570 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb4ffff700 1 -- 172.21.15.46:0/2106420351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb6009eee0 con 0x55d697db4570 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.643+0000 7ffb4ffff700 1 -- 172.21.15.46:0/2106420351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60099520 con 0x55d697db4570 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.644+0000 7ffb77553b80 1 -- 172.21.15.46:0/2106420351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 msgr2=0x55d697dc0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.644+0000 7ffb77553b80 1 --2- 172.21.15.46:0/2106420351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 0x55d697dc0e20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ffb6009adb0 tx=0x7ffb6002e040 comp rx=0 tx=0).stop 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.644+0000 7ffb77553b80 1 -- 172.21.15.46:0/2106420351 shutdown_connections 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.644+0000 7ffb77553b80 1 --2- 172.21.15.46:0/2106420351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 0x55d697dc0e20 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.644+0000 7ffb77553b80 1 --2- 172.21.15.46:0/2106420351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697dc41c0 0x55d697dcf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.644+0000 7ffb77553b80 1 --2- 172.21.15.46:0/2106420351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697db4e90 0x55d697dc36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.644+0000 7ffb77553b80 1 -- 172.21.15.46:0/2106420351 >> 172.21.15.46:0/2106420351 conn(0x55d697d2b1c0 msgr2=0x55d697dc0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.644+0000 7ffb77553b80 1 -- 172.21.15.46:0/2106420351 shutdown_connections 2024-03-18T20:42:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.644+0000 7ffb77553b80 1 -- 172.21.15.46:0/2106420351 wait complete. 2024-03-18T20:42:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.645+0000 7ffb77553b80 1 -- start start 2024-03-18T20:42:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.645+0000 7ffb77553b80 1 Processor -- start 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.645+0000 7ffb77553b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 0x55d697dbe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.645+0000 7ffb77553b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697db4e90 0x55d697dbe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.645+0000 7ffb77553b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697dc41c0 0x55d697dbd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.645+0000 7ffb77553b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d697dbefc0 con 0x55d697db4570 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.645+0000 7ffb77553b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d697dbd670 con 0x55d697db4e90 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.645+0000 7ffb77553b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d697dbd830 con 0x55d697dc41c0 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.645+0000 7ffb77553b80 10 client.0 _ll_register_callbacks cb 0x7ffb600058f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.645+0000 7ffb6915a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 0x55d697dbe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.645+0000 7ffb6915a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 0x55d697dbe370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52538/0 (socket says 172.21.15.46:52538) 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.645+0000 7ffb6915a700 1 -- 172.21.15.46:0/3506282695 learned_addr learned my addr 172.21.15.46:0/3506282695 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb6995b700 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697dc41c0 0x55d697dbd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb68959700 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697db4e90 0x55d697dbe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb6915a700 1 -- 172.21.15.46:0/3506282695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697dc41c0 msgr2=0x55d697dbd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb6915a700 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697dc41c0 0x55d697dbd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb6915a700 1 -- 172.21.15.46:0/3506282695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697db4e90 msgr2=0x55d697dbe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb6915a700 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697db4e90 0x55d697dbe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb6915a700 1 -- 172.21.15.46:0/3506282695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb641a41a0 con 0x55d697db4570 2024-03-18T20:42:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb6995b700 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697dc41c0 0x55d697dbd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb6915a700 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 0x55d697dbe370 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ffb60097b20 tx=0x7ffb600226c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff6c224c2afdfcb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb4cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb4cff9700 1 -- 172.21.15.46:0/3506282695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6009e040 con 0x55d697db4570 2024-03-18T20:42:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb4cff9700 1 -- 172.21.15.46:0/3506282695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7ffb600b9310 con 0x55d697db4570 2024-03-18T20:42:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb77553b80 1 -- 172.21.15.46:0/3506282695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb54011c30 con 0x55d697db4570 2024-03-18T20:42:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb4cff9700 1 -- 172.21.15.46:0/3506282695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb60099450 con 0x55d697db4570 2024-03-18T20:42:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb77553b80 10 client.17715 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.646+0000 7ffb77553b80 1 -- 172.21.15.46:0/3506282695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55d697b8dc70 con 0x55d697db4570 2024-03-18T20:42:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.647+0000 7ffb4cff9700 1 -- 172.21.15.46:0/3506282695 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb54011c30 con 0x55d697db4570 2024-03-18T20:42:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.647+0000 7ffb4cff9700 1 -- 172.21.15.46:0/3506282695 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ffb6007b070 con 0x55d697db4570 2024-03-18T20:42:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.647+0000 7ffb77553b80 10 client.17715 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.647+0000 7ffb77553b80 10 client.17715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.647+0000 7ffb77553b80 20 client.17715 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.647+0000 7ffb77553b80 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d697e28620 0x55d697e2aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.647+0000 7ffb77553b80 4 client.17715 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:42:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.647+0000 7ffb77553b80 1 -- 172.21.15.46:0/3506282695 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d697b56c90 con 0x55d697e28620 2024-03-18T20:42:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.647+0000 7ffb68959700 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d697e28620 0x55d697e2aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.647+0000 7ffb4cff9700 10 client.17715 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.647+0000 7ffb68959700 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d697e28620 0x55d697e2aa40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da2e08049f6c50bc server_cookie=1c565d7fe2cfe5b8 in_seq=0 out_seq=0 2024-03-18T20:42:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb4cff9700 1 -- 172.21.15.46:0/3506282695 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d697b56c90 con 0x55d697e28620 2024-03-18T20:42:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb4cff9700 10 client.17715 handle_command_reply: tid=0 2024-03-18T20:42:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb77553b80 1 client.17715 shutdown 2024-03-18T20:42:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb77553b80 1 -- 172.21.15.46:0/3506282695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 msgr2=0x55d697dbe370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb77553b80 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 0x55d697dbe370 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ffb60097b20 tx=0x7ffb600226c0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb77553b80 1 -- 172.21.15.46:0/3506282695 shutdown_connections 2024-03-18T20:42:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb77553b80 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d697e28620 0x55d697e2aa40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb77553b80 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697db4570 0x55d697dbe370 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb77553b80 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d697dc41c0 0x55d697dbd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb4cff9700 0 client.17715 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb77553b80 1 --2- 172.21.15.46:0/3506282695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d697db4e90 0x55d697dbe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb77553b80 1 -- 172.21.15.46:0/3506282695 >> 172.21.15.46:0/3506282695 conn(0x55d697d2b1c0 msgr2=0x55d697dbf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb77553b80 1 -- 172.21.15.46:0/3506282695 shutdown_connections 2024-03-18T20:42:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb77553b80 1 -- 172.21.15.46:0/3506282695 wait complete. 2024-03-18T20:42:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb77553b80 20 client.17715 trim_cache size 0 max 16384 2024-03-18T20:42:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:42:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:42:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:42:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:42:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb6b3be700 1 -- 172.21.15.46:0/2367299160 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ffb500a1310 msgr2=0x7ffb500a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb6b3be700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ffb500a1310 0x7ffb500a37d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005950 tx=0x7ffb540058e0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb6b3be700 1 -- 172.21.15.46:0/2367299160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb64100f10 msgr2=0x7ffb641971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:19.687 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb6b3be700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb64100f10 0x7ffb641971e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ffb60005770 tx=0x7ffb60009580 comp rx=0 tx=0).stop 2024-03-18T20:42:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.682+0000 7ffb6b3be700 1 -- 172.21.15.46:0/2367299160 shutdown_connections 2024-03-18T20:42:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.683+0000 7ffb6b3be700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ffb500a1310 0x7ffb500a37d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.683+0000 7ffb6b3be700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb64100f10 0x7ffb641971e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.683+0000 7ffb6b3be700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb6419bac0 0x7ffb6419bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.683+0000 7ffb6b3be700 1 --2- 172.21.15.46:0/2367299160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb64106030 0x7ffb64197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.683+0000 7ffb6b3be700 1 -- 172.21.15.46:0/2367299160 >> 172.21.15.46:0/2367299160 conn(0x7ffb640fc770 msgr2=0x7ffb640fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.683+0000 7ffb6b3be700 1 -- 172.21.15.46:0/2367299160 shutdown_connections 2024-03-18T20:42:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:19.683+0000 7ffb6b3be700 1 -- 172.21.15.46:0/2367299160 wait complete. 2024-03-18T20:42:19.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 203878 3.38 MB/sec execute 194 sec latency 2240.466 ms 2024-03-18T20:42:20.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:20 smithi073 ceph-mon[47384]: pgmap v190: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 2.4 MiB/s wr, 204 op/s 2024-03-18T20:42:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:20 smithi155 ceph-mon[48344]: pgmap v190: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 2.4 MiB/s wr, 204 op/s 2024-03-18T20:42:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:20 smithi046 ceph-mon[43495]: pgmap v190: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 2.4 MiB/s wr, 204 op/s 2024-03-18T20:42:20.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 204763 3.37 MB/sec execute 195 sec latency 2871.878 ms 2024-03-18T20:42:21.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 206282 3.41 MB/sec execute 196 sec latency 22.483 ms 2024-03-18T20:42:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:22 smithi073 ceph-mon[47384]: pgmap v191: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-03-18T20:42:22.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:22 smithi155 ceph-mon[48344]: pgmap v191: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-03-18T20:42:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:22 smithi046 ceph-mon[43495]: pgmap v191: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-03-18T20:42:22.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 207421 3.42 MB/sec execute 197 sec latency 336.389 ms 2024-03-18T20:42:23.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 207421 3.41 MB/sec execute 198 sec latency 1336.481 ms 2024-03-18T20:42:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:24 smithi073 ceph-mon[47384]: pgmap v192: 97 pgs: 97 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 2.8 MiB/s wr, 221 op/s 2024-03-18T20:42:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:24 smithi155 ceph-mon[48344]: pgmap v192: 97 pgs: 97 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 2.8 MiB/s wr, 221 op/s 2024-03-18T20:42:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:24 smithi046 ceph-mon[43495]: pgmap v192: 97 pgs: 97 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 2.8 MiB/s wr, 221 op/s 2024-03-18T20:42:24.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 207421 3.39 MB/sec execute 199 sec latency 2336.577 ms 2024-03-18T20:42:25.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 209392 3.43 MB/sec execute 200 sec latency 2496.733 ms 2024-03-18T20:42:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:25 smithi073 ceph-mon[47384]: pgmap v193: 97 pgs: 97 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-03-18T20:42:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:25 smithi155 ceph-mon[48344]: pgmap v193: 97 pgs: 97 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-03-18T20:42:26.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:25 smithi046 ceph-mon[43495]: pgmap v193: 97 pgs: 97 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-03-18T20:42:26.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 210688 3.45 MB/sec execute 201 sec latency 17.515 ms 2024-03-18T20:42:27.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 210970 3.43 MB/sec execute 202 sec latency 834.867 ms 2024-03-18T20:42:28.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:28 smithi073 ceph-mon[47384]: pgmap v194: 97 pgs: 97 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-03-18T20:42:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:28 smithi155 ceph-mon[48344]: pgmap v194: 97 pgs: 97 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-03-18T20:42:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:28 smithi046 ceph-mon[43495]: pgmap v194: 97 pgs: 97 active+clean; 82 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-03-18T20:42:28.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 210970 3.42 MB/sec execute 203 sec latency 1834.935 ms 2024-03-18T20:42:29.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 210970 3.40 MB/sec execute 204 sec latency 2835.047 ms 2024-03-18T20:42:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:30 smithi073 ceph-mon[47384]: pgmap v195: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.4 MiB/s wr, 216 op/s 2024-03-18T20:42:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:30 smithi155 ceph-mon[48344]: pgmap v195: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.4 MiB/s wr, 216 op/s 2024-03-18T20:42:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:30 smithi046 ceph-mon[43495]: pgmap v195: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 2.4 MiB/s wr, 216 op/s 2024-03-18T20:42:30.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 211860 3.40 MB/sec execute 205 sec latency 3461.501 ms 2024-03-18T20:42:31.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 213302 3.43 MB/sec execute 206 sec latency 22.705 ms 2024-03-18T20:42:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:32 smithi073 ceph-mon[47384]: pgmap v196: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.6 MiB/s wr, 146 op/s 2024-03-18T20:42:32.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:42:32.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:42:32.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:32 smithi155 ceph-mon[48344]: pgmap v196: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.6 MiB/s wr, 146 op/s 2024-03-18T20:42:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:32 smithi046 ceph-mon[43495]: pgmap v196: 97 pgs: 97 active+clean; 84 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.6 MiB/s wr, 146 op/s 2024-03-18T20:42:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.667+0000 7faa08b86700 1 -- 172.21.15.46:0/3033716222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa04105ce0 msgr2=0x7faa041060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.667+0000 7faa08b86700 1 --2- 172.21.15.46:0/3033716222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa04105ce0 0x7faa041060c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009990 tx=0x7fa9f8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.668+0000 7faa08b86700 1 -- 172.21.15.46:0/3033716222 shutdown_connections 2024-03-18T20:42:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.668+0000 7faa08b86700 1 --2- 172.21.15.46:0/3033716222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa04105ce0 0x7faa041060c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.668+0000 7faa08b86700 1 --2- 172.21.15.46:0/3033716222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa040687f0 0x7faa0410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.668+0000 7faa08b86700 1 --2- 172.21.15.46:0/3033716222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa04067e30 0x7faa040682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.668+0000 7faa08b86700 1 -- 172.21.15.46:0/3033716222 >> 172.21.15.46:0/3033716222 conn(0x7faa04076d40 msgr2=0x7faa04077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.668+0000 7faa08b86700 1 -- 172.21.15.46:0/3033716222 shutdown_connections 2024-03-18T20:42:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.668+0000 7faa08b86700 1 -- 172.21.15.46:0/3033716222 wait complete. 2024-03-18T20:42:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.668+0000 7faa08b86700 1 Processor -- start 2024-03-18T20:42:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa08b86700 1 -- start start 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa08b86700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa04067e30 0x7faa04197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa08b86700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa040687f0 0x7faa04197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa08b86700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa04105ce0 0x7faa0419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa08b86700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0419c160 con 0x7faa04067e30 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa08b86700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0419c2d0 con 0x7faa04105ce0 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa08b86700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0419c490 con 0x7faa040687f0 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa0259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa04067e30 0x7faa04197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa0259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa04067e30 0x7faa04197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59314/0 (socket says 172.21.15.46:59314) 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa0259c700 1 -- 172.21.15.46:0/3482979231 learned_addr learned my addr 172.21.15.46:0/3482979231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa01d9b700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa040687f0 0x7faa04197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa02d9d700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa04105ce0 0x7faa0419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa0259c700 1 -- 172.21.15.46:0/3482979231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa040687f0 msgr2=0x7faa04197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa0259c700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa040687f0 0x7faa04197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa0259c700 1 -- 172.21.15.46:0/3482979231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa04105ce0 msgr2=0x7faa0419bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa0259c700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa04105ce0 0x7faa0419bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa0259c700 1 -- 172.21.15.46:0/3482979231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f80095f0 con 0x7faa04067e30 2024-03-18T20:42:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.669+0000 7faa0259c700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa04067e30 0x7faa04197240 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8005910 tx=0x7fa9f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.670+0000 7fa9f37fe700 1 -- 172.21.15.46:0/3482979231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f801d070 con 0x7faa04067e30 2024-03-18T20:42:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.670+0000 7faa08b86700 1 -- 172.21.15.46:0/3482979231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0419c710 con 0x7faa04067e30 2024-03-18T20:42:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.670+0000 7fa9f37fe700 1 -- 172.21.15.46:0/3482979231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f8022470 con 0x7faa04067e30 2024-03-18T20:42:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.670+0000 7fa9f37fe700 1 -- 172.21.15.46:0/3482979231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8017650 con 0x7faa04067e30 2024-03-18T20:42:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.670+0000 7faa08b86700 1 -- 172.21.15.46:0/3482979231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa041a3fb0 con 0x7faa04067e30 2024-03-18T20:42:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.672+0000 7fa9f37fe700 1 -- 172.21.15.46:0/3482979231 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa9f80225e0 con 0x7faa04067e30 2024-03-18T20:42:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.673+0000 7fa9f37fe700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9ec0a1290 0x7fa9ec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.673+0000 7faa01d9b700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9ec0a1290 0x7fa9ec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.673+0000 7fa9f37fe700 1 -- 172.21.15.46:0/3482979231 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fa9f80cdf70 con 0x7faa04067e30 2024-03-18T20:42:32.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.673+0000 7faa01d9b700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9ec0a1290 0x7fa9ec0a3750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faa04198790 tx=0x7fa9f4009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa14d1bb80 1 Processor -- start 2024-03-18T20:42:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa14d1bb80 1 -- start start 2024-03-18T20:42:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa14d1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8c3a60 0x55f02f8c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa14d1bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa14d1bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8cf220 0x55f02f8d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa14d1bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02f81d990 con 0x55f02f8c4380 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa14d1bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02f81b350 con 0x55f02f8cf220 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa14d1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02f816c40 con 0x55f02f8c3a60 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa01d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa01d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8cece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59322/0 (socket says 172.21.15.46:59322) 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa01d9b700 1 -- 172.21.15.46:0/1649294568 learned_addr learned my addr 172.21.15.46:0/1649294568 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa0259c700 1 --2- 172.21.15.46:0/1649294568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8c3a60 0x55f02f8c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.674+0000 7faa02d9d700 1 --2- 172.21.15.46:0/1649294568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8cf220 0x55f02f8d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.675+0000 7faa01d9b700 1 -- 172.21.15.46:0/1649294568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8c3a60 msgr2=0x55f02f8c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.675+0000 7faa01d9b700 1 --2- 172.21.15.46:0/1649294568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8c3a60 0x55f02f8c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.675+0000 7faa01d9b700 1 -- 172.21.15.46:0/1649294568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8cf220 msgr2=0x55f02f8d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.675+0000 7faa01d9b700 1 --2- 172.21.15.46:0/1649294568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8cf220 0x55f02f8d1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.675+0000 7faa01d9b700 1 -- 172.21.15.46:0/1649294568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f02f6983b0 con 0x55f02f8c4380 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.675+0000 7faa01d9b700 1 --2- 172.21.15.46:0/1649294568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8cece0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4011d50 tx=0x7fa9f4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729644d884ac6ddd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.675+0000 7fa9f0ff9700 1 -- 172.21.15.46:0/1649294568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f400eb20 con 0x55f02f8c4380 2024-03-18T20:42:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.675+0000 7fa9f0ff9700 1 -- 172.21.15.46:0/1649294568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f4021070 con 0x55f02f8c4380 2024-03-18T20:42:32.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.675+0000 7fa9f0ff9700 1 -- 172.21.15.46:0/1649294568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f401d670 con 0x55f02f8c4380 2024-03-18T20:42:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.676+0000 7faa14d1bb80 1 -- 172.21.15.46:0/1649294568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 msgr2=0x55f02f8cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.676+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/1649294568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8cece0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4011d50 tx=0x7fa9f4014000 comp rx=0 tx=0).stop 2024-03-18T20:42:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.676+0000 7faa14d1bb80 1 -- 172.21.15.46:0/1649294568 shutdown_connections 2024-03-18T20:42:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.676+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/1649294568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8cf220 0x55f02f8d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.676+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/1649294568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8c3a60 0x55f02f8c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.676+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/1649294568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8cece0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.676+0000 7faa14d1bb80 1 -- 172.21.15.46:0/1649294568 >> 172.21.15.46:0/1649294568 conn(0x55f02f83a220 msgr2=0x55f02f8c0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.676+0000 7faa14d1bb80 1 -- 172.21.15.46:0/1649294568 shutdown_connections 2024-03-18T20:42:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.676+0000 7faa14d1bb80 1 -- 172.21.15.46:0/1649294568 wait complete. 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.676+0000 7faa14d1bb80 1 -- start start 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.676+0000 7faa14d1bb80 1 Processor -- start 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa14d1bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c3a60 0x55f02f8cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa14d1bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa14d1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa14d1bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02f7b2d80 con 0x55f02f8c4380 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa14d1bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02f7b2a20 con 0x55f02f8c3a60 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa14d1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02f69a6a0 con 0x55f02f8cf220 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa14d1bb80 10 client.0 _ll_register_callbacks cb 0x55f02f61e3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa01d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa0259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c3a60 0x55f02f8cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa01d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59332/0 (socket says 172.21.15.46:59332) 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa01d9b700 1 -- 172.21.15.46:0/4063264378 learned_addr learned my addr 172.21.15.46:0/4063264378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:32.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa02d9d700 1 --2- 172.21.15.46:0/4063264378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa0259c700 1 -- 172.21.15.46:0/4063264378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 msgr2=0x55f02f8cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa0259c700 1 --2- 172.21.15.46:0/4063264378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa0259c700 1 -- 172.21.15.46:0/4063264378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 msgr2=0x55f02f8cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa0259c700 1 --2- 172.21.15.46:0/4063264378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa0259c700 1 -- 172.21.15.46:0/4063264378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f4017070 con 0x55f02f8c3a60 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7faa0259c700 1 --2- 172.21.15.46:0/4063264378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c3a60 0x55f02f8cecc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80cb690 tx=0x7fa9f8022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51df209b58305a07 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.677+0000 7fa9e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7fa9e9ffb700 1 -- 172.21.15.46:0/4063264378 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80b2070 con 0x55f02f8c3a60 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7fa9e9ffb700 1 -- 172.21.15.46:0/4063264378 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fa9f809b7e0 con 0x55f02f8c3a60 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7faa14d1bb80 1 -- 172.21.15.46:0/4063264378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f02f8b7ad0 con 0x55f02f8c3a60 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7fa9e9ffb700 1 -- 172.21.15.46:0/4063264378 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9f809b0e0 con 0x55f02f8c3a60 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7faa14d1bb80 10 client.27407 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7faa14d1bb80 1 -- 172.21.15.46:0/4063264378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f02f644b60 con 0x55f02f8c3a60 2024-03-18T20:42:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7fa9e9ffb700 1 -- 172.21.15.46:0/4063264378 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f02f8b7ad0 con 0x55f02f8c3a60 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7fa9e9ffb700 1 -- 172.21.15.46:0/4063264378 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa9f809a5d0 con 0x55f02f8c3a60 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7faa14d1bb80 10 client.27407 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7faa14d1bb80 10 client.27407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7faa14d1bb80 20 client.27407 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/4063264378 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f02f8dea80 0x55f02f8e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7faa14d1bb80 4 client.27407 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7faa14d1bb80 1 -- 172.21.15.46:0/4063264378 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f02f80f700 con 0x55f02f8dea80 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.678+0000 7faa01d9b700 1 --2- 172.21.15.46:0/4063264378 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f02f8dea80 0x55f02f8e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.679+0000 7fa9e9ffb700 10 client.27407 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.679+0000 7faa01d9b700 1 --2- 172.21.15.46:0/4063264378 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f02f8dea80 0x55f02f8e0ea0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=313504555a8bb6fd server_cookie=4fe6d5fa383c2e1 in_seq=0 out_seq=0 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.679+0000 7fa9e9ffb700 1 -- 172.21.15.46:0/4063264378 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f02f80f700 con 0x55f02f8dea80 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.679+0000 7fa9e9ffb700 10 client.27407 handle_command_reply: tid=0 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.679+0000 7faa14d1bb80 1 client.27407 shutdown 2024-03-18T20:42:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.680+0000 7faa14d1bb80 1 -- 172.21.15.46:0/4063264378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c3a60 msgr2=0x55f02f8cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.680+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/4063264378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c3a60 0x55f02f8cecc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80cb690 tx=0x7fa9f8022c90 comp rx=0 tx=0).stop 2024-03-18T20:42:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.680+0000 7faa14d1bb80 1 -- 172.21.15.46:0/4063264378 shutdown_connections 2024-03-18T20:42:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.680+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/4063264378 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f02f8dea80 0x55f02f8e0ea0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.680+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/4063264378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.680+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/4063264378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c3a60 0x55f02f8cecc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.680+0000 7fa9e9ffb700 0 client.27407 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.680+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/4063264378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.680+0000 7faa14d1bb80 1 -- 172.21.15.46:0/4063264378 >> 172.21.15.46:0/4063264378 conn(0x55f02f83a220 msgr2=0x55f02f8bfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.680+0000 7faa14d1bb80 1 -- 172.21.15.46:0/4063264378 shutdown_connections 2024-03-18T20:42:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.680+0000 7faa14d1bb80 1 -- 172.21.15.46:0/4063264378 wait complete. 2024-03-18T20:42:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.680+0000 7faa14d1bb80 20 client.27407 trim_cache size 0 max 16384 2024-03-18T20:42:32.695 INFO:tasks.workunit.client.0.smithi046.stdout: 1 214512 3.44 MB/sec execute 207 sec latency 177.758 ms 2024-03-18T20:42:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa14d1bb80 1 Processor -- start 2024-03-18T20:42:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa14d1bb80 1 -- start start 2024-03-18T20:42:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa14d1bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c3a60 0x55f02f8cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa14d1bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa14d1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa14d1bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02f8c1230 con 0x55f02f8c4380 2024-03-18T20:42:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa14d1bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02f8b9f80 con 0x55f02f8c3a60 2024-03-18T20:42:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa14d1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02f8d6390 con 0x55f02f8cf220 2024-03-18T20:42:32.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa01d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa01d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8d5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59340/0 (socket says 172.21.15.46:59340) 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa01d9b700 1 -- 172.21.15.46:0/934214265 learned_addr learned my addr 172.21.15.46:0/934214265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa02d9d700 1 --2- 172.21.15.46:0/934214265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.698+0000 7faa0259c700 1 --2- 172.21.15.46:0/934214265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c3a60 0x55f02f8cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.699+0000 7faa01d9b700 1 -- 172.21.15.46:0/934214265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 msgr2=0x55f02f8c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.699+0000 7faa01d9b700 1 --2- 172.21.15.46:0/934214265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.699+0000 7faa01d9b700 1 -- 172.21.15.46:0/934214265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c3a60 msgr2=0x55f02f8cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.699+0000 7faa01d9b700 1 --2- 172.21.15.46:0/934214265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c3a60 0x55f02f8cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.699+0000 7faa01d9b700 1 -- 172.21.15.46:0/934214265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f02f7dcea0 con 0x55f02f8c4380 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.699+0000 7faa01d9b700 1 --2- 172.21.15.46:0/934214265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8d5e50 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4011b30 tx=0x7fa9f401cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ea481cd0a065e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.699+0000 7fa9f17fa700 1 -- 172.21.15.46:0/934214265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4011cd0 con 0x55f02f8c4380 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.699+0000 7fa9f17fa700 1 -- 172.21.15.46:0/934214265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f401cd00 con 0x55f02f8c4380 2024-03-18T20:42:32.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.699+0000 7fa9f17fa700 1 -- 172.21.15.46:0/934214265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40333d0 con 0x55f02f8c4380 2024-03-18T20:42:32.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.700+0000 7faa14d1bb80 1 -- 172.21.15.46:0/934214265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 msgr2=0x55f02f8d5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.700+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/934214265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8d5e50 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4011b30 tx=0x7fa9f401cb60 comp rx=0 tx=0).stop 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.700+0000 7faa14d1bb80 1 -- 172.21.15.46:0/934214265 shutdown_connections 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.700+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/934214265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.700+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/934214265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c3a60 0x55f02f8cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.700+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/934214265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c4380 0x55f02f8d5e50 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.700+0000 7faa14d1bb80 1 -- 172.21.15.46:0/934214265 >> 172.21.15.46:0/934214265 conn(0x55f02f83a220 msgr2=0x55f02f8ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.700+0000 7faa14d1bb80 1 -- 172.21.15.46:0/934214265 shutdown_connections 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.700+0000 7faa14d1bb80 1 -- 172.21.15.46:0/934214265 wait complete. 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.700+0000 7faa14d1bb80 1 -- start start 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.700+0000 7faa14d1bb80 1 Processor -- start 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa14d1bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c3a60 0x55f02f8d5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa14d1bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c4380 0x55f02f8d3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa14d1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8d43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa14d1bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02f8d4920 con 0x55f02f8c3a60 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa14d1bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02f8d4a60 con 0x55f02f8c4380 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa14d1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02f8d4bf0 con 0x55f02f8cf220 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa14d1bb80 10 client.0 _ll_register_callbacks cb 0x7fa9f8097420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa0259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c3a60 0x55f02f8d5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa0259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c3a60 0x55f02f8d5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59346/0 (socket says 172.21.15.46:59346) 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa0259c700 1 -- 172.21.15.46:0/891815380 learned_addr learned my addr 172.21.15.46:0/891815380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa02d9d700 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8d43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa01d9b700 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c4380 0x55f02f8d3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa0259c700 1 -- 172.21.15.46:0/891815380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 msgr2=0x55f02f8d43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa0259c700 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8d43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa0259c700 1 -- 172.21.15.46:0/891815380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c4380 msgr2=0x55f02f8d3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa0259c700 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c4380 0x55f02f8d3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa0259c700 1 -- 172.21.15.46:0/891815380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f402a040 con 0x55f02f8c3a60 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa01d9b700 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c4380 0x55f02f8d3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa02d9d700 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8d43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7faa0259c700 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c3a60 0x55f02f8d5ae0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800f740 tx=0x7fa9f800b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba80696f9f1596df server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.701+0000 7fa9e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7fa9e9ffb700 1 -- 172.21.15.46:0/891815380 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80b2070 con 0x55f02f8c3a60 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7fa9e9ffb700 1 -- 172.21.15.46:0/891815380 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fa9f809b7e0 con 0x55f02f8c3a60 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7faa14d1bb80 1 -- 172.21.15.46:0/891815380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9f4021070 con 0x55f02f8c3a60 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7fa9e9ffb700 1 -- 172.21.15.46:0/891815380 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9f8063070 con 0x55f02f8c3a60 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7faa14d1bb80 10 client.17733 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7faa14d1bb80 1 -- 172.21.15.46:0/891815380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f02f69ccd0 con 0x55f02f8c3a60 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7fa9e9ffb700 1 -- 172.21.15.46:0/891815380 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f4021070 con 0x55f02f8c3a60 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7fa9e9ffb700 1 -- 172.21.15.46:0/891815380 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa9f80b9290 con 0x55f02f8c3a60 2024-03-18T20:42:32.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7faa14d1bb80 10 client.17733 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7faa14d1bb80 10 client.17733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7faa14d1bb80 20 client.17733 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f02f933940 0x55f02f935d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7faa14d1bb80 4 client.17733 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7faa14d1bb80 1 -- 172.21.15.46:0/891815380 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f02f665cf0 con 0x55f02f933940 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.702+0000 7faa01d9b700 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f02f933940 0x55f02f935d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7faa01d9b700 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f02f933940 0x55f02f935d30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c413cf7cd0ea091a server_cookie=208d32e5a9dccb87 in_seq=0 out_seq=0 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7fa9e9ffb700 10 client.17733 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7fa9e9ffb700 1 -- 172.21.15.46:0/891815380 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f02f665cf0 con 0x55f02f933940 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7fa9e9ffb700 10 client.17733 handle_command_reply: tid=0 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7faa14d1bb80 1 client.17733 shutdown 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7faa14d1bb80 1 -- 172.21.15.46:0/891815380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c3a60 msgr2=0x55f02f8d5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c3a60 0x55f02f8d5ae0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800f740 tx=0x7fa9f800b6b0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7faa14d1bb80 1 -- 172.21.15.46:0/891815380 shutdown_connections 2024-03-18T20:42:32.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f02f933940 0x55f02f935d30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f02f8c3a60 0x55f02f8d5ae0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f02f8cf220 0x55f02f8d43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7faa14d1bb80 1 --2- 172.21.15.46:0/891815380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f02f8c4380 0x55f02f8d3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7fa9e9ffb700 0 client.17733 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7faa14d1bb80 1 -- 172.21.15.46:0/891815380 >> 172.21.15.46:0/891815380 conn(0x55f02f83a220 msgr2=0x55f02f8bfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7faa14d1bb80 1 -- 172.21.15.46:0/891815380 shutdown_connections 2024-03-18T20:42:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.703+0000 7faa14d1bb80 1 -- 172.21.15.46:0/891815380 wait complete. 2024-03-18T20:42:32.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa14d1bb80 20 client.17733 trim_cache size 0 max 16384 2024-03-18T20:42:32.712 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:42:32.712 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:42:32.713 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:42:32.713 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:42:32.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa08b86700 1 -- 172.21.15.46:0/3482979231 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9ec0a1290 msgr2=0x7fa9ec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa08b86700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9ec0a1290 0x7fa9ec0a3750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faa04198790 tx=0x7fa9f4009260 comp rx=0 tx=0).stop 2024-03-18T20:42:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa08b86700 1 -- 172.21.15.46:0/3482979231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa04067e30 msgr2=0x7faa04197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa08b86700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa04067e30 0x7faa04197240 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8005910 tx=0x7fa9f8004730 comp rx=0 tx=0).stop 2024-03-18T20:42:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa08b86700 1 -- 172.21.15.46:0/3482979231 shutdown_connections 2024-03-18T20:42:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa08b86700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9ec0a1290 0x7fa9ec0a3750 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa08b86700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa04067e30 0x7faa04197240 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa08b86700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa04105ce0 0x7faa0419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa08b86700 1 --2- 172.21.15.46:0/3482979231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa040687f0 0x7faa04197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa08b86700 1 -- 172.21.15.46:0/3482979231 >> 172.21.15.46:0/3482979231 conn(0x7faa04076d40 msgr2=0x7faa040fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa08b86700 1 -- 172.21.15.46:0/3482979231 shutdown_connections 2024-03-18T20:42:32.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.704+0000 7faa08b86700 1 -- 172.21.15.46:0/3482979231 wait complete. 2024-03-18T20:42:32.721 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:42:32.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:42:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.810+0000 7f16e32fb700 1 -- 172.21.15.46:0/2917510876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc103110 msgr2=0x7f16dc1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.810+0000 7f16e32fb700 1 --2- 172.21.15.46:0/2917510876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc103110 0x7f16dc1034f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009990 tx=0x7f16d8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.810+0000 7f16e32fb700 1 -- 172.21.15.46:0/2917510876 shutdown_connections 2024-03-18T20:42:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.810+0000 7f16e32fb700 1 --2- 172.21.15.46:0/2917510876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc103110 0x7f16dc1034f0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.810+0000 7f16e32fb700 1 --2- 172.21.15.46:0/2917510876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc108230 0x7f16dc10fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.810+0000 7f16e32fb700 1 --2- 172.21.15.46:0/2917510876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc103ac0 0x7f16dc107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.810+0000 7f16e32fb700 1 -- 172.21.15.46:0/2917510876 >> 172.21.15.46:0/2917510876 conn(0x7f16dc0fe930 msgr2=0x7f16dc100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.810+0000 7f16e32fb700 1 -- 172.21.15.46:0/2917510876 shutdown_connections 2024-03-18T20:42:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.810+0000 7f16e32fb700 1 -- 172.21.15.46:0/2917510876 wait complete. 2024-03-18T20:42:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.810+0000 7f16e32fb700 1 Processor -- start 2024-03-18T20:42:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e32fb700 1 -- start start 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e32fb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc103110 0x7f16dc199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e32fb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc108230 0x7f16dc1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e32fb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc19dd00 0x7f16dc19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e32fb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc19e780 con 0x7f16dc19dd00 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e32fb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc19e8f0 con 0x7f16dc108230 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e32fb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc19eab0 con 0x7f16dc103110 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e0896700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc108230 0x7f16dc1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e1898700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc19dd00 0x7f16dc19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e0896700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc108230 0x7f16dc1999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40164/0 (socket says 172.21.15.46:40164) 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e0896700 1 -- 172.21.15.46:0/4152936381 learned_addr learned my addr 172.21.15.46:0/4152936381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e1097700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc103110 0x7f16dc199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e0896700 1 -- 172.21.15.46:0/4152936381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc103110 msgr2=0x7f16dc199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e0896700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc103110 0x7f16dc199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e0896700 1 -- 172.21.15.46:0/4152936381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc19dd00 msgr2=0x7f16dc19e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e0896700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc19dd00 0x7f16dc19e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e0896700 1 -- 172.21.15.46:0/4152936381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d80095f0 con 0x7f16dc108230 2024-03-18T20:42:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.811+0000 7f16e1898700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc19dd00 0x7f16dc19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.812+0000 7f16e0896700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc108230 0x7f16dc1999b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00d950 tx=0x7f16cc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.812+0000 7f16d27fc700 1 -- 172.21.15.46:0/4152936381 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc004220 con 0x7f16dc108230 2024-03-18T20:42:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.812+0000 7f16d27fc700 1 -- 172.21.15.46:0/4152936381 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16cc007620 con 0x7f16dc108230 2024-03-18T20:42:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.812+0000 7f16e32fb700 1 -- 172.21.15.46:0/4152936381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16dc1a5fb0 con 0x7f16dc108230 2024-03-18T20:42:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.812+0000 7f16d27fc700 1 -- 172.21.15.46:0/4152936381 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc0056c0 con 0x7f16dc108230 2024-03-18T20:42:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.812+0000 7f16e32fb700 1 -- 172.21.15.46:0/4152936381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc1a6480 con 0x7f16dc108230 2024-03-18T20:42:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.815+0000 7f16d27fc700 1 -- 172.21.15.46:0/4152936381 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f16cc004380 con 0x7f16dc108230 2024-03-18T20:42:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.815+0000 7f16d27fc700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f16c80a1290 0x7f16c80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.816+0000 7f16d27fc700 1 -- 172.21.15.46:0/4152936381 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f16cc0cca10 con 0x7f16dc108230 2024-03-18T20:42:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.816+0000 7f16e1097700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f16c80a1290 0x7f16c80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.816+0000 7f16e1097700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f16c80a1290 0x7f16c80a3750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f16d800b2e0 tx=0x7f16d8005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16ef490b80 1 Processor -- start 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16ef490b80 1 -- start start 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16ef490b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 0x556500e48950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16ef490b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e48e90 0x556500e53870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16ef490b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e53db0 0x556500e561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16ef490b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556500da2930 con 0x556500e48570 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16ef490b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556500da02f0 con 0x556500e53db0 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16ef490b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556500d9bbe0 con 0x556500e48e90 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16e1097700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 0x556500e48950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16e1097700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 0x556500e48950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59386/0 (socket says 172.21.15.46:59386) 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16e1097700 1 -- 172.21.15.46:0/1980319547 learned_addr learned my addr 172.21.15.46:0/1980319547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16e1898700 1 --2- 172.21.15.46:0/1980319547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e53db0 0x556500e561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16e0896700 1 --2- 172.21.15.46:0/1980319547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e48e90 0x556500e53870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16e1097700 1 -- 172.21.15.46:0/1980319547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e48e90 msgr2=0x556500e53870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16e1097700 1 --2- 172.21.15.46:0/1980319547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e48e90 0x556500e53870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16e1097700 1 -- 172.21.15.46:0/1980319547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e53db0 msgr2=0x556500e561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16e1097700 1 --2- 172.21.15.46:0/1980319547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e53db0 0x556500e561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.817+0000 7f16e1097700 1 -- 172.21.15.46:0/1980319547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556500c1d350 con 0x556500e48570 2024-03-18T20:42:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.818+0000 7f16e1097700 1 --2- 172.21.15.46:0/1980319547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 0x556500e48950 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f16d8015000 tx=0x7f16d8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b8a5c0ba336cc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.818+0000 7f16c77fe700 1 -- 172.21.15.46:0/1980319547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8004700 con 0x556500e48570 2024-03-18T20:42:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.818+0000 7f16c77fe700 1 -- 172.21.15.46:0/1980319547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d801e040 con 0x556500e48570 2024-03-18T20:42:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.818+0000 7f16c77fe700 1 -- 172.21.15.46:0/1980319547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d800f960 con 0x556500e48570 2024-03-18T20:42:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.819+0000 7f16ef490b80 1 -- 172.21.15.46:0/1980319547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 msgr2=0x556500e48950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.819+0000 7f16ef490b80 1 --2- 172.21.15.46:0/1980319547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 0x556500e48950 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f16d8015000 tx=0x7f16d8020000 comp rx=0 tx=0).stop 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.819+0000 7f16ef490b80 1 -- 172.21.15.46:0/1980319547 shutdown_connections 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.819+0000 7f16ef490b80 1 --2- 172.21.15.46:0/1980319547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 0x556500e48950 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.819+0000 7f16ef490b80 1 --2- 172.21.15.46:0/1980319547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e53db0 0x556500e561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.819+0000 7f16ef490b80 1 --2- 172.21.15.46:0/1980319547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e48e90 0x556500e53870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.819+0000 7f16ef490b80 1 -- 172.21.15.46:0/1980319547 >> 172.21.15.46:0/1980319547 conn(0x556500dbf1c0 msgr2=0x556500e44a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.819+0000 7f16ef490b80 1 -- 172.21.15.46:0/1980319547 shutdown_connections 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.819+0000 7f16ef490b80 1 -- 172.21.15.46:0/1980319547 wait complete. 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.819+0000 7f16ef490b80 1 -- start start 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.819+0000 7f16ef490b80 1 Processor -- start 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16ef490b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 0x556500e537d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16ef490b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48e90 0x556500e50ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16ef490b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e53db0 0x556500e510f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16ef490b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556500d37d20 con 0x556500e48570 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16ef490b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556500d379c0 con 0x556500e48e90 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16ef490b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556500c1f640 con 0x556500e53db0 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16ef490b80 10 client.0 _ll_register_callbacks cb 0x556500ba3340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16e1097700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 0x556500e537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16e1097700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 0x556500e537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59388/0 (socket says 172.21.15.46:59388) 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16e1097700 1 -- 172.21.15.46:0/1064382861 learned_addr learned my addr 172.21.15.46:0/1064382861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16e1898700 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e53db0 0x556500e510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16e0896700 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48e90 0x556500e50ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16e1097700 1 -- 172.21.15.46:0/1064382861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e53db0 msgr2=0x556500e510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16e1097700 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e53db0 0x556500e510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16e1097700 1 -- 172.21.15.46:0/1064382861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48e90 msgr2=0x556500e50ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.820+0000 7f16e1097700 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48e90 0x556500e50ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.821+0000 7f16e1097700 1 -- 172.21.15.46:0/1064382861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc110620 con 0x556500e48570 2024-03-18T20:42:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.821+0000 7f16e1097700 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 0x556500e537d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009ce0 tx=0x7f16d80057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8da0214fe48dd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.821+0000 7f16c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.821+0000 7f16c4ff9700 1 -- 172.21.15.46:0/1064382861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8004700 con 0x556500e48570 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.821+0000 7f16c4ff9700 1 -- 172.21.15.46:0/1064382861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f16d800be60 con 0x556500e48570 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.821+0000 7f16ef490b80 1 -- 172.21.15.46:0/1064382861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556500e3c6c0 con 0x556500e48570 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.821+0000 7f16c4ff9700 1 -- 172.21.15.46:0/1064382861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d8004860 con 0x556500e48570 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.821+0000 7f16ef490b80 10 client.17745 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.821+0000 7f16ef490b80 1 -- 172.21.15.46:0/1064382861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x556500bc9b00 con 0x556500e48570 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.821+0000 7f16c4ff9700 1 -- 172.21.15.46:0/1064382861 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d8047020 con 0x556500e48570 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.822+0000 7f16c4ff9700 1 -- 172.21.15.46:0/1064382861 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f16d8043aa0 con 0x556500e48570 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.822+0000 7f16ef490b80 10 client.17745 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.822+0000 7f16ef490b80 10 client.17745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.822+0000 7f16ef490b80 20 client.17745 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.822+0000 7f16ef490b80 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556500e63480 0x556500e658a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.822+0000 7f16ef490b80 4 client.17745 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.822+0000 7f16ef490b80 1 -- 172.21.15.46:0/1064382861 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556500d946a0 con 0x556500e63480 2024-03-18T20:42:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.822+0000 7f16e0896700 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556500e63480 0x556500e658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.822+0000 7f16c4ff9700 10 client.17745 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.822+0000 7f16e0896700 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556500e63480 0x556500e658a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faba0531aa71e9c6 server_cookie=a1411f5e6fe2ddc4 in_seq=0 out_seq=0 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16c4ff9700 1 -- 172.21.15.46:0/1064382861 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556500d946a0 con 0x556500e63480 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16c4ff9700 10 client.17745 handle_command_reply: tid=0 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16ef490b80 1 client.17745 shutdown 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16ef490b80 1 -- 172.21.15.46:0/1064382861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 msgr2=0x556500e537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16ef490b80 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 0x556500e537d0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009ce0 tx=0x7f16d80057e0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16ef490b80 1 -- 172.21.15.46:0/1064382861 shutdown_connections 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16ef490b80 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556500e63480 0x556500e658a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16ef490b80 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48570 0x556500e537d0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16c4ff9700 0 client.17745 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16ef490b80 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e53db0 0x556500e510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16ef490b80 1 --2- 172.21.15.46:0/1064382861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48e90 0x556500e50ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16ef490b80 1 -- 172.21.15.46:0/1064382861 >> 172.21.15.46:0/1064382861 conn(0x556500dbf1c0 msgr2=0x556500e44650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16ef490b80 1 -- 172.21.15.46:0/1064382861 shutdown_connections 2024-03-18T20:42:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16ef490b80 1 -- 172.21.15.46:0/1064382861 wait complete. 2024-03-18T20:42:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.823+0000 7f16ef490b80 20 client.17745 trim_cache size 0 max 16384 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16ef490b80 1 Processor -- start 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16ef490b80 1 -- start start 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16ef490b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48570 0x556500e510f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16ef490b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48e90 0x556500e5a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16ef490b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e53db0 0x556500e4f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16ef490b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556500e516d0 con 0x556500e48e90 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16ef490b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556500e5af20 con 0x556500e48570 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16ef490b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556500e4f010 con 0x556500e53db0 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16e0896700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48e90 0x556500e5a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16e0896700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48e90 0x556500e5a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59404/0 (socket says 172.21.15.46:59404) 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16e0896700 1 -- 172.21.15.46:0/3736610232 learned_addr learned my addr 172.21.15.46:0/3736610232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16e1898700 1 --2- 172.21.15.46:0/3736610232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e53db0 0x556500e4f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.839+0000 7f16e1097700 1 --2- 172.21.15.46:0/3736610232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48570 0x556500e510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.840+0000 7f16e0896700 1 -- 172.21.15.46:0/3736610232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e53db0 msgr2=0x556500e4f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.840+0000 7f16e0896700 1 --2- 172.21.15.46:0/3736610232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e53db0 0x556500e4f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.840+0000 7f16e0896700 1 -- 172.21.15.46:0/3736610232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48570 msgr2=0x556500e510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.840+0000 7f16e0896700 1 --2- 172.21.15.46:0/3736610232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48570 0x556500e510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.840+0000 7f16e0896700 1 -- 172.21.15.46:0/3736610232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556500d61e40 con 0x556500e48e90 2024-03-18T20:42:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.840+0000 7f16e0896700 1 --2- 172.21.15.46:0/3736610232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48e90 0x556500e5a9e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f16cc095ff0 tx=0x7f16cc099760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f874c21592c33a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.840+0000 7f16c7fff700 1 -- 172.21.15.46:0/3736610232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc09d920 con 0x556500e48e90 2024-03-18T20:42:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.840+0000 7f16c7fff700 1 -- 172.21.15.46:0/3736610232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16cc0a1f70 con 0x556500e48e90 2024-03-18T20:42:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.840+0000 7f16c7fff700 1 -- 172.21.15.46:0/3736610232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc099f50 con 0x556500e48e90 2024-03-18T20:42:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.841+0000 7f16ef490b80 1 -- 172.21.15.46:0/3736610232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48e90 msgr2=0x556500e5a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.841+0000 7f16ef490b80 1 --2- 172.21.15.46:0/3736610232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48e90 0x556500e5a9e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f16cc095ff0 tx=0x7f16cc099760 comp rx=0 tx=0).stop 2024-03-18T20:42:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.841+0000 7f16ef490b80 1 -- 172.21.15.46:0/3736610232 shutdown_connections 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.841+0000 7f16ef490b80 1 --2- 172.21.15.46:0/3736610232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e53db0 0x556500e4f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.841+0000 7f16ef490b80 1 --2- 172.21.15.46:0/3736610232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48570 0x556500e510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.841+0000 7f16ef490b80 1 --2- 172.21.15.46:0/3736610232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e48e90 0x556500e5a9e0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.841+0000 7f16ef490b80 1 -- 172.21.15.46:0/3736610232 >> 172.21.15.46:0/3736610232 conn(0x556500dbf1c0 msgr2=0x556500e43120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.841+0000 7f16ef490b80 1 -- 172.21.15.46:0/3736610232 shutdown_connections 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.841+0000 7f16ef490b80 1 -- 172.21.15.46:0/3736610232 wait complete. 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.841+0000 7f16ef490b80 1 -- start start 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.841+0000 7f16ef490b80 1 Processor -- start 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16ef490b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e48570 0x556500e569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16ef490b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48e90 0x556500e56ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16ef490b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e53db0 0x556500eb62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16ef490b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556500e57800 con 0x556500e53db0 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16ef490b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556500eb67e0 con 0x556500e48e90 2024-03-18T20:42:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16ef490b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556500eb6920 con 0x556500e48570 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16ef490b80 10 client.0 _ll_register_callbacks cb 0x7f16d4009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e1898700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e53db0 0x556500eb62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e1097700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e48570 0x556500e569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e1898700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e53db0 0x556500eb62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59416/0 (socket says 172.21.15.46:59416) 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e1097700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e48570 0x556500e569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:38010/0 (socket says 172.21.15.46:38010) 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e1898700 1 -- 172.21.15.46:0/2529591227 learned_addr learned my addr 172.21.15.46:0/2529591227 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e0896700 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48e90 0x556500e56ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e1898700 1 -- 172.21.15.46:0/2529591227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e48570 msgr2=0x556500e569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e1898700 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e48570 0x556500e569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e1898700 1 -- 172.21.15.46:0/2529591227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48e90 msgr2=0x556500e56ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e1898700 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48e90 0x556500e56ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e1898700 1 -- 172.21.15.46:0/2529591227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc1a6480 con 0x556500e53db0 2024-03-18T20:42:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e0896700 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48e90 0x556500e56ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.842+0000 7f16e1097700 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e48570 0x556500e569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16e1898700 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e53db0 0x556500eb62a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f16d4018700 tx=0x7f16d400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bbdd6c209c9c23 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16c4ff9700 1 -- 172.21.15.46:0/2529591227 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d401d070 con 0x556500e53db0 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16c4ff9700 1 -- 172.21.15.46:0/2529591227 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f16d4004bb0 con 0x556500e53db0 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16ef490b80 1 -- 172.21.15.46:0/2529591227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16d801e040 con 0x556500e53db0 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16c4ff9700 1 -- 172.21.15.46:0/2529591227 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d4004d70 con 0x556500e53db0 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16ef490b80 10 client.17751 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16ef490b80 1 -- 172.21.15.46:0/2529591227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x556500c21c70 con 0x556500e53db0 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16c4ff9700 1 -- 172.21.15.46:0/2529591227 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d801e040 con 0x556500e53db0 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16c4ff9700 1 -- 172.21.15.46:0/2529591227 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f16d4029070 con 0x556500e53db0 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16ef490b80 10 client.17751 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16ef490b80 10 client.17751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16ef490b80 20 client.17751 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16ef490b80 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556500eb8110 0x556500eba530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16ef490b80 4 client.17751 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:42:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.843+0000 7f16ef490b80 1 -- 172.21.15.46:0/2529591227 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556500beac90 con 0x556500eb8110 2024-03-18T20:42:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.844+0000 7f16e1097700 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556500eb8110 0x556500eba530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.844+0000 7f16e1097700 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556500eb8110 0x556500eba530 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18cfc5927e296ef3 server_cookie=1a2fae77780ff18e in_seq=0 out_seq=0 2024-03-18T20:42:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.844+0000 7f16c4ff9700 10 client.17751 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16c4ff9700 1 -- 172.21.15.46:0/2529591227 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556500beac90 con 0x556500eb8110 2024-03-18T20:42:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16c4ff9700 10 client.17751 handle_command_reply: tid=0 2024-03-18T20:42:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16ef490b80 1 client.17751 shutdown 2024-03-18T20:42:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16ef490b80 1 -- 172.21.15.46:0/2529591227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e53db0 msgr2=0x556500eb62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16ef490b80 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e53db0 0x556500eb62a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f16d4018700 tx=0x7f16d400efd0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16ef490b80 1 -- 172.21.15.46:0/2529591227 shutdown_connections 2024-03-18T20:42:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16ef490b80 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556500eb8110 0x556500eba530 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16ef490b80 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556500e53db0 0x556500eb62a0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16ef490b80 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556500e48570 0x556500e569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16ef490b80 1 --2- 172.21.15.46:0/2529591227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556500e48e90 0x556500e56ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16ef490b80 1 -- 172.21.15.46:0/2529591227 >> 172.21.15.46:0/2529591227 conn(0x556500dbf1c0 msgr2=0x556500e4f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16c4ff9700 0 client.17751 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16ef490b80 1 -- 172.21.15.46:0/2529591227 shutdown_connections 2024-03-18T20:42:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16ef490b80 1 -- 172.21.15.46:0/2529591227 wait complete. 2024-03-18T20:42:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.875+0000 7f16ef490b80 20 client.17751 trim_cache size 0 max 16384 2024-03-18T20:42:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:42:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:42:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:42:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:42:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.876+0000 7f16e32fb700 1 -- 172.21.15.46:0/4152936381 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f16c80a1290 msgr2=0x7f16c80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.876+0000 7f16e32fb700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f16c80a1290 0x7f16c80a3750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f16d800b2e0 tx=0x7f16d8005960 comp rx=0 tx=0).stop 2024-03-18T20:42:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.876+0000 7f16e32fb700 1 -- 172.21.15.46:0/4152936381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc108230 msgr2=0x7f16dc1999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.876+0000 7f16e32fb700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc108230 0x7f16dc1999b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00d950 tx=0x7f16cc00dc60 comp rx=0 tx=0).stop 2024-03-18T20:42:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.876+0000 7f16e32fb700 1 -- 172.21.15.46:0/4152936381 shutdown_connections 2024-03-18T20:42:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.876+0000 7f16e32fb700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f16c80a1290 0x7f16c80a3750 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.876+0000 7f16e32fb700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc19dd00 0x7f16dc19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.876+0000 7f16e32fb700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc103110 0x7f16dc199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.876+0000 7f16e32fb700 1 --2- 172.21.15.46:0/4152936381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc108230 0x7f16dc1999b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.876+0000 7f16e32fb700 1 -- 172.21.15.46:0/4152936381 >> 172.21.15.46:0/4152936381 conn(0x7f16dc0fe930 msgr2=0x7f16dc0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.876+0000 7f16e32fb700 1 -- 172.21.15.46:0/4152936381 shutdown_connections 2024-03-18T20:42:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:32.876+0000 7f16e32fb700 1 -- 172.21.15.46:0/4152936381 wait complete. 2024-03-18T20:42:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.910+0000 7f7e1d87d700 1 -- 172.21.15.46:0/4201794469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e181004a0 msgr2=0x7f7e1810c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.910+0000 7f7e1d87d700 1 --2- 172.21.15.46:0/4201794469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e181004a0 0x7f7e1810c9b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f7e1000b0a0 tx=0x7f7e1000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.910+0000 7f7e1d87d700 1 -- 172.21.15.46:0/4201794469 shutdown_connections 2024-03-18T20:42:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.910+0000 7f7e1d87d700 1 --2- 172.21.15.46:0/4201794469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e181004a0 0x7f7e1810c9b0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.910+0000 7f7e1d87d700 1 --2- 172.21.15.46:0/4201794469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e18105b80 0x7f7e18105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.910+0000 7f7e1d87d700 1 --2- 172.21.15.46:0/4201794469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e180ffae0 0x7f7e180fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.910+0000 7f7e1d87d700 1 -- 172.21.15.46:0/4201794469 >> 172.21.15.46:0/4201794469 conn(0x7f7e180fb5f0 msgr2=0x7f7e180fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:32.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.911+0000 7f7e1d87d700 1 -- 172.21.15.46:0/4201794469 shutdown_connections 2024-03-18T20:42:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.911+0000 7f7e1d87d700 1 -- 172.21.15.46:0/4201794469 wait complete. 2024-03-18T20:42:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.911+0000 7f7e1d87d700 1 Processor -- start 2024-03-18T20:42:32.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.911+0000 7f7e1d87d700 1 -- start start 2024-03-18T20:42:32.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e1d87d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e180ffae0 0x7f7e180fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e1d87d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e181004a0 0x7f7e180fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e1d87d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e18105b80 0x7f7e180fc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e1d87d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e181a53f0 con 0x7f7e181004a0 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e1d87d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e181a5560 con 0x7f7e180ffae0 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e1d87d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e181a5720 con 0x7f7e18105b80 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e16ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e180ffae0 0x7f7e180fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e167fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e181004a0 0x7f7e180fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e16ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e180ffae0 0x7f7e180fbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40222/0 (socket says 172.21.15.46:40222) 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e167fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e181004a0 0x7f7e180fc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59446/0 (socket says 172.21.15.46:59446) 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e16ffd700 1 -- 172.21.15.46:0/3121456954 learned_addr learned my addr 172.21.15.46:0/3121456954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e177fe700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e18105b80 0x7f7e180fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e16ffd700 1 -- 172.21.15.46:0/3121456954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e18105b80 msgr2=0x7f7e180fc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e16ffd700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e18105b80 0x7f7e180fc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e16ffd700 1 -- 172.21.15.46:0/3121456954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e181004a0 msgr2=0x7f7e180fc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e16ffd700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e181004a0 0x7f7e180fc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e16ffd700 1 -- 172.21.15.46:0/3121456954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10009d00 con 0x7f7e180ffae0 2024-03-18T20:42:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e16ffd700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e180ffae0 0x7f7e180fbb70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800e9a0 tx=0x7f7e0800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e1c87b700 1 -- 172.21.15.46:0/3121456954 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08009790 con 0x7f7e180ffae0 2024-03-18T20:42:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e1c87b700 1 -- 172.21.15.46:0/3121456954 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0800cbd0 con 0x7f7e180ffae0 2024-03-18T20:42:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e1d87d700 1 -- 172.21.15.46:0/3121456954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e181a5a00 con 0x7f7e180ffae0 2024-03-18T20:42:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.912+0000 7f7e1c87b700 1 -- 172.21.15.46:0/3121456954 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08018680 con 0x7f7e180ffae0 2024-03-18T20:42:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.913+0000 7f7e1d87d700 1 -- 172.21.15.46:0/3121456954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e181a5f10 con 0x7f7e180ffae0 2024-03-18T20:42:32.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.914+0000 7f7e1d87d700 1 -- 172.21.15.46:0/3121456954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e18109110 con 0x7f7e180ffae0 2024-03-18T20:42:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.915+0000 7f7e1c87b700 1 -- 172.21.15.46:0/3121456954 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7e0800cd40 con 0x7f7e180ffae0 2024-03-18T20:42:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.915+0000 7f7e1c87b700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e000a1290 0x7f7e000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.915+0000 7f7e167fc700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e000a1290 0x7f7e000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.915+0000 7f7e1c87b700 1 -- 172.21.15.46:0/3121456954 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f7e08014070 con 0x7f7e180ffae0 2024-03-18T20:42:32.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.916+0000 7f7e167fc700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e000a1290 0x7f7e000a3750 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e180fd2a0 tx=0x7f7e0c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:32.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:32.919+0000 7f7e1c87b700 1 -- 172.21.15.46:0/3121456954 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7e08096400 con 0x7f7e180ffae0 2024-03-18T20:42:33.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.197+0000 7f7e1d87d700 1 -- 172.21.15.46:0/3121456954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e181011e0 con 0x7f7e180ffae0 2024-03-18T20:42:33.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.198+0000 7f7e1c87b700 1 -- 172.21.15.46:0/3121456954 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+10919 (secure 0 0 0) 0x7f7e08099a20 con 0x7f7e180ffae0 2024-03-18T20:42:33.201 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:42:33.201 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:41:55.183335+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:42:33.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.200+0000 7f7e1d87d700 1 -- 172.21.15.46:0/3121456954 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e000a1290 msgr2=0x7f7e000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.200+0000 7f7e1d87d700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e000a1290 0x7f7e000a3750 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e180fd2a0 tx=0x7f7e0c009260 comp rx=0 tx=0).stop 2024-03-18T20:42:33.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.200+0000 7f7e1d87d700 1 -- 172.21.15.46:0/3121456954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e180ffae0 msgr2=0x7f7e180fbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.200+0000 7f7e1d87d700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e180ffae0 0x7f7e180fbb70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800e9a0 tx=0x7f7e0800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.200+0000 7f7e1d87d700 1 -- 172.21.15.46:0/3121456954 shutdown_connections 2024-03-18T20:42:33.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.200+0000 7f7e1d87d700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e000a1290 0x7f7e000a3750 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.200+0000 7f7e1d87d700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e18105b80 0x7f7e180fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.200+0000 7f7e1d87d700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e180ffae0 0x7f7e180fbb70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.200+0000 7f7e1d87d700 1 --2- 172.21.15.46:0/3121456954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e181004a0 0x7f7e180fc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.200+0000 7f7e1d87d700 1 -- 172.21.15.46:0/3121456954 >> 172.21.15.46:0/3121456954 conn(0x7f7e180fb5f0 msgr2=0x7f7e18069550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.200+0000 7f7e1d87d700 1 -- 172.21.15.46:0/3121456954 shutdown_connections 2024-03-18T20:42:33.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.200+0000 7f7e1d87d700 1 -- 172.21.15.46:0/3121456954 wait complete. 2024-03-18T20:42:33.204 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 43 2024-03-18T20:42:33.217 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:42:33.217 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:42:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.397+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/3068433221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b280ffe50 msgr2=0x7f6b28100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.397+0000 7f6b2ddf8700 1 --2- 172.21.15.46:0/3068433221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b280ffe50 0x7f6b28100230 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009990 tx=0x7f6b18009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.398+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/3068433221 shutdown_connections 2024-03-18T20:42:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.398+0000 7f6b2ddf8700 1 --2- 172.21.15.46:0/3068433221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b280ffe50 0x7f6b28100230 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.398+0000 7f6b2ddf8700 1 --2- 172.21.15.46:0/3068433221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b28104f60 0x7f6b2810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.398+0000 7f6b2ddf8700 1 --2- 172.21.15.46:0/3068433221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b28100800 0x7f6b28104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.398+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/3068433221 >> 172.21.15.46:0/3068433221 conn(0x7f6b28076d40 msgr2=0x7f6b28077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.398+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/3068433221 shutdown_connections 2024-03-18T20:42:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.398+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/3068433221 wait complete. 2024-03-18T20:42:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.398+0000 7f6b2ddf8700 1 Processor -- start 2024-03-18T20:42:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b2ddf8700 1 -- start start 2024-03-18T20:42:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b2ddf8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b280ffe50 0x7f6b281018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b2ddf8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b28104f60 0x7f6b28101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b2ddf8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b28102350 0x7f6b281a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b2ddf8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b281a5a30 con 0x7f6b28104f60 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b2ddf8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b281a5ba0 con 0x7f6b280ffe50 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b2ddf8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b281a5d60 con 0x7f6b28102350 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b26ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b28104f60 0x7f6b28101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b26ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b28104f60 0x7f6b28101e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59464/0 (socket says 172.21.15.46:59464) 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b26ffd700 1 -- 172.21.15.46:0/2887045984 learned_addr learned my addr 172.21.15.46:0/2887045984 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b277fe700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b280ffe50 0x7f6b281018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b27fff700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b28102350 0x7f6b281a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b26ffd700 1 -- 172.21.15.46:0/2887045984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b28102350 msgr2=0x7f6b281a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b26ffd700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b28102350 0x7f6b281a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b26ffd700 1 -- 172.21.15.46:0/2887045984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b280ffe50 msgr2=0x7f6b281018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b26ffd700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b280ffe50 0x7f6b281018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b26ffd700 1 -- 172.21.15.46:0/2887045984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c009520 con 0x7f6b28104f60 2024-03-18T20:42:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b27fff700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b28102350 0x7f6b281a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.399+0000 7f6b26ffd700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b28104f60 0x7f6b28101e10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00ea30 tx=0x7f6b1c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.400+0000 7f6b24ff9700 1 -- 172.21.15.46:0/2887045984 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c009820 con 0x7f6b28104f60 2024-03-18T20:42:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.400+0000 7f6b24ff9700 1 -- 172.21.15.46:0/2887045984 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b1c0044e0 con 0x7f6b28104f60 2024-03-18T20:42:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.400+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/2887045984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b180095f0 con 0x7f6b28104f60 2024-03-18T20:42:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.400+0000 7f6b24ff9700 1 -- 172.21.15.46:0/2887045984 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c003dc0 con 0x7f6b28104f60 2024-03-18T20:42:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.400+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/2887045984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b281a6330 con 0x7f6b28104f60 2024-03-18T20:42:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.402+0000 7f6b24ff9700 1 -- 172.21.15.46:0/2887045984 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6b1c004000 con 0x7f6b28104f60 2024-03-18T20:42:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.403+0000 7f6b24ff9700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6b100a1290 0x7f6b100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.403+0000 7f6b277fe700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6b100a1290 0x7f6b100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.403+0000 7f6b24ff9700 1 -- 172.21.15.46:0/2887045984 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6b1c014070 con 0x7f6b28104f60 2024-03-18T20:42:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.403+0000 7f6b277fe700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6b100a1290 0x7f6b100a3750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800b2e0 tx=0x7f6b18005fd0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.403+0000 7f6b39f8db80 1 Processor -- start 2024-03-18T20:42:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.403+0000 7f6b39f8db80 1 -- start start 2024-03-18T20:42:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b39f8db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60c9ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b39f8db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60c9f380 0x55ff60ca9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b39f8db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60caa220 0x55ff60cac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b39f8db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60bf8990 con 0x55ff60c9ea60 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b39f8db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60bf6350 con 0x55ff60caa220 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b39f8db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60bf1c40 con 0x55ff60c9f380 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60c9ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60c9ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59476/0 (socket says 172.21.15.46:59476) 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b277fe700 1 -- 172.21.15.46:0/536085325 learned_addr learned my addr 172.21.15.46:0/536085325 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b27fff700 1 --2- 172.21.15.46:0/536085325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60caa220 0x55ff60cac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b26ffd700 1 --2- 172.21.15.46:0/536085325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60c9f380 0x55ff60ca9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b277fe700 1 -- 172.21.15.46:0/536085325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60c9f380 msgr2=0x55ff60ca9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b277fe700 1 --2- 172.21.15.46:0/536085325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60c9f380 0x55ff60ca9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b277fe700 1 -- 172.21.15.46:0/536085325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60caa220 msgr2=0x55ff60cac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b277fe700 1 --2- 172.21.15.46:0/536085325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60caa220 0x55ff60cac610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b277fe700 1 -- 172.21.15.46:0/536085325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff60a733b0 con 0x55ff60c9ea60 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b27fff700 1 --2- 172.21.15.46:0/536085325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60caa220 0x55ff60cac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.404+0000 7f6b277fe700 1 --2- 172.21.15.46:0/536085325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60c9ee40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6b180095c0 tx=0x7f6b18020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37caf64efcecb9e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.405+0000 7f6b0dffb700 1 -- 172.21.15.46:0/536085325 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18005660 con 0x55ff60c9ea60 2024-03-18T20:42:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.405+0000 7f6b0dffb700 1 -- 172.21.15.46:0/536085325 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b1801e040 con 0x55ff60c9ea60 2024-03-18T20:42:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.405+0000 7f6b0dffb700 1 -- 172.21.15.46:0/536085325 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800f8c0 con 0x55ff60c9ea60 2024-03-18T20:42:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.405+0000 7f6b39f8db80 1 -- 172.21.15.46:0/536085325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 msgr2=0x55ff60c9ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.405+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/536085325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60c9ee40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6b180095c0 tx=0x7f6b18020000 comp rx=0 tx=0).stop 2024-03-18T20:42:33.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.405+0000 7f6b39f8db80 1 -- 172.21.15.46:0/536085325 shutdown_connections 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.405+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/536085325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60c9ee40 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.405+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/536085325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60caa220 0x55ff60cac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.405+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/536085325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60c9f380 0x55ff60ca9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.405+0000 7f6b39f8db80 1 -- 172.21.15.46:0/536085325 >> 172.21.15.46:0/536085325 conn(0x55ff60c15220 msgr2=0x55ff60c9b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.405+0000 7f6b39f8db80 1 -- 172.21.15.46:0/536085325 shutdown_connections 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.405+0000 7f6b39f8db80 1 -- 172.21.15.46:0/536085325 wait complete. 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.406+0000 7f6b39f8db80 1 -- start start 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.406+0000 7f6b39f8db80 1 Processor -- start 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.406+0000 7f6b39f8db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60ca9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.406+0000 7f6b39f8db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60c9f380 0x55ff60ca6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.406+0000 7f6b39f8db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60caa220 0x55ff60ca76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.406+0000 7f6b39f8db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60b8dd80 con 0x55ff60c9ea60 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.406+0000 7f6b39f8db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60b8da20 con 0x55ff60c9f380 2024-03-18T20:42:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.406+0000 7f6b39f8db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60a756a0 con 0x55ff60caa220 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.406+0000 7f6b39f8db80 10 client.0 _ll_register_callbacks cb 0x55ff609f93a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.406+0000 7f6b277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60ca9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60ca9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59486/0 (socket says 172.21.15.46:59486) 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b277fe700 1 -- 172.21.15.46:0/2072426477 learned_addr learned my addr 172.21.15.46:0/2072426477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b27fff700 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60caa220 0x55ff60ca76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b26ffd700 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60c9f380 0x55ff60ca6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b277fe700 1 -- 172.21.15.46:0/2072426477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60caa220 msgr2=0x55ff60ca76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b277fe700 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60caa220 0x55ff60ca76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b277fe700 1 -- 172.21.15.46:0/2072426477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60c9f380 msgr2=0x55ff60ca6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b277fe700 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60c9f380 0x55ff60ca6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b277fe700 1 -- 172.21.15.46:0/2072426477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b20009050 con 0x55ff60c9ea60 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b277fe700 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60ca9ca0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009ce0 tx=0x7f6b1800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad354cc257efc09 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b06ffd700 1 -- 172.21.15.46:0/2072426477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18005720 con 0x55ff60c9ea60 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2072426477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff60c92ad0 con 0x55ff60c9ea60 2024-03-18T20:42:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b06ffd700 1 -- 172.21.15.46:0/2072426477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6b1800f870 con 0x55ff60c9ea60 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b06ffd700 1 -- 172.21.15.46:0/2072426477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b18004500 con 0x55ff60c9ea60 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b39f8db80 10 client.17769 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.407+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2072426477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f6b2810d350 con 0x55ff60c9ea60 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.408+0000 7f6b06ffd700 1 -- 172.21.15.46:0/2072426477 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ff60c92ad0 con 0x55ff60c9ea60 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.408+0000 7f6b06ffd700 1 -- 172.21.15.46:0/2072426477 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6b1802f9e0 con 0x55ff60c9ea60 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.408+0000 7f6b39f8db80 10 client.17769 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.408+0000 7f6b39f8db80 10 client.17769 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.408+0000 7f6b39f8db80 20 client.17769 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.408+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ff60cb98d0 0x55ff60cbbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.408+0000 7f6b39f8db80 4 client.17769 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.408+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2072426477 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff60bea700 con 0x55ff60cb98d0 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.408+0000 7f6b26ffd700 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ff60cb98d0 0x55ff60cbbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b06ffd700 10 client.17769 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b26ffd700 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ff60cb98d0 0x55ff60cbbcc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2ff37b2a74690e45 server_cookie=219f344e2fca31b in_seq=0 out_seq=0 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b06ffd700 1 -- 172.21.15.46:0/2072426477 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ff60bea700 con 0x55ff60cb98d0 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b06ffd700 10 client.17769 handle_command_reply: tid=0 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b39f8db80 1 client.17769 shutdown 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2072426477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 msgr2=0x55ff60ca9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60ca9ca0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009ce0 tx=0x7f6b1800b390 comp rx=0 tx=0).stop 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2072426477 shutdown_connections 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ff60cb98d0 0x55ff60cbbcc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60ca9ca0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b06ffd700 0 client.17769 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60caa220 0x55ff60ca76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/2072426477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60c9f380 0x55ff60ca6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2072426477 >> 172.21.15.46:0/2072426477 conn(0x55ff60c15220 msgr2=0x55ff60c9ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2072426477 shutdown_connections 2024-03-18T20:42:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.409+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2072426477 wait complete. 2024-03-18T20:42:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.410+0000 7f6b39f8db80 20 client.17769 trim_cache size 0 max 16384 2024-03-18T20:42:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.423+0000 7f6b39f8db80 1 Processor -- start 2024-03-18T20:42:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.423+0000 7f6b39f8db80 1 -- start start 2024-03-18T20:42:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.423+0000 7f6b39f8db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60ca19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.423+0000 7f6b39f8db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60c9f380 0x55ff60cb0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.423+0000 7f6b39f8db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60caa220 0x55ff60ca6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.423+0000 7f6b39f8db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60ca8590 con 0x55ff60c9ea60 2024-03-18T20:42:33.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.423+0000 7f6b39f8db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60c9fd30 con 0x55ff60caa220 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.423+0000 7f6b39f8db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60c94f80 con 0x55ff60c9f380 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60ca19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60ca19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59490/0 (socket says 172.21.15.46:59490) 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b277fe700 1 -- 172.21.15.46:0/2889376296 learned_addr learned my addr 172.21.15.46:0/2889376296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b26ffd700 1 --2- 172.21.15.46:0/2889376296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60c9f380 0x55ff60cb0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b27fff700 1 --2- 172.21.15.46:0/2889376296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60caa220 0x55ff60ca6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b277fe700 1 -- 172.21.15.46:0/2889376296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60c9f380 msgr2=0x55ff60cb0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b277fe700 1 --2- 172.21.15.46:0/2889376296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60c9f380 0x55ff60cb0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b277fe700 1 -- 172.21.15.46:0/2889376296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60caa220 msgr2=0x55ff60ca6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b277fe700 1 --2- 172.21.15.46:0/2889376296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60caa220 0x55ff60ca6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b277fe700 1 -- 172.21.15.46:0/2889376296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff60a1fb60 con 0x55ff60c9ea60 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b277fe700 1 --2- 172.21.15.46:0/2889376296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60ca19c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800b690 tx=0x7f6b18029f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddddab3210b1c82b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b0e7fc700 1 -- 172.21.15.46:0/2889376296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18047070 con 0x55ff60c9ea60 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b0e7fc700 1 -- 172.21.15.46:0/2889376296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b1802fe90 con 0x55ff60c9ea60 2024-03-18T20:42:33.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.424+0000 7f6b0e7fc700 1 -- 172.21.15.46:0/2889376296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800e3e0 con 0x55ff60c9ea60 2024-03-18T20:42:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.425+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2889376296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 msgr2=0x55ff60ca19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.425+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/2889376296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60ca19c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800b690 tx=0x7f6b18029f20 comp rx=0 tx=0).stop 2024-03-18T20:42:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.425+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2889376296 shutdown_connections 2024-03-18T20:42:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.425+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/2889376296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9ea60 0x55ff60ca19c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.425+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/2889376296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60caa220 0x55ff60ca6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.425+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/2889376296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60c9f380 0x55ff60cb0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.425+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2889376296 >> 172.21.15.46:0/2889376296 conn(0x55ff60c15220 msgr2=0x55ff60ca2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.425+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2889376296 shutdown_connections 2024-03-18T20:42:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.425+0000 7f6b39f8db80 1 -- 172.21.15.46:0/2889376296 wait complete. 2024-03-18T20:42:33.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.425+0000 7f6b39f8db80 1 -- start start 2024-03-18T20:42:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b39f8db80 1 Processor -- start 2024-03-18T20:42:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b39f8db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60c9ea60 0x55ff60caee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b39f8db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9f380 0x55ff60caf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b39f8db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60caa220 0x55ff60caf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b39f8db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60d0c650 con 0x55ff60c9f380 2024-03-18T20:42:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b39f8db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60d0c7c0 con 0x55ff60c9ea60 2024-03-18T20:42:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b39f8db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60d0c980 con 0x55ff60caa220 2024-03-18T20:42:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b39f8db80 10 client.0 _ll_register_callbacks cb 0x7f6b20009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b26ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9f380 0x55ff60caf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b26ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9f380 0x55ff60caf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59498/0 (socket says 172.21.15.46:59498) 2024-03-18T20:42:33.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b26ffd700 1 -- 172.21.15.46:0/3897587760 learned_addr learned my addr 172.21.15.46:0/3897587760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b27fff700 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60caa220 0x55ff60caf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.426+0000 7f6b277fe700 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60c9ea60 0x55ff60caee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b26ffd700 1 -- 172.21.15.46:0/3897587760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60caa220 msgr2=0x55ff60caf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b26ffd700 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60caa220 0x55ff60caf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b26ffd700 1 -- 172.21.15.46:0/3897587760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60c9ea60 msgr2=0x55ff60caee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b26ffd700 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60c9ea60 0x55ff60caee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b26ffd700 1 -- 172.21.15.46:0/3897587760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2810d350 con 0x55ff60c9f380 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b27fff700 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60caa220 0x55ff60caf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b26ffd700 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9f380 0x55ff60caf340 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c096bf0 tx=0x7f6b1c09a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68336990ede36a0b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b06ffd700 1 -- 172.21.15.46:0/3897587760 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c063070 con 0x55ff60c9f380 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b06ffd700 1 -- 172.21.15.46:0/3897587760 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f6b1c09c040 con 0x55ff60c9f380 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b39f8db80 1 -- 172.21.15.46:0/3897587760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b1801e040 con 0x55ff60c9f380 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b06ffd700 1 -- 172.21.15.46:0/3897587760 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b1c0b1040 con 0x55ff60c9f380 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b39f8db80 10 client.17775 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:33.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b39f8db80 1 -- 172.21.15.46:0/3897587760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55ff60bb7ea0 con 0x55ff60c9f380 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.427+0000 7f6b06ffd700 1 -- 172.21.15.46:0/3897587760 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b1801e040 con 0x55ff60c9f380 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b06ffd700 1 -- 172.21.15.46:0/3897587760 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6b1c0caa40 con 0x55ff60c9f380 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b39f8db80 10 client.17775 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b39f8db80 10 client.17775 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b39f8db80 20 client.17775 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ff60d0e0e0 0x55ff60d10500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b39f8db80 4 client.17775 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b39f8db80 1 -- 172.21.15.46:0/3897587760 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ff60a40cf0 con 0x55ff60d0e0e0 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b277fe700 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ff60d0e0e0 0x55ff60d10500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b277fe700 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ff60d0e0e0 0x55ff60d10500 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9edc6567b39bf68f server_cookie=3f5b0bdf67659164 in_seq=0 out_seq=0 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b06ffd700 10 client.17775 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b06ffd700 1 -- 172.21.15.46:0/3897587760 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ff60a40cf0 con 0x55ff60d0e0e0 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b06ffd700 10 client.17775 handle_command_reply: tid=0 2024-03-18T20:42:33.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.428+0000 7f6b39f8db80 1 client.17775 shutdown 2024-03-18T20:42:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b39f8db80 1 -- 172.21.15.46:0/3897587760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9f380 msgr2=0x55ff60caf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9f380 0x55ff60caf340 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c096bf0 tx=0x7f6b1c09a9c0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b39f8db80 1 -- 172.21.15.46:0/3897587760 shutdown_connections 2024-03-18T20:42:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ff60d0e0e0 0x55ff60d10500 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ff60caa220 0x55ff60caf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff60c9ea60 0x55ff60caee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b06ffd700 0 client.17775 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:42:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b39f8db80 1 --2- 172.21.15.46:0/3897587760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff60c9f380 0x55ff60caf340 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b39f8db80 1 -- 172.21.15.46:0/3897587760 >> 172.21.15.46:0/3897587760 conn(0x55ff60c15220 msgr2=0x55ff60ca6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b39f8db80 1 -- 172.21.15.46:0/3897587760 shutdown_connections 2024-03-18T20:42:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b39f8db80 1 -- 172.21.15.46:0/3897587760 wait complete. 2024-03-18T20:42:33.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b39f8db80 20 client.17775 trim_cache size 0 max 16384 2024-03-18T20:42:33.438 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:42:33.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/2887045984 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6b100a1290 msgr2=0x7f6b100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b2ddf8700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6b100a1290 0x7f6b100a3750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800b2e0 tx=0x7f6b18005fd0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/2887045984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b28104f60 msgr2=0x7f6b28101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.429+0000 7f6b2ddf8700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b28104f60 0x7f6b28101e10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00ea30 tx=0x7f6b1c00ed40 comp rx=0 tx=0).stop 2024-03-18T20:42:33.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.430+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/2887045984 shutdown_connections 2024-03-18T20:42:33.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.430+0000 7f6b2ddf8700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6b100a1290 0x7f6b100a3750 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.430+0000 7f6b2ddf8700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b28102350 0x7f6b281a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.430+0000 7f6b2ddf8700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b280ffe50 0x7f6b281018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.430+0000 7f6b2ddf8700 1 --2- 172.21.15.46:0/2887045984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b28104f60 0x7f6b28101e10 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.430+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/2887045984 >> 172.21.15.46:0/2887045984 conn(0x7f6b28076d40 msgr2=0x7f6b28068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.430+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/2887045984 shutdown_connections 2024-03-18T20:42:33.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.430+0000 7f6b2ddf8700 1 -- 172.21.15.46:0/2887045984 wait complete. 2024-03-18T20:42:33.446 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:42:33.446 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:42:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.582+0000 7f5255c13700 1 -- 172.21.15.46:0/826564055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5250100820 msgr2=0x7f5250104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.582+0000 7f5255c13700 1 --2- 172.21.15.46:0/826564055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5250100820 0x7f5250104860 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f52440098f0 tx=0x7f5244009c00 comp rx=0 tx=0).stop 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.582+0000 7f5255c13700 1 -- 172.21.15.46:0/826564055 shutdown_connections 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.582+0000 7f5255c13700 1 --2- 172.21.15.46:0/826564055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52500ffe70 0x7f5250100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.582+0000 7f5255c13700 1 --2- 172.21.15.46:0/826564055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5250104f90 0x7f525010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.582+0000 7f5255c13700 1 --2- 172.21.15.46:0/826564055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5250100820 0x7f5250104860 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.582+0000 7f5255c13700 1 -- 172.21.15.46:0/826564055 >> 172.21.15.46:0/826564055 conn(0x7f5250076d80 msgr2=0x7f5250077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.582+0000 7f5255c13700 1 -- 172.21.15.46:0/826564055 shutdown_connections 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.582+0000 7f5255c13700 1 -- 172.21.15.46:0/826564055 wait complete. 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.583+0000 7f5255c13700 1 Processor -- start 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.583+0000 7f5255c13700 1 -- start start 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.583+0000 7f5255c13700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52500ffe70 0x7f5250197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.583+0000 7f5255c13700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5250104f90 0x7f52501977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.583+0000 7f5255c13700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f525019bb50 0x7f525019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.583+0000 7f5255c13700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525019c5d0 con 0x7f5250104f90 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.583+0000 7f5255c13700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525019c740 con 0x7f52500ffe70 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.583+0000 7f5255c13700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525019c900 con 0x7f525019bb50 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.583+0000 7f524effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5250104f90 0x7f52501977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5250104f90 0x7f52501977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59506/0 (socket says 172.21.15.46:59506) 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524effd700 1 -- 172.21.15.46:0/2872265499 learned_addr learned my addr 172.21.15.46:0/2872265499 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524ffff700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f525019bb50 0x7f525019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524f7fe700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52500ffe70 0x7f5250197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524effd700 1 -- 172.21.15.46:0/2872265499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f525019bb50 msgr2=0x7f525019bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524effd700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f525019bb50 0x7f525019bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524effd700 1 -- 172.21.15.46:0/2872265499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52500ffe70 msgr2=0x7f5250197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524effd700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52500ffe70 0x7f5250197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524effd700 1 -- 172.21.15.46:0/2872265499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52400095f0 con 0x7f5250104f90 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524ffff700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f525019bb50 0x7f525019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524f7fe700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52500ffe70 0x7f5250197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524effd700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5250104f90 0x7f52501977b0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5244009230 tx=0x7f5244010ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524cff9700 1 -- 172.21.15.46:0/2872265499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524401c070 con 0x7f5250104f90 2024-03-18T20:42:33.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f5255c13700 1 -- 172.21.15.46:0/2872265499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5248009c40 con 0x7f5250104f90 2024-03-18T20:42:33.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524cff9700 1 -- 172.21.15.46:0/2872265499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f524400bb20 con 0x7f5250104f90 2024-03-18T20:42:33.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f524cff9700 1 -- 172.21.15.46:0/2872265499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52440038e0 con 0x7f5250104f90 2024-03-18T20:42:33.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.584+0000 7f5255c13700 1 -- 172.21.15.46:0/2872265499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5244009520 con 0x7f5250104f90 2024-03-18T20:42:33.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.587+0000 7f524cff9700 1 -- 172.21.15.46:0/2872265499 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5244020030 con 0x7f5250104f90 2024-03-18T20:42:33.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.587+0000 7f524cff9700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f52380a1290 0x7f52380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.587+0000 7f524cff9700 1 -- 172.21.15.46:0/2872265499 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f52440cdb70 con 0x7f5250104f90 2024-03-18T20:42:33.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.587+0000 7f524f7fe700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f52380a1290 0x7f52380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.588+0000 7f524f7fe700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f52380a1290 0x7f52380a3750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5240005fd0 tx=0x7f5240009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.588+0000 7f5261da8b80 1 Processor -- start 2024-03-18T20:42:33.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.588+0000 7f5261da8b80 1 -- start start 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f5261da8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5474fa60 0x559e5474fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f5261da8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e5475ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f5261da8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e5475d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f5261da8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e546a9990 con 0x559e5475b220 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f5261da8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e546a7350 con 0x559e5474fa60 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f5261da8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e546a2c40 con 0x559e54750380 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f524ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e5475d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f524ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e5475d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59516/0 (socket says 172.21.15.46:59516) 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f524ffff700 1 -- 172.21.15.46:0/1809520582 learned_addr learned my addr 172.21.15.46:0/1809520582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f524f7fe700 1 --2- 172.21.15.46:0/1809520582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5474fa60 0x559e5474fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f524effd700 1 --2- 172.21.15.46:0/1809520582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e5475ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f524ffff700 1 -- 172.21.15.46:0/1809520582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 msgr2=0x559e5475ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f524ffff700 1 --2- 172.21.15.46:0/1809520582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e5475ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f524ffff700 1 -- 172.21.15.46:0/1809520582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5474fa60 msgr2=0x559e5474fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f524ffff700 1 --2- 172.21.15.46:0/1809520582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5474fa60 0x559e5474fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f524ffff700 1 -- 172.21.15.46:0/1809520582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e545243b0 con 0x559e5475b220 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f524f7fe700 1 --2- 172.21.15.46:0/1809520582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5474fa60 0x559e5474fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.589+0000 7f524ffff700 1 --2- 172.21.15.46:0/1809520582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e5475d610 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5248007610 tx=0x7f5248009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d25364717a98a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.590+0000 7f5235ffb700 1 -- 172.21.15.46:0/1809520582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248009100 con 0x559e5475b220 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.590+0000 7f5235ffb700 1 -- 172.21.15.46:0/1809520582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f524800f040 con 0x559e5475b220 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.590+0000 7f5235ffb700 1 -- 172.21.15.46:0/1809520582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248017d80 con 0x559e5475b220 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.590+0000 7f5261da8b80 1 -- 172.21.15.46:0/1809520582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 msgr2=0x559e5475d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.590+0000 7f5261da8b80 1 --2- 172.21.15.46:0/1809520582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e5475d610 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5248007610 tx=0x7f5248009920 comp rx=0 tx=0).stop 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.590+0000 7f5261da8b80 1 -- 172.21.15.46:0/1809520582 shutdown_connections 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.590+0000 7f5261da8b80 1 --2- 172.21.15.46:0/1809520582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e5475d610 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.590+0000 7f5261da8b80 1 --2- 172.21.15.46:0/1809520582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5474fa60 0x559e5474fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.590+0000 7f5261da8b80 1 --2- 172.21.15.46:0/1809520582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e5475ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.590+0000 7f5261da8b80 1 -- 172.21.15.46:0/1809520582 >> 172.21.15.46:0/1809520582 conn(0x559e546c6220 msgr2=0x559e5474c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.590+0000 7f5261da8b80 1 -- 172.21.15.46:0/1809520582 shutdown_connections 2024-03-18T20:42:33.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.590+0000 7f5261da8b80 1 -- 172.21.15.46:0/1809520582 wait complete. 2024-03-18T20:42:33.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.591+0000 7f5261da8b80 1 -- start start 2024-03-18T20:42:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.591+0000 7f5261da8b80 1 Processor -- start 2024-03-18T20:42:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.591+0000 7f5261da8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5474fa60 0x559e5475ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.591+0000 7f5261da8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e54757ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.591+0000 7f5261da8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5475b220 0x559e54758610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.591+0000 7f5261da8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e5463ed80 con 0x559e5474fa60 2024-03-18T20:42:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.591+0000 7f5261da8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e5463ea20 con 0x559e5475b220 2024-03-18T20:42:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.591+0000 7f5261da8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e545266a0 con 0x559e54750380 2024-03-18T20:42:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.591+0000 7f5261da8b80 10 client.0 _ll_register_callbacks cb 0x559e544aa3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f524f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5474fa60 0x559e5475ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f524f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5474fa60 0x559e5475ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59518/0 (socket says 172.21.15.46:59518) 2024-03-18T20:42:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f524f7fe700 1 -- 172.21.15.46:0/2343799710 learned_addr learned my addr 172.21.15.46:0/2343799710 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f524effd700 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e54757ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f524ffff700 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5475b220 0x559e54758610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f524f7fe700 1 -- 172.21.15.46:0/2343799710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 msgr2=0x559e54757ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f524f7fe700 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e54757ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f524f7fe700 1 -- 172.21.15.46:0/2343799710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5475b220 msgr2=0x559e54758610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f524f7fe700 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5475b220 0x559e54758610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f524f7fe700 1 -- 172.21.15.46:0/2343799710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5240011da0 con 0x559e5474fa60 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f524effd700 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e54757ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f524f7fe700 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5474fa60 0x559e5475ac50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5240012cd0 tx=0x7f52400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9eee07afbeea077 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f522effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f522effd700 1 -- 172.21.15.46:0/2343799710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240004220 con 0x559e5474fa60 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f522effd700 1 -- 172.21.15.46:0/2343799710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5240018070 con 0x559e5474fa60 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f5261da8b80 1 -- 172.21.15.46:0/2343799710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e54743ad0 con 0x559e5474fa60 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f522effd700 1 -- 172.21.15.46:0/2343799710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5240012e60 con 0x559e5474fa60 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f5261da8b80 10 client.17787 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.592+0000 7f5261da8b80 1 -- 172.21.15.46:0/2343799710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f5248009260 con 0x559e5474fa60 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.593+0000 7f522effd700 1 -- 172.21.15.46:0/2343799710 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559e54743ad0 con 0x559e5474fa60 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.593+0000 7f522effd700 1 -- 172.21.15.46:0/2343799710 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f524001fb50 con 0x559e5474fa60 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.593+0000 7f5261da8b80 10 client.17787 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.593+0000 7f5261da8b80 10 client.17787 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.593+0000 7f5261da8b80 20 client.17787 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.593+0000 7f5261da8b80 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x559e5476ad70 0x559e5476d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.593+0000 7f5261da8b80 4 client.17787 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.593+0000 7f5261da8b80 1 -- 172.21.15.46:0/2343799710 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e5469b700 con 0x559e5476ad70 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.593+0000 7f524effd700 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x559e5476ad70 0x559e5476d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.594+0000 7f524effd700 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x559e5476ad70 0x559e5476d190 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=10db5c3f1f9d3a27 server_cookie=38ce7e2d96427f38 in_seq=0 out_seq=0 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.594+0000 7f522effd700 10 client.17787 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.594+0000 7f522effd700 1 -- 172.21.15.46:0/2343799710 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559e5469b700 con 0x559e5476ad70 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.594+0000 7f522effd700 10 client.17787 handle_command_reply: tid=0 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.594+0000 7f5261da8b80 1 client.17787 shutdown 2024-03-18T20:42:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.594+0000 7f5261da8b80 1 -- 172.21.15.46:0/2343799710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5474fa60 msgr2=0x559e5475ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.594+0000 7f5261da8b80 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5474fa60 0x559e5475ac50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5240012cd0 tx=0x7f52400047c0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.595+0000 7f5261da8b80 1 -- 172.21.15.46:0/2343799710 shutdown_connections 2024-03-18T20:42:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.595+0000 7f5261da8b80 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x559e5476ad70 0x559e5476d190 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.595+0000 7f5261da8b80 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5474fa60 0x559e5475ac50 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.595+0000 7f5261da8b80 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5475b220 0x559e54758610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.595+0000 7f5261da8b80 1 --2- 172.21.15.46:0/2343799710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e54757ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.595+0000 7f522effd700 0 client.17787 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:42:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.595+0000 7f5261da8b80 1 -- 172.21.15.46:0/2343799710 >> 172.21.15.46:0/2343799710 conn(0x559e546c6220 msgr2=0x559e5474bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.595+0000 7f5261da8b80 1 -- 172.21.15.46:0/2343799710 shutdown_connections 2024-03-18T20:42:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.595+0000 7f5261da8b80 1 -- 172.21.15.46:0/2343799710 wait complete. 2024-03-18T20:42:33.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.595+0000 7f5261da8b80 20 client.17787 trim_cache size 0 max 16384 2024-03-18T20:42:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.608+0000 7f5261da8b80 1 Processor -- start 2024-03-18T20:42:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.608+0000 7f5261da8b80 1 -- start start 2024-03-18T20:42:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f5261da8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5474fa60 0x559e5476d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f5261da8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e54761e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f5261da8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e5475a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f5261da8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e5474c560 con 0x559e5475b220 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f5261da8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e54758e40 con 0x559e5474fa60 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f5261da8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e547623c0 con 0x559e54750380 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f524ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e5475a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f524ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e5475a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59528/0 (socket says 172.21.15.46:59528) 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f524ffff700 1 -- 172.21.15.46:0/1749711080 learned_addr learned my addr 172.21.15.46:0/1749711080 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f524effd700 1 --2- 172.21.15.46:0/1749711080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e54761e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f524f7fe700 1 --2- 172.21.15.46:0/1749711080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5474fa60 0x559e5476d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f524ffff700 1 -- 172.21.15.46:0/1749711080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 msgr2=0x559e54761e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f524ffff700 1 --2- 172.21.15.46:0/1749711080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e54761e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f524ffff700 1 -- 172.21.15.46:0/1749711080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5474fa60 msgr2=0x559e5476d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f524ffff700 1 --2- 172.21.15.46:0/1749711080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5474fa60 0x559e5476d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f524ffff700 1 -- 172.21.15.46:0/1749711080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e544d0b60 con 0x559e5475b220 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f524ffff700 1 --2- 172.21.15.46:0/1749711080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e5475a780 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f524801c000 tx=0x7f52480174c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32694f8ff62866a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f52367fc700 1 -- 172.21.15.46:0/1749711080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248006000 con 0x559e5475b220 2024-03-18T20:42:33.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f52367fc700 1 -- 172.21.15.46:0/1749711080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5248003c00 con 0x559e5475b220 2024-03-18T20:42:33.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.609+0000 7f52367fc700 1 -- 172.21.15.46:0/1749711080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248017b80 con 0x559e5475b220 2024-03-18T20:42:33.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.610+0000 7f5261da8b80 1 -- 172.21.15.46:0/1749711080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 msgr2=0x559e5475a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.610+0000 7f5261da8b80 1 --2- 172.21.15.46:0/1749711080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e5475a780 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f524801c000 tx=0x7f52480174c0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.610+0000 7f5261da8b80 1 -- 172.21.15.46:0/1749711080 shutdown_connections 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.610+0000 7f5261da8b80 1 --2- 172.21.15.46:0/1749711080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e5475a780 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.610+0000 7f5261da8b80 1 --2- 172.21.15.46:0/1749711080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e5474fa60 0x559e5476d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.610+0000 7f5261da8b80 1 --2- 172.21.15.46:0/1749711080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e54750380 0x559e54761e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.610+0000 7f5261da8b80 1 -- 172.21.15.46:0/1749711080 >> 172.21.15.46:0/1749711080 conn(0x559e546c6220 msgr2=0x559e547531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.610+0000 7f5261da8b80 1 -- 172.21.15.46:0/1749711080 shutdown_connections 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.610+0000 7f5261da8b80 1 -- 172.21.15.46:0/1749711080 wait complete. 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.610+0000 7f5261da8b80 1 -- start start 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f5261da8b80 1 Processor -- start 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f5261da8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e5474fa60 0x559e54760050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f5261da8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e54750380 0x559e54760590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f5261da8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e547b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f5261da8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e547b8190 con 0x559e5475b220 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f5261da8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e547b8300 con 0x559e54750380 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f5261da8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e547b84c0 con 0x559e5474fa60 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f5261da8b80 10 client.0 _ll_register_callbacks cb 0x7f52400128b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f524ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e547b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f524ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e547b7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59542/0 (socket says 172.21.15.46:59542) 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f524ffff700 1 -- 172.21.15.46:0/4085204723 learned_addr learned my addr 172.21.15.46:0/4085204723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f524effd700 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e54750380 0x559e54760590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f524f7fe700 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e5474fa60 0x559e54760050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f524ffff700 1 -- 172.21.15.46:0/4085204723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e5474fa60 msgr2=0x559e54760050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f524ffff700 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e5474fa60 0x559e54760050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f524ffff700 1 -- 172.21.15.46:0/4085204723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e54750380 msgr2=0x559e54760590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.611+0000 7f524ffff700 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e54750380 0x559e54760590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f524ffff700 1 -- 172.21.15.46:0/4085204723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e545243b0 con 0x559e5475b220 2024-03-18T20:42:33.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f524effd700 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e54750380 0x559e54760590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f524ffff700 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e547b7c50 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5248017490 tx=0x7f524800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3700a3d73df147 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f522effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f522effd700 1 -- 172.21.15.46:0/4085204723 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52480177a0 con 0x559e5475b220 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f522effd700 1 -- 172.21.15.46:0/4085204723 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5248017900 con 0x559e5475b220 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f5261da8b80 1 -- 172.21.15.46:0/4085204723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f524800f040 con 0x559e5475b220 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f522effd700 1 -- 172.21.15.46:0/4085204723 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5248017ac0 con 0x559e5475b220 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f5261da8b80 10 client.17793 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f5261da8b80 1 -- 172.21.15.46:0/4085204723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x559e54668ea0 con 0x559e5475b220 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f522effd700 1 -- 172.21.15.46:0/4085204723 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f524800f040 con 0x559e5475b220 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f522effd700 1 -- 172.21.15.46:0/4085204723 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5248026030 con 0x559e5475b220 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f5261da8b80 10 client.17793 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f5261da8b80 10 client.17793 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f5261da8b80 20 client.17793 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.612+0000 7f5261da8b80 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x559e547bf920 0x559e547c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f5261da8b80 4 client.17793 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:42:33.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f5261da8b80 1 -- 172.21.15.46:0/4085204723 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559e544f1cf0 con 0x559e547bf920 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f524f7fe700 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x559e547bf920 0x559e547c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f522effd700 10 client.17793 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f524f7fe700 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x559e547bf920 0x559e547c1d10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8596d1344aa8699a server_cookie=f1fa15f46b8a4085 in_seq=0 out_seq=0 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f522effd700 1 -- 172.21.15.46:0/4085204723 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559e544f1cf0 con 0x559e547bf920 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f522effd700 10 client.17793 handle_command_reply: tid=0 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f5261da8b80 1 client.17793 shutdown 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f5261da8b80 1 -- 172.21.15.46:0/4085204723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 msgr2=0x559e547b7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f5261da8b80 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e547b7c50 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5248017490 tx=0x7f524800ed60 comp rx=0 tx=0).stop 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f5261da8b80 1 -- 172.21.15.46:0/4085204723 shutdown_connections 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f5261da8b80 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x559e547bf920 0x559e547c1d10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f5261da8b80 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e5475b220 0x559e547b7c50 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f5261da8b80 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e5474fa60 0x559e54760050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f522effd700 0 client.17793 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f5261da8b80 1 --2- 172.21.15.46:0/4085204723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e54750380 0x559e54760590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.613+0000 7f5261da8b80 1 -- 172.21.15.46:0/4085204723 >> 172.21.15.46:0/4085204723 conn(0x559e546c6220 msgr2=0x559e5474b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5261da8b80 1 -- 172.21.15.46:0/4085204723 shutdown_connections 2024-03-18T20:42:33.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5261da8b80 1 -- 172.21.15.46:0/4085204723 wait complete. 2024-03-18T20:42:33.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5261da8b80 20 client.17793 trim_cache size 0 max 16384 2024-03-18T20:42:33.622 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:42:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5255c13700 1 -- 172.21.15.46:0/2872265499 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f52380a1290 msgr2=0x7f52380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5255c13700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f52380a1290 0x7f52380a3750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5240005fd0 tx=0x7f5240009310 comp rx=0 tx=0).stop 2024-03-18T20:42:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5255c13700 1 -- 172.21.15.46:0/2872265499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5250104f90 msgr2=0x7f52501977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5255c13700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5250104f90 0x7f52501977b0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5244009230 tx=0x7f5244010ff0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5255c13700 1 -- 172.21.15.46:0/2872265499 shutdown_connections 2024-03-18T20:42:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5255c13700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f52380a1290 0x7f52380a3750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5255c13700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f525019bb50 0x7f525019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5255c13700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52500ffe70 0x7f5250197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5255c13700 1 --2- 172.21.15.46:0/2872265499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5250104f90 0x7f52501977b0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5255c13700 1 -- 172.21.15.46:0/2872265499 >> 172.21.15.46:0/2872265499 conn(0x7f5250076d80 msgr2=0x7f5250069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5255c13700 1 -- 172.21.15.46:0/2872265499 shutdown_connections 2024-03-18T20:42:33.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.614+0000 7f5255c13700 1 -- 172.21.15.46:0/2872265499 wait complete. 2024-03-18T20:42:33.631 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:42:33.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:42:33.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 214512 3.43 MB/sec execute 208 sec latency 1177.799 ms 2024-03-18T20:42:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.773+0000 7fdb924ab700 1 -- 172.21.15.46:0/1183265199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb8c101920 msgr2=0x7fdb8c105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.773+0000 7fdb924ab700 1 --2- 172.21.15.46:0/1183265199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb8c101920 0x7fdb8c105960 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009990 tx=0x7fdb80009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.774+0000 7fdb924ab700 1 -- 172.21.15.46:0/1183265199 shutdown_connections 2024-03-18T20:42:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.774+0000 7fdb924ab700 1 --2- 172.21.15.46:0/1183265199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c106090 0x7fdb8c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.774+0000 7fdb924ab700 1 --2- 172.21.15.46:0/1183265199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb8c100f70 0x7fdb8c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.774+0000 7fdb924ab700 1 --2- 172.21.15.46:0/1183265199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb8c101920 0x7fdb8c105960 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.774+0000 7fdb924ab700 1 -- 172.21.15.46:0/1183265199 >> 172.21.15.46:0/1183265199 conn(0x7fdb8c0fc7f0 msgr2=0x7fdb8c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.774+0000 7fdb924ab700 1 -- 172.21.15.46:0/1183265199 shutdown_connections 2024-03-18T20:42:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.774+0000 7fdb924ab700 1 -- 172.21.15.46:0/1183265199 wait complete. 2024-03-18T20:42:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.774+0000 7fdb924ab700 1 Processor -- start 2024-03-18T20:42:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.774+0000 7fdb924ab700 1 -- start start 2024-03-18T20:42:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb924ab700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c100f70 0x7fdb8c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb924ab700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb8c106090 0x7fdb8c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb924ab700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb8c073b30 0x7fdb8c1a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb924ab700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c1a5990 con 0x7fdb8c073b30 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb924ab700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c1a5b00 con 0x7fdb8c100f70 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb924ab700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c1a5cc0 con 0x7fdb8c106090 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb8bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c100f70 0x7fdb8c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb8bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c100f70 0x7fdb8c0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40364/0 (socket says 172.21.15.46:40364) 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb8bfff700 1 -- 172.21.15.46:0/3004186120 learned_addr learned my addr 172.21.15.46:0/3004186120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb90a48700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb8c073b30 0x7fdb8c1a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb8b7fe700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb8c106090 0x7fdb8c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb8bfff700 1 -- 172.21.15.46:0/3004186120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb8c106090 msgr2=0x7fdb8c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb8c106090 0x7fdb8c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb8bfff700 1 -- 172.21.15.46:0/3004186120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb8c073b30 msgr2=0x7fdb8c1a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb8c073b30 0x7fdb8c1a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb8bfff700 1 -- 172.21.15.46:0/3004186120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c009520 con 0x7fdb8c100f70 2024-03-18T20:42:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.775+0000 7fdb90a48700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb8c073b30 0x7fdb8c1a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.776+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c100f70 0x7fdb8c0730b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00eb30 tx=0x7fdb7c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.776+0000 7fdb897fa700 1 -- 172.21.15.46:0/3004186120 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c009880 con 0x7fdb8c100f70 2024-03-18T20:42:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.776+0000 7fdb897fa700 1 -- 172.21.15.46:0/3004186120 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb7c0044e0 con 0x7fdb8c100f70 2024-03-18T20:42:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.776+0000 7fdb924ab700 1 -- 172.21.15.46:0/3004186120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb800095f0 con 0x7fdb8c100f70 2024-03-18T20:42:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.776+0000 7fdb897fa700 1 -- 172.21.15.46:0/3004186120 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c003e90 con 0x7fdb8c100f70 2024-03-18T20:42:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.776+0000 7fdb924ab700 1 -- 172.21.15.46:0/3004186120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c1a6290 con 0x7fdb8c100f70 2024-03-18T20:42:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.778+0000 7fdb897fa700 1 -- 172.21.15.46:0/3004186120 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fdb7c004020 con 0x7fdb8c100f70 2024-03-18T20:42:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.779+0000 7fdb897fa700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdb740a1290 0x7fdb740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.779+0000 7fdb897fa700 1 -- 172.21.15.46:0/3004186120 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fdb7c014070 con 0x7fdb8c100f70 2024-03-18T20:42:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.779+0000 7fdb8b7fe700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdb740a1290 0x7fdb740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.779+0000 7fdb8b7fe700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdb740a1290 0x7fdb740a3750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000b2e0 tx=0x7fdb80005fd0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb9e640b80 1 Processor -- start 2024-03-18T20:42:33.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb9e640b80 1 -- start start 2024-03-18T20:42:33.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb9e640b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb9e640b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2bcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb9e640b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 0x55c60d2bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb9e640b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60d20b990 con 0x55c60d2b1a60 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb9e640b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60d209350 con 0x55c60d2bd220 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb9e640b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60d204c40 con 0x55c60d2b2380 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59572/0 (socket says 172.21.15.46:59572) 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb8bfff700 1 -- 172.21.15.46:0/408227937 learned_addr learned my addr 172.21.15.46:0/408227937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb90a48700 1 --2- 172.21.15.46:0/408227937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 0x55c60d2bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.780+0000 7fdb8b7fe700 1 --2- 172.21.15.46:0/408227937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2bcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.781+0000 7fdb8bfff700 1 -- 172.21.15.46:0/408227937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 msgr2=0x55c60d2bcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.781+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/408227937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2bcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.781+0000 7fdb8bfff700 1 -- 172.21.15.46:0/408227937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 msgr2=0x55c60d2bf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.781+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/408227937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 0x55c60d2bf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.781+0000 7fdb8bfff700 1 -- 172.21.15.46:0/408227937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c60d0863b0 con 0x55c60d2b1a60 2024-03-18T20:42:33.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.781+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/408227937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2b1e40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0a2190 tx=0x7fdb7c09a090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c57c62b9bdaa23 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.781+0000 7fdb727fc700 1 -- 172.21.15.46:0/408227937 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c09a940 con 0x55c60d2b1a60 2024-03-18T20:42:33.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.781+0000 7fdb727fc700 1 -- 172.21.15.46:0/408227937 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb7c00cc90 con 0x55c60d2b1a60 2024-03-18T20:42:33.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.781+0000 7fdb727fc700 1 -- 172.21.15.46:0/408227937 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c01f340 con 0x55c60d2b1a60 2024-03-18T20:42:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.782+0000 7fdb9e640b80 1 -- 172.21.15.46:0/408227937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 msgr2=0x55c60d2b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.782+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/408227937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2b1e40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0a2190 tx=0x7fdb7c09a090 comp rx=0 tx=0).stop 2024-03-18T20:42:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.782+0000 7fdb9e640b80 1 -- 172.21.15.46:0/408227937 shutdown_connections 2024-03-18T20:42:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.782+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/408227937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2b1e40 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.782+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/408227937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 0x55c60d2bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.782+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/408227937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2bcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.782+0000 7fdb9e640b80 1 -- 172.21.15.46:0/408227937 >> 172.21.15.46:0/408227937 conn(0x55c60d228220 msgr2=0x55c60d2ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.782+0000 7fdb9e640b80 1 -- 172.21.15.46:0/408227937 shutdown_connections 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.782+0000 7fdb9e640b80 1 -- 172.21.15.46:0/408227937 wait complete. 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.782+0000 7fdb9e640b80 1 -- start start 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.782+0000 7fdb9e640b80 1 Processor -- start 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb9e640b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2ba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb9e640b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2ba590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb9e640b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 0x55c60d2baad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb9e640b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60d1a0d80 con 0x55c60d2b1a60 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb9e640b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60d1a0a20 con 0x55c60d2bd220 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb9e640b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60d0886a0 con 0x55c60d2b2380 2024-03-18T20:42:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb9e640b80 10 client.0 _ll_register_callbacks cb 0x55c60d00c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2ba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2ba050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59574/0 (socket says 172.21.15.46:59574) 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb8bfff700 1 -- 172.21.15.46:0/2861021166 learned_addr learned my addr 172.21.15.46:0/2861021166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb90a48700 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 0x55c60d2baad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb8b7fe700 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2ba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb8bfff700 1 -- 172.21.15.46:0/2861021166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 msgr2=0x55c60d2ba590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2ba590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb8bfff700 1 -- 172.21.15.46:0/2861021166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 msgr2=0x55c60d2baad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 0x55c60d2baad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb8bfff700 1 -- 172.21.15.46:0/2861021166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c1a6290 con 0x55c60d2b1a60 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb8b7fe700 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2ba590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2ba050 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0cafd0 tx=0x7fdb7c0ca8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b70fd3e8040efd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb6b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.783+0000 7fdb6b7fe700 1 -- 172.21.15.46:0/2861021166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c09e070 con 0x55c60d2b1a60 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb6b7fe700 1 -- 172.21.15.46:0/2861021166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fdb7c01f340 con 0x55c60d2b1a60 2024-03-18T20:42:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2861021166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c60d2a5ad0 con 0x55c60d2b1a60 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb6b7fe700 1 -- 172.21.15.46:0/2861021166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb7c01f500 con 0x55c60d2b1a60 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb9e640b80 10 client.17805 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2861021166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c60d032b60 con 0x55c60d2b1a60 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb6b7fe700 1 -- 172.21.15.46:0/2861021166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c60d2a5ad0 con 0x55c60d2b1a60 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb6b7fe700 1 -- 172.21.15.46:0/2861021166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdb7c0a2230 con 0x55c60d2b1a60 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb9e640b80 10 client.17805 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb9e640b80 10 client.17805 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb9e640b80 20 client.17805 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c60d2cc980 0x55c60d2ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb9e640b80 4 client.17805 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2861021166 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c60d1fd700 con 0x55c60d2cc980 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.784+0000 7fdb8b7fe700 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c60d2cc980 0x55c60d2ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.785+0000 7fdb6b7fe700 10 client.17805 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.785+0000 7fdb8b7fe700 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c60d2cc980 0x55c60d2ced70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15b4e674d838d4bd server_cookie=51cdf7dd8b615909 in_seq=0 out_seq=0 2024-03-18T20:42:33.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb6b7fe700 1 -- 172.21.15.46:0/2861021166 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c60d1fd700 con 0x55c60d2cc980 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb6b7fe700 10 client.17805 handle_command_reply: tid=0 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb9e640b80 1 client.17805 shutdown 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2861021166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 msgr2=0x55c60d2ba050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2ba050 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0cafd0 tx=0x7fdb7c0ca8a0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2861021166 shutdown_connections 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c60d2cc980 0x55c60d2ced70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2ba050 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 0x55c60d2baad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb6b7fe700 0 client.17805 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/2861021166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2ba590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2861021166 >> 172.21.15.46:0/2861021166 conn(0x55c60d228220 msgr2=0x55c60d2adbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2861021166 shutdown_connections 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2861021166 wait complete. 2024-03-18T20:42:33.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.786+0000 7fdb9e640b80 20 client.17805 trim_cache size 0 max 16384 2024-03-18T20:42:33.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.800+0000 7fdb9e640b80 1 Processor -- start 2024-03-18T20:42:33.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.800+0000 7fdb9e640b80 1 -- start start 2024-03-18T20:42:33.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.800+0000 7fdb9e640b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2b1a60 0x55c60d2bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.800+0000 7fdb9e640b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb9e640b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2bd220 0x55c60d2ba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb9e640b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60d2bcff0 con 0x55c60d2bd220 2024-03-18T20:42:33.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb9e640b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60d2c99b0 con 0x55c60d2b1a60 2024-03-18T20:42:33.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb9e640b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60d2b2d60 con 0x55c60d2b2380 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb90a48700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2bd220 0x55c60d2ba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb90a48700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2bd220 0x55c60d2ba8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59586/0 (socket says 172.21.15.46:59586) 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb90a48700 1 -- 172.21.15.46:0/3453496901 learned_addr learned my addr 172.21.15.46:0/3453496901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/3453496901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2b1a60 0x55c60d2bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb8b7fe700 1 --2- 172.21.15.46:0/3453496901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb90a48700 1 -- 172.21.15.46:0/3453496901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 msgr2=0x55c60d2c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb90a48700 1 --2- 172.21.15.46:0/3453496901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb90a48700 1 -- 172.21.15.46:0/3453496901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2b1a60 msgr2=0x55c60d2bf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb90a48700 1 --2- 172.21.15.46:0/3453496901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2b1a60 0x55c60d2bf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb90a48700 1 -- 172.21.15.46:0/3453496901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c60d1caea0 con 0x55c60d2bd220 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/3453496901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2b1a60 0x55c60d2bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb90a48700 1 --2- 172.21.15.46:0/3453496901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2bd220 0x55c60d2ba8f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400ea90 tx=0x7fdb84009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39beb88ec99344de server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb72ffd700 1 -- 172.21.15.46:0/3453496901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400e5e0 con 0x55c60d2bd220 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb72ffd700 1 -- 172.21.15.46:0/3453496901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb8400f040 con 0x55c60d2bd220 2024-03-18T20:42:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.801+0000 7fdb72ffd700 1 -- 172.21.15.46:0/3453496901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84007e00 con 0x55c60d2bd220 2024-03-18T20:42:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.802+0000 7fdb9e640b80 1 -- 172.21.15.46:0/3453496901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2bd220 msgr2=0x55c60d2ba8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.802+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/3453496901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2bd220 0x55c60d2ba8f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400ea90 tx=0x7fdb84009920 comp rx=0 tx=0).stop 2024-03-18T20:42:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.802+0000 7fdb9e640b80 1 -- 172.21.15.46:0/3453496901 shutdown_connections 2024-03-18T20:42:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.802+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/3453496901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2bd220 0x55c60d2ba8f0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.802+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/3453496901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2b1a60 0x55c60d2bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.802+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/3453496901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.802+0000 7fdb9e640b80 1 -- 172.21.15.46:0/3453496901 >> 172.21.15.46:0/3453496901 conn(0x55c60d228220 msgr2=0x55c60d2c85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.802+0000 7fdb9e640b80 1 -- 172.21.15.46:0/3453496901 shutdown_connections 2024-03-18T20:42:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.802+0000 7fdb9e640b80 1 -- 172.21.15.46:0/3453496901 wait complete. 2024-03-18T20:42:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb9e640b80 1 -- start start 2024-03-18T20:42:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb9e640b80 1 Processor -- start 2024-03-18T20:42:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb9e640b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb9e640b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb9e640b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 0x55c60d31f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb9e640b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60d2c2750 con 0x55c60d2b1a60 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb9e640b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60d31fc60 con 0x55c60d2bd220 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb9e640b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c60d31fe20 con 0x55c60d2b2380 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb9e640b80 10 client.0 _ll_register_callbacks cb 0x55c60d00c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2c1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59602/0 (socket says 172.21.15.46:59602) 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb8bfff700 1 -- 172.21.15.46:0/2771148372 learned_addr learned my addr 172.21.15.46:0/2771148372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb8b7fe700 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.803+0000 7fdb90a48700 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 0x55c60d31f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb8bfff700 1 -- 172.21.15.46:0/2771148372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 msgr2=0x55c60d2c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2c2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb8bfff700 1 -- 172.21.15.46:0/2771148372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 msgr2=0x55c60d31f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 0x55c60d31f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb8bfff700 1 -- 172.21.15.46:0/2771148372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8400e740 con 0x55c60d2b1a60 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb8bfff700 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2c1ac0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c09a1d0 tx=0x7fdb7c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66ed2e031f542d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb6b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb6b7fe700 1 -- 172.21.15.46:0/2771148372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c099930 con 0x55c60d2b1a60 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb6b7fe700 1 -- 172.21.15.46:0/2771148372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fdb7c09e070 con 0x55c60d2b1a60 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2771148372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb7c00cc90 con 0x55c60d2b1a60 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb6b7fe700 1 -- 172.21.15.46:0/2771148372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb7c09a010 con 0x55c60d2b1a60 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb9e640b80 10 client.17811 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2771148372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c60d08acd0 con 0x55c60d2b1a60 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.804+0000 7fdb6b7fe700 1 -- 172.21.15.46:0/2771148372 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb7c00cc90 con 0x55c60d2b1a60 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb6b7fe700 1 -- 172.21.15.46:0/2771148372 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdb7c09a4a0 con 0x55c60d2b1a60 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb9e640b80 10 client.17811 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb9e640b80 10 client.17811 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb9e640b80 20 client.17811 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c60d3210f0 0x55c60d3234e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb9e640b80 4 client.17811 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2771148372 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c60d053cf0 con 0x55c60d3210f0 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb8b7fe700 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c60d3210f0 0x55c60d3234e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb6b7fe700 10 client.17811 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb8b7fe700 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c60d3210f0 0x55c60d3234e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d46eac3b8a23a617 server_cookie=b8b5a345d710cee4 in_seq=0 out_seq=0 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb6b7fe700 1 -- 172.21.15.46:0/2771148372 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c60d053cf0 con 0x55c60d3210f0 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb6b7fe700 10 client.17811 handle_command_reply: tid=0 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.805+0000 7fdb9e640b80 1 client.17811 shutdown 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2771148372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 msgr2=0x55c60d2c1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2c1ac0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c09a1d0 tx=0x7fdb7c00cf80 comp rx=0 tx=0).stop 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2771148372 shutdown_connections 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c60d3210f0 0x55c60d3234e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c60d2b1a60 0x55c60d2c1ac0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c60d2bd220 0x55c60d31f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb6b7fe700 0 client.17811 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb9e640b80 1 --2- 172.21.15.46:0/2771148372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c60d2b2380 0x55c60d2c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2771148372 >> 172.21.15.46:0/2771148372 conn(0x55c60d228220 msgr2=0x55c60d2b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2771148372 shutdown_connections 2024-03-18T20:42:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb9e640b80 1 -- 172.21.15.46:0/2771148372 wait complete. 2024-03-18T20:42:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb9e640b80 20 client.17811 trim_cache size 0 max 16384 2024-03-18T20:42:33.814 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb924ab700 1 -- 172.21.15.46:0/3004186120 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdb740a1290 msgr2=0x7fdb740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb924ab700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdb740a1290 0x7fdb740a3750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000b2e0 tx=0x7fdb80005fd0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb924ab700 1 -- 172.21.15.46:0/3004186120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c100f70 msgr2=0x7fdb8c0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.806+0000 7fdb924ab700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c100f70 0x7fdb8c0730b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00eb30 tx=0x7fdb7c00ee40 comp rx=0 tx=0).stop 2024-03-18T20:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.807+0000 7fdb924ab700 1 -- 172.21.15.46:0/3004186120 shutdown_connections 2024-03-18T20:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.807+0000 7fdb924ab700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdb740a1290 0x7fdb740a3750 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.807+0000 7fdb924ab700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb8c073b30 0x7fdb8c1a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.807+0000 7fdb924ab700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c100f70 0x7fdb8c0730b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.807+0000 7fdb924ab700 1 --2- 172.21.15.46:0/3004186120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb8c106090 0x7fdb8c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.807+0000 7fdb924ab700 1 -- 172.21.15.46:0/3004186120 >> 172.21.15.46:0/3004186120 conn(0x7fdb8c0fc7f0 msgr2=0x7fdb8c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.807+0000 7fdb924ab700 1 -- 172.21.15.46:0/3004186120 shutdown_connections 2024-03-18T20:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.807+0000 7fdb924ab700 1 -- 172.21.15.46:0/3004186120 wait complete. 2024-03-18T20:42:33.823 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:42:33.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:42:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.960+0000 7fb4e0900700 1 -- 172.21.15.46:0/2102026077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4dc069010 msgr2=0x7fb4dc0693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.960+0000 7fb4e0900700 1 --2- 172.21.15.46:0/2102026077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4dc069010 0x7fb4dc0693f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0098f0 tx=0x7fb4cc009c00 comp rx=0 tx=0).stop 2024-03-18T20:42:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.960+0000 7fb4e0900700 1 -- 172.21.15.46:0/2102026077 shutdown_connections 2024-03-18T20:42:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.960+0000 7fb4e0900700 1 --2- 172.21.15.46:0/2102026077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4dc10d750 0x7fb4dc10fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.960+0000 7fb4e0900700 1 --2- 172.21.15.46:0/2102026077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4dc069010 0x7fb4dc0693f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.960+0000 7fb4e0900700 1 --2- 172.21.15.46:0/2102026077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc069930 0x7fb4dc10d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.960+0000 7fb4e0900700 1 -- 172.21.15.46:0/2102026077 >> 172.21.15.46:0/2102026077 conn(0x7fb4dc076d80 msgr2=0x7fb4dc077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.960+0000 7fb4e0900700 1 -- 172.21.15.46:0/2102026077 shutdown_connections 2024-03-18T20:42:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.960+0000 7fb4e0900700 1 -- 172.21.15.46:0/2102026077 wait complete. 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.961+0000 7fb4e0900700 1 Processor -- start 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.961+0000 7fb4e0900700 1 -- start start 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.961+0000 7fb4e0900700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4dc069010 0x7fb4dc1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.961+0000 7fb4e0900700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc069930 0x7fb4dc1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.961+0000 7fb4e0900700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4dc10d750 0x7fb4dc19dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.961+0000 7fb4e0900700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc19e3c0 con 0x7fb4dc069930 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.961+0000 7fb4e0900700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc19e530 con 0x7fb4dc069010 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.961+0000 7fb4e0900700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc19e6f0 con 0x7fb4dc10d750 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4da59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4dc069010 0x7fb4dc1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4d9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc069930 0x7fb4dc1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4da59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4dc069010 0x7fb4dc1994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40408/0 (socket says 172.21.15.46:40408) 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4da59c700 1 -- 172.21.15.46:0/1851749774 learned_addr learned my addr 172.21.15.46:0/1851749774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4dad9d700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4dc10d750 0x7fb4dc19dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4da59c700 1 -- 172.21.15.46:0/1851749774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4dc10d750 msgr2=0x7fb4dc19dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4dc10d750 0x7fb4dc19dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4da59c700 1 -- 172.21.15.46:0/1851749774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc069930 msgr2=0x7fb4dc1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc069930 0x7fb4dc1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4da59c700 1 -- 172.21.15.46:0/1851749774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc009520 con 0x7fb4dc069010 2024-03-18T20:42:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc069930 0x7fb4dc1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4dc069010 0x7fb4dc1994a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00fa00 tx=0x7fb4cc00fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4cb7fe700 1 -- 172.21.15.46:0/1851749774 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc01c070 con 0x7fb4dc069010 2024-03-18T20:42:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4e0900700 1 -- 172.21.15.46:0/1851749774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4dc19e970 con 0x7fb4dc069010 2024-03-18T20:42:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4cb7fe700 1 -- 172.21.15.46:0/1851749774 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4cc00bba0 con 0x7fb4dc069010 2024-03-18T20:42:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4cb7fe700 1 -- 172.21.15.46:0/1851749774 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc004930 con 0x7fb4dc069010 2024-03-18T20:42:33.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.962+0000 7fb4e0900700 1 -- 172.21.15.46:0/1851749774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc1a6210 con 0x7fb4dc069010 2024-03-18T20:42:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.964+0000 7fb4cb7fe700 1 -- 172.21.15.46:0/1851749774 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fb4cc020070 con 0x7fb4dc069010 2024-03-18T20:42:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.965+0000 7fb4cb7fe700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb4c40a1290 0x7fb4c40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.965+0000 7fb4cb7fe700 1 -- 172.21.15.46:0/1851749774 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb4cc0cd960 con 0x7fb4dc069010 2024-03-18T20:42:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.965+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb4c40a1290 0x7fb4c40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.966+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb4c40a1290 0x7fb4c40a3750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc19a9f0 tx=0x7fb4d0009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.966+0000 7fb4eca95b80 1 Processor -- start 2024-03-18T20:42:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.966+0000 7fb4eca95b80 1 -- start start 2024-03-18T20:42:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4eca95b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fe9da60 0x55f31fe9de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4eca95b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 0x55f31fea8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4eca95b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fea9220 0x55f31feab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4eca95b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31fdf7990 con 0x55f31fea9220 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4eca95b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31fdf5350 con 0x55f31fe9da60 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4eca95b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31fdf0c40 con 0x55f31fe9e380 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4da59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fe9da60 0x55f31fe9de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4dad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fea9220 0x55f31feab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4da59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fe9da60 0x55f31fe9de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40418/0 (socket says 172.21.15.46:40418) 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4da59c700 1 -- 172.21.15.46:0/1955423158 learned_addr learned my addr 172.21.15.46:0/1955423158 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4dad9d700 1 --2- 172.21.15.46:0/1955423158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fea9220 0x55f31feab610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59622/0 (socket says 172.21.15.46:59622) 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1955423158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 0x55f31fea8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4da59c700 1 -- 172.21.15.46:0/1955423158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 msgr2=0x55f31fea8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1955423158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 0x55f31fea8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4da59c700 1 -- 172.21.15.46:0/1955423158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fea9220 msgr2=0x55f31feab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1955423158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fea9220 0x55f31feab610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4da59c700 1 -- 172.21.15.46:0/1955423158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f31fc723b0 con 0x55f31fe9da60 2024-03-18T20:42:33.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1955423158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fe9da60 0x55f31fe9de40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0a2990 tx=0x7fb4cc0054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc5eed9087057246 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.967+0000 7fb4c8ff9700 1 -- 172.21.15.46:0/1955423158 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc09b1d0 con 0x55f31fe9da60 2024-03-18T20:42:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.968+0000 7fb4c8ff9700 1 -- 172.21.15.46:0/1955423158 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4cc09aad0 con 0x55f31fe9da60 2024-03-18T20:42:33.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.968+0000 7fb4c8ff9700 1 -- 172.21.15.46:0/1955423158 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc09ad30 con 0x55f31fe9da60 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.968+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1955423158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fe9da60 msgr2=0x55f31fe9de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.968+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1955423158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fe9da60 0x55f31fe9de40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0a2990 tx=0x7fb4cc0054e0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.968+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1955423158 shutdown_connections 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.968+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1955423158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fea9220 0x55f31feab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.968+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1955423158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fe9da60 0x55f31fe9de40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.968+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1955423158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 0x55f31fea8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.968+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1955423158 >> 172.21.15.46:0/1955423158 conn(0x55f31fe14220 msgr2=0x55f31fe9a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.968+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1955423158 shutdown_connections 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.968+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1955423158 wait complete. 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4eca95b80 1 -- start start 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4eca95b80 1 Processor -- start 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4eca95b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 0x55f31fea9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4eca95b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 0x55f31fea9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4eca95b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31feb4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4eca95b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31fd8cd80 con 0x55f31fe9da60 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4eca95b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31fd8ca20 con 0x55f31fea9220 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4eca95b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31fc746a0 con 0x55f31fe9e380 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4eca95b80 10 client.0 _ll_register_callbacks cb 0x55f31fbf83a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4da59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 0x55f31fea9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4da59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 0x55f31fea9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59638/0 (socket says 172.21.15.46:59638) 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4da59c700 1 -- 172.21.15.46:0/1610064824 learned_addr learned my addr 172.21.15.46:0/1610064824 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.969+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 0x55f31fea9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4dad9d700 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31feb4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4da59c700 1 -- 172.21.15.46:0/1610064824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 msgr2=0x55f31fea9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 0x55f31fea9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4da59c700 1 -- 172.21.15.46:0/1610064824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 msgr2=0x55f31feb4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31feb4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4da59c700 1 -- 172.21.15.46:0/1610064824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc1a6210 con 0x55f31fe9da60 2024-03-18T20:42:33.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4dad9d700 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31feb4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 0x55f31fea9600 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0098c0 tx=0x7fb4cc005370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130b5dfdd2627a9c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4c1ffb700 1 -- 172.21.15.46:0/1610064824 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0a2ec0 con 0x55f31fe9da60 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4c1ffb700 1 -- 172.21.15.46:0/1610064824 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb4cc09a1d0 con 0x55f31fe9da60 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1610064824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f31fe91ad0 con 0x55f31fe9da60 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4c1ffb700 1 -- 172.21.15.46:0/1610064824 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4cc066300 con 0x55f31fe9da60 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4eca95b80 10 client.17823 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1610064824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f31fc1eb60 con 0x55f31fe9da60 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.970+0000 7fb4c1ffb700 1 -- 172.21.15.46:0/1610064824 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4cc07d020 con 0x55f31fe9da60 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.971+0000 7fb4c1ffb700 1 -- 172.21.15.46:0/1610064824 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb4cc0a2a30 con 0x55f31fe9da60 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.971+0000 7fb4eca95b80 10 client.17823 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.971+0000 7fb4eca95b80 10 client.17823 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.971+0000 7fb4eca95b80 20 client.17823 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:33.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.971+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f31feb8560 0x55f31feba980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.971+0000 7fb4eca95b80 4 client.17823 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.971+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1610064824 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f31fde9700 con 0x55f31feb8560 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.971+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f31feb8560 0x55f31feba980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.971+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f31feb8560 0x55f31feba980 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9019795430756da2 server_cookie=81d0ae84cfa29080 in_seq=0 out_seq=0 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.971+0000 7fb4c1ffb700 10 client.17823 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4c1ffb700 1 -- 172.21.15.46:0/1610064824 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f31fde9700 con 0x55f31feb8560 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4c1ffb700 10 client.17823 handle_command_reply: tid=0 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4eca95b80 1 client.17823 shutdown 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1610064824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 msgr2=0x55f31fea9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 0x55f31fea9600 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0098c0 tx=0x7fb4cc005370 comp rx=0 tx=0).stop 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1610064824 shutdown_connections 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f31feb8560 0x55f31feba980 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 0x55f31fea9600 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31feb4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1610064824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 0x55f31fea9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4c1ffb700 0 client.17823 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:42:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1610064824 >> 172.21.15.46:0/1610064824 conn(0x55f31fe14220 msgr2=0x55f31fe99c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1610064824 shutdown_connections 2024-03-18T20:42:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1610064824 wait complete. 2024-03-18T20:42:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.972+0000 7fb4eca95b80 20 client.17823 trim_cache size 0 max 16384 2024-03-18T20:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.986+0000 7fb4eca95b80 1 Processor -- start 2024-03-18T20:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.986+0000 7fb4eca95b80 1 -- start start 2024-03-18T20:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.986+0000 7fb4eca95b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9da60 0x55f31feba950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.986+0000 7fb4eca95b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9e380 0x55f31feae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.986+0000 7fb4eca95b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31fea7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.986+0000 7fb4eca95b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31fe9ed30 con 0x55f31fe9e380 2024-03-18T20:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.986+0000 7fb4eca95b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31fe9b230 con 0x55f31fea9220 2024-03-18T20:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.986+0000 7fb4eca95b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31feaea80 con 0x55f31fe9da60 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.986+0000 7fb4d9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9e380 0x55f31feae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.986+0000 7fb4d9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9e380 0x55f31feae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59646/0 (socket says 172.21.15.46:59646) 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.986+0000 7fb4d9d9b700 1 -- 172.21.15.46:0/1153963750 learned_addr learned my addr 172.21.15.46:0/1153963750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.987+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1153963750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9da60 0x55f31feba950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.987+0000 7fb4dad9d700 1 --2- 172.21.15.46:0/1153963750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31fea7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.987+0000 7fb4d9d9b700 1 -- 172.21.15.46:0/1153963750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9da60 msgr2=0x55f31feba950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.987+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1153963750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9da60 0x55f31feba950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.987+0000 7fb4d9d9b700 1 -- 172.21.15.46:0/1153963750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 msgr2=0x55f31fea7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.987+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1153963750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31fea7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.987+0000 7fb4d9d9b700 1 -- 172.21.15.46:0/1153963750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f31fdb6ea0 con 0x55f31fe9e380 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.987+0000 7fb4dad9d700 1 --2- 172.21.15.46:0/1153963750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31fea7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.987+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1153963750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9e380 0x55f31feae540 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x55f31fea89f0 tx=0x7fb4d0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced5995f173d571c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.987+0000 7fb4c97fa700 1 -- 172.21.15.46:0/1153963750 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0024040 con 0x55f31fe9e380 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.987+0000 7fb4c97fa700 1 -- 172.21.15.46:0/1153963750 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d0029dc0 con 0x55f31fe9e380 2024-03-18T20:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.987+0000 7fb4c97fa700 1 -- 172.21.15.46:0/1153963750 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d001e7d0 con 0x55f31fe9e380 2024-03-18T20:42:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1153963750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9e380 msgr2=0x55f31feae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1153963750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9e380 0x55f31feae540 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x55f31fea89f0 tx=0x7fb4d0013040 comp rx=0 tx=0).stop 2024-03-18T20:42:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1153963750 shutdown_connections 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1153963750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31fea7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1153963750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9da60 0x55f31feba950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1153963750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9e380 0x55f31feae540 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1153963750 >> 172.21.15.46:0/1153963750 conn(0x55f31fe14220 msgr2=0x55f31fe98750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1153963750 shutdown_connections 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1153963750 wait complete. 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 -- start start 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 Processor -- start 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 0x55f31ff00510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.988+0000 7fb4eca95b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 0x55f31ff00a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4eca95b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31ff053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4eca95b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31ff05950 con 0x55f31fe9da60 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4eca95b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31ff05ac0 con 0x55f31fea9220 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4eca95b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f31ff05c80 con 0x55f31fe9e380 2024-03-18T20:42:33.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4eca95b80 10 client.0 _ll_register_callbacks cb 0x7fb4d4009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4da59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 0x55f31ff00510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4da59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 0x55f31ff00510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59662/0 (socket says 172.21.15.46:59662) 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4da59c700 1 -- 172.21.15.46:0/1307663215 learned_addr learned my addr 172.21.15.46:0/1307663215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4dad9d700 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31ff053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 0x55f31ff00a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4da59c700 1 -- 172.21.15.46:0/1307663215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 msgr2=0x55f31ff00a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 0x55f31ff00a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4da59c700 1 -- 172.21.15.46:0/1307663215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 msgr2=0x55f31ff053e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31ff053e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4da59c700 1 -- 172.21.15.46:0/1307663215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d0017070 con 0x55f31fe9da60 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4da59c700 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 0x55f31ff00510 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0094f0 tx=0x7fb4cc0054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1b0f03228442be server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4c1ffb700 1 -- 172.21.15.46:0/1307663215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0a2c30 con 0x55f31fe9da60 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4c1ffb700 1 -- 172.21.15.46:0/1307663215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fb4cc07e080 con 0x55f31fe9da60 2024-03-18T20:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.989+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1307663215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4cc09aad0 con 0x55f31fe9da60 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4c1ffb700 1 -- 172.21.15.46:0/1307663215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4cc09b1d0 con 0x55f31fe9da60 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4eca95b80 10 client.17829 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1307663215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f31fc76cd0 con 0x55f31fe9da60 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4c1ffb700 1 -- 172.21.15.46:0/1307663215 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4cc09aad0 con 0x55f31fe9da60 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4c1ffb700 1 -- 172.21.15.46:0/1307663215 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb4cc09ae10 con 0x55f31fe9da60 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4eca95b80 10 client.17829 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4eca95b80 10 client.17829 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4eca95b80 20 client.17829 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f31ff0cfd0 0x55f31ff0f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4eca95b80 4 client.17829 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1307663215 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f31fc3fcf0 con 0x55f31ff0cfd0 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f31ff0cfd0 0x55f31ff0f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:33.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4d9d9b700 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f31ff0cfd0 0x55f31ff0f3c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8e33578760d8d4b server_cookie=25ed21c6ced50114 in_seq=0 out_seq=0 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.990+0000 7fb4c1ffb700 10 client.17829 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4c1ffb700 1 -- 172.21.15.46:0/1307663215 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f31fc3fcf0 con 0x55f31ff0cfd0 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4c1ffb700 10 client.17829 handle_command_reply: tid=0 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4eca95b80 1 client.17829 shutdown 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1307663215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 msgr2=0x55f31ff00510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 0x55f31ff00510 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0094f0 tx=0x7fb4cc0054e0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1307663215 shutdown_connections 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f31ff0cfd0 0x55f31ff0f3c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f31fe9da60 0x55f31ff00510 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f31fea9220 0x55f31ff053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4eca95b80 1 --2- 172.21.15.46:0/1307663215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f31fe9e380 0x55f31ff00a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1307663215 >> 172.21.15.46:0/1307663215 conn(0x55f31fe14220 msgr2=0x55f31fea7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4c1ffb700 0 client.17829 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1307663215 shutdown_connections 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4eca95b80 1 -- 172.21.15.46:0/1307663215 wait complete. 2024-03-18T20:42:33.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.991+0000 7fb4eca95b80 20 client.17829 trim_cache size 0 max 16384 2024-03-18T20:42:34.000 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:42:34.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.992+0000 7fb4e0900700 1 -- 172.21.15.46:0/1851749774 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb4c40a1290 msgr2=0x7fb4c40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:34.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.992+0000 7fb4e0900700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb4c40a1290 0x7fb4c40a3750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc19a9f0 tx=0x7fb4d0009310 comp rx=0 tx=0).stop 2024-03-18T20:42:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.992+0000 7fb4e0900700 1 -- 172.21.15.46:0/1851749774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4dc069010 msgr2=0x7fb4dc1994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.992+0000 7fb4e0900700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4dc069010 0x7fb4dc1994a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00fa00 tx=0x7fb4cc00fa30 comp rx=0 tx=0).stop 2024-03-18T20:42:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.992+0000 7fb4e0900700 1 -- 172.21.15.46:0/1851749774 shutdown_connections 2024-03-18T20:42:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.992+0000 7fb4e0900700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb4c40a1290 0x7fb4c40a3750 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.992+0000 7fb4e0900700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4dc10d750 0x7fb4dc19dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.992+0000 7fb4e0900700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4dc069010 0x7fb4dc1994a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.992+0000 7fb4e0900700 1 --2- 172.21.15.46:0/1851749774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4dc069930 0x7fb4dc1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.992+0000 7fb4e0900700 1 -- 172.21.15.46:0/1851749774 >> 172.21.15.46:0/1851749774 conn(0x7fb4dc076d80 msgr2=0x7fb4dc10ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.992+0000 7fb4e0900700 1 -- 172.21.15.46:0/1851749774 shutdown_connections 2024-03-18T20:42:34.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:33.992+0000 7fb4e0900700 1 -- 172.21.15.46:0/1851749774 wait complete. 2024-03-18T20:42:34.008 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:42:34.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:42:34.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.144+0000 7f7fc266b700 1 -- 172.21.15.46:0/3758808939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc067e30 msgr2=0x7f7fbc0682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:34.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.144+0000 7f7fc266b700 1 --2- 172.21.15.46:0/3758808939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc067e30 0x7f7fbc0682b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009990 tx=0x7f7fa4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.144+0000 7f7fc266b700 1 -- 172.21.15.46:0/3758808939 shutdown_connections 2024-03-18T20:42:34.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.144+0000 7f7fc266b700 1 --2- 172.21.15.46:0/3758808939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fbc0687f0 0x7f7fbc10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.144+0000 7f7fc266b700 1 --2- 172.21.15.46:0/3758808939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fbc105ce0 0x7f7fbc1060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.144+0000 7f7fc266b700 1 --2- 172.21.15.46:0/3758808939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc067e30 0x7f7fbc0682b0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.144+0000 7f7fc266b700 1 -- 172.21.15.46:0/3758808939 >> 172.21.15.46:0/3758808939 conn(0x7f7fbc076d40 msgr2=0x7f7fbc077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.144+0000 7f7fc266b700 1 -- 172.21.15.46:0/3758808939 shutdown_connections 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.144+0000 7f7fc266b700 1 -- 172.21.15.46:0/3758808939 wait complete. 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.145+0000 7f7fc266b700 1 Processor -- start 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.145+0000 7f7fc266b700 1 -- start start 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.145+0000 7f7fc266b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc067e30 0x7f7fbc199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.145+0000 7f7fc266b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fbc0687f0 0x7f7fbc199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.145+0000 7f7fc266b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fbc105ce0 0x7f7fbc19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.145+0000 7f7fc266b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc19e330 con 0x7f7fbc067e30 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.145+0000 7f7fc266b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc19e4a0 con 0x7f7fbc0687f0 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.145+0000 7f7fc266b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc19e660 con 0x7f7fbc105ce0 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.145+0000 7f7fbbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc067e30 0x7f7fbc199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.145+0000 7f7fbbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc067e30 0x7f7fbc199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59690/0 (socket says 172.21.15.46:59690) 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.145+0000 7f7fbbfff700 1 -- 172.21.15.46:0/2958853261 learned_addr learned my addr 172.21.15.46:0/2958853261 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:34.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fc0c08700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fbc105ce0 0x7f7fbc19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fb3fff700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fbc0687f0 0x7f7fbc199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fbbfff700 1 -- 172.21.15.46:0/2958853261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fbc105ce0 msgr2=0x7f7fbc19dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fbc105ce0 0x7f7fbc19dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fbbfff700 1 -- 172.21.15.46:0/2958853261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fbc0687f0 msgr2=0x7f7fbc199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fbc0687f0 0x7f7fbc199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fbbfff700 1 -- 172.21.15.46:0/2958853261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa40095f0 con 0x7f7fbc067e30 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fc0c08700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fbc105ce0 0x7f7fbc19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fb3fff700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fbc0687f0 0x7f7fbc199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc067e30 0x7f7fbc199410 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00e9a0 tx=0x7f7fac00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fb9ffb700 1 -- 172.21.15.46:0/2958853261 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac009790 con 0x7f7fbc067e30 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fb9ffb700 1 -- 172.21.15.46:0/2958853261 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fac00cbd0 con 0x7f7fbc067e30 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fc266b700 1 -- 172.21.15.46:0/2958853261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fbc19e940 con 0x7f7fbc067e30 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fb9ffb700 1 -- 172.21.15.46:0/2958853261 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac018680 con 0x7f7fbc067e30 2024-03-18T20:42:34.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.146+0000 7f7fc266b700 1 -- 172.21.15.46:0/2958853261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fbc1a6180 con 0x7f7fbc067e30 2024-03-18T20:42:34.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.149+0000 7f7fb9ffb700 1 -- 172.21.15.46:0/2958853261 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7fac00cd40 con 0x7f7fbc067e30 2024-03-18T20:42:34.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.149+0000 7f7fb9ffb700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7fa80a1290 0x7f7fa80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.149+0000 7f7fb9ffb700 1 -- 172.21.15.46:0/2958853261 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f7fac014070 con 0x7f7fbc067e30 2024-03-18T20:42:34.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.149+0000 7f7fb3fff700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7fa80a1290 0x7f7fa80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fce800b80 1 Processor -- start 2024-03-18T20:42:34.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fce800b80 1 -- start start 2024-03-18T20:42:34.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fb3fff700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7fa80a1290 0x7f7fa80a3750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400b2e0 tx=0x7f7fa4005fd0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:34.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fce800b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124089a60 0x55e124089e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fce800b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e124094ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fce800b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124095220 0x55e124097610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fce800b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e123fe3990 con 0x55e12408a380 2024-03-18T20:42:34.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fce800b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e123fe1350 con 0x55e124089a60 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fce800b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e123fdcc40 con 0x55e124095220 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fb3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e124094ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fb3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e124094ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59706/0 (socket says 172.21.15.46:59706) 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fbbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124089a60 0x55e124089e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fb3fff700 1 -- 172.21.15.46:0/2368504365 learned_addr learned my addr 172.21.15.46:0/2368504365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.150+0000 7f7fc0c08700 1 --2- 172.21.15.46:0/2368504365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124095220 0x55e124097610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.151+0000 7f7fbbfff700 1 -- 172.21.15.46:0/2368504365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124095220 msgr2=0x55e124097610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.151+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/2368504365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124095220 0x55e124097610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.151+0000 7f7fbbfff700 1 -- 172.21.15.46:0/2368504365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 msgr2=0x55e124094ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.151+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/2368504365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e124094ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.151+0000 7f7fbbfff700 1 -- 172.21.15.46:0/2368504365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e123e5e3b0 con 0x55e124089a60 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.151+0000 7f7fb3fff700 1 --2- 172.21.15.46:0/2368504365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e124094ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.151+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/2368504365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124089a60 0x55e124089e40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0ca6f0 tx=0x7f7fac09a880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44d7879830b14726 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.151+0000 7f7fb27fc700 1 -- 172.21.15.46:0/2368504365 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0a2530 con 0x55e124089a60 2024-03-18T20:42:34.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.151+0000 7f7fb27fc700 1 -- 172.21.15.46:0/2368504365 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fac09a310 con 0x55e124089a60 2024-03-18T20:42:34.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.151+0000 7f7fb27fc700 1 -- 172.21.15.46:0/2368504365 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0a22c0 con 0x55e124089a60 2024-03-18T20:42:34.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.152+0000 7f7fce800b80 1 -- 172.21.15.46:0/2368504365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124089a60 msgr2=0x55e124089e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:34.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.152+0000 7f7fce800b80 1 --2- 172.21.15.46:0/2368504365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124089a60 0x55e124089e40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0ca6f0 tx=0x7f7fac09a880 comp rx=0 tx=0).stop 2024-03-18T20:42:34.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.152+0000 7f7fce800b80 1 -- 172.21.15.46:0/2368504365 shutdown_connections 2024-03-18T20:42:34.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.152+0000 7f7fce800b80 1 --2- 172.21.15.46:0/2368504365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124095220 0x55e124097610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.152+0000 7f7fce800b80 1 --2- 172.21.15.46:0/2368504365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124089a60 0x55e124089e40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.152+0000 7f7fce800b80 1 --2- 172.21.15.46:0/2368504365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e124094ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.152+0000 7f7fce800b80 1 -- 172.21.15.46:0/2368504365 >> 172.21.15.46:0/2368504365 conn(0x55e124000220 msgr2=0x55e124086020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.152+0000 7f7fce800b80 1 -- 172.21.15.46:0/2368504365 shutdown_connections 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.152+0000 7f7fce800b80 1 -- 172.21.15.46:0/2368504365 wait complete. 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.152+0000 7f7fce800b80 1 -- start start 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.152+0000 7f7fce800b80 1 Processor -- start 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fce800b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124089a60 0x55e1240920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fce800b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e124092630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fce800b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124095220 0x55e124092b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fce800b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e123f78d80 con 0x55e12408a380 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fce800b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e123f78a20 con 0x55e124095220 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fce800b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e123e606a0 con 0x55e124089a60 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fce800b80 10 client.0 _ll_register_callbacks cb 0x55e123de43a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:34.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fb3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e124092630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fb3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e124092630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59718/0 (socket says 172.21.15.46:59718) 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fb3fff700 1 -- 172.21.15.46:0/3322007586 learned_addr learned my addr 172.21.15.46:0/3322007586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fc0c08700 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124095220 0x55e124092b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124089a60 0x55e1240920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fb3fff700 1 -- 172.21.15.46:0/3322007586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124089a60 msgr2=0x55e1240920f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fb3fff700 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124089a60 0x55e1240920f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fb3fff700 1 -- 172.21.15.46:0/3322007586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124095220 msgr2=0x55e124092b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fb3fff700 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124095220 0x55e124092b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fb3fff700 1 -- 172.21.15.46:0/3322007586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fbc10e550 con 0x55e12408a380 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7fb3fff700 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e124092630 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400b6f0 tx=0x7f7fa401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdcf88b4b115536 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.153+0000 7f7f9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7f9b7fe700 1 -- 172.21.15.46:0/3322007586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa402b050 con 0x55e12408a380 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7f9b7fe700 1 -- 172.21.15.46:0/3322007586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f7fa4024610 con 0x55e12408a380 2024-03-18T20:42:34.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7fce800b80 1 -- 172.21.15.46:0/3322007586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e12407dad0 con 0x55e12408a380 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7f9b7fe700 1 -- 172.21.15.46:0/3322007586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fa400fe80 con 0x55e12408a380 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7fce800b80 10 client.17841 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7fce800b80 1 -- 172.21.15.46:0/3322007586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f7fbc1a6180 con 0x55e12408a380 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7f9b7fe700 1 -- 172.21.15.46:0/3322007586 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e12407dad0 con 0x55e12408a380 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7f9b7fe700 1 -- 172.21.15.46:0/3322007586 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7fa4030a00 con 0x55e12408a380 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7fce800b80 10 client.17841 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7fce800b80 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7fce800b80 20 client.17841 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7fce800b80 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e1240a5000 0x55e1240a7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7fce800b80 4 client.17841 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.154+0000 7f7fce800b80 1 -- 172.21.15.46:0/3322007586 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e123fd5700 con 0x55e1240a5000 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.155+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e1240a5000 0x55e1240a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.155+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e1240a5000 0x55e1240a7420 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cce7d8515eb99ebb server_cookie=4734da17a0e260ee in_seq=0 out_seq=0 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.155+0000 7f7f9b7fe700 10 client.17841 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:34.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7f9b7fe700 1 -- 172.21.15.46:0/3322007586 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e123fd5700 con 0x55e1240a5000 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7f9b7fe700 10 client.17841 handle_command_reply: tid=0 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7fce800b80 1 client.17841 shutdown 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7fce800b80 1 -- 172.21.15.46:0/3322007586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 msgr2=0x55e124092630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7fce800b80 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e124092630 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400b6f0 tx=0x7f7fa401a050 comp rx=0 tx=0).stop 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7fce800b80 1 -- 172.21.15.46:0/3322007586 shutdown_connections 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7fce800b80 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e1240a5000 0x55e1240a7420 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7fce800b80 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124095220 0x55e124092b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7fce800b80 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124089a60 0x55e1240920f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7fce800b80 1 --2- 172.21.15.46:0/3322007586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e124092630 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7f9b7fe700 0 client.17841 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7fce800b80 1 -- 172.21.15.46:0/3322007586 >> 172.21.15.46:0/3322007586 conn(0x55e124000220 msgr2=0x55e124085c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7fce800b80 1 -- 172.21.15.46:0/3322007586 shutdown_connections 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7fce800b80 1 -- 172.21.15.46:0/3322007586 wait complete. 2024-03-18T20:42:34.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.156+0000 7f7fce800b80 20 client.17841 trim_cache size 0 max 16384 2024-03-18T20:42:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.172+0000 7f7fce800b80 1 Processor -- start 2024-03-18T20:42:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.172+0000 7f7fce800b80 1 -- start start 2024-03-18T20:42:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.172+0000 7f7fce800b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124089a60 0x55e12408c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.172+0000 7f7fce800b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e12408a380 0x55e124092990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.172+0000 7f7fce800b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e124095220 0x55e12408f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.172+0000 7f7fce800b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1240a1a50 con 0x55e124095220 2024-03-18T20:42:34.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.172+0000 7f7fce800b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1240936a0 con 0x55e12408a380 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.172+0000 7f7fce800b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1240a25e0 con 0x55e124089a60 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.172+0000 7f7fbbfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124089a60 0x55e12408c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.172+0000 7f7fbbfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124089a60 0x55e12408c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:38266/0 (socket says 172.21.15.46:38266) 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.172+0000 7f7fbbfff700 1 -- 172.21.15.46:0/90658200 learned_addr learned my addr 172.21.15.46:0/90658200 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.172+0000 7f7fc0c08700 1 --2- 172.21.15.46:0/90658200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e124095220 0x55e12408f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.173+0000 7f7fbbfff700 1 -- 172.21.15.46:0/90658200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e12408a380 msgr2=0x55e124092990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.173+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/90658200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e12408a380 0x55e124092990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.173+0000 7f7fbbfff700 1 -- 172.21.15.46:0/90658200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e124095220 msgr2=0x55e12408f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.173+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/90658200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e124095220 0x55e12408f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.173+0000 7f7fbbfff700 1 -- 172.21.15.46:0/90658200 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e123e0ab60 con 0x55e124089a60 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.173+0000 7f7fc0c08700 1 --2- 172.21.15.46:0/90658200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e124095220 0x55e12408f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.173+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/90658200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124089a60 0x55e12408c9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7fac000c00 tx=0x7f7fac099f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=783f5cfcaee81bb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.173+0000 7f7fb2ffd700 1 -- 172.21.15.46:0/90658200 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac09a480 con 0x55e124089a60 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.173+0000 7f7fb2ffd700 1 -- 172.21.15.46:0/90658200 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fac0b8300 con 0x55e124089a60 2024-03-18T20:42:34.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.173+0000 7f7fb2ffd700 1 -- 172.21.15.46:0/90658200 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac099870 con 0x55e124089a60 2024-03-18T20:42:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.174+0000 7f7fce800b80 1 -- 172.21.15.46:0/90658200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124089a60 msgr2=0x55e12408c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.174+0000 7f7fce800b80 1 --2- 172.21.15.46:0/90658200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124089a60 0x55e12408c9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7fac000c00 tx=0x7f7fac099f00 comp rx=0 tx=0).stop 2024-03-18T20:42:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.174+0000 7f7fce800b80 1 -- 172.21.15.46:0/90658200 shutdown_connections 2024-03-18T20:42:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.174+0000 7f7fce800b80 1 --2- 172.21.15.46:0/90658200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e124095220 0x55e12408f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.174+0000 7f7fce800b80 1 --2- 172.21.15.46:0/90658200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124089a60 0x55e12408c9c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.174+0000 7f7fce800b80 1 --2- 172.21.15.46:0/90658200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e12408a380 0x55e124092990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.174+0000 7f7fce800b80 1 -- 172.21.15.46:0/90658200 >> 172.21.15.46:0/90658200 conn(0x55e124000220 msgr2=0x55e1240909c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.174+0000 7f7fce800b80 1 -- 172.21.15.46:0/90658200 shutdown_connections 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.174+0000 7f7fce800b80 1 -- 172.21.15.46:0/90658200 wait complete. 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.175+0000 7f7fce800b80 1 -- start start 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.175+0000 7f7fce800b80 1 Processor -- start 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.175+0000 7f7fce800b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124089a60 0x55e124099e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.175+0000 7f7fce800b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e12409a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.175+0000 7f7fce800b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124095220 0x55e12409a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.175+0000 7f7fce800b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1240f7f10 con 0x55e12408a380 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.175+0000 7f7fce800b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1240f8080 con 0x55e124089a60 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.175+0000 7f7fce800b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1240f8240 con 0x55e124095220 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.175+0000 7f7fce800b80 10 client.0 _ll_register_callbacks cb 0x55e123de43a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.175+0000 7f7fb3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e12409a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fb3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e12409a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59730/0 (socket says 172.21.15.46:59730) 2024-03-18T20:42:34.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fbbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124089a60 0x55e124099e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fb3fff700 1 -- 172.21.15.46:0/1283663877 learned_addr learned my addr 172.21.15.46:0/1283663877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fc0c08700 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124095220 0x55e12409a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fbbfff700 1 -- 172.21.15.46:0/1283663877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124095220 msgr2=0x55e12409a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124095220 0x55e12409a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fbbfff700 1 -- 172.21.15.46:0/1283663877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 msgr2=0x55e12409a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e12409a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fbbfff700 1 -- 172.21.15.46:0/1283663877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb4009370 con 0x55e124089a60 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fb3fff700 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e12409a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fbbfff700 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124089a60 0x55e124099e30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7fac09a040 tx=0x7f7fac09e060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=210387735955a567 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7f9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7f9b7fe700 1 -- 172.21.15.46:0/1283663877 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac065030 con 0x55e124089a60 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7f9b7fe700 1 -- 172.21.15.46:0/1283663877 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f7fac07a070 con 0x55e124089a60 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fce800b80 1 -- 172.21.15.46:0/1283663877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fac09a310 con 0x55e124089a60 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7f9b7fe700 1 -- 172.21.15.46:0/1283663877 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fac09a7b0 con 0x55e124089a60 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fce800b80 10 client.27524 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:34.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.176+0000 7f7fce800b80 1 -- 172.21.15.46:0/1283663877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55e123e5e3b0 con 0x55e124089a60 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.177+0000 7f7f9b7fe700 1 -- 172.21.15.46:0/1283663877 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fac09a310 con 0x55e124089a60 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.177+0000 7f7f9b7fe700 1 -- 172.21.15.46:0/1283663877 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7fac0b8020 con 0x55e124089a60 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.177+0000 7f7fce800b80 10 client.27524 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.177+0000 7f7fce800b80 10 client.27524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.177+0000 7f7fce800b80 20 client.27524 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.177+0000 7f7fce800b80 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e1240f9600 0x55e1240fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.177+0000 7f7fce800b80 4 client.27524 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.177+0000 7f7fce800b80 1 -- 172.21.15.46:0/1283663877 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e123e2bcf0 con 0x55e1240f9600 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.177+0000 7f7fb3fff700 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e1240f9600 0x55e1240fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fb3fff700 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e1240f9600 0x55e1240fb9f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e14a532428305cb server_cookie=1b9037a14428171c in_seq=0 out_seq=0 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7f9b7fe700 10 client.27524 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7f9b7fe700 1 -- 172.21.15.46:0/1283663877 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e123e2bcf0 con 0x55e1240f9600 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7f9b7fe700 10 client.27524 handle_command_reply: tid=0 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fce800b80 1 client.27524 shutdown 2024-03-18T20:42:34.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fce800b80 1 -- 172.21.15.46:0/1283663877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124089a60 msgr2=0x55e124099e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:34.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fce800b80 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124089a60 0x55e124099e30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7fac09a040 tx=0x7f7fac09e060 comp rx=0 tx=0).stop 2024-03-18T20:42:34.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fce800b80 1 -- 172.21.15.46:0/1283663877 shutdown_connections 2024-03-18T20:42:34.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fce800b80 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e1240f9600 0x55e1240fb9f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fce800b80 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e124095220 0x55e12409a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fce800b80 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e124089a60 0x55e124099e30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7f9b7fe700 0 client.27524 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:34.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fce800b80 1 --2- 172.21.15.46:0/1283663877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e12408a380 0x55e12409a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fce800b80 1 -- 172.21.15.46:0/1283663877 >> 172.21.15.46:0/1283663877 conn(0x55e124000220 msgr2=0x55e1240938d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:34.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fce800b80 1 -- 172.21.15.46:0/1283663877 shutdown_connections 2024-03-18T20:42:34.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fce800b80 1 -- 172.21.15.46:0/1283663877 wait complete. 2024-03-18T20:42:34.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.178+0000 7f7fce800b80 20 client.27524 trim_cache size 0 max 16384 2024-03-18T20:42:34.185 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:42:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.179+0000 7f7fc266b700 1 -- 172.21.15.46:0/2958853261 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7fa80a1290 msgr2=0x7f7fa80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.179+0000 7f7fc266b700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7fa80a1290 0x7f7fa80a3750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400b2e0 tx=0x7f7fa4005fd0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.179+0000 7f7fc266b700 1 -- 172.21.15.46:0/2958853261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc067e30 msgr2=0x7f7fbc199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.179+0000 7f7fc266b700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc067e30 0x7f7fbc199410 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00e9a0 tx=0x7f7fac00ed60 comp rx=0 tx=0).stop 2024-03-18T20:42:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.179+0000 7f7fc266b700 1 -- 172.21.15.46:0/2958853261 shutdown_connections 2024-03-18T20:42:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.179+0000 7f7fc266b700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7fa80a1290 0x7f7fa80a3750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.179+0000 7f7fc266b700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fbc067e30 0x7f7fbc199410 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.179+0000 7f7fc266b700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7fbc105ce0 0x7f7fbc19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.179+0000 7f7fc266b700 1 --2- 172.21.15.46:0/2958853261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fbc0687f0 0x7f7fbc199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.179+0000 7f7fc266b700 1 -- 172.21.15.46:0/2958853261 >> 172.21.15.46:0/2958853261 conn(0x7f7fbc076d40 msgr2=0x7f7fbc102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.179+0000 7f7fc266b700 1 -- 172.21.15.46:0/2958853261 shutdown_connections 2024-03-18T20:42:34.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:34.179+0000 7f7fc266b700 1 -- 172.21.15.46:0/2958853261 wait complete. 2024-03-18T20:42:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:34 smithi073 ceph-mon[47384]: pgmap v197: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-03-18T20:42:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:34 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3121456954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:42:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:34 smithi155 ceph-mon[48344]: pgmap v197: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-03-18T20:42:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:34 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3121456954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:42:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:34 smithi046 ceph-mon[43495]: pgmap v197: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 2.2 MiB/s wr, 211 op/s 2024-03-18T20:42:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:34 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3121456954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:42:34.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 214512 3.41 MB/sec execute 209 sec latency 2177.877 ms 2024-03-18T20:42:35.198 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:42:35.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:42:35.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:42:35.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.348+0000 7fc15c347700 1 -- 172.21.15.46:0/1510810340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc154069ef0 msgr2=0x7fc154105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.348+0000 7fc15c347700 1 --2- 172.21.15.46:0/1510810340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc154069ef0 0x7fc154105970 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc1440099c0 tx=0x7fc144009cd0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.348+0000 7fc15c347700 1 -- 172.21.15.46:0/1510810340 shutdown_connections 2024-03-18T20:42:35.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.348+0000 7fc15c347700 1 --2- 172.21.15.46:0/1510810340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1541060a0 0x7fc15410dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.348+0000 7fc15c347700 1 --2- 172.21.15.46:0/1510810340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc154069540 0x7fc154069920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.348+0000 7fc15c347700 1 --2- 172.21.15.46:0/1510810340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc154069ef0 0x7fc154105970 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.348+0000 7fc15c347700 1 -- 172.21.15.46:0/1510810340 >> 172.21.15.46:0/1510810340 conn(0x7fc154076d50 msgr2=0x7fc154077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.348+0000 7fc15c347700 1 -- 172.21.15.46:0/1510810340 shutdown_connections 2024-03-18T20:42:35.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.348+0000 7fc15c347700 1 -- 172.21.15.46:0/1510810340 wait complete. 2024-03-18T20:42:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc15c347700 1 Processor -- start 2024-03-18T20:42:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc15c347700 1 -- start start 2024-03-18T20:42:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc15c347700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc154069540 0x7fc1541971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc15c347700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1541060a0 0x7fc154197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc15c347700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc15419baa0 0x7fc15419bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc15c347700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15419c520 con 0x7fc15419baa0 2024-03-18T20:42:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc15c347700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15419c690 con 0x7fc1541060a0 2024-03-18T20:42:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc15c347700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15419c850 con 0x7fc154069540 2024-03-18T20:42:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc15a8e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc15419baa0 0x7fc15419bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc15a8e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc15419baa0 0x7fc15419bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58148/0 (socket says 172.21.15.46:58148) 2024-03-18T20:42:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc15a8e4700 1 -- 172.21.15.46:0/2571946168 learned_addr learned my addr 172.21.15.46:0/2571946168 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:35.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc1598e2700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1541060a0 0x7fc154197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.349+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc154069540 0x7fc1541971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.350+0000 7fc15a8e4700 1 -- 172.21.15.46:0/2571946168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc154069540 msgr2=0x7fc1541971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.350+0000 7fc15a8e4700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc154069540 0x7fc1541971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.350+0000 7fc15a8e4700 1 -- 172.21.15.46:0/2571946168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1541060a0 msgr2=0x7fc154197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.350+0000 7fc15a8e4700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1541060a0 0x7fc154197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.350+0000 7fc15a8e4700 1 -- 172.21.15.46:0/2571946168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1440095f0 con 0x7fc15419baa0 2024-03-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.350+0000 7fc15a8e4700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc15419baa0 0x7fc15419bf20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00ef30 tx=0x7fc14c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.350+0000 7fc14b7fe700 1 -- 172.21.15.46:0/2571946168 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c018070 con 0x7fc15419baa0 2024-03-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.350+0000 7fc14b7fe700 1 -- 172.21.15.46:0/2571946168 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc14c00f040 con 0x7fc15419baa0 2024-03-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.350+0000 7fc15c347700 1 -- 172.21.15.46:0/2571946168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1541a3d50 con 0x7fc15419baa0 2024-03-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.350+0000 7fc14b7fe700 1 -- 172.21.15.46:0/2571946168 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c00cd00 con 0x7fc15419baa0 2024-03-18T20:42:35.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.350+0000 7fc15c347700 1 -- 172.21.15.46:0/2571946168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1541a41e0 con 0x7fc15419baa0 2024-03-18T20:42:35.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.353+0000 7fc14b7fe700 1 -- 172.21.15.46:0/2571946168 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc14c003a00 con 0x7fc15419baa0 2024-03-18T20:42:35.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.353+0000 7fc14b7fe700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1400a1290 0x7fc1400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.353+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1400a1290 0x7fc1400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.353+0000 7fc14b7fe700 1 -- 172.21.15.46:0/2571946168 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc14c0cc7c0 con 0x7fc15419baa0 2024-03-18T20:42:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.354+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1400a1290 0x7fc1400a3750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc150009c60 tx=0x7fc150009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.354+0000 7fc1684dcb80 1 Processor -- start 2024-03-18T20:42:35.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.354+0000 7fc1684dcb80 1 -- start start 2024-03-18T20:42:35.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc1684dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6bae0 0x5577dfd6bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc1684dcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 0x5577dfd7b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc1684dcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd7b6b0 0x5577dfd7daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc1684dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577dfcc5a10 con 0x5577dfd6bae0 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc1684dcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577dfcc33d0 con 0x5577dfd7b6b0 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc1684dcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577dfcbecc0 con 0x5577dfd6c400 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc15a0e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6bae0 0x5577dfd6bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc15a0e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6bae0 0x5577dfd6bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58158/0 (socket says 172.21.15.46:58158) 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc15a0e3700 1 -- 172.21.15.46:0/782727482 learned_addr learned my addr 172.21.15.46:0/782727482 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc1598e2700 1 --2- 172.21.15.46:0/782727482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 0x5577dfd7b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc15a8e4700 1 --2- 172.21.15.46:0/782727482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd7b6b0 0x5577dfd7daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc15a0e3700 1 -- 172.21.15.46:0/782727482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 msgr2=0x5577dfd7b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/782727482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 0x5577dfd7b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc15a0e3700 1 -- 172.21.15.46:0/782727482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd7b6b0 msgr2=0x5577dfd7daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/782727482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd7b6b0 0x5577dfd7daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc15a0e3700 1 -- 172.21.15.46:0/782727482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577dfb40430 con 0x5577dfd6bae0 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.355+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/782727482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6bae0 0x5577dfd6bec0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc15000e4e0 tx=0x7fc150014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae10adf6d9de054 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.356+0000 7fc148da8700 1 -- 172.21.15.46:0/782727482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15000ee00 con 0x5577dfd6bae0 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.356+0000 7fc148da8700 1 -- 172.21.15.46:0/782727482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc150021070 con 0x5577dfd6bae0 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.356+0000 7fc148da8700 1 -- 172.21.15.46:0/782727482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15001d7b0 con 0x5577dfd6bae0 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.356+0000 7fc1684dcb80 1 -- 172.21.15.46:0/782727482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6bae0 msgr2=0x5577dfd6bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.356+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/782727482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6bae0 0x5577dfd6bec0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc15000e4e0 tx=0x7fc150014000 comp rx=0 tx=0).stop 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.356+0000 7fc1684dcb80 1 -- 172.21.15.46:0/782727482 shutdown_connections 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.356+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/782727482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6bae0 0x5577dfd6bec0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.356+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/782727482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd7b6b0 0x5577dfd7daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.356+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/782727482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 0x5577dfd7b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.356+0000 7fc1684dcb80 1 -- 172.21.15.46:0/782727482 >> 172.21.15.46:0/782727482 conn(0x5577dfce22a0 msgr2=0x5577dfd680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:35.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.357+0000 7fc1684dcb80 1 -- 172.21.15.46:0/782727482 shutdown_connections 2024-03-18T20:42:35.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.357+0000 7fc1684dcb80 1 -- 172.21.15.46:0/782727482 wait complete. 2024-03-18T20:42:35.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.357+0000 7fc1684dcb80 1 -- start start 2024-03-18T20:42:35.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.357+0000 7fc1684dcb80 1 Processor -- start 2024-03-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.357+0000 7fc1684dcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 0x5577dfd7b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.357+0000 7fc1684dcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 0x5577dfd78410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.357+0000 7fc1684dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 0x5577dfd78b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.357+0000 7fc1684dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577dfc5ae00 con 0x5577dfd7b6b0 2024-03-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.357+0000 7fc1684dcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577dfc5aaa0 con 0x5577dfd6bae0 2024-03-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.357+0000 7fc1684dcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577dfb42720 con 0x5577dfd6c400 2024-03-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.357+0000 7fc1684dcb80 10 client.0 _ll_register_callbacks cb 0x5577dfac6420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc15a0e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 0x5577dfd7b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc15a8e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 0x5577dfd78b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc15a0e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 0x5577dfd7b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:39354/0 (socket says 172.21.15.46:39354) 2024-03-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc15a0e3700 1 -- 172.21.15.46:0/1393808290 learned_addr learned my addr 172.21.15.46:0/1393808290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc1598e2700 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 0x5577dfd78410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc15a8e4700 1 -- 172.21.15.46:0/1393808290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 msgr2=0x5577dfd78410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc15a8e4700 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 0x5577dfd78410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc15a8e4700 1 -- 172.21.15.46:0/1393808290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 msgr2=0x5577dfd7b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc15a8e4700 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 0x5577dfd7b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc15a8e4700 1 -- 172.21.15.46:0/1393808290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc150017070 con 0x5577dfd7b6b0 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 0x5577dfd7b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc15a8e4700 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 0x5577dfd78b50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0c9f20 tx=0x7fc14c0992d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5724599ece954 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc13dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc13dffb700 1 -- 172.21.15.46:0/1393808290 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c09a070 con 0x5577dfd7b6b0 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc13dffb700 1 -- 172.21.15.46:0/1393808290 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc14c00ce60 con 0x5577dfd7b6b0 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1393808290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577dfd5fb50 con 0x5577dfd7b6b0 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc13dffb700 1 -- 172.21.15.46:0/1393808290 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc14c099970 con 0x5577dfd7b6b0 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc1684dcb80 10 client.17859 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.358+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1393808290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5577dfaecbe0 con 0x5577dfd7b6b0 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.359+0000 7fc13dffb700 1 -- 172.21.15.46:0/1393808290 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577dfd5fb50 con 0x5577dfd7b6b0 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.359+0000 7fc13dffb700 1 -- 172.21.15.46:0/1393808290 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc14c098e60 con 0x5577dfd7b6b0 2024-03-18T20:42:35.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.359+0000 7fc1684dcb80 10 client.17859 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.359+0000 7fc1684dcb80 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.359+0000 7fc1684dcb80 20 client.17859 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.359+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577dfd8b110 0x5577dfd8d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.359+0000 7fc1684dcb80 4 client.17859 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.359+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1393808290 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577dfcb7780 con 0x5577dfd8b110 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.359+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577dfd8b110 0x5577dfd8d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.360+0000 7fc13dffb700 10 client.17859 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.360+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577dfd8b110 0x5577dfd8d530 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea35e9d7ca42a12b server_cookie=1c41b772cbdf3ee1 in_seq=0 out_seq=0 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc13dffb700 1 -- 172.21.15.46:0/1393808290 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5577dfcb7780 con 0x5577dfd8b110 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc13dffb700 10 client.17859 handle_command_reply: tid=0 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc1684dcb80 1 client.17859 shutdown 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1393808290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 msgr2=0x5577dfd78b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 0x5577dfd78b50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0c9f20 tx=0x7fc14c0992d0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1393808290 shutdown_connections 2024-03-18T20:42:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577dfd8b110 0x5577dfd8d530 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 0x5577dfd78b50 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 0x5577dfd7b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/1393808290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 0x5577dfd78410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc13dffb700 0 client.17859 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1393808290 >> 172.21.15.46:0/1393808290 conn(0x5577dfce22a0 msgr2=0x5577dfd67c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1393808290 shutdown_connections 2024-03-18T20:42:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.361+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1393808290 wait complete. 2024-03-18T20:42:35.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.362+0000 7fc1684dcb80 20 client.17859 trim_cache size 0 max 16384 2024-03-18T20:42:35.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.379+0000 7fc1684dcb80 1 Processor -- start 2024-03-18T20:42:35.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.379+0000 7fc1684dcb80 1 -- start start 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc1684dcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 0x5577dfd822e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc1684dcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 0x5577dfd7ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc1684dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 0x5577dfd7b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc1684dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577dfd793c0 con 0x5577dfd7b6b0 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc1684dcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577dfd329f0 con 0x5577dfd6bae0 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc1684dcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577dfd799e0 con 0x5577dfd6c400 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc15a8e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 0x5577dfd7b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc15a8e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 0x5577dfd7b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58172/0 (socket says 172.21.15.46:58172) 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc15a0e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 0x5577dfd822e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc15a8e4700 1 -- 172.21.15.46:0/3242542204 learned_addr learned my addr 172.21.15.46:0/3242542204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc1598e2700 1 --2- 172.21.15.46:0/3242542204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 0x5577dfd7ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc15a0e3700 1 -- 172.21.15.46:0/3242542204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 msgr2=0x5577dfd7ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/3242542204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 0x5577dfd7ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc15a0e3700 1 -- 172.21.15.46:0/3242542204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 msgr2=0x5577dfd7b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/3242542204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 0x5577dfd7b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc15a0e3700 1 -- 172.21.15.46:0/3242542204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577dfc84f20 con 0x5577dfd6bae0 2024-03-18T20:42:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc15a8e4700 1 --2- 172.21.15.46:0/3242542204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 0x5577dfd7b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.380+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/3242542204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 0x5577dfd822e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5577dfd8ddd0 tx=0x7fc15000eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdbb3ca1a82a433b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.381+0000 7fc1495a9700 1 -- 172.21.15.46:0/3242542204 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15000ed10 con 0x5577dfd6bae0 2024-03-18T20:42:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.381+0000 7fc1495a9700 1 -- 172.21.15.46:0/3242542204 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc150033170 con 0x5577dfd6bae0 2024-03-18T20:42:35.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.381+0000 7fc1495a9700 1 -- 172.21.15.46:0/3242542204 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150005490 con 0x5577dfd6bae0 2024-03-18T20:42:35.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.381+0000 7fc1684dcb80 1 -- 172.21.15.46:0/3242542204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 msgr2=0x5577dfd822e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.381+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/3242542204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 0x5577dfd822e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5577dfd8ddd0 tx=0x7fc15000eca0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.381+0000 7fc1684dcb80 1 -- 172.21.15.46:0/3242542204 shutdown_connections 2024-03-18T20:42:35.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.381+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/3242542204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd7b6b0 0x5577dfd7b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.381+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/3242542204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd6bae0 0x5577dfd822e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.381+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/3242542204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6c400 0x5577dfd7ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.381+0000 7fc1684dcb80 1 -- 172.21.15.46:0/3242542204 >> 172.21.15.46:0/3242542204 conn(0x5577dfce22a0 msgr2=0x5577dfd73710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:35.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.381+0000 7fc1684dcb80 1 -- 172.21.15.46:0/3242542204 shutdown_connections 2024-03-18T20:42:35.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.381+0000 7fc1684dcb80 1 -- 172.21.15.46:0/3242542204 wait complete. 2024-03-18T20:42:35.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc1684dcb80 1 -- start start 2024-03-18T20:42:35.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc1684dcb80 1 Processor -- start 2024-03-18T20:42:35.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc1684dcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6bae0 0x5577dfd81300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc1684dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6c400 0x5577dfd81840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc1684dcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd7b6b0 0x5577dfdd8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc1684dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577dfdd9490 con 0x5577dfd6c400 2024-03-18T20:42:35.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc1684dcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577dfdd9600 con 0x5577dfd7b6b0 2024-03-18T20:42:35.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc1684dcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577dfdd97c0 con 0x5577dfd6bae0 2024-03-18T20:42:35.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc1684dcb80 10 client.0 _ll_register_callbacks cb 0x7fc14c099830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:35.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc1598e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6c400 0x5577dfd81840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc15a0e3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6bae0 0x5577dfd81300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc1598e2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6c400 0x5577dfd81840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58176/0 (socket says 172.21.15.46:58176) 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc15a8e4700 1 --2- 172.21.15.46:0/1636365093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd7b6b0 0x5577dfdd8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.382+0000 7fc1598e2700 1 -- 172.21.15.46:0/1636365093 learned_addr learned my addr 172.21.15.46:0/1636365093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc1598e2700 1 -- 172.21.15.46:0/1636365093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6bae0 msgr2=0x5577dfd81300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc1598e2700 1 --2- 172.21.15.46:0/1636365093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6bae0 0x5577dfd81300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc1598e2700 1 -- 172.21.15.46:0/1636365093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd7b6b0 msgr2=0x5577dfdd8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc1598e2700 1 --2- 172.21.15.46:0/1636365093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd7b6b0 0x5577dfdd8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc1598e2700 1 -- 172.21.15.46:0/1636365093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577dfaecbe0 con 0x5577dfd6c400 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc1598e2700 1 --2- 172.21.15.46:0/1636365093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6c400 0x5577dfd81840 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc1440099c0 tx=0x7fc14400bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a4daf69db8e7a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc13dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc13dffb700 1 -- 172.21.15.46:0/1636365093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14401c040 con 0x5577dfd6c400 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc13dffb700 1 -- 172.21.15.46:0/1636365093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7fc144003680 con 0x5577dfd6c400 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1636365093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc150021070 con 0x5577dfd6c400 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc13dffb700 1 -- 172.21.15.46:0/1636365093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc144003840 con 0x5577dfd6c400 2024-03-18T20:42:35.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc1684dcb80 10 client.17865 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1636365093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fc15002a040 con 0x5577dfd6c400 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.383+0000 7fc13dffb700 1 -- 172.21.15.46:0/1636365093 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc150021070 con 0x5577dfd6c400 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.384+0000 7fc13dffb700 1 -- 172.21.15.46:0/1636365093 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc144035070 con 0x5577dfd6c400 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.384+0000 7fc1684dcb80 10 client.17865 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.384+0000 7fc1684dcb80 10 client.17865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.384+0000 7fc1684dcb80 20 client.17865 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.384+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/1636365093 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577dfde1120 0x5577dfde3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.384+0000 7fc1684dcb80 4 client.17865 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.384+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1636365093 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5577dfb0dd70 con 0x5577dfde1120 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.384+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/1636365093 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577dfde1120 0x5577dfde3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.384+0000 7fc13dffb700 10 client.17865 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.384+0000 7fc15a0e3700 1 --2- 172.21.15.46:0/1636365093 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577dfde1120 0x5577dfde3540 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=616991942e047cf2 server_cookie=63e07097b87fc0fc in_seq=0 out_seq=0 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.385+0000 7fc13dffb700 1 -- 172.21.15.46:0/1636365093 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5577dfb0dd70 con 0x5577dfde1120 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.385+0000 7fc13dffb700 10 client.17865 handle_command_reply: tid=0 2024-03-18T20:42:35.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.385+0000 7fc1684dcb80 1 client.17865 shutdown 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.385+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1636365093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6c400 msgr2=0x5577dfd81840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.385+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/1636365093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6c400 0x5577dfd81840 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc1440099c0 tx=0x7fc14400bfc0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.385+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1636365093 shutdown_connections 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.385+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/1636365093 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577dfde1120 0x5577dfde3540 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.385+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/1636365093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577dfd7b6b0 0x5577dfdd8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.385+0000 7fc13dffb700 0 client.17865 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.385+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/1636365093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577dfd6bae0 0x5577dfd81300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.385+0000 7fc1684dcb80 1 --2- 172.21.15.46:0/1636365093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577dfd6c400 0x5577dfd81840 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.385+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1636365093 >> 172.21.15.46:0/1636365093 conn(0x5577dfce22a0 msgr2=0x5577dfd67270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.386+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1636365093 shutdown_connections 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.386+0000 7fc1684dcb80 1 -- 172.21.15.46:0/1636365093 wait complete. 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.386+0000 7fc1684dcb80 20 client.17865 trim_cache size 0 max 16384 2024-03-18T20:42:35.393 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:42:35.394 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:42:35.394 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "fad4b874-9dfe-45d3-a39a-cf632b045bdc", 2024-03-18T20:42:35.394 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:42:35.394 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:42:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.386+0000 7fc15c347700 1 -- 172.21.15.46:0/2571946168 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1400a1290 msgr2=0x7fc1400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.386+0000 7fc15c347700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1400a1290 0x7fc1400a3750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc150009c60 tx=0x7fc150009260 comp rx=0 tx=0).stop 2024-03-18T20:42:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.386+0000 7fc15c347700 1 -- 172.21.15.46:0/2571946168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc15419baa0 msgr2=0x7fc15419bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.386+0000 7fc15c347700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc15419baa0 0x7fc15419bf20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00ef30 tx=0x7fc14c00c3a0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.386+0000 7fc15c347700 1 -- 172.21.15.46:0/2571946168 shutdown_connections 2024-03-18T20:42:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.386+0000 7fc15c347700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1400a1290 0x7fc1400a3750 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.386+0000 7fc15c347700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc15419baa0 0x7fc15419bf20 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.386+0000 7fc15c347700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc154069540 0x7fc1541971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.386+0000 7fc15c347700 1 --2- 172.21.15.46:0/2571946168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1541060a0 0x7fc154197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.387+0000 7fc15c347700 1 -- 172.21.15.46:0/2571946168 >> 172.21.15.46:0/2571946168 conn(0x7fc154076d50 msgr2=0x7fc1540fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.387+0000 7fc15c347700 1 -- 172.21.15.46:0/2571946168 shutdown_connections 2024-03-18T20:42:35.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.387+0000 7fc15c347700 1 -- 172.21.15.46:0/2571946168 wait complete. 2024-03-18T20:42:35.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:42:35.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:42:35.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.547+0000 7f09c864a700 1 -- 172.21.15.46:0/119601294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c00687f0 msgr2=0x7f09c010db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.547+0000 7f09c864a700 1 --2- 172.21.15.46:0/119601294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c00687f0 0x7f09c010db50 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f09b800b0a0 tx=0x7f09b800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.547+0000 7f09c864a700 1 -- 172.21.15.46:0/119601294 shutdown_connections 2024-03-18T20:42:35.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.547+0000 7f09c864a700 1 --2- 172.21.15.46:0/119601294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c00687f0 0x7f09c010db50 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.547+0000 7f09c864a700 1 --2- 172.21.15.46:0/119601294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c0105ce0 0x7f09c01060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.547+0000 7f09c864a700 1 --2- 172.21.15.46:0/119601294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09c0067e30 0x7f09c00682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.547+0000 7f09c864a700 1 -- 172.21.15.46:0/119601294 >> 172.21.15.46:0/119601294 conn(0x7f09c0076d40 msgr2=0x7f09c0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:35.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.547+0000 7f09c864a700 1 -- 172.21.15.46:0/119601294 shutdown_connections 2024-03-18T20:42:35.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.547+0000 7f09c864a700 1 -- 172.21.15.46:0/119601294 wait complete. 2024-03-18T20:42:35.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c864a700 1 Processor -- start 2024-03-18T20:42:35.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c864a700 1 -- start start 2024-03-18T20:42:35.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c864a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c0067e30 0x7f09c0195080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c864a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09c00687f0 0x7f09c01955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c864a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c0105ce0 0x7f09c0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c864a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c0199fa0 con 0x7f09c0067e30 2024-03-18T20:42:35.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c864a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c019a110 con 0x7f09c00687f0 2024-03-18T20:42:35.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c864a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c019a2d0 con 0x7f09c0105ce0 2024-03-18T20:42:35.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c63e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c0067e30 0x7f09c0195080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c63e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c0067e30 0x7f09c0195080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58194/0 (socket says 172.21.15.46:58194) 2024-03-18T20:42:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c63e6700 1 -- 172.21.15.46:0/1094757716 learned_addr learned my addr 172.21.15.46:0/1094757716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c6be7700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c0105ce0 0x7f09c0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.548+0000 7f09c5be5700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09c00687f0 0x7f09c01955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.549+0000 7f09c63e6700 1 -- 172.21.15.46:0/1094757716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c0105ce0 msgr2=0x7f09c0199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.549+0000 7f09c63e6700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c0105ce0 0x7f09c0199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.549+0000 7f09c63e6700 1 -- 172.21.15.46:0/1094757716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09c00687f0 msgr2=0x7f09c01955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.549+0000 7f09c63e6700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09c00687f0 0x7f09c01955c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.549+0000 7f09c63e6700 1 -- 172.21.15.46:0/1094757716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b8009d00 con 0x7f09c0067e30 2024-03-18T20:42:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.549+0000 7f09c63e6700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c0067e30 0x7f09c0195080 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00d950 tx=0x7f09bc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.549+0000 7f09b77fe700 1 -- 172.21.15.46:0/1094757716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00f890 con 0x7f09c0067e30 2024-03-18T20:42:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.549+0000 7f09b77fe700 1 -- 172.21.15.46:0/1094757716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09bc010460 con 0x7f09c0067e30 2024-03-18T20:42:35.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.549+0000 7f09c864a700 1 -- 172.21.15.46:0/1094757716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c019a5b0 con 0x7f09c0067e30 2024-03-18T20:42:35.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.549+0000 7f09b77fe700 1 -- 172.21.15.46:0/1094757716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00e610 con 0x7f09c0067e30 2024-03-18T20:42:35.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.549+0000 7f09c864a700 1 -- 172.21.15.46:0/1094757716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c01a1df0 con 0x7f09c0067e30 2024-03-18T20:42:35.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.551+0000 7f09b77fe700 1 -- 172.21.15.46:0/1094757716 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f09bc0108d0 con 0x7f09c0067e30 2024-03-18T20:42:35.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.552+0000 7f09b77fe700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f09ac0a1290 0x7f09ac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.552+0000 7f09c5be5700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f09ac0a1290 0x7f09ac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.552+0000 7f09b77fe700 1 -- 172.21.15.46:0/1094757716 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f09bc0cca90 con 0x7f09c0067e30 2024-03-18T20:42:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.553+0000 7f09c5be5700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f09ac0a1290 0x7f09ac0a3750 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f09c01965d0 tx=0x7f09b0009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.553+0000 7f09d47dfb80 1 Processor -- start 2024-03-18T20:42:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.553+0000 7f09d47dfb80 1 -- start start 2024-03-18T20:42:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09d47dfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 0x558641709e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09d47dfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864170a380 0x558641714ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09d47dfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641715220 0x558641717610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09d47dfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641663990 con 0x55864170a380 2024-03-18T20:42:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09d47dfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641661350 con 0x558641709a60 2024-03-18T20:42:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09d47dfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864165cc40 con 0x558641715220 2024-03-18T20:42:35.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09c5be5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864170a380 0x558641714ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09c63e6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 0x558641709e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09c63e6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 0x558641709e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:39392/0 (socket says 172.21.15.46:39392) 2024-03-18T20:42:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09c63e6700 1 -- 172.21.15.46:0/2378352586 learned_addr learned my addr 172.21.15.46:0/2378352586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09c5be5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864170a380 0x558641714ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58210/0 (socket says 172.21.15.46:58210) 2024-03-18T20:42:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09c6be7700 1 --2- 172.21.15.46:0/2378352586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641715220 0x558641717610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09c63e6700 1 -- 172.21.15.46:0/2378352586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641715220 msgr2=0x558641717610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09c63e6700 1 --2- 172.21.15.46:0/2378352586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641715220 0x558641717610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09c63e6700 1 -- 172.21.15.46:0/2378352586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864170a380 msgr2=0x558641714ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09c63e6700 1 --2- 172.21.15.46:0/2378352586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864170a380 0x558641714ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09c63e6700 1 -- 172.21.15.46:0/2378352586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586414de3b0 con 0x558641709a60 2024-03-18T20:42:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09c63e6700 1 --2- 172.21.15.46:0/2378352586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 0x558641709e40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0a1b00 tx=0x7f09bc09a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c829333671210f0f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.554+0000 7f09b4ff9700 1 -- 172.21.15.46:0/2378352586 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc09d9e0 con 0x558641709a60 2024-03-18T20:42:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.555+0000 7f09b4ff9700 1 -- 172.21.15.46:0/2378352586 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09bc0a2030 con 0x558641709a60 2024-03-18T20:42:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.555+0000 7f09b4ff9700 1 -- 172.21.15.46:0/2378352586 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0992e0 con 0x558641709a60 2024-03-18T20:42:35.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.555+0000 7f09d47dfb80 1 -- 172.21.15.46:0/2378352586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 msgr2=0x558641709e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.555+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/2378352586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 0x558641709e40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0a1b00 tx=0x7f09bc09a330 comp rx=0 tx=0).stop 2024-03-18T20:42:35.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.555+0000 7f09d47dfb80 1 -- 172.21.15.46:0/2378352586 shutdown_connections 2024-03-18T20:42:35.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.555+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/2378352586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641715220 0x558641717610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.555+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/2378352586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 0x558641709e40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.555+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/2378352586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55864170a380 0x558641714ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.555+0000 7f09d47dfb80 1 -- 172.21.15.46:0/2378352586 >> 172.21.15.46:0/2378352586 conn(0x558641680220 msgr2=0x558641706020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.555+0000 7f09d47dfb80 1 -- 172.21.15.46:0/2378352586 shutdown_connections 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.555+0000 7f09d47dfb80 1 -- 172.21.15.46:0/2378352586 wait complete. 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09d47dfb80 1 -- start start 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09d47dfb80 1 Processor -- start 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09d47dfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641709a60 0x558641711f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09d47dfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55864170a380 0x5586417124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09d47dfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 0x558641712a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09d47dfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586415f8d80 con 0x558641715220 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09d47dfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586415f8a20 con 0x55864170a380 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09d47dfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586414e06a0 con 0x558641709a60 2024-03-18T20:42:35.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09d47dfb80 10 client.0 _ll_register_callbacks cb 0x5586414643a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09c6be7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 0x558641712a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09c5be5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55864170a380 0x5586417124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09c63e6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641709a60 0x558641711f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09c6be7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 0x558641712a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58224/0 (socket says 172.21.15.46:58224) 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09c6be7700 1 -- 172.21.15.46:0/1655934323 learned_addr learned my addr 172.21.15.46:0/1655934323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.556+0000 7f09c5be5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55864170a380 0x5586417124c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:39398/0 (socket says 172.21.15.46:39398) 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09c5be5700 1 -- 172.21.15.46:0/1655934323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641709a60 msgr2=0x558641711f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09c5be5700 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641709a60 0x558641711f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09c5be5700 1 -- 172.21.15.46:0/1655934323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 msgr2=0x558641712a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09c5be5700 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 0x558641712a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09c5be5700 1 -- 172.21.15.46:0/1655934323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c01a1df0 con 0x55864170a380 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09c63e6700 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641709a60 0x558641711f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09c6be7700 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 0x558641712a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:35.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09c5be5700 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55864170a380 0x5586417124c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f09b00097c0 tx=0x7f09b00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e58ebc7a65b8e89 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09a9ffb700 1 -- 172.21.15.46:0/1655934323 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0022070 con 0x55864170a380 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09d47dfb80 1 -- 172.21.15.46:0/1655934323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586416fdad0 con 0x55864170a380 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09a9ffb700 1 -- 172.21.15.46:0/1655934323 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f09b0003d40 con 0x55864170a380 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09a9ffb700 1 -- 172.21.15.46:0/1655934323 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09b0027420 con 0x55864170a380 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09d47dfb80 10 client.27554 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.557+0000 7f09d47dfb80 1 -- 172.21.15.46:0/1655934323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55864148ab60 con 0x55864170a380 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.558+0000 7f09a9ffb700 1 -- 172.21.15.46:0/1655934323 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09b0033080 con 0x55864170a380 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.558+0000 7f09a9ffb700 1 -- 172.21.15.46:0/1655934323 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f09b001d5a0 con 0x55864170a380 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.558+0000 7f09d47dfb80 10 client.27554 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.558+0000 7f09d47dfb80 10 client.27554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.558+0000 7f09d47dfb80 20 client.27554 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.558+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558641724f30 0x558641727350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.558+0000 7f09d47dfb80 4 client.27554 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.558+0000 7f09d47dfb80 1 -- 172.21.15.46:0/1655934323 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558641655700 con 0x558641724f30 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.558+0000 7f09c63e6700 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558641724f30 0x558641727350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.558+0000 7f09a9ffb700 10 client.27554 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.558+0000 7f09c63e6700 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558641724f30 0x558641727350 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97096cb9fb5f023c server_cookie=fe7490b633c30f66 in_seq=0 out_seq=0 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09a9ffb700 1 -- 172.21.15.46:0/1655934323 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558641655700 con 0x558641724f30 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09a9ffb700 10 client.27554 handle_command_reply: tid=0 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09d47dfb80 1 client.27554 shutdown 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09d47dfb80 1 -- 172.21.15.46:0/1655934323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55864170a380 msgr2=0x5586417124c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55864170a380 0x5586417124c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f09b00097c0 tx=0x7f09b00047c0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09d47dfb80 1 -- 172.21.15.46:0/1655934323 shutdown_connections 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558641724f30 0x558641727350 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 0x558641712a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641709a60 0x558641711f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09a9ffb700 0 client.27554 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:35.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/1655934323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55864170a380 0x5586417124c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09d47dfb80 1 -- 172.21.15.46:0/1655934323 >> 172.21.15.46:0/1655934323 conn(0x558641680220 msgr2=0x558641705c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09d47dfb80 1 -- 172.21.15.46:0/1655934323 shutdown_connections 2024-03-18T20:42:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09d47dfb80 1 -- 172.21.15.46:0/1655934323 wait complete. 2024-03-18T20:42:35.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.560+0000 7f09d47dfb80 20 client.27554 trim_cache size 0 max 16384 2024-03-18T20:42:35.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.578+0000 7f09d47dfb80 1 Processor -- start 2024-03-18T20:42:35.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.578+0000 7f09d47dfb80 1 -- start start 2024-03-18T20:42:35.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.578+0000 7f09d47dfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641709a60 0x55864171bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.578+0000 7f09d47dfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55864170a380 0x55864170d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.578+0000 7f09d47dfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641715220 0x55864170d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.578+0000 7f09d47dfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586417224e0 con 0x558641709a60 2024-03-18T20:42:35.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.578+0000 7f09d47dfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864170b610 con 0x55864170a380 2024-03-18T20:42:35.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.578+0000 7f09d47dfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586416fff80 con 0x558641715220 2024-03-18T20:42:35.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.578+0000 7f09c63e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641709a60 0x55864171bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.578+0000 7f09c63e6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641709a60 0x55864171bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58238/0 (socket says 172.21.15.46:58238) 2024-03-18T20:42:35.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.578+0000 7f09c63e6700 1 -- 172.21.15.46:0/2952772764 learned_addr learned my addr 172.21.15.46:0/2952772764 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:35.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.578+0000 7f09c5be5700 1 --2- 172.21.15.46:0/2952772764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55864170a380 0x55864170d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.579+0000 7f09c6be7700 1 --2- 172.21.15.46:0/2952772764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641715220 0x55864170d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.579+0000 7f09c63e6700 1 -- 172.21.15.46:0/2952772764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641715220 msgr2=0x55864170d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.579+0000 7f09c63e6700 1 --2- 172.21.15.46:0/2952772764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641715220 0x55864170d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.579+0000 7f09c63e6700 1 -- 172.21.15.46:0/2952772764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55864170a380 msgr2=0x55864170d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.579+0000 7f09c63e6700 1 --2- 172.21.15.46:0/2952772764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55864170a380 0x55864170d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.579+0000 7f09c63e6700 1 -- 172.21.15.46:0/2952772764 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558641622ea0 con 0x558641709a60 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.579+0000 7f09c6be7700 1 --2- 172.21.15.46:0/2952772764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641715220 0x55864170d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.579+0000 7f09c63e6700 1 --2- 172.21.15.46:0/2952772764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641709a60 0x55864171bdf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f09bc099d70 tx=0x7f09bc096020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7979694e40d421 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.579+0000 7f09b57fa700 1 -- 172.21.15.46:0/2952772764 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0962c0 con 0x558641709a60 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.579+0000 7f09b57fa700 1 -- 172.21.15.46:0/2952772764 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09bc096420 con 0x558641709a60 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.579+0000 7f09b57fa700 1 -- 172.21.15.46:0/2952772764 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0a1ba0 con 0x558641709a60 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.580+0000 7f09d47dfb80 1 -- 172.21.15.46:0/2952772764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641709a60 msgr2=0x55864171bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.580+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/2952772764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641709a60 0x55864171bdf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f09bc099d70 tx=0x7f09bc096020 comp rx=0 tx=0).stop 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.580+0000 7f09d47dfb80 1 -- 172.21.15.46:0/2952772764 shutdown_connections 2024-03-18T20:42:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.580+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/2952772764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641709a60 0x55864171bdf0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.580+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/2952772764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558641715220 0x55864170d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.580+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/2952772764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55864170a380 0x55864170d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.580+0000 7f09d47dfb80 1 -- 172.21.15.46:0/2952772764 >> 172.21.15.46:0/2952772764 conn(0x558641680220 msgr2=0x558641720540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.580+0000 7f09d47dfb80 1 -- 172.21.15.46:0/2952772764 shutdown_connections 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.580+0000 7f09d47dfb80 1 -- 172.21.15.46:0/2952772764 wait complete. 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.580+0000 7f09d47dfb80 1 -- start start 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.580+0000 7f09d47dfb80 1 Processor -- start 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09d47dfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 0x55864176ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09d47dfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55864170a380 0x55864176d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09d47dfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 0x558641777d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09d47dfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864176da30 con 0x558641715220 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09d47dfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864176dba0 con 0x558641709a60 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09d47dfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558641778330 con 0x55864170a380 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09d47dfb80 10 client.0 _ll_register_callbacks cb 0x7f09b0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c63e6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 0x55864176ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c6be7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 0x558641777d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c63e6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 0x55864176ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:39408/0 (socket says 172.21.15.46:39408) 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c63e6700 1 -- 172.21.15.46:0/3101230199 learned_addr learned my addr 172.21.15.46:0/3101230199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c5be5700 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55864170a380 0x55864176d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c63e6700 1 -- 172.21.15.46:0/3101230199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55864170a380 msgr2=0x55864176d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c63e6700 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55864170a380 0x55864176d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c63e6700 1 -- 172.21.15.46:0/3101230199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 msgr2=0x558641777d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c63e6700 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 0x558641777d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c63e6700 1 -- 172.21.15.46:0/3101230199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586414de3b0 con 0x558641709a60 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c6be7700 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 0x558641777d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c5be5700 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55864170a380 0x55864176d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09c63e6700 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 0x55864176ccf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0a1b00 tx=0x7f09bc096590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0daf94aacdff1a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.581+0000 7f09a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09a9ffb700 1 -- 172.21.15.46:0/3101230199 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc099e10 con 0x558641709a60 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09a9ffb700 1 -- 172.21.15.46:0/3101230199 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f09bc099f70 con 0x558641709a60 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09d47dfb80 1 -- 172.21.15.46:0/3101230199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09bc0a2030 con 0x558641709a60 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09a9ffb700 1 -- 172.21.15.46:0/3101230199 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09bc083020 con 0x558641709a60 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09d47dfb80 10 client.27560 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09d47dfb80 1 -- 172.21.15.46:0/3101230199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5586414e2cd0 con 0x558641709a60 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09a9ffb700 1 -- 172.21.15.46:0/3101230199 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09bc0a2030 con 0x558641709a60 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09a9ffb700 1 -- 172.21.15.46:0/3101230199 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f09bc065280 con 0x558641709a60 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09d47dfb80 10 client.27560 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09d47dfb80 10 client.27560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09d47dfb80 20 client.27560 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5586417795b0 0x55864177b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09d47dfb80 4 client.27560 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.582+0000 7f09d47dfb80 1 -- 172.21.15.46:0/3101230199 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5586414abcf0 con 0x5586417795b0 2024-03-18T20:42:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09c5be5700 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5586417795b0 0x55864177b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09c5be5700 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5586417795b0 0x55864177b9d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9814bc801276648a server_cookie=78b58e29cc3c5a34 in_seq=0 out_seq=0 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09a9ffb700 10 client.27560 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09a9ffb700 1 -- 172.21.15.46:0/3101230199 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5586414abcf0 con 0x5586417795b0 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09a9ffb700 10 client.27560 handle_command_reply: tid=0 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09d47dfb80 1 client.27560 shutdown 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09d47dfb80 1 -- 172.21.15.46:0/3101230199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 msgr2=0x55864176ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 0x55864176ccf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0a1b00 tx=0x7f09bc096590 comp rx=0 tx=0).stop 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09d47dfb80 1 -- 172.21.15.46:0/3101230199 shutdown_connections 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5586417795b0 0x55864177b9d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558641715220 0x558641777d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558641709a60 0x55864176ccf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09d47dfb80 1 --2- 172.21.15.46:0/3101230199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55864170a380 0x55864176d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09a9ffb700 0 client.27560 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09d47dfb80 1 -- 172.21.15.46:0/3101230199 >> 172.21.15.46:0/3101230199 conn(0x558641680220 msgr2=0x55864170d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09d47dfb80 1 -- 172.21.15.46:0/3101230199 shutdown_connections 2024-03-18T20:42:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.583+0000 7f09d47dfb80 1 -- 172.21.15.46:0/3101230199 wait complete. 2024-03-18T20:42:35.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.584+0000 7f09d47dfb80 20 client.27560 trim_cache size 0 max 16384 2024-03-18T20:42:35.593 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:42:35.593 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:42:35.593 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:42:35.593 INFO:teuthology.orchestra.run.smithi046.stdout: "fad4b874-9dfe-45d3-a39a-cf632b045bdc": { 2024-03-18T20:42:35.593 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:42:35.593 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "fad4b874-9dfe-45d3-a39a-cf632b045bdc", 2024-03-18T20:42:35.593 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:42:35.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:42:35.593 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:42:35.593 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:42:35.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.584+0000 7f09c864a700 1 -- 172.21.15.46:0/1094757716 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f09ac0a1290 msgr2=0x7f09ac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.584+0000 7f09c864a700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f09ac0a1290 0x7f09ac0a3750 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f09c01965d0 tx=0x7f09b0009260 comp rx=0 tx=0).stop 2024-03-18T20:42:35.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.584+0000 7f09c864a700 1 -- 172.21.15.46:0/1094757716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c0067e30 msgr2=0x7f09c0195080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:35.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.584+0000 7f09c864a700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c0067e30 0x7f09c0195080 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00d950 tx=0x7f09bc00dc60 comp rx=0 tx=0).stop 2024-03-18T20:42:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.584+0000 7f09c864a700 1 -- 172.21.15.46:0/1094757716 shutdown_connections 2024-03-18T20:42:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.584+0000 7f09c864a700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f09ac0a1290 0x7f09ac0a3750 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.584+0000 7f09c864a700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09c0067e30 0x7f09c0195080 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.584+0000 7f09c864a700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c0105ce0 0x7f09c0199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.584+0000 7f09c864a700 1 --2- 172.21.15.46:0/1094757716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09c00687f0 0x7f09c01955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.584+0000 7f09c864a700 1 -- 172.21.15.46:0/1094757716 >> 172.21.15.46:0/1094757716 conn(0x7f09c0076d40 msgr2=0x7f09c00fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.585+0000 7f09c864a700 1 -- 172.21.15.46:0/1094757716 shutdown_connections 2024-03-18T20:42:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:42:35.585+0000 7f09c864a700 1 -- 172.21.15.46:0/1094757716 wait complete. 2024-03-18T20:42:35.601 INFO:tasks.cephfs.filesystem:scrub status for tag:fad4b874-9dfe-45d3-a39a-cf632b045bdc - {'path': '/', 'tag': 'fad4b874-9dfe-45d3-a39a-cf632b045bdc', 'options': 'recursive,force'} 2024-03-18T20:42:35.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 215292 3.40 MB/sec execute 210 sec latency 2815.179 ms 2024-03-18T20:42:35.922 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:35 smithi046 ceph-mon[43495]: pgmap v198: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-03-18T20:42:35.923 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:35 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:36.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:35 smithi073 ceph-mon[47384]: pgmap v198: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-03-18T20:42:36.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:35 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:36.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:35 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:35 smithi155 ceph-mon[48344]: pgmap v198: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-03-18T20:42:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:35 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:35 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:35 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:36.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 216889 3.44 MB/sec execute 211 sec latency 12.297 ms 2024-03-18T20:42:37.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:42:37.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:37.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:37.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:37.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:37.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:42:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:36 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:42:37.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:37.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:37.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:37.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:37.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:37.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:37.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:37.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:37.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:42:37.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:36 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:42:37.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:37.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:37.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:36 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:42:37.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 218114 3.45 MB/sec execute 212 sec latency 137.978 ms 2024-03-18T20:42:38.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:37 smithi073 ceph-mon[47384]: pgmap v199: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-03-18T20:42:38.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:37 smithi155 ceph-mon[48344]: pgmap v199: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-03-18T20:42:38.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:37 smithi046 ceph-mon[43495]: pgmap v199: 97 pgs: 97 active+clean; 83 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-03-18T20:42:38.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 218114 3.44 MB/sec execute 213 sec latency 1138.082 ms 2024-03-18T20:42:39.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 218114 3.42 MB/sec execute 214 sec latency 2138.158 ms 2024-03-18T20:42:40.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:40 smithi073 ceph-mon[47384]: pgmap v200: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-03-18T20:42:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:40 smithi155 ceph-mon[48344]: pgmap v200: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-03-18T20:42:40.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:40 smithi046 ceph-mon[43495]: pgmap v200: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-03-18T20:42:40.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 219002 3.42 MB/sec execute 215 sec latency 2772.826 ms 2024-03-18T20:42:41.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 220491 3.45 MB/sec execute 216 sec latency 16.524 ms 2024-03-18T20:42:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:42 smithi073 ceph-mon[47384]: pgmap v201: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-03-18T20:42:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:42 smithi155 ceph-mon[48344]: pgmap v201: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-03-18T20:42:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:42 smithi046 ceph-mon[43495]: pgmap v201: 97 pgs: 97 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-03-18T20:42:42.696 INFO:tasks.workunit.client.0.smithi046.stdout: 1 221657 3.46 MB/sec execute 217 sec latency 139.373 ms 2024-03-18T20:42:43.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:43 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:42:43.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:43 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:42:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:43 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:42:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:43 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:42:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:43 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:42:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:43 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:42:43.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 221657 3.44 MB/sec execute 218 sec latency 1139.469 ms 2024-03-18T20:42:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:44 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:42:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:44 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:42:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:44 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:42:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:44 smithi073 ceph-mon[47384]: pgmap v202: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-03-18T20:42:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:44 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:42:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:44 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:42:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:44 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:42:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:44 smithi155 ceph-mon[48344]: pgmap v202: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-03-18T20:42:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:44 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:42:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:44 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:42:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:44 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:42:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:44 smithi046 ceph-mon[43495]: pgmap v202: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-03-18T20:42:44.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 221657 3.43 MB/sec execute 219 sec latency 2139.574 ms 2024-03-18T20:42:45.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 223600 3.47 MB/sec execute 220 sec latency 2302.582 ms 2024-03-18T20:42:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.019+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3508273985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fec106020 msgr2=0x7f5fec10dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.019+0000 7f5ff23fd700 1 --2- 172.21.15.46:0/3508273985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fec106020 0x7f5fec10dba0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009f90 tx=0x7f5fe400b330 comp rx=0 tx=0).stop 2024-03-18T20:42:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.019+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3508273985 shutdown_connections 2024-03-18T20:42:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.019+0000 7f5ff23fd700 1 --2- 172.21.15.46:0/3508273985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fec106020 0x7f5fec10dba0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.019+0000 7f5ff23fd700 1 --2- 172.21.15.46:0/3508273985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5fec100f00 0x7f5fec1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.019+0000 7f5ff23fd700 1 --2- 172.21.15.46:0/3508273985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec1018b0 0x7f5fec1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.019+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3508273985 >> 172.21.15.46:0/3508273985 conn(0x7f5fec0fc7c0 msgr2=0x7f5fec0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.019+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3508273985 shutdown_connections 2024-03-18T20:42:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.019+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3508273985 wait complete. 2024-03-18T20:42:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5ff23fd700 1 Processor -- start 2024-03-18T20:42:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5ff23fd700 1 -- start start 2024-03-18T20:42:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5ff23fd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fec100f00 0x7f5fec0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5ff23fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec106020 0x7f5fec0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5ff23fd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5fec073b30 0x7f5fec1a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5ff23fd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec1a58e0 con 0x7f5fec106020 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5ff23fd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec1a5a50 con 0x7f5fec100f00 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5ff23fd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec1a5c10 con 0x7f5fec073b30 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5fe3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec106020 0x7f5fec0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5ff099a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5fec073b30 0x7f5fec1a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5fe3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec106020 0x7f5fec0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48950/0 (socket says 172.21.15.46:48950) 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5fe3fff700 1 -- 172.21.15.46:0/3687474893 learned_addr learned my addr 172.21.15.46:0/3687474893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.020+0000 7f5febfff700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fec100f00 0x7f5fec0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.021+0000 7f5fe3fff700 1 -- 172.21.15.46:0/3687474893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5fec073b30 msgr2=0x7f5fec1a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.021+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5fec073b30 0x7f5fec1a5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.021+0000 7f5fe3fff700 1 -- 172.21.15.46:0/3687474893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fec100f00 msgr2=0x7f5fec0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.021+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fec100f00 0x7f5fec0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.021+0000 7f5fe3fff700 1 -- 172.21.15.46:0/3687474893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0095f0 con 0x7f5fec106020 2024-03-18T20:42:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.021+0000 7f5ff099a700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5fec073b30 0x7f5fec1a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.021+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec106020 0x7f5fec0735f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400e970 tx=0x7f5fd400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.021+0000 7f5fe9ffb700 1 -- 172.21.15.46:0/3687474893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4009790 con 0x7f5fec106020 2024-03-18T20:42:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.021+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3687474893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe4009c40 con 0x7f5fec106020 2024-03-18T20:42:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.021+0000 7f5fe9ffb700 1 -- 172.21.15.46:0/3687474893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fd400c7e0 con 0x7f5fec106020 2024-03-18T20:42:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.021+0000 7f5fe9ffb700 1 -- 172.21.15.46:0/3687474893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40054e0 con 0x7f5fec106020 2024-03-18T20:42:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.021+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3687474893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec1a6260 con 0x7f5fec106020 2024-03-18T20:42:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.024+0000 7f5fe9ffb700 1 -- 172.21.15.46:0/3687474893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5fd400c950 con 0x7f5fec106020 2024-03-18T20:42:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.024+0000 7f5fe9ffb700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5fd80a1290 0x7f5fd80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.025+0000 7f5fe9ffb700 1 -- 172.21.15.46:0/3687474893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5fd4014070 con 0x7f5fec106020 2024-03-18T20:42:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.025+0000 7f5febfff700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5fd80a1290 0x7f5fd80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.025+0000 7f5febfff700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5fd80a1290 0x7f5fd80a3750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005fd0 tx=0x7f5fdc009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.025+0000 7f5ffe592b80 1 Processor -- start 2024-03-18T20:42:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5ffe592b80 1 -- start start 2024-03-18T20:42:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5ffe592b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e658950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5ffe592b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e663870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5ffe592b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e6661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5ffe592b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56343e5b2930 con 0x56343e663db0 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5ffe592b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56343e5b02f0 con 0x56343e658570 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5ffe592b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56343e5abbe0 con 0x56343e658e90 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5febfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e658950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5febfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e658950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:41220/0 (socket says 172.21.15.46:41220) 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5fe3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e663870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5febfff700 1 -- 172.21.15.46:0/1456536216 learned_addr learned my addr 172.21.15.46:0/1456536216 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5ff099a700 1 --2- 172.21.15.46:0/1456536216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e6661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5febfff700 1 -- 172.21.15.46:0/1456536216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 msgr2=0x56343e663870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5febfff700 1 --2- 172.21.15.46:0/1456536216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e663870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5febfff700 1 -- 172.21.15.46:0/1456536216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 msgr2=0x56343e6661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5febfff700 1 --2- 172.21.15.46:0/1456536216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e6661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5febfff700 1 -- 172.21.15.46:0/1456536216 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56343e42d350 con 0x56343e658570 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.026+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/1456536216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e663870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.027+0000 7f5febfff700 1 --2- 172.21.15.46:0/1456536216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e658950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e670 tx=0x7f5fdc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f49d9e875dcb11b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.027+0000 7f5fe27fc700 1 -- 172.21.15.46:0/1456536216 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00ed30 con 0x56343e658570 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.027+0000 7f5fe27fc700 1 -- 172.21.15.46:0/1456536216 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fdc021070 con 0x56343e658570 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.027+0000 7f5fe27fc700 1 -- 172.21.15.46:0/1456536216 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc01d6d0 con 0x56343e658570 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.027+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1456536216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 msgr2=0x56343e658950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.027+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/1456536216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e658950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e670 tx=0x7f5fdc014000 comp rx=0 tx=0).stop 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.028+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1456536216 shutdown_connections 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.028+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/1456536216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e6661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.028+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/1456536216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e658950 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.028+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/1456536216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e663870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.028+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1456536216 >> 172.21.15.46:0/1456536216 conn(0x56343e5cf1c0 msgr2=0x56343e654a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.028+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1456536216 shutdown_connections 2024-03-18T20:42:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.028+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1456536216 wait complete. 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.028+0000 7f5ffe592b80 1 -- start start 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.028+0000 7f5ffe592b80 1 Processor -- start 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5ffe592b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e658570 0x56343e6637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5ffe592b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658e90 0x56343e660ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5ffe592b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e663db0 0x56343e6610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5ffe592b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56343e547d20 con 0x56343e658570 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5ffe592b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56343e5479c0 con 0x56343e658e90 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5ffe592b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56343e42f640 con 0x56343e663db0 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5ffe592b80 10 client.0 _ll_register_callbacks cb 0x56343e3b3340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5febfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e658570 0x56343e6637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5febfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e658570 0x56343e6637c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48958/0 (socket says 172.21.15.46:48958) 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5febfff700 1 -- 172.21.15.46:0/1420393401 learned_addr learned my addr 172.21.15.46:0/1420393401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658e90 0x56343e660ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5ff099a700 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e663db0 0x56343e6610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5febfff700 1 -- 172.21.15.46:0/1420393401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e663db0 msgr2=0x56343e6610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5febfff700 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e663db0 0x56343e6610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5febfff700 1 -- 172.21.15.46:0/1420393401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658e90 msgr2=0x56343e660ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5febfff700 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658e90 0x56343e660ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.029+0000 7f5febfff700 1 -- 172.21.15.46:0/1420393401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec1a6260 con 0x56343e658570 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.030+0000 7f5febfff700 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e658570 0x56343e6637c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005b40 tx=0x7f5fdc00eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd247ae9d7751b70 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.030+0000 7f5fcb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.030+0000 7f5fcb7fe700 1 -- 172.21.15.46:0/1420393401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00ee00 con 0x56343e658570 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.030+0000 7f5fcb7fe700 1 -- 172.21.15.46:0/1420393401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5fdc01c620 con 0x56343e658570 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.030+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1420393401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56343e64c6c0 con 0x56343e658570 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.030+0000 7f5fcb7fe700 1 -- 172.21.15.46:0/1420393401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fdc01c7e0 con 0x56343e658570 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.030+0000 7f5ffe592b80 10 client.17895 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.030+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1420393401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f5fdc017070 con 0x56343e658570 2024-03-18T20:42:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.030+0000 7f5fcb7fe700 1 -- 172.21.15.46:0/1420393401 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fdc03c020 con 0x56343e658570 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.030+0000 7f5fcb7fe700 1 -- 172.21.15.46:0/1420393401 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5fdc004890 con 0x56343e658570 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.031+0000 7f5ffe592b80 10 client.17895 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.031+0000 7f5ffe592b80 10 client.17895 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.031+0000 7f5ffe592b80 20 client.17895 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.031+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56343e673860 0x56343e675c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.031+0000 7f5ffe592b80 4 client.17895 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.031+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1420393401 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56343e5a46a0 con 0x56343e673860 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.031+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56343e673860 0x56343e675c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.031+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56343e673860 0x56343e675c80 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e104269c2af8b60a server_cookie=9da7d82af624eab0 in_seq=0 out_seq=0 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.031+0000 7f5fcb7fe700 10 client.17895 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5fcb7fe700 1 -- 172.21.15.46:0/1420393401 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56343e5a46a0 con 0x56343e673860 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5fcb7fe700 10 client.17895 handle_command_reply: tid=0 2024-03-18T20:42:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5ffe592b80 1 client.17895 shutdown 2024-03-18T20:42:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1420393401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e658570 msgr2=0x56343e6637c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e658570 0x56343e6637c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005b40 tx=0x7f5fdc00eb30 comp rx=0 tx=0).stop 2024-03-18T20:42:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1420393401 shutdown_connections 2024-03-18T20:42:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56343e673860 0x56343e675c80 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e658570 0x56343e6637c0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e663db0 0x56343e6610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5fcb7fe700 0 client.17895 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/1420393401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658e90 0x56343e660ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1420393401 >> 172.21.15.46:0/1420393401 conn(0x56343e5cf1c0 msgr2=0x56343e654650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1420393401 shutdown_connections 2024-03-18T20:42:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5ffe592b80 1 -- 172.21.15.46:0/1420393401 wait complete. 2024-03-18T20:42:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.032+0000 7f5ffe592b80 20 client.17895 trim_cache size 0 max 16384 2024-03-18T20:42:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5ffe592b80 1 Processor -- start 2024-03-18T20:42:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5ffe592b80 1 -- start start 2024-03-18T20:42:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5ffe592b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e65b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5ffe592b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e66aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5ffe592b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e654500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5ffe592b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56343e661760 con 0x56343e663db0 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5ffe592b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56343e66afb0 con 0x56343e658570 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5ffe592b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56343e659530 con 0x56343e658e90 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5febfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e65b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5ff099a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e654500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5febfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e65b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:41244/0 (socket says 172.21.15.46:41244) 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5ff099a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e654500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48966/0 (socket says 172.21.15.46:48966) 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5febfff700 1 -- 172.21.15.46:0/2350733292 learned_addr learned my addr 172.21.15.46:0/2350733292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/2350733292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e66aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5febfff700 1 -- 172.21.15.46:0/2350733292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 msgr2=0x56343e66aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5febfff700 1 --2- 172.21.15.46:0/2350733292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e66aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5febfff700 1 -- 172.21.15.46:0/2350733292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 msgr2=0x56343e654500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.049+0000 7f5febfff700 1 --2- 172.21.15.46:0/2350733292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e654500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.050+0000 7f5febfff700 1 -- 172.21.15.46:0/2350733292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56343e3d9b00 con 0x56343e658570 2024-03-18T20:42:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.050+0000 7f5ff099a700 1 --2- 172.21.15.46:0/2350733292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e654500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.050+0000 7f5febfff700 1 --2- 172.21.15.46:0/2350733292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e65b4e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc01cec0 tx=0x7f5fdc01cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=603a876144336ffd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.050+0000 7f5fe2ffd700 1 -- 172.21.15.46:0/2350733292 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00ec50 con 0x56343e658570 2024-03-18T20:42:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.050+0000 7f5fe2ffd700 1 -- 172.21.15.46:0/2350733292 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fdc03c070 con 0x56343e658570 2024-03-18T20:42:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.050+0000 7f5fe2ffd700 1 -- 172.21.15.46:0/2350733292 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc01c9e0 con 0x56343e658570 2024-03-18T20:42:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.051+0000 7f5ffe592b80 1 -- 172.21.15.46:0/2350733292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 msgr2=0x56343e65b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.051+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/2350733292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e65b4e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc01cec0 tx=0x7f5fdc01cde0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.051+0000 7f5ffe592b80 1 -- 172.21.15.46:0/2350733292 shutdown_connections 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.051+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/2350733292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e654500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.051+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/2350733292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e65b4e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.051+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/2350733292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e66aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.051+0000 7f5ffe592b80 1 -- 172.21.15.46:0/2350733292 >> 172.21.15.46:0/2350733292 conn(0x56343e5cf1c0 msgr2=0x56343e65f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.051+0000 7f5ffe592b80 1 -- 172.21.15.46:0/2350733292 shutdown_connections 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.051+0000 7f5ffe592b80 1 -- 172.21.15.46:0/2350733292 wait complete. 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.051+0000 7f5ffe592b80 1 -- start start 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5ffe592b80 1 Processor -- start 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5ffe592b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e666a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5ffe592b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e666f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5ffe592b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e6674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5ffe592b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56343e6c6850 con 0x56343e663db0 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5ffe592b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56343e6c69c0 con 0x56343e658570 2024-03-18T20:42:46.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5ffe592b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56343e6c6b80 con 0x56343e658e90 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5ffe592b80 10 client.0 _ll_register_callbacks cb 0x7f5fd4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5ff099a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e6674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5febfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e666a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5ff099a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e6674d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48976/0 (socket says 172.21.15.46:48976) 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5fe3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e666f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5ff099a700 1 -- 172.21.15.46:0/3315675547 learned_addr learned my addr 172.21.15.46:0/3315675547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5febfff700 1 -- 172.21.15.46:0/3315675547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 msgr2=0x56343e666f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5febfff700 1 --2- 172.21.15.46:0/3315675547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e666f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5febfff700 1 -- 172.21.15.46:0/3315675547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 msgr2=0x56343e6674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5febfff700 1 --2- 172.21.15.46:0/3315675547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e6674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.052+0000 7f5febfff700 1 -- 172.21.15.46:0/3315675547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec10e410 con 0x56343e658570 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5febfff700 1 --2- 172.21.15.46:0/3315675547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e666a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009d10 tx=0x7f5fdc01ce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d329d762f35354b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5fcb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:42:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5fcb7fe700 1 -- 172.21.15.46:0/3315675547 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00ea30 con 0x56343e658570 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5fcb7fe700 1 -- 172.21.15.46:0/3315675547 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f5fdc03d080 con 0x56343e658570 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5ffe592b80 1 -- 172.21.15.46:0/3315675547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fdc021070 con 0x56343e658570 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5fcb7fe700 1 -- 172.21.15.46:0/3315675547 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fdc0048e0 con 0x56343e658570 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5ffe592b80 10 client.27578 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5ffe592b80 1 -- 172.21.15.46:0/3315675547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f5fdc017070 con 0x56343e658570 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5fcb7fe700 1 -- 172.21.15.46:0/3315675547 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fdc050020 con 0x56343e658570 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5fcb7fe700 1 -- 172.21.15.46:0/3315675547 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5fdc005410 con 0x56343e658570 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5ffe592b80 10 client.27578 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5ffe592b80 10 client.27578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5ffe592b80 20 client.27578 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/3315675547 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56343e6c7f20 0x56343e6ca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5ffe592b80 4 client.27578 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.053+0000 7f5ffe592b80 1 -- 172.21.15.46:0/3315675547 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56343e3fac90 con 0x56343e6c7f20 2024-03-18T20:42:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.054+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/3315675547 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56343e6c7f20 0x56343e6ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.054+0000 7f5fcb7fe700 10 client.27578 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.054+0000 7f5fe3fff700 1 --2- 172.21.15.46:0/3315675547 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56343e6c7f20 0x56343e6ca340 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8edd9e3cb43834 server_cookie=98e31a656829fa35 in_seq=0 out_seq=0 2024-03-18T20:42:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.076+0000 7f5fcb7fe700 1 -- 172.21.15.46:0/3315675547 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56343e3fac90 con 0x56343e6c7f20 2024-03-18T20:42:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.076+0000 7f5fcb7fe700 10 client.27578 handle_command_reply: tid=0 2024-03-18T20:42:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.076+0000 7f5ffe592b80 1 client.27578 shutdown 2024-03-18T20:42:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.077+0000 7f5ffe592b80 1 -- 172.21.15.46:0/3315675547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 msgr2=0x56343e666a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.077+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/3315675547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e666a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009d10 tx=0x7f5fdc01ce50 comp rx=0 tx=0).stop 2024-03-18T20:42:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.077+0000 7f5ffe592b80 1 -- 172.21.15.46:0/3315675547 shutdown_connections 2024-03-18T20:42:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.077+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/3315675547 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56343e6c7f20 0x56343e6ca340 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.077+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/3315675547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56343e663db0 0x56343e6674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.077+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/3315675547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56343e658570 0x56343e666a10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.077+0000 7f5fcb7fe700 0 client.27578 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.077+0000 7f5ffe592b80 1 --2- 172.21.15.46:0/3315675547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56343e658e90 0x56343e666f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.077+0000 7f5ffe592b80 1 -- 172.21.15.46:0/3315675547 >> 172.21.15.46:0/3315675547 conn(0x56343e5cf1c0 msgr2=0x56343e654470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.077+0000 7f5ffe592b80 1 -- 172.21.15.46:0/3315675547 shutdown_connections 2024-03-18T20:42:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.077+0000 7f5ffe592b80 1 -- 172.21.15.46:0/3315675547 wait complete. 2024-03-18T20:42:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.077+0000 7f5ffe592b80 20 client.27578 trim_cache size 0 max 16384 2024-03-18T20:42:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:42:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:42:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:42:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:42:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.078+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3687474893 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5fd80a1290 msgr2=0x7f5fd80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.078+0000 7f5ff23fd700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5fd80a1290 0x7f5fd80a3750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005fd0 tx=0x7f5fdc009310 comp rx=0 tx=0).stop 2024-03-18T20:42:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.078+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3687474893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec106020 msgr2=0x7f5fec0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.078+0000 7f5ff23fd700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec106020 0x7f5fec0735f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400e970 tx=0x7f5fd400ec80 comp rx=0 tx=0).stop 2024-03-18T20:42:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.078+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3687474893 shutdown_connections 2024-03-18T20:42:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.078+0000 7f5ff23fd700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5fd80a1290 0x7f5fd80a3750 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.078+0000 7f5ff23fd700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5fec073b30 0x7f5fec1a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.078+0000 7f5ff23fd700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fec100f00 0x7f5fec0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.078+0000 7f5ff23fd700 1 --2- 172.21.15.46:0/3687474893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fec106020 0x7f5fec0735f0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.078+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3687474893 >> 172.21.15.46:0/3687474893 conn(0x7f5fec0fc7c0 msgr2=0x7f5fec0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.078+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3687474893 shutdown_connections 2024-03-18T20:42:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:46.078+0000 7f5ff23fd700 1 -- 172.21.15.46:0/3687474893 wait complete. 2024-03-18T20:42:46.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:45 smithi073 ceph-mon[47384]: pgmap v203: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-18T20:42:46.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:45 smithi155 ceph-mon[48344]: pgmap v203: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-18T20:42:46.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:45 smithi046 ceph-mon[43495]: pgmap v203: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-18T20:42:46.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 224701 3.47 MB/sec execute 221 sec latency 21.092 ms 2024-03-18T20:42:47.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 225205 3.47 MB/sec execute 222 sec latency 585.689 ms 2024-03-18T20:42:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:48 smithi073 ceph-mon[47384]: pgmap v204: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-18T20:42:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:48 smithi155 ceph-mon[48344]: pgmap v204: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-18T20:42:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:48 smithi046 ceph-mon[43495]: pgmap v204: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-18T20:42:48.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 225205 3.45 MB/sec execute 223 sec latency 1585.811 ms 2024-03-18T20:42:49.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 225205 3.44 MB/sec execute 224 sec latency 2585.885 ms 2024-03-18T20:42:50.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:50 smithi073 ceph-mon[47384]: pgmap v205: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2024-03-18T20:42:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:50 smithi155 ceph-mon[48344]: pgmap v205: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2024-03-18T20:42:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:50 smithi046 ceph-mon[43495]: pgmap v205: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 1.7 MiB/s wr, 184 op/s 2024-03-18T20:42:50.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 226096 3.43 MB/sec execute 225 sec latency 3212.223 ms 2024-03-18T20:42:51.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 227644 3.47 MB/sec execute 226 sec latency 13.677 ms 2024-03-18T20:42:52.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:52 smithi073 ceph-mon[47384]: pgmap v206: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:42:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:52 smithi155 ceph-mon[48344]: pgmap v206: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:42:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:52 smithi046 ceph-mon[43495]: pgmap v206: 97 pgs: 97 active+clean; 87 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:42:52.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 228748 3.48 MB/sec execute 227 sec latency 239.954 ms 2024-03-18T20:42:53.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 228748 3.46 MB/sec execute 228 sec latency 1240.015 ms 2024-03-18T20:42:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:54 smithi073 ceph-mon[47384]: pgmap v207: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-03-18T20:42:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:54 smithi155 ceph-mon[48344]: pgmap v207: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-03-18T20:42:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:54 smithi046 ceph-mon[43495]: pgmap v207: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-03-18T20:42:54.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 228748 3.45 MB/sec execute 229 sec latency 2240.111 ms 2024-03-18T20:42:55.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 229657 3.44 MB/sec execute 230 sec latency 2865.800 ms 2024-03-18T20:42:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:55 smithi073 ceph-mon[47384]: pgmap v208: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:42:56.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:55 smithi155 ceph-mon[48344]: pgmap v208: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:42:56.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:55 smithi046 ceph-mon[43495]: pgmap v208: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:42:56.697 INFO:tasks.workunit.client.0.smithi046.stdout: 1 231204 3.48 MB/sec execute 231 sec latency 16.590 ms 2024-03-18T20:42:57.698 INFO:tasks.workunit.client.0.smithi046.stdout: 1 232296 3.48 MB/sec execute 232 sec latency 244.604 ms 2024-03-18T20:42:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:42:58 smithi073 ceph-mon[47384]: pgmap v209: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:42:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:42:58 smithi155 ceph-mon[48344]: pgmap v209: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:42:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:42:58 smithi046 ceph-mon[43495]: pgmap v209: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:42:58.698 INFO:tasks.workunit.client.0.smithi046.stdout: 1 232296 3.47 MB/sec execute 233 sec latency 1244.714 ms 2024-03-18T20:42:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.207+0000 7f4e34019700 1 -- 172.21.15.46:0/3159877296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2c1018c0 msgr2=0x7f4e2c105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.207+0000 7f4e34019700 1 --2- 172.21.15.46:0/3159877296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2c1018c0 0x7f4e2c105900 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009990 tx=0x7f4e1c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.207+0000 7f4e34019700 1 -- 172.21.15.46:0/3159877296 shutdown_connections 2024-03-18T20:42:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.207+0000 7f4e34019700 1 --2- 172.21.15.46:0/3159877296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e2c106030 0x7f4e2c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.207+0000 7f4e34019700 1 --2- 172.21.15.46:0/3159877296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c100f10 0x7f4e2c1012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.207+0000 7f4e34019700 1 --2- 172.21.15.46:0/3159877296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2c1018c0 0x7f4e2c105900 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.207+0000 7f4e34019700 1 -- 172.21.15.46:0/3159877296 >> 172.21.15.46:0/3159877296 conn(0x7f4e2c0fc770 msgr2=0x7f4e2c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.207+0000 7f4e34019700 1 -- 172.21.15.46:0/3159877296 shutdown_connections 2024-03-18T20:42:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.208+0000 7f4e34019700 1 -- 172.21.15.46:0/3159877296 wait complete. 2024-03-18T20:42:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.208+0000 7f4e34019700 1 Processor -- start 2024-03-18T20:42:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.208+0000 7f4e34019700 1 -- start start 2024-03-18T20:42:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.208+0000 7f4e34019700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c100f10 0x7f4e2c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.208+0000 7f4e34019700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2c106030 0x7f4e2c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.208+0000 7f4e34019700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e2c19bb00 0x7f4e2c19bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.208+0000 7f4e34019700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c19c580 con 0x7f4e2c106030 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.208+0000 7f4e34019700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c19c6f0 con 0x7f4e2c19bb00 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.208+0000 7f4e34019700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c19c8b0 con 0x7f4e2c100f10 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e315b4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2c106030 0x7f4e2c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e315b4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2c106030 0x7f4e2c1977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60822/0 (socket says 172.21.15.46:60822) 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e31db5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c100f10 0x7f4e2c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e315b4700 1 -- 172.21.15.46:0/2162023293 learned_addr learned my addr 172.21.15.46:0/2162023293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e325b6700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e2c19bb00 0x7f4e2c19bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e315b4700 1 -- 172.21.15.46:0/2162023293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c100f10 msgr2=0x7f4e2c197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e315b4700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c100f10 0x7f4e2c197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e315b4700 1 -- 172.21.15.46:0/2162023293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e2c19bb00 msgr2=0x7f4e2c19bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e315b4700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e2c19bb00 0x7f4e2c19bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e315b4700 1 -- 172.21.15.46:0/2162023293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c0095f0 con 0x7f4e2c106030 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e31db5700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c100f10 0x7f4e2c197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:42:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e315b4700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2c106030 0x7f4e2c1977b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c000c00 tx=0x7f4e1c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e22ffd700 1 -- 172.21.15.46:0/2162023293 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c01d070 con 0x7f4e2c106030 2024-03-18T20:42:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e34019700 1 -- 172.21.15.46:0/2162023293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2c1a3db0 con 0x7f4e2c106030 2024-03-18T20:42:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e22ffd700 1 -- 172.21.15.46:0/2162023293 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e1c00fb30 con 0x7f4e2c106030 2024-03-18T20:42:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e22ffd700 1 -- 172.21.15.46:0/2162023293 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c022bb0 con 0x7f4e2c106030 2024-03-18T20:42:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.209+0000 7f4e34019700 1 -- 172.21.15.46:0/2162023293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2c1a41d0 con 0x7f4e2c106030 2024-03-18T20:42:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.212+0000 7f4e22ffd700 1 -- 172.21.15.46:0/2162023293 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f4e1c00fca0 con 0x7f4e2c106030 2024-03-18T20:42:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.212+0000 7f4e22ffd700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e180a1290 0x7f4e180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.212+0000 7f4e22ffd700 1 -- 172.21.15.46:0/2162023293 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f4e1c0ce080 con 0x7f4e2c106030 2024-03-18T20:42:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.212+0000 7f4e31db5700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e180a1290 0x7f4e180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.213+0000 7f4e31db5700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e180a1290 0x7f4e180a3750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4e28007fd0 tx=0x7f4e280073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.213+0000 7f4e401aeb80 1 Processor -- start 2024-03-18T20:42:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e401aeb80 1 -- start start 2024-03-18T20:42:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e401aeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1ce6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e401aeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6e90 0x564bd1cf1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e401aeb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1cf1db0 0x564bd1cf41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e401aeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bd1c40930 con 0x564bd1ce6e90 2024-03-18T20:42:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e401aeb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bd1c3e2f0 con 0x564bd1cf1db0 2024-03-18T20:42:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e401aeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bd1c39be0 con 0x564bd1ce6570 2024-03-18T20:42:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e315b4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6e90 0x564bd1cf1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e315b4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6e90 0x564bd1cf1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60832/0 (socket says 172.21.15.46:60832) 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e315b4700 1 -- 172.21.15.46:0/3879847663 learned_addr learned my addr 172.21.15.46:0/3879847663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e31db5700 1 --2- 172.21.15.46:0/3879847663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1ce6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e325b6700 1 --2- 172.21.15.46:0/3879847663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1cf1db0 0x564bd1cf41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e315b4700 1 -- 172.21.15.46:0/3879847663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 msgr2=0x564bd1ce6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e315b4700 1 --2- 172.21.15.46:0/3879847663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1ce6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e315b4700 1 -- 172.21.15.46:0/3879847663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1cf1db0 msgr2=0x564bd1cf41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e315b4700 1 --2- 172.21.15.46:0/3879847663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1cf1db0 0x564bd1cf41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e315b4700 1 -- 172.21.15.46:0/3879847663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bd1abb350 con 0x564bd1ce6e90 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.214+0000 7f4e31db5700 1 --2- 172.21.15.46:0/3879847663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1ce6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.215+0000 7f4e315b4700 1 --2- 172.21.15.46:0/3879847663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6e90 0x564bd1cf1870 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c097ca0 tx=0x7f4e1c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297d5e27256bda21 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.215+0000 7f4e17fff700 1 -- 172.21.15.46:0/3879847663 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c09b230 con 0x564bd1ce6e90 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.215+0000 7f4e17fff700 1 -- 172.21.15.46:0/3879847663 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e1c09f060 con 0x564bd1ce6e90 2024-03-18T20:42:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.215+0000 7f4e17fff700 1 -- 172.21.15.46:0/3879847663 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c09a920 con 0x564bd1ce6e90 2024-03-18T20:42:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.216+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3879847663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6e90 msgr2=0x564bd1cf1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.216+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/3879847663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6e90 0x564bd1cf1870 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c097ca0 tx=0x7f4e1c003680 comp rx=0 tx=0).stop 2024-03-18T20:42:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.216+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3879847663 shutdown_connections 2024-03-18T20:42:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.216+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/3879847663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1cf1db0 0x564bd1cf41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.216+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/3879847663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1ce6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.216+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/3879847663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6e90 0x564bd1cf1870 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.216+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3879847663 >> 172.21.15.46:0/3879847663 conn(0x564bd1c5d1c0 msgr2=0x564bd1ce2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.216+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3879847663 shutdown_connections 2024-03-18T20:42:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.216+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3879847663 wait complete. 2024-03-18T20:42:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.216+0000 7f4e401aeb80 1 -- start start 2024-03-18T20:42:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.216+0000 7f4e401aeb80 1 Processor -- start 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e401aeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1cf17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e401aeb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 0x564bd1ceea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e401aeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 0x564bd1cef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e401aeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bd1bd5d20 con 0x564bd1cf1db0 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e401aeb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bd1bd59c0 con 0x564bd1ce6e90 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e401aeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bd1abd640 con 0x564bd1ce6570 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e401aeb80 10 client.0 _ll_register_callbacks cb 0x564bd1a41340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e325b6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 0x564bd1cef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e325b6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 0x564bd1cef1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60844/0 (socket says 172.21.15.46:60844) 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e325b6700 1 -- 172.21.15.46:0/289227106 learned_addr learned my addr 172.21.15.46:0/289227106 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e315b4700 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 0x564bd1ceea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e31db5700 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1cf17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e325b6700 1 -- 172.21.15.46:0/289227106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 msgr2=0x564bd1cf17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e325b6700 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1cf17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e325b6700 1 -- 172.21.15.46:0/289227106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 msgr2=0x564bd1ceea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e325b6700 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 0x564bd1ceea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e325b6700 1 -- 172.21.15.46:0/289227106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2c1a41d0 con 0x564bd1cf1db0 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e325b6700 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 0x564bd1cef1a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009f50 tx=0x7f4e24004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233acab144b7d199 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.217+0000 7f4e157fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e157fa700 1 -- 172.21.15.46:0/289227106 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e24003ab0 con 0x564bd1cf1db0 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e157fa700 1 -- 172.21.15.46:0/289227106 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f4e24007910 con 0x564bd1cf1db0 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e401aeb80 1 -- 172.21.15.46:0/289227106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bd1cda6c0 con 0x564bd1cf1db0 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e157fa700 1 -- 172.21.15.46:0/289227106 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e24007ad0 con 0x564bd1cf1db0 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e401aeb80 10 client.17913 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e401aeb80 1 -- 172.21.15.46:0/289227106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x564bd1a67b00 con 0x564bd1cf1db0 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e157fa700 1 -- 172.21.15.46:0/289227106 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564bd1cda6c0 con 0x564bd1cf1db0 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e157fa700 1 -- 172.21.15.46:0/289227106 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4e24014040 con 0x564bd1cf1db0 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e401aeb80 10 client.17913 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:59.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e401aeb80 10 client.17913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e401aeb80 20 client.17913 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564bd1d01820 0x564bd1d03c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e401aeb80 4 client.17913 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.218+0000 7f4e401aeb80 1 -- 172.21.15.46:0/289227106 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bd1c326a0 con 0x564bd1d01820 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.219+0000 7f4e31db5700 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564bd1d01820 0x564bd1d03c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.219+0000 7f4e31db5700 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564bd1d01820 0x564bd1d03c40 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4404f063b071b3a1 server_cookie=a2d4a5efe26439f1 in_seq=0 out_seq=0 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.219+0000 7f4e157fa700 10 client.17913 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e157fa700 1 -- 172.21.15.46:0/289227106 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564bd1c326a0 con 0x564bd1d01820 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e157fa700 10 client.17913 handle_command_reply: tid=0 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e401aeb80 1 client.17913 shutdown 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e401aeb80 1 -- 172.21.15.46:0/289227106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 msgr2=0x564bd1cef1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 0x564bd1cef1a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009f50 tx=0x7f4e24004320 comp rx=0 tx=0).stop 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e401aeb80 1 -- 172.21.15.46:0/289227106 shutdown_connections 2024-03-18T20:42:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564bd1d01820 0x564bd1d03c40 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 0x564bd1cef1a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1cf17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/289227106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 0x564bd1ceea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e401aeb80 1 -- 172.21.15.46:0/289227106 >> 172.21.15.46:0/289227106 conn(0x564bd1c5d1c0 msgr2=0x564bd1ce2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e157fa700 0 client.17913 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e401aeb80 1 -- 172.21.15.46:0/289227106 shutdown_connections 2024-03-18T20:42:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e401aeb80 1 -- 172.21.15.46:0/289227106 wait complete. 2024-03-18T20:42:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.220+0000 7f4e401aeb80 20 client.17913 trim_cache size 0 max 16384 2024-03-18T20:42:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e401aeb80 1 Processor -- start 2024-03-18T20:42:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e401aeb80 1 -- start start 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e401aeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6570 0x564bd1ce2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e401aeb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 0x564bd1cfd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e401aeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1cf1db0 0x564bd1cee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e401aeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bd1cfe0c0 con 0x564bd1ce6570 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e401aeb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bd1cecfb0 con 0x564bd1ce6e90 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e401aeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bd1cea8a0 con 0x564bd1cf1db0 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e31db5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6570 0x564bd1ce2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e31db5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6570 0x564bd1ce2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60852/0 (socket says 172.21.15.46:60852) 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e31db5700 1 -- 172.21.15.46:0/178316509 learned_addr learned my addr 172.21.15.46:0/178316509 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e315b4700 1 --2- 172.21.15.46:0/178316509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 0x564bd1cfd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e325b6700 1 --2- 172.21.15.46:0/178316509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1cf1db0 0x564bd1cee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e31db5700 1 -- 172.21.15.46:0/178316509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1cf1db0 msgr2=0x564bd1cee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e31db5700 1 --2- 172.21.15.46:0/178316509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1cf1db0 0x564bd1cee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e31db5700 1 -- 172.21.15.46:0/178316509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 msgr2=0x564bd1cfd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e31db5700 1 --2- 172.21.15.46:0/178316509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 0x564bd1cfd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e31db5700 1 -- 172.21.15.46:0/178316509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bd1bffe40 con 0x564bd1ce6570 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e325b6700 1 --2- 172.21.15.46:0/178316509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1cf1db0 0x564bd1cee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.236+0000 7f4e31db5700 1 --2- 172.21.15.46:0/178316509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6570 0x564bd1ce2bd0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x564bd1d04550 tx=0x7f4e28013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f857a49ba29539f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.237+0000 7f4e20ff9700 1 -- 172.21.15.46:0/178316509 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28026330 con 0x564bd1ce6570 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.237+0000 7f4e20ff9700 1 -- 172.21.15.46:0/178316509 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e28003e50 con 0x564bd1ce6570 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.237+0000 7f4e20ff9700 1 -- 172.21.15.46:0/178316509 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28026ea0 con 0x564bd1ce6570 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.237+0000 7f4e401aeb80 1 -- 172.21.15.46:0/178316509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6570 msgr2=0x564bd1ce2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.237+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/178316509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6570 0x564bd1ce2bd0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x564bd1d04550 tx=0x7f4e28013000 comp rx=0 tx=0).stop 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.237+0000 7f4e401aeb80 1 -- 172.21.15.46:0/178316509 shutdown_connections 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.237+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/178316509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1ce6570 0x564bd1ce2bd0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.237+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/178316509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1cf1db0 0x564bd1cee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.237+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/178316509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 0x564bd1cfd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.237+0000 7f4e401aeb80 1 -- 172.21.15.46:0/178316509 >> 172.21.15.46:0/178316509 conn(0x564bd1c5d1c0 msgr2=0x564bd1ce84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.237+0000 7f4e401aeb80 1 -- 172.21.15.46:0/178316509 shutdown_connections 2024-03-18T20:42:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.237+0000 7f4e401aeb80 1 -- 172.21.15.46:0/178316509 wait complete. 2024-03-18T20:42:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e401aeb80 1 -- start start 2024-03-18T20:42:59.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e401aeb80 1 Processor -- start 2024-03-18T20:42:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e401aeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1cf4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e401aeb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 0x564bd1cf4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e401aeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 0x564bd1d58a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e401aeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bd1d58f90 con 0x564bd1cf1db0 2024-03-18T20:42:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e401aeb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bd1d590d0 con 0x564bd1ce6e90 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e401aeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bd1d59290 con 0x564bd1ce6570 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e401aeb80 10 client.0 _ll_register_callbacks cb 0x7f4e24009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e325b6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 0x564bd1d58a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e325b6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 0x564bd1d58a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60862/0 (socket says 172.21.15.46:60862) 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e325b6700 1 -- 172.21.15.46:0/3178347144 learned_addr learned my addr 172.21.15.46:0/3178347144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e315b4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 0x564bd1cf4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.238+0000 7f4e31db5700 1 --2- 172.21.15.46:0/3178347144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1cf4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e325b6700 1 -- 172.21.15.46:0/3178347144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 msgr2=0x564bd1cf4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e325b6700 1 --2- 172.21.15.46:0/3178347144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1cf4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e325b6700 1 -- 172.21.15.46:0/3178347144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 msgr2=0x564bd1cf4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e325b6700 1 --2- 172.21.15.46:0/3178347144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 0x564bd1cf4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e325b6700 1 -- 172.21.15.46:0/3178347144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e280042f0 con 0x564bd1cf1db0 2024-03-18T20:42:59.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e325b6700 1 --2- 172.21.15.46:0/3178347144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 0x564bd1d58a50 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4e24000f80 tx=0x7f4e24003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566cae5d18a55d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e157fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e157fa700 1 -- 172.21.15.46:0/3178347144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e24015940 con 0x564bd1cf1db0 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3178347144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e1c09f060 con 0x564bd1cf1db0 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e157fa700 1 -- 172.21.15.46:0/3178347144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 8512+0+0 (secure 0 0 0) 0x7f4e24015aa0 con 0x564bd1cf1db0 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e157fa700 1 -- 172.21.15.46:0/3178347144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e24015ca0 con 0x564bd1cf1db0 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e401aeb80 10 client.17919 fetch_fsmap learned FSMap version 43 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.239+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3178347144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x564bd1abfc70 con 0x564bd1cf1db0 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.240+0000 7f4e157fa700 1 -- 172.21.15.46:0/3178347144 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e24036020 con 0x564bd1cf1db0 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.240+0000 7f4e157fa700 1 -- 172.21.15.46:0/3178347144 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4e24004040 con 0x564bd1cf1db0 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.240+0000 7f4e401aeb80 10 client.17919 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.240+0000 7f4e401aeb80 10 client.17919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.240+0000 7f4e401aeb80 20 client.17919 populate_metadata read hostname 'smithi046' 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.240+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/3178347144 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564bd1d5aac0 0x564bd1d5cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:42:59.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.240+0000 7f4e401aeb80 4 client.17919 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:42:59.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.240+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3178347144 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564bd1a88c90 con 0x564bd1d5aac0 2024-03-18T20:42:59.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.240+0000 7f4e31db5700 1 --2- 172.21.15.46:0/3178347144 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564bd1d5aac0 0x564bd1d5cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:42:59.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.240+0000 7f4e157fa700 10 client.17919 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:59.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.240+0000 7f4e31db5700 1 --2- 172.21.15.46:0/3178347144 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564bd1d5aac0 0x564bd1d5cee0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9cdc882e31ac0d4 server_cookie=8daa42c19362ad4a in_seq=0 out_seq=0 2024-03-18T20:42:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e157fa700 1 -- 172.21.15.46:0/3178347144 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564bd1a88c90 con 0x564bd1d5aac0 2024-03-18T20:42:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e157fa700 10 client.17919 handle_command_reply: tid=0 2024-03-18T20:42:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e401aeb80 1 client.17919 shutdown 2024-03-18T20:42:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3178347144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 msgr2=0x564bd1d58a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/3178347144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 0x564bd1d58a50 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4e24000f80 tx=0x7f4e24003ab0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3178347144 shutdown_connections 2024-03-18T20:42:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/3178347144 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564bd1d5aac0 0x564bd1d5cee0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/3178347144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bd1cf1db0 0x564bd1d58a50 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/3178347144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564bd1ce6570 0x564bd1cf4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e401aeb80 1 --2- 172.21.15.46:0/3178347144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564bd1ce6e90 0x564bd1cf4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3178347144 >> 172.21.15.46:0/3178347144 conn(0x564bd1c5d1c0 msgr2=0x564bd1ce2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e157fa700 0 client.17919 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:42:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3178347144 shutdown_connections 2024-03-18T20:42:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.269+0000 7f4e401aeb80 1 -- 172.21.15.46:0/3178347144 wait complete. 2024-03-18T20:42:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.270+0000 7f4e401aeb80 20 client.17919 trim_cache size 0 max 16384 2024-03-18T20:42:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:42:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:42:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:42:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:42:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.270+0000 7f4e34019700 1 -- 172.21.15.46:0/2162023293 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e180a1290 msgr2=0x7f4e180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.270+0000 7f4e34019700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e180a1290 0x7f4e180a3750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4e28007fd0 tx=0x7f4e280073d0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.270+0000 7f4e34019700 1 -- 172.21.15.46:0/2162023293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2c106030 msgr2=0x7f4e2c1977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:42:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.270+0000 7f4e34019700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2c106030 0x7f4e2c1977b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c000c00 tx=0x7f4e1c009580 comp rx=0 tx=0).stop 2024-03-18T20:42:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.270+0000 7f4e34019700 1 -- 172.21.15.46:0/2162023293 shutdown_connections 2024-03-18T20:42:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.270+0000 7f4e34019700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e180a1290 0x7f4e180a3750 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.270+0000 7f4e34019700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e2c19bb00 0x7f4e2c19bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.270+0000 7f4e34019700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e2c100f10 0x7f4e2c197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.270+0000 7f4e34019700 1 --2- 172.21.15.46:0/2162023293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e2c106030 0x7f4e2c1977b0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:42:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.270+0000 7f4e34019700 1 -- 172.21.15.46:0/2162023293 >> 172.21.15.46:0/2162023293 conn(0x7f4e2c0fc770 msgr2=0x7f4e2c0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:42:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.271+0000 7f4e34019700 1 -- 172.21.15.46:0/2162023293 shutdown_connections 2024-03-18T20:42:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:42:59.271+0000 7f4e34019700 1 -- 172.21.15.46:0/2162023293 wait complete. 2024-03-18T20:42:59.698 INFO:tasks.workunit.client.0.smithi046.stdout: 1 232296 3.45 MB/sec execute 234 sec latency 2244.813 ms 2024-03-18T20:43:00.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:00 smithi073 ceph-mon[47384]: pgmap v210: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-03-18T20:43:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:00 smithi155 ceph-mon[48344]: pgmap v210: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-03-18T20:43:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:00 smithi046 ceph-mon[43495]: pgmap v210: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-03-18T20:43:00.698 INFO:tasks.workunit.client.0.smithi046.stdout: 1 233859 3.48 MB/sec execute 235 sec latency 2404.108 ms 2024-03-18T20:43:01.698 INFO:tasks.workunit.client.0.smithi046.stdout: 1 234739 3.48 MB/sec execute 236 sec latency 57.114 ms 2024-03-18T20:43:02.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:02 smithi073 ceph-mon[47384]: pgmap v211: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-03-18T20:43:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:02 smithi155 ceph-mon[48344]: pgmap v211: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-03-18T20:43:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:02 smithi046 ceph-mon[43495]: pgmap v211: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-03-18T20:43:02.698 INFO:tasks.workunit.client.0.smithi046.stdout: 1 235839 3.49 MB/sec execute 237 sec latency 379.087 ms 2024-03-18T20:43:03.698 INFO:tasks.workunit.client.0.smithi046.stdout: 1 235839 3.48 MB/sec execute 238 sec latency 1379.204 ms 2024-03-18T20:43:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:04 smithi073 ceph-mon[47384]: pgmap v212: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-03-18T20:43:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:04 smithi073 ceph-mon[47384]: osdmap e83: 12 total, 12 up, 12 in 2024-03-18T20:43:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:04 smithi155 ceph-mon[48344]: pgmap v212: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-03-18T20:43:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:04 smithi155 ceph-mon[48344]: osdmap e83: 12 total, 12 up, 12 in 2024-03-18T20:43:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:04 smithi046 ceph-mon[43495]: pgmap v212: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 1.8 MiB/s wr, 211 op/s 2024-03-18T20:43:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:04 smithi046 ceph-mon[43495]: osdmap e83: 12 total, 12 up, 12 in 2024-03-18T20:43:04.698 INFO:tasks.workunit.client.0.smithi046.stdout: 1 235839 3.46 MB/sec execute 239 sec latency 2379.315 ms 2024-03-18T20:43:05.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:43:05.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:43:05.698 INFO:tasks.workunit.client.0.smithi046.stdout: 1 236645 3.46 MB/sec execute 240 sec latency 3003.874 ms 2024-03-18T20:43:05.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.752+0000 7eff36411700 1 -- 172.21.15.46:0/2692828237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff30101920 msgr2=0x7eff30105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.752+0000 7eff36411700 1 --2- 172.21.15.46:0/2692828237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff30101920 0x7eff30105960 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7eff18009990 tx=0x7eff18009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.752+0000 7eff36411700 1 -- 172.21.15.46:0/2692828237 shutdown_connections 2024-03-18T20:43:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.752+0000 7eff36411700 1 --2- 172.21.15.46:0/2692828237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff30106090 0x7eff3010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.752+0000 7eff36411700 1 --2- 172.21.15.46:0/2692828237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff30100f70 0x7eff30101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.752+0000 7eff36411700 1 --2- 172.21.15.46:0/2692828237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff30101920 0x7eff30105960 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.752+0000 7eff36411700 1 -- 172.21.15.46:0/2692828237 >> 172.21.15.46:0/2692828237 conn(0x7eff300fc7f0 msgr2=0x7eff300fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.752+0000 7eff36411700 1 -- 172.21.15.46:0/2692828237 shutdown_connections 2024-03-18T20:43:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.752+0000 7eff36411700 1 -- 172.21.15.46:0/2692828237 wait complete. 2024-03-18T20:43:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.753+0000 7eff36411700 1 Processor -- start 2024-03-18T20:43:05.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.753+0000 7eff36411700 1 -- start start 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.753+0000 7eff36411700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff30100f70 0x7eff30199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.753+0000 7eff36411700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff30106090 0x7eff30199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.753+0000 7eff36411700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3019dca0 0x7eff3019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.753+0000 7eff36411700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3019e720 con 0x7eff30106090 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.753+0000 7eff36411700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3019e890 con 0x7eff3019dca0 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.753+0000 7eff36411700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3019ea50 con 0x7eff30100f70 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.753+0000 7eff277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff30106090 0x7eff30199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff30106090 0x7eff30199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59840/0 (socket says 172.21.15.46:59840) 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff277fe700 1 -- 172.21.15.46:0/1169483399 learned_addr learned my addr 172.21.15.46:0/1169483399 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff349ae700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3019dca0 0x7eff3019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff2ffff700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff30100f70 0x7eff30199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff277fe700 1 -- 172.21.15.46:0/1169483399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff30100f70 msgr2=0x7eff30199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff277fe700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff30100f70 0x7eff30199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff277fe700 1 -- 172.21.15.46:0/1169483399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3019dca0 msgr2=0x7eff3019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff277fe700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3019dca0 0x7eff3019e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff277fe700 1 -- 172.21.15.46:0/1169483399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff28009520 con 0x7eff30106090 2024-03-18T20:43:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff349ae700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3019dca0 0x7eff3019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff277fe700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff30106090 0x7eff30199950 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7eff18005770 tx=0x7eff180057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff2dffb700 1 -- 172.21.15.46:0/1169483399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1801d070 con 0x7eff30106090 2024-03-18T20:43:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff36411700 1 -- 172.21.15.46:0/1169483399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff180095f0 con 0x7eff30106090 2024-03-18T20:43:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff2dffb700 1 -- 172.21.15.46:0/1169483399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff18021410 con 0x7eff30106090 2024-03-18T20:43:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.754+0000 7eff2dffb700 1 -- 172.21.15.46:0/1169483399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18017650 con 0x7eff30106090 2024-03-18T20:43:05.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.755+0000 7eff36411700 1 -- 172.21.15.46:0/1169483399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff301a6230 con 0x7eff30106090 2024-03-18T20:43:05.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.757+0000 7eff2dffb700 1 -- 172.21.15.46:0/1169483399 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7eff180177b0 con 0x7eff30106090 2024-03-18T20:43:05.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.757+0000 7eff2dffb700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff1c0a1290 0x7eff1c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.758+0000 7eff2dffb700 1 -- 172.21.15.46:0/1169483399 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7eff180cdea0 con 0x7eff30106090 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.758+0000 7eff2ffff700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff1c0a1290 0x7eff1c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.758+0000 7eff2ffff700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff1c0a1290 0x7eff1c0a3750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff2000f540 tx=0x7eff2000b040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.758+0000 7eff425a6b80 1 Processor -- start 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff425a6b80 1 -- start start 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff425a6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 0x563258c3ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff425a6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258c45ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff425a6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c46220 0x563258c48610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff425a6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563258b94990 con 0x563258c3aa60 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff425a6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563258b92350 con 0x563258c3b380 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff425a6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563258b8dc40 con 0x563258c46220 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 0x563258c3ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 0x563258c3ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59846/0 (socket says 172.21.15.46:59846) 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff2ffff700 1 -- 172.21.15.46:0/3532481923 learned_addr learned my addr 172.21.15.46:0/3532481923 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff277fe700 1 --2- 172.21.15.46:0/3532481923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258c45ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff349ae700 1 --2- 172.21.15.46:0/3532481923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c46220 0x563258c48610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff277fe700 1 -- 172.21.15.46:0/3532481923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c46220 msgr2=0x563258c48610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff277fe700 1 --2- 172.21.15.46:0/3532481923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c46220 0x563258c48610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff277fe700 1 -- 172.21.15.46:0/3532481923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 msgr2=0x563258c3ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff277fe700 1 --2- 172.21.15.46:0/3532481923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 0x563258c3ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff277fe700 1 -- 172.21.15.46:0/3532481923 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563258a0f3b0 con 0x563258c3b380 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff2ffff700 1 --2- 172.21.15.46:0/3532481923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 0x563258c3ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.759+0000 7eff277fe700 1 --2- 172.21.15.46:0/3532481923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258c45ce0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7eff180cb5a0 tx=0x7eff18009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc622645ee0b4ed server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.760+0000 7eff267fc700 1 -- 172.21.15.46:0/3532481923 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1809a950 con 0x563258c3b380 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.760+0000 7eff267fc700 1 -- 172.21.15.46:0/3532481923 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff1809b6f0 con 0x563258c3b380 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.760+0000 7eff267fc700 1 -- 172.21.15.46:0/3532481923 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18021810 con 0x563258c3b380 2024-03-18T20:43:05.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.760+0000 7eff425a6b80 1 -- 172.21.15.46:0/3532481923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 msgr2=0x563258c45ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.760+0000 7eff425a6b80 1 --2- 172.21.15.46:0/3532481923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258c45ce0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7eff180cb5a0 tx=0x7eff18009530 comp rx=0 tx=0).stop 2024-03-18T20:43:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.760+0000 7eff425a6b80 1 -- 172.21.15.46:0/3532481923 shutdown_connections 2024-03-18T20:43:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.760+0000 7eff425a6b80 1 --2- 172.21.15.46:0/3532481923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 0x563258c3ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.760+0000 7eff425a6b80 1 --2- 172.21.15.46:0/3532481923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c46220 0x563258c48610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.760+0000 7eff425a6b80 1 --2- 172.21.15.46:0/3532481923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258c45ce0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.760+0000 7eff425a6b80 1 -- 172.21.15.46:0/3532481923 >> 172.21.15.46:0/3532481923 conn(0x563258bb1220 msgr2=0x563258c37020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.761+0000 7eff425a6b80 1 -- 172.21.15.46:0/3532481923 shutdown_connections 2024-03-18T20:43:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.761+0000 7eff425a6b80 1 -- 172.21.15.46:0/3532481923 wait complete. 2024-03-18T20:43:05.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.761+0000 7eff425a6b80 1 -- start start 2024-03-18T20:43:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.761+0000 7eff425a6b80 1 Processor -- start 2024-03-18T20:43:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.761+0000 7eff425a6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 0x563258c46600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.761+0000 7eff425a6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3b380 0x563258c46b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.761+0000 7eff425a6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c46220 0x563258c51680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff425a6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563258b29d80 con 0x563258c3b380 2024-03-18T20:43:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff425a6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563258b29a20 con 0x563258c46220 2024-03-18T20:43:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff425a6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563258a116a0 con 0x563258c3aa60 2024-03-18T20:43:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff425a6b80 10 client.0 _ll_register_callbacks cb 0x5632589953a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff2ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 0x563258c46600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff277fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3b380 0x563258c46b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff349ae700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c46220 0x563258c51680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff2ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 0x563258c46600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58650/0 (socket says 172.21.15.46:58650) 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff2ffff700 1 -- 172.21.15.46:0/135289702 learned_addr learned my addr 172.21.15.46:0/135289702 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff2ffff700 1 -- 172.21.15.46:0/135289702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c46220 msgr2=0x563258c51680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff2ffff700 1 --2- 172.21.15.46:0/135289702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c46220 0x563258c51680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff2ffff700 1 -- 172.21.15.46:0/135289702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3b380 msgr2=0x563258c46b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff2ffff700 1 --2- 172.21.15.46:0/135289702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3b380 0x563258c46b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff2ffff700 1 -- 172.21.15.46:0/135289702 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff20003a00 con 0x563258c3aa60 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff277fe700 1 --2- 172.21.15.46:0/135289702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3b380 0x563258c46b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff2ffff700 1 --2- 172.21.15.46:0/135289702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 0x563258c46600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7eff20009fd0 tx=0x7eff20004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=807c4e14bfe55ed server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.762+0000 7eff0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff0f7fe700 1 -- 172.21.15.46:0/135289702 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2000e040 con 0x563258c3aa60 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff0f7fe700 1 -- 172.21.15.46:0/135289702 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7eff20015d40 con 0x563258c3aa60 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff425a6b80 1 -- 172.21.15.46:0/135289702 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563258c2ead0 con 0x563258c3aa60 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff0f7fe700 1 -- 172.21.15.46:0/135289702 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff20017440 con 0x563258c3aa60 2024-03-18T20:43:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff425a6b80 10 client.27622 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff425a6b80 1 -- 172.21.15.46:0/135289702 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7eff301a6230 con 0x563258c3aa60 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff0f7fe700 1 -- 172.21.15.46:0/135289702 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563258c2ead0 con 0x563258c3aa60 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff0f7fe700 1 -- 172.21.15.46:0/135289702 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7eff20023510 con 0x563258c3aa60 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff425a6b80 10 client.27622 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff425a6b80 10 client.27622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff425a6b80 20 client.27622 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff425a6b80 1 --2- 172.21.15.46:0/135289702 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563258c55b10 0x563258c57f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff425a6b80 4 client.27622 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff425a6b80 1 -- 172.21.15.46:0/135289702 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563258b86700 con 0x563258c55b10 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.763+0000 7eff277fe700 1 --2- 172.21.15.46:0/135289702 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563258c55b10 0x563258c57f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.764+0000 7eff277fe700 1 --2- 172.21.15.46:0/135289702 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563258c55b10 0x563258c57f30 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9184aace73c36c00 server_cookie=800cefc5a271ae7b in_seq=0 out_seq=0 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.764+0000 7eff0f7fe700 10 client.27622 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.764+0000 7eff0f7fe700 1 -- 172.21.15.46:0/135289702 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563258b86700 con 0x563258c55b10 2024-03-18T20:43:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.764+0000 7eff0f7fe700 10 client.27622 handle_command_reply: tid=0 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.764+0000 7eff425a6b80 1 client.27622 shutdown 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.765+0000 7eff425a6b80 1 -- 172.21.15.46:0/135289702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 msgr2=0x563258c46600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.765+0000 7eff425a6b80 1 --2- 172.21.15.46:0/135289702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 0x563258c46600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7eff20009fd0 tx=0x7eff20004ad0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.765+0000 7eff425a6b80 1 -- 172.21.15.46:0/135289702 shutdown_connections 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.765+0000 7eff425a6b80 1 --2- 172.21.15.46:0/135289702 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563258c55b10 0x563258c57f30 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.765+0000 7eff425a6b80 1 --2- 172.21.15.46:0/135289702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c46220 0x563258c51680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.765+0000 7eff425a6b80 1 --2- 172.21.15.46:0/135289702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 0x563258c46600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.765+0000 7eff0f7fe700 0 client.27622 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.765+0000 7eff425a6b80 1 --2- 172.21.15.46:0/135289702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3b380 0x563258c46b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.765+0000 7eff425a6b80 1 -- 172.21.15.46:0/135289702 >> 172.21.15.46:0/135289702 conn(0x563258bb1220 msgr2=0x563258c36c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.765+0000 7eff425a6b80 1 -- 172.21.15.46:0/135289702 shutdown_connections 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.765+0000 7eff425a6b80 1 -- 172.21.15.46:0/135289702 wait complete. 2024-03-18T20:43:05.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.765+0000 7eff425a6b80 20 client.27622 trim_cache size 0 max 16384 2024-03-18T20:43:05.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.788+0000 7eff425a6b80 1 Processor -- start 2024-03-18T20:43:05.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.788+0000 7eff425a6b80 1 -- start start 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.788+0000 7eff425a6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 0x563258c4b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.788+0000 7eff425a6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258c46600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.788+0000 7eff425a6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c46220 0x563258c46b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.788+0000 7eff425a6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563258c2e640 con 0x563258c46220 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff425a6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563258c335e0 con 0x563258c3b380 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff425a6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563258c40340 con 0x563258c3aa60 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff2ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 0x563258c4b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff277fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258c46600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff349ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c46220 0x563258c46b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff277fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258c46600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:50942/0 (socket says 172.21.15.46:50942) 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff349ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c46220 0x563258c46b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59868/0 (socket says 172.21.15.46:59868) 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff2ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 0x563258c4b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58664/0 (socket says 172.21.15.46:58664) 2024-03-18T20:43:05.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff277fe700 1 -- 172.21.15.46:0/148030638 learned_addr learned my addr 172.21.15.46:0/148030638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff277fe700 1 -- 172.21.15.46:0/148030638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 msgr2=0x563258c4b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff277fe700 1 --2- 172.21.15.46:0/148030638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 0x563258c4b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff277fe700 1 -- 172.21.15.46:0/148030638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c46220 msgr2=0x563258c46b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff277fe700 1 --2- 172.21.15.46:0/148030638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c46220 0x563258c46b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff277fe700 1 -- 172.21.15.46:0/148030638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563258c36e20 con 0x563258c3b380 2024-03-18T20:43:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff277fe700 1 --2- 172.21.15.46:0/148030638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258c46600 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7eff180cb650 tx=0x7eff18021be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e5185f47b1ec0e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.789+0000 7eff26ffd700 1 -- 172.21.15.46:0/148030638 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1809adc0 con 0x563258c3b380 2024-03-18T20:43:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.790+0000 7eff26ffd700 1 -- 172.21.15.46:0/148030638 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff1809ee20 con 0x563258c3b380 2024-03-18T20:43:05.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.790+0000 7eff26ffd700 1 -- 172.21.15.46:0/148030638 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1809b490 con 0x563258c3b380 2024-03-18T20:43:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.791+0000 7eff425a6b80 1 -- 172.21.15.46:0/148030638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 msgr2=0x563258c46600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.791+0000 7eff425a6b80 1 --2- 172.21.15.46:0/148030638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258c46600 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7eff180cb650 tx=0x7eff18021be0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.791+0000 7eff425a6b80 1 -- 172.21.15.46:0/148030638 shutdown_connections 2024-03-18T20:43:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.791+0000 7eff425a6b80 1 --2- 172.21.15.46:0/148030638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c46220 0x563258c46b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.791+0000 7eff425a6b80 1 --2- 172.21.15.46:0/148030638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c3aa60 0x563258c4b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.791+0000 7eff425a6b80 1 --2- 172.21.15.46:0/148030638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258c46600 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.791+0000 7eff425a6b80 1 -- 172.21.15.46:0/148030638 >> 172.21.15.46:0/148030638 conn(0x563258bb1220 msgr2=0x563258c35750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.791+0000 7eff425a6b80 1 -- 172.21.15.46:0/148030638 shutdown_connections 2024-03-18T20:43:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.791+0000 7eff425a6b80 1 -- 172.21.15.46:0/148030638 wait complete. 2024-03-18T20:43:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.791+0000 7eff425a6b80 1 -- start start 2024-03-18T20:43:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff425a6b80 1 Processor -- start 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff425a6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 0x563258ca3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff425a6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258ca3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff425a6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c46220 0x563258c9d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff425a6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563258c9df70 con 0x563258c3aa60 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff425a6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563258c9e0e0 con 0x563258c3b380 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff425a6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563258c9e2a0 con 0x563258c46220 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff425a6b80 10 client.0 _ll_register_callbacks cb 0x7eff200043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 0x563258ca3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff2ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 0x563258ca3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59884/0 (socket says 172.21.15.46:59884) 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff2ffff700 1 -- 172.21.15.46:0/2357630258 learned_addr learned my addr 172.21.15.46:0/2357630258 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff277fe700 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258ca3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff349ae700 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c46220 0x563258c9d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff2ffff700 1 -- 172.21.15.46:0/2357630258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c46220 msgr2=0x563258c9d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff2ffff700 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c46220 0x563258c9d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff2ffff700 1 -- 172.21.15.46:0/2357630258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 msgr2=0x563258ca3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff2ffff700 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258ca3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff2ffff700 1 -- 172.21.15.46:0/2357630258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563258a0f3b0 con 0x563258c3aa60 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.792+0000 7eff277fe700 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258ca3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.793+0000 7eff2ffff700 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 0x563258ca3550 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7eff20017fd0 tx=0x7eff20009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1baf4c2ab4a6637b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.793+0000 7eff0f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.793+0000 7eff0f7fe700 1 -- 172.21.15.46:0/2357630258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2000e040 con 0x563258c3aa60 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.793+0000 7eff0f7fe700 1 -- 172.21.15.46:0/2357630258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7eff20004b40 con 0x563258c3aa60 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.793+0000 7eff425a6b80 1 -- 172.21.15.46:0/2357630258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff1809b6f0 con 0x563258c3aa60 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.793+0000 7eff0f7fe700 1 -- 172.21.15.46:0/2357630258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff20004d00 con 0x563258c3aa60 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.793+0000 7eff425a6b80 10 client.17937 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.793+0000 7eff425a6b80 1 -- 172.21.15.46:0/2357630258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5632589bbb60 con 0x563258c3aa60 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.793+0000 7eff0f7fe700 1 -- 172.21.15.46:0/2357630258 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff1809b6f0 con 0x563258c3aa60 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.793+0000 7eff0f7fe700 1 -- 172.21.15.46:0/2357630258 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7eff200178e0 con 0x563258c3aa60 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.793+0000 7eff425a6b80 10 client.17937 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:05.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.793+0000 7eff425a6b80 10 client.17937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.794+0000 7eff425a6b80 20 client.17937 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.794+0000 7eff425a6b80 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563258cab510 0x563258cad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.794+0000 7eff425a6b80 4 client.17937 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.794+0000 7eff425a6b80 1 -- 172.21.15.46:0/2357630258 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5632589dccf0 con 0x563258cab510 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.794+0000 7eff277fe700 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563258cab510 0x563258cad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.794+0000 7eff0f7fe700 10 client.17937 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.794+0000 7eff277fe700 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563258cab510 0x563258cad930 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166394ea6dfd63e7 server_cookie=6582c09ec05e79af in_seq=0 out_seq=0 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff0f7fe700 1 -- 172.21.15.46:0/2357630258 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5632589dccf0 con 0x563258cab510 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff0f7fe700 10 client.17937 handle_command_reply: tid=0 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff425a6b80 1 client.17937 shutdown 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff425a6b80 1 -- 172.21.15.46:0/2357630258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 msgr2=0x563258ca3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff425a6b80 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 0x563258ca3550 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7eff20017fd0 tx=0x7eff20009df0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff425a6b80 1 -- 172.21.15.46:0/2357630258 shutdown_connections 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff425a6b80 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563258cab510 0x563258cad930 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff425a6b80 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563258c3aa60 0x563258ca3550 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff425a6b80 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563258c46220 0x563258c9d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff0f7fe700 0 client.17937 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff425a6b80 1 --2- 172.21.15.46:0/2357630258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563258c3b380 0x563258ca3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff425a6b80 1 -- 172.21.15.46:0/2357630258 >> 172.21.15.46:0/2357630258 conn(0x563258bb1220 msgr2=0x563258c46600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff425a6b80 1 -- 172.21.15.46:0/2357630258 shutdown_connections 2024-03-18T20:43:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff425a6b80 1 -- 172.21.15.46:0/2357630258 wait complete. 2024-03-18T20:43:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.795+0000 7eff425a6b80 20 client.17937 trim_cache size 0 max 16384 2024-03-18T20:43:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:43:05.803 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:43:05.803 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:43:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:43:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.796+0000 7eff36411700 1 -- 172.21.15.46:0/1169483399 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff1c0a1290 msgr2=0x7eff1c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.796+0000 7eff36411700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff1c0a1290 0x7eff1c0a3750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff2000f540 tx=0x7eff2000b040 comp rx=0 tx=0).stop 2024-03-18T20:43:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.796+0000 7eff36411700 1 -- 172.21.15.46:0/1169483399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff30106090 msgr2=0x7eff30199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.796+0000 7eff36411700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff30106090 0x7eff30199950 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7eff18005770 tx=0x7eff180057a0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.796+0000 7eff36411700 1 -- 172.21.15.46:0/1169483399 shutdown_connections 2024-03-18T20:43:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.796+0000 7eff36411700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff1c0a1290 0x7eff1c0a3750 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.796+0000 7eff36411700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3019dca0 0x7eff3019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.796+0000 7eff36411700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff30100f70 0x7eff30199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.796+0000 7eff36411700 1 --2- 172.21.15.46:0/1169483399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff30106090 0x7eff30199950 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.796+0000 7eff36411700 1 -- 172.21.15.46:0/1169483399 >> 172.21.15.46:0/1169483399 conn(0x7eff300fc7f0 msgr2=0x7eff300feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:05.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.796+0000 7eff36411700 1 -- 172.21.15.46:0/1169483399 shutdown_connections 2024-03-18T20:43:05.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.796+0000 7eff36411700 1 -- 172.21.15.46:0/1169483399 wait complete. 2024-03-18T20:43:05.814 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:43:05.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:43:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.973+0000 7f9082632700 1 -- 172.21.15.46:0/3383313585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f907c105fd0 msgr2=0x7f907c10db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.973+0000 7f9082632700 1 --2- 172.21.15.46:0/3383313585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f907c105fd0 0x7f907c10db30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9074009fc0 tx=0x7f907400b330 comp rx=0 tx=0).stop 2024-03-18T20:43:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.973+0000 7f9082632700 1 -- 172.21.15.46:0/3383313585 shutdown_connections 2024-03-18T20:43:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.973+0000 7f9082632700 1 --2- 172.21.15.46:0/3383313585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c0ff3c0 0x7f907c0ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.973+0000 7f9082632700 1 --2- 172.21.15.46:0/3383313585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f907c105fd0 0x7f907c10db30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.973+0000 7f9082632700 1 --2- 172.21.15.46:0/3383313585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f907c0ffd70 0x7f907c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.973+0000 7f9082632700 1 -- 172.21.15.46:0/3383313585 >> 172.21.15.46:0/3383313585 conn(0x7f907c076070 msgr2=0x7f907c0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:05.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.973+0000 7f9082632700 1 -- 172.21.15.46:0/3383313585 shutdown_connections 2024-03-18T20:43:05.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.973+0000 7f9082632700 1 -- 172.21.15.46:0/3383313585 wait complete. 2024-03-18T20:43:05.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.973+0000 7f9082632700 1 Processor -- start 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f9082632700 1 -- start start 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f9082632700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f907c0ff3c0 0x7f907c197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f9082632700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c105fd0 0x7f907c197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f9082632700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f907c19b840 0x7f907c19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f9082632700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c19c2c0 con 0x7f907c105fd0 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f9082632700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c19c430 con 0x7f907c19b840 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f9082632700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c19c5f0 con 0x7f907c0ff3c0 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f907bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f907c0ff3c0 0x7f907c197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f907b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c105fd0 0x7f907c197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f907bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f907c0ff3c0 0x7f907c197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58702/0 (socket says 172.21.15.46:58702) 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f907b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c105fd0 0x7f907c197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59916/0 (socket says 172.21.15.46:59916) 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f907bfff700 1 -- 172.21.15.46:0/2462472480 learned_addr learned my addr 172.21.15.46:0/2462472480 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f9080bcf700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f907c19b840 0x7f907c19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f907bfff700 1 -- 172.21.15.46:0/2462472480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f907c19b840 msgr2=0x7f907c19bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:05.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f907bfff700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f907c19b840 0x7f907c19bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f907bfff700 1 -- 172.21.15.46:0/2462472480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c105fd0 msgr2=0x7f907c197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:05.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f907bfff700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c105fd0 0x7f907c197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:05.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.974+0000 7f907bfff700 1 -- 172.21.15.46:0/2462472480 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9074009c40 con 0x7f907c0ff3c0 2024-03-18T20:43:05.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.975+0000 7f907bfff700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f907c0ff3c0 0x7f907c197050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e9c0 tx=0x7f906c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:05.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.975+0000 7f90797fa700 1 -- 172.21.15.46:0/2462472480 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c004a30 con 0x7f907c0ff3c0 2024-03-18T20:43:05.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.975+0000 7f90797fa700 1 -- 172.21.15.46:0/2462472480 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f906c00cc80 con 0x7f907c0ff3c0 2024-03-18T20:43:05.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.975+0000 7f9082632700 1 -- 172.21.15.46:0/2462472480 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907c1a3b50 con 0x7f907c0ff3c0 2024-03-18T20:43:05.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.975+0000 7f90797fa700 1 -- 172.21.15.46:0/2462472480 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c0054e0 con 0x7f907c0ff3c0 2024-03-18T20:43:05.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.975+0000 7f9082632700 1 -- 172.21.15.46:0/2462472480 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907c1a4030 con 0x7f907c0ff3c0 2024-03-18T20:43:05.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.976+0000 7f9082632700 1 -- 172.21.15.46:0/2462472480 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907c1091f0 con 0x7f907c0ff3c0 2024-03-18T20:43:05.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.978+0000 7f90797fa700 1 -- 172.21.15.46:0/2462472480 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f906c004b90 con 0x7f907c0ff3c0 2024-03-18T20:43:05.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.979+0000 7f90797fa700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90640a1290 0x7f90640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:05.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.979+0000 7f907b7fe700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90640a1290 0x7f90640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:05.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.979+0000 7f90797fa700 1 -- 172.21.15.46:0/2462472480 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f906c014070 con 0x7f907c0ff3c0 2024-03-18T20:43:05.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.979+0000 7f907b7fe700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90640a1290 0x7f90640a3750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f907c198550 tx=0x7f9070009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:05.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:05.982+0000 7f90797fa700 1 -- 172.21.15.46:0/2462472480 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f906c0961a0 con 0x7f907c0ff3c0 2024-03-18T20:43:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:05 smithi073 ceph-mon[47384]: pgmap v214: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-03-18T20:43:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:05 smithi155 ceph-mon[48344]: pgmap v214: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-03-18T20:43:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:05 smithi046 ceph-mon[43495]: pgmap v214: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-03-18T20:43:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.310+0000 7f9082632700 1 -- 172.21.15.46:0/2462472480 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f907c102470 con 0x7f907c0ff3c0 2024-03-18T20:43:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.311+0000 7f90797fa700 1 -- 172.21.15.46:0/2462472480 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+10919 (secure 0 0 0) 0x7f906c0997c0 con 0x7f907c0ff3c0 2024-03-18T20:43:06.314 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:43:06.315 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:41:55.183335+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.314+0000 7f9082632700 1 -- 172.21.15.46:0/2462472480 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90640a1290 msgr2=0x7f90640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.314+0000 7f9082632700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90640a1290 0x7f90640a3750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f907c198550 tx=0x7f9070009260 comp rx=0 tx=0).stop 2024-03-18T20:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.314+0000 7f9082632700 1 -- 172.21.15.46:0/2462472480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f907c0ff3c0 msgr2=0x7f907c197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.314+0000 7f9082632700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f907c0ff3c0 0x7f907c197050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e9c0 tx=0x7f906c00ecd0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.314+0000 7f9082632700 1 -- 172.21.15.46:0/2462472480 shutdown_connections 2024-03-18T20:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.314+0000 7f9082632700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90640a1290 0x7f90640a3750 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.314+0000 7f9082632700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f907c19b840 0x7f907c19bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.314+0000 7f9082632700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f907c0ff3c0 0x7f907c197050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.314+0000 7f9082632700 1 --2- 172.21.15.46:0/2462472480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c105fd0 0x7f907c197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.314+0000 7f9082632700 1 -- 172.21.15.46:0/2462472480 >> 172.21.15.46:0/2462472480 conn(0x7f907c076070 msgr2=0x7f907c104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.314+0000 7f9082632700 1 -- 172.21.15.46:0/2462472480 shutdown_connections 2024-03-18T20:43:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.314+0000 7f9082632700 1 -- 172.21.15.46:0/2462472480 wait complete. 2024-03-18T20:43:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 43 2024-03-18T20:43:06.331 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:43:06.331 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:43:06.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.471+0000 7f1ff0515700 1 -- 172.21.15.46:0/431908162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe8100710 msgr2=0x7f1fe8100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.471+0000 7f1ff0515700 1 --2- 172.21.15.46:0/431908162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe8100710 0x7f1fe8100af0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009990 tx=0x7f1fe4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.471+0000 7f1ff0515700 1 -- 172.21.15.46:0/431908162 shutdown_connections 2024-03-18T20:43:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.471+0000 7f1ff0515700 1 --2- 172.21.15.46:0/431908162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe8100710 0x7f1fe8100af0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.471+0000 7f1ff0515700 1 --2- 172.21.15.46:0/431908162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe810d750 0x7f1fe810fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.471+0000 7f1ff0515700 1 --2- 172.21.15.46:0/431908162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe8101030 0x7f1fe810d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.471+0000 7f1ff0515700 1 -- 172.21.15.46:0/431908162 >> 172.21.15.46:0/431908162 conn(0x7f1fe8076d40 msgr2=0x7f1fe8077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.471+0000 7f1ff0515700 1 -- 172.21.15.46:0/431908162 shutdown_connections 2024-03-18T20:43:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.471+0000 7f1ff0515700 1 -- 172.21.15.46:0/431908162 wait complete. 2024-03-18T20:43:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1ff0515700 1 Processor -- start 2024-03-18T20:43:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1ff0515700 1 -- start start 2024-03-18T20:43:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1ff0515700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe8100710 0x7f1fe8199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1ff0515700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe8101030 0x7f1fe81999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1ff0515700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe810d750 0x7f1fe819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1ff0515700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe819e3a0 con 0x7f1fe8101030 2024-03-18T20:43:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1ff0515700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe819e510 con 0x7f1fe8100710 2024-03-18T20:43:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1ff0515700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe819e6d0 con 0x7f1fe810d750 2024-03-18T20:43:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1fedab0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe8101030 0x7f1fe81999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1fee2b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe8100710 0x7f1fe8199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1fee2b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe8100710 0x7f1fe8199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:50996/0 (socket says 172.21.15.46:50996) 2024-03-18T20:43:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1feeab2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe810d750 0x7f1fe819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1fee2b1700 1 -- 172.21.15.46:0/1374085980 learned_addr learned my addr 172.21.15.46:0/1374085980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1fedab0700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe8101030 0x7f1fe81999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59934/0 (socket says 172.21.15.46:59934) 2024-03-18T20:43:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1fee2b1700 1 -- 172.21.15.46:0/1374085980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe810d750 msgr2=0x7f1fe819dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe810d750 0x7f1fe819dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1fee2b1700 1 -- 172.21.15.46:0/1374085980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe8101030 msgr2=0x7f1fe81999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe8101030 0x7f1fe81999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1fee2b1700 1 -- 172.21.15.46:0/1374085980 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe40095f0 con 0x7f1fe8100710 2024-03-18T20:43:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1fedab0700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe8101030 0x7f1fe81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.472+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe8100710 0x7f1fe8199480 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4004730 tx=0x7f1fe4004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.473+0000 7f1fdf7fe700 1 -- 172.21.15.46:0/1374085980 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe401d070 con 0x7f1fe8100710 2024-03-18T20:43:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.473+0000 7f1fdf7fe700 1 -- 172.21.15.46:0/1374085980 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fe4022470 con 0x7f1fe8100710 2024-03-18T20:43:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.473+0000 7f1ff0515700 1 -- 172.21.15.46:0/1374085980 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe819e950 con 0x7f1fe8100710 2024-03-18T20:43:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.473+0000 7f1fdf7fe700 1 -- 172.21.15.46:0/1374085980 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4017650 con 0x7f1fe8100710 2024-03-18T20:43:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.473+0000 7f1ff0515700 1 -- 172.21.15.46:0/1374085980 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe81a61f0 con 0x7f1fe8100710 2024-03-18T20:43:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.475+0000 7f1fdf7fe700 1 -- 172.21.15.46:0/1374085980 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1fe40225e0 con 0x7f1fe8100710 2024-03-18T20:43:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.476+0000 7f1fdf7fe700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1fd40a1290 0x7f1fd40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.476+0000 7f1fedab0700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1fd40a1290 0x7f1fd40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.476+0000 7f1fdf7fe700 1 -- 172.21.15.46:0/1374085980 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f1fe40cdee0 con 0x7f1fe8100710 2024-03-18T20:43:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.477+0000 7f1fedab0700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1fd40a1290 0x7f1fd40a3750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1fe819a9d0 tx=0x7f1fd8009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.477+0000 7f1ffc6aab80 1 Processor -- start 2024-03-18T20:43:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.477+0000 7f1ffc6aab80 1 -- start start 2024-03-18T20:43:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.477+0000 7f1ffc6aab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 0x55adb7759e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.477+0000 7f1ffc6aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 0x55adb7764ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.477+0000 7f1ffc6aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb7765220 0x55adb7767610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.477+0000 7f1ffc6aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb76b3990 con 0x55adb7765220 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.477+0000 7f1ffc6aab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb76b1350 con 0x55adb7759a60 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.477+0000 7f1ffc6aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb76acc40 con 0x55adb775a380 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.477+0000 7f1fedab0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 0x55adb7764ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.477+0000 7f1fee2b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 0x55adb7759e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fedab0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 0x55adb7764ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58730/0 (socket says 172.21.15.46:58730) 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fee2b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 0x55adb7759e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51000/0 (socket says 172.21.15.46:51000) 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fedab0700 1 -- 172.21.15.46:0/4213983729 learned_addr learned my addr 172.21.15.46:0/4213983729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1feeab2700 1 --2- 172.21.15.46:0/4213983729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb7765220 0x55adb7767610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fee2b1700 1 -- 172.21.15.46:0/4213983729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 msgr2=0x55adb7764ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/4213983729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 0x55adb7764ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fee2b1700 1 -- 172.21.15.46:0/4213983729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb7765220 msgr2=0x55adb7767610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/4213983729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb7765220 0x55adb7767610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fee2b1700 1 -- 172.21.15.46:0/4213983729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adb752e3b0 con 0x55adb7759a60 2024-03-18T20:43:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fedab0700 1 --2- 172.21.15.46:0/4213983729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 0x55adb7764ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/4213983729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 0x55adb7759e40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40cb580 tx=0x7f1fe409ae10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c017fe254dffbb39 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fdcff9700 1 -- 172.21.15.46:0/4213983729 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40a10d0 con 0x55adb7759a60 2024-03-18T20:43:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fdcff9700 1 -- 172.21.15.46:0/4213983729 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fe409b6d0 con 0x55adb7759a60 2024-03-18T20:43:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.478+0000 7f1fdcff9700 1 -- 172.21.15.46:0/4213983729 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4095e40 con 0x55adb7759a60 2024-03-18T20:43:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.479+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/4213983729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 msgr2=0x55adb7759e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.479+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/4213983729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 0x55adb7759e40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40cb580 tx=0x7f1fe409ae10 comp rx=0 tx=0).stop 2024-03-18T20:43:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.479+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/4213983729 shutdown_connections 2024-03-18T20:43:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.479+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/4213983729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb7765220 0x55adb7767610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.479+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/4213983729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 0x55adb7759e40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.479+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/4213983729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 0x55adb7764ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.479+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/4213983729 >> 172.21.15.46:0/4213983729 conn(0x55adb76d0220 msgr2=0x55adb7756020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.479+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/4213983729 shutdown_connections 2024-03-18T20:43:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.479+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/4213983729 wait complete. 2024-03-18T20:43:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1ffc6aab80 1 -- start start 2024-03-18T20:43:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1ffc6aab80 1 Processor -- start 2024-03-18T20:43:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1ffc6aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb7759a60 0x55adb7765600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1ffc6aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb775a380 0x55adb7765b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1ffc6aab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7765220 0x55adb7770680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1ffc6aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7648d80 con 0x55adb775a380 2024-03-18T20:43:06.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1ffc6aab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7648a20 con 0x55adb7765220 2024-03-18T20:43:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1ffc6aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb75306a0 con 0x55adb7759a60 2024-03-18T20:43:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1ffc6aab80 10 client.0 _ll_register_callbacks cb 0x55adb74b43a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1fee2b1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb7759a60 0x55adb7765600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1fedab0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb775a380 0x55adb7765b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1fee2b1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb7759a60 0x55adb7765600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58736/0 (socket says 172.21.15.46:58736) 2024-03-18T20:43:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1fee2b1700 1 -- 172.21.15.46:0/2022055049 learned_addr learned my addr 172.21.15.46:0/2022055049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1feeab2700 1 --2- 172.21.15.46:0/2022055049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7765220 0x55adb7770680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1fee2b1700 1 -- 172.21.15.46:0/2022055049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7765220 msgr2=0x55adb7770680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/2022055049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7765220 0x55adb7770680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1fee2b1700 1 -- 172.21.15.46:0/2022055049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb775a380 msgr2=0x55adb7765b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.480+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/2022055049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb775a380 0x55adb7765b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.481+0000 7f1fee2b1700 1 -- 172.21.15.46:0/2022055049 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd8017070 con 0x55adb7759a60 2024-03-18T20:43:06.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.481+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/2022055049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb7759a60 0x55adb7765600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40974d0 tx=0x7f1fe4095e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43f86d3dc5b75898 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.481+0000 7f1fd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:43:06.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.481+0000 7f1fd1ffb700 1 -- 172.21.15.46:0/2022055049 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40a10d0 con 0x55adb7759a60 2024-03-18T20:43:06.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.481+0000 7f1fd1ffb700 1 -- 172.21.15.46:0/2022055049 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f1fe409afd0 con 0x55adb7759a60 2024-03-18T20:43:06.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.481+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2022055049 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55adb774dad0 con 0x55adb7759a60 2024-03-18T20:43:06.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.481+0000 7f1fd1ffb700 1 -- 172.21.15.46:0/2022055049 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fe40a33c0 con 0x55adb7759a60 2024-03-18T20:43:06.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.481+0000 7f1ffc6aab80 10 client.27646 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:06.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.481+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2022055049 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f1fe81a61f0 con 0x55adb7759a60 2024-03-18T20:43:06.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.481+0000 7f1fd1ffb700 1 -- 172.21.15.46:0/2022055049 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55adb774dad0 con 0x55adb7759a60 2024-03-18T20:43:06.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.482+0000 7f1fd1ffb700 1 -- 172.21.15.46:0/2022055049 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1fe409fd20 con 0x55adb7759a60 2024-03-18T20:43:06.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.482+0000 7f1ffc6aab80 10 client.27646 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:06.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.482+0000 7f1ffc6aab80 10 client.27646 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:43:06.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.482+0000 7f1ffc6aab80 20 client.27646 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:06.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.482+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/2022055049 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55adb7774640 0x55adb7776a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.482+0000 7f1ffc6aab80 4 client.27646 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:06.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.482+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2022055049 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adb76a5700 con 0x55adb7774640 2024-03-18T20:43:06.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.482+0000 7f1fedab0700 1 --2- 172.21.15.46:0/2022055049 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55adb7774640 0x55adb7776a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.482+0000 7f1fedab0700 1 --2- 172.21.15.46:0/2022055049 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55adb7774640 0x55adb7776a30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=937f3a61bda9692b server_cookie=3d8299ec1c854642 in_seq=0 out_seq=0 2024-03-18T20:43:06.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.482+0000 7f1fd1ffb700 10 client.27646 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:43:06.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1fd1ffb700 1 -- 172.21.15.46:0/2022055049 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55adb76a5700 con 0x55adb7774640 2024-03-18T20:43:06.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1fd1ffb700 10 client.27646 handle_command_reply: tid=0 2024-03-18T20:43:06.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1ffc6aab80 1 client.27646 shutdown 2024-03-18T20:43:06.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2022055049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb7759a60 msgr2=0x55adb7765600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/2022055049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb7759a60 0x55adb7765600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40974d0 tx=0x7f1fe4095e90 comp rx=0 tx=0).stop 2024-03-18T20:43:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2022055049 shutdown_connections 2024-03-18T20:43:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/2022055049 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55adb7774640 0x55adb7776a30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/2022055049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7765220 0x55adb7770680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/2022055049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb7759a60 0x55adb7765600 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/2022055049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb775a380 0x55adb7765b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1fd1ffb700 0 client.27646 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:43:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2022055049 >> 172.21.15.46:0/2022055049 conn(0x55adb76d0220 msgr2=0x55adb7755c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2022055049 shutdown_connections 2024-03-18T20:43:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2022055049 wait complete. 2024-03-18T20:43:06.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.483+0000 7f1ffc6aab80 20 client.27646 trim_cache size 0 max 16384 2024-03-18T20:43:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.497+0000 7f1ffc6aab80 1 Processor -- start 2024-03-18T20:43:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.497+0000 7f1ffc6aab80 1 -- start start 2024-03-18T20:43:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.497+0000 7f1ffc6aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb7759a60 0x55adb776a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.497+0000 7f1ffc6aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 0x55adb775ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.497+0000 7f1ffc6aab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7765220 0x55adb7765600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.497+0000 7f1ffc6aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb77716a0 con 0x55adb7759a60 2024-03-18T20:43:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.497+0000 7f1ffc6aab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb7771000 con 0x55adb7765220 2024-03-18T20:43:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.497+0000 7f1ffc6aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb774ffd0 con 0x55adb775a380 2024-03-18T20:43:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fee2b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb7759a60 0x55adb776a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fedab0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 0x55adb775ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fee2b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb7759a60 0x55adb776a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59948/0 (socket says 172.21.15.46:59948) 2024-03-18T20:43:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fee2b1700 1 -- 172.21.15.46:0/2614054946 learned_addr learned my addr 172.21.15.46:0/2614054946 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1feeab2700 1 --2- 172.21.15.46:0/2614054946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7765220 0x55adb7765600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fee2b1700 1 -- 172.21.15.46:0/2614054946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 msgr2=0x55adb775ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/2614054946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 0x55adb775ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fee2b1700 1 -- 172.21.15.46:0/2614054946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7765220 msgr2=0x55adb7765600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/2614054946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7765220 0x55adb7765600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fee2b1700 1 -- 172.21.15.46:0/2614054946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adb74dab60 con 0x55adb7759a60 2024-03-18T20:43:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fedab0700 1 --2- 172.21.15.46:0/2614054946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 0x55adb775ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/2614054946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb7759a60 0x55adb776a620 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1fe409af60 tx=0x7f1fe400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377c6852b3595d35 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fdd7fa700 1 -- 172.21.15.46:0/2614054946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4097130 con 0x55adb7759a60 2024-03-18T20:43:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fdd7fa700 1 -- 172.21.15.46:0/2614054946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fe409afd0 con 0x55adb7759a60 2024-03-18T20:43:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.498+0000 7f1fdd7fa700 1 -- 172.21.15.46:0/2614054946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4096dc0 con 0x55adb7759a60 2024-03-18T20:43:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.499+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2614054946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb7759a60 msgr2=0x55adb776a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.499+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/2614054946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb7759a60 0x55adb776a620 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1fe409af60 tx=0x7f1fe400bf90 comp rx=0 tx=0).stop 2024-03-18T20:43:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.499+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2614054946 shutdown_connections 2024-03-18T20:43:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.499+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/2614054946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb7759a60 0x55adb776a620 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.499+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/2614054946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7765220 0x55adb7765600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.499+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/2614054946 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb775a380 0x55adb775ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.499+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2614054946 >> 172.21.15.46:0/2614054946 conn(0x55adb76d0220 msgr2=0x55adb7755b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.499+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2614054946 shutdown_connections 2024-03-18T20:43:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.499+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/2614054946 wait complete. 2024-03-18T20:43:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.499+0000 7f1ffc6aab80 1 -- start start 2024-03-18T20:43:06.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1ffc6aab80 1 Processor -- start 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1ffc6aab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 0x55adb77bc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1ffc6aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb775a380 0x55adb77bcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1ffc6aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb7765220 0x55adb77c7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1ffc6aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb77c79a0 con 0x55adb775a380 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1ffc6aab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb77c7ae0 con 0x55adb7759a60 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1ffc6aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adb77c7c50 con 0x55adb7765220 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1ffc6aab80 10 client.0 _ll_register_callbacks cb 0x7f1fe0009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1fedab0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb775a380 0x55adb77bcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1fee2b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 0x55adb77bc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1fedab0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb775a380 0x55adb77bcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59956/0 (socket says 172.21.15.46:59956) 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1feeab2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb7765220 0x55adb77c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1fedab0700 1 -- 172.21.15.46:0/862087747 learned_addr learned my addr 172.21.15.46:0/862087747 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 0x55adb77bc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51030/0 (socket says 172.21.15.46:51030) 2024-03-18T20:43:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1fee2b1700 1 -- 172.21.15.46:0/862087747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb7765220 msgr2=0x55adb77c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb7765220 0x55adb77c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1fee2b1700 1 -- 172.21.15.46:0/862087747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb775a380 msgr2=0x55adb77bcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb775a380 0x55adb77bcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1fee2b1700 1 -- 172.21.15.46:0/862087747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd8004560 con 0x55adb7759a60 2024-03-18T20:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.500+0000 7f1fedab0700 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb775a380 0x55adb77bcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1fee2b1700 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 0x55adb77bc5d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40974d0 tx=0x7f1fe4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3827391c4d90d4a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1fd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1fd1ffb700 1 -- 172.21.15.46:0/862087747 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40a33c0 con 0x55adb7759a60 2024-03-18T20:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1fd1ffb700 1 -- 172.21.15.46:0/862087747 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f1fe409ee00 con 0x55adb7759a60 2024-03-18T20:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/862087747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fe409b6d0 con 0x55adb7759a60 2024-03-18T20:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1fd1ffb700 1 -- 172.21.15.46:0/862087747 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fe40a3050 con 0x55adb7759a60 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1ffc6aab80 10 client.27638 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/862087747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f1fe81a61f0 con 0x55adb7759a60 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1fd1ffb700 1 -- 172.21.15.46:0/862087747 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fe409b6d0 con 0x55adb7759a60 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1fd1ffb700 1 -- 172.21.15.46:0/862087747 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1fe409fd20 con 0x55adb7759a60 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1ffc6aab80 10 client.27638 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1ffc6aab80 10 client.27638 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1ffc6aab80 20 client.27638 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55adb77c8f70 0x55adb77cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1ffc6aab80 4 client.27638 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.501+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/862087747 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55adb74fbcf0 con 0x55adb77c8f70 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1fedab0700 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55adb77c8f70 0x55adb77cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1fedab0700 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55adb77c8f70 0x55adb77cb360 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=218f204b70e65c66 server_cookie=2a60c38ac409144d in_seq=0 out_seq=0 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1fd1ffb700 10 client.27638 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:43:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1fd1ffb700 1 -- 172.21.15.46:0/862087747 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55adb74fbcf0 con 0x55adb77c8f70 2024-03-18T20:43:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1fd1ffb700 10 client.27638 handle_command_reply: tid=0 2024-03-18T20:43:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1ffc6aab80 1 client.27638 shutdown 2024-03-18T20:43:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/862087747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 msgr2=0x55adb77bc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 0x55adb77bc5d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40974d0 tx=0x7f1fe4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/862087747 shutdown_connections 2024-03-18T20:43:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55adb77c8f70 0x55adb77cb360 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55adb7765220 0x55adb77c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55adb7759a60 0x55adb77bc5d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1ffc6aab80 1 --2- 172.21.15.46:0/862087747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adb775a380 0x55adb77bcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/862087747 >> 172.21.15.46:0/862087747 conn(0x55adb76d0220 msgr2=0x55adb7755aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.502+0000 7f1fd1ffb700 0 client.27638 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:43:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/862087747 shutdown_connections 2024-03-18T20:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ffc6aab80 1 -- 172.21.15.46:0/862087747 wait complete. 2024-03-18T20:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ffc6aab80 20 client.27638 trim_cache size 0 max 16384 2024-03-18T20:43:06.513 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:43:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ff0515700 1 -- 172.21.15.46:0/1374085980 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1fd40a1290 msgr2=0x7f1fd40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ff0515700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1fd40a1290 0x7f1fd40a3750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1fe819a9d0 tx=0x7f1fd8009260 comp rx=0 tx=0).stop 2024-03-18T20:43:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ff0515700 1 -- 172.21.15.46:0/1374085980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe8100710 msgr2=0x7f1fe8199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ff0515700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe8100710 0x7f1fe8199480 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4004730 tx=0x7f1fe4004760 comp rx=0 tx=0).stop 2024-03-18T20:43:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ff0515700 1 -- 172.21.15.46:0/1374085980 shutdown_connections 2024-03-18T20:43:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ff0515700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1fd40a1290 0x7f1fd40a3750 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ff0515700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1fe810d750 0x7f1fe819dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ff0515700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe8100710 0x7f1fe8199480 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ff0515700 1 --2- 172.21.15.46:0/1374085980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fe8101030 0x7f1fe81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ff0515700 1 -- 172.21.15.46:0/1374085980 >> 172.21.15.46:0/1374085980 conn(0x7f1fe8076d40 msgr2=0x7f1fe810eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.503+0000 7f1ff0515700 1 -- 172.21.15.46:0/1374085980 shutdown_connections 2024-03-18T20:43:06.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.504+0000 7f1ff0515700 1 -- 172.21.15.46:0/1374085980 wait complete. 2024-03-18T20:43:06.520 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:43:06.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:43:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.662+0000 7f4c3c43b700 1 -- 172.21.15.46:0/2874924703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c34101900 msgr2=0x7f4c34105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.662+0000 7f4c3c43b700 1 --2- 172.21.15.46:0/2874924703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c34101900 0x7f4c34105940 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009990 tx=0x7f4c24009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.662+0000 7f4c3c43b700 1 -- 172.21.15.46:0/2874924703 shutdown_connections 2024-03-18T20:43:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.662+0000 7f4c3c43b700 1 --2- 172.21.15.46:0/2874924703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34106070 0x7f4c3410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.662+0000 7f4c3c43b700 1 --2- 172.21.15.46:0/2874924703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c34100f50 0x7f4c34101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.662+0000 7f4c3c43b700 1 --2- 172.21.15.46:0/2874924703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c34101900 0x7f4c34105940 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.662+0000 7f4c3c43b700 1 -- 172.21.15.46:0/2874924703 >> 172.21.15.46:0/2874924703 conn(0x7f4c340fc7f0 msgr2=0x7f4c340fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.662+0000 7f4c3c43b700 1 -- 172.21.15.46:0/2874924703 shutdown_connections 2024-03-18T20:43:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.662+0000 7f4c3c43b700 1 -- 172.21.15.46:0/2874924703 wait complete. 2024-03-18T20:43:06.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.663+0000 7f4c3c43b700 1 Processor -- start 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.663+0000 7f4c3c43b700 1 -- start start 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.663+0000 7f4c3c43b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c34100f50 0x7f4c34197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.663+0000 7f4c3c43b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34106070 0x7f4c34197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.663+0000 7f4c3c43b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c3419bae0 0x7f4c3419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.663+0000 7f4c3c43b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3419c560 con 0x7f4c34100f50 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.663+0000 7f4c3c43b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3419c6d0 con 0x7f4c34106070 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.663+0000 7f4c3c43b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3419c890 con 0x7f4c3419bae0 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.663+0000 7f4c3a1d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c34100f50 0x7f4c34197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.663+0000 7f4c3a1d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c34100f50 0x7f4c34197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59964/0 (socket says 172.21.15.46:59964) 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.663+0000 7f4c3a1d7700 1 -- 172.21.15.46:0/922368597 learned_addr learned my addr 172.21.15.46:0/922368597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.663+0000 7f4c3a9d8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c3419bae0 0x7f4c3419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c399d6700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34106070 0x7f4c34197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c3a1d7700 1 -- 172.21.15.46:0/922368597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c3419bae0 msgr2=0x7f4c3419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c3a1d7700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c3419bae0 0x7f4c3419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c3a1d7700 1 -- 172.21.15.46:0/922368597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34106070 msgr2=0x7f4c34197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c3a1d7700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34106070 0x7f4c34197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c3a1d7700 1 -- 172.21.15.46:0/922368597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240095f0 con 0x7f4c34100f50 2024-03-18T20:43:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c3a9d8700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c3419bae0 0x7f4c3419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c399d6700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34106070 0x7f4c34197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c3a1d7700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c34100f50 0x7f4c34197200 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000e9a0 tx=0x7f4c3000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c2b7fe700 1 -- 172.21.15.46:0/922368597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30009790 con 0x7f4c34100f50 2024-03-18T20:43:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c2b7fe700 1 -- 172.21.15.46:0/922368597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c3000cbd0 con 0x7f4c34100f50 2024-03-18T20:43:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c3c43b700 1 -- 172.21.15.46:0/922368597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c341a3df0 con 0x7f4c34100f50 2024-03-18T20:43:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c2b7fe700 1 -- 172.21.15.46:0/922368597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30018680 con 0x7f4c34100f50 2024-03-18T20:43:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.664+0000 7f4c3c43b700 1 -- 172.21.15.46:0/922368597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c341a4270 con 0x7f4c34100f50 2024-03-18T20:43:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.667+0000 7f4c2b7fe700 1 -- 172.21.15.46:0/922368597 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f4c3000cd40 con 0x7f4c34100f50 2024-03-18T20:43:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.667+0000 7f4c2b7fe700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4c200a1290 0x7f4c200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.667+0000 7f4c399d6700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4c200a1290 0x7f4c200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.667+0000 7f4c2b7fe700 1 -- 172.21.15.46:0/922368597 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f4c30014070 con 0x7f4c34100f50 2024-03-18T20:43:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.668+0000 7f4c485d0b80 1 Processor -- start 2024-03-18T20:43:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.668+0000 7f4c399d6700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4c200a1290 0x7f4c200a3750 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006010 tx=0x7f4c24005940 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.668+0000 7f4c485d0b80 1 -- start start 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.668+0000 7f4c485d0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30ba60 0x55a3ae30be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.668+0000 7f4c485d0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae30c380 0x55a3ae316ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.668+0000 7f4c485d0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae317220 0x55a3ae319610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.668+0000 7f4c485d0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ae265990 con 0x55a3ae317220 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.668+0000 7f4c485d0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ae263350 con 0x55a3ae30ba60 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.668+0000 7f4c485d0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ae25ec40 con 0x55a3ae30c380 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.668+0000 7f4c3a9d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae317220 0x55a3ae319610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.668+0000 7f4c399d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae30c380 0x55a3ae316ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c399d6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae30c380 0x55a3ae316ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58774/0 (socket says 172.21.15.46:58774) 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c3a9d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae317220 0x55a3ae319610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59980/0 (socket says 172.21.15.46:59980) 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c399d6700 1 -- 172.21.15.46:0/4135607869 learned_addr learned my addr 172.21.15.46:0/4135607869 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c3a1d7700 1 --2- 172.21.15.46:0/4135607869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30ba60 0x55a3ae30be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c399d6700 1 -- 172.21.15.46:0/4135607869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30ba60 msgr2=0x55a3ae30be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c399d6700 1 --2- 172.21.15.46:0/4135607869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30ba60 0x55a3ae30be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c399d6700 1 -- 172.21.15.46:0/4135607869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae317220 msgr2=0x55a3ae319610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c399d6700 1 --2- 172.21.15.46:0/4135607869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae317220 0x55a3ae319610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c399d6700 1 -- 172.21.15.46:0/4135607869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3ae0e03b0 con 0x55a3ae30c380 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c399d6700 1 --2- 172.21.15.46:0/4135607869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae30c380 0x55a3ae316ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c24015000 tx=0x7f4c2401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8083a881054b07a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c28da8700 1 -- 172.21.15.46:0/4135607869 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2402b070 con 0x55a3ae30c380 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c28da8700 1 -- 172.21.15.46:0/4135607869 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c2401e040 con 0x55a3ae30c380 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.669+0000 7f4c28da8700 1 -- 172.21.15.46:0/4135607869 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24004e80 con 0x55a3ae30c380 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.670+0000 7f4c485d0b80 1 -- 172.21.15.46:0/4135607869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae30c380 msgr2=0x55a3ae316ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.670+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/4135607869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae30c380 0x55a3ae316ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c24015000 tx=0x7f4c2401f000 comp rx=0 tx=0).stop 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.670+0000 7f4c485d0b80 1 -- 172.21.15.46:0/4135607869 shutdown_connections 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.670+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/4135607869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae317220 0x55a3ae319610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.670+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/4135607869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30ba60 0x55a3ae30be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.670+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/4135607869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae30c380 0x55a3ae316ce0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.670+0000 7f4c485d0b80 1 -- 172.21.15.46:0/4135607869 >> 172.21.15.46:0/4135607869 conn(0x55a3ae282220 msgr2=0x55a3ae308020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.670+0000 7f4c485d0b80 1 -- 172.21.15.46:0/4135607869 shutdown_connections 2024-03-18T20:43:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.670+0000 7f4c485d0b80 1 -- 172.21.15.46:0/4135607869 wait complete. 2024-03-18T20:43:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c485d0b80 1 -- start start 2024-03-18T20:43:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c485d0b80 1 Processor -- start 2024-03-18T20:43:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c485d0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 0x55a3ae316cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c485d0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30c380 0x55a3ae313fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c485d0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 0x55a3ae314700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c485d0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ae1fad80 con 0x55a3ae30ba60 2024-03-18T20:43:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c485d0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ae1faa20 con 0x55a3ae30c380 2024-03-18T20:43:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c485d0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ae0e26a0 con 0x55a3ae317220 2024-03-18T20:43:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c485d0b80 10 client.0 _ll_register_callbacks cb 0x55a3ae0663a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:06.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c3a1d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 0x55a3ae316cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c3a1d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 0x55a3ae316cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59982/0 (socket says 172.21.15.46:59982) 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c3a1d7700 1 -- 172.21.15.46:0/2707538589 learned_addr learned my addr 172.21.15.46:0/2707538589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.671+0000 7f4c3a9d8700 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 0x55a3ae314700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c399d6700 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30c380 0x55a3ae313fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c3a1d7700 1 -- 172.21.15.46:0/2707538589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 msgr2=0x55a3ae314700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c3a1d7700 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 0x55a3ae314700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c3a1d7700 1 -- 172.21.15.46:0/2707538589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30c380 msgr2=0x55a3ae313fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c3a1d7700 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30c380 0x55a3ae313fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c3a1d7700 1 -- 172.21.15.46:0/2707538589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3410e460 con 0x55a3ae30ba60 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c399d6700 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30c380 0x55a3ae313fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c3a1d7700 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 0x55a3ae316cd0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4c30099d50 tx=0x7f4c30099ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13e148cc0797ee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c1dffb700 1 -- 172.21.15.46:0/2707538589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3009de80 con 0x55a3ae30ba60 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c1dffb700 1 -- 172.21.15.46:0/2707538589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f4c300a24d0 con 0x55a3ae30ba60 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2707538589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3ae2ffad0 con 0x55a3ae30ba60 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c1dffb700 1 -- 172.21.15.46:0/2707538589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c300b8360 con 0x55a3ae30ba60 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c485d0b80 10 client.17973 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.672+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2707538589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55a3ae08cb60 con 0x55a3ae30ba60 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.673+0000 7f4c1dffb700 1 -- 172.21.15.46:0/2707538589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c30063020 con 0x55a3ae30ba60 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.673+0000 7f4c1dffb700 1 -- 172.21.15.46:0/2707538589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4c30099120 con 0x55a3ae30ba60 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.673+0000 7f4c485d0b80 10 client.17973 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.673+0000 7f4c485d0b80 10 client.17973 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.673+0000 7f4c485d0b80 20 client.17973 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.673+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55a3ae32b200 0x55a3ae32d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.673+0000 7f4c485d0b80 4 client.17973 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.673+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2707538589 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3ae257700 con 0x55a3ae32b200 2024-03-18T20:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.673+0000 7f4c399d6700 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55a3ae32b200 0x55a3ae32d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.673+0000 7f4c1dffb700 10 client.17973 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.673+0000 7f4c399d6700 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55a3ae32b200 0x55a3ae32d5f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ca586791ba97b12 server_cookie=504d63b1d064406c in_seq=0 out_seq=0 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.674+0000 7f4c1dffb700 1 -- 172.21.15.46:0/2707538589 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a3ae257700 con 0x55a3ae32b200 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.674+0000 7f4c1dffb700 10 client.17973 handle_command_reply: tid=0 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.674+0000 7f4c485d0b80 1 client.17973 shutdown 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.674+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2707538589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 msgr2=0x55a3ae316cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.674+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 0x55a3ae316cd0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4c30099d50 tx=0x7f4c30099ac0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.674+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2707538589 shutdown_connections 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.674+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55a3ae32b200 0x55a3ae32d5f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.674+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 0x55a3ae316cd0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.674+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 0x55a3ae314700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.674+0000 7f4c1dffb700 0 client.17973 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.674+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/2707538589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30c380 0x55a3ae313fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.674+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2707538589 >> 172.21.15.46:0/2707538589 conn(0x55a3ae282220 msgr2=0x55a3ae307c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.675+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2707538589 shutdown_connections 2024-03-18T20:43:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.675+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2707538589 wait complete. 2024-03-18T20:43:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.675+0000 7f4c485d0b80 20 client.17973 trim_cache size 0 max 16384 2024-03-18T20:43:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c485d0b80 1 Processor -- start 2024-03-18T20:43:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c485d0b80 1 -- start start 2024-03-18T20:43:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c485d0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30ba60 0x55a3ae322280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c485d0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30c380 0x55a3ae3168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c485d0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 0x55a3ae307b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c485d0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ae314f70 con 0x55a3ae30c380 2024-03-18T20:43:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c485d0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ae301f80 con 0x55a3ae30ba60 2024-03-18T20:43:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c485d0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ae327710 con 0x55a3ae317220 2024-03-18T20:43:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c399d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30c380 0x55a3ae3168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c399d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30c380 0x55a3ae3168e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59994/0 (socket says 172.21.15.46:59994) 2024-03-18T20:43:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c399d6700 1 -- 172.21.15.46:0/2879297604 learned_addr learned my addr 172.21.15.46:0/2879297604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c3a1d7700 1 --2- 172.21.15.46:0/2879297604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30ba60 0x55a3ae322280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c3a9d8700 1 --2- 172.21.15.46:0/2879297604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 0x55a3ae307b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c399d6700 1 -- 172.21.15.46:0/2879297604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 msgr2=0x55a3ae307b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c399d6700 1 --2- 172.21.15.46:0/2879297604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 0x55a3ae307b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c399d6700 1 -- 172.21.15.46:0/2879297604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30ba60 msgr2=0x55a3ae322280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c399d6700 1 --2- 172.21.15.46:0/2879297604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30ba60 0x55a3ae322280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.690+0000 7f4c399d6700 1 -- 172.21.15.46:0/2879297604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3ae224ea0 con 0x55a3ae30c380 2024-03-18T20:43:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.691+0000 7f4c399d6700 1 --2- 172.21.15.46:0/2879297604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30c380 0x55a3ae3168e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f4c24015000 tx=0x7f4c24009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c67349605d4205 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.691+0000 7f4c295a9700 1 -- 172.21.15.46:0/2879297604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24034040 con 0x55a3ae30c380 2024-03-18T20:43:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.691+0000 7f4c295a9700 1 -- 172.21.15.46:0/2879297604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c2402c040 con 0x55a3ae30c380 2024-03-18T20:43:06.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.691+0000 7f4c295a9700 1 -- 172.21.15.46:0/2879297604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24038730 con 0x55a3ae30c380 2024-03-18T20:43:06.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.691+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2879297604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30c380 msgr2=0x55a3ae3168e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.691+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/2879297604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30c380 0x55a3ae3168e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f4c24015000 tx=0x7f4c24009310 comp rx=0 tx=0).stop 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2879297604 shutdown_connections 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/2879297604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 0x55a3ae307b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/2879297604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30ba60 0x55a3ae322280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/2879297604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30c380 0x55a3ae3168e0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2879297604 >> 172.21.15.46:0/2879297604 conn(0x55a3ae282220 msgr2=0x55a3ae3071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2879297604 shutdown_connections 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 -- 172.21.15.46:0/2879297604 wait complete. 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 -- start start 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 Processor -- start 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 0x55a3ae321c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30c380 0x55a3ae3221a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 0x55a3ae320260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ae320840 con 0x55a3ae30ba60 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ae3209b0 con 0x55a3ae30c380 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3ae320b70 con 0x55a3ae317220 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.692+0000 7f4c485d0b80 10 client.0 _ll_register_callbacks cb 0x7f4c30099bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c3a1d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 0x55a3ae321c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c3a1d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 0x55a3ae321c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60008/0 (socket says 172.21.15.46:60008) 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c3a1d7700 1 -- 172.21.15.46:0/450443231 learned_addr learned my addr 172.21.15.46:0/450443231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c399d6700 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30c380 0x55a3ae3221a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c3a9d8700 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 0x55a3ae320260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c3a1d7700 1 -- 172.21.15.46:0/450443231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 msgr2=0x55a3ae320260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c3a1d7700 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 0x55a3ae320260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c3a1d7700 1 -- 172.21.15.46:0/450443231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30c380 msgr2=0x55a3ae3221a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c3a1d7700 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30c380 0x55a3ae3221a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c3a1d7700 1 -- 172.21.15.46:0/450443231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3ae0e03b0 con 0x55a3ae30ba60 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c3a1d7700 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 0x55a3ae321c60 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000c5b0 tx=0x7f4c300a2400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753f0503a3c9fe05 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c1dffb700 1 -- 172.21.15.46:0/450443231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3007a070 con 0x55a3ae30ba60 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c1dffb700 1 -- 172.21.15.46:0/450443231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f4c30064080 con 0x55a3ae30ba60 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.693+0000 7f4c485d0b80 1 -- 172.21.15.46:0/450443231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c2401e040 con 0x55a3ae30ba60 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c1dffb700 1 -- 172.21.15.46:0/450443231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c300a24d0 con 0x55a3ae30ba60 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c485d0b80 10 client.17979 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c485d0b80 1 -- 172.21.15.46:0/450443231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55a3ae0e4cd0 con 0x55a3ae30ba60 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c1dffb700 1 -- 172.21.15.46:0/450443231 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c2401e040 con 0x55a3ae30ba60 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c1dffb700 1 -- 172.21.15.46:0/450443231 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4c3009eda0 con 0x55a3ae30ba60 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c485d0b80 10 client.17979 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c485d0b80 10 client.17979 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c485d0b80 20 client.17979 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55a3ae37fee0 0x55a3ae382300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c485d0b80 4 client.17979 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c485d0b80 1 -- 172.21.15.46:0/450443231 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a3ae0adcf0 con 0x55a3ae37fee0 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c399d6700 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55a3ae37fee0 0x55a3ae382300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c1dffb700 10 client.17979 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.694+0000 7f4c399d6700 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55a3ae37fee0 0x55a3ae382300 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a724ba961c1b69ff server_cookie=12efb96fa52ffdce in_seq=0 out_seq=0 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c1dffb700 1 -- 172.21.15.46:0/450443231 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a3ae0adcf0 con 0x55a3ae37fee0 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c1dffb700 10 client.17979 handle_command_reply: tid=0 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c485d0b80 1 client.17979 shutdown 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c485d0b80 1 -- 172.21.15.46:0/450443231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 msgr2=0x55a3ae321c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 0x55a3ae321c60 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000c5b0 tx=0x7f4c300a2400 comp rx=0 tx=0).stop 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c485d0b80 1 -- 172.21.15.46:0/450443231 shutdown_connections 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55a3ae37fee0 0x55a3ae382300 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a3ae30ba60 0x55a3ae321c60 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a3ae317220 0x55a3ae320260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c485d0b80 1 --2- 172.21.15.46:0/450443231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a3ae30c380 0x55a3ae3221a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c485d0b80 1 -- 172.21.15.46:0/450443231 >> 172.21.15.46:0/450443231 conn(0x55a3ae282220 msgr2=0x55a3ae312760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c1dffb700 0 client.17979 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:43:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c485d0b80 1 -- 172.21.15.46:0/450443231 shutdown_connections 2024-03-18T20:43:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c485d0b80 1 -- 172.21.15.46:0/450443231 wait complete. 2024-03-18T20:43:06.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.695+0000 7f4c485d0b80 20 client.17979 trim_cache size 0 max 16384 2024-03-18T20:43:06.704 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:43:06.704 INFO:tasks.workunit.client.0.smithi046.stdout: 1 238171 3.49 MB/sec execute 241 sec latency 18.821 ms 2024-03-18T20:43:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.696+0000 7f4c3c43b700 1 -- 172.21.15.46:0/922368597 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4c200a1290 msgr2=0x7f4c200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.696+0000 7f4c3c43b700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4c200a1290 0x7f4c200a3750 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006010 tx=0x7f4c24005940 comp rx=0 tx=0).stop 2024-03-18T20:43:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.696+0000 7f4c3c43b700 1 -- 172.21.15.46:0/922368597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c34100f50 msgr2=0x7f4c34197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.696+0000 7f4c3c43b700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c34100f50 0x7f4c34197200 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000e9a0 tx=0x7f4c3000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.696+0000 7f4c3c43b700 1 -- 172.21.15.46:0/922368597 shutdown_connections 2024-03-18T20:43:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.696+0000 7f4c3c43b700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4c200a1290 0x7f4c200a3750 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.696+0000 7f4c3c43b700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c34100f50 0x7f4c34197200 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.696+0000 7f4c3c43b700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c3419bae0 0x7f4c3419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.696+0000 7f4c3c43b700 1 --2- 172.21.15.46:0/922368597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34106070 0x7f4c34197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.696+0000 7f4c3c43b700 1 -- 172.21.15.46:0/922368597 >> 172.21.15.46:0/922368597 conn(0x7f4c340fc7f0 msgr2=0x7f4c340fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.696+0000 7f4c3c43b700 1 -- 172.21.15.46:0/922368597 shutdown_connections 2024-03-18T20:43:06.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.696+0000 7f4c3c43b700 1 -- 172.21.15.46:0/922368597 wait complete. 2024-03-18T20:43:06.712 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:43:06.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:43:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.857+0000 7f150406e700 1 -- 172.21.15.46:0/3707377771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14fc100820 msgr2=0x7f14fc104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.857+0000 7f150406e700 1 --2- 172.21.15.46:0/3707377771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14fc100820 0x7f14fc104840 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009990 tx=0x7f14ec009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.857+0000 7f150406e700 1 -- 172.21.15.46:0/3707377771 shutdown_connections 2024-03-18T20:43:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.857+0000 7f150406e700 1 --2- 172.21.15.46:0/3707377771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14fc104f70 0x7f14fc10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.857+0000 7f150406e700 1 --2- 172.21.15.46:0/3707377771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14fc0ffe70 0x7f14fc100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.857+0000 7f150406e700 1 --2- 172.21.15.46:0/3707377771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14fc100820 0x7f14fc104840 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.857+0000 7f150406e700 1 -- 172.21.15.46:0/3707377771 >> 172.21.15.46:0/3707377771 conn(0x7f14fc076d50 msgr2=0x7f14fc077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.857+0000 7f150406e700 1 -- 172.21.15.46:0/3707377771 shutdown_connections 2024-03-18T20:43:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.857+0000 7f150406e700 1 -- 172.21.15.46:0/3707377771 wait complete. 2024-03-18T20:43:06.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.857+0000 7f150406e700 1 Processor -- start 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f150406e700 1 -- start start 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f150406e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14fc0ffe70 0x7f14fc198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f150406e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14fc104f70 0x7f14fc1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f150406e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14fc198ae0 0x7f14fc1a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f150406e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc1a3780 con 0x7f14fc104f70 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f150406e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc1a38f0 con 0x7f14fc0ffe70 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f150406e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc1a3ab0 con 0x7f14fc198ae0 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f1501609700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14fc104f70 0x7f14fc1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f1501e0a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14fc0ffe70 0x7f14fc198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f1501609700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14fc104f70 0x7f14fc1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60030/0 (socket says 172.21.15.46:60030) 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f1501609700 1 -- 172.21.15.46:0/3958887329 learned_addr learned my addr 172.21.15.46:0/3958887329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f1501609700 1 -- 172.21.15.46:0/3958887329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14fc198ae0 msgr2=0x7f14fc1a30e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f1501609700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14fc198ae0 0x7f14fc1a30e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f1501609700 1 -- 172.21.15.46:0/3958887329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14fc0ffe70 msgr2=0x7f14fc198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f1501609700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14fc0ffe70 0x7f14fc198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f1501609700 1 -- 172.21.15.46:0/3958887329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec0095f0 con 0x7f14fc104f70 2024-03-18T20:43:06.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f1501e0a700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14fc0ffe70 0x7f14fc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:06.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.858+0000 7f1501609700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14fc104f70 0x7f14fc1985a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f14ec005770 tx=0x7f14ec0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.859+0000 7f14f2ffd700 1 -- 172.21.15.46:0/3958887329 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec01d070 con 0x7f14fc104f70 2024-03-18T20:43:06.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.859+0000 7f14f2ffd700 1 -- 172.21.15.46:0/3958887329 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14ec021410 con 0x7f14fc104f70 2024-03-18T20:43:06.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.859+0000 7f150406e700 1 -- 172.21.15.46:0/3958887329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14fc1a3d30 con 0x7f14fc104f70 2024-03-18T20:43:06.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.859+0000 7f14f2ffd700 1 -- 172.21.15.46:0/3958887329 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec017650 con 0x7f14fc104f70 2024-03-18T20:43:06.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.859+0000 7f150406e700 1 -- 172.21.15.46:0/3958887329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14fc1a4190 con 0x7f14fc104f70 2024-03-18T20:43:06.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.862+0000 7f14f2ffd700 1 -- 172.21.15.46:0/3958887329 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f14ec00fac0 con 0x7f14fc104f70 2024-03-18T20:43:06.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.862+0000 7f1510203b80 1 Processor -- start 2024-03-18T20:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f1510203b80 1 -- start start 2024-03-18T20:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f14f2ffd700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14e80a1290 0x7f14e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f1510203b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 0x561af7b81e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f1510203b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f150260b700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14e80a1290 0x7f14e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f1510203b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b91630 0x561af7b93a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f1510203b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af7adb990 con 0x561af7b91630 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f1510203b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af7ad9350 con 0x561af7b82380 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f1510203b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af7ad4c40 con 0x561af7b81a60 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f14f2ffd700 1 -- 172.21.15.46:0/3958887329 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f14ec0cdf00 con 0x7f14fc104f70 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f1501e0a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 0x561af7b81e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f1501e0a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 0x561af7b81e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58842/0 (socket says 172.21.15.46:58842) 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f1501e0a700 1 -- 172.21.15.46:0/3420413826 learned_addr learned my addr 172.21.15.46:0/3420413826 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f150260b700 1 --2- 172.21.15.46:0/3420413826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b91630 0x561af7b93a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.863+0000 7f1501609700 1 --2- 172.21.15.46:0/3420413826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.864+0000 7f1501e0a700 1 -- 172.21.15.46:0/3420413826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 msgr2=0x561af7b910f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.864+0000 7f1501e0a700 1 --2- 172.21.15.46:0/3420413826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b910f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.864+0000 7f1501e0a700 1 -- 172.21.15.46:0/3420413826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b91630 msgr2=0x561af7b93a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.864+0000 7f1501e0a700 1 --2- 172.21.15.46:0/3420413826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b91630 0x561af7b93a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.864+0000 7f1501e0a700 1 -- 172.21.15.46:0/3420413826 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561af79563b0 con 0x561af7b81a60 2024-03-18T20:43:06.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.864+0000 7f1501609700 1 --2- 172.21.15.46:0/3420413826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b910f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.864+0000 7f1501e0a700 1 --2- 172.21.15.46:0/3420413826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 0x561af7b81e40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14f80115f0 tx=0x7f14f8011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abe19f0fa39fa67a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.864+0000 7f150260b700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14e80a1290 0x7f14e80a3750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f14f4013b30 tx=0x7f14f4009920 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.864+0000 7f14e7fff700 1 -- 172.21.15.46:0/3420413826 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8019630 con 0x561af7b81a60 2024-03-18T20:43:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.864+0000 7f14e7fff700 1 -- 172.21.15.46:0/3420413826 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f8019c50 con 0x561af7b81a60 2024-03-18T20:43:06.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.864+0000 7f14e7fff700 1 -- 172.21.15.46:0/3420413826 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800eb90 con 0x561af7b81a60 2024-03-18T20:43:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.865+0000 7f1510203b80 1 -- 172.21.15.46:0/3420413826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 msgr2=0x561af7b81e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.865+0000 7f1510203b80 1 --2- 172.21.15.46:0/3420413826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 0x561af7b81e40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14f80115f0 tx=0x7f14f8011900 comp rx=0 tx=0).stop 2024-03-18T20:43:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.865+0000 7f1510203b80 1 -- 172.21.15.46:0/3420413826 shutdown_connections 2024-03-18T20:43:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.865+0000 7f1510203b80 1 --2- 172.21.15.46:0/3420413826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b91630 0x561af7b93a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.865+0000 7f1510203b80 1 --2- 172.21.15.46:0/3420413826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 0x561af7b81e40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.865+0000 7f1510203b80 1 --2- 172.21.15.46:0/3420413826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b910f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.865+0000 7f1510203b80 1 -- 172.21.15.46:0/3420413826 >> 172.21.15.46:0/3420413826 conn(0x561af7af8220 msgr2=0x561af7b7e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.865+0000 7f1510203b80 1 -- 172.21.15.46:0/3420413826 shutdown_connections 2024-03-18T20:43:06.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.865+0000 7f1510203b80 1 -- 172.21.15.46:0/3420413826 wait complete. 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.865+0000 7f1510203b80 1 -- start start 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.865+0000 7f1510203b80 1 Processor -- start 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1510203b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b81a60 0x561af7b910d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1510203b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b8e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1510203b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b91630 0x561af7b8eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1510203b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af7a70d80 con 0x561af7b81a60 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1510203b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af7a70a20 con 0x561af7b82380 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1510203b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af79586a0 con 0x561af7b91630 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1510203b80 10 client.0 _ll_register_callbacks cb 0x561af78dc3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1501e0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b81a60 0x561af7b910d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f150260b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b91630 0x561af7b8eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1501e0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b81a60 0x561af7b910d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60052/0 (socket says 172.21.15.46:60052) 2024-03-18T20:43:06.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1501e0a700 1 -- 172.21.15.46:0/1601677202 learned_addr learned my addr 172.21.15.46:0/1601677202 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1501609700 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b8e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1501e0a700 1 -- 172.21.15.46:0/1601677202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b91630 msgr2=0x561af7b8eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1501e0a700 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b91630 0x561af7b8eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1501e0a700 1 -- 172.21.15.46:0/1601677202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 msgr2=0x561af7b8e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1501e0a700 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b8e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1501e0a700 1 -- 172.21.15.46:0/1601677202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f8009dd0 con 0x561af7b81a60 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.866+0000 7f1501609700 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b8e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.867+0000 7f1501e0a700 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b81a60 0x561af7b910d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f14f8000c00 tx=0x7f14f800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ff27968f06ddec server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.867+0000 7f14e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.867+0000 7f14e57fa700 1 -- 172.21.15.46:0/1601677202 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800ec20 con 0x561af7b81a60 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.867+0000 7f1510203b80 1 -- 172.21.15.46:0/1601677202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561af7b75ad0 con 0x561af7b81a60 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.867+0000 7f14e57fa700 1 -- 172.21.15.46:0/1601677202 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f14f8005ce0 con 0x561af7b81a60 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.867+0000 7f14e57fa700 1 -- 172.21.15.46:0/1601677202 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14f800ed80 con 0x561af7b81a60 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.867+0000 7f1510203b80 10 client.17991 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:06.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.867+0000 7f1510203b80 1 -- 172.21.15.46:0/1601677202 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x561af7902b60 con 0x561af7b81a60 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.867+0000 7f14e57fa700 1 -- 172.21.15.46:0/1601677202 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f8037080 con 0x561af7b81a60 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.867+0000 7f14e57fa700 1 -- 172.21.15.46:0/1601677202 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f14f8031030 con 0x561af7b81a60 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.868+0000 7f1510203b80 10 client.17991 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.868+0000 7f1510203b80 10 client.17991 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.868+0000 7f1510203b80 20 client.17991 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.868+0000 7f1510203b80 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561af7ba1110 0x561af7ba3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.868+0000 7f1510203b80 4 client.17991 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.868+0000 7f1510203b80 1 -- 172.21.15.46:0/1601677202 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561af7acd700 con 0x561af7ba1110 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.868+0000 7f1501609700 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561af7ba1110 0x561af7ba3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.868+0000 7f14e57fa700 10 client.17991 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.868+0000 7f1501609700 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561af7ba1110 0x561af7ba3500 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a99413254a9c32d4 server_cookie=2124694e3669ffd7 in_seq=0 out_seq=0 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.869+0000 7f14e57fa700 1 -- 172.21.15.46:0/1601677202 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561af7acd700 con 0x561af7ba1110 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.869+0000 7f14e57fa700 10 client.17991 handle_command_reply: tid=0 2024-03-18T20:43:06.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.869+0000 7f1510203b80 1 client.17991 shutdown 2024-03-18T20:43:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.869+0000 7f1510203b80 1 -- 172.21.15.46:0/1601677202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b81a60 msgr2=0x561af7b910d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.869+0000 7f1510203b80 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b81a60 0x561af7b910d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f14f8000c00 tx=0x7f14f800e530 comp rx=0 tx=0).stop 2024-03-18T20:43:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.869+0000 7f1510203b80 1 -- 172.21.15.46:0/1601677202 shutdown_connections 2024-03-18T20:43:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.869+0000 7f1510203b80 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561af7ba1110 0x561af7ba3500 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.869+0000 7f1510203b80 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b81a60 0x561af7b910d0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.869+0000 7f1510203b80 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b91630 0x561af7b8eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.869+0000 7f1510203b80 1 --2- 172.21.15.46:0/1601677202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b8e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.869+0000 7f14e57fa700 0 client.17991 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:43:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.869+0000 7f1510203b80 1 -- 172.21.15.46:0/1601677202 >> 172.21.15.46:0/1601677202 conn(0x561af7af8220 msgr2=0x561af7b7dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.870+0000 7f1510203b80 1 -- 172.21.15.46:0/1601677202 shutdown_connections 2024-03-18T20:43:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.870+0000 7f1510203b80 1 -- 172.21.15.46:0/1601677202 wait complete. 2024-03-18T20:43:06.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.870+0000 7f1510203b80 20 client.17991 trim_cache size 0 max 16384 2024-03-18T20:43:06.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1510203b80 1 Processor -- start 2024-03-18T20:43:06.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1510203b80 1 -- start start 2024-03-18T20:43:06.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1510203b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 0x561af7b8eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1510203b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b98260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1510203b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b91630 0x561af7b9c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1510203b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af7b7a5e0 con 0x561af7b91630 2024-03-18T20:43:06.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1510203b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af7b83300 con 0x561af7b82380 2024-03-18T20:43:06.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1510203b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af7b987a0 con 0x561af7b81a60 2024-03-18T20:43:06.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1501e0a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 0x561af7b8eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1501609700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b98260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1501609700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b98260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51140/0 (socket says 172.21.15.46:51140) 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1501e0a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 0x561af7b8eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:58860/0 (socket says 172.21.15.46:58860) 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.887+0000 7f1501609700 1 -- 172.21.15.46:0/1532435956 learned_addr learned my addr 172.21.15.46:0/1532435956 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.888+0000 7f150260b700 1 --2- 172.21.15.46:0/1532435956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b91630 0x561af7b9c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.888+0000 7f1501609700 1 -- 172.21.15.46:0/1532435956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 msgr2=0x561af7b8eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.888+0000 7f1501609700 1 --2- 172.21.15.46:0/1532435956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 0x561af7b8eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.888+0000 7f1501609700 1 -- 172.21.15.46:0/1532435956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b91630 msgr2=0x561af7b9c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.888+0000 7f1501609700 1 --2- 172.21.15.46:0/1532435956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b91630 0x561af7b9c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.888+0000 7f1501609700 1 -- 172.21.15.46:0/1532435956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561af7a9aea0 con 0x561af7b82380 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.888+0000 7f150260b700 1 --2- 172.21.15.46:0/1532435956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b91630 0x561af7b9c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.888+0000 7f1501609700 1 --2- 172.21.15.46:0/1532435956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b98260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00f740 tx=0x7f14ec0a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8175afe0a530108 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.888+0000 7f14f0ff9700 1 -- 172.21.15.46:0/1532435956 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec09b750 con 0x561af7b82380 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.888+0000 7f14f0ff9700 1 -- 172.21.15.46:0/1532435956 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14ec0aee90 con 0x561af7b82380 2024-03-18T20:43:06.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.888+0000 7f14f0ff9700 1 -- 172.21.15.46:0/1532435956 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0a1120 con 0x561af7b82380 2024-03-18T20:43:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.889+0000 7f1510203b80 1 -- 172.21.15.46:0/1532435956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 msgr2=0x561af7b98260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.889+0000 7f1510203b80 1 --2- 172.21.15.46:0/1532435956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b98260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00f740 tx=0x7f14ec0a1040 comp rx=0 tx=0).stop 2024-03-18T20:43:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.889+0000 7f1510203b80 1 -- 172.21.15.46:0/1532435956 shutdown_connections 2024-03-18T20:43:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.889+0000 7f1510203b80 1 --2- 172.21.15.46:0/1532435956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b91630 0x561af7b9c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.889+0000 7f1510203b80 1 --2- 172.21.15.46:0/1532435956 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b81a60 0x561af7b8eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.889+0000 7f1510203b80 1 --2- 172.21.15.46:0/1532435956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b98260 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.889+0000 7f1510203b80 1 -- 172.21.15.46:0/1532435956 >> 172.21.15.46:0/1532435956 conn(0x561af7af8220 msgr2=0x561af7b79a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.889+0000 7f1510203b80 1 -- 172.21.15.46:0/1532435956 shutdown_connections 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.889+0000 7f1510203b80 1 -- 172.21.15.46:0/1532435956 wait complete. 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.889+0000 7f1510203b80 1 -- start start 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.889+0000 7f1510203b80 1 Processor -- start 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1510203b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b81a60 0x561af7b95770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1510203b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b95cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1510203b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b91630 0x561af7b961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1510203b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af7bf3f30 con 0x561af7b81a60 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1510203b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af7bf4070 con 0x561af7b82380 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1510203b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af7bf41b0 con 0x561af7b91630 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1510203b80 10 client.0 _ll_register_callbacks cb 0x7f14f4013680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1501609700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b95cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1501e0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b81a60 0x561af7b95770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1501609700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b95cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51146/0 (socket says 172.21.15.46:51146) 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f150260b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b91630 0x561af7b961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1501609700 1 -- 172.21.15.46:0/1547434937 learned_addr learned my addr 172.21.15.46:0/1547434937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1501609700 1 -- 172.21.15.46:0/1547434937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b91630 msgr2=0x561af7b961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1501609700 1 --2- 172.21.15.46:0/1547434937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b91630 0x561af7b961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1501609700 1 -- 172.21.15.46:0/1547434937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b81a60 msgr2=0x561af7b95770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1501609700 1 --2- 172.21.15.46:0/1547434937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b81a60 0x561af7b95770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.890+0000 7f1501609700 1 -- 172.21.15.46:0/1547434937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14fc1a4190 con 0x561af7b82380 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f1501609700 1 --2- 172.21.15.46:0/1547434937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b95cb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f14ec097ac0 tx=0x7f14ec005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b498e9d9f3e0ed74 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f14e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f14e57fa700 1 -- 172.21.15.46:0/1547434937 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec07b030 con 0x561af7b82380 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f1510203b80 1 -- 172.21.15.46:0/1547434937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14f8019c50 con 0x561af7b82380 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f14e57fa700 1 -- 172.21.15.46:0/1547434937 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f14ec07c070 con 0x561af7b82380 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f14e57fa700 1 -- 172.21.15.46:0/1547434937 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14ec09ee80 con 0x561af7b82380 2024-03-18T20:43:06.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f1510203b80 10 client.27674 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f1510203b80 1 -- 172.21.15.46:0/1547434937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x561af795acd0 con 0x561af7b82380 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f14e57fa700 1 -- 172.21.15.46:0/1547434937 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f8019c50 con 0x561af7b82380 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f14e57fa700 1 -- 172.21.15.46:0/1547434937 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f14ec09a540 con 0x561af7b82380 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f1510203b80 10 client.27674 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f1510203b80 10 client.27674 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f1510203b80 20 client.27674 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f1510203b80 1 --2- 172.21.15.46:0/1547434937 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561af7bf54c0 0x561af7bf78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f1510203b80 4 client.27674 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.891+0000 7f1510203b80 1 -- 172.21.15.46:0/1547434937 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561af7923cf0 con 0x561af7bf54c0 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.892+0000 7f1501e0a700 1 --2- 172.21.15.46:0/1547434937 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561af7bf54c0 0x561af7bf78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.892+0000 7f1501e0a700 1 --2- 172.21.15.46:0/1547434937 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561af7bf54c0 0x561af7bf78b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35fbb71be976fcc4 server_cookie=cf2235bb6165dfa8 in_seq=0 out_seq=0 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.892+0000 7f14e57fa700 10 client.27674 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:43:06.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f14e57fa700 1 -- 172.21.15.46:0/1547434937 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561af7923cf0 con 0x561af7bf54c0 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f14e57fa700 10 client.27674 handle_command_reply: tid=0 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f1510203b80 1 client.27674 shutdown 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f1510203b80 1 -- 172.21.15.46:0/1547434937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 msgr2=0x561af7b95cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f1510203b80 1 --2- 172.21.15.46:0/1547434937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b95cb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f14ec097ac0 tx=0x7f14ec005c00 comp rx=0 tx=0).stop 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f1510203b80 1 -- 172.21.15.46:0/1547434937 shutdown_connections 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f1510203b80 1 --2- 172.21.15.46:0/1547434937 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561af7bf54c0 0x561af7bf78b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f1510203b80 1 --2- 172.21.15.46:0/1547434937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561af7b81a60 0x561af7b95770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f14e57fa700 0 client.27674 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f1510203b80 1 --2- 172.21.15.46:0/1547434937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561af7b91630 0x561af7b961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f1510203b80 1 --2- 172.21.15.46:0/1547434937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561af7b82380 0x561af7b95cb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f1510203b80 1 -- 172.21.15.46:0/1547434937 >> 172.21.15.46:0/1547434937 conn(0x561af7af8220 msgr2=0x561af7b9c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f1510203b80 1 -- 172.21.15.46:0/1547434937 shutdown_connections 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f1510203b80 1 -- 172.21.15.46:0/1547434937 wait complete. 2024-03-18T20:43:06.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.893+0000 7f1510203b80 20 client.27674 trim_cache size 0 max 16384 2024-03-18T20:43:06.902 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:43:06.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.894+0000 7f150406e700 1 -- 172.21.15.46:0/3958887329 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14e80a1290 msgr2=0x7f14e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.894+0000 7f150406e700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14e80a1290 0x7f14e80a3750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f14f4013b30 tx=0x7f14f4009920 comp rx=0 tx=0).stop 2024-03-18T20:43:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.894+0000 7f150406e700 1 -- 172.21.15.46:0/3958887329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14fc104f70 msgr2=0x7f14fc1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.894+0000 7f150406e700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14fc104f70 0x7f14fc1985a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f14ec005770 tx=0x7f14ec0057a0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.894+0000 7f150406e700 1 -- 172.21.15.46:0/3958887329 shutdown_connections 2024-03-18T20:43:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.894+0000 7f150406e700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14e80a1290 0x7f14e80a3750 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.894+0000 7f150406e700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14fc198ae0 0x7f14fc1a30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.894+0000 7f150406e700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14fc0ffe70 0x7f14fc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.894+0000 7f150406e700 1 --2- 172.21.15.46:0/3958887329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14fc104f70 0x7f14fc1985a0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.894+0000 7f150406e700 1 -- 172.21.15.46:0/3958887329 >> 172.21.15.46:0/3958887329 conn(0x7f14fc076d50 msgr2=0x7f14fc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.894+0000 7f150406e700 1 -- 172.21.15.46:0/3958887329 shutdown_connections 2024-03-18T20:43:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:06.894+0000 7f150406e700 1 -- 172.21.15.46:0/3958887329 wait complete. 2024-03-18T20:43:06.910 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:43:06.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:43:07.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:06 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2462472480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:43:07.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.057+0000 7f86e8cdc700 1 -- 172.21.15.46:0/2832196735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4067e30 msgr2=0x7f86e40682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.057+0000 7f86e8cdc700 1 --2- 172.21.15.46:0/2832196735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4067e30 0x7f86e40682b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009990 tx=0x7f86d8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.057+0000 7f86e8cdc700 1 -- 172.21.15.46:0/2832196735 shutdown_connections 2024-03-18T20:43:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.057+0000 7f86e8cdc700 1 --2- 172.21.15.46:0/2832196735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e40687f0 0x7f86e410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.057+0000 7f86e8cdc700 1 --2- 172.21.15.46:0/2832196735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e4105ce0 0x7f86e41060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.057+0000 7f86e8cdc700 1 --2- 172.21.15.46:0/2832196735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4067e30 0x7f86e40682b0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.057+0000 7f86e8cdc700 1 -- 172.21.15.46:0/2832196735 >> 172.21.15.46:0/2832196735 conn(0x7f86e4076d40 msgr2=0x7f86e4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.057+0000 7f86e8cdc700 1 -- 172.21.15.46:0/2832196735 shutdown_connections 2024-03-18T20:43:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.057+0000 7f86e8cdc700 1 -- 172.21.15.46:0/2832196735 wait complete. 2024-03-18T20:43:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.058+0000 7f86e8cdc700 1 Processor -- start 2024-03-18T20:43:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.058+0000 7f86e8cdc700 1 -- start start 2024-03-18T20:43:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e8cdc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4067e30 0x7f86e4197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e8cdc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e40687f0 0x7f86e4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e8cdc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e4105ce0 0x7f86e419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e8cdc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e419c130 con 0x7f86e4067e30 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e8cdc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e419c2a0 con 0x7f86e40687f0 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e8cdc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e419c460 con 0x7f86e4105ce0 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4067e30 0x7f86e4197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4067e30 0x7f86e4197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60102/0 (socket says 172.21.15.46:60102) 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e259c700 1 -- 172.21.15.46:0/4108872638 learned_addr learned my addr 172.21.15.46:0/4108872638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e40687f0 0x7f86e4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e2d9d700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e4105ce0 0x7f86e419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e259c700 1 -- 172.21.15.46:0/4108872638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e4105ce0 msgr2=0x7f86e419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e259c700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e4105ce0 0x7f86e419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e259c700 1 -- 172.21.15.46:0/4108872638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e40687f0 msgr2=0x7f86e4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e259c700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e40687f0 0x7f86e4197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e259c700 1 -- 172.21.15.46:0/4108872638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80095f0 con 0x7f86e4067e30 2024-03-18T20:43:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e40687f0 0x7f86e4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e259c700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4067e30 0x7f86e4197210 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f86d400e9a0 tx=0x7f86d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86d37fe700 1 -- 172.21.15.46:0/4108872638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4009790 con 0x7f86e4067e30 2024-03-18T20:43:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86d37fe700 1 -- 172.21.15.46:0/4108872638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86d400cbd0 con 0x7f86e4067e30 2024-03-18T20:43:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86e8cdc700 1 -- 172.21.15.46:0/4108872638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e419c740 con 0x7f86e4067e30 2024-03-18T20:43:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.059+0000 7f86d37fe700 1 -- 172.21.15.46:0/4108872638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4018680 con 0x7f86e4067e30 2024-03-18T20:43:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.060+0000 7f86e8cdc700 1 -- 172.21.15.46:0/4108872638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e41a3f80 con 0x7f86e4067e30 2024-03-18T20:43:07.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.062+0000 7f86d37fe700 1 -- 172.21.15.46:0/4108872638 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f86d400cd40 con 0x7f86e4067e30 2024-03-18T20:43:07.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.063+0000 7f86d37fe700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f86cc0a1290 0x7f86cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.063+0000 7f86d37fe700 1 -- 172.21.15.46:0/4108872638 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f86d4014070 con 0x7f86e4067e30 2024-03-18T20:43:07.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.063+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f86cc0a1290 0x7f86cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.064+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f86cc0a1290 0x7f86cc0a3750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86d80098e0 tx=0x7f86d8005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:07.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.064+0000 7f86f4e71b80 1 Processor -- start 2024-03-18T20:43:07.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.064+0000 7f86f4e71b80 1 -- start start 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86f4e71b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec118a60 0x55cdec118e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86f4e71b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec123ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86f4e71b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec126610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86f4e71b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec072990 con 0x55cdec119380 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86f4e71b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec070350 con 0x55cdec124220 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86f4e71b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec06bc40 con 0x55cdec118a60 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec123ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec126610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec123ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60114/0 (socket says 172.21.15.46:60114) 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec126610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51168/0 (socket says 172.21.15.46:51168) 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e1d9b700 1 -- 172.21.15.46:0/1008365986 learned_addr learned my addr 172.21.15.46:0/1008365986 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e259c700 1 --2- 172.21.15.46:0/1008365986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec118a60 0x55cdec118e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e259c700 1 -- 172.21.15.46:0/1008365986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 msgr2=0x55cdec126610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e259c700 1 --2- 172.21.15.46:0/1008365986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec126610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e259c700 1 -- 172.21.15.46:0/1008365986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 msgr2=0x55cdec123ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e259c700 1 --2- 172.21.15.46:0/1008365986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec123ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e259c700 1 -- 172.21.15.46:0/1008365986 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdebeed3b0 con 0x55cdec118a60 2024-03-18T20:43:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e2d9d700 1 --2- 172.21.15.46:0/1008365986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec126610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/1008365986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec123ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.065+0000 7f86e259c700 1 --2- 172.21.15.46:0/1008365986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec118a60 0x55cdec118e40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f86d40ca690 tx=0x7f86d40999b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9074145b2827b38 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.066+0000 7f86d0ff9700 1 -- 172.21.15.46:0/1008365986 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d409de80 con 0x55cdec118a60 2024-03-18T20:43:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.066+0000 7f86d0ff9700 1 -- 172.21.15.46:0/1008365986 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86d40a24d0 con 0x55cdec118a60 2024-03-18T20:43:07.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.066+0000 7f86d0ff9700 1 -- 172.21.15.46:0/1008365986 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40a2040 con 0x55cdec118a60 2024-03-18T20:43:07.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.066+0000 7f86f4e71b80 1 -- 172.21.15.46:0/1008365986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec118a60 msgr2=0x55cdec118e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.066+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/1008365986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec118a60 0x55cdec118e40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f86d40ca690 tx=0x7f86d40999b0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.066+0000 7f86f4e71b80 1 -- 172.21.15.46:0/1008365986 shutdown_connections 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.066+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/1008365986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec126610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.066+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/1008365986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec118a60 0x55cdec118e40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.066+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/1008365986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec123ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.066+0000 7f86f4e71b80 1 -- 172.21.15.46:0/1008365986 >> 172.21.15.46:0/1008365986 conn(0x55cdec08f220 msgr2=0x55cdec115020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.066+0000 7f86f4e71b80 1 -- 172.21.15.46:0/1008365986 shutdown_connections 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.066+0000 7f86f4e71b80 1 -- 172.21.15.46:0/1008365986 wait complete. 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86f4e71b80 1 -- start start 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86f4e71b80 1 Processor -- start 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86f4e71b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec118a60 0x55cdec1210d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86f4e71b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec121610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86f4e71b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec124220 0x55cdec121b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86f4e71b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec007d80 con 0x55cdec119380 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86f4e71b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec007a20 con 0x55cdec118a60 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86f4e71b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdebeef6a0 con 0x55cdec124220 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86f4e71b80 10 client.0 _ll_register_callbacks cb 0x55cdebe733a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec121610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86e259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec118a60 0x55cdec1210d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86e2d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec124220 0x55cdec121b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec121610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60128/0 (socket says 172.21.15.46:60128) 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86e1d9b700 1 -- 172.21.15.46:0/2118357432 learned_addr learned my addr 172.21.15.46:0/2118357432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.067+0000 7f86e259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec118a60 0x55cdec1210d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51180/0 (socket says 172.21.15.46:51180) 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86e1d9b700 1 -- 172.21.15.46:0/2118357432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec124220 msgr2=0x55cdec121b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec124220 0x55cdec121b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86e1d9b700 1 -- 172.21.15.46:0/2118357432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec118a60 msgr2=0x55cdec1210d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec118a60 0x55cdec1210d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86e1d9b700 1 -- 172.21.15.46:0/2118357432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc009050 con 0x55cdec119380 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86e2d9d700 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec124220 0x55cdec121b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86e259c700 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec118a60 0x55cdec1210d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec121610 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f86d800b2e0 tx=0x7f86d8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8cb994f7a8aae6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86c9ffb700 1 -- 172.21.15.46:0/2118357432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d802d040 con 0x55cdec119380 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86c9ffb700 1 -- 172.21.15.46:0/2118357432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f86d8004e40 con 0x55cdec119380 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2118357432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdec10cad0 con 0x55cdec119380 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86c9ffb700 1 -- 172.21.15.46:0/2118357432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86d800f870 con 0x55cdec119380 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86f4e71b80 10 client.18009 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.068+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2118357432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f86e41a3f80 con 0x55cdec119380 2024-03-18T20:43:07.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.069+0000 7f86c9ffb700 1 -- 172.21.15.46:0/2118357432 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86d803d020 con 0x55cdec119380 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.069+0000 7f86c9ffb700 1 -- 172.21.15.46:0/2118357432 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f86d80329f0 con 0x55cdec119380 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.069+0000 7f86f4e71b80 10 client.18009 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.069+0000 7f86f4e71b80 10 client.18009 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.069+0000 7f86f4e71b80 20 client.18009 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.069+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55cdec133f00 0x55cdec136320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.069+0000 7f86f4e71b80 4 client.18009 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.069+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2118357432 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdec064700 con 0x55cdec133f00 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.069+0000 7f86e259c700 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55cdec133f00 0x55cdec136320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.069+0000 7f86c9ffb700 10 client.18009 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.069+0000 7f86e259c700 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55cdec133f00 0x55cdec136320 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c38384aad0eaf1d2 server_cookie=caf871a84dfd333f in_seq=0 out_seq=0 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86c9ffb700 1 -- 172.21.15.46:0/2118357432 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cdec064700 con 0x55cdec133f00 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86c9ffb700 10 client.18009 handle_command_reply: tid=0 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86f4e71b80 1 client.18009 shutdown 2024-03-18T20:43:07.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2118357432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 msgr2=0x55cdec121610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec121610 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f86d800b2e0 tx=0x7f86d8020000 comp rx=0 tx=0).stop 2024-03-18T20:43:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2118357432 shutdown_connections 2024-03-18T20:43:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55cdec133f00 0x55cdec136320 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec124220 0x55cdec121b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec118a60 0x55cdec1210d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/2118357432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec121610 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2118357432 >> 172.21.15.46:0/2118357432 conn(0x55cdec08f220 msgr2=0x55cdec114c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86c9ffb700 0 client.18009 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:43:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2118357432 shutdown_connections 2024-03-18T20:43:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.070+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2118357432 wait complete. 2024-03-18T20:43:07.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.071+0000 7f86f4e71b80 20 client.18009 trim_cache size 0 max 16384 2024-03-18T20:43:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86f4e71b80 1 Processor -- start 2024-03-18T20:43:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86f4e71b80 1 -- start start 2024-03-18T20:43:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86f4e71b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec118a60 0x55cdec11bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86f4e71b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec119380 0x55cdec121940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86f4e71b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec11f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86f4e71b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec11a440 con 0x55cdec118a60 2024-03-18T20:43:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86f4e71b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec122630 con 0x55cdec124220 2024-03-18T20:43:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86f4e71b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec119d30 con 0x55cdec119380 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86e259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec118a60 0x55cdec11bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86e259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec118a60 0x55cdec11bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60136/0 (socket says 172.21.15.46:60136) 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86e259c700 1 -- 172.21.15.46:0/1281024161 learned_addr learned my addr 172.21.15.46:0/1281024161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86e2d9d700 1 --2- 172.21.15.46:0/1281024161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec11f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/1281024161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec119380 0x55cdec121940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86e2d9d700 1 -- 172.21.15.46:0/1281024161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec119380 msgr2=0x55cdec121940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86e2d9d700 1 --2- 172.21.15.46:0/1281024161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec119380 0x55cdec121940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86e2d9d700 1 -- 172.21.15.46:0/1281024161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec118a60 msgr2=0x55cdec11bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86e2d9d700 1 --2- 172.21.15.46:0/1281024161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec118a60 0x55cdec11bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86e2d9d700 1 -- 172.21.15.46:0/1281024161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdebe99b60 con 0x55cdec124220 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86e259c700 1 --2- 172.21.15.46:0/1281024161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec118a60 0x55cdec11bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.085+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/1281024161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec119380 0x55cdec121940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.086+0000 7f86e2d9d700 1 --2- 172.21.15.46:0/1281024161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec11f9a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f86dc007a40 tx=0x7f86dc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7671da2e7f909eae server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.086+0000 7f86d17fa700 1 -- 172.21.15.46:0/1281024161 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc00e040 con 0x55cdec124220 2024-03-18T20:43:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.086+0000 7f86d17fa700 1 -- 172.21.15.46:0/1281024161 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86dc00f040 con 0x55cdec124220 2024-03-18T20:43:07.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.086+0000 7f86d17fa700 1 -- 172.21.15.46:0/1281024161 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc020cf0 con 0x55cdec124220 2024-03-18T20:43:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 -- 172.21.15.46:0/1281024161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 msgr2=0x55cdec11f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/1281024161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec11f9a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f86dc007a40 tx=0x7f86dc009920 comp rx=0 tx=0).stop 2024-03-18T20:43:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 -- 172.21.15.46:0/1281024161 shutdown_connections 2024-03-18T20:43:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/1281024161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec118a60 0x55cdec11bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/1281024161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec11f9a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/1281024161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec119380 0x55cdec121940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 -- 172.21.15.46:0/1281024161 >> 172.21.15.46:0/1281024161 conn(0x55cdec08f220 msgr2=0x55cdec11c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 -- 172.21.15.46:0/1281024161 shutdown_connections 2024-03-18T20:43:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 -- 172.21.15.46:0/1281024161 wait complete. 2024-03-18T20:43:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 -- start start 2024-03-18T20:43:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 Processor -- start 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec118a60 0x55cdec128e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec129370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec1298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec18b270 con 0x55cdec119380 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec18b3e0 con 0x55cdec124220 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdec18b5a0 con 0x55cdec118a60 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86f4e71b80 10 client.0 _ll_register_callbacks cb 0x7f86d8005610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec129370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec129370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60152/0 (socket says 172.21.15.46:60152) 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86e1d9b700 1 -- 172.21.15.46:0/2960081126 learned_addr learned my addr 172.21.15.46:0/2960081126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.087+0000 7f86e2d9d700 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec1298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86e259c700 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec118a60 0x55cdec128e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86e1d9b700 1 -- 172.21.15.46:0/2960081126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec118a60 msgr2=0x55cdec128e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec118a60 0x55cdec128e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86e1d9b700 1 -- 172.21.15.46:0/2960081126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 msgr2=0x55cdec1298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec1298b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86e1d9b700 1 -- 172.21.15.46:0/2960081126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d40b1040 con 0x55cdec119380 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86e1d9b700 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec129370 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f86d8005f50 tx=0x7f86d8004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9b34d9a6eca93d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86c9ffb700 1 -- 172.21.15.46:0/2960081126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d802d040 con 0x55cdec119380 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2960081126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86d40a24d0 con 0x55cdec119380 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86c9ffb700 1 -- 172.21.15.46:0/2960081126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f86d803e080 con 0x55cdec119380 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86c9ffb700 1 -- 172.21.15.46:0/2960081126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86d8024550 con 0x55cdec119380 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86f4e71b80 10 client.18015 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.088+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2960081126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f86e41a3f80 con 0x55cdec119380 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.089+0000 7f86c9ffb700 1 -- 172.21.15.46:0/2960081126 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86d804d020 con 0x55cdec119380 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.089+0000 7f86c9ffb700 1 -- 172.21.15.46:0/2960081126 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f86d800f670 con 0x55cdec119380 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.089+0000 7f86f4e71b80 10 client.18015 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.089+0000 7f86f4e71b80 10 client.18015 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.089+0000 7f86f4e71b80 20 client.18015 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.089+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55cdec18c850 0x55cdec18ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.089+0000 7f86f4e71b80 4 client.18015 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.089+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2960081126 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cdebebacf0 con 0x55cdec18c850 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.089+0000 7f86e259c700 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55cdec18c850 0x55cdec18ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.089+0000 7f86c9ffb700 10 client.18015 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.089+0000 7f86e259c700 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55cdec18c850 0x55cdec18ec40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f66ed8a774ec25d5 server_cookie=75012e2cecfa2045 in_seq=0 out_seq=0 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86c9ffb700 1 -- 172.21.15.46:0/2960081126 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cdebebacf0 con 0x55cdec18c850 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86c9ffb700 10 client.18015 handle_command_reply: tid=0 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86f4e71b80 1 client.18015 shutdown 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2960081126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 msgr2=0x55cdec129370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec129370 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f86d8005f50 tx=0x7f86d8004f70 comp rx=0 tx=0).stop 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2960081126 shutdown_connections 2024-03-18T20:43:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55cdec18c850 0x55cdec18ec40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdec124220 0x55cdec1298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86c9ffb700 0 client.18015 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:43:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cdec118a60 0x55cdec128e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86f4e71b80 1 --2- 172.21.15.46:0/2960081126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cdec119380 0x55cdec129370 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2960081126 >> 172.21.15.46:0/2960081126 conn(0x55cdec08f220 msgr2=0x55cdec11f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2960081126 shutdown_connections 2024-03-18T20:43:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86f4e71b80 1 -- 172.21.15.46:0/2960081126 wait complete. 2024-03-18T20:43:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86f4e71b80 20 client.18015 trim_cache size 0 max 16384 2024-03-18T20:43:07.099 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:43:07.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86e8cdc700 1 -- 172.21.15.46:0/4108872638 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f86cc0a1290 msgr2=0x7f86cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86e8cdc700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f86cc0a1290 0x7f86cc0a3750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86d80098e0 tx=0x7f86d8005960 comp rx=0 tx=0).stop 2024-03-18T20:43:07.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86e8cdc700 1 -- 172.21.15.46:0/4108872638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4067e30 msgr2=0x7f86e4197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.090+0000 7f86e8cdc700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4067e30 0x7f86e4197210 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f86d400e9a0 tx=0x7f86d400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.091+0000 7f86e8cdc700 1 -- 172.21.15.46:0/4108872638 shutdown_connections 2024-03-18T20:43:07.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.091+0000 7f86e8cdc700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f86cc0a1290 0x7f86cc0a3750 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.091+0000 7f86e8cdc700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86e4067e30 0x7f86e4197210 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.091+0000 7f86e8cdc700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e4105ce0 0x7f86e419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.091+0000 7f86e8cdc700 1 --2- 172.21.15.46:0/4108872638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86e40687f0 0x7f86e4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.091+0000 7f86e8cdc700 1 -- 172.21.15.46:0/4108872638 >> 172.21.15.46:0/4108872638 conn(0x7f86e4076d40 msgr2=0x7f86e40fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:07.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.091+0000 7f86e8cdc700 1 -- 172.21.15.46:0/4108872638 shutdown_connections 2024-03-18T20:43:07.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.091+0000 7f86e8cdc700 1 -- 172.21.15.46:0/4108872638 wait complete. 2024-03-18T20:43:07.107 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:43:07.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:43:07.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:06 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2462472480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:43:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.249+0000 7fb08a2d9700 1 -- 172.21.15.46:0/1347594190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb084104f60 msgr2=0x7fb08410cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.249+0000 7fb08a2d9700 1 --2- 172.21.15.46:0/1347594190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb084104f60 0x7fb08410cae0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00b0a0 tx=0x7fb07c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.249+0000 7fb08a2d9700 1 -- 172.21.15.46:0/1347594190 shutdown_connections 2024-03-18T20:43:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.249+0000 7fb08a2d9700 1 --2- 172.21.15.46:0/1347594190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb084104f60 0x7fb08410cae0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.249+0000 7fb08a2d9700 1 --2- 172.21.15.46:0/1347594190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0840ffe50 0x7fb084100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.249+0000 7fb08a2d9700 1 --2- 172.21.15.46:0/1347594190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb084100800 0x7fb084104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.249+0000 7fb08a2d9700 1 -- 172.21.15.46:0/1347594190 >> 172.21.15.46:0/1347594190 conn(0x7fb084076d40 msgr2=0x7fb084077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.249+0000 7fb08a2d9700 1 -- 172.21.15.46:0/1347594190 shutdown_connections 2024-03-18T20:43:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.249+0000 7fb08a2d9700 1 -- 172.21.15.46:0/1347594190 wait complete. 2024-03-18T20:43:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.250+0000 7fb08a2d9700 1 Processor -- start 2024-03-18T20:43:07.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.250+0000 7fb08a2d9700 1 -- start start 2024-03-18T20:43:07.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb08a2d9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0840ffe50 0x7fb08410e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb08a2d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb084104f60 0x7fb08410eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb08a2d9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb08410f060 0x7fb0841a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb08a2d9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0841a5a70 con 0x7fb084104f60 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb08a2d9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0841a5bb0 con 0x7fb0840ffe50 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb08a2d9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0841a5d70 con 0x7fb08410f060 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb083fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0840ffe50 0x7fb08410e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb0837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb084104f60 0x7fb08410eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb083fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0840ffe50 0x7fb08410e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51206/0 (socket says 172.21.15.46:51206) 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb083fff700 1 -- 172.21.15.46:0/2575162761 learned_addr learned my addr 172.21.15.46:0/2575162761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb088876700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb08410f060 0x7fb0841a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb0837fe700 1 -- 172.21.15.46:0/2575162761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb08410f060 msgr2=0x7fb0841a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb0837fe700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb08410f060 0x7fb0841a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb0837fe700 1 -- 172.21.15.46:0/2575162761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0840ffe50 msgr2=0x7fb08410e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb0837fe700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0840ffe50 0x7fb08410e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb0837fe700 1 -- 172.21.15.46:0/2575162761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078009520 con 0x7fb084104f60 2024-03-18T20:43:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb083fff700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0840ffe50 0x7fb08410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb088876700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb08410f060 0x7fb0841a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.251+0000 7fb0837fe700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb084104f60 0x7fb08410eb20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb07800e9c0 tx=0x7fb07800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.252+0000 7fb0817fa700 1 -- 172.21.15.46:0/2575162761 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07800cb70 con 0x7fb084104f60 2024-03-18T20:43:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.252+0000 7fb0817fa700 1 -- 172.21.15.46:0/2575162761 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb07800ee60 con 0x7fb084104f60 2024-03-18T20:43:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.252+0000 7fb08a2d9700 1 -- 172.21.15.46:0/2575162761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c009d00 con 0x7fb084104f60 2024-03-18T20:43:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.252+0000 7fb0817fa700 1 -- 172.21.15.46:0/2575162761 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb078003c60 con 0x7fb084104f60 2024-03-18T20:43:07.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.252+0000 7fb08a2d9700 1 -- 172.21.15.46:0/2575162761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0841a6340 con 0x7fb084104f60 2024-03-18T20:43:07.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.254+0000 7fb0817fa700 1 -- 172.21.15.46:0/2575162761 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fb078003e00 con 0x7fb084104f60 2024-03-18T20:43:07.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.255+0000 7fb0817fa700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb06c0a1290 0x7fb06c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.255+0000 7fb083fff700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb06c0a1290 0x7fb06c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.255+0000 7fb0817fa700 1 -- 172.21.15.46:0/2575162761 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fb078014070 con 0x7fb084104f60 2024-03-18T20:43:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb083fff700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb06c0a1290 0x7fb06c0a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb074011550 tx=0x7fb074009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb09646eb80 1 Processor -- start 2024-03-18T20:43:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb09646eb80 1 -- start start 2024-03-18T20:43:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb09646eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb09646eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a1380 0x55dbbd0abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb09646eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0ac220 0x55dbbd0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb09646eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbcffa990 con 0x55dbbd0a0a60 2024-03-18T20:43:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb09646eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbcff8350 con 0x55dbbd0ac220 2024-03-18T20:43:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb09646eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbcff3c40 con 0x55dbbd0a1380 2024-03-18T20:43:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb083fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb083fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60188/0 (socket says 172.21.15.46:60188) 2024-03-18T20:43:07.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb083fff700 1 -- 172.21.15.46:0/1883521674 learned_addr learned my addr 172.21.15.46:0/1883521674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb0837fe700 1 --2- 172.21.15.46:0/1883521674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a1380 0x55dbbd0abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.256+0000 7fb088876700 1 --2- 172.21.15.46:0/1883521674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0ac220 0x55dbbd0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.257+0000 7fb083fff700 1 -- 172.21.15.46:0/1883521674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a1380 msgr2=0x55dbbd0abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.257+0000 7fb083fff700 1 --2- 172.21.15.46:0/1883521674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a1380 0x55dbbd0abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.257+0000 7fb083fff700 1 -- 172.21.15.46:0/1883521674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0ac220 msgr2=0x55dbbd0ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.257+0000 7fb083fff700 1 --2- 172.21.15.46:0/1883521674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0ac220 0x55dbbd0ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.257+0000 7fb083fff700 1 -- 172.21.15.46:0/1883521674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbbce753b0 con 0x55dbbd0a0a60 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.257+0000 7fb088876700 1 --2- 172.21.15.46:0/1883521674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0ac220 0x55dbbd0ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.257+0000 7fb0837fe700 1 --2- 172.21.15.46:0/1883521674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a1380 0x55dbbd0abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.257+0000 7fb083fff700 1 --2- 172.21.15.46:0/1883521674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0a0e40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb074009880 tx=0x7fb074014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9891ee52c74d849 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.257+0000 7fb06a7fc700 1 -- 172.21.15.46:0/1883521674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074026630 con 0x55dbbd0a0a60 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.257+0000 7fb06a7fc700 1 -- 172.21.15.46:0/1883521674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb074021070 con 0x55dbbd0a0a60 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.257+0000 7fb06a7fc700 1 -- 172.21.15.46:0/1883521674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07401d670 con 0x55dbbd0a0a60 2024-03-18T20:43:07.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.258+0000 7fb09646eb80 1 -- 172.21.15.46:0/1883521674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 msgr2=0x55dbbd0a0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.258+0000 7fb09646eb80 1 --2- 172.21.15.46:0/1883521674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0a0e40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb074009880 tx=0x7fb074014000 comp rx=0 tx=0).stop 2024-03-18T20:43:07.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.258+0000 7fb09646eb80 1 -- 172.21.15.46:0/1883521674 shutdown_connections 2024-03-18T20:43:07.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.258+0000 7fb09646eb80 1 --2- 172.21.15.46:0/1883521674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0a0e40 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.258+0000 7fb09646eb80 1 --2- 172.21.15.46:0/1883521674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0ac220 0x55dbbd0ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.258+0000 7fb09646eb80 1 --2- 172.21.15.46:0/1883521674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a1380 0x55dbbd0abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.258+0000 7fb09646eb80 1 -- 172.21.15.46:0/1883521674 >> 172.21.15.46:0/1883521674 conn(0x55dbbd017220 msgr2=0x55dbbd09d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:07.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.258+0000 7fb09646eb80 1 -- 172.21.15.46:0/1883521674 shutdown_connections 2024-03-18T20:43:07.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.258+0000 7fb09646eb80 1 -- 172.21.15.46:0/1883521674 wait complete. 2024-03-18T20:43:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.258+0000 7fb09646eb80 1 -- start start 2024-03-18T20:43:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb09646eb80 1 Processor -- start 2024-03-18T20:43:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb09646eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb09646eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 0x55dbbd0a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb09646eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0ac220 0x55dbbd0a9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb09646eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbcf8fd80 con 0x55dbbd0ac220 2024-03-18T20:43:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb09646eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbcf8fa20 con 0x55dbbd0a1380 2024-03-18T20:43:07.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb09646eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbce776a0 con 0x55dbbd0a0a60 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb09646eb80 10 client.0 _ll_register_callbacks cb 0x55dbbcdfb3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb0837fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 0x55dbbd0a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb088876700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0ac220 0x55dbbd0a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb0837fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 0x55dbbd0a8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51232/0 (socket says 172.21.15.46:51232) 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb0837fe700 1 -- 172.21.15.46:0/21514590 learned_addr learned my addr 172.21.15.46:0/21514590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb083fff700 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb0837fe700 1 -- 172.21.15.46:0/21514590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a0a60 msgr2=0x55dbbd0abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb0837fe700 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb0837fe700 1 -- 172.21.15.46:0/21514590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0ac220 msgr2=0x55dbbd0a9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb0837fe700 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0ac220 0x55dbbd0a9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb0837fe700 1 -- 172.21.15.46:0/21514590 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08410d350 con 0x55dbbd0a1380 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb083fff700 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.259+0000 7fb088876700 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0ac220 0x55dbbd0a9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb0837fe700 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 0x55dbbd0a8ee0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb078099d00 tx=0x7fb07809a700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e754b87938f30ae server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb0637fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb0637fe700 1 -- 172.21.15.46:0/21514590 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07809de30 con 0x55dbbd0a1380 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb0637fe700 1 -- 172.21.15.46:0/21514590 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fb0780a2480 con 0x55dbbd0a1380 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb09646eb80 1 -- 172.21.15.46:0/21514590 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbbd094ad0 con 0x55dbbd0a1380 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb0637fe700 1 -- 172.21.15.46:0/21514590 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb07807b050 con 0x55dbbd0a1380 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb09646eb80 10 client.27704 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb09646eb80 1 -- 172.21.15.46:0/21514590 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fb074017070 con 0x55dbbd0a1380 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb0637fe700 1 -- 172.21.15.46:0/21514590 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dbbd094ad0 con 0x55dbbd0a1380 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb0637fe700 1 -- 172.21.15.46:0/21514590 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb0780a1ff0 con 0x55dbbd0a1380 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb09646eb80 10 client.27704 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb09646eb80 10 client.27704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.260+0000 7fb09646eb80 20 client.27704 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.261+0000 7fb09646eb80 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dbbd0bbee0 0x55dbbd0be300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.261+0000 7fb09646eb80 4 client.27704 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.261+0000 7fb09646eb80 1 -- 172.21.15.46:0/21514590 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbbcfec700 con 0x55dbbd0bbee0 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.261+0000 7fb083fff700 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dbbd0bbee0 0x55dbbd0be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.261+0000 7fb0637fe700 10 client.27704 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.261+0000 7fb083fff700 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dbbd0bbee0 0x55dbbd0be300 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1207a2fe9e9dad36 server_cookie=450c2ef3318d38aa in_seq=0 out_seq=0 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb0637fe700 1 -- 172.21.15.46:0/21514590 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dbbcfec700 con 0x55dbbd0bbee0 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb0637fe700 10 client.27704 handle_command_reply: tid=0 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb09646eb80 1 client.27704 shutdown 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb09646eb80 1 -- 172.21.15.46:0/21514590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 msgr2=0x55dbbd0a8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb09646eb80 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 0x55dbbd0a8ee0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb078099d00 tx=0x7fb07809a700 comp rx=0 tx=0).stop 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb09646eb80 1 -- 172.21.15.46:0/21514590 shutdown_connections 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb09646eb80 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dbbd0bbee0 0x55dbbd0be300 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb09646eb80 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0ac220 0x55dbbd0a9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb0637fe700 0 client.27704 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:07.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb09646eb80 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb09646eb80 1 --2- 172.21.15.46:0/21514590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 0x55dbbd0a8ee0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb09646eb80 1 -- 172.21.15.46:0/21514590 >> 172.21.15.46:0/21514590 conn(0x55dbbd017220 msgr2=0x55dbbd09cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:07.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb09646eb80 1 -- 172.21.15.46:0/21514590 shutdown_connections 2024-03-18T20:43:07.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb09646eb80 1 -- 172.21.15.46:0/21514590 wait complete. 2024-03-18T20:43:07.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.262+0000 7fb09646eb80 20 client.27704 trim_cache size 0 max 16384 2024-03-18T20:43:07.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.276+0000 7fb09646eb80 1 Processor -- start 2024-03-18T20:43:07.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.276+0000 7fb09646eb80 1 -- start start 2024-03-18T20:43:07.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.276+0000 7fb09646eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.276+0000 7fb09646eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a1380 0x55dbbd0b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.276+0000 7fb09646eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0ac220 0x55dbbd09cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.276+0000 7fb09646eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbd0a9e40 con 0x55dbbd0a0a60 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.276+0000 7fb09646eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbd0b8900 con 0x55dbbd0ac220 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.276+0000 7fb09646eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbd0a1d60 con 0x55dbbd0a1380 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb083fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb083fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60198/0 (socket says 172.21.15.46:60198) 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb083fff700 1 -- 172.21.15.46:0/3763210402 learned_addr learned my addr 172.21.15.46:0/3763210402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb0837fe700 1 --2- 172.21.15.46:0/3763210402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a1380 0x55dbbd0b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb088876700 1 --2- 172.21.15.46:0/3763210402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0ac220 0x55dbbd09cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb083fff700 1 -- 172.21.15.46:0/3763210402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a1380 msgr2=0x55dbbd0b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb083fff700 1 --2- 172.21.15.46:0/3763210402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a1380 0x55dbbd0b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb083fff700 1 -- 172.21.15.46:0/3763210402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0ac220 msgr2=0x55dbbd09cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb083fff700 1 --2- 172.21.15.46:0/3763210402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0ac220 0x55dbbd09cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb083fff700 1 -- 172.21.15.46:0/3763210402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbbce21b60 con 0x55dbbd0a0a60 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb0837fe700 1 --2- 172.21.15.46:0/3763210402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a1380 0x55dbbd0b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb083fff700 1 --2- 172.21.15.46:0/3763210402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0ae610 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fb074009880 tx=0x7fb07401cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b12009f699ba42 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:07.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb06affd700 1 -- 172.21.15.46:0/3763210402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074009e90 con 0x55dbbd0a0a60 2024-03-18T20:43:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb06affd700 1 -- 172.21.15.46:0/3763210402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb07401cc40 con 0x55dbbd0a0a60 2024-03-18T20:43:07.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.277+0000 7fb06affd700 1 -- 172.21.15.46:0/3763210402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07401c700 con 0x55dbbd0a0a60 2024-03-18T20:43:07.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.278+0000 7fb09646eb80 1 -- 172.21.15.46:0/3763210402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 msgr2=0x55dbbd0ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.278+0000 7fb09646eb80 1 --2- 172.21.15.46:0/3763210402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0ae610 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fb074009880 tx=0x7fb07401cbd0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.278+0000 7fb09646eb80 1 -- 172.21.15.46:0/3763210402 shutdown_connections 2024-03-18T20:43:07.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.278+0000 7fb09646eb80 1 --2- 172.21.15.46:0/3763210402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0ae610 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.278+0000 7fb09646eb80 1 --2- 172.21.15.46:0/3763210402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0ac220 0x55dbbd09cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.278+0000 7fb09646eb80 1 --2- 172.21.15.46:0/3763210402 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0a1380 0x55dbbd0b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.278+0000 7fb09646eb80 1 -- 172.21.15.46:0/3763210402 >> 172.21.15.46:0/3763210402 conn(0x55dbbd017220 msgr2=0x55dbbd0ab820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.278+0000 7fb09646eb80 1 -- 172.21.15.46:0/3763210402 shutdown_connections 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.278+0000 7fb09646eb80 1 -- 172.21.15.46:0/3763210402 wait complete. 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.279+0000 7fb09646eb80 1 -- start start 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.279+0000 7fb09646eb80 1 Processor -- start 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.279+0000 7fb09646eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.279+0000 7fb09646eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 0x55dbbd0b1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.279+0000 7fb09646eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0ac220 0x55dbbd0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.279+0000 7fb09646eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbd0b3440 con 0x55dbbd0a0a60 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.279+0000 7fb09646eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbd10ed00 con 0x55dbbd0a1380 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.279+0000 7fb09646eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbd10eec0 con 0x55dbbd0ac220 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.279+0000 7fb09646eb80 10 client.0 _ll_register_callbacks cb 0x7fb078099bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:07.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.279+0000 7fb0837fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 0x55dbbd0b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb083fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb0837fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 0x55dbbd0b1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51248/0 (socket says 172.21.15.46:51248) 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb0837fe700 1 -- 172.21.15.46:0/480277531 learned_addr learned my addr 172.21.15.46:0/480277531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb088876700 1 --2- 172.21.15.46:0/480277531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0ac220 0x55dbbd0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb0837fe700 1 -- 172.21.15.46:0/480277531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0ac220 msgr2=0x55dbbd0b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb0837fe700 1 --2- 172.21.15.46:0/480277531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0ac220 0x55dbbd0b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb0837fe700 1 -- 172.21.15.46:0/480277531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 msgr2=0x55dbbd0b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb0837fe700 1 --2- 172.21.15.46:0/480277531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb0837fe700 1 -- 172.21.15.46:0/480277531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb074017070 con 0x55dbbd0a1380 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb0837fe700 1 --2- 172.21.15.46:0/480277531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 0x55dbbd0b1370 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb07800c5b0 tx=0x7fb0780caba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8509645443105049 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb0637fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb0637fe700 1 -- 172.21.15.46:0/480277531 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07807b070 con 0x55dbbd0a1380 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb0637fe700 1 -- 172.21.15.46:0/480277531 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fb0780a2480 con 0x55dbbd0a1380 2024-03-18T20:43:07.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb09646eb80 1 -- 172.21.15.46:0/480277531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb074021070 con 0x55dbbd0a1380 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb0637fe700 1 -- 172.21.15.46:0/480277531 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb07809a770 con 0x55dbbd0a1380 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb09646eb80 10 client.27710 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.280+0000 7fb09646eb80 1 -- 172.21.15.46:0/480277531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fb07402a040 con 0x55dbbd0a1380 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.281+0000 7fb0637fe700 1 -- 172.21.15.46:0/480277531 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb074021070 con 0x55dbbd0a1380 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.281+0000 7fb0637fe700 1 -- 172.21.15.46:0/480277531 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb07807e060 con 0x55dbbd0a1380 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.281+0000 7fb09646eb80 10 client.27710 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.281+0000 7fb09646eb80 10 client.27710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.281+0000 7fb09646eb80 20 client.27710 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.281+0000 7fb09646eb80 1 --2- 172.21.15.46:0/480277531 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dbbd1103a0 0x55dbbd1127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.281+0000 7fb09646eb80 4 client.27710 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.281+0000 7fb09646eb80 1 -- 172.21.15.46:0/480277531 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dbbce42cf0 con 0x55dbbd1103a0 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.281+0000 7fb083fff700 1 --2- 172.21.15.46:0/480277531 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dbbd1103a0 0x55dbbd1127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.281+0000 7fb0637fe700 10 client.27710 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:07.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.281+0000 7fb083fff700 1 --2- 172.21.15.46:0/480277531 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dbbd1103a0 0x55dbbd1127c0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=419e52997c09b6c7 server_cookie=9c2fa1890570385a in_seq=0 out_seq=0 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb0637fe700 1 -- 172.21.15.46:0/480277531 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dbbce42cf0 con 0x55dbbd1103a0 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb0637fe700 10 client.27710 handle_command_reply: tid=0 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb09646eb80 1 client.27710 shutdown 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb09646eb80 1 -- 172.21.15.46:0/480277531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 msgr2=0x55dbbd0b1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb09646eb80 1 --2- 172.21.15.46:0/480277531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 0x55dbbd0b1370 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb07800c5b0 tx=0x7fb0780caba0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb09646eb80 1 -- 172.21.15.46:0/480277531 shutdown_connections 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb09646eb80 1 --2- 172.21.15.46:0/480277531 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dbbd1103a0 0x55dbbd1127c0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb09646eb80 1 --2- 172.21.15.46:0/480277531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbd0a0a60 0x55dbbd0b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb0637fe700 0 client.27710 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb09646eb80 1 --2- 172.21.15.46:0/480277531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbd0ac220 0x55dbbd0b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb09646eb80 1 --2- 172.21.15.46:0/480277531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbd0a1380 0x55dbbd0b1370 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb09646eb80 1 -- 172.21.15.46:0/480277531 >> 172.21.15.46:0/480277531 conn(0x55dbbd017220 msgr2=0x55dbbd09cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb09646eb80 1 -- 172.21.15.46:0/480277531 shutdown_connections 2024-03-18T20:43:07.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb09646eb80 1 -- 172.21.15.46:0/480277531 wait complete. 2024-03-18T20:43:07.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.282+0000 7fb09646eb80 20 client.27710 trim_cache size 0 max 16384 2024-03-18T20:43:07.291 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:43:07.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.283+0000 7fb08a2d9700 1 -- 172.21.15.46:0/2575162761 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb06c0a1290 msgr2=0x7fb06c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.283+0000 7fb08a2d9700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb06c0a1290 0x7fb06c0a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb074011550 tx=0x7fb074009260 comp rx=0 tx=0).stop 2024-03-18T20:43:07.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.283+0000 7fb08a2d9700 1 -- 172.21.15.46:0/2575162761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb084104f60 msgr2=0x7fb08410eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:07.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.283+0000 7fb08a2d9700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb084104f60 0x7fb08410eb20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb07800e9c0 tx=0x7fb07800ecd0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.283+0000 7fb08a2d9700 1 -- 172.21.15.46:0/2575162761 shutdown_connections 2024-03-18T20:43:07.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.283+0000 7fb08a2d9700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb06c0a1290 0x7fb06c0a3750 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.283+0000 7fb08a2d9700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb08410f060 0x7fb0841a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.283+0000 7fb08a2d9700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0840ffe50 0x7fb08410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.283+0000 7fb08a2d9700 1 --2- 172.21.15.46:0/2575162761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb084104f60 0x7fb08410eb20 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:07.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.283+0000 7fb08a2d9700 1 -- 172.21.15.46:0/2575162761 >> 172.21.15.46:0/2575162761 conn(0x7fb084076d40 msgr2=0x7fb0840fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:07.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.283+0000 7fb08a2d9700 1 -- 172.21.15.46:0/2575162761 shutdown_connections 2024-03-18T20:43:07.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:07.283+0000 7fb08a2d9700 1 -- 172.21.15.46:0/2575162761 wait complete. 2024-03-18T20:43:07.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:06 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2462472480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:43:07.698 INFO:tasks.workunit.client.0.smithi046.stdout: 1 239388 3.50 MB/sec execute 242 sec latency 297.704 ms 2024-03-18T20:43:08.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:07 smithi073 ceph-mon[47384]: pgmap v215: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-03-18T20:43:08.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:07 smithi155 ceph-mon[48344]: pgmap v215: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-03-18T20:43:08.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:07 smithi046 ceph-mon[43495]: pgmap v215: 97 pgs: 97 active+clean; 91 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-03-18T20:43:08.301 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:43:08.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:43:08.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:43:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.449+0000 7fc47752f700 1 -- 172.21.15.46:0/1689540069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc470100820 msgr2=0x7fc470104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.449+0000 7fc47752f700 1 --2- 172.21.15.46:0/1689540069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc470100820 0x7fc470104860 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc460009990 tx=0x7fc460009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.449+0000 7fc47752f700 1 -- 172.21.15.46:0/1689540069 shutdown_connections 2024-03-18T20:43:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.450+0000 7fc47752f700 1 --2- 172.21.15.46:0/1689540069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc470104f90 0x7fc47010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.450+0000 7fc47752f700 1 --2- 172.21.15.46:0/1689540069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4700ffe70 0x7fc470100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.450+0000 7fc47752f700 1 --2- 172.21.15.46:0/1689540069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc470100820 0x7fc470104860 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.450+0000 7fc47752f700 1 -- 172.21.15.46:0/1689540069 >> 172.21.15.46:0/1689540069 conn(0x7fc470076d80 msgr2=0x7fc470077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.450+0000 7fc47752f700 1 -- 172.21.15.46:0/1689540069 shutdown_connections 2024-03-18T20:43:08.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.450+0000 7fc47752f700 1 -- 172.21.15.46:0/1689540069 wait complete. 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.450+0000 7fc47752f700 1 Processor -- start 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.450+0000 7fc47752f700 1 -- start start 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.450+0000 7fc47752f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4700ffe70 0x7fc470197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.450+0000 7fc47752f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc470104f90 0x7fc470197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.450+0000 7fc47752f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc47019bab0 0x7fc47019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc47752f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47019c530 con 0x7fc4700ffe70 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc47752f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47019c6a0 con 0x7fc470104f90 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc47752f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47019c860 con 0x7fc47019bab0 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc4752cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4700ffe70 0x7fc470197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc4752cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4700ffe70 0x7fc470197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60228/0 (socket says 172.21.15.46:60228) 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc4752cb700 1 -- 172.21.15.46:0/1475220391 learned_addr learned my addr 172.21.15.46:0/1475220391 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc475acc700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc47019bab0 0x7fc47019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc474aca700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc470104f90 0x7fc470197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc4752cb700 1 -- 172.21.15.46:0/1475220391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc47019bab0 msgr2=0x7fc47019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc4752cb700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc47019bab0 0x7fc47019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc4752cb700 1 -- 172.21.15.46:0/1475220391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc470104f90 msgr2=0x7fc470197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc4752cb700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc470104f90 0x7fc470197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc4752cb700 1 -- 172.21.15.46:0/1475220391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4600095f0 con 0x7fc4700ffe70 2024-03-18T20:43:08.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc474aca700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc470104f90 0x7fc470197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:08.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc4752cb700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4700ffe70 0x7fc470197220 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00e9a0 tx=0x7fc46c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:08.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc4667fc700 1 -- 172.21.15.46:0/1475220391 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c009790 con 0x7fc4700ffe70 2024-03-18T20:43:08.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc47752f700 1 -- 172.21.15.46:0/1475220391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4701a3dc0 con 0x7fc4700ffe70 2024-03-18T20:43:08.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc4667fc700 1 -- 172.21.15.46:0/1475220391 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc46c00cbd0 con 0x7fc4700ffe70 2024-03-18T20:43:08.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc4667fc700 1 -- 172.21.15.46:0/1475220391 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c018680 con 0x7fc4700ffe70 2024-03-18T20:43:08.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.451+0000 7fc47752f700 1 -- 172.21.15.46:0/1475220391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4701a4250 con 0x7fc4700ffe70 2024-03-18T20:43:08.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.454+0000 7fc4667fc700 1 -- 172.21.15.46:0/1475220391 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc46c00cd40 con 0x7fc4700ffe70 2024-03-18T20:43:08.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.454+0000 7fc4667fc700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc45c0a1290 0x7fc45c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.455+0000 7fc474aca700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc45c0a1290 0x7fc45c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.455+0000 7fc4667fc700 1 -- 172.21.15.46:0/1475220391 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fc46c014070 con 0x7fc4700ffe70 2024-03-18T20:43:08.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.455+0000 7fc474aca700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc45c0a1290 0x7fc45c0a3750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc460005b40 tx=0x7fc460005ab0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:08.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.455+0000 7fc4836c4b80 1 Processor -- start 2024-03-18T20:43:08.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc4836c4b80 1 -- start start 2024-03-18T20:43:08.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc4836c4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863cdfae0 0x55c863cdfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc4836c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863ce0400 0x55c863cef170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc4836c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cef6b0 0x55c863cf1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc4836c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c863c39a10 con 0x55c863ce0400 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc4836c4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c863c373d0 con 0x55c863cdfae0 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc4836c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c863c32cc0 con 0x55c863cef6b0 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc474aca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863ce0400 0x55c863cef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc474aca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863ce0400 0x55c863cef170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60232/0 (socket says 172.21.15.46:60232) 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc474aca700 1 -- 172.21.15.46:0/726175089 learned_addr learned my addr 172.21.15.46:0/726175089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc4752cb700 1 --2- 172.21.15.46:0/726175089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863cdfae0 0x55c863cdfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc475acc700 1 --2- 172.21.15.46:0/726175089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cef6b0 0x55c863cf1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc474aca700 1 -- 172.21.15.46:0/726175089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cef6b0 msgr2=0x55c863cf1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc474aca700 1 --2- 172.21.15.46:0/726175089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cef6b0 0x55c863cf1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc474aca700 1 -- 172.21.15.46:0/726175089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863cdfae0 msgr2=0x55c863cdfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc474aca700 1 --2- 172.21.15.46:0/726175089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863cdfae0 0x55c863cdfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc474aca700 1 -- 172.21.15.46:0/726175089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c863ab4430 con 0x55c863ce0400 2024-03-18T20:43:08.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.456+0000 7fc474aca700 1 --2- 172.21.15.46:0/726175089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863ce0400 0x55c863cef170 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc460005110 tx=0x7fc46001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e04913103b8522 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:08.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.457+0000 7fc45b7fe700 1 -- 172.21.15.46:0/726175089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46002b070 con 0x55c863ce0400 2024-03-18T20:43:08.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.457+0000 7fc45b7fe700 1 -- 172.21.15.46:0/726175089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc46001e040 con 0x55c863ce0400 2024-03-18T20:43:08.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.457+0000 7fc45b7fe700 1 -- 172.21.15.46:0/726175089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460004db0 con 0x55c863ce0400 2024-03-18T20:43:08.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.457+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726175089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863ce0400 msgr2=0x55c863cef170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.457+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/726175089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863ce0400 0x55c863cef170 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc460005110 tx=0x7fc46001f000 comp rx=0 tx=0).stop 2024-03-18T20:43:08.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.458+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726175089 shutdown_connections 2024-03-18T20:43:08.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.458+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/726175089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cef6b0 0x55c863cf1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.458+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/726175089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863cdfae0 0x55c863cdfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.458+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/726175089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863ce0400 0x55c863cef170 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.458+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726175089 >> 172.21.15.46:0/726175089 conn(0x55c863c562a0 msgr2=0x55c863cdc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.458+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726175089 shutdown_connections 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.458+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726175089 wait complete. 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.458+0000 7fc4836c4b80 1 -- start start 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.458+0000 7fc4836c4b80 1 Processor -- start 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4836c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cdfae0 0x55c863cef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4836c4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863ce0400 0x55c863cec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4836c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cef6b0 0x55c863ceca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4836c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c863bcee00 con 0x55c863cdfae0 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4836c4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c863bceaa0 con 0x55c863ce0400 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4836c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c863ab6720 con 0x55c863cef6b0 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4836c4b80 10 client.0 _ll_register_callbacks cb 0x55c863a3a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4752cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cdfae0 0x55c863cef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4752cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cdfae0 0x55c863cef080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60242/0 (socket says 172.21.15.46:60242) 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4752cb700 1 -- 172.21.15.46:0/726379071 learned_addr learned my addr 172.21.15.46:0/726379071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc475acc700 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cef6b0 0x55c863ceca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc474aca700 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863ce0400 0x55c863cec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4752cb700 1 -- 172.21.15.46:0/726379071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cef6b0 msgr2=0x55c863ceca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4752cb700 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cef6b0 0x55c863ceca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4752cb700 1 -- 172.21.15.46:0/726379071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863ce0400 msgr2=0x55c863cec410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4752cb700 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863ce0400 0x55c863cec410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4752cb700 1 -- 172.21.15.46:0/726379071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47010d380 con 0x55c863cdfae0 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4752cb700 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cdfae0 0x55c863cef080 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc46c099d50 tx=0x7fc46c09a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477c47a742987b32 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc458ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc458ff9700 1 -- 172.21.15.46:0/726379071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c09de80 con 0x55c863cdfae0 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc458ff9700 1 -- 172.21.15.46:0/726379071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fc46c0a24d0 con 0x55c863cdfae0 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.459+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726379071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c863cd3b50 con 0x55c863cdfae0 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.460+0000 7fc458ff9700 1 -- 172.21.15.46:0/726379071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc46c0b81a0 con 0x55c863cdfae0 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.460+0000 7fc4836c4b80 10 client.18045 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.460+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726379071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c863a60be0 con 0x55c863cdfae0 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.460+0000 7fc458ff9700 1 -- 172.21.15.46:0/726379071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c863cd3b50 con 0x55c863cdfae0 2024-03-18T20:43:08.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.460+0000 7fc458ff9700 1 -- 172.21.15.46:0/726379071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc46c098fb0 con 0x55c863cdfae0 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.460+0000 7fc4836c4b80 10 client.18045 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.460+0000 7fc4836c4b80 10 client.18045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.460+0000 7fc4836c4b80 20 client.18045 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.460+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c863cff110 0x55c863d01530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.460+0000 7fc4836c4b80 4 client.18045 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.460+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726379071 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c863c2b780 con 0x55c863cff110 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.460+0000 7fc474aca700 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c863cff110 0x55c863d01530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.461+0000 7fc458ff9700 10 client.18045 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.461+0000 7fc474aca700 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c863cff110 0x55c863d01530 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db64b5907408decd server_cookie=edc5f0cb8573835b in_seq=0 out_seq=0 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.461+0000 7fc458ff9700 1 -- 172.21.15.46:0/726379071 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c863c2b780 con 0x55c863cff110 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.461+0000 7fc458ff9700 10 client.18045 handle_command_reply: tid=0 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.461+0000 7fc4836c4b80 1 client.18045 shutdown 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.462+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726379071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cdfae0 msgr2=0x55c863cef080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.462+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cdfae0 0x55c863cef080 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc46c099d50 tx=0x7fc46c09a7d0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.462+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726379071 shutdown_connections 2024-03-18T20:43:08.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.462+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c863cff110 0x55c863d01530 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.462+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cdfae0 0x55c863cef080 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.462+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cef6b0 0x55c863ceca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.462+0000 7fc458ff9700 0 client.18045 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:08.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.462+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/726379071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863ce0400 0x55c863cec410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.462+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726379071 >> 172.21.15.46:0/726379071 conn(0x55c863c562a0 msgr2=0x55c863cdbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:08.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.462+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726379071 shutdown_connections 2024-03-18T20:43:08.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.462+0000 7fc4836c4b80 1 -- 172.21.15.46:0/726379071 wait complete. 2024-03-18T20:43:08.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.462+0000 7fc4836c4b80 20 client.18045 trim_cache size 0 max 16384 2024-03-18T20:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc4836c4b80 1 Processor -- start 2024-03-18T20:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc4836c4b80 1 -- start start 2024-03-18T20:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc4836c4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863cdfae0 0x55c863cfb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc4836c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863ce0400 0x55c863ce9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc4836c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 0x55c863ce9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc4836c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c863ce0240 con 0x55c863cef6b0 2024-03-18T20:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc4836c4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c863cd6000 con 0x55c863cdfae0 2024-03-18T20:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc4836c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c863cfc860 con 0x55c863ce0400 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc475acc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 0x55c863ce9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc475acc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 0x55c863ce9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60250/0 (socket says 172.21.15.46:60250) 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc475acc700 1 -- 172.21.15.46:0/859443164 learned_addr learned my addr 172.21.15.46:0/859443164 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc474aca700 1 --2- 172.21.15.46:0/859443164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863ce0400 0x55c863ce9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.480+0000 7fc4752cb700 1 --2- 172.21.15.46:0/859443164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863cdfae0 0x55c863cfb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.481+0000 7fc475acc700 1 -- 172.21.15.46:0/859443164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863ce0400 msgr2=0x55c863ce9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.481+0000 7fc475acc700 1 --2- 172.21.15.46:0/859443164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863ce0400 0x55c863ce9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.481+0000 7fc475acc700 1 -- 172.21.15.46:0/859443164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863cdfae0 msgr2=0x55c863cfb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.481+0000 7fc475acc700 1 --2- 172.21.15.46:0/859443164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863cdfae0 0x55c863cfb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.481+0000 7fc475acc700 1 -- 172.21.15.46:0/859443164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c863bf8f20 con 0x55c863cef6b0 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.481+0000 7fc474aca700 1 --2- 172.21.15.46:0/859443164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863ce0400 0x55c863ce9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.481+0000 7fc4752cb700 1 --2- 172.21.15.46:0/859443164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863cdfae0 0x55c863cfb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.481+0000 7fc475acc700 1 --2- 172.21.15.46:0/859443164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 0x55c863ce9bb0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc468009ec0 tx=0x7fc468009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb62f27e4cb3a178 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.481+0000 7fc45bfff700 1 -- 172.21.15.46:0/859443164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468007dd0 con 0x55c863cef6b0 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.481+0000 7fc45bfff700 1 -- 172.21.15.46:0/859443164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc46800f040 con 0x55c863cef6b0 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.481+0000 7fc45bfff700 1 -- 172.21.15.46:0/859443164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46801f7a0 con 0x55c863cef6b0 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 -- 172.21.15.46:0/859443164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 msgr2=0x55c863ce9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/859443164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 0x55c863ce9bb0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc468009ec0 tx=0x7fc468009ef0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 -- 172.21.15.46:0/859443164 shutdown_connections 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/859443164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 0x55c863ce9bb0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/859443164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863cdfae0 0x55c863cfb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/859443164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863ce0400 0x55c863ce9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 -- 172.21.15.46:0/859443164 >> 172.21.15.46:0/859443164 conn(0x55c863c562a0 msgr2=0x55c863cfa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 -- 172.21.15.46:0/859443164 shutdown_connections 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 -- 172.21.15.46:0/859443164 wait complete. 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 -- start start 2024-03-18T20:43:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 Processor -- start 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cdfae0 0x55c863d4c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863ce0400 0x55c863d4c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 0x55c863cf53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c863cf5980 con 0x55c863cef6b0 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c863cf5af0 con 0x55c863ce0400 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.482+0000 7fc4836c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c863cf5cb0 con 0x55c863cdfae0 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc4836c4b80 10 client.0 _ll_register_callbacks cb 0x7fc46c099c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc475acc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 0x55c863cf53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc475acc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 0x55c863cf53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60252/0 (socket says 172.21.15.46:60252) 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc475acc700 1 -- 172.21.15.46:0/615223068 learned_addr learned my addr 172.21.15.46:0/615223068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc474aca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863ce0400 0x55c863d4c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc4752cb700 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cdfae0 0x55c863d4c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc475acc700 1 -- 172.21.15.46:0/615223068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cdfae0 msgr2=0x55c863d4c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc475acc700 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cdfae0 0x55c863d4c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc475acc700 1 -- 172.21.15.46:0/615223068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863ce0400 msgr2=0x55c863d4c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc475acc700 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863ce0400 0x55c863d4c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc475acc700 1 -- 172.21.15.46:0/615223068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc468009b70 con 0x55c863cef6b0 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc474aca700 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863ce0400 0x55c863d4c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc4752cb700 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cdfae0 0x55c863d4c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc475acc700 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 0x55c863cf53f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc468017ee0 tx=0x7fc468004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e744e90d1f7b3f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc458ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.483+0000 7fc458ff9700 1 -- 172.21.15.46:0/615223068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468017820 con 0x55c863cef6b0 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc4836c4b80 1 -- 172.21.15.46:0/615223068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc46001e040 con 0x55c863cef6b0 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc458ff9700 1 -- 172.21.15.46:0/615223068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fc468017980 con 0x55c863cef6b0 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc458ff9700 1 -- 172.21.15.46:0/615223068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc468017b40 con 0x55c863cef6b0 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc4836c4b80 10 client.18051 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc4836c4b80 1 -- 172.21.15.46:0/615223068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c863ab8d50 con 0x55c863cef6b0 2024-03-18T20:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc458ff9700 1 -- 172.21.15.46:0/615223068 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc46001e040 con 0x55c863cef6b0 2024-03-18T20:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc458ff9700 1 -- 172.21.15.46:0/615223068 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc46801d020 con 0x55c863cef6b0 2024-03-18T20:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc4836c4b80 10 client.18051 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc4836c4b80 10 client.18051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc4836c4b80 20 client.18051 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c863d55280 0x55c863d576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc4836c4b80 4 client.18051 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc4836c4b80 1 -- 172.21.15.46:0/615223068 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c863a81d70 con 0x55c863d55280 2024-03-18T20:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.484+0000 7fc4752cb700 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c863d55280 0x55c863d576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.485+0000 7fc458ff9700 10 client.18051 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.485+0000 7fc4752cb700 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c863d55280 0x55c863d576a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c8874bb93305e74 server_cookie=c3a633d41a93d075 in_seq=0 out_seq=0 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc458ff9700 1 -- 172.21.15.46:0/615223068 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c863a81d70 con 0x55c863d55280 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc458ff9700 10 client.18051 handle_command_reply: tid=0 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc4836c4b80 1 client.18051 shutdown 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc4836c4b80 1 -- 172.21.15.46:0/615223068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 msgr2=0x55c863cf53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 0x55c863cf53f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc468017ee0 tx=0x7fc468004d60 comp rx=0 tx=0).stop 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc4836c4b80 1 -- 172.21.15.46:0/615223068 shutdown_connections 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c863d55280 0x55c863d576a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c863cef6b0 0x55c863cf53f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc458ff9700 0 client.18051 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c863cdfae0 0x55c863d4c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc4836c4b80 1 --2- 172.21.15.46:0/615223068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c863ce0400 0x55c863d4c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc4836c4b80 1 -- 172.21.15.46:0/615223068 >> 172.21.15.46:0/615223068 conn(0x55c863c562a0 msgr2=0x55c863cedc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc4836c4b80 1 -- 172.21.15.46:0/615223068 shutdown_connections 2024-03-18T20:43:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.486+0000 7fc4836c4b80 1 -- 172.21.15.46:0/615223068 wait complete. 2024-03-18T20:43:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc4836c4b80 20 client.18051 trim_cache size 0 max 16384 2024-03-18T20:43:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:43:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:43:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "5ca2f8f5-5c9a-4eec-9cbc-a936acf48e34", 2024-03-18T20:43:08.495 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:43:08.495 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc47752f700 1 -- 172.21.15.46:0/1475220391 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc45c0a1290 msgr2=0x7fc45c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc47752f700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc45c0a1290 0x7fc45c0a3750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc460005b40 tx=0x7fc460005ab0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc47752f700 1 -- 172.21.15.46:0/1475220391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4700ffe70 msgr2=0x7fc470197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc47752f700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4700ffe70 0x7fc470197220 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00e9a0 tx=0x7fc46c00ecb0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc47752f700 1 -- 172.21.15.46:0/1475220391 shutdown_connections 2024-03-18T20:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc47752f700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc45c0a1290 0x7fc45c0a3750 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc47752f700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4700ffe70 0x7fc470197220 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc47752f700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc47019bab0 0x7fc47019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc47752f700 1 --2- 172.21.15.46:0/1475220391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc470104f90 0x7fc470197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc47752f700 1 -- 172.21.15.46:0/1475220391 >> 172.21.15.46:0/1475220391 conn(0x7fc470076d80 msgr2=0x7fc470106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc47752f700 1 -- 172.21.15.46:0/1475220391 shutdown_connections 2024-03-18T20:43:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.487+0000 7fc47752f700 1 -- 172.21.15.46:0/1475220391 wait complete. 2024-03-18T20:43:08.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:43:08.507 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:43:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.648+0000 7f0eee4af700 1 -- 172.21.15.46:0/4203416805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee8106070 msgr2=0x7f0ee810dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.648+0000 7f0eee4af700 1 --2- 172.21.15.46:0/4203416805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee8106070 0x7f0ee810dbf0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b0a0 tx=0x7f0ee000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.648+0000 7f0eee4af700 1 -- 172.21.15.46:0/4203416805 shutdown_connections 2024-03-18T20:43:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.648+0000 7f0eee4af700 1 --2- 172.21.15.46:0/4203416805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee8106070 0x7f0ee810dbf0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.648+0000 7f0eee4af700 1 --2- 172.21.15.46:0/4203416805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee8100f50 0x7f0ee8101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.648+0000 7f0eee4af700 1 --2- 172.21.15.46:0/4203416805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ee8101900 0x7f0ee8105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.648+0000 7f0eee4af700 1 -- 172.21.15.46:0/4203416805 >> 172.21.15.46:0/4203416805 conn(0x7f0ee80fc7f0 msgr2=0x7f0ee80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:08.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.648+0000 7f0eee4af700 1 -- 172.21.15.46:0/4203416805 shutdown_connections 2024-03-18T20:43:08.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.648+0000 7f0eee4af700 1 -- 172.21.15.46:0/4203416805 wait complete. 2024-03-18T20:43:08.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.649+0000 7f0eee4af700 1 Processor -- start 2024-03-18T20:43:08.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.649+0000 7f0eee4af700 1 -- start start 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.649+0000 7f0eee4af700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee8100f50 0x7f0ee80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.649+0000 7f0eee4af700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ee8106070 0x7f0ee80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.649+0000 7f0eee4af700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee8073b30 0x7f0ee81a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.649+0000 7f0eee4af700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee81a5b30 con 0x7f0ee8100f50 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0eee4af700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee81a5ca0 con 0x7f0ee8106070 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0eee4af700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee81a5e60 con 0x7f0ee8073b30 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ee8106070 0x7f0ee80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee8100f50 0x7f0ee80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ee8106070 0x7f0ee80735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51294/0 (socket says 172.21.15.46:51294) 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee77fe700 1 -- 172.21.15.46:0/2040933265 learned_addr learned my addr 172.21.15.46:0/2040933265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0eeca4c700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee8073b30 0x7f0ee81a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee77fe700 1 -- 172.21.15.46:0/2040933265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee8073b30 msgr2=0x7f0ee81a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee77fe700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee8073b30 0x7f0ee81a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee77fe700 1 -- 172.21.15.46:0/2040933265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee8100f50 msgr2=0x7f0ee80730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee77fe700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee8100f50 0x7f0ee80730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee77fe700 1 -- 172.21.15.46:0/2040933265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0009d00 con 0x7f0ee8106070 2024-03-18T20:43:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee77fe700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ee8106070 0x7f0ee80735f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00e970 tx=0x7f0edc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee57fa700 1 -- 172.21.15.46:0/2040933265 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc009790 con 0x7f0ee8106070 2024-03-18T20:43:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee57fa700 1 -- 172.21.15.46:0/2040933265 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0edc00c7e0 con 0x7f0ee8106070 2024-03-18T20:43:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0eee4af700 1 -- 172.21.15.46:0/2040933265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee81a6140 con 0x7f0ee8106070 2024-03-18T20:43:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.650+0000 7f0ee57fa700 1 -- 172.21.15.46:0/2040933265 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0054e0 con 0x7f0ee8106070 2024-03-18T20:43:08.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.651+0000 7f0eee4af700 1 -- 172.21.15.46:0/2040933265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee81a6650 con 0x7f0ee8106070 2024-03-18T20:43:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.653+0000 7f0ee57fa700 1 -- 172.21.15.46:0/2040933265 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0edc00c950 con 0x7f0ee8106070 2024-03-18T20:43:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.654+0000 7f0ee57fa700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0ed00a1290 0x7f0ed00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.654+0000 7f0ee7fff700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0ed00a1290 0x7f0ed00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.654+0000 7f0ee57fa700 1 -- 172.21.15.46:0/2040933265 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f0edc014070 con 0x7f0ee8106070 2024-03-18T20:43:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.654+0000 7f0efa644b80 1 Processor -- start 2024-03-18T20:43:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.654+0000 7f0ee7fff700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0ed00a1290 0x7f0ed00a3750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009c60 tx=0x7f0ed8009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.654+0000 7f0efa644b80 1 -- start start 2024-03-18T20:43:08.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0efa644b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 0x558fe415de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0efa644b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe4168ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0efa644b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe416b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0efa644b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe40b7990 con 0x558fe4169220 2024-03-18T20:43:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0efa644b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe40b5350 con 0x558fe415e380 2024-03-18T20:43:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0efa644b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe40b0c40 con 0x558fe415da60 2024-03-18T20:43:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0eeca4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe416b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe4168ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe4168ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51304/0 (socket says 172.21.15.46:51304) 2024-03-18T20:43:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0ee77fe700 1 -- 172.21.15.46:0/3667760220 learned_addr learned my addr 172.21.15.46:0/3667760220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:08.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0eeca4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe416b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60288/0 (socket says 172.21.15.46:60288) 2024-03-18T20:43:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0ee7fff700 1 --2- 172.21.15.46:0/3667760220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 0x558fe415de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0ee77fe700 1 -- 172.21.15.46:0/3667760220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 msgr2=0x558fe415de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0ee77fe700 1 --2- 172.21.15.46:0/3667760220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 0x558fe415de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0ee77fe700 1 -- 172.21.15.46:0/3667760220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 msgr2=0x558fe416b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0ee77fe700 1 --2- 172.21.15.46:0/3667760220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe416b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0ee77fe700 1 -- 172.21.15.46:0/3667760220 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fe3f323b0 con 0x558fe415e380 2024-03-18T20:43:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0eeca4c700 1 --2- 172.21.15.46:0/3667760220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe416b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.655+0000 7f0ee77fe700 1 --2- 172.21.15.46:0/3667760220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe4168ce0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0a1f00 tx=0x7f0edc00ccd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e9a8a6bed83b573 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.656+0000 7f0ece7fc700 1 -- 172.21.15.46:0/3667760220 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc09dde0 con 0x558fe415e380 2024-03-18T20:43:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.656+0000 7f0ece7fc700 1 -- 172.21.15.46:0/3667760220 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0edc0a2430 con 0x558fe415e380 2024-03-18T20:43:08.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.656+0000 7f0ece7fc700 1 -- 172.21.15.46:0/3667760220 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc016590 con 0x558fe415e380 2024-03-18T20:43:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.656+0000 7f0efa644b80 1 -- 172.21.15.46:0/3667760220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 msgr2=0x558fe4168ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.656+0000 7f0efa644b80 1 --2- 172.21.15.46:0/3667760220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe4168ce0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0a1f00 tx=0x7f0edc00ccd0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.657+0000 7f0efa644b80 1 -- 172.21.15.46:0/3667760220 shutdown_connections 2024-03-18T20:43:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.657+0000 7f0efa644b80 1 --2- 172.21.15.46:0/3667760220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe416b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.657+0000 7f0efa644b80 1 --2- 172.21.15.46:0/3667760220 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 0x558fe415de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.657+0000 7f0efa644b80 1 --2- 172.21.15.46:0/3667760220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe4168ce0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.657+0000 7f0efa644b80 1 -- 172.21.15.46:0/3667760220 >> 172.21.15.46:0/3667760220 conn(0x558fe40d4220 msgr2=0x558fe415a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.657+0000 7f0efa644b80 1 -- 172.21.15.46:0/3667760220 shutdown_connections 2024-03-18T20:43:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.657+0000 7f0efa644b80 1 -- 172.21.15.46:0/3667760220 wait complete. 2024-03-18T20:43:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.657+0000 7f0efa644b80 1 -- start start 2024-03-18T20:43:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.657+0000 7f0efa644b80 1 Processor -- start 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0efa644b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 0x558fe4166090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0efa644b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe41665d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0efa644b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe4166b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0efa644b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe404cd80 con 0x558fe4169220 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0efa644b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe404ca20 con 0x558fe415e380 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0efa644b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe3f346a0 con 0x558fe415da60 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0efa644b80 10 client.0 _ll_register_callbacks cb 0x558fe3eb83a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe41665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe41665d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51316/0 (socket says 172.21.15.46:51316) 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ee7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 0x558fe4166090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ee77fe700 1 -- 172.21.15.46:0/1503157143 learned_addr learned my addr 172.21.15.46:0/1503157143 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0eeca4c700 1 --2- 172.21.15.46:0/1503157143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe4166b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ee77fe700 1 -- 172.21.15.46:0/1503157143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 msgr2=0x558fe4166090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ee77fe700 1 --2- 172.21.15.46:0/1503157143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 0x558fe4166090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ee77fe700 1 -- 172.21.15.46:0/1503157143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 msgr2=0x558fe4166b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ee77fe700 1 --2- 172.21.15.46:0/1503157143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe4166b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ee77fe700 1 -- 172.21.15.46:0/1503157143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee810e460 con 0x558fe415e380 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ee77fe700 1 --2- 172.21.15.46:0/1503157143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe41665d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00cf90 tx=0x7f0edc005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=829c67fdca6f194b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ec77fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ec77fe700 1 -- 172.21.15.46:0/1503157143 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0b81d0 con 0x558fe415e380 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0ec77fe700 1 -- 172.21.15.46:0/1503157143 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f0edc0b8330 con 0x558fe415e380 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.658+0000 7f0efa644b80 1 -- 172.21.15.46:0/1503157143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fe4151ad0 con 0x558fe415e380 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.659+0000 7f0ec77fe700 1 -- 172.21.15.46:0/1503157143 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0edc09a210 con 0x558fe415e380 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.659+0000 7f0efa644b80 10 client.27740 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.659+0000 7f0efa644b80 1 -- 172.21.15.46:0/1503157143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f0ee81a6650 con 0x558fe415e380 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.659+0000 7f0ec77fe700 1 -- 172.21.15.46:0/1503157143 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558fe4151ad0 con 0x558fe415e380 2024-03-18T20:43:08.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.659+0000 7f0ec77fe700 1 -- 172.21.15.46:0/1503157143 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0edc0a1fa0 con 0x558fe415e380 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.659+0000 7f0efa644b80 10 client.27740 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.659+0000 7f0efa644b80 10 client.27740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.659+0000 7f0efa644b80 20 client.27740 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.659+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1503157143 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558fe4178c30 0x558fe417b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.659+0000 7f0efa644b80 4 client.27740 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.659+0000 7f0efa644b80 1 -- 172.21.15.46:0/1503157143 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fe40a9700 con 0x558fe4178c30 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.659+0000 7f0ee7fff700 1 --2- 172.21.15.46:0/1503157143 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558fe4178c30 0x558fe417b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.660+0000 7f0ec77fe700 10 client.27740 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.660+0000 7f0ee7fff700 1 --2- 172.21.15.46:0/1503157143 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558fe4178c30 0x558fe417b050 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a48de7842e9097b server_cookie=46ac25a38506dada in_seq=0 out_seq=0 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0ec77fe700 1 -- 172.21.15.46:0/1503157143 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558fe40a9700 con 0x558fe4178c30 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0ec77fe700 10 client.27740 handle_command_reply: tid=0 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0efa644b80 1 client.27740 shutdown 2024-03-18T20:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0efa644b80 1 -- 172.21.15.46:0/1503157143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 msgr2=0x558fe41665d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1503157143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe41665d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00cf90 tx=0x7f0edc005870 comp rx=0 tx=0).stop 2024-03-18T20:43:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0efa644b80 1 -- 172.21.15.46:0/1503157143 shutdown_connections 2024-03-18T20:43:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1503157143 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558fe4178c30 0x558fe417b050 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1503157143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe4166b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0ec77fe700 0 client.27740 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1503157143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 0x558fe4166090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1503157143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe41665d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0efa644b80 1 -- 172.21.15.46:0/1503157143 >> 172.21.15.46:0/1503157143 conn(0x558fe40d4220 msgr2=0x558fe4159c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0efa644b80 1 -- 172.21.15.46:0/1503157143 shutdown_connections 2024-03-18T20:43:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0efa644b80 1 -- 172.21.15.46:0/1503157143 wait complete. 2024-03-18T20:43:08.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.661+0000 7f0efa644b80 20 client.27740 trim_cache size 0 max 16384 2024-03-18T20:43:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0efa644b80 1 Processor -- start 2024-03-18T20:43:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0efa644b80 1 -- start start 2024-03-18T20:43:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0efa644b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415da60 0x558fe416fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0efa644b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe415e380 0x558fe4166930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0efa644b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe4169220 0x558fe4159b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0efa644b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe415f550 con 0x558fe415e380 2024-03-18T20:43:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0efa644b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe4170330 con 0x558fe415da60 2024-03-18T20:43:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0efa644b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe417b6b0 con 0x558fe4169220 2024-03-18T20:43:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe415e380 0x558fe4166930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0ee7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415da60 0x558fe416fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0ee7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415da60 0x558fe416fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51326/0 (socket says 172.21.15.46:51326) 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe415e380 0x558fe4166930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60294/0 (socket says 172.21.15.46:60294) 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0ee7fff700 1 -- 172.21.15.46:0/1697000452 learned_addr learned my addr 172.21.15.46:0/1697000452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0eeca4c700 1 --2- 172.21.15.46:0/1697000452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe4169220 0x558fe4159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0ee7fff700 1 -- 172.21.15.46:0/1697000452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe4169220 msgr2=0x558fe4159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0ee7fff700 1 --2- 172.21.15.46:0/1697000452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe4169220 0x558fe4159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0ee7fff700 1 -- 172.21.15.46:0/1697000452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe415e380 msgr2=0x558fe4166930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0ee7fff700 1 --2- 172.21.15.46:0/1697000452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe415e380 0x558fe4166930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0ee7fff700 1 -- 172.21.15.46:0/1697000452 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fe3edeb60 con 0x558fe415da60 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.680+0000 7f0ee77fe700 1 --2- 172.21.15.46:0/1697000452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe415e380 0x558fe4166930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.681+0000 7f0ee7fff700 1 --2- 172.21.15.46:0/1697000452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415da60 0x558fe416fdf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8013010 tx=0x7f0ed8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75aee8ad85a1cb9c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.681+0000 7f0eceffd700 1 -- 172.21.15.46:0/1697000452 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed802e070 con 0x558fe415da60 2024-03-18T20:43:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.681+0000 7f0eceffd700 1 -- 172.21.15.46:0/1697000452 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ed80045a0 con 0x558fe415da60 2024-03-18T20:43:08.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.681+0000 7f0eceffd700 1 -- 172.21.15.46:0/1697000452 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8024040 con 0x558fe415da60 2024-03-18T20:43:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.682+0000 7f0efa644b80 1 -- 172.21.15.46:0/1697000452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415da60 msgr2=0x558fe416fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.682+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1697000452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415da60 0x558fe416fdf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8013010 tx=0x7f0ed8014000 comp rx=0 tx=0).stop 2024-03-18T20:43:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.682+0000 7f0efa644b80 1 -- 172.21.15.46:0/1697000452 shutdown_connections 2024-03-18T20:43:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.682+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1697000452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe4169220 0x558fe4159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.682+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1697000452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415da60 0x558fe416fdf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.682+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1697000452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe415e380 0x558fe4166930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.682+0000 7f0efa644b80 1 -- 172.21.15.46:0/1697000452 >> 172.21.15.46:0/1697000452 conn(0x558fe40d4220 msgr2=0x558fe41613c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.682+0000 7f0efa644b80 1 -- 172.21.15.46:0/1697000452 shutdown_connections 2024-03-18T20:43:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.682+0000 7f0efa644b80 1 -- 172.21.15.46:0/1697000452 wait complete. 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.682+0000 7f0efa644b80 1 -- start start 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.682+0000 7f0efa644b80 1 Processor -- start 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0efa644b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 0x558fe416fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0efa644b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe416e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0efa644b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe416e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0efa644b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe416ee90 con 0x558fe4169220 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0efa644b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe416efd0 con 0x558fe415e380 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0efa644b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fe41cca30 con 0x558fe415da60 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0efa644b80 10 client.0 _ll_register_callbacks cb 0x7f0edc005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0eeca4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe416e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe416e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0eeca4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe416e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60306/0 (socket says 172.21.15.46:60306) 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0eeca4c700 1 -- 172.21.15.46:0/1642707076 learned_addr learned my addr 172.21.15.46:0/1642707076 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0ee7fff700 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 0x558fe416fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0eeca4c700 1 -- 172.21.15.46:0/1642707076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 msgr2=0x558fe416fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0eeca4c700 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 0x558fe416fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0eeca4c700 1 -- 172.21.15.46:0/1642707076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 msgr2=0x558fe416e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0eeca4c700 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe416e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0eeca4c700 1 -- 172.21.15.46:0/1642707076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee81a6650 con 0x558fe4169220 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0ee77fe700 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe416e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0eeca4c700 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe416e7b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b0a0 tx=0x7f0ee000bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411da7adff61cbb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.683+0000 7f0ec77fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0ec77fe700 1 -- 172.21.15.46:0/1642707076 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0021070 con 0x558fe4169220 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0efa644b80 1 -- 172.21.15.46:0/1642707076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0edc0a2430 con 0x558fe4169220 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0ec77fe700 1 -- 172.21.15.46:0/1642707076 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f0ee00074e0 con 0x558fe4169220 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0ec77fe700 1 -- 172.21.15.46:0/1642707076 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ee000e040 con 0x558fe4169220 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0efa644b80 10 client.18069 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0efa644b80 1 -- 172.21.15.46:0/1642707076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f0ed8017070 con 0x558fe4169220 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0ec77fe700 1 -- 172.21.15.46:0/1642707076 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ee0031020 con 0x558fe4169220 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0ec77fe700 1 -- 172.21.15.46:0/1642707076 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0ee0019050 con 0x558fe4169220 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0efa644b80 10 client.18069 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0efa644b80 10 client.18069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0efa644b80 20 client.18069 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558fe41ce3b0 0x558fe41d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0efa644b80 4 client.18069 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.684+0000 7f0efa644b80 1 -- 172.21.15.46:0/1642707076 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558fe3effcf0 con 0x558fe41ce3b0 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0ee7fff700 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558fe41ce3b0 0x558fe41d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0ec77fe700 10 client.18069 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0ee7fff700 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558fe41ce3b0 0x558fe41d07d0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0d271174cc0388 server_cookie=bfec210d4f2e4875 in_seq=0 out_seq=0 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0ec77fe700 1 -- 172.21.15.46:0/1642707076 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558fe3effcf0 con 0x558fe41ce3b0 2024-03-18T20:43:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0ec77fe700 10 client.18069 handle_command_reply: tid=0 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0efa644b80 1 client.18069 shutdown 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0efa644b80 1 -- 172.21.15.46:0/1642707076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 msgr2=0x558fe416e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe416e7b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b0a0 tx=0x7f0ee000bc60 comp rx=0 tx=0).stop 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0efa644b80 1 -- 172.21.15.46:0/1642707076 shutdown_connections 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558fe41ce3b0 0x558fe41d07d0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558fe4169220 0x558fe416e7b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558fe415da60 0x558fe416fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0ec77fe700 0 client.18069 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0efa644b80 1 --2- 172.21.15.46:0/1642707076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fe415e380 0x558fe416e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0efa644b80 1 -- 172.21.15.46:0/1642707076 >> 172.21.15.46:0/1642707076 conn(0x558fe40d4220 msgr2=0x558fe4159aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0efa644b80 1 -- 172.21.15.46:0/1642707076 shutdown_connections 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.685+0000 7f0efa644b80 1 -- 172.21.15.46:0/1642707076 wait complete. 2024-03-18T20:43:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0efa644b80 20 client.18069 trim_cache size 0 max 16384 2024-03-18T20:43:08.695 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:43:08.695 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:43:08.695 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:43:08.695 INFO:teuthology.orchestra.run.smithi046.stdout: "5ca2f8f5-5c9a-4eec-9cbc-a936acf48e34": { 2024-03-18T20:43:08.695 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:43:08.695 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "5ca2f8f5-5c9a-4eec-9cbc-a936acf48e34", 2024-03-18T20:43:08.695 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:43:08.695 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:43:08.695 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:43:08.695 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:43:08.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0eee4af700 1 -- 172.21.15.46:0/2040933265 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0ed00a1290 msgr2=0x7f0ed00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0eee4af700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0ed00a1290 0x7f0ed00a3750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009c60 tx=0x7f0ed8009260 comp rx=0 tx=0).stop 2024-03-18T20:43:08.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0eee4af700 1 -- 172.21.15.46:0/2040933265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ee8106070 msgr2=0x7f0ee80735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:08.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0eee4af700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ee8106070 0x7f0ee80735f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00e970 tx=0x7f0edc00ec80 comp rx=0 tx=0).stop 2024-03-18T20:43:08.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0eee4af700 1 -- 172.21.15.46:0/2040933265 shutdown_connections 2024-03-18T20:43:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0eee4af700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f0ed00a1290 0x7f0ed00a3750 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0eee4af700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ee8100f50 0x7f0ee80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0eee4af700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ee8073b30 0x7f0ee81a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0eee4af700 1 --2- 172.21.15.46:0/2040933265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ee8106070 0x7f0ee80735f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0eee4af700 1 -- 172.21.15.46:0/2040933265 >> 172.21.15.46:0/2040933265 conn(0x7f0ee80fc7f0 msgr2=0x7f0ee80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0eee4af700 1 -- 172.21.15.46:0/2040933265 shutdown_connections 2024-03-18T20:43:08.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:08.686+0000 7f0eee4af700 1 -- 172.21.15.46:0/2040933265 wait complete. 2024-03-18T20:43:08.698 INFO:tasks.workunit.client.0.smithi046.stdout: 1 239896 3.49 MB/sec execute 243 sec latency 1124.294 ms 2024-03-18T20:43:08.703 INFO:tasks.cephfs.filesystem:scrub status for tag:5ca2f8f5-5c9a-4eec-9cbc-a936acf48e34 - {'path': '/', 'tag': '5ca2f8f5-5c9a-4eec-9cbc-a936acf48e34', 'options': 'recursive,force'} 2024-03-18T20:43:09.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:08 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:09.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:08 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:09.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:08 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:09.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:08 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:09.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:08 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:09.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:08 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:09.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 241609 3.53 MB/sec execute 244 sec latency 15.662 ms 2024-03-18T20:43:10.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:43:10.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:10.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:10.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:10.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:10.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:10.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:10.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:09 smithi073 ceph-mon[47384]: pgmap v216: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-03-18T20:43:10.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:43:10.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:10.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:10.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:10.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:10.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:10.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:10.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:10.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:10.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:09 smithi155 ceph-mon[48344]: pgmap v216: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-03-18T20:43:10.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:43:10.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:10.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:10.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:10.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:10.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:10.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:10.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:10.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:10.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:10.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:09 smithi046 ceph-mon[43495]: pgmap v216: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-03-18T20:43:10.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 242931 3.53 MB/sec execute 245 sec latency 109.259 ms 2024-03-18T20:43:11.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 242931 3.52 MB/sec execute 246 sec latency 1109.367 ms 2024-03-18T20:43:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.399+0000 7fa13371e700 1 -- 172.21.15.46:0/1747665333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa12c106050 msgr2=0x7fa12c069970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.399+0000 7fa13371e700 1 --2- 172.21.15.46:0/1747665333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa12c106050 0x7fa12c069970 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009990 tx=0x7fa11c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.399+0000 7fa13371e700 1 -- 172.21.15.46:0/1747665333 shutdown_connections 2024-03-18T20:43:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.399+0000 7fa13371e700 1 --2- 172.21.15.46:0/1747665333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12c06a050 0x7fa12c103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.399+0000 7fa13371e700 1 --2- 172.21.15.46:0/1747665333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa12c105be0 0x7fa12c069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.399+0000 7fa13371e700 1 --2- 172.21.15.46:0/1747665333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa12c106050 0x7fa12c069970 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.399+0000 7fa13371e700 1 -- 172.21.15.46:0/1747665333 >> 172.21.15.46:0/1747665333 conn(0x7fa12c076a80 msgr2=0x7fa12c076e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.399+0000 7fa13371e700 1 -- 172.21.15.46:0/1747665333 shutdown_connections 2024-03-18T20:43:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.399+0000 7fa13371e700 1 -- 172.21.15.46:0/1747665333 wait complete. 2024-03-18T20:43:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.399+0000 7fa13371e700 1 Processor -- start 2024-03-18T20:43:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.399+0000 7fa13371e700 1 -- start start 2024-03-18T20:43:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa13371e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12c105be0 0x7fa12c19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa13371e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa12c106050 0x7fa12c19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa13371e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa12c1a0fe0 0x7fa12c1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa13371e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c1a19d0 con 0x7fa12c1a0fe0 2024-03-18T20:43:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa13371e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c1a1b40 con 0x7fa12c106050 2024-03-18T20:43:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa13371e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c1a1d00 con 0x7fa12c105be0 2024-03-18T20:43:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa131cbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa12c1a0fe0 0x7fa12c1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa131cbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa12c1a0fe0 0x7fa12c1a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60320/0 (socket says 172.21.15.46:60320) 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa131cbb700 1 -- 172.21.15.46:0/3214617888 learned_addr learned my addr 172.21.15.46:0/3214617888 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa130cb9700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa12c106050 0x7fa12c19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa1314ba700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12c105be0 0x7fa12c19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa131cbb700 1 -- 172.21.15.46:0/3214617888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12c105be0 msgr2=0x7fa12c19d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa131cbb700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12c105be0 0x7fa12c19d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa131cbb700 1 -- 172.21.15.46:0/3214617888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa12c106050 msgr2=0x7fa12c19dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa131cbb700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa12c106050 0x7fa12c19dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa131cbb700 1 -- 172.21.15.46:0/3214617888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0095f0 con 0x7fa12c1a0fe0 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa1314ba700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12c105be0 0x7fa12c19d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.400+0000 7fa131cbb700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa12c1a0fe0 0x7fa12c1a1490 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fa12400b9e0 tx=0x7fa12400bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.401+0000 7fa1227fc700 1 -- 172.21.15.46:0/3214617888 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12400d8e0 con 0x7fa12c1a0fe0 2024-03-18T20:43:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.401+0000 7fa1227fc700 1 -- 172.21.15.46:0/3214617888 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa124015420 con 0x7fa12c1a0fe0 2024-03-18T20:43:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.401+0000 7fa13371e700 1 -- 172.21.15.46:0/3214617888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12c1aa370 con 0x7fa12c1a0fe0 2024-03-18T20:43:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.401+0000 7fa1227fc700 1 -- 172.21.15.46:0/3214617888 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124014630 con 0x7fa12c1a0fe0 2024-03-18T20:43:12.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.401+0000 7fa13371e700 1 -- 172.21.15.46:0/3214617888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12c1aa850 con 0x7fa12c1a0fe0 2024-03-18T20:43:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.403+0000 7fa1227fc700 1 -- 172.21.15.46:0/3214617888 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa124015590 con 0x7fa12c1a0fe0 2024-03-18T20:43:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.404+0000 7fa1227fc700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa1180a1340 0x7fa1180a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.404+0000 7fa1314ba700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa1180a1340 0x7fa1180a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.404+0000 7fa1227fc700 1 -- 172.21.15.46:0/3214617888 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fa124011070 con 0x7fa12c1a0fe0 2024-03-18T20:43:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.404+0000 7fa1314ba700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa1180a1340 0x7fa1180a3800 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa128005950 tx=0x7fa128005f50 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa13f8b3b80 1 Processor -- start 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa13f8b3b80 1 -- start start 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa13f8b3b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cba3570 0x558a7cba3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa13f8b3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbae870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa13f8b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 0x558a7cbb11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa13f8b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7cafd930 con 0x558a7cbaedb0 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa13f8b3b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7cafb2f0 con 0x558a7cba3570 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa13f8b3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7caf6be0 con 0x558a7cba3e90 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa131cbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 0x558a7cbb11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa131cbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 0x558a7cbb11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60328/0 (socket says 172.21.15.46:60328) 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa131cbb700 1 -- 172.21.15.46:0/1911959777 learned_addr learned my addr 172.21.15.46:0/1911959777 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa130cb9700 1 --2- 172.21.15.46:0/1911959777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.405+0000 7fa1314ba700 1 --2- 172.21.15.46:0/1911959777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cba3570 0x558a7cba3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.406+0000 7fa131cbb700 1 -- 172.21.15.46:0/1911959777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 msgr2=0x558a7cbae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.406+0000 7fa131cbb700 1 --2- 172.21.15.46:0/1911959777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.406+0000 7fa131cbb700 1 -- 172.21.15.46:0/1911959777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cba3570 msgr2=0x558a7cba3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.406+0000 7fa131cbb700 1 --2- 172.21.15.46:0/1911959777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cba3570 0x558a7cba3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.406+0000 7fa131cbb700 1 -- 172.21.15.46:0/1911959777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a7c978350 con 0x558a7cbaedb0 2024-03-18T20:43:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.406+0000 7fa130cb9700 1 --2- 172.21.15.46:0/1911959777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.406+0000 7fa1314ba700 1 --2- 172.21.15.46:0/1911959777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cba3570 0x558a7cba3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.406+0000 7fa131cbb700 1 --2- 172.21.15.46:0/1911959777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 0x558a7cbb11a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fa124095be0 tx=0x7fa124004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4e2642622d8192 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.406+0000 7fa1177fe700 1 -- 172.21.15.46:0/1911959777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12409d510 con 0x558a7cbaedb0 2024-03-18T20:43:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.406+0000 7fa1177fe700 1 -- 172.21.15.46:0/1911959777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1240a1b60 con 0x558a7cbaedb0 2024-03-18T20:43:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.406+0000 7fa1177fe700 1 -- 172.21.15.46:0/1911959777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124098bd0 con 0x558a7cbaedb0 2024-03-18T20:43:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.407+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/1911959777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 msgr2=0x558a7cbb11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.407+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/1911959777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 0x558a7cbb11a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fa124095be0 tx=0x7fa124004be0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.407+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/1911959777 shutdown_connections 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.407+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/1911959777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 0x558a7cbb11a0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.407+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/1911959777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cba3570 0x558a7cba3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.407+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/1911959777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.407+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/1911959777 >> 172.21.15.46:0/1911959777 conn(0x558a7cb1a1c0 msgr2=0x558a7cb9fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.407+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/1911959777 shutdown_connections 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.407+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/1911959777 wait complete. 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.407+0000 7fa13f8b3b80 1 -- start start 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.407+0000 7fa13f8b3b80 1 Processor -- start 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa13f8b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3570 0x558a7cbae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa13f8b3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbabb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa13f8b3b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cbaedb0 0x558a7cbac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa13f8b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ca92d20 con 0x558a7cba3570 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa13f8b3b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ca929c0 con 0x558a7cbaedb0 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa13f8b3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7c97a640 con 0x558a7cba3e90 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa13f8b3b80 10 client.0 _ll_register_callbacks cb 0x558a7c8fe340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa1314ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3570 0x558a7cbae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa1314ba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3570 0x558a7cbae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60332/0 (socket says 172.21.15.46:60332) 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa1314ba700 1 -- 172.21.15.46:0/2125899391 learned_addr learned my addr 172.21.15.46:0/2125899391 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa131cbb700 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cbaedb0 0x558a7cbac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa130cb9700 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbabb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa1314ba700 1 -- 172.21.15.46:0/2125899391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 msgr2=0x558a7cbabb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa1314ba700 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbabb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa1314ba700 1 -- 172.21.15.46:0/2125899391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cbaedb0 msgr2=0x558a7cbac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa1314ba700 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cbaedb0 0x558a7cbac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.408+0000 7fa1314ba700 1 -- 172.21.15.46:0/2125899391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12c1aa850 con 0x558a7cba3570 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.409+0000 7fa1314ba700 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3570 0x558a7cbae820 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa12800aa10 tx=0x7fa128004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337614f3ccdba29e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.409+0000 7fa114ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.409+0000 7fa114ff9700 1 -- 172.21.15.46:0/2125899391 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12800fa50 con 0x558a7cba3570 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.409+0000 7fa114ff9700 1 -- 172.21.15.46:0/2125899391 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fa12800ee80 con 0x558a7cba3570 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.409+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/2125899391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a7cb976c0 con 0x558a7cba3570 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.409+0000 7fa114ff9700 1 -- 172.21.15.46:0/2125899391 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa12800fbb0 con 0x558a7cba3570 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.409+0000 7fa13f8b3b80 10 client.18081 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.409+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/2125899391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x558a7c924b00 con 0x558a7cba3570 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.409+0000 7fa114ff9700 1 -- 172.21.15.46:0/2125899391 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a7cb976c0 con 0x558a7cba3570 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.409+0000 7fa114ff9700 1 -- 172.21.15.46:0/2125899391 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa128024030 con 0x558a7cba3570 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.410+0000 7fa13f8b3b80 10 client.18081 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.410+0000 7fa13f8b3b80 10 client.18081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.410+0000 7fa13f8b3b80 20 client.18081 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.410+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a7cbbeba0 0x558a7cbc0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.410+0000 7fa13f8b3b80 4 client.18081 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.410+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/2125899391 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a7caef6a0 con 0x558a7cbbeba0 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.410+0000 7fa130cb9700 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a7cbbeba0 0x558a7cbc0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.410+0000 7fa130cb9700 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a7cbbeba0 0x558a7cbc0fc0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e908ed10edf4089 server_cookie=19e6c96515c5426a in_seq=0 out_seq=0 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.410+0000 7fa114ff9700 10 client.18081 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa114ff9700 1 -- 172.21.15.46:0/2125899391 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558a7caef6a0 con 0x558a7cbbeba0 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa114ff9700 10 client.18081 handle_command_reply: tid=0 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa13f8b3b80 1 client.18081 shutdown 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/2125899391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3570 msgr2=0x558a7cbae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3570 0x558a7cbae820 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa12800aa10 tx=0x7fa128004750 comp rx=0 tx=0).stop 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/2125899391 shutdown_connections 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a7cbbeba0 0x558a7cbc0fc0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3570 0x558a7cbae820 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cbaedb0 0x558a7cbac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa114ff9700 0 client.18081 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/2125899391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbabb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/2125899391 >> 172.21.15.46:0/2125899391 conn(0x558a7cb1a1c0 msgr2=0x558a7cb9f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/2125899391 shutdown_connections 2024-03-18T20:43:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/2125899391 wait complete. 2024-03-18T20:43:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.411+0000 7fa13f8b3b80 20 client.18081 trim_cache size 0 max 16384 2024-03-18T20:43:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa13f8b3b80 1 Processor -- start 2024-03-18T20:43:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa13f8b3b80 1 -- start start 2024-03-18T20:43:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa13f8b3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3570 0x558a7cbb5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa13f8b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3e90 0x558a7cba6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa13f8b3b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cbaedb0 0x558a7cba73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa13f8b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7cba0d60 con 0x558a7cba3e90 2024-03-18T20:43:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa13f8b3b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7cb99c10 con 0x558a7cbaedb0 2024-03-18T20:43:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa13f8b3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7cba0f70 con 0x558a7cba3570 2024-03-18T20:43:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa130cb9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3e90 0x558a7cba6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa131cbb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cbaedb0 0x558a7cba73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa130cb9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3e90 0x558a7cba6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60344/0 (socket says 172.21.15.46:60344) 2024-03-18T20:43:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa130cb9700 1 -- 172.21.15.46:0/3821595927 learned_addr learned my addr 172.21.15.46:0/3821595927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa1314ba700 1 --2- 172.21.15.46:0/3821595927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3570 0x558a7cbb5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa130cb9700 1 -- 172.21.15.46:0/3821595927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3570 msgr2=0x558a7cbb5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa130cb9700 1 --2- 172.21.15.46:0/3821595927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3570 0x558a7cbb5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa130cb9700 1 -- 172.21.15.46:0/3821595927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cbaedb0 msgr2=0x558a7cba73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa130cb9700 1 --2- 172.21.15.46:0/3821595927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cbaedb0 0x558a7cba73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa130cb9700 1 -- 172.21.15.46:0/3821595927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a7cabce40 con 0x558a7cba3e90 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.427+0000 7fa131cbb700 1 --2- 172.21.15.46:0/3821595927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cbaedb0 0x558a7cba73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.428+0000 7fa130cb9700 1 --2- 172.21.15.46:0/3821595927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3e90 0x558a7cba6eb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00b720 tx=0x7fa11c005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68acf64ac431892 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.428+0000 7fa117fff700 1 -- 172.21.15.46:0/3821595927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c01d040 con 0x558a7cba3e90 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.428+0000 7fa117fff700 1 -- 172.21.15.46:0/3821595927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa11c01e040 con 0x558a7cba3e90 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.428+0000 7fa117fff700 1 -- 172.21.15.46:0/3821595927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00f6c0 con 0x558a7cba3e90 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.428+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/3821595927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3e90 msgr2=0x558a7cba6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.428+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/3821595927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3e90 0x558a7cba6eb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00b720 tx=0x7fa11c005960 comp rx=0 tx=0).stop 2024-03-18T20:43:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.428+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/3821595927 shutdown_connections 2024-03-18T20:43:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.428+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/3821595927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cbaedb0 0x558a7cba73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.428+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/3821595927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3570 0x558a7cbb5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.428+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/3821595927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cba3e90 0x558a7cba6eb0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.428+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/3821595927 >> 172.21.15.46:0/3821595927 conn(0x558a7cb1a1c0 msgr2=0x558a7cb9e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.429+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/3821595927 shutdown_connections 2024-03-18T20:43:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.429+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/3821595927 wait complete. 2024-03-18T20:43:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.429+0000 7fa13f8b3b80 1 -- start start 2024-03-18T20:43:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.429+0000 7fa13f8b3b80 1 Processor -- start 2024-03-18T20:43:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.429+0000 7fa13f8b3b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cba3570 0x558a7cbb3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.429+0000 7fa13f8b3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbb4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.429+0000 7fa13f8b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 0x558a7cc11990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.429+0000 7fa13f8b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7cc11ed0 con 0x558a7cbaedb0 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.429+0000 7fa13f8b3b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7cc12010 con 0x558a7cba3570 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.429+0000 7fa13f8b3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7cc12150 con 0x558a7cba3e90 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.429+0000 7fa13f8b3b80 10 client.0 _ll_register_callbacks cb 0x7fa128010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa131cbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 0x558a7cc11990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa130cb9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbb4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa131cbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 0x558a7cc11990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60356/0 (socket says 172.21.15.46:60356) 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa131cbb700 1 -- 172.21.15.46:0/925188413 learned_addr learned my addr 172.21.15.46:0/925188413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa1314ba700 1 --2- 172.21.15.46:0/925188413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cba3570 0x558a7cbb3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa131cbb700 1 -- 172.21.15.46:0/925188413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 msgr2=0x558a7cbb4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa131cbb700 1 --2- 172.21.15.46:0/925188413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbb4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa131cbb700 1 -- 172.21.15.46:0/925188413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cba3570 msgr2=0x558a7cbb3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa131cbb700 1 --2- 172.21.15.46:0/925188413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cba3570 0x558a7cbb3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa131cbb700 1 -- 172.21.15.46:0/925188413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a7c978350 con 0x558a7cbaedb0 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa131cbb700 1 --2- 172.21.15.46:0/925188413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 0x558a7cc11990 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa124015fd0 tx=0x7fa124009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a231b4de53c14e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa114ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa114ff9700 1 -- 172.21.15.46:0/925188413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12409d510 con 0x558a7cbaedb0 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa114ff9700 1 -- 172.21.15.46:0/925188413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fa124006e40 con 0x558a7cbaedb0 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/925188413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1240a1b60 con 0x558a7cbaedb0 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa114ff9700 1 -- 172.21.15.46:0/925188413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa12409d160 con 0x558a7cbaedb0 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa13f8b3b80 10 client.18087 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.430+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/925188413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fa12c104400 con 0x558a7cbaedb0 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.431+0000 7fa114ff9700 1 -- 172.21.15.46:0/925188413 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1240a1b60 con 0x558a7cbaedb0 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.431+0000 7fa114ff9700 1 -- 172.21.15.46:0/925188413 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa1240a16d0 con 0x558a7cbaedb0 2024-03-18T20:43:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.431+0000 7fa13f8b3b80 10 client.18087 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.431+0000 7fa13f8b3b80 10 client.18087 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.431+0000 7fa13f8b3b80 20 client.18087 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.431+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/925188413 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a7cc135a0 0x558a7cc159c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.431+0000 7fa13f8b3b80 4 client.18087 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:43:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.431+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/925188413 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558a7c945c90 con 0x558a7cc135a0 2024-03-18T20:43:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.431+0000 7fa1314ba700 1 --2- 172.21.15.46:0/925188413 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a7cc135a0 0x558a7cc159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.432+0000 7fa114ff9700 10 client.18087 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.432+0000 7fa1314ba700 1 --2- 172.21.15.46:0/925188413 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a7cc135a0 0x558a7cc159c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65fdd36dde91d703 server_cookie=9f11aafe859e596c in_seq=0 out_seq=0 2024-03-18T20:43:12.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:12 smithi073 ceph-mon[47384]: pgmap v217: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-03-18T20:43:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa114ff9700 1 -- 172.21.15.46:0/925188413 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558a7c945c90 con 0x558a7cc135a0 2024-03-18T20:43:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa114ff9700 10 client.18087 handle_command_reply: tid=0 2024-03-18T20:43:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa13f8b3b80 1 client.18087 shutdown 2024-03-18T20:43:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/925188413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 msgr2=0x558a7cc11990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/925188413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 0x558a7cc11990 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa124015fd0 tx=0x7fa124009f90 comp rx=0 tx=0).stop 2024-03-18T20:43:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/925188413 shutdown_connections 2024-03-18T20:43:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/925188413 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558a7cc135a0 0x558a7cc159c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/925188413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558a7cbaedb0 0x558a7cc11990 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/925188413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558a7cba3570 0x558a7cbb3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa13f8b3b80 1 --2- 172.21.15.46:0/925188413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a7cba3e90 0x558a7cbb4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa114ff9700 0 client.18087 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/925188413 >> 172.21.15.46:0/925188413 conn(0x558a7cb1a1c0 msgr2=0x558a7cb9f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/925188413 shutdown_connections 2024-03-18T20:43:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa13f8b3b80 1 -- 172.21.15.46:0/925188413 wait complete. 2024-03-18T20:43:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.471+0000 7fa13f8b3b80 20 client.18087 trim_cache size 0 max 16384 2024-03-18T20:43:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:43:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:43:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:43:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:43:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.472+0000 7fa13371e700 1 -- 172.21.15.46:0/3214617888 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa1180a1340 msgr2=0x7fa1180a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.472+0000 7fa13371e700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa1180a1340 0x7fa1180a3800 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa128005950 tx=0x7fa128005f50 comp rx=0 tx=0).stop 2024-03-18T20:43:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.472+0000 7fa13371e700 1 -- 172.21.15.46:0/3214617888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa12c1a0fe0 msgr2=0x7fa12c1a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.472+0000 7fa13371e700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa12c1a0fe0 0x7fa12c1a1490 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fa12400b9e0 tx=0x7fa12400bcf0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.472+0000 7fa13371e700 1 -- 172.21.15.46:0/3214617888 shutdown_connections 2024-03-18T20:43:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.472+0000 7fa13371e700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa1180a1340 0x7fa1180a3800 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.472+0000 7fa13371e700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa12c1a0fe0 0x7fa12c1a1490 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.472+0000 7fa13371e700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa12c105be0 0x7fa12c19d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.472+0000 7fa13371e700 1 --2- 172.21.15.46:0/3214617888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa12c106050 0x7fa12c19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.472+0000 7fa13371e700 1 -- 172.21.15.46:0/3214617888 >> 172.21.15.46:0/3214617888 conn(0x7fa12c076a80 msgr2=0x7fa12c0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.472+0000 7fa13371e700 1 -- 172.21.15.46:0/3214617888 shutdown_connections 2024-03-18T20:43:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:12.472+0000 7fa13371e700 1 -- 172.21.15.46:0/3214617888 wait complete. 2024-03-18T20:43:12.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:12 smithi155 ceph-mon[48344]: pgmap v217: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-03-18T20:43:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:12 smithi046 ceph-mon[43495]: pgmap v217: 97 pgs: 97 active+clean; 94 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-03-18T20:43:12.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 242931 3.50 MB/sec execute 247 sec latency 2109.455 ms 2024-03-18T20:43:13.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 242931 3.49 MB/sec execute 248 sec latency 3109.560 ms 2024-03-18T20:43:14.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:14 smithi073 ceph-mon[47384]: pgmap v218: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-03-18T20:43:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:14 smithi155 ceph-mon[48344]: pgmap v218: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-03-18T20:43:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:14 smithi046 ceph-mon[43495]: pgmap v218: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-03-18T20:43:14.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 242931 3.48 MB/sec execute 249 sec latency 4109.670 ms 2024-03-18T20:43:15.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 245007 3.52 MB/sec execute 250 sec latency 4173.012 ms 2024-03-18T20:43:16.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:15 smithi073 ceph-mon[47384]: osdmap e84: 12 total, 12 up, 12 in 2024-03-18T20:43:16.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:15 smithi073 ceph-mon[47384]: pgmap v220: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-03-18T20:43:16.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:15 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:43:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:15 smithi155 ceph-mon[48344]: osdmap e84: 12 total, 12 up, 12 in 2024-03-18T20:43:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:15 smithi155 ceph-mon[48344]: pgmap v220: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-03-18T20:43:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:15 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:43:16.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:15 smithi046 ceph-mon[43495]: osdmap e84: 12 total, 12 up, 12 in 2024-03-18T20:43:16.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:15 smithi046 ceph-mon[43495]: pgmap v220: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-03-18T20:43:16.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:15 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:43:16.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 246254 3.52 MB/sec execute 251 sec latency 16.548 ms 2024-03-18T20:43:17.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:16 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:43:17.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:16 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:43:17.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:16 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:43:17.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:16 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:43:17.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:16 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:43:17.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:16 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:43:17.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:16 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:43:17.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:16 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:43:17.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:16 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:43:17.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 246478 3.51 MB/sec execute 252 sec latency 761.983 ms 2024-03-18T20:43:18.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:17 smithi073 ceph-mon[47384]: pgmap v221: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-03-18T20:43:18.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:17 smithi155 ceph-mon[48344]: pgmap v221: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-03-18T20:43:18.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:17 smithi046 ceph-mon[43495]: pgmap v221: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-03-18T20:43:18.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 246478 3.50 MB/sec execute 253 sec latency 1762.104 ms 2024-03-18T20:43:19.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 246478 3.48 MB/sec execute 254 sec latency 2762.197 ms 2024-03-18T20:43:20.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:20 smithi073 ceph-mon[47384]: pgmap v222: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-03-18T20:43:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:20 smithi155 ceph-mon[48344]: pgmap v222: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-03-18T20:43:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:20 smithi046 ceph-mon[43495]: pgmap v222: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-03-18T20:43:20.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 247382 3.48 MB/sec execute 255 sec latency 3392.345 ms 2024-03-18T20:43:21.699 INFO:tasks.workunit.client.0.smithi046.stdout: 1 248890 3.51 MB/sec execute 256 sec latency 14.719 ms 2024-03-18T20:43:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:22 smithi073 ceph-mon[47384]: pgmap v223: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-03-18T20:43:22.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:22 smithi155 ceph-mon[48344]: pgmap v223: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-03-18T20:43:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:22 smithi046 ceph-mon[43495]: pgmap v223: 97 pgs: 97 active+clean; 98 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.6 MiB/s wr, 157 op/s 2024-03-18T20:43:22.700 INFO:tasks.workunit.client.0.smithi046.stdout: 1 250021 3.52 MB/sec execute 257 sec latency 229.025 ms 2024-03-18T20:43:23.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:23 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:43:23.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:23 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:43:23.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:23 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:43:23.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:23 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:43:23.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:23 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:43:23.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:23 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:43:23.700 INFO:tasks.workunit.client.0.smithi046.stdout: 1 250021 3.50 MB/sec execute 258 sec latency 1229.120 ms 2024-03-18T20:43:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:24 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:43:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:24 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:43:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:24 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:43:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:24 smithi073 ceph-mon[47384]: pgmap v224: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.7 MiB/s wr, 134 op/s 2024-03-18T20:43:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:24 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:43:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:24 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:43:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:24 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:43:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:24 smithi155 ceph-mon[48344]: pgmap v224: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.7 MiB/s wr, 134 op/s 2024-03-18T20:43:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:24 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:43:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:24 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:43:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:24 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:43:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:24 smithi046 ceph-mon[43495]: pgmap v224: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.7 MiB/s wr, 134 op/s 2024-03-18T20:43:24.700 INFO:tasks.workunit.client.0.smithi046.stdout: 1 250021 3.49 MB/sec execute 259 sec latency 2229.247 ms 2024-03-18T20:43:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.605+0000 7fc1a8838700 1 -- 172.21.15.46:0/2374050845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a40ffbb0 msgr2=0x7fc1a4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.605+0000 7fc1a8838700 1 --2- 172.21.15.46:0/2374050845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a40ffbb0 0x7fc1a4100030 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009990 tx=0x7fc18c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.605+0000 7fc1a8838700 1 -- 172.21.15.46:0/2374050845 shutdown_connections 2024-03-18T20:43:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.605+0000 7fc1a8838700 1 --2- 172.21.15.46:0/2374050845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a4100570 0x7fc1a410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.605+0000 7fc1a8838700 1 --2- 172.21.15.46:0/2374050845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a4105c60 0x7fc1a4106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.605+0000 7fc1a8838700 1 --2- 172.21.15.46:0/2374050845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a40ffbb0 0x7fc1a4100030 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.605+0000 7fc1a8838700 1 -- 172.21.15.46:0/2374050845 >> 172.21.15.46:0/2374050845 conn(0x7fc1a4076d50 msgr2=0x7fc1a4077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.605+0000 7fc1a8838700 1 -- 172.21.15.46:0/2374050845 shutdown_connections 2024-03-18T20:43:25.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.605+0000 7fc1a8838700 1 -- 172.21.15.46:0/2374050845 wait complete. 2024-03-18T20:43:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a8838700 1 Processor -- start 2024-03-18T20:43:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a8838700 1 -- start start 2024-03-18T20:43:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a8838700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a40ffbb0 0x7fc1a4196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a8838700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a4100570 0x7fc1a4196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a8838700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a4105c60 0x7fc1a419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a8838700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a419af00 con 0x7fc1a4100570 2024-03-18T20:43:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a8838700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a419b070 con 0x7fc1a4105c60 2024-03-18T20:43:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a8838700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a419b230 con 0x7fc1a40ffbb0 2024-03-18T20:43:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a4100570 0x7fc1a4196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a40ffbb0 0x7fc1a4196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a4100570 0x7fc1a4196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44558/0 (socket says 172.21.15.46:44558) 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a1d9b700 1 -- 172.21.15.46:0/284338105 learned_addr learned my addr 172.21.15.46:0/284338105 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a259c700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a40ffbb0 0x7fc1a4196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37102/0 (socket says 172.21.15.46:37102) 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.606+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a4105c60 0x7fc1a419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.607+0000 7fc1a2d9d700 1 -- 172.21.15.46:0/284338105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a40ffbb0 msgr2=0x7fc1a4196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.607+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a40ffbb0 0x7fc1a4196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.607+0000 7fc1a2d9d700 1 -- 172.21.15.46:0/284338105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a4100570 msgr2=0x7fc1a4196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.607+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a4100570 0x7fc1a4196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.607+0000 7fc1a2d9d700 1 -- 172.21.15.46:0/284338105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c009c40 con 0x7fc1a4105c60 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.607+0000 7fc1a1d9b700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a4100570 0x7fc1a4196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.607+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a4105c60 0x7fc1a419a8c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00efb0 tx=0x7fc19c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.607+0000 7fc19b7fe700 1 -- 172.21.15.46:0/284338105 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c003a00 con 0x7fc1a4105c60 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.607+0000 7fc1a8838700 1 -- 172.21.15.46:0/284338105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c0095f0 con 0x7fc1a4105c60 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.607+0000 7fc19b7fe700 1 -- 172.21.15.46:0/284338105 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19c00f040 con 0x7fc1a4105c60 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.607+0000 7fc19b7fe700 1 -- 172.21.15.46:0/284338105 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c004540 con 0x7fc1a4105c60 2024-03-18T20:43:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.607+0000 7fc1a8838700 1 -- 172.21.15.46:0/284338105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a41a2b70 con 0x7fc1a4105c60 2024-03-18T20:43:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.610+0000 7fc19b7fe700 1 -- 172.21.15.46:0/284338105 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc19c003bf0 con 0x7fc1a4105c60 2024-03-18T20:43:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.611+0000 7fc19b7fe700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1900a1310 0x7fc1900a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.611+0000 7fc19b7fe700 1 -- 172.21.15.46:0/284338105 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc19c0cc900 con 0x7fc1a4105c60 2024-03-18T20:43:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.611+0000 7fc1a259c700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1900a1310 0x7fc1900a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.611+0000 7fc1b49cdb80 1 Processor -- start 2024-03-18T20:43:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.611+0000 7fc1a259c700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1900a1310 0x7fc1900a37d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc194009c60 tx=0x7fc194009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.611+0000 7fc1b49cdb80 1 -- start start 2024-03-18T20:43:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1b49cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0570 0x5560890a0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1b49cdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0e90 0x5560890ab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1b49cdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890abdb0 0x5560890ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1b49cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556088ffa930 con 0x5560890a0570 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1b49cdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556088ff82f0 con 0x5560890abdb0 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1b49cdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556088ff3be0 con 0x5560890a0e90 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1a259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0570 0x5560890a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1a259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0570 0x5560890a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44572/0 (socket says 172.21.15.46:44572) 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1a1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0e90 0x5560890ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1a2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890abdb0 0x5560890ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1a259c700 1 -- 172.21.15.46:0/2182297350 learned_addr learned my addr 172.21.15.46:0/2182297350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1a2d9d700 1 -- 172.21.15.46:0/2182297350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0e90 msgr2=0x5560890ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/2182297350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0e90 0x5560890ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1a2d9d700 1 -- 172.21.15.46:0/2182297350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0570 msgr2=0x5560890a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/2182297350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0570 0x5560890a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1a2d9d700 1 -- 172.21.15.46:0/2182297350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556088e75350 con 0x5560890abdb0 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/2182297350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890abdb0 0x5560890ae1a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0c9fd0 tx=0x7fc19c099660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7148f2659321be49 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.612+0000 7fc198ff9700 1 -- 172.21.15.46:0/2182297350 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c099a20 con 0x5560890abdb0 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.613+0000 7fc198ff9700 1 -- 172.21.15.46:0/2182297350 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19c09d850 con 0x5560890abdb0 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.613+0000 7fc198ff9700 1 -- 172.21.15.46:0/2182297350 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0046c0 con 0x5560890abdb0 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.613+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/2182297350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890abdb0 msgr2=0x5560890ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.613+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/2182297350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890abdb0 0x5560890ae1a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0c9fd0 tx=0x7fc19c099660 comp rx=0 tx=0).stop 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.613+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/2182297350 shutdown_connections 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.613+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/2182297350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0570 0x5560890a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.613+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/2182297350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890abdb0 0x5560890ae1a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.613+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/2182297350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0e90 0x5560890ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.613+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/2182297350 >> 172.21.15.46:0/2182297350 conn(0x5560890171c0 msgr2=0x55608909ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.613+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/2182297350 shutdown_connections 2024-03-18T20:43:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.613+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/2182297350 wait complete. 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1b49cdb80 1 -- start start 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1b49cdb80 1 Processor -- start 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1b49cdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890a0570 0x5560890ab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1b49cdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0e90 0x5560890a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1b49cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890abdb0 0x5560890a9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1b49cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556088f8fd20 con 0x5560890abdb0 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1b49cdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556088f8f9c0 con 0x5560890a0570 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1b49cdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556088e77640 con 0x5560890a0e90 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1b49cdb80 10 client.0 _ll_register_callbacks cb 0x556088dfb340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1a259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890a0570 0x5560890ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1a259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890a0570 0x5560890ab840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:58654/0 (socket says 172.21.15.46:58654) 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1a259c700 1 -- 172.21.15.46:0/3637069975 learned_addr learned my addr 172.21.15.46:0/3637069975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890abdb0 0x5560890a9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.614+0000 7fc1a1d9b700 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0e90 0x5560890a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc1a259c700 1 -- 172.21.15.46:0/3637069975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0e90 msgr2=0x5560890a8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc1a259c700 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0e90 0x5560890a8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc1a259c700 1 -- 172.21.15.46:0/3637069975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890abdb0 msgr2=0x5560890a9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc1a259c700 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890abdb0 0x5560890a9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc1a259c700 1 -- 172.21.15.46:0/3637069975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a41a2b70 con 0x5560890a0570 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc1a1d9b700 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0e90 0x5560890a8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890abdb0 0x5560890a9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc1a259c700 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890a0570 0x5560890ab840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc194003900 tx=0x7fc1940047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7758ae92e1373f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc189ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc189ffb700 1 -- 172.21.15.46:0/3637069975 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194022070 con 0x5560890a0570 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc189ffb700 1 -- 172.21.15.46:0/3637069975 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc19401d7e0 con 0x5560890a0570 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/3637069975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560890946c0 con 0x5560890a0570 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc189ffb700 1 -- 172.21.15.46:0/3637069975 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc19401d9a0 con 0x5560890a0570 2024-03-18T20:43:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc1b49cdb80 10 client.27773 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.615+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/3637069975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x556088e21b00 con 0x5560890a0570 2024-03-18T20:43:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.616+0000 7fc189ffb700 1 -- 172.21.15.46:0/3637069975 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc194033080 con 0x5560890a0570 2024-03-18T20:43:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.616+0000 7fc189ffb700 1 -- 172.21.15.46:0/3637069975 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc19402fb50 con 0x5560890a0570 2024-03-18T20:43:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.616+0000 7fc1b49cdb80 10 client.27773 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.616+0000 7fc1b49cdb80 10 client.27773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.616+0000 7fc1b49cdb80 20 client.27773 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.616+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5560890bbb70 0x5560890bdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.616+0000 7fc1b49cdb80 4 client.27773 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.616+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/3637069975 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556088fec6a0 con 0x5560890bbb70 2024-03-18T20:43:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.616+0000 7fc1a1d9b700 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5560890bbb70 0x5560890bdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.616+0000 7fc189ffb700 10 client.27773 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.616+0000 7fc1a1d9b700 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5560890bbb70 0x5560890bdf90 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7a926ffe6073ec server_cookie=dbdaa56a6fdcc24a in_seq=0 out_seq=0 2024-03-18T20:43:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc189ffb700 1 -- 172.21.15.46:0/3637069975 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556088fec6a0 con 0x5560890bbb70 2024-03-18T20:43:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc189ffb700 10 client.27773 handle_command_reply: tid=0 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc1b49cdb80 1 client.27773 shutdown 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/3637069975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890a0570 msgr2=0x5560890ab840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890a0570 0x5560890ab840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc194003900 tx=0x7fc1940047c0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/3637069975 shutdown_connections 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5560890bbb70 0x5560890bdf90 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890abdb0 0x5560890a9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc189ffb700 0 client.27773 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890a0570 0x5560890ab840 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/3637069975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0e90 0x5560890a8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/3637069975 >> 172.21.15.46:0/3637069975 conn(0x5560890171c0 msgr2=0x55608909c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/3637069975 shutdown_connections 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/3637069975 wait complete. 2024-03-18T20:43:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.617+0000 7fc1b49cdb80 20 client.27773 trim_cache size 0 max 16384 2024-03-18T20:43:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.634+0000 7fc1b49cdb80 1 Processor -- start 2024-03-18T20:43:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.634+0000 7fc1b49cdb80 1 -- start start 2024-03-18T20:43:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.634+0000 7fc1b49cdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0570 0x5560890b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1b49cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 0x5560890a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1b49cdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890abdb0 0x5560890a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1b49cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560890a1850 con 0x5560890a0e90 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1b49cdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560890a0cd0 con 0x5560890abdb0 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1b49cdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608909dd60 con 0x5560890a0570 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1a1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 0x5560890a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1a1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 0x5560890a3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44600/0 (socket says 172.21.15.46:44600) 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1a1d9b700 1 -- 172.21.15.46:0/35474694 learned_addr learned my addr 172.21.15.46:0/35474694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/35474694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890abdb0 0x5560890a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1a259c700 1 --2- 172.21.15.46:0/35474694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0570 0x5560890b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1a1d9b700 1 -- 172.21.15.46:0/35474694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0570 msgr2=0x5560890b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1a1d9b700 1 --2- 172.21.15.46:0/35474694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0570 0x5560890b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1a1d9b700 1 -- 172.21.15.46:0/35474694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890abdb0 msgr2=0x5560890a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1a1d9b700 1 --2- 172.21.15.46:0/35474694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890abdb0 0x5560890a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1a1d9b700 1 -- 172.21.15.46:0/35474694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556088fb9e40 con 0x5560890a0e90 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1a1d9b700 1 --2- 172.21.15.46:0/35474694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 0x5560890a3e10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0098e0 tx=0x7fc18c00bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ae284230f324e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.635+0000 7fc1997fa700 1 -- 172.21.15.46:0/35474694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c01d040 con 0x5560890a0e90 2024-03-18T20:43:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.636+0000 7fc1997fa700 1 -- 172.21.15.46:0/35474694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc18c028070 con 0x5560890a0e90 2024-03-18T20:43:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.636+0000 7fc1997fa700 1 -- 172.21.15.46:0/35474694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00fa60 con 0x5560890a0e90 2024-03-18T20:43:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.636+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/35474694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 msgr2=0x5560890a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.636+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/35474694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 0x5560890a3e10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0098e0 tx=0x7fc18c00bb80 comp rx=0 tx=0).stop 2024-03-18T20:43:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.636+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/35474694 shutdown_connections 2024-03-18T20:43:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.636+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/35474694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890abdb0 0x5560890a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.636+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/35474694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890a0570 0x5560890b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.636+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/35474694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 0x5560890a3e10 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.636+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/35474694 >> 172.21.15.46:0/35474694 conn(0x5560890171c0 msgr2=0x556089098620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.636+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/35474694 shutdown_connections 2024-03-18T20:43:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.636+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/35474694 wait complete. 2024-03-18T20:43:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1b49cdb80 1 -- start start 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1b49cdb80 1 Processor -- start 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1b49cdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890a0570 0x5560890b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1b49cdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 0x5560890b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1b49cdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890abdb0 0x55608910e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1b49cdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560890b1770 con 0x5560890a0e90 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1b49cdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560890b18e0 con 0x5560890a0570 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1b49cdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55608911ff30 con 0x5560890abdb0 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1b49cdb80 10 client.0 _ll_register_callbacks cb 0x7fc194004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1a1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 0x5560890b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1a1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 0x5560890b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44606/0 (socket says 172.21.15.46:44606) 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1a2d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890abdb0 0x55608910e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1a1d9b700 1 -- 172.21.15.46:0/1742926943 learned_addr learned my addr 172.21.15.46:0/1742926943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.637+0000 7fc1a259c700 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890a0570 0x5560890b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc1a2d9d700 1 -- 172.21.15.46:0/1742926943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890a0570 msgr2=0x5560890b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890a0570 0x5560890b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc1a2d9d700 1 -- 172.21.15.46:0/1742926943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 msgr2=0x5560890b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 0x5560890b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc1a2d9d700 1 -- 172.21.15.46:0/1742926943 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a410d2f0 con 0x5560890abdb0 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc1a1d9b700 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 0x5560890b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc1a2d9d700 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890abdb0 0x55608910e960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc19c004000 tx=0x7fc19c095e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=518f97880e0dbf92 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc189ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc189ffb700 1 -- 172.21.15.46:0/1742926943 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0a1e10 con 0x5560890abdb0 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/1742926943 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc19c09d850 con 0x5560890abdb0 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc189ffb700 1 -- 172.21.15.46:0/1742926943 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc19c099c80 con 0x5560890abdb0 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc189ffb700 1 -- 172.21.15.46:0/1742926943 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc19c099e40 con 0x5560890abdb0 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc1b49cdb80 10 client.27790 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.638+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/1742926943 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x556088e79c70 con 0x5560890abdb0 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.639+0000 7fc189ffb700 1 -- 172.21.15.46:0/1742926943 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc19c09d850 con 0x5560890abdb0 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.639+0000 7fc189ffb700 1 -- 172.21.15.46:0/1742926943 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc19c004cf0 con 0x5560890abdb0 2024-03-18T20:43:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.639+0000 7fc1b49cdb80 10 client.27790 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.639+0000 7fc1b49cdb80 10 client.27790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.639+0000 7fc1b49cdb80 20 client.27790 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.639+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5560891214a0 0x5560891238c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.639+0000 7fc1b49cdb80 4 client.27790 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:43:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.639+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/1742926943 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556088e42c90 con 0x5560891214a0 2024-03-18T20:43:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.639+0000 7fc1a259c700 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5560891214a0 0x5560891238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.639+0000 7fc189ffb700 10 client.27790 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.639+0000 7fc1a259c700 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5560891214a0 0x5560891238c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ed07959565fb7f server_cookie=bc511d0df1535e67 in_seq=0 out_seq=0 2024-03-18T20:43:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.665+0000 7fc189ffb700 1 -- 172.21.15.46:0/1742926943 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556088e42c90 con 0x5560891214a0 2024-03-18T20:43:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.665+0000 7fc189ffb700 10 client.27790 handle_command_reply: tid=0 2024-03-18T20:43:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.665+0000 7fc1b49cdb80 1 client.27790 shutdown 2024-03-18T20:43:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.665+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/1742926943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890abdb0 msgr2=0x55608910e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.665+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890abdb0 0x55608910e960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc19c004000 tx=0x7fc19c095e90 comp rx=0 tx=0).stop 2024-03-18T20:43:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.665+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/1742926943 shutdown_connections 2024-03-18T20:43:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.665+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5560891214a0 0x5560891238c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.665+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5560890abdb0 0x55608910e960 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc189ffb700 0 client.27790 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.665+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5560890a0570 0x5560890b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1b49cdb80 1 --2- 172.21.15.46:0/1742926943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5560890a0e90 0x5560890b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/1742926943 >> 172.21.15.46:0/1742926943 conn(0x5560890171c0 msgr2=0x5560890a3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/1742926943 shutdown_connections 2024-03-18T20:43:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1b49cdb80 1 -- 172.21.15.46:0/1742926943 wait complete. 2024-03-18T20:43:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1b49cdb80 20 client.27790 trim_cache size 0 max 16384 2024-03-18T20:43:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:43:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:43:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:43:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:43:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1a8838700 1 -- 172.21.15.46:0/284338105 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1900a1310 msgr2=0x7fc1900a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1a8838700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1900a1310 0x7fc1900a37d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc194009c60 tx=0x7fc194009260 comp rx=0 tx=0).stop 2024-03-18T20:43:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1a8838700 1 -- 172.21.15.46:0/284338105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a4105c60 msgr2=0x7fc1a419a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1a8838700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a4105c60 0x7fc1a419a8c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00efb0 tx=0x7fc19c00c3a0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1a8838700 1 -- 172.21.15.46:0/284338105 shutdown_connections 2024-03-18T20:43:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1a8838700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1900a1310 0x7fc1900a37d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1a8838700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a4105c60 0x7fc1a419a8c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1a8838700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a40ffbb0 0x7fc1a4196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1a8838700 1 --2- 172.21.15.46:0/284338105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a4100570 0x7fc1a4196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.666+0000 7fc1a8838700 1 -- 172.21.15.46:0/284338105 >> 172.21.15.46:0/284338105 conn(0x7fc1a4076d50 msgr2=0x7fc1a40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.667+0000 7fc1a8838700 1 -- 172.21.15.46:0/284338105 shutdown_connections 2024-03-18T20:43:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:25.667+0000 7fc1a8838700 1 -- 172.21.15.46:0/284338105 wait complete. 2024-03-18T20:43:25.700 INFO:tasks.workunit.client.0.smithi046.stdout: 1 250924 3.49 MB/sec execute 260 sec latency 2854.438 ms 2024-03-18T20:43:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:25 smithi073 ceph-mon[47384]: pgmap v225: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-03-18T20:43:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:25 smithi155 ceph-mon[48344]: pgmap v225: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-03-18T20:43:26.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:25 smithi046 ceph-mon[43495]: pgmap v225: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-03-18T20:43:26.700 INFO:tasks.workunit.client.0.smithi046.stdout: 1 252413 3.52 MB/sec execute 261 sec latency 15.675 ms 2024-03-18T20:43:27.700 INFO:tasks.workunit.client.0.smithi046.stdout: 1 253572 3.52 MB/sec execute 262 sec latency 154.266 ms 2024-03-18T20:43:28.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:28 smithi073 ceph-mon[47384]: pgmap v226: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-18T20:43:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:28 smithi155 ceph-mon[48344]: pgmap v226: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-18T20:43:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:28 smithi046 ceph-mon[43495]: pgmap v226: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-18T20:43:28.700 INFO:tasks.workunit.client.0.smithi046.stdout: 1 253572 3.51 MB/sec execute 263 sec latency 1154.373 ms 2024-03-18T20:43:29.700 INFO:tasks.workunit.client.0.smithi046.stdout: 1 253572 3.50 MB/sec execute 264 sec latency 2154.482 ms 2024-03-18T20:43:30.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:30 smithi073 ceph-mon[47384]: pgmap v227: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.0 MiB/s wr, 177 op/s 2024-03-18T20:43:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:30 smithi155 ceph-mon[48344]: pgmap v227: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.0 MiB/s wr, 177 op/s 2024-03-18T20:43:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:30 smithi046 ceph-mon[43495]: pgmap v227: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.0 MiB/s wr, 177 op/s 2024-03-18T20:43:30.700 INFO:tasks.workunit.client.0.smithi046.stdout: 1 255510 3.53 MB/sec execute 265 sec latency 2307.394 ms 2024-03-18T20:43:31.700 INFO:tasks.workunit.client.0.smithi046.stdout: 1 256672 3.53 MB/sec execute 266 sec latency 14.807 ms 2024-03-18T20:43:32.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:32 smithi073 ceph-mon[47384]: pgmap v228: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-18T20:43:32.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:32 smithi155 ceph-mon[48344]: pgmap v228: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-18T20:43:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:32 smithi046 ceph-mon[43495]: pgmap v228: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-03-18T20:43:32.700 INFO:tasks.workunit.client.0.smithi046.stdout: 1 257113 3.53 MB/sec execute 267 sec latency 794.439 ms 2024-03-18T20:43:33.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 257113 3.52 MB/sec execute 268 sec latency 1794.546 ms 2024-03-18T20:43:34.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:34 smithi073 ceph-mon[47384]: pgmap v229: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-03-18T20:43:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:34 smithi155 ceph-mon[48344]: pgmap v229: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-03-18T20:43:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:34 smithi046 ceph-mon[43495]: pgmap v229: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-03-18T20:43:34.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 257113 3.50 MB/sec execute 269 sec latency 2794.639 ms 2024-03-18T20:43:35.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 257877 3.50 MB/sec execute 270 sec latency 3421.560 ms 2024-03-18T20:43:36.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:35 smithi073 ceph-mon[47384]: pgmap v230: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-18T20:43:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:35 smithi155 ceph-mon[48344]: pgmap v230: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-18T20:43:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:35 smithi046 ceph-mon[43495]: pgmap v230: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-18T20:43:36.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 259424 3.53 MB/sec execute 271 sec latency 17.956 ms 2024-03-18T20:43:37.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 260661 3.53 MB/sec execute 272 sec latency 97.185 ms 2024-03-18T20:43:38.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:38 smithi073 ceph-mon[47384]: pgmap v231: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-18T20:43:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:38 smithi155 ceph-mon[48344]: pgmap v231: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-18T20:43:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:38 smithi046 ceph-mon[43495]: pgmap v231: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-18T20:43:38.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 260661 3.52 MB/sec execute 273 sec latency 1097.235 ms 2024-03-18T20:43:38.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:43:38.704 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:43:38.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.797+0000 7f37c24ec700 1 -- 172.21.15.46:0/3228880247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37bc068aa0 msgr2=0x7f37bc10dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.797+0000 7f37c24ec700 1 --2- 172.21.15.46:0/3228880247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37bc068aa0 0x7f37bc10dab0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f37b400b0a0 tx=0x7f37b400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.797+0000 7f37c24ec700 1 -- 172.21.15.46:0/3228880247 shutdown_connections 2024-03-18T20:43:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.797+0000 7f37c24ec700 1 --2- 172.21.15.46:0/3228880247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37bc068aa0 0x7f37bc10dab0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.797+0000 7f37c24ec700 1 --2- 172.21.15.46:0/3228880247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37bc106e10 0x7f37bc1071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.797+0000 7f37c24ec700 1 --2- 172.21.15.46:0/3228880247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37bc0680e0 0x7f37bc068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.797+0000 7f37c24ec700 1 -- 172.21.15.46:0/3228880247 >> 172.21.15.46:0/3228880247 conn(0x7f37bc076d50 msgr2=0x7f37bc077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.797+0000 7f37c24ec700 1 -- 172.21.15.46:0/3228880247 shutdown_connections 2024-03-18T20:43:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.797+0000 7f37c24ec700 1 -- 172.21.15.46:0/3228880247 wait complete. 2024-03-18T20:43:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.798+0000 7f37c24ec700 1 Processor -- start 2024-03-18T20:43:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.798+0000 7f37c24ec700 1 -- start start 2024-03-18T20:43:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.798+0000 7f37c24ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37bc0680e0 0x7f37bc1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.798+0000 7f37c24ec700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37bc068aa0 0x7f37bc199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.798+0000 7f37c24ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37bc106e10 0x7f37bc19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.798+0000 7f37c24ec700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc19e2b0 con 0x7f37bc106e10 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.798+0000 7f37c24ec700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc19e420 con 0x7f37bc0680e0 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.798+0000 7f37c24ec700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc19e5e0 con 0x7f37bc068aa0 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.798+0000 7f37c0a89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37bc106e10 0x7f37bc19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.798+0000 7f37bbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37bc0680e0 0x7f37bc1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.798+0000 7f37c0a89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37bc106e10 0x7f37bc19dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59544/0 (socket says 172.21.15.46:59544) 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.798+0000 7f37c0a89700 1 -- 172.21.15.46:0/2880188285 learned_addr learned my addr 172.21.15.46:0/2880188285 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37bb7fe700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37bc068aa0 0x7f37bc199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37c0a89700 1 -- 172.21.15.46:0/2880188285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37bc068aa0 msgr2=0x7f37bc199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37c0a89700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37bc068aa0 0x7f37bc199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37c0a89700 1 -- 172.21.15.46:0/2880188285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37bc0680e0 msgr2=0x7f37bc1993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37c0a89700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37bc0680e0 0x7f37bc1993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37c0a89700 1 -- 172.21.15.46:0/2880188285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b4009d00 con 0x7f37bc106e10 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37bbfff700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37bc0680e0 0x7f37bc1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37c0a89700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37bc106e10 0x7f37bc19dc70 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f37b400b070 tx=0x7f37b4007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37b97fa700 1 -- 172.21.15.46:0/2880188285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b400e040 con 0x7f37bc106e10 2024-03-18T20:43:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37b97fa700 1 -- 172.21.15.46:0/2880188285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b401d470 con 0x7f37bc106e10 2024-03-18T20:43:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37c24ec700 1 -- 172.21.15.46:0/2880188285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc19e860 con 0x7f37bc106e10 2024-03-18T20:43:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37b97fa700 1 -- 172.21.15.46:0/2880188285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b40052a0 con 0x7f37bc106e10 2024-03-18T20:43:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.799+0000 7f37c24ec700 1 -- 172.21.15.46:0/2880188285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc1a6100 con 0x7f37bc106e10 2024-03-18T20:43:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.801+0000 7f37b97fa700 1 -- 172.21.15.46:0/2880188285 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f37b401d870 con 0x7f37bc106e10 2024-03-18T20:43:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.802+0000 7f37b97fa700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37a40a1290 0x7f37a40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.802+0000 7f37b97fa700 1 -- 172.21.15.46:0/2880188285 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f37b40ce0e0 con 0x7f37bc106e10 2024-03-18T20:43:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.802+0000 7f37bbfff700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37a40a1290 0x7f37a40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.803+0000 7f37bbfff700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37a40a1290 0x7f37a40a3750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f37ac011490 tx=0x7f37ac009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.804+0000 7f37ce681b80 1 Processor -- start 2024-03-18T20:43:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.804+0000 7f37ce681b80 1 -- start start 2024-03-18T20:43:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.804+0000 7f37ce681b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 0x55b7f3cf3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.804+0000 7f37ce681b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.804+0000 7f37ce681b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cfedb0 0x55b7f3d011a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.804+0000 7f37ce681b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3c4d930 con 0x55b7f3cf3570 2024-03-18T20:43:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.804+0000 7f37ce681b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3c4b2f0 con 0x55b7f3cfedb0 2024-03-18T20:43:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.804+0000 7f37ce681b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3c46be0 con 0x55b7f3cf3e90 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.804+0000 7f37bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 0x55b7f3cf3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.804+0000 7f37bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 0x55b7f3cf3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59550/0 (socket says 172.21.15.46:59550) 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.804+0000 7f37bbfff700 1 -- 172.21.15.46:0/2026651590 learned_addr learned my addr 172.21.15.46:0/2026651590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.804+0000 7f37bb7fe700 1 --2- 172.21.15.46:0/2026651590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.805+0000 7f37c0a89700 1 --2- 172.21.15.46:0/2026651590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cfedb0 0x55b7f3d011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.805+0000 7f37bbfff700 1 -- 172.21.15.46:0/2026651590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cf3e90 msgr2=0x55b7f3cfe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.805+0000 7f37bbfff700 1 --2- 172.21.15.46:0/2026651590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfe870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.805+0000 7f37bbfff700 1 -- 172.21.15.46:0/2026651590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cfedb0 msgr2=0x55b7f3d011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.805+0000 7f37bbfff700 1 --2- 172.21.15.46:0/2026651590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cfedb0 0x55b7f3d011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.805+0000 7f37bbfff700 1 -- 172.21.15.46:0/2026651590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7f3ac8350 con 0x55b7f3cf3570 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.805+0000 7f37bbfff700 1 --2- 172.21.15.46:0/2026651590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 0x55b7f3cf3950 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0043c0 tx=0x7f37ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db74dd783d278fc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.805+0000 7f37a27fc700 1 -- 172.21.15.46:0/2026651590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac003d10 con 0x55b7f3cf3570 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.805+0000 7f37a27fc700 1 -- 172.21.15.46:0/2026651590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37ac003e70 con 0x55b7f3cf3570 2024-03-18T20:43:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.805+0000 7f37a27fc700 1 -- 172.21.15.46:0/2026651590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac01e7d0 con 0x55b7f3cf3570 2024-03-18T20:43:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.806+0000 7f37ce681b80 1 -- 172.21.15.46:0/2026651590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 msgr2=0x55b7f3cf3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.806+0000 7f37ce681b80 1 --2- 172.21.15.46:0/2026651590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 0x55b7f3cf3950 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0043c0 tx=0x7f37ac014000 comp rx=0 tx=0).stop 2024-03-18T20:43:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.806+0000 7f37ce681b80 1 -- 172.21.15.46:0/2026651590 shutdown_connections 2024-03-18T20:43:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.806+0000 7f37ce681b80 1 --2- 172.21.15.46:0/2026651590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 0x55b7f3cf3950 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.806+0000 7f37ce681b80 1 --2- 172.21.15.46:0/2026651590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cfedb0 0x55b7f3d011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.806+0000 7f37ce681b80 1 --2- 172.21.15.46:0/2026651590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.806+0000 7f37ce681b80 1 -- 172.21.15.46:0/2026651590 >> 172.21.15.46:0/2026651590 conn(0x55b7f3c6a1c0 msgr2=0x55b7f3cefa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.806+0000 7f37ce681b80 1 -- 172.21.15.46:0/2026651590 shutdown_connections 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.806+0000 7f37ce681b80 1 -- 172.21.15.46:0/2026651590 wait complete. 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.807+0000 7f37ce681b80 1 -- start start 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.807+0000 7f37ce681b80 1 Processor -- start 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.807+0000 7f37ce681b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3570 0x55b7f3cfe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.807+0000 7f37ce681b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.807+0000 7f37ce681b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cfedb0 0x55b7f3cfc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.807+0000 7f37ce681b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3be2d20 con 0x55b7f3cfedb0 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.807+0000 7f37ce681b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3be29c0 con 0x55b7f3cf3570 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.807+0000 7f37ce681b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3aca640 con 0x55b7f3cf3e90 2024-03-18T20:43:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.807+0000 7f37ce681b80 10 client.0 _ll_register_callbacks cb 0x55b7f3a4e340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.807+0000 7f37c0a89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cfedb0 0x55b7f3cfc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f37bbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3570 0x55b7f3cfe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f37c0a89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cfedb0 0x55b7f3cfc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59566/0 (socket says 172.21.15.46:59566) 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f37c0a89700 1 -- 172.21.15.46:0/654867725 learned_addr learned my addr 172.21.15.46:0/654867725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f37bb7fe700 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f37bbfff700 1 -- 172.21.15.46:0/654867725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cf3e90 msgr2=0x55b7f3cfbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f37bbfff700 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f37bbfff700 1 -- 172.21.15.46:0/654867725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cfedb0 msgr2=0x55b7f3cfc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f37bbfff700 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cfedb0 0x55b7f3cfc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f37bbfff700 1 -- 172.21.15.46:0/654867725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac017070 con 0x55b7f3cf3570 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f37c0a89700 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cfedb0 0x55b7f3cfc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f37bbfff700 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3570 0x55b7f3cfe800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f37ac004390 tx=0x7f37ac026630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a94b9e5ca1ecb67 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f379b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f379b7fe700 1 -- 172.21.15.46:0/654867725 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac02b070 con 0x55b7f3cf3570 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f379b7fe700 1 -- 172.21.15.46:0/654867725 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f37ac026a30 con 0x55b7f3cf3570 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.808+0000 7f37ce681b80 1 -- 172.21.15.46:0/654867725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7f3ce76c0 con 0x55b7f3cf3570 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.809+0000 7f379b7fe700 1 -- 172.21.15.46:0/654867725 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37ac026bf0 con 0x55b7f3cf3570 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.809+0000 7f37ce681b80 10 client.27791 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.809+0000 7f37ce681b80 1 -- 172.21.15.46:0/654867725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55b7f3a74b00 con 0x55b7f3cf3570 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.809+0000 7f379b7fe700 1 -- 172.21.15.46:0/654867725 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7f3ce76c0 con 0x55b7f3cf3570 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.809+0000 7f379b7fe700 1 -- 172.21.15.46:0/654867725 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f37ac024020 con 0x55b7f3cf3570 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.809+0000 7f37ce681b80 10 client.27791 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.809+0000 7f37ce681b80 10 client.27791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.809+0000 7f37ce681b80 20 client.27791 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.809+0000 7f37ce681b80 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b7f3d0e950 0x55b7f3d10d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.809+0000 7f37ce681b80 4 client.27791 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.809+0000 7f37ce681b80 1 -- 172.21.15.46:0/654867725 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7f3c3f6a0 con 0x55b7f3d0e950 2024-03-18T20:43:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.809+0000 7f37bb7fe700 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b7f3d0e950 0x55b7f3d10d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.810+0000 7f379b7fe700 10 client.27791 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.810+0000 7f37bb7fe700 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b7f3d0e950 0x55b7f3d10d70 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89dec8ffdb4372f7 server_cookie=f672cc4cfbbe4430 in_seq=0 out_seq=0 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.810+0000 7f379b7fe700 1 -- 172.21.15.46:0/654867725 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b7f3c3f6a0 con 0x55b7f3d0e950 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.810+0000 7f379b7fe700 10 client.27791 handle_command_reply: tid=0 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.810+0000 7f37ce681b80 1 client.27791 shutdown 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.811+0000 7f37ce681b80 1 -- 172.21.15.46:0/654867725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3570 msgr2=0x55b7f3cfe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.811+0000 7f37ce681b80 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3570 0x55b7f3cfe800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f37ac004390 tx=0x7f37ac026630 comp rx=0 tx=0).stop 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.811+0000 7f37ce681b80 1 -- 172.21.15.46:0/654867725 shutdown_connections 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.811+0000 7f37ce681b80 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b7f3d0e950 0x55b7f3d10d70 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.811+0000 7f37ce681b80 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cfedb0 0x55b7f3cfc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.811+0000 7f379b7fe700 0 client.27791 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.811+0000 7f37ce681b80 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3570 0x55b7f3cfe800 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.811+0000 7f37ce681b80 1 --2- 172.21.15.46:0/654867725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.811+0000 7f37ce681b80 1 -- 172.21.15.46:0/654867725 >> 172.21.15.46:0/654867725 conn(0x55b7f3c6a1c0 msgr2=0x55b7f3cef650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.811+0000 7f37ce681b80 1 -- 172.21.15.46:0/654867725 shutdown_connections 2024-03-18T20:43:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.811+0000 7f37ce681b80 1 -- 172.21.15.46:0/654867725 wait complete. 2024-03-18T20:43:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.811+0000 7f37ce681b80 20 client.27791 trim_cache size 0 max 16384 2024-03-18T20:43:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37ce681b80 1 Processor -- start 2024-03-18T20:43:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37ce681b80 1 -- start start 2024-03-18T20:43:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37ce681b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3570 0x55b7f3d05a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37ce681b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37ce681b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cfedb0 0x55b7f3cfb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37ce681b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3cf43a0 con 0x55b7f3cf3e90 2024-03-18T20:43:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37ce681b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3cfbe50 con 0x55b7f3cf3570 2024-03-18T20:43:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37ce681b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3d05f80 con 0x55b7f3cfedb0 2024-03-18T20:43:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37bb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37bb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59574/0 (socket says 172.21.15.46:59574) 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37bb7fe700 1 -- 172.21.15.46:0/3935355637 learned_addr learned my addr 172.21.15.46:0/3935355637 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37bbfff700 1 --2- 172.21.15.46:0/3935355637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3570 0x55b7f3d05a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.843+0000 7f37c0a89700 1 --2- 172.21.15.46:0/3935355637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cfedb0 0x55b7f3cfb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.844+0000 7f37bb7fe700 1 -- 172.21.15.46:0/3935355637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cfedb0 msgr2=0x55b7f3cfb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.844+0000 7f37bb7fe700 1 --2- 172.21.15.46:0/3935355637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cfedb0 0x55b7f3cfb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.844+0000 7f37bb7fe700 1 -- 172.21.15.46:0/3935355637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3570 msgr2=0x55b7f3d05a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.844+0000 7f37bb7fe700 1 --2- 172.21.15.46:0/3935355637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3570 0x55b7f3d05a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.844+0000 7f37bb7fe700 1 -- 172.21.15.46:0/3935355637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7f3c0ce40 con 0x55b7f3cf3e90 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.844+0000 7f37bb7fe700 1 --2- 172.21.15.46:0/3935355637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfb3d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f37b001fbf0 tx=0x7f37b000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad673573f06f13a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.844+0000 7f37a2ffd700 1 -- 172.21.15.46:0/3935355637 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0016070 con 0x55b7f3cf3e90 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.844+0000 7f37a2ffd700 1 -- 172.21.15.46:0/3935355637 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b0017040 con 0x55b7f3cf3e90 2024-03-18T20:43:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.844+0000 7f37a2ffd700 1 -- 172.21.15.46:0/3935355637 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b00053a0 con 0x55b7f3cf3e90 2024-03-18T20:43:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.845+0000 7f37ce681b80 1 -- 172.21.15.46:0/3935355637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3e90 msgr2=0x55b7f3cfb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.845+0000 7f37ce681b80 1 --2- 172.21.15.46:0/3935355637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfb3d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f37b001fbf0 tx=0x7f37b000b410 comp rx=0 tx=0).stop 2024-03-18T20:43:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.845+0000 7f37ce681b80 1 -- 172.21.15.46:0/3935355637 shutdown_connections 2024-03-18T20:43:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.845+0000 7f37ce681b80 1 --2- 172.21.15.46:0/3935355637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cfedb0 0x55b7f3cfb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.845+0000 7f37ce681b80 1 --2- 172.21.15.46:0/3935355637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3570 0x55b7f3d05a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.845+0000 7f37ce681b80 1 --2- 172.21.15.46:0/3935355637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3e90 0x55b7f3cfb3d0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.845+0000 7f37ce681b80 1 -- 172.21.15.46:0/3935355637 >> 172.21.15.46:0/3935355637 conn(0x55b7f3c6a1c0 msgr2=0x55b7f3cf6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.845+0000 7f37ce681b80 1 -- 172.21.15.46:0/3935355637 shutdown_connections 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.845+0000 7f37ce681b80 1 -- 172.21.15.46:0/3935355637 wait complete. 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.845+0000 7f37ce681b80 1 -- start start 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37ce681b80 1 Processor -- start 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37ce681b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 0x55b7f3d05180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37ce681b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3e90 0x55b7f3d056c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37ce681b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cfedb0 0x55b7f3d03a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37ce681b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3d03ff0 con 0x55b7f3cf3570 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37ce681b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3d04160 con 0x55b7f3cf3e90 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37ce681b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7f3d04320 con 0x55b7f3cfedb0 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37ce681b80 10 client.0 _ll_register_callbacks cb 0x7f37b401df50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 0x55b7f3d05180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 0x55b7f3d05180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59580/0 (socket says 172.21.15.46:59580) 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37bbfff700 1 -- 172.21.15.46:0/1007268041 learned_addr learned my addr 172.21.15.46:0/1007268041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37c0a89700 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cfedb0 0x55b7f3d03a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37bb7fe700 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3e90 0x55b7f3d056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37bbfff700 1 -- 172.21.15.46:0/1007268041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cfedb0 msgr2=0x55b7f3d03a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37bbfff700 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cfedb0 0x55b7f3d03a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37bbfff700 1 -- 172.21.15.46:0/1007268041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3e90 msgr2=0x55b7f3d056c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37bbfff700 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3e90 0x55b7f3d056c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37bbfff700 1 -- 172.21.15.46:0/1007268041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7f3accc70 con 0x55b7f3cf3570 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37bb7fe700 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3e90 0x55b7f3d056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.846+0000 7f37c0a89700 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cfedb0 0x55b7f3d03a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f37bbfff700 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 0x55b7f3d05180 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f37ac026fd0 tx=0x7f37ac00ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e7e5de67ac19b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f379b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f379b7fe700 1 -- 172.21.15.46:0/1007268041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac02b070 con 0x55b7f3cf3570 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f379b7fe700 1 -- 172.21.15.46:0/1007268041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f37ac0049a0 con 0x55b7f3cf3570 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f37ce681b80 1 -- 172.21.15.46:0/1007268041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37ac003e70 con 0x55b7f3cf3570 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f379b7fe700 1 -- 172.21.15.46:0/1007268041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37ac004e70 con 0x55b7f3cf3570 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f37ce681b80 10 client.18123 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f37ce681b80 1 -- 172.21.15.46:0/1007268041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55b7f3a72fd0 con 0x55b7f3cf3570 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f379b7fe700 1 -- 172.21.15.46:0/1007268041 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37ac003e70 con 0x55b7f3cf3570 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f379b7fe700 1 -- 172.21.15.46:0/1007268041 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f37ac024050 con 0x55b7f3cf3570 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f37ce681b80 10 client.18123 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f37ce681b80 10 client.18123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f37ce681b80 20 client.18123 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.847+0000 7f37ce681b80 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b7f3d0e950 0x55b7f3d65600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.848+0000 7f37ce681b80 4 client.18123 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:43:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.848+0000 7f37ce681b80 1 -- 172.21.15.46:0/1007268041 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b7f3a95c90 con 0x55b7f3d0e950 2024-03-18T20:43:38.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.848+0000 7f37bb7fe700 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b7f3d0e950 0x55b7f3d65600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.848+0000 7f379b7fe700 10 client.18123 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:38.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.848+0000 7f37bb7fe700 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b7f3d0e950 0x55b7f3d65600 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a4e0da9eb904a8d server_cookie=81c7de15ca6d8361 in_seq=0 out_seq=0 2024-03-18T20:43:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.877+0000 7f379b7fe700 1 -- 172.21.15.46:0/1007268041 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b7f3a95c90 con 0x55b7f3d0e950 2024-03-18T20:43:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.877+0000 7f379b7fe700 10 client.18123 handle_command_reply: tid=0 2024-03-18T20:43:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.877+0000 7f37ce681b80 1 client.18123 shutdown 2024-03-18T20:43:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.877+0000 7f37ce681b80 1 -- 172.21.15.46:0/1007268041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 msgr2=0x55b7f3d05180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.877+0000 7f37ce681b80 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 0x55b7f3d05180 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f37ac026fd0 tx=0x7f37ac00ffb0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37ce681b80 1 -- 172.21.15.46:0/1007268041 shutdown_connections 2024-03-18T20:43:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37ce681b80 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b7f3d0e950 0x55b7f3d65600 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37ce681b80 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b7f3cf3570 0x55b7f3d05180 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37ce681b80 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b7f3cfedb0 0x55b7f3d03a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f379b7fe700 0 client.18123 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37ce681b80 1 --2- 172.21.15.46:0/1007268041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b7f3cf3e90 0x55b7f3d056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37ce681b80 1 -- 172.21.15.46:0/1007268041 >> 172.21.15.46:0/1007268041 conn(0x55b7f3c6a1c0 msgr2=0x55b7f3cf5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37ce681b80 1 -- 172.21.15.46:0/1007268041 shutdown_connections 2024-03-18T20:43:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37ce681b80 1 -- 172.21.15.46:0/1007268041 wait complete. 2024-03-18T20:43:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37ce681b80 20 client.18123 trim_cache size 0 max 16384 2024-03-18T20:43:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:43:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:43:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:43:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:43:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37c24ec700 1 -- 172.21.15.46:0/2880188285 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37a40a1290 msgr2=0x7f37a40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37c24ec700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37a40a1290 0x7f37a40a3750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f37ac011490 tx=0x7f37ac009260 comp rx=0 tx=0).stop 2024-03-18T20:43:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37c24ec700 1 -- 172.21.15.46:0/2880188285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37bc106e10 msgr2=0x7f37bc19dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.878+0000 7f37c24ec700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37bc106e10 0x7f37bc19dc70 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f37b400b070 tx=0x7f37b4007930 comp rx=0 tx=0).stop 2024-03-18T20:43:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.879+0000 7f37c24ec700 1 -- 172.21.15.46:0/2880188285 shutdown_connections 2024-03-18T20:43:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.879+0000 7f37c24ec700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37a40a1290 0x7f37a40a3750 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.879+0000 7f37c24ec700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37bc106e10 0x7f37bc19dc70 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.879+0000 7f37c24ec700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37bc0680e0 0x7f37bc1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.879+0000 7f37c24ec700 1 --2- 172.21.15.46:0/2880188285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37bc068aa0 0x7f37bc199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.879+0000 7f37c24ec700 1 -- 172.21.15.46:0/2880188285 >> 172.21.15.46:0/2880188285 conn(0x7f37bc076d50 msgr2=0x7f37bc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.879+0000 7f37c24ec700 1 -- 172.21.15.46:0/2880188285 shutdown_connections 2024-03-18T20:43:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:38.879+0000 7f37c24ec700 1 -- 172.21.15.46:0/2880188285 wait complete. 2024-03-18T20:43:38.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.886+0000 7f02018af700 1 -- 172.21.15.46:0/2885853112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01fc073160 msgr2=0x7f01fc073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.886+0000 7f02018af700 1 --2- 172.21.15.46:0/2885853112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01fc073160 0x7f01fc073540 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009990 tx=0x7f01ec009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.886+0000 7f02018af700 1 -- 172.21.15.46:0/2885853112 shutdown_connections 2024-03-18T20:43:38.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.886+0000 7f02018af700 1 --2- 172.21.15.46:0/2885853112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01fc073160 0x7f01fc073540 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.886+0000 7f02018af700 1 --2- 172.21.15.46:0/2885853112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01fc10d5a0 0x7f01fc10f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.886+0000 7f02018af700 1 --2- 172.21.15.46:0/2885853112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc073a80 0x7f01fc10d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.886+0000 7f02018af700 1 -- 172.21.15.46:0/2885853112 >> 172.21.15.46:0/2885853112 conn(0x7f01fc0fc7f0 msgr2=0x7f01fc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:38.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.886+0000 7f02018af700 1 -- 172.21.15.46:0/2885853112 shutdown_connections 2024-03-18T20:43:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.886+0000 7f02018af700 1 -- 172.21.15.46:0/2885853112 wait complete. 2024-03-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.887+0000 7f02018af700 1 Processor -- start 2024-03-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.887+0000 7f02018af700 1 -- start start 2024-03-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.887+0000 7f02018af700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc073160 0x7f01fc1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.887+0000 7f02018af700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01fc073a80 0x7f01fc199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.887+0000 7f02018af700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01fc10d5a0 0x7f01fc19dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.887+0000 7f02018af700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc19e310 con 0x7f01fc10d5a0 2024-03-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.887+0000 7f02018af700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc19e480 con 0x7f01fc073a80 2024-03-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.887+0000 7f02018af700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc19e640 con 0x7f01fc073160 2024-03-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.888+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01fc10d5a0 0x7f01fc19dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.888+0000 7f01fa7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01fc073a80 0x7f01fc199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.888+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01fc10d5a0 0x7f01fc19dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59602/0 (socket says 172.21.15.46:59602) 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.888+0000 7f01fb7fe700 1 -- 172.21.15.46:0/4175236694 learned_addr learned my addr 172.21.15.46:0/4175236694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.888+0000 7f01faffd700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc073160 0x7f01fc1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.888+0000 7f01fa7fc700 1 -- 172.21.15.46:0/4175236694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc073160 msgr2=0x7f01fc1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.888+0000 7f01fa7fc700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc073160 0x7f01fc1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.888+0000 7f01fa7fc700 1 -- 172.21.15.46:0/4175236694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01fc10d5a0 msgr2=0x7f01fc19dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.888+0000 7f01fa7fc700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01fc10d5a0 0x7f01fc19dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.888+0000 7f01fa7fc700 1 -- 172.21.15.46:0/4175236694 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0095f0 con 0x7f01fc073a80 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.888+0000 7f01fa7fc700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01fc073a80 0x7f01fc199930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f01f000e9f0 tx=0x7f01f000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.888+0000 7f02008ad700 1 -- 172.21.15.46:0/4175236694 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0004220 con 0x7f01fc073a80 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.889+0000 7f02018af700 1 -- 172.21.15.46:0/4175236694 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01fc19e920 con 0x7f01fc073a80 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.889+0000 7f02008ad700 1 -- 172.21.15.46:0/4175236694 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01f0004cf0 con 0x7f01fc073a80 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.889+0000 7f02008ad700 1 -- 172.21.15.46:0/4175236694 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f000ce70 con 0x7f01fc073a80 2024-03-18T20:43:38.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.889+0000 7f02018af700 1 -- 172.21.15.46:0/4175236694 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01fc1a6160 con 0x7f01fc073a80 2024-03-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.891+0000 7f02008ad700 1 -- 172.21.15.46:0/4175236694 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f01f0004750 con 0x7f01fc073a80 2024-03-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.892+0000 7f02008ad700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01e40a1310 0x7f01e40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.892+0000 7f01faffd700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01e40a1310 0x7f01e40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.892+0000 7f02008ad700 1 -- 172.21.15.46:0/4175236694 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f01f0014070 con 0x7f01fc073a80 2024-03-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.892+0000 7f020da44b80 1 Processor -- start 2024-03-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f020da44b80 1 -- start start 2024-03-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f020da44b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d0a60 0x55af442d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f020da44b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442d1380 0x55af442dbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f020da44b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442dc220 0x55af442de610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f020da44b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4422a990 con 0x55af442d1380 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f020da44b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af44228350 con 0x55af442dc220 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f020da44b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af44223c40 con 0x55af442d0a60 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f01faffd700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01e40a1310 0x7f01e40a37d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0098e0 tx=0x7f01ec005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f01fa7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442d1380 0x55af442dbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f01fa7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442d1380 0x55af442dbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59604/0 (socket says 172.21.15.46:59604) 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f01fa7fc700 1 -- 172.21.15.46:0/3060217870 learned_addr learned my addr 172.21.15.46:0/3060217870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f01faffd700 1 --2- 172.21.15.46:0/3060217870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d0a60 0x55af442d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f01fb7fe700 1 --2- 172.21.15.46:0/3060217870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442dc220 0x55af442de610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f01fa7fc700 1 -- 172.21.15.46:0/3060217870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d0a60 msgr2=0x55af442d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f01fa7fc700 1 --2- 172.21.15.46:0/3060217870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d0a60 0x55af442d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f01fa7fc700 1 -- 172.21.15.46:0/3060217870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442dc220 msgr2=0x55af442de610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.893+0000 7f01fa7fc700 1 --2- 172.21.15.46:0/3060217870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442dc220 0x55af442de610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.894+0000 7f01fa7fc700 1 -- 172.21.15.46:0/3060217870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af440a53b0 con 0x55af442d1380 2024-03-18T20:43:38.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.894+0000 7f01fb7fe700 1 --2- 172.21.15.46:0/3060217870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442dc220 0x55af442de610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.894+0000 7f01fa7fc700 1 --2- 172.21.15.46:0/3060217870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442d1380 0x55af442dbce0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f01f00966a0 tx=0x7f01f0099690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9c535c97db8098 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.894+0000 7f01e1ffb700 1 -- 172.21.15.46:0/3060217870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0099c30 con 0x55af442d1380 2024-03-18T20:43:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.894+0000 7f01e1ffb700 1 -- 172.21.15.46:0/3060217870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01f009da60 con 0x55af442d1380 2024-03-18T20:43:38.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.894+0000 7f01e1ffb700 1 -- 172.21.15.46:0/3060217870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f009a0f0 con 0x55af442d1380 2024-03-18T20:43:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.895+0000 7f020da44b80 1 -- 172.21.15.46:0/3060217870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442d1380 msgr2=0x55af442dbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.895+0000 7f020da44b80 1 --2- 172.21.15.46:0/3060217870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442d1380 0x55af442dbce0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f01f00966a0 tx=0x7f01f0099690 comp rx=0 tx=0).stop 2024-03-18T20:43:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.895+0000 7f020da44b80 1 -- 172.21.15.46:0/3060217870 shutdown_connections 2024-03-18T20:43:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.895+0000 7f020da44b80 1 --2- 172.21.15.46:0/3060217870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442dc220 0x55af442de610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.895+0000 7f020da44b80 1 --2- 172.21.15.46:0/3060217870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d0a60 0x55af442d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.895+0000 7f020da44b80 1 --2- 172.21.15.46:0/3060217870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442d1380 0x55af442dbce0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.895+0000 7f020da44b80 1 -- 172.21.15.46:0/3060217870 >> 172.21.15.46:0/3060217870 conn(0x55af44247220 msgr2=0x55af442cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.895+0000 7f020da44b80 1 -- 172.21.15.46:0/3060217870 shutdown_connections 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.895+0000 7f020da44b80 1 -- 172.21.15.46:0/3060217870 wait complete. 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.895+0000 7f020da44b80 1 -- start start 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.896+0000 7f020da44b80 1 Processor -- start 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.896+0000 7f020da44b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d0a60 0x55af442d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.896+0000 7f020da44b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d1380 0x55af442d95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.896+0000 7f020da44b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af442d9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.896+0000 7f020da44b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af441bfd80 con 0x55af442dc220 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.896+0000 7f020da44b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af441bfa20 con 0x55af442d0a60 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.896+0000 7f020da44b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af440a76a0 con 0x55af442d1380 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.896+0000 7f020da44b80 10 client.0 _ll_register_callbacks cb 0x55af4402b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.896+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af442d9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af442d9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59610/0 (socket says 172.21.15.46:59610) 2024-03-18T20:43:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01fb7fe700 1 -- 172.21.15.46:0/3757843299 learned_addr learned my addr 172.21.15.46:0/3757843299 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01fa7fc700 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d1380 0x55af442d95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01faffd700 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d0a60 0x55af442d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01fb7fe700 1 -- 172.21.15.46:0/3757843299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d1380 msgr2=0x55af442d95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01fb7fe700 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d1380 0x55af442d95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01fb7fe700 1 -- 172.21.15.46:0/3757843299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d0a60 msgr2=0x55af442d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01fb7fe700 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d0a60 0x55af442d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01fb7fe700 1 -- 172.21.15.46:0/3757843299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f4009a50 con 0x55af442dc220 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01fa7fc700 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d1380 0x55af442d95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01fb7fe700 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af442d9b10 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f01f4007f60 tx=0x7f01f4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3700500b126c2a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01daffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01daffd700 1 -- 172.21.15.46:0/3757843299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40139b0 con 0x55af442dc220 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f020da44b80 1 -- 172.21.15.46:0/3757843299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af442c4ad0 con 0x55af442dc220 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01daffd700 1 -- 172.21.15.46:0/3757843299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f01f4013b10 con 0x55af442dc220 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f01daffd700 1 -- 172.21.15.46:0/3757843299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01f4013d10 con 0x55af442dc220 2024-03-18T20:43:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f020da44b80 10 client.18135 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.897+0000 7f020da44b80 1 -- 172.21.15.46:0/3757843299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f01fc1a6160 con 0x55af442dc220 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.898+0000 7f01daffd700 1 -- 172.21.15.46:0/3757843299 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55af442c4ad0 con 0x55af442dc220 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.898+0000 7f01daffd700 1 -- 172.21.15.46:0/3757843299 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f01f401f410 con 0x55af442dc220 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.898+0000 7f020da44b80 10 client.18135 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.898+0000 7f020da44b80 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.898+0000 7f020da44b80 20 client.18135 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.898+0000 7f020da44b80 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55af442ebf00 0x55af442ee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.898+0000 7f020da44b80 4 client.18135 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.898+0000 7f020da44b80 1 -- 172.21.15.46:0/3757843299 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af4421c700 con 0x55af442ebf00 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.898+0000 7f01faffd700 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55af442ebf00 0x55af442ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.899+0000 7f01faffd700 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55af442ebf00 0x55af442ee320 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc804de524f5c363 server_cookie=73303e3efbcdf99c in_seq=0 out_seq=0 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.899+0000 7f01daffd700 10 client.18135 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.899+0000 7f01daffd700 1 -- 172.21.15.46:0/3757843299 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55af4421c700 con 0x55af442ebf00 2024-03-18T20:43:38.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.899+0000 7f01daffd700 10 client.18135 handle_command_reply: tid=0 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.899+0000 7f020da44b80 1 client.18135 shutdown 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.900+0000 7f020da44b80 1 -- 172.21.15.46:0/3757843299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 msgr2=0x55af442d9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.900+0000 7f020da44b80 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af442d9b10 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f01f4007f60 tx=0x7f01f4007f90 comp rx=0 tx=0).stop 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.900+0000 7f020da44b80 1 -- 172.21.15.46:0/3757843299 shutdown_connections 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.900+0000 7f020da44b80 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55af442ebf00 0x55af442ee320 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.900+0000 7f020da44b80 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af442d9b10 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.900+0000 7f020da44b80 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d0a60 0x55af442d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.900+0000 7f01daffd700 0 client.18135 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.900+0000 7f020da44b80 1 --2- 172.21.15.46:0/3757843299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d1380 0x55af442d95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.900+0000 7f020da44b80 1 -- 172.21.15.46:0/3757843299 >> 172.21.15.46:0/3757843299 conn(0x55af44247220 msgr2=0x55af442ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.900+0000 7f020da44b80 1 -- 172.21.15.46:0/3757843299 shutdown_connections 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.900+0000 7f020da44b80 1 -- 172.21.15.46:0/3757843299 wait complete. 2024-03-18T20:43:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.900+0000 7f020da44b80 20 client.18135 trim_cache size 0 max 16384 2024-03-18T20:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f020da44b80 1 Processor -- start 2024-03-18T20:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f020da44b80 1 -- start start 2024-03-18T20:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f020da44b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d0a60 0x55af442e2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f020da44b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d1380 0x55af442d43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f020da44b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af442d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f020da44b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af442da690 con 0x55af442dc220 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f020da44b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af442e94b0 con 0x55af442d0a60 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f020da44b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af442c95e0 con 0x55af442d1380 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af442d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af442d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59618/0 (socket says 172.21.15.46:59618) 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f01fb7fe700 1 -- 172.21.15.46:0/2572143979 learned_addr learned my addr 172.21.15.46:0/2572143979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f01fa7fc700 1 --2- 172.21.15.46:0/2572143979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d1380 0x55af442d43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.918+0000 7f01faffd700 1 --2- 172.21.15.46:0/2572143979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d0a60 0x55af442e2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.919+0000 7f01fb7fe700 1 -- 172.21.15.46:0/2572143979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d1380 msgr2=0x55af442d43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.919+0000 7f01fb7fe700 1 --2- 172.21.15.46:0/2572143979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d1380 0x55af442d43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.919+0000 7f01fb7fe700 1 -- 172.21.15.46:0/2572143979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d0a60 msgr2=0x55af442e2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.919+0000 7f01fb7fe700 1 --2- 172.21.15.46:0/2572143979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d0a60 0x55af442e2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.919+0000 7f01fb7fe700 1 -- 172.21.15.46:0/2572143979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af44051b60 con 0x55af442dc220 2024-03-18T20:43:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.919+0000 7f01fb7fe700 1 --2- 172.21.15.46:0/2572143979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af442d4900 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009120 tx=0x7f01f4009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4a0e1ec6a47048 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.919+0000 7f01e27fc700 1 -- 172.21.15.46:0/2572143979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f401f410 con 0x55af442dc220 2024-03-18T20:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.919+0000 7f01e27fc700 1 -- 172.21.15.46:0/2572143979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01f400f040 con 0x55af442dc220 2024-03-18T20:43:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.919+0000 7f01e27fc700 1 -- 172.21.15.46:0/2572143979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40139b0 con 0x55af442dc220 2024-03-18T20:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.920+0000 7f020da44b80 1 -- 172.21.15.46:0/2572143979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 msgr2=0x55af442d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.920+0000 7f020da44b80 1 --2- 172.21.15.46:0/2572143979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af442d4900 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009120 tx=0x7f01f4009410 comp rx=0 tx=0).stop 2024-03-18T20:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.920+0000 7f020da44b80 1 -- 172.21.15.46:0/2572143979 shutdown_connections 2024-03-18T20:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.920+0000 7f020da44b80 1 --2- 172.21.15.46:0/2572143979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af442d4900 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.920+0000 7f020da44b80 1 --2- 172.21.15.46:0/2572143979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d0a60 0x55af442e2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.920+0000 7f020da44b80 1 --2- 172.21.15.46:0/2572143979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d1380 0x55af442d43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.920+0000 7f020da44b80 1 -- 172.21.15.46:0/2572143979 >> 172.21.15.46:0/2572143979 conn(0x55af44247220 msgr2=0x55af442e7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.920+0000 7f020da44b80 1 -- 172.21.15.46:0/2572143979 shutdown_connections 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.920+0000 7f020da44b80 1 -- 172.21.15.46:0/2572143979 wait complete. 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.920+0000 7f020da44b80 1 -- start start 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f020da44b80 1 Processor -- start 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f020da44b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d0a60 0x55af44333eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f020da44b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d1380 0x55af443343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f020da44b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af44338d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f020da44b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af44339250 con 0x55af442dc220 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f020da44b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af443393c0 con 0x55af442d1380 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f020da44b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af44339580 con 0x55af442d0a60 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f020da44b80 10 client.0 _ll_register_callbacks cb 0x7f01f4007810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af44338d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f01fa7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d1380 0x55af443343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f01faffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d0a60 0x55af44333eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f01fa7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d1380 0x55af443343f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54206/0 (socket says 172.21.15.46:54206) 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af44338d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59632/0 (socket says 172.21.15.46:59632) 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f01fa7fc700 1 -- 172.21.15.46:0/845891242 learned_addr learned my addr 172.21.15.46:0/845891242 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f01faffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d0a60 0x55af44333eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:32820/0 (socket says 172.21.15.46:32820) 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f01fa7fc700 1 -- 172.21.15.46:0/845891242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d0a60 msgr2=0x55af44333eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f01fa7fc700 1 --2- 172.21.15.46:0/845891242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d0a60 0x55af44333eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f01fa7fc700 1 -- 172.21.15.46:0/845891242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 msgr2=0x55af44338d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f01fa7fc700 1 --2- 172.21.15.46:0/845891242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af44338d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.921+0000 7f01fa7fc700 1 -- 172.21.15.46:0/845891242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01fc1a6160 con 0x55af442d1380 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f01fa7fc700 1 --2- 172.21.15.46:0/845891242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d1380 0x55af443343f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01f0097f80 tx=0x7f01f0099880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d76a203393405fb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f01daffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f01daffd700 1 -- 172.21.15.46:0/845891242 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f00a1e20 con 0x55af442d1380 2024-03-18T20:43:38.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f01daffd700 1 -- 172.21.15.46:0/845891242 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f01f00a20b0 con 0x55af442d1380 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f020da44b80 1 -- 172.21.15.46:0/845891242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01f009da60 con 0x55af442d1380 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f01daffd700 1 -- 172.21.15.46:0/845891242 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01f001eb80 con 0x55af442d1380 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f020da44b80 10 client.27812 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f020da44b80 1 -- 172.21.15.46:0/845891242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55af441e9ea0 con 0x55af442d1380 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f01daffd700 1 -- 172.21.15.46:0/845891242 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01f009da60 con 0x55af442d1380 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f01daffd700 1 -- 172.21.15.46:0/845891242 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f01f0095a20 con 0x55af442d1380 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f020da44b80 10 client.27812 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f020da44b80 10 client.27812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f020da44b80 20 client.27812 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f020da44b80 1 --2- 172.21.15.46:0/845891242 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55af443409e0 0x55af44342e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f020da44b80 4 client.27812 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.922+0000 7f020da44b80 1 -- 172.21.15.46:0/845891242 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55af44072cf0 con 0x55af443409e0 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f01faffd700 1 --2- 172.21.15.46:0/845891242 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55af443409e0 0x55af44342e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f01faffd700 1 --2- 172.21.15.46:0/845891242 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55af443409e0 0x55af44342e00 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c993181d89df7048 server_cookie=7594c0b91ba02154 in_seq=0 out_seq=0 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f01daffd700 10 client.27812 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f01daffd700 1 -- 172.21.15.46:0/845891242 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55af44072cf0 con 0x55af443409e0 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f01daffd700 10 client.27812 handle_command_reply: tid=0 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f020da44b80 1 client.27812 shutdown 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f020da44b80 1 -- 172.21.15.46:0/845891242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d1380 msgr2=0x55af443343f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f020da44b80 1 --2- 172.21.15.46:0/845891242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d1380 0x55af443343f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01f0097f80 tx=0x7f01f0099880 comp rx=0 tx=0).stop 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f020da44b80 1 -- 172.21.15.46:0/845891242 shutdown_connections 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f020da44b80 1 --2- 172.21.15.46:0/845891242 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55af443409e0 0x55af44342e00 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f020da44b80 1 --2- 172.21.15.46:0/845891242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af442dc220 0x55af44338d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f020da44b80 1 --2- 172.21.15.46:0/845891242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55af442d0a60 0x55af44333eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f01daffd700 0 client.27812 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f020da44b80 1 --2- 172.21.15.46:0/845891242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af442d1380 0x55af443343f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f020da44b80 1 -- 172.21.15.46:0/845891242 >> 172.21.15.46:0/845891242 conn(0x55af44247220 msgr2=0x55af442d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f020da44b80 1 -- 172.21.15.46:0/845891242 shutdown_connections 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.923+0000 7f020da44b80 1 -- 172.21.15.46:0/845891242 wait complete. 2024-03-18T20:43:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f020da44b80 20 client.27812 trim_cache size 0 max 16384 2024-03-18T20:43:38.933 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:43:38.933 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:43:38.933 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:43:38.933 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:43:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f02018af700 1 -- 172.21.15.46:0/4175236694 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01e40a1310 msgr2=0x7f01e40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f02018af700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01e40a1310 0x7f01e40a37d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0098e0 tx=0x7f01ec005960 comp rx=0 tx=0).stop 2024-03-18T20:43:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f02018af700 1 -- 172.21.15.46:0/4175236694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01fc073a80 msgr2=0x7f01fc199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f02018af700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01fc073a80 0x7f01fc199930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f01f000e9f0 tx=0x7f01f000ed00 comp rx=0 tx=0).stop 2024-03-18T20:43:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f02018af700 1 -- 172.21.15.46:0/4175236694 shutdown_connections 2024-03-18T20:43:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f02018af700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01e40a1310 0x7f01e40a37d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f02018af700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01fc10d5a0 0x7f01fc19dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f02018af700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc073160 0x7f01fc1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f02018af700 1 --2- 172.21.15.46:0/4175236694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01fc073a80 0x7f01fc199930 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f02018af700 1 -- 172.21.15.46:0/4175236694 >> 172.21.15.46:0/4175236694 conn(0x7f01fc0fc7f0 msgr2=0x7f01fc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f02018af700 1 -- 172.21.15.46:0/4175236694 shutdown_connections 2024-03-18T20:43:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:38.924+0000 7f02018af700 1 -- 172.21.15.46:0/4175236694 wait complete. 2024-03-18T20:43:38.940 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:43:38.940 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:43:39.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.118+0000 7f8507659700 1 -- 172.21.15.46:0/3320555768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8500101830 msgr2=0x7f8500105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.118+0000 7f8507659700 1 --2- 172.21.15.46:0/3320555768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8500101830 0x7f8500105870 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009990 tx=0x7f84f0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.118+0000 7f8507659700 1 -- 172.21.15.46:0/3320555768 shutdown_connections 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.118+0000 7f8507659700 1 --2- 172.21.15.46:0/3320555768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8500105fa0 0x7f850010db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.118+0000 7f8507659700 1 --2- 172.21.15.46:0/3320555768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8500100e80 0x7f8500101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.118+0000 7f8507659700 1 --2- 172.21.15.46:0/3320555768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8500101830 0x7f8500105870 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.118+0000 7f8507659700 1 -- 172.21.15.46:0/3320555768 >> 172.21.15.46:0/3320555768 conn(0x7f85000fc720 msgr2=0x7f85000feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.118+0000 7f8507659700 1 -- 172.21.15.46:0/3320555768 shutdown_connections 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.118+0000 7f8507659700 1 -- 172.21.15.46:0/3320555768 wait complete. 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8507659700 1 Processor -- start 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8507659700 1 -- start start 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8507659700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8500100e80 0x7f8500197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8507659700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8500105fa0 0x7f8500197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8507659700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850019baa0 0x7f850019bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8507659700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850019c520 con 0x7f850019baa0 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8507659700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850019c690 con 0x7f8500100e80 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8507659700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850019c850 con 0x7f8500105fa0 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8505bf6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850019baa0 0x7f850019bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8505bf6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850019baa0 0x7f850019bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59650/0 (socket says 172.21.15.46:59650) 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8505bf6700 1 -- 172.21.15.46:0/427124494 learned_addr learned my addr 172.21.15.46:0/427124494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f85053f5700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8500100e80 0x7f8500197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8504bf4700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8500105fa0 0x7f8500197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8505bf6700 1 -- 172.21.15.46:0/427124494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8500105fa0 msgr2=0x7f8500197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8505bf6700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8500105fa0 0x7f8500197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8505bf6700 1 -- 172.21.15.46:0/427124494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8500100e80 msgr2=0x7f8500197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8505bf6700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8500100e80 0x7f8500197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.119+0000 7f8505bf6700 1 -- 172.21.15.46:0/427124494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f00095f0 con 0x7f850019baa0 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.120+0000 7f8505bf6700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850019baa0 0x7f850019bf20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f84f800ef30 tx=0x7f84f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.120+0000 7f84f67fc700 1 -- 172.21.15.46:0/427124494 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8018070 con 0x7f850019baa0 2024-03-18T20:43:39.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.120+0000 7f84f67fc700 1 -- 172.21.15.46:0/427124494 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84f800f040 con 0x7f850019baa0 2024-03-18T20:43:39.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.120+0000 7f8507659700 1 -- 172.21.15.46:0/427124494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85001a3d50 con 0x7f850019baa0 2024-03-18T20:43:39.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.120+0000 7f84f67fc700 1 -- 172.21.15.46:0/427124494 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f800cdb0 con 0x7f850019baa0 2024-03-18T20:43:39.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.120+0000 7f8507659700 1 -- 172.21.15.46:0/427124494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85001a41e0 con 0x7f850019baa0 2024-03-18T20:43:39.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.121+0000 7f8507659700 1 -- 172.21.15.46:0/427124494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8500109270 con 0x7f850019baa0 2024-03-18T20:43:39.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.122+0000 7f84f67fc700 1 -- 172.21.15.46:0/427124494 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f84f8003a00 con 0x7f850019baa0 2024-03-18T20:43:39.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.123+0000 7f84f67fc700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84ec0a1290 0x7f84ec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.123+0000 7f85053f5700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84ec0a1290 0x7f84ec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.123+0000 7f84f67fc700 1 -- 172.21.15.46:0/427124494 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f84f80ccc90 con 0x7f850019baa0 2024-03-18T20:43:39.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.123+0000 7f85053f5700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84ec0a1290 0x7f84ec0a3750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f84fc007be0 tx=0x7f84fc0073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.127+0000 7f84f67fc700 1 -- 172.21.15.46:0/427124494 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f84f80960f0 con 0x7f850019baa0 2024-03-18T20:43:39.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.402+0000 7f8507659700 1 -- 172.21.15.46:0/427124494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8500102350 con 0x7f850019baa0 2024-03-18T20:43:39.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.402+0000 7f84f67fc700 1 -- 172.21.15.46:0/427124494 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+10919 (secure 0 0 0) 0x7f84f8099710 con 0x7f850019baa0 2024-03-18T20:43:39.405 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:43:39.406 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:41:55.183335+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:43:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.405+0000 7f8507659700 1 -- 172.21.15.46:0/427124494 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84ec0a1290 msgr2=0x7f84ec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.405+0000 7f8507659700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84ec0a1290 0x7f84ec0a3750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f84fc007be0 tx=0x7f84fc0073d0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.405+0000 7f8507659700 1 -- 172.21.15.46:0/427124494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850019baa0 msgr2=0x7f850019bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.405+0000 7f8507659700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850019baa0 0x7f850019bf20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f84f800ef30 tx=0x7f84f800c3a0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.405+0000 7f8507659700 1 -- 172.21.15.46:0/427124494 shutdown_connections 2024-03-18T20:43:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.405+0000 7f8507659700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84ec0a1290 0x7f84ec0a3750 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.405+0000 7f8507659700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f850019baa0 0x7f850019bf20 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.405+0000 7f8507659700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8500100e80 0x7f8500197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.405+0000 7f8507659700 1 --2- 172.21.15.46:0/427124494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8500105fa0 0x7f8500197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.405+0000 7f8507659700 1 -- 172.21.15.46:0/427124494 >> 172.21.15.46:0/427124494 conn(0x7f85000fc720 msgr2=0x7f85000fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.405+0000 7f8507659700 1 -- 172.21.15.46:0/427124494 shutdown_connections 2024-03-18T20:43:39.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.405+0000 7f8507659700 1 -- 172.21.15.46:0/427124494 wait complete. 2024-03-18T20:43:39.409 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 43 2024-03-18T20:43:39.421 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:43:39.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:43:39.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.557+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/515418410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe580ffe70 msgr2=0x7fbe58100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.557+0000 7fbe5e9c0700 1 --2- 172.21.15.46:0/515418410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe580ffe70 0x7fbe58100250 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009990 tx=0x7fbe48009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.557+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/515418410 shutdown_connections 2024-03-18T20:43:39.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.557+0000 7fbe5e9c0700 1 --2- 172.21.15.46:0/515418410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe580ffe70 0x7fbe58100250 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.557+0000 7fbe5e9c0700 1 --2- 172.21.15.46:0/515418410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe58104f90 0x7fbe5810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.557+0000 7fbe5e9c0700 1 --2- 172.21.15.46:0/515418410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe58100820 0x7fbe58104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.557+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/515418410 >> 172.21.15.46:0/515418410 conn(0x7fbe58076d80 msgr2=0x7fbe58077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.557+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/515418410 shutdown_connections 2024-03-18T20:43:39.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.557+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/515418410 wait complete. 2024-03-18T20:43:39.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.558+0000 7fbe5e9c0700 1 Processor -- start 2024-03-18T20:43:39.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.558+0000 7fbe5e9c0700 1 -- start start 2024-03-18T20:43:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.558+0000 7fbe5e9c0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe580ffe70 0x7fbe581971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.558+0000 7fbe5e9c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe58104f90 0x7fbe58197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.558+0000 7fbe5e9c0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5819bab0 0x7fbe5819bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe5e9c0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819c530 con 0x7fbe58104f90 2024-03-18T20:43:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe5e9c0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819c6a0 con 0x7fbe5819bab0 2024-03-18T20:43:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe5e9c0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5819c860 con 0x7fbe580ffe70 2024-03-18T20:43:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe58104f90 0x7fbe58197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe58104f90 0x7fbe58197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59670/0 (socket says 172.21.15.46:59670) 2024-03-18T20:43:39.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe577fe700 1 -- 172.21.15.46:0/2522922260 learned_addr learned my addr 172.21.15.46:0/2522922260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe57fff700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe580ffe70 0x7fbe581971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe5cf5d700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5819bab0 0x7fbe5819bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe577fe700 1 -- 172.21.15.46:0/2522922260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe580ffe70 msgr2=0x7fbe581971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe577fe700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe580ffe70 0x7fbe581971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe577fe700 1 -- 172.21.15.46:0/2522922260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5819bab0 msgr2=0x7fbe5819bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe577fe700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5819bab0 0x7fbe5819bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe577fe700 1 -- 172.21.15.46:0/2522922260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe480095f0 con 0x7fbe58104f90 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe577fe700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe58104f90 0x7fbe58197710 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e970 tx=0x7fbe4c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe557fa700 1 -- 172.21.15.46:0/2522922260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c009790 con 0x7fbe58104f90 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe557fa700 1 -- 172.21.15.46:0/2522922260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe4c00eec0 con 0x7fbe58104f90 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/2522922260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe581a3dc0 con 0x7fbe58104f90 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe557fa700 1 -- 172.21.15.46:0/2522922260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c005590 con 0x7fbe58104f90 2024-03-18T20:43:39.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.559+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/2522922260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe581a4240 con 0x7fbe58104f90 2024-03-18T20:43:39.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.562+0000 7fbe557fa700 1 -- 172.21.15.46:0/2522922260 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fbe4c00cbd0 con 0x7fbe58104f90 2024-03-18T20:43:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.563+0000 7fbe557fa700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbe400a1290 0x7fbe400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.563+0000 7fbe557fa700 1 -- 172.21.15.46:0/2522922260 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fbe4c014070 con 0x7fbe58104f90 2024-03-18T20:43:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.563+0000 7fbe57fff700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbe400a1290 0x7fbe400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.563+0000 7fbe57fff700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbe400a1290 0x7fbe400a3750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800b2e0 tx=0x7fbe48005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.563+0000 7fbe6ab55b80 1 Processor -- start 2024-03-18T20:43:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe6ab55b80 1 -- start start 2024-03-18T20:43:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe6ab55b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f28e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe6ab55b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f29380 0x556c51f380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe6ab55b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 0x556c51f3aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe6ab55b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c51e82990 con 0x556c51f28a60 2024-03-18T20:43:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe6ab55b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c51e80350 con 0x556c51f29380 2024-03-18T20:43:39.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe6ab55b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c51e7bc40 con 0x556c51f38630 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe57fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f28e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe57fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f28e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59678/0 (socket says 172.21.15.46:59678) 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe57fff700 1 -- 172.21.15.46:0/3509792507 learned_addr learned my addr 172.21.15.46:0/3509792507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f29380 0x556c51f380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe5cf5d700 1 --2- 172.21.15.46:0/3509792507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 0x556c51f3aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe577fe700 1 -- 172.21.15.46:0/3509792507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 msgr2=0x556c51f3aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe577fe700 1 --2- 172.21.15.46:0/3509792507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 0x556c51f3aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe577fe700 1 -- 172.21.15.46:0/3509792507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 msgr2=0x556c51f28e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe577fe700 1 --2- 172.21.15.46:0/3509792507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f28e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe577fe700 1 -- 172.21.15.46:0/3509792507 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c51cfd3b0 con 0x556c51f29380 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.564+0000 7fbe577fe700 1 --2- 172.21.15.46:0/3509792507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f29380 0x556c51f380f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0ca6b0 tx=0x7fbe4c09a770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a54a8796eaf3c48 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.565+0000 7fbe3e7fc700 1 -- 172.21.15.46:0/3509792507 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c09dea0 con 0x556c51f29380 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.565+0000 7fbe3e7fc700 1 -- 172.21.15.46:0/3509792507 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe4c0a24f0 con 0x556c51f29380 2024-03-18T20:43:39.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.565+0000 7fbe3e7fc700 1 -- 172.21.15.46:0/3509792507 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c09a2d0 con 0x556c51f29380 2024-03-18T20:43:39.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.565+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/3509792507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f29380 msgr2=0x556c51f380f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.565+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/3509792507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f29380 0x556c51f380f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0ca6b0 tx=0x7fbe4c09a770 comp rx=0 tx=0).stop 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.566+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/3509792507 shutdown_connections 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.566+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/3509792507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f28e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.566+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/3509792507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 0x556c51f3aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.566+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/3509792507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f29380 0x556c51f380f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.566+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/3509792507 >> 172.21.15.46:0/3509792507 conn(0x556c51e9f220 msgr2=0x556c51f25020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.566+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/3509792507 shutdown_connections 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.566+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/3509792507 wait complete. 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.566+0000 7fbe6ab55b80 1 -- start start 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.566+0000 7fbe6ab55b80 1 Processor -- start 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe6ab55b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f28a60 0x556c51f38050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe6ab55b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f29380 0x556c51f35390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe6ab55b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 0x556c51f35a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe6ab55b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c51e17d80 con 0x556c51f29380 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe6ab55b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c51e17a20 con 0x556c51f28a60 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe6ab55b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c51cff6a0 con 0x556c51f38630 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe6ab55b80 10 client.0 _ll_register_callbacks cb 0x556c51c833a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f29380 0x556c51f35390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f29380 0x556c51f35390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59692/0 (socket says 172.21.15.46:59692) 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe577fe700 1 -- 172.21.15.46:0/1670291031 learned_addr learned my addr 172.21.15.46:0/1670291031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe5cf5d700 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 0x556c51f35a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe57fff700 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f28a60 0x556c51f38050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe577fe700 1 -- 172.21.15.46:0/1670291031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 msgr2=0x556c51f35a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe577fe700 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 0x556c51f35a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe577fe700 1 -- 172.21.15.46:0/1670291031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f28a60 msgr2=0x556c51f38050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe577fe700 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f28a60 0x556c51f38050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe577fe700 1 -- 172.21.15.46:0/1670291031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe581a4240 con 0x556c51f29380 2024-03-18T20:43:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe577fe700 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f29380 0x556c51f35390 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c099d70 tx=0x7fbe4c09df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3b712040141a85 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe377fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe377fe700 1 -- 172.21.15.46:0/1670291031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c016330 con 0x556c51f29380 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe377fe700 1 -- 172.21.15.46:0/1670291031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fbe4c063030 con 0x556c51f29380 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.567+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1670291031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c51f1cad0 con 0x556c51f29380 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.568+0000 7fbe377fe700 1 -- 172.21.15.46:0/1670291031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe4c016490 con 0x556c51f29380 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.568+0000 7fbe6ab55b80 10 client.18159 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.568+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1670291031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x556c51ca9b60 con 0x556c51f29380 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.568+0000 7fbe377fe700 1 -- 172.21.15.46:0/1670291031 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe4c0a6080 con 0x556c51f29380 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.568+0000 7fbe377fe700 1 -- 172.21.15.46:0/1670291031 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbe4c0a2060 con 0x556c51f29380 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.568+0000 7fbe6ab55b80 10 client.18159 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.568+0000 7fbe6ab55b80 10 client.18159 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.568+0000 7fbe6ab55b80 20 client.18159 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.568+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x556c51f47f10 0x556c51f4a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.568+0000 7fbe6ab55b80 4 client.18159 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.568+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1670291031 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c51e74700 con 0x556c51f47f10 2024-03-18T20:43:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.568+0000 7fbe57fff700 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x556c51f47f10 0x556c51f4a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.569+0000 7fbe57fff700 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x556c51f47f10 0x556c51f4a330 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=448f4411213facd6 server_cookie=28255d78b9a25acf in_seq=0 out_seq=0 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.569+0000 7fbe377fe700 10 client.18159 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.569+0000 7fbe377fe700 1 -- 172.21.15.46:0/1670291031 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556c51e74700 con 0x556c51f47f10 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.569+0000 7fbe377fe700 10 client.18159 handle_command_reply: tid=0 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.569+0000 7fbe6ab55b80 1 client.18159 shutdown 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.570+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1670291031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f29380 msgr2=0x556c51f35390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.570+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f29380 0x556c51f35390 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c099d70 tx=0x7fbe4c09df10 comp rx=0 tx=0).stop 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.570+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1670291031 shutdown_connections 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.570+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x556c51f47f10 0x556c51f4a330 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.570+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 0x556c51f35a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.570+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f28a60 0x556c51f38050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.570+0000 7fbe377fe700 0 client.18159 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.570+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/1670291031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f29380 0x556c51f35390 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.570+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1670291031 >> 172.21.15.46:0/1670291031 conn(0x556c51e9f220 msgr2=0x556c51f24c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.570+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1670291031 shutdown_connections 2024-03-18T20:43:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.570+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1670291031 wait complete. 2024-03-18T20:43:39.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.570+0000 7fbe6ab55b80 20 client.18159 trim_cache size 0 max 16384 2024-03-18T20:43:39.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.583+0000 7fbe6ab55b80 1 Processor -- start 2024-03-18T20:43:39.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.583+0000 7fbe6ab55b80 1 -- start start 2024-03-18T20:43:39.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe6ab55b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe6ab55b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f29380 0x556c51f34c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe6ab55b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f38630 0x556c51f351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe6ab55b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c51e74700 con 0x556c51f28a60 2024-03-18T20:43:39.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe6ab55b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c51f29d30 con 0x556c51f38630 2024-03-18T20:43:39.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe6ab55b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c51f1c610 con 0x556c51f29380 2024-03-18T20:43:39.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe57fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe57fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59706/0 (socket says 172.21.15.46:59706) 2024-03-18T20:43:39.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe57fff700 1 -- 172.21.15.46:0/2372104638 learned_addr learned my addr 172.21.15.46:0/2372104638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe577fe700 1 --2- 172.21.15.46:0/2372104638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f29380 0x556c51f34c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe5cf5d700 1 --2- 172.21.15.46:0/2372104638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f38630 0x556c51f351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe57fff700 1 -- 172.21.15.46:0/2372104638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f29380 msgr2=0x556c51f34c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe57fff700 1 --2- 172.21.15.46:0/2372104638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f29380 0x556c51f34c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe57fff700 1 -- 172.21.15.46:0/2372104638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f38630 msgr2=0x556c51f351b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe57fff700 1 --2- 172.21.15.46:0/2372104638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f38630 0x556c51f351b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe57fff700 1 -- 172.21.15.46:0/2372104638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c51e41ea0 con 0x556c51f28a60 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe577fe700 1 --2- 172.21.15.46:0/2372104638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f29380 0x556c51f34c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe5cf5d700 1 --2- 172.21.15.46:0/2372104638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f38630 0x556c51f351b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.584+0000 7fbe57fff700 1 --2- 172.21.15.46:0/2372104638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f440f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009990 tx=0x7fbe48020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5a5f9c070fd82c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.585+0000 7fbe3effd700 1 -- 172.21.15.46:0/2372104638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48036b00 con 0x556c51f28a60 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.585+0000 7fbe3effd700 1 -- 172.21.15.46:0/2372104638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe4801e040 con 0x556c51f28a60 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.585+0000 7fbe3effd700 1 -- 172.21.15.46:0/2372104638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48004a60 con 0x556c51f28a60 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.585+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/2372104638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 msgr2=0x556c51f440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.585+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/2372104638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f440f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009990 tx=0x7fbe48020000 comp rx=0 tx=0).stop 2024-03-18T20:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.585+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/2372104638 shutdown_connections 2024-03-18T20:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.585+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/2372104638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f440f0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.585+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/2372104638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f38630 0x556c51f351b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.585+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/2372104638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f29380 0x556c51f34c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.585+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/2372104638 >> 172.21.15.46:0/2372104638 conn(0x556c51e9f220 msgr2=0x556c51f439e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.585+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/2372104638 shutdown_connections 2024-03-18T20:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.585+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/2372104638 wait complete. 2024-03-18T20:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.586+0000 7fbe6ab55b80 1 -- start start 2024-03-18T20:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.586+0000 7fbe6ab55b80 1 Processor -- start 2024-03-18T20:43:39.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.586+0000 7fbe6ab55b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f3d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.586+0000 7fbe6ab55b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f29380 0x556c51f3d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.586+0000 7fbe6ab55b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 0x556c51f94cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.586+0000 7fbe6ab55b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c51f95210 con 0x556c51f28a60 2024-03-18T20:43:39.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.586+0000 7fbe6ab55b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c51f95380 con 0x556c51f29380 2024-03-18T20:43:39.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.586+0000 7fbe6ab55b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c51f95540 con 0x556c51f38630 2024-03-18T20:43:39.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.586+0000 7fbe6ab55b80 10 client.0 _ll_register_callbacks cb 0x7fbe48005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:39.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.586+0000 7fbe57fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f3d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe57fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f3d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59708/0 (socket says 172.21.15.46:59708) 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe57fff700 1 -- 172.21.15.46:0/1643065540 learned_addr learned my addr 172.21.15.46:0/1643065540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe5cf5d700 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 0x556c51f94cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe577fe700 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f29380 0x556c51f3d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe57fff700 1 -- 172.21.15.46:0/1643065540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 msgr2=0x556c51f94cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe57fff700 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 0x556c51f94cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe57fff700 1 -- 172.21.15.46:0/1643065540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f29380 msgr2=0x556c51f3d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe57fff700 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f29380 0x556c51f3d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe57fff700 1 -- 172.21.15.46:0/1643065540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5810d380 con 0x556c51f28a60 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe577fe700 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f29380 0x556c51f3d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe57fff700 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f3d390 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fbe48004680 tx=0x7fbe48036b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a56787edcaa3b91 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe377fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe377fe700 1 -- 172.21.15.46:0/1643065540 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe480364d0 con 0x556c51f28a60 2024-03-18T20:43:39.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe377fe700 1 -- 172.21.15.46:0/1643065540 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fbe480046e0 con 0x556c51f28a60 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1643065540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe4c0a24f0 con 0x556c51f28a60 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe377fe700 1 -- 172.21.15.46:0/1643065540 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe480048a0 con 0x556c51f28a60 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe6ab55b80 10 client.18165 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.587+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1643065540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x556c51d01cd0 con 0x556c51f28a60 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.588+0000 7fbe377fe700 1 -- 172.21.15.46:0/1643065540 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe4804b080 con 0x556c51f28a60 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.588+0000 7fbe377fe700 1 -- 172.21.15.46:0/1643065540 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbe4800f8e0 con 0x556c51f28a60 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.588+0000 7fbe6ab55b80 10 client.18165 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.588+0000 7fbe6ab55b80 10 client.18165 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.588+0000 7fbe6ab55b80 20 client.18165 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.588+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x556c51f9ca80 0x556c51f9eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.588+0000 7fbe6ab55b80 4 client.18165 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.588+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1643065540 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556c51ccacf0 con 0x556c51f9ca80 2024-03-18T20:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.588+0000 7fbe577fe700 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x556c51f9ca80 0x556c51f9eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe377fe700 10 client.18165 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe577fe700 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x556c51f9ca80 0x556c51f9eea0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a8c7658bd29c7723 server_cookie=22a9ffb983007efa in_seq=0 out_seq=0 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe377fe700 1 -- 172.21.15.46:0/1643065540 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556c51ccacf0 con 0x556c51f9ca80 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe377fe700 10 client.18165 handle_command_reply: tid=0 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe6ab55b80 1 client.18165 shutdown 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1643065540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 msgr2=0x556c51f3d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f3d390 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fbe48004680 tx=0x7fbe48036b00 comp rx=0 tx=0).stop 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1643065540 shutdown_connections 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x556c51f9ca80 0x556c51f9eea0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c51f28a60 0x556c51f3d390 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556c51f38630 0x556c51f94cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe377fe700 0 client.18165 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe6ab55b80 1 --2- 172.21.15.46:0/1643065540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c51f29380 0x556c51f3d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1643065540 >> 172.21.15.46:0/1643065540 conn(0x556c51e9f220 msgr2=0x556c51f37470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1643065540 shutdown_connections 2024-03-18T20:43:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe6ab55b80 1 -- 172.21.15.46:0/1643065540 wait complete. 2024-03-18T20:43:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.589+0000 7fbe6ab55b80 20 client.18165 trim_cache size 0 max 16384 2024-03-18T20:43:39.599 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:43:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.590+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/2522922260 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbe400a1290 msgr2=0x7fbe400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.590+0000 7fbe5e9c0700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbe400a1290 0x7fbe400a3750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800b2e0 tx=0x7fbe48005960 comp rx=0 tx=0).stop 2024-03-18T20:43:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.590+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/2522922260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe58104f90 msgr2=0x7fbe58197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.590+0000 7fbe5e9c0700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe58104f90 0x7fbe58197710 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e970 tx=0x7fbe4c00ec80 comp rx=0 tx=0).stop 2024-03-18T20:43:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.590+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/2522922260 shutdown_connections 2024-03-18T20:43:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.590+0000 7fbe5e9c0700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbe400a1290 0x7fbe400a3750 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.590+0000 7fbe5e9c0700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5819bab0 0x7fbe5819bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.590+0000 7fbe5e9c0700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe580ffe70 0x7fbe581971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.590+0000 7fbe5e9c0700 1 --2- 172.21.15.46:0/2522922260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe58104f90 0x7fbe58197710 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.590+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/2522922260 >> 172.21.15.46:0/2522922260 conn(0x7fbe58076d80 msgr2=0x7fbe58069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.590+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/2522922260 shutdown_connections 2024-03-18T20:43:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.590+0000 7fbe5e9c0700 1 -- 172.21.15.46:0/2522922260 wait complete. 2024-03-18T20:43:39.606 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:43:39.606 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:43:39.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 260661 3.51 MB/sec execute 274 sec latency 2097.290 ms 2024-03-18T20:43:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.741+0000 7f0163fb1700 1 -- 172.21.15.46:0/1304186575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f015c101900 msgr2=0x7f015c105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.742+0000 7f0163fb1700 1 --2- 172.21.15.46:0/1304186575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f015c101900 0x7f015c105940 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f014c009990 tx=0x7f014c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.742+0000 7f0163fb1700 1 -- 172.21.15.46:0/1304186575 shutdown_connections 2024-03-18T20:43:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.742+0000 7f0163fb1700 1 --2- 172.21.15.46:0/1304186575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f015c106070 0x7f015c10dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.742+0000 7f0163fb1700 1 --2- 172.21.15.46:0/1304186575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c100f50 0x7f015c101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.742+0000 7f0163fb1700 1 --2- 172.21.15.46:0/1304186575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f015c101900 0x7f015c105940 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.742+0000 7f0163fb1700 1 -- 172.21.15.46:0/1304186575 >> 172.21.15.46:0/1304186575 conn(0x7f015c0fc7f0 msgr2=0x7f015c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.742+0000 7f0163fb1700 1 -- 172.21.15.46:0/1304186575 shutdown_connections 2024-03-18T20:43:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.742+0000 7f0163fb1700 1 -- 172.21.15.46:0/1304186575 wait complete. 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f0163fb1700 1 Processor -- start 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f0163fb1700 1 -- start start 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f0163fb1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c100f50 0x7f015c197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f0163fb1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f015c106070 0x7f015c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f0163fb1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f015c19bae0 0x7f015c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f0163fb1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c19c560 con 0x7f015c19bae0 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f0163fb1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c19c6d0 con 0x7f015c106070 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f0163fb1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c19c890 con 0x7f015c100f50 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f016254e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f015c19bae0 0x7f015c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f016254e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f015c19bae0 0x7f015c19bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59726/0 (socket says 172.21.15.46:59726) 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f016254e700 1 -- 172.21.15.46:0/1377138913 learned_addr learned my addr 172.21.15.46:0/1377138913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f0161d4d700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c100f50 0x7f015c197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f016154c700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f015c106070 0x7f015c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f016254e700 1 -- 172.21.15.46:0/1377138913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c100f50 msgr2=0x7f015c197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f016254e700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c100f50 0x7f015c197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f016254e700 1 -- 172.21.15.46:0/1377138913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f015c106070 msgr2=0x7f015c197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f016254e700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f015c106070 0x7f015c197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f016254e700 1 -- 172.21.15.46:0/1377138913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c0095f0 con 0x7f015c19bae0 2024-03-18T20:43:39.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.743+0000 7f0161d4d700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c100f50 0x7f015c197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:39.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.744+0000 7f016254e700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f015c19bae0 0x7f015c19bf60 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f015400ada0 tx=0x7f015400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.744+0000 7f0152ffd700 1 -- 172.21.15.46:0/1377138913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015400f070 con 0x7f015c19bae0 2024-03-18T20:43:39.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.744+0000 7f0152ffd700 1 -- 172.21.15.46:0/1377138913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0154007b30 con 0x7f015c19bae0 2024-03-18T20:43:39.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.744+0000 7f0163fb1700 1 -- 172.21.15.46:0/1377138913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c1a3df0 con 0x7f015c19bae0 2024-03-18T20:43:39.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.744+0000 7f0152ffd700 1 -- 172.21.15.46:0/1377138913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154005440 con 0x7f015c19bae0 2024-03-18T20:43:39.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.744+0000 7f0163fb1700 1 -- 172.21.15.46:0/1377138913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c1a4280 con 0x7f015c19bae0 2024-03-18T20:43:39.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.746+0000 7f0152ffd700 1 -- 172.21.15.46:0/1377138913 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0154004ad0 con 0x7f015c19bae0 2024-03-18T20:43:39.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.747+0000 7f0152ffd700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01480a1290 0x7f01480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.747+0000 7f0152ffd700 1 -- 172.21.15.46:0/1377138913 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f01540cc360 con 0x7f015c19bae0 2024-03-18T20:43:39.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.747+0000 7f0161d4d700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01480a1290 0x7f01480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f0161d4d700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01480a1290 0x7f01480a3750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0158005950 tx=0x7f01580058e0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f0170146b80 1 Processor -- start 2024-03-18T20:43:39.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f0170146b80 1 -- start start 2024-03-18T20:43:39.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f0170146b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdba60 0x557d60fdbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f0170146b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fdc380 0x557d60fe6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f0170146b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fe7220 0x557d60fe9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f0170146b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d60f35990 con 0x557d60fdc380 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f0170146b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d60f33350 con 0x557d60fe7220 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f0170146b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d60f2ec40 con 0x557d60fdba60 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f016154c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fdc380 0x557d60fe6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f016154c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fdc380 0x557d60fe6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59736/0 (socket says 172.21.15.46:59736) 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f016154c700 1 -- 172.21.15.46:0/95680526 learned_addr learned my addr 172.21.15.46:0/95680526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.748+0000 7f0161d4d700 1 --2- 172.21.15.46:0/95680526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdba60 0x557d60fdbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.749+0000 7f016254e700 1 --2- 172.21.15.46:0/95680526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fe7220 0x557d60fe9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.749+0000 7f016154c700 1 -- 172.21.15.46:0/95680526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdba60 msgr2=0x557d60fdbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.749+0000 7f016154c700 1 --2- 172.21.15.46:0/95680526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdba60 0x557d60fdbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.749+0000 7f016154c700 1 -- 172.21.15.46:0/95680526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fe7220 msgr2=0x557d60fe9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.749+0000 7f016154c700 1 --2- 172.21.15.46:0/95680526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fe7220 0x557d60fe9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.749+0000 7f016154c700 1 -- 172.21.15.46:0/95680526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d60db03b0 con 0x557d60fdc380 2024-03-18T20:43:39.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.749+0000 7f016154c700 1 --2- 172.21.15.46:0/95680526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fdc380 0x557d60fe6ce0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f014c015000 tx=0x7f014c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65d308566c71f57 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.749+0000 7f0147fff700 1 -- 172.21.15.46:0/95680526 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c005340 con 0x557d60fdc380 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.749+0000 7f0147fff700 1 -- 172.21.15.46:0/95680526 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f014c01e040 con 0x557d60fdc380 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.749+0000 7f0147fff700 1 -- 172.21.15.46:0/95680526 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c004920 con 0x557d60fdc380 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.750+0000 7f0170146b80 1 -- 172.21.15.46:0/95680526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fdc380 msgr2=0x557d60fe6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.750+0000 7f0170146b80 1 --2- 172.21.15.46:0/95680526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fdc380 0x557d60fe6ce0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f014c015000 tx=0x7f014c009ee0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.750+0000 7f0170146b80 1 -- 172.21.15.46:0/95680526 shutdown_connections 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.750+0000 7f0170146b80 1 --2- 172.21.15.46:0/95680526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fe7220 0x557d60fe9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.750+0000 7f0170146b80 1 --2- 172.21.15.46:0/95680526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdba60 0x557d60fdbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.750+0000 7f0170146b80 1 --2- 172.21.15.46:0/95680526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fdc380 0x557d60fe6ce0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.750+0000 7f0170146b80 1 -- 172.21.15.46:0/95680526 >> 172.21.15.46:0/95680526 conn(0x557d60f52220 msgr2=0x557d60fd8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.750+0000 7f0170146b80 1 -- 172.21.15.46:0/95680526 shutdown_connections 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.750+0000 7f0170146b80 1 -- 172.21.15.46:0/95680526 wait complete. 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.750+0000 7f0170146b80 1 -- start start 2024-03-18T20:43:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.750+0000 7f0170146b80 1 Processor -- start 2024-03-18T20:43:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f0170146b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdba60 0x557d60fe6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f0170146b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdc380 0x557d60fe3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f0170146b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d60fe46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f0170146b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d60ecad80 con 0x557d60fe7220 2024-03-18T20:43:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f0170146b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d60ecaa20 con 0x557d60fdc380 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f0170146b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d60db26a0 con 0x557d60fdba60 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f0170146b80 10 client.0 _ll_register_callbacks cb 0x557d60d363a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f016254e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d60fe46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f016254e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d60fe46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59742/0 (socket says 172.21.15.46:59742) 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f016254e700 1 -- 172.21.15.46:0/1880277327 learned_addr learned my addr 172.21.15.46:0/1880277327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f016154c700 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdc380 0x557d60fe3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f0161d4d700 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdba60 0x557d60fe6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f016254e700 1 -- 172.21.15.46:0/1880277327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdba60 msgr2=0x557d60fe6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f016254e700 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdba60 0x557d60fe6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f016254e700 1 -- 172.21.15.46:0/1880277327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdc380 msgr2=0x557d60fe3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f016254e700 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdc380 0x557d60fe3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f016254e700 1 -- 172.21.15.46:0/1880277327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c10e460 con 0x557d60fe7220 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.751+0000 7f0161d4d700 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdba60 0x557d60fe6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:43:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f016254e700 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d60fe46b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0154095f50 tx=0x7f01540990a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca054c971bf639a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f01457fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f01457fa700 1 -- 172.21.15.46:0/1880277327 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015409d120 con 0x557d60fe7220 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f01457fa700 1 -- 172.21.15.46:0/1880277327 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f01540994e0 con 0x557d60fe7220 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f0170146b80 1 -- 172.21.15.46:0/1880277327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d60fcfad0 con 0x557d60fe7220 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f01457fa700 1 -- 172.21.15.46:0/1880277327 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f015409d310 con 0x557d60fe7220 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f0170146b80 10 client.18177 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f0170146b80 1 -- 172.21.15.46:0/1880277327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x557d60d5cb60 con 0x557d60fe7220 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f01457fa700 1 -- 172.21.15.46:0/1880277327 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d60fcfad0 con 0x557d60fe7220 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f01457fa700 1 -- 172.21.15.46:0/1880277327 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0154098ad0 con 0x557d60fe7220 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f0170146b80 10 client.18177 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f0170146b80 10 client.18177 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f0170146b80 20 client.18177 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f0170146b80 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557d60ff6da0 0x557d60ff91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f0170146b80 4 client.18177 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.752+0000 7f0170146b80 1 -- 172.21.15.46:0/1880277327 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d60f27700 con 0x557d60ff6da0 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.753+0000 7f0161d4d700 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557d60ff6da0 0x557d60ff91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.753+0000 7f01457fa700 10 client.18177 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.753+0000 7f0161d4d700 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557d60ff6da0 0x557d60ff91c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5e964d84411e9d6c server_cookie=fb2a6749cd208d29 in_seq=0 out_seq=0 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f01457fa700 1 -- 172.21.15.46:0/1880277327 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557d60f27700 con 0x557d60ff6da0 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f01457fa700 10 client.18177 handle_command_reply: tid=0 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f0170146b80 1 client.18177 shutdown 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f0170146b80 1 -- 172.21.15.46:0/1880277327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 msgr2=0x557d60fe46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f0170146b80 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d60fe46b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0154095f50 tx=0x7f01540990a0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f0170146b80 1 -- 172.21.15.46:0/1880277327 shutdown_connections 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f0170146b80 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557d60ff6da0 0x557d60ff91c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f0170146b80 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d60fe46b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f0170146b80 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdba60 0x557d60fe6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f0170146b80 1 --2- 172.21.15.46:0/1880277327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdc380 0x557d60fe3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f01457fa700 0 client.18177 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:43:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f0170146b80 1 -- 172.21.15.46:0/1880277327 >> 172.21.15.46:0/1880277327 conn(0x557d60f52220 msgr2=0x557d60fd7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f0170146b80 1 -- 172.21.15.46:0/1880277327 shutdown_connections 2024-03-18T20:43:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f0170146b80 1 -- 172.21.15.46:0/1880277327 wait complete. 2024-03-18T20:43:39.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.754+0000 7f0170146b80 20 client.18177 trim_cache size 0 max 16384 2024-03-18T20:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.768+0000 7f0170146b80 1 Processor -- start 2024-03-18T20:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.768+0000 7f0170146b80 1 -- start start 2024-03-18T20:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.768+0000 7f0170146b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdba60 0x557d60feddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.768+0000 7f0170146b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdc380 0x557d60fd6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.768+0000 7f0170146b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d60ff2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.768+0000 7f0170146b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d60fd9230 con 0x557d60fe7220 2024-03-18T20:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.768+0000 7f0170146b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d60fee330 con 0x557d60fdba60 2024-03-18T20:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.768+0000 7f0170146b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d60fdfdf0 con 0x557d60fdc380 2024-03-18T20:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.768+0000 7f016254e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d60ff2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.768+0000 7f016254e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d60ff2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59746/0 (socket says 172.21.15.46:59746) 2024-03-18T20:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.768+0000 7f016254e700 1 -- 172.21.15.46:0/2469114469 learned_addr learned my addr 172.21.15.46:0/2469114469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f016154c700 1 --2- 172.21.15.46:0/2469114469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdc380 0x557d60fd6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f0161d4d700 1 --2- 172.21.15.46:0/2469114469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdba60 0x557d60feddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f016254e700 1 -- 172.21.15.46:0/2469114469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdc380 msgr2=0x557d60fd6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f016254e700 1 --2- 172.21.15.46:0/2469114469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdc380 0x557d60fd6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f016254e700 1 -- 172.21.15.46:0/2469114469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdba60 msgr2=0x557d60feddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f016254e700 1 --2- 172.21.15.46:0/2469114469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdba60 0x557d60feddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f016254e700 1 -- 172.21.15.46:0/2469114469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d60ef4ea0 con 0x557d60fe7220 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f016154c700 1 --2- 172.21.15.46:0/2469114469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdc380 0x557d60fd6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f0161d4d700 1 --2- 172.21.15.46:0/2469114469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdba60 0x557d60feddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f016254e700 1 --2- 172.21.15.46:0/2469114469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d60ff2540 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f01540959e0 tx=0x7f015401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dd8ea94a0ebd22 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f0150ff9700 1 -- 172.21.15.46:0/2469114469 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154098e40 con 0x557d60fe7220 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f0150ff9700 1 -- 172.21.15.46:0/2469114469 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f015409d120 con 0x557d60fe7220 2024-03-18T20:43:39.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.769+0000 7f0150ff9700 1 -- 172.21.15.46:0/2469114469 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154098c30 con 0x557d60fe7220 2024-03-18T20:43:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.770+0000 7f0170146b80 1 -- 172.21.15.46:0/2469114469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 msgr2=0x557d60ff2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.770+0000 7f0170146b80 1 --2- 172.21.15.46:0/2469114469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d60ff2540 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f01540959e0 tx=0x7f015401b000 comp rx=0 tx=0).stop 2024-03-18T20:43:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.770+0000 7f0170146b80 1 -- 172.21.15.46:0/2469114469 shutdown_connections 2024-03-18T20:43:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.770+0000 7f0170146b80 1 --2- 172.21.15.46:0/2469114469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d60ff2540 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.770+0000 7f0170146b80 1 --2- 172.21.15.46:0/2469114469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdba60 0x557d60feddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.770+0000 7f0170146b80 1 --2- 172.21.15.46:0/2469114469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdc380 0x557d60fd6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.770+0000 7f0170146b80 1 -- 172.21.15.46:0/2469114469 >> 172.21.15.46:0/2469114469 conn(0x557d60f52220 msgr2=0x557d60fd3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.770+0000 7f0170146b80 1 -- 172.21.15.46:0/2469114469 shutdown_connections 2024-03-18T20:43:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.770+0000 7f0170146b80 1 -- 172.21.15.46:0/2469114469 wait complete. 2024-03-18T20:43:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.770+0000 7f0170146b80 1 -- start start 2024-03-18T20:43:39.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.770+0000 7f0170146b80 1 Processor -- start 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f0170146b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdba60 0x557d60febfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f0170146b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdc380 0x557d60fec500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f0170146b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d6104e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f0170146b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d60fecc80 con 0x557d60fe7220 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f0170146b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6104e570 con 0x557d60fdba60 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f0170146b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6104e730 con 0x557d60fdc380 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f0170146b80 10 client.0 _ll_register_callbacks cb 0x7f01540958d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f016254e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d6104e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f0161d4d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdba60 0x557d60febfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f016254e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d6104e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59754/0 (socket says 172.21.15.46:59754) 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f016254e700 1 -- 172.21.15.46:0/2806092988 learned_addr learned my addr 172.21.15.46:0/2806092988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f016154c700 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdc380 0x557d60fec500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f016254e700 1 -- 172.21.15.46:0/2806092988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdc380 msgr2=0x557d60fec500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f016254e700 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdc380 0x557d60fec500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f016254e700 1 -- 172.21.15.46:0/2806092988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdba60 msgr2=0x557d60febfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f016254e700 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdba60 0x557d60febfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f016254e700 1 -- 172.21.15.46:0/2806092988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d60d5cb60 con 0x557d60fe7220 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f016154c700 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdc380 0x557d60fec500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f016254e700 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d6104e000 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f015400c730 tx=0x7f0154098d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ca49c13a39cd10 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f01457fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.771+0000 7f01457fa700 1 -- 172.21.15.46:0/2806092988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154099940 con 0x557d60fe7220 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f01457fa700 1 -- 172.21.15.46:0/2806092988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f015409d310 con 0x557d60fe7220 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f0170146b80 1 -- 172.21.15.46:0/2806092988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f014c01e040 con 0x557d60fe7220 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f01457fa700 1 -- 172.21.15.46:0/2806092988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01540a18d0 con 0x557d60fe7220 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f0170146b80 10 client.18183 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f0170146b80 1 -- 172.21.15.46:0/2806092988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x557d60db4cd0 con 0x557d60fe7220 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f01457fa700 1 -- 172.21.15.46:0/2806092988 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f014c01e040 con 0x557d60fe7220 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f01457fa700 1 -- 172.21.15.46:0/2806092988 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f015409f470 con 0x557d60fe7220 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f0170146b80 10 client.18183 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:39.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f0170146b80 10 client.18183 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f0170146b80 20 client.18183 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f0170146b80 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557d6104f9a0 0x557d61051d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f0170146b80 4 client.18183 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f0170146b80 1 -- 172.21.15.46:0/2806092988 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d60d7dcf0 con 0x557d6104f9a0 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.772+0000 7f0161d4d700 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557d6104f9a0 0x557d61051d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f01457fa700 10 client.18183 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f0161d4d700 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557d6104f9a0 0x557d61051d90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aa61624d9576cedc server_cookie=bf7b3aab68a7cb6 in_seq=0 out_seq=0 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f01457fa700 1 -- 172.21.15.46:0/2806092988 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d60d7dcf0 con 0x557d6104f9a0 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f01457fa700 10 client.18183 handle_command_reply: tid=0 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f0170146b80 1 client.18183 shutdown 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f0170146b80 1 -- 172.21.15.46:0/2806092988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 msgr2=0x557d6104e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f0170146b80 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d6104e000 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f015400c730 tx=0x7f0154098d40 comp rx=0 tx=0).stop 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f0170146b80 1 -- 172.21.15.46:0/2806092988 shutdown_connections 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f0170146b80 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557d6104f9a0 0x557d61051d90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f0170146b80 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557d60fe7220 0x557d6104e000 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f0170146b80 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557d60fdba60 0x557d60febfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f01457fa700 0 client.18183 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:43:39.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f0170146b80 1 --2- 172.21.15.46:0/2806092988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557d60fdc380 0x557d60fec500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f0170146b80 1 -- 172.21.15.46:0/2806092988 >> 172.21.15.46:0/2806092988 conn(0x557d60f52220 msgr2=0x557d60ff2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f0170146b80 1 -- 172.21.15.46:0/2806092988 shutdown_connections 2024-03-18T20:43:39.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.773+0000 7f0170146b80 1 -- 172.21.15.46:0/2806092988 wait complete. 2024-03-18T20:43:39.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0170146b80 20 client.18183 trim_cache size 0 max 16384 2024-03-18T20:43:39.782 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:43:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0163fb1700 1 -- 172.21.15.46:0/1377138913 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01480a1290 msgr2=0x7f01480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0163fb1700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01480a1290 0x7f01480a3750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0158005950 tx=0x7f01580058e0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0163fb1700 1 -- 172.21.15.46:0/1377138913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f015c19bae0 msgr2=0x7f015c19bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0163fb1700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f015c19bae0 0x7f015c19bf60 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f015400ada0 tx=0x7f015400c330 comp rx=0 tx=0).stop 2024-03-18T20:43:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0163fb1700 1 -- 172.21.15.46:0/1377138913 shutdown_connections 2024-03-18T20:43:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0163fb1700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01480a1290 0x7f01480a3750 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0163fb1700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f015c19bae0 0x7f015c19bf60 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0163fb1700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f015c100f50 0x7f015c197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0163fb1700 1 --2- 172.21.15.46:0/1377138913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f015c106070 0x7f015c197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0163fb1700 1 -- 172.21.15.46:0/1377138913 >> 172.21.15.46:0/1377138913 conn(0x7f015c0fc7f0 msgr2=0x7f015c0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0163fb1700 1 -- 172.21.15.46:0/1377138913 shutdown_connections 2024-03-18T20:43:39.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.774+0000 7f0163fb1700 1 -- 172.21.15.46:0/1377138913 wait complete. 2024-03-18T20:43:39.791 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:43:39.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:43:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.927+0000 7efd53d80700 1 -- 172.21.15.46:0/3874686893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c10d750 msgr2=0x7efd4c10fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.927+0000 7efd53d80700 1 --2- 172.21.15.46:0/3874686893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c10d750 0x7efd4c10fb40 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7efd4400b0a0 tx=0x7efd4400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.927+0000 7efd53d80700 1 -- 172.21.15.46:0/3874686893 shutdown_connections 2024-03-18T20:43:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.927+0000 7efd53d80700 1 --2- 172.21.15.46:0/3874686893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c10d750 0x7efd4c10fb40 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.927+0000 7efd53d80700 1 --2- 172.21.15.46:0/3874686893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c068ff0 0x7efd4c0693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.927+0000 7efd53d80700 1 --2- 172.21.15.46:0/3874686893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd4c069910 0x7efd4c10d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.927+0000 7efd53d80700 1 -- 172.21.15.46:0/3874686893 >> 172.21.15.46:0/3874686893 conn(0x7efd4c076d80 msgr2=0x7efd4c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.927+0000 7efd53d80700 1 -- 172.21.15.46:0/3874686893 shutdown_connections 2024-03-18T20:43:39.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.927+0000 7efd53d80700 1 -- 172.21.15.46:0/3874686893 wait complete. 2024-03-18T20:43:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.928+0000 7efd53d80700 1 Processor -- start 2024-03-18T20:43:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.928+0000 7efd53d80700 1 -- start start 2024-03-18T20:43:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.928+0000 7efd53d80700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c068ff0 0x7efd4c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.928+0000 7efd53d80700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd4c069910 0x7efd4c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.928+0000 7efd53d80700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c10d750 0x7efd4c19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.928+0000 7efd53d80700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c19e3d0 con 0x7efd4c068ff0 2024-03-18T20:43:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.928+0000 7efd53d80700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c19e540 con 0x7efd4c069910 2024-03-18T20:43:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.928+0000 7efd53d80700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c19e700 con 0x7efd4c10d750 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.928+0000 7efd51b1c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c068ff0 0x7efd4c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.928+0000 7efd51b1c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c068ff0 0x7efd4c1994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59782/0 (socket says 172.21.15.46:59782) 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.928+0000 7efd51b1c700 1 -- 172.21.15.46:0/2784159979 learned_addr learned my addr 172.21.15.46:0/2784159979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.928+0000 7efd5231d700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c10d750 0x7efd4c19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.929+0000 7efd5131b700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd4c069910 0x7efd4c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.929+0000 7efd51b1c700 1 -- 172.21.15.46:0/2784159979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c10d750 msgr2=0x7efd4c19dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.929+0000 7efd51b1c700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c10d750 0x7efd4c19dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.929+0000 7efd51b1c700 1 -- 172.21.15.46:0/2784159979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd4c069910 msgr2=0x7efd4c1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.929+0000 7efd51b1c700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd4c069910 0x7efd4c1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.929+0000 7efd51b1c700 1 -- 172.21.15.46:0/2784159979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd44009d00 con 0x7efd4c068ff0 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.929+0000 7efd51b1c700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c068ff0 0x7efd4c1994b0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7efd4800c9b0 tx=0x7efd4800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.929+0000 7efd42ffd700 1 -- 172.21.15.46:0/2784159979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48007b00 con 0x7efd4c068ff0 2024-03-18T20:43:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.929+0000 7efd53d80700 1 -- 172.21.15.46:0/2784159979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4c19e9e0 con 0x7efd4c068ff0 2024-03-18T20:43:39.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.929+0000 7efd42ffd700 1 -- 172.21.15.46:0/2784159979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd48007c60 con 0x7efd4c068ff0 2024-03-18T20:43:39.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.929+0000 7efd42ffd700 1 -- 172.21.15.46:0/2784159979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48018660 con 0x7efd4c068ff0 2024-03-18T20:43:39.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.929+0000 7efd53d80700 1 -- 172.21.15.46:0/2784159979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c1a6220 con 0x7efd4c068ff0 2024-03-18T20:43:39.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.933+0000 7efd42ffd700 1 -- 172.21.15.46:0/2784159979 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7efd4800f450 con 0x7efd4c068ff0 2024-03-18T20:43:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.933+0000 7efd42ffd700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efd380a1290 0x7efd380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.933+0000 7efd5ff15b80 1 Processor -- start 2024-03-18T20:43:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.933+0000 7efd42ffd700 1 -- 172.21.15.46:0/2784159979 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7efd480cccb0 con 0x7efd4c068ff0 2024-03-18T20:43:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.933+0000 7efd5131b700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efd380a1290 0x7efd380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5ff15b80 1 -- start start 2024-03-18T20:43:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5ff15b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e336ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5ff15b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 0x561e3370ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5ff15b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e3370b220 0x561e3370d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5131b700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efd380a1290 0x7efd380a3750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efd4c19aa00 tx=0x7efd3c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5ff15b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33659990 con 0x561e33700380 2024-03-18T20:43:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5ff15b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33657350 con 0x561e3370b220 2024-03-18T20:43:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5ff15b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33652c40 con 0x561e336ffa60 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5131b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 0x561e3370ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5131b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 0x561e3370ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59788/0 (socket says 172.21.15.46:59788) 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5131b700 1 -- 172.21.15.46:0/1565929178 learned_addr learned my addr 172.21.15.46:0/1565929178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd51b1c700 1 --2- 172.21.15.46:0/1565929178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e336ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5231d700 1 --2- 172.21.15.46:0/1565929178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e3370b220 0x561e3370d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5131b700 1 -- 172.21.15.46:0/1565929178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 msgr2=0x561e336ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5131b700 1 --2- 172.21.15.46:0/1565929178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e336ffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5131b700 1 -- 172.21.15.46:0/1565929178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e3370b220 msgr2=0x561e3370d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.934+0000 7efd5131b700 1 --2- 172.21.15.46:0/1565929178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e3370b220 0x561e3370d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.935+0000 7efd5131b700 1 -- 172.21.15.46:0/1565929178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e334d43b0 con 0x561e33700380 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.935+0000 7efd5131b700 1 --2- 172.21.15.46:0/1565929178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 0x561e3370ace0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7efd3c010e00 tx=0x7efd3c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea15b5f8c6688594 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.935+0000 7efd37fff700 1 -- 172.21.15.46:0/1565929178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c009960 con 0x561e33700380 2024-03-18T20:43:39.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.935+0000 7efd37fff700 1 -- 172.21.15.46:0/1565929178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd3c015070 con 0x561e33700380 2024-03-18T20:43:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.935+0000 7efd37fff700 1 -- 172.21.15.46:0/1565929178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c01c610 con 0x561e33700380 2024-03-18T20:43:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.936+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1565929178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 msgr2=0x561e3370ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.936+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/1565929178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 0x561e3370ace0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7efd3c010e00 tx=0x7efd3c013000 comp rx=0 tx=0).stop 2024-03-18T20:43:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.936+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1565929178 shutdown_connections 2024-03-18T20:43:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.936+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/1565929178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e3370b220 0x561e3370d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.936+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/1565929178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e336ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.936+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/1565929178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 0x561e3370ace0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.936+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1565929178 >> 172.21.15.46:0/1565929178 conn(0x561e33676220 msgr2=0x561e336fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.936+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1565929178 shutdown_connections 2024-03-18T20:43:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.936+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1565929178 wait complete. 2024-03-18T20:43:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.936+0000 7efd5ff15b80 1 -- start start 2024-03-18T20:43:39.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.936+0000 7efd5ff15b80 1 Processor -- start 2024-03-18T20:43:39.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5ff15b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e33707fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5ff15b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 0x561e33708510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5ff15b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e3370b220 0x561e33708a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5ff15b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e335eed80 con 0x561e33700380 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5ff15b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e335eea20 con 0x561e3370b220 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5ff15b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e334d66a0 con 0x561e336ffa60 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5ff15b80 10 client.0 _ll_register_callbacks cb 0x561e3345a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5131b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 0x561e33708510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5131b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 0x561e33708510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59800/0 (socket says 172.21.15.46:59800) 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5131b700 1 -- 172.21.15.46:0/1740487340 learned_addr learned my addr 172.21.15.46:0/1740487340 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5231d700 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e3370b220 0x561e33708a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd51b1c700 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e33707fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5131b700 1 -- 172.21.15.46:0/1740487340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 msgr2=0x561e33707fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5131b700 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e33707fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5131b700 1 -- 172.21.15.46:0/1740487340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e3370b220 msgr2=0x561e33708a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5131b700 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e3370b220 0x561e33708a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.937+0000 7efd5131b700 1 -- 172.21.15.46:0/1740487340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c010ae0 con 0x561e33700380 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd5131b700 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 0x561e33708510 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7efd3c010760 tx=0x7efd3c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd25c3583fe4346a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd357fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd357fa700 1 -- 172.21.15.46:0/1740487340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c02b070 con 0x561e33700380 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd357fa700 1 -- 172.21.15.46:0/1740487340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7efd3c017070 con 0x561e33700380 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1740487340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e336f3ad0 con 0x561e33700380 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd357fa700 1 -- 172.21.15.46:0/1740487340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd3c009940 con 0x561e33700380 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd5ff15b80 10 client.18195 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1740487340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x561e33480b60 con 0x561e33700380 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd357fa700 1 -- 172.21.15.46:0/1740487340 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd3c03d020 con 0x561e33700380 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd357fa700 1 -- 172.21.15.46:0/1740487340 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efd3c004a30 con 0x561e33700380 2024-03-18T20:43:39.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd5ff15b80 10 client.18195 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd5ff15b80 10 client.18195 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd5ff15b80 20 client.18195 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561e3371ad30 0x561e3371d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd5ff15b80 4 client.18195 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.938+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1740487340 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e3364b700 con 0x561e3371ad30 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.939+0000 7efd51b1c700 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561e3371ad30 0x561e3371d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.939+0000 7efd357fa700 10 client.18195 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.939+0000 7efd51b1c700 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561e3371ad30 0x561e3371d150 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1b738f4611efbc8 server_cookie=493b4919818c7f96 in_seq=0 out_seq=0 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd357fa700 1 -- 172.21.15.46:0/1740487340 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561e3364b700 con 0x561e3371ad30 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd357fa700 10 client.18195 handle_command_reply: tid=0 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd5ff15b80 1 client.18195 shutdown 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1740487340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 msgr2=0x561e33708510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 0x561e33708510 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7efd3c010760 tx=0x7efd3c01ec50 comp rx=0 tx=0).stop 2024-03-18T20:43:39.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1740487340 shutdown_connections 2024-03-18T20:43:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e3370b220 0x561e33708a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e33707fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561e3371ad30 0x561e3371d150 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/1740487340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e33700380 0x561e33708510 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd357fa700 0 client.18195 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:43:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1740487340 >> 172.21.15.46:0/1740487340 conn(0x561e33676220 msgr2=0x561e336fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1740487340 shutdown_connections 2024-03-18T20:43:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd5ff15b80 1 -- 172.21.15.46:0/1740487340 wait complete. 2024-03-18T20:43:39.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.940+0000 7efd5ff15b80 20 client.18195 trim_cache size 0 max 16384 2024-03-18T20:43:39.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.954+0000 7efd5ff15b80 1 Processor -- start 2024-03-18T20:43:39.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.954+0000 7efd5ff15b80 1 -- start start 2024-03-18T20:43:39.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.954+0000 7efd5ff15b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e3371d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.954+0000 7efd5ff15b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e33700380 0x561e33711e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.954+0000 7efd5ff15b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 0x561e33708870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.954+0000 7efd5ff15b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33709640 con 0x561e3370b220 2024-03-18T20:43:39.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.954+0000 7efd5ff15b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e336f85e0 con 0x561e33700380 2024-03-18T20:43:39.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.954+0000 7efd5ff15b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e336fd230 con 0x561e336ffa60 2024-03-18T20:43:39.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.954+0000 7efd5231d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 0x561e33708870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.954+0000 7efd51b1c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e3371d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.954+0000 7efd5231d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 0x561e33708870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59804/0 (socket says 172.21.15.46:59804) 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.954+0000 7efd5231d700 1 -- 172.21.15.46:0/11634636 learned_addr learned my addr 172.21.15.46:0/11634636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.955+0000 7efd5131b700 1 --2- 172.21.15.46:0/11634636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e33700380 0x561e33711e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.955+0000 7efd51b1c700 1 -- 172.21.15.46:0/11634636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e33700380 msgr2=0x561e33711e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.955+0000 7efd51b1c700 1 --2- 172.21.15.46:0/11634636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e33700380 0x561e33711e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.955+0000 7efd51b1c700 1 -- 172.21.15.46:0/11634636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 msgr2=0x561e33708870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.955+0000 7efd51b1c700 1 --2- 172.21.15.46:0/11634636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 0x561e33708870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.955+0000 7efd51b1c700 1 -- 172.21.15.46:0/11634636 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e33618ea0 con 0x561e336ffa60 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.955+0000 7efd5231d700 1 --2- 172.21.15.46:0/11634636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 0x561e33708870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.955+0000 7efd51b1c700 1 --2- 172.21.15.46:0/11634636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e3371d120 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efd480ca3e0 tx=0x7efd4809c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b6d4969fa2a2d14 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.955+0000 7efd40ff9700 1 -- 172.21.15.46:0/11634636 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4809dbd0 con 0x561e336ffa60 2024-03-18T20:43:39.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.955+0000 7efd40ff9700 1 -- 172.21.15.46:0/11634636 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd480b6070 con 0x561e336ffa60 2024-03-18T20:43:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.955+0000 7efd40ff9700 1 -- 172.21.15.46:0/11634636 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800f700 con 0x561e336ffa60 2024-03-18T20:43:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.956+0000 7efd5ff15b80 1 -- 172.21.15.46:0/11634636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 msgr2=0x561e3371d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.956+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/11634636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e3371d120 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efd480ca3e0 tx=0x7efd4809c040 comp rx=0 tx=0).stop 2024-03-18T20:43:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.956+0000 7efd5ff15b80 1 -- 172.21.15.46:0/11634636 shutdown_connections 2024-03-18T20:43:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.956+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/11634636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 0x561e33708870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.956+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/11634636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e336ffa60 0x561e3371d120 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.956+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/11634636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e33700380 0x561e33711e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.956+0000 7efd5ff15b80 1 -- 172.21.15.46:0/11634636 >> 172.21.15.46:0/11634636 conn(0x561e33676220 msgr2=0x561e336fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.956+0000 7efd5ff15b80 1 -- 172.21.15.46:0/11634636 shutdown_connections 2024-03-18T20:43:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.956+0000 7efd5ff15b80 1 -- 172.21.15.46:0/11634636 wait complete. 2024-03-18T20:43:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.956+0000 7efd5ff15b80 1 -- start start 2024-03-18T20:43:39.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.956+0000 7efd5ff15b80 1 Processor -- start 2024-03-18T20:43:39.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5ff15b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e336ffa60 0x561e33710040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5ff15b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e33700380 0x561e33710580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5ff15b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 0x561e33767b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5ff15b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33768090 con 0x561e3370b220 2024-03-18T20:43:39.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5ff15b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e33768200 con 0x561e336ffa60 2024-03-18T20:43:39.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5ff15b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e337683c0 con 0x561e33700380 2024-03-18T20:43:39.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5ff15b80 10 client.0 _ll_register_callbacks cb 0x7efd3c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:39.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5231d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 0x561e33767b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5231d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 0x561e33767b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59818/0 (socket says 172.21.15.46:59818) 2024-03-18T20:43:39.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5231d700 1 -- 172.21.15.46:0/799274711 learned_addr learned my addr 172.21.15.46:0/799274711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd51b1c700 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e336ffa60 0x561e33710040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5131b700 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e33700380 0x561e33710580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5231d700 1 -- 172.21.15.46:0/799274711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e33700380 msgr2=0x561e33710580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5231d700 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e33700380 0x561e33710580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5231d700 1 -- 172.21.15.46:0/799274711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e336ffa60 msgr2=0x561e33710040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5231d700 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e336ffa60 0x561e33710040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5231d700 1 -- 172.21.15.46:0/799274711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c110670 con 0x561e3370b220 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd5231d700 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 0x561e33767b50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7efd44015000 tx=0x7efd4400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e318f1776bdf01 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.957+0000 7efd357fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd357fa700 1 -- 172.21.15.46:0/799274711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4400e040 con 0x561e3370b220 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd357fa700 1 -- 172.21.15.46:0/799274711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7efd44003e10 con 0x561e3370b220 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd5ff15b80 1 -- 172.21.15.46:0/799274711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd3c015070 con 0x561e3370b220 2024-03-18T20:43:39.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd357fa700 1 -- 172.21.15.46:0/799274711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd440074e0 con 0x561e3370b220 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd5ff15b80 10 client.18201 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd5ff15b80 1 -- 172.21.15.46:0/799274711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7efd4c1a6220 con 0x561e3370b220 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd357fa700 1 -- 172.21.15.46:0/799274711 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd3c015070 con 0x561e3370b220 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd357fa700 1 -- 172.21.15.46:0/799274711 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efd4400b0d0 con 0x561e3370b220 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd5ff15b80 10 client.18201 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd5ff15b80 10 client.18201 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd5ff15b80 20 client.18201 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561e3376fc40 0x561e33772060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd5ff15b80 4 client.18201 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.958+0000 7efd5ff15b80 1 -- 172.21.15.46:0/799274711 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e334a1cf0 con 0x561e3376fc40 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd51b1c700 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561e3376fc40 0x561e33772060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd357fa700 10 client.18201 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd51b1c700 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561e3376fc40 0x561e33772060 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2bf86a8fd4572e07 server_cookie=44a538fae3ebf4eb in_seq=0 out_seq=0 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd357fa700 1 -- 172.21.15.46:0/799274711 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e334a1cf0 con 0x561e3376fc40 2024-03-18T20:43:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd357fa700 10 client.18201 handle_command_reply: tid=0 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd5ff15b80 1 client.18201 shutdown 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd5ff15b80 1 -- 172.21.15.46:0/799274711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 msgr2=0x561e33767b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 0x561e33767b50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7efd44015000 tx=0x7efd4400be20 comp rx=0 tx=0).stop 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd5ff15b80 1 -- 172.21.15.46:0/799274711 shutdown_connections 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x561e3376fc40 0x561e33772060 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e3370b220 0x561e33767b50 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561e336ffa60 0x561e33710040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd357fa700 0 client.18201 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd5ff15b80 1 --2- 172.21.15.46:0/799274711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e33700380 0x561e33710580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd5ff15b80 1 -- 172.21.15.46:0/799274711 >> 172.21.15.46:0/799274711 conn(0x561e33676220 msgr2=0x561e337032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd5ff15b80 1 -- 172.21.15.46:0/799274711 shutdown_connections 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.959+0000 7efd5ff15b80 1 -- 172.21.15.46:0/799274711 wait complete. 2024-03-18T20:43:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd5ff15b80 20 client.18201 trim_cache size 0 max 16384 2024-03-18T20:43:39.969 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:43:39.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd53d80700 1 -- 172.21.15.46:0/2784159979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efd380a1290 msgr2=0x7efd380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd53d80700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efd380a1290 0x7efd380a3750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efd4c19aa00 tx=0x7efd3c009260 comp rx=0 tx=0).stop 2024-03-18T20:43:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd53d80700 1 -- 172.21.15.46:0/2784159979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c068ff0 msgr2=0x7efd4c1994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd53d80700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c068ff0 0x7efd4c1994b0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7efd4800c9b0 tx=0x7efd4800ccc0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd53d80700 1 -- 172.21.15.46:0/2784159979 shutdown_connections 2024-03-18T20:43:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd53d80700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efd380a1290 0x7efd380a3750 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd53d80700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd4c068ff0 0x7efd4c1994b0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd53d80700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd4c10d750 0x7efd4c19dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd53d80700 1 --2- 172.21.15.46:0/2784159979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd4c069910 0x7efd4c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd53d80700 1 -- 172.21.15.46:0/2784159979 >> 172.21.15.46:0/2784159979 conn(0x7efd4c076d80 msgr2=0x7efd4c10ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd53d80700 1 -- 172.21.15.46:0/2784159979 shutdown_connections 2024-03-18T20:43:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:39.960+0000 7efd53d80700 1 -- 172.21.15.46:0/2784159979 wait complete. 2024-03-18T20:43:39.976 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:43:39.976 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:43:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.111+0000 7f6cda4ec700 1 -- 172.21.15.46:0/395557256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ffe50 msgr2=0x7f6cd4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.111+0000 7f6cda4ec700 1 --2- 172.21.15.46:0/395557256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ffe50 0x7f6cd4100230 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009990 tx=0x7f6cc4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.112+0000 7f6cda4ec700 1 -- 172.21.15.46:0/395557256 shutdown_connections 2024-03-18T20:43:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.112+0000 7f6cda4ec700 1 --2- 172.21.15.46:0/395557256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ffe50 0x7f6cd4100230 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.112+0000 7f6cda4ec700 1 --2- 172.21.15.46:0/395557256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd4104f60 0x7f6cd410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.112+0000 7f6cda4ec700 1 --2- 172.21.15.46:0/395557256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4100800 0x7f6cd4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.112+0000 7f6cda4ec700 1 -- 172.21.15.46:0/395557256 >> 172.21.15.46:0/395557256 conn(0x7f6cd4076d40 msgr2=0x7f6cd4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:40.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.112+0000 7f6cda4ec700 1 -- 172.21.15.46:0/395557256 shutdown_connections 2024-03-18T20:43:40.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.112+0000 7f6cda4ec700 1 -- 172.21.15.46:0/395557256 wait complete. 2024-03-18T20:43:40.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.112+0000 7f6cda4ec700 1 Processor -- start 2024-03-18T20:43:40.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.112+0000 7f6cda4ec700 1 -- start start 2024-03-18T20:43:40.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cda4ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ffe50 0x7f6cd4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cda4ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd4104f60 0x7f6cd4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cda4ec700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4198ae0 0x7f6cd41a3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cda4ec700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd41a37a0 con 0x7f6cd40ffe50 2024-03-18T20:43:40.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cda4ec700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd41a3910 con 0x7f6cd4104f60 2024-03-18T20:43:40.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cda4ec700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd41a3ad0 con 0x7f6cd4198ae0 2024-03-18T20:43:40.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ffe50 0x7f6cd4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ffe50 0x7f6cd4198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59836/0 (socket says 172.21.15.46:59836) 2024-03-18T20:43:40.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd3fff700 1 -- 172.21.15.46:0/2862233439 learned_addr learned my addr 172.21.15.46:0/2862233439 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:40.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd37fe700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd4104f60 0x7f6cd4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd8a89700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4198ae0 0x7f6cd41a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd3fff700 1 -- 172.21.15.46:0/2862233439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4198ae0 msgr2=0x7f6cd41a3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4198ae0 0x7f6cd41a3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd3fff700 1 -- 172.21.15.46:0/2862233439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd4104f60 msgr2=0x7f6cd4198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd4104f60 0x7f6cd4198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd3fff700 1 -- 172.21.15.46:0/2862233439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc40095f0 con 0x7f6cd40ffe50 2024-03-18T20:43:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd8a89700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4198ae0 0x7f6cd41a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:43:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ffe50 0x7f6cd4198010 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4005910 tx=0x7f6cc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.113+0000 7f6cd17fa700 1 -- 172.21.15.46:0/2862233439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc401d070 con 0x7f6cd40ffe50 2024-03-18T20:43:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.114+0000 7f6cd17fa700 1 -- 172.21.15.46:0/2862233439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc4005a00 con 0x7f6cd40ffe50 2024-03-18T20:43:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.114+0000 7f6cda4ec700 1 -- 172.21.15.46:0/2862233439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd41a3d50 con 0x7f6cd40ffe50 2024-03-18T20:43:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.114+0000 7f6cd17fa700 1 -- 172.21.15.46:0/2862233439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4022c20 con 0x7f6cd40ffe50 2024-03-18T20:43:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.114+0000 7f6cda4ec700 1 -- 172.21.15.46:0/2862233439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd41a4170 con 0x7f6cd40ffe50 2024-03-18T20:43:40.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.116+0000 7f6cd17fa700 1 -- 172.21.15.46:0/2862233439 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6cc400f670 con 0x7f6cd40ffe50 2024-03-18T20:43:40.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.117+0000 7f6cd17fa700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6cbc0a1290 0x7f6cbc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.117+0000 7f6cd37fe700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6cbc0a1290 0x7f6cbc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.117+0000 7f6cd17fa700 1 -- 172.21.15.46:0/2862233439 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f6cc40ce160 con 0x7f6cd40ffe50 2024-03-18T20:43:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.118+0000 7f6cd37fe700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6cbc0a1290 0x7f6cbc0a3750 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4068640 tx=0x7f6cc800b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.118+0000 7f6ce6681b80 1 Processor -- start 2024-03-18T20:43:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.118+0000 7f6ce6681b80 1 -- start start 2024-03-18T20:43:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.118+0000 7f6ce6681b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7260a60 0x55a5e7260e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.118+0000 7f6ce6681b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7261380 0x55a5e726bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.118+0000 7f6ce6681b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e726e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.118+0000 7f6ce6681b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e71ba990 con 0x55a5e726c220 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.118+0000 7f6ce6681b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e71b8350 con 0x55a5e7260a60 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.118+0000 7f6ce6681b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e71b3c40 con 0x55a5e7261380 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.118+0000 7f6cd8a89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e726e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cd8a89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e726e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59838/0 (socket says 172.21.15.46:59838) 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cd8a89700 1 -- 172.21.15.46:0/3463311863 learned_addr learned my addr 172.21.15.46:0/3463311863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cd37fe700 1 --2- 172.21.15.46:0/3463311863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7261380 0x55a5e726bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/3463311863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7260a60 0x55a5e7260e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cd8a89700 1 -- 172.21.15.46:0/3463311863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7261380 msgr2=0x55a5e726bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cd8a89700 1 --2- 172.21.15.46:0/3463311863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7261380 0x55a5e726bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cd8a89700 1 -- 172.21.15.46:0/3463311863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7260a60 msgr2=0x55a5e7260e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cd8a89700 1 --2- 172.21.15.46:0/3463311863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7260a60 0x55a5e7260e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cd8a89700 1 -- 172.21.15.46:0/3463311863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5e70353b0 con 0x55a5e726c220 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cd8a89700 1 --2- 172.21.15.46:0/3463311863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e726e610 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009cd0 tx=0x7f6ccc007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa8e427e1932b08 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:40.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cba7fc700 1 -- 172.21.15.46:0/3463311863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc014400 con 0x55a5e726c220 2024-03-18T20:43:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cba7fc700 1 -- 172.21.15.46:0/3463311863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ccc014a20 con 0x55a5e726c220 2024-03-18T20:43:40.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.119+0000 7f6cba7fc700 1 -- 172.21.15.46:0/3463311863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc013a40 con 0x55a5e726c220 2024-03-18T20:43:40.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.120+0000 7f6ce6681b80 1 -- 172.21.15.46:0/3463311863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 msgr2=0x55a5e726e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.120+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/3463311863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e726e610 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009cd0 tx=0x7f6ccc007500 comp rx=0 tx=0).stop 2024-03-18T20:43:40.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.120+0000 7f6ce6681b80 1 -- 172.21.15.46:0/3463311863 shutdown_connections 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.120+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/3463311863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e726e610 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.120+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/3463311863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7260a60 0x55a5e7260e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.120+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/3463311863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7261380 0x55a5e726bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.120+0000 7f6ce6681b80 1 -- 172.21.15.46:0/3463311863 >> 172.21.15.46:0/3463311863 conn(0x55a5e71d7220 msgr2=0x55a5e725d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.120+0000 7f6ce6681b80 1 -- 172.21.15.46:0/3463311863 shutdown_connections 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.120+0000 7f6ce6681b80 1 -- 172.21.15.46:0/3463311863 wait complete. 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.121+0000 7f6ce6681b80 1 -- start start 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.121+0000 7f6ce6681b80 1 Processor -- start 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.121+0000 7f6ce6681b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 0x55a5e726bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.121+0000 7f6ce6681b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7261380 0x55a5e7269010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.121+0000 7f6ce6681b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e7269750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.121+0000 7f6ce6681b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e714fd80 con 0x55a5e726c220 2024-03-18T20:43:40.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.121+0000 7f6ce6681b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e714fa20 con 0x55a5e7261380 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.121+0000 7f6ce6681b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e70376a0 con 0x55a5e7260a60 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.121+0000 7f6ce6681b80 10 client.0 _ll_register_callbacks cb 0x55a5e6fbb3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.121+0000 7f6cd3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 0x55a5e726bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.121+0000 7f6cd3fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 0x55a5e726bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33054/0 (socket says 172.21.15.46:33054) 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.121+0000 7f6cd3fff700 1 -- 172.21.15.46:0/1475028084 learned_addr learned my addr 172.21.15.46:0/1475028084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.122+0000 7f6cd8a89700 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e7269750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.122+0000 7f6cd37fe700 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7261380 0x55a5e7269010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.122+0000 7f6cd3fff700 1 -- 172.21.15.46:0/1475028084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7261380 msgr2=0x55a5e7269010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.122+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7261380 0x55a5e7269010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.122+0000 7f6cd3fff700 1 -- 172.21.15.46:0/1475028084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 msgr2=0x55a5e7269750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.122+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e7269750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.122+0000 7f6cd3fff700 1 -- 172.21.15.46:0/1475028084 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc009900 con 0x55a5e7260a60 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.122+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 0x55a5e726bc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6cc409b270 tx=0x7f6cc409ac40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=153be15a47a0cb18 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.123+0000 7f6cb37fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:43:40.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.123+0000 7f6cb37fe700 1 -- 172.21.15.46:0/1475028084 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc409b740 con 0x55a5e7260a60 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.123+0000 7f6cb37fe700 1 -- 172.21.15.46:0/1475028084 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f6cc409b2e0 con 0x55a5e7260a60 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.123+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1475028084 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5e7254ad0 con 0x55a5e7260a60 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.123+0000 7f6cb37fe700 1 -- 172.21.15.46:0/1475028084 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cc409f110 con 0x55a5e7260a60 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.123+0000 7f6ce6681b80 10 client.27886 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.123+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1475028084 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55a5e6fe1b60 con 0x55a5e7260a60 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.123+0000 7f6cb37fe700 1 -- 172.21.15.46:0/1475028084 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a5e7254ad0 con 0x55a5e7260a60 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.123+0000 7f6cb37fe700 1 -- 172.21.15.46:0/1475028084 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6cc40b3020 con 0x55a5e7260a60 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.124+0000 7f6ce6681b80 10 client.27886 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.124+0000 7f6ce6681b80 10 client.27886 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.124+0000 7f6ce6681b80 20 client.27886 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.124+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a5e727bad0 0x55a5e727dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.124+0000 7f6ce6681b80 4 client.27886 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.124+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1475028084 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5e71ac700 con 0x55a5e727bad0 2024-03-18T20:43:40.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.124+0000 7f6cd37fe700 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a5e727bad0 0x55a5e727dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.124+0000 7f6cb37fe700 10 client.27886 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:43:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.124+0000 7f6cd37fe700 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a5e727bad0 0x55a5e727dec0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcbd3e66e5a58d90 server_cookie=65655894aba148ad in_seq=0 out_seq=0 2024-03-18T20:43:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6cb37fe700 1 -- 172.21.15.46:0/1475028084 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a5e71ac700 con 0x55a5e727bad0 2024-03-18T20:43:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6cb37fe700 10 client.27886 handle_command_reply: tid=0 2024-03-18T20:43:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6ce6681b80 1 client.27886 shutdown 2024-03-18T20:43:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1475028084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 msgr2=0x55a5e726bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 0x55a5e726bc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6cc409b270 tx=0x7f6cc409ac40 comp rx=0 tx=0).stop 2024-03-18T20:43:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1475028084 shutdown_connections 2024-03-18T20:43:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a5e727bad0 0x55a5e727dec0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e7269750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 0x55a5e726bc90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6cb37fe700 0 client.27886 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:43:40.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/1475028084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7261380 0x55a5e7269010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1475028084 >> 172.21.15.46:0/1475028084 conn(0x55a5e71d7220 msgr2=0x55a5e725cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:40.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1475028084 shutdown_connections 2024-03-18T20:43:40.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.125+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1475028084 wait complete. 2024-03-18T20:43:40.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.126+0000 7f6ce6681b80 20 client.27886 trim_cache size 0 max 16384 2024-03-18T20:43:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.139+0000 7f6ce6681b80 1 Processor -- start 2024-03-18T20:43:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.139+0000 7f6ce6681b80 1 -- start start 2024-03-18T20:43:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.139+0000 7f6ce6681b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e7260a60 0x55a5e72639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.139+0000 7f6ce6681b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7261380 0x55a5e72688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.139+0000 7f6ce6681b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e726c220 0x55a5e7268e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.139+0000 7f6ce6681b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e72595e0 con 0x55a5e7260a60 2024-03-18T20:43:40.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.139+0000 7f6ce6681b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e7256f80 con 0x55a5e726c220 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.139+0000 7f6ce6681b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e72623d0 con 0x55a5e7261380 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cd3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e7260a60 0x55a5e72639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cd3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e7260a60 0x55a5e72639c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59850/0 (socket says 172.21.15.46:59850) 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cd3fff700 1 -- 172.21.15.46:0/1792447377 learned_addr learned my addr 172.21.15.46:0/1792447377 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cd8a89700 1 --2- 172.21.15.46:0/1792447377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e726c220 0x55a5e7268e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cd37fe700 1 --2- 172.21.15.46:0/1792447377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7261380 0x55a5e72688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cd3fff700 1 -- 172.21.15.46:0/1792447377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7261380 msgr2=0x55a5e72688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/1792447377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7261380 0x55a5e72688d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cd3fff700 1 -- 172.21.15.46:0/1792447377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e726c220 msgr2=0x55a5e7268e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/1792447377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e726c220 0x55a5e7268e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cd3fff700 1 -- 172.21.15.46:0/1792447377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5e7179ea0 con 0x55a5e7260a60 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cd8a89700 1 --2- 172.21.15.46:0/1792447377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e726c220 0x55a5e7268e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/1792447377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e7260a60 0x55a5e72639c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40cbe90 tx=0x7f6cc401a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d577c97700bb8cda server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:40.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cbaffd700 1 -- 172.21.15.46:0/1792447377 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc409b2e0 con 0x55a5e7260a60 2024-03-18T20:43:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cbaffd700 1 -- 172.21.15.46:0/1792447377 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc409f110 con 0x55a5e7260a60 2024-03-18T20:43:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.140+0000 7f6cbaffd700 1 -- 172.21.15.46:0/1792447377 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc409b7f0 con 0x55a5e7260a60 2024-03-18T20:43:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.141+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1792447377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e7260a60 msgr2=0x55a5e72639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.141+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/1792447377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e7260a60 0x55a5e72639c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40cbe90 tx=0x7f6cc401a2c0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.141+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1792447377 shutdown_connections 2024-03-18T20:43:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.141+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/1792447377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e7260a60 0x55a5e72639c0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.141+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/1792447377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e726c220 0x55a5e7268e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.141+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/1792447377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7261380 0x55a5e72688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.141+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1792447377 >> 172.21.15.46:0/1792447377 conn(0x55a5e71d7220 msgr2=0x55a5e7264320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.141+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1792447377 shutdown_connections 2024-03-18T20:43:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.141+0000 7f6ce6681b80 1 -- 172.21.15.46:0/1792447377 wait complete. 2024-03-18T20:43:40.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.141+0000 7f6ce6681b80 1 -- start start 2024-03-18T20:43:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6ce6681b80 1 Processor -- start 2024-03-18T20:43:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6ce6681b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 0x55a5e72707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6ce6681b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7261380 0x55a5e7270cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6ce6681b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e7271230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6ce6681b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e72ce9c0 con 0x55a5e726c220 2024-03-18T20:43:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6ce6681b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e72ceb30 con 0x55a5e7261380 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6ce6681b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e72cecf0 con 0x55a5e7260a60 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6ce6681b80 10 client.0 _ll_register_callbacks cb 0x7f6cc409ae20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6cd8a89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e7271230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6cd8a89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e7271230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59856/0 (socket says 172.21.15.46:59856) 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6cd8a89700 1 -- 172.21.15.46:0/2656979912 learned_addr learned my addr 172.21.15.46:0/2656979912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 0x55a5e72707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.142+0000 7f6cd37fe700 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7261380 0x55a5e7270cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cd8a89700 1 -- 172.21.15.46:0/2656979912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 msgr2=0x55a5e72707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cd8a89700 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 0x55a5e72707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cd8a89700 1 -- 172.21.15.46:0/2656979912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7261380 msgr2=0x55a5e7270cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cd8a89700 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7261380 0x55a5e7270cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cd8a89700 1 -- 172.21.15.46:0/2656979912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5e6fe1b60 con 0x55a5e726c220 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cd37fe700 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7261380 0x55a5e7270cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:40.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 0x55a5e72707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cd8a89700 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e7271230 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc007850 tx=0x7f6ccc004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b2c93cf0aa0e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cb37fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cb37fe700 1 -- 172.21.15.46:0/2656979912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0095e0 con 0x55a5e726c220 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cb37fe700 1 -- 172.21.15.46:0/2656979912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f6ccc0133a0 con 0x55a5e726c220 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6ce6681b80 1 -- 172.21.15.46:0/2656979912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ccc014a20 con 0x55a5e726c220 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cb37fe700 1 -- 172.21.15.46:0/2656979912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ccc014810 con 0x55a5e726c220 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6ce6681b80 10 client.18219 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6ce6681b80 1 -- 172.21.15.46:0/2656979912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55a5e7039cd0 con 0x55a5e726c220 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.143+0000 7f6cb37fe700 1 -- 172.21.15.46:0/2656979912 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ccc033080 con 0x55a5e726c220 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6cb37fe700 1 -- 172.21.15.46:0/2656979912 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6ccc025070 con 0x55a5e726c220 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6ce6681b80 10 client.18219 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6ce6681b80 10 client.18219 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6ce6681b80 20 client.18219 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a5e72d0460 0x55a5e72d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6ce6681b80 4 client.18219 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6ce6681b80 1 -- 172.21.15.46:0/2656979912 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a5e7002cf0 con 0x55a5e72d0460 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a5e72d0460 0x55a5e72d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6cb37fe700 10 client.18219 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6cd3fff700 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a5e72d0460 0x55a5e72d2850 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7c5b9594d0430a9 server_cookie=f6b726e109da2f3e in_seq=0 out_seq=0 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6cb37fe700 1 -- 172.21.15.46:0/2656979912 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a5e7002cf0 con 0x55a5e72d0460 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6cb37fe700 10 client.18219 handle_command_reply: tid=0 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.144+0000 7f6ce6681b80 1 client.18219 shutdown 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6ce6681b80 1 -- 172.21.15.46:0/2656979912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 msgr2=0x55a5e7271230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e7271230 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc007850 tx=0x7f6ccc004ca0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6ce6681b80 1 -- 172.21.15.46:0/2656979912 shutdown_connections 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55a5e72d0460 0x55a5e72d2850 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5e726c220 0x55a5e7271230 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6cb37fe700 0 client.18219 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:43:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a5e7260a60 0x55a5e72707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6ce6681b80 1 --2- 172.21.15.46:0/2656979912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5e7261380 0x55a5e7270cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6ce6681b80 1 -- 172.21.15.46:0/2656979912 >> 172.21.15.46:0/2656979912 conn(0x55a5e71d7220 msgr2=0x55a5e725ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6ce6681b80 1 -- 172.21.15.46:0/2656979912 shutdown_connections 2024-03-18T20:43:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6ce6681b80 1 -- 172.21.15.46:0/2656979912 wait complete. 2024-03-18T20:43:40.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6ce6681b80 20 client.18219 trim_cache size 0 max 16384 2024-03-18T20:43:40.154 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:43:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6cda4ec700 1 -- 172.21.15.46:0/2862233439 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6cbc0a1290 msgr2=0x7f6cbc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6cda4ec700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6cbc0a1290 0x7f6cbc0a3750 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4068640 tx=0x7f6cc800b410 comp rx=0 tx=0).stop 2024-03-18T20:43:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6cda4ec700 1 -- 172.21.15.46:0/2862233439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ffe50 msgr2=0x7f6cd4198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.145+0000 7f6cda4ec700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ffe50 0x7f6cd4198010 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4005910 tx=0x7f6cc4004730 comp rx=0 tx=0).stop 2024-03-18T20:43:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.146+0000 7f6cda4ec700 1 -- 172.21.15.46:0/2862233439 shutdown_connections 2024-03-18T20:43:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.146+0000 7f6cda4ec700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6cbc0a1290 0x7f6cbc0a3750 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.146+0000 7f6cda4ec700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd40ffe50 0x7f6cd4198010 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.146+0000 7f6cda4ec700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cd4198ae0 0x7f6cd41a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.146+0000 7f6cda4ec700 1 --2- 172.21.15.46:0/2862233439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd4104f60 0x7f6cd4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.146+0000 7f6cda4ec700 1 -- 172.21.15.46:0/2862233439 >> 172.21.15.46:0/2862233439 conn(0x7f6cd4076d40 msgr2=0x7f6cd40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:40.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.146+0000 7f6cda4ec700 1 -- 172.21.15.46:0/2862233439 shutdown_connections 2024-03-18T20:43:40.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.146+0000 7f6cda4ec700 1 -- 172.21.15.46:0/2862233439 wait complete. 2024-03-18T20:43:40.162 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:43:40.162 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:43:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.308+0000 7f7b61b70700 1 -- 172.21.15.46:0/800481734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b5c1060b0 msgr2=0x7f7b5c10dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.308+0000 7f7b61b70700 1 --2- 172.21.15.46:0/800481734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b5c1060b0 0x7f7b5c10dc30 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400b0d0 tx=0x7f7b5400b3e0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.308+0000 7f7b61b70700 1 -- 172.21.15.46:0/800481734 shutdown_connections 2024-03-18T20:43:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.308+0000 7f7b61b70700 1 --2- 172.21.15.46:0/800481734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b5c1060b0 0x7f7b5c10dc30 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.308+0000 7f7b61b70700 1 --2- 172.21.15.46:0/800481734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b5c069260 0x7f7b5c069640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.308+0000 7f7b61b70700 1 --2- 172.21.15.46:0/800481734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b5c069c10 0x7f7b5c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.308+0000 7f7b61b70700 1 -- 172.21.15.46:0/800481734 >> 172.21.15.46:0/800481734 conn(0x7f7b5c076d80 msgr2=0x7f7b5c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.308+0000 7f7b61b70700 1 -- 172.21.15.46:0/800481734 shutdown_connections 2024-03-18T20:43:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.308+0000 7f7b61b70700 1 -- 172.21.15.46:0/800481734 wait complete. 2024-03-18T20:43:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.309+0000 7f7b61b70700 1 Processor -- start 2024-03-18T20:43:40.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.309+0000 7f7b61b70700 1 -- start start 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b61b70700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b5c069260 0x7f7b5c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b61b70700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b5c1060b0 0x7f7b5c1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b61b70700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b5c19bb70 0x7f7b5c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b61b70700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c19c5f0 con 0x7f7b5c1060b0 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b61b70700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c19c760 con 0x7f7b5c19bb70 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b61b70700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c19c920 con 0x7f7b5c069260 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b5affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b5c1060b0 0x7f7b5c1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b5affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b5c1060b0 0x7f7b5c1977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59876/0 (socket says 172.21.15.46:59876) 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b5affd700 1 -- 172.21.15.46:0/3666698832 learned_addr learned my addr 172.21.15.46:0/3666698832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b5c069260 0x7f7b5c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b5bfff700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b5c19bb70 0x7f7b5c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b5affd700 1 -- 172.21.15.46:0/3666698832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b5c069260 msgr2=0x7f7b5c197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b5affd700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b5c069260 0x7f7b5c197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b5affd700 1 -- 172.21.15.46:0/3666698832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b5c19bb70 msgr2=0x7f7b5c19bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b5affd700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b5c19bb70 0x7f7b5c19bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b5affd700 1 -- 172.21.15.46:0/3666698832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b54009d00 con 0x7f7b5c1060b0 2024-03-18T20:43:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.310+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b5c069260 0x7f7b5c197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.311+0000 7f7b5affd700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b5c1060b0 0x7f7b5c1977d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400d980 tx=0x7f7b4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.311+0000 7f7b58ff9700 1 -- 172.21.15.46:0/3666698832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4400f890 con 0x7f7b5c1060b0 2024-03-18T20:43:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.311+0000 7f7b58ff9700 1 -- 172.21.15.46:0/3666698832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b44010460 con 0x7f7b5c1060b0 2024-03-18T20:43:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.311+0000 7f7b58ff9700 1 -- 172.21.15.46:0/3666698832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4400e610 con 0x7f7b5c1060b0 2024-03-18T20:43:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.311+0000 7f7b61b70700 1 -- 172.21.15.46:0/3666698832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5c1a3e80 con 0x7f7b5c1060b0 2024-03-18T20:43:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.311+0000 7f7b61b70700 1 -- 172.21.15.46:0/3666698832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5c1a42b0 con 0x7f7b5c1060b0 2024-03-18T20:43:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.314+0000 7f7b58ff9700 1 -- 172.21.15.46:0/3666698832 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7b440105d0 con 0x7f7b5c1060b0 2024-03-18T20:43:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.315+0000 7f7b58ff9700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7b480a1290 0x7f7b480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.315+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7b480a1290 0x7f7b480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.315+0000 7f7b58ff9700 1 -- 172.21.15.46:0/3666698832 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7b440cc630 con 0x7f7b5c1060b0 2024-03-18T20:43:40.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.315+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7b480a1290 0x7f7b480a3750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c005950 tx=0x7f7b4c00b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.316+0000 7f7b6dd05b80 1 Processor -- start 2024-03-18T20:43:40.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.316+0000 7f7b6dd05b80 1 -- start start 2024-03-18T20:43:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b6dd05b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6953e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b6dd05b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6954380 0x55e8e69630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b6dd05b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6963630 0x55e8e6965a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b6dd05b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e68ad990 con 0x55e8e6954380 2024-03-18T20:43:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b6dd05b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e68ab350 con 0x55e8e6953a60 2024-03-18T20:43:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b6dd05b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e68a6c40 con 0x55e8e6963630 2024-03-18T20:43:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b5b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6953e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b5affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6954380 0x55e8e69630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b5b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6953e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54476/0 (socket says 172.21.15.46:54476) 2024-03-18T20:43:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6963630 0x55e8e6965a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b5b7fe700 1 -- 172.21.15.46:0/3838666210 learned_addr learned my addr 172.21.15.46:0/3838666210 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b5b7fe700 1 -- 172.21.15.46:0/3838666210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6963630 msgr2=0x55e8e6965a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/3838666210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6963630 0x55e8e6965a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b5b7fe700 1 -- 172.21.15.46:0/3838666210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6954380 msgr2=0x55e8e69630f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/3838666210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6954380 0x55e8e69630f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b5b7fe700 1 -- 172.21.15.46:0/3838666210 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8e67283b0 con 0x55e8e6953a60 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.317+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/3838666210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6953e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c011790 tx=0x7f7b4c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f042697d6f317f5c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.318+0000 7f7b51ffb700 1 -- 172.21.15.46:0/3838666210 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c01e620 con 0x55e8e6953a60 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.318+0000 7f7b51ffb700 1 -- 172.21.15.46:0/3838666210 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b4c015070 con 0x55e8e6953a60 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.318+0000 7f7b51ffb700 1 -- 172.21.15.46:0/3838666210 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c01ca50 con 0x55e8e6953a60 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.318+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/3838666210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 msgr2=0x55e8e6953e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.318+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/3838666210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6953e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c011790 tx=0x7f7b4c013000 comp rx=0 tx=0).stop 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.318+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/3838666210 shutdown_connections 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.318+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/3838666210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6963630 0x55e8e6965a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.318+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/3838666210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6953e40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.318+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/3838666210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6954380 0x55e8e69630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.318+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/3838666210 >> 172.21.15.46:0/3838666210 conn(0x55e8e68ca220 msgr2=0x55e8e6950020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.319+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/3838666210 shutdown_connections 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.319+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/3838666210 wait complete. 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.319+0000 7f7b6dd05b80 1 -- start start 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.319+0000 7f7b6dd05b80 1 Processor -- start 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.319+0000 7f7b6dd05b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6962d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.319+0000 7f7b6dd05b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6954380 0x55e8e695bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.319+0000 7f7b6dd05b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e695c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.319+0000 7f7b6dd05b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e6842d80 con 0x55e8e6963630 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.319+0000 7f7b6dd05b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e6842a20 con 0x55e8e6953a60 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.319+0000 7f7b6dd05b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e672a6a0 con 0x55e8e6954380 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.320+0000 7f7b6dd05b80 10 client.0 _ll_register_callbacks cb 0x55e8e66ae3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.320+0000 7f7b5b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6962d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.320+0000 7f7b5b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6962d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54482/0 (socket says 172.21.15.46:54482) 2024-03-18T20:43:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.320+0000 7f7b5b7fe700 1 -- 172.21.15.46:0/2398738869 learned_addr learned my addr 172.21.15.46:0/2398738869 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.320+0000 7f7b5bfff700 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e695c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.320+0000 7f7b5affd700 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6954380 0x55e8e695bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.320+0000 7f7b5affd700 1 -- 172.21.15.46:0/2398738869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 msgr2=0x55e8e6962d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.320+0000 7f7b5affd700 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6962d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.320+0000 7f7b5affd700 1 -- 172.21.15.46:0/2398738869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 msgr2=0x55e8e695c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.320+0000 7f7b5affd700 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e695c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.320+0000 7f7b5affd700 1 -- 172.21.15.46:0/2398738869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4c011440 con 0x55e8e6954380 2024-03-18T20:43:40.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.329+0000 7f7b5affd700 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6954380 0x55e8e695bfb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b440c9d60 tx=0x7f7b44099370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea992750af67518b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.330+0000 7f7b3affd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:43:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.330+0000 7f7b3affd700 1 -- 172.21.15.46:0/2398738869 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b440a1ba0 con 0x55e8e6954380 2024-03-18T20:43:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.330+0000 7f7b3affd700 1 -- 172.21.15.46:0/2398738869 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7b4409d120 con 0x55e8e6954380 2024-03-18T20:43:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.330+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/2398738869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8e6947ad0 con 0x55e8e6954380 2024-03-18T20:43:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.331+0000 7f7b3affd700 1 -- 172.21.15.46:0/2398738869 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b4409d2e0 con 0x55e8e6954380 2024-03-18T20:43:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.331+0000 7f7b6dd05b80 10 client.27904 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.331+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/2398738869 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55e8e66d4b60 con 0x55e8e6954380 2024-03-18T20:43:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.332+0000 7f7b3affd700 1 -- 172.21.15.46:0/2398738869 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e8e6947ad0 con 0x55e8e6954380 2024-03-18T20:43:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.332+0000 7f7b3affd700 1 -- 172.21.15.46:0/2398738869 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7b44098d30 con 0x55e8e6954380 2024-03-18T20:43:40.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.336+0000 7f7b6dd05b80 10 client.27904 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:40.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.336+0000 7f7b6dd05b80 10 client.27904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:40.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.336+0000 7f7b6dd05b80 20 client.27904 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:40.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.336+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8e6977780 0x55e8e6979ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.336+0000 7f7b6dd05b80 4 client.27904 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:40.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.336+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/2398738869 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8e689f700 con 0x55e8e6977780 2024-03-18T20:43:40.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.339+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8e6977780 0x55e8e6979ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.341+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8e6977780 0x55e8e6979ba0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6c3c301016b620 server_cookie=b7622ef4406b37e in_seq=0 out_seq=0 2024-03-18T20:43:40.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.341+0000 7f7b3affd700 10 client.27904 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:40.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b3affd700 1 -- 172.21.15.46:0/2398738869 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e8e689f700 con 0x55e8e6977780 2024-03-18T20:43:40.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b3affd700 10 client.27904 handle_command_reply: tid=0 2024-03-18T20:43:40.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b6dd05b80 1 client.27904 shutdown 2024-03-18T20:43:40.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/2398738869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6954380 msgr2=0x55e8e695bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6954380 0x55e8e695bfb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b440c9d60 tx=0x7f7b44099370 comp rx=0 tx=0).stop 2024-03-18T20:43:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/2398738869 shutdown_connections 2024-03-18T20:43:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8e6977780 0x55e8e6979ba0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e695c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6962d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b3affd700 0 client.27904 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/2398738869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6954380 0x55e8e695bfb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/2398738869 >> 172.21.15.46:0/2398738869 conn(0x55e8e68ca220 msgr2=0x55e8e694fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/2398738869 shutdown_connections 2024-03-18T20:43:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/2398738869 wait complete. 2024-03-18T20:43:40.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.344+0000 7f7b6dd05b80 20 client.27904 trim_cache size 0 max 16384 2024-03-18T20:43:40.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.360+0000 7f7b6dd05b80 1 Processor -- start 2024-03-18T20:43:40.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.360+0000 7f7b6dd05b80 1 -- start start 2024-03-18T20:43:40.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.360+0000 7f7b6dd05b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6962930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b6dd05b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6954380 0x55e8e69572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b6dd05b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e6957800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b6dd05b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e6960f80 con 0x55e8e6963630 2024-03-18T20:43:40.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b6dd05b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e694edd0 con 0x55e8e6953a60 2024-03-18T20:43:40.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b6dd05b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e695bd80 con 0x55e8e6954380 2024-03-18T20:43:40.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e6957800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e6957800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59914/0 (socket says 172.21.15.46:59914) 2024-03-18T20:43:40.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b5bfff700 1 -- 172.21.15.46:0/846151709 learned_addr learned my addr 172.21.15.46:0/846151709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:40.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b5affd700 1 --2- 172.21.15.46:0/846151709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6954380 0x55e8e69572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/846151709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6962930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b5bfff700 1 -- 172.21.15.46:0/846151709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6954380 msgr2=0x55e8e69572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b5bfff700 1 --2- 172.21.15.46:0/846151709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6954380 0x55e8e69572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b5bfff700 1 -- 172.21.15.46:0/846151709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 msgr2=0x55e8e6962930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b5bfff700 1 --2- 172.21.15.46:0/846151709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6962930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b5bfff700 1 -- 172.21.15.46:0/846151709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8e686cea0 con 0x55e8e6963630 2024-03-18T20:43:40.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b5bfff700 1 --2- 172.21.15.46:0/846151709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e6957800 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7b54007e40 tx=0x7f7b5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69a333250c9420 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:40.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.361+0000 7f7b527fc700 1 -- 172.21.15.46:0/846151709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5400e030 con 0x55e8e6963630 2024-03-18T20:43:40.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.362+0000 7f7b527fc700 1 -- 172.21.15.46:0/846151709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b540092d0 con 0x55e8e6963630 2024-03-18T20:43:40.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.362+0000 7f7b527fc700 1 -- 172.21.15.46:0/846151709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54005310 con 0x55e8e6963630 2024-03-18T20:43:40.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.362+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/846151709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 msgr2=0x55e8e6957800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.362+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/846151709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e6957800 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7b54007e40 tx=0x7f7b5400b040 comp rx=0 tx=0).stop 2024-03-18T20:43:40.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.362+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/846151709 shutdown_connections 2024-03-18T20:43:40.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.362+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/846151709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e6957800 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.362+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/846151709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6953a60 0x55e8e6962930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.362+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/846151709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6954380 0x55e8e69572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.362+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/846151709 >> 172.21.15.46:0/846151709 conn(0x55e8e68ca220 msgr2=0x55e8e695bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:40.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.362+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/846151709 shutdown_connections 2024-03-18T20:43:40.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.362+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/846151709 wait complete. 2024-03-18T20:43:40.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.363+0000 7f7b6dd05b80 1 -- start start 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.363+0000 7f7b6dd05b80 1 Processor -- start 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.363+0000 7f7b6dd05b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6953a60 0x55e8e6974000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.363+0000 7f7b6dd05b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6954380 0x55e8e6974540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.363+0000 7f7b6dd05b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e6970560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.363+0000 7f7b6dd05b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e6970c70 con 0x55e8e6963630 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.363+0000 7f7b6dd05b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e6970de0 con 0x55e8e6954380 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.363+0000 7f7b6dd05b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e6970fa0 con 0x55e8e6953a60 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.363+0000 7f7b6dd05b80 10 client.0 _ll_register_callbacks cb 0x55e8e66ae3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6953a60 0x55e8e6974000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6954380 0x55e8e6974540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e6970560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6953a60 0x55e8e6974000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33138/0 (socket says 172.21.15.46:33138) 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6954380 0x55e8e6974540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54498/0 (socket says 172.21.15.46:54498) 2024-03-18T20:43:40.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5b7fe700 1 -- 172.21.15.46:0/4089614381 learned_addr learned my addr 172.21.15.46:0/4089614381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5b7fe700 1 -- 172.21.15.46:0/4089614381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6954380 msgr2=0x55e8e6974540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6954380 0x55e8e6974540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5b7fe700 1 -- 172.21.15.46:0/4089614381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 msgr2=0x55e8e6970560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e6970560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5b7fe700 1 -- 172.21.15.46:0/4089614381 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5c10e470 con 0x55e8e6953a60 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5affd700 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6954380 0x55e8e6974540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5bfff700 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e6970560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b5b7fe700 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6953a60 0x55e8e6974000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c011d80 tx=0x7f7b4c005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b000c18ec346e210 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b3affd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b3affd700 1 -- 172.21.15.46:0/4089614381 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c035bf0 con 0x55e8e6953a60 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b3affd700 1 -- 172.21.15.46:0/4089614381 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7b4c017070 con 0x55e8e6953a60 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.364+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/4089614381 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b4c015070 con 0x55e8e6953a60 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.365+0000 7f7b3affd700 1 -- 172.21.15.46:0/4089614381 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b4c01c3b0 con 0x55e8e6953a60 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.365+0000 7f7b6dd05b80 10 client.27910 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.365+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/4089614381 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55e8e672ccd0 con 0x55e8e6953a60 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.365+0000 7f7b3affd700 1 -- 172.21.15.46:0/4089614381 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b4c015070 con 0x55e8e6953a60 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.365+0000 7f7b3affd700 1 -- 172.21.15.46:0/4089614381 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7b4c03b070 con 0x55e8e6953a60 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.365+0000 7f7b6dd05b80 10 client.27910 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.365+0000 7f7b6dd05b80 10 client.27910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.365+0000 7f7b6dd05b80 20 client.27910 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.365+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8e69cc810 0x55e8e69cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.365+0000 7f7b6dd05b80 4 client.27910 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.365+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/4089614381 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e8e66f5cf0 con 0x55e8e69cc810 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.365+0000 7f7b5affd700 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8e69cc810 0x55e8e69cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.366+0000 7f7b3affd700 10 client.27910 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.366+0000 7f7b5affd700 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8e69cc810 0x55e8e69cec30 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=776e7a0c8e656fdd server_cookie=d8e9a84f44aec478 in_seq=0 out_seq=0 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.366+0000 7f7b3affd700 1 -- 172.21.15.46:0/4089614381 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e8e66f5cf0 con 0x55e8e69cc810 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.366+0000 7f7b3affd700 10 client.27910 handle_command_reply: tid=0 2024-03-18T20:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.366+0000 7f7b6dd05b80 1 client.27910 shutdown 2024-03-18T20:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/4089614381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6953a60 msgr2=0x55e8e6974000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6953a60 0x55e8e6974000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c011d80 tx=0x7f7b4c005720 comp rx=0 tx=0).stop 2024-03-18T20:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/4089614381 shutdown_connections 2024-03-18T20:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8e69cc810 0x55e8e69cec30 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8e6963630 0x55e8e6970560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e6953a60 0x55e8e6974000 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b3affd700 0 client.27910 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b6dd05b80 1 --2- 172.21.15.46:0/4089614381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8e6954380 0x55e8e6974540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/4089614381 >> 172.21.15.46:0/4089614381 conn(0x55e8e68ca220 msgr2=0x55e8e695bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/4089614381 shutdown_connections 2024-03-18T20:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b6dd05b80 1 -- 172.21.15.46:0/4089614381 wait complete. 2024-03-18T20:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b6dd05b80 20 client.27910 trim_cache size 0 max 16384 2024-03-18T20:43:40.375 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b61b70700 1 -- 172.21.15.46:0/3666698832 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7b480a1290 msgr2=0x7f7b480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b61b70700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7b480a1290 0x7f7b480a3750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c005950 tx=0x7f7b4c00b410 comp rx=0 tx=0).stop 2024-03-18T20:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b61b70700 1 -- 172.21.15.46:0/3666698832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b5c1060b0 msgr2=0x7f7b5c1977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:40.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b61b70700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b5c1060b0 0x7f7b5c1977d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400d980 tx=0x7f7b4400dc90 comp rx=0 tx=0).stop 2024-03-18T20:43:40.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b61b70700 1 -- 172.21.15.46:0/3666698832 shutdown_connections 2024-03-18T20:43:40.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b61b70700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7b480a1290 0x7f7b480a3750 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b61b70700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b5c19bb70 0x7f7b5c19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b61b70700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b5c069260 0x7f7b5c197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b61b70700 1 --2- 172.21.15.46:0/3666698832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b5c1060b0 0x7f7b5c1977d0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:40.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.367+0000 7f7b61b70700 1 -- 172.21.15.46:0/3666698832 >> 172.21.15.46:0/3666698832 conn(0x7f7b5c076d80 msgr2=0x7f7b5c1071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:40.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.368+0000 7f7b61b70700 1 -- 172.21.15.46:0/3666698832 shutdown_connections 2024-03-18T20:43:40.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:40.368+0000 7f7b61b70700 1 -- 172.21.15.46:0/3666698832 wait complete. 2024-03-18T20:43:40.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:40 smithi073 ceph-mon[47384]: pgmap v232: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-03-18T20:43:40.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:40 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/427124494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:43:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:40 smithi155 ceph-mon[48344]: pgmap v232: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-03-18T20:43:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:40 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/427124494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:43:40.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:40 smithi046 ceph-mon[43495]: pgmap v232: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-03-18T20:43:40.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:40 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/427124494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:43:40.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 261459 3.50 MB/sec execute 275 sec latency 2744.014 ms 2024-03-18T20:43:41.384 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:43:41.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:43:41.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:43:41.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.535+0000 7ff973b75700 1 -- 172.21.15.46:0/3637908761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c0ffe50 msgr2=0x7ff96c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.535+0000 7ff973b75700 1 --2- 172.21.15.46:0/3637908761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c0ffe50 0x7ff96c100230 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff9680098f0 tx=0x7ff968009c00 comp rx=0 tx=0).stop 2024-03-18T20:43:41.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.535+0000 7ff973b75700 1 -- 172.21.15.46:0/3637908761 shutdown_connections 2024-03-18T20:43:41.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.535+0000 7ff973b75700 1 --2- 172.21.15.46:0/3637908761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c0ffe50 0x7ff96c100230 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.535+0000 7ff973b75700 1 --2- 172.21.15.46:0/3637908761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff96c104f60 0x7ff96c10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.535+0000 7ff973b75700 1 --2- 172.21.15.46:0/3637908761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff96c100800 0x7ff96c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.535+0000 7ff973b75700 1 -- 172.21.15.46:0/3637908761 >> 172.21.15.46:0/3637908761 conn(0x7ff96c076d40 msgr2=0x7ff96c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.535+0000 7ff973b75700 1 -- 172.21.15.46:0/3637908761 shutdown_connections 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.535+0000 7ff973b75700 1 -- 172.21.15.46:0/3637908761 wait complete. 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff973b75700 1 Processor -- start 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff973b75700 1 -- start start 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff973b75700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff96c0ffe50 0x7ff96c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff973b75700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff96c104f60 0x7ff96c1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff973b75700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c19bb70 0x7ff96c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff973b75700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c19c5f0 con 0x7ff96c19bb70 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff973b75700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c19c760 con 0x7ff96c0ffe50 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff973b75700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c19c920 con 0x7ff96c104f60 2024-03-18T20:43:41.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff972112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c19bb70 0x7ff96c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff972112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c19bb70 0x7ff96c19bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59942/0 (socket says 172.21.15.46:59942) 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff971911700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff96c0ffe50 0x7ff96c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff972112700 1 -- 172.21.15.46:0/2967508823 learned_addr learned my addr 172.21.15.46:0/2967508823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.536+0000 7ff971110700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff96c104f60 0x7ff96c1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.537+0000 7ff971911700 1 -- 172.21.15.46:0/2967508823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff96c104f60 msgr2=0x7ff96c1977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.537+0000 7ff971911700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff96c104f60 0x7ff96c1977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.537+0000 7ff971911700 1 -- 172.21.15.46:0/2967508823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c19bb70 msgr2=0x7ff96c19bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.537+0000 7ff971911700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c19bb70 0x7ff96c19bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.537+0000 7ff971911700 1 -- 172.21.15.46:0/2967508823 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff964009c40 con 0x7ff96c0ffe50 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.537+0000 7ff972112700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c19bb70 0x7ff96c19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.537+0000 7ff971911700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff96c0ffe50 0x7ff96c197290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff968009fd0 tx=0x7ff96800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.537+0000 7ff95effd700 1 -- 172.21.15.46:0/2967508823 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96801c070 con 0x7ff96c0ffe50 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.537+0000 7ff973b75700 1 -- 172.21.15.46:0/2967508823 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff968009520 con 0x7ff96c0ffe50 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.537+0000 7ff95effd700 1 -- 172.21.15.46:0/2967508823 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff96800b9c0 con 0x7ff96c0ffe50 2024-03-18T20:43:41.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.537+0000 7ff95effd700 1 -- 172.21.15.46:0/2967508823 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9680036c0 con 0x7ff96c0ffe50 2024-03-18T20:43:41.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.537+0000 7ff973b75700 1 -- 172.21.15.46:0/2967508823 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c1a4110 con 0x7ff96c0ffe50 2024-03-18T20:43:41.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.540+0000 7ff95effd700 1 -- 172.21.15.46:0/2967508823 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff968020030 con 0x7ff96c0ffe50 2024-03-18T20:43:41.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.540+0000 7ff95effd700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9580a1290 0x7ff9580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.540+0000 7ff95effd700 1 -- 172.21.15.46:0/2967508823 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff9680cda00 con 0x7ff96c0ffe50 2024-03-18T20:43:41.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.540+0000 7ff971110700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9580a1290 0x7ff9580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.541+0000 7ff971110700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9580a1290 0x7ff9580a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff96c1987e0 tx=0x7ff960009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:41.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.541+0000 7ff97fd0ab80 1 Processor -- start 2024-03-18T20:43:41.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.541+0000 7ff97fd0ab80 1 -- start start 2024-03-18T20:43:41.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff97fd0ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539bdae0 0x5597539bdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff97fd0ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539be400 0x5597539c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff97fd0ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 0x5597539cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff97fd0ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559753917a10 con 0x5597539be400 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff97fd0ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597539153d0 con 0x5597539bdae0 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff97fd0ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559753910cc0 con 0x5597539c92a0 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff971110700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539be400 0x5597539c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff971911700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539bdae0 0x5597539bdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff972112700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 0x5597539cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff971110700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539be400 0x5597539c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59954/0 (socket says 172.21.15.46:59954) 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff971110700 1 -- 172.21.15.46:0/1959074967 learned_addr learned my addr 172.21.15.46:0/1959074967 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff971911700 1 -- 172.21.15.46:0/1959074967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 msgr2=0x5597539cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff971911700 1 --2- 172.21.15.46:0/1959074967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 0x5597539cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff971911700 1 -- 172.21.15.46:0/1959074967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539be400 msgr2=0x5597539c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff971911700 1 --2- 172.21.15.46:0/1959074967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539be400 0x5597539c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff971911700 1 -- 172.21.15.46:0/1959074967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559753792430 con 0x5597539bdae0 2024-03-18T20:43:41.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.542+0000 7ff971911700 1 --2- 172.21.15.46:0/1959074967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539bdae0 0x5597539bdec0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff9680cb130 tx=0x7ff968009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66ea8b76452b63eb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.543+0000 7ff957fff700 1 -- 172.21.15.46:0/1959074967 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96809b280 con 0x5597539bdae0 2024-03-18T20:43:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.543+0000 7ff957fff700 1 -- 172.21.15.46:0/1959074967 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff96809ab80 con 0x5597539bdae0 2024-03-18T20:43:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.543+0000 7ff957fff700 1 -- 172.21.15.46:0/1959074967 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968097290 con 0x5597539bdae0 2024-03-18T20:43:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.543+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/1959074967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539bdae0 msgr2=0x5597539bdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.543+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/1959074967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539bdae0 0x5597539bdec0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff9680cb130 tx=0x7ff968009c00 comp rx=0 tx=0).stop 2024-03-18T20:43:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.543+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/1959074967 shutdown_connections 2024-03-18T20:43:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.543+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/1959074967 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 0x5597539cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.543+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/1959074967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539bdae0 0x5597539bdec0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.543+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/1959074967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539be400 0x5597539c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.543+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/1959074967 >> 172.21.15.46:0/1959074967 conn(0x5597539342a0 msgr2=0x5597539ba0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.543+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/1959074967 shutdown_connections 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.544+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/1959074967 wait complete. 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.544+0000 7ff97fd0ab80 1 -- start start 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.544+0000 7ff97fd0ab80 1 Processor -- start 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff97fd0ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 0x5597539c9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff97fd0ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x5597539c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff97fd0ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 0x5597539d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff97fd0ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597538ace00 con 0x5597539bdae0 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff97fd0ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597538acaa0 con 0x5597539be400 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff97fd0ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559753794720 con 0x5597539c92a0 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff97fd0ab80 10 client.0 _ll_register_callbacks cb 0x559753718420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff971911700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 0x5597539c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff971911700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 0x5597539c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59968/0 (socket says 172.21.15.46:59968) 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff971911700 1 -- 172.21.15.46:0/237570017 learned_addr learned my addr 172.21.15.46:0/237570017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff971110700 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x5597539c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff972112700 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 0x5597539d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff971911700 1 -- 172.21.15.46:0/237570017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 msgr2=0x5597539d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff971911700 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 0x5597539d4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff971911700 1 -- 172.21.15.46:0/237570017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 msgr2=0x5597539c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff971911700 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x5597539c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff971911700 1 -- 172.21.15.46:0/237570017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c1a4110 con 0x5597539bdae0 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff971911700 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 0x5597539c9680 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ff9680cb190 tx=0x7ff96802afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5d6489d3736444 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff9557fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.545+0000 7ff9557fa700 1 -- 172.21.15.46:0/237570017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968064070 con 0x5597539bdae0 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff9557fa700 1 -- 172.21.15.46:0/237570017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff96809e9b0 con 0x5597539bdae0 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/237570017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597539b1b50 con 0x5597539bdae0 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff9557fa700 1 -- 172.21.15.46:0/237570017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9680a2f70 con 0x5597539bdae0 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff97fd0ab80 10 client.18249 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/237570017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55975373ebe0 con 0x5597539bdae0 2024-03-18T20:43:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff9557fa700 1 -- 172.21.15.46:0/237570017 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5597539b1b50 con 0x5597539bdae0 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff9557fa700 1 -- 172.21.15.46:0/237570017 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff9680b32c0 con 0x5597539bdae0 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff97fd0ab80 10 client.18249 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff97fd0ab80 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff97fd0ab80 20 client.18249 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5597539d8560 0x5597539da950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff97fd0ab80 4 client.18249 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/237570017 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559753909780 con 0x5597539d8560 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.546+0000 7ff971110700 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5597539d8560 0x5597539da950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.547+0000 7ff971110700 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5597539d8560 0x5597539da950 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3a2f92c52a091d1 server_cookie=4bb8c440efd7520f in_seq=0 out_seq=0 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.547+0000 7ff9557fa700 10 client.18249 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.547+0000 7ff9557fa700 1 -- 172.21.15.46:0/237570017 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559753909780 con 0x5597539d8560 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.547+0000 7ff9557fa700 10 client.18249 handle_command_reply: tid=0 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.547+0000 7ff97fd0ab80 1 client.18249 shutdown 2024-03-18T20:43:41.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.547+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/237570017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 msgr2=0x5597539c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.547+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 0x5597539c9680 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ff9680cb190 tx=0x7ff96802afc0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.548+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/237570017 shutdown_connections 2024-03-18T20:43:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.548+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5597539d8560 0x5597539da950 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.548+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 0x5597539c9680 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.548+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 0x5597539d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.548+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/237570017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x5597539c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.548+0000 7ff9557fa700 0 client.18249 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.548+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/237570017 >> 172.21.15.46:0/237570017 conn(0x5597539342a0 msgr2=0x5597539b9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.548+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/237570017 shutdown_connections 2024-03-18T20:43:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.548+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/237570017 wait complete. 2024-03-18T20:43:41.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.548+0000 7ff97fd0ab80 20 client.18249 trim_cache size 0 max 16384 2024-03-18T20:43:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.566+0000 7ff97fd0ab80 1 Processor -- start 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.566+0000 7ff97fd0ab80 1 -- start start 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff97fd0ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 0x5597539ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff97fd0ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x5597539c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff97fd0ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 0x5597539c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff97fd0ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597539d5590 con 0x5597539bdae0 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff97fd0ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597539bedb0 con 0x5597539be400 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff97fd0ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597539b4000 con 0x5597539c92a0 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff971911700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 0x5597539ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff971110700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x5597539c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff971110700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x5597539c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54546/0 (socket says 172.21.15.46:54546) 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff971911700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 0x5597539ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59984/0 (socket says 172.21.15.46:59984) 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff971110700 1 -- 172.21.15.46:0/3041471073 learned_addr learned my addr 172.21.15.46:0/3041471073 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff972112700 1 --2- 172.21.15.46:0/3041471073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 0x5597539c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff971110700 1 -- 172.21.15.46:0/3041471073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 msgr2=0x5597539c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff971110700 1 --2- 172.21.15.46:0/3041471073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 0x5597539c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff971110700 1 -- 172.21.15.46:0/3041471073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 msgr2=0x5597539ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff971110700 1 --2- 172.21.15.46:0/3041471073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 0x5597539ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff971110700 1 -- 172.21.15.46:0/3041471073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597538d6f20 con 0x5597539be400 2024-03-18T20:43:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff971911700 1 --2- 172.21.15.46:0/3041471073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 0x5597539ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff971110700 1 --2- 172.21.15.46:0/3041471073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x5597539c6d60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff960005fd0 tx=0x7ff960014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23cb61a36bc14a21 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff95cda8700 1 -- 172.21.15.46:0/3041471073 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96002e070 con 0x5597539be400 2024-03-18T20:43:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.567+0000 7ff95cda8700 1 -- 172.21.15.46:0/3041471073 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff960029460 con 0x5597539be400 2024-03-18T20:43:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.568+0000 7ff95cda8700 1 -- 172.21.15.46:0/3041471073 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960024040 con 0x5597539be400 2024-03-18T20:43:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.568+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3041471073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 msgr2=0x5597539c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.568+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/3041471073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x5597539c6d60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff960005fd0 tx=0x7ff960014000 comp rx=0 tx=0).stop 2024-03-18T20:43:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.568+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3041471073 shutdown_connections 2024-03-18T20:43:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.568+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/3041471073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539bdae0 0x5597539ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.568+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/3041471073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539c92a0 0x5597539c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.568+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/3041471073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x5597539c6d60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.568+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3041471073 >> 172.21.15.46:0/3041471073 conn(0x5597539342a0 msgr2=0x5597539c7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.568+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3041471073 shutdown_connections 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.568+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3041471073 wait complete. 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.569+0000 7ff97fd0ab80 1 -- start start 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.569+0000 7ff97fd0ab80 1 Processor -- start 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.569+0000 7ff97fd0ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539bdae0 0x559753a265a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.569+0000 7ff97fd0ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x559753a26ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.569+0000 7ff97fd0ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539c92a0 0x559753a20af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.569+0000 7ff97fd0ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559753a210d0 con 0x5597539c92a0 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.569+0000 7ff97fd0ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559753a21240 con 0x5597539be400 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.569+0000 7ff97fd0ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559753a21400 con 0x5597539bdae0 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.569+0000 7ff97fd0ab80 10 client.0 _ll_register_callbacks cb 0x7ff960009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff971911700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539bdae0 0x559753a265a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff971110700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x559753a26ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff971911700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539bdae0 0x559753a265a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33192/0 (socket says 172.21.15.46:33192) 2024-03-18T20:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff971110700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x559753a26ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54562/0 (socket says 172.21.15.46:54562) 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff971911700 1 -- 172.21.15.46:0/3804914044 learned_addr learned my addr 172.21.15.46:0/3804914044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff972112700 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539c92a0 0x559753a20af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff971911700 1 -- 172.21.15.46:0/3804914044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 msgr2=0x559753a26ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff971911700 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x559753a26ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff971911700 1 -- 172.21.15.46:0/3804914044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539c92a0 msgr2=0x559753a20af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff971911700 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539c92a0 0x559753a20af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff971911700 1 -- 172.21.15.46:0/3804914044 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55975373ebe0 con 0x5597539bdae0 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff971110700 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x559753a26ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff972112700 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539c92a0 0x559753a20af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff971911700 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539bdae0 0x559753a265a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff9680aefd0 tx=0x7ff968005300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e6d3f0c9300b5b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff9557fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff9557fa700 1 -- 172.21.15.46:0/3804914044 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968064070 con 0x5597539bdae0 2024-03-18T20:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff9557fa700 1 -- 172.21.15.46:0/3804914044 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff96809e9b0 con 0x5597539bdae0 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3804914044 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff96809ab80 con 0x5597539bdae0 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.570+0000 7ff9557fa700 1 -- 172.21.15.46:0/3804914044 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9680b3440 con 0x5597539bdae0 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.571+0000 7ff97fd0ab80 10 client.27928 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.571+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3804914044 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7ff960017070 con 0x5597539bdae0 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.571+0000 7ff9557fa700 1 -- 172.21.15.46:0/3804914044 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9680ab020 con 0x5597539bdae0 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.571+0000 7ff9557fa700 1 -- 172.21.15.46:0/3804914044 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff9680a2ae0 con 0x5597539bdae0 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.571+0000 7ff97fd0ab80 10 client.27928 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.571+0000 7ff97fd0ab80 10 client.27928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.571+0000 7ff97fd0ab80 20 client.27928 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.571+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559753a2e1a0 0x559753a305c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.571+0000 7ff97fd0ab80 4 client.27928 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.571+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3804914044 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55975375fd70 con 0x559753a2e1a0 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.571+0000 7ff971110700 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559753a2e1a0 0x559753a305c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.572+0000 7ff971110700 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559753a2e1a0 0x559753a305c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9b8582d646b80ef server_cookie=f8979c5f70dbe33c in_seq=0 out_seq=0 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.572+0000 7ff9557fa700 10 client.27928 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff9557fa700 1 -- 172.21.15.46:0/3804914044 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55975375fd70 con 0x559753a2e1a0 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff9557fa700 10 client.27928 handle_command_reply: tid=0 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff97fd0ab80 1 client.27928 shutdown 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3804914044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539bdae0 msgr2=0x559753a265a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539bdae0 0x559753a265a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff9680aefd0 tx=0x7ff968005300 comp rx=0 tx=0).stop 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3804914044 shutdown_connections 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559753a2e1a0 0x559753a305c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5597539c92a0 0x559753a20af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5597539bdae0 0x559753a265a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff97fd0ab80 1 --2- 172.21.15.46:0/3804914044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5597539be400 0x559753a26ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff9557fa700 0 client.27928 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3804914044 >> 172.21.15.46:0/3804914044 conn(0x5597539342a0 msgr2=0x5597539c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3804914044 shutdown_connections 2024-03-18T20:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff97fd0ab80 1 -- 172.21.15.46:0/3804914044 wait complete. 2024-03-18T20:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.573+0000 7ff97fd0ab80 20 client.27928 trim_cache size 0 max 16384 2024-03-18T20:43:41.581 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:43:41.581 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:43:41.581 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "d0e199dd-bd3b-448d-afe7-42d98d7a808b", 2024-03-18T20:43:41.581 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:43:41.581 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.574+0000 7ff973b75700 1 -- 172.21.15.46:0/2967508823 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9580a1290 msgr2=0x7ff9580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.574+0000 7ff973b75700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9580a1290 0x7ff9580a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff96c1987e0 tx=0x7ff960009310 comp rx=0 tx=0).stop 2024-03-18T20:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.574+0000 7ff973b75700 1 -- 172.21.15.46:0/2967508823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff96c0ffe50 msgr2=0x7ff96c197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.574+0000 7ff973b75700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff96c0ffe50 0x7ff96c197290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff968009fd0 tx=0x7ff96800fa00 comp rx=0 tx=0).stop 2024-03-18T20:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.574+0000 7ff973b75700 1 -- 172.21.15.46:0/2967508823 shutdown_connections 2024-03-18T20:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.574+0000 7ff973b75700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9580a1290 0x7ff9580a3750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.574+0000 7ff973b75700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c19bb70 0x7ff96c19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.574+0000 7ff973b75700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff96c0ffe50 0x7ff96c197290 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.574+0000 7ff973b75700 1 --2- 172.21.15.46:0/2967508823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff96c104f60 0x7ff96c1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.574+0000 7ff973b75700 1 -- 172.21.15.46:0/2967508823 >> 172.21.15.46:0/2967508823 conn(0x7ff96c076d40 msgr2=0x7ff96c0682b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.574+0000 7ff973b75700 1 -- 172.21.15.46:0/2967508823 shutdown_connections 2024-03-18T20:43:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.574+0000 7ff973b75700 1 -- 172.21.15.46:0/2967508823 wait complete. 2024-03-18T20:43:41.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:43:41.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:43:41.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 262969 3.53 MB/sec execute 276 sec latency 24.690 ms 2024-03-18T20:43:41.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.737+0000 7f5a4353a700 1 -- 172.21.15.46:0/2369267729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a3c0687f0 msgr2=0x7f5a3c10db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.737+0000 7f5a4353a700 1 --2- 172.21.15.46:0/2369267729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a3c0687f0 0x7f5a3c10db50 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400b0a0 tx=0x7f5a3400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.737+0000 7f5a4353a700 1 -- 172.21.15.46:0/2369267729 shutdown_connections 2024-03-18T20:43:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.737+0000 7f5a4353a700 1 --2- 172.21.15.46:0/2369267729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a3c0687f0 0x7f5a3c10db50 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.737+0000 7f5a4353a700 1 --2- 172.21.15.46:0/2369267729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a3c105ce0 0x7f5a3c1060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.737+0000 7f5a4353a700 1 --2- 172.21.15.46:0/2369267729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a3c067e30 0x7f5a3c0682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.737+0000 7f5a4353a700 1 -- 172.21.15.46:0/2369267729 >> 172.21.15.46:0/2369267729 conn(0x7f5a3c076d40 msgr2=0x7f5a3c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.737+0000 7f5a4353a700 1 -- 172.21.15.46:0/2369267729 shutdown_connections 2024-03-18T20:43:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.737+0000 7f5a4353a700 1 -- 172.21.15.46:0/2369267729 wait complete. 2024-03-18T20:43:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a4353a700 1 Processor -- start 2024-03-18T20:43:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a4353a700 1 -- start start 2024-03-18T20:43:41.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a4353a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a3c067e30 0x7f5a3c199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a4353a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a3c0687f0 0x7f5a3c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a4353a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a3c105ce0 0x7f5a3c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a4353a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c19e450 con 0x7f5a3c067e30 2024-03-18T20:43:41.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a4353a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c19e5c0 con 0x7f5a3c105ce0 2024-03-18T20:43:41.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a4353a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c19e780 con 0x7f5a3c0687f0 2024-03-18T20:43:41.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a412d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a3c067e30 0x7f5a3c199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a412d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a3c067e30 0x7f5a3c199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60000/0 (socket says 172.21.15.46:60000) 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a412d6700 1 -- 172.21.15.46:0/437459043 learned_addr learned my addr 172.21.15.46:0/437459043 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a41ad7700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a3c105ce0 0x7f5a3c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.738+0000 7f5a40ad5700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a3c0687f0 0x7f5a3c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.739+0000 7f5a412d6700 1 -- 172.21.15.46:0/437459043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a3c0687f0 msgr2=0x7f5a3c199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.739+0000 7f5a412d6700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a3c0687f0 0x7f5a3c199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.739+0000 7f5a412d6700 1 -- 172.21.15.46:0/437459043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a3c105ce0 msgr2=0x7f5a3c19de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.739+0000 7f5a412d6700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a3c105ce0 0x7f5a3c19de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.739+0000 7f5a412d6700 1 -- 172.21.15.46:0/437459043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a34009d00 con 0x7f5a3c067e30 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.739+0000 7f5a41ad7700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a3c105ce0 0x7f5a3c19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.739+0000 7f5a412d6700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a3c067e30 0x7f5a3c199530 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800e9a0 tx=0x7f5a3800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.739+0000 7f5a327fc700 1 -- 172.21.15.46:0/437459043 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38009790 con 0x7f5a3c067e30 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.739+0000 7f5a327fc700 1 -- 172.21.15.46:0/437459043 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a38010430 con 0x7f5a3c067e30 2024-03-18T20:43:41.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.739+0000 7f5a4353a700 1 -- 172.21.15.46:0/437459043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c19ea60 con 0x7f5a3c067e30 2024-03-18T20:43:41.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.739+0000 7f5a327fc700 1 -- 172.21.15.46:0/437459043 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38018680 con 0x7f5a3c067e30 2024-03-18T20:43:41.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.739+0000 7f5a4353a700 1 -- 172.21.15.46:0/437459043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c1a62a0 con 0x7f5a3c067e30 2024-03-18T20:43:41.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.741+0000 7f5a327fc700 1 -- 172.21.15.46:0/437459043 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5a380105a0 con 0x7f5a3c067e30 2024-03-18T20:43:41.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.742+0000 7f5a327fc700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5a280a1310 0x7f5a280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.742+0000 7f5a327fc700 1 -- 172.21.15.46:0/437459043 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5a38014070 con 0x7f5a3c067e30 2024-03-18T20:43:41.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.742+0000 7f5a40ad5700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5a280a1310 0x7f5a280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.743+0000 7f5a40ad5700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5a280a1310 0x7f5a280a37d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c19aa80 tx=0x7f5a2c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:41.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.743+0000 7f5a4f6cfb80 1 Processor -- start 2024-03-18T20:43:41.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.743+0000 7f5a4f6cfb80 1 -- start start 2024-03-18T20:43:41.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a4f6cfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a82a60 0x55f5a2a82e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a4f6cfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 0x55f5a2a8dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a4f6cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a8e220 0x55f5a2a90610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a4f6cfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a29dc990 con 0x55f5a2a83380 2024-03-18T20:43:41.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a4f6cfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a29da350 con 0x55f5a2a82a60 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a4f6cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a29d5c40 con 0x55f5a2a8e220 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a40ad5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 0x55f5a2a8dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a412d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a82a60 0x55f5a2a82e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a412d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a82a60 0x55f5a2a82e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54594/0 (socket says 172.21.15.46:54594) 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a412d6700 1 -- 172.21.15.46:0/1566043927 learned_addr learned my addr 172.21.15.46:0/1566043927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a40ad5700 1 --2- 172.21.15.46:0/1566043927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 0x55f5a2a8dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60006/0 (socket says 172.21.15.46:60006) 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a41ad7700 1 --2- 172.21.15.46:0/1566043927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a8e220 0x55f5a2a90610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a412d6700 1 -- 172.21.15.46:0/1566043927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a8e220 msgr2=0x55f5a2a90610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a412d6700 1 --2- 172.21.15.46:0/1566043927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a8e220 0x55f5a2a90610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a412d6700 1 -- 172.21.15.46:0/1566043927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 msgr2=0x55f5a2a8dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a412d6700 1 --2- 172.21.15.46:0/1566043927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 0x55f5a2a8dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a412d6700 1 -- 172.21.15.46:0/1566043927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5a28573b0 con 0x55f5a2a82a60 2024-03-18T20:43:41.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a412d6700 1 --2- 172.21.15.46:0/1566043927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a82a60 0x55f5a2a82e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5a380a1e80 tx=0x7f5a3809a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5121da6ec6914ae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a277fe700 1 -- 172.21.15.46:0/1566043927 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380a23b0 con 0x55f5a2a82a60 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a277fe700 1 -- 172.21.15.46:0/1566043927 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a38010880 con 0x55f5a2a82a60 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.744+0000 7f5a277fe700 1 -- 172.21.15.46:0/1566043927 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380996d0 con 0x55f5a2a82a60 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.745+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/1566043927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a82a60 msgr2=0x55f5a2a82e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.745+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/1566043927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a82a60 0x55f5a2a82e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5a380a1e80 tx=0x7f5a3809a630 comp rx=0 tx=0).stop 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.745+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/1566043927 shutdown_connections 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.745+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/1566043927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a8e220 0x55f5a2a90610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.745+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/1566043927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a82a60 0x55f5a2a82e40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.745+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/1566043927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 0x55f5a2a8dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.745+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/1566043927 >> 172.21.15.46:0/1566043927 conn(0x55f5a29f9220 msgr2=0x55f5a2a7f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.745+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/1566043927 shutdown_connections 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.745+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/1566043927 wait complete. 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a4f6cfb80 1 -- start start 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a4f6cfb80 1 Processor -- start 2024-03-18T20:43:41.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a4f6cfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 0x55f5a2a8b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a4f6cfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a83380 0x55f5a2a8b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a4f6cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a8e220 0x55f5a2a8bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a4f6cfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a2971d80 con 0x55f5a2a82a60 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a4f6cfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a2971a20 con 0x55f5a2a83380 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a4f6cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a28596a0 con 0x55f5a2a8e220 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a4f6cfb80 10 client.0 _ll_register_callbacks cb 0x55f5a27dd3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a412d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 0x55f5a2a8b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a40ad5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a83380 0x55f5a2a8b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a412d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 0x55f5a2a8b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60014/0 (socket says 172.21.15.46:60014) 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.746+0000 7f5a412d6700 1 -- 172.21.15.46:0/2741917372 learned_addr learned my addr 172.21.15.46:0/2741917372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a41ad7700 1 --2- 172.21.15.46:0/2741917372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a8e220 0x55f5a2a8bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a40ad5700 1 -- 172.21.15.46:0/2741917372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a8e220 msgr2=0x55f5a2a8bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a40ad5700 1 --2- 172.21.15.46:0/2741917372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a8e220 0x55f5a2a8bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a40ad5700 1 -- 172.21.15.46:0/2741917372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 msgr2=0x55f5a2a8b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a40ad5700 1 --2- 172.21.15.46:0/2741917372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 0x55f5a2a8b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a40ad5700 1 -- 172.21.15.46:0/2741917372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c1a62a0 con 0x55f5a2a83380 2024-03-18T20:43:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a40ad5700 1 --2- 172.21.15.46:0/2741917372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a83380 0x55f5a2a8b580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0117d0 tx=0x7f5a2c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c0ad64446f5119 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:43:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a24ff9700 1 -- 172.21.15.46:0/2741917372 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c022070 con 0x55f5a2a83380 2024-03-18T20:43:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a24ff9700 1 -- 172.21.15.46:0/2741917372 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5a2c01d410 con 0x55f5a2a83380 2024-03-18T20:43:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/2741917372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5a2a76ad0 con 0x55f5a2a83380 2024-03-18T20:43:41.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a24ff9700 1 -- 172.21.15.46:0/2741917372 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a2c004580 con 0x55f5a2a83380 2024-03-18T20:43:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a4f6cfb80 10 client.27923 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.747+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/2741917372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f5a2803b60 con 0x55f5a2a83380 2024-03-18T20:43:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.748+0000 7f5a24ff9700 1 -- 172.21.15.46:0/2741917372 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a2c033080 con 0x55f5a2a83380 2024-03-18T20:43:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.748+0000 7f5a24ff9700 1 -- 172.21.15.46:0/2741917372 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5a2c02f720 con 0x55f5a2a83380 2024-03-18T20:43:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.748+0000 7f5a4f6cfb80 10 client.27923 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.748+0000 7f5a4f6cfb80 10 client.27923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.748+0000 7f5a4f6cfb80 20 client.27923 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.748+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/2741917372 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f5a2a9dfb0 0x55f5a2aa03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.748+0000 7f5a4f6cfb80 4 client.27923 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.748+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/2741917372 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5a29ce700 con 0x55f5a2a9dfb0 2024-03-18T20:43:41.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.748+0000 7f5a412d6700 1 --2- 172.21.15.46:0/2741917372 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f5a2a9dfb0 0x55f5a2aa03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.748+0000 7f5a24ff9700 10 client.27923 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.748+0000 7f5a412d6700 1 --2- 172.21.15.46:0/2741917372 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f5a2a9dfb0 0x55f5a2aa03d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82e0a24f47ffa9a0 server_cookie=7da0db5920bffd7b in_seq=0 out_seq=0 2024-03-18T20:43:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.749+0000 7f5a24ff9700 1 -- 172.21.15.46:0/2741917372 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f5a29ce700 con 0x55f5a2a9dfb0 2024-03-18T20:43:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.749+0000 7f5a24ff9700 10 client.27923 handle_command_reply: tid=0 2024-03-18T20:43:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.749+0000 7f5a4f6cfb80 1 client.27923 shutdown 2024-03-18T20:43:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.749+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/2741917372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a83380 msgr2=0x55f5a2a8b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.749+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/2741917372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a83380 0x55f5a2a8b580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0117d0 tx=0x7f5a2c0047c0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.749+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/2741917372 shutdown_connections 2024-03-18T20:43:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.749+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/2741917372 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f5a2a9dfb0 0x55f5a2aa03d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.749+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/2741917372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 0x55f5a2a8b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.749+0000 7f5a24ff9700 0 client.27923 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.749+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/2741917372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a8e220 0x55f5a2a8bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.749+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/2741917372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a83380 0x55f5a2a8b580 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.749+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/2741917372 >> 172.21.15.46:0/2741917372 conn(0x55f5a29f9220 msgr2=0x55f5a2a7ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.750+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/2741917372 shutdown_connections 2024-03-18T20:43:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.750+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/2741917372 wait complete. 2024-03-18T20:43:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.750+0000 7f5a4f6cfb80 20 client.27923 trim_cache size 0 max 16384 2024-03-18T20:43:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.768+0000 7f5a4f6cfb80 1 Processor -- start 2024-03-18T20:43:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.768+0000 7f5a4f6cfb80 1 -- start start 2024-03-18T20:43:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a4f6cfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 0x55f5a2a94e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a4f6cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a83380 0x55f5a2a7eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a4f6cfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a8e220 0x55f5a2a995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a4f6cfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a2a84440 con 0x55f5a2a82a60 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a4f6cfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a2aa0760 con 0x55f5a2a8e220 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a4f6cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a2a95390 con 0x55f5a2a83380 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a412d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 0x55f5a2a94e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a412d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 0x55f5a2a94e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60032/0 (socket says 172.21.15.46:60032) 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a412d6700 1 -- 172.21.15.46:0/917707039 learned_addr learned my addr 172.21.15.46:0/917707039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a41ad7700 1 --2- 172.21.15.46:0/917707039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a8e220 0x55f5a2a995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a40ad5700 1 --2- 172.21.15.46:0/917707039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a83380 0x55f5a2a7eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a412d6700 1 -- 172.21.15.46:0/917707039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a83380 msgr2=0x55f5a2a7eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a412d6700 1 --2- 172.21.15.46:0/917707039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a83380 0x55f5a2a7eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a412d6700 1 -- 172.21.15.46:0/917707039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a8e220 msgr2=0x55f5a2a995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a412d6700 1 --2- 172.21.15.46:0/917707039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a8e220 0x55f5a2a995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a412d6700 1 -- 172.21.15.46:0/917707039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5a299bea0 con 0x55f5a2a82a60 2024-03-18T20:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a40ad5700 1 --2- 172.21.15.46:0/917707039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a83380 0x55f5a2a7eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a41ad7700 1 --2- 172.21.15.46:0/917707039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a8e220 0x55f5a2a995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a412d6700 1 --2- 172.21.15.46:0/917707039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 0x55f5a2a94e50 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5a38098280 tx=0x7f5a380962f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a50372f676e0158 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.769+0000 7f5a27fff700 1 -- 172.21.15.46:0/917707039 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380a23b0 con 0x55f5a2a82a60 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.770+0000 7f5a27fff700 1 -- 172.21.15.46:0/917707039 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a380982d0 con 0x55f5a2a82a60 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.770+0000 7f5a27fff700 1 -- 172.21.15.46:0/917707039 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3809a460 con 0x55f5a2a82a60 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.770+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/917707039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 msgr2=0x55f5a2a94e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.770+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/917707039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 0x55f5a2a94e50 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5a38098280 tx=0x7f5a380962f0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.770+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/917707039 shutdown_connections 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.770+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/917707039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a82a60 0x55f5a2a94e50 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.770+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/917707039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a8e220 0x55f5a2a995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.770+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/917707039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a83380 0x55f5a2a7eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.770+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/917707039 >> 172.21.15.46:0/917707039 conn(0x55f5a29f9220 msgr2=0x55f5a2a7aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.770+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/917707039 shutdown_connections 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.770+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/917707039 wait complete. 2024-03-18T20:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.771+0000 7f5a4f6cfb80 1 -- start start 2024-03-18T20:43:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.771+0000 7f5a4f6cfb80 1 Processor -- start 2024-03-18T20:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.771+0000 7f5a4f6cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a82a60 0x55f5a2a94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.771+0000 7f5a4f6cfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 0x55f5a2a94b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.771+0000 7f5a4f6cfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a8e220 0x55f5a2a92e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.771+0000 7f5a4f6cfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a2a933c0 con 0x55f5a2a83380 2024-03-18T20:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.771+0000 7f5a4f6cfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a2a93530 con 0x55f5a2a8e220 2024-03-18T20:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.771+0000 7f5a4f6cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a2a936f0 con 0x55f5a2a82a60 2024-03-18T20:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.771+0000 7f5a4f6cfb80 10 client.0 _ll_register_callbacks cb 0x55f5a27dd3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.771+0000 7f5a40ad5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 0x55f5a2a94b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a40ad5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 0x55f5a2a94b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60042/0 (socket says 172.21.15.46:60042) 2024-03-18T20:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a40ad5700 1 -- 172.21.15.46:0/131803392 learned_addr learned my addr 172.21.15.46:0/131803392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a41ad7700 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a8e220 0x55f5a2a92e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a412d6700 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a82a60 0x55f5a2a94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a40ad5700 1 -- 172.21.15.46:0/131803392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a82a60 msgr2=0x55f5a2a94640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a40ad5700 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a82a60 0x55f5a2a94640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a40ad5700 1 -- 172.21.15.46:0/131803392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a8e220 msgr2=0x55f5a2a92e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a40ad5700 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a8e220 0x55f5a2a92e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a40ad5700 1 -- 172.21.15.46:0/131803392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c10e550 con 0x55f5a2a83380 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a40ad5700 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 0x55f5a2a94b80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c016040 tx=0x7f5a2c02f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc92f1fa346681a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a24ff9700 1 -- 172.21.15.46:0/131803392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c022070 con 0x55f5a2a83380 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a24ff9700 1 -- 172.21.15.46:0/131803392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5a2c027630 con 0x55f5a2a83380 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/131803392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a38010880 con 0x55f5a2a83380 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a24ff9700 1 -- 172.21.15.46:0/131803392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a2c0092d0 con 0x55f5a2a83380 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a4f6cfb80 10 client.18273 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.772+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/131803392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f5a28573b0 con 0x55f5a2a83380 2024-03-18T20:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a24ff9700 1 -- 172.21.15.46:0/131803392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a38010880 con 0x55f5a2a83380 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a24ff9700 1 -- 172.21.15.46:0/131803392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5a2c039890 con 0x55f5a2a83380 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a4f6cfb80 10 client.18273 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a4f6cfb80 10 client.18273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a4f6cfb80 20 client.18273 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f5a2af31a0 0x55f5a2af55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a4f6cfb80 4 client.18273 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/131803392 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f5a2824cf0 con 0x55f5a2af31a0 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a412d6700 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f5a2af31a0 0x55f5a2af55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a412d6700 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f5a2af31a0 0x55f5a2af55c0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22ceb4edd6567c27 server_cookie=be6cab1aa6fb1753 in_seq=0 out_seq=0 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a24ff9700 10 client.18273 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a24ff9700 1 -- 172.21.15.46:0/131803392 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f5a2824cf0 con 0x55f5a2af31a0 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a24ff9700 10 client.18273 handle_command_reply: tid=0 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.773+0000 7f5a4f6cfb80 1 client.18273 shutdown 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/131803392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 msgr2=0x55f5a2a94b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 0x55f5a2a94b80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c016040 tx=0x7f5a2c02f980 comp rx=0 tx=0).stop 2024-03-18T20:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/131803392 shutdown_connections 2024-03-18T20:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f5a2af31a0 0x55f5a2af55c0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f5a2a8e220 0x55f5a2a92e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a24ff9700 0 client.18273 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f5a2a82a60 0x55f5a2a94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4f6cfb80 1 --2- 172.21.15.46:0/131803392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5a2a83380 0x55f5a2a94b80 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/131803392 >> 172.21.15.46:0/131803392 conn(0x55f5a29f9220 msgr2=0x55f5a2a7eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/131803392 shutdown_connections 2024-03-18T20:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4f6cfb80 1 -- 172.21.15.46:0/131803392 wait complete. 2024-03-18T20:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4f6cfb80 20 client.18273 trim_cache size 0 max 16384 2024-03-18T20:43:41.782 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:43:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:43:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:43:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "d0e199dd-bd3b-448d-afe7-42d98d7a808b": { 2024-03-18T20:43:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:43:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "d0e199dd-bd3b-448d-afe7-42d98d7a808b", 2024-03-18T20:43:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:43:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:43:41.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:43:41.783 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4353a700 1 -- 172.21.15.46:0/437459043 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5a280a1310 msgr2=0x7f5a280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4353a700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5a280a1310 0x7f5a280a37d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c19aa80 tx=0x7f5a2c009260 comp rx=0 tx=0).stop 2024-03-18T20:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4353a700 1 -- 172.21.15.46:0/437459043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a3c067e30 msgr2=0x7f5a3c199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.774+0000 7f5a4353a700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a3c067e30 0x7f5a3c199530 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800e9a0 tx=0x7f5a3800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.775+0000 7f5a4353a700 1 -- 172.21.15.46:0/437459043 shutdown_connections 2024-03-18T20:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.775+0000 7f5a4353a700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5a280a1310 0x7f5a280a37d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.775+0000 7f5a4353a700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a3c067e30 0x7f5a3c199530 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.775+0000 7f5a4353a700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a3c105ce0 0x7f5a3c19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.775+0000 7f5a4353a700 1 --2- 172.21.15.46:0/437459043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a3c0687f0 0x7f5a3c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.775+0000 7f5a4353a700 1 -- 172.21.15.46:0/437459043 >> 172.21.15.46:0/437459043 conn(0x7f5a3c076d40 msgr2=0x7f5a3c10c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.775+0000 7f5a4353a700 1 -- 172.21.15.46:0/437459043 shutdown_connections 2024-03-18T20:43:41.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:43:41.775+0000 7f5a4353a700 1 -- 172.21.15.46:0/437459043 wait complete. 2024-03-18T20:43:41.793 INFO:tasks.cephfs.filesystem:scrub status for tag:d0e199dd-bd3b-448d-afe7-42d98d7a808b - {'path': '/', 'tag': 'd0e199dd-bd3b-448d-afe7-42d98d7a808b', 'options': 'recursive,force'} 2024-03-18T20:43:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:42 smithi073 ceph-mon[47384]: pgmap v233: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-03-18T20:43:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:42 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:43:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:42 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:42 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:42 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:42 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:42 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:42 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:42 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:42.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:42 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:42 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:42 smithi155 ceph-mon[48344]: pgmap v233: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-03-18T20:43:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:42 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:43:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:42 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:42 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:42 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:42 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:42 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:42 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:42 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:42 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:42 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:42 smithi046 ceph-mon[43495]: pgmap v233: 97 pgs: 97 active+clean; 107 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-03-18T20:43:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:42 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:43:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:42 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:42 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:42 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:42.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:42 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:42.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:42 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:42 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:42 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:42.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:42 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:42 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:42.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 264204 3.54 MB/sec execute 277 sec latency 109.508 ms 2024-03-18T20:43:43.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:43 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:43.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:43 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:43 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:43.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:43 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:43.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:43 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:43 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:43 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:43.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:43 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:43 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:43:43.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:43 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:43 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:43 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:43 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:43 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:43 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:43 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:43 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:43.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:43 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:43 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:43:43.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:43 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:43 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:43 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:43 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:43 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:43 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:43 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:43 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:43 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:43 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:43:43.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:43 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:43:43.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 264204 3.53 MB/sec execute 278 sec latency 1109.628 ms 2024-03-18T20:43:44.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:44 smithi073 ceph-mon[47384]: pgmap v234: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-03-18T20:43:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:44 smithi155 ceph-mon[48344]: pgmap v234: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-03-18T20:43:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:44 smithi046 ceph-mon[43495]: pgmap v234: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-03-18T20:43:44.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 264204 3.51 MB/sec execute 279 sec latency 2109.733 ms 2024-03-18T20:43:45.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 265074 3.51 MB/sec execute 280 sec latency 2735.826 ms 2024-03-18T20:43:46.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:45 smithi073 ceph-mon[47384]: pgmap v235: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-03-18T20:43:46.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:45 smithi155 ceph-mon[48344]: pgmap v235: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-03-18T20:43:46.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:45 smithi046 ceph-mon[43495]: pgmap v235: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-03-18T20:43:46.701 INFO:tasks.workunit.client.0.smithi046.stdout: 1 266531 3.54 MB/sec execute 281 sec latency 15.145 ms 2024-03-18T20:43:47.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 267752 3.54 MB/sec execute 282 sec latency 127.172 ms 2024-03-18T20:43:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:48 smithi073 ceph-mon[47384]: pgmap v236: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-03-18T20:43:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:48 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:43:48.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:48 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:43:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:48 smithi155 ceph-mon[48344]: pgmap v236: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-03-18T20:43:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:48 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:43:48.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:48 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:43:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:48 smithi046 ceph-mon[43495]: pgmap v236: 97 pgs: 97 active+clean; 102 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-03-18T20:43:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:48 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:43:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:48 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:43:48.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 267752 3.53 MB/sec execute 283 sec latency 1127.280 ms 2024-03-18T20:43:49.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:49 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:43:49.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:49 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:43:49.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:49 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:43:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:49 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:43:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:49 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:43:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:49 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:43:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:49 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:43:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:49 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:43:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:49 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:43:49.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 267752 3.52 MB/sec execute 284 sec latency 2127.374 ms 2024-03-18T20:43:50.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:50 smithi073 ceph-mon[47384]: pgmap v237: 97 pgs: 97 active+clean; 105 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-03-18T20:43:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:50 smithi155 ceph-mon[48344]: pgmap v237: 97 pgs: 97 active+clean; 105 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-03-18T20:43:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:50 smithi046 ceph-mon[43495]: pgmap v237: 97 pgs: 97 active+clean; 105 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-03-18T20:43:50.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 269682 3.55 MB/sec execute 285 sec latency 2283.845 ms 2024-03-18T20:43:51.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 270857 3.55 MB/sec execute 286 sec latency 15.741 ms 2024-03-18T20:43:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.010+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1457237042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc4410b280 msgr2=0x7fcc4410d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.010+0000 7fcc4a5e3700 1 --2- 172.21.15.46:0/1457237042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc4410b280 0x7fcc4410d740 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00b0a0 tx=0x7fcc3c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.010+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1457237042 shutdown_connections 2024-03-18T20:43:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.010+0000 7fcc4a5e3700 1 --2- 172.21.15.46:0/1457237042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc4410b280 0x7fcc4410d740 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.010+0000 7fcc4a5e3700 1 --2- 172.21.15.46:0/1457237042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc44101e60 0x7fcc44102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.010+0000 7fcc4a5e3700 1 --2- 172.21.15.46:0/1457237042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc44102780 0x7fcc4410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.010+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1457237042 >> 172.21.15.46:0/1457237042 conn(0x7fcc440fb680 msgr2=0x7fcc440fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.010+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1457237042 shutdown_connections 2024-03-18T20:43:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.010+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1457237042 wait complete. 2024-03-18T20:43:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.011+0000 7fcc4a5e3700 1 Processor -- start 2024-03-18T20:43:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.011+0000 7fcc4a5e3700 1 -- start start 2024-03-18T20:43:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.011+0000 7fcc4a5e3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc44101e60 0x7fcc441972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc4a5e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc44102780 0x7fcc44197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc4a5e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc4410b280 0x7fcc4419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc4a5e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4419c1f0 con 0x7fcc44102780 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc4a5e3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4419c360 con 0x7fcc4410b280 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc4a5e3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4419c520 con 0x7fcc44101e60 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc437fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc44102780 0x7fcc44197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc437fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc44102780 0x7fcc44197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45970/0 (socket says 172.21.15.46:45970) 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc437fe700 1 -- 172.21.15.46:0/1743924184 learned_addr learned my addr 172.21.15.46:0/1743924184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc48b80700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc4410b280 0x7fcc4419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc43fff700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc44101e60 0x7fcc441972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc437fe700 1 -- 172.21.15.46:0/1743924184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc44101e60 msgr2=0x7fcc441972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc437fe700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc44101e60 0x7fcc441972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc437fe700 1 -- 172.21.15.46:0/1743924184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc4410b280 msgr2=0x7fcc4419bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc437fe700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc4410b280 0x7fcc4419bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc437fe700 1 -- 172.21.15.46:0/1743924184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c009d00 con 0x7fcc44102780 2024-03-18T20:43:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc437fe700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc44102780 0x7fcc44197810 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00e9f0 tx=0x7fcc2c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc417fa700 1 -- 172.21.15.46:0/1743924184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c00c9c0 con 0x7fcc44102780 2024-03-18T20:43:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc417fa700 1 -- 172.21.15.46:0/1743924184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc2c010430 con 0x7fcc44102780 2024-03-18T20:43:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.012+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1743924184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4419c800 con 0x7fcc44102780 2024-03-18T20:43:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.013+0000 7fcc417fa700 1 -- 172.21.15.46:0/1743924184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c018680 con 0x7fcc44102780 2024-03-18T20:43:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.013+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1743924184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc441a4040 con 0x7fcc44102780 2024-03-18T20:43:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.015+0000 7fcc417fa700 1 -- 172.21.15.46:0/1743924184 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fcc2c010860 con 0x7fcc44102780 2024-03-18T20:43:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.015+0000 7fcc417fa700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcc300a1290 0x7fcc300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.015+0000 7fcc417fa700 1 -- 172.21.15.46:0/1743924184 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fcc2c014070 con 0x7fcc44102780 2024-03-18T20:43:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.015+0000 7fcc43fff700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcc300a1290 0x7fcc300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.016+0000 7fcc43fff700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcc300a1290 0x7fcc300a3750 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009c60 tx=0x7fcc34009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc56778b80 1 Processor -- start 2024-03-18T20:43:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc56778b80 1 -- start start 2024-03-18T20:43:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc56778b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1570 0x55b5cabf1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc56778b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 0x55b5cabfc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc56778b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabfcdb0 0x55b5cabff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc56778b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5cab4b930 con 0x55b5cabf1e90 2024-03-18T20:43:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc56778b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5cab492f0 con 0x55b5cabf1570 2024-03-18T20:43:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc56778b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5cab44be0 con 0x55b5cabfcdb0 2024-03-18T20:43:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc437fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 0x55b5cabfc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc43fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1570 0x55b5cabf1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc437fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 0x55b5cabfc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45980/0 (socket says 172.21.15.46:45980) 2024-03-18T20:43:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc437fe700 1 -- 172.21.15.46:0/2503610115 learned_addr learned my addr 172.21.15.46:0/2503610115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc48b80700 1 --2- 172.21.15.46:0/2503610115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabfcdb0 0x55b5cabff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc43fff700 1 -- 172.21.15.46:0/2503610115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabfcdb0 msgr2=0x55b5cabff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc43fff700 1 --2- 172.21.15.46:0/2503610115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabfcdb0 0x55b5cabff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc43fff700 1 -- 172.21.15.46:0/2503610115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 msgr2=0x55b5cabfc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.017+0000 7fcc43fff700 1 --2- 172.21.15.46:0/2503610115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 0x55b5cabfc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.018+0000 7fcc43fff700 1 -- 172.21.15.46:0/2503610115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5ca9c6350 con 0x55b5cabf1570 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.018+0000 7fcc437fe700 1 --2- 172.21.15.46:0/2503610115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 0x55b5cabfc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.018+0000 7fcc43fff700 1 --2- 172.21.15.46:0/2503610115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1570 0x55b5cabf1950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e3b0 tx=0x7fcc34014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50095851eae6d666 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.018+0000 7fcc3a7fc700 1 -- 172.21.15.46:0/2503610115 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3400ed50 con 0x55b5cabf1570 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.018+0000 7fcc3a7fc700 1 -- 172.21.15.46:0/2503610115 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc34021070 con 0x55b5cabf1570 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.018+0000 7fcc3a7fc700 1 -- 172.21.15.46:0/2503610115 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34003ea0 con 0x55b5cabf1570 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.019+0000 7fcc56778b80 1 -- 172.21.15.46:0/2503610115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1570 msgr2=0x55b5cabf1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.019+0000 7fcc56778b80 1 --2- 172.21.15.46:0/2503610115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1570 0x55b5cabf1950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e3b0 tx=0x7fcc34014000 comp rx=0 tx=0).stop 2024-03-18T20:43:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.019+0000 7fcc56778b80 1 -- 172.21.15.46:0/2503610115 shutdown_connections 2024-03-18T20:43:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.019+0000 7fcc56778b80 1 --2- 172.21.15.46:0/2503610115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabfcdb0 0x55b5cabff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.019+0000 7fcc56778b80 1 --2- 172.21.15.46:0/2503610115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1570 0x55b5cabf1950 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.019+0000 7fcc56778b80 1 --2- 172.21.15.46:0/2503610115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 0x55b5cabfc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.019+0000 7fcc56778b80 1 -- 172.21.15.46:0/2503610115 >> 172.21.15.46:0/2503610115 conn(0x55b5cab681c0 msgr2=0x55b5cabeda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.019+0000 7fcc56778b80 1 -- 172.21.15.46:0/2503610115 shutdown_connections 2024-03-18T20:43:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.019+0000 7fcc56778b80 1 -- 172.21.15.46:0/2503610115 wait complete. 2024-03-18T20:43:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.019+0000 7fcc56778b80 1 -- start start 2024-03-18T20:43:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.019+0000 7fcc56778b80 1 Processor -- start 2024-03-18T20:43:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc56778b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1570 0x55b5cabfc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc56778b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 0x55b5cabf9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc56778b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabfcdb0 0x55b5cabfa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc56778b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5caae0d20 con 0x55b5cabf1e90 2024-03-18T20:43:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc56778b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5caae09c0 con 0x55b5cabfcdb0 2024-03-18T20:43:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc56778b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ca9c8640 con 0x55b5cabf1570 2024-03-18T20:43:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc56778b80 10 client.0 _ll_register_callbacks cb 0x55b5ca94c340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc437fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 0x55b5cabf9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc437fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 0x55b5cabf9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45982/0 (socket says 172.21.15.46:45982) 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc437fe700 1 -- 172.21.15.46:0/2279956030 learned_addr learned my addr 172.21.15.46:0/2279956030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc43fff700 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1570 0x55b5cabfc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc48b80700 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabfcdb0 0x55b5cabfa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc437fe700 1 -- 172.21.15.46:0/2279956030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1570 msgr2=0x55b5cabfc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc437fe700 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1570 0x55b5cabfc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc437fe700 1 -- 172.21.15.46:0/2279956030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabfcdb0 msgr2=0x55b5cabfa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc437fe700 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabfcdb0 0x55b5cabfa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc437fe700 1 -- 172.21.15.46:0/2279956030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc441a4040 con 0x55b5cabf1e90 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc437fe700 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 0x55b5cabf9ac0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0965b0 tx=0x7fcc2c099b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3099440b7a2042f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.020+0000 7fcc237fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc237fe700 1 -- 172.21.15.46:0/2279956030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0997a0 con 0x55b5cabf1e90 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc237fe700 1 -- 172.21.15.46:0/2279956030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fcc2c0cae40 con 0x55b5cabf1e90 2024-03-18T20:43:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc56778b80 1 -- 172.21.15.46:0/2279956030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5cabe56c0 con 0x55b5cabf1e90 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc237fe700 1 -- 172.21.15.46:0/2279956030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc2c09dee0 con 0x55b5cabf1e90 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc56778b80 10 client.18285 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc56778b80 1 -- 172.21.15.46:0/2279956030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fcc34017070 con 0x55b5cabf1e90 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc237fe700 1 -- 172.21.15.46:0/2279956030 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc2c063070 con 0x55b5cabf1e90 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc237fe700 1 -- 172.21.15.46:0/2279956030 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcc2c0a2130 con 0x55b5cabf1e90 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc56778b80 10 client.18285 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc56778b80 10 client.18285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc56778b80 20 client.18285 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc56778b80 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b5cac0cb00 0x55b5cac0ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc56778b80 4 client.18285 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc56778b80 1 -- 172.21.15.46:0/2279956030 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5cab3d6a0 con 0x55b5cac0cb00 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.021+0000 7fcc43fff700 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b5cac0cb00 0x55b5cac0ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.022+0000 7fcc43fff700 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b5cac0cb00 0x55b5cac0ef20 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeda15a1241f6aad server_cookie=c3e4c03b2e811b7b in_seq=0 out_seq=0 2024-03-18T20:43:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.022+0000 7fcc237fe700 10 client.18285 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.022+0000 7fcc237fe700 1 -- 172.21.15.46:0/2279956030 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b5cab3d6a0 con 0x55b5cac0cb00 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.022+0000 7fcc237fe700 10 client.18285 handle_command_reply: tid=0 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.022+0000 7fcc56778b80 1 client.18285 shutdown 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.023+0000 7fcc56778b80 1 -- 172.21.15.46:0/2279956030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 msgr2=0x55b5cabf9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.023+0000 7fcc56778b80 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 0x55b5cabf9ac0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0965b0 tx=0x7fcc2c099b00 comp rx=0 tx=0).stop 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.023+0000 7fcc56778b80 1 -- 172.21.15.46:0/2279956030 shutdown_connections 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.023+0000 7fcc56778b80 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b5cac0cb00 0x55b5cac0ef20 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.023+0000 7fcc56778b80 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabfcdb0 0x55b5cabfa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.023+0000 7fcc237fe700 0 client.18285 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.023+0000 7fcc56778b80 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1570 0x55b5cabfc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.023+0000 7fcc56778b80 1 --2- 172.21.15.46:0/2279956030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabf1e90 0x55b5cabf9ac0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.023+0000 7fcc56778b80 1 -- 172.21.15.46:0/2279956030 >> 172.21.15.46:0/2279956030 conn(0x55b5cab681c0 msgr2=0x55b5cabed5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.023+0000 7fcc56778b80 1 -- 172.21.15.46:0/2279956030 shutdown_connections 2024-03-18T20:43:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.023+0000 7fcc56778b80 1 -- 172.21.15.46:0/2279956030 wait complete. 2024-03-18T20:43:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.023+0000 7fcc56778b80 20 client.18285 trim_cache size 0 max 16384 2024-03-18T20:43:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.038+0000 7fcc56778b80 1 Processor -- start 2024-03-18T20:43:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.038+0000 7fcc56778b80 1 -- start start 2024-03-18T20:43:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.038+0000 7fcc56778b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1570 0x55b5cabedbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.038+0000 7fcc56778b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1e90 0x55b5cac03a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.038+0000 7fcc56778b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 0x55b5cac08190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc56778b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5cac09500 con 0x55b5cabfcdb0 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc56778b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5cabf3140 con 0x55b5cabf1e90 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc56778b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5cabec7a0 con 0x55b5cabf1570 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc48b80700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 0x55b5cac08190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc48b80700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 0x55b5cac08190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45998/0 (socket says 172.21.15.46:45998) 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc48b80700 1 -- 172.21.15.46:0/188852083 learned_addr learned my addr 172.21.15.46:0/188852083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc437fe700 1 --2- 172.21.15.46:0/188852083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1e90 0x55b5cac03a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc43fff700 1 --2- 172.21.15.46:0/188852083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1570 0x55b5cabedbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc48b80700 1 -- 172.21.15.46:0/188852083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1570 msgr2=0x55b5cabedbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc48b80700 1 --2- 172.21.15.46:0/188852083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1570 0x55b5cabedbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc48b80700 1 -- 172.21.15.46:0/188852083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1e90 msgr2=0x55b5cac03a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc48b80700 1 --2- 172.21.15.46:0/188852083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1e90 0x55b5cac03a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc48b80700 1 -- 172.21.15.46:0/188852083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5ca972b00 con 0x55b5cabfcdb0 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc437fe700 1 --2- 172.21.15.46:0/188852083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1e90 0x55b5cac03a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:43:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc48b80700 1 --2- 172.21.15.46:0/188852083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 0x55b5cac08190 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0094d0 tx=0x7fcc3c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ee938c627845b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc3affd700 1 -- 172.21.15.46:0/188852083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c00e040 con 0x55b5cabfcdb0 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc3affd700 1 -- 172.21.15.46:0/188852083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc3c007db0 con 0x55b5cabfcdb0 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.039+0000 7fcc3affd700 1 -- 172.21.15.46:0/188852083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c007980 con 0x55b5cabfcdb0 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.040+0000 7fcc56778b80 1 -- 172.21.15.46:0/188852083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 msgr2=0x55b5cac08190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.040+0000 7fcc56778b80 1 --2- 172.21.15.46:0/188852083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 0x55b5cac08190 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0094d0 tx=0x7fcc3c009500 comp rx=0 tx=0).stop 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.040+0000 7fcc56778b80 1 -- 172.21.15.46:0/188852083 shutdown_connections 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.040+0000 7fcc56778b80 1 --2- 172.21.15.46:0/188852083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 0x55b5cac08190 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.040+0000 7fcc56778b80 1 --2- 172.21.15.46:0/188852083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1570 0x55b5cabedbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.040+0000 7fcc56778b80 1 --2- 172.21.15.46:0/188852083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1e90 0x55b5cac03a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.040+0000 7fcc56778b80 1 -- 172.21.15.46:0/188852083 >> 172.21.15.46:0/188852083 conn(0x55b5cab681c0 msgr2=0x55b5cabf8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.040+0000 7fcc56778b80 1 -- 172.21.15.46:0/188852083 shutdown_connections 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.040+0000 7fcc56778b80 1 -- 172.21.15.46:0/188852083 wait complete. 2024-03-18T20:43:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.040+0000 7fcc56778b80 1 -- start start 2024-03-18T20:43:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.041+0000 7fcc56778b80 1 Processor -- start 2024-03-18T20:43:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.041+0000 7fcc56778b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1570 0x55b5cac03460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.041+0000 7fcc56778b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1e90 0x55b5cac039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.041+0000 7fcc56778b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 0x55b5cac01a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.041+0000 7fcc56778b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5cac02130 con 0x55b5cabfcdb0 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.041+0000 7fcc56778b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5cac022a0 con 0x55b5cabf1570 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.041+0000 7fcc56778b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5cac02460 con 0x55b5cabf1e90 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.041+0000 7fcc56778b80 10 client.0 _ll_register_callbacks cb 0x55b5ca94c340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.041+0000 7fcc48b80700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 0x55b5cac01a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.041+0000 7fcc437fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1e90 0x55b5cac039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.041+0000 7fcc48b80700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 0x55b5cac01a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46006/0 (socket says 172.21.15.46:46006) 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc48b80700 1 -- 172.21.15.46:0/3770126998 learned_addr learned my addr 172.21.15.46:0/3770126998 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.041+0000 7fcc437fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1e90 0x55b5cac039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59966/0 (socket says 172.21.15.46:59966) 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc43fff700 1 --2- 172.21.15.46:0/3770126998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1570 0x55b5cac03460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc48b80700 1 -- 172.21.15.46:0/3770126998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1e90 msgr2=0x55b5cac039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc48b80700 1 --2- 172.21.15.46:0/3770126998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1e90 0x55b5cac039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc48b80700 1 -- 172.21.15.46:0/3770126998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1570 msgr2=0x55b5cac03460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc48b80700 1 --2- 172.21.15.46:0/3770126998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1570 0x55b5cac03460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc48b80700 1 -- 172.21.15.46:0/3770126998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc34017070 con 0x55b5cabfcdb0 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc48b80700 1 --2- 172.21.15.46:0/3770126998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 0x55b5cac01a20 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009570 tx=0x7fcc3c00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef1cb756d04c184 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc237fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc237fe700 1 -- 172.21.15.46:0/3770126998 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c00e070 con 0x55b5cabfcdb0 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc237fe700 1 -- 172.21.15.46:0/3770126998 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fcc3c007790 con 0x55b5cabfcdb0 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc56778b80 1 -- 172.21.15.46:0/3770126998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc34021070 con 0x55b5cabfcdb0 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc237fe700 1 -- 172.21.15.46:0/3770126998 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc3c003d70 con 0x55b5cabfcdb0 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc56778b80 10 client.18291 fetch_fsmap learned FSMap version 43 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.042+0000 7fcc56778b80 1 -- 172.21.15.46:0/3770126998 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fcc4410e400 con 0x55b5cabfcdb0 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.043+0000 7fcc237fe700 1 -- 172.21.15.46:0/3770126998 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc34021070 con 0x55b5cabfcdb0 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.043+0000 7fcc237fe700 1 -- 172.21.15.46:0/3770126998 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcc3c00b0d0 con 0x55b5cabfcdb0 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.043+0000 7fcc56778b80 10 client.18291 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.043+0000 7fcc56778b80 10 client.18291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.043+0000 7fcc56778b80 20 client.18291 populate_metadata read hostname 'smithi046' 2024-03-18T20:43:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.043+0000 7fcc56778b80 1 --2- 172.21.15.46:0/3770126998 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b5cac618b0 0x55b5cac63cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:43:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.043+0000 7fcc56778b80 4 client.18291 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:43:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.043+0000 7fcc56778b80 1 -- 172.21.15.46:0/3770126998 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b5ca993c90 con 0x55b5cac618b0 2024-03-18T20:43:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.043+0000 7fcc43fff700 1 --2- 172.21.15.46:0/3770126998 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b5cac618b0 0x55b5cac63cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:43:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.043+0000 7fcc237fe700 10 client.18291 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.043+0000 7fcc43fff700 1 --2- 172.21.15.46:0/3770126998 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b5cac618b0 0x55b5cac63cd0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b590cd3994fa5fe server_cookie=7fd69fffc7ec0b60 in_seq=0 out_seq=0 2024-03-18T20:43:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.070+0000 7fcc237fe700 1 -- 172.21.15.46:0/3770126998 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b5ca993c90 con 0x55b5cac618b0 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.070+0000 7fcc237fe700 10 client.18291 handle_command_reply: tid=0 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.070+0000 7fcc56778b80 1 client.18291 shutdown 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.070+0000 7fcc56778b80 1 -- 172.21.15.46:0/3770126998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 msgr2=0x55b5cac01a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.070+0000 7fcc56778b80 1 --2- 172.21.15.46:0/3770126998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 0x55b5cac01a20 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009570 tx=0x7fcc3c00bf30 comp rx=0 tx=0).stop 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.070+0000 7fcc56778b80 1 -- 172.21.15.46:0/3770126998 shutdown_connections 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.070+0000 7fcc56778b80 1 --2- 172.21.15.46:0/3770126998 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b5cac618b0 0x55b5cac63cd0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.070+0000 7fcc56778b80 1 --2- 172.21.15.46:0/3770126998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5cabfcdb0 0x55b5cac01a20 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.070+0000 7fcc56778b80 1 --2- 172.21.15.46:0/3770126998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b5cabf1570 0x55b5cac03460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.070+0000 7fcc237fe700 0 client.18291 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.070+0000 7fcc56778b80 1 --2- 172.21.15.46:0/3770126998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b5cabf1e90 0x55b5cac039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.070+0000 7fcc56778b80 1 -- 172.21.15.46:0/3770126998 >> 172.21.15.46:0/3770126998 conn(0x55b5cab681c0 msgr2=0x55b5cac08190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc56778b80 1 -- 172.21.15.46:0/3770126998 shutdown_connections 2024-03-18T20:43:52.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc56778b80 1 -- 172.21.15.46:0/3770126998 wait complete. 2024-03-18T20:43:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc56778b80 20 client.18291 trim_cache size 0 max 16384 2024-03-18T20:43:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:43:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:43:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:43:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:43:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1743924184 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcc300a1290 msgr2=0x7fcc300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc4a5e3700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcc300a1290 0x7fcc300a3750 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009c60 tx=0x7fcc34009260 comp rx=0 tx=0).stop 2024-03-18T20:43:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1743924184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc44102780 msgr2=0x7fcc44197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:43:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc4a5e3700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc44102780 0x7fcc44197810 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00e9f0 tx=0x7fcc2c00ed00 comp rx=0 tx=0).stop 2024-03-18T20:43:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1743924184 shutdown_connections 2024-03-18T20:43:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc4a5e3700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcc300a1290 0x7fcc300a3750 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc4a5e3700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc4410b280 0x7fcc4419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc4a5e3700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc44101e60 0x7fcc441972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc4a5e3700 1 --2- 172.21.15.46:0/1743924184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc44102780 0x7fcc44197810 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:43:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1743924184 >> 172.21.15.46:0/1743924184 conn(0x7fcc440fb680 msgr2=0x7fcc440fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:43:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1743924184 shutdown_connections 2024-03-18T20:43:52.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:43:52.071+0000 7fcc4a5e3700 1 -- 172.21.15.46:0/1743924184 wait complete. 2024-03-18T20:43:52.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:52 smithi073 ceph-mon[47384]: pgmap v238: 97 pgs: 97 active+clean; 105 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-03-18T20:43:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:52 smithi155 ceph-mon[48344]: pgmap v238: 97 pgs: 97 active+clean; 105 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-03-18T20:43:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:52 smithi046 ceph-mon[43495]: pgmap v238: 97 pgs: 97 active+clean; 105 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-03-18T20:43:52.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 271294 3.55 MB/sec execute 287 sec latency 803.817 ms 2024-03-18T20:43:53.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 271294 3.54 MB/sec execute 288 sec latency 1803.932 ms 2024-03-18T20:43:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:54 smithi073 ceph-mon[47384]: pgmap v239: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-03-18T20:43:54.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:54 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:43:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:54 smithi155 ceph-mon[48344]: pgmap v239: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-03-18T20:43:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:54 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:43:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:54 smithi046 ceph-mon[43495]: pgmap v239: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-03-18T20:43:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:54 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:43:54.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 271294 3.52 MB/sec execute 289 sec latency 2804.028 ms 2024-03-18T20:43:55.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 272184 3.52 MB/sec execute 290 sec latency 3425.854 ms 2024-03-18T20:43:56.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:55 smithi073 ceph-mon[47384]: pgmap v240: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1014 KiB/s wr, 116 op/s 2024-03-18T20:43:56.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:55 smithi155 ceph-mon[48344]: pgmap v240: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1014 KiB/s wr, 116 op/s 2024-03-18T20:43:56.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:55 smithi046 ceph-mon[43495]: pgmap v240: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1014 KiB/s wr, 116 op/s 2024-03-18T20:43:56.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 273628 3.55 MB/sec execute 291 sec latency 14.200 ms 2024-03-18T20:43:57.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 274843 3.55 MB/sec execute 292 sec latency 297.520 ms 2024-03-18T20:43:58.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:58 smithi073 ceph-mon[47384]: pgmap v241: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1014 KiB/s wr, 116 op/s 2024-03-18T20:43:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:58 smithi155 ceph-mon[48344]: pgmap v241: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1014 KiB/s wr, 116 op/s 2024-03-18T20:43:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:58 smithi046 ceph-mon[43495]: pgmap v241: 97 pgs: 97 active+clean; 104 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 1014 KiB/s wr, 116 op/s 2024-03-18T20:43:58.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 274843 3.54 MB/sec execute 293 sec latency 1297.580 ms 2024-03-18T20:43:59.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:43:59 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:43:59.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:43:59 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:43:59.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:43:59 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:43:59.702 INFO:tasks.workunit.client.0.smithi046.stdout: 1 274843 3.53 MB/sec execute 294 sec latency 2297.702 ms 2024-03-18T20:44:00.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:00 smithi073 ceph-mon[47384]: pgmap v242: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-03-18T20:44:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:00 smithi155 ceph-mon[48344]: pgmap v242: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-03-18T20:44:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:00 smithi046 ceph-mon[43495]: pgmap v242: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-03-18T20:44:00.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 275438 3.52 MB/sec execute 295 sec latency 2915.429 ms 2024-03-18T20:44:01.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:01 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:44:01.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:01 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:44:01.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 276905 3.55 MB/sec execute 296 sec latency 39.548 ms 2024-03-18T20:44:01.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:01 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:44:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:02 smithi155 ceph-mon[48344]: pgmap v243: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-03-18T20:44:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:02 smithi046 ceph-mon[43495]: pgmap v243: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-03-18T20:44:02.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 278199 3.56 MB/sec execute 297 sec latency 15.452 ms 2024-03-18T20:44:02.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:02 smithi073 ceph-mon[47384]: pgmap v243: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 1.1 MiB/s wr, 118 op/s 2024-03-18T20:44:03.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 278385 3.55 MB/sec execute 298 sec latency 885.821 ms 2024-03-18T20:44:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:04 smithi155 ceph-mon[48344]: pgmap v244: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-03-18T20:44:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:04 smithi155 ceph-mon[48344]: osdmap e85: 12 total, 12 up, 12 in 2024-03-18T20:44:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:04 smithi046 ceph-mon[43495]: pgmap v244: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-03-18T20:44:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:04 smithi046 ceph-mon[43495]: osdmap e85: 12 total, 12 up, 12 in 2024-03-18T20:44:04.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 278385 3.53 MB/sec execute 299 sec latency 1885.924 ms 2024-03-18T20:44:04.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:04 smithi073 ceph-mon[47384]: pgmap v244: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-03-18T20:44:04.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:04 smithi073 ceph-mon[47384]: osdmap e85: 12 total, 12 up, 12 in 2024-03-18T20:44:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.205+0000 7fe7616ac700 1 -- 172.21.15.46:0/2889618128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe75c106e10 msgr2=0x7fe75c1071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.205+0000 7fe7616ac700 1 --2- 172.21.15.46:0/2889618128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe75c106e10 0x7fe75c1071f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009990 tx=0x7fe74c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.205+0000 7fe7616ac700 1 -- 172.21.15.46:0/2889618128 shutdown_connections 2024-03-18T20:44:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.205+0000 7fe7616ac700 1 --2- 172.21.15.46:0/2889618128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe75c106e10 0x7fe75c1071f0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.205+0000 7fe7616ac700 1 --2- 172.21.15.46:0/2889618128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe75c068aa0 0x7fe75c10dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.205+0000 7fe7616ac700 1 --2- 172.21.15.46:0/2889618128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe75c0680e0 0x7fe75c068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.205+0000 7fe7616ac700 1 -- 172.21.15.46:0/2889618128 >> 172.21.15.46:0/2889618128 conn(0x7fe75c076d50 msgr2=0x7fe75c077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.205+0000 7fe7616ac700 1 -- 172.21.15.46:0/2889618128 shutdown_connections 2024-03-18T20:44:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.205+0000 7fe7616ac700 1 -- 172.21.15.46:0/2889618128 wait complete. 2024-03-18T20:44:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.206+0000 7fe7616ac700 1 Processor -- start 2024-03-18T20:44:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.206+0000 7fe7616ac700 1 -- start start 2024-03-18T20:44:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.206+0000 7fe7616ac700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe75c0680e0 0x7fe75c1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.206+0000 7fe7616ac700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe75c068aa0 0x7fe75c1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.206+0000 7fe7616ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe75c106e10 0x7fe75c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.206+0000 7fe7616ac700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c19bfd0 con 0x7fe75c106e10 2024-03-18T20:44:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.206+0000 7fe7616ac700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c19c140 con 0x7fe75c068aa0 2024-03-18T20:44:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.206+0000 7fe7616ac700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c19c300 con 0x7fe75c0680e0 2024-03-18T20:44:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.206+0000 7fe75b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe75c106e10 0x7fe75c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe75b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe75c106e10 0x7fe75c19b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42992/0 (socket says 172.21.15.46:42992) 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe75b7fe700 1 -- 172.21.15.46:0/1657734257 learned_addr learned my addr 172.21.15.46:0/1657734257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe7527fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe75c068aa0 0x7fe75c1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe75affd700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe75c0680e0 0x7fe75c1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe75b7fe700 1 -- 172.21.15.46:0/1657734257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe75c0680e0 msgr2=0x7fe75c1970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe75c0680e0 0x7fe75c1970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe75b7fe700 1 -- 172.21.15.46:0/1657734257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe75c068aa0 msgr2=0x7fe75c1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe75c068aa0 0x7fe75c1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe75b7fe700 1 -- 172.21.15.46:0/1657734257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754009c40 con 0x7fe75c106e10 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe7527fc700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe75c068aa0 0x7fe75c1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe75c106e10 0x7fe75c19b990 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c370 tx=0x7fe75400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe758ff9700 1 -- 172.21.15.46:0/1657734257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754017070 con 0x7fe75c106e10 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe7616ac700 1 -- 172.21.15.46:0/1657734257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74c0095f0 con 0x7fe75c106e10 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe758ff9700 1 -- 172.21.15.46:0/1657734257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe75400f040 con 0x7fe75c106e10 2024-03-18T20:44:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe758ff9700 1 -- 172.21.15.46:0/1657734257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75400e050 con 0x7fe75c106e10 2024-03-18T20:44:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.207+0000 7fe7616ac700 1 -- 172.21.15.46:0/1657734257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c1a3c40 con 0x7fe75c106e10 2024-03-18T20:44:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.209+0000 7fe758ff9700 1 -- 172.21.15.46:0/1657734257 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe75400cb60 con 0x7fe75c106e10 2024-03-18T20:44:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.210+0000 7fe758ff9700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe7480a1290 0x7fe7480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.210+0000 7fe75affd700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe7480a1290 0x7fe7480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.210+0000 7fe758ff9700 1 -- 172.21.15.46:0/1657734257 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fe7540cc6c0 con 0x7fe75c106e10 2024-03-18T20:44:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.211+0000 7fe75affd700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe7480a1290 0x7fe7480a3750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0098e0 tx=0x7fe74c005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.211+0000 7fe76d841b80 1 Processor -- start 2024-03-18T20:44:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.211+0000 7fe76d841b80 1 -- start start 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe76d841b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260af950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe76d841b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260afe90 0x55be260ba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe76d841b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260badb0 0x55be260bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe76d841b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be26009930 con 0x55be260af570 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe76d841b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be260072f0 con 0x55be260afe90 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe76d841b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be26002be0 con 0x55be260badb0 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe75affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe7527fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260afe90 0x55be260ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe75b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260badb0 0x55be260bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe75affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43006/0 (socket says 172.21.15.46:43006) 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe7527fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260afe90 0x55be260ba870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52460/0 (socket says 172.21.15.46:52460) 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe75affd700 1 -- 172.21.15.46:0/2116940713 learned_addr learned my addr 172.21.15.46:0/2116940713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe75b7fe700 1 -- 172.21.15.46:0/2116940713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260afe90 msgr2=0x55be260ba870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/2116940713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260afe90 0x55be260ba870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe75b7fe700 1 -- 172.21.15.46:0/2116940713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 msgr2=0x55be260af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/2116940713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.212+0000 7fe75b7fe700 1 -- 172.21.15.46:0/2116940713 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be25e84350 con 0x55be260badb0 2024-03-18T20:44:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.213+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/2116940713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260badb0 0x55be260bd1a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe7540c9d60 tx=0x7fe754099080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=987ad9ee710a7264 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.213+0000 7fe7517fa700 1 -- 172.21.15.46:0/2116940713 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754099720 con 0x55be260badb0 2024-03-18T20:44:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.213+0000 7fe7517fa700 1 -- 172.21.15.46:0/2116940713 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe75409d550 con 0x55be260badb0 2024-03-18T20:44:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.213+0000 7fe7517fa700 1 -- 172.21.15.46:0/2116940713 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75409d170 con 0x55be260badb0 2024-03-18T20:44:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.213+0000 7fe76d841b80 1 -- 172.21.15.46:0/2116940713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260badb0 msgr2=0x55be260bd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.213+0000 7fe76d841b80 1 --2- 172.21.15.46:0/2116940713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260badb0 0x55be260bd1a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe7540c9d60 tx=0x7fe754099080 comp rx=0 tx=0).stop 2024-03-18T20:44:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.214+0000 7fe76d841b80 1 -- 172.21.15.46:0/2116940713 shutdown_connections 2024-03-18T20:44:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.214+0000 7fe76d841b80 1 --2- 172.21.15.46:0/2116940713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.214+0000 7fe76d841b80 1 --2- 172.21.15.46:0/2116940713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260badb0 0x55be260bd1a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.214+0000 7fe76d841b80 1 --2- 172.21.15.46:0/2116940713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260afe90 0x55be260ba870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.214+0000 7fe76d841b80 1 -- 172.21.15.46:0/2116940713 >> 172.21.15.46:0/2116940713 conn(0x55be260261c0 msgr2=0x55be260aba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.214+0000 7fe76d841b80 1 -- 172.21.15.46:0/2116940713 shutdown_connections 2024-03-18T20:44:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.214+0000 7fe76d841b80 1 -- 172.21.15.46:0/2116940713 wait complete. 2024-03-18T20:44:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.214+0000 7fe76d841b80 1 -- start start 2024-03-18T20:44:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.214+0000 7fe76d841b80 1 Processor -- start 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe76d841b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe76d841b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260afe90 0x55be260b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe76d841b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 0x55be260b8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe76d841b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be25f9ed20 con 0x55be260af570 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe76d841b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be25f9e9c0 con 0x55be260badb0 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe76d841b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be25e86640 con 0x55be260afe90 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe76d841b80 10 client.0 _ll_register_callbacks cb 0x55be25e0a340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe75affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe75affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260b7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43014/0 (socket says 172.21.15.46:43014) 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe75affd700 1 -- 172.21.15.46:0/2630476592 learned_addr learned my addr 172.21.15.46:0/2630476592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe7527fc700 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260afe90 0x55be260b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 0x55be260b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe75affd700 1 -- 172.21.15.46:0/2630476592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260afe90 msgr2=0x55be260b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe75affd700 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260afe90 0x55be260b8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe75affd700 1 -- 172.21.15.46:0/2630476592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 msgr2=0x55be260b8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe75affd700 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 0x55be260b8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe75affd700 1 -- 172.21.15.46:0/2630476592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c1a3c40 con 0x55be260af570 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 0x55be260b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe7527fc700 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260afe90 0x55be260b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe75affd700 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260b7bd0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0059d0 tx=0x7fe74c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ffe6e0271eb775 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.215+0000 7fe7427fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe7427fc700 1 -- 172.21.15.46:0/2630476592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c004e50 con 0x55be260af570 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe7427fc700 1 -- 172.21.15.46:0/2630476592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fe74c00f460 con 0x55be260af570 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe76d841b80 1 -- 172.21.15.46:0/2630476592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be260a36c0 con 0x55be260af570 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe7427fc700 1 -- 172.21.15.46:0/2630476592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe74c00f620 con 0x55be260af570 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe76d841b80 10 client.18303 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe76d841b80 1 -- 172.21.15.46:0/2630476592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55be25e30b00 con 0x55be260af570 2024-03-18T20:44:05.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe7427fc700 1 -- 172.21.15.46:0/2630476592 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be260a36c0 con 0x55be260af570 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe7427fc700 1 -- 172.21.15.46:0/2630476592 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe74c02f070 con 0x55be260af570 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe76d841b80 10 client.18303 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe76d841b80 10 client.18303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe76d841b80 20 client.18303 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe76d841b80 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55be260caab0 0x55be260cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe76d841b80 4 client.18303 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe76d841b80 1 -- 172.21.15.46:0/2630476592 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be25ffb6a0 con 0x55be260caab0 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.216+0000 7fe7527fc700 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55be260caab0 0x55be260cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.217+0000 7fe7427fc700 10 client.18303 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.217+0000 7fe7527fc700 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55be260caab0 0x55be260cced0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522e731506525df7 server_cookie=1ee881a7698e9246 in_seq=0 out_seq=0 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe7427fc700 1 -- 172.21.15.46:0/2630476592 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55be25ffb6a0 con 0x55be260caab0 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe7427fc700 10 client.18303 handle_command_reply: tid=0 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe76d841b80 1 client.18303 shutdown 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe76d841b80 1 -- 172.21.15.46:0/2630476592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 msgr2=0x55be260b7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:05.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe76d841b80 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260b7bd0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0059d0 tx=0x7fe74c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe76d841b80 1 -- 172.21.15.46:0/2630476592 shutdown_connections 2024-03-18T20:44:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe76d841b80 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55be260caab0 0x55be260cced0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe76d841b80 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260b7bd0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe76d841b80 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 0x55be260b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe7427fc700 0 client.18303 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe76d841b80 1 --2- 172.21.15.46:0/2630476592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260afe90 0x55be260b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe76d841b80 1 -- 172.21.15.46:0/2630476592 >> 172.21.15.46:0/2630476592 conn(0x55be260261c0 msgr2=0x55be260ab650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe76d841b80 1 -- 172.21.15.46:0/2630476592 shutdown_connections 2024-03-18T20:44:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe76d841b80 1 -- 172.21.15.46:0/2630476592 wait complete. 2024-03-18T20:44:05.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.218+0000 7fe76d841b80 20 client.18303 trim_cache size 0 max 16384 2024-03-18T20:44:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.234+0000 7fe76d841b80 1 Processor -- start 2024-03-18T20:44:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.234+0000 7fe76d841b80 1 -- start start 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe76d841b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe76d841b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260afe90 0x55be260b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe76d841b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260badb0 0x55be260b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe76d841b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be260bac10 con 0x55be260af570 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe76d841b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be260acfa0 con 0x55be260afe90 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe76d841b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be260a3230 con 0x55be260badb0 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe75affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe75affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43026/0 (socket says 172.21.15.46:43026) 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe75affd700 1 -- 172.21.15.46:0/3200981423 learned_addr learned my addr 172.21.15.46:0/3200981423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe7527fc700 1 --2- 172.21.15.46:0/3200981423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260afe90 0x55be260b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/3200981423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260badb0 0x55be260b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe75affd700 1 -- 172.21.15.46:0/3200981423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260badb0 msgr2=0x55be260b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe75affd700 1 --2- 172.21.15.46:0/3200981423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260badb0 0x55be260b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe75affd700 1 -- 172.21.15.46:0/3200981423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260afe90 msgr2=0x55be260b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe75affd700 1 --2- 172.21.15.46:0/3200981423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260afe90 0x55be260b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe75affd700 1 -- 172.21.15.46:0/3200981423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be25fc8e40 con 0x55be260af570 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/3200981423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260badb0 0x55be260b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.235+0000 7fe7527fc700 1 --2- 172.21.15.46:0/3200981423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260afe90 0x55be260b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.236+0000 7fe75affd700 1 --2- 172.21.15.46:0/3200981423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260b25e0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00b360 tx=0x7fe74c00bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8242a9f3c46baae server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.236+0000 7fe751ffb700 1 -- 172.21.15.46:0/3200981423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c005570 con 0x55be260af570 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.236+0000 7fe751ffb700 1 -- 172.21.15.46:0/3200981423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe74c028070 con 0x55be260af570 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.236+0000 7fe751ffb700 1 -- 172.21.15.46:0/3200981423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c00f920 con 0x55be260af570 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.236+0000 7fe76d841b80 1 -- 172.21.15.46:0/3200981423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 msgr2=0x55be260b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.236+0000 7fe76d841b80 1 --2- 172.21.15.46:0/3200981423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260b25e0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00b360 tx=0x7fe74c00bf70 comp rx=0 tx=0).stop 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 -- 172.21.15.46:0/3200981423 shutdown_connections 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 --2- 172.21.15.46:0/3200981423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260af570 0x55be260b25e0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 --2- 172.21.15.46:0/3200981423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260badb0 0x55be260b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.241 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 --2- 172.21.15.46:0/3200981423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260afe90 0x55be260b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 -- 172.21.15.46:0/3200981423 >> 172.21.15.46:0/3200981423 conn(0x55be260261c0 msgr2=0x55be260b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 -- 172.21.15.46:0/3200981423 shutdown_connections 2024-03-18T20:44:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 -- 172.21.15.46:0/3200981423 wait complete. 2024-03-18T20:44:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 -- start start 2024-03-18T20:44:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 Processor -- start 2024-03-18T20:44:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260af570 0x55be260bf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260afe90 0x55be260bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 0x55be260c03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be2611d920 con 0x55be260afe90 2024-03-18T20:44:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be2611da90 con 0x55be260badb0 2024-03-18T20:44:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be2611dc50 con 0x55be260af570 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.237+0000 7fe76d841b80 10 client.0 _ll_register_callbacks cb 0x7fe74c00ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe75b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 0x55be260c03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe7527fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260afe90 0x55be260bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe75b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 0x55be260c03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52480/0 (socket says 172.21.15.46:52480) 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe75b7fe700 1 -- 172.21.15.46:0/4198374128 learned_addr learned my addr 172.21.15.46:0/4198374128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe75affd700 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260af570 0x55be260bf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe75b7fe700 1 -- 172.21.15.46:0/4198374128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260af570 msgr2=0x55be260bf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260af570 0x55be260bf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe75b7fe700 1 -- 172.21.15.46:0/4198374128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260afe90 msgr2=0x55be260bfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260afe90 0x55be260bfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe75b7fe700 1 -- 172.21.15.46:0/4198374128 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe744012ae0 con 0x55be260badb0 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe75affd700 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260af570 0x55be260bf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe75b7fe700 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 0x55be260c03e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe754096190 tx=0x7fe754095b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=304a6c9c0b17eded server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe7427fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe7427fc700 1 -- 172.21.15.46:0/4198374128 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754099720 con 0x55be260badb0 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe7427fc700 1 -- 172.21.15.46:0/4198374128 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fe754099980 con 0x55be260badb0 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe76d841b80 1 -- 172.21.15.46:0/4198374128 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe75409d550 con 0x55be260badb0 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe7427fc700 1 -- 172.21.15.46:0/4198374128 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe754099b40 con 0x55be260badb0 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe76d841b80 10 client.27965 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.238+0000 7fe76d841b80 1 -- 172.21.15.46:0/4198374128 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55be25e84350 con 0x55be260badb0 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.239+0000 7fe7427fc700 1 -- 172.21.15.46:0/4198374128 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe75409d550 con 0x55be260badb0 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.239+0000 7fe7427fc700 1 -- 172.21.15.46:0/4198374128 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe754098cf0 con 0x55be260badb0 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.239+0000 7fe76d841b80 10 client.27965 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.239+0000 7fe76d841b80 10 client.27965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.239+0000 7fe76d841b80 20 client.27965 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.239+0000 7fe76d841b80 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55be2611f120 0x55be26121510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.239+0000 7fe76d841b80 4 client.27965 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:44:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.239+0000 7fe76d841b80 1 -- 172.21.15.46:0/4198374128 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be25e51c90 con 0x55be2611f120 2024-03-18T20:44:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.239+0000 7fe75affd700 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55be2611f120 0x55be26121510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.239+0000 7fe75affd700 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55be2611f120 0x55be26121510 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3951daef74b295d3 server_cookie=f0a582c4c1c664ed in_seq=0 out_seq=0 2024-03-18T20:44:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.239+0000 7fe7427fc700 10 client.27965 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe7427fc700 1 -- 172.21.15.46:0/4198374128 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be25e51c90 con 0x55be2611f120 2024-03-18T20:44:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe7427fc700 10 client.27965 handle_command_reply: tid=0 2024-03-18T20:44:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe76d841b80 1 client.27965 shutdown 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe76d841b80 1 -- 172.21.15.46:0/4198374128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 msgr2=0x55be260c03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe76d841b80 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 0x55be260c03e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe754096190 tx=0x7fe754095b90 comp rx=0 tx=0).stop 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe76d841b80 1 -- 172.21.15.46:0/4198374128 shutdown_connections 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe76d841b80 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55be2611f120 0x55be26121510 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe76d841b80 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55be260badb0 0x55be260c03e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe76d841b80 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55be260af570 0x55be260bf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe7427fc700 0 client.27965 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe76d841b80 1 --2- 172.21.15.46:0/4198374128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be260afe90 0x55be260bfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe76d841b80 1 -- 172.21.15.46:0/4198374128 >> 172.21.15.46:0/4198374128 conn(0x55be260261c0 msgr2=0x55be260b9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe76d841b80 1 -- 172.21.15.46:0/4198374128 shutdown_connections 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe76d841b80 1 -- 172.21.15.46:0/4198374128 wait complete. 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.267+0000 7fe76d841b80 20 client.27965 trim_cache size 0 max 16384 2024-03-18T20:44:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:44:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:44:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:44:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:44:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.268+0000 7fe7616ac700 1 -- 172.21.15.46:0/1657734257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe7480a1290 msgr2=0x7fe7480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.268+0000 7fe7616ac700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe7480a1290 0x7fe7480a3750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0098e0 tx=0x7fe74c005960 comp rx=0 tx=0).stop 2024-03-18T20:44:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.268+0000 7fe7616ac700 1 -- 172.21.15.46:0/1657734257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe75c106e10 msgr2=0x7fe75c19b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.268+0000 7fe7616ac700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe75c106e10 0x7fe75c19b990 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe75400c370 tx=0x7fe75400c680 comp rx=0 tx=0).stop 2024-03-18T20:44:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.268+0000 7fe7616ac700 1 -- 172.21.15.46:0/1657734257 shutdown_connections 2024-03-18T20:44:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.268+0000 7fe7616ac700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe7480a1290 0x7fe7480a3750 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.268+0000 7fe7616ac700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe75c106e10 0x7fe75c19b990 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.268+0000 7fe7616ac700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe75c0680e0 0x7fe75c1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.268+0000 7fe7616ac700 1 --2- 172.21.15.46:0/1657734257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe75c068aa0 0x7fe75c1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.268+0000 7fe7616ac700 1 -- 172.21.15.46:0/1657734257 >> 172.21.15.46:0/1657734257 conn(0x7fe75c076d50 msgr2=0x7fe75c0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.268+0000 7fe7616ac700 1 -- 172.21.15.46:0/1657734257 shutdown_connections 2024-03-18T20:44:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:05.268+0000 7fe7616ac700 1 -- 172.21.15.46:0/1657734257 wait complete. 2024-03-18T20:44:05.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 280346 3.56 MB/sec execute 300 sec latency 2039.389 ms 2024-03-18T20:44:06.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:05 smithi073 ceph-mon[47384]: pgmap v246: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:44:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:05 smithi155 ceph-mon[48344]: pgmap v246: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:44:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:05 smithi046 ceph-mon[43495]: pgmap v246: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:44:06.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 281447 3.56 MB/sec execute 301 sec latency 17.590 ms 2024-03-18T20:44:07.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 281934 3.56 MB/sec execute 302 sec latency 769.938 ms 2024-03-18T20:44:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:08 smithi073 ceph-mon[47384]: pgmap v247: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:44:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:08 smithi155 ceph-mon[48344]: pgmap v247: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:44:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:08 smithi046 ceph-mon[43495]: pgmap v247: 97 pgs: 97 active+clean; 110 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:44:08.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 281934 3.55 MB/sec execute 303 sec latency 1770.073 ms 2024-03-18T20:44:09.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 281934 3.54 MB/sec execute 304 sec latency 2770.180 ms 2024-03-18T20:44:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:10 smithi073 ceph-mon[47384]: pgmap v248: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-03-18T20:44:10.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:10 smithi073 ceph-mon[47384]: osdmap e86: 12 total, 12 up, 12 in 2024-03-18T20:44:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:10 smithi155 ceph-mon[48344]: pgmap v248: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-03-18T20:44:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:10 smithi155 ceph-mon[48344]: osdmap e86: 12 total, 12 up, 12 in 2024-03-18T20:44:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:10 smithi046 ceph-mon[43495]: pgmap v248: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-03-18T20:44:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:10 smithi046 ceph-mon[43495]: osdmap e86: 12 total, 12 up, 12 in 2024-03-18T20:44:10.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 282832 3.54 MB/sec execute 305 sec latency 3387.626 ms 2024-03-18T20:44:11.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 284341 3.56 MB/sec execute 306 sec latency 19.327 ms 2024-03-18T20:44:11.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:44:11.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:44:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.942+0000 7ff33408e700 1 -- 172.21.15.46:0/159890625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff32c100820 msgr2=0x7ff32c104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.942+0000 7ff33408e700 1 --2- 172.21.15.46:0/159890625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff32c100820 0x7ff32c104860 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009990 tx=0x7ff31c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.942+0000 7ff33408e700 1 -- 172.21.15.46:0/159890625 shutdown_connections 2024-03-18T20:44:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.942+0000 7ff33408e700 1 --2- 172.21.15.46:0/159890625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c104f90 0x7ff32c10cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.942+0000 7ff33408e700 1 --2- 172.21.15.46:0/159890625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff32c0ffe70 0x7ff32c100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.942+0000 7ff33408e700 1 --2- 172.21.15.46:0/159890625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff32c100820 0x7ff32c104860 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.942+0000 7ff33408e700 1 -- 172.21.15.46:0/159890625 >> 172.21.15.46:0/159890625 conn(0x7ff32c076d80 msgr2=0x7ff32c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.942+0000 7ff33408e700 1 -- 172.21.15.46:0/159890625 shutdown_connections 2024-03-18T20:44:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.942+0000 7ff33408e700 1 -- 172.21.15.46:0/159890625 wait complete. 2024-03-18T20:44:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.942+0000 7ff33408e700 1 Processor -- start 2024-03-18T20:44:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.942+0000 7ff33408e700 1 -- start start 2024-03-18T20:44:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff33408e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff32c0ffe70 0x7ff32c197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff33408e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c104f90 0x7ff32c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff33408e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff32c19bbb0 0x7ff32c19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff33408e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c19c630 con 0x7ff32c0ffe70 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff33408e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c19c7a0 con 0x7ff32c104f90 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff33408e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c19c960 con 0x7ff32c19bbb0 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff331e2a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff32c0ffe70 0x7ff32c197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff331629700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c104f90 0x7ff32c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff331e2a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff32c0ffe70 0x7ff32c197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56290/0 (socket says 172.21.15.46:56290) 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff331e2a700 1 -- 172.21.15.46:0/3810602331 learned_addr learned my addr 172.21.15.46:0/3810602331 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff33262b700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff32c19bbb0 0x7ff32c19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff331e2a700 1 -- 172.21.15.46:0/3810602331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff32c19bbb0 msgr2=0x7ff32c19c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff331e2a700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff32c19bbb0 0x7ff32c19c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff331e2a700 1 -- 172.21.15.46:0/3810602331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c104f90 msgr2=0x7ff32c197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff331e2a700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c104f90 0x7ff32c197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff331e2a700 1 -- 172.21.15.46:0/3810602331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff324009c40 con 0x7ff32c0ffe70 2024-03-18T20:44:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff331629700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c104f90 0x7ff32c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.943+0000 7ff331e2a700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff32c0ffe70 0x7ff32c197320 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff32800e9a0 tx=0x7ff32800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.944+0000 7ff322ffd700 1 -- 172.21.15.46:0/3810602331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328009790 con 0x7ff32c0ffe70 2024-03-18T20:44:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.944+0000 7ff322ffd700 1 -- 172.21.15.46:0/3810602331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff32800ce80 con 0x7ff32c0ffe70 2024-03-18T20:44:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.944+0000 7ff33408e700 1 -- 172.21.15.46:0/3810602331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31c0095f0 con 0x7ff32c0ffe70 2024-03-18T20:44:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.944+0000 7ff322ffd700 1 -- 172.21.15.46:0/3810602331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328018680 con 0x7ff32c0ffe70 2024-03-18T20:44:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.944+0000 7ff33408e700 1 -- 172.21.15.46:0/3810602331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c1a41c0 con 0x7ff32c0ffe70 2024-03-18T20:44:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.946+0000 7ff322ffd700 1 -- 172.21.15.46:0/3810602331 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff328010430 con 0x7ff32c0ffe70 2024-03-18T20:44:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.947+0000 7ff322ffd700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff3180a1290 0x7ff3180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.947+0000 7ff331629700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff3180a1290 0x7ff3180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.947+0000 7ff322ffd700 1 -- 172.21.15.46:0/3810602331 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff328014070 con 0x7ff32c0ffe70 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.947+0000 7ff331629700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff3180a1290 0x7ff3180a3750 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff32c198820 tx=0x7ff31c00b260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.947+0000 7ff340223b80 1 Processor -- start 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff340223b80 1 -- start start 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff340223b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342ea60 0x55a74342ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff340223b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 0x55a74343e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff340223b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 0x55a743440a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff340223b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a743388990 con 0x55a74342ea60 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff340223b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a743386350 con 0x55a74342f380 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff340223b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a743381c40 con 0x55a74343e630 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff331629700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 0x55a74343e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff331629700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 0x55a74343e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55606/0 (socket says 172.21.15.46:55606) 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff33262b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 0x55a743440a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff331629700 1 -- 172.21.15.46:0/810794998 learned_addr learned my addr 172.21.15.46:0/810794998 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff331e2a700 1 --2- 172.21.15.46:0/810794998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342ea60 0x55a74342ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff331629700 1 -- 172.21.15.46:0/810794998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 msgr2=0x55a743440a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff331629700 1 --2- 172.21.15.46:0/810794998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 0x55a743440a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff331629700 1 -- 172.21.15.46:0/810794998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342ea60 msgr2=0x55a74342ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff331629700 1 --2- 172.21.15.46:0/810794998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342ea60 0x55a74342ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.948+0000 7ff331629700 1 -- 172.21.15.46:0/810794998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7432033b0 con 0x55a74342f380 2024-03-18T20:44:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.949+0000 7ff331629700 1 --2- 172.21.15.46:0/810794998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 0x55a74343e0f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00bb10 tx=0x7ff31c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f2ce6edbd274ee6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.949+0000 7ff317fff700 1 -- 172.21.15.46:0/810794998 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c02b070 con 0x55a74342f380 2024-03-18T20:44:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.949+0000 7ff317fff700 1 -- 172.21.15.46:0/810794998 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff31c01e040 con 0x55a74342f380 2024-03-18T20:44:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.949+0000 7ff317fff700 1 -- 172.21.15.46:0/810794998 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c00f930 con 0x55a74342f380 2024-03-18T20:44:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.950+0000 7ff340223b80 1 -- 172.21.15.46:0/810794998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 msgr2=0x55a74343e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.950+0000 7ff340223b80 1 --2- 172.21.15.46:0/810794998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 0x55a74343e0f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00bb10 tx=0x7ff31c020000 comp rx=0 tx=0).stop 2024-03-18T20:44:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.950+0000 7ff340223b80 1 -- 172.21.15.46:0/810794998 shutdown_connections 2024-03-18T20:44:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.950+0000 7ff340223b80 1 --2- 172.21.15.46:0/810794998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342ea60 0x55a74342ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.950+0000 7ff340223b80 1 --2- 172.21.15.46:0/810794998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 0x55a743440a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.950+0000 7ff340223b80 1 --2- 172.21.15.46:0/810794998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 0x55a74343e0f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.950+0000 7ff340223b80 1 -- 172.21.15.46:0/810794998 >> 172.21.15.46:0/810794998 conn(0x55a7433a5220 msgr2=0x55a74342b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.950+0000 7ff340223b80 1 -- 172.21.15.46:0/810794998 shutdown_connections 2024-03-18T20:44:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.950+0000 7ff340223b80 1 -- 172.21.15.46:0/810794998 wait complete. 2024-03-18T20:44:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.950+0000 7ff340223b80 1 -- start start 2024-03-18T20:44:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.950+0000 7ff340223b80 1 Processor -- start 2024-03-18T20:44:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff340223b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 0x55a74343dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff340223b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342f380 0x55a74343b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff340223b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 0x55a74343b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff340223b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74331dd80 con 0x55a74342f380 2024-03-18T20:44:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff340223b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74331da20 con 0x55a74342ea60 2024-03-18T20:44:11.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff340223b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7432056a0 con 0x55a74343e630 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff340223b80 10 client.0 _ll_register_callbacks cb 0x55a7431893a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff331e2a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 0x55a74343dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff331629700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342f380 0x55a74343b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff331e2a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 0x55a74343dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55614/0 (socket says 172.21.15.46:55614) 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff33262b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 0x55a74343b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff331e2a700 1 -- 172.21.15.46:0/4026622325 learned_addr learned my addr 172.21.15.46:0/4026622325 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff331e2a700 1 -- 172.21.15.46:0/4026622325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 msgr2=0x55a74343b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff331e2a700 1 --2- 172.21.15.46:0/4026622325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 0x55a74343b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff331e2a700 1 -- 172.21.15.46:0/4026622325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342f380 msgr2=0x55a74343b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff331e2a700 1 --2- 172.21.15.46:0/4026622325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342f380 0x55a74343b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff331e2a700 1 -- 172.21.15.46:0/4026622325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c10d380 con 0x55a74342ea60 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff331e2a700 1 --2- 172.21.15.46:0/4026622325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 0x55a74343dfd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff3280ca680 tx=0x7ff32809a780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c0b58b089bf6a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff3157fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:11.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff3157fa700 1 -- 172.21.15.46:0/4026622325 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32809a3f0 con 0x55a74342ea60 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff3157fa700 1 -- 172.21.15.46:0/4026622325 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff3280a2410 con 0x55a74342ea60 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.951+0000 7ff340223b80 1 -- 172.21.15.46:0/4026622325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a743422ad0 con 0x55a74342ea60 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.952+0000 7ff3157fa700 1 -- 172.21.15.46:0/4026622325 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3280b8170 con 0x55a74342ea60 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.952+0000 7ff340223b80 10 client.27977 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.952+0000 7ff340223b80 1 -- 172.21.15.46:0/4026622325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55a7431afb60 con 0x55a74342ea60 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.952+0000 7ff3157fa700 1 -- 172.21.15.46:0/4026622325 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff328063080 con 0x55a74342ea60 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.952+0000 7ff3157fa700 1 -- 172.21.15.46:0/4026622325 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff3280a1f80 con 0x55a74342ea60 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.952+0000 7ff340223b80 10 client.27977 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.952+0000 7ff340223b80 10 client.27977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.952+0000 7ff340223b80 20 client.27977 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.952+0000 7ff340223b80 1 --2- 172.21.15.46:0/4026622325 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a74344e050 0x55a743450470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.952+0000 7ff340223b80 4 client.27977 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.952+0000 7ff340223b80 1 -- 172.21.15.46:0/4026622325 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a74337a700 con 0x55a74344e050 2024-03-18T20:44:11.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.952+0000 7ff331629700 1 --2- 172.21.15.46:0/4026622325 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a74344e050 0x55a743450470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.953+0000 7ff3157fa700 10 client.27977 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.953+0000 7ff331629700 1 --2- 172.21.15.46:0/4026622325 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a74344e050 0x55a743450470 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b59be8c34740c5b server_cookie=d4c2df50389e2980 in_seq=0 out_seq=0 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.953+0000 7ff3157fa700 1 -- 172.21.15.46:0/4026622325 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a74337a700 con 0x55a74344e050 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.953+0000 7ff3157fa700 10 client.27977 handle_command_reply: tid=0 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.953+0000 7ff340223b80 1 client.27977 shutdown 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.953+0000 7ff340223b80 1 -- 172.21.15.46:0/4026622325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 msgr2=0x55a74343dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.953+0000 7ff340223b80 1 --2- 172.21.15.46:0/4026622325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 0x55a74343dfd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff3280ca680 tx=0x7ff32809a780 comp rx=0 tx=0).stop 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.954+0000 7ff340223b80 1 -- 172.21.15.46:0/4026622325 shutdown_connections 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.954+0000 7ff340223b80 1 --2- 172.21.15.46:0/4026622325 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a74344e050 0x55a743450470 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.954+0000 7ff340223b80 1 --2- 172.21.15.46:0/4026622325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 0x55a74343b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.954+0000 7ff340223b80 1 --2- 172.21.15.46:0/4026622325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 0x55a74343dfd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.954+0000 7ff340223b80 1 --2- 172.21.15.46:0/4026622325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342f380 0x55a74343b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.954+0000 7ff3157fa700 0 client.27977 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.954+0000 7ff340223b80 1 -- 172.21.15.46:0/4026622325 >> 172.21.15.46:0/4026622325 conn(0x55a7433a5220 msgr2=0x55a74342ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:11.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.954+0000 7ff340223b80 1 -- 172.21.15.46:0/4026622325 shutdown_connections 2024-03-18T20:44:11.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.954+0000 7ff340223b80 1 -- 172.21.15.46:0/4026622325 wait complete. 2024-03-18T20:44:11.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.954+0000 7ff340223b80 20 client.27977 trim_cache size 0 max 16384 2024-03-18T20:44:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff340223b80 1 Processor -- start 2024-03-18T20:44:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff340223b80 1 -- start start 2024-03-18T20:44:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff340223b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74342ea60 0x55a7434451a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff340223b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 0x55a74342ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff340223b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74343e630 0x55a74342b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff340223b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a74342fd30 con 0x55a74343e630 2024-03-18T20:44:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff340223b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7434275e0 con 0x55a74342f380 2024-03-18T20:44:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff340223b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a743424f80 con 0x55a74342ea60 2024-03-18T20:44:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff33262b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74343e630 0x55a74342b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff331629700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 0x55a74342ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff331e2a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74342ea60 0x55a7434451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff33262b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74343e630 0x55a74342b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56312/0 (socket says 172.21.15.46:56312) 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff331629700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 0x55a74342ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55630/0 (socket says 172.21.15.46:55630) 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff33262b700 1 -- 172.21.15.46:0/417677452 learned_addr learned my addr 172.21.15.46:0/417677452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff33262b700 1 -- 172.21.15.46:0/417677452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74342ea60 msgr2=0x55a7434451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff33262b700 1 --2- 172.21.15.46:0/417677452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74342ea60 0x55a7434451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff33262b700 1 -- 172.21.15.46:0/417677452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 msgr2=0x55a74342ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff33262b700 1 --2- 172.21.15.46:0/417677452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 0x55a74342ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff33262b700 1 -- 172.21.15.46:0/417677452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a743347ea0 con 0x55a74343e630 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.972+0000 7ff331e2a700 1 --2- 172.21.15.46:0/417677452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74342ea60 0x55a7434451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.973+0000 7ff33262b700 1 --2- 172.21.15.46:0/417677452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74343e630 0x55a74342b090 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7ff32400e9c0 tx=0x7ff324009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b7e707da655855 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.973+0000 7ff320ff9700 1 -- 172.21.15.46:0/417677452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32400e040 con 0x55a74343e630 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.973+0000 7ff320ff9700 1 -- 172.21.15.46:0/417677452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff32400f040 con 0x55a74343e630 2024-03-18T20:44:11.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.973+0000 7ff320ff9700 1 -- 172.21.15.46:0/417677452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff324007b40 con 0x55a74343e630 2024-03-18T20:44:11.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.974+0000 7ff340223b80 1 -- 172.21.15.46:0/417677452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74343e630 msgr2=0x55a74342b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:11.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.974+0000 7ff340223b80 1 --2- 172.21.15.46:0/417677452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74343e630 0x55a74342b090 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7ff32400e9c0 tx=0x7ff324009920 comp rx=0 tx=0).stop 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.974+0000 7ff340223b80 1 -- 172.21.15.46:0/417677452 shutdown_connections 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.974+0000 7ff340223b80 1 --2- 172.21.15.46:0/417677452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74343e630 0x55a74342b090 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.974+0000 7ff340223b80 1 --2- 172.21.15.46:0/417677452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74342ea60 0x55a7434451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.974+0000 7ff340223b80 1 --2- 172.21.15.46:0/417677452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342f380 0x55a74342ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.974+0000 7ff340223b80 1 -- 172.21.15.46:0/417677452 >> 172.21.15.46:0/417677452 conn(0x55a7433a5220 msgr2=0x55a743429750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.974+0000 7ff340223b80 1 -- 172.21.15.46:0/417677452 shutdown_connections 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.974+0000 7ff340223b80 1 -- 172.21.15.46:0/417677452 wait complete. 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.974+0000 7ff340223b80 1 -- start start 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.974+0000 7ff340223b80 1 Processor -- start 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff340223b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 0x55a743443860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff340223b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342f380 0x55a743443da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff340223b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 0x55a7434442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff340223b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7434a1f70 con 0x55a74342f380 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff340223b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7434a20e0 con 0x55a74342ea60 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff340223b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7434a22a0 con 0x55a74343e630 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff340223b80 10 client.0 _ll_register_callbacks cb 0x7ff3280998f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff331e2a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 0x55a743443860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff331629700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342f380 0x55a743443da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff331e2a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 0x55a743443860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55638/0 (socket says 172.21.15.46:55638) 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff331e2a700 1 -- 172.21.15.46:0/388372171 learned_addr learned my addr 172.21.15.46:0/388372171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff33262b700 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 0x55a7434442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff331e2a700 1 -- 172.21.15.46:0/388372171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 msgr2=0x55a7434442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff331e2a700 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 0x55a7434442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff331e2a700 1 -- 172.21.15.46:0/388372171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342f380 msgr2=0x55a743443da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff331e2a700 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342f380 0x55a743443da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff331e2a700 1 -- 172.21.15.46:0/388372171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7431afb60 con 0x55a74342ea60 2024-03-18T20:44:11.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff331629700 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342f380 0x55a743443da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff331e2a700 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 0x55a743443860 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff328099f20 tx=0x7ff3280a21b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d2194347ab9eae server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.975+0000 7ff3157fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff3157fa700 1 -- 172.21.15.46:0/388372171 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328099680 con 0x55a74342ea60 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff340223b80 1 -- 172.21.15.46:0/388372171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff31c01e040 con 0x55a74342ea60 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff3157fa700 1 -- 172.21.15.46:0/388372171 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff32805fe40 con 0x55a74342ea60 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff3157fa700 1 -- 172.21.15.46:0/388372171 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3280b8380 con 0x55a74342ea60 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff340223b80 10 client.27983 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff340223b80 1 -- 172.21.15.46:0/388372171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7ff32400e610 con 0x55a74342ea60 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff3157fa700 1 -- 172.21.15.46:0/388372171 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff31c01e040 con 0x55a74342ea60 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff3157fa700 1 -- 172.21.15.46:0/388372171 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff328094ba0 con 0x55a74342ea60 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff340223b80 10 client.27983 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff340223b80 10 client.27983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff340223b80 20 client.27983 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff340223b80 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a7434a3640 0x55a7434a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:11.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff340223b80 4 client.27983 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff340223b80 1 -- 172.21.15.46:0/388372171 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7431d0cf0 con 0x55a7434a3640 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.976+0000 7ff331629700 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a7434a3640 0x55a7434a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff331629700 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a7434a3640 0x55a7434a5a60 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1871c7bf4a579ba server_cookie=5e1eb6e1ec77ea78 in_seq=0 out_seq=0 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff3157fa700 10 client.27983 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff3157fa700 1 -- 172.21.15.46:0/388372171 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a7431d0cf0 con 0x55a7434a3640 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff3157fa700 10 client.27983 handle_command_reply: tid=0 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff340223b80 1 client.27983 shutdown 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff340223b80 1 -- 172.21.15.46:0/388372171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 msgr2=0x55a743443860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff340223b80 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 0x55a743443860 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff328099f20 tx=0x7ff3280a21b0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff340223b80 1 -- 172.21.15.46:0/388372171 shutdown_connections 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff340223b80 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55a7434a3640 0x55a7434a5a60 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff340223b80 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a74343e630 0x55a7434442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff340223b80 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a74342ea60 0x55a743443860 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff3157fa700 0 client.27983 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:11.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff340223b80 1 --2- 172.21.15.46:0/388372171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a74342f380 0x55a743443da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff340223b80 1 -- 172.21.15.46:0/388372171 >> 172.21.15.46:0/388372171 conn(0x55a7433a5220 msgr2=0x55a74342aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:11.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff340223b80 1 -- 172.21.15.46:0/388372171 shutdown_connections 2024-03-18T20:44:11.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.977+0000 7ff340223b80 1 -- 172.21.15.46:0/388372171 wait complete. 2024-03-18T20:44:11.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff340223b80 20 client.27983 trim_cache size 0 max 16384 2024-03-18T20:44:11.986 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:44:11.986 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:44:11.986 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:44:11.986 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:44:11.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff33408e700 1 -- 172.21.15.46:0/3810602331 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff3180a1290 msgr2=0x7ff3180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:11.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff33408e700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff3180a1290 0x7ff3180a3750 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff32c198820 tx=0x7ff31c00b260 comp rx=0 tx=0).stop 2024-03-18T20:44:11.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff33408e700 1 -- 172.21.15.46:0/3810602331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff32c0ffe70 msgr2=0x7ff32c197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:11.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff33408e700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff32c0ffe70 0x7ff32c197320 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff32800e9a0 tx=0x7ff32800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff33408e700 1 -- 172.21.15.46:0/3810602331 shutdown_connections 2024-03-18T20:44:11.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff33408e700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff3180a1290 0x7ff3180a3750 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff33408e700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff32c0ffe70 0x7ff32c197320 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff33408e700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff32c19bbb0 0x7ff32c19c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff33408e700 1 --2- 172.21.15.46:0/3810602331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c104f90 0x7ff32c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:11.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff33408e700 1 -- 172.21.15.46:0/3810602331 >> 172.21.15.46:0/3810602331 conn(0x7ff32c076d80 msgr2=0x7ff32c069470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:11.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff33408e700 1 -- 172.21.15.46:0/3810602331 shutdown_connections 2024-03-18T20:44:11.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:11.978+0000 7ff33408e700 1 -- 172.21.15.46:0/3810602331 wait complete. 2024-03-18T20:44:11.994 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:44:11.994 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:44:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.137+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1491390979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2acc101e90 msgr2=0x7f2acc102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.137+0000 7f2ad12a4700 1 --2- 172.21.15.46:0/1491390979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2acc101e90 0x7f2acc102270 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00b0a0 tx=0x7f2abc00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.137+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1491390979 shutdown_connections 2024-03-18T20:44:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.137+0000 7f2ad12a4700 1 --2- 172.21.15.46:0/1491390979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2acc101e90 0x7f2acc102270 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.137+0000 7f2ad12a4700 1 --2- 172.21.15.46:0/1491390979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2acc10b260 0x7f2acc10d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.137+0000 7f2ad12a4700 1 --2- 172.21.15.46:0/1491390979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2acc1027b0 0x7f2acc10ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.137+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1491390979 >> 172.21.15.46:0/1491390979 conn(0x7f2acc0fb5f0 msgr2=0x7f2acc0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.137+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1491390979 shutdown_connections 2024-03-18T20:44:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.137+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1491390979 wait complete. 2024-03-18T20:44:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.138+0000 7f2ad12a4700 1 Processor -- start 2024-03-18T20:44:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.138+0000 7f2ad12a4700 1 -- start start 2024-03-18T20:44:12.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2ad12a4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2acc101e90 0x7f2acc1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2ad12a4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2acc1027b0 0x7f2acc197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2ad12a4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2acc10b260 0x7f2acc19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2ad12a4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc19c100 con 0x7f2acc101e90 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2ad12a4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc19c270 con 0x7f2acc1027b0 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2ad12a4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2acc19c430 con 0x7f2acc10b260 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2acaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2acc101e90 0x7f2acc1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2acaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2acc101e90 0x7f2acc1971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56334/0 (socket says 172.21.15.46:56334) 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2acaffd700 1 -- 172.21.15.46:0/1492179858 learned_addr learned my addr 172.21.15.46:0/1492179858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2aca7fc700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2acc1027b0 0x7f2acc197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2acb7fe700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2acc10b260 0x7f2acc19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2acaffd700 1 -- 172.21.15.46:0/1492179858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2acc10b260 msgr2=0x7f2acc19bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2acaffd700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2acc10b260 0x7f2acc19bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2acaffd700 1 -- 172.21.15.46:0/1492179858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2acc1027b0 msgr2=0x7f2acc197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2acaffd700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2acc1027b0 0x7f2acc197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2acaffd700 1 -- 172.21.15.46:0/1492179858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc009d00 con 0x7f2acc101e90 2024-03-18T20:44:12.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2aca7fc700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2acc1027b0 0x7f2acc197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:12.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.139+0000 7f2acaffd700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2acc101e90 0x7f2acc1971e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00b070 tx=0x7f2abc0079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.140+0000 7f2ac3fff700 1 -- 172.21.15.46:0/1492179858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc00e040 con 0x7f2acc101e90 2024-03-18T20:44:12.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.140+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1492179858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2acc19c6b0 con 0x7f2acc101e90 2024-03-18T20:44:12.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.140+0000 7f2ac3fff700 1 -- 172.21.15.46:0/1492179858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2abc01d470 con 0x7f2acc101e90 2024-03-18T20:44:12.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.140+0000 7f2ac3fff700 1 -- 172.21.15.46:0/1492179858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc01b6c0 con 0x7f2acc101e90 2024-03-18T20:44:12.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.140+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1492179858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2acc1a3f50 con 0x7f2acc101e90 2024-03-18T20:44:12.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.141+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1492179858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2acc1063e0 con 0x7f2acc101e90 2024-03-18T20:44:12.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.142+0000 7f2ac3fff700 1 -- 172.21.15.46:0/1492179858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2abc01d5e0 con 0x7f2acc101e90 2024-03-18T20:44:12.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.143+0000 7f2ac3fff700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ab80a1290 0x7f2ab80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.143+0000 7f2ac3fff700 1 -- 172.21.15.46:0/1492179858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f2abc0cdb80 con 0x7f2acc101e90 2024-03-18T20:44:12.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.143+0000 7f2aca7fc700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ab80a1290 0x7f2ab80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.143+0000 7f2aca7fc700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ab80a1290 0x7f2ab80a3750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2acc198730 tx=0x7f2ab4009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.150+0000 7f2ac3fff700 1 -- 172.21.15.46:0/1492179858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2abc097040 con 0x7f2acc101e90 2024-03-18T20:44:12.213 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:12 smithi046 ceph-mon[43495]: pgmap v250: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 824 KiB/s wr, 95 op/s 2024-03-18T20:44:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.439+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1492179858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2acc067e60 con 0x7f2acc101e90 2024-03-18T20:44:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.440+0000 7f2ac3fff700 1 -- 172.21.15.46:0/1492179858 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10911 (secure 0 0 0) 0x7f2abc09a660 con 0x7f2acc101e90 2024-03-18T20:44:12.443 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:44:12.443 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:44:00.211948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:44:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.443+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1492179858 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ab80a1290 msgr2=0x7f2ab80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.443+0000 7f2ad12a4700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ab80a1290 0x7f2ab80a3750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2acc198730 tx=0x7f2ab4009260 comp rx=0 tx=0).stop 2024-03-18T20:44:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.443+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1492179858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2acc101e90 msgr2=0x7f2acc1971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.443+0000 7f2ad12a4700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2acc101e90 0x7f2acc1971e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00b070 tx=0x7f2abc0079b0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.443+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1492179858 shutdown_connections 2024-03-18T20:44:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.443+0000 7f2ad12a4700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ab80a1290 0x7f2ab80a3750 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.443+0000 7f2ad12a4700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2acc101e90 0x7f2acc1971e0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.443+0000 7f2ad12a4700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2acc10b260 0x7f2acc19bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.443+0000 7f2ad12a4700 1 --2- 172.21.15.46:0/1492179858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2acc1027b0 0x7f2acc197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.443+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1492179858 >> 172.21.15.46:0/1492179858 conn(0x7f2acc0fb5f0 msgr2=0x7f2acc0fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.443+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1492179858 shutdown_connections 2024-03-18T20:44:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.443+0000 7f2ad12a4700 1 -- 172.21.15.46:0/1492179858 wait complete. 2024-03-18T20:44:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 46 2024-03-18T20:44:12.459 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:44:12.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:44:12.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:12 smithi073 ceph-mon[47384]: pgmap v250: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 824 KiB/s wr, 95 op/s 2024-03-18T20:44:12.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:12 smithi155 ceph-mon[48344]: pgmap v250: 97 pgs: 97 active+clean; 106 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 824 KiB/s wr, 95 op/s 2024-03-18T20:44:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.595+0000 7f47ac686700 1 -- 172.21.15.46:0/1602294022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a40ffe70 msgr2=0x7f47a4100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.595+0000 7f47ac686700 1 --2- 172.21.15.46:0/1602294022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a40ffe70 0x7f47a4100250 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009990 tx=0x7f47a0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.595+0000 7f47ac686700 1 -- 172.21.15.46:0/1602294022 shutdown_connections 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.595+0000 7f47ac686700 1 --2- 172.21.15.46:0/1602294022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a40ffe70 0x7f47a4100250 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.595+0000 7f47ac686700 1 --2- 172.21.15.46:0/1602294022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a4104f90 0x7f47a410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.595+0000 7f47ac686700 1 --2- 172.21.15.46:0/1602294022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a4100820 0x7f47a4104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.595+0000 7f47ac686700 1 -- 172.21.15.46:0/1602294022 >> 172.21.15.46:0/1602294022 conn(0x7f47a4076d80 msgr2=0x7f47a4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.595+0000 7f47ac686700 1 -- 172.21.15.46:0/1602294022 shutdown_connections 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.595+0000 7f47ac686700 1 -- 172.21.15.46:0/1602294022 wait complete. 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.596+0000 7f47ac686700 1 Processor -- start 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.596+0000 7f47ac686700 1 -- start start 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.596+0000 7f47ac686700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a40ffe70 0x7f47a4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.596+0000 7f47ac686700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a4104f90 0x7f47a41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.596+0000 7f47ac686700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a4198b10 0x7f47a41a3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.596+0000 7f47ac686700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a41a37b0 con 0x7f47a4104f90 2024-03-18T20:44:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.596+0000 7f47ac686700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a41a3920 con 0x7f47a4198b10 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.596+0000 7f47ac686700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a41a3ae0 con 0x7f47a40ffe70 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47a9c21700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a4104f90 0x7f47a41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47a9c21700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a4104f90 0x7f47a41985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56360/0 (socket says 172.21.15.46:56360) 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47a9c21700 1 -- 172.21.15.46:0/2552139233 learned_addr learned my addr 172.21.15.46:0/2552139233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47aac23700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a4198b10 0x7f47a41a3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47aa422700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a40ffe70 0x7f47a4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47a9c21700 1 -- 172.21.15.46:0/2552139233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a40ffe70 msgr2=0x7f47a4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47a9c21700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a40ffe70 0x7f47a4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47a9c21700 1 -- 172.21.15.46:0/2552139233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a4198b10 msgr2=0x7f47a41a3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47a9c21700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a4198b10 0x7f47a41a3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47a9c21700 1 -- 172.21.15.46:0/2552139233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a00095f0 con 0x7f47a4104f90 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47aa422700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a40ffe70 0x7f47a4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:44:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47a9c21700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a4104f90 0x7f47a41985d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f479800e970 tx=0x7f479800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47977fe700 1 -- 172.21.15.46:0/2552139233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798009790 con 0x7f47a4104f90 2024-03-18T20:44:12.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47ac686700 1 -- 172.21.15.46:0/2552139233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a41a3dc0 con 0x7f47a4104f90 2024-03-18T20:44:12.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47977fe700 1 -- 172.21.15.46:0/2552139233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f479800eec0 con 0x7f47a4104f90 2024-03-18T20:44:12.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47977fe700 1 -- 172.21.15.46:0/2552139233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798005590 con 0x7f47a4104f90 2024-03-18T20:44:12.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.597+0000 7f47ac686700 1 -- 172.21.15.46:0/2552139233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a41a42a0 con 0x7f47a4104f90 2024-03-18T20:44:12.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.600+0000 7f47977fe700 1 -- 172.21.15.46:0/2552139233 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f479800cbd0 con 0x7f47a4104f90 2024-03-18T20:44:12.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.600+0000 7f47977fe700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f47900a1290 0x7f47900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.601+0000 7f47977fe700 1 -- 172.21.15.46:0/2552139233 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f4798014070 con 0x7f47a4104f90 2024-03-18T20:44:12.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.601+0000 7f47aa422700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f47900a1290 0x7f47900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.601+0000 7f47aa422700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f47900a1290 0x7f47900a3750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f47a000b2e0 tx=0x7f47a0005fb0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.601+0000 7f47b881bb80 1 Processor -- start 2024-03-18T20:44:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47b881bb80 1 -- start start 2024-03-18T20:44:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47b881bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fca60 0x56049b0fce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47b881bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fd380 0x56049b107ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47b881bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b108220 0x56049b10a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47b881bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049b056990 con 0x56049b0fca60 2024-03-18T20:44:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47b881bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049b054350 con 0x56049b0fd380 2024-03-18T20:44:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47b881bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049b04fc40 con 0x56049b108220 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47aa422700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fca60 0x56049b0fce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47aa422700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fca60 0x56049b0fce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56366/0 (socket says 172.21.15.46:56366) 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47aa422700 1 -- 172.21.15.46:0/1004681173 learned_addr learned my addr 172.21.15.46:0/1004681173 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47a9c21700 1 --2- 172.21.15.46:0/1004681173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fd380 0x56049b107ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47aac23700 1 --2- 172.21.15.46:0/1004681173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b108220 0x56049b10a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47aa422700 1 -- 172.21.15.46:0/1004681173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b108220 msgr2=0x56049b10a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47aa422700 1 --2- 172.21.15.46:0/1004681173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b108220 0x56049b10a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47aa422700 1 -- 172.21.15.46:0/1004681173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fd380 msgr2=0x56049b107ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47aa422700 1 --2- 172.21.15.46:0/1004681173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fd380 0x56049b107ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47aa422700 1 -- 172.21.15.46:0/1004681173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56049aed13b0 con 0x56049b0fca60 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.602+0000 7f47a9c21700 1 --2- 172.21.15.46:0/1004681173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fd380 0x56049b107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.603+0000 7f47aa422700 1 --2- 172.21.15.46:0/1004681173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fca60 0x56049b0fce40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f47a000b690 tx=0x7f47a001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a7412744b26eeb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.603+0000 7f4794ff9700 1 -- 172.21.15.46:0/1004681173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a002b070 con 0x56049b0fca60 2024-03-18T20:44:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.603+0000 7f4794ff9700 1 -- 172.21.15.46:0/1004681173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a001e040 con 0x56049b0fca60 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.603+0000 7f4794ff9700 1 -- 172.21.15.46:0/1004681173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0024610 con 0x56049b0fca60 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 -- 172.21.15.46:0/1004681173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fca60 msgr2=0x56049b0fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 --2- 172.21.15.46:0/1004681173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fca60 0x56049b0fce40 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f47a000b690 tx=0x7f47a001f000 comp rx=0 tx=0).stop 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 -- 172.21.15.46:0/1004681173 shutdown_connections 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 --2- 172.21.15.46:0/1004681173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fca60 0x56049b0fce40 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 --2- 172.21.15.46:0/1004681173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b108220 0x56049b10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 --2- 172.21.15.46:0/1004681173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fd380 0x56049b107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 -- 172.21.15.46:0/1004681173 >> 172.21.15.46:0/1004681173 conn(0x56049b073220 msgr2=0x56049b0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 -- 172.21.15.46:0/1004681173 shutdown_connections 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 -- 172.21.15.46:0/1004681173 wait complete. 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 -- start start 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 Processor -- start 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b0fca60 0x56049b107bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fd380 0x56049b104ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 0x56049b105500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049afebd80 con 0x56049b108220 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.604+0000 7f47b881bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049afeba20 con 0x56049b0fd380 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47b881bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049aed36a0 con 0x56049b0fca60 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47b881bb80 10 client.0 _ll_register_callbacks cb 0x56049ae573a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47aac23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 0x56049b105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47aac23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 0x56049b105500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56376/0 (socket says 172.21.15.46:56376) 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47aac23700 1 -- 172.21.15.46:0/1485103663 learned_addr learned my addr 172.21.15.46:0/1485103663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47a9c21700 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fd380 0x56049b104ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47aa422700 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b0fca60 0x56049b107bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47aac23700 1 -- 172.21.15.46:0/1485103663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b0fca60 msgr2=0x56049b107bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47aac23700 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b0fca60 0x56049b107bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47aac23700 1 -- 172.21.15.46:0/1485103663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fd380 msgr2=0x56049b104ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47aac23700 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fd380 0x56049b104ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47aac23700 1 -- 172.21.15.46:0/1485103663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a41a42a0 con 0x56049b108220 2024-03-18T20:44:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47a9c21700 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fd380 0x56049b104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47aac23700 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 0x56049b105500 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f479c011e80 tx=0x7f479c014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d8e3e505515956 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f478dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f478dffb700 1 -- 172.21.15.46:0/1485103663 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c00e040 con 0x56049b108220 2024-03-18T20:44:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f478dffb700 1 -- 172.21.15.46:0/1485103663 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f479c01a070 con 0x56049b108220 2024-03-18T20:44:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.605+0000 7f47b881bb80 1 -- 172.21.15.46:0/1485103663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56049b0f0ad0 con 0x56049b108220 2024-03-18T20:44:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.606+0000 7f478dffb700 1 -- 172.21.15.46:0/1485103663 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f479c027480 con 0x56049b108220 2024-03-18T20:44:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.606+0000 7f47b881bb80 10 client.18345 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.606+0000 7f47b881bb80 1 -- 172.21.15.46:0/1485103663 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f47a410d380 con 0x56049b108220 2024-03-18T20:44:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.606+0000 7f478dffb700 1 -- 172.21.15.46:0/1485103663 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56049b0f0ad0 con 0x56049b108220 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.606+0000 7f478dffb700 1 -- 172.21.15.46:0/1485103663 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f479c016490 con 0x56049b108220 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.606+0000 7f47b881bb80 10 client.18345 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.606+0000 7f47b881bb80 10 client.18345 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.606+0000 7f47b881bb80 20 client.18345 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.606+0000 7f47b881bb80 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56049b117d80 0x56049b11a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.606+0000 7f47b881bb80 4 client.18345 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.606+0000 7f47b881bb80 1 -- 172.21.15.46:0/1485103663 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56049b048700 con 0x56049b117d80 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.606+0000 7f47aa422700 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56049b117d80 0x56049b11a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.607+0000 7f47aa422700 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56049b117d80 0x56049b11a1a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c2f0c02e8bf2b164 server_cookie=726dd0b9c58f4163 in_seq=0 out_seq=0 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.607+0000 7f478dffb700 10 client.18345 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.607+0000 7f478dffb700 1 -- 172.21.15.46:0/1485103663 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56049b048700 con 0x56049b117d80 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.607+0000 7f478dffb700 10 client.18345 handle_command_reply: tid=0 2024-03-18T20:44:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.607+0000 7f47b881bb80 1 client.18345 shutdown 2024-03-18T20:44:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.608+0000 7f47b881bb80 1 -- 172.21.15.46:0/1485103663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 msgr2=0x56049b105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.608+0000 7f47b881bb80 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 0x56049b105500 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f479c011e80 tx=0x7f479c014600 comp rx=0 tx=0).stop 2024-03-18T20:44:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.608+0000 7f47b881bb80 1 -- 172.21.15.46:0/1485103663 shutdown_connections 2024-03-18T20:44:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.608+0000 7f47b881bb80 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56049b117d80 0x56049b11a1a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.608+0000 7f47b881bb80 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 0x56049b105500 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.608+0000 7f47b881bb80 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b0fca60 0x56049b107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.608+0000 7f478dffb700 0 client.18345 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:44:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.608+0000 7f47b881bb80 1 --2- 172.21.15.46:0/1485103663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fd380 0x56049b104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.608+0000 7f47b881bb80 1 -- 172.21.15.46:0/1485103663 >> 172.21.15.46:0/1485103663 conn(0x56049b073220 msgr2=0x56049b0f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.608+0000 7f47b881bb80 1 -- 172.21.15.46:0/1485103663 shutdown_connections 2024-03-18T20:44:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.608+0000 7f47b881bb80 1 -- 172.21.15.46:0/1485103663 wait complete. 2024-03-18T20:44:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.608+0000 7f47b881bb80 20 client.18345 trim_cache size 0 max 16384 2024-03-18T20:44:12.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.621+0000 7f47b881bb80 1 Processor -- start 2024-03-18T20:44:12.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.621+0000 7f47b881bb80 1 -- start start 2024-03-18T20:44:12.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47b881bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fca60 0x56049b105480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47b881bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b0fd380 0x56049b10ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47b881bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 0x56049b100220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47b881bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049b114800 con 0x56049b108220 2024-03-18T20:44:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47b881bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049b0fe5d0 con 0x56049b0fca60 2024-03-18T20:44:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47b881bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049b1153e0 con 0x56049b0fd380 2024-03-18T20:44:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47aac23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 0x56049b100220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47aac23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 0x56049b100220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56386/0 (socket says 172.21.15.46:56386) 2024-03-18T20:44:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47aac23700 1 -- 172.21.15.46:0/564634456 learned_addr learned my addr 172.21.15.46:0/564634456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47aa422700 1 --2- 172.21.15.46:0/564634456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fca60 0x56049b105480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47a9c21700 1 --2- 172.21.15.46:0/564634456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b0fd380 0x56049b10ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47aa422700 1 -- 172.21.15.46:0/564634456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b0fd380 msgr2=0x56049b10ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47aa422700 1 --2- 172.21.15.46:0/564634456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b0fd380 0x56049b10ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47aa422700 1 -- 172.21.15.46:0/564634456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 msgr2=0x56049b100220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47aa422700 1 --2- 172.21.15.46:0/564634456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 0x56049b100220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47aa422700 1 -- 172.21.15.46:0/564634456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56049ae7db60 con 0x56049b0fca60 2024-03-18T20:44:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47aa422700 1 --2- 172.21.15.46:0/564634456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fca60 0x56049b105480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f47a0005f50 tx=0x7f47a0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=210895ab7112699c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47957fa700 1 -- 172.21.15.46:0/564634456 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a002b040 con 0x56049b0fca60 2024-03-18T20:44:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47957fa700 1 -- 172.21.15.46:0/564634456 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a002c040 con 0x56049b0fca60 2024-03-18T20:44:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.622+0000 7f47957fa700 1 -- 172.21.15.46:0/564634456 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a000f960 con 0x56049b0fca60 2024-03-18T20:44:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.623+0000 7f47b881bb80 1 -- 172.21.15.46:0/564634456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fca60 msgr2=0x56049b105480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.623+0000 7f47b881bb80 1 --2- 172.21.15.46:0/564634456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fca60 0x56049b105480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f47a0005f50 tx=0x7f47a0000bc0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.623+0000 7f47b881bb80 1 -- 172.21.15.46:0/564634456 shutdown_connections 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.623+0000 7f47b881bb80 1 --2- 172.21.15.46:0/564634456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b108220 0x56049b100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.623+0000 7f47b881bb80 1 --2- 172.21.15.46:0/564634456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fca60 0x56049b105480 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.623+0000 7f47b881bb80 1 --2- 172.21.15.46:0/564634456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b0fd380 0x56049b10ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.623+0000 7f47b881bb80 1 -- 172.21.15.46:0/564634456 >> 172.21.15.46:0/564634456 conn(0x56049b073220 msgr2=0x56049b0f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.623+0000 7f47b881bb80 1 -- 172.21.15.46:0/564634456 shutdown_connections 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.623+0000 7f47b881bb80 1 -- 172.21.15.46:0/564634456 wait complete. 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.623+0000 7f47b881bb80 1 -- start start 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.623+0000 7f47b881bb80 1 Processor -- start 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47b881bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fca60 0x56049b10df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47b881bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fd380 0x56049b10e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47b881bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b108220 0x56049b16abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47b881bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049b10ed40 con 0x56049b0fd380 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47b881bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049b16b0f0 con 0x56049b0fca60 2024-03-18T20:44:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47b881bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049b16b230 con 0x56049b108220 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47b881bb80 10 client.0 _ll_register_callbacks cb 0x7f479c0117b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47a9c21700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fd380 0x56049b10e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47a9c21700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fd380 0x56049b10e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56400/0 (socket says 172.21.15.46:56400) 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47a9c21700 1 -- 172.21.15.46:0/484103224 learned_addr learned my addr 172.21.15.46:0/484103224 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47aa422700 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fca60 0x56049b10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47aac23700 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b108220 0x56049b16abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47a9c21700 1 -- 172.21.15.46:0/484103224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b108220 msgr2=0x56049b16abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47a9c21700 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b108220 0x56049b16abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47a9c21700 1 -- 172.21.15.46:0/484103224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fca60 msgr2=0x56049b10df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47a9c21700 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fca60 0x56049b10df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47a9c21700 1 -- 172.21.15.46:0/484103224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56049aed13b0 con 0x56049b0fd380 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47aa422700 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fca60 0x56049b10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f47a9c21700 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fd380 0x56049b10e480 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4798096570 tx=0x7f479809a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0c0024bf8ce21 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.624+0000 7f478dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f478dffb700 1 -- 172.21.15.46:0/484103224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47980a24f0 con 0x56049b0fd380 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f478dffb700 1 -- 172.21.15.46:0/484103224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f479809a4e0 con 0x56049b0fd380 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f47b881bb80 1 -- 172.21.15.46:0/484103224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47a001e040 con 0x56049b0fd380 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f478dffb700 1 -- 172.21.15.46:0/484103224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47980b82a0 con 0x56049b0fd380 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f47b881bb80 10 client.18351 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f47b881bb80 1 -- 172.21.15.46:0/484103224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56049b015ea0 con 0x56049b0fd380 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f478dffb700 1 -- 172.21.15.46:0/484103224 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a001e040 con 0x56049b0fd380 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f478dffb700 1 -- 172.21.15.46:0/484103224 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47980a2180 con 0x56049b0fd380 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f47b881bb80 10 client.18351 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f47b881bb80 10 client.18351 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f47b881bb80 20 client.18351 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f47b881bb80 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56049b16c960 0x56049b16ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f47b881bb80 4 client.18351 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f47b881bb80 1 -- 172.21.15.46:0/484103224 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56049ae9ecf0 con 0x56049b16c960 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.625+0000 7f47aa422700 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56049b16c960 0x56049b16ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f478dffb700 10 client.18351 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47aa422700 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56049b16c960 0x56049b16ed80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=776d0bf752a6deca server_cookie=c48aa5d642f3cbf3 in_seq=0 out_seq=0 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f478dffb700 1 -- 172.21.15.46:0/484103224 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56049ae9ecf0 con 0x56049b16c960 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f478dffb700 10 client.18351 handle_command_reply: tid=0 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47b881bb80 1 client.18351 shutdown 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47b881bb80 1 -- 172.21.15.46:0/484103224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fd380 msgr2=0x56049b10e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47b881bb80 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fd380 0x56049b10e480 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4798096570 tx=0x7f479809a070 comp rx=0 tx=0).stop 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47b881bb80 1 -- 172.21.15.46:0/484103224 shutdown_connections 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47b881bb80 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x56049b16c960 0x56049b16ed80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47b881bb80 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56049b108220 0x56049b16abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47b881bb80 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56049b0fca60 0x56049b10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f478dffb700 0 client.18351 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:44:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47b881bb80 1 --2- 172.21.15.46:0/484103224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56049b0fd380 0x56049b10e480 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47b881bb80 1 -- 172.21.15.46:0/484103224 >> 172.21.15.46:0/484103224 conn(0x56049b073220 msgr2=0x56049b103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47b881bb80 1 -- 172.21.15.46:0/484103224 shutdown_connections 2024-03-18T20:44:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47b881bb80 1 -- 172.21.15.46:0/484103224 wait complete. 2024-03-18T20:44:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.626+0000 7f47b881bb80 20 client.18351 trim_cache size 0 max 16384 2024-03-18T20:44:12.636 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:44:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.627+0000 7f47ac686700 1 -- 172.21.15.46:0/2552139233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f47900a1290 msgr2=0x7f47900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.627+0000 7f47ac686700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f47900a1290 0x7f47900a3750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f47a000b2e0 tx=0x7f47a0005fb0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.627+0000 7f47ac686700 1 -- 172.21.15.46:0/2552139233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a4104f90 msgr2=0x7f47a41985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.627+0000 7f47ac686700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a4104f90 0x7f47a41985d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f479800e970 tx=0x7f479800ec80 comp rx=0 tx=0).stop 2024-03-18T20:44:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.627+0000 7f47ac686700 1 -- 172.21.15.46:0/2552139233 shutdown_connections 2024-03-18T20:44:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.627+0000 7f47ac686700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f47900a1290 0x7f47900a3750 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.627+0000 7f47ac686700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a4198b10 0x7f47a41a3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.627+0000 7f47ac686700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47a40ffe70 0x7f47a4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.627+0000 7f47ac686700 1 --2- 172.21.15.46:0/2552139233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47a4104f90 0x7f47a41985d0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.627+0000 7f47ac686700 1 -- 172.21.15.46:0/2552139233 >> 172.21.15.46:0/2552139233 conn(0x7f47a4076d80 msgr2=0x7f47a4106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.627+0000 7f47ac686700 1 -- 172.21.15.46:0/2552139233 shutdown_connections 2024-03-18T20:44:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.627+0000 7f47ac686700 1 -- 172.21.15.46:0/2552139233 wait complete. 2024-03-18T20:44:12.643 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:44:12.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:44:12.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 285476 3.57 MB/sec execute 307 sec latency 308.833 ms 2024-03-18T20:44:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.779+0000 7f61e08d8700 1 -- 172.21.15.46:0/443965260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61dc0ff490 msgr2=0x7f61dc0ff870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.779+0000 7f61e08d8700 1 --2- 172.21.15.46:0/443965260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61dc0ff490 0x7f61dc0ff870 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009990 tx=0x7f61c4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.779+0000 7f61e08d8700 1 -- 172.21.15.46:0/443965260 shutdown_connections 2024-03-18T20:44:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.779+0000 7f61e08d8700 1 --2- 172.21.15.46:0/443965260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61dc0ff490 0x7f61dc0ff870 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.779+0000 7f61e08d8700 1 --2- 172.21.15.46:0/443965260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61dc106050 0x7f61dc10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.779+0000 7f61e08d8700 1 --2- 172.21.15.46:0/443965260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc0ffe40 0x7f61dc105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.779+0000 7f61e08d8700 1 -- 172.21.15.46:0/443965260 >> 172.21.15.46:0/443965260 conn(0x7f61dc0761e0 msgr2=0x7f61dc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.779+0000 7f61e08d8700 1 -- 172.21.15.46:0/443965260 shutdown_connections 2024-03-18T20:44:12.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.779+0000 7f61e08d8700 1 -- 172.21.15.46:0/443965260 wait complete. 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61e08d8700 1 Processor -- start 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61e08d8700 1 -- start start 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61e08d8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61dc0ff490 0x7f61dc1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61e08d8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc106050 0x7f61dc197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61e08d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61dc19bab0 0x7f61dc19bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61e08d8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc19c530 con 0x7f61dc19bab0 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61e08d8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc19c6a0 con 0x7f61dc106050 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61e08d8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc19c860 con 0x7f61dc0ff490 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61dad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61dc19bab0 0x7f61dc19bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61da59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61dc0ff490 0x7f61dc1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61dad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61dc19bab0 0x7f61dc19bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56414/0 (socket says 172.21.15.46:56414) 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61dad9d700 1 -- 172.21.15.46:0/2273939805 learned_addr learned my addr 172.21.15.46:0/2273939805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.780+0000 7f61d9d9b700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc106050 0x7f61dc197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.781+0000 7f61dad9d700 1 -- 172.21.15.46:0/2273939805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61dc0ff490 msgr2=0x7f61dc1971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.781+0000 7f61dad9d700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61dc0ff490 0x7f61dc1971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.781+0000 7f61dad9d700 1 -- 172.21.15.46:0/2273939805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc106050 msgr2=0x7f61dc197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.781+0000 7f61dad9d700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc106050 0x7f61dc197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.781+0000 7f61dad9d700 1 -- 172.21.15.46:0/2273939805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c40095f0 con 0x7f61dc19bab0 2024-03-18T20:44:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.781+0000 7f61da59c700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61dc0ff490 0x7f61dc1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.781+0000 7f61dad9d700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61dc19bab0 0x7f61dc19bf30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f61d4007ee0 tx=0x7f61d400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.781+0000 7f61d37fe700 1 -- 172.21.15.46:0/2273939805 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4017070 con 0x7f61dc19bab0 2024-03-18T20:44:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.781+0000 7f61e08d8700 1 -- 172.21.15.46:0/2273939805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61dc1a3dc0 con 0x7f61dc19bab0 2024-03-18T20:44:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.781+0000 7f61d37fe700 1 -- 172.21.15.46:0/2273939805 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61d400f040 con 0x7f61dc19bab0 2024-03-18T20:44:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.781+0000 7f61d37fe700 1 -- 172.21.15.46:0/2273939805 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4005580 con 0x7f61dc19bab0 2024-03-18T20:44:12.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.781+0000 7f61e08d8700 1 -- 172.21.15.46:0/2273939805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61dc1a42a0 con 0x7f61dc19bab0 2024-03-18T20:44:12.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.784+0000 7f61d37fe700 1 -- 172.21.15.46:0/2273939805 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f61d400db20 con 0x7f61dc19bab0 2024-03-18T20:44:12.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.784+0000 7f61d37fe700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f61c80a1290 0x7f61c80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.784+0000 7f61d37fe700 1 -- 172.21.15.46:0/2273939805 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f61d40cc3d0 con 0x7f61dc19bab0 2024-03-18T20:44:12.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.785+0000 7f61da59c700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f61c80a1290 0x7f61c80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.785+0000 7f61da59c700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f61c80a1290 0x7f61c80a3750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61c400b5c0 tx=0x7f61c401a040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.785+0000 7f61eca6db80 1 Processor -- start 2024-03-18T20:44:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.785+0000 7f61eca6db80 1 -- start start 2024-03-18T20:44:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61eca6db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b44a60 0x557812b44e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61eca6db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 0x557812b4fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61eca6db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b50220 0x557812b52610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61eca6db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557812a9e990 con 0x557812b45380 2024-03-18T20:44:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61eca6db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557812a9c350 con 0x557812b50220 2024-03-18T20:44:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61eca6db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557812a97c40 con 0x557812b44a60 2024-03-18T20:44:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61d9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 0x557812b4fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61d9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 0x557812b4fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56418/0 (socket says 172.21.15.46:56418) 2024-03-18T20:44:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61da59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b44a60 0x557812b44e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61d9d9b700 1 -- 172.21.15.46:0/1561721097 learned_addr learned my addr 172.21.15.46:0/1561721097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61dad9d700 1 --2- 172.21.15.46:0/1561721097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b50220 0x557812b52610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61d9d9b700 1 -- 172.21.15.46:0/1561721097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b44a60 msgr2=0x557812b44e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61d9d9b700 1 --2- 172.21.15.46:0/1561721097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b44a60 0x557812b44e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61d9d9b700 1 -- 172.21.15.46:0/1561721097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b50220 msgr2=0x557812b52610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61d9d9b700 1 --2- 172.21.15.46:0/1561721097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b50220 0x557812b52610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61d9d9b700 1 -- 172.21.15.46:0/1561721097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578129193b0 con 0x557812b45380 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61dad9d700 1 --2- 172.21.15.46:0/1561721097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b50220 0x557812b52610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61da59c700 1 --2- 172.21.15.46:0/1561721097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b44a60 0x557812b44e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.786+0000 7f61d9d9b700 1 --2- 172.21.15.46:0/1561721097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 0x557812b4fce0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00aca0 tx=0x7f61cc00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa83756a8f39c46 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.787+0000 7f61d0ff9700 1 -- 172.21.15.46:0/1561721097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00fe50 con 0x557812b45380 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.787+0000 7f61d0ff9700 1 -- 172.21.15.46:0/1561721097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61cc0044f0 con 0x557812b45380 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.787+0000 7f61d0ff9700 1 -- 172.21.15.46:0/1561721097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc018430 con 0x557812b45380 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.787+0000 7f61eca6db80 1 -- 172.21.15.46:0/1561721097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 msgr2=0x557812b4fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.787+0000 7f61eca6db80 1 --2- 172.21.15.46:0/1561721097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 0x557812b4fce0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00aca0 tx=0x7f61cc00f3c0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.787+0000 7f61eca6db80 1 -- 172.21.15.46:0/1561721097 shutdown_connections 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.787+0000 7f61eca6db80 1 --2- 172.21.15.46:0/1561721097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b50220 0x557812b52610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.787+0000 7f61eca6db80 1 --2- 172.21.15.46:0/1561721097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b44a60 0x557812b44e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.787+0000 7f61eca6db80 1 --2- 172.21.15.46:0/1561721097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 0x557812b4fce0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.787+0000 7f61eca6db80 1 -- 172.21.15.46:0/1561721097 >> 172.21.15.46:0/1561721097 conn(0x557812abb220 msgr2=0x557812b41020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61eca6db80 1 -- 172.21.15.46:0/1561721097 shutdown_connections 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61eca6db80 1 -- 172.21.15.46:0/1561721097 wait complete. 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61eca6db80 1 -- start start 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61eca6db80 1 Processor -- start 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61eca6db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b44a60 0x557812b4fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61eca6db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 0x557812b4cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61eca6db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b50220 0x557812b4d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61eca6db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557812a33d80 con 0x557812b45380 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61eca6db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557812a33a20 con 0x557812b50220 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61eca6db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781291b6a0 con 0x557812b44a60 2024-03-18T20:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61eca6db80 10 client.0 _ll_register_callbacks cb 0x55781289f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61d9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 0x557812b4cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61da59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b44a60 0x557812b4fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61d9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 0x557812b4cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56426/0 (socket says 172.21.15.46:56426) 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.788+0000 7f61d9d9b700 1 -- 172.21.15.46:0/881863687 learned_addr learned my addr 172.21.15.46:0/881863687 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61dad9d700 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b50220 0x557812b4d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61da59c700 1 -- 172.21.15.46:0/881863687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b50220 msgr2=0x557812b4d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61da59c700 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b50220 0x557812b4d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61da59c700 1 -- 172.21.15.46:0/881863687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 msgr2=0x557812b4cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61da59c700 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 0x557812b4cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61da59c700 1 -- 172.21.15.46:0/881863687 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61dc10e460 con 0x557812b44a60 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61d9d9b700 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 0x557812b4cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61dad9d700 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b50220 0x557812b4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61da59c700 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b44a60 0x557812b4fcb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f61c400c010 tx=0x7f61c4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d99e985d0fd72c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61c1ffb700 1 -- 172.21.15.46:0/881863687 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c402c040 con 0x557812b44a60 2024-03-18T20:44:12.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61eca6db80 1 -- 172.21.15.46:0/881863687 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557812b38ad0 con 0x557812b44a60 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61c1ffb700 1 -- 172.21.15.46:0/881863687 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f61c400f660 con 0x557812b44a60 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61c1ffb700 1 -- 172.21.15.46:0/881863687 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61c400f860 con 0x557812b44a60 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61eca6db80 10 client.28036 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.789+0000 7f61eca6db80 1 -- 172.21.15.46:0/881863687 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f61cc00a8d0 con 0x557812b44a60 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.790+0000 7f61c1ffb700 1 -- 172.21.15.46:0/881863687 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61c403d020 con 0x557812b44a60 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.790+0000 7f61c1ffb700 1 -- 172.21.15.46:0/881863687 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f61c4026ab0 con 0x557812b44a60 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.790+0000 7f61eca6db80 10 client.28036 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.790+0000 7f61eca6db80 10 client.28036 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.790+0000 7f61eca6db80 20 client.28036 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.790+0000 7f61eca6db80 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557812b5fe50 0x557812b62270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.790+0000 7f61eca6db80 4 client.28036 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.790+0000 7f61eca6db80 1 -- 172.21.15.46:0/881863687 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557812a90700 con 0x557812b5fe50 2024-03-18T20:44:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.790+0000 7f61d9d9b700 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557812b5fe50 0x557812b62270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.791+0000 7f61d9d9b700 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557812b5fe50 0x557812b62270 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d5061b46b3bca286 server_cookie=29f622fe150ec5c3 in_seq=0 out_seq=0 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.791+0000 7f61c1ffb700 10 client.28036 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.791+0000 7f61c1ffb700 1 -- 172.21.15.46:0/881863687 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557812a90700 con 0x557812b5fe50 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.791+0000 7f61c1ffb700 10 client.28036 handle_command_reply: tid=0 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.791+0000 7f61eca6db80 1 client.28036 shutdown 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.792+0000 7f61eca6db80 1 -- 172.21.15.46:0/881863687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b44a60 msgr2=0x557812b4fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.792+0000 7f61eca6db80 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b44a60 0x557812b4fcb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f61c400c010 tx=0x7f61c4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.792+0000 7f61eca6db80 1 -- 172.21.15.46:0/881863687 shutdown_connections 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.792+0000 7f61eca6db80 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b50220 0x557812b4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.792+0000 7f61eca6db80 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b44a60 0x557812b4fcb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.792+0000 7f61eca6db80 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557812b5fe50 0x557812b62270 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.792+0000 7f61eca6db80 1 --2- 172.21.15.46:0/881863687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b45380 0x557812b4cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.792+0000 7f61c1ffb700 0 client.28036 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:44:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.792+0000 7f61eca6db80 1 -- 172.21.15.46:0/881863687 >> 172.21.15.46:0/881863687 conn(0x557812abb220 msgr2=0x557812b40c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.792+0000 7f61eca6db80 1 -- 172.21.15.46:0/881863687 shutdown_connections 2024-03-18T20:44:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.792+0000 7f61eca6db80 1 -- 172.21.15.46:0/881863687 wait complete. 2024-03-18T20:44:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.792+0000 7f61eca6db80 20 client.28036 trim_cache size 0 max 16384 2024-03-18T20:44:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.805+0000 7f61eca6db80 1 Processor -- start 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61eca6db80 1 -- start start 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61eca6db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b44a60 0x557812b4d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61eca6db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b45380 0x557812b56e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61eca6db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 0x557812b40b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61eca6db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557812b3b210 con 0x557812b50220 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61eca6db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557812b4e5a0 con 0x557812b44a60 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61eca6db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557812b45d60 con 0x557812b45380 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61dad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 0x557812b40b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61dad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 0x557812b40b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56438/0 (socket says 172.21.15.46:56438) 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61dad9d700 1 -- 172.21.15.46:0/29324110 learned_addr learned my addr 172.21.15.46:0/29324110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61da59c700 1 --2- 172.21.15.46:0/29324110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b44a60 0x557812b4d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61d9d9b700 1 --2- 172.21.15.46:0/29324110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b45380 0x557812b56e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61dad9d700 1 -- 172.21.15.46:0/29324110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b45380 msgr2=0x557812b56e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61dad9d700 1 --2- 172.21.15.46:0/29324110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b45380 0x557812b56e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61dad9d700 1 -- 172.21.15.46:0/29324110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b44a60 msgr2=0x557812b4d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61dad9d700 1 --2- 172.21.15.46:0/29324110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b44a60 0x557812b4d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61dad9d700 1 -- 172.21.15.46:0/29324110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578128c5b60 con 0x557812b50220 2024-03-18T20:44:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.806+0000 7f61dad9d700 1 --2- 172.21.15.46:0/29324110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 0x557812b40b50 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f61d4095240 tx=0x7f61d4084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adec2d1f215ea78d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.807+0000 7f61d17fa700 1 -- 172.21.15.46:0/29324110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d40994c0 con 0x557812b50220 2024-03-18T20:44:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.807+0000 7f61d17fa700 1 -- 172.21.15.46:0/29324110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61d409d2f0 con 0x557812b50220 2024-03-18T20:44:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.807+0000 7f61d17fa700 1 -- 172.21.15.46:0/29324110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4022250 con 0x557812b50220 2024-03-18T20:44:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.807+0000 7f61eca6db80 1 -- 172.21.15.46:0/29324110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 msgr2=0x557812b40b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.807+0000 7f61eca6db80 1 --2- 172.21.15.46:0/29324110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 0x557812b40b50 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f61d4095240 tx=0x7f61d4084040 comp rx=0 tx=0).stop 2024-03-18T20:44:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.807+0000 7f61eca6db80 1 -- 172.21.15.46:0/29324110 shutdown_connections 2024-03-18T20:44:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.807+0000 7f61eca6db80 1 --2- 172.21.15.46:0/29324110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 0x557812b40b50 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.807+0000 7f61eca6db80 1 --2- 172.21.15.46:0/29324110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b44a60 0x557812b4d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.807+0000 7f61eca6db80 1 --2- 172.21.15.46:0/29324110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b45380 0x557812b56e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.807+0000 7f61eca6db80 1 -- 172.21.15.46:0/29324110 >> 172.21.15.46:0/29324110 conn(0x557812abb220 msgr2=0x557812b3f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.807+0000 7f61eca6db80 1 -- 172.21.15.46:0/29324110 shutdown_connections 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.807+0000 7f61eca6db80 1 -- 172.21.15.46:0/29324110 wait complete. 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61eca6db80 1 -- start start 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61eca6db80 1 Processor -- start 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61eca6db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b44a60 0x557812b547a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61eca6db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b45380 0x557812b54ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61eca6db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 0x557812b55260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61eca6db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557812bb72b0 con 0x557812b50220 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61eca6db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557812bb7420 con 0x557812b44a60 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61eca6db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557812bb75e0 con 0x557812b45380 2024-03-18T20:44:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61eca6db80 10 client.0 _ll_register_callbacks cb 0x55781289f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61dad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 0x557812b55260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61dad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 0x557812b55260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56450/0 (socket says 172.21.15.46:56450) 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61dad9d700 1 -- 172.21.15.46:0/2305156089 learned_addr learned my addr 172.21.15.46:0/2305156089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61da59c700 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b44a60 0x557812b547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61d9d9b700 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b45380 0x557812b54ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61dad9d700 1 -- 172.21.15.46:0/2305156089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b45380 msgr2=0x557812b54ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61dad9d700 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b45380 0x557812b54ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61dad9d700 1 -- 172.21.15.46:0/2305156089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b44a60 msgr2=0x557812b547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61dad9d700 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b44a60 0x557812b547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61dad9d700 1 -- 172.21.15.46:0/2305156089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61dc1a42a0 con 0x557812b50220 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.808+0000 7f61da59c700 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b44a60 0x557812b547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61dad9d700 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 0x557812b55260 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f61d4099450 tx=0x7f61d4007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ba8de9e08f2c49 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61c1ffb700 1 -- 172.21.15.46:0/2305156089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d40994c0 con 0x557812b50220 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61c1ffb700 1 -- 172.21.15.46:0/2305156089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f61d40b5070 con 0x557812b50220 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61eca6db80 1 -- 172.21.15.46:0/2305156089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61cc0044f0 con 0x557812b50220 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61c1ffb700 1 -- 172.21.15.46:0/2305156089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61d4097860 con 0x557812b50220 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61eca6db80 10 client.18369 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61eca6db80 1 -- 172.21.15.46:0/2305156089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x557812a5dea0 con 0x557812b50220 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61c1ffb700 1 -- 172.21.15.46:0/2305156089 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61d407e080 con 0x557812b50220 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61c1ffb700 1 -- 172.21.15.46:0/2305156089 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f61d4098ab0 con 0x557812b50220 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61eca6db80 10 client.18369 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61eca6db80 10 client.18369 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61eca6db80 20 client.18369 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61eca6db80 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557812bb8990 0x557812bbadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61eca6db80 4 client.18369 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:44:12.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.809+0000 7f61eca6db80 1 -- 172.21.15.46:0/2305156089 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5578128e6cf0 con 0x557812bb8990 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61da59c700 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557812bb8990 0x557812bbadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61c1ffb700 10 client.18369 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61da59c700 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557812bb8990 0x557812bbadb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=26bd3af787e05d14 server_cookie=72c1d7725d4b268 in_seq=0 out_seq=0 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61c1ffb700 1 -- 172.21.15.46:0/2305156089 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5578128e6cf0 con 0x557812bb8990 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61c1ffb700 10 client.18369 handle_command_reply: tid=0 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61eca6db80 1 client.18369 shutdown 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61eca6db80 1 -- 172.21.15.46:0/2305156089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 msgr2=0x557812b55260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61eca6db80 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 0x557812b55260 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f61d4099450 tx=0x7f61d4007420 comp rx=0 tx=0).stop 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61eca6db80 1 -- 172.21.15.46:0/2305156089 shutdown_connections 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61eca6db80 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x557812bb8990 0x557812bbadb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61eca6db80 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557812b50220 0x557812b55260 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61c1ffb700 0 client.18369 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61eca6db80 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557812b44a60 0x557812b547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61eca6db80 1 --2- 172.21.15.46:0/2305156089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557812b45380 0x557812b54ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61eca6db80 1 -- 172.21.15.46:0/2305156089 >> 172.21.15.46:0/2305156089 conn(0x557812abb220 msgr2=0x557812b40aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61eca6db80 1 -- 172.21.15.46:0/2305156089 shutdown_connections 2024-03-18T20:44:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.810+0000 7f61eca6db80 1 -- 172.21.15.46:0/2305156089 wait complete. 2024-03-18T20:44:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61eca6db80 20 client.18369 trim_cache size 0 max 16384 2024-03-18T20:44:12.820 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:44:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61e08d8700 1 -- 172.21.15.46:0/2273939805 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f61c80a1290 msgr2=0x7f61c80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61e08d8700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f61c80a1290 0x7f61c80a3750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61c400b5c0 tx=0x7f61c401a040 comp rx=0 tx=0).stop 2024-03-18T20:44:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61e08d8700 1 -- 172.21.15.46:0/2273939805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61dc19bab0 msgr2=0x7f61dc19bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61e08d8700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61dc19bab0 0x7f61dc19bf30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f61d4007ee0 tx=0x7f61d400d3b0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61e08d8700 1 -- 172.21.15.46:0/2273939805 shutdown_connections 2024-03-18T20:44:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61e08d8700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f61c80a1290 0x7f61c80a3750 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61e08d8700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61dc19bab0 0x7f61dc19bf30 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61e08d8700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61dc0ff490 0x7f61dc1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61e08d8700 1 --2- 172.21.15.46:0/2273939805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61dc106050 0x7f61dc197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61e08d8700 1 -- 172.21.15.46:0/2273939805 >> 172.21.15.46:0/2273939805 conn(0x7f61dc0761e0 msgr2=0x7f61dc1043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61e08d8700 1 -- 172.21.15.46:0/2273939805 shutdown_connections 2024-03-18T20:44:12.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.811+0000 7f61e08d8700 1 -- 172.21.15.46:0/2273939805 wait complete. 2024-03-18T20:44:12.827 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:44:12.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:44:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.967+0000 7fa104d3c700 1 -- 172.21.15.46:0/729876339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa100104f60 msgr2=0x7fa10010cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.967+0000 7fa104d3c700 1 --2- 172.21.15.46:0/729876339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa100104f60 0x7fa10010cae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009f90 tx=0x7fa0f800b330 comp rx=0 tx=0).stop 2024-03-18T20:44:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.967+0000 7fa104d3c700 1 -- 172.21.15.46:0/729876339 shutdown_connections 2024-03-18T20:44:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.967+0000 7fa104d3c700 1 --2- 172.21.15.46:0/729876339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1000ffe50 0x7fa100100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.967+0000 7fa104d3c700 1 --2- 172.21.15.46:0/729876339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa100104f60 0x7fa10010cae0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.967+0000 7fa104d3c700 1 --2- 172.21.15.46:0/729876339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa100100800 0x7fa100104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.967+0000 7fa104d3c700 1 -- 172.21.15.46:0/729876339 >> 172.21.15.46:0/729876339 conn(0x7fa100076d40 msgr2=0x7fa100077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.967+0000 7fa104d3c700 1 -- 172.21.15.46:0/729876339 shutdown_connections 2024-03-18T20:44:12.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.967+0000 7fa104d3c700 1 -- 172.21.15.46:0/729876339 wait complete. 2024-03-18T20:44:12.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa104d3c700 1 Processor -- start 2024-03-18T20:44:12.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa104d3c700 1 -- start start 2024-03-18T20:44:12.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa104d3c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1000ffe50 0x7fa1001994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa104d3c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa100104f60 0x7fa100199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa104d3c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10019ddc0 0x7fa10019e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa104d3c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10019e840 con 0x7fa10019ddc0 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa104d3c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10019e9b0 con 0x7fa100104f60 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa104d3c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10019eb70 con 0x7fa1000ffe50 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa0fed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10019ddc0 0x7fa10019e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa0fed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10019ddc0 0x7fa10019e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56462/0 (socket says 172.21.15.46:56462) 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa0fed9d700 1 -- 172.21.15.46:0/1678077342 learned_addr learned my addr 172.21.15.46:0/1678077342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa0fe59c700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1000ffe50 0x7fa1001994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.968+0000 7fa0fdd9b700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa100104f60 0x7fa100199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.969+0000 7fa0fed9d700 1 -- 172.21.15.46:0/1678077342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1000ffe50 msgr2=0x7fa1001994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.969+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1000ffe50 0x7fa1001994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.969+0000 7fa0fed9d700 1 -- 172.21.15.46:0/1678077342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa100104f60 msgr2=0x7fa100199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.969+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa100104f60 0x7fa100199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.969+0000 7fa0fed9d700 1 -- 172.21.15.46:0/1678077342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f8009c40 con 0x7fa10019ddc0 2024-03-18T20:44:12.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.969+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10019ddc0 0x7fa10019e240 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8006070 tx=0x7fa0f8008c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.969+0000 7fa0ef7fe700 1 -- 172.21.15.46:0/1678077342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f800e040 con 0x7fa10019ddc0 2024-03-18T20:44:12.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.969+0000 7fa104d3c700 1 -- 172.21.15.46:0/1678077342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1001a6070 con 0x7fa10019ddc0 2024-03-18T20:44:12.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.969+0000 7fa0ef7fe700 1 -- 172.21.15.46:0/1678077342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0f800bda0 con 0x7fa10019ddc0 2024-03-18T20:44:12.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.969+0000 7fa0ef7fe700 1 -- 172.21.15.46:0/1678077342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80052a0 con 0x7fa10019ddc0 2024-03-18T20:44:12.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.969+0000 7fa104d3c700 1 -- 172.21.15.46:0/1678077342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1001a64f0 con 0x7fa10019ddc0 2024-03-18T20:44:12.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.972+0000 7fa0ef7fe700 1 -- 172.21.15.46:0/1678077342 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa0f801d470 con 0x7fa10019ddc0 2024-03-18T20:44:12.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.973+0000 7fa0ef7fe700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa0e80a1290 0x7fa0e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.973+0000 7fa0ef7fe700 1 -- 172.21.15.46:0/1678077342 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa0f80ce2c0 con 0x7fa10019ddc0 2024-03-18T20:44:12.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.973+0000 7fa0fe59c700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa0e80a1290 0x7fa0e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.973+0000 7fa0fe59c700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa0e80a1290 0x7fa0e80a3750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0005fd0 tx=0x7fa0f0009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.973+0000 7fa110ed1b80 1 Processor -- start 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa110ed1b80 1 -- start start 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa110ed1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e444a60 0x56096e444e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa110ed1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e445380 0x56096e44fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa110ed1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 0x56096e452610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa110ed1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096e39e990 con 0x56096e450220 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa110ed1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096e39c350 con 0x56096e444a60 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa110ed1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096e397c40 con 0x56096e445380 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa0fed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 0x56096e452610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa0fed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 0x56096e452610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56476/0 (socket says 172.21.15.46:56476) 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa0fed9d700 1 -- 172.21.15.46:0/1751203517 learned_addr learned my addr 172.21.15.46:0/1751203517 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa0fdd9b700 1 --2- 172.21.15.46:0/1751203517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e445380 0x56096e44fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa0fe59c700 1 --2- 172.21.15.46:0/1751203517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e444a60 0x56096e444e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa0fed9d700 1 -- 172.21.15.46:0/1751203517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e445380 msgr2=0x56096e44fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/1751203517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e445380 0x56096e44fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa0fed9d700 1 -- 172.21.15.46:0/1751203517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e444a60 msgr2=0x56096e444e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/1751203517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e444a60 0x56096e444e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa0fed9d700 1 -- 172.21.15.46:0/1751203517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56096e2193b0 con 0x56096e450220 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.974+0000 7fa0fdd9b700 1 --2- 172.21.15.46:0/1751203517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e445380 0x56096e44fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.975+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/1751203517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 0x56096e452610 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80978e0 tx=0x7fa0f80093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85968a3ba642bc45 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.975+0000 7fa0ecff9700 1 -- 172.21.15.46:0/1751203517 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80a1040 con 0x56096e450220 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.975+0000 7fa0ecff9700 1 -- 172.21.15.46:0/1751203517 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0f809ad40 con 0x56096e450220 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.975+0000 7fa0ecff9700 1 -- 172.21.15.46:0/1751203517 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80a37d0 con 0x56096e450220 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 -- 172.21.15.46:0/1751203517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 msgr2=0x56096e452610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/1751203517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 0x56096e452610 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80978e0 tx=0x7fa0f80093d0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 -- 172.21.15.46:0/1751203517 shutdown_connections 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/1751203517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 0x56096e452610 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/1751203517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e444a60 0x56096e444e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/1751203517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e445380 0x56096e44fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 -- 172.21.15.46:0/1751203517 >> 172.21.15.46:0/1751203517 conn(0x56096e3bb220 msgr2=0x56096e441020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 -- 172.21.15.46:0/1751203517 shutdown_connections 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 -- 172.21.15.46:0/1751203517 wait complete. 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 -- start start 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 Processor -- start 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e444a60 0x56096e44d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e445380 0x56096e44d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 0x56096e44daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096e333d80 con 0x56096e450220 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096e333a20 con 0x56096e445380 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.976+0000 7fa110ed1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096e21b6a0 con 0x56096e444a60 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa110ed1b80 10 client.0 _ll_register_callbacks cb 0x56096e19f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0fed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 0x56096e44daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0fe59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e444a60 0x56096e44d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0fed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 0x56096e44daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56492/0 (socket says 172.21.15.46:56492) 2024-03-18T20:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0fed9d700 1 -- 172.21.15.46:0/2535425936 learned_addr learned my addr 172.21.15.46:0/2535425936 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0fdd9b700 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e445380 0x56096e44d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0fed9d700 1 -- 172.21.15.46:0/2535425936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e444a60 msgr2=0x56096e44d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e444a60 0x56096e44d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0fed9d700 1 -- 172.21.15.46:0/2535425936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e445380 msgr2=0x56096e44d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e445380 0x56096e44d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0fed9d700 1 -- 172.21.15.46:0/2535425936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1001a64f0 con 0x56096e450220 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0fdd9b700 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e445380 0x56096e44d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 0x56096e44daf0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80a32a0 tx=0x7fa0f8007490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0fbcd31e2ff1d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0e5ffb700 1 -- 172.21.15.46:0/2535425936 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80a1040 con 0x56096e450220 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0e5ffb700 1 -- 172.21.15.46:0/2535425936 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa0f809bae0 con 0x56096e450220 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa110ed1b80 1 -- 172.21.15.46:0/2535425936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56096e438ad0 con 0x56096e450220 2024-03-18T20:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa0e5ffb700 1 -- 172.21.15.46:0/2535425936 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0f80a37d0 con 0x56096e450220 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa110ed1b80 10 client.18381 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.977+0000 7fa110ed1b80 1 -- 172.21.15.46:0/2535425936 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56096e1c5b60 con 0x56096e450220 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.978+0000 7fa0e5ffb700 1 -- 172.21.15.46:0/2535425936 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56096e438ad0 con 0x56096e450220 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.978+0000 7fa0e5ffb700 1 -- 172.21.15.46:0/2535425936 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa0f809a960 con 0x56096e450220 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.978+0000 7fa110ed1b80 10 client.18381 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.978+0000 7fa110ed1b80 10 client.18381 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.978+0000 7fa110ed1b80 20 client.18381 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.978+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56096e45f690 0x56096e461ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.978+0000 7fa110ed1b80 4 client.18381 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.978+0000 7fa110ed1b80 1 -- 172.21.15.46:0/2535425936 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56096e390700 con 0x56096e45f690 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.978+0000 7fa0fe59c700 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56096e45f690 0x56096e461ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa0e5ffb700 10 client.18381 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa0fe59c700 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56096e45f690 0x56096e461ab0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e97272890bf764 server_cookie=12b4dfaf76bfcc12 in_seq=0 out_seq=0 2024-03-18T20:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa0e5ffb700 1 -- 172.21.15.46:0/2535425936 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56096e390700 con 0x56096e45f690 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa0e5ffb700 10 client.18381 handle_command_reply: tid=0 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa110ed1b80 1 client.18381 shutdown 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa110ed1b80 1 -- 172.21.15.46:0/2535425936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 msgr2=0x56096e44daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 0x56096e44daf0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80a32a0 tx=0x7fa0f8007490 comp rx=0 tx=0).stop 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa110ed1b80 1 -- 172.21.15.46:0/2535425936 shutdown_connections 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56096e45f690 0x56096e461ab0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e450220 0x56096e44daf0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e444a60 0x56096e44d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa0e5ffb700 0 client.18381 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/2535425936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e445380 0x56096e44d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa110ed1b80 1 -- 172.21.15.46:0/2535425936 >> 172.21.15.46:0/2535425936 conn(0x56096e3bb220 msgr2=0x56096e440c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa110ed1b80 1 -- 172.21.15.46:0/2535425936 shutdown_connections 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.979+0000 7fa110ed1b80 1 -- 172.21.15.46:0/2535425936 wait complete. 2024-03-18T20:44:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.980+0000 7fa110ed1b80 20 client.18381 trim_cache size 0 max 16384 2024-03-18T20:44:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.993+0000 7fa110ed1b80 1 Processor -- start 2024-03-18T20:44:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.993+0000 7fa110ed1b80 1 -- start start 2024-03-18T20:44:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.993+0000 7fa110ed1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 0x56096e454630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.993+0000 7fa110ed1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e445380 0x56096e44d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.993+0000 7fa110ed1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e450220 0x56096e45ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.993+0000 7fa110ed1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096e43d5e0 con 0x56096e444a60 2024-03-18T20:44:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.993+0000 7fa110ed1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096e447ed0 con 0x56096e445380 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.993+0000 7fa110ed1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096e445090 con 0x56096e450220 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0fe59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 0x56096e454630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0fe59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 0x56096e454630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56496/0 (socket says 172.21.15.46:56496) 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0fe59c700 1 -- 172.21.15.46:0/3735100619 learned_addr learned my addr 172.21.15.46:0/3735100619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0fdd9b700 1 --2- 172.21.15.46:0/3735100619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e445380 0x56096e44d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/3735100619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e450220 0x56096e45ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0fe59c700 1 -- 172.21.15.46:0/3735100619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e450220 msgr2=0x56096e45ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0fe59c700 1 --2- 172.21.15.46:0/3735100619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e450220 0x56096e45ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0fe59c700 1 -- 172.21.15.46:0/3735100619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e445380 msgr2=0x56096e44d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0fe59c700 1 --2- 172.21.15.46:0/3735100619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e445380 0x56096e44d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0fe59c700 1 -- 172.21.15.46:0/3735100619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56096e35dea0 con 0x56096e444a60 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0fe59c700 1 --2- 172.21.15.46:0/3735100619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 0x56096e454630 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x56096e462380 tx=0x7fa0f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867ce309c849d2bd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0ed7fa700 1 -- 172.21.15.46:0/3735100619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f002f070 con 0x56096e444a60 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0ed7fa700 1 -- 172.21.15.46:0/3735100619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0f002ad00 con 0x56096e444a60 2024-03-18T20:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.994+0000 7fa0ed7fa700 1 -- 172.21.15.46:0/3735100619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0024040 con 0x56096e444a60 2024-03-18T20:44:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.995+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3735100619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 msgr2=0x56096e454630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.995+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/3735100619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 0x56096e454630 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x56096e462380 tx=0x7fa0f0014000 comp rx=0 tx=0).stop 2024-03-18T20:44:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.995+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3735100619 shutdown_connections 2024-03-18T20:44:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.995+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/3735100619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 0x56096e454630 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.995+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/3735100619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e450220 0x56096e45ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.995+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/3735100619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e445380 0x56096e44d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.995+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3735100619 >> 172.21.15.46:0/3735100619 conn(0x56096e3bb220 msgr2=0x56096e440b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.995+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3735100619 shutdown_connections 2024-03-18T20:44:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.995+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3735100619 wait complete. 2024-03-18T20:44:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.995+0000 7fa110ed1b80 1 -- start start 2024-03-18T20:44:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.995+0000 7fa110ed1b80 1 Processor -- start 2024-03-18T20:44:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa110ed1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 0x56096e453e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa110ed1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e445380 0x56096e4543b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa110ed1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e450220 0x56096e4524c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa110ed1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096e452bd0 con 0x56096e444a60 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa110ed1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096e452d40 con 0x56096e450220 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa110ed1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096e452f00 con 0x56096e445380 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa110ed1b80 10 client.0 _ll_register_callbacks cb 0x7fa0f8009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0fe59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 0x56096e453e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0fed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e450220 0x56096e4524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0fe59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 0x56096e453e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56512/0 (socket says 172.21.15.46:56512) 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0fe59c700 1 -- 172.21.15.46:0/3495978312 learned_addr learned my addr 172.21.15.46:0/3495978312 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0fdd9b700 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e445380 0x56096e4543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0fed9d700 1 -- 172.21.15.46:0/3495978312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e445380 msgr2=0x56096e4543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e445380 0x56096e4543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0fed9d700 1 -- 172.21.15.46:0/3495978312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 msgr2=0x56096e453e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 0x56096e453e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0fed9d700 1 -- 172.21.15.46:0/3495978312 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f0017070 con 0x56096e450220 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0fe59c700 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 0x56096e453e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0fed9d700 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e450220 0x56096e4524c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8097e50 tx=0x7fa0f8007420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d49f2fe4eb9df5b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.996+0000 7fa0e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa0e5ffb700 1 -- 172.21.15.46:0/3495978312 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80a1040 con 0x56096e450220 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3495978312 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0f809ad40 con 0x56096e450220 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa0e5ffb700 1 -- 172.21.15.46:0/3495978312 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa0f809bae0 con 0x56096e450220 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa0e5ffb700 1 -- 172.21.15.46:0/3495978312 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0f80b3460 con 0x56096e450220 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa110ed1b80 10 client.28040 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3495978312 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56096e21dcd0 con 0x56096e450220 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa0e5ffb700 1 -- 172.21.15.46:0/3495978312 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0f809ad40 con 0x56096e450220 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa0e5ffb700 1 -- 172.21.15.46:0/3495978312 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa0f80971d0 con 0x56096e450220 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa110ed1b80 10 client.28040 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:13.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa110ed1b80 10 client.28040 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:44:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa110ed1b80 20 client.28040 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56096e4b4200 0x56096e4b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa110ed1b80 4 client.28040 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:44:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3495978312 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56096e1e6cf0 con 0x56096e4b4200 2024-03-18T20:44:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.997+0000 7fa0fe59c700 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56096e4b4200 0x56096e4b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa0fe59c700 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56096e4b4200 0x56096e4b65f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6aaf98beb1f0ef6 server_cookie=568e5bcbde214c5a in_seq=0 out_seq=0 2024-03-18T20:44:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa0e5ffb700 10 client.28040 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:44:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa0e5ffb700 1 -- 172.21.15.46:0/3495978312 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56096e1e6cf0 con 0x56096e4b4200 2024-03-18T20:44:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa0e5ffb700 10 client.28040 handle_command_reply: tid=0 2024-03-18T20:44:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa110ed1b80 1 client.28040 shutdown 2024-03-18T20:44:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3495978312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e450220 msgr2=0x56096e4524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e450220 0x56096e4524c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8097e50 tx=0x7fa0f8007420 comp rx=0 tx=0).stop 2024-03-18T20:44:13.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3495978312 shutdown_connections 2024-03-18T20:44:13.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x56096e4b4200 0x56096e4b65f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56096e444a60 0x56096e453e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56096e450220 0x56096e4524c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa110ed1b80 1 --2- 172.21.15.46:0/3495978312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56096e445380 0x56096e4543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa0e5ffb700 0 client.28040 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:44:13.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3495978312 >> 172.21.15.46:0/3495978312 conn(0x56096e3bb220 msgr2=0x56096e44d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3495978312 shutdown_connections 2024-03-18T20:44:13.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa110ed1b80 1 -- 172.21.15.46:0/3495978312 wait complete. 2024-03-18T20:44:13.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.998+0000 7fa110ed1b80 20 client.28040 trim_cache size 0 max 16384 2024-03-18T20:44:13.007 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:44:13.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.999+0000 7fa104d3c700 1 -- 172.21.15.46:0/1678077342 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa0e80a1290 msgr2=0x7fa0e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.999+0000 7fa104d3c700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa0e80a1290 0x7fa0e80a3750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0005fd0 tx=0x7fa0f0009310 comp rx=0 tx=0).stop 2024-03-18T20:44:13.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.999+0000 7fa104d3c700 1 -- 172.21.15.46:0/1678077342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10019ddc0 msgr2=0x7fa10019e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.999+0000 7fa104d3c700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10019ddc0 0x7fa10019e240 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8006070 tx=0x7fa0f8008c40 comp rx=0 tx=0).stop 2024-03-18T20:44:13.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.999+0000 7fa104d3c700 1 -- 172.21.15.46:0/1678077342 shutdown_connections 2024-03-18T20:44:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.999+0000 7fa104d3c700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa0e80a1290 0x7fa0e80a3750 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.999+0000 7fa104d3c700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa10019ddc0 0x7fa10019e240 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.999+0000 7fa104d3c700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1000ffe50 0x7fa1001994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.999+0000 7fa104d3c700 1 --2- 172.21.15.46:0/1678077342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa100104f60 0x7fa100199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.999+0000 7fa104d3c700 1 -- 172.21.15.46:0/1678077342 >> 172.21.15.46:0/1678077342 conn(0x7fa100076d40 msgr2=0x7fa100068330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.999+0000 7fa104d3c700 1 -- 172.21.15.46:0/1678077342 shutdown_connections 2024-03-18T20:44:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:12.999+0000 7fa104d3c700 1 -- 172.21.15.46:0/1678077342 wait complete. 2024-03-18T20:44:13.015 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:44:13.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:44:13.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.152+0000 7f5985a86700 1 -- 172.21.15.46:0/2739812114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5980100f50 msgr2=0x7f5980101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.152+0000 7f5985a86700 1 --2- 172.21.15.46:0/2739812114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5980100f50 0x7f5980101330 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f5974009990 tx=0x7f5974009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.152+0000 7f5985a86700 1 -- 172.21.15.46:0/2739812114 shutdown_connections 2024-03-18T20:44:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.152+0000 7f5985a86700 1 --2- 172.21.15.46:0/2739812114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5980100f50 0x7f5980101330 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.152+0000 7f5985a86700 1 --2- 172.21.15.46:0/2739812114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5980106070 0x7f598010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.152+0000 7f5985a86700 1 --2- 172.21.15.46:0/2739812114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5980101900 0x7f5980105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.152+0000 7f5985a86700 1 -- 172.21.15.46:0/2739812114 >> 172.21.15.46:0/2739812114 conn(0x7f59800fc7f0 msgr2=0x7f59800fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.152+0000 7f5985a86700 1 -- 172.21.15.46:0/2739812114 shutdown_connections 2024-03-18T20:44:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.152+0000 7f5985a86700 1 -- 172.21.15.46:0/2739812114 wait complete. 2024-03-18T20:44:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.153+0000 7f5985a86700 1 Processor -- start 2024-03-18T20:44:13.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.153+0000 7f5985a86700 1 -- start start 2024-03-18T20:44:13.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.153+0000 7f5985a86700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5980100f50 0x7f59800730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.153+0000 7f5985a86700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5980106070 0x7f59800735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.153+0000 7f5985a86700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5980073b30 0x7f59801a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.153+0000 7f5985a86700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59801a59d0 con 0x7f5980106070 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.153+0000 7f5985a86700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59801a5b40 con 0x7f5980073b30 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.153+0000 7f5985a86700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59801a5d00 con 0x7f5980100f50 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5980106070 0x7f59800735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5980106070 0x7f59800735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56530/0 (socket says 172.21.15.46:56530) 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597effd700 1 -- 172.21.15.46:0/1854540798 learned_addr learned my addr 172.21.15.46:0/1854540798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597ffff700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5980073b30 0x7f59801a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597f7fe700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5980100f50 0x7f59800730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597effd700 1 -- 172.21.15.46:0/1854540798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5980100f50 msgr2=0x7f59800730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597effd700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5980100f50 0x7f59800730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597effd700 1 -- 172.21.15.46:0/1854540798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5980073b30 msgr2=0x7f59801a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597effd700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5980073b30 0x7f59801a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597effd700 1 -- 172.21.15.46:0/1854540798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59740095f0 con 0x7f5980106070 2024-03-18T20:44:13.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597f7fe700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5980100f50 0x7f59800730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:44:13.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597effd700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5980106070 0x7f59800735f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f597000e970 tx=0x7f597000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597cff9700 1 -- 172.21.15.46:0/1854540798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970009790 con 0x7f5980106070 2024-03-18T20:44:13.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f5985a86700 1 -- 172.21.15.46:0/1854540798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59801a5fe0 con 0x7f5980106070 2024-03-18T20:44:13.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597cff9700 1 -- 172.21.15.46:0/1854540798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f597000eec0 con 0x7f5980106070 2024-03-18T20:44:13.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f597cff9700 1 -- 172.21.15.46:0/1854540798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970005590 con 0x7f5980106070 2024-03-18T20:44:13.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.154+0000 7f5985a86700 1 -- 172.21.15.46:0/1854540798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59801a6460 con 0x7f5980106070 2024-03-18T20:44:13.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.157+0000 7f597cff9700 1 -- 172.21.15.46:0/1854540798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5970021440 con 0x7f5980106070 2024-03-18T20:44:13.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.158+0000 7f597cff9700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f59680a1290 0x7f59680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.158+0000 7f597cff9700 1 -- 172.21.15.46:0/1854540798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5970014070 con 0x7f5980106070 2024-03-18T20:44:13.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.159+0000 7f5991c1bb80 1 Processor -- start 2024-03-18T20:44:13.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.159+0000 7f597f7fe700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f59680a1290 0x7f59680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.159+0000 7f5991c1bb80 1 -- start start 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.159+0000 7f5991c1bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95c9270 0x55bae95c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.159+0000 7f5991c1bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae95d4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.159+0000 7f5991c1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95d4a40 0x55bae95d6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.159+0000 7f5991c1bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae951a990 con 0x55bae95c9270 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.159+0000 7f5991c1bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae9518350 con 0x55bae95c9b90 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.159+0000 7f5991c1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae9513c40 con 0x55bae95d4a40 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae95d4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae95d4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55830/0 (socket says 172.21.15.46:55830) 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597effd700 1 -- 172.21.15.46:0/3245708740 learned_addr learned my addr 172.21.15.46:0/3245708740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597f7fe700 1 --2- 172.21.15.46:0/3245708740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95c9270 0x55bae95c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597ffff700 1 --2- 172.21.15.46:0/3245708740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95d4a40 0x55bae95d6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597effd700 1 -- 172.21.15.46:0/3245708740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95d4a40 msgr2=0x55bae95d6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597effd700 1 --2- 172.21.15.46:0/3245708740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95d4a40 0x55bae95d6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597f7fe700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f59680a1290 0x7f59680a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f597400b2e0 tx=0x7f5974005ab0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597effd700 1 -- 172.21.15.46:0/3245708740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95c9270 msgr2=0x55bae95c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597effd700 1 --2- 172.21.15.46:0/3245708740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95c9270 0x55bae95c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597effd700 1 -- 172.21.15.46:0/3245708740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bae93953b0 con 0x55bae95c9b90 2024-03-18T20:44:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597ffff700 1 --2- 172.21.15.46:0/3245708740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95d4a40 0x55bae95d6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:44:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f597effd700 1 --2- 172.21.15.46:0/3245708740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae95d4500 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5970096ac0 tx=0x7f59700057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa9460b93c6e14b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f5965ffb700 1 -- 172.21.15.46:0/3245708740 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597009de80 con 0x55bae95c9b90 2024-03-18T20:44:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f5965ffb700 1 -- 172.21.15.46:0/3245708740 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59700a24d0 con 0x55bae95c9b90 2024-03-18T20:44:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.160+0000 7f5965ffb700 1 -- 172.21.15.46:0/3245708740 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970016570 con 0x55bae95c9b90 2024-03-18T20:44:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.161+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3245708740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 msgr2=0x55bae95d4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.161+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/3245708740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae95d4500 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5970096ac0 tx=0x7f59700057b0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.161+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3245708740 shutdown_connections 2024-03-18T20:44:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.161+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/3245708740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95c9270 0x55bae95c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.161+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/3245708740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95d4a40 0x55bae95d6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.161+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/3245708740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae95d4500 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.161+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3245708740 >> 172.21.15.46:0/3245708740 conn(0x55bae9537220 msgr2=0x55bae95bcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.161+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3245708740 shutdown_connections 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.161+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3245708740 wait complete. 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f5991c1bb80 1 -- start start 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f5991c1bb80 1 Processor -- start 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f5991c1bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9270 0x55bae95d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f5991c1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95c9b90 0x55bae95d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f5991c1bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95d4a40 0x55bae95d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f5991c1bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae94afd80 con 0x55bae95d4a40 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f5991c1bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae94afa20 con 0x55bae95c9270 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f5991c1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae93976a0 con 0x55bae95c9b90 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f5991c1bb80 10 client.0 _ll_register_callbacks cb 0x55bae931b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f597f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9270 0x55bae95d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f597ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95d4a40 0x55bae95d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f597f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9270 0x55bae95d18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55846/0 (socket says 172.21.15.46:55846) 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f597f7fe700 1 -- 172.21.15.46:0/320512315 learned_addr learned my addr 172.21.15.46:0/320512315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.162+0000 7f597effd700 1 --2- 172.21.15.46:0/320512315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95c9b90 0x55bae95d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f597f7fe700 1 -- 172.21.15.46:0/320512315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95c9b90 msgr2=0x55bae95d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f597f7fe700 1 --2- 172.21.15.46:0/320512315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95c9b90 0x55bae95d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f597f7fe700 1 -- 172.21.15.46:0/320512315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95d4a40 msgr2=0x55bae95d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f597f7fe700 1 --2- 172.21.15.46:0/320512315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95d4a40 0x55bae95d2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f597f7fe700 1 -- 172.21.15.46:0/320512315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59801a6460 con 0x55bae95c9270 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f597f7fe700 1 --2- 172.21.15.46:0/320512315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9270 0x55bae95d18a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5974009990 tx=0x7f5974009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea17205d8045865 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f595effd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f595effd700 1 -- 172.21.15.46:0/320512315 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5974004ae0 con 0x55bae95c9270 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f595effd700 1 -- 172.21.15.46:0/320512315 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5974004c40 con 0x55bae95c9270 2024-03-18T20:44:13.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f5991c1bb80 1 -- 172.21.15.46:0/320512315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bae95b4ad0 con 0x55bae95c9270 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f595effd700 1 -- 172.21.15.46:0/320512315 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5974004e00 con 0x55bae95c9270 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f5991c1bb80 10 client.28052 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.163+0000 7f5991c1bb80 1 -- 172.21.15.46:0/320512315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55bae9341b60 con 0x55bae95c9270 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.164+0000 7f595effd700 1 -- 172.21.15.46:0/320512315 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f597403d080 con 0x55bae95c9270 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.164+0000 7f595effd700 1 -- 172.21.15.46:0/320512315 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5974031600 con 0x55bae95c9270 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.164+0000 7f5991c1bb80 10 client.28052 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.164+0000 7f5991c1bb80 10 client.28052 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.164+0000 7f5991c1bb80 20 client.28052 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.164+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/320512315 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55bae95e4780 0x55bae95e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.164+0000 7f5991c1bb80 4 client.28052 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.164+0000 7f5991c1bb80 1 -- 172.21.15.46:0/320512315 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bae950c700 con 0x55bae95e4780 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.164+0000 7f597effd700 1 --2- 172.21.15.46:0/320512315 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55bae95e4780 0x55bae95e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.164+0000 7f595effd700 10 client.28052 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.164+0000 7f597effd700 1 --2- 172.21.15.46:0/320512315 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55bae95e4780 0x55bae95e6ba0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89088a457b3be0b2 server_cookie=6893bb8fd3b0597e in_seq=0 out_seq=0 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f595effd700 1 -- 172.21.15.46:0/320512315 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bae950c700 con 0x55bae95e4780 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f595effd700 10 client.28052 handle_command_reply: tid=0 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f5991c1bb80 1 client.28052 shutdown 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f5991c1bb80 1 -- 172.21.15.46:0/320512315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9270 msgr2=0x55bae95d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/320512315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9270 0x55bae95d18a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5974009990 tx=0x7f5974009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f5991c1bb80 1 -- 172.21.15.46:0/320512315 shutdown_connections 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/320512315 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55bae95e4780 0x55bae95e6ba0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/320512315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95d4a40 0x55bae95d2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/320512315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9270 0x55bae95d18a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/320512315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95c9b90 0x55bae95d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f595effd700 0 client.28052 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f5991c1bb80 1 -- 172.21.15.46:0/320512315 >> 172.21.15.46:0/320512315 conn(0x55bae9537220 msgr2=0x55bae95bcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f5991c1bb80 1 -- 172.21.15.46:0/320512315 shutdown_connections 2024-03-18T20:44:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.165+0000 7f5991c1bb80 1 -- 172.21.15.46:0/320512315 wait complete. 2024-03-18T20:44:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.166+0000 7f5991c1bb80 20 client.28052 trim_cache size 0 max 16384 2024-03-18T20:44:13.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.179+0000 7f5991c1bb80 1 Processor -- start 2024-03-18T20:44:13.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.179+0000 7f5991c1bb80 1 -- start start 2024-03-18T20:44:13.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.179+0000 7f5991c1bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95c9270 0x55bae95cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.179+0000 7f5991c1bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae95db670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.179+0000 7f5991c1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95d4a40 0x55bae95ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.179+0000 7f5991c1bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae95be230 con 0x55bae95c9270 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.179+0000 7f5991c1bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae95d2ea0 con 0x55bae95c9b90 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.179+0000 7f5991c1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae95ca4e0 con 0x55bae95d4a40 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.179+0000 7f597f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95c9270 0x55bae95cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.179+0000 7f597f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95c9270 0x55bae95cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56554/0 (socket says 172.21.15.46:56554) 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.179+0000 7f597f7fe700 1 -- 172.21.15.46:0/569135503 learned_addr learned my addr 172.21.15.46:0/569135503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.180+0000 7f597effd700 1 --2- 172.21.15.46:0/569135503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae95db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.180+0000 7f597ffff700 1 --2- 172.21.15.46:0/569135503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95d4a40 0x55bae95ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.180+0000 7f597f7fe700 1 -- 172.21.15.46:0/569135503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95d4a40 msgr2=0x55bae95ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.180+0000 7f597f7fe700 1 --2- 172.21.15.46:0/569135503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95d4a40 0x55bae95ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.180+0000 7f597f7fe700 1 -- 172.21.15.46:0/569135503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 msgr2=0x55bae95db670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.180+0000 7f597f7fe700 1 --2- 172.21.15.46:0/569135503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae95db670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.180+0000 7f597f7fe700 1 -- 172.21.15.46:0/569135503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bae95eb7d0 con 0x55bae95c9270 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.180+0000 7f597f7fe700 1 --2- 172.21.15.46:0/569135503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95c9270 0x55bae95cc320 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f597400b690 tx=0x7f59740055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce95b791d43a2a99 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.180+0000 7f59667fc700 1 -- 172.21.15.46:0/569135503 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5974031440 con 0x55bae95c9270 2024-03-18T20:44:13.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.180+0000 7f59667fc700 1 -- 172.21.15.46:0/569135503 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bae95b4ad0 con 0x55bae95c9270 2024-03-18T20:44:13.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.180+0000 7f59667fc700 1 -- 172.21.15.46:0/569135503 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597400f950 con 0x55bae95c9270 2024-03-18T20:44:13.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.181+0000 7f5991c1bb80 1 -- 172.21.15.46:0/569135503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95c9270 msgr2=0x55bae95cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.181+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/569135503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95c9270 0x55bae95cc320 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f597400b690 tx=0x7f59740055c0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.181+0000 7f5991c1bb80 1 -- 172.21.15.46:0/569135503 shutdown_connections 2024-03-18T20:44:13.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.181+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/569135503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95c9270 0x55bae95cc320 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.181+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/569135503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95d4a40 0x55bae95ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.181+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/569135503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae95db670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.181+0000 7f5991c1bb80 1 -- 172.21.15.46:0/569135503 >> 172.21.15.46:0/569135503 conn(0x55bae9537220 msgr2=0x55bae95ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.181+0000 7f5991c1bb80 1 -- 172.21.15.46:0/569135503 shutdown_connections 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.181+0000 7f5991c1bb80 1 -- 172.21.15.46:0/569135503 wait complete. 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.181+0000 7f5991c1bb80 1 -- start start 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f5991c1bb80 1 Processor -- start 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f5991c1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95c9270 0x55bae9631080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f5991c1bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae96315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f5991c1bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95d4a40 0x55bae95dad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f5991c1bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae95db320 con 0x55bae95d4a40 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f5991c1bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae95db490 con 0x55bae95c9b90 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f5991c1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae95db650 con 0x55bae95c9270 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f5991c1bb80 10 client.0 _ll_register_callbacks cb 0x7f597400ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f597ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95d4a40 0x55bae95dad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f597ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95d4a40 0x55bae95dad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56562/0 (socket says 172.21.15.46:56562) 2024-03-18T20:44:13.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f597ffff700 1 -- 172.21.15.46:0/3395087635 learned_addr learned my addr 172.21.15.46:0/3395087635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f597f7fe700 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95c9270 0x55bae9631080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f597effd700 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae96315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f597ffff700 1 -- 172.21.15.46:0/3395087635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95c9270 msgr2=0x55bae9631080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f597ffff700 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95c9270 0x55bae9631080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f597ffff700 1 -- 172.21.15.46:0/3395087635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 msgr2=0x55bae96315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f597ffff700 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae96315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.182+0000 7f597ffff700 1 -- 172.21.15.46:0/3395087635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bae9341b60 con 0x55bae95d4a40 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f597ffff700 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95d4a40 0x55bae95dad40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f59780061e0 tx=0x7f5978007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98e6222bf63b6c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f595effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f595effd700 1 -- 172.21.15.46:0/3395087635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597800e040 con 0x55bae95d4a40 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f595effd700 1 -- 172.21.15.46:0/3395087635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5978016790 con 0x55bae95d4a40 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3395087635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59700a24d0 con 0x55bae95d4a40 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f595effd700 1 -- 172.21.15.46:0/3395087635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5978016950 con 0x55bae95d4a40 2024-03-18T20:44:13.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f5991c1bb80 10 client.18405 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3395087635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55bae94d9ea0 con 0x55bae95d4a40 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f595effd700 1 -- 172.21.15.46:0/3395087635 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f597802c080 con 0x55bae95d4a40 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f595effd700 1 -- 172.21.15.46:0/3395087635 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5978020cf0 con 0x55bae95d4a40 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f5991c1bb80 10 client.18405 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f5991c1bb80 10 client.18405 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f5991c1bb80 20 client.18405 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55bae9639080 0x55bae963b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f5991c1bb80 4 client.18405 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.183+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3395087635 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bae9362cf0 con 0x55bae9639080 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f597f7fe700 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55bae9639080 0x55bae963b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f595effd700 10 client.18405 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f597f7fe700 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55bae9639080 0x55bae963b4a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a213d43a7f57fbda server_cookie=8ba2f3d9bfe52425 in_seq=0 out_seq=0 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f595effd700 1 -- 172.21.15.46:0/3395087635 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bae9362cf0 con 0x55bae9639080 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f595effd700 10 client.18405 handle_command_reply: tid=0 2024-03-18T20:44:13.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f5991c1bb80 1 client.18405 shutdown 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3395087635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95d4a40 msgr2=0x55bae95dad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95d4a40 0x55bae95dad40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f59780061e0 tx=0x7f5978007be0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3395087635 shutdown_connections 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55bae9639080 0x55bae963b4a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bae95d4a40 0x55bae95dad40 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bae95c9270 0x55bae9631080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f595effd700 0 client.18405 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f5991c1bb80 1 --2- 172.21.15.46:0/3395087635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bae95c9b90 0x55bae96315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3395087635 >> 172.21.15.46:0/3395087635 conn(0x55bae9537220 msgr2=0x55bae95bc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.184+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3395087635 shutdown_connections 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5991c1bb80 1 -- 172.21.15.46:0/3395087635 wait complete. 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5991c1bb80 20 client.18405 trim_cache size 0 max 16384 2024-03-18T20:44:13.193 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:44:13.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5985a86700 1 -- 172.21.15.46:0/1854540798 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f59680a1290 msgr2=0x7f59680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5985a86700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f59680a1290 0x7f59680a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f597400b2e0 tx=0x7f5974005ab0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5985a86700 1 -- 172.21.15.46:0/1854540798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5980106070 msgr2=0x7f59800735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5985a86700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5980106070 0x7f59800735f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f597000e970 tx=0x7f597000ec80 comp rx=0 tx=0).stop 2024-03-18T20:44:13.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5985a86700 1 -- 172.21.15.46:0/1854540798 shutdown_connections 2024-03-18T20:44:13.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5985a86700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f59680a1290 0x7f59680a3750 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5985a86700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5980073b30 0x7f59801a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5985a86700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5980100f50 0x7f59800730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5985a86700 1 --2- 172.21.15.46:0/1854540798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5980106070 0x7f59800735f0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5985a86700 1 -- 172.21.15.46:0/1854540798 >> 172.21.15.46:0/1854540798 conn(0x7f59800fc7f0 msgr2=0x7f59800fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5985a86700 1 -- 172.21.15.46:0/1854540798 shutdown_connections 2024-03-18T20:44:13.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.185+0000 7f5985a86700 1 -- 172.21.15.46:0/1854540798 wait complete. 2024-03-18T20:44:13.201 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:44:13.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:44:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.339+0000 7f890c2cf700 1 -- 172.21.15.46:0/338939259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89040ffe70 msgr2=0x7f8904100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.339+0000 7f890c2cf700 1 --2- 172.21.15.46:0/338939259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89040ffe70 0x7f8904100250 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f8900009990 tx=0x7f8900009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.339+0000 7f890c2cf700 1 -- 172.21.15.46:0/338939259 shutdown_connections 2024-03-18T20:44:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.339+0000 7f890c2cf700 1 --2- 172.21.15.46:0/338939259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89040ffe70 0x7f8904100250 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.339+0000 7f890c2cf700 1 --2- 172.21.15.46:0/338939259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8904104f70 0x7f890410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.339+0000 7f890c2cf700 1 --2- 172.21.15.46:0/338939259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8904100820 0x7f8904104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.339+0000 7f890c2cf700 1 -- 172.21.15.46:0/338939259 >> 172.21.15.46:0/338939259 conn(0x7f8904076d50 msgr2=0x7f8904077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.339+0000 7f890c2cf700 1 -- 172.21.15.46:0/338939259 shutdown_connections 2024-03-18T20:44:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.339+0000 7f890c2cf700 1 -- 172.21.15.46:0/338939259 wait complete. 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890c2cf700 1 Processor -- start 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890c2cf700 1 -- start start 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890c2cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89040ffe70 0x7f89041971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890c2cf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8904104f70 0x7f89041976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890c2cf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f890419ba90 0x7f890419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890c2cf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890419c510 con 0x7f89040ffe70 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890c2cf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890419c680 con 0x7f890419ba90 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890c2cf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890419c840 con 0x7f8904104f70 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890a06b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89040ffe70 0x7f89041971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890a06b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89040ffe70 0x7f89041971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56576/0 (socket says 172.21.15.46:56576) 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890a06b700 1 -- 172.21.15.46:0/2878240851 learned_addr learned my addr 172.21.15.46:0/2878240851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890a86c700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f890419ba90 0x7f890419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890986a700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8904104f70 0x7f89041976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890a06b700 1 -- 172.21.15.46:0/2878240851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8904104f70 msgr2=0x7f89041976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890a06b700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8904104f70 0x7f89041976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890a06b700 1 -- 172.21.15.46:0/2878240851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f890419ba90 msgr2=0x7f890419bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890a06b700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f890419ba90 0x7f890419bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.340+0000 7f890a06b700 1 -- 172.21.15.46:0/2878240851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89000095f0 con 0x7f89040ffe70 2024-03-18T20:44:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.341+0000 7f890a06b700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89040ffe70 0x7f89041971b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f8900004760 tx=0x7f8900004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.341+0000 7f88fb7fe700 1 -- 172.21.15.46:0/2878240851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890001d070 con 0x7f89040ffe70 2024-03-18T20:44:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.341+0000 7f88fb7fe700 1 -- 172.21.15.46:0/2878240851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f890000fb30 con 0x7f89040ffe70 2024-03-18T20:44:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.341+0000 7f890c2cf700 1 -- 172.21.15.46:0/2878240851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89041a3d40 con 0x7f89040ffe70 2024-03-18T20:44:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.341+0000 7f88fb7fe700 1 -- 172.21.15.46:0/2878240851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900022db0 con 0x7f89040ffe70 2024-03-18T20:44:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.341+0000 7f890c2cf700 1 -- 172.21.15.46:0/2878240851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89041a4160 con 0x7f89040ffe70 2024-03-18T20:44:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.344+0000 7f88fb7fe700 1 -- 172.21.15.46:0/2878240851 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f890000fca0 con 0x7f89040ffe70 2024-03-18T20:44:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.344+0000 7f88fb7fe700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88f00a1290 0x7f88f00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.344+0000 7f88fb7fe700 1 -- 172.21.15.46:0/2878240851 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f89000cdf60 con 0x7f89040ffe70 2024-03-18T20:44:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.344+0000 7f890986a700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88f00a1290 0x7f88f00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.345+0000 7f890986a700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88f00a1290 0x7f88f00a3750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8904198700 tx=0x7f88f40073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.345+0000 7f8918464b80 1 Processor -- start 2024-03-18T20:44:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.345+0000 7f8918464b80 1 -- start start 2024-03-18T20:44:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f8918464b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 0x5650c34e9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f8918464b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 0x5650c34f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f8918464b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34f5220 0x5650c34f7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f8918464b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c3443990 con 0x5650c34ea380 2024-03-18T20:44:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f8918464b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c3441350 con 0x5650c34f5220 2024-03-18T20:44:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f8918464b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c343cc40 con 0x5650c34e9a60 2024-03-18T20:44:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f890986a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 0x5650c34f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f890986a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 0x5650c34f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56588/0 (socket says 172.21.15.46:56588) 2024-03-18T20:44:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f890986a700 1 -- 172.21.15.46:0/106335882 learned_addr learned my addr 172.21.15.46:0/106335882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f890a86c700 1 --2- 172.21.15.46:0/106335882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34f5220 0x5650c34f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f890a06b700 1 --2- 172.21.15.46:0/106335882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 0x5650c34e9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f890986a700 1 -- 172.21.15.46:0/106335882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 msgr2=0x5650c34e9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f890986a700 1 --2- 172.21.15.46:0/106335882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 0x5650c34e9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f890986a700 1 -- 172.21.15.46:0/106335882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34f5220 msgr2=0x5650c34f7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f890986a700 1 --2- 172.21.15.46:0/106335882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34f5220 0x5650c34f7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f890986a700 1 -- 172.21.15.46:0/106335882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650c32be3b0 con 0x5650c34ea380 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f890986a700 1 --2- 172.21.15.46:0/106335882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 0x5650c34f4ce0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f88f4004500 tx=0x7f88f4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6295f38bbafa7631 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.346+0000 7f88f8da8700 1 -- 172.21.15.46:0/106335882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4004530 con 0x5650c34ea380 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.347+0000 7f88f8da8700 1 -- 172.21.15.46:0/106335882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f4003b40 con 0x5650c34ea380 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.347+0000 7f88f8da8700 1 -- 172.21.15.46:0/106335882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4011ac0 con 0x5650c34ea380 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.347+0000 7f8918464b80 1 -- 172.21.15.46:0/106335882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 msgr2=0x5650c34f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.347+0000 7f8918464b80 1 --2- 172.21.15.46:0/106335882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 0x5650c34f4ce0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f88f4004500 tx=0x7f88f4013000 comp rx=0 tx=0).stop 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.347+0000 7f8918464b80 1 -- 172.21.15.46:0/106335882 shutdown_connections 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.347+0000 7f8918464b80 1 --2- 172.21.15.46:0/106335882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34f5220 0x5650c34f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.347+0000 7f8918464b80 1 --2- 172.21.15.46:0/106335882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 0x5650c34e9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.347+0000 7f8918464b80 1 --2- 172.21.15.46:0/106335882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 0x5650c34f4ce0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.347+0000 7f8918464b80 1 -- 172.21.15.46:0/106335882 >> 172.21.15.46:0/106335882 conn(0x5650c3460220 msgr2=0x5650c34e6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.347+0000 7f8918464b80 1 -- 172.21.15.46:0/106335882 shutdown_connections 2024-03-18T20:44:13.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.347+0000 7f8918464b80 1 -- 172.21.15.46:0/106335882 wait complete. 2024-03-18T20:44:13.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f8918464b80 1 -- start start 2024-03-18T20:44:13.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f8918464b80 1 Processor -- start 2024-03-18T20:44:13.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f8918464b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34e9a60 0x5650c34f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f8918464b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34ea380 0x5650c34f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f8918464b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34f5220 0x5650c34f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f8918464b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c33d8d80 con 0x5650c34e9a60 2024-03-18T20:44:13.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f8918464b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c33d8a20 con 0x5650c34ea380 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f8918464b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c32c06a0 con 0x5650c34f5220 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f8918464b80 10 client.0 _ll_register_callbacks cb 0x5650c32443a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f890a06b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34e9a60 0x5650c34f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f890a06b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34e9a60 0x5650c34f4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56590/0 (socket says 172.21.15.46:56590) 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f890a06b700 1 -- 172.21.15.46:0/1339810144 learned_addr learned my addr 172.21.15.46:0/1339810144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f890a86c700 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34f5220 0x5650c34f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.348+0000 7f890986a700 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34ea380 0x5650c34f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f890a06b700 1 -- 172.21.15.46:0/1339810144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34f5220 msgr2=0x5650c34f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f890a06b700 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34f5220 0x5650c34f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f890a06b700 1 -- 172.21.15.46:0/1339810144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34ea380 msgr2=0x5650c34f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f890a06b700 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34ea380 0x5650c34f1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f890a06b700 1 -- 172.21.15.46:0/1339810144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f40041e0 con 0x5650c34e9a60 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f890a06b700 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34e9a60 0x5650c34f4c50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8900004730 tx=0x7f890009abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34d5443867c3ed5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f88edffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:13.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f88edffb700 1 -- 172.21.15.46:0/1339810144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89000b3070 con 0x5650c34e9a60 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f88edffb700 1 -- 172.21.15.46:0/1339810144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f890009b200 con 0x5650c34e9a60 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f8918464b80 1 -- 172.21.15.46:0/1339810144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650c34ddad0 con 0x5650c34e9a60 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f88edffb700 1 -- 172.21.15.46:0/1339810144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f890009f030 con 0x5650c34e9a60 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f8918464b80 10 client.18417 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.349+0000 7f8918464b80 1 -- 172.21.15.46:0/1339810144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5650c326ab60 con 0x5650c34e9a60 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.350+0000 7f88edffb700 1 -- 172.21.15.46:0/1339810144 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650c34ddad0 con 0x5650c34e9a60 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.350+0000 7f88edffb700 1 -- 172.21.15.46:0/1339810144 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f89000b3d20 con 0x5650c34e9a60 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.350+0000 7f8918464b80 10 client.18417 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.350+0000 7f8918464b80 10 client.18417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.350+0000 7f8918464b80 20 client.18417 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.350+0000 7f8918464b80 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650c350d310 0x5650c350f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.350+0000 7f8918464b80 4 client.18417 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.350+0000 7f8918464b80 1 -- 172.21.15.46:0/1339810144 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650c3435700 con 0x5650c350d310 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.350+0000 7f890986a700 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650c350d310 0x5650c350f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.350+0000 7f88edffb700 10 client.18417 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.350+0000 7f890986a700 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650c350d310 0x5650c350f700 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97747bfc6320737 server_cookie=41ebba2eae6aa59 in_seq=0 out_seq=0 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f88edffb700 1 -- 172.21.15.46:0/1339810144 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5650c3435700 con 0x5650c350d310 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f88edffb700 10 client.18417 handle_command_reply: tid=0 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f8918464b80 1 client.18417 shutdown 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f8918464b80 1 -- 172.21.15.46:0/1339810144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34e9a60 msgr2=0x5650c34f4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f8918464b80 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34e9a60 0x5650c34f4c50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8900004730 tx=0x7f890009abe0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f8918464b80 1 -- 172.21.15.46:0/1339810144 shutdown_connections 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f8918464b80 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650c350d310 0x5650c350f700 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f8918464b80 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34e9a60 0x5650c34f4c50 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f8918464b80 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34f5220 0x5650c34f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f88edffb700 0 client.18417 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f8918464b80 1 --2- 172.21.15.46:0/1339810144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34ea380 0x5650c34f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f8918464b80 1 -- 172.21.15.46:0/1339810144 >> 172.21.15.46:0/1339810144 conn(0x5650c3460220 msgr2=0x5650c34e5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f8918464b80 1 -- 172.21.15.46:0/1339810144 shutdown_connections 2024-03-18T20:44:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.351+0000 7f8918464b80 1 -- 172.21.15.46:0/1339810144 wait complete. 2024-03-18T20:44:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.352+0000 7f8918464b80 20 client.18417 trim_cache size 0 max 16384 2024-03-18T20:44:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.365+0000 7f8918464b80 1 Processor -- start 2024-03-18T20:44:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.365+0000 7f8918464b80 1 -- start start 2024-03-18T20:44:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.365+0000 7f8918464b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 0x5650c34f0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.365+0000 7f8918464b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34ea380 0x5650c34f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.365+0000 7f8918464b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34f5220 0x5650c34f6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.365+0000 7f8918464b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c34f2f30 con 0x5650c34f5220 2024-03-18T20:44:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.365+0000 7f8918464b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c34fba20 con 0x5650c34ea380 2024-03-18T20:44:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.365+0000 7f8918464b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c34e7230 con 0x5650c34e9a60 2024-03-18T20:44:13.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f890a86c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34f5220 0x5650c34f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f890a86c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34f5220 0x5650c34f6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56592/0 (socket says 172.21.15.46:56592) 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f890a86c700 1 -- 172.21.15.46:0/4254681895 learned_addr learned my addr 172.21.15.46:0/4254681895 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f890986a700 1 --2- 172.21.15.46:0/4254681895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34ea380 0x5650c34f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f890a06b700 1 --2- 172.21.15.46:0/4254681895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 0x5650c34f0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f890a86c700 1 -- 172.21.15.46:0/4254681895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 msgr2=0x5650c34f0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f890a86c700 1 --2- 172.21.15.46:0/4254681895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 0x5650c34f0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f890a86c700 1 -- 172.21.15.46:0/4254681895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34ea380 msgr2=0x5650c34f0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f890a86c700 1 --2- 172.21.15.46:0/4254681895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34ea380 0x5650c34f0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f890a86c700 1 -- 172.21.15.46:0/4254681895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650c3402ea0 con 0x5650c34f5220 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f890986a700 1 --2- 172.21.15.46:0/4254681895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34ea380 0x5650c34f0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f890a86c700 1 --2- 172.21.15.46:0/4254681895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34f5220 0x5650c34f6610 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009ee0 tx=0x7f88fc009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a04a50465f11cb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f88f95a9700 1 -- 172.21.15.46:0/4254681895 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc007e20 con 0x5650c34f5220 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f88f95a9700 1 -- 172.21.15.46:0/4254681895 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88fc00f040 con 0x5650c34f5220 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.366+0000 7f88f95a9700 1 -- 172.21.15.46:0/4254681895 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc005020 con 0x5650c34f5220 2024-03-18T20:44:13.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.367+0000 7f8918464b80 1 -- 172.21.15.46:0/4254681895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34f5220 msgr2=0x5650c34f6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.367+0000 7f8918464b80 1 --2- 172.21.15.46:0/4254681895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34f5220 0x5650c34f6610 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009ee0 tx=0x7f88fc009f10 comp rx=0 tx=0).stop 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.367+0000 7f8918464b80 1 -- 172.21.15.46:0/4254681895 shutdown_connections 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.367+0000 7f8918464b80 1 --2- 172.21.15.46:0/4254681895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34f5220 0x5650c34f6610 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.367+0000 7f8918464b80 1 --2- 172.21.15.46:0/4254681895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 0x5650c34f0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.367+0000 7f8918464b80 1 --2- 172.21.15.46:0/4254681895 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34ea380 0x5650c34f0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.367+0000 7f8918464b80 1 -- 172.21.15.46:0/4254681895 >> 172.21.15.46:0/4254681895 conn(0x5650c3460220 msgr2=0x5650c34ed280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.367+0000 7f8918464b80 1 -- 172.21.15.46:0/4254681895 shutdown_connections 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.367+0000 7f8918464b80 1 -- 172.21.15.46:0/4254681895 wait complete. 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.367+0000 7f8918464b80 1 -- start start 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.367+0000 7f8918464b80 1 Processor -- start 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f8918464b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 0x5650c34ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f8918464b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 0x5650c3509b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f8918464b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34f5220 0x5650c350a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f8918464b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c34f0080 con 0x5650c34ea380 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f8918464b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c350a5e0 con 0x5650c34f5220 2024-03-18T20:44:13.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f8918464b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c350a7a0 con 0x5650c34e9a60 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f8918464b80 10 client.0 _ll_register_callbacks cb 0x7f890009ad40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f890986a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 0x5650c3509b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f890986a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 0x5650c3509b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56604/0 (socket says 172.21.15.46:56604) 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f890986a700 1 -- 172.21.15.46:0/2738502233 learned_addr learned my addr 172.21.15.46:0/2738502233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f890a06b700 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 0x5650c34ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f890a86c700 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34f5220 0x5650c350a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f890986a700 1 -- 172.21.15.46:0/2738502233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 msgr2=0x5650c34ef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f890986a700 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 0x5650c34ef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f890986a700 1 -- 172.21.15.46:0/2738502233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34f5220 msgr2=0x5650c350a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f890986a700 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34f5220 0x5650c350a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f890986a700 1 -- 172.21.15.46:0/2738502233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650c32be3b0 con 0x5650c34ea380 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f890986a700 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 0x5650c3509b30 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f88f4007830 tx=0x7f88f4005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2122765e368979d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f88edffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f88edffb700 1 -- 172.21.15.46:0/2738502233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40056d0 con 0x5650c34ea380 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f88edffb700 1 -- 172.21.15.46:0/2738502233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f88f4017070 con 0x5650c34ea380 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.368+0000 7f8918464b80 1 -- 172.21.15.46:0/2738502233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88f4003b40 con 0x5650c34ea380 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.369+0000 7f88edffb700 1 -- 172.21.15.46:0/2738502233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88f4010e50 con 0x5650c34ea380 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.369+0000 7f8918464b80 10 client.18423 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.369+0000 7f8918464b80 1 -- 172.21.15.46:0/2738502233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f88fc009b90 con 0x5650c34ea380 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.369+0000 7f88edffb700 1 -- 172.21.15.46:0/2738502233 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88f4003b40 con 0x5650c34ea380 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.369+0000 7f88edffb700 1 -- 172.21.15.46:0/2738502233 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f88f4031830 con 0x5650c34ea380 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.369+0000 7f8918464b80 10 client.18423 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.369+0000 7f8918464b80 10 client.18423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.369+0000 7f8918464b80 20 client.18423 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.369+0000 7f8918464b80 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650c35623a0 0x5650c35647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.369+0000 7f8918464b80 4 client.18423 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.369+0000 7f8918464b80 1 -- 172.21.15.46:0/2738502233 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5650c328bcf0 con 0x5650c35623a0 2024-03-18T20:44:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.369+0000 7f890a06b700 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650c35623a0 0x5650c35647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:13.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f88edffb700 10 client.18423 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:13.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f890a06b700 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650c35623a0 0x5650c35647c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432018a653e55d6f server_cookie=a8c08760ccf03dae in_seq=0 out_seq=0 2024-03-18T20:44:13.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f88edffb700 1 -- 172.21.15.46:0/2738502233 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5650c328bcf0 con 0x5650c35623a0 2024-03-18T20:44:13.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f88edffb700 10 client.18423 handle_command_reply: tid=0 2024-03-18T20:44:13.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f8918464b80 1 client.18423 shutdown 2024-03-18T20:44:13.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f8918464b80 1 -- 172.21.15.46:0/2738502233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 msgr2=0x5650c3509b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f8918464b80 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 0x5650c3509b30 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f88f4007830 tx=0x7f88f4005660 comp rx=0 tx=0).stop 2024-03-18T20:44:13.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f8918464b80 1 -- 172.21.15.46:0/2738502233 shutdown_connections 2024-03-18T20:44:13.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f8918464b80 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5650c35623a0 0x5650c35647c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f8918464b80 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650c34f5220 0x5650c350a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f8918464b80 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5650c34e9a60 0x5650c34ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f88edffb700 0 client.18423 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f8918464b80 1 --2- 172.21.15.46:0/2738502233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5650c34ea380 0x5650c3509b30 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f8918464b80 1 -- 172.21.15.46:0/2738502233 >> 172.21.15.46:0/2738502233 conn(0x5650c3460220 msgr2=0x5650c34f0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f8918464b80 1 -- 172.21.15.46:0/2738502233 shutdown_connections 2024-03-18T20:44:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f8918464b80 1 -- 172.21.15.46:0/2738502233 wait complete. 2024-03-18T20:44:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.370+0000 7f8918464b80 20 client.18423 trim_cache size 0 max 16384 2024-03-18T20:44:13.379 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:44:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.371+0000 7f890c2cf700 1 -- 172.21.15.46:0/2878240851 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88f00a1290 msgr2=0x7f88f00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.371+0000 7f890c2cf700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88f00a1290 0x7f88f00a3750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8904198700 tx=0x7f88f40073d0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.371+0000 7f890c2cf700 1 -- 172.21.15.46:0/2878240851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89040ffe70 msgr2=0x7f89041971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.371+0000 7f890c2cf700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89040ffe70 0x7f89041971b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f8900004760 tx=0x7f8900004790 comp rx=0 tx=0).stop 2024-03-18T20:44:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.371+0000 7f890c2cf700 1 -- 172.21.15.46:0/2878240851 shutdown_connections 2024-03-18T20:44:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.371+0000 7f890c2cf700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f88f00a1290 0x7f88f00a3750 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.371+0000 7f890c2cf700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89040ffe70 0x7f89041971b0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.371+0000 7f890c2cf700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f890419ba90 0x7f890419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.371+0000 7f890c2cf700 1 --2- 172.21.15.46:0/2878240851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8904104f70 0x7f89041976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.371+0000 7f890c2cf700 1 -- 172.21.15.46:0/2878240851 >> 172.21.15.46:0/2878240851 conn(0x7f8904076d50 msgr2=0x7f8904069650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.371+0000 7f890c2cf700 1 -- 172.21.15.46:0/2878240851 shutdown_connections 2024-03-18T20:44:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:13.371+0000 7f890c2cf700 1 -- 172.21.15.46:0/2878240851 wait complete. 2024-03-18T20:44:13.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:13 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1492179858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:44:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:13 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1492179858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:44:13.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:13 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1492179858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:44:13.703 INFO:tasks.workunit.client.0.smithi046.stdout: 1 285476 3.56 MB/sec execute 308 sec latency 1308.907 ms 2024-03-18T20:44:14.387 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:44:14.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:44:14.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:44:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.535+0000 7f3acfc6a700 1 -- 172.21.15.46:0/4035349605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac8100800 msgr2=0x7f3ac8104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.535+0000 7f3acfc6a700 1 --2- 172.21.15.46:0/4035349605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac8100800 0x7f3ac8104830 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009990 tx=0x7f3ab8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.535+0000 7f3acfc6a700 1 -- 172.21.15.46:0/4035349605 shutdown_connections 2024-03-18T20:44:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.535+0000 7f3acfc6a700 1 --2- 172.21.15.46:0/4035349605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ac8104f60 0x7f3ac810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.535+0000 7f3acfc6a700 1 --2- 172.21.15.46:0/4035349605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac80ffe50 0x7f3ac8100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.535+0000 7f3acfc6a700 1 --2- 172.21.15.46:0/4035349605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac8100800 0x7f3ac8104830 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.535+0000 7f3acfc6a700 1 -- 172.21.15.46:0/4035349605 >> 172.21.15.46:0/4035349605 conn(0x7f3ac8076d40 msgr2=0x7f3ac8077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.535+0000 7f3acfc6a700 1 -- 172.21.15.46:0/4035349605 shutdown_connections 2024-03-18T20:44:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.535+0000 7f3acfc6a700 1 -- 172.21.15.46:0/4035349605 wait complete. 2024-03-18T20:44:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3acfc6a700 1 Processor -- start 2024-03-18T20:44:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3acfc6a700 1 -- start start 2024-03-18T20:44:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3acfc6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac80ffe50 0x7f3ac8197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3acfc6a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ac8104f60 0x7f3ac8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3acfc6a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac819baa0 0x7f3ac819bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3acfc6a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac819c520 con 0x7f3ac80ffe50 2024-03-18T20:44:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3acfc6a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac819c690 con 0x7f3ac8104f60 2024-03-18T20:44:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3acfc6a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac819c850 con 0x7f3ac819baa0 2024-03-18T20:44:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3acda06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac80ffe50 0x7f3ac8197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3acda06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac80ffe50 0x7f3ac8197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56630/0 (socket says 172.21.15.46:56630) 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3acda06700 1 -- 172.21.15.46:0/2552948108 learned_addr learned my addr 172.21.15.46:0/2552948108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3ace207700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac819baa0 0x7f3ac819bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.536+0000 7f3acd205700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ac8104f60 0x7f3ac8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.537+0000 7f3acda06700 1 -- 172.21.15.46:0/2552948108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac819baa0 msgr2=0x7f3ac819bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.537+0000 7f3acda06700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac819baa0 0x7f3ac819bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.537+0000 7f3acda06700 1 -- 172.21.15.46:0/2552948108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ac8104f60 msgr2=0x7f3ac8197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.537+0000 7f3acda06700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ac8104f60 0x7f3ac8197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.537+0000 7f3acda06700 1 -- 172.21.15.46:0/2552948108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80095f0 con 0x7f3ac80ffe50 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.537+0000 7f3acda06700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac80ffe50 0x7f3ac8197210 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400e9f0 tx=0x7f3ac400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.537+0000 7f3abeffd700 1 -- 172.21.15.46:0/2552948108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400eee0 con 0x7f3ac80ffe50 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.537+0000 7f3abeffd700 1 -- 172.21.15.46:0/2552948108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ac4004cf0 con 0x7f3ac80ffe50 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.537+0000 7f3acfc6a700 1 -- 172.21.15.46:0/2552948108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac81a3db0 con 0x7f3ac80ffe50 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.537+0000 7f3abeffd700 1 -- 172.21.15.46:0/2552948108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4005540 con 0x7f3ac80ffe50 2024-03-18T20:44:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.537+0000 7f3acfc6a700 1 -- 172.21.15.46:0/2552948108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac81a4230 con 0x7f3ac80ffe50 2024-03-18T20:44:14.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:14 smithi155 ceph-mon[48344]: pgmap v251: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-18T20:44:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.539+0000 7f3abeffd700 1 -- 172.21.15.46:0/2552948108 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3ac400cbd0 con 0x7f3ac80ffe50 2024-03-18T20:44:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.540+0000 7f3abeffd700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ab40a1310 0x7f3ab40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.540+0000 7f3abeffd700 1 -- 172.21.15.46:0/2552948108 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3ac4014070 con 0x7f3ac80ffe50 2024-03-18T20:44:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.540+0000 7f3acd205700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ab40a1310 0x7f3ab40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3acd205700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ab40a1310 0x7f3ab40a37d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8006010 tx=0x7f3ab800b260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3adbdffb80 1 Processor -- start 2024-03-18T20:44:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3adbdffb80 1 -- start start 2024-03-18T20:44:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3adbdffb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078cdae0 0x564d078cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3adbdffb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078ce400 0x564d078dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3adbdffb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 0x564d078dfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3adbdffb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d07827a10 con 0x564d078dd6b0 2024-03-18T20:44:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3adbdffb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d078253d0 con 0x564d078ce400 2024-03-18T20:44:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3adbdffb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d07820cc0 con 0x564d078cdae0 2024-03-18T20:44:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3ace207700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 0x564d078dfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3ace207700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 0x564d078dfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56636/0 (socket says 172.21.15.46:56636) 2024-03-18T20:44:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3ace207700 1 -- 172.21.15.46:0/3741196897 learned_addr learned my addr 172.21.15.46:0/3741196897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.541+0000 7f3acda06700 1 --2- 172.21.15.46:0/3741196897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078cdae0 0x564d078cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.542+0000 7f3acd205700 1 --2- 172.21.15.46:0/3741196897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078ce400 0x564d078dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.542+0000 7f3ace207700 1 -- 172.21.15.46:0/3741196897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078cdae0 msgr2=0x564d078cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.542+0000 7f3ace207700 1 --2- 172.21.15.46:0/3741196897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078cdae0 0x564d078cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.542+0000 7f3ace207700 1 -- 172.21.15.46:0/3741196897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078ce400 msgr2=0x564d078dd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.542+0000 7f3ace207700 1 --2- 172.21.15.46:0/3741196897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078ce400 0x564d078dd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.542+0000 7f3ace207700 1 -- 172.21.15.46:0/3741196897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d076a2430 con 0x564d078dd6b0 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.542+0000 7f3ace207700 1 --2- 172.21.15.46:0/3741196897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 0x564d078dfaa0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0003fa0 tx=0x7f3ac0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570a50c9f0541845 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.542+0000 7f3ab3fff700 1 -- 172.21.15.46:0/3741196897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0019070 con 0x564d078dd6b0 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.542+0000 7f3ab3fff700 1 -- 172.21.15.46:0/3741196897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ac000f040 con 0x564d078dd6b0 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.542+0000 7f3ab3fff700 1 -- 172.21.15.46:0/3741196897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00079a0 con 0x564d078dd6b0 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.543+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3741196897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 msgr2=0x564d078dfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.543+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3741196897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 0x564d078dfaa0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0003fa0 tx=0x7f3ac0009ba0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.543+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3741196897 shutdown_connections 2024-03-18T20:44:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.543+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3741196897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 0x564d078dfaa0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.543+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3741196897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078cdae0 0x564d078cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.543+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3741196897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078ce400 0x564d078dd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.543+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3741196897 >> 172.21.15.46:0/3741196897 conn(0x564d078442a0 msgr2=0x564d078ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:14.549 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:14 smithi046 ceph-mon[43495]: pgmap v251: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.543+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3741196897 shutdown_connections 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.543+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3741196897 wait complete. 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.544+0000 7f3adbdffb80 1 -- start start 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.544+0000 7f3adbdffb80 1 Processor -- start 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.544+0000 7f3adbdffb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 0x564d078dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.544+0000 7f3adbdffb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078ce400 0x564d078da370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.544+0000 7f3adbdffb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078dd6b0 0x564d078da9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.544+0000 7f3adbdffb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d077bce00 con 0x564d078cdae0 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.544+0000 7f3adbdffb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d077bcaa0 con 0x564d078dd6b0 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3adbdffb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d076a4720 con 0x564d078ce400 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3adbdffb80 10 client.0 _ll_register_callbacks cb 0x564d07628420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3acda06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 0x564d078dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3acda06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 0x564d078dd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56640/0 (socket says 172.21.15.46:56640) 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3acda06700 1 -- 172.21.15.46:0/3619540041 learned_addr learned my addr 172.21.15.46:0/3619540041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3acd205700 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078ce400 0x564d078da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3ace207700 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078dd6b0 0x564d078da9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3acda06700 1 -- 172.21.15.46:0/3619540041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078ce400 msgr2=0x564d078da370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3acda06700 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078ce400 0x564d078da370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3acda06700 1 -- 172.21.15.46:0/3619540041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078dd6b0 msgr2=0x564d078da9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3acda06700 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078dd6b0 0x564d078da9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3acda06700 1 -- 172.21.15.46:0/3619540041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac0003bd0 con 0x564d078cdae0 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3acda06700 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 0x564d078dd080 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400c5b0 tx=0x7f3ac409a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc95de6034217e9f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3ab17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3ab17fa700 1 -- 172.21.15.46:0/3619540041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40a24f0 con 0x564d078cdae0 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3ab17fa700 1 -- 172.21.15.46:0/3619540041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3ac40b8270 con 0x564d078cdae0 2024-03-18T20:44:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.545+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3619540041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d078c1b50 con 0x564d078cdae0 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.546+0000 7f3ab17fa700 1 -- 172.21.15.46:0/3619540041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ac40b8430 con 0x564d078cdae0 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.546+0000 7f3adbdffb80 10 client.18435 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.546+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3619540041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x564d0764ebe0 con 0x564d078cdae0 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.546+0000 7f3ab17fa700 1 -- 172.21.15.46:0/3619540041 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ac4082030 con 0x564d078cdae0 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.546+0000 7f3ab17fa700 1 -- 172.21.15.46:0/3619540041 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ac40a2060 con 0x564d078cdae0 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.546+0000 7f3adbdffb80 10 client.18435 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.546+0000 7f3adbdffb80 10 client.18435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.546+0000 7f3adbdffb80 20 client.18435 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.546+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564d078ed370 0x564d078ef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.546+0000 7f3adbdffb80 4 client.18435 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.546+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3619540041 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d07819780 con 0x564d078ed370 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.546+0000 7f3acd205700 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564d078ed370 0x564d078ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.547+0000 7f3acd205700 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564d078ed370 0x564d078ef760 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9400a5e99d18d55a server_cookie=1e8456de0ab558c6 in_seq=0 out_seq=0 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.547+0000 7f3ab17fa700 10 client.18435 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.547+0000 7f3ab17fa700 1 -- 172.21.15.46:0/3619540041 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564d07819780 con 0x564d078ed370 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.547+0000 7f3ab17fa700 10 client.18435 handle_command_reply: tid=0 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.547+0000 7f3adbdffb80 1 client.18435 shutdown 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.548+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3619540041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 msgr2=0x564d078dd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.548+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 0x564d078dd080 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400c5b0 tx=0x7f3ac409a140 comp rx=0 tx=0).stop 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.548+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3619540041 shutdown_connections 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.548+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564d078ed370 0x564d078ef760 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.548+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 0x564d078dd080 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.548+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078dd6b0 0x564d078da9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.548+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3619540041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078ce400 0x564d078da370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.548+0000 7f3ab17fa700 0 client.18435 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.548+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3619540041 >> 172.21.15.46:0/3619540041 conn(0x564d078442a0 msgr2=0x564d078c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.548+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3619540041 shutdown_connections 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.548+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3619540041 wait complete. 2024-03-18T20:44:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.548+0000 7f3adbdffb80 20 client.18435 trim_cache size 0 max 16384 2024-03-18T20:44:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.566+0000 7f3adbdffb80 1 Processor -- start 2024-03-18T20:44:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.566+0000 7f3adbdffb80 1 -- start start 2024-03-18T20:44:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.566+0000 7f3adbdffb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 0x564d078e4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.566+0000 7f3adbdffb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078ce400 0x564d078e8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.566+0000 7f3adbdffb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078dd6b0 0x564d078e8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.566+0000 7f3adbdffb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d078ed150 con 0x564d078cdae0 2024-03-18T20:44:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.566+0000 7f3adbdffb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d078c4000 con 0x564d078dd6b0 2024-03-18T20:44:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.566+0000 7f3adbdffb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d078db2d0 con 0x564d078ce400 2024-03-18T20:44:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.566+0000 7f3acda06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 0x564d078e4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.566+0000 7f3acda06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 0x564d078e4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56646/0 (socket says 172.21.15.46:56646) 2024-03-18T20:44:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.566+0000 7f3acda06700 1 -- 172.21.15.46:0/1788394752 learned_addr learned my addr 172.21.15.46:0/1788394752 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3ace207700 1 --2- 172.21.15.46:0/1788394752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078dd6b0 0x564d078e8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3acd205700 1 --2- 172.21.15.46:0/1788394752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078ce400 0x564d078e8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3acda06700 1 -- 172.21.15.46:0/1788394752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078ce400 msgr2=0x564d078e8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3acda06700 1 --2- 172.21.15.46:0/1788394752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078ce400 0x564d078e8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3acda06700 1 -- 172.21.15.46:0/1788394752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078dd6b0 msgr2=0x564d078e8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3acda06700 1 --2- 172.21.15.46:0/1788394752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078dd6b0 0x564d078e8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3acda06700 1 -- 172.21.15.46:0/1788394752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d077e6f20 con 0x564d078cdae0 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3acd205700 1 --2- 172.21.15.46:0/1788394752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078ce400 0x564d078e8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3ace207700 1 --2- 172.21.15.46:0/1788394752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078dd6b0 0x564d078e8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3acda06700 1 --2- 172.21.15.46:0/1788394752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 0x564d078e4300 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40cacb0 tx=0x7f3ac4009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2aeb1466b4c41a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3abcda8700 1 -- 172.21.15.46:0/1788394752 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40a24f0 con 0x564d078cdae0 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3abcda8700 1 -- 172.21.15.46:0/1788394752 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ac4082070 con 0x564d078cdae0 2024-03-18T20:44:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3abcda8700 1 -- 172.21.15.46:0/1788394752 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40b8270 con 0x564d078cdae0 2024-03-18T20:44:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.567+0000 7f3adbdffb80 1 -- 172.21.15.46:0/1788394752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 msgr2=0x564d078e4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/1788394752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 0x564d078e4300 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40cacb0 tx=0x7f3ac4009ef0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 -- 172.21.15.46:0/1788394752 shutdown_connections 2024-03-18T20:44:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/1788394752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078cdae0 0x564d078e4300 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/1788394752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078dd6b0 0x564d078e8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/1788394752 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078ce400 0x564d078e8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 -- 172.21.15.46:0/1788394752 >> 172.21.15.46:0/1788394752 conn(0x564d078442a0 msgr2=0x564d078d8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 -- 172.21.15.46:0/1788394752 shutdown_connections 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 -- 172.21.15.46:0/1788394752 wait complete. 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 -- start start 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 Processor -- start 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078cdae0 0x564d078e0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078ce400 0x564d078e1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 0x564d078e1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.568+0000 7f3adbdffb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d07941250 con 0x564d078dd6b0 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3adbdffb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d07941390 con 0x564d078ce400 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3adbdffb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d07941550 con 0x564d078cdae0 2024-03-18T20:44:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3adbdffb80 10 client.0 _ll_register_callbacks cb 0x7f3ac4099a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3ace207700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 0x564d078e1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3ace207700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 0x564d078e1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56648/0 (socket says 172.21.15.46:56648) 2024-03-18T20:44:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3ace207700 1 -- 172.21.15.46:0/3237609258 learned_addr learned my addr 172.21.15.46:0/3237609258 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3acda06700 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078cdae0 0x564d078e0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3acd205700 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078ce400 0x564d078e1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3ace207700 1 -- 172.21.15.46:0/3237609258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078cdae0 msgr2=0x564d078e0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3ace207700 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078cdae0 0x564d078e0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3ace207700 1 -- 172.21.15.46:0/3237609258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078ce400 msgr2=0x564d078e1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3ace207700 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078ce400 0x564d078e1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3ace207700 1 -- 172.21.15.46:0/3237609258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d0764ebe0 con 0x564d078dd6b0 2024-03-18T20:44:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3acda06700 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078cdae0 0x564d078e0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:44:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3acd205700 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078ce400 0x564d078e1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3ace207700 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 0x564d078e1650 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0003ec0 tx=0x7f3ac0004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214ce565799da3a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3ab17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3ab17fa700 1 -- 172.21.15.46:0/3237609258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0019040 con 0x564d078dd6b0 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3ab17fa700 1 -- 172.21.15.46:0/3237609258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3ac0007980 con 0x564d078dd6b0 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.569+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3237609258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ac000f040 con 0x564d078dd6b0 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.570+0000 7f3ab17fa700 1 -- 172.21.15.46:0/3237609258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ac00094e0 con 0x564d078dd6b0 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.570+0000 7f3adbdffb80 10 client.18441 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.570+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3237609258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x564d076a6d50 con 0x564d078dd6b0 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.570+0000 7f3ab17fa700 1 -- 172.21.15.46:0/3237609258 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ac000f040 con 0x564d078dd6b0 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.570+0000 7f3ab17fa700 1 -- 172.21.15.46:0/3237609258 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ac0015c60 con 0x564d078dd6b0 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.570+0000 7f3adbdffb80 10 client.18441 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.570+0000 7f3adbdffb80 10 client.18441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.570+0000 7f3adbdffb80 20 client.18441 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.570+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564d07942ca0 0x564d079450c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.570+0000 7f3adbdffb80 4 client.18441 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.570+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3237609258 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564d0766fd70 con 0x564d07942ca0 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.570+0000 7f3acda06700 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564d07942ca0 0x564d079450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.571+0000 7f3ab17fa700 10 client.18441 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.571+0000 7f3acda06700 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564d07942ca0 0x564d079450c0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297481e61a26ec4e server_cookie=1903829786d7a19a in_seq=0 out_seq=0 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3ab17fa700 1 -- 172.21.15.46:0/3237609258 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564d0766fd70 con 0x564d07942ca0 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3ab17fa700 10 client.18441 handle_command_reply: tid=0 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3adbdffb80 1 client.18441 shutdown 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3237609258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 msgr2=0x564d078e1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 0x564d078e1650 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0003ec0 tx=0x7f3ac0004cc0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3237609258 shutdown_connections 2024-03-18T20:44:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564d07942ca0 0x564d079450c0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564d078dd6b0 0x564d078e1650 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564d078cdae0 0x564d078e0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3ab17fa700 0 client.18441 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3adbdffb80 1 --2- 172.21.15.46:0/3237609258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564d078ce400 0x564d078e1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3237609258 >> 172.21.15.46:0/3237609258 conn(0x564d078442a0 msgr2=0x564d078e8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3237609258 shutdown_connections 2024-03-18T20:44:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.572+0000 7f3adbdffb80 1 -- 172.21.15.46:0/3237609258 wait complete. 2024-03-18T20:44:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3adbdffb80 20 client.18441 trim_cache size 0 max 16384 2024-03-18T20:44:14.580 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:44:14.580 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:44:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "fb01c11a-ae20-4cb1-9609-907e25722bb7", 2024-03-18T20:44:14.581 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:44:14.581 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:44:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3acfc6a700 1 -- 172.21.15.46:0/2552948108 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ab40a1310 msgr2=0x7f3ab40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3acfc6a700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ab40a1310 0x7f3ab40a37d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8006010 tx=0x7f3ab800b260 comp rx=0 tx=0).stop 2024-03-18T20:44:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3acfc6a700 1 -- 172.21.15.46:0/2552948108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac80ffe50 msgr2=0x7f3ac8197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3acfc6a700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac80ffe50 0x7f3ac8197210 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400e9f0 tx=0x7f3ac400ed00 comp rx=0 tx=0).stop 2024-03-18T20:44:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3acfc6a700 1 -- 172.21.15.46:0/2552948108 shutdown_connections 2024-03-18T20:44:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3acfc6a700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ab40a1310 0x7f3ab40a37d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3acfc6a700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ac80ffe50 0x7f3ac8197210 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3acfc6a700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac819baa0 0x7f3ac819bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3acfc6a700 1 --2- 172.21.15.46:0/2552948108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ac8104f60 0x7f3ac8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3acfc6a700 1 -- 172.21.15.46:0/2552948108 >> 172.21.15.46:0/2552948108 conn(0x7f3ac8076d40 msgr2=0x7f3ac80681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3acfc6a700 1 -- 172.21.15.46:0/2552948108 shutdown_connections 2024-03-18T20:44:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.573+0000 7f3acfc6a700 1 -- 172.21.15.46:0/2552948108 wait complete. 2024-03-18T20:44:14.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:44:14.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:44:14.704 INFO:tasks.workunit.client.0.smithi046.stdout: 1 285703 3.54 MB/sec execute 309 sec latency 2211.175 ms 2024-03-18T20:44:14.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:14 smithi073 ceph-mon[47384]: pgmap v251: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-18T20:44:14.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.736+0000 7ffa072fa700 1 -- 172.21.15.46:0/1028414204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00103170 msgr2=0x7ffa00103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.736+0000 7ffa072fa700 1 --2- 172.21.15.46:0/1028414204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00103170 0x7ffa00103550 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00b0a0 tx=0x7ff9fc00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.736+0000 7ffa072fa700 1 -- 172.21.15.46:0/1028414204 shutdown_connections 2024-03-18T20:44:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.737+0000 7ffa072fa700 1 --2- 172.21.15.46:0/1028414204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00103170 0x7ffa00103550 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.737+0000 7ffa072fa700 1 --2- 172.21.15.46:0/1028414204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa00108290 0x7ffa0010fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.737+0000 7ffa072fa700 1 --2- 172.21.15.46:0/1028414204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa00103b20 0x7ffa00107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.737+0000 7ffa072fa700 1 -- 172.21.15.46:0/1028414204 >> 172.21.15.46:0/1028414204 conn(0x7ffa000fe990 msgr2=0x7ffa00100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.737+0000 7ffa072fa700 1 -- 172.21.15.46:0/1028414204 shutdown_connections 2024-03-18T20:44:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.737+0000 7ffa072fa700 1 -- 172.21.15.46:0/1028414204 wait complete. 2024-03-18T20:44:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.737+0000 7ffa072fa700 1 Processor -- start 2024-03-18T20:44:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.737+0000 7ffa072fa700 1 -- start start 2024-03-18T20:44:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.737+0000 7ffa072fa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa00103170 0x7ffa00199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.737+0000 7ffa072fa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00108290 0x7ffa00199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa072fa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0019de70 0x7ffa0019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa072fa700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0019e8f0 con 0x7ffa00108290 2024-03-18T20:44:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa072fa700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0019ea60 con 0x7ffa00103170 2024-03-18T20:44:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa072fa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0019ec20 con 0x7ffa0019de70 2024-03-18T20:44:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa04895700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00108290 0x7ffa00199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa04895700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00108290 0x7ffa00199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56668/0 (socket says 172.21.15.46:56668) 2024-03-18T20:44:14.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa04895700 1 -- 172.21.15.46:0/2974037484 learned_addr learned my addr 172.21.15.46:0/2974037484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa05096700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa00103170 0x7ffa00199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa05897700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0019de70 0x7ffa0019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa04895700 1 -- 172.21.15.46:0/2974037484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0019de70 msgr2=0x7ffa0019e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa04895700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0019de70 0x7ffa0019e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa04895700 1 -- 172.21.15.46:0/2974037484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa00103170 msgr2=0x7ffa00199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa04895700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa00103170 0x7ffa00199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa04895700 1 -- 172.21.15.46:0/2974037484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc009d00 con 0x7ffa00108290 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa05096700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa00103170 0x7ffa00199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa05897700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0019de70 0x7ffa0019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa04895700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00108290 0x7ffa00199ad0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000e970 tx=0x7ff9f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ff9f67fc700 1 -- 172.21.15.46:0/2974037484 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0009790 con 0x7ffa00108290 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa072fa700 1 -- 172.21.15.46:0/2974037484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa001a6180 con 0x7ffa00108290 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ff9f67fc700 1 -- 172.21.15.46:0/2974037484 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9f000eec0 con 0x7ffa00108290 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ff9f67fc700 1 -- 172.21.15.46:0/2974037484 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0005590 con 0x7ffa00108290 2024-03-18T20:44:14.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.738+0000 7ffa072fa700 1 -- 172.21.15.46:0/2974037484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa001a6660 con 0x7ffa00108290 2024-03-18T20:44:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.741+0000 7ff9f67fc700 1 -- 172.21.15.46:0/2974037484 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff9f000cbd0 con 0x7ffa00108290 2024-03-18T20:44:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.741+0000 7ff9f67fc700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9ec0a1290 0x7ff9ec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.742+0000 7ff9f67fc700 1 -- 172.21.15.46:0/2974037484 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff9f0014070 con 0x7ffa00108290 2024-03-18T20:44:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.742+0000 7ffa05096700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9ec0a1290 0x7ff9ec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.742+0000 7ffa05096700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9ec0a1290 0x7ff9ec0a3750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000f80 tx=0x7ff9fc00b740 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa1348fb80 1 Processor -- start 2024-03-18T20:44:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa1348fb80 1 -- start start 2024-03-18T20:44:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa1348fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e589be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa1348fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e589c380 0x5577e58a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa1348fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e58a7220 0x5577e58a9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa1348fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e57f5990 con 0x5577e58a7220 2024-03-18T20:44:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa1348fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e57f3350 con 0x5577e589ba60 2024-03-18T20:44:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa1348fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e57eec40 con 0x5577e589c380 2024-03-18T20:44:14.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa05096700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e589be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa05096700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e589be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56004/0 (socket says 172.21.15.46:56004) 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa05096700 1 -- 172.21.15.46:0/4190887341 learned_addr learned my addr 172.21.15.46:0/4190887341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa04895700 1 --2- 172.21.15.46:0/4190887341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e589c380 0x5577e58a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa05897700 1 --2- 172.21.15.46:0/4190887341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e58a7220 0x5577e58a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa05096700 1 -- 172.21.15.46:0/4190887341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e589c380 msgr2=0x5577e58a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa05096700 1 --2- 172.21.15.46:0/4190887341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e589c380 0x5577e58a6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa05096700 1 -- 172.21.15.46:0/4190887341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e58a7220 msgr2=0x5577e58a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.743+0000 7ffa05096700 1 --2- 172.21.15.46:0/4190887341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e58a7220 0x5577e58a9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.744+0000 7ffa05096700 1 -- 172.21.15.46:0/4190887341 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577e56703b0 con 0x5577e589ba60 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.744+0000 7ffa05096700 1 --2- 172.21.15.46:0/4190887341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e589be40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc007bc0 tx=0x7ff9fc01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fe359beeea70d59 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.744+0000 7ff9eb7fe700 1 -- 172.21.15.46:0/4190887341 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc023040 con 0x5577e589ba60 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.744+0000 7ff9eb7fe700 1 -- 172.21.15.46:0/4190887341 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9fc003c00 con 0x5577e589ba60 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.744+0000 7ff9eb7fe700 1 -- 172.21.15.46:0/4190887341 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00e040 con 0x5577e589ba60 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.745+0000 7ffa1348fb80 1 -- 172.21.15.46:0/4190887341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 msgr2=0x5577e589be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.745+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/4190887341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e589be40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc007bc0 tx=0x7ff9fc01c000 comp rx=0 tx=0).stop 2024-03-18T20:44:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.745+0000 7ffa1348fb80 1 -- 172.21.15.46:0/4190887341 shutdown_connections 2024-03-18T20:44:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.745+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/4190887341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e58a7220 0x5577e58a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.745+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/4190887341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e589be40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.745+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/4190887341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e589c380 0x5577e58a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.745+0000 7ffa1348fb80 1 -- 172.21.15.46:0/4190887341 >> 172.21.15.46:0/4190887341 conn(0x5577e5812220 msgr2=0x5577e5898020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.745+0000 7ffa1348fb80 1 -- 172.21.15.46:0/4190887341 shutdown_connections 2024-03-18T20:44:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.745+0000 7ffa1348fb80 1 -- 172.21.15.46:0/4190887341 wait complete. 2024-03-18T20:44:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.745+0000 7ffa1348fb80 1 -- start start 2024-03-18T20:44:14.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.745+0000 7ffa1348fb80 1 Processor -- start 2024-03-18T20:44:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa1348fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e58a4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa1348fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 0x5577e58a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa1348fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e58a7220 0x5577e58a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa1348fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e578ad80 con 0x5577e589c380 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa1348fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e578aa20 con 0x5577e589ba60 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa1348fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e56726a0 con 0x5577e58a7220 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa1348fb80 10 client.0 _ll_register_callbacks cb 0x5577e55f63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa04895700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 0x5577e58a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa05096700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e58a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa04895700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 0x5577e58a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56686/0 (socket says 172.21.15.46:56686) 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa05096700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e58a4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56010/0 (socket says 172.21.15.46:56010) 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa04895700 1 -- 172.21.15.46:0/2722434902 learned_addr learned my addr 172.21.15.46:0/2722434902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.746+0000 7ffa05897700 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e58a7220 0x5577e58a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ffa05096700 1 -- 172.21.15.46:0/2722434902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e58a7220 msgr2=0x5577e58a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ffa05096700 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e58a7220 0x5577e58a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ffa05096700 1 -- 172.21.15.46:0/2722434902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 msgr2=0x5577e58a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ffa05096700 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 0x5577e58a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ffa05096700 1 -- 172.21.15.46:0/2722434902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa00110680 con 0x5577e589ba60 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ffa04895700 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 0x5577e58a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ffa05096700 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e58a4070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00b0a0 tx=0x7ff9fc007ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85659dc00f1b2de0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ff9e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ff9e8ff9700 1 -- 172.21.15.46:0/2722434902 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00e070 con 0x5577e589ba60 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ffa1348fb80 1 -- 172.21.15.46:0/2722434902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577e588fad0 con 0x5577e589ba60 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ff9e8ff9700 1 -- 172.21.15.46:0/2722434902 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff9fc01c650 con 0x5577e589ba60 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ff9e8ff9700 1 -- 172.21.15.46:0/2722434902 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9fc01c810 con 0x5577e589ba60 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ffa1348fb80 10 client.28103 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.747+0000 7ffa1348fb80 1 -- 172.21.15.46:0/2722434902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5577e561cb60 con 0x5577e589ba60 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.748+0000 7ff9e8ff9700 1 -- 172.21.15.46:0/2722434902 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577e588fad0 con 0x5577e589ba60 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.748+0000 7ff9e8ff9700 1 -- 172.21.15.46:0/2722434902 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9fc0055d0 con 0x5577e589ba60 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.748+0000 7ffa1348fb80 10 client.28103 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.748+0000 7ffa1348fb80 10 client.28103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.748+0000 7ffa1348fb80 20 client.28103 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:14.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.748+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577e58b6940 0x5577e58b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.748+0000 7ffa1348fb80 4 client.28103 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.748+0000 7ffa1348fb80 1 -- 172.21.15.46:0/2722434902 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577e57e7700 con 0x5577e58b6940 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.748+0000 7ffa04895700 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577e58b6940 0x5577e58b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.748+0000 7ff9e8ff9700 10 client.28103 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.748+0000 7ffa04895700 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577e58b6940 0x5577e58b8d30 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc9ce4a36e83dd1 server_cookie=3d465994362da494 in_seq=0 out_seq=0 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ff9e8ff9700 1 -- 172.21.15.46:0/2722434902 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5577e57e7700 con 0x5577e58b6940 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ff9e8ff9700 10 client.28103 handle_command_reply: tid=0 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ffa1348fb80 1 client.28103 shutdown 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ffa1348fb80 1 -- 172.21.15.46:0/2722434902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 msgr2=0x5577e58a4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e58a4070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00b0a0 tx=0x7ff9fc007ed0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ffa1348fb80 1 -- 172.21.15.46:0/2722434902 shutdown_connections 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577e58b6940 0x5577e58b8d30 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e58a7220 0x5577e58a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e58a4070 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/2722434902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 0x5577e58a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ffa1348fb80 1 -- 172.21.15.46:0/2722434902 >> 172.21.15.46:0/2722434902 conn(0x5577e5812220 msgr2=0x5577e5897c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:14.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ff9e8ff9700 0 client.28103 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:14.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.749+0000 7ffa1348fb80 1 -- 172.21.15.46:0/2722434902 shutdown_connections 2024-03-18T20:44:14.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.750+0000 7ffa1348fb80 1 -- 172.21.15.46:0/2722434902 wait complete. 2024-03-18T20:44:14.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.750+0000 7ffa1348fb80 20 client.28103 trim_cache size 0 max 16384 2024-03-18T20:44:14.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.768+0000 7ffa1348fb80 1 Processor -- start 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.768+0000 7ffa1348fb80 1 -- start start 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa1348fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e58addf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa1348fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 0x5577e58a4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa1348fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e58a7220 0x5577e58b8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa1348fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e589cd30 con 0x5577e589c380 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa1348fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e58945e0 con 0x5577e589ba60 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa1348fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e589d610 con 0x5577e58a7220 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa04895700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 0x5577e58a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa04895700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 0x5577e58a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56690/0 (socket says 172.21.15.46:56690) 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa05096700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e58addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa04895700 1 -- 172.21.15.46:0/1652495254 learned_addr learned my addr 172.21.15.46:0/1652495254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa05897700 1 --2- 172.21.15.46:0/1652495254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e58a7220 0x5577e58b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa04895700 1 -- 172.21.15.46:0/1652495254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e58a7220 msgr2=0x5577e58b8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa04895700 1 --2- 172.21.15.46:0/1652495254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e58a7220 0x5577e58b8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa04895700 1 -- 172.21.15.46:0/1652495254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 msgr2=0x5577e58addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa04895700 1 --2- 172.21.15.46:0/1652495254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e58addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa04895700 1 -- 172.21.15.46:0/1652495254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577e57b4ea0 con 0x5577e589c380 2024-03-18T20:44:14.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ffa04895700 1 --2- 172.21.15.46:0/1652495254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 0x5577e58a4910 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c5b0 tx=0x7ff9f00cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b00fed9d0b49cf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:14.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ff9ebfff700 1 -- 172.21.15.46:0/1652495254 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0063070 con 0x5577e589c380 2024-03-18T20:44:14.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ff9ebfff700 1 -- 172.21.15.46:0/1652495254 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9f00b6070 con 0x5577e589c380 2024-03-18T20:44:14.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.769+0000 7ff9ebfff700 1 -- 172.21.15.46:0/1652495254 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0016330 con 0x5577e589c380 2024-03-18T20:44:14.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.770+0000 7ffa1348fb80 1 -- 172.21.15.46:0/1652495254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 msgr2=0x5577e58a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.770+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/1652495254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 0x5577e58a4910 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c5b0 tx=0x7ff9f00cc040 comp rx=0 tx=0).stop 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.770+0000 7ffa1348fb80 1 -- 172.21.15.46:0/1652495254 shutdown_connections 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.770+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/1652495254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e58a7220 0x5577e58b8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.770+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/1652495254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589ba60 0x5577e58addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.770+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/1652495254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e589c380 0x5577e58a4910 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.770+0000 7ffa1348fb80 1 -- 172.21.15.46:0/1652495254 >> 172.21.15.46:0/1652495254 conn(0x5577e5812220 msgr2=0x5577e589f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.770+0000 7ffa1348fb80 1 -- 172.21.15.46:0/1652495254 shutdown_connections 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.770+0000 7ffa1348fb80 1 -- 172.21.15.46:0/1652495254 wait complete. 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.771+0000 7ffa1348fb80 1 -- start start 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.771+0000 7ffa1348fb80 1 Processor -- start 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.771+0000 7ffa1348fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e589ba60 0x5577e58ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.771+0000 7ffa1348fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589c380 0x5577e58abc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.771+0000 7ffa1348fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e58a7220 0x5577e58ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.771+0000 7ffa1348fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e5909750 con 0x5577e58a7220 2024-03-18T20:44:14.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.771+0000 7ffa1348fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e59098c0 con 0x5577e589c380 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.771+0000 7ffa1348fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e5909a80 con 0x5577e589ba60 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.771+0000 7ffa1348fb80 10 client.0 _ll_register_callbacks cb 0x7ff9f8009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa04895700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589c380 0x5577e58abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa05096700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e589ba60 0x5577e58ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa04895700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589c380 0x5577e58abc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56016/0 (socket says 172.21.15.46:56016) 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa04895700 1 -- 172.21.15.46:0/3296808351 learned_addr learned my addr 172.21.15.46:0/3296808351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa05897700 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e58a7220 0x5577e58ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa04895700 1 -- 172.21.15.46:0/3296808351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e589ba60 msgr2=0x5577e58ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa04895700 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e589ba60 0x5577e58ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa04895700 1 -- 172.21.15.46:0/3296808351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e58a7220 msgr2=0x5577e58ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa04895700 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e58a7220 0x5577e58ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa04895700 1 -- 172.21.15.46:0/3296808351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa001a6660 con 0x5577e589c380 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa05096700 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e589ba60 0x5577e58ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:14.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa04895700 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589c380 0x5577e58abc50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00ca710 tx=0x7ff9f00a24f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4147a170065243ef server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ff9e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ff9e8ff9700 1 -- 172.21.15.46:0/3296808351 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0063050 con 0x5577e589c380 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ff9e8ff9700 1 -- 172.21.15.46:0/3296808351 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff9f00b8210 con 0x5577e589c380 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa1348fb80 1 -- 172.21.15.46:0/3296808351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9fc003c00 con 0x5577e589c380 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ff9e8ff9700 1 -- 172.21.15.46:0/3296808351 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9f00b83d0 con 0x5577e589c380 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.772+0000 7ffa1348fb80 10 client.28109 fetch_fsmap learned FSMap version 46 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.773+0000 7ffa1348fb80 1 -- 172.21.15.46:0/3296808351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5577e5674cd0 con 0x5577e589c380 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.773+0000 7ff9e8ff9700 1 -- 172.21.15.46:0/3296808351 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9fc003c00 con 0x5577e589c380 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.773+0000 7ff9e8ff9700 1 -- 172.21.15.46:0/3296808351 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 46) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9f00995f0 con 0x5577e589c380 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.773+0000 7ffa1348fb80 10 client.28109 fetch_fsmap finished waiting for FSMap version 46 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.773+0000 7ffa1348fb80 10 client.28109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.773+0000 7ffa1348fb80 20 client.28109 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.773+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577e590af10 0x5577e590d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.773+0000 7ffa1348fb80 4 client.28109 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:44:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.773+0000 7ffa1348fb80 1 -- 172.21.15.46:0/3296808351 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577e563dcf0 con 0x5577e590af10 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.773+0000 7ffa05096700 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577e590af10 0x5577e590d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.773+0000 7ff9e8ff9700 10 client.28109 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.773+0000 7ffa05096700 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577e590af10 0x5577e590d330 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59bf64d261ec3f01 server_cookie=bcfd9236e32137f2 in_seq=0 out_seq=0 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ff9e8ff9700 1 -- 172.21.15.46:0/3296808351 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5577e563dcf0 con 0x5577e590af10 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ff9e8ff9700 10 client.28109 handle_command_reply: tid=0 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ffa1348fb80 1 client.28109 shutdown 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ffa1348fb80 1 -- 172.21.15.46:0/3296808351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589c380 msgr2=0x5577e58abc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589c380 0x5577e58abc50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00ca710 tx=0x7ff9f00a24f0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ffa1348fb80 1 -- 172.21.15.46:0/3296808351 shutdown_connections 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5577e590af10 0x5577e590d330 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577e58a7220 0x5577e58ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5577e589ba60 0x5577e58ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ffa1348fb80 1 --2- 172.21.15.46:0/3296808351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e589c380 0x5577e58abc50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ff9e8ff9700 0 client.28109 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:14.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ffa1348fb80 1 -- 172.21.15.46:0/3296808351 >> 172.21.15.46:0/3296808351 conn(0x5577e5812220 msgr2=0x5577e58b8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ffa1348fb80 1 -- 172.21.15.46:0/3296808351 shutdown_connections 2024-03-18T20:44:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ffa1348fb80 1 -- 172.21.15.46:0/3296808351 wait complete. 2024-03-18T20:44:14.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.774+0000 7ffa1348fb80 20 client.28109 trim_cache size 0 max 16384 2024-03-18T20:44:14.783 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:44:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:44:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:44:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "fb01c11a-ae20-4cb1-9609-907e25722bb7": { 2024-03-18T20:44:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:44:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "fb01c11a-ae20-4cb1-9609-907e25722bb7", 2024-03-18T20:44:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:44:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:44:14.783 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:44:14.784 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:44:14.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.775+0000 7ffa072fa700 1 -- 172.21.15.46:0/2974037484 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9ec0a1290 msgr2=0x7ff9ec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.775+0000 7ffa072fa700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9ec0a1290 0x7ff9ec0a3750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000f80 tx=0x7ff9fc00b740 comp rx=0 tx=0).stop 2024-03-18T20:44:14.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.775+0000 7ffa072fa700 1 -- 172.21.15.46:0/2974037484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00108290 msgr2=0x7ffa00199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:14.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.775+0000 7ffa072fa700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00108290 0x7ffa00199ad0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000e970 tx=0x7ff9f000ec80 comp rx=0 tx=0).stop 2024-03-18T20:44:14.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.775+0000 7ffa072fa700 1 -- 172.21.15.46:0/2974037484 shutdown_connections 2024-03-18T20:44:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.775+0000 7ffa072fa700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9ec0a1290 0x7ff9ec0a3750 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.775+0000 7ffa072fa700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0019de70 0x7ffa0019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.775+0000 7ffa072fa700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa00103170 0x7ffa00199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.775+0000 7ffa072fa700 1 --2- 172.21.15.46:0/2974037484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00108290 0x7ffa00199ad0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.775+0000 7ffa072fa700 1 -- 172.21.15.46:0/2974037484 >> 172.21.15.46:0/2974037484 conn(0x7ffa000fe990 msgr2=0x7ffa000ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.775+0000 7ffa072fa700 1 -- 172.21.15.46:0/2974037484 shutdown_connections 2024-03-18T20:44:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:14.775+0000 7ffa072fa700 1 -- 172.21.15.46:0/2974037484 wait complete. 2024-03-18T20:44:14.791 INFO:tasks.cephfs.filesystem:scrub status for tag:fb01c11a-ae20-4cb1-9609-907e25722bb7 - {'path': '/', 'tag': 'fb01c11a-ae20-4cb1-9609-907e25722bb7', 'options': 'recursive,force'} 2024-03-18T20:44:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:15 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:44:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:15 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:15 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:15 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:15 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:15 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:15 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:15 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:15.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:15 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:15 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:15 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:44:15.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:15 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:15 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:15.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:15 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:15.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:15 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:15 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:15.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:15 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:15 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:15.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:15 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:15 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.704 INFO:tasks.workunit.client.0.smithi046.stdout: 1 287712 3.58 MB/sec execute 310 sec latency 16.642 ms 2024-03-18T20:44:15.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:15 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:44:15.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:15 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:15 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:15.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:15 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:15.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:15 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:15 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:15.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:15 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:15 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:15.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:15 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:15.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:15 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: pgmap v252: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:44:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:44:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:16 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:44:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: pgmap v252: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:44:16.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:44:16.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:16 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:44:16.704 INFO:tasks.workunit.client.0.smithi046.stdout: 1 289024 3.58 MB/sec execute 311 sec latency 67.048 ms 2024-03-18T20:44:16.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:16.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: pgmap v252: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:44:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:44:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:16 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:44:17.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:17 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:44:17.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:17 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:44:17.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:17 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:44:17.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:17 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:44:17.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:17 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:44:17.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:17 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:44:17.704 INFO:tasks.workunit.client.0.smithi046.stdout: 1 289024 3.57 MB/sec execute 312 sec latency 1067.098 ms 2024-03-18T20:44:17.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:17 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:44:17.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:17 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:44:17.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:17 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:44:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.396+0000 7f38e0661700 1 -- 172.21.15.46:0/2332578395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d8100ab0 msgr2=0x7f38d8104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.396+0000 7f38e0661700 1 --2- 172.21.15.46:0/2332578395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d8100ab0 0x7f38d8104860 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009990 tx=0x7f38d4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.396+0000 7f38e0661700 1 -- 172.21.15.46:0/2332578395 shutdown_connections 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.396+0000 7f38e0661700 1 --2- 172.21.15.46:0/2332578395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d8104f40 0x7f38d810cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.396+0000 7f38e0661700 1 --2- 172.21.15.46:0/2332578395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38d8100100 0x7f38d81004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.396+0000 7f38e0661700 1 --2- 172.21.15.46:0/2332578395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d8100ab0 0x7f38d8104860 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.396+0000 7f38e0661700 1 -- 172.21.15.46:0/2332578395 >> 172.21.15.46:0/2332578395 conn(0x7f38d8076a80 msgr2=0x7f38d8076e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.397+0000 7f38e0661700 1 -- 172.21.15.46:0/2332578395 shutdown_connections 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.397+0000 7f38e0661700 1 -- 172.21.15.46:0/2332578395 wait complete. 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.397+0000 7f38e0661700 1 Processor -- start 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.397+0000 7f38e0661700 1 -- start start 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.397+0000 7f38e0661700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d8100100 0x7f38d81970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.397+0000 7f38e0661700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38d8104f40 0x7f38d8197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.397+0000 7f38e0661700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d819b9d0 0x7f38d819be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.397+0000 7f38e0661700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d819c450 con 0x7f38d8100100 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.397+0000 7f38e0661700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d819c5c0 con 0x7f38d819b9d0 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.397+0000 7f38e0661700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d819c780 con 0x7f38d8104f40 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38de3fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d8100100 0x7f38d81970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38de3fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d8100100 0x7f38d81970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53714/0 (socket says 172.21.15.46:53714) 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38de3fd700 1 -- 172.21.15.46:0/307294736 learned_addr learned my addr 172.21.15.46:0/307294736 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38debfe700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d819b9d0 0x7f38d819be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38d8104f40 0x7f38d8197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38de3fd700 1 -- 172.21.15.46:0/307294736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38d8104f40 msgr2=0x7f38d8197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38de3fd700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38d8104f40 0x7f38d8197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38de3fd700 1 -- 172.21.15.46:0/307294736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d819b9d0 msgr2=0x7f38d819be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38de3fd700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d819b9d0 0x7f38d819be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38de3fd700 1 -- 172.21.15.46:0/307294736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40095f0 con 0x7f38d8100100 2024-03-18T20:44:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38de3fd700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d8100100 0x7f38d81970f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f38c800d980 tx=0x7f38c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38cf7fe700 1 -- 172.21.15.46:0/307294736 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8004220 con 0x7f38d8100100 2024-03-18T20:44:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38e0661700 1 -- 172.21.15.46:0/307294736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d81a3ce0 con 0x7f38d8100100 2024-03-18T20:44:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38cf7fe700 1 -- 172.21.15.46:0/307294736 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38c8007640 con 0x7f38d8100100 2024-03-18T20:44:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38cf7fe700 1 -- 172.21.15.46:0/307294736 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8005690 con 0x7f38d8100100 2024-03-18T20:44:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.398+0000 7f38e0661700 1 -- 172.21.15.46:0/307294736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d81a41c0 con 0x7f38d8100100 2024-03-18T20:44:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.401+0000 7f38cf7fe700 1 -- 172.21.15.46:0/307294736 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f38c8004750 con 0x7f38d8100100 2024-03-18T20:44:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.401+0000 7f38cf7fe700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38c40a1290 0x7f38c40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.402+0000 7f38cf7fe700 1 -- 172.21.15.46:0/307294736 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f38c80ccb70 con 0x7f38d8100100 2024-03-18T20:44:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.402+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38c40a1290 0x7f38c40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.402+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38c40a1290 0x7f38c40a3750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f38d40059d0 tx=0x7f38d4005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.402+0000 7f38ec7f6b80 1 Processor -- start 2024-03-18T20:44:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.403+0000 7f38ec7f6b80 1 -- start start 2024-03-18T20:44:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.403+0000 7f38ec7f6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 0x55740b011950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.403+0000 7f38ec7f6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011e90 0x55740b020c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.403+0000 7f38ec7f6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 0x55740b0235b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.403+0000 7f38ec7f6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740af6b930 con 0x55740b011570 2024-03-18T20:44:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.403+0000 7f38ec7f6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740af692f0 con 0x55740b011e90 2024-03-18T20:44:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.403+0000 7f38ec7f6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740af64be0 con 0x55740b0211c0 2024-03-18T20:44:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.403+0000 7f38de3fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 0x55740b011950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.403+0000 7f38de3fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 0x55740b011950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53728/0 (socket says 172.21.15.46:53728) 2024-03-18T20:44:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.403+0000 7f38de3fd700 1 -- 172.21.15.46:0/1022890887 learned_addr learned my addr 172.21.15.46:0/1022890887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.403+0000 7f38debfe700 1 --2- 172.21.15.46:0/1022890887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 0x55740b0235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.403+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/1022890887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011e90 0x55740b020c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.404+0000 7f38de3fd700 1 -- 172.21.15.46:0/1022890887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 msgr2=0x55740b0235b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.404+0000 7f38de3fd700 1 --2- 172.21.15.46:0/1022890887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 0x55740b0235b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.404+0000 7f38de3fd700 1 -- 172.21.15.46:0/1022890887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011e90 msgr2=0x55740b020c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.404+0000 7f38de3fd700 1 --2- 172.21.15.46:0/1022890887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011e90 0x55740b020c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.404+0000 7f38de3fd700 1 -- 172.21.15.46:0/1022890887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55740ade6350 con 0x55740b011570 2024-03-18T20:44:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.404+0000 7f38de3fd700 1 --2- 172.21.15.46:0/1022890887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 0x55740b011950 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f38c80a1b50 tx=0x7f38c809a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f3227f10efab38 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.404+0000 7f38ccff9700 1 -- 172.21.15.46:0/1022890887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c809da30 con 0x55740b011570 2024-03-18T20:44:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.404+0000 7f38ccff9700 1 -- 172.21.15.46:0/1022890887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38c80a2080 con 0x55740b011570 2024-03-18T20:44:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.404+0000 7f38ccff9700 1 -- 172.21.15.46:0/1022890887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c80992f0 con 0x55740b011570 2024-03-18T20:44:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.405+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1022890887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 msgr2=0x55740b011950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.405+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/1022890887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 0x55740b011950 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f38c80a1b50 tx=0x7f38c809a450 comp rx=0 tx=0).stop 2024-03-18T20:44:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.405+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1022890887 shutdown_connections 2024-03-18T20:44:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.405+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/1022890887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 0x55740b011950 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.405+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/1022890887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 0x55740b0235b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.405+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/1022890887 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011e90 0x55740b020c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.405+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1022890887 >> 172.21.15.46:0/1022890887 conn(0x55740af881c0 msgr2=0x55740b00da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.405+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1022890887 shutdown_connections 2024-03-18T20:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.405+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1022890887 wait complete. 2024-03-18T20:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.405+0000 7f38ec7f6b80 1 -- start start 2024-03-18T20:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.405+0000 7f38ec7f6b80 1 Processor -- start 2024-03-18T20:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38ec7f6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 0x55740b020c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38ec7f6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b011e90 0x55740b01de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38ec7f6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b0211c0 0x55740b01e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38ec7f6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740af00d20 con 0x55740b011570 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38ec7f6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740af009c0 con 0x55740b0211c0 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38ec7f6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740ade8640 con 0x55740b011e90 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38ec7f6b80 10 client.0 _ll_register_callbacks cb 0x55740ad6c340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38de3fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 0x55740b020c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38de3fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 0x55740b020c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53744/0 (socket says 172.21.15.46:53744) 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38de3fd700 1 -- 172.21.15.46:0/4097223765 learned_addr learned my addr 172.21.15.46:0/4097223765 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38debfe700 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b0211c0 0x55740b01e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b011e90 0x55740b01de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38de3fd700 1 -- 172.21.15.46:0/4097223765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b011e90 msgr2=0x55740b01de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38de3fd700 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b011e90 0x55740b01de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38de3fd700 1 -- 172.21.15.46:0/4097223765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b0211c0 msgr2=0x55740b01e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38de3fd700 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b0211c0 0x55740b01e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38de3fd700 1 -- 172.21.15.46:0/4097223765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d81a41c0 con 0x55740b011570 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38de3fd700 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 0x55740b020c10 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f38c8012040 tx=0x7f38c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30bc29725fc3915 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38c1ffb700 1 -- 172.21.15.46:0/4097223765 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8004380 con 0x55740b011570 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38c1ffb700 1 -- 172.21.15.46:0/4097223765 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f38c809da30 con 0x55740b011570 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.406+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/4097223765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55740b0056c0 con 0x55740b011570 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.407+0000 7f38c1ffb700 1 -- 172.21.15.46:0/4097223765 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38c809a070 con 0x55740b011570 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.407+0000 7f38ec7f6b80 10 client.18471 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.407+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/4097223765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55740ad92b00 con 0x55740b011570 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.407+0000 7f38c1ffb700 1 -- 172.21.15.46:0/4097223765 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55740b0056c0 con 0x55740b011570 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.407+0000 7f38c1ffb700 1 -- 172.21.15.46:0/4097223765 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f38c80959f0 con 0x55740b011570 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.407+0000 7f38ec7f6b80 10 client.18471 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.407+0000 7f38ec7f6b80 10 client.18471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.407+0000 7f38ec7f6b80 20 client.18471 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.407+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55740b030790 0x55740b032b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.407+0000 7f38ec7f6b80 4 client.18471 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.407+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/4097223765 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55740af5d6a0 con 0x55740b030790 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.407+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55740b030790 0x55740b032b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.408+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55740b030790 0x55740b032b80 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=174b67e551f6d263 server_cookie=882af76859081347 in_seq=0 out_seq=0 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.408+0000 7f38c1ffb700 10 client.18471 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.408+0000 7f38c1ffb700 1 -- 172.21.15.46:0/4097223765 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55740af5d6a0 con 0x55740b030790 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.408+0000 7f38c1ffb700 10 client.18471 handle_command_reply: tid=0 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.408+0000 7f38ec7f6b80 1 client.18471 shutdown 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.409+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/4097223765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 msgr2=0x55740b020c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.409+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 0x55740b020c10 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f38c8012040 tx=0x7f38c8000bc0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.409+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/4097223765 shutdown_connections 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.409+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55740b030790 0x55740b032b80 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.409+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011570 0x55740b020c10 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.409+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b0211c0 0x55740b01e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.409+0000 7f38c1ffb700 0 client.18471 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.409+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/4097223765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b011e90 0x55740b01de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.409+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/4097223765 >> 172.21.15.46:0/4097223765 conn(0x55740af881c0 msgr2=0x55740b00d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.409+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/4097223765 shutdown_connections 2024-03-18T20:44:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.409+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/4097223765 wait complete. 2024-03-18T20:44:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.409+0000 7f38ec7f6b80 20 client.18471 trim_cache size 0 max 16384 2024-03-18T20:44:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.424+0000 7f38ec7f6b80 1 Processor -- start 2024-03-18T20:44:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.424+0000 7f38ec7f6b80 1 -- start start 2024-03-18T20:44:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ec7f6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011570 0x55740b020740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ec7f6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 0x55740b01d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ec7f6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 0x55740b01dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ec7f6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740b012880 con 0x55740b011e90 2024-03-18T20:44:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ec7f6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740b00a1d0 con 0x55740b011570 2024-03-18T20:44:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ec7f6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740b012f20 con 0x55740b0211c0 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ddbfc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 0x55740b01d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ddbfc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 0x55740b01d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53748/0 (socket says 172.21.15.46:53748) 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ddbfc700 1 -- 172.21.15.46:0/2223828442 learned_addr learned my addr 172.21.15.46:0/2223828442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38de3fd700 1 --2- 172.21.15.46:0/2223828442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011570 0x55740b020740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ddbfc700 1 -- 172.21.15.46:0/2223828442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 msgr2=0x55740b01dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/2223828442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 0x55740b01dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ddbfc700 1 -- 172.21.15.46:0/2223828442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011570 msgr2=0x55740b020740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/2223828442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011570 0x55740b020740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ddbfc700 1 -- 172.21.15.46:0/2223828442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55740af2ae40 con 0x55740b011e90 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38de3fd700 1 --2- 172.21.15.46:0/2223828442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011570 0x55740b020740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/2223828442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 0x55740b01d770 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f38d400b5c0 tx=0x7f38d4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0c5c3b82c7cb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38cd7fa700 1 -- 172.21.15.46:0/2223828442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4005690 con 0x55740b011e90 2024-03-18T20:44:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38cd7fa700 1 -- 172.21.15.46:0/2223828442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38d402a040 con 0x55740b011e90 2024-03-18T20:44:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.425+0000 7f38cd7fa700 1 -- 172.21.15.46:0/2223828442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40039f0 con 0x55740b011e90 2024-03-18T20:44:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.426+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/2223828442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 msgr2=0x55740b01d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.426+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/2223828442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 0x55740b01d770 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f38d400b5c0 tx=0x7f38d4021000 comp rx=0 tx=0).stop 2024-03-18T20:44:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.426+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/2223828442 shutdown_connections 2024-03-18T20:44:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.426+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/2223828442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 0x55740b01dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.426+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/2223828442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011570 0x55740b020740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.426+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/2223828442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 0x55740b01d770 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.426+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/2223828442 >> 172.21.15.46:0/2223828442 conn(0x55740af881c0 msgr2=0x55740b019180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.426+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/2223828442 shutdown_connections 2024-03-18T20:44:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.426+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/2223828442 wait complete. 2024-03-18T20:44:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.426+0000 7f38ec7f6b80 1 -- start start 2024-03-18T20:44:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.426+0000 7f38ec7f6b80 1 Processor -- start 2024-03-18T20:44:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ec7f6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011570 0x55740b027a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ec7f6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 0x55740b025e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ec7f6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 0x55740b026390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ec7f6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740b0268d0 con 0x55740b011e90 2024-03-18T20:44:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ec7f6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740b026a40 con 0x55740b011570 2024-03-18T20:44:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ec7f6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740b026c00 con 0x55740b0211c0 2024-03-18T20:44:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ec7f6b80 10 client.0 _ll_register_callbacks cb 0x7f38d0009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ddbfc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 0x55740b025e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ddbfc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 0x55740b025e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53764/0 (socket says 172.21.15.46:53764) 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ddbfc700 1 -- 172.21.15.46:0/1959961334 learned_addr learned my addr 172.21.15.46:0/1959961334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38debfe700 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 0x55740b026390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38de3fd700 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011570 0x55740b027a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ddbfc700 1 -- 172.21.15.46:0/1959961334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 msgr2=0x55740b026390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 0x55740b026390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ddbfc700 1 -- 172.21.15.46:0/1959961334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011570 msgr2=0x55740b027a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011570 0x55740b027a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ddbfc700 1 -- 172.21.15.46:0/1959961334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d810d2e0 con 0x55740b011e90 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38debfe700 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 0x55740b026390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38ddbfc700 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 0x55740b025e50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005f50 tx=0x7f38d4032610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e4e4f561552c1e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.427+0000 7f38c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38c1ffb700 1 -- 172.21.15.46:0/1959961334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4027070 con 0x55740b011e90 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38c1ffb700 1 -- 172.21.15.46:0/1959961334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f38d4032a00 con 0x55740b011e90 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1959961334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38c80a2080 con 0x55740b011e90 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38c1ffb700 1 -- 172.21.15.46:0/1959961334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38d4005710 con 0x55740b011e90 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38ec7f6b80 10 client.18477 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1959961334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55740adeac70 con 0x55740b011e90 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38c1ffb700 1 -- 172.21.15.46:0/1959961334 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38d4049020 con 0x55740b011e90 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38c1ffb700 1 -- 172.21.15.46:0/1959961334 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f38d400f580 con 0x55740b011e90 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38ec7f6b80 10 client.18477 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38ec7f6b80 10 client.18477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38ec7f6b80 20 client.18477 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55740b085510 0x55740b087930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38ec7f6b80 4 client.18477 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:44:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1959961334 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55740adb3c90 con 0x55740b085510 2024-03-18T20:44:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.428+0000 7f38de3fd700 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55740b085510 0x55740b087930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.429+0000 7f38c1ffb700 10 client.18477 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:18.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.429+0000 7f38de3fd700 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55740b085510 0x55740b087930 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93b7e24fcbf8be31 server_cookie=1f6412fe48a61aeb in_seq=0 out_seq=0 2024-03-18T20:44:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38c1ffb700 1 -- 172.21.15.46:0/1959961334 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55740adb3c90 con 0x55740b085510 2024-03-18T20:44:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38c1ffb700 10 client.18477 handle_command_reply: tid=0 2024-03-18T20:44:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38ec7f6b80 1 client.18477 shutdown 2024-03-18T20:44:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1959961334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 msgr2=0x55740b025e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 0x55740b025e50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005f50 tx=0x7f38d4032610 comp rx=0 tx=0).stop 2024-03-18T20:44:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1959961334 shutdown_connections 2024-03-18T20:44:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55740b085510 0x55740b087930 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55740b0211c0 0x55740b026390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55740b011570 0x55740b027a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38c1ffb700 0 client.18477 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38ec7f6b80 1 --2- 172.21.15.46:0/1959961334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55740b011e90 0x55740b025e50 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1959961334 >> 172.21.15.46:0/1959961334 conn(0x55740af881c0 msgr2=0x55740b0135d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1959961334 shutdown_connections 2024-03-18T20:44:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38ec7f6b80 1 -- 172.21.15.46:0/1959961334 wait complete. 2024-03-18T20:44:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.457+0000 7f38ec7f6b80 20 client.18477 trim_cache size 0 max 16384 2024-03-18T20:44:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:44:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:44:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:44:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:44:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.458+0000 7f38e0661700 1 -- 172.21.15.46:0/307294736 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38c40a1290 msgr2=0x7f38c40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.458+0000 7f38e0661700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38c40a1290 0x7f38c40a3750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f38d40059d0 tx=0x7f38d4005960 comp rx=0 tx=0).stop 2024-03-18T20:44:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.458+0000 7f38e0661700 1 -- 172.21.15.46:0/307294736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d8100100 msgr2=0x7f38d81970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.458+0000 7f38e0661700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d8100100 0x7f38d81970f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f38c800d980 tx=0x7f38c800dc90 comp rx=0 tx=0).stop 2024-03-18T20:44:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.458+0000 7f38e0661700 1 -- 172.21.15.46:0/307294736 shutdown_connections 2024-03-18T20:44:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.458+0000 7f38e0661700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38c40a1290 0x7f38c40a3750 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.458+0000 7f38e0661700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d8100100 0x7f38d81970f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.458+0000 7f38e0661700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d819b9d0 0x7f38d819be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.458+0000 7f38e0661700 1 --2- 172.21.15.46:0/307294736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38d8104f40 0x7f38d8197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.458+0000 7f38e0661700 1 -- 172.21.15.46:0/307294736 >> 172.21.15.46:0/307294736 conn(0x7f38d8076a80 msgr2=0x7f38d8069810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.458+0000 7f38e0661700 1 -- 172.21.15.46:0/307294736 shutdown_connections 2024-03-18T20:44:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:18.458+0000 7f38e0661700 1 -- 172.21.15.46:0/307294736 wait complete. 2024-03-18T20:44:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:18 smithi155 ceph-mon[48344]: pgmap v253: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:44:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:18 smithi046 ceph-mon[43495]: pgmap v253: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:44:18.704 INFO:tasks.workunit.client.0.smithi046.stdout: 1 289024 3.56 MB/sec execute 313 sec latency 2067.176 ms 2024-03-18T20:44:18.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:18 smithi073 ceph-mon[47384]: pgmap v253: 97 pgs: 97 active+clean; 109 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:44:19.704 INFO:tasks.workunit.client.0.smithi046.stdout: 1 289024 3.55 MB/sec execute 314 sec latency 3067.278 ms 2024-03-18T20:44:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:20 smithi155 ceph-mon[48344]: pgmap v254: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-03-18T20:44:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:20 smithi046 ceph-mon[43495]: pgmap v254: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-03-18T20:44:20.704 INFO:tasks.workunit.client.0.smithi046.stdout: 1 290984 3.58 MB/sec execute 315 sec latency 3228.775 ms 2024-03-18T20:44:20.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:20 smithi073 ceph-mon[47384]: pgmap v254: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-03-18T20:44:21.704 INFO:tasks.workunit.client.0.smithi046.stdout: 1 292260 3.59 MB/sec execute 316 sec latency 16.926 ms 2024-03-18T20:44:22.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:22 smithi155 ceph-mon[48344]: pgmap v255: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-18T20:44:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:22 smithi046 ceph-mon[43495]: pgmap v255: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-18T20:44:22.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 292567 3.57 MB/sec execute 317 sec latency 807.646 ms 2024-03-18T20:44:22.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:22 smithi073 ceph-mon[47384]: pgmap v255: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-18T20:44:23.704 INFO:tasks.workunit.client.0.smithi046.stdout: 1 292567 3.56 MB/sec execute 318 sec latency 1807.757 ms 2024-03-18T20:44:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:24 smithi155 ceph-mon[48344]: pgmap v256: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-03-18T20:44:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:24 smithi046 ceph-mon[43495]: pgmap v256: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-03-18T20:44:24.704 INFO:tasks.workunit.client.0.smithi046.stdout: 1 292567 3.55 MB/sec execute 319 sec latency 2807.870 ms 2024-03-18T20:44:24.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:24 smithi073 ceph-mon[47384]: pgmap v256: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-03-18T20:44:25.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 293470 3.55 MB/sec execute 320 sec latency 3430.314 ms 2024-03-18T20:44:26.219 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:25 smithi073 ceph-mon[47384]: pgmap v257: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-03-18T20:44:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:25 smithi155 ceph-mon[48344]: pgmap v257: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-03-18T20:44:26.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:25 smithi046 ceph-mon[43495]: pgmap v257: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-03-18T20:44:26.704 INFO:tasks.workunit.client.0.smithi046.stdout: 1 295043 3.57 MB/sec execute 321 sec latency 13.136 ms 2024-03-18T20:44:27.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 296117 3.58 MB/sec execute 322 sec latency 433.584 ms 2024-03-18T20:44:28.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:28 smithi073 ceph-mon[47384]: pgmap v258: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-03-18T20:44:28.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:28 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:44:28.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:28 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:44:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:28 smithi155 ceph-mon[48344]: pgmap v258: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-03-18T20:44:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:28 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:44:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:28 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:44:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:28 smithi046 ceph-mon[43495]: pgmap v258: 97 pgs: 97 active+clean; 113 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-03-18T20:44:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:28 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:44:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:28 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:44:28.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 296117 3.57 MB/sec execute 323 sec latency 1433.619 ms 2024-03-18T20:44:29.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:29 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:44:29.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:29 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:44:29.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:29 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:44:29.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:29 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:44:29.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:29 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:44:29.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:29 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:44:29.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:29 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:44:29.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:29 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:44:29.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:29 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:44:29.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 296117 3.56 MB/sec execute 324 sec latency 2433.712 ms 2024-03-18T20:44:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:30 smithi155 ceph-mon[48344]: pgmap v259: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:44:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:30 smithi046 ceph-mon[43495]: pgmap v259: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:44:30.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 297028 3.55 MB/sec execute 325 sec latency 3053.174 ms 2024-03-18T20:44:30.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:30 smithi073 ceph-mon[47384]: pgmap v259: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:44:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.591+0000 7fc1779ad700 1 -- 172.21.15.46:0/4009893940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc170100f10 msgr2=0x7fc1701012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.591+0000 7fc1779ad700 1 --2- 172.21.15.46:0/4009893940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc170100f10 0x7fc1701012f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009990 tx=0x7fc16c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.591+0000 7fc1779ad700 1 -- 172.21.15.46:0/4009893940 shutdown_connections 2024-03-18T20:44:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.591+0000 7fc1779ad700 1 --2- 172.21.15.46:0/4009893940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc170100f10 0x7fc1701012f0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.591+0000 7fc1779ad700 1 --2- 172.21.15.46:0/4009893940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc170106030 0x7fc17010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.591+0000 7fc1779ad700 1 --2- 172.21.15.46:0/4009893940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1701018c0 0x7fc170105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.591+0000 7fc1779ad700 1 -- 172.21.15.46:0/4009893940 >> 172.21.15.46:0/4009893940 conn(0x7fc1700fc770 msgr2=0x7fc1700feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.591+0000 7fc1779ad700 1 -- 172.21.15.46:0/4009893940 shutdown_connections 2024-03-18T20:44:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.591+0000 7fc1779ad700 1 -- 172.21.15.46:0/4009893940 wait complete. 2024-03-18T20:44:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc1779ad700 1 Processor -- start 2024-03-18T20:44:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc1779ad700 1 -- start start 2024-03-18T20:44:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc1779ad700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc170100f10 0x7fc1700730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc1779ad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc170106030 0x7fc1700735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc1779ad700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc170073b30 0x7fc1701a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc1779ad700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1701a59f0 con 0x7fc170106030 2024-03-18T20:44:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc1779ad700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1701a5b60 con 0x7fc170073b30 2024-03-18T20:44:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc1779ad700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1701a5d20 con 0x7fc170100f10 2024-03-18T20:44:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc174f48700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc170106030 0x7fc1700735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc175749700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc170100f10 0x7fc1700730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc175749700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc170100f10 0x7fc1700730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37472/0 (socket says 172.21.15.46:37472) 2024-03-18T20:44:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc174f48700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc170106030 0x7fc1700735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37550/0 (socket says 172.21.15.46:37550) 2024-03-18T20:44:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc175749700 1 -- 172.21.15.46:0/2078719737 learned_addr learned my addr 172.21.15.46:0/2078719737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.592+0000 7fc175f4a700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc170073b30 0x7fc1701a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.593+0000 7fc175749700 1 -- 172.21.15.46:0/2078719737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc170073b30 msgr2=0x7fc1701a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.593+0000 7fc175749700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc170073b30 0x7fc1701a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.593+0000 7fc175749700 1 -- 172.21.15.46:0/2078719737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc170106030 msgr2=0x7fc1700735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.593+0000 7fc175749700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc170106030 0x7fc1700735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.593+0000 7fc175749700 1 -- 172.21.15.46:0/2078719737 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc160009520 con 0x7fc170100f10 2024-03-18T20:44:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.593+0000 7fc174f48700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc170106030 0x7fc1700735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.593+0000 7fc175749700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc170100f10 0x7fc1700730b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc16c004760 tx=0x7fc16c004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.593+0000 7fc1667fc700 1 -- 172.21.15.46:0/2078719737 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c01d070 con 0x7fc170100f10 2024-03-18T20:44:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.593+0000 7fc1779ad700 1 -- 172.21.15.46:0/2078719737 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c0095f0 con 0x7fc170100f10 2024-03-18T20:44:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.593+0000 7fc1667fc700 1 -- 172.21.15.46:0/2078719737 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc16c022470 con 0x7fc170100f10 2024-03-18T20:44:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.593+0000 7fc1667fc700 1 -- 172.21.15.46:0/2078719737 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c017650 con 0x7fc170100f10 2024-03-18T20:44:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.593+0000 7fc1779ad700 1 -- 172.21.15.46:0/2078719737 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1701a61d0 con 0x7fc170100f10 2024-03-18T20:44:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.595+0000 7fc1667fc700 1 -- 172.21.15.46:0/2078719737 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc16c0225e0 con 0x7fc170100f10 2024-03-18T20:44:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.596+0000 7fc1667fc700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc15c0a1290 0x7fc15c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.596+0000 7fc1667fc700 1 -- 172.21.15.46:0/2078719737 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc16c0cdea0 con 0x7fc170100f10 2024-03-18T20:44:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.596+0000 7fc174f48700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc15c0a1290 0x7fc15c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.597+0000 7fc174f48700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc15c0a1290 0x7fc15c0a3750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc160009fd0 tx=0x7fc160009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.597+0000 7fc183b42b80 1 Processor -- start 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.597+0000 7fc183b42b80 1 -- start start 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.597+0000 7fc183b42b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9570 0x563e9ddd9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.597+0000 7fc183b42b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9dde4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.597+0000 7fc183b42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9dde4db0 0x563e9dde71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.597+0000 7fc183b42b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9dd33930 con 0x563e9ddd9570 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.597+0000 7fc183b42b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9dd312f0 con 0x563e9ddd9e90 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.597+0000 7fc183b42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9dd2cbe0 con 0x563e9dde4db0 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc175749700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9570 0x563e9ddd9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc174f48700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9dde4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc174f48700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9dde4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:36384/0 (socket says 172.21.15.46:36384) 2024-03-18T20:44:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc175749700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9570 0x563e9ddd9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37560/0 (socket says 172.21.15.46:37560) 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc174f48700 1 -- 172.21.15.46:0/3416869848 learned_addr learned my addr 172.21.15.46:0/3416869848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc175f4a700 1 --2- 172.21.15.46:0/3416869848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9dde4db0 0x563e9dde71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc174f48700 1 -- 172.21.15.46:0/3416869848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9dde4db0 msgr2=0x563e9dde71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc174f48700 1 --2- 172.21.15.46:0/3416869848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9dde4db0 0x563e9dde71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc174f48700 1 -- 172.21.15.46:0/3416869848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9570 msgr2=0x563e9ddd9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc174f48700 1 --2- 172.21.15.46:0/3416869848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9570 0x563e9ddd9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc174f48700 1 -- 172.21.15.46:0/3416869848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e9dbae350 con 0x563e9ddd9e90 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc175f4a700 1 --2- 172.21.15.46:0/3416869848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9dde4db0 0x563e9dde71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc175749700 1 --2- 172.21.15.46:0/3416869848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9570 0x563e9ddd9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc174f48700 1 --2- 172.21.15.46:0/3416869848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9dde4870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc16000e670 tx=0x7fc160014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6813c2a0b25016e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc15b7fe700 1 -- 172.21.15.46:0/3416869848 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160026630 con 0x563e9ddd9e90 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.598+0000 7fc15b7fe700 1 -- 172.21.15.46:0/3416869848 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc160021070 con 0x563e9ddd9e90 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.599+0000 7fc15b7fe700 1 -- 172.21.15.46:0/3416869848 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16001d630 con 0x563e9ddd9e90 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.599+0000 7fc183b42b80 1 -- 172.21.15.46:0/3416869848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 msgr2=0x563e9dde4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.599+0000 7fc183b42b80 1 --2- 172.21.15.46:0/3416869848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9dde4870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc16000e670 tx=0x7fc160014000 comp rx=0 tx=0).stop 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.599+0000 7fc183b42b80 1 -- 172.21.15.46:0/3416869848 shutdown_connections 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.599+0000 7fc183b42b80 1 --2- 172.21.15.46:0/3416869848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9570 0x563e9ddd9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.599+0000 7fc183b42b80 1 --2- 172.21.15.46:0/3416869848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9dde4db0 0x563e9dde71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.599+0000 7fc183b42b80 1 --2- 172.21.15.46:0/3416869848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9dde4870 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.599+0000 7fc183b42b80 1 -- 172.21.15.46:0/3416869848 >> 172.21.15.46:0/3416869848 conn(0x563e9dd501c0 msgr2=0x563e9ddd5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.599+0000 7fc183b42b80 1 -- 172.21.15.46:0/3416869848 shutdown_connections 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.599+0000 7fc183b42b80 1 -- 172.21.15.46:0/3416869848 wait complete. 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc183b42b80 1 -- start start 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc183b42b80 1 Processor -- start 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc183b42b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9570 0x563e9dde4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc183b42b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9e90 0x563e9dde1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc183b42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9dde4db0 0x563e9dde2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc183b42b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9dcc8d20 con 0x563e9ddd9e90 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc183b42b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9dcc89c0 con 0x563e9ddd9570 2024-03-18T20:44:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc183b42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9dbb0640 con 0x563e9dde4db0 2024-03-18T20:44:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc183b42b80 10 client.0 _ll_register_callbacks cb 0x563e9db34340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc174f48700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9e90 0x563e9dde1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc175749700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9570 0x563e9dde4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc174f48700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9e90 0x563e9dde1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37566/0 (socket says 172.21.15.46:37566) 2024-03-18T20:44:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc174f48700 1 -- 172.21.15.46:0/2902735682 learned_addr learned my addr 172.21.15.46:0/2902735682 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.600+0000 7fc175f4a700 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9dde4db0 0x563e9dde2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc174f48700 1 -- 172.21.15.46:0/2902735682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9dde4db0 msgr2=0x563e9dde2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc174f48700 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9dde4db0 0x563e9dde2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc174f48700 1 -- 172.21.15.46:0/2902735682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9570 msgr2=0x563e9dde4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc174f48700 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9570 0x563e9dde4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc174f48700 1 -- 172.21.15.46:0/2902735682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc168011230 con 0x563e9ddd9e90 2024-03-18T20:44:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc175749700 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9570 0x563e9dde4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc175f4a700 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9dde4db0 0x563e9dde2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc174f48700 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9e90 0x563e9dde1a10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc160009cc0 tx=0x7fc16000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e95af3670254e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc158ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc158ff9700 1 -- 172.21.15.46:0/2902735682 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160005140 con 0x563e9ddd9e90 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc158ff9700 1 -- 172.21.15.46:0/2902735682 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc1600052a0 con 0x563e9ddd9e90 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc183b42b80 1 -- 172.21.15.46:0/2902735682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e9ddcd6c0 con 0x563e9ddd9e90 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc158ff9700 1 -- 172.21.15.46:0/2902735682 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc160005460 con 0x563e9ddd9e90 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc183b42b80 10 client.18489 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.601+0000 7fc183b42b80 1 -- 172.21.15.46:0/2902735682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7fc160017050 con 0x563e9ddd9e90 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.602+0000 7fc158ff9700 1 -- 172.21.15.46:0/2902735682 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e9ddcd6c0 con 0x563e9ddd9e90 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.602+0000 7fc158ff9700 1 -- 172.21.15.46:0/2902735682 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc16001c570 con 0x563e9ddd9e90 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.602+0000 7fc183b42b80 10 client.18489 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.602+0000 7fc183b42b80 10 client.18489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.602+0000 7fc183b42b80 20 client.18489 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.602+0000 7fc183b42b80 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563e9ddf4940 0x563e9ddf6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.602+0000 7fc183b42b80 4 client.18489 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.602+0000 7fc183b42b80 1 -- 172.21.15.46:0/2902735682 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e9dd256a0 con 0x563e9ddf4940 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.602+0000 7fc175749700 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563e9ddf4940 0x563e9ddf6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.602+0000 7fc175749700 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563e9ddf4940 0x563e9ddf6d60 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d2d39cb33a8549 server_cookie=649887da219d8272 in_seq=0 out_seq=0 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.602+0000 7fc158ff9700 10 client.18489 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.603+0000 7fc158ff9700 1 -- 172.21.15.46:0/2902735682 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563e9dd256a0 con 0x563e9ddf4940 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.603+0000 7fc158ff9700 10 client.18489 handle_command_reply: tid=0 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.603+0000 7fc183b42b80 1 client.18489 shutdown 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.603+0000 7fc183b42b80 1 -- 172.21.15.46:0/2902735682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9e90 msgr2=0x563e9dde1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.603+0000 7fc183b42b80 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9e90 0x563e9dde1a10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc160009cc0 tx=0x7fc16000ef10 comp rx=0 tx=0).stop 2024-03-18T20:44:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.603+0000 7fc183b42b80 1 -- 172.21.15.46:0/2902735682 shutdown_connections 2024-03-18T20:44:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.603+0000 7fc183b42b80 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563e9ddf4940 0x563e9ddf6d60 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.603+0000 7fc183b42b80 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9dde4db0 0x563e9dde2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.604+0000 7fc183b42b80 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9570 0x563e9dde4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.604+0000 7fc183b42b80 1 --2- 172.21.15.46:0/2902735682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9ddd9e90 0x563e9dde1a10 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.604+0000 7fc158ff9700 0 client.18489 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.604+0000 7fc183b42b80 1 -- 172.21.15.46:0/2902735682 >> 172.21.15.46:0/2902735682 conn(0x563e9dd501c0 msgr2=0x563e9ddd5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.604+0000 7fc183b42b80 1 -- 172.21.15.46:0/2902735682 shutdown_connections 2024-03-18T20:44:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.604+0000 7fc183b42b80 1 -- 172.21.15.46:0/2902735682 wait complete. 2024-03-18T20:44:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.604+0000 7fc183b42b80 20 client.18489 trim_cache size 0 max 16384 2024-03-18T20:44:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.620+0000 7fc183b42b80 1 Processor -- start 2024-03-18T20:44:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.620+0000 7fc183b42b80 1 -- start start 2024-03-18T20:44:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.620+0000 7fc183b42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 0x563e9ddf6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.620+0000 7fc183b42b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9dddc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.620+0000 7fc183b42b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 0x563e9ddd4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.620+0000 7fc183b42b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9ddcd290 con 0x563e9dde4db0 2024-03-18T20:44:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.620+0000 7fc183b42b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9dde28e0 con 0x563e9ddd9e90 2024-03-18T20:44:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.620+0000 7fc183b42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9dde2600 con 0x563e9ddd9570 2024-03-18T20:44:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc175f4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 0x563e9ddd4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc174f48700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9dddc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc175749700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 0x563e9ddf6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc174f48700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9dddc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:36410/0 (socket says 172.21.15.46:36410) 2024-03-18T20:44:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc175f4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 0x563e9ddd4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37582/0 (socket says 172.21.15.46:37582) 2024-03-18T20:44:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc174f48700 1 -- 172.21.15.46:0/2051689257 learned_addr learned my addr 172.21.15.46:0/2051689257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc175749700 1 -- 172.21.15.46:0/2051689257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 msgr2=0x563e9dddc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc175749700 1 --2- 172.21.15.46:0/2051689257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9dddc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc175749700 1 -- 172.21.15.46:0/2051689257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 msgr2=0x563e9ddd4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc175749700 1 --2- 172.21.15.46:0/2051689257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 0x563e9ddd4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc175749700 1 -- 172.21.15.46:0/2051689257 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e9db5ab00 con 0x563e9ddd9570 2024-03-18T20:44:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc174f48700 1 --2- 172.21.15.46:0/2051689257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9dddc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc175f4a700 1 --2- 172.21.15.46:0/2051689257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 0x563e9ddd4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc175749700 1 --2- 172.21.15.46:0/2051689257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 0x563e9ddf6d30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0cb570 tx=0x7fc16c01a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f3549a2ca1820f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc15bfff700 1 -- 172.21.15.46:0/2051689257 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c09b6c0 con 0x563e9ddd9570 2024-03-18T20:44:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc15bfff700 1 -- 172.21.15.46:0/2051689257 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc16c09afc0 con 0x563e9ddd9570 2024-03-18T20:44:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.621+0000 7fc15bfff700 1 -- 172.21.15.46:0/2051689257 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c099470 con 0x563e9ddd9570 2024-03-18T20:44:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.622+0000 7fc183b42b80 1 -- 172.21.15.46:0/2051689257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 msgr2=0x563e9ddf6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.622+0000 7fc183b42b80 1 --2- 172.21.15.46:0/2051689257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 0x563e9ddf6d30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0cb570 tx=0x7fc16c01a040 comp rx=0 tx=0).stop 2024-03-18T20:44:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.622+0000 7fc183b42b80 1 -- 172.21.15.46:0/2051689257 shutdown_connections 2024-03-18T20:44:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.622+0000 7fc183b42b80 1 --2- 172.21.15.46:0/2051689257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 0x563e9ddd4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.622+0000 7fc183b42b80 1 --2- 172.21.15.46:0/2051689257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 0x563e9ddf6d30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.622+0000 7fc183b42b80 1 --2- 172.21.15.46:0/2051689257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9dddc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.622+0000 7fc183b42b80 1 -- 172.21.15.46:0/2051689257 >> 172.21.15.46:0/2051689257 conn(0x563e9dd501c0 msgr2=0x563e9ddfb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.622+0000 7fc183b42b80 1 -- 172.21.15.46:0/2051689257 shutdown_connections 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.622+0000 7fc183b42b80 1 -- 172.21.15.46:0/2051689257 wait complete. 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.622+0000 7fc183b42b80 1 -- start start 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc183b42b80 1 Processor -- start 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc183b42b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 0x563e9ddddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc183b42b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9ddde200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc183b42b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 0x563e9ddde740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc183b42b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9dddec80 con 0x563e9dde4db0 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc183b42b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9dddedf0 con 0x563e9ddd9e90 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc183b42b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e9dddefb0 con 0x563e9ddd9570 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc183b42b80 10 client.0 _ll_register_callbacks cb 0x7fc16c005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc175f4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 0x563e9ddde740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc175749700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 0x563e9ddddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc174f48700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9ddde200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc175f4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 0x563e9ddde740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37584/0 (socket says 172.21.15.46:37584) 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc175f4a700 1 -- 172.21.15.46:0/3271734531 learned_addr learned my addr 172.21.15.46:0/3271734531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.623+0000 7fc175749700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 0x563e9ddddcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37504/0 (socket says 172.21.15.46:37504) 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc175f4a700 1 -- 172.21.15.46:0/3271734531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 msgr2=0x563e9ddddcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc175f4a700 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 0x563e9ddddcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc175f4a700 1 -- 172.21.15.46:0/3271734531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 msgr2=0x563e9ddde200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc175f4a700 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9ddde200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc175f4a700 1 -- 172.21.15.46:0/3271734531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc160017050 con 0x563e9dde4db0 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc174f48700 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9ddde200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc175749700 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 0x563e9ddddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc175f4a700 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 0x563e9ddde740 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc168008e70 tx=0x7fc168004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2c669887591487 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc158ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc158ff9700 1 -- 172.21.15.46:0/3271734531 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16801c070 con 0x563e9dde4db0 2024-03-18T20:44:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc183b42b80 1 -- 172.21.15.46:0/3271734531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc160021070 con 0x563e9dde4db0 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc158ff9700 1 -- 172.21.15.46:0/3271734531 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc168020410 con 0x563e9dde4db0 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc158ff9700 1 -- 172.21.15.46:0/3271734531 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc16800e040 con 0x563e9dde4db0 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc183b42b80 10 client.18495 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.624+0000 7fc183b42b80 1 -- 172.21.15.46:0/3271734531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7fc168007600 con 0x563e9dde4db0 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.625+0000 7fc158ff9700 1 -- 172.21.15.46:0/3271734531 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc16802d020 con 0x563e9dde4db0 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.625+0000 7fc158ff9700 1 -- 172.21.15.46:0/3271734531 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc168029530 con 0x563e9dde4db0 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.625+0000 7fc183b42b80 10 client.18495 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.625+0000 7fc183b42b80 10 client.18495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.625+0000 7fc183b42b80 20 client.18495 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.625+0000 7fc183b42b80 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563e9de49190 0x563e9de4b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.625+0000 7fc183b42b80 4 client.18495 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.625+0000 7fc183b42b80 1 -- 172.21.15.46:0/3271734531 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563e9db7bc90 con 0x563e9de49190 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.625+0000 7fc175749700 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563e9de49190 0x563e9de4b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.625+0000 7fc175749700 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563e9de49190 0x563e9de4b5b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=551c81ccb0aaca29 server_cookie=db358fb14606cf27 in_seq=0 out_seq=0 2024-03-18T20:44:31.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.625+0000 7fc158ff9700 10 client.18495 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.653+0000 7fc158ff9700 1 -- 172.21.15.46:0/3271734531 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563e9db7bc90 con 0x563e9de49190 2024-03-18T20:44:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.653+0000 7fc158ff9700 10 client.18495 handle_command_reply: tid=0 2024-03-18T20:44:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc183b42b80 1 client.18495 shutdown 2024-03-18T20:44:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc183b42b80 1 -- 172.21.15.46:0/3271734531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 msgr2=0x563e9ddde740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc183b42b80 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 0x563e9ddde740 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc168008e70 tx=0x7fc168004320 comp rx=0 tx=0).stop 2024-03-18T20:44:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc183b42b80 1 -- 172.21.15.46:0/3271734531 shutdown_connections 2024-03-18T20:44:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc183b42b80 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x563e9de49190 0x563e9de4b5b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc183b42b80 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563e9dde4db0 0x563e9ddde740 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc183b42b80 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563e9ddd9570 0x563e9ddddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc183b42b80 1 --2- 172.21.15.46:0/3271734531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563e9ddd9e90 0x563e9ddde200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc183b42b80 1 -- 172.21.15.46:0/3271734531 >> 172.21.15.46:0/3271734531 conn(0x563e9dd501c0 msgr2=0x563e9dde0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc158ff9700 0 client.18495 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc183b42b80 1 -- 172.21.15.46:0/3271734531 shutdown_connections 2024-03-18T20:44:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc183b42b80 1 -- 172.21.15.46:0/3271734531 wait complete. 2024-03-18T20:44:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.654+0000 7fc183b42b80 20 client.18495 trim_cache size 0 max 16384 2024-03-18T20:44:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:44:31.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:44:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:44:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:44:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.655+0000 7fc1779ad700 1 -- 172.21.15.46:0/2078719737 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc15c0a1290 msgr2=0x7fc15c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.655+0000 7fc1779ad700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc15c0a1290 0x7fc15c0a3750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc160009fd0 tx=0x7fc160009260 comp rx=0 tx=0).stop 2024-03-18T20:44:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.655+0000 7fc1779ad700 1 -- 172.21.15.46:0/2078719737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc170100f10 msgr2=0x7fc1700730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.655+0000 7fc1779ad700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc170100f10 0x7fc1700730b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc16c004760 tx=0x7fc16c004790 comp rx=0 tx=0).stop 2024-03-18T20:44:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.655+0000 7fc1779ad700 1 -- 172.21.15.46:0/2078719737 shutdown_connections 2024-03-18T20:44:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.655+0000 7fc1779ad700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc15c0a1290 0x7fc15c0a3750 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.655+0000 7fc1779ad700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc170073b30 0x7fc1701a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.655+0000 7fc1779ad700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc170100f10 0x7fc1700730b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.655+0000 7fc1779ad700 1 --2- 172.21.15.46:0/2078719737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc170106030 0x7fc1700735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.655+0000 7fc1779ad700 1 -- 172.21.15.46:0/2078719737 >> 172.21.15.46:0/2078719737 conn(0x7fc1700fc770 msgr2=0x7fc1700fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.655+0000 7fc1779ad700 1 -- 172.21.15.46:0/2078719737 shutdown_connections 2024-03-18T20:44:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:31.655+0000 7fc1779ad700 1 -- 172.21.15.46:0/2078719737 wait complete. 2024-03-18T20:44:31.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 298570 3.58 MB/sec execute 326 sec latency 16.161 ms 2024-03-18T20:44:32.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:32 smithi155 ceph-mon[48344]: pgmap v260: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-03-18T20:44:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:32 smithi046 ceph-mon[43495]: pgmap v260: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-03-18T20:44:32.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 299665 3.58 MB/sec execute 327 sec latency 418.721 ms 2024-03-18T20:44:32.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:32 smithi073 ceph-mon[47384]: pgmap v260: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.3 MiB/s wr, 123 op/s 2024-03-18T20:44:33.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 299665 3.57 MB/sec execute 328 sec latency 1418.814 ms 2024-03-18T20:44:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:34 smithi155 ceph-mon[48344]: pgmap v261: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-03-18T20:44:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:34 smithi046 ceph-mon[43495]: pgmap v261: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-03-18T20:44:34.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 299665 3.56 MB/sec execute 329 sec latency 2418.921 ms 2024-03-18T20:44:34.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:34 smithi073 ceph-mon[47384]: pgmap v261: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-03-18T20:44:35.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 301670 3.59 MB/sec execute 330 sec latency 2568.932 ms 2024-03-18T20:44:36.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:35 smithi073 ceph-mon[47384]: pgmap v262: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-18T20:44:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:35 smithi155 ceph-mon[48344]: pgmap v262: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-18T20:44:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:35 smithi046 ceph-mon[43495]: pgmap v262: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-18T20:44:36.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 302957 3.60 MB/sec execute 331 sec latency 19.753 ms 2024-03-18T20:44:37.705 INFO:tasks.workunit.client.0.smithi046.stdout: 1 303214 3.59 MB/sec execute 332 sec latency 852.283 ms 2024-03-18T20:44:38.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:38 smithi073 ceph-mon[47384]: pgmap v263: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-18T20:44:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:38 smithi155 ceph-mon[48344]: pgmap v263: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-18T20:44:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:38 smithi046 ceph-mon[43495]: pgmap v263: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-03-18T20:44:38.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 303214 3.58 MB/sec execute 333 sec latency 1852.381 ms 2024-03-18T20:44:39.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 303214 3.56 MB/sec execute 334 sec latency 2852.490 ms 2024-03-18T20:44:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:40 smithi155 ceph-mon[48344]: pgmap v264: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.8 MiB/s wr, 176 op/s 2024-03-18T20:44:40.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:40 smithi046 ceph-mon[43495]: pgmap v264: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.8 MiB/s wr, 176 op/s 2024-03-18T20:44:40.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 303881 3.56 MB/sec execute 335 sec latency 3469.557 ms 2024-03-18T20:44:40.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:40 smithi073 ceph-mon[47384]: pgmap v264: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.8 MiB/s wr, 176 op/s 2024-03-18T20:44:41.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 305483 3.58 MB/sec execute 336 sec latency 12.776 ms 2024-03-18T20:44:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:42 smithi155 ceph-mon[48344]: pgmap v265: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:44:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:42 smithi046 ceph-mon[43495]: pgmap v265: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:44:42.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 306757 3.59 MB/sec execute 337 sec latency 170.329 ms 2024-03-18T20:44:42.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:42 smithi073 ceph-mon[47384]: pgmap v265: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:44:43.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 306757 3.58 MB/sec execute 338 sec latency 1170.430 ms 2024-03-18T20:44:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:44 smithi155 ceph-mon[48344]: pgmap v266: 97 pgs: 97 active+clean; 119 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-03-18T20:44:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:44 smithi046 ceph-mon[43495]: pgmap v266: 97 pgs: 97 active+clean; 119 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-03-18T20:44:44.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 306757 3.57 MB/sec execute 339 sec latency 2170.500 ms 2024-03-18T20:44:44.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:44 smithi073 ceph-mon[47384]: pgmap v266: 97 pgs: 97 active+clean; 119 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-03-18T20:44:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.786+0000 7f5c8c32a700 1 -- 172.21.15.46:0/2828337431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c84100f10 msgr2=0x7f5c841012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.786+0000 7f5c8c32a700 1 --2- 172.21.15.46:0/2828337431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c84100f10 0x7f5c841012f0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009990 tx=0x7f5c80009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.786+0000 7f5c8c32a700 1 -- 172.21.15.46:0/2828337431 shutdown_connections 2024-03-18T20:44:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.786+0000 7f5c8c32a700 1 --2- 172.21.15.46:0/2828337431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c84100f10 0x7f5c841012f0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.786+0000 7f5c8c32a700 1 --2- 172.21.15.46:0/2828337431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c84106030 0x7f5c8410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.786+0000 7f5c8c32a700 1 --2- 172.21.15.46:0/2828337431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c841018c0 0x7f5c84105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.786+0000 7f5c8c32a700 1 -- 172.21.15.46:0/2828337431 >> 172.21.15.46:0/2828337431 conn(0x7f5c840fc7b0 msgr2=0x7f5c840febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.786+0000 7f5c8c32a700 1 -- 172.21.15.46:0/2828337431 shutdown_connections 2024-03-18T20:44:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.786+0000 7f5c8c32a700 1 -- 172.21.15.46:0/2828337431 wait complete. 2024-03-18T20:44:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c8c32a700 1 Processor -- start 2024-03-18T20:44:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c8c32a700 1 -- start start 2024-03-18T20:44:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c8c32a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c84100f10 0x7f5c84197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c8c32a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c84106030 0x7f5c841976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c8c32a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8419ba60 0x7f5c8419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c8c32a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8419c4e0 con 0x7f5c8419ba60 2024-03-18T20:44:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c8c32a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8419c650 con 0x7f5c84100f10 2024-03-18T20:44:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c8c32a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8419c810 con 0x7f5c84106030 2024-03-18T20:44:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c8a8c7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8419ba60 0x7f5c8419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c8a8c7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8419ba60 0x7f5c8419bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40278/0 (socket says 172.21.15.46:40278) 2024-03-18T20:44:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/27625827 learned_addr learned my addr 172.21.15.46:0/27625827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c898c5700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c84106030 0x7f5c841976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.787+0000 7f5c8a0c6700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c84100f10 0x7f5c84197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.788+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/27625827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c84106030 msgr2=0x7f5c841976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.788+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c84106030 0x7f5c841976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.788+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/27625827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c84100f10 msgr2=0x7f5c84197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.788+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c84100f10 0x7f5c84197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.788+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/27625827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c800095f0 con 0x7f5c8419ba60 2024-03-18T20:44:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.788+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8419ba60 0x7f5c8419bee0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00ef30 tx=0x7f5c7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.788+0000 7f5c7b7fe700 1 -- 172.21.15.46:0/27625827 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c018070 con 0x7f5c8419ba60 2024-03-18T20:44:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.788+0000 7f5c8c32a700 1 -- 172.21.15.46:0/27625827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c841a3d10 con 0x7f5c8419ba60 2024-03-18T20:44:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.788+0000 7f5c7b7fe700 1 -- 172.21.15.46:0/27625827 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c7c00f040 con 0x7f5c8419ba60 2024-03-18T20:44:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.788+0000 7f5c7b7fe700 1 -- 172.21.15.46:0/27625827 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c00cd00 con 0x7f5c8419ba60 2024-03-18T20:44:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.788+0000 7f5c8c32a700 1 -- 172.21.15.46:0/27625827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c841a41a0 con 0x7f5c8419ba60 2024-03-18T20:44:44.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:44:44.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:44:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.790+0000 7f5c7b7fe700 1 -- 172.21.15.46:0/27625827 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5c7c003a00 con 0x7f5c8419ba60 2024-03-18T20:44:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.791+0000 7f5c7b7fe700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5c700a1290 0x7f5c700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.791+0000 7f5c7b7fe700 1 -- 172.21.15.46:0/27625827 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5c7c0cc840 con 0x7f5c8419ba60 2024-03-18T20:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.791+0000 7f5c8a0c6700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5c700a1290 0x7f5c700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.792+0000 7f5c8a0c6700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5c700a1290 0x7f5c700a3750 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000b2e0 tx=0x7f5c80005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.792+0000 7f5c984bfb80 1 Processor -- start 2024-03-18T20:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.792+0000 7f5c984bfb80 1 -- start start 2024-03-18T20:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.792+0000 7f5c984bfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b5185570 0x5637b5185950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.792+0000 7f5c984bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185e90 0x5637b5194c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.792+0000 7f5c984bfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 0x5637b51975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.792+0000 7f5c984bfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b50df930 con 0x5637b51951c0 2024-03-18T20:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.792+0000 7f5c984bfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b50dd2f0 con 0x5637b5185570 2024-03-18T20:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.792+0000 7f5c984bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b50d8be0 con 0x5637b5185e90 2024-03-18T20:44:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.792+0000 7f5c8a8c7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 0x5637b51975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.792+0000 7f5c8a8c7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 0x5637b51975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40292/0 (socket says 172.21.15.46:40292) 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c898c5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185e90 0x5637b5194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/994593763 learned_addr learned my addr 172.21.15.46:0/994593763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c8a0c6700 1 --2- 172.21.15.46:0/994593763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b5185570 0x5637b5185950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/994593763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185e90 msgr2=0x5637b5194c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/994593763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185e90 0x5637b5194c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/994593763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b5185570 msgr2=0x5637b5185950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/994593763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b5185570 0x5637b5185950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/994593763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637b4f5a350 con 0x5637b51951c0 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c898c5700 1 --2- 172.21.15.46:0/994593763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185e90 0x5637b5194c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/994593763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 0x5637b51975b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0a1820 tx=0x7f5c7c0994a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76939763cd5bd9d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c78da8700 1 -- 172.21.15.46:0/994593763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c004ad0 con 0x5637b51951c0 2024-03-18T20:44:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c78da8700 1 -- 172.21.15.46:0/994593763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c7c01b320 con 0x5637b51951c0 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.793+0000 7f5c78da8700 1 -- 172.21.15.46:0/994593763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c099bc0 con 0x5637b51951c0 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.794+0000 7f5c984bfb80 1 -- 172.21.15.46:0/994593763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 msgr2=0x5637b51975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.794+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/994593763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 0x5637b51975b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0a1820 tx=0x7f5c7c0994a0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.794+0000 7f5c984bfb80 1 -- 172.21.15.46:0/994593763 shutdown_connections 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.794+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/994593763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 0x5637b51975b0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.794+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/994593763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b5185570 0x5637b5185950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.794+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/994593763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185e90 0x5637b5194c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.794+0000 7f5c984bfb80 1 -- 172.21.15.46:0/994593763 >> 172.21.15.46:0/994593763 conn(0x5637b50fc1c0 msgr2=0x5637b5181a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.794+0000 7f5c984bfb80 1 -- 172.21.15.46:0/994593763 shutdown_connections 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.794+0000 7f5c984bfb80 1 -- 172.21.15.46:0/994593763 wait complete. 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.794+0000 7f5c984bfb80 1 -- start start 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c984bfb80 1 Processor -- start 2024-03-18T20:44:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c984bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185570 0x5637b5194c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c984bfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b5185e90 0x5637b5191ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c984bfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b51951c0 0x5637b5192610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c984bfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b5074d20 con 0x5637b5185e90 2024-03-18T20:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c984bfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b50749c0 con 0x5637b51951c0 2024-03-18T20:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c984bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b4f5c640 con 0x5637b5185570 2024-03-18T20:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c984bfb80 10 client.0 _ll_register_callbacks cb 0x5637b4ee0340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c898c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b5185e90 0x5637b5191ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c898c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b5185e90 0x5637b5191ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40300/0 (socket says 172.21.15.46:40300) 2024-03-18T20:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c898c5700 1 -- 172.21.15.46:0/4127283902 learned_addr learned my addr 172.21.15.46:0/4127283902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c8a0c6700 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185570 0x5637b5194c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.795+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b51951c0 0x5637b5192610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c898c5700 1 -- 172.21.15.46:0/4127283902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185570 msgr2=0x5637b5194c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c898c5700 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185570 0x5637b5194c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c898c5700 1 -- 172.21.15.46:0/4127283902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b51951c0 msgr2=0x5637b5192610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c898c5700 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b51951c0 0x5637b5192610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c898c5700 1 -- 172.21.15.46:0/4127283902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c740098e0 con 0x5637b5185e90 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c898c5700 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b5185e90 0x5637b5191ed0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5c74012980 tx=0x7f5c74012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c4b328b48c375b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c6dffb700 1 -- 172.21.15.46:0/4127283902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74017070 con 0x5637b5185e90 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c6dffb700 1 -- 172.21.15.46:0/4127283902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5c74004030 con 0x5637b5185e90 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c984bfb80 1 -- 172.21.15.46:0/4127283902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637b51796c0 con 0x5637b5185e90 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c6dffb700 1 -- 172.21.15.46:0/4127283902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c74004230 con 0x5637b5185e90 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c984bfb80 10 client.18507 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.796+0000 7f5c984bfb80 1 -- 172.21.15.46:0/4127283902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f5c7c084040 con 0x5637b5185e90 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.797+0000 7f5c6dffb700 1 -- 172.21.15.46:0/4127283902 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5637b51796c0 con 0x5637b5185e90 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.797+0000 7f5c6dffb700 1 -- 172.21.15.46:0/4127283902 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5c7400f670 con 0x5637b5185e90 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.797+0000 7f5c984bfb80 10 client.18507 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.797+0000 7f5c984bfb80 10 client.18507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.797+0000 7f5c984bfb80 20 client.18507 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.797+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5637b51a4d50 0x5637b51a7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.797+0000 7f5c984bfb80 4 client.18507 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.797+0000 7f5c984bfb80 1 -- 172.21.15.46:0/4127283902 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637b50d16a0 con 0x5637b51a4d50 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.797+0000 7f5c8a0c6700 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5637b51a4d50 0x5637b51a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.798+0000 7f5c6dffb700 10 client.18507 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.798+0000 7f5c8a0c6700 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5637b51a4d50 0x5637b51a7170 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09da5dab9738303 server_cookie=de0daa0cadfb4305 in_seq=0 out_seq=0 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.798+0000 7f5c6dffb700 1 -- 172.21.15.46:0/4127283902 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5637b50d16a0 con 0x5637b51a4d50 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.798+0000 7f5c6dffb700 10 client.18507 handle_command_reply: tid=0 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.798+0000 7f5c984bfb80 1 client.18507 shutdown 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.798+0000 7f5c984bfb80 1 -- 172.21.15.46:0/4127283902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b5185e90 msgr2=0x5637b5191ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.798+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b5185e90 0x5637b5191ed0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5c74012980 tx=0x7f5c74012c90 comp rx=0 tx=0).stop 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.799+0000 7f5c984bfb80 1 -- 172.21.15.46:0/4127283902 shutdown_connections 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.799+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5637b51a4d50 0x5637b51a7170 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.799+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b51951c0 0x5637b5192610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.799+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185570 0x5637b5194c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.799+0000 7f5c6dffb700 0 client.18507 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.799+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/4127283902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b5185e90 0x5637b5191ed0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.799+0000 7f5c984bfb80 1 -- 172.21.15.46:0/4127283902 >> 172.21.15.46:0/4127283902 conn(0x5637b50fc1c0 msgr2=0x5637b5181650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.799+0000 7f5c984bfb80 1 -- 172.21.15.46:0/4127283902 shutdown_connections 2024-03-18T20:44:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.799+0000 7f5c984bfb80 1 -- 172.21.15.46:0/4127283902 wait complete. 2024-03-18T20:44:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.799+0000 7f5c984bfb80 20 client.18507 trim_cache size 0 max 16384 2024-03-18T20:44:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.814+0000 7f5c984bfb80 1 Processor -- start 2024-03-18T20:44:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.814+0000 7f5c984bfb80 1 -- start start 2024-03-18T20:44:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c984bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185570 0x5637b51925d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c984bfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b5185e90 0x5637b5191e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c984bfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b51951c0 0x5637b519bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c984bfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b5182fe0 con 0x5637b5185e90 2024-03-18T20:44:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c984bfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b519c2a0 con 0x5637b51951c0 2024-03-18T20:44:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c984bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b5192c10 con 0x5637b5185570 2024-03-18T20:44:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c898c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b5185e90 0x5637b5191e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c8a0c6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185570 0x5637b51925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c8a8c7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b51951c0 0x5637b519bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c8a0c6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185570 0x5637b51925d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43282/0 (socket says 172.21.15.46:43282) 2024-03-18T20:44:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c8a8c7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b51951c0 0x5637b519bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:58854/0 (socket says 172.21.15.46:58854) 2024-03-18T20:44:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c8a0c6700 1 -- 172.21.15.46:0/487487619 learned_addr learned my addr 172.21.15.46:0/487487619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/487487619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185570 msgr2=0x5637b51925d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/487487619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185570 0x5637b51925d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/487487619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b5185e90 msgr2=0x5637b5191e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/487487619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b5185e90 0x5637b5191e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/487487619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637b4f06b00 con 0x5637b51951c0 2024-03-18T20:44:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.815+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/487487619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b51951c0 0x5637b519bd30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0a1820 tx=0x7f5c7c0993c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c8491a4f4daddc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.816+0000 7f5c795a9700 1 -- 172.21.15.46:0/487487619 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c004ad0 con 0x5637b51951c0 2024-03-18T20:44:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.816+0000 7f5c795a9700 1 -- 172.21.15.46:0/487487619 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c7c0b2260 con 0x5637b51951c0 2024-03-18T20:44:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.816+0000 7f5c795a9700 1 -- 172.21.15.46:0/487487619 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c098e50 con 0x5637b51951c0 2024-03-18T20:44:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.816+0000 7f5c984bfb80 1 -- 172.21.15.46:0/487487619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b51951c0 msgr2=0x5637b519bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.816+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/487487619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b51951c0 0x5637b519bd30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0a1820 tx=0x7f5c7c0993c0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.816+0000 7f5c984bfb80 1 -- 172.21.15.46:0/487487619 shutdown_connections 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.816+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/487487619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b51951c0 0x5637b519bd30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.816+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/487487619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185570 0x5637b51925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.816+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/487487619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b5185e90 0x5637b5191e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.816+0000 7f5c984bfb80 1 -- 172.21.15.46:0/487487619 >> 172.21.15.46:0/487487619 conn(0x5637b50fc1c0 msgr2=0x5637b5191760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c984bfb80 1 -- 172.21.15.46:0/487487619 shutdown_connections 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c984bfb80 1 -- 172.21.15.46:0/487487619 wait complete. 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c984bfb80 1 -- start start 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c984bfb80 1 Processor -- start 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c984bfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b5185570 0x5637b5199d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c984bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185e90 0x5637b519a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c984bfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 0x5637b51f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c984bfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b519aa10 con 0x5637b51951c0 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c984bfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b519ab80 con 0x5637b5185570 2024-03-18T20:44:44.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c984bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b51f80f0 con 0x5637b5185e90 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c984bfb80 10 client.0 _ll_register_callbacks cb 0x7f5c74012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c8a8c7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 0x5637b51f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c8a8c7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 0x5637b51f7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40304/0 (socket says 172.21.15.46:40304) 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.817+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/2970893115 learned_addr learned my addr 172.21.15.46:0/2970893115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c898c5700 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185e90 0x5637b519a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c8a0c6700 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b5185570 0x5637b5199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/2970893115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185e90 msgr2=0x5637b519a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185e90 0x5637b519a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/2970893115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b5185570 msgr2=0x5637b5199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b5185570 0x5637b5199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c8a8c7700 1 -- 172.21.15.46:0/2970893115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637b4f5a350 con 0x5637b51951c0 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c8a8c7700 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 0x5637b51f7b80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c6f0 tx=0x7f5c7c0994a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a45195393efbb4b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c6dffb700 1 -- 172.21.15.46:0/2970893115 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c09d790 con 0x5637b51951c0 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c6dffb700 1 -- 172.21.15.46:0/2970893115 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5c7c09a060 con 0x5637b51951c0 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c984bfb80 1 -- 172.21.15.46:0/2970893115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c7c01b320 con 0x5637b51951c0 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c6dffb700 1 -- 172.21.15.46:0/2970893115 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c7c099960 con 0x5637b51951c0 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c984bfb80 10 client.18513 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.818+0000 7f5c984bfb80 1 -- 172.21.15.46:0/2970893115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5637b509ee40 con 0x5637b51951c0 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.819+0000 7f5c6dffb700 1 -- 172.21.15.46:0/2970893115 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c7c01b320 con 0x5637b51951c0 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.819+0000 7f5c6dffb700 1 -- 172.21.15.46:0/2970893115 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5c7c0a19e0 con 0x5637b51951c0 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.819+0000 7f5c984bfb80 10 client.18513 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.819+0000 7f5c984bfb80 10 client.18513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.819+0000 7f5c984bfb80 20 client.18513 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.819+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5637b51f93d0 0x5637b51fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.819+0000 7f5c984bfb80 4 client.18513 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:44:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.819+0000 7f5c984bfb80 1 -- 172.21.15.46:0/2970893115 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5637b4f27c90 con 0x5637b51f93d0 2024-03-18T20:44:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.819+0000 7f5c8a0c6700 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5637b51f93d0 0x5637b51fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.820+0000 7f5c8a0c6700 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5637b51f93d0 0x5637b51fb7c0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e33aa886b59172 server_cookie=228024338b917813 in_seq=0 out_seq=0 2024-03-18T20:44:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.820+0000 7f5c6dffb700 10 client.18513 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.850+0000 7f5c6dffb700 1 -- 172.21.15.46:0/2970893115 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5637b4f27c90 con 0x5637b51f93d0 2024-03-18T20:44:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.850+0000 7f5c6dffb700 10 client.18513 handle_command_reply: tid=0 2024-03-18T20:44:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.850+0000 7f5c984bfb80 1 client.18513 shutdown 2024-03-18T20:44:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.850+0000 7f5c984bfb80 1 -- 172.21.15.46:0/2970893115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 msgr2=0x5637b51f7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.850+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 0x5637b51f7b80 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c6f0 tx=0x7f5c7c0994a0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.850+0000 7f5c984bfb80 1 -- 172.21.15.46:0/2970893115 shutdown_connections 2024-03-18T20:44:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.850+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5637b51f93d0 0x5637b51fb7c0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.850+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637b51951c0 0x5637b51f7b80 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.850+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5637b5185570 0x5637b5199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.850+0000 7f5c6dffb700 0 client.18513 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:44.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.850+0000 7f5c984bfb80 1 --2- 172.21.15.46:0/2970893115 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5637b5185e90 0x5637b519a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.850+0000 7f5c984bfb80 1 -- 172.21.15.46:0/2970893115 >> 172.21.15.46:0/2970893115 conn(0x5637b50fc1c0 msgr2=0x5637b51907a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c984bfb80 1 -- 172.21.15.46:0/2970893115 shutdown_connections 2024-03-18T20:44:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c984bfb80 1 -- 172.21.15.46:0/2970893115 wait complete. 2024-03-18T20:44:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c984bfb80 20 client.18513 trim_cache size 0 max 16384 2024-03-18T20:44:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:44:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:44:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:44:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:44:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c8c32a700 1 -- 172.21.15.46:0/27625827 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5c700a1290 msgr2=0x7f5c700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c8c32a700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5c700a1290 0x7f5c700a3750 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000b2e0 tx=0x7f5c80005960 comp rx=0 tx=0).stop 2024-03-18T20:44:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c8c32a700 1 -- 172.21.15.46:0/27625827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8419ba60 msgr2=0x7f5c8419bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c8c32a700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8419ba60 0x7f5c8419bee0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00ef30 tx=0x7f5c7c00c3a0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c8c32a700 1 -- 172.21.15.46:0/27625827 shutdown_connections 2024-03-18T20:44:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c8c32a700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5c700a1290 0x7f5c700a3750 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c8c32a700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8419ba60 0x7f5c8419bee0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c8c32a700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c84100f10 0x7f5c84197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c8c32a700 1 --2- 172.21.15.46:0/27625827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c84106030 0x7f5c841976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c8c32a700 1 -- 172.21.15.46:0/27625827 >> 172.21.15.46:0/27625827 conn(0x7f5c840fc7b0 msgr2=0x7f5c840fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c8c32a700 1 -- 172.21.15.46:0/27625827 shutdown_connections 2024-03-18T20:44:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:44.851+0000 7f5c8c32a700 1 -- 172.21.15.46:0/27625827 wait complete. 2024-03-18T20:44:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.943+0000 7f24a759e700 1 -- 172.21.15.46:0/3598402150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a8101760 msgr2=0x7f24a81057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.943+0000 7f24a759e700 1 --2- 172.21.15.46:0/3598402150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a8101760 0x7f24a81057a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f249c009990 tx=0x7f249c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.943+0000 7f24a759e700 1 -- 172.21.15.46:0/3598402150 shutdown_connections 2024-03-18T20:44:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.943+0000 7f24a759e700 1 --2- 172.21.15.46:0/3598402150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24a8105ed0 0x7f24a810da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.943+0000 7f24a759e700 1 --2- 172.21.15.46:0/3598402150 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a8100db0 0x7f24a8101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.943+0000 7f24a759e700 1 --2- 172.21.15.46:0/3598402150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a8101760 0x7f24a81057a0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.943+0000 7f24a759e700 1 -- 172.21.15.46:0/3598402150 >> 172.21.15.46:0/3598402150 conn(0x7f24a80fc670 msgr2=0x7f24a80fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.944+0000 7f24a759e700 1 -- 172.21.15.46:0/3598402150 shutdown_connections 2024-03-18T20:44:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.944+0000 7f24a759e700 1 -- 172.21.15.46:0/3598402150 wait complete. 2024-03-18T20:44:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.944+0000 7f24a759e700 1 Processor -- start 2024-03-18T20:44:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.944+0000 7f24a759e700 1 -- start start 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a759e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a8100db0 0x7f24a80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a759e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24a8105ed0 0x7f24a80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a8073b30 0x7f24a81a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a81a57c0 con 0x7f24a8073b30 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a759e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a81a5930 con 0x7f24a8105ed0 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a759e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a81a5af0 con 0x7f24a8100db0 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a8073b30 0x7f24a81a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a8100db0 0x7f24a80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a8100db0 0x7f24a80730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43310/0 (socket says 172.21.15.46:43310) 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a659c700 1 -- 172.21.15.46:0/1345166032 learned_addr learned my addr 172.21.15.46:0/1345166032 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a5d9b700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24a8105ed0 0x7f24a80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a6d9d700 1 -- 172.21.15.46:0/1345166032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a8100db0 msgr2=0x7f24a80730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a6d9d700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a8100db0 0x7f24a80730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a6d9d700 1 -- 172.21.15.46:0/1345166032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24a8105ed0 msgr2=0x7f24a80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a6d9d700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24a8105ed0 0x7f24a80735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a6d9d700 1 -- 172.21.15.46:0/1345166032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c0095f0 con 0x7f24a8073b30 2024-03-18T20:44:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.945+0000 7f24a6d9d700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a8073b30 0x7f24a81a51f0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f24a0007f10 tx=0x7f24a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.946+0000 7f24977fe700 1 -- 172.21.15.46:0/1345166032 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0009ea0 con 0x7f24a8073b30 2024-03-18T20:44:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.946+0000 7f24a759e700 1 -- 172.21.15.46:0/1345166032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a81a5dd0 con 0x7f24a8073b30 2024-03-18T20:44:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.946+0000 7f24977fe700 1 -- 172.21.15.46:0/1345166032 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24a000f040 con 0x7f24a8073b30 2024-03-18T20:44:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.946+0000 7f24977fe700 1 -- 172.21.15.46:0/1345166032 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0004350 con 0x7f24a8073b30 2024-03-18T20:44:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.946+0000 7f24a759e700 1 -- 172.21.15.46:0/1345166032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a81a6290 con 0x7f24a8073b30 2024-03-18T20:44:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.948+0000 7f24977fe700 1 -- 172.21.15.46:0/1345166032 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f24a000db20 con 0x7f24a8073b30 2024-03-18T20:44:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.949+0000 7f24977fe700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f24900a1290 0x7f24900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.949+0000 7f24a659c700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f24900a1290 0x7f24900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.949+0000 7f24977fe700 1 -- 172.21.15.46:0/1345166032 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f24a00cc540 con 0x7f24a8073b30 2024-03-18T20:44:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.950+0000 7f24a659c700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f24900a1290 0x7f24900a3750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2498005950 tx=0x7f249800b500 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.950+0000 7f24b9195b80 1 Processor -- start 2024-03-18T20:44:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.950+0000 7f24b9195b80 1 -- start start 2024-03-18T20:44:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.950+0000 7f24b9195b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe0fca0 0x560a9fe10080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.950+0000 7f24b9195b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe105c0 0x560a9fe1af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.950+0000 7f24b9195b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe1b450 0x560a9fe1d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.950+0000 7f24b9195b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9fd69990 con 0x560a9fe105c0 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.950+0000 7f24b9195b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9fd67350 con 0x560a9fe0fca0 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.950+0000 7f24b9195b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9fd62c40 con 0x560a9fe1b450 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.951+0000 7f24a5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe105c0 0x560a9fe1af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.951+0000 7f24a5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe105c0 0x560a9fe1af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40334/0 (socket says 172.21.15.46:40334) 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.951+0000 7f24a659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe0fca0 0x560a9fe10080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.951+0000 7f24a5d9b700 1 -- 172.21.15.46:0/15742566 learned_addr learned my addr 172.21.15.46:0/15742566 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.951+0000 7f24a6d9d700 1 --2- 172.21.15.46:0/15742566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe1b450 0x560a9fe1d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.951+0000 7f24a659c700 1 -- 172.21.15.46:0/15742566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe1b450 msgr2=0x560a9fe1d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.951+0000 7f24a659c700 1 --2- 172.21.15.46:0/15742566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe1b450 0x560a9fe1d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.951+0000 7f24a659c700 1 -- 172.21.15.46:0/15742566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe105c0 msgr2=0x560a9fe1af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.951+0000 7f24a659c700 1 --2- 172.21.15.46:0/15742566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe105c0 0x560a9fe1af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.951+0000 7f24a659c700 1 -- 172.21.15.46:0/15742566 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a9fbe43b0 con 0x560a9fe0fca0 2024-03-18T20:44:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.951+0000 7f24a5d9b700 1 --2- 172.21.15.46:0/15742566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe105c0 0x560a9fe1af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.951+0000 7f24a659c700 1 --2- 172.21.15.46:0/15742566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe0fca0 0x560a9fe10080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2498010a40 tx=0x7f2498013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc436956aa676a93 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.952+0000 7f2494ff9700 1 -- 172.21.15.46:0/15742566 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498011d30 con 0x560a9fe0fca0 2024-03-18T20:44:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.952+0000 7f2494ff9700 1 -- 172.21.15.46:0/15742566 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2498015070 con 0x560a9fe0fca0 2024-03-18T20:44:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.952+0000 7f2494ff9700 1 -- 172.21.15.46:0/15742566 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249801c5f0 con 0x560a9fe0fca0 2024-03-18T20:44:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.952+0000 7f24b9195b80 1 -- 172.21.15.46:0/15742566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe0fca0 msgr2=0x560a9fe10080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.952+0000 7f24b9195b80 1 --2- 172.21.15.46:0/15742566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe0fca0 0x560a9fe10080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2498010a40 tx=0x7f2498013000 comp rx=0 tx=0).stop 2024-03-18T20:44:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.952+0000 7f24b9195b80 1 -- 172.21.15.46:0/15742566 shutdown_connections 2024-03-18T20:44:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.952+0000 7f24b9195b80 1 --2- 172.21.15.46:0/15742566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe1b450 0x560a9fe1d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.952+0000 7f24b9195b80 1 --2- 172.21.15.46:0/15742566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe0fca0 0x560a9fe10080 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.952+0000 7f24b9195b80 1 --2- 172.21.15.46:0/15742566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe105c0 0x560a9fe1af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.952+0000 7f24b9195b80 1 -- 172.21.15.46:0/15742566 >> 172.21.15.46:0/15742566 conn(0x560a9fd86220 msgr2=0x560a9fe0c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24b9195b80 1 -- 172.21.15.46:0/15742566 shutdown_connections 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24b9195b80 1 -- 172.21.15.46:0/15742566 wait complete. 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24b9195b80 1 -- start start 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24b9195b80 1 Processor -- start 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24b9195b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 0x560a9fe182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24b9195b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 0x560a9fe187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24b9195b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe1b450 0x560a9fe18d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24b9195b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9fcfed80 con 0x560a9fe0fca0 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24b9195b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9fcfea20 con 0x560a9fe1b450 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24b9195b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9fbe66a0 con 0x560a9fe105c0 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24b9195b80 10 client.0 _ll_register_callbacks cb 0x560a9fb6a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24a659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 0x560a9fe182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.953+0000 7f24a659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 0x560a9fe182b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40350/0 (socket says 172.21.15.46:40350) 2024-03-18T20:44:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f24a659c700 1 -- 172.21.15.46:0/4163664618 learned_addr learned my addr 172.21.15.46:0/4163664618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f24a5d9b700 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 0x560a9fe187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f24a6d9d700 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe1b450 0x560a9fe18d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f24a659c700 1 -- 172.21.15.46:0/4163664618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 msgr2=0x560a9fe187f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f24a659c700 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 0x560a9fe187f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f24a659c700 1 -- 172.21.15.46:0/4163664618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe1b450 msgr2=0x560a9fe18d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f24a659c700 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe1b450 0x560a9fe18d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f24a659c700 1 -- 172.21.15.46:0/4163664618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a810e2c0 con 0x560a9fe0fca0 2024-03-18T20:44:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f24a659c700 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 0x560a9fe182b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2498010bc0 tx=0x7f2498012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c5328b29e083cb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f248dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f248dffb700 1 -- 172.21.15.46:0/4163664618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249802b070 con 0x560a9fe0fca0 2024-03-18T20:44:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f248dffb700 1 -- 172.21.15.46:0/4163664618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f2498017070 con 0x560a9fe0fca0 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f24b9195b80 1 -- 172.21.15.46:0/4163664618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a9fe03c30 con 0x560a9fe0fca0 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f248dffb700 1 -- 172.21.15.46:0/4163664618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f249801ebe0 con 0x560a9fe0fca0 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.954+0000 7f24b9195b80 10 client.18525 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.955+0000 7f24b9195b80 1 -- 172.21.15.46:0/4163664618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f2498010c50 con 0x560a9fe0fca0 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.955+0000 7f248dffb700 1 -- 172.21.15.46:0/4163664618 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f249803c020 con 0x560a9fe0fca0 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.955+0000 7f248dffb700 1 -- 172.21.15.46:0/4163664618 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2498003840 con 0x560a9fe0fca0 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.955+0000 7f24b9195b80 10 client.18525 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.955+0000 7f24b9195b80 10 client.18525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.955+0000 7f24b9195b80 20 client.18525 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.955+0000 7f24b9195b80 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560a9fe2ae70 0x560a9fe2d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.955+0000 7f24b9195b80 4 client.18525 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.955+0000 7f24b9195b80 1 -- 172.21.15.46:0/4163664618 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a9fd5b700 con 0x560a9fe2ae70 2024-03-18T20:44:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.955+0000 7f24a5d9b700 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560a9fe2ae70 0x560a9fe2d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f248dffb700 10 client.18525 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f24a5d9b700 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560a9fe2ae70 0x560a9fe2d290 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe4c390e8346f57 server_cookie=428b13706727393a in_seq=0 out_seq=0 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f248dffb700 1 -- 172.21.15.46:0/4163664618 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560a9fd5b700 con 0x560a9fe2ae70 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f248dffb700 10 client.18525 handle_command_reply: tid=0 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f24b9195b80 1 client.18525 shutdown 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f24b9195b80 1 -- 172.21.15.46:0/4163664618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 msgr2=0x560a9fe182b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f24b9195b80 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 0x560a9fe182b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2498010bc0 tx=0x7f2498012d50 comp rx=0 tx=0).stop 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f24b9195b80 1 -- 172.21.15.46:0/4163664618 shutdown_connections 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f24b9195b80 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560a9fe2ae70 0x560a9fe2d290 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f24b9195b80 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 0x560a9fe182b0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f24b9195b80 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe1b450 0x560a9fe18d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f248dffb700 0 client.18525 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f24b9195b80 1 --2- 172.21.15.46:0/4163664618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 0x560a9fe187f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f24b9195b80 1 -- 172.21.15.46:0/4163664618 >> 172.21.15.46:0/4163664618 conn(0x560a9fd86220 msgr2=0x560a9fe0bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.956+0000 7f24b9195b80 1 -- 172.21.15.46:0/4163664618 shutdown_connections 2024-03-18T20:44:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.957+0000 7f24b9195b80 1 -- 172.21.15.46:0/4163664618 wait complete. 2024-03-18T20:44:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.957+0000 7f24b9195b80 20 client.18525 trim_cache size 0 max 16384 2024-03-18T20:44:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.974+0000 7f24b9195b80 1 Processor -- start 2024-03-18T20:44:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.974+0000 7f24b9195b80 1 -- start start 2024-03-18T20:44:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24b9195b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 0x560a9fe0c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24b9195b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 0x560a9fe18b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24b9195b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe1b450 0x560a9fe17160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24b9195b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9fe0ccd0 con 0x560a9fe0fca0 2024-03-18T20:44:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24b9195b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9fe102d0 con 0x560a9fe1b450 2024-03-18T20:44:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24b9195b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9fe13100 con 0x560a9fe105c0 2024-03-18T20:44:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24a659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 0x560a9fe0c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24a659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 0x560a9fe0c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40354/0 (socket says 172.21.15.46:40354) 2024-03-18T20:44:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24a659c700 1 -- 172.21.15.46:0/2375685557 learned_addr learned my addr 172.21.15.46:0/2375685557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24a6d9d700 1 --2- 172.21.15.46:0/2375685557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe1b450 0x560a9fe17160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24a5d9b700 1 --2- 172.21.15.46:0/2375685557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 0x560a9fe18b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24a659c700 1 -- 172.21.15.46:0/2375685557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 msgr2=0x560a9fe18b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24a659c700 1 --2- 172.21.15.46:0/2375685557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 0x560a9fe18b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24a659c700 1 -- 172.21.15.46:0/2375685557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe1b450 msgr2=0x560a9fe17160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24a659c700 1 --2- 172.21.15.46:0/2375685557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe1b450 0x560a9fe17160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24a659c700 1 -- 172.21.15.46:0/2375685557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a9fb90b60 con 0x560a9fe0fca0 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24a659c700 1 --2- 172.21.15.46:0/2375685557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 0x560a9fe0c360 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f2498010f40 tx=0x7f2498005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620c16fd0743c8f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24957fa700 1 -- 172.21.15.46:0/2375685557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249802b040 con 0x560a9fe0fca0 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.975+0000 7f24957fa700 1 -- 172.21.15.46:0/2375685557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f249803c070 con 0x560a9fe0fca0 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.976+0000 7f24957fa700 1 -- 172.21.15.46:0/2375685557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498005500 con 0x560a9fe0fca0 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.976+0000 7f24b9195b80 1 -- 172.21.15.46:0/2375685557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 msgr2=0x560a9fe0c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.976+0000 7f24b9195b80 1 --2- 172.21.15.46:0/2375685557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 0x560a9fe0c360 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f2498010f40 tx=0x7f2498005a00 comp rx=0 tx=0).stop 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.976+0000 7f24b9195b80 1 -- 172.21.15.46:0/2375685557 shutdown_connections 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.976+0000 7f24b9195b80 1 --2- 172.21.15.46:0/2375685557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe0fca0 0x560a9fe0c360 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.976+0000 7f24b9195b80 1 --2- 172.21.15.46:0/2375685557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe1b450 0x560a9fe17160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.976+0000 7f24b9195b80 1 --2- 172.21.15.46:0/2375685557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 0x560a9fe18b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.976+0000 7f24b9195b80 1 -- 172.21.15.46:0/2375685557 >> 172.21.15.46:0/2375685557 conn(0x560a9fd86220 msgr2=0x560a9fe16a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.976+0000 7f24b9195b80 1 -- 172.21.15.46:0/2375685557 shutdown_connections 2024-03-18T20:44:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.976+0000 7f24b9195b80 1 -- 172.21.15.46:0/2375685557 wait complete. 2024-03-18T20:44:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.976+0000 7f24b9195b80 1 -- start start 2024-03-18T20:44:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24b9195b80 1 Processor -- start 2024-03-18T20:44:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24b9195b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe0fca0 0x560a9fe1fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24b9195b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 0x560a9fe20420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24b9195b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe1b450 0x560a9fe7dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24b9195b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9fe20b30 con 0x560a9fe1b450 2024-03-18T20:44:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24b9195b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9fe7e1a0 con 0x560a9fe0fca0 2024-03-18T20:44:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24b9195b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9fe7e310 con 0x560a9fe105c0 2024-03-18T20:44:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24b9195b80 10 client.0 _ll_register_callbacks cb 0x7f249c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24a6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe1b450 0x560a9fe7dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24a659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe0fca0 0x560a9fe1fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24a6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe1b450 0x560a9fe7dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40360/0 (socket says 172.21.15.46:40360) 2024-03-18T20:44:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24a6d9d700 1 -- 172.21.15.46:0/852534143 learned_addr learned my addr 172.21.15.46:0/852534143 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24a5d9b700 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 0x560a9fe20420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24a659c700 1 -- 172.21.15.46:0/852534143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 msgr2=0x560a9fe20420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24a659c700 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 0x560a9fe20420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24a659c700 1 -- 172.21.15.46:0/852534143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe1b450 msgr2=0x560a9fe7dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:44.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24a659c700 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe1b450 0x560a9fe7dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24a659c700 1 -- 172.21.15.46:0/852534143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2498010c50 con 0x560a9fe0fca0 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.977+0000 7f24a6d9d700 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe1b450 0x560a9fe7dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f24a659c700 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe0fca0 0x560a9fe1fee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2498003cf0 tx=0x7f2498003930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f67c6fc36d2d27fb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f248dffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f248dffb700 1 -- 172.21.15.46:0/852534143 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249802b070 con 0x560a9fe0fca0 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f248dffb700 1 -- 172.21.15.46:0/852534143 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f2498017070 con 0x560a9fe0fca0 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f24b9195b80 1 -- 172.21.15.46:0/852534143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2498015070 con 0x560a9fe0fca0 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f248dffb700 1 -- 172.21.15.46:0/852534143 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24980055c0 con 0x560a9fe0fca0 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f24b9195b80 10 client.28178 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f24b9195b80 1 -- 172.21.15.46:0/852534143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560a9fd28ea0 con 0x560a9fe0fca0 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f248dffb700 1 -- 172.21.15.46:0/852534143 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2498015070 con 0x560a9fe0fca0 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f248dffb700 1 -- 172.21.15.46:0/852534143 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2498048070 con 0x560a9fe0fca0 2024-03-18T20:44:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f24b9195b80 10 client.28178 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f24b9195b80 10 client.28178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f24b9195b80 20 client.28178 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f24b9195b80 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560a9fe7f610 0x560a9fe81a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f24b9195b80 4 client.28178 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:44:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.978+0000 7f24b9195b80 1 -- 172.21.15.46:0/852534143 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560a9fbb1cf0 con 0x560a9fe7f610 2024-03-18T20:44:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24a5d9b700 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560a9fe7f610 0x560a9fe81a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24a5d9b700 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560a9fe7f610 0x560a9fe81a00 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dde2b28124ffc90 server_cookie=71b9dee5fccbdd62 in_seq=0 out_seq=0 2024-03-18T20:44:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f248dffb700 10 client.28178 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f248dffb700 1 -- 172.21.15.46:0/852534143 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560a9fbb1cf0 con 0x560a9fe7f610 2024-03-18T20:44:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f248dffb700 10 client.28178 handle_command_reply: tid=0 2024-03-18T20:44:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24b9195b80 1 client.28178 shutdown 2024-03-18T20:44:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24b9195b80 1 -- 172.21.15.46:0/852534143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe0fca0 msgr2=0x560a9fe1fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24b9195b80 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe0fca0 0x560a9fe1fee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2498003cf0 tx=0x7f2498003930 comp rx=0 tx=0).stop 2024-03-18T20:44:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24b9195b80 1 -- 172.21.15.46:0/852534143 shutdown_connections 2024-03-18T20:44:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24b9195b80 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560a9fe7f610 0x560a9fe81a00 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24b9195b80 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a9fe1b450 0x560a9fe7dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f248dffb700 0 client.28178 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24b9195b80 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a9fe0fca0 0x560a9fe1fee0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24b9195b80 1 --2- 172.21.15.46:0/852534143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a9fe105c0 0x560a9fe20420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24b9195b80 1 -- 172.21.15.46:0/852534143 >> 172.21.15.46:0/852534143 conn(0x560a9fd86220 msgr2=0x560a9fe172e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24b9195b80 1 -- 172.21.15.46:0/852534143 shutdown_connections 2024-03-18T20:44:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24b9195b80 1 -- 172.21.15.46:0/852534143 wait complete. 2024-03-18T20:44:44.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.979+0000 7f24b9195b80 20 client.28178 trim_cache size 0 max 16384 2024-03-18T20:44:44.991 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:44:44.991 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:44:44.991 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:44:44.991 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:44:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.980+0000 7f24a759e700 1 -- 172.21.15.46:0/1345166032 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f24900a1290 msgr2=0x7f24900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.980+0000 7f24a759e700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f24900a1290 0x7f24900a3750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2498005950 tx=0x7f249800b500 comp rx=0 tx=0).stop 2024-03-18T20:44:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.980+0000 7f24a759e700 1 -- 172.21.15.46:0/1345166032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a8073b30 msgr2=0x7f24a81a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.980+0000 7f24a759e700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a8073b30 0x7f24a81a51f0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f24a0007f10 tx=0x7f24a000d3b0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.980+0000 7f24a759e700 1 -- 172.21.15.46:0/1345166032 shutdown_connections 2024-03-18T20:44:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.980+0000 7f24a759e700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f24900a1290 0x7f24900a3750 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.980+0000 7f24a759e700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a8073b30 0x7f24a81a51f0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.980+0000 7f24a759e700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24a8100db0 0x7f24a80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.980+0000 7f24a759e700 1 --2- 172.21.15.46:0/1345166032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24a8105ed0 0x7f24a80735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.980+0000 7f24a759e700 1 -- 172.21.15.46:0/1345166032 >> 172.21.15.46:0/1345166032 conn(0x7f24a80fc670 msgr2=0x7f24a80fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.980+0000 7f24a759e700 1 -- 172.21.15.46:0/1345166032 shutdown_connections 2024-03-18T20:44:44.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:44.980+0000 7f24a759e700 1 -- 172.21.15.46:0/1345166032 wait complete. 2024-03-18T20:44:44.996 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:44:44.996 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:44:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.174+0000 7f0444a21700 1 -- 172.21.15.46:0/1155108701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0440105c00 msgr2=0x7f0440105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.174+0000 7f0444a21700 1 --2- 172.21.15.46:0/1155108701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0440105c00 0x7f0440105fe0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f04300099c0 tx=0x7f0430009cd0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.174+0000 7f0444a21700 1 -- 172.21.15.46:0/1155108701 shutdown_connections 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.174+0000 7f0444a21700 1 --2- 172.21.15.46:0/1155108701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0440105c00 0x7f0440105fe0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.174+0000 7f0444a21700 1 --2- 172.21.15.46:0/1155108701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0440100500 0x7f044010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.174+0000 7f0444a21700 1 --2- 172.21.15.46:0/1155108701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04400ffb40 0x7f04400fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.174+0000 7f0444a21700 1 -- 172.21.15.46:0/1155108701 >> 172.21.15.46:0/1155108701 conn(0x7f04400fb5f0 msgr2=0x7f04400fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.174+0000 7f0444a21700 1 -- 172.21.15.46:0/1155108701 shutdown_connections 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.174+0000 7f0444a21700 1 -- 172.21.15.46:0/1155108701 wait complete. 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.175+0000 7f0444a21700 1 Processor -- start 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.175+0000 7f0444a21700 1 -- start start 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.175+0000 7f0444a21700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04400ffb40 0x7f04401017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.175+0000 7f0444a21700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0440100500 0x7f0440101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.175+0000 7f0444a21700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0440105c00 0x7f0440102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.175+0000 7f0444a21700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04401a53a0 con 0x7f0440105c00 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.175+0000 7f0444a21700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04401a5510 con 0x7f04400ffb40 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.175+0000 7f0444a21700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04401a56d0 con 0x7f0440100500 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f043ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0440105c00 0x7f0440102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f043ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0440105c00 0x7f0440102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40382/0 (socket says 172.21.15.46:40382) 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f043ed9d700 1 -- 172.21.15.46:0/4032400397 learned_addr learned my addr 172.21.15.46:0/4032400397 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f043dd9b700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0440100500 0x7f0440101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f043e59c700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04400ffb40 0x7f04401017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f043ed9d700 1 -- 172.21.15.46:0/4032400397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0440100500 msgr2=0x7f0440101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f043ed9d700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0440100500 0x7f0440101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f043ed9d700 1 -- 172.21.15.46:0/4032400397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04400ffb40 msgr2=0x7f04401017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f043ed9d700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04400ffb40 0x7f04401017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f043ed9d700 1 -- 172.21.15.46:0/4032400397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0434009520 con 0x7f0440105c00 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f043ed9d700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0440105c00 0x7f0440102230 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0438007ee0 tx=0x7f043800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f042f7fe700 1 -- 172.21.15.46:0/4032400397 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438017070 con 0x7f0440105c00 2024-03-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f042f7fe700 1 -- 172.21.15.46:0/4032400397 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f043800f040 con 0x7f0440105c00 2024-03-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f0444a21700 1 -- 172.21.15.46:0/4032400397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04300095f0 con 0x7f0440105c00 2024-03-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f042f7fe700 1 -- 172.21.15.46:0/4032400397 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438005580 con 0x7f0440105c00 2024-03-18T20:44:45.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.176+0000 7f0444a21700 1 -- 172.21.15.46:0/4032400397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04401a5ce0 con 0x7f0440105c00 2024-03-18T20:44:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.177+0000 7f0444a21700 1 -- 172.21.15.46:0/4032400397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04401011f0 con 0x7f0440105c00 2024-03-18T20:44:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.179+0000 7f042f7fe700 1 -- 172.21.15.46:0/4032400397 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0438022020 con 0x7f0440105c00 2024-03-18T20:44:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.179+0000 7f042f7fe700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f04280a1290 0x7f04280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.180+0000 7f043e59c700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f04280a1290 0x7f04280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.180+0000 7f042f7fe700 1 -- 172.21.15.46:0/4032400397 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f04380cc3d0 con 0x7f0440105c00 2024-03-18T20:44:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.180+0000 7f043e59c700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f04280a1290 0x7f04280a3750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0430009fd0 tx=0x7f0430019040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.183+0000 7f042f7fe700 1 -- 172.21.15.46:0/4032400397 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f04380957a0 con 0x7f0440105c00 2024-03-18T20:44:45.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.465+0000 7f0444a21700 1 -- 172.21.15.46:0/4032400397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04401a5f00 con 0x7f0440105c00 2024-03-18T20:44:45.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.466+0000 7f042f7fe700 1 -- 172.21.15.46:0/4032400397 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+10910 (secure 0 0 0) 0x7f0438098dc0 con 0x7f0440105c00 2024-03-18T20:44:45.469 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:44:45.469 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:44:15.219685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:44:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.468+0000 7f0444a21700 1 -- 172.21.15.46:0/4032400397 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f04280a1290 msgr2=0x7f04280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.468+0000 7f0444a21700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f04280a1290 0x7f04280a3750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0430009fd0 tx=0x7f0430019040 comp rx=0 tx=0).stop 2024-03-18T20:44:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.468+0000 7f0444a21700 1 -- 172.21.15.46:0/4032400397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0440105c00 msgr2=0x7f0440102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.468+0000 7f0444a21700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0440105c00 0x7f0440102230 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0438007ee0 tx=0x7f043800d3b0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.468+0000 7f0444a21700 1 -- 172.21.15.46:0/4032400397 shutdown_connections 2024-03-18T20:44:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.468+0000 7f0444a21700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f04280a1290 0x7f04280a3750 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.468+0000 7f0444a21700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0440105c00 0x7f0440102230 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.468+0000 7f0444a21700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04400ffb40 0x7f04401017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.468+0000 7f0444a21700 1 --2- 172.21.15.46:0/4032400397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0440100500 0x7f0440101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.468+0000 7f0444a21700 1 -- 172.21.15.46:0/4032400397 >> 172.21.15.46:0/4032400397 conn(0x7f04400fb5f0 msgr2=0x7f04400fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.468+0000 7f0444a21700 1 -- 172.21.15.46:0/4032400397 shutdown_connections 2024-03-18T20:44:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.468+0000 7f0444a21700 1 -- 172.21.15.46:0/4032400397 wait complete. 2024-03-18T20:44:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 47 2024-03-18T20:44:45.485 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:44:45.485 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:44:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.633+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/832538795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a4100820 msgr2=0x7fd5a4104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.633+0000 7fd5a9e1f700 1 --2- 172.21.15.46:0/832538795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a4100820 0x7fd5a4104840 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009990 tx=0x7fd58c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.633+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/832538795 shutdown_connections 2024-03-18T20:44:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.633+0000 7fd5a9e1f700 1 --2- 172.21.15.46:0/832538795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5a4104f70 0x7fd5a410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.633+0000 7fd5a9e1f700 1 --2- 172.21.15.46:0/832538795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5a40ffe70 0x7fd5a4100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.633+0000 7fd5a9e1f700 1 --2- 172.21.15.46:0/832538795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a4100820 0x7fd5a4104840 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.633+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/832538795 >> 172.21.15.46:0/832538795 conn(0x7fd5a4076d50 msgr2=0x7fd5a4077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.633+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/832538795 shutdown_connections 2024-03-18T20:44:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.633+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/832538795 wait complete. 2024-03-18T20:44:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.634+0000 7fd5a9e1f700 1 Processor -- start 2024-03-18T20:44:45.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.634+0000 7fd5a9e1f700 1 -- start start 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.634+0000 7fd5a9e1f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a40ffe70 0x7fd5a41018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.634+0000 7fd5a9e1f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5a4104f70 0x7fd5a4101e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.634+0000 7fd5a9e1f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5a4102360 0x7fd5a41a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.634+0000 7fd5a9e1f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a41a5970 con 0x7fd5a40ffe70 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.634+0000 7fd5a9e1f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a41a5ab0 con 0x7fd5a4102360 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.634+0000 7fd5a9e1f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a41a5c70 con 0x7fd5a4104f70 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a40ffe70 0x7fd5a41018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5a4102360 0x7fd5a41a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a40ffe70 0x7fd5a41018e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53540/0 (socket says 172.21.15.46:53540) 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a37fe700 1 -- 172.21.15.46:0/2510459396 learned_addr learned my addr 172.21.15.46:0/2510459396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a2ffd700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5a4104f70 0x7fd5a4101e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a37fe700 1 -- 172.21.15.46:0/2510459396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5a4104f70 msgr2=0x7fd5a4101e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5a4104f70 0x7fd5a4101e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a37fe700 1 -- 172.21.15.46:0/2510459396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5a4102360 msgr2=0x7fd5a41a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5a4102360 0x7fd5a41a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a37fe700 1 -- 172.21.15.46:0/2510459396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd594009c40 con 0x7fd5a40ffe70 2024-03-18T20:44:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a3fff700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5a4102360 0x7fd5a41a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a40ffe70 0x7fd5a41018e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c370 tx=0x7fd59400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a0ff9700 1 -- 172.21.15.46:0/2510459396 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594017070 con 0x7fd5a40ffe70 2024-03-18T20:44:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a0ff9700 1 -- 172.21.15.46:0/2510459396 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd59400f040 con 0x7fd5a40ffe70 2024-03-18T20:44:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/2510459396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c009520 con 0x7fd5a40ffe70 2024-03-18T20:44:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a0ff9700 1 -- 172.21.15.46:0/2510459396 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59400e050 con 0x7fd5a40ffe70 2024-03-18T20:44:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.635+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/2510459396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c0095f0 con 0x7fd5a40ffe70 2024-03-18T20:44:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.638+0000 7fd5a0ff9700 1 -- 172.21.15.46:0/2510459396 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd594003a00 con 0x7fd5a40ffe70 2024-03-18T20:44:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.639+0000 7fd5a0ff9700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd5900a1290 0x7fd5900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.639+0000 7fd5a2ffd700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd5900a1290 0x7fd5900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.639+0000 7fd5a0ff9700 1 -- 172.21.15.46:0/2510459396 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd5940cc670 con 0x7fd5a40ffe70 2024-03-18T20:44:45.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.639+0000 7fd5a2ffd700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd5900a1290 0x7fd5900a3750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4102fc0 tx=0x7fd58c005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5b5fb4b80 1 Processor -- start 2024-03-18T20:44:45.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5b5fb4b80 1 -- start start 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5b5fb4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 0x560dd92a6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5b5fb4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5b5fb4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92b2220 0x560dd92b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5b5fb4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd9200990 con 0x560dd92b2220 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5b5fb4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd91fe350 con 0x560dd92a6a60 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5b5fb4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd91f9c40 con 0x560dd92a7380 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92b2220 0x560dd92b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5a37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 0x560dd92a6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92b2220 0x560dd92b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53554/0 (socket says 172.21.15.46:53554) 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5a3fff700 1 -- 172.21.15.46:0/1102630851 learned_addr learned my addr 172.21.15.46:0/1102630851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5a2ffd700 1 --2- 172.21.15.46:0/1102630851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5a37fe700 1 -- 172.21.15.46:0/1102630851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 msgr2=0x560dd92b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/1102630851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5a37fe700 1 -- 172.21.15.46:0/1102630851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92b2220 msgr2=0x560dd92b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/1102630851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92b2220 0x560dd92b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.641+0000 7fd5a37fe700 1 -- 172.21.15.46:0/1102630851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dd907b3b0 con 0x560dd92a6a60 2024-03-18T20:44:45.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.642+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/1102630851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 0x560dd92a6e40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd5940c9da0 tx=0x7fd5940995b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af30eb742ca32c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.642+0000 7fd599ffb700 1 -- 172.21.15.46:0/1102630851 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940192c0 con 0x560dd92a6a60 2024-03-18T20:44:45.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.642+0000 7fd599ffb700 1 -- 172.21.15.46:0/1102630851 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd594099e60 con 0x560dd92a6a60 2024-03-18T20:44:45.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.642+0000 7fd599ffb700 1 -- 172.21.15.46:0/1102630851 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940b9230 con 0x560dd92a6a60 2024-03-18T20:44:45.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.643+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/1102630851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 msgr2=0x560dd92a6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.643+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/1102630851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 0x560dd92a6e40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd5940c9da0 tx=0x7fd5940995b0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.643+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/1102630851 shutdown_connections 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.643+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/1102630851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92b2220 0x560dd92b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.643+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/1102630851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 0x560dd92a6e40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.643+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/1102630851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.643+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/1102630851 >> 172.21.15.46:0/1102630851 conn(0x560dd921d220 msgr2=0x560dd92a3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.643+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/1102630851 shutdown_connections 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.643+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/1102630851 wait complete. 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.643+0000 7fd5b5fb4b80 1 -- start start 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5b5fb4b80 1 Processor -- start 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5b5fb4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 0x560dd92af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5b5fb4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5b5fb4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92b2220 0x560dd92afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5b5fb4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd9195d80 con 0x560dd92a6a60 2024-03-18T20:44:45.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5b5fb4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd9195a20 con 0x560dd92b2220 2024-03-18T20:44:45.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5b5fb4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd907d6a0 con 0x560dd92a7380 2024-03-18T20:44:45.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5b5fb4b80 10 client.0 _ll_register_callbacks cb 0x560dd90013a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:45.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 0x560dd92af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 0x560dd92af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53566/0 (socket says 172.21.15.46:53566) 2024-03-18T20:44:45.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5a37fe700 1 -- 172.21.15.46:0/4124946800 learned_addr learned my addr 172.21.15.46:0/4124946800 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:45.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5a3fff700 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92b2220 0x560dd92afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5a2ffd700 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5a37fe700 1 -- 172.21.15.46:0/4124946800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 msgr2=0x560dd92af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5a37fe700 1 -- 172.21.15.46:0/4124946800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92b2220 msgr2=0x560dd92afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92b2220 0x560dd92afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5a37fe700 1 -- 172.21.15.46:0/4124946800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c0095f0 con 0x560dd92a6a60 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.644+0000 7fd5a3fff700 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92b2220 0x560dd92afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 0x560dd92af150 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fd594000f80 tx=0x7fd5940b9340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41732726d87f284 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd582ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd582ffd700 1 -- 172.21.15.46:0/4124946800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940a1be0 con 0x560dd92a6a60 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd582ffd700 1 -- 172.21.15.46:0/4124946800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd594099760 con 0x560dd92a6a60 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/4124946800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560dd929aad0 con 0x560dd92a6a60 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd582ffd700 1 -- 172.21.15.46:0/4124946800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd59409d590 con 0x560dd92a6a60 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd5b5fb4b80 10 client.18549 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/4124946800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560dd9027b60 con 0x560dd92a6a60 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd582ffd700 1 -- 172.21.15.46:0/4124946800 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560dd929aad0 con 0x560dd92a6a60 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd582ffd700 1 -- 172.21.15.46:0/4124946800 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd594004bd0 con 0x560dd92a6a60 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd5b5fb4b80 10 client.18549 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd5b5fb4b80 10 client.18549 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:44:45.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd5b5fb4b80 20 client.18549 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:45.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560dd92c1ae0 0x560dd92c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd5b5fb4b80 4 client.18549 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:45.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.645+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/4124946800 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dd91f2700 con 0x560dd92c1ae0 2024-03-18T20:44:45.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.646+0000 7fd5a2ffd700 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560dd92c1ae0 0x560dd92c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.646+0000 7fd582ffd700 10 client.18549 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:44:45.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.646+0000 7fd5a2ffd700 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560dd92c1ae0 0x560dd92c3f00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e0570d61650d25d server_cookie=8504bea4e0ef9d09 in_seq=0 out_seq=0 2024-03-18T20:44:45.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd582ffd700 1 -- 172.21.15.46:0/4124946800 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560dd91f2700 con 0x560dd92c1ae0 2024-03-18T20:44:45.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd582ffd700 10 client.18549 handle_command_reply: tid=0 2024-03-18T20:44:45.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd5b5fb4b80 1 client.18549 shutdown 2024-03-18T20:44:45.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/4124946800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 msgr2=0x560dd92af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 0x560dd92af150 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fd594000f80 tx=0x7fd5940b9340 comp rx=0 tx=0).stop 2024-03-18T20:44:45.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/4124946800 shutdown_connections 2024-03-18T20:44:45.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560dd92c1ae0 0x560dd92c3f00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 0x560dd92af150 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92b2220 0x560dd92afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd582ffd700 0 client.18549 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:44:45.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/4124946800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/4124946800 >> 172.21.15.46:0/4124946800 conn(0x560dd921d220 msgr2=0x560dd92a2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/4124946800 shutdown_connections 2024-03-18T20:44:45.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/4124946800 wait complete. 2024-03-18T20:44:45.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.647+0000 7fd5b5fb4b80 20 client.18549 trim_cache size 0 max 16384 2024-03-18T20:44:45.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.662+0000 7fd5b5fb4b80 1 Processor -- start 2024-03-18T20:44:45.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.662+0000 7fd5b5fb4b80 1 -- start start 2024-03-18T20:44:45.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.662+0000 7fd5b5fb4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 0x560dd92bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.662+0000 7fd5b5fb4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.662+0000 7fd5b5fb4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92b2220 0x560dd92a1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.662+0000 7fd5b5fb4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd92a4440 con 0x560dd92a6a60 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.662+0000 7fd5b5fb4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd92b1bd0 con 0x560dd92b2220 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.662+0000 7fd5b5fb4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd92a7d30 con 0x560dd92a7380 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 0x560dd92bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92b2220 0x560dd92a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92b8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37236/0 (socket says 172.21.15.46:37236) 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a2ffd700 1 -- 172.21.15.46:0/3258358600 learned_addr learned my addr 172.21.15.46:0/3258358600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a3fff700 1 -- 172.21.15.46:0/3258358600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 msgr2=0x560dd92b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a3fff700 1 --2- 172.21.15.46:0/3258358600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92b8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a3fff700 1 -- 172.21.15.46:0/3258358600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 msgr2=0x560dd92bdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a3fff700 1 --2- 172.21.15.46:0/3258358600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 0x560dd92bdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a3fff700 1 -- 172.21.15.46:0/3258358600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dd91bfea0 con 0x560dd92b2220 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a2ffd700 1 --2- 172.21.15.46:0/3258358600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/3258358600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 0x560dd92bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd5a3fff700 1 --2- 172.21.15.46:0/3258358600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92b2220 0x560dd92a1750 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0043b0 tx=0x7fd59c011750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c34be959cc5d992f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd59a7fc700 1 -- 172.21.15.46:0/3258358600 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c01f070 con 0x560dd92b2220 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd59a7fc700 1 -- 172.21.15.46:0/3258358600 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd59c017040 con 0x560dd92b2220 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.663+0000 7fd59a7fc700 1 -- 172.21.15.46:0/3258358600 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c0056d0 con 0x560dd92b2220 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.664+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3258358600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92b2220 msgr2=0x560dd92a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.664+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/3258358600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92b2220 0x560dd92a1750 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0043b0 tx=0x7fd59c011750 comp rx=0 tx=0).stop 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.664+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3258358600 shutdown_connections 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.664+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/3258358600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a6a60 0x560dd92bdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.664+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/3258358600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92b2220 0x560dd92a1750 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.664+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/3258358600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92a7380 0x560dd92b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.664+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3258358600 >> 172.21.15.46:0/3258358600 conn(0x560dd921d220 msgr2=0x560dd92afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.664+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3258358600 shutdown_connections 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.664+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3258358600 wait complete. 2024-03-18T20:44:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5b5fb4b80 1 -- start start 2024-03-18T20:44:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5b5fb4b80 1 Processor -- start 2024-03-18T20:44:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5b5fb4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 0x560dd9309860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5b5fb4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a7380 0x560dd9309da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5b5fb4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92b2220 0x560dd930a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5b5fb4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd93148e0 con 0x560dd92a7380 2024-03-18T20:44:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5b5fb4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd9314a20 con 0x560dd92a6a60 2024-03-18T20:44:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5b5fb4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd9314b60 con 0x560dd92b2220 2024-03-18T20:44:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5b5fb4b80 10 client.0 _ll_register_callbacks cb 0x7fd59c009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5a2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a7380 0x560dd9309da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5a37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 0x560dd9309860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5a37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 0x560dd9309860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49696/0 (socket says 172.21.15.46:49696) 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5a37fe700 1 -- 172.21.15.46:0/3629832000 learned_addr learned my addr 172.21.15.46:0/3629832000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.665+0000 7fd5a3fff700 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92b2220 0x560dd930a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd5a37fe700 1 -- 172.21.15.46:0/3629832000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92b2220 msgr2=0x560dd930a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92b2220 0x560dd930a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd5a37fe700 1 -- 172.21.15.46:0/3629832000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a7380 msgr2=0x560dd9309da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a7380 0x560dd9309da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd5a37fe700 1 -- 172.21.15.46:0/3629832000 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560dd9027b60 con 0x560dd92a6a60 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd5a2ffd700 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a7380 0x560dd9309da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd5a37fe700 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 0x560dd9309860 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd594097ab0 tx=0x7fd594009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8e69d0375053a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd582ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd582ffd700 1 -- 172.21.15.46:0/3629832000 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940a1be0 con 0x560dd92a6a60 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd582ffd700 1 -- 172.21.15.46:0/3629832000 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd594099760 con 0x560dd92a6a60 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3629832000 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd594099e60 con 0x560dd92a6a60 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd582ffd700 1 -- 172.21.15.46:0/3629832000 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd594099530 con 0x560dd92a6a60 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd5b5fb4b80 10 client.28202 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3629832000 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7fd58c02c040 con 0x560dd92a6a60 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.666+0000 7fd582ffd700 1 -- 172.21.15.46:0/3629832000 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd594099e60 con 0x560dd92a6a60 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd582ffd700 1 -- 172.21.15.46:0/3629832000 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd594095550 con 0x560dd92a6a60 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd5b5fb4b80 10 client.28202 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd5b5fb4b80 10 client.28202 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd5b5fb4b80 20 client.28202 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560dd92c1ae0 0x560dd9317e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd5b5fb4b80 4 client.28202 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3629832000 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560dd9048cf0 con 0x560dd92c1ae0 2024-03-18T20:44:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd5a2ffd700 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560dd92c1ae0 0x560dd9317e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd582ffd700 10 client.28202 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:44:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd5a2ffd700 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560dd92c1ae0 0x560dd9317e60 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=807c1863a45c4699 server_cookie=95b91919ff5bf4a3 in_seq=0 out_seq=0 2024-03-18T20:44:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd582ffd700 1 -- 172.21.15.46:0/3629832000 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560dd9048cf0 con 0x560dd92c1ae0 2024-03-18T20:44:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd582ffd700 10 client.28202 handle_command_reply: tid=0 2024-03-18T20:44:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.667+0000 7fd5b5fb4b80 1 client.28202 shutdown 2024-03-18T20:44:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3629832000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 msgr2=0x560dd9309860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 0x560dd9309860 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd594097ab0 tx=0x7fd594009190 comp rx=0 tx=0).stop 2024-03-18T20:44:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3629832000 shutdown_connections 2024-03-18T20:44:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560dd92c1ae0 0x560dd9317e60 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560dd92b2220 0x560dd930a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560dd92a6a60 0x560dd9309860 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5b5fb4b80 1 --2- 172.21.15.46:0/3629832000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560dd92a7380 0x560dd9309da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd582ffd700 0 client.28202 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:44:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3629832000 >> 172.21.15.46:0/3629832000 conn(0x560dd921d220 msgr2=0x560dd92aa420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3629832000 shutdown_connections 2024-03-18T20:44:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5b5fb4b80 1 -- 172.21.15.46:0/3629832000 wait complete. 2024-03-18T20:44:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5b5fb4b80 20 client.28202 trim_cache size 0 max 16384 2024-03-18T20:44:45.679 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:44:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/2510459396 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd5900a1290 msgr2=0x7fd5900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5a9e1f700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd5900a1290 0x7fd5900a3750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4102fc0 tx=0x7fd58c005960 comp rx=0 tx=0).stop 2024-03-18T20:44:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/2510459396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a40ffe70 msgr2=0x7fd5a41018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.668+0000 7fd5a9e1f700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a40ffe70 0x7fd5a41018e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c370 tx=0x7fd59400c680 comp rx=0 tx=0).stop 2024-03-18T20:44:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.669+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/2510459396 shutdown_connections 2024-03-18T20:44:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.669+0000 7fd5a9e1f700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd5900a1290 0x7fd5900a3750 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.669+0000 7fd5a9e1f700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a40ffe70 0x7fd5a41018e0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.669+0000 7fd5a9e1f700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5a4102360 0x7fd5a41a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.669+0000 7fd5a9e1f700 1 --2- 172.21.15.46:0/2510459396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5a4104f70 0x7fd5a4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.669+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/2510459396 >> 172.21.15.46:0/2510459396 conn(0x7fd5a4076d50 msgr2=0x7fd5a40696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.669+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/2510459396 shutdown_connections 2024-03-18T20:44:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.669+0000 7fd5a9e1f700 1 -- 172.21.15.46:0/2510459396 wait complete. 2024-03-18T20:44:45.685 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:44:45.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:44:45.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 307661 3.57 MB/sec execute 340 sec latency 2787.997 ms 2024-03-18T20:44:45.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.834+0000 7f8582f03700 1 -- 172.21.15.46:0/4235199938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f857c105ce0 msgr2=0x7f857c1060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.834+0000 7f8582f03700 1 --2- 172.21.15.46:0/4235199938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f857c105ce0 0x7f857c1060c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8570009920 tx=0x7f8570009c30 comp rx=0 tx=0).stop 2024-03-18T20:44:45.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.834+0000 7f8582f03700 1 -- 172.21.15.46:0/4235199938 shutdown_connections 2024-03-18T20:44:45.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.834+0000 7f8582f03700 1 --2- 172.21.15.46:0/4235199938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857c0687f0 0x7f857c10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.834+0000 7f8582f03700 1 --2- 172.21.15.46:0/4235199938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f857c105ce0 0x7f857c1060c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.834+0000 7f8582f03700 1 --2- 172.21.15.46:0/4235199938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f857c067e30 0x7f857c0682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.834+0000 7f8582f03700 1 -- 172.21.15.46:0/4235199938 >> 172.21.15.46:0/4235199938 conn(0x7f857c076d40 msgr2=0x7f857c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.834+0000 7f8582f03700 1 -- 172.21.15.46:0/4235199938 shutdown_connections 2024-03-18T20:44:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.834+0000 7f8582f03700 1 -- 172.21.15.46:0/4235199938 wait complete. 2024-03-18T20:44:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.835+0000 7f8582f03700 1 Processor -- start 2024-03-18T20:44:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.835+0000 7f8582f03700 1 -- start start 2024-03-18T20:44:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.835+0000 7f8582f03700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f857c067e30 0x7f857c1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.835+0000 7f8582f03700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f857c0687f0 0x7f857c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.835+0000 7f8582f03700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857c105ce0 0x7f857c19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.835+0000 7f8582f03700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c19e400 con 0x7f857c105ce0 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.835+0000 7f8582f03700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c19e570 con 0x7f857c0687f0 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.835+0000 7f8582f03700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c19e730 con 0x7f857c067e30 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f8580c9f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f857c067e30 0x7f857c1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f857bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f857c0687f0 0x7f857c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f8580c9f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f857c067e30 0x7f857c1994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37260/0 (socket says 172.21.15.46:37260) 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f8580c9f700 1 -- 172.21.15.46:0/3105387592 learned_addr learned my addr 172.21.15.46:0/3105387592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f85814a0700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857c105ce0 0x7f857c19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f857bfff700 1 -- 172.21.15.46:0/3105387592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f857c067e30 msgr2=0x7f857c1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f857bfff700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f857c067e30 0x7f857c1994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f857bfff700 1 -- 172.21.15.46:0/3105387592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857c105ce0 msgr2=0x7f857c19ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f857bfff700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857c105ce0 0x7f857c19ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f857bfff700 1 -- 172.21.15.46:0/3105387592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8574009d00 con 0x7f857c0687f0 2024-03-18T20:44:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f8580c9f700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f857c067e30 0x7f857c1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f857bfff700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f857c0687f0 0x7f857c199a20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f856800e9f0 tx=0x7f856800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f8579ffb700 1 -- 172.21.15.46:0/3105387592 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856800c9c0 con 0x7f857c0687f0 2024-03-18T20:44:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f8579ffb700 1 -- 172.21.15.46:0/3105387592 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8568010430 con 0x7f857c0687f0 2024-03-18T20:44:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.836+0000 7f8582f03700 1 -- 172.21.15.46:0/3105387592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8570009520 con 0x7f857c0687f0 2024-03-18T20:44:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.837+0000 7f8579ffb700 1 -- 172.21.15.46:0/3105387592 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568018680 con 0x7f857c0687f0 2024-03-18T20:44:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.837+0000 7f8582f03700 1 -- 172.21.15.46:0/3105387592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857c19ec00 con 0x7f857c0687f0 2024-03-18T20:44:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.839+0000 7f8579ffb700 1 -- 172.21.15.46:0/3105387592 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8568010860 con 0x7f857c0687f0 2024-03-18T20:44:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.840+0000 7f8579ffb700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f856c0a1310 0x7f856c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.840+0000 7f8580c9f700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f856c0a1310 0x7f856c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.840+0000 7f8579ffb700 1 -- 172.21.15.46:0/3105387592 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8568014070 con 0x7f857c0687f0 2024-03-18T20:44:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.840+0000 7f8580c9f700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f856c0a1310 0x7f856c0a37d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8570009810 tx=0x7f8570019040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.841+0000 7f858f098b80 1 Processor -- start 2024-03-18T20:44:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.841+0000 7f858f098b80 1 -- start start 2024-03-18T20:44:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f858f098b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 0x55cef8e57e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f858f098b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e58380 0x55cef8e62ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f858f098b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e63220 0x55cef8e65610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f858f098b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8db1990 con 0x55cef8e58380 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f858f098b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8daf350 con 0x55cef8e63220 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f858f098b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8daac40 con 0x55cef8e57a60 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f857bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e58380 0x55cef8e62ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f8580c9f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 0x55cef8e57e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f857bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e58380 0x55cef8e62ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53620/0 (socket says 172.21.15.46:53620) 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f857bfff700 1 -- 172.21.15.46:0/679002712 learned_addr learned my addr 172.21.15.46:0/679002712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f8580c9f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 0x55cef8e57e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37276/0 (socket says 172.21.15.46:37276) 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f85814a0700 1 --2- 172.21.15.46:0/679002712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e63220 0x55cef8e65610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f857bfff700 1 -- 172.21.15.46:0/679002712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 msgr2=0x55cef8e57e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f857bfff700 1 --2- 172.21.15.46:0/679002712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 0x55cef8e57e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f857bfff700 1 -- 172.21.15.46:0/679002712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e63220 msgr2=0x55cef8e65610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f857bfff700 1 --2- 172.21.15.46:0/679002712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e63220 0x55cef8e65610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f857bfff700 1 -- 172.21.15.46:0/679002712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cef8c2c3b0 con 0x55cef8e58380 2024-03-18T20:44:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f857bfff700 1 --2- 172.21.15.46:0/679002712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e58380 0x55cef8e62ce0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f85680968c0 tx=0x7f85680998b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26680794d5f3d91e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.842+0000 7f8566ffd700 1 -- 172.21.15.46:0/679002712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856809dc80 con 0x55cef8e58380 2024-03-18T20:44:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.843+0000 7f8566ffd700 1 -- 172.21.15.46:0/679002712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85680a22d0 con 0x55cef8e58380 2024-03-18T20:44:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.843+0000 7f8566ffd700 1 -- 172.21.15.46:0/679002712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856809a310 con 0x55cef8e58380 2024-03-18T20:44:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.843+0000 7f858f098b80 1 -- 172.21.15.46:0/679002712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e58380 msgr2=0x55cef8e62ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.843+0000 7f858f098b80 1 --2- 172.21.15.46:0/679002712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e58380 0x55cef8e62ce0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f85680968c0 tx=0x7f85680998b0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.843+0000 7f858f098b80 1 -- 172.21.15.46:0/679002712 shutdown_connections 2024-03-18T20:44:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.843+0000 7f858f098b80 1 --2- 172.21.15.46:0/679002712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e63220 0x55cef8e65610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.843+0000 7f858f098b80 1 --2- 172.21.15.46:0/679002712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 0x55cef8e57e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.843+0000 7f858f098b80 1 --2- 172.21.15.46:0/679002712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e58380 0x55cef8e62ce0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.843+0000 7f858f098b80 1 -- 172.21.15.46:0/679002712 >> 172.21.15.46:0/679002712 conn(0x55cef8dce220 msgr2=0x55cef8e54020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.843+0000 7f858f098b80 1 -- 172.21.15.46:0/679002712 shutdown_connections 2024-03-18T20:44:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.843+0000 7f858f098b80 1 -- 172.21.15.46:0/679002712 wait complete. 2024-03-18T20:44:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.844+0000 7f858f098b80 1 -- start start 2024-03-18T20:44:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.844+0000 7f858f098b80 1 Processor -- start 2024-03-18T20:44:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.844+0000 7f858f098b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e57a60 0x55cef8e62c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.844+0000 7f858f098b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e58380 0x55cef8e5ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.844+0000 7f858f098b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e63220 0x55cef8e60700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.844+0000 7f858f098b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8d46d80 con 0x55cef8e57a60 2024-03-18T20:44:45.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.844+0000 7f858f098b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8d46a20 con 0x55cef8e58380 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.844+0000 7f858f098b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8c2e6a0 con 0x55cef8e63220 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.844+0000 7f858f098b80 10 client.0 _ll_register_callbacks cb 0x55cef8bb23a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.845+0000 7f8580c9f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e57a60 0x55cef8e62c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.845+0000 7f857bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e58380 0x55cef8e5ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.845+0000 7f8580c9f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e57a60 0x55cef8e62c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53624/0 (socket says 172.21.15.46:53624) 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.845+0000 7f8580c9f700 1 -- 172.21.15.46:0/2467871398 learned_addr learned my addr 172.21.15.46:0/2467871398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.845+0000 7f85814a0700 1 --2- 172.21.15.46:0/2467871398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e63220 0x55cef8e60700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.845+0000 7f857bfff700 1 -- 172.21.15.46:0/2467871398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e63220 msgr2=0x55cef8e60700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.845+0000 7f857bfff700 1 --2- 172.21.15.46:0/2467871398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e63220 0x55cef8e60700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.845+0000 7f857bfff700 1 -- 172.21.15.46:0/2467871398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e57a60 msgr2=0x55cef8e62c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.845+0000 7f857bfff700 1 --2- 172.21.15.46:0/2467871398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e57a60 0x55cef8e62c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.845+0000 7f857bfff700 1 -- 172.21.15.46:0/2467871398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857c19ec00 con 0x55cef8e58380 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.845+0000 7f857bfff700 1 --2- 172.21.15.46:0/2467871398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e58380 0x55cef8e5ffc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f85680968c0 tx=0x7f8568099fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7a50bf4ead68cf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.845+0000 7f855bfff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f855bfff700 1 -- 172.21.15.46:0/2467871398 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85680b6070 con 0x55cef8e58380 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f855bfff700 1 -- 172.21.15.46:0/2467871398 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f85680981f0 con 0x55cef8e58380 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f858f098b80 1 -- 172.21.15.46:0/2467871398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cef8e4bad0 con 0x55cef8e58380 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f855bfff700 1 -- 172.21.15.46:0/2467871398 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f856809dd00 con 0x55cef8e58380 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f858f098b80 10 client.28214 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f858f098b80 1 -- 172.21.15.46:0/2467871398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55cef8bd8b60 con 0x55cef8e58380 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f855bfff700 1 -- 172.21.15.46:0/2467871398 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8568080080 con 0x55cef8e58380 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f855bfff700 1 -- 172.21.15.46:0/2467871398 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f85680add20 con 0x55cef8e58380 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f858f098b80 10 client.28214 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f858f098b80 10 client.28214 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f858f098b80 20 client.28214 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f858f098b80 1 --2- 172.21.15.46:0/2467871398 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cef8e72ad0 0x55cef8e74ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f858f098b80 4 client.28214 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f858f098b80 1 -- 172.21.15.46:0/2467871398 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cef8da3700 con 0x55cef8e72ad0 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.846+0000 7f8580c9f700 1 --2- 172.21.15.46:0/2467871398 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cef8e72ad0 0x55cef8e74ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.847+0000 7f855bfff700 10 client.28214 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.847+0000 7f8580c9f700 1 --2- 172.21.15.46:0/2467871398 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cef8e72ad0 0x55cef8e74ef0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab8319ea2a2af8c2 server_cookie=9e092521dca78f7a in_seq=0 out_seq=0 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.847+0000 7f855bfff700 1 -- 172.21.15.46:0/2467871398 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cef8da3700 con 0x55cef8e72ad0 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.847+0000 7f855bfff700 10 client.28214 handle_command_reply: tid=0 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.847+0000 7f858f098b80 1 client.28214 shutdown 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.848+0000 7f858f098b80 1 -- 172.21.15.46:0/2467871398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e58380 msgr2=0x55cef8e5ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.848+0000 7f858f098b80 1 --2- 172.21.15.46:0/2467871398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e58380 0x55cef8e5ffc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f85680968c0 tx=0x7f8568099fd0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.848+0000 7f858f098b80 1 -- 172.21.15.46:0/2467871398 shutdown_connections 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.848+0000 7f858f098b80 1 --2- 172.21.15.46:0/2467871398 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cef8e72ad0 0x55cef8e74ef0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.848+0000 7f858f098b80 1 --2- 172.21.15.46:0/2467871398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e57a60 0x55cef8e62c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.848+0000 7f858f098b80 1 --2- 172.21.15.46:0/2467871398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e63220 0x55cef8e60700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.848+0000 7f858f098b80 1 --2- 172.21.15.46:0/2467871398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e58380 0x55cef8e5ffc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.848+0000 7f858f098b80 1 -- 172.21.15.46:0/2467871398 >> 172.21.15.46:0/2467871398 conn(0x55cef8dce220 msgr2=0x55cef8e53c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.848+0000 7f855bfff700 0 client.28214 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:44:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.848+0000 7f858f098b80 1 -- 172.21.15.46:0/2467871398 shutdown_connections 2024-03-18T20:44:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.848+0000 7f858f098b80 1 -- 172.21.15.46:0/2467871398 wait complete. 2024-03-18T20:44:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.848+0000 7f858f098b80 20 client.28214 trim_cache size 0 max 16384 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f858f098b80 1 Processor -- start 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f858f098b80 1 -- start start 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f858f098b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e57a60 0x55cef8e69d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f858f098b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e58380 0x55cef8e5f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f858f098b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e63220 0x55cef8e628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f858f098b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8e58d30 con 0x55cef8e58380 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f858f098b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8e59550 con 0x55cef8e57a60 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f858f098b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8e4df80 con 0x55cef8e63220 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f8580c9f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e57a60 0x55cef8e69d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f8580c9f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e57a60 0x55cef8e69d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49760/0 (socket says 172.21.15.46:49760) 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f8580c9f700 1 -- 172.21.15.46:0/20313911 learned_addr learned my addr 172.21.15.46:0/20313911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f857bfff700 1 --2- 172.21.15.46:0/20313911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e58380 0x55cef8e5f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f85814a0700 1 --2- 172.21.15.46:0/20313911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e63220 0x55cef8e628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f8580c9f700 1 -- 172.21.15.46:0/20313911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e63220 msgr2=0x55cef8e628b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f8580c9f700 1 --2- 172.21.15.46:0/20313911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e63220 0x55cef8e628b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f8580c9f700 1 -- 172.21.15.46:0/20313911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e58380 msgr2=0x55cef8e5f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.868+0000 7f8580c9f700 1 --2- 172.21.15.46:0/20313911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e58380 0x55cef8e5f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.869+0000 7f8580c9f700 1 -- 172.21.15.46:0/20313911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cef8d70ea0 con 0x55cef8e57a60 2024-03-18T20:44:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.869+0000 7f85814a0700 1 --2- 172.21.15.46:0/20313911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e63220 0x55cef8e628b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:44:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.869+0000 7f8580c9f700 1 --2- 172.21.15.46:0/20313911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e57a60 0x55cef8e69d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f857000b6c0 tx=0x7f857001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aba2cae36ae3cd82 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.869+0000 7f85677fe700 1 -- 172.21.15.46:0/20313911 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570036070 con 0x55cef8e57a60 2024-03-18T20:44:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.869+0000 7f85677fe700 1 -- 172.21.15.46:0/20313911 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f857001d040 con 0x55cef8e57a60 2024-03-18T20:44:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.869+0000 7f85677fe700 1 -- 172.21.15.46:0/20313911 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570005520 con 0x55cef8e57a60 2024-03-18T20:44:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.870+0000 7f858f098b80 1 -- 172.21.15.46:0/20313911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e57a60 msgr2=0x55cef8e69d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.870+0000 7f858f098b80 1 --2- 172.21.15.46:0/20313911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e57a60 0x55cef8e69d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f857000b6c0 tx=0x7f857001f000 comp rx=0 tx=0).stop 2024-03-18T20:44:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.870+0000 7f858f098b80 1 -- 172.21.15.46:0/20313911 shutdown_connections 2024-03-18T20:44:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.870+0000 7f858f098b80 1 --2- 172.21.15.46:0/20313911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e63220 0x55cef8e628b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.870+0000 7f858f098b80 1 --2- 172.21.15.46:0/20313911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e57a60 0x55cef8e69d90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.870+0000 7f858f098b80 1 --2- 172.21.15.46:0/20313911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e58380 0x55cef8e5f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.870+0000 7f858f098b80 1 -- 172.21.15.46:0/20313911 >> 172.21.15.46:0/20313911 conn(0x55cef8dce220 msgr2=0x55cef8e6e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.870+0000 7f858f098b80 1 -- 172.21.15.46:0/20313911 shutdown_connections 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.870+0000 7f858f098b80 1 -- 172.21.15.46:0/20313911 wait complete. 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.870+0000 7f858f098b80 1 -- start start 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.870+0000 7f858f098b80 1 Processor -- start 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f858f098b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 0x55cef8e677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f858f098b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e58380 0x55cef8e67cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f858f098b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e63220 0x55cef8ec58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f858f098b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8e685d0 con 0x55cef8e63220 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f858f098b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8ec5e00 con 0x55cef8e58380 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f858f098b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cef8ec5f40 con 0x55cef8e57a60 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f858f098b80 10 client.0 _ll_register_callbacks cb 0x7f8574009050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f8580c9f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 0x55cef8e677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f8580c9f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 0x55cef8e677b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37290/0 (socket says 172.21.15.46:37290) 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f8580c9f700 1 -- 172.21.15.46:0/3624354803 learned_addr learned my addr 172.21.15.46:0/3624354803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f857bfff700 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e58380 0x55cef8e67cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f85814a0700 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e63220 0x55cef8ec58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f8580c9f700 1 -- 172.21.15.46:0/3624354803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e58380 msgr2=0x55cef8e67cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f8580c9f700 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e58380 0x55cef8e67cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f8580c9f700 1 -- 172.21.15.46:0/3624354803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e63220 msgr2=0x55cef8ec58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f8580c9f700 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e63220 0x55cef8ec58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.871+0000 7f8580c9f700 1 -- 172.21.15.46:0/3624354803 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857c10e550 con 0x55cef8e57a60 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f8580c9f700 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 0x55cef8e677b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f85700050e0 tx=0x7f8570005260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f417428c75d5309 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f855bfff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f855bfff700 1 -- 172.21.15.46:0/3624354803 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570036040 con 0x55cef8e57a60 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f855bfff700 1 -- 172.21.15.46:0/3624354803 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f85700055a0 con 0x55cef8e57a60 2024-03-18T20:44:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f858f098b80 1 -- 172.21.15.46:0/3624354803 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85680a22d0 con 0x55cef8e57a60 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f855bfff700 1 -- 172.21.15.46:0/3624354803 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8570003680 con 0x55cef8e57a60 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f858f098b80 10 client.28231 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f858f098b80 1 -- 172.21.15.46:0/3624354803 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55cef8c30cd0 con 0x55cef8e57a60 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f855bfff700 1 -- 172.21.15.46:0/3624354803 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85680a22d0 con 0x55cef8e57a60 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f855bfff700 1 -- 172.21.15.46:0/3624354803 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f857002e070 con 0x55cef8e57a60 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f858f098b80 10 client.28231 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f858f098b80 10 client.28231 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f858f098b80 20 client.28231 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f858f098b80 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cef8ec7120 0x55cef8ec9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f858f098b80 4 client.28231 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.872+0000 7f858f098b80 1 -- 172.21.15.46:0/3624354803 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cef8bf9cf0 con 0x55cef8ec7120 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f857bfff700 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cef8ec7120 0x55cef8ec9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f855bfff700 10 client.28231 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f857bfff700 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cef8ec7120 0x55cef8ec9510 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e2a13e6b05908688 server_cookie=6fedd11e8cc2c95f in_seq=0 out_seq=0 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f855bfff700 1 -- 172.21.15.46:0/3624354803 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cef8bf9cf0 con 0x55cef8ec7120 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f855bfff700 10 client.28231 handle_command_reply: tid=0 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f858f098b80 1 client.28231 shutdown 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f858f098b80 1 -- 172.21.15.46:0/3624354803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 msgr2=0x55cef8e677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f858f098b80 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 0x55cef8e677b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f85700050e0 tx=0x7f8570005260 comp rx=0 tx=0).stop 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f858f098b80 1 -- 172.21.15.46:0/3624354803 shutdown_connections 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f858f098b80 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cef8ec7120 0x55cef8ec9510 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f858f098b80 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cef8e63220 0x55cef8ec58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f858f098b80 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cef8e57a60 0x55cef8e677b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f855bfff700 0 client.28231 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f858f098b80 1 --2- 172.21.15.46:0/3624354803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cef8e58380 0x55cef8e67cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.873+0000 7f858f098b80 1 -- 172.21.15.46:0/3624354803 >> 172.21.15.46:0/3624354803 conn(0x55cef8dce220 msgr2=0x55cef8e5b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.874+0000 7f858f098b80 1 -- 172.21.15.46:0/3624354803 shutdown_connections 2024-03-18T20:44:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.874+0000 7f858f098b80 1 -- 172.21.15.46:0/3624354803 wait complete. 2024-03-18T20:44:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.874+0000 7f858f098b80 20 client.28231 trim_cache size 0 max 16384 2024-03-18T20:44:45.884 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:44:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.874+0000 7f8582f03700 1 -- 172.21.15.46:0/3105387592 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f856c0a1310 msgr2=0x7f856c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.874+0000 7f8582f03700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f856c0a1310 0x7f856c0a37d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8570009810 tx=0x7f8570019040 comp rx=0 tx=0).stop 2024-03-18T20:44:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.874+0000 7f8582f03700 1 -- 172.21.15.46:0/3105387592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f857c0687f0 msgr2=0x7f857c199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.874+0000 7f8582f03700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f857c0687f0 0x7f857c199a20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f856800e9f0 tx=0x7f856800ed00 comp rx=0 tx=0).stop 2024-03-18T20:44:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.874+0000 7f8582f03700 1 -- 172.21.15.46:0/3105387592 shutdown_connections 2024-03-18T20:44:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.874+0000 7f8582f03700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f856c0a1310 0x7f856c0a37d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.874+0000 7f8582f03700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857c105ce0 0x7f857c19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.874+0000 7f8582f03700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f857c067e30 0x7f857c1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.874+0000 7f8582f03700 1 --2- 172.21.15.46:0/3105387592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f857c0687f0 0x7f857c199a20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.875+0000 7f8582f03700 1 -- 172.21.15.46:0/3105387592 >> 172.21.15.46:0/3105387592 conn(0x7f857c076d40 msgr2=0x7f857c10c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:45.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.875+0000 7f8582f03700 1 -- 172.21.15.46:0/3105387592 shutdown_connections 2024-03-18T20:44:45.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:45.875+0000 7f8582f03700 1 -- 172.21.15.46:0/3105387592 wait complete. 2024-03-18T20:44:45.892 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:44:45.892 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:44:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.104+0000 7f0131622700 1 -- 172.21.15.46:0/1937609779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012c104f70 msgr2=0x7f012c10caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.104+0000 7f0131622700 1 --2- 172.21.15.46:0/1937609779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012c104f70 0x7f012c10caf0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f012400b0a0 tx=0x7f012400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.104+0000 7f0131622700 1 -- 172.21.15.46:0/1937609779 shutdown_connections 2024-03-18T20:44:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.104+0000 7f0131622700 1 --2- 172.21.15.46:0/1937609779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012c104f70 0x7f012c10caf0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.104+0000 7f0131622700 1 --2- 172.21.15.46:0/1937609779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c0ffe70 0x7f012c100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.104+0000 7f0131622700 1 --2- 172.21.15.46:0/1937609779 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f012c100820 0x7f012c104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.104+0000 7f0131622700 1 -- 172.21.15.46:0/1937609779 >> 172.21.15.46:0/1937609779 conn(0x7f012c076d50 msgr2=0x7f012c077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.104+0000 7f0131622700 1 -- 172.21.15.46:0/1937609779 shutdown_connections 2024-03-18T20:44:46.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.104+0000 7f0131622700 1 -- 172.21.15.46:0/1937609779 wait complete. 2024-03-18T20:44:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.105+0000 7f0131622700 1 Processor -- start 2024-03-18T20:44:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.105+0000 7f0131622700 1 -- start start 2024-03-18T20:44:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.105+0000 7f0131622700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c0ffe70 0x7f012c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.105+0000 7f0131622700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012c104f70 0x7f012c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.105+0000 7f0131622700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f012c198bb0 0x7f012c1a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.105+0000 7f0131622700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c1a3890 con 0x7f012c104f70 2024-03-18T20:44:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.105+0000 7f0131622700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c1a3a00 con 0x7f012c0ffe70 2024-03-18T20:44:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.105+0000 7f0131622700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c1a3bc0 con 0x7f012c198bb0 2024-03-18T20:44:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.105+0000 7f012affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c0ffe70 0x7f012c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.105+0000 7f012affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c0ffe70 0x7f012c198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49780/0 (socket says 172.21.15.46:49780) 2024-03-18T20:44:46.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.105+0000 7f012affd700 1 -- 172.21.15.46:0/2228838034 learned_addr learned my addr 172.21.15.46:0/2228838034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.105+0000 7f012b7fe700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f012c198bb0 0x7f012c1a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f0123fff700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012c104f70 0x7f012c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f012affd700 1 -- 172.21.15.46:0/2228838034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f012c198bb0 msgr2=0x7f012c1a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f012affd700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f012c198bb0 0x7f012c1a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f012affd700 1 -- 172.21.15.46:0/2228838034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012c104f70 msgr2=0x7f012c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f012affd700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012c104f70 0x7f012c198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f012affd700 1 -- 172.21.15.46:0/2228838034 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0124009d00 con 0x7f012c0ffe70 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f012b7fe700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f012c198bb0 0x7f012c1a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f012affd700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c0ffe70 0x7f012c198130 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f011c00e9a0 tx=0x7f011c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f0128ff9700 1 -- 172.21.15.46:0/2228838034 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c009790 con 0x7f012c0ffe70 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f0131622700 1 -- 172.21.15.46:0/2228838034 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012c1a3ea0 con 0x7f012c0ffe70 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f0128ff9700 1 -- 172.21.15.46:0/2228838034 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f011c010430 con 0x7f012c0ffe70 2024-03-18T20:44:46.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f0128ff9700 1 -- 172.21.15.46:0/2228838034 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c018680 con 0x7f012c0ffe70 2024-03-18T20:44:46.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.106+0000 7f0131622700 1 -- 172.21.15.46:0/2228838034 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012c1a4320 con 0x7f012c0ffe70 2024-03-18T20:44:46.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.109+0000 7f0128ff9700 1 -- 172.21.15.46:0/2228838034 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f011c0105a0 con 0x7f012c0ffe70 2024-03-18T20:44:46.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.110+0000 7f0128ff9700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01180a1290 0x7f01180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.110+0000 7f0123fff700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01180a1290 0x7f01180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.110+0000 7f013d7b7b80 1 Processor -- start 2024-03-18T20:44:46.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.110+0000 7f0128ff9700 1 -- 172.21.15.46:0/2228838034 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f011c014070 con 0x7f012c0ffe70 2024-03-18T20:44:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.110+0000 7f013d7b7b80 1 -- start start 2024-03-18T20:44:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f0123fff700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01180a1290 0x7f01180a3750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f012c069a60 tx=0x7f0114009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f013d7b7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79769a60 0x559c79769e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f013d7b7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c7976a380 0x559c79774ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f013d7b7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79775220 0x559c79777610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f013d7b7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c796c3990 con 0x559c7976a380 2024-03-18T20:44:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f013d7b7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c796c1350 con 0x559c79769a60 2024-03-18T20:44:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f013d7b7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c796bcc40 con 0x559c79775220 2024-03-18T20:44:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f0123fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c7976a380 0x559c79774ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f012affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79769a60 0x559c79769e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f012affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79769a60 0x559c79769e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49782/0 (socket says 172.21.15.46:49782) 2024-03-18T20:44:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f0123fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c7976a380 0x559c79774ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53680/0 (socket says 172.21.15.46:53680) 2024-03-18T20:44:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f012affd700 1 -- 172.21.15.46:0/1468242276 learned_addr learned my addr 172.21.15.46:0/1468242276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f012b7fe700 1 --2- 172.21.15.46:0/1468242276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79775220 0x559c79777610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f012affd700 1 -- 172.21.15.46:0/1468242276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79775220 msgr2=0x559c79777610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f012affd700 1 --2- 172.21.15.46:0/1468242276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79775220 0x559c79777610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f012affd700 1 -- 172.21.15.46:0/1468242276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c7976a380 msgr2=0x559c79774ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f012affd700 1 --2- 172.21.15.46:0/1468242276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c7976a380 0x559c79774ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.111+0000 7f012affd700 1 -- 172.21.15.46:0/1468242276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c7953e3b0 con 0x559c79769a60 2024-03-18T20:44:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.112+0000 7f012b7fe700 1 --2- 172.21.15.46:0/1468242276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79775220 0x559c79777610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:44:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.112+0000 7f012affd700 1 --2- 172.21.15.46:0/1468242276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79769a60 0x559c79769e40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f011c0a1f00 tx=0x7f011c09a6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abacf0bb1f300457 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.112+0000 7f01217fa700 1 -- 172.21.15.46:0/1468242276 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c0a2430 con 0x559c79769a60 2024-03-18T20:44:46.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.112+0000 7f01217fa700 1 -- 172.21.15.46:0/1468242276 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f011c010880 con 0x559c79769a60 2024-03-18T20:44:46.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.112+0000 7f01217fa700 1 -- 172.21.15.46:0/1468242276 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c099710 con 0x559c79769a60 2024-03-18T20:44:46.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.113+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1468242276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79769a60 msgr2=0x559c79769e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.113+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/1468242276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79769a60 0x559c79769e40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f011c0a1f00 tx=0x7f011c09a6b0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.113+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1468242276 shutdown_connections 2024-03-18T20:44:46.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.113+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/1468242276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79775220 0x559c79777610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.113+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/1468242276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79769a60 0x559c79769e40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.113+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/1468242276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c7976a380 0x559c79774ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.113+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1468242276 >> 172.21.15.46:0/1468242276 conn(0x559c796e0220 msgr2=0x559c79766020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.113+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1468242276 shutdown_connections 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.113+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1468242276 wait complete. 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.113+0000 7f013d7b7b80 1 -- start start 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.113+0000 7f013d7b7b80 1 Processor -- start 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f013d7b7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 0x559c79772100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f013d7b7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c7976a380 0x559c79772640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f013d7b7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 0x559c79772b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f013d7b7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79658d80 con 0x559c79769a60 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f013d7b7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79658a20 con 0x559c79775220 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f013d7b7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c795406a0 con 0x559c7976a380 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f013d7b7b80 10 client.0 _ll_register_callbacks cb 0x559c794c43a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f012affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 0x559c79772100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f012b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 0x559c79772b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f012b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 0x559c79772b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49784/0 (socket says 172.21.15.46:49784) 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f0123fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c7976a380 0x559c79772640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f012affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 0x559c79772100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53688/0 (socket says 172.21.15.46:53688) 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f012b7fe700 1 -- 172.21.15.46:0/3614792802 learned_addr learned my addr 172.21.15.46:0/3614792802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f012affd700 1 -- 172.21.15.46:0/3614792802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c7976a380 msgr2=0x559c79772640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f012affd700 1 --2- 172.21.15.46:0/3614792802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c7976a380 0x559c79772640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f012affd700 1 -- 172.21.15.46:0/3614792802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 msgr2=0x559c79772b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f012affd700 1 --2- 172.21.15.46:0/3614792802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 0x559c79772b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f012affd700 1 -- 172.21.15.46:0/3614792802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012c1a4320 con 0x559c79769a60 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f012b7fe700 1 --2- 172.21.15.46:0/3614792802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 0x559c79772b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f012affd700 1 --2- 172.21.15.46:0/3614792802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 0x559c79772100 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f011c0a1f00 tx=0x7f011c099750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9a289b53591574 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.114+0000 7f01127fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.115+0000 7f01127fc700 1 -- 172.21.15.46:0/3614792802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c0a2430 con 0x559c79769a60 2024-03-18T20:44:46.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.115+0000 7f01127fc700 1 -- 172.21.15.46:0/3614792802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f011c0a21b0 con 0x559c79769a60 2024-03-18T20:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.115+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3614792802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c7975dad0 con 0x559c79769a60 2024-03-18T20:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.115+0000 7f01127fc700 1 -- 172.21.15.46:0/3614792802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f011c094bc0 con 0x559c79769a60 2024-03-18T20:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.115+0000 7f013d7b7b80 10 client.18585 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.115+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3614792802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x559c794eab60 con 0x559c79769a60 2024-03-18T20:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.115+0000 7f01127fc700 1 -- 172.21.15.46:0/3614792802 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c7975dad0 con 0x559c79769a60 2024-03-18T20:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.116+0000 7f01127fc700 1 -- 172.21.15.46:0/3614792802 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f011c095da0 con 0x559c79769a60 2024-03-18T20:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.116+0000 7f013d7b7b80 10 client.18585 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.116+0000 7f013d7b7b80 10 client.18585 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.116+0000 7f013d7b7b80 20 client.18585 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.116+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/3614792802 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559c797849f0 0x559c79786e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.116+0000 7f013d7b7b80 4 client.18585 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.116+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3614792802 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c796b5700 con 0x559c797849f0 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.116+0000 7f0123fff700 1 --2- 172.21.15.46:0/3614792802 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559c797849f0 0x559c79786e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.116+0000 7f01127fc700 10 client.18585 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.116+0000 7f0123fff700 1 --2- 172.21.15.46:0/3614792802 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559c797849f0 0x559c79786e10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d01a55f3a19bd8d6 server_cookie=e60332db1ff68260 in_seq=0 out_seq=0 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f01127fc700 1 -- 172.21.15.46:0/3614792802 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559c796b5700 con 0x559c797849f0 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f01127fc700 10 client.18585 handle_command_reply: tid=0 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f013d7b7b80 1 client.18585 shutdown 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3614792802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 msgr2=0x559c79772100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/3614792802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 0x559c79772100 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f011c0a1f00 tx=0x7f011c099750 comp rx=0 tx=0).stop 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3614792802 shutdown_connections 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/3614792802 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559c797849f0 0x559c79786e10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/3614792802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 0x559c79772100 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f01127fc700 0 client.18585 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:44:46.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/3614792802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 0x559c79772b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/3614792802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c7976a380 0x559c79772640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3614792802 >> 172.21.15.46:0/3614792802 conn(0x559c796e0220 msgr2=0x559c79765c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.117+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3614792802 shutdown_connections 2024-03-18T20:44:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.118+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3614792802 wait complete. 2024-03-18T20:44:46.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.118+0000 7f013d7b7b80 20 client.18585 trim_cache size 0 max 16384 2024-03-18T20:44:46.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.133+0000 7f013d7b7b80 1 Processor -- start 2024-03-18T20:44:46.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.133+0000 7f013d7b7b80 1 -- start start 2024-03-18T20:44:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f013d7b7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 0x559c7976cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f013d7b7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c7976a380 0x559c797729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f013d7b7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 0x559c79765b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f013d7b7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c797625e0 con 0x559c79769a60 2024-03-18T20:44:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f013d7b7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7976b300 con 0x559c79775220 2024-03-18T20:44:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f013d7b7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7975d610 con 0x559c7976a380 2024-03-18T20:44:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f012affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 0x559c7976cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f012b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 0x559c79765b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f012affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 0x559c7976cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53694/0 (socket says 172.21.15.46:53694) 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f012affd700 1 -- 172.21.15.46:0/3867780539 learned_addr learned my addr 172.21.15.46:0/3867780539 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f0123fff700 1 --2- 172.21.15.46:0/3867780539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c7976a380 0x559c797729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f012affd700 1 -- 172.21.15.46:0/3867780539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c7976a380 msgr2=0x559c797729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f012affd700 1 --2- 172.21.15.46:0/3867780539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c7976a380 0x559c797729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f012affd700 1 -- 172.21.15.46:0/3867780539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 msgr2=0x559c79765b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f012affd700 1 --2- 172.21.15.46:0/3867780539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 0x559c79765b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f012affd700 1 -- 172.21.15.46:0/3867780539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c79682ea0 con 0x559c79769a60 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f012b7fe700 1 --2- 172.21.15.46:0/3867780539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 0x559c79765b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.134+0000 7f012affd700 1 --2- 172.21.15.46:0/3867780539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 0x559c7976cac0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f011c0964b0 tx=0x7f011c099eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d11ae93138c05e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.135+0000 7f0121ffb700 1 -- 172.21.15.46:0/3867780539 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c0a2430 con 0x559c79769a60 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.135+0000 7f0121ffb700 1 -- 172.21.15.46:0/3867780539 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f011c0962f0 con 0x559c79769a60 2024-03-18T20:44:46.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.135+0000 7f0121ffb700 1 -- 172.21.15.46:0/3867780539 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c094de0 con 0x559c79769a60 2024-03-18T20:44:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.136+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3867780539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 msgr2=0x559c7976cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.136+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/3867780539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 0x559c7976cac0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f011c0964b0 tx=0x7f011c099eb0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.136+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3867780539 shutdown_connections 2024-03-18T20:44:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.136+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/3867780539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79769a60 0x559c7976cac0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.136+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/3867780539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c79775220 0x559c79765b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.136+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/3867780539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c7976a380 0x559c797729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.136+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3867780539 >> 172.21.15.46:0/3867780539 conn(0x559c796e0220 msgr2=0x559c7976d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.136+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3867780539 shutdown_connections 2024-03-18T20:44:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.136+0000 7f013d7b7b80 1 -- 172.21.15.46:0/3867780539 wait complete. 2024-03-18T20:44:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.136+0000 7f013d7b7b80 1 -- start start 2024-03-18T20:44:46.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.136+0000 7f013d7b7b80 1 Processor -- start 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f013d7b7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79769a60 0x559c79779620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f013d7b7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c7976a380 0x559c79779b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f013d7b7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79775220 0x559c7977a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f013d7b7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c797d77e0 con 0x559c79775220 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f013d7b7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c797d7920 con 0x559c7976a380 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f013d7b7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c797d7ae0 con 0x559c79769a60 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f013d7b7b80 10 client.0 _ll_register_callbacks cb 0x559c794c43a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f012b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79775220 0x559c7977a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f0123fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c7976a380 0x559c79779b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f0123fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c7976a380 0x559c79779b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49800/0 (socket says 172.21.15.46:49800) 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f0123fff700 1 -- 172.21.15.46:0/1706650484 learned_addr learned my addr 172.21.15.46:0/1706650484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f012b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79775220 0x559c7977a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53712/0 (socket says 172.21.15.46:53712) 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f012affd700 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79769a60 0x559c79779620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f0123fff700 1 -- 172.21.15.46:0/1706650484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79769a60 msgr2=0x559c79779620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f0123fff700 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79769a60 0x559c79779620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f0123fff700 1 -- 172.21.15.46:0/1706650484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79775220 msgr2=0x559c7977a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f0123fff700 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79775220 0x559c7977a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f0123fff700 1 -- 172.21.15.46:0/1706650484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012c10d360 con 0x559c7976a380 2024-03-18T20:44:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f012affd700 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79769a60 0x559c79779620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f012b7fe700 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79775220 0x559c7977a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f0123fff700 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c7976a380 0x559c79779b60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0114005950 tx=0x7f01140047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6febb01ff543044c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.137+0000 7f01127fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.138+0000 7f01127fc700 1 -- 172.21.15.46:0/1706650484 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114024040 con 0x559c7976a380 2024-03-18T20:44:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.138+0000 7f01127fc700 1 -- 172.21.15.46:0/1706650484 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f0114004030 con 0x559c7976a380 2024-03-18T20:44:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.138+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1706650484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f011c010880 con 0x559c7976a380 2024-03-18T20:44:46.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.138+0000 7f01127fc700 1 -- 172.21.15.46:0/1706650484 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0114029580 con 0x559c7976a380 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.138+0000 7f013d7b7b80 10 client.28238 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.138+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1706650484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x559c794eab60 con 0x559c7976a380 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.138+0000 7f01127fc700 1 -- 172.21.15.46:0/1706650484 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f011c010880 con 0x559c7976a380 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f01127fc700 1 -- 172.21.15.46:0/1706650484 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f011401e420 con 0x559c7976a380 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f013d7b7b80 10 client.28238 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f013d7b7b80 10 client.28238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f013d7b7b80 20 client.28238 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559c797d9390 0x559c797db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f013d7b7b80 4 client.28238 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1706650484 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559c7950bcf0 con 0x559c797d9390 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f012affd700 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559c797d9390 0x559c797db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f012affd700 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559c797d9390 0x559c797db7b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0004d9c07d10b69 server_cookie=db2ceb9c5790a20c in_seq=0 out_seq=0 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f01127fc700 10 client.28238 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:44:46.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f01127fc700 1 -- 172.21.15.46:0/1706650484 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559c7950bcf0 con 0x559c797d9390 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f01127fc700 10 client.28238 handle_command_reply: tid=0 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.139+0000 7f013d7b7b80 1 client.28238 shutdown 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.140+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1706650484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c7976a380 msgr2=0x559c79779b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.140+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c7976a380 0x559c79779b60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0114005950 tx=0x7f01140047c0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.140+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1706650484 shutdown_connections 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.140+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559c797d9390 0x559c797db7b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.140+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c79775220 0x559c7977a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.140+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79769a60 0x559c79779620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.140+0000 7f013d7b7b80 1 --2- 172.21.15.46:0/1706650484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c7976a380 0x559c79779b60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.140+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1706650484 >> 172.21.15.46:0/1706650484 conn(0x559c796e0220 msgr2=0x559c79772660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.140+0000 7f01127fc700 0 client.28238 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.140+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1706650484 shutdown_connections 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.140+0000 7f013d7b7b80 1 -- 172.21.15.46:0/1706650484 wait complete. 2024-03-18T20:44:46.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.140+0000 7f013d7b7b80 20 client.28238 trim_cache size 0 max 16384 2024-03-18T20:44:46.149 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:44:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.141+0000 7f0131622700 1 -- 172.21.15.46:0/2228838034 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01180a1290 msgr2=0x7f01180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.141+0000 7f0131622700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01180a1290 0x7f01180a3750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f012c069a60 tx=0x7f0114009260 comp rx=0 tx=0).stop 2024-03-18T20:44:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.141+0000 7f0131622700 1 -- 172.21.15.46:0/2228838034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c0ffe70 msgr2=0x7f012c198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.141+0000 7f0131622700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c0ffe70 0x7f012c198130 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f011c00e9a0 tx=0x7f011c00ecb0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.141+0000 7f0131622700 1 -- 172.21.15.46:0/2228838034 shutdown_connections 2024-03-18T20:44:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.141+0000 7f0131622700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f01180a1290 0x7f01180a3750 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.141+0000 7f0131622700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f012c198bb0 0x7f012c1a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.141+0000 7f0131622700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f012c0ffe70 0x7f012c198130 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.141+0000 7f0131622700 1 --2- 172.21.15.46:0/2228838034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012c104f70 0x7f012c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.141+0000 7f0131622700 1 -- 172.21.15.46:0/2228838034 >> 172.21.15.46:0/2228838034 conn(0x7f012c076d50 msgr2=0x7f012c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.141+0000 7f0131622700 1 -- 172.21.15.46:0/2228838034 shutdown_connections 2024-03-18T20:44:46.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.141+0000 7f0131622700 1 -- 172.21.15.46:0/2228838034 wait complete. 2024-03-18T20:44:46.157 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:45 smithi046 ceph-mon[43495]: pgmap v267: 97 pgs: 97 active+clean; 119 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-03-18T20:44:46.157 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:45 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/4032400397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:44:46.158 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:44:46.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:44:46.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:45 smithi073 ceph-mon[47384]: pgmap v267: 97 pgs: 97 active+clean; 119 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-03-18T20:44:46.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:45 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/4032400397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:44:46.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:45 smithi155 ceph-mon[48344]: pgmap v267: 97 pgs: 97 active+clean; 119 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-03-18T20:44:46.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:45 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/4032400397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:44:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.308+0000 7f1d15bf5700 1 -- 172.21.15.46:0/1171150091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d10104f60 msgr2=0x7f1d1010cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.308+0000 7f1d15bf5700 1 --2- 172.21.15.46:0/1171150091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d10104f60 0x7f1d1010cae0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800b0a0 tx=0x7f1d0800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.308+0000 7f1d15bf5700 1 -- 172.21.15.46:0/1171150091 shutdown_connections 2024-03-18T20:44:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.308+0000 7f1d15bf5700 1 --2- 172.21.15.46:0/1171150091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d10104f60 0x7f1d1010cae0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.308+0000 7f1d15bf5700 1 --2- 172.21.15.46:0/1171150091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d100ffe50 0x7f1d10100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.308+0000 7f1d15bf5700 1 --2- 172.21.15.46:0/1171150091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d10100800 0x7f1d10104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.308+0000 7f1d15bf5700 1 -- 172.21.15.46:0/1171150091 >> 172.21.15.46:0/1171150091 conn(0x7f1d10076d40 msgr2=0x7f1d10077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.308+0000 7f1d15bf5700 1 -- 172.21.15.46:0/1171150091 shutdown_connections 2024-03-18T20:44:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.308+0000 7f1d15bf5700 1 -- 172.21.15.46:0/1171150091 wait complete. 2024-03-18T20:44:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.309+0000 7f1d15bf5700 1 Processor -- start 2024-03-18T20:44:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.309+0000 7f1d15bf5700 1 -- start start 2024-03-18T20:44:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d15bf5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d100ffe50 0x7f1d10199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d15bf5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d10104f60 0x7f1d10199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d15bf5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d1019dda0 0x7f1d1019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d15bf5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1019e820 con 0x7f1d100ffe50 2024-03-18T20:44:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d15bf5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1019e990 con 0x7f1d1019dda0 2024-03-18T20:44:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d15bf5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1019eb50 con 0x7f1d10104f60 2024-03-18T20:44:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d100ffe50 0x7f1d10199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d0effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d10104f60 0x7f1d10199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d100ffe50 0x7f1d10199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53732/0 (socket says 172.21.15.46:53732) 2024-03-18T20:44:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/103829955 learned_addr learned my addr 172.21.15.46:0/103829955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d0ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d1019dda0 0x7f1d1019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/103829955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d10104f60 msgr2=0x7f1d10199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d0f7fe700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d10104f60 0x7f1d10199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/103829955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d1019dda0 msgr2=0x7f1d1019e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d0f7fe700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d1019dda0 0x7f1d1019e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/103829955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d08009d00 con 0x7f1d100ffe50 2024-03-18T20:44:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d0ffff700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d1019dda0 0x7f1d1019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.310+0000 7f1d0f7fe700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d100ffe50 0x7f1d10199510 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000e9a0 tx=0x7f1d0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.311+0000 7f1d0cff9700 1 -- 172.21.15.46:0/103829955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00009790 con 0x7f1d100ffe50 2024-03-18T20:44:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.311+0000 7f1d15bf5700 1 -- 172.21.15.46:0/103829955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d101a60b0 con 0x7f1d100ffe50 2024-03-18T20:44:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.311+0000 7f1d0cff9700 1 -- 172.21.15.46:0/103829955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d00010430 con 0x7f1d100ffe50 2024-03-18T20:44:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.311+0000 7f1d0cff9700 1 -- 172.21.15.46:0/103829955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00018680 con 0x7f1d100ffe50 2024-03-18T20:44:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.311+0000 7f1d15bf5700 1 -- 172.21.15.46:0/103829955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d101a65e0 con 0x7f1d100ffe50 2024-03-18T20:44:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.313+0000 7f1d0cff9700 1 -- 172.21.15.46:0/103829955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1d000187e0 con 0x7f1d100ffe50 2024-03-18T20:44:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.314+0000 7f1d0cff9700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1cf80a1290 0x7f1cf80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.314+0000 7f1d0effd700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1cf80a1290 0x7f1cf80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.314+0000 7f1d0cff9700 1 -- 172.21.15.46:0/103829955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f1d00014070 con 0x7f1d100ffe50 2024-03-18T20:44:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.314+0000 7f1d21d8ab80 1 Processor -- start 2024-03-18T20:44:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.314+0000 7f1d0effd700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1cf80a1290 0x7f1cf80a3750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d1019aa10 tx=0x7f1d04009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.314+0000 7f1d21d8ab80 1 -- start start 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d21d8ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422abe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d21d8ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d21d8ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422b9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d21d8ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a42205990 con 0x561a422ac380 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d21d8ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a42203350 con 0x561a422b7220 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d21d8ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a421fec40 con 0x561a422aba60 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d0effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d0effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422b6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53738/0 (socket says 172.21.15.46:53738) 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d0effd700 1 -- 172.21.15.46:0/2153259205 learned_addr learned my addr 172.21.15.46:0/2153259205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d0ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d0f7fe700 1 --2- 172.21.15.46:0/2153259205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d0effd700 1 -- 172.21.15.46:0/2153259205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 msgr2=0x561a422abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d0effd700 1 --2- 172.21.15.46:0/2153259205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d0effd700 1 -- 172.21.15.46:0/2153259205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 msgr2=0x561a422b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d0effd700 1 --2- 172.21.15.46:0/2153259205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422b9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d0effd700 1 -- 172.21.15.46:0/2153259205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a420803b0 con 0x561a422ac380 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d0ffff700 1 --2- 172.21.15.46:0/2153259205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.315+0000 7f1d0effd700 1 --2- 172.21.15.46:0/2153259205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422b6ce0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1d04011d10 tx=0x7f1d04014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b26ad97a42fb0a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.316+0000 7f1cf5ffb700 1 -- 172.21.15.46:0/2153259205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04011d40 con 0x561a422ac380 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.316+0000 7f1cf5ffb700 1 -- 172.21.15.46:0/2153259205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d04004020 con 0x561a422ac380 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.316+0000 7f1cf5ffb700 1 -- 172.21.15.46:0/2153259205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04003c70 con 0x561a422ac380 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.316+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/2153259205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 msgr2=0x561a422b6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.316+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/2153259205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422b6ce0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1d04011d10 tx=0x7f1d04014000 comp rx=0 tx=0).stop 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.316+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/2153259205 shutdown_connections 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.316+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/2153259205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.316+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/2153259205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.316+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/2153259205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422b6ce0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.316+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/2153259205 >> 172.21.15.46:0/2153259205 conn(0x561a42222220 msgr2=0x561a422a8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.316+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/2153259205 shutdown_connections 2024-03-18T20:44:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.316+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/2153259205 wait complete. 2024-03-18T20:44:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.317+0000 7f1d21d8ab80 1 -- start start 2024-03-18T20:44:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.317+0000 7f1d21d8ab80 1 Processor -- start 2024-03-18T20:44:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.317+0000 7f1d21d8ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.317+0000 7f1d21d8ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.317+0000 7f1d21d8ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d21d8ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a4219ad80 con 0x561a422ac380 2024-03-18T20:44:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d21d8ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a4219aa20 con 0x561a422b7220 2024-03-18T20:44:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d21d8ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a420826a0 con 0x561a422aba60 2024-03-18T20:44:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d21d8ab80 10 client.0 _ll_register_callbacks cb 0x561a420063a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d0effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d0effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53750/0 (socket says 172.21.15.46:53750) 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d0effd700 1 -- 172.21.15.46:0/471875483 learned_addr learned my addr 172.21.15.46:0/471875483 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d0ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d0effd700 1 -- 172.21.15.46:0/471875483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 msgr2=0x561a422b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d0effd700 1 --2- 172.21.15.46:0/471875483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d0effd700 1 -- 172.21.15.46:0/471875483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 msgr2=0x561a422b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d0effd700 1 --2- 172.21.15.46:0/471875483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d0effd700 1 -- 172.21.15.46:0/471875483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1010d350 con 0x561a422ac380 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d0effd700 1 --2- 172.21.15.46:0/471875483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422b4040 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f1d040037f0 tx=0x7f1d0400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263e152ed2c99577 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1ceeffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1ceeffd700 1 -- 172.21.15.46:0/471875483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04003850 con 0x561a422ac380 2024-03-18T20:44:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/471875483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a4229fad0 con 0x561a422ac380 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.318+0000 7f1ceeffd700 1 -- 172.21.15.46:0/471875483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f1d040039b0 con 0x561a422ac380 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.319+0000 7f1ceeffd700 1 -- 172.21.15.46:0/471875483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d04003b70 con 0x561a422ac380 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.319+0000 7f1d21d8ab80 10 client.18603 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.319+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/471875483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f1d04017070 con 0x561a422ac380 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.319+0000 7f1ceeffd700 1 -- 172.21.15.46:0/471875483 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d0403c020 con 0x561a422ac380 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.319+0000 7f1ceeffd700 1 -- 172.21.15.46:0/471875483 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1d04024030 con 0x561a422ac380 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.319+0000 7f1d21d8ab80 10 client.18603 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.319+0000 7f1d21d8ab80 10 client.18603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.319+0000 7f1d21d8ab80 20 client.18603 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.319+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/471875483 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561a422c6d10 0x561a422c9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.319+0000 7f1d21d8ab80 4 client.18603 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.319+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/471875483 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a421f7700 con 0x561a422c6d10 2024-03-18T20:44:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.319+0000 7f1d0f7fe700 1 --2- 172.21.15.46:0/471875483 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561a422c6d10 0x561a422c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.320+0000 7f1ceeffd700 10 client.18603 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.320+0000 7f1d0f7fe700 1 --2- 172.21.15.46:0/471875483 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561a422c6d10 0x561a422c9130 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7123d2ba095a9a3 server_cookie=b576c8be3d1249e5 in_seq=0 out_seq=0 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.320+0000 7f1ceeffd700 1 -- 172.21.15.46:0/471875483 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561a421f7700 con 0x561a422c6d10 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.320+0000 7f1ceeffd700 10 client.18603 handle_command_reply: tid=0 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.320+0000 7f1d21d8ab80 1 client.18603 shutdown 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.321+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/471875483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 msgr2=0x561a422b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.321+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/471875483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422b4040 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f1d040037f0 tx=0x7f1d0400eb20 comp rx=0 tx=0).stop 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.321+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/471875483 shutdown_connections 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.321+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/471875483 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561a422c6d10 0x561a422c9130 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.321+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/471875483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.321+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/471875483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.321+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/471875483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422b4040 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.321+0000 7f1ceeffd700 0 client.18603 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:44:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.321+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/471875483 >> 172.21.15.46:0/471875483 conn(0x561a42222220 msgr2=0x561a422a7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.321+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/471875483 shutdown_connections 2024-03-18T20:44:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.321+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/471875483 wait complete. 2024-03-18T20:44:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.321+0000 7f1d21d8ab80 20 client.18603 trim_cache size 0 max 16384 2024-03-18T20:44:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.335+0000 7f1d21d8ab80 1 Processor -- start 2024-03-18T20:44:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.335+0000 7f1d21d8ab80 1 -- start start 2024-03-18T20:44:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d21d8ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422aba60 0x561a422b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d21d8ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422ac380 0x561a422b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d21d8ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422b7220 0x561a422bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d21d8ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a422b3e40 con 0x561a422b7220 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d21d8ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a422be390 con 0x561a422aba60 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d21d8ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a422afe60 con 0x561a422ac380 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422aba60 0x561a422b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422b7220 0x561a422bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422b7220 0x561a422bde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53754/0 (socket says 172.21.15.46:53754) 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422aba60 0x561a422b46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49856/0 (socket says 172.21.15.46:49856) 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0ffff700 1 -- 172.21.15.46:0/169467966 learned_addr learned my addr 172.21.15.46:0/169467966 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0effd700 1 --2- 172.21.15.46:0/169467966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422ac380 0x561a422b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0ffff700 1 -- 172.21.15.46:0/169467966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422ac380 msgr2=0x561a422b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0ffff700 1 --2- 172.21.15.46:0/169467966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422ac380 0x561a422b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0ffff700 1 -- 172.21.15.46:0/169467966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422aba60 msgr2=0x561a422b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0ffff700 1 --2- 172.21.15.46:0/169467966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422aba60 0x561a422b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0ffff700 1 -- 172.21.15.46:0/169467966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a4202cb60 con 0x561a422b7220 2024-03-18T20:44:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0f7fe700 1 --2- 172.21.15.46:0/169467966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422aba60 0x561a422b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.336+0000 7f1d0ffff700 1 --2- 172.21.15.46:0/169467966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422b7220 0x561a422bde50 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800b0a0 tx=0x7f1d08009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af28d3fe864f1cb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.337+0000 7f1cf67fc700 1 -- 172.21.15.46:0/169467966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0800e040 con 0x561a422b7220 2024-03-18T20:44:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.337+0000 7f1cf67fc700 1 -- 172.21.15.46:0/169467966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d08007d40 con 0x561a422b7220 2024-03-18T20:44:46.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.337+0000 7f1cf67fc700 1 -- 172.21.15.46:0/169467966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d080276c0 con 0x561a422b7220 2024-03-18T20:44:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.337+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/169467966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422b7220 msgr2=0x561a422bde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.337+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/169467966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422b7220 0x561a422bde50 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800b0a0 tx=0x7f1d08009520 comp rx=0 tx=0).stop 2024-03-18T20:44:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.337+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/169467966 shutdown_connections 2024-03-18T20:44:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.337+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/169467966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422b7220 0x561a422bde50 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.337+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/169467966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422aba60 0x561a422b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.337+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/169467966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422ac380 0x561a422b3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.337+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/169467966 >> 172.21.15.46:0/169467966 conn(0x561a42222220 msgr2=0x561a422b6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.337+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/169467966 shutdown_connections 2024-03-18T20:44:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.338+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/169467966 wait complete. 2024-03-18T20:44:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.338+0000 7f1d21d8ab80 1 -- start start 2024-03-18T20:44:46.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d21d8ab80 1 Processor -- start 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d21d8ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d21d8ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422bbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d21d8ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d21d8ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a422bc930 con 0x561a422ac380 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d21d8ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a422bcaa0 con 0x561a422b7220 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d21d8ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a422bcc60 con 0x561a422aba60 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d21d8ab80 10 client.0 _ll_register_callbacks cb 0x7f1d04005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d0effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422bbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422bda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37412/0 (socket says 172.21.15.46:37412) 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/586564395 learned_addr learned my addr 172.21.15.46:0/586564395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.339+0000 7f1d0ffff700 1 --2- 172.21.15.46:0/586564395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/586564395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 msgr2=0x561a422bc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1d0f7fe700 1 --2- 172.21.15.46:0/586564395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422bc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/586564395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 msgr2=0x561a422bbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1d0f7fe700 1 --2- 172.21.15.46:0/586564395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422bbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1d0f7fe700 1 -- 172.21.15.46:0/586564395 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d101a65e0 con 0x561a422aba60 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1d0f7fe700 1 --2- 172.21.15.46:0/586564395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422bda10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d000ca660 tx=0x7f1d0001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d95e75b22b8c0e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1ceeffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1ceeffd700 1 -- 172.21.15.46:0/586564395 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00010880 con 0x561a422aba60 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1ceeffd700 1 -- 172.21.15.46:0/586564395 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f1d000b8330 con 0x561a422aba60 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/586564395 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d04004020 con 0x561a422aba60 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1ceeffd700 1 -- 172.21.15.46:0/586564395 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d000109e0 con 0x561a422aba60 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1d21d8ab80 10 client.28264 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/586564395 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f1d080092e0 con 0x561a422aba60 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.340+0000 7f1ceeffd700 1 -- 172.21.15.46:0/586564395 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d04004020 con 0x561a422aba60 2024-03-18T20:44:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1ceeffd700 1 -- 172.21.15.46:0/586564395 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1d00099540 con 0x561a422aba60 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1d21d8ab80 10 client.28264 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1d21d8ab80 10 client.28264 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1d21d8ab80 20 client.28264 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/586564395 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561a4231bff0 0x561a4231e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1d21d8ab80 4 client.28264 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/586564395 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561a4204dcf0 con 0x561a4231bff0 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1d0effd700 1 --2- 172.21.15.46:0/586564395 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561a4231bff0 0x561a4231e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1ceeffd700 10 client.28264 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1d0effd700 1 --2- 172.21.15.46:0/586564395 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561a4231bff0 0x561a4231e410 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93342220ba8b2314 server_cookie=5cb9d64ce662bef8 in_seq=0 out_seq=0 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1ceeffd700 1 -- 172.21.15.46:0/586564395 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561a4204dcf0 con 0x561a4231bff0 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1ceeffd700 10 client.28264 handle_command_reply: tid=0 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.341+0000 7f1d21d8ab80 1 client.28264 shutdown 2024-03-18T20:44:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/586564395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 msgr2=0x561a422bda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/586564395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422bda10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d000ca660 tx=0x7f1d0001a040 comp rx=0 tx=0).stop 2024-03-18T20:44:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/586564395 shutdown_connections 2024-03-18T20:44:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/586564395 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561a4231bff0 0x561a4231e410 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/586564395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561a422b7220 0x561a422bc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1ceeffd700 0 client.28264 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:44:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/586564395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a422aba60 0x561a422bda10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d21d8ab80 1 --2- 172.21.15.46:0/586564395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561a422ac380 0x561a422bbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/586564395 >> 172.21.15.46:0/586564395 conn(0x561a42222220 msgr2=0x561a422af320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/586564395 shutdown_connections 2024-03-18T20:44:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d21d8ab80 1 -- 172.21.15.46:0/586564395 wait complete. 2024-03-18T20:44:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d21d8ab80 20 client.28264 trim_cache size 0 max 16384 2024-03-18T20:44:46.351 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:44:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d15bf5700 1 -- 172.21.15.46:0/103829955 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1cf80a1290 msgr2=0x7f1cf80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d15bf5700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1cf80a1290 0x7f1cf80a3750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d1019aa10 tx=0x7f1d04009260 comp rx=0 tx=0).stop 2024-03-18T20:44:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d15bf5700 1 -- 172.21.15.46:0/103829955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d100ffe50 msgr2=0x7f1d10199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d15bf5700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d100ffe50 0x7f1d10199510 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000e9a0 tx=0x7f1d0000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d15bf5700 1 -- 172.21.15.46:0/103829955 shutdown_connections 2024-03-18T20:44:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d15bf5700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1cf80a1290 0x7f1cf80a3750 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d15bf5700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d100ffe50 0x7f1d10199510 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d15bf5700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d1019dda0 0x7f1d1019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d15bf5700 1 --2- 172.21.15.46:0/103829955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d10104f60 0x7f1d10199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.342+0000 7f1d15bf5700 1 -- 172.21.15.46:0/103829955 >> 172.21.15.46:0/103829955 conn(0x7f1d10076d40 msgr2=0x7f1d100fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.343+0000 7f1d15bf5700 1 -- 172.21.15.46:0/103829955 shutdown_connections 2024-03-18T20:44:46.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.343+0000 7f1d15bf5700 1 -- 172.21.15.46:0/103829955 wait complete. 2024-03-18T20:44:46.359 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:44:46.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:44:46.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.504+0000 7f3392197700 1 -- 172.21.15.46:0/2405873983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f338c0fffd0 msgr2=0x7f338c103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.504+0000 7f3392197700 1 --2- 172.21.15.46:0/2405873983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f338c0fffd0 0x7f338c103fc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f33800098f0 tx=0x7f3380009c00 comp rx=0 tx=0).stop 2024-03-18T20:44:46.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.504+0000 7f3392197700 1 -- 172.21.15.46:0/2405873983 shutdown_connections 2024-03-18T20:44:46.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.504+0000 7f3392197700 1 --2- 172.21.15.46:0/2405873983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338c1046f0 0x7f338c10c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.504+0000 7f3392197700 1 --2- 172.21.15.46:0/2405873983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c0ff620 0x7f338c0ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.504+0000 7f3392197700 1 --2- 172.21.15.46:0/2405873983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f338c0fffd0 0x7f338c103fc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.504+0000 7f3392197700 1 -- 172.21.15.46:0/2405873983 >> 172.21.15.46:0/2405873983 conn(0x7f338c0761e0 msgr2=0x7f338c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.504+0000 7f3392197700 1 -- 172.21.15.46:0/2405873983 shutdown_connections 2024-03-18T20:44:46.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.504+0000 7f3392197700 1 -- 172.21.15.46:0/2405873983 wait complete. 2024-03-18T20:44:46.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.505+0000 7f3392197700 1 Processor -- start 2024-03-18T20:44:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.505+0000 7f3392197700 1 -- start start 2024-03-18T20:44:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.505+0000 7f3392197700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c0ff620 0x7f338c0fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.505+0000 7f3392197700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f338c1046f0 0x7f338c0fbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.505+0000 7f3392197700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338c0fc070 0x7f338c1a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.505+0000 7f3392197700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c1a5a80 con 0x7f338c0fc070 2024-03-18T20:44:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.505+0000 7f3392197700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c1a5bf0 con 0x7f338c1046f0 2024-03-18T20:44:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.505+0000 7f3392197700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c1a5db0 con 0x7f338c0ff620 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338c0fc070 0x7f338c1a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c0ff620 0x7f338c0fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f338c1046f0 0x7f338c0fbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338c0fc070 0x7f338c1a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53770/0 (socket says 172.21.15.46:53770) 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c0ff620 0x7f338c0fb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37430/0 (socket says 172.21.15.46:37430) 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338bfff700 1 -- 172.21.15.46:0/1088534526 learned_addr learned my addr 172.21.15.46:0/1088534526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338affd700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f338c1046f0 0x7f338c0fbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49880/0 (socket says 172.21.15.46:49880) 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338affd700 1 -- 172.21.15.46:0/1088534526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c0ff620 msgr2=0x7f338c0fb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338affd700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c0ff620 0x7f338c0fb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338affd700 1 -- 172.21.15.46:0/1088534526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338c0fc070 msgr2=0x7f338c1a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338affd700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338c0fc070 0x7f338c1a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338affd700 1 -- 172.21.15.46:0/1088534526 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3380009520 con 0x7f338c1046f0 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338bfff700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338c0fc070 0x7f338c1a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338b7fe700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c0ff620 0x7f338c0fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f338affd700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f338c1046f0 0x7f338c0fbb30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3380009230 tx=0x7f338000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f3388ff9700 1 -- 172.21.15.46:0/1088534526 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338001c070 con 0x7f338c1046f0 2024-03-18T20:44:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f3388ff9700 1 -- 172.21.15.46:0/1088534526 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f338000ba20 con 0x7f338c1046f0 2024-03-18T20:44:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f3388ff9700 1 -- 172.21.15.46:0/1088534526 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800038e0 con 0x7f338c1046f0 2024-03-18T20:44:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.506+0000 7f3392197700 1 -- 172.21.15.46:0/1088534526 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338c1a6030 con 0x7f338c1046f0 2024-03-18T20:44:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.507+0000 7f3392197700 1 -- 172.21.15.46:0/1088534526 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c1a6460 con 0x7f338c1046f0 2024-03-18T20:44:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.510+0000 7f3388ff9700 1 -- 172.21.15.46:0/1088534526 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3380020070 con 0x7f338c1046f0 2024-03-18T20:44:46.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.510+0000 7f3388ff9700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f33740a1310 0x7f33740a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.511+0000 7f338b7fe700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f33740a1310 0x7f33740a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.511+0000 7f3388ff9700 1 -- 172.21.15.46:0/1088534526 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f33800cd4b0 con 0x7f338c1046f0 2024-03-18T20:44:46.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.511+0000 7f339e32cb80 1 Processor -- start 2024-03-18T20:44:46.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.511+0000 7f339e32cb80 1 -- start start 2024-03-18T20:44:46.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.511+0000 7f339e32cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1aa60 0x55bea0f1ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.511+0000 7f339e32cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1b380 0x55bea0f25ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.511+0000 7f339e32cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f28610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.511+0000 7f339e32cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea0e74990 con 0x55bea0f26220 2024-03-18T20:44:46.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.511+0000 7f339e32cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea0e72350 con 0x55bea0f1aa60 2024-03-18T20:44:46.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.511+0000 7f339e32cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea0e6dc40 con 0x55bea0f1b380 2024-03-18T20:44:46.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.511+0000 7f338b7fe700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f33740a1310 0x7f33740a37d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f337c011490 tx=0x7f337c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f338bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f28610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f338bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f28610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53776/0 (socket says 172.21.15.46:53776) 2024-03-18T20:44:46.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f338bfff700 1 -- 172.21.15.46:0/1939611327 learned_addr learned my addr 172.21.15.46:0/1939611327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f338b7fe700 1 --2- 172.21.15.46:0/1939611327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1aa60 0x55bea0f1ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f338affd700 1 --2- 172.21.15.46:0/1939611327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1b380 0x55bea0f25ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f338bfff700 1 -- 172.21.15.46:0/1939611327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1b380 msgr2=0x55bea0f25ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f338bfff700 1 --2- 172.21.15.46:0/1939611327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1b380 0x55bea0f25ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f338bfff700 1 -- 172.21.15.46:0/1939611327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1aa60 msgr2=0x55bea0f1ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f338bfff700 1 --2- 172.21.15.46:0/1939611327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1aa60 0x55bea0f1ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f338bfff700 1 -- 172.21.15.46:0/1939611327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bea0cef3b0 con 0x55bea0f26220 2024-03-18T20:44:46.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f338affd700 1 --2- 172.21.15.46:0/1939611327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1b380 0x55bea0f25ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f338bfff700 1 --2- 172.21.15.46:0/1939611327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f28610 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f33840083f0 tx=0x7f33840089b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30434646609575e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f3371ffb700 1 -- 172.21.15.46:0/1939611327 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33840078b0 con 0x55bea0f26220 2024-03-18T20:44:46.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f3371ffb700 1 -- 172.21.15.46:0/1939611327 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3384010040 con 0x55bea0f26220 2024-03-18T20:44:46.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.512+0000 7f3371ffb700 1 -- 172.21.15.46:0/1939611327 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3384016bc0 con 0x55bea0f26220 2024-03-18T20:44:46.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.513+0000 7f339e32cb80 1 -- 172.21.15.46:0/1939611327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 msgr2=0x55bea0f28610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.513+0000 7f339e32cb80 1 --2- 172.21.15.46:0/1939611327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f28610 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f33840083f0 tx=0x7f33840089b0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.513+0000 7f339e32cb80 1 -- 172.21.15.46:0/1939611327 shutdown_connections 2024-03-18T20:44:46.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.513+0000 7f339e32cb80 1 --2- 172.21.15.46:0/1939611327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f28610 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.513+0000 7f339e32cb80 1 --2- 172.21.15.46:0/1939611327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1aa60 0x55bea0f1ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.513+0000 7f339e32cb80 1 --2- 172.21.15.46:0/1939611327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1b380 0x55bea0f25ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.513+0000 7f339e32cb80 1 -- 172.21.15.46:0/1939611327 >> 172.21.15.46:0/1939611327 conn(0x55bea0e91220 msgr2=0x55bea0f17020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.513+0000 7f339e32cb80 1 -- 172.21.15.46:0/1939611327 shutdown_connections 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.513+0000 7f339e32cb80 1 -- 172.21.15.46:0/1939611327 wait complete. 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f339e32cb80 1 -- start start 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f339e32cb80 1 Processor -- start 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f339e32cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f1aa60 0x55bea0f25c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f339e32cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1b380 0x55bea0f22f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f339e32cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f26220 0x55bea0f23590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f339e32cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea0e09d80 con 0x55bea0f1aa60 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f339e32cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea0e09a20 con 0x55bea0f1b380 2024-03-18T20:44:46.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f339e32cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea0cf16a0 con 0x55bea0f26220 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f339e32cb80 10 client.0 _ll_register_callbacks cb 0x55bea0c753a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f338bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f26220 0x55bea0f23590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f338b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f1aa60 0x55bea0f25c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f338bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f26220 0x55bea0f23590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37442/0 (socket says 172.21.15.46:37442) 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f338affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1b380 0x55bea0f22f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.514+0000 7f338bfff700 1 -- 172.21.15.46:0/2870951828 learned_addr learned my addr 172.21.15.46:0/2870951828 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f338affd700 1 -- 172.21.15.46:0/2870951828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f26220 msgr2=0x55bea0f23590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f338affd700 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f26220 0x55bea0f23590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f338affd700 1 -- 172.21.15.46:0/2870951828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f1aa60 msgr2=0x55bea0f25c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f338affd700 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f1aa60 0x55bea0f25c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f338affd700 1 -- 172.21.15.46:0/2870951828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c1a6460 con 0x55bea0f1b380 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f338bfff700 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f26220 0x55bea0f23590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f338b7fe700 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f1aa60 0x55bea0f25c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f338affd700 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1b380 0x55bea0f22f10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f33800cab80 tx=0x7f3380009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d45f10fe093905d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f336affd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f336affd700 1 -- 172.21.15.46:0/2870951828 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338009a5d0 con 0x55bea0f1b380 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f336affd700 1 -- 172.21.15.46:0/2870951828 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f33800b5020 con 0x55bea0f1b380 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f339e32cb80 1 -- 172.21.15.46:0/2870951828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bea0f0ead0 con 0x55bea0f1b380 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f336affd700 1 -- 172.21.15.46:0/2870951828 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f338009e400 con 0x55bea0f1b380 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f339e32cb80 10 client.28268 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f339e32cb80 1 -- 172.21.15.46:0/2870951828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f338400a560 con 0x55bea0f1b380 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.515+0000 7f336affd700 1 -- 172.21.15.46:0/2870951828 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bea0f0ead0 con 0x55bea0f1b380 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.516+0000 7f336affd700 1 -- 172.21.15.46:0/2870951828 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f33800050d0 con 0x55bea0f1b380 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.516+0000 7f339e32cb80 10 client.28268 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.516+0000 7f339e32cb80 10 client.28268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.516+0000 7f339e32cb80 20 client.28268 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.516+0000 7f339e32cb80 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bea0f3e260 0x55bea0f40680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.516+0000 7f339e32cb80 4 client.28268 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:46.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.516+0000 7f339e32cb80 1 -- 172.21.15.46:0/2870951828 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bea0e66700 con 0x55bea0f3e260 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.516+0000 7f338b7fe700 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bea0f3e260 0x55bea0f40680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.516+0000 7f338b7fe700 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bea0f3e260 0x55bea0f40680 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28df4fba965a28a2 server_cookie=a674418d9b291355 in_seq=0 out_seq=0 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.516+0000 7f336affd700 10 client.28268 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f336affd700 1 -- 172.21.15.46:0/2870951828 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bea0e66700 con 0x55bea0f3e260 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f336affd700 10 client.28268 handle_command_reply: tid=0 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f339e32cb80 1 client.28268 shutdown 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f339e32cb80 1 -- 172.21.15.46:0/2870951828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1b380 msgr2=0x55bea0f22f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f339e32cb80 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1b380 0x55bea0f22f10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f33800cab80 tx=0x7f3380009c00 comp rx=0 tx=0).stop 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f339e32cb80 1 -- 172.21.15.46:0/2870951828 shutdown_connections 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f339e32cb80 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bea0f3e260 0x55bea0f40680 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f339e32cb80 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f1aa60 0x55bea0f25c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f339e32cb80 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f26220 0x55bea0f23590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f336affd700 0 client.28268 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f339e32cb80 1 --2- 172.21.15.46:0/2870951828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1b380 0x55bea0f22f10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f339e32cb80 1 -- 172.21.15.46:0/2870951828 >> 172.21.15.46:0/2870951828 conn(0x55bea0e91220 msgr2=0x55bea0f16c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f339e32cb80 1 -- 172.21.15.46:0/2870951828 shutdown_connections 2024-03-18T20:44:46.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f339e32cb80 1 -- 172.21.15.46:0/2870951828 wait complete. 2024-03-18T20:44:46.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.517+0000 7f339e32cb80 20 client.28268 trim_cache size 0 max 16384 2024-03-18T20:44:46.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.531+0000 7f339e32cb80 1 Processor -- start 2024-03-18T20:44:46.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.531+0000 7f339e32cb80 1 -- start start 2024-03-18T20:44:46.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f339e32cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1aa60 0x55bea0f23590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f339e32cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1b380 0x55bea0f35650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f339e32cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f217e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f339e32cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea0f23ea0 con 0x55bea0f26220 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f339e32cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea0f1c3d0 con 0x55bea0f1aa60 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f339e32cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea0f35b90 con 0x55bea0f1b380 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f338bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f338bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f217e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53800/0 (socket says 172.21.15.46:53800) 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f338b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1aa60 0x55bea0f23590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f338affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1b380 0x55bea0f35650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f338bfff700 1 -- 172.21.15.46:0/3582603564 learned_addr learned my addr 172.21.15.46:0/3582603564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f338bfff700 1 -- 172.21.15.46:0/3582603564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1b380 msgr2=0x55bea0f35650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f338bfff700 1 --2- 172.21.15.46:0/3582603564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1b380 0x55bea0f35650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f338bfff700 1 -- 172.21.15.46:0/3582603564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1aa60 msgr2=0x55bea0f23590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f338bfff700 1 --2- 172.21.15.46:0/3582603564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1aa60 0x55bea0f23590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f338bfff700 1 -- 172.21.15.46:0/3582603564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bea0c9bb60 con 0x55bea0f26220 2024-03-18T20:44:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.532+0000 7f338bfff700 1 --2- 172.21.15.46:0/3582603564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f217e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f33840062a0 tx=0x7f3384004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b4288776d372c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.533+0000 7f33727fc700 1 -- 172.21.15.46:0/3582603564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3384016480 con 0x55bea0f26220 2024-03-18T20:44:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.533+0000 7f33727fc700 1 -- 172.21.15.46:0/3582603564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3384016c30 con 0x55bea0f26220 2024-03-18T20:44:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.533+0000 7f33727fc700 1 -- 172.21.15.46:0/3582603564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3384007ce0 con 0x55bea0f26220 2024-03-18T20:44:46.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.533+0000 7f339e32cb80 1 -- 172.21.15.46:0/3582603564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 msgr2=0x55bea0f217e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.533+0000 7f339e32cb80 1 --2- 172.21.15.46:0/3582603564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f217e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f33840062a0 tx=0x7f3384004a30 comp rx=0 tx=0).stop 2024-03-18T20:44:46.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.533+0000 7f339e32cb80 1 -- 172.21.15.46:0/3582603564 shutdown_connections 2024-03-18T20:44:46.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.533+0000 7f339e32cb80 1 --2- 172.21.15.46:0/3582603564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f217e0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.533+0000 7f339e32cb80 1 --2- 172.21.15.46:0/3582603564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1aa60 0x55bea0f23590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.533+0000 7f339e32cb80 1 --2- 172.21.15.46:0/3582603564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1b380 0x55bea0f35650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.533+0000 7f339e32cb80 1 -- 172.21.15.46:0/3582603564 >> 172.21.15.46:0/3582603564 conn(0x55bea0e91220 msgr2=0x55bea0f15750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.533+0000 7f339e32cb80 1 -- 172.21.15.46:0/3582603564 shutdown_connections 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.534+0000 7f339e32cb80 1 -- 172.21.15.46:0/3582603564 wait complete. 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.534+0000 7f339e32cb80 1 -- start start 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.534+0000 7f339e32cb80 1 Processor -- start 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.534+0000 7f339e32cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1aa60 0x55bea0f31630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.534+0000 7f339e32cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1b380 0x55bea0f31b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.534+0000 7f339e32cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f320b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.534+0000 7f339e32cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea0f911c0 con 0x55bea0f26220 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.534+0000 7f339e32cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea0f91330 con 0x55bea0f1b380 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.534+0000 7f339e32cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bea0f914f0 con 0x55bea0f1aa60 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f339e32cb80 10 client.0 _ll_register_callbacks cb 0x7f338009a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:46.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1aa60 0x55bea0f31630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1aa60 0x55bea0f31630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37464/0 (socket says 172.21.15.46:37464) 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f320b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53810/0 (socket says 172.21.15.46:53810) 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338b7fe700 1 -- 172.21.15.46:0/2065097154 learned_addr learned my addr 172.21.15.46:0/2065097154 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338affd700 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1b380 0x55bea0f31b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338bfff700 1 -- 172.21.15.46:0/2065097154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1aa60 msgr2=0x55bea0f31630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338bfff700 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1aa60 0x55bea0f31630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338bfff700 1 -- 172.21.15.46:0/2065097154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1b380 msgr2=0x55bea0f31b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338bfff700 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1b380 0x55bea0f31b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338bfff700 1 -- 172.21.15.46:0/2065097154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bea0cef3b0 con 0x55bea0f26220 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338b7fe700 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1aa60 0x55bea0f31630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f338bfff700 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f320b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f338400afd0 tx=0x7f3384016b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674f77c60744a373 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f336affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:46.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f336affd700 1 -- 172.21.15.46:0/2065097154 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3384007b60 con 0x55bea0f26220 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.535+0000 7f336affd700 1 -- 172.21.15.46:0/2065097154 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3384007cc0 con 0x55bea0f26220 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f339e32cb80 1 -- 172.21.15.46:0/2065097154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3384010040 con 0x55bea0f26220 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f336affd700 1 -- 172.21.15.46:0/2065097154 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3384007e80 con 0x55bea0f26220 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f339e32cb80 10 client.18627 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f339e32cb80 1 -- 172.21.15.46:0/2065097154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55bea0e33ea0 con 0x55bea0f26220 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f336affd700 1 -- 172.21.15.46:0/2065097154 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3384010040 con 0x55bea0f26220 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f336affd700 1 -- 172.21.15.46:0/2065097154 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3384028420 con 0x55bea0f26220 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f339e32cb80 10 client.18627 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f339e32cb80 10 client.18627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f339e32cb80 20 client.18627 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f339e32cb80 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bea0f92900 0x55bea0f94cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f339e32cb80 4 client.18627 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:44:46.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f339e32cb80 1 -- 172.21.15.46:0/2065097154 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bea0cbccf0 con 0x55bea0f92900 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.536+0000 7f338b7fe700 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bea0f92900 0x55bea0f94cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f336affd700 10 client.18627 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f338b7fe700 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bea0f92900 0x55bea0f94cf0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9385eee42e2fc28 server_cookie=2161e27ebd32d99b in_seq=0 out_seq=0 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f336affd700 1 -- 172.21.15.46:0/2065097154 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bea0cbccf0 con 0x55bea0f92900 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f336affd700 10 client.18627 handle_command_reply: tid=0 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f339e32cb80 1 client.18627 shutdown 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f339e32cb80 1 -- 172.21.15.46:0/2065097154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 msgr2=0x55bea0f320b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f339e32cb80 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f320b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f338400afd0 tx=0x7f3384016b30 comp rx=0 tx=0).stop 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f339e32cb80 1 -- 172.21.15.46:0/2065097154 shutdown_connections 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f339e32cb80 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bea0f92900 0x55bea0f94cf0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f339e32cb80 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bea0f26220 0x55bea0f320b0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f336affd700 0 client.18627 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:46.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f339e32cb80 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bea0f1aa60 0x55bea0f31630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f339e32cb80 1 --2- 172.21.15.46:0/2065097154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bea0f1b380 0x55bea0f31b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f339e32cb80 1 -- 172.21.15.46:0/2065097154 >> 172.21.15.46:0/2065097154 conn(0x55bea0e91220 msgr2=0x55bea0f21750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f339e32cb80 1 -- 172.21.15.46:0/2065097154 shutdown_connections 2024-03-18T20:44:46.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.537+0000 7f339e32cb80 1 -- 172.21.15.46:0/2065097154 wait complete. 2024-03-18T20:44:46.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f339e32cb80 20 client.18627 trim_cache size 0 max 16384 2024-03-18T20:44:46.546 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:44:46.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f3392197700 1 -- 172.21.15.46:0/1088534526 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f33740a1310 msgr2=0x7f33740a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f3392197700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f33740a1310 0x7f33740a37d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f337c011490 tx=0x7f337c009260 comp rx=0 tx=0).stop 2024-03-18T20:44:46.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f3392197700 1 -- 172.21.15.46:0/1088534526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f338c1046f0 msgr2=0x7f338c0fbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:46.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f3392197700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f338c1046f0 0x7f338c0fbb30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3380009230 tx=0x7f338000fa00 comp rx=0 tx=0).stop 2024-03-18T20:44:46.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f3392197700 1 -- 172.21.15.46:0/1088534526 shutdown_connections 2024-03-18T20:44:46.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f3392197700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f33740a1310 0x7f33740a37d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f3392197700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f338c0fc070 0x7f338c1a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f3392197700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f338c0ff620 0x7f338c0fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f3392197700 1 --2- 172.21.15.46:0/1088534526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f338c1046f0 0x7f338c0fbb30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:46.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f3392197700 1 -- 172.21.15.46:0/1088534526 >> 172.21.15.46:0/1088534526 conn(0x7f338c0761e0 msgr2=0x7f338c102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:46.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f3392197700 1 -- 172.21.15.46:0/1088534526 shutdown_connections 2024-03-18T20:44:46.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:46.538+0000 7f3392197700 1 -- 172.21.15.46:0/1088534526 wait complete. 2024-03-18T20:44:46.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 309170 3.59 MB/sec execute 341 sec latency 16.788 ms 2024-03-18T20:44:47.555 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:44:47.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:44:47.556 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:44:47.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 310305 3.59 MB/sec execute 342 sec latency 327.333 ms 2024-03-18T20:44:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.704+0000 7f4b6593d700 1 -- 172.21.15.46:0/2721387499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b60101900 msgr2=0x7f4b60105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.704+0000 7f4b6593d700 1 --2- 172.21.15.46:0/2721387499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b60101900 0x7f4b60105940 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009990 tx=0x7f4b48009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.704+0000 7f4b6593d700 1 -- 172.21.15.46:0/2721387499 shutdown_connections 2024-03-18T20:44:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.704+0000 7f4b6593d700 1 --2- 172.21.15.46:0/2721387499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b60106070 0x7f4b6010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.704+0000 7f4b6593d700 1 --2- 172.21.15.46:0/2721387499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b60100f50 0x7f4b60101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.704+0000 7f4b6593d700 1 --2- 172.21.15.46:0/2721387499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b60101900 0x7f4b60105940 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.704+0000 7f4b6593d700 1 -- 172.21.15.46:0/2721387499 >> 172.21.15.46:0/2721387499 conn(0x7f4b600fc7f0 msgr2=0x7f4b600fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.704+0000 7f4b6593d700 1 -- 172.21.15.46:0/2721387499 shutdown_connections 2024-03-18T20:44:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.704+0000 7f4b6593d700 1 -- 172.21.15.46:0/2721387499 wait complete. 2024-03-18T20:44:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.705+0000 7f4b6593d700 1 Processor -- start 2024-03-18T20:44:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.705+0000 7f4b6593d700 1 -- start start 2024-03-18T20:44:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.705+0000 7f4b6593d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b60100f50 0x7f4b60197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.705+0000 7f4b6593d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b60106070 0x7f4b601977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.705+0000 7f4b6593d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b6019baf0 0x7f4b6019bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.705+0000 7f4b6593d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6019c570 con 0x7f4b6019baf0 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.705+0000 7f4b6593d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6019c6e0 con 0x7f4b60106070 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.705+0000 7f4b6593d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6019c8a0 con 0x7f4b60100f50 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b6019baf0 0x7f4b6019bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b567fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b60106070 0x7f4b601977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b6019baf0 0x7f4b6019bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53826/0 (socket says 172.21.15.46:53826) 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b5f7fe700 1 -- 172.21.15.46:0/3099629469 learned_addr learned my addr 172.21.15.46:0/3099629469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b5effd700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b60100f50 0x7f4b60197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b5f7fe700 1 -- 172.21.15.46:0/3099629469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b60100f50 msgr2=0x7f4b60197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b5f7fe700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b60100f50 0x7f4b60197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b5f7fe700 1 -- 172.21.15.46:0/3099629469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b60106070 msgr2=0x7f4b601977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b5f7fe700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b60106070 0x7f4b601977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b5f7fe700 1 -- 172.21.15.46:0/3099629469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b480095f0 con 0x7f4b6019baf0 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b567fc700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b60106070 0x7f4b601977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b5f7fe700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b6019baf0 0x7f4b6019bf70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800ef30 tx=0x7f4b5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:47.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b6493b700 1 -- 172.21.15.46:0/3099629469 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58018070 con 0x7f4b6019baf0 2024-03-18T20:44:47.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b6493b700 1 -- 172.21.15.46:0/3099629469 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5800f040 con 0x7f4b6019baf0 2024-03-18T20:44:47.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b6593d700 1 -- 172.21.15.46:0/3099629469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b601a3da0 con 0x7f4b6019baf0 2024-03-18T20:44:47.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b6493b700 1 -- 172.21.15.46:0/3099629469 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800cd00 con 0x7f4b6019baf0 2024-03-18T20:44:47.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.706+0000 7f4b6593d700 1 -- 172.21.15.46:0/3099629469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b601a4230 con 0x7f4b6019baf0 2024-03-18T20:44:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.709+0000 7f4b6493b700 1 -- 172.21.15.46:0/3099629469 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f4b58003a00 con 0x7f4b6019baf0 2024-03-18T20:44:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.710+0000 7f4b6493b700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4b4c0a1290 0x7f4b4c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.710+0000 7f4b5effd700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4b4c0a1290 0x7f4b4c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.710+0000 7f4b6493b700 1 -- 172.21.15.46:0/3099629469 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f4b580cc850 con 0x7f4b6019baf0 2024-03-18T20:44:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.710+0000 7f4b5effd700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4b4c0a1290 0x7f4b4c0a3750 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b50005950 tx=0x7f4b5000b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:47.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.710+0000 7f4b71ad2b80 1 Processor -- start 2024-03-18T20:44:47.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b71ad2b80 1 -- start start 2024-03-18T20:44:47.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b71ad2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37deae0 0x557de37deec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b71ad2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37e9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b71ad2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37ea2a0 0x557de37ec690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b71ad2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de3738a10 con 0x557de37df400 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b71ad2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de37363d0 con 0x557de37ea2a0 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b71ad2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de3731cc0 con 0x557de37deae0 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37e9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37e9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53834/0 (socket says 172.21.15.46:53834) 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b567fc700 1 -- 172.21.15.46:0/2468553926 learned_addr learned my addr 172.21.15.46:0/2468553926 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b5effd700 1 --2- 172.21.15.46:0/2468553926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37deae0 0x557de37deec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b5f7fe700 1 --2- 172.21.15.46:0/2468553926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37ea2a0 0x557de37ec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b567fc700 1 -- 172.21.15.46:0/2468553926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37deae0 msgr2=0x557de37deec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b567fc700 1 --2- 172.21.15.46:0/2468553926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37deae0 0x557de37deec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b567fc700 1 -- 172.21.15.46:0/2468553926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37ea2a0 msgr2=0x557de37ec690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b567fc700 1 --2- 172.21.15.46:0/2468553926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37ea2a0 0x557de37ec690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b567fc700 1 -- 172.21.15.46:0/2468553926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557de35b3430 con 0x557de37df400 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.711+0000 7f4b567fc700 1 --2- 172.21.15.46:0/2468553926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37e9d60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009990 tx=0x7f4b48005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23bef3e10c39c1e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.712+0000 7f4b55ffb700 1 -- 172.21.15.46:0/2468553926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4801d050 con 0x557de37df400 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.712+0000 7f4b55ffb700 1 -- 172.21.15.46:0/2468553926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b4801e040 con 0x557de37df400 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.712+0000 7f4b55ffb700 1 -- 172.21.15.46:0/2468553926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48005620 con 0x557de37df400 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.712+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/2468553926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 msgr2=0x557de37e9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.712+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/2468553926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37e9d60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009990 tx=0x7f4b48005bc0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.712+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/2468553926 shutdown_connections 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.712+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/2468553926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37ea2a0 0x557de37ec690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.712+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/2468553926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37deae0 0x557de37deec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.712+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/2468553926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37e9d60 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.712+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/2468553926 >> 172.21.15.46:0/2468553926 conn(0x557de37552a0 msgr2=0x557de37db0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.712+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/2468553926 shutdown_connections 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.712+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/2468553926 wait complete. 2024-03-18T20:44:47.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.713+0000 7f4b71ad2b80 1 -- start start 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.713+0000 7f4b71ad2b80 1 Processor -- start 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.713+0000 7f4b71ad2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37deae0 0x557de37e9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.713+0000 7f4b71ad2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37e7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.713+0000 7f4b71ad2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37ea2a0 0x557de37e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.713+0000 7f4b71ad2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de36cde00 con 0x557de37df400 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.713+0000 7f4b71ad2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de36cdaa0 con 0x557de37deae0 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.713+0000 7f4b71ad2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de35b5720 con 0x557de37ea2a0 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b71ad2b80 10 client.0 _ll_register_callbacks cb 0x557de3539420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37e7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37e7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53842/0 (socket says 172.21.15.46:53842) 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b567fc700 1 -- 172.21.15.46:0/1497966058 learned_addr learned my addr 172.21.15.46:0/1497966058 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b5f7fe700 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37ea2a0 0x557de37e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b5effd700 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37deae0 0x557de37e9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b567fc700 1 -- 172.21.15.46:0/1497966058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37ea2a0 msgr2=0x557de37e76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b567fc700 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37ea2a0 0x557de37e76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b567fc700 1 -- 172.21.15.46:0/1497966058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37deae0 msgr2=0x557de37e9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b567fc700 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37deae0 0x557de37e9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b567fc700 1 -- 172.21.15.46:0/1497966058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6010e460 con 0x557de37df400 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b5f7fe700 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37ea2a0 0x557de37e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b567fc700 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37e7030 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f4b480051a0 tx=0x7f4b480054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b0a18a7bbc61bd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b3effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b3effd700 1 -- 172.21.15.46:0/1497966058 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480056c0 con 0x557de37df400 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b3effd700 1 -- 172.21.15.46:0/1497966058 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f4b480049f0 con 0x557de37df400 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.714+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/1497966058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557de37d2b50 con 0x557de37df400 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.715+0000 7f4b3effd700 1 -- 172.21.15.46:0/1497966058 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b4801d020 con 0x557de37df400 2024-03-18T20:44:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.715+0000 7f4b71ad2b80 10 client.18639 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.715+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/1497966058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x557de355fbe0 con 0x557de37df400 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.715+0000 7f4b3effd700 1 -- 172.21.15.46:0/1497966058 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557de37d2b50 con 0x557de37df400 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.715+0000 7f4b3effd700 1 -- 172.21.15.46:0/1497966058 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4b48023cf0 con 0x557de37df400 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.715+0000 7f4b71ad2b80 10 client.18639 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.715+0000 7f4b71ad2b80 10 client.18639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.715+0000 7f4b71ad2b80 20 client.18639 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.715+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557de37fddd0 0x557de38001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.715+0000 7f4b71ad2b80 4 client.18639 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.715+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/1497966058 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557de372a780 con 0x557de37fddd0 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.715+0000 7f4b5effd700 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557de37fddd0 0x557de38001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b3effd700 10 client.18639 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b5effd700 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557de37fddd0 0x557de38001c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d47371537c5fe70c server_cookie=e28478f244328670 in_seq=0 out_seq=0 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b3effd700 1 -- 172.21.15.46:0/1497966058 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557de372a780 con 0x557de37fddd0 2024-03-18T20:44:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b3effd700 10 client.18639 handle_command_reply: tid=0 2024-03-18T20:44:47.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b71ad2b80 1 client.18639 shutdown 2024-03-18T20:44:47.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/1497966058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 msgr2=0x557de37e7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37e7030 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f4b480051a0 tx=0x7f4b480054d0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/1497966058 shutdown_connections 2024-03-18T20:44:47.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557de37fddd0 0x557de38001c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37ea2a0 0x557de37e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37deae0 0x557de37e9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/1497966058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37e7030 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b3effd700 0 client.18639 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:47.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.716+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/1497966058 >> 172.21.15.46:0/1497966058 conn(0x557de37552a0 msgr2=0x557de37dac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:47.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.717+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/1497966058 shutdown_connections 2024-03-18T20:44:47.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.717+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/1497966058 wait complete. 2024-03-18T20:44:47.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.717+0000 7f4b71ad2b80 20 client.18639 trim_cache size 0 max 16384 2024-03-18T20:44:47.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b71ad2b80 1 Processor -- start 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b71ad2b80 1 -- start start 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b71ad2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37deae0 0x557de37fa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b71ad2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37f5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b71ad2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37ea2a0 0x557de37e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b71ad2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de37a59c0 con 0x557de37df400 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b71ad2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de37fad60 con 0x557de37ea2a0 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b71ad2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de37dc2b0 con 0x557de37deae0 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53858/0 (socket says 172.21.15.46:53858) 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b567fc700 1 -- 172.21.15.46:0/3394431385 learned_addr learned my addr 172.21.15.46:0/3394431385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b5f7fe700 1 --2- 172.21.15.46:0/3394431385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37ea2a0 0x557de37e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b5effd700 1 --2- 172.21.15.46:0/3394431385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37deae0 0x557de37fa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b567fc700 1 -- 172.21.15.46:0/3394431385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37deae0 msgr2=0x557de37fa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b567fc700 1 --2- 172.21.15.46:0/3394431385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37deae0 0x557de37fa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b567fc700 1 -- 172.21.15.46:0/3394431385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37ea2a0 msgr2=0x557de37e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b567fc700 1 --2- 172.21.15.46:0/3394431385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37ea2a0 0x557de37e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b567fc700 1 -- 172.21.15.46:0/3394431385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557de36f7f20 con 0x557de37df400 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.735+0000 7f4b567fc700 1 --2- 172.21.15.46:0/3394431385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37f5300 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800f4b0 tx=0x7f4b48009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e43ec4b618f8f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.736+0000 7f4b56ffd700 1 -- 172.21.15.46:0/3394431385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4801d040 con 0x557de37df400 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.736+0000 7f4b56ffd700 1 -- 172.21.15.46:0/3394431385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b4800f7a0 con 0x557de37df400 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.736+0000 7f4b56ffd700 1 -- 172.21.15.46:0/3394431385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48023460 con 0x557de37df400 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.736+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3394431385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 msgr2=0x557de37f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.736+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/3394431385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37f5300 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800f4b0 tx=0x7f4b48009310 comp rx=0 tx=0).stop 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.736+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3394431385 shutdown_connections 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.736+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/3394431385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37ea2a0 0x557de37e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.736+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/3394431385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37deae0 0x557de37fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.736+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/3394431385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37f5300 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.736+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3394431385 >> 172.21.15.46:0/3394431385 conn(0x557de37552a0 msgr2=0x557de37f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.736+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3394431385 shutdown_connections 2024-03-18T20:44:47.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.736+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3394431385 wait complete. 2024-03-18T20:44:47.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b71ad2b80 1 -- start start 2024-03-18T20:44:47.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b71ad2b80 1 Processor -- start 2024-03-18T20:44:47.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b71ad2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37deae0 0x557de37f5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b71ad2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37f3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b71ad2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37ea2a0 0x557de37f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b71ad2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de37f4620 con 0x557de37df400 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b71ad2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de37f4790 con 0x557de37deae0 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b71ad2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557de3851bd0 con 0x557de37ea2a0 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b71ad2b80 10 client.0 _ll_register_callbacks cb 0x7f4b5000b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37deae0 0x557de37f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37f3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53870/0 (socket says 172.21.15.46:53870) 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b5f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37ea2a0 0x557de37f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b5effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37deae0 0x557de37f5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49964/0 (socket says 172.21.15.46:49964) 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.737+0000 7f4b567fc700 1 -- 172.21.15.46:0/3861660820 learned_addr learned my addr 172.21.15.46:0/3861660820 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b567fc700 1 -- 172.21.15.46:0/3861660820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37ea2a0 msgr2=0x557de37f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b567fc700 1 --2- 172.21.15.46:0/3861660820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37ea2a0 0x557de37f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b567fc700 1 -- 172.21.15.46:0/3861660820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37deae0 msgr2=0x557de37f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b567fc700 1 --2- 172.21.15.46:0/3861660820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37deae0 0x557de37f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b567fc700 1 -- 172.21.15.46:0/3861660820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557de355fbe0 con 0x557de37df400 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b5effd700 1 --2- 172.21.15.46:0/3861660820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37deae0 0x557de37f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b567fc700 1 --2- 172.21.15.46:0/3861660820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37f3910 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800b2e0 tx=0x7f4b48010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78364f590a4f3e47 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b3effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b3effd700 1 -- 172.21.15.46:0/3861660820 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4801d040 con 0x557de37df400 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b3effd700 1 -- 172.21.15.46:0/3861660820 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f4b48004c00 con 0x557de37df400 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3861660820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b4801e040 con 0x557de37df400 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b3effd700 1 -- 172.21.15.46:0/3861660820 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b48004e00 con 0x557de37df400 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b71ad2b80 10 client.18645 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3861660820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x557de35b7d50 con 0x557de37df400 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.738+0000 7f4b3effd700 1 -- 172.21.15.46:0/3861660820 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b4801e040 con 0x557de37df400 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.739+0000 7f4b3effd700 1 -- 172.21.15.46:0/3861660820 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4b48023460 con 0x557de37df400 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.739+0000 7f4b71ad2b80 10 client.18645 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.739+0000 7f4b71ad2b80 10 client.18645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:47.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.739+0000 7f4b71ad2b80 20 client.18645 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.739+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/3861660820 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557de38530a0 0x557de38554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.739+0000 7f4b71ad2b80 4 client.18645 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.739+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3861660820 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557de3580d70 con 0x557de38530a0 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.739+0000 7f4b5effd700 1 --2- 172.21.15.46:0/3861660820 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557de38530a0 0x557de38554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.739+0000 7f4b5effd700 1 --2- 172.21.15.46:0/3861660820 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557de38530a0 0x557de38554c0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d4d8b7706c98b4a server_cookie=97cc35e4609de8cd in_seq=0 out_seq=0 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.739+0000 7f4b3effd700 10 client.18645 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.740+0000 7f4b3effd700 1 -- 172.21.15.46:0/3861660820 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557de3580d70 con 0x557de38530a0 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.740+0000 7f4b3effd700 10 client.18645 handle_command_reply: tid=0 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.740+0000 7f4b71ad2b80 1 client.18645 shutdown 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.740+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3861660820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 msgr2=0x557de37f3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.740+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/3861660820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37f3910 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800b2e0 tx=0x7f4b48010d10 comp rx=0 tx=0).stop 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.740+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3861660820 shutdown_connections 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.740+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/3861660820 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557de38530a0 0x557de38554c0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.740+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/3861660820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557de37ea2a0 0x557de37f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.740+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/3861660820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557de37deae0 0x557de37f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.740+0000 7f4b71ad2b80 1 --2- 172.21.15.46:0/3861660820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557de37df400 0x557de37f3910 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.740+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3861660820 >> 172.21.15.46:0/3861660820 conn(0x557de37552a0 msgr2=0x557de37e8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:47.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.740+0000 7f4b3effd700 0 client.18645 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:47.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3861660820 shutdown_connections 2024-03-18T20:44:47.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b71ad2b80 1 -- 172.21.15.46:0/3861660820 wait complete. 2024-03-18T20:44:47.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b71ad2b80 20 client.18645 trim_cache size 0 max 16384 2024-03-18T20:44:47.749 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:44:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:44:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "9e5fd927-c63e-42cf-a274-6b47897bcc4d", 2024-03-18T20:44:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:44:47.750 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:44:47.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b6593d700 1 -- 172.21.15.46:0/3099629469 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4b4c0a1290 msgr2=0x7f4b4c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b6593d700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4b4c0a1290 0x7f4b4c0a3750 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4b50005950 tx=0x7f4b5000b410 comp rx=0 tx=0).stop 2024-03-18T20:44:47.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b6593d700 1 -- 172.21.15.46:0/3099629469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b6019baf0 msgr2=0x7f4b6019bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b6593d700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b6019baf0 0x7f4b6019bf70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800ef30 tx=0x7f4b5800c3a0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b6593d700 1 -- 172.21.15.46:0/3099629469 shutdown_connections 2024-03-18T20:44:47.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b6593d700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4b4c0a1290 0x7f4b4c0a3750 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b6593d700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b6019baf0 0x7f4b6019bf70 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b6593d700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b60100f50 0x7f4b60197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b6593d700 1 --2- 172.21.15.46:0/3099629469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b60106070 0x7f4b601977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b6593d700 1 -- 172.21.15.46:0/3099629469 >> 172.21.15.46:0/3099629469 conn(0x7f4b600fc7f0 msgr2=0x7f4b600fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:47.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.741+0000 7f4b6593d700 1 -- 172.21.15.46:0/3099629469 shutdown_connections 2024-03-18T20:44:47.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.742+0000 7f4b6593d700 1 -- 172.21.15.46:0/3099629469 wait complete. 2024-03-18T20:44:47.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:44:47.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.910+0000 7f8de4e03700 1 -- 172.21.15.46:0/936938137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de0101900 msgr2=0x7f8de0105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.910+0000 7f8de4e03700 1 --2- 172.21.15.46:0/936938137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de0101900 0x7f8de0105940 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009990 tx=0x7f8dd4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.910+0000 7f8de4e03700 1 -- 172.21.15.46:0/936938137 shutdown_connections 2024-03-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.910+0000 7f8de4e03700 1 --2- 172.21.15.46:0/936938137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8de0106070 0x7f8de010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.910+0000 7f8de4e03700 1 --2- 172.21.15.46:0/936938137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8de0100f50 0x7f8de0101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.910+0000 7f8de4e03700 1 --2- 172.21.15.46:0/936938137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de0101900 0x7f8de0105940 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.910+0000 7f8de4e03700 1 -- 172.21.15.46:0/936938137 >> 172.21.15.46:0/936938137 conn(0x7f8de00fc7f0 msgr2=0x7f8de00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.910+0000 7f8de4e03700 1 -- 172.21.15.46:0/936938137 shutdown_connections 2024-03-18T20:44:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.910+0000 7f8de4e03700 1 -- 172.21.15.46:0/936938137 wait complete. 2024-03-18T20:44:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8de4e03700 1 Processor -- start 2024-03-18T20:44:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8de4e03700 1 -- start start 2024-03-18T20:44:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8de4e03700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8de0100f50 0x7f8de0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8de4e03700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de0106070 0x7f8de0197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8de4e03700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8de019ba90 0x7f8de019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8de4e03700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de019c510 con 0x7f8de0106070 2024-03-18T20:44:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8de4e03700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de019c680 con 0x7f8de0100f50 2024-03-18T20:44:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8de4e03700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de019c840 con 0x7f8de019ba90 2024-03-18T20:44:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8dddd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de0106070 0x7f8de0197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8dde59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8de0100f50 0x7f8de0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8dddd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de0106070 0x7f8de0197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53878/0 (socket says 172.21.15.46:53878) 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8dddd9b700 1 -- 172.21.15.46:0/2180028598 learned_addr learned my addr 172.21.15.46:0/2180028598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8dde59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8de0100f50 0x7f8de0197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49982/0 (socket says 172.21.15.46:49982) 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.911+0000 7f8dded9d700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8de019ba90 0x7f8de019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8dde59c700 1 -- 172.21.15.46:0/2180028598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8de019ba90 msgr2=0x7f8de019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8dde59c700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8de019ba90 0x7f8de019bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8dde59c700 1 -- 172.21.15.46:0/2180028598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de0106070 msgr2=0x7f8de0197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8dde59c700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de0106070 0x7f8de0197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8dde59c700 1 -- 172.21.15.46:0/2180028598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40095f0 con 0x7f8de0100f50 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de0106070 0x7f8de0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8dded9d700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8de019ba90 0x7f8de019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8dde59c700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8de0100f50 0x7f8de0197200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000e9f0 tx=0x7f8dd000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8dcf7fe700 1 -- 172.21.15.46:0/2180028598 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd000c9c0 con 0x7f8de0100f50 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8de4e03700 1 -- 172.21.15.46:0/2180028598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de01a3da0 con 0x7f8de0100f50 2024-03-18T20:44:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8dcf7fe700 1 -- 172.21.15.46:0/2180028598 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dd0010430 con 0x7f8de0100f50 2024-03-18T20:44:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8dcf7fe700 1 -- 172.21.15.46:0/2180028598 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0018680 con 0x7f8de0100f50 2024-03-18T20:44:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.912+0000 7f8de4e03700 1 -- 172.21.15.46:0/2180028598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de01a4270 con 0x7f8de0100f50 2024-03-18T20:44:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.915+0000 7f8dcf7fe700 1 -- 172.21.15.46:0/2180028598 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8dd0010860 con 0x7f8de0100f50 2024-03-18T20:44:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.915+0000 7f8dcf7fe700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8dc80a1290 0x7f8dc80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.916+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8dc80a1290 0x7f8dc80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.916+0000 7f8dcf7fe700 1 -- 172.21.15.46:0/2180028598 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8dd0014070 con 0x7f8de0100f50 2024-03-18T20:44:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.916+0000 7f8df0f98b80 1 Processor -- start 2024-03-18T20:44:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.916+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8dc80a1290 0x7f8dc80a3750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8de0198700 tx=0x7f8dd4005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:47.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.916+0000 7f8df0f98b80 1 -- start start 2024-03-18T20:44:47.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8df0f98b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba2a60 0x56367cba2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8df0f98b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba3380 0x56367cbadce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8df0f98b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cbae220 0x56367cbb0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8df0f98b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367cafc990 con 0x56367cba3380 2024-03-18T20:44:47.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8df0f98b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367cafa350 con 0x56367cbae220 2024-03-18T20:44:47.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8df0f98b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367caf5c40 con 0x56367cba2a60 2024-03-18T20:44:47.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8dddd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba3380 0x56367cbadce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8dddd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba3380 0x56367cbadce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53888/0 (socket says 172.21.15.46:53888) 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8dddd9b700 1 -- 172.21.15.46:0/3803967406 learned_addr learned my addr 172.21.15.46:0/3803967406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8dded9d700 1 --2- 172.21.15.46:0/3803967406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cbae220 0x56367cbb0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8dde59c700 1 --2- 172.21.15.46:0/3803967406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba2a60 0x56367cba2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8dddd9b700 1 -- 172.21.15.46:0/3803967406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba2a60 msgr2=0x56367cba2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/3803967406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba2a60 0x56367cba2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8dddd9b700 1 -- 172.21.15.46:0/3803967406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cbae220 msgr2=0x56367cbb0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/3803967406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cbae220 0x56367cbb0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8dddd9b700 1 -- 172.21.15.46:0/3803967406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56367c9773b0 con 0x56367cba3380 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8dded9d700 1 --2- 172.21.15.46:0/3803967406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cbae220 0x56367cbb0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.917+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/3803967406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba3380 0x56367cbadce0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4005680 tx=0x7f8dd4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e0fcee1278697 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.918+0000 7f8dccff9700 1 -- 172.21.15.46:0/3803967406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd402b070 con 0x56367cba3380 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.918+0000 7f8dccff9700 1 -- 172.21.15.46:0/3803967406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dd401e040 con 0x56367cba3380 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.918+0000 7f8dccff9700 1 -- 172.21.15.46:0/3803967406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4024d60 con 0x56367cba3380 2024-03-18T20:44:47.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.918+0000 7f8df0f98b80 1 -- 172.21.15.46:0/3803967406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba3380 msgr2=0x56367cbadce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.918+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/3803967406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba3380 0x56367cbadce0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4005680 tx=0x7f8dd4020000 comp rx=0 tx=0).stop 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 -- 172.21.15.46:0/3803967406 shutdown_connections 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/3803967406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cbae220 0x56367cbb0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/3803967406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba2a60 0x56367cba2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/3803967406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba3380 0x56367cbadce0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 -- 172.21.15.46:0/3803967406 >> 172.21.15.46:0/3803967406 conn(0x56367cb19220 msgr2=0x56367cb9f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 -- 172.21.15.46:0/3803967406 shutdown_connections 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 -- 172.21.15.46:0/3803967406 wait complete. 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 -- start start 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 Processor -- start 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba2a60 0x56367cbadcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 0x56367cbab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cbae220 0x56367cbab730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367ca91d80 con 0x56367cba2a60 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367ca91a20 con 0x56367cba3380 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367c9796a0 con 0x56367cbae220 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.919+0000 7f8df0f98b80 10 client.0 _ll_register_callbacks cb 0x56367c8fd3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dde59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba2a60 0x56367cbadcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dddd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 0x56367cbab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dde59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba2a60 0x56367cbadcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53890/0 (socket says 172.21.15.46:53890) 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dded9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cbae220 0x56367cbab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dde59c700 1 -- 172.21.15.46:0/827193947 learned_addr learned my addr 172.21.15.46:0/827193947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dddd9b700 1 -- 172.21.15.46:0/827193947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cbae220 msgr2=0x56367cbab730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cbae220 0x56367cbab730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dddd9b700 1 -- 172.21.15.46:0/827193947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba2a60 msgr2=0x56367cbadcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba2a60 0x56367cbadcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dddd9b700 1 -- 172.21.15.46:0/827193947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd8003a60 con 0x56367cba3380 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dded9d700 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cbae220 0x56367cbab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dde59c700 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba2a60 0x56367cbadcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 0x56367cbab040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4031040 tx=0x7f8dd402f6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ea5be35ac410be3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dc5ffb700 1 -- 172.21.15.46:0/827193947 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd402b040 con 0x56367cba3380 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8dc5ffb700 1 -- 172.21.15.46:0/827193947 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8dd402fcd0 con 0x56367cba3380 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.920+0000 7f8df0f98b80 1 -- 172.21.15.46:0/827193947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56367cb96ad0 con 0x56367cba3380 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.921+0000 7f8dc5ffb700 1 -- 172.21.15.46:0/827193947 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dd4040430 con 0x56367cba3380 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.921+0000 7f8df0f98b80 10 client.28304 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.921+0000 7f8df0f98b80 1 -- 172.21.15.46:0/827193947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f8de010e460 con 0x56367cba3380 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.921+0000 7f8dc5ffb700 1 -- 172.21.15.46:0/827193947 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dd4044020 con 0x56367cba3380 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.921+0000 7f8dc5ffb700 1 -- 172.21.15.46:0/827193947 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8dd4037400 con 0x56367cba3380 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.921+0000 7f8df0f98b80 10 client.28304 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.921+0000 7f8df0f98b80 10 client.28304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.921+0000 7f8df0f98b80 20 client.28304 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.921+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56367cbbd890 0x56367cbbfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.921+0000 7f8df0f98b80 4 client.28304 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.921+0000 7f8df0f98b80 1 -- 172.21.15.46:0/827193947 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56367caee700 con 0x56367cbbd890 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.921+0000 7f8dde59c700 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56367cbbd890 0x56367cbbfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.922+0000 7f8dc5ffb700 10 client.28304 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.922+0000 7f8dde59c700 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56367cbbd890 0x56367cbbfc80 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=749135930065754e server_cookie=5bdc8d04e9ebb745 in_seq=0 out_seq=0 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8dc5ffb700 1 -- 172.21.15.46:0/827193947 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56367caee700 con 0x56367cbbd890 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8dc5ffb700 10 client.28304 handle_command_reply: tid=0 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8df0f98b80 1 client.28304 shutdown 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8df0f98b80 1 -- 172.21.15.46:0/827193947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 msgr2=0x56367cbab040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 0x56367cbab040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4031040 tx=0x7f8dd402f6d0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8df0f98b80 1 -- 172.21.15.46:0/827193947 shutdown_connections 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56367cbbd890 0x56367cbbfc80 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cba2a60 0x56367cbadcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cbae220 0x56367cbab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8dc5ffb700 0 client.28304 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:47.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/827193947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 0x56367cbab040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8df0f98b80 1 -- 172.21.15.46:0/827193947 >> 172.21.15.46:0/827193947 conn(0x56367cb19220 msgr2=0x56367cb9ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:47.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8df0f98b80 1 -- 172.21.15.46:0/827193947 shutdown_connections 2024-03-18T20:44:47.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8df0f98b80 1 -- 172.21.15.46:0/827193947 wait complete. 2024-03-18T20:44:47.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.923+0000 7f8df0f98b80 20 client.28304 trim_cache size 0 max 16384 2024-03-18T20:44:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8df0f98b80 1 Processor -- start 2024-03-18T20:44:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8df0f98b80 1 -- start start 2024-03-18T20:44:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8df0f98b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba2a60 0x56367cbaa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8df0f98b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba3380 0x56367cbb4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8df0f98b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cbae220 0x56367cba6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8df0f98b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367cbb52d0 con 0x56367cbae220 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8df0f98b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367cba6e60 con 0x56367cba2a60 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8df0f98b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367cb96610 con 0x56367cba3380 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8dded9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cbae220 0x56367cba6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8dde59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba2a60 0x56367cbaa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8dde59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba2a60 0x56367cbaa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:50008/0 (socket says 172.21.15.46:50008) 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8dded9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cbae220 0x56367cba6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53906/0 (socket says 172.21.15.46:53906) 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8dde59c700 1 -- 172.21.15.46:0/611309184 learned_addr learned my addr 172.21.15.46:0/611309184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.942+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/611309184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba3380 0x56367cbb4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.943+0000 7f8dde59c700 1 -- 172.21.15.46:0/611309184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba3380 msgr2=0x56367cbb4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.943+0000 7f8dde59c700 1 --2- 172.21.15.46:0/611309184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba3380 0x56367cbb4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.943+0000 7f8dde59c700 1 -- 172.21.15.46:0/611309184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cbae220 msgr2=0x56367cba6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.943+0000 7f8dde59c700 1 --2- 172.21.15.46:0/611309184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cbae220 0x56367cba6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.943+0000 7f8dde59c700 1 -- 172.21.15.46:0/611309184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56367c923b60 con 0x56367cba2a60 2024-03-18T20:44:47.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.943+0000 7f8dde59c700 1 --2- 172.21.15.46:0/611309184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba2a60 0x56367cbaa900 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009fd0 tx=0x7f8dd001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53bdd1bdd734376b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:47.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.943+0000 7f8dcd7fa700 1 -- 172.21.15.46:0/611309184 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0099f00 con 0x56367cba2a60 2024-03-18T20:44:47.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.943+0000 7f8dcd7fa700 1 -- 172.21.15.46:0/611309184 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dd009dd30 con 0x56367cba2a60 2024-03-18T20:44:47.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.943+0000 7f8dcd7fa700 1 -- 172.21.15.46:0/611309184 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0099650 con 0x56367cba2a60 2024-03-18T20:44:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.944+0000 7f8df0f98b80 1 -- 172.21.15.46:0/611309184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba2a60 msgr2=0x56367cbaa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.944+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/611309184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba2a60 0x56367cbaa900 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009fd0 tx=0x7f8dd001a040 comp rx=0 tx=0).stop 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.944+0000 7f8df0f98b80 1 -- 172.21.15.46:0/611309184 shutdown_connections 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.944+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/611309184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cbae220 0x56367cba6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.944+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/611309184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba2a60 0x56367cbaa900 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.944+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/611309184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba3380 0x56367cbb4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.944+0000 7f8df0f98b80 1 -- 172.21.15.46:0/611309184 >> 172.21.15.46:0/611309184 conn(0x56367cb19220 msgr2=0x56367cbad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.944+0000 7f8df0f98b80 1 -- 172.21.15.46:0/611309184 shutdown_connections 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.944+0000 7f8df0f98b80 1 -- 172.21.15.46:0/611309184 wait complete. 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.944+0000 7f8df0f98b80 1 -- start start 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.944+0000 7f8df0f98b80 1 Processor -- start 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8df0f98b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba2a60 0x56367cbb4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8df0f98b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 0x56367cbb2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8df0f98b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cbae220 0x56367cbb3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8df0f98b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367cbb5170 con 0x56367cbae220 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8df0f98b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367cbb37b0 con 0x56367cba3380 2024-03-18T20:44:47.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8df0f98b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367cbb3970 con 0x56367cba2a60 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8df0f98b80 10 client.0 _ll_register_callbacks cb 0x7f8dd8009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8dddd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 0x56367cbb2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8dded9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cbae220 0x56367cbb3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8dddd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 0x56367cbb2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:50016/0 (socket says 172.21.15.46:50016) 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8dddd9b700 1 -- 172.21.15.46:0/1796788974 learned_addr learned my addr 172.21.15.46:0/1796788974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8dde59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba2a60 0x56367cbb4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8dddd9b700 1 -- 172.21.15.46:0/1796788974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba2a60 msgr2=0x56367cbb4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/1796788974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba2a60 0x56367cbb4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8dddd9b700 1 -- 172.21.15.46:0/1796788974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cbae220 msgr2=0x56367cbb3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/1796788974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cbae220 0x56367cbb3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8dddd9b700 1 -- 172.21.15.46:0/1796788974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de01a4270 con 0x56367cba3380 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8dddd9b700 1 --2- 172.21.15.46:0/1796788974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 0x56367cbb2d30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009ce0 tx=0x7f8dd4037630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80a69c92f24a895a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.945+0000 7f8dc5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:44:47.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8dc5ffb700 1 -- 172.21.15.46:0/1796788974 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd402b040 con 0x56367cba3380 2024-03-18T20:44:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8dc5ffb700 1 -- 172.21.15.46:0/1796788974 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8dd4045080 con 0x56367cba3380 2024-03-18T20:44:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8df0f98b80 1 -- 172.21.15.46:0/1796788974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8dd401e040 con 0x56367cba3380 2024-03-18T20:44:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8dc5ffb700 1 -- 172.21.15.46:0/1796788974 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dd402f740 con 0x56367cba3380 2024-03-18T20:44:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8df0f98b80 10 client.28310 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8df0f98b80 1 -- 172.21.15.46:0/1796788974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x56367cabbea0 con 0x56367cba3380 2024-03-18T20:44:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8dc5ffb700 1 -- 172.21.15.46:0/1796788974 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dd401e040 con 0x56367cba3380 2024-03-18T20:44:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8dc5ffb700 1 -- 172.21.15.46:0/1796788974 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8dd4004b80 con 0x56367cba3380 2024-03-18T20:44:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8df0f98b80 10 client.28310 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8df0f98b80 10 client.28310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:47.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8df0f98b80 20 client.28310 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/1796788974 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56367cc169b0 0x56367cc18da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8df0f98b80 4 client.28310 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:44:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8df0f98b80 1 -- 172.21.15.46:0/1796788974 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56367c944cf0 con 0x56367cc169b0 2024-03-18T20:44:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.946+0000 7f8dde59c700 1 --2- 172.21.15.46:0/1796788974 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56367cc169b0 0x56367cc18da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8dde59c700 1 --2- 172.21.15.46:0/1796788974 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56367cc169b0 0x56367cc18da0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4f3d8d3b6898b0 server_cookie=421e7a103546c30b in_seq=0 out_seq=0 2024-03-18T20:44:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8dc5ffb700 10 client.28310 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8dc5ffb700 1 -- 172.21.15.46:0/1796788974 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56367c944cf0 con 0x56367cc169b0 2024-03-18T20:44:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8dc5ffb700 10 client.28310 handle_command_reply: tid=0 2024-03-18T20:44:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8df0f98b80 1 client.28310 shutdown 2024-03-18T20:44:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8df0f98b80 1 -- 172.21.15.46:0/1796788974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 msgr2=0x56367cbb2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/1796788974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 0x56367cbb2d30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009ce0 tx=0x7f8dd4037630 comp rx=0 tx=0).stop 2024-03-18T20:44:47.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8df0f98b80 1 -- 172.21.15.46:0/1796788974 shutdown_connections 2024-03-18T20:44:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/1796788974 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56367cc169b0 0x56367cc18da0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/1796788974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56367cbae220 0x56367cbb3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8dc5ffb700 0 client.28310 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/1796788974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56367cba2a60 0x56367cbb4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8df0f98b80 1 --2- 172.21.15.46:0/1796788974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56367cba3380 0x56367cbb2d30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8df0f98b80 1 -- 172.21.15.46:0/1796788974 >> 172.21.15.46:0/1796788974 conn(0x56367cb19220 msgr2=0x56367cb9eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8df0f98b80 1 -- 172.21.15.46:0/1796788974 shutdown_connections 2024-03-18T20:44:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8df0f98b80 1 -- 172.21.15.46:0/1796788974 wait complete. 2024-03-18T20:44:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.947+0000 7f8df0f98b80 20 client.28310 trim_cache size 0 max 16384 2024-03-18T20:44:47.957 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:44:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:44:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:44:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "9e5fd927-c63e-42cf-a274-6b47897bcc4d": { 2024-03-18T20:44:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:44:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "9e5fd927-c63e-42cf-a274-6b47897bcc4d", 2024-03-18T20:44:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:44:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:44:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:44:47.957 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:44:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.948+0000 7f8de4e03700 1 -- 172.21.15.46:0/2180028598 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8dc80a1290 msgr2=0x7f8dc80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.948+0000 7f8de4e03700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8dc80a1290 0x7f8dc80a3750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8de0198700 tx=0x7f8dd4005960 comp rx=0 tx=0).stop 2024-03-18T20:44:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.948+0000 7f8de4e03700 1 -- 172.21.15.46:0/2180028598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8de0100f50 msgr2=0x7f8de0197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.948+0000 7f8de4e03700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8de0100f50 0x7f8de0197200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000e9f0 tx=0x7f8dd000ed00 comp rx=0 tx=0).stop 2024-03-18T20:44:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.948+0000 7f8de4e03700 1 -- 172.21.15.46:0/2180028598 shutdown_connections 2024-03-18T20:44:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.948+0000 7f8de4e03700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8dc80a1290 0x7f8dc80a3750 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.948+0000 7f8de4e03700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8de019ba90 0x7f8de019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.948+0000 7f8de4e03700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8de0100f50 0x7f8de0197200 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.948+0000 7f8de4e03700 1 --2- 172.21.15.46:0/2180028598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8de0106070 0x7f8de0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.948+0000 7f8de4e03700 1 -- 172.21.15.46:0/2180028598 >> 172.21.15.46:0/2180028598 conn(0x7f8de00fc7f0 msgr2=0x7f8de00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.948+0000 7f8de4e03700 1 -- 172.21.15.46:0/2180028598 shutdown_connections 2024-03-18T20:44:47.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:44:47.948+0000 7f8de4e03700 1 -- 172.21.15.46:0/2180028598 wait complete. 2024-03-18T20:44:47.965 INFO:tasks.cephfs.filesystem:scrub status for tag:9e5fd927-c63e-42cf-a274-6b47897bcc4d - {'path': '/', 'tag': '9e5fd927-c63e-42cf-a274-6b47897bcc4d', 'options': 'recursive,force'} 2024-03-18T20:44:48.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:48 smithi073 ceph-mon[47384]: pgmap v268: 97 pgs: 97 active+clean; 119 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-03-18T20:44:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:48 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:44:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:48 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:48 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:48 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:48 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:48 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:48 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:48 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:48 smithi155 ceph-mon[48344]: pgmap v268: 97 pgs: 97 active+clean; 119 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-03-18T20:44:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:48 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:44:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:48 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:48 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:48 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:48 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:48 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:48 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:48.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:48 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:48 smithi046 ceph-mon[43495]: pgmap v268: 97 pgs: 97 active+clean; 119 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-03-18T20:44:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:48 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:44:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:48 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:48 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:48 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:48 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:48 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:48 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:48 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:48.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 312379 3.62 MB/sec execute 343 sec latency 387.471 ms 2024-03-18T20:44:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:49 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:49 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:49 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:49 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:49 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:49 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:44:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:49 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:49 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:49 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:49 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:49 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:49 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:49 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:49 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:44:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:49 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:49.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:49 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:49.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 313565 3.63 MB/sec execute 344 sec latency 11.870 ms 2024-03-18T20:44:49.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:49 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:49.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:49 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:49.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:49 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:49.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:49 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:49.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:49 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:49.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:49 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:44:49.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:49 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:49.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:49 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:44:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:50 smithi155 ceph-mon[48344]: pgmap v269: 97 pgs: 97 active+clean; 120 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-03-18T20:44:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:50 smithi046 ceph-mon[43495]: pgmap v269: 97 pgs: 97 active+clean; 120 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-03-18T20:44:50.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 314580 3.62 MB/sec execute 345 sec latency 293.826 ms 2024-03-18T20:44:50.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:50 smithi073 ceph-mon[47384]: pgmap v269: 97 pgs: 97 active+clean; 120 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-03-18T20:44:51.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 316127 3.65 MB/sec execute 346 sec latency 17.863 ms 2024-03-18T20:44:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:52 smithi155 ceph-mon[48344]: pgmap v270: 97 pgs: 97 active+clean; 120 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-03-18T20:44:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:52 smithi046 ceph-mon[43495]: pgmap v270: 97 pgs: 97 active+clean; 120 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-03-18T20:44:52.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 317396 3.65 MB/sec execute 347 sec latency 175.927 ms 2024-03-18T20:44:52.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:52 smithi073 ceph-mon[47384]: pgmap v270: 97 pgs: 97 active+clean; 120 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-03-18T20:44:53.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 317396 3.64 MB/sec execute 348 sec latency 1176.012 ms 2024-03-18T20:44:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:54 smithi155 ceph-mon[48344]: pgmap v271: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-03-18T20:44:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:54 smithi046 ceph-mon[43495]: pgmap v271: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-03-18T20:44:54.706 INFO:tasks.workunit.client.0.smithi046.stdout: 1 317396 3.63 MB/sec execute 349 sec latency 2176.078 ms 2024-03-18T20:44:54.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:54 smithi073 ceph-mon[47384]: pgmap v271: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-03-18T20:44:55.707 INFO:tasks.workunit.client.0.smithi046.stdout: 1 319365 3.66 MB/sec execute 350 sec latency 2324.613 ms 2024-03-18T20:44:56.219 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:55 smithi073 ceph-mon[47384]: pgmap v272: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-03-18T20:44:56.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:55 smithi155 ceph-mon[48344]: pgmap v272: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-03-18T20:44:56.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:55 smithi046 ceph-mon[43495]: pgmap v272: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-03-18T20:44:56.707 INFO:tasks.workunit.client.0.smithi046.stdout: 1 320497 3.66 MB/sec execute 351 sec latency 28.001 ms 2024-03-18T20:44:57.707 INFO:tasks.workunit.client.0.smithi046.stdout: 1 320938 3.65 MB/sec execute 352 sec latency 790.325 ms 2024-03-18T20:44:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.978+0000 7f4916143700 1 -- 172.21.15.46:0/941330084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49100ff220 msgr2=0x7f49100ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.978+0000 7f4916143700 1 --2- 172.21.15.46:0/941330084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49100ff220 0x7f49100ff6a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f48f80099c0 tx=0x7f48f8009cd0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.978+0000 7f4916143700 1 -- 172.21.15.46:0/941330084 shutdown_connections 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.978+0000 7f4916143700 1 --2- 172.21.15.46:0/941330084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49100ffbe0 0x7f491010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.978+0000 7f4916143700 1 --2- 172.21.15.46:0/941330084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4910106e30 0x7f4910107210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.978+0000 7f4916143700 1 --2- 172.21.15.46:0/941330084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49100ff220 0x7f49100ff6a0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.978+0000 7f4916143700 1 -- 172.21.15.46:0/941330084 >> 172.21.15.46:0/941330084 conn(0x7f49100761b0 msgr2=0x7f49100fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.978+0000 7f4916143700 1 -- 172.21.15.46:0/941330084 shutdown_connections 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.978+0000 7f4916143700 1 -- 172.21.15.46:0/941330084 wait complete. 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.979+0000 7f4916143700 1 Processor -- start 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.979+0000 7f4916143700 1 -- start start 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.979+0000 7f4916143700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49100ff220 0x7f4910197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.979+0000 7f4916143700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49100ffbe0 0x7f4910197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.979+0000 7f4916143700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4910106e30 0x7f491019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.979+0000 7f4916143700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491019c110 con 0x7f49100ff220 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.979+0000 7f4916143700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491019c280 con 0x7f49100ffbe0 2024-03-18T20:44:57.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.979+0000 7f4916143700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491019c440 con 0x7f4910106e30 2024-03-18T20:44:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.979+0000 7f490f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49100ff220 0x7f4910197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.979+0000 7f490f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49100ff220 0x7f4910197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43820/0 (socket says 172.21.15.46:43820) 2024-03-18T20:44:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.979+0000 7f490f7fe700 1 -- 172.21.15.46:0/1314796424 learned_addr learned my addr 172.21.15.46:0/1314796424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f490ffff700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4910106e30 0x7f491019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f490effd700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49100ffbe0 0x7f4910197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f490f7fe700 1 -- 172.21.15.46:0/1314796424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4910106e30 msgr2=0x7f491019bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f490f7fe700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4910106e30 0x7f491019bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f490f7fe700 1 -- 172.21.15.46:0/1314796424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49100ffbe0 msgr2=0x7f4910197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f490f7fe700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49100ffbe0 0x7f4910197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f490f7fe700 1 -- 172.21.15.46:0/1314796424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f80095f0 con 0x7f49100ff220 2024-03-18T20:44:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f490f7fe700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49100ff220 0x7f4910197240 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f490000e9a0 tx=0x7f490000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:57.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f490cff9700 1 -- 172.21.15.46:0/1314796424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900009790 con 0x7f49100ff220 2024-03-18T20:44:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f490cff9700 1 -- 172.21.15.46:0/1314796424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f490000cbd0 con 0x7f49100ff220 2024-03-18T20:44:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f4916143700 1 -- 172.21.15.46:0/1314796424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491019c720 con 0x7f49100ff220 2024-03-18T20:44:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f490cff9700 1 -- 172.21.15.46:0/1314796424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900018680 con 0x7f49100ff220 2024-03-18T20:44:57.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.980+0000 7f4916143700 1 -- 172.21.15.46:0/1314796424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49101a3f60 con 0x7f49100ff220 2024-03-18T20:44:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.982+0000 7f490cff9700 1 -- 172.21.15.46:0/1314796424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f490000cd40 con 0x7f49100ff220 2024-03-18T20:44:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.983+0000 7f490cff9700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48fc0a1310 0x7f48fc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.983+0000 7f490cff9700 1 -- 172.21.15.46:0/1314796424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f4900014070 con 0x7f49100ff220 2024-03-18T20:44:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.983+0000 7f490effd700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48fc0a1310 0x7f48fc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:57.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f490effd700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48fc0a1310 0x7f48fc0a37d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009fd0 tx=0x7f48f8005a90 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f49222d8b80 1 Processor -- start 2024-03-18T20:44:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f49222d8b80 1 -- start start 2024-03-18T20:44:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f49222d8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b07950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f49222d8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b07e90 0x562af5b12870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f49222d8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b12db0 0x562af5b151a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f49222d8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5a61930 con 0x562af5b12db0 2024-03-18T20:44:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f49222d8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5a5f2f0 con 0x562af5b07570 2024-03-18T20:44:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f49222d8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5a5abe0 con 0x562af5b07e90 2024-03-18T20:44:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f490ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b12db0 0x562af5b151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f490effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b07e90 0x562af5b12870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:57.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f490ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b12db0 0x562af5b151a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43828/0 (socket says 172.21.15.46:43828) 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f490ffff700 1 -- 172.21.15.46:0/1761125127 learned_addr learned my addr 172.21.15.46:0/1761125127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.984+0000 7f490f7fe700 1 --2- 172.21.15.46:0/1761125127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b07950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.985+0000 7f490f7fe700 1 -- 172.21.15.46:0/1761125127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b07e90 msgr2=0x562af5b12870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.985+0000 7f490f7fe700 1 --2- 172.21.15.46:0/1761125127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b07e90 0x562af5b12870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.985+0000 7f490f7fe700 1 -- 172.21.15.46:0/1761125127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b12db0 msgr2=0x562af5b151a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.985+0000 7f490f7fe700 1 --2- 172.21.15.46:0/1761125127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b12db0 0x562af5b151a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.985+0000 7f490f7fe700 1 -- 172.21.15.46:0/1761125127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562af58dc350 con 0x562af5b07570 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.985+0000 7f490f7fe700 1 --2- 172.21.15.46:0/1761125127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b07950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f49000ca670 tx=0x7f490009a7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0997ab7562cc601 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.985+0000 7f4905ffb700 1 -- 172.21.15.46:0/1761125127 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49000a24b0 con 0x562af5b07570 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.985+0000 7f4905ffb700 1 -- 172.21.15.46:0/1761125127 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f490009a290 con 0x562af5b07570 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.985+0000 7f4905ffb700 1 -- 172.21.15.46:0/1761125127 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49000a2280 con 0x562af5b07570 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.986+0000 7f49222d8b80 1 -- 172.21.15.46:0/1761125127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 msgr2=0x562af5b07950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.986+0000 7f49222d8b80 1 --2- 172.21.15.46:0/1761125127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b07950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f49000ca670 tx=0x7f490009a7b0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.986+0000 7f49222d8b80 1 -- 172.21.15.46:0/1761125127 shutdown_connections 2024-03-18T20:44:57.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.986+0000 7f49222d8b80 1 --2- 172.21.15.46:0/1761125127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b12db0 0x562af5b151a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.986+0000 7f49222d8b80 1 --2- 172.21.15.46:0/1761125127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b07950 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.986+0000 7f49222d8b80 1 --2- 172.21.15.46:0/1761125127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b07e90 0x562af5b12870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.986+0000 7f49222d8b80 1 -- 172.21.15.46:0/1761125127 >> 172.21.15.46:0/1761125127 conn(0x562af5a7e1c0 msgr2=0x562af5b03a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.986+0000 7f49222d8b80 1 -- 172.21.15.46:0/1761125127 shutdown_connections 2024-03-18T20:44:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.986+0000 7f49222d8b80 1 -- 172.21.15.46:0/1761125127 wait complete. 2024-03-18T20:44:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.986+0000 7f49222d8b80 1 -- start start 2024-03-18T20:44:57.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f49222d8b80 1 Processor -- start 2024-03-18T20:44:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f49222d8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b12820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f49222d8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 0x562af5b0fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f49222d8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b10200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f49222d8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af59f6d20 con 0x562af5b07e90 2024-03-18T20:44:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f49222d8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af59f69c0 con 0x562af5b07570 2024-03-18T20:44:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f49222d8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af58de640 con 0x562af5b12db0 2024-03-18T20:44:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f490effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 0x562af5b0fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f49222d8b80 10 client.0 _ll_register_callbacks cb 0x562af5862340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f490effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 0x562af5b0fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43832/0 (socket says 172.21.15.46:43832) 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f490effd700 1 -- 172.21.15.46:0/3749050863 learned_addr learned my addr 172.21.15.46:0/3749050863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f490f7fe700 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b12820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f490ffff700 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b10200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f490effd700 1 -- 172.21.15.46:0/3749050863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 msgr2=0x562af5b10200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f490effd700 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b10200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f490effd700 1 -- 172.21.15.46:0/3749050863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 msgr2=0x562af5b12820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f490effd700 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b12820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.987+0000 7f490effd700 1 -- 172.21.15.46:0/3749050863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49101a3f60 con 0x562af5b07e90 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f490effd700 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 0x562af5b0fb10 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f48f80099c0 tx=0x7f48f8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f4d90159d5e9dd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f48eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f48eeffd700 1 -- 172.21.15.46:0/3749050863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f80039b0 con 0x562af5b07e90 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f48eeffd700 1 -- 172.21.15.46:0/3749050863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f48f8003b10 con 0x562af5b07e90 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f49222d8b80 1 -- 172.21.15.46:0/3749050863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562af5afb6c0 con 0x562af5b07e90 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f48eeffd700 1 -- 172.21.15.46:0/3749050863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48f8003cd0 con 0x562af5b07e90 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f49222d8b80 10 client.18675 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f49222d8b80 1 -- 172.21.15.46:0/3749050863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x562af5888b00 con 0x562af5b07e90 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f48eeffd700 1 -- 172.21.15.46:0/3749050863 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562af5afb6c0 con 0x562af5b07e90 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f48eeffd700 1 -- 172.21.15.46:0/3749050863 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f48f802e070 con 0x562af5b07e90 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f49222d8b80 10 client.18675 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f49222d8b80 10 client.18675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f49222d8b80 20 client.18675 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f49222d8b80 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562af5b22a60 0x562af5b24e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f49222d8b80 4 client.18675 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.988+0000 7f49222d8b80 1 -- 172.21.15.46:0/3749050863 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562af5a536a0 con 0x562af5b22a60 2024-03-18T20:44:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.989+0000 7f490f7fe700 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562af5b22a60 0x562af5b24e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.989+0000 7f48eeffd700 10 client.18675 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.989+0000 7f490f7fe700 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562af5b22a60 0x562af5b24e80 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=968426d6221521cb server_cookie=baad4da5a4d730a9 in_seq=0 out_seq=0 2024-03-18T20:44:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f48eeffd700 1 -- 172.21.15.46:0/3749050863 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562af5a536a0 con 0x562af5b22a60 2024-03-18T20:44:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f48eeffd700 10 client.18675 handle_command_reply: tid=0 2024-03-18T20:44:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f49222d8b80 1 client.18675 shutdown 2024-03-18T20:44:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f49222d8b80 1 -- 172.21.15.46:0/3749050863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 msgr2=0x562af5b0fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f49222d8b80 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 0x562af5b0fb10 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f48f80099c0 tx=0x7f48f8009cd0 comp rx=0 tx=0).stop 2024-03-18T20:44:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f49222d8b80 1 -- 172.21.15.46:0/3749050863 shutdown_connections 2024-03-18T20:44:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f49222d8b80 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562af5b22a60 0x562af5b24e80 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f49222d8b80 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b10200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f48eeffd700 0 client.18675 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f49222d8b80 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b12820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f49222d8b80 1 --2- 172.21.15.46:0/3749050863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 0x562af5b0fb10 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f49222d8b80 1 -- 172.21.15.46:0/3749050863 >> 172.21.15.46:0/3749050863 conn(0x562af5a7e1c0 msgr2=0x562af5b035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f49222d8b80 1 -- 172.21.15.46:0/3749050863 shutdown_connections 2024-03-18T20:44:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f49222d8b80 1 -- 172.21.15.46:0/3749050863 wait complete. 2024-03-18T20:44:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:57.990+0000 7f49222d8b80 20 client.18675 trim_cache size 0 max 16384 2024-03-18T20:44:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f49222d8b80 1 Processor -- start 2024-03-18T20:44:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f49222d8b80 1 -- start start 2024-03-18T20:44:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f49222d8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b19a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f49222d8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 0x562af5b0e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f49222d8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b0e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f49222d8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5b047d0 con 0x562af5b07e90 2024-03-18T20:44:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f49222d8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5b157d0 con 0x562af5b07570 2024-03-18T20:44:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f49222d8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5b0eeb0 con 0x562af5b12db0 2024-03-18T20:44:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f490effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 0x562af5b0e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f490effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 0x562af5b0e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43848/0 (socket says 172.21.15.46:43848) 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f490effd700 1 -- 172.21.15.46:0/371508711 learned_addr learned my addr 172.21.15.46:0/371508711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f490f7fe700 1 --2- 172.21.15.46:0/371508711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b19a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f490ffff700 1 --2- 172.21.15.46:0/371508711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b0e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f490effd700 1 -- 172.21.15.46:0/371508711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 msgr2=0x562af5b0e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f490effd700 1 --2- 172.21.15.46:0/371508711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b0e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f490effd700 1 -- 172.21.15.46:0/371508711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 msgr2=0x562af5b19a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f490effd700 1 --2- 172.21.15.46:0/371508711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b19a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f490effd700 1 -- 172.21.15.46:0/371508711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562af5a20e40 con 0x562af5b07e90 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f490ffff700 1 --2- 172.21.15.46:0/371508711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b0e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.006+0000 7f490effd700 1 --2- 172.21.15.46:0/371508711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 0x562af5b0e400 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f48f8005850 tx=0x7f48f800b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae10d00eeb7afb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.007+0000 7f49067fc700 1 -- 172.21.15.46:0/371508711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f80229c0 con 0x562af5b07e90 2024-03-18T20:44:58.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.007+0000 7f49067fc700 1 -- 172.21.15.46:0/371508711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f8027070 con 0x562af5b07e90 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.007+0000 7f49067fc700 1 -- 172.21.15.46:0/371508711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8041760 con 0x562af5b07e90 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.007+0000 7f49222d8b80 1 -- 172.21.15.46:0/371508711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 msgr2=0x562af5b0e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.007+0000 7f49222d8b80 1 --2- 172.21.15.46:0/371508711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 0x562af5b0e400 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f48f8005850 tx=0x7f48f800b5c0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.007+0000 7f49222d8b80 1 -- 172.21.15.46:0/371508711 shutdown_connections 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.007+0000 7f49222d8b80 1 --2- 172.21.15.46:0/371508711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b0e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.007+0000 7f49222d8b80 1 --2- 172.21.15.46:0/371508711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07570 0x562af5b19a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.007+0000 7f49222d8b80 1 --2- 172.21.15.46:0/371508711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07e90 0x562af5b0e400 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.007+0000 7f49222d8b80 1 -- 172.21.15.46:0/371508711 >> 172.21.15.46:0/371508711 conn(0x562af5a7e1c0 msgr2=0x562af5b0ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.007+0000 7f49222d8b80 1 -- 172.21.15.46:0/371508711 shutdown_connections 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.007+0000 7f49222d8b80 1 -- 172.21.15.46:0/371508711 wait complete. 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f49222d8b80 1 -- start start 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f49222d8b80 1 Processor -- start 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f49222d8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07570 0x562af5b17a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f49222d8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07e90 0x562af5b17f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:58.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f49222d8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b75890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f49222d8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5b188a0 con 0x562af5b07570 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f49222d8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5b75dd0 con 0x562af5b07e90 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f49222d8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af5b75f10 con 0x562af5b12db0 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f49222d8b80 10 client.0 _ll_register_callbacks cb 0x7f48f800b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f490f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07570 0x562af5b17a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f490f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07570 0x562af5b17a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43858/0 (socket says 172.21.15.46:43858) 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f490f7fe700 1 -- 172.21.15.46:0/2065359464 learned_addr learned my addr 172.21.15.46:0/2065359464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f490ffff700 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b75890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.008+0000 7f490effd700 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07e90 0x562af5b17f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f490f7fe700 1 -- 172.21.15.46:0/2065359464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 msgr2=0x562af5b75890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f490f7fe700 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b75890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f490f7fe700 1 -- 172.21.15.46:0/2065359464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07e90 msgr2=0x562af5b17f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f490f7fe700 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07e90 0x562af5b17f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f490f7fe700 1 -- 172.21.15.46:0/2065359464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562af5888b00 con 0x562af5b07570 2024-03-18T20:44:58.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f490f7fe700 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07570 0x562af5b17a20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f49000964b0 tx=0x7f490000efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87725e25ad3ebdd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f48eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f48eeffd700 1 -- 172.21.15.46:0/2065359464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900099b70 con 0x562af5b07570 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f48eeffd700 1 -- 172.21.15.46:0/2065359464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f490009a400 con 0x562af5b07570 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f49222d8b80 1 -- 172.21.15.46:0/2065359464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f490009a290 con 0x562af5b07570 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f48eeffd700 1 -- 172.21.15.46:0/2065359464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49000b2be0 con 0x562af5b07570 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f49222d8b80 10 client.18681 fetch_fsmap learned FSMap version 47 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f49222d8b80 1 -- 172.21.15.46:0/2065359464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x562af58e0c70 con 0x562af5b07570 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f48eeffd700 1 -- 172.21.15.46:0/2065359464 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f490009a290 con 0x562af5b07570 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.009+0000 7f48eeffd700 1 -- 172.21.15.46:0/2065359464 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f49000a2020 con 0x562af5b07570 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.010+0000 7f49222d8b80 10 client.18681 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.010+0000 7f49222d8b80 10 client.18681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.010+0000 7f49222d8b80 20 client.18681 populate_metadata read hostname 'smithi046' 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.010+0000 7f49222d8b80 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562af5b770e0 0x562af5b79500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:44:58.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.010+0000 7f49222d8b80 4 client.18681 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:44:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.010+0000 7f49222d8b80 1 -- 172.21.15.46:0/2065359464 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562af58a9c90 con 0x562af5b770e0 2024-03-18T20:44:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.010+0000 7f490effd700 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562af5b770e0 0x562af5b79500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:44:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.010+0000 7f490effd700 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562af5b770e0 0x562af5b79500 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12ec86a281446789 server_cookie=bea8e72f0b22f2fe in_seq=0 out_seq=0 2024-03-18T20:44:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.010+0000 7f48eeffd700 10 client.18681 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.046+0000 7f48eeffd700 1 -- 172.21.15.46:0/2065359464 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562af58a9c90 con 0x562af5b770e0 2024-03-18T20:44:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.046+0000 7f48eeffd700 10 client.18681 handle_command_reply: tid=0 2024-03-18T20:44:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.046+0000 7f49222d8b80 1 client.18681 shutdown 2024-03-18T20:44:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.046+0000 7f49222d8b80 1 -- 172.21.15.46:0/2065359464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07570 msgr2=0x562af5b17a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.046+0000 7f49222d8b80 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07570 0x562af5b17a20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f49000964b0 tx=0x7f490000efa0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.047+0000 7f49222d8b80 1 -- 172.21.15.46:0/2065359464 shutdown_connections 2024-03-18T20:44:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.047+0000 7f49222d8b80 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562af5b770e0 0x562af5b79500 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.047+0000 7f49222d8b80 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562af5b07570 0x562af5b17a20 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.047+0000 7f49222d8b80 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562af5b12db0 0x562af5b75890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.047+0000 7f49222d8b80 1 --2- 172.21.15.46:0/2065359464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562af5b07e90 0x562af5b17f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.047+0000 7f49222d8b80 1 -- 172.21.15.46:0/2065359464 >> 172.21.15.46:0/2065359464 conn(0x562af5a7e1c0 msgr2=0x562af5b03470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.047+0000 7f48eeffd700 0 client.18681 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:44:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.047+0000 7f49222d8b80 1 -- 172.21.15.46:0/2065359464 shutdown_connections 2024-03-18T20:44:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.047+0000 7f49222d8b80 1 -- 172.21.15.46:0/2065359464 wait complete. 2024-03-18T20:44:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.047+0000 7f49222d8b80 20 client.18681 trim_cache size 0 max 16384 2024-03-18T20:44:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:44:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:44:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:44:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:44:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.048+0000 7f4916143700 1 -- 172.21.15.46:0/1314796424 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48fc0a1310 msgr2=0x7f48fc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.048+0000 7f4916143700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48fc0a1310 0x7f48fc0a37d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009fd0 tx=0x7f48f8005a90 comp rx=0 tx=0).stop 2024-03-18T20:44:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.048+0000 7f4916143700 1 -- 172.21.15.46:0/1314796424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49100ff220 msgr2=0x7f4910197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:44:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.048+0000 7f4916143700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49100ff220 0x7f4910197240 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f490000e9a0 tx=0x7f490000ed60 comp rx=0 tx=0).stop 2024-03-18T20:44:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.048+0000 7f4916143700 1 -- 172.21.15.46:0/1314796424 shutdown_connections 2024-03-18T20:44:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.048+0000 7f4916143700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48fc0a1310 0x7f48fc0a37d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.048+0000 7f4916143700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49100ff220 0x7f4910197240 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.048+0000 7f4916143700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4910106e30 0x7f491019bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.048+0000 7f4916143700 1 --2- 172.21.15.46:0/1314796424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49100ffbe0 0x7f4910197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:44:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.049+0000 7f4916143700 1 -- 172.21.15.46:0/1314796424 >> 172.21.15.46:0/1314796424 conn(0x7f49100761b0 msgr2=0x7f4910103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:44:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.049+0000 7f4916143700 1 -- 172.21.15.46:0/1314796424 shutdown_connections 2024-03-18T20:44:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:44:58.049+0000 7f4916143700 1 -- 172.21.15.46:0/1314796424 wait complete. 2024-03-18T20:44:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:58 smithi155 ceph-mon[48344]: pgmap v273: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-03-18T20:44:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:58 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:44:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:58 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:44:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:58 smithi046 ceph-mon[43495]: pgmap v273: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-03-18T20:44:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:58 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:44:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:58 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:44:58.707 INFO:tasks.workunit.client.0.smithi046.stdout: 1 320938 3.64 MB/sec execute 353 sec latency 1790.437 ms 2024-03-18T20:44:58.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:58 smithi073 ceph-mon[47384]: pgmap v273: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-03-18T20:44:58.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:58 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:44:58.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:58 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:44:59.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:59 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:44:59.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:59 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:44:59.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:44:59 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:44:59.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:59 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:44:59.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:59 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:44:59.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:44:59 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:44:59.707 INFO:tasks.workunit.client.0.smithi046.stdout: 1 320938 3.63 MB/sec execute 354 sec latency 2790.574 ms 2024-03-18T20:44:59.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:59 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:44:59.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:59 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:44:59.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:44:59 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:45:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:00 smithi155 ceph-mon[48344]: pgmap v274: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-03-18T20:45:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:00 smithi046 ceph-mon[43495]: pgmap v274: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-03-18T20:45:00.707 INFO:tasks.workunit.client.0.smithi046.stdout: 1 321829 3.63 MB/sec execute 355 sec latency 3412.116 ms 2024-03-18T20:45:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:00 smithi073 ceph-mon[47384]: pgmap v274: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-03-18T20:45:01.707 INFO:tasks.workunit.client.0.smithi046.stdout: 1 323079 3.65 MB/sec execute 356 sec latency 32.143 ms 2024-03-18T20:45:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:02 smithi155 ceph-mon[48344]: pgmap v275: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-03-18T20:45:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:02 smithi046 ceph-mon[43495]: pgmap v275: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-03-18T20:45:02.707 INFO:tasks.workunit.client.0.smithi046.stdout: 1 324389 3.66 MB/sec execute 357 sec latency 22.650 ms 2024-03-18T20:45:02.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:02 smithi073 ceph-mon[47384]: pgmap v275: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 1.5 MiB/s wr, 153 op/s 2024-03-18T20:45:03.707 INFO:tasks.workunit.client.0.smithi046.stdout: 1 324487 3.65 MB/sec execute 358 sec latency 945.892 ms 2024-03-18T20:45:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:04 smithi155 ceph-mon[48344]: pgmap v276: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-03-18T20:45:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:04 smithi155 ceph-mon[48344]: osdmap e87: 12 total, 12 up, 12 in 2024-03-18T20:45:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:04 smithi046 ceph-mon[43495]: pgmap v276: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-03-18T20:45:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:04 smithi046 ceph-mon[43495]: osdmap e87: 12 total, 12 up, 12 in 2024-03-18T20:45:04.707 INFO:tasks.workunit.client.0.smithi046.stdout: 1 324487 3.64 MB/sec execute 359 sec latency 1946.008 ms 2024-03-18T20:45:04.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:04 smithi073 ceph-mon[47384]: pgmap v276: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-03-18T20:45:04.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:04 smithi073 ceph-mon[47384]: osdmap e87: 12 total, 12 up, 12 in 2024-03-18T20:45:05.708 INFO:tasks.workunit.client.0.smithi046.stdout: 1 325385 3.63 MB/sec execute 360 sec latency 2562.635 ms 2024-03-18T20:45:06.219 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:05 smithi073 ceph-mon[47384]: pgmap v278: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-03-18T20:45:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:05 smithi155 ceph-mon[48344]: pgmap v278: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-03-18T20:45:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:05 smithi046 ceph-mon[43495]: pgmap v278: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-03-18T20:45:06.707 INFO:tasks.workunit.client.0.smithi046.stdout: 1 326872 3.65 MB/sec execute 361 sec latency 12.472 ms 2024-03-18T20:45:07.708 INFO:tasks.workunit.client.0.smithi046.stdout: 1 328029 3.66 MB/sec execute 362 sec latency 173.647 ms 2024-03-18T20:45:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:08 smithi155 ceph-mon[48344]: pgmap v279: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-03-18T20:45:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:08 smithi046 ceph-mon[43495]: pgmap v279: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-03-18T20:45:08.708 INFO:tasks.workunit.client.0.smithi046.stdout: 1 328029 3.65 MB/sec execute 363 sec latency 1173.773 ms 2024-03-18T20:45:08.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:08 smithi073 ceph-mon[47384]: pgmap v279: 97 pgs: 97 active+clean; 126 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-03-18T20:45:09.708 INFO:tasks.workunit.client.0.smithi046.stdout: 1 328029 3.64 MB/sec execute 364 sec latency 2173.880 ms 2024-03-18T20:45:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:10 smithi155 ceph-mon[48344]: pgmap v280: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-03-18T20:45:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:10 smithi155 ceph-mon[48344]: osdmap e88: 12 total, 12 up, 12 in 2024-03-18T20:45:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:10 smithi046 ceph-mon[43495]: pgmap v280: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-03-18T20:45:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:10 smithi046 ceph-mon[43495]: osdmap e88: 12 total, 12 up, 12 in 2024-03-18T20:45:10.708 INFO:tasks.workunit.client.0.smithi046.stdout: 1 329961 3.66 MB/sec execute 365 sec latency 2322.051 ms 2024-03-18T20:45:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:10 smithi073 ceph-mon[47384]: pgmap v280: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-03-18T20:45:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:10 smithi073 ceph-mon[47384]: osdmap e88: 12 total, 12 up, 12 in 2024-03-18T20:45:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.185+0000 7f696d159700 1 -- 172.21.15.46:0/226529667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69681007b0 msgr2=0x7f69681047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.185+0000 7f696d159700 1 --2- 172.21.15.46:0/226529667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69681007b0 0x7f69681047f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f6950009990 tx=0x7f6950009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.185+0000 7f696d159700 1 -- 172.21.15.46:0/226529667 shutdown_connections 2024-03-18T20:45:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.185+0000 7f696d159700 1 --2- 172.21.15.46:0/226529667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6968104f20 0x7f696810caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.185+0000 7f696d159700 1 --2- 172.21.15.46:0/226529667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69680ffe00 0x7f69681001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.185+0000 7f696d159700 1 --2- 172.21.15.46:0/226529667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69681007b0 0x7f69681047f0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.185+0000 7f696d159700 1 -- 172.21.15.46:0/226529667 >> 172.21.15.46:0/226529667 conn(0x7f69680fb680 msgr2=0x7f69680fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.186+0000 7f696d159700 1 -- 172.21.15.46:0/226529667 shutdown_connections 2024-03-18T20:45:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.186+0000 7f696d159700 1 -- 172.21.15.46:0/226529667 wait complete. 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.187+0000 7f696d159700 1 Processor -- start 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.187+0000 7f696d159700 1 -- start start 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.187+0000 7f696d159700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680ffe00 0x7f6968197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.187+0000 7f696d159700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6968104f20 0x7f6968198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.187+0000 7f696d159700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6968198a70 0x7f69681a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.187+0000 7f696d159700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69681a3750 con 0x7f69680ffe00 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.187+0000 7f696d159700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69681a38c0 con 0x7f6968104f20 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.187+0000 7f696d159700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69681a3a80 con 0x7f6968198a70 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.187+0000 7f6966d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680ffe00 0x7f6968197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.187+0000 7f6966d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680ffe00 0x7f6968197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47328/0 (socket says 172.21.15.46:47328) 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.187+0000 7f6966d9d700 1 -- 172.21.15.46:0/520210685 learned_addr learned my addr 172.21.15.46:0/520210685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.187+0000 7f696759e700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6968198a70 0x7f69681a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f695e59c700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6968104f20 0x7f6968198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f696759e700 1 -- 172.21.15.46:0/520210685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6968104f20 msgr2=0x7f6968198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f696759e700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6968104f20 0x7f6968198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f696759e700 1 -- 172.21.15.46:0/520210685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680ffe00 msgr2=0x7f6968197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f696759e700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680ffe00 0x7f6968197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f696759e700 1 -- 172.21.15.46:0/520210685 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69500095f0 con 0x7f6968198a70 2024-03-18T20:45:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f6966d9d700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680ffe00 0x7f6968197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f696759e700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6968198a70 0x7f69681a3070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f696000efc0 tx=0x7f696000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f6964d99700 1 -- 172.21.15.46:0/520210685 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960018070 con 0x7f6968198a70 2024-03-18T20:45:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f6964d99700 1 -- 172.21.15.46:0/520210685 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f696000f040 con 0x7f6968198a70 2024-03-18T20:45:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f696d159700 1 -- 172.21.15.46:0/520210685 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69681a3d00 con 0x7f6968198a70 2024-03-18T20:45:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f6964d99700 1 -- 172.21.15.46:0/520210685 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696000cd40 con 0x7f6968198a70 2024-03-18T20:45:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.188+0000 7f696d159700 1 -- 172.21.15.46:0/520210685 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69681a41c0 con 0x7f6968198a70 2024-03-18T20:45:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.191+0000 7f6964d99700 1 -- 172.21.15.46:0/520210685 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6960004000 con 0x7f6968198a70 2024-03-18T20:45:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.192+0000 7f6964d99700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f69540a1290 0x7f69540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.192+0000 7f6964d99700 1 -- 172.21.15.46:0/520210685 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f69600cc860 con 0x7f6968198a70 2024-03-18T20:45:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.192+0000 7f6966d9d700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f69540a1290 0x7f69540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f69792eeb80 1 Processor -- start 2024-03-18T20:45:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f6966d9d700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f69540a1290 0x7f69540a3750 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6958011550 tx=0x7f6958009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:11.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f69792eeb80 1 -- start start 2024-03-18T20:45:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f69792eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 0x561166042950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f69792eeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042e90 0x561166051c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f69792eeb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611660521c0 0x5611660545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f69792eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561165f9c930 con 0x561166042570 2024-03-18T20:45:11.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f69792eeb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561165f9a2f0 con 0x5611660521c0 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f69792eeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561165f95be0 con 0x561166042e90 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f6966d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 0x561166042950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f6966d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 0x561166042950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47338/0 (socket says 172.21.15.46:47338) 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f6966d9d700 1 -- 172.21.15.46:0/164462694 learned_addr learned my addr 172.21.15.46:0/164462694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.193+0000 7f695e59c700 1 --2- 172.21.15.46:0/164462694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042e90 0x561166051c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.194+0000 7f696759e700 1 --2- 172.21.15.46:0/164462694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611660521c0 0x5611660545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.194+0000 7f6966d9d700 1 -- 172.21.15.46:0/164462694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042e90 msgr2=0x561166051c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.194+0000 7f6966d9d700 1 --2- 172.21.15.46:0/164462694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042e90 0x561166051c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.194+0000 7f6966d9d700 1 -- 172.21.15.46:0/164462694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611660521c0 msgr2=0x5611660545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.194+0000 7f6966d9d700 1 --2- 172.21.15.46:0/164462694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611660521c0 0x5611660545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.194+0000 7f6966d9d700 1 -- 172.21.15.46:0/164462694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561165e17350 con 0x561166042570 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.194+0000 7f695e59c700 1 --2- 172.21.15.46:0/164462694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042e90 0x561166051c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.194+0000 7f6966d9d700 1 --2- 172.21.15.46:0/164462694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 0x561166042950 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6958009880 tx=0x7f6958014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61d41d09d26865b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.194+0000 7f695d59a700 1 -- 172.21.15.46:0/164462694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958026630 con 0x561166042570 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.194+0000 7f695d59a700 1 -- 172.21.15.46:0/164462694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6958021070 con 0x561166042570 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.194+0000 7f695d59a700 1 -- 172.21.15.46:0/164462694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695801d650 con 0x561166042570 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.195+0000 7f69792eeb80 1 -- 172.21.15.46:0/164462694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 msgr2=0x561166042950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.195+0000 7f69792eeb80 1 --2- 172.21.15.46:0/164462694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 0x561166042950 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6958009880 tx=0x7f6958014000 comp rx=0 tx=0).stop 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.195+0000 7f69792eeb80 1 -- 172.21.15.46:0/164462694 shutdown_connections 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.195+0000 7f69792eeb80 1 --2- 172.21.15.46:0/164462694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 0x561166042950 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.195+0000 7f69792eeb80 1 --2- 172.21.15.46:0/164462694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611660521c0 0x5611660545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.195+0000 7f69792eeb80 1 --2- 172.21.15.46:0/164462694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042e90 0x561166051c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.195+0000 7f69792eeb80 1 -- 172.21.15.46:0/164462694 >> 172.21.15.46:0/164462694 conn(0x561165fb91c0 msgr2=0x56116603ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.195+0000 7f69792eeb80 1 -- 172.21.15.46:0/164462694 shutdown_connections 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.195+0000 7f69792eeb80 1 -- 172.21.15.46:0/164462694 wait complete. 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f69792eeb80 1 -- start start 2024-03-18T20:45:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f69792eeb80 1 Processor -- start 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f69792eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 0x561166051c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f69792eeb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042e90 0x56116604efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f69792eeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611660521c0 0x56116604f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f69792eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561165f31d20 con 0x561166042570 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f69792eeb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561165f319c0 con 0x561166042e90 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f69792eeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561165e19640 con 0x5611660521c0 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f69792eeb80 10 client.0 _ll_register_callbacks cb 0x561165d9d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f6966d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 0x561166051c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f696759e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611660521c0 0x56116604f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f6966d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 0x561166051c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47344/0 (socket says 172.21.15.46:47344) 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f6966d9d700 1 -- 172.21.15.46:0/190020647 learned_addr learned my addr 172.21.15.46:0/190020647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.196+0000 7f695e59c700 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042e90 0x56116604efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f696759e700 1 -- 172.21.15.46:0/190020647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042e90 msgr2=0x56116604efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f696759e700 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042e90 0x56116604efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f696759e700 1 -- 172.21.15.46:0/190020647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 msgr2=0x561166051c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f696759e700 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 0x561166051c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f696759e700 1 -- 172.21.15.46:0/190020647 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696810d310 con 0x5611660521c0 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f695e59c700 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042e90 0x56116604efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f6966d9d700 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 0x561166051c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f696759e700 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611660521c0 0x56116604f6f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69600c9f90 tx=0x7f69600995f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb8e3d4be07f5fba server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f694e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f694e7fc700 1 -- 172.21.15.46:0/190020647 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960004ad0 con 0x5611660521c0 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f694e7fc700 1 -- 172.21.15.46:0/190020647 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f696009a0e0 con 0x5611660521c0 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f69792eeb80 1 -- 172.21.15.46:0/190020647 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611660366c0 con 0x5611660521c0 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f694e7fc700 1 -- 172.21.15.46:0/190020647 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69600999e0 con 0x5611660521c0 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f69792eeb80 10 client.28345 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f69792eeb80 1 -- 172.21.15.46:0/190020647 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f6958017070 con 0x5611660521c0 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.197+0000 7f694e7fc700 1 -- 172.21.15.46:0/190020647 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5611660366c0 con 0x5611660521c0 2024-03-18T20:45:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.198+0000 7f694e7fc700 1 -- 172.21.15.46:0/190020647 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6960098ff0 con 0x5611660521c0 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.198+0000 7f69792eeb80 10 client.28345 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.198+0000 7f69792eeb80 10 client.28345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.198+0000 7f69792eeb80 20 client.28345 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.198+0000 7f69792eeb80 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561166061cd0 0x5611660640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.198+0000 7f69792eeb80 4 client.28345 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.198+0000 7f69792eeb80 1 -- 172.21.15.46:0/190020647 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561165f8e6a0 con 0x561166061cd0 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.198+0000 7f6966d9d700 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561166061cd0 0x5611660640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.198+0000 7f6966d9d700 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561166061cd0 0x5611660640c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d33f9c109fa06c server_cookie=7e6358fc5f42b860 in_seq=0 out_seq=0 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.198+0000 7f694e7fc700 10 client.28345 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f694e7fc700 1 -- 172.21.15.46:0/190020647 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561165f8e6a0 con 0x561166061cd0 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f694e7fc700 10 client.28345 handle_command_reply: tid=0 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f69792eeb80 1 client.28345 shutdown 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f69792eeb80 1 -- 172.21.15.46:0/190020647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611660521c0 msgr2=0x56116604f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f69792eeb80 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611660521c0 0x56116604f6f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69600c9f90 tx=0x7f69600995f0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f69792eeb80 1 -- 172.21.15.46:0/190020647 shutdown_connections 2024-03-18T20:45:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f69792eeb80 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561166061cd0 0x5611660640c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f69792eeb80 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042570 0x561166051c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f69792eeb80 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5611660521c0 0x56116604f6f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f694e7fc700 0 client.28345 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f69792eeb80 1 --2- 172.21.15.46:0/190020647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042e90 0x56116604efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f69792eeb80 1 -- 172.21.15.46:0/190020647 >> 172.21.15.46:0/190020647 conn(0x561165fb91c0 msgr2=0x56116603e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f69792eeb80 1 -- 172.21.15.46:0/190020647 shutdown_connections 2024-03-18T20:45:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f69792eeb80 1 -- 172.21.15.46:0/190020647 wait complete. 2024-03-18T20:45:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.199+0000 7f69792eeb80 20 client.28345 trim_cache size 0 max 16384 2024-03-18T20:45:11.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.217+0000 7f69792eeb80 1 Processor -- start 2024-03-18T20:45:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.217+0000 7f69792eeb80 1 -- start start 2024-03-18T20:45:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.217+0000 7f69792eeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042570 0x56116604e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.217+0000 7f69792eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042e90 0x561166058e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.217+0000 7f69792eeb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611660521c0 0x56116603d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.217+0000 7f69792eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116603fd60 con 0x561166042e90 2024-03-18T20:45:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.217+0000 7f69792eeb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561166038c10 con 0x5611660521c0 2024-03-18T20:45:11.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.217+0000 7f69792eeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56116603ff70 con 0x561166042570 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f695e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042e90 0x561166058e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f695e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042e90 0x561166058e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47356/0 (socket says 172.21.15.46:47356) 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f695e59c700 1 -- 172.21.15.46:0/2336308734 learned_addr learned my addr 172.21.15.46:0/2336308734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f6966d9d700 1 --2- 172.21.15.46:0/2336308734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042570 0x56116604e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f696759e700 1 --2- 172.21.15.46:0/2336308734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611660521c0 0x56116603d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f695e59c700 1 -- 172.21.15.46:0/2336308734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042570 msgr2=0x56116604e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f695e59c700 1 --2- 172.21.15.46:0/2336308734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042570 0x56116604e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f695e59c700 1 -- 172.21.15.46:0/2336308734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611660521c0 msgr2=0x56116603d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f695e59c700 1 --2- 172.21.15.46:0/2336308734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611660521c0 0x56116603d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f695e59c700 1 -- 172.21.15.46:0/2336308734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561165dc3b00 con 0x561166042e90 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f696759e700 1 --2- 172.21.15.46:0/2336308734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611660521c0 0x56116603d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f695e59c700 1 --2- 172.21.15.46:0/2336308734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042e90 0x561166058e50 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6950009990 tx=0x7f6950005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d3ce373fc262a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:11.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f695dd9b700 1 -- 172.21.15.46:0/2336308734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695001d070 con 0x561166042e90 2024-03-18T20:45:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f695dd9b700 1 -- 172.21.15.46:0/2336308734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f695001e040 con 0x561166042e90 2024-03-18T20:45:11.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.218+0000 7f695dd9b700 1 -- 172.21.15.46:0/2336308734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950004e00 con 0x561166042e90 2024-03-18T20:45:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.219+0000 7f69792eeb80 1 -- 172.21.15.46:0/2336308734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042e90 msgr2=0x561166058e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:11.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.219+0000 7f69792eeb80 1 --2- 172.21.15.46:0/2336308734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042e90 0x561166058e50 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6950009990 tx=0x7f6950005590 comp rx=0 tx=0).stop 2024-03-18T20:45:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.219+0000 7f69792eeb80 1 -- 172.21.15.46:0/2336308734 shutdown_connections 2024-03-18T20:45:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.219+0000 7f69792eeb80 1 --2- 172.21.15.46:0/2336308734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5611660521c0 0x56116603d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.219+0000 7f69792eeb80 1 --2- 172.21.15.46:0/2336308734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042570 0x56116604e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.219+0000 7f69792eeb80 1 --2- 172.21.15.46:0/2336308734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561166042e90 0x561166058e50 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.219+0000 7f69792eeb80 1 -- 172.21.15.46:0/2336308734 >> 172.21.15.46:0/2336308734 conn(0x561165fb91c0 msgr2=0x561166068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.219+0000 7f69792eeb80 1 -- 172.21.15.46:0/2336308734 shutdown_connections 2024-03-18T20:45:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.219+0000 7f69792eeb80 1 -- 172.21.15.46:0/2336308734 wait complete. 2024-03-18T20:45:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f69792eeb80 1 -- start start 2024-03-18T20:45:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f69792eeb80 1 Processor -- start 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f69792eeb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042570 0x561166056e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f69792eeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042e90 0x561166057370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f69792eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611660521c0 0x5611660578d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f69792eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611660b4cb0 con 0x5611660521c0 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f69792eeb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611660b4e20 con 0x561166042570 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f69792eeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611660b4fe0 con 0x561166042e90 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f69792eeb80 10 client.0 _ll_register_callbacks cb 0x7f69600997b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f6966d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042570 0x561166056e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f695e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042e90 0x561166057370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f696759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611660521c0 0x5611660578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f6966d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042570 0x561166056e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:59910/0 (socket says 172.21.15.46:59910) 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.220+0000 7f6966d9d700 1 -- 172.21.15.46:0/4216785898 learned_addr learned my addr 172.21.15.46:0/4216785898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f6966d9d700 1 -- 172.21.15.46:0/4216785898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042e90 msgr2=0x561166057370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f6966d9d700 1 --2- 172.21.15.46:0/4216785898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042e90 0x561166057370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f6966d9d700 1 -- 172.21.15.46:0/4216785898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611660521c0 msgr2=0x5611660578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f6966d9d700 1 --2- 172.21.15.46:0/4216785898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611660521c0 0x5611660578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f6966d9d700 1 -- 172.21.15.46:0/4216785898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6950005350 con 0x561166042570 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f6966d9d700 1 --2- 172.21.15.46:0/4216785898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042570 0x561166056e30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6958009cf0 tx=0x7f695800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c44cae35fce0d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f694e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f694e7fc700 1 -- 172.21.15.46:0/4216785898 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958033a90 con 0x561166042570 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f694e7fc700 1 -- 172.21.15.46:0/4216785898 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f6958033bf0 con 0x561166042570 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f69792eeb80 1 -- 172.21.15.46:0/4216785898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6958021070 con 0x561166042570 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f694e7fc700 1 -- 172.21.15.46:0/4216785898 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6958033db0 con 0x561166042570 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f69792eeb80 10 client.28346 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f69792eeb80 1 -- 172.21.15.46:0/4216785898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x561165f5be40 con 0x561166042570 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.221+0000 7f694e7fc700 1 -- 172.21.15.46:0/4216785898 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6958021070 con 0x561166042570 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.222+0000 7f694e7fc700 1 -- 172.21.15.46:0/4216785898 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f69580046b0 con 0x561166042570 2024-03-18T20:45:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.222+0000 7f69792eeb80 10 client.28346 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.222+0000 7f69792eeb80 10 client.28346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.222+0000 7f69792eeb80 20 client.28346 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.222+0000 7f69792eeb80 1 --2- 172.21.15.46:0/4216785898 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5611660b6700 0x5611660b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.222+0000 7f69792eeb80 4 client.28346 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:45:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.222+0000 7f69792eeb80 1 -- 172.21.15.46:0/4216785898 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561165de4c90 con 0x5611660b6700 2024-03-18T20:45:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.222+0000 7f695e59c700 1 --2- 172.21.15.46:0/4216785898 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5611660b6700 0x5611660b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.222+0000 7f694e7fc700 10 client.28346 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.222+0000 7f695e59c700 1 --2- 172.21.15.46:0/4216785898 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5611660b6700 0x5611660b8af0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=937f2652b35fefd5 server_cookie=bfcaa396224c1eed in_seq=0 out_seq=0 2024-03-18T20:45:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.259+0000 7f694e7fc700 1 -- 172.21.15.46:0/4216785898 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561165de4c90 con 0x5611660b6700 2024-03-18T20:45:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.259+0000 7f694e7fc700 10 client.28346 handle_command_reply: tid=0 2024-03-18T20:45:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.259+0000 7f69792eeb80 1 client.28346 shutdown 2024-03-18T20:45:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.260+0000 7f69792eeb80 1 -- 172.21.15.46:0/4216785898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042570 msgr2=0x561166056e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.260+0000 7f69792eeb80 1 --2- 172.21.15.46:0/4216785898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042570 0x561166056e30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6958009cf0 tx=0x7f695800edc0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.260+0000 7f69792eeb80 1 -- 172.21.15.46:0/4216785898 shutdown_connections 2024-03-18T20:45:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.260+0000 7f69792eeb80 1 --2- 172.21.15.46:0/4216785898 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5611660b6700 0x5611660b8af0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.260+0000 7f69792eeb80 1 --2- 172.21.15.46:0/4216785898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5611660521c0 0x5611660578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.260+0000 7f69792eeb80 1 --2- 172.21.15.46:0/4216785898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561166042570 0x561166056e30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.260+0000 7f69792eeb80 1 --2- 172.21.15.46:0/4216785898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561166042e90 0x561166057370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.260+0000 7f69792eeb80 1 -- 172.21.15.46:0/4216785898 >> 172.21.15.46:0/4216785898 conn(0x561165fb91c0 msgr2=0x56116603d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.260+0000 7f694e7fc700 0 client.28346 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.260+0000 7f69792eeb80 1 -- 172.21.15.46:0/4216785898 shutdown_connections 2024-03-18T20:45:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.260+0000 7f69792eeb80 1 -- 172.21.15.46:0/4216785898 wait complete. 2024-03-18T20:45:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.260+0000 7f69792eeb80 20 client.28346 trim_cache size 0 max 16384 2024-03-18T20:45:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:45:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:45:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:45:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:45:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.261+0000 7f696d159700 1 -- 172.21.15.46:0/520210685 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f69540a1290 msgr2=0x7f69540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.261+0000 7f696d159700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f69540a1290 0x7f69540a3750 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6958011550 tx=0x7f6958009260 comp rx=0 tx=0).stop 2024-03-18T20:45:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.261+0000 7f696d159700 1 -- 172.21.15.46:0/520210685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6968198a70 msgr2=0x7f69681a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.261+0000 7f696d159700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6968198a70 0x7f69681a3070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f696000efc0 tx=0x7f696000c3a0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.261+0000 7f696d159700 1 -- 172.21.15.46:0/520210685 shutdown_connections 2024-03-18T20:45:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.261+0000 7f696d159700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f69540a1290 0x7f69540a3750 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.261+0000 7f696d159700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680ffe00 0x7f6968197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.261+0000 7f696d159700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6968198a70 0x7f69681a3070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.261+0000 7f696d159700 1 --2- 172.21.15.46:0/520210685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6968104f20 0x7f6968198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.261+0000 7f696d159700 1 -- 172.21.15.46:0/520210685 >> 172.21.15.46:0/520210685 conn(0x7f69680fb680 msgr2=0x7f69680fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.261+0000 7f696d159700 1 -- 172.21.15.46:0/520210685 shutdown_connections 2024-03-18T20:45:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:11.261+0000 7f696d159700 1 -- 172.21.15.46:0/520210685 wait complete. 2024-03-18T20:45:11.708 INFO:tasks.workunit.client.0.smithi046.stdout: 1 330954 3.66 MB/sec execute 366 sec latency 20.308 ms 2024-03-18T20:45:12.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:12 smithi155 ceph-mon[48344]: pgmap v282: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 924 KiB/s wr, 82 op/s 2024-03-18T20:45:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:12 smithi046 ceph-mon[43495]: pgmap v282: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 924 KiB/s wr, 82 op/s 2024-03-18T20:45:12.708 INFO:tasks.workunit.client.0.smithi046.stdout: 1 331577 3.66 MB/sec execute 367 sec latency 500.221 ms 2024-03-18T20:45:12.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:12 smithi073 ceph-mon[47384]: pgmap v282: 97 pgs: 97 active+clean; 128 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 924 KiB/s wr, 82 op/s 2024-03-18T20:45:13.708 INFO:tasks.workunit.client.0.smithi046.stdout: 1 331577 3.65 MB/sec execute 368 sec latency 1500.340 ms 2024-03-18T20:45:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:14 smithi155 ceph-mon[48344]: pgmap v283: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:45:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:14 smithi046 ceph-mon[43495]: pgmap v283: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:45:14.708 INFO:tasks.workunit.client.0.smithi046.stdout: 1 331577 3.64 MB/sec execute 369 sec latency 2500.450 ms 2024-03-18T20:45:14.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:14 smithi073 ceph-mon[47384]: pgmap v283: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:45:15.708 INFO:tasks.workunit.client.0.smithi046.stdout: 1 332481 3.64 MB/sec execute 370 sec latency 3130.311 ms 2024-03-18T20:45:16.219 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:15 smithi073 ceph-mon[47384]: pgmap v284: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-03-18T20:45:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:15 smithi155 ceph-mon[48344]: pgmap v284: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-03-18T20:45:16.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:15 smithi046 ceph-mon[43495]: pgmap v284: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-03-18T20:45:16.708 INFO:tasks.workunit.client.0.smithi046.stdout: 1 333894 3.66 MB/sec execute 371 sec latency 16.207 ms 2024-03-18T20:45:17.219 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:16 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:45:17.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:16 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:45:17.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:16 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:45:17.709 INFO:tasks.workunit.client.0.smithi046.stdout: 1 335120 3.66 MB/sec execute 372 sec latency 19.990 ms 2024-03-18T20:45:17.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:45:17.966 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:45:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.113+0000 7f3dfbffb700 1 -- 172.21.15.46:0/4099836864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3df40ffe50 msgr2=0x7f3df4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.113+0000 7f3dfbffb700 1 --2- 172.21.15.46:0/4099836864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3df40ffe50 0x7f3df4100230 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009990 tx=0x7f3de4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.113+0000 7f3dfbffb700 1 -- 172.21.15.46:0/4099836864 shutdown_connections 2024-03-18T20:45:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.113+0000 7f3dfbffb700 1 --2- 172.21.15.46:0/4099836864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3df40ffe50 0x7f3df4100230 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.113+0000 7f3dfbffb700 1 --2- 172.21.15.46:0/4099836864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df4104f60 0x7f3df410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.113+0000 7f3dfbffb700 1 --2- 172.21.15.46:0/4099836864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3df4100800 0x7f3df4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.113+0000 7f3dfbffb700 1 -- 172.21.15.46:0/4099836864 >> 172.21.15.46:0/4099836864 conn(0x7f3df4076d40 msgr2=0x7f3df4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.113+0000 7f3dfbffb700 1 -- 172.21.15.46:0/4099836864 shutdown_connections 2024-03-18T20:45:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.113+0000 7f3dfbffb700 1 -- 172.21.15.46:0/4099836864 wait complete. 2024-03-18T20:45:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.114+0000 7f3dfbffb700 1 Processor -- start 2024-03-18T20:45:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.114+0000 7f3dfbffb700 1 -- start start 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.114+0000 7f3dfbffb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df40ffe50 0x7f3df4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.114+0000 7f3dfbffb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3df4104f60 0x7f3df4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.114+0000 7f3dfbffb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3df4198a90 0x7f3df41a3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.114+0000 7f3dfbffb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41a3730 con 0x7f3df4104f60 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.114+0000 7f3dfbffb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41a38a0 con 0x7f3df40ffe50 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.114+0000 7f3dfbffb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41a3a60 con 0x7f3df4198a90 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.114+0000 7f3df9596700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3df4104f60 0x7f3df4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3df9596700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3df4104f60 0x7f3df4198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38118/0 (socket says 172.21.15.46:38118) 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3df9596700 1 -- 172.21.15.46:0/2653677318 learned_addr learned my addr 172.21.15.46:0/2653677318 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3df9d97700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df40ffe50 0x7f3df4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3dfa598700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3df4198a90 0x7f3df41a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3df9596700 1 -- 172.21.15.46:0/2653677318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3df4198a90 msgr2=0x7f3df41a3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3df9596700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3df4198a90 0x7f3df41a3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3df9596700 1 -- 172.21.15.46:0/2653677318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df40ffe50 msgr2=0x7f3df4198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3df9596700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df40ffe50 0x7f3df4198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3df9596700 1 -- 172.21.15.46:0/2653677318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40095f0 con 0x7f3df4104f60 2024-03-18T20:45:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3df9596700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3df4104f60 0x7f3df4198550 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3df000d980 tx=0x7f3df000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3deaffd700 1 -- 172.21.15.46:0/2653677318 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df000f890 con 0x7f3df4104f60 2024-03-18T20:45:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3deaffd700 1 -- 172.21.15.46:0/2653677318 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3df0010460 con 0x7f3df4104f60 2024-03-18T20:45:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3dfbffb700 1 -- 172.21.15.46:0/2653677318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df41a3d40 con 0x7f3df4104f60 2024-03-18T20:45:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3deaffd700 1 -- 172.21.15.46:0/2653677318 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df000e610 con 0x7f3df4104f60 2024-03-18T20:45:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.115+0000 7f3dfbffb700 1 -- 172.21.15.46:0/2653677318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df41a41f0 con 0x7f3df4104f60 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.118+0000 7f3deaffd700 1 -- 172.21.15.46:0/2653677318 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3df00105d0 con 0x7f3df4104f60 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.118+0000 7f3deaffd700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3de00a1290 0x7f3de00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.119+0000 7f3df9d97700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3de00a1290 0x7f3de00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.119+0000 7f3deaffd700 1 -- 172.21.15.46:0/2653677318 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3df00cc6e0 con 0x7f3df4104f60 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.119+0000 7f3df9d97700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3de00a1290 0x7f3de00a3750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3de400b2e0 tx=0x7f3de4005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.119+0000 7f3e08190b80 1 Processor -- start 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3e08190b80 1 -- start start 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3e08190b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 0x55ee86645e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3e08190b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86646380 0x55ee86650ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3e08190b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 0x55ee86653610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3e08190b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8659f990 con 0x55ee86645a60 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3e08190b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8659d350 con 0x55ee86646380 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3e08190b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee86598c40 con 0x55ee86651220 2024-03-18T20:45:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3df9d97700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 0x55ee86645e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3df9d97700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 0x55ee86645e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38128/0 (socket says 172.21.15.46:38128) 2024-03-18T20:45:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3df9d97700 1 -- 172.21.15.46:0/1458092458 learned_addr learned my addr 172.21.15.46:0/1458092458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3dfa598700 1 --2- 172.21.15.46:0/1458092458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 0x55ee86653610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3df9596700 1 --2- 172.21.15.46:0/1458092458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86646380 0x55ee86650ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3df9d97700 1 -- 172.21.15.46:0/1458092458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 msgr2=0x55ee86653610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3df9d97700 1 --2- 172.21.15.46:0/1458092458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 0x55ee86653610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3df9d97700 1 -- 172.21.15.46:0/1458092458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86646380 msgr2=0x55ee86650ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3df9d97700 1 --2- 172.21.15.46:0/1458092458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86646380 0x55ee86650ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3df9d97700 1 -- 172.21.15.46:0/1458092458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee8641a3b0 con 0x55ee86645a60 2024-03-18T20:45:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.120+0000 7f3df9596700 1 --2- 172.21.15.46:0/1458092458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86646380 0x55ee86650ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.121+0000 7f3df9d97700 1 --2- 172.21.15.46:0/1458092458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 0x55ee86645e40 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f3de4015000 tx=0x7f3de4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea526491654b490d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.121+0000 7f3ddffff700 1 -- 172.21.15.46:0/1458092458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4004700 con 0x55ee86645a60 2024-03-18T20:45:18.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.121+0000 7f3ddffff700 1 -- 172.21.15.46:0/1458092458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de401e040 con 0x55ee86645a60 2024-03-18T20:45:18.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.121+0000 7f3ddffff700 1 -- 172.21.15.46:0/1458092458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400f960 con 0x55ee86645a60 2024-03-18T20:45:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.122+0000 7f3e08190b80 1 -- 172.21.15.46:0/1458092458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 msgr2=0x55ee86645e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.122+0000 7f3e08190b80 1 --2- 172.21.15.46:0/1458092458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 0x55ee86645e40 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f3de4015000 tx=0x7f3de4020000 comp rx=0 tx=0).stop 2024-03-18T20:45:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.122+0000 7f3e08190b80 1 -- 172.21.15.46:0/1458092458 shutdown_connections 2024-03-18T20:45:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.122+0000 7f3e08190b80 1 --2- 172.21.15.46:0/1458092458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 0x55ee86645e40 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.122+0000 7f3e08190b80 1 --2- 172.21.15.46:0/1458092458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 0x55ee86653610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.122+0000 7f3e08190b80 1 --2- 172.21.15.46:0/1458092458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86646380 0x55ee86650ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.122+0000 7f3e08190b80 1 -- 172.21.15.46:0/1458092458 >> 172.21.15.46:0/1458092458 conn(0x55ee865bc220 msgr2=0x55ee86642020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.122+0000 7f3e08190b80 1 -- 172.21.15.46:0/1458092458 shutdown_connections 2024-03-18T20:45:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.122+0000 7f3e08190b80 1 -- 172.21.15.46:0/1458092458 wait complete. 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.122+0000 7f3e08190b80 1 -- start start 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.122+0000 7f3e08190b80 1 Processor -- start 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3e08190b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 0x55ee86650c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3e08190b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86646380 0x55ee8664df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3e08190b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86651220 0x55ee8664e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3e08190b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee86534d80 con 0x55ee86645a60 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3e08190b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee86534a20 con 0x55ee86651220 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3e08190b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8641c6a0 con 0x55ee86646380 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3e08190b80 10 client.0 _ll_register_callbacks cb 0x55ee863a03a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3df9d97700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 0x55ee86650c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3df9d97700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 0x55ee86650c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38140/0 (socket says 172.21.15.46:38140) 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3df9d97700 1 -- 172.21.15.46:0/3886041385 learned_addr learned my addr 172.21.15.46:0/3886041385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3df9596700 1 --2- 172.21.15.46:0/3886041385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86646380 0x55ee8664df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3df9d97700 1 -- 172.21.15.46:0/3886041385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86646380 msgr2=0x55ee8664df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3df9d97700 1 --2- 172.21.15.46:0/3886041385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86646380 0x55ee8664df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3df9d97700 1 -- 172.21.15.46:0/3886041385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86651220 msgr2=0x55ee8664e690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3df9d97700 1 --2- 172.21.15.46:0/3886041385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86651220 0x55ee8664e690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.123+0000 7f3df9d97700 1 -- 172.21.15.46:0/3886041385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df410d350 con 0x55ee86645a60 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.124+0000 7f3df9d97700 1 --2- 172.21.15.46:0/3886041385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 0x55ee86650c60 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009ce0 tx=0x7f3de40057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b7861608ab1d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.124+0000 7f3ddd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.124+0000 7f3ddd7fa700 1 -- 172.21.15.46:0/3886041385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4004910 con 0x55ee86645a60 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.124+0000 7f3ddd7fa700 1 -- 172.21.15.46:0/3886041385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3de400be60 con 0x55ee86645a60 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.124+0000 7f3e08190b80 1 -- 172.21.15.46:0/3886041385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee86639ad0 con 0x55ee86645a60 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.124+0000 7f3ddd7fa700 1 -- 172.21.15.46:0/3886041385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3de4004a70 con 0x55ee86645a60 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.124+0000 7f3e08190b80 10 client.18711 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.124+0000 7f3e08190b80 1 -- 172.21.15.46:0/3886041385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55ee863c6b60 con 0x55ee86645a60 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.124+0000 7f3ddd7fa700 1 -- 172.21.15.46:0/3886041385 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de4047020 con 0x55ee86645a60 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.124+0000 7f3ddd7fa700 1 -- 172.21.15.46:0/3886041385 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3de402fbd0 con 0x55ee86645a60 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.124+0000 7f3e08190b80 10 client.18711 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.124+0000 7f3e08190b80 10 client.18711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.125+0000 7f3e08190b80 20 client.18711 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.125+0000 7f3e08190b80 1 --2- 172.21.15.46:0/3886041385 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ee866608a0 0x55ee86662cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.125+0000 7f3e08190b80 4 client.18711 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.125+0000 7f3e08190b80 1 -- 172.21.15.46:0/3886041385 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee86591700 con 0x55ee866608a0 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.125+0000 7f3df9596700 1 --2- 172.21.15.46:0/3886041385 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ee866608a0 0x55ee86662cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.125+0000 7f3ddd7fa700 10 client.18711 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.125+0000 7f3df9596700 1 --2- 172.21.15.46:0/3886041385 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ee866608a0 0x55ee86662cc0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f555d0d7d4c9942 server_cookie=13c410f98b5cea8c in_seq=0 out_seq=0 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3ddd7fa700 1 -- 172.21.15.46:0/3886041385 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ee86591700 con 0x55ee866608a0 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3ddd7fa700 10 client.18711 handle_command_reply: tid=0 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3e08190b80 1 client.18711 shutdown 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3e08190b80 1 -- 172.21.15.46:0/3886041385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 msgr2=0x55ee86650c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3e08190b80 1 --2- 172.21.15.46:0/3886041385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 0x55ee86650c60 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009ce0 tx=0x7f3de40057e0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3e08190b80 1 -- 172.21.15.46:0/3886041385 shutdown_connections 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3e08190b80 1 --2- 172.21.15.46:0/3886041385 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ee866608a0 0x55ee86662cc0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3e08190b80 1 --2- 172.21.15.46:0/3886041385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86645a60 0x55ee86650c60 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3e08190b80 1 --2- 172.21.15.46:0/3886041385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86651220 0x55ee8664e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3e08190b80 1 --2- 172.21.15.46:0/3886041385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86646380 0x55ee8664df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3ddd7fa700 0 client.18711 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3e08190b80 1 -- 172.21.15.46:0/3886041385 >> 172.21.15.46:0/3886041385 conn(0x55ee865bc220 msgr2=0x55ee86641c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3e08190b80 1 -- 172.21.15.46:0/3886041385 shutdown_connections 2024-03-18T20:45:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3e08190b80 1 -- 172.21.15.46:0/3886041385 wait complete. 2024-03-18T20:45:18.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.126+0000 7f3e08190b80 20 client.18711 trim_cache size 0 max 16384 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3e08190b80 1 Processor -- start 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3e08190b80 1 -- start start 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3e08190b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86645a60 0x55ee866489c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3e08190b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 0x55ee86657e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3e08190b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 0x55ee8665c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3e08190b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee86658360 con 0x55ee86646380 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3e08190b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee8665d1c0 con 0x55ee86645a60 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3e08190b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee86649dd0 con 0x55ee86651220 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3df9596700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 0x55ee86657e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3df9596700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 0x55ee86657e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38144/0 (socket says 172.21.15.46:38144) 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3df9596700 1 -- 172.21.15.46:0/3344723361 learned_addr learned my addr 172.21.15.46:0/3344723361 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3dfa598700 1 --2- 172.21.15.46:0/3344723361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 0x55ee8665c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3df9d97700 1 --2- 172.21.15.46:0/3344723361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86645a60 0x55ee866489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3df9596700 1 -- 172.21.15.46:0/3344723361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 msgr2=0x55ee8665c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3df9596700 1 --2- 172.21.15.46:0/3344723361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 0x55ee8665c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3df9596700 1 -- 172.21.15.46:0/3344723361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86645a60 msgr2=0x55ee866489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3df9596700 1 --2- 172.21.15.46:0/3344723361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86645a60 0x55ee866489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3df9596700 1 -- 172.21.15.46:0/3344723361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee8655eea0 con 0x55ee86646380 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.144+0000 7f3df9d97700 1 --2- 172.21.15.46:0/3344723361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86645a60 0x55ee866489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.145+0000 7f3df9596700 1 --2- 172.21.15.46:0/3344723361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 0x55ee86657e20 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3df0096240 tx=0x7f3df00b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3631d27ea91e909a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.145+0000 7f3de8ff9700 1 -- 172.21.15.46:0/3344723361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df009d600 con 0x55ee86646380 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.145+0000 7f3de8ff9700 1 -- 172.21.15.46:0/3344723361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3df00a1c50 con 0x55ee86646380 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.145+0000 7f3de8ff9700 1 -- 172.21.15.46:0/3344723361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df00b90b0 con 0x55ee86646380 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.145+0000 7f3e08190b80 1 -- 172.21.15.46:0/3344723361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 msgr2=0x55ee86657e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.145+0000 7f3e08190b80 1 --2- 172.21.15.46:0/3344723361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 0x55ee86657e20 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3df0096240 tx=0x7f3df00b9040 comp rx=0 tx=0).stop 2024-03-18T20:45:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.145+0000 7f3e08190b80 1 -- 172.21.15.46:0/3344723361 shutdown_connections 2024-03-18T20:45:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.145+0000 7f3e08190b80 1 --2- 172.21.15.46:0/3344723361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 0x55ee8665c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.145+0000 7f3e08190b80 1 --2- 172.21.15.46:0/3344723361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86645a60 0x55ee866489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.145+0000 7f3e08190b80 1 --2- 172.21.15.46:0/3344723361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 0x55ee86657e20 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.146+0000 7f3e08190b80 1 -- 172.21.15.46:0/3344723361 >> 172.21.15.46:0/3344723361 conn(0x55ee865bc220 msgr2=0x55ee8663da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.146+0000 7f3e08190b80 1 -- 172.21.15.46:0/3344723361 shutdown_connections 2024-03-18T20:45:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.146+0000 7f3e08190b80 1 -- 172.21.15.46:0/3344723361 wait complete. 2024-03-18T20:45:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.146+0000 7f3e08190b80 1 -- start start 2024-03-18T20:45:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.146+0000 7f3e08190b80 1 Processor -- start 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.146+0000 7f3e08190b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86645a60 0x55ee86656570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.146+0000 7f3e08190b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 0x55ee86656ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.146+0000 7f3e08190b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 0x55ee86656ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.146+0000 7f3e08190b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee866b4760 con 0x55ee86646380 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.146+0000 7f3e08190b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee866b48d0 con 0x55ee86645a60 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.146+0000 7f3e08190b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee866b4a90 con 0x55ee86651220 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3e08190b80 10 client.0 _ll_register_callbacks cb 0x7f3dec009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3df9596700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 0x55ee86656ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3df9596700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 0x55ee86656ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38148/0 (socket says 172.21.15.46:38148) 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3df9d97700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86645a60 0x55ee86656570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3df9596700 1 -- 172.21.15.46:0/1138168252 learned_addr learned my addr 172.21.15.46:0/1138168252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3dfa598700 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 0x55ee86656ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3df9596700 1 -- 172.21.15.46:0/1138168252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 msgr2=0x55ee86656ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3df9596700 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 0x55ee86656ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3df9596700 1 -- 172.21.15.46:0/1138168252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86645a60 msgr2=0x55ee86656570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3df9596700 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86645a60 0x55ee86656570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3df9596700 1 -- 172.21.15.46:0/1138168252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df41a41f0 con 0x55ee86646380 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3df9d97700 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86645a60 0x55ee86656570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3df9596700 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 0x55ee86656ab0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3df00994d0 tx=0x7f3df000b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0b4101fb85948b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3ddd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3ddd7fa700 1 -- 172.21.15.46:0/1138168252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df009d600 con 0x55ee86646380 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3ddd7fa700 1 -- 172.21.15.46:0/1138168252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3df00108b0 con 0x55ee86646380 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3e08190b80 1 -- 172.21.15.46:0/1138168252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3de401e040 con 0x55ee86646380 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.147+0000 7f3ddd7fa700 1 -- 172.21.15.46:0/1138168252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3df00b90b0 con 0x55ee86646380 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.148+0000 7f3e08190b80 10 client.18717 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.148+0000 7f3e08190b80 1 -- 172.21.15.46:0/1138168252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55ee8641ecd0 con 0x55ee86646380 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.148+0000 7f3ddd7fa700 1 -- 172.21.15.46:0/1138168252 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de401e040 con 0x55ee86646380 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.148+0000 7f3ddd7fa700 1 -- 172.21.15.46:0/1138168252 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3df0098cc0 con 0x55ee86646380 2024-03-18T20:45:18.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.148+0000 7f3e08190b80 10 client.18717 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:18.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.148+0000 7f3e08190b80 10 client.18717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:18.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.148+0000 7f3e08190b80 20 client.18717 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:18.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.148+0000 7f3e08190b80 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ee866b5e20 0x55ee866b8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.148+0000 7f3e08190b80 4 client.18717 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:45:18.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.148+0000 7f3e08190b80 1 -- 172.21.15.46:0/1138168252 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ee863e7cf0 con 0x55ee866b5e20 2024-03-18T20:45:18.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.148+0000 7f3df9d97700 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ee866b5e20 0x55ee866b8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3ddd7fa700 10 client.18717 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:18.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3df9d97700 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ee866b5e20 0x55ee866b8210 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4174ce57b15bb588 server_cookie=ae3fc5a18093851c in_seq=0 out_seq=0 2024-03-18T20:45:18.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3ddd7fa700 1 -- 172.21.15.46:0/1138168252 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ee863e7cf0 con 0x55ee866b5e20 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3ddd7fa700 10 client.18717 handle_command_reply: tid=0 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3e08190b80 1 client.18717 shutdown 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3e08190b80 1 -- 172.21.15.46:0/1138168252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 msgr2=0x55ee86656ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3e08190b80 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 0x55ee86656ab0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3df00994d0 tx=0x7f3df000b510 comp rx=0 tx=0).stop 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3e08190b80 1 -- 172.21.15.46:0/1138168252 shutdown_connections 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3e08190b80 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ee866b5e20 0x55ee866b8210 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3e08190b80 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee86651220 0x55ee86656ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3e08190b80 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee86645a60 0x55ee86656570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3ddd7fa700 0 client.18717 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3e08190b80 1 --2- 172.21.15.46:0/1138168252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ee86646380 0x55ee86656ab0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3e08190b80 1 -- 172.21.15.46:0/1138168252 >> 172.21.15.46:0/1138168252 conn(0x55ee865bc220 msgr2=0x55ee8665c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3e08190b80 1 -- 172.21.15.46:0/1138168252 shutdown_connections 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3e08190b80 1 -- 172.21.15.46:0/1138168252 wait complete. 2024-03-18T20:45:18.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.149+0000 7f3e08190b80 20 client.18717 trim_cache size 0 max 16384 2024-03-18T20:45:18.159 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:45:18.159 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:45:18.159 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:45:18.159 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:45:18.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.150+0000 7f3dfbffb700 1 -- 172.21.15.46:0/2653677318 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3de00a1290 msgr2=0x7f3de00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.150+0000 7f3dfbffb700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3de00a1290 0x7f3de00a3750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3de400b2e0 tx=0x7f3de4005960 comp rx=0 tx=0).stop 2024-03-18T20:45:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.150+0000 7f3dfbffb700 1 -- 172.21.15.46:0/2653677318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3df4104f60 msgr2=0x7f3df4198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.150+0000 7f3dfbffb700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3df4104f60 0x7f3df4198550 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3df000d980 tx=0x7f3df000dc90 comp rx=0 tx=0).stop 2024-03-18T20:45:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.150+0000 7f3dfbffb700 1 -- 172.21.15.46:0/2653677318 shutdown_connections 2024-03-18T20:45:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.150+0000 7f3dfbffb700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3de00a1290 0x7f3de00a3750 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.150+0000 7f3dfbffb700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3df4198a90 0x7f3df41a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.150+0000 7f3dfbffb700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3df40ffe50 0x7f3df4198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.150+0000 7f3dfbffb700 1 --2- 172.21.15.46:0/2653677318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3df4104f60 0x7f3df4198550 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.150+0000 7f3dfbffb700 1 -- 172.21.15.46:0/2653677318 >> 172.21.15.46:0/2653677318 conn(0x7f3df4076d40 msgr2=0x7f3df40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.150+0000 7f3dfbffb700 1 -- 172.21.15.46:0/2653677318 shutdown_connections 2024-03-18T20:45:18.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.150+0000 7f3dfbffb700 1 -- 172.21.15.46:0/2653677318 wait complete. 2024-03-18T20:45:18.165 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:45:18.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:45:18.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:17 smithi155 ceph-mon[48344]: pgmap v285: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-03-18T20:45:18.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:17 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:45:18.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:17 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:45:18.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:17 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:45:18.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:17 smithi046 ceph-mon[43495]: pgmap v285: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-03-18T20:45:18.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:17 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:45:18.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:17 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:45:18.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:17 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:45:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.316+0000 7fc041e10700 1 -- 172.21.15.46:0/1569279640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c100500 msgr2=0x7fc03c10c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.316+0000 7fc041e10700 1 --2- 172.21.15.46:0/1569279640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c100500 0x7fc03c10c9c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fc03400b0d0 tx=0x7fc03400b3e0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.316+0000 7fc041e10700 1 -- 172.21.15.46:0/1569279640 shutdown_connections 2024-03-18T20:45:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.316+0000 7fc041e10700 1 --2- 172.21.15.46:0/1569279640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c100500 0x7fc03c10c9c0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.316+0000 7fc041e10700 1 --2- 172.21.15.46:0/1569279640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc03c105c00 0x7fc03c105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.316+0000 7fc041e10700 1 --2- 172.21.15.46:0/1569279640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc03c0ffb40 0x7fc03c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.316+0000 7fc041e10700 1 -- 172.21.15.46:0/1569279640 >> 172.21.15.46:0/1569279640 conn(0x7fc03c0fb5f0 msgr2=0x7fc03c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.316+0000 7fc041e10700 1 -- 172.21.15.46:0/1569279640 shutdown_connections 2024-03-18T20:45:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.316+0000 7fc041e10700 1 -- 172.21.15.46:0/1569279640 wait complete. 2024-03-18T20:45:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.317+0000 7fc041e10700 1 Processor -- start 2024-03-18T20:45:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.317+0000 7fc041e10700 1 -- start start 2024-03-18T20:45:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.317+0000 7fc041e10700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc03c0ffb40 0x7fc03c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.317+0000 7fc041e10700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c100500 0x7fc03c1976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.317+0000 7fc041e10700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc03c105c00 0x7fc03c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.317+0000 7fc041e10700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c19c060 con 0x7fc03c100500 2024-03-18T20:45:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.317+0000 7fc041e10700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c19c1d0 con 0x7fc03c105c00 2024-03-18T20:45:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.317+0000 7fc041e10700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c19c390 con 0x7fc03c0ffb40 2024-03-18T20:45:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc03affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c100500 0x7fc03c1976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc03b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc03c0ffb40 0x7fc03c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc03affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c100500 0x7fc03c1976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38158/0 (socket says 172.21.15.46:38158) 2024-03-18T20:45:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc03affd700 1 -- 172.21.15.46:0/2568085194 learned_addr learned my addr 172.21.15.46:0/2568085194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc03bfff700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc03c105c00 0x7fc03c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc03affd700 1 -- 172.21.15.46:0/2568085194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc03c0ffb40 msgr2=0x7fc03c197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc03affd700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc03c0ffb40 0x7fc03c197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc03affd700 1 -- 172.21.15.46:0/2568085194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc03c105c00 msgr2=0x7fc03c19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc03affd700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc03c105c00 0x7fc03c19ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc03affd700 1 -- 172.21.15.46:0/2568085194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c009520 con 0x7fc03c100500 2024-03-18T20:45:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc03affd700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c100500 0x7fc03c1976d0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fc03000e9a0 tx=0x7fc03000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc038ff9700 1 -- 172.21.15.46:0/2568085194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030009790 con 0x7fc03c100500 2024-03-18T20:45:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc041e10700 1 -- 172.21.15.46:0/2568085194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc034009d00 con 0x7fc03c100500 2024-03-18T20:45:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc038ff9700 1 -- 172.21.15.46:0/2568085194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc030010430 con 0x7fc03c100500 2024-03-18T20:45:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc038ff9700 1 -- 172.21.15.46:0/2568085194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030018680 con 0x7fc03c100500 2024-03-18T20:45:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.318+0000 7fc041e10700 1 -- 172.21.15.46:0/2568085194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c1a3cd0 con 0x7fc03c100500 2024-03-18T20:45:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.320+0000 7fc041e10700 1 -- 172.21.15.46:0/2568085194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c1011f0 con 0x7fc03c100500 2024-03-18T20:45:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.321+0000 7fc038ff9700 1 -- 172.21.15.46:0/2568085194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc0300105a0 con 0x7fc03c100500 2024-03-18T20:45:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.321+0000 7fc038ff9700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0240a1290 0x7fc0240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.321+0000 7fc038ff9700 1 -- 172.21.15.46:0/2568085194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc030014070 con 0x7fc03c100500 2024-03-18T20:45:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.321+0000 7fc03b7fe700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0240a1290 0x7fc0240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.322+0000 7fc03b7fe700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0240a1290 0x7fc0240a3750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009df0 tx=0x7fc02c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.326+0000 7fc038ff9700 1 -- 172.21.15.46:0/2568085194 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc0300529f0 con 0x7fc03c100500 2024-03-18T20:45:18.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:17 smithi073 ceph-mon[47384]: pgmap v285: 97 pgs: 97 active+clean; 127 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-03-18T20:45:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:17 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:45:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:17 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:45:18.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:17 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:45:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.591+0000 7fc041e10700 1 -- 172.21.15.46:0/2568085194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc03c067180 con 0x7fc03c100500 2024-03-18T20:45:18.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.592+0000 7fc038ff9700 1 -- 172.21.15.46:0/2568085194 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+10910 (secure 0 0 0) 0x7fc030096300 con 0x7fc03c100500 2024-03-18T20:45:18.594 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:45:18.595 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:44:15.219685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:45:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.594+0000 7fc041e10700 1 -- 172.21.15.46:0/2568085194 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0240a1290 msgr2=0x7fc0240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.594+0000 7fc041e10700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0240a1290 0x7fc0240a3750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009df0 tx=0x7fc02c009260 comp rx=0 tx=0).stop 2024-03-18T20:45:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.594+0000 7fc041e10700 1 -- 172.21.15.46:0/2568085194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c100500 msgr2=0x7fc03c1976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.594+0000 7fc041e10700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c100500 0x7fc03c1976d0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fc03000e9a0 tx=0x7fc03000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.594+0000 7fc041e10700 1 -- 172.21.15.46:0/2568085194 shutdown_connections 2024-03-18T20:45:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.594+0000 7fc041e10700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0240a1290 0x7fc0240a3750 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.594+0000 7fc041e10700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc03c105c00 0x7fc03c19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.594+0000 7fc041e10700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc03c0ffb40 0x7fc03c197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.594+0000 7fc041e10700 1 --2- 172.21.15.46:0/2568085194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c100500 0x7fc03c1976d0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.594+0000 7fc041e10700 1 -- 172.21.15.46:0/2568085194 >> 172.21.15.46:0/2568085194 conn(0x7fc03c0fb5f0 msgr2=0x7fc03c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.594+0000 7fc041e10700 1 -- 172.21.15.46:0/2568085194 shutdown_connections 2024-03-18T20:45:18.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.594+0000 7fc041e10700 1 -- 172.21.15.46:0/2568085194 wait complete. 2024-03-18T20:45:18.598 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 47 2024-03-18T20:45:18.611 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:45:18.611 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:45:18.709 INFO:tasks.workunit.client.0.smithi046.stdout: 1 335120 3.65 MB/sec execute 373 sec latency 1019.731 ms 2024-03-18T20:45:18.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.747+0000 7f37ae8a8700 1 -- 172.21.15.46:0/313771038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a8101900 msgr2=0x7f37a8105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.747+0000 7f37ae8a8700 1 --2- 172.21.15.46:0/313771038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a8101900 0x7f37a8105940 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f379c009990 tx=0x7f379c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.747+0000 7f37ae8a8700 1 -- 172.21.15.46:0/313771038 shutdown_connections 2024-03-18T20:45:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.747+0000 7f37ae8a8700 1 --2- 172.21.15.46:0/313771038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37a8106070 0x7f37a810dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.747+0000 7f37ae8a8700 1 --2- 172.21.15.46:0/313771038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a8100f50 0x7f37a8101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.747+0000 7f37ae8a8700 1 --2- 172.21.15.46:0/313771038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a8101900 0x7f37a8105940 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.747+0000 7f37ae8a8700 1 -- 172.21.15.46:0/313771038 >> 172.21.15.46:0/313771038 conn(0x7f37a80fc7f0 msgr2=0x7f37a80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.747+0000 7f37ae8a8700 1 -- 172.21.15.46:0/313771038 shutdown_connections 2024-03-18T20:45:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.747+0000 7f37ae8a8700 1 -- 172.21.15.46:0/313771038 wait complete. 2024-03-18T20:45:18.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.748+0000 7f37ae8a8700 1 Processor -- start 2024-03-18T20:45:18.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.748+0000 7f37ae8a8700 1 -- start start 2024-03-18T20:45:18.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37ae8a8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a8100f50 0x7f37a8197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37ae8a8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a8106070 0x7f37a8197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37ae8a8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37a819ba90 0x7f37a819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37ae8a8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a819c510 con 0x7f37a8106070 2024-03-18T20:45:18.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37ae8a8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a819c680 con 0x7f37a819ba90 2024-03-18T20:45:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37ae8a8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a819c840 con 0x7f37a8100f50 2024-03-18T20:45:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a8106070 0x7f37a8197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a8106070 0x7f37a8197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38168/0 (socket says 172.21.15.46:38168) 2024-03-18T20:45:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37a77fe700 1 -- 172.21.15.46:0/3189943953 learned_addr learned my addr 172.21.15.46:0/3189943953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37a7fff700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a8100f50 0x7f37a8197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37ace45700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37a819ba90 0x7f37a819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37a77fe700 1 -- 172.21.15.46:0/3189943953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a8100f50 msgr2=0x7f37a8197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37a77fe700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a8100f50 0x7f37a8197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37a77fe700 1 -- 172.21.15.46:0/3189943953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37a819ba90 msgr2=0x7f37a819bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37a77fe700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37a819ba90 0x7f37a819bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37a77fe700 1 -- 172.21.15.46:0/3189943953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c0095f0 con 0x7f37a8106070 2024-03-18T20:45:18.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.749+0000 7f37ace45700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37a819ba90 0x7f37a819bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:18.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.750+0000 7f37a77fe700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a8106070 0x7f37a8197740 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f379c005770 tx=0x7f379c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.750+0000 7f37a57fa700 1 -- 172.21.15.46:0/3189943953 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c01d070 con 0x7f37a8106070 2024-03-18T20:45:18.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.750+0000 7f37a57fa700 1 -- 172.21.15.46:0/3189943953 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f379c021410 con 0x7f37a8106070 2024-03-18T20:45:18.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.750+0000 7f37ae8a8700 1 -- 172.21.15.46:0/3189943953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a81a3d40 con 0x7f37a8106070 2024-03-18T20:45:18.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.750+0000 7f37a57fa700 1 -- 172.21.15.46:0/3189943953 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c017650 con 0x7f37a8106070 2024-03-18T20:45:18.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.750+0000 7f37ae8a8700 1 -- 172.21.15.46:0/3189943953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a81a41c0 con 0x7f37a8106070 2024-03-18T20:45:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.752+0000 7f37a57fa700 1 -- 172.21.15.46:0/3189943953 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f379c00fb40 con 0x7f37a8106070 2024-03-18T20:45:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.753+0000 7f37a57fa700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37900a1310 0x7f37900a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.753+0000 7f37a57fa700 1 -- 172.21.15.46:0/3189943953 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f379c0cde60 con 0x7f37a8106070 2024-03-18T20:45:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.753+0000 7f37a7fff700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37900a1310 0x7f37900a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.753+0000 7f37a7fff700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37900a1310 0x7f37900a37d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3798009c60 tx=0x7f3798009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37baa3db80 1 Processor -- start 2024-03-18T20:45:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37baa3db80 1 -- start start 2024-03-18T20:45:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37baa3db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 0x55ae92c01e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37baa3db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c0cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37baa3db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c0f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37baa3db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae92b5b990 con 0x55ae92c02380 2024-03-18T20:45:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37baa3db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae92b59350 con 0x55ae92c0d220 2024-03-18T20:45:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37baa3db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae92b54c40 con 0x55ae92c01a60 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c0cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c0cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38170/0 (socket says 172.21.15.46:38170) 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37a77fe700 1 -- 172.21.15.46:0/2151494449 learned_addr learned my addr 172.21.15.46:0/2151494449 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37a7fff700 1 --2- 172.21.15.46:0/2151494449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 0x55ae92c01e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.754+0000 7f37ace45700 1 --2- 172.21.15.46:0/2151494449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c0f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.755+0000 7f37a77fe700 1 -- 172.21.15.46:0/2151494449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 msgr2=0x55ae92c01e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.755+0000 7f37a77fe700 1 --2- 172.21.15.46:0/2151494449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 0x55ae92c01e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.755+0000 7f37a77fe700 1 -- 172.21.15.46:0/2151494449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 msgr2=0x55ae92c0f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.755+0000 7f37a77fe700 1 --2- 172.21.15.46:0/2151494449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c0f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.755+0000 7f37a77fe700 1 -- 172.21.15.46:0/2151494449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae929d63b0 con 0x55ae92c02380 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.755+0000 7f37ace45700 1 --2- 172.21.15.46:0/2151494449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c0f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.755+0000 7f37a77fe700 1 --2- 172.21.15.46:0/2151494449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c0cce0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f379c00f740 tx=0x7f379c09ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561759725586d181 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.755+0000 7f378e7fc700 1 -- 172.21.15.46:0/2151494449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c09a8e0 con 0x55ae92c02380 2024-03-18T20:45:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.755+0000 7f378e7fc700 1 -- 172.21.15.46:0/2151494449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f379c09b680 con 0x55ae92c02380 2024-03-18T20:45:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.755+0000 7f378e7fc700 1 -- 172.21.15.46:0/2151494449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c021810 con 0x55ae92c02380 2024-03-18T20:45:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.756+0000 7f37baa3db80 1 -- 172.21.15.46:0/2151494449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 msgr2=0x55ae92c0cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.756+0000 7f37baa3db80 1 --2- 172.21.15.46:0/2151494449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c0cce0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f379c00f740 tx=0x7f379c09ad50 comp rx=0 tx=0).stop 2024-03-18T20:45:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.756+0000 7f37baa3db80 1 -- 172.21.15.46:0/2151494449 shutdown_connections 2024-03-18T20:45:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.756+0000 7f37baa3db80 1 --2- 172.21.15.46:0/2151494449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c0f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.756+0000 7f37baa3db80 1 --2- 172.21.15.46:0/2151494449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 0x55ae92c01e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.756+0000 7f37baa3db80 1 --2- 172.21.15.46:0/2151494449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c0cce0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.756+0000 7f37baa3db80 1 -- 172.21.15.46:0/2151494449 >> 172.21.15.46:0/2151494449 conn(0x55ae92b78220 msgr2=0x55ae92bfe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.756+0000 7f37baa3db80 1 -- 172.21.15.46:0/2151494449 shutdown_connections 2024-03-18T20:45:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.756+0000 7f37baa3db80 1 -- 172.21.15.46:0/2151494449 wait complete. 2024-03-18T20:45:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.756+0000 7f37baa3db80 1 -- start start 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.756+0000 7f37baa3db80 1 Processor -- start 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37baa3db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 0x55ae92c0d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37baa3db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c0dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37baa3db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c18680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37baa3db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae92af0d80 con 0x55ae92c02380 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37baa3db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae92af0a20 con 0x55ae92c0d220 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37baa3db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae929d86a0 con 0x55ae92c01a60 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37baa3db80 10 client.0 _ll_register_callbacks cb 0x55ae9295c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c0dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c0dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38172/0 (socket says 172.21.15.46:38172) 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37a77fe700 1 -- 172.21.15.46:0/3873713792 learned_addr learned my addr 172.21.15.46:0/3873713792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37a7fff700 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 0x55ae92c0d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37ace45700 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c18680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37a77fe700 1 -- 172.21.15.46:0/3873713792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 msgr2=0x55ae92c0d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37a77fe700 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 0x55ae92c0d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37a77fe700 1 -- 172.21.15.46:0/3873713792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 msgr2=0x55ae92c18680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37a77fe700 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c18680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37a77fe700 1 -- 172.21.15.46:0/3873713792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a81a41c0 con 0x55ae92c02380 2024-03-18T20:45:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37a77fe700 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c0dd30 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f379c0979f0 tx=0x7f379c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdffcfcd734b9db2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.757+0000 7f37877fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37877fe700 1 -- 172.21.15.46:0/3873713792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0b2070 con 0x55ae92c02380 2024-03-18T20:45:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37877fe700 1 -- 172.21.15.46:0/3873713792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f379c09af80 con 0x55ae92c02380 2024-03-18T20:45:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37baa3db80 1 -- 172.21.15.46:0/3873713792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae92bf5ad0 con 0x55ae92c02380 2024-03-18T20:45:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37877fe700 1 -- 172.21.15.46:0/3873713792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f379c09a8e0 con 0x55ae92c02380 2024-03-18T20:45:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37baa3db80 10 client.18735 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37baa3db80 1 -- 172.21.15.46:0/3873713792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55ae92982b60 con 0x55ae92c02380 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37877fe700 1 -- 172.21.15.46:0/3873713792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f379c07e080 con 0x55ae92c02380 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37877fe700 1 -- 172.21.15.46:0/3873713792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f379c09a470 con 0x55ae92c02380 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37baa3db80 10 client.18735 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37baa3db80 10 client.18735 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37baa3db80 20 client.18735 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37baa3db80 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ae92c1c610 0x55ae92c1ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37baa3db80 4 client.18735 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.758+0000 7f37baa3db80 1 -- 172.21.15.46:0/3873713792 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae92b4d700 con 0x55ae92c1c610 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.759+0000 7f37a7fff700 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ae92c1c610 0x55ae92c1ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.759+0000 7f37a7fff700 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ae92c1c610 0x55ae92c1ea00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=55dddb6a596a4af6 server_cookie=7bfe47a5762c2dc5 in_seq=0 out_seq=0 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.759+0000 7f37877fe700 10 client.18735 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37877fe700 1 -- 172.21.15.46:0/3873713792 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ae92b4d700 con 0x55ae92c1c610 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37877fe700 10 client.18735 handle_command_reply: tid=0 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37baa3db80 1 client.18735 shutdown 2024-03-18T20:45:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37baa3db80 1 -- 172.21.15.46:0/3873713792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 msgr2=0x55ae92c0dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37baa3db80 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c0dd30 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f379c0979f0 tx=0x7f379c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37baa3db80 1 -- 172.21.15.46:0/3873713792 shutdown_connections 2024-03-18T20:45:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37baa3db80 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ae92c1c610 0x55ae92c1ea00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37baa3db80 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c18680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37baa3db80 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 0x55ae92c0d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37baa3db80 1 --2- 172.21.15.46:0/3873713792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c0dd30 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37baa3db80 1 -- 172.21.15.46:0/3873713792 >> 172.21.15.46:0/3873713792 conn(0x55ae92b78220 msgr2=0x55ae92bfdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37877fe700 0 client.18735 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:45:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37baa3db80 1 -- 172.21.15.46:0/3873713792 shutdown_connections 2024-03-18T20:45:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37baa3db80 1 -- 172.21.15.46:0/3873713792 wait complete. 2024-03-18T20:45:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.760+0000 7f37baa3db80 20 client.18735 trim_cache size 0 max 16384 2024-03-18T20:45:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.774+0000 7f37baa3db80 1 Processor -- start 2024-03-18T20:45:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.774+0000 7f37baa3db80 1 -- start start 2024-03-18T20:45:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.774+0000 7f37baa3db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 0x55ae92c125e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.774+0000 7f37baa3db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.774+0000 7f37baa3db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c07df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.774+0000 7f37baa3db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae92bc8940 con 0x55ae92c02380 2024-03-18T20:45:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.774+0000 7f37baa3db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae92c0e350 con 0x55ae92c0d220 2024-03-18T20:45:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.774+0000 7f37baa3db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae92c03620 con 0x55ae92c01a60 2024-03-18T20:45:18.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.774+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.774+0000 7f37a77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38180/0 (socket says 172.21.15.46:38180) 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.774+0000 7f37a77fe700 1 -- 172.21.15.46:0/3497580763 learned_addr learned my addr 172.21.15.46:0/3497580763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.775+0000 7f37a7fff700 1 --2- 172.21.15.46:0/3497580763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 0x55ae92c125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.775+0000 7f37ace45700 1 --2- 172.21.15.46:0/3497580763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c07df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.775+0000 7f37a77fe700 1 -- 172.21.15.46:0/3497580763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 msgr2=0x55ae92c125e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.775+0000 7f37a77fe700 1 --2- 172.21.15.46:0/3497580763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 0x55ae92c125e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.775+0000 7f37a77fe700 1 -- 172.21.15.46:0/3497580763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 msgr2=0x55ae92c07df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.775+0000 7f37a77fe700 1 --2- 172.21.15.46:0/3497580763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c07df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.775+0000 7f37a77fe700 1 -- 172.21.15.46:0/3497580763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae92c235f0 con 0x55ae92c02380 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.775+0000 7f37ace45700 1 --2- 172.21.15.46:0/3497580763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c07df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.775+0000 7f37a77fe700 1 --2- 172.21.15.46:0/3497580763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c078b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f379c00f740 tx=0x7f379c099530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dd9aff1660ac76 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.775+0000 7f378effd700 1 -- 172.21.15.46:0/3497580763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c09a8e0 con 0x55ae92c02380 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.775+0000 7f378effd700 1 -- 172.21.15.46:0/3497580763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae92bf5ad0 con 0x55ae92c02380 2024-03-18T20:45:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.775+0000 7f378effd700 1 -- 172.21.15.46:0/3497580763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0a31a0 con 0x55ae92c02380 2024-03-18T20:45:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.776+0000 7f37baa3db80 1 -- 172.21.15.46:0/3497580763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 msgr2=0x55ae92c078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.776+0000 7f37baa3db80 1 --2- 172.21.15.46:0/3497580763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c078b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f379c00f740 tx=0x7f379c099530 comp rx=0 tx=0).stop 2024-03-18T20:45:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.776+0000 7f37baa3db80 1 -- 172.21.15.46:0/3497580763 shutdown_connections 2024-03-18T20:45:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.776+0000 7f37baa3db80 1 --2- 172.21.15.46:0/3497580763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c0d220 0x55ae92c07df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.776+0000 7f37baa3db80 1 --2- 172.21.15.46:0/3497580763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c01a60 0x55ae92c125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.776+0000 7f37baa3db80 1 --2- 172.21.15.46:0/3497580763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c02380 0x55ae92c078b0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.776+0000 7f37baa3db80 1 -- 172.21.15.46:0/3497580763 >> 172.21.15.46:0/3497580763 conn(0x55ae92b78220 msgr2=0x55ae92c06c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.776+0000 7f37baa3db80 1 -- 172.21.15.46:0/3497580763 shutdown_connections 2024-03-18T20:45:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.776+0000 7f37baa3db80 1 -- 172.21.15.46:0/3497580763 wait complete. 2024-03-18T20:45:18.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.776+0000 7f37baa3db80 1 -- start start 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.776+0000 7f37baa3db80 1 Processor -- start 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.776+0000 7f37baa3db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c01a60 0x55ae92c12650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37baa3db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c02380 0x55ae92c0fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37baa3db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c0d220 0x55ae92c0ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37baa3db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae92c106f0 con 0x55ae92c0d220 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37baa3db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae92c10860 con 0x55ae92c01a60 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37baa3db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae92c6f3d0 con 0x55ae92c02380 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37baa3db80 10 client.0 _ll_register_callbacks cb 0x7f37980096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37ace45700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c0d220 0x55ae92c0ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37a7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c01a60 0x55ae92c12650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37a7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c01a60 0x55ae92c12650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:36104/0 (socket says 172.21.15.46:36104) 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37a7fff700 1 -- 172.21.15.46:0/2807499574 learned_addr learned my addr 172.21.15.46:0/2807499574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37a77fe700 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c02380 0x55ae92c0fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37a77fe700 1 -- 172.21.15.46:0/2807499574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c01a60 msgr2=0x55ae92c12650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37a77fe700 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c01a60 0x55ae92c12650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37a77fe700 1 -- 172.21.15.46:0/2807499574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c0d220 msgr2=0x55ae92c0ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37a77fe700 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c0d220 0x55ae92c0ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37a77fe700 1 -- 172.21.15.46:0/2807499574 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae92982b60 con 0x55ae92c02380 2024-03-18T20:45:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37ace45700 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c0d220 0x55ae92c0ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37a7fff700 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c01a60 0x55ae92c12650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37a77fe700 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c02380 0x55ae92c0fa80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f379c021b90 tx=0x7f379c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f49b2ef09de4700f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.777+0000 7f37877fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:45:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37877fe700 1 -- 172.21.15.46:0/2807499574 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c09a8e0 con 0x55ae92c02380 2024-03-18T20:45:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37baa3db80 1 -- 172.21.15.46:0/2807499574 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f379c09b680 con 0x55ae92c02380 2024-03-18T20:45:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37877fe700 1 -- 172.21.15.46:0/2807499574 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f379c09edb0 con 0x55ae92c02380 2024-03-18T20:45:18.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37877fe700 1 -- 172.21.15.46:0/2807499574 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f379c0b2050 con 0x55ae92c02380 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37baa3db80 10 client.28393 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37baa3db80 1 -- 172.21.15.46:0/2807499574 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55ae92b1aea0 con 0x55ae92c02380 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37877fe700 1 -- 172.21.15.46:0/2807499574 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f379c09b680 con 0x55ae92c02380 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37877fe700 1 -- 172.21.15.46:0/2807499574 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f379c09a470 con 0x55ae92c02380 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37baa3db80 10 client.28393 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37baa3db80 10 client.28393 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37baa3db80 20 client.28393 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37baa3db80 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ae92c705c0 0x55ae92c729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37baa3db80 4 client.28393 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.778+0000 7f37baa3db80 1 -- 172.21.15.46:0/2807499574 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ae929a3cf0 con 0x55ae92c705c0 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37a7fff700 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ae92c705c0 0x55ae92c729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37877fe700 10 client.28393 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:45:18.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37a7fff700 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ae92c705c0 0x55ae92c729b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d785f2e758081afb server_cookie=4978401ec5937646 in_seq=0 out_seq=0 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37877fe700 1 -- 172.21.15.46:0/2807499574 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ae929a3cf0 con 0x55ae92c705c0 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37877fe700 10 client.28393 handle_command_reply: tid=0 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37baa3db80 1 client.28393 shutdown 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37baa3db80 1 -- 172.21.15.46:0/2807499574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c02380 msgr2=0x55ae92c0fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37baa3db80 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c02380 0x55ae92c0fa80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f379c021b90 tx=0x7f379c009310 comp rx=0 tx=0).stop 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37baa3db80 1 -- 172.21.15.46:0/2807499574 shutdown_connections 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37baa3db80 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55ae92c705c0 0x55ae92c729b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37baa3db80 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae92c0d220 0x55ae92c0ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37baa3db80 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae92c01a60 0x55ae92c12650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37877fe700 0 client.28393 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37baa3db80 1 --2- 172.21.15.46:0/2807499574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae92c02380 0x55ae92c0fa80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37baa3db80 1 -- 172.21.15.46:0/2807499574 >> 172.21.15.46:0/2807499574 conn(0x55ae92b78220 msgr2=0x55ae92c0d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37baa3db80 1 -- 172.21.15.46:0/2807499574 shutdown_connections 2024-03-18T20:45:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37baa3db80 1 -- 172.21.15.46:0/2807499574 wait complete. 2024-03-18T20:45:18.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.779+0000 7f37baa3db80 20 client.28393 trim_cache size 0 max 16384 2024-03-18T20:45:18.789 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:45:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.780+0000 7f37ae8a8700 1 -- 172.21.15.46:0/3189943953 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37900a1310 msgr2=0x7f37900a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.780+0000 7f37ae8a8700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37900a1310 0x7f37900a37d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3798009c60 tx=0x7f3798009260 comp rx=0 tx=0).stop 2024-03-18T20:45:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.780+0000 7f37ae8a8700 1 -- 172.21.15.46:0/3189943953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a8106070 msgr2=0x7f37a8197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.780+0000 7f37ae8a8700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a8106070 0x7f37a8197740 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f379c005770 tx=0x7f379c0057a0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.780+0000 7f37ae8a8700 1 -- 172.21.15.46:0/3189943953 shutdown_connections 2024-03-18T20:45:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.780+0000 7f37ae8a8700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37900a1310 0x7f37900a37d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.780+0000 7f37ae8a8700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37a819ba90 0x7f37a819bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.780+0000 7f37ae8a8700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37a8100f50 0x7f37a8197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.780+0000 7f37ae8a8700 1 --2- 172.21.15.46:0/3189943953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37a8106070 0x7f37a8197740 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.780+0000 7f37ae8a8700 1 -- 172.21.15.46:0/3189943953 >> 172.21.15.46:0/3189943953 conn(0x7f37a80fc7f0 msgr2=0x7f37a80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.780+0000 7f37ae8a8700 1 -- 172.21.15.46:0/3189943953 shutdown_connections 2024-03-18T20:45:18.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.780+0000 7f37ae8a8700 1 -- 172.21.15.46:0/3189943953 wait complete. 2024-03-18T20:45:18.797 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:45:18.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:45:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.936+0000 7f086303f700 1 -- 172.21.15.46:0/891457932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c0692b0 msgr2=0x7f085c069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.936+0000 7f086303f700 1 --2- 172.21.15.46:0/891457932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c0692b0 0x7f085c069690 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f084c009990 tx=0x7f084c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.936+0000 7f086303f700 1 -- 172.21.15.46:0/891457932 shutdown_connections 2024-03-18T20:45:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.936+0000 7f086303f700 1 --2- 172.21.15.46:0/891457932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c0692b0 0x7f085c069690 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.936+0000 7f086303f700 1 --2- 172.21.15.46:0/891457932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c10d750 0x7f085c10fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.936+0000 7f086303f700 1 --2- 172.21.15.46:0/891457932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f085c069bd0 0x7f085c10d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.936+0000 7f086303f700 1 -- 172.21.15.46:0/891457932 >> 172.21.15.46:0/891457932 conn(0x7f085c076d50 msgr2=0x7f085c077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.936+0000 7f086303f700 1 -- 172.21.15.46:0/891457932 shutdown_connections 2024-03-18T20:45:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.936+0000 7f086303f700 1 -- 172.21.15.46:0/891457932 wait complete. 2024-03-18T20:45:18.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.937+0000 7f086303f700 1 Processor -- start 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.937+0000 7f086303f700 1 -- start start 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.937+0000 7f086303f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f085c0692b0 0x7f085c1994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.937+0000 7f086303f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c069bd0 0x7f085c199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.937+0000 7f086303f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c10d750 0x7f085c19ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.937+0000 7f086303f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c19e410 con 0x7f085c10d750 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.937+0000 7f086303f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c19e580 con 0x7f085c069bd0 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.937+0000 7f086303f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c19e740 con 0x7f085c0692b0 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.937+0000 7f08615dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c10d750 0x7f085c19ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f08615dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c10d750 0x7f085c19ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38206/0 (socket says 172.21.15.46:38206) 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f08615dc700 1 -- 172.21.15.46:0/3431889732 learned_addr learned my addr 172.21.15.46:0/3431889732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f0860ddb700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f085c0692b0 0x7f085c1994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f085bfff700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c069bd0 0x7f085c199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f08615dc700 1 -- 172.21.15.46:0/3431889732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f085c0692b0 msgr2=0x7f085c1994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f08615dc700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f085c0692b0 0x7f085c1994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f08615dc700 1 -- 172.21.15.46:0/3431889732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c069bd0 msgr2=0x7f085c199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f08615dc700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c069bd0 0x7f085c199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f08615dc700 1 -- 172.21.15.46:0/3431889732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0850009520 con 0x7f085c10d750 2024-03-18T20:45:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f08615dc700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c10d750 0x7f085c19ddd0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f0854007ec0 tx=0x7f085400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f0859ffb700 1 -- 172.21.15.46:0/3431889732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854017070 con 0x7f085c10d750 2024-03-18T20:45:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f0859ffb700 1 -- 172.21.15.46:0/3431889732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f085400f040 con 0x7f085c10d750 2024-03-18T20:45:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f086303f700 1 -- 172.21.15.46:0/3431889732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084c0095f0 con 0x7f085c10d750 2024-03-18T20:45:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f0859ffb700 1 -- 172.21.15.46:0/3431889732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854004020 con 0x7f085c10d750 2024-03-18T20:45:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.938+0000 7f086303f700 1 -- 172.21.15.46:0/3431889732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c1a6080 con 0x7f085c10d750 2024-03-18T20:45:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.941+0000 7f0859ffb700 1 -- 172.21.15.46:0/3431889732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f085400db20 con 0x7f085c10d750 2024-03-18T20:45:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.941+0000 7f0859ffb700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f08440a1290 0x7f08440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.941+0000 7f0859ffb700 1 -- 172.21.15.46:0/3431889732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f08540cc0e0 con 0x7f085c10d750 2024-03-18T20:45:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.941+0000 7f0860ddb700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f08440a1290 0x7f08440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.942+0000 7f0860ddb700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f08440a1290 0x7f08440a3750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f084c006010 tx=0x7f084c00b260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.942+0000 7f086f1d4b80 1 Processor -- start 2024-03-18T20:45:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f086f1d4b80 1 -- start start 2024-03-18T20:45:18.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f086f1d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8bee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f086f1d4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8bf380 0x555bfc8c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f086f1d4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8ca220 0x555bfc8cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f086f1d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc818990 con 0x555bfc8bea60 2024-03-18T20:45:18.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f086f1d4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc816350 con 0x555bfc8bf380 2024-03-18T20:45:18.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f086f1d4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc811c40 con 0x555bfc8ca220 2024-03-18T20:45:18.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f0860ddb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8bee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f0860ddb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8bee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38218/0 (socket says 172.21.15.46:38218) 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f0860ddb700 1 -- 172.21.15.46:0/2335722255 learned_addr learned my addr 172.21.15.46:0/2335722255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f085bfff700 1 --2- 172.21.15.46:0/2335722255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8bf380 0x555bfc8c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f08615dc700 1 --2- 172.21.15.46:0/2335722255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8ca220 0x555bfc8cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f0860ddb700 1 -- 172.21.15.46:0/2335722255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8ca220 msgr2=0x555bfc8cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f0860ddb700 1 --2- 172.21.15.46:0/2335722255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8ca220 0x555bfc8cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f0860ddb700 1 -- 172.21.15.46:0/2335722255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8bf380 msgr2=0x555bfc8c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f0860ddb700 1 --2- 172.21.15.46:0/2335722255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8bf380 0x555bfc8c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f0860ddb700 1 -- 172.21.15.46:0/2335722255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bfc6933b0 con 0x555bfc8bea60 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.943+0000 7f08615dc700 1 --2- 172.21.15.46:0/2335722255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8ca220 0x555bfc8cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.944+0000 7f0860ddb700 1 --2- 172.21.15.46:0/2335722255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8bee40 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f084c015000 tx=0x7f084c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd138170e3058a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.944+0000 7f0842ffd700 1 -- 172.21.15.46:0/2335722255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c004700 con 0x555bfc8bea60 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.944+0000 7f0842ffd700 1 -- 172.21.15.46:0/2335722255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f084c01e040 con 0x555bfc8bea60 2024-03-18T20:45:18.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.944+0000 7f0842ffd700 1 -- 172.21.15.46:0/2335722255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c00f900 con 0x555bfc8bea60 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.945+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2335722255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 msgr2=0x555bfc8bee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.945+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2335722255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8bee40 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f084c015000 tx=0x7f084c020000 comp rx=0 tx=0).stop 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.945+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2335722255 shutdown_connections 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.945+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2335722255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8bee40 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.945+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2335722255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8ca220 0x555bfc8cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.945+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2335722255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8bf380 0x555bfc8c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.945+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2335722255 >> 172.21.15.46:0/2335722255 conn(0x555bfc835220 msgr2=0x555bfc8bb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.945+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2335722255 shutdown_connections 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.945+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2335722255 wait complete. 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.945+0000 7f086f1d4b80 1 -- start start 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.945+0000 7f086f1d4b80 1 Processor -- start 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f086f1d4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bea60 0x555bfc8c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f086f1d4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8bf380 0x555bfc8c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f086f1d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8ca220 0x555bfc8c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f086f1d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc7add80 con 0x555bfc8ca220 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f086f1d4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc7ada20 con 0x555bfc8bf380 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f086f1d4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc6956a0 con 0x555bfc8bea60 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f086f1d4b80 10 client.0 _ll_register_callbacks cb 0x555bfc6193a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f08615dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8ca220 0x555bfc8c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f08615dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8ca220 0x555bfc8c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38222/0 (socket says 172.21.15.46:38222) 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f08615dc700 1 -- 172.21.15.46:0/2506035462 learned_addr learned my addr 172.21.15.46:0/2506035462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f085bfff700 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8bf380 0x555bfc8c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f0860ddb700 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bea60 0x555bfc8c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f08615dc700 1 -- 172.21.15.46:0/2506035462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bea60 msgr2=0x555bfc8c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f08615dc700 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bea60 0x555bfc8c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f08615dc700 1 -- 172.21.15.46:0/2506035462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8bf380 msgr2=0x555bfc8c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f08615dc700 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8bf380 0x555bfc8c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.946+0000 7f08615dc700 1 -- 172.21.15.46:0/2506035462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c110670 con 0x555bfc8ca220 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f08615dc700 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8ca220 0x555bfc8c7550 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f08540c9930 tx=0x7f0854019340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdacdfbe6a7a8c07 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f083bfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f083bfff700 1 -- 172.21.15.46:0/2506035462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085409d130 con 0x555bfc8ca220 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f083bfff700 1 -- 172.21.15.46:0/2506035462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f08540a16f0 con 0x555bfc8ca220 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2506035462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bfc8b2ad0 con 0x555bfc8ca220 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f083bfff700 1 -- 172.21.15.46:0/2506035462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0854004d40 con 0x555bfc8ca220 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f086f1d4b80 10 client.18753 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2506035462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x555bfc63fb60 con 0x555bfc8ca220 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f083bfff700 1 -- 172.21.15.46:0/2506035462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555bfc8b2ad0 con 0x555bfc8ca220 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f083bfff700 1 -- 172.21.15.46:0/2506035462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0854098880 con 0x555bfc8ca220 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f086f1d4b80 10 client.18753 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f086f1d4b80 10 client.18753 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f086f1d4b80 20 client.18753 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:18.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555bfc8d9e00 0x555bfc8dc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f086f1d4b80 4 client.18753 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.947+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2506035462 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bfc80a700 con 0x555bfc8d9e00 2024-03-18T20:45:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.948+0000 7f0860ddb700 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555bfc8d9e00 0x555bfc8dc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.948+0000 7f0860ddb700 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555bfc8d9e00 0x555bfc8dc220 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3e24472986e79189 server_cookie=dc426af1a7f03260 in_seq=0 out_seq=0 2024-03-18T20:45:18.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.948+0000 7f083bfff700 10 client.18753 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:45:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f083bfff700 1 -- 172.21.15.46:0/2506035462 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555bfc80a700 con 0x555bfc8d9e00 2024-03-18T20:45:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f083bfff700 10 client.18753 handle_command_reply: tid=0 2024-03-18T20:45:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f086f1d4b80 1 client.18753 shutdown 2024-03-18T20:45:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2506035462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8ca220 msgr2=0x555bfc8c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8ca220 0x555bfc8c7550 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f08540c9930 tx=0x7f0854019340 comp rx=0 tx=0).stop 2024-03-18T20:45:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2506035462 shutdown_connections 2024-03-18T20:45:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555bfc8d9e00 0x555bfc8dc220 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8ca220 0x555bfc8c7550 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bea60 0x555bfc8c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f083bfff700 0 client.18753 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:45:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2506035462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8bf380 0x555bfc8c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2506035462 >> 172.21.15.46:0/2506035462 conn(0x555bfc835220 msgr2=0x555bfc8bac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2506035462 shutdown_connections 2024-03-18T20:45:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2506035462 wait complete. 2024-03-18T20:45:18.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.949+0000 7f086f1d4b80 20 client.18753 trim_cache size 0 max 16384 2024-03-18T20:45:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.963+0000 7f086f1d4b80 1 Processor -- start 2024-03-18T20:45:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.963+0000 7f086f1d4b80 1 -- start start 2024-03-18T20:45:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.963+0000 7f086f1d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8c7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.963+0000 7f086f1d4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bf380 0x555bfc8d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.963+0000 7f086f1d4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8ca220 0x555bfc8cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.963+0000 7f086f1d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc8b9dd0 con 0x555bfc8bea60 2024-03-18T20:45:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.963+0000 7f086f1d4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc8b4f80 con 0x555bfc8ca220 2024-03-18T20:45:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.963+0000 7f086f1d4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc8d1350 con 0x555bfc8bf380 2024-03-18T20:45:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.963+0000 7f0860ddb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.963+0000 7f0860ddb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38226/0 (socket says 172.21.15.46:38226) 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.963+0000 7f0860ddb700 1 -- 172.21.15.46:0/2449496583 learned_addr learned my addr 172.21.15.46:0/2449496583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f08615dc700 1 --2- 172.21.15.46:0/2449496583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8ca220 0x555bfc8cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f085bfff700 1 --2- 172.21.15.46:0/2449496583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bf380 0x555bfc8d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f0860ddb700 1 -- 172.21.15.46:0/2449496583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bf380 msgr2=0x555bfc8d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f0860ddb700 1 --2- 172.21.15.46:0/2449496583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bf380 0x555bfc8d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f0860ddb700 1 -- 172.21.15.46:0/2449496583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8ca220 msgr2=0x555bfc8cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f0860ddb700 1 --2- 172.21.15.46:0/2449496583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8ca220 0x555bfc8cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f0860ddb700 1 -- 172.21.15.46:0/2449496583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bfc7d7ea0 con 0x555bfc8bea60 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f08615dc700 1 --2- 172.21.15.46:0/2449496583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8ca220 0x555bfc8cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f085bfff700 1 --2- 172.21.15.46:0/2449496583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bf380 0x555bfc8d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f0860ddb700 1 --2- 172.21.15.46:0/2449496583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8c7440 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f084c00b6f0 tx=0x7f084c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c0daf297cd32f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f08437fe700 1 -- 172.21.15.46:0/2449496583 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c03dd30 con 0x555bfc8bea60 2024-03-18T20:45:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f08437fe700 1 -- 172.21.15.46:0/2449496583 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f084c03de90 con 0x555bfc8bea60 2024-03-18T20:45:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.964+0000 7f08437fe700 1 -- 172.21.15.46:0/2449496583 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c00fc70 con 0x555bfc8bea60 2024-03-18T20:45:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.965+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2449496583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 msgr2=0x555bfc8c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.965+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2449496583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8c7440 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f084c00b6f0 tx=0x7f084c01a040 comp rx=0 tx=0).stop 2024-03-18T20:45:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.965+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2449496583 shutdown_connections 2024-03-18T20:45:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.965+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2449496583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8c7440 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.965+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2449496583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8ca220 0x555bfc8cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.965+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/2449496583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bf380 0x555bfc8d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.965+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2449496583 >> 172.21.15.46:0/2449496583 conn(0x555bfc835220 msgr2=0x555bfc8c57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.965+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2449496583 shutdown_connections 2024-03-18T20:45:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.965+0000 7f086f1d4b80 1 -- 172.21.15.46:0/2449496583 wait complete. 2024-03-18T20:45:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.965+0000 7f086f1d4b80 1 -- start start 2024-03-18T20:45:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.965+0000 7f086f1d4b80 1 Processor -- start 2024-03-18T20:45:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f086f1d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f086f1d4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bf380 0x555bfc8c4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f086f1d4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8ca220 0x555bfc8c3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f086f1d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc8c36a0 con 0x555bfc8bea60 2024-03-18T20:45:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f086f1d4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc8c3810 con 0x555bfc8ca220 2024-03-18T20:45:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f086f1d4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bfc8c3980 con 0x555bfc8bf380 2024-03-18T20:45:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f086f1d4b80 10 client.0 _ll_register_callbacks cb 0x7f0854098e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f0860ddb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f0860ddb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8c42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38234/0 (socket says 172.21.15.46:38234) 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f0860ddb700 1 -- 172.21.15.46:0/3176704097 learned_addr learned my addr 172.21.15.46:0/3176704097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f085bfff700 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bf380 0x555bfc8c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f08615dc700 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8ca220 0x555bfc8c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f0860ddb700 1 -- 172.21.15.46:0/3176704097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bf380 msgr2=0x555bfc8c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f0860ddb700 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bf380 0x555bfc8c4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f0860ddb700 1 -- 172.21.15.46:0/3176704097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8ca220 msgr2=0x555bfc8c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f0860ddb700 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8ca220 0x555bfc8c3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f0860ddb700 1 -- 172.21.15.46:0/3176704097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bfc6933b0 con 0x555bfc8bea60 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f085bfff700 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bf380 0x555bfc8c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.966+0000 7f08615dc700 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8ca220 0x555bfc8c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f0860ddb700 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8c42c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f084c005850 tx=0x7f084c03dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30969c3c9d7cd420 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f083bfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f083bfff700 1 -- 172.21.15.46:0/3176704097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c00f570 con 0x555bfc8bea60 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f083bfff700 1 -- 172.21.15.46:0/3176704097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f084c00f6d0 con 0x555bfc8bea60 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f086f1d4b80 1 -- 172.21.15.46:0/3176704097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f084c01e040 con 0x555bfc8bea60 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f083bfff700 1 -- 172.21.15.46:0/3176704097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f084c00f890 con 0x555bfc8bea60 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f086f1d4b80 10 client.18759 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f086f1d4b80 1 -- 172.21.15.46:0/3176704097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x555bfc697cd0 con 0x555bfc8bea60 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f083bfff700 1 -- 172.21.15.46:0/3176704097 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f084c01e040 con 0x555bfc8bea60 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f083bfff700 1 -- 172.21.15.46:0/3176704097 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f084c045020 con 0x555bfc8bea60 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f086f1d4b80 10 client.18759 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f086f1d4b80 10 client.18759 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f086f1d4b80 20 client.18759 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555bfc92e680 0x555bfc930aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f086f1d4b80 4 client.18759 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:45:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.967+0000 7f086f1d4b80 1 -- 172.21.15.46:0/3176704097 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555bfc660cf0 con 0x555bfc92e680 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f085bfff700 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555bfc92e680 0x555bfc930aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f085bfff700 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555bfc92e680 0x555bfc930aa0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dfe34327831257b5 server_cookie=91297ff8b27d87c6 in_seq=0 out_seq=0 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f083bfff700 10 client.18759 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f083bfff700 1 -- 172.21.15.46:0/3176704097 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555bfc660cf0 con 0x555bfc92e680 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f083bfff700 10 client.18759 handle_command_reply: tid=0 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f086f1d4b80 1 client.18759 shutdown 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f086f1d4b80 1 -- 172.21.15.46:0/3176704097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 msgr2=0x555bfc8c42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8c42c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f084c005850 tx=0x7f084c03dc60 comp rx=0 tx=0).stop 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f086f1d4b80 1 -- 172.21.15.46:0/3176704097 shutdown_connections 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555bfc92e680 0x555bfc930aa0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bfc8bea60 0x555bfc8c42c0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f083bfff700 0 client.18759 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bfc8ca220 0x555bfc8c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f086f1d4b80 1 --2- 172.21.15.46:0/3176704097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555bfc8bf380 0x555bfc8c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f086f1d4b80 1 -- 172.21.15.46:0/3176704097 >> 172.21.15.46:0/3176704097 conn(0x555bfc835220 msgr2=0x555bfc8c8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f086f1d4b80 1 -- 172.21.15.46:0/3176704097 shutdown_connections 2024-03-18T20:45:18.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.968+0000 7f086f1d4b80 1 -- 172.21.15.46:0/3176704097 wait complete. 2024-03-18T20:45:18.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086f1d4b80 20 client.18759 trim_cache size 0 max 16384 2024-03-18T20:45:18.978 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:45:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086303f700 1 -- 172.21.15.46:0/3431889732 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f08440a1290 msgr2=0x7f08440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086303f700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f08440a1290 0x7f08440a3750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f084c006010 tx=0x7f084c00b260 comp rx=0 tx=0).stop 2024-03-18T20:45:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086303f700 1 -- 172.21.15.46:0/3431889732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c10d750 msgr2=0x7f085c19ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086303f700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c10d750 0x7f085c19ddd0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f0854007ec0 tx=0x7f085400d3b0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086303f700 1 -- 172.21.15.46:0/3431889732 shutdown_connections 2024-03-18T20:45:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086303f700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f08440a1290 0x7f08440a3750 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086303f700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c10d750 0x7f085c19ddd0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086303f700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f085c0692b0 0x7f085c1994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086303f700 1 --2- 172.21.15.46:0/3431889732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c069bd0 0x7f085c199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086303f700 1 -- 172.21.15.46:0/3431889732 >> 172.21.15.46:0/3431889732 conn(0x7f085c076d50 msgr2=0x7f085c10ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086303f700 1 -- 172.21.15.46:0/3431889732 shutdown_connections 2024-03-18T20:45:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:18.969+0000 7f086303f700 1 -- 172.21.15.46:0/3431889732 wait complete. 2024-03-18T20:45:18.986 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:45:18.986 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:45:19.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.123+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/4027101568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd74100800 msgr2=0x7fdd74104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.123+0000 7fdd7aa7b700 1 --2- 172.21.15.46:0/4027101568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd74100800 0x7fdd74104830 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009990 tx=0x7fdd60009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.123+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/4027101568 shutdown_connections 2024-03-18T20:45:19.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.123+0000 7fdd7aa7b700 1 --2- 172.21.15.46:0/4027101568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd74104f60 0x7fdd7410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.123+0000 7fdd7aa7b700 1 --2- 172.21.15.46:0/4027101568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd740ffe50 0x7fdd74100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.123+0000 7fdd7aa7b700 1 --2- 172.21.15.46:0/4027101568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd74100800 0x7fdd74104830 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.123+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/4027101568 >> 172.21.15.46:0/4027101568 conn(0x7fdd74076d40 msgr2=0x7fdd74077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.123+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/4027101568 shutdown_connections 2024-03-18T20:45:19.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.123+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/4027101568 wait complete. 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd7aa7b700 1 Processor -- start 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd7aa7b700 1 -- start start 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd7aa7b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740ffe50 0x7fdd74197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd7aa7b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd74104f60 0x7fdd74197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd7aa7b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7419bb10 0x7fdd7419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd7aa7b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7419c590 con 0x7fdd740ffe50 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd7aa7b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7419c700 con 0x7fdd7419bb10 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd7aa7b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7419c8c0 con 0x7fdd74104f60 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd78817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740ffe50 0x7fdd74197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd78817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740ffe50 0x7fdd74197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38264/0 (socket says 172.21.15.46:38264) 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd78817700 1 -- 172.21.15.46:0/1623327845 learned_addr learned my addr 172.21.15.46:0/1623327845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd79018700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7419bb10 0x7fdd7419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd73fff700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd74104f60 0x7fdd74197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd78817700 1 -- 172.21.15.46:0/1623327845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd74104f60 msgr2=0x7fdd74197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd78817700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd74104f60 0x7fdd74197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd78817700 1 -- 172.21.15.46:0/1623327845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7419bb10 msgr2=0x7fdd7419bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd78817700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7419bb10 0x7fdd7419bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd78817700 1 -- 172.21.15.46:0/1623327845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600095f0 con 0x7fdd740ffe50 2024-03-18T20:45:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.124+0000 7fdd79018700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7419bb10 0x7fdd7419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.125+0000 7fdd78817700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740ffe50 0x7fdd74197230 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800e9a0 tx=0x7fdd6800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.125+0000 7fdd71ffb700 1 -- 172.21.15.46:0/1623327845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68009790 con 0x7fdd740ffe50 2024-03-18T20:45:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.125+0000 7fdd71ffb700 1 -- 172.21.15.46:0/1623327845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd6800cbd0 con 0x7fdd740ffe50 2024-03-18T20:45:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.125+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/1623327845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd741a3e20 con 0x7fdd740ffe50 2024-03-18T20:45:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.125+0000 7fdd71ffb700 1 -- 172.21.15.46:0/1623327845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68018680 con 0x7fdd740ffe50 2024-03-18T20:45:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.125+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/1623327845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd741a42f0 con 0x7fdd740ffe50 2024-03-18T20:45:19.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.127+0000 7fdd71ffb700 1 -- 172.21.15.46:0/1623327845 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fdd6800cd40 con 0x7fdd740ffe50 2024-03-18T20:45:19.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.128+0000 7fdd71ffb700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdd640a1310 0x7fdd640a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.128+0000 7fdd71ffb700 1 -- 172.21.15.46:0/1623327845 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fdd68014070 con 0x7fdd740ffe50 2024-03-18T20:45:19.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.128+0000 7fdd73fff700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdd640a1310 0x7fdd640a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd73fff700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdd640a1310 0x7fdd640a37d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000b2e0 tx=0x7fdd60005fd0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd86c10b80 1 Processor -- start 2024-03-18T20:45:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd86c10b80 1 -- start start 2024-03-18T20:45:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd86c10b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 0x557239906e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd86c10b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239907380 0x557239911ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd86c10b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239912220 0x557239914610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd86c10b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239860990 con 0x557239906a60 2024-03-18T20:45:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd86c10b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723985e350 con 0x557239907380 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd86c10b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239859c40 con 0x557239912220 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd78817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 0x557239906e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd78817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 0x557239906e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38276/0 (socket says 172.21.15.46:38276) 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd78817700 1 -- 172.21.15.46:0/1235556495 learned_addr learned my addr 172.21.15.46:0/1235556495 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd79018700 1 --2- 172.21.15.46:0/1235556495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239912220 0x557239914610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.129+0000 7fdd73fff700 1 --2- 172.21.15.46:0/1235556495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239907380 0x557239911ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.130+0000 7fdd78817700 1 -- 172.21.15.46:0/1235556495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239912220 msgr2=0x557239914610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.130+0000 7fdd78817700 1 --2- 172.21.15.46:0/1235556495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239912220 0x557239914610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.130+0000 7fdd78817700 1 -- 172.21.15.46:0/1235556495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239907380 msgr2=0x557239911ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.130+0000 7fdd78817700 1 --2- 172.21.15.46:0/1235556495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239907380 0x557239911ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.130+0000 7fdd78817700 1 -- 172.21.15.46:0/1235556495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572396db3b0 con 0x557239906a60 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.130+0000 7fdd78817700 1 --2- 172.21.15.46:0/1235556495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 0x557239906e40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fdd68096b00 tx=0x7fdd68099b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3206dae7f8044cd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.130+0000 7fdd5effd700 1 -- 172.21.15.46:0/1235556495 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680a2510 con 0x557239906a60 2024-03-18T20:45:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.130+0000 7fdd5effd700 1 -- 172.21.15.46:0/1235556495 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd6809a2f0 con 0x557239906a60 2024-03-18T20:45:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.130+0000 7fdd5effd700 1 -- 172.21.15.46:0/1235556495 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680a22c0 con 0x557239906a60 2024-03-18T20:45:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.131+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1235556495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 msgr2=0x557239906e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.131+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/1235556495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 0x557239906e40 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fdd68096b00 tx=0x7fdd68099b00 comp rx=0 tx=0).stop 2024-03-18T20:45:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.131+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1235556495 shutdown_connections 2024-03-18T20:45:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.131+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/1235556495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 0x557239906e40 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.131+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/1235556495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239912220 0x557239914610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.131+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/1235556495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239907380 0x557239911ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.131+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1235556495 >> 172.21.15.46:0/1235556495 conn(0x55723987d220 msgr2=0x557239903020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.131+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1235556495 shutdown_connections 2024-03-18T20:45:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.131+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1235556495 wait complete. 2024-03-18T20:45:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.131+0000 7fdd86c10b80 1 -- start start 2024-03-18T20:45:19.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.131+0000 7fdd86c10b80 1 Processor -- start 2024-03-18T20:45:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.132+0000 7fdd86c10b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 0x55723990f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.132+0000 7fdd86c10b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239907380 0x55723990f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.132+0000 7fdd86c10b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239912220 0x55723990fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.132+0000 7fdd86c10b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572397f5d80 con 0x557239906a60 2024-03-18T20:45:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.132+0000 7fdd86c10b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572397f5a20 con 0x557239912220 2024-03-18T20:45:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.132+0000 7fdd86c10b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572396dd6a0 con 0x557239907380 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.132+0000 7fdd86c10b80 10 client.0 _ll_register_callbacks cb 0x5572396613a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.132+0000 7fdd78817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 0x55723990f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.132+0000 7fdd78817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 0x55723990f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38292/0 (socket says 172.21.15.46:38292) 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.132+0000 7fdd78817700 1 -- 172.21.15.46:0/1490768260 learned_addr learned my addr 172.21.15.46:0/1490768260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.132+0000 7fdd79018700 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239912220 0x55723990fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.132+0000 7fdd73fff700 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239907380 0x55723990f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd78817700 1 -- 172.21.15.46:0/1490768260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239907380 msgr2=0x55723990f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd78817700 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239907380 0x55723990f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd78817700 1 -- 172.21.15.46:0/1490768260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239912220 msgr2=0x55723990fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd78817700 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239912220 0x55723990fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd78817700 1 -- 172.21.15.46:0/1490768260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd741a42f0 con 0x557239906a60 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd78817700 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 0x55723990f0b0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fdd68099fd0 tx=0x7fdd6800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585f9932770a4cf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd53fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd53fff700 1 -- 172.21.15.46:0/1490768260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680a2510 con 0x557239906a60 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd53fff700 1 -- 172.21.15.46:0/1490768260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fdd68098430 con 0x557239906a60 2024-03-18T20:45:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1490768260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572398faad0 con 0x557239906a60 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd53fff700 1 -- 172.21.15.46:0/1490768260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd680a2290 con 0x557239906a60 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd86c10b80 10 client.18771 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1490768260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x557239687b60 con 0x557239906a60 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd53fff700 1 -- 172.21.15.46:0/1490768260 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5572398faad0 con 0x557239906a60 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.133+0000 7fdd53fff700 1 -- 172.21.15.46:0/1490768260 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fdd6809ede0 con 0x557239906a60 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.134+0000 7fdd86c10b80 10 client.18771 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.134+0000 7fdd86c10b80 10 client.18771 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.134+0000 7fdd86c10b80 20 client.18771 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.134+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557239921a30 0x557239923e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.134+0000 7fdd86c10b80 4 client.18771 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.134+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1490768260 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557239852700 con 0x557239921a30 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.134+0000 7fdd73fff700 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557239921a30 0x557239923e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.134+0000 7fdd53fff700 10 client.18771 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.134+0000 7fdd73fff700 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557239921a30 0x557239923e50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=590c2bd177f240ff server_cookie=dd72f0c9d4746c08 in_seq=0 out_seq=0 2024-03-18T20:45:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd53fff700 1 -- 172.21.15.46:0/1490768260 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557239852700 con 0x557239921a30 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd53fff700 10 client.18771 handle_command_reply: tid=0 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd86c10b80 1 client.18771 shutdown 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1490768260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 msgr2=0x55723990f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 0x55723990f0b0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fdd68099fd0 tx=0x7fdd6800c740 comp rx=0 tx=0).stop 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1490768260 shutdown_connections 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557239921a30 0x557239923e50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239906a60 0x55723990f0b0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239912220 0x55723990fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd53fff700 0 client.18771 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/1490768260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239907380 0x55723990f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1490768260 >> 172.21.15.46:0/1490768260 conn(0x55723987d220 msgr2=0x557239902c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1490768260 shutdown_connections 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd86c10b80 1 -- 172.21.15.46:0/1490768260 wait complete. 2024-03-18T20:45:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.135+0000 7fdd86c10b80 20 client.18771 trim_cache size 0 max 16384 2024-03-18T20:45:19.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd86c10b80 1 Processor -- start 2024-03-18T20:45:19.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd86c10b80 1 -- start start 2024-03-18T20:45:19.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd86c10b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239906a60 0x5572399099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd86c10b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 0x55723990f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd86c10b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239912220 0x55723990df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd86c10b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572398fd210 con 0x557239907380 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd86c10b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239910660 con 0x557239906a60 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd86c10b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572399085d0 con 0x557239912220 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd73fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 0x55723990f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd73fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 0x55723990f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38294/0 (socket says 172.21.15.46:38294) 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd73fff700 1 -- 172.21.15.46:0/92782225 learned_addr learned my addr 172.21.15.46:0/92782225 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd79018700 1 --2- 172.21.15.46:0/92782225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239912220 0x55723990df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.149+0000 7fdd78817700 1 --2- 172.21.15.46:0/92782225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239906a60 0x5572399099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.150+0000 7fdd73fff700 1 -- 172.21.15.46:0/92782225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239912220 msgr2=0x55723990df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.150+0000 7fdd73fff700 1 --2- 172.21.15.46:0/92782225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239912220 0x55723990df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.150+0000 7fdd73fff700 1 -- 172.21.15.46:0/92782225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239906a60 msgr2=0x5572399099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.150+0000 7fdd73fff700 1 --2- 172.21.15.46:0/92782225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239906a60 0x5572399099c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.150+0000 7fdd73fff700 1 -- 172.21.15.46:0/92782225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55723981fea0 con 0x557239907380 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.150+0000 7fdd78817700 1 --2- 172.21.15.46:0/92782225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239906a60 0x5572399099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.150+0000 7fdd73fff700 1 --2- 172.21.15.46:0/92782225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 0x55723990f950 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fdd7406a4d0 tx=0x7fdd6001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f091e7ebb3de5579 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.150+0000 7fdd5f7fe700 1 -- 172.21.15.46:0/92782225 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6002b040 con 0x557239907380 2024-03-18T20:45:19.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.150+0000 7fdd5f7fe700 1 -- 172.21.15.46:0/92782225 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd6002c040 con 0x557239907380 2024-03-18T20:45:19.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.150+0000 7fdd5f7fe700 1 -- 172.21.15.46:0/92782225 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60004810 con 0x557239907380 2024-03-18T20:45:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.151+0000 7fdd86c10b80 1 -- 172.21.15.46:0/92782225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 msgr2=0x55723990f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.151+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/92782225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 0x55723990f950 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fdd7406a4d0 tx=0x7fdd6001f000 comp rx=0 tx=0).stop 2024-03-18T20:45:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.151+0000 7fdd86c10b80 1 -- 172.21.15.46:0/92782225 shutdown_connections 2024-03-18T20:45:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.151+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/92782225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239912220 0x55723990df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.151+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/92782225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239906a60 0x5572399099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.151+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/92782225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 0x55723990f950 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.151+0000 7fdd86c10b80 1 -- 172.21.15.46:0/92782225 >> 172.21.15.46:0/92782225 conn(0x55723987d220 msgr2=0x55723990d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.151+0000 7fdd86c10b80 1 -- 172.21.15.46:0/92782225 shutdown_connections 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.151+0000 7fdd86c10b80 1 -- 172.21.15.46:0/92782225 wait complete. 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.151+0000 7fdd86c10b80 1 -- start start 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.151+0000 7fdd86c10b80 1 Processor -- start 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.151+0000 7fdd86c10b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239906a60 0x55723996eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd86c10b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 0x55723996f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd86c10b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239912220 0x557239916e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd86c10b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239917430 con 0x557239907380 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd86c10b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572399175a0 con 0x557239912220 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd86c10b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557239917760 con 0x557239906a60 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd86c10b80 10 client.0 _ll_register_callbacks cb 0x7fdd6c0091d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd79018700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239912220 0x557239916e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd79018700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239912220 0x557239916e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:36222/0 (socket says 172.21.15.46:36222) 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd79018700 1 -- 172.21.15.46:0/65215041 learned_addr learned my addr 172.21.15.46:0/65215041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd78817700 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239906a60 0x55723996eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd73fff700 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 0x55723996f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd79018700 1 -- 172.21.15.46:0/65215041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239906a60 msgr2=0x55723996eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd79018700 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239906a60 0x55723996eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd79018700 1 -- 172.21.15.46:0/65215041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 msgr2=0x55723996f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd79018700 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 0x55723996f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd79018700 1 -- 172.21.15.46:0/65215041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7410d350 con 0x557239912220 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd73fff700 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 0x55723996f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd79018700 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239912220 0x557239916e60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c018bc0 tx=0x7fdd6c018bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a12e6ddb59ab292 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.152+0000 7fdd53fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd53fff700 1 -- 172.21.15.46:0/65215041 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c007aa0 con 0x557239912220 2024-03-18T20:45:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd86c10b80 1 -- 172.21.15.46:0/65215041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd6809a2f0 con 0x557239912220 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd53fff700 1 -- 172.21.15.46:0/65215041 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fdd6c007c00 con 0x557239912220 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd53fff700 1 -- 172.21.15.46:0/65215041 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd6c007dc0 con 0x557239912220 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd86c10b80 10 client.28415 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd86c10b80 1 -- 172.21.15.46:0/65215041 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x5572396dfcd0 con 0x557239912220 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd53fff700 1 -- 172.21.15.46:0/65215041 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd6c02d020 con 0x557239912220 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd53fff700 1 -- 172.21.15.46:0/65215041 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fdd6c0164f0 con 0x557239912220 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd86c10b80 10 client.28415 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd86c10b80 10 client.28415 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd86c10b80 20 client.28415 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557239976aa0 0x557239978ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd86c10b80 4 client.28415 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd86c10b80 1 -- 172.21.15.46:0/65215041 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5572396a8cf0 con 0x557239976aa0 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.153+0000 7fdd78817700 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557239976aa0 0x557239978ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd78817700 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557239976aa0 0x557239978ec0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23dd308b8dc812e9 server_cookie=9b862d6c9e0a5395 in_seq=0 out_seq=0 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd53fff700 10 client.28415 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd53fff700 1 -- 172.21.15.46:0/65215041 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5572396a8cf0 con 0x557239976aa0 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd53fff700 10 client.28415 handle_command_reply: tid=0 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd86c10b80 1 client.28415 shutdown 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd86c10b80 1 -- 172.21.15.46:0/65215041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239912220 msgr2=0x557239916e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239912220 0x557239916e60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c018bc0 tx=0x7fdd6c018bf0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd86c10b80 1 -- 172.21.15.46:0/65215041 shutdown_connections 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557239912220 0x557239916e60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557239906a60 0x55723996eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x557239976aa0 0x557239978ec0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd86c10b80 1 --2- 172.21.15.46:0/65215041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557239907380 0x55723996f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd53fff700 0 client.28415 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:45:19.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd86c10b80 1 -- 172.21.15.46:0/65215041 >> 172.21.15.46:0/65215041 conn(0x55723987d220 msgr2=0x55723990e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd86c10b80 1 -- 172.21.15.46:0/65215041 shutdown_connections 2024-03-18T20:45:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd86c10b80 1 -- 172.21.15.46:0/65215041 wait complete. 2024-03-18T20:45:19.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.154+0000 7fdd86c10b80 20 client.28415 trim_cache size 0 max 16384 2024-03-18T20:45:19.164 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:45:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.155+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/1623327845 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdd640a1310 msgr2=0x7fdd640a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.155+0000 7fdd7aa7b700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdd640a1310 0x7fdd640a37d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000b2e0 tx=0x7fdd60005fd0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.155+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/1623327845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740ffe50 msgr2=0x7fdd74197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.155+0000 7fdd7aa7b700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740ffe50 0x7fdd74197230 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800e9a0 tx=0x7fdd6800ed60 comp rx=0 tx=0).stop 2024-03-18T20:45:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.155+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/1623327845 shutdown_connections 2024-03-18T20:45:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.155+0000 7fdd7aa7b700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdd640a1310 0x7fdd640a37d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.155+0000 7fdd7aa7b700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740ffe50 0x7fdd74197230 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.155+0000 7fdd7aa7b700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7419bb10 0x7fdd7419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.155+0000 7fdd7aa7b700 1 --2- 172.21.15.46:0/1623327845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd74104f60 0x7fdd74197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.155+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/1623327845 >> 172.21.15.46:0/1623327845 conn(0x7fdd74076d40 msgr2=0x7fdd74068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.155+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/1623327845 shutdown_connections 2024-03-18T20:45:19.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.155+0000 7fdd7aa7b700 1 -- 172.21.15.46:0/1623327845 wait complete. 2024-03-18T20:45:19.171 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:45:19.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:45:19.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:18 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2568085194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:45:19.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:18 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2568085194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:45:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.310+0000 7feb4d94f700 1 -- 172.21.15.46:0/3433023345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb48100820 msgr2=0x7feb48104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.310+0000 7feb4d94f700 1 --2- 172.21.15.46:0/3433023345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb48100820 0x7feb48104840 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7feb38009990 tx=0x7feb38009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.310+0000 7feb4d94f700 1 -- 172.21.15.46:0/3433023345 shutdown_connections 2024-03-18T20:45:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.310+0000 7feb4d94f700 1 --2- 172.21.15.46:0/3433023345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb48104f70 0x7feb4810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.310+0000 7feb4d94f700 1 --2- 172.21.15.46:0/3433023345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb480ffe70 0x7feb48100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.310+0000 7feb4d94f700 1 --2- 172.21.15.46:0/3433023345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb48100820 0x7feb48104840 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.310+0000 7feb4d94f700 1 -- 172.21.15.46:0/3433023345 >> 172.21.15.46:0/3433023345 conn(0x7feb48076d50 msgr2=0x7feb48077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.310+0000 7feb4d94f700 1 -- 172.21.15.46:0/3433023345 shutdown_connections 2024-03-18T20:45:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.310+0000 7feb4d94f700 1 -- 172.21.15.46:0/3433023345 wait complete. 2024-03-18T20:45:19.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.311+0000 7feb4d94f700 1 Processor -- start 2024-03-18T20:45:19.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.311+0000 7feb4d94f700 1 -- start start 2024-03-18T20:45:19.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.311+0000 7feb4d94f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb480ffe70 0x7feb48196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.311+0000 7feb4d94f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb48104f70 0x7feb48196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.311+0000 7feb4d94f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4819a9f0 0x7feb4819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.311+0000 7feb4d94f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4819b470 con 0x7feb480ffe70 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.311+0000 7feb4d94f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4819b5e0 con 0x7feb48104f70 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.311+0000 7feb4d94f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4819b7a0 con 0x7feb4819a9f0 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb480ffe70 0x7feb48196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb480ffe70 0x7feb48196110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38322/0 (socket says 172.21.15.46:38322) 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb46ffd700 1 -- 172.21.15.46:0/2460463268 learned_addr learned my addr 172.21.15.46:0/2460463268 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb477fe700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4819a9f0 0x7feb4819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb467fc700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb48104f70 0x7feb48196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb46ffd700 1 -- 172.21.15.46:0/2460463268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4819a9f0 msgr2=0x7feb4819ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb46ffd700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4819a9f0 0x7feb4819ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb46ffd700 1 -- 172.21.15.46:0/2460463268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb48104f70 msgr2=0x7feb48196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb46ffd700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb48104f70 0x7feb48196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb46ffd700 1 -- 172.21.15.46:0/2460463268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb380095f0 con 0x7feb480ffe70 2024-03-18T20:45:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb46ffd700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb480ffe70 0x7feb48196110 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7feb3000eab0 tx=0x7feb3000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb4c94d700 1 -- 172.21.15.46:0/2460463268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30013070 con 0x7feb480ffe70 2024-03-18T20:45:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb4d94f700 1 -- 172.21.15.46:0/2460463268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb48069050 con 0x7feb480ffe70 2024-03-18T20:45:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb4c94d700 1 -- 172.21.15.46:0/2460463268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb3000be80 con 0x7feb480ffe70 2024-03-18T20:45:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb4c94d700 1 -- 172.21.15.46:0/2460463268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb300163b0 con 0x7feb480ffe70 2024-03-18T20:45:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.312+0000 7feb4d94f700 1 -- 172.21.15.46:0/2460463268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb480695b0 con 0x7feb480ffe70 2024-03-18T20:45:19.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.315+0000 7feb4c94d700 1 -- 172.21.15.46:0/2460463268 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7feb300175a0 con 0x7feb480ffe70 2024-03-18T20:45:19.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.315+0000 7feb4c94d700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7feb340a1290 0x7feb340a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.315+0000 7feb467fc700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7feb340a1290 0x7feb340a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.315+0000 7feb4c94d700 1 -- 172.21.15.46:0/2460463268 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7feb300ce290 con 0x7feb480ffe70 2024-03-18T20:45:19.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.316+0000 7feb467fc700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7feb340a1290 0x7feb340a3750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feb48197660 tx=0x7feb38005fd0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.316+0000 7feb59ae4b80 1 Processor -- start 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.316+0000 7feb59ae4b80 1 -- start start 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb59ae4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df5a60 0x563de7df5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb59ae4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df6380 0x563de7e050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb59ae4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e07a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb59ae4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de7d4f990 con 0x563de7e05630 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb59ae4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de7d4d350 con 0x563de7df5a60 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb59ae4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de7d48c40 con 0x563de7df6380 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e07a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e07a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38336/0 (socket says 172.21.15.46:38336) 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb46ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df5a60 0x563de7df5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb477fe700 1 -- 172.21.15.46:0/1638217453 learned_addr learned my addr 172.21.15.46:0/1638217453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb467fc700 1 --2- 172.21.15.46:0/1638217453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df6380 0x563de7e050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb477fe700 1 -- 172.21.15.46:0/1638217453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df6380 msgr2=0x563de7e050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb477fe700 1 --2- 172.21.15.46:0/1638217453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df6380 0x563de7e050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb477fe700 1 -- 172.21.15.46:0/1638217453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df5a60 msgr2=0x563de7df5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb477fe700 1 --2- 172.21.15.46:0/1638217453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df5a60 0x563de7df5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb477fe700 1 -- 172.21.15.46:0/1638217453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563de7bca3b0 con 0x563de7e05630 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb46ffd700 1 --2- 172.21.15.46:0/1638217453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df5a60 0x563de7df5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb477fe700 1 --2- 172.21.15.46:0/1638217453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e07a20 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7feb40011680 tx=0x7feb40009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad10618368aa9be server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.317+0000 7feb3dffb700 1 -- 172.21.15.46:0/1638217453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4000e040 con 0x563de7e05630 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.318+0000 7feb3dffb700 1 -- 172.21.15.46:0/1638217453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb4000f040 con 0x563de7e05630 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.318+0000 7feb3dffb700 1 -- 172.21.15.46:0/1638217453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40004860 con 0x563de7e05630 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.318+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1638217453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 msgr2=0x563de7e07a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.318+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1638217453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e07a20 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7feb40011680 tx=0x7feb40009920 comp rx=0 tx=0).stop 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.318+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1638217453 shutdown_connections 2024-03-18T20:45:19.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.318+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1638217453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e07a20 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.318+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1638217453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df5a60 0x563de7df5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.318+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1638217453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df6380 0x563de7e050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.318+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1638217453 >> 172.21.15.46:0/1638217453 conn(0x563de7d6c220 msgr2=0x563de7df2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.318+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1638217453 shutdown_connections 2024-03-18T20:45:19.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.318+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1638217453 wait complete. 2024-03-18T20:45:19.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb59ae4b80 1 -- start start 2024-03-18T20:45:19.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb59ae4b80 1 Processor -- start 2024-03-18T20:45:19.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb59ae4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7df5a60 0x563de7e05030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb59ae4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e02320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb59ae4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7e05630 0x563de7e029a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb59ae4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de7ce4d80 con 0x563de7df5a60 2024-03-18T20:45:19.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb59ae4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de7ce4a20 con 0x563de7df6380 2024-03-18T20:45:19.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb59ae4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de7bcc6a0 con 0x563de7e05630 2024-03-18T20:45:19.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb59ae4b80 10 client.0 _ll_register_callbacks cb 0x563de7b503a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:19.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7df5a60 0x563de7e05030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7df5a60 0x563de7e05030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38348/0 (socket says 172.21.15.46:38348) 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb46ffd700 1 -- 172.21.15.46:0/285133732 learned_addr learned my addr 172.21.15.46:0/285133732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.319+0000 7feb477fe700 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7e05630 0x563de7e029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb467fc700 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e02320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb46ffd700 1 -- 172.21.15.46:0/285133732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7e05630 msgr2=0x563de7e029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb46ffd700 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7e05630 0x563de7e029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb46ffd700 1 -- 172.21.15.46:0/285133732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 msgr2=0x563de7e02320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb46ffd700 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e02320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb46ffd700 1 -- 172.21.15.46:0/285133732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb480695b0 con 0x563de7df5a60 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb46ffd700 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7df5a60 0x563de7e05030 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7feb300cbfc0 tx=0x7feb3009b1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a7648588d52318 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb26ffd700 1 -- 172.21.15.46:0/285133732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3009ace0 con 0x563de7df5a60 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb26ffd700 1 -- 172.21.15.46:0/285133732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7feb3009ba80 con 0x563de7df5a60 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb59ae4b80 1 -- 172.21.15.46:0/285133732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563de7de9ad0 con 0x563de7df5a60 2024-03-18T20:45:19.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb26ffd700 1 -- 172.21.15.46:0/285133732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb3009b380 con 0x563de7df5a60 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb59ae4b80 10 client.18786 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb59ae4b80 1 -- 172.21.15.46:0/285133732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7feb40011330 con 0x563de7df5a60 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.320+0000 7feb26ffd700 1 -- 172.21.15.46:0/285133732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563de7de9ad0 con 0x563de7df5a60 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.321+0000 7feb26ffd700 1 -- 172.21.15.46:0/285133732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7feb300b1070 con 0x563de7df5a60 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.321+0000 7feb59ae4b80 10 client.18786 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.321+0000 7feb59ae4b80 10 client.18786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.321+0000 7feb59ae4b80 20 client.18786 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.321+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563de7e152b0 0x563de7e176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.321+0000 7feb59ae4b80 4 client.18786 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.321+0000 7feb59ae4b80 1 -- 172.21.15.46:0/285133732 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563de7d41700 con 0x563de7e152b0 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.321+0000 7feb467fc700 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563de7e152b0 0x563de7e176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.321+0000 7feb467fc700 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563de7e152b0 0x563de7e176a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a82561b33bcf9068 server_cookie=e0d2c994da1c0f0 in_seq=0 out_seq=0 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.321+0000 7feb26ffd700 10 client.18786 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:45:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb26ffd700 1 -- 172.21.15.46:0/285133732 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563de7d41700 con 0x563de7e152b0 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb26ffd700 10 client.18786 handle_command_reply: tid=0 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb59ae4b80 1 client.18786 shutdown 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb59ae4b80 1 -- 172.21.15.46:0/285133732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7df5a60 msgr2=0x563de7e05030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7df5a60 0x563de7e05030 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7feb300cbfc0 tx=0x7feb3009b1d0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb59ae4b80 1 -- 172.21.15.46:0/285133732 shutdown_connections 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563de7e152b0 0x563de7e176a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7df5a60 0x563de7e05030 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb26ffd700 0 client.18786 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7e05630 0x563de7e029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/285133732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e02320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb59ae4b80 1 -- 172.21.15.46:0/285133732 >> 172.21.15.46:0/285133732 conn(0x563de7d6c220 msgr2=0x563de7df1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb59ae4b80 1 -- 172.21.15.46:0/285133732 shutdown_connections 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb59ae4b80 1 -- 172.21.15.46:0/285133732 wait complete. 2024-03-18T20:45:19.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.322+0000 7feb59ae4b80 20 client.18786 trim_cache size 0 max 16384 2024-03-18T20:45:19.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.336+0000 7feb59ae4b80 1 Processor -- start 2024-03-18T20:45:19.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.336+0000 7feb59ae4b80 1 -- start start 2024-03-18T20:45:19.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.336+0000 7feb59ae4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df5a60 0x563de7e0c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.336+0000 7feb59ae4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e00bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.336+0000 7feb59ae4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e01130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.336+0000 7feb59ae4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de7e032e0 con 0x563de7e05630 2024-03-18T20:45:19.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.336+0000 7feb59ae4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de7e01670 con 0x563de7df6380 2024-03-18T20:45:19.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.336+0000 7feb59ae4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de7e0c7c0 con 0x563de7df5a60 2024-03-18T20:45:19.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.336+0000 7feb477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e01130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.336+0000 7feb477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e01130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38362/0 (socket says 172.21.15.46:38362) 2024-03-18T20:45:19.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.336+0000 7feb477fe700 1 -- 172.21.15.46:0/1890814334 learned_addr learned my addr 172.21.15.46:0/1890814334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.337+0000 7feb46ffd700 1 --2- 172.21.15.46:0/1890814334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df5a60 0x563de7e0c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.337+0000 7feb467fc700 1 --2- 172.21.15.46:0/1890814334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e00bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.337+0000 7feb477fe700 1 -- 172.21.15.46:0/1890814334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df5a60 msgr2=0x563de7e0c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.337+0000 7feb477fe700 1 --2- 172.21.15.46:0/1890814334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df5a60 0x563de7e0c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.337+0000 7feb477fe700 1 -- 172.21.15.46:0/1890814334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 msgr2=0x563de7e00bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.337+0000 7feb477fe700 1 --2- 172.21.15.46:0/1890814334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e00bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.337+0000 7feb477fe700 1 -- 172.21.15.46:0/1890814334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563de7b76b60 con 0x563de7e05630 2024-03-18T20:45:19.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.337+0000 7feb477fe700 1 --2- 172.21.15.46:0/1890814334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e01130 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7feb40011680 tx=0x7feb400060b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c592574f018a67 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.337+0000 7feb3e7fc700 1 -- 172.21.15.46:0/1890814334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4000e050 con 0x563de7e05630 2024-03-18T20:45:19.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.337+0000 7feb3e7fc700 1 -- 172.21.15.46:0/1890814334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb400092e0 con 0x563de7e05630 2024-03-18T20:45:19.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.337+0000 7feb3e7fc700 1 -- 172.21.15.46:0/1890814334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40005a30 con 0x563de7e05630 2024-03-18T20:45:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.338+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1890814334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 msgr2=0x563de7e01130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.338+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1890814334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e01130 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7feb40011680 tx=0x7feb400060b0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.338+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1890814334 shutdown_connections 2024-03-18T20:45:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.338+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1890814334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e01130 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.338+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1890814334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df5a60 0x563de7e0c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.338+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1890814334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e00bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.338+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1890814334 >> 172.21.15.46:0/1890814334 conn(0x563de7d6c220 msgr2=0x563de7df0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.338+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1890814334 shutdown_connections 2024-03-18T20:45:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.338+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1890814334 wait complete. 2024-03-18T20:45:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.338+0000 7feb59ae4b80 1 -- start start 2024-03-18T20:45:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.338+0000 7feb59ae4b80 1 Processor -- start 2024-03-18T20:45:19.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb59ae4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df5a60 0x563de7e61390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb59ae4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb59ae4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e08230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb59ae4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de7e08910 con 0x563de7e05630 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb59ae4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de7e08a80 con 0x563de7df6380 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb59ae4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563de7e08c40 con 0x563de7df5a60 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb59ae4b80 10 client.0 _ll_register_callbacks cb 0x7feb30086130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e08230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e08230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38364/0 (socket says 172.21.15.46:38364) 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb467fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb477fe700 1 -- 172.21.15.46:0/1640290785 learned_addr learned my addr 172.21.15.46:0/1640290785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb46ffd700 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df5a60 0x563de7e61390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb477fe700 1 -- 172.21.15.46:0/1640290785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df5a60 msgr2=0x563de7e61390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb477fe700 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df5a60 0x563de7e61390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb477fe700 1 -- 172.21.15.46:0/1640290785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 msgr2=0x563de7e618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb477fe700 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb477fe700 1 -- 172.21.15.46:0/1640290785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563de7bca3b0 con 0x563de7e05630 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb467fc700 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb477fe700 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e08230 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7feb400042f0 tx=0x7feb400095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ca9219d352eb31 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.339+0000 7feb26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb26ffd700 1 -- 172.21.15.46:0/1640290785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40003cd0 con 0x563de7e05630 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb26ffd700 1 -- 172.21.15.46:0/1640290785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7feb40003e30 con 0x563de7e05630 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1640290785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb4000f040 con 0x563de7e05630 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb26ffd700 1 -- 172.21.15.46:0/1640290785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb4000e020 con 0x563de7e05630 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb59ae4b80 10 client.18792 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1640290785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x563de7d0eea0 con 0x563de7e05630 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb26ffd700 1 -- 172.21.15.46:0/1640290785 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb4000f040 con 0x563de7e05630 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb26ffd700 1 -- 172.21.15.46:0/1640290785 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7feb4001e4d0 con 0x563de7e05630 2024-03-18T20:45:19.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb59ae4b80 10 client.18792 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb59ae4b80 10 client.18792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb59ae4b80 20 client.18792 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563de7e69f30 0x563de7e6c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb59ae4b80 4 client.18792 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1640290785 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563de7b97cf0 con 0x563de7e69f30 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.340+0000 7feb46ffd700 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563de7e69f30 0x563de7e6c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb46ffd700 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563de7e69f30 0x563de7e6c320 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23fd6e4f16b4f80d server_cookie=afd651c77fcf7bb9 in_seq=0 out_seq=0 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb26ffd700 10 client.18792 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb26ffd700 1 -- 172.21.15.46:0/1640290785 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563de7b97cf0 con 0x563de7e69f30 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb26ffd700 10 client.18792 handle_command_reply: tid=0 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb59ae4b80 1 client.18792 shutdown 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1640290785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 msgr2=0x563de7e08230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e08230 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7feb400042f0 tx=0x7feb400095e0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1640290785 shutdown_connections 2024-03-18T20:45:19.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x563de7e69f30 0x563de7e6c320 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563de7e05630 0x563de7e08230 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563de7df5a60 0x563de7e61390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb26ffd700 0 client.18792 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:45:19.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb59ae4b80 1 --2- 172.21.15.46:0/1640290785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563de7df6380 0x563de7e618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1640290785 >> 172.21.15.46:0/1640290785 conn(0x563de7d6c220 msgr2=0x563de7e00b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1640290785 shutdown_connections 2024-03-18T20:45:19.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb59ae4b80 1 -- 172.21.15.46:0/1640290785 wait complete. 2024-03-18T20:45:19.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.341+0000 7feb59ae4b80 20 client.18792 trim_cache size 0 max 16384 2024-03-18T20:45:19.352 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:45:19.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.342+0000 7feb4d94f700 1 -- 172.21.15.46:0/2460463268 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7feb340a1290 msgr2=0x7feb340a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.342+0000 7feb4d94f700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7feb340a1290 0x7feb340a3750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feb48197660 tx=0x7feb38005fd0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.342+0000 7feb4d94f700 1 -- 172.21.15.46:0/2460463268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb480ffe70 msgr2=0x7feb48196110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.342+0000 7feb4d94f700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb480ffe70 0x7feb48196110 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7feb3000eab0 tx=0x7feb3000edc0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.342+0000 7feb4d94f700 1 -- 172.21.15.46:0/2460463268 shutdown_connections 2024-03-18T20:45:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.342+0000 7feb4d94f700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7feb340a1290 0x7feb340a3750 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.342+0000 7feb4d94f700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb480ffe70 0x7feb48196110 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.342+0000 7feb4d94f700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb4819a9f0 0x7feb4819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.342+0000 7feb4d94f700 1 --2- 172.21.15.46:0/2460463268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb48104f70 0x7feb48196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.342+0000 7feb4d94f700 1 -- 172.21.15.46:0/2460463268 >> 172.21.15.46:0/2460463268 conn(0x7feb48076d50 msgr2=0x7feb480fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.342+0000 7feb4d94f700 1 -- 172.21.15.46:0/2460463268 shutdown_connections 2024-03-18T20:45:19.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.342+0000 7feb4d94f700 1 -- 172.21.15.46:0/2460463268 wait complete. 2024-03-18T20:45:19.358 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:45:19.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:45:19.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:18 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2568085194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:45:19.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.495+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3448084865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640ffe70 msgr2=0x7fef64100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.495+0000 7fef6b1e0700 1 --2- 172.21.15.46:0/3448084865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640ffe70 0x7fef64100250 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fef54009990 tx=0x7fef54009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.495+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3448084865 shutdown_connections 2024-03-18T20:45:19.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.495+0000 7fef6b1e0700 1 --2- 172.21.15.46:0/3448084865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640ffe70 0x7fef64100250 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.495+0000 7fef6b1e0700 1 --2- 172.21.15.46:0/3448084865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef64104f90 0x7fef6410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.495+0000 7fef6b1e0700 1 --2- 172.21.15.46:0/3448084865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef64100820 0x7fef64104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.495+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3448084865 >> 172.21.15.46:0/3448084865 conn(0x7fef64076d80 msgr2=0x7fef64077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.495+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3448084865 shutdown_connections 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.495+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3448084865 wait complete. 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.495+0000 7fef6b1e0700 1 Processor -- start 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.495+0000 7fef6b1e0700 1 -- start start 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef6b1e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640ffe70 0x7fef64197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef6b1e0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef64104f90 0x7fef64197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef6b1e0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef6419bb10 0x7fef6419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef6b1e0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6419c590 con 0x7fef640ffe70 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef6b1e0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6419c700 con 0x7fef6419bb10 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef6b1e0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6419c8c0 con 0x7fef64104f90 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef68f7c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640ffe70 0x7fef64197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef68f7c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640ffe70 0x7fef64197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38384/0 (socket says 172.21.15.46:38384) 2024-03-18T20:45:19.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef68f7c700 1 -- 172.21.15.46:0/3099321334 learned_addr learned my addr 172.21.15.46:0/3099321334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef6977d700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef6419bb10 0x7fef6419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef63fff700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef64104f90 0x7fef64197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef68f7c700 1 -- 172.21.15.46:0/3099321334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef64104f90 msgr2=0x7fef64197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef68f7c700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef64104f90 0x7fef64197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef68f7c700 1 -- 172.21.15.46:0/3099321334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef6419bb10 msgr2=0x7fef6419bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef68f7c700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef6419bb10 0x7fef6419bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef68f7c700 1 -- 172.21.15.46:0/3099321334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef540095f0 con 0x7fef640ffe70 2024-03-18T20:45:19.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.496+0000 7fef68f7c700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640ffe70 0x7fef64197230 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fef54000c00 tx=0x7fef54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.497+0000 7fef61ffb700 1 -- 172.21.15.46:0/3099321334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5401d070 con 0x7fef640ffe70 2024-03-18T20:45:19.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.497+0000 7fef61ffb700 1 -- 172.21.15.46:0/3099321334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef54005960 con 0x7fef640ffe70 2024-03-18T20:45:19.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.497+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3099321334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef641a3dc0 con 0x7fef640ffe70 2024-03-18T20:45:19.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.497+0000 7fef61ffb700 1 -- 172.21.15.46:0/3099321334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5400f670 con 0x7fef640ffe70 2024-03-18T20:45:19.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.497+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3099321334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef641a4180 con 0x7fef640ffe70 2024-03-18T20:45:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.499+0000 7fef61ffb700 1 -- 172.21.15.46:0/3099321334 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fef54005020 con 0x7fef640ffe70 2024-03-18T20:45:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.500+0000 7fef61ffb700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fef4c0a1290 0x7fef4c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.500+0000 7fef61ffb700 1 -- 172.21.15.46:0/3099321334 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fef540cdd70 con 0x7fef640ffe70 2024-03-18T20:45:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.500+0000 7fef63fff700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fef4c0a1290 0x7fef4c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.500+0000 7fef63fff700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fef4c0a1290 0x7fef4c0a3750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef58005950 tx=0x7fef580058e0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef77375b80 1 Processor -- start 2024-03-18T20:45:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef77375b80 1 -- start start 2024-03-18T20:45:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef77375b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab01fa60 0x55edab01fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef77375b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab020380 0x55edab02ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef77375b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab02b220 0x55edab02d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef77375b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaaf79990 con 0x55edab01fa60 2024-03-18T20:45:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef77375b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaaf77350 con 0x55edab020380 2024-03-18T20:45:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef77375b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaaf72c40 con 0x55edab02b220 2024-03-18T20:45:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef68f7c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab01fa60 0x55edab01fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef68f7c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab01fa60 0x55edab01fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38386/0 (socket says 172.21.15.46:38386) 2024-03-18T20:45:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef68f7c700 1 -- 172.21.15.46:0/1530027095 learned_addr learned my addr 172.21.15.46:0/1530027095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef6977d700 1 --2- 172.21.15.46:0/1530027095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab02b220 0x55edab02d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.501+0000 7fef63fff700 1 --2- 172.21.15.46:0/1530027095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab020380 0x55edab02ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.502+0000 7fef68f7c700 1 -- 172.21.15.46:0/1530027095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab02b220 msgr2=0x55edab02d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.502+0000 7fef68f7c700 1 --2- 172.21.15.46:0/1530027095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab02b220 0x55edab02d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.502+0000 7fef68f7c700 1 -- 172.21.15.46:0/1530027095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab020380 msgr2=0x55edab02ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.502+0000 7fef68f7c700 1 --2- 172.21.15.46:0/1530027095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab020380 0x55edab02ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.502+0000 7fef68f7c700 1 -- 172.21.15.46:0/1530027095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edaadf43b0 con 0x55edab01fa60 2024-03-18T20:45:19.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.502+0000 7fef68f7c700 1 --2- 172.21.15.46:0/1530027095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab01fa60 0x55edab01fe40 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fef54028000 tx=0x7fef54009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ef63849521e9ce server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.502+0000 7fef4affd700 1 -- 172.21.15.46:0/1530027095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5409bea0 con 0x55edab01fa60 2024-03-18T20:45:19.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.502+0000 7fef4affd700 1 -- 172.21.15.46:0/1530027095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef5409ecc0 con 0x55edab01fa60 2024-03-18T20:45:19.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.502+0000 7fef4affd700 1 -- 172.21.15.46:0/1530027095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5409b330 con 0x55edab01fa60 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.503+0000 7fef77375b80 1 -- 172.21.15.46:0/1530027095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab01fa60 msgr2=0x55edab01fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.503+0000 7fef77375b80 1 --2- 172.21.15.46:0/1530027095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab01fa60 0x55edab01fe40 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fef54028000 tx=0x7fef54009310 comp rx=0 tx=0).stop 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.503+0000 7fef77375b80 1 -- 172.21.15.46:0/1530027095 shutdown_connections 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.503+0000 7fef77375b80 1 --2- 172.21.15.46:0/1530027095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab01fa60 0x55edab01fe40 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.503+0000 7fef77375b80 1 --2- 172.21.15.46:0/1530027095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab02b220 0x55edab02d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.503+0000 7fef77375b80 1 --2- 172.21.15.46:0/1530027095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab020380 0x55edab02ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.503+0000 7fef77375b80 1 -- 172.21.15.46:0/1530027095 >> 172.21.15.46:0/1530027095 conn(0x55edaaf96220 msgr2=0x55edab01c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.503+0000 7fef77375b80 1 -- 172.21.15.46:0/1530027095 shutdown_connections 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.503+0000 7fef77375b80 1 -- 172.21.15.46:0/1530027095 wait complete. 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.503+0000 7fef77375b80 1 -- start start 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.503+0000 7fef77375b80 1 Processor -- start 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.504+0000 7fef77375b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab01fa60 0x55edab02b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.504+0000 7fef77375b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab020380 0x55edab02bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.504+0000 7fef77375b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab02b220 0x55edab036680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.504+0000 7fef77375b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaaf0ed80 con 0x55edab01fa60 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.504+0000 7fef77375b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaaf0ea20 con 0x55edab02b220 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.504+0000 7fef77375b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edaadf66a0 con 0x55edab020380 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.504+0000 7fef77375b80 10 client.0 _ll_register_callbacks cb 0x55edaad7a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.504+0000 7fef63fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab020380 0x55edab02bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.504+0000 7fef63fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab020380 0x55edab02bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59370/0 (socket says 172.21.15.46:59370) 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.504+0000 7fef63fff700 1 -- 172.21.15.46:0/3378840189 learned_addr learned my addr 172.21.15.46:0/3378840189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.504+0000 7fef6977d700 1 --2- 172.21.15.46:0/3378840189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab02b220 0x55edab036680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef63fff700 1 -- 172.21.15.46:0/3378840189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab02b220 msgr2=0x55edab036680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef63fff700 1 --2- 172.21.15.46:0/3378840189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab02b220 0x55edab036680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef63fff700 1 -- 172.21.15.46:0/3378840189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab01fa60 msgr2=0x55edab02b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef63fff700 1 --2- 172.21.15.46:0/3378840189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab01fa60 0x55edab02b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef63fff700 1 -- 172.21.15.46:0/3378840189 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef641a4180 con 0x55edab020380 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef63fff700 1 --2- 172.21.15.46:0/3378840189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab020380 0x55edab02bb40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fef58010010 tx=0x7fef58004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6cc7406967c39c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef43fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef43fff700 1 -- 172.21.15.46:0/3378840189 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58028670 con 0x55edab020380 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef77375b80 1 -- 172.21.15.46:0/3378840189 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edab013ad0 con 0x55edab020380 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef43fff700 1 -- 172.21.15.46:0/3378840189 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fef580287d0 con 0x55edab020380 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef43fff700 1 -- 172.21.15.46:0/3378840189 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef58011e70 con 0x55edab020380 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef77375b80 10 client.28456 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef77375b80 1 -- 172.21.15.46:0/3378840189 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55edaada0b60 con 0x55edab020380 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.505+0000 7fef43fff700 1 -- 172.21.15.46:0/3378840189 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55edab013ad0 con 0x55edab020380 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.506+0000 7fef43fff700 1 -- 172.21.15.46:0/3378840189 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fef5801f690 con 0x55edab020380 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.506+0000 7fef77375b80 10 client.28456 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.506+0000 7fef77375b80 10 client.28456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.506+0000 7fef77375b80 20 client.28456 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.506+0000 7fef77375b80 1 --2- 172.21.15.46:0/3378840189 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edab03aa40 0x55edab03ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.506+0000 7fef77375b80 4 client.28456 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.506+0000 7fef77375b80 1 -- 172.21.15.46:0/3378840189 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edaaf6b700 con 0x55edab03aa40 2024-03-18T20:45:19.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.506+0000 7fef68f7c700 1 --2- 172.21.15.46:0/3378840189 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edab03aa40 0x55edab03ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.506+0000 7fef43fff700 10 client.28456 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.506+0000 7fef68f7c700 1 --2- 172.21.15.46:0/3378840189 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edab03aa40 0x55edab03ce60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77ac8115de2998c server_cookie=a9d219fb1a09aef0 in_seq=0 out_seq=0 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.507+0000 7fef43fff700 1 -- 172.21.15.46:0/3378840189 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55edaaf6b700 con 0x55edab03aa40 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.507+0000 7fef43fff700 10 client.28456 handle_command_reply: tid=0 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.507+0000 7fef77375b80 1 client.28456 shutdown 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.507+0000 7fef77375b80 1 -- 172.21.15.46:0/3378840189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab020380 msgr2=0x55edab02bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.507+0000 7fef77375b80 1 --2- 172.21.15.46:0/3378840189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab020380 0x55edab02bb40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fef58010010 tx=0x7fef58004750 comp rx=0 tx=0).stop 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.507+0000 7fef77375b80 1 -- 172.21.15.46:0/3378840189 shutdown_connections 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.508+0000 7fef77375b80 1 --2- 172.21.15.46:0/3378840189 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edab03aa40 0x55edab03ce60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.508+0000 7fef77375b80 1 --2- 172.21.15.46:0/3378840189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab01fa60 0x55edab02b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.508+0000 7fef77375b80 1 --2- 172.21.15.46:0/3378840189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab02b220 0x55edab036680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.508+0000 7fef43fff700 0 client.28456 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.508+0000 7fef77375b80 1 --2- 172.21.15.46:0/3378840189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab020380 0x55edab02bb40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.508+0000 7fef77375b80 1 -- 172.21.15.46:0/3378840189 >> 172.21.15.46:0/3378840189 conn(0x55edaaf96220 msgr2=0x55edab01bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.508+0000 7fef77375b80 1 -- 172.21.15.46:0/3378840189 shutdown_connections 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.508+0000 7fef77375b80 1 -- 172.21.15.46:0/3378840189 wait complete. 2024-03-18T20:45:19.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.508+0000 7fef77375b80 20 client.28456 trim_cache size 0 max 16384 2024-03-18T20:45:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef77375b80 1 Processor -- start 2024-03-18T20:45:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef77375b80 1 -- start start 2024-03-18T20:45:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef77375b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab01fa60 0x55edab030500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef77375b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab020380 0x55edab025810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef77375b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab02b220 0x55edab025d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef77375b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edab020d30 con 0x55edab020380 2024-03-18T20:45:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef77375b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edab038170 con 0x55edab01fa60 2024-03-18T20:45:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef77375b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edab03d1f0 con 0x55edab02b220 2024-03-18T20:45:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef63fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab020380 0x55edab025810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef63fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab020380 0x55edab025810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38408/0 (socket says 172.21.15.46:38408) 2024-03-18T20:45:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef63fff700 1 -- 172.21.15.46:0/737045217 learned_addr learned my addr 172.21.15.46:0/737045217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef6977d700 1 --2- 172.21.15.46:0/737045217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab02b220 0x55edab025d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef68f7c700 1 --2- 172.21.15.46:0/737045217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab01fa60 0x55edab030500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef63fff700 1 -- 172.21.15.46:0/737045217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab02b220 msgr2=0x55edab025d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef63fff700 1 --2- 172.21.15.46:0/737045217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab02b220 0x55edab025d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef63fff700 1 -- 172.21.15.46:0/737045217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab01fa60 msgr2=0x55edab030500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef63fff700 1 --2- 172.21.15.46:0/737045217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab01fa60 0x55edab030500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef63fff700 1 -- 172.21.15.46:0/737045217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edab036f30 con 0x55edab020380 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.522+0000 7fef68f7c700 1 --2- 172.21.15.46:0/737045217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab01fa60 0x55edab030500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef63fff700 1 --2- 172.21.15.46:0/737045217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab020380 0x55edab025810 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fef58014040 tx=0x7fef5800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1058d2bead84b8fc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef4b7fe700 1 -- 172.21.15.46:0/737045217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5801f630 con 0x55edab020380 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef4b7fe700 1 -- 172.21.15.46:0/737045217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef58011ab0 con 0x55edab020380 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef4b7fe700 1 -- 172.21.15.46:0/737045217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58031450 con 0x55edab020380 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef77375b80 1 -- 172.21.15.46:0/737045217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab020380 msgr2=0x55edab025810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef77375b80 1 --2- 172.21.15.46:0/737045217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab020380 0x55edab025810 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fef58014040 tx=0x7fef5800b940 comp rx=0 tx=0).stop 2024-03-18T20:45:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef77375b80 1 -- 172.21.15.46:0/737045217 shutdown_connections 2024-03-18T20:45:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef77375b80 1 --2- 172.21.15.46:0/737045217 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab02b220 0x55edab025d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef77375b80 1 --2- 172.21.15.46:0/737045217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab01fa60 0x55edab030500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef77375b80 1 --2- 172.21.15.46:0/737045217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab020380 0x55edab025810 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef77375b80 1 -- 172.21.15.46:0/737045217 >> 172.21.15.46:0/737045217 conn(0x55edaaf96220 msgr2=0x55edab029d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef77375b80 1 -- 172.21.15.46:0/737045217 shutdown_connections 2024-03-18T20:45:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.523+0000 7fef77375b80 1 -- 172.21.15.46:0/737045217 wait complete. 2024-03-18T20:45:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.524+0000 7fef77375b80 1 -- start start 2024-03-18T20:45:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.524+0000 7fef77375b80 1 Processor -- start 2024-03-18T20:45:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.524+0000 7fef77375b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab01fa60 0x55edab02e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.524+0000 7fef77375b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab020380 0x55edab02ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.524+0000 7fef77375b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab02b220 0x55edab0878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.524+0000 7fef77375b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edab087df0 con 0x55edab02b220 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.524+0000 7fef77375b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edab087f60 con 0x55edab020380 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.524+0000 7fef77375b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edab088120 con 0x55edab01fa60 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.524+0000 7fef77375b80 10 client.0 _ll_register_callbacks cb 0x55edaad7a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.524+0000 7fef6977d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab02b220 0x55edab0878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef6977d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab02b220 0x55edab0878b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38420/0 (socket says 172.21.15.46:38420) 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef68f7c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab01fa60 0x55edab02e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef6977d700 1 -- 172.21.15.46:0/2378423961 learned_addr learned my addr 172.21.15.46:0/2378423961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef63fff700 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab020380 0x55edab02ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef68f7c700 1 -- 172.21.15.46:0/2378423961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab020380 msgr2=0x55edab02ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef68f7c700 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab020380 0x55edab02ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef68f7c700 1 -- 172.21.15.46:0/2378423961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab02b220 msgr2=0x55edab0878b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:19.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef68f7c700 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab02b220 0x55edab0878b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef68f7c700 1 -- 172.21.15.46:0/2378423961 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edaada0b60 con 0x55edab01fa60 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef63fff700 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab020380 0x55edab02ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef6977d700 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab02b220 0x55edab0878b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef68f7c700 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab01fa60 0x55edab02e680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef540a2d50 tx=0x7fef540a33c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=910a047a0d321995 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef43fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef43fff700 1 -- 172.21.15.46:0/2378423961 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5409bea0 con 0x55edab01fa60 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef77375b80 1 -- 172.21.15.46:0/2378423961 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef5409ecc0 con 0x55edab01fa60 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef43fff700 1 -- 172.21.15.46:0/2378423961 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fef5409fde0 con 0x55edab01fa60 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef43fff700 1 -- 172.21.15.46:0/2378423961 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef5409ac60 con 0x55edab01fa60 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef77375b80 10 client.28462 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.525+0000 7fef77375b80 1 -- 172.21.15.46:0/2378423961 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55edaaf38ea0 con 0x55edab01fa60 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.526+0000 7fef43fff700 1 -- 172.21.15.46:0/2378423961 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef5409ecc0 con 0x55edab01fa60 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.526+0000 7fef43fff700 1 -- 172.21.15.46:0/2378423961 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fef540a2df0 con 0x55edab01fa60 2024-03-18T20:45:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.526+0000 7fef77375b80 10 client.28462 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.526+0000 7fef77375b80 10 client.28462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.526+0000 7fef77375b80 20 client.28462 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.526+0000 7fef77375b80 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edab08f550 0x55edab091970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.526+0000 7fef77375b80 4 client.28462 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:45:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.526+0000 7fef77375b80 1 -- 172.21.15.46:0/2378423961 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55edaadc1cf0 con 0x55edab08f550 2024-03-18T20:45:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.526+0000 7fef63fff700 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edab08f550 0x55edab091970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef43fff700 10 client.28462 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef63fff700 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edab08f550 0x55edab091970 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc60b01dee39fe8 server_cookie=fd8f102b80a4c54c in_seq=0 out_seq=0 2024-03-18T20:45:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef43fff700 1 -- 172.21.15.46:0/2378423961 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55edaadc1cf0 con 0x55edab08f550 2024-03-18T20:45:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef43fff700 10 client.28462 handle_command_reply: tid=0 2024-03-18T20:45:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef77375b80 1 client.28462 shutdown 2024-03-18T20:45:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef77375b80 1 -- 172.21.15.46:0/2378423961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab01fa60 msgr2=0x55edab02e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef77375b80 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab01fa60 0x55edab02e680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef540a2d50 tx=0x7fef540a33c0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef77375b80 1 -- 172.21.15.46:0/2378423961 shutdown_connections 2024-03-18T20:45:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef77375b80 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edab08f550 0x55edab091970 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef77375b80 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edab02b220 0x55edab0878b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef43fff700 0 client.28462 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef77375b80 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edab01fa60 0x55edab02e680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef77375b80 1 --2- 172.21.15.46:0/2378423961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edab020380 0x55edab02ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef77375b80 1 -- 172.21.15.46:0/2378423961 >> 172.21.15.46:0/2378423961 conn(0x55edaaf96220 msgr2=0x55edab025810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef77375b80 1 -- 172.21.15.46:0/2378423961 shutdown_connections 2024-03-18T20:45:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef77375b80 1 -- 172.21.15.46:0/2378423961 wait complete. 2024-03-18T20:45:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.527+0000 7fef77375b80 20 client.28462 trim_cache size 0 max 16384 2024-03-18T20:45:19.537 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:45:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.528+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3099321334 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fef4c0a1290 msgr2=0x7fef4c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.528+0000 7fef6b1e0700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fef4c0a1290 0x7fef4c0a3750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef58005950 tx=0x7fef580058e0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.528+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3099321334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640ffe70 msgr2=0x7fef64197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.528+0000 7fef6b1e0700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640ffe70 0x7fef64197230 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fef54000c00 tx=0x7fef54004730 comp rx=0 tx=0).stop 2024-03-18T20:45:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.528+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3099321334 shutdown_connections 2024-03-18T20:45:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.528+0000 7fef6b1e0700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fef4c0a1290 0x7fef4c0a3750 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.528+0000 7fef6b1e0700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef640ffe70 0x7fef64197230 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.528+0000 7fef6b1e0700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef6419bb10 0x7fef6419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.528+0000 7fef6b1e0700 1 --2- 172.21.15.46:0/3099321334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef64104f90 0x7fef64197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.528+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3099321334 >> 172.21.15.46:0/3099321334 conn(0x7fef64076d80 msgr2=0x7fef64069410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.528+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3099321334 shutdown_connections 2024-03-18T20:45:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:19.528+0000 7fef6b1e0700 1 -- 172.21.15.46:0/3099321334 wait complete. 2024-03-18T20:45:19.709 INFO:tasks.workunit.client.0.smithi046.stdout: 1 335120 3.64 MB/sec execute 374 sec latency 2019.817 ms 2024-03-18T20:45:20.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:19 smithi155 ceph-mon[48344]: pgmap v286: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-03-18T20:45:20.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:19 smithi046 ceph-mon[43495]: pgmap v286: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-03-18T20:45:20.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:19 smithi073 ceph-mon[47384]: pgmap v286: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-03-18T20:45:20.545 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:45:20.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:45:20.546 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:45:20.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.697+0000 7f5de5018700 1 -- 172.21.15.46:0/2233974918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0069540 msgr2=0x7f5de0069920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.697+0000 7f5de5018700 1 --2- 172.21.15.46:0/2233974918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0069540 0x7f5de0069920 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00098f0 tx=0x7f5dd0009c00 comp rx=0 tx=0).stop 2024-03-18T20:45:20.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.697+0000 7f5de5018700 1 -- 172.21.15.46:0/2233974918 shutdown_connections 2024-03-18T20:45:20.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.697+0000 7f5de5018700 1 --2- 172.21.15.46:0/2233974918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5de01060a0 0x7f5de010dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.697+0000 7f5de5018700 1 --2- 172.21.15.46:0/2233974918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0069540 0x7f5de0069920 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.697+0000 7f5de5018700 1 --2- 172.21.15.46:0/2233974918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de0069ef0 0x7f5de0105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.697+0000 7f5de5018700 1 -- 172.21.15.46:0/2233974918 >> 172.21.15.46:0/2233974918 conn(0x7f5de0076d50 msgr2=0x7f5de0077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:20.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.697+0000 7f5de5018700 1 -- 172.21.15.46:0/2233974918 shutdown_connections 2024-03-18T20:45:20.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.697+0000 7f5de5018700 1 -- 172.21.15.46:0/2233974918 wait complete. 2024-03-18T20:45:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5de5018700 1 Processor -- start 2024-03-18T20:45:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5de5018700 1 -- start start 2024-03-18T20:45:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5de5018700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0069540 0x7f5de00ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5de5018700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5de01060a0 0x7f5de01001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5de5018700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de01006f0 0x7f5de01a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5de5018700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de01a5980 con 0x7f5de01006f0 2024-03-18T20:45:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5de5018700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de01a5af0 con 0x7f5de0069540 2024-03-18T20:45:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5de5018700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de01a5cb0 con 0x7f5de01060a0 2024-03-18T20:45:20.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5ddf59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de01006f0 0x7f5de01a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5dded9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0069540 0x7f5de00ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5dded9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0069540 0x7f5de00ffc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:36358/0 (socket says 172.21.15.46:36358) 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5dded9d700 1 -- 172.21.15.46:0/2533606133 learned_addr learned my addr 172.21.15.46:0/2533606133 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.698+0000 7f5dde59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5de01060a0 0x7f5de01001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.699+0000 7f5dded9d700 1 -- 172.21.15.46:0/2533606133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5de01060a0 msgr2=0x7f5de01001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.699+0000 7f5dded9d700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5de01060a0 0x7f5de01001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.699+0000 7f5dded9d700 1 -- 172.21.15.46:0/2533606133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de01006f0 msgr2=0x7f5de01a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.699+0000 7f5dded9d700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de01006f0 0x7f5de01a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.699+0000 7f5dded9d700 1 -- 172.21.15.46:0/2533606133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40095f0 con 0x7f5de0069540 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.699+0000 7f5ddf59e700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de01006f0 0x7f5de01a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.699+0000 7f5dded9d700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0069540 0x7f5de00ffc70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000fa00 tx=0x7f5dd000fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.699+0000 7f5dc7fff700 1 -- 172.21.15.46:0/2533606133 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd001c070 con 0x7f5de0069540 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.699+0000 7f5dc7fff700 1 -- 172.21.15.46:0/2533606133 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd000baa0 con 0x7f5de0069540 2024-03-18T20:45:20.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.699+0000 7f5de5018700 1 -- 172.21.15.46:0/2533606133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd0009520 con 0x7f5de0069540 2024-03-18T20:45:20.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.699+0000 7f5dc7fff700 1 -- 172.21.15.46:0/2533606133 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0004930 con 0x7f5de0069540 2024-03-18T20:45:20.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.699+0000 7f5de5018700 1 -- 172.21.15.46:0/2533606133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de01a61c0 con 0x7f5de0069540 2024-03-18T20:45:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.701+0000 7f5dc7fff700 1 -- 172.21.15.46:0/2533606133 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5dd0020070 con 0x7f5de0069540 2024-03-18T20:45:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.702+0000 7f5dc7fff700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5dc80a1290 0x7f5dc80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.702+0000 7f5dde59c700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5dc80a1290 0x7f5dc80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.702+0000 7f5dc7fff700 1 -- 172.21.15.46:0/2533606133 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5dd00cd8d0 con 0x7f5de0069540 2024-03-18T20:45:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.703+0000 7f5dde59c700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5dc80a1290 0x7f5dc80a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5de01013a0 tx=0x7f5dd4009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.703+0000 7f5df11adb80 1 Processor -- start 2024-03-18T20:45:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.703+0000 7f5df11adb80 1 -- start start 2024-03-18T20:45:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.703+0000 7f5df11adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf18fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.703+0000 7f5df11adb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf190400 0x556fbf19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.703+0000 7f5df11adb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf19d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.703+0000 7f5df11adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbf0e9a10 con 0x556fbf18fae0 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.703+0000 7f5df11adb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbf0e73d0 con 0x556fbf19b2a0 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.703+0000 7f5df11adb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbf0e2cc0 con 0x556fbf190400 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.703+0000 7f5dded9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf18fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dded9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf18fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38454/0 (socket says 172.21.15.46:38454) 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dded9d700 1 -- 172.21.15.46:0/3908354882 learned_addr learned my addr 172.21.15.46:0/3908354882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dde59c700 1 --2- 172.21.15.46:0/3908354882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf190400 0x556fbf19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5ddf59e700 1 --2- 172.21.15.46:0/3908354882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dded9d700 1 -- 172.21.15.46:0/3908354882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf190400 msgr2=0x556fbf19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dded9d700 1 --2- 172.21.15.46:0/3908354882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf190400 0x556fbf19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dded9d700 1 -- 172.21.15.46:0/3908354882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 msgr2=0x556fbf19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dded9d700 1 --2- 172.21.15.46:0/3908354882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dded9d700 1 -- 172.21.15.46:0/3908354882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fbef64430 con 0x556fbf18fae0 2024-03-18T20:45:20.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dde59c700 1 --2- 172.21.15.46:0/3908354882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf190400 0x556fbf19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5ddf59e700 1 --2- 172.21.15.46:0/3908354882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dded9d700 1 --2- 172.21.15.46:0/3908354882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf18fec0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00cb0b0 tx=0x7f5dd00054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cb97553b4b39b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dc57fa700 1 -- 172.21.15.46:0/3908354882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00a2eb0 con 0x556fbf18fae0 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dc57fa700 1 -- 172.21.15.46:0/3908354882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd0099d20 con 0x556fbf18fae0 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.704+0000 7f5dc57fa700 1 -- 172.21.15.46:0/3908354882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd009eab0 con 0x556fbf18fae0 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.705+0000 7f5df11adb80 1 -- 172.21.15.46:0/3908354882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 msgr2=0x556fbf18fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.705+0000 7f5df11adb80 1 --2- 172.21.15.46:0/3908354882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf18fec0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00cb0b0 tx=0x7f5dd00054e0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.705+0000 7f5df11adb80 1 -- 172.21.15.46:0/3908354882 shutdown_connections 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.705+0000 7f5df11adb80 1 --2- 172.21.15.46:0/3908354882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf18fec0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.705+0000 7f5df11adb80 1 --2- 172.21.15.46:0/3908354882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.705+0000 7f5df11adb80 1 --2- 172.21.15.46:0/3908354882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf190400 0x556fbf19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.705+0000 7f5df11adb80 1 -- 172.21.15.46:0/3908354882 >> 172.21.15.46:0/3908354882 conn(0x556fbf1062a0 msgr2=0x556fbf18c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.705+0000 7f5df11adb80 1 -- 172.21.15.46:0/3908354882 shutdown_connections 2024-03-18T20:45:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.705+0000 7f5df11adb80 1 -- 172.21.15.46:0/3908354882 wait complete. 2024-03-18T20:45:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.705+0000 7f5df11adb80 1 -- start start 2024-03-18T20:45:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5df11adb80 1 Processor -- start 2024-03-18T20:45:20.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 336023 3.64 MB/sec execute 375 sec latency 2639.249 ms 2024-03-18T20:45:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5df11adb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf18fae0 0x556fbf19b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5df11adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf190400 0x556fbf19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5df11adb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf19c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5df11adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbf07ee00 con 0x556fbf190400 2024-03-18T20:45:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5df11adb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbf07eaa0 con 0x556fbf19b2a0 2024-03-18T20:45:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5df11adb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbef66720 con 0x556fbf18fae0 2024-03-18T20:45:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5df11adb80 10 client.0 _ll_register_callbacks cb 0x556fbeeea420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5dde59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf190400 0x556fbf19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5dde59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf190400 0x556fbf19bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38458/0 (socket says 172.21.15.46:38458) 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5dde59c700 1 -- 172.21.15.46:0/2389937123 learned_addr learned my addr 172.21.15.46:0/2389937123 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5dded9d700 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf18fae0 0x556fbf19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5ddf59e700 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf19c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5dde59c700 1 -- 172.21.15.46:0/2389937123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf18fae0 msgr2=0x556fbf19b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5dde59c700 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf18fae0 0x556fbf19b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5dde59c700 1 -- 172.21.15.46:0/2389937123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 msgr2=0x556fbf19c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5dde59c700 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf19c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5dde59c700 1 -- 172.21.15.46:0/2389937123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd8003a60 con 0x556fbf190400 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.706+0000 7f5ddf59e700 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf19c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5dde59c700 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf190400 0x556fbf19bbc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400ea30 tx=0x7f5dd40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e85555f6d3f7a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5dc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5dc2ff5700 1 -- 172.21.15.46:0/2389937123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4022070 con 0x556fbf190400 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5dc2ff5700 1 -- 172.21.15.46:0/2389937123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5dd401da30 con 0x556fbf190400 2024-03-18T20:45:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5df11adb80 1 -- 172.21.15.46:0/2389937123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556fbf183b50 con 0x556fbf190400 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5dc2ff5700 1 -- 172.21.15.46:0/2389937123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dd401dc30 con 0x556fbf190400 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5df11adb80 10 client.18819 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5df11adb80 1 -- 172.21.15.46:0/2389937123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f5de01a61c0 con 0x556fbf190400 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5dc2ff5700 1 -- 172.21.15.46:0/2389937123 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd4033020 con 0x556fbf190400 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5dc2ff5700 1 -- 172.21.15.46:0/2389937123 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5dd4026ce0 con 0x556fbf190400 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5df11adb80 10 client.18819 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5df11adb80 10 client.18819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5df11adb80 20 client.18819 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5df11adb80 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556fbf1aab50 0x556fbf1acf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5df11adb80 4 client.18819 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.707+0000 7f5df11adb80 1 -- 172.21.15.46:0/2389937123 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fbf0db780 con 0x556fbf1aab50 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.708+0000 7f5dded9d700 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556fbf1aab50 0x556fbf1acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.708+0000 7f5dc2ff5700 10 client.18819 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.708+0000 7f5dded9d700 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556fbf1aab50 0x556fbf1acf70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c37be6e2e90fca2 server_cookie=7fc3ca9a9f47ad1b in_seq=0 out_seq=0 2024-03-18T20:45:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.708+0000 7f5dc2ff5700 1 -- 172.21.15.46:0/2389937123 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556fbf0db780 con 0x556fbf1aab50 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.708+0000 7f5dc2ff5700 10 client.18819 handle_command_reply: tid=0 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5df11adb80 1 client.18819 shutdown 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5df11adb80 1 -- 172.21.15.46:0/2389937123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf190400 msgr2=0x556fbf19bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5df11adb80 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf190400 0x556fbf19bbc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400ea30 tx=0x7f5dd40047c0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5df11adb80 1 -- 172.21.15.46:0/2389937123 shutdown_connections 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5df11adb80 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556fbf1aab50 0x556fbf1acf70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5df11adb80 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf19c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5df11adb80 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf18fae0 0x556fbf19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5dc2ff5700 0 client.18819 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5df11adb80 1 --2- 172.21.15.46:0/2389937123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf190400 0x556fbf19bbc0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5df11adb80 1 -- 172.21.15.46:0/2389937123 >> 172.21.15.46:0/2389937123 conn(0x556fbf1062a0 msgr2=0x556fbf18bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5df11adb80 1 -- 172.21.15.46:0/2389937123 shutdown_connections 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5df11adb80 1 -- 172.21.15.46:0/2389937123 wait complete. 2024-03-18T20:45:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.709+0000 7f5df11adb80 20 client.18819 trim_cache size 0 max 16384 2024-03-18T20:45:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.727+0000 7f5df11adb80 1 Processor -- start 2024-03-18T20:45:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5df11adb80 1 -- start start 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5df11adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5df11adb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf190400 0x556fbf195850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5df11adb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf19b2a0 0x556fbf195d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5df11adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbf18ff50 con 0x556fbf18fae0 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5df11adb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbf19fbf0 con 0x556fbf190400 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5df11adb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbf198aa0 con 0x556fbf19b2a0 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5dded9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5dded9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38474/0 (socket says 172.21.15.46:38474) 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5dded9d700 1 -- 172.21.15.46:0/886683654 learned_addr learned my addr 172.21.15.46:0/886683654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5dde59c700 1 --2- 172.21.15.46:0/886683654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf190400 0x556fbf195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5ddf59e700 1 --2- 172.21.15.46:0/886683654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf19b2a0 0x556fbf195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5dded9d700 1 -- 172.21.15.46:0/886683654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf19b2a0 msgr2=0x556fbf195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5dded9d700 1 --2- 172.21.15.46:0/886683654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf19b2a0 0x556fbf195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5dded9d700 1 -- 172.21.15.46:0/886683654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf190400 msgr2=0x556fbf195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5dded9d700 1 --2- 172.21.15.46:0/886683654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf190400 0x556fbf195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5dded9d700 1 -- 172.21.15.46:0/886683654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fbf1b1ba0 con 0x556fbf18fae0 2024-03-18T20:45:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.728+0000 7f5dded9d700 1 --2- 172.21.15.46:0/886683654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf19f6b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00cb600 tx=0x7f5dd001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993ddefbf4ffd193 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.729+0000 7f5dc5ffb700 1 -- 172.21.15.46:0/886683654 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0098090 con 0x556fbf18fae0 2024-03-18T20:45:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.729+0000 7f5dc5ffb700 1 -- 172.21.15.46:0/886683654 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd00981f0 con 0x556fbf18fae0 2024-03-18T20:45:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.729+0000 7f5dc5ffb700 1 -- 172.21.15.46:0/886683654 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd009eab0 con 0x556fbf18fae0 2024-03-18T20:45:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.729+0000 7f5df11adb80 1 -- 172.21.15.46:0/886683654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 msgr2=0x556fbf19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.729+0000 7f5df11adb80 1 --2- 172.21.15.46:0/886683654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf19f6b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00cb600 tx=0x7f5dd001f000 comp rx=0 tx=0).stop 2024-03-18T20:45:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.729+0000 7f5df11adb80 1 -- 172.21.15.46:0/886683654 shutdown_connections 2024-03-18T20:45:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.729+0000 7f5df11adb80 1 --2- 172.21.15.46:0/886683654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf19f6b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.729+0000 7f5df11adb80 1 --2- 172.21.15.46:0/886683654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf19b2a0 0x556fbf195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.729+0000 7f5df11adb80 1 --2- 172.21.15.46:0/886683654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf190400 0x556fbf195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.729+0000 7f5df11adb80 1 -- 172.21.15.46:0/886683654 >> 172.21.15.46:0/886683654 conn(0x556fbf1062a0 msgr2=0x556fbf199a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.729+0000 7f5df11adb80 1 -- 172.21.15.46:0/886683654 shutdown_connections 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.729+0000 7f5df11adb80 1 -- 172.21.15.46:0/886683654 wait complete. 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5df11adb80 1 -- start start 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5df11adb80 1 Processor -- start 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5df11adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf1f8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5df11adb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf190400 0x556fbf1f8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5df11adb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf1f2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5df11adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbf1f3020 con 0x556fbf18fae0 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5df11adb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbf1f3190 con 0x556fbf19b2a0 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5df11adb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fbf1f3350 con 0x556fbf190400 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5df11adb80 10 client.0 _ll_register_callbacks cb 0x7f5dd400e5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5dded9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf1f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5dde59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf190400 0x556fbf1f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5dded9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf1f8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38490/0 (socket says 172.21.15.46:38490) 2024-03-18T20:45:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5ddf59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf1f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.730+0000 7f5dded9d700 1 -- 172.21.15.46:0/569709080 learned_addr learned my addr 172.21.15.46:0/569709080 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5dded9d700 1 -- 172.21.15.46:0/569709080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf190400 msgr2=0x556fbf1f8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5dded9d700 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf190400 0x556fbf1f8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5dded9d700 1 -- 172.21.15.46:0/569709080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 msgr2=0x556fbf1f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5dded9d700 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf1f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5dded9d700 1 -- 172.21.15.46:0/569709080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556fbef64430 con 0x556fbf18fae0 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5dde59c700 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf190400 0x556fbf1f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5ddf59e700 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf1f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5dded9d700 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf1f8450 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00983f0 tx=0x7f5dd009b300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be81f29210b3e674 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5dc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5dc2ff5700 1 -- 172.21.15.46:0/569709080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00a2eb0 con 0x556fbf18fae0 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5dc2ff5700 1 -- 172.21.15.46:0/569709080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5dd00672c0 con 0x556fbf18fae0 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5df11adb80 1 -- 172.21.15.46:0/569709080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5dd0099d20 con 0x556fbf18fae0 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5dc2ff5700 1 -- 172.21.15.46:0/569709080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dd009ae70 con 0x556fbf18fae0 2024-03-18T20:45:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5df11adb80 10 client.18825 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.731+0000 7f5df11adb80 1 -- 172.21.15.46:0/569709080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x556fbef10be0 con 0x556fbf18fae0 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.732+0000 7f5dc2ff5700 1 -- 172.21.15.46:0/569709080 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dd0099d20 con 0x556fbf18fae0 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.732+0000 7f5dc2ff5700 1 -- 172.21.15.46:0/569709080 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5dd009a8e0 con 0x556fbf18fae0 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.732+0000 7f5df11adb80 10 client.18825 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.732+0000 7f5df11adb80 10 client.18825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.732+0000 7f5df11adb80 20 client.18825 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.732+0000 7f5df11adb80 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556fbf1ffe20 0x556fbf202210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.732+0000 7f5df11adb80 4 client.18825 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.732+0000 7f5df11adb80 1 -- 172.21.15.46:0/569709080 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556fbef31d70 con 0x556fbf1ffe20 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.732+0000 7f5dde59c700 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556fbf1ffe20 0x556fbf202210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.732+0000 7f5dc2ff5700 10 client.18825 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.732+0000 7f5dde59c700 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556fbf1ffe20 0x556fbf202210 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a1bccf2661e961 server_cookie=e0393fe49c421daf in_seq=0 out_seq=0 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5dc2ff5700 1 -- 172.21.15.46:0/569709080 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556fbef31d70 con 0x556fbf1ffe20 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5dc2ff5700 10 client.18825 handle_command_reply: tid=0 2024-03-18T20:45:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5df11adb80 1 client.18825 shutdown 2024-03-18T20:45:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5df11adb80 1 -- 172.21.15.46:0/569709080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 msgr2=0x556fbf1f8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5df11adb80 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf1f8450 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00983f0 tx=0x7f5dd009b300 comp rx=0 tx=0).stop 2024-03-18T20:45:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5df11adb80 1 -- 172.21.15.46:0/569709080 shutdown_connections 2024-03-18T20:45:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5df11adb80 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556fbf1ffe20 0x556fbf202210 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5df11adb80 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556fbf18fae0 0x556fbf1f8450 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5dc2ff5700 0 client.18825 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5df11adb80 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fbf19b2a0 0x556fbf1f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5df11adb80 1 --2- 172.21.15.46:0/569709080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556fbf190400 0x556fbf1f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5df11adb80 1 -- 172.21.15.46:0/569709080 >> 172.21.15.46:0/569709080 conn(0x556fbf1062a0 msgr2=0x556fbf198d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5df11adb80 1 -- 172.21.15.46:0/569709080 shutdown_connections 2024-03-18T20:45:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5df11adb80 1 -- 172.21.15.46:0/569709080 wait complete. 2024-03-18T20:45:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5df11adb80 20 client.18825 trim_cache size 0 max 16384 2024-03-18T20:45:20.743 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:45:20.743 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:45:20.743 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "869d0f85-1944-45cc-b9a5-96d93fadbf1d", 2024-03-18T20:45:20.743 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:45:20.743 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:45:20.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5de5018700 1 -- 172.21.15.46:0/2533606133 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5dc80a1290 msgr2=0x7f5dc80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5de5018700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5dc80a1290 0x7f5dc80a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5de01013a0 tx=0x7f5dd4009310 comp rx=0 tx=0).stop 2024-03-18T20:45:20.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.734+0000 7f5de5018700 1 -- 172.21.15.46:0/2533606133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0069540 msgr2=0x7f5de00ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.735+0000 7f5de5018700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0069540 0x7f5de00ffc70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000fa00 tx=0x7f5dd000fa30 comp rx=0 tx=0).stop 2024-03-18T20:45:20.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.735+0000 7f5de5018700 1 -- 172.21.15.46:0/2533606133 shutdown_connections 2024-03-18T20:45:20.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.735+0000 7f5de5018700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5dc80a1290 0x7f5dc80a3750 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.735+0000 7f5de5018700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5de01006f0 0x7f5de01a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.735+0000 7f5de5018700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5de0069540 0x7f5de00ffc70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.735+0000 7f5de5018700 1 --2- 172.21.15.46:0/2533606133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5de01060a0 0x7f5de01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.735+0000 7f5de5018700 1 -- 172.21.15.46:0/2533606133 >> 172.21.15.46:0/2533606133 conn(0x7f5de0076d50 msgr2=0x7f5de00fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:20.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.735+0000 7f5de5018700 1 -- 172.21.15.46:0/2533606133 shutdown_connections 2024-03-18T20:45:20.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.735+0000 7f5de5018700 1 -- 172.21.15.46:0/2533606133 wait complete. 2024-03-18T20:45:20.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:45:20.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:45:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.908+0000 7fd04aba6700 1 -- 172.21.15.46:0/121564215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd044068810 msgr2=0x7fd04410db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.908+0000 7fd04aba6700 1 --2- 172.21.15.46:0/121564215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd044068810 0x7fd04410db00 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00b0a0 tx=0x7fd03c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.908+0000 7fd04aba6700 1 -- 172.21.15.46:0/121564215 shutdown_connections 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.908+0000 7fd04aba6700 1 --2- 172.21.15.46:0/121564215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd044068810 0x7fd04410db00 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.908+0000 7fd04aba6700 1 --2- 172.21.15.46:0/121564215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd044106e60 0x7fd044107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.908+0000 7fd04aba6700 1 --2- 172.21.15.46:0/121564215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd044067e50 0x7fd0440682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.908+0000 7fd04aba6700 1 -- 172.21.15.46:0/121564215 >> 172.21.15.46:0/121564215 conn(0x7fd044076d40 msgr2=0x7fd044077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.908+0000 7fd04aba6700 1 -- 172.21.15.46:0/121564215 shutdown_connections 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.908+0000 7fd04aba6700 1 -- 172.21.15.46:0/121564215 wait complete. 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.909+0000 7fd04aba6700 1 Processor -- start 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.909+0000 7fd04aba6700 1 -- start start 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.909+0000 7fd04aba6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd044067e50 0x7fd0441971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.909+0000 7fd04aba6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd044068810 0x7fd0441976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.909+0000 7fd04aba6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd044106e60 0x7fd04419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.909+0000 7fd04aba6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04419c0d0 con 0x7fd044067e50 2024-03-18T20:45:20.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.909+0000 7fd04aba6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04419c240 con 0x7fd044106e60 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.909+0000 7fd04aba6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04419c400 con 0x7fd044068810 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd048942700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd044067e50 0x7fd0441971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd048942700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd044067e50 0x7fd0441971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38510/0 (socket says 172.21.15.46:38510) 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd048942700 1 -- 172.21.15.46:0/426970969 learned_addr learned my addr 172.21.15.46:0/426970969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd049143700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd044106e60 0x7fd04419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd043fff700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd044068810 0x7fd0441976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd049143700 1 -- 172.21.15.46:0/426970969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd044068810 msgr2=0x7fd0441976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd049143700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd044068810 0x7fd0441976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd049143700 1 -- 172.21.15.46:0/426970969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd044067e50 msgr2=0x7fd0441971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd049143700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd044067e50 0x7fd0441971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd049143700 1 -- 172.21.15.46:0/426970969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd038009520 con 0x7fd044106e60 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd048942700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd044067e50 0x7fd0441971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd049143700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd044106e60 0x7fd04419ba90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00b0a0 tx=0x7fd03c0078e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:20.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd041ffb700 1 -- 172.21.15.46:0/426970969 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c00e040 con 0x7fd044106e60 2024-03-18T20:45:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd041ffb700 1 -- 172.21.15.46:0/426970969 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd03c01d470 con 0x7fd044106e60 2024-03-18T20:45:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd04aba6700 1 -- 172.21.15.46:0/426970969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c009d00 con 0x7fd044106e60 2024-03-18T20:45:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.910+0000 7fd041ffb700 1 -- 172.21.15.46:0/426970969 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c005260 con 0x7fd044106e60 2024-03-18T20:45:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.911+0000 7fd04aba6700 1 -- 172.21.15.46:0/426970969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0441a3d40 con 0x7fd044106e60 2024-03-18T20:45:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.914+0000 7fd041ffb700 1 -- 172.21.15.46:0/426970969 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd03c01d5e0 con 0x7fd044106e60 2024-03-18T20:45:20.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.914+0000 7fd041ffb700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0340a1290 0x7fd0340a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd041ffb700 1 -- 172.21.15.46:0/426970969 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd03c0cdee0 con 0x7fd044106e60 2024-03-18T20:45:20.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd056d3bb80 1 Processor -- start 2024-03-18T20:45:20.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd048942700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0340a1290 0x7fd0340a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd056d3bb80 1 -- start start 2024-03-18T20:45:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd056d3bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd056d3bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1dace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd056d3bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1dd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd056d3bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5c129990 con 0x560e5c1d0380 2024-03-18T20:45:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd056d3bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5c127350 con 0x560e5c1cfa60 2024-03-18T20:45:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd056d3bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5c122c40 con 0x560e5c1db220 2024-03-18T20:45:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd043fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1dace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd048942700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd043fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1dace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38520/0 (socket says 172.21.15.46:38520) 2024-03-18T20:45:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.915+0000 7fd043fff700 1 -- 172.21.15.46:0/583844142 learned_addr learned my addr 172.21.15.46:0/583844142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.916+0000 7fd048942700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0340a1290 0x7fd0340a3750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd038009fd0 tx=0x7fd038009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.916+0000 7fd049143700 1 --2- 172.21.15.46:0/583844142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1dd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.916+0000 7fd048942700 1 -- 172.21.15.46:0/583844142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 msgr2=0x560e5c1dd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.916+0000 7fd048942700 1 --2- 172.21.15.46:0/583844142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1dd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.916+0000 7fd048942700 1 -- 172.21.15.46:0/583844142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 msgr2=0x560e5c1dace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.916+0000 7fd048942700 1 --2- 172.21.15.46:0/583844142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1dace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.916+0000 7fd048942700 1 -- 172.21.15.46:0/583844142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e5bfa43b0 con 0x560e5c1cfa60 2024-03-18T20:45:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.916+0000 7fd048942700 1 --2- 172.21.15.46:0/583844142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1cfe40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd03800f770 tx=0x7fd038014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7185d75c4e1e3a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.916+0000 7fd02effd700 1 -- 172.21.15.46:0/583844142 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038026640 con 0x560e5c1cfa60 2024-03-18T20:45:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.916+0000 7fd02effd700 1 -- 172.21.15.46:0/583844142 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd03801c8d0 con 0x560e5c1cfa60 2024-03-18T20:45:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.916+0000 7fd02effd700 1 -- 172.21.15.46:0/583844142 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03801b790 con 0x560e5c1cfa60 2024-03-18T20:45:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.917+0000 7fd056d3bb80 1 -- 172.21.15.46:0/583844142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 msgr2=0x560e5c1cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.917+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/583844142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1cfe40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd03800f770 tx=0x7fd038014000 comp rx=0 tx=0).stop 2024-03-18T20:45:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.917+0000 7fd056d3bb80 1 -- 172.21.15.46:0/583844142 shutdown_connections 2024-03-18T20:45:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.917+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/583844142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1dd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.917+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/583844142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1cfe40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.917+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/583844142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1dace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.917+0000 7fd056d3bb80 1 -- 172.21.15.46:0/583844142 >> 172.21.15.46:0/583844142 conn(0x560e5c146220 msgr2=0x560e5c1cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.917+0000 7fd056d3bb80 1 -- 172.21.15.46:0/583844142 shutdown_connections 2024-03-18T20:45:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.917+0000 7fd056d3bb80 1 -- 172.21.15.46:0/583844142 wait complete. 2024-03-18T20:45:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.917+0000 7fd056d3bb80 1 -- start start 2024-03-18T20:45:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.918+0000 7fd056d3bb80 1 Processor -- start 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.918+0000 7fd056d3bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1dac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.918+0000 7fd056d3bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1d7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.918+0000 7fd056d3bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1d85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.918+0000 7fd056d3bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5c0bed80 con 0x560e5c1d0380 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.918+0000 7fd056d3bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5c0bea20 con 0x560e5c1cfa60 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.918+0000 7fd056d3bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5bfa66a0 con 0x560e5c1db220 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.918+0000 7fd056d3bb80 10 client.0 _ll_register_callbacks cb 0x560e5bf2a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.918+0000 7fd043fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1d7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.918+0000 7fd043fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1d7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38524/0 (socket says 172.21.15.46:38524) 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.918+0000 7fd043fff700 1 -- 172.21.15.46:0/439640462 learned_addr learned my addr 172.21.15.46:0/439640462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd049143700 1 --2- 172.21.15.46:0/439640462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1d85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd043fff700 1 -- 172.21.15.46:0/439640462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 msgr2=0x560e5c1d85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd043fff700 1 --2- 172.21.15.46:0/439640462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1d85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd043fff700 1 -- 172.21.15.46:0/439640462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 msgr2=0x560e5c1dac30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd043fff700 1 --2- 172.21.15.46:0/439640462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1dac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd043fff700 1 -- 172.21.15.46:0/439640462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd038017050 con 0x560e5c1d0380 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd043fff700 1 --2- 172.21.15.46:0/439640462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1d7f20 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fd030012840 tx=0x7fd030012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6d7115a482142 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd023fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd023fff700 1 -- 172.21.15.46:0/439640462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030017070 con 0x560e5c1d0380 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd023fff700 1 -- 172.21.15.46:0/439640462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd030004050 con 0x560e5c1d0380 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd056d3bb80 1 -- 172.21.15.46:0/439640462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e5c1c3ad0 con 0x560e5c1d0380 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd023fff700 1 -- 172.21.15.46:0/439640462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd030004250 con 0x560e5c1d0380 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd056d3bb80 10 client.18837 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.919+0000 7fd056d3bb80 1 -- 172.21.15.46:0/439640462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560e5bf50b60 con 0x560e5c1d0380 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.920+0000 7fd023fff700 1 -- 172.21.15.46:0/439640462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e5c1c3ad0 con 0x560e5c1d0380 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.920+0000 7fd023fff700 1 -- 172.21.15.46:0/439640462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd03000f650 con 0x560e5c1d0380 2024-03-18T20:45:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.920+0000 7fd056d3bb80 10 client.18837 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.920+0000 7fd056d3bb80 10 client.18837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.920+0000 7fd056d3bb80 20 client.18837 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.920+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/439640462 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560e5c1eae60 0x560e5c1ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.920+0000 7fd056d3bb80 4 client.18837 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.920+0000 7fd056d3bb80 1 -- 172.21.15.46:0/439640462 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e5c11b700 con 0x560e5c1eae60 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.920+0000 7fd048942700 1 --2- 172.21.15.46:0/439640462 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560e5c1eae60 0x560e5c1ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.920+0000 7fd023fff700 10 client.18837 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd048942700 1 --2- 172.21.15.46:0/439640462 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560e5c1eae60 0x560e5c1ed280 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2400c445cca8f4d1 server_cookie=65023c26576e84d6 in_seq=0 out_seq=0 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd023fff700 1 -- 172.21.15.46:0/439640462 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560e5c11b700 con 0x560e5c1eae60 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd023fff700 10 client.18837 handle_command_reply: tid=0 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd056d3bb80 1 client.18837 shutdown 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd056d3bb80 1 -- 172.21.15.46:0/439640462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 msgr2=0x560e5c1d7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/439640462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1d7f20 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fd030012840 tx=0x7fd030012b50 comp rx=0 tx=0).stop 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd056d3bb80 1 -- 172.21.15.46:0/439640462 shutdown_connections 2024-03-18T20:45:20.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/439640462 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560e5c1eae60 0x560e5c1ed280 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/439640462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1d85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/439640462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1dac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd023fff700 0 client.18837 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/439640462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1d7f20 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.921+0000 7fd056d3bb80 1 -- 172.21.15.46:0/439640462 >> 172.21.15.46:0/439640462 conn(0x560e5c146220 msgr2=0x560e5c1cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.922+0000 7fd056d3bb80 1 -- 172.21.15.46:0/439640462 shutdown_connections 2024-03-18T20:45:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.922+0000 7fd056d3bb80 1 -- 172.21.15.46:0/439640462 wait complete. 2024-03-18T20:45:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.922+0000 7fd056d3bb80 20 client.18837 trim_cache size 0 max 16384 2024-03-18T20:45:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.941+0000 7fd056d3bb80 1 Processor -- start 2024-03-18T20:45:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.941+0000 7fd056d3bb80 1 -- start start 2024-03-18T20:45:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.941+0000 7fd056d3bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1d85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.941+0000 7fd056d3bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1e1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.941+0000 7fd056d3bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1d7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.941+0000 7fd056d3bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5c196940 con 0x560e5c1d0380 2024-03-18T20:45:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.941+0000 7fd056d3bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5c1d0200 con 0x560e5c1cfa60 2024-03-18T20:45:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.941+0000 7fd056d3bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5c1e8530 con 0x560e5c1db220 2024-03-18T20:45:20.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd048942700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1d85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd043fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1e1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd049143700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1d7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd049143700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1d7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59486/0 (socket says 172.21.15.46:59486) 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd043fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1e1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38530/0 (socket says 172.21.15.46:38530) 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd048942700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1d85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:36428/0 (socket says 172.21.15.46:36428) 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd049143700 1 -- 172.21.15.46:0/919708446 learned_addr learned my addr 172.21.15.46:0/919708446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd049143700 1 -- 172.21.15.46:0/919708446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 msgr2=0x560e5c1d85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd049143700 1 --2- 172.21.15.46:0/919708446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1d85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd049143700 1 -- 172.21.15.46:0/919708446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 msgr2=0x560e5c1e1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd049143700 1 --2- 172.21.15.46:0/919708446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1e1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd049143700 1 -- 172.21.15.46:0/919708446 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e5c0e8ea0 con 0x560e5c1db220 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd049143700 1 --2- 172.21.15.46:0/919708446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1d7460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0cb670 tx=0x7fd03c00b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a04a45bc574ae10 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:20.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd02f7fe700 1 -- 172.21.15.46:0/919708446 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c0aeea0 con 0x560e5c1db220 2024-03-18T20:45:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd02f7fe700 1 -- 172.21.15.46:0/919708446 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd03c09a9c0 con 0x560e5c1db220 2024-03-18T20:45:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.942+0000 7fd02f7fe700 1 -- 172.21.15.46:0/919708446 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c099400 con 0x560e5c1db220 2024-03-18T20:45:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.943+0000 7fd056d3bb80 1 -- 172.21.15.46:0/919708446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 msgr2=0x560e5c1d7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.943+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/919708446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1d7460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0cb670 tx=0x7fd03c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.943+0000 7fd056d3bb80 1 -- 172.21.15.46:0/919708446 shutdown_connections 2024-03-18T20:45:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.943+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/919708446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1db220 0x560e5c1d7460 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.943+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/919708446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1cfa60 0x560e5c1d85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.943+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/919708446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1d0380 0x560e5c1e1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.943+0000 7fd056d3bb80 1 -- 172.21.15.46:0/919708446 >> 172.21.15.46:0/919708446 conn(0x560e5c146220 msgr2=0x560e5c1ca750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:20.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.943+0000 7fd056d3bb80 1 -- 172.21.15.46:0/919708446 shutdown_connections 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.943+0000 7fd056d3bb80 1 -- 172.21.15.46:0/919708446 wait complete. 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd056d3bb80 1 -- start start 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd056d3bb80 1 Processor -- start 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd056d3bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1cfa60 0x560e5c1df730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd056d3bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1d0380 0x560e5c1dfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd056d3bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1db220 0x560e5c1e01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd056d3bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5c23dde0 con 0x560e5c1cfa60 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd056d3bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5c23df50 con 0x560e5c1db220 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd056d3bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5c23e110 con 0x560e5c1d0380 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd056d3bb80 10 client.0 _ll_register_callbacks cb 0x7fd030009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd043fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1d0380 0x560e5c1dfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd048942700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1cfa60 0x560e5c1df730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd043fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1d0380 0x560e5c1dfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59492/0 (socket says 172.21.15.46:59492) 2024-03-18T20:45:20.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.944+0000 7fd043fff700 1 -- 172.21.15.46:0/855788031 learned_addr learned my addr 172.21.15.46:0/855788031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd049143700 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1db220 0x560e5c1e01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd043fff700 1 -- 172.21.15.46:0/855788031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1db220 msgr2=0x560e5c1e01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd043fff700 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1db220 0x560e5c1e01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd043fff700 1 -- 172.21.15.46:0/855788031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1cfa60 msgr2=0x560e5c1df730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd043fff700 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1cfa60 0x560e5c1df730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd043fff700 1 -- 172.21.15.46:0/855788031 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0441a3d40 con 0x560e5c1d0380 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd049143700 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1db220 0x560e5c1e01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd043fff700 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1d0380 0x560e5c1dfc70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x560e5c1d0b90 tx=0x7fd030009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20653f5cacb14ac4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd023fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd023fff700 1 -- 172.21.15.46:0/855788031 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030017040 con 0x560e5c1d0380 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd023fff700 1 -- 172.21.15.46:0/855788031 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd0300052d0 con 0x560e5c1d0380 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd056d3bb80 1 -- 172.21.15.46:0/855788031 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd03801c8d0 con 0x560e5c1d0380 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd023fff700 1 -- 172.21.15.46:0/855788031 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd03000f7d0 con 0x560e5c1d0380 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd056d3bb80 10 client.28498 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:20.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.945+0000 7fd056d3bb80 1 -- 172.21.15.46:0/855788031 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560e5bfa8cd0 con 0x560e5c1d0380 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.946+0000 7fd023fff700 1 -- 172.21.15.46:0/855788031 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd03801c8d0 con 0x560e5c1d0380 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.946+0000 7fd023fff700 1 -- 172.21.15.46:0/855788031 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd030025450 con 0x560e5c1d0380 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.946+0000 7fd056d3bb80 10 client.28498 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.946+0000 7fd056d3bb80 10 client.28498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.946+0000 7fd056d3bb80 20 client.28498 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.946+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560e5c23f8b0 0x560e5c241cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.946+0000 7fd056d3bb80 4 client.28498 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.946+0000 7fd056d3bb80 1 -- 172.21.15.46:0/855788031 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560e5bf71cf0 con 0x560e5c23f8b0 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.946+0000 7fd048942700 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560e5c23f8b0 0x560e5c241cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.946+0000 7fd048942700 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560e5c23f8b0 0x560e5c241cd0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f35dd9975f8f2b8 server_cookie=5fb46baefd43165a in_seq=0 out_seq=0 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.946+0000 7fd023fff700 10 client.28498 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd023fff700 1 -- 172.21.15.46:0/855788031 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560e5bf71cf0 con 0x560e5c23f8b0 2024-03-18T20:45:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd023fff700 10 client.28498 handle_command_reply: tid=0 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd056d3bb80 1 client.28498 shutdown 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd056d3bb80 1 -- 172.21.15.46:0/855788031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1d0380 msgr2=0x560e5c1dfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1d0380 0x560e5c1dfc70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x560e5c1d0b90 tx=0x7fd030009760 comp rx=0 tx=0).stop 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd056d3bb80 1 -- 172.21.15.46:0/855788031 shutdown_connections 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560e5c23f8b0 0x560e5c241cd0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560e5c1cfa60 0x560e5c1df730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e5c1db220 0x560e5c1e01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd023fff700 0 client.28498 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd056d3bb80 1 --2- 172.21.15.46:0/855788031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e5c1d0380 0x560e5c1dfc70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd056d3bb80 1 -- 172.21.15.46:0/855788031 >> 172.21.15.46:0/855788031 conn(0x560e5c146220 msgr2=0x560e5c1cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd056d3bb80 1 -- 172.21.15.46:0/855788031 shutdown_connections 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd056d3bb80 1 -- 172.21.15.46:0/855788031 wait complete. 2024-03-18T20:45:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.947+0000 7fd056d3bb80 20 client.28498 trim_cache size 0 max 16384 2024-03-18T20:45:20.956 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:45:20.956 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:45:20.956 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:45:20.956 INFO:teuthology.orchestra.run.smithi046.stdout: "869d0f85-1944-45cc-b9a5-96d93fadbf1d": { 2024-03-18T20:45:20.956 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:45:20.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "869d0f85-1944-45cc-b9a5-96d93fadbf1d", 2024-03-18T20:45:20.956 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:45:20.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:45:20.956 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:45:20.956 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:45:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.948+0000 7fd04aba6700 1 -- 172.21.15.46:0/426970969 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0340a1290 msgr2=0x7fd0340a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.948+0000 7fd04aba6700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0340a1290 0x7fd0340a3750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd038009fd0 tx=0x7fd038009260 comp rx=0 tx=0).stop 2024-03-18T20:45:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.948+0000 7fd04aba6700 1 -- 172.21.15.46:0/426970969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd044106e60 msgr2=0x7fd04419ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.948+0000 7fd04aba6700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd044106e60 0x7fd04419ba90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00b0a0 tx=0x7fd03c0078e0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.948+0000 7fd04aba6700 1 -- 172.21.15.46:0/426970969 shutdown_connections 2024-03-18T20:45:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.948+0000 7fd04aba6700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0340a1290 0x7fd0340a3750 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.948+0000 7fd04aba6700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd044067e50 0x7fd0441971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.948+0000 7fd04aba6700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd044106e60 0x7fd04419ba90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.948+0000 7fd04aba6700 1 --2- 172.21.15.46:0/426970969 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd044068810 0x7fd0441976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.948+0000 7fd04aba6700 1 -- 172.21.15.46:0/426970969 >> 172.21.15.46:0/426970969 conn(0x7fd044076d40 msgr2=0x7fd0440fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.948+0000 7fd04aba6700 1 -- 172.21.15.46:0/426970969 shutdown_connections 2024-03-18T20:45:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:20.948+0000 7fd04aba6700 1 -- 172.21.15.46:0/426970969 wait complete. 2024-03-18T20:45:20.965 INFO:tasks.cephfs.filesystem:scrub status for tag:869d0f85-1944-45cc-b9a5-96d93fadbf1d - {'path': '/', 'tag': '869d0f85-1944-45cc-b9a5-96d93fadbf1d', 'options': 'recursive,force'} 2024-03-18T20:45:21.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:20 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:21.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:20 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:21.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:20 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:21.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:20 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:21.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:20 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:21.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:20 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:21.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 337511 3.66 MB/sec execute 376 sec latency 10.738 ms 2024-03-18T20:45:22.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:45:22.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:22.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:22.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:21 smithi155 ceph-mon[48344]: pgmap v287: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-03-18T20:45:22.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:45:22.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:22.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:22.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:22.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:22.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:22.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:22.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:22.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:22.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:21 smithi046 ceph-mon[43495]: pgmap v287: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-03-18T20:45:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:45:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:22.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:22.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:22.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:22.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:22.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:21 smithi073 ceph-mon[47384]: pgmap v287: 97 pgs: 97 active+clean; 131 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-03-18T20:45:22.709 INFO:tasks.workunit.client.0.smithi046.stdout: 1 338670 3.67 MB/sec execute 377 sec latency 166.379 ms 2024-03-18T20:45:23.709 INFO:tasks.workunit.client.0.smithi046.stdout: 1 338670 3.66 MB/sec execute 378 sec latency 1166.491 ms 2024-03-18T20:45:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.390+0000 7f054d9b5700 1 -- 172.21.15.46:0/2909542646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548069530 msgr2=0x7f05480699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.390+0000 7f054d9b5700 1 --2- 172.21.15.46:0/2909542646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548069530 0x7f05480699b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f053c009990 tx=0x7f053c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.390+0000 7f054d9b5700 1 -- 172.21.15.46:0/2909542646 shutdown_connections 2024-03-18T20:45:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.390+0000 7f054d9b5700 1 --2- 172.21.15.46:0/2909542646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0548069ef0 0x7f054810dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.390+0000 7f054d9b5700 1 --2- 172.21.15.46:0/2909542646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548106e10 0x7f05481071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.390+0000 7f054d9b5700 1 --2- 172.21.15.46:0/2909542646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548069530 0x7f05480699b0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.390+0000 7f054d9b5700 1 -- 172.21.15.46:0/2909542646 >> 172.21.15.46:0/2909542646 conn(0x7f0548076db0 msgr2=0x7f05480771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.390+0000 7f054d9b5700 1 -- 172.21.15.46:0/2909542646 shutdown_connections 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.390+0000 7f054d9b5700 1 -- 172.21.15.46:0/2909542646 wait complete. 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.390+0000 7f054d9b5700 1 Processor -- start 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f054d9b5700 1 -- start start 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f054d9b5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0548069530 0x7f0548197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f054d9b5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548069ef0 0x7f05481975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f054d9b5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548106e10 0x7f054819b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f054d9b5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054819bfa0 con 0x7f0548069ef0 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f054d9b5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054819c110 con 0x7f0548069530 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f054d9b5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054819c2d0 con 0x7f0548106e10 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f05467fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548069ef0 0x7f05481975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f05467fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548069ef0 0x7f05481975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38562/0 (socket says 172.21.15.46:38562) 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f0546ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0548069530 0x7f0548197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f05467fc700 1 -- 172.21.15.46:0/2537858068 learned_addr learned my addr 172.21.15.46:0/2537858068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:24.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f05477fe700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548106e10 0x7f054819b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.391+0000 7f0546ffd700 1 -- 172.21.15.46:0/2537858068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548106e10 msgr2=0x7f054819b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.392+0000 7f0546ffd700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548106e10 0x7f054819b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.392+0000 7f0546ffd700 1 -- 172.21.15.46:0/2537858068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548069ef0 msgr2=0x7f05481975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.392+0000 7f0546ffd700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548069ef0 0x7f05481975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.392+0000 7f0546ffd700 1 -- 172.21.15.46:0/2537858068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c0095f0 con 0x7f0548069530 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.392+0000 7f05477fe700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548106e10 0x7f054819b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.392+0000 7f05467fc700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548069ef0 0x7f05481975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.392+0000 7f0546ffd700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0548069530 0x7f0548197080 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f053800e9f0 tx=0x7f053800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.392+0000 7f054c9b3700 1 -- 172.21.15.46:0/2537858068 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053800eee0 con 0x7f0548069530 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.392+0000 7f054d9b5700 1 -- 172.21.15.46:0/2537858068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054819c5b0 con 0x7f0548069530 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.392+0000 7f054c9b3700 1 -- 172.21.15.46:0/2537858068 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f053800cbd0 con 0x7f0548069530 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.392+0000 7f054c9b3700 1 -- 172.21.15.46:0/2537858068 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538018680 con 0x7f0548069530 2024-03-18T20:45:24.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.392+0000 7f054d9b5700 1 -- 172.21.15.46:0/2537858068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05481a3df0 con 0x7f0548069530 2024-03-18T20:45:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.395+0000 7f054c9b3700 1 -- 172.21.15.46:0/2537858068 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f053800cd40 con 0x7f0548069530 2024-03-18T20:45:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.396+0000 7f054c9b3700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f05300a1290 0x7f05300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.396+0000 7f05467fc700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f05300a1290 0x7f05300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.396+0000 7f054c9b3700 1 -- 172.21.15.46:0/2537858068 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f0538014070 con 0x7f0548069530 2024-03-18T20:45:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.396+0000 7f0559b4ab80 1 Processor -- start 2024-03-18T20:45:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f0559b4ab80 1 -- start start 2024-03-18T20:45:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f05467fc700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f05300a1290 0x7f05300a3750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f05481985d0 tx=0x7f053c005bc0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:24.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f0559b4ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9570 0x55eecf2e9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f0559b4ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 0x55eecf2f4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f0559b4ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf2f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f0559b4ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecf243930 con 0x55eecf2e9e90 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f0559b4ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecf2412f0 con 0x55eecf2e9570 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f0559b4ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecf23cbe0 con 0x55eecf2f4db0 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f05467fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 0x55eecf2f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f05467fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 0x55eecf2f4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38574/0 (socket says 172.21.15.46:38574) 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f05467fc700 1 -- 172.21.15.46:0/4200356173 learned_addr learned my addr 172.21.15.46:0/4200356173 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f0546ffd700 1 --2- 172.21.15.46:0/4200356173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9570 0x55eecf2e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f05477fe700 1 --2- 172.21.15.46:0/4200356173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf2f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f05467fc700 1 -- 172.21.15.46:0/4200356173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 msgr2=0x55eecf2f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f05467fc700 1 --2- 172.21.15.46:0/4200356173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf2f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f05467fc700 1 -- 172.21.15.46:0/4200356173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9570 msgr2=0x55eecf2e9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:24.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f05467fc700 1 --2- 172.21.15.46:0/4200356173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9570 0x55eecf2e9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f05467fc700 1 -- 172.21.15.46:0/4200356173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eecf0be350 con 0x55eecf2e9e90 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.397+0000 7f05467fc700 1 --2- 172.21.15.46:0/4200356173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 0x55eecf2f4870 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f053c0098e0 tx=0x7f053c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402ce791077b6b57 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.398+0000 7f052dffb700 1 -- 172.21.15.46:0/4200356173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c02b070 con 0x55eecf2e9e90 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.398+0000 7f052dffb700 1 -- 172.21.15.46:0/4200356173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f053c01e040 con 0x55eecf2e9e90 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.398+0000 7f052dffb700 1 -- 172.21.15.46:0/4200356173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c024d30 con 0x55eecf2e9e90 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.398+0000 7f0559b4ab80 1 -- 172.21.15.46:0/4200356173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 msgr2=0x55eecf2f4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.398+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/4200356173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 0x55eecf2f4870 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f053c0098e0 tx=0x7f053c020000 comp rx=0 tx=0).stop 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.398+0000 7f0559b4ab80 1 -- 172.21.15.46:0/4200356173 shutdown_connections 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.398+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/4200356173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf2f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.398+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/4200356173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9570 0x55eecf2e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.398+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/4200356173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 0x55eecf2f4870 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.398+0000 7f0559b4ab80 1 -- 172.21.15.46:0/4200356173 >> 172.21.15.46:0/4200356173 conn(0x55eecf2601c0 msgr2=0x55eecf2e5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.398+0000 7f0559b4ab80 1 -- 172.21.15.46:0/4200356173 shutdown_connections 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.398+0000 7f0559b4ab80 1 -- 172.21.15.46:0/4200356173 wait complete. 2024-03-18T20:45:24.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.399+0000 7f0559b4ab80 1 -- start start 2024-03-18T20:45:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.399+0000 7f0559b4ab80 1 Processor -- start 2024-03-18T20:45:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.399+0000 7f0559b4ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 0x55eecf2f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.399+0000 7f0559b4ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9e90 0x55eecf2f2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.399+0000 7f0559b4ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf2f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.399+0000 7f0559b4ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecf1d8d20 con 0x55eecf2e9570 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.399+0000 7f0559b4ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecf1d89c0 con 0x55eecf2e9e90 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.399+0000 7f0559b4ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecf0c0640 con 0x55eecf2f4db0 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0559b4ab80 10 client.0 _ll_register_callbacks cb 0x55eecf044340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0546ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 0x55eecf2f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0546ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 0x55eecf2f1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38580/0 (socket says 172.21.15.46:38580) 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0546ffd700 1 -- 172.21.15.46:0/2409268750 learned_addr learned my addr 172.21.15.46:0/2409268750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f05477fe700 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf2f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f05467fc700 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9e90 0x55eecf2f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0546ffd700 1 -- 172.21.15.46:0/2409268750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 msgr2=0x55eecf2f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0546ffd700 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf2f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0546ffd700 1 -- 172.21.15.46:0/2409268750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9e90 msgr2=0x55eecf2f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0546ffd700 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9e90 0x55eecf2f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0546ffd700 1 -- 172.21.15.46:0/2409268750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054810e440 con 0x55eecf2e9570 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f05477fe700 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf2f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0546ffd700 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 0x55eecf2f1bf0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f0538009800 tx=0x7f053809a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a858f90eb12d2390 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0526ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0526ffd700 1 -- 172.21.15.46:0/2409268750 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05380b1070 con 0x55eecf2e9570 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0526ffd700 1 -- 172.21.15.46:0/2409268750 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f05380a2410 con 0x55eecf2e9570 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2409268750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eecf2dd6c0 con 0x55eecf2e9570 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0526ffd700 1 -- 172.21.15.46:0/2409268750 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f053809a400 con 0x55eecf2e9570 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0559b4ab80 10 client.18855 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.400+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2409268750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55eecf06ab00 con 0x55eecf2e9570 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.401+0000 7f0526ffd700 1 -- 172.21.15.46:0/2409268750 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0538063080 con 0x55eecf2e9570 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.401+0000 7f0526ffd700 1 -- 172.21.15.46:0/2409268750 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f05380a1f80 con 0x55eecf2e9570 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.401+0000 7f0559b4ab80 10 client.18855 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.401+0000 7f0559b4ab80 10 client.18855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:24.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.401+0000 7f0559b4ab80 20 client.18855 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.401+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55eecf3049b0 0x55eecf306da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.401+0000 7f0559b4ab80 4 client.18855 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.401+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2409268750 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eecf2356a0 con 0x55eecf3049b0 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.401+0000 7f05467fc700 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55eecf3049b0 0x55eecf306da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f05467fc700 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55eecf3049b0 0x55eecf306da0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac39d4206046b0bf server_cookie=bc21daf734d8c036 in_seq=0 out_seq=0 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f0526ffd700 10 client.18855 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f0526ffd700 1 -- 172.21.15.46:0/2409268750 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eecf2356a0 con 0x55eecf3049b0 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f0526ffd700 10 client.18855 handle_command_reply: tid=0 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f0559b4ab80 1 client.18855 shutdown 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2409268750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 msgr2=0x55eecf2f1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 0x55eecf2f1bf0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f0538009800 tx=0x7f053809a740 comp rx=0 tx=0).stop 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2409268750 shutdown_connections 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55eecf3049b0 0x55eecf306da0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 0x55eecf2f1bf0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf2f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f0526ffd700 0 client.18855 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.402+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/2409268750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9e90 0x55eecf2f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.403+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2409268750 >> 172.21.15.46:0/2409268750 conn(0x55eecf2601c0 msgr2=0x55eecf2e5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.403+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2409268750 shutdown_connections 2024-03-18T20:45:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.403+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2409268750 wait complete. 2024-03-18T20:45:24.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.403+0000 7f0559b4ab80 20 client.18855 trim_cache size 0 max 16384 2024-03-18T20:45:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0559b4ab80 1 Processor -- start 2024-03-18T20:45:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0559b4ab80 1 -- start start 2024-03-18T20:45:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0559b4ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 0x55eecf2f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0559b4ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2e9e90 0x55eecf2f2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0559b4ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2f4db0 0x55eecf2eeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0559b4ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecf2ea850 con 0x55eecf2e9570 2024-03-18T20:45:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0559b4ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecf2dfc10 con 0x55eecf2f4db0 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0559b4ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecf2f0180 con 0x55eecf2e9e90 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0546ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 0x55eecf2f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0546ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 0x55eecf2f71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38596/0 (socket says 172.21.15.46:38596) 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0546ffd700 1 -- 172.21.15.46:0/3263531873 learned_addr learned my addr 172.21.15.46:0/3263531873 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0546ffd700 1 -- 172.21.15.46:0/3263531873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2e9e90 msgr2=0x55eecf2f2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f05467fc700 1 --2- 172.21.15.46:0/3263531873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2e9e90 0x55eecf2f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f05477fe700 1 --2- 172.21.15.46:0/3263531873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2f4db0 0x55eecf2eeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0546ffd700 1 --2- 172.21.15.46:0/3263531873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2e9e90 0x55eecf2f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0546ffd700 1 -- 172.21.15.46:0/3263531873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2f4db0 msgr2=0x55eecf2eeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0546ffd700 1 --2- 172.21.15.46:0/3263531873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2f4db0 0x55eecf2eeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0546ffd700 1 -- 172.21.15.46:0/3263531873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eecf202e40 con 0x55eecf2e9570 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f05477fe700 1 --2- 172.21.15.46:0/3263531873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2f4db0 0x55eecf2eeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.419+0000 7f0546ffd700 1 --2- 172.21.15.46:0/3263531873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 0x55eecf2f71a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f0538099ed0 tx=0x7f053809de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7d1da476583da server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.420+0000 7f052e7fc700 1 -- 172.21.15.46:0/3263531873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05380b1030 con 0x55eecf2e9570 2024-03-18T20:45:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.420+0000 7f052e7fc700 1 -- 172.21.15.46:0/3263531873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eecf2dd6c0 con 0x55eecf2e9570 2024-03-18T20:45:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.420+0000 7f052e7fc700 1 -- 172.21.15.46:0/3263531873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538099480 con 0x55eecf2e9570 2024-03-18T20:45:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.420+0000 7f0559b4ab80 1 -- 172.21.15.46:0/3263531873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 msgr2=0x55eecf2f71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.420+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/3263531873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 0x55eecf2f71a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f0538099ed0 tx=0x7f053809de80 comp rx=0 tx=0).stop 2024-03-18T20:45:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.420+0000 7f0559b4ab80 1 -- 172.21.15.46:0/3263531873 shutdown_connections 2024-03-18T20:45:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.420+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/3263531873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9570 0x55eecf2f71a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.420+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/3263531873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2f4db0 0x55eecf2eeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.420+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/3263531873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2e9e90 0x55eecf2f2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.420+0000 7f0559b4ab80 1 -- 172.21.15.46:0/3263531873 >> 172.21.15.46:0/3263531873 conn(0x55eecf2601c0 msgr2=0x55eecf2e4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.420+0000 7f0559b4ab80 1 -- 172.21.15.46:0/3263531873 shutdown_connections 2024-03-18T20:45:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.420+0000 7f0559b4ab80 1 -- 172.21.15.46:0/3263531873 wait complete. 2024-03-18T20:45:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f0559b4ab80 1 -- start start 2024-03-18T20:45:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f0559b4ab80 1 Processor -- start 2024-03-18T20:45:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f0559b4ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9570 0x55eecf2f9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f0559b4ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 0x55eecf2f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f0559b4ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf3577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f0559b4ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecf2fa2c0 con 0x55eecf2e9e90 2024-03-18T20:45:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f0559b4ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecf357cf0 con 0x55eecf2e9570 2024-03-18T20:45:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f0559b4ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecf357e30 con 0x55eecf2f4db0 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f0559b4ab80 10 client.0 _ll_register_callbacks cb 0x7f05380998f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f05467fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 0x55eecf2f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f05467fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 0x55eecf2f99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38606/0 (socket says 172.21.15.46:38606) 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f05467fc700 1 -- 172.21.15.46:0/2043410801 learned_addr learned my addr 172.21.15.46:0/2043410801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.421+0000 7f05477fe700 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf3577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f0546ffd700 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9570 0x55eecf2f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f05467fc700 1 -- 172.21.15.46:0/2043410801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 msgr2=0x55eecf3577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f05467fc700 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf3577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f05467fc700 1 -- 172.21.15.46:0/2043410801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9570 msgr2=0x55eecf2f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f05467fc700 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9570 0x55eecf2f9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f05467fc700 1 -- 172.21.15.46:0/2043410801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eecf06ab00 con 0x55eecf2e9e90 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f05467fc700 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 0x55eecf2f99b0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f053c006010 tx=0x7f053c00bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6674c2075d4615e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f0526ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f0526ffd700 1 -- 172.21.15.46:0/2043410801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c02b040 con 0x55eecf2e9e90 2024-03-18T20:45:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f0526ffd700 1 -- 172.21.15.46:0/2043410801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f053c03f030 con 0x55eecf2e9e90 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2043410801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f053c01e040 con 0x55eecf2e9e90 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f0526ffd700 1 -- 172.21.15.46:0/2043410801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f053c00fea0 con 0x55eecf2e9e90 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f0559b4ab80 10 client.18861 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2043410801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55eecf0c2c70 con 0x55eecf2e9e90 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.422+0000 7f0526ffd700 1 -- 172.21.15.46:0/2043410801 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f053c01e040 con 0x55eecf2e9e90 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.423+0000 7f0526ffd700 1 -- 172.21.15.46:0/2043410801 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f053c024950 con 0x55eecf2e9e90 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.423+0000 7f0559b4ab80 10 client.18861 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.423+0000 7f0559b4ab80 10 client.18861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.423+0000 7f0559b4ab80 20 client.18861 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.423+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55eecf3591f0 0x55eecf35b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.423+0000 7f0559b4ab80 4 client.18861 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.423+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2043410801 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eecf08bc90 con 0x55eecf3591f0 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.423+0000 7f0546ffd700 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55eecf3591f0 0x55eecf35b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.423+0000 7f0526ffd700 10 client.18861 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.423+0000 7f0546ffd700 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55eecf3591f0 0x55eecf35b610 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baeeb3bcaefc7892 server_cookie=dfc373d9b2cb8108 in_seq=0 out_seq=0 2024-03-18T20:45:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.455+0000 7f0526ffd700 1 -- 172.21.15.46:0/2043410801 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eecf08bc90 con 0x55eecf3591f0 2024-03-18T20:45:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.455+0000 7f0526ffd700 10 client.18861 handle_command_reply: tid=0 2024-03-18T20:45:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.455+0000 7f0559b4ab80 1 client.18861 shutdown 2024-03-18T20:45:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.456+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2043410801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 msgr2=0x55eecf2f99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.456+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 0x55eecf2f99b0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f053c006010 tx=0x7f053c00bfa0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.456+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2043410801 shutdown_connections 2024-03-18T20:45:24.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.456+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55eecf3591f0 0x55eecf35b610 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.456+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eecf2f4db0 0x55eecf3577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.456+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55eecf2e9570 0x55eecf2f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.456+0000 7f0526ffd700 0 client.18861 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.456+0000 7f0559b4ab80 1 --2- 172.21.15.46:0/2043410801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eecf2e9e90 0x55eecf2f99b0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.456+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2043410801 >> 172.21.15.46:0/2043410801 conn(0x55eecf2601c0 msgr2=0x55eecf2f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.456+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2043410801 shutdown_connections 2024-03-18T20:45:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.456+0000 7f0559b4ab80 1 -- 172.21.15.46:0/2043410801 wait complete. 2024-03-18T20:45:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.456+0000 7f0559b4ab80 20 client.18861 trim_cache size 0 max 16384 2024-03-18T20:45:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:45:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:45:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:45:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:45:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.457+0000 7f054d9b5700 1 -- 172.21.15.46:0/2537858068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f05300a1290 msgr2=0x7f05300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.457+0000 7f054d9b5700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f05300a1290 0x7f05300a3750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f05481985d0 tx=0x7f053c005bc0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.457+0000 7f054d9b5700 1 -- 172.21.15.46:0/2537858068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0548069530 msgr2=0x7f0548197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.457+0000 7f054d9b5700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0548069530 0x7f0548197080 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f053800e9f0 tx=0x7f053800ed00 comp rx=0 tx=0).stop 2024-03-18T20:45:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.457+0000 7f054d9b5700 1 -- 172.21.15.46:0/2537858068 shutdown_connections 2024-03-18T20:45:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.457+0000 7f054d9b5700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f05300a1290 0x7f05300a3750 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.457+0000 7f054d9b5700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548106e10 0x7f054819b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.457+0000 7f054d9b5700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0548069530 0x7f0548197080 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.457+0000 7f054d9b5700 1 --2- 172.21.15.46:0/2537858068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0548069ef0 0x7f05481975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.457+0000 7f054d9b5700 1 -- 172.21.15.46:0/2537858068 >> 172.21.15.46:0/2537858068 conn(0x7f0548076db0 msgr2=0x7f05480fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.457+0000 7f054d9b5700 1 -- 172.21.15.46:0/2537858068 shutdown_connections 2024-03-18T20:45:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:24.457+0000 7f054d9b5700 1 -- 172.21.15.46:0/2537858068 wait complete. 2024-03-18T20:45:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:24 smithi155 ceph-mon[48344]: pgmap v288: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-03-18T20:45:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:24 smithi046 ceph-mon[43495]: pgmap v288: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-03-18T20:45:24.709 INFO:tasks.workunit.client.0.smithi046.stdout: 1 338670 3.65 MB/sec execute 379 sec latency 2166.601 ms 2024-03-18T20:45:24.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:24 smithi073 ceph-mon[47384]: pgmap v288: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-03-18T20:45:25.709 INFO:tasks.workunit.client.0.smithi046.stdout: 1 339579 3.64 MB/sec execute 380 sec latency 2791.082 ms 2024-03-18T20:45:26.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:25 smithi073 ceph-mon[47384]: pgmap v289: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-18T20:45:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:25 smithi155 ceph-mon[48344]: pgmap v289: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-18T20:45:26.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:25 smithi046 ceph-mon[43495]: pgmap v289: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-18T20:45:26.709 INFO:tasks.workunit.client.0.smithi046.stdout: 1 341028 3.66 MB/sec execute 381 sec latency 14.296 ms 2024-03-18T20:45:27.709 INFO:tasks.workunit.client.0.smithi046.stdout: 1 342210 3.67 MB/sec execute 382 sec latency 142.093 ms 2024-03-18T20:45:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:28 smithi155 ceph-mon[48344]: pgmap v290: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-18T20:45:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:28 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:45:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:28 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:45:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:28 smithi046 ceph-mon[43495]: pgmap v290: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-18T20:45:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:28 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:45:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:28 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:45:28.709 INFO:tasks.workunit.client.0.smithi046.stdout: 1 342210 3.66 MB/sec execute 383 sec latency 1142.197 ms 2024-03-18T20:45:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:28 smithi073 ceph-mon[47384]: pgmap v290: 97 pgs: 97 active+clean; 137 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-18T20:45:28.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:28 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:45:28.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:28 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:45:29.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:29 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:45:29.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:29 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:45:29.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:29 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:45:29.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:29 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:45:29.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:29 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:45:29.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:29 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:45:29.709 INFO:tasks.workunit.client.0.smithi046.stdout: 1 342210 3.65 MB/sec execute 384 sec latency 2142.319 ms 2024-03-18T20:45:29.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:29 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:45:29.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:29 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:45:29.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:29 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:45:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:30 smithi155 ceph-mon[48344]: pgmap v291: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-03-18T20:45:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:30 smithi046 ceph-mon[43495]: pgmap v291: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-03-18T20:45:30.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 344153 3.67 MB/sec execute 385 sec latency 2302.292 ms 2024-03-18T20:45:30.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:30 smithi073 ceph-mon[47384]: pgmap v291: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-03-18T20:45:31.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 345246 3.67 MB/sec execute 386 sec latency 20.367 ms 2024-03-18T20:45:32.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:32 smithi155 ceph-mon[48344]: pgmap v292: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-03-18T20:45:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:32 smithi046 ceph-mon[43495]: pgmap v292: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-03-18T20:45:32.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 345759 3.67 MB/sec execute 387 sec latency 562.752 ms 2024-03-18T20:45:32.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:32 smithi073 ceph-mon[47384]: pgmap v292: 97 pgs: 97 active+clean; 133 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-03-18T20:45:33.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 345759 3.66 MB/sec execute 388 sec latency 1562.860 ms 2024-03-18T20:45:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:34 smithi155 ceph-mon[48344]: pgmap v293: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-03-18T20:45:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:34 smithi046 ceph-mon[43495]: pgmap v293: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-03-18T20:45:34.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 345759 3.65 MB/sec execute 389 sec latency 2562.984 ms 2024-03-18T20:45:34.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:34 smithi073 ceph-mon[47384]: pgmap v293: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-03-18T20:45:35.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 346521 3.65 MB/sec execute 390 sec latency 3183.839 ms 2024-03-18T20:45:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:35 smithi155 ceph-mon[48344]: pgmap v294: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-03-18T20:45:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:35 smithi046 ceph-mon[43495]: pgmap v294: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-03-18T20:45:36.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:35 smithi073 ceph-mon[47384]: pgmap v294: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-03-18T20:45:36.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 348031 3.67 MB/sec execute 391 sec latency 13.973 ms 2024-03-18T20:45:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.586+0000 7f7ea3134700 1 -- 172.21.15.46:0/2345232871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0ffbb0 msgr2=0x7f7e9c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.586+0000 7f7ea3134700 1 --2- 172.21.15.46:0/2345232871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0ffbb0 0x7f7e9c100030 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7e880098f0 tx=0x7f7e88009c00 comp rx=0 tx=0).stop 2024-03-18T20:45:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.586+0000 7f7ea3134700 1 -- 172.21.15.46:0/2345232871 shutdown_connections 2024-03-18T20:45:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.586+0000 7f7ea3134700 1 --2- 172.21.15.46:0/2345232871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c105c60 0x7f7e9c106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.586+0000 7f7ea3134700 1 --2- 172.21.15.46:0/2345232871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e9c100570 0x7f7e9c10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.586+0000 7f7ea3134700 1 --2- 172.21.15.46:0/2345232871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0ffbb0 0x7f7e9c100030 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.586+0000 7f7ea3134700 1 -- 172.21.15.46:0/2345232871 >> 172.21.15.46:0/2345232871 conn(0x7f7e9c076d50 msgr2=0x7f7e9c077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.586+0000 7f7ea3134700 1 -- 172.21.15.46:0/2345232871 shutdown_connections 2024-03-18T20:45:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.586+0000 7f7ea3134700 1 -- 172.21.15.46:0/2345232871 wait complete. 2024-03-18T20:45:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.587+0000 7f7ea3134700 1 Processor -- start 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.587+0000 7f7ea3134700 1 -- start start 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.587+0000 7f7ea3134700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0ffbb0 0x7f7e9c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.587+0000 7f7ea3134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c100570 0x7f7e9c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.587+0000 7f7ea3134700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e9c105c60 0x7f7e9c19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.587+0000 7f7ea3134700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c19c0f0 con 0x7f7e9c100570 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.587+0000 7f7ea3134700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c19c260 con 0x7f7e9c0ffbb0 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.587+0000 7f7ea3134700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c19c420 con 0x7f7e9c105c60 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.587+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c100570 0x7f7e9c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.587+0000 7f7ea0ed0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0ffbb0 0x7f7e9c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.587+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c100570 0x7f7e9c197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43960/0 (socket says 172.21.15.46:43960) 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7e9bfff700 1 -- 172.21.15.46:0/448090937 learned_addr learned my addr 172.21.15.46:0/448090937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7ea16d1700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e9c105c60 0x7f7e9c19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7e9bfff700 1 -- 172.21.15.46:0/448090937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e9c105c60 msgr2=0x7f7e9c19bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e9c105c60 0x7f7e9c19bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7e9bfff700 1 -- 172.21.15.46:0/448090937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0ffbb0 msgr2=0x7f7e9c1971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0ffbb0 0x7f7e9c1971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7e9bfff700 1 -- 172.21.15.46:0/448090937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e900095f0 con 0x7f7e9c100570 2024-03-18T20:45:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c100570 0x7f7e9c197710 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f7e88005950 tx=0x7f7e8800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7e99ffb700 1 -- 172.21.15.46:0/448090937 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8801c070 con 0x7f7e9c100570 2024-03-18T20:45:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7ea3134700 1 -- 172.21.15.46:0/448090937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e88009520 con 0x7f7e9c100570 2024-03-18T20:45:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7e99ffb700 1 -- 172.21.15.46:0/448090937 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e8800bb20 con 0x7f7e9c100570 2024-03-18T20:45:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7e99ffb700 1 -- 172.21.15.46:0/448090937 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e880038e0 con 0x7f7e9c100570 2024-03-18T20:45:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.588+0000 7f7ea3134700 1 -- 172.21.15.46:0/448090937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c1a3d60 con 0x7f7e9c100570 2024-03-18T20:45:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.591+0000 7f7e99ffb700 1 -- 172.21.15.46:0/448090937 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7e88020030 con 0x7f7e9c100570 2024-03-18T20:45:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.591+0000 7f7e99ffb700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e8c0a1290 0x7f7e8c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.592+0000 7f7e99ffb700 1 -- 172.21.15.46:0/448090937 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7e880cdc00 con 0x7f7e9c100570 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.592+0000 7f7ea0ed0700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e8c0a1290 0x7f7e8c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.592+0000 7f7ea0ed0700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e8c0a1290 0x7f7e8c0a3750 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7e90005fd0 tx=0x7f7e90009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.592+0000 7f7eaf2c9b80 1 Processor -- start 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.592+0000 7f7eaf2c9b80 1 -- start start 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7eaf2c9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55962007c570 0x55962007c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7eaf2c9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007ce90 0x559620087870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7eaf2c9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559620087db0 0x55962008a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7eaf2c9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55961ffd6930 con 0x55962007ce90 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7eaf2c9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55961ffd42f0 con 0x55962007c570 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7eaf2c9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55961ffcfbe0 con 0x559620087db0 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007ce90 0x559620087870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7ea0ed0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55962007c570 0x55962007c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007ce90 0x559620087870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43962/0 (socket says 172.21.15.46:43962) 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7e9bfff700 1 -- 172.21.15.46:0/2458982995 learned_addr learned my addr 172.21.15.46:0/2458982995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7ea16d1700 1 --2- 172.21.15.46:0/2458982995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559620087db0 0x55962008a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7e9bfff700 1 -- 172.21.15.46:0/2458982995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559620087db0 msgr2=0x55962008a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/2458982995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559620087db0 0x55962008a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7e9bfff700 1 -- 172.21.15.46:0/2458982995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55962007c570 msgr2=0x55962007c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/2458982995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55962007c570 0x55962007c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7e9bfff700 1 -- 172.21.15.46:0/2458982995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55961fe51350 con 0x55962007ce90 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7ea0ed0700 1 --2- 172.21.15.46:0/2458982995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55962007c570 0x55962007c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.593+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/2458982995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007ce90 0x559620087870 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f7e88097790 tx=0x7f7e88005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a078898a113c11af server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.594+0000 7f7e86ffd700 1 -- 172.21.15.46:0/2458982995 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8809eb50 con 0x55962007ce90 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.594+0000 7f7e86ffd700 1 -- 172.21.15.46:0/2458982995 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e880a3110 con 0x55962007ce90 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.594+0000 7f7e86ffd700 1 -- 172.21.15.46:0/2458982995 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e880053f0 con 0x55962007ce90 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.594+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/2458982995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007ce90 msgr2=0x559620087870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:37.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.594+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/2458982995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007ce90 0x559620087870 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f7e88097790 tx=0x7f7e88005570 comp rx=0 tx=0).stop 2024-03-18T20:45:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.594+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/2458982995 shutdown_connections 2024-03-18T20:45:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.594+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/2458982995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559620087db0 0x55962008a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.594+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/2458982995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55962007c570 0x55962007c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.594+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/2458982995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007ce90 0x559620087870 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.594+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/2458982995 >> 172.21.15.46:0/2458982995 conn(0x55961fff31c0 msgr2=0x559620078a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.594+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/2458982995 shutdown_connections 2024-03-18T20:45:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.594+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/2458982995 wait complete. 2024-03-18T20:45:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.595+0000 7f7eaf2c9b80 1 -- start start 2024-03-18T20:45:37.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.595+0000 7f7eaf2c9b80 1 Processor -- start 2024-03-18T20:45:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.595+0000 7f7eaf2c9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 0x5596200877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.595+0000 7f7eaf2c9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55962007ce90 0x559620084b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.595+0000 7f7eaf2c9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559620087db0 0x559620085130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.595+0000 7f7eaf2c9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55961ff6bd20 con 0x55962007c570 2024-03-18T20:45:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.595+0000 7f7eaf2c9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55961ff6b9c0 con 0x55962007ce90 2024-03-18T20:45:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.595+0000 7f7eaf2c9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55961fe53640 con 0x559620087db0 2024-03-18T20:45:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.595+0000 7f7eaf2c9b80 10 client.0 _ll_register_callbacks cb 0x55961fdd7340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7ea0ed0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 0x5596200877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55962007ce90 0x559620084b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7ea0ed0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 0x5596200877c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43970/0 (socket says 172.21.15.46:43970) 2024-03-18T20:45:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7ea0ed0700 1 -- 172.21.15.46:0/1402370301 learned_addr learned my addr 172.21.15.46:0/1402370301 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7ea16d1700 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559620087db0 0x559620085130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7e9bfff700 1 -- 172.21.15.46:0/1402370301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559620087db0 msgr2=0x559620085130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559620087db0 0x559620085130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7e9bfff700 1 -- 172.21.15.46:0/1402370301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 msgr2=0x5596200877c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 0x5596200877c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7e9bfff700 1 -- 172.21.15.46:0/1402370301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e90017070 con 0x55962007ce90 2024-03-18T20:45:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7ea0ed0700 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 0x5596200877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55962007ce90 0x559620084b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55962007d3a0 tx=0x7f7e88019fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27c558f47b918045 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7e7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7e7bfff700 1 -- 172.21.15.46:0/1402370301 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e880cb760 con 0x55962007ce90 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7e7bfff700 1 -- 172.21.15.46:0/1402370301 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7e8809eb50 con 0x55962007ce90 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/1402370301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596200706c0 con 0x55962007ce90 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7e7bfff700 1 -- 172.21.15.46:0/1402370301 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e880051d0 con 0x55962007ce90 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7eaf2c9b80 10 client.28508 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.596+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/1402370301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f7e9c1a3d60 con 0x55962007ce90 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.597+0000 7f7e7bfff700 1 -- 172.21.15.46:0/1402370301 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596200706c0 con 0x55962007ce90 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.597+0000 7f7e7bfff700 1 -- 172.21.15.46:0/1402370301 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7e8809a210 con 0x55962007ce90 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.597+0000 7f7eaf2c9b80 10 client.28508 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.597+0000 7f7eaf2c9b80 10 client.28508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.597+0000 7f7eaf2c9b80 20 client.28508 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.597+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596200971e0 0x559620099600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.597+0000 7f7eaf2c9b80 4 client.28508 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.597+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/1402370301 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55961ffc86a0 con 0x5596200971e0 2024-03-18T20:45:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.597+0000 7f7ea0ed0700 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596200971e0 0x559620099600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7e7bfff700 10 client.28508 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7ea0ed0700 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596200971e0 0x559620099600 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8005a281e3ff59d server_cookie=5c03ba868188235c in_seq=0 out_seq=0 2024-03-18T20:45:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7e7bfff700 1 -- 172.21.15.46:0/1402370301 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55961ffc86a0 con 0x5596200971e0 2024-03-18T20:45:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7e7bfff700 10 client.28508 handle_command_reply: tid=0 2024-03-18T20:45:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7eaf2c9b80 1 client.28508 shutdown 2024-03-18T20:45:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/1402370301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55962007ce90 msgr2=0x559620084b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55962007ce90 0x559620084b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55962007d3a0 tx=0x7f7e88019fd0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/1402370301 shutdown_connections 2024-03-18T20:45:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596200971e0 0x559620099600 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 0x5596200877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559620087db0 0x559620085130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/1402370301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55962007ce90 0x559620084b00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7e7bfff700 0 client.28508 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.598+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/1402370301 >> 172.21.15.46:0/1402370301 conn(0x55961fff31c0 msgr2=0x559620078650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.599+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/1402370301 shutdown_connections 2024-03-18T20:45:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.599+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/1402370301 wait complete. 2024-03-18T20:45:37.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.599+0000 7f7eaf2c9b80 20 client.28508 trim_cache size 0 max 16384 2024-03-18T20:45:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.614+0000 7f7eaf2c9b80 1 Processor -- start 2024-03-18T20:45:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.614+0000 7f7eaf2c9b80 1 -- start start 2024-03-18T20:45:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7eaf2c9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 0x55962008e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7eaf2c9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007ce90 0x5596200935d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7eaf2c9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559620087db0 0x5596200872f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7eaf2c9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596200777b0 con 0x55962007c570 2024-03-18T20:45:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7eaf2c9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559620079780 con 0x559620087db0 2024-03-18T20:45:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7eaf2c9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559620072c10 con 0x55962007ce90 2024-03-18T20:45:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7ea0ed0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 0x55962008e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7ea0ed0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 0x55962008e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43980/0 (socket says 172.21.15.46:43980) 2024-03-18T20:45:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7ea0ed0700 1 -- 172.21.15.46:0/3414968871 learned_addr learned my addr 172.21.15.46:0/3414968871 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7ea16d1700 1 --2- 172.21.15.46:0/3414968871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559620087db0 0x5596200872f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/3414968871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007ce90 0x5596200935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7ea0ed0700 1 -- 172.21.15.46:0/3414968871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007ce90 msgr2=0x5596200935d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7ea0ed0700 1 --2- 172.21.15.46:0/3414968871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007ce90 0x5596200935d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7ea0ed0700 1 -- 172.21.15.46:0/3414968871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559620087db0 msgr2=0x5596200872f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7ea0ed0700 1 --2- 172.21.15.46:0/3414968871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559620087db0 0x5596200872f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7ea0ed0700 1 -- 172.21.15.46:0/3414968871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55961fdfdb00 con 0x55962007c570 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/3414968871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007ce90 0x5596200935d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7ea16d1700 1 --2- 172.21.15.46:0/3414968871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559620087db0 0x5596200872f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.615+0000 7f7ea0ed0700 1 --2- 172.21.15.46:0/3414968871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 0x55962008e770 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7e90011ab0 tx=0x7f7e90014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e68cbb80a8bc4af server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.616+0000 7f7e877fe700 1 -- 172.21.15.46:0/3414968871 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90022070 con 0x55962007c570 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.616+0000 7f7e877fe700 1 -- 172.21.15.46:0/3414968871 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e90023040 con 0x55962007c570 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.616+0000 7f7e877fe700 1 -- 172.21.15.46:0/3414968871 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9001d800 con 0x55962007c570 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.616+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3414968871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 msgr2=0x55962008e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.616+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/3414968871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 0x55962008e770 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f7e90011ab0 tx=0x7f7e90014000 comp rx=0 tx=0).stop 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.616+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3414968871 shutdown_connections 2024-03-18T20:45:37.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.616+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/3414968871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007c570 0x55962008e770 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.616+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/3414968871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559620087db0 0x5596200872f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.616+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/3414968871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007ce90 0x5596200935d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.616+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3414968871 >> 172.21.15.46:0/3414968871 conn(0x55961fff31c0 msgr2=0x559620092ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.617+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3414968871 shutdown_connections 2024-03-18T20:45:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.617+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3414968871 wait complete. 2024-03-18T20:45:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.617+0000 7f7eaf2c9b80 1 -- start start 2024-03-18T20:45:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.617+0000 7f7eaf2c9b80 1 Processor -- start 2024-03-18T20:45:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.617+0000 7f7eaf2c9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007c570 0x55962008c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.617+0000 7f7eaf2c9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007ce90 0x55962008cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.617+0000 7f7eaf2c9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559620087db0 0x5596200e3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.617+0000 7f7eaf2c9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596200e44e0 con 0x55962007ce90 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7eaf2c9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596200e4620 con 0x559620087db0 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7eaf2c9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596200e47e0 con 0x55962007c570 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7eaf2c9b80 10 client.0 _ll_register_callbacks cb 0x7f7e90009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7ea0ed0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007c570 0x55962008c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007ce90 0x55962008cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7ea0ed0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007c570 0x55962008c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:39962/0 (socket says 172.21.15.46:39962) 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7ea16d1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559620087db0 0x5596200e3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7ea0ed0700 1 -- 172.21.15.46:0/3702012763 learned_addr learned my addr 172.21.15.46:0/3702012763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7ea0ed0700 1 -- 172.21.15.46:0/3702012763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559620087db0 msgr2=0x5596200e3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7ea0ed0700 1 --2- 172.21.15.46:0/3702012763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559620087db0 0x5596200e3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7ea0ed0700 1 -- 172.21.15.46:0/3702012763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007ce90 msgr2=0x55962008cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7ea0ed0700 1 --2- 172.21.15.46:0/3702012763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007ce90 0x55962008cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7ea0ed0700 1 -- 172.21.15.46:0/3702012763 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c1a3d60 con 0x55962007c570 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7ea16d1700 1 --2- 172.21.15.46:0/3702012763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559620087db0 0x5596200e3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7ea0ed0700 1 --2- 172.21.15.46:0/3702012763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007c570 0x55962008c610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7e900095c0 tx=0x7f7e90005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaec2a80c207cdc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7e7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.618+0000 7f7e7bfff700 1 -- 172.21.15.46:0/3702012763 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90022040 con 0x55962007c570 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3702012763 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e880a3110 con 0x55962007c570 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7e7bfff700 1 -- 172.21.15.46:0/3702012763 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7e9002b0b0 con 0x55962007c570 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7e7bfff700 1 -- 172.21.15.46:0/3702012763 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e9002b270 con 0x55962007c570 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7eaf2c9b80 10 client.28531 fetch_fsmap learned FSMap version 47 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3702012763 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f7e94009b30 con 0x55962007c570 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7e7bfff700 1 -- 172.21.15.46:0/3702012763 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e880a3110 con 0x55962007c570 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7e7bfff700 1 -- 172.21.15.46:0/3702012763 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7e9001d410 con 0x55962007c570 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7eaf2c9b80 10 client.28531 fetch_fsmap finished waiting for FSMap version 47 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7eaf2c9b80 10 client.28531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7eaf2c9b80 20 client.28531 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/3702012763 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596200ec0a0 0x5596200ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7eaf2c9b80 4 client.28531 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:45:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3702012763 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55961fe1ec90 con 0x5596200ec0a0 2024-03-18T20:45:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.619+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/3702012763 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596200ec0a0 0x5596200ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.620+0000 7f7e9bfff700 1 --2- 172.21.15.46:0/3702012763 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596200ec0a0 0x5596200ee4c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=690801d875440610 server_cookie=d09bc5b88337f74d in_seq=0 out_seq=0 2024-03-18T20:45:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.620+0000 7f7e7bfff700 10 client.28531 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7e7bfff700 1 -- 172.21.15.46:0/3702012763 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55961fe1ec90 con 0x5596200ec0a0 2024-03-18T20:45:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7e7bfff700 10 client.28531 handle_command_reply: tid=0 2024-03-18T20:45:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7eaf2c9b80 1 client.28531 shutdown 2024-03-18T20:45:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3702012763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007c570 msgr2=0x55962008c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/3702012763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007c570 0x55962008c610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7e900095c0 tx=0x7f7e90005dd0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3702012763 shutdown_connections 2024-03-18T20:45:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/3702012763 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596200ec0a0 0x5596200ee4c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/3702012763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559620087db0 0x5596200e3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7e7bfff700 0 client.28531 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/3702012763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55962007c570 0x55962008c610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7eaf2c9b80 1 --2- 172.21.15.46:0/3702012763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55962007ce90 0x55962008cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3702012763 >> 172.21.15.46:0/3702012763 conn(0x55961fff31c0 msgr2=0x5596200833d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3702012763 shutdown_connections 2024-03-18T20:45:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7eaf2c9b80 1 -- 172.21.15.46:0/3702012763 wait complete. 2024-03-18T20:45:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.658+0000 7f7eaf2c9b80 20 client.28531 trim_cache size 0 max 16384 2024-03-18T20:45:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:45:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:45:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:45:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:45:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.659+0000 7f7ea3134700 1 -- 172.21.15.46:0/448090937 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e8c0a1290 msgr2=0x7f7e8c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.659+0000 7f7ea3134700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e8c0a1290 0x7f7e8c0a3750 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7e90005fd0 tx=0x7f7e90009310 comp rx=0 tx=0).stop 2024-03-18T20:45:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.659+0000 7f7ea3134700 1 -- 172.21.15.46:0/448090937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c100570 msgr2=0x7f7e9c197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.659+0000 7f7ea3134700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c100570 0x7f7e9c197710 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f7e88005950 tx=0x7f7e8800fa00 comp rx=0 tx=0).stop 2024-03-18T20:45:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.659+0000 7f7ea3134700 1 -- 172.21.15.46:0/448090937 shutdown_connections 2024-03-18T20:45:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.659+0000 7f7ea3134700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7e8c0a1290 0x7f7e8c0a3750 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.659+0000 7f7ea3134700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e9c105c60 0x7f7e9c19bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.659+0000 7f7ea3134700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0ffbb0 0x7f7e9c1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.659+0000 7f7ea3134700 1 --2- 172.21.15.46:0/448090937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c100570 0x7f7e9c197710 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.659+0000 7f7ea3134700 1 -- 172.21.15.46:0/448090937 >> 172.21.15.46:0/448090937 conn(0x7f7e9c076d50 msgr2=0x7f7e9c068460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.659+0000 7f7ea3134700 1 -- 172.21.15.46:0/448090937 shutdown_connections 2024-03-18T20:45:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:37.659+0000 7f7ea3134700 1 -- 172.21.15.46:0/448090937 wait complete. 2024-03-18T20:45:37.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 349223 3.67 MB/sec execute 392 sec latency 66.596 ms 2024-03-18T20:45:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:38 smithi155 ceph-mon[48344]: pgmap v295: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-03-18T20:45:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:38 smithi046 ceph-mon[43495]: pgmap v295: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-03-18T20:45:38.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 349302 3.66 MB/sec execute 393 sec latency 927.533 ms 2024-03-18T20:45:38.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:38 smithi073 ceph-mon[47384]: pgmap v295: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-03-18T20:45:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:39 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:45:39.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:39 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:45:39.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 349302 3.65 MB/sec execute 394 sec latency 1927.641 ms 2024-03-18T20:45:39.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:39 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:45:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:40 smithi155 ceph-mon[48344]: pgmap v296: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-03-18T20:45:40.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:40 smithi046 ceph-mon[43495]: pgmap v296: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-03-18T20:45:40.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 350108 3.65 MB/sec execute 395 sec latency 2543.331 ms 2024-03-18T20:45:40.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:40 smithi073 ceph-mon[47384]: pgmap v296: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-03-18T20:45:41.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 351626 3.67 MB/sec execute 396 sec latency 15.874 ms 2024-03-18T20:45:42.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:42 smithi155 ceph-mon[48344]: pgmap v297: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-03-18T20:45:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:42 smithi046 ceph-mon[43495]: pgmap v297: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-03-18T20:45:42.710 INFO:tasks.workunit.client.0.smithi046.stdout: 1 352850 3.67 MB/sec execute 397 sec latency 89.168 ms 2024-03-18T20:45:42.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:42 smithi073 ceph-mon[47384]: pgmap v297: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-03-18T20:45:43.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 352850 3.66 MB/sec execute 398 sec latency 1089.285 ms 2024-03-18T20:45:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:44 smithi155 ceph-mon[48344]: pgmap v298: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-03-18T20:45:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:44 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:45:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:44 smithi046 ceph-mon[43495]: pgmap v298: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-03-18T20:45:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:44 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:45:44.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 352850 3.65 MB/sec execute 399 sec latency 2089.383 ms 2024-03-18T20:45:44.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:44 smithi073 ceph-mon[47384]: pgmap v298: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-03-18T20:45:44.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:44 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:45:45.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 354809 3.68 MB/sec execute 400 sec latency 2237.440 ms 2024-03-18T20:45:46.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:45 smithi155 ceph-mon[48344]: pgmap v299: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1016 KiB/s wr, 117 op/s 2024-03-18T20:45:46.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:45 smithi046 ceph-mon[43495]: pgmap v299: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1016 KiB/s wr, 117 op/s 2024-03-18T20:45:46.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:45 smithi073 ceph-mon[47384]: pgmap v299: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1016 KiB/s wr, 117 op/s 2024-03-18T20:45:46.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 355933 3.68 MB/sec execute 401 sec latency 39.473 ms 2024-03-18T20:45:47.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 356392 3.68 MB/sec execute 402 sec latency 739.781 ms 2024-03-18T20:45:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:48 smithi155 ceph-mon[48344]: pgmap v300: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1016 KiB/s wr, 117 op/s 2024-03-18T20:45:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:48 smithi046 ceph-mon[43495]: pgmap v300: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1016 KiB/s wr, 117 op/s 2024-03-18T20:45:48.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 356392 3.67 MB/sec execute 403 sec latency 1739.904 ms 2024-03-18T20:45:48.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:48 smithi073 ceph-mon[47384]: pgmap v300: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1016 KiB/s wr, 117 op/s 2024-03-18T20:45:49.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 356392 3.66 MB/sec execute 404 sec latency 2739.967 ms 2024-03-18T20:45:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:50 smithi155 ceph-mon[48344]: pgmap v301: 97 pgs: 97 active+clean; 138 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-03-18T20:45:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:50 smithi046 ceph-mon[43495]: pgmap v301: 97 pgs: 97 active+clean; 138 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-03-18T20:45:50.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 357282 3.65 MB/sec execute 405 sec latency 3357.158 ms 2024-03-18T20:45:50.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:50 smithi073 ceph-mon[47384]: pgmap v301: 97 pgs: 97 active+clean; 138 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-03-18T20:45:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.793+0000 7fee94f7d700 1 -- 172.21.15.46:0/383965268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee901018c0 msgr2=0x7fee90105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.793+0000 7fee94f7d700 1 --2- 172.21.15.46:0/383965268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee901018c0 0x7fee90105900 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fee84009990 tx=0x7fee84009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.793+0000 7fee94f7d700 1 -- 172.21.15.46:0/383965268 shutdown_connections 2024-03-18T20:45:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.793+0000 7fee94f7d700 1 --2- 172.21.15.46:0/383965268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee90106030 0x7fee9010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.793+0000 7fee94f7d700 1 --2- 172.21.15.46:0/383965268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90100f10 0x7fee901012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.793+0000 7fee94f7d700 1 --2- 172.21.15.46:0/383965268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee901018c0 0x7fee90105900 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.793+0000 7fee94f7d700 1 -- 172.21.15.46:0/383965268 >> 172.21.15.46:0/383965268 conn(0x7fee900fc770 msgr2=0x7fee900feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.793+0000 7fee94f7d700 1 -- 172.21.15.46:0/383965268 shutdown_connections 2024-03-18T20:45:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.793+0000 7fee94f7d700 1 -- 172.21.15.46:0/383965268 wait complete. 2024-03-18T20:45:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.794+0000 7fee94f7d700 1 Processor -- start 2024-03-18T20:45:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.794+0000 7fee94f7d700 1 -- start start 2024-03-18T20:45:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee94f7d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90100f10 0x7fee900730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee94f7d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee90106030 0x7fee900735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee94f7d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073b30 0x7fee901a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee94f7d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee901a59d0 con 0x7fee90100f10 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee94f7d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee901a5b10 con 0x7fee90106030 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee94f7d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee901a5cd0 con 0x7fee90073b30 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee8dd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee90106030 0x7fee900735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee8dd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee90106030 0x7fee900735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56296/0 (socket says 172.21.15.46:56296) 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90100f10 0x7fee900730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee8dd9b700 1 -- 172.21.15.46:0/1036781621 learned_addr learned my addr 172.21.15.46:0/1036781621 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee8ed9d700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073b30 0x7fee901a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee8dd9b700 1 -- 172.21.15.46:0/1036781621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073b30 msgr2=0x7fee901a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee8dd9b700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073b30 0x7fee901a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee8dd9b700 1 -- 172.21.15.46:0/1036781621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90100f10 msgr2=0x7fee900730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee8dd9b700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90100f10 0x7fee900730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee8dd9b700 1 -- 172.21.15.46:0/1036781621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee840095f0 con 0x7fee90106030 2024-03-18T20:45:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.795+0000 7fee8dd9b700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee90106030 0x7fee900735f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee84005770 tx=0x7fee840057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.796+0000 7fee7f7fe700 1 -- 172.21.15.46:0/1036781621 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8401d070 con 0x7fee90106030 2024-03-18T20:45:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.796+0000 7fee7f7fe700 1 -- 172.21.15.46:0/1036781621 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee84021410 con 0x7fee90106030 2024-03-18T20:45:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.796+0000 7fee94f7d700 1 -- 172.21.15.46:0/1036781621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee901a5f50 con 0x7fee90106030 2024-03-18T20:45:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.796+0000 7fee7f7fe700 1 -- 172.21.15.46:0/1036781621 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84017650 con 0x7fee90106030 2024-03-18T20:45:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.796+0000 7fee94f7d700 1 -- 172.21.15.46:0/1036781621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee901a6370 con 0x7fee90106030 2024-03-18T20:45:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.798+0000 7fee7f7fe700 1 -- 172.21.15.46:0/1036781621 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fee8400fb40 con 0x7fee90106030 2024-03-18T20:45:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.799+0000 7fee7f7fe700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fee780a1290 0x7fee780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.799+0000 7fee8e59c700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fee780a1290 0x7fee780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.799+0000 7fee7f7fe700 1 -- 172.21.15.46:0/1036781621 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fee840cdf40 con 0x7fee90106030 2024-03-18T20:45:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.800+0000 7fee8e59c700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fee780a1290 0x7fee780a3750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fee80007be0 tx=0x7fee800073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.800+0000 7feea1112b80 1 Processor -- start 2024-03-18T20:45:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7feea1112b80 1 -- start start 2024-03-18T20:45:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7feea1112b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 0x56256a582950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7feea1112b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582e90 0x56256a58d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7feea1112b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a58ddb0 0x56256a5901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7feea1112b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256a4dc930 con 0x56256a58ddb0 2024-03-18T20:45:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7feea1112b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256a4da2f0 con 0x56256a582570 2024-03-18T20:45:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7feea1112b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256a4d5be0 con 0x56256a582e90 2024-03-18T20:45:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7fee8e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 0x56256a582950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7fee8e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 0x56256a582950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56312/0 (socket says 172.21.15.46:56312) 2024-03-18T20:45:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7fee8e59c700 1 -- 172.21.15.46:0/3141940134 learned_addr learned my addr 172.21.15.46:0/3141940134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7fee8dd9b700 1 --2- 172.21.15.46:0/3141940134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582e90 0x56256a58d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7fee8ed9d700 1 --2- 172.21.15.46:0/3141940134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a58ddb0 0x56256a5901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7fee8e59c700 1 -- 172.21.15.46:0/3141940134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582e90 msgr2=0x56256a58d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7fee8e59c700 1 --2- 172.21.15.46:0/3141940134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582e90 0x56256a58d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7fee8e59c700 1 -- 172.21.15.46:0/3141940134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a58ddb0 msgr2=0x56256a5901a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7fee8e59c700 1 --2- 172.21.15.46:0/3141940134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a58ddb0 0x56256a5901a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7fee8e59c700 1 -- 172.21.15.46:0/3141940134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56256a357350 con 0x56256a582570 2024-03-18T20:45:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.801+0000 7fee8ed9d700 1 --2- 172.21.15.46:0/3141940134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a58ddb0 0x56256a5901a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.802+0000 7fee8e59c700 1 --2- 172.21.15.46:0/3141940134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 0x56256a582950 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fee80010e80 tx=0x7fee80013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d36082597db9cc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.802+0000 7fee7cff9700 1 -- 172.21.15.46:0/3141940134 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80010eb0 con 0x56256a582570 2024-03-18T20:45:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.802+0000 7fee7cff9700 1 -- 172.21.15.46:0/3141940134 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee80003b40 con 0x56256a582570 2024-03-18T20:45:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.802+0000 7fee7cff9700 1 -- 172.21.15.46:0/3141940134 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80011c00 con 0x56256a582570 2024-03-18T20:45:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.803+0000 7feea1112b80 1 -- 172.21.15.46:0/3141940134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 msgr2=0x56256a582950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.803+0000 7feea1112b80 1 --2- 172.21.15.46:0/3141940134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 0x56256a582950 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fee80010e80 tx=0x7fee80013000 comp rx=0 tx=0).stop 2024-03-18T20:45:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.803+0000 7feea1112b80 1 -- 172.21.15.46:0/3141940134 shutdown_connections 2024-03-18T20:45:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.803+0000 7feea1112b80 1 --2- 172.21.15.46:0/3141940134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a58ddb0 0x56256a5901a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.803+0000 7feea1112b80 1 --2- 172.21.15.46:0/3141940134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 0x56256a582950 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.803+0000 7feea1112b80 1 --2- 172.21.15.46:0/3141940134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582e90 0x56256a58d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.803+0000 7feea1112b80 1 -- 172.21.15.46:0/3141940134 >> 172.21.15.46:0/3141940134 conn(0x56256a4f91c0 msgr2=0x56256a57ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.803+0000 7feea1112b80 1 -- 172.21.15.46:0/3141940134 shutdown_connections 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.803+0000 7feea1112b80 1 -- 172.21.15.46:0/3141940134 wait complete. 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.803+0000 7feea1112b80 1 -- start start 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.803+0000 7feea1112b80 1 Processor -- start 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7feea1112b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582570 0x56256a58d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7feea1112b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582e90 0x56256a58aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7feea1112b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a58ddb0 0x56256a58b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7feea1112b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256a471d20 con 0x56256a582e90 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7feea1112b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256a4719c0 con 0x56256a58ddb0 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7feea1112b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256a359640 con 0x56256a582570 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7feea1112b80 10 client.0 _ll_register_callbacks cb 0x56256a2dd340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7fee8e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582570 0x56256a58d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7fee8dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582e90 0x56256a58aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7fee8e59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582570 0x56256a58d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:38810/0 (socket says 172.21.15.46:38810) 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7fee8e59c700 1 -- 172.21.15.46:0/432314246 learned_addr learned my addr 172.21.15.46:0/432314246 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7fee8dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582e90 0x56256a58aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41628/0 (socket says 172.21.15.46:41628) 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7fee8ed9d700 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a58ddb0 0x56256a58b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7fee8e59c700 1 -- 172.21.15.46:0/432314246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a58ddb0 msgr2=0x56256a58b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7fee8e59c700 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a58ddb0 0x56256a58b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7fee8e59c700 1 -- 172.21.15.46:0/432314246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582e90 msgr2=0x56256a58aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7fee8e59c700 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582e90 0x56256a58aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7fee8e59c700 1 -- 172.21.15.46:0/432314246 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee80010b60 con 0x56256a582570 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.804+0000 7fee8ed9d700 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a58ddb0 0x56256a58b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7fee8e59c700 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582570 0x56256a58d780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee80011780 tx=0x7fee80003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3995d68a524e33e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7fee75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7fee75ffb700 1 -- 172.21.15.46:0/432314246 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80030070 con 0x56256a582570 2024-03-18T20:45:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7feea1112b80 1 -- 172.21.15.46:0/432314246 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56256a5766c0 con 0x56256a582570 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7fee75ffb700 1 -- 172.21.15.46:0/432314246 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fee80017070 con 0x56256a582570 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7fee75ffb700 1 -- 172.21.15.46:0/432314246 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee80010eb0 con 0x56256a582570 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7feea1112b80 10 client.28543 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7feea1112b80 1 -- 172.21.15.46:0/432314246 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56256a303b00 con 0x56256a582570 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7fee75ffb700 1 -- 172.21.15.46:0/432314246 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56256a5766c0 con 0x56256a582570 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7fee75ffb700 1 -- 172.21.15.46:0/432314246 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fee80033020 con 0x56256a582570 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7feea1112b80 10 client.28543 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7feea1112b80 10 client.28543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7feea1112b80 20 client.28543 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7feea1112b80 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56256a59d920 0x56256a59fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7feea1112b80 4 client.28543 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.805+0000 7feea1112b80 1 -- 172.21.15.46:0/432314246 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56256a4ce6a0 con 0x56256a59d920 2024-03-18T20:45:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.806+0000 7fee8dd9b700 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56256a59d920 0x56256a59fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.806+0000 7fee75ffb700 10 client.28543 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.806+0000 7fee8dd9b700 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56256a59d920 0x56256a59fd40 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d27c5b9dfad7be11 server_cookie=c4fc9c5235068040 in_seq=0 out_seq=0 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7fee75ffb700 1 -- 172.21.15.46:0/432314246 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56256a4ce6a0 con 0x56256a59d920 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7fee75ffb700 10 client.28543 handle_command_reply: tid=0 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7feea1112b80 1 client.28543 shutdown 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7feea1112b80 1 -- 172.21.15.46:0/432314246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582570 msgr2=0x56256a58d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7feea1112b80 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582570 0x56256a58d780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee80011780 tx=0x7fee80003f60 comp rx=0 tx=0).stop 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7feea1112b80 1 -- 172.21.15.46:0/432314246 shutdown_connections 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7feea1112b80 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56256a59d920 0x56256a59fd40 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7feea1112b80 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a58ddb0 0x56256a58b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7feea1112b80 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582570 0x56256a58d780 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7feea1112b80 1 --2- 172.21.15.46:0/432314246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582e90 0x56256a58aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7feea1112b80 1 -- 172.21.15.46:0/432314246 >> 172.21.15.46:0/432314246 conn(0x56256a4f91c0 msgr2=0x56256a57e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7fee75ffb700 0 client.28543 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7feea1112b80 1 -- 172.21.15.46:0/432314246 shutdown_connections 2024-03-18T20:45:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7feea1112b80 1 -- 172.21.15.46:0/432314246 wait complete. 2024-03-18T20:45:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.807+0000 7feea1112b80 20 client.28543 trim_cache size 0 max 16384 2024-03-18T20:45:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.826+0000 7feea1112b80 1 Processor -- start 2024-03-18T20:45:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.826+0000 7feea1112b80 1 -- start start 2024-03-18T20:45:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.826+0000 7feea1112b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 0x56256a58d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.826+0000 7feea1112b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582e90 0x56256a58a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.826+0000 7feea1112b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a58ddb0 0x56256a58a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.826+0000 7feea1112b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256a583850 con 0x56256a58ddb0 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.826+0000 7feea1112b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256a57b1d0 con 0x56256a582570 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.826+0000 7feea1112b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256a57fd60 con 0x56256a582e90 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee8e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 0x56256a58d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee8e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 0x56256a58d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56332/0 (socket says 172.21.15.46:56332) 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee8ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a58ddb0 0x56256a58a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee8e59c700 1 -- 172.21.15.46:0/1298585341 learned_addr learned my addr 172.21.15.46:0/1298585341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee8dd9b700 1 --2- 172.21.15.46:0/1298585341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582e90 0x56256a58a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee8e59c700 1 -- 172.21.15.46:0/1298585341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582e90 msgr2=0x56256a58a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee8e59c700 1 --2- 172.21.15.46:0/1298585341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582e90 0x56256a58a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee8e59c700 1 -- 172.21.15.46:0/1298585341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a58ddb0 msgr2=0x56256a58a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee8e59c700 1 --2- 172.21.15.46:0/1298585341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a58ddb0 0x56256a58a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee8e59c700 1 -- 172.21.15.46:0/1298585341 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56256a49be40 con 0x56256a582570 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee8e59c700 1 --2- 172.21.15.46:0/1298585341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 0x56256a58d2b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fee80004310 tx=0x7fee80005980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20cb7d119bd592b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee7d7fa700 1 -- 172.21.15.46:0/1298585341 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80030030 con 0x56256a582570 2024-03-18T20:45:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.827+0000 7fee7d7fa700 1 -- 172.21.15.46:0/1298585341 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee80003890 con 0x56256a582570 2024-03-18T20:45:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.828+0000 7fee7d7fa700 1 -- 172.21.15.46:0/1298585341 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8001fe60 con 0x56256a582570 2024-03-18T20:45:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.828+0000 7feea1112b80 1 -- 172.21.15.46:0/1298585341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 msgr2=0x56256a58d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.828+0000 7feea1112b80 1 --2- 172.21.15.46:0/1298585341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 0x56256a58d2b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fee80004310 tx=0x7fee80005980 comp rx=0 tx=0).stop 2024-03-18T20:45:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.828+0000 7feea1112b80 1 -- 172.21.15.46:0/1298585341 shutdown_connections 2024-03-18T20:45:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.828+0000 7feea1112b80 1 --2- 172.21.15.46:0/1298585341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a58ddb0 0x56256a58a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.828+0000 7feea1112b80 1 --2- 172.21.15.46:0/1298585341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582570 0x56256a58d2b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.828+0000 7feea1112b80 1 --2- 172.21.15.46:0/1298585341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a582e90 0x56256a58a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.828+0000 7feea1112b80 1 -- 172.21.15.46:0/1298585341 >> 172.21.15.46:0/1298585341 conn(0x56256a4f91c0 msgr2=0x56256a589330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.828+0000 7feea1112b80 1 -- 172.21.15.46:0/1298585341 shutdown_connections 2024-03-18T20:45:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.828+0000 7feea1112b80 1 -- 172.21.15.46:0/1298585341 wait complete. 2024-03-18T20:45:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7feea1112b80 1 -- start start 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7feea1112b80 1 Processor -- start 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7feea1112b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582570 0x56256a594c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7feea1112b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582e90 0x56256a586cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7feea1112b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a58ddb0 0x56256a5872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7feea1112b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256a5879c0 con 0x56256a582570 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7feea1112b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256a587b30 con 0x56256a582e90 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7feea1112b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256a587cf0 con 0x56256a58ddb0 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7feea1112b80 10 client.0 _ll_register_callbacks cb 0x7fee84005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7fee8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582570 0x56256a594c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7fee8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582570 0x56256a594c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41646/0 (socket says 172.21.15.46:41646) 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7fee8e59c700 1 -- 172.21.15.46:0/291504782 learned_addr learned my addr 172.21.15.46:0/291504782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7fee8dd9b700 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582e90 0x56256a586cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.829+0000 7fee8ed9d700 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a58ddb0 0x56256a5872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7fee8e59c700 1 -- 172.21.15.46:0/291504782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a58ddb0 msgr2=0x56256a5872e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7fee8e59c700 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a58ddb0 0x56256a5872e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7fee8e59c700 1 -- 172.21.15.46:0/291504782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582e90 msgr2=0x56256a586cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7fee8e59c700 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582e90 0x56256a586cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7fee8e59c700 1 -- 172.21.15.46:0/291504782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56256a303b00 con 0x56256a582570 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7fee8dd9b700 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582e90 0x56256a586cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7fee8e59c700 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582570 0x56256a594c30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fee80005a80 tx=0x7fee80011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be075c74b272e5b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7fee75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7fee75ffb700 1 -- 172.21.15.46:0/291504782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8001fcf0 con 0x56256a582570 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7fee75ffb700 1 -- 172.21.15.46:0/291504782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fee80031070 con 0x56256a582570 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7feea1112b80 1 -- 172.21.15.46:0/291504782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee80003b40 con 0x56256a582570 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7fee75ffb700 1 -- 172.21.15.46:0/291504782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee8001fe50 con 0x56256a582570 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7feea1112b80 10 client.18897 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7feea1112b80 1 -- 172.21.15.46:0/291504782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56256a35bc70 con 0x56256a582570 2024-03-18T20:45:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.830+0000 7fee75ffb700 1 -- 172.21.15.46:0/291504782 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee80003b40 con 0x56256a582570 2024-03-18T20:45:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.831+0000 7fee75ffb700 1 -- 172.21.15.46:0/291504782 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fee80033070 con 0x56256a582570 2024-03-18T20:45:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.831+0000 7feea1112b80 10 client.18897 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.831+0000 7feea1112b80 10 client.18897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.831+0000 7feea1112b80 20 client.18897 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.831+0000 7feea1112b80 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56256a5f1ad0 0x56256a5f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.831+0000 7feea1112b80 4 client.18897 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:45:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.831+0000 7feea1112b80 1 -- 172.21.15.46:0/291504782 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56256a324c90 con 0x56256a5f1ad0 2024-03-18T20:45:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.831+0000 7fee8dd9b700 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56256a5f1ad0 0x56256a5f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.833+0000 7fee8dd9b700 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56256a5f1ad0 0x56256a5f3ef0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=518ffac4c0f2a635 server_cookie=ea341a22dc73397d in_seq=0 out_seq=0 2024-03-18T20:45:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.833+0000 7fee75ffb700 10 client.18897 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.860+0000 7fee75ffb700 1 -- 172.21.15.46:0/291504782 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56256a324c90 con 0x56256a5f1ad0 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.860+0000 7fee75ffb700 10 client.18897 handle_command_reply: tid=0 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.860+0000 7feea1112b80 1 client.18897 shutdown 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.860+0000 7feea1112b80 1 -- 172.21.15.46:0/291504782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582570 msgr2=0x56256a594c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.860+0000 7feea1112b80 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582570 0x56256a594c30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fee80005a80 tx=0x7fee80011440 comp rx=0 tx=0).stop 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.860+0000 7feea1112b80 1 -- 172.21.15.46:0/291504782 shutdown_connections 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.860+0000 7feea1112b80 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56256a5f1ad0 0x56256a5f3ef0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.860+0000 7feea1112b80 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256a582570 0x56256a594c30 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.860+0000 7feea1112b80 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256a58ddb0 0x56256a5872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.860+0000 7feea1112b80 1 --2- 172.21.15.46:0/291504782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256a582e90 0x56256a586cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.860+0000 7feea1112b80 1 -- 172.21.15.46:0/291504782 >> 172.21.15.46:0/291504782 conn(0x56256a4f91c0 msgr2=0x56256a58a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.860+0000 7fee75ffb700 0 client.18897 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.861+0000 7feea1112b80 1 -- 172.21.15.46:0/291504782 shutdown_connections 2024-03-18T20:45:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.861+0000 7feea1112b80 1 -- 172.21.15.46:0/291504782 wait complete. 2024-03-18T20:45:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.861+0000 7feea1112b80 20 client.18897 trim_cache size 0 max 16384 2024-03-18T20:45:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:45:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:45:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:45:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:45:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.861+0000 7fee94f7d700 1 -- 172.21.15.46:0/1036781621 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fee780a1290 msgr2=0x7fee780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.861+0000 7fee94f7d700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fee780a1290 0x7fee780a3750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fee80007be0 tx=0x7fee800073d0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.861+0000 7fee94f7d700 1 -- 172.21.15.46:0/1036781621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee90106030 msgr2=0x7fee900735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.861+0000 7fee94f7d700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee90106030 0x7fee900735f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee84005770 tx=0x7fee840057a0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.864+0000 7fee94f7d700 1 -- 172.21.15.46:0/1036781621 shutdown_connections 2024-03-18T20:45:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.864+0000 7fee94f7d700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fee780a1290 0x7fee780a3750 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.864+0000 7fee94f7d700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90100f10 0x7fee900730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.864+0000 7fee94f7d700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee90073b30 0x7fee901a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.864+0000 7fee94f7d700 1 --2- 172.21.15.46:0/1036781621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee90106030 0x7fee900735f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.864+0000 7fee94f7d700 1 -- 172.21.15.46:0/1036781621 >> 172.21.15.46:0/1036781621 conn(0x7fee900fc770 msgr2=0x7fee900fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.864+0000 7fee94f7d700 1 -- 172.21.15.46:0/1036781621 shutdown_connections 2024-03-18T20:45:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:45:50.864+0000 7fee94f7d700 1 -- 172.21.15.46:0/1036781621 wait complete. 2024-03-18T20:45:50.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:45:50.966 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:45:51.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.119+0000 7fbfcebac700 1 -- 172.21.15.46:0/1484601321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8106090 msgr2=0x7fbfc810dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.119+0000 7fbfcebac700 1 --2- 172.21.15.46:0/1484601321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8106090 0x7fbfc810dc10 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b0a0 tx=0x7fbfc000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.120+0000 7fbfcebac700 1 -- 172.21.15.46:0/1484601321 shutdown_connections 2024-03-18T20:45:51.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.120+0000 7fbfcebac700 1 --2- 172.21.15.46:0/1484601321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8106090 0x7fbfc810dc10 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.120+0000 7fbfcebac700 1 --2- 172.21.15.46:0/1484601321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfc8100f70 0x7fbfc8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.120+0000 7fbfcebac700 1 --2- 172.21.15.46:0/1484601321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc8101920 0x7fbfc8105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.120+0000 7fbfcebac700 1 -- 172.21.15.46:0/1484601321 >> 172.21.15.46:0/1484601321 conn(0x7fbfc80fc7f0 msgr2=0x7fbfc80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.120+0000 7fbfcebac700 1 -- 172.21.15.46:0/1484601321 shutdown_connections 2024-03-18T20:45:51.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.120+0000 7fbfcebac700 1 -- 172.21.15.46:0/1484601321 wait complete. 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.120+0000 7fbfcebac700 1 Processor -- start 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.121+0000 7fbfcebac700 1 -- start start 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.121+0000 7fbfcebac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8100f70 0x7fbfc8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.121+0000 7fbfcebac700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfc8106090 0x7fbfc8197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.121+0000 7fbfcebac700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc819bbd0 0x7fbfc819c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.121+0000 7fbfcebac700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc819c650 con 0x7fbfc8100f70 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.121+0000 7fbfcebac700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc819c7c0 con 0x7fbfc8106090 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.121+0000 7fbfcebac700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc819c980 con 0x7fbfc819bbd0 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.121+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfc8106090 0x7fbfc8197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.121+0000 7fbfcc948700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8100f70 0x7fbfc8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.121+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfc8106090 0x7fbfc8197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56360/0 (socket says 172.21.15.46:56360) 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.121+0000 7fbfc7fff700 1 -- 172.21.15.46:0/2616839788 learned_addr learned my addr 172.21.15.46:0/2616839788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:51.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.121+0000 7fbfcd149700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc819bbd0 0x7fbfc819c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.122+0000 7fbfc7fff700 1 -- 172.21.15.46:0/2616839788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc819bbd0 msgr2=0x7fbfc819c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.122+0000 7fbfc7fff700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc819bbd0 0x7fbfc819c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.122+0000 7fbfc7fff700 1 -- 172.21.15.46:0/2616839788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8100f70 msgr2=0x7fbfc8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.122+0000 7fbfc7fff700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8100f70 0x7fbfc8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.122+0000 7fbfc7fff700 1 -- 172.21.15.46:0/2616839788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc009520 con 0x7fbfc8106090 2024-03-18T20:45:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.122+0000 7fbfcd149700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc819bbd0 0x7fbfc819c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.122+0000 7fbfc7fff700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfc8106090 0x7fbfc8197880 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400d980 tx=0x7fbfb400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.122+0000 7fbfc5ffb700 1 -- 172.21.15.46:0/2616839788 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4009930 con 0x7fbfc8106090 2024-03-18T20:45:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.122+0000 7fbfc5ffb700 1 -- 172.21.15.46:0/2616839788 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfb4010460 con 0x7fbfc8106090 2024-03-18T20:45:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.122+0000 7fbfcebac700 1 -- 172.21.15.46:0/2616839788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc0009d00 con 0x7fbfc8106090 2024-03-18T20:45:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.122+0000 7fbfc5ffb700 1 -- 172.21.15.46:0/2616839788 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400f620 con 0x7fbfc8106090 2024-03-18T20:45:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.122+0000 7fbfcebac700 1 -- 172.21.15.46:0/2616839788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc80733c0 con 0x7fbfc8106090 2024-03-18T20:45:51.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.124+0000 7fbfc5ffb700 1 -- 172.21.15.46:0/2616839788 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fbfb40108d0 con 0x7fbfc8106090 2024-03-18T20:45:51.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.125+0000 7fbfc5ffb700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbfb80a1290 0x7fbfb80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.125+0000 7fbfc5ffb700 1 -- 172.21.15.46:0/2616839788 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fbfb40cc7b0 con 0x7fbfc8106090 2024-03-18T20:45:51.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.125+0000 7fbfcc948700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbfb80a1290 0x7fbfb80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.126+0000 7fbfcc948700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbfb80a1290 0x7fbfb80a3750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009df0 tx=0x7fbfbc012040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.126+0000 7fbfdad41b80 1 Processor -- start 2024-03-18T20:45:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.126+0000 7fbfdad41b80 1 -- start start 2024-03-18T20:45:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.126+0000 7fbfdad41b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.126+0000 7fbfdad41b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5bfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.126+0000 7fbfdad41b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5c2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.126+0000 7fbfdad41b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9b50e990 con 0x557a9b5b4a60 2024-03-18T20:45:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.126+0000 7fbfdad41b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9b50c350 con 0x557a9b5c0220 2024-03-18T20:45:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.126+0000 7fbfdad41b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9b507c40 con 0x557a9b5b5380 2024-03-18T20:45:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfcc948700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5bfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5bfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:38846/0 (socket says 172.21.15.46:38846) 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfcc948700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41672/0 (socket says 172.21.15.46:41672) 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfc7fff700 1 -- 172.21.15.46:0/832407397 learned_addr learned my addr 172.21.15.46:0/832407397 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfcd149700 1 --2- 172.21.15.46:0/832407397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfc7fff700 1 -- 172.21.15.46:0/832407397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 msgr2=0x557a9b5c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfc7fff700 1 --2- 172.21.15.46:0/832407397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5c2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfc7fff700 1 -- 172.21.15.46:0/832407397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 msgr2=0x557a9b5b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfc7fff700 1 --2- 172.21.15.46:0/832407397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfc7fff700 1 -- 172.21.15.46:0/832407397 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a9b3893b0 con 0x557a9b5b5380 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfcc948700 1 --2- 172.21.15.46:0/832407397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfc7fff700 1 --2- 172.21.15.46:0/832407397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5bfce0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40a1790 tx=0x7fbfb4099690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ddbb3770e6e752 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfb2ffd700 1 -- 172.21.15.46:0/832407397 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40a1cc0 con 0x557a9b5b5380 2024-03-18T20:45:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfb2ffd700 1 -- 172.21.15.46:0/832407397 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfb4095f50 con 0x557a9b5b5380 2024-03-18T20:45:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.127+0000 7fbfb2ffd700 1 -- 172.21.15.46:0/832407397 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4099cc0 con 0x557a9b5b5380 2024-03-18T20:45:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.128+0000 7fbfdad41b80 1 -- 172.21.15.46:0/832407397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 msgr2=0x557a9b5bfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.128+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/832407397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5bfce0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40a1790 tx=0x7fbfb4099690 comp rx=0 tx=0).stop 2024-03-18T20:45:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.128+0000 7fbfdad41b80 1 -- 172.21.15.46:0/832407397 shutdown_connections 2024-03-18T20:45:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.128+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/832407397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.128+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/832407397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.128+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/832407397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5bfce0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.128+0000 7fbfdad41b80 1 -- 172.21.15.46:0/832407397 >> 172.21.15.46:0/832407397 conn(0x557a9b52b220 msgr2=0x557a9b5b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.128+0000 7fbfdad41b80 1 -- 172.21.15.46:0/832407397 shutdown_connections 2024-03-18T20:45:51.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.128+0000 7fbfdad41b80 1 -- 172.21.15.46:0/832407397 wait complete. 2024-03-18T20:45:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.129+0000 7fbfdad41b80 1 -- start start 2024-03-18T20:45:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.129+0000 7fbfdad41b80 1 Processor -- start 2024-03-18T20:45:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.129+0000 7fbfdad41b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5bfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.129+0000 7fbfdad41b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5bcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.129+0000 7fbfdad41b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5bd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.129+0000 7fbfdad41b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9b4a3d80 con 0x557a9b5b4a60 2024-03-18T20:45:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.129+0000 7fbfdad41b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9b4a3a20 con 0x557a9b5c0220 2024-03-18T20:45:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.129+0000 7fbfdad41b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9b38b6a0 con 0x557a9b5b5380 2024-03-18T20:45:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfdad41b80 10 client.0 _ll_register_callbacks cb 0x557a9b30f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfcc948700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5bfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfcd149700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5bd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5bcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfcc948700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5bfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41688/0 (socket says 172.21.15.46:41688) 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfcc948700 1 -- 172.21.15.46:0/206277383 learned_addr learned my addr 172.21.15.46:0/206277383 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfcd149700 1 -- 172.21.15.46:0/206277383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 msgr2=0x557a9b5bcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfcd149700 1 --2- 172.21.15.46:0/206277383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5bcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfcd149700 1 -- 172.21.15.46:0/206277383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 msgr2=0x557a9b5bfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfcd149700 1 --2- 172.21.15.46:0/206277383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5bfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfcd149700 1 -- 172.21.15.46:0/206277383 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc021040 con 0x557a9b5c0220 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfcd149700 1 --2- 172.21.15.46:0/206277383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5bd670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b780 tx=0x7fbfc0007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad90e2634fbd1a7d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfa7fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfa7fff700 1 -- 172.21.15.46:0/206277383 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000e040 con 0x557a9b5c0220 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfa7fff700 1 -- 172.21.15.46:0/206277383 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fbfc0022070 con 0x557a9b5c0220 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.130+0000 7fbfdad41b80 1 -- 172.21.15.46:0/206277383 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a9b5a8ad0 con 0x557a9b5c0220 2024-03-18T20:45:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.131+0000 7fbfa7fff700 1 -- 172.21.15.46:0/206277383 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfc000ba50 con 0x557a9b5c0220 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.131+0000 7fbfdad41b80 10 client.28544 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.131+0000 7fbfdad41b80 1 -- 172.21.15.46:0/206277383 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7fbfc80733c0 con 0x557a9b5c0220 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.131+0000 7fbfa7fff700 1 -- 172.21.15.46:0/206277383 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a9b5a8ad0 con 0x557a9b5c0220 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.131+0000 7fbfa7fff700 1 -- 172.21.15.46:0/206277383 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbfc000b0d0 con 0x557a9b5c0220 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.131+0000 7fbfdad41b80 10 client.28544 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.131+0000 7fbfdad41b80 10 client.28544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.131+0000 7fbfdad41b80 20 client.28544 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.131+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/206277383 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557a9b5cfd40 0x557a9b5d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.131+0000 7fbfdad41b80 4 client.28544 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.131+0000 7fbfdad41b80 1 -- 172.21.15.46:0/206277383 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a9b500700 con 0x557a9b5cfd40 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.131+0000 7fbfcc948700 1 --2- 172.21.15.46:0/206277383 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557a9b5cfd40 0x557a9b5d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.132+0000 7fbfa7fff700 10 client.28544 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.132+0000 7fbfcc948700 1 --2- 172.21.15.46:0/206277383 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557a9b5cfd40 0x557a9b5d2160 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3e8676b8e1d4104 server_cookie=5ba808b1fb06fe11 in_seq=0 out_seq=0 2024-03-18T20:45:51.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfa7fff700 1 -- 172.21.15.46:0/206277383 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557a9b500700 con 0x557a9b5cfd40 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfa7fff700 10 client.28544 handle_command_reply: tid=0 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfdad41b80 1 client.28544 shutdown 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfdad41b80 1 -- 172.21.15.46:0/206277383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 msgr2=0x557a9b5bd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/206277383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5bd670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b780 tx=0x7fbfc0007f70 comp rx=0 tx=0).stop 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfdad41b80 1 -- 172.21.15.46:0/206277383 shutdown_connections 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/206277383 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557a9b5cfd40 0x557a9b5d2160 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/206277383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5bfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/206277383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5bd670 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfa7fff700 0 client.28544 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/206277383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5bcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfdad41b80 1 -- 172.21.15.46:0/206277383 >> 172.21.15.46:0/206277383 conn(0x557a9b52b220 msgr2=0x557a9b5b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfdad41b80 1 -- 172.21.15.46:0/206277383 shutdown_connections 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.133+0000 7fbfdad41b80 1 -- 172.21.15.46:0/206277383 wait complete. 2024-03-18T20:45:51.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.134+0000 7fbfdad41b80 20 client.28544 trim_cache size 0 max 16384 2024-03-18T20:45:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.155+0000 7fbfdad41b80 1 Processor -- start 2024-03-18T20:45:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.155+0000 7fbfdad41b80 1 -- start start 2024-03-18T20:45:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfdad41b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5b4a60 0x557a9b5c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfdad41b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b5380 0x557a9b5b8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfdad41b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5c0220 0x557a9b5b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfdad41b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9b5cd420 con 0x557a9b5b5380 2024-03-18T20:45:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfdad41b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9b5b5d30 con 0x557a9b5b4a60 2024-03-18T20:45:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfdad41b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9b5b6300 con 0x557a9b5c0220 2024-03-18T20:45:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b5380 0x557a9b5b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b5380 0x557a9b5b8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41698/0 (socket says 172.21.15.46:41698) 2024-03-18T20:45:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfcc948700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5b4a60 0x557a9b5c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfc7fff700 1 -- 172.21.15.46:0/4149051434 learned_addr learned my addr 172.21.15.46:0/4149051434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfcd149700 1 --2- 172.21.15.46:0/4149051434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5c0220 0x557a9b5b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfc7fff700 1 -- 172.21.15.46:0/4149051434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5c0220 msgr2=0x557a9b5b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfc7fff700 1 --2- 172.21.15.46:0/4149051434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5c0220 0x557a9b5b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfc7fff700 1 -- 172.21.15.46:0/4149051434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5b4a60 msgr2=0x557a9b5c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfc7fff700 1 --2- 172.21.15.46:0/4149051434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5b4a60 0x557a9b5c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.156+0000 7fbfc7fff700 1 -- 172.21.15.46:0/4149051434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a9b335b60 con 0x557a9b5b5380 2024-03-18T20:45:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.157+0000 7fbfc7fff700 1 --2- 172.21.15.46:0/4149051434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b5380 0x557a9b5b8220 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40994f0 tx=0x7fbfb40ca3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fce59cec9a12b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.157+0000 7fbfb37fe700 1 -- 172.21.15.46:0/4149051434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40a1cc0 con 0x557a9b5b5380 2024-03-18T20:45:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.157+0000 7fbfb37fe700 1 -- 172.21.15.46:0/4149051434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfb4097be0 con 0x557a9b5b5380 2024-03-18T20:45:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.157+0000 7fbfb37fe700 1 -- 172.21.15.46:0/4149051434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4099d00 con 0x557a9b5b5380 2024-03-18T20:45:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.158+0000 7fbfdad41b80 1 -- 172.21.15.46:0/4149051434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b5380 msgr2=0x557a9b5b8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.158+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/4149051434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b5380 0x557a9b5b8220 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40994f0 tx=0x7fbfb40ca3e0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.158+0000 7fbfdad41b80 1 -- 172.21.15.46:0/4149051434 shutdown_connections 2024-03-18T20:45:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.158+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/4149051434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5c0220 0x557a9b5b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.158+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/4149051434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5b4a60 0x557a9b5c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.158+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/4149051434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b5380 0x557a9b5b8220 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.158+0000 7fbfdad41b80 1 -- 172.21.15.46:0/4149051434 >> 172.21.15.46:0/4149051434 conn(0x557a9b52b220 msgr2=0x557a9b5b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.158+0000 7fbfdad41b80 1 -- 172.21.15.46:0/4149051434 shutdown_connections 2024-03-18T20:45:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.158+0000 7fbfdad41b80 1 -- 172.21.15.46:0/4149051434 wait complete. 2024-03-18T20:45:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.158+0000 7fbfdad41b80 1 -- start start 2024-03-18T20:45:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.158+0000 7fbfdad41b80 1 Processor -- start 2024-03-18T20:45:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.159+0000 7fbfdad41b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5c6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.159+0000 7fbfdad41b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.159+0000 7fbfdad41b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.159+0000 7fbfdad41b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9b5c53f0 con 0x557a9b5b4a60 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.159+0000 7fbfdad41b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9b5c5560 con 0x557a9b5c0220 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.159+0000 7fbfdad41b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9b5c5720 con 0x557a9b5b5380 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.159+0000 7fbfdad41b80 10 client.0 _ll_register_callbacks cb 0x7fbfc0007ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.159+0000 7fbfcc948700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.159+0000 7fbfc7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.159+0000 7fbfcc948700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41700/0 (socket says 172.21.15.46:41700) 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.159+0000 7fbfcc948700 1 -- 172.21.15.46:0/1077284078 learned_addr learned my addr 172.21.15.46:0/1077284078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.159+0000 7fbfcd149700 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfc7fff700 1 -- 172.21.15.46:0/1077284078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 msgr2=0x557a9b5c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfc7fff700 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5c4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfc7fff700 1 -- 172.21.15.46:0/1077284078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 msgr2=0x557a9b5c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfc7fff700 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfc7fff700 1 -- 172.21.15.46:0/1077284078 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a9b3893b0 con 0x557a9b5b5380 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfcc948700 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfcd149700 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfc7fff700 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5c6b90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x557a9b5be040 tx=0x7fbfb4097d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a7d388a1c2248e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfa7fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfa7fff700 1 -- 172.21.15.46:0/1077284078 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4099ca0 con 0x557a9b5b5380 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfa7fff700 1 -- 172.21.15.46:0/1077284078 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fbfb40b92c0 con 0x557a9b5b5380 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfdad41b80 1 -- 172.21.15.46:0/1077284078 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfb4095f50 con 0x557a9b5b5380 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfa7fff700 1 -- 172.21.15.46:0/1077284078 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfb40a1cc0 con 0x557a9b5b5380 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfdad41b80 10 client.28567 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfdad41b80 1 -- 172.21.15.46:0/1077284078 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557a9b4cdea0 con 0x557a9b5b5380 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.160+0000 7fbfa7fff700 1 -- 172.21.15.46:0/1077284078 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfb4095f50 con 0x557a9b5b5380 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.161+0000 7fbfa7fff700 1 -- 172.21.15.46:0/1077284078 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbfb4095630 con 0x557a9b5b5380 2024-03-18T20:45:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.161+0000 7fbfdad41b80 10 client.28567 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.161+0000 7fbfdad41b80 10 client.28567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.161+0000 7fbfdad41b80 20 client.28567 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.161+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557a9b624900 0x557a9b626d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.161+0000 7fbfdad41b80 4 client.28567 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.161+0000 7fbfdad41b80 1 -- 172.21.15.46:0/1077284078 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557a9b356cf0 con 0x557a9b624900 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.161+0000 7fbfcc948700 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557a9b624900 0x557a9b626d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.161+0000 7fbfa7fff700 10 client.28567 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.161+0000 7fbfcc948700 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557a9b624900 0x557a9b626d20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46b4c3e1707e52cf server_cookie=d535dd1631b7d732 in_seq=0 out_seq=0 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.161+0000 7fbfa7fff700 1 -- 172.21.15.46:0/1077284078 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557a9b356cf0 con 0x557a9b624900 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.161+0000 7fbfa7fff700 10 client.28567 handle_command_reply: tid=0 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfdad41b80 1 client.28567 shutdown 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfdad41b80 1 -- 172.21.15.46:0/1077284078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 msgr2=0x557a9b5c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5c6b90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x557a9b5be040 tx=0x7fbfb4097d50 comp rx=0 tx=0).stop 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfdad41b80 1 -- 172.21.15.46:0/1077284078 shutdown_connections 2024-03-18T20:45:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557a9b624900 0x557a9b626d20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a9b5b4a60 0x557a9b5c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a9b5c0220 0x557a9b5c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfa7fff700 0 client.28567 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfdad41b80 1 --2- 172.21.15.46:0/1077284078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9b5b5380 0x557a9b5c6b90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfdad41b80 1 -- 172.21.15.46:0/1077284078 >> 172.21.15.46:0/1077284078 conn(0x557a9b52b220 msgr2=0x557a9b5b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfdad41b80 1 -- 172.21.15.46:0/1077284078 shutdown_connections 2024-03-18T20:45:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfdad41b80 1 -- 172.21.15.46:0/1077284078 wait complete. 2024-03-18T20:45:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.162+0000 7fbfdad41b80 20 client.28567 trim_cache size 0 max 16384 2024-03-18T20:45:51.170 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:45:51.170 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:45:51.171 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:45:51.171 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:45:51.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.163+0000 7fbfcebac700 1 -- 172.21.15.46:0/2616839788 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbfb80a1290 msgr2=0x7fbfb80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.163+0000 7fbfcebac700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbfb80a1290 0x7fbfb80a3750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009df0 tx=0x7fbfbc012040 comp rx=0 tx=0).stop 2024-03-18T20:45:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.163+0000 7fbfcebac700 1 -- 172.21.15.46:0/2616839788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfc8106090 msgr2=0x7fbfc8197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.163+0000 7fbfcebac700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfc8106090 0x7fbfc8197880 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400d980 tx=0x7fbfb400dc90 comp rx=0 tx=0).stop 2024-03-18T20:45:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.163+0000 7fbfcebac700 1 -- 172.21.15.46:0/2616839788 shutdown_connections 2024-03-18T20:45:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.163+0000 7fbfcebac700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbfb80a1290 0x7fbfb80a3750 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.163+0000 7fbfcebac700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc8100f70 0x7fbfc8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.163+0000 7fbfcebac700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc819bbd0 0x7fbfc819c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.163+0000 7fbfcebac700 1 --2- 172.21.15.46:0/2616839788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfc8106090 0x7fbfc8197880 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.163+0000 7fbfcebac700 1 -- 172.21.15.46:0/2616839788 >> 172.21.15.46:0/2616839788 conn(0x7fbfc80fc7f0 msgr2=0x7fbfc80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.163+0000 7fbfcebac700 1 -- 172.21.15.46:0/2616839788 shutdown_connections 2024-03-18T20:45:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.163+0000 7fbfcebac700 1 -- 172.21.15.46:0/2616839788 wait complete. 2024-03-18T20:45:51.180 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:45:51.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:45:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.335+0000 7fe7a54b8700 1 -- 172.21.15.46:0/3943605685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a0101df0 msgr2=0x7fe7a01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.335+0000 7fe7a54b8700 1 --2- 172.21.15.46:0/3943605685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a0101df0 0x7fe7a01021d0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe790009990 tx=0x7fe790009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.335+0000 7fe7a54b8700 1 -- 172.21.15.46:0/3943605685 shutdown_connections 2024-03-18T20:45:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.335+0000 7fe7a54b8700 1 --2- 172.21.15.46:0/3943605685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a0101df0 0x7fe7a01021d0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.335+0000 7fe7a54b8700 1 --2- 172.21.15.46:0/3943605685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7a010b210 0x7fe7a010d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.335+0000 7fe7a54b8700 1 --2- 172.21.15.46:0/3943605685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7a0102710 0x7fe7a010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.335+0000 7fe7a54b8700 1 -- 172.21.15.46:0/3943605685 >> 172.21.15.46:0/3943605685 conn(0x7fe7a00fb5f0 msgr2=0x7fe7a00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.335+0000 7fe7a54b8700 1 -- 172.21.15.46:0/3943605685 shutdown_connections 2024-03-18T20:45:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.335+0000 7fe7a54b8700 1 -- 172.21.15.46:0/3943605685 wait complete. 2024-03-18T20:45:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe7a54b8700 1 Processor -- start 2024-03-18T20:45:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe7a54b8700 1 -- start start 2024-03-18T20:45:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe7a54b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7a0101df0 0x7fe7a0199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe7a54b8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7a0102710 0x7fe7a0199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe7a54b8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a010b210 0x7fe7a019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe7a54b8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a019e1c0 con 0x7fe7a010b210 2024-03-18T20:45:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe7a54b8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a019e330 con 0x7fe7a0101df0 2024-03-18T20:45:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe7a54b8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a019e4f0 con 0x7fe7a0102710 2024-03-18T20:45:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe79effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7a0101df0 0x7fe7a0199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe79f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a010b210 0x7fe7a019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe79e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7a0102710 0x7fe7a0199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe79f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a010b210 0x7fe7a019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41726/0 (socket says 172.21.15.46:41726) 2024-03-18T20:45:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe79f7fe700 1 -- 172.21.15.46:0/4204287113 learned_addr learned my addr 172.21.15.46:0/4204287113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.336+0000 7fe79effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7a0101df0 0x7fe7a0199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56418/0 (socket says 172.21.15.46:56418) 2024-03-18T20:45:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.337+0000 7fe79effd700 1 -- 172.21.15.46:0/4204287113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7a0102710 msgr2=0x7fe7a0199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.337+0000 7fe79effd700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7a0102710 0x7fe7a0199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.337+0000 7fe79effd700 1 -- 172.21.15.46:0/4204287113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a010b210 msgr2=0x7fe7a019db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.337+0000 7fe79effd700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a010b210 0x7fe7a019db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.337+0000 7fe79effd700 1 -- 172.21.15.46:0/4204287113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7900095f0 con 0x7fe7a0101df0 2024-03-18T20:45:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.337+0000 7fe79f7fe700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a010b210 0x7fe7a019db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.337+0000 7fe79effd700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7a0101df0 0x7fe7a0199310 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe790000c00 tx=0x7fe790004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.337+0000 7fe797fff700 1 -- 172.21.15.46:0/4204287113 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79001d070 con 0x7fe7a0101df0 2024-03-18T20:45:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.337+0000 7fe797fff700 1 -- 172.21.15.46:0/4204287113 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe790021410 con 0x7fe7a0101df0 2024-03-18T20:45:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.337+0000 7fe7a54b8700 1 -- 172.21.15.46:0/4204287113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a019e770 con 0x7fe7a0101df0 2024-03-18T20:45:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.337+0000 7fe797fff700 1 -- 172.21.15.46:0/4204287113 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790017650 con 0x7fe7a0101df0 2024-03-18T20:45:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.337+0000 7fe7a54b8700 1 -- 172.21.15.46:0/4204287113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a01a6010 con 0x7fe7a0101df0 2024-03-18T20:45:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.338+0000 7fe7a54b8700 1 -- 172.21.15.46:0/4204287113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a00698f0 con 0x7fe7a0101df0 2024-03-18T20:45:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.340+0000 7fe797fff700 1 -- 172.21.15.46:0/4204287113 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe790021580 con 0x7fe7a0101df0 2024-03-18T20:45:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.340+0000 7fe797fff700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe78c0a1290 0x7fe78c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.340+0000 7fe797fff700 1 -- 172.21.15.46:0/4204287113 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe7900ce010 con 0x7fe7a0101df0 2024-03-18T20:45:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.340+0000 7fe79e7fc700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe78c0a1290 0x7fe78c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.341+0000 7fe79e7fc700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe78c0a1290 0x7fe78c0a3750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00fda30 tx=0x7fe788009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.344+0000 7fe797fff700 1 -- 172.21.15.46:0/4204287113 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe7900974d0 con 0x7fe7a0101df0 2024-03-18T20:45:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.637+0000 7fe7a54b8700 1 -- 172.21.15.46:0/4204287113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7a01a61a0 con 0x7fe7a0101df0 2024-03-18T20:45:51.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.638+0000 7fe797fff700 1 -- 172.21.15.46:0/4204287113 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+10916 (secure 0 0 0) 0x7fe79009aaf0 con 0x7fe7a0101df0 2024-03-18T20:45:51.641 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:45:51.641 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:45:43.243732+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:45:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.640+0000 7fe7a54b8700 1 -- 172.21.15.46:0/4204287113 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe78c0a1290 msgr2=0x7fe78c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.640+0000 7fe7a54b8700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe78c0a1290 0x7fe78c0a3750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00fda30 tx=0x7fe788009260 comp rx=0 tx=0).stop 2024-03-18T20:45:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.640+0000 7fe7a54b8700 1 -- 172.21.15.46:0/4204287113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7a0101df0 msgr2=0x7fe7a0199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.640+0000 7fe7a54b8700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7a0101df0 0x7fe7a0199310 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe790000c00 tx=0x7fe790004730 comp rx=0 tx=0).stop 2024-03-18T20:45:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.640+0000 7fe7a54b8700 1 -- 172.21.15.46:0/4204287113 shutdown_connections 2024-03-18T20:45:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.640+0000 7fe7a54b8700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe78c0a1290 0x7fe78c0a3750 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.640+0000 7fe7a54b8700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a010b210 0x7fe7a019db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.640+0000 7fe7a54b8700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7a0101df0 0x7fe7a0199310 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.640+0000 7fe7a54b8700 1 --2- 172.21.15.46:0/4204287113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7a0102710 0x7fe7a0199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.640+0000 7fe7a54b8700 1 -- 172.21.15.46:0/4204287113 >> 172.21.15.46:0/4204287113 conn(0x7fe7a00fb5f0 msgr2=0x7fe7a00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.641+0000 7fe7a54b8700 1 -- 172.21.15.46:0/4204287113 shutdown_connections 2024-03-18T20:45:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.641+0000 7fe7a54b8700 1 -- 172.21.15.46:0/4204287113 wait complete. 2024-03-18T20:45:51.644 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 49 2024-03-18T20:45:51.658 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:45:51.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:45:51.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 358717 3.67 MB/sec execute 406 sec latency 16.844 ms 2024-03-18T20:45:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.810+0000 7f7c43126700 1 -- 172.21.15.46:0/2736269362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c0ffe50 msgr2=0x7f7c3c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.810+0000 7f7c43126700 1 --2- 172.21.15.46:0/2736269362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c0ffe50 0x7f7c3c100230 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f7c300098f0 tx=0x7f7c30009c00 comp rx=0 tx=0).stop 2024-03-18T20:45:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.810+0000 7f7c43126700 1 -- 172.21.15.46:0/2736269362 shutdown_connections 2024-03-18T20:45:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.810+0000 7f7c43126700 1 --2- 172.21.15.46:0/2736269362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c0ffe50 0x7f7c3c100230 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.810+0000 7f7c43126700 1 --2- 172.21.15.46:0/2736269362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c3c104f60 0x7f7c3c10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.810+0000 7f7c43126700 1 --2- 172.21.15.46:0/2736269362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c3c100800 0x7f7c3c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.810+0000 7f7c43126700 1 -- 172.21.15.46:0/2736269362 >> 172.21.15.46:0/2736269362 conn(0x7f7c3c076d40 msgr2=0x7f7c3c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.810+0000 7f7c43126700 1 -- 172.21.15.46:0/2736269362 shutdown_connections 2024-03-18T20:45:51.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.810+0000 7f7c43126700 1 -- 172.21.15.46:0/2736269362 wait complete. 2024-03-18T20:45:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.810+0000 7f7c43126700 1 Processor -- start 2024-03-18T20:45:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.810+0000 7f7c43126700 1 -- start start 2024-03-18T20:45:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c43126700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c3c0ffe50 0x7f7c3c196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c43126700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c104f60 0x7f7c3c196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c43126700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c3c19aa30 0x7f7c3c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c43126700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c19b4b0 con 0x7f7c3c104f60 2024-03-18T20:45:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c43126700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c19b620 con 0x7f7c3c0ffe50 2024-03-18T20:45:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c43126700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c19b7e0 con 0x7f7c3c19aa30 2024-03-18T20:45:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c104f60 0x7f7c3c196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c40ec2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c3c0ffe50 0x7f7c3c196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c104f60 0x7f7c3c196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41740/0 (socket says 172.21.15.46:41740) 2024-03-18T20:45:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c3bfff700 1 -- 172.21.15.46:0/1330563782 learned_addr learned my addr 172.21.15.46:0/1330563782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c416c3700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c3c19aa30 0x7f7c3c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c40ec2700 1 -- 172.21.15.46:0/1330563782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c3c19aa30 msgr2=0x7f7c3c19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c40ec2700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c3c19aa30 0x7f7c3c19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c40ec2700 1 -- 172.21.15.46:0/1330563782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c104f60 msgr2=0x7f7c3c196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c40ec2700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c104f60 0x7f7c3c196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c40ec2700 1 -- 172.21.15.46:0/1330563782 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30009520 con 0x7f7c3c0ffe50 2024-03-18T20:45:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.811+0000 7f7c40ec2700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c3c0ffe50 0x7f7c3c196150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009230 tx=0x7f7c3000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.812+0000 7f7c39ffb700 1 -- 172.21.15.46:0/1330563782 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3001c070 con 0x7f7c3c0ffe50 2024-03-18T20:45:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.812+0000 7f7c39ffb700 1 -- 172.21.15.46:0/1330563782 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c3000b9c0 con 0x7f7c3c0ffe50 2024-03-18T20:45:51.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.812+0000 7f7c43126700 1 -- 172.21.15.46:0/1330563782 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3c067df0 con 0x7f7c3c0ffe50 2024-03-18T20:45:51.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.812+0000 7f7c39ffb700 1 -- 172.21.15.46:0/1330563782 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c300038e0 con 0x7f7c3c0ffe50 2024-03-18T20:45:51.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.812+0000 7f7c43126700 1 -- 172.21.15.46:0/1330563782 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3c0682f0 con 0x7f7c3c0ffe50 2024-03-18T20:45:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.812+0000 7f7c416c3700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c3c19aa30 0x7f7c3c19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.814+0000 7f7c39ffb700 1 -- 172.21.15.46:0/1330563782 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7c30020030 con 0x7f7c3c0ffe50 2024-03-18T20:45:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.815+0000 7f7c39ffb700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c2c0a1390 0x7f7c2c0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.815+0000 7f7c39ffb700 1 -- 172.21.15.46:0/1330563782 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7c300cd9e0 con 0x7f7c3c0ffe50 2024-03-18T20:45:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.815+0000 7f7c3bfff700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c2c0a1390 0x7f7c2c0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.816+0000 7f7c3bfff700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c2c0a1390 0x7f7c2c0a3850 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c1976a0 tx=0x7f7c28010040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.816+0000 7f7c4f2bbb80 1 Processor -- start 2024-03-18T20:45:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.816+0000 7f7c4f2bbb80 1 -- start start 2024-03-18T20:45:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.816+0000 7f7c4f2bbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679651a60 0x55a679651e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c4f2bbb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 0x55a67965cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c4f2bbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a67965d220 0x55a67965f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c4f2bbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6795ab990 con 0x55a67965d220 2024-03-18T20:45:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c4f2bbb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6795a9350 con 0x55a679652380 2024-03-18T20:45:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c4f2bbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6795a4c40 con 0x55a679651a60 2024-03-18T20:45:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c416c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a67965d220 0x55a67965f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 0x55a67965cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 0x55a67965cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56452/0 (socket says 172.21.15.46:56452) 2024-03-18T20:45:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c416c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a67965d220 0x55a67965f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41746/0 (socket says 172.21.15.46:41746) 2024-03-18T20:45:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c3bfff700 1 -- 172.21.15.46:0/4260683653 learned_addr learned my addr 172.21.15.46:0/4260683653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c40ec2700 1 --2- 172.21.15.46:0/4260683653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679651a60 0x55a679651e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c3bfff700 1 -- 172.21.15.46:0/4260683653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679651a60 msgr2=0x55a679651e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c3bfff700 1 --2- 172.21.15.46:0/4260683653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679651a60 0x55a679651e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c3bfff700 1 -- 172.21.15.46:0/4260683653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a67965d220 msgr2=0x55a67965f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c3bfff700 1 --2- 172.21.15.46:0/4260683653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a67965d220 0x55a67965f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c3bfff700 1 -- 172.21.15.46:0/4260683653 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6794263b0 con 0x55a679652380 2024-03-18T20:45:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c3bfff700 1 --2- 172.21.15.46:0/4260683653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 0x55a67965cce0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7c28005950 tx=0x7f7c28016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c16578b8d5aad9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.817+0000 7f7c26ffd700 1 -- 172.21.15.46:0/4260683653 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28028070 con 0x55a679652380 2024-03-18T20:45:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.818+0000 7f7c26ffd700 1 -- 172.21.15.46:0/4260683653 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c28014040 con 0x55a679652380 2024-03-18T20:45:51.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.818+0000 7f7c26ffd700 1 -- 172.21.15.46:0/4260683653 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28005660 con 0x55a679652380 2024-03-18T20:45:51.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.818+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/4260683653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 msgr2=0x55a67965cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.818+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/4260683653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 0x55a67965cce0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7c28005950 tx=0x7f7c28016000 comp rx=0 tx=0).stop 2024-03-18T20:45:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.818+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/4260683653 shutdown_connections 2024-03-18T20:45:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.818+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/4260683653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a67965d220 0x55a67965f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.818+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/4260683653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679651a60 0x55a679651e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.818+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/4260683653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 0x55a67965cce0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.818+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/4260683653 >> 172.21.15.46:0/4260683653 conn(0x55a6795c8220 msgr2=0x55a67964e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.819+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/4260683653 shutdown_connections 2024-03-18T20:45:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.819+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/4260683653 wait complete. 2024-03-18T20:45:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.819+0000 7f7c4f2bbb80 1 -- start start 2024-03-18T20:45:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.819+0000 7f7c4f2bbb80 1 Processor -- start 2024-03-18T20:45:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c4f2bbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 0x55a67965cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c4f2bbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679652380 0x55a679659f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c4f2bbb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a67965d220 0x55a67965a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c4f2bbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a679540d80 con 0x55a679651a60 2024-03-18T20:45:51.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c4f2bbb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a679540a20 con 0x55a67965d220 2024-03-18T20:45:51.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c4f2bbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6794286a0 con 0x55a679652380 2024-03-18T20:45:51.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c4f2bbb80 10 client.0 _ll_register_callbacks cb 0x55a6793ac3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:51.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c40ec2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 0x55a67965cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c416c3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a67965d220 0x55a67965a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679652380 0x55a679659f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c40ec2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 0x55a67965cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41756/0 (socket says 172.21.15.46:41756) 2024-03-18T20:45:51.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c40ec2700 1 -- 172.21.15.46:0/133350833 learned_addr learned my addr 172.21.15.46:0/133350833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:51.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c3bfff700 1 -- 172.21.15.46:0/133350833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a67965d220 msgr2=0x55a67965a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c3bfff700 1 --2- 172.21.15.46:0/133350833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a67965d220 0x55a67965a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c3bfff700 1 -- 172.21.15.46:0/133350833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 msgr2=0x55a67965cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c3bfff700 1 --2- 172.21.15.46:0/133350833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 0x55a67965cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c3bfff700 1 -- 172.21.15.46:0/133350833 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2801a3f0 con 0x55a679652380 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c40ec2700 1 --2- 172.21.15.46:0/133350833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 0x55a67965cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c3bfff700 1 --2- 172.21.15.46:0/133350833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679652380 0x55a679659f50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800eb50 tx=0x7f7c28009af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9cb906a5fc210a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.820+0000 7f7c1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c1bfff700 1 -- 172.21.15.46:0/133350833 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28028040 con 0x55a679652380 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c1bfff700 1 -- 172.21.15.46:0/133350833 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7c280046f0 con 0x55a679652380 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/133350833 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a679645ad0 con 0x55a679652380 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c1bfff700 1 -- 172.21.15.46:0/133350833 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c28005710 con 0x55a679652380 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c4f2bbb80 10 client.28585 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/133350833 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55a6793d2b60 con 0x55a679652380 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c1bfff700 1 -- 172.21.15.46:0/133350833 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a679645ad0 con 0x55a679652380 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c1bfff700 1 -- 172.21.15.46:0/133350833 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7c2802db40 con 0x55a679652380 2024-03-18T20:45:51.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c4f2bbb80 10 client.28585 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c4f2bbb80 10 client.28585 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:45:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c4f2bbb80 20 client.28585 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/133350833 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a67966ccb0 0x55a67966f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c4f2bbb80 4 client.28585 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/133350833 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a67959d700 con 0x55a67966ccb0 2024-03-18T20:45:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.821+0000 7f7c40ec2700 1 --2- 172.21.15.46:0/133350833 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a67966ccb0 0x55a67966f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.822+0000 7f7c40ec2700 1 --2- 172.21.15.46:0/133350833 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a67966ccb0 0x55a67966f0d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8fbec534ac113daa server_cookie=50c34ec449c4abbe in_seq=0 out_seq=0 2024-03-18T20:45:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.822+0000 7f7c1bfff700 10 client.28585 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:45:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.822+0000 7f7c1bfff700 1 -- 172.21.15.46:0/133350833 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a67959d700 con 0x55a67966ccb0 2024-03-18T20:45:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.822+0000 7f7c1bfff700 10 client.28585 handle_command_reply: tid=0 2024-03-18T20:45:51.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.822+0000 7f7c4f2bbb80 1 client.28585 shutdown 2024-03-18T20:45:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.823+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/133350833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679652380 msgr2=0x55a679659f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.823+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/133350833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679652380 0x55a679659f50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800eb50 tx=0x7f7c28009af0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.823+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/133350833 shutdown_connections 2024-03-18T20:45:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.823+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/133350833 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a67966ccb0 0x55a67966f0d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.823+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/133350833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 0x55a67965cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.823+0000 7f7c1bfff700 0 client.28585 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:45:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.823+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/133350833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a67965d220 0x55a67965a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.823+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/133350833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679652380 0x55a679659f50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.823+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/133350833 >> 172.21.15.46:0/133350833 conn(0x55a6795c8220 msgr2=0x55a67964dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.823+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/133350833 shutdown_connections 2024-03-18T20:45:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.823+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/133350833 wait complete. 2024-03-18T20:45:51.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.823+0000 7f7c4f2bbb80 20 client.28585 trim_cache size 0 max 16384 2024-03-18T20:45:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c4f2bbb80 1 Processor -- start 2024-03-18T20:45:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c4f2bbb80 1 -- start start 2024-03-18T20:45:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c4f2bbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 0x55a679659f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c4f2bbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679652380 0x55a679663e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c4f2bbb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a67965d220 0x55a6796685a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c4f2bbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a67966f460 con 0x55a679651a60 2024-03-18T20:45:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c4f2bbb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a679653550 con 0x55a67965d220 2024-03-18T20:45:51.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c4f2bbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a679664390 con 0x55a679652380 2024-03-18T20:45:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c40ec2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 0x55a679659f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c40ec2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 0x55a679659f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41764/0 (socket says 172.21.15.46:41764) 2024-03-18T20:45:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c40ec2700 1 -- 172.21.15.46:0/3388488359 learned_addr learned my addr 172.21.15.46:0/3388488359 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c416c3700 1 --2- 172.21.15.46:0/3388488359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a67965d220 0x55a6796685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c3bfff700 1 --2- 172.21.15.46:0/3388488359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679652380 0x55a679663e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c416c3700 1 -- 172.21.15.46:0/3388488359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679652380 msgr2=0x55a679663e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c416c3700 1 --2- 172.21.15.46:0/3388488359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679652380 0x55a679663e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c416c3700 1 -- 172.21.15.46:0/3388488359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 msgr2=0x55a679659f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c416c3700 1 --2- 172.21.15.46:0/3388488359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 0x55a679659f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.837+0000 7f7c416c3700 1 -- 172.21.15.46:0/3388488359 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a67956aea0 con 0x55a67965d220 2024-03-18T20:45:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.838+0000 7f7c40ec2700 1 --2- 172.21.15.46:0/3388488359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 0x55a679659f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:51.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.838+0000 7f7c416c3700 1 --2- 172.21.15.46:0/3388488359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a67965d220 0x55a6796685a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400e950 tx=0x7f7c34009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77da197dc0189b9b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.838+0000 7f7c277fe700 1 -- 172.21.15.46:0/3388488359 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34004db0 con 0x55a67965d220 2024-03-18T20:45:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.838+0000 7f7c277fe700 1 -- 172.21.15.46:0/3388488359 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c3400f040 con 0x55a67965d220 2024-03-18T20:45:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.838+0000 7f7c277fe700 1 -- 172.21.15.46:0/3388488359 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34007880 con 0x55a67965d220 2024-03-18T20:45:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.839+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/3388488359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a67965d220 msgr2=0x55a6796685a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.839+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/3388488359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a67965d220 0x55a6796685a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400e950 tx=0x7f7c34009920 comp rx=0 tx=0).stop 2024-03-18T20:45:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.839+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/3388488359 shutdown_connections 2024-03-18T20:45:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.839+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/3388488359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a679651a60 0x55a679659f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.839+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/3388488359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a67965d220 0x55a6796685a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.839+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/3388488359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679652380 0x55a679663e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.839+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/3388488359 >> 172.21.15.46:0/3388488359 conn(0x55a6795c8220 msgr2=0x55a679649a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.839+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/3388488359 shutdown_connections 2024-03-18T20:45:51.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.839+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/3388488359 wait complete. 2024-03-18T20:45:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.839+0000 7f7c4f2bbb80 1 -- start start 2024-03-18T20:45:51.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.839+0000 7f7c4f2bbb80 1 Processor -- start 2024-03-18T20:45:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c4f2bbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679651a60 0x55a6796639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c4f2bbb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 0x55a679661e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c4f2bbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a67965d220 0x55a6796623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c4f2bbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a679662900 con 0x55a67965d220 2024-03-18T20:45:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c4f2bbb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a679662a70 con 0x55a679652380 2024-03-18T20:45:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c4f2bbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a679662c30 con 0x55a679651a60 2024-03-18T20:45:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c4f2bbb80 10 client.0 _ll_register_callbacks cb 0x7f7c34009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 0x55a679661e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c3bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 0x55a679661e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56478/0 (socket says 172.21.15.46:56478) 2024-03-18T20:45:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c3bfff700 1 -- 172.21.15.46:0/1503799477 learned_addr learned my addr 172.21.15.46:0/1503799477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c416c3700 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a67965d220 0x55a6796623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c40ec2700 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679651a60 0x55a6796639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c3bfff700 1 -- 172.21.15.46:0/1503799477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679651a60 msgr2=0x55a6796639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c3bfff700 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679651a60 0x55a6796639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c3bfff700 1 -- 172.21.15.46:0/1503799477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a67965d220 msgr2=0x55a6796623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c3bfff700 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a67965d220 0x55a6796623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.840+0000 7f7c3bfff700 1 -- 172.21.15.46:0/1503799477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3c0682f0 con 0x55a679652380 2024-03-18T20:45:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c3bfff700 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 0x55a679661e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7c28005230 tx=0x7f7c28009950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=680c488fdcb01af3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:51.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:45:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c1bfff700 1 -- 172.21.15.46:0/1503799477 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28028040 con 0x55a679652380 2024-03-18T20:45:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c1bfff700 1 -- 172.21.15.46:0/1503799477 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7c280249d0 con 0x55a679652380 2024-03-18T20:45:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/1503799477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c28014040 con 0x55a679652380 2024-03-18T20:45:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c1bfff700 1 -- 172.21.15.46:0/1503799477 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c28005720 con 0x55a679652380 2024-03-18T20:45:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c4f2bbb80 10 client.28574 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/1503799477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f7c3400e600 con 0x55a679652380 2024-03-18T20:45:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c1bfff700 1 -- 172.21.15.46:0/1503799477 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c2804e080 con 0x55a679652380 2024-03-18T20:45:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c1bfff700 1 -- 172.21.15.46:0/1503799477 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7c2802d460 con 0x55a679652380 2024-03-18T20:45:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c4f2bbb80 10 client.28574 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c4f2bbb80 10 client.28574 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:45:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c4f2bbb80 20 client.28574 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:51.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a6796c1b20 0x55a6796c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.841+0000 7f7c4f2bbb80 4 client.28574 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:45:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.842+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/1503799477 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a6793f3cf0 con 0x55a6796c1b20 2024-03-18T20:45:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.842+0000 7f7c40ec2700 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a6796c1b20 0x55a6796c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.842+0000 7f7c1bfff700 10 client.28574 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:45:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.842+0000 7f7c40ec2700 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a6796c1b20 0x55a6796c3f40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30e4089652503474 server_cookie=6b67039ec2a2ddba in_seq=0 out_seq=0 2024-03-18T20:45:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.842+0000 7f7c1bfff700 1 -- 172.21.15.46:0/1503799477 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a6793f3cf0 con 0x55a6796c1b20 2024-03-18T20:45:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.842+0000 7f7c1bfff700 10 client.28574 handle_command_reply: tid=0 2024-03-18T20:45:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.842+0000 7f7c4f2bbb80 1 client.28574 shutdown 2024-03-18T20:45:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/1503799477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 msgr2=0x55a679661e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 0x55a679661e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7c28005230 tx=0x7f7c28009950 comp rx=0 tx=0).stop 2024-03-18T20:45:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/1503799477 shutdown_connections 2024-03-18T20:45:51.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x55a6796c1b20 0x55a6796c3f40 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a67965d220 0x55a6796623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a679651a60 0x55a6796639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c4f2bbb80 1 --2- 172.21.15.46:0/1503799477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a679652380 0x55a679661e80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c1bfff700 0 client.28574 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:45:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/1503799477 >> 172.21.15.46:0/1503799477 conn(0x55a6795c8220 msgr2=0x55a6796685a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/1503799477 shutdown_connections 2024-03-18T20:45:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c4f2bbb80 1 -- 172.21.15.46:0/1503799477 wait complete. 2024-03-18T20:45:51.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c4f2bbb80 20 client.28574 trim_cache size 0 max 16384 2024-03-18T20:45:51.854 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:45:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c43126700 1 -- 172.21.15.46:0/1330563782 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c2c0a1390 msgr2=0x7f7c2c0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c43126700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c2c0a1390 0x7f7c2c0a3850 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c1976a0 tx=0x7f7c28010040 comp rx=0 tx=0).stop 2024-03-18T20:45:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c43126700 1 -- 172.21.15.46:0/1330563782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c3c0ffe50 msgr2=0x7f7c3c196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.843+0000 7f7c43126700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c3c0ffe50 0x7f7c3c196150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009230 tx=0x7f7c3000fa00 comp rx=0 tx=0).stop 2024-03-18T20:45:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.844+0000 7f7c43126700 1 -- 172.21.15.46:0/1330563782 shutdown_connections 2024-03-18T20:45:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.844+0000 7f7c43126700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c2c0a1390 0x7f7c2c0a3850 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.844+0000 7f7c43126700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c3c19aa30 0x7f7c3c19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.844+0000 7f7c43126700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c3c0ffe50 0x7f7c3c196150 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.844+0000 7f7c43126700 1 --2- 172.21.15.46:0/1330563782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c104f60 0x7f7c3c196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.844+0000 7f7c43126700 1 -- 172.21.15.46:0/1330563782 >> 172.21.15.46:0/1330563782 conn(0x7f7c3c076d40 msgr2=0x7f7c3c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.844+0000 7f7c43126700 1 -- 172.21.15.46:0/1330563782 shutdown_connections 2024-03-18T20:45:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:51.844+0000 7f7c43126700 1 -- 172.21.15.46:0/1330563782 wait complete. 2024-03-18T20:45:51.861 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:45:51.861 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:45:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.006+0000 7f6dd9112700 1 -- 172.21.15.46:0/1855269828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd4100820 msgr2=0x7f6dd4104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.006+0000 7f6dd9112700 1 --2- 172.21.15.46:0/1855269828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd4100820 0x7f6dd4104860 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0098f0 tx=0x7f6dbc009c00 comp rx=0 tx=0).stop 2024-03-18T20:45:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.007+0000 7f6dd9112700 1 -- 172.21.15.46:0/1855269828 shutdown_connections 2024-03-18T20:45:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.007+0000 7f6dd9112700 1 --2- 172.21.15.46:0/1855269828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd40ffe70 0x7f6dd4100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.007+0000 7f6dd9112700 1 --2- 172.21.15.46:0/1855269828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dd4104f90 0x7f6dd410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.007+0000 7f6dd9112700 1 --2- 172.21.15.46:0/1855269828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd4100820 0x7f6dd4104860 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.007+0000 7f6dd9112700 1 -- 172.21.15.46:0/1855269828 >> 172.21.15.46:0/1855269828 conn(0x7f6dd4076d80 msgr2=0x7f6dd4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.007+0000 7f6dd9112700 1 -- 172.21.15.46:0/1855269828 shutdown_connections 2024-03-18T20:45:52.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.007+0000 7f6dd9112700 1 -- 172.21.15.46:0/1855269828 wait complete. 2024-03-18T20:45:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.008+0000 7f6dd9112700 1 Processor -- start 2024-03-18T20:45:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.008+0000 7f6dd9112700 1 -- start start 2024-03-18T20:45:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.008+0000 7f6dd9112700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd40ffe70 0x7f6dd4197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.008+0000 7f6dd9112700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dd4104f90 0x7f6dd41977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.008+0000 7f6dd9112700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd419bb50 0x7f6dd419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.008+0000 7f6dd9112700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd419c5d0 con 0x7f6dd40ffe70 2024-03-18T20:45:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.008+0000 7f6dd9112700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd419c740 con 0x7f6dd4104f90 2024-03-18T20:45:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.008+0000 7f6dd9112700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd419c900 con 0x7f6dd419bb50 2024-03-18T20:45:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd40ffe70 0x7f6dd4197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd359e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd419bb50 0x7f6dd419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd359e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd419bb50 0x7f6dd419bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:38940/0 (socket says 172.21.15.46:38940) 2024-03-18T20:45:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd359e700 1 -- 172.21.15.46:0/3459664770 learned_addr learned my addr 172.21.15.46:0/3459664770 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd259c700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dd4104f90 0x7f6dd41977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd40ffe70 0x7f6dd4197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41782/0 (socket says 172.21.15.46:41782) 2024-03-18T20:45:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd359e700 1 -- 172.21.15.46:0/3459664770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dd4104f90 msgr2=0x7f6dd41977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd359e700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dd4104f90 0x7f6dd41977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd359e700 1 -- 172.21.15.46:0/3459664770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd40ffe70 msgr2=0x7f6dd4197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd359e700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd40ffe70 0x7f6dd4197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd359e700 1 -- 172.21.15.46:0/3459664770 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc40095f0 con 0x7f6dd419bb50 2024-03-18T20:45:52.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd359e700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd419bb50 0x7f6dd419bfd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00ef30 tx=0x7f6dcc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dcbfff700 1 -- 172.21.15.46:0/3459664770 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc018070 con 0x7f6dd419bb50 2024-03-18T20:45:52.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd9112700 1 -- 172.21.15.46:0/3459664770 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dbc009520 con 0x7f6dd419bb50 2024-03-18T20:45:52.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dcbfff700 1 -- 172.21.15.46:0/3459664770 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dcc00f040 con 0x7f6dd419bb50 2024-03-18T20:45:52.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dcbfff700 1 -- 172.21.15.46:0/3459664770 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc00cd00 con 0x7f6dd419bb50 2024-03-18T20:45:52.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.009+0000 7f6dd9112700 1 -- 172.21.15.46:0/3459664770 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd41a4150 con 0x7f6dd419bb50 2024-03-18T20:45:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.012+0000 7f6dcbfff700 1 -- 172.21.15.46:0/3459664770 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6dcc003a00 con 0x7f6dd419bb50 2024-03-18T20:45:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.013+0000 7f6dcbfff700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6dc00a1290 0x7f6dc00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.013+0000 7f6dd2d9d700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6dc00a1290 0x7f6dc00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.013+0000 7f6dcbfff700 1 -- 172.21.15.46:0/3459664770 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f6dcc0cc7f0 con 0x7f6dd419bb50 2024-03-18T20:45:52.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.014+0000 7f6dd2d9d700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6dc00a1290 0x7f6dc00a3750 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4005fd0 tx=0x7f6dc4009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.014+0000 7f6de52a7b80 1 Processor -- start 2024-03-18T20:45:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.014+0000 7f6de52a7b80 1 -- start start 2024-03-18T20:45:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.014+0000 7f6de52a7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e441e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.014+0000 7f6de52a7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e44cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.014+0000 7f6de52a7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e44f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.014+0000 7f6de52a7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e39b990 con 0x56361e441a60 2024-03-18T20:45:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.014+0000 7f6de52a7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e399350 con 0x56361e442380 2024-03-18T20:45:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.014+0000 7f6de52a7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e394c40 con 0x56361e44d220 2024-03-18T20:45:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e441e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e441e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41784/0 (socket says 172.21.15.46:41784) 2024-03-18T20:45:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd2d9d700 1 -- 172.21.15.46:0/3857518954 learned_addr learned my addr 172.21.15.46:0/3857518954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd259c700 1 --2- 172.21.15.46:0/3857518954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e44cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd359e700 1 --2- 172.21.15.46:0/3857518954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e44f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd259c700 1 -- 172.21.15.46:0/3857518954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 msgr2=0x56361e44f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd259c700 1 --2- 172.21.15.46:0/3857518954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e44f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd259c700 1 -- 172.21.15.46:0/3857518954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 msgr2=0x56361e441e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd259c700 1 --2- 172.21.15.46:0/3857518954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e441e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd259c700 1 -- 172.21.15.46:0/3857518954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361e2163b0 con 0x56361e442380 2024-03-18T20:45:52.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd359e700 1 --2- 172.21.15.46:0/3857518954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e44f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:52.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd2d9d700 1 --2- 172.21.15.46:0/3857518954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e441e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dd259c700 1 --2- 172.21.15.46:0/3857518954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e44cce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc000c00 tx=0x7f6dbc009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74762c73a2e9b397 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.015+0000 7f6dc97fa700 1 -- 172.21.15.46:0/3857518954 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00fcb0 con 0x56361e442380 2024-03-18T20:45:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.016+0000 7f6dc97fa700 1 -- 172.21.15.46:0/3857518954 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dbc01d040 con 0x56361e442380 2024-03-18T20:45:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.016+0000 7f6dc97fa700 1 -- 172.21.15.46:0/3857518954 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc005040 con 0x56361e442380 2024-03-18T20:45:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.016+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3857518954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 msgr2=0x56361e44cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.016+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/3857518954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e44cce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc000c00 tx=0x7f6dbc009400 comp rx=0 tx=0).stop 2024-03-18T20:45:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.016+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3857518954 shutdown_connections 2024-03-18T20:45:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.016+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/3857518954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e441e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.016+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/3857518954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e44f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.016+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/3857518954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e44cce0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.016+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3857518954 >> 172.21.15.46:0/3857518954 conn(0x56361e3b8220 msgr2=0x56361e43e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.016+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3857518954 shutdown_connections 2024-03-18T20:45:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.016+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3857518954 wait complete. 2024-03-18T20:45:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6de52a7b80 1 -- start start 2024-03-18T20:45:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6de52a7b80 1 Processor -- start 2024-03-18T20:45:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6de52a7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e44ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6de52a7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e44a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6de52a7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e44a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6de52a7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e330d80 con 0x56361e441a60 2024-03-18T20:45:52.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6de52a7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e330a20 con 0x56361e442380 2024-03-18T20:45:52.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6de52a7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e2186a0 con 0x56361e44d220 2024-03-18T20:45:52.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6de52a7b80 10 client.0 _ll_register_callbacks cb 0x56361e19c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:52.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e44ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e44ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41796/0 (socket says 172.21.15.46:41796) 2024-03-18T20:45:52.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6dd2d9d700 1 -- 172.21.15.46:0/3290252147 learned_addr learned my addr 172.21.15.46:0/3290252147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6dd259c700 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e44a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.017+0000 7f6dd359e700 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e44a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6dd2d9d700 1 -- 172.21.15.46:0/3290252147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 msgr2=0x56361e44a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6dd2d9d700 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e44a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6dd2d9d700 1 -- 172.21.15.46:0/3290252147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 msgr2=0x56361e44a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6dd2d9d700 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e44a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6dd2d9d700 1 -- 172.21.15.46:0/3290252147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc4017070 con 0x56361e441a60 2024-03-18T20:45:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6dd259c700 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e44a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6dd359e700 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e44a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6dd2d9d700 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e44ccc0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400ee50 tx=0x7f6dc40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f521706cf8507de server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6dba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6dba7fc700 1 -- 172.21.15.46:0/3290252147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4022070 con 0x56361e441a60 2024-03-18T20:45:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6dba7fc700 1 -- 172.21.15.46:0/3290252147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f6dc401da30 con 0x56361e441a60 2024-03-18T20:45:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3290252147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56361e435ad0 con 0x56361e441a60 2024-03-18T20:45:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6dba7fc700 1 -- 172.21.15.46:0/3290252147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dc400ee80 con 0x56361e441a60 2024-03-18T20:45:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6de52a7b80 10 client.18951 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.018+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3290252147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f6dd410d380 con 0x56361e441a60 2024-03-18T20:45:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.019+0000 7f6dba7fc700 1 -- 172.21.15.46:0/3290252147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc4033020 con 0x56361e441a60 2024-03-18T20:45:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.019+0000 7f6dba7fc700 1 -- 172.21.15.46:0/3290252147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6dc4026cd0 con 0x56361e441a60 2024-03-18T20:45:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.019+0000 7f6de52a7b80 10 client.18951 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.019+0000 7f6de52a7b80 10 client.18951 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:45:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.019+0000 7f6de52a7b80 20 client.18951 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.019+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56361e45ce00 0x56361e45f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.019+0000 7f6de52a7b80 4 client.18951 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.019+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3290252147 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56361e38d700 con 0x56361e45ce00 2024-03-18T20:45:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.019+0000 7f6dd259c700 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56361e45ce00 0x56361e45f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.019+0000 7f6dba7fc700 10 client.18951 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:45:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.019+0000 7f6dd259c700 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56361e45ce00 0x56361e45f220 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62b050243ffb43e1 server_cookie=98fe39efd124f7da in_seq=0 out_seq=0 2024-03-18T20:45:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6dba7fc700 1 -- 172.21.15.46:0/3290252147 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56361e38d700 con 0x56361e45ce00 2024-03-18T20:45:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6dba7fc700 10 client.18951 handle_command_reply: tid=0 2024-03-18T20:45:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6de52a7b80 1 client.18951 shutdown 2024-03-18T20:45:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3290252147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 msgr2=0x56361e44ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e44ccc0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400ee50 tx=0x7f6dc40047c0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3290252147 shutdown_connections 2024-03-18T20:45:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56361e45ce00 0x56361e45f220 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e44ccc0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e44a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6dba7fc700 0 client.18951 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:45:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/3290252147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e44a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3290252147 >> 172.21.15.46:0/3290252147 conn(0x56361e3b8220 msgr2=0x56361e43dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3290252147 shutdown_connections 2024-03-18T20:45:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.020+0000 7f6de52a7b80 1 -- 172.21.15.46:0/3290252147 wait complete. 2024-03-18T20:45:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.021+0000 7f6de52a7b80 20 client.18951 trim_cache size 0 max 16384 2024-03-18T20:45:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.035+0000 7f6de52a7b80 1 Processor -- start 2024-03-18T20:45:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.035+0000 7f6de52a7b80 1 -- start start 2024-03-18T20:45:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.035+0000 7f6de52a7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e458c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.035+0000 7f6de52a7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e449970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.035+0000 7f6de52a7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e449eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.035+0000 7f6de52a7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e4420d0 con 0x56361e441a60 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.035+0000 7f6de52a7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e444eb0 con 0x56361e442380 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.035+0000 7f6de52a7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e435610 con 0x56361e44d220 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.035+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e458c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.035+0000 7f6dd259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e449970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e458c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41808/0 (socket says 172.21.15.46:41808) 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dd2d9d700 1 -- 172.21.15.46:0/2184587528 learned_addr learned my addr 172.21.15.46:0/2184587528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dd359e700 1 --2- 172.21.15.46:0/2184587528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e449eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dd259c700 1 -- 172.21.15.46:0/2184587528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 msgr2=0x56361e449eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dd259c700 1 --2- 172.21.15.46:0/2184587528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e449eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dd259c700 1 -- 172.21.15.46:0/2184587528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 msgr2=0x56361e458c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dd259c700 1 --2- 172.21.15.46:0/2184587528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e458c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dd259c700 1 -- 172.21.15.46:0/2184587528 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361e1c2b60 con 0x56361e442380 2024-03-18T20:45:52.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dd2d9d700 1 --2- 172.21.15.46:0/2184587528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e458c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dd259c700 1 --2- 172.21.15.46:0/2184587528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e449970 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc005e00 tx=0x7f6dbc00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea82418771330a5a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dc9ffb700 1 -- 172.21.15.46:0/2184587528 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00fcb0 con 0x56361e442380 2024-03-18T20:45:52.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dc9ffb700 1 -- 172.21.15.46:0/2184587528 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dbc00bbb0 con 0x56361e442380 2024-03-18T20:45:52.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.036+0000 7f6dc9ffb700 1 -- 172.21.15.46:0/2184587528 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0055e0 con 0x56361e442380 2024-03-18T20:45:52.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.037+0000 7f6de52a7b80 1 -- 172.21.15.46:0/2184587528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 msgr2=0x56361e449970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.037+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/2184587528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e449970 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc005e00 tx=0x7f6dbc00bf10 comp rx=0 tx=0).stop 2024-03-18T20:45:52.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.037+0000 7f6de52a7b80 1 -- 172.21.15.46:0/2184587528 shutdown_connections 2024-03-18T20:45:52.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.037+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/2184587528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e441a60 0x56361e458c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.037+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/2184587528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e44d220 0x56361e449eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.037+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/2184587528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e442380 0x56361e449970 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.037+0000 7f6de52a7b80 1 -- 172.21.15.46:0/2184587528 >> 172.21.15.46:0/2184587528 conn(0x56361e3b8220 msgr2=0x56361e458540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.037+0000 7f6de52a7b80 1 -- 172.21.15.46:0/2184587528 shutdown_connections 2024-03-18T20:45:52.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.037+0000 7f6de52a7b80 1 -- 172.21.15.46:0/2184587528 wait complete. 2024-03-18T20:45:52.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6de52a7b80 1 -- start start 2024-03-18T20:45:52.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6de52a7b80 1 Processor -- start 2024-03-18T20:45:52.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6de52a7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e441a60 0x56361e451dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6de52a7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e442380 0x56361e452310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6de52a7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e44d220 0x56361e452850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6de52a7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e4afd10 con 0x56361e442380 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6de52a7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e4afe80 con 0x56361e44d220 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6de52a7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e4b0040 con 0x56361e441a60 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6de52a7b80 10 client.0 _ll_register_callbacks cb 0x7f6dc400ea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e441a60 0x56361e451dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6dd259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e442380 0x56361e452310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6dd2d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e441a60 0x56361e451dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:38984/0 (socket says 172.21.15.46:38984) 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6dd2d9d700 1 -- 172.21.15.46:0/958809645 learned_addr learned my addr 172.21.15.46:0/958809645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.038+0000 7f6dd359e700 1 --2- 172.21.15.46:0/958809645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e44d220 0x56361e452850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6dd2d9d700 1 -- 172.21.15.46:0/958809645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e44d220 msgr2=0x56361e452850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6dd2d9d700 1 --2- 172.21.15.46:0/958809645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e44d220 0x56361e452850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6dd2d9d700 1 -- 172.21.15.46:0/958809645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e442380 msgr2=0x56361e452310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6dd2d9d700 1 --2- 172.21.15.46:0/958809645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e442380 0x56361e452310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6dd2d9d700 1 -- 172.21.15.46:0/958809645 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd410d380 con 0x56361e441a60 2024-03-18T20:45:52.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6dd2d9d700 1 --2- 172.21.15.46:0/958809645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e441a60 0x56361e451dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6dc401c480 tx=0x7f6dc400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a38524fc667cc35 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6dba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:45:52.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6dba7fc700 1 -- 172.21.15.46:0/958809645 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4022070 con 0x56361e441a60 2024-03-18T20:45:52.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6dba7fc700 1 -- 172.21.15.46:0/958809645 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f6dc4034080 con 0x56361e441a60 2024-03-18T20:45:52.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6de52a7b80 1 -- 172.21.15.46:0/958809645 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6dbc01d040 con 0x56361e441a60 2024-03-18T20:45:52.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6dba7fc700 1 -- 172.21.15.46:0/958809645 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dc401dd40 con 0x56361e441a60 2024-03-18T20:45:52.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6de52a7b80 10 client.28609 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.039+0000 7f6de52a7b80 1 -- 172.21.15.46:0/958809645 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56361e2163b0 con 0x56361e441a60 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6dba7fc700 1 -- 172.21.15.46:0/958809645 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dc4044080 con 0x56361e441a60 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6dba7fc700 1 -- 172.21.15.46:0/958809645 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6dc40268b0 con 0x56361e441a60 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6de52a7b80 10 client.28609 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6de52a7b80 10 client.28609 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6de52a7b80 20 client.28609 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/958809645 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56361e4b1790 0x56361e4b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6de52a7b80 4 client.28609 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6de52a7b80 1 -- 172.21.15.46:0/958809645 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56361e1e3cf0 con 0x56361e4b1790 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6dd259c700 1 --2- 172.21.15.46:0/958809645 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56361e4b1790 0x56361e4b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6dba7fc700 10 client.28609 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6dd259c700 1 --2- 172.21.15.46:0/958809645 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56361e4b1790 0x56361e4b3bb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca3c5568552847fe server_cookie=af196ee9a11e4ccb in_seq=0 out_seq=0 2024-03-18T20:45:52.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6dba7fc700 1 -- 172.21.15.46:0/958809645 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56361e1e3cf0 con 0x56361e4b1790 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.040+0000 7f6dba7fc700 10 client.28609 handle_command_reply: tid=0 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6de52a7b80 1 client.28609 shutdown 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6de52a7b80 1 -- 172.21.15.46:0/958809645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e441a60 msgr2=0x56361e451dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/958809645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e441a60 0x56361e451dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6dc401c480 tx=0x7f6dc400ee50 comp rx=0 tx=0).stop 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6de52a7b80 1 -- 172.21.15.46:0/958809645 shutdown_connections 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/958809645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56361e44d220 0x56361e452850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/958809645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56361e441a60 0x56361e451dd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/958809645 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56361e4b1790 0x56361e4b3bb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6de52a7b80 1 --2- 172.21.15.46:0/958809645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56361e442380 0x56361e452310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6dba7fc700 0 client.28609 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6de52a7b80 1 -- 172.21.15.46:0/958809645 >> 172.21.15.46:0/958809645 conn(0x56361e3b8220 msgr2=0x56361e458dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6de52a7b80 1 -- 172.21.15.46:0/958809645 shutdown_connections 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6de52a7b80 1 -- 172.21.15.46:0/958809645 wait complete. 2024-03-18T20:45:52.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.041+0000 7f6de52a7b80 20 client.28609 trim_cache size 0 max 16384 2024-03-18T20:45:52.050 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:45:52.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.042+0000 7f6dd9112700 1 -- 172.21.15.46:0/3459664770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6dc00a1290 msgr2=0x7f6dc00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.042+0000 7f6dd9112700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6dc00a1290 0x7f6dc00a3750 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4005fd0 tx=0x7f6dc4009310 comp rx=0 tx=0).stop 2024-03-18T20:45:52.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.042+0000 7f6dd9112700 1 -- 172.21.15.46:0/3459664770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd419bb50 msgr2=0x7f6dd419bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.042+0000 7f6dd9112700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd419bb50 0x7f6dd419bfd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00ef30 tx=0x7f6dcc00c3a0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.042+0000 7f6dd9112700 1 -- 172.21.15.46:0/3459664770 shutdown_connections 2024-03-18T20:45:52.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.042+0000 7f6dd9112700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6dc00a1290 0x7f6dc00a3750 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.042+0000 7f6dd9112700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6dd40ffe70 0x7f6dd4197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.042+0000 7f6dd9112700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6dd419bb50 0x7f6dd419bfd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.042+0000 7f6dd9112700 1 --2- 172.21.15.46:0/3459664770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dd4104f90 0x7f6dd41977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.042+0000 7f6dd9112700 1 -- 172.21.15.46:0/3459664770 >> 172.21.15.46:0/3459664770 conn(0x7f6dd4076d80 msgr2=0x7f6dd4103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.042+0000 7f6dd9112700 1 -- 172.21.15.46:0/3459664770 shutdown_connections 2024-03-18T20:45:52.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.042+0000 7f6dd9112700 1 -- 172.21.15.46:0/3459664770 wait complete. 2024-03-18T20:45:52.059 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:45:52.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:45:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.205+0000 7f298b84f700 1 -- 172.21.15.46:0/4141248216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2984101920 msgr2=0x7f2984105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.205+0000 7f298b84f700 1 --2- 172.21.15.46:0/4141248216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2984101920 0x7f2984105960 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2974009990 tx=0x7f2974009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.205+0000 7f298b84f700 1 -- 172.21.15.46:0/4141248216 shutdown_connections 2024-03-18T20:45:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.205+0000 7f298b84f700 1 --2- 172.21.15.46:0/4141248216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984106090 0x7f298410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.205+0000 7f298b84f700 1 --2- 172.21.15.46:0/4141248216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2984100f70 0x7f2984101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.205+0000 7f298b84f700 1 --2- 172.21.15.46:0/4141248216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2984101920 0x7f2984105960 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.205+0000 7f298b84f700 1 -- 172.21.15.46:0/4141248216 >> 172.21.15.46:0/4141248216 conn(0x7f29840fc7f0 msgr2=0x7f29840fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.205+0000 7f298b84f700 1 -- 172.21.15.46:0/4141248216 shutdown_connections 2024-03-18T20:45:52.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.205+0000 7f298b84f700 1 -- 172.21.15.46:0/4141248216 wait complete. 2024-03-18T20:45:52.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.205+0000 7f298b84f700 1 Processor -- start 2024-03-18T20:45:52.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.205+0000 7f298b84f700 1 -- start start 2024-03-18T20:45:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f298b84f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984100f70 0x7f29840730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f298b84f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2984106090 0x7f29840735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f298b84f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2984073b30 0x7f29841a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f298b84f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29841a5960 con 0x7f2984106090 2024-03-18T20:45:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f298b84f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29841a5ad0 con 0x7f2984100f70 2024-03-18T20:45:52.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f298b84f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29841a5c90 con 0x7f2984073b30 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f29895eb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984100f70 0x7f29840730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f2989dec700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2984073b30 0x7f29841a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f2988dea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2984106090 0x7f29840735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f29895eb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984100f70 0x7f29840730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56544/0 (socket says 172.21.15.46:56544) 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f29895eb700 1 -- 172.21.15.46:0/539124273 learned_addr learned my addr 172.21.15.46:0/539124273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f29895eb700 1 -- 172.21.15.46:0/539124273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2984073b30 msgr2=0x7f29841a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f29895eb700 1 --2- 172.21.15.46:0/539124273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2984073b30 0x7f29841a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f29895eb700 1 -- 172.21.15.46:0/539124273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2984106090 msgr2=0x7f29840735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f29895eb700 1 --2- 172.21.15.46:0/539124273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2984106090 0x7f29840735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.206+0000 7f29895eb700 1 -- 172.21.15.46:0/539124273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29740095f0 con 0x7f2984100f70 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.207+0000 7f29895eb700 1 --2- 172.21.15.46:0/539124273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984100f70 0x7f29840730b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f298000d950 tx=0x7f298000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.207+0000 7f297a7fc700 1 -- 172.21.15.46:0/539124273 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298000f890 con 0x7f2984100f70 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.207+0000 7f297a7fc700 1 -- 172.21.15.46:0/539124273 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2980010460 con 0x7f2984100f70 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.207+0000 7f297a7fc700 1 -- 172.21.15.46:0/539124273 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298000e610 con 0x7f2984100f70 2024-03-18T20:45:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.207+0000 7f298b84f700 1 -- 172.21.15.46:0/539124273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29841a5f70 con 0x7f2984100f70 2024-03-18T20:45:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.207+0000 7f298b84f700 1 -- 172.21.15.46:0/539124273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29841a6340 con 0x7f2984100f70 2024-03-18T20:45:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.210+0000 7f297a7fc700 1 -- 172.21.15.46:0/539124273 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f29800108d0 con 0x7f2984100f70 2024-03-18T20:45:52.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.210+0000 7f297a7fc700 1 --2- 172.21.15.46:0/539124273 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f29700a1310 0x7f29700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.211+0000 7f297a7fc700 1 -- 172.21.15.46:0/539124273 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f29800cc990 con 0x7f2984100f70 2024-03-18T20:45:52.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.211+0000 7f2988dea700 1 --2- 172.21.15.46:0/539124273 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f29700a1310 0x7f29700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.211+0000 7f29979e4b80 1 Processor -- start 2024-03-18T20:45:52.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.211+0000 7f29979e4b80 1 -- start start 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f29979e4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f29979e4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1c5380 0x55dbbc1cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f29979e4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1d0220 0x55dbbc1d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f29979e4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbc11e990 con 0x55dbbc1c4a60 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f29979e4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbc11c350 con 0x55dbbc1d0220 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f29979e4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbc117c40 con 0x55dbbc1c5380 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f29895eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f2989dec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1d0220 0x55dbbc1d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f29895eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41836/0 (socket says 172.21.15.46:41836) 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f29895eb700 1 -- 172.21.15.46:0/4277849898 learned_addr learned my addr 172.21.15.46:0/4277849898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f2988dea700 1 --2- 172.21.15.46:0/4277849898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1c5380 0x55dbbc1cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f2989dec700 1 -- 172.21.15.46:0/4277849898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1c5380 msgr2=0x55dbbc1cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f2988dea700 1 --2- 172.21.15.46:0/539124273 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f29700a1310 0x7f29700a37d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f297400b2e0 tx=0x7f2974005a90 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f2989dec700 1 --2- 172.21.15.46:0/4277849898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1c5380 0x55dbbc1cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f2989dec700 1 -- 172.21.15.46:0/4277849898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 msgr2=0x55dbbc1c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f2989dec700 1 --2- 172.21.15.46:0/4277849898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1c4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.212+0000 7f2989dec700 1 -- 172.21.15.46:0/4277849898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbbbf993b0 con 0x55dbbc1d0220 2024-03-18T20:45:52.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.213+0000 7f2989dec700 1 --2- 172.21.15.46:0/4277849898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1d0220 0x55dbbc1d2610 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f297c011580 tx=0x7f297c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3a194c09bd64697 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.213+0000 7f296f7fe700 1 -- 172.21.15.46:0/4277849898 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c00e040 con 0x55dbbc1d0220 2024-03-18T20:45:52.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.213+0000 7f296f7fe700 1 -- 172.21.15.46:0/4277849898 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f297c00f040 con 0x55dbbc1d0220 2024-03-18T20:45:52.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.213+0000 7f296f7fe700 1 -- 172.21.15.46:0/4277849898 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c0044f0 con 0x55dbbc1d0220 2024-03-18T20:45:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.214+0000 7f29979e4b80 1 -- 172.21.15.46:0/4277849898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1d0220 msgr2=0x55dbbc1d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.214+0000 7f29979e4b80 1 --2- 172.21.15.46:0/4277849898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1d0220 0x55dbbc1d2610 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f297c011580 tx=0x7f297c009920 comp rx=0 tx=0).stop 2024-03-18T20:45:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.214+0000 7f29979e4b80 1 -- 172.21.15.46:0/4277849898 shutdown_connections 2024-03-18T20:45:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.214+0000 7f29979e4b80 1 --2- 172.21.15.46:0/4277849898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.214+0000 7f29979e4b80 1 --2- 172.21.15.46:0/4277849898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1d0220 0x55dbbc1d2610 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.214+0000 7f29979e4b80 1 --2- 172.21.15.46:0/4277849898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1c5380 0x55dbbc1cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.214+0000 7f29979e4b80 1 -- 172.21.15.46:0/4277849898 >> 172.21.15.46:0/4277849898 conn(0x55dbbc13b220 msgr2=0x55dbbc1c1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.214+0000 7f29979e4b80 1 -- 172.21.15.46:0/4277849898 shutdown_connections 2024-03-18T20:45:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.214+0000 7f29979e4b80 1 -- 172.21.15.46:0/4277849898 wait complete. 2024-03-18T20:45:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.214+0000 7f29979e4b80 1 -- start start 2024-03-18T20:45:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.214+0000 7f29979e4b80 1 Processor -- start 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f29979e4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f29979e4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 0x55dbbc1ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f29979e4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1d0220 0x55dbbc1cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f29979e4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbc0b3d80 con 0x55dbbc1c5380 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f29979e4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbc0b3a20 con 0x55dbbc1c4a60 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f29979e4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbbf9b6a0 con 0x55dbbc1d0220 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f29979e4b80 10 client.0 _ll_register_callbacks cb 0x55dbbbf1f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f2988dea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 0x55dbbc1ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f29895eb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f2988dea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 0x55dbbc1ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41840/0 (socket says 172.21.15.46:41840) 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f29895eb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56556/0 (socket says 172.21.15.46:56556) 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f2988dea700 1 -- 172.21.15.46:0/2605894241 learned_addr learned my addr 172.21.15.46:0/2605894241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f2989dec700 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1d0220 0x55dbbc1cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f2988dea700 1 -- 172.21.15.46:0/2605894241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1d0220 msgr2=0x55dbbc1cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f2988dea700 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1d0220 0x55dbbc1cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f2988dea700 1 -- 172.21.15.46:0/2605894241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c4a60 msgr2=0x55dbbc1cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f2988dea700 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f2988dea700 1 -- 172.21.15.46:0/2605894241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c011230 con 0x55dbbc1c5380 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f29895eb700 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f2988dea700 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 0x55dbbc1ccfa0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2974015000 tx=0x7f2974009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed4225596e4608c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.215+0000 7f296cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.216+0000 7f296cff9700 1 -- 172.21.15.46:0/2605894241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974004df0 con 0x55dbbc1c5380 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.216+0000 7f296cff9700 1 -- 172.21.15.46:0/2605894241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f297400f460 con 0x55dbbc1c5380 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.216+0000 7f29979e4b80 1 -- 172.21.15.46:0/2605894241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbbc1b8ad0 con 0x55dbbc1c5380 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.216+0000 7f296cff9700 1 -- 172.21.15.46:0/2605894241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f297400f620 con 0x55dbbc1c5380 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.216+0000 7f29979e4b80 10 client.18969 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.216+0000 7f29979e4b80 1 -- 172.21.15.46:0/2605894241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55dbbbf45b60 con 0x55dbbc1c5380 2024-03-18T20:45:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.216+0000 7f296cff9700 1 -- 172.21.15.46:0/2605894241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f297403c020 con 0x55dbbc1c5380 2024-03-18T20:45:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.216+0000 7f296cff9700 1 -- 172.21.15.46:0/2605894241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2974024a10 con 0x55dbbc1c5380 2024-03-18T20:45:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.216+0000 7f29979e4b80 10 client.18969 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.216+0000 7f29979e4b80 10 client.18969 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:45:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.217+0000 7f29979e4b80 20 client.18969 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.217+0000 7f29979e4b80 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55dbbc1f0df0 0x55dbbc1f3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.217+0000 7f29979e4b80 4 client.18969 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.217+0000 7f29979e4b80 1 -- 172.21.15.46:0/2605894241 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbbc110700 con 0x55dbbc1f0df0 2024-03-18T20:45:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.217+0000 7f29895eb700 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55dbbc1f0df0 0x55dbbc1f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.217+0000 7f296cff9700 10 client.18969 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.217+0000 7f29895eb700 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55dbbc1f0df0 0x55dbbc1f3210 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28a0906ce852337b server_cookie=dc336843f53a6204 in_seq=0 out_seq=0 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f296cff9700 1 -- 172.21.15.46:0/2605894241 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dbbc110700 con 0x55dbbc1f0df0 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f296cff9700 10 client.18969 handle_command_reply: tid=0 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f29979e4b80 1 client.18969 shutdown 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f29979e4b80 1 -- 172.21.15.46:0/2605894241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 msgr2=0x55dbbc1ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f29979e4b80 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 0x55dbbc1ccfa0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2974015000 tx=0x7f2974009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f29979e4b80 1 -- 172.21.15.46:0/2605894241 shutdown_connections 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f29979e4b80 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1d0220 0x55dbbc1cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f29979e4b80 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f29979e4b80 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55dbbc1f0df0 0x55dbbc1f3210 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f29979e4b80 1 --2- 172.21.15.46:0/2605894241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 0x55dbbc1ccfa0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f296cff9700 0 client.18969 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f29979e4b80 1 -- 172.21.15.46:0/2605894241 >> 172.21.15.46:0/2605894241 conn(0x55dbbc13b220 msgr2=0x55dbbc1c0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f29979e4b80 1 -- 172.21.15.46:0/2605894241 shutdown_connections 2024-03-18T20:45:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f29979e4b80 1 -- 172.21.15.46:0/2605894241 wait complete. 2024-03-18T20:45:52.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.218+0000 7f29979e4b80 20 client.18969 trim_cache size 0 max 16384 2024-03-18T20:45:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29979e4b80 1 Processor -- start 2024-03-18T20:45:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29979e4b80 1 -- start start 2024-03-18T20:45:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29979e4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1e7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29979e4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c5380 0x55dbbc1c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29979e4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1d0220 0x55dbbc1c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29979e4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbc1e3bf0 con 0x55dbbc1c4a60 2024-03-18T20:45:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29979e4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbc1bd5e0 con 0x55dbbc1c5380 2024-03-18T20:45:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29979e4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbc1c2230 con 0x55dbbc1d0220 2024-03-18T20:45:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29895eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1e7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f2988dea700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c5380 0x55dbbc1c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29895eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1e7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41848/0 (socket says 172.21.15.46:41848) 2024-03-18T20:45:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29895eb700 1 -- 172.21.15.46:0/521638238 learned_addr learned my addr 172.21.15.46:0/521638238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f2989dec700 1 --2- 172.21.15.46:0/521638238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1d0220 0x55dbbc1c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29895eb700 1 -- 172.21.15.46:0/521638238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1d0220 msgr2=0x55dbbc1c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29895eb700 1 --2- 172.21.15.46:0/521638238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1d0220 0x55dbbc1c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29895eb700 1 -- 172.21.15.46:0/521638238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c5380 msgr2=0x55dbbc1c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29895eb700 1 --2- 172.21.15.46:0/521638238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c5380 0x55dbbc1c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f29895eb700 1 -- 172.21.15.46:0/521638238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbbc0ddea0 con 0x55dbbc1c4a60 2024-03-18T20:45:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.234+0000 7f2988dea700 1 --2- 172.21.15.46:0/521638238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c5380 0x55dbbc1c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.235+0000 7f29895eb700 1 --2- 172.21.15.46:0/521638238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1e7e60 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f2980010fd0 tx=0x7f2980000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe7ce7acfad1c6e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.235+0000 7f296ffff700 1 -- 172.21.15.46:0/521638238 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980097e20 con 0x55dbbc1c4a60 2024-03-18T20:45:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.235+0000 7f296ffff700 1 -- 172.21.15.46:0/521638238 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2980014050 con 0x55dbbc1c4a60 2024-03-18T20:45:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.235+0000 7f296ffff700 1 -- 172.21.15.46:0/521638238 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800991d0 con 0x55dbbc1c4a60 2024-03-18T20:45:52.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.236+0000 7f29979e4b80 1 -- 172.21.15.46:0/521638238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 msgr2=0x55dbbc1e7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.236+0000 7f29979e4b80 1 --2- 172.21.15.46:0/521638238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1e7e60 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f2980010fd0 tx=0x7f2980000bc0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.236+0000 7f29979e4b80 1 -- 172.21.15.46:0/521638238 shutdown_connections 2024-03-18T20:45:52.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.236+0000 7f29979e4b80 1 --2- 172.21.15.46:0/521638238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1e7e60 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.236+0000 7f29979e4b80 1 --2- 172.21.15.46:0/521638238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1d0220 0x55dbbc1c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.236+0000 7f29979e4b80 1 --2- 172.21.15.46:0/521638238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1c5380 0x55dbbc1c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.236+0000 7f29979e4b80 1 -- 172.21.15.46:0/521638238 >> 172.21.15.46:0/521638238 conn(0x55dbbc13b220 msgr2=0x55dbbc1bf750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.236+0000 7f29979e4b80 1 -- 172.21.15.46:0/521638238 shutdown_connections 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.236+0000 7f29979e4b80 1 -- 172.21.15.46:0/521638238 wait complete. 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.236+0000 7f29979e4b80 1 -- start start 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.236+0000 7f29979e4b80 1 Processor -- start 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29979e4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1e6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29979e4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 0x55dbbc1e6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29979e4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1d0220 0x55dbbc243c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29979e4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbc244160 con 0x55dbbc1c5380 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29979e4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbc2442a0 con 0x55dbbc1d0220 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29979e4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbc2443e0 con 0x55dbbc1c4a60 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29979e4b80 10 client.0 _ll_register_callbacks cb 0x7f297400ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:52.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f2988dea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 0x55dbbc1e6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29895eb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1e6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f2988dea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 0x55dbbc1e6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41852/0 (socket says 172.21.15.46:41852) 2024-03-18T20:45:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f2988dea700 1 -- 172.21.15.46:0/2967884212 learned_addr learned my addr 172.21.15.46:0/2967884212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f2989dec700 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1d0220 0x55dbbc243c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29895eb700 1 -- 172.21.15.46:0/2967884212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1d0220 msgr2=0x55dbbc243c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29895eb700 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1d0220 0x55dbbc243c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29895eb700 1 -- 172.21.15.46:0/2967884212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 msgr2=0x55dbbc1e6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29895eb700 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 0x55dbbc1e6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f29895eb700 1 -- 172.21.15.46:0/2967884212 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbbbf45b60 con 0x55dbbc1c4a60 2024-03-18T20:45:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.237+0000 7f2989dec700 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1d0220 0x55dbbc243c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.238+0000 7f29895eb700 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1e6030 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2980095f80 tx=0x7f2980097c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e11d2cdc80678649 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.238+0000 7f296cff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:45:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.239+0000 7f296cff9700 1 -- 172.21.15.46:0/2967884212 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800962d0 con 0x55dbbc1c4a60 2024-03-18T20:45:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.239+0000 7f296cff9700 1 -- 172.21.15.46:0/2967884212 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f2980099ce0 con 0x55dbbc1c4a60 2024-03-18T20:45:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.239+0000 7f29979e4b80 1 -- 172.21.15.46:0/2967884212 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f297c00f040 con 0x55dbbc1c4a60 2024-03-18T20:45:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.239+0000 7f296cff9700 1 -- 172.21.15.46:0/2967884212 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2980099ea0 con 0x55dbbc1c4a60 2024-03-18T20:45:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.240+0000 7f29979e4b80 10 client.28627 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.240+0000 7f29979e4b80 1 -- 172.21.15.46:0/2967884212 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f29841a6340 con 0x55dbbc1c4a60 2024-03-18T20:45:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.240+0000 7f296cff9700 1 -- 172.21.15.46:0/2967884212 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f297c00f040 con 0x55dbbc1c4a60 2024-03-18T20:45:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.240+0000 7f296cff9700 1 -- 172.21.15.46:0/2967884212 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2980078b60 con 0x55dbbc1c4a60 2024-03-18T20:45:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.240+0000 7f29979e4b80 10 client.28627 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.240+0000 7f29979e4b80 10 client.28627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:45:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.240+0000 7f29979e4b80 20 client.28627 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:52.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.240+0000 7f29979e4b80 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55dbbc245640 0x55dbbc247a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.240+0000 7f29979e4b80 4 client.28627 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.240+0000 7f29979e4b80 1 -- 172.21.15.46:0/2967884212 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dbbbf66cf0 con 0x55dbbc245640 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.240+0000 7f2988dea700 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55dbbc245640 0x55dbbc247a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f296cff9700 10 client.28627 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f2988dea700 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55dbbc245640 0x55dbbc247a30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7abddd35a31659c2 server_cookie=435faf25704a5ec in_seq=0 out_seq=0 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f296cff9700 1 -- 172.21.15.46:0/2967884212 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dbbbf66cf0 con 0x55dbbc245640 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f296cff9700 10 client.28627 handle_command_reply: tid=0 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f29979e4b80 1 client.28627 shutdown 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f29979e4b80 1 -- 172.21.15.46:0/2967884212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1c4a60 msgr2=0x55dbbc1e6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f29979e4b80 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1e6030 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2980095f80 tx=0x7f2980097c30 comp rx=0 tx=0).stop 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f29979e4b80 1 -- 172.21.15.46:0/2967884212 shutdown_connections 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f29979e4b80 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbbc1d0220 0x55dbbc243c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f29979e4b80 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dbbc1c4a60 0x55dbbc1e6030 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f29979e4b80 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55dbbc245640 0x55dbbc247a30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f29979e4b80 1 --2- 172.21.15.46:0/2967884212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dbbc1c5380 0x55dbbc1e6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f296cff9700 0 client.28627 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:45:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f29979e4b80 1 -- 172.21.15.46:0/2967884212 >> 172.21.15.46:0/2967884212 conn(0x55dbbc13b220 msgr2=0x55dbbc1c82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f29979e4b80 1 -- 172.21.15.46:0/2967884212 shutdown_connections 2024-03-18T20:45:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f29979e4b80 1 -- 172.21.15.46:0/2967884212 wait complete. 2024-03-18T20:45:52.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.241+0000 7f29979e4b80 20 client.28627 trim_cache size 0 max 16384 2024-03-18T20:45:52.248 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:45:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.242+0000 7f298b84f700 1 -- 172.21.15.46:0/539124273 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f29700a1310 msgr2=0x7f29700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.242+0000 7f298b84f700 1 --2- 172.21.15.46:0/539124273 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f29700a1310 0x7f29700a37d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f297400b2e0 tx=0x7f2974005a90 comp rx=0 tx=0).stop 2024-03-18T20:45:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.242+0000 7f298b84f700 1 -- 172.21.15.46:0/539124273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984100f70 msgr2=0x7f29840730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.242+0000 7f298b84f700 1 --2- 172.21.15.46:0/539124273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984100f70 0x7f29840730b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f298000d950 tx=0x7f298000dc60 comp rx=0 tx=0).stop 2024-03-18T20:45:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.242+0000 7f298b84f700 1 -- 172.21.15.46:0/539124273 shutdown_connections 2024-03-18T20:45:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.242+0000 7f298b84f700 1 --2- 172.21.15.46:0/539124273 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f29700a1310 0x7f29700a37d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.242+0000 7f298b84f700 1 --2- 172.21.15.46:0/539124273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2984073b30 0x7f29841a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.242+0000 7f298b84f700 1 --2- 172.21.15.46:0/539124273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984100f70 0x7f29840730b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.242+0000 7f298b84f700 1 --2- 172.21.15.46:0/539124273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2984106090 0x7f29840735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.242+0000 7f298b84f700 1 -- 172.21.15.46:0/539124273 >> 172.21.15.46:0/539124273 conn(0x7f29840fc7f0 msgr2=0x7f29840fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.242+0000 7f298b84f700 1 -- 172.21.15.46:0/539124273 shutdown_connections 2024-03-18T20:45:52.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.242+0000 7f298b84f700 1 -- 172.21.15.46:0/539124273 wait complete. 2024-03-18T20:45:52.259 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:45:52.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:45:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.409+0000 7f64004a4700 1 -- 172.21.15.46:0/4066086855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8105ce0 msgr2=0x7f63f81060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.409+0000 7f64004a4700 1 --2- 172.21.15.46:0/4066086855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8105ce0 0x7f63f81060c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009990 tx=0x7f63f4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.410+0000 7f64004a4700 1 -- 172.21.15.46:0/4066086855 shutdown_connections 2024-03-18T20:45:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.410+0000 7f64004a4700 1 --2- 172.21.15.46:0/4066086855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8105ce0 0x7f63f81060c0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.410+0000 7f64004a4700 1 --2- 172.21.15.46:0/4066086855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f80687f0 0x7f63f810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.410+0000 7f64004a4700 1 --2- 172.21.15.46:0/4066086855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63f8067e30 0x7f63f80682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.410+0000 7f64004a4700 1 -- 172.21.15.46:0/4066086855 >> 172.21.15.46:0/4066086855 conn(0x7f63f8076d40 msgr2=0x7f63f8077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.410+0000 7f64004a4700 1 -- 172.21.15.46:0/4066086855 shutdown_connections 2024-03-18T20:45:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.410+0000 7f64004a4700 1 -- 172.21.15.46:0/4066086855 wait complete. 2024-03-18T20:45:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.410+0000 7f64004a4700 1 Processor -- start 2024-03-18T20:45:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.410+0000 7f64004a4700 1 -- start start 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f64004a4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f8067e30 0x7f63f8197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f64004a4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63f80687f0 0x7f63f81977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f64004a4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8105ce0 0x7f63f819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f64004a4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f819c130 con 0x7f63f8105ce0 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f64004a4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f819c2a0 con 0x7f63f80687f0 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f64004a4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f819c460 con 0x7f63f8067e30 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fe240700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f8067e30 0x7f63f8197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fda3f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63f80687f0 0x7f63f81977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fe240700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f8067e30 0x7f63f8197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:39078/0 (socket says 172.21.15.46:39078) 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fe240700 1 -- 172.21.15.46:0/167504771 learned_addr learned my addr 172.21.15.46:0/167504771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fea41700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8105ce0 0x7f63f819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fe240700 1 -- 172.21.15.46:0/167504771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63f80687f0 msgr2=0x7f63f81977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fe240700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63f80687f0 0x7f63f81977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fe240700 1 -- 172.21.15.46:0/167504771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8105ce0 msgr2=0x7f63f819baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fe240700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8105ce0 0x7f63f819baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fe240700 1 -- 172.21.15.46:0/167504771 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e8009520 con 0x7f63f8067e30 2024-03-18T20:45:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fea41700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8105ce0 0x7f63f819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fda3f700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63f80687f0 0x7f63f81977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.411+0000 7f63fe240700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f8067e30 0x7f63f8197260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f63f4005910 tx=0x7f63f4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.412+0000 7f63ef7fe700 1 -- 172.21.15.46:0/167504771 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f401d070 con 0x7f63f8067e30 2024-03-18T20:45:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.412+0000 7f63ef7fe700 1 -- 172.21.15.46:0/167504771 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f4022470 con 0x7f63f8067e30 2024-03-18T20:45:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.412+0000 7f64004a4700 1 -- 172.21.15.46:0/167504771 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f40095f0 con 0x7f63f8067e30 2024-03-18T20:45:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.412+0000 7f63ef7fe700 1 -- 172.21.15.46:0/167504771 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4017650 con 0x7f63f8067e30 2024-03-18T20:45:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.412+0000 7f64004a4700 1 -- 172.21.15.46:0/167504771 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f81a3da0 con 0x7f63f8067e30 2024-03-18T20:45:52.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.414+0000 7f63ef7fe700 1 -- 172.21.15.46:0/167504771 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f63f40225e0 con 0x7f63f8067e30 2024-03-18T20:45:52.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.415+0000 7f63ef7fe700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f63e40a1310 0x7f63e40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.415+0000 7f63ef7fe700 1 -- 172.21.15.46:0/167504771 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f63f40cdf50 con 0x7f63f8067e30 2024-03-18T20:45:52.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.415+0000 7f63fda3f700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f63e40a1310 0x7f63e40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.416+0000 7f63fda3f700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f63e40a1310 0x7f63e40a37d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009fd0 tx=0x7f63e8009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.416+0000 7f640c639b80 1 Processor -- start 2024-03-18T20:45:52.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.416+0000 7f640c639b80 1 -- start start 2024-03-18T20:45:52.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f640c639b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 0x55de3d814e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f640c639b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d815380 0x55de3d8240f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f640c639b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d826a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f640c639b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3d76e990 con 0x55de3d815380 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f640c639b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3d76c350 con 0x55de3d814a60 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f640c639b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3d767c40 con 0x55de3d824630 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f63fe240700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 0x55de3d814e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f63fe240700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 0x55de3d814e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56600/0 (socket says 172.21.15.46:56600) 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f63fe240700 1 -- 172.21.15.46:0/2926507913 learned_addr learned my addr 172.21.15.46:0/2926507913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f63fda3f700 1 --2- 172.21.15.46:0/2926507913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d815380 0x55de3d8240f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f63fea41700 1 --2- 172.21.15.46:0/2926507913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d826a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f63fe240700 1 -- 172.21.15.46:0/2926507913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 msgr2=0x55de3d826a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f63fe240700 1 --2- 172.21.15.46:0/2926507913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d826a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f63fe240700 1 -- 172.21.15.46:0/2926507913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d815380 msgr2=0x55de3d8240f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f63fe240700 1 --2- 172.21.15.46:0/2926507913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d815380 0x55de3d8240f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f63fe240700 1 -- 172.21.15.46:0/2926507913 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de3d5e93b0 con 0x55de3d814a60 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.417+0000 7f63fe240700 1 --2- 172.21.15.46:0/2926507913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 0x55de3d814e40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f63f40976f0 tx=0x7f63f4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=715285befc4d22b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.418+0000 7f63ecda8700 1 -- 172.21.15.46:0/2926507913 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f40b2070 con 0x55de3d814a60 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.418+0000 7f63ecda8700 1 -- 172.21.15.46:0/2926507913 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f409ab50 con 0x55de3d814a60 2024-03-18T20:45:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.418+0000 7f63ecda8700 1 -- 172.21.15.46:0/2926507913 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f40a10c0 con 0x55de3d814a60 2024-03-18T20:45:52.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 -- 172.21.15.46:0/2926507913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 msgr2=0x55de3d814e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 --2- 172.21.15.46:0/2926507913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 0x55de3d814e40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f63f40976f0 tx=0x7f63f4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 -- 172.21.15.46:0/2926507913 shutdown_connections 2024-03-18T20:45:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 --2- 172.21.15.46:0/2926507913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d826a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 --2- 172.21.15.46:0/2926507913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 0x55de3d814e40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 --2- 172.21.15.46:0/2926507913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d815380 0x55de3d8240f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 -- 172.21.15.46:0/2926507913 >> 172.21.15.46:0/2926507913 conn(0x55de3d78b220 msgr2=0x55de3d811020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 -- 172.21.15.46:0/2926507913 shutdown_connections 2024-03-18T20:45:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 -- 172.21.15.46:0/2926507913 wait complete. 2024-03-18T20:45:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 -- start start 2024-03-18T20:45:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 Processor -- start 2024-03-18T20:45:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 0x55de3d824c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d815380 0x55de3d825140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d824630 0x55de3d82fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3d703d80 con 0x55de3d814a60 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3d703a20 con 0x55de3d824630 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.419+0000 7f640c639b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3d5eb6a0 con 0x55de3d815380 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f640c639b80 10 client.0 _ll_register_callbacks cb 0x55de3d56f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63fe240700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 0x55de3d824c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63fea41700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d824630 0x55de3d82fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63fe240700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 0x55de3d824c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41882/0 (socket says 172.21.15.46:41882) 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63fe240700 1 -- 172.21.15.46:0/17885099 learned_addr learned my addr 172.21.15.46:0/17885099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63fda3f700 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d815380 0x55de3d825140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63fea41700 1 -- 172.21.15.46:0/17885099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d815380 msgr2=0x55de3d825140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63fea41700 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d815380 0x55de3d825140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63fea41700 1 -- 172.21.15.46:0/17885099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 msgr2=0x55de3d824c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63fea41700 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 0x55de3d824c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63fea41700 1 -- 172.21.15.46:0/17885099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f81a3da0 con 0x55de3d824630 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63fe240700 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 0x55de3d824c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63fea41700 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d824630 0x55de3d82fa90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f63f0007a70 tx=0x7f63f0007aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0120fa6233e405f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63e1ffb700 1 -- 172.21.15.46:0/17885099 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f000e030 con 0x55de3d824630 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f63e1ffb700 1 -- 172.21.15.46:0/17885099 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f63f0003c00 con 0x55de3d824630 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.420+0000 7f640c639b80 1 -- 172.21.15.46:0/17885099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de3d808ad0 con 0x55de3d824630 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.421+0000 7f63e1ffb700 1 -- 172.21.15.46:0/17885099 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63f0003dc0 con 0x55de3d824630 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.421+0000 7f640c639b80 10 client.28622 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.421+0000 7f640c639b80 1 -- 172.21.15.46:0/17885099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55de3d595b60 con 0x55de3d824630 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.421+0000 7f63e1ffb700 1 -- 172.21.15.46:0/17885099 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de3d808ad0 con 0x55de3d824630 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.421+0000 7f63e1ffb700 1 -- 172.21.15.46:0/17885099 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f63f0004410 con 0x55de3d824630 2024-03-18T20:45:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.421+0000 7f640c639b80 10 client.28622 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.421+0000 7f640c639b80 10 client.28622 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.421+0000 7f640c639b80 20 client.28622 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.421+0000 7f640c639b80 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55de3d833ef0 0x55de3d836310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.421+0000 7f640c639b80 4 client.28622 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.421+0000 7f640c639b80 1 -- 172.21.15.46:0/17885099 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de3d760700 con 0x55de3d833ef0 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.421+0000 7f63fe240700 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55de3d833ef0 0x55de3d836310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.422+0000 7f63e1ffb700 10 client.28622 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.422+0000 7f63fe240700 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55de3d833ef0 0x55de3d836310 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33b864fccefb607c server_cookie=18f59eb3673615a4 in_seq=0 out_seq=0 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.422+0000 7f63e1ffb700 1 -- 172.21.15.46:0/17885099 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55de3d760700 con 0x55de3d833ef0 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.422+0000 7f63e1ffb700 10 client.28622 handle_command_reply: tid=0 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.422+0000 7f640c639b80 1 client.28622 shutdown 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.423+0000 7f640c639b80 1 -- 172.21.15.46:0/17885099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d824630 msgr2=0x55de3d82fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.423+0000 7f640c639b80 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d824630 0x55de3d82fa90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f63f0007a70 tx=0x7f63f0007aa0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.423+0000 7f640c639b80 1 -- 172.21.15.46:0/17885099 shutdown_connections 2024-03-18T20:45:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.423+0000 7f640c639b80 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55de3d833ef0 0x55de3d836310 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.423+0000 7f640c639b80 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 0x55de3d824c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.423+0000 7f640c639b80 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d824630 0x55de3d82fa90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.423+0000 7f63e1ffb700 0 client.28622 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:45:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.423+0000 7f640c639b80 1 --2- 172.21.15.46:0/17885099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d815380 0x55de3d825140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.423+0000 7f640c639b80 1 -- 172.21.15.46:0/17885099 >> 172.21.15.46:0/17885099 conn(0x55de3d78b220 msgr2=0x55de3d810c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.423+0000 7f640c639b80 1 -- 172.21.15.46:0/17885099 shutdown_connections 2024-03-18T20:45:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.423+0000 7f640c639b80 1 -- 172.21.15.46:0/17885099 wait complete. 2024-03-18T20:45:52.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.423+0000 7f640c639b80 20 client.28622 trim_cache size 0 max 16384 2024-03-18T20:45:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f640c639b80 1 Processor -- start 2024-03-18T20:45:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f640c639b80 1 -- start start 2024-03-18T20:45:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f640c639b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 0x55de3d8299f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f640c639b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d815380 0x55de3d81e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f640c639b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d81ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f640c639b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3d831650 con 0x55de3d814a60 2024-03-18T20:45:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f640c639b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3d830a40 con 0x55de3d815380 2024-03-18T20:45:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f640c639b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3d8303a0 con 0x55de3d824630 2024-03-18T20:45:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f63fe240700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 0x55de3d8299f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f63fda3f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d815380 0x55de3d81e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f63fda3f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d815380 0x55de3d81e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56628/0 (socket says 172.21.15.46:56628) 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f63fda3f700 1 -- 172.21.15.46:0/888866153 learned_addr learned my addr 172.21.15.46:0/888866153 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f63fe240700 1 --2- 172.21.15.46:0/888866153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 0x55de3d8299f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41886/0 (socket says 172.21.15.46:41886) 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.437+0000 7f63fea41700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d81ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.438+0000 7f63fda3f700 1 -- 172.21.15.46:0/888866153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 msgr2=0x55de3d81ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.438+0000 7f63fda3f700 1 --2- 172.21.15.46:0/888866153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d81ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.438+0000 7f63fda3f700 1 -- 172.21.15.46:0/888866153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 msgr2=0x55de3d8299f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.438+0000 7f63fda3f700 1 --2- 172.21.15.46:0/888866153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 0x55de3d8299f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.438+0000 7f63fda3f700 1 -- 172.21.15.46:0/888866153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de3d72dea0 con 0x55de3d815380 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.438+0000 7f63fea41700 1 --2- 172.21.15.46:0/888866153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d81ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.438+0000 7f63fda3f700 1 --2- 172.21.15.46:0/888866153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d815380 0x55de3d81e0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f63e800e8a0 tx=0x7f63e8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7e734f278f02f6d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.438+0000 7f63ed5a9700 1 -- 172.21.15.46:0/888866153 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8023070 con 0x55de3d815380 2024-03-18T20:45:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.438+0000 7f63ed5a9700 1 -- 172.21.15.46:0/888866153 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63e801d970 con 0x55de3d815380 2024-03-18T20:45:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.438+0000 7f63ed5a9700 1 -- 172.21.15.46:0/888866153 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e80277d0 con 0x55de3d815380 2024-03-18T20:45:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.439+0000 7f640c639b80 1 -- 172.21.15.46:0/888866153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d815380 msgr2=0x55de3d81e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.439+0000 7f640c639b80 1 --2- 172.21.15.46:0/888866153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d815380 0x55de3d81e0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f63e800e8a0 tx=0x7f63e8014000 comp rx=0 tx=0).stop 2024-03-18T20:45:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.439+0000 7f640c639b80 1 -- 172.21.15.46:0/888866153 shutdown_connections 2024-03-18T20:45:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.439+0000 7f640c639b80 1 --2- 172.21.15.46:0/888866153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d814a60 0x55de3d8299f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.439+0000 7f640c639b80 1 --2- 172.21.15.46:0/888866153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d81ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.439+0000 7f640c639b80 1 --2- 172.21.15.46:0/888866153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d815380 0x55de3d81e0a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.439+0000 7f640c639b80 1 -- 172.21.15.46:0/888866153 >> 172.21.15.46:0/888866153 conn(0x55de3d78b220 msgr2=0x55de3d810b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.439+0000 7f640c639b80 1 -- 172.21.15.46:0/888866153 shutdown_connections 2024-03-18T20:45:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.439+0000 7f640c639b80 1 -- 172.21.15.46:0/888866153 wait complete. 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.439+0000 7f640c639b80 1 -- start start 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f640c639b80 1 Processor -- start 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f640c639b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 0x55de3d87be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f640c639b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d815380 0x55de3d87c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f640c639b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d886cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f640c639b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3d87cb10 con 0x55de3d815380 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f640c639b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3d887230 con 0x55de3d814a60 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f640c639b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de3d8873f0 con 0x55de3d824630 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f640c639b80 10 client.0 _ll_register_callbacks cb 0x7f63f0009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f63fe240700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 0x55de3d87be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f63fda3f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d815380 0x55de3d87c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f63fe240700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 0x55de3d87be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56632/0 (socket says 172.21.15.46:56632) 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f63fe240700 1 -- 172.21.15.46:0/3562694129 learned_addr learned my addr 172.21.15.46:0/3562694129 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f63fea41700 1 --2- 172.21.15.46:0/3562694129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d886cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f63fe240700 1 -- 172.21.15.46:0/3562694129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 msgr2=0x55de3d886cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f63fe240700 1 --2- 172.21.15.46:0/3562694129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d886cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f63fe240700 1 -- 172.21.15.46:0/3562694129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d815380 msgr2=0x55de3d87c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f63fe240700 1 --2- 172.21.15.46:0/3562694129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d815380 0x55de3d87c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.440+0000 7f63fe240700 1 -- 172.21.15.46:0/3562694129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de3d595b60 con 0x55de3d814a60 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f63fe240700 1 --2- 172.21.15.46:0/3562694129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 0x55de3d87be60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f63f4000c00 tx=0x7f63f4022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=911c7ece8722b395 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f63e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f63e1ffb700 1 -- 172.21.15.46:0/3562694129 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f407e070 con 0x55de3d814a60 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f63e1ffb700 1 -- 172.21.15.46:0/3562694129 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f63f40b8040 con 0x55de3d814a60 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f640c639b80 1 -- 172.21.15.46:0/3562694129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63f409ab50 con 0x55de3d814a60 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f63e1ffb700 1 -- 172.21.15.46:0/3562694129 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63f40b9040 con 0x55de3d814a60 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f640c639b80 10 client.28628 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f640c639b80 1 -- 172.21.15.46:0/3562694129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f63e8017050 con 0x55de3d814a60 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f63e1ffb700 1 -- 172.21.15.46:0/3562694129 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f409ab50 con 0x55de3d814a60 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f63e1ffb700 1 -- 172.21.15.46:0/3562694129 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f63f40a10b0 con 0x55de3d814a60 2024-03-18T20:45:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f640c639b80 10 client.28628 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f640c639b80 10 client.28628 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f640c639b80 20 client.28628 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f640c639b80 1 --2- 172.21.15.46:0/3562694129 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55de3d8886e0 0x55de3d88aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f640c639b80 4 client.28628 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.441+0000 7f640c639b80 1 -- 172.21.15.46:0/3562694129 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55de3d5b6cf0 con 0x55de3d8886e0 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f63fda3f700 1 --2- 172.21.15.46:0/3562694129 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55de3d8886e0 0x55de3d88aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f63e1ffb700 10 client.28628 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f63fda3f700 1 --2- 172.21.15.46:0/3562694129 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55de3d8886e0 0x55de3d88aad0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ebc6b64e56bea22d server_cookie=745c7e74f7742c6c in_seq=0 out_seq=0 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f63e1ffb700 1 -- 172.21.15.46:0/3562694129 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55de3d5b6cf0 con 0x55de3d8886e0 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f63e1ffb700 10 client.28628 handle_command_reply: tid=0 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f640c639b80 1 client.28628 shutdown 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f640c639b80 1 -- 172.21.15.46:0/3562694129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 msgr2=0x55de3d87be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f640c639b80 1 --2- 172.21.15.46:0/3562694129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 0x55de3d87be60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f63f4000c00 tx=0x7f63f4022f90 comp rx=0 tx=0).stop 2024-03-18T20:45:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f640c639b80 1 -- 172.21.15.46:0/3562694129 shutdown_connections 2024-03-18T20:45:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f640c639b80 1 --2- 172.21.15.46:0/3562694129 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55de3d8886e0 0x55de3d88aad0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f640c639b80 1 --2- 172.21.15.46:0/3562694129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de3d824630 0x55de3d886cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f640c639b80 1 --2- 172.21.15.46:0/3562694129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3d814a60 0x55de3d87be60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f63e1ffb700 0 client.28628 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:45:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f640c639b80 1 --2- 172.21.15.46:0/3562694129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de3d815380 0x55de3d87c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f640c639b80 1 -- 172.21.15.46:0/3562694129 >> 172.21.15.46:0/3562694129 conn(0x55de3d78b220 msgr2=0x55de3d81e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f640c639b80 1 -- 172.21.15.46:0/3562694129 shutdown_connections 2024-03-18T20:45:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.442+0000 7f640c639b80 1 -- 172.21.15.46:0/3562694129 wait complete. 2024-03-18T20:45:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f640c639b80 20 client.28628 trim_cache size 0 max 16384 2024-03-18T20:45:52.451 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:45:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f64004a4700 1 -- 172.21.15.46:0/167504771 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f63e40a1310 msgr2=0x7f63e40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f64004a4700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f63e40a1310 0x7f63e40a37d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009fd0 tx=0x7f63e8009260 comp rx=0 tx=0).stop 2024-03-18T20:45:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f64004a4700 1 -- 172.21.15.46:0/167504771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f8067e30 msgr2=0x7f63f8197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f64004a4700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f8067e30 0x7f63f8197260 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f63f4005910 tx=0x7f63f4004730 comp rx=0 tx=0).stop 2024-03-18T20:45:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f64004a4700 1 -- 172.21.15.46:0/167504771 shutdown_connections 2024-03-18T20:45:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f64004a4700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f63e40a1310 0x7f63e40a37d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f64004a4700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63f8105ce0 0x7f63f819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f64004a4700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f63f8067e30 0x7f63f8197260 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f64004a4700 1 --2- 172.21.15.46:0/167504771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63f80687f0 0x7f63f81977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f64004a4700 1 -- 172.21.15.46:0/167504771 >> 172.21.15.46:0/167504771 conn(0x7f63f8076d40 msgr2=0x7f63f80fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f64004a4700 1 -- 172.21.15.46:0/167504771 shutdown_connections 2024-03-18T20:45:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.443+0000 7f64004a4700 1 -- 172.21.15.46:0/167504771 wait complete. 2024-03-18T20:45:52.458 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:52 smithi046 ceph-mon[43495]: pgmap v302: 97 pgs: 97 active+clean; 138 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-03-18T20:45:52.459 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:52 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/4204287113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:45:52.460 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:45:52.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:45:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:52 smithi155 ceph-mon[48344]: pgmap v302: 97 pgs: 97 active+clean; 138 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-03-18T20:45:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:52 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/4204287113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:45:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.598+0000 7f2c18948700 1 -- 172.21.15.46:0/3792079385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c140692a0 msgr2=0x7f2c14069680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.598+0000 7f2c18948700 1 --2- 172.21.15.46:0/3792079385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c140692a0 0x7f2c14069680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2c040099c0 tx=0x7f2c04009cd0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.598+0000 7f2c18948700 1 -- 172.21.15.46:0/3792079385 shutdown_connections 2024-03-18T20:45:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.598+0000 7f2c18948700 1 --2- 172.21.15.46:0/3792079385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c141060b0 0x7f2c1410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.598+0000 7f2c18948700 1 --2- 172.21.15.46:0/3792079385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c140692a0 0x7f2c14069680 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.598+0000 7f2c18948700 1 --2- 172.21.15.46:0/3792079385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c14069c50 0x7f2c14105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.598+0000 7f2c18948700 1 -- 172.21.15.46:0/3792079385 >> 172.21.15.46:0/3792079385 conn(0x7f2c14076d80 msgr2=0x7f2c14077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.598+0000 7f2c18948700 1 -- 172.21.15.46:0/3792079385 shutdown_connections 2024-03-18T20:45:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.598+0000 7f2c18948700 1 -- 172.21.15.46:0/3792079385 wait complete. 2024-03-18T20:45:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c18948700 1 Processor -- start 2024-03-18T20:45:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c18948700 1 -- start start 2024-03-18T20:45:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c18948700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c140692a0 0x7f2c14107c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c18948700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c141060b0 0x7f2c14108160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c18948700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c141086a0 0x7f2c141a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c18948700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1410be10 con 0x7f2c140692a0 2024-03-18T20:45:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c18948700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c141a9ef0 con 0x7f2c141086a0 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c18948700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c141aa0b0 con 0x7f2c141060b0 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c1259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c140692a0 0x7f2c14107c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c1259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c140692a0 0x7f2c14107c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41908/0 (socket says 172.21.15.46:41908) 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c1259c700 1 -- 172.21.15.46:0/2010077411 learned_addr learned my addr 172.21.15.46:0/2010077411 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c11d9b700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c141060b0 0x7f2c14108160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.599+0000 7f2c12d9d700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c141086a0 0x7f2c141a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.600+0000 7f2c1259c700 1 -- 172.21.15.46:0/2010077411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c141060b0 msgr2=0x7f2c14108160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.600+0000 7f2c1259c700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c141060b0 0x7f2c14108160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.600+0000 7f2c1259c700 1 -- 172.21.15.46:0/2010077411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c141086a0 msgr2=0x7f2c141a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.600+0000 7f2c1259c700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c141086a0 0x7f2c141a9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.600+0000 7f2c1259c700 1 -- 172.21.15.46:0/2010077411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c040095f0 con 0x7f2c140692a0 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.600+0000 7f2c1259c700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c140692a0 0x7f2c14107c20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400f690 tx=0x7f2c0400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.600+0000 7f2c037fe700 1 -- 172.21.15.46:0/2010077411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0401c070 con 0x7f2c140692a0 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.600+0000 7f2c18948700 1 -- 172.21.15.46:0/2010077411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c141aa330 con 0x7f2c140692a0 2024-03-18T20:45:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.600+0000 7f2c037fe700 1 -- 172.21.15.46:0/2010077411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0400fcf0 con 0x7f2c140692a0 2024-03-18T20:45:52.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.600+0000 7f2c037fe700 1 -- 172.21.15.46:0/2010077411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04022c60 con 0x7f2c140692a0 2024-03-18T20:45:52.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.600+0000 7f2c18948700 1 -- 172.21.15.46:0/2010077411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c141aa7b0 con 0x7f2c140692a0 2024-03-18T20:45:52.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.602+0000 7f2c037fe700 1 -- 172.21.15.46:0/2010077411 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2c04020070 con 0x7f2c140692a0 2024-03-18T20:45:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.603+0000 7f2c037fe700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2bfc0a1290 0x7f2bfc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.603+0000 7f2c037fe700 1 -- 172.21.15.46:0/2010077411 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f2c040cdc60 con 0x7f2c140692a0 2024-03-18T20:45:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.603+0000 7f2c11d9b700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2bfc0a1290 0x7f2bfc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.604+0000 7f2c11d9b700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2bfc0a1290 0x7f2bfc0a3750 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2c14109350 tx=0x7f2c08009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.604+0000 7f2c24addb80 1 Processor -- start 2024-03-18T20:45:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.604+0000 7f2c24addb80 1 -- start start 2024-03-18T20:45:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c24addb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120a90e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c24addb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a91380 0x55b120a9bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c24addb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a9c220 0x55b120a9e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c24addb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1209ea990 con 0x55b120a90a60 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c24addb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1209e8350 con 0x55b120a91380 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c24addb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1209e3c40 con 0x55b120a9c220 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c1259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120a90e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c1259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120a90e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41920/0 (socket says 172.21.15.46:41920) 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c1259c700 1 -- 172.21.15.46:0/624056382 learned_addr learned my addr 172.21.15.46:0/624056382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c11d9b700 1 --2- 172.21.15.46:0/624056382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a91380 0x55b120a9bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c12d9d700 1 --2- 172.21.15.46:0/624056382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a9c220 0x55b120a9e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c1259c700 1 -- 172.21.15.46:0/624056382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a9c220 msgr2=0x55b120a9e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c1259c700 1 --2- 172.21.15.46:0/624056382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a9c220 0x55b120a9e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c1259c700 1 -- 172.21.15.46:0/624056382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a91380 msgr2=0x55b120a9bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c1259c700 1 --2- 172.21.15.46:0/624056382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a91380 0x55b120a9bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c1259c700 1 -- 172.21.15.46:0/624056382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1208653b0 con 0x55b120a90a60 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c1259c700 1 --2- 172.21.15.46:0/624056382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120a90e40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f2c0409ae90 tx=0x7f2c0409a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cb962790b2f845 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c00ff9700 1 -- 172.21.15.46:0/624056382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0409af00 con 0x55b120a90a60 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c00ff9700 1 -- 172.21.15.46:0/624056382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0409ed30 con 0x55b120a90a60 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.605+0000 7f2c00ff9700 1 -- 172.21.15.46:0/624056382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0409a3f0 con 0x55b120a90a60 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.606+0000 7f2c24addb80 1 -- 172.21.15.46:0/624056382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 msgr2=0x55b120a90e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.606+0000 7f2c24addb80 1 --2- 172.21.15.46:0/624056382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120a90e40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f2c0409ae90 tx=0x7f2c0409a860 comp rx=0 tx=0).stop 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.606+0000 7f2c24addb80 1 -- 172.21.15.46:0/624056382 shutdown_connections 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.606+0000 7f2c24addb80 1 --2- 172.21.15.46:0/624056382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120a90e40 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.606+0000 7f2c24addb80 1 --2- 172.21.15.46:0/624056382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a9c220 0x55b120a9e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.606+0000 7f2c24addb80 1 --2- 172.21.15.46:0/624056382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a91380 0x55b120a9bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.606+0000 7f2c24addb80 1 -- 172.21.15.46:0/624056382 >> 172.21.15.46:0/624056382 conn(0x55b120a07220 msgr2=0x55b120a8d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.606+0000 7f2c24addb80 1 -- 172.21.15.46:0/624056382 shutdown_connections 2024-03-18T20:45:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.606+0000 7f2c24addb80 1 -- 172.21.15.46:0/624056382 wait complete. 2024-03-18T20:45:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c24addb80 1 -- start start 2024-03-18T20:45:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c24addb80 1 Processor -- start 2024-03-18T20:45:52.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c24addb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120a9bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c24addb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a91380 0x55b120a98f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c24addb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a9c220 0x55b120a99590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c24addb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b12097fd80 con 0x55b120a90a60 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c24addb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b12097fa20 con 0x55b120a91380 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c24addb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1208676a0 con 0x55b120a9c220 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c24addb80 10 client.0 _ll_register_callbacks cb 0x55b1207eb3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c1259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120a9bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c1259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120a9bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41928/0 (socket says 172.21.15.46:41928) 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c1259c700 1 -- 172.21.15.46:0/167001412 learned_addr learned my addr 172.21.15.46:0/167001412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c11d9b700 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a91380 0x55b120a98f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.607+0000 7f2c12d9d700 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a9c220 0x55b120a99590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2c1259c700 1 -- 172.21.15.46:0/167001412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a9c220 msgr2=0x55b120a99590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2c1259c700 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a9c220 0x55b120a99590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2c1259c700 1 -- 172.21.15.46:0/167001412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a91380 msgr2=0x55b120a98f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2c1259c700 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a91380 0x55b120a98f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2c1259c700 1 -- 172.21.15.46:0/167001412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c141aa7b0 con 0x55b120a90a60 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2c12d9d700 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a9c220 0x55b120a99590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2c11d9b700 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a91380 0x55b120a98f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2c1259c700 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120a9bc20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f2c04029000 tx=0x7f2c04009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb114a804c22cc41 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2bf9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2bf9ffb700 1 -- 172.21.15.46:0/167001412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0409af00 con 0x55b120a90a60 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2bf9ffb700 1 -- 172.21.15.46:0/167001412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f2c040a32f0 con 0x55b120a90a60 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2c24addb80 1 -- 172.21.15.46:0/167001412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b120a84ad0 con 0x55b120a90a60 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2bf9ffb700 1 -- 172.21.15.46:0/167001412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c0409b370 con 0x55b120a90a60 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2c24addb80 10 client.19005 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2c24addb80 1 -- 172.21.15.46:0/167001412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55b120811b60 con 0x55b120a90a60 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.608+0000 7f2bf9ffb700 1 -- 172.21.15.46:0/167001412 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c04064080 con 0x55b120a90a60 2024-03-18T20:45:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.609+0000 7f2bf9ffb700 1 -- 172.21.15.46:0/167001412 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2c04096e10 con 0x55b120a90a60 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.609+0000 7f2c24addb80 10 client.19005 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.609+0000 7f2c24addb80 10 client.19005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.609+0000 7f2c24addb80 20 client.19005 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.609+0000 7f2c24addb80 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b120aab760 0x55b120aadb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.609+0000 7f2c24addb80 4 client.19005 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.609+0000 7f2c24addb80 1 -- 172.21.15.46:0/167001412 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1209dc700 con 0x55b120aab760 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.609+0000 7f2c11d9b700 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b120aab760 0x55b120aadb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.609+0000 7f2bf9ffb700 10 client.19005 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.609+0000 7f2c11d9b700 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b120aab760 0x55b120aadb50 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a3f721c0dbcb343 server_cookie=6af9bf0e38f71a05 in_seq=0 out_seq=0 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2bf9ffb700 1 -- 172.21.15.46:0/167001412 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b1209dc700 con 0x55b120aab760 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2bf9ffb700 10 client.19005 handle_command_reply: tid=0 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2c24addb80 1 client.19005 shutdown 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2c24addb80 1 -- 172.21.15.46:0/167001412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 msgr2=0x55b120a9bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2c24addb80 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120a9bc20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f2c04029000 tx=0x7f2c04009cd0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2c24addb80 1 -- 172.21.15.46:0/167001412 shutdown_connections 2024-03-18T20:45:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2c24addb80 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b120aab760 0x55b120aadb50 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2c24addb80 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120a9bc20 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2c24addb80 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a9c220 0x55b120a99590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2bf9ffb700 0 client.19005 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2c24addb80 1 --2- 172.21.15.46:0/167001412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a91380 0x55b120a98f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2c24addb80 1 -- 172.21.15.46:0/167001412 >> 172.21.15.46:0/167001412 conn(0x55b120a07220 msgr2=0x55b120a8cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2c24addb80 1 -- 172.21.15.46:0/167001412 shutdown_connections 2024-03-18T20:45:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2c24addb80 1 -- 172.21.15.46:0/167001412 wait complete. 2024-03-18T20:45:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.610+0000 7f2c24addb80 20 client.19005 trim_cache size 0 max 16384 2024-03-18T20:45:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.624+0000 7f2c24addb80 1 Processor -- start 2024-03-18T20:45:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.624+0000 7f2c24addb80 1 -- start start 2024-03-18T20:45:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.624+0000 7f2c24addb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120aa2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.624+0000 7f2c24addb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a91380 0x55b120a8cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.624+0000 7f2c24addb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a9c220 0x55b120a8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.624+0000 7f2c24addb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120a91d30 con 0x55b120a90a60 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.624+0000 7f2c24addb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120aa3150 con 0x55b120a9c220 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.624+0000 7f2c24addb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120a94db0 con 0x55b120a91380 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c1259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120aa2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c11d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a91380 0x55b120a8cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c1259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120aa2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41932/0 (socket says 172.21.15.46:41932) 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c1259c700 1 -- 172.21.15.46:0/2550466594 learned_addr learned my addr 172.21.15.46:0/2550466594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c12d9d700 1 --2- 172.21.15.46:0/2550466594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a9c220 0x55b120a8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c1259c700 1 -- 172.21.15.46:0/2550466594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a91380 msgr2=0x55b120a8cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c1259c700 1 --2- 172.21.15.46:0/2550466594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a91380 0x55b120a8cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c1259c700 1 -- 172.21.15.46:0/2550466594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a9c220 msgr2=0x55b120a8d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c1259c700 1 --2- 172.21.15.46:0/2550466594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a9c220 0x55b120a8d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c1259c700 1 -- 172.21.15.46:0/2550466594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1209a9ea0 con 0x55b120a90a60 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c11d9b700 1 --2- 172.21.15.46:0/2550466594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a91380 0x55b120a8cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c12d9d700 1 --2- 172.21.15.46:0/2550466594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a9c220 0x55b120a8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c1259c700 1 --2- 172.21.15.46:0/2550466594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120aa2c10 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2c040a1190 tx=0x7f2c04097370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11ac977c6da1c14 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c017fa700 1 -- 172.21.15.46:0/2550466594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c040a32f0 con 0x55b120a90a60 2024-03-18T20:45:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c017fa700 1 -- 172.21.15.46:0/2550466594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b120a84ad0 con 0x55b120a90a60 2024-03-18T20:45:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.625+0000 7f2c017fa700 1 -- 172.21.15.46:0/2550466594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0409b400 con 0x55b120a90a60 2024-03-18T20:45:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.626+0000 7f2c24addb80 1 -- 172.21.15.46:0/2550466594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 msgr2=0x55b120aa2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.626+0000 7f2c24addb80 1 --2- 172.21.15.46:0/2550466594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120aa2c10 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2c040a1190 tx=0x7f2c04097370 comp rx=0 tx=0).stop 2024-03-18T20:45:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.626+0000 7f2c24addb80 1 -- 172.21.15.46:0/2550466594 shutdown_connections 2024-03-18T20:45:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.626+0000 7f2c24addb80 1 --2- 172.21.15.46:0/2550466594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120aa2c10 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.626+0000 7f2c24addb80 1 --2- 172.21.15.46:0/2550466594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a9c220 0x55b120a8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.626+0000 7f2c24addb80 1 --2- 172.21.15.46:0/2550466594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a91380 0x55b120a8cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.626+0000 7f2c24addb80 1 -- 172.21.15.46:0/2550466594 >> 172.21.15.46:0/2550466594 conn(0x55b120a07220 msgr2=0x55b120a9b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.626+0000 7f2c24addb80 1 -- 172.21.15.46:0/2550466594 shutdown_connections 2024-03-18T20:45:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.626+0000 7f2c24addb80 1 -- 172.21.15.46:0/2550466594 wait complete. 2024-03-18T20:45:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.626+0000 7f2c24addb80 1 -- start start 2024-03-18T20:45:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.626+0000 7f2c24addb80 1 Processor -- start 2024-03-18T20:45:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c24addb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120aa0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c24addb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a91380 0x55b120aa1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c24addb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a9c220 0x55b120aa1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c24addb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120afe690 con 0x55b120a90a60 2024-03-18T20:45:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c24addb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120afe800 con 0x55b120a9c220 2024-03-18T20:45:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c24addb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b120afe9c0 con 0x55b120a91380 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c24addb80 10 client.0 _ll_register_callbacks cb 0x7f2c080096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c1259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120aa0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c1259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120aa0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41942/0 (socket says 172.21.15.46:41942) 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c1259c700 1 -- 172.21.15.46:0/4237859414 learned_addr learned my addr 172.21.15.46:0/4237859414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c11d9b700 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a91380 0x55b120aa1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c12d9d700 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a9c220 0x55b120aa1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c1259c700 1 -- 172.21.15.46:0/4237859414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a91380 msgr2=0x55b120aa1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c1259c700 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a91380 0x55b120aa1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c1259c700 1 -- 172.21.15.46:0/4237859414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a9c220 msgr2=0x55b120aa1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c1259c700 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a9c220 0x55b120aa1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c1259c700 1 -- 172.21.15.46:0/4237859414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c08017070 con 0x55b120a90a60 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2c1259c700 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120aa0bf0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f2c04022700 tx=0x7f2c04009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b9c5b859e3a76a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.627+0000 7f2bf9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2bf9ffb700 1 -- 172.21.15.46:0/4237859414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c040a3060 con 0x55b120a90a60 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2bf9ffb700 1 -- 172.21.15.46:0/4237859414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f2c040a32f0 con 0x55b120a90a60 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2c24addb80 1 -- 172.21.15.46:0/4237859414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c0409ed30 con 0x55b120a90a60 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2bf9ffb700 1 -- 172.21.15.46:0/4237859414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c0409a860 con 0x55b120a90a60 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2c24addb80 10 client.19011 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2c24addb80 1 -- 172.21.15.46:0/4237859414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f2c0c009100 con 0x55b120a90a60 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2bf9ffb700 1 -- 172.21.15.46:0/4237859414 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c040a7020 con 0x55b120a90a60 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2bf9ffb700 1 -- 172.21.15.46:0/4237859414 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2c04096d80 con 0x55b120a90a60 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2c24addb80 10 client.19011 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2c24addb80 10 client.19011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2c24addb80 20 client.19011 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2c24addb80 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b120affd80 0x55b120b021a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2c24addb80 4 client.19011 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.628+0000 7f2c24addb80 1 -- 172.21.15.46:0/4237859414 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b120832cf0 con 0x55b120affd80 2024-03-18T20:45:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c11d9b700 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b120affd80 0x55b120b021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2bf9ffb700 10 client.19011 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c11d9b700 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b120affd80 0x55b120b021a0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e85676446786fdc2 server_cookie=4fb25bcace88c207 in_seq=0 out_seq=0 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2bf9ffb700 1 -- 172.21.15.46:0/4237859414 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b120832cf0 con 0x55b120affd80 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2bf9ffb700 10 client.19011 handle_command_reply: tid=0 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c24addb80 1 client.19011 shutdown 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c24addb80 1 -- 172.21.15.46:0/4237859414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 msgr2=0x55b120aa0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c24addb80 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120aa0bf0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f2c04022700 tx=0x7f2c04009cd0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c24addb80 1 -- 172.21.15.46:0/4237859414 shutdown_connections 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c24addb80 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b120affd80 0x55b120b021a0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c24addb80 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b120a90a60 0x55b120aa0bf0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2bf9ffb700 0 client.19011 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c24addb80 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b120a9c220 0x55b120aa1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c24addb80 1 --2- 172.21.15.46:0/4237859414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b120a91380 0x55b120aa1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c24addb80 1 -- 172.21.15.46:0/4237859414 >> 172.21.15.46:0/4237859414 conn(0x55b120a07220 msgr2=0x55b120a94240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c24addb80 1 -- 172.21.15.46:0/4237859414 shutdown_connections 2024-03-18T20:45:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c24addb80 1 -- 172.21.15.46:0/4237859414 wait complete. 2024-03-18T20:45:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.629+0000 7f2c24addb80 20 client.19011 trim_cache size 0 max 16384 2024-03-18T20:45:52.639 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:45:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.630+0000 7f2c18948700 1 -- 172.21.15.46:0/2010077411 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2bfc0a1290 msgr2=0x7f2bfc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.630+0000 7f2c18948700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2bfc0a1290 0x7f2bfc0a3750 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2c14109350 tx=0x7f2c08009260 comp rx=0 tx=0).stop 2024-03-18T20:45:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.630+0000 7f2c18948700 1 -- 172.21.15.46:0/2010077411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c140692a0 msgr2=0x7f2c14107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.630+0000 7f2c18948700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c140692a0 0x7f2c14107c20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400f690 tx=0x7f2c0400bfd0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.630+0000 7f2c18948700 1 -- 172.21.15.46:0/2010077411 shutdown_connections 2024-03-18T20:45:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.630+0000 7f2c18948700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2bfc0a1290 0x7f2bfc0a3750 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.630+0000 7f2c18948700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c140692a0 0x7f2c14107c20 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.630+0000 7f2c18948700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c141086a0 0x7f2c141a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.630+0000 7f2c18948700 1 --2- 172.21.15.46:0/2010077411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c141060b0 0x7f2c14108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.630+0000 7f2c18948700 1 -- 172.21.15.46:0/2010077411 >> 172.21.15.46:0/2010077411 conn(0x7f2c14076d80 msgr2=0x7f2c1410c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.630+0000 7f2c18948700 1 -- 172.21.15.46:0/2010077411 shutdown_connections 2024-03-18T20:45:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:52.630+0000 7f2c18948700 1 -- 172.21.15.46:0/2010077411 wait complete. 2024-03-18T20:45:52.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 359941 3.68 MB/sec execute 407 sec latency 248.860 ms 2024-03-18T20:45:52.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:52 smithi073 ceph-mon[47384]: pgmap v302: 97 pgs: 97 active+clean; 138 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-03-18T20:45:52.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:52 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/4204287113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:45:53.646 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:45:53.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:45:53.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:45:53.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 359941 3.67 MB/sec execute 408 sec latency 1248.903 ms 2024-03-18T20:45:53.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.794+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/1253545684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf68100800 msgr2=0x7fdf68104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.794+0000 7fdf6c9a5700 1 --2- 172.21.15.46:0/1253545684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf68100800 0x7fdf68104830 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009990 tx=0x7fdf50009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.794+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/1253545684 shutdown_connections 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.794+0000 7fdf6c9a5700 1 --2- 172.21.15.46:0/1253545684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf68104f60 0x7fdf6810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.794+0000 7fdf6c9a5700 1 --2- 172.21.15.46:0/1253545684 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf680ffe50 0x7fdf68100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.794+0000 7fdf6c9a5700 1 --2- 172.21.15.46:0/1253545684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf68100800 0x7fdf68104830 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.794+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/1253545684 >> 172.21.15.46:0/1253545684 conn(0x7fdf68076d40 msgr2=0x7fdf68077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.794+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/1253545684 shutdown_connections 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.794+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/1253545684 wait complete. 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.795+0000 7fdf6c9a5700 1 Processor -- start 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.795+0000 7fdf6c9a5700 1 -- start start 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.795+0000 7fdf6c9a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf680ffe50 0x7fdf681971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.795+0000 7fdf6c9a5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf68104f60 0x7fdf681976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.795+0000 7fdf6c9a5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf6819ba80 0x7fdf6819bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.795+0000 7fdf6c9a5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6819c500 con 0x7fdf680ffe50 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.795+0000 7fdf6c9a5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6819c670 con 0x7fdf6819ba80 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.795+0000 7fdf6c9a5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6819c830 con 0x7fdf68104f60 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf6659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf680ffe50 0x7fdf681971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf6659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf680ffe50 0x7fdf681971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41948/0 (socket says 172.21.15.46:41948) 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf6659c700 1 -- 172.21.15.46:0/4104660261 learned_addr learned my addr 172.21.15.46:0/4104660261 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf66d9d700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf6819ba80 0x7fdf6819bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf65d9b700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf68104f60 0x7fdf681976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf6659c700 1 -- 172.21.15.46:0/4104660261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf68104f60 msgr2=0x7fdf681976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf6659c700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf68104f60 0x7fdf681976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf6659c700 1 -- 172.21.15.46:0/4104660261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf6819ba80 msgr2=0x7fdf6819bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf6659c700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf6819ba80 0x7fdf6819bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf6659c700 1 -- 172.21.15.46:0/4104660261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf500095f0 con 0x7fdf680ffe50 2024-03-18T20:45:53.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf66d9d700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf6819ba80 0x7fdf6819bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:53.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf6659c700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf680ffe50 0x7fdf681971a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800d980 tx=0x7fdf5800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:53.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf5f7fe700 1 -- 172.21.15.46:0/4104660261 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5800bb60 con 0x7fdf680ffe50 2024-03-18T20:45:53.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/4104660261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf681a3d90 con 0x7fdf680ffe50 2024-03-18T20:45:53.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf5f7fe700 1 -- 172.21.15.46:0/4104660261 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf58007640 con 0x7fdf680ffe50 2024-03-18T20:45:53.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf5f7fe700 1 -- 172.21.15.46:0/4104660261 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58004e90 con 0x7fdf680ffe50 2024-03-18T20:45:53.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.796+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/4104660261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf681a4260 con 0x7fdf680ffe50 2024-03-18T20:45:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.799+0000 7fdf5f7fe700 1 -- 172.21.15.46:0/4104660261 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fdf5800bcc0 con 0x7fdf680ffe50 2024-03-18T20:45:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.799+0000 7fdf5f7fe700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdf540a1290 0x7fdf540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.799+0000 7fdf65d9b700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdf540a1290 0x7fdf540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.799+0000 7fdf5f7fe700 1 -- 172.21.15.46:0/4104660261 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fdf580ccb50 con 0x7fdf680ffe50 2024-03-18T20:45:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.800+0000 7fdf65d9b700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdf540a1290 0x7fdf540a3750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000b5c0 tx=0x7fdf50003680 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.800+0000 7fdf78b3ab80 1 Processor -- start 2024-03-18T20:45:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.800+0000 7fdf78b3ab80 1 -- start start 2024-03-18T20:45:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf78b3ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088efae0 0x5607088efec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf78b3ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088f0400 0x5607088fad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf78b3ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088fb2a0 0x5607088fd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf78b3ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560708849a10 con 0x5607088fb2a0 2024-03-18T20:45:53.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf78b3ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607088473d0 con 0x5607088f0400 2024-03-18T20:45:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf78b3ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560708842cc0 con 0x5607088efae0 2024-03-18T20:45:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf66d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088fb2a0 0x5607088fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf6659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088efae0 0x5607088efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf66d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088fb2a0 0x5607088fd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41954/0 (socket says 172.21.15.46:41954) 2024-03-18T20:45:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf66d9d700 1 -- 172.21.15.46:0/672138099 learned_addr learned my addr 172.21.15.46:0/672138099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf65d9b700 1 --2- 172.21.15.46:0/672138099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088f0400 0x5607088fad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf66d9d700 1 -- 172.21.15.46:0/672138099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088efae0 msgr2=0x5607088efec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf66d9d700 1 --2- 172.21.15.46:0/672138099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088efae0 0x5607088efec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf66d9d700 1 -- 172.21.15.46:0/672138099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088f0400 msgr2=0x5607088fad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf66d9d700 1 --2- 172.21.15.46:0/672138099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088f0400 0x5607088fad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf66d9d700 1 -- 172.21.15.46:0/672138099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607086c4430 con 0x5607088fb2a0 2024-03-18T20:45:53.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf6659c700 1 --2- 172.21.15.46:0/672138099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088efae0 0x5607088efec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.801+0000 7fdf66d9d700 1 --2- 172.21.15.46:0/672138099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088fb2a0 0x5607088fd690 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009dd0 tx=0x7fdf600077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be41b80ca0561bd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.802+0000 7fdf5cff9700 1 -- 172.21.15.46:0/672138099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60018070 con 0x5607088fb2a0 2024-03-18T20:45:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.802+0000 7fdf5cff9700 1 -- 172.21.15.46:0/672138099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf6000f050 con 0x5607088fb2a0 2024-03-18T20:45:53.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.802+0000 7fdf5cff9700 1 -- 172.21.15.46:0/672138099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60005a70 con 0x5607088fb2a0 2024-03-18T20:45:53.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.802+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/672138099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088fb2a0 msgr2=0x5607088fd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:53.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.802+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/672138099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088fb2a0 0x5607088fd690 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009dd0 tx=0x7fdf600077e0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/672138099 shutdown_connections 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/672138099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088fb2a0 0x5607088fd690 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/672138099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088efae0 0x5607088efec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/672138099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088f0400 0x5607088fad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/672138099 >> 172.21.15.46:0/672138099 conn(0x5607088662a0 msgr2=0x5607088ec0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/672138099 shutdown_connections 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/672138099 wait complete. 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 -- start start 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 Processor -- start 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 0x5607088fad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088f0400 0x5607088f8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 0x5607088f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607087dee00 con 0x5607088efae0 2024-03-18T20:45:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607087deaa0 con 0x5607088fb2a0 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607086c6720 con 0x5607088f0400 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.803+0000 7fdf78b3ab80 10 client.0 _ll_register_callbacks cb 0x56070864a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf6659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 0x5607088fad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf6659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 0x5607088fad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41968/0 (socket says 172.21.15.46:41968) 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf6659c700 1 -- 172.21.15.46:0/737105514 learned_addr learned my addr 172.21.15.46:0/737105514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf66d9d700 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 0x5607088f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf65d9b700 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088f0400 0x5607088f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf6659c700 1 -- 172.21.15.46:0/737105514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088f0400 msgr2=0x5607088f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf6659c700 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088f0400 0x5607088f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf6659c700 1 -- 172.21.15.46:0/737105514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 msgr2=0x5607088f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf6659c700 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 0x5607088f87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf6659c700 1 -- 172.21.15.46:0/737105514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf681a4260 con 0x5607088efae0 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf6659c700 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 0x5607088fad30 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fdf580ca2e0 tx=0x7fdf5800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce5516e72f5d734 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:53.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf4dffb700 1 -- 172.21.15.46:0/737105514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5809a0a0 con 0x5607088efae0 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf4dffb700 1 -- 172.21.15.46:0/737105514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fdf580a20c0 con 0x5607088efae0 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/737105514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607088e3b50 con 0x5607088efae0 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf4dffb700 1 -- 172.21.15.46:0/737105514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf58099130 con 0x5607088efae0 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf78b3ab80 10 client.19023 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.804+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/737105514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7fdf60009a00 con 0x5607088efae0 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.805+0000 7fdf4dffb700 1 -- 172.21.15.46:0/737105514 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf58063080 con 0x5607088efae0 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.805+0000 7fdf4dffb700 1 -- 172.21.15.46:0/737105514 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdf5805faf0 con 0x5607088efae0 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.805+0000 7fdf78b3ab80 10 client.19023 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.805+0000 7fdf78b3ab80 10 client.19023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.805+0000 7fdf78b3ab80 20 client.19023 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.805+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56070890ae20 0x56070890d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.805+0000 7fdf78b3ab80 4 client.19023 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.805+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/737105514 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56070883b780 con 0x56070890ae20 2024-03-18T20:45:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.805+0000 7fdf65d9b700 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56070890ae20 0x56070890d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf4dffb700 10 client.19023 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf65d9b700 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56070890ae20 0x56070890d240 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=970dea63c8f95365 server_cookie=1fa0ae3c5d9bb515 in_seq=0 out_seq=0 2024-03-18T20:45:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf4dffb700 1 -- 172.21.15.46:0/737105514 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56070883b780 con 0x56070890ae20 2024-03-18T20:45:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf4dffb700 10 client.19023 handle_command_reply: tid=0 2024-03-18T20:45:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf78b3ab80 1 client.19023 shutdown 2024-03-18T20:45:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/737105514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 msgr2=0x5607088fad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 0x5607088fad30 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fdf580ca2e0 tx=0x7fdf5800df70 comp rx=0 tx=0).stop 2024-03-18T20:45:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/737105514 shutdown_connections 2024-03-18T20:45:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56070890ae20 0x56070890d240 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 0x5607088fad30 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 0x5607088f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf4dffb700 0 client.19023 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/737105514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088f0400 0x5607088f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/737105514 >> 172.21.15.46:0/737105514 conn(0x5607088662a0 msgr2=0x5607088ebc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/737105514 shutdown_connections 2024-03-18T20:45:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.806+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/737105514 wait complete. 2024-03-18T20:45:53.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.807+0000 7fdf78b3ab80 20 client.19023 trim_cache size 0 max 16384 2024-03-18T20:45:53.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.824+0000 7fdf78b3ab80 1 Processor -- start 2024-03-18T20:45:53.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf78b3ab80 1 -- start start 2024-03-18T20:45:53.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf78b3ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 0x560708901ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf78b3ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088f0400 0x560708906d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf78b3ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 0x5607088f7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf78b3ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607088f0db0 con 0x5607088efae0 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf78b3ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607088f8d90 con 0x5607088fb2a0 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf78b3ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607088e8660 con 0x5607088f0400 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf6659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 0x560708901ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf6659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 0x560708901ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41980/0 (socket says 172.21.15.46:41980) 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf6659c700 1 -- 172.21.15.46:0/1967772987 learned_addr learned my addr 172.21.15.46:0/1967772987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf65d9b700 1 --2- 172.21.15.46:0/1967772987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088f0400 0x560708906d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf6659c700 1 -- 172.21.15.46:0/1967772987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088f0400 msgr2=0x560708906d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf6659c700 1 --2- 172.21.15.46:0/1967772987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088f0400 0x560708906d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf6659c700 1 -- 172.21.15.46:0/1967772987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 msgr2=0x5607088f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf6659c700 1 --2- 172.21.15.46:0/1967772987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 0x5607088f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf6659c700 1 -- 172.21.15.46:0/1967772987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560708670be0 con 0x5607088efae0 2024-03-18T20:45:53.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf6659c700 1 --2- 172.21.15.46:0/1967772987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 0x560708901ed0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fdf58099940 tx=0x7fdf580981f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c272febb8d9aeca5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.825+0000 7fdf5d7fa700 1 -- 172.21.15.46:0/1967772987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58014040 con 0x5607088efae0 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.826+0000 7fdf5d7fa700 1 -- 172.21.15.46:0/1967772987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607088e3b50 con 0x5607088efae0 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.826+0000 7fdf5d7fa700 1 -- 172.21.15.46:0/1967772987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5809a100 con 0x5607088efae0 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.826+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/1967772987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 msgr2=0x560708901ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.826+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/1967772987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 0x560708901ed0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fdf58099940 tx=0x7fdf580981f0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.826+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/1967772987 shutdown_connections 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.826+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/1967772987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088efae0 0x560708901ed0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.826+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/1967772987 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 0x5607088f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.826+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/1967772987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088f0400 0x560708906d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.826+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/1967772987 >> 172.21.15.46:0/1967772987 conn(0x5607088662a0 msgr2=0x560708906620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.826+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/1967772987 shutdown_connections 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.826+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/1967772987 wait complete. 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.826+0000 7fdf78b3ab80 1 -- start start 2024-03-18T20:45:53.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf78b3ab80 1 Processor -- start 2024-03-18T20:45:53.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf78b3ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088efae0 0x560708900c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf78b3ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088f0400 0x560708901180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf78b3ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 0x56070895ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf78b3ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560708901890 con 0x5607088f0400 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf78b3ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070895f160 con 0x5607088fb2a0 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf78b3ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56070895f2d0 con 0x5607088efae0 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf78b3ab80 10 client.0 _ll_register_callbacks cb 0x7fdf58004750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088f0400 0x560708901180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf65d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088f0400 0x560708901180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41990/0 (socket says 172.21.15.46:41990) 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf65d9b700 1 -- 172.21.15.46:0/3349174011 learned_addr learned my addr 172.21.15.46:0/3349174011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf6659c700 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088efae0 0x560708900c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf66d9d700 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 0x56070895ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf65d9b700 1 -- 172.21.15.46:0/3349174011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088efae0 msgr2=0x560708900c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf65d9b700 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088efae0 0x560708900c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf65d9b700 1 -- 172.21.15.46:0/3349174011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 msgr2=0x56070895ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf65d9b700 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 0x56070895ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf65d9b700 1 -- 172.21.15.46:0/3349174011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf60009a00 con 0x5607088f0400 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf66d9d700 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 0x56070895ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.827+0000 7fdf6659c700 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088efae0 0x560708900c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf65d9b700 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088f0400 0x560708901180 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fdf50005850 tx=0x7fdf50009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfb65f72fda34fc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf4dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf4dffb700 1 -- 172.21.15.46:0/3349174011 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5002b040 con 0x5607088f0400 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/3349174011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf6000f050 con 0x5607088f0400 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf4dffb700 1 -- 172.21.15.46:0/3349174011 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fdf50034520 con 0x5607088f0400 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf4dffb700 1 -- 172.21.15.46:0/3349174011 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf5000b770 con 0x5607088f0400 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf78b3ab80 10 client.19029 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/3349174011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560708808f20 con 0x5607088f0400 2024-03-18T20:45:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf4dffb700 1 -- 172.21.15.46:0/3349174011 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf6000f050 con 0x5607088f0400 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf4dffb700 1 -- 172.21.15.46:0/3349174011 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdf50004a60 con 0x5607088f0400 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf78b3ab80 10 client.19029 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf78b3ab80 10 client.19029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf78b3ab80 20 client.19029 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560708960c40 0x560708963060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf78b3ab80 4 client.19029 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.828+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/3349174011 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560708691d70 con 0x560708960c40 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.829+0000 7fdf6659c700 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560708960c40 0x560708963060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.829+0000 7fdf6659c700 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560708960c40 0x560708963060 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52c0d32e980b75d server_cookie=c9c592caed3daf23 in_seq=0 out_seq=0 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.829+0000 7fdf4dffb700 10 client.19029 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.830+0000 7fdf4dffb700 1 -- 172.21.15.46:0/3349174011 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560708691d70 con 0x560708960c40 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.830+0000 7fdf4dffb700 10 client.19029 handle_command_reply: tid=0 2024-03-18T20:45:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.830+0000 7fdf78b3ab80 1 client.19029 shutdown 2024-03-18T20:45:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.830+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/3349174011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088f0400 msgr2=0x560708901180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.830+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088f0400 0x560708901180 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fdf50005850 tx=0x7fdf50009ca0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/3349174011 shutdown_connections 2024-03-18T20:45:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560708960c40 0x560708963060 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5607088fb2a0 0x56070895ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5607088efae0 0x560708900c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf4dffb700 0 client.19029 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf78b3ab80 1 --2- 172.21.15.46:0/3349174011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5607088f0400 0x560708901180 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/3349174011 >> 172.21.15.46:0/3349174011 conn(0x5607088662a0 msgr2=0x560708906eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/3349174011 shutdown_connections 2024-03-18T20:45:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf78b3ab80 1 -- 172.21.15.46:0/3349174011 wait complete. 2024-03-18T20:45:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf78b3ab80 20 client.19029 trim_cache size 0 max 16384 2024-03-18T20:45:53.840 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:45:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:45:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "5a86dfc4-ef71-424a-af7e-454561037ec0", 2024-03-18T20:45:53.840 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:45:53.840 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:45:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/4104660261 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdf540a1290 msgr2=0x7fdf540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf6c9a5700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdf540a1290 0x7fdf540a3750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000b5c0 tx=0x7fdf50003680 comp rx=0 tx=0).stop 2024-03-18T20:45:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/4104660261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf680ffe50 msgr2=0x7fdf681971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf6c9a5700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf680ffe50 0x7fdf681971a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800d980 tx=0x7fdf5800dd40 comp rx=0 tx=0).stop 2024-03-18T20:45:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/4104660261 shutdown_connections 2024-03-18T20:45:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf6c9a5700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdf540a1290 0x7fdf540a3750 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf6c9a5700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf680ffe50 0x7fdf681971a0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf6c9a5700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf6819ba80 0x7fdf6819bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf6c9a5700 1 --2- 172.21.15.46:0/4104660261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf68104f60 0x7fdf681976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.831+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/4104660261 >> 172.21.15.46:0/4104660261 conn(0x7fdf68076d40 msgr2=0x7fdf680fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.832+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/4104660261 shutdown_connections 2024-03-18T20:45:53.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:53.832+0000 7fdf6c9a5700 1 -- 172.21.15.46:0/4104660261 wait complete. 2024-03-18T20:45:53.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:45:53.851 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:45:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.001+0000 7f69d3934700 1 -- 172.21.15.46:0/2248490741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69cc0687f0 msgr2=0x7f69cc10db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.001+0000 7f69d3934700 1 --2- 172.21.15.46:0/2248490741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69cc0687f0 0x7f69cc10db50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f69c400b0d0 tx=0x7f69c400b3e0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.001+0000 7f69d3934700 1 -- 172.21.15.46:0/2248490741 shutdown_connections 2024-03-18T20:45:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.001+0000 7f69d3934700 1 --2- 172.21.15.46:0/2248490741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69cc0687f0 0x7f69cc10db50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.001+0000 7f69d3934700 1 --2- 172.21.15.46:0/2248490741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69cc105ce0 0x7f69cc1060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.001+0000 7f69d3934700 1 --2- 172.21.15.46:0/2248490741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc067e30 0x7f69cc0682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.001+0000 7f69d3934700 1 -- 172.21.15.46:0/2248490741 >> 172.21.15.46:0/2248490741 conn(0x7f69cc076d40 msgr2=0x7f69cc077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.001+0000 7f69d3934700 1 -- 172.21.15.46:0/2248490741 shutdown_connections 2024-03-18T20:45:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.001+0000 7f69d3934700 1 -- 172.21.15.46:0/2248490741 wait complete. 2024-03-18T20:45:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d3934700 1 Processor -- start 2024-03-18T20:45:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d3934700 1 -- start start 2024-03-18T20:45:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d3934700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69cc067e30 0x7f69cc0ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d3934700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69cc0687f0 0x7f69cc1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d3934700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc105ce0 0x7f69cc1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d3934700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc1a56c0 con 0x7f69cc105ce0 2024-03-18T20:45:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d3934700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc1a5830 con 0x7f69cc0687f0 2024-03-18T20:45:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d3934700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc1a59f0 con 0x7f69cc067e30 2024-03-18T20:45:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d1ed1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc105ce0 0x7f69cc1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d16d0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69cc067e30 0x7f69cc0ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d0ecf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69cc0687f0 0x7f69cc1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d1ed1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc105ce0 0x7f69cc1006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42018/0 (socket says 172.21.15.46:42018) 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d16d0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69cc067e30 0x7f69cc0ffc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:39222/0 (socket says 172.21.15.46:39222) 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d1ed1700 1 -- 172.21.15.46:0/632741814 learned_addr learned my addr 172.21.15.46:0/632741814 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d0ecf700 1 -- 172.21.15.46:0/632741814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69cc067e30 msgr2=0x7f69cc0ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d0ecf700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69cc067e30 0x7f69cc0ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d0ecf700 1 -- 172.21.15.46:0/632741814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc105ce0 msgr2=0x7f69cc1006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d0ecf700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc105ce0 0x7f69cc1006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d0ecf700 1 -- 172.21.15.46:0/632741814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c4009d00 con 0x7f69cc0687f0 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.002+0000 7f69d1ed1700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc105ce0 0x7f69cc1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.003+0000 7f69d0ecf700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69cc0687f0 0x7f69cc1001b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00e9f0 tx=0x7f69bc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.003+0000 7f69c27fc700 1 -- 172.21.15.46:0/632741814 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc00c9c0 con 0x7f69cc0687f0 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.003+0000 7f69d3934700 1 -- 172.21.15.46:0/632741814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69cc1a5cd0 con 0x7f69cc0687f0 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.003+0000 7f69c27fc700 1 -- 172.21.15.46:0/632741814 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69bc010430 con 0x7f69cc0687f0 2024-03-18T20:45:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.003+0000 7f69c27fc700 1 -- 172.21.15.46:0/632741814 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc018680 con 0x7f69cc0687f0 2024-03-18T20:45:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.003+0000 7f69d3934700 1 -- 172.21.15.46:0/632741814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc1a6190 con 0x7f69cc0687f0 2024-03-18T20:45:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.005+0000 7f69c27fc700 1 -- 172.21.15.46:0/632741814 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f69bc010860 con 0x7f69cc0687f0 2024-03-18T20:45:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.006+0000 7f69c27fc700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f69b80a1290 0x7f69b80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.006+0000 7f69d16d0700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f69b80a1290 0x7f69b80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.006+0000 7f69c27fc700 1 -- 172.21.15.46:0/632741814 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f69bc014070 con 0x7f69cc0687f0 2024-03-18T20:45:54.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.007+0000 7f69d16d0700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f69b80a1290 0x7f69b80a3750 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009c60 tx=0x7f69c8009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.007+0000 7f69dfac9b80 1 Processor -- start 2024-03-18T20:45:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.007+0000 7f69dfac9b80 1 -- start start 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69dfac9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 0x56256158ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69dfac9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158f380 0x562561599ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69dfac9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 0x56256159c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69dfac9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625614e8990 con 0x56256159a220 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69dfac9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625614e6350 con 0x56256158ea60 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69dfac9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625614e1c40 con 0x56256158f380 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d16d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 0x56256158ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d1ed1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 0x56256159c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d16d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 0x56256158ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56778/0 (socket says 172.21.15.46:56778) 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d1ed1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 0x56256159c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42028/0 (socket says 172.21.15.46:42028) 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d16d0700 1 -- 172.21.15.46:0/2579949623 learned_addr learned my addr 172.21.15.46:0/2579949623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d0ecf700 1 --2- 172.21.15.46:0/2579949623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158f380 0x562561599ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d16d0700 1 -- 172.21.15.46:0/2579949623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158f380 msgr2=0x562561599ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d16d0700 1 --2- 172.21.15.46:0/2579949623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158f380 0x562561599ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d16d0700 1 -- 172.21.15.46:0/2579949623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 msgr2=0x56256159c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d16d0700 1 --2- 172.21.15.46:0/2579949623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 0x56256159c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d16d0700 1 -- 172.21.15.46:0/2579949623 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625613633b0 con 0x56256158ea60 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d1ed1700 1 --2- 172.21.15.46:0/2579949623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 0x56256159c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.008+0000 7f69d16d0700 1 --2- 172.21.15.46:0/2579949623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 0x56256158ee40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f69c800e4e0 tx=0x7f69c8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f14552963b0373 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.009+0000 7f69b77fe700 1 -- 172.21.15.46:0/2579949623 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c800ee00 con 0x56256158ea60 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.009+0000 7f69b77fe700 1 -- 172.21.15.46:0/2579949623 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69c8021070 con 0x56256158ea60 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.009+0000 7f69b77fe700 1 -- 172.21.15.46:0/2579949623 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c801d7f0 con 0x56256158ea60 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.009+0000 7f69dfac9b80 1 -- 172.21.15.46:0/2579949623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 msgr2=0x56256158ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.009+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/2579949623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 0x56256158ee40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f69c800e4e0 tx=0x7f69c8014000 comp rx=0 tx=0).stop 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.009+0000 7f69dfac9b80 1 -- 172.21.15.46:0/2579949623 shutdown_connections 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.009+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/2579949623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 0x56256159c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.009+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/2579949623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 0x56256158ee40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.009+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/2579949623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158f380 0x562561599ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.009+0000 7f69dfac9b80 1 -- 172.21.15.46:0/2579949623 >> 172.21.15.46:0/2579949623 conn(0x562561505220 msgr2=0x56256158b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.010+0000 7f69dfac9b80 1 -- 172.21.15.46:0/2579949623 shutdown_connections 2024-03-18T20:45:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.010+0000 7f69dfac9b80 1 -- 172.21.15.46:0/2579949623 wait complete. 2024-03-18T20:45:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.010+0000 7f69dfac9b80 1 -- start start 2024-03-18T20:45:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.010+0000 7f69dfac9b80 1 Processor -- start 2024-03-18T20:45:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.010+0000 7f69dfac9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158ea60 0x562561597110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.010+0000 7f69dfac9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256158f380 0x562561597650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.010+0000 7f69dfac9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 0x562561597b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.010+0000 7f69dfac9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256147dd80 con 0x56256158f380 2024-03-18T20:45:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.010+0000 7f69dfac9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256147da20 con 0x56256159a220 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.010+0000 7f69dfac9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625613656a0 con 0x56256158ea60 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69dfac9b80 10 client.0 _ll_register_callbacks cb 0x5625612e93a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69d0ecf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256158f380 0x562561597650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69d1ed1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 0x562561597b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69d16d0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158ea60 0x562561597110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69d1ed1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 0x562561597b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56782/0 (socket says 172.21.15.46:56782) 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69d1ed1700 1 -- 172.21.15.46:0/3484633384 learned_addr learned my addr 172.21.15.46:0/3484633384 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69d1ed1700 1 -- 172.21.15.46:0/3484633384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158ea60 msgr2=0x562561597110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69d1ed1700 1 --2- 172.21.15.46:0/3484633384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158ea60 0x562561597110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69d1ed1700 1 -- 172.21.15.46:0/3484633384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256158f380 msgr2=0x562561597650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69d1ed1700 1 --2- 172.21.15.46:0/3484633384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256158f380 0x562561597650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69d1ed1700 1 -- 172.21.15.46:0/3484633384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc10e550 con 0x56256159a220 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69d1ed1700 1 --2- 172.21.15.46:0/3484633384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 0x562561597b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f69c4007ea0 tx=0x7f69c4007ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a23155842123a9c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:45:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69b4ff9700 1 -- 172.21.15.46:0/3484633384 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c400e040 con 0x56256159a220 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69b4ff9700 1 -- 172.21.15.46:0/3484633384 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f69c400bda0 con 0x56256159a220 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.011+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3484633384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562561582ad0 con 0x56256159a220 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.012+0000 7f69b4ff9700 1 -- 172.21.15.46:0/3484633384 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69c4003a00 con 0x56256159a220 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.012+0000 7f69dfac9b80 10 client.28670 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.012+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3484633384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f69c8017070 con 0x56256159a220 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.012+0000 7f69b4ff9700 1 -- 172.21.15.46:0/3484633384 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562561582ad0 con 0x56256159a220 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.012+0000 7f69b4ff9700 1 -- 172.21.15.46:0/3484633384 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f69c400b100 con 0x56256159a220 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.012+0000 7f69dfac9b80 10 client.28670 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.012+0000 7f69dfac9b80 10 client.28670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.012+0000 7f69dfac9b80 20 client.28670 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.012+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3484633384 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5625615a9fa0 0x5625615ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.012+0000 7f69dfac9b80 4 client.28670 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.012+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3484633384 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625614da700 con 0x5625615a9fa0 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.012+0000 7f69d16d0700 1 --2- 172.21.15.46:0/3484633384 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5625615a9fa0 0x5625615ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69b4ff9700 10 client.28670 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69d16d0700 1 --2- 172.21.15.46:0/3484633384 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5625615a9fa0 0x5625615ac3c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf5b9bcad1374ede server_cookie=1242e95c04c1dcfa in_seq=0 out_seq=0 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69b4ff9700 1 -- 172.21.15.46:0/3484633384 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5625614da700 con 0x5625615a9fa0 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69b4ff9700 10 client.28670 handle_command_reply: tid=0 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69dfac9b80 1 client.28670 shutdown 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3484633384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 msgr2=0x562561597b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3484633384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 0x562561597b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f69c4007ea0 tx=0x7f69c4007ed0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3484633384 shutdown_connections 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3484633384 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5625615a9fa0 0x5625615ac3c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3484633384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 0x562561597b90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3484633384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158ea60 0x562561597110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69b4ff9700 0 client.28670 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3484633384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256158f380 0x562561597650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.013+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3484633384 >> 172.21.15.46:0/3484633384 conn(0x562561505220 msgr2=0x56256158ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:54.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.014+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3484633384 shutdown_connections 2024-03-18T20:45:54.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.014+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3484633384 wait complete. 2024-03-18T20:45:54.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.014+0000 7f69dfac9b80 20 client.28670 trim_cache size 0 max 16384 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69dfac9b80 1 Processor -- start 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69dfac9b80 1 -- start start 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69dfac9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 0x5625615a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69dfac9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158f380 0x562561596a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69dfac9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 0x5625615a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69dfac9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625615a1380 con 0x56256159a220 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69dfac9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562561599b80 con 0x56256158ea60 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69dfac9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562561595680 con 0x56256158f380 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69d16d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 0x5625615a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69d0ecf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158f380 0x562561596a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69d1ed1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 0x5625615a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69d16d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 0x5625615a5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56792/0 (socket says 172.21.15.46:56792) 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.033+0000 7f69d16d0700 1 -- 172.21.15.46:0/3826010724 learned_addr learned my addr 172.21.15.46:0/3826010724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.034+0000 7f69d16d0700 1 -- 172.21.15.46:0/3826010724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158f380 msgr2=0x562561596a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.034+0000 7f69d16d0700 1 --2- 172.21.15.46:0/3826010724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158f380 0x562561596a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.034+0000 7f69d16d0700 1 -- 172.21.15.46:0/3826010724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 msgr2=0x5625615a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.034+0000 7f69d16d0700 1 --2- 172.21.15.46:0/3826010724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 0x5625615a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.034+0000 7f69d16d0700 1 -- 172.21.15.46:0/3826010724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56256130fb60 con 0x56256158ea60 2024-03-18T20:45:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.034+0000 7f69d0ecf700 1 --2- 172.21.15.46:0/3826010724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158f380 0x562561596a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.034+0000 7f69d1ed1700 1 --2- 172.21.15.46:0/3826010724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 0x5625615a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.034+0000 7f69d16d0700 1 --2- 172.21.15.46:0/3826010724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 0x5625615a5fa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009ec0 tx=0x7f69c8005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1687ff4c9af44a60 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.034+0000 7f69b7fff700 1 -- 172.21.15.46:0/3826010724 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c800ecc0 con 0x56256158ea60 2024-03-18T20:45:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.034+0000 7f69b7fff700 1 -- 172.21.15.46:0/3826010724 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69c8033170 con 0x56256158ea60 2024-03-18T20:45:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.034+0000 7f69b7fff700 1 -- 172.21.15.46:0/3826010724 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8005680 con 0x56256158ea60 2024-03-18T20:45:54.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.035+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3826010724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 msgr2=0x5625615a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.035+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3826010724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 0x5625615a5fa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009ec0 tx=0x7f69c8005ce0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.035+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3826010724 shutdown_connections 2024-03-18T20:45:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.035+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3826010724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256159a220 0x5625615a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.035+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3826010724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256158ea60 0x5625615a5fa0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.035+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3826010724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158f380 0x562561596a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.035+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3826010724 >> 172.21.15.46:0/3826010724 conn(0x562561505220 msgr2=0x562561589750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.035+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3826010724 shutdown_connections 2024-03-18T20:45:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.035+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3826010724 wait complete. 2024-03-18T20:45:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.035+0000 7f69dfac9b80 1 -- start start 2024-03-18T20:45:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.035+0000 7f69dfac9b80 1 Processor -- start 2024-03-18T20:45:54.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69dfac9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158ea60 0x5625615a0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69dfac9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256158f380 0x5625615a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69dfac9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 0x56256159ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69dfac9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256159f410 con 0x56256158f380 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69dfac9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256159f580 con 0x56256159a220 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69dfac9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256159f740 con 0x56256158ea60 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69dfac9b80 10 client.0 _ll_register_callbacks cb 0x7f69c4009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d0ecf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256158f380 0x5625615a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d1ed1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 0x56256159ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d1ed1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 0x56256159ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56798/0 (socket says 172.21.15.46:56798) 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d0ecf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256158f380 0x5625615a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42056/0 (socket says 172.21.15.46:42056) 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d1ed1700 1 -- 172.21.15.46:0/3832892749 learned_addr learned my addr 172.21.15.46:0/3832892749 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d16d0700 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158ea60 0x5625615a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d1ed1700 1 -- 172.21.15.46:0/3832892749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158ea60 msgr2=0x5625615a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:54.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d1ed1700 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158ea60 0x5625615a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d1ed1700 1 -- 172.21.15.46:0/3832892749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256158f380 msgr2=0x5625615a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:45:54.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d1ed1700 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256158f380 0x5625615a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d1ed1700 1 -- 172.21.15.46:0/3832892749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc1a6190 con 0x56256159a220 2024-03-18T20:45:54.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d0ecf700 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256158f380 0x5625615a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:45:54.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69d1ed1700 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 0x56256159ee30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f69c4006260 tx=0x7f69c4007510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d272e9d7a2e44726 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:45:54.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.036+0000 7f69b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:45:54.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69b4ff9700 1 -- 172.21.15.46:0/3832892749 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c400e040 con 0x56256159a220 2024-03-18T20:45:54.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3832892749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69c8021070 con 0x56256159a220 2024-03-18T20:45:54.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69b4ff9700 1 -- 172.21.15.46:0/3832892749 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f69c401d7f0 con 0x56256159a220 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69b4ff9700 1 -- 172.21.15.46:0/3832892749 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69c400baa0 con 0x56256159a220 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69dfac9b80 10 client.28676 fetch_fsmap learned FSMap version 49 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3832892749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f69c8017070 con 0x56256159a220 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69b4ff9700 1 -- 172.21.15.46:0/3832892749 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69c8021070 con 0x56256159a220 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69b4ff9700 1 -- 172.21.15.46:0/3832892749 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f69c4005040 con 0x56256159a220 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69dfac9b80 10 client.28676 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69dfac9b80 10 client.28676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69dfac9b80 20 client.28676 populate_metadata read hostname 'smithi046' 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5625615fed70 0x562561601190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69dfac9b80 4 client.28676 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3832892749 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562561330cf0 con 0x5625615fed70 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.037+0000 7f69d16d0700 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5625615fed70 0x562561601190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69b4ff9700 10 client.28676 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:54.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69d16d0700 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5625615fed70 0x562561601190 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1663fae32249073c server_cookie=a13ac438ad685467 in_seq=0 out_seq=0 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69b4ff9700 1 -- 172.21.15.46:0/3832892749 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562561330cf0 con 0x5625615fed70 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69b4ff9700 10 client.28676 handle_command_reply: tid=0 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69dfac9b80 1 client.28676 shutdown 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3832892749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 msgr2=0x56256159ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 0x56256159ee30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f69c4006260 tx=0x7f69c4007510 comp rx=0 tx=0).stop 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3832892749 shutdown_connections 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5625615fed70 0x562561601190 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56256159a220 0x56256159ee30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56256158ea60 0x5625615a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69b4ff9700 0 client.28676 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69dfac9b80 1 --2- 172.21.15.46:0/3832892749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56256158f380 0x5625615a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3832892749 >> 172.21.15.46:0/3832892749 conn(0x562561505220 msgr2=0x5625615a5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:54.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3832892749 shutdown_connections 2024-03-18T20:45:54.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69dfac9b80 1 -- 172.21.15.46:0/3832892749 wait complete. 2024-03-18T20:45:54.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.038+0000 7f69dfac9b80 20 client.28676 trim_cache size 0 max 16384 2024-03-18T20:45:54.048 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:45:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:45:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:45:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: "5a86dfc4-ef71-424a-af7e-454561037ec0": { 2024-03-18T20:45:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:45:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "5a86dfc4-ef71-424a-af7e-454561037ec0", 2024-03-18T20:45:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:45:54.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:45:54.049 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:45:54.049 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:45:54.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.039+0000 7f69d3934700 1 -- 172.21.15.46:0/632741814 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f69b80a1290 msgr2=0x7f69b80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:54.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.039+0000 7f69d3934700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f69b80a1290 0x7f69b80a3750 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009c60 tx=0x7f69c8009260 comp rx=0 tx=0).stop 2024-03-18T20:45:54.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.039+0000 7f69d3934700 1 -- 172.21.15.46:0/632741814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69cc0687f0 msgr2=0x7f69cc1001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:45:54.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.039+0000 7f69d3934700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69cc0687f0 0x7f69cc1001b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00e9f0 tx=0x7f69bc00ed00 comp rx=0 tx=0).stop 2024-03-18T20:45:54.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.039+0000 7f69d3934700 1 -- 172.21.15.46:0/632741814 shutdown_connections 2024-03-18T20:45:54.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.039+0000 7f69d3934700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f69b80a1290 0x7f69b80a3750 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.039+0000 7f69d3934700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69cc105ce0 0x7f69cc1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.039+0000 7f69d3934700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69cc067e30 0x7f69cc0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.039+0000 7f69d3934700 1 --2- 172.21.15.46:0/632741814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69cc0687f0 0x7f69cc1001b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:45:54.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.039+0000 7f69d3934700 1 -- 172.21.15.46:0/632741814 >> 172.21.15.46:0/632741814 conn(0x7f69cc076d40 msgr2=0x7f69cc0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:45:54.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.039+0000 7f69d3934700 1 -- 172.21.15.46:0/632741814 shutdown_connections 2024-03-18T20:45:54.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:45:54.039+0000 7f69d3934700 1 -- 172.21.15.46:0/632741814 wait complete. 2024-03-18T20:45:54.055 INFO:tasks.cephfs.filesystem:scrub status for tag:5a86dfc4-ef71-424a-af7e-454561037ec0 - {'path': '/', 'tag': '5a86dfc4-ef71-424a-af7e-454561037ec0', 'options': 'recursive,force'} 2024-03-18T20:45:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:54 smithi155 ceph-mon[48344]: pgmap v303: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-03-18T20:45:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:54 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:45:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:54 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:54 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:54 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:54 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:54 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:54 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:54 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:54 smithi046 ceph-mon[43495]: pgmap v303: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-03-18T20:45:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:54 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:45:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:54 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:54 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:54 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:54 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:54 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:54 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:54 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:54.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 361910 3.69 MB/sec execute 409 sec latency 1404.200 ms 2024-03-18T20:45:54.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:54 smithi073 ceph-mon[47384]: pgmap v303: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-03-18T20:45:54.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:54 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:45:54.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:54 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:54.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:54 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:54.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:54 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:54.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:54 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:54.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:54 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:54.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:54 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:54.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:54 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:55.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:55.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:55.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:55.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:45:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:55 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:55.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:55.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:45:55.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:55 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.711 INFO:tasks.workunit.client.0.smithi046.stdout: 1 363217 3.70 MB/sec execute 410 sec latency 20.185 ms 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:55.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:45:55.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:55.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:55 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:45:56.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:56 smithi155 ceph-mon[48344]: pgmap v304: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-18T20:45:56.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:56 smithi046 ceph-mon[43495]: pgmap v304: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-18T20:45:56.712 INFO:tasks.workunit.client.0.smithi046.stdout: 1 363483 3.69 MB/sec execute 411 sec latency 697.287 ms 2024-03-18T20:45:56.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:56 smithi073 ceph-mon[47384]: pgmap v304: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-18T20:45:57.712 INFO:tasks.workunit.client.0.smithi046.stdout: 1 363483 3.68 MB/sec execute 412 sec latency 1697.407 ms 2024-03-18T20:45:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:45:58 smithi155 ceph-mon[48344]: pgmap v305: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-18T20:45:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:45:58 smithi046 ceph-mon[43495]: pgmap v305: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-18T20:45:58.712 INFO:tasks.workunit.client.0.smithi046.stdout: 1 363483 3.67 MB/sec execute 413 sec latency 2697.527 ms 2024-03-18T20:45:58.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:45:58 smithi073 ceph-mon[47384]: pgmap v305: 97 pgs: 97 active+clean; 137 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-03-18T20:45:59.712 INFO:tasks.workunit.client.0.smithi046.stdout: 1 364711 3.68 MB/sec execute 414 sec latency 3179.575 ms 2024-03-18T20:46:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:00 smithi155 ceph-mon[48344]: pgmap v306: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-03-18T20:46:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:00 smithi046 ceph-mon[43495]: pgmap v306: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-03-18T20:46:00.712 INFO:tasks.workunit.client.0.smithi046.stdout: 1 366006 3.69 MB/sec execute 415 sec latency 60.246 ms 2024-03-18T20:46:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:00 smithi073 ceph-mon[47384]: pgmap v306: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-03-18T20:46:01.712 INFO:tasks.workunit.client.0.smithi046.stdout: 1 366006 3.68 MB/sec execute 416 sec latency 1060.370 ms 2024-03-18T20:46:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:02 smithi155 ceph-mon[48344]: pgmap v307: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-03-18T20:46:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:02 smithi046 ceph-mon[43495]: pgmap v307: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-03-18T20:46:02.712 INFO:tasks.workunit.client.0.smithi046.stdout: 1 366561 3.67 MB/sec execute 417 sec latency 1400.264 ms 2024-03-18T20:46:02.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:02 smithi073 ceph-mon[47384]: pgmap v307: 97 pgs: 97 active+clean; 140 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-03-18T20:46:03.712 INFO:tasks.workunit.client.0.smithi046.stdout: 1 367032 3.67 MB/sec execute 418 sec latency 753.706 ms 2024-03-18T20:46:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.992+0000 7fbb3aadb700 1 -- 172.21.15.46:0/2184613804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34068aa0 msgr2=0x7fbb3410dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.992+0000 7fbb3aadb700 1 --2- 172.21.15.46:0/2184613804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34068aa0 0x7fbb3410dab0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00b0a0 tx=0x7fbb2c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:46:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.992+0000 7fbb3aadb700 1 -- 172.21.15.46:0/2184613804 shutdown_connections 2024-03-18T20:46:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.992+0000 7fbb3aadb700 1 --2- 172.21.15.46:0/2184613804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb34068aa0 0x7fbb3410dab0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.992+0000 7fbb3aadb700 1 --2- 172.21.15.46:0/2184613804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb34106e10 0x7fbb341071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.992+0000 7fbb3aadb700 1 --2- 172.21.15.46:0/2184613804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb340680e0 0x7fbb34068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.992+0000 7fbb3aadb700 1 -- 172.21.15.46:0/2184613804 >> 172.21.15.46:0/2184613804 conn(0x7fbb34076d50 msgr2=0x7fbb34077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.992+0000 7fbb3aadb700 1 -- 172.21.15.46:0/2184613804 shutdown_connections 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.992+0000 7fbb3aadb700 1 -- 172.21.15.46:0/2184613804 wait complete. 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.992+0000 7fbb3aadb700 1 Processor -- start 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb3aadb700 1 -- start start 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb3aadb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340680e0 0x7fbb341971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb3aadb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb34068aa0 0x7fbb341976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb3aadb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb34106e10 0x7fbb3419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb3aadb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3419c0d0 con 0x7fbb340680e0 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb3aadb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3419c240 con 0x7fbb34068aa0 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb3aadb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3419c400 con 0x7fbb34106e10 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb38877700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340680e0 0x7fbb341971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb38877700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340680e0 0x7fbb341971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50628/0 (socket says 172.21.15.46:50628) 2024-03-18T20:46:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb38877700 1 -- 172.21.15.46:0/1571800251 learned_addr learned my addr 172.21.15.46:0/1571800251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb39078700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb34106e10 0x7fbb3419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb33fff700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb34068aa0 0x7fbb341976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb38877700 1 -- 172.21.15.46:0/1571800251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb34106e10 msgr2=0x7fbb3419ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb38877700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb34106e10 0x7fbb3419ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb38877700 1 -- 172.21.15.46:0/1571800251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb34068aa0 msgr2=0x7fbb341976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb38877700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb34068aa0 0x7fbb341976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb38877700 1 -- 172.21.15.46:0/1571800251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb28009520 con 0x7fbb340680e0 2024-03-18T20:46:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb33fff700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb34068aa0 0x7fbb341976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.993+0000 7fbb38877700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340680e0 0x7fbb341971b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400e9a0 tx=0x7fbb2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.994+0000 7fbb31ffb700 1 -- 172.21.15.46:0/1571800251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24009790 con 0x7fbb340680e0 2024-03-18T20:46:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.994+0000 7fbb31ffb700 1 -- 172.21.15.46:0/1571800251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb2400cbd0 con 0x7fbb340680e0 2024-03-18T20:46:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.994+0000 7fbb3aadb700 1 -- 172.21.15.46:0/1571800251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c009d00 con 0x7fbb340680e0 2024-03-18T20:46:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.994+0000 7fbb31ffb700 1 -- 172.21.15.46:0/1571800251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24018680 con 0x7fbb340680e0 2024-03-18T20:46:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.994+0000 7fbb3aadb700 1 -- 172.21.15.46:0/1571800251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3419c8d0 con 0x7fbb340680e0 2024-03-18T20:46:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.996+0000 7fbb31ffb700 1 -- 172.21.15.46:0/1571800251 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fbb2400cd40 con 0x7fbb340680e0 2024-03-18T20:46:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.997+0000 7fbb31ffb700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbb1c0a1310 0x7fbb1c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.997+0000 7fbb31ffb700 1 -- 172.21.15.46:0/1571800251 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fbb24014070 con 0x7fbb340680e0 2024-03-18T20:46:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.997+0000 7fbb33fff700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbb1c0a1310 0x7fbb1c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb33fff700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbb1c0a1310 0x7fbb1c0a37d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb34198700 tx=0x7fbb28009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb46c70b80 1 Processor -- start 2024-03-18T20:46:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb46c70b80 1 -- start start 2024-03-18T20:46:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb46c70b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2570 0x564e26be2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb46c70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26be2e90 0x564e26bed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb46c70b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26beddb0 0x564e26bf01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb46c70b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26b3c930 con 0x564e26beddb0 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb46c70b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26b3a2f0 con 0x564e26be2570 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb46c70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26b35be0 con 0x564e26be2e90 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb39078700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26beddb0 0x564e26bf01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb39078700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26beddb0 0x564e26bf01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50636/0 (socket says 172.21.15.46:50636) 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb39078700 1 -- 172.21.15.46:0/1974713002 learned_addr learned my addr 172.21.15.46:0/1974713002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb33fff700 1 --2- 172.21.15.46:0/1974713002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26be2e90 0x564e26bed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.998+0000 7fbb38877700 1 --2- 172.21.15.46:0/1974713002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2570 0x564e26be2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.999+0000 7fbb39078700 1 -- 172.21.15.46:0/1974713002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26be2e90 msgr2=0x564e26bed870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.999+0000 7fbb39078700 1 --2- 172.21.15.46:0/1974713002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26be2e90 0x564e26bed870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.999+0000 7fbb39078700 1 -- 172.21.15.46:0/1974713002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2570 msgr2=0x564e26be2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.999+0000 7fbb39078700 1 --2- 172.21.15.46:0/1974713002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2570 0x564e26be2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.999+0000 7fbb39078700 1 -- 172.21.15.46:0/1974713002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e269b7350 con 0x564e26beddb0 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.999+0000 7fbb39078700 1 --2- 172.21.15.46:0/1974713002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26beddb0 0x564e26bf01a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c015000 tx=0x7fbb2c009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a5c826c558336a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.999+0000 7fbb1affd700 1 -- 172.21.15.46:0/1974713002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00e040 con 0x564e26beddb0 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.999+0000 7fbb1affd700 1 -- 172.21.15.46:0/1974713002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb2c007d30 con 0x564e26beddb0 2024-03-18T20:46:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:03.999+0000 7fbb1affd700 1 -- 172.21.15.46:0/1974713002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c007740 con 0x564e26beddb0 2024-03-18T20:46:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.000+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1974713002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26beddb0 msgr2=0x564e26bf01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.000+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/1974713002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26beddb0 0x564e26bf01a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c015000 tx=0x7fbb2c009c40 comp rx=0 tx=0).stop 2024-03-18T20:46:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.000+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1974713002 shutdown_connections 2024-03-18T20:46:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.000+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/1974713002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26beddb0 0x564e26bf01a0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.000+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/1974713002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2570 0x564e26be2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.000+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/1974713002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26be2e90 0x564e26bed870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.000+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1974713002 >> 172.21.15.46:0/1974713002 conn(0x564e26b591c0 msgr2=0x564e26bdea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.000+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1974713002 shutdown_connections 2024-03-18T20:46:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.000+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1974713002 wait complete. 2024-03-18T20:46:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.000+0000 7fbb46c70b80 1 -- start start 2024-03-18T20:46:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.000+0000 7fbb46c70b80 1 Processor -- start 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb46c70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26be2570 0x564e26bed790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb46c70b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 0x564e26beaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb46c70b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26beddb0 0x564e26beb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb46c70b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26ad1d20 con 0x564e26be2e90 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb46c70b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26ad19c0 con 0x564e26beddb0 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb46c70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e269b9640 con 0x564e26be2570 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb46c70b80 10 client.0 _ll_register_callbacks cb 0x564e2693d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 0x564e26beaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 0x564e26beaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50650/0 (socket says 172.21.15.46:50650) 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb33fff700 1 -- 172.21.15.46:0/1685455687 learned_addr learned my addr 172.21.15.46:0/1685455687 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb39078700 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26beddb0 0x564e26beb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb38877700 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26be2570 0x564e26bed790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb33fff700 1 -- 172.21.15.46:0/1685455687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26be2570 msgr2=0x564e26bed790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb33fff700 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26be2570 0x564e26bed790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb33fff700 1 -- 172.21.15.46:0/1685455687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26beddb0 msgr2=0x564e26beb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb33fff700 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26beddb0 0x564e26beb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb33fff700 1 -- 172.21.15.46:0/1685455687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3410e440 con 0x564e26be2e90 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.001+0000 7fbb38877700 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26be2570 0x564e26bed790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb33fff700 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 0x564e26beaa10 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbb28012c40 tx=0x7fbb280047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329b7b4231051e49 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb13fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb13fff700 1 -- 172.21.15.46:0/1685455687 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2801ea40 con 0x564e26be2e90 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb13fff700 1 -- 172.21.15.46:0/1685455687 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fbb28018070 con 0x564e26be2e90 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1685455687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e26bd66c0 con 0x564e26be2e90 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb13fff700 1 -- 172.21.15.46:0/1685455687 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb2801eba0 con 0x564e26be2e90 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb46c70b80 10 client.19059 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1685455687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564e26963b00 con 0x564e26be2e90 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb13fff700 1 -- 172.21.15.46:0/1685455687 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e26bd66c0 con 0x564e26be2e90 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb13fff700 1 -- 172.21.15.46:0/1685455687 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbb2801f420 con 0x564e26be2e90 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb46c70b80 10 client.19059 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb46c70b80 10 client.19059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb46c70b80 20 client.19059 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564e26bfdb10 0x564e26bfff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb46c70b80 4 client.19059 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.002+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1685455687 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e26b2e6a0 con 0x564e26bfdb10 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.003+0000 7fbb38877700 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564e26bfdb10 0x564e26bfff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.003+0000 7fbb13fff700 10 client.19059 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.003+0000 7fbb38877700 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564e26bfdb10 0x564e26bfff30 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ffe34c854a2a81 server_cookie=17bbec330630b9bd in_seq=0 out_seq=0 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.003+0000 7fbb13fff700 1 -- 172.21.15.46:0/1685455687 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564e26b2e6a0 con 0x564e26bfdb10 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb13fff700 10 client.19059 handle_command_reply: tid=0 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb46c70b80 1 client.19059 shutdown 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1685455687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 msgr2=0x564e26beaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 0x564e26beaa10 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbb28012c40 tx=0x7fbb280047c0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1685455687 shutdown_connections 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564e26bfdb10 0x564e26bfff30 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26beddb0 0x564e26beb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26be2570 0x564e26bed790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb13fff700 0 client.19059 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/1685455687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 0x564e26beaa10 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1685455687 >> 172.21.15.46:0/1685455687 conn(0x564e26b591c0 msgr2=0x564e26bde650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1685455687 shutdown_connections 2024-03-18T20:46:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb46c70b80 1 -- 172.21.15.46:0/1685455687 wait complete. 2024-03-18T20:46:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.004+0000 7fbb46c70b80 20 client.19059 trim_cache size 0 max 16384 2024-03-18T20:46:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb46c70b80 1 Processor -- start 2024-03-18T20:46:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb46c70b80 1 -- start start 2024-03-18T20:46:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb46c70b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2570 0x564e26be54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb46c70b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2e90 0x564e26bfff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb46c70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26beddb0 0x564e26bf49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb46c70b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26be8f60 con 0x564e26be2570 2024-03-18T20:46:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb46c70b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26be6870 con 0x564e26be2e90 2024-03-18T20:46:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb46c70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26bd6200 con 0x564e26beddb0 2024-03-18T20:46:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb38877700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2570 0x564e26be54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb38877700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2570 0x564e26be54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50656/0 (socket says 172.21.15.46:50656) 2024-03-18T20:46:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb38877700 1 -- 172.21.15.46:0/4005127559 learned_addr learned my addr 172.21.15.46:0/4005127559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb39078700 1 --2- 172.21.15.46:0/4005127559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26beddb0 0x564e26bf49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb33fff700 1 --2- 172.21.15.46:0/4005127559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2e90 0x564e26bfff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb38877700 1 -- 172.21.15.46:0/4005127559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26beddb0 msgr2=0x564e26bf49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb38877700 1 --2- 172.21.15.46:0/4005127559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26beddb0 0x564e26bf49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.020+0000 7fbb38877700 1 -- 172.21.15.46:0/4005127559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2e90 msgr2=0x564e26bfff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.021+0000 7fbb38877700 1 --2- 172.21.15.46:0/4005127559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2e90 0x564e26bfff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.021+0000 7fbb38877700 1 -- 172.21.15.46:0/4005127559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e26afbe40 con 0x564e26be2570 2024-03-18T20:46:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.021+0000 7fbb38877700 1 --2- 172.21.15.46:0/4005127559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2570 0x564e26be54e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbb240964f0 tx=0x7fbb24005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18357e965f75dce server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.021+0000 7fbb1b7fe700 1 -- 172.21.15.46:0/4005127559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb240a2470 con 0x564e26be2570 2024-03-18T20:46:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.021+0000 7fbb1b7fe700 1 -- 172.21.15.46:0/4005127559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb24098390 con 0x564e26be2570 2024-03-18T20:46:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.021+0000 7fbb1b7fe700 1 -- 172.21.15.46:0/4005127559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb240a1fe0 con 0x564e26be2570 2024-03-18T20:46:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.022+0000 7fbb46c70b80 1 -- 172.21.15.46:0/4005127559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2570 msgr2=0x564e26be54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.022+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/4005127559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2570 0x564e26be54e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbb240964f0 tx=0x7fbb24005b60 comp rx=0 tx=0).stop 2024-03-18T20:46:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.022+0000 7fbb46c70b80 1 -- 172.21.15.46:0/4005127559 shutdown_connections 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.022+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/4005127559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2570 0x564e26be54e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.022+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/4005127559 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26beddb0 0x564e26bf49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.022+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/4005127559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2e90 0x564e26bfff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.022+0000 7fbb46c70b80 1 -- 172.21.15.46:0/4005127559 >> 172.21.15.46:0/4005127559 conn(0x564e26b591c0 msgr2=0x564e26bda620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.022+0000 7fbb46c70b80 1 -- 172.21.15.46:0/4005127559 shutdown_connections 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.022+0000 7fbb46c70b80 1 -- 172.21.15.46:0/4005127559 wait complete. 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.022+0000 7fbb46c70b80 1 -- start start 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.022+0000 7fbb46c70b80 1 Processor -- start 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb46c70b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2570 0x564e26be7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb46c70b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 0x564e26be8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb46c70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26beddb0 0x564e26be6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb46c70b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26be8a30 con 0x564e26be2e90 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb46c70b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26be71c0 con 0x564e26be2570 2024-03-18T20:46:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb46c70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e26be7380 con 0x564e26beddb0 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb46c70b80 10 client.0 _ll_register_callbacks cb 0x7fbb280127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 0x564e26be8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb33fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 0x564e26be8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50672/0 (socket says 172.21.15.46:50672) 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb33fff700 1 -- 172.21.15.46:0/3496061857 learned_addr learned my addr 172.21.15.46:0/3496061857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb38877700 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2570 0x564e26be7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb39078700 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26beddb0 0x564e26be6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb33fff700 1 -- 172.21.15.46:0/3496061857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26beddb0 msgr2=0x564e26be6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb33fff700 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26beddb0 0x564e26be6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb33fff700 1 -- 172.21.15.46:0/3496061857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2570 msgr2=0x564e26be7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb33fff700 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2570 0x564e26be7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb33fff700 1 -- 172.21.15.46:0/3496061857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3419c8d0 con 0x564e26be2e90 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb38877700 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2570 0x564e26be7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb33fff700 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 0x564e26be8320 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fbb2801f850 tx=0x7fbb2801f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4c2c32a6a45881 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb13fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb13fff700 1 -- 172.21.15.46:0/3496061857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb28004020 con 0x564e26be2e90 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb13fff700 1 -- 172.21.15.46:0/3496061857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fbb28018040 con 0x564e26be2e90 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.023+0000 7fbb46c70b80 1 -- 172.21.15.46:0/3496061857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb2c007d30 con 0x564e26be2e90 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.024+0000 7fbb13fff700 1 -- 172.21.15.46:0/3496061857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb2801fb80 con 0x564e26be2e90 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.024+0000 7fbb46c70b80 10 client.19065 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.024+0000 7fbb46c70b80 1 -- 172.21.15.46:0/3496061857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564e269bbc70 con 0x564e26be2e90 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.024+0000 7fbb13fff700 1 -- 172.21.15.46:0/3496061857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb2c007d30 con 0x564e26be2e90 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.024+0000 7fbb13fff700 1 -- 172.21.15.46:0/3496061857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbb28027ab0 con 0x564e26be2e90 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.024+0000 7fbb46c70b80 10 client.19065 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.024+0000 7fbb46c70b80 10 client.19065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.024+0000 7fbb46c70b80 20 client.19065 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.024+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564e26c527c0 0x564e26c54be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.024+0000 7fbb46c70b80 4 client.19065 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.024+0000 7fbb46c70b80 1 -- 172.21.15.46:0/3496061857 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564e26984c90 con 0x564e26c527c0 2024-03-18T20:46:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.024+0000 7fbb38877700 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564e26c527c0 0x564e26c54be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.025+0000 7fbb38877700 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564e26c527c0 0x564e26c54be0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e759a29fbe375a1 server_cookie=5bbd5183e57180db in_seq=0 out_seq=0 2024-03-18T20:46:04.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.025+0000 7fbb13fff700 10 client.19065 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.051+0000 7fbb13fff700 1 -- 172.21.15.46:0/3496061857 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564e26984c90 con 0x564e26c527c0 2024-03-18T20:46:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.051+0000 7fbb13fff700 10 client.19065 handle_command_reply: tid=0 2024-03-18T20:46:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.051+0000 7fbb46c70b80 1 client.19065 shutdown 2024-03-18T20:46:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.051+0000 7fbb46c70b80 1 -- 172.21.15.46:0/3496061857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 msgr2=0x564e26be8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.052+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 0x564e26be8320 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fbb2801f850 tx=0x7fbb2801f4a0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.052+0000 7fbb46c70b80 1 -- 172.21.15.46:0/3496061857 shutdown_connections 2024-03-18T20:46:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.052+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564e26c527c0 0x564e26c54be0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.052+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e26beddb0 0x564e26be6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.052+0000 7fbb13fff700 0 client.19065 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.052+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564e26be2570 0x564e26be7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.052+0000 7fbb46c70b80 1 --2- 172.21.15.46:0/3496061857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564e26be2e90 0x564e26be8320 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.052+0000 7fbb46c70b80 1 -- 172.21.15.46:0/3496061857 >> 172.21.15.46:0/3496061857 conn(0x564e26b591c0 msgr2=0x564e26be5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.052+0000 7fbb46c70b80 1 -- 172.21.15.46:0/3496061857 shutdown_connections 2024-03-18T20:46:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.052+0000 7fbb46c70b80 1 -- 172.21.15.46:0/3496061857 wait complete. 2024-03-18T20:46:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.052+0000 7fbb46c70b80 20 client.19065 trim_cache size 0 max 16384 2024-03-18T20:46:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:46:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:46:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:46:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:46:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.053+0000 7fbb3aadb700 1 -- 172.21.15.46:0/1571800251 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbb1c0a1310 msgr2=0x7fbb1c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.053+0000 7fbb3aadb700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbb1c0a1310 0x7fbb1c0a37d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb34198700 tx=0x7fbb28009260 comp rx=0 tx=0).stop 2024-03-18T20:46:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.053+0000 7fbb3aadb700 1 -- 172.21.15.46:0/1571800251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340680e0 msgr2=0x7fbb341971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.053+0000 7fbb3aadb700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340680e0 0x7fbb341971b0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400e9a0 tx=0x7fbb2400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.053+0000 7fbb3aadb700 1 -- 172.21.15.46:0/1571800251 shutdown_connections 2024-03-18T20:46:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.053+0000 7fbb3aadb700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbb1c0a1310 0x7fbb1c0a37d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.053+0000 7fbb3aadb700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb340680e0 0x7fbb341971b0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.053+0000 7fbb3aadb700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbb34106e10 0x7fbb3419ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.053+0000 7fbb3aadb700 1 --2- 172.21.15.46:0/1571800251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb34068aa0 0x7fbb341976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.053+0000 7fbb3aadb700 1 -- 172.21.15.46:0/1571800251 >> 172.21.15.46:0/1571800251 conn(0x7fbb34076d50 msgr2=0x7fbb340fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.053+0000 7fbb3aadb700 1 -- 172.21.15.46:0/1571800251 shutdown_connections 2024-03-18T20:46:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:04.053+0000 7fbb3aadb700 1 -- 172.21.15.46:0/1571800251 wait complete. 2024-03-18T20:46:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:04 smithi155 ceph-mon[48344]: pgmap v308: 97 pgs: 97 active+clean; 144 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-03-18T20:46:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:04 smithi155 ceph-mon[48344]: osdmap e89: 12 total, 12 up, 12 in 2024-03-18T20:46:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:04 smithi046 ceph-mon[43495]: pgmap v308: 97 pgs: 97 active+clean; 144 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-03-18T20:46:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:04 smithi046 ceph-mon[43495]: osdmap e89: 12 total, 12 up, 12 in 2024-03-18T20:46:04.712 INFO:tasks.workunit.client.0.smithi046.stdout: 1 367032 3.66 MB/sec execute 419 sec latency 1753.799 ms 2024-03-18T20:46:04.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:04 smithi073 ceph-mon[47384]: pgmap v308: 97 pgs: 97 active+clean; 144 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-03-18T20:46:04.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:04 smithi073 ceph-mon[47384]: osdmap e89: 12 total, 12 up, 12 in 2024-03-18T20:46:05.712 INFO:tasks.workunit.client.0.smithi046.stdout: 1 368999 3.68 MB/sec execute 420 sec latency 1900.026 ms 2024-03-18T20:46:06.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:05 smithi155 ceph-mon[48344]: pgmap v310: 97 pgs: 97 active+clean; 144 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-03-18T20:46:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:05 smithi046 ceph-mon[43495]: pgmap v310: 97 pgs: 97 active+clean; 144 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-03-18T20:46:06.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:05 smithi073 ceph-mon[47384]: pgmap v310: 97 pgs: 97 active+clean; 144 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-03-18T20:46:06.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 370163 3.68 MB/sec execute 421 sec latency 18.944 ms 2024-03-18T20:46:07.712 INFO:tasks.workunit.client.0.smithi046.stdout: 1 370575 3.68 MB/sec execute 422 sec latency 602.265 ms 2024-03-18T20:46:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:08 smithi155 ceph-mon[48344]: pgmap v311: 97 pgs: 97 active+clean; 144 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-03-18T20:46:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:08 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:46:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:08 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:46:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:08 smithi046 ceph-mon[43495]: pgmap v311: 97 pgs: 97 active+clean; 144 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-03-18T20:46:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:08 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:46:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:08 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:46:08.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 370575 3.67 MB/sec execute 423 sec latency 1602.377 ms 2024-03-18T20:46:08.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:08 smithi073 ceph-mon[47384]: pgmap v311: 97 pgs: 97 active+clean; 144 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-03-18T20:46:08.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:08 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:46:08.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:08 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:46:09.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:09 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:46:09.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:09 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:46:09.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:09 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:46:09.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:09 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:46:09.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:09 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:46:09.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:09 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:46:09.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 370575 3.66 MB/sec execute 424 sec latency 2602.490 ms 2024-03-18T20:46:09.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:09 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:46:09.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:09 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:46:09.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:09 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:46:10.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:10 smithi155 ceph-mon[48344]: pgmap v312: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-03-18T20:46:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:10 smithi155 ceph-mon[48344]: osdmap e90: 12 total, 12 up, 12 in 2024-03-18T20:46:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:10 smithi046 ceph-mon[43495]: pgmap v312: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-03-18T20:46:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:10 smithi046 ceph-mon[43495]: osdmap e90: 12 total, 12 up, 12 in 2024-03-18T20:46:10.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 371455 3.66 MB/sec execute 425 sec latency 3226.628 ms 2024-03-18T20:46:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:10 smithi073 ceph-mon[47384]: pgmap v312: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-03-18T20:46:10.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:10 smithi073 ceph-mon[47384]: osdmap e90: 12 total, 12 up, 12 in 2024-03-18T20:46:11.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 372957 3.68 MB/sec execute 426 sec latency 15.927 ms 2024-03-18T20:46:12.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:12 smithi155 ceph-mon[48344]: pgmap v314: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 909 KiB/s wr, 97 op/s 2024-03-18T20:46:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:12 smithi046 ceph-mon[43495]: pgmap v314: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 909 KiB/s wr, 97 op/s 2024-03-18T20:46:12.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 374123 3.68 MB/sec execute 427 sec latency 72.876 ms 2024-03-18T20:46:12.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:12 smithi073 ceph-mon[47384]: pgmap v314: 97 pgs: 97 active+clean; 142 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 909 KiB/s wr, 97 op/s 2024-03-18T20:46:13.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 374123 3.68 MB/sec execute 428 sec latency 1032.593 ms 2024-03-18T20:46:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:14 smithi155 ceph-mon[48344]: pgmap v315: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-18T20:46:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:14 smithi046 ceph-mon[43495]: pgmap v315: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-18T20:46:14.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 374123 3.67 MB/sec execute 429 sec latency 2032.692 ms 2024-03-18T20:46:14.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:14 smithi073 ceph-mon[47384]: pgmap v315: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-03-18T20:46:15.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 375002 3.66 MB/sec execute 430 sec latency 2653.816 ms 2024-03-18T20:46:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:15 smithi155 ceph-mon[48344]: pgmap v316: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:46:16.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:15 smithi046 ceph-mon[43495]: pgmap v316: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:46:16.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:15 smithi073 ceph-mon[47384]: pgmap v316: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:46:16.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 376439 3.68 MB/sec execute 431 sec latency 16.114 ms 2024-03-18T20:46:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.186+0000 7fa5ab102700 1 -- 172.21.15.46:0/1102872280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a4068aa0 msgr2=0x7fa5a410dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.186+0000 7fa5ab102700 1 --2- 172.21.15.46:0/1102872280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a4068aa0 0x7fa5a410dab0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00b0a0 tx=0x7fa59c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.186+0000 7fa5ab102700 1 -- 172.21.15.46:0/1102872280 shutdown_connections 2024-03-18T20:46:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.186+0000 7fa5ab102700 1 --2- 172.21.15.46:0/1102872280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a4068aa0 0x7fa5a410dab0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.186+0000 7fa5ab102700 1 --2- 172.21.15.46:0/1102872280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a4106e10 0x7fa5a41071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.186+0000 7fa5ab102700 1 --2- 172.21.15.46:0/1102872280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a40680e0 0x7fa5a4068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.186+0000 7fa5ab102700 1 -- 172.21.15.46:0/1102872280 >> 172.21.15.46:0/1102872280 conn(0x7fa5a4076d50 msgr2=0x7fa5a4077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.186+0000 7fa5ab102700 1 -- 172.21.15.46:0/1102872280 shutdown_connections 2024-03-18T20:46:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.186+0000 7fa5ab102700 1 -- 172.21.15.46:0/1102872280 wait complete. 2024-03-18T20:46:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.187+0000 7fa5ab102700 1 Processor -- start 2024-03-18T20:46:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.187+0000 7fa5ab102700 1 -- start start 2024-03-18T20:46:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5ab102700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a40680e0 0x7fa5a4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5ab102700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a4068aa0 0x7fa5a4197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5ab102700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a4106e10 0x7fa5a419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5ab102700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a419c120 con 0x7fa5a40680e0 2024-03-18T20:46:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5ab102700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a419c290 con 0x7fa5a4068aa0 2024-03-18T20:46:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5ab102700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a419c450 con 0x7fa5a4106e10 2024-03-18T20:46:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5a8e9e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a40680e0 0x7fa5a4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5a8e9e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a40680e0 0x7fa5a4197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49052/0 (socket says 172.21.15.46:49052) 2024-03-18T20:46:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5a8e9e700 1 -- 172.21.15.46:0/4097584091 learned_addr learned my addr 172.21.15.46:0/4097584091 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa59bfff700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a4068aa0 0x7fa5a4197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5a969f700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a4106e10 0x7fa5a419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5a8e9e700 1 -- 172.21.15.46:0/4097584091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a4106e10 msgr2=0x7fa5a419bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5a8e9e700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a4106e10 0x7fa5a419bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5a8e9e700 1 -- 172.21.15.46:0/4097584091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a4068aa0 msgr2=0x7fa5a4197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5a8e9e700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a4068aa0 0x7fa5a4197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5a8e9e700 1 -- 172.21.15.46:0/4097584091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c009d00 con 0x7fa5a40680e0 2024-03-18T20:46:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.188+0000 7fa5a8e9e700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a40680e0 0x7fa5a4197200 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000e9a0 tx=0x7fa5a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.189+0000 7fa599ffb700 1 -- 172.21.15.46:0/4097584091 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0009790 con 0x7fa5a40680e0 2024-03-18T20:46:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.189+0000 7fa599ffb700 1 -- 172.21.15.46:0/4097584091 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5a0010430 con 0x7fa5a40680e0 2024-03-18T20:46:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.189+0000 7fa599ffb700 1 -- 172.21.15.46:0/4097584091 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0018680 con 0x7fa5a40680e0 2024-03-18T20:46:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.189+0000 7fa5ab102700 1 -- 172.21.15.46:0/4097584091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a419c730 con 0x7fa5a40680e0 2024-03-18T20:46:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.189+0000 7fa5ab102700 1 -- 172.21.15.46:0/4097584091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a41a3f70 con 0x7fa5a40680e0 2024-03-18T20:46:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.191+0000 7fa599ffb700 1 -- 172.21.15.46:0/4097584091 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa5a00105a0 con 0x7fa5a40680e0 2024-03-18T20:46:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.192+0000 7fa599ffb700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa58c0a1290 0x7fa58c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.192+0000 7fa599ffb700 1 -- 172.21.15.46:0/4097584091 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa5a0014070 con 0x7fa5a40680e0 2024-03-18T20:46:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.192+0000 7fa59bfff700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa58c0a1290 0x7fa58c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa59bfff700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa58c0a1290 0x7fa58c0a3750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4198750 tx=0x7fa59400b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa5b7297b80 1 Processor -- start 2024-03-18T20:46:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa5b7297b80 1 -- start start 2024-03-18T20:46:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa5b7297b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105570 0x559e30105950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa5b7297b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105e90 0x559e30110870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa5b7297b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30110db0 0x559e301131a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa5b7297b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3005f930 con 0x559e30105570 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa5b7297b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3005d2f0 con 0x559e30110db0 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa5b7297b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e30058be0 con 0x559e30105e90 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa5a8e9e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105570 0x559e30105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa59bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105e90 0x559e30110870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa5a8e9e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105570 0x559e30105950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49066/0 (socket says 172.21.15.46:49066) 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa5a8e9e700 1 -- 172.21.15.46:0/1525439032 learned_addr learned my addr 172.21.15.46:0/1525439032 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.193+0000 7fa5a969f700 1 --2- 172.21.15.46:0/1525439032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30110db0 0x559e301131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.194+0000 7fa59bfff700 1 -- 172.21.15.46:0/1525439032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30110db0 msgr2=0x559e301131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.194+0000 7fa59bfff700 1 --2- 172.21.15.46:0/1525439032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30110db0 0x559e301131a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.194+0000 7fa59bfff700 1 -- 172.21.15.46:0/1525439032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105570 msgr2=0x559e30105950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.194+0000 7fa59bfff700 1 --2- 172.21.15.46:0/1525439032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105570 0x559e30105950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.194+0000 7fa59bfff700 1 -- 172.21.15.46:0/1525439032 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e2feda350 con 0x559e30105e90 2024-03-18T20:46:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.194+0000 7fa5a8e9e700 1 --2- 172.21.15.46:0/1525439032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105570 0x559e30105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.194+0000 7fa5a969f700 1 --2- 172.21.15.46:0/1525439032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30110db0 0x559e301131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.194+0000 7fa59bfff700 1 --2- 172.21.15.46:0/1525439032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105e90 0x559e30110870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa594005d90 tx=0x7fa594013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76a10372af509a2c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.194+0000 7fa58affd700 1 -- 172.21.15.46:0/1525439032 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594011b80 con 0x559e30105e90 2024-03-18T20:46:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.194+0000 7fa58affd700 1 -- 172.21.15.46:0/1525439032 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa594015070 con 0x559e30105e90 2024-03-18T20:46:17.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.194+0000 7fa58affd700 1 -- 172.21.15.46:0/1525439032 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594005580 con 0x559e30105e90 2024-03-18T20:46:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.195+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1525439032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105e90 msgr2=0x559e30110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.195+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1525439032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105e90 0x559e30110870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa594005d90 tx=0x7fa594013000 comp rx=0 tx=0).stop 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.195+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1525439032 shutdown_connections 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.195+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1525439032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105570 0x559e30105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.195+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1525439032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30110db0 0x559e301131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.195+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1525439032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105e90 0x559e30110870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.195+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1525439032 >> 172.21.15.46:0/1525439032 conn(0x559e3007c1c0 msgr2=0x559e30101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.195+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1525439032 shutdown_connections 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.195+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1525439032 wait complete. 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.195+0000 7fa5b7297b80 1 -- start start 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.195+0000 7fa5b7297b80 1 Processor -- start 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5b7297b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30105570 0x559e30110730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5b7297b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 0x559e3010da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5b7297b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30110db0 0x559e3010e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5b7297b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2fff4d20 con 0x559e30105e90 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5b7297b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2fff49c0 con 0x559e30105570 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5b7297b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e2fedc640 con 0x559e30110db0 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5b7297b80 10 client.0 _ll_register_callbacks cb 0x559e2fe60340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa59bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 0x559e3010da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5a8e9e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30105570 0x559e30110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa59bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 0x559e3010da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49070/0 (socket says 172.21.15.46:49070) 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa59bfff700 1 -- 172.21.15.46:0/2139803475 learned_addr learned my addr 172.21.15.46:0/2139803475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5a969f700 1 --2- 172.21.15.46:0/2139803475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30110db0 0x559e3010e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5a8e9e700 1 -- 172.21.15.46:0/2139803475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30110db0 msgr2=0x559e3010e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5a8e9e700 1 --2- 172.21.15.46:0/2139803475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30110db0 0x559e3010e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5a8e9e700 1 -- 172.21.15.46:0/2139803475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 msgr2=0x559e3010da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5a8e9e700 1 --2- 172.21.15.46:0/2139803475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 0x559e3010da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5a8e9e700 1 -- 172.21.15.46:0/2139803475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a410e440 con 0x559e30105570 2024-03-18T20:46:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa5a8e9e700 1 --2- 172.21.15.46:0/2139803475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30105570 0x559e30110730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00969c0 tx=0x7fa5a00999c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5463bb716df5327 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.196+0000 7fa583fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:46:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa583fff700 1 -- 172.21.15.46:0/2139803475 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0099640 con 0x559e30105570 2024-03-18T20:46:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa583fff700 1 -- 172.21.15.46:0/2139803475 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa5a0099f50 con 0x559e30105570 2024-03-18T20:46:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa5b7297b80 1 -- 172.21.15.46:0/2139803475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e300f96c0 con 0x559e30105570 2024-03-18T20:46:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa583fff700 1 -- 172.21.15.46:0/2139803475 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5a00b82e0 con 0x559e30105570 2024-03-18T20:46:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa5b7297b80 10 client.28706 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa5b7297b80 1 -- 172.21.15.46:0/2139803475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7fa594010c20 con 0x559e30105570 2024-03-18T20:46:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa583fff700 1 -- 172.21.15.46:0/2139803475 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5a0063070 con 0x559e30105570 2024-03-18T20:46:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa583fff700 1 -- 172.21.15.46:0/2139803475 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa5a0095d40 con 0x559e30105570 2024-03-18T20:46:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa5b7297b80 10 client.28706 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa5b7297b80 10 client.28706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa5b7297b80 20 client.28706 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/2139803475 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559e30120a30 0x559e30122e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa5b7297b80 4 client.28706 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa5b7297b80 1 -- 172.21.15.46:0/2139803475 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e300516a0 con 0x559e30120a30 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.197+0000 7fa59bfff700 1 --2- 172.21.15.46:0/2139803475 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559e30120a30 0x559e30122e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.198+0000 7fa583fff700 10 client.28706 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.198+0000 7fa59bfff700 1 --2- 172.21.15.46:0/2139803475 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559e30120a30 0x559e30122e50 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72393d0a4f07d0e server_cookie=425bc44523f7559c in_seq=0 out_seq=0 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.198+0000 7fa583fff700 1 -- 172.21.15.46:0/2139803475 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559e300516a0 con 0x559e30120a30 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.198+0000 7fa583fff700 10 client.28706 handle_command_reply: tid=0 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.198+0000 7fa5b7297b80 1 client.28706 shutdown 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.198+0000 7fa5b7297b80 1 -- 172.21.15.46:0/2139803475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30105570 msgr2=0x559e30110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.198+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/2139803475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30105570 0x559e30110730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00969c0 tx=0x7fa5a00999c0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.199+0000 7fa5b7297b80 1 -- 172.21.15.46:0/2139803475 shutdown_connections 2024-03-18T20:46:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.199+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/2139803475 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559e30120a30 0x559e30122e50 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.199+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/2139803475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30110db0 0x559e3010e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.199+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/2139803475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30105570 0x559e30110730 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.199+0000 7fa583fff700 0 client.28706 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.199+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/2139803475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 0x559e3010da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.199+0000 7fa5b7297b80 1 -- 172.21.15.46:0/2139803475 >> 172.21.15.46:0/2139803475 conn(0x559e3007c1c0 msgr2=0x559e30101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.199+0000 7fa5b7297b80 1 -- 172.21.15.46:0/2139803475 shutdown_connections 2024-03-18T20:46:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.199+0000 7fa5b7297b80 1 -- 172.21.15.46:0/2139803475 wait complete. 2024-03-18T20:46:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.199+0000 7fa5b7297b80 20 client.28706 trim_cache size 0 max 16384 2024-03-18T20:46:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.214+0000 7fa5b7297b80 1 Processor -- start 2024-03-18T20:46:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.214+0000 7fa5b7297b80 1 -- start start 2024-03-18T20:46:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa5b7297b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105570 0x559e30105950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa5b7297b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 0x559e30108d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa5b7297b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30110db0 0x559e301092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa5b7297b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e300fbc10 con 0x559e30105e90 2024-03-18T20:46:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa5b7297b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e30102f70 con 0x559e30110db0 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa5b7297b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e30106530 con 0x559e30105570 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa59bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 0x559e30108d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa59bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 0x559e30108d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49072/0 (socket says 172.21.15.46:49072) 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa59bfff700 1 -- 172.21.15.46:0/1251880362 learned_addr learned my addr 172.21.15.46:0/1251880362 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa5a969f700 1 --2- 172.21.15.46:0/1251880362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30110db0 0x559e301092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa5a8e9e700 1 --2- 172.21.15.46:0/1251880362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105570 0x559e30105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa59bfff700 1 -- 172.21.15.46:0/1251880362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105570 msgr2=0x559e30105950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa59bfff700 1 --2- 172.21.15.46:0/1251880362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105570 0x559e30105950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa59bfff700 1 -- 172.21.15.46:0/1251880362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30110db0 msgr2=0x559e301092b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa59bfff700 1 --2- 172.21.15.46:0/1251880362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30110db0 0x559e301092b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa59bfff700 1 -- 172.21.15.46:0/1251880362 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e2fe86b00 con 0x559e30105e90 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa59bfff700 1 --2- 172.21.15.46:0/1251880362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 0x559e30108d70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa594005d90 tx=0x7fa59400ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ec92d9a58879fb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa58b7fe700 1 -- 172.21.15.46:0/1251880362 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59402c040 con 0x559e30105e90 2024-03-18T20:46:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa58b7fe700 1 -- 172.21.15.46:0/1251880362 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa59402d040 con 0x559e30105e90 2024-03-18T20:46:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.215+0000 7fa58b7fe700 1 -- 172.21.15.46:0/1251880362 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5940342b0 con 0x559e30105e90 2024-03-18T20:46:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.216+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1251880362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 msgr2=0x559e30108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.216+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1251880362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 0x559e30108d70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa594005d90 tx=0x7fa59400ba80 comp rx=0 tx=0).stop 2024-03-18T20:46:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.216+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1251880362 shutdown_connections 2024-03-18T20:46:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.216+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1251880362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30110db0 0x559e301092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.216+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1251880362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105570 0x559e30105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.216+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1251880362 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30105e90 0x559e30108d70 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.216+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1251880362 >> 172.21.15.46:0/1251880362 conn(0x559e3007c1c0 msgr2=0x559e30100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.216+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1251880362 shutdown_connections 2024-03-18T20:46:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.216+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1251880362 wait complete. 2024-03-18T20:46:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.216+0000 7fa5b7297b80 1 -- start start 2024-03-18T20:46:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa5b7297b80 1 Processor -- start 2024-03-18T20:46:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa5b7297b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30105570 0x559e30113a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa5b7297b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105e90 0x559e30113f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa5b7297b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30110db0 0x559e30114490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa5b7297b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e301738f0 con 0x559e30110db0 2024-03-18T20:46:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa5b7297b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e30173a30 con 0x559e30105570 2024-03-18T20:46:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa5b7297b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e30173bf0 con 0x559e30105e90 2024-03-18T20:46:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa5b7297b80 10 client.0 _ll_register_callbacks cb 0x7fa5a000e8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa5a969f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30110db0 0x559e30114490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa5a8e9e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30105570 0x559e30113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa5a969f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30110db0 0x559e30114490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49082/0 (socket says 172.21.15.46:49082) 2024-03-18T20:46:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa5a969f700 1 -- 172.21.15.46:0/1551753666 learned_addr learned my addr 172.21.15.46:0/1551753666 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.217+0000 7fa59bfff700 1 --2- 172.21.15.46:0/1551753666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105e90 0x559e30113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa5a8e9e700 1 -- 172.21.15.46:0/1551753666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105e90 msgr2=0x559e30113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa5a8e9e700 1 --2- 172.21.15.46:0/1551753666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105e90 0x559e30113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa5a8e9e700 1 -- 172.21.15.46:0/1551753666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30110db0 msgr2=0x559e30114490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa5a8e9e700 1 --2- 172.21.15.46:0/1551753666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30110db0 0x559e30114490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa5a8e9e700 1 -- 172.21.15.46:0/1551753666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e2feda350 con 0x559e30105570 2024-03-18T20:46:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa5a8e9e700 1 --2- 172.21.15.46:0/1551753666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30105570 0x559e30113a10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0000c00 tx=0x7fa5a00961c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f212032d30fc61a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa583fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa583fff700 1 -- 172.21.15.46:0/1551753666 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00a2150 con 0x559e30105570 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa583fff700 1 -- 172.21.15.46:0/1551753666 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa5a0099f50 con 0x559e30105570 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1551753666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa594015070 con 0x559e30105570 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa583fff700 1 -- 172.21.15.46:0/1551753666 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5a00a23d0 con 0x559e30105570 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa5b7297b80 10 client.28712 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1551753666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x559e3001ee40 con 0x559e30105570 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa583fff700 1 -- 172.21.15.46:0/1551753666 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa594015070 con 0x559e30105570 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa583fff700 1 -- 172.21.15.46:0/1551753666 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa5a0094c60 con 0x559e30105570 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa5b7297b80 10 client.28712 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa5b7297b80 10 client.28712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.218+0000 7fa5b7297b80 20 client.28712 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.219+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1551753666 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559e30175110 0x559e30177530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.219+0000 7fa5b7297b80 4 client.28712 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:46:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.219+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1551753666 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559e2fea7c90 con 0x559e30175110 2024-03-18T20:46:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.219+0000 7fa59bfff700 1 --2- 172.21.15.46:0/1551753666 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559e30175110 0x559e30177530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.219+0000 7fa583fff700 10 client.28712 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.219+0000 7fa59bfff700 1 --2- 172.21.15.46:0/1551753666 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559e30175110 0x559e30177530 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ce6f148e8456fe server_cookie=488e2030b38f708a in_seq=0 out_seq=0 2024-03-18T20:46:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.249+0000 7fa583fff700 1 -- 172.21.15.46:0/1551753666 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559e2fea7c90 con 0x559e30175110 2024-03-18T20:46:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.249+0000 7fa583fff700 10 client.28712 handle_command_reply: tid=0 2024-03-18T20:46:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.249+0000 7fa5b7297b80 1 client.28712 shutdown 2024-03-18T20:46:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.249+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1551753666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30105570 msgr2=0x559e30113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.249+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1551753666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30105570 0x559e30113a10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0000c00 tx=0x7fa5a00961c0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.250+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1551753666 shutdown_connections 2024-03-18T20:46:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.250+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1551753666 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559e30175110 0x559e30177530 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.250+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1551753666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e30110db0 0x559e30114490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.250+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1551753666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559e30105570 0x559e30113a10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.250+0000 7fa583fff700 0 client.28712 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.250+0000 7fa5b7297b80 1 --2- 172.21.15.46:0/1551753666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559e30105e90 0x559e30113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.250+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1551753666 >> 172.21.15.46:0/1551753666 conn(0x559e3007c1c0 msgr2=0x559e30101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.250+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1551753666 shutdown_connections 2024-03-18T20:46:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.250+0000 7fa5b7297b80 1 -- 172.21.15.46:0/1551753666 wait complete. 2024-03-18T20:46:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.250+0000 7fa5b7297b80 20 client.28712 trim_cache size 0 max 16384 2024-03-18T20:46:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:46:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:46:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:46:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:46:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.251+0000 7fa5ab102700 1 -- 172.21.15.46:0/4097584091 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa58c0a1290 msgr2=0x7fa58c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.251+0000 7fa5ab102700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa58c0a1290 0x7fa58c0a3750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4198750 tx=0x7fa59400b410 comp rx=0 tx=0).stop 2024-03-18T20:46:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.251+0000 7fa5ab102700 1 -- 172.21.15.46:0/4097584091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a40680e0 msgr2=0x7fa5a4197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.251+0000 7fa5ab102700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a40680e0 0x7fa5a4197200 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000e9a0 tx=0x7fa5a000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.251+0000 7fa5ab102700 1 -- 172.21.15.46:0/4097584091 shutdown_connections 2024-03-18T20:46:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.251+0000 7fa5ab102700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa58c0a1290 0x7fa58c0a3750 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.251+0000 7fa5ab102700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a40680e0 0x7fa5a4197200 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.251+0000 7fa5ab102700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5a4106e10 0x7fa5a419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.251+0000 7fa5ab102700 1 --2- 172.21.15.46:0/4097584091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a4068aa0 0x7fa5a4197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.251+0000 7fa5ab102700 1 -- 172.21.15.46:0/4097584091 >> 172.21.15.46:0/4097584091 conn(0x7fa5a4076d50 msgr2=0x7fa5a40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.251+0000 7fa5ab102700 1 -- 172.21.15.46:0/4097584091 shutdown_connections 2024-03-18T20:46:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:17.251+0000 7fa5ab102700 1 -- 172.21.15.46:0/4097584091 wait complete. 2024-03-18T20:46:17.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 377583 3.69 MB/sec execute 432 sec latency 62.596 ms 2024-03-18T20:46:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:18 smithi155 ceph-mon[48344]: pgmap v317: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:46:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:18 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:46:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:18 smithi046 ceph-mon[43495]: pgmap v317: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:46:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:18 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:46:18.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 377666 3.68 MB/sec execute 433 sec latency 898.832 ms 2024-03-18T20:46:18.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:18 smithi073 ceph-mon[47384]: pgmap v317: 97 pgs: 97 active+clean; 145 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:46:18.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:18 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:46:19.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 379035 3.69 MB/sec execute 434 sec latency 1266.473 ms 2024-03-18T20:46:20.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: pgmap v318: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-03-18T20:46:20.220 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.221 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi073 to 182.4M 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: Unable to set osd_memory_target on smithi073 to 191350476: error parsing value: Value '191350476' is below minimum 939524096 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.222 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:46:20.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:46:20.223 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:19 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: pgmap v318: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi073 to 182.4M 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: Unable to set osd_memory_target on smithi073 to 191350476: error parsing value: Value '191350476' is below minimum 939524096 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.292 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:46:20.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:46:20.293 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:19 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: pgmap v318: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi073 to 182.4M 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: Unable to set osd_memory_target on smithi073 to 191350476: error parsing value: Value '191350476' is below minimum 939524096 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:46:20.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:46:20.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:19 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:46:20.713 INFO:tasks.workunit.client.0.smithi046.stdout: 1 380343 3.69 MB/sec execute 435 sec latency 15.678 ms 2024-03-18T20:46:21.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:20 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi046 to 183.5M 2024-03-18T20:46:21.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:20 smithi155 ceph-mon[48344]: Unable to set osd_memory_target on smithi046 to 192483737: error parsing value: Value '192483737' is below minimum 939524096 2024-03-18T20:46:21.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:20 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi046 to 183.5M 2024-03-18T20:46:21.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:20 smithi046 ceph-mon[43495]: Unable to set osd_memory_target on smithi046 to 192483737: error parsing value: Value '192483737' is below minimum 939524096 2024-03-18T20:46:21.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:20 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi046 to 183.5M 2024-03-18T20:46:21.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:20 smithi073 ceph-mon[47384]: Unable to set osd_memory_target on smithi046 to 192483737: error parsing value: Value '192483737' is below minimum 939524096 2024-03-18T20:46:21.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 381255 3.70 MB/sec execute 436 sec latency 309.993 ms 2024-03-18T20:46:22.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:21 smithi155 ceph-mon[48344]: pgmap v319: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-18T20:46:22.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:21 smithi046 ceph-mon[43495]: pgmap v319: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-18T20:46:22.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:21 smithi073 ceph-mon[47384]: pgmap v319: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-03-18T20:46:22.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 381255 3.69 MB/sec execute 437 sec latency 1310.111 ms 2024-03-18T20:46:23.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 381255 3.68 MB/sec execute 438 sec latency 2310.235 ms 2024-03-18T20:46:24.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:46:24.056 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:46:24.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.215+0000 7f57ced2c700 1 -- 172.21.15.46:0/1129409048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c80ffe50 msgr2=0x7f57c8100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.215+0000 7f57ced2c700 1 --2- 172.21.15.46:0/1129409048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c80ffe50 0x7f57c8100230 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009990 tx=0x7f57b8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.215+0000 7f57ced2c700 1 -- 172.21.15.46:0/1129409048 shutdown_connections 2024-03-18T20:46:24.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.215+0000 7f57ced2c700 1 --2- 172.21.15.46:0/1129409048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c80ffe50 0x7f57c8100230 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.215+0000 7f57ced2c700 1 --2- 172.21.15.46:0/1129409048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c8104f60 0x7f57c810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.215+0000 7f57ced2c700 1 --2- 172.21.15.46:0/1129409048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57c8100800 0x7f57c8104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.215+0000 7f57ced2c700 1 -- 172.21.15.46:0/1129409048 >> 172.21.15.46:0/1129409048 conn(0x7f57c8076d40 msgr2=0x7f57c8077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.215+0000 7f57ced2c700 1 -- 172.21.15.46:0/1129409048 shutdown_connections 2024-03-18T20:46:24.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.215+0000 7f57ced2c700 1 -- 172.21.15.46:0/1129409048 wait complete. 2024-03-18T20:46:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57ced2c700 1 Processor -- start 2024-03-18T20:46:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57ced2c700 1 -- start start 2024-03-18T20:46:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57ced2c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c80ffe50 0x7f57c8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57ced2c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57c8104f60 0x7f57c8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57ced2c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c8198a70 0x7f57c81a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57ced2c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c81a3750 con 0x7f57c8198a70 2024-03-18T20:46:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57ced2c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c81a38c0 con 0x7f57c80ffe50 2024-03-18T20:46:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57ced2c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c81a3a80 con 0x7f57c8104f60 2024-03-18T20:46:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57cd2c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c8198a70 0x7f57c81a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57cd2c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c8198a70 0x7f57c81a3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49100/0 (socket says 172.21.15.46:49100) 2024-03-18T20:46:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57cd2c9700 1 -- 172.21.15.46:0/1464363442 learned_addr learned my addr 172.21.15.46:0/1464363442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57c7fff700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57c8104f60 0x7f57c8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.216+0000 7f57ccac8700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c80ffe50 0x7f57c8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.217+0000 7f57cd2c9700 1 -- 172.21.15.46:0/1464363442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57c8104f60 msgr2=0x7f57c8198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.217+0000 7f57cd2c9700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57c8104f60 0x7f57c8198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.217+0000 7f57cd2c9700 1 -- 172.21.15.46:0/1464363442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c80ffe50 msgr2=0x7f57c8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.217+0000 7f57cd2c9700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c80ffe50 0x7f57c8197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.217+0000 7f57cd2c9700 1 -- 172.21.15.46:0/1464363442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b80095f0 con 0x7f57c8198a70 2024-03-18T20:46:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.217+0000 7f57cd2c9700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c8198a70 0x7f57c81a3070 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f57c000ef30 tx=0x7f57c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.217+0000 7f57c5ffb700 1 -- 172.21.15.46:0/1464363442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0018070 con 0x7f57c8198a70 2024-03-18T20:46:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.217+0000 7f57c5ffb700 1 -- 172.21.15.46:0/1464363442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c000f040 con 0x7f57c8198a70 2024-03-18T20:46:24.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.217+0000 7f57ced2c700 1 -- 172.21.15.46:0/1464363442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c81a3d00 con 0x7f57c8198a70 2024-03-18T20:46:24.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.217+0000 7f57c5ffb700 1 -- 172.21.15.46:0/1464363442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c000cd00 con 0x7f57c8198a70 2024-03-18T20:46:24.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.217+0000 7f57ced2c700 1 -- 172.21.15.46:0/1464363442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c81a4210 con 0x7f57c8198a70 2024-03-18T20:46:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.219+0000 7f57c5ffb700 1 -- 172.21.15.46:0/1464363442 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f57c0003a00 con 0x7f57c8198a70 2024-03-18T20:46:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.220+0000 7f57c5ffb700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57b00a1290 0x7f57b00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.220+0000 7f57c5ffb700 1 -- 172.21.15.46:0/1464363442 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f57c00cc7b0 con 0x7f57c8198a70 2024-03-18T20:46:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.220+0000 7f57ccac8700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57b00a1290 0x7f57b00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.221+0000 7f57ccac8700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57b00a1290 0x7f57b00a3750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f57b800b2e0 tx=0x7f57b8005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.221+0000 7f57daec1b80 1 Processor -- start 2024-03-18T20:46:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.221+0000 7f57daec1b80 1 -- start start 2024-03-18T20:46:24.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.221+0000 7f57daec1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8ca60 0x558f8df8ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.221+0000 7f57daec1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df8d380 0x558f8df97ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.221+0000 7f57daec1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df98220 0x558f8df9a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.221+0000 7f57daec1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8dee6990 con 0x558f8df98220 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.221+0000 7f57daec1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8dee4350 con 0x558f8df8d380 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.221+0000 7f57daec1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8dedfc40 con 0x558f8df8ca60 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57cd2c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df98220 0x558f8df9a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57cd2c9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df98220 0x558f8df9a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49116/0 (socket says 172.21.15.46:49116) 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df8d380 0x558f8df97ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57cd2c9700 1 -- 172.21.15.46:0/3425274438 learned_addr learned my addr 172.21.15.46:0/3425274438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57cd2c9700 1 -- 172.21.15.46:0/3425274438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8ca60 msgr2=0x558f8df8ce40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57cd2c9700 1 --2- 172.21.15.46:0/3425274438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8ca60 0x558f8df8ce40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57cd2c9700 1 -- 172.21.15.46:0/3425274438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df8d380 msgr2=0x558f8df97ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57cd2c9700 1 --2- 172.21.15.46:0/3425274438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df8d380 0x558f8df97ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57cd2c9700 1 -- 172.21.15.46:0/3425274438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f8dd613b0 con 0x558f8df98220 2024-03-18T20:46:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57c7fff700 1 --2- 172.21.15.46:0/3425274438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df8d380 0x558f8df97ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57cd2c9700 1 --2- 172.21.15.46:0/3425274438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df98220 0x558f8df9a610 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f57c0095e30 tx=0x7f57c0095da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab3f7094f737b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57aeffd700 1 -- 172.21.15.46:0/3425274438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0084040 con 0x558f8df98220 2024-03-18T20:46:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57aeffd700 1 -- 172.21.15.46:0/3425274438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c00b7e90 con 0x558f8df98220 2024-03-18T20:46:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.222+0000 7f57aeffd700 1 -- 172.21.15.46:0/3425274438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c009d760 con 0x558f8df98220 2024-03-18T20:46:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.223+0000 7f57daec1b80 1 -- 172.21.15.46:0/3425274438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df98220 msgr2=0x558f8df9a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.223+0000 7f57daec1b80 1 --2- 172.21.15.46:0/3425274438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df98220 0x558f8df9a610 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f57c0095e30 tx=0x7f57c0095da0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.223+0000 7f57daec1b80 1 -- 172.21.15.46:0/3425274438 shutdown_connections 2024-03-18T20:46:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.223+0000 7f57daec1b80 1 --2- 172.21.15.46:0/3425274438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df98220 0x558f8df9a610 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.223+0000 7f57daec1b80 1 --2- 172.21.15.46:0/3425274438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8ca60 0x558f8df8ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.223+0000 7f57daec1b80 1 --2- 172.21.15.46:0/3425274438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df8d380 0x558f8df97ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.223+0000 7f57daec1b80 1 -- 172.21.15.46:0/3425274438 >> 172.21.15.46:0/3425274438 conn(0x558f8df03220 msgr2=0x558f8df89020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.224+0000 7f57daec1b80 1 -- 172.21.15.46:0/3425274438 shutdown_connections 2024-03-18T20:46:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.224+0000 7f57daec1b80 1 -- 172.21.15.46:0/3425274438 wait complete. 2024-03-18T20:46:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.224+0000 7f57daec1b80 1 -- start start 2024-03-18T20:46:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.224+0000 7f57daec1b80 1 Processor -- start 2024-03-18T20:46:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.224+0000 7f57daec1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8ca60 0x558f8df94fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.224+0000 7f57daec1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8d380 0x558f8df954f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.224+0000 7f57daec1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df98220 0x558f8df95a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.224+0000 7f57daec1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8de7bd80 con 0x558f8df8ca60 2024-03-18T20:46:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.224+0000 7f57daec1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8de7ba20 con 0x558f8df98220 2024-03-18T20:46:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.225+0000 7f57daec1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8dd636a0 con 0x558f8df8d380 2024-03-18T20:46:24.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.225+0000 7f57daec1b80 10 client.0 _ll_register_callbacks cb 0x558f8dce73a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:24.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.225+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8d380 0x558f8df954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.225+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8d380 0x558f8df954f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:47396/0 (socket says 172.21.15.46:47396) 2024-03-18T20:46:24.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.225+0000 7f57c7fff700 1 -- 172.21.15.46:0/519081619 learned_addr learned my addr 172.21.15.46:0/519081619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:24.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.225+0000 7f57c7fff700 1 -- 172.21.15.46:0/519081619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df98220 msgr2=0x558f8df95a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:46:24.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.225+0000 7f57c7fff700 1 --2- 172.21.15.46:0/519081619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df98220 0x558f8df95a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.225+0000 7f57c7fff700 1 -- 172.21.15.46:0/519081619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8ca60 msgr2=0x558f8df94fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.225+0000 7f57c7fff700 1 --2- 172.21.15.46:0/519081619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8ca60 0x558f8df94fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.225+0000 7f57c7fff700 1 -- 172.21.15.46:0/519081619 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57bc0098e0 con 0x558f8df8d380 2024-03-18T20:46:24.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.225+0000 7f57c7fff700 1 --2- 172.21.15.46:0/519081619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8d380 0x558f8df954f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f57bc012960 tx=0x7f57bc012d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd2189b5552d84c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.225+0000 7f57a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:46:24.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.226+0000 7f57a7fff700 1 -- 172.21.15.46:0/519081619 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc017070 con 0x558f8df8d380 2024-03-18T20:46:24.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.226+0000 7f57a7fff700 1 -- 172.21.15.46:0/519081619 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f57bc004030 con 0x558f8df8d380 2024-03-18T20:46:24.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.226+0000 7f57daec1b80 1 -- 172.21.15.46:0/519081619 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f8df80ad0 con 0x558f8df8d380 2024-03-18T20:46:24.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.226+0000 7f57a7fff700 1 -- 172.21.15.46:0/519081619 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57bc0041f0 con 0x558f8df8d380 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.226+0000 7f57daec1b80 10 client.28741 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.226+0000 7f57daec1b80 1 -- 172.21.15.46:0/519081619 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f57c81a4210 con 0x558f8df8d380 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.227+0000 7f57a7fff700 1 -- 172.21.15.46:0/519081619 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f8df80ad0 con 0x558f8df8d380 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.227+0000 7f57a7fff700 1 -- 172.21.15.46:0/519081619 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f57bc003950 con 0x558f8df8d380 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.227+0000 7f57daec1b80 10 client.28741 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.227+0000 7f57daec1b80 10 client.28741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.227+0000 7f57daec1b80 20 client.28741 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.227+0000 7f57daec1b80 1 --2- 172.21.15.46:0/519081619 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558f8dfac2c0 0x558f8dfae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.227+0000 7f57daec1b80 4 client.28741 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.227+0000 7f57daec1b80 1 -- 172.21.15.46:0/519081619 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f8ded8700 con 0x558f8dfac2c0 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.228+0000 7f57ccac8700 1 --2- 172.21.15.46:0/519081619 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558f8dfac2c0 0x558f8dfae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.228+0000 7f57a7fff700 10 client.28741 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.228+0000 7f57ccac8700 1 --2- 172.21.15.46:0/519081619 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558f8dfac2c0 0x558f8dfae6e0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24164a81d6d2d5 server_cookie=a9076e1ba980a105 in_seq=0 out_seq=0 2024-03-18T20:46:24.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.229+0000 7f57a7fff700 1 -- 172.21.15.46:0/519081619 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558f8ded8700 con 0x558f8dfac2c0 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.229+0000 7f57a7fff700 10 client.28741 handle_command_reply: tid=0 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.229+0000 7f57daec1b80 1 client.28741 shutdown 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.229+0000 7f57daec1b80 1 -- 172.21.15.46:0/519081619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8d380 msgr2=0x558f8df954f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.229+0000 7f57daec1b80 1 --2- 172.21.15.46:0/519081619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8d380 0x558f8df954f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f57bc012960 tx=0x7f57bc012d20 comp rx=0 tx=0).stop 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.229+0000 7f57daec1b80 1 -- 172.21.15.46:0/519081619 shutdown_connections 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.229+0000 7f57daec1b80 1 --2- 172.21.15.46:0/519081619 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558f8dfac2c0 0x558f8dfae6e0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.229+0000 7f57daec1b80 1 --2- 172.21.15.46:0/519081619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8ca60 0x558f8df94fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.229+0000 7f57daec1b80 1 --2- 172.21.15.46:0/519081619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df98220 0x558f8df95a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.229+0000 7f57daec1b80 1 --2- 172.21.15.46:0/519081619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8d380 0x558f8df954f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.229+0000 7f57a7fff700 0 client.28741 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.229+0000 7f57daec1b80 1 -- 172.21.15.46:0/519081619 >> 172.21.15.46:0/519081619 conn(0x558f8df03220 msgr2=0x558f8df88c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.230+0000 7f57daec1b80 1 -- 172.21.15.46:0/519081619 shutdown_connections 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.230+0000 7f57daec1b80 1 -- 172.21.15.46:0/519081619 wait complete. 2024-03-18T20:46:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.230+0000 7f57daec1b80 20 client.28741 trim_cache size 0 max 16384 2024-03-18T20:46:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.249+0000 7f57daec1b80 1 Processor -- start 2024-03-18T20:46:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.249+0000 7f57daec1b80 1 -- start start 2024-03-18T20:46:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.249+0000 7f57daec1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8ca60 0x558f8dfa32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.249+0000 7f57daec1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 0x558f8df88b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.249+0000 7f57daec1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df98220 0x558f8df89090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.249+0000 7f57daec1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8df96620 con 0x558f8df8d380 2024-03-18T20:46:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.249+0000 7f57daec1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8df855e0 con 0x558f8df98220 2024-03-18T20:46:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.249+0000 7f57daec1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8df8e610 con 0x558f8df8ca60 2024-03-18T20:46:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 0x558f8df88b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 0x558f8df88b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49144/0 (socket says 172.21.15.46:49144) 2024-03-18T20:46:24.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57c7fff700 1 -- 172.21.15.46:0/840825695 learned_addr learned my addr 172.21.15.46:0/840825695 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57cd2c9700 1 --2- 172.21.15.46:0/840825695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df98220 0x558f8df89090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57ccac8700 1 --2- 172.21.15.46:0/840825695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8ca60 0x558f8dfa32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57c7fff700 1 -- 172.21.15.46:0/840825695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8ca60 msgr2=0x558f8dfa32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57c7fff700 1 --2- 172.21.15.46:0/840825695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8ca60 0x558f8dfa32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57c7fff700 1 -- 172.21.15.46:0/840825695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df98220 msgr2=0x558f8df89090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57c7fff700 1 --2- 172.21.15.46:0/840825695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df98220 0x558f8df89090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57c7fff700 1 -- 172.21.15.46:0/840825695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f8dd0db60 con 0x558f8df8d380 2024-03-18T20:46:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57c7fff700 1 --2- 172.21.15.46:0/840825695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 0x558f8df88b50 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f57bc000c00 tx=0x7f57bc003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774bbc70bd540e47 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57af7fe700 1 -- 172.21.15.46:0/840825695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc017040 con 0x558f8df8d380 2024-03-18T20:46:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57af7fe700 1 -- 172.21.15.46:0/840825695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57bc003940 con 0x558f8df8d380 2024-03-18T20:46:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.250+0000 7f57af7fe700 1 -- 172.21.15.46:0/840825695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0056c0 con 0x558f8df8d380 2024-03-18T20:46:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.251+0000 7f57daec1b80 1 -- 172.21.15.46:0/840825695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 msgr2=0x558f8df88b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.251+0000 7f57daec1b80 1 --2- 172.21.15.46:0/840825695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 0x558f8df88b50 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f57bc000c00 tx=0x7f57bc003e80 comp rx=0 tx=0).stop 2024-03-18T20:46:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.251+0000 7f57daec1b80 1 -- 172.21.15.46:0/840825695 shutdown_connections 2024-03-18T20:46:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.251+0000 7f57daec1b80 1 --2- 172.21.15.46:0/840825695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df98220 0x558f8df89090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.251+0000 7f57daec1b80 1 --2- 172.21.15.46:0/840825695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df8ca60 0x558f8dfa32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.251+0000 7f57daec1b80 1 --2- 172.21.15.46:0/840825695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 0x558f8df88b50 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.251+0000 7f57daec1b80 1 -- 172.21.15.46:0/840825695 >> 172.21.15.46:0/840825695 conn(0x558f8df03220 msgr2=0x558f8df881f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.251+0000 7f57daec1b80 1 -- 172.21.15.46:0/840825695 shutdown_connections 2024-03-18T20:46:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.251+0000 7f57daec1b80 1 -- 172.21.15.46:0/840825695 wait complete. 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.251+0000 7f57daec1b80 1 -- start start 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57daec1b80 1 Processor -- start 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57daec1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df8ca60 0x558f8dfa3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57daec1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 0x558f8dfa18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57daec1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df98220 0x558f8dfa1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57daec1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8dfa2500 con 0x558f8df8d380 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57daec1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8dfa2670 con 0x558f8df8ca60 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57daec1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f8e000100 con 0x558f8df98220 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57daec1b80 10 client.0 _ll_register_callbacks cb 0x7f57bc012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 0x558f8dfa18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 0x558f8dfa18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49152/0 (socket says 172.21.15.46:49152) 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57c7fff700 1 -- 172.21.15.46:0/4012958019 learned_addr learned my addr 172.21.15.46:0/4012958019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:24.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57ccac8700 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df8ca60 0x558f8dfa3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.252+0000 7f57cd2c9700 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df98220 0x558f8dfa1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57c7fff700 1 -- 172.21.15.46:0/4012958019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df98220 msgr2=0x558f8dfa1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57c7fff700 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df98220 0x558f8dfa1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57c7fff700 1 -- 172.21.15.46:0/4012958019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df8ca60 msgr2=0x558f8dfa3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57c7fff700 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df8ca60 0x558f8dfa3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57c7fff700 1 -- 172.21.15.46:0/4012958019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c81a4210 con 0x558f8df8d380 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57ccac8700 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df8ca60 0x558f8dfa3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57c7fff700 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 0x558f8dfa18e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f57bc010470 tx=0x7f57bc0051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7812e382dfe39047 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57a7fff700 1 -- 172.21.15.46:0/4012958019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc017070 con 0x558f8df8d380 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57a7fff700 1 -- 172.21.15.46:0/4012958019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f57bc02b030 con 0x558f8df8d380 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57daec1b80 1 -- 172.21.15.46:0/4012958019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57c00b7e90 con 0x558f8df8d380 2024-03-18T20:46:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57a7fff700 1 -- 172.21.15.46:0/4012958019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57bc005510 con 0x558f8df8d380 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57daec1b80 10 client.19098 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57daec1b80 1 -- 172.21.15.46:0/4012958019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x558f8dea5ea0 con 0x558f8df8d380 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57a7fff700 1 -- 172.21.15.46:0/4012958019 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c00b7e90 con 0x558f8df8d380 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57a7fff700 1 -- 172.21.15.46:0/4012958019 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f57bc01c680 con 0x558f8df8d380 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57daec1b80 10 client.19098 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.253+0000 7f57daec1b80 10 client.19098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.254+0000 7f57daec1b80 20 client.19098 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.254+0000 7f57daec1b80 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558f8e0014c0 0x558f8e0038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.254+0000 7f57daec1b80 4 client.19098 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.254+0000 7f57daec1b80 1 -- 172.21.15.46:0/4012958019 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558f8dd2ecf0 con 0x558f8e0014c0 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.254+0000 7f57ccac8700 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558f8e0014c0 0x558f8e0038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.254+0000 7f57a7fff700 10 client.19098 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.254+0000 7f57ccac8700 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558f8e0014c0 0x558f8e0038e0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f72f6464b57a33 server_cookie=cd8622258c4272a5 in_seq=0 out_seq=0 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.254+0000 7f57a7fff700 1 -- 172.21.15.46:0/4012958019 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558f8dd2ecf0 con 0x558f8e0014c0 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.254+0000 7f57a7fff700 10 client.19098 handle_command_reply: tid=0 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.254+0000 7f57daec1b80 1 client.19098 shutdown 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.254+0000 7f57daec1b80 1 -- 172.21.15.46:0/4012958019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 msgr2=0x558f8dfa18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.254+0000 7f57daec1b80 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 0x558f8dfa18e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f57bc010470 tx=0x7f57bc0051c0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57daec1b80 1 -- 172.21.15.46:0/4012958019 shutdown_connections 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57daec1b80 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558f8e0014c0 0x558f8e0038e0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57daec1b80 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f8df98220 0x558f8dfa1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57a7fff700 0 client.19098 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57daec1b80 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f8df8ca60 0x558f8dfa3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57daec1b80 1 --2- 172.21.15.46:0/4012958019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f8df8d380 0x558f8dfa18e0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57daec1b80 1 -- 172.21.15.46:0/4012958019 >> 172.21.15.46:0/4012958019 conn(0x558f8df03220 msgr2=0x558f8df902c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57daec1b80 1 -- 172.21.15.46:0/4012958019 shutdown_connections 2024-03-18T20:46:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57daec1b80 1 -- 172.21.15.46:0/4012958019 wait complete. 2024-03-18T20:46:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57daec1b80 20 client.19098 trim_cache size 0 max 16384 2024-03-18T20:46:24.267 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:46:24.267 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:46:24.267 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:46:24.267 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:46:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57ced2c700 1 -- 172.21.15.46:0/1464363442 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57b00a1290 msgr2=0x7f57b00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57ced2c700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57b00a1290 0x7f57b00a3750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f57b800b2e0 tx=0x7f57b8005960 comp rx=0 tx=0).stop 2024-03-18T20:46:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57ced2c700 1 -- 172.21.15.46:0/1464363442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c8198a70 msgr2=0x7f57c81a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57ced2c700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c8198a70 0x7f57c81a3070 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f57c000ef30 tx=0x7f57c000c3a0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57ced2c700 1 -- 172.21.15.46:0/1464363442 shutdown_connections 2024-03-18T20:46:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57ced2c700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f57b00a1290 0x7f57b00a3750 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57ced2c700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c8198a70 0x7f57c81a3070 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57ced2c700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c80ffe50 0x7f57c8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57ced2c700 1 --2- 172.21.15.46:0/1464363442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57c8104f60 0x7f57c8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57ced2c700 1 -- 172.21.15.46:0/1464363442 >> 172.21.15.46:0/1464363442 conn(0x7f57c8076d40 msgr2=0x7f57c80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.255+0000 7f57ced2c700 1 -- 172.21.15.46:0/1464363442 shutdown_connections 2024-03-18T20:46:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.256+0000 7f57ced2c700 1 -- 172.21.15.46:0/1464363442 wait complete. 2024-03-18T20:46:24.271 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:46:24.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:46:24.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.416+0000 7effaf8a0700 1 -- 172.21.15.46:0/1737219190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa80ffb40 msgr2=0x7effa80fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.416+0000 7effaf8a0700 1 --2- 172.21.15.46:0/1737219190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa80ffb40 0x7effa80fffc0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7eff980099c0 tx=0x7eff98009cd0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.416+0000 7effaf8a0700 1 -- 172.21.15.46:0/1737219190 shutdown_connections 2024-03-18T20:46:24.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.417+0000 7effaf8a0700 1 --2- 172.21.15.46:0/1737219190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa8100500 0x7effa810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.417+0000 7effaf8a0700 1 --2- 172.21.15.46:0/1737219190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa8105c00 0x7effa8105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.417+0000 7effaf8a0700 1 --2- 172.21.15.46:0/1737219190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa80ffb40 0x7effa80fffc0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.417+0000 7effaf8a0700 1 -- 172.21.15.46:0/1737219190 >> 172.21.15.46:0/1737219190 conn(0x7effa80fb5f0 msgr2=0x7effa80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.417+0000 7effaf8a0700 1 -- 172.21.15.46:0/1737219190 shutdown_connections 2024-03-18T20:46:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.417+0000 7effaf8a0700 1 -- 172.21.15.46:0/1737219190 wait complete. 2024-03-18T20:46:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.417+0000 7effaf8a0700 1 Processor -- start 2024-03-18T20:46:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.417+0000 7effaf8a0700 1 -- start start 2024-03-18T20:46:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.417+0000 7effaf8a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa80ffb40 0x7effa8197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7effaf8a0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa8100500 0x7effa8197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7effaf8a0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa8105c00 0x7effa819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7effaf8a0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa819bfb0 con 0x7effa80ffb40 2024-03-18T20:46:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7effaf8a0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa819c120 con 0x7effa8100500 2024-03-18T20:46:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7effaf8a0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa819c2e0 con 0x7effa8105c00 2024-03-18T20:46:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7effad63c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa80ffb40 0x7effa8197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7efface3b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa8100500 0x7effa8197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7effad63c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa80ffb40 0x7effa8197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49166/0 (socket says 172.21.15.46:49166) 2024-03-18T20:46:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7effad63c700 1 -- 172.21.15.46:0/942966275 learned_addr learned my addr 172.21.15.46:0/942966275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7effade3d700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa8105c00 0x7effa819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7efface3b700 1 -- 172.21.15.46:0/942966275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa8105c00 msgr2=0x7effa819b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7efface3b700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa8105c00 0x7effa819b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7efface3b700 1 -- 172.21.15.46:0/942966275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa80ffb40 msgr2=0x7effa8197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7efface3b700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa80ffb40 0x7effa8197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7efface3b700 1 -- 172.21.15.46:0/942966275 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff980095f0 con 0x7effa8100500 2024-03-18T20:46:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7effad63c700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa80ffb40 0x7effa8197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7efface3b700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa8100500 0x7effa8197690 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7eff980098e0 tx=0x7eff9800b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7eff9e7fc700 1 -- 172.21.15.46:0/942966275 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9801d070 con 0x7effa8100500 2024-03-18T20:46:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7effaf8a0700 1 -- 172.21.15.46:0/942966275 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa819c560 con 0x7effa8100500 2024-03-18T20:46:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7eff9e7fc700 1 -- 172.21.15.46:0/942966275 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff9800bde0 con 0x7effa8100500 2024-03-18T20:46:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.418+0000 7eff9e7fc700 1 -- 172.21.15.46:0/942966275 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800fa10 con 0x7effa8100500 2024-03-18T20:46:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.419+0000 7effaf8a0700 1 -- 172.21.15.46:0/942966275 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa81a3e20 con 0x7effa8100500 2024-03-18T20:46:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.419+0000 7effaf8a0700 1 -- 172.21.15.46:0/942966275 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa81011f0 con 0x7effa8100500 2024-03-18T20:46:24.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.421+0000 7eff9e7fc700 1 -- 172.21.15.46:0/942966275 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7eff98004cd0 con 0x7effa8100500 2024-03-18T20:46:24.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.421+0000 7eff9e7fc700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff940a1290 0x7eff940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.421+0000 7effad63c700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff940a1290 0x7eff940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.421+0000 7eff9e7fc700 1 -- 172.21.15.46:0/942966275 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7eff980ce130 con 0x7effa8100500 2024-03-18T20:46:24.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.422+0000 7effad63c700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff940a1290 0x7eff940a3750 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7effa80fe0f0 tx=0x7effa400b500 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.425+0000 7eff9e7fc700 1 -- 172.21.15.46:0/942966275 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7eff98097690 con 0x7effa8100500 2024-03-18T20:46:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:24 smithi155 ceph-mon[48344]: pgmap v320: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-03-18T20:46:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:24 smithi046 ceph-mon[43495]: pgmap v320: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-03-18T20:46:24.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.692+0000 7effaf8a0700 1 -- 172.21.15.46:0/942966275 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effa8067180 con 0x7effa8100500 2024-03-18T20:46:24.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.693+0000 7eff9e7fc700 1 -- 172.21.15.46:0/942966275 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+10916 (secure 0 0 0) 0x7eff9809acb0 con 0x7effa8100500 2024-03-18T20:46:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:46:24.696 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:45:43.243732+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:46:24.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.695+0000 7effaf8a0700 1 -- 172.21.15.46:0/942966275 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff940a1290 msgr2=0x7eff940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.695+0000 7effaf8a0700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff940a1290 0x7eff940a3750 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7effa80fe0f0 tx=0x7effa400b500 comp rx=0 tx=0).stop 2024-03-18T20:46:24.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.695+0000 7effaf8a0700 1 -- 172.21.15.46:0/942966275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa8100500 msgr2=0x7effa8197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.695+0000 7effaf8a0700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa8100500 0x7effa8197690 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7eff980098e0 tx=0x7eff9800b9a0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.695+0000 7effaf8a0700 1 -- 172.21.15.46:0/942966275 shutdown_connections 2024-03-18T20:46:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.695+0000 7effaf8a0700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7eff940a1290 0x7eff940a3750 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.695+0000 7effaf8a0700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effa80ffb40 0x7effa8197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.695+0000 7effaf8a0700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa8105c00 0x7effa819b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.695+0000 7effaf8a0700 1 --2- 172.21.15.46:0/942966275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa8100500 0x7effa8197690 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.695+0000 7effaf8a0700 1 -- 172.21.15.46:0/942966275 >> 172.21.15.46:0/942966275 conn(0x7effa80fb5f0 msgr2=0x7effa80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.695+0000 7effaf8a0700 1 -- 172.21.15.46:0/942966275 shutdown_connections 2024-03-18T20:46:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.695+0000 7effaf8a0700 1 -- 172.21.15.46:0/942966275 wait complete. 2024-03-18T20:46:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 49 2024-03-18T20:46:24.712 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:46:24.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:46:24.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 381255 3.67 MB/sec execute 439 sec latency 3310.279 ms 2024-03-18T20:46:24.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:24 smithi073 ceph-mon[47384]: pgmap v320: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 1.8 MiB/s wr, 175 op/s 2024-03-18T20:46:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.850+0000 7f56e8e09700 1 -- 172.21.15.46:0/2633096443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56e4100f70 msgr2=0x7f56e4101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.850+0000 7f56e8e09700 1 --2- 172.21.15.46:0/2633096443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56e4100f70 0x7f56e4101350 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f56d40098f0 tx=0x7f56d4009c00 comp rx=0 tx=0).stop 2024-03-18T20:46:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.850+0000 7f56e8e09700 1 -- 172.21.15.46:0/2633096443 shutdown_connections 2024-03-18T20:46:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.850+0000 7f56e8e09700 1 --2- 172.21.15.46:0/2633096443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56e4106090 0x7f56e410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.850+0000 7f56e8e09700 1 --2- 172.21.15.46:0/2633096443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56e4100f70 0x7f56e4101350 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.850+0000 7f56e8e09700 1 --2- 172.21.15.46:0/2633096443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e4101920 0x7f56e4105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.850+0000 7f56e8e09700 1 -- 172.21.15.46:0/2633096443 >> 172.21.15.46:0/2633096443 conn(0x7f56e40fc7f0 msgr2=0x7f56e40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.850+0000 7f56e8e09700 1 -- 172.21.15.46:0/2633096443 shutdown_connections 2024-03-18T20:46:24.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.850+0000 7f56e8e09700 1 -- 172.21.15.46:0/2633096443 wait complete. 2024-03-18T20:46:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.851+0000 7f56e8e09700 1 Processor -- start 2024-03-18T20:46:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.851+0000 7f56e8e09700 1 -- start start 2024-03-18T20:46:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e8e09700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56e4100f70 0x7f56e40730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e8e09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e4106090 0x7f56e40735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e8e09700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56e4073b30 0x7f56e41a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e8e09700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41a5a50 con 0x7f56e4106090 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e8e09700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41a5bc0 con 0x7f56e4100f70 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e8e09700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41a5d80 con 0x7f56e4073b30 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e4106090 0x7f56e40735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e4106090 0x7f56e40735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49186/0 (socket says 172.21.15.46:49186) 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e1d9b700 1 -- 172.21.15.46:0/2690031440 learned_addr learned my addr 172.21.15.46:0/2690031440 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e259c700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56e4100f70 0x7f56e40730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e2d9d700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56e4073b30 0x7f56e41a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e1d9b700 1 -- 172.21.15.46:0/2690031440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56e4073b30 msgr2=0x7f56e41a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e1d9b700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56e4073b30 0x7f56e41a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e1d9b700 1 -- 172.21.15.46:0/2690031440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56e4100f70 msgr2=0x7f56e40730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e1d9b700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56e4100f70 0x7f56e40730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e1d9b700 1 -- 172.21.15.46:0/2690031440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d80095f0 con 0x7f56e4106090 2024-03-18T20:46:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e259c700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56e4100f70 0x7f56e40730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.852+0000 7f56e1d9b700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e4106090 0x7f56e40735f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f56d800ec70 tx=0x7f56d800ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.853+0000 7f56d37fe700 1 -- 172.21.15.46:0/2690031440 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d80099d0 con 0x7f56e4106090 2024-03-18T20:46:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.853+0000 7f56e8e09700 1 -- 172.21.15.46:0/2690031440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d4009520 con 0x7f56e4106090 2024-03-18T20:46:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.853+0000 7f56d37fe700 1 -- 172.21.15.46:0/2690031440 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d80044e0 con 0x7f56e4106090 2024-03-18T20:46:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.853+0000 7f56d37fe700 1 -- 172.21.15.46:0/2690031440 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8010430 con 0x7f56e4106090 2024-03-18T20:46:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.853+0000 7f56e8e09700 1 -- 172.21.15.46:0/2690031440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e41a6430 con 0x7f56e4106090 2024-03-18T20:46:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.855+0000 7f56d37fe700 1 -- 172.21.15.46:0/2690031440 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f56d8010590 con 0x7f56e4106090 2024-03-18T20:46:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.856+0000 7f56d37fe700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f56cc0a1290 0x7f56cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.856+0000 7f56d37fe700 1 -- 172.21.15.46:0/2690031440 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f56d8014070 con 0x7f56e4106090 2024-03-18T20:46:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.856+0000 7f56e259c700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f56cc0a1290 0x7f56cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.857+0000 7f56f4f9eb80 1 Processor -- start 2024-03-18T20:46:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.857+0000 7f56e259c700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f56cc0a1290 0x7f56cc0a3750 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009fd0 tx=0x7f56d4019040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.857+0000 7f56f4f9eb80 1 -- start start 2024-03-18T20:46:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.857+0000 7f56f4f9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b60e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.857+0000 7f56f4f9eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 0x5619f7b6bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.857+0000 7f56f4f9eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b6c220 0x5619f7b6e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.857+0000 7f56f4f9eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f7aba990 con 0x5619f7b61380 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.857+0000 7f56f4f9eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f7ab8350 con 0x5619f7b6c220 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.857+0000 7f56f4f9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f7ab3c40 con 0x5619f7b60a60 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.857+0000 7f56e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 0x5619f7b6bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.857+0000 7f56e259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b60e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.858+0000 7f56e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 0x5619f7b6bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49198/0 (socket says 172.21.15.46:49198) 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.858+0000 7f56e1d9b700 1 -- 172.21.15.46:0/54993084 learned_addr learned my addr 172.21.15.46:0/54993084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.858+0000 7f56e259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b60e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:47466/0 (socket says 172.21.15.46:47466) 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.858+0000 7f56e2d9d700 1 --2- 172.21.15.46:0/54993084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b6c220 0x5619f7b6e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.858+0000 7f56e259c700 1 -- 172.21.15.46:0/54993084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b6c220 msgr2=0x5619f7b6e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.858+0000 7f56e259c700 1 --2- 172.21.15.46:0/54993084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b6c220 0x5619f7b6e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.858+0000 7f56e259c700 1 -- 172.21.15.46:0/54993084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 msgr2=0x5619f7b6bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.858+0000 7f56e259c700 1 --2- 172.21.15.46:0/54993084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 0x5619f7b6bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.858+0000 7f56e259c700 1 -- 172.21.15.46:0/54993084 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619f79353b0 con 0x5619f7b60a60 2024-03-18T20:46:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.858+0000 7f56e259c700 1 --2- 172.21.15.46:0/54993084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b60e40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f56d40098c0 tx=0x7f56d401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=532d5170bbc7b995 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.858+0000 7f56d0ff9700 1 -- 172.21.15.46:0/54993084 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d400b700 con 0x5619f7b60a60 2024-03-18T20:46:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.859+0000 7f56d0ff9700 1 -- 172.21.15.46:0/54993084 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d401d040 con 0x5619f7b60a60 2024-03-18T20:46:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.859+0000 7f56d0ff9700 1 -- 172.21.15.46:0/54993084 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d402d760 con 0x5619f7b60a60 2024-03-18T20:46:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.859+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/54993084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 msgr2=0x5619f7b60e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.859+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/54993084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b60e40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f56d40098c0 tx=0x7f56d401f000 comp rx=0 tx=0).stop 2024-03-18T20:46:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.859+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/54993084 shutdown_connections 2024-03-18T20:46:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.859+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/54993084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b6c220 0x5619f7b6e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.859+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/54993084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b60e40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.859+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/54993084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 0x5619f7b6bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.859+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/54993084 >> 172.21.15.46:0/54993084 conn(0x5619f7ad7220 msgr2=0x5619f7b5d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.860+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/54993084 shutdown_connections 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.860+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/54993084 wait complete. 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.860+0000 7f56f4f9eb80 1 -- start start 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.860+0000 7f56f4f9eb80 1 Processor -- start 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.860+0000 7f56f4f9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b6bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.860+0000 7f56f4f9eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 0x5619f7b68f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.860+0000 7f56f4f9eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b6c220 0x5619f7b695e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56f4f9eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f7a4fd80 con 0x5619f7b61380 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56f4f9eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f7a4fa20 con 0x5619f7b6c220 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56f4f9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f79376a0 con 0x5619f7b60a60 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56f4f9eb80 10 client.0 _ll_register_callbacks cb 0x5619f78bb3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 0x5619f7b68f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56e2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b6c220 0x5619f7b695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 0x5619f7b68f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49202/0 (socket says 172.21.15.46:49202) 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56e1d9b700 1 -- 172.21.15.46:0/3050901233 learned_addr learned my addr 172.21.15.46:0/3050901233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56e259c700 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b6bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56e1d9b700 1 -- 172.21.15.46:0/3050901233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 msgr2=0x5619f7b6bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56e1d9b700 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b6bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56e1d9b700 1 -- 172.21.15.46:0/3050901233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b6c220 msgr2=0x5619f7b695e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56e1d9b700 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b6c220 0x5619f7b695e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56e1d9b700 1 -- 172.21.15.46:0/3050901233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e410e480 con 0x5619f7b61380 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56e2d9d700 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b6c220 0x5619f7b695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56e1d9b700 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 0x5619f7b68f60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f56d8096d20 tx=0x7f56d8099df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae075e1013482d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.861+0000 7f56c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56c9ffb700 1 -- 172.21.15.46:0/3050901233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8063020 con 0x5619f7b61380 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56c9ffb700 1 -- 172.21.15.46:0/3050901233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f56d809a9b0 con 0x5619f7b61380 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/3050901233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619f7b54ad0 con 0x5619f7b61380 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56c9ffb700 1 -- 172.21.15.46:0/3050901233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56d809a2b0 con 0x5619f7b61380 2024-03-18T20:46:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56f4f9eb80 10 client.19116 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/3050901233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5619f78e1b60 con 0x5619f7b61380 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56c9ffb700 1 -- 172.21.15.46:0/3050901233 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619f7b54ad0 con 0x5619f7b61380 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56c9ffb700 1 -- 172.21.15.46:0/3050901233 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f56d809a510 con 0x5619f7b61380 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56f4f9eb80 10 client.19116 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56f4f9eb80 10 client.19116 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56f4f9eb80 20 client.19116 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5619f7b7be20 0x5619f7b7e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56f4f9eb80 4 client.19116 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.862+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/3050901233 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619f7aac700 con 0x5619f7b7be20 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.863+0000 7f56e259c700 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5619f7b7be20 0x5619f7b7e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.863+0000 7f56e259c700 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5619f7b7be20 0x5619f7b7e240 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1abeded3676c720 server_cookie=4a00a07fb650f5dc in_seq=0 out_seq=0 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.863+0000 7f56c9ffb700 10 client.19116 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56c9ffb700 1 -- 172.21.15.46:0/3050901233 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5619f7aac700 con 0x5619f7b7be20 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56c9ffb700 10 client.19116 handle_command_reply: tid=0 2024-03-18T20:46:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56f4f9eb80 1 client.19116 shutdown 2024-03-18T20:46:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/3050901233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 msgr2=0x5619f7b68f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 0x5619f7b68f60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f56d8096d20 tx=0x7f56d8099df0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/3050901233 shutdown_connections 2024-03-18T20:46:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5619f7b7be20 0x5619f7b7e240 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b6c220 0x5619f7b695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b6bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/3050901233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b61380 0x5619f7b68f60 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56c9ffb700 0 client.19116 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:46:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/3050901233 >> 172.21.15.46:0/3050901233 conn(0x5619f7ad7220 msgr2=0x5619f7b5cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/3050901233 shutdown_connections 2024-03-18T20:46:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/3050901233 wait complete. 2024-03-18T20:46:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.864+0000 7f56f4f9eb80 20 client.19116 trim_cache size 0 max 16384 2024-03-18T20:46:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56f4f9eb80 1 Processor -- start 2024-03-18T20:46:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56f4f9eb80 1 -- start start 2024-03-18T20:46:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56f4f9eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b60a60 0x5619f7b639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56f4f9eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b61380 0x5619f7b72df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56f4f9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b6c220 0x5619f7b5cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56f4f9eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f7b27940 con 0x5619f7b60a60 2024-03-18T20:46:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56f4f9eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f7b57210 con 0x5619f7b61380 2024-03-18T20:46:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56f4f9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f7b595e0 con 0x5619f7b6c220 2024-03-18T20:46:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56e259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b60a60 0x5619f7b639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56e259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b60a60 0x5619f7b639c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49214/0 (socket says 172.21.15.46:49214) 2024-03-18T20:46:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56e259c700 1 -- 172.21.15.46:0/1315456592 learned_addr learned my addr 172.21.15.46:0/1315456592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56e1d9b700 1 --2- 172.21.15.46:0/1315456592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b61380 0x5619f7b72df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56e2d9d700 1 --2- 172.21.15.46:0/1315456592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b6c220 0x5619f7b5cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56e259c700 1 -- 172.21.15.46:0/1315456592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b6c220 msgr2=0x5619f7b5cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.879+0000 7f56e259c700 1 --2- 172.21.15.46:0/1315456592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b6c220 0x5619f7b5cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.880+0000 7f56e259c700 1 -- 172.21.15.46:0/1315456592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b61380 msgr2=0x5619f7b72df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.880+0000 7f56e259c700 1 --2- 172.21.15.46:0/1315456592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b61380 0x5619f7b72df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.880+0000 7f56e259c700 1 -- 172.21.15.46:0/1315456592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619f7a79ea0 con 0x5619f7b60a60 2024-03-18T20:46:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.880+0000 7f56e1d9b700 1 --2- 172.21.15.46:0/1315456592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b61380 0x5619f7b72df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.880+0000 7f56e259c700 1 --2- 172.21.15.46:0/1315456592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b60a60 0x5619f7b639c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f56d40098f0 tx=0x7f56d4023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20b1843c4a2e58a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.880+0000 7f56d17fa700 1 -- 172.21.15.46:0/1315456592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40050c0 con 0x5619f7b60a60 2024-03-18T20:46:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.880+0000 7f56d17fa700 1 -- 172.21.15.46:0/1315456592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d40053e0 con 0x5619f7b60a60 2024-03-18T20:46:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.880+0000 7f56d17fa700 1 -- 172.21.15.46:0/1315456592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d400b760 con 0x5619f7b60a60 2024-03-18T20:46:24.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.881+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/1315456592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b60a60 msgr2=0x5619f7b639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.881+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/1315456592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b60a60 0x5619f7b639c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f56d40098f0 tx=0x7f56d4023000 comp rx=0 tx=0).stop 2024-03-18T20:46:24.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.881+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/1315456592 shutdown_connections 2024-03-18T20:46:24.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.881+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/1315456592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b60a60 0x5619f7b639c0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.881+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/1315456592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b6c220 0x5619f7b5cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.881+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/1315456592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b61380 0x5619f7b72df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.881+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/1315456592 >> 172.21.15.46:0/1315456592 conn(0x5619f7ad7220 msgr2=0x5619f7b64280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.881+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/1315456592 shutdown_connections 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.881+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/1315456592 wait complete. 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.881+0000 7f56f4f9eb80 1 -- start start 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.881+0000 7f56f4f9eb80 1 Processor -- start 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56f4f9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b70fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56f4f9eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b61380 0x5619f7b71500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56f4f9eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b6c220 0x5619f7bcec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56f4f9eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f7b71bc0 con 0x5619f7b6c220 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56f4f9eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f7bcf190 con 0x5619f7b61380 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56f4f9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f7bcf2d0 con 0x5619f7b60a60 2024-03-18T20:46:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56f4f9eb80 10 client.0 _ll_register_callbacks cb 0x5619f78bb3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56e2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b6c220 0x5619f7bcec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56e2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b6c220 0x5619f7bcec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49224/0 (socket says 172.21.15.46:49224) 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56e2d9d700 1 -- 172.21.15.46:0/2644180456 learned_addr learned my addr 172.21.15.46:0/2644180456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56e1d9b700 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b61380 0x5619f7b71500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56e259c700 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b70fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56e2d9d700 1 -- 172.21.15.46:0/2644180456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 msgr2=0x5619f7b70fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56e2d9d700 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b70fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56e2d9d700 1 -- 172.21.15.46:0/2644180456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b61380 msgr2=0x5619f7b71500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56e2d9d700 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b61380 0x5619f7b71500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56e2d9d700 1 -- 172.21.15.46:0/2644180456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619f78e1b60 con 0x5619f7b6c220 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56e259c700 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b70fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56e2d9d700 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b6c220 0x5619f7bcec50 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f56dc018740 tx=0x7f56dc018a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa8b7422a8e7b52 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.882+0000 7f56c9ffb700 1 -- 172.21.15.46:0/2644180456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc01d070 con 0x5619f7b6c220 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56c9ffb700 1 -- 172.21.15.46:0/2644180456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f56dc003a00 con 0x5619f7b6c220 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/2644180456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56d401d040 con 0x5619f7b6c220 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56c9ffb700 1 -- 172.21.15.46:0/2644180456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56dc00e040 con 0x5619f7b6c220 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56f4f9eb80 10 client.19122 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/2644180456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5619f79353b0 con 0x5619f7b6c220 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56c9ffb700 1 -- 172.21.15.46:0/2644180456 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d401d040 con 0x5619f7b6c220 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56c9ffb700 1 -- 172.21.15.46:0/2644180456 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f56dc0074a0 con 0x5619f7b6c220 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56f4f9eb80 10 client.19122 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56f4f9eb80 10 client.19122 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56f4f9eb80 20 client.19122 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5619f7bd0b20 0x5619f7bd2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56f4f9eb80 4 client.19122 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/2644180456 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5619f7902cf0 con 0x5619f7bd0b20 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.883+0000 7f56e259c700 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5619f7bd0b20 0x5619f7bd2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56e259c700 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5619f7bd0b20 0x5619f7bd2f40 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e2a8f8bdfc47fd7 server_cookie=aadc38f594d0d3ea in_seq=0 out_seq=0 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56c9ffb700 10 client.19122 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56c9ffb700 1 -- 172.21.15.46:0/2644180456 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5619f7902cf0 con 0x5619f7bd0b20 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56c9ffb700 10 client.19122 handle_command_reply: tid=0 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56f4f9eb80 1 client.19122 shutdown 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/2644180456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b6c220 msgr2=0x5619f7bcec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b6c220 0x5619f7bcec50 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f56dc018740 tx=0x7f56dc018a50 comp rx=0 tx=0).stop 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/2644180456 shutdown_connections 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5619f7bd0b20 0x5619f7bd2f40 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5619f7b6c220 0x5619f7bcec50 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5619f7b60a60 0x5619f7b70fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56f4f9eb80 1 --2- 172.21.15.46:0/2644180456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619f7b61380 0x5619f7b71500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/2644180456 >> 172.21.15.46:0/2644180456 conn(0x5619f7ad7220 msgr2=0x5619f7b5cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56c9ffb700 0 client.19122 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:46:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/2644180456 shutdown_connections 2024-03-18T20:46:24.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56f4f9eb80 1 -- 172.21.15.46:0/2644180456 wait complete. 2024-03-18T20:46:24.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.884+0000 7f56f4f9eb80 20 client.19122 trim_cache size 0 max 16384 2024-03-18T20:46:24.893 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:46:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.885+0000 7f56e8e09700 1 -- 172.21.15.46:0/2690031440 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f56cc0a1290 msgr2=0x7f56cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.885+0000 7f56e8e09700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f56cc0a1290 0x7f56cc0a3750 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009fd0 tx=0x7f56d4019040 comp rx=0 tx=0).stop 2024-03-18T20:46:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.885+0000 7f56e8e09700 1 -- 172.21.15.46:0/2690031440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e4106090 msgr2=0x7f56e40735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.885+0000 7f56e8e09700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e4106090 0x7f56e40735f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f56d800ec70 tx=0x7f56d800ef80 comp rx=0 tx=0).stop 2024-03-18T20:46:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.885+0000 7f56e8e09700 1 -- 172.21.15.46:0/2690031440 shutdown_connections 2024-03-18T20:46:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.885+0000 7f56e8e09700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f56cc0a1290 0x7f56cc0a3750 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.885+0000 7f56e8e09700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56e4073b30 0x7f56e41a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.885+0000 7f56e8e09700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56e4100f70 0x7f56e40730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.885+0000 7f56e8e09700 1 --2- 172.21.15.46:0/2690031440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56e4106090 0x7f56e40735f0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.885+0000 7f56e8e09700 1 -- 172.21.15.46:0/2690031440 >> 172.21.15.46:0/2690031440 conn(0x7f56e40fc7f0 msgr2=0x7f56e40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.885+0000 7f56e8e09700 1 -- 172.21.15.46:0/2690031440 shutdown_connections 2024-03-18T20:46:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:24.885+0000 7f56e8e09700 1 -- 172.21.15.46:0/2690031440 wait complete. 2024-03-18T20:46:24.903 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:46:24.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:46:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.047+0000 7ff100e15700 1 -- 172.21.15.46:0/2672087929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc0687f0 msgr2=0x7ff0fc10db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.047+0000 7ff100e15700 1 --2- 172.21.15.46:0/2672087929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc0687f0 0x7ff0fc10db50 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400b0a0 tx=0x7ff0f400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.047+0000 7ff100e15700 1 -- 172.21.15.46:0/2672087929 shutdown_connections 2024-03-18T20:46:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.047+0000 7ff100e15700 1 --2- 172.21.15.46:0/2672087929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc0687f0 0x7ff0fc10db50 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.047+0000 7ff100e15700 1 --2- 172.21.15.46:0/2672087929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc105ce0 0x7ff0fc1060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.047+0000 7ff100e15700 1 --2- 172.21.15.46:0/2672087929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0fc067e30 0x7ff0fc0682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.047+0000 7ff100e15700 1 -- 172.21.15.46:0/2672087929 >> 172.21.15.46:0/2672087929 conn(0x7ff0fc076d40 msgr2=0x7ff0fc077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.047+0000 7ff100e15700 1 -- 172.21.15.46:0/2672087929 shutdown_connections 2024-03-18T20:46:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.047+0000 7ff100e15700 1 -- 172.21.15.46:0/2672087929 wait complete. 2024-03-18T20:46:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.048+0000 7ff100e15700 1 Processor -- start 2024-03-18T20:46:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.048+0000 7ff100e15700 1 -- start start 2024-03-18T20:46:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff100e15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc067e30 0x7ff0fc197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff100e15700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0fc0687f0 0x7ff0fc197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff100e15700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc105ce0 0x7ff0fc19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff100e15700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc19c230 con 0x7ff0fc067e30 2024-03-18T20:46:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff100e15700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc19c3a0 con 0x7ff0fc105ce0 2024-03-18T20:46:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff100e15700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc19c560 con 0x7ff0fc0687f0 2024-03-18T20:46:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff0fa59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc067e30 0x7ff0fc197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff0fa59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc067e30 0x7ff0fc197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49252/0 (socket says 172.21.15.46:49252) 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff0fa59c700 1 -- 172.21.15.46:0/2081879846 learned_addr learned my addr 172.21.15.46:0/2081879846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc105ce0 0x7ff0fc19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff0f9d9b700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0fc0687f0 0x7ff0fc197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff0fad9d700 1 -- 172.21.15.46:0/2081879846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0fc0687f0 msgr2=0x7ff0fc197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0fc0687f0 0x7ff0fc197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff0fad9d700 1 -- 172.21.15.46:0/2081879846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc067e30 msgr2=0x7ff0fc197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc067e30 0x7ff0fc197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff0fad9d700 1 -- 172.21.15.46:0/2081879846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f4009d00 con 0x7ff0fc105ce0 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff0fa59c700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc067e30 0x7ff0fc197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.049+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc105ce0 0x7ff0fc19bbf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400b070 tx=0x7ff0f400bdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.050+0000 7ff0eb7fe700 1 -- 172.21.15.46:0/2081879846 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f400e040 con 0x7ff0fc105ce0 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.050+0000 7ff0eb7fe700 1 -- 172.21.15.46:0/2081879846 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f401d470 con 0x7ff0fc105ce0 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.050+0000 7ff100e15700 1 -- 172.21.15.46:0/2081879846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc19c7e0 con 0x7ff0fc105ce0 2024-03-18T20:46:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.050+0000 7ff0eb7fe700 1 -- 172.21.15.46:0/2081879846 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40079d0 con 0x7ff0fc105ce0 2024-03-18T20:46:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.050+0000 7ff100e15700 1 -- 172.21.15.46:0/2081879846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc1a4080 con 0x7ff0fc105ce0 2024-03-18T20:46:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.052+0000 7ff0eb7fe700 1 -- 172.21.15.46:0/2081879846 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff0f401d5e0 con 0x7ff0fc105ce0 2024-03-18T20:46:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.053+0000 7ff0eb7fe700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff0e40a1290 0x7ff0e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.053+0000 7ff0fa59c700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff0e40a1290 0x7ff0e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.053+0000 7ff0eb7fe700 1 -- 172.21.15.46:0/2081879846 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff0f40cdfe0 con 0x7ff0fc105ce0 2024-03-18T20:46:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.053+0000 7ff0fa59c700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff0e40a1290 0x7ff0e40a3750 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec011550 tx=0x7ff0ec009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff10cfaab80 1 Processor -- start 2024-03-18T20:46:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff10cfaab80 1 -- start start 2024-03-18T20:46:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff10cfaab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a395a60 0x555c9a395e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff10cfaab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a396380 0x555c9a3a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff10cfaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a3a1220 0x555c9a3a3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff10cfaab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9a2ef990 con 0x555c9a395a60 2024-03-18T20:46:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff10cfaab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9a2ed350 con 0x555c9a396380 2024-03-18T20:46:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff10cfaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9a2e8c40 con 0x555c9a3a1220 2024-03-18T20:46:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff0fa59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a395a60 0x555c9a395e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff0fa59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a395a60 0x555c9a395e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49258/0 (socket says 172.21.15.46:49258) 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff0fa59c700 1 -- 172.21.15.46:0/318442810 learned_addr learned my addr 172.21.15.46:0/318442810 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff0f9d9b700 1 --2- 172.21.15.46:0/318442810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a396380 0x555c9a3a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.054+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/318442810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a3a1220 0x555c9a3a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.055+0000 7ff0f9d9b700 1 -- 172.21.15.46:0/318442810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a3a1220 msgr2=0x555c9a3a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.055+0000 7ff0f9d9b700 1 --2- 172.21.15.46:0/318442810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a3a1220 0x555c9a3a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.055+0000 7ff0f9d9b700 1 -- 172.21.15.46:0/318442810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a395a60 msgr2=0x555c9a395e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.055+0000 7ff0f9d9b700 1 --2- 172.21.15.46:0/318442810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a395a60 0x555c9a395e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.055+0000 7ff0f9d9b700 1 -- 172.21.15.46:0/318442810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c9a16a3b0 con 0x555c9a396380 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.055+0000 7ff0fa59c700 1 --2- 172.21.15.46:0/318442810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a395a60 0x555c9a395e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.055+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/318442810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a3a1220 0x555c9a3a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.055+0000 7ff0f9d9b700 1 --2- 172.21.15.46:0/318442810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a396380 0x555c9a3a0ce0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0005950 tx=0x7ff0f000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a5da2a6c94d388 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.055+0000 7ff0e8ff9700 1 -- 172.21.15.46:0/318442810 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0010c70 con 0x555c9a396380 2024-03-18T20:46:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.055+0000 7ff0e8ff9700 1 -- 172.21.15.46:0/318442810 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f0015070 con 0x555c9a396380 2024-03-18T20:46:25.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.055+0000 7ff0e8ff9700 1 -- 172.21.15.46:0/318442810 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0005020 con 0x555c9a396380 2024-03-18T20:46:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.056+0000 7ff10cfaab80 1 -- 172.21.15.46:0/318442810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a396380 msgr2=0x555c9a3a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.056+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/318442810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a396380 0x555c9a3a0ce0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0005950 tx=0x7ff0f000bf60 comp rx=0 tx=0).stop 2024-03-18T20:46:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.056+0000 7ff10cfaab80 1 -- 172.21.15.46:0/318442810 shutdown_connections 2024-03-18T20:46:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.056+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/318442810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a395a60 0x555c9a395e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.056+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/318442810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a3a1220 0x555c9a3a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.056+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/318442810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a396380 0x555c9a3a0ce0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.056+0000 7ff10cfaab80 1 -- 172.21.15.46:0/318442810 >> 172.21.15.46:0/318442810 conn(0x555c9a30c220 msgr2=0x555c9a392020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.056+0000 7ff10cfaab80 1 -- 172.21.15.46:0/318442810 shutdown_connections 2024-03-18T20:46:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.056+0000 7ff10cfaab80 1 -- 172.21.15.46:0/318442810 wait complete. 2024-03-18T20:46:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.056+0000 7ff10cfaab80 1 -- start start 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.056+0000 7ff10cfaab80 1 Processor -- start 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff10cfaab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a395a60 0x555c9a3a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff10cfaab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a396380 0x555c9a39df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff10cfaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a3a1220 0x555c9a39e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff10cfaab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9a284d80 con 0x555c9a396380 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff10cfaab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9a284a20 con 0x555c9a395a60 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff10cfaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9a16c6a0 con 0x555c9a3a1220 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff10cfaab80 10 client.0 _ll_register_callbacks cb 0x555c9a0f03a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0f9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a396380 0x555c9a39df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0fa59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a395a60 0x555c9a3a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0f9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a396380 0x555c9a39df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49264/0 (socket says 172.21.15.46:49264) 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0f9d9b700 1 -- 172.21.15.46:0/585651313 learned_addr learned my addr 172.21.15.46:0/585651313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a3a1220 0x555c9a39e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0fa59c700 1 -- 172.21.15.46:0/585651313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a3a1220 msgr2=0x555c9a39e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0fa59c700 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a3a1220 0x555c9a39e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0fa59c700 1 -- 172.21.15.46:0/585651313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a396380 msgr2=0x555c9a39df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0fa59c700 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a396380 0x555c9a39df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0fa59c700 1 -- 172.21.15.46:0/585651313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ec017070 con 0x555c9a395a60 2024-03-18T20:46:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0f9d9b700 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a396380 0x555c9a39df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0fa59c700 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a395a60 0x555c9a3a0bd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec011ca0 tx=0x7ff0ec0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d36ba35f32d456 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.057+0000 7ff0e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:46:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff0e1ffb700 1 -- 172.21.15.46:0/585651313 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec022070 con 0x555c9a395a60 2024-03-18T20:46:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff0e1ffb700 1 -- 172.21.15.46:0/585651313 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff0ec01dce0 con 0x555c9a395a60 2024-03-18T20:46:25.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff10cfaab80 1 -- 172.21.15.46:0/585651313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c9a389ad0 con 0x555c9a395a60 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff0e1ffb700 1 -- 172.21.15.46:0/585651313 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0ec01dea0 con 0x555c9a395a60 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff10cfaab80 10 client.28763 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff10cfaab80 1 -- 172.21.15.46:0/585651313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7ff0f000b8e0 con 0x555c9a395a60 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff0e1ffb700 1 -- 172.21.15.46:0/585651313 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0ec033020 con 0x555c9a395a60 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff0e1ffb700 1 -- 172.21.15.46:0/585651313 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff0ec01c520 con 0x555c9a395a60 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff10cfaab80 10 client.28763 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff10cfaab80 10 client.28763 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff10cfaab80 20 client.28763 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555c9a3b0e50 0x555c9a3b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff10cfaab80 4 client.28763 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff10cfaab80 1 -- 172.21.15.46:0/585651313 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c9a2e1700 con 0x555c9a3b0e50 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.058+0000 7ff0f9d9b700 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555c9a3b0e50 0x555c9a3b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.059+0000 7ff0e1ffb700 10 client.28763 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.059+0000 7ff0f9d9b700 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555c9a3b0e50 0x555c9a3b3270 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64b86a3d7badddd6 server_cookie=232b6f5364864e5d in_seq=0 out_seq=0 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff0e1ffb700 1 -- 172.21.15.46:0/585651313 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555c9a2e1700 con 0x555c9a3b0e50 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff0e1ffb700 10 client.28763 handle_command_reply: tid=0 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff10cfaab80 1 client.28763 shutdown 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff10cfaab80 1 -- 172.21.15.46:0/585651313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a395a60 msgr2=0x555c9a3a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a395a60 0x555c9a3a0bd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec011ca0 tx=0x7ff0ec0047c0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff10cfaab80 1 -- 172.21.15.46:0/585651313 shutdown_connections 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a3a1220 0x555c9a39e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a395a60 0x555c9a3a0bd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555c9a3b0e50 0x555c9a3b3270 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/585651313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a396380 0x555c9a39df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff10cfaab80 1 -- 172.21.15.46:0/585651313 >> 172.21.15.46:0/585651313 conn(0x555c9a30c220 msgr2=0x555c9a391c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff0e1ffb700 0 client.28763 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff10cfaab80 1 -- 172.21.15.46:0/585651313 shutdown_connections 2024-03-18T20:46:25.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff10cfaab80 1 -- 172.21.15.46:0/585651313 wait complete. 2024-03-18T20:46:25.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.060+0000 7ff10cfaab80 20 client.28763 trim_cache size 0 max 16384 2024-03-18T20:46:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.075+0000 7ff10cfaab80 1 Processor -- start 2024-03-18T20:46:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.075+0000 7ff10cfaab80 1 -- start start 2024-03-18T20:46:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.075+0000 7ff10cfaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a395a60 0x555c9a3a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.075+0000 7ff10cfaab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a396380 0x555c9a391b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.075+0000 7ff10cfaab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a3a1220 0x555c9a392090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.075+0000 7ff10cfaab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9a396d30 con 0x555c9a3a1220 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.075+0000 7ff10cfaab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9a38e5e0 con 0x555c9a396380 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.075+0000 7ff10cfaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9a3961c0 con 0x555c9a395a60 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.075+0000 7ff0fad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a3a1220 0x555c9a392090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.075+0000 7ff0fad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a3a1220 0x555c9a392090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49270/0 (socket says 172.21.15.46:49270) 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.075+0000 7ff0fad9d700 1 -- 172.21.15.46:0/1019699851 learned_addr learned my addr 172.21.15.46:0/1019699851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.075+0000 7ff0f9d9b700 1 --2- 172.21.15.46:0/1019699851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a396380 0x555c9a391b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.076+0000 7ff0fa59c700 1 --2- 172.21.15.46:0/1019699851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a395a60 0x555c9a3a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.076+0000 7ff0fad9d700 1 -- 172.21.15.46:0/1019699851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a395a60 msgr2=0x555c9a3a7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.076+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/1019699851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a395a60 0x555c9a3a7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.076+0000 7ff0fad9d700 1 -- 172.21.15.46:0/1019699851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a396380 msgr2=0x555c9a391b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.076+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/1019699851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a396380 0x555c9a391b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.076+0000 7ff0fad9d700 1 -- 172.21.15.46:0/1019699851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c9a116b60 con 0x555c9a3a1220 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.076+0000 7ff0f9d9b700 1 --2- 172.21.15.46:0/1019699851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a396380 0x555c9a391b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.076+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/1019699851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a3a1220 0x555c9a392090 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ff0f401d890 tx=0x7ff0f401dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20ce0e92ed395e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.076+0000 7ff0e97fa700 1 -- 172.21.15.46:0/1019699851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f409b800 con 0x555c9a3a1220 2024-03-18T20:46:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.076+0000 7ff0e97fa700 1 -- 172.21.15.46:0/1019699851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f409b100 con 0x555c9a3a1220 2024-03-18T20:46:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.076+0000 7ff0e97fa700 1 -- 172.21.15.46:0/1019699851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40cb930 con 0x555c9a3a1220 2024-03-18T20:46:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.077+0000 7ff10cfaab80 1 -- 172.21.15.46:0/1019699851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a3a1220 msgr2=0x555c9a392090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.077+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/1019699851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a3a1220 0x555c9a392090 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ff0f401d890 tx=0x7ff0f401dc80 comp rx=0 tx=0).stop 2024-03-18T20:46:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.077+0000 7ff10cfaab80 1 -- 172.21.15.46:0/1019699851 shutdown_connections 2024-03-18T20:46:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.077+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/1019699851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a3a1220 0x555c9a392090 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.077+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/1019699851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a395a60 0x555c9a3a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.077+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/1019699851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a396380 0x555c9a391b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.077+0000 7ff10cfaab80 1 -- 172.21.15.46:0/1019699851 >> 172.21.15.46:0/1019699851 conn(0x555c9a30c220 msgr2=0x555c9a399280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.077+0000 7ff10cfaab80 1 -- 172.21.15.46:0/1019699851 shutdown_connections 2024-03-18T20:46:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.077+0000 7ff10cfaab80 1 -- 172.21.15.46:0/1019699851 wait complete. 2024-03-18T20:46:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.077+0000 7ff10cfaab80 1 -- start start 2024-03-18T20:46:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff10cfaab80 1 Processor -- start 2024-03-18T20:46:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff10cfaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a395a60 0x555c9a3a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff10cfaab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a396380 0x555c9a3a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff10cfaab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a3a1220 0x555c9a403c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff10cfaab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9a3a7d90 con 0x555c9a396380 2024-03-18T20:46:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff10cfaab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9a404180 con 0x555c9a3a1220 2024-03-18T20:46:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff10cfaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9a4042c0 con 0x555c9a395a60 2024-03-18T20:46:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff10cfaab80 10 client.0 _ll_register_callbacks cb 0x7ff0ec00ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff0fa59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a395a60 0x555c9a3a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff0fad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a3a1220 0x555c9a403c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff0f9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a396380 0x555c9a3a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff0fad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a3a1220 0x555c9a403c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55934/0 (socket says 172.21.15.46:55934) 2024-03-18T20:46:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff0fa59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a395a60 0x555c9a3a6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:47546/0 (socket says 172.21.15.46:47546) 2024-03-18T20:46:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff0fad9d700 1 -- 172.21.15.46:0/4082429022 learned_addr learned my addr 172.21.15.46:0/4082429022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff0fad9d700 1 -- 172.21.15.46:0/4082429022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a395a60 msgr2=0x555c9a3a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/4082429022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a395a60 0x555c9a3a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff0fad9d700 1 -- 172.21.15.46:0/4082429022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a396380 msgr2=0x555c9a3a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/4082429022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a396380 0x555c9a3a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.078+0000 7ff0fad9d700 1 -- 172.21.15.46:0/4082429022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c9a16a3b0 con 0x555c9a3a1220 2024-03-18T20:46:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff0fad9d700 1 --2- 172.21.15.46:0/4082429022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a3a1220 0x555c9a403c40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff0f409adb0 tx=0x7ff0f409b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9ab9b659c233b31 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff0e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:46:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff0e1ffb700 1 -- 172.21.15.46:0/4082429022 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f409ef30 con 0x555c9a3a1220 2024-03-18T20:46:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff10cfaab80 1 -- 172.21.15.46:0/4082429022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0f0015070 con 0x555c9a3a1220 2024-03-18T20:46:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff0e1ffb700 1 -- 172.21.15.46:0/4082429022 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff0f40b9270 con 0x555c9a3a1220 2024-03-18T20:46:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff0e1ffb700 1 -- 172.21.15.46:0/4082429022 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0f40b9430 con 0x555c9a3a1220 2024-03-18T20:46:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff10cfaab80 10 client.28769 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff10cfaab80 1 -- 172.21.15.46:0/4082429022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7ff0fc1a4080 con 0x555c9a3a1220 2024-03-18T20:46:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff0e1ffb700 1 -- 172.21.15.46:0/4082429022 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f0015070 con 0x555c9a3a1220 2024-03-18T20:46:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff0e1ffb700 1 -- 172.21.15.46:0/4082429022 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff0f407d070 con 0x555c9a3a1220 2024-03-18T20:46:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff10cfaab80 10 client.28769 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff10cfaab80 10 client.28769 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:46:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff10cfaab80 20 client.28769 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/4082429022 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555c9a405560 0x555c9a407980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff10cfaab80 4 client.28769 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:46:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.079+0000 7ff10cfaab80 1 -- 172.21.15.46:0/4082429022 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c9a137cf0 con 0x555c9a405560 2024-03-18T20:46:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff0fa59c700 1 --2- 172.21.15.46:0/4082429022 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555c9a405560 0x555c9a407980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff0e1ffb700 10 client.28769 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:46:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff0fa59c700 1 --2- 172.21.15.46:0/4082429022 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555c9a405560 0x555c9a407980 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce9e32fdadd5da34 server_cookie=2e3cd94900c7dcd4 in_seq=0 out_seq=0 2024-03-18T20:46:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff0e1ffb700 1 -- 172.21.15.46:0/4082429022 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c9a137cf0 con 0x555c9a405560 2024-03-18T20:46:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff0e1ffb700 10 client.28769 handle_command_reply: tid=0 2024-03-18T20:46:25.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff10cfaab80 1 client.28769 shutdown 2024-03-18T20:46:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff10cfaab80 1 -- 172.21.15.46:0/4082429022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a3a1220 msgr2=0x555c9a403c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/4082429022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a3a1220 0x555c9a403c40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff0f409adb0 tx=0x7ff0f409b870 comp rx=0 tx=0).stop 2024-03-18T20:46:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff10cfaab80 1 -- 172.21.15.46:0/4082429022 shutdown_connections 2024-03-18T20:46:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/4082429022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c9a3a1220 0x555c9a403c40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/4082429022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c9a395a60 0x555c9a3a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/4082429022 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x555c9a405560 0x555c9a407980 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff10cfaab80 1 --2- 172.21.15.46:0/4082429022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c9a396380 0x555c9a3a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff0e1ffb700 0 client.28769 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:46:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.080+0000 7ff10cfaab80 1 -- 172.21.15.46:0/4082429022 >> 172.21.15.46:0/4082429022 conn(0x555c9a30c220 msgr2=0x555c9a39c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff10cfaab80 1 -- 172.21.15.46:0/4082429022 shutdown_connections 2024-03-18T20:46:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff10cfaab80 1 -- 172.21.15.46:0/4082429022 wait complete. 2024-03-18T20:46:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff10cfaab80 20 client.28769 trim_cache size 0 max 16384 2024-03-18T20:46:25.093 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:46:25.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff100e15700 1 -- 172.21.15.46:0/2081879846 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff0e40a1290 msgr2=0x7ff0e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff100e15700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff0e40a1290 0x7ff0e40a3750 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec011550 tx=0x7ff0ec009260 comp rx=0 tx=0).stop 2024-03-18T20:46:25.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff100e15700 1 -- 172.21.15.46:0/2081879846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc105ce0 msgr2=0x7ff0fc19bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff100e15700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc105ce0 0x7ff0fc19bbf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400b070 tx=0x7ff0f400bdd0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff100e15700 1 -- 172.21.15.46:0/2081879846 shutdown_connections 2024-03-18T20:46:25.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff100e15700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff0e40a1290 0x7ff0e40a3750 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff100e15700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0fc067e30 0x7ff0fc197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff100e15700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc105ce0 0x7ff0fc19bbf0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff100e15700 1 --2- 172.21.15.46:0/2081879846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0fc0687f0 0x7ff0fc197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff100e15700 1 -- 172.21.15.46:0/2081879846 >> 172.21.15.46:0/2081879846 conn(0x7ff0fc076d40 msgr2=0x7ff0fc0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff100e15700 1 -- 172.21.15.46:0/2081879846 shutdown_connections 2024-03-18T20:46:25.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.081+0000 7ff100e15700 1 -- 172.21.15.46:0/2081879846 wait complete. 2024-03-18T20:46:25.098 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:46:25.098 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:46:25.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.248+0000 7f79f724f700 1 -- 172.21.15.46:0/1971277722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f010a070 msgr2=0x7f79f01127c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.248+0000 7f79f724f700 1 --2- 172.21.15.46:0/1971277722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f010a070 0x7f79f01127c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009990 tx=0x7f79e4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.249+0000 7f79f724f700 1 -- 172.21.15.46:0/1971277722 shutdown_connections 2024-03-18T20:46:25.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.249+0000 7f79f724f700 1 --2- 172.21.15.46:0/1971277722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f0112d00 0x7f79f01150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.249+0000 7f79f724f700 1 --2- 172.21.15.46:0/1971277722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f0109750 0x7f79f0109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.249+0000 7f79f724f700 1 --2- 172.21.15.46:0/1971277722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f010a070 0x7f79f01127c0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.249+0000 7f79f724f700 1 -- 172.21.15.46:0/1971277722 >> 172.21.15.46:0/1971277722 conn(0x7f79f006bdf0 msgr2=0x7f79f006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.249+0000 7f79f724f700 1 -- 172.21.15.46:0/1971277722 shutdown_connections 2024-03-18T20:46:25.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.249+0000 7f79f724f700 1 -- 172.21.15.46:0/1971277722 wait complete. 2024-03-18T20:46:25.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.249+0000 7f79f724f700 1 Processor -- start 2024-03-18T20:46:25.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f724f700 1 -- start start 2024-03-18T20:46:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f724f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f0109750 0x7f79f0115c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f724f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f010a070 0x7f79f01161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f724f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f0112d00 0x7f79f01166e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f724f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f01add90 con 0x7f79f0112d00 2024-03-18T20:46:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f724f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f01adf00 con 0x7f79f010a070 2024-03-18T20:46:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f724f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f01ae0c0 con 0x7f79f0109750 2024-03-18T20:46:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f57ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f0112d00 0x7f79f01166e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f4feb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f0109750 0x7f79f0115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f57ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f0112d00 0x7f79f01166e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43298/0 (socket says 172.21.15.46:43298) 2024-03-18T20:46:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f57ec700 1 -- 172.21.15.46:0/3723885268 learned_addr learned my addr 172.21.15.46:0/3723885268 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79effff700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f010a070 0x7f79f01161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f57ec700 1 -- 172.21.15.46:0/3723885268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f0109750 msgr2=0x7f79f0115c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f57ec700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f0109750 0x7f79f0115c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f57ec700 1 -- 172.21.15.46:0/3723885268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f010a070 msgr2=0x7f79f01161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f57ec700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f010a070 0x7f79f01161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.250+0000 7f79f57ec700 1 -- 172.21.15.46:0/3723885268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc009520 con 0x7f79f0112d00 2024-03-18T20:46:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.251+0000 7f79f57ec700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f0112d00 0x7f79f01166e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f79e80094f0 tx=0x7f79e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.251+0000 7f79edffb700 1 -- 172.21.15.46:0/3723885268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8004db0 con 0x7f79f0112d00 2024-03-18T20:46:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.251+0000 7f79edffb700 1 -- 172.21.15.46:0/3723885268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79e800f040 con 0x7f79f0112d00 2024-03-18T20:46:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.251+0000 7f79edffb700 1 -- 172.21.15.46:0/3723885268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8005980 con 0x7f79f0112d00 2024-03-18T20:46:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.251+0000 7f79f724f700 1 -- 172.21.15.46:0/3723885268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e40095f0 con 0x7f79f0112d00 2024-03-18T20:46:25.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.251+0000 7f79f724f700 1 -- 172.21.15.46:0/3723885268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f01ae6d0 con 0x7f79f0112d00 2024-03-18T20:46:25.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.253+0000 7f79edffb700 1 -- 172.21.15.46:0/3723885268 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f79e8005020 con 0x7f79f0112d00 2024-03-18T20:46:25.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.254+0000 7f79edffb700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79e00a1290 0x7f79e00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.254+0000 7f79f4feb700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79e00a1290 0x7f79e00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.254+0000 7f79edffb700 1 -- 172.21.15.46:0/3723885268 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f79e80cc670 con 0x7f79f0112d00 2024-03-18T20:46:25.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.255+0000 7f79f4feb700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79e00a1290 0x7f79e00a3750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009df0 tx=0x7f79dc009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.255+0000 7f7a033e4b80 1 Processor -- start 2024-03-18T20:46:25.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.255+0000 7f7a033e4b80 1 -- start start 2024-03-18T20:46:25.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.255+0000 7f7a033e4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c862ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.255+0000 7f7a033e4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c8637ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.255+0000 7f7a033e4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c863a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.255+0000 7f7a033e4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c8586990 con 0x55a4c8638220 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.255+0000 7f7a033e4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c8584350 con 0x55a4c862ca60 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.255+0000 7f7a033e4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c857fc40 con 0x55a4c862d380 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79f4feb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c862ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79f4feb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c862ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52486/0 (socket says 172.21.15.46:52486) 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79f4feb700 1 -- 172.21.15.46:0/61296586 learned_addr learned my addr 172.21.15.46:0/61296586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79f57ec700 1 --2- 172.21.15.46:0/61296586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c863a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79effff700 1 --2- 172.21.15.46:0/61296586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c8637ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79f4feb700 1 -- 172.21.15.46:0/61296586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 msgr2=0x55a4c8637ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79f4feb700 1 --2- 172.21.15.46:0/61296586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c8637ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79f4feb700 1 -- 172.21.15.46:0/61296586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 msgr2=0x55a4c863a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79f4feb700 1 --2- 172.21.15.46:0/61296586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c863a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79f4feb700 1 -- 172.21.15.46:0/61296586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4c84013b0 con 0x55a4c862ca60 2024-03-18T20:46:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79effff700 1 --2- 172.21.15.46:0/61296586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c8637ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79f4feb700 1 --2- 172.21.15.46:0/61296586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c862ce40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0094f0 tx=0x7f79dc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=665978a85e9c89ca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79daffd700 1 -- 172.21.15.46:0/61296586 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc026630 con 0x55a4c862ca60 2024-03-18T20:46:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79daffd700 1 -- 172.21.15.46:0/61296586 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79dc021070 con 0x55a4c862ca60 2024-03-18T20:46:25.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.256+0000 7f79daffd700 1 -- 172.21.15.46:0/61296586 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc01d650 con 0x55a4c862ca60 2024-03-18T20:46:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.257+0000 7f7a033e4b80 1 -- 172.21.15.46:0/61296586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 msgr2=0x55a4c862ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.257+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/61296586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c862ce40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0094f0 tx=0x7f79dc014000 comp rx=0 tx=0).stop 2024-03-18T20:46:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.257+0000 7f7a033e4b80 1 -- 172.21.15.46:0/61296586 shutdown_connections 2024-03-18T20:46:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.257+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/61296586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c863a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.257+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/61296586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c862ce40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.257+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/61296586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c8637ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.257+0000 7f7a033e4b80 1 -- 172.21.15.46:0/61296586 >> 172.21.15.46:0/61296586 conn(0x55a4c85a3220 msgr2=0x55a4c8629020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.257+0000 7f7a033e4b80 1 -- 172.21.15.46:0/61296586 shutdown_connections 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.257+0000 7f7a033e4b80 1 -- 172.21.15.46:0/61296586 wait complete. 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.258+0000 7f7a033e4b80 1 -- start start 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.258+0000 7f7a033e4b80 1 Processor -- start 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.258+0000 7f7a033e4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c8637bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.258+0000 7f7a033e4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c8634ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.258+0000 7f7a033e4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c8635560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.258+0000 7f7a033e4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c851bd80 con 0x55a4c8638220 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.258+0000 7f7a033e4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c851ba20 con 0x55a4c862ca60 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.258+0000 7f7a033e4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c84036a0 con 0x55a4c862d380 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.258+0000 7f7a033e4b80 10 client.0 _ll_register_callbacks cb 0x55a4c83873a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.258+0000 7f79f4feb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c8637bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.258+0000 7f79f57ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c8635560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79f4feb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c8637bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52492/0 (socket says 172.21.15.46:52492) 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79f4feb700 1 -- 172.21.15.46:0/2368191111 learned_addr learned my addr 172.21.15.46:0/2368191111 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79effff700 1 --2- 172.21.15.46:0/2368191111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c8634ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79f4feb700 1 -- 172.21.15.46:0/2368191111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 msgr2=0x55a4c8634ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79f4feb700 1 --2- 172.21.15.46:0/2368191111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c8634ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79f4feb700 1 -- 172.21.15.46:0/2368191111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 msgr2=0x55a4c8635560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79f4feb700 1 --2- 172.21.15.46:0/2368191111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c8635560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79f4feb700 1 -- 172.21.15.46:0/2368191111 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc017070 con 0x55a4c862ca60 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79f4feb700 1 --2- 172.21.15.46:0/2368191111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c8637bf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f79dc01ce00 tx=0x7f79dc01cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af0378c7ddd20286 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79cffff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79cffff700 1 -- 172.21.15.46:0/2368191111 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc01c620 con 0x55a4c862ca60 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79cffff700 1 -- 172.21.15.46:0/2368191111 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f79dc01c780 con 0x55a4c862ca60 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f7a033e4b80 1 -- 172.21.15.46:0/2368191111 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4c8620ad0 con 0x55a4c862ca60 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f79cffff700 1 -- 172.21.15.46:0/2368191111 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79dc01c940 con 0x55a4c862ca60 2024-03-18T20:46:25.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f7a033e4b80 10 client.28781 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.259+0000 7f7a033e4b80 1 -- 172.21.15.46:0/2368191111 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55a4c83adb60 con 0x55a4c862ca60 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.260+0000 7f79cffff700 1 -- 172.21.15.46:0/2368191111 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79dc03b020 con 0x55a4c862ca60 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.260+0000 7f79cffff700 1 -- 172.21.15.46:0/2368191111 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f79dc003b30 con 0x55a4c862ca60 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.260+0000 7f7a033e4b80 10 client.28781 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.260+0000 7f7a033e4b80 10 client.28781 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.260+0000 7f7a033e4b80 20 client.28781 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.260+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/2368191111 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a4c8647cc0 0x55a4c864a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.260+0000 7f7a033e4b80 4 client.28781 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.260+0000 7f7a033e4b80 1 -- 172.21.15.46:0/2368191111 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4c8578700 con 0x55a4c8647cc0 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.260+0000 7f79effff700 1 --2- 172.21.15.46:0/2368191111 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a4c8647cc0 0x55a4c864a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f79cffff700 10 client.28781 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f79effff700 1 --2- 172.21.15.46:0/2368191111 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a4c8647cc0 0x55a4c864a0e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc52b726c3a5bb3b server_cookie=ace4ac9b7e7035fc in_seq=0 out_seq=0 2024-03-18T20:46:25.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f79cffff700 1 -- 172.21.15.46:0/2368191111 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a4c8578700 con 0x55a4c8647cc0 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f79cffff700 10 client.28781 handle_command_reply: tid=0 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f7a033e4b80 1 client.28781 shutdown 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f7a033e4b80 1 -- 172.21.15.46:0/2368191111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 msgr2=0x55a4c8637bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/2368191111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c8637bf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f79dc01ce00 tx=0x7f79dc01cf70 comp rx=0 tx=0).stop 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f7a033e4b80 1 -- 172.21.15.46:0/2368191111 shutdown_connections 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/2368191111 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a4c8647cc0 0x55a4c864a0e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/2368191111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c8635560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/2368191111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c8637bf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/2368191111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c8634ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f79cffff700 0 client.28781 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.261+0000 7f7a033e4b80 1 -- 172.21.15.46:0/2368191111 >> 172.21.15.46:0/2368191111 conn(0x55a4c85a3220 msgr2=0x55a4c8628c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.262+0000 7f7a033e4b80 1 -- 172.21.15.46:0/2368191111 shutdown_connections 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.262+0000 7f7a033e4b80 1 -- 172.21.15.46:0/2368191111 wait complete. 2024-03-18T20:46:25.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.262+0000 7f7a033e4b80 20 client.28781 trim_cache size 0 max 16384 2024-03-18T20:46:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.276+0000 7f7a033e4b80 1 Processor -- start 2024-03-18T20:46:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.276+0000 7f7a033e4b80 1 -- start start 2024-03-18T20:46:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.276+0000 7f7a033e4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862ca60 0x55a4c863ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.276+0000 7f7a033e4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862d380 0x55a4c8628b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.276+0000 7f7a033e4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c8629090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.276+0000 7f7a033e4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c8620610 con 0x55a4c8638220 2024-03-18T20:46:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.276+0000 7f7a033e4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c8622f80 con 0x55a4c862d380 2024-03-18T20:46:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.276+0000 7f7a033e4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c863a970 con 0x55a4c862ca60 2024-03-18T20:46:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79f57ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c8629090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79effff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862d380 0x55a4c8628b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79effff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862d380 0x55a4c8628b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52504/0 (socket says 172.21.15.46:52504) 2024-03-18T20:46:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79f57ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c8629090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43324/0 (socket says 172.21.15.46:43324) 2024-03-18T20:46:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79effff700 1 -- 172.21.15.46:0/337866701 learned_addr learned my addr 172.21.15.46:0/337866701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79f4feb700 1 --2- 172.21.15.46:0/337866701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862ca60 0x55a4c863ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79effff700 1 -- 172.21.15.46:0/337866701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862ca60 msgr2=0x55a4c863ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79effff700 1 --2- 172.21.15.46:0/337866701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862ca60 0x55a4c863ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79effff700 1 -- 172.21.15.46:0/337866701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 msgr2=0x55a4c8629090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79effff700 1 --2- 172.21.15.46:0/337866701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c8629090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79effff700 1 -- 172.21.15.46:0/337866701 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4c8545ea0 con 0x55a4c862d380 2024-03-18T20:46:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79f4feb700 1 --2- 172.21.15.46:0/337866701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862ca60 0x55a4c863ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79f57ec700 1 --2- 172.21.15.46:0/337866701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c8629090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79effff700 1 --2- 172.21.15.46:0/337866701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862d380 0x55a4c8628b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f79e40098e0 tx=0x7f79e4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0e59af6a0aeb7fd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79db7fe700 1 -- 172.21.15.46:0/337866701 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e401d040 con 0x55a4c862d380 2024-03-18T20:46:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79db7fe700 1 -- 172.21.15.46:0/337866701 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79e401e040 con 0x55a4c862d380 2024-03-18T20:46:25.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.277+0000 7f79db7fe700 1 -- 172.21.15.46:0/337866701 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400ba40 con 0x55a4c862d380 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.278+0000 7f7a033e4b80 1 -- 172.21.15.46:0/337866701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862d380 msgr2=0x55a4c8628b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.278+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/337866701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862d380 0x55a4c8628b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f79e40098e0 tx=0x7f79e4005fd0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.278+0000 7f7a033e4b80 1 -- 172.21.15.46:0/337866701 shutdown_connections 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.278+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/337866701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c8629090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.278+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/337866701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862ca60 0x55a4c863ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.278+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/337866701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862d380 0x55a4c8628b50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.278+0000 7f7a033e4b80 1 -- 172.21.15.46:0/337866701 >> 172.21.15.46:0/337866701 conn(0x55a4c85a3220 msgr2=0x55a4c8630220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.278+0000 7f7a033e4b80 1 -- 172.21.15.46:0/337866701 shutdown_connections 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.278+0000 7f7a033e4b80 1 -- 172.21.15.46:0/337866701 wait complete. 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.279+0000 7f7a033e4b80 1 -- start start 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.279+0000 7f7a033e4b80 1 Processor -- start 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.279+0000 7f7a033e4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c863ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.279+0000 7f7a033e4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c863d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.279+0000 7f7a033e4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c863d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.279+0000 7f7a033e4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c863dce0 con 0x55a4c8638220 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.279+0000 7f7a033e4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c863de50 con 0x55a4c862ca60 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.279+0000 7f7a033e4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c863dfc0 con 0x55a4c862d380 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.279+0000 7f7a033e4b80 10 client.0 _ll_register_callbacks cb 0x7f79e8009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79f4feb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c863ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79f4feb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c863ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52518/0 (socket says 172.21.15.46:52518) 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79f4feb700 1 -- 172.21.15.46:0/432206838 learned_addr learned my addr 172.21.15.46:0/432206838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79effff700 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c863d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79f57ec700 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c863d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79f4feb700 1 -- 172.21.15.46:0/432206838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 msgr2=0x55a4c863d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79f4feb700 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c863d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79f4feb700 1 -- 172.21.15.46:0/432206838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 msgr2=0x55a4c863d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79f4feb700 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c863d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79f4feb700 1 -- 172.21.15.46:0/432206838 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4c83adb60 con 0x55a4c862ca60 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79f4feb700 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c863ec50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f79dc014040 tx=0x7f79dc01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599ce7c131f464f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79cffff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79cffff700 1 -- 172.21.15.46:0/432206838 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc00ee70 con 0x55a4c862ca60 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79cffff700 1 -- 172.21.15.46:0/432206838 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f79dc03c080 con 0x55a4c862ca60 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f7a033e4b80 1 -- 172.21.15.46:0/432206838 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79dc021070 con 0x55a4c862ca60 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f79cffff700 1 -- 172.21.15.46:0/432206838 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79dc00ec00 con 0x55a4c862ca60 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f7a033e4b80 10 client.28787 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.280+0000 7f7a033e4b80 1 -- 172.21.15.46:0/432206838 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f79f01ae6d0 con 0x55a4c862ca60 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.281+0000 7f79cffff700 1 -- 172.21.15.46:0/432206838 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79dc021070 con 0x55a4c862ca60 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.281+0000 7f79cffff700 1 -- 172.21.15.46:0/432206838 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f79dc003680 con 0x55a4c862ca60 2024-03-18T20:46:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.281+0000 7f7a033e4b80 10 client.28787 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.281+0000 7f7a033e4b80 10 client.28787 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.281+0000 7f7a033e4b80 20 client.28787 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.281+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a4c869ccc0 0x55a4c869f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.281+0000 7f7a033e4b80 4 client.28787 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.281+0000 7f7a033e4b80 1 -- 172.21.15.46:0/432206838 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a4c83cecf0 con 0x55a4c869ccc0 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.281+0000 7f79effff700 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a4c869ccc0 0x55a4c869f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f79effff700 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a4c869ccc0 0x55a4c869f0e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4537f3bda3e7bae0 server_cookie=ea79970bff74901e in_seq=0 out_seq=0 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f79cffff700 10 client.28787 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f79cffff700 1 -- 172.21.15.46:0/432206838 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a4c83cecf0 con 0x55a4c869ccc0 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f79cffff700 10 client.28787 handle_command_reply: tid=0 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f7a033e4b80 1 client.28787 shutdown 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f7a033e4b80 1 -- 172.21.15.46:0/432206838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 msgr2=0x55a4c863ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c863ec50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f79dc014040 tx=0x7f79dc01c8b0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f7a033e4b80 1 -- 172.21.15.46:0/432206838 shutdown_connections 2024-03-18T20:46:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a4c869ccc0 0x55a4c869f0e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a4c8638220 0x55a4c863d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f79cffff700 0 client.28787 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:46:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a4c862ca60 0x55a4c863ec50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f7a033e4b80 1 --2- 172.21.15.46:0/432206838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a4c862d380 0x55a4c863d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f7a033e4b80 1 -- 172.21.15.46:0/432206838 >> 172.21.15.46:0/432206838 conn(0x55a4c85a3220 msgr2=0x55a4c8633710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f7a033e4b80 1 -- 172.21.15.46:0/432206838 shutdown_connections 2024-03-18T20:46:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f7a033e4b80 1 -- 172.21.15.46:0/432206838 wait complete. 2024-03-18T20:46:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.282+0000 7f7a033e4b80 20 client.28787 trim_cache size 0 max 16384 2024-03-18T20:46:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:46:25.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.283+0000 7f79f724f700 1 -- 172.21.15.46:0/3723885268 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79e00a1290 msgr2=0x7f79e00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.283+0000 7f79f724f700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79e00a1290 0x7f79e00a3750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009df0 tx=0x7f79dc009260 comp rx=0 tx=0).stop 2024-03-18T20:46:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.283+0000 7f79f724f700 1 -- 172.21.15.46:0/3723885268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f0112d00 msgr2=0x7f79f01166e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.283+0000 7f79f724f700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f0112d00 0x7f79f01166e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f79e80094f0 tx=0x7f79e800d3b0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.283+0000 7f79f724f700 1 -- 172.21.15.46:0/3723885268 shutdown_connections 2024-03-18T20:46:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.283+0000 7f79f724f700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f79e00a1290 0x7f79e00a3750 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.283+0000 7f79f724f700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f0112d00 0x7f79f01166e0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.283+0000 7f79f724f700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f0109750 0x7f79f0115c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.283+0000 7f79f724f700 1 --2- 172.21.15.46:0/3723885268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f010a070 0x7f79f01161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.283+0000 7f79f724f700 1 -- 172.21.15.46:0/3723885268 >> 172.21.15.46:0/3723885268 conn(0x7f79f006bdf0 msgr2=0x7f79f0071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.283+0000 7f79f724f700 1 -- 172.21.15.46:0/3723885268 shutdown_connections 2024-03-18T20:46:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.283+0000 7f79f724f700 1 -- 172.21.15.46:0/3723885268 wait complete. 2024-03-18T20:46:25.300 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:46:25.300 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:46:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.447+0000 7ffbed3ac700 1 -- 172.21.15.46:0/254376024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe81043b0 msgr2=0x7ffbe8104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.447+0000 7ffbed3ac700 1 --2- 172.21.15.46:0/254376024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe81043b0 0x7ffbe8104790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009990 tx=0x7ffbd8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.447+0000 7ffbed3ac700 1 -- 172.21.15.46:0/254376024 shutdown_connections 2024-03-18T20:46:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.447+0000 7ffbed3ac700 1 --2- 172.21.15.46:0/254376024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe8100030 0x7ffbe810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.447+0000 7ffbed3ac700 1 --2- 172.21.15.46:0/254376024 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe81043b0 0x7ffbe8104790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.447+0000 7ffbed3ac700 1 --2- 172.21.15.46:0/254376024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe80ff670 0x7ffbe80ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.447+0000 7ffbed3ac700 1 -- 172.21.15.46:0/254376024 >> 172.21.15.46:0/254376024 conn(0x7ffbe8074f00 msgr2=0x7ffbe8075310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.447+0000 7ffbed3ac700 1 -- 172.21.15.46:0/254376024 shutdown_connections 2024-03-18T20:46:25.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.447+0000 7ffbed3ac700 1 -- 172.21.15.46:0/254376024 wait complete. 2024-03-18T20:46:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.447+0000 7ffbed3ac700 1 Processor -- start 2024-03-18T20:46:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.448+0000 7ffbed3ac700 1 -- start start 2024-03-18T20:46:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.448+0000 7ffbed3ac700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe80ff670 0x7ffbe8197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.448+0000 7ffbed3ac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe8100030 0x7ffbe81976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.448+0000 7ffbed3ac700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe81043b0 0x7ffbe819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.448+0000 7ffbed3ac700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe819c0b0 con 0x7ffbe8100030 2024-03-18T20:46:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.448+0000 7ffbed3ac700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe819c220 con 0x7ffbe81043b0 2024-03-18T20:46:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.448+0000 7ffbed3ac700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe819c3e0 con 0x7ffbe80ff670 2024-03-18T20:46:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.448+0000 7ffbe67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe8100030 0x7ffbe81976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.448+0000 7ffbe67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe8100030 0x7ffbe81976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43346/0 (socket says 172.21.15.46:43346) 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.448+0000 7ffbe67fc700 1 -- 172.21.15.46:0/133145835 learned_addr learned my addr 172.21.15.46:0/133145835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.448+0000 7ffbe77fe700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe81043b0 0x7ffbe819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.448+0000 7ffbe6ffd700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe80ff670 0x7ffbe8197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.449+0000 7ffbe67fc700 1 -- 172.21.15.46:0/133145835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe80ff670 msgr2=0x7ffbe8197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.449+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe80ff670 0x7ffbe8197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.449+0000 7ffbe67fc700 1 -- 172.21.15.46:0/133145835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe81043b0 msgr2=0x7ffbe819ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.449+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe81043b0 0x7ffbe819ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.449+0000 7ffbe67fc700 1 -- 172.21.15.46:0/133145835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd80095f0 con 0x7ffbe8100030 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.449+0000 7ffbe77fe700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe81043b0 0x7ffbe819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.449+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe8100030 0x7ffbe81976d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00e9a0 tx=0x7ffbdc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.449+0000 7ffbcffff700 1 -- 172.21.15.46:0/133145835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc009790 con 0x7ffbe8100030 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.449+0000 7ffbcffff700 1 -- 172.21.15.46:0/133145835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbdc010430 con 0x7ffbe8100030 2024-03-18T20:46:25.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.449+0000 7ffbed3ac700 1 -- 172.21.15.46:0/133145835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe819c6c0 con 0x7ffbe8100030 2024-03-18T20:46:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.449+0000 7ffbcffff700 1 -- 172.21.15.46:0/133145835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc018680 con 0x7ffbe8100030 2024-03-18T20:46:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.449+0000 7ffbed3ac700 1 -- 172.21.15.46:0/133145835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe81011b0 con 0x7ffbe8100030 2024-03-18T20:46:25.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.452+0000 7ffbcffff700 1 -- 172.21.15.46:0/133145835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ffbdc0105a0 con 0x7ffbe8100030 2024-03-18T20:46:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.453+0000 7ffbcffff700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ffbd00a1290 0x7ffbd00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.453+0000 7ffbf9541b80 1 Processor -- start 2024-03-18T20:46:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.453+0000 7ffbe6ffd700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ffbd00a1290 0x7ffbd00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.453+0000 7ffbcffff700 1 -- 172.21.15.46:0/133145835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ffbdc014070 con 0x7ffbe8100030 2024-03-18T20:46:25.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.453+0000 7ffbf9541b80 1 -- start start 2024-03-18T20:46:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbf9541b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 0x560b37fbde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbf9541b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 0x560b37fc8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbf9541b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fc9220 0x560b37fcb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbf9541b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b37f17990 con 0x560b37fbe380 2024-03-18T20:46:25.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbf9541b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b37f15350 con 0x560b37fc9220 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbf9541b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b37f10c40 con 0x560b37fbda60 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbe6ffd700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ffbd00a1290 0x7ffbd00a3750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800b2e0 tx=0x7ffbd8005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbe67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 0x560b37fc8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbe77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fc9220 0x560b37fcb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbe6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 0x560b37fbde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbe67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 0x560b37fc8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43352/0 (socket says 172.21.15.46:43352) 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbe67fc700 1 -- 172.21.15.46:0/3282051899 learned_addr learned my addr 172.21.15.46:0/3282051899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbe67fc700 1 -- 172.21.15.46:0/3282051899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 msgr2=0x560b37fbde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/3282051899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 0x560b37fbde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbe67fc700 1 -- 172.21.15.46:0/3282051899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fc9220 msgr2=0x560b37fcb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/3282051899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fc9220 0x560b37fcb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbe67fc700 1 -- 172.21.15.46:0/3282051899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b37d923b0 con 0x560b37fbe380 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.454+0000 7ffbe6ffd700 1 --2- 172.21.15.46:0/3282051899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 0x560b37fbde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.455+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/3282051899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 0x560b37fc8ce0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0a1f00 tx=0x7ffbdc09a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45a03a411e5b5c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.455+0000 7ffbcd7fa700 1 -- 172.21.15.46:0/3282051899 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0a2430 con 0x560b37fbe380 2024-03-18T20:46:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.455+0000 7ffbcd7fa700 1 -- 172.21.15.46:0/3282051899 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbdc010880 con 0x560b37fbe380 2024-03-18T20:46:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.455+0000 7ffbcd7fa700 1 -- 172.21.15.46:0/3282051899 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc099750 con 0x560b37fbe380 2024-03-18T20:46:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.456+0000 7ffbf9541b80 1 -- 172.21.15.46:0/3282051899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 msgr2=0x560b37fc8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.456+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/3282051899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 0x560b37fc8ce0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc0a1f00 tx=0x7ffbdc09a6b0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.456+0000 7ffbf9541b80 1 -- 172.21.15.46:0/3282051899 shutdown_connections 2024-03-18T20:46:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.456+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/3282051899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fc9220 0x560b37fcb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.456+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/3282051899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 0x560b37fbde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.456+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/3282051899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 0x560b37fc8ce0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.456+0000 7ffbf9541b80 1 -- 172.21.15.46:0/3282051899 >> 172.21.15.46:0/3282051899 conn(0x560b37f34220 msgr2=0x560b37fba020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.456+0000 7ffbf9541b80 1 -- 172.21.15.46:0/3282051899 shutdown_connections 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.456+0000 7ffbf9541b80 1 -- 172.21.15.46:0/3282051899 wait complete. 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.456+0000 7ffbf9541b80 1 -- start start 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbf9541b80 1 Processor -- start 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbf9541b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbda60 0x560b37fc6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbf9541b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 0x560b37fc6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbf9541b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fc9220 0x560b37fc6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbf9541b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b37eacd80 con 0x560b37fbe380 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbf9541b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b37eaca20 con 0x560b37fbda60 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbf9541b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b37d946a0 con 0x560b37fc9220 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbf9541b80 10 client.0 _ll_register_callbacks cb 0x560b37d183a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbe67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 0x560b37fc6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbe6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbda60 0x560b37fc6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbe67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 0x560b37fc6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43360/0 (socket says 172.21.15.46:43360) 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbe6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbda60 0x560b37fc6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52566/0 (socket says 172.21.15.46:52566) 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbe67fc700 1 -- 172.21.15.46:0/677484699 learned_addr learned my addr 172.21.15.46:0/677484699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbe77fe700 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fc9220 0x560b37fc6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbe67fc700 1 -- 172.21.15.46:0/677484699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fc9220 msgr2=0x560b37fc6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fc9220 0x560b37fc6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbe67fc700 1 -- 172.21.15.46:0/677484699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbda60 msgr2=0x560b37fc6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbda60 0x560b37fc6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbe67fc700 1 -- 172.21.15.46:0/677484699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe810cb40 con 0x560b37fbe380 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.457+0000 7ffbe6ffd700 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbda60 0x560b37fc6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.458+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 0x560b37fc6560 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009230 tx=0x7ffbdc099750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6a1ebee2690216 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.458+0000 7ffbcaff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.458+0000 7ffbcaff5700 1 -- 172.21.15.46:0/677484699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0a2430 con 0x560b37fbe380 2024-03-18T20:46:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.458+0000 7ffbcaff5700 1 -- 172.21.15.46:0/677484699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ffbdc0a21b0 con 0x560b37fbe380 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.458+0000 7ffbf9541b80 1 -- 172.21.15.46:0/677484699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b37fb1ad0 con 0x560b37fbe380 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.458+0000 7ffbcaff5700 1 -- 172.21.15.46:0/677484699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbdc0b8170 con 0x560b37fbe380 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.458+0000 7ffbf9541b80 10 client.19170 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.458+0000 7ffbf9541b80 1 -- 172.21.15.46:0/677484699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7ffbe81011b0 con 0x560b37fbe380 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.459+0000 7ffbcaff5700 1 -- 172.21.15.46:0/677484699 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbdc063080 con 0x560b37fbe380 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.459+0000 7ffbcaff5700 1 -- 172.21.15.46:0/677484699 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ffbdc098f10 con 0x560b37fbe380 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.459+0000 7ffbf9541b80 10 client.19170 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.459+0000 7ffbf9541b80 10 client.19170 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.459+0000 7ffbf9541b80 20 client.19170 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.459+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x560b37fd8b50 0x560b37fdaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.459+0000 7ffbf9541b80 4 client.19170 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.459+0000 7ffbf9541b80 1 -- 172.21.15.46:0/677484699 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b37f09700 con 0x560b37fd8b50 2024-03-18T20:46:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.459+0000 7ffbe6ffd700 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x560b37fd8b50 0x560b37fdaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.459+0000 7ffbcaff5700 10 client.19170 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.459+0000 7ffbe6ffd700 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x560b37fd8b50 0x560b37fdaf40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d04b522ec0d734bc server_cookie=a63d7a9483822423 in_seq=0 out_seq=0 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbcaff5700 1 -- 172.21.15.46:0/677484699 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560b37f09700 con 0x560b37fd8b50 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbcaff5700 10 client.19170 handle_command_reply: tid=0 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbf9541b80 1 client.19170 shutdown 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbf9541b80 1 -- 172.21.15.46:0/677484699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 msgr2=0x560b37fc6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 0x560b37fc6560 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009230 tx=0x7ffbdc099750 comp rx=0 tx=0).stop 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbf9541b80 1 -- 172.21.15.46:0/677484699 shutdown_connections 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x560b37fd8b50 0x560b37fdaf40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fc9220 0x560b37fc6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbda60 0x560b37fc6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/677484699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbe380 0x560b37fc6560 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbf9541b80 1 -- 172.21.15.46:0/677484699 >> 172.21.15.46:0/677484699 conn(0x560b37f34220 msgr2=0x560b37fb9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbcaff5700 0 client.19170 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:46:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbf9541b80 1 -- 172.21.15.46:0/677484699 shutdown_connections 2024-03-18T20:46:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbf9541b80 1 -- 172.21.15.46:0/677484699 wait complete. 2024-03-18T20:46:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.460+0000 7ffbf9541b80 20 client.19170 trim_cache size 0 max 16384 2024-03-18T20:46:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.476+0000 7ffbf9541b80 1 Processor -- start 2024-03-18T20:46:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.476+0000 7ffbf9541b80 1 -- start start 2024-03-18T20:46:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.476+0000 7ffbf9541b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbda60 0x560b37fc0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.476+0000 7ffbf9541b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbe380 0x560b37fcfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.476+0000 7ffbf9541b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fc9220 0x560b37fc68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.476+0000 7ffbf9541b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b37f84970 con 0x560b37fbda60 2024-03-18T20:46:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.476+0000 7ffbf9541b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b37fbb470 con 0x560b37fbe380 2024-03-18T20:46:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.476+0000 7ffbf9541b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b37fbb230 con 0x560b37fc9220 2024-03-18T20:46:25.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.476+0000 7ffbe6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbda60 0x560b37fc0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.476+0000 7ffbe67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbe380 0x560b37fcfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbe6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbda60 0x560b37fc0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43368/0 (socket says 172.21.15.46:43368) 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbe6ffd700 1 -- 172.21.15.46:0/879846670 learned_addr learned my addr 172.21.15.46:0/879846670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbe77fe700 1 --2- 172.21.15.46:0/879846670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fc9220 0x560b37fc68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbe67fc700 1 -- 172.21.15.46:0/879846670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fc9220 msgr2=0x560b37fc68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/879846670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fc9220 0x560b37fc68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbe67fc700 1 -- 172.21.15.46:0/879846670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbda60 msgr2=0x560b37fc0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/879846670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbda60 0x560b37fc0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbe67fc700 1 -- 172.21.15.46:0/879846670 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b37fdfb70 con 0x560b37fbe380 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbe6ffd700 1 --2- 172.21.15.46:0/879846670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbda60 0x560b37fc0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbe77fe700 1 --2- 172.21.15.46:0/879846670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fc9220 0x560b37fc68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/879846670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbe380 0x560b37fcfe40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc096a20 tx=0x7ffbdc094f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a74fdfc83cc0a77 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbcdffb700 1 -- 172.21.15.46:0/879846670 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0962f0 con 0x560b37fbe380 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbcdffb700 1 -- 172.21.15.46:0/879846670 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560b37fb1ad0 con 0x560b37fbe380 2024-03-18T20:46:25.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.477+0000 7ffbcdffb700 1 -- 172.21.15.46:0/879846670 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc098f10 con 0x560b37fbe380 2024-03-18T20:46:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.478+0000 7ffbf9541b80 1 -- 172.21.15.46:0/879846670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbe380 msgr2=0x560b37fcfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.478+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/879846670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbe380 0x560b37fcfe40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc096a20 tx=0x7ffbdc094f60 comp rx=0 tx=0).stop 2024-03-18T20:46:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.478+0000 7ffbf9541b80 1 -- 172.21.15.46:0/879846670 shutdown_connections 2024-03-18T20:46:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.478+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/879846670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fbda60 0x560b37fc0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.478+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/879846670 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fc9220 0x560b37fc68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.478+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/879846670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbe380 0x560b37fcfe40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.478+0000 7ffbf9541b80 1 -- 172.21.15.46:0/879846670 >> 172.21.15.46:0/879846670 conn(0x560b37f34220 msgr2=0x560b37fc1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.478+0000 7ffbf9541b80 1 -- 172.21.15.46:0/879846670 shutdown_connections 2024-03-18T20:46:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.478+0000 7ffbf9541b80 1 -- 172.21.15.46:0/879846670 wait complete. 2024-03-18T20:46:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbf9541b80 1 -- start start 2024-03-18T20:46:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbf9541b80 1 Processor -- start 2024-03-18T20:46:25.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbf9541b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 0x560b37fcfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbf9541b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbe380 0x560b37fcd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbf9541b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fc9220 0x560b37fcd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbf9541b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b37fd0420 con 0x560b37fc9220 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbf9541b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b37fcdac0 con 0x560b37fbe380 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbf9541b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b37fcdc80 con 0x560b37fbda60 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbf9541b80 10 client.0 _ll_register_callbacks cb 0x560b37d183a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbe6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 0x560b37fcfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbe67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbe380 0x560b37fcd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbe6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 0x560b37fcfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:51374/0 (socket says 172.21.15.46:51374) 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbe6ffd700 1 -- 172.21.15.46:0/24747300 learned_addr learned my addr 172.21.15.46:0/24747300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.479+0000 7ffbe77fe700 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fc9220 0x560b37fcd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbe6ffd700 1 -- 172.21.15.46:0/24747300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbe380 msgr2=0x560b37fcd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbe6ffd700 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbe380 0x560b37fcd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbe6ffd700 1 -- 172.21.15.46:0/24747300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fc9220 msgr2=0x560b37fcd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbe6ffd700 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fc9220 0x560b37fcd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbe6ffd700 1 -- 172.21.15.46:0/24747300 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd801de90 con 0x560b37fbda60 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbe77fe700 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fc9220 0x560b37fcd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbe6ffd700 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 0x560b37fcfb10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8006010 tx=0x7ffbd8024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=518c4ad7dcdb84af server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbcaff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbcaff5700 1 -- 172.21.15.46:0/24747300 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8039340 con 0x560b37fbda60 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbcaff5700 1 -- 172.21.15.46:0/24747300 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ffbd80394a0 con 0x560b37fbda60 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbf9541b80 1 -- 172.21.15.46:0/24747300 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffbdc010880 con 0x560b37fbda60 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbcaff5700 1 -- 172.21.15.46:0/24747300 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbd800b750 con 0x560b37fbda60 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbf9541b80 10 client.28825 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbf9541b80 1 -- 172.21.15.46:0/24747300 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7ffbe81011b0 con 0x560b37fbda60 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.480+0000 7ffbcaff5700 1 -- 172.21.15.46:0/24747300 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbd804e020 con 0x560b37fbda60 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.481+0000 7ffbcaff5700 1 -- 172.21.15.46:0/24747300 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ffbd8004910 con 0x560b37fbda60 2024-03-18T20:46:25.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.481+0000 7ffbf9541b80 10 client.28825 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.481+0000 7ffbf9541b80 10 client.28825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.481+0000 7ffbf9541b80 20 client.28825 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.481+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x560b3802cd20 0x560b3802f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.481+0000 7ffbf9541b80 4 client.28825 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.481+0000 7ffbf9541b80 1 -- 172.21.15.46:0/24747300 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560b37d5fcf0 con 0x560b3802cd20 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.481+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x560b3802cd20 0x560b3802f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.481+0000 7ffbcaff5700 10 client.28825 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.481+0000 7ffbe67fc700 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x560b3802cd20 0x560b3802f140 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21844d0be3abcd80 server_cookie=91b8999395f5c38 in_seq=0 out_seq=0 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.482+0000 7ffbcaff5700 1 -- 172.21.15.46:0/24747300 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560b37d5fcf0 con 0x560b3802cd20 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.482+0000 7ffbcaff5700 10 client.28825 handle_command_reply: tid=0 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.482+0000 7ffbf9541b80 1 client.28825 shutdown 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.482+0000 7ffbf9541b80 1 -- 172.21.15.46:0/24747300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 msgr2=0x560b37fcfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 0x560b37fcfb10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8006010 tx=0x7ffbd8024000 comp rx=0 tx=0).stop 2024-03-18T20:46:25.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbf9541b80 1 -- 172.21.15.46:0/24747300 shutdown_connections 2024-03-18T20:46:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x560b3802cd20 0x560b3802f140 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b37fc9220 0x560b37fcd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560b37fbda60 0x560b37fcfb10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbf9541b80 1 --2- 172.21.15.46:0/24747300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b37fbe380 0x560b37fcd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbcaff5700 0 client.28825 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:46:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbf9541b80 1 -- 172.21.15.46:0/24747300 >> 172.21.15.46:0/24747300 conn(0x560b37f34220 msgr2=0x560b37fc47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbf9541b80 1 -- 172.21.15.46:0/24747300 shutdown_connections 2024-03-18T20:46:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbf9541b80 1 -- 172.21.15.46:0/24747300 wait complete. 2024-03-18T20:46:25.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbf9541b80 20 client.28825 trim_cache size 0 max 16384 2024-03-18T20:46:25.490 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:46:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbed3ac700 1 -- 172.21.15.46:0/133145835 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ffbd00a1290 msgr2=0x7ffbd00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbed3ac700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ffbd00a1290 0x7ffbd00a3750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800b2e0 tx=0x7ffbd8005960 comp rx=0 tx=0).stop 2024-03-18T20:46:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbed3ac700 1 -- 172.21.15.46:0/133145835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe8100030 msgr2=0x7ffbe81976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbed3ac700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe8100030 0x7ffbe81976d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00e9a0 tx=0x7ffbdc00ecb0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbed3ac700 1 -- 172.21.15.46:0/133145835 shutdown_connections 2024-03-18T20:46:25.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbed3ac700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ffbd00a1290 0x7ffbd00a3750 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbed3ac700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbe81043b0 0x7ffbe819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbed3ac700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbe80ff670 0x7ffbe8197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbed3ac700 1 --2- 172.21.15.46:0/133145835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbe8100030 0x7ffbe81976d0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.483+0000 7ffbed3ac700 1 -- 172.21.15.46:0/133145835 >> 172.21.15.46:0/133145835 conn(0x7ffbe8074f00 msgr2=0x7ffbe81061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.484+0000 7ffbed3ac700 1 -- 172.21.15.46:0/133145835 shutdown_connections 2024-03-18T20:46:25.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.484+0000 7ffbed3ac700 1 -- 172.21.15.46:0/133145835 wait complete. 2024-03-18T20:46:25.499 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:25 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/942966275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:46:25.501 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:46:25.501 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:46:25.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:25 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/942966275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:46:25.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.646+0000 7f150e30c700 1 -- 172.21.15.46:0/1478802240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508069540 msgr2=0x7f1508069920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.646+0000 7f150e30c700 1 --2- 172.21.15.46:0/1478802240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508069540 0x7f1508069920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009990 tx=0x7f14f8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.646+0000 7f150e30c700 1 -- 172.21.15.46:0/1478802240 shutdown_connections 2024-03-18T20:46:25.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.646+0000 7f150e30c700 1 --2- 172.21.15.46:0/1478802240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15081060a0 0x7f150810dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.646+0000 7f150e30c700 1 --2- 172.21.15.46:0/1478802240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508069540 0x7f1508069920 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.646+0000 7f150e30c700 1 --2- 172.21.15.46:0/1478802240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1508069ef0 0x7f1508105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.646+0000 7f150e30c700 1 -- 172.21.15.46:0/1478802240 >> 172.21.15.46:0/1478802240 conn(0x7f1508076d50 msgr2=0x7f1508077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.646+0000 7f150e30c700 1 -- 172.21.15.46:0/1478802240 shutdown_connections 2024-03-18T20:46:25.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.646+0000 7f150e30c700 1 -- 172.21.15.46:0/1478802240 wait complete. 2024-03-18T20:46:25.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.647+0000 7f150e30c700 1 Processor -- start 2024-03-18T20:46:25.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.647+0000 7f150e30c700 1 -- start start 2024-03-18T20:46:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f150e30c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508069540 0x7f15080ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f150e30c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15081060a0 0x7f15081001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f150e30c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15081006f0 0x7f15081a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f150e30c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15081a5920 con 0x7f15081006f0 2024-03-18T20:46:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f150e30c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15081a5a90 con 0x7f1508069540 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f150e30c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15081a5c50 con 0x7f15081060a0 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f1507fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508069540 0x7f15080ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f15077fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15081060a0 0x7f15081001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f1507fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508069540 0x7f15080ffc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52602/0 (socket says 172.21.15.46:52602) 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f1507fff700 1 -- 172.21.15.46:0/1516721126 learned_addr learned my addr 172.21.15.46:0/1516721126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f150c8a9700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15081006f0 0x7f15081a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f1507fff700 1 -- 172.21.15.46:0/1516721126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15081060a0 msgr2=0x7f15081001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f1507fff700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15081060a0 0x7f15081001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f1507fff700 1 -- 172.21.15.46:0/1516721126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15081006f0 msgr2=0x7f15081a5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f1507fff700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15081006f0 0x7f15081a5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f1507fff700 1 -- 172.21.15.46:0/1516721126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f80095f0 con 0x7f1508069540 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.648+0000 7f1507fff700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508069540 0x7f15080ffc70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f14f800b2e0 tx=0x7f14f8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.649+0000 7f15057fa700 1 -- 172.21.15.46:0/1516721126 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f801d070 con 0x7f1508069540 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.649+0000 7f15057fa700 1 -- 172.21.15.46:0/1516721126 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f8021410 con 0x7f1508069540 2024-03-18T20:46:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.649+0000 7f150e30c700 1 -- 172.21.15.46:0/1516721126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15081a5ed0 con 0x7f1508069540 2024-03-18T20:46:25.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.649+0000 7f15057fa700 1 -- 172.21.15.46:0/1516721126 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8017650 con 0x7f1508069540 2024-03-18T20:46:25.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.649+0000 7f150e30c700 1 -- 172.21.15.46:0/1516721126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15081a62f0 con 0x7f1508069540 2024-03-18T20:46:25.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.651+0000 7f15057fa700 1 -- 172.21.15.46:0/1516721126 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f14f80177b0 con 0x7f1508069540 2024-03-18T20:46:25.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.652+0000 7f15057fa700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14f00a1310 0x7f14f00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.652+0000 7f15077fe700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14f00a1310 0x7f14f00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.652+0000 7f15057fa700 1 -- 172.21.15.46:0/1516721126 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f14f80ce270 con 0x7f1508069540 2024-03-18T20:46:25.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.653+0000 7f15077fe700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14f00a1310 0x7f14f00a37d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f15081013a0 tx=0x7f14fc00b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.653+0000 7f151a4a1b80 1 Processor -- start 2024-03-18T20:46:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.653+0000 7f151a4a1b80 1 -- start start 2024-03-18T20:46:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f151a4a1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597d6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f151a4a1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 0x55c5597e1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f151a4a1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 0x55c5597e4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f151a4a1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c559730990 con 0x55c5597d6a60 2024-03-18T20:46:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f151a4a1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c55972e350 con 0x55c5597d7380 2024-03-18T20:46:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f151a4a1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c559729c40 con 0x55c5597e2220 2024-03-18T20:46:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f15077fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 0x55c5597e1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f150c8a9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 0x55c5597e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f15077fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 0x55c5597e1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52608/0 (socket says 172.21.15.46:52608) 2024-03-18T20:46:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f15077fe700 1 -- 172.21.15.46:0/1579102748 learned_addr learned my addr 172.21.15.46:0/1579102748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f15077fe700 1 -- 172.21.15.46:0/1579102748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 msgr2=0x55c5597e4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f15077fe700 1 --2- 172.21.15.46:0/1579102748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 0x55c5597e4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f15077fe700 1 -- 172.21.15.46:0/1579102748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 msgr2=0x55c5597d6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f15077fe700 1 --2- 172.21.15.46:0/1579102748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597d6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f15077fe700 1 -- 172.21.15.46:0/1579102748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5595ab3b0 con 0x55c5597d7380 2024-03-18T20:46:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f15077fe700 1 --2- 172.21.15.46:0/1579102748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 0x55c5597e1ce0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f14fc010f40 tx=0x7f14fc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af85d0aa3678105f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.654+0000 7f14ee7fc700 1 -- 172.21.15.46:0/1579102748 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc00b950 con 0x55c5597d7380 2024-03-18T20:46:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.655+0000 7f14ee7fc700 1 -- 172.21.15.46:0/1579102748 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14fc015070 con 0x55c5597d7380 2024-03-18T20:46:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.655+0000 7f14ee7fc700 1 -- 172.21.15.46:0/1579102748 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc01c5d0 con 0x55c5597d7380 2024-03-18T20:46:25.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.655+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1579102748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 msgr2=0x55c5597e1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.655+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/1579102748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 0x55c5597e1ce0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f14fc010f40 tx=0x7f14fc013000 comp rx=0 tx=0).stop 2024-03-18T20:46:25.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.655+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1579102748 shutdown_connections 2024-03-18T20:46:25.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.655+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/1579102748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597d6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.655+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/1579102748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 0x55c5597e4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.655+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/1579102748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 0x55c5597e1ce0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.655+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1579102748 >> 172.21.15.46:0/1579102748 conn(0x55c55974d220 msgr2=0x55c5597d3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.655+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1579102748 shutdown_connections 2024-03-18T20:46:25.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.655+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1579102748 wait complete. 2024-03-18T20:46:25.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.656+0000 7f151a4a1b80 1 -- start start 2024-03-18T20:46:25.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.656+0000 7f151a4a1b80 1 Processor -- start 2024-03-18T20:46:25.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.656+0000 7f151a4a1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597e1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.656+0000 7f151a4a1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597d7380 0x55c5597def60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.656+0000 7f151a4a1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597e2220 0x55c5597df6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.656+0000 7f151a4a1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5596c5d80 con 0x55c5597d6a60 2024-03-18T20:46:25.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.656+0000 7f151a4a1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5596c5a20 con 0x55c5597e2220 2024-03-18T20:46:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.656+0000 7f151a4a1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5595ad6a0 con 0x55c5597d7380 2024-03-18T20:46:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f151a4a1b80 10 client.0 _ll_register_callbacks cb 0x55c5595313a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f1507fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597e1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f1507fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597e1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43410/0 (socket says 172.21.15.46:43410) 2024-03-18T20:46:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f1507fff700 1 -- 172.21.15.46:0/674852010 learned_addr learned my addr 172.21.15.46:0/674852010 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f150c8a9700 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597e2220 0x55c5597df6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f15077fe700 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597d7380 0x55c5597def60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f1507fff700 1 -- 172.21.15.46:0/674852010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597d7380 msgr2=0x55c5597def60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f1507fff700 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597d7380 0x55c5597def60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f1507fff700 1 -- 172.21.15.46:0/674852010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597e2220 msgr2=0x55c5597df6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f1507fff700 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597e2220 0x55c5597df6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f1507fff700 1 -- 172.21.15.46:0/674852010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14fc010c20 con 0x55c5597d6a60 2024-03-18T20:46:25.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f150c8a9700 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597e2220 0x55c5597df6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f1507fff700 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597e1c70 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f14f80986e0 tx=0x7f14f809bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7352df063e1689eb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f14e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:25.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f14e77fe700 1 -- 172.21.15.46:0/674852010 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f809bb40 con 0x55c5597d6a60 2024-03-18T20:46:25.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f14e77fe700 1 -- 172.21.15.46:0/674852010 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f14f80219d0 con 0x55c5597d6a60 2024-03-18T20:46:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.657+0000 7f151a4a1b80 1 -- 172.21.15.46:0/674852010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5597caad0 con 0x55c5597d6a60 2024-03-18T20:46:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.658+0000 7f14e77fe700 1 -- 172.21.15.46:0/674852010 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14f809c8e0 con 0x55c5597d6a60 2024-03-18T20:46:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.658+0000 7f151a4a1b80 10 client.19185 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.658+0000 7f151a4a1b80 1 -- 172.21.15.46:0/674852010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c559557b60 con 0x55c5597d6a60 2024-03-18T20:46:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.658+0000 7f14e77fe700 1 -- 172.21.15.46:0/674852010 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5597caad0 con 0x55c5597d6a60 2024-03-18T20:46:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.658+0000 7f14e77fe700 1 -- 172.21.15.46:0/674852010 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f14f809b140 con 0x55c5597d6a60 2024-03-18T20:46:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.658+0000 7f151a4a1b80 10 client.19185 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.658+0000 7f151a4a1b80 10 client.19185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.658+0000 7f151a4a1b80 20 client.19185 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.658+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c5597f1b80 0x55c5597f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.658+0000 7f151a4a1b80 4 client.19185 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:25.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.658+0000 7f151a4a1b80 1 -- 172.21.15.46:0/674852010 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c559722700 con 0x55c5597f1b80 2024-03-18T20:46:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.658+0000 7f15077fe700 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c5597f1b80 0x55c5597f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.659+0000 7f15077fe700 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c5597f1b80 0x55c5597f3fa0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e925f33bd14962 server_cookie=cd5488ee9c85038c in_seq=0 out_seq=0 2024-03-18T20:46:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.659+0000 7f14e77fe700 10 client.19185 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.659+0000 7f14e77fe700 1 -- 172.21.15.46:0/674852010 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c559722700 con 0x55c5597f1b80 2024-03-18T20:46:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f14e77fe700 10 client.19185 handle_command_reply: tid=0 2024-03-18T20:46:25.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f151a4a1b80 1 client.19185 shutdown 2024-03-18T20:46:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f151a4a1b80 1 -- 172.21.15.46:0/674852010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 msgr2=0x55c5597e1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597e1c70 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f14f80986e0 tx=0x7f14f809bfb0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f151a4a1b80 1 -- 172.21.15.46:0/674852010 shutdown_connections 2024-03-18T20:46:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c5597f1b80 0x55c5597f3fa0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597e1c70 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f14e77fe700 0 client.19185 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:25.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597e2220 0x55c5597df6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/674852010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597d7380 0x55c5597def60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f151a4a1b80 1 -- 172.21.15.46:0/674852010 >> 172.21.15.46:0/674852010 conn(0x55c55974d220 msgr2=0x55c5597d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f151a4a1b80 1 -- 172.21.15.46:0/674852010 shutdown_connections 2024-03-18T20:46:25.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f151a4a1b80 1 -- 172.21.15.46:0/674852010 wait complete. 2024-03-18T20:46:25.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.660+0000 7f151a4a1b80 20 client.19185 trim_cache size 0 max 16384 2024-03-18T20:46:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.677+0000 7f151a4a1b80 1 Processor -- start 2024-03-18T20:46:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.677+0000 7f151a4a1b80 1 -- start start 2024-03-18T20:46:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.677+0000 7f151a4a1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d6a60 0x55c5597e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.677+0000 7f151a4a1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d7380 0x55c5597ed5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.677+0000 7f151a4a1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 0x55c5597da280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.677+0000 7f151a4a1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5597ccf80 con 0x55c5597d7380 2024-03-18T20:46:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.677+0000 7f151a4a1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5597ca610 con 0x55c5597d6a60 2024-03-18T20:46:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.677+0000 7f151a4a1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5597e4970 con 0x55c5597e2220 2024-03-18T20:46:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.677+0000 7f1507fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d6a60 0x55c5597e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.677+0000 7f15077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d7380 0x55c5597ed5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.677+0000 7f1507fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d6a60 0x55c5597e8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52624/0 (socket says 172.21.15.46:52624) 2024-03-18T20:46:25.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.677+0000 7f1507fff700 1 -- 172.21.15.46:0/297119915 learned_addr learned my addr 172.21.15.46:0/297119915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.678+0000 7f150c8a9700 1 --2- 172.21.15.46:0/297119915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 0x55c5597da280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.678+0000 7f1507fff700 1 -- 172.21.15.46:0/297119915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 msgr2=0x55c5597da280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.678+0000 7f1507fff700 1 --2- 172.21.15.46:0/297119915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 0x55c5597da280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.678+0000 7f1507fff700 1 -- 172.21.15.46:0/297119915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d7380 msgr2=0x55c5597ed5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.678+0000 7f1507fff700 1 --2- 172.21.15.46:0/297119915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d7380 0x55c5597ed5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.678+0000 7f1507fff700 1 -- 172.21.15.46:0/297119915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5596efea0 con 0x55c5597d6a60 2024-03-18T20:46:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.678+0000 7f15077fe700 1 --2- 172.21.15.46:0/297119915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d7380 0x55c5597ed5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.678+0000 7f1507fff700 1 --2- 172.21.15.46:0/297119915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d6a60 0x55c5597e8e80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f14f809c170 tx=0x7f14f8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba2236cc4ff148bb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.678+0000 7f14eeffd700 1 -- 172.21.15.46:0/297119915 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8017a60 con 0x55c5597d6a60 2024-03-18T20:46:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.678+0000 7f14eeffd700 1 -- 172.21.15.46:0/297119915 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f80219d0 con 0x55c5597d6a60 2024-03-18T20:46:25.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.678+0000 7f14eeffd700 1 -- 172.21.15.46:0/297119915 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8097fd0 con 0x55c5597d6a60 2024-03-18T20:46:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.679+0000 7f151a4a1b80 1 -- 172.21.15.46:0/297119915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d6a60 msgr2=0x55c5597e8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.679+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/297119915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d6a60 0x55c5597e8e80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f14f809c170 tx=0x7f14f8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.679+0000 7f151a4a1b80 1 -- 172.21.15.46:0/297119915 shutdown_connections 2024-03-18T20:46:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.679+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/297119915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 0x55c5597da280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.679+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/297119915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d6a60 0x55c5597e8e80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.679+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/297119915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d7380 0x55c5597ed5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.679+0000 7f151a4a1b80 1 -- 172.21.15.46:0/297119915 >> 172.21.15.46:0/297119915 conn(0x55c55974d220 msgr2=0x55c5597d1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.679+0000 7f151a4a1b80 1 -- 172.21.15.46:0/297119915 shutdown_connections 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.679+0000 7f151a4a1b80 1 -- 172.21.15.46:0/297119915 wait complete. 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.679+0000 7f151a4a1b80 1 -- start start 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.680+0000 7f151a4a1b80 1 Processor -- start 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.680+0000 7f151a4a1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.680+0000 7f151a4a1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 0x55c5597e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.680+0000 7f151a4a1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 0x55c5597e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.680+0000 7f151a4a1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c559844970 con 0x55c5597d6a60 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.680+0000 7f151a4a1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c559844ae0 con 0x55c5597d7380 2024-03-18T20:46:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.680+0000 7f151a4a1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c559844ca0 con 0x55c5597e2220 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.680+0000 7f151a4a1b80 10 client.0 _ll_register_callbacks cb 0x55c5595313a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.680+0000 7f1507fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597e6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.680+0000 7f1507fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597e6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43426/0 (socket says 172.21.15.46:43426) 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f1507fff700 1 -- 172.21.15.46:0/1273297274 learned_addr learned my addr 172.21.15.46:0/1273297274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f15077fe700 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 0x55c5597e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f150c8a9700 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 0x55c5597e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f15077fe700 1 -- 172.21.15.46:0/1273297274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 msgr2=0x55c5597e78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f15077fe700 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 0x55c5597e78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f15077fe700 1 -- 172.21.15.46:0/1273297274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 msgr2=0x55c5597e6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f15077fe700 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597e6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f15077fe700 1 -- 172.21.15.46:0/1273297274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5595ab3b0 con 0x55c5597d7380 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f1507fff700 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597e6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f15077fe700 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 0x55c5597e73a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f14fc011a00 tx=0x7f14fc00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f9cb02751ab85c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f14e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:46:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f14e77fe700 1 -- 172.21.15.46:0/1273297274 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc02b070 con 0x55c5597d7380 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1273297274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14fc015070 con 0x55c5597d7380 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f14e77fe700 1 -- 172.21.15.46:0/1273297274 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f14fc017070 con 0x55c5597d7380 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.681+0000 7f14e77fe700 1 -- 172.21.15.46:0/1273297274 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14fc0325a0 con 0x55c5597d7380 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.682+0000 7f151a4a1b80 10 client.28823 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.682+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1273297274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c559557b60 con 0x55c5597d7380 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.682+0000 7f14e77fe700 1 -- 172.21.15.46:0/1273297274 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14fc015070 con 0x55c5597d7380 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.682+0000 7f14e77fe700 1 -- 172.21.15.46:0/1273297274 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f14fc038020 con 0x55c5597d7380 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.682+0000 7f151a4a1b80 10 client.28823 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.682+0000 7f151a4a1b80 10 client.28823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.682+0000 7f151a4a1b80 20 client.28823 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.682+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c5598463f0 0x55c559848810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.682+0000 7f151a4a1b80 4 client.28823 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.682+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1273297274 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c559578cf0 con 0x55c5598463f0 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.682+0000 7f1507fff700 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c5598463f0 0x55c559848810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f1507fff700 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c5598463f0 0x55c559848810 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33be43f051c46d65 server_cookie=29e23b3f5fb74d1e in_seq=0 out_seq=0 2024-03-18T20:46:25.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f14e77fe700 10 client.28823 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f14e77fe700 1 -- 172.21.15.46:0/1273297274 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c559578cf0 con 0x55c5598463f0 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f14e77fe700 10 client.28823 handle_command_reply: tid=0 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f151a4a1b80 1 client.28823 shutdown 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1273297274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 msgr2=0x55c5597e73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 0x55c5597e73a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f14fc011a00 tx=0x7f14fc00bfd0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1273297274 shutdown_connections 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c5598463f0 0x55c559848810 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5597d6a60 0x55c5597e6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5597e2220 0x55c5597e78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f14e77fe700 0 client.28823 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f151a4a1b80 1 --2- 172.21.15.46:0/1273297274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c5597d7380 0x55c5597e73a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1273297274 >> 172.21.15.46:0/1273297274 conn(0x55c55974d220 msgr2=0x55c5597da280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1273297274 shutdown_connections 2024-03-18T20:46:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f151a4a1b80 1 -- 172.21.15.46:0/1273297274 wait complete. 2024-03-18T20:46:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.683+0000 7f151a4a1b80 20 client.28823 trim_cache size 0 max 16384 2024-03-18T20:46:25.691 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:46:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.684+0000 7f150e30c700 1 -- 172.21.15.46:0/1516721126 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14f00a1310 msgr2=0x7f14f00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.684+0000 7f150e30c700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14f00a1310 0x7f14f00a37d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f15081013a0 tx=0x7f14fc00b410 comp rx=0 tx=0).stop 2024-03-18T20:46:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.684+0000 7f150e30c700 1 -- 172.21.15.46:0/1516721126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508069540 msgr2=0x7f15080ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.684+0000 7f150e30c700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508069540 0x7f15080ffc70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f14f800b2e0 tx=0x7f14f8004730 comp rx=0 tx=0).stop 2024-03-18T20:46:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.684+0000 7f150e30c700 1 -- 172.21.15.46:0/1516721126 shutdown_connections 2024-03-18T20:46:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.684+0000 7f150e30c700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f14f00a1310 0x7f14f00a37d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.684+0000 7f150e30c700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15081006f0 0x7f15081a5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.684+0000 7f150e30c700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1508069540 0x7f15080ffc70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.684+0000 7f150e30c700 1 --2- 172.21.15.46:0/1516721126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15081060a0 0x7f15081001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.684+0000 7f150e30c700 1 -- 172.21.15.46:0/1516721126 >> 172.21.15.46:0/1516721126 conn(0x7f1508076d50 msgr2=0x7f15080fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.684+0000 7f150e30c700 1 -- 172.21.15.46:0/1516721126 shutdown_connections 2024-03-18T20:46:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:25.684+0000 7f150e30c700 1 -- 172.21.15.46:0/1516721126 wait complete. 2024-03-18T20:46:25.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 382931 3.68 MB/sec execute 440 sec latency 3457.034 ms 2024-03-18T20:46:25.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:25 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/942966275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:46:26.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:26 smithi155 ceph-mon[48344]: pgmap v321: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:46:26.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:26 smithi046 ceph-mon[43495]: pgmap v321: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:46:26.702 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:46:26.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:46:26.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:46:26.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 384173 3.69 MB/sec execute 441 sec latency 19.914 ms 2024-03-18T20:46:26.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:26 smithi073 ceph-mon[47384]: pgmap v321: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:46:26.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.855+0000 7f487fbbb700 1 -- 172.21.15.46:0/912160266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4878103170 msgr2=0x7f4878103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:26.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.855+0000 7f487fbbb700 1 --2- 172.21.15.46:0/912160266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4878103170 0x7f4878103550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f487400b0a0 tx=0x7f487400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.855+0000 7f487fbbb700 1 -- 172.21.15.46:0/912160266 shutdown_connections 2024-03-18T20:46:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.855+0000 7f487fbbb700 1 --2- 172.21.15.46:0/912160266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4878108290 0x7f487810fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.855+0000 7f487fbbb700 1 --2- 172.21.15.46:0/912160266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4878103170 0x7f4878103550 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.855+0000 7f487fbbb700 1 --2- 172.21.15.46:0/912160266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878103b20 0x7f4878107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.855+0000 7f487fbbb700 1 -- 172.21.15.46:0/912160266 >> 172.21.15.46:0/912160266 conn(0x7f48780fe990 msgr2=0x7f4878100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.855+0000 7f487fbbb700 1 -- 172.21.15.46:0/912160266 shutdown_connections 2024-03-18T20:46:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.855+0000 7f487fbbb700 1 -- 172.21.15.46:0/912160266 wait complete. 2024-03-18T20:46:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.856+0000 7f487fbbb700 1 Processor -- start 2024-03-18T20:46:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.856+0000 7f487fbbb700 1 -- start start 2024-03-18T20:46:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.856+0000 7f487fbbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878103170 0x7f4878199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.856+0000 7f487fbbb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4878108290 0x7f4878199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.856+0000 7f487fbbb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f487819de30 0x7f487819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.856+0000 7f487fbbb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487819e8b0 con 0x7f4878103170 2024-03-18T20:46:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.856+0000 7f487fbbb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487819ea20 con 0x7f487819de30 2024-03-18T20:46:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.856+0000 7f487fbbb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487819ebe0 con 0x7f4878108290 2024-03-18T20:46:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.856+0000 7f487d957700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878103170 0x7f4878199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f487d957700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878103170 0x7f4878199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43434/0 (socket says 172.21.15.46:43434) 2024-03-18T20:46:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f487d957700 1 -- 172.21.15.46:0/2423186181 learned_addr learned my addr 172.21.15.46:0/2423186181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f487e158700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f487819de30 0x7f487819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f487d156700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4878108290 0x7f4878199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f487d957700 1 -- 172.21.15.46:0/2423186181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4878108290 msgr2=0x7f4878199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:26.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f487d957700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4878108290 0x7f4878199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f487d957700 1 -- 172.21.15.46:0/2423186181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f487819de30 msgr2=0x7f487819e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:26.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f487d957700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f487819de30 0x7f487819e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f487d957700 1 -- 172.21.15.46:0/2423186181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4874009d00 con 0x7f4878103170 2024-03-18T20:46:26.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f487d957700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878103170 0x7f4878199550 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f487400d010 tx=0x7f487400be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:26.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f486effd700 1 -- 172.21.15.46:0/2423186181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487400e040 con 0x7f4878103170 2024-03-18T20:46:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f486effd700 1 -- 172.21.15.46:0/2423186181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4874007e90 con 0x7f4878103170 2024-03-18T20:46:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f487fbbb700 1 -- 172.21.15.46:0/2423186181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48781a60e0 con 0x7f4878103170 2024-03-18T20:46:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f486effd700 1 -- 172.21.15.46:0/2423186181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487401b450 con 0x7f4878103170 2024-03-18T20:46:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.857+0000 7f487fbbb700 1 -- 172.21.15.46:0/2423186181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48781a6560 con 0x7f4878103170 2024-03-18T20:46:26.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.860+0000 7f486effd700 1 -- 172.21.15.46:0/2423186181 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f487401d480 con 0x7f4878103170 2024-03-18T20:46:26.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.861+0000 7f486effd700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48640a1290 0x7f48640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.861+0000 7f487d156700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48640a1290 0x7f48640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.861+0000 7f486effd700 1 -- 172.21.15.46:0/2423186181 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f48740ce540 con 0x7f4878103170 2024-03-18T20:46:26.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.861+0000 7f487d156700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48640a1290 0x7f48640a3750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f487819aaa0 tx=0x7f4868009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:26.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.861+0000 7f488bd50b80 1 Processor -- start 2024-03-18T20:46:26.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.862+0000 7f488bd50b80 1 -- start start 2024-03-18T20:46:26.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.862+0000 7f488bd50b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb289ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.862+0000 7f488bd50b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb294d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.862+0000 7f488bd50b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb2952a0 0x5632eb297690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.862+0000 7f488bd50b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb1e3a10 con 0x5632eb289ae0 2024-03-18T20:46:26.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.862+0000 7f488bd50b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb1e13d0 con 0x5632eb2952a0 2024-03-18T20:46:26.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.862+0000 7f488bd50b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb1dccc0 con 0x5632eb28a400 2024-03-18T20:46:26.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.862+0000 7f487d957700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb289ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.862+0000 7f487d957700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb289ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43436/0 (socket says 172.21.15.46:43436) 2024-03-18T20:46:26.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.862+0000 7f487d957700 1 -- 172.21.15.46:0/3361770244 learned_addr learned my addr 172.21.15.46:0/3361770244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:26.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.862+0000 7f487d156700 1 --2- 172.21.15.46:0/3361770244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb294d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.862+0000 7f487e158700 1 --2- 172.21.15.46:0/3361770244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb2952a0 0x5632eb297690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.863+0000 7f487d957700 1 -- 172.21.15.46:0/3361770244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 msgr2=0x5632eb294d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.863+0000 7f487d957700 1 --2- 172.21.15.46:0/3361770244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb294d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.863+0000 7f487d957700 1 -- 172.21.15.46:0/3361770244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb2952a0 msgr2=0x5632eb297690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.863+0000 7f487d957700 1 --2- 172.21.15.46:0/3361770244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb2952a0 0x5632eb297690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.863+0000 7f487d957700 1 -- 172.21.15.46:0/3361770244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632eb05e430 con 0x5632eb289ae0 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.863+0000 7f487d156700 1 --2- 172.21.15.46:0/3361770244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb294d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.863+0000 7f487d957700 1 --2- 172.21.15.46:0/3361770244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb289ec0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f48740980d0 tx=0x7f4874000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d6d6a973f68971 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.863+0000 7f4863fff700 1 -- 172.21.15.46:0/3361770244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487409b660 con 0x5632eb289ae0 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.863+0000 7f4863fff700 1 -- 172.21.15.46:0/3361770244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f487409f490 con 0x5632eb289ae0 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.863+0000 7f4863fff700 1 -- 172.21.15.46:0/3361770244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874097d70 con 0x5632eb289ae0 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.864+0000 7f488bd50b80 1 -- 172.21.15.46:0/3361770244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 msgr2=0x5632eb289ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.864+0000 7f488bd50b80 1 --2- 172.21.15.46:0/3361770244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb289ec0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f48740980d0 tx=0x7f4874000f40 comp rx=0 tx=0).stop 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.864+0000 7f488bd50b80 1 -- 172.21.15.46:0/3361770244 shutdown_connections 2024-03-18T20:46:26.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.864+0000 7f488bd50b80 1 --2- 172.21.15.46:0/3361770244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb289ec0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.864+0000 7f488bd50b80 1 --2- 172.21.15.46:0/3361770244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb2952a0 0x5632eb297690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.864+0000 7f488bd50b80 1 --2- 172.21.15.46:0/3361770244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb294d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.864+0000 7f488bd50b80 1 -- 172.21.15.46:0/3361770244 >> 172.21.15.46:0/3361770244 conn(0x5632eb2002a0 msgr2=0x5632eb2860a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.864+0000 7f488bd50b80 1 -- 172.21.15.46:0/3361770244 shutdown_connections 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.864+0000 7f488bd50b80 1 -- 172.21.15.46:0/3361770244 wait complete. 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.865+0000 7f488bd50b80 1 -- start start 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.865+0000 7f488bd50b80 1 Processor -- start 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.865+0000 7f488bd50b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb294ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.865+0000 7f488bd50b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb291fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.865+0000 7f488bd50b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb2952a0 0x5632eb292710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.865+0000 7f488bd50b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb178e00 con 0x5632eb289ae0 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.865+0000 7f488bd50b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb178aa0 con 0x5632eb2952a0 2024-03-18T20:46:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.865+0000 7f488bd50b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb060720 con 0x5632eb28a400 2024-03-18T20:46:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.865+0000 7f488bd50b80 10 client.0 _ll_register_callbacks cb 0x5632eafe4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.865+0000 7f487d957700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb294ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f487d957700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb294ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43440/0 (socket says 172.21.15.46:43440) 2024-03-18T20:46:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f487d957700 1 -- 172.21.15.46:0/159038809 learned_addr learned my addr 172.21.15.46:0/159038809 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f487d156700 1 --2- 172.21.15.46:0/159038809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb291fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f487d957700 1 -- 172.21.15.46:0/159038809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 msgr2=0x5632eb291fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f487d957700 1 --2- 172.21.15.46:0/159038809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb291fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f487d957700 1 -- 172.21.15.46:0/159038809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb2952a0 msgr2=0x5632eb292710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f487d957700 1 --2- 172.21.15.46:0/159038809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb2952a0 0x5632eb292710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f487d957700 1 -- 172.21.15.46:0/159038809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4868017070 con 0x5632eb289ae0 2024-03-18T20:46:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f487d957700 1 --2- 172.21.15.46:0/159038809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb294ce0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f48740cc190 tx=0x7f48740a3a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18627ff774211a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f48617fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f48617fa700 1 -- 172.21.15.46:0/159038809 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487409b6e0 con 0x5632eb289ae0 2024-03-18T20:46:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f48617fa700 1 -- 172.21.15.46:0/159038809 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f487409f160 con 0x5632eb289ae0 2024-03-18T20:46:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f488bd50b80 1 -- 172.21.15.46:0/159038809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632eb27db50 con 0x5632eb289ae0 2024-03-18T20:46:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f48617fa700 1 -- 172.21.15.46:0/159038809 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f487401d730 con 0x5632eb289ae0 2024-03-18T20:46:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f488bd50b80 10 client.19203 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.866+0000 7f488bd50b80 1 -- 172.21.15.46:0/159038809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f48781a6560 con 0x5632eb289ae0 2024-03-18T20:46:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.867+0000 7f48617fa700 1 -- 172.21.15.46:0/159038809 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f487407e020 con 0x5632eb289ae0 2024-03-18T20:46:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.867+0000 7f48617fa700 1 -- 172.21.15.46:0/159038809 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f487409b8c0 con 0x5632eb289ae0 2024-03-18T20:46:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.867+0000 7f488bd50b80 10 client.19203 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.867+0000 7f488bd50b80 10 client.19203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.867+0000 7f488bd50b80 20 client.19203 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.867+0000 7f488bd50b80 1 --2- 172.21.15.46:0/159038809 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632eb2a4900 0x5632eb2a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.867+0000 7f488bd50b80 4 client.19203 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.867+0000 7f488bd50b80 1 -- 172.21.15.46:0/159038809 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632eb1d5780 con 0x5632eb2a4900 2024-03-18T20:46:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.867+0000 7f487d156700 1 --2- 172.21.15.46:0/159038809 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632eb2a4900 0x5632eb2a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.867+0000 7f48617fa700 10 client.19203 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.867+0000 7f487d156700 1 --2- 172.21.15.46:0/159038809 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632eb2a4900 0x5632eb2a6d20 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83970320d042db6a server_cookie=2b54d61b7016a632 in_seq=0 out_seq=0 2024-03-18T20:46:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.868+0000 7f48617fa700 1 -- 172.21.15.46:0/159038809 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5632eb1d5780 con 0x5632eb2a4900 2024-03-18T20:46:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.868+0000 7f48617fa700 10 client.19203 handle_command_reply: tid=0 2024-03-18T20:46:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.868+0000 7f488bd50b80 1 client.19203 shutdown 2024-03-18T20:46:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.868+0000 7f488bd50b80 1 -- 172.21.15.46:0/159038809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 msgr2=0x5632eb294ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.868+0000 7f488bd50b80 1 --2- 172.21.15.46:0/159038809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb294ce0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f48740cc190 tx=0x7f48740a3a50 comp rx=0 tx=0).stop 2024-03-18T20:46:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.869+0000 7f488bd50b80 1 -- 172.21.15.46:0/159038809 shutdown_connections 2024-03-18T20:46:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.869+0000 7f488bd50b80 1 --2- 172.21.15.46:0/159038809 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632eb2a4900 0x5632eb2a6d20 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.869+0000 7f488bd50b80 1 --2- 172.21.15.46:0/159038809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb294ce0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.869+0000 7f488bd50b80 1 --2- 172.21.15.46:0/159038809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb2952a0 0x5632eb292710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.869+0000 7f488bd50b80 1 --2- 172.21.15.46:0/159038809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb291fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.869+0000 7f48617fa700 0 client.19203 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.869+0000 7f488bd50b80 1 -- 172.21.15.46:0/159038809 >> 172.21.15.46:0/159038809 conn(0x5632eb2002a0 msgr2=0x5632eb285c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.869+0000 7f488bd50b80 1 -- 172.21.15.46:0/159038809 shutdown_connections 2024-03-18T20:46:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.869+0000 7f488bd50b80 1 -- 172.21.15.46:0/159038809 wait complete. 2024-03-18T20:46:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.869+0000 7f488bd50b80 20 client.19203 trim_cache size 0 max 16384 2024-03-18T20:46:26.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f488bd50b80 1 Processor -- start 2024-03-18T20:46:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f488bd50b80 1 -- start start 2024-03-18T20:46:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f488bd50b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb289ae0 0x5632eb29bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f488bd50b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb291550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f488bd50b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb2952a0 0x5632eb285bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f488bd50b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb280290 con 0x5632eb2952a0 2024-03-18T20:46:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f488bd50b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb282660 con 0x5632eb289ae0 2024-03-18T20:46:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f488bd50b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb28adb0 con 0x5632eb28a400 2024-03-18T20:46:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f487d957700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb289ae0 0x5632eb29bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f487e158700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb2952a0 0x5632eb285bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f487d957700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb289ae0 0x5632eb29bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52678/0 (socket says 172.21.15.46:52678) 2024-03-18T20:46:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f487d957700 1 -- 172.21.15.46:0/3684032801 learned_addr learned my addr 172.21.15.46:0/3684032801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f487d156700 1 --2- 172.21.15.46:0/3684032801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb291550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f487d957700 1 -- 172.21.15.46:0/3684032801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 msgr2=0x5632eb291550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f487d957700 1 --2- 172.21.15.46:0/3684032801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb291550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f487d957700 1 -- 172.21.15.46:0/3684032801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb2952a0 msgr2=0x5632eb285bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f487d957700 1 --2- 172.21.15.46:0/3684032801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb2952a0 0x5632eb285bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.888+0000 7f487d957700 1 -- 172.21.15.46:0/3684032801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632eb00abe0 con 0x5632eb289ae0 2024-03-18T20:46:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.889+0000 7f487d957700 1 --2- 172.21.15.46:0/3684032801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb289ae0 0x5632eb29bc90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f487409b360 tx=0x7f4874009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d618339246058c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.889+0000 7f486cda8700 1 -- 172.21.15.46:0/3684032801 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487409acc0 con 0x5632eb289ae0 2024-03-18T20:46:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.889+0000 7f486cda8700 1 -- 172.21.15.46:0/3684032801 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f487407e070 con 0x5632eb289ae0 2024-03-18T20:46:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.889+0000 7f486cda8700 1 -- 172.21.15.46:0/3684032801 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487409b8c0 con 0x5632eb289ae0 2024-03-18T20:46:26.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.889+0000 7f488bd50b80 1 -- 172.21.15.46:0/3684032801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb289ae0 msgr2=0x5632eb29bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:26.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.889+0000 7f488bd50b80 1 --2- 172.21.15.46:0/3684032801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb289ae0 0x5632eb29bc90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f487409b360 tx=0x7f4874009040 comp rx=0 tx=0).stop 2024-03-18T20:46:26.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.889+0000 7f488bd50b80 1 -- 172.21.15.46:0/3684032801 shutdown_connections 2024-03-18T20:46:26.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 --2- 172.21.15.46:0/3684032801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb2952a0 0x5632eb285bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 --2- 172.21.15.46:0/3684032801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb289ae0 0x5632eb29bc90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 --2- 172.21.15.46:0/3684032801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb28a400 0x5632eb291550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 -- 172.21.15.46:0/3684032801 >> 172.21.15.46:0/3684032801 conn(0x5632eb2002a0 msgr2=0x5632eb2948f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 -- 172.21.15.46:0/3684032801 shutdown_connections 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 -- 172.21.15.46:0/3684032801 wait complete. 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 -- start start 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 Processor -- start 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb29aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb28a400 0x5632eb29afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb2952a0 0x5632eb2f8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb2f8c80 con 0x5632eb289ae0 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb2f8dc0 con 0x5632eb28a400 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eb2f8f30 con 0x5632eb2952a0 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.890+0000 7f488bd50b80 10 client.0 _ll_register_callbacks cb 0x7f4870009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f487d957700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb29aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f487d156700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb28a400 0x5632eb29afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f487d957700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb29aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43468/0 (socket says 172.21.15.46:43468) 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f487e158700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb2952a0 0x5632eb2f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f487d957700 1 -- 172.21.15.46:0/520970727 learned_addr learned my addr 172.21.15.46:0/520970727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f487d957700 1 -- 172.21.15.46:0/520970727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb2952a0 msgr2=0x5632eb2f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f487d957700 1 --2- 172.21.15.46:0/520970727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb2952a0 0x5632eb2f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f487d957700 1 -- 172.21.15.46:0/520970727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb28a400 msgr2=0x5632eb29afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f487d957700 1 --2- 172.21.15.46:0/520970727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb28a400 0x5632eb29afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f487d957700 1 -- 172.21.15.46:0/520970727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48781a6560 con 0x5632eb289ae0 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f487d156700 1 --2- 172.21.15.46:0/520970727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb28a400 0x5632eb29afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f487d957700 1 --2- 172.21.15.46:0/520970727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb29aa80 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f48740062a0 tx=0x7f487409b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d3956908a148d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f48617fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f48617fa700 1 -- 172.21.15.46:0/520970727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487409acc0 con 0x5632eb289ae0 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f48617fa700 1 -- 172.21.15.46:0/520970727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f487407f080 con 0x5632eb289ae0 2024-03-18T20:46:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f488bd50b80 1 -- 172.21.15.46:0/520970727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f487409f490 con 0x5632eb289ae0 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f48617fa700 1 -- 172.21.15.46:0/520970727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f487409f160 con 0x5632eb289ae0 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f488bd50b80 10 client.19209 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.891+0000 7f488bd50b80 1 -- 172.21.15.46:0/520970727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5632eb1a2f20 con 0x5632eb289ae0 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.892+0000 7f48617fa700 1 -- 172.21.15.46:0/520970727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f487409f490 con 0x5632eb289ae0 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.892+0000 7f48617fa700 1 -- 172.21.15.46:0/520970727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f48740860b0 con 0x5632eb289ae0 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.892+0000 7f488bd50b80 10 client.19209 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.892+0000 7f488bd50b80 10 client.19209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.892+0000 7f488bd50b80 20 client.19209 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.892+0000 7f488bd50b80 1 --2- 172.21.15.46:0/520970727 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632eb2fa240 0x5632eb2fc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.892+0000 7f488bd50b80 4 client.19209 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.892+0000 7f488bd50b80 1 -- 172.21.15.46:0/520970727 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5632eb02bd70 con 0x5632eb2fa240 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.892+0000 7f487d156700 1 --2- 172.21.15.46:0/520970727 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632eb2fa240 0x5632eb2fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.892+0000 7f48617fa700 10 client.19209 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.892+0000 7f487d156700 1 --2- 172.21.15.46:0/520970727 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632eb2fa240 0x5632eb2fc630 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3afd0d4c8e726f server_cookie=59e1f2889a6d9966 in_seq=0 out_seq=0 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.893+0000 7f48617fa700 1 -- 172.21.15.46:0/520970727 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5632eb02bd70 con 0x5632eb2fa240 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.893+0000 7f48617fa700 10 client.19209 handle_command_reply: tid=0 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.893+0000 7f488bd50b80 1 client.19209 shutdown 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f488bd50b80 1 -- 172.21.15.46:0/520970727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 msgr2=0x5632eb29aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f488bd50b80 1 --2- 172.21.15.46:0/520970727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb29aa80 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f48740062a0 tx=0x7f487409b0c0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f488bd50b80 1 -- 172.21.15.46:0/520970727 shutdown_connections 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f488bd50b80 1 --2- 172.21.15.46:0/520970727 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5632eb2fa240 0x5632eb2fc630 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f488bd50b80 1 --2- 172.21.15.46:0/520970727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632eb289ae0 0x5632eb29aa80 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f488bd50b80 1 --2- 172.21.15.46:0/520970727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5632eb2952a0 0x5632eb2f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f488bd50b80 1 --2- 172.21.15.46:0/520970727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eb28a400 0x5632eb29afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f48617fa700 0 client.19209 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f488bd50b80 1 -- 172.21.15.46:0/520970727 >> 172.21.15.46:0/520970727 conn(0x5632eb2002a0 msgr2=0x5632eb28d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f488bd50b80 1 -- 172.21.15.46:0/520970727 shutdown_connections 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f488bd50b80 1 -- 172.21.15.46:0/520970727 wait complete. 2024-03-18T20:46:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f488bd50b80 20 client.19209 trim_cache size 0 max 16384 2024-03-18T20:46:26.902 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:46:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:46:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "abe6e7b0-9f2d-4209-b839-79859c4d1837", 2024-03-18T20:46:26.902 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:46:26.902 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:46:26.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f487fbbb700 1 -- 172.21.15.46:0/2423186181 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48640a1290 msgr2=0x7f48640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:26.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f487fbbb700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48640a1290 0x7f48640a3750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f487819aaa0 tx=0x7f4868009260 comp rx=0 tx=0).stop 2024-03-18T20:46:26.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f487fbbb700 1 -- 172.21.15.46:0/2423186181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878103170 msgr2=0x7f4878199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:26.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.894+0000 7f487fbbb700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878103170 0x7f4878199550 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f487400d010 tx=0x7f487400be40 comp rx=0 tx=0).stop 2024-03-18T20:46:26.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.895+0000 7f487fbbb700 1 -- 172.21.15.46:0/2423186181 shutdown_connections 2024-03-18T20:46:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.895+0000 7f487fbbb700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f48640a1290 0x7f48640a3750 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.895+0000 7f487fbbb700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4878103170 0x7f4878199550 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.895+0000 7f487fbbb700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f487819de30 0x7f487819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.895+0000 7f487fbbb700 1 --2- 172.21.15.46:0/2423186181 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4878108290 0x7f4878199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.895+0000 7f487fbbb700 1 -- 172.21.15.46:0/2423186181 >> 172.21.15.46:0/2423186181 conn(0x7f48780fe990 msgr2=0x7f48780ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.895+0000 7f487fbbb700 1 -- 172.21.15.46:0/2423186181 shutdown_connections 2024-03-18T20:46:26.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:26.895+0000 7f487fbbb700 1 -- 172.21.15.46:0/2423186181 wait complete. 2024-03-18T20:46:26.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:46:26.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:46:27.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.101+0000 7ff5c08a6700 1 -- 172.21.15.46:0/1168885492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5bc100820 msgr2=0x7ff5bc104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:27.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.101+0000 7ff5c08a6700 1 --2- 172.21.15.46:0/1168885492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5bc100820 0x7ff5bc104840 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009990 tx=0x7ff5b0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.101+0000 7ff5c08a6700 1 -- 172.21.15.46:0/1168885492 shutdown_connections 2024-03-18T20:46:27.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.101+0000 7ff5c08a6700 1 --2- 172.21.15.46:0/1168885492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5bc104f70 0x7ff5bc10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.101+0000 7ff5c08a6700 1 --2- 172.21.15.46:0/1168885492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5bc0ffe70 0x7ff5bc100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.101+0000 7ff5c08a6700 1 --2- 172.21.15.46:0/1168885492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5bc100820 0x7ff5bc104840 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.101+0000 7ff5c08a6700 1 -- 172.21.15.46:0/1168885492 >> 172.21.15.46:0/1168885492 conn(0x7ff5bc076d50 msgr2=0x7ff5bc077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:27.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.101+0000 7ff5c08a6700 1 -- 172.21.15.46:0/1168885492 shutdown_connections 2024-03-18T20:46:27.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.101+0000 7ff5c08a6700 1 -- 172.21.15.46:0/1168885492 wait complete. 2024-03-18T20:46:27.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.102+0000 7ff5c08a6700 1 Processor -- start 2024-03-18T20:46:27.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.102+0000 7ff5c08a6700 1 -- start start 2024-03-18T20:46:27.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.102+0000 7ff5c08a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5bc0ffe70 0x7ff5bc1018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.102+0000 7ff5c08a6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5bc104f70 0x7ff5bc101e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.102+0000 7ff5c08a6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5bc102360 0x7ff5bc1a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.102+0000 7ff5c08a6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc1a59f0 con 0x7ff5bc0ffe70 2024-03-18T20:46:27.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.102+0000 7ff5c08a6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc1a5b60 con 0x7ff5bc104f70 2024-03-18T20:46:27.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.102+0000 7ff5c08a6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc1a5d20 con 0x7ff5bc102360 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5ba59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5bc0ffe70 0x7ff5bc1018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5b9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5bc104f70 0x7ff5bc101e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5b9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5bc104f70 0x7ff5bc101e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52698/0 (socket says 172.21.15.46:52698) 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5ba59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5bc0ffe70 0x7ff5bc1018e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43494/0 (socket says 172.21.15.46:43494) 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5b9d9b700 1 -- 172.21.15.46:0/3226642892 learned_addr learned my addr 172.21.15.46:0/3226642892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5bad9d700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5bc102360 0x7ff5bc1a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5b9d9b700 1 -- 172.21.15.46:0/3226642892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5bc102360 msgr2=0x7ff5bc1a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5b9d9b700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5bc102360 0x7ff5bc1a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5b9d9b700 1 -- 172.21.15.46:0/3226642892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5bc0ffe70 msgr2=0x7ff5bc1018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5b9d9b700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5bc0ffe70 0x7ff5bc1018e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5b9d9b700 1 -- 172.21.15.46:0/3226642892 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac009520 con 0x7ff5bc104f70 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5bc0ffe70 0x7ff5bc1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5b9d9b700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5bc104f70 0x7ff5bc101e20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005770 tx=0x7ff5b00057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5ab7fe700 1 -- 172.21.15.46:0/3226642892 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b001d070 con 0x7ff5bc104f70 2024-03-18T20:46:27.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5ab7fe700 1 -- 172.21.15.46:0/3226642892 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b0021410 con 0x7ff5bc104f70 2024-03-18T20:46:27.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5ab7fe700 1 -- 172.21.15.46:0/3226642892 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0017650 con 0x7ff5bc104f70 2024-03-18T20:46:27.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5c08a6700 1 -- 172.21.15.46:0/3226642892 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b00095f0 con 0x7ff5bc104f70 2024-03-18T20:46:27.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.103+0000 7ff5c08a6700 1 -- 172.21.15.46:0/3226642892 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc1a6260 con 0x7ff5bc104f70 2024-03-18T20:46:27.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.106+0000 7ff5ab7fe700 1 -- 172.21.15.46:0/3226642892 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff5b000fb40 con 0x7ff5bc104f70 2024-03-18T20:46:27.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.107+0000 7ff5ab7fe700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff5a40a1310 0x7ff5a40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.107+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff5a40a1310 0x7ff5a40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.107+0000 7ff5ab7fe700 1 -- 172.21.15.46:0/3226642892 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff5b00cde20 con 0x7ff5bc104f70 2024-03-18T20:46:27.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.107+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff5a40a1310 0x7ff5a40a37d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009fd0 tx=0x7ff5ac009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:27.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.108+0000 7ff5cca3bb80 1 Processor -- start 2024-03-18T20:46:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.108+0000 7ff5cca3bb80 1 -- start start 2024-03-18T20:46:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.108+0000 7ff5cca3bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 0x557e9e642e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.108+0000 7ff5cca3bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e643380 0x557e9e64dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.108+0000 7ff5cca3bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e64e220 0x557e9e650610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.108+0000 7ff5cca3bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e59c990 con 0x557e9e642a60 2024-03-18T20:46:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.108+0000 7ff5cca3bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e59a350 con 0x557e9e64e220 2024-03-18T20:46:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.108+0000 7ff5cca3bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e595c40 con 0x557e9e643380 2024-03-18T20:46:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5ba59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 0x557e9e642e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5b9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e643380 0x557e9e64dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5ba59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 0x557e9e642e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43506/0 (socket says 172.21.15.46:43506) 2024-03-18T20:46:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5ba59c700 1 -- 172.21.15.46:0/1201015158 learned_addr learned my addr 172.21.15.46:0/1201015158 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5bad9d700 1 --2- 172.21.15.46:0/1201015158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e64e220 0x557e9e650610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5ba59c700 1 -- 172.21.15.46:0/1201015158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e643380 msgr2=0x557e9e64dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/1201015158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e643380 0x557e9e64dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5ba59c700 1 -- 172.21.15.46:0/1201015158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e64e220 msgr2=0x557e9e650610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:27.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/1201015158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e64e220 0x557e9e650610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5ba59c700 1 -- 172.21.15.46:0/1201015158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e9e4173b0 con 0x557e9e642a60 2024-03-18T20:46:27.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/1201015158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 0x557e9e642e40 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac012fd0 tx=0x7ff5ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fb8e752a9f60c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:27.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5a8ff9700 1 -- 172.21.15.46:0/1201015158 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac01b600 con 0x557e9e642a60 2024-03-18T20:46:27.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5a8ff9700 1 -- 172.21.15.46:0/1201015158 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5ac00ee40 con 0x557e9e642a60 2024-03-18T20:46:27.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.109+0000 7ff5a8ff9700 1 -- 172.21.15.46:0/1201015158 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac026b60 con 0x557e9e642a60 2024-03-18T20:46:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.110+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/1201015158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 msgr2=0x557e9e642e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.110+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/1201015158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 0x557e9e642e40 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac012fd0 tx=0x7ff5ac014000 comp rx=0 tx=0).stop 2024-03-18T20:46:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.110+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/1201015158 shutdown_connections 2024-03-18T20:46:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.110+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/1201015158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 0x557e9e642e40 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.110+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/1201015158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e64e220 0x557e9e650610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.110+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/1201015158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e643380 0x557e9e64dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.110+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/1201015158 >> 172.21.15.46:0/1201015158 conn(0x557e9e5b9220 msgr2=0x557e9e63f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:27.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.110+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/1201015158 shutdown_connections 2024-03-18T20:46:27.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.110+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/1201015158 wait complete. 2024-03-18T20:46:27.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5cca3bb80 1 -- start start 2024-03-18T20:46:27.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5cca3bb80 1 Processor -- start 2024-03-18T20:46:27.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5cca3bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 0x557e9e64dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5cca3bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e643380 0x557e9e64af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5cca3bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e64e220 0x557e9e64b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5cca3bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e531d80 con 0x557e9e64e220 2024-03-18T20:46:27.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5cca3bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e531a20 con 0x557e9e642a60 2024-03-18T20:46:27.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5cca3bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e4196a0 con 0x557e9e643380 2024-03-18T20:46:27.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5cca3bb80 10 client.0 _ll_register_callbacks cb 0x557e9e39d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:27.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5bad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e64e220 0x557e9e64b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5ba59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 0x557e9e64dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5ba59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 0x557e9e64dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52718/0 (socket says 172.21.15.46:52718) 2024-03-18T20:46:27.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5bad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e64e220 0x557e9e64b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43522/0 (socket says 172.21.15.46:43522) 2024-03-18T20:46:27.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5ba59c700 1 -- 172.21.15.46:0/332629707 learned_addr learned my addr 172.21.15.46:0/332629707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:27.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.111+0000 7ff5b9d9b700 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e643380 0x557e9e64af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5ba59c700 1 -- 172.21.15.46:0/332629707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e643380 msgr2=0x557e9e64af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:27.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e643380 0x557e9e64af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5ba59c700 1 -- 172.21.15.46:0/332629707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e64e220 msgr2=0x557e9e64b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:27.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e64e220 0x557e9e64b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5ba59c700 1 -- 172.21.15.46:0/332629707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac017050 con 0x557e9e642a60 2024-03-18T20:46:27.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5b9d9b700 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e643380 0x557e9e64af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5bad9d700 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e64e220 0x557e9e64b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 0x557e9e64dc80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00efd0 tx=0x7ff5ac005500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdcb115dc699f5a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5a1ffb700 1 -- 172.21.15.46:0/332629707 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0269e0 con 0x557e9e642a60 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5a1ffb700 1 -- 172.21.15.46:0/332629707 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff5ac026b40 con 0x557e9e642a60 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/332629707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e9e636ad0 con 0x557e9e642a60 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5a1ffb700 1 -- 172.21.15.46:0/332629707 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5ac026500 con 0x557e9e642a60 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5cca3bb80 10 client.28850 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/332629707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557e9e3c3b60 con 0x557e9e642a60 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.112+0000 7ff5a1ffb700 1 -- 172.21.15.46:0/332629707 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e9e636ad0 con 0x557e9e642a60 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.113+0000 7ff5a1ffb700 1 -- 172.21.15.46:0/332629707 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff5ac02f600 con 0x557e9e642a60 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.113+0000 7ff5cca3bb80 10 client.28850 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.113+0000 7ff5cca3bb80 10 client.28850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:27.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.113+0000 7ff5cca3bb80 20 client.28850 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.113+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e9e65dce0 0x557e9e660100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.113+0000 7ff5cca3bb80 4 client.28850 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.113+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/332629707 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e9e58e700 con 0x557e9e65dce0 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.113+0000 7ff5b9d9b700 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e9e65dce0 0x557e9e660100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.113+0000 7ff5a1ffb700 10 client.28850 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.113+0000 7ff5b9d9b700 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e9e65dce0 0x557e9e660100 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12c397618300fb95 server_cookie=4c36c2460d3f4f1 in_seq=0 out_seq=0 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5a1ffb700 1 -- 172.21.15.46:0/332629707 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557e9e58e700 con 0x557e9e65dce0 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5a1ffb700 10 client.28850 handle_command_reply: tid=0 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5cca3bb80 1 client.28850 shutdown 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/332629707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 msgr2=0x557e9e64dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 0x557e9e64dc80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00efd0 tx=0x7ff5ac005500 comp rx=0 tx=0).stop 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/332629707 shutdown_connections 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e9e65dce0 0x557e9e660100 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e64e220 0x557e9e64b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 0x557e9e64dc80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5a1ffb700 0 client.28850 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/332629707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e643380 0x557e9e64af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/332629707 >> 172.21.15.46:0/332629707 conn(0x557e9e5b9220 msgr2=0x557e9e63ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/332629707 shutdown_connections 2024-03-18T20:46:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/332629707 wait complete. 2024-03-18T20:46:27.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.114+0000 7ff5cca3bb80 20 client.28850 trim_cache size 0 max 16384 2024-03-18T20:46:27.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.132+0000 7ff5cca3bb80 1 Processor -- start 2024-03-18T20:46:27.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.132+0000 7ff5cca3bb80 1 -- start start 2024-03-18T20:46:27.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5cca3bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 0x557e9e6459c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5cca3bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e643380 0x557e9e654eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5cca3bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e64e220 0x557e9e63eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5cca3bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e63b5e0 con 0x557e9e643380 2024-03-18T20:46:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5cca3bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e643d30 con 0x557e9e642a60 2024-03-18T20:46:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5cca3bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e64bcb0 con 0x557e9e64e220 2024-03-18T20:46:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5b9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e643380 0x557e9e654eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5ba59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 0x557e9e6459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5ba59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 0x557e9e6459c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52730/0 (socket says 172.21.15.46:52730) 2024-03-18T20:46:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5b9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e643380 0x557e9e654eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43530/0 (socket says 172.21.15.46:43530) 2024-03-18T20:46:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5ba59c700 1 -- 172.21.15.46:0/2376202075 learned_addr learned my addr 172.21.15.46:0/2376202075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:27.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5bad9d700 1 --2- 172.21.15.46:0/2376202075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e64e220 0x557e9e63eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5ba59c700 1 -- 172.21.15.46:0/2376202075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e64e220 msgr2=0x557e9e63eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/2376202075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e64e220 0x557e9e63eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5ba59c700 1 -- 172.21.15.46:0/2376202075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e643380 msgr2=0x557e9e654eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/2376202075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e643380 0x557e9e654eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5ba59c700 1 -- 172.21.15.46:0/2376202075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e9e55bea0 con 0x557e9e642a60 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.133+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/2376202075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 0x557e9e6459c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00e6a0 tx=0x7ff5ac009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=755b53d6284e7e7b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.134+0000 7ff5a97fa700 1 -- 172.21.15.46:0/2376202075 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac02f440 con 0x557e9e642a60 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.134+0000 7ff5a97fa700 1 -- 172.21.15.46:0/2376202075 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5ac01f070 con 0x557e9e642a60 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.134+0000 7ff5a97fa700 1 -- 172.21.15.46:0/2376202075 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac01b660 con 0x557e9e642a60 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.134+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2376202075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 msgr2=0x557e9e6459c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.134+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/2376202075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 0x557e9e6459c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00e6a0 tx=0x7ff5ac009330 comp rx=0 tx=0).stop 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.134+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2376202075 shutdown_connections 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.134+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/2376202075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e64e220 0x557e9e63eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.134+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/2376202075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e642a60 0x557e9e6459c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.134+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/2376202075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e643380 0x557e9e654eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.134+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2376202075 >> 172.21.15.46:0/2376202075 conn(0x557e9e5b9220 msgr2=0x557e9e659600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.134+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2376202075 shutdown_connections 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.134+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2376202075 wait complete. 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5cca3bb80 1 -- start start 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5cca3bb80 1 Processor -- start 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5cca3bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 0x557e9e652e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5cca3bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e643380 0x557e9e6533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5cca3bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e64e220 0x557e9e653910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5cca3bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e6b0c70 con 0x557e9e642a60 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5cca3bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e6b0de0 con 0x557e9e643380 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5cca3bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e9e6b0fa0 con 0x557e9e64e220 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5cca3bb80 10 client.0 _ll_register_callbacks cb 0x7ff5b4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5b9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e643380 0x557e9e6533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5b9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e643380 0x557e9e6533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52736/0 (socket says 172.21.15.46:52736) 2024-03-18T20:46:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5b9d9b700 1 -- 172.21.15.46:0/2462737856 learned_addr learned my addr 172.21.15.46:0/2462737856 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.135+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 0x557e9e652e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5bad9d700 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e64e220 0x557e9e653910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5b9d9b700 1 -- 172.21.15.46:0/2462737856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e64e220 msgr2=0x557e9e653910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5b9d9b700 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e64e220 0x557e9e653910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5b9d9b700 1 -- 172.21.15.46:0/2462737856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 msgr2=0x557e9e652e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5b9d9b700 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 0x557e9e652e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5b9d9b700 1 -- 172.21.15.46:0/2462737856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e9e3c3b60 con 0x557e9e643380 2024-03-18T20:46:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 0x557e9e652e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5b9d9b700 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e643380 0x557e9e6533d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00cb5b0 tx=0x7ff5b0005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcc0ac9528053285 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:46:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5a1ffb700 1 -- 172.21.15.46:0/2462737856 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b009edd0 con 0x557e9e643380 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5a1ffb700 1 -- 172.21.15.46:0/2462737856 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff5b00a3390 con 0x557e9e643380 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2462737856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5ac00ee40 con 0x557e9e643380 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5a1ffb700 1 -- 172.21.15.46:0/2462737856 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5b00b9250 con 0x557e9e643380 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5cca3bb80 10 client.28856 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2462737856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557e9e41bcd0 con 0x557e9e643380 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5a1ffb700 1 -- 172.21.15.46:0/2462737856 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5ac00ee40 con 0x557e9e643380 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5a1ffb700 1 -- 172.21.15.46:0/2462737856 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff5b009b320 con 0x557e9e643380 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5cca3bb80 10 client.28856 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5cca3bb80 10 client.28856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.136+0000 7ff5cca3bb80 20 client.28856 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e9e6b24a0 0x557e9e6b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5cca3bb80 4 client.28856 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:46:27.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2462737856 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557e9e3e4cf0 con 0x557e9e6b24a0 2024-03-18T20:46:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e9e6b24a0 0x557e9e6b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5ba59c700 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e9e6b24a0 0x557e9e6b48c0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f2fee6e6adfd5dc server_cookie=d80017d87c3e1c22 in_seq=0 out_seq=0 2024-03-18T20:46:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5a1ffb700 10 client.28856 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5a1ffb700 1 -- 172.21.15.46:0/2462737856 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557e9e3e4cf0 con 0x557e9e6b24a0 2024-03-18T20:46:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5a1ffb700 10 client.28856 handle_command_reply: tid=0 2024-03-18T20:46:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5cca3bb80 1 client.28856 shutdown 2024-03-18T20:46:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2462737856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e643380 msgr2=0x557e9e6533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e643380 0x557e9e6533d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00cb5b0 tx=0x7ff5b0005c00 comp rx=0 tx=0).stop 2024-03-18T20:46:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2462737856 shutdown_connections 2024-03-18T20:46:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e9e6b24a0 0x557e9e6b48c0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e9e642a60 0x557e9e652e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e9e64e220 0x557e9e653910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5cca3bb80 1 --2- 172.21.15.46:0/2462737856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e9e643380 0x557e9e6533d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5a1ffb700 0 client.28856 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.137+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2462737856 >> 172.21.15.46:0/2462737856 conn(0x557e9e5b9220 msgr2=0x557e9e6462c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2462737856 shutdown_connections 2024-03-18T20:46:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5cca3bb80 1 -- 172.21.15.46:0/2462737856 wait complete. 2024-03-18T20:46:27.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5cca3bb80 20 client.28856 trim_cache size 0 max 16384 2024-03-18T20:46:27.149 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:46:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:46:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:46:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: "abe6e7b0-9f2d-4209-b839-79859c4d1837": { 2024-03-18T20:46:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:46:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "abe6e7b0-9f2d-4209-b839-79859c4d1837", 2024-03-18T20:46:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:46:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:46:27.149 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:46:27.149 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:46:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5c08a6700 1 -- 172.21.15.46:0/3226642892 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff5a40a1310 msgr2=0x7ff5a40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5c08a6700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff5a40a1310 0x7ff5a40a37d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009fd0 tx=0x7ff5ac009260 comp rx=0 tx=0).stop 2024-03-18T20:46:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5c08a6700 1 -- 172.21.15.46:0/3226642892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5bc104f70 msgr2=0x7ff5bc101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5c08a6700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5bc104f70 0x7ff5bc101e20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005770 tx=0x7ff5b00057a0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5c08a6700 1 -- 172.21.15.46:0/3226642892 shutdown_connections 2024-03-18T20:46:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5c08a6700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff5a40a1310 0x7ff5a40a37d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5c08a6700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5bc0ffe70 0x7ff5bc1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5c08a6700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5bc102360 0x7ff5bc1a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5c08a6700 1 --2- 172.21.15.46:0/3226642892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5bc104f70 0x7ff5bc101e20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5c08a6700 1 -- 172.21.15.46:0/3226642892 >> 172.21.15.46:0/3226642892 conn(0x7ff5bc076d50 msgr2=0x7ff5bc0695f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5c08a6700 1 -- 172.21.15.46:0/3226642892 shutdown_connections 2024-03-18T20:46:27.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:27.138+0000 7ff5c08a6700 1 -- 172.21.15.46:0/3226642892 wait complete. 2024-03-18T20:46:27.155 INFO:tasks.cephfs.filesystem:scrub status for tag:abe6e7b0-9f2d-4209-b839-79859c4d1837 - {'path': '/', 'tag': 'abe6e7b0-9f2d-4209-b839-79859c4d1837', 'options': 'recursive,force'} 2024-03-18T20:46:27.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:46:27.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:46:27.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:46:27.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:46:27.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:46:27.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:46:27.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:46:27.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:27 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:46:27.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:46:27.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:46:27.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:46:27.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:46:27.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:46:27.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:46:27.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:27 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 384796 3.69 MB/sec execute 442 sec latency 417.734 ms 2024-03-18T20:46:27.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:46:27.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:46:27.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:46:27.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:46:27.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:46:27.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:46:27.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:46:27.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:27.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:27 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:28 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:46:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:28 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:28 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:46:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:28 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:28 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:46:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:28 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:28 smithi155 ceph-mon[48344]: pgmap v322: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:46:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:28 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:46:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:28 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:28 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:46:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:28 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:28 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:46:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:28 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:28.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:28 smithi046 ceph-mon[43495]: pgmap v322: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:46:28.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 384796 3.68 MB/sec execute 443 sec latency 1417.849 ms 2024-03-18T20:46:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:28 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:46:28.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:28 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:28.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:28 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:46:28.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:28 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:28.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:28 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:46:28.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:28 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:46:28.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:28 smithi073 ceph-mon[47384]: pgmap v322: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T20:46:29.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 384796 3.67 MB/sec execute 444 sec latency 2417.938 ms 2024-03-18T20:46:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.385+0000 7f065dd1b700 1 -- 172.21.15.46:0/1657604655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0658105c60 msgr2=0x7f0658106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.385+0000 7f065dd1b700 1 --2- 172.21.15.46:0/1657604655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0658105c60 0x7f0658106040 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f0648009990 tx=0x7f0648009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.385+0000 7f065dd1b700 1 -- 172.21.15.46:0/1657604655 shutdown_connections 2024-03-18T20:46:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.385+0000 7f065dd1b700 1 --2- 172.21.15.46:0/1657604655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0658105c60 0x7f0658106040 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.385+0000 7f065dd1b700 1 --2- 172.21.15.46:0/1657604655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0658100570 0x7f065810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.385+0000 7f065dd1b700 1 --2- 172.21.15.46:0/1657604655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06580ffbb0 0x7f0658100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.385+0000 7f065dd1b700 1 -- 172.21.15.46:0/1657604655 >> 172.21.15.46:0/1657604655 conn(0x7f0658076d50 msgr2=0x7f0658077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.385+0000 7f065dd1b700 1 -- 172.21.15.46:0/1657604655 shutdown_connections 2024-03-18T20:46:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.385+0000 7f065dd1b700 1 -- 172.21.15.46:0/1657604655 wait complete. 2024-03-18T20:46:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.386+0000 7f065dd1b700 1 Processor -- start 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.386+0000 7f065dd1b700 1 -- start start 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.386+0000 7f065dd1b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06580ffbb0 0x7f0658194f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.386+0000 7f065dd1b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0658100570 0x7f0658195440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.386+0000 7f065dd1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0658105c60 0x7f0658199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.386+0000 7f065dd1b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0658199db0 con 0x7f0658105c60 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.386+0000 7f065dd1b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0658199f20 con 0x7f0658100570 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.386+0000 7f065dd1b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065819a0e0 con 0x7f06580ffbb0 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.386+0000 7f0657fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0658105c60 0x7f0658199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f0657fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0658105c60 0x7f0658199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43552/0 (socket says 172.21.15.46:43552) 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f0657fff700 1 -- 172.21.15.46:0/1222559068 learned_addr learned my addr 172.21.15.46:0/1222559068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f064ffff700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0658100570 0x7f0658195440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f06577fe700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06580ffbb0 0x7f0658194f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f0657fff700 1 -- 172.21.15.46:0/1222559068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06580ffbb0 msgr2=0x7f0658194f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f0657fff700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06580ffbb0 0x7f0658194f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f0657fff700 1 -- 172.21.15.46:0/1222559068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0658100570 msgr2=0x7f0658195440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f0657fff700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0658100570 0x7f0658195440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f0657fff700 1 -- 172.21.15.46:0/1222559068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06480095f0 con 0x7f0658105c60 2024-03-18T20:46:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f0657fff700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0658105c60 0x7f0658199770 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f065000ef30 tx=0x7f065000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f06557fa700 1 -- 172.21.15.46:0/1222559068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650018070 con 0x7f0658105c60 2024-03-18T20:46:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f06557fa700 1 -- 172.21.15.46:0/1222559068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f065000f040 con 0x7f0658105c60 2024-03-18T20:46:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f065dd1b700 1 -- 172.21.15.46:0/1222559068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065819a360 con 0x7f0658105c60 2024-03-18T20:46:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f06557fa700 1 -- 172.21.15.46:0/1222559068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065000cd00 con 0x7f0658105c60 2024-03-18T20:46:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.387+0000 7f065dd1b700 1 -- 172.21.15.46:0/1222559068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06581a1c00 con 0x7f0658105c60 2024-03-18T20:46:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.390+0000 7f06557fa700 1 -- 172.21.15.46:0/1222559068 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0650003a00 con 0x7f0658105c60 2024-03-18T20:46:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.390+0000 7f06557fa700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f06440a34a0 0x7f06440a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.390+0000 7f06557fa700 1 -- 172.21.15.46:0/1222559068 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f06500cc7f0 con 0x7f0658105c60 2024-03-18T20:46:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.390+0000 7f06577fe700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f06440a34a0 0x7f06440a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.391+0000 7f06577fe700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f06440a34a0 0x7f06440a5960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f064800b2e0 tx=0x7f0648005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.391+0000 7f0669eb0b80 1 Processor -- start 2024-03-18T20:46:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f0669eb0b80 1 -- start start 2024-03-18T20:46:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f0669eb0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6cb570 0x556d2b6cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f0669eb0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cbe90 0x556d2b6d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f0669eb0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6d6db0 0x556d2b6d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f0669eb0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2b625930 con 0x556d2b6cb570 2024-03-18T20:46:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f0669eb0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2b6232f0 con 0x556d2b6cbe90 2024-03-18T20:46:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f0669eb0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2b61ebe0 con 0x556d2b6d6db0 2024-03-18T20:46:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f06577fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6cb570 0x556d2b6cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f06577fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6cb570 0x556d2b6cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43566/0 (socket says 172.21.15.46:43566) 2024-03-18T20:46:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f06577fe700 1 -- 172.21.15.46:0/174175027 learned_addr learned my addr 172.21.15.46:0/174175027 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f064ffff700 1 --2- 172.21.15.46:0/174175027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cbe90 0x556d2b6d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f0657fff700 1 --2- 172.21.15.46:0/174175027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6d6db0 0x556d2b6d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f06577fe700 1 -- 172.21.15.46:0/174175027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6d6db0 msgr2=0x556d2b6d91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f06577fe700 1 --2- 172.21.15.46:0/174175027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6d6db0 0x556d2b6d91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f06577fe700 1 -- 172.21.15.46:0/174175027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cbe90 msgr2=0x556d2b6d6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f06577fe700 1 --2- 172.21.15.46:0/174175027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cbe90 0x556d2b6d6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f06577fe700 1 -- 172.21.15.46:0/174175027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d2b4a0350 con 0x556d2b6cb570 2024-03-18T20:46:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.392+0000 7f064ffff700 1 --2- 172.21.15.46:0/174175027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cbe90 0x556d2b6d6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.393+0000 7f06577fe700 1 --2- 172.21.15.46:0/174175027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6cb570 0x556d2b6cb950 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f0648009990 tx=0x7f0648020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2babcdd809ffb888 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.393+0000 7f064dffb700 1 -- 172.21.15.46:0/174175027 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648004500 con 0x556d2b6cb570 2024-03-18T20:46:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.393+0000 7f064dffb700 1 -- 172.21.15.46:0/174175027 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f064801e040 con 0x556d2b6cb570 2024-03-18T20:46:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.393+0000 7f064dffb700 1 -- 172.21.15.46:0/174175027 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064800f8e0 con 0x556d2b6cb570 2024-03-18T20:46:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.394+0000 7f0669eb0b80 1 -- 172.21.15.46:0/174175027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6cb570 msgr2=0x556d2b6cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.394+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/174175027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6cb570 0x556d2b6cb950 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f0648009990 tx=0x7f0648020000 comp rx=0 tx=0).stop 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.394+0000 7f0669eb0b80 1 -- 172.21.15.46:0/174175027 shutdown_connections 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.394+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/174175027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6cb570 0x556d2b6cb950 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.394+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/174175027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6d6db0 0x556d2b6d91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.394+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/174175027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cbe90 0x556d2b6d6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.394+0000 7f0669eb0b80 1 -- 172.21.15.46:0/174175027 >> 172.21.15.46:0/174175027 conn(0x556d2b6421c0 msgr2=0x556d2b6c7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.394+0000 7f0669eb0b80 1 -- 172.21.15.46:0/174175027 shutdown_connections 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.394+0000 7f0669eb0b80 1 -- 172.21.15.46:0/174175027 wait complete. 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.394+0000 7f0669eb0b80 1 -- start start 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.394+0000 7f0669eb0b80 1 Processor -- start 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0669eb0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6cb570 0x556d2b6d67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0669eb0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cbe90 0x556d2b6d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0669eb0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6d6db0 0x556d2b6d41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0669eb0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2b5bad20 con 0x556d2b6cb570 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0669eb0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2b5ba9c0 con 0x556d2b6d6db0 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0669eb0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2b4a2640 con 0x556d2b6cbe90 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0669eb0b80 10 client.0 _ll_register_callbacks cb 0x556d2b426340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0657fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6d6db0 0x556d2b6d41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0657fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6d6db0 0x556d2b6d41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52776/0 (socket says 172.21.15.46:52776) 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0657fff700 1 -- 172.21.15.46:0/3519850889 learned_addr learned my addr 172.21.15.46:0/3519850889 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f064ffff700 1 --2- 172.21.15.46:0/3519850889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cbe90 0x556d2b6d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0657fff700 1 -- 172.21.15.46:0/3519850889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cbe90 msgr2=0x556d2b6d3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0657fff700 1 --2- 172.21.15.46:0/3519850889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cbe90 0x556d2b6d3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0657fff700 1 -- 172.21.15.46:0/3519850889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6cb570 msgr2=0x556d2b6d67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0657fff700 1 --2- 172.21.15.46:0/3519850889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6cb570 0x556d2b6d67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.395+0000 7f0657fff700 1 -- 172.21.15.46:0/3519850889 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06400098e0 con 0x556d2b6d6db0 2024-03-18T20:46:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0657fff700 1 --2- 172.21.15.46:0/3519850889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6d6db0 0x556d2b6d41a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f06500090a0 tx=0x7f0650099380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a518aaa95ac3c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0636ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0636ffd700 1 -- 172.21.15.46:0/3519850889 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065009a070 con 0x556d2b6d6db0 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0636ffd700 1 -- 172.21.15.46:0/3519850889 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f065000ce60 con 0x556d2b6d6db0 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0669eb0b80 1 -- 172.21.15.46:0/3519850889 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d2b6bf6c0 con 0x556d2b6d6db0 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0636ffd700 1 -- 172.21.15.46:0/3519850889 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0650099970 con 0x556d2b6d6db0 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0669eb0b80 10 client.28868 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0669eb0b80 1 -- 172.21.15.46:0/3519850889 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f065810d2f0 con 0x556d2b6d6db0 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0636ffd700 1 -- 172.21.15.46:0/3519850889 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0650063020 con 0x556d2b6d6db0 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0636ffd700 1 -- 172.21.15.46:0/3519850889 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0650098fb0 con 0x556d2b6d6db0 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0669eb0b80 10 client.28868 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0669eb0b80 10 client.28868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0669eb0b80 20 client.28868 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/3519850889 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556d2b6e6830 0x556d2b6e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0669eb0b80 4 client.28868 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f0669eb0b80 1 -- 172.21.15.46:0/3519850889 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d2b6176a0 con 0x556d2b6e6830 2024-03-18T20:46:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.396+0000 7f06577fe700 1 --2- 172.21.15.46:0/3519850889 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556d2b6e6830 0x556d2b6e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.397+0000 7f06577fe700 1 --2- 172.21.15.46:0/3519850889 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556d2b6e6830 0x556d2b6e8c50 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da0dcde6185de84a server_cookie=2aa992781af86dbf in_seq=0 out_seq=0 2024-03-18T20:46:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.397+0000 7f0636ffd700 10 client.28868 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.397+0000 7f0636ffd700 1 -- 172.21.15.46:0/3519850889 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556d2b6176a0 con 0x556d2b6e6830 2024-03-18T20:46:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.397+0000 7f0636ffd700 10 client.28868 handle_command_reply: tid=0 2024-03-18T20:46:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.397+0000 7f0669eb0b80 1 client.28868 shutdown 2024-03-18T20:46:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.398+0000 7f0669eb0b80 1 -- 172.21.15.46:0/3519850889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6d6db0 msgr2=0x556d2b6d41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.398+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/3519850889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6d6db0 0x556d2b6d41a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f06500090a0 tx=0x7f0650099380 comp rx=0 tx=0).stop 2024-03-18T20:46:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.398+0000 7f0669eb0b80 1 -- 172.21.15.46:0/3519850889 shutdown_connections 2024-03-18T20:46:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.398+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/3519850889 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556d2b6e6830 0x556d2b6e8c50 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.398+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/3519850889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6cb570 0x556d2b6d67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.398+0000 7f0636ffd700 0 client.28868 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.398+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/3519850889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6d6db0 0x556d2b6d41a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.398+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/3519850889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cbe90 0x556d2b6d3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.398+0000 7f0669eb0b80 1 -- 172.21.15.46:0/3519850889 >> 172.21.15.46:0/3519850889 conn(0x556d2b6421c0 msgr2=0x556d2b6c7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.398+0000 7f0669eb0b80 1 -- 172.21.15.46:0/3519850889 shutdown_connections 2024-03-18T20:46:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.398+0000 7f0669eb0b80 1 -- 172.21.15.46:0/3519850889 wait complete. 2024-03-18T20:46:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.398+0000 7f0669eb0b80 20 client.28868 trim_cache size 0 max 16384 2024-03-18T20:46:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.414+0000 7f0669eb0b80 1 Processor -- start 2024-03-18T20:46:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.414+0000 7f0669eb0b80 1 -- start start 2024-03-18T20:46:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.414+0000 7f0669eb0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cb570 0x556d2b6d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.414+0000 7f0669eb0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cbe90 0x556d2b6d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.414+0000 7f0669eb0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 0x556d2b6d6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.414+0000 7f0669eb0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2b6cc850 con 0x556d2b6d6db0 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.414+0000 7f0669eb0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2b6c1c10 con 0x556d2b6cb570 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.414+0000 7f0669eb0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2b6d4a40 con 0x556d2b6cbe90 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f0657fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 0x556d2b6d6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f06577fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cb570 0x556d2b6d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f0657fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 0x556d2b6d6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43580/0 (socket says 172.21.15.46:43580) 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f0657fff700 1 -- 172.21.15.46:0/4234051748 learned_addr learned my addr 172.21.15.46:0/4234051748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f064ffff700 1 --2- 172.21.15.46:0/4234051748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cbe90 0x556d2b6d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f0657fff700 1 -- 172.21.15.46:0/4234051748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cbe90 msgr2=0x556d2b6d4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f0657fff700 1 --2- 172.21.15.46:0/4234051748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cbe90 0x556d2b6d4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f0657fff700 1 -- 172.21.15.46:0/4234051748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cb570 msgr2=0x556d2b6d91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f0657fff700 1 --2- 172.21.15.46:0/4234051748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cb570 0x556d2b6d91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f0657fff700 1 -- 172.21.15.46:0/4234051748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d2b44cb00 con 0x556d2b6d6db0 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f0657fff700 1 --2- 172.21.15.46:0/4234051748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 0x556d2b6d6310 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0650097cc0 tx=0x7f06500092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54926804dd8dfdcb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f064e7fc700 1 -- 172.21.15.46:0/4234051748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06500a1b60 con 0x556d2b6d6db0 2024-03-18T20:46:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f064e7fc700 1 -- 172.21.15.46:0/4234051748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0650063030 con 0x556d2b6d6db0 2024-03-18T20:46:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.415+0000 7f064e7fc700 1 -- 172.21.15.46:0/4234051748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065009d7a0 con 0x556d2b6d6db0 2024-03-18T20:46:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.416+0000 7f0669eb0b80 1 -- 172.21.15.46:0/4234051748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 msgr2=0x556d2b6d6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.416+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/4234051748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 0x556d2b6d6310 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0650097cc0 tx=0x7f06500092e0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.416+0000 7f0669eb0b80 1 -- 172.21.15.46:0/4234051748 shutdown_connections 2024-03-18T20:46:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.416+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/4234051748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 0x556d2b6d6310 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.416+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/4234051748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cb570 0x556d2b6d91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.416+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/4234051748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cbe90 0x556d2b6d4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.416+0000 7f0669eb0b80 1 -- 172.21.15.46:0/4234051748 >> 172.21.15.46:0/4234051748 conn(0x556d2b6421c0 msgr2=0x556d2b6ed5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.416+0000 7f0669eb0b80 1 -- 172.21.15.46:0/4234051748 shutdown_connections 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.416+0000 7f0669eb0b80 1 -- 172.21.15.46:0/4234051748 wait complete. 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f0669eb0b80 1 -- start start 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f0669eb0b80 1 Processor -- start 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f0669eb0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cb570 0x556d2b6dbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f0669eb0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cbe90 0x556d2b6dc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f0669eb0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 0x556d2b73da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f0669eb0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2b6dc7b0 con 0x556d2b6d6db0 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f0669eb0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2b73dfb0 con 0x556d2b6cbe90 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f0669eb0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2b73e0f0 con 0x556d2b6cb570 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f0669eb0b80 10 client.0 _ll_register_callbacks cb 0x7f0650099810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f0657fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 0x556d2b73da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f0657fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 0x556d2b73da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43590/0 (socket says 172.21.15.46:43590) 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f0657fff700 1 -- 172.21.15.46:0/2465952204 learned_addr learned my addr 172.21.15.46:0/2465952204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f064ffff700 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cbe90 0x556d2b6dc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.417+0000 7f06577fe700 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cb570 0x556d2b6dbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0657fff700 1 -- 172.21.15.46:0/2465952204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cb570 msgr2=0x556d2b6dbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0657fff700 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cb570 0x556d2b6dbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0657fff700 1 -- 172.21.15.46:0/2465952204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cbe90 msgr2=0x556d2b6dc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0657fff700 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cbe90 0x556d2b6dc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0657fff700 1 -- 172.21.15.46:0/2465952204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065810d2f0 con 0x556d2b6d6db0 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f064ffff700 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cbe90 0x556d2b6dc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0657fff700 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 0x556d2b73da70 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f06500090a0 tx=0x7f065009a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3098285de8a47a51 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0636ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0636ffd700 1 -- 172.21.15.46:0/2465952204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065000ce60 con 0x556d2b6d6db0 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0636ffd700 1 -- 172.21.15.46:0/2465952204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f0650064080 con 0x556d2b6d6db0 2024-03-18T20:46:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0669eb0b80 1 -- 172.21.15.46:0/2465952204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f064801e040 con 0x556d2b6d6db0 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0636ffd700 1 -- 172.21.15.46:0/2465952204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06500a1b60 con 0x556d2b6d6db0 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0669eb0b80 10 client.19242 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.418+0000 7f0669eb0b80 1 -- 172.21.15.46:0/2465952204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x556d2b5e4e40 con 0x556d2b6d6db0 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.419+0000 7f0636ffd700 1 -- 172.21.15.46:0/2465952204 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f065007e020 con 0x556d2b6d6db0 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.419+0000 7f0636ffd700 1 -- 172.21.15.46:0/2465952204 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f06500988d0 con 0x556d2b6d6db0 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.419+0000 7f0669eb0b80 10 client.19242 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.419+0000 7f0669eb0b80 10 client.19242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.419+0000 7f0669eb0b80 20 client.19242 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.419+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556d2b73f4b0 0x556d2b7418a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.419+0000 7f0669eb0b80 4 client.19242 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.419+0000 7f0669eb0b80 1 -- 172.21.15.46:0/2465952204 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556d2b46dc90 con 0x556d2b73f4b0 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.419+0000 7f06577fe700 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556d2b73f4b0 0x556d2b7418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.419+0000 7f0636ffd700 10 client.19242 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.419+0000 7f06577fe700 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556d2b73f4b0 0x556d2b7418a0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec6a7f0cd5da25b server_cookie=8ce1da734c145a4b in_seq=0 out_seq=0 2024-03-18T20:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.449+0000 7f0636ffd700 1 -- 172.21.15.46:0/2465952204 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556d2b46dc90 con 0x556d2b73f4b0 2024-03-18T20:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.449+0000 7f0636ffd700 10 client.19242 handle_command_reply: tid=0 2024-03-18T20:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.449+0000 7f0669eb0b80 1 client.19242 shutdown 2024-03-18T20:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f0669eb0b80 1 -- 172.21.15.46:0/2465952204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 msgr2=0x556d2b73da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 0x556d2b73da70 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f06500090a0 tx=0x7f065009a070 comp rx=0 tx=0).stop 2024-03-18T20:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f0669eb0b80 1 -- 172.21.15.46:0/2465952204 shutdown_connections 2024-03-18T20:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556d2b73f4b0 0x556d2b7418a0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d2b6d6db0 0x556d2b73da70 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556d2b6cb570 0x556d2b6dbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f0636ffd700 0 client.19242 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f0669eb0b80 1 --2- 172.21.15.46:0/2465952204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556d2b6cbe90 0x556d2b6dc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f0669eb0b80 1 -- 172.21.15.46:0/2465952204 >> 172.21.15.46:0/2465952204 conn(0x556d2b6421c0 msgr2=0x556d2b6e2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f0669eb0b80 1 -- 172.21.15.46:0/2465952204 shutdown_connections 2024-03-18T20:46:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f0669eb0b80 1 -- 172.21.15.46:0/2465952204 wait complete. 2024-03-18T20:46:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f0669eb0b80 20 client.19242 trim_cache size 0 max 16384 2024-03-18T20:46:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:46:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:46:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:46:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:46:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f065dd1b700 1 -- 172.21.15.46:0/1222559068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f06440a34a0 msgr2=0x7f06440a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f065dd1b700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f06440a34a0 0x7f06440a5960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f064800b2e0 tx=0x7f0648005960 comp rx=0 tx=0).stop 2024-03-18T20:46:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.450+0000 7f065dd1b700 1 -- 172.21.15.46:0/1222559068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0658105c60 msgr2=0x7f0658199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.451+0000 7f065dd1b700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0658105c60 0x7f0658199770 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f065000ef30 tx=0x7f065000c3a0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.451+0000 7f065dd1b700 1 -- 172.21.15.46:0/1222559068 shutdown_connections 2024-03-18T20:46:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.451+0000 7f065dd1b700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f06440a34a0 0x7f06440a5960 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.451+0000 7f065dd1b700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0658105c60 0x7f0658199770 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.451+0000 7f065dd1b700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06580ffbb0 0x7f0658194f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.451+0000 7f065dd1b700 1 --2- 172.21.15.46:0/1222559068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0658100570 0x7f0658195440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.451+0000 7f065dd1b700 1 -- 172.21.15.46:0/1222559068 >> 172.21.15.46:0/1222559068 conn(0x7f0658076d50 msgr2=0x7f06580fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.451+0000 7f065dd1b700 1 -- 172.21.15.46:0/1222559068 shutdown_connections 2024-03-18T20:46:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:30.451+0000 7f065dd1b700 1 -- 172.21.15.46:0/1222559068 wait complete. 2024-03-18T20:46:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:30 smithi155 ceph-mon[48344]: pgmap v323: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-03-18T20:46:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:30 smithi046 ceph-mon[43495]: pgmap v323: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-03-18T20:46:30.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 385536 3.67 MB/sec execute 445 sec latency 3062.093 ms 2024-03-18T20:46:30.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:30 smithi073 ceph-mon[47384]: pgmap v323: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-03-18T20:46:31.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 387119 3.69 MB/sec execute 446 sec latency 16.830 ms 2024-03-18T20:46:32.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:32 smithi155 ceph-mon[48344]: pgmap v324: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-03-18T20:46:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:32 smithi046 ceph-mon[43495]: pgmap v324: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-03-18T20:46:32.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 388368 3.69 MB/sec execute 447 sec latency 100.622 ms 2024-03-18T20:46:32.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:32 smithi073 ceph-mon[47384]: pgmap v324: 97 pgs: 97 active+clean; 150 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2024-03-18T20:46:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:33 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:46:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:33 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:46:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:33 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:46:33.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:33 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:46:33.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:33 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:46:33.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:33 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:46:33.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 388368 3.68 MB/sec execute 448 sec latency 1100.725 ms 2024-03-18T20:46:33.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:33 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:46:33.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:33 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:46:33.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:33 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:46:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:34 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:46:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:34 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:46:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:34 smithi155 ceph-mon[48344]: pgmap v325: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:46:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:34 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:46:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:34 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:46:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:34 smithi046 ceph-mon[43495]: pgmap v325: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:46:34.714 INFO:tasks.workunit.client.0.smithi046.stdout: 1 389435 3.69 MB/sec execute 449 sec latency 1567.947 ms 2024-03-18T20:46:34.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:34 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:46:34.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:34 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:46:34.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:34 smithi073 ceph-mon[47384]: pgmap v325: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:46:35.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 390760 3.70 MB/sec execute 450 sec latency 19.827 ms 2024-03-18T20:46:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:35 smithi155 ceph-mon[48344]: pgmap v326: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-03-18T20:46:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:35 smithi046 ceph-mon[43495]: pgmap v326: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-03-18T20:46:36.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:35 smithi073 ceph-mon[47384]: pgmap v326: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-03-18T20:46:36.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 391911 3.70 MB/sec execute 451 sec latency 350.728 ms 2024-03-18T20:46:37.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 391911 3.69 MB/sec execute 452 sec latency 1350.832 ms 2024-03-18T20:46:38.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:38 smithi155 ceph-mon[48344]: pgmap v327: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-03-18T20:46:38.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:38 smithi046 ceph-mon[43495]: pgmap v327: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-03-18T20:46:38.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 391911 3.68 MB/sec execute 453 sec latency 2350.942 ms 2024-03-18T20:46:38.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:38 smithi073 ceph-mon[47384]: pgmap v327: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-03-18T20:46:39.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 391911 3.68 MB/sec execute 454 sec latency 3351.046 ms 2024-03-18T20:46:40.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:40 smithi155 ceph-mon[48344]: pgmap v328: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:46:40.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:40 smithi046 ceph-mon[43495]: pgmap v328: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:46:40.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 393852 3.69 MB/sec execute 455 sec latency 3496.980 ms 2024-03-18T20:46:40.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:40 smithi073 ceph-mon[47384]: pgmap v328: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.8 MiB/s wr, 185 op/s 2024-03-18T20:46:41.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 395152 3.70 MB/sec execute 456 sec latency 16.436 ms 2024-03-18T20:46:42.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:42 smithi155 ceph-mon[48344]: pgmap v329: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-03-18T20:46:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:42 smithi046 ceph-mon[43495]: pgmap v329: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-03-18T20:46:42.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 395459 3.69 MB/sec execute 457 sec latency 796.791 ms 2024-03-18T20:46:42.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:42 smithi073 ceph-mon[47384]: pgmap v329: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-03-18T20:46:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.579+0000 7f1b92b47700 1 -- 172.21.15.46:0/3723512226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c1027a0 msgr2=0x7f1b8c10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.579+0000 7f1b92b47700 1 --2- 172.21.15.46:0/3723512226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c1027a0 0x7f1b8c10ac80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009990 tx=0x7f1b78009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.579+0000 7f1b92b47700 1 -- 172.21.15.46:0/3723512226 shutdown_connections 2024-03-18T20:46:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.579+0000 7f1b92b47700 1 --2- 172.21.15.46:0/3723512226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b8c10b2a0 0x7f1b8c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.579+0000 7f1b92b47700 1 --2- 172.21.15.46:0/3723512226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b8c101e80 0x7f1b8c102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.579+0000 7f1b92b47700 1 --2- 172.21.15.46:0/3723512226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c1027a0 0x7f1b8c10ac80 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.579+0000 7f1b92b47700 1 -- 172.21.15.46:0/3723512226 >> 172.21.15.46:0/3723512226 conn(0x7f1b8c0fb680 msgr2=0x7f1b8c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.579+0000 7f1b92b47700 1 -- 172.21.15.46:0/3723512226 shutdown_connections 2024-03-18T20:46:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.579+0000 7f1b92b47700 1 -- 172.21.15.46:0/3723512226 wait complete. 2024-03-18T20:46:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.579+0000 7f1b92b47700 1 Processor -- start 2024-03-18T20:46:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.579+0000 7f1b92b47700 1 -- start start 2024-03-18T20:46:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b92b47700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c101e80 0x7f1b8c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b92b47700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b8c1027a0 0x7f1b8c1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b92b47700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b8c10b2a0 0x7f1b8c19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b92b47700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c19c080 con 0x7f1b8c101e80 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b92b47700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c19c1f0 con 0x7f1b8c1027a0 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b92b47700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c19c3b0 con 0x7f1b8c10b2a0 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b908e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c101e80 0x7f1b8c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b908e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c101e80 0x7f1b8c197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58524/0 (socket says 172.21.15.46:58524) 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b908e3700 1 -- 172.21.15.46:0/4058305032 learned_addr learned my addr 172.21.15.46:0/4058305032 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b8bfff700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b8c1027a0 0x7f1b8c1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b910e4700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b8c10b2a0 0x7f1b8c19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b908e3700 1 -- 172.21.15.46:0/4058305032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b8c10b2a0 msgr2=0x7f1b8c19ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b908e3700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b8c10b2a0 0x7f1b8c19ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b908e3700 1 -- 172.21.15.46:0/4058305032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b8c1027a0 msgr2=0x7f1b8c1976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b908e3700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b8c1027a0 0x7f1b8c1976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b908e3700 1 -- 172.21.15.46:0/4058305032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b780095f0 con 0x7f1b8c101e80 2024-03-18T20:46:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b908e3700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c101e80 0x7f1b8c197160 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000e9a0 tx=0x7f1b8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.580+0000 7f1b89ffb700 1 -- 172.21.15.46:0/4058305032 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80009790 con 0x7f1b8c101e80 2024-03-18T20:46:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.581+0000 7f1b89ffb700 1 -- 172.21.15.46:0/4058305032 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b8000cbd0 con 0x7f1b8c101e80 2024-03-18T20:46:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.581+0000 7f1b89ffb700 1 -- 172.21.15.46:0/4058305032 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80018680 con 0x7f1b8c101e80 2024-03-18T20:46:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.581+0000 7f1b92b47700 1 -- 172.21.15.46:0/4058305032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8c19c690 con 0x7f1b8c101e80 2024-03-18T20:46:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.581+0000 7f1b92b47700 1 -- 172.21.15.46:0/4058305032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8c1a3ed0 con 0x7f1b8c101e80 2024-03-18T20:46:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.583+0000 7f1b89ffb700 1 -- 172.21.15.46:0/4058305032 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1b8000cd40 con 0x7f1b8c101e80 2024-03-18T20:46:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.584+0000 7f1b89ffb700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1b7c0a1310 0x7f1b7c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.584+0000 7f1b89ffb700 1 -- 172.21.15.46:0/4058305032 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f1b80014070 con 0x7f1b8c101e80 2024-03-18T20:46:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.584+0000 7f1b8bfff700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1b7c0a1310 0x7f1b7c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.584+0000 7f1b9ecdcb80 1 Processor -- start 2024-03-18T20:46:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b8bfff700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1b7c0a1310 0x7f1b7c0a37d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800b2e0 tx=0x7f1b78005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b9ecdcb80 1 -- start start 2024-03-18T20:46:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b9ecdcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7570 0x55d7188e7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b9ecdcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7e90 0x55d7188f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b9ecdcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188f71c0 0x55d7188f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b9ecdcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d718841930 con 0x55d7188e7e90 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b9ecdcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71883f2f0 con 0x55d7188f71c0 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b9ecdcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71883abe0 con 0x55d7188e7570 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7e90 0x55d7188f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7e90 0x55d7188f6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58538/0 (socket says 172.21.15.46:58538) 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b8bfff700 1 -- 172.21.15.46:0/1983830268 learned_addr learned my addr 172.21.15.46:0/1983830268 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b908e3700 1 --2- 172.21.15.46:0/1983830268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7570 0x55d7188e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b910e4700 1 --2- 172.21.15.46:0/1983830268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188f71c0 0x55d7188f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b8bfff700 1 -- 172.21.15.46:0/1983830268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7570 msgr2=0x55d7188e7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b8bfff700 1 --2- 172.21.15.46:0/1983830268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7570 0x55d7188e7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b8bfff700 1 -- 172.21.15.46:0/1983830268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188f71c0 msgr2=0x55d7188f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b8bfff700 1 --2- 172.21.15.46:0/1983830268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188f71c0 0x55d7188f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b8bfff700 1 -- 172.21.15.46:0/1983830268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7186bc350 con 0x55d7188e7e90 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.585+0000 7f1b910e4700 1 --2- 172.21.15.46:0/1983830268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188f71c0 0x55d7188f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.586+0000 7f1b8bfff700 1 --2- 172.21.15.46:0/1983830268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7e90 0x55d7188f6c80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f1b78015000 tx=0x7f1b78020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1cff322afb1a3d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.586+0000 7f1b76ffd700 1 -- 172.21.15.46:0/1983830268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800bec0 con 0x55d7188e7e90 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.586+0000 7f1b76ffd700 1 -- 172.21.15.46:0/1983830268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b7801e040 con 0x55d7188e7e90 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.586+0000 7f1b76ffd700 1 -- 172.21.15.46:0/1983830268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800f860 con 0x55d7188e7e90 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.587+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/1983830268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7e90 msgr2=0x55d7188f6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.587+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/1983830268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7e90 0x55d7188f6c80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f1b78015000 tx=0x7f1b78020000 comp rx=0 tx=0).stop 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.587+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/1983830268 shutdown_connections 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.587+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/1983830268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188f71c0 0x55d7188f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.587+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/1983830268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7570 0x55d7188e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.587+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/1983830268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7e90 0x55d7188f6c80 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.587+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/1983830268 >> 172.21.15.46:0/1983830268 conn(0x55d71885e1c0 msgr2=0x55d7188e3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.587+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/1983830268 shutdown_connections 2024-03-18T20:46:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.587+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/1983830268 wait complete. 2024-03-18T20:46:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.587+0000 7f1b9ecdcb80 1 -- start start 2024-03-18T20:46:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.587+0000 7f1b9ecdcb80 1 Processor -- start 2024-03-18T20:46:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b9ecdcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7570 0x55d7188f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b9ecdcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7e90 0x55d7188f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b9ecdcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 0x55d7188f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b9ecdcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7187d6d20 con 0x55d7188f71c0 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b9ecdcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7187d69c0 con 0x55d7188e7e90 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b9ecdcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7186be640 con 0x55d7188e7570 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b9ecdcb80 10 client.0 _ll_register_callbacks cb 0x55d718642340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b910e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 0x55d7188f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b910e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 0x55d7188f45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58546/0 (socket says 172.21.15.46:58546) 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b910e4700 1 -- 172.21.15.46:0/3217616171 learned_addr learned my addr 172.21.15.46:0/3217616171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b8bfff700 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7e90 0x55d7188f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b908e3700 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7570 0x55d7188f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b910e4700 1 -- 172.21.15.46:0/3217616171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7570 msgr2=0x55d7188f6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b910e4700 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7570 0x55d7188f6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b910e4700 1 -- 172.21.15.46:0/3217616171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7e90 msgr2=0x55d7188f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b910e4700 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7e90 0x55d7188f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b910e4700 1 -- 172.21.15.46:0/3217616171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8c10e420 con 0x55d7188f71c0 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b908e3700 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7570 0x55d7188f6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b910e4700 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 0x55d7188f45e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f1b84007bc0 tx=0x7f1b84007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da3bb691a3beb86 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.588+0000 7f1b6bfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b6bfff700 1 -- 172.21.15.46:0/3217616171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400e070 con 0x55d7188f71c0 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b6bfff700 1 -- 172.21.15.46:0/3217616171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f1b84012be0 con 0x55d7188f71c0 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3217616171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7188db6c0 con 0x55d7188f71c0 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b6bfff700 1 -- 172.21.15.46:0/3217616171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b84012da0 con 0x55d7188f71c0 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b9ecdcb80 10 client.19254 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3217616171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d718668b00 con 0x55d7188f71c0 2024-03-18T20:46:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b6bfff700 1 -- 172.21.15.46:0/3217616171 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7188db6c0 con 0x55d7188f71c0 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b6bfff700 1 -- 172.21.15.46:0/3217616171 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1b84014040 con 0x55d7188f71c0 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b9ecdcb80 10 client.19254 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b9ecdcb80 10 client.19254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b9ecdcb80 20 client.19254 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d718906eb0 0x55d7189092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b9ecdcb80 4 client.19254 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.589+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3217616171 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7188336a0 con 0x55d718906eb0 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.590+0000 7f1b908e3700 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d718906eb0 0x55d7189092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.590+0000 7f1b6bfff700 10 client.19254 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.590+0000 7f1b908e3700 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d718906eb0 0x55d7189092d0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b055f01772b62f5c server_cookie=af62d96b47d79cab in_seq=0 out_seq=0 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b6bfff700 1 -- 172.21.15.46:0/3217616171 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d7188336a0 con 0x55d718906eb0 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b6bfff700 10 client.19254 handle_command_reply: tid=0 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b9ecdcb80 1 client.19254 shutdown 2024-03-18T20:46:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3217616171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 msgr2=0x55d7188f45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 0x55d7188f45e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f1b84007bc0 tx=0x7f1b84007ed0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3217616171 shutdown_connections 2024-03-18T20:46:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d718906eb0 0x55d7189092d0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 0x55d7188f45e0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7570 0x55d7188f6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/3217616171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7e90 0x55d7188f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b6bfff700 0 client.19254 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3217616171 >> 172.21.15.46:0/3217616171 conn(0x55d71885e1c0 msgr2=0x55d7188e3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3217616171 shutdown_connections 2024-03-18T20:46:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3217616171 wait complete. 2024-03-18T20:46:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.591+0000 7f1b9ecdcb80 20 client.19254 trim_cache size 0 max 16384 2024-03-18T20:46:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.606+0000 7f1b9ecdcb80 1 Processor -- start 2024-03-18T20:46:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.606+0000 7f1b9ecdcb80 1 -- start start 2024-03-18T20:46:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b9ecdcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7570 0x55d7188f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b9ecdcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7e90 0x55d7188fdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b9ecdcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 0x55d7188e2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b9ecdcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7188f54c0 con 0x55d7188f71c0 2024-03-18T20:46:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b9ecdcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7188fe320 con 0x55d7188e7570 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b9ecdcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7188f23f0 con 0x55d7188e7e90 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b910e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 0x55d7188e2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b910e4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 0x55d7188e2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58560/0 (socket says 172.21.15.46:58560) 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b908e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7570 0x55d7188f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b910e4700 1 -- 172.21.15.46:0/2168807347 learned_addr learned my addr 172.21.15.46:0/2168807347 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b8bfff700 1 --2- 172.21.15.46:0/2168807347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7e90 0x55d7188fdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b910e4700 1 -- 172.21.15.46:0/2168807347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7e90 msgr2=0x55d7188fdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b910e4700 1 --2- 172.21.15.46:0/2168807347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7e90 0x55d7188fdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b910e4700 1 -- 172.21.15.46:0/2168807347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7570 msgr2=0x55d7188f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b910e4700 1 --2- 172.21.15.46:0/2168807347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7570 0x55d7188f94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b910e4700 1 -- 172.21.15.46:0/2168807347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d718800e40 con 0x55d7188f71c0 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b8bfff700 1 --2- 172.21.15.46:0/2168807347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7e90 0x55d7188fdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b908e3700 1 --2- 172.21.15.46:0/2168807347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7570 0x55d7188f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b910e4700 1 --2- 172.21.15.46:0/2168807347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 0x55d7188e2120 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009010 tx=0x7f1b84009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24843d1f398130b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b777fe700 1 -- 172.21.15.46:0/2168807347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400e040 con 0x55d7188f71c0 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b777fe700 1 -- 172.21.15.46:0/2168807347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b8400f040 con 0x55d7188f71c0 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.607+0000 7f1b777fe700 1 -- 172.21.15.46:0/2168807347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84012e70 con 0x55d7188f71c0 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.608+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/2168807347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 msgr2=0x55d7188e2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.608+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/2168807347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 0x55d7188e2120 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009010 tx=0x7f1b84009d80 comp rx=0 tx=0).stop 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.608+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/2168807347 shutdown_connections 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.608+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/2168807347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188f71c0 0x55d7188e2120 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.608+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/2168807347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7570 0x55d7188f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.608+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/2168807347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188e7e90 0x55d7188fdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.608+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/2168807347 >> 172.21.15.46:0/2168807347 conn(0x55d71885e1c0 msgr2=0x55d71890dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.608+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/2168807347 shutdown_connections 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.608+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/2168807347 wait complete. 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.608+0000 7f1b9ecdcb80 1 -- start start 2024-03-18T20:46:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.608+0000 7f1b9ecdcb80 1 Processor -- start 2024-03-18T20:46:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b9ecdcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7570 0x55d7188f9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b9ecdcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7e90 0x55d7188fa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b9ecdcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188f71c0 0x55d718959cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b9ecdcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7188faa30 con 0x55d7188e7570 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b9ecdcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7188faba0 con 0x55d7188e7e90 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b9ecdcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d71895a210 con 0x55d7188f71c0 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b9ecdcb80 10 client.0 _ll_register_callbacks cb 0x7f1b84007590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b908e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7570 0x55d7188f9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b908e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7570 0x55d7188f9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58576/0 (socket says 172.21.15.46:58576) 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b908e3700 1 -- 172.21.15.46:0/3549440456 learned_addr learned my addr 172.21.15.46:0/3549440456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b8bfff700 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7e90 0x55d7188fa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b910e4700 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188f71c0 0x55d718959cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b908e3700 1 -- 172.21.15.46:0/3549440456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188f71c0 msgr2=0x55d718959cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b908e3700 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188f71c0 0x55d718959cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b908e3700 1 -- 172.21.15.46:0/3549440456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7e90 msgr2=0x55d7188fa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:43.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b908e3700 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7e90 0x55d7188fa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.609+0000 7f1b908e3700 1 -- 172.21.15.46:0/3549440456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8c1a3ed0 con 0x55d7188e7570 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b908e3700 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7570 0x55d7188f9dc0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f1b800964f0 tx=0x7f1b800cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a313eb36d52195b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b6bfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b6bfff700 1 -- 172.21.15.46:0/3549440456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b800a2470 con 0x55d7188e7570 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b6bfff700 1 -- 172.21.15.46:0/3549440456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f1b800b6070 con 0x55d7188e7570 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3549440456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b7801e040 con 0x55d7188e7570 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b6bfff700 1 -- 172.21.15.46:0/3549440456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b8009a250 con 0x55d7188e7570 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b9ecdcb80 10 client.19260 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3549440456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d718668b00 con 0x55d7188e7570 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b6bfff700 1 -- 172.21.15.46:0/3549440456 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b80083080 con 0x55d7188e7570 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b6bfff700 1 -- 172.21.15.46:0/3549440456 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1b800a1fe0 con 0x55d7188e7570 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b9ecdcb80 10 client.19260 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b9ecdcb80 10 client.19260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b9ecdcb80 20 client.19260 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d71895b9d0 0x55d71895ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b9ecdcb80 4 client.19260 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.610+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3549440456 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d718689c90 con 0x55d71895b9d0 2024-03-18T20:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.611+0000 7f1b8bfff700 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d71895b9d0 0x55d71895ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.611+0000 7f1b6bfff700 10 client.19260 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.611+0000 7f1b8bfff700 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d71895b9d0 0x55d71895ddf0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60296e4cff6cf424 server_cookie=581b88be8ef0638a in_seq=0 out_seq=0 2024-03-18T20:46:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b6bfff700 1 -- 172.21.15.46:0/3549440456 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d718689c90 con 0x55d71895b9d0 2024-03-18T20:46:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b6bfff700 10 client.19260 handle_command_reply: tid=0 2024-03-18T20:46:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b9ecdcb80 1 client.19260 shutdown 2024-03-18T20:46:43.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3549440456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7570 msgr2=0x55d7188f9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7570 0x55d7188f9dc0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f1b800964f0 tx=0x7f1b800cc040 comp rx=0 tx=0).stop 2024-03-18T20:46:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3549440456 shutdown_connections 2024-03-18T20:46:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d71895b9d0 0x55d71895ddf0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d7188e7570 0x55d7188f9dc0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d7188f71c0 0x55d718959cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b9ecdcb80 1 --2- 172.21.15.46:0/3549440456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d7188e7e90 0x55d7188fa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3549440456 >> 172.21.15.46:0/3549440456 conn(0x55d71885e1c0 msgr2=0x55d7188ef180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b6bfff700 0 client.19260 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3549440456 shutdown_connections 2024-03-18T20:46:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b9ecdcb80 1 -- 172.21.15.46:0/3549440456 wait complete. 2024-03-18T20:46:43.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.639+0000 7f1b9ecdcb80 20 client.19260 trim_cache size 0 max 16384 2024-03-18T20:46:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:46:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:46:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:46:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:46:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.640+0000 7f1b92b47700 1 -- 172.21.15.46:0/4058305032 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1b7c0a1310 msgr2=0x7f1b7c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.640+0000 7f1b92b47700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1b7c0a1310 0x7f1b7c0a37d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800b2e0 tx=0x7f1b78005960 comp rx=0 tx=0).stop 2024-03-18T20:46:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.640+0000 7f1b92b47700 1 -- 172.21.15.46:0/4058305032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c101e80 msgr2=0x7f1b8c197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.640+0000 7f1b92b47700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c101e80 0x7f1b8c197160 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000e9a0 tx=0x7f1b8000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.640+0000 7f1b92b47700 1 -- 172.21.15.46:0/4058305032 shutdown_connections 2024-03-18T20:46:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.640+0000 7f1b92b47700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1b7c0a1310 0x7f1b7c0a37d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.640+0000 7f1b92b47700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b8c101e80 0x7f1b8c197160 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.640+0000 7f1b92b47700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b8c10b2a0 0x7f1b8c19ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.640+0000 7f1b92b47700 1 --2- 172.21.15.46:0/4058305032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b8c1027a0 0x7f1b8c1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.640+0000 7f1b92b47700 1 -- 172.21.15.46:0/4058305032 >> 172.21.15.46:0/4058305032 conn(0x7f1b8c0fb680 msgr2=0x7f1b8c0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.640+0000 7f1b92b47700 1 -- 172.21.15.46:0/4058305032 shutdown_connections 2024-03-18T20:46:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:43.641+0000 7f1b92b47700 1 -- 172.21.15.46:0/4058305032 wait complete. 2024-03-18T20:46:43.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 395459 3.69 MB/sec execute 458 sec latency 1796.901 ms 2024-03-18T20:46:44.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:44 smithi155 ceph-mon[48344]: pgmap v330: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-03-18T20:46:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:44 smithi046 ceph-mon[43495]: pgmap v330: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-03-18T20:46:44.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 395459 3.68 MB/sec execute 459 sec latency 2796.948 ms 2024-03-18T20:46:44.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:44 smithi073 ceph-mon[47384]: pgmap v330: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-03-18T20:46:45.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 396363 3.68 MB/sec execute 460 sec latency 3412.271 ms 2024-03-18T20:46:46.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:45 smithi155 ceph-mon[48344]: pgmap v331: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-03-18T20:46:46.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:45 smithi046 ceph-mon[43495]: pgmap v331: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-03-18T20:46:46.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:45 smithi073 ceph-mon[47384]: pgmap v331: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-03-18T20:46:46.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 397787 3.69 MB/sec execute 461 sec latency 15.316 ms 2024-03-18T20:46:47.715 INFO:tasks.workunit.client.0.smithi046.stdout: 1 399001 3.69 MB/sec execute 462 sec latency 55.811 ms 2024-03-18T20:46:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:48 smithi155 ceph-mon[48344]: pgmap v332: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-03-18T20:46:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:48 smithi046 ceph-mon[43495]: pgmap v332: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-03-18T20:46:48.716 INFO:tasks.workunit.client.0.smithi046.stdout: 1 399001 3.69 MB/sec execute 463 sec latency 1037.748 ms 2024-03-18T20:46:48.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:48 smithi073 ceph-mon[47384]: pgmap v332: 97 pgs: 97 active+clean; 156 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-03-18T20:46:49.716 INFO:tasks.workunit.client.0.smithi046.stdout: 1 399001 3.68 MB/sec execute 464 sec latency 2037.820 ms 2024-03-18T20:46:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:50 smithi155 ceph-mon[48344]: pgmap v333: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-03-18T20:46:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:50 smithi046 ceph-mon[43495]: pgmap v333: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-03-18T20:46:50.716 INFO:tasks.workunit.client.0.smithi046.stdout: 1 399905 3.68 MB/sec execute 465 sec latency 2657.218 ms 2024-03-18T20:46:50.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:50 smithi073 ceph-mon[47384]: pgmap v333: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-03-18T20:46:51.716 INFO:tasks.workunit.client.0.smithi046.stdout: 1 401392 3.69 MB/sec execute 466 sec latency 15.157 ms 2024-03-18T20:46:52.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:52 smithi155 ceph-mon[48344]: pgmap v334: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-03-18T20:46:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:52 smithi046 ceph-mon[43495]: pgmap v334: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-03-18T20:46:52.716 INFO:tasks.workunit.client.0.smithi046.stdout: 1 402551 3.70 MB/sec execute 467 sec latency 205.783 ms 2024-03-18T20:46:52.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:52 smithi073 ceph-mon[47384]: pgmap v334: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-03-18T20:46:53.716 INFO:tasks.workunit.client.0.smithi046.stdout: 1 402551 3.69 MB/sec execute 468 sec latency 1205.857 ms 2024-03-18T20:46:54.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:54 smithi155 ceph-mon[48344]: pgmap v335: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-03-18T20:46:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:54 smithi046 ceph-mon[43495]: pgmap v335: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-03-18T20:46:54.716 INFO:tasks.workunit.client.0.smithi046.stdout: 1 403781 3.70 MB/sec execute 469 sec latency 1671.996 ms 2024-03-18T20:46:54.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:54 smithi073 ceph-mon[47384]: pgmap v335: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-03-18T20:46:55.716 INFO:tasks.workunit.client.0.smithi046.stdout: 1 404993 3.70 MB/sec execute 470 sec latency 22.265 ms 2024-03-18T20:46:56.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:55 smithi155 ceph-mon[48344]: pgmap v336: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-18T20:46:56.295 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:55 smithi046 ceph-mon[43495]: pgmap v336: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-18T20:46:56.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:55 smithi073 ceph-mon[47384]: pgmap v336: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-18T20:46:56.716 INFO:tasks.workunit.client.0.smithi046.stdout: 1 406092 3.71 MB/sec execute 471 sec latency 394.920 ms 2024-03-18T20:46:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.768+0000 7f7c4db61700 1 -- 172.21.15.46:0/599659993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c481007d0 msgr2=0x7f7c48104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.768+0000 7f7c4db61700 1 --2- 172.21.15.46:0/599659993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c481007d0 0x7f7c48104810 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009990 tx=0x7f7c3c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.768+0000 7f7c4db61700 1 -- 172.21.15.46:0/599659993 shutdown_connections 2024-03-18T20:46:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.768+0000 7f7c4db61700 1 --2- 172.21.15.46:0/599659993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c48104f40 0x7f7c4810cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.768+0000 7f7c4db61700 1 --2- 172.21.15.46:0/599659993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c480ffe20 0x7f7c48100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.768+0000 7f7c4db61700 1 --2- 172.21.15.46:0/599659993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c481007d0 0x7f7c48104810 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.768+0000 7f7c4db61700 1 -- 172.21.15.46:0/599659993 >> 172.21.15.46:0/599659993 conn(0x7f7c480fb680 msgr2=0x7f7c480fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.768+0000 7f7c4db61700 1 -- 172.21.15.46:0/599659993 shutdown_connections 2024-03-18T20:46:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.768+0000 7f7c4db61700 1 -- 172.21.15.46:0/599659993 wait complete. 2024-03-18T20:46:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.769+0000 7f7c4db61700 1 Processor -- start 2024-03-18T20:46:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.769+0000 7f7c4db61700 1 -- start start 2024-03-18T20:46:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.769+0000 7f7c4db61700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c480ffe20 0x7f7c48197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.769+0000 7f7c4db61700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c48104f40 0x7f7c48198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.769+0000 7f7c4db61700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c48198a70 0x7f7c481a3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c4db61700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c481a3700 con 0x7f7c48198a70 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c4db61700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c481a3870 con 0x7f7c48104f40 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c4db61700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c481a3a30 con 0x7f7c480ffe20 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c47fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c48198a70 0x7f7c481a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c46ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c48104f40 0x7f7c48198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c47fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c48198a70 0x7f7c481a3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51654/0 (socket says 172.21.15.46:51654) 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c477fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c480ffe20 0x7f7c48197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c47fff700 1 -- 172.21.15.46:0/1518326196 learned_addr learned my addr 172.21.15.46:0/1518326196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c47fff700 1 -- 172.21.15.46:0/1518326196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c480ffe20 msgr2=0x7f7c48197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c47fff700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c480ffe20 0x7f7c48197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c47fff700 1 -- 172.21.15.46:0/1518326196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c48104f40 msgr2=0x7f7c48198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c47fff700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c48104f40 0x7f7c48198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c47fff700 1 -- 172.21.15.46:0/1518326196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3c0095f0 con 0x7f7c48198a70 2024-03-18T20:46:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c477fe700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c480ffe20 0x7f7c48197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c46ffd700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c48104f40 0x7f7c48198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c47fff700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c48198a70 0x7f7c481a3070 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000ef30 tx=0x7f7c4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c44ff9700 1 -- 172.21.15.46:0/1518326196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c40018070 con 0x7f7c48198a70 2024-03-18T20:46:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c44ff9700 1 -- 172.21.15.46:0/1518326196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c4000f040 con 0x7f7c48198a70 2024-03-18T20:46:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.770+0000 7f7c4db61700 1 -- 172.21.15.46:0/1518326196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c481a3cb0 con 0x7f7c48198a70 2024-03-18T20:46:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.771+0000 7f7c44ff9700 1 -- 172.21.15.46:0/1518326196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4000cd00 con 0x7f7c48198a70 2024-03-18T20:46:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.771+0000 7f7c4db61700 1 -- 172.21.15.46:0/1518326196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c481a4140 con 0x7f7c48198a70 2024-03-18T20:46:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.773+0000 7f7c44ff9700 1 -- 172.21.15.46:0/1518326196 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7c40003a00 con 0x7f7c48198a70 2024-03-18T20:46:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.774+0000 7f7c44ff9700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c300a1290 0x7f7c300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.774+0000 7f7c44ff9700 1 -- 172.21.15.46:0/1518326196 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7c400cc850 con 0x7f7c48198a70 2024-03-18T20:46:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.774+0000 7f7c477fe700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c300a1290 0x7f7c300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.774+0000 7f7c477fe700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c300a1290 0x7f7c300a3750 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7c38007fd0 tx=0x7f7c380073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.774+0000 7f7c59cf6b80 1 Processor -- start 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.774+0000 7f7c59cf6b80 1 -- start start 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c59cf6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 0x562ac8060950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c59cf6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060e90 0x562ac806b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c59cf6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac806bdb0 0x562ac806e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c59cf6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac7fba930 con 0x562ac8060570 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c59cf6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac7fb82f0 con 0x562ac806bdb0 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c59cf6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac7fb3be0 con 0x562ac8060e90 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 0x562ac8060950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 0x562ac8060950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51666/0 (socket says 172.21.15.46:51666) 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c477fe700 1 -- 172.21.15.46:0/463004789 learned_addr learned my addr 172.21.15.46:0/463004789 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c46ffd700 1 --2- 172.21.15.46:0/463004789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060e90 0x562ac806b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c47fff700 1 --2- 172.21.15.46:0/463004789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac806bdb0 0x562ac806e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c477fe700 1 -- 172.21.15.46:0/463004789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060e90 msgr2=0x562ac806b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c477fe700 1 --2- 172.21.15.46:0/463004789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060e90 0x562ac806b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c477fe700 1 -- 172.21.15.46:0/463004789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac806bdb0 msgr2=0x562ac806e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c477fe700 1 --2- 172.21.15.46:0/463004789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac806bdb0 0x562ac806e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c477fe700 1 -- 172.21.15.46:0/463004789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ac7e35350 con 0x562ac8060570 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c47fff700 1 --2- 172.21.15.46:0/463004789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac806bdb0 0x562ac806e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.775+0000 7f7c477fe700 1 --2- 172.21.15.46:0/463004789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 0x562ac8060950 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7c38004350 tx=0x7f7c38013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0aa4b4855c2f0ed server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.776+0000 7f7c2dffb700 1 -- 172.21.15.46:0/463004789 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38003980 con 0x562ac8060570 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.776+0000 7f7c2dffb700 1 -- 172.21.15.46:0/463004789 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c38011440 con 0x562ac8060570 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.776+0000 7f7c2dffb700 1 -- 172.21.15.46:0/463004789 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3801f690 con 0x562ac8060570 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.776+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/463004789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 msgr2=0x562ac8060950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.776+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/463004789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 0x562ac8060950 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7c38004350 tx=0x7f7c38013000 comp rx=0 tx=0).stop 2024-03-18T20:46:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.776+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/463004789 shutdown_connections 2024-03-18T20:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.776+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/463004789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 0x562ac8060950 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.776+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/463004789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac806bdb0 0x562ac806e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.776+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/463004789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060e90 0x562ac806b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.776+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/463004789 >> 172.21.15.46:0/463004789 conn(0x562ac7fd71c0 msgr2=0x562ac805ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.776+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/463004789 shutdown_connections 2024-03-18T20:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.777+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/463004789 wait complete. 2024-03-18T20:46:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.777+0000 7f7c59cf6b80 1 -- start start 2024-03-18T20:46:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.777+0000 7f7c59cf6b80 1 Processor -- start 2024-03-18T20:46:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.777+0000 7f7c59cf6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060570 0x562ac806b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.777+0000 7f7c59cf6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac8060e90 0x562ac8068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.777+0000 7f7c59cf6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac806bdb0 0x562ac80690f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.777+0000 7f7c59cf6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac7f4fd20 con 0x562ac806bdb0 2024-03-18T20:46:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.777+0000 7f7c59cf6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac7f4f9c0 con 0x562ac8060e90 2024-03-18T20:46:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.777+0000 7f7c59cf6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac7e37640 con 0x562ac8060570 2024-03-18T20:46:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.777+0000 7f7c59cf6b80 10 client.0 _ll_register_callbacks cb 0x562ac7dbb340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c47fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac806bdb0 0x562ac80690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c47fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac806bdb0 0x562ac80690f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51668/0 (socket says 172.21.15.46:51668) 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c47fff700 1 -- 172.21.15.46:0/31316880 learned_addr learned my addr 172.21.15.46:0/31316880 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c46ffd700 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac8060e90 0x562ac8068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c477fe700 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060570 0x562ac806b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c47fff700 1 -- 172.21.15.46:0/31316880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060570 msgr2=0x562ac806b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c47fff700 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060570 0x562ac806b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c47fff700 1 -- 172.21.15.46:0/31316880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac8060e90 msgr2=0x562ac8068ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c47fff700 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac8060e90 0x562ac8068ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c47fff700 1 -- 172.21.15.46:0/31316880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c38004030 con 0x562ac806bdb0 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c46ffd700 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac8060e90 0x562ac8068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c47fff700 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac806bdb0 0x562ac80690f0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f7c400963e0 tx=0x7f7c40004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593260be10129303 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c26ffd700 1 -- 172.21.15.46:0/31316880 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4000ce60 con 0x562ac806bdb0 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c26ffd700 1 -- 172.21.15.46:0/31316880 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7c40099970 con 0x562ac806bdb0 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/31316880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ac80546c0 con 0x562ac806bdb0 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c26ffd700 1 -- 172.21.15.46:0/31316880 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c4009d7a0 con 0x562ac806bdb0 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c59cf6b80 10 client.19272 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.778+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/31316880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x562ac7de1b00 con 0x562ac806bdb0 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.779+0000 7f7c26ffd700 1 -- 172.21.15.46:0/31316880 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ac80546c0 con 0x562ac806bdb0 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.779+0000 7f7c26ffd700 1 -- 172.21.15.46:0/31316880 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7c400a18d0 con 0x562ac806bdb0 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.779+0000 7f7c59cf6b80 10 client.19272 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.779+0000 7f7c59cf6b80 10 client.19272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.779+0000 7f7c59cf6b80 20 client.19272 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.779+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ac807ba10 0x562ac807de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.779+0000 7f7c59cf6b80 4 client.19272 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.779+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/31316880 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ac7fac6a0 con 0x562ac807ba10 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.779+0000 7f7c477fe700 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ac807ba10 0x562ac807de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.780+0000 7f7c26ffd700 10 client.19272 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.780+0000 7f7c477fe700 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ac807ba10 0x562ac807de30 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6327f7e89d6b50a9 server_cookie=6366d6125b46c568 in_seq=0 out_seq=0 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.780+0000 7f7c26ffd700 1 -- 172.21.15.46:0/31316880 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562ac7fac6a0 con 0x562ac807ba10 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.780+0000 7f7c26ffd700 10 client.19272 handle_command_reply: tid=0 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.780+0000 7f7c59cf6b80 1 client.19272 shutdown 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.781+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/31316880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac806bdb0 msgr2=0x562ac80690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.781+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac806bdb0 0x562ac80690f0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f7c400963e0 tx=0x7f7c40004b50 comp rx=0 tx=0).stop 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.781+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/31316880 shutdown_connections 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.781+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ac807ba10 0x562ac807de30 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.781+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac806bdb0 0x562ac80690f0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.781+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060570 0x562ac806b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.781+0000 7f7c26ffd700 0 client.19272 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.781+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/31316880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac8060e90 0x562ac8068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.781+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/31316880 >> 172.21.15.46:0/31316880 conn(0x562ac7fd71c0 msgr2=0x562ac805c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.781+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/31316880 shutdown_connections 2024-03-18T20:46:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.781+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/31316880 wait complete. 2024-03-18T20:46:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.781+0000 7f7c59cf6b80 20 client.19272 trim_cache size 0 max 16384 2024-03-18T20:46:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.796+0000 7f7c59cf6b80 1 Processor -- start 2024-03-18T20:46:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.796+0000 7f7c59cf6b80 1 -- start start 2024-03-18T20:46:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.796+0000 7f7c59cf6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac8060570 0x562ac805cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.796+0000 7f7c59cf6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060e90 0x562ac8072a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.796+0000 7f7c59cf6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac806bdb0 0x562ac80778a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.796+0000 7f7c59cf6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac805d780 con 0x562ac8060e90 2024-03-18T20:46:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.796+0000 7f7c59cf6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac805dd60 con 0x562ac8060570 2024-03-18T20:46:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.796+0000 7f7c59cf6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac80699d0 con 0x562ac806bdb0 2024-03-18T20:46:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.796+0000 7f7c46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060e90 0x562ac8072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c46ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060e90 0x562ac8072a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51678/0 (socket says 172.21.15.46:51678) 2024-03-18T20:46:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c46ffd700 1 -- 172.21.15.46:0/1388326652 learned_addr learned my addr 172.21.15.46:0/1388326652 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c47fff700 1 --2- 172.21.15.46:0/1388326652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac806bdb0 0x562ac80778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c477fe700 1 --2- 172.21.15.46:0/1388326652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac8060570 0x562ac805cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c46ffd700 1 -- 172.21.15.46:0/1388326652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac806bdb0 msgr2=0x562ac80778a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c46ffd700 1 --2- 172.21.15.46:0/1388326652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac806bdb0 0x562ac80778a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c46ffd700 1 -- 172.21.15.46:0/1388326652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac8060570 msgr2=0x562ac805cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c46ffd700 1 --2- 172.21.15.46:0/1388326652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac8060570 0x562ac805cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c46ffd700 1 -- 172.21.15.46:0/1388326652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ac7f79e40 con 0x562ac8060e90 2024-03-18T20:46:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c46ffd700 1 --2- 172.21.15.46:0/1388326652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060e90 0x562ac8072a30 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f7c48067d10 tx=0x7f7c3c005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d00d1002d34151 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c2e7fc700 1 -- 172.21.15.46:0/1388326652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c01d030 con 0x562ac8060e90 2024-03-18T20:46:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c2e7fc700 1 -- 172.21.15.46:0/1388326652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c3c01e040 con 0x562ac8060e90 2024-03-18T20:46:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.797+0000 7f7c2e7fc700 1 -- 172.21.15.46:0/1388326652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c0256c0 con 0x562ac8060e90 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.798+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/1388326652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060e90 msgr2=0x562ac8072a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.798+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/1388326652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060e90 0x562ac8072a30 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f7c48067d10 tx=0x7f7c3c005960 comp rx=0 tx=0).stop 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.798+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/1388326652 shutdown_connections 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.798+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/1388326652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac806bdb0 0x562ac80778a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.798+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/1388326652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac8060570 0x562ac805cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.798+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/1388326652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060e90 0x562ac8072a30 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.798+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/1388326652 >> 172.21.15.46:0/1388326652 conn(0x562ac7fd71c0 msgr2=0x562ac8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.798+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/1388326652 shutdown_connections 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.798+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/1388326652 wait complete. 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.798+0000 7f7c59cf6b80 1 -- start start 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.798+0000 7f7c59cf6b80 1 Processor -- start 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c59cf6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 0x562ac806ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c59cf6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060e90 0x562ac806f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c59cf6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac806bdb0 0x562ac80ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c59cf6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac80ced70 con 0x562ac8060570 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c59cf6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac80ceeb0 con 0x562ac806bdb0 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c59cf6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac80ceff0 con 0x562ac8060e90 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c59cf6b80 10 client.0 _ll_register_callbacks cb 0x7f7c400994f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 0x562ac806ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 0x562ac806ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51688/0 (socket says 172.21.15.46:51688) 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c477fe700 1 -- 172.21.15.46:0/802176590 learned_addr learned my addr 172.21.15.46:0/802176590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c46ffd700 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060e90 0x562ac806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c47fff700 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac806bdb0 0x562ac80ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c477fe700 1 -- 172.21.15.46:0/802176590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060e90 msgr2=0x562ac806f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c477fe700 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060e90 0x562ac806f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c477fe700 1 -- 172.21.15.46:0/802176590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac806bdb0 msgr2=0x562ac80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c477fe700 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac806bdb0 0x562ac80ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c477fe700 1 -- 172.21.15.46:0/802176590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4810d330 con 0x562ac8060570 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c477fe700 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 0x562ac806ec00 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f7c38022040 tx=0x7f7c38011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1366448ff0a5d72d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.799+0000 7f7c26ffd700 1 -- 172.21.15.46:0/802176590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38031330 con 0x562ac8060570 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c26ffd700 1 -- 172.21.15.46:0/802176590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7c38017070 con 0x562ac8060570 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/802176590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c38011440 con 0x562ac8060570 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c26ffd700 1 -- 172.21.15.46:0/802176590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c380118c0 con 0x562ac8060570 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c59cf6b80 10 client.19278 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/802176590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x562ac7e39c70 con 0x562ac8060570 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c26ffd700 1 -- 172.21.15.46:0/802176590 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c38011440 con 0x562ac8060570 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c26ffd700 1 -- 172.21.15.46:0/802176590 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7c38003780 con 0x562ac8060570 2024-03-18T20:46:56.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c59cf6b80 10 client.19278 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c59cf6b80 10 client.19278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c59cf6b80 20 client.19278 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ac80d07d0 0x562ac80d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c59cf6b80 4 client.19278 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:46:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.800+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/802176590 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ac7e02c90 con 0x562ac80d07d0 2024-03-18T20:46:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.801+0000 7f7c46ffd700 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ac80d07d0 0x562ac80d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.801+0000 7f7c46ffd700 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ac80d07d0 0x562ac80d2bf0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abfb3d8b4720a8c7 server_cookie=ddc4812a28e989c1 in_seq=0 out_seq=0 2024-03-18T20:46:56.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.801+0000 7f7c26ffd700 10 client.19278 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.826+0000 7f7c26ffd700 1 -- 172.21.15.46:0/802176590 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ac7e02c90 con 0x562ac80d07d0 2024-03-18T20:46:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.826+0000 7f7c26ffd700 10 client.19278 handle_command_reply: tid=0 2024-03-18T20:46:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.826+0000 7f7c59cf6b80 1 client.19278 shutdown 2024-03-18T20:46:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.827+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/802176590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 msgr2=0x562ac806ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.827+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 0x562ac806ec00 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f7c38022040 tx=0x7f7c38011d90 comp rx=0 tx=0).stop 2024-03-18T20:46:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.827+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/802176590 shutdown_connections 2024-03-18T20:46:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.827+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ac80d07d0 0x562ac80d2bf0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.827+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ac8060570 0x562ac806ec00 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.827+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ac806bdb0 0x562ac80ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.827+0000 7f7c59cf6b80 1 --2- 172.21.15.46:0/802176590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ac8060e90 0x562ac806f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.827+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/802176590 >> 172.21.15.46:0/802176590 conn(0x562ac7fd71c0 msgr2=0x562ac8077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.827+0000 7f7c26ffd700 0 client.19278 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.827+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/802176590 shutdown_connections 2024-03-18T20:46:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.827+0000 7f7c59cf6b80 1 -- 172.21.15.46:0/802176590 wait complete. 2024-03-18T20:46:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.827+0000 7f7c59cf6b80 20 client.19278 trim_cache size 0 max 16384 2024-03-18T20:46:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:46:56.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:46:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:46:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:46:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.828+0000 7f7c4db61700 1 -- 172.21.15.46:0/1518326196 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c300a1290 msgr2=0x7f7c300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.828+0000 7f7c4db61700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c300a1290 0x7f7c300a3750 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7c38007fd0 tx=0x7f7c380073d0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.828+0000 7f7c4db61700 1 -- 172.21.15.46:0/1518326196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c48198a70 msgr2=0x7f7c481a3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.828+0000 7f7c4db61700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c48198a70 0x7f7c481a3070 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000ef30 tx=0x7f7c4000c3a0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.828+0000 7f7c4db61700 1 -- 172.21.15.46:0/1518326196 shutdown_connections 2024-03-18T20:46:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.828+0000 7f7c4db61700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c300a1290 0x7f7c300a3750 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.828+0000 7f7c4db61700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c48198a70 0x7f7c481a3070 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.828+0000 7f7c4db61700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c480ffe20 0x7f7c48197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.828+0000 7f7c4db61700 1 --2- 172.21.15.46:0/1518326196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c48104f40 0x7f7c48198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.828+0000 7f7c4db61700 1 -- 172.21.15.46:0/1518326196 >> 172.21.15.46:0/1518326196 conn(0x7f7c480fb680 msgr2=0x7f7c480fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.828+0000 7f7c4db61700 1 -- 172.21.15.46:0/1518326196 shutdown_connections 2024-03-18T20:46:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:46:56.828+0000 7f7c4db61700 1 -- 172.21.15.46:0/1518326196 wait complete. 2024-03-18T20:46:57.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:46:57.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:46:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.301+0000 7fe3f6290700 1 -- 172.21.15.46:0/1557363424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0101900 msgr2=0x7fe3f0105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.301+0000 7fe3f6290700 1 --2- 172.21.15.46:0/1557363424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0101900 0x7fe3f0105940 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009990 tx=0x7fe3e4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.301+0000 7fe3f6290700 1 -- 172.21.15.46:0/1557363424 shutdown_connections 2024-03-18T20:46:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.301+0000 7fe3f6290700 1 --2- 172.21.15.46:0/1557363424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3f0106070 0x7fe3f010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.301+0000 7fe3f6290700 1 --2- 172.21.15.46:0/1557363424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0100f50 0x7fe3f0101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.301+0000 7fe3f6290700 1 --2- 172.21.15.46:0/1557363424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0101900 0x7fe3f0105940 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.301+0000 7fe3f6290700 1 -- 172.21.15.46:0/1557363424 >> 172.21.15.46:0/1557363424 conn(0x7fe3f00fc7f0 msgr2=0x7fe3f00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.302+0000 7fe3f6290700 1 -- 172.21.15.46:0/1557363424 shutdown_connections 2024-03-18T20:46:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.302+0000 7fe3f6290700 1 -- 172.21.15.46:0/1557363424 wait complete. 2024-03-18T20:46:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.302+0000 7fe3f6290700 1 Processor -- start 2024-03-18T20:46:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.302+0000 7fe3f6290700 1 -- start start 2024-03-18T20:46:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f6290700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0100f50 0x7fe3f0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f6290700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3f0106070 0x7fe3f0197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f6290700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f019bae0 0x7fe3f019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f6290700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f019c560 con 0x7fe3f019bae0 2024-03-18T20:46:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f6290700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f019c6d0 con 0x7fe3f0100f50 2024-03-18T20:46:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f6290700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f019c890 con 0x7fe3f0106070 2024-03-18T20:46:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f482d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f019bae0 0x7fe3f019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f482d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f019bae0 0x7fe3f019bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51712/0 (socket says 172.21.15.46:51712) 2024-03-18T20:46:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f482d700 1 -- 172.21.15.46:0/2647091875 learned_addr learned my addr 172.21.15.46:0/2647091875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3effff700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0100f50 0x7fe3f0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3ef7fe700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3f0106070 0x7fe3f0197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f482d700 1 -- 172.21.15.46:0/2647091875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3f0106070 msgr2=0x7fe3f0197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f482d700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3f0106070 0x7fe3f0197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f482d700 1 -- 172.21.15.46:0/2647091875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0100f50 msgr2=0x7fe3f0197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f482d700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0100f50 0x7fe3f0197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f482d700 1 -- 172.21.15.46:0/2647091875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e40095f0 con 0x7fe3f019bae0 2024-03-18T20:46:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3effff700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0100f50 0x7fe3f0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3f482d700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f019bae0 0x7fe3f019bf60 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8007ee0 tx=0x7fe3e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.303+0000 7fe3ed7fa700 1 -- 172.21.15.46:0/2647091875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8017070 con 0x7fe3f019bae0 2024-03-18T20:46:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.304+0000 7fe3ed7fa700 1 -- 172.21.15.46:0/2647091875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e800f040 con 0x7fe3f019bae0 2024-03-18T20:46:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.304+0000 7fe3f6290700 1 -- 172.21.15.46:0/2647091875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f01a3df0 con 0x7fe3f019bae0 2024-03-18T20:46:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.304+0000 7fe3ed7fa700 1 -- 172.21.15.46:0/2647091875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8005740 con 0x7fe3f019bae0 2024-03-18T20:46:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.304+0000 7fe3f6290700 1 -- 172.21.15.46:0/2647091875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f01a4280 con 0x7fe3f019bae0 2024-03-18T20:46:57.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.306+0000 7fe3ed7fa700 1 -- 172.21.15.46:0/2647091875 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe3e800dc60 con 0x7fe3f019bae0 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.307+0000 7fe3ed7fa700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe3d80a1290 0x7fe3d80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.307+0000 7fe3ed7fa700 1 -- 172.21.15.46:0/2647091875 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe3e80cc470 con 0x7fe3f019bae0 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.307+0000 7fe3effff700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe3d80a1290 0x7fe3d80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe402425b80 1 Processor -- start 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe3effff700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe3d80a1290 0x7fe3d80a3750 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0011490 tx=0x7fe3e0009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe402425b80 1 -- start start 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe402425b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588da60 0x55c3c588de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe402425b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588e380 0x55c3c5898ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe402425b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c589b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe402425b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c57e7990 con 0x55c3c5899220 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe402425b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c57e5350 con 0x55c3c588e380 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe402425b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c57e0c40 con 0x55c3c588da60 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe3f482d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c589b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe3f482d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c589b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51716/0 (socket says 172.21.15.46:51716) 2024-03-18T20:46:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe3f482d700 1 -- 172.21.15.46:0/2335801019 learned_addr learned my addr 172.21.15.46:0/2335801019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe3effff700 1 --2- 172.21.15.46:0/2335801019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588da60 0x55c3c588de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.308+0000 7fe3ef7fe700 1 --2- 172.21.15.46:0/2335801019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588e380 0x55c3c5898ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.309+0000 7fe3f482d700 1 -- 172.21.15.46:0/2335801019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588da60 msgr2=0x55c3c588de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.309+0000 7fe3f482d700 1 --2- 172.21.15.46:0/2335801019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588da60 0x55c3c588de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.309+0000 7fe3f482d700 1 -- 172.21.15.46:0/2335801019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588e380 msgr2=0x55c3c5898ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.309+0000 7fe3f482d700 1 --2- 172.21.15.46:0/2335801019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588e380 0x55c3c5898ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.309+0000 7fe3f482d700 1 -- 172.21.15.46:0/2335801019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3c56623b0 con 0x55c3c5899220 2024-03-18T20:46:57.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.309+0000 7fe3f482d700 1 --2- 172.21.15.46:0/2335801019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c589b610 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800d7b0 tx=0x7fe3e800df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c092707699e47bc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.309+0000 7fe3d67fc700 1 -- 172.21.15.46:0/2335801019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e809d170 con 0x55c3c5899220 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.309+0000 7fe3d67fc700 1 -- 172.21.15.46:0/2335801019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e8099560 con 0x55c3c5899220 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.309+0000 7fe3d67fc700 1 -- 172.21.15.46:0/2335801019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8022250 con 0x55c3c5899220 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.310+0000 7fe402425b80 1 -- 172.21.15.46:0/2335801019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 msgr2=0x55c3c589b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.310+0000 7fe402425b80 1 --2- 172.21.15.46:0/2335801019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c589b610 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800d7b0 tx=0x7fe3e800df60 comp rx=0 tx=0).stop 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.310+0000 7fe402425b80 1 -- 172.21.15.46:0/2335801019 shutdown_connections 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.310+0000 7fe402425b80 1 --2- 172.21.15.46:0/2335801019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c589b610 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.310+0000 7fe402425b80 1 --2- 172.21.15.46:0/2335801019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588da60 0x55c3c588de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.310+0000 7fe402425b80 1 --2- 172.21.15.46:0/2335801019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588e380 0x55c3c5898ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.310+0000 7fe402425b80 1 -- 172.21.15.46:0/2335801019 >> 172.21.15.46:0/2335801019 conn(0x55c3c5804220 msgr2=0x55c3c588a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.310+0000 7fe402425b80 1 -- 172.21.15.46:0/2335801019 shutdown_connections 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.310+0000 7fe402425b80 1 -- 172.21.15.46:0/2335801019 wait complete. 2024-03-18T20:46:57.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.310+0000 7fe402425b80 1 -- start start 2024-03-18T20:46:57.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.310+0000 7fe402425b80 1 Processor -- start 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe402425b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588da60 0x55c3c5898ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe402425b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c588e380 0x55c3c5895f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe402425b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c5899220 0x55c3c5896680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe402425b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c577cd80 con 0x55c3c588e380 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe402425b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c577ca20 con 0x55c3c588da60 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe402425b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c56646a0 con 0x55c3c5899220 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe402425b80 10 client.0 _ll_register_callbacks cb 0x55c3c55e83a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe3ef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c588e380 0x55c3c5895f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe3ef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c588e380 0x55c3c5895f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51732/0 (socket says 172.21.15.46:51732) 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe3ef7fe700 1 -- 172.21.15.46:0/452908812 learned_addr learned my addr 172.21.15.46:0/452908812 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe3effff700 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588da60 0x55c3c5898ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe3f482d700 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c5899220 0x55c3c5896680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe3ef7fe700 1 -- 172.21.15.46:0/452908812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c5899220 msgr2=0x55c3c5896680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe3ef7fe700 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c5899220 0x55c3c5896680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe3ef7fe700 1 -- 172.21.15.46:0/452908812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588da60 msgr2=0x55c3c5898ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe3ef7fe700 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588da60 0x55c3c5898ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe3ef7fe700 1 -- 172.21.15.46:0/452908812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f01a4280 con 0x55c3c588e380 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe3effff700 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588da60 0x55c3c5898ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.311+0000 7fe3ef7fe700 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c588e380 0x55c3c5895f90 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00679c0 tx=0x7fe3e4005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed5e12bbc0db089 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe3cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe3cf7fe700 1 -- 172.21.15.46:0/452908812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400bee0 con 0x55c3c588e380 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe3cf7fe700 1 -- 172.21.15.46:0/452908812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe3e4004500 con 0x55c3c588e380 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe402425b80 1 -- 172.21.15.46:0/452908812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3c5881ad0 con 0x55c3c588e380 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe3cf7fe700 1 -- 172.21.15.46:0/452908812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3e4004700 con 0x55c3c588e380 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe402425b80 10 client.19290 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe402425b80 1 -- 172.21.15.46:0/452908812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c3c560eb60 con 0x55c3c588e380 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe3cf7fe700 1 -- 172.21.15.46:0/452908812 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e4031020 con 0x55c3c588e380 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe3cf7fe700 1 -- 172.21.15.46:0/452908812 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe3e400f9b0 con 0x55c3c588e380 2024-03-18T20:46:57.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe402425b80 10 client.19290 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe402425b80 10 client.19290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe402425b80 20 client.19290 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe402425b80 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c3c58a8e30 0x55c3c58ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe402425b80 4 client.19290 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.312+0000 7fe402425b80 1 -- 172.21.15.46:0/452908812 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3c57d9700 con 0x55c3c58a8e30 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.313+0000 7fe3effff700 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c3c58a8e30 0x55c3c58ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.313+0000 7fe3effff700 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c3c58a8e30 0x55c3c58ab250 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35690d1832120cab server_cookie=92e3868c63a12c54 in_seq=0 out_seq=0 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.313+0000 7fe3cf7fe700 10 client.19290 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe3cf7fe700 1 -- 172.21.15.46:0/452908812 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c3c57d9700 con 0x55c3c58a8e30 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe3cf7fe700 10 client.19290 handle_command_reply: tid=0 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe402425b80 1 client.19290 shutdown 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe402425b80 1 -- 172.21.15.46:0/452908812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c588e380 msgr2=0x55c3c5895f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe402425b80 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c588e380 0x55c3c5895f90 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00679c0 tx=0x7fe3e4005770 comp rx=0 tx=0).stop 2024-03-18T20:46:57.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe402425b80 1 -- 172.21.15.46:0/452908812 shutdown_connections 2024-03-18T20:46:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe402425b80 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c3c58a8e30 0x55c3c58ab250 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe402425b80 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c5899220 0x55c3c5896680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe402425b80 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588da60 0x55c3c5898ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe3cf7fe700 0 client.19290 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe402425b80 1 --2- 172.21.15.46:0/452908812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c588e380 0x55c3c5895f90 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe402425b80 1 -- 172.21.15.46:0/452908812 >> 172.21.15.46:0/452908812 conn(0x55c3c5804220 msgr2=0x55c3c5889bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe402425b80 1 -- 172.21.15.46:0/452908812 shutdown_connections 2024-03-18T20:46:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe402425b80 1 -- 172.21.15.46:0/452908812 wait complete. 2024-03-18T20:46:57.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.314+0000 7fe402425b80 20 client.19290 trim_cache size 0 max 16384 2024-03-18T20:46:57.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe402425b80 1 Processor -- start 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe402425b80 1 -- start start 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe402425b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588da60 0x55c3c589fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe402425b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588e380 0x55c3c58a4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe402425b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c58a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe402425b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c58a64e0 con 0x55c3c5899220 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe402425b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c588b230 con 0x55c3c588da60 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe402425b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c58865e0 con 0x55c3c588e380 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe3f482d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c58a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe3f482d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c58a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51746/0 (socket says 172.21.15.46:51746) 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe3f482d700 1 -- 172.21.15.46:0/2706208461 learned_addr learned my addr 172.21.15.46:0/2706208461 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe3effff700 1 --2- 172.21.15.46:0/2706208461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588da60 0x55c3c589fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.332+0000 7fe3ef7fe700 1 --2- 172.21.15.46:0/2706208461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588e380 0x55c3c58a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.333+0000 7fe3f482d700 1 -- 172.21.15.46:0/2706208461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588e380 msgr2=0x55c3c58a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.333+0000 7fe3f482d700 1 --2- 172.21.15.46:0/2706208461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588e380 0x55c3c58a4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.333+0000 7fe3f482d700 1 -- 172.21.15.46:0/2706208461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588da60 msgr2=0x55c3c589fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.333+0000 7fe3f482d700 1 --2- 172.21.15.46:0/2706208461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588da60 0x55c3c589fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.333+0000 7fe3f482d700 1 -- 172.21.15.46:0/2706208461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3c57a6ea0 con 0x55c3c5899220 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.333+0000 7fe3ef7fe700 1 --2- 172.21.15.46:0/2706208461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588e380 0x55c3c58a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.333+0000 7fe3f482d700 1 --2- 172.21.15.46:0/2706208461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c58a4a80 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009510 tx=0x7fe3e8004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b32b1cecdcbeca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.333+0000 7fe3d6ffd700 1 -- 172.21.15.46:0/2706208461 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e809d170 con 0x55c3c5899220 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.333+0000 7fe3d6ffd700 1 -- 172.21.15.46:0/2706208461 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e80a19e0 con 0x55c3c5899220 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.333+0000 7fe3d6ffd700 1 -- 172.21.15.46:0/2706208461 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8022250 con 0x55c3c5899220 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.333+0000 7fe402425b80 1 -- 172.21.15.46:0/2706208461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 msgr2=0x55c3c58a4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.333+0000 7fe402425b80 1 --2- 172.21.15.46:0/2706208461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c58a4a80 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009510 tx=0x7fe3e8004c40 comp rx=0 tx=0).stop 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.334+0000 7fe402425b80 1 -- 172.21.15.46:0/2706208461 shutdown_connections 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.334+0000 7fe402425b80 1 --2- 172.21.15.46:0/2706208461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c58a4a80 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.334+0000 7fe402425b80 1 --2- 172.21.15.46:0/2706208461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588da60 0x55c3c589fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.334+0000 7fe402425b80 1 --2- 172.21.15.46:0/2706208461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588e380 0x55c3c58a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.334+0000 7fe402425b80 1 -- 172.21.15.46:0/2706208461 >> 172.21.15.46:0/2706208461 conn(0x55c3c5804220 msgr2=0x55c3c5894860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.334+0000 7fe402425b80 1 -- 172.21.15.46:0/2706208461 shutdown_connections 2024-03-18T20:46:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.334+0000 7fe402425b80 1 -- 172.21.15.46:0/2706208461 wait complete. 2024-03-18T20:46:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.334+0000 7fe402425b80 1 -- start start 2024-03-18T20:46:57.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.334+0000 7fe402425b80 1 Processor -- start 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe402425b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588da60 0x55c3c589f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe402425b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588e380 0x55c3c589fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe402425b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c589de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe402425b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c589e520 con 0x55c3c5899220 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe402425b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c589e690 con 0x55c3c588e380 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe402425b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c589e850 con 0x55c3c588da60 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe402425b80 10 client.0 _ll_register_callbacks cb 0x7fe3e4005400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3f482d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c589de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3f482d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c589de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51756/0 (socket says 172.21.15.46:51756) 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3f482d700 1 -- 172.21.15.46:0/1646583477 learned_addr learned my addr 172.21.15.46:0/1646583477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3effff700 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588da60 0x55c3c589f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3ef7fe700 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588e380 0x55c3c589fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3f482d700 1 -- 172.21.15.46:0/1646583477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588da60 msgr2=0x55c3c589f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3f482d700 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588da60 0x55c3c589f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3f482d700 1 -- 172.21.15.46:0/1646583477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588e380 msgr2=0x55c3c589fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3f482d700 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588e380 0x55c3c589fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3f482d700 1 -- 172.21.15.46:0/1646583477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3c56623b0 con 0x55c3c5899220 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3effff700 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588da60 0x55c3c589f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3f482d700 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c589de10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80ca120 tx=0x7fe3e8095f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475443cebd0f94f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.335+0000 7fe3cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe3cf7fe700 1 -- 172.21.15.46:0/1646583477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e809d390 con 0x55c3c5899220 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe3cf7fe700 1 -- 172.21.15.46:0/1646583477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe3e8022250 con 0x55c3c5899220 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe402425b80 1 -- 172.21.15.46:0/1646583477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3e8099560 con 0x55c3c5899220 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe3cf7fe700 1 -- 172.21.15.46:0/1646583477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3e8022410 con 0x55c3c5899220 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe402425b80 10 client.19296 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe402425b80 1 -- 172.21.15.46:0/1646583477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c3c5666cd0 con 0x55c3c5899220 2024-03-18T20:46:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe3cf7fe700 1 -- 172.21.15.46:0/1646583477 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e8099560 con 0x55c3c5899220 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe3cf7fe700 1 -- 172.21.15.46:0/1646583477 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe3e80997c0 con 0x55c3c5899220 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe402425b80 10 client.19296 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe402425b80 10 client.19296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe402425b80 20 client.19296 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe402425b80 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c3c58fda30 0x55c3c58ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe402425b80 4 client.19296 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.336+0000 7fe402425b80 1 -- 172.21.15.46:0/1646583477 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c3c562fcf0 con 0x55c3c58fda30 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe3effff700 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c3c58fda30 0x55c3c58ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe3effff700 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c3c58fda30 0x55c3c58ffe50 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a862aa1c29943a01 server_cookie=81333c9d90d66e19 in_seq=0 out_seq=0 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe3cf7fe700 10 client.19296 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe3cf7fe700 1 -- 172.21.15.46:0/1646583477 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c3c562fcf0 con 0x55c3c58fda30 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe3cf7fe700 10 client.19296 handle_command_reply: tid=0 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe402425b80 1 client.19296 shutdown 2024-03-18T20:46:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe402425b80 1 -- 172.21.15.46:0/1646583477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 msgr2=0x55c3c589de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe402425b80 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c589de10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80ca120 tx=0x7fe3e8095f40 comp rx=0 tx=0).stop 2024-03-18T20:46:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe402425b80 1 -- 172.21.15.46:0/1646583477 shutdown_connections 2024-03-18T20:46:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe402425b80 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c3c58fda30 0x55c3c58ffe50 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe402425b80 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c3c5899220 0x55c3c589de10 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe3cf7fe700 0 client.19296 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe402425b80 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c3c588da60 0x55c3c589f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe402425b80 1 --2- 172.21.15.46:0/1646583477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c3c588e380 0x55c3c589fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.337+0000 7fe402425b80 1 -- 172.21.15.46:0/1646583477 >> 172.21.15.46:0/1646583477 conn(0x55c3c5804220 msgr2=0x55c3c5895d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe402425b80 1 -- 172.21.15.46:0/1646583477 shutdown_connections 2024-03-18T20:46:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe402425b80 1 -- 172.21.15.46:0/1646583477 wait complete. 2024-03-18T20:46:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe402425b80 20 client.19296 trim_cache size 0 max 16384 2024-03-18T20:46:57.347 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:46:57.347 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:46:57.347 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:46:57.347 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:46:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe3f6290700 1 -- 172.21.15.46:0/2647091875 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe3d80a1290 msgr2=0x7fe3d80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe3f6290700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe3d80a1290 0x7fe3d80a3750 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0011490 tx=0x7fe3e0009260 comp rx=0 tx=0).stop 2024-03-18T20:46:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe3f6290700 1 -- 172.21.15.46:0/2647091875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f019bae0 msgr2=0x7fe3f019bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe3f6290700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f019bae0 0x7fe3f019bf60 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8007ee0 tx=0x7fe3e800d3b0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe3f6290700 1 -- 172.21.15.46:0/2647091875 shutdown_connections 2024-03-18T20:46:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe3f6290700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe3d80a1290 0x7fe3d80a3750 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe3f6290700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f019bae0 0x7fe3f019bf60 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe3f6290700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0100f50 0x7fe3f0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe3f6290700 1 --2- 172.21.15.46:0/2647091875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3f0106070 0x7fe3f0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe3f6290700 1 -- 172.21.15.46:0/2647091875 >> 172.21.15.46:0/2647091875 conn(0x7fe3f00fc7f0 msgr2=0x7fe3f00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe3f6290700 1 -- 172.21.15.46:0/2647091875 shutdown_connections 2024-03-18T20:46:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.338+0000 7fe3f6290700 1 -- 172.21.15.46:0/2647091875 wait complete. 2024-03-18T20:46:57.357 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:46:57.357 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:46:57.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.495+0000 7febccc39700 1 -- 172.21.15.46:0/1592577816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8105bf0 msgr2=0x7febc8105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.495+0000 7febccc39700 1 --2- 172.21.15.46:0/1592577816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8105bf0 0x7febc8105fd0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7febb8009990 tx=0x7febb8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.495+0000 7febccc39700 1 -- 172.21.15.46:0/1592577816 shutdown_connections 2024-03-18T20:46:57.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.495+0000 7febccc39700 1 --2- 172.21.15.46:0/1592577816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8105bf0 0x7febc8105fd0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.495+0000 7febccc39700 1 --2- 172.21.15.46:0/1592577816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc80ffad0 0x7febc810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.495+0000 7febccc39700 1 --2- 172.21.15.46:0/1592577816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febc80ff110 0x7febc80ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.495+0000 7febccc39700 1 -- 172.21.15.46:0/1592577816 >> 172.21.15.46:0/1592577816 conn(0x7febc8076070 msgr2=0x7febc80fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.495+0000 7febccc39700 1 -- 172.21.15.46:0/1592577816 shutdown_connections 2024-03-18T20:46:57.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.495+0000 7febccc39700 1 -- 172.21.15.46:0/1592577816 wait complete. 2024-03-18T20:46:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febccc39700 1 Processor -- start 2024-03-18T20:46:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febccc39700 1 -- start start 2024-03-18T20:46:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febccc39700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc80ff110 0x7febc8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febccc39700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febc80ffad0 0x7febc8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febccc39700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8105bf0 0x7febc819dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febccc39700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc819e270 con 0x7febc8105bf0 2024-03-18T20:46:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febccc39700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc819e3e0 con 0x7febc80ff110 2024-03-18T20:46:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febccc39700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc819e5a0 con 0x7febc80ffad0 2024-03-18T20:46:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febc6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8105bf0 0x7febc819dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febc6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8105bf0 0x7febc819dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51772/0 (socket says 172.21.15.46:51772) 2024-03-18T20:46:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febc6d9d700 1 -- 172.21.15.46:0/3151826964 learned_addr learned my addr 172.21.15.46:0/3151826964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:57.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febc5d9b700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febc80ffad0 0x7febc8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.496+0000 7febc659c700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc80ff110 0x7febc8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.497+0000 7febc6d9d700 1 -- 172.21.15.46:0/3151826964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febc80ffad0 msgr2=0x7febc8199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.497+0000 7febc6d9d700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febc80ffad0 0x7febc8199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.497+0000 7febc6d9d700 1 -- 172.21.15.46:0/3151826964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc80ff110 msgr2=0x7febc8199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.497+0000 7febc6d9d700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc80ff110 0x7febc8199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.497+0000 7febc6d9d700 1 -- 172.21.15.46:0/3151826964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb80095f0 con 0x7febc8105bf0 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.497+0000 7febc6d9d700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8105bf0 0x7febc819dc30 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7febc0007ee0 tx=0x7febc000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.497+0000 7febb77fe700 1 -- 172.21.15.46:0/3151826964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0017070 con 0x7febc8105bf0 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.497+0000 7febb77fe700 1 -- 172.21.15.46:0/3151826964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc000f040 con 0x7febc8105bf0 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.497+0000 7febccc39700 1 -- 172.21.15.46:0/3151826964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc819e880 con 0x7febc8105bf0 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.497+0000 7febb77fe700 1 -- 172.21.15.46:0/3151826964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0005580 con 0x7febc8105bf0 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.497+0000 7febccc39700 1 -- 172.21.15.46:0/3151826964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc81a60c0 con 0x7febc8105bf0 2024-03-18T20:46:57.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.498+0000 7febccc39700 1 -- 172.21.15.46:0/3151826964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc8102430 con 0x7febc8105bf0 2024-03-18T20:46:57.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.499+0000 7febb77fe700 1 -- 172.21.15.46:0/3151826964 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7febc000db20 con 0x7febc8105bf0 2024-03-18T20:46:57.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.500+0000 7febb77fe700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7febb00a1310 0x7febb00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.500+0000 7febb77fe700 1 -- 172.21.15.46:0/3151826964 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7febc00cca30 con 0x7febc8105bf0 2024-03-18T20:46:57.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.500+0000 7febc659c700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7febb00a1310 0x7febb00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.501+0000 7febc659c700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7febb00a1310 0x7febb00a37d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7febb8006010 tx=0x7febb800b260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.504+0000 7febb77fe700 1 -- 172.21.15.46:0/3151826964 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7febc0095e60 con 0x7febc8105bf0 2024-03-18T20:46:57.716 INFO:tasks.workunit.client.0.smithi046.stdout: 1 406092 3.70 MB/sec execute 472 sec latency 1394.975 ms 2024-03-18T20:46:57.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.771+0000 7febccc39700 1 -- 172.21.15.46:0/3151826964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febc8067180 con 0x7febc8105bf0 2024-03-18T20:46:57.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.772+0000 7febb77fe700 1 -- 172.21.15.46:0/3151826964 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+10916 (secure 0 0 0) 0x7febc0099480 con 0x7febc8105bf0 2024-03-18T20:46:57.775 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:46:57.775 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:45:43.243732+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:46:57.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.775+0000 7febccc39700 1 -- 172.21.15.46:0/3151826964 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7febb00a1310 msgr2=0x7febb00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.775+0000 7febccc39700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7febb00a1310 0x7febb00a37d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7febb8006010 tx=0x7febb800b260 comp rx=0 tx=0).stop 2024-03-18T20:46:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.775+0000 7febccc39700 1 -- 172.21.15.46:0/3151826964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8105bf0 msgr2=0x7febc819dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.775+0000 7febccc39700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8105bf0 0x7febc819dc30 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7febc0007ee0 tx=0x7febc000d3b0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.775+0000 7febccc39700 1 -- 172.21.15.46:0/3151826964 shutdown_connections 2024-03-18T20:46:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.775+0000 7febccc39700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7febb00a1310 0x7febb00a37d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.775+0000 7febccc39700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febc8105bf0 0x7febc819dc30 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.775+0000 7febccc39700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc80ff110 0x7febc8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.775+0000 7febccc39700 1 --2- 172.21.15.46:0/3151826964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febc80ffad0 0x7febc8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.775+0000 7febccc39700 1 -- 172.21.15.46:0/3151826964 >> 172.21.15.46:0/3151826964 conn(0x7febc8076070 msgr2=0x7febc8103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.775+0000 7febccc39700 1 -- 172.21.15.46:0/3151826964 shutdown_connections 2024-03-18T20:46:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.775+0000 7febccc39700 1 -- 172.21.15.46:0/3151826964 wait complete. 2024-03-18T20:46:57.779 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 49 2024-03-18T20:46:57.791 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:46:57.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:46:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.927+0000 7f7c6f697700 1 -- 172.21.15.46:0/2795778789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c680687f0 msgr2=0x7f7c6810db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.927+0000 7f7c6f697700 1 --2- 172.21.15.46:0/2795778789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c680687f0 0x7f7c6810db50 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000b0a0 tx=0x7f7c6000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.927+0000 7f7c6f697700 1 -- 172.21.15.46:0/2795778789 shutdown_connections 2024-03-18T20:46:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.927+0000 7f7c6f697700 1 --2- 172.21.15.46:0/2795778789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c680687f0 0x7f7c6810db50 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.927+0000 7f7c6f697700 1 --2- 172.21.15.46:0/2795778789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c68105ce0 0x7f7c681060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.927+0000 7f7c6f697700 1 --2- 172.21.15.46:0/2795778789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68067e30 0x7f7c680682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.927+0000 7f7c6f697700 1 -- 172.21.15.46:0/2795778789 >> 172.21.15.46:0/2795778789 conn(0x7f7c68076d40 msgr2=0x7f7c68077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.927+0000 7f7c6f697700 1 -- 172.21.15.46:0/2795778789 shutdown_connections 2024-03-18T20:46:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.927+0000 7f7c6f697700 1 -- 172.21.15.46:0/2795778789 wait complete. 2024-03-18T20:46:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.928+0000 7f7c6f697700 1 Processor -- start 2024-03-18T20:46:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.928+0000 7f7c6f697700 1 -- start start 2024-03-18T20:46:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6f697700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68067e30 0x7f7c68197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6f697700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c680687f0 0x7f7c68197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6f697700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c68105ce0 0x7f7c6819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6f697700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6819c230 con 0x7f7c680687f0 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6f697700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6819c3a0 con 0x7f7c68067e30 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6f697700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6819c560 con 0x7f7c68105ce0 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6cc32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c680687f0 0x7f7c68197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6cc32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c680687f0 0x7f7c68197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51794/0 (socket says 172.21.15.46:51794) 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6cc32700 1 -- 172.21.15.46:0/3791458589 learned_addr learned my addr 172.21.15.46:0/3791458589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6dc34700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c68105ce0 0x7f7c6819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6d433700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68067e30 0x7f7c68197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6cc32700 1 -- 172.21.15.46:0/3791458589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c68105ce0 msgr2=0x7f7c6819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6cc32700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c68105ce0 0x7f7c6819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6cc32700 1 -- 172.21.15.46:0/3791458589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68067e30 msgr2=0x7f7c68197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6cc32700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68067e30 0x7f7c68197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6cc32700 1 -- 172.21.15.46:0/3791458589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c60009d00 con 0x7f7c680687f0 2024-03-18T20:46:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6d433700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68067e30 0x7f7c68197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.929+0000 7f7c6cc32700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c680687f0 0x7f7c68197850 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00d950 tx=0x7f7c5c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.930+0000 7f7c5a7fc700 1 -- 172.21.15.46:0/3791458589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c009940 con 0x7f7c680687f0 2024-03-18T20:46:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.930+0000 7f7c5a7fc700 1 -- 172.21.15.46:0/3791458589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c5c010460 con 0x7f7c680687f0 2024-03-18T20:46:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.930+0000 7f7c6f697700 1 -- 172.21.15.46:0/3791458589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6819c840 con 0x7f7c680687f0 2024-03-18T20:46:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.930+0000 7f7c5a7fc700 1 -- 172.21.15.46:0/3791458589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c00f620 con 0x7f7c680687f0 2024-03-18T20:46:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.930+0000 7f7c6f697700 1 -- 172.21.15.46:0/3791458589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c681a4080 con 0x7f7c680687f0 2024-03-18T20:46:57.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.932+0000 7f7c5a7fc700 1 -- 172.21.15.46:0/3791458589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7c5c0108d0 con 0x7f7c680687f0 2024-03-18T20:46:57.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.933+0000 7f7c5a7fc700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c540a1290 0x7f7c540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.933+0000 7f7c6d433700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c540a1290 0x7f7c540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.933+0000 7f7c5a7fc700 1 -- 172.21.15.46:0/3791458589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7c5c0ccc00 con 0x7f7c680687f0 2024-03-18T20:46:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.933+0000 7f7c6d433700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c540a1290 0x7f7c540a3750 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7c64005950 tx=0x7f7c640058e0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.934+0000 7f7c7b82cb80 1 Processor -- start 2024-03-18T20:46:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.934+0000 7f7c7b82cb80 1 -- start start 2024-03-18T20:46:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.934+0000 7f7c7b82cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 0x562eb5bbee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.934+0000 7f7c7b82cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbf380 0x562eb5bc9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.934+0000 7f7c7b82cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bca220 0x562eb5bcc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.934+0000 7f7c7b82cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb5b18990 con 0x562eb5bbea60 2024-03-18T20:46:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.934+0000 7f7c7b82cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb5b16350 con 0x562eb5bbf380 2024-03-18T20:46:57.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.934+0000 7f7c7b82cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb5b11c40 con 0x562eb5bca220 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.934+0000 7f7c6d433700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 0x562eb5bbee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.934+0000 7f7c6d433700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 0x562eb5bbee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51804/0 (socket says 172.21.15.46:51804) 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.934+0000 7f7c6d433700 1 -- 172.21.15.46:0/918093071 learned_addr learned my addr 172.21.15.46:0/918093071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.935+0000 7f7c6dc34700 1 --2- 172.21.15.46:0/918093071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bca220 0x562eb5bcc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.935+0000 7f7c6cc32700 1 --2- 172.21.15.46:0/918093071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbf380 0x562eb5bc9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.935+0000 7f7c6d433700 1 -- 172.21.15.46:0/918093071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bca220 msgr2=0x562eb5bcc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.935+0000 7f7c6d433700 1 --2- 172.21.15.46:0/918093071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bca220 0x562eb5bcc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.935+0000 7f7c6d433700 1 -- 172.21.15.46:0/918093071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbf380 msgr2=0x562eb5bc9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.935+0000 7f7c6d433700 1 --2- 172.21.15.46:0/918093071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbf380 0x562eb5bc9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.935+0000 7f7c6d433700 1 -- 172.21.15.46:0/918093071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eb59933b0 con 0x562eb5bbea60 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.935+0000 7f7c6cc32700 1 --2- 172.21.15.46:0/918093071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbf380 0x562eb5bc9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.935+0000 7f7c6d433700 1 --2- 172.21.15.46:0/918093071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 0x562eb5bbee40 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400ff00 tx=0x7f7c64012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a309d91d2c2ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.935+0000 7f7c537fe700 1 -- 172.21.15.46:0/918093071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64017040 con 0x562eb5bbea60 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.935+0000 7f7c537fe700 1 -- 172.21.15.46:0/918093071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c64003b40 con 0x562eb5bbea60 2024-03-18T20:46:57.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.935+0000 7f7c537fe700 1 -- 172.21.15.46:0/918093071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6401fcb0 con 0x562eb5bbea60 2024-03-18T20:46:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.936+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/918093071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 msgr2=0x562eb5bbee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.936+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/918093071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 0x562eb5bbee40 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400ff00 tx=0x7f7c64012000 comp rx=0 tx=0).stop 2024-03-18T20:46:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.936+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/918093071 shutdown_connections 2024-03-18T20:46:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.936+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/918093071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 0x562eb5bbee40 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.936+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/918093071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bca220 0x562eb5bcc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.936+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/918093071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbf380 0x562eb5bc9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.936+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/918093071 >> 172.21.15.46:0/918093071 conn(0x562eb5b35220 msgr2=0x562eb5bbb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.936+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/918093071 shutdown_connections 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.936+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/918093071 wait complete. 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.936+0000 7f7c7b82cb80 1 -- start start 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.936+0000 7f7c7b82cb80 1 Processor -- start 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c7b82cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 0x562eb5bc9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c7b82cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbf380 0x562eb5bc6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c7b82cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bca220 0x562eb5bc7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c7b82cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb5aadd80 con 0x562eb5bbea60 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c7b82cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb5aada20 con 0x562eb5bbf380 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c7b82cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb59956a0 con 0x562eb5bca220 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c7b82cb80 10 client.0 _ll_register_callbacks cb 0x562eb59193a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c6d433700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 0x562eb5bc9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c6d433700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 0x562eb5bc9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51816/0 (socket says 172.21.15.46:51816) 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c6d433700 1 -- 172.21.15.46:0/2495224165 learned_addr learned my addr 172.21.15.46:0/2495224165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c6cc32700 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbf380 0x562eb5bc6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c6dc34700 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bca220 0x562eb5bc7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c6d433700 1 -- 172.21.15.46:0/2495224165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bca220 msgr2=0x562eb5bc7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c6d433700 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bca220 0x562eb5bc7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c6d433700 1 -- 172.21.15.46:0/2495224165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbf380 msgr2=0x562eb5bc6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c6d433700 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbf380 0x562eb5bc6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c6d433700 1 -- 172.21.15.46:0/2495224165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6400fbe0 con 0x562eb5bbea60 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.937+0000 7f7c6dc34700 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bca220 0x562eb5bc7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c6d433700 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 0x562eb5bc9bb0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7c6401f810 tx=0x7f7c64003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be4ada1b4f58c19 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c50ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c50ff9700 1 -- 172.21.15.46:0/2495224165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64003890 con 0x562eb5bbea60 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c50ff9700 1 -- 172.21.15.46:0/2495224165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7c6401f910 con 0x562eb5bbea60 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/2495224165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562eb5bb2ad0 con 0x562eb5bbea60 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c50ff9700 1 -- 172.21.15.46:0/2495224165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c6401fad0 con 0x562eb5bbea60 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c7b82cb80 10 client.19314 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/2495224165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x562eb593fb60 con 0x562eb5bbea60 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c50ff9700 1 -- 172.21.15.46:0/2495224165 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562eb5bb2ad0 con 0x562eb5bbea60 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c50ff9700 1 -- 172.21.15.46:0/2495224165 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7c6401d040 con 0x562eb5bbea60 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c7b82cb80 10 client.19314 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c7b82cb80 10 client.19314 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c7b82cb80 20 client.19314 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x562eb5bd9c20 0x562eb5bdc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c7b82cb80 4 client.19314 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.938+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/2495224165 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562eb5b0a700 con 0x562eb5bd9c20 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.939+0000 7f7c6cc32700 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x562eb5bd9c20 0x562eb5bdc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.939+0000 7f7c50ff9700 10 client.19314 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.939+0000 7f7c6cc32700 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x562eb5bd9c20 0x562eb5bdc040 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8aebd3efd5a16d6b server_cookie=5668c5ca1780e99f in_seq=0 out_seq=0 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c50ff9700 1 -- 172.21.15.46:0/2495224165 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562eb5b0a700 con 0x562eb5bd9c20 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c50ff9700 10 client.19314 handle_command_reply: tid=0 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c7b82cb80 1 client.19314 shutdown 2024-03-18T20:46:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/2495224165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 msgr2=0x562eb5bc9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 0x562eb5bc9bb0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7c6401f810 tx=0x7f7c64003ee0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/2495224165 shutdown_connections 2024-03-18T20:46:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x562eb5bd9c20 0x562eb5bdc040 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bbea60 0x562eb5bc9bb0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bca220 0x562eb5bc7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c50ff9700 0 client.19314 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:46:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/2495224165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbf380 0x562eb5bc6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/2495224165 >> 172.21.15.46:0/2495224165 conn(0x562eb5b35220 msgr2=0x562eb5bbac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/2495224165 shutdown_connections 2024-03-18T20:46:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/2495224165 wait complete. 2024-03-18T20:46:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.940+0000 7f7c7b82cb80 20 client.19314 trim_cache size 0 max 16384 2024-03-18T20:46:57.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.954+0000 7f7c7b82cb80 1 Processor -- start 2024-03-18T20:46:57.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.954+0000 7f7c7b82cb80 1 -- start start 2024-03-18T20:46:57.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c7b82cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 0x562eb5bc19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c7b82cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bbf380 0x562eb5bdc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c7b82cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 0x562eb5bd0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c7b82cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb5bbfd30 con 0x562eb5bca220 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c7b82cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb5bb75e0 con 0x562eb5bbea60 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c7b82cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb5bc7e10 con 0x562eb5bbf380 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c6dc34700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 0x562eb5bd0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c6d433700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 0x562eb5bc19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c6cc32700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bbf380 0x562eb5bdc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c6d433700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 0x562eb5bc19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49608/0 (socket says 172.21.15.46:49608) 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c6dc34700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 0x562eb5bd0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51832/0 (socket says 172.21.15.46:51832) 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c6d433700 1 -- 172.21.15.46:0/3457504128 learned_addr learned my addr 172.21.15.46:0/3457504128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c6cc32700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bbf380 0x562eb5bdc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:51984/0 (socket says 172.21.15.46:51984) 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c6d433700 1 -- 172.21.15.46:0/3457504128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bbf380 msgr2=0x562eb5bdc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c6d433700 1 --2- 172.21.15.46:0/3457504128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bbf380 0x562eb5bdc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c6d433700 1 -- 172.21.15.46:0/3457504128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 msgr2=0x562eb5bd0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c6d433700 1 --2- 172.21.15.46:0/3457504128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 0x562eb5bd0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.955+0000 7f7c6d433700 1 -- 172.21.15.46:0/3457504128 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eb5ad7ea0 con 0x562eb5bbea60 2024-03-18T20:46:57.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.956+0000 7f7c6d433700 1 --2- 172.21.15.46:0/3457504128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 0x562eb5bc19c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400f9d0 tx=0x7f7c6402af70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ffd1d3d45214ebe server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.956+0000 7f7c53fff700 1 -- 172.21.15.46:0/3457504128 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6400feb0 con 0x562eb5bbea60 2024-03-18T20:46:57.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.956+0000 7f7c53fff700 1 -- 172.21.15.46:0/3457504128 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c6401f460 con 0x562eb5bbea60 2024-03-18T20:46:57.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.956+0000 7f7c53fff700 1 -- 172.21.15.46:0/3457504128 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6402a7c0 con 0x562eb5bbea60 2024-03-18T20:46:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.956+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/3457504128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 msgr2=0x562eb5bc19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.956+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/3457504128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 0x562eb5bc19c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400f9d0 tx=0x7f7c6402af70 comp rx=0 tx=0).stop 2024-03-18T20:46:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.957+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/3457504128 shutdown_connections 2024-03-18T20:46:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.957+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/3457504128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 0x562eb5bd0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.957+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/3457504128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 0x562eb5bc19c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.957+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/3457504128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bbf380 0x562eb5bdc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.957+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/3457504128 >> 172.21.15.46:0/3457504128 conn(0x562eb5b35220 msgr2=0x562eb5bb6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.957+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/3457504128 shutdown_connections 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.957+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/3457504128 wait complete. 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.957+0000 7f7c7b82cb80 1 -- start start 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.957+0000 7f7c7b82cb80 1 Processor -- start 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c7b82cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 0x562eb5bcee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c7b82cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bbf380 0x562eb5bcf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c7b82cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 0x562eb5c2ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c7b82cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb5bcfae0 con 0x562eb5bca220 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c7b82cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb5bcfc50 con 0x562eb5bbea60 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c7b82cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eb5c2cf90 con 0x562eb5bbf380 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c7b82cb80 10 client.0 _ll_register_callbacks cb 0x7f7c60009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6dc34700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 0x562eb5c2ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6d433700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 0x562eb5bcee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6d433700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 0x562eb5bcee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49624/0 (socket says 172.21.15.46:49624) 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6d433700 1 -- 172.21.15.46:0/1204380853 learned_addr learned my addr 172.21.15.46:0/1204380853 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6cc32700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bbf380 0x562eb5bcf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6dc34700 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 0x562eb5c2ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51838/0 (socket says 172.21.15.46:51838) 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6d433700 1 -- 172.21.15.46:0/1204380853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bbf380 msgr2=0x562eb5bcf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6d433700 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bbf380 0x562eb5bcf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6d433700 1 -- 172.21.15.46:0/1204380853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 msgr2=0x562eb5c2ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6d433700 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 0x562eb5c2ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6d433700 1 -- 172.21.15.46:0/1204380853 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562eb593fb60 con 0x562eb5bbea60 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6cc32700 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bbf380 0x562eb5bcf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6dc34700 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 0x562eb5c2ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c6d433700 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 0x562eb5bcee30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c6402ad70 tx=0x7f7c6401fc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471ab95dba95bd0f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.958+0000 7f7c50ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c50ff9700 1 -- 172.21.15.46:0/1204380853 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6400b4a0 con 0x562eb5bbea60 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c50ff9700 1 -- 172.21.15.46:0/1204380853 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7c64003890 con 0x562eb5bbea60 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/1204380853 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c64003b40 con 0x562eb5bbea60 2024-03-18T20:46:57.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c50ff9700 1 -- 172.21.15.46:0/1204380853 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c6400feb0 con 0x562eb5bbea60 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c7b82cb80 10 client.28949 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/1204380853 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x562eb5997cd0 con 0x562eb5bbea60 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c50ff9700 1 -- 172.21.15.46:0/1204380853 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c64003b40 con 0x562eb5bbea60 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c50ff9700 1 -- 172.21.15.46:0/1204380853 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7c6401d070 con 0x562eb5bbea60 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c7b82cb80 10 client.28949 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c7b82cb80 10 client.28949 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c7b82cb80 20 client.28949 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x562eb5c2e250 0x562eb5c30640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c7b82cb80 4 client.28949 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.959+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/1204380853 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562eb5960cf0 con 0x562eb5c2e250 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c6cc32700 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x562eb5c2e250 0x562eb5c30640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c6cc32700 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x562eb5c2e250 0x562eb5c30640 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=83230e9e589f65ba server_cookie=40ae7ea7fd419bdc in_seq=0 out_seq=0 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c50ff9700 10 client.28949 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:46:57.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c50ff9700 1 -- 172.21.15.46:0/1204380853 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562eb5960cf0 con 0x562eb5c2e250 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c50ff9700 10 client.28949 handle_command_reply: tid=0 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c7b82cb80 1 client.28949 shutdown 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/1204380853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 msgr2=0x562eb5bcee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 0x562eb5bcee30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c6402ad70 tx=0x7f7c6401fc60 comp rx=0 tx=0).stop 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/1204380853 shutdown_connections 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x562eb5c2e250 0x562eb5c30640 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eb5bca220 0x562eb5c2ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c50ff9700 0 client.28949 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eb5bbea60 0x562eb5bcee30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c7b82cb80 1 --2- 172.21.15.46:0/1204380853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eb5bbf380 0x562eb5bcf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.960+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/1204380853 >> 172.21.15.46:0/1204380853 conn(0x562eb5b35220 msgr2=0x562eb5bbaac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/1204380853 shutdown_connections 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c7b82cb80 1 -- 172.21.15.46:0/1204380853 wait complete. 2024-03-18T20:46:57.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c7b82cb80 20 client.28949 trim_cache size 0 max 16384 2024-03-18T20:46:57.969 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:46:57.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c6f697700 1 -- 172.21.15.46:0/3791458589 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c540a1290 msgr2=0x7f7c540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c6f697700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c540a1290 0x7f7c540a3750 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7c64005950 tx=0x7f7c640058e0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c6f697700 1 -- 172.21.15.46:0/3791458589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c680687f0 msgr2=0x7f7c68197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:57.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c6f697700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c680687f0 0x7f7c68197850 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00d950 tx=0x7f7c5c00dc60 comp rx=0 tx=0).stop 2024-03-18T20:46:57.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c6f697700 1 -- 172.21.15.46:0/3791458589 shutdown_connections 2024-03-18T20:46:57.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c6f697700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7c540a1290 0x7f7c540a3750 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c6f697700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c68105ce0 0x7f7c6819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c6f697700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68067e30 0x7f7c68197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c6f697700 1 --2- 172.21.15.46:0/3791458589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c680687f0 0x7f7c68197850 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:57.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c6f697700 1 -- 172.21.15.46:0/3791458589 >> 172.21.15.46:0/3791458589 conn(0x7f7c68076d40 msgr2=0x7f7c680fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:57.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c6f697700 1 -- 172.21.15.46:0/3791458589 shutdown_connections 2024-03-18T20:46:57.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:57.961+0000 7f7c6f697700 1 -- 172.21.15.46:0/3791458589 wait complete. 2024-03-18T20:46:57.977 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:46:57.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:46:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.114+0000 7faa8536d700 1 -- 172.21.15.46:0/1160641836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa800ffe60 msgr2=0x7faa80100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.114+0000 7faa8536d700 1 --2- 172.21.15.46:0/1160641836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa800ffe60 0x7faa80100240 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7faa70009990 tx=0x7faa70009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.115+0000 7faa8536d700 1 -- 172.21.15.46:0/1160641836 shutdown_connections 2024-03-18T20:46:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.115+0000 7faa8536d700 1 --2- 172.21.15.46:0/1160641836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa800ffe60 0x7faa80100240 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.115+0000 7faa8536d700 1 --2- 172.21.15.46:0/1160641836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa80104f80 0x7faa8010cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.115+0000 7faa8536d700 1 --2- 172.21.15.46:0/1160641836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa80100810 0x7faa80104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.115+0000 7faa8536d700 1 -- 172.21.15.46:0/1160641836 >> 172.21.15.46:0/1160641836 conn(0x7faa800fb6c0 msgr2=0x7faa800fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.115+0000 7faa8536d700 1 -- 172.21.15.46:0/1160641836 shutdown_connections 2024-03-18T20:46:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.115+0000 7faa8536d700 1 -- 172.21.15.46:0/1160641836 wait complete. 2024-03-18T20:46:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.115+0000 7faa8536d700 1 Processor -- start 2024-03-18T20:46:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa8536d700 1 -- start start 2024-03-18T20:46:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa8536d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa800ffe60 0x7faa801971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa8536d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa80104f80 0x7faa80197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa8536d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa8019bac0 0x7faa8019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa8536d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8019c540 con 0x7faa800ffe60 2024-03-18T20:46:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa8536d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8019c6b0 con 0x7faa80104f80 2024-03-18T20:46:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa8536d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8019c870 con 0x7faa8019bac0 2024-03-18T20:46:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa800ffe60 0x7faa801971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa800ffe60 0x7faa801971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51858/0 (socket says 172.21.15.46:51858) 2024-03-18T20:46:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa7effd700 1 -- 172.21.15.46:0/2002752426 learned_addr learned my addr 172.21.15.46:0/2002752426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa7e7fc700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa80104f80 0x7faa80197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa7f7fe700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa8019bac0 0x7faa8019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa7effd700 1 -- 172.21.15.46:0/2002752426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa8019bac0 msgr2=0x7faa8019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa7effd700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa8019bac0 0x7faa8019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa7effd700 1 -- 172.21.15.46:0/2002752426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa80104f80 msgr2=0x7faa80197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa7effd700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa80104f80 0x7faa80197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa7effd700 1 -- 172.21.15.46:0/2002752426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa700095f0 con 0x7faa800ffe60 2024-03-18T20:46:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.116+0000 7faa7effd700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa800ffe60 0x7faa801971e0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7faa70004760 tx=0x7faa70004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.117+0000 7faa67fff700 1 -- 172.21.15.46:0/2002752426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7001d070 con 0x7faa800ffe60 2024-03-18T20:46:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.117+0000 7faa8536d700 1 -- 172.21.15.46:0/2002752426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa801a3d70 con 0x7faa800ffe60 2024-03-18T20:46:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.117+0000 7faa67fff700 1 -- 172.21.15.46:0/2002752426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa7000fb30 con 0x7faa800ffe60 2024-03-18T20:46:58.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.117+0000 7faa67fff700 1 -- 172.21.15.46:0/2002752426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70022db0 con 0x7faa800ffe60 2024-03-18T20:46:58.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.117+0000 7faa8536d700 1 -- 172.21.15.46:0/2002752426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa801a41a0 con 0x7faa800ffe60 2024-03-18T20:46:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.119+0000 7faa67fff700 1 -- 172.21.15.46:0/2002752426 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7faa7000fca0 con 0x7faa800ffe60 2024-03-18T20:46:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.120+0000 7faa67fff700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7faa680a1290 0x7faa680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.120+0000 7faa67fff700 1 -- 172.21.15.46:0/2002752426 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7faa700cdfb0 con 0x7faa800ffe60 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.120+0000 7faa7e7fc700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7faa680a1290 0x7faa680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.120+0000 7faa7e7fc700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7faa680a1290 0x7faa680a3750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faa80198730 tx=0x7faa7400b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.120+0000 7faa91502b80 1 Processor -- start 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.120+0000 7faa91502b80 1 -- start start 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa91502b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x561218009e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa91502b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121800a380 0x5612180190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa91502b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561218019630 0x56121801ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa91502b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561217f63990 con 0x561218009a60 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa91502b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561217f61350 con 0x56121800a380 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa91502b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561217f5cc40 con 0x561218019630 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x561218009e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x561218009e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51872/0 (socket says 172.21.15.46:51872) 2024-03-18T20:46:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa7effd700 1 -- 172.21.15.46:0/1792252555 learned_addr learned my addr 172.21.15.46:0/1792252555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa7f7fe700 1 --2- 172.21.15.46:0/1792252555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561218019630 0x56121801ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa7e7fc700 1 --2- 172.21.15.46:0/1792252555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121800a380 0x5612180190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa7effd700 1 -- 172.21.15.46:0/1792252555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561218019630 msgr2=0x56121801ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa7effd700 1 --2- 172.21.15.46:0/1792252555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561218019630 0x56121801ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa7effd700 1 -- 172.21.15.46:0/1792252555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121800a380 msgr2=0x5612180190f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa7effd700 1 --2- 172.21.15.46:0/1792252555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121800a380 0x5612180190f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa7effd700 1 -- 172.21.15.46:0/1792252555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561217dde3b0 con 0x561218009a60 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.121+0000 7faa7effd700 1 --2- 172.21.15.46:0/1792252555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x561218009e40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7faa700976f0 tx=0x7faa7009abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19a8eba8dba087d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.122+0000 7faa657fa700 1 -- 172.21.15.46:0/1792252555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7009b1f0 con 0x561218009a60 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.122+0000 7faa657fa700 1 -- 172.21.15.46:0/1792252555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa7009f020 con 0x561218009a60 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.122+0000 7faa657fa700 1 -- 172.21.15.46:0/1792252555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7009b720 con 0x561218009a60 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.122+0000 7faa91502b80 1 -- 172.21.15.46:0/1792252555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 msgr2=0x561218009e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.122+0000 7faa91502b80 1 --2- 172.21.15.46:0/1792252555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x561218009e40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7faa700976f0 tx=0x7faa7009abd0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.122+0000 7faa91502b80 1 -- 172.21.15.46:0/1792252555 shutdown_connections 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.122+0000 7faa91502b80 1 --2- 172.21.15.46:0/1792252555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x561218009e40 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.122+0000 7faa91502b80 1 --2- 172.21.15.46:0/1792252555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561218019630 0x56121801ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.122+0000 7faa91502b80 1 --2- 172.21.15.46:0/1792252555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121800a380 0x5612180190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.122+0000 7faa91502b80 1 -- 172.21.15.46:0/1792252555 >> 172.21.15.46:0/1792252555 conn(0x561217f80220 msgr2=0x561218006020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa91502b80 1 -- 172.21.15.46:0/1792252555 shutdown_connections 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa91502b80 1 -- 172.21.15.46:0/1792252555 wait complete. 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa91502b80 1 -- start start 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa91502b80 1 Processor -- start 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa91502b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x561218019a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa91502b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 0x561218019f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa91502b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x561218024a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa91502b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561217ef8d80 con 0x561218009a60 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa91502b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561217ef8a20 con 0x561218019630 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa91502b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561217de06a0 con 0x56121800a380 2024-03-18T20:46:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa91502b80 10 client.0 _ll_register_callbacks cb 0x561217d643a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x561218019a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x561218019a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51880/0 (socket says 172.21.15.46:51880) 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.123+0000 7faa7effd700 1 -- 172.21.15.46:0/2021999048 learned_addr learned my addr 172.21.15.46:0/2021999048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa7e7fc700 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 0x561218019f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa7f7fe700 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x561218024a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa7effd700 1 -- 172.21.15.46:0/2021999048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 msgr2=0x561218019f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa7effd700 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 0x561218019f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa7effd700 1 -- 172.21.15.46:0/2021999048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 msgr2=0x561218024a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa7effd700 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x561218024a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa7effd700 1 -- 172.21.15.46:0/2021999048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa801a41a0 con 0x561218009a60 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa7f7fe700 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x561218024a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa7effd700 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x561218019a10 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7faa700cbc00 tx=0x7faa70005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932fc5f9fe208ac9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa62ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:58.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa62ff5700 1 -- 172.21.15.46:0/2021999048 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70099330 con 0x561218009a60 2024-03-18T20:46:58.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa62ff5700 1 -- 172.21.15.46:0/2021999048 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7faa7009b1f0 con 0x561218009a60 2024-03-18T20:46:58.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa91502b80 1 -- 172.21.15.46:0/2021999048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561217ffdad0 con 0x561218009a60 2024-03-18T20:46:58.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa62ff5700 1 -- 172.21.15.46:0/2021999048 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa7009a8f0 con 0x561218009a60 2024-03-18T20:46:58.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa91502b80 10 client.19332 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:58.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.124+0000 7faa91502b80 1 -- 172.21.15.46:0/2021999048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561217d8ab60 con 0x561218009a60 2024-03-18T20:46:58.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.125+0000 7faa62ff5700 1 -- 172.21.15.46:0/2021999048 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561217ffdad0 con 0x561218009a60 2024-03-18T20:46:58.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.125+0000 7faa62ff5700 1 -- 172.21.15.46:0/2021999048 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7faa700b9200 con 0x561218009a60 2024-03-18T20:46:58.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.125+0000 7faa91502b80 10 client.19332 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:58.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.125+0000 7faa91502b80 10 client.19332 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:46:58.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.125+0000 7faa91502b80 20 client.19332 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:58.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.125+0000 7faa91502b80 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x561218028920 0x56121802ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.125+0000 7faa91502b80 4 client.19332 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:58.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.125+0000 7faa91502b80 1 -- 172.21.15.46:0/2021999048 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561217f55700 con 0x561218028920 2024-03-18T20:46:58.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.125+0000 7faa7e7fc700 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x561218028920 0x56121802ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa7e7fc700 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x561218028920 0x56121802ad40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c4045c736494ebcb server_cookie=8a67e41c2bc49d9b in_seq=0 out_seq=0 2024-03-18T20:46:58.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa62ff5700 10 client.19332 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:46:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa62ff5700 1 -- 172.21.15.46:0/2021999048 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561217f55700 con 0x561218028920 2024-03-18T20:46:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa62ff5700 10 client.19332 handle_command_reply: tid=0 2024-03-18T20:46:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa91502b80 1 client.19332 shutdown 2024-03-18T20:46:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa91502b80 1 -- 172.21.15.46:0/2021999048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 msgr2=0x561218019a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa91502b80 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x561218019a10 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7faa700cbc00 tx=0x7faa70005980 comp rx=0 tx=0).stop 2024-03-18T20:46:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa91502b80 1 -- 172.21.15.46:0/2021999048 shutdown_connections 2024-03-18T20:46:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa91502b80 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x561218028920 0x56121802ad40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa91502b80 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x561218019a10 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa91502b80 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x561218024a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa62ff5700 0 client.19332 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:46:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa91502b80 1 --2- 172.21.15.46:0/2021999048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 0x561218019f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa91502b80 1 -- 172.21.15.46:0/2021999048 >> 172.21.15.46:0/2021999048 conn(0x561217f80220 msgr2=0x561218005c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa91502b80 1 -- 172.21.15.46:0/2021999048 shutdown_connections 2024-03-18T20:46:58.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.126+0000 7faa91502b80 1 -- 172.21.15.46:0/2021999048 wait complete. 2024-03-18T20:46:58.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.127+0000 7faa91502b80 20 client.19332 trim_cache size 0 max 16384 2024-03-18T20:46:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa91502b80 1 Processor -- start 2024-03-18T20:46:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa91502b80 1 -- start start 2024-03-18T20:46:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa91502b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x56121802ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa91502b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 0x56121801e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa91502b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x561218004750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa91502b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121801ee40 con 0x561218009a60 2024-03-18T20:46:58.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa91502b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561218016db0 con 0x561218019630 2024-03-18T20:46:58.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa91502b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612180146a0 con 0x56121800a380 2024-03-18T20:46:58.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x56121802ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x56121802ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51886/0 (socket says 172.21.15.46:51886) 2024-03-18T20:46:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa7effd700 1 -- 172.21.15.46:0/3978971955 learned_addr learned my addr 172.21.15.46:0/3978971955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa7e7fc700 1 --2- 172.21.15.46:0/3978971955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 0x56121801e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.141+0000 7faa7f7fe700 1 --2- 172.21.15.46:0/3978971955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x561218004750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.142+0000 7faa7effd700 1 -- 172.21.15.46:0/3978971955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 msgr2=0x56121801e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.142+0000 7faa7effd700 1 --2- 172.21.15.46:0/3978971955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 0x56121801e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.142+0000 7faa7effd700 1 -- 172.21.15.46:0/3978971955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 msgr2=0x561218004750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.142+0000 7faa7effd700 1 --2- 172.21.15.46:0/3978971955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x561218004750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.142+0000 7faa7effd700 1 -- 172.21.15.46:0/3978971955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561217f22ea0 con 0x561218009a60 2024-03-18T20:46:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.142+0000 7faa7effd700 1 --2- 172.21.15.46:0/3978971955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x56121802ad10 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7faa7000ff50 tx=0x7faa70009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea17e224e3a40ff4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.142+0000 7faa65ffb700 1 -- 172.21.15.46:0/3978971955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70099330 con 0x561218009a60 2024-03-18T20:46:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.142+0000 7faa65ffb700 1 -- 172.21.15.46:0/3978971955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa7009b1f0 con 0x561218009a60 2024-03-18T20:46:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.142+0000 7faa65ffb700 1 -- 172.21.15.46:0/3978971955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa700a3150 con 0x561218009a60 2024-03-18T20:46:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 -- 172.21.15.46:0/3978971955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 msgr2=0x56121802ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 --2- 172.21.15.46:0/3978971955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x56121802ad10 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7faa7000ff50 tx=0x7faa70009310 comp rx=0 tx=0).stop 2024-03-18T20:46:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 -- 172.21.15.46:0/3978971955 shutdown_connections 2024-03-18T20:46:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 --2- 172.21.15.46:0/3978971955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x56121802ad10 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 --2- 172.21.15.46:0/3978971955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x561218004750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 --2- 172.21.15.46:0/3978971955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 0x56121801e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 -- 172.21.15.46:0/3978971955 >> 172.21.15.46:0/3978971955 conn(0x561217f80220 msgr2=0x561218013030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 -- 172.21.15.46:0/3978971955 shutdown_connections 2024-03-18T20:46:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 -- 172.21.15.46:0/3978971955 wait complete. 2024-03-18T20:46:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 -- start start 2024-03-18T20:46:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 Processor -- start 2024-03-18T20:46:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x56121801c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 0x56121801c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x56121807b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121801d010 con 0x561218009a60 2024-03-18T20:46:58.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121801d180 con 0x561218019630 2024-03-18T20:46:58.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.143+0000 7faa91502b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121807bc90 con 0x56121800a380 2024-03-18T20:46:58.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa91502b80 10 client.0 _ll_register_callbacks cb 0x7faa78009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:58.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x56121801c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x56121801c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51898/0 (socket says 172.21.15.46:51898) 2024-03-18T20:46:58.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa7effd700 1 -- 172.21.15.46:0/3808645994 learned_addr learned my addr 172.21.15.46:0/3808645994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa7f7fe700 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x56121807b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa7e7fc700 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 0x56121801c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa7effd700 1 -- 172.21.15.46:0/3808645994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 msgr2=0x56121801c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa7effd700 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 0x56121801c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa7effd700 1 -- 172.21.15.46:0/3808645994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 msgr2=0x56121807b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa7effd700 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x56121807b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa7effd700 1 -- 172.21.15.46:0/3808645994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561217d8ab60 con 0x561218009a60 2024-03-18T20:46:58.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa7f7fe700 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x56121807b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:58.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa7effd700 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x56121801c360 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7faa700b95e0 tx=0x7faa700997a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f81a4e9f2ddcb9c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa62ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:58.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa62ff5700 1 -- 172.21.15.46:0/3808645994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70099330 con 0x561218009a60 2024-03-18T20:46:58.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa62ff5700 1 -- 172.21.15.46:0/3808645994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7faa7009a8f0 con 0x561218009a60 2024-03-18T20:46:58.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.144+0000 7faa91502b80 1 -- 172.21.15.46:0/3808645994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa7009f020 con 0x561218009a60 2024-03-18T20:46:58.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.145+0000 7faa62ff5700 1 -- 172.21.15.46:0/3808645994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa70097310 con 0x561218009a60 2024-03-18T20:46:58.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.145+0000 7faa91502b80 10 client.19338 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:58.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.145+0000 7faa91502b80 1 -- 172.21.15.46:0/3808645994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x561217de2cd0 con 0x561218009a60 2024-03-18T20:46:58.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.145+0000 7faa62ff5700 1 -- 172.21.15.46:0/3808645994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa7009f020 con 0x561218009a60 2024-03-18T20:46:58.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.145+0000 7faa62ff5700 1 -- 172.21.15.46:0/3808645994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7faa7009b570 con 0x561218009a60 2024-03-18T20:46:58.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.145+0000 7faa91502b80 10 client.19338 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:58.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.145+0000 7faa91502b80 10 client.19338 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:46:58.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.145+0000 7faa91502b80 20 client.19338 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:58.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.145+0000 7faa91502b80 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56121807cf50 0x56121807f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.145+0000 7faa91502b80 4 client.19338 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:46:58.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.145+0000 7faa91502b80 1 -- 172.21.15.46:0/3808645994 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561217dabcf0 con 0x56121807cf50 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.145+0000 7faa7e7fc700 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56121807cf50 0x56121807f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa7e7fc700 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56121807cf50 0x56121807f340 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f52d9229b0fb8bc5 server_cookie=6a6e8f36d7404051 in_seq=0 out_seq=0 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa62ff5700 10 client.19338 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa62ff5700 1 -- 172.21.15.46:0/3808645994 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561217dabcf0 con 0x56121807cf50 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa62ff5700 10 client.19338 handle_command_reply: tid=0 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa91502b80 1 client.19338 shutdown 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa91502b80 1 -- 172.21.15.46:0/3808645994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 msgr2=0x56121801c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa91502b80 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x56121801c360 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7faa700b95e0 tx=0x7faa700997a0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa91502b80 1 -- 172.21.15.46:0/3808645994 shutdown_connections 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa91502b80 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x56121807cf50 0x56121807f340 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa91502b80 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561218009a60 0x56121801c360 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa62ff5700 0 client.19338 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:46:58.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa91502b80 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561218019630 0x56121807b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa91502b80 1 --2- 172.21.15.46:0/3808645994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56121800a380 0x56121801c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa91502b80 1 -- 172.21.15.46:0/3808645994 >> 172.21.15.46:0/3808645994 conn(0x561217f80220 msgr2=0x561218013b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa91502b80 1 -- 172.21.15.46:0/3808645994 shutdown_connections 2024-03-18T20:46:58.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa91502b80 1 -- 172.21.15.46:0/3808645994 wait complete. 2024-03-18T20:46:58.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.146+0000 7faa91502b80 20 client.19338 trim_cache size 0 max 16384 2024-03-18T20:46:58.165 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:46:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.147+0000 7faa8536d700 1 -- 172.21.15.46:0/2002752426 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7faa680a1290 msgr2=0x7faa680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.147+0000 7faa8536d700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7faa680a1290 0x7faa680a3750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faa80198730 tx=0x7faa7400b410 comp rx=0 tx=0).stop 2024-03-18T20:46:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.147+0000 7faa8536d700 1 -- 172.21.15.46:0/2002752426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa800ffe60 msgr2=0x7faa801971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.147+0000 7faa8536d700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa800ffe60 0x7faa801971e0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7faa70004760 tx=0x7faa70004790 comp rx=0 tx=0).stop 2024-03-18T20:46:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.147+0000 7faa8536d700 1 -- 172.21.15.46:0/2002752426 shutdown_connections 2024-03-18T20:46:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.147+0000 7faa8536d700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7faa680a1290 0x7faa680a3750 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.147+0000 7faa8536d700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa800ffe60 0x7faa801971e0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.147+0000 7faa8536d700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faa8019bac0 0x7faa8019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.147+0000 7faa8536d700 1 --2- 172.21.15.46:0/2002752426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa80104f80 0x7faa80197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.147+0000 7faa8536d700 1 -- 172.21.15.46:0/2002752426 >> 172.21.15.46:0/2002752426 conn(0x7faa800fb6c0 msgr2=0x7faa800696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.147+0000 7faa8536d700 1 -- 172.21.15.46:0/2002752426 shutdown_connections 2024-03-18T20:46:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.147+0000 7faa8536d700 1 -- 172.21.15.46:0/2002752426 wait complete. 2024-03-18T20:46:58.168 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:46:58.168 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:46:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.307+0000 7fd5e7a40700 1 -- 172.21.15.46:0/2781949428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e010d750 msgr2=0x7fd5e010fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.307+0000 7fd5e7a40700 1 --2- 172.21.15.46:0/2781949428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e010d750 0x7fd5e010fb40 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800b0a0 tx=0x7fd5d800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.307+0000 7fd5e7a40700 1 -- 172.21.15.46:0/2781949428 shutdown_connections 2024-03-18T20:46:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.307+0000 7fd5e7a40700 1 --2- 172.21.15.46:0/2781949428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e010d750 0x7fd5e010fb40 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.307+0000 7fd5e7a40700 1 --2- 172.21.15.46:0/2781949428 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e0100710 0x7fd5e0100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.307+0000 7fd5e7a40700 1 --2- 172.21.15.46:0/2781949428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5e0101030 0x7fd5e010d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.307+0000 7fd5e7a40700 1 -- 172.21.15.46:0/2781949428 >> 172.21.15.46:0/2781949428 conn(0x7fd5e0076d40 msgr2=0x7fd5e0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.307+0000 7fd5e7a40700 1 -- 172.21.15.46:0/2781949428 shutdown_connections 2024-03-18T20:46:58.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.307+0000 7fd5e7a40700 1 -- 172.21.15.46:0/2781949428 wait complete. 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e7a40700 1 Processor -- start 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e7a40700 1 -- start start 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e7a40700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0100710 0x7fd5e01995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e7a40700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e0101030 0x7fd5e0199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e7a40700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5e010d750 0x7fd5e019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e7a40700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e019e4d0 con 0x7fd5e0100710 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e7a40700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e019e640 con 0x7fd5e010d750 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e7a40700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e019e800 con 0x7fd5e0101030 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e57dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0100710 0x7fd5e01995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e57dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0100710 0x7fd5e01995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51928/0 (socket says 172.21.15.46:51928) 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e57dc700 1 -- 172.21.15.46:0/1732943134 learned_addr learned my addr 172.21.15.46:0/1732943134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e4fdb700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e0101030 0x7fd5e0199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.308+0000 7fd5e5fdd700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5e010d750 0x7fd5e019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.309+0000 7fd5e57dc700 1 -- 172.21.15.46:0/1732943134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e0101030 msgr2=0x7fd5e0199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.309+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e0101030 0x7fd5e0199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.309+0000 7fd5e57dc700 1 -- 172.21.15.46:0/1732943134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5e010d750 msgr2=0x7fd5e019de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.309+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5e010d750 0x7fd5e019de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.309+0000 7fd5e57dc700 1 -- 172.21.15.46:0/1732943134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d8009d00 con 0x7fd5e0100710 2024-03-18T20:46:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.309+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0100710 0x7fd5e01995b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00e9a0 tx=0x7fd5dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.309+0000 7fd5d67fc700 1 -- 172.21.15.46:0/1732943134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc009790 con 0x7fd5e0100710 2024-03-18T20:46:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.309+0000 7fd5d67fc700 1 -- 172.21.15.46:0/1732943134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5dc010430 con 0x7fd5e0100710 2024-03-18T20:46:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.309+0000 7fd5e7a40700 1 -- 172.21.15.46:0/1732943134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e019eae0 con 0x7fd5e0100710 2024-03-18T20:46:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.309+0000 7fd5d67fc700 1 -- 172.21.15.46:0/1732943134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc018680 con 0x7fd5e0100710 2024-03-18T20:46:58.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.309+0000 7fd5e7a40700 1 -- 172.21.15.46:0/1732943134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e01a6320 con 0x7fd5e0100710 2024-03-18T20:46:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.312+0000 7fd5d67fc700 1 -- 172.21.15.46:0/1732943134 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd5dc0105a0 con 0x7fd5e0100710 2024-03-18T20:46:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.313+0000 7fd5d67fc700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd5cc0a1290 0x7fd5cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.313+0000 7fd5d67fc700 1 -- 172.21.15.46:0/1732943134 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd5dc014070 con 0x7fd5e0100710 2024-03-18T20:46:58.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.313+0000 7fd5e4fdb700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd5cc0a1290 0x7fd5cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.313+0000 7fd5e4fdb700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd5cc0a1290 0x7fd5cc0a3750 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd5e019ab00 tx=0x7fd5d0009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.313+0000 7fd5f3bd5b80 1 Processor -- start 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5f3bd5b80 1 -- start start 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5f3bd5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688149a60 0x556688149e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5f3bd5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55668814a380 0x556688154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5f3bd5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556688155220 0x556688157610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5f3bd5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566880a3990 con 0x55668814a380 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5f3bd5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566880a1350 con 0x556688149a60 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5f3bd5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668809cc40 con 0x556688155220 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5e4fdb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55668814a380 0x556688154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5e4fdb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55668814a380 0x556688154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51940/0 (socket says 172.21.15.46:51940) 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5e4fdb700 1 -- 172.21.15.46:0/1451250054 learned_addr learned my addr 172.21.15.46:0/1451250054 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/1451250054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688149a60 0x556688149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5e5fdd700 1 --2- 172.21.15.46:0/1451250054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556688155220 0x556688157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5e4fdb700 1 -- 172.21.15.46:0/1451250054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556688155220 msgr2=0x556688157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5e4fdb700 1 --2- 172.21.15.46:0/1451250054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556688155220 0x556688157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5e4fdb700 1 -- 172.21.15.46:0/1451250054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688149a60 msgr2=0x556688149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5e4fdb700 1 --2- 172.21.15.46:0/1451250054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688149a60 0x556688149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5e4fdb700 1 -- 172.21.15.46:0/1451250054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556687f1e3b0 con 0x55668814a380 2024-03-18T20:46:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.314+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/1451250054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688149a60 0x556688149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.315+0000 7fd5e4fdb700 1 --2- 172.21.15.46:0/1451250054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55668814a380 0x556688154ce0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0011d50 tx=0x7fd5d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c24991c1d16db9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.315+0000 7fd5cb7fe700 1 -- 172.21.15.46:0/1451250054 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000eb20 con 0x55668814a380 2024-03-18T20:46:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.315+0000 7fd5cb7fe700 1 -- 172.21.15.46:0/1451250054 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5d0021070 con 0x55668814a380 2024-03-18T20:46:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.315+0000 7fd5cb7fe700 1 -- 172.21.15.46:0/1451250054 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d001d410 con 0x55668814a380 2024-03-18T20:46:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.316+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/1451250054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55668814a380 msgr2=0x556688154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.316+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/1451250054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55668814a380 0x556688154ce0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0011d50 tx=0x7fd5d0014000 comp rx=0 tx=0).stop 2024-03-18T20:46:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.316+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/1451250054 shutdown_connections 2024-03-18T20:46:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.316+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/1451250054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556688155220 0x556688157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.316+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/1451250054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688149a60 0x556688149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.316+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/1451250054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55668814a380 0x556688154ce0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.316+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/1451250054 >> 172.21.15.46:0/1451250054 conn(0x5566880c0220 msgr2=0x556688146020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.316+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/1451250054 shutdown_connections 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.316+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/1451250054 wait complete. 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.316+0000 7fd5f3bd5b80 1 -- start start 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.316+0000 7fd5f3bd5b80 1 Processor -- start 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5f3bd5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x556688154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5f3bd5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55668814a380 0x556688151f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5f3bd5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688155220 0x556688152600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5f3bd5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556688038d80 con 0x556688149a60 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5f3bd5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556688038a20 con 0x556688155220 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5f3bd5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556687f206a0 con 0x55668814a380 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5f3bd5b80 10 client.0 _ll_register_callbacks cb 0x556687ea43a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5e57dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x556688154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5e57dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x556688154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51944/0 (socket says 172.21.15.46:51944) 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5e57dc700 1 -- 172.21.15.46:0/2372104934 learned_addr learned my addr 172.21.15.46:0/2372104934 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5e4fdb700 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55668814a380 0x556688151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5e5fdd700 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688155220 0x556688152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5e57dc700 1 -- 172.21.15.46:0/2372104934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55668814a380 msgr2=0x556688151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55668814a380 0x556688151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5e57dc700 1 -- 172.21.15.46:0/2372104934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688155220 msgr2=0x556688152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688155220 0x556688152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5e57dc700 1 -- 172.21.15.46:0/2372104934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e01a6320 con 0x556688149a60 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.317+0000 7fd5e5fdd700 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688155220 0x556688152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x556688154bf0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc096a20 tx=0x7fd5dc099fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbdb6dda16b8ceb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5c8ff9700 1 -- 172.21.15.46:0/2372104934 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0996a0 con 0x556688149a60 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5c8ff9700 1 -- 172.21.15.46:0/2372104934 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd5dc0a2430 con 0x556688149a60 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2372104934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55668813dad0 con 0x556688149a60 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5c8ff9700 1 -- 172.21.15.46:0/2372104934 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5dc0a1fa0 con 0x556688149a60 2024-03-18T20:46:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5f3bd5b80 10 client.19350 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2372104934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7fd5d0017070 con 0x556688149a60 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5c8ff9700 1 -- 172.21.15.46:0/2372104934 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55668813dad0 con 0x556688149a60 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5c8ff9700 1 -- 172.21.15.46:0/2372104934 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd5dc098f10 con 0x556688149a60 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5f3bd5b80 10 client.19350 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5f3bd5b80 10 client.19350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5f3bd5b80 20 client.19350 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x556688164d70 0x556688167160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5f3bd5b80 4 client.19350 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.318+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2372104934 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556688095700 con 0x556688164d70 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.319+0000 7fd5e4fdb700 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x556688164d70 0x556688167160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.319+0000 7fd5c8ff9700 10 client.19350 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.319+0000 7fd5e4fdb700 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x556688164d70 0x556688167160 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9cf10e06b81324d server_cookie=313ef79974e7d4af in_seq=0 out_seq=0 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5c8ff9700 1 -- 172.21.15.46:0/2372104934 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556688095700 con 0x556688164d70 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5c8ff9700 10 client.19350 handle_command_reply: tid=0 2024-03-18T20:46:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5f3bd5b80 1 client.19350 shutdown 2024-03-18T20:46:58.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2372104934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 msgr2=0x556688154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x556688154bf0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc096a20 tx=0x7fd5dc099fb0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2372104934 shutdown_connections 2024-03-18T20:46:58.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x556688164d70 0x556688167160 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x556688154bf0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5c8ff9700 0 client.19350 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:46:58.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688155220 0x556688152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/2372104934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55668814a380 0x556688151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2372104934 >> 172.21.15.46:0/2372104934 conn(0x5566880c0220 msgr2=0x556688145c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2372104934 shutdown_connections 2024-03-18T20:46:58.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2372104934 wait complete. 2024-03-18T20:46:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.320+0000 7fd5f3bd5b80 20 client.19350 trim_cache size 0 max 16384 2024-03-18T20:46:58.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.334+0000 7fd5f3bd5b80 1 Processor -- start 2024-03-18T20:46:58.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.334+0000 7fd5f3bd5b80 1 -- start start 2024-03-18T20:46:58.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.334+0000 7fd5f3bd5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x5566881605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.334+0000 7fd5f3bd5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668814a380 0x556688160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.334+0000 7fd5f3bd5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556688155220 0x55668814f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.334+0000 7fd5f3bd5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556688147230 con 0x556688149a60 2024-03-18T20:46:58.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.334+0000 7fd5f3bd5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668814ad30 con 0x55668814a380 2024-03-18T20:46:58.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.334+0000 7fd5f3bd5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55668814a1c0 con 0x556688155220 2024-03-18T20:46:58.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.334+0000 7fd5e57dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x5566881605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5e57dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x5566881605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51960/0 (socket says 172.21.15.46:51960) 2024-03-18T20:46:58.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5e57dc700 1 -- 172.21.15.46:0/2020039703 learned_addr learned my addr 172.21.15.46:0/2020039703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5e4fdb700 1 --2- 172.21.15.46:0/2020039703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668814a380 0x556688160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5e5fdd700 1 --2- 172.21.15.46:0/2020039703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556688155220 0x55668814f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5e57dc700 1 -- 172.21.15.46:0/2020039703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556688155220 msgr2=0x55668814f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/2020039703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556688155220 0x55668814f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5e57dc700 1 -- 172.21.15.46:0/2020039703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668814a380 msgr2=0x556688160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/2020039703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668814a380 0x556688160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5e57dc700 1 -- 172.21.15.46:0/2020039703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556687ecab60 con 0x556688149a60 2024-03-18T20:46:58.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5e5fdd700 1 --2- 172.21.15.46:0/2020039703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556688155220 0x55668814f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:46:58.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/2020039703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x5566881605a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc098300 tx=0x7fd5dc010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a3f6c812b065ef server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5cbfff700 1 -- 172.21.15.46:0/2020039703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0a2430 con 0x556688149a60 2024-03-18T20:46:58.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5cbfff700 1 -- 172.21.15.46:0/2020039703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5dc0a1fa0 con 0x556688149a60 2024-03-18T20:46:58.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.335+0000 7fd5cbfff700 1 -- 172.21.15.46:0/2020039703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0b2dc0 con 0x556688149a60 2024-03-18T20:46:58.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.336+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2020039703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 msgr2=0x5566881605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.336+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/2020039703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x5566881605a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc098300 tx=0x7fd5dc010b80 comp rx=0 tx=0).stop 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.336+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2020039703 shutdown_connections 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.336+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/2020039703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x5566881605a0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.336+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/2020039703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556688155220 0x55668814f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.336+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/2020039703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55668814a380 0x556688160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.336+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2020039703 >> 172.21.15.46:0/2020039703 conn(0x5566880c0220 msgr2=0x556688150850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.336+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2020039703 shutdown_connections 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.336+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/2020039703 wait complete. 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.337+0000 7fd5f3bd5b80 1 -- start start 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.337+0000 7fd5f3bd5b80 1 Processor -- start 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5f3bd5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x556688159790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5f3bd5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55668814a380 0x556688159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5f3bd5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688155220 0x55668815a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5f3bd5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566881b7cf0 con 0x556688149a60 2024-03-18T20:46:58.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5f3bd5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566881b7e60 con 0x556688155220 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5f3bd5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566881b8020 con 0x55668814a380 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5f3bd5b80 10 client.0 _ll_register_callbacks cb 0x7fd5dc099bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5e57dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x556688159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5e57dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x556688159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51972/0 (socket says 172.21.15.46:51972) 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5e5fdd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688155220 0x55668815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5e57dc700 1 -- 172.21.15.46:0/3074997064 learned_addr learned my addr 172.21.15.46:0/3074997064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5e4fdb700 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55668814a380 0x556688159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5e5fdd700 1 -- 172.21.15.46:0/3074997064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55668814a380 msgr2=0x556688159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5e5fdd700 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55668814a380 0x556688159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5e5fdd700 1 -- 172.21.15.46:0/3074997064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 msgr2=0x556688159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5e5fdd700 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x556688159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5e5fdd700 1 -- 172.21.15.46:0/3074997064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d0017070 con 0x556688155220 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.338+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x556688159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.339+0000 7fd5e5fdd700 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688155220 0x55668815a210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800b070 tx=0x7fd5d800bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e6de689a8bdb32b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.339+0000 7fd5c8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.339+0000 7fd5c8ff9700 1 -- 172.21.15.46:0/3074997064 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800e040 con 0x556688155220 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.339+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/3074997064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5d0021070 con 0x556688155220 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.339+0000 7fd5c8ff9700 1 -- 172.21.15.46:0/3074997064 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd5d8003a00 con 0x556688155220 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.339+0000 7fd5c8ff9700 1 -- 172.21.15.46:0/3074997064 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5d8009b80 con 0x556688155220 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.339+0000 7fd5f3bd5b80 10 client.28985 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.339+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/3074997064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x556688062ea0 con 0x556688155220 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.339+0000 7fd5c8ff9700 1 -- 172.21.15.46:0/3074997064 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5d0021070 con 0x556688155220 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.340+0000 7fd5c8ff9700 1 -- 172.21.15.46:0/3074997064 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd5d800b0d0 con 0x556688155220 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.340+0000 7fd5f3bd5b80 10 client.28985 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.340+0000 7fd5f3bd5b80 10 client.28985 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.340+0000 7fd5f3bd5b80 20 client.28985 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.340+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5566881b9730 0x5566881bbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.340+0000 7fd5f3bd5b80 4 client.28985 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:46:58.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.340+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/3074997064 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556687eebcf0 con 0x5566881b9730 2024-03-18T20:46:58.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.340+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5566881b9730 0x5566881bbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5e57dc700 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5566881b9730 0x5566881bbb50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d8751dba80358aa server_cookie=2163142fd9f8c9c6 in_seq=0 out_seq=0 2024-03-18T20:46:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5c8ff9700 10 client.28985 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:46:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5c8ff9700 1 -- 172.21.15.46:0/3074997064 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556687eebcf0 con 0x5566881b9730 2024-03-18T20:46:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5c8ff9700 10 client.28985 handle_command_reply: tid=0 2024-03-18T20:46:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5f3bd5b80 1 client.28985 shutdown 2024-03-18T20:46:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/3074997064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688155220 msgr2=0x55668815a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688155220 0x55668815a210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800b070 tx=0x7fd5d800bce0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/3074997064 shutdown_connections 2024-03-18T20:46:58.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x5566881b9730 0x5566881bbb50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556688149a60 0x556688159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556688155220 0x55668815a210 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5c8ff9700 0 client.28985 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:46:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5f3bd5b80 1 --2- 172.21.15.46:0/3074997064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55668814a380 0x556688159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/3074997064 >> 172.21.15.46:0/3074997064 conn(0x5566880c0220 msgr2=0x5566881605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/3074997064 shutdown_connections 2024-03-18T20:46:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5f3bd5b80 1 -- 172.21.15.46:0/3074997064 wait complete. 2024-03-18T20:46:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.341+0000 7fd5f3bd5b80 20 client.28985 trim_cache size 0 max 16384 2024-03-18T20:46:58.348 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:46:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.342+0000 7fd5e7a40700 1 -- 172.21.15.46:0/1732943134 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd5cc0a1290 msgr2=0x7fd5cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.342+0000 7fd5e7a40700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd5cc0a1290 0x7fd5cc0a3750 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd5e019ab00 tx=0x7fd5d0009260 comp rx=0 tx=0).stop 2024-03-18T20:46:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.342+0000 7fd5e7a40700 1 -- 172.21.15.46:0/1732943134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0100710 msgr2=0x7fd5e01995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.342+0000 7fd5e7a40700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0100710 0x7fd5e01995b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00e9a0 tx=0x7fd5dc00ecb0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.342+0000 7fd5e7a40700 1 -- 172.21.15.46:0/1732943134 shutdown_connections 2024-03-18T20:46:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.342+0000 7fd5e7a40700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd5cc0a1290 0x7fd5cc0a3750 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.342+0000 7fd5e7a40700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e0100710 0x7fd5e01995b0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.342+0000 7fd5e7a40700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5e010d750 0x7fd5e019de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.342+0000 7fd5e7a40700 1 --2- 172.21.15.46:0/1732943134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5e0101030 0x7fd5e0199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.342+0000 7fd5e7a40700 1 -- 172.21.15.46:0/1732943134 >> 172.21.15.46:0/1732943134 conn(0x7fd5e0076d40 msgr2=0x7fd5e010edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.343+0000 7fd5e7a40700 1 -- 172.21.15.46:0/1732943134 shutdown_connections 2024-03-18T20:46:58.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.343+0000 7fd5e7a40700 1 -- 172.21.15.46:0/1732943134 wait complete. 2024-03-18T20:46:58.364 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:46:58.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:46:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.501+0000 7f28c8741700 1 -- 172.21.15.46:0/3171483820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c0101900 msgr2=0x7f28c0105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.501+0000 7f28c8741700 1 --2- 172.21.15.46:0/3171483820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c0101900 0x7f28c0105940 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009990 tx=0x7f28bc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.501+0000 7f28c8741700 1 -- 172.21.15.46:0/3171483820 shutdown_connections 2024-03-18T20:46:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.501+0000 7f28c8741700 1 --2- 172.21.15.46:0/3171483820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c0106070 0x7f28c010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.501+0000 7f28c8741700 1 --2- 172.21.15.46:0/3171483820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28c0100f50 0x7f28c0101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.501+0000 7f28c8741700 1 --2- 172.21.15.46:0/3171483820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c0101900 0x7f28c0105940 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.501+0000 7f28c8741700 1 -- 172.21.15.46:0/3171483820 >> 172.21.15.46:0/3171483820 conn(0x7f28c00fc7f0 msgr2=0x7f28c00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.501+0000 7f28c8741700 1 -- 172.21.15.46:0/3171483820 shutdown_connections 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.501+0000 7f28c8741700 1 -- 172.21.15.46:0/3171483820 wait complete. 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c8741700 1 Processor -- start 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c8741700 1 -- start start 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c8741700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28c0100f50 0x7f28c0197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c8741700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c0106070 0x7f28c0197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c8741700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c019ba00 0x7f28c019be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c8741700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c019c480 con 0x7f28c019ba00 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c8741700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c019c5f0 con 0x7f28c0100f50 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c8741700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c019c7b0 con 0x7f28c0106070 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c6cde700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c019ba00 0x7f28c019be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c6cde700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c019ba00 0x7f28c019be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51986/0 (socket says 172.21.15.46:51986) 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c6cde700 1 -- 172.21.15.46:0/721073957 learned_addr learned my addr 172.21.15.46:0/721073957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c64dd700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28c0100f50 0x7f28c0197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c5cdc700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c0106070 0x7f28c0197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c6cde700 1 -- 172.21.15.46:0/721073957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c0106070 msgr2=0x7f28c0197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c6cde700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c0106070 0x7f28c0197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c6cde700 1 -- 172.21.15.46:0/721073957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28c0100f50 msgr2=0x7f28c0197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.502+0000 7f28c6cde700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28c0100f50 0x7f28c0197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.503+0000 7f28c6cde700 1 -- 172.21.15.46:0/721073957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc0095f0 con 0x7f28c019ba00 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.503+0000 7f28c5cdc700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c0106070 0x7f28c0197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.503+0000 7f28c6cde700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c019ba00 0x7f28c019be80 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f28b8007d70 tx=0x7f28b800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.503+0000 7f28b37fe700 1 -- 172.21.15.46:0/721073957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b800ad40 con 0x7f28c019ba00 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.503+0000 7f28b37fe700 1 -- 172.21.15.46:0/721073957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28b80128e0 con 0x7f28c019ba00 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.503+0000 7f28c8741700 1 -- 172.21.15.46:0/721073957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c01a3d10 con 0x7f28c019ba00 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.503+0000 7f28b37fe700 1 -- 172.21.15.46:0/721073957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80116b0 con 0x7f28c019ba00 2024-03-18T20:46:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.503+0000 7f28c8741700 1 -- 172.21.15.46:0/721073957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c01a41a0 con 0x7f28c019ba00 2024-03-18T20:46:58.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.506+0000 7f28b37fe700 1 -- 172.21.15.46:0/721073957 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f28b8012420 con 0x7f28c019ba00 2024-03-18T20:46:58.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.506+0000 7f28b37fe700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f28ac0a1290 0x7f28ac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.506+0000 7f28b37fe700 1 -- 172.21.15.46:0/721073957 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f28b80cce00 con 0x7f28c019ba00 2024-03-18T20:46:58.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.506+0000 7f28c64dd700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f28ac0a1290 0x7f28ac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.507+0000 7f28d48d6b80 1 Processor -- start 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.507+0000 7f28c64dd700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f28ac0a1290 0x7f28ac0a3750 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f28b4005950 tx=0x7f28b400b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.507+0000 7f28d48d6b80 1 -- start start 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.507+0000 7f28d48d6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e7a60 0x55f04d7e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.507+0000 7f28d48d6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 0x55f04d7f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.507+0000 7f28d48d6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7f7630 0x55f04d7f9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.507+0000 7f28d48d6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04d741990 con 0x55f04d7e7a60 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.507+0000 7f28d48d6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04d73f350 con 0x55f04d7f7630 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28d48d6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04d73ac40 con 0x55f04d7e8380 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28c64dd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e7a60 0x55f04d7e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28c64dd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e7a60 0x55f04d7e7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52000/0 (socket says 172.21.15.46:52000) 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28c64dd700 1 -- 172.21.15.46:0/360595972 learned_addr learned my addr 172.21.15.46:0/360595972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28c6cde700 1 --2- 172.21.15.46:0/360595972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7f7630 0x55f04d7f9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28c5cdc700 1 --2- 172.21.15.46:0/360595972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 0x55f04d7f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28c64dd700 1 -- 172.21.15.46:0/360595972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 msgr2=0x55f04d7f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28c64dd700 1 --2- 172.21.15.46:0/360595972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 0x55f04d7f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28c64dd700 1 -- 172.21.15.46:0/360595972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7f7630 msgr2=0x55f04d7f9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28c64dd700 1 --2- 172.21.15.46:0/360595972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7f7630 0x55f04d7f9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28c64dd700 1 -- 172.21.15.46:0/360595972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f04d5bc3b0 con 0x55f04d7e7a60 2024-03-18T20:46:58.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28c64dd700 1 --2- 172.21.15.46:0/360595972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e7a60 0x55f04d7e7e40 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f28b4010fa0 tx=0x7f28b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeac96cc4f10fcfa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28b0ff9700 1 -- 172.21.15.46:0/360595972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4011db0 con 0x55f04d7e7a60 2024-03-18T20:46:58.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28b0ff9700 1 -- 172.21.15.46:0/360595972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28b4015070 con 0x55f04d7e7a60 2024-03-18T20:46:58.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.508+0000 7f28b0ff9700 1 -- 172.21.15.46:0/360595972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b401c610 con 0x55f04d7e7a60 2024-03-18T20:46:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.509+0000 7f28d48d6b80 1 -- 172.21.15.46:0/360595972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e7a60 msgr2=0x55f04d7e7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.509+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/360595972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e7a60 0x55f04d7e7e40 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f28b4010fa0 tx=0x7f28b4013000 comp rx=0 tx=0).stop 2024-03-18T20:46:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.509+0000 7f28d48d6b80 1 -- 172.21.15.46:0/360595972 shutdown_connections 2024-03-18T20:46:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.509+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/360595972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e7a60 0x55f04d7e7e40 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.509+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/360595972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7f7630 0x55f04d7f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.509+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/360595972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 0x55f04d7f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.509+0000 7f28d48d6b80 1 -- 172.21.15.46:0/360595972 >> 172.21.15.46:0/360595972 conn(0x55f04d75e220 msgr2=0x55f04d7e4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.509+0000 7f28d48d6b80 1 -- 172.21.15.46:0/360595972 shutdown_connections 2024-03-18T20:46:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.509+0000 7f28d48d6b80 1 -- 172.21.15.46:0/360595972 wait complete. 2024-03-18T20:46:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28d48d6b80 1 -- start start 2024-03-18T20:46:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28d48d6b80 1 Processor -- start 2024-03-18T20:46:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28d48d6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e7a60 0x55f04d7f7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28d48d6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e8380 0x55f04d7f4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28d48d6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7f7630 0x55f04d7f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28d48d6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04d6d6d80 con 0x55f04d7e8380 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28d48d6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04d6d6a20 con 0x55f04d7f7630 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28d48d6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04d5be6a0 con 0x55f04d7e7a60 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28d48d6b80 10 client.0 _ll_register_callbacks cb 0x55f04d5423a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28c5cdc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e8380 0x55f04d7f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28c5cdc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e8380 0x55f04d7f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52012/0 (socket says 172.21.15.46:52012) 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28c5cdc700 1 -- 172.21.15.46:0/105737724 learned_addr learned my addr 172.21.15.46:0/105737724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28c64dd700 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e7a60 0x55f04d7f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.510+0000 7f28c6cde700 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7f7630 0x55f04d7f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28c5cdc700 1 -- 172.21.15.46:0/105737724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e7a60 msgr2=0x55f04d7f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28c5cdc700 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e7a60 0x55f04d7f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28c5cdc700 1 -- 172.21.15.46:0/105737724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7f7630 msgr2=0x55f04d7f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28c5cdc700 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7f7630 0x55f04d7f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28c5cdc700 1 -- 172.21.15.46:0/105737724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b4010c50 con 0x55f04d7e8380 2024-03-18T20:46:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28c64dd700 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e7a60 0x55f04d7f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28c6cde700 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7f7630 0x55f04d7f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28c5cdc700 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e8380 0x55f04d7f4390 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f28c00679c0 tx=0x7f28bc005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed513cf92bca269 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28a9ffb700 1 -- 172.21.15.46:0/105737724 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc00bb80 con 0x55f04d7e8380 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28a9ffb700 1 -- 172.21.15.46:0/105737724 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f28bc00bce0 con 0x55f04d7e8380 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28d48d6b80 1 -- 172.21.15.46:0/105737724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f04d7dbad0 con 0x55f04d7e8380 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28a9ffb700 1 -- 172.21.15.46:0/105737724 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28bc00bea0 con 0x55f04d7e8380 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28d48d6b80 10 client.19368 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.511+0000 7f28d48d6b80 1 -- 172.21.15.46:0/105737724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55f04d568b60 con 0x55f04d7e8380 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.512+0000 7f28a9ffb700 1 -- 172.21.15.46:0/105737724 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28bc031020 con 0x55f04d7e8380 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.512+0000 7f28a9ffb700 1 -- 172.21.15.46:0/105737724 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f28bc00f730 con 0x55f04d7e8380 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.512+0000 7f28d48d6b80 10 client.19368 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.512+0000 7f28d48d6b80 10 client.19368 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:46:58.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.512+0000 7f28d48d6b80 20 client.19368 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.512+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f04d8071f0 0x55f04d809610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.512+0000 7f28d48d6b80 4 client.19368 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.512+0000 7f28d48d6b80 1 -- 172.21.15.46:0/105737724 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f04d733700 con 0x55f04d8071f0 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.512+0000 7f28c64dd700 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f04d8071f0 0x55f04d809610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.512+0000 7f28a9ffb700 10 client.19368 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.512+0000 7f28c64dd700 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f04d8071f0 0x55f04d809610 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=689ae5ea006ee72f server_cookie=ec66fdf489bd4acd in_seq=0 out_seq=0 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28a9ffb700 1 -- 172.21.15.46:0/105737724 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f04d733700 con 0x55f04d8071f0 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28a9ffb700 10 client.19368 handle_command_reply: tid=0 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28d48d6b80 1 client.19368 shutdown 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28d48d6b80 1 -- 172.21.15.46:0/105737724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e8380 msgr2=0x55f04d7f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e8380 0x55f04d7f4390 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f28c00679c0 tx=0x7f28bc005750 comp rx=0 tx=0).stop 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28d48d6b80 1 -- 172.21.15.46:0/105737724 shutdown_connections 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f04d8071f0 0x55f04d809610 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7f7630 0x55f04d7f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e7a60 0x55f04d7f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/105737724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7e8380 0x55f04d7f4390 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28a9ffb700 0 client.19368 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:46:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28d48d6b80 1 -- 172.21.15.46:0/105737724 >> 172.21.15.46:0/105737724 conn(0x55f04d75e220 msgr2=0x55f04d7e3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28d48d6b80 1 -- 172.21.15.46:0/105737724 shutdown_connections 2024-03-18T20:46:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28d48d6b80 1 -- 172.21.15.46:0/105737724 wait complete. 2024-03-18T20:46:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.513+0000 7f28d48d6b80 20 client.19368 trim_cache size 0 max 16384 2024-03-18T20:46:58.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.527+0000 7f28d48d6b80 1 Processor -- start 2024-03-18T20:46:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.527+0000 7f28d48d6b80 1 -- start start 2024-03-18T20:46:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.527+0000 7f28d48d6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7e7a60 0x55f04d7fe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.527+0000 7f28d48d6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 0x55f04d7f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28d48d6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 0x55f04d7f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28d48d6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04d7de210 con 0x55f04d7f7630 2024-03-18T20:46:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28d48d6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04d7e8a30 con 0x55f04d7e7a60 2024-03-18T20:46:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28d48d6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04d7e9620 con 0x55f04d7e8380 2024-03-18T20:46:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28c6cde700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 0x55f04d7f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28c64dd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7e7a60 0x55f04d7fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28c6cde700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 0x55f04d7f70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52022/0 (socket says 172.21.15.46:52022) 2024-03-18T20:46:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28c6cde700 1 -- 172.21.15.46:0/3076732009 learned_addr learned my addr 172.21.15.46:0/3076732009 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28c5cdc700 1 --2- 172.21.15.46:0/3076732009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 0x55f04d7f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28c6cde700 1 -- 172.21.15.46:0/3076732009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 msgr2=0x55f04d7f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28c6cde700 1 --2- 172.21.15.46:0/3076732009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 0x55f04d7f6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28c6cde700 1 -- 172.21.15.46:0/3076732009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7e7a60 msgr2=0x55f04d7fe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28c6cde700 1 --2- 172.21.15.46:0/3076732009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7e7a60 0x55f04d7fe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28c6cde700 1 -- 172.21.15.46:0/3076732009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f04d803680 con 0x55f04d7f7630 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28c64dd700 1 --2- 172.21.15.46:0/3076732009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7e7a60 0x55f04d7fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28c6cde700 1 --2- 172.21.15.46:0/3076732009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 0x55f04d7f70e0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f28b8099d00 tx=0x7f28b8096470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed0cf7502859a8d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28b17fa700 1 -- 172.21.15.46:0/3076732009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b809de30 con 0x55f04d7f7630 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28b17fa700 1 -- 172.21.15.46:0/3076732009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28b80a2480 con 0x55f04d7f7630 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.528+0000 7f28b17fa700 1 -- 172.21.15.46:0/3076732009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b809a260 con 0x55f04d7f7630 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.529+0000 7f28d48d6b80 1 -- 172.21.15.46:0/3076732009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 msgr2=0x55f04d7f70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.529+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/3076732009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 0x55f04d7f70e0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f28b8099d00 tx=0x7f28b8096470 comp rx=0 tx=0).stop 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.529+0000 7f28d48d6b80 1 -- 172.21.15.46:0/3076732009 shutdown_connections 2024-03-18T20:46:58.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.529+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/3076732009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 0x55f04d7f70e0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.529+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/3076732009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7e7a60 0x55f04d7fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.529+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/3076732009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 0x55f04d7f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.529+0000 7f28d48d6b80 1 -- 172.21.15.46:0/3076732009 >> 172.21.15.46:0/3076732009 conn(0x55f04d75e220 msgr2=0x55f04d7ef690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.529+0000 7f28d48d6b80 1 -- 172.21.15.46:0/3076732009 shutdown_connections 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.529+0000 7f28d48d6b80 1 -- 172.21.15.46:0/3076732009 wait complete. 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.529+0000 7f28d48d6b80 1 -- start start 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.529+0000 7f28d48d6b80 1 Processor -- start 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28d48d6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7e7a60 0x55f04d7fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28d48d6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 0x55f04d7fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28d48d6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 0x55f04d859fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28d48d6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04d85a520 con 0x55f04d7f7630 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28d48d6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04d85a660 con 0x55f04d7e7a60 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28d48d6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04d85a7a0 con 0x55f04d7e8380 2024-03-18T20:46:58.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28d48d6b80 10 client.0 _ll_register_callbacks cb 0x7f28bc009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28c6cde700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 0x55f04d859fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28c6cde700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 0x55f04d859fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52024/0 (socket says 172.21.15.46:52024) 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28c6cde700 1 -- 172.21.15.46:0/4178564209 learned_addr learned my addr 172.21.15.46:0/4178564209 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28c5cdc700 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 0x55f04d7fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28c64dd700 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7e7a60 0x55f04d7fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28c6cde700 1 -- 172.21.15.46:0/4178564209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 msgr2=0x55f04d7fc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28c6cde700 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 0x55f04d7fc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28c6cde700 1 -- 172.21.15.46:0/4178564209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7e7a60 msgr2=0x55f04d7fc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28c6cde700 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7e7a60 0x55f04d7fc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28c6cde700 1 -- 172.21.15.46:0/4178564209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f04d5bc3b0 con 0x55f04d7f7630 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.530+0000 7f28c64dd700 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7e7a60 0x55f04d7fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28c6cde700 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 0x55f04d859fe0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f28b800afd0 tx=0x7f28b8099860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76519ab5f5c39c38 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28a9ffb700 1 -- 172.21.15.46:0/4178564209 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b809a2e0 con 0x55f04d7f7630 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28a9ffb700 1 -- 172.21.15.46:0/4178564209 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f28b8007aa0 con 0x55f04d7f7630 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28d48d6b80 1 -- 172.21.15.46:0/4178564209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28b4015070 con 0x55f04d7f7630 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28a9ffb700 1 -- 172.21.15.46:0/4178564209 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28b809de30 con 0x55f04d7f7630 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28d48d6b80 10 client.19374 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28d48d6b80 1 -- 172.21.15.46:0/4178564209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f28c01a41a0 con 0x55f04d7f7630 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28a9ffb700 1 -- 172.21.15.46:0/4178564209 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28b8082020 con 0x55f04d7f7630 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28a9ffb700 1 -- 172.21.15.46:0/4178564209 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f28b80b2bb0 con 0x55f04d7f7630 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28d48d6b80 10 client.19374 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28d48d6b80 10 client.19374 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28d48d6b80 20 client.19374 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f04d85bc50 0x55f04d85e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28d48d6b80 4 client.19374 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28d48d6b80 1 -- 172.21.15.46:0/4178564209 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f04d589cf0 con 0x55f04d85bc50 2024-03-18T20:46:58.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.531+0000 7f28c64dd700 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f04d85bc50 0x55f04d85e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28a9ffb700 10 client.19374 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28c64dd700 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f04d85bc50 0x55f04d85e040 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b94182189284efcc server_cookie=d647b9daa4d9f4f2 in_seq=0 out_seq=0 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28a9ffb700 1 -- 172.21.15.46:0/4178564209 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f04d589cf0 con 0x55f04d85bc50 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28a9ffb700 10 client.19374 handle_command_reply: tid=0 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28d48d6b80 1 client.19374 shutdown 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28d48d6b80 1 -- 172.21.15.46:0/4178564209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 msgr2=0x55f04d859fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 0x55f04d859fe0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f28b800afd0 tx=0x7f28b8099860 comp rx=0 tx=0).stop 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28d48d6b80 1 -- 172.21.15.46:0/4178564209 shutdown_connections 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55f04d85bc50 0x55f04d85e040 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f04d7f7630 0x55f04d859fe0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f04d7e7a60 0x55f04d7fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28a9ffb700 0 client.19374 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28d48d6b80 1 --2- 172.21.15.46:0/4178564209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f04d7e8380 0x55f04d7fc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28d48d6b80 1 -- 172.21.15.46:0/4178564209 >> 172.21.15.46:0/4178564209 conn(0x55f04d75e220 msgr2=0x55f04d7f3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28d48d6b80 1 -- 172.21.15.46:0/4178564209 shutdown_connections 2024-03-18T20:46:58.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.532+0000 7f28d48d6b80 1 -- 172.21.15.46:0/4178564209 wait complete. 2024-03-18T20:46:58.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28d48d6b80 20 client.19374 trim_cache size 0 max 16384 2024-03-18T20:46:58.542 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:46:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28c8741700 1 -- 172.21.15.46:0/721073957 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f28ac0a1290 msgr2=0x7f28ac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28c8741700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f28ac0a1290 0x7f28ac0a3750 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f28b4005950 tx=0x7f28b400b410 comp rx=0 tx=0).stop 2024-03-18T20:46:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28c8741700 1 -- 172.21.15.46:0/721073957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c019ba00 msgr2=0x7f28c019be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28c8741700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c019ba00 0x7f28c019be80 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f28b8007d70 tx=0x7f28b800a340 comp rx=0 tx=0).stop 2024-03-18T20:46:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28c8741700 1 -- 172.21.15.46:0/721073957 shutdown_connections 2024-03-18T20:46:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28c8741700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f28ac0a1290 0x7f28ac0a3750 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28c8741700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28c019ba00 0x7f28c019be80 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28c8741700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28c0100f50 0x7f28c0197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28c8741700 1 --2- 172.21.15.46:0/721073957 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28c0106070 0x7f28c0197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28c8741700 1 -- 172.21.15.46:0/721073957 >> 172.21.15.46:0/721073957 conn(0x7f28c00fc7f0 msgr2=0x7f28c00feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28c8741700 1 -- 172.21.15.46:0/721073957 shutdown_connections 2024-03-18T20:46:58.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.533+0000 7f28c8741700 1 -- 172.21.15.46:0/721073957 wait complete. 2024-03-18T20:46:58.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:58 smithi155 ceph-mon[48344]: pgmap v337: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-18T20:46:58.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:46:58 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3151826964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:46:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:58 smithi046 ceph-mon[43495]: pgmap v337: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-18T20:46:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:46:58 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3151826964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:46:58.549 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:46:58.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:46:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.685+0000 7f86aa637700 1 -- 172.21.15.46:0/3433411786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a40ffe70 msgr2=0x7f86a4100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.685+0000 7f86aa637700 1 --2- 172.21.15.46:0/3433411786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a40ffe70 0x7f86a4100250 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f8694009990 tx=0x7f8694009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.685+0000 7f86aa637700 1 -- 172.21.15.46:0/3433411786 shutdown_connections 2024-03-18T20:46:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.685+0000 7f86aa637700 1 --2- 172.21.15.46:0/3433411786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a40ffe70 0x7f86a4100250 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.685+0000 7f86aa637700 1 --2- 172.21.15.46:0/3433411786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86a4104f90 0x7f86a410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.685+0000 7f86aa637700 1 --2- 172.21.15.46:0/3433411786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86a4100820 0x7f86a4104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.685+0000 7f86aa637700 1 -- 172.21.15.46:0/3433411786 >> 172.21.15.46:0/3433411786 conn(0x7f86a4076d80 msgr2=0x7f86a4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.685+0000 7f86aa637700 1 -- 172.21.15.46:0/3433411786 shutdown_connections 2024-03-18T20:46:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.685+0000 7f86aa637700 1 -- 172.21.15.46:0/3433411786 wait complete. 2024-03-18T20:46:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.686+0000 7f86aa637700 1 Processor -- start 2024-03-18T20:46:58.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.686+0000 7f86aa637700 1 -- start start 2024-03-18T20:46:58.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.686+0000 7f86aa637700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86a40ffe70 0x7f86a4197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.686+0000 7f86aa637700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a4104f90 0x7f86a4197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.686+0000 7f86aa637700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86a419bad0 0x7f86a419bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.686+0000 7f86aa637700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a419c550 con 0x7f86a4104f90 2024-03-18T20:46:58.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.686+0000 7f86aa637700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a419c6c0 con 0x7f86a40ffe70 2024-03-18T20:46:58.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.686+0000 7f86aa637700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a419c880 con 0x7f86a419bad0 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a4104f90 0x7f86a4197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a4104f90 0x7f86a4197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52028/0 (socket says 172.21.15.46:52028) 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a37fe700 1 -- 172.21.15.46:0/3268338068 learned_addr learned my addr 172.21.15.46:0/3268338068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a8bd4700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86a419bad0 0x7f86a419bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a3fff700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86a40ffe70 0x7f86a4197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a37fe700 1 -- 172.21.15.46:0/3268338068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86a419bad0 msgr2=0x7f86a419bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a37fe700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86a419bad0 0x7f86a419bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a37fe700 1 -- 172.21.15.46:0/3268338068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86a40ffe70 msgr2=0x7f86a4197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a37fe700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86a40ffe70 0x7f86a4197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a37fe700 1 -- 172.21.15.46:0/3268338068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940095f0 con 0x7f86a4104f90 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a8bd4700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86a419bad0 0x7f86a419bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a37fe700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a4104f90 0x7f86a4197780 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f869800e970 tx=0x7f869800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a17fa700 1 -- 172.21.15.46:0/3268338068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698009790 con 0x7f86a4104f90 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86aa637700 1 -- 172.21.15.46:0/3268338068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a41a3de0 con 0x7f86a4104f90 2024-03-18T20:46:58.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a17fa700 1 -- 172.21.15.46:0/3268338068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8698004cf0 con 0x7f86a4104f90 2024-03-18T20:46:58.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86a17fa700 1 -- 172.21.15.46:0/3268338068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698005680 con 0x7f86a4104f90 2024-03-18T20:46:58.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.687+0000 7f86aa637700 1 -- 172.21.15.46:0/3268338068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a41a4270 con 0x7f86a4104f90 2024-03-18T20:46:58.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.690+0000 7f86a17fa700 1 -- 172.21.15.46:0/3268338068 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8698004750 con 0x7f86a4104f90 2024-03-18T20:46:58.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.690+0000 7f86a17fa700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f868c0a1310 0x7f868c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.691+0000 7f86a3fff700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f868c0a1310 0x7f868c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.691+0000 7f86a17fa700 1 -- 172.21.15.46:0/3268338068 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8698014070 con 0x7f86a4104f90 2024-03-18T20:46:58.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.691+0000 7f86a3fff700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f868c0a1310 0x7f868c0a37d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8694006010 tx=0x7f869400b260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.691+0000 7f86b67ccb80 1 Processor -- start 2024-03-18T20:46:58.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86b67ccb80 1 -- start start 2024-03-18T20:46:58.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86b67ccb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d2a60 0x55e6907d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86b67ccb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 0x55e6907ddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86b67ccb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 0x55e6907e0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86b67ccb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69072c990 con 0x55e6907d3380 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86b67ccb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e69072a350 con 0x55e6907de220 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86b67ccb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e690725c40 con 0x55e6907d2a60 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 0x55e6907ddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 0x55e6907ddce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52042/0 (socket says 172.21.15.46:52042) 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86a37fe700 1 -- 172.21.15.46:0/2690061860 learned_addr learned my addr 172.21.15.46:0/2690061860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86a8bd4700 1 --2- 172.21.15.46:0/2690061860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 0x55e6907e0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86a3fff700 1 --2- 172.21.15.46:0/2690061860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d2a60 0x55e6907d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86a37fe700 1 -- 172.21.15.46:0/2690061860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d2a60 msgr2=0x55e6907d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86a37fe700 1 --2- 172.21.15.46:0/2690061860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d2a60 0x55e6907d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86a37fe700 1 -- 172.21.15.46:0/2690061860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 msgr2=0x55e6907e0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86a37fe700 1 --2- 172.21.15.46:0/2690061860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 0x55e6907e0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86a37fe700 1 -- 172.21.15.46:0/2690061860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6905a73b0 con 0x55e6907d3380 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.692+0000 7f86a37fe700 1 --2- 172.21.15.46:0/2690061860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 0x55e6907ddce0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8698096570 tx=0x7f8698099ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f427893cd2ab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.693+0000 7f868a7fc700 1 -- 172.21.15.46:0/2690061860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980a24f0 con 0x55e6907d3380 2024-03-18T20:46:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.693+0000 7f868a7fc700 1 -- 172.21.15.46:0/2690061860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8698016e10 con 0x55e6907d3380 2024-03-18T20:46:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.693+0000 7f868a7fc700 1 -- 172.21.15.46:0/2690061860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980ca930 con 0x55e6907d3380 2024-03-18T20:46:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.693+0000 7f86b67ccb80 1 -- 172.21.15.46:0/2690061860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 msgr2=0x55e6907ddce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.693+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/2690061860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 0x55e6907ddce0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8698096570 tx=0x7f8698099ae0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.693+0000 7f86b67ccb80 1 -- 172.21.15.46:0/2690061860 shutdown_connections 2024-03-18T20:46:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.693+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/2690061860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 0x55e6907e0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.693+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/2690061860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d2a60 0x55e6907d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.693+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/2690061860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 0x55e6907ddce0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.693+0000 7f86b67ccb80 1 -- 172.21.15.46:0/2690061860 >> 172.21.15.46:0/2690061860 conn(0x55e690749220 msgr2=0x55e6907cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.693+0000 7f86b67ccb80 1 -- 172.21.15.46:0/2690061860 shutdown_connections 2024-03-18T20:46:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.693+0000 7f86b67ccb80 1 -- 172.21.15.46:0/2690061860 wait complete. 2024-03-18T20:46:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.694+0000 7f86b67ccb80 1 -- start start 2024-03-18T20:46:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.694+0000 7f86b67ccb80 1 Processor -- start 2024-03-18T20:46:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.694+0000 7f86b67ccb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 0x55e6907ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.694+0000 7f86b67ccb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d3380 0x55e6907dafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.694+0000 7f86b67ccb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 0x55e6907db700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.694+0000 7f86b67ccb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6906c1d80 con 0x55e6907d2a60 2024-03-18T20:46:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.694+0000 7f86b67ccb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6906c1a20 con 0x55e6907de220 2024-03-18T20:46:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.694+0000 7f86b67ccb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6905a96a0 con 0x55e6907d3380 2024-03-18T20:46:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.694+0000 7f86b67ccb80 10 client.0 _ll_register_callbacks cb 0x55e69052d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 0x55e6907ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 0x55e6907ddcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52048/0 (socket says 172.21.15.46:52048) 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86a3fff700 1 -- 172.21.15.46:0/296807632 learned_addr learned my addr 172.21.15.46:0/296807632 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86a8bd4700 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 0x55e6907db700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86a37fe700 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d3380 0x55e6907dafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86a3fff700 1 -- 172.21.15.46:0/296807632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d3380 msgr2=0x55e6907dafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86a3fff700 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d3380 0x55e6907dafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86a3fff700 1 -- 172.21.15.46:0/296807632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 msgr2=0x55e6907db700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86a3fff700 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 0x55e6907db700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86a3fff700 1 -- 172.21.15.46:0/296807632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a41a4270 con 0x55e6907d2a60 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86a3fff700 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 0x55e6907ddcd0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f869400be00 tx=0x7f8694009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852449008ddaafc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86837fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86837fe700 1 -- 172.21.15.46:0/296807632 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694004910 con 0x55e6907d2a60 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86837fe700 1 -- 172.21.15.46:0/296807632 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8694004a70 con 0x55e6907d2a60 2024-03-18T20:46:58.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86b67ccb80 1 -- 172.21.15.46:0/296807632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6907c6ad0 con 0x55e6907d2a60 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86837fe700 1 -- 172.21.15.46:0/296807632 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8694004c30 con 0x55e6907d2a60 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86b67ccb80 10 client.19386 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.695+0000 7f86b67ccb80 1 -- 172.21.15.46:0/296807632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55e690553b60 con 0x55e6907d2a60 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.696+0000 7f86837fe700 1 -- 172.21.15.46:0/296807632 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f869403d020 con 0x55e6907d2a60 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.696+0000 7f86837fe700 1 -- 172.21.15.46:0/296807632 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8694030430 con 0x55e6907d2a60 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.696+0000 7f86b67ccb80 10 client.19386 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.696+0000 7f86b67ccb80 10 client.19386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.696+0000 7f86b67ccb80 20 client.19386 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.696+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e6907edf00 0x55e6907f0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.696+0000 7f86b67ccb80 4 client.19386 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.696+0000 7f86b67ccb80 1 -- 172.21.15.46:0/296807632 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e69071e700 con 0x55e6907edf00 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.696+0000 7f86a37fe700 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e6907edf00 0x55e6907f0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.697+0000 7f86837fe700 10 client.19386 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:58.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.697+0000 7f86a37fe700 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e6907edf00 0x55e6907f0320 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899bea098fd6dbf1 server_cookie=9e163aab1275a92 in_seq=0 out_seq=0 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.697+0000 7f86837fe700 1 -- 172.21.15.46:0/296807632 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e69071e700 con 0x55e6907edf00 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.697+0000 7f86837fe700 10 client.19386 handle_command_reply: tid=0 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.697+0000 7f86b67ccb80 1 client.19386 shutdown 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.697+0000 7f86b67ccb80 1 -- 172.21.15.46:0/296807632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 msgr2=0x55e6907ddcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.697+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 0x55e6907ddcd0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f869400be00 tx=0x7f8694009ca0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.698+0000 7f86b67ccb80 1 -- 172.21.15.46:0/296807632 shutdown_connections 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.698+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e6907edf00 0x55e6907f0320 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.698+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 0x55e6907ddcd0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.698+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 0x55e6907db700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.698+0000 7f86837fe700 0 client.19386 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.698+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/296807632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d3380 0x55e6907dafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.698+0000 7f86b67ccb80 1 -- 172.21.15.46:0/296807632 >> 172.21.15.46:0/296807632 conn(0x55e690749220 msgr2=0x55e6907cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.698+0000 7f86b67ccb80 1 -- 172.21.15.46:0/296807632 shutdown_connections 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.698+0000 7f86b67ccb80 1 -- 172.21.15.46:0/296807632 wait complete. 2024-03-18T20:46:58.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.698+0000 7f86b67ccb80 20 client.19386 trim_cache size 0 max 16384 2024-03-18T20:46:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.711+0000 7f86b67ccb80 1 Processor -- start 2024-03-18T20:46:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.711+0000 7f86b67ccb80 1 -- start start 2024-03-18T20:46:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86b67ccb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 0x55e6907e4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86b67ccb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d3380 0x55e6907e95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86b67ccb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 0x55e6907ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86b67ccb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6907d45c0 con 0x55e6907d2a60 2024-03-18T20:46:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86b67ccb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6907dba90 con 0x55e6907de220 2024-03-18T20:46:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86b67ccb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6907e53c0 con 0x55e6907d3380 2024-03-18T20:46:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 0x55e6907e4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 0x55e6907e4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52056/0 (socket says 172.21.15.46:52056) 2024-03-18T20:46:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86a3fff700 1 -- 172.21.15.46:0/3498848396 learned_addr learned my addr 172.21.15.46:0/3498848396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86a37fe700 1 --2- 172.21.15.46:0/3498848396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d3380 0x55e6907e95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86a3fff700 1 -- 172.21.15.46:0/3498848396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d3380 msgr2=0x55e6907e95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86a3fff700 1 --2- 172.21.15.46:0/3498848396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d3380 0x55e6907e95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86a3fff700 1 -- 172.21.15.46:0/3498848396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 msgr2=0x55e6907ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86a3fff700 1 --2- 172.21.15.46:0/3498848396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 0x55e6907ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86a3fff700 1 -- 172.21.15.46:0/3498848396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6906ebea0 con 0x55e6907d2a60 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f86a3fff700 1 --2- 172.21.15.46:0/3498848396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 0x55e6907e4e50 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f8694030790 tx=0x7f8694005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc58458cdd48c2a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f868affd700 1 -- 172.21.15.46:0/3498848396 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86940309b0 con 0x55e6907d2a60 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f868affd700 1 -- 172.21.15.46:0/3498848396 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f869403d070 con 0x55e6907d2a60 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.712+0000 7f868affd700 1 -- 172.21.15.46:0/3498848396 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869400f770 con 0x55e6907d2a60 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.713+0000 7f86b67ccb80 1 -- 172.21.15.46:0/3498848396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 msgr2=0x55e6907e4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.713+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/3498848396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 0x55e6907e4e50 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f8694030790 tx=0x7f8694005960 comp rx=0 tx=0).stop 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.713+0000 7f86b67ccb80 1 -- 172.21.15.46:0/3498848396 shutdown_connections 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.713+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/3498848396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d2a60 0x55e6907e4e50 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.713+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/3498848396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907de220 0x55e6907ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.713+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/3498848396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907d3380 0x55e6907e95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.713+0000 7f86b67ccb80 1 -- 172.21.15.46:0/3498848396 >> 172.21.15.46:0/3498848396 conn(0x55e690749220 msgr2=0x55e6907dd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.713+0000 7f86b67ccb80 1 -- 172.21.15.46:0/3498848396 shutdown_connections 2024-03-18T20:46:58.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.713+0000 7f86b67ccb80 1 -- 172.21.15.46:0/3498848396 wait complete. 2024-03-18T20:46:58.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.713+0000 7f86b67ccb80 1 -- start start 2024-03-18T20:46:58.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 406092 3.69 MB/sec execute 473 sec latency 2395.011 ms 2024-03-18T20:46:58.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:58 smithi073 ceph-mon[47384]: pgmap v337: 97 pgs: 97 active+clean; 155 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-18T20:46:58.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:46:58 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3151826964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:46:58.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86b67ccb80 1 Processor -- start 2024-03-18T20:46:58.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86b67ccb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907d2a60 0x55e6907e2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86b67ccb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 0x55e6907e3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86b67ccb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907de220 0x55e690840cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86b67ccb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6907e3c60 con 0x55e6907d3380 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86b67ccb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e690841230 con 0x55e6907d2a60 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86b67ccb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e690841370 con 0x55e6907de220 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86b67ccb80 10 client.0 _ll_register_callbacks cb 0x7f869400ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 0x55e6907e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86a37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 0x55e6907e3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52066/0 (socket says 172.21.15.46:52066) 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86a37fe700 1 -- 172.21.15.46:0/1082098422 learned_addr learned my addr 172.21.15.46:0/1082098422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86a8bd4700 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907de220 0x55e690840cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86a3fff700 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907d2a60 0x55e6907e2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86a37fe700 1 -- 172.21.15.46:0/1082098422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907de220 msgr2=0x55e690840cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.714+0000 7f86a37fe700 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907de220 0x55e690840cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86a37fe700 1 -- 172.21.15.46:0/1082098422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907d2a60 msgr2=0x55e6907e2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:58.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86a37fe700 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907d2a60 0x55e6907e2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86a37fe700 1 -- 172.21.15.46:0/1082098422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e690553b60 con 0x55e6907d3380 2024-03-18T20:46:58.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86a37fe700 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 0x55e6907e3370 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f8698005950 tx=0x7f8698016f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8d65da46005bc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:58.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86837fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:58.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86837fe700 1 -- 172.21.15.46:0/1082098422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980a2570 con 0x55e6907d3380 2024-03-18T20:46:58.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86837fe700 1 -- 172.21.15.46:0/1082098422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8698004e60 con 0x55e6907d3380 2024-03-18T20:46:58.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86b67ccb80 1 -- 172.21.15.46:0/1082098422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8698016e10 con 0x55e6907d3380 2024-03-18T20:46:58.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86837fe700 1 -- 172.21.15.46:0/1082098422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8698099e40 con 0x55e6907d3380 2024-03-18T20:46:58.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86b67ccb80 10 client.19392 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:58.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86b67ccb80 1 -- 172.21.15.46:0/1082098422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55e6905abcd0 con 0x55e6907d3380 2024-03-18T20:46:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86837fe700 1 -- 172.21.15.46:0/1082098422 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8698016e10 con 0x55e6907d3380 2024-03-18T20:46:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86837fe700 1 -- 172.21.15.46:0/1082098422 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8698096780 con 0x55e6907d3380 2024-03-18T20:46:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86b67ccb80 10 client.19392 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86b67ccb80 10 client.19392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86b67ccb80 20 client.19392 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e690842760 0x55e690844b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86b67ccb80 4 client.19392 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:46:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.715+0000 7f86b67ccb80 1 -- 172.21.15.46:0/1082098422 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e690574cf0 con 0x55e690842760 2024-03-18T20:46:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.716+0000 7f86a3fff700 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e690842760 0x55e690844b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.716+0000 7f86a3fff700 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e690842760 0x55e690844b80 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61411ddfef1dad1b server_cookie=b5b89841fd4890ab in_seq=0 out_seq=0 2024-03-18T20:46:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.716+0000 7f86837fe700 10 client.19392 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:58.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.716+0000 7f86837fe700 1 -- 172.21.15.46:0/1082098422 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e690574cf0 con 0x55e690842760 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.716+0000 7f86837fe700 10 client.19392 handle_command_reply: tid=0 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.716+0000 7f86b67ccb80 1 client.19392 shutdown 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86b67ccb80 1 -- 172.21.15.46:0/1082098422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 msgr2=0x55e6907e3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 0x55e6907e3370 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f8698005950 tx=0x7f8698016f80 comp rx=0 tx=0).stop 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86b67ccb80 1 -- 172.21.15.46:0/1082098422 shutdown_connections 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e690842760 0x55e690844b80 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e6907de220 0x55e690840cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6907d2a60 0x55e6907e2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86837fe700 0 client.19392 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86b67ccb80 1 --2- 172.21.15.46:0/1082098422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6907d3380 0x55e6907e3370 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86b67ccb80 1 -- 172.21.15.46:0/1082098422 >> 172.21.15.46:0/1082098422 conn(0x55e690749220 msgr2=0x55e6907d6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86b67ccb80 1 -- 172.21.15.46:0/1082098422 shutdown_connections 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86b67ccb80 1 -- 172.21.15.46:0/1082098422 wait complete. 2024-03-18T20:46:58.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86b67ccb80 20 client.19392 trim_cache size 0 max 16384 2024-03-18T20:46:58.728 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:46:58.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86aa637700 1 -- 172.21.15.46:0/3268338068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f868c0a1310 msgr2=0x7f868c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86aa637700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f868c0a1310 0x7f868c0a37d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8694006010 tx=0x7f869400b260 comp rx=0 tx=0).stop 2024-03-18T20:46:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86aa637700 1 -- 172.21.15.46:0/3268338068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a4104f90 msgr2=0x7f86a4197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86aa637700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a4104f90 0x7f86a4197780 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f869800e970 tx=0x7f869800ed30 comp rx=0 tx=0).stop 2024-03-18T20:46:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86aa637700 1 -- 172.21.15.46:0/3268338068 shutdown_connections 2024-03-18T20:46:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86aa637700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f868c0a1310 0x7f868c0a37d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86aa637700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86a419bad0 0x7f86a419bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86aa637700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86a40ffe70 0x7f86a4197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86aa637700 1 --2- 172.21.15.46:0/3268338068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86a4104f90 0x7f86a4197780 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.717+0000 7f86aa637700 1 -- 172.21.15.46:0/3268338068 >> 172.21.15.46:0/3268338068 conn(0x7f86a4076d80 msgr2=0x7f86a4069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.718+0000 7f86aa637700 1 -- 172.21.15.46:0/3268338068 shutdown_connections 2024-03-18T20:46:58.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:58.718+0000 7f86aa637700 1 -- 172.21.15.46:0/3268338068 wait complete. 2024-03-18T20:46:59.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 406092 3.68 MB/sec execute 474 sec latency 3395.103 ms 2024-03-18T20:46:59.734 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:46:59.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:46:59.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:46:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.887+0000 7f4827369700 1 -- 172.21.15.46:0/1658433478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4820106e60 msgr2=0x7f4820107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.887+0000 7f4827369700 1 --2- 172.21.15.46:0/1658433478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4820106e60 0x7f4820107240 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f481c0098f0 tx=0x7f481c009c00 comp rx=0 tx=0).stop 2024-03-18T20:46:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.887+0000 7f4827369700 1 -- 172.21.15.46:0/1658433478 shutdown_connections 2024-03-18T20:46:59.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.887+0000 7f4827369700 1 --2- 172.21.15.46:0/1658433478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4820068810 0x7f482010db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.887+0000 7f4827369700 1 --2- 172.21.15.46:0/1658433478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4820106e60 0x7f4820107240 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.887+0000 7f4827369700 1 --2- 172.21.15.46:0/1658433478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4820067e50 0x7f48200682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.887+0000 7f4827369700 1 -- 172.21.15.46:0/1658433478 >> 172.21.15.46:0/1658433478 conn(0x7f4820076d40 msgr2=0x7f4820077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.887+0000 7f4827369700 1 -- 172.21.15.46:0/1658433478 shutdown_connections 2024-03-18T20:46:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.887+0000 7f4827369700 1 -- 172.21.15.46:0/1658433478 wait complete. 2024-03-18T20:46:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4827369700 1 Processor -- start 2024-03-18T20:46:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4827369700 1 -- start start 2024-03-18T20:46:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4827369700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4820067e50 0x7f48200ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4827369700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4820068810 0x7f48201001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4827369700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4820106e60 0x7f48201006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4827369700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48201a5500 con 0x7f4820106e60 2024-03-18T20:46:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4827369700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48201a5670 con 0x7f4820067e50 2024-03-18T20:46:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4827369700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48201a5830 con 0x7f4820068810 2024-03-18T20:46:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4825105700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4820067e50 0x7f48200ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4825105700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4820067e50 0x7f48200ffc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49838/0 (socket says 172.21.15.46:49838) 2024-03-18T20:46:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4824904700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4820068810 0x7f48201001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4825105700 1 -- 172.21.15.46:0/849708147 learned_addr learned my addr 172.21.15.46:0/849708147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.888+0000 7f4825906700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4820106e60 0x7f48201006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.889+0000 7f4825105700 1 -- 172.21.15.46:0/849708147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4820068810 msgr2=0x7f48201001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.889+0000 7f4825105700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4820068810 0x7f48201001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.889+0000 7f4825105700 1 -- 172.21.15.46:0/849708147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4820106e60 msgr2=0x7f48201006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.889+0000 7f4825105700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4820106e60 0x7f48201006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.889+0000 7f4825105700 1 -- 172.21.15.46:0/849708147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48100095f0 con 0x7f4820067e50 2024-03-18T20:46:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.889+0000 7f4825105700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4820067e50 0x7f48200ffc70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f481c005950 tx=0x7f481c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.889+0000 7f48167fc700 1 -- 172.21.15.46:0/849708147 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c01c070 con 0x7f4820067e50 2024-03-18T20:46:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.889+0000 7f48167fc700 1 -- 172.21.15.46:0/849708147 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f481c00baa0 con 0x7f4820067e50 2024-03-18T20:46:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.889+0000 7f4827369700 1 -- 172.21.15.46:0/849708147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c009520 con 0x7f4820067e50 2024-03-18T20:46:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.889+0000 7f48167fc700 1 -- 172.21.15.46:0/849708147 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c0038e0 con 0x7f4820067e50 2024-03-18T20:46:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.889+0000 7f4827369700 1 -- 172.21.15.46:0/849708147 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48201a5da0 con 0x7f4820067e50 2024-03-18T20:46:59.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.892+0000 7f48167fc700 1 -- 172.21.15.46:0/849708147 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f481c020030 con 0x7f4820067e50 2024-03-18T20:46:59.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.892+0000 7f48167fc700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f480c0a1310 0x7f480c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.893+0000 7f48167fc700 1 -- 172.21.15.46:0/849708147 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f481c0cd9f0 con 0x7f4820067e50 2024-03-18T20:46:59.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.893+0000 7f4824904700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f480c0a1310 0x7f480c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.893+0000 7f4824904700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f480c0a1310 0x7f480c0a37d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f48201013a0 tx=0x7f4810009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.894+0000 7f48334feb80 1 Processor -- start 2024-03-18T20:46:59.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.894+0000 7f48334feb80 1 -- start start 2024-03-18T20:46:59.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f48334feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396bae0 0x55dd2396bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f48334feb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396c400 0x55dd23976d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f48334feb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd23979690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f48334feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd238c5a10 con 0x55dd2396bae0 2024-03-18T20:46:59.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f48334feb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd238c33d0 con 0x55dd239772a0 2024-03-18T20:46:59.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f48334feb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd238becc0 con 0x55dd2396c400 2024-03-18T20:46:59.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f4825105700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396bae0 0x55dd2396bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f4825105700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396bae0 0x55dd2396bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52090/0 (socket says 172.21.15.46:52090) 2024-03-18T20:46:59.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f4825105700 1 -- 172.21.15.46:0/4153455437 learned_addr learned my addr 172.21.15.46:0/4153455437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:59.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f4825906700 1 --2- 172.21.15.46:0/4153455437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd23979690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f4824904700 1 --2- 172.21.15.46:0/4153455437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396c400 0x55dd23976d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f4825105700 1 -- 172.21.15.46:0/4153455437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396c400 msgr2=0x55dd23976d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f4825105700 1 --2- 172.21.15.46:0/4153455437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396c400 0x55dd23976d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f4825105700 1 -- 172.21.15.46:0/4153455437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 msgr2=0x55dd23979690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f4825105700 1 --2- 172.21.15.46:0/4153455437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd23979690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f4825105700 1 -- 172.21.15.46:0/4153455437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd23740430 con 0x55dd2396bae0 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f4825906700 1 --2- 172.21.15.46:0/4153455437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd23979690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f4825105700 1 --2- 172.21.15.46:0/4153455437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396bae0 0x55dd2396bec0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f481c0975e0 tx=0x7f481c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1d05016ff3f6de server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.895+0000 7f480b7fe700 1 -- 172.21.15.46:0/4153455437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c09ab70 con 0x55dd2396bae0 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.896+0000 7f480b7fe700 1 -- 172.21.15.46:0/4153455437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f481c09e9a0 con 0x55dd2396bae0 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.896+0000 7f480b7fe700 1 -- 172.21.15.46:0/4153455437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c021380 con 0x55dd2396bae0 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.896+0000 7f48334feb80 1 -- 172.21.15.46:0/4153455437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396bae0 msgr2=0x55dd2396bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.896+0000 7f48334feb80 1 --2- 172.21.15.46:0/4153455437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396bae0 0x55dd2396bec0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f481c0975e0 tx=0x7f481c005040 comp rx=0 tx=0).stop 2024-03-18T20:46:59.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.896+0000 7f48334feb80 1 -- 172.21.15.46:0/4153455437 shutdown_connections 2024-03-18T20:46:59.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.896+0000 7f48334feb80 1 --2- 172.21.15.46:0/4153455437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396bae0 0x55dd2396bec0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.896+0000 7f48334feb80 1 --2- 172.21.15.46:0/4153455437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd23979690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.896+0000 7f48334feb80 1 --2- 172.21.15.46:0/4153455437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396c400 0x55dd23976d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.896+0000 7f48334feb80 1 -- 172.21.15.46:0/4153455437 >> 172.21.15.46:0/4153455437 conn(0x55dd238e22a0 msgr2=0x55dd239680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:59.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.896+0000 7f48334feb80 1 -- 172.21.15.46:0/4153455437 shutdown_connections 2024-03-18T20:46:59.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.896+0000 7f48334feb80 1 -- 172.21.15.46:0/4153455437 wait complete. 2024-03-18T20:46:59.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.897+0000 7f48334feb80 1 -- start start 2024-03-18T20:46:59.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.897+0000 7f48334feb80 1 Processor -- start 2024-03-18T20:46:59.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f48334feb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd23977680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f48334feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396c400 0x55dd23977bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f48334feb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd23982700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f48334feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2385ae00 con 0x55dd2396c400 2024-03-18T20:46:59.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f48334feb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2385aaa0 con 0x55dd239772a0 2024-03-18T20:46:59.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f48334feb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd23742720 con 0x55dd2396bae0 2024-03-18T20:46:59.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f48334feb80 10 client.0 _ll_register_callbacks cb 0x55dd236c6420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:59.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f4825105700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd23977680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f4825105700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd23977680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:52216/0 (socket says 172.21.15.46:52216) 2024-03-18T20:46:59.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f4825105700 1 -- 172.21.15.46:0/1448856212 learned_addr learned my addr 172.21.15.46:0/1448856212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:59.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f4824904700 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396c400 0x55dd23977bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f4825906700 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd23982700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f4825105700 1 -- 172.21.15.46:0/1448856212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 msgr2=0x55dd23982700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:59.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f4825105700 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd23982700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f4825105700 1 -- 172.21.15.46:0/1448856212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396c400 msgr2=0x55dd23977bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:59.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f4825105700 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396c400 0x55dd23977bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.898+0000 7f4825105700 1 -- 172.21.15.46:0/1448856212 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4818011230 con 0x55dd2396bae0 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.899+0000 7f4825105700 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd23977680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f481c0a2a30 tx=0x7f481c00fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15f08e71f06ddb1c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.899+0000 7f4808ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.899+0000 7f4808ff9700 1 -- 172.21.15.46:0/1448856212 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c0a2f60 con 0x55dd2396bae0 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.899+0000 7f48334feb80 1 -- 172.21.15.46:0/1448856212 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd2395fb50 con 0x55dd2396bae0 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.899+0000 7f4808ff9700 1 -- 172.21.15.46:0/1448856212 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f481c09ab70 con 0x55dd2396bae0 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.899+0000 7f4808ff9700 1 -- 172.21.15.46:0/1448856212 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f481c021380 con 0x55dd2396bae0 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.899+0000 7f48334feb80 10 client.29047 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.899+0000 7f48334feb80 1 -- 172.21.15.46:0/1448856212 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f48201a5da0 con 0x55dd2396bae0 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.900+0000 7f4808ff9700 1 -- 172.21.15.46:0/1448856212 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd2395fb50 con 0x55dd2396bae0 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.900+0000 7f4808ff9700 1 -- 172.21.15.46:0/1448856212 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f481c09a060 con 0x55dd2396bae0 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.900+0000 7f48334feb80 10 client.29047 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:59.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.900+0000 7f48334feb80 10 client.29047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.900+0000 7f48334feb80 20 client.29047 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.900+0000 7f48334feb80 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd23986670 0x55dd23988a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.900+0000 7f48334feb80 4 client.29047 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.900+0000 7f48334feb80 1 -- 172.21.15.46:0/1448856212 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd238b7780 con 0x55dd23986670 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.900+0000 7f4824904700 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd23986670 0x55dd23988a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.901+0000 7f4808ff9700 10 client.29047 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.901+0000 7f4824904700 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd23986670 0x55dd23988a60 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353492fcc86aae02 server_cookie=f32150d1d4009e3b in_seq=0 out_seq=0 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f4808ff9700 1 -- 172.21.15.46:0/1448856212 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dd238b7780 con 0x55dd23986670 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f4808ff9700 10 client.29047 handle_command_reply: tid=0 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f48334feb80 1 client.29047 shutdown 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f48334feb80 1 -- 172.21.15.46:0/1448856212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 msgr2=0x55dd23977680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f48334feb80 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd23977680 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f481c0a2a30 tx=0x7f481c00fa70 comp rx=0 tx=0).stop 2024-03-18T20:46:59.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f48334feb80 1 -- 172.21.15.46:0/1448856212 shutdown_connections 2024-03-18T20:46:59.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f48334feb80 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd23986670 0x55dd23988a60 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f48334feb80 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd23982700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f48334feb80 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd23977680 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f48334feb80 1 --2- 172.21.15.46:0/1448856212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396c400 0x55dd23977bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f4808ff9700 0 client.29047 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:59.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f48334feb80 1 -- 172.21.15.46:0/1448856212 >> 172.21.15.46:0/1448856212 conn(0x55dd238e22a0 msgr2=0x55dd23967c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:59.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f48334feb80 1 -- 172.21.15.46:0/1448856212 shutdown_connections 2024-03-18T20:46:59.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f48334feb80 1 -- 172.21.15.46:0/1448856212 wait complete. 2024-03-18T20:46:59.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.902+0000 7f48334feb80 20 client.29047 trim_cache size 0 max 16384 2024-03-18T20:46:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f48334feb80 1 Processor -- start 2024-03-18T20:46:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f48334feb80 1 -- start start 2024-03-18T20:46:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f48334feb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd2397c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f48334feb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd2396c400 0x55dd239718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f48334feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd239772a0 0x55dd23971df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f48334feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd238b77b0 con 0x55dd239772a0 2024-03-18T20:46:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f48334feb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2396cae0 con 0x55dd2396c400 2024-03-18T20:46:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f48334feb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd23962290 con 0x55dd2396bae0 2024-03-18T20:46:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f4824904700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd2396c400 0x55dd239718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f4825906700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd239772a0 0x55dd23971df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f4825105700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd2397c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f4824904700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd2396c400 0x55dd239718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49880/0 (socket says 172.21.15.46:49880) 2024-03-18T20:46:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f4825906700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd239772a0 0x55dd23971df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52108/0 (socket says 172.21.15.46:52108) 2024-03-18T20:46:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.922+0000 7f4824904700 1 -- 172.21.15.46:0/1178380323 learned_addr learned my addr 172.21.15.46:0/1178380323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.923+0000 7f4824904700 1 -- 172.21.15.46:0/1178380323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 msgr2=0x55dd2397c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.923+0000 7f4824904700 1 --2- 172.21.15.46:0/1178380323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd2397c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.923+0000 7f4824904700 1 -- 172.21.15.46:0/1178380323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd239772a0 msgr2=0x55dd23971df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.923+0000 7f4824904700 1 --2- 172.21.15.46:0/1178380323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd239772a0 0x55dd23971df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.923+0000 7f4824904700 1 -- 172.21.15.46:0/1178380323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd2398d660 con 0x55dd2396c400 2024-03-18T20:46:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.923+0000 7f4825906700 1 --2- 172.21.15.46:0/1178380323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd239772a0 0x55dd23971df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.923+0000 7f4825105700 1 --2- 172.21.15.46:0/1178380323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd2397c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.923+0000 7f4824904700 1 --2- 172.21.15.46:0/1178380323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd2396c400 0x55dd239718b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55dd23989340 tx=0x7f4810014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bd97bfc5751714d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.923+0000 7f480bfff700 1 -- 172.21.15.46:0/1178380323 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810022040 con 0x55dd2396c400 2024-03-18T20:46:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.923+0000 7f480bfff700 1 -- 172.21.15.46:0/1178380323 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4810023040 con 0x55dd2396c400 2024-03-18T20:46:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.923+0000 7f480bfff700 1 -- 172.21.15.46:0/1178380323 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481001d870 con 0x55dd2396c400 2024-03-18T20:46:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.924+0000 7f48334feb80 1 -- 172.21.15.46:0/1178380323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd2396c400 msgr2=0x55dd239718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.924+0000 7f48334feb80 1 --2- 172.21.15.46:0/1178380323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd2396c400 0x55dd239718b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55dd23989340 tx=0x7f4810014000 comp rx=0 tx=0).stop 2024-03-18T20:46:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.924+0000 7f48334feb80 1 -- 172.21.15.46:0/1178380323 shutdown_connections 2024-03-18T20:46:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.924+0000 7f48334feb80 1 --2- 172.21.15.46:0/1178380323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd239772a0 0x55dd23971df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.924+0000 7f48334feb80 1 --2- 172.21.15.46:0/1178380323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd2397c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.924+0000 7f48334feb80 1 --2- 172.21.15.46:0/1178380323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd2396c400 0x55dd239718b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.924+0000 7f48334feb80 1 -- 172.21.15.46:0/1178380323 >> 172.21.15.46:0/1178380323 conn(0x55dd238e22a0 msgr2=0x55dd23977680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.924+0000 7f48334feb80 1 -- 172.21.15.46:0/1178380323 shutdown_connections 2024-03-18T20:46:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.924+0000 7f48334feb80 1 -- 172.21.15.46:0/1178380323 wait complete. 2024-03-18T20:46:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.924+0000 7f48334feb80 1 -- start start 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f48334feb80 1 Processor -- start 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f48334feb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd239d4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f48334feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396c400 0x55dd239d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f48334feb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd239ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f48334feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd239ceed0 con 0x55dd2396c400 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f48334feb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd239cf040 con 0x55dd239772a0 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f48334feb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd239cf200 con 0x55dd2396bae0 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f48334feb80 10 client.0 _ll_register_callbacks cb 0x7f481c09aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f4824904700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396c400 0x55dd239d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f4824904700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396c400 0x55dd239d4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52120/0 (socket says 172.21.15.46:52120) 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f4824904700 1 -- 172.21.15.46:0/680846337 learned_addr learned my addr 172.21.15.46:0/680846337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f4825906700 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd239ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.925+0000 7f4825105700 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd239d4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f4824904700 1 -- 172.21.15.46:0/680846337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 msgr2=0x55dd239d4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f4824904700 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd239d4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f4824904700 1 -- 172.21.15.46:0/680846337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 msgr2=0x55dd239ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:46:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f4824904700 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd239ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f4824904700 1 -- 172.21.15.46:0/680846337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4818009180 con 0x55dd2396c400 2024-03-18T20:46:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f4825906700 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd239ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:46:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f4824904700 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396c400 0x55dd239d4980 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f4810033000 tx=0x7f481000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8793161ef4d89892 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:46:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f4808ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:46:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f4808ff9700 1 -- 172.21.15.46:0/680846337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810022040 con 0x55dd2396c400 2024-03-18T20:46:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f4808ff9700 1 -- 172.21.15.46:0/680846337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f481002b300 con 0x55dd2396c400 2024-03-18T20:46:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f48334feb80 1 -- 172.21.15.46:0/680846337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f481c09e9a0 con 0x55dd2396c400 2024-03-18T20:46:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f4808ff9700 1 -- 172.21.15.46:0/680846337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f481002b500 con 0x55dd2396c400 2024-03-18T20:46:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f48334feb80 10 client.19410 fetch_fsmap learned FSMap version 49 2024-03-18T20:46:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f48334feb80 1 -- 172.21.15.46:0/680846337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f4810017070 con 0x55dd2396c400 2024-03-18T20:46:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.926+0000 7f4808ff9700 1 -- 172.21.15.46:0/680846337 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f481c09e9a0 con 0x55dd2396c400 2024-03-18T20:46:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.927+0000 7f4808ff9700 1 -- 172.21.15.46:0/680846337 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4810004a40 con 0x55dd2396c400 2024-03-18T20:46:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.927+0000 7f48334feb80 10 client.19410 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:46:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.927+0000 7f48334feb80 10 client.19410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:46:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.927+0000 7f48334feb80 20 client.19410 populate_metadata read hostname 'smithi046' 2024-03-18T20:46:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.927+0000 7f48334feb80 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd239dc480 0x55dd239de8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:46:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.927+0000 7f48334feb80 4 client.19410 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:46:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.927+0000 7f48334feb80 1 -- 172.21.15.46:0/680846337 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dd2370dd70 con 0x55dd239dc480 2024-03-18T20:46:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.927+0000 7f4825105700 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd239dc480 0x55dd239de8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:46:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.927+0000 7f4808ff9700 10 client.19410 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.927+0000 7f4825105700 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd239dc480 0x55dd239de8a0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4b31bedc31457e0 server_cookie=8b50af3c8f00f76f in_seq=0 out_seq=0 2024-03-18T20:46:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.928+0000 7f4808ff9700 1 -- 172.21.15.46:0/680846337 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dd2370dd70 con 0x55dd239dc480 2024-03-18T20:46:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.928+0000 7f4808ff9700 10 client.19410 handle_command_reply: tid=0 2024-03-18T20:46:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.928+0000 7f48334feb80 1 client.19410 shutdown 2024-03-18T20:46:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.929+0000 7f48334feb80 1 -- 172.21.15.46:0/680846337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396c400 msgr2=0x55dd239d4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.929+0000 7f48334feb80 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396c400 0x55dd239d4980 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f4810033000 tx=0x7f481000ed40 comp rx=0 tx=0).stop 2024-03-18T20:46:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.929+0000 7f48334feb80 1 -- 172.21.15.46:0/680846337 shutdown_connections 2024-03-18T20:46:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.929+0000 7f48334feb80 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd239dc480 0x55dd239de8a0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.929+0000 7f48334feb80 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd239772a0 0x55dd239ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.929+0000 7f48334feb80 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd2396bae0 0x55dd239d4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.929+0000 7f4808ff9700 0 client.19410 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:46:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.929+0000 7f48334feb80 1 --2- 172.21.15.46:0/680846337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd2396c400 0x55dd239d4980 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.929+0000 7f48334feb80 1 -- 172.21.15.46:0/680846337 >> 172.21.15.46:0/680846337 conn(0x55dd238e22a0 msgr2=0x55dd239718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.929+0000 7f48334feb80 1 -- 172.21.15.46:0/680846337 shutdown_connections 2024-03-18T20:46:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.929+0000 7f48334feb80 1 -- 172.21.15.46:0/680846337 wait complete. 2024-03-18T20:46:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.929+0000 7f48334feb80 20 client.19410 trim_cache size 0 max 16384 2024-03-18T20:46:59.940 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:46:59.940 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:46:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "6cad749a-3ed2-44e2-a0de-9b87ef81ac96", 2024-03-18T20:46:59.941 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:46:59.941 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:46:59.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.930+0000 7f4827369700 1 -- 172.21.15.46:0/849708147 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f480c0a1310 msgr2=0x7f480c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:59.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.930+0000 7f4827369700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f480c0a1310 0x7f480c0a37d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f48201013a0 tx=0x7f4810009310 comp rx=0 tx=0).stop 2024-03-18T20:46:59.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.930+0000 7f4827369700 1 -- 172.21.15.46:0/849708147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4820067e50 msgr2=0x7f48200ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:46:59.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.930+0000 7f4827369700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4820067e50 0x7f48200ffc70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f481c005950 tx=0x7f481c00fa00 comp rx=0 tx=0).stop 2024-03-18T20:46:59.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.930+0000 7f4827369700 1 -- 172.21.15.46:0/849708147 shutdown_connections 2024-03-18T20:46:59.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.930+0000 7f4827369700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f480c0a1310 0x7f480c0a37d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.930+0000 7f4827369700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4820106e60 0x7f48201006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.930+0000 7f4827369700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4820067e50 0x7f48200ffc70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.930+0000 7f4827369700 1 --2- 172.21.15.46:0/849708147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4820068810 0x7f48201001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:46:59.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.930+0000 7f4827369700 1 -- 172.21.15.46:0/849708147 >> 172.21.15.46:0/849708147 conn(0x7f4820076d40 msgr2=0x7f48200fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:46:59.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.930+0000 7f4827369700 1 -- 172.21.15.46:0/849708147 shutdown_connections 2024-03-18T20:46:59.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:46:59.930+0000 7f4827369700 1 -- 172.21.15.46:0/849708147 wait complete. 2024-03-18T20:46:59.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:46:59.950 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:47:00.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.107+0000 7fd71ed54700 1 -- 172.21.15.46:0/1850432296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7180ffe50 msgr2=0x7fd718100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:00.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.107+0000 7fd71ed54700 1 --2- 172.21.15.46:0/1850432296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7180ffe50 0x7fd718100230 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009990 tx=0x7fd70c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.107+0000 7fd71ed54700 1 -- 172.21.15.46:0/1850432296 shutdown_connections 2024-03-18T20:47:00.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.107+0000 7fd71ed54700 1 --2- 172.21.15.46:0/1850432296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7180ffe50 0x7fd718100230 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.107+0000 7fd71ed54700 1 --2- 172.21.15.46:0/1850432296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd718104f60 0x7fd71810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.107+0000 7fd71ed54700 1 --2- 172.21.15.46:0/1850432296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd718100800 0x7fd718104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.107+0000 7fd71ed54700 1 -- 172.21.15.46:0/1850432296 >> 172.21.15.46:0/1850432296 conn(0x7fd718076d40 msgr2=0x7fd718077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.107+0000 7fd71ed54700 1 -- 172.21.15.46:0/1850432296 shutdown_connections 2024-03-18T20:47:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.107+0000 7fd71ed54700 1 -- 172.21.15.46:0/1850432296 wait complete. 2024-03-18T20:47:00.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd71ed54700 1 Processor -- start 2024-03-18T20:47:00.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd71ed54700 1 -- start start 2024-03-18T20:47:00.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd71ed54700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7180ffe50 0x7fd718197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd71ed54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd718104f60 0x7fd718197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd71ed54700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd71819baf0 0x7fd71819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd71ed54700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71819c570 con 0x7fd718104f60 2024-03-18T20:47:00.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd71ed54700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71819c6e0 con 0x7fd71819baf0 2024-03-18T20:47:00.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd71ed54700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71819c8a0 con 0x7fd7180ffe50 2024-03-18T20:47:00.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd717fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd718104f60 0x7fd718197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd71caf0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7180ffe50 0x7fd718197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd717fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd718104f60 0x7fd718197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52130/0 (socket says 172.21.15.46:52130) 2024-03-18T20:47:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd71d2f1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd71819baf0 0x7fd71819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.108+0000 7fd717fff700 1 -- 172.21.15.46:0/3523377689 learned_addr learned my addr 172.21.15.46:0/3523377689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd71d2f1700 1 -- 172.21.15.46:0/3523377689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7180ffe50 msgr2=0x7fd718197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd71d2f1700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7180ffe50 0x7fd718197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd71d2f1700 1 -- 172.21.15.46:0/3523377689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd718104f60 msgr2=0x7fd718197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd71d2f1700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd718104f60 0x7fd718197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd71d2f1700 1 -- 172.21.15.46:0/3523377689 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c0095f0 con 0x7fd71819baf0 2024-03-18T20:47:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd717fff700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd718104f60 0x7fd718197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd71caf0700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7180ffe50 0x7fd718197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd71d2f1700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd71819baf0 0x7fd71819bf70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd71000efc0 tx=0x7fd71000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:00.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd715ffb700 1 -- 172.21.15.46:0/3523377689 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710018070 con 0x7fd71819baf0 2024-03-18T20:47:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd715ffb700 1 -- 172.21.15.46:0/3523377689 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd71000f040 con 0x7fd71819baf0 2024-03-18T20:47:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd715ffb700 1 -- 172.21.15.46:0/3523377689 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71000cd40 con 0x7fd71819baf0 2024-03-18T20:47:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd71ed54700 1 -- 172.21.15.46:0/3523377689 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7181a3da0 con 0x7fd71819baf0 2024-03-18T20:47:00.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.109+0000 7fd71ed54700 1 -- 172.21.15.46:0/3523377689 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7181a4230 con 0x7fd71819baf0 2024-03-18T20:47:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.112+0000 7fd715ffb700 1 -- 172.21.15.46:0/3523377689 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd710003a00 con 0x7fd71819baf0 2024-03-18T20:47:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.112+0000 7fd715ffb700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd7080a1310 0x7fd7080a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.113+0000 7fd71caf0700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd7080a1310 0x7fd7080a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.113+0000 7fd715ffb700 1 -- 172.21.15.46:0/3523377689 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd7100cc7f0 con 0x7fd71819baf0 2024-03-18T20:47:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.113+0000 7fd71caf0700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd7080a1310 0x7fd7080a37d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd70c005b20 tx=0x7fd70c005a90 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd72aee9b80 1 Processor -- start 2024-03-18T20:47:00.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd72aee9b80 1 -- start start 2024-03-18T20:47:00.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd72aee9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6363a60 0x557bc6363e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd72aee9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc636ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd72aee9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc636f220 0x557bc6371610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd72aee9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc62bd990 con 0x557bc6364380 2024-03-18T20:47:00.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd72aee9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc62bb350 con 0x557bc6363a60 2024-03-18T20:47:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd72aee9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc62b6c40 con 0x557bc636f220 2024-03-18T20:47:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd717fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc636ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd71caf0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6363a60 0x557bc6363e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd71caf0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6363a60 0x557bc6363e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49942/0 (socket says 172.21.15.46:49942) 2024-03-18T20:47:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd71caf0700 1 -- 172.21.15.46:0/1972671067 learned_addr learned my addr 172.21.15.46:0/1972671067 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.114+0000 7fd717fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc636ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52132/0 (socket says 172.21.15.46:52132) 2024-03-18T20:47:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.115+0000 7fd71d2f1700 1 --2- 172.21.15.46:0/1972671067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc636f220 0x557bc6371610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.115+0000 7fd71caf0700 1 -- 172.21.15.46:0/1972671067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc636f220 msgr2=0x557bc6371610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.115+0000 7fd71caf0700 1 --2- 172.21.15.46:0/1972671067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc636f220 0x557bc6371610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.115+0000 7fd71caf0700 1 -- 172.21.15.46:0/1972671067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 msgr2=0x557bc636ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.115+0000 7fd71caf0700 1 --2- 172.21.15.46:0/1972671067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc636ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.115+0000 7fd71caf0700 1 -- 172.21.15.46:0/1972671067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bc61383b0 con 0x557bc6363a60 2024-03-18T20:47:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.115+0000 7fd71d2f1700 1 --2- 172.21.15.46:0/1972671067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc636f220 0x557bc6371610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.115+0000 7fd71caf0700 1 --2- 172.21.15.46:0/1972671067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6363a60 0x557bc6363e40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd70c015000 tx=0x7fd70c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebf8d62d0b8ab431 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.115+0000 7fd702ffd700 1 -- 172.21.15.46:0/1972671067 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c02b070 con 0x557bc6363a60 2024-03-18T20:47:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.115+0000 7fd702ffd700 1 -- 172.21.15.46:0/1972671067 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd70c01e040 con 0x557bc6363a60 2024-03-18T20:47:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.115+0000 7fd702ffd700 1 -- 172.21.15.46:0/1972671067 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c024cb0 con 0x557bc6363a60 2024-03-18T20:47:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.116+0000 7fd72aee9b80 1 -- 172.21.15.46:0/1972671067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6363a60 msgr2=0x557bc6363e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.116+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/1972671067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6363a60 0x557bc6363e40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd70c015000 tx=0x7fd70c020000 comp rx=0 tx=0).stop 2024-03-18T20:47:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.116+0000 7fd72aee9b80 1 -- 172.21.15.46:0/1972671067 shutdown_connections 2024-03-18T20:47:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.116+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/1972671067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc636f220 0x557bc6371610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.116+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/1972671067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6363a60 0x557bc6363e40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.116+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/1972671067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc636ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.116+0000 7fd72aee9b80 1 -- 172.21.15.46:0/1972671067 >> 172.21.15.46:0/1972671067 conn(0x557bc62da220 msgr2=0x557bc6360020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.116+0000 7fd72aee9b80 1 -- 172.21.15.46:0/1972671067 shutdown_connections 2024-03-18T20:47:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.116+0000 7fd72aee9b80 1 -- 172.21.15.46:0/1972671067 wait complete. 2024-03-18T20:47:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.117+0000 7fd72aee9b80 1 -- start start 2024-03-18T20:47:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.117+0000 7fd72aee9b80 1 Processor -- start 2024-03-18T20:47:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd72aee9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc636ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd72aee9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6364380 0x557bc636bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd72aee9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc636f220 0x557bc636c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd72aee9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc6252d80 con 0x557bc636f220 2024-03-18T20:47:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd72aee9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc6252a20 con 0x557bc6364380 2024-03-18T20:47:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd72aee9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc613a6a0 con 0x557bc6363a60 2024-03-18T20:47:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd72aee9b80 10 client.0 _ll_register_callbacks cb 0x557bc60be3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd717fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6364380 0x557bc636bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd71d2f1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc636f220 0x557bc636c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd717fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6364380 0x557bc636bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49958/0 (socket says 172.21.15.46:49958) 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd717fff700 1 -- 172.21.15.46:0/3407799865 learned_addr learned my addr 172.21.15.46:0/3407799865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd71caf0700 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc636ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd717fff700 1 -- 172.21.15.46:0/3407799865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 msgr2=0x557bc636ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd717fff700 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc636ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd717fff700 1 -- 172.21.15.46:0/3407799865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc636f220 msgr2=0x557bc636c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd717fff700 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc636f220 0x557bc636c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd717fff700 1 -- 172.21.15.46:0/3407799865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7181a4230 con 0x557bc6364380 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd71caf0700 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc636ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd717fff700 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6364380 0x557bc636bf40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd704012d20 tx=0x7fd70400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89dd6edfa9c0c410 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.118+0000 7fd6f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd6f7fff700 1 -- 172.21.15.46:0/3407799865 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704017070 con 0x557bc6364380 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd6f7fff700 1 -- 172.21.15.46:0/3407799865 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd70400fc30 con 0x557bc6364380 2024-03-18T20:47:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd72aee9b80 1 -- 172.21.15.46:0/3407799865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557bc6357ad0 con 0x557bc6364380 2024-03-18T20:47:00.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd6f7fff700 1 -- 172.21.15.46:0/3407799865 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd70400fdf0 con 0x557bc6364380 2024-03-18T20:47:00.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd72aee9b80 10 client.29048 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:00.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd72aee9b80 1 -- 172.21.15.46:0/3407799865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7fd71810d350 con 0x557bc6364380 2024-03-18T20:47:00.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd6f7fff700 1 -- 172.21.15.46:0/3407799865 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557bc6357ad0 con 0x557bc6364380 2024-03-18T20:47:00.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd6f7fff700 1 -- 172.21.15.46:0/3407799865 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd704003c80 con 0x557bc6364380 2024-03-18T20:47:00.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd72aee9b80 10 client.29048 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd72aee9b80 10 client.29048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd72aee9b80 20 client.29048 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557bc637ee40 0x557bc6381260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd72aee9b80 4 client.29048 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.119+0000 7fd72aee9b80 1 -- 172.21.15.46:0/3407799865 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bc62af700 con 0x557bc637ee40 2024-03-18T20:47:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.120+0000 7fd71caf0700 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557bc637ee40 0x557bc6381260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.120+0000 7fd6f7fff700 10 client.29048 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.120+0000 7fd71caf0700 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557bc637ee40 0x557bc6381260 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63e3bd906a01fda server_cookie=3d27fa2a2292d1d1 in_seq=0 out_seq=0 2024-03-18T20:47:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd6f7fff700 1 -- 172.21.15.46:0/3407799865 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557bc62af700 con 0x557bc637ee40 2024-03-18T20:47:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd6f7fff700 10 client.29048 handle_command_reply: tid=0 2024-03-18T20:47:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd72aee9b80 1 client.29048 shutdown 2024-03-18T20:47:00.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd72aee9b80 1 -- 172.21.15.46:0/3407799865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6364380 msgr2=0x557bc636bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:00.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6364380 0x557bc636bf40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd704012d20 tx=0x7fd70400f6a0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd72aee9b80 1 -- 172.21.15.46:0/3407799865 shutdown_connections 2024-03-18T20:47:00.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557bc637ee40 0x557bc6381260 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc636f220 0x557bc636c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd6f7fff700 0 client.29048 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:00.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc636ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/3407799865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc6364380 0x557bc636bf40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd72aee9b80 1 -- 172.21.15.46:0/3407799865 >> 172.21.15.46:0/3407799865 conn(0x557bc62da220 msgr2=0x557bc635fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:00.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd72aee9b80 1 -- 172.21.15.46:0/3407799865 shutdown_connections 2024-03-18T20:47:00.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd72aee9b80 1 -- 172.21.15.46:0/3407799865 wait complete. 2024-03-18T20:47:00.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.121+0000 7fd72aee9b80 20 client.29048 trim_cache size 0 max 16384 2024-03-18T20:47:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.143+0000 7fd72aee9b80 1 Processor -- start 2024-03-18T20:47:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.143+0000 7fd72aee9b80 1 -- start start 2024-03-18T20:47:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.143+0000 7fd72aee9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc63669c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.143+0000 7fd72aee9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc6375df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.143+0000 7fd72aee9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc636f220 0x557bc6367220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.143+0000 7fd72aee9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc637b900 con 0x557bc6364380 2024-03-18T20:47:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.143+0000 7fd72aee9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc6364d30 con 0x557bc636f220 2024-03-18T20:47:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.143+0000 7fd72aee9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc6359f80 con 0x557bc6363a60 2024-03-18T20:47:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.143+0000 7fd717fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc6375df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.143+0000 7fd717fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc6375df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52160/0 (socket says 172.21.15.46:52160) 2024-03-18T20:47:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.143+0000 7fd717fff700 1 -- 172.21.15.46:0/1866536120 learned_addr learned my addr 172.21.15.46:0/1866536120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.144+0000 7fd71caf0700 1 --2- 172.21.15.46:0/1866536120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc63669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.144+0000 7fd717fff700 1 -- 172.21.15.46:0/1866536120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 msgr2=0x557bc63669c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.144+0000 7fd717fff700 1 --2- 172.21.15.46:0/1866536120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc63669c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.144+0000 7fd717fff700 1 -- 172.21.15.46:0/1866536120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc636f220 msgr2=0x557bc6367220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:47:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.144+0000 7fd717fff700 1 --2- 172.21.15.46:0/1866536120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc636f220 0x557bc6367220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.144+0000 7fd717fff700 1 -- 172.21.15.46:0/1866536120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bc60e4b60 con 0x557bc6364380 2024-03-18T20:47:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.144+0000 7fd717fff700 1 --2- 172.21.15.46:0/1866536120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc6375df0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fd704010430 tx=0x7fd704012320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90edc19b0ecea802 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.144+0000 7fd7037fe700 1 -- 172.21.15.46:0/1866536120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704017040 con 0x557bc6364380 2024-03-18T20:47:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.144+0000 7fd7037fe700 1 -- 172.21.15.46:0/1866536120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd704010e20 con 0x557bc6364380 2024-03-18T20:47:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.145+0000 7fd7037fe700 1 -- 172.21.15.46:0/1866536120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704004310 con 0x557bc6364380 2024-03-18T20:47:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.145+0000 7fd72aee9b80 1 -- 172.21.15.46:0/1866536120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 msgr2=0x557bc6375df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.145+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/1866536120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc6375df0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fd704010430 tx=0x7fd704012320 comp rx=0 tx=0).stop 2024-03-18T20:47:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.146+0000 7fd72aee9b80 1 -- 172.21.15.46:0/1866536120 shutdown_connections 2024-03-18T20:47:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.146+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/1866536120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc636f220 0x557bc6367220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.146+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/1866536120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc63669c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.146+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/1866536120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc6375df0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.146+0000 7fd72aee9b80 1 -- 172.21.15.46:0/1866536120 >> 172.21.15.46:0/1866536120 conn(0x557bc62da220 msgr2=0x557bc636e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.146+0000 7fd72aee9b80 1 -- 172.21.15.46:0/1866536120 shutdown_connections 2024-03-18T20:47:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.146+0000 7fd72aee9b80 1 -- 172.21.15.46:0/1866536120 wait complete. 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.146+0000 7fd72aee9b80 1 -- start start 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.146+0000 7fd72aee9b80 1 Processor -- start 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.147+0000 7fd72aee9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc6375930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.147+0000 7fd72aee9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc6373d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.147+0000 7fd72aee9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc636f220 0x557bc6374260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.147+0000 7fd72aee9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc63747a0 con 0x557bc6364380 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.147+0000 7fd72aee9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc6374910 con 0x557bc636f220 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.147+0000 7fd72aee9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bc6374ad0 con 0x557bc6363a60 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.147+0000 7fd72aee9b80 10 client.0 _ll_register_callbacks cb 0x7fd7040127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.147+0000 7fd71caf0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc6375930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.147+0000 7fd71caf0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc6375930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:52296/0 (socket says 172.21.15.46:52296) 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.147+0000 7fd71caf0700 1 -- 172.21.15.46:0/63584120 learned_addr learned my addr 172.21.15.46:0/63584120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.147+0000 7fd717fff700 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc6373d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.147+0000 7fd71d2f1700 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc636f220 0x557bc6374260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.148+0000 7fd71caf0700 1 -- 172.21.15.46:0/63584120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc636f220 msgr2=0x557bc6374260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.148+0000 7fd71caf0700 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc636f220 0x557bc6374260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.148+0000 7fd71caf0700 1 -- 172.21.15.46:0/63584120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 msgr2=0x557bc6373d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.148+0000 7fd71caf0700 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc6373d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.148+0000 7fd71caf0700 1 -- 172.21.15.46:0/63584120 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71810d350 con 0x557bc6363a60 2024-03-18T20:47:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.148+0000 7fd71d2f1700 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc636f220 0x557bc6374260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:47:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.148+0000 7fd71caf0700 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc6375930 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd70c000c00 tx=0x7fd70c00fce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bda4b7ee1bc7ffe server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.148+0000 7fd6f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:47:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.148+0000 7fd6f7fff700 1 -- 172.21.15.46:0/63584120 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c02b040 con 0x557bc6363a60 2024-03-18T20:47:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.148+0000 7fd6f7fff700 1 -- 172.21.15.46:0/63584120 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd70c03c0f0 con 0x557bc6363a60 2024-03-18T20:47:00.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.148+0000 7fd72aee9b80 1 -- 172.21.15.46:0/63584120 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd70c01e040 con 0x557bc6363a60 2024-03-18T20:47:00.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.148+0000 7fd6f7fff700 1 -- 172.21.15.46:0/63584120 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd70c03c2b0 con 0x557bc6363a60 2024-03-18T20:47:00.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.149+0000 7fd72aee9b80 10 client.29071 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:00.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.149+0000 7fd72aee9b80 1 -- 172.21.15.46:0/63584120 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557bc627cea0 con 0x557bc6363a60 2024-03-18T20:47:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.149+0000 7fd6f7fff700 1 -- 172.21.15.46:0/63584120 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd70c01e040 con 0x557bc6363a60 2024-03-18T20:47:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.149+0000 7fd6f7fff700 1 -- 172.21.15.46:0/63584120 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd70c00f740 con 0x557bc6363a60 2024-03-18T20:47:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.149+0000 7fd72aee9b80 10 client.29071 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.149+0000 7fd72aee9b80 10 client.29071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.149+0000 7fd72aee9b80 20 client.29071 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.149+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557bc637ee40 0x557bc63d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.149+0000 7fd72aee9b80 4 client.29071 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:47:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.149+0000 7fd72aee9b80 1 -- 172.21.15.46:0/63584120 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557bc6105cf0 con 0x557bc637ee40 2024-03-18T20:47:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.149+0000 7fd717fff700 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557bc637ee40 0x557bc63d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd6f7fff700 10 client.29071 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd717fff700 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557bc637ee40 0x557bc63d4e00 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d6f662d837f3eaf server_cookie=646863ce7144cb9 in_seq=0 out_seq=0 2024-03-18T20:47:00.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd6f7fff700 1 -- 172.21.15.46:0/63584120 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557bc6105cf0 con 0x557bc637ee40 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd6f7fff700 10 client.29071 handle_command_reply: tid=0 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd72aee9b80 1 client.29071 shutdown 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd72aee9b80 1 -- 172.21.15.46:0/63584120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 msgr2=0x557bc6375930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc6375930 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd70c000c00 tx=0x7fd70c00fce0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd72aee9b80 1 -- 172.21.15.46:0/63584120 shutdown_connections 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557bc637ee40 0x557bc63d4e00 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557bc636f220 0x557bc6374260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557bc6363a60 0x557bc6375930 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd72aee9b80 1 --2- 172.21.15.46:0/63584120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557bc6364380 0x557bc6373d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd72aee9b80 1 -- 172.21.15.46:0/63584120 >> 172.21.15.46:0/63584120 conn(0x557bc62da220 msgr2=0x557bc636a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd6f7fff700 0 client.29071 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.150+0000 7fd72aee9b80 1 -- 172.21.15.46:0/63584120 shutdown_connections 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.151+0000 7fd72aee9b80 1 -- 172.21.15.46:0/63584120 wait complete. 2024-03-18T20:47:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.151+0000 7fd72aee9b80 20 client.29071 trim_cache size 0 max 16384 2024-03-18T20:47:00.158 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:47:00.158 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:47:00.158 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:47:00.158 INFO:teuthology.orchestra.run.smithi046.stdout: "6cad749a-3ed2-44e2-a0de-9b87ef81ac96": { 2024-03-18T20:47:00.158 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:47:00.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "6cad749a-3ed2-44e2-a0de-9b87ef81ac96", 2024-03-18T20:47:00.158 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:47:00.159 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:47:00.159 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:47:00.159 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:47:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.152+0000 7fd71ed54700 1 -- 172.21.15.46:0/3523377689 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd7080a1310 msgr2=0x7fd7080a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.152+0000 7fd71ed54700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd7080a1310 0x7fd7080a37d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd70c005b20 tx=0x7fd70c005a90 comp rx=0 tx=0).stop 2024-03-18T20:47:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.152+0000 7fd71ed54700 1 -- 172.21.15.46:0/3523377689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd71819baf0 msgr2=0x7fd71819bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.152+0000 7fd71ed54700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd71819baf0 0x7fd71819bf70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd71000efc0 tx=0x7fd71000c3a0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.152+0000 7fd71ed54700 1 -- 172.21.15.46:0/3523377689 shutdown_connections 2024-03-18T20:47:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.152+0000 7fd71ed54700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd7080a1310 0x7fd7080a37d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.152+0000 7fd71ed54700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd71819baf0 0x7fd71819bf70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.152+0000 7fd71ed54700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7180ffe50 0x7fd718197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.152+0000 7fd71ed54700 1 --2- 172.21.15.46:0/3523377689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd718104f60 0x7fd718197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.152+0000 7fd71ed54700 1 -- 172.21.15.46:0/3523377689 >> 172.21.15.46:0/3523377689 conn(0x7fd718076d40 msgr2=0x7fd718068270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.152+0000 7fd71ed54700 1 -- 172.21.15.46:0/3523377689 shutdown_connections 2024-03-18T20:47:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:00.152+0000 7fd71ed54700 1 -- 172.21.15.46:0/3523377689 wait complete. 2024-03-18T20:47:00.172 INFO:tasks.cephfs.filesystem:scrub status for tag:6cad749a-3ed2-44e2-a0de-9b87ef81ac96 - {'path': '/', 'tag': '6cad749a-3ed2-44e2-a0de-9b87ef81ac96', 'options': 'recursive,force'} 2024-03-18T20:47:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: pgmap v338: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.8 MiB/s wr, 184 op/s 2024-03-18T20:47:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:47:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:00.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:00.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:00 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: pgmap v338: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.8 MiB/s wr, 184 op/s 2024-03-18T20:47:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:47:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:00 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.716 INFO:tasks.workunit.client.0.smithi046.stdout: 1 407641 3.69 MB/sec execute 475 sec latency 3537.814 ms 2024-03-18T20:47:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: pgmap v338: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 1.8 MiB/s wr, 184 op/s 2024-03-18T20:47:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:47:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:00 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:01.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:01 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:01.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:01 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:01.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:01 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:01.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:01 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:01.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:01 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:01.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:01 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:01.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:01 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:01.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:01 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:01.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:01 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:01.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:01 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:01.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:01 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:01.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:01 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:01 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:01 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:01 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:01.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:01 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:01.716 INFO:tasks.workunit.client.0.smithi046.stdout: 1 408518 3.70 MB/sec execute 476 sec latency 25.898 ms 2024-03-18T20:47:01.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:01 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:01.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:01 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:01.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:01 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:01.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:01 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:01.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:01 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:01.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:01 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:01.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:01 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:01.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:01 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:02 smithi155 ceph-mon[48344]: pgmap v339: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-03-18T20:47:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:02 smithi046 ceph-mon[43495]: pgmap v339: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-03-18T20:47:02.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 409641 3.70 MB/sec execute 477 sec latency 133.005 ms 2024-03-18T20:47:02.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:02 smithi073 ceph-mon[47384]: pgmap v339: 97 pgs: 97 active+clean; 154 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-03-18T20:47:03.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 409641 3.69 MB/sec execute 478 sec latency 1133.044 ms 2024-03-18T20:47:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:04 smithi155 ceph-mon[48344]: pgmap v340: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-03-18T20:47:04.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:04 smithi155 ceph-mon[48344]: osdmap e91: 12 total, 12 up, 12 in 2024-03-18T20:47:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:04 smithi046 ceph-mon[43495]: pgmap v340: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-03-18T20:47:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:04 smithi046 ceph-mon[43495]: osdmap e91: 12 total, 12 up, 12 in 2024-03-18T20:47:04.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 409641 3.68 MB/sec execute 479 sec latency 2133.139 ms 2024-03-18T20:47:04.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:04 smithi073 ceph-mon[47384]: pgmap v340: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-03-18T20:47:04.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:04 smithi073 ceph-mon[47384]: osdmap e91: 12 total, 12 up, 12 in 2024-03-18T20:47:05.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 410446 3.68 MB/sec execute 480 sec latency 2759.544 ms 2024-03-18T20:47:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:05 smithi155 ceph-mon[48344]: pgmap v342: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-03-18T20:47:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:05 smithi046 ceph-mon[43495]: pgmap v342: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-03-18T20:47:06.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:05 smithi073 ceph-mon[47384]: pgmap v342: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-03-18T20:47:06.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 411944 3.70 MB/sec execute 481 sec latency 16.810 ms 2024-03-18T20:47:07.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 413183 3.70 MB/sec execute 482 sec latency 39.480 ms 2024-03-18T20:47:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:08 smithi155 ceph-mon[48344]: pgmap v343: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-03-18T20:47:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:08 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:47:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:08 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:47:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:08 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:47:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:08 smithi046 ceph-mon[43495]: pgmap v343: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-03-18T20:47:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:08 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:47:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:08 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:47:08.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:08 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:47:08.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 413183 3.69 MB/sec execute 483 sec latency 1039.587 ms 2024-03-18T20:47:08.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:08 smithi073 ceph-mon[47384]: pgmap v343: 97 pgs: 97 active+clean; 157 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-03-18T20:47:08.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:08 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:47:08.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:08 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:47:08.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:08 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:47:09.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:09 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:47:09.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:09 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:47:09.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:09 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:47:09.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:09 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:47:09.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 413183 3.68 MB/sec execute 484 sec latency 2039.714 ms 2024-03-18T20:47:09.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:09 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:47:09.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:09 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:47:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.954+0000 7f5c84423700 1 -- 172.21.15.46:0/326919818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c7c1007d0 msgr2=0x7f5c7c104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.954+0000 7f5c84423700 1 --2- 172.21.15.46:0/326919818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c7c1007d0 0x7f5c7c104810 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009990 tx=0x7f5c6c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.954+0000 7f5c84423700 1 -- 172.21.15.46:0/326919818 shutdown_connections 2024-03-18T20:47:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.954+0000 7f5c84423700 1 --2- 172.21.15.46:0/326919818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c7c104f40 0x7f5c7c10cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.954+0000 7f5c84423700 1 --2- 172.21.15.46:0/326919818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c7c0ffe20 0x7f5c7c100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.954+0000 7f5c84423700 1 --2- 172.21.15.46:0/326919818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c7c1007d0 0x7f5c7c104810 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.954+0000 7f5c84423700 1 -- 172.21.15.46:0/326919818 >> 172.21.15.46:0/326919818 conn(0x7f5c7c0fb680 msgr2=0x7f5c7c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.954+0000 7f5c84423700 1 -- 172.21.15.46:0/326919818 shutdown_connections 2024-03-18T20:47:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.954+0000 7f5c84423700 1 -- 172.21.15.46:0/326919818 wait complete. 2024-03-18T20:47:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.954+0000 7f5c84423700 1 Processor -- start 2024-03-18T20:47:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c84423700 1 -- start start 2024-03-18T20:47:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c84423700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c7c0ffe20 0x7f5c7c197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c84423700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c7c104f40 0x7f5c7c1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c84423700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c7c19ba50 0x7f5c7c19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c84423700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c19c4d0 con 0x7f5c7c19ba50 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c84423700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c19c640 con 0x7f5c7c0ffe20 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c84423700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c19c800 con 0x7f5c7c104f40 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c829c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c7c19ba50 0x7f5c7c19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c821bf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c7c0ffe20 0x7f5c7c197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c829c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c7c19ba50 0x7f5c7c19bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38964/0 (socket says 172.21.15.46:38964) 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c829c0700 1 -- 172.21.15.46:0/1837770124 learned_addr learned my addr 172.21.15.46:0/1837770124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c819be700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c7c104f40 0x7f5c7c1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c829c0700 1 -- 172.21.15.46:0/1837770124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c7c104f40 msgr2=0x7f5c7c1976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c829c0700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c7c104f40 0x7f5c7c1976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c829c0700 1 -- 172.21.15.46:0/1837770124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c7c0ffe20 msgr2=0x7f5c7c197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c829c0700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c7c0ffe20 0x7f5c7c197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c829c0700 1 -- 172.21.15.46:0/1837770124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0095f0 con 0x7f5c7c19ba50 2024-03-18T20:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.955+0000 7f5c821bf700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c7c0ffe20 0x7f5c7c197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.956+0000 7f5c829c0700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c7c19ba50 0x7f5c7c19bed0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5c74007ee0 tx=0x7f5c7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.956+0000 7f5c737fe700 1 -- 172.21.15.46:0/1837770124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74017070 con 0x7f5c7c19ba50 2024-03-18T20:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.956+0000 7f5c737fe700 1 -- 172.21.15.46:0/1837770124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c7400f040 con 0x7f5c7c19ba50 2024-03-18T20:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.956+0000 7f5c84423700 1 -- 172.21.15.46:0/1837770124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7c1a3d60 con 0x7f5c7c19ba50 2024-03-18T20:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.956+0000 7f5c737fe700 1 -- 172.21.15.46:0/1837770124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74005580 con 0x7f5c7c19ba50 2024-03-18T20:47:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.956+0000 7f5c84423700 1 -- 172.21.15.46:0/1837770124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c1a41f0 con 0x7f5c7c19ba50 2024-03-18T20:47:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.958+0000 7f5c737fe700 1 -- 172.21.15.46:0/1837770124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5c74005760 con 0x7f5c7c19ba50 2024-03-18T20:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.958+0000 7f5c737fe700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5c680a1290 0x7f5c680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.959+0000 7f5c821bf700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5c680a1290 0x7f5c680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.959+0000 7f5c737fe700 1 -- 172.21.15.46:0/1837770124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f5c740cc3e0 con 0x7f5c7c19ba50 2024-03-18T20:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.959+0000 7f5c821bf700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5c680a1290 0x7f5c680a3750 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5c78005950 tx=0x7f5c7800b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c905b8b80 1 Processor -- start 2024-03-18T20:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c905b8b80 1 -- start start 2024-03-18T20:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c905b8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3570 0x565270bc3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c905b8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3e90 0x565270bce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c905b8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 0x565270bd11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c905b8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x565270b1d930 con 0x565270bcedb0 2024-03-18T20:47:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c905b8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x565270b1b2f0 con 0x565270bc3e90 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c905b8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x565270b16be0 con 0x565270bc3570 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c829c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 0x565270bd11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c829c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 0x565270bd11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38972/0 (socket says 172.21.15.46:38972) 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c829c0700 1 -- 172.21.15.46:0/3503404022 learned_addr learned my addr 172.21.15.46:0/3503404022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c821bf700 1 --2- 172.21.15.46:0/3503404022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3570 0x565270bc3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.960+0000 7f5c819be700 1 --2- 172.21.15.46:0/3503404022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3e90 0x565270bce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.961+0000 7f5c829c0700 1 -- 172.21.15.46:0/3503404022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3570 msgr2=0x565270bc3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.961+0000 7f5c829c0700 1 --2- 172.21.15.46:0/3503404022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3570 0x565270bc3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.961+0000 7f5c829c0700 1 -- 172.21.15.46:0/3503404022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3e90 msgr2=0x565270bce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.961+0000 7f5c829c0700 1 --2- 172.21.15.46:0/3503404022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3e90 0x565270bce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.961+0000 7f5c829c0700 1 -- 172.21.15.46:0/3503404022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565270998350 con 0x565270bcedb0 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.961+0000 7f5c829c0700 1 --2- 172.21.15.46:0/3503404022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 0x565270bd11a0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5c740959a0 tx=0x7f5c740951e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0655836b997026 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.961+0000 7f5c70da8700 1 -- 172.21.15.46:0/3503404022 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7409d2d0 con 0x565270bcedb0 2024-03-18T20:47:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.961+0000 7f5c70da8700 1 -- 172.21.15.46:0/3503404022 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c740a1920 con 0x565270bcedb0 2024-03-18T20:47:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.961+0000 7f5c70da8700 1 -- 172.21.15.46:0/3503404022 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74095bb0 con 0x565270bcedb0 2024-03-18T20:47:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.962+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3503404022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 msgr2=0x565270bd11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.962+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3503404022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 0x565270bd11a0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5c740959a0 tx=0x7f5c740951e0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.962+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3503404022 shutdown_connections 2024-03-18T20:47:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.962+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3503404022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 0x565270bd11a0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.962+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3503404022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3570 0x565270bc3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.962+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3503404022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3e90 0x565270bce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.962+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3503404022 >> 172.21.15.46:0/3503404022 conn(0x565270b3a1c0 msgr2=0x565270bbfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.962+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3503404022 shutdown_connections 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.962+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3503404022 wait complete. 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.962+0000 7f5c905b8b80 1 -- start start 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.962+0000 7f5c905b8b80 1 Processor -- start 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c905b8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3570 0x565270bce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c905b8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3e90 0x565270bcbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c905b8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 0x565270bcc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c905b8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x565270ab2d20 con 0x565270bcedb0 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c905b8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x565270ab29c0 con 0x565270bc3570 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c905b8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527099a640 con 0x565270bc3e90 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c905b8b80 10 client.0 _ll_register_callbacks cb 0x56527091e340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:09.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c829c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 0x565270bcc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c829c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 0x565270bcc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38984/0 (socket says 172.21.15.46:38984) 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c829c0700 1 -- 172.21.15.46:0/3769738786 learned_addr learned my addr 172.21.15.46:0/3769738786 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c821bf700 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3570 0x565270bce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c819be700 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3e90 0x565270bcbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c829c0700 1 -- 172.21.15.46:0/3769738786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3e90 msgr2=0x565270bcbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c829c0700 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3e90 0x565270bcbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c829c0700 1 -- 172.21.15.46:0/3769738786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3570 msgr2=0x565270bce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c829c0700 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3570 0x565270bce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.963+0000 7f5c829c0700 1 -- 172.21.15.46:0/3769738786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c1a41f0 con 0x565270bcedb0 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c829c0700 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 0x565270bcc210 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5c740c9b90 tx=0x7f5c74004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19eae8bab301a23 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c65ffb700 1 -- 172.21.15.46:0/3769738786 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7409d2d0 con 0x565270bcedb0 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c65ffb700 1 -- 172.21.15.46:0/3769738786 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f5c74084050 con 0x565270bcedb0 2024-03-18T20:47:09.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3769738786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565270bb76c0 con 0x565270bcedb0 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c65ffb700 1 -- 172.21.15.46:0/3769738786 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c74022250 con 0x565270bcedb0 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c905b8b80 10 client.19440 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3769738786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x565270944b00 con 0x565270bcedb0 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c65ffb700 1 -- 172.21.15.46:0/3769738786 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565270bb76c0 con 0x565270bcedb0 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c65ffb700 1 -- 172.21.15.46:0/3769738786 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5c74098990 con 0x565270bcedb0 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c905b8b80 10 client.19440 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c905b8b80 10 client.19440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c905b8b80 20 client.19440 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x565270bde6f0 0x565270be0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c905b8b80 4 client.19440 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.964+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3769738786 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565270b0f6a0 con 0x565270bde6f0 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.965+0000 7f5c821bf700 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x565270bde6f0 0x565270be0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.965+0000 7f5c821bf700 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x565270bde6f0 0x565270be0b10 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06d081c0aad8f2 server_cookie=fead821567bc7216 in_seq=0 out_seq=0 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.965+0000 7f5c65ffb700 10 client.19440 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c65ffb700 1 -- 172.21.15.46:0/3769738786 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x565270b0f6a0 con 0x565270bde6f0 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c65ffb700 10 client.19440 handle_command_reply: tid=0 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c905b8b80 1 client.19440 shutdown 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3769738786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 msgr2=0x565270bcc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 0x565270bcc210 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5c740c9b90 tx=0x7f5c74004f10 comp rx=0 tx=0).stop 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3769738786 shutdown_connections 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x565270bde6f0 0x565270be0b10 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bcedb0 0x565270bcc210 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3570 0x565270bce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3769738786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3e90 0x565270bcbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c65ffb700 0 client.19440 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:09.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3769738786 >> 172.21.15.46:0/3769738786 conn(0x565270b3a1c0 msgr2=0x565270bbf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3769738786 shutdown_connections 2024-03-18T20:47:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3769738786 wait complete. 2024-03-18T20:47:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.966+0000 7f5c905b8b80 20 client.19440 trim_cache size 0 max 16384 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.982+0000 7f5c905b8b80 1 Processor -- start 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.982+0000 7f5c905b8b80 1 -- start start 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.982+0000 7f5c905b8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3570 0x565270bbfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.982+0000 7f5c905b8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3e90 0x565270bcb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.982+0000 7f5c905b8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bcedb0 0x565270bcb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.982+0000 7f5c905b8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x565270bbc1d0 con 0x565270bc3e90 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.982+0000 7f5c905b8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x565270bc5070 con 0x565270bc3570 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.982+0000 7f5c905b8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x565270bc4600 con 0x565270bcedb0 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.982+0000 7f5c819be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3e90 0x565270bcb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.982+0000 7f5c819be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3e90 0x565270bcb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38990/0 (socket says 172.21.15.46:38990) 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.982+0000 7f5c819be700 1 -- 172.21.15.46:0/3774434916 learned_addr learned my addr 172.21.15.46:0/3774434916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.982+0000 7f5c829c0700 1 --2- 172.21.15.46:0/3774434916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bcedb0 0x565270bcb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.983+0000 7f5c821bf700 1 --2- 172.21.15.46:0/3774434916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3570 0x565270bbfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.983+0000 7f5c819be700 1 -- 172.21.15.46:0/3774434916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bcedb0 msgr2=0x565270bcb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.983+0000 7f5c819be700 1 --2- 172.21.15.46:0/3774434916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bcedb0 0x565270bcb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.983+0000 7f5c819be700 1 -- 172.21.15.46:0/3774434916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3570 msgr2=0x565270bbfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.983+0000 7f5c819be700 1 --2- 172.21.15.46:0/3774434916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3570 0x565270bbfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.983+0000 7f5c819be700 1 -- 172.21.15.46:0/3774434916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565270adce40 con 0x565270bc3e90 2024-03-18T20:47:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.983+0000 7f5c821bf700 1 --2- 172.21.15.46:0/3774434916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3570 0x565270bbfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:47:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.983+0000 7f5c819be700 1 --2- 172.21.15.46:0/3774434916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3e90 0x565270bcb400 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009990 tx=0x7f5c6c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fe7a707c39c1fe server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.983+0000 7f5c715a9700 1 -- 172.21.15.46:0/3774434916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c01d030 con 0x565270bc3e90 2024-03-18T20:47:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.983+0000 7f5c715a9700 1 -- 172.21.15.46:0/3774434916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c6c01e040 con 0x565270bc3e90 2024-03-18T20:47:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.983+0000 7f5c715a9700 1 -- 172.21.15.46:0/3774434916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c004870 con 0x565270bc3e90 2024-03-18T20:47:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3774434916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3e90 msgr2=0x565270bcb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3774434916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3e90 0x565270bcb400 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009990 tx=0x7f5c6c005a90 comp rx=0 tx=0).stop 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3774434916 shutdown_connections 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3774434916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bcedb0 0x565270bcb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3774434916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bc3570 0x565270bbfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/3774434916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3e90 0x565270bcb400 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3774434916 >> 172.21.15.46:0/3774434916 conn(0x565270b3a1c0 msgr2=0x565270bbf500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3774434916 shutdown_connections 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 -- 172.21.15.46:0/3774434916 wait complete. 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 -- start start 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 Processor -- start 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3570 0x565270bd1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3e90 0x565270bd21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bcedb0 0x565270bd5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x565270bd56c0 con 0x565270bc3570 2024-03-18T20:47:09.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x565270bd5830 con 0x565270bcedb0 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x565270bd59f0 con 0x565270bc3e90 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.984+0000 7f5c905b8b80 10 client.0 _ll_register_callbacks cb 0x7f5c7800bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c821bf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3570 0x565270bd1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c821bf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3570 0x565270bd1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38994/0 (socket says 172.21.15.46:38994) 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c821bf700 1 -- 172.21.15.46:0/1591498660 learned_addr learned my addr 172.21.15.46:0/1591498660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c829c0700 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bcedb0 0x565270bd5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c819be700 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3e90 0x565270bd21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c821bf700 1 -- 172.21.15.46:0/1591498660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3e90 msgr2=0x565270bd21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c821bf700 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3e90 0x565270bd21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c821bf700 1 -- 172.21.15.46:0/1591498660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bcedb0 msgr2=0x565270bd5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c821bf700 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bcedb0 0x565270bd5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c821bf700 1 -- 172.21.15.46:0/1591498660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c10d330 con 0x565270bc3570 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c821bf700 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3570 0x565270bd1c60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x565270be13b0 tx=0x7f5c780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a339564385ba2ea1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c65ffb700 1 -- 172.21.15.46:0/1591498660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78024040 con 0x565270bc3570 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c65ffb700 1 -- 172.21.15.46:0/1591498660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f5c78017070 con 0x565270bc3570 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c905b8b80 1 -- 172.21.15.46:0/1591498660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c740a1920 con 0x565270bc3570 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c65ffb700 1 -- 172.21.15.46:0/1591498660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c780295c0 con 0x565270bc3570 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c905b8b80 10 client.19446 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.985+0000 7f5c905b8b80 1 -- 172.21.15.46:0/1591498660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x56527099cc70 con 0x565270bc3570 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.986+0000 7f5c65ffb700 1 -- 172.21.15.46:0/1591498660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c740a1920 con 0x565270bc3570 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.986+0000 7f5c65ffb700 1 -- 172.21.15.46:0/1591498660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5c7801fb60 con 0x565270bc3570 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.986+0000 7f5c905b8b80 10 client.19446 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.986+0000 7f5c905b8b80 10 client.19446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.986+0000 7f5c905b8b80 20 client.19446 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.986+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x565270c33880 0x565270c35ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.986+0000 7f5c905b8b80 4 client.19446 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:47:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.986+0000 7f5c905b8b80 1 -- 172.21.15.46:0/1591498660 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565270965c90 con 0x565270c33880 2024-03-18T20:47:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.986+0000 7f5c819be700 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x565270c33880 0x565270c35ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.987+0000 7f5c65ffb700 10 client.19446 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:09.987+0000 7f5c819be700 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x565270c33880 0x565270c35ca0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=222765c808796a25 server_cookie=963a026d1c04e025 in_seq=0 out_seq=0 2024-03-18T20:47:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.027+0000 7f5c65ffb700 1 -- 172.21.15.46:0/1591498660 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565270965c90 con 0x565270c33880 2024-03-18T20:47:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.027+0000 7f5c65ffb700 10 client.19446 handle_command_reply: tid=0 2024-03-18T20:47:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c905b8b80 1 client.19446 shutdown 2024-03-18T20:47:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c905b8b80 1 -- 172.21.15.46:0/1591498660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3570 msgr2=0x565270bd1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3570 0x565270bd1c60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x565270be13b0 tx=0x7f5c780047c0 comp rx=0 tx=0).stop 2024-03-18T20:47:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c905b8b80 1 -- 172.21.15.46:0/1591498660 shutdown_connections 2024-03-18T20:47:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x565270c33880 0x565270c35ca0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x565270bc3570 0x565270bd1c60 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x565270bcedb0 0x565270bd5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c905b8b80 1 --2- 172.21.15.46:0/1591498660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565270bc3e90 0x565270bd21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c905b8b80 1 -- 172.21.15.46:0/1591498660 >> 172.21.15.46:0/1591498660 conn(0x565270b3a1c0 msgr2=0x565270bda190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c65ffb700 0 client.19446 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c905b8b80 1 -- 172.21.15.46:0/1591498660 shutdown_connections 2024-03-18T20:47:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c905b8b80 1 -- 172.21.15.46:0/1591498660 wait complete. 2024-03-18T20:47:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.028+0000 7f5c905b8b80 20 client.19446 trim_cache size 0 max 16384 2024-03-18T20:47:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:47:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:47:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:47:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:47:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.029+0000 7f5c84423700 1 -- 172.21.15.46:0/1837770124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5c680a1290 msgr2=0x7f5c680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.029+0000 7f5c84423700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5c680a1290 0x7f5c680a3750 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5c78005950 tx=0x7f5c7800b410 comp rx=0 tx=0).stop 2024-03-18T20:47:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.029+0000 7f5c84423700 1 -- 172.21.15.46:0/1837770124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c7c19ba50 msgr2=0x7f5c7c19bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.029+0000 7f5c84423700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c7c19ba50 0x7f5c7c19bed0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5c74007ee0 tx=0x7f5c7400d3b0 comp rx=0 tx=0).stop 2024-03-18T20:47:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.029+0000 7f5c84423700 1 -- 172.21.15.46:0/1837770124 shutdown_connections 2024-03-18T20:47:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.029+0000 7f5c84423700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5c680a1290 0x7f5c680a3750 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.029+0000 7f5c84423700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c7c19ba50 0x7f5c7c19bed0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.029+0000 7f5c84423700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c7c0ffe20 0x7f5c7c197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.029+0000 7f5c84423700 1 --2- 172.21.15.46:0/1837770124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c7c104f40 0x7f5c7c1976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.029+0000 7f5c84423700 1 -- 172.21.15.46:0/1837770124 >> 172.21.15.46:0/1837770124 conn(0x7f5c7c0fb680 msgr2=0x7f5c7c069370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.029+0000 7f5c84423700 1 -- 172.21.15.46:0/1837770124 shutdown_connections 2024-03-18T20:47:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:10.029+0000 7f5c84423700 1 -- 172.21.15.46:0/1837770124 wait complete. 2024-03-18T20:47:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:10 smithi155 ceph-mon[48344]: pgmap v344: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-03-18T20:47:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:10 smithi155 ceph-mon[48344]: osdmap e92: 12 total, 12 up, 12 in 2024-03-18T20:47:10.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:10 smithi046 ceph-mon[43495]: pgmap v344: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-03-18T20:47:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:10 smithi046 ceph-mon[43495]: osdmap e92: 12 total, 12 up, 12 in 2024-03-18T20:47:10.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 414069 3.68 MB/sec execute 485 sec latency 2658.236 ms 2024-03-18T20:47:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:10 smithi073 ceph-mon[47384]: pgmap v344: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-03-18T20:47:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:10 smithi073 ceph-mon[47384]: osdmap e92: 12 total, 12 up, 12 in 2024-03-18T20:47:11.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 415495 3.70 MB/sec execute 486 sec latency 16.083 ms 2024-03-18T20:47:12.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:12 smithi155 ceph-mon[48344]: pgmap v346: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-03-18T20:47:12.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:12 smithi046 ceph-mon[43495]: pgmap v346: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-03-18T20:47:12.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 416732 3.70 MB/sec execute 487 sec latency 40.497 ms 2024-03-18T20:47:12.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:12 smithi073 ceph-mon[47384]: pgmap v346: 97 pgs: 97 active+clean; 161 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-03-18T20:47:13.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 416732 3.69 MB/sec execute 488 sec latency 1040.557 ms 2024-03-18T20:47:14.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:14 smithi155 ceph-mon[48344]: pgmap v347: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-18T20:47:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:14 smithi046 ceph-mon[43495]: pgmap v347: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-18T20:47:14.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 417636 3.69 MB/sec execute 489 sec latency 1507.039 ms 2024-03-18T20:47:14.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:14 smithi073 ceph-mon[47384]: pgmap v347: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-03-18T20:47:15.717 INFO:tasks.workunit.client.0.smithi046.stdout: 1 419061 3.71 MB/sec execute 490 sec latency 16.922 ms 2024-03-18T20:47:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:15 smithi155 ceph-mon[48344]: pgmap v348: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-03-18T20:47:16.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:15 smithi046 ceph-mon[43495]: pgmap v348: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-03-18T20:47:16.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:15 smithi073 ceph-mon[47384]: pgmap v348: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-03-18T20:47:16.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 420275 3.71 MB/sec execute 491 sec latency 258.107 ms 2024-03-18T20:47:17.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 420275 3.70 MB/sec execute 492 sec latency 1258.198 ms 2024-03-18T20:47:18.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:18 smithi155 ceph-mon[48344]: pgmap v349: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-03-18T20:47:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:18 smithi046 ceph-mon[43495]: pgmap v349: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-03-18T20:47:18.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 420275 3.70 MB/sec execute 493 sec latency 2258.260 ms 2024-03-18T20:47:18.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:18 smithi073 ceph-mon[47384]: pgmap v349: 97 pgs: 97 active+clean; 160 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-03-18T20:47:19.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 420275 3.69 MB/sec execute 494 sec latency 3258.312 ms 2024-03-18T20:47:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:20 smithi155 ceph-mon[48344]: pgmap v350: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-03-18T20:47:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:20 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:47:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:20 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:47:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:20 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:47:20.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:20 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:47:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:20 smithi046 ceph-mon[43495]: pgmap v350: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-03-18T20:47:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:20 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:47:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:20 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:47:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:20 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:47:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:20 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:47:20.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 422200 3.71 MB/sec execute 495 sec latency 3401.162 ms 2024-03-18T20:47:20.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:20 smithi073 ceph-mon[47384]: pgmap v350: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-03-18T20:47:20.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:20 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:47:20.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:20 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:47:20.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:20 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:47:20.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:20 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:47:21.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 423268 3.70 MB/sec execute 496 sec latency 15.393 ms 2024-03-18T20:47:22.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:22 smithi155 ceph-mon[48344]: pgmap v351: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-03-18T20:47:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:22 smithi046 ceph-mon[43495]: pgmap v351: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-03-18T20:47:22.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 423824 3.70 MB/sec execute 497 sec latency 682.208 ms 2024-03-18T20:47:22.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:22 smithi073 ceph-mon[47384]: pgmap v351: 97 pgs: 97 active+clean; 163 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-03-18T20:47:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.157+0000 7ff610d00700 1 -- 172.21.15.46:0/785743961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60c101e80 msgr2=0x7ff60c102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.157+0000 7ff610d00700 1 --2- 172.21.15.46:0/785743961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60c101e80 0x7ff60c102260 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009990 tx=0x7ff5fc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.157+0000 7ff610d00700 1 -- 172.21.15.46:0/785743961 shutdown_connections 2024-03-18T20:47:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.157+0000 7ff610d00700 1 --2- 172.21.15.46:0/785743961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60c101e80 0x7ff60c102260 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.157+0000 7ff610d00700 1 --2- 172.21.15.46:0/785743961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff60c10b2a0 0x7ff60c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.157+0000 7ff610d00700 1 --2- 172.21.15.46:0/785743961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff60c1027a0 0x7ff60c10ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.157+0000 7ff610d00700 1 -- 172.21.15.46:0/785743961 >> 172.21.15.46:0/785743961 conn(0x7ff60c0fb680 msgr2=0x7ff60c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.157+0000 7ff610d00700 1 -- 172.21.15.46:0/785743961 shutdown_connections 2024-03-18T20:47:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.157+0000 7ff610d00700 1 -- 172.21.15.46:0/785743961 wait complete. 2024-03-18T20:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff610d00700 1 Processor -- start 2024-03-18T20:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff610d00700 1 -- start start 2024-03-18T20:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff610d00700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff60c101e80 0x7ff60c0ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff610d00700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff60c1027a0 0x7ff60c100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff610d00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60c10b2a0 0x7ff60c100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff610d00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c1a5500 con 0x7ff60c10b2a0 2024-03-18T20:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff610d00700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c1a5670 con 0x7ff60c101e80 2024-03-18T20:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff610d00700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c1a5830 con 0x7ff60c1027a0 2024-03-18T20:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff60ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60c10b2a0 0x7ff60c100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff60ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60c10b2a0 0x7ff60c100550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52912/0 (socket says 172.21.15.46:52912) 2024-03-18T20:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff60ad9d700 1 -- 172.21.15.46:0/2128880942 learned_addr learned my addr 172.21.15.46:0/2128880942 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff60a59c700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff60c101e80 0x7ff60c0ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff609d9b700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff60c1027a0 0x7ff60c100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff60ad9d700 1 -- 172.21.15.46:0/2128880942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff60c1027a0 msgr2=0x7ff60c100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff60ad9d700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff60c1027a0 0x7ff60c100010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff60ad9d700 1 -- 172.21.15.46:0/2128880942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff60c101e80 msgr2=0x7ff60c0ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff60ad9d700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff60c101e80 0x7ff60c0ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff60ad9d700 1 -- 172.21.15.46:0/2128880942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc0095f0 con 0x7ff60c10b2a0 2024-03-18T20:47:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.158+0000 7ff60a59c700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff60c101e80 0x7ff60c0ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:47:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.159+0000 7ff60ad9d700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60c10b2a0 0x7ff60c100550 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ff604007ee0 tx=0x7ff60400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.159+0000 7ff5fb7fe700 1 -- 172.21.15.46:0/2128880942 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604017070 con 0x7ff60c10b2a0 2024-03-18T20:47:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.159+0000 7ff5fb7fe700 1 -- 172.21.15.46:0/2128880942 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff60400f040 con 0x7ff60c10b2a0 2024-03-18T20:47:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.159+0000 7ff610d00700 1 -- 172.21.15.46:0/2128880942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60c1a5b10 con 0x7ff60c10b2a0 2024-03-18T20:47:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.159+0000 7ff5fb7fe700 1 -- 172.21.15.46:0/2128880942 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604005740 con 0x7ff60c10b2a0 2024-03-18T20:47:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.159+0000 7ff610d00700 1 -- 172.21.15.46:0/2128880942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c1a5fd0 con 0x7ff60c10b2a0 2024-03-18T20:47:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.161+0000 7ff5fb7fe700 1 -- 172.21.15.46:0/2128880942 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff6040058a0 con 0x7ff60c10b2a0 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.162+0000 7ff5fb7fe700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff5f40a1290 0x7ff5f40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.162+0000 7ff60a59c700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff5f40a1290 0x7ff5f40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.162+0000 7ff5fb7fe700 1 -- 172.21.15.46:0/2128880942 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff6040cc580 con 0x7ff60c10b2a0 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.162+0000 7ff60a59c700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff5f40a1290 0x7ff5f40a3750 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc006010 tx=0x7ff5fc01a040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.163+0000 7ff61ce95b80 1 Processor -- start 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.163+0000 7ff61ce95b80 1 -- start start 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.163+0000 7ff61ce95b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bb570 0x55f79c5bb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.163+0000 7ff61ce95b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bbe90 0x55f79c5c6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.163+0000 7ff61ce95b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 0x55f79c5c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.163+0000 7ff61ce95b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79c515930 con 0x55f79c5bbe90 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.163+0000 7ff61ce95b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79c5132f0 con 0x55f79c5bb570 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.163+0000 7ff61ce95b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79c50ebe0 con 0x55f79c5c6db0 2024-03-18T20:47:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.163+0000 7ff609d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bbe90 0x55f79c5c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.163+0000 7ff609d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bbe90 0x55f79c5c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52918/0 (socket says 172.21.15.46:52918) 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.163+0000 7ff609d9b700 1 -- 172.21.15.46:0/4102997080 learned_addr learned my addr 172.21.15.46:0/4102997080 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.163+0000 7ff60a59c700 1 --2- 172.21.15.46:0/4102997080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bb570 0x55f79c5bb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.164+0000 7ff60ad9d700 1 --2- 172.21.15.46:0/4102997080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 0x55f79c5c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.164+0000 7ff609d9b700 1 -- 172.21.15.46:0/4102997080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 msgr2=0x55f79c5c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.164+0000 7ff609d9b700 1 --2- 172.21.15.46:0/4102997080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 0x55f79c5c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.164+0000 7ff609d9b700 1 -- 172.21.15.46:0/4102997080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bb570 msgr2=0x55f79c5bb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.164+0000 7ff609d9b700 1 --2- 172.21.15.46:0/4102997080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bb570 0x55f79c5bb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.164+0000 7ff609d9b700 1 -- 172.21.15.46:0/4102997080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f79c390350 con 0x55f79c5bbe90 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.164+0000 7ff609d9b700 1 --2- 172.21.15.46:0/4102997080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bbe90 0x55f79c5c6870 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7ff60000bc20 tx=0x7ff60000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcba9e08ccd3bc81 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.164+0000 7ff5f8ff9700 1 -- 172.21.15.46:0/4102997080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000fb10 con 0x55f79c5bbe90 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.164+0000 7ff5f8ff9700 1 -- 172.21.15.46:0/4102997080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff60000fc70 con 0x55f79c5bbe90 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.164+0000 7ff5f8ff9700 1 -- 172.21.15.46:0/4102997080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600011680 con 0x55f79c5bbe90 2024-03-18T20:47:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.165+0000 7ff61ce95b80 1 -- 172.21.15.46:0/4102997080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bbe90 msgr2=0x55f79c5c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.165+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/4102997080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bbe90 0x55f79c5c6870 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7ff60000bc20 tx=0x7ff60000bf30 comp rx=0 tx=0).stop 2024-03-18T20:47:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.165+0000 7ff61ce95b80 1 -- 172.21.15.46:0/4102997080 shutdown_connections 2024-03-18T20:47:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.165+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/4102997080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 0x55f79c5c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.165+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/4102997080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bb570 0x55f79c5bb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.165+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/4102997080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bbe90 0x55f79c5c6870 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.165+0000 7ff61ce95b80 1 -- 172.21.15.46:0/4102997080 >> 172.21.15.46:0/4102997080 conn(0x55f79c5321c0 msgr2=0x55f79c5b7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.165+0000 7ff61ce95b80 1 -- 172.21.15.46:0/4102997080 shutdown_connections 2024-03-18T20:47:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.165+0000 7ff61ce95b80 1 -- 172.21.15.46:0/4102997080 wait complete. 2024-03-18T20:47:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.165+0000 7ff61ce95b80 1 -- start start 2024-03-18T20:47:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff61ce95b80 1 Processor -- start 2024-03-18T20:47:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff61ce95b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5bb570 0x55f79c5c6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff61ce95b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 0x55f79c5c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff61ce95b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5c6db0 0x55f79c5c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff61ce95b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79c4aad20 con 0x55f79c5c6db0 2024-03-18T20:47:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff61ce95b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79c4aa9c0 con 0x55f79c5bbe90 2024-03-18T20:47:23.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff61ce95b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79c392640 con 0x55f79c5bb570 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff61ce95b80 10 client.0 _ll_register_callbacks cb 0x55f79c316340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff60ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5c6db0 0x55f79c5c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff60ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5c6db0 0x55f79c5c40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52922/0 (socket says 172.21.15.46:52922) 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff60ad9d700 1 -- 172.21.15.46:0/2860157355 learned_addr learned my addr 172.21.15.46:0/2860157355 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff60a59c700 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5bb570 0x55f79c5c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff609d9b700 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 0x55f79c5c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff60ad9d700 1 -- 172.21.15.46:0/2860157355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5bb570 msgr2=0x55f79c5c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff60ad9d700 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5bb570 0x55f79c5c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff60ad9d700 1 -- 172.21.15.46:0/2860157355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 msgr2=0x55f79c5c3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff60ad9d700 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 0x55f79c5c3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff60ad9d700 1 -- 172.21.15.46:0/2860157355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60000b850 con 0x55f79c5c6db0 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.166+0000 7ff60ad9d700 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5c6db0 0x55f79c5c40f0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ff604099310 tx=0x7ff604098ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42881eede32dc3fe server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff5f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff5f1ffb700 1 -- 172.21.15.46:0/2860157355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60400dde0 con 0x55f79c5c6db0 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff5f1ffb700 1 -- 172.21.15.46:0/2860157355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff604099610 con 0x55f79c5c6db0 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff61ce95b80 1 -- 172.21.15.46:0/2860157355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f79c5af6c0 con 0x55f79c5c6db0 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff5f1ffb700 1 -- 172.21.15.46:0/2860157355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff60409d440 con 0x55f79c5c6db0 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff61ce95b80 10 client.19458 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff61ce95b80 1 -- 172.21.15.46:0/2860157355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55f79c33cb00 con 0x55f79c5c6db0 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff5f1ffb700 1 -- 172.21.15.46:0/2860157355 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff604063080 con 0x55f79c5c6db0 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff5f1ffb700 1 -- 172.21.15.46:0/2860157355 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff604099870 con 0x55f79c5c6db0 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff61ce95b80 10 client.19458 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff61ce95b80 10 client.19458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff61ce95b80 20 client.19458 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f79c5d6b20 0x55f79c5d8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff61ce95b80 4 client.19458 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.167+0000 7ff61ce95b80 1 -- 172.21.15.46:0/2860157355 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f79c5076a0 con 0x55f79c5d6b20 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.168+0000 7ff60a59c700 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f79c5d6b20 0x55f79c5d8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.168+0000 7ff60a59c700 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f79c5d6b20 0x55f79c5d8f40 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e44ffd40ee8e41 server_cookie=6ac5537e56c593d5 in_seq=0 out_seq=0 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.168+0000 7ff5f1ffb700 10 client.19458 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff5f1ffb700 1 -- 172.21.15.46:0/2860157355 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f79c5076a0 con 0x55f79c5d6b20 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff5f1ffb700 10 client.19458 handle_command_reply: tid=0 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff61ce95b80 1 client.19458 shutdown 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff61ce95b80 1 -- 172.21.15.46:0/2860157355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5c6db0 msgr2=0x55f79c5c40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5c6db0 0x55f79c5c40f0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ff604099310 tx=0x7ff604098ff0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff61ce95b80 1 -- 172.21.15.46:0/2860157355 shutdown_connections 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f79c5d6b20 0x55f79c5d8f40 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5c6db0 0x55f79c5c40f0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5bb570 0x55f79c5c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff5f1ffb700 0 client.19458 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/2860157355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 0x55f79c5c3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff61ce95b80 1 -- 172.21.15.46:0/2860157355 >> 172.21.15.46:0/2860157355 conn(0x55f79c5321c0 msgr2=0x55f79c5b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff61ce95b80 1 -- 172.21.15.46:0/2860157355 shutdown_connections 2024-03-18T20:47:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff61ce95b80 1 -- 172.21.15.46:0/2860157355 wait complete. 2024-03-18T20:47:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.169+0000 7ff61ce95b80 20 client.19458 trim_cache size 0 max 16384 2024-03-18T20:47:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff61ce95b80 1 Processor -- start 2024-03-18T20:47:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff61ce95b80 1 -- start start 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff61ce95b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 0x55f79c5b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff61ce95b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 0x55f79c5c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff61ce95b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 0x55f79c5cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff61ce95b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79c5bc850 con 0x55f79c5bb570 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff61ce95b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79c5b41d0 con 0x55f79c5bbe90 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff61ce95b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79c5b8f90 con 0x55f79c5c6db0 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff60a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 0x55f79c5b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff60a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 0x55f79c5b7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52936/0 (socket says 172.21.15.46:52936) 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff60a59c700 1 -- 172.21.15.46:0/4167206816 learned_addr learned my addr 172.21.15.46:0/4167206816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff60a59c700 1 -- 172.21.15.46:0/4167206816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 msgr2=0x55f79c5cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff60a59c700 1 --2- 172.21.15.46:0/4167206816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 0x55f79c5cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff609d9b700 1 --2- 172.21.15.46:0/4167206816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 0x55f79c5c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff60a59c700 1 -- 172.21.15.46:0/4167206816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 msgr2=0x55f79c5c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff60a59c700 1 --2- 172.21.15.46:0/4167206816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 0x55f79c5c3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff60a59c700 1 -- 172.21.15.46:0/4167206816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f79c4d4e40 con 0x55f79c5bb570 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.185+0000 7ff60a59c700 1 --2- 172.21.15.46:0/4167206816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 0x55f79c5b7bd0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009990 tx=0x7ff5fc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36de7d1cb656bcf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff5f97fa700 1 -- 172.21.15.46:0/4167206816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc02a040 con 0x55f79c5bb570 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff5f97fa700 1 -- 172.21.15.46:0/4167206816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5fc02b040 con 0x55f79c5bb570 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff5f97fa700 1 -- 172.21.15.46:0/4167206816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc01d040 con 0x55f79c5bb570 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff61ce95b80 1 -- 172.21.15.46:0/4167206816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 msgr2=0x55f79c5b7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/4167206816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 0x55f79c5b7bd0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009990 tx=0x7ff5fc01e000 comp rx=0 tx=0).stop 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff61ce95b80 1 -- 172.21.15.46:0/4167206816 shutdown_connections 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/4167206816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 0x55f79c5b7bd0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/4167206816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 0x55f79c5cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/4167206816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 0x55f79c5c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff61ce95b80 1 -- 172.21.15.46:0/4167206816 >> 172.21.15.46:0/4167206816 conn(0x55f79c5321c0 msgr2=0x55f79c5d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff61ce95b80 1 -- 172.21.15.46:0/4167206816 shutdown_connections 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff61ce95b80 1 -- 172.21.15.46:0/4167206816 wait complete. 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.186+0000 7ff61ce95b80 1 -- start start 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff61ce95b80 1 Processor -- start 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff61ce95b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 0x55f79c5cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff61ce95b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 0x55f79c5c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff61ce95b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 0x55f79c5ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff61ce95b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79c5ca620 con 0x55f79c5bb570 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff61ce95b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79c5ca790 con 0x55f79c5bbe90 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff61ce95b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79c5ca950 con 0x55f79c5c6db0 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff61ce95b80 10 client.0 _ll_register_callbacks cb 0x7ff604099150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff60a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 0x55f79c5cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff60a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 0x55f79c5cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52946/0 (socket says 172.21.15.46:52946) 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff60a59c700 1 -- 172.21.15.46:0/694467464 learned_addr learned my addr 172.21.15.46:0/694467464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff60ad9d700 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 0x55f79c5ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff609d9b700 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 0x55f79c5c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff60a59c700 1 -- 172.21.15.46:0/694467464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 msgr2=0x55f79c5ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff60a59c700 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 0x55f79c5ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff60a59c700 1 -- 172.21.15.46:0/694467464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 msgr2=0x55f79c5c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff60a59c700 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 0x55f79c5c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff60a59c700 1 -- 172.21.15.46:0/694467464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c10e420 con 0x55f79c5bb570 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff60ad9d700 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 0x55f79c5ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff60a59c700 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 0x55f79c5cd740 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00b580 tx=0x7ff5fc0039a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d714e6fe5e426e2d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.187+0000 7ff5f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff5f1ffb700 1 -- 172.21.15.46:0/694467464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc01d050 con 0x55f79c5bb570 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff5f1ffb700 1 -- 172.21.15.46:0/694467464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff5fc003ce0 con 0x55f79c5bb570 2024-03-18T20:47:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff61ce95b80 1 -- 172.21.15.46:0/694467464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff60000fc70 con 0x55f79c5bb570 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff5f1ffb700 1 -- 172.21.15.46:0/694467464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5fc02a040 con 0x55f79c5bb570 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff61ce95b80 10 client.19464 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff61ce95b80 1 -- 172.21.15.46:0/694467464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55f79c394c70 con 0x55f79c5bb570 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff5f1ffb700 1 -- 172.21.15.46:0/694467464 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5fc04a020 con 0x55f79c5bb570 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff5f1ffb700 1 -- 172.21.15.46:0/694467464 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff5fc03c030 con 0x55f79c5bb570 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff61ce95b80 10 client.19464 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff61ce95b80 10 client.19464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff61ce95b80 20 client.19464 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f79c62bc60 0x55f79c62e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff61ce95b80 4 client.19464 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff61ce95b80 1 -- 172.21.15.46:0/694467464 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f79c35dc90 con 0x55f79c62bc60 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.188+0000 7ff609d9b700 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f79c62bc60 0x55f79c62e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.189+0000 7ff5f1ffb700 10 client.19464 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.189+0000 7ff609d9b700 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f79c62bc60 0x55f79c62e080 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df2c154191a2607d server_cookie=b742a94095e3b750 in_seq=0 out_seq=0 2024-03-18T20:47:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff5f1ffb700 1 -- 172.21.15.46:0/694467464 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f79c35dc90 con 0x55f79c62bc60 2024-03-18T20:47:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff5f1ffb700 10 client.19464 handle_command_reply: tid=0 2024-03-18T20:47:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff61ce95b80 1 client.19464 shutdown 2024-03-18T20:47:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff61ce95b80 1 -- 172.21.15.46:0/694467464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 msgr2=0x55f79c5cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 0x55f79c5cd740 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00b580 tx=0x7ff5fc0039a0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff61ce95b80 1 -- 172.21.15.46:0/694467464 shutdown_connections 2024-03-18T20:47:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f79c62bc60 0x55f79c62e080 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f79c5bb570 0x55f79c5cd740 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff5f1ffb700 0 client.19464 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f79c5c6db0 0x55f79c5ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff61ce95b80 1 --2- 172.21.15.46:0/694467464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f79c5bbe90 0x55f79c5c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff61ce95b80 1 -- 172.21.15.46:0/694467464 >> 172.21.15.46:0/694467464 conn(0x55f79c5321c0 msgr2=0x55f79c5d2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff61ce95b80 1 -- 172.21.15.46:0/694467464 shutdown_connections 2024-03-18T20:47:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff61ce95b80 1 -- 172.21.15.46:0/694467464 wait complete. 2024-03-18T20:47:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.221+0000 7ff61ce95b80 20 client.19464 trim_cache size 0 max 16384 2024-03-18T20:47:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:47:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:47:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:47:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.222+0000 7ff610d00700 1 -- 172.21.15.46:0/2128880942 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff5f40a1290 msgr2=0x7ff5f40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.222+0000 7ff610d00700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff5f40a1290 0x7ff5f40a3750 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc006010 tx=0x7ff5fc01a040 comp rx=0 tx=0).stop 2024-03-18T20:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.222+0000 7ff610d00700 1 -- 172.21.15.46:0/2128880942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60c10b2a0 msgr2=0x7ff60c100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.222+0000 7ff610d00700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60c10b2a0 0x7ff60c100550 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ff604007ee0 tx=0x7ff60400d3b0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.222+0000 7ff610d00700 1 -- 172.21.15.46:0/2128880942 shutdown_connections 2024-03-18T20:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.222+0000 7ff610d00700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff5f40a1290 0x7ff5f40a3750 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.222+0000 7ff610d00700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60c10b2a0 0x7ff60c100550 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.222+0000 7ff610d00700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff60c101e80 0x7ff60c0ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.222+0000 7ff610d00700 1 --2- 172.21.15.46:0/2128880942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff60c1027a0 0x7ff60c100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.222+0000 7ff610d00700 1 -- 172.21.15.46:0/2128880942 >> 172.21.15.46:0/2128880942 conn(0x7ff60c0fb680 msgr2=0x7ff60c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.222+0000 7ff610d00700 1 -- 172.21.15.46:0/2128880942 shutdown_connections 2024-03-18T20:47:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:23.222+0000 7ff610d00700 1 -- 172.21.15.46:0/2128880942 wait complete. 2024-03-18T20:47:23.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 423824 3.70 MB/sec execute 498 sec latency 1682.325 ms 2024-03-18T20:47:24.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:24 smithi155 ceph-mon[48344]: pgmap v352: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-03-18T20:47:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:24 smithi046 ceph-mon[43495]: pgmap v352: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-03-18T20:47:24.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 423824 3.69 MB/sec execute 499 sec latency 2682.443 ms 2024-03-18T20:47:24.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:24 smithi073 ceph-mon[47384]: pgmap v352: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-03-18T20:47:25.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 424601 3.69 MB/sec execute 500 sec latency 3294.627 ms 2024-03-18T20:47:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:26 smithi155 ceph-mon[48344]: pgmap v353: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-03-18T20:47:26.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:26 smithi046 ceph-mon[43495]: pgmap v353: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-03-18T20:47:26.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:26 smithi073 ceph-mon[47384]: pgmap v353: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-03-18T20:47:26.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 426131 3.70 MB/sec execute 501 sec latency 17.000 ms 2024-03-18T20:47:27.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 427365 3.70 MB/sec execute 502 sec latency 203.939 ms 2024-03-18T20:47:28.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:28 smithi155 ceph-mon[48344]: pgmap v354: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-03-18T20:47:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:28 smithi046 ceph-mon[43495]: pgmap v354: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-03-18T20:47:28.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 427365 3.70 MB/sec execute 503 sec latency 1204.048 ms 2024-03-18T20:47:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:28 smithi073 ceph-mon[47384]: pgmap v354: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-03-18T20:47:29.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 427365 3.69 MB/sec execute 504 sec latency 2204.141 ms 2024-03-18T20:47:30.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:47:30.173 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:47:30.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.323+0000 7f65703fa700 1 -- 172.21.15.46:0/3784400485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65680ff490 msgr2=0x7f65680ff870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.323+0000 7f65703fa700 1 --2- 172.21.15.46:0/3784400485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65680ff490 0x7f65680ff870 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f6564009990 tx=0x7f6564009ca0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.323+0000 7f65703fa700 1 -- 172.21.15.46:0/3784400485 shutdown_connections 2024-03-18T20:47:30.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.323+0000 7f65703fa700 1 --2- 172.21.15.46:0/3784400485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65680ff490 0x7f65680ff870 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.323+0000 7f65703fa700 1 --2- 172.21.15.46:0/3784400485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6568106050 0x7f656810dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.323+0000 7f65703fa700 1 --2- 172.21.15.46:0/3784400485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65680ffe40 0x7f6568105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.323+0000 7f65703fa700 1 -- 172.21.15.46:0/3784400485 >> 172.21.15.46:0/3784400485 conn(0x7f65680761e0 msgr2=0x7f65680fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.323+0000 7f65703fa700 1 -- 172.21.15.46:0/3784400485 shutdown_connections 2024-03-18T20:47:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.323+0000 7f65703fa700 1 -- 172.21.15.46:0/3784400485 wait complete. 2024-03-18T20:47:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.324+0000 7f65703fa700 1 Processor -- start 2024-03-18T20:47:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.324+0000 7f65703fa700 1 -- start start 2024-03-18T20:47:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.324+0000 7f65703fa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65680ff490 0x7f6568197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.324+0000 7f65703fa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6568106050 0x7f6568197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.324+0000 7f65703fa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f656819bb10 0x7f656819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.324+0000 7f65703fa700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656819c590 con 0x7f656819bb10 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.324+0000 7f65703fa700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656819c700 con 0x7f65680ff490 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.324+0000 7f65703fa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656819c8c0 con 0x7f6568106050 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f656e196700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65680ff490 0x7f6568197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f656e997700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f656819bb10 0x7f656819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f656e196700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65680ff490 0x7f6568197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54622/0 (socket says 172.21.15.46:54622) 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f656e196700 1 -- 172.21.15.46:0/1586179601 learned_addr learned my addr 172.21.15.46:0/1586179601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f656d995700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6568106050 0x7f6568197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f656e196700 1 -- 172.21.15.46:0/1586179601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6568106050 msgr2=0x7f6568197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f656e196700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6568106050 0x7f6568197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f656e196700 1 -- 172.21.15.46:0/1586179601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f656819bb10 msgr2=0x7f656819bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f656e196700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f656819bb10 0x7f656819bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f656e196700 1 -- 172.21.15.46:0/1586179601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65640095f0 con 0x7f65680ff490 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f656e997700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f656819bb10 0x7f656819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f656e196700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65680ff490 0x7f6568197230 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6564004730 tx=0x7f6564004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f655f7fe700 1 -- 172.21.15.46:0/1586179601 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656401d070 con 0x7f65680ff490 2024-03-18T20:47:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f655f7fe700 1 -- 172.21.15.46:0/1586179601 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6564022470 con 0x7f65680ff490 2024-03-18T20:47:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f65703fa700 1 -- 172.21.15.46:0/1586179601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65681a3dc0 con 0x7f65680ff490 2024-03-18T20:47:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.325+0000 7f655f7fe700 1 -- 172.21.15.46:0/1586179601 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564017650 con 0x7f65680ff490 2024-03-18T20:47:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.326+0000 7f65703fa700 1 -- 172.21.15.46:0/1586179601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65681a4190 con 0x7f65680ff490 2024-03-18T20:47:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.328+0000 7f655f7fe700 1 -- 172.21.15.46:0/1586179601 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f65640225e0 con 0x7f65680ff490 2024-03-18T20:47:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.329+0000 7f655f7fe700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f65540a1290 0x7f65540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.329+0000 7f656d995700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f65540a1290 0x7f65540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.329+0000 7f655f7fe700 1 -- 172.21.15.46:0/1586179601 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f65640cde50 con 0x7f65680ff490 2024-03-18T20:47:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.329+0000 7f656d995700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f65540a1290 0x7f65540a3750 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6568198780 tx=0x7f6558009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f657c58fb80 1 Processor -- start 2024-03-18T20:47:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f657c58fb80 1 -- start start 2024-03-18T20:47:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f657c58fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c5621a60 0x5608c5621e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f657c58fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c562cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f657c58fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c562d220 0x5608c562f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f657c58fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608c557b990 con 0x5608c5622380 2024-03-18T20:47:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f657c58fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608c5579350 con 0x5608c562d220 2024-03-18T20:47:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f657c58fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608c5574c40 con 0x5608c5621a60 2024-03-18T20:47:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f656d995700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c562cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f656d995700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c562cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60548/0 (socket says 172.21.15.46:60548) 2024-03-18T20:47:30.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f656d995700 1 -- 172.21.15.46:0/1106475284 learned_addr learned my addr 172.21.15.46:0/1106475284 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f656e997700 1 --2- 172.21.15.46:0/1106475284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c562d220 0x5608c562f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.330+0000 7f656e196700 1 --2- 172.21.15.46:0/1106475284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c5621a60 0x5608c5621e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.331+0000 7f656d995700 1 -- 172.21.15.46:0/1106475284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c5621a60 msgr2=0x5608c5621e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.331+0000 7f656d995700 1 --2- 172.21.15.46:0/1106475284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c5621a60 0x5608c5621e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.331+0000 7f656d995700 1 -- 172.21.15.46:0/1106475284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c562d220 msgr2=0x5608c562f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.331+0000 7f656d995700 1 --2- 172.21.15.46:0/1106475284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c562d220 0x5608c562f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.331+0000 7f656d995700 1 -- 172.21.15.46:0/1106475284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608c53f63b0 con 0x5608c5622380 2024-03-18T20:47:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.331+0000 7f656d995700 1 --2- 172.21.15.46:0/1106475284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c562cce0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f6558011cf0 tx=0x7f6558014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abe991b0371d25c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.331+0000 7f655cda8700 1 -- 172.21.15.46:0/1106475284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655800ea40 con 0x5608c5622380 2024-03-18T20:47:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.331+0000 7f655cda8700 1 -- 172.21.15.46:0/1106475284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6558021070 con 0x5608c5622380 2024-03-18T20:47:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.331+0000 7f655cda8700 1 -- 172.21.15.46:0/1106475284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655801d410 con 0x5608c5622380 2024-03-18T20:47:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.332+0000 7f657c58fb80 1 -- 172.21.15.46:0/1106475284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 msgr2=0x5608c562cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.332+0000 7f657c58fb80 1 --2- 172.21.15.46:0/1106475284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c562cce0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f6558011cf0 tx=0x7f6558014000 comp rx=0 tx=0).stop 2024-03-18T20:47:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.332+0000 7f657c58fb80 1 -- 172.21.15.46:0/1106475284 shutdown_connections 2024-03-18T20:47:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.332+0000 7f657c58fb80 1 --2- 172.21.15.46:0/1106475284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c562d220 0x5608c562f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.332+0000 7f657c58fb80 1 --2- 172.21.15.46:0/1106475284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c5621a60 0x5608c5621e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.332+0000 7f657c58fb80 1 --2- 172.21.15.46:0/1106475284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c562cce0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.332+0000 7f657c58fb80 1 -- 172.21.15.46:0/1106475284 >> 172.21.15.46:0/1106475284 conn(0x5608c5598220 msgr2=0x5608c561e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.332+0000 7f657c58fb80 1 -- 172.21.15.46:0/1106475284 shutdown_connections 2024-03-18T20:47:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.332+0000 7f657c58fb80 1 -- 172.21.15.46:0/1106475284 wait complete. 2024-03-18T20:47:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.332+0000 7f657c58fb80 1 -- start start 2024-03-18T20:47:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f657c58fb80 1 Processor -- start 2024-03-18T20:47:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f657c58fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5621a60 0x5608c562cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f657c58fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5622380 0x5608c5629f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f657c58fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c562a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f657c58fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608c5510d80 con 0x5608c5621a60 2024-03-18T20:47:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f657c58fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608c5510a20 con 0x5608c5622380 2024-03-18T20:47:30.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f657c58fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608c53f86a0 con 0x5608c562d220 2024-03-18T20:47:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f657c58fb80 10 client.0 _ll_register_callbacks cb 0x5608c537c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f656e196700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5621a60 0x5608c562cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f656e196700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5621a60 0x5608c562cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60554/0 (socket says 172.21.15.46:60554) 2024-03-18T20:47:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f656e196700 1 -- 172.21.15.46:0/2415751483 learned_addr learned my addr 172.21.15.46:0/2415751483 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f656d995700 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5622380 0x5608c5629f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f656e997700 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c562a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f656e196700 1 -- 172.21.15.46:0/2415751483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 msgr2=0x5608c562a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f656e196700 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c562a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f656e196700 1 -- 172.21.15.46:0/2415751483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5622380 msgr2=0x5608c5629f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f656e196700 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5622380 0x5608c5629f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f656e196700 1 -- 172.21.15.46:0/2415751483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6558017070 con 0x5608c5621a60 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.333+0000 7f656e997700 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c562a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.334+0000 7f656e196700 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5621a60 0x5608c562cc10 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f656409acd0 tx=0x7f656409ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3339b6455726f9f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.334+0000 7f6551ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.334+0000 7f6551ffb700 1 -- 172.21.15.46:0/2415751483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6564095de0 con 0x5608c5621a60 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.334+0000 7f657c58fb80 1 -- 172.21.15.46:0/2415751483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608c5615ad0 con 0x5608c5621a60 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.334+0000 7f6551ffb700 1 -- 172.21.15.46:0/2415751483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f656409afd0 con 0x5608c5621a60 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.334+0000 7f6551ffb700 1 -- 172.21.15.46:0/2415751483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f656409ee00 con 0x5608c5621a60 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.334+0000 7f657c58fb80 10 client.19476 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.334+0000 7f657c58fb80 1 -- 172.21.15.46:0/2415751483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5608c53a2b60 con 0x5608c5621a60 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.334+0000 7f6551ffb700 1 -- 172.21.15.46:0/2415751483 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65640b7080 con 0x5608c5621a60 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.334+0000 7f6551ffb700 1 -- 172.21.15.46:0/2415751483 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f656409fd20 con 0x5608c5621a60 2024-03-18T20:47:30.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.335+0000 7f657c58fb80 10 client.19476 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.335+0000 7f657c58fb80 10 client.19476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.335+0000 7f657c58fb80 20 client.19476 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.335+0000 7f657c58fb80 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608c5640f50 0x5608c5643370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.335+0000 7f657c58fb80 4 client.19476 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.335+0000 7f657c58fb80 1 -- 172.21.15.46:0/2415751483 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608c556d700 con 0x5608c5640f50 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.335+0000 7f656d995700 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608c5640f50 0x5608c5643370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.335+0000 7f6551ffb700 10 client.19476 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.335+0000 7f656d995700 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608c5640f50 0x5608c5643370 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ce4caec8e334d3 server_cookie=4c1865d67b2b8828 in_seq=0 out_seq=0 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f6551ffb700 1 -- 172.21.15.46:0/2415751483 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5608c556d700 con 0x5608c5640f50 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f6551ffb700 10 client.19476 handle_command_reply: tid=0 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f657c58fb80 1 client.19476 shutdown 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f657c58fb80 1 -- 172.21.15.46:0/2415751483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5621a60 msgr2=0x5608c562cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f657c58fb80 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5621a60 0x5608c562cc10 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f656409acd0 tx=0x7f656409ab10 comp rx=0 tx=0).stop 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f657c58fb80 1 -- 172.21.15.46:0/2415751483 shutdown_connections 2024-03-18T20:47:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f657c58fb80 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608c5640f50 0x5608c5643370 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f657c58fb80 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5621a60 0x5608c562cc10 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f657c58fb80 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c562a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f6551ffb700 0 client.19476 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f657c58fb80 1 --2- 172.21.15.46:0/2415751483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5622380 0x5608c5629f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f657c58fb80 1 -- 172.21.15.46:0/2415751483 >> 172.21.15.46:0/2415751483 conn(0x5608c5598220 msgr2=0x5608c561dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f657c58fb80 1 -- 172.21.15.46:0/2415751483 shutdown_connections 2024-03-18T20:47:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f657c58fb80 1 -- 172.21.15.46:0/2415751483 wait complete. 2024-03-18T20:47:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.336+0000 7f657c58fb80 20 client.19476 trim_cache size 0 max 16384 2024-03-18T20:47:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.358+0000 7f657c58fb80 1 Processor -- start 2024-03-18T20:47:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.358+0000 7f657c58fb80 1 -- start start 2024-03-18T20:47:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.358+0000 7f657c58fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 0x5608c5638310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.358+0000 7f657c58fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c561db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.358+0000 7f657c58fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c5625220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.358+0000 7f657c58fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608c55e8940 con 0x5608c5622380 2024-03-18T20:47:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.358+0000 7f657c58fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608c561e5a0 con 0x5608c5621a60 2024-03-18T20:47:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.358+0000 7f657c58fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608c561a5e0 con 0x5608c562d220 2024-03-18T20:47:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.358+0000 7f656d995700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c561db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f656d995700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c561db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60566/0 (socket says 172.21.15.46:60566) 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f656e196700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 0x5608c5638310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f656d995700 1 -- 172.21.15.46:0/2837393916 learned_addr learned my addr 172.21.15.46:0/2837393916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f656e997700 1 --2- 172.21.15.46:0/2837393916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c5625220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f656d995700 1 -- 172.21.15.46:0/2837393916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 msgr2=0x5608c5625220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f656d995700 1 --2- 172.21.15.46:0/2837393916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c5625220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f656d995700 1 -- 172.21.15.46:0/2837393916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 msgr2=0x5608c5638310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f656d995700 1 --2- 172.21.15.46:0/2837393916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 0x5608c5638310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f656d995700 1 -- 172.21.15.46:0/2837393916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608c553aea0 con 0x5608c5622380 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f656e196700 1 --2- 172.21.15.46:0/2837393916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 0x5608c5638310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f656d995700 1 --2- 172.21.15.46:0/2837393916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c561db50 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f655801cfd0 tx=0x7f65580058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3eb18c28fdbbf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f655d5a9700 1 -- 172.21.15.46:0/2837393916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655801cb60 con 0x5608c5622380 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f655d5a9700 1 -- 172.21.15.46:0/2837393916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f655801c410 con 0x5608c5622380 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.359+0000 7f655d5a9700 1 -- 172.21.15.46:0/2837393916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558005040 con 0x5608c5622380 2024-03-18T20:47:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.360+0000 7f657c58fb80 1 -- 172.21.15.46:0/2837393916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 msgr2=0x5608c561db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.360+0000 7f657c58fb80 1 --2- 172.21.15.46:0/2837393916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c561db50 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f655801cfd0 tx=0x7f65580058e0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.360+0000 7f657c58fb80 1 -- 172.21.15.46:0/2837393916 shutdown_connections 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.360+0000 7f657c58fb80 1 --2- 172.21.15.46:0/2837393916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c5625220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.360+0000 7f657c58fb80 1 --2- 172.21.15.46:0/2837393916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 0x5608c5638310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.360+0000 7f657c58fb80 1 --2- 172.21.15.46:0/2837393916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c561db50 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.360+0000 7f657c58fb80 1 -- 172.21.15.46:0/2837393916 >> 172.21.15.46:0/2837393916 conn(0x5608c5598220 msgr2=0x5608c561c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.360+0000 7f657c58fb80 1 -- 172.21.15.46:0/2837393916 shutdown_connections 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.360+0000 7f657c58fb80 1 -- 172.21.15.46:0/2837393916 wait complete. 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.361+0000 7f657c58fb80 1 -- start start 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.361+0000 7f657c58fb80 1 Processor -- start 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.361+0000 7f657c58fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 0x5608c56363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.361+0000 7f657c58fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c56368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.361+0000 7f657c58fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c568ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.361+0000 7f657c58fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608c568e300 con 0x5608c5622380 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.361+0000 7f657c58fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608c568e470 con 0x5608c5621a60 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.361+0000 7f657c58fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608c568e5e0 con 0x5608c562d220 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.361+0000 7f657c58fb80 10 client.0 _ll_register_callbacks cb 0x7f656409ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f656e196700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 0x5608c56363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f656d995700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c56368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f656e196700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 0x5608c56363a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54662/0 (socket says 172.21.15.46:54662) 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f656e196700 1 -- 172.21.15.46:0/3874425133 learned_addr learned my addr 172.21.15.46:0/3874425133 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f656e997700 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c568ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f656e196700 1 -- 172.21.15.46:0/3874425133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 msgr2=0x5608c568ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f656e196700 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c568ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f656e196700 1 -- 172.21.15.46:0/3874425133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 msgr2=0x5608c56368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f656e196700 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c56368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f656e196700 1 -- 172.21.15.46:0/3874425133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608c53a2b60 con 0x5608c5621a60 2024-03-18T20:47:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f656d995700 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c56368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f656e196700 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 0x5608c56363a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f65640cb630 tx=0x7f656409b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f4e7befb61d0bbc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f6551ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f6551ffb700 1 -- 172.21.15.46:0/3874425133 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656409a710 con 0x5608c5621a60 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f6551ffb700 1 -- 172.21.15.46:0/3874425133 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f656409afd0 con 0x5608c5621a60 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.362+0000 7f657c58fb80 1 -- 172.21.15.46:0/3874425133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6558021070 con 0x5608c5621a60 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.363+0000 7f6551ffb700 1 -- 172.21.15.46:0/3874425133 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f656409ee00 con 0x5608c5621a60 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.363+0000 7f657c58fb80 10 client.29099 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.363+0000 7f657c58fb80 1 -- 172.21.15.46:0/3874425133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f655802a040 con 0x5608c5621a60 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.363+0000 7f6551ffb700 1 -- 172.21.15.46:0/3874425133 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f656407c030 con 0x5608c5621a60 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.363+0000 7f6551ffb700 1 -- 172.21.15.46:0/3874425133 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f65640976e0 con 0x5608c5621a60 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.363+0000 7f657c58fb80 10 client.29099 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.363+0000 7f657c58fb80 10 client.29099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.363+0000 7f657c58fb80 20 client.29099 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.363+0000 7f657c58fb80 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608c5695ba0 0x5608c5697fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.363+0000 7f657c58fb80 4 client.29099 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:47:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.363+0000 7f657c58fb80 1 -- 172.21.15.46:0/3874425133 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5608c53c3cf0 con 0x5608c5695ba0 2024-03-18T20:47:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.363+0000 7f656d995700 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608c5695ba0 0x5608c5697fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f656d995700 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608c5695ba0 0x5608c5697fc0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c38c0befb09a5acd server_cookie=9c5ec885b404f4ad in_seq=0 out_seq=0 2024-03-18T20:47:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f6551ffb700 10 client.29099 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f6551ffb700 1 -- 172.21.15.46:0/3874425133 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5608c53c3cf0 con 0x5608c5695ba0 2024-03-18T20:47:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f6551ffb700 10 client.29099 handle_command_reply: tid=0 2024-03-18T20:47:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f657c58fb80 1 client.29099 shutdown 2024-03-18T20:47:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f657c58fb80 1 -- 172.21.15.46:0/3874425133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 msgr2=0x5608c56363a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f657c58fb80 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 0x5608c56363a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f65640cb630 tx=0x7f656409b460 comp rx=0 tx=0).stop 2024-03-18T20:47:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f657c58fb80 1 -- 172.21.15.46:0/3874425133 shutdown_connections 2024-03-18T20:47:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f657c58fb80 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5608c5695ba0 0x5608c5697fc0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f657c58fb80 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608c562d220 0x5608c568ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f657c58fb80 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5608c5621a60 0x5608c56363a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f657c58fb80 1 --2- 172.21.15.46:0/3874425133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5608c5622380 0x5608c56368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f6551ffb700 0 client.29099 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f657c58fb80 1 -- 172.21.15.46:0/3874425133 >> 172.21.15.46:0/3874425133 conn(0x5608c5598220 msgr2=0x5608c5625220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f657c58fb80 1 -- 172.21.15.46:0/3874425133 shutdown_connections 2024-03-18T20:47:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f657c58fb80 1 -- 172.21.15.46:0/3874425133 wait complete. 2024-03-18T20:47:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.364+0000 7f657c58fb80 20 client.29099 trim_cache size 0 max 16384 2024-03-18T20:47:30.374 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:47:30.374 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:47:30.375 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:47:30.375 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:47:30.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.365+0000 7f65703fa700 1 -- 172.21.15.46:0/1586179601 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f65540a1290 msgr2=0x7f65540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.365+0000 7f65703fa700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f65540a1290 0x7f65540a3750 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6568198780 tx=0x7f6558009260 comp rx=0 tx=0).stop 2024-03-18T20:47:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.365+0000 7f65703fa700 1 -- 172.21.15.46:0/1586179601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65680ff490 msgr2=0x7f6568197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.365+0000 7f65703fa700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65680ff490 0x7f6568197230 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6564004730 tx=0x7f6564004760 comp rx=0 tx=0).stop 2024-03-18T20:47:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.365+0000 7f65703fa700 1 -- 172.21.15.46:0/1586179601 shutdown_connections 2024-03-18T20:47:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.365+0000 7f65703fa700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f65540a1290 0x7f65540a3750 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.365+0000 7f65703fa700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f656819bb10 0x7f656819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.365+0000 7f65703fa700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65680ff490 0x7f6568197230 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.365+0000 7f65703fa700 1 --2- 172.21.15.46:0/1586179601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6568106050 0x7f6568197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.365+0000 7f65703fa700 1 -- 172.21.15.46:0/1586179601 >> 172.21.15.46:0/1586179601 conn(0x7f65680761e0 msgr2=0x7f6568104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.365+0000 7f65703fa700 1 -- 172.21.15.46:0/1586179601 shutdown_connections 2024-03-18T20:47:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.365+0000 7f65703fa700 1 -- 172.21.15.46:0/1586179601 wait complete. 2024-03-18T20:47:30.381 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:47:30.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:47:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.532+0000 7fd38387f700 1 -- 172.21.15.46:0/1155959675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c101dc0 msgr2=0x7fd37c1021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.532+0000 7fd38387f700 1 --2- 172.21.15.46:0/1155959675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c101dc0 0x7fd37c1021a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd3780098f0 tx=0x7fd378009c00 comp rx=0 tx=0).stop 2024-03-18T20:47:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.532+0000 7fd38387f700 1 -- 172.21.15.46:0/1155959675 shutdown_connections 2024-03-18T20:47:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.532+0000 7fd38387f700 1 --2- 172.21.15.46:0/1155959675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c10b1e0 0x7fd37c10d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.532+0000 7fd38387f700 1 --2- 172.21.15.46:0/1155959675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c101dc0 0x7fd37c1021a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.532+0000 7fd38387f700 1 --2- 172.21.15.46:0/1155959675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c1026e0 0x7fd37c10abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.532+0000 7fd38387f700 1 -- 172.21.15.46:0/1155959675 >> 172.21.15.46:0/1155959675 conn(0x7fd37c0fb620 msgr2=0x7fd37c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.532+0000 7fd38387f700 1 -- 172.21.15.46:0/1155959675 shutdown_connections 2024-03-18T20:47:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.532+0000 7fd38387f700 1 -- 172.21.15.46:0/1155959675 wait complete. 2024-03-18T20:47:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd38387f700 1 Processor -- start 2024-03-18T20:47:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd38387f700 1 -- start start 2024-03-18T20:47:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd38387f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c101dc0 0x7fd37c0ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd38387f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c1026e0 0x7fd37c0fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd38387f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c10b1e0 0x7fd37c100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd38387f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1a5500 con 0x7fd37c10b1e0 2024-03-18T20:47:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd38387f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1a5670 con 0x7fd37c101dc0 2024-03-18T20:47:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd38387f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1a5830 con 0x7fd37c1026e0 2024-03-18T20:47:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd38161b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c101dc0 0x7fd37c0ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd381e1c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c10b1e0 0x7fd37c100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd38161b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c101dc0 0x7fd37c0ffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54678/0 (socket says 172.21.15.46:54678) 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd38161b700 1 -- 172.21.15.46:0/2446432628 learned_addr learned my addr 172.21.15.46:0/2446432628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.533+0000 7fd380e1a700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c1026e0 0x7fd37c0fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.534+0000 7fd38161b700 1 -- 172.21.15.46:0/2446432628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c1026e0 msgr2=0x7fd37c0fff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.534+0000 7fd38161b700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c1026e0 0x7fd37c0fff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.534+0000 7fd38161b700 1 -- 172.21.15.46:0/2446432628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c10b1e0 msgr2=0x7fd37c100490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.534+0000 7fd38161b700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c10b1e0 0x7fd37c100490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.534+0000 7fd38161b700 1 -- 172.21.15.46:0/2446432628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c0095f0 con 0x7fd37c101dc0 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.534+0000 7fd38161b700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c101dc0 0x7fd37c0ffa10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd37800fa00 tx=0x7fd37800fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.534+0000 7fd3727fc700 1 -- 172.21.15.46:0/2446432628 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37801c070 con 0x7fd37c101dc0 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.534+0000 7fd3727fc700 1 -- 172.21.15.46:0/2446432628 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd378005040 con 0x7fd37c101dc0 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.534+0000 7fd38387f700 1 -- 172.21.15.46:0/2446432628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd378009520 con 0x7fd37c101dc0 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.534+0000 7fd3727fc700 1 -- 172.21.15.46:0/2446432628 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378004a90 con 0x7fd37c101dc0 2024-03-18T20:47:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.534+0000 7fd38387f700 1 -- 172.21.15.46:0/2446432628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c1a5dd0 con 0x7fd37c101dc0 2024-03-18T20:47:30.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:30 smithi155 ceph-mon[48344]: pgmap v355: 97 pgs: 97 active+clean; 166 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-03-18T20:47:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.535+0000 7fd38387f700 1 -- 172.21.15.46:0/2446432628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c106310 con 0x7fd37c101dc0 2024-03-18T20:47:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.536+0000 7fd3727fc700 1 -- 172.21.15.46:0/2446432628 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd3780051b0 con 0x7fd37c101dc0 2024-03-18T20:47:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.537+0000 7fd3727fc700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd3680a1290 0x7fd3680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.537+0000 7fd380e1a700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd3680a1290 0x7fd3680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.537+0000 7fd3727fc700 1 -- 172.21.15.46:0/2446432628 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd3780cddf0 con 0x7fd37c101dc0 2024-03-18T20:47:30.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.538+0000 7fd380e1a700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd3680a1290 0x7fd3680a3750 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd37c101140 tx=0x7fd36c009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:30.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.541+0000 7fd3727fc700 1 -- 172.21.15.46:0/2446432628 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd3780972b0 con 0x7fd37c101dc0 2024-03-18T20:47:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:30 smithi046 ceph-mon[43495]: pgmap v355: 97 pgs: 97 active+clean; 166 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-03-18T20:47:30.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 428270 3.69 MB/sec execute 505 sec latency 2809.687 ms 2024-03-18T20:47:30.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:30 smithi073 ceph-mon[47384]: pgmap v355: 97 pgs: 97 active+clean; 166 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 961 KiB/s rd, 2.0 MiB/s wr, 181 op/s 2024-03-18T20:47:30.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.832+0000 7fd38387f700 1 -- 172.21.15.46:0/2446432628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd37c069930 con 0x7fd37c101dc0 2024-03-18T20:47:30.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.833+0000 7fd3727fc700 1 -- 172.21.15.46:0/2446432628 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+10916 (secure 0 0 0) 0x7fd37809a8d0 con 0x7fd37c101dc0 2024-03-18T20:47:30.836 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:47:30.836 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:45:43.243732+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:47:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.835+0000 7fd38387f700 1 -- 172.21.15.46:0/2446432628 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd3680a1290 msgr2=0x7fd3680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.835+0000 7fd38387f700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd3680a1290 0x7fd3680a3750 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd37c101140 tx=0x7fd36c009310 comp rx=0 tx=0).stop 2024-03-18T20:47:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.835+0000 7fd38387f700 1 -- 172.21.15.46:0/2446432628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c101dc0 msgr2=0x7fd37c0ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.835+0000 7fd38387f700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c101dc0 0x7fd37c0ffa10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd37800fa00 tx=0x7fd37800fa30 comp rx=0 tx=0).stop 2024-03-18T20:47:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.835+0000 7fd38387f700 1 -- 172.21.15.46:0/2446432628 shutdown_connections 2024-03-18T20:47:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.835+0000 7fd38387f700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd3680a1290 0x7fd3680a3750 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.835+0000 7fd38387f700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c10b1e0 0x7fd37c100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.836+0000 7fd38387f700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c101dc0 0x7fd37c0ffa10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.836+0000 7fd38387f700 1 --2- 172.21.15.46:0/2446432628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd37c1026e0 0x7fd37c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.836+0000 7fd38387f700 1 -- 172.21.15.46:0/2446432628 >> 172.21.15.46:0/2446432628 conn(0x7fd37c0fb620 msgr2=0x7fd37c0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.836+0000 7fd38387f700 1 -- 172.21.15.46:0/2446432628 shutdown_connections 2024-03-18T20:47:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:30.836+0000 7fd38387f700 1 -- 172.21.15.46:0/2446432628 wait complete. 2024-03-18T20:47:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 49 2024-03-18T20:47:30.853 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:47:30.853 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:47:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.005+0000 7f36f8e47700 1 -- 172.21.15.46:0/765874977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36f4069f80 msgr2=0x7f36f4105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.005+0000 7f36f8e47700 1 --2- 172.21.15.46:0/765874977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36f4069f80 0x7f36f4105960 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009990 tx=0x7f36dc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.005+0000 7f36f8e47700 1 -- 172.21.15.46:0/765874977 shutdown_connections 2024-03-18T20:47:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.005+0000 7f36f8e47700 1 --2- 172.21.15.46:0/765874977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f4106090 0x7f36f410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.005+0000 7f36f8e47700 1 --2- 172.21.15.46:0/765874977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36f40695d0 0x7f36f40699b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.005+0000 7f36f8e47700 1 --2- 172.21.15.46:0/765874977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36f4069f80 0x7f36f4105960 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.005+0000 7f36f8e47700 1 -- 172.21.15.46:0/765874977 >> 172.21.15.46:0/765874977 conn(0x7f36f4076db0 msgr2=0x7f36f40771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.005+0000 7f36f8e47700 1 -- 172.21.15.46:0/765874977 shutdown_connections 2024-03-18T20:47:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.005+0000 7f36f8e47700 1 -- 172.21.15.46:0/765874977 wait complete. 2024-03-18T20:47:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.006+0000 7f36f8e47700 1 Processor -- start 2024-03-18T20:47:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.006+0000 7f36f8e47700 1 -- start start 2024-03-18T20:47:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.006+0000 7f36f8e47700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f40695d0 0x7f36f4197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.006+0000 7f36f8e47700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36f4106090 0x7f36f4197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.006+0000 7f36f8e47700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36f419bad0 0x7f36f419bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.006+0000 7f36f8e47700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f419c550 con 0x7f36f4106090 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.006+0000 7f36f8e47700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f419c6c0 con 0x7f36f419bad0 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.006+0000 7f36f8e47700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f419c880 con 0x7f36f40695d0 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.006+0000 7f36f259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f40695d0 0x7f36f4197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.006+0000 7f36f259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f40695d0 0x7f36f4197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43448/0 (socket says 172.21.15.46:43448) 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.006+0000 7f36f2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36f419bad0 0x7f36f419bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.006+0000 7f36f259c700 1 -- 172.21.15.46:0/3421678 learned_addr learned my addr 172.21.15.46:0/3421678 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36ebfff700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36f4106090 0x7f36f4197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36f259c700 1 -- 172.21.15.46:0/3421678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36f419bad0 msgr2=0x7f36f419bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36f259c700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36f419bad0 0x7f36f419bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36f259c700 1 -- 172.21.15.46:0/3421678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36f4106090 msgr2=0x7f36f4197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36f259c700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36f4106090 0x7f36f4197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36f259c700 1 -- 172.21.15.46:0/3421678 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc0095f0 con 0x7f36f40695d0 2024-03-18T20:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36f2d9d700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36f419bad0 0x7f36f419bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36f259c700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f40695d0 0x7f36f4197240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f36e400e9a0 tx=0x7f36e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36eb7fe700 1 -- 172.21.15.46:0/3421678 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4009790 con 0x7f36f40695d0 2024-03-18T20:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36eb7fe700 1 -- 172.21.15.46:0/3421678 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36e400cbd0 con 0x7f36f40695d0 2024-03-18T20:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36f8e47700 1 -- 172.21.15.46:0/3421678 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f41a3de0 con 0x7f36f40695d0 2024-03-18T20:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36eb7fe700 1 -- 172.21.15.46:0/3421678 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4018680 con 0x7f36f40695d0 2024-03-18T20:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.007+0000 7f36f8e47700 1 -- 172.21.15.46:0/3421678 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f41a4270 con 0x7f36f40695d0 2024-03-18T20:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.010+0000 7f36eb7fe700 1 -- 172.21.15.46:0/3421678 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f36e400cd40 con 0x7f36f40695d0 2024-03-18T20:47:31.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.011+0000 7f36eb7fe700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36e00a1310 0x7f36e00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.011+0000 7f36ebfff700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36e00a1310 0x7f36e00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.011+0000 7f36eb7fe700 1 -- 172.21.15.46:0/3421678 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f36e4014070 con 0x7f36f40695d0 2024-03-18T20:47:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.011+0000 7f36ebfff700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36e00a1310 0x7f36e00a37d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f36dc006010 tx=0x7f36dc00b260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.012+0000 7f3704fdcb80 1 Processor -- start 2024-03-18T20:47:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.012+0000 7f3704fdcb80 1 -- start start 2024-03-18T20:47:31.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f3704fdcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e0a60 0x555db87e0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f3704fdcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e1380 0x555db87f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f3704fdcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f3704fdcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db873a990 con 0x555db87e1380 2024-03-18T20:47:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f3704fdcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db8738350 con 0x555db87e0a60 2024-03-18T20:47:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f3704fdcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db8733c40 con 0x555db87f0630 2024-03-18T20:47:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f36f259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e0a60 0x555db87e0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f36f2d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f36f259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e0a60 0x555db87e0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54710/0 (socket says 172.21.15.46:54710) 2024-03-18T20:47:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f36ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e1380 0x555db87f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f36f259c700 1 -- 172.21.15.46:0/867933053 learned_addr learned my addr 172.21.15.46:0/867933053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f36f259c700 1 -- 172.21.15.46:0/867933053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 msgr2=0x555db87f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f36f259c700 1 --2- 172.21.15.46:0/867933053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f36f259c700 1 -- 172.21.15.46:0/867933053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e1380 msgr2=0x555db87f00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f36f259c700 1 --2- 172.21.15.46:0/867933053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e1380 0x555db87f00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f36f259c700 1 -- 172.21.15.46:0/867933053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db85b53b0 con 0x555db87e0a60 2024-03-18T20:47:31.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f36f259c700 1 --2- 172.21.15.46:0/867933053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e0a60 0x555db87e0e40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f36e4096a40 tx=0x7f36e40999b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d599e36b7ddee99 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.013+0000 7f36e8ff9700 1 -- 172.21.15.46:0/867933053 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e40adea0 con 0x555db87e0a60 2024-03-18T20:47:31.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.014+0000 7f36e8ff9700 1 -- 172.21.15.46:0/867933053 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36e409de00 con 0x555db87e0a60 2024-03-18T20:47:31.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.014+0000 7f36e8ff9700 1 -- 172.21.15.46:0/867933053 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e40a1fc0 con 0x555db87e0a60 2024-03-18T20:47:31.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.014+0000 7f3704fdcb80 1 -- 172.21.15.46:0/867933053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e0a60 msgr2=0x555db87e0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.014+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/867933053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e0a60 0x555db87e0e40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f36e4096a40 tx=0x7f36e40999b0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.015+0000 7f3704fdcb80 1 -- 172.21.15.46:0/867933053 shutdown_connections 2024-03-18T20:47:31.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.015+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/867933053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.015+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/867933053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e0a60 0x555db87e0e40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.015+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/867933053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e1380 0x555db87f00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.015+0000 7f3704fdcb80 1 -- 172.21.15.46:0/867933053 >> 172.21.15.46:0/867933053 conn(0x555db8757220 msgr2=0x555db87dd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.015+0000 7f3704fdcb80 1 -- 172.21.15.46:0/867933053 shutdown_connections 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.015+0000 7f3704fdcb80 1 -- 172.21.15.46:0/867933053 wait complete. 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.015+0000 7f3704fdcb80 1 -- start start 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.015+0000 7f3704fdcb80 1 Processor -- start 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f3704fdcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87e0a60 0x555db87ed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f3704fdcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e1380 0x555db87ed990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f3704fdcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87f0630 0x555db87eded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f3704fdcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db86cfd80 con 0x555db87e1380 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f3704fdcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db86cfa20 con 0x555db87f0630 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f3704fdcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db85b76a0 con 0x555db87e0a60 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f3704fdcb80 10 client.0 _ll_register_callbacks cb 0x555db853b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:31.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e1380 0x555db87ed990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e1380 0x555db87ed990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60626/0 (socket says 172.21.15.46:60626) 2024-03-18T20:47:31.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36ebfff700 1 -- 172.21.15.46:0/3435395376 learned_addr learned my addr 172.21.15.46:0/3435395376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36f2d9d700 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87f0630 0x555db87eded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36f259c700 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87e0a60 0x555db87ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36ebfff700 1 -- 172.21.15.46:0/3435395376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87e0a60 msgr2=0x555db87ed450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36ebfff700 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87e0a60 0x555db87ed450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36ebfff700 1 -- 172.21.15.46:0/3435395376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87f0630 msgr2=0x555db87eded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36ebfff700 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87f0630 0x555db87eded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36ebfff700 1 -- 172.21.15.46:0/3435395376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f41a4270 con 0x555db87e1380 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36f2d9d700 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87f0630 0x555db87eded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36ebfff700 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e1380 0x555db87ed990 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f36dc015000 tx=0x7f36dc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b444ac834023ee9d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.016+0000 7f36d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f36d9ffb700 1 -- 172.21.15.46:0/3435395376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc02d040 con 0x555db87e1380 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f36d9ffb700 1 -- 172.21.15.46:0/3435395376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f36dc004910 con 0x555db87e1380 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f3704fdcb80 1 -- 172.21.15.46:0/3435395376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555db87d4ad0 con 0x555db87e1380 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f36d9ffb700 1 -- 172.21.15.46:0/3435395376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36dc004ad0 con 0x555db87e1380 2024-03-18T20:47:31.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f3704fdcb80 10 client.19500 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f3704fdcb80 1 -- 172.21.15.46:0/3435395376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x555db8561b60 con 0x555db87e1380 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f36d9ffb700 1 -- 172.21.15.46:0/3435395376 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555db87d4ad0 con 0x555db87e1380 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f36d9ffb700 1 -- 172.21.15.46:0/3435395376 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f36dc00f5f0 con 0x555db87e1380 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f3704fdcb80 10 client.19500 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f3704fdcb80 10 client.19500 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f3704fdcb80 20 client.19500 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x555db8800250 0x555db8802670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f3704fdcb80 4 client.19500 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.017+0000 7f3704fdcb80 1 -- 172.21.15.46:0/3435395376 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555db872c700 con 0x555db8800250 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.018+0000 7f36f259c700 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x555db8800250 0x555db8802670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.018+0000 7f36f259c700 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x555db8800250 0x555db8802670 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3af4c2974a548611 server_cookie=dd0d6f7745efd247 in_seq=0 out_seq=0 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.018+0000 7f36d9ffb700 10 client.19500 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:47:31.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.018+0000 7f36d9ffb700 1 -- 172.21.15.46:0/3435395376 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555db872c700 con 0x555db8800250 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.018+0000 7f36d9ffb700 10 client.19500 handle_command_reply: tid=0 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f3704fdcb80 1 client.19500 shutdown 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f3704fdcb80 1 -- 172.21.15.46:0/3435395376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e1380 msgr2=0x555db87ed990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e1380 0x555db87ed990 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f36dc015000 tx=0x7f36dc01f000 comp rx=0 tx=0).stop 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f3704fdcb80 1 -- 172.21.15.46:0/3435395376 shutdown_connections 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x555db8800250 0x555db8802670 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87f0630 0x555db87eded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87e0a60 0x555db87ed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/3435395376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e1380 0x555db87ed990 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f3704fdcb80 1 -- 172.21.15.46:0/3435395376 >> 172.21.15.46:0/3435395376 conn(0x555db8757220 msgr2=0x555db87dcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f36d9ffb700 0 client.19500 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f3704fdcb80 1 -- 172.21.15.46:0/3435395376 shutdown_connections 2024-03-18T20:47:31.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f3704fdcb80 1 -- 172.21.15.46:0/3435395376 wait complete. 2024-03-18T20:47:31.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.019+0000 7f3704fdcb80 20 client.19500 trim_cache size 0 max 16384 2024-03-18T20:47:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.034+0000 7f3704fdcb80 1 Processor -- start 2024-03-18T20:47:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.034+0000 7f3704fdcb80 1 -- start start 2024-03-18T20:47:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.034+0000 7f3704fdcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 0x555db87f7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.034+0000 7f3704fdcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e1380 0x555db87edcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.034+0000 7f3704fdcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.034+0000 7f3704fdcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db87f0420 con 0x555db87e0a60 2024-03-18T20:47:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.034+0000 7f3704fdcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db87fda10 con 0x555db87e1380 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.034+0000 7f3704fdcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db87eea50 con 0x555db87f0630 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36f259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 0x555db87f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36ebfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e1380 0x555db87edcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36ebfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e1380 0x555db87edcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54732/0 (socket says 172.21.15.46:54732) 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36f259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 0x555db87f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60640/0 (socket says 172.21.15.46:60640) 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36ebfff700 1 -- 172.21.15.46:0/1895247642 learned_addr learned my addr 172.21.15.46:0/1895247642 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36f2d9d700 1 --2- 172.21.15.46:0/1895247642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36ebfff700 1 -- 172.21.15.46:0/1895247642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 msgr2=0x555db87fb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36ebfff700 1 --2- 172.21.15.46:0/1895247642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87fb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36ebfff700 1 -- 172.21.15.46:0/1895247642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 msgr2=0x555db87f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36ebfff700 1 --2- 172.21.15.46:0/1895247642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 0x555db87f7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36ebfff700 1 -- 172.21.15.46:0/1895247642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555db86f9ea0 con 0x555db87e1380 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36f259c700 1 --2- 172.21.15.46:0/1895247642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 0x555db87f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36ebfff700 1 --2- 172.21.15.46:0/1895247642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e1380 0x555db87edcf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f36dc000c00 tx=0x7f36dc004f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1537d9efb6ab26cb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36e97fa700 1 -- 172.21.15.46:0/1895247642 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc02d070 con 0x555db87e1380 2024-03-18T20:47:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36e97fa700 1 -- 172.21.15.46:0/1895247642 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36dc01e040 con 0x555db87e1380 2024-03-18T20:47:31.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.035+0000 7f36e97fa700 1 -- 172.21.15.46:0/1895247642 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc03c760 con 0x555db87e1380 2024-03-18T20:47:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.036+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1895247642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e1380 msgr2=0x555db87edcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.036+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/1895247642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e1380 0x555db87edcf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f36dc000c00 tx=0x7f36dc004f60 comp rx=0 tx=0).stop 2024-03-18T20:47:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.036+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1895247642 shutdown_connections 2024-03-18T20:47:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/1895247642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 0x555db87f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/1895247642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/1895247642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e1380 0x555db87edcf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1895247642 >> 172.21.15.46:0/1895247642 conn(0x555db8757220 msgr2=0x555db87dcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1895247642 shutdown_connections 2024-03-18T20:47:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1895247642 wait complete. 2024-03-18T20:47:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 -- start start 2024-03-18T20:47:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 Processor -- start 2024-03-18T20:47:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 0x555db87f5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e1380 0x555db87f5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db8853080 con 0x555db87e0a60 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db88531c0 con 0x555db87e1380 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555db8853350 con 0x555db87f0630 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f3704fdcb80 10 client.0 _ll_register_callbacks cb 0x555db853b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f36f259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 0x555db87f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f36f259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 0x555db87f5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60642/0 (socket says 172.21.15.46:60642) 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f36f259c700 1 -- 172.21.15.46:0/1324425680 learned_addr learned my addr 172.21.15.46:0/1324425680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f36f2d9d700 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.037+0000 7f36ebfff700 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e1380 0x555db87f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f36f2d9d700 1 -- 172.21.15.46:0/1324425680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e1380 msgr2=0x555db87f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f36f2d9d700 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e1380 0x555db87f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f36f2d9d700 1 -- 172.21.15.46:0/1324425680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 msgr2=0x555db87f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f36f2d9d700 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 0x555db87f5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f36f2d9d700 1 -- 172.21.15.46:0/1324425680 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555db85b53b0 con 0x555db87f0630 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f36f2d9d700 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87f5cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x555db87d7100 tx=0x7f36ec0077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d5fc0be9d4fe468 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f36d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f36d9ffb700 1 -- 172.21.15.46:0/1324425680 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec01c070 con 0x555db87f0630 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f36d9ffb700 1 -- 172.21.15.46:0/1324425680 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f36ec007dd0 con 0x555db87f0630 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1324425680 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36e409de00 con 0x555db87f0630 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f36d9ffb700 1 -- 172.21.15.46:0/1324425680 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36ec004bb0 con 0x555db87f0630 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f3704fdcb80 10 client.29146 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.038+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1324425680 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x555db8561b60 con 0x555db87f0630 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.039+0000 7f36d9ffb700 1 -- 172.21.15.46:0/1324425680 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36ec02d020 con 0x555db87f0630 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.039+0000 7f36d9ffb700 1 -- 172.21.15.46:0/1324425680 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f36ec004040 con 0x555db87f0630 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.039+0000 7f3704fdcb80 10 client.29146 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.039+0000 7f3704fdcb80 10 client.29146 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.039+0000 7f3704fdcb80 20 client.29146 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.039+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x555db8854b60 0x555db8856f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.039+0000 7f3704fdcb80 4 client.29146 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.039+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1324425680 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555db8582cf0 con 0x555db8854b60 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.039+0000 7f36f259c700 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x555db8854b60 0x555db8856f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.039+0000 7f36f259c700 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x555db8854b60 0x555db8856f80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7796e292c7907a68 server_cookie=7a1e1bfeb5095a38 in_seq=0 out_seq=0 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.039+0000 7f36d9ffb700 10 client.29146 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f36d9ffb700 1 -- 172.21.15.46:0/1324425680 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555db8582cf0 con 0x555db8854b60 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f36d9ffb700 10 client.29146 handle_command_reply: tid=0 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f3704fdcb80 1 client.29146 shutdown 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1324425680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 msgr2=0x555db87f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87f5cc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x555db87d7100 tx=0x7f36ec0077e0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1324425680 shutdown_connections 2024-03-18T20:47:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x555db8854b60 0x555db8856f80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555db87e0a60 0x555db87f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555db87f0630 0x555db87f5cc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f3704fdcb80 1 --2- 172.21.15.46:0/1324425680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555db87e1380 0x555db87f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f36d9ffb700 0 client.29146 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:47:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1324425680 >> 172.21.15.46:0/1324425680 conn(0x555db8757220 msgr2=0x555db87fb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1324425680 shutdown_connections 2024-03-18T20:47:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f3704fdcb80 1 -- 172.21.15.46:0/1324425680 wait complete. 2024-03-18T20:47:31.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.040+0000 7f3704fdcb80 20 client.29146 trim_cache size 0 max 16384 2024-03-18T20:47:31.048 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:47:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.041+0000 7f36f8e47700 1 -- 172.21.15.46:0/3421678 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36e00a1310 msgr2=0x7f36e00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.041+0000 7f36f8e47700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36e00a1310 0x7f36e00a37d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f36dc006010 tx=0x7f36dc00b260 comp rx=0 tx=0).stop 2024-03-18T20:47:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.041+0000 7f36f8e47700 1 -- 172.21.15.46:0/3421678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f40695d0 msgr2=0x7f36f4197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.041+0000 7f36f8e47700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f40695d0 0x7f36f4197240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f36e400e9a0 tx=0x7f36e400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.041+0000 7f36f8e47700 1 -- 172.21.15.46:0/3421678 shutdown_connections 2024-03-18T20:47:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.041+0000 7f36f8e47700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36e00a1310 0x7f36e00a37d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.041+0000 7f36f8e47700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36f419bad0 0x7f36f419bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.041+0000 7f36f8e47700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36f40695d0 0x7f36f4197240 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.041+0000 7f36f8e47700 1 --2- 172.21.15.46:0/3421678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36f4106090 0x7f36f4197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.041+0000 7f36f8e47700 1 -- 172.21.15.46:0/3421678 >> 172.21.15.46:0/3421678 conn(0x7f36f4076db0 msgr2=0x7f36f4107210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.042+0000 7f36f8e47700 1 -- 172.21.15.46:0/3421678 shutdown_connections 2024-03-18T20:47:31.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.042+0000 7f36f8e47700 1 -- 172.21.15.46:0/3421678 wait complete. 2024-03-18T20:47:31.060 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:47:31.060 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:47:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.208+0000 7fe2100ea700 1 -- 172.21.15.46:0/3166069875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080ffe70 msgr2=0x7fe208100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.208+0000 7fe2100ea700 1 --2- 172.21.15.46:0/3166069875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080ffe70 0x7fe208100250 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fe204009990 tx=0x7fe204009ca0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.209+0000 7fe2100ea700 1 -- 172.21.15.46:0/3166069875 shutdown_connections 2024-03-18T20:47:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.209+0000 7fe2100ea700 1 --2- 172.21.15.46:0/3166069875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080ffe70 0x7fe208100250 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.209+0000 7fe2100ea700 1 --2- 172.21.15.46:0/3166069875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe208104f90 0x7fe20810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.209+0000 7fe2100ea700 1 --2- 172.21.15.46:0/3166069875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe208100820 0x7fe208104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.209+0000 7fe2100ea700 1 -- 172.21.15.46:0/3166069875 >> 172.21.15.46:0/3166069875 conn(0x7fe208076d80 msgr2=0x7fe208077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.209+0000 7fe2100ea700 1 -- 172.21.15.46:0/3166069875 shutdown_connections 2024-03-18T20:47:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.209+0000 7fe2100ea700 1 -- 172.21.15.46:0/3166069875 wait complete. 2024-03-18T20:47:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.209+0000 7fe2100ea700 1 Processor -- start 2024-03-18T20:47:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.209+0000 7fe2100ea700 1 -- start start 2024-03-18T20:47:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe2100ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080ffe70 0x7fe208197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe2100ea700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe208104f90 0x7fe208197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe2100ea700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe20819bb00 0x7fe20819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe2100ea700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20819c580 con 0x7fe2080ffe70 2024-03-18T20:47:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe2100ea700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20819c6f0 con 0x7fe20819bb00 2024-03-18T20:47:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe2100ea700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20819c8b0 con 0x7fe208104f90 2024-03-18T20:47:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe20de86700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080ffe70 0x7fe208197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe20de86700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080ffe70 0x7fe208197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60660/0 (socket says 172.21.15.46:60660) 2024-03-18T20:47:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe20de86700 1 -- 172.21.15.46:0/3130454877 learned_addr learned my addr 172.21.15.46:0/3130454877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe20d685700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe208104f90 0x7fe208197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe20e687700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe20819bb00 0x7fe20819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe20de86700 1 -- 172.21.15.46:0/3130454877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe208104f90 msgr2=0x7fe208197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe20de86700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe208104f90 0x7fe208197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe20de86700 1 -- 172.21.15.46:0/3130454877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe20819bb00 msgr2=0x7fe20819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe20de86700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe20819bb00 0x7fe20819bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe20de86700 1 -- 172.21.15.46:0/3130454877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2040095f0 con 0x7fe2080ffe70 2024-03-18T20:47:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.210+0000 7fe20e687700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe20819bb00 0x7fe20819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.211+0000 7fe20de86700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080ffe70 0x7fe208197220 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fe204005910 tx=0x7fe204004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.211+0000 7fe1feffd700 1 -- 172.21.15.46:0/3130454877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20401d070 con 0x7fe2080ffe70 2024-03-18T20:47:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.211+0000 7fe2100ea700 1 -- 172.21.15.46:0/3130454877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2081a3db0 con 0x7fe2080ffe70 2024-03-18T20:47:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.211+0000 7fe1feffd700 1 -- 172.21.15.46:0/3130454877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe204022470 con 0x7fe2080ffe70 2024-03-18T20:47:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.211+0000 7fe2100ea700 1 -- 172.21.15.46:0/3130454877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2081a41d0 con 0x7fe2080ffe70 2024-03-18T20:47:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.211+0000 7fe1feffd700 1 -- 172.21.15.46:0/3130454877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204017650 con 0x7fe2080ffe70 2024-03-18T20:47:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.213+0000 7fe1feffd700 1 -- 172.21.15.46:0/3130454877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe2040225e0 con 0x7fe2080ffe70 2024-03-18T20:47:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.214+0000 7fe1feffd700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe1f40a1290 0x7fe1f40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.214+0000 7fe1feffd700 1 -- 172.21.15.46:0/3130454877 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe2040cdf00 con 0x7fe2080ffe70 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.214+0000 7fe20d685700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe1f40a1290 0x7fe1f40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.215+0000 7fe20d685700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe1f40a1290 0x7fe1f40a3750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe208198770 tx=0x7fe1f8009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.215+0000 7fe21c27fb80 1 Processor -- start 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.215+0000 7fe21c27fb80 1 -- start start 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe21c27fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44be9a60 0x55ec44be9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe21c27fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bea380 0x55ec44bf90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe21c27fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bf9630 0x55ec44bfba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe21c27fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec44b43990 con 0x55ec44bea380 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe21c27fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec44b41350 con 0x55ec44bf9630 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe21c27fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec44b3cc40 con 0x55ec44be9a60 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe20e687700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bf9630 0x55ec44bfba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe20d685700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bea380 0x55ec44bf90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe20e687700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bf9630 0x55ec44bfba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54778/0 (socket says 172.21.15.46:54778) 2024-03-18T20:47:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe20e687700 1 -- 172.21.15.46:0/95889209 learned_addr learned my addr 172.21.15.46:0/95889209 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe20de86700 1 --2- 172.21.15.46:0/95889209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44be9a60 0x55ec44be9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe20e687700 1 -- 172.21.15.46:0/95889209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44be9a60 msgr2=0x55ec44be9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe20e687700 1 --2- 172.21.15.46:0/95889209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44be9a60 0x55ec44be9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe20e687700 1 -- 172.21.15.46:0/95889209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bea380 msgr2=0x55ec44bf90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe20e687700 1 --2- 172.21.15.46:0/95889209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bea380 0x55ec44bf90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe20e687700 1 -- 172.21.15.46:0/95889209 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec449be3b0 con 0x55ec44bf9630 2024-03-18T20:47:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.216+0000 7fe20e687700 1 --2- 172.21.15.46:0/95889209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bf9630 0x55ec44bfba20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe200003eb0 tx=0x7fe200009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6273d263ba528aa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.217+0000 7fe1f3fff700 1 -- 172.21.15.46:0/95889209 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2000092d0 con 0x55ec44bf9630 2024-03-18T20:47:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.217+0000 7fe1f3fff700 1 -- 172.21.15.46:0/95889209 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe20000f040 con 0x55ec44bf9630 2024-03-18T20:47:31.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.217+0000 7fe1f3fff700 1 -- 172.21.15.46:0/95889209 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20000e8b0 con 0x55ec44bf9630 2024-03-18T20:47:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.217+0000 7fe21c27fb80 1 -- 172.21.15.46:0/95889209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bf9630 msgr2=0x55ec44bfba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.217+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/95889209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bf9630 0x55ec44bfba20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe200003eb0 tx=0x7fe200009920 comp rx=0 tx=0).stop 2024-03-18T20:47:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.218+0000 7fe21c27fb80 1 -- 172.21.15.46:0/95889209 shutdown_connections 2024-03-18T20:47:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.218+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/95889209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bf9630 0x55ec44bfba20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.218+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/95889209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44be9a60 0x55ec44be9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.218+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/95889209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bea380 0x55ec44bf90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.218+0000 7fe21c27fb80 1 -- 172.21.15.46:0/95889209 >> 172.21.15.46:0/95889209 conn(0x55ec44b60220 msgr2=0x55ec44be6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.218+0000 7fe21c27fb80 1 -- 172.21.15.46:0/95889209 shutdown_connections 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.218+0000 7fe21c27fb80 1 -- 172.21.15.46:0/95889209 wait complete. 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.218+0000 7fe21c27fb80 1 -- start start 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.218+0000 7fe21c27fb80 1 Processor -- start 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe21c27fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44be9a60 0x55ec44bf90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe21c27fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bea380 0x55ec44bf63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe21c27fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44bf9630 0x55ec44bf6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe21c27fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec44ad8d80 con 0x55ec44bea380 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe21c27fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec44ad8a20 con 0x55ec44be9a60 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe21c27fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec449c06a0 con 0x55ec44bf9630 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe21c27fb80 10 client.0 _ll_register_callbacks cb 0x55ec449443a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe20de86700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44be9a60 0x55ec44bf90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe20e687700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44bf9630 0x55ec44bf6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe20de86700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44be9a60 0x55ec44bf90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54792/0 (socket says 172.21.15.46:54792) 2024-03-18T20:47:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe20d685700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bea380 0x55ec44bf63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe20de86700 1 -- 172.21.15.46:0/2819070360 learned_addr learned my addr 172.21.15.46:0/2819070360 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe20de86700 1 -- 172.21.15.46:0/2819070360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44bf9630 msgr2=0x55ec44bf6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe20de86700 1 --2- 172.21.15.46:0/2819070360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44bf9630 0x55ec44bf6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe20de86700 1 -- 172.21.15.46:0/2819070360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bea380 msgr2=0x55ec44bf63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe20de86700 1 --2- 172.21.15.46:0/2819070360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bea380 0x55ec44bf63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe20de86700 1 -- 172.21.15.46:0/2819070360 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe200003b90 con 0x55ec44be9a60 2024-03-18T20:47:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe20de86700 1 --2- 172.21.15.46:0/2819070360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44be9a60 0x55ec44bf90e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe2040976a0 tx=0x7fe204022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41655a9fc4ad1a48 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.219+0000 7fe1f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:47:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe1f17fa700 1 -- 172.21.15.46:0/2819070360 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040b2070 con 0x55ec44be9a60 2024-03-18T20:47:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe1f17fa700 1 -- 172.21.15.46:0/2819070360 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe20409b8a0 con 0x55ec44be9a60 2024-03-18T20:47:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2819070360 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec44bddad0 con 0x55ec44be9a60 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe1f17fa700 1 -- 172.21.15.46:0/2819070360 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe20409b1a0 con 0x55ec44be9a60 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe21c27fb80 10 client.29135 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2819070360 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55ec4496ab60 con 0x55ec44be9a60 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe1f17fa700 1 -- 172.21.15.46:0/2819070360 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec44bddad0 con 0x55ec44be9a60 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe1f17fa700 1 -- 172.21.15.46:0/2819070360 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe20409a690 con 0x55ec44be9a60 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe21c27fb80 10 client.29135 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe21c27fb80 10 client.29135 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe21c27fb80 20 client.29135 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2819070360 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55ec44c08ec0 0x55ec44c0b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe21c27fb80 4 client.29135 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2819070360 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec44b35700 con 0x55ec44c08ec0 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.220+0000 7fe20d685700 1 --2- 172.21.15.46:0/2819070360 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55ec44c08ec0 0x55ec44c0b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.221+0000 7fe20d685700 1 --2- 172.21.15.46:0/2819070360 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55ec44c08ec0 0x55ec44c0b2e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4cf3712fd14b9d68 server_cookie=82782eeeb6c3edd0 in_seq=0 out_seq=0 2024-03-18T20:47:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.221+0000 7fe1f17fa700 10 client.29135 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:47:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.221+0000 7fe1f17fa700 1 -- 172.21.15.46:0/2819070360 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ec44b35700 con 0x55ec44c08ec0 2024-03-18T20:47:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.221+0000 7fe1f17fa700 10 client.29135 handle_command_reply: tid=0 2024-03-18T20:47:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.221+0000 7fe21c27fb80 1 client.29135 shutdown 2024-03-18T20:47:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.222+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2819070360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44be9a60 msgr2=0x55ec44bf90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.222+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2819070360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44be9a60 0x55ec44bf90e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe2040976a0 tx=0x7fe204022c90 comp rx=0 tx=0).stop 2024-03-18T20:47:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.222+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2819070360 shutdown_connections 2024-03-18T20:47:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.222+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2819070360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44bf9630 0x55ec44bf6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.222+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2819070360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44be9a60 0x55ec44bf90e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.222+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2819070360 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55ec44c08ec0 0x55ec44c0b2e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.222+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2819070360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bea380 0x55ec44bf63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.222+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2819070360 >> 172.21.15.46:0/2819070360 conn(0x55ec44b60220 msgr2=0x55ec44be5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.222+0000 7fe1f17fa700 0 client.29135 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:47:31.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.222+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2819070360 shutdown_connections 2024-03-18T20:47:31.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.222+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2819070360 wait complete. 2024-03-18T20:47:31.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.222+0000 7fe21c27fb80 20 client.29135 trim_cache size 0 max 16384 2024-03-18T20:47:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe21c27fb80 1 Processor -- start 2024-03-18T20:47:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe21c27fb80 1 -- start start 2024-03-18T20:47:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe21c27fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44be9a60 0x55ec44c053b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe21c27fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 0x55ec44c049b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe21c27fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bf9630 0x55ec44bf16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe21c27fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec44be25e0 con 0x55ec44bf9630 2024-03-18T20:47:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe21c27fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec44bf4920 con 0x55ec44bea380 2024-03-18T20:47:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe21c27fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec44bf2210 con 0x55ec44be9a60 2024-03-18T20:47:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe20d685700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 0x55ec44c049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe20de86700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44be9a60 0x55ec44c053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe20d685700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 0x55ec44c049b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54798/0 (socket says 172.21.15.46:54798) 2024-03-18T20:47:31.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe20de86700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44be9a60 0x55ec44c053b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43540/0 (socket says 172.21.15.46:43540) 2024-03-18T20:47:31.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe20e687700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bf9630 0x55ec44bf16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.237+0000 7fe20d685700 1 -- 172.21.15.46:0/2843673107 learned_addr learned my addr 172.21.15.46:0/2843673107 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.238+0000 7fe20d685700 1 -- 172.21.15.46:0/2843673107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44be9a60 msgr2=0x55ec44c053b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.238+0000 7fe20d685700 1 --2- 172.21.15.46:0/2843673107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44be9a60 0x55ec44c053b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.238+0000 7fe20d685700 1 -- 172.21.15.46:0/2843673107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bf9630 msgr2=0x55ec44bf16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.238+0000 7fe20d685700 1 --2- 172.21.15.46:0/2843673107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bf9630 0x55ec44bf16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.238+0000 7fe20d685700 1 -- 172.21.15.46:0/2843673107 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec44b02ea0 con 0x55ec44bea380 2024-03-18T20:47:31.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.238+0000 7fe20d685700 1 --2- 172.21.15.46:0/2843673107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 0x55ec44c049b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55ec44bf7910 tx=0x7fe1f8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa7c532ee683d4a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.238+0000 7fe1fcff9700 1 -- 172.21.15.46:0/2843673107 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8022030 con 0x55ec44bea380 2024-03-18T20:47:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.238+0000 7fe1fcff9700 1 -- 172.21.15.46:0/2843673107 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1f8023040 con 0x55ec44bea380 2024-03-18T20:47:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.238+0000 7fe1fcff9700 1 -- 172.21.15.46:0/2843673107 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80038c0 con 0x55ec44bea380 2024-03-18T20:47:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.239+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2843673107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 msgr2=0x55ec44c049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.239+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2843673107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 0x55ec44c049b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55ec44bf7910 tx=0x7fe1f8014000 comp rx=0 tx=0).stop 2024-03-18T20:47:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.239+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2843673107 shutdown_connections 2024-03-18T20:47:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.239+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2843673107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44bf9630 0x55ec44bf16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.239+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2843673107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44be9a60 0x55ec44c053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.239+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2843673107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 0x55ec44c049b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.239+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2843673107 >> 172.21.15.46:0/2843673107 conn(0x55ec44b60220 msgr2=0x55ec44c0aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.239+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2843673107 shutdown_connections 2024-03-18T20:47:31.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.239+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2843673107 wait complete. 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.239+0000 7fe21c27fb80 1 -- start start 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe21c27fb80 1 Processor -- start 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe21c27fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44be9a60 0x55ec44bfda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe21c27fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 0x55ec44bfdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe21c27fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44bf9630 0x55ec44bfe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe21c27fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec44c5bcf0 con 0x55ec44be9a60 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe21c27fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec44c5be30 con 0x55ec44bea380 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe21c27fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec44c5bf70 con 0x55ec44bf9630 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe21c27fb80 10 client.0 _ll_register_callbacks cb 0x7fe204097570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe20de86700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44be9a60 0x55ec44bfda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe20e687700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44bf9630 0x55ec44bfe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe20de86700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44be9a60 0x55ec44bfda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60704/0 (socket says 172.21.15.46:60704) 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe20e687700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44bf9630 0x55ec44bfe510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43542/0 (socket says 172.21.15.46:43542) 2024-03-18T20:47:31.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe20de86700 1 -- 172.21.15.46:0/2024752690 learned_addr learned my addr 172.21.15.46:0/2024752690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.240+0000 7fe20d685700 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 0x55ec44bfdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe20de86700 1 -- 172.21.15.46:0/2024752690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44bf9630 msgr2=0x55ec44bfe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe20de86700 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44bf9630 0x55ec44bfe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe20de86700 1 -- 172.21.15.46:0/2024752690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 msgr2=0x55ec44bfdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe20de86700 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 0x55ec44bfdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe20de86700 1 -- 172.21.15.46:0/2024752690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f8017070 con 0x55ec44be9a60 2024-03-18T20:47:31.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe20e687700 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44bf9630 0x55ec44bfe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe20d685700 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 0x55ec44bfdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe20de86700 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44be9a60 0x55ec44bfda90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe2040994f0 tx=0x7fe20409a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede33631886cc4f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe1f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe1f17fa700 1 -- 172.21.15.46:0/2024752690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040b2070 con 0x55ec44be9a60 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe1f17fa700 1 -- 172.21.15.46:0/2024752690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe2040b5e40 con 0x55ec44be9a60 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2024752690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe20000f040 con 0x55ec44be9a60 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe1f17fa700 1 -- 172.21.15.46:0/2024752690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe20409b8a0 con 0x55ec44be9a60 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe21c27fb80 10 client.19524 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2024752690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55ec449c2cd0 con 0x55ec44be9a60 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.241+0000 7fe1f17fa700 1 -- 172.21.15.46:0/2024752690 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe20000f040 con 0x55ec44be9a60 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.242+0000 7fe1f17fa700 1 -- 172.21.15.46:0/2024752690 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe204096f90 con 0x55ec44be9a60 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.242+0000 7fe21c27fb80 10 client.19524 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.242+0000 7fe21c27fb80 10 client.19524 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.242+0000 7fe21c27fb80 20 client.19524 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.242+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55ec44c5d230 0x55ec44c5f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.242+0000 7fe21c27fb80 4 client.19524 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:47:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.242+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2024752690 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ec4498bcf0 con 0x55ec44c5d230 2024-03-18T20:47:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.242+0000 7fe20d685700 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55ec44c5d230 0x55ec44c5f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe1f17fa700 10 client.19524 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:47:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe20d685700 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55ec44c5d230 0x55ec44c5f620 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8856f45ee83197db server_cookie=346b5ae2168aa465 in_seq=0 out_seq=0 2024-03-18T20:47:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe1f17fa700 1 -- 172.21.15.46:0/2024752690 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ec4498bcf0 con 0x55ec44c5d230 2024-03-18T20:47:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe1f17fa700 10 client.19524 handle_command_reply: tid=0 2024-03-18T20:47:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe21c27fb80 1 client.19524 shutdown 2024-03-18T20:47:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2024752690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44be9a60 msgr2=0x55ec44bfda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44be9a60 0x55ec44bfda90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe2040994f0 tx=0x7fe20409a8c0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2024752690 shutdown_connections 2024-03-18T20:47:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55ec44c5d230 0x55ec44c5f620 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec44be9a60 0x55ec44bfda90 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ec44bf9630 0x55ec44bfe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe21c27fb80 1 --2- 172.21.15.46:0/2024752690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec44bea380 0x55ec44bfdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe1f17fa700 0 client.19524 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:47:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2024752690 >> 172.21.15.46:0/2024752690 conn(0x55ec44b60220 msgr2=0x55ec44c049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2024752690 shutdown_connections 2024-03-18T20:47:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe21c27fb80 1 -- 172.21.15.46:0/2024752690 wait complete. 2024-03-18T20:47:31.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.243+0000 7fe21c27fb80 20 client.19524 trim_cache size 0 max 16384 2024-03-18T20:47:31.252 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:47:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.244+0000 7fe2100ea700 1 -- 172.21.15.46:0/3130454877 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe1f40a1290 msgr2=0x7fe1f40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.244+0000 7fe2100ea700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe1f40a1290 0x7fe1f40a3750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe208198770 tx=0x7fe1f8009260 comp rx=0 tx=0).stop 2024-03-18T20:47:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.244+0000 7fe2100ea700 1 -- 172.21.15.46:0/3130454877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080ffe70 msgr2=0x7fe208197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.244+0000 7fe2100ea700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080ffe70 0x7fe208197220 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fe204005910 tx=0x7fe204004730 comp rx=0 tx=0).stop 2024-03-18T20:47:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.244+0000 7fe2100ea700 1 -- 172.21.15.46:0/3130454877 shutdown_connections 2024-03-18T20:47:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.244+0000 7fe2100ea700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe1f40a1290 0x7fe1f40a3750 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.244+0000 7fe2100ea700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2080ffe70 0x7fe208197220 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.244+0000 7fe2100ea700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe20819bb00 0x7fe20819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.244+0000 7fe2100ea700 1 --2- 172.21.15.46:0/3130454877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe208104f90 0x7fe208197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.244+0000 7fe2100ea700 1 -- 172.21.15.46:0/3130454877 >> 172.21.15.46:0/3130454877 conn(0x7fe208076d80 msgr2=0x7fe2080693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.244+0000 7fe2100ea700 1 -- 172.21.15.46:0/3130454877 shutdown_connections 2024-03-18T20:47:31.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.244+0000 7fe2100ea700 1 -- 172.21.15.46:0/3130454877 wait complete. 2024-03-18T20:47:31.263 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:47:31.263 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:47:31.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.422+0000 7fc5282e0700 1 -- 172.21.15.46:0/408983069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5200ffe50 msgr2=0x7fc520100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.422+0000 7fc5282e0700 1 --2- 172.21.15.46:0/408983069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5200ffe50 0x7fc520100230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009990 tx=0x7fc51c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.422+0000 7fc5282e0700 1 -- 172.21.15.46:0/408983069 shutdown_connections 2024-03-18T20:47:31.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.422+0000 7fc5282e0700 1 --2- 172.21.15.46:0/408983069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520104f60 0x7fc52010cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.422+0000 7fc5282e0700 1 --2- 172.21.15.46:0/408983069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5200ffe50 0x7fc520100230 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.422+0000 7fc5282e0700 1 --2- 172.21.15.46:0/408983069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc520100800 0x7fc520104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.422+0000 7fc5282e0700 1 -- 172.21.15.46:0/408983069 >> 172.21.15.46:0/408983069 conn(0x7fc520076d40 msgr2=0x7fc520077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.422+0000 7fc5282e0700 1 -- 172.21.15.46:0/408983069 shutdown_connections 2024-03-18T20:47:31.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.422+0000 7fc5282e0700 1 -- 172.21.15.46:0/408983069 wait complete. 2024-03-18T20:47:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.423+0000 7fc5282e0700 1 Processor -- start 2024-03-18T20:47:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.423+0000 7fc5282e0700 1 -- start start 2024-03-18T20:47:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.423+0000 7fc5282e0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5200ffe50 0x7fc5201971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.423+0000 7fc5282e0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520104f60 0x7fc520197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.423+0000 7fc5282e0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc52019bab0 0x7fc52019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.423+0000 7fc5282e0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52019c530 con 0x7fc520104f60 2024-03-18T20:47:31.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.423+0000 7fc5282e0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52019c6a0 con 0x7fc5200ffe50 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.423+0000 7fc5282e0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52019c860 con 0x7fc52019bab0 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52587b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520104f60 0x7fc520197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52607c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5200ffe50 0x7fc5201971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52687d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc52019bab0 0x7fc52019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52587b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520104f60 0x7fc520197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60712/0 (socket says 172.21.15.46:60712) 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52607c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5200ffe50 0x7fc5201971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54822/0 (socket says 172.21.15.46:54822) 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52587b700 1 -- 172.21.15.46:0/4012472665 learned_addr learned my addr 172.21.15.46:0/4012472665 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52587b700 1 -- 172.21.15.46:0/4012472665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc52019bab0 msgr2=0x7fc52019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52587b700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc52019bab0 0x7fc52019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52587b700 1 -- 172.21.15.46:0/4012472665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5200ffe50 msgr2=0x7fc5201971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52587b700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5200ffe50 0x7fc5201971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52587b700 1 -- 172.21.15.46:0/4012472665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c0095f0 con 0x7fc520104f60 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52687d700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc52019bab0 0x7fc52019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc52587b700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520104f60 0x7fc520197710 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fc51000d980 tx=0x7fc51000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc5177fe700 1 -- 172.21.15.46:0/4012472665 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000f890 con 0x7fc520104f60 2024-03-18T20:47:31.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc5177fe700 1 -- 172.21.15.46:0/4012472665 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc510010460 con 0x7fc520104f60 2024-03-18T20:47:31.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc5282e0700 1 -- 172.21.15.46:0/4012472665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5201a3dc0 con 0x7fc520104f60 2024-03-18T20:47:31.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc5177fe700 1 -- 172.21.15.46:0/4012472665 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000e610 con 0x7fc520104f60 2024-03-18T20:47:31.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.424+0000 7fc5282e0700 1 -- 172.21.15.46:0/4012472665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5201a4240 con 0x7fc520104f60 2024-03-18T20:47:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.427+0000 7fc5177fe700 1 -- 172.21.15.46:0/4012472665 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc5100105d0 con 0x7fc520104f60 2024-03-18T20:47:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.427+0000 7fc5177fe700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc50c0a1290 0x7fc50c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.427+0000 7fc52607c700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc50c0a1290 0x7fc50c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.427+0000 7fc5177fe700 1 -- 172.21.15.46:0/4012472665 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc5100cca70 con 0x7fc520104f60 2024-03-18T20:47:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.428+0000 7fc52607c700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc50c0a1290 0x7fc50c0a3750 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00b2e0 tx=0x7fc51c005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.428+0000 7fc534475b80 1 Processor -- start 2024-03-18T20:47:31.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc534475b80 1 -- start start 2024-03-18T20:47:31.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc534475b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64925a60 0x562b64925e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc534475b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 0x562b64930ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc534475b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b64933610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc534475b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6487f990 con 0x562b64926380 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc534475b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b6487d350 con 0x562b64931220 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc534475b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b64878c40 con 0x562b64925a60 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc52607c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64925a60 0x562b64925e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc52607c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64925a60 0x562b64925e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43568/0 (socket says 172.21.15.46:43568) 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc52607c700 1 -- 172.21.15.46:0/2588104151 learned_addr learned my addr 172.21.15.46:0/2588104151 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc52587b700 1 --2- 172.21.15.46:0/2588104151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 0x562b64930ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc52687d700 1 --2- 172.21.15.46:0/2588104151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b64933610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc52607c700 1 -- 172.21.15.46:0/2588104151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 msgr2=0x562b64933610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc52607c700 1 --2- 172.21.15.46:0/2588104151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b64933610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc52607c700 1 -- 172.21.15.46:0/2588104151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 msgr2=0x562b64930ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc52607c700 1 --2- 172.21.15.46:0/2588104151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 0x562b64930ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc52607c700 1 -- 172.21.15.46:0/2588104151 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b646fa3b0 con 0x562b64925a60 2024-03-18T20:47:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc52687d700 1 --2- 172.21.15.46:0/2588104151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b64933610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:47:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.429+0000 7fc52587b700 1 --2- 172.21.15.46:0/2588104151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 0x562b64930ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.430+0000 7fc52607c700 1 --2- 172.21.15.46:0/2588104151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64925a60 0x562b64925e40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc51c015000 tx=0x7fc51c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a731f249e36ea015 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.430+0000 7fc514da8700 1 -- 172.21.15.46:0/2588104151 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c004700 con 0x562b64925a60 2024-03-18T20:47:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.430+0000 7fc514da8700 1 -- 172.21.15.46:0/2588104151 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc51c01e040 con 0x562b64925a60 2024-03-18T20:47:31.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.430+0000 7fc514da8700 1 -- 172.21.15.46:0/2588104151 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00f960 con 0x562b64925a60 2024-03-18T20:47:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 -- 172.21.15.46:0/2588104151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64925a60 msgr2=0x562b64925e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 --2- 172.21.15.46:0/2588104151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64925a60 0x562b64925e40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc51c015000 tx=0x7fc51c020000 comp rx=0 tx=0).stop 2024-03-18T20:47:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 -- 172.21.15.46:0/2588104151 shutdown_connections 2024-03-18T20:47:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 --2- 172.21.15.46:0/2588104151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b64933610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 --2- 172.21.15.46:0/2588104151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64925a60 0x562b64925e40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 --2- 172.21.15.46:0/2588104151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 0x562b64930ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 -- 172.21.15.46:0/2588104151 >> 172.21.15.46:0/2588104151 conn(0x562b6489c220 msgr2=0x562b64922020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 -- 172.21.15.46:0/2588104151 shutdown_connections 2024-03-18T20:47:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 -- 172.21.15.46:0/2588104151 wait complete. 2024-03-18T20:47:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 -- start start 2024-03-18T20:47:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 Processor -- start 2024-03-18T20:47:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 0x562b64930c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64926380 0x562b6492df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b6492e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b64814d80 con 0x562b64925a60 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b64814a20 con 0x562b64931220 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.431+0000 7fc534475b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b646fc6a0 con 0x562b64926380 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc534475b80 10 client.0 _ll_register_callbacks cb 0x562b646803a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc52607c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 0x562b64930c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc52607c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 0x562b64930c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60742/0 (socket says 172.21.15.46:60742) 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc52607c700 1 -- 172.21.15.46:0/3390875368 learned_addr learned my addr 172.21.15.46:0/3390875368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc52687d700 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b6492e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc52587b700 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64926380 0x562b6492df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc52607c700 1 -- 172.21.15.46:0/3390875368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64926380 msgr2=0x562b6492df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc52607c700 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64926380 0x562b6492df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc52607c700 1 -- 172.21.15.46:0/3390875368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 msgr2=0x562b6492e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc52607c700 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b6492e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc52607c700 1 -- 172.21.15.46:0/3390875368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5201a4240 con 0x562b64925a60 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc52687d700 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b6492e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc52607c700 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 0x562b64930c60 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009ce0 tx=0x7fc51c0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a59e2957cf0592c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc509ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc509ffb700 1 -- 172.21.15.46:0/3390875368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c004700 con 0x562b64925a60 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc534475b80 1 -- 172.21.15.46:0/3390875368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b64919ad0 con 0x562b64925a60 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.432+0000 7fc509ffb700 1 -- 172.21.15.46:0/3390875368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc51c00be60 con 0x562b64925a60 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.433+0000 7fc509ffb700 1 -- 172.21.15.46:0/3390875368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc51c004860 con 0x562b64925a60 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.433+0000 7fc534475b80 10 client.19536 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.433+0000 7fc534475b80 1 -- 172.21.15.46:0/3390875368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7fc52010d350 con 0x562b64925a60 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.433+0000 7fc509ffb700 1 -- 172.21.15.46:0/3390875368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc51c047020 con 0x562b64925a60 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.433+0000 7fc509ffb700 1 -- 172.21.15.46:0/3390875368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc51c024a80 con 0x562b64925a60 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.433+0000 7fc534475b80 10 client.19536 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.433+0000 7fc534475b80 10 client.19536 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:47:31.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.433+0000 7fc534475b80 20 client.19536 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.433+0000 7fc534475b80 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x562b64940910 0x562b64942d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.433+0000 7fc534475b80 4 client.19536 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.433+0000 7fc534475b80 1 -- 172.21.15.46:0/3390875368 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b64871700 con 0x562b64940910 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.433+0000 7fc52587b700 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x562b64940910 0x562b64942d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.434+0000 7fc52587b700 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x562b64940910 0x562b64942d00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a727936391df3de9 server_cookie=56726afb09d2bcd3 in_seq=0 out_seq=0 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.434+0000 7fc509ffb700 10 client.19536 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.434+0000 7fc509ffb700 1 -- 172.21.15.46:0/3390875368 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562b64871700 con 0x562b64940910 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.434+0000 7fc509ffb700 10 client.19536 handle_command_reply: tid=0 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.434+0000 7fc534475b80 1 client.19536 shutdown 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.434+0000 7fc534475b80 1 -- 172.21.15.46:0/3390875368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 msgr2=0x562b64930c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.434+0000 7fc534475b80 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 0x562b64930c60 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009ce0 tx=0x7fc51c0057e0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.435+0000 7fc534475b80 1 -- 172.21.15.46:0/3390875368 shutdown_connections 2024-03-18T20:47:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.435+0000 7fc534475b80 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x562b64940910 0x562b64942d00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.435+0000 7fc534475b80 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 0x562b64930c60 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.435+0000 7fc534475b80 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b6492e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.435+0000 7fc534475b80 1 --2- 172.21.15.46:0/3390875368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64926380 0x562b6492df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.435+0000 7fc534475b80 1 -- 172.21.15.46:0/3390875368 >> 172.21.15.46:0/3390875368 conn(0x562b6489c220 msgr2=0x562b64921c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.435+0000 7fc509ffb700 0 client.19536 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:47:31.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.435+0000 7fc534475b80 1 -- 172.21.15.46:0/3390875368 shutdown_connections 2024-03-18T20:47:31.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.435+0000 7fc534475b80 1 -- 172.21.15.46:0/3390875368 wait complete. 2024-03-18T20:47:31.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.435+0000 7fc534475b80 20 client.19536 trim_cache size 0 max 16384 2024-03-18T20:47:31.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.449+0000 7fc534475b80 1 Processor -- start 2024-03-18T20:47:31.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.449+0000 7fc534475b80 1 -- start start 2024-03-18T20:47:31.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.449+0000 7fc534475b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 0x562b64937e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.449+0000 7fc534475b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64926380 0x562b64929280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.449+0000 7fc534475b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64931220 0x562b649297c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.449+0000 7fc534475b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b649267e0 con 0x562b64925a60 2024-03-18T20:47:31.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.449+0000 7fc534475b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b64926d30 con 0x562b64926380 2024-03-18T20:47:31.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.449+0000 7fc534475b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b649273d0 con 0x562b64931220 2024-03-18T20:47:31.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc52607c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 0x562b64937e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc52607c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 0x562b64937e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60752/0 (socket says 172.21.15.46:60752) 2024-03-18T20:47:31.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc52607c700 1 -- 172.21.15.46:0/3493444862 learned_addr learned my addr 172.21.15.46:0/3493444862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc52687d700 1 --2- 172.21.15.46:0/3493444862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64931220 0x562b649297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc52587b700 1 --2- 172.21.15.46:0/3493444862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64926380 0x562b64929280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc52607c700 1 -- 172.21.15.46:0/3493444862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64931220 msgr2=0x562b649297c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc52607c700 1 --2- 172.21.15.46:0/3493444862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64931220 0x562b649297c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc52607c700 1 -- 172.21.15.46:0/3493444862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64926380 msgr2=0x562b64929280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc52607c700 1 --2- 172.21.15.46:0/3493444862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64926380 0x562b64929280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc52607c700 1 -- 172.21.15.46:0/3493444862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b646a6b60 con 0x562b64925a60 2024-03-18T20:47:31.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc52607c700 1 --2- 172.21.15.46:0/3493444862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 0x562b64937e20 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc51c005740 tx=0x7fc51c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1063c715cd001e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc5155a9700 1 -- 172.21.15.46:0/3493444862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00beb0 con 0x562b64925a60 2024-03-18T20:47:31.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc5155a9700 1 -- 172.21.15.46:0/3493444862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc51c047030 con 0x562b64925a60 2024-03-18T20:47:31.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.450+0000 7fc5155a9700 1 -- 172.21.15.46:0/3493444862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c024700 con 0x562b64925a60 2024-03-18T20:47:31.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.451+0000 7fc534475b80 1 -- 172.21.15.46:0/3493444862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 msgr2=0x562b64937e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.451+0000 7fc534475b80 1 --2- 172.21.15.46:0/3493444862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 0x562b64937e20 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc51c005740 tx=0x7fc51c005fb0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.451+0000 7fc534475b80 1 -- 172.21.15.46:0/3493444862 shutdown_connections 2024-03-18T20:47:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.451+0000 7fc534475b80 1 --2- 172.21.15.46:0/3493444862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64925a60 0x562b64937e20 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.451+0000 7fc534475b80 1 --2- 172.21.15.46:0/3493444862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64931220 0x562b649297c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.451+0000 7fc534475b80 1 --2- 172.21.15.46:0/3493444862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64926380 0x562b64929280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.451+0000 7fc534475b80 1 -- 172.21.15.46:0/3493444862 >> 172.21.15.46:0/3493444862 conn(0x562b6489c220 msgr2=0x562b64930870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.451+0000 7fc534475b80 1 -- 172.21.15.46:0/3493444862 shutdown_connections 2024-03-18T20:47:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.451+0000 7fc534475b80 1 -- 172.21.15.46:0/3493444862 wait complete. 2024-03-18T20:47:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.451+0000 7fc534475b80 1 -- start start 2024-03-18T20:47:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc534475b80 1 Processor -- start 2024-03-18T20:47:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc534475b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64925a60 0x562b64937de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc534475b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 0x562b64935150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc534475b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b64935840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc534475b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b64935f50 con 0x562b64926380 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc534475b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b64997ae0 con 0x562b64931220 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc534475b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b64997c20 con 0x562b64925a60 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc534475b80 10 client.0 _ll_register_callbacks cb 0x7fc510095f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc52587b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 0x562b64935150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc52607c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64925a60 0x562b64937de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc52687d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b64935840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc52587b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 0x562b64935150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60760/0 (socket says 172.21.15.46:60760) 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc52587b700 1 -- 172.21.15.46:0/500054742 learned_addr learned my addr 172.21.15.46:0/500054742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc52687d700 1 -- 172.21.15.46:0/500054742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64925a60 msgr2=0x562b64937de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc52687d700 1 --2- 172.21.15.46:0/500054742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64925a60 0x562b64937de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc52687d700 1 -- 172.21.15.46:0/500054742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 msgr2=0x562b64935150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc52687d700 1 --2- 172.21.15.46:0/500054742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 0x562b64935150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc52687d700 1 -- 172.21.15.46:0/500054742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52010d350 con 0x562b64931220 2024-03-18T20:47:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc52587b700 1 --2- 172.21.15.46:0/500054742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 0x562b64935150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc52687d700 1 --2- 172.21.15.46:0/500054742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b64935840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc5180094f0 tx=0x7fc5180046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a2e30aaf4425168 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.452+0000 7fc509ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:47:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc509ffb700 1 -- 172.21.15.46:0/500054742 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51800ebc0 con 0x562b64931220 2024-03-18T20:47:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc509ffb700 1 -- 172.21.15.46:0/500054742 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc51800ed20 con 0x562b64931220 2024-03-18T20:47:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc534475b80 1 -- 172.21.15.46:0/500054742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc51c01e040 con 0x562b64931220 2024-03-18T20:47:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc509ffb700 1 -- 172.21.15.46:0/500054742 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc51800eee0 con 0x562b64931220 2024-03-18T20:47:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc534475b80 10 client.29159 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc534475b80 1 -- 172.21.15.46:0/500054742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x562b6483eea0 con 0x562b64931220 2024-03-18T20:47:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc509ffb700 1 -- 172.21.15.46:0/500054742 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc51c01e040 con 0x562b64931220 2024-03-18T20:47:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc509ffb700 1 -- 172.21.15.46:0/500054742 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc518017530 con 0x562b64931220 2024-03-18T20:47:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc534475b80 10 client.29159 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc534475b80 10 client.29159 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:47:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc534475b80 20 client.29159 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc534475b80 1 --2- 172.21.15.46:0/500054742 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x562b64999440 0x562b6499b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc534475b80 4 client.29159 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:47:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.453+0000 7fc534475b80 1 -- 172.21.15.46:0/500054742 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562b646c7cf0 con 0x562b64999440 2024-03-18T20:47:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc52607c700 1 --2- 172.21.15.46:0/500054742 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x562b64999440 0x562b6499b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc509ffb700 10 client.29159 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:47:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc52607c700 1 --2- 172.21.15.46:0/500054742 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x562b64999440 0x562b6499b860 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=898766a38aabae60 server_cookie=430b89024872ebf2 in_seq=0 out_seq=0 2024-03-18T20:47:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc509ffb700 1 -- 172.21.15.46:0/500054742 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562b646c7cf0 con 0x562b64999440 2024-03-18T20:47:31.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc509ffb700 10 client.29159 handle_command_reply: tid=0 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc534475b80 1 client.29159 shutdown 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc534475b80 1 -- 172.21.15.46:0/500054742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 msgr2=0x562b64935840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc534475b80 1 --2- 172.21.15.46:0/500054742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b64935840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc5180094f0 tx=0x7fc5180046e0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc534475b80 1 -- 172.21.15.46:0/500054742 shutdown_connections 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc534475b80 1 --2- 172.21.15.46:0/500054742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b64931220 0x562b64935840 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc534475b80 1 --2- 172.21.15.46:0/500054742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b64925a60 0x562b64937de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc534475b80 1 --2- 172.21.15.46:0/500054742 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x562b64999440 0x562b6499b860 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.454+0000 7fc534475b80 1 --2- 172.21.15.46:0/500054742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b64926380 0x562b64935150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc509ffb700 0 client.29159 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc534475b80 1 -- 172.21.15.46:0/500054742 >> 172.21.15.46:0/500054742 conn(0x562b6489c220 msgr2=0x562b6492d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc534475b80 1 -- 172.21.15.46:0/500054742 shutdown_connections 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc534475b80 1 -- 172.21.15.46:0/500054742 wait complete. 2024-03-18T20:47:31.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc534475b80 20 client.29159 trim_cache size 0 max 16384 2024-03-18T20:47:31.464 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:47:31.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc5282e0700 1 -- 172.21.15.46:0/4012472665 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc50c0a1290 msgr2=0x7fc50c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc5282e0700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc50c0a1290 0x7fc50c0a3750 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00b2e0 tx=0x7fc51c005960 comp rx=0 tx=0).stop 2024-03-18T20:47:31.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc5282e0700 1 -- 172.21.15.46:0/4012472665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520104f60 msgr2=0x7fc520197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc5282e0700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520104f60 0x7fc520197710 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fc51000d980 tx=0x7fc51000dc90 comp rx=0 tx=0).stop 2024-03-18T20:47:31.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc5282e0700 1 -- 172.21.15.46:0/4012472665 shutdown_connections 2024-03-18T20:47:31.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc5282e0700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc50c0a1290 0x7fc50c0a3750 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc5282e0700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc52019bab0 0x7fc52019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc5282e0700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5200ffe50 0x7fc5201971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc5282e0700 1 --2- 172.21.15.46:0/4012472665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520104f60 0x7fc520197710 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc5282e0700 1 -- 172.21.15.46:0/4012472665 >> 172.21.15.46:0/4012472665 conn(0x7fc520076d40 msgr2=0x7fc520068220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc5282e0700 1 -- 172.21.15.46:0/4012472665 shutdown_connections 2024-03-18T20:47:31.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.455+0000 7fc5282e0700 1 -- 172.21.15.46:0/4012472665 wait complete. 2024-03-18T20:47:31.472 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:47:31.472 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:47:31.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:31 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2446432628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:47:31.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:31 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2446432628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:47:31.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.618+0000 7f3edb8ca700 1 -- 172.21.15.46:0/2444665613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed40ffe50 msgr2=0x7f3ed4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.618+0000 7f3edb8ca700 1 --2- 172.21.15.46:0/2444665613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed40ffe50 0x7f3ed4100230 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00098f0 tx=0x7f3ed0009c00 comp rx=0 tx=0).stop 2024-03-18T20:47:31.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.618+0000 7f3edb8ca700 1 -- 172.21.15.46:0/2444665613 shutdown_connections 2024-03-18T20:47:31.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.618+0000 7f3edb8ca700 1 --2- 172.21.15.46:0/2444665613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ed4104f60 0x7f3ed410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.618+0000 7f3edb8ca700 1 --2- 172.21.15.46:0/2444665613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed40ffe50 0x7f3ed4100230 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.618+0000 7f3edb8ca700 1 --2- 172.21.15.46:0/2444665613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed4100800 0x7f3ed4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.618+0000 7f3edb8ca700 1 -- 172.21.15.46:0/2444665613 >> 172.21.15.46:0/2444665613 conn(0x7f3ed4076d40 msgr2=0x7f3ed4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.618+0000 7f3edb8ca700 1 -- 172.21.15.46:0/2444665613 shutdown_connections 2024-03-18T20:47:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.618+0000 7f3edb8ca700 1 -- 172.21.15.46:0/2444665613 wait complete. 2024-03-18T20:47:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.619+0000 7f3edb8ca700 1 Processor -- start 2024-03-18T20:47:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.619+0000 7f3edb8ca700 1 -- start start 2024-03-18T20:47:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.619+0000 7f3edb8ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed40ffe50 0x7f3ed41960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.619+0000 7f3edb8ca700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ed4104f60 0x7f3ed41965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.619+0000 7f3edb8ca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed419a990 0x7f3ed419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.619+0000 7f3edb8ca700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed419b410 con 0x7f3ed40ffe50 2024-03-18T20:47:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.619+0000 7f3edb8ca700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed419b580 con 0x7f3ed419a990 2024-03-18T20:47:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.619+0000 7f3edb8ca700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed419b740 con 0x7f3ed4104f60 2024-03-18T20:47:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3ed9666700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed40ffe50 0x7f3ed41960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3ed8e65700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ed4104f60 0x7f3ed41965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3ed8e65700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ed4104f60 0x7f3ed41965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43608/0 (socket says 172.21.15.46:43608) 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3ed9666700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed40ffe50 0x7f3ed41960b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60778/0 (socket says 172.21.15.46:60778) 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3ed8e65700 1 -- 172.21.15.46:0/3990722271 learned_addr learned my addr 172.21.15.46:0/3990722271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3ed9e67700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed419a990 0x7f3ed419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3ed8e65700 1 -- 172.21.15.46:0/3990722271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed419a990 msgr2=0x7f3ed419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3ed8e65700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed419a990 0x7f3ed419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3ed8e65700 1 -- 172.21.15.46:0/3990722271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed40ffe50 msgr2=0x7f3ed41960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3ed8e65700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed40ffe50 0x7f3ed41960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3ed8e65700 1 -- 172.21.15.46:0/3990722271 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed0009520 con 0x7f3ed4104f60 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3ed8e65700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ed4104f60 0x7f3ed41965f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e9f0 tx=0x7f3ec400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3eca7fc700 1 -- 172.21.15.46:0/3990722271 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4009690 con 0x7f3ed4104f60 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3edb8ca700 1 -- 172.21.15.46:0/3990722271 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed4067e50 con 0x7f3ed4104f60 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3eca7fc700 1 -- 172.21.15.46:0/3990722271 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ec400ce80 con 0x7f3ed4104f60 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.620+0000 7f3eca7fc700 1 -- 172.21.15.46:0/3990722271 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4018420 con 0x7f3ed4104f60 2024-03-18T20:47:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.621+0000 7f3edb8ca700 1 -- 172.21.15.46:0/3990722271 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed4068360 con 0x7f3ed4104f60 2024-03-18T20:47:31.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.623+0000 7f3eca7fc700 1 -- 172.21.15.46:0/3990722271 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3ec400c9c0 con 0x7f3ed4104f60 2024-03-18T20:47:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.623+0000 7f3eca7fc700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ec00a1290 0x7f3ec00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.623+0000 7f3ed9666700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ec00a1290 0x7f3ec00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.624+0000 7f3eca7fc700 1 -- 172.21.15.46:0/3990722271 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3ec4014070 con 0x7f3ed4104f60 2024-03-18T20:47:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.624+0000 7f3ed9666700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ec00a1290 0x7f3ec00a3750 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009fd0 tx=0x7f3ed0019040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.624+0000 7f3ee7a5fb80 1 Processor -- start 2024-03-18T20:47:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ee7a5fb80 1 -- start start 2024-03-18T20:47:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ee7a5fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28ea60 0x55ac9d28ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ee7a5fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d299ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ee7a5fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 0x55ac9d29c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ee7a5fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9d1e8990 con 0x55ac9d29a220 2024-03-18T20:47:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ee7a5fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9d1e6350 con 0x55ac9d28ea60 2024-03-18T20:47:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ee7a5fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9d1e1c40 con 0x55ac9d28f380 2024-03-18T20:47:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ed9e67700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 0x55ac9d29c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ed8e65700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d299ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ed9e67700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 0x55ac9d29c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60790/0 (socket says 172.21.15.46:60790) 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ed9e67700 1 -- 172.21.15.46:0/508375512 learned_addr learned my addr 172.21.15.46:0/508375512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ed9666700 1 --2- 172.21.15.46:0/508375512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28ea60 0x55ac9d28ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ed8e65700 1 -- 172.21.15.46:0/508375512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28ea60 msgr2=0x55ac9d28ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ed8e65700 1 --2- 172.21.15.46:0/508375512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28ea60 0x55ac9d28ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ed8e65700 1 -- 172.21.15.46:0/508375512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 msgr2=0x55ac9d29c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ed8e65700 1 --2- 172.21.15.46:0/508375512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 0x55ac9d29c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ed8e65700 1 -- 172.21.15.46:0/508375512 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac9d0633b0 con 0x55ac9d28f380 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.625+0000 7f3ed9e67700 1 --2- 172.21.15.46:0/508375512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 0x55ac9d29c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.626+0000 7f3ed8e65700 1 --2- 172.21.15.46:0/508375512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d299ce0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40ca540 tx=0x7f3ec4099b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b38f08261889a63 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.626+0000 7f3ebf7fe700 1 -- 172.21.15.46:0/508375512 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40a2380 con 0x55ac9d28f380 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.626+0000 7f3ebf7fe700 1 -- 172.21.15.46:0/508375512 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ec4010430 con 0x55ac9d28f380 2024-03-18T20:47:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.626+0000 7f3ebf7fe700 1 -- 172.21.15.46:0/508375512 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40993f0 con 0x55ac9d28f380 2024-03-18T20:47:31.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.627+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/508375512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 msgr2=0x55ac9d299ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.627+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/508375512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d299ce0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40ca540 tx=0x7f3ec4099b70 comp rx=0 tx=0).stop 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.627+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/508375512 shutdown_connections 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.627+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/508375512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 0x55ac9d29c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.627+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/508375512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28ea60 0x55ac9d28ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.627+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/508375512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d299ce0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.627+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/508375512 >> 172.21.15.46:0/508375512 conn(0x55ac9d205220 msgr2=0x55ac9d28b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.627+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/508375512 shutdown_connections 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.627+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/508375512 wait complete. 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.627+0000 7f3ee7a5fb80 1 -- start start 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.627+0000 7f3ee7a5fb80 1 Processor -- start 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ee7a5fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 0x55ac9d2970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ee7a5fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28f380 0x55ac9d297630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ee7a5fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d29a220 0x55ac9d297b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ee7a5fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9d17dd80 con 0x55ac9d28ea60 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ee7a5fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9d17da20 con 0x55ac9d28f380 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ee7a5fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9d0656a0 con 0x55ac9d29a220 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ee7a5fb80 10 client.0 _ll_register_callbacks cb 0x55ac9cfe93a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ed9666700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 0x55ac9d2970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ed9666700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 0x55ac9d2970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60802/0 (socket says 172.21.15.46:60802) 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ed9666700 1 -- 172.21.15.46:0/4262264902 learned_addr learned my addr 172.21.15.46:0/4262264902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ed9e67700 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d29a220 0x55ac9d297b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ed8e65700 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28f380 0x55ac9d297630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ed9666700 1 -- 172.21.15.46:0/4262264902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d29a220 msgr2=0x55ac9d297b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ed9666700 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d29a220 0x55ac9d297b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ed9666700 1 -- 172.21.15.46:0/4262264902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28f380 msgr2=0x55ac9d297630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ed9666700 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28f380 0x55ac9d297630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ed9666700 1 -- 172.21.15.46:0/4262264902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc011330 con 0x55ac9d28ea60 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ed9666700 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 0x55ac9d2970f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009230 tx=0x7f3ed0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a05977242370dc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.628+0000 7f3ebcff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ebcff9700 1 -- 172.21.15.46:0/4262264902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed002b040 con 0x55ac9d28ea60 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ebcff9700 1 -- 172.21.15.46:0/4262264902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3ed0005040 con 0x55ac9d28ea60 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4262264902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac9d282ad0 con 0x55ac9d28ea60 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ebcff9700 1 -- 172.21.15.46:0/4262264902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ed0005200 con 0x55ac9d28ea60 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ee7a5fb80 10 client.19554 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4262264902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f3ed4068360 con 0x55ac9d28ea60 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ebcff9700 1 -- 172.21.15.46:0/4262264902 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac9d282ad0 con 0x55ac9d28ea60 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ebcff9700 1 -- 172.21.15.46:0/4262264902 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3ed0038440 con 0x55ac9d28ea60 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ee7a5fb80 10 client.19554 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ee7a5fb80 10 client.19554 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ee7a5fb80 20 client.19554 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ac9d2a9e60 0x55ac9d2ac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ee7a5fb80 4 client.19554 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4262264902 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac9d1da700 con 0x55ac9d2a9e60 2024-03-18T20:47:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.629+0000 7f3ed8e65700 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ac9d2a9e60 0x55ac9d2ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.630+0000 7f3ed8e65700 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ac9d2a9e60 0x55ac9d2ac280 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51c0934e2d0cd900 server_cookie=e45c7fbcd81e1e5d in_seq=0 out_seq=0 2024-03-18T20:47:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.630+0000 7f3ebcff9700 10 client.19554 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:47:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.630+0000 7f3ebcff9700 1 -- 172.21.15.46:0/4262264902 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ac9d1da700 con 0x55ac9d2a9e60 2024-03-18T20:47:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.630+0000 7f3ebcff9700 10 client.19554 handle_command_reply: tid=0 2024-03-18T20:47:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.630+0000 7f3ee7a5fb80 1 client.19554 shutdown 2024-03-18T20:47:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.630+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4262264902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 msgr2=0x55ac9d2970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.630+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 0x55ac9d2970f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009230 tx=0x7f3ed0009c00 comp rx=0 tx=0).stop 2024-03-18T20:47:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.631+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4262264902 shutdown_connections 2024-03-18T20:47:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.631+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ac9d2a9e60 0x55ac9d2ac280 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.631+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 0x55ac9d2970f0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.631+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d29a220 0x55ac9d297b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.631+0000 7f3ebcff9700 0 client.19554 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:47:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.631+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/4262264902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28f380 0x55ac9d297630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.631+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4262264902 >> 172.21.15.46:0/4262264902 conn(0x55ac9d205220 msgr2=0x55ac9d28ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.631+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4262264902 shutdown_connections 2024-03-18T20:47:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.631+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4262264902 wait complete. 2024-03-18T20:47:31.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.631+0000 7f3ee7a5fb80 20 client.19554 trim_cache size 0 max 16384 2024-03-18T20:47:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ee7a5fb80 1 Processor -- start 2024-03-18T20:47:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ee7a5fb80 1 -- start start 2024-03-18T20:47:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ee7a5fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28ea60 0x55ac9d297990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ee7a5fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d2960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ee7a5fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 0x55ac9d294370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ee7a5fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9d2875e0 con 0x55ac9d29a220 2024-03-18T20:47:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ee7a5fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9d28c230 con 0x55ac9d28ea60 2024-03-18T20:47:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ee7a5fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9d289dd0 con 0x55ac9d28f380 2024-03-18T20:47:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ed9e67700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 0x55ac9d294370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ed9666700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28ea60 0x55ac9d297990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ed9e67700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 0x55ac9d294370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60808/0 (socket says 172.21.15.46:60808) 2024-03-18T20:47:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ed9666700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28ea60 0x55ac9d297990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54902/0 (socket says 172.21.15.46:54902) 2024-03-18T20:47:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ed9e67700 1 -- 172.21.15.46:0/2637151841 learned_addr learned my addr 172.21.15.46:0/2637151841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.645+0000 7f3ed8e65700 1 --2- 172.21.15.46:0/2637151841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d2960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.646+0000 7f3ed9e67700 1 -- 172.21.15.46:0/2637151841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 msgr2=0x55ac9d2960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.646+0000 7f3ed9e67700 1 --2- 172.21.15.46:0/2637151841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d2960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.646+0000 7f3ed9e67700 1 -- 172.21.15.46:0/2637151841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28ea60 msgr2=0x55ac9d297990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.646+0000 7f3ed9e67700 1 --2- 172.21.15.46:0/2637151841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28ea60 0x55ac9d297990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.646+0000 7f3ed9e67700 1 -- 172.21.15.46:0/2637151841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac9d00fb60 con 0x55ac9d29a220 2024-03-18T20:47:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.646+0000 7f3ed8e65700 1 --2- 172.21.15.46:0/2637151841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d2960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.646+0000 7f3ed9666700 1 --2- 172.21.15.46:0/2637151841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28ea60 0x55ac9d297990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.646+0000 7f3ed9e67700 1 --2- 172.21.15.46:0/2637151841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 0x55ac9d294370 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc011b60 tx=0x7f3ecc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b5085d7f9f9dca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.646+0000 7f3ebffff700 1 -- 172.21.15.46:0/2637151841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc00e040 con 0x55ac9d29a220 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.646+0000 7f3ebffff700 1 -- 172.21.15.46:0/2637151841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ecc00f040 con 0x55ac9d29a220 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.646+0000 7f3ebffff700 1 -- 172.21.15.46:0/2637151841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc007d90 con 0x55ac9d29a220 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.647+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/2637151841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 msgr2=0x55ac9d294370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.647+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/2637151841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 0x55ac9d294370 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc011b60 tx=0x7f3ecc009920 comp rx=0 tx=0).stop 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.647+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/2637151841 shutdown_connections 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.647+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/2637151841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d29a220 0x55ac9d294370 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.647+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/2637151841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d28ea60 0x55ac9d297990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.647+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/2637151841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d2960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.647+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/2637151841 >> 172.21.15.46:0/2637151841 conn(0x55ac9d205220 msgr2=0x55ac9d2959c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.647+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/2637151841 shutdown_connections 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.647+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/2637151841 wait complete. 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.647+0000 7f3ee7a5fb80 1 -- start start 2024-03-18T20:47:31.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ee7a5fb80 1 Processor -- start 2024-03-18T20:47:31.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ee7a5fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 0x55ac9d29ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ee7a5fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d29f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ee7a5fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d29a220 0x55ac9d2fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ee7a5fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9d2fd1d0 con 0x55ac9d28ea60 2024-03-18T20:47:31.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ee7a5fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9d2fd310 con 0x55ac9d29a220 2024-03-18T20:47:31.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ee7a5fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9d2fd450 con 0x55ac9d28f380 2024-03-18T20:47:31.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ee7a5fb80 10 client.0 _ll_register_callbacks cb 0x55ac9cfe93a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:31.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ed9666700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 0x55ac9d29ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ed8e65700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d29f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ed9e67700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d29a220 0x55ac9d2fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ed9666700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 0x55ac9d29ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60812/0 (socket says 172.21.15.46:60812) 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ed8e65700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d29f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43652/0 (socket says 172.21.15.46:43652) 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.648+0000 7f3ed9666700 1 -- 172.21.15.46:0/4240158306 learned_addr learned my addr 172.21.15.46:0/4240158306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ed9e67700 1 -- 172.21.15.46:0/4240158306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 msgr2=0x55ac9d29f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ed9e67700 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d29f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ed9e67700 1 -- 172.21.15.46:0/4240158306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 msgr2=0x55ac9d29ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ed9e67700 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 0x55ac9d29ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ed9e67700 1 -- 172.21.15.46:0/4240158306 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed4068360 con 0x55ac9d29a220 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ed8e65700 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d29f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ed9666700 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 0x55ac9d29ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ed9e67700 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d29a220 0x55ac9d2fcc90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc000f80 tx=0x7f3ecc004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a58fe514fdd4ae16 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ebcff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:47:31.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ebcff9700 1 -- 172.21.15.46:0/4240158306 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc00e030 con 0x55ac9d29a220 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ebcff9700 1 -- 172.21.15.46:0/4240158306 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3ecc007500 con 0x55ac9d29a220 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4240158306 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ec4010430 con 0x55ac9d29a220 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ebcff9700 1 -- 172.21.15.46:0/4240158306 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ecc007be0 con 0x55ac9d29a220 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ee7a5fb80 10 client.29177 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4240158306 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55ac9d0633b0 con 0x55ac9d29a220 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.649+0000 7f3ebcff9700 1 -- 172.21.15.46:0/4240158306 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ec4010430 con 0x55ac9d29a220 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ebcff9700 1 -- 172.21.15.46:0/4240158306 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3ecc020770 con 0x55ac9d29a220 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ee7a5fb80 10 client.29177 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ee7a5fb80 10 client.29177 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ee7a5fb80 20 client.29177 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ac9d2feb80 0x55ac9d300f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ee7a5fb80 4 client.29177 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4240158306 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ac9d030cf0 con 0x55ac9d2feb80 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ed9666700 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ac9d2feb80 0x55ac9d300f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ebcff9700 10 client.29177 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:47:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ed9666700 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ac9d2feb80 0x55ac9d300f70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a1422f9b7b91b34 server_cookie=5271a81f7de412c5 in_seq=0 out_seq=0 2024-03-18T20:47:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ebcff9700 1 -- 172.21.15.46:0/4240158306 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ac9d030cf0 con 0x55ac9d2feb80 2024-03-18T20:47:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ebcff9700 10 client.29177 handle_command_reply: tid=0 2024-03-18T20:47:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.650+0000 7f3ee7a5fb80 1 client.29177 shutdown 2024-03-18T20:47:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4240158306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d29a220 msgr2=0x55ac9d2fcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d29a220 0x55ac9d2fcc90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc000f80 tx=0x7f3ecc004f90 comp rx=0 tx=0).stop 2024-03-18T20:47:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4240158306 shutdown_connections 2024-03-18T20:47:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x55ac9d2feb80 0x55ac9d300f70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9d28ea60 0x55ac9d29ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ac9d29a220 0x55ac9d2fcc90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3ee7a5fb80 1 --2- 172.21.15.46:0/4240158306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ac9d28f380 0x55ac9d29f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3ebcff9700 0 client.29177 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:47:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4240158306 >> 172.21.15.46:0/4240158306 conn(0x55ac9d205220 msgr2=0x55ac9d2979d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4240158306 shutdown_connections 2024-03-18T20:47:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3ee7a5fb80 1 -- 172.21.15.46:0/4240158306 wait complete. 2024-03-18T20:47:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3ee7a5fb80 20 client.29177 trim_cache size 0 max 16384 2024-03-18T20:47:31.659 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:47:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3edb8ca700 1 -- 172.21.15.46:0/3990722271 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ec00a1290 msgr2=0x7f3ec00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3edb8ca700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ec00a1290 0x7f3ec00a3750 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009fd0 tx=0x7f3ed0019040 comp rx=0 tx=0).stop 2024-03-18T20:47:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3edb8ca700 1 -- 172.21.15.46:0/3990722271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ed4104f60 msgr2=0x7f3ed41965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3edb8ca700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ed4104f60 0x7f3ed41965f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e9f0 tx=0x7f3ec400ed00 comp rx=0 tx=0).stop 2024-03-18T20:47:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3edb8ca700 1 -- 172.21.15.46:0/3990722271 shutdown_connections 2024-03-18T20:47:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3edb8ca700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ec00a1290 0x7f3ec00a3750 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3edb8ca700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed40ffe50 0x7f3ed41960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3edb8ca700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed419a990 0x7f3ed419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3edb8ca700 1 --2- 172.21.15.46:0/3990722271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ed4104f60 0x7f3ed41965f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.651+0000 7f3edb8ca700 1 -- 172.21.15.46:0/3990722271 >> 172.21.15.46:0/3990722271 conn(0x7f3ed4076d40 msgr2=0x7f3ed40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.652+0000 7f3edb8ca700 1 -- 172.21.15.46:0/3990722271 shutdown_connections 2024-03-18T20:47:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.652+0000 7f3edb8ca700 1 -- 172.21.15.46:0/3990722271 wait complete. 2024-03-18T20:47:31.667 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:47:31.668 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:47:31.718 INFO:tasks.workunit.client.0.smithi046.stdout: 1 429668 3.70 MB/sec execute 506 sec latency 28.664 ms 2024-03-18T20:47:31.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:31 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2446432628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:47:31.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.813+0000 7f85439c3700 1 -- 172.21.15.46:0/2248342992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f853c100f50 msgr2=0x7f853c101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.813+0000 7f85439c3700 1 --2- 172.21.15.46:0/2248342992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f853c100f50 0x7f853c101330 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f852c009990 tx=0x7f852c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.813+0000 7f85439c3700 1 -- 172.21.15.46:0/2248342992 shutdown_connections 2024-03-18T20:47:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.813+0000 7f85439c3700 1 --2- 172.21.15.46:0/2248342992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f853c100f50 0x7f853c101330 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.813+0000 7f85439c3700 1 --2- 172.21.15.46:0/2248342992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f853c106070 0x7f853c10dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.813+0000 7f85439c3700 1 --2- 172.21.15.46:0/2248342992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f853c101900 0x7f853c105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.813+0000 7f85439c3700 1 -- 172.21.15.46:0/2248342992 >> 172.21.15.46:0/2248342992 conn(0x7f853c0fc7f0 msgr2=0x7f853c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.813+0000 7f85439c3700 1 -- 172.21.15.46:0/2248342992 shutdown_connections 2024-03-18T20:47:31.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.813+0000 7f85439c3700 1 -- 172.21.15.46:0/2248342992 wait complete. 2024-03-18T20:47:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.813+0000 7f85439c3700 1 Processor -- start 2024-03-18T20:47:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.813+0000 7f85439c3700 1 -- start start 2024-03-18T20:47:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f85439c3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f853c100f50 0x7f853c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f85439c3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f853c106070 0x7f853c197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f85439c3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f853c19bb10 0x7f853c19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f85439c3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c19c590 con 0x7f853c100f50 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f85439c3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c19c700 con 0x7f853c106070 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f85439c3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c19c8c0 con 0x7f853c19bb10 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f854175f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f853c100f50 0x7f853c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f854175f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f853c100f50 0x7f853c197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60836/0 (socket says 172.21.15.46:60836) 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f854175f700 1 -- 172.21.15.46:0/3453774442 learned_addr learned my addr 172.21.15.46:0/3453774442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f8541f60700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f853c19bb10 0x7f853c19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f8540f5e700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f853c106070 0x7f853c197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f854175f700 1 -- 172.21.15.46:0/3453774442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f853c19bb10 msgr2=0x7f853c19bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f854175f700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f853c19bb10 0x7f853c19bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f854175f700 1 -- 172.21.15.46:0/3453774442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f853c106070 msgr2=0x7f853c197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f854175f700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f853c106070 0x7f853c197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f854175f700 1 -- 172.21.15.46:0/3453774442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8534009c40 con 0x7f853c100f50 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.814+0000 7f854175f700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f853c100f50 0x7f853c197230 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f852c004760 tx=0x7f852c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.815+0000 7f853a7fc700 1 -- 172.21.15.46:0/3453774442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c01d070 con 0x7f853c100f50 2024-03-18T20:47:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.815+0000 7f853a7fc700 1 -- 172.21.15.46:0/3453774442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f852c022470 con 0x7f853c100f50 2024-03-18T20:47:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.815+0000 7f85439c3700 1 -- 172.21.15.46:0/3453774442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c0095f0 con 0x7f853c100f50 2024-03-18T20:47:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.815+0000 7f853a7fc700 1 -- 172.21.15.46:0/3453774442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c017650 con 0x7f853c100f50 2024-03-18T20:47:31.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.815+0000 7f85439c3700 1 -- 172.21.15.46:0/3453774442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c1a4050 con 0x7f853c100f50 2024-03-18T20:47:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.817+0000 7f853a7fc700 1 -- 172.21.15.46:0/3453774442 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f852c0225e0 con 0x7f853c100f50 2024-03-18T20:47:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.818+0000 7f853a7fc700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f85240a1290 0x7f85240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.818+0000 7f8540f5e700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f85240a1290 0x7f85240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.818+0000 7f853a7fc700 1 -- 172.21.15.46:0/3453774442 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f852c0cdde0 con 0x7f853c100f50 2024-03-18T20:47:31.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.819+0000 7f8540f5e700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f85240a1290 0x7f85240a3750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f853c198780 tx=0x7f8530009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.819+0000 7f854fb58b80 1 Processor -- start 2024-03-18T20:47:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.819+0000 7f854fb58b80 1 -- start start 2024-03-18T20:47:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854fb58b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde1e8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854fb58b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1e9380 0x557fde1f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854fb58b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1f4220 0x557fde1f6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854fb58b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fde142990 con 0x557fde1f4220 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854fb58b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fde140350 con 0x557fde1e8a60 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854fb58b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fde13bc40 con 0x557fde1e9380 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854175f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde1e8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f8541f60700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1f4220 0x557fde1f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854175f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde1e8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:54940/0 (socket says 172.21.15.46:54940) 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f8540f5e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1e9380 0x557fde1f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f8541f60700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1f4220 0x557fde1f6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60850/0 (socket says 172.21.15.46:60850) 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854175f700 1 -- 172.21.15.46:0/3523073337 learned_addr learned my addr 172.21.15.46:0/3523073337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854175f700 1 -- 172.21.15.46:0/3523073337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1e9380 msgr2=0x557fde1f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854175f700 1 --2- 172.21.15.46:0/3523073337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1e9380 0x557fde1f3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854175f700 1 -- 172.21.15.46:0/3523073337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1f4220 msgr2=0x557fde1f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854175f700 1 --2- 172.21.15.46:0/3523073337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1f4220 0x557fde1f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854175f700 1 -- 172.21.15.46:0/3523073337 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fddfbd3b0 con 0x557fde1e8a60 2024-03-18T20:47:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f8541f60700 1 --2- 172.21.15.46:0/3523073337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1f4220 0x557fde1f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f854175f700 1 --2- 172.21.15.46:0/3523073337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde1e8e40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f852c09aef0 tx=0x7f852c09ad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b7822006a564db server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.820+0000 7f85237fe700 1 -- 172.21.15.46:0/3523073337 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c0a1070 con 0x557fde1e8a60 2024-03-18T20:47:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.821+0000 7f85237fe700 1 -- 172.21.15.46:0/3523073337 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f852c09b660 con 0x557fde1e8a60 2024-03-18T20:47:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.821+0000 7f85237fe700 1 -- 172.21.15.46:0/3523073337 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c09b420 con 0x557fde1e8a60 2024-03-18T20:47:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.821+0000 7f854fb58b80 1 -- 172.21.15.46:0/3523073337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 msgr2=0x557fde1e8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.821+0000 7f854fb58b80 1 --2- 172.21.15.46:0/3523073337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde1e8e40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f852c09aef0 tx=0x7f852c09ad30 comp rx=0 tx=0).stop 2024-03-18T20:47:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.821+0000 7f854fb58b80 1 -- 172.21.15.46:0/3523073337 shutdown_connections 2024-03-18T20:47:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.821+0000 7f854fb58b80 1 --2- 172.21.15.46:0/3523073337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1f4220 0x557fde1f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.821+0000 7f854fb58b80 1 --2- 172.21.15.46:0/3523073337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde1e8e40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.821+0000 7f854fb58b80 1 --2- 172.21.15.46:0/3523073337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1e9380 0x557fde1f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.821+0000 7f854fb58b80 1 -- 172.21.15.46:0/3523073337 >> 172.21.15.46:0/3523073337 conn(0x557fde15f220 msgr2=0x557fde1e5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.821+0000 7f854fb58b80 1 -- 172.21.15.46:0/3523073337 shutdown_connections 2024-03-18T20:47:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.821+0000 7f854fb58b80 1 -- 172.21.15.46:0/3523073337 wait complete. 2024-03-18T20:47:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.822+0000 7f854fb58b80 1 -- start start 2024-03-18T20:47:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.822+0000 7f854fb58b80 1 Processor -- start 2024-03-18T20:47:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854fb58b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e8a60 0x557fde1f4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854fb58b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1e9380 0x557fde1f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854fb58b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1f4220 0x557fde1ff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854fb58b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fde0d7d80 con 0x557fde1e8a60 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854fb58b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fde0d7a20 con 0x557fde1f4220 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854fb58b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fddfbf6a0 con 0x557fde1e9380 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854fb58b80 10 client.0 _ll_register_callbacks cb 0x557fddf433a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854175f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e8a60 0x557fde1f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f8540f5e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1e9380 0x557fde1f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f8541f60700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1f4220 0x557fde1ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854175f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e8a60 0x557fde1f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60856/0 (socket says 172.21.15.46:60856) 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854175f700 1 -- 172.21.15.46:0/3026674698 learned_addr learned my addr 172.21.15.46:0/3026674698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854175f700 1 -- 172.21.15.46:0/3026674698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1e9380 msgr2=0x557fde1f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854175f700 1 --2- 172.21.15.46:0/3026674698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1e9380 0x557fde1f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854175f700 1 -- 172.21.15.46:0/3026674698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1f4220 msgr2=0x557fde1ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854175f700 1 --2- 172.21.15.46:0/3026674698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1f4220 0x557fde1ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854175f700 1 -- 172.21.15.46:0/3026674698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853400e330 con 0x557fde1e8a60 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f854175f700 1 --2- 172.21.15.46:0/3026674698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e8a60 0x557fde1f4600 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f852c0979d0 tx=0x7f852c022b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1dfbce76893c9a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f8520ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f8520ff9700 1 -- 172.21.15.46:0/3026674698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c09af60 con 0x557fde1e8a60 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.823+0000 7f8520ff9700 1 -- 172.21.15.46:0/3026674698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f852c09ed90 con 0x557fde1e8a60 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f854fb58b80 1 -- 172.21.15.46:0/3026674698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fde1dcad0 con 0x557fde1e8a60 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f8520ff9700 1 -- 172.21.15.46:0/3026674698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f852c0a3350 con 0x557fde1e8a60 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f854fb58b80 10 client.19572 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f854fb58b80 1 -- 172.21.15.46:0/3026674698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f853c1a4050 con 0x557fde1e8a60 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f8520ff9700 1 -- 172.21.15.46:0/3026674698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557fde1dcad0 con 0x557fde1e8a60 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f8520ff9700 1 -- 172.21.15.46:0/3026674698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f852c095b70 con 0x557fde1e8a60 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f854fb58b80 10 client.19572 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f854fb58b80 10 client.19572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f854fb58b80 20 client.19572 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:31.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f854fb58b80 1 --2- 172.21.15.46:0/3026674698 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557fde203760 0x557fde205b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f854fb58b80 4 client.19572 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f854fb58b80 1 -- 172.21.15.46:0/3026674698 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fde134700 con 0x557fde203760 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.824+0000 7f8540f5e700 1 --2- 172.21.15.46:0/3026674698 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557fde203760 0x557fde205b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.825+0000 7f8520ff9700 10 client.19572 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.825+0000 7f8540f5e700 1 --2- 172.21.15.46:0/3026674698 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557fde203760 0x557fde205b80 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f783cde57e79c4dd server_cookie=651cb7506c1cdebb in_seq=0 out_seq=0 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.825+0000 7f8520ff9700 1 -- 172.21.15.46:0/3026674698 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557fde134700 con 0x557fde203760 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.825+0000 7f8520ff9700 10 client.19572 handle_command_reply: tid=0 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.825+0000 7f854fb58b80 1 client.19572 shutdown 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.826+0000 7f854fb58b80 1 -- 172.21.15.46:0/3026674698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e8a60 msgr2=0x557fde1f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.826+0000 7f854fb58b80 1 --2- 172.21.15.46:0/3026674698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e8a60 0x557fde1f4600 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f852c0979d0 tx=0x7f852c022b10 comp rx=0 tx=0).stop 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.826+0000 7f854fb58b80 1 -- 172.21.15.46:0/3026674698 shutdown_connections 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.826+0000 7f854fb58b80 1 --2- 172.21.15.46:0/3026674698 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557fde203760 0x557fde205b80 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.826+0000 7f854fb58b80 1 --2- 172.21.15.46:0/3026674698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e8a60 0x557fde1f4600 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.826+0000 7f854fb58b80 1 --2- 172.21.15.46:0/3026674698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1f4220 0x557fde1ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.826+0000 7f8520ff9700 0 client.19572 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.826+0000 7f854fb58b80 1 --2- 172.21.15.46:0/3026674698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1e9380 0x557fde1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.826+0000 7f854fb58b80 1 -- 172.21.15.46:0/3026674698 >> 172.21.15.46:0/3026674698 conn(0x557fde15f220 msgr2=0x557fde1e4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.826+0000 7f854fb58b80 1 -- 172.21.15.46:0/3026674698 shutdown_connections 2024-03-18T20:47:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.826+0000 7f854fb58b80 1 -- 172.21.15.46:0/3026674698 wait complete. 2024-03-18T20:47:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.826+0000 7f854fb58b80 20 client.19572 trim_cache size 0 max 16384 2024-03-18T20:47:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f854fb58b80 1 Processor -- start 2024-03-18T20:47:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f854fb58b80 1 -- start start 2024-03-18T20:47:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f854fb58b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde1f9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f854fb58b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 0x557fde1edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f854fb58b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1f4220 0x557fde1f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f854fb58b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fde1e9d30 con 0x557fde1e9380 2024-03-18T20:47:31.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f854fb58b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fde1e15e0 con 0x557fde1e8a60 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f854fb58b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fde1e6230 con 0x557fde1f4220 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f8540f5e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 0x557fde1edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f8541f60700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1f4220 0x557fde1f07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f8540f5e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 0x557fde1edc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60864/0 (socket says 172.21.15.46:60864) 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f8540f5e700 1 -- 172.21.15.46:0/1730424433 learned_addr learned my addr 172.21.15.46:0/1730424433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.847+0000 7f854175f700 1 --2- 172.21.15.46:0/1730424433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde1f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.848+0000 7f8541f60700 1 -- 172.21.15.46:0/1730424433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 msgr2=0x557fde1f9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.848+0000 7f8541f60700 1 --2- 172.21.15.46:0/1730424433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde1f9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.848+0000 7f8541f60700 1 -- 172.21.15.46:0/1730424433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 msgr2=0x557fde1edc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.848+0000 7f8541f60700 1 --2- 172.21.15.46:0/1730424433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 0x557fde1edc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.848+0000 7f8541f60700 1 -- 172.21.15.46:0/1730424433 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fddf69b60 con 0x557fde1f4220 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.848+0000 7f8540f5e700 1 --2- 172.21.15.46:0/1730424433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 0x557fde1edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.848+0000 7f854175f700 1 --2- 172.21.15.46:0/1730424433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde1f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.848+0000 7f8541f60700 1 --2- 172.21.15.46:0/1730424433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1f4220 0x557fde1f07a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f853400ea40 tx=0x7f8534009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=280570725a1d834d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.848+0000 7f8523fff700 1 -- 172.21.15.46:0/1730424433 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853400e040 con 0x557fde1f4220 2024-03-18T20:47:31.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.848+0000 7f8523fff700 1 -- 172.21.15.46:0/1730424433 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f853400f040 con 0x557fde1f4220 2024-03-18T20:47:31.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.848+0000 7f8523fff700 1 -- 172.21.15.46:0/1730424433 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8534017d00 con 0x557fde1f4220 2024-03-18T20:47:31.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.849+0000 7f854fb58b80 1 -- 172.21.15.46:0/1730424433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1f4220 msgr2=0x557fde1f07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.849+0000 7f854fb58b80 1 --2- 172.21.15.46:0/1730424433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1f4220 0x557fde1f07a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f853400ea40 tx=0x7f8534009920 comp rx=0 tx=0).stop 2024-03-18T20:47:31.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.849+0000 7f854fb58b80 1 -- 172.21.15.46:0/1730424433 shutdown_connections 2024-03-18T20:47:31.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.849+0000 7f854fb58b80 1 --2- 172.21.15.46:0/1730424433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1f4220 0x557fde1f07a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.849+0000 7f854fb58b80 1 --2- 172.21.15.46:0/1730424433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde1f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.849+0000 7f854fb58b80 1 --2- 172.21.15.46:0/1730424433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 0x557fde1edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.849+0000 7f854fb58b80 1 -- 172.21.15.46:0/1730424433 >> 172.21.15.46:0/1730424433 conn(0x557fde15f220 msgr2=0x557fde1f2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.849+0000 7f854fb58b80 1 -- 172.21.15.46:0/1730424433 shutdown_connections 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.849+0000 7f854fb58b80 1 -- 172.21.15.46:0/1730424433 wait complete. 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.850+0000 7f854fb58b80 1 -- start start 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.850+0000 7f854fb58b80 1 Processor -- start 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.850+0000 7f854fb58b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde24b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.850+0000 7f854fb58b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 0x557fde24bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.850+0000 7f854fb58b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1f4220 0x557fde256560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.850+0000 7f854fb58b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fde256aa0 con 0x557fde1e9380 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.850+0000 7f854fb58b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fde256be0 con 0x557fde1e8a60 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.850+0000 7f854fb58b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fde256d20 con 0x557fde1f4220 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.850+0000 7f854fb58b80 10 client.0 _ll_register_callbacks cb 0x7f853400e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f8540f5e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 0x557fde24bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f8540f5e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 0x557fde24bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60878/0 (socket says 172.21.15.46:60878) 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f8540f5e700 1 -- 172.21.15.46:0/2070832916 learned_addr learned my addr 172.21.15.46:0/2070832916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f854175f700 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde24b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f8541f60700 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1f4220 0x557fde256560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f854175f700 1 -- 172.21.15.46:0/2070832916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1f4220 msgr2=0x557fde256560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f854175f700 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1f4220 0x557fde256560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f854175f700 1 -- 172.21.15.46:0/2070832916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 msgr2=0x557fde24bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f854175f700 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 0x557fde24bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f854175f700 1 -- 172.21.15.46:0/2070832916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c1a4050 con 0x557fde1e8a60 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f8540f5e700 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 0x557fde24bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f854175f700 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde24b6d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f852c09ac60 tx=0x7f852c098ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fabf44de5b8b9728 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f8520ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f8520ff9700 1 -- 172.21.15.46:0/2070832916 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c0a3350 con 0x557fde1e8a60 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f8520ff9700 1 -- 172.21.15.46:0/2070832916 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f852c09ed90 con 0x557fde1e8a60 2024-03-18T20:47:31.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.851+0000 7f854fb58b80 1 -- 172.21.15.46:0/2070832916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f852c09b660 con 0x557fde1e8a60 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.852+0000 7f8520ff9700 1 -- 172.21.15.46:0/2070832916 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f852c0a10b0 con 0x557fde1e8a60 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.852+0000 7f854fb58b80 10 client.29195 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.852+0000 7f854fb58b80 1 -- 172.21.15.46:0/2070832916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f8530017070 con 0x557fde1e8a60 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.852+0000 7f8520ff9700 1 -- 172.21.15.46:0/2070832916 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f852c09b660 con 0x557fde1e8a60 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.852+0000 7f8520ff9700 1 -- 172.21.15.46:0/2070832916 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f852c099ec0 con 0x557fde1e8a60 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.852+0000 7f854fb58b80 10 client.29195 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.852+0000 7f854fb58b80 10 client.29195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.852+0000 7f854fb58b80 20 client.29195 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.852+0000 7f854fb58b80 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557fde258040 0x557fde25a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.852+0000 7f854fb58b80 4 client.29195 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.852+0000 7f854fb58b80 1 -- 172.21.15.46:0/2070832916 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557fddf8acf0 con 0x557fde258040 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f8540f5e700 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557fde258040 0x557fde25a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f8520ff9700 10 client.29195 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f8540f5e700 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557fde258040 0x557fde25a460 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80d77877813ec2 server_cookie=f93eca6a2dbed416 in_seq=0 out_seq=0 2024-03-18T20:47:31.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f8520ff9700 1 -- 172.21.15.46:0/2070832916 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557fddf8acf0 con 0x557fde258040 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f8520ff9700 10 client.29195 handle_command_reply: tid=0 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f854fb58b80 1 client.29195 shutdown 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f854fb58b80 1 -- 172.21.15.46:0/2070832916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 msgr2=0x557fde24b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f854fb58b80 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde24b6d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f852c09ac60 tx=0x7f852c098ff0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f854fb58b80 1 -- 172.21.15.46:0/2070832916 shutdown_connections 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f854fb58b80 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557fde258040 0x557fde25a460 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f854fb58b80 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557fde1f4220 0x557fde256560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f854fb58b80 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557fde1e8a60 0x557fde24b6d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f8520ff9700 0 client.29195 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f854fb58b80 1 --2- 172.21.15.46:0/2070832916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fde1e9380 0x557fde24bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.853+0000 7f854fb58b80 1 -- 172.21.15.46:0/2070832916 >> 172.21.15.46:0/2070832916 conn(0x557fde15f220 msgr2=0x557fde1edc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f854fb58b80 1 -- 172.21.15.46:0/2070832916 shutdown_connections 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f854fb58b80 1 -- 172.21.15.46:0/2070832916 wait complete. 2024-03-18T20:47:31.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f854fb58b80 20 client.29195 trim_cache size 0 max 16384 2024-03-18T20:47:31.863 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:47:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f85439c3700 1 -- 172.21.15.46:0/3453774442 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f85240a1290 msgr2=0x7f85240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f85439c3700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f85240a1290 0x7f85240a3750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f853c198780 tx=0x7f8530009260 comp rx=0 tx=0).stop 2024-03-18T20:47:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f85439c3700 1 -- 172.21.15.46:0/3453774442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f853c100f50 msgr2=0x7f853c197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f85439c3700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f853c100f50 0x7f853c197230 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f852c004760 tx=0x7f852c004790 comp rx=0 tx=0).stop 2024-03-18T20:47:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f85439c3700 1 -- 172.21.15.46:0/3453774442 shutdown_connections 2024-03-18T20:47:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f85439c3700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f85240a1290 0x7f85240a3750 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f85439c3700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f853c100f50 0x7f853c197230 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f85439c3700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f853c19bb10 0x7f853c19bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f85439c3700 1 --2- 172.21.15.46:0/3453774442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f853c106070 0x7f853c197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.854+0000 7f85439c3700 1 -- 172.21.15.46:0/3453774442 >> 172.21.15.46:0/3453774442 conn(0x7f853c0fc7f0 msgr2=0x7f853c0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.855+0000 7f85439c3700 1 -- 172.21.15.46:0/3453774442 shutdown_connections 2024-03-18T20:47:31.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:31.855+0000 7f85439c3700 1 -- 172.21.15.46:0/3453774442 wait complete. 2024-03-18T20:47:32.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:32 smithi155 ceph-mon[48344]: pgmap v356: 97 pgs: 97 active+clean; 166 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-03-18T20:47:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:32 smithi046 ceph-mon[43495]: pgmap v356: 97 pgs: 97 active+clean; 166 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-03-18T20:47:32.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 430914 3.71 MB/sec execute 507 sec latency 217.098 ms 2024-03-18T20:47:32.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:32 smithi073 ceph-mon[47384]: pgmap v356: 97 pgs: 97 active+clean; 166 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-03-18T20:47:32.873 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:47:32.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:47:32.874 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:47:33.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.022+0000 7f53ec060700 1 -- 172.21.15.46:0/4024940542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4073b00 msgr2=0x7f53e410d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.022+0000 7f53ec060700 1 --2- 172.21.15.46:0/4024940542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4073b00 0x7f53e410d000 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009990 tx=0x7f53d4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.022+0000 7f53ec060700 1 -- 172.21.15.46:0/4024940542 shutdown_connections 2024-03-18T20:47:33.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.022+0000 7f53ec060700 1 --2- 172.21.15.46:0/4024940542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e410d540 0x7f53e410f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.022+0000 7f53ec060700 1 --2- 172.21.15.46:0/4024940542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e40731e0 0x7f53e40735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.022+0000 7f53ec060700 1 --2- 172.21.15.46:0/4024940542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e4073b00 0x7f53e410d000 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.022+0000 7f53ec060700 1 -- 172.21.15.46:0/4024940542 >> 172.21.15.46:0/4024940542 conn(0x7f53e40fc7f0 msgr2=0x7f53e40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.022+0000 7f53ec060700 1 -- 172.21.15.46:0/4024940542 shutdown_connections 2024-03-18T20:47:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.022+0000 7f53ec060700 1 -- 172.21.15.46:0/4024940542 wait complete. 2024-03-18T20:47:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.023+0000 7f53ec060700 1 Processor -- start 2024-03-18T20:47:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.023+0000 7f53ec060700 1 -- start start 2024-03-18T20:47:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.023+0000 7f53ec060700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e40731e0 0x7f53e41993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53ec060700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e4073b00 0x7f53e4199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53ec060700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e410d540 0x7f53e419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53ec060700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e419e2e0 con 0x7f53e40731e0 2024-03-18T20:47:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53ec060700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e419e450 con 0x7f53e410d540 2024-03-18T20:47:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53ec060700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e419e610 con 0x7f53e4073b00 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53e9dfc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e40731e0 0x7f53e41993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53e9dfc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e40731e0 0x7f53e41993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60888/0 (socket says 172.21.15.46:60888) 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53e9dfc700 1 -- 172.21.15.46:0/1116970112 learned_addr learned my addr 172.21.15.46:0/1116970112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53e95fb700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e4073b00 0x7f53e4199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53ea5fd700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e410d540 0x7f53e419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53e9dfc700 1 -- 172.21.15.46:0/1116970112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e4073b00 msgr2=0x7f53e4199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e4073b00 0x7f53e4199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53e9dfc700 1 -- 172.21.15.46:0/1116970112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e410d540 msgr2=0x7f53e419dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e410d540 0x7f53e419dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53e9dfc700 1 -- 172.21.15.46:0/1116970112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d40095f0 con 0x7f53e40731e0 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53ea5fd700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e410d540 0x7f53e419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53e95fb700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e4073b00 0x7f53e4199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e40731e0 0x7f53e41993c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f53e000e9a0 tx=0x7f53e000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53daffd700 1 -- 172.21.15.46:0/1116970112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0009790 con 0x7f53e40731e0 2024-03-18T20:47:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53daffd700 1 -- 172.21.15.46:0/1116970112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e000cbd0 con 0x7f53e40731e0 2024-03-18T20:47:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53daffd700 1 -- 172.21.15.46:0/1116970112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0018680 con 0x7f53e40731e0 2024-03-18T20:47:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.024+0000 7f53ec060700 1 -- 172.21.15.46:0/1116970112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e419e8f0 con 0x7f53e40731e0 2024-03-18T20:47:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.025+0000 7f53ec060700 1 -- 172.21.15.46:0/1116970112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e41a6130 con 0x7f53e40731e0 2024-03-18T20:47:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.027+0000 7f53daffd700 1 -- 172.21.15.46:0/1116970112 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f53e000cd40 con 0x7f53e40731e0 2024-03-18T20:47:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.027+0000 7f53daffd700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f53d00a1310 0x7f53d00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.027+0000 7f53daffd700 1 -- 172.21.15.46:0/1116970112 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f53e0014070 con 0x7f53e40731e0 2024-03-18T20:47:33.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.027+0000 7f53e95fb700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f53d00a1310 0x7f53d00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.028+0000 7f53e95fb700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f53d00a1310 0x7f53d00a37d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f53d400b2e0 tx=0x7f53d4005fd0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:33.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.028+0000 7f53f81f5b80 1 Processor -- start 2024-03-18T20:47:33.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53f81f5b80 1 -- start start 2024-03-18T20:47:33.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53f81f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 0x5596e27feec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53f81f5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e27ff400 0x5596e2809d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53f81f5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e280a2a0 0x5596e280c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53f81f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e2758a10 con 0x5596e27feae0 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53f81f5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e27563d0 con 0x5596e27ff400 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53f81f5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e2751cc0 con 0x5596e280a2a0 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53e9dfc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 0x5596e27feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53e9dfc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 0x5596e27feec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60890/0 (socket says 172.21.15.46:60890) 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53e9dfc700 1 -- 172.21.15.46:0/1766866982 learned_addr learned my addr 172.21.15.46:0/1766866982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53ea5fd700 1 --2- 172.21.15.46:0/1766866982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e280a2a0 0x5596e280c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53e95fb700 1 --2- 172.21.15.46:0/1766866982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e27ff400 0x5596e2809d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53e9dfc700 1 -- 172.21.15.46:0/1766866982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e280a2a0 msgr2=0x5596e280c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/1766866982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e280a2a0 0x5596e280c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53e9dfc700 1 -- 172.21.15.46:0/1766866982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e27ff400 msgr2=0x5596e2809d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/1766866982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e27ff400 0x5596e2809d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.029+0000 7f53e9dfc700 1 -- 172.21.15.46:0/1766866982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596e25d3430 con 0x5596e27feae0 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.030+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/1766866982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 0x5596e27feec0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f53e00a2030 tx=0x7f53e009a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3496f7917453bdcd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.030+0000 7f53cffff700 1 -- 172.21.15.46:0/1766866982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00a2560 con 0x5596e27feae0 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.030+0000 7f53cffff700 1 -- 172.21.15.46:0/1766866982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e009a340 con 0x5596e27feae0 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.030+0000 7f53cffff700 1 -- 172.21.15.46:0/1766866982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00a22f0 con 0x5596e27feae0 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.031+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1766866982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 msgr2=0x5596e27feec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.031+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/1766866982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 0x5596e27feec0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f53e00a2030 tx=0x7f53e009a8d0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.031+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1766866982 shutdown_connections 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.031+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/1766866982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 0x5596e27feec0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.031+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/1766866982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e280a2a0 0x5596e280c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.031+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/1766866982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e27ff400 0x5596e2809d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.031+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1766866982 >> 172.21.15.46:0/1766866982 conn(0x5596e27752a0 msgr2=0x5596e27fb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.031+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1766866982 shutdown_connections 2024-03-18T20:47:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.031+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1766866982 wait complete. 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.031+0000 7f53f81f5b80 1 -- start start 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.031+0000 7f53f81f5b80 1 Processor -- start 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53f81f5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27feae0 0x5596e2807130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53f81f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27ff400 0x5596e2807670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53f81f5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e280a2a0 0x5596e2807bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53f81f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e26ede00 con 0x5596e27ff400 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53f81f5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e26edaa0 con 0x5596e280a2a0 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53f81f5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e25d5720 con 0x5596e27feae0 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53f81f5b80 10 client.0 _ll_register_callbacks cb 0x5596e2559420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53e95fb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27ff400 0x5596e2807670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53e95fb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27ff400 0x5596e2807670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60892/0 (socket says 172.21.15.46:60892) 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53e95fb700 1 -- 172.21.15.46:0/1977056603 learned_addr learned my addr 172.21.15.46:0/1977056603 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53ea5fd700 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e280a2a0 0x5596e2807bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27feae0 0x5596e2807130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53e95fb700 1 -- 172.21.15.46:0/1977056603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27feae0 msgr2=0x5596e2807130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53e95fb700 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27feae0 0x5596e2807130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53e95fb700 1 -- 172.21.15.46:0/1977056603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e280a2a0 msgr2=0x5596e2807bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53e95fb700 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e280a2a0 0x5596e2807bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.032+0000 7f53e95fb700 1 -- 172.21.15.46:0/1977056603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e41a6130 con 0x5596e27ff400 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53e95fb700 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27ff400 0x5596e2807670 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f53d400f6e0 tx=0x7f53d401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea15113e7211a0a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53cd7fa700 1 -- 172.21.15.46:0/1977056603 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d402b070 con 0x5596e27ff400 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53cd7fa700 1 -- 172.21.15.46:0/1977056603 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f53d4005050 con 0x5596e27ff400 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1977056603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596e27f2b50 con 0x5596e27ff400 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53cd7fa700 1 -- 172.21.15.46:0/1977056603 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53d4005250 con 0x5596e27ff400 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53f81f5b80 10 client.19590 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1977056603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5596e257fbe0 con 0x5596e27ff400 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53cd7fa700 1 -- 172.21.15.46:0/1977056603 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53d403e020 con 0x5596e27ff400 2024-03-18T20:47:33.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53cd7fa700 1 -- 172.21.15.46:0/1977056603 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f53d40314f0 con 0x5596e27ff400 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53f81f5b80 10 client.19590 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53f81f5b80 10 client.19590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53f81f5b80 20 client.19590 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596e2819f60 0x5596e281c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53f81f5b80 4 client.19590 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.033+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1977056603 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596e274a780 con 0x5596e2819f60 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.034+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596e2819f60 0x5596e281c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.034+0000 7f53cd7fa700 10 client.19590 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.034+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596e2819f60 0x5596e281c380 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b70292b937fb4f server_cookie=38d307c969fd1a55 in_seq=0 out_seq=0 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.034+0000 7f53cd7fa700 1 -- 172.21.15.46:0/1977056603 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5596e274a780 con 0x5596e2819f60 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.034+0000 7f53cd7fa700 10 client.19590 handle_command_reply: tid=0 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53f81f5b80 1 client.19590 shutdown 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1977056603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27ff400 msgr2=0x5596e2807670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27ff400 0x5596e2807670 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f53d400f6e0 tx=0x7f53d401a050 comp rx=0 tx=0).stop 2024-03-18T20:47:33.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1977056603 shutdown_connections 2024-03-18T20:47:33.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596e2819f60 0x5596e281c380 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e280a2a0 0x5596e2807bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27feae0 0x5596e2807130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53cd7fa700 0 client.19590 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:33.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/1977056603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27ff400 0x5596e2807670 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1977056603 >> 172.21.15.46:0/1977056603 conn(0x5596e27752a0 msgr2=0x5596e27fac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:33.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1977056603 shutdown_connections 2024-03-18T20:47:33.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53f81f5b80 1 -- 172.21.15.46:0/1977056603 wait complete. 2024-03-18T20:47:33.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.035+0000 7f53f81f5b80 20 client.19590 trim_cache size 0 max 16384 2024-03-18T20:47:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.053+0000 7f53f81f5b80 1 Processor -- start 2024-03-18T20:47:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.053+0000 7f53f81f5b80 1 -- start start 2024-03-18T20:47:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.053+0000 7f53f81f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 0x5596e2810ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.053+0000 7f53f81f5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27ff400 0x5596e28079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.053+0000 7f53f81f5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e280a2a0 0x5596e2805a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.053+0000 7f53f81f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e280a040 con 0x5596e27feae0 2024-03-18T20:47:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.053+0000 7f53f81f5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e2808690 con 0x5596e280a2a0 2024-03-18T20:47:33.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.053+0000 7f53f81f5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e28175c0 con 0x5596e27ff400 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.053+0000 7f53e9dfc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 0x5596e2810ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.053+0000 7f53e9dfc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 0x5596e2810ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60902/0 (socket says 172.21.15.46:60902) 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53e9dfc700 1 -- 172.21.15.46:0/3709473445 learned_addr learned my addr 172.21.15.46:0/3709473445 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53e95fb700 1 --2- 172.21.15.46:0/3709473445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27ff400 0x5596e28079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53ea5fd700 1 --2- 172.21.15.46:0/3709473445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e280a2a0 0x5596e2805a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53e9dfc700 1 -- 172.21.15.46:0/3709473445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27ff400 msgr2=0x5596e28079d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/3709473445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27ff400 0x5596e28079d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53e9dfc700 1 -- 172.21.15.46:0/3709473445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e280a2a0 msgr2=0x5596e2805a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/3709473445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e280a2a0 0x5596e2805a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53e9dfc700 1 -- 172.21.15.46:0/3709473445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596e2717f20 con 0x5596e27feae0 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53e95fb700 1 --2- 172.21.15.46:0/3709473445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27ff400 0x5596e28079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53ea5fd700 1 --2- 172.21.15.46:0/3709473445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e280a2a0 0x5596e2805a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:47:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/3709473445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 0x5596e2810ed0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f53e00ca7d0 tx=0x7f53e0099f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236d6c54bc113590 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53d8ff9700 1 -- 172.21.15.46:0/3709473445 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00a25e0 con 0x5596e27feae0 2024-03-18T20:47:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53d8ff9700 1 -- 172.21.15.46:0/3709473445 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e0082070 con 0x5596e27feae0 2024-03-18T20:47:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.054+0000 7f53d8ff9700 1 -- 172.21.15.46:0/3709473445 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00a2330 con 0x5596e27feae0 2024-03-18T20:47:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.055+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3709473445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 msgr2=0x5596e2810ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.055+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/3709473445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 0x5596e2810ed0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f53e00ca7d0 tx=0x7f53e0099f30 comp rx=0 tx=0).stop 2024-03-18T20:47:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.055+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3709473445 shutdown_connections 2024-03-18T20:47:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.055+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/3709473445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e27feae0 0x5596e2810ed0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.055+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/3709473445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e280a2a0 0x5596e2805a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.055+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/3709473445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27ff400 0x5596e28079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.055+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3709473445 >> 172.21.15.46:0/3709473445 conn(0x5596e27752a0 msgr2=0x5596e27fabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.055+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3709473445 shutdown_connections 2024-03-18T20:47:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.055+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3709473445 wait complete. 2024-03-18T20:47:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.055+0000 7f53f81f5b80 1 -- start start 2024-03-18T20:47:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.055+0000 7f53f81f5b80 1 Processor -- start 2024-03-18T20:47:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53f81f5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27feae0 0x5596e280fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53f81f5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e27ff400 0x5596e280ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53f81f5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e280a2a0 0x5596e286dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53f81f5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e28108b0 con 0x5596e280a2a0 2024-03-18T20:47:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53f81f5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e286e2e0 con 0x5596e27ff400 2024-03-18T20:47:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53f81f5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596e286e420 con 0x5596e27feae0 2024-03-18T20:47:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53f81f5b80 10 client.0 _ll_register_callbacks cb 0x7f53d400bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53ea5fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e280a2a0 0x5596e286dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53ea5fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e280a2a0 0x5596e286dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60904/0 (socket says 172.21.15.46:60904) 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53ea5fd700 1 -- 172.21.15.46:0/3039512488 learned_addr learned my addr 172.21.15.46:0/3039512488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27feae0 0x5596e280fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53e95fb700 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e27ff400 0x5596e280ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53ea5fd700 1 -- 172.21.15.46:0/3039512488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27feae0 msgr2=0x5596e280fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53ea5fd700 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27feae0 0x5596e280fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53ea5fd700 1 -- 172.21.15.46:0/3039512488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e27ff400 msgr2=0x5596e280ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53ea5fd700 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e27ff400 0x5596e280ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.056+0000 7f53ea5fd700 1 -- 172.21.15.46:0/3039512488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596e25d3430 con 0x5596e280a2a0 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53ea5fd700 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e280a2a0 0x5596e286dda0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d50 tx=0x7f53dc007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bc8ca6387bf0c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53cd7fa700 1 -- 172.21.15.46:0/3039512488 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00e040 con 0x5596e280a2a0 2024-03-18T20:47:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53cd7fa700 1 -- 172.21.15.46:0/3039512488 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f53dc0179a0 con 0x5596e280a2a0 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3039512488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53e009a340 con 0x5596e280a2a0 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53cd7fa700 1 -- 172.21.15.46:0/3039512488 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53dc009390 con 0x5596e280a2a0 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53f81f5b80 10 client.19596 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3039512488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5596e25d7d50 con 0x5596e280a2a0 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53cd7fa700 1 -- 172.21.15.46:0/3039512488 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53dc02d080 con 0x5596e280a2a0 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53cd7fa700 1 -- 172.21.15.46:0/3039512488 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f53dc029420 con 0x5596e280a2a0 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53f81f5b80 10 client.19596 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53f81f5b80 10 client.19596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53f81f5b80 20 client.19596 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596e286fd20 0x5596e2872140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53f81f5b80 4 client.19596 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.057+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3039512488 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5596e25a0d70 con 0x5596e286fd20 2024-03-18T20:47:33.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.058+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596e286fd20 0x5596e2872140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.058+0000 7f53e9dfc700 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596e286fd20 0x5596e2872140 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23fe0bd1e77c4746 server_cookie=20cef8eee536a564 in_seq=0 out_seq=0 2024-03-18T20:47:33.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.058+0000 7f53cd7fa700 10 client.19596 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53cd7fa700 1 -- 172.21.15.46:0/3039512488 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5596e25a0d70 con 0x5596e286fd20 2024-03-18T20:47:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53cd7fa700 10 client.19596 handle_command_reply: tid=0 2024-03-18T20:47:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53f81f5b80 1 client.19596 shutdown 2024-03-18T20:47:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3039512488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e280a2a0 msgr2=0x5596e286dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e280a2a0 0x5596e286dda0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d50 tx=0x7f53dc007f10 comp rx=0 tx=0).stop 2024-03-18T20:47:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3039512488 shutdown_connections 2024-03-18T20:47:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5596e286fd20 0x5596e2872140 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596e280a2a0 0x5596e286dda0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5596e27feae0 0x5596e280fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53cd7fa700 0 client.19596 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53f81f5b80 1 --2- 172.21.15.46:0/3039512488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596e27ff400 0x5596e280ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3039512488 >> 172.21.15.46:0/3039512488 conn(0x5596e27752a0 msgr2=0x5596e28058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3039512488 shutdown_connections 2024-03-18T20:47:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53f81f5b80 1 -- 172.21.15.46:0/3039512488 wait complete. 2024-03-18T20:47:33.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.059+0000 7f53f81f5b80 20 client.19596 trim_cache size 0 max 16384 2024-03-18T20:47:33.069 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:47:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:47:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "26fb7a65-8c29-4b93-af76-eba0604f5736", 2024-03-18T20:47:33.069 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:47:33.069 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:47:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.060+0000 7f53ec060700 1 -- 172.21.15.46:0/1116970112 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f53d00a1310 msgr2=0x7f53d00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.060+0000 7f53ec060700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f53d00a1310 0x7f53d00a37d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f53d400b2e0 tx=0x7f53d4005fd0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.060+0000 7f53ec060700 1 -- 172.21.15.46:0/1116970112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e40731e0 msgr2=0x7f53e41993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.060+0000 7f53ec060700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e40731e0 0x7f53e41993c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f53e000e9a0 tx=0x7f53e000ed60 comp rx=0 tx=0).stop 2024-03-18T20:47:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.060+0000 7f53ec060700 1 -- 172.21.15.46:0/1116970112 shutdown_connections 2024-03-18T20:47:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.060+0000 7f53ec060700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f53d00a1310 0x7f53d00a37d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.060+0000 7f53ec060700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e40731e0 0x7f53e41993c0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.060+0000 7f53ec060700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e410d540 0x7f53e419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.060+0000 7f53ec060700 1 --2- 172.21.15.46:0/1116970112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53e4073b00 0x7f53e4199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.060+0000 7f53ec060700 1 -- 172.21.15.46:0/1116970112 >> 172.21.15.46:0/1116970112 conn(0x7f53e40fc7f0 msgr2=0x7f53e40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.060+0000 7f53ec060700 1 -- 172.21.15.46:0/1116970112 shutdown_connections 2024-03-18T20:47:33.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.060+0000 7f53ec060700 1 -- 172.21.15.46:0/1116970112 wait complete. 2024-03-18T20:47:33.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:47:33.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:47:33.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.226+0000 7f39155d7700 1 -- 172.21.15.46:0/2139590562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39100687f0 msgr2=0x7f391010db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.226+0000 7f39155d7700 1 --2- 172.21.15.46:0/2139590562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39100687f0 0x7f391010db50 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f390800b0a0 tx=0x7f390800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.226+0000 7f39155d7700 1 -- 172.21.15.46:0/2139590562 shutdown_connections 2024-03-18T20:47:33.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.226+0000 7f39155d7700 1 --2- 172.21.15.46:0/2139590562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39100687f0 0x7f391010db50 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.226+0000 7f39155d7700 1 --2- 172.21.15.46:0/2139590562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3910105ce0 0x7f39101060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.226+0000 7f39155d7700 1 --2- 172.21.15.46:0/2139590562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3910067e30 0x7f39100682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.226+0000 7f39155d7700 1 -- 172.21.15.46:0/2139590562 >> 172.21.15.46:0/2139590562 conn(0x7f3910076d40 msgr2=0x7f3910077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:33.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.227+0000 7f39155d7700 1 -- 172.21.15.46:0/2139590562 shutdown_connections 2024-03-18T20:47:33.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.227+0000 7f39155d7700 1 -- 172.21.15.46:0/2139590562 wait complete. 2024-03-18T20:47:33.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.227+0000 7f39155d7700 1 Processor -- start 2024-03-18T20:47:33.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.227+0000 7f39155d7700 1 -- start start 2024-03-18T20:47:33.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f39155d7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3910067e30 0x7f3910199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f39155d7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39100687f0 0x7f3910199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f39155d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3910105ce0 0x7f391019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f39155d7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391019e460 con 0x7f3910105ce0 2024-03-18T20:47:33.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f39155d7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391019e5d0 con 0x7f3910067e30 2024-03-18T20:47:33.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f39155d7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391019e790 con 0x7f39100687f0 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3910105ce0 0x7f391019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3910067e30 0x7f3910199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3910067e30 0x7f3910199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55062/0 (socket says 172.21.15.46:55062) 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390effd700 1 -- 172.21.15.46:0/3599665704 learned_addr learned my addr 172.21.15.46:0/3599665704 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3910105ce0 0x7f391019de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60918/0 (socket says 172.21.15.46:60918) 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390e7fc700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39100687f0 0x7f3910199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390f7fe700 1 -- 172.21.15.46:0/3599665704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39100687f0 msgr2=0x7f3910199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390f7fe700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39100687f0 0x7f3910199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390f7fe700 1 -- 172.21.15.46:0/3599665704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3910067e30 msgr2=0x7f3910199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390f7fe700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3910067e30 0x7f3910199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390f7fe700 1 -- 172.21.15.46:0/3599665704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900009520 con 0x7f3910105ce0 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390e7fc700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39100687f0 0x7f3910199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.228+0000 7f390effd700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3910067e30 0x7f3910199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.229+0000 7f390f7fe700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3910105ce0 0x7f391019de20 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f390800b0a0 tx=0x7f390800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.229+0000 7f38f7fff700 1 -- 172.21.15.46:0/3599665704 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390800e040 con 0x7f3910105ce0 2024-03-18T20:47:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.229+0000 7f38f7fff700 1 -- 172.21.15.46:0/3599665704 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f390801d470 con 0x7f3910105ce0 2024-03-18T20:47:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.229+0000 7f39155d7700 1 -- 172.21.15.46:0/3599665704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3908009d00 con 0x7f3910105ce0 2024-03-18T20:47:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.229+0000 7f38f7fff700 1 -- 172.21.15.46:0/3599665704 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908005260 con 0x7f3910105ce0 2024-03-18T20:47:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.229+0000 7f39155d7700 1 -- 172.21.15.46:0/3599665704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39101a60d0 con 0x7f3910105ce0 2024-03-18T20:47:33.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.232+0000 7f38f7fff700 1 -- 172.21.15.46:0/3599665704 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f390801d5e0 con 0x7f3910105ce0 2024-03-18T20:47:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.232+0000 7f38f7fff700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38f80a1290 0x7f38f80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.233+0000 7f38f7fff700 1 -- 172.21.15.46:0/3599665704 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f39080cdf00 con 0x7f3910105ce0 2024-03-18T20:47:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.233+0000 7f390effd700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38f80a1290 0x7f38f80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.233+0000 7f390effd700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38f80a1290 0x7f38f80a3750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3900009fd0 tx=0x7f3900009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.233+0000 7f392176cb80 1 Processor -- start 2024-03-18T20:47:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.233+0000 7f392176cb80 1 -- start start 2024-03-18T20:47:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f392176cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7135a60 0x559cc7135e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f392176cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7136380 0x559cc7140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f392176cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7141220 0x559cc7143610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f392176cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc708f990 con 0x559cc7135a60 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f392176cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc708d350 con 0x559cc7141220 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f392176cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc7088c40 con 0x559cc7136380 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f390effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7135a60 0x559cc7135e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7141220 0x559cc7143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f390effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7135a60 0x559cc7135e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60924/0 (socket says 172.21.15.46:60924) 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f390effd700 1 -- 172.21.15.46:0/671554451 learned_addr learned my addr 172.21.15.46:0/671554451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f390e7fc700 1 --2- 172.21.15.46:0/671554451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7136380 0x559cc7140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f390f7fe700 1 -- 172.21.15.46:0/671554451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7136380 msgr2=0x559cc7140ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f390f7fe700 1 --2- 172.21.15.46:0/671554451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7136380 0x559cc7140ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f390f7fe700 1 -- 172.21.15.46:0/671554451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7135a60 msgr2=0x559cc7135e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f390f7fe700 1 --2- 172.21.15.46:0/671554451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7135a60 0x559cc7135e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f390f7fe700 1 -- 172.21.15.46:0/671554451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cc6f0a3b0 con 0x559cc7141220 2024-03-18T20:47:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f390effd700 1 --2- 172.21.15.46:0/671554451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7135a60 0x559cc7135e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f390f7fe700 1 --2- 172.21.15.46:0/671554451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7141220 0x559cc7143610 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f39080cb630 tx=0x7f3908097440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=231dd9f66d2a8a6d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.234+0000 7f38f57fa700 1 -- 172.21.15.46:0/671554451 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390809e030 con 0x559cc7141220 2024-03-18T20:47:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.235+0000 7f38f57fa700 1 -- 172.21.15.46:0/671554451 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f390809a9e0 con 0x559cc7141220 2024-03-18T20:47:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.235+0000 7f38f57fa700 1 -- 172.21.15.46:0/671554451 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39080a10b0 con 0x559cc7141220 2024-03-18T20:47:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.235+0000 7f392176cb80 1 -- 172.21.15.46:0/671554451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7141220 msgr2=0x559cc7143610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.235+0000 7f392176cb80 1 --2- 172.21.15.46:0/671554451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7141220 0x559cc7143610 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f39080cb630 tx=0x7f3908097440 comp rx=0 tx=0).stop 2024-03-18T20:47:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.236+0000 7f392176cb80 1 -- 172.21.15.46:0/671554451 shutdown_connections 2024-03-18T20:47:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.236+0000 7f392176cb80 1 --2- 172.21.15.46:0/671554451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7135a60 0x559cc7135e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.236+0000 7f392176cb80 1 --2- 172.21.15.46:0/671554451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7141220 0x559cc7143610 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.236+0000 7f392176cb80 1 --2- 172.21.15.46:0/671554451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7136380 0x559cc7140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.236+0000 7f392176cb80 1 -- 172.21.15.46:0/671554451 >> 172.21.15.46:0/671554451 conn(0x559cc70ac220 msgr2=0x559cc7132020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:33.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.236+0000 7f392176cb80 1 -- 172.21.15.46:0/671554451 shutdown_connections 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.236+0000 7f392176cb80 1 -- 172.21.15.46:0/671554451 wait complete. 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.236+0000 7f392176cb80 1 -- start start 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.236+0000 7f392176cb80 1 Processor -- start 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.236+0000 7f392176cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 0x559cc713e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.236+0000 7f392176cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7136380 0x559cc713e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f392176cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7141220 0x559cc713eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f392176cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc7024d80 con 0x559cc7136380 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f392176cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc7024a20 con 0x559cc7135a60 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f392176cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc6f0c6a0 con 0x559cc7141220 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f392176cb80 10 client.0 _ll_register_callbacks cb 0x559cc6e903a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f390effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 0x559cc713e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f390e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7136380 0x559cc713e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f390effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 0x559cc713e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55092/0 (socket says 172.21.15.46:55092) 2024-03-18T20:47:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f390effd700 1 -- 172.21.15.46:0/1591047321 learned_addr learned my addr 172.21.15.46:0/1591047321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f390f7fe700 1 --2- 172.21.15.46:0/1591047321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7141220 0x559cc713eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f390effd700 1 -- 172.21.15.46:0/1591047321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7141220 msgr2=0x559cc713eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f390effd700 1 --2- 172.21.15.46:0/1591047321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7141220 0x559cc713eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f390effd700 1 -- 172.21.15.46:0/1591047321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7136380 msgr2=0x559cc713e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f390effd700 1 --2- 172.21.15.46:0/1591047321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7136380 0x559cc713e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f390effd700 1 -- 172.21.15.46:0/1591047321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3900017050 con 0x559cc7135a60 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f390effd700 1 --2- 172.21.15.46:0/1591047321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 0x559cc713e0b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3900013010 tx=0x7f39000047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ce6b93ccb7ddd0f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f38f2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f38f2ff5700 1 -- 172.21.15.46:0/1591047321 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900022070 con 0x559cc7135a60 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f38f2ff5700 1 -- 172.21.15.46:0/1591047321 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f390001c820 con 0x559cc7135a60 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.237+0000 7f392176cb80 1 -- 172.21.15.46:0/1591047321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559cc7129ad0 con 0x559cc7135a60 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.238+0000 7f38f2ff5700 1 -- 172.21.15.46:0/1591047321 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f390001c9e0 con 0x559cc7135a60 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.238+0000 7f392176cb80 10 client.29225 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.238+0000 7f392176cb80 1 -- 172.21.15.46:0/1591047321 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f39101a60d0 con 0x559cc7135a60 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.238+0000 7f38f2ff5700 1 -- 172.21.15.46:0/1591047321 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3900035020 con 0x559cc7135a60 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.238+0000 7f38f2ff5700 1 -- 172.21.15.46:0/1591047321 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3900031c20 con 0x559cc7135a60 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.238+0000 7f392176cb80 10 client.29225 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.238+0000 7f392176cb80 10 client.29225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.238+0000 7f392176cb80 20 client.29225 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.238+0000 7f392176cb80 1 --2- 172.21.15.46:0/1591047321 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559cc7150d50 0x559cc7153170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.238+0000 7f392176cb80 4 client.29225 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.238+0000 7f392176cb80 1 -- 172.21.15.46:0/1591047321 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cc7081700 con 0x559cc7150d50 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.238+0000 7f390e7fc700 1 --2- 172.21.15.46:0/1591047321 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559cc7150d50 0x559cc7153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.239+0000 7f390e7fc700 1 --2- 172.21.15.46:0/1591047321 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559cc7150d50 0x559cc7153170 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b36e9379773c39 server_cookie=bab0bbe76740941b in_seq=0 out_seq=0 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.239+0000 7f38f2ff5700 10 client.29225 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.239+0000 7f38f2ff5700 1 -- 172.21.15.46:0/1591047321 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559cc7081700 con 0x559cc7150d50 2024-03-18T20:47:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.239+0000 7f38f2ff5700 10 client.29225 handle_command_reply: tid=0 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.239+0000 7f392176cb80 1 client.29225 shutdown 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.240+0000 7f392176cb80 1 -- 172.21.15.46:0/1591047321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 msgr2=0x559cc713e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.240+0000 7f392176cb80 1 --2- 172.21.15.46:0/1591047321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 0x559cc713e0b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3900013010 tx=0x7f39000047c0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.240+0000 7f392176cb80 1 -- 172.21.15.46:0/1591047321 shutdown_connections 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.240+0000 7f392176cb80 1 --2- 172.21.15.46:0/1591047321 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559cc7150d50 0x559cc7153170 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.240+0000 7f392176cb80 1 --2- 172.21.15.46:0/1591047321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7141220 0x559cc713eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.240+0000 7f392176cb80 1 --2- 172.21.15.46:0/1591047321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 0x559cc713e0b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.240+0000 7f392176cb80 1 --2- 172.21.15.46:0/1591047321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7136380 0x559cc713e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.240+0000 7f38f2ff5700 0 client.29225 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.240+0000 7f392176cb80 1 -- 172.21.15.46:0/1591047321 >> 172.21.15.46:0/1591047321 conn(0x559cc70ac220 msgr2=0x559cc7131c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.240+0000 7f392176cb80 1 -- 172.21.15.46:0/1591047321 shutdown_connections 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.240+0000 7f392176cb80 1 -- 172.21.15.46:0/1591047321 wait complete. 2024-03-18T20:47:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.240+0000 7f392176cb80 20 client.29225 trim_cache size 0 max 16384 2024-03-18T20:47:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f392176cb80 1 Processor -- start 2024-03-18T20:47:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f392176cb80 1 -- start start 2024-03-18T20:47:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f392176cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7135a60 0x559cc7138a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f392176cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7136380 0x559cc713e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f392176cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 0x559cc713b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f392176cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc713f610 con 0x559cc7141220 2024-03-18T20:47:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f392176cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc712e5e0 con 0x559cc7136380 2024-03-18T20:47:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f392176cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc712bf80 con 0x559cc7135a60 2024-03-18T20:47:33.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 0x559cc713b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 0x559cc713b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60940/0 (socket says 172.21.15.46:60940) 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f390e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7136380 0x559cc713e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f390f7fe700 1 -- 172.21.15.46:0/2363389102 learned_addr learned my addr 172.21.15.46:0/2363389102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.259+0000 7f390effd700 1 --2- 172.21.15.46:0/2363389102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7135a60 0x559cc7138a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.260+0000 7f390f7fe700 1 -- 172.21.15.46:0/2363389102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7135a60 msgr2=0x559cc7138a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.260+0000 7f390f7fe700 1 --2- 172.21.15.46:0/2363389102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7135a60 0x559cc7138a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.260+0000 7f390f7fe700 1 -- 172.21.15.46:0/2363389102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7136380 msgr2=0x559cc713e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.260+0000 7f390f7fe700 1 --2- 172.21.15.46:0/2363389102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7136380 0x559cc713e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.260+0000 7f390f7fe700 1 -- 172.21.15.46:0/2363389102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cc6eb6b60 con 0x559cc7141220 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.260+0000 7f390e7fc700 1 --2- 172.21.15.46:0/2363389102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7136380 0x559cc713e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.260+0000 7f390effd700 1 --2- 172.21.15.46:0/2363389102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7135a60 0x559cc7138a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.260+0000 7f390f7fe700 1 --2- 172.21.15.46:0/2363389102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 0x559cc713b350 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3908097580 tx=0x7f39080099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b19066503aa181 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.260+0000 7f38f5ffb700 1 -- 172.21.15.46:0/2363389102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390809e070 con 0x559cc7141220 2024-03-18T20:47:33.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.260+0000 7f38f5ffb700 1 -- 172.21.15.46:0/2363389102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39080b2070 con 0x559cc7141220 2024-03-18T20:47:33.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.260+0000 7f38f5ffb700 1 -- 172.21.15.46:0/2363389102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39080a3500 con 0x559cc7141220 2024-03-18T20:47:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.261+0000 7f392176cb80 1 -- 172.21.15.46:0/2363389102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 msgr2=0x559cc713b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.261+0000 7f392176cb80 1 --2- 172.21.15.46:0/2363389102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 0x559cc713b350 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3908097580 tx=0x7f39080099e0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.261+0000 7f392176cb80 1 -- 172.21.15.46:0/2363389102 shutdown_connections 2024-03-18T20:47:33.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.261+0000 7f392176cb80 1 --2- 172.21.15.46:0/2363389102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 0x559cc713b350 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.261+0000 7f392176cb80 1 --2- 172.21.15.46:0/2363389102 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7135a60 0x559cc7138a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.261+0000 7f392176cb80 1 --2- 172.21.15.46:0/2363389102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7136380 0x559cc713e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.261+0000 7f392176cb80 1 -- 172.21.15.46:0/2363389102 >> 172.21.15.46:0/2363389102 conn(0x559cc70ac220 msgr2=0x559cc713c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.261+0000 7f392176cb80 1 -- 172.21.15.46:0/2363389102 shutdown_connections 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.261+0000 7f392176cb80 1 -- 172.21.15.46:0/2363389102 wait complete. 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.261+0000 7f392176cb80 1 -- start start 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.261+0000 7f392176cb80 1 Processor -- start 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f392176cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 0x559cc7146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f392176cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7136380 0x559cc7147300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f392176cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 0x559cc71a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f392176cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc7147c10 con 0x559cc7141220 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f392176cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc71a40c0 con 0x559cc7135a60 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f392176cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cc71a4200 con 0x559cc7136380 2024-03-18T20:47:33.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f392176cb80 10 client.0 _ll_register_callbacks cb 0x7f390000fa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 0x559cc71a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 0x559cc71a3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60942/0 (socket says 172.21.15.46:60942) 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f390f7fe700 1 -- 172.21.15.46:0/2958651797 learned_addr learned my addr 172.21.15.46:0/2958651797 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f390effd700 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 0x559cc7146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f390e7fc700 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7136380 0x559cc7147300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f390f7fe700 1 -- 172.21.15.46:0/2958651797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7136380 msgr2=0x559cc7147300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f390f7fe700 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7136380 0x559cc7147300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f390f7fe700 1 -- 172.21.15.46:0/2958651797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 msgr2=0x559cc7146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f390f7fe700 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 0x559cc7146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f390f7fe700 1 -- 172.21.15.46:0/2958651797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39101a60d0 con 0x559cc7141220 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.262+0000 7f390effd700 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 0x559cc7146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f390f7fe700 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 0x559cc71a3b80 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f390809ab50 tx=0x7f3908006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5e895cdda1bf71 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f38f2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:33.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f38f2ff5700 1 -- 172.21.15.46:0/2958651797 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390809e070 con 0x559cc7141220 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f38f2ff5700 1 -- 172.21.15.46:0/2958651797 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f390809b080 con 0x559cc7141220 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f392176cb80 1 -- 172.21.15.46:0/2958651797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f390809a9e0 con 0x559cc7141220 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f38f2ff5700 1 -- 172.21.15.46:0/2958651797 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f390809b780 con 0x559cc7141220 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f392176cb80 10 client.19614 fetch_fsmap learned FSMap version 49 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f392176cb80 1 -- 172.21.15.46:0/2958651797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f3904009eb0 con 0x559cc7141220 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f38f2ff5700 1 -- 172.21.15.46:0/2958651797 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3908083020 con 0x559cc7141220 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f38f2ff5700 1 -- 172.21.15.46:0/2958651797 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f39080b9280 con 0x559cc7141220 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f392176cb80 10 client.19614 fetch_fsmap finished waiting for FSMap version 49 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f392176cb80 10 client.19614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f392176cb80 20 client.19614 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f392176cb80 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559cc71a5400 0x559cc71a7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f392176cb80 4 client.19614 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.263+0000 7f392176cb80 1 -- 172.21.15.46:0/2958651797 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559cc6ed7cf0 con 0x559cc71a5400 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.264+0000 7f390effd700 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559cc71a5400 0x559cc71a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:33.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.264+0000 7f38f2ff5700 10 client.19614 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.264+0000 7f390effd700 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559cc71a5400 0x559cc71a7820 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c88530859e57b543 server_cookie=f07a7650adf52d93 in_seq=0 out_seq=0 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.264+0000 7f38f2ff5700 1 -- 172.21.15.46:0/2958651797 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559cc6ed7cf0 con 0x559cc71a5400 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.264+0000 7f38f2ff5700 10 client.19614 handle_command_reply: tid=0 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.264+0000 7f392176cb80 1 client.19614 shutdown 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.264+0000 7f392176cb80 1 -- 172.21.15.46:0/2958651797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 msgr2=0x559cc71a3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.264+0000 7f392176cb80 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 0x559cc71a3b80 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f390809ab50 tx=0x7f3908006000 comp rx=0 tx=0).stop 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.264+0000 7f392176cb80 1 -- 172.21.15.46:0/2958651797 shutdown_connections 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.264+0000 7f392176cb80 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559cc71a5400 0x559cc71a7820 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.264+0000 7f392176cb80 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559cc7141220 0x559cc71a3b80 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.264+0000 7f392176cb80 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cc7135a60 0x559cc7146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f38f2ff5700 0 client.19614 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f392176cb80 1 --2- 172.21.15.46:0/2958651797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559cc7136380 0x559cc7147300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f392176cb80 1 -- 172.21.15.46:0/2958651797 >> 172.21.15.46:0/2958651797 conn(0x559cc70ac220 msgr2=0x559cc713f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:33.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f392176cb80 1 -- 172.21.15.46:0/2958651797 shutdown_connections 2024-03-18T20:47:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f392176cb80 1 -- 172.21.15.46:0/2958651797 wait complete. 2024-03-18T20:47:33.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f392176cb80 20 client.19614 trim_cache size 0 max 16384 2024-03-18T20:47:33.274 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:47:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:47:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:47:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "26fb7a65-8c29-4b93-af76-eba0604f5736": { 2024-03-18T20:47:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:47:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "26fb7a65-8c29-4b93-af76-eba0604f5736", 2024-03-18T20:47:33.274 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:47:33.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:47:33.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:47:33.275 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:47:33.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f39155d7700 1 -- 172.21.15.46:0/3599665704 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38f80a1290 msgr2=0x7f38f80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f39155d7700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38f80a1290 0x7f38f80a3750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3900009fd0 tx=0x7f3900009260 comp rx=0 tx=0).stop 2024-03-18T20:47:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f39155d7700 1 -- 172.21.15.46:0/3599665704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3910105ce0 msgr2=0x7f391019de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f39155d7700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3910105ce0 0x7f391019de20 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f390800b0a0 tx=0x7f390800bdd0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f39155d7700 1 -- 172.21.15.46:0/3599665704 shutdown_connections 2024-03-18T20:47:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f39155d7700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38f80a1290 0x7f38f80a3750 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f39155d7700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3910105ce0 0x7f391019de20 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f39155d7700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3910067e30 0x7f3910199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f39155d7700 1 --2- 172.21.15.46:0/3599665704 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39100687f0 0x7f3910199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f39155d7700 1 -- 172.21.15.46:0/3599665704 >> 172.21.15.46:0/3599665704 conn(0x7f3910076d40 msgr2=0x7f3910104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.265+0000 7f39155d7700 1 -- 172.21.15.46:0/3599665704 shutdown_connections 2024-03-18T20:47:33.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:47:33.266+0000 7f39155d7700 1 -- 172.21.15.46:0/3599665704 wait complete. 2024-03-18T20:47:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:33 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:33 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:33 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:33 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:33 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:33 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:33 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:33 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:33 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.277 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:33 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:33 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:33.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:33 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.284 INFO:tasks.cephfs.filesystem:scrub status for tag:26fb7a65-8c29-4b93-af76-eba0604f5736 - {'path': '/', 'tag': '26fb7a65-8c29-4b93-af76-eba0604f5736', 'options': 'recursive,force'} 2024-03-18T20:47:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:33 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:33 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:33 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:33 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:33 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:33 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:33 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:33 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:33 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:33 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:33 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:33.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:33 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 432388 3.71 MB/sec execute 508 sec latency 584.771 ms 2024-03-18T20:47:33.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:33 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:33.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:33 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:33 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:33.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:33 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:33 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:33.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:33 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:33 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:33.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:33 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:33.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:33 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:33 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:33.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:33 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:33.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:33 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: pgmap v357: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 2.0 MiB/s wr, 182 op/s 2024-03-18T20:47:34.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:34 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:47:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:47:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:47:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: pgmap v357: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 2.0 MiB/s wr, 182 op/s 2024-03-18T20:47:34.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:34 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:47:34.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 433566 3.72 MB/sec execute 509 sec latency 19.076 ms 2024-03-18T20:47:34.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:47:34.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:47:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:47:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: pgmap v357: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 2.0 MiB/s wr, 182 op/s 2024-03-18T20:47:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:34 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:47:35.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 434458 3.72 MB/sec execute 510 sec latency 335.253 ms 2024-03-18T20:47:36.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:36 smithi155 ceph-mon[48344]: pgmap v358: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-18T20:47:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:36 smithi046 ceph-mon[43495]: pgmap v358: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-18T20:47:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.352+0000 7fa220e4d700 1 -- 172.21.15.46:0/3508788959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c1001a0 msgr2=0x7fa21c103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.352+0000 7fa220e4d700 1 --2- 172.21.15.46:0/3508788959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c1001a0 0x7fa21c103f50 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa210009990 tx=0x7fa210009ca0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.352+0000 7fa220e4d700 1 -- 172.21.15.46:0/3508788959 shutdown_connections 2024-03-18T20:47:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.352+0000 7fa220e4d700 1 --2- 172.21.15.46:0/3508788959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c104680 0x7fa21c10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.352+0000 7fa220e4d700 1 --2- 172.21.15.46:0/3508788959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa21c0ff7f0 0x7fa21c0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.352+0000 7fa220e4d700 1 --2- 172.21.15.46:0/3508788959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c1001a0 0x7fa21c103f50 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.352+0000 7fa220e4d700 1 -- 172.21.15.46:0/3508788959 >> 172.21.15.46:0/3508788959 conn(0x7fa21c076180 msgr2=0x7fa21c076590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.352+0000 7fa220e4d700 1 -- 172.21.15.46:0/3508788959 shutdown_connections 2024-03-18T20:47:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.352+0000 7fa220e4d700 1 -- 172.21.15.46:0/3508788959 wait complete. 2024-03-18T20:47:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.353+0000 7fa220e4d700 1 Processor -- start 2024-03-18T20:47:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.353+0000 7fa220e4d700 1 -- start start 2024-03-18T20:47:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.353+0000 7fa220e4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c0ff7f0 0x7fa21c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.353+0000 7fa220e4d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa21c104680 0x7fa21c1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.353+0000 7fa220e4d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c19ba40 0x7fa21c19bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.353+0000 7fa220e4d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c19c4c0 con 0x7fa21c0ff7f0 2024-03-18T20:47:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.353+0000 7fa220e4d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c19c630 con 0x7fa21c104680 2024-03-18T20:47:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.353+0000 7fa220e4d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c19c7f0 con 0x7fa21c19ba40 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c0ff7f0 0x7fa21c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c0ff7f0 0x7fa21c197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54904/0 (socket says 172.21.15.46:54904) 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa21a59c700 1 -- 172.21.15.46:0/3940047639 learned_addr learned my addr 172.21.15.46:0/3940047639 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa21c104680 0x7fa21c1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa21ad9d700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c19ba40 0x7fa21c19bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa21a59c700 1 -- 172.21.15.46:0/3940047639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c19ba40 msgr2=0x7fa21c19bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa21a59c700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c19ba40 0x7fa21c19bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa21a59c700 1 -- 172.21.15.46:0/3940047639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa21c104680 msgr2=0x7fa21c1976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa21a59c700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa21c104680 0x7fa21c1976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa21a59c700 1 -- 172.21.15.46:0/3940047639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100095f0 con 0x7fa21c0ff7f0 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa21a59c700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c0ff7f0 0x7fa21c197160 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00d950 tx=0x7fa20c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa20b7fe700 1 -- 172.21.15.46:0/3940047639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c00f890 con 0x7fa21c0ff7f0 2024-03-18T20:47:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa20b7fe700 1 -- 172.21.15.46:0/3940047639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa20c010460 con 0x7fa21c0ff7f0 2024-03-18T20:47:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa220e4d700 1 -- 172.21.15.46:0/3940047639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c1a3d50 con 0x7fa21c0ff7f0 2024-03-18T20:47:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa20b7fe700 1 -- 172.21.15.46:0/3940047639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c00e610 con 0x7fa21c0ff7f0 2024-03-18T20:47:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.354+0000 7fa220e4d700 1 -- 172.21.15.46:0/3940047639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c1a4220 con 0x7fa21c0ff7f0 2024-03-18T20:47:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.357+0000 7fa20b7fe700 1 -- 172.21.15.46:0/3940047639 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa20c0108d0 con 0x7fa21c0ff7f0 2024-03-18T20:47:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.358+0000 7fa20b7fe700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2040a1310 0x7fa2040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.358+0000 7fa20b7fe700 1 -- 172.21.15.46:0/3940047639 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa20c0cca30 con 0x7fa21c0ff7f0 2024-03-18T20:47:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.358+0000 7fa219d9b700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2040a1310 0x7fa2040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.359+0000 7fa219d9b700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2040a1310 0x7fa2040a37d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa21000b5c0 tx=0x7fa210005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.359+0000 7fa22cfe2b80 1 Processor -- start 2024-03-18T20:47:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.359+0000 7fa22cfe2b80 1 -- start start 2024-03-18T20:47:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.359+0000 7fa22cfe2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44a97950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.359+0000 7fa22cfe2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aa6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.359+0000 7fa22cfe2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44aa95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.359+0000 7fa22cfe2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd449f1930 con 0x55dd44aa71c0 2024-03-18T20:47:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.359+0000 7fa22cfe2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd449ef2f0 con 0x55dd44a97570 2024-03-18T20:47:36.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.359+0000 7fa22cfe2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd449eabe0 con 0x55dd44a97e90 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa21ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44aa95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa21ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44aa95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54914/0 (socket says 172.21.15.46:54914) 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa21ad9d700 1 -- 172.21.15.46:0/3150779707 learned_addr learned my addr 172.21.15.46:0/3150779707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa21a59c700 1 --2- 172.21.15.46:0/3150779707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44a97950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa219d9b700 1 --2- 172.21.15.46:0/3150779707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aa6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa21ad9d700 1 -- 172.21.15.46:0/3150779707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 msgr2=0x55dd44aa6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa21ad9d700 1 --2- 172.21.15.46:0/3150779707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aa6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa21ad9d700 1 -- 172.21.15.46:0/3150779707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 msgr2=0x55dd44a97950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa21ad9d700 1 --2- 172.21.15.46:0/3150779707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44a97950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa21ad9d700 1 -- 172.21.15.46:0/3150779707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd4486c350 con 0x55dd44aa71c0 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa21a59c700 1 --2- 172.21.15.46:0/3150779707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44a97950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa21ad9d700 1 --2- 172.21.15.46:0/3150779707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44aa95b0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fa2140077b0 tx=0x7fa214007ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7d02f874a9cd45 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:36.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa208ff9700 1 -- 172.21.15.46:0/3150779707 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400f040 con 0x55dd44aa71c0 2024-03-18T20:47:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa208ff9700 1 -- 172.21.15.46:0/3150779707 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa21400a650 con 0x55dd44aa71c0 2024-03-18T20:47:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.360+0000 7fa208ff9700 1 -- 172.21.15.46:0/3150779707 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214004030 con 0x55dd44aa71c0 2024-03-18T20:47:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.361+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/3150779707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 msgr2=0x55dd44aa95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.361+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/3150779707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44aa95b0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fa2140077b0 tx=0x7fa214007ac0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.361+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/3150779707 shutdown_connections 2024-03-18T20:47:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.361+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/3150779707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44aa95b0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.361+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/3150779707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44a97950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.361+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/3150779707 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aa6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.361+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/3150779707 >> 172.21.15.46:0/3150779707 conn(0x55dd44a0e1c0 msgr2=0x55dd44a93a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.361+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/3150779707 shutdown_connections 2024-03-18T20:47:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.361+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/3150779707 wait complete. 2024-03-18T20:47:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa22cfe2b80 1 -- start start 2024-03-18T20:47:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa22cfe2b80 1 Processor -- start 2024-03-18T20:47:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa22cfe2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97570 0x55dd44aa6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa22cfe2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44a97e90 0x55dd44aa3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa22cfe2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44aa71c0 0x55dd44aa4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa22cfe2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd44986d20 con 0x55dd44a97e90 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa22cfe2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd449869c0 con 0x55dd44aa71c0 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa22cfe2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4486e640 con 0x55dd44a97570 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa22cfe2b80 10 client.0 _ll_register_callbacks cb 0x55dd447f2340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44a97e90 0x55dd44aa3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97570 0x55dd44aa6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44a97e90 0x55dd44aa3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54922/0 (socket says 172.21.15.46:54922) 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa219d9b700 1 -- 172.21.15.46:0/2875969291 learned_addr learned my addr 172.21.15.46:0/2875969291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa21ad9d700 1 --2- 172.21.15.46:0/2875969291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44aa71c0 0x55dd44aa4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa219d9b700 1 -- 172.21.15.46:0/2875969291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97570 msgr2=0x55dd44aa6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa219d9b700 1 --2- 172.21.15.46:0/2875969291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97570 0x55dd44aa6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.362+0000 7fa219d9b700 1 -- 172.21.15.46:0/2875969291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44aa71c0 msgr2=0x55dd44aa4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.363+0000 7fa219d9b700 1 --2- 172.21.15.46:0/2875969291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44aa71c0 0x55dd44aa4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.363+0000 7fa219d9b700 1 -- 172.21.15.46:0/2875969291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c1a4220 con 0x55dd44a97e90 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.363+0000 7fa219d9b700 1 --2- 172.21.15.46:0/2875969291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44a97e90 0x55dd44aa3f40 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fa210009990 tx=0x7fa21001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3090b99f98d2fbb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.363+0000 7fa201ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.363+0000 7fa201ffb700 1 -- 172.21.15.46:0/2875969291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210004810 con 0x55dd44a97e90 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.363+0000 7fa201ffb700 1 -- 172.21.15.46:0/2875969291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa210004970 con 0x55dd44a97e90 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.363+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/2875969291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd44a8b6c0 con 0x55dd44a97e90 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.363+0000 7fa201ffb700 1 -- 172.21.15.46:0/2875969291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa210004b30 con 0x55dd44a97e90 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.363+0000 7fa22cfe2b80 10 client.19626 fetch_fsmap learned FSMap version 50 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.363+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/2875969291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7fa2140073e0 con 0x55dd44a97e90 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.363+0000 7fa201ffb700 1 -- 172.21.15.46:0/2875969291 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa21003d020 con 0x55dd44a97e90 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.363+0000 7fa201ffb700 1 -- 172.21.15.46:0/2875969291 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa21000fd20 con 0x55dd44a97e90 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.364+0000 7fa22cfe2b80 10 client.19626 fetch_fsmap finished waiting for FSMap version 50 2024-03-18T20:47:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.364+0000 7fa22cfe2b80 10 client.19626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.364+0000 7fa22cfe2b80 20 client.19626 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.364+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/2875969291 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd44ab6e90 0x55dd44ab92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.364+0000 7fa22cfe2b80 4 client.19626 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.364+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/2875969291 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd449e36a0 con 0x55dd44ab6e90 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.364+0000 7fa21a59c700 1 --2- 172.21.15.46:0/2875969291 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd44ab6e90 0x55dd44ab92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.364+0000 7fa21a59c700 1 --2- 172.21.15.46:0/2875969291 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd44ab6e90 0x55dd44ab92b0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=296238e8b9841a74 server_cookie=6c7f0375db7e19a5 in_seq=0 out_seq=0 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.364+0000 7fa201ffb700 10 client.19626 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa201ffb700 1 -- 172.21.15.46:0/2875969291 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dd449e36a0 con 0x55dd44ab6e90 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa201ffb700 10 client.19626 handle_command_reply: tid=0 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa22cfe2b80 1 client.19626 shutdown 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/2875969291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44a97e90 msgr2=0x55dd44aa3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/2875969291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44a97e90 0x55dd44aa3f40 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fa210009990 tx=0x7fa21001a050 comp rx=0 tx=0).stop 2024-03-18T20:47:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/2875969291 shutdown_connections 2024-03-18T20:47:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/2875969291 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd44ab6e90 0x55dd44ab92b0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/2875969291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44aa71c0 0x55dd44aa4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/2875969291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97570 0x55dd44aa6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa201ffb700 0 client.19626 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/2875969291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44a97e90 0x55dd44aa3f40 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/2875969291 >> 172.21.15.46:0/2875969291 conn(0x55dd44a0e1c0 msgr2=0x55dd44a93650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/2875969291 shutdown_connections 2024-03-18T20:47:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/2875969291 wait complete. 2024-03-18T20:47:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.365+0000 7fa22cfe2b80 20 client.19626 trim_cache size 0 max 16384 2024-03-18T20:47:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.390+0000 7fa22cfe2b80 1 Processor -- start 2024-03-18T20:47:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.391+0000 7fa22cfe2b80 1 -- start start 2024-03-18T20:47:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.391+0000 7fa22cfe2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44ab9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.391+0000 7fa22cfe2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aa66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.391+0000 7fa22cfe2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44ab25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.391+0000 7fa22cfe2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd44aa9c50 con 0x55dd44aa71c0 2024-03-18T20:47:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.391+0000 7fa22cfe2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd44a94760 con 0x55dd44a97570 2024-03-18T20:47:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.391+0000 7fa22cfe2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd44aa24f0 con 0x55dd44a97e90 2024-03-18T20:47:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.391+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44ab9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.391+0000 7fa21ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44ab25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.391+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44ab9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52784/0 (socket says 172.21.15.46:52784) 2024-03-18T20:47:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.391+0000 7fa21a59c700 1 -- 172.21.15.46:0/3272827778 learned_addr learned my addr 172.21.15.46:0/3272827778 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.391+0000 7fa219d9b700 1 --2- 172.21.15.46:0/3272827778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aa66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.392+0000 7fa21a59c700 1 -- 172.21.15.46:0/3272827778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 msgr2=0x55dd44aa66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.392+0000 7fa21a59c700 1 --2- 172.21.15.46:0/3272827778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aa66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.392+0000 7fa21a59c700 1 -- 172.21.15.46:0/3272827778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 msgr2=0x55dd44ab25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.392+0000 7fa21a59c700 1 --2- 172.21.15.46:0/3272827778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44ab25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.392+0000 7fa21a59c700 1 -- 172.21.15.46:0/3272827778 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd44818b00 con 0x55dd44a97570 2024-03-18T20:47:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.392+0000 7fa21ad9d700 1 --2- 172.21.15.46:0/3272827778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44ab25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.392+0000 7fa21a59c700 1 --2- 172.21.15.46:0/3272827778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44ab9280 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa20c099ab0 tx=0x7fa20c0cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2331c5a8a4c3bd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.392+0000 7fa2097fa700 1 -- 172.21.15.46:0/3272827778 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c014070 con 0x55dd44a97570 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.392+0000 7fa2097fa700 1 -- 172.21.15.46:0/3272827778 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa20c0a1fa0 con 0x55dd44a97570 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.392+0000 7fa2097fa700 1 -- 172.21.15.46:0/3272827778 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c099270 con 0x55dd44a97570 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.393+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/3272827778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 msgr2=0x55dd44ab9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.393+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/3272827778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44ab9280 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa20c099ab0 tx=0x7fa20c0cc040 comp rx=0 tx=0).stop 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.393+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/3272827778 shutdown_connections 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.393+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/3272827778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44ab25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.393+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/3272827778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44ab9280 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.393+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/3272827778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aa66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.393+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/3272827778 >> 172.21.15.46:0/3272827778 conn(0x55dd44a0e1c0 msgr2=0x55dd44a92120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.394+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/3272827778 shutdown_connections 2024-03-18T20:47:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.394+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/3272827778 wait complete. 2024-03-18T20:47:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.394+0000 7fa22cfe2b80 1 -- start start 2024-03-18T20:47:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.394+0000 7fa22cfe2b80 1 Processor -- start 2024-03-18T20:47:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.395+0000 7fa22cfe2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44aac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.395+0000 7fa22cfe2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.395+0000 7fa22cfe2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44b09cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.395+0000 7fa22cfe2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd44b0a200 con 0x55dd44aa71c0 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.395+0000 7fa22cfe2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd44b0a340 con 0x55dd44a97570 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.395+0000 7fa22cfe2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd44b0a480 con 0x55dd44a97e90 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.395+0000 7fa22cfe2b80 10 client.0 _ll_register_callbacks cb 0x7fa2100056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.395+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.395+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aac590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:51366/0 (socket says 172.21.15.46:51366) 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.395+0000 7fa219d9b700 1 -- 172.21.15.46:0/1277451112 learned_addr learned my addr 172.21.15.46:0/1277451112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.395+0000 7fa21ad9d700 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44b09cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.395+0000 7fa21a59c700 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44aac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa219d9b700 1 -- 172.21.15.46:0/1277451112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 msgr2=0x55dd44aac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa219d9b700 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44aac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa219d9b700 1 -- 172.21.15.46:0/1277451112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 msgr2=0x55dd44b09cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa219d9b700 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44b09cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa219d9b700 1 -- 172.21.15.46:0/1277451112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd4486c350 con 0x55dd44a97e90 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa21ad9d700 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44b09cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa21a59c700 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44aac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa219d9b700 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aac590 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa2100305a0 tx=0x7fa2100308f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dea9752693f24729 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa201ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa201ffb700 1 -- 172.21.15.46:0/1277451112 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210030c80 con 0x55dd44a97e90 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa201ffb700 1 -- 172.21.15.46:0/1277451112 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa21003e080 con 0x55dd44a97e90 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/1277451112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa21400a650 con 0x55dd44a97e90 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.396+0000 7fa201ffb700 1 -- 172.21.15.46:0/1277451112 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa210030de0 con 0x55dd44a97e90 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.397+0000 7fa22cfe2b80 10 client.29272 fetch_fsmap learned FSMap version 50 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.397+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/1277451112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7fa21c1a3d50 con 0x55dd44a97e90 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.397+0000 7fa201ffb700 1 -- 172.21.15.46:0/1277451112 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa21400a650 con 0x55dd44a97e90 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.397+0000 7fa201ffb700 1 -- 172.21.15.46:0/1277451112 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 50) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa21000b9f0 con 0x55dd44a97e90 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.398+0000 7fa22cfe2b80 10 client.29272 fetch_fsmap finished waiting for FSMap version 50 2024-03-18T20:47:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.398+0000 7fa22cfe2b80 10 client.29272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.398+0000 7fa22cfe2b80 20 client.29272 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.398+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd44b0b800 0x55dd44b0dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.398+0000 7fa22cfe2b80 4 client.29272 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:47:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.398+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/1277451112 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dd44839c90 con 0x55dd44b0b800 2024-03-18T20:47:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.398+0000 7fa21a59c700 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd44b0b800 0x55dd44b0dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.398+0000 7fa201ffb700 10 client.29272 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.398+0000 7fa21a59c700 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd44b0b800 0x55dd44b0dc20 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59df36383345f89e server_cookie=cb1d59d2bb760f2e in_seq=0 out_seq=0 2024-03-18T20:47:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.428+0000 7fa201ffb700 1 -- 172.21.15.46:0/1277451112 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dd44839c90 con 0x55dd44b0b800 2024-03-18T20:47:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.428+0000 7fa201ffb700 10 client.29272 handle_command_reply: tid=0 2024-03-18T20:47:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.428+0000 7fa22cfe2b80 1 client.29272 shutdown 2024-03-18T20:47:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.429+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/1277451112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 msgr2=0x55dd44aac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.429+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aac590 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa2100305a0 tx=0x7fa2100308f0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.429+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/1277451112 shutdown_connections 2024-03-18T20:47:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.429+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55dd44b0b800 0x55dd44b0dc20 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.429+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dd44aa71c0 0x55dd44b09cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.429+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd44a97570 0x55dd44aac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.429+0000 7fa22cfe2b80 1 --2- 172.21.15.46:0/1277451112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dd44a97e90 0x55dd44aac590 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.429+0000 7fa201ffb700 0 client.29272 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.429+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/1277451112 >> 172.21.15.46:0/1277451112 conn(0x55dd44a0e1c0 msgr2=0x55dd44ab25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.429+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/1277451112 shutdown_connections 2024-03-18T20:47:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.429+0000 7fa22cfe2b80 1 -- 172.21.15.46:0/1277451112 wait complete. 2024-03-18T20:47:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.429+0000 7fa22cfe2b80 20 client.29272 trim_cache size 0 max 16384 2024-03-18T20:47:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:47:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:47:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:47:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.430+0000 7fa220e4d700 1 -- 172.21.15.46:0/3940047639 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2040a1310 msgr2=0x7fa2040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.430+0000 7fa220e4d700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2040a1310 0x7fa2040a37d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa21000b5c0 tx=0x7fa210005960 comp rx=0 tx=0).stop 2024-03-18T20:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.430+0000 7fa220e4d700 1 -- 172.21.15.46:0/3940047639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c0ff7f0 msgr2=0x7fa21c197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.430+0000 7fa220e4d700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c0ff7f0 0x7fa21c197160 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00d950 tx=0x7fa20c00dc60 comp rx=0 tx=0).stop 2024-03-18T20:47:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.430+0000 7fa220e4d700 1 -- 172.21.15.46:0/3940047639 shutdown_connections 2024-03-18T20:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.430+0000 7fa220e4d700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2040a1310 0x7fa2040a37d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.430+0000 7fa220e4d700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa21c0ff7f0 0x7fa21c197160 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.430+0000 7fa220e4d700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa21c19ba40 0x7fa21c19bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.430+0000 7fa220e4d700 1 --2- 172.21.15.46:0/3940047639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa21c104680 0x7fa21c1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.430+0000 7fa220e4d700 1 -- 172.21.15.46:0/3940047639 >> 172.21.15.46:0/3940047639 conn(0x7fa21c076180 msgr2=0x7fa21c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.430+0000 7fa220e4d700 1 -- 172.21.15.46:0/3940047639 shutdown_connections 2024-03-18T20:47:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:36.430+0000 7fa220e4d700 1 -- 172.21.15.46:0/3940047639 wait complete. 2024-03-18T20:47:36.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:36 smithi073 ceph-mon[47384]: pgmap v358: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-18T20:47:36.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 434458 3.71 MB/sec execute 511 sec latency 1335.332 ms 2024-03-18T20:47:37.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 434458 3.71 MB/sec execute 512 sec latency 2335.423 ms 2024-03-18T20:47:38.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:38 smithi155 ceph-mon[48344]: pgmap v359: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-18T20:47:38.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:38 smithi046 ceph-mon[43495]: pgmap v359: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-18T20:47:38.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 434458 3.70 MB/sec execute 513 sec latency 3335.537 ms 2024-03-18T20:47:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:38 smithi073 ceph-mon[47384]: pgmap v359: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-18T20:47:39.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 435693 3.71 MB/sec execute 514 sec latency 3804.321 ms 2024-03-18T20:47:40.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:40 smithi155 ceph-mon[48344]: pgmap v360: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-03-18T20:47:40.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:40 smithi046 ceph-mon[43495]: pgmap v360: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-03-18T20:47:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:40 smithi073 ceph-mon[47384]: pgmap v360: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-03-18T20:47:40.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 436979 3.71 MB/sec execute 515 sec latency 16.240 ms 2024-03-18T20:47:41.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 438005 3.72 MB/sec execute 516 sec latency 367.969 ms 2024-03-18T20:47:42.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:42 smithi155 ceph-mon[48344]: pgmap v361: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-03-18T20:47:42.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:42 smithi046 ceph-mon[43495]: pgmap v361: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-03-18T20:47:42.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:42 smithi073 ceph-mon[47384]: pgmap v361: 97 pgs: 97 active+clean; 169 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-03-18T20:47:42.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 438895 3.71 MB/sec execute 517 sec latency 973.295 ms 2024-03-18T20:47:43.719 INFO:tasks.workunit.client.0.smithi046.stdout: 1 440382 3.73 MB/sec execute 518 sec latency 11.909 ms 2024-03-18T20:47:44.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:44 smithi155 ceph-mon[48344]: pgmap v362: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-03-18T20:47:44.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:44 smithi046 ceph-mon[43495]: pgmap v362: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-03-18T20:47:44.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:44 smithi073 ceph-mon[47384]: pgmap v362: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-03-18T20:47:44.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 441547 3.73 MB/sec execute 519 sec latency 112.946 ms 2024-03-18T20:47:45.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 443118 3.74 MB/sec execute 520 sec latency 249.609 ms 2024-03-18T20:47:46.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:46 smithi155 ceph-mon[48344]: pgmap v363: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-03-18T20:47:46.295 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:46 smithi046 ceph-mon[43495]: pgmap v363: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-03-18T20:47:46.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:46 smithi073 ceph-mon[47384]: pgmap v363: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-03-18T20:47:46.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 444195 3.74 MB/sec execute 521 sec latency 20.302 ms 2024-03-18T20:47:47.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 445096 3.75 MB/sec execute 522 sec latency 468.557 ms 2024-03-18T20:47:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:48 smithi155 ceph-mon[48344]: pgmap v364: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-03-18T20:47:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:48 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:47:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:48 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:47:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:48 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:47:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:48 smithi046 ceph-mon[43495]: pgmap v364: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-03-18T20:47:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:48 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:47:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:48 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:47:48.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:48 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:47:48.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:48 smithi073 ceph-mon[47384]: pgmap v364: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-03-18T20:47:48.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:48 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:47:48.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:48 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:47:48.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:48 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:47:48.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 445096 3.74 MB/sec execute 523 sec latency 1468.647 ms 2024-03-18T20:47:49.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:49 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:47:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:49 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:47:49.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:49 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:47:49.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:49 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:47:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:49 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:47:49.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:49 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:47:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.556+0000 7fa2d1692700 1 -- 172.21.15.46:0/991895746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2cc1018b0 msgr2=0x7fa2cc1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.556+0000 7fa2d1692700 1 --2- 172.21.15.46:0/991895746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2cc1018b0 0x7fa2cc1058f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009990 tx=0x7fa2bc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.556+0000 7fa2d1692700 1 -- 172.21.15.46:0/991895746 shutdown_connections 2024-03-18T20:47:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.556+0000 7fa2d1692700 1 --2- 172.21.15.46:0/991895746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2cc106020 0x7fa2cc10dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.556+0000 7fa2d1692700 1 --2- 172.21.15.46:0/991895746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2cc100f00 0x7fa2cc1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.556+0000 7fa2d1692700 1 --2- 172.21.15.46:0/991895746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2cc1018b0 0x7fa2cc1058f0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.556+0000 7fa2d1692700 1 -- 172.21.15.46:0/991895746 >> 172.21.15.46:0/991895746 conn(0x7fa2cc0fc7c0 msgr2=0x7fa2cc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.556+0000 7fa2d1692700 1 -- 172.21.15.46:0/991895746 shutdown_connections 2024-03-18T20:47:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.556+0000 7fa2d1692700 1 -- 172.21.15.46:0/991895746 wait complete. 2024-03-18T20:47:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.557+0000 7fa2d1692700 1 Processor -- start 2024-03-18T20:47:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.557+0000 7fa2d1692700 1 -- start start 2024-03-18T20:47:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.557+0000 7fa2d1692700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2cc100f00 0x7fa2cc1971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2d1692700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2cc106020 0x7fa2cc197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2d1692700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2cc19baa0 0x7fa2cc19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2d1692700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc19c520 con 0x7fa2cc19baa0 2024-03-18T20:47:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2d1692700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc19c690 con 0x7fa2cc106020 2024-03-18T20:47:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2d1692700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc19c850 con 0x7fa2cc100f00 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2cb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2cc19baa0 0x7fa2cc19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2cb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2cc19baa0 0x7fa2cc19bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48984/0 (socket says 172.21.15.46:48984) 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2cb7fe700 1 -- 172.21.15.46:0/443629133 learned_addr learned my addr 172.21.15.46:0/443629133 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2caffd700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2cc100f00 0x7fa2cc1971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2ca7fc700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2cc106020 0x7fa2cc197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2cb7fe700 1 -- 172.21.15.46:0/443629133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2cc100f00 msgr2=0x7fa2cc1971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2cb7fe700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2cc100f00 0x7fa2cc1971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2cb7fe700 1 -- 172.21.15.46:0/443629133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2cc106020 msgr2=0x7fa2cc197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2cb7fe700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2cc106020 0x7fa2cc197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2cb7fe700 1 -- 172.21.15.46:0/443629133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc0095f0 con 0x7fa2cc19baa0 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2cb7fe700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2cc19baa0 0x7fa2cc19bf20 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400ef30 tx=0x7fa2c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2c3fff700 1 -- 172.21.15.46:0/443629133 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4018070 con 0x7fa2cc19baa0 2024-03-18T20:47:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2c3fff700 1 -- 172.21.15.46:0/443629133 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2c400f040 con 0x7fa2cc19baa0 2024-03-18T20:47:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.558+0000 7fa2d1692700 1 -- 172.21.15.46:0/443629133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc1a3d50 con 0x7fa2cc19baa0 2024-03-18T20:47:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.559+0000 7fa2c3fff700 1 -- 172.21.15.46:0/443629133 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c400cd00 con 0x7fa2cc19baa0 2024-03-18T20:47:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.559+0000 7fa2d1692700 1 -- 172.21.15.46:0/443629133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc1a41e0 con 0x7fa2cc19baa0 2024-03-18T20:47:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.561+0000 7fa2c3fff700 1 -- 172.21.15.46:0/443629133 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa2c4003a00 con 0x7fa2cc19baa0 2024-03-18T20:47:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.562+0000 7fa2c3fff700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2b80a1290 0x7fa2b80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.562+0000 7fa2c3fff700 1 -- 172.21.15.46:0/443629133 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa2c40cc850 con 0x7fa2cc19baa0 2024-03-18T20:47:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.562+0000 7fa2caffd700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2b80a1290 0x7fa2b80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.562+0000 7fa2caffd700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2b80a1290 0x7fa2b80a3750 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4005950 tx=0x7fa2b400b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.562+0000 7fa2dd827b80 1 Processor -- start 2024-03-18T20:47:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2dd827b80 1 -- start start 2024-03-18T20:47:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2dd827b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce8f570 0x55e96ce8f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2dd827b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8fe90 0x55e96ce9a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2dd827b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce9adb0 0x55e96ce9d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2dd827b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96cde9930 con 0x55e96ce8fe90 2024-03-18T20:47:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2dd827b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96cde72f0 con 0x55e96ce8f570 2024-03-18T20:47:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2dd827b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96cde2be0 con 0x55e96ce9adb0 2024-03-18T20:47:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2ca7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8fe90 0x55e96ce9a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2ca7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8fe90 0x55e96ce9a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48996/0 (socket says 172.21.15.46:48996) 2024-03-18T20:47:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2ca7fc700 1 -- 172.21.15.46:0/405982139 learned_addr learned my addr 172.21.15.46:0/405982139 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2cb7fe700 1 --2- 172.21.15.46:0/405982139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce9adb0 0x55e96ce9d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2caffd700 1 --2- 172.21.15.46:0/405982139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce8f570 0x55e96ce8f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2ca7fc700 1 -- 172.21.15.46:0/405982139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce9adb0 msgr2=0x55e96ce9d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2ca7fc700 1 --2- 172.21.15.46:0/405982139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce9adb0 0x55e96ce9d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2ca7fc700 1 -- 172.21.15.46:0/405982139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce8f570 msgr2=0x55e96ce8f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2ca7fc700 1 --2- 172.21.15.46:0/405982139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce8f570 0x55e96ce8f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.563+0000 7fa2ca7fc700 1 -- 172.21.15.46:0/405982139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e96cc64350 con 0x55e96ce8fe90 2024-03-18T20:47:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.564+0000 7fa2caffd700 1 --2- 172.21.15.46:0/405982139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce8f570 0x55e96ce8f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:47:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.564+0000 7fa2ca7fc700 1 --2- 172.21.15.46:0/405982139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8fe90 0x55e96ce9a870 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009990 tx=0x7fa2bc005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad1b2e6cf2dbf30 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.564+0000 7fa2c17fa700 1 -- 172.21.15.46:0/405982139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc01d070 con 0x55e96ce8fe90 2024-03-18T20:47:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.564+0000 7fa2c17fa700 1 -- 172.21.15.46:0/405982139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2bc01e040 con 0x55e96ce8fe90 2024-03-18T20:47:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.564+0000 7fa2c17fa700 1 -- 172.21.15.46:0/405982139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc004860 con 0x55e96ce8fe90 2024-03-18T20:47:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.565+0000 7fa2dd827b80 1 -- 172.21.15.46:0/405982139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8fe90 msgr2=0x55e96ce9a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:49.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.565+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/405982139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8fe90 0x55e96ce9a870 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009990 tx=0x7fa2bc005bc0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.565+0000 7fa2dd827b80 1 -- 172.21.15.46:0/405982139 shutdown_connections 2024-03-18T20:47:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.565+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/405982139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce9adb0 0x55e96ce9d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.565+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/405982139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce8f570 0x55e96ce8f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.565+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/405982139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8fe90 0x55e96ce9a870 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.565+0000 7fa2dd827b80 1 -- 172.21.15.46:0/405982139 >> 172.21.15.46:0/405982139 conn(0x55e96ce061c0 msgr2=0x55e96ce8ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.565+0000 7fa2dd827b80 1 -- 172.21.15.46:0/405982139 shutdown_connections 2024-03-18T20:47:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.565+0000 7fa2dd827b80 1 -- 172.21.15.46:0/405982139 wait complete. 2024-03-18T20:47:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.565+0000 7fa2dd827b80 1 -- start start 2024-03-18T20:47:49.575 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.565+0000 7fa2dd827b80 1 Processor -- start 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2dd827b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce9a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2dd827b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce8fe90 0x55e96ce97ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2dd827b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce9adb0 0x55e96ce98210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2dd827b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96cd7ed20 con 0x55e96ce8f570 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2dd827b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96cd7e9c0 con 0x55e96ce9adb0 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2dd827b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96cc66640 con 0x55e96ce8fe90 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2dd827b80 10 client.0 _ll_register_callbacks cb 0x55e96cbea340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2caffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce9a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2caffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce9a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49008/0 (socket says 172.21.15.46:49008) 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2caffd700 1 -- 172.21.15.46:0/2804701505 learned_addr learned my addr 172.21.15.46:0/2804701505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2ca7fc700 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce8fe90 0x55e96ce97ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2cb7fe700 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce9adb0 0x55e96ce98210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2caffd700 1 -- 172.21.15.46:0/2804701505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce8fe90 msgr2=0x55e96ce97ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:49.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2caffd700 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce8fe90 0x55e96ce97ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2caffd700 1 -- 172.21.15.46:0/2804701505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce9adb0 msgr2=0x55e96ce98210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2caffd700 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce9adb0 0x55e96ce98210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2caffd700 1 -- 172.21.15.46:0/2804701505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc10e410 con 0x55e96ce8f570 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2cb7fe700 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce9adb0 0x55e96ce98210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2caffd700 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce9a7e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4011770 tx=0x7fa2b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49130657dbfbad7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.566+0000 7fa2b27fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2b27fc700 1 -- 172.21.15.46:0/2804701505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4004a30 con 0x55e96ce8f570 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2b27fc700 1 -- 172.21.15.46:0/2804701505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa2b4017070 con 0x55e96ce8f570 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2804701505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e96ce836c0 con 0x55e96ce8f570 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2b27fc700 1 -- 172.21.15.46:0/2804701505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2b4010ea0 con 0x55e96ce8f570 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2dd827b80 10 client.19644 fetch_fsmap learned FSMap version 51 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2804701505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55e96cc10b00 con 0x55e96ce8f570 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2b27fc700 1 -- 172.21.15.46:0/2804701505 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e96ce836c0 con 0x55e96ce8f570 2024-03-18T20:47:49.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2b27fc700 1 -- 172.21.15.46:0/2804701505 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa2b401f760 con 0x55e96ce8f570 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2dd827b80 10 client.19644 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2dd827b80 10 client.19644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2dd827b80 20 client.19644 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e96ceaa990 0x55e96ceacdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2dd827b80 4 client.19644 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.567+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2804701505 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e96cddb6a0 con 0x55e96ceaa990 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.568+0000 7fa2ca7fc700 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e96ceaa990 0x55e96ceacdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.568+0000 7fa2ca7fc700 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e96ceaa990 0x55e96ceacdb0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ca77b6cdc94d054 server_cookie=6c3eb4c360a308f in_seq=0 out_seq=0 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.568+0000 7fa2b27fc700 10 client.19644 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2b27fc700 1 -- 172.21.15.46:0/2804701505 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e96cddb6a0 con 0x55e96ceaa990 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2b27fc700 10 client.19644 handle_command_reply: tid=0 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2dd827b80 1 client.19644 shutdown 2024-03-18T20:47:49.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2804701505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 msgr2=0x55e96ce9a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce9a7e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4011770 tx=0x7fa2b40047c0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2804701505 shutdown_connections 2024-03-18T20:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e96ceaa990 0x55e96ceacdb0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce9a7e0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce9adb0 0x55e96ce98210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/2804701505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce8fe90 0x55e96ce97ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2804701505 >> 172.21.15.46:0/2804701505 conn(0x55e96ce061c0 msgr2=0x55e96ce8b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2b27fc700 0 client.19644 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2804701505 shutdown_connections 2024-03-18T20:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2804701505 wait complete. 2024-03-18T20:47:49.579 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.569+0000 7fa2dd827b80 20 client.19644 trim_cache size 0 max 16384 2024-03-18T20:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2dd827b80 1 Processor -- start 2024-03-18T20:47:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2dd827b80 1 -- start start 2024-03-18T20:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2dd827b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2dd827b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce8fe90 0x55e96cea19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2dd827b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce9adb0 0x55e96ce92e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2dd827b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96ce99080 con 0x55e96ce8f570 2024-03-18T20:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2dd827b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96cea1f20 con 0x55e96ce8fe90 2024-03-18T20:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2dd827b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96ce83200 con 0x55e96ce9adb0 2024-03-18T20:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2caffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2caffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce924e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49020/0 (socket says 172.21.15.46:49020) 2024-03-18T20:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2caffd700 1 -- 172.21.15.46:0/2270281794 learned_addr learned my addr 172.21.15.46:0/2270281794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:49.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2ca7fc700 1 --2- 172.21.15.46:0/2270281794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce8fe90 0x55e96cea19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.585+0000 7fa2cb7fe700 1 --2- 172.21.15.46:0/2270281794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce9adb0 0x55e96ce92e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.586+0000 7fa2caffd700 1 -- 172.21.15.46:0/2270281794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce9adb0 msgr2=0x55e96ce92e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.586+0000 7fa2caffd700 1 --2- 172.21.15.46:0/2270281794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce9adb0 0x55e96ce92e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.586+0000 7fa2caffd700 1 -- 172.21.15.46:0/2270281794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce8fe90 msgr2=0x55e96cea19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.586+0000 7fa2caffd700 1 --2- 172.21.15.46:0/2270281794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce8fe90 0x55e96cea19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.586+0000 7fa2caffd700 1 -- 172.21.15.46:0/2270281794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e96cda8e40 con 0x55e96ce8f570 2024-03-18T20:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.586+0000 7fa2caffd700 1 --2- 172.21.15.46:0/2270281794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce924e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400b5a0 tx=0x7fa2b400b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b7ccfd430a0e4d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.586+0000 7fa2c1ffb700 1 -- 172.21.15.46:0/2270281794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b401f640 con 0x55e96ce8f570 2024-03-18T20:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.586+0000 7fa2c1ffb700 1 -- 172.21.15.46:0/2270281794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2b4015070 con 0x55e96ce8f570 2024-03-18T20:47:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.586+0000 7fa2c1ffb700 1 -- 172.21.15.46:0/2270281794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4005380 con 0x55e96ce8f570 2024-03-18T20:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2270281794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 msgr2=0x55e96ce924e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/2270281794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce924e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400b5a0 tx=0x7fa2b400b970 comp rx=0 tx=0).stop 2024-03-18T20:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2270281794 shutdown_connections 2024-03-18T20:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/2270281794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce924e0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/2270281794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce9adb0 0x55e96ce92e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/2270281794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce8fe90 0x55e96cea19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2270281794 >> 172.21.15.46:0/2270281794 conn(0x55e96ce061c0 msgr2=0x55e96cea6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2270281794 shutdown_connections 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 -- 172.21.15.46:0/2270281794 wait complete. 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 -- start start 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 Processor -- start 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce93d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce8fe90 0x55e96ce942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce9adb0 0x55e96ce947e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96ce94d20 con 0x55e96ce8f570 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96ce94e90 con 0x55e96ce9adb0 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.587+0000 7fa2dd827b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96ce95050 con 0x55e96ce8fe90 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2dd827b80 10 client.0 _ll_register_callbacks cb 0x7fa2b4010d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2caffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce93d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2caffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce93d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49022/0 (socket says 172.21.15.46:49022) 2024-03-18T20:47:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2caffd700 1 -- 172.21.15.46:0/4070031528 learned_addr learned my addr 172.21.15.46:0/4070031528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2cb7fe700 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce9adb0 0x55e96ce947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2ca7fc700 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce8fe90 0x55e96ce942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2cb7fe700 1 -- 172.21.15.46:0/4070031528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce8fe90 msgr2=0x55e96ce942a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2cb7fe700 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce8fe90 0x55e96ce942a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2cb7fe700 1 -- 172.21.15.46:0/4070031528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 msgr2=0x55e96ce93d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2cb7fe700 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce93d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2cb7fe700 1 -- 172.21.15.46:0/4070031528 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e96cc10b00 con 0x55e96ce9adb0 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2caffd700 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce93d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2cb7fe700 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce9adb0 0x55e96ce947e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40c9f20 tx=0x7fa2c4099850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21bd9a6a46d3868c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2b27fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2b27fc700 1 -- 172.21.15.46:0/4070031528 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c401b320 con 0x55e96ce9adb0 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2dd827b80 1 -- 172.21.15.46:0/4070031528 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2bc01e040 con 0x55e96ce9adb0 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2b27fc700 1 -- 172.21.15.46:0/4070031528 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa2c409a070 con 0x55e96ce9adb0 2024-03-18T20:47:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.588+0000 7fa2b27fc700 1 -- 172.21.15.46:0/4070031528 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2c400ce60 con 0x55e96ce9adb0 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.589+0000 7fa2dd827b80 10 client.29264 fetch_fsmap learned FSMap version 51 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.589+0000 7fa2dd827b80 1 -- 172.21.15.46:0/4070031528 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55e96cc68c70 con 0x55e96ce9adb0 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.589+0000 7fa2b27fc700 1 -- 172.21.15.46:0/4070031528 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2bc01e040 con 0x55e96ce9adb0 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.589+0000 7fa2b27fc700 1 -- 172.21.15.46:0/4070031528 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa2c4098e60 con 0x55e96ce9adb0 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.589+0000 7fa2dd827b80 10 client.29264 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.589+0000 7fa2dd827b80 10 client.29264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.589+0000 7fa2dd827b80 20 client.29264 populate_metadata read hostname 'smithi046' 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.589+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e96ceff160 0x55e96cf01550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.589+0000 7fa2dd827b80 4 client.29264 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.589+0000 7fa2dd827b80 1 -- 172.21.15.46:0/4070031528 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e96cc31c90 con 0x55e96ceff160 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.589+0000 7fa2caffd700 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e96ceff160 0x55e96cf01550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.590+0000 7fa2b27fc700 10 client.29264 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.590+0000 7fa2caffd700 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e96ceff160 0x55e96cf01550 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46a646638249172 server_cookie=1bf0620b95a8d9c8 in_seq=0 out_seq=0 2024-03-18T20:47:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2b27fc700 1 -- 172.21.15.46:0/4070031528 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e96cc31c90 con 0x55e96ceff160 2024-03-18T20:47:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2b27fc700 10 client.29264 handle_command_reply: tid=0 2024-03-18T20:47:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2dd827b80 1 client.29264 shutdown 2024-03-18T20:47:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2dd827b80 1 -- 172.21.15.46:0/4070031528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce9adb0 msgr2=0x55e96ce947e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce9adb0 0x55e96ce947e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40c9f20 tx=0x7fa2c4099850 comp rx=0 tx=0).stop 2024-03-18T20:47:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2dd827b80 1 -- 172.21.15.46:0/4070031528 shutdown_connections 2024-03-18T20:47:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e96ceff160 0x55e96cf01550 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e96ce8f570 0x55e96ce93d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e96ce9adb0 0x55e96ce947e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2dd827b80 1 --2- 172.21.15.46:0/4070031528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e96ce8fe90 0x55e96ce942a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2b27fc700 0 client.29264 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:47:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2dd827b80 1 -- 172.21.15.46:0/4070031528 >> 172.21.15.46:0/4070031528 conn(0x55e96ce061c0 msgr2=0x55e96ce92e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2dd827b80 1 -- 172.21.15.46:0/4070031528 shutdown_connections 2024-03-18T20:47:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2dd827b80 1 -- 172.21.15.46:0/4070031528 wait complete. 2024-03-18T20:47:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.620+0000 7fa2dd827b80 20 client.29264 trim_cache size 0 max 16384 2024-03-18T20:47:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:47:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:47:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:47:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:47:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.621+0000 7fa2d1692700 1 -- 172.21.15.46:0/443629133 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2b80a1290 msgr2=0x7fa2b80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.621+0000 7fa2d1692700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2b80a1290 0x7fa2b80a3750 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4005950 tx=0x7fa2b400b410 comp rx=0 tx=0).stop 2024-03-18T20:47:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.621+0000 7fa2d1692700 1 -- 172.21.15.46:0/443629133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2cc19baa0 msgr2=0x7fa2cc19bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:47:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.621+0000 7fa2d1692700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2cc19baa0 0x7fa2cc19bf20 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400ef30 tx=0x7fa2c400c3a0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.621+0000 7fa2d1692700 1 -- 172.21.15.46:0/443629133 shutdown_connections 2024-03-18T20:47:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.621+0000 7fa2d1692700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa2b80a1290 0x7fa2b80a3750 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.621+0000 7fa2d1692700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2cc19baa0 0x7fa2cc19bf20 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.621+0000 7fa2d1692700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2cc100f00 0x7fa2cc1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.621+0000 7fa2d1692700 1 --2- 172.21.15.46:0/443629133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2cc106020 0x7fa2cc197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:47:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.621+0000 7fa2d1692700 1 -- 172.21.15.46:0/443629133 >> 172.21.15.46:0/443629133 conn(0x7fa2cc0fc7c0 msgr2=0x7fa2cc0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:47:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.621+0000 7fa2d1692700 1 -- 172.21.15.46:0/443629133 shutdown_connections 2024-03-18T20:47:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:47:49.621+0000 7fa2d1692700 1 -- 172.21.15.46:0/443629133 wait complete. 2024-03-18T20:47:49.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:49 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:47:49.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:49 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:47:49.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:49 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:47:49.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 445096 3.73 MB/sec execute 524 sec latency 2468.743 ms 2024-03-18T20:47:50.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:50 smithi155 ceph-mon[48344]: pgmap v365: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-03-18T20:47:50.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:50 smithi046 ceph-mon[43495]: pgmap v365: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-03-18T20:47:50.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:50 smithi073 ceph-mon[47384]: pgmap v365: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 220 op/s 2024-03-18T20:47:50.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 445895 3.73 MB/sec execute 525 sec latency 3074.441 ms 2024-03-18T20:47:51.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 447365 3.74 MB/sec execute 526 sec latency 20.930 ms 2024-03-18T20:47:52.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:52 smithi155 ceph-mon[48344]: pgmap v366: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-03-18T20:47:52.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:52 smithi046 ceph-mon[43495]: pgmap v366: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-03-18T20:47:52.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:52 smithi073 ceph-mon[47384]: pgmap v366: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-03-18T20:47:52.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 448638 3.75 MB/sec execute 527 sec latency 110.175 ms 2024-03-18T20:47:53.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 448638 3.74 MB/sec execute 528 sec latency 1110.285 ms 2024-03-18T20:47:54.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:54 smithi155 ceph-mon[48344]: pgmap v367: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 219 op/s 2024-03-18T20:47:54.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:54 smithi046 ceph-mon[43495]: pgmap v367: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 219 op/s 2024-03-18T20:47:54.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:54 smithi073 ceph-mon[47384]: pgmap v367: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 219 op/s 2024-03-18T20:47:54.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 448638 3.73 MB/sec execute 529 sec latency 2110.340 ms 2024-03-18T20:47:55.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 449273 3.73 MB/sec execute 530 sec latency 2716.602 ms 2024-03-18T20:47:56.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:56 smithi155 ceph-mon[48344]: pgmap v368: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:47:56.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:56 smithi046 ceph-mon[43495]: pgmap v368: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:47:56.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:56 smithi073 ceph-mon[47384]: pgmap v368: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:47:56.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 450832 3.74 MB/sec execute 531 sec latency 17.731 ms 2024-03-18T20:47:57.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 452135 3.75 MB/sec execute 532 sec latency 47.697 ms 2024-03-18T20:47:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:47:58 smithi155 ceph-mon[48344]: pgmap v369: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:47:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:47:58 smithi046 ceph-mon[43495]: pgmap v369: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:47:58.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:47:58 smithi073 ceph-mon[47384]: pgmap v369: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T20:47:58.720 INFO:tasks.workunit.client.0.smithi046.stdout: 1 452187 3.74 MB/sec execute 533 sec latency 955.795 ms 2024-03-18T20:47:59.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 453373 3.74 MB/sec execute 534 sec latency 1422.271 ms 2024-03-18T20:48:00.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:00 smithi155 ceph-mon[48344]: pgmap v370: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-03-18T20:48:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:00 smithi046 ceph-mon[43495]: pgmap v370: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-03-18T20:48:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:00 smithi073 ceph-mon[47384]: pgmap v370: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-03-18T20:48:00.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 454300 3.75 MB/sec execute 535 sec latency 429.748 ms 2024-03-18T20:48:01.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 454300 3.74 MB/sec execute 536 sec latency 1429.862 ms 2024-03-18T20:48:02.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:02 smithi155 ceph-mon[48344]: pgmap v371: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-03-18T20:48:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:02 smithi046 ceph-mon[43495]: pgmap v371: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-03-18T20:48:02.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:02 smithi073 ceph-mon[47384]: pgmap v371: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-03-18T20:48:02.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 454921 3.74 MB/sec execute 537 sec latency 1766.015 ms 2024-03-18T20:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.762+0000 7f808e615700 1 -- 172.21.15.46:0/772736741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088105c60 msgr2=0x7f8088106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.762+0000 7f808e615700 1 --2- 172.21.15.46:0/772736741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088105c60 0x7f8088106040 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f8078009990 tx=0x7f8078009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.762+0000 7f808e615700 1 -- 172.21.15.46:0/772736741 shutdown_connections 2024-03-18T20:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.762+0000 7f808e615700 1 --2- 172.21.15.46:0/772736741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088105c60 0x7f8088106040 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.762+0000 7f808e615700 1 --2- 172.21.15.46:0/772736741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8088100570 0x7f808810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.762+0000 7f808e615700 1 --2- 172.21.15.46:0/772736741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80880ffbb0 0x7f8088100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.762+0000 7f808e615700 1 -- 172.21.15.46:0/772736741 >> 172.21.15.46:0/772736741 conn(0x7f8088076d50 msgr2=0x7f8088077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.762+0000 7f808e615700 1 -- 172.21.15.46:0/772736741 shutdown_connections 2024-03-18T20:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.762+0000 7f808e615700 1 -- 172.21.15.46:0/772736741 wait complete. 2024-03-18T20:48:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.763+0000 7f808e615700 1 Processor -- start 2024-03-18T20:48:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.763+0000 7f808e615700 1 -- start start 2024-03-18T20:48:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f808e615700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80880ffbb0 0x7f8088197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f808e615700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8088100570 0x7f8088198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f808e615700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088105c60 0x7f80881a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f808e615700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8088198cd0 con 0x7f8088105c60 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f808e615700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80881a34e0 con 0x7f8088100570 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f808e615700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80881a3620 con 0x7f80880ffbb0 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f808cbb2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088105c60 0x7f80881a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f8087fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80880ffbb0 0x7f8088197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f808cbb2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088105c60 0x7f80881a2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44304/0 (socket says 172.21.15.46:44304) 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f808cbb2700 1 -- 172.21.15.46:0/1627673124 learned_addr learned my addr 172.21.15.46:0/1627673124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f80877fe700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8088100570 0x7f8088198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f80877fe700 1 -- 172.21.15.46:0/1627673124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80880ffbb0 msgr2=0x7f8088197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f80877fe700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80880ffbb0 0x7f8088197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f80877fe700 1 -- 172.21.15.46:0/1627673124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088105c60 msgr2=0x7f80881a2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f80877fe700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088105c60 0x7f80881a2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f80877fe700 1 -- 172.21.15.46:0/1627673124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c009520 con 0x7f8088100570 2024-03-18T20:48:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f808cbb2700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088105c60 0x7f80881a2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.764+0000 7f80877fe700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8088100570 0x7f8088198460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f807c00e9f0 tx=0x7f807c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.765+0000 7f80857fa700 1 -- 172.21.15.46:0/1627673124 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00cb70 con 0x7f8088100570 2024-03-18T20:48:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.765+0000 7f80857fa700 1 -- 172.21.15.46:0/1627673124 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f807c00ee90 con 0x7f8088100570 2024-03-18T20:48:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.765+0000 7f808e615700 1 -- 172.21.15.46:0/1627673124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80780095f0 con 0x7f8088100570 2024-03-18T20:48:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.765+0000 7f80857fa700 1 -- 172.21.15.46:0/1627673124 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c003cf0 con 0x7f8088100570 2024-03-18T20:48:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.765+0000 7f808e615700 1 -- 172.21.15.46:0/1627673124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80881a3c80 con 0x7f8088100570 2024-03-18T20:48:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.768+0000 7f80857fa700 1 -- 172.21.15.46:0/1627673124 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f807c00ccd0 con 0x7f8088100570 2024-03-18T20:48:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.768+0000 7f80857fa700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f80700a1310 0x7f80700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.769+0000 7f8087fff700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f80700a1310 0x7f80700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.769+0000 7f80857fa700 1 -- 172.21.15.46:0/1627673124 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f807c014070 con 0x7f8088100570 2024-03-18T20:48:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.769+0000 7f8087fff700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f80700a1310 0x7f80700a37d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8078005b00 tx=0x7f8078005a90 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.769+0000 7f809a7aab80 1 Processor -- start 2024-03-18T20:48:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.769+0000 7f809a7aab80 1 -- start start 2024-03-18T20:48:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.769+0000 7f809a7aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffb570 0x55aa86ffb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f809a7aab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffbe90 0x55aa8700ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f809a7aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa8700d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f809a7aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa86f55930 con 0x55aa8700b1c0 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f809a7aab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa86f532f0 con 0x55aa86ffbe90 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f809a7aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa86f4ebe0 con 0x55aa86ffb570 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f8087fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffb570 0x55aa86ffb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f8087fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffb570 0x55aa86ffb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:34850/0 (socket says 172.21.15.46:34850) 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f8087fff700 1 -- 172.21.15.46:0/1316613325 learned_addr learned my addr 172.21.15.46:0/1316613325 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f808cbb2700 1 --2- 172.21.15.46:0/1316613325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa8700d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f80877fe700 1 --2- 172.21.15.46:0/1316613325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffbe90 0x55aa8700ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f8087fff700 1 -- 172.21.15.46:0/1316613325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffbe90 msgr2=0x55aa8700ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f8087fff700 1 --2- 172.21.15.46:0/1316613325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffbe90 0x55aa8700ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f8087fff700 1 -- 172.21.15.46:0/1316613325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 msgr2=0x55aa8700d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f8087fff700 1 --2- 172.21.15.46:0/1316613325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa8700d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f8087fff700 1 -- 172.21.15.46:0/1316613325 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa86dd0350 con 0x55aa86ffb570 2024-03-18T20:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f80877fe700 1 --2- 172.21.15.46:0/1316613325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffbe90 0x55aa8700ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f8087fff700 1 --2- 172.21.15.46:0/1316613325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffb570 0x55aa86ffb950 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8078015000 tx=0x7f8078020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f69a3420b143c3a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.770+0000 7f806e7fc700 1 -- 172.21.15.46:0/1316613325 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807800f660 con 0x55aa86ffb570 2024-03-18T20:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.771+0000 7f806e7fc700 1 -- 172.21.15.46:0/1316613325 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f807801e040 con 0x55aa86ffb570 2024-03-18T20:48:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.771+0000 7f806e7fc700 1 -- 172.21.15.46:0/1316613325 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807802fab0 con 0x55aa86ffb570 2024-03-18T20:48:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.771+0000 7f809a7aab80 1 -- 172.21.15.46:0/1316613325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffb570 msgr2=0x55aa86ffb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.771+0000 7f809a7aab80 1 --2- 172.21.15.46:0/1316613325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffb570 0x55aa86ffb950 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8078015000 tx=0x7f8078020000 comp rx=0 tx=0).stop 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.771+0000 7f809a7aab80 1 -- 172.21.15.46:0/1316613325 shutdown_connections 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.771+0000 7f809a7aab80 1 --2- 172.21.15.46:0/1316613325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa8700d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.771+0000 7f809a7aab80 1 --2- 172.21.15.46:0/1316613325 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffb570 0x55aa86ffb950 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.771+0000 7f809a7aab80 1 --2- 172.21.15.46:0/1316613325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffbe90 0x55aa8700ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.771+0000 7f809a7aab80 1 -- 172.21.15.46:0/1316613325 >> 172.21.15.46:0/1316613325 conn(0x55aa86f721c0 msgr2=0x55aa86ff7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.772+0000 7f809a7aab80 1 -- 172.21.15.46:0/1316613325 shutdown_connections 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.772+0000 7f809a7aab80 1 -- 172.21.15.46:0/1316613325 wait complete. 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.772+0000 7f809a7aab80 1 -- start start 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.772+0000 7f809a7aab80 1 Processor -- start 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f809a7aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa86ffb570 0x55aa8700ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f809a7aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa87007e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f809a7aab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa8700b1c0 0x55aa87008480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f809a7aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa86eead20 con 0x55aa86ffb570 2024-03-18T20:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f809a7aab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa86eea9c0 con 0x55aa8700b1c0 2024-03-18T20:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f809a7aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa86dd2640 con 0x55aa86ffbe90 2024-03-18T20:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f809a7aab80 10 client.0 _ll_register_callbacks cb 0x55aa86d56340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f8087fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa86ffb570 0x55aa8700ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f80877fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa87007e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f8087fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa86ffb570 0x55aa8700ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44328/0 (socket says 172.21.15.46:44328) 2024-03-18T20:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f8087fff700 1 -- 172.21.15.46:0/1139151766 learned_addr learned my addr 172.21.15.46:0/1139151766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f808cbb2700 1 --2- 172.21.15.46:0/1139151766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa8700b1c0 0x55aa87008480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f8087fff700 1 -- 172.21.15.46:0/1139151766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 msgr2=0x55aa87007e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f8087fff700 1 --2- 172.21.15.46:0/1139151766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa87007e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f8087fff700 1 -- 172.21.15.46:0/1139151766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa8700b1c0 msgr2=0x55aa87008480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f8087fff700 1 --2- 172.21.15.46:0/1139151766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa8700b1c0 0x55aa87008480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.773+0000 7f8087fff700 1 -- 172.21.15.46:0/1139151766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808810d2f0 con 0x55aa86ffb570 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f8087fff700 1 --2- 172.21.15.46:0/1139151766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa86ffb570 0x55aa8700ab60 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f8078009350 tx=0x7f807802fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9dba3e7be6036f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f80677fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f80677fe700 1 -- 172.21.15.46:0/1139151766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807800f870 con 0x55aa86ffb570 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f80677fe700 1 -- 172.21.15.46:0/1139151766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f807802f440 con 0x55aa86ffb570 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f809a7aab80 1 -- 172.21.15.46:0/1139151766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa86fef6c0 con 0x55aa86ffb570 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f80677fe700 1 -- 172.21.15.46:0/1139151766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f807800f9d0 con 0x55aa86ffb570 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f809a7aab80 10 client.19662 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f809a7aab80 1 -- 172.21.15.46:0/1139151766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55aa86d7cb00 con 0x55aa86ffb570 2024-03-18T20:48:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f80677fe700 1 -- 172.21.15.46:0/1139151766 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa86fef6c0 con 0x55aa86ffb570 2024-03-18T20:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f80677fe700 1 -- 172.21.15.46:0/1139151766 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8078024410 con 0x55aa86ffb570 2024-03-18T20:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f809a7aab80 10 client.19662 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f809a7aab80 10 client.19662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f809a7aab80 20 client.19662 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f809a7aab80 1 --2- 172.21.15.46:0/1139151766 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55aa8702b870 0x55aa8702dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f809a7aab80 4 client.19662 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.774+0000 7f809a7aab80 1 -- 172.21.15.46:0/1139151766 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa86f476a0 con 0x55aa8702b870 2024-03-18T20:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.775+0000 7f80877fe700 1 --2- 172.21.15.46:0/1139151766 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55aa8702b870 0x55aa8702dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.775+0000 7f80677fe700 10 client.19662 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.775+0000 7f80877fe700 1 --2- 172.21.15.46:0/1139151766 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55aa8702b870 0x55aa8702dc60 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51cb85a34a38a643 server_cookie=caf6387eaa1abfa in_seq=0 out_seq=0 2024-03-18T20:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.775+0000 7f80677fe700 1 -- 172.21.15.46:0/1139151766 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aa86f476a0 con 0x55aa8702b870 2024-03-18T20:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.775+0000 7f80677fe700 10 client.19662 handle_command_reply: tid=0 2024-03-18T20:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.775+0000 7f809a7aab80 1 client.19662 shutdown 2024-03-18T20:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.776+0000 7f809a7aab80 1 -- 172.21.15.46:0/1139151766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa86ffb570 msgr2=0x55aa8700ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.776+0000 7f809a7aab80 1 --2- 172.21.15.46:0/1139151766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa86ffb570 0x55aa8700ab60 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f8078009350 tx=0x7f807802fd90 comp rx=0 tx=0).stop 2024-03-18T20:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.776+0000 7f809a7aab80 1 -- 172.21.15.46:0/1139151766 shutdown_connections 2024-03-18T20:48:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.776+0000 7f809a7aab80 1 --2- 172.21.15.46:0/1139151766 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55aa8702b870 0x55aa8702dc60 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.776+0000 7f809a7aab80 1 --2- 172.21.15.46:0/1139151766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa86ffb570 0x55aa8700ab60 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.776+0000 7f809a7aab80 1 --2- 172.21.15.46:0/1139151766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa8700b1c0 0x55aa87008480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.776+0000 7f809a7aab80 1 --2- 172.21.15.46:0/1139151766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa87007e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.776+0000 7f80677fe700 0 client.19662 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.776+0000 7f809a7aab80 1 -- 172.21.15.46:0/1139151766 >> 172.21.15.46:0/1139151766 conn(0x55aa86f721c0 msgr2=0x55aa86ff7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.776+0000 7f809a7aab80 1 -- 172.21.15.46:0/1139151766 shutdown_connections 2024-03-18T20:48:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.776+0000 7f809a7aab80 1 -- 172.21.15.46:0/1139151766 wait complete. 2024-03-18T20:48:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.776+0000 7f809a7aab80 20 client.19662 trim_cache size 0 max 16384 2024-03-18T20:48:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f809a7aab80 1 Processor -- start 2024-03-18T20:48:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f809a7aab80 1 -- start start 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f809a7aab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 0x55aa8700d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f809a7aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa87003120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f809a7aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa87003660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f809a7aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa86ff8760 con 0x55aa8700b1c0 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f809a7aab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa86ff8550 con 0x55aa86ffb570 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f809a7aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa87002c20 con 0x55aa86ffbe90 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f808cbb2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa87003660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f8087fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 0x55aa8700d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f80877fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa87003120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f8087fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 0x55aa8700d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49138/0 (socket says 172.21.15.46:49138) 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f808cbb2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa87003660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44330/0 (socket says 172.21.15.46:44330) 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f80877fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa87003120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:34882/0 (socket says 172.21.15.46:34882) 2024-03-18T20:48:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f8087fff700 1 -- 172.21.15.46:0/3530117737 learned_addr learned my addr 172.21.15.46:0/3530117737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f8087fff700 1 -- 172.21.15.46:0/3530117737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 msgr2=0x55aa87003120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f8087fff700 1 --2- 172.21.15.46:0/3530117737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa87003120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f8087fff700 1 -- 172.21.15.46:0/3530117737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 msgr2=0x55aa87003660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f8087fff700 1 --2- 172.21.15.46:0/3530117737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa87003660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.793+0000 7f8087fff700 1 -- 172.21.15.46:0/3530117737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa86f14e40 con 0x55aa86ffb570 2024-03-18T20:48:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.794+0000 7f8087fff700 1 --2- 172.21.15.46:0/3530117737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 0x55aa8700d5b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f807800b6f0 tx=0x7f807800b280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df4739f97d44e423 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.794+0000 7f806effd700 1 -- 172.21.15.46:0/3530117737 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80780383a0 con 0x55aa86ffb570 2024-03-18T20:48:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.794+0000 7f806effd700 1 -- 172.21.15.46:0/3530117737 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8078024a40 con 0x55aa86ffb570 2024-03-18T20:48:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.794+0000 7f806effd700 1 -- 172.21.15.46:0/3530117737 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807800fd50 con 0x55aa86ffb570 2024-03-18T20:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.795+0000 7f809a7aab80 1 -- 172.21.15.46:0/3530117737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 msgr2=0x55aa8700d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.795+0000 7f809a7aab80 1 --2- 172.21.15.46:0/3530117737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 0x55aa8700d5b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f807800b6f0 tx=0x7f807800b280 comp rx=0 tx=0).stop 2024-03-18T20:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.795+0000 7f809a7aab80 1 -- 172.21.15.46:0/3530117737 shutdown_connections 2024-03-18T20:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.795+0000 7f809a7aab80 1 --2- 172.21.15.46:0/3530117737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa87003660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.795+0000 7f809a7aab80 1 --2- 172.21.15.46:0/3530117737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 0x55aa8700d5b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.795+0000 7f809a7aab80 1 --2- 172.21.15.46:0/3530117737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa87003120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.795+0000 7f809a7aab80 1 -- 172.21.15.46:0/3530117737 >> 172.21.15.46:0/3530117737 conn(0x55aa86f721c0 msgr2=0x55aa87016540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.795+0000 7f809a7aab80 1 -- 172.21.15.46:0/3530117737 shutdown_connections 2024-03-18T20:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.795+0000 7f809a7aab80 1 -- 172.21.15.46:0/3530117737 wait complete. 2024-03-18T20:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.795+0000 7f809a7aab80 1 -- start start 2024-03-18T20:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.795+0000 7f809a7aab80 1 Processor -- start 2024-03-18T20:48:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f809a7aab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 0x55aa8700f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f809a7aab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa8700fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f809a7aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa87010280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f809a7aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8707e6b0 con 0x55aa8700b1c0 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f809a7aab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8707e820 con 0x55aa86ffb570 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f809a7aab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa8707e9e0 con 0x55aa86ffbe90 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f809a7aab80 10 client.0 _ll_register_callbacks cb 0x7f8080009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f8087fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 0x55aa8700f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f808cbb2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa87010280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f8087fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 0x55aa8700f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49144/0 (socket says 172.21.15.46:49144) 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f8087fff700 1 -- 172.21.15.46:0/48205055 learned_addr learned my addr 172.21.15.46:0/48205055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f80877fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa8700fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f8087fff700 1 -- 172.21.15.46:0/48205055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 msgr2=0x55aa8700fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f8087fff700 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa8700fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f8087fff700 1 -- 172.21.15.46:0/48205055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 msgr2=0x55aa87010280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f8087fff700 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa87010280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f8087fff700 1 -- 172.21.15.46:0/48205055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa86d7cb00 con 0x55aa86ffb570 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f808cbb2700 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa87010280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f80877fe700 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa8700fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f8087fff700 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 0x55aa8700f800 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f80780094f0 tx=0x7f8078024480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f15ed6c94d701f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f80677fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f80677fe700 1 -- 172.21.15.46:0/48205055 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078028070 con 0x55aa86ffb570 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.796+0000 7f80677fe700 1 -- 172.21.15.46:0/48205055 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f807800f870 con 0x55aa86ffb570 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f809a7aab80 1 -- 172.21.15.46:0/48205055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f807801e040 con 0x55aa86ffb570 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f80677fe700 1 -- 172.21.15.46:0/48205055 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f807800fa30 con 0x55aa86ffb570 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f809a7aab80 10 client.29282 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f809a7aab80 1 -- 172.21.15.46:0/48205055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55aa86dd4c70 con 0x55aa86ffb570 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f80677fe700 1 -- 172.21.15.46:0/48205055 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f807801e040 con 0x55aa86ffb570 2024-03-18T20:48:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f80677fe700 1 -- 172.21.15.46:0/48205055 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8078039a00 con 0x55aa86ffb570 2024-03-18T20:48:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f809a7aab80 10 client.29282 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f809a7aab80 10 client.29282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f809a7aab80 20 client.29282 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f809a7aab80 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55aa8707fd40 0x55aa87082160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f809a7aab80 4 client.29282 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:48:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f809a7aab80 1 -- 172.21.15.46:0/48205055 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aa86d9dc90 con 0x55aa8707fd40 2024-03-18T20:48:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.797+0000 7f80877fe700 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55aa8707fd40 0x55aa87082160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.798+0000 7f80677fe700 10 client.29282 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.798+0000 7f80877fe700 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55aa8707fd40 0x55aa87082160 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b148f7e615f51d86 server_cookie=f7b45334e0b7fc8a in_seq=0 out_seq=0 2024-03-18T20:48:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.831+0000 7f80677fe700 1 -- 172.21.15.46:0/48205055 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aa86d9dc90 con 0x55aa8707fd40 2024-03-18T20:48:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.831+0000 7f80677fe700 10 client.29282 handle_command_reply: tid=0 2024-03-18T20:48:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.831+0000 7f809a7aab80 1 client.29282 shutdown 2024-03-18T20:48:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.831+0000 7f809a7aab80 1 -- 172.21.15.46:0/48205055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 msgr2=0x55aa8700f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.831+0000 7f809a7aab80 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 0x55aa8700f800 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f80780094f0 tx=0x7f8078024480 comp rx=0 tx=0).stop 2024-03-18T20:48:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.832+0000 7f809a7aab80 1 -- 172.21.15.46:0/48205055 shutdown_connections 2024-03-18T20:48:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.832+0000 7f809a7aab80 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55aa8707fd40 0x55aa87082160 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.832+0000 7f809a7aab80 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55aa8700b1c0 0x55aa87010280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.832+0000 7f809a7aab80 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aa86ffb570 0x55aa8700f800 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.832+0000 7f80677fe700 0 client.29282 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.832+0000 7f809a7aab80 1 --2- 172.21.15.46:0/48205055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa86ffbe90 0x55aa8700fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.832+0000 7f809a7aab80 1 -- 172.21.15.46:0/48205055 >> 172.21.15.46:0/48205055 conn(0x55aa86f721c0 msgr2=0x55aa87003120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.832+0000 7f809a7aab80 1 -- 172.21.15.46:0/48205055 shutdown_connections 2024-03-18T20:48:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.832+0000 7f809a7aab80 1 -- 172.21.15.46:0/48205055 wait complete. 2024-03-18T20:48:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.832+0000 7f809a7aab80 20 client.29282 trim_cache size 0 max 16384 2024-03-18T20:48:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:48:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:48:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:48:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:48:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.833+0000 7f808e615700 1 -- 172.21.15.46:0/1627673124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f80700a1310 msgr2=0x7f80700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.833+0000 7f808e615700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f80700a1310 0x7f80700a37d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8078005b00 tx=0x7f8078005a90 comp rx=0 tx=0).stop 2024-03-18T20:48:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.833+0000 7f808e615700 1 -- 172.21.15.46:0/1627673124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8088100570 msgr2=0x7f8088198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.833+0000 7f808e615700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8088100570 0x7f8088198460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f807c00e9f0 tx=0x7f807c00ed00 comp rx=0 tx=0).stop 2024-03-18T20:48:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.833+0000 7f808e615700 1 -- 172.21.15.46:0/1627673124 shutdown_connections 2024-03-18T20:48:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.833+0000 7f808e615700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f80700a1310 0x7f80700a37d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.833+0000 7f808e615700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8088105c60 0x7f80881a2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.833+0000 7f808e615700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80880ffbb0 0x7f8088197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.833+0000 7f808e615700 1 --2- 172.21.15.46:0/1627673124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8088100570 0x7f8088198460 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.833+0000 7f808e615700 1 -- 172.21.15.46:0/1627673124 >> 172.21.15.46:0/1627673124 conn(0x7f8088076d50 msgr2=0x7f80880fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.833+0000 7f808e615700 1 -- 172.21.15.46:0/1627673124 shutdown_connections 2024-03-18T20:48:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:02.833+0000 7f808e615700 1 -- 172.21.15.46:0/1627673124 wait complete. 2024-03-18T20:48:03.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:48:03.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:48:03.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.481+0000 7fc3705c8700 1 -- 172.21.15.46:0/90242344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc368100290 msgr2=0x7fc3681059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.481+0000 7fc3705c8700 1 --2- 172.21.15.46:0/90242344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc368100290 0x7fc3681059c0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009990 tx=0x7fc35c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.481+0000 7fc3705c8700 1 -- 172.21.15.46:0/90242344 shutdown_connections 2024-03-18T20:48:03.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.481+0000 7fc3705c8700 1 --2- 172.21.15.46:0/90242344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3681060f0 0x7fc36810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.481+0000 7fc3705c8700 1 --2- 172.21.15.46:0/90242344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3680ff8e0 0x7fc3680ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.481+0000 7fc3705c8700 1 --2- 172.21.15.46:0/90242344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc368100290 0x7fc3681059c0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.481+0000 7fc3705c8700 1 -- 172.21.15.46:0/90242344 >> 172.21.15.46:0/90242344 conn(0x7fc368074f00 msgr2=0x7fc368075310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:03.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.481+0000 7fc3705c8700 1 -- 172.21.15.46:0/90242344 shutdown_connections 2024-03-18T20:48:03.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.481+0000 7fc3705c8700 1 -- 172.21.15.46:0/90242344 wait complete. 2024-03-18T20:48:03.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc3705c8700 1 Processor -- start 2024-03-18T20:48:03.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc3705c8700 1 -- start start 2024-03-18T20:48:03.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc3705c8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3680ff8e0 0x7fc368197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc3705c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3681060f0 0x7fc3681977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc3705c8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc36819bb60 0x7fc36819bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc3705c8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36819c5e0 con 0x7fc3681060f0 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc3705c8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36819c750 con 0x7fc3680ff8e0 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc3705c8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36819c910 con 0x7fc36819bb60 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc36e364700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3680ff8e0 0x7fc368197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc36db63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3681060f0 0x7fc3681977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc36e364700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3680ff8e0 0x7fc368197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49160/0 (socket says 172.21.15.46:49160) 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc36eb65700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc36819bb60 0x7fc36819bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.482+0000 7fc36e364700 1 -- 172.21.15.46:0/2366734173 learned_addr learned my addr 172.21.15.46:0/2366734173 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.483+0000 7fc36e364700 1 -- 172.21.15.46:0/2366734173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc36819bb60 msgr2=0x7fc36819bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.483+0000 7fc36e364700 1 --2- 172.21.15.46:0/2366734173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc36819bb60 0x7fc36819bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.483+0000 7fc36e364700 1 -- 172.21.15.46:0/2366734173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3681060f0 msgr2=0x7fc3681977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.483+0000 7fc36e364700 1 --2- 172.21.15.46:0/2366734173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3681060f0 0x7fc3681977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.483+0000 7fc36e364700 1 -- 172.21.15.46:0/2366734173 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc358009c40 con 0x7fc3680ff8e0 2024-03-18T20:48:03.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.483+0000 7fc36e364700 1 --2- 172.21.15.46:0/2366734173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3680ff8e0 0x7fc368197280 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc35800c370 tx=0x7fc35800c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:03.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.483+0000 7fc3677fe700 1 -- 172.21.15.46:0/2366734173 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358017070 con 0x7fc3680ff8e0 2024-03-18T20:48:03.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.483+0000 7fc3677fe700 1 -- 172.21.15.46:0/2366734173 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35800f040 con 0x7fc3680ff8e0 2024-03-18T20:48:03.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.483+0000 7fc3705c8700 1 -- 172.21.15.46:0/2366734173 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c0095f0 con 0x7fc3680ff8e0 2024-03-18T20:48:03.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.483+0000 7fc3677fe700 1 -- 172.21.15.46:0/2366734173 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35800e050 con 0x7fc3680ff8e0 2024-03-18T20:48:03.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.483+0000 7fc3705c8700 1 -- 172.21.15.46:0/2366734173 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc368100b50 con 0x7fc3680ff8e0 2024-03-18T20:48:03.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.485+0000 7fc3677fe700 1 -- 172.21.15.46:0/2366734173 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc358004440 con 0x7fc3680ff8e0 2024-03-18T20:48:03.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.486+0000 7fc3677fe700 1 --2- 172.21.15.46:0/2366734173 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc3500a1310 0x7fc3500a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.486+0000 7fc36db63700 1 --2- 172.21.15.46:0/2366734173 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc3500a1310 0x7fc3500a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.486+0000 7fc3677fe700 1 -- 172.21.15.46:0/2366734173 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fc3580cc6a0 con 0x7fc3680ff8e0 2024-03-18T20:48:03.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.487+0000 7fc36db63700 1 --2- 172.21.15.46:0/2366734173 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc3500a1310 0x7fc3500a37d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3681987d0 tx=0x7fc35c00b260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:03.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.487+0000 7fc37c75db80 1 Processor -- start 2024-03-18T20:48:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc37c75db80 1 -- start start 2024-03-18T20:48:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc37c75db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003ea60 0x555fd003ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc37c75db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003f380 0x555fd0049ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc37c75db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd004c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc37c75db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcff98990 con 0x555fd003f380 2024-03-18T20:48:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc37c75db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcff96350 con 0x555fd003ea60 2024-03-18T20:48:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc37c75db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcff91c40 con 0x555fd004a220 2024-03-18T20:48:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc36e364700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003ea60 0x555fd003ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc36db63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003f380 0x555fd0049ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc36eb65700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd004c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc36e364700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003ea60 0x555fd003ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49174/0 (socket says 172.21.15.46:49174) 2024-03-18T20:48:03.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc36eb65700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd004c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:34908/0 (socket says 172.21.15.46:34908) 2024-03-18T20:48:03.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.488+0000 7fc36e364700 1 -- 172.21.15.46:0/1904701136 learned_addr learned my addr 172.21.15.46:0/1904701136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:03.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.489+0000 7fc36db63700 1 -- 172.21.15.46:0/1904701136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 msgr2=0x555fd004c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:03.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.489+0000 7fc36db63700 1 --2- 172.21.15.46:0/1904701136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd004c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.489+0000 7fc36db63700 1 -- 172.21.15.46:0/1904701136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003ea60 msgr2=0x555fd003ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.489+0000 7fc36db63700 1 --2- 172.21.15.46:0/1904701136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003ea60 0x555fd003ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.489+0000 7fc36db63700 1 -- 172.21.15.46:0/1904701136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fcfe133b0 con 0x555fd003f380 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.489+0000 7fc36e364700 1 --2- 172.21.15.46:0/1904701136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003ea60 0x555fd003ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.489+0000 7fc36db63700 1 --2- 172.21.15.46:0/1904701136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003f380 0x555fd0049ce0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0098e0 tx=0x7fc35c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa1c96eaedbdf70 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.489+0000 7fc364ff9700 1 -- 172.21.15.46:0/1904701136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c005640 con 0x555fd003f380 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.489+0000 7fc364ff9700 1 -- 172.21.15.46:0/1904701136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c01e040 con 0x555fd003f380 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.489+0000 7fc364ff9700 1 -- 172.21.15.46:0/1904701136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00f7c0 con 0x555fd003f380 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.490+0000 7fc37c75db80 1 -- 172.21.15.46:0/1904701136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003f380 msgr2=0x555fd0049ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.490+0000 7fc37c75db80 1 --2- 172.21.15.46:0/1904701136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003f380 0x555fd0049ce0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0098e0 tx=0x7fc35c020000 comp rx=0 tx=0).stop 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.490+0000 7fc37c75db80 1 -- 172.21.15.46:0/1904701136 shutdown_connections 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.490+0000 7fc37c75db80 1 --2- 172.21.15.46:0/1904701136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd004c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.490+0000 7fc37c75db80 1 --2- 172.21.15.46:0/1904701136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003ea60 0x555fd003ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.490+0000 7fc37c75db80 1 --2- 172.21.15.46:0/1904701136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003f380 0x555fd0049ce0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.490+0000 7fc37c75db80 1 -- 172.21.15.46:0/1904701136 >> 172.21.15.46:0/1904701136 conn(0x555fcffb5220 msgr2=0x555fd003b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:03.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.491+0000 7fc37c75db80 1 -- 172.21.15.46:0/1904701136 shutdown_connections 2024-03-18T20:48:03.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.491+0000 7fc37c75db80 1 -- 172.21.15.46:0/1904701136 wait complete. 2024-03-18T20:48:03.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.491+0000 7fc37c75db80 1 -- start start 2024-03-18T20:48:03.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.491+0000 7fc37c75db80 1 Processor -- start 2024-03-18T20:48:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.491+0000 7fc37c75db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd0046fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.491+0000 7fc37c75db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd003f380 0x555fd00474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.491+0000 7fc37c75db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd004a220 0x555fd0047a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.491+0000 7fc37c75db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcff2dd80 con 0x555fd003ea60 2024-03-18T20:48:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.491+0000 7fc37c75db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcff2da20 con 0x555fd004a220 2024-03-18T20:48:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.491+0000 7fc37c75db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcfe156a0 con 0x555fd003f380 2024-03-18T20:48:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.491+0000 7fc37c75db80 10 client.0 _ll_register_callbacks cb 0x555fcfd993a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:03.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36e364700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd0046fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36eb65700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd004a220 0x555fd0047a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36db63700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd003f380 0x555fd00474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36eb65700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd004a220 0x555fd0047a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49182/0 (socket says 172.21.15.46:49182) 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36e364700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd0046fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44378/0 (socket says 172.21.15.46:44378) 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36eb65700 1 -- 172.21.15.46:0/2139908576 learned_addr learned my addr 172.21.15.46:0/2139908576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36db63700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd003f380 0x555fd00474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:34912/0 (socket says 172.21.15.46:34912) 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36db63700 1 -- 172.21.15.46:0/2139908576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd004a220 msgr2=0x555fd0047a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36db63700 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd004a220 0x555fd0047a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36db63700 1 -- 172.21.15.46:0/2139908576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 msgr2=0x555fd0046fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36db63700 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd0046fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36db63700 1 -- 172.21.15.46:0/2139908576 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc368100b50 con 0x555fd003f380 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36e364700 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd0046fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36eb65700 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd004a220 0x555fd0047a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:03.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc36db63700 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd003f380 0x555fd00474f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc35c032040 tx=0x7fc35c00b3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aa6ac991e80de46 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc34dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.492+0000 7fc34dffb700 1 -- 172.21.15.46:0/2139908576 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c004700 con 0x555fd003f380 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc34dffb700 1 -- 172.21.15.46:0/2139908576 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fc35c004860 con 0x555fd003f380 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc37c75db80 1 -- 172.21.15.46:0/2139908576 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fd0032ad0 con 0x555fd003f380 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc34dffb700 1 -- 172.21.15.46:0/2139908576 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc35c004a20 con 0x555fd003f380 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc37c75db80 10 client.29320 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc37c75db80 1 -- 172.21.15.46:0/2139908576 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fc36810e500 con 0x555fd003f380 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc34dffb700 1 -- 172.21.15.46:0/2139908576 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555fd0032ad0 con 0x555fd003f380 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc34dffb700 1 -- 172.21.15.46:0/2139908576 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc35c038bf0 con 0x555fd003f380 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc37c75db80 10 client.29320 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc37c75db80 10 client.29320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc37c75db80 20 client.29320 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc37c75db80 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555fd0059940 0x555fd005bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc37c75db80 4 client.29320 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc37c75db80 1 -- 172.21.15.46:0/2139908576 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fcff8a700 con 0x555fd0059940 2024-03-18T20:48:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.493+0000 7fc36e364700 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555fd0059940 0x555fd005bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.494+0000 7fc34dffb700 10 client.29320 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.494+0000 7fc36e364700 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555fd0059940 0x555fd005bd30 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564742fce7e2333b server_cookie=c9c20b4a9de1c27a in_seq=0 out_seq=0 2024-03-18T20:48:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.495+0000 7fc34dffb700 1 -- 172.21.15.46:0/2139908576 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555fcff8a700 con 0x555fd0059940 2024-03-18T20:48:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.495+0000 7fc34dffb700 10 client.29320 handle_command_reply: tid=0 2024-03-18T20:48:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.495+0000 7fc37c75db80 1 client.29320 shutdown 2024-03-18T20:48:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.496+0000 7fc37c75db80 1 -- 172.21.15.46:0/2139908576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd003f380 msgr2=0x555fd00474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.496+0000 7fc37c75db80 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd003f380 0x555fd00474f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc35c032040 tx=0x7fc35c00b3f0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.496+0000 7fc37c75db80 1 -- 172.21.15.46:0/2139908576 shutdown_connections 2024-03-18T20:48:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.496+0000 7fc37c75db80 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555fd0059940 0x555fd005bd30 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.496+0000 7fc37c75db80 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd0046fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.496+0000 7fc37c75db80 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd004a220 0x555fd0047a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.496+0000 7fc34dffb700 0 client.29320 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.496+0000 7fc37c75db80 1 --2- 172.21.15.46:0/2139908576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd003f380 0x555fd00474f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.496+0000 7fc37c75db80 1 -- 172.21.15.46:0/2139908576 >> 172.21.15.46:0/2139908576 conn(0x555fcffb5220 msgr2=0x555fd003ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.496+0000 7fc37c75db80 1 -- 172.21.15.46:0/2139908576 shutdown_connections 2024-03-18T20:48:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.496+0000 7fc37c75db80 1 -- 172.21.15.46:0/2139908576 wait complete. 2024-03-18T20:48:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.496+0000 7fc37c75db80 20 client.29320 trim_cache size 0 max 16384 2024-03-18T20:48:03.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.515+0000 7fc37c75db80 1 Processor -- start 2024-03-18T20:48:03.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.515+0000 7fc37c75db80 1 -- start start 2024-03-18T20:48:03.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.515+0000 7fc37c75db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd0047850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.515+0000 7fc37c75db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003f380 0x555fd0045e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.515+0000 7fc37c75db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd0044210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.515+0000 7fc37c75db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd003c440 con 0x555fd003ea60 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.515+0000 7fc37c75db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd0048620 con 0x555fd003f380 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.515+0000 7fc37c75db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd003c230 con 0x555fd004a220 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.515+0000 7fc36e364700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd0047850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc36e364700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd0047850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44380/0 (socket says 172.21.15.46:44380) 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc36e364700 1 -- 172.21.15.46:0/3751992153 learned_addr learned my addr 172.21.15.46:0/3751992153 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc36db63700 1 --2- 172.21.15.46:0/3751992153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003f380 0x555fd0045e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc36eb65700 1 --2- 172.21.15.46:0/3751992153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd0044210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc36db63700 1 -- 172.21.15.46:0/3751992153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 msgr2=0x555fd0044210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc36db63700 1 --2- 172.21.15.46:0/3751992153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd0044210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc36db63700 1 -- 172.21.15.46:0/3751992153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 msgr2=0x555fd0047850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc36db63700 1 --2- 172.21.15.46:0/3751992153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd0047850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc36db63700 1 -- 172.21.15.46:0/3751992153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fcfdbfb60 con 0x555fd003f380 2024-03-18T20:48:03.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc36e364700 1 --2- 172.21.15.46:0/3751992153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd0047850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:03.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc36eb65700 1 --2- 172.21.15.46:0/3751992153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd0044210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:03.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc36db63700 1 --2- 172.21.15.46:0/3751992153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003f380 0x555fd0045e60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0098e0 tx=0x7fc35c005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e24bac52232852b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:03.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc3657fa700 1 -- 172.21.15.46:0/3751992153 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c024a60 con 0x555fd003f380 2024-03-18T20:48:03.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc3657fa700 1 -- 172.21.15.46:0/3751992153 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c0056c0 con 0x555fd003f380 2024-03-18T20:48:03.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.516+0000 7fc3657fa700 1 -- 172.21.15.46:0/3751992153 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c040780 con 0x555fd003f380 2024-03-18T20:48:03.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.517+0000 7fc37c75db80 1 -- 172.21.15.46:0/3751992153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003f380 msgr2=0x555fd0045e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.517+0000 7fc37c75db80 1 --2- 172.21.15.46:0/3751992153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003f380 0x555fd0045e60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0098e0 tx=0x7fc35c005ce0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.517+0000 7fc37c75db80 1 -- 172.21.15.46:0/3751992153 shutdown_connections 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.517+0000 7fc37c75db80 1 --2- 172.21.15.46:0/3751992153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd0047850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.517+0000 7fc37c75db80 1 --2- 172.21.15.46:0/3751992153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd0044210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.517+0000 7fc37c75db80 1 --2- 172.21.15.46:0/3751992153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003f380 0x555fd0045e60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.517+0000 7fc37c75db80 1 -- 172.21.15.46:0/3751992153 >> 172.21.15.46:0/3751992153 conn(0x555fcffb5220 msgr2=0x555fd0045750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.517+0000 7fc37c75db80 1 -- 172.21.15.46:0/3751992153 shutdown_connections 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.517+0000 7fc37c75db80 1 -- 172.21.15.46:0/3751992153 wait complete. 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.517+0000 7fc37c75db80 1 -- start start 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc37c75db80 1 Processor -- start 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc37c75db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd004edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc37c75db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003f380 0x555fd004f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc37c75db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd00ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc37c75db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd004fbf0 con 0x555fd003ea60 2024-03-18T20:48:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc37c75db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd00acb70 con 0x555fd003f380 2024-03-18T20:48:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc37c75db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd00accb0 con 0x555fd004a220 2024-03-18T20:48:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc37c75db80 10 client.0 _ll_register_callbacks cb 0x7fc3580092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc36e364700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd004edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc36db63700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003f380 0x555fd004f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc36e364700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd004edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44394/0 (socket says 172.21.15.46:44394) 2024-03-18T20:48:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc36e364700 1 -- 172.21.15.46:0/181995337 learned_addr learned my addr 172.21.15.46:0/181995337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:03.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc36eb65700 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd00ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc36e364700 1 -- 172.21.15.46:0/181995337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 msgr2=0x555fd00ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc36e364700 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd00ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc36e364700 1 -- 172.21.15.46:0/181995337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003f380 msgr2=0x555fd004f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc36e364700 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003f380 0x555fd004f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc36e364700 1 -- 172.21.15.46:0/181995337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc358021070 con 0x555fd003ea60 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc36db63700 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003f380 0x555fd004f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.518+0000 7fc36eb65700 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd00ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc36e364700 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd004edd0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc3580090a0 tx=0x7fc358084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c17b224ff04353a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc34dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc34dffb700 1 -- 172.21.15.46:0/181995337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358099700 con 0x555fd003ea60 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc34dffb700 1 -- 172.21.15.46:0/181995337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fc35809d530 con 0x555fd003ea60 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc37c75db80 1 -- 172.21.15.46:0/181995337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc35c01e040 con 0x555fd003ea60 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc34dffb700 1 -- 172.21.15.46:0/181995337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3580a1b80 con 0x555fd003ea60 2024-03-18T20:48:03.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc37c75db80 10 client.19686 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc37c75db80 1 -- 172.21.15.46:0/181995337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fc36810e500 con 0x555fd003ea60 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc34dffb700 1 -- 172.21.15.46:0/181995337 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c01e040 con 0x555fd003ea60 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc34dffb700 1 -- 172.21.15.46:0/181995337 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc3580b3020 con 0x555fd003ea60 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc37c75db80 10 client.19686 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc37c75db80 10 client.19686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc37c75db80 20 client.19686 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc37c75db80 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555fd00ae3f0 0x555fd00b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.519+0000 7fc37c75db80 4 client.19686 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.520+0000 7fc37c75db80 1 -- 172.21.15.46:0/181995337 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555fcfde0cf0 con 0x555fd00ae3f0 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.520+0000 7fc36db63700 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555fd00ae3f0 0x555fd00b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.520+0000 7fc36db63700 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555fd00ae3f0 0x555fd00b0810 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8abdac532d8e20 server_cookie=e35c8cb814c6908a in_seq=0 out_seq=0 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.520+0000 7fc34dffb700 10 client.19686 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:03.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc34dffb700 1 -- 172.21.15.46:0/181995337 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555fcfde0cf0 con 0x555fd00ae3f0 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc34dffb700 10 client.19686 handle_command_reply: tid=0 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc37c75db80 1 client.19686 shutdown 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc37c75db80 1 -- 172.21.15.46:0/181995337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 msgr2=0x555fd004edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc37c75db80 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd004edd0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc3580090a0 tx=0x7fc358084040 comp rx=0 tx=0).stop 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc37c75db80 1 -- 172.21.15.46:0/181995337 shutdown_connections 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc37c75db80 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555fd00ae3f0 0x555fd00b0810 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc37c75db80 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555fd003ea60 0x555fd004edd0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc37c75db80 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fd004a220 0x555fd00ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc34dffb700 0 client.19686 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc37c75db80 1 --2- 172.21.15.46:0/181995337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555fd003f380 0x555fd004f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc37c75db80 1 -- 172.21.15.46:0/181995337 >> 172.21.15.46:0/181995337 conn(0x555fcffb5220 msgr2=0x555fd00412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc37c75db80 1 -- 172.21.15.46:0/181995337 shutdown_connections 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc37c75db80 1 -- 172.21.15.46:0/181995337 wait complete. 2024-03-18T20:48:03.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.521+0000 7fc37c75db80 20 client.19686 trim_cache size 0 max 16384 2024-03-18T20:48:03.531 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:48:03.531 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:48:03.531 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:48:03.531 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:48:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.522+0000 7fc3705c8700 1 -- 172.21.15.46:0/2366734173 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc3500a1310 msgr2=0x7fc3500a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.522+0000 7fc3705c8700 1 --2- 172.21.15.46:0/2366734173 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc3500a1310 0x7fc3500a37d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3681987d0 tx=0x7fc35c00b260 comp rx=0 tx=0).stop 2024-03-18T20:48:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.522+0000 7fc3705c8700 1 -- 172.21.15.46:0/2366734173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3680ff8e0 msgr2=0x7fc368197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.522+0000 7fc3705c8700 1 --2- 172.21.15.46:0/2366734173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3680ff8e0 0x7fc368197280 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc35800c370 tx=0x7fc35800c680 comp rx=0 tx=0).stop 2024-03-18T20:48:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.522+0000 7fc3705c8700 1 -- 172.21.15.46:0/2366734173 shutdown_connections 2024-03-18T20:48:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.522+0000 7fc3705c8700 1 --2- 172.21.15.46:0/2366734173 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc3500a1310 0x7fc3500a37d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.522+0000 7fc3705c8700 1 --2- 172.21.15.46:0/2366734173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc36819bb60 0x7fc36819bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.522+0000 7fc3705c8700 1 --2- 172.21.15.46:0/2366734173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3680ff8e0 0x7fc368197280 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.522+0000 7fc3705c8700 1 --2- 172.21.15.46:0/2366734173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3681060f0 0x7fc3681977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.522+0000 7fc3705c8700 1 -- 172.21.15.46:0/2366734173 >> 172.21.15.46:0/2366734173 conn(0x7fc368074f00 msgr2=0x7fc3680fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.522+0000 7fc3705c8700 1 -- 172.21.15.46:0/2366734173 shutdown_connections 2024-03-18T20:48:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.522+0000 7fc3705c8700 1 -- 172.21.15.46:0/2366734173 wait complete. 2024-03-18T20:48:03.538 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:48:03.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:48:03.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.681+0000 7f561835e700 1 -- 172.21.15.46:0/1099939621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5610101830 msgr2=0x7f5610105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.681+0000 7f561835e700 1 --2- 172.21.15.46:0/1099939621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5610101830 0x7f5610105870 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f5600009990 tx=0x7f5600009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.682+0000 7f561835e700 1 -- 172.21.15.46:0/1099939621 shutdown_connections 2024-03-18T20:48:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.682+0000 7f561835e700 1 --2- 172.21.15.46:0/1099939621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5610105fa0 0x7f561010db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.682+0000 7f561835e700 1 --2- 172.21.15.46:0/1099939621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5610100e80 0x7f5610101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.682+0000 7f561835e700 1 --2- 172.21.15.46:0/1099939621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5610101830 0x7f5610105870 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.682+0000 7f561835e700 1 -- 172.21.15.46:0/1099939621 >> 172.21.15.46:0/1099939621 conn(0x7f56100fc720 msgr2=0x7f56100feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.682+0000 7f561835e700 1 -- 172.21.15.46:0/1099939621 shutdown_connections 2024-03-18T20:48:03.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.682+0000 7f561835e700 1 -- 172.21.15.46:0/1099939621 wait complete. 2024-03-18T20:48:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f561835e700 1 Processor -- start 2024-03-18T20:48:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f561835e700 1 -- start start 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f561835e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5610100e80 0x7f5610197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f561835e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5610105fa0 0x7f5610197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f561835e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561019baa0 0x7f561019bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f561835e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561019c520 con 0x7f5610105fa0 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f561835e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561019c690 con 0x7f561019baa0 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f561835e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561019c850 con 0x7f5610100e80 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f56158f9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5610105fa0 0x7f5610197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f56158f9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5610105fa0 0x7f5610197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44406/0 (socket says 172.21.15.46:44406) 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f56168fb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561019baa0 0x7f561019bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f56158f9700 1 -- 172.21.15.46:0/2453098973 learned_addr learned my addr 172.21.15.46:0/2453098973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f56160fa700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5610100e80 0x7f5610197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f56168fb700 1 -- 172.21.15.46:0/2453098973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5610100e80 msgr2=0x7f5610197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f56168fb700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5610100e80 0x7f5610197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f56168fb700 1 -- 172.21.15.46:0/2453098973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5610105fa0 msgr2=0x7f5610197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f56168fb700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5610105fa0 0x7f5610197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.683+0000 7f56168fb700 1 -- 172.21.15.46:0/2453098973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56000095f0 con 0x7f561019baa0 2024-03-18T20:48:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.684+0000 7f56158f9700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5610105fa0 0x7f5610197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.684+0000 7f56168fb700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561019baa0 0x7f561019bf20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f560800efc0 tx=0x7f560800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.684+0000 7f56077fe700 1 -- 172.21.15.46:0/2453098973 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608018070 con 0x7f561019baa0 2024-03-18T20:48:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.684+0000 7f561835e700 1 -- 172.21.15.46:0/2453098973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56101a3d50 con 0x7f561019baa0 2024-03-18T20:48:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.684+0000 7f56077fe700 1 -- 172.21.15.46:0/2453098973 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f560800f040 con 0x7f561019baa0 2024-03-18T20:48:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.684+0000 7f56077fe700 1 -- 172.21.15.46:0/2453098973 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560800cd40 con 0x7f561019baa0 2024-03-18T20:48:03.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.684+0000 7f561835e700 1 -- 172.21.15.46:0/2453098973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56101a4230 con 0x7f561019baa0 2024-03-18T20:48:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.686+0000 7f561835e700 1 -- 172.21.15.46:0/2453098973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5610109270 con 0x7f561019baa0 2024-03-18T20:48:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.686+0000 7f56077fe700 1 -- 172.21.15.46:0/2453098973 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5608003a00 con 0x7f561019baa0 2024-03-18T20:48:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.687+0000 7f56077fe700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f55fc0a1290 0x7f55fc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.687+0000 7f56160fa700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f55fc0a1290 0x7f55fc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.687+0000 7f56077fe700 1 -- 172.21.15.46:0/2453098973 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f56080cc830 con 0x7f561019baa0 2024-03-18T20:48:03.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.688+0000 7f56160fa700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f55fc0a1290 0x7f55fc0a3750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f560c009c60 tx=0x7f560c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:03.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.693+0000 7f56077fe700 1 -- 172.21.15.46:0/2453098973 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5608095cc0 con 0x7f561019baa0 2024-03-18T20:48:03.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 455730 3.74 MB/sec execute 538 sec latency 40.463 ms 2024-03-18T20:48:03.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.963+0000 7f561835e700 1 -- 172.21.15.46:0/2453098973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56101984c0 con 0x7f561019baa0 2024-03-18T20:48:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.964+0000 7f56077fe700 1 -- 172.21.15.46:0/2453098973 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f560801b320 con 0x7f561019baa0 2024-03-18T20:48:03.967 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:48:03.967 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:47:48.264206+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:48:03.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.967+0000 7f561835e700 1 -- 172.21.15.46:0/2453098973 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f55fc0a1290 msgr2=0x7f55fc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.967+0000 7f561835e700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f55fc0a1290 0x7f55fc0a3750 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f560c009c60 tx=0x7f560c009260 comp rx=0 tx=0).stop 2024-03-18T20:48:03.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.967+0000 7f561835e700 1 -- 172.21.15.46:0/2453098973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561019baa0 msgr2=0x7f561019bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:03.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.967+0000 7f561835e700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561019baa0 0x7f561019bf20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f560800efc0 tx=0x7f560800c3a0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.967+0000 7f561835e700 1 -- 172.21.15.46:0/2453098973 shutdown_connections 2024-03-18T20:48:03.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.967+0000 7f561835e700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f55fc0a1290 0x7f55fc0a3750 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.967+0000 7f561835e700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f561019baa0 0x7f561019bf20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.967+0000 7f561835e700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5610100e80 0x7f5610197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.967+0000 7f561835e700 1 --2- 172.21.15.46:0/2453098973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5610105fa0 0x7f5610197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:03.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.967+0000 7f561835e700 1 -- 172.21.15.46:0/2453098973 >> 172.21.15.46:0/2453098973 conn(0x7f56100fc720 msgr2=0x7f56100fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:03.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.967+0000 7f561835e700 1 -- 172.21.15.46:0/2453098973 shutdown_connections 2024-03-18T20:48:03.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:03.967+0000 7f561835e700 1 -- 172.21.15.46:0/2453098973 wait complete. 2024-03-18T20:48:03.971 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 51 2024-03-18T20:48:03.983 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:48:03.983 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:48:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.120+0000 7f67d71a6700 1 -- 172.21.15.46:0/3605360147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67d00ffe50 msgr2=0x7f67d0100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.120+0000 7f67d71a6700 1 --2- 172.21.15.46:0/3605360147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67d00ffe50 0x7f67d0100230 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f67c40098f0 tx=0x7f67c4009c00 comp rx=0 tx=0).stop 2024-03-18T20:48:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.120+0000 7f67d71a6700 1 -- 172.21.15.46:0/3605360147 shutdown_connections 2024-03-18T20:48:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.120+0000 7f67d71a6700 1 --2- 172.21.15.46:0/3605360147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67d0104f60 0x7f67d010cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.120+0000 7f67d71a6700 1 --2- 172.21.15.46:0/3605360147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67d00ffe50 0x7f67d0100230 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.120+0000 7f67d71a6700 1 --2- 172.21.15.46:0/3605360147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d0100800 0x7f67d0104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.120+0000 7f67d71a6700 1 -- 172.21.15.46:0/3605360147 >> 172.21.15.46:0/3605360147 conn(0x7f67d0076d40 msgr2=0x7f67d0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.121+0000 7f67d71a6700 1 -- 172.21.15.46:0/3605360147 shutdown_connections 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.121+0000 7f67d71a6700 1 -- 172.21.15.46:0/3605360147 wait complete. 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.121+0000 7f67d71a6700 1 Processor -- start 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.121+0000 7f67d71a6700 1 -- start start 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.121+0000 7f67d71a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d00ffe50 0x7f67d0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.121+0000 7f67d71a6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67d0104f60 0x7f67d0197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.121+0000 7f67d71a6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67d019bae0 0x7f67d019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.121+0000 7f67d71a6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d019c560 con 0x7f67d00ffe50 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.121+0000 7f67d71a6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d019c6d0 con 0x7f67d019bae0 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67d71a6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d019c890 con 0x7f67d0104f60 2024-03-18T20:48:04.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67d4f42700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d00ffe50 0x7f67d0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67d4f42700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d00ffe50 0x7f67d0197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44428/0 (socket says 172.21.15.46:44428) 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67d4f42700 1 -- 172.21.15.46:0/2654829655 learned_addr learned my addr 172.21.15.46:0/2654829655 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67d5743700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67d019bae0 0x7f67d019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67cffff700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67d0104f60 0x7f67d0197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67d4f42700 1 -- 172.21.15.46:0/2654829655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67d0104f60 msgr2=0x7f67d0197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67d4f42700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67d0104f60 0x7f67d0197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67d4f42700 1 -- 172.21.15.46:0/2654829655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67d019bae0 msgr2=0x7f67d019bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67d4f42700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67d019bae0 0x7f67d019bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67d4f42700 1 -- 172.21.15.46:0/2654829655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c4009520 con 0x7f67d00ffe50 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67d4f42700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d00ffe50 0x7f67d0197200 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f67c4005950 tx=0x7f67c400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67cdffb700 1 -- 172.21.15.46:0/2654829655 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c401c070 con 0x7f67d00ffe50 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67cdffb700 1 -- 172.21.15.46:0/2654829655 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67c400bb20 con 0x7f67d00ffe50 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67d71a6700 1 -- 172.21.15.46:0/2654829655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d01a3d90 con 0x7f67d00ffe50 2024-03-18T20:48:04.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.122+0000 7f67cdffb700 1 -- 172.21.15.46:0/2654829655 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c40038e0 con 0x7f67d00ffe50 2024-03-18T20:48:04.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.123+0000 7f67d71a6700 1 -- 172.21.15.46:0/2654829655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d01a41c0 con 0x7f67d00ffe50 2024-03-18T20:48:04.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67e333bb80 1 Processor -- start 2024-03-18T20:48:04.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67e333bb80 1 -- start start 2024-03-18T20:48:04.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67e333bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 0x560186300e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67e333bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186301380 0x5601863100f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67e333bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186310630 0x560186312a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67e333bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018625a990 con 0x560186301380 2024-03-18T20:48:04.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67e333bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560186258350 con 0x560186300a60 2024-03-18T20:48:04.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67e333bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560186253c40 con 0x560186310630 2024-03-18T20:48:04.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67cffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186310630 0x560186312a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67cffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186310630 0x560186312a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:34960/0 (socket says 172.21.15.46:34960) 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67cffff700 1 -- 172.21.15.46:0/720302830 learned_addr learned my addr 172.21.15.46:0/720302830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67d5743700 1 --2- 172.21.15.46:0/720302830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 0x560186300e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67cffff700 1 -- 172.21.15.46:0/720302830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 msgr2=0x560186300e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67cffff700 1 --2- 172.21.15.46:0/720302830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 0x560186300e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67cffff700 1 -- 172.21.15.46:0/720302830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186301380 msgr2=0x5601863100f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.127+0000 7f67cffff700 1 --2- 172.21.15.46:0/720302830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186301380 0x5601863100f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.128+0000 7f67cffff700 1 -- 172.21.15.46:0/720302830 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601860d53b0 con 0x560186310630 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.128+0000 7f67cdffb700 1 -- 172.21.15.46:0/2654829655 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f67c4020030 con 0x7f67d00ffe50 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.128+0000 7f67cffff700 1 --2- 172.21.15.46:0/720302830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186310630 0x560186312a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009c70 tx=0x7f67bc009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d05a84858bfe054c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.128+0000 7f67baffd700 1 -- 172.21.15.46:0/720302830 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc009740 con 0x560186310630 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.128+0000 7f67baffd700 1 -- 172.21.15.46:0/720302830 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67bc011bf0 con 0x560186310630 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.128+0000 7f67baffd700 1 -- 172.21.15.46:0/720302830 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc00f6a0 con 0x560186310630 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.128+0000 7f67cdffb700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f67c00a1290 0x7f67c00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.128+0000 7f67d5743700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f67c00a1290 0x7f67c00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.128+0000 7f67cdffb700 1 -- 172.21.15.46:0/2654829655 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f67c40cdbe0 con 0x7f67d00ffe50 2024-03-18T20:48:04.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.129+0000 7f67e333bb80 1 -- 172.21.15.46:0/720302830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186310630 msgr2=0x560186312a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.129+0000 7f67e333bb80 1 --2- 172.21.15.46:0/720302830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186310630 0x560186312a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009c70 tx=0x7f67bc009f80 comp rx=0 tx=0).stop 2024-03-18T20:48:04.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.129+0000 7f67d5743700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f67c00a1290 0x7f67c00a3750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f67c80127f0 tx=0x7f67c800b040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.129+0000 7f67e333bb80 1 -- 172.21.15.46:0/720302830 shutdown_connections 2024-03-18T20:48:04.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.129+0000 7f67e333bb80 1 --2- 172.21.15.46:0/720302830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186310630 0x560186312a20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.129+0000 7f67e333bb80 1 --2- 172.21.15.46:0/720302830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 0x560186300e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.129+0000 7f67e333bb80 1 --2- 172.21.15.46:0/720302830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186301380 0x5601863100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.129+0000 7f67e333bb80 1 -- 172.21.15.46:0/720302830 >> 172.21.15.46:0/720302830 conn(0x560186277220 msgr2=0x5601862fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.129+0000 7f67e333bb80 1 -- 172.21.15.46:0/720302830 shutdown_connections 2024-03-18T20:48:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.129+0000 7f67e333bb80 1 -- 172.21.15.46:0/720302830 wait complete. 2024-03-18T20:48:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.129+0000 7f67e333bb80 1 -- start start 2024-03-18T20:48:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.129+0000 7f67e333bb80 1 Processor -- start 2024-03-18T20:48:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67e333bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 0x560186310070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67e333bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186301380 0x56018630d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67e333bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 0x56018630daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67e333bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601861efd80 con 0x560186310630 2024-03-18T20:48:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67e333bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601861efa20 con 0x560186300a60 2024-03-18T20:48:04.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67e333bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601860d76a0 con 0x560186301380 2024-03-18T20:48:04.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67e333bb80 10 client.0 _ll_register_callbacks cb 0x56018605b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:04.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67d5743700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 0x56018630daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67d5743700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 0x56018630daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44440/0 (socket says 172.21.15.46:44440) 2024-03-18T20:48:04.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67d5743700 1 -- 172.21.15.46:0/3068611545 learned_addr learned my addr 172.21.15.46:0/3068611545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67cffff700 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186301380 0x56018630d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67d4f42700 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 0x560186310070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67d5743700 1 -- 172.21.15.46:0/3068611545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186301380 msgr2=0x56018630d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67d5743700 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186301380 0x56018630d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67d5743700 1 -- 172.21.15.46:0/3068611545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 msgr2=0x560186310070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67d5743700 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 0x560186310070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.130+0000 7f67d5743700 1 -- 172.21.15.46:0/3068611545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc0098a0 con 0x560186310630 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67d5743700 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 0x56018630daa0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f67c8004680 tx=0x7f67c8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a80944b553c372 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67affff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67affff700 1 -- 172.21.15.46:0/3068611545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c800e040 con 0x560186310630 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67affff700 1 -- 172.21.15.46:0/3068611545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f67c8004db0 con 0x560186310630 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67e333bb80 1 -- 172.21.15.46:0/3068611545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601862f4ad0 con 0x560186310630 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67affff700 1 -- 172.21.15.46:0/3068611545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67c8003e80 con 0x560186310630 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67e333bb80 10 client.19701 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67e333bb80 1 -- 172.21.15.46:0/3068611545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x560186081b60 con 0x560186310630 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67affff700 1 -- 172.21.15.46:0/3068611545 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5601862f4ad0 con 0x560186310630 2024-03-18T20:48:04.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67affff700 1 -- 172.21.15.46:0/3068611545 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f67c8023b40 con 0x560186310630 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67e333bb80 10 client.19701 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67e333bb80 10 client.19701 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.131+0000 7f67e333bb80 20 client.19701 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.132+0000 7f67e333bb80 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560186320290 0x5601863226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.132+0000 7f67e333bb80 4 client.19701 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.132+0000 7f67e333bb80 1 -- 172.21.15.46:0/3068611545 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56018624c700 con 0x560186320290 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.132+0000 7f67d4f42700 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560186320290 0x5601863226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.132+0000 7f67d4f42700 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560186320290 0x5601863226b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a01d34aa3a66e891 server_cookie=afaf10355de8d15d in_seq=0 out_seq=0 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.132+0000 7f67affff700 10 client.19701 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67affff700 1 -- 172.21.15.46:0/3068611545 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56018624c700 con 0x560186320290 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67affff700 10 client.19701 handle_command_reply: tid=0 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67e333bb80 1 client.19701 shutdown 2024-03-18T20:48:04.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67e333bb80 1 -- 172.21.15.46:0/3068611545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 msgr2=0x56018630daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67e333bb80 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 0x56018630daa0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f67c8004680 tx=0x7f67c8004b40 comp rx=0 tx=0).stop 2024-03-18T20:48:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67e333bb80 1 -- 172.21.15.46:0/3068611545 shutdown_connections 2024-03-18T20:48:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67e333bb80 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560186320290 0x5601863226b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67e333bb80 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 0x56018630daa0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67e333bb80 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 0x560186310070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67affff700 0 client.19701 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:48:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67e333bb80 1 --2- 172.21.15.46:0/3068611545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186301380 0x56018630d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67e333bb80 1 -- 172.21.15.46:0/3068611545 >> 172.21.15.46:0/3068611545 conn(0x560186277220 msgr2=0x5601862fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67e333bb80 1 -- 172.21.15.46:0/3068611545 shutdown_connections 2024-03-18T20:48:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67e333bb80 1 -- 172.21.15.46:0/3068611545 wait complete. 2024-03-18T20:48:04.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.133+0000 7f67e333bb80 20 client.19701 trim_cache size 0 max 16384 2024-03-18T20:48:04.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67e333bb80 1 Processor -- start 2024-03-18T20:48:04.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67e333bb80 1 -- start start 2024-03-18T20:48:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67e333bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 0x560186317290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67e333bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186301380 0x56018631b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67e333bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 0x56018630fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67e333bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560186301d30 con 0x560186310630 2024-03-18T20:48:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67e333bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601863177d0 con 0x560186300a60 2024-03-18T20:48:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67e333bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601862fe3d0 con 0x560186301380 2024-03-18T20:48:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67d5743700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 0x56018630fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67d5743700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 0x56018630fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44450/0 (socket says 172.21.15.46:44450) 2024-03-18T20:48:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67d5743700 1 -- 172.21.15.46:0/2423869982 learned_addr learned my addr 172.21.15.46:0/2423869982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67d4f42700 1 --2- 172.21.15.46:0/2423869982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 0x560186317290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67cffff700 1 --2- 172.21.15.46:0/2423869982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186301380 0x56018631b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67d5743700 1 -- 172.21.15.46:0/2423869982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186301380 msgr2=0x56018631b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67d5743700 1 --2- 172.21.15.46:0/2423869982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186301380 0x56018631b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67d5743700 1 -- 172.21.15.46:0/2423869982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 msgr2=0x560186317290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67d5743700 1 --2- 172.21.15.46:0/2423869982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 0x560186317290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.147+0000 7f67d5743700 1 -- 172.21.15.46:0/2423869982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560186219ea0 con 0x560186310630 2024-03-18T20:48:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.148+0000 7f67d5743700 1 --2- 172.21.15.46:0/2423869982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 0x56018630fc80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f67c8006070 tx=0x7f67c80099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d673d9b245e6b190 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.148+0000 7f67bb7fe700 1 -- 172.21.15.46:0/2423869982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c800e070 con 0x560186310630 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.148+0000 7f67bb7fe700 1 -- 172.21.15.46:0/2423869982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67c803a070 con 0x560186310630 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.148+0000 7f67bb7fe700 1 -- 172.21.15.46:0/2423869982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80092a0 con 0x560186310630 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.148+0000 7f67e333bb80 1 -- 172.21.15.46:0/2423869982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 msgr2=0x56018630fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.148+0000 7f67e333bb80 1 --2- 172.21.15.46:0/2423869982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 0x56018630fc80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f67c8006070 tx=0x7f67c80099a0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.148+0000 7f67e333bb80 1 -- 172.21.15.46:0/2423869982 shutdown_connections 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.148+0000 7f67e333bb80 1 --2- 172.21.15.46:0/2423869982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186310630 0x56018630fc80 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.148+0000 7f67e333bb80 1 --2- 172.21.15.46:0/2423869982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186300a60 0x560186317290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.148+0000 7f67e333bb80 1 --2- 172.21.15.46:0/2423869982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186301380 0x56018631b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.148+0000 7f67e333bb80 1 -- 172.21.15.46:0/2423869982 >> 172.21.15.46:0/2423869982 conn(0x560186277220 msgr2=0x5601862f8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.148+0000 7f67e333bb80 1 -- 172.21.15.46:0/2423869982 shutdown_connections 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.149+0000 7f67e333bb80 1 -- 172.21.15.46:0/2423869982 wait complete. 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.149+0000 7f67e333bb80 1 -- start start 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.149+0000 7f67e333bb80 1 Processor -- start 2024-03-18T20:48:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.149+0000 7f67e333bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186300a60 0x560186315050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.149+0000 7f67e333bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186301380 0x560186315590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.149+0000 7f67e333bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186310630 0x5601863730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.149+0000 7f67e333bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560186373600 con 0x560186301380 2024-03-18T20:48:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.149+0000 7f67e333bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560186373740 con 0x560186310630 2024-03-18T20:48:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.149+0000 7f67e333bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601863738d0 con 0x560186300a60 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.149+0000 7f67e333bb80 10 client.0 _ll_register_callbacks cb 0x7f67c8003dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.149+0000 7f67cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186301380 0x560186315590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186301380 0x560186315590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44452/0 (socket says 172.21.15.46:44452) 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67cffff700 1 -- 172.21.15.46:0/374408324 learned_addr learned my addr 172.21.15.46:0/374408324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67d4f42700 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186300a60 0x560186315050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67d5743700 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186310630 0x5601863730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67d4f42700 1 -- 172.21.15.46:0/374408324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186310630 msgr2=0x5601863730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67d4f42700 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186310630 0x5601863730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67d4f42700 1 -- 172.21.15.46:0/374408324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186301380 msgr2=0x560186315590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67d4f42700 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186301380 0x560186315590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67d4f42700 1 -- 172.21.15.46:0/374408324 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560186081b60 con 0x560186300a60 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67cffff700 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186301380 0x560186315590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67d4f42700 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186300a60 0x560186315050 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f67c40cb2e0 tx=0x7f67c400fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cd04a7dea556839 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67affff700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67affff700 1 -- 172.21.15.46:0/374408324 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c409eb60 con 0x560186300a60 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67affff700 1 -- 172.21.15.46:0/374408324 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f67c40a3120 con 0x560186300a60 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67e333bb80 1 -- 172.21.15.46:0/374408324 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67bc011bf0 con 0x560186300a60 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67affff700 1 -- 172.21.15.46:0/374408324 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67c4097440 con 0x560186300a60 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67e333bb80 10 client.29350 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.150+0000 7f67e333bb80 1 -- 172.21.15.46:0/374408324 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5601860d9cd0 con 0x560186300a60 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.151+0000 7f67affff700 1 -- 172.21.15.46:0/374408324 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67bc011bf0 con 0x560186300a60 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.151+0000 7f67affff700 1 -- 172.21.15.46:0/374408324 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f67c4021410 con 0x560186300a60 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.151+0000 7f67e333bb80 10 client.29350 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.151+0000 7f67e333bb80 10 client.29350 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.151+0000 7f67e333bb80 20 client.29350 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.151+0000 7f67e333bb80 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560186374d40 0x560186377160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.151+0000 7f67e333bb80 4 client.29350 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.151+0000 7f67e333bb80 1 -- 172.21.15.46:0/374408324 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5601860a2cf0 con 0x560186374d40 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.151+0000 7f67cffff700 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560186374d40 0x560186377160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.151+0000 7f67cffff700 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560186374d40 0x560186377160 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aabf5ce647dd48d1 server_cookie=3825486e4ed83855 in_seq=0 out_seq=0 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.151+0000 7f67affff700 10 client.29350 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67affff700 1 -- 172.21.15.46:0/374408324 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5601860a2cf0 con 0x560186374d40 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67affff700 10 client.29350 handle_command_reply: tid=0 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67e333bb80 1 client.29350 shutdown 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67e333bb80 1 -- 172.21.15.46:0/374408324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186300a60 msgr2=0x560186315050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67e333bb80 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186300a60 0x560186315050 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f67c40cb2e0 tx=0x7f67c400fa70 comp rx=0 tx=0).stop 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67e333bb80 1 -- 172.21.15.46:0/374408324 shutdown_connections 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67e333bb80 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x560186374d40 0x560186377160 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67e333bb80 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560186310630 0x5601863730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67affff700 0 client.29350 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:48:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67e333bb80 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560186300a60 0x560186315050 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67e333bb80 1 --2- 172.21.15.46:0/374408324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560186301380 0x560186315590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67e333bb80 1 -- 172.21.15.46:0/374408324 >> 172.21.15.46:0/374408324 conn(0x560186277220 msgr2=0x56018631b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67e333bb80 1 -- 172.21.15.46:0/374408324 shutdown_connections 2024-03-18T20:48:04.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67e333bb80 1 -- 172.21.15.46:0/374408324 wait complete. 2024-03-18T20:48:04.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.152+0000 7f67e333bb80 20 client.29350 trim_cache size 0 max 16384 2024-03-18T20:48:04.166 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:48:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.153+0000 7f67d71a6700 1 -- 172.21.15.46:0/2654829655 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f67c00a1290 msgr2=0x7f67c00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.153+0000 7f67d71a6700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f67c00a1290 0x7f67c00a3750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f67c80127f0 tx=0x7f67c800b040 comp rx=0 tx=0).stop 2024-03-18T20:48:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.153+0000 7f67d71a6700 1 -- 172.21.15.46:0/2654829655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d00ffe50 msgr2=0x7f67d0197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.153+0000 7f67d71a6700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d00ffe50 0x7f67d0197200 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f67c4005950 tx=0x7f67c400fa00 comp rx=0 tx=0).stop 2024-03-18T20:48:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.153+0000 7f67d71a6700 1 -- 172.21.15.46:0/2654829655 shutdown_connections 2024-03-18T20:48:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.153+0000 7f67d71a6700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f67c00a1290 0x7f67c00a3750 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.153+0000 7f67d71a6700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67d00ffe50 0x7f67d0197200 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.153+0000 7f67d71a6700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67d019bae0 0x7f67d019bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.153+0000 7f67d71a6700 1 --2- 172.21.15.46:0/2654829655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67d0104f60 0x7f67d0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.153+0000 7f67d71a6700 1 -- 172.21.15.46:0/2654829655 >> 172.21.15.46:0/2654829655 conn(0x7f67d0076d40 msgr2=0x7f67d0068270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.153+0000 7f67d71a6700 1 -- 172.21.15.46:0/2654829655 shutdown_connections 2024-03-18T20:48:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.153+0000 7f67d71a6700 1 -- 172.21.15.46:0/2654829655 wait complete. 2024-03-18T20:48:04.169 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:48:04.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:48:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.309+0000 7fbe483be700 1 -- 172.21.15.46:0/505306270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe400692a0 msgr2=0x7fbe40069680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.309+0000 7fbe483be700 1 --2- 172.21.15.46:0/505306270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe400692a0 0x7fbe40069680 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbe300099c0 tx=0x7fbe30009cd0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.309+0000 7fbe483be700 1 -- 172.21.15.46:0/505306270 shutdown_connections 2024-03-18T20:48:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.309+0000 7fbe483be700 1 --2- 172.21.15.46:0/505306270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe401060b0 0x7fbe4010dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.309+0000 7fbe483be700 1 --2- 172.21.15.46:0/505306270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe400692a0 0x7fbe40069680 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.309+0000 7fbe483be700 1 --2- 172.21.15.46:0/505306270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe40069c50 0x7fbe40105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.309+0000 7fbe483be700 1 -- 172.21.15.46:0/505306270 >> 172.21.15.46:0/505306270 conn(0x7fbe40076d80 msgr2=0x7fbe40077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.309+0000 7fbe483be700 1 -- 172.21.15.46:0/505306270 shutdown_connections 2024-03-18T20:48:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.309+0000 7fbe483be700 1 -- 172.21.15.46:0/505306270 wait complete. 2024-03-18T20:48:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe483be700 1 Processor -- start 2024-03-18T20:48:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe483be700 1 -- start start 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe483be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe400692a0 0x7fbe40197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe483be700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe401060b0 0x7fbe40197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe483be700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe4019baf0 0x7fbe4019bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe483be700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4019c570 con 0x7fbe400692a0 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe483be700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4019c6e0 con 0x7fbe4019baf0 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe483be700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4019c8a0 con 0x7fbe401060b0 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe4615a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe400692a0 0x7fbe40197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe4615a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe400692a0 0x7fbe40197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44460/0 (socket says 172.21.15.46:44460) 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe4615a700 1 -- 172.21.15.46:0/418577061 learned_addr learned my addr 172.21.15.46:0/418577061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe45959700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe401060b0 0x7fbe40197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe4695b700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe4019baf0 0x7fbe4019bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe4615a700 1 -- 172.21.15.46:0/418577061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe401060b0 msgr2=0x7fbe40197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe4615a700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe401060b0 0x7fbe40197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe4615a700 1 -- 172.21.15.46:0/418577061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe4019baf0 msgr2=0x7fbe4019bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe4615a700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe4019baf0 0x7fbe4019bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.310+0000 7fbe4615a700 1 -- 172.21.15.46:0/418577061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe300095f0 con 0x7fbe400692a0 2024-03-18T20:48:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.311+0000 7fbe4615a700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe400692a0 0x7fbe40197210 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fbe30006010 tx=0x7fbe3000b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.311+0000 7fbe377fe700 1 -- 172.21.15.46:0/418577061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3001d070 con 0x7fbe400692a0 2024-03-18T20:48:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.311+0000 7fbe483be700 1 -- 172.21.15.46:0/418577061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe401a3da0 con 0x7fbe400692a0 2024-03-18T20:48:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.311+0000 7fbe377fe700 1 -- 172.21.15.46:0/418577061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe3000beb0 con 0x7fbe400692a0 2024-03-18T20:48:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.311+0000 7fbe377fe700 1 -- 172.21.15.46:0/418577061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000fae0 con 0x7fbe400692a0 2024-03-18T20:48:04.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.311+0000 7fbe483be700 1 -- 172.21.15.46:0/418577061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe401a41c0 con 0x7fbe400692a0 2024-03-18T20:48:04.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.313+0000 7fbe377fe700 1 -- 172.21.15.46:0/418577061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fbe3002b440 con 0x7fbe400692a0 2024-03-18T20:48:04.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.314+0000 7fbe377fe700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbe2c0a1290 0x7fbe2c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.314+0000 7fbe377fe700 1 -- 172.21.15.46:0/418577061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fbe300cda00 con 0x7fbe400692a0 2024-03-18T20:48:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.314+0000 7fbe45959700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbe2c0a1290 0x7fbe2c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.314+0000 7fbe45959700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbe2c0a1290 0x7fbe2c0a3750 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbe40198760 tx=0x7fbe3c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.314+0000 7fbe54553b80 1 Processor -- start 2024-03-18T20:48:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe54553b80 1 -- start start 2024-03-18T20:48:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe54553b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 0x55c4415e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe54553b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e7380 0x55c4415f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe54553b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 0x55c4415f4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe54553b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c441540990 con 0x55c4415f2220 2024-03-18T20:48:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe54553b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c44153e350 con 0x55c4415e6a60 2024-03-18T20:48:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe54553b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c441539c40 con 0x55c4415e7380 2024-03-18T20:48:04.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe4695b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 0x55c4415f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe4695b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 0x55c4415f4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44470/0 (socket says 172.21.15.46:44470) 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe4695b700 1 -- 172.21.15.46:0/3699456275 learned_addr learned my addr 172.21.15.46:0/3699456275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe45959700 1 --2- 172.21.15.46:0/3699456275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e7380 0x55c4415f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe4615a700 1 --2- 172.21.15.46:0/3699456275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 0x55c4415e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe4695b700 1 -- 172.21.15.46:0/3699456275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e7380 msgr2=0x55c4415f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe4695b700 1 --2- 172.21.15.46:0/3699456275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e7380 0x55c4415f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe4695b700 1 -- 172.21.15.46:0/3699456275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 msgr2=0x55c4415e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.315+0000 7fbe4695b700 1 --2- 172.21.15.46:0/3699456275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 0x55c4415e6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.316+0000 7fbe4695b700 1 -- 172.21.15.46:0/3699456275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4413bb3b0 con 0x55c4415f2220 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.316+0000 7fbe4695b700 1 --2- 172.21.15.46:0/3699456275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 0x55c4415f4610 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009440 tx=0x7fbe38009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673cd5a5f8eb6620 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.316+0000 7fbe34da8700 1 -- 172.21.15.46:0/3699456275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3800e070 con 0x55c4415f2220 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.316+0000 7fbe34da8700 1 -- 172.21.15.46:0/3699456275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe3800f040 con 0x55c4415f2220 2024-03-18T20:48:04.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.316+0000 7fbe34da8700 1 -- 172.21.15.46:0/3699456275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38015b00 con 0x55c4415f2220 2024-03-18T20:48:04.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 -- 172.21.15.46:0/3699456275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 msgr2=0x55c4415f4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 --2- 172.21.15.46:0/3699456275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 0x55c4415f4610 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009440 tx=0x7fbe38009920 comp rx=0 tx=0).stop 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 -- 172.21.15.46:0/3699456275 shutdown_connections 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 --2- 172.21.15.46:0/3699456275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 0x55c4415f4610 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 --2- 172.21.15.46:0/3699456275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 0x55c4415e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 --2- 172.21.15.46:0/3699456275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e7380 0x55c4415f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 -- 172.21.15.46:0/3699456275 >> 172.21.15.46:0/3699456275 conn(0x55c44155d220 msgr2=0x55c4415e3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 -- 172.21.15.46:0/3699456275 shutdown_connections 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 -- 172.21.15.46:0/3699456275 wait complete. 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 -- start start 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 Processor -- start 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e6a60 0x55c4415f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.317+0000 7fbe54553b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 0x55c4415eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe54553b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415f2220 0x55c4415ef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe54553b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4414d5d80 con 0x55c4415e7380 2024-03-18T20:48:04.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe54553b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4414d5a20 con 0x55c4415f2220 2024-03-18T20:48:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe54553b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4413bd6a0 con 0x55c4415e6a60 2024-03-18T20:48:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe54553b80 10 client.0 _ll_register_callbacks cb 0x55c4413413a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe45959700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 0x55c4415eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe4695b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415f2220 0x55c4415ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe4695b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415f2220 0x55c4415ef610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49304/0 (socket says 172.21.15.46:49304) 2024-03-18T20:48:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe4615a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e6a60 0x55c4415f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe45959700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 0x55c4415eeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44486/0 (socket says 172.21.15.46:44486) 2024-03-18T20:48:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe4695b700 1 -- 172.21.15.46:0/1781323782 learned_addr learned my addr 172.21.15.46:0/1781323782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe45959700 1 -- 172.21.15.46:0/1781323782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e6a60 msgr2=0x55c4415f1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe45959700 1 --2- 172.21.15.46:0/1781323782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e6a60 0x55c4415f1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe45959700 1 -- 172.21.15.46:0/1781323782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415f2220 msgr2=0x55c4415ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe45959700 1 --2- 172.21.15.46:0/1781323782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415f2220 0x55c4415ef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe45959700 1 -- 172.21.15.46:0/1781323782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe380090f0 con 0x55c4415e7380 2024-03-18T20:48:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe4695b700 1 --2- 172.21.15.46:0/1781323782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415f2220 0x55c4415ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe45959700 1 --2- 172.21.15.46:0/1781323782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 0x55c4415eeed0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fbe40067bd0 tx=0x7fbe3c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fe74da3159ff49 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe29ffb700 1 -- 172.21.15.46:0/1781323782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c022070 con 0x55c4415e7380 2024-03-18T20:48:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe29ffb700 1 -- 172.21.15.46:0/1781323782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fbe3c0044f0 con 0x55c4415e7380 2024-03-18T20:48:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.318+0000 7fbe54553b80 1 -- 172.21.15.46:0/1781323782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4415daad0 con 0x55c4415e7380 2024-03-18T20:48:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.319+0000 7fbe29ffb700 1 -- 172.21.15.46:0/1781323782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe3c003d70 con 0x55c4415e7380 2024-03-18T20:48:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.319+0000 7fbe54553b80 10 client.19719 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.319+0000 7fbe54553b80 1 -- 172.21.15.46:0/1781323782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55c441367b60 con 0x55c4415e7380 2024-03-18T20:48:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.319+0000 7fbe29ffb700 1 -- 172.21.15.46:0/1781323782 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe3c033080 con 0x55c4415e7380 2024-03-18T20:48:04.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.319+0000 7fbe29ffb700 1 -- 172.21.15.46:0/1781323782 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbe3c02d030 con 0x55c4415e7380 2024-03-18T20:48:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.319+0000 7fbe54553b80 10 client.19719 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.319+0000 7fbe54553b80 10 client.19719 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:48:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.319+0000 7fbe54553b80 20 client.19719 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.319+0000 7fbe54553b80 1 --2- 172.21.15.46:0/1781323782 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55c441601f00 0x55c441604320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.319+0000 7fbe54553b80 4 client.19719 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.319+0000 7fbe54553b80 1 -- 172.21.15.46:0/1781323782 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c441532700 con 0x55c441601f00 2024-03-18T20:48:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.319+0000 7fbe4615a700 1 --2- 172.21.15.46:0/1781323782 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55c441601f00 0x55c441604320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.320+0000 7fbe4615a700 1 --2- 172.21.15.46:0/1781323782 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55c441601f00 0x55c441604320 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e28d675dd8c8740c server_cookie=8ff14ad6150d830d in_seq=0 out_seq=0 2024-03-18T20:48:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.320+0000 7fbe29ffb700 10 client.19719 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:48:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.320+0000 7fbe29ffb700 1 -- 172.21.15.46:0/1781323782 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c441532700 con 0x55c441601f00 2024-03-18T20:48:04.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.320+0000 7fbe29ffb700 10 client.19719 handle_command_reply: tid=0 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.320+0000 7fbe54553b80 1 client.19719 shutdown 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.320+0000 7fbe54553b80 1 -- 172.21.15.46:0/1781323782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 msgr2=0x55c4415eeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.320+0000 7fbe54553b80 1 --2- 172.21.15.46:0/1781323782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 0x55c4415eeed0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fbe40067bd0 tx=0x7fbe3c0047c0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.321+0000 7fbe54553b80 1 -- 172.21.15.46:0/1781323782 shutdown_connections 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.321+0000 7fbe54553b80 1 --2- 172.21.15.46:0/1781323782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415f2220 0x55c4415ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.321+0000 7fbe54553b80 1 --2- 172.21.15.46:0/1781323782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e6a60 0x55c4415f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.321+0000 7fbe54553b80 1 --2- 172.21.15.46:0/1781323782 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55c441601f00 0x55c441604320 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.321+0000 7fbe54553b80 1 --2- 172.21.15.46:0/1781323782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 0x55c4415eeed0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.321+0000 7fbe29ffb700 0 client.19719 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.321+0000 7fbe54553b80 1 -- 172.21.15.46:0/1781323782 >> 172.21.15.46:0/1781323782 conn(0x55c44155d220 msgr2=0x55c4415e2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.321+0000 7fbe54553b80 1 -- 172.21.15.46:0/1781323782 shutdown_connections 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.321+0000 7fbe54553b80 1 -- 172.21.15.46:0/1781323782 wait complete. 2024-03-18T20:48:04.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.321+0000 7fbe54553b80 20 client.19719 trim_cache size 0 max 16384 2024-03-18T20:48:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.334+0000 7fbe54553b80 1 Processor -- start 2024-03-18T20:48:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.334+0000 7fbe54553b80 1 -- start start 2024-03-18T20:48:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe54553b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 0x55c4415e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe54553b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e7380 0x55c4415f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe54553b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 0x55c4415f1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe54553b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4415efc60 con 0x55c4415f2220 2024-03-18T20:48:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe54553b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4415e3560 con 0x55c4415e6a60 2024-03-18T20:48:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe54553b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4415e7ae0 con 0x55c4415e7380 2024-03-18T20:48:04.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe4695b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 0x55c4415f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe4695b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 0x55c4415f1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44500/0 (socket says 172.21.15.46:44500) 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe4695b700 1 -- 172.21.15.46:0/3314354346 learned_addr learned my addr 172.21.15.46:0/3314354346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe4615a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 0x55c4415e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe45959700 1 --2- 172.21.15.46:0/3314354346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e7380 0x55c4415f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe4695b700 1 -- 172.21.15.46:0/3314354346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e7380 msgr2=0x55c4415f8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe4695b700 1 --2- 172.21.15.46:0/3314354346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e7380 0x55c4415f8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe4695b700 1 -- 172.21.15.46:0/3314354346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 msgr2=0x55c4415e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe4695b700 1 --2- 172.21.15.46:0/3314354346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 0x55c4415e6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe4695b700 1 -- 172.21.15.46:0/3314354346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4414ffea0 con 0x55c4415f2220 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe4695b700 1 --2- 172.21.15.46:0/3314354346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 0x55c4415f1780 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fbe38005b70 tx=0x7fbe380060c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60385dc028460471 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.335+0000 7fbe355a9700 1 -- 172.21.15.46:0/3314354346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3800e040 con 0x55c4415f2220 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.336+0000 7fbe355a9700 1 -- 172.21.15.46:0/3314354346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe38016c20 con 0x55c4415f2220 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.336+0000 7fbe355a9700 1 -- 172.21.15.46:0/3314354346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe380165a0 con 0x55c4415f2220 2024-03-18T20:48:04.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.336+0000 7fbe54553b80 1 -- 172.21.15.46:0/3314354346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 msgr2=0x55c4415f1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.336+0000 7fbe54553b80 1 --2- 172.21.15.46:0/3314354346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 0x55c4415f1780 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fbe38005b70 tx=0x7fbe380060c0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.336+0000 7fbe54553b80 1 -- 172.21.15.46:0/3314354346 shutdown_connections 2024-03-18T20:48:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.336+0000 7fbe54553b80 1 --2- 172.21.15.46:0/3314354346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415f2220 0x55c4415f1780 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.336+0000 7fbe54553b80 1 --2- 172.21.15.46:0/3314354346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 0x55c4415e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.336+0000 7fbe54553b80 1 --2- 172.21.15.46:0/3314354346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415e7380 0x55c4415f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.336+0000 7fbe54553b80 1 -- 172.21.15.46:0/3314354346 >> 172.21.15.46:0/3314354346 conn(0x55c44155d220 msgr2=0x55c4415ea1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.336+0000 7fbe54553b80 1 -- 172.21.15.46:0/3314354346 shutdown_connections 2024-03-18T20:48:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.336+0000 7fbe54553b80 1 -- 172.21.15.46:0/3314354346 wait complete. 2024-03-18T20:48:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.336+0000 7fbe54553b80 1 -- start start 2024-03-18T20:48:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe54553b80 1 Processor -- start 2024-03-18T20:48:04.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe54553b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 0x55c4415f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe54553b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 0x55c4415f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe54553b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415f2220 0x55c4415f7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe54553b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c441659340 con 0x55c4415e7380 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe54553b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4416594b0 con 0x55c4415e6a60 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe54553b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c441659670 con 0x55c4415f2220 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe54553b80 10 client.0 _ll_register_callbacks cb 0x55c4413413a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe45959700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 0x55c4415f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe4615a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 0x55c4415f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe45959700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 0x55c4415f73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44512/0 (socket says 172.21.15.46:44512) 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe45959700 1 -- 172.21.15.46:0/2504352683 learned_addr learned my addr 172.21.15.46:0/2504352683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe4695b700 1 --2- 172.21.15.46:0/2504352683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415f2220 0x55c4415f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe45959700 1 -- 172.21.15.46:0/2504352683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415f2220 msgr2=0x55c4415f7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe45959700 1 --2- 172.21.15.46:0/2504352683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415f2220 0x55c4415f7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe45959700 1 -- 172.21.15.46:0/2504352683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 msgr2=0x55c4415f6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe45959700 1 --2- 172.21.15.46:0/2504352683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 0x55c4415f6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.337+0000 7fbe45959700 1 -- 172.21.15.46:0/2504352683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4413bb3b0 con 0x55c4415e7380 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe45959700 1 --2- 172.21.15.46:0/2504352683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 0x55c4415f73a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x55c4415f0410 tx=0x7fbe3c009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bcefeb6563e3ee server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe29ffb700 1 -- 172.21.15.46:0/2504352683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c022040 con 0x55c4415e7380 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe29ffb700 1 -- 172.21.15.46:0/2504352683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fbe3c03a630 con 0x55c4415e7380 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe54553b80 1 -- 172.21.15.46:0/2504352683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe3800f040 con 0x55c4415e7380 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe29ffb700 1 -- 172.21.15.46:0/2504352683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe3c03a7f0 con 0x55c4415e7380 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe54553b80 10 client.19725 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe54553b80 1 -- 172.21.15.46:0/2504352683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55c4413bfcd0 con 0x55c4415e7380 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe29ffb700 1 -- 172.21.15.46:0/2504352683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe3c046080 con 0x55c4415e7380 2024-03-18T20:48:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe29ffb700 1 -- 172.21.15.46:0/2504352683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbe3c040030 con 0x55c4415e7380 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe54553b80 10 client.19725 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe54553b80 10 client.19725 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe54553b80 20 client.19725 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe54553b80 1 --2- 172.21.15.46:0/2504352683 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55c44165acb0 0x55c44165d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe54553b80 4 client.19725 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.338+0000 7fbe54553b80 1 -- 172.21.15.46:0/2504352683 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c441388cf0 con 0x55c44165acb0 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe4615a700 1 --2- 172.21.15.46:0/2504352683 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55c44165acb0 0x55c44165d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe4615a700 1 --2- 172.21.15.46:0/2504352683 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55c44165acb0 0x55c44165d0d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7fdba975dd382981 server_cookie=e01099d3c387699b in_seq=0 out_seq=0 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe29ffb700 10 client.19725 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe29ffb700 1 -- 172.21.15.46:0/2504352683 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c441388cf0 con 0x55c44165acb0 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe29ffb700 10 client.19725 handle_command_reply: tid=0 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe54553b80 1 client.19725 shutdown 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe54553b80 1 -- 172.21.15.46:0/2504352683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 msgr2=0x55c4415f73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe54553b80 1 --2- 172.21.15.46:0/2504352683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 0x55c4415f73a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x55c4415f0410 tx=0x7fbe3c009340 comp rx=0 tx=0).stop 2024-03-18T20:48:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe54553b80 1 -- 172.21.15.46:0/2504352683 shutdown_connections 2024-03-18T20:48:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe54553b80 1 --2- 172.21.15.46:0/2504352683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c4415f2220 0x55c4415f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe54553b80 1 --2- 172.21.15.46:0/2504352683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c4415e6a60 0x55c4415f6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe54553b80 1 --2- 172.21.15.46:0/2504352683 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55c44165acb0 0x55c44165d0d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe54553b80 1 --2- 172.21.15.46:0/2504352683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4415e7380 0x55c4415f73a0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe29ffb700 0 client.19725 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:48:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe54553b80 1 -- 172.21.15.46:0/2504352683 >> 172.21.15.46:0/2504352683 conn(0x55c44155d220 msgr2=0x55c4415ee760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.339+0000 7fbe54553b80 1 -- 172.21.15.46:0/2504352683 shutdown_connections 2024-03-18T20:48:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe54553b80 1 -- 172.21.15.46:0/2504352683 wait complete. 2024-03-18T20:48:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe54553b80 20 client.19725 trim_cache size 0 max 16384 2024-03-18T20:48:04.348 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:48:04.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe483be700 1 -- 172.21.15.46:0/418577061 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbe2c0a1290 msgr2=0x7fbe2c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe483be700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbe2c0a1290 0x7fbe2c0a3750 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbe40198760 tx=0x7fbe3c009260 comp rx=0 tx=0).stop 2024-03-18T20:48:04.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe483be700 1 -- 172.21.15.46:0/418577061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe400692a0 msgr2=0x7fbe40197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe483be700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe400692a0 0x7fbe40197210 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fbe30006010 tx=0x7fbe3000b9c0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe483be700 1 -- 172.21.15.46:0/418577061 shutdown_connections 2024-03-18T20:48:04.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe483be700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbe2c0a1290 0x7fbe2c0a3750 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe483be700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe400692a0 0x7fbe40197210 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe483be700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe4019baf0 0x7fbe4019bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe483be700 1 --2- 172.21.15.46:0/418577061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe401060b0 0x7fbe40197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe483be700 1 -- 172.21.15.46:0/418577061 >> 172.21.15.46:0/418577061 conn(0x7fbe40076d80 msgr2=0x7fbe400fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe483be700 1 -- 172.21.15.46:0/418577061 shutdown_connections 2024-03-18T20:48:04.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.340+0000 7fbe483be700 1 -- 172.21.15.46:0/418577061 wait complete. 2024-03-18T20:48:04.356 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:48:04.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:48:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.501+0000 7f779945b700 1 -- 172.21.15.46:0/1931885650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794100f50 msgr2=0x7f7794101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.501+0000 7f779945b700 1 --2- 172.21.15.46:0/1931885650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794100f50 0x7f7794101330 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f7784009990 tx=0x7f7784009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.501+0000 7f779945b700 1 -- 172.21.15.46:0/1931885650 shutdown_connections 2024-03-18T20:48:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.501+0000 7f779945b700 1 --2- 172.21.15.46:0/1931885650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794100f50 0x7f7794101330 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.501+0000 7f779945b700 1 --2- 172.21.15.46:0/1931885650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794106070 0x7f779410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.501+0000 7f779945b700 1 --2- 172.21.15.46:0/1931885650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7794101900 0x7f7794105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.501+0000 7f779945b700 1 -- 172.21.15.46:0/1931885650 >> 172.21.15.46:0/1931885650 conn(0x7f77940fc7f0 msgr2=0x7f77940fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.501+0000 7f779945b700 1 -- 172.21.15.46:0/1931885650 shutdown_connections 2024-03-18T20:48:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.501+0000 7f779945b700 1 -- 172.21.15.46:0/1931885650 wait complete. 2024-03-18T20:48:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.501+0000 7f779945b700 1 Processor -- start 2024-03-18T20:48:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f779945b700 1 -- start start 2024-03-18T20:48:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f779945b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794100f50 0x7f77940730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f779945b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7794106070 0x7f77940735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f779945b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794073b30 0x7f77941a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f779945b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77941a5a00 con 0x7f7794100f50 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f779945b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77941a5b70 con 0x7f7794106070 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f779945b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77941a5d30 con 0x7f7794073b30 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f7792ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794100f50 0x7f77940730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f7792ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794100f50 0x7f77940730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44520/0 (socket says 172.21.15.46:44520) 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f7792ffd700 1 -- 172.21.15.46:0/3328834651 learned_addr learned my addr 172.21.15.46:0/3328834651 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f77927fc700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7794106070 0x7f77940735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f77937fe700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794073b30 0x7f77941a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f7792ffd700 1 -- 172.21.15.46:0/3328834651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794073b30 msgr2=0x7f77941a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f7792ffd700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794073b30 0x7f77941a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f7792ffd700 1 -- 172.21.15.46:0/3328834651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7794106070 msgr2=0x7f77940735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f7792ffd700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7794106070 0x7f77940735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f7792ffd700 1 -- 172.21.15.46:0/3328834651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77840095f0 con 0x7f7794100f50 2024-03-18T20:48:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.502+0000 7f7792ffd700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794100f50 0x7f77940730b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f7784004760 tx=0x7f7784004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.503+0000 7f777bfff700 1 -- 172.21.15.46:0/3328834651 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778401d070 con 0x7f7794100f50 2024-03-18T20:48:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.503+0000 7f777bfff700 1 -- 172.21.15.46:0/3328834651 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7784022470 con 0x7f7794100f50 2024-03-18T20:48:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.503+0000 7f779945b700 1 -- 172.21.15.46:0/3328834651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77941a5fb0 con 0x7f7794100f50 2024-03-18T20:48:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.503+0000 7f777bfff700 1 -- 172.21.15.46:0/3328834651 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784017650 con 0x7f7794100f50 2024-03-18T20:48:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.503+0000 7f779945b700 1 -- 172.21.15.46:0/3328834651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77941a6370 con 0x7f7794100f50 2024-03-18T20:48:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.505+0000 7f777bfff700 1 -- 172.21.15.46:0/3328834651 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f77840225e0 con 0x7f7794100f50 2024-03-18T20:48:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.506+0000 7f777bfff700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f777c0a1290 0x7f777c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.506+0000 7f777bfff700 1 -- 172.21.15.46:0/3328834651 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f77840cded0 con 0x7f7794100f50 2024-03-18T20:48:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.506+0000 7f77927fc700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f777c0a1290 0x7f777c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.506+0000 7f77927fc700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f777c0a1290 0x7f777c0a3750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7794074790 tx=0x7f7788009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77a55f0b80 1 Processor -- start 2024-03-18T20:48:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77a55f0b80 1 -- start start 2024-03-18T20:48:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77a55f0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d998fa60 0x55a1d998fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77a55f0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d9990380 0x55a1d999ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77a55f0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 0x55a1d999d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77a55f0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d98e9990 con 0x55a1d999b220 2024-03-18T20:48:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77a55f0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d98e7350 con 0x55a1d998fa60 2024-03-18T20:48:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77a55f0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d98e2c40 con 0x55a1d9990380 2024-03-18T20:48:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 0x55a1d999d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 0x55a1d999d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44524/0 (socket says 172.21.15.46:44524) 2024-03-18T20:48:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77937fe700 1 -- 172.21.15.46:0/3915592285 learned_addr learned my addr 172.21.15.46:0/3915592285 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77927fc700 1 --2- 172.21.15.46:0/3915592285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d9990380 0x55a1d999ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f7792ffd700 1 --2- 172.21.15.46:0/3915592285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d998fa60 0x55a1d998fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77937fe700 1 -- 172.21.15.46:0/3915592285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d9990380 msgr2=0x55a1d999ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77937fe700 1 --2- 172.21.15.46:0/3915592285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d9990380 0x55a1d999ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77937fe700 1 -- 172.21.15.46:0/3915592285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d998fa60 msgr2=0x55a1d998fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77937fe700 1 --2- 172.21.15.46:0/3915592285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d998fa60 0x55a1d998fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.507+0000 7f77937fe700 1 -- 172.21.15.46:0/3915592285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1d97643b0 con 0x55a1d999b220 2024-03-18T20:48:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.508+0000 7f77937fe700 1 --2- 172.21.15.46:0/3915592285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 0x55a1d999d610 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f778c011680 tx=0x7f778c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2915720b598f459e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.508+0000 7f77797fa700 1 -- 172.21.15.46:0/3915592285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c00e040 con 0x55a1d999b220 2024-03-18T20:48:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.508+0000 7f77797fa700 1 -- 172.21.15.46:0/3915592285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f778c00f040 con 0x55a1d999b220 2024-03-18T20:48:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.508+0000 7f77797fa700 1 -- 172.21.15.46:0/3915592285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c004860 con 0x55a1d999b220 2024-03-18T20:48:04.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.509+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3915592285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 msgr2=0x55a1d999d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.509+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/3915592285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 0x55a1d999d610 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f778c011680 tx=0x7f778c009920 comp rx=0 tx=0).stop 2024-03-18T20:48:04.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.509+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3915592285 shutdown_connections 2024-03-18T20:48:04.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.509+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/3915592285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 0x55a1d999d610 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.509+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/3915592285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d998fa60 0x55a1d998fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.509+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/3915592285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d9990380 0x55a1d999ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.509+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3915592285 >> 172.21.15.46:0/3915592285 conn(0x55a1d9906220 msgr2=0x55a1d998c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.509+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3915592285 shutdown_connections 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.509+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3915592285 wait complete. 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.509+0000 7f77a55f0b80 1 -- start start 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.509+0000 7f77a55f0b80 1 Processor -- start 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77a55f0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d998fa60 0x55a1d999ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77a55f0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 0x55a1d9997f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77a55f0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d999b220 0x55a1d9998590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77a55f0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d987ed80 con 0x55a1d9990380 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77a55f0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d987ea20 con 0x55a1d998fa60 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77a55f0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d97666a0 con 0x55a1d999b220 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77a55f0b80 10 client.0 _ll_register_callbacks cb 0x55a1d96ea3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77927fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 0x55a1d9997f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77927fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 0x55a1d9997f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44534/0 (socket says 172.21.15.46:44534) 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77927fc700 1 -- 172.21.15.46:0/3774699742 learned_addr learned my addr 172.21.15.46:0/3774699742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77937fe700 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d999b220 0x55a1d9998590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f7792ffd700 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d998fa60 0x55a1d999ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77927fc700 1 -- 172.21.15.46:0/3774699742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d999b220 msgr2=0x55a1d9998590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77927fc700 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d999b220 0x55a1d9998590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77927fc700 1 -- 172.21.15.46:0/3774699742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d998fa60 msgr2=0x55a1d999ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77927fc700 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d998fa60 0x55a1d999ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77927fc700 1 -- 172.21.15.46:0/3774699742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778c011330 con 0x55a1d9990380 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f7792ffd700 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d998fa60 0x55a1d999ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77937fe700 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d999b220 0x55a1d9998590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f77927fc700 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 0x55a1d9997f10 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f7788004520 tx=0x7f77880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d13df3cf2fc144b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.510+0000 7f7776ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f7776ff5700 1 -- 172.21.15.46:0/3774699742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788022070 con 0x55a1d9990380 2024-03-18T20:48:04.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f7776ff5700 1 -- 172.21.15.46:0/3774699742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f7788004550 con 0x55a1d9990380 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3774699742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1d9983ad0 con 0x55a1d9990380 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f7776ff5700 1 -- 172.21.15.46:0/3774699742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7788003e20 con 0x55a1d9990380 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f77a55f0b80 10 client.19737 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3774699742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a1d9710b60 con 0x55a1d9990380 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f7776ff5700 1 -- 172.21.15.46:0/3774699742 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7788033080 con 0x55a1d9990380 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f7776ff5700 1 -- 172.21.15.46:0/3774699742 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f778802d030 con 0x55a1d9990380 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f77a55f0b80 10 client.19737 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f77a55f0b80 10 client.19737 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f77a55f0b80 20 client.19737 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a1d99aaf10 0x55a1d99ad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f77a55f0b80 4 client.19737 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.511+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3774699742 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1d98db700 con 0x55a1d99aaf10 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.512+0000 7f7792ffd700 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a1d99aaf10 0x55a1d99ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.512+0000 7f7792ffd700 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a1d99aaf10 0x55a1d99ad330 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9e8aae2acb70f0a server_cookie=795ab02afc5eb227 in_seq=0 out_seq=0 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.512+0000 7f7776ff5700 10 client.19737 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.512+0000 7f7776ff5700 1 -- 172.21.15.46:0/3774699742 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a1d98db700 con 0x55a1d99aaf10 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.512+0000 7f7776ff5700 10 client.19737 handle_command_reply: tid=0 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.512+0000 7f77a55f0b80 1 client.19737 shutdown 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.513+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3774699742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 msgr2=0x55a1d9997f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.513+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 0x55a1d9997f10 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f7788004520 tx=0x7f77880047c0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.513+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3774699742 shutdown_connections 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.513+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d999b220 0x55a1d9998590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.513+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d998fa60 0x55a1d999ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.513+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a1d99aaf10 0x55a1d99ad330 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.513+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/3774699742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 0x55a1d9997f10 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.513+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3774699742 >> 172.21.15.46:0/3774699742 conn(0x55a1d9906220 msgr2=0x55a1d998bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.513+0000 7f7776ff5700 0 client.19737 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.513+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3774699742 shutdown_connections 2024-03-18T20:48:04.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.513+0000 7f77a55f0b80 1 -- 172.21.15.46:0/3774699742 wait complete. 2024-03-18T20:48:04.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.513+0000 7f77a55f0b80 20 client.19737 trim_cache size 0 max 16384 2024-03-18T20:48:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.526+0000 7f77a55f0b80 1 Processor -- start 2024-03-18T20:48:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.526+0000 7f77a55f0b80 1 -- start start 2024-03-18T20:48:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77a55f0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d998fa60 0x55a1d99a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77a55f0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 0x55a1d99931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77a55f0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d999b220 0x55a1d9993720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77a55f0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d9990d30 con 0x55a1d9990380 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77a55f0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d99988d0 con 0x55a1d999b220 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77a55f0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d99a2380 con 0x55a1d998fa60 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77927fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 0x55a1d99931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77927fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 0x55a1d99931e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44550/0 (socket says 172.21.15.46:44550) 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77927fc700 1 -- 172.21.15.46:0/927198086 learned_addr learned my addr 172.21.15.46:0/927198086 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f7792ffd700 1 --2- 172.21.15.46:0/927198086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d998fa60 0x55a1d99a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77937fe700 1 --2- 172.21.15.46:0/927198086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d999b220 0x55a1d9993720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77927fc700 1 -- 172.21.15.46:0/927198086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d998fa60 msgr2=0x55a1d99a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77927fc700 1 --2- 172.21.15.46:0/927198086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d998fa60 0x55a1d99a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77927fc700 1 -- 172.21.15.46:0/927198086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d999b220 msgr2=0x55a1d9993720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77927fc700 1 --2- 172.21.15.46:0/927198086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d999b220 0x55a1d9993720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77927fc700 1 -- 172.21.15.46:0/927198086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1d98a8ea0 con 0x55a1d9990380 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f7792ffd700 1 --2- 172.21.15.46:0/927198086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d998fa60 0x55a1d99a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77937fe700 1 --2- 172.21.15.46:0/927198086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d999b220 0x55a1d9993720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.527+0000 7f77927fc700 1 --2- 172.21.15.46:0/927198086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 0x55a1d99931e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f7788016040 tx=0x7f7788009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3efd0ab9b59d1c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.528+0000 7f7779ffb700 1 -- 172.21.15.46:0/927198086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788022070 con 0x55a1d9990380 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.528+0000 7f7779ffb700 1 -- 172.21.15.46:0/927198086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a1d9983ad0 con 0x55a1d9990380 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.528+0000 7f7779ffb700 1 -- 172.21.15.46:0/927198086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788027420 con 0x55a1d9990380 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.528+0000 7f77a55f0b80 1 -- 172.21.15.46:0/927198086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 msgr2=0x55a1d99931e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.528+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/927198086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 0x55a1d99931e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f7788016040 tx=0x7f7788009390 comp rx=0 tx=0).stop 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.528+0000 7f77a55f0b80 1 -- 172.21.15.46:0/927198086 shutdown_connections 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.528+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/927198086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d999b220 0x55a1d9993720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.528+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/927198086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d998fa60 0x55a1d99a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.528+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/927198086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d9990380 0x55a1d99931e0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.528+0000 7f77a55f0b80 1 -- 172.21.15.46:0/927198086 >> 172.21.15.46:0/927198086 conn(0x55a1d9906220 msgr2=0x55a1d99acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.528+0000 7f77a55f0b80 1 -- 172.21.15.46:0/927198086 shutdown_connections 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.528+0000 7f77a55f0b80 1 -- 172.21.15.46:0/927198086 wait complete. 2024-03-18T20:48:04.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77a55f0b80 1 -- start start 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77a55f0b80 1 Processor -- start 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77a55f0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d998fa60 0x55a1d999de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77a55f0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d9990380 0x55a1d999e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77a55f0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 0x55a1d999e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77a55f0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d99fdd30 con 0x55a1d999b220 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77a55f0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d99fde70 con 0x55a1d9990380 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77a55f0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1d99fdfe0 con 0x55a1d998fa60 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77a55f0b80 10 client.0 _ll_register_callbacks cb 0x55a1d96ea3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 0x55a1d999e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77937fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 0x55a1d999e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44562/0 (socket says 172.21.15.46:44562) 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77937fe700 1 -- 172.21.15.46:0/1573292731 learned_addr learned my addr 172.21.15.46:0/1573292731 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f7792ffd700 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d998fa60 0x55a1d999de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77927fc700 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d9990380 0x55a1d999e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77937fe700 1 -- 172.21.15.46:0/1573292731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d998fa60 msgr2=0x55a1d999de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77937fe700 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d998fa60 0x55a1d999de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77937fe700 1 -- 172.21.15.46:0/1573292731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d9990380 msgr2=0x55a1d999e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77937fe700 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d9990380 0x55a1d999e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.529+0000 7f77937fe700 1 -- 172.21.15.46:0/1573292731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77941a6370 con 0x55a1d999b220 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f77937fe700 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 0x55a1d999e8a0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f778c011620 tx=0x7f778c009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d9839f9f3e1673 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f7776ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f7776ff5700 1 -- 172.21.15.46:0/1573292731 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c00e040 con 0x55a1d999b220 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f7776ff5700 1 -- 172.21.15.46:0/1573292731 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f778c004710 con 0x55a1d999b220 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f77a55f0b80 1 -- 172.21.15.46:0/1573292731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f778c00f040 con 0x55a1d999b220 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f7776ff5700 1 -- 172.21.15.46:0/1573292731 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f778c004230 con 0x55a1d999b220 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f77a55f0b80 10 client.19743 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f77a55f0b80 1 -- 172.21.15.46:0/1573292731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a1d9710b60 con 0x55a1d999b220 2024-03-18T20:48:04.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f7776ff5700 1 -- 172.21.15.46:0/1573292731 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f778c032080 con 0x55a1d999b220 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f7776ff5700 1 -- 172.21.15.46:0/1573292731 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f778c02c030 con 0x55a1d999b220 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f77a55f0b80 10 client.19743 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f77a55f0b80 10 client.19743 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f77a55f0b80 20 client.19743 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a1d99ff6e0 0x55a1d9a01b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f77a55f0b80 4 client.19743 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.530+0000 7f77a55f0b80 1 -- 172.21.15.46:0/1573292731 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a1d9731cf0 con 0x55a1d99ff6e0 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f7792ffd700 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a1d99ff6e0 0x55a1d9a01b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f7792ffd700 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a1d99ff6e0 0x55a1d9a01b00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8620b97f849f2da server_cookie=3fb473bf41653880 in_seq=0 out_seq=0 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f7776ff5700 10 client.19743 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f7776ff5700 1 -- 172.21.15.46:0/1573292731 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a1d9731cf0 con 0x55a1d99ff6e0 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f7776ff5700 10 client.19743 handle_command_reply: tid=0 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f77a55f0b80 1 client.19743 shutdown 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f77a55f0b80 1 -- 172.21.15.46:0/1573292731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 msgr2=0x55a1d999e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 0x55a1d999e8a0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f778c011620 tx=0x7f778c009e70 comp rx=0 tx=0).stop 2024-03-18T20:48:04.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f77a55f0b80 1 -- 172.21.15.46:0/1573292731 shutdown_connections 2024-03-18T20:48:04.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55a1d99ff6e0 0x55a1d9a01b00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1d999b220 0x55a1d999e8a0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a1d998fa60 0x55a1d999de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f7776ff5700 0 client.19743 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:48:04.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f77a55f0b80 1 --2- 172.21.15.46:0/1573292731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a1d9990380 0x55a1d999e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f77a55f0b80 1 -- 172.21.15.46:0/1573292731 >> 172.21.15.46:0/1573292731 conn(0x55a1d9906220 msgr2=0x55a1d99931e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f77a55f0b80 1 -- 172.21.15.46:0/1573292731 shutdown_connections 2024-03-18T20:48:04.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.531+0000 7f77a55f0b80 1 -- 172.21.15.46:0/1573292731 wait complete. 2024-03-18T20:48:04.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f77a55f0b80 20 client.19743 trim_cache size 0 max 16384 2024-03-18T20:48:04.541 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:48:04.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f779945b700 1 -- 172.21.15.46:0/3328834651 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f777c0a1290 msgr2=0x7f777c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f779945b700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f777c0a1290 0x7f777c0a3750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7794074790 tx=0x7f7788009260 comp rx=0 tx=0).stop 2024-03-18T20:48:04.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f779945b700 1 -- 172.21.15.46:0/3328834651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794100f50 msgr2=0x7f77940730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f779945b700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794100f50 0x7f77940730b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f7784004760 tx=0x7f7784004790 comp rx=0 tx=0).stop 2024-03-18T20:48:04.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f779945b700 1 -- 172.21.15.46:0/3328834651 shutdown_connections 2024-03-18T20:48:04.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f779945b700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f777c0a1290 0x7f777c0a3750 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f779945b700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7794100f50 0x7f77940730b0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f779945b700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7794073b30 0x7f77941a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f779945b700 1 --2- 172.21.15.46:0/3328834651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7794106070 0x7f77940735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f779945b700 1 -- 172.21.15.46:0/3328834651 >> 172.21.15.46:0/3328834651 conn(0x7f77940fc7f0 msgr2=0x7f77940fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f779945b700 1 -- 172.21.15.46:0/3328834651 shutdown_connections 2024-03-18T20:48:04.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.532+0000 7f779945b700 1 -- 172.21.15.46:0/3328834651 wait complete. 2024-03-18T20:48:04.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:04 smithi155 ceph-mon[48344]: pgmap v372: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.8 MiB/s wr, 206 op/s 2024-03-18T20:48:04.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:04 smithi155 ceph-mon[48344]: osdmap e93: 12 total, 12 up, 12 in 2024-03-18T20:48:04.544 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:04 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2453098973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:48:04.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:04 smithi046 ceph-mon[43495]: pgmap v372: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.8 MiB/s wr, 206 op/s 2024-03-18T20:48:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:04 smithi046 ceph-mon[43495]: osdmap e93: 12 total, 12 up, 12 in 2024-03-18T20:48:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:04 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2453098973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:48:04.548 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:48:04.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:48:04.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.685+0000 7f1a85387700 1 -- 172.21.15.46:0/3884073980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800ffc40 msgr2=0x7f1a801000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.685+0000 7f1a85387700 1 --2- 172.21.15.46:0/3884073980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800ffc40 0x7f1a801000c0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009990 tx=0x7f1a74009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.685+0000 7f1a85387700 1 -- 172.21.15.46:0/3884073980 shutdown_connections 2024-03-18T20:48:04.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.685+0000 7f1a85387700 1 --2- 172.21.15.46:0/3884073980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a80100600 0x7f1a8010caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.685+0000 7f1a85387700 1 --2- 172.21.15.46:0/3884073980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a80105d00 0x7f1a801060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.685+0000 7f1a85387700 1 --2- 172.21.15.46:0/3884073980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800ffc40 0x7f1a801000c0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.685+0000 7f1a85387700 1 -- 172.21.15.46:0/3884073980 >> 172.21.15.46:0/3884073980 conn(0x7f1a80076de0 msgr2=0x7f1a800771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.685+0000 7f1a85387700 1 -- 172.21.15.46:0/3884073980 shutdown_connections 2024-03-18T20:48:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.685+0000 7f1a85387700 1 -- 172.21.15.46:0/3884073980 wait complete. 2024-03-18T20:48:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.686+0000 7f1a85387700 1 Processor -- start 2024-03-18T20:48:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.686+0000 7f1a85387700 1 -- start start 2024-03-18T20:48:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.686+0000 7f1a85387700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800ffc40 0x7f1a80197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.686+0000 7f1a85387700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a80100600 0x7f1a80197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.686+0000 7f1a85387700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a80105d00 0x7f1a8019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.686+0000 7f1a85387700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8019c040 con 0x7f1a800ffc40 2024-03-18T20:48:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.686+0000 7f1a85387700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8019c1b0 con 0x7f1a80105d00 2024-03-18T20:48:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.686+0000 7f1a85387700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8019c370 con 0x7f1a80100600 2024-03-18T20:48:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.686+0000 7f1a7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800ffc40 0x7f1a80197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.686+0000 7f1a7e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a80100600 0x7f1a80197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.686+0000 7f1a7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800ffc40 0x7f1a80197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44580/0 (socket says 172.21.15.46:44580) 2024-03-18T20:48:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.686+0000 7f1a7effd700 1 -- 172.21.15.46:0/3813042972 learned_addr learned my addr 172.21.15.46:0/3813042972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a7f7fe700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a80105d00 0x7f1a8019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a7effd700 1 -- 172.21.15.46:0/3813042972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a80100600 msgr2=0x7f1a80197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a7effd700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a80100600 0x7f1a80197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a7effd700 1 -- 172.21.15.46:0/3813042972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a80105d00 msgr2=0x7f1a8019ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a7effd700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a80105d00 0x7f1a8019ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a7effd700 1 -- 172.21.15.46:0/3813042972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70009520 con 0x7f1a800ffc40 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a80100600 0x7f1a80197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a7effd700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800ffc40 0x7f1a80197120 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000eb30 tx=0x7f1a7000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a67fff700 1 -- 172.21.15.46:0/3813042972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70009880 con 0x7f1a800ffc40 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a85387700 1 -- 172.21.15.46:0/3813042972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a740095f0 con 0x7f1a800ffc40 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a67fff700 1 -- 172.21.15.46:0/3813042972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a700044e0 con 0x7f1a800ffc40 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a67fff700 1 -- 172.21.15.46:0/3813042972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70003d80 con 0x7f1a800ffc40 2024-03-18T20:48:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.687+0000 7f1a85387700 1 -- 172.21.15.46:0/3813042972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a801a3cb0 con 0x7f1a800ffc40 2024-03-18T20:48:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.690+0000 7f1a67fff700 1 -- 172.21.15.46:0/3813042972 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1a70004000 con 0x7f1a800ffc40 2024-03-18T20:48:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.691+0000 7f1a67fff700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1a680a1290 0x7f1a680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.691+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1a680a1290 0x7f1a680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.691+0000 7f1a67fff700 1 -- 172.21.15.46:0/3813042972 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f1a70014070 con 0x7f1a800ffc40 2024-03-18T20:48:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.691+0000 7f1a9151cb80 1 Processor -- start 2024-03-18T20:48:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.691+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1a680a1290 0x7f1a680a3750 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400b2e0 tx=0x7f1a74005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.691+0000 7f1a9151cb80 1 -- start start 2024-03-18T20:48:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a9151cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abaa60 0x561234abae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a9151cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abb380 0x561234ac5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a9151cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234ac6220 0x561234ac8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a9151cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561234a14990 con 0x561234abaa60 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a9151cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561234a12350 con 0x561234ac6220 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a9151cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561234a0dc40 con 0x561234abb380 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abaa60 0x561234abae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a7effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abaa60 0x561234abae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44590/0 (socket says 172.21.15.46:44590) 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a7effd700 1 -- 172.21.15.46:0/733888052 learned_addr learned my addr 172.21.15.46:0/733888052 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a7f7fe700 1 --2- 172.21.15.46:0/733888052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234ac6220 0x561234ac8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/733888052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abb380 0x561234ac5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a7effd700 1 -- 172.21.15.46:0/733888052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abb380 msgr2=0x561234ac5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a7effd700 1 --2- 172.21.15.46:0/733888052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abb380 0x561234ac5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a7effd700 1 -- 172.21.15.46:0/733888052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234ac6220 msgr2=0x561234ac8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a7effd700 1 --2- 172.21.15.46:0/733888052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234ac6220 0x561234ac8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a7effd700 1 -- 172.21.15.46:0/733888052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56123488f3b0 con 0x561234abaa60 2024-03-18T20:48:04.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/733888052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abb380 0x561234ac5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a7effd700 1 --2- 172.21.15.46:0/733888052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abaa60 0x561234abae40 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f1a70096d90 tx=0x7f1a70099ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683dd67fc6a4f286 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.692+0000 7f1a657fa700 1 -- 172.21.15.46:0/733888052 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7009aa20 con 0x561234abaa60 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.693+0000 7f1a657fa700 1 -- 172.21.15.46:0/733888052 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a7009a320 con 0x561234abaa60 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.693+0000 7f1a657fa700 1 -- 172.21.15.46:0/733888052 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7009a580 con 0x561234abaa60 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.693+0000 7f1a9151cb80 1 -- 172.21.15.46:0/733888052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abaa60 msgr2=0x561234abae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.693+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/733888052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abaa60 0x561234abae40 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f1a70096d90 tx=0x7f1a70099ee0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.693+0000 7f1a9151cb80 1 -- 172.21.15.46:0/733888052 shutdown_connections 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.693+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/733888052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abaa60 0x561234abae40 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.693+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/733888052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234ac6220 0x561234ac8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.693+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/733888052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abb380 0x561234ac5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.693+0000 7f1a9151cb80 1 -- 172.21.15.46:0/733888052 >> 172.21.15.46:0/733888052 conn(0x561234a31220 msgr2=0x561234ab7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.693+0000 7f1a9151cb80 1 -- 172.21.15.46:0/733888052 shutdown_connections 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.693+0000 7f1a9151cb80 1 -- 172.21.15.46:0/733888052 wait complete. 2024-03-18T20:48:04.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a9151cb80 1 -- start start 2024-03-18T20:48:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a9151cb80 1 Processor -- start 2024-03-18T20:48:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a9151cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 0x561234acfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a9151cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234abb380 0x561234ad02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a9151cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234ac6220 0x561234aca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a9151cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612349a9d80 con 0x561234ac6220 2024-03-18T20:48:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a9151cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612349a9a20 con 0x561234abb380 2024-03-18T20:48:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a9151cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612348916a0 con 0x561234abaa60 2024-03-18T20:48:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a9151cb80 10 client.0 _ll_register_callbacks cb 0x5612348153a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a7effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 0x561234acfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a7f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234ac6220 0x561234aca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a7effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 0x561234acfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:35154/0 (socket says 172.21.15.46:35154) 2024-03-18T20:48:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a7f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234ac6220 0x561234aca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44600/0 (socket says 172.21.15.46:44600) 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a7effd700 1 -- 172.21.15.46:0/2547124780 learned_addr learned my addr 172.21.15.46:0/2547124780 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.694+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234abb380 0x561234ad02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a7effd700 1 -- 172.21.15.46:0/2547124780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234abb380 msgr2=0x561234ad02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a7effd700 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234abb380 0x561234ad02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a7effd700 1 -- 172.21.15.46:0/2547124780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234ac6220 msgr2=0x561234aca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a7effd700 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234ac6220 0x561234aca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a7effd700 1 -- 172.21.15.46:0/2547124780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a801a3cb0 con 0x561234abaa60 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a7f7fe700 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234ac6220 0x561234aca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a7effd700 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 0x561234acfd60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1a700caee0 tx=0x7f1a70003760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed9658749cf76ce server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a62ff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a62ff5700 1 -- 172.21.15.46:0/2547124780 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7009e150 con 0x561234abaa60 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a62ff5700 1 -- 172.21.15.46:0/2547124780 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f1a7000cdd0 con 0x561234abaa60 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a9151cb80 1 -- 172.21.15.46:0/2547124780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561234aaead0 con 0x561234abaa60 2024-03-18T20:48:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a62ff5700 1 -- 172.21.15.46:0/2547124780 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a7001fde0 con 0x561234abaa60 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a9151cb80 10 client.29398 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a9151cb80 1 -- 172.21.15.46:0/2547124780 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x56123483bb60 con 0x561234abaa60 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.695+0000 7f1a62ff5700 1 -- 172.21.15.46:0/2547124780 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561234aaead0 con 0x561234abaa60 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.696+0000 7f1a62ff5700 1 -- 172.21.15.46:0/2547124780 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1a7009a680 con 0x561234abaa60 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.696+0000 7f1a9151cb80 10 client.29398 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.696+0000 7f1a9151cb80 10 client.29398 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.696+0000 7f1a9151cb80 20 client.29398 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.696+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561234ad9eb0 0x561234adc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.696+0000 7f1a9151cb80 4 client.29398 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.696+0000 7f1a9151cb80 1 -- 172.21.15.46:0/2547124780 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561234a06700 con 0x561234ad9eb0 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.696+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561234ad9eb0 0x561234adc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.696+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561234ad9eb0 0x561234adc2d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff267478eebb6dd5 server_cookie=3d8c125077dd3f33 in_seq=0 out_seq=0 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.696+0000 7f1a62ff5700 10 client.29398 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:48:04.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a62ff5700 1 -- 172.21.15.46:0/2547124780 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561234a06700 con 0x561234ad9eb0 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a62ff5700 10 client.29398 handle_command_reply: tid=0 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a9151cb80 1 client.29398 shutdown 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a9151cb80 1 -- 172.21.15.46:0/2547124780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 msgr2=0x561234acfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 0x561234acfd60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1a700caee0 tx=0x7f1a70003760 comp rx=0 tx=0).stop 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a9151cb80 1 -- 172.21.15.46:0/2547124780 shutdown_connections 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561234ad9eb0 0x561234adc2d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234ac6220 0x561234aca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 0x561234acfd60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a62ff5700 0 client.29398 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/2547124780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234abb380 0x561234ad02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a9151cb80 1 -- 172.21.15.46:0/2547124780 >> 172.21.15.46:0/2547124780 conn(0x561234a31220 msgr2=0x561234ab6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a9151cb80 1 -- 172.21.15.46:0/2547124780 shutdown_connections 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a9151cb80 1 -- 172.21.15.46:0/2547124780 wait complete. 2024-03-18T20:48:04.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.697+0000 7f1a9151cb80 20 client.29398 trim_cache size 0 max 16384 2024-03-18T20:48:04.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a9151cb80 1 Processor -- start 2024-03-18T20:48:04.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a9151cb80 1 -- start start 2024-03-18T20:48:04.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a9151cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234abaa60 0x561234ac5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a9151cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 0x561234ac2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a9151cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234ac6220 0x561234ac2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a9151cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561234ab0f80 con 0x561234abb380 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a9151cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561234abc3d0 con 0x561234abaa60 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a9151cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561234ac5960 con 0x561234ac6220 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a7e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 0x561234ac2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a7e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 0x561234ac2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44606/0 (socket says 172.21.15.46:44606) 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a7e7fc700 1 -- 172.21.15.46:0/1613178094 learned_addr learned my addr 172.21.15.46:0/1613178094 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a7f7fe700 1 --2- 172.21.15.46:0/1613178094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234ac6220 0x561234ac2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a7effd700 1 --2- 172.21.15.46:0/1613178094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234abaa60 0x561234ac5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a7e7fc700 1 -- 172.21.15.46:0/1613178094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234ac6220 msgr2=0x561234ac2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/1613178094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234ac6220 0x561234ac2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a7e7fc700 1 -- 172.21.15.46:0/1613178094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234abaa60 msgr2=0x561234ac5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/1613178094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234abaa60 0x561234ac5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.711+0000 7f1a7e7fc700 1 -- 172.21.15.46:0/1613178094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612349d3ea0 con 0x561234abb380 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/1613178094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 0x561234ac2730 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009990 tx=0x7f1a74023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39eecf99f37e83d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a65ffb700 1 -- 172.21.15.46:0/1613178094 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74005020 con 0x561234abb380 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a65ffb700 1 -- 172.21.15.46:0/1613178094 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a7402c040 con 0x561234abb380 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a65ffb700 1 -- 172.21.15.46:0/1613178094 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7400fe30 con 0x561234abb380 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a9151cb80 1 -- 172.21.15.46:0/1613178094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 msgr2=0x561234ac2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/1613178094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 0x561234ac2730 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009990 tx=0x7f1a74023000 comp rx=0 tx=0).stop 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a9151cb80 1 -- 172.21.15.46:0/1613178094 shutdown_connections 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/1613178094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234ac6220 0x561234ac2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/1613178094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234abaa60 0x561234ac5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/1613178094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 0x561234ac2730 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a9151cb80 1 -- 172.21.15.46:0/1613178094 >> 172.21.15.46:0/1613178094 conn(0x561234a31220 msgr2=0x561234ac8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a9151cb80 1 -- 172.21.15.46:0/1613178094 shutdown_connections 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.712+0000 7f1a9151cb80 1 -- 172.21.15.46:0/1613178094 wait complete. 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a9151cb80 1 -- start start 2024-03-18T20:48:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a9151cb80 1 Processor -- start 2024-03-18T20:48:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a9151cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 0x561234ad40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a9151cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 0x561234ad4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a9151cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234ac6220 0x561234ad4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a9151cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561234b2cca0 con 0x561234abb380 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a9151cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561234b2cde0 con 0x561234ac6220 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a9151cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561234b2cf20 con 0x561234abaa60 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a9151cb80 10 client.0 _ll_register_callbacks cb 0x7f1a78009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a7e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 0x561234ad4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a7effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 0x561234ad40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a7effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 0x561234ad40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:35176/0 (socket says 172.21.15.46:35176) 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a7effd700 1 -- 172.21.15.46:0/4072413017 learned_addr learned my addr 172.21.15.46:0/4072413017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.713+0000 7f1a7f7fe700 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234ac6220 0x561234ad4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a7effd700 1 -- 172.21.15.46:0/4072413017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234ac6220 msgr2=0x561234ad4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a7effd700 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234ac6220 0x561234ad4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a7effd700 1 -- 172.21.15.46:0/4072413017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 msgr2=0x561234ad4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a7effd700 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 0x561234ad4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a7effd700 1 -- 172.21.15.46:0/4072413017 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8010d390 con 0x561234abaa60 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 0x561234ad4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a7effd700 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 0x561234ad40e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a70000c00 tx=0x7f1a70098980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bebc0cbf5cfe653 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a62ff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a62ff5700 1 -- 172.21.15.46:0/4072413017 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a700a24d0 con 0x561234abaa60 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a62ff5700 1 -- 172.21.15.46:0/4072413017 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f1a7000cdd0 con 0x561234abaa60 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a9151cb80 1 -- 172.21.15.46:0/4072413017 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a7009a320 con 0x561234abaa60 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a62ff5700 1 -- 172.21.15.46:0/4072413017 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a70099c80 con 0x561234abaa60 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a9151cb80 10 client.29404 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a9151cb80 1 -- 172.21.15.46:0/4072413017 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x561234893cd0 con 0x561234abaa60 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.714+0000 7f1a62ff5700 1 -- 172.21.15.46:0/4072413017 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a7009a320 con 0x561234abaa60 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a62ff5700 1 -- 172.21.15.46:0/4072413017 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1a70096110 con 0x561234abaa60 2024-03-18T20:48:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a9151cb80 10 client.29404 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a9151cb80 10 client.29404 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a9151cb80 20 client.29404 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561234b2e190 0x561234b30580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a9151cb80 4 client.29404 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a9151cb80 1 -- 172.21.15.46:0/4072413017 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56123485ccf0 con 0x561234b2e190 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561234b2e190 0x561234b30580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a62ff5700 10 client.29404 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a7e7fc700 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561234b2e190 0x561234b30580 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2a52f522af96823 server_cookie=9ece282ce6291272 in_seq=0 out_seq=0 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a62ff5700 1 -- 172.21.15.46:0/4072413017 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56123485ccf0 con 0x561234b2e190 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a62ff5700 10 client.29404 handle_command_reply: tid=0 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.715+0000 7f1a9151cb80 1 client.29404 shutdown 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a9151cb80 1 -- 172.21.15.46:0/4072413017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 msgr2=0x561234ad40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 0x561234ad40e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a70000c00 tx=0x7f1a70098980 comp rx=0 tx=0).stop 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a9151cb80 1 -- 172.21.15.46:0/4072413017 shutdown_connections 2024-03-18T20:48:04.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x561234b2e190 0x561234b30580 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561234ac6220 0x561234ad4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a62ff5700 0 client.29404 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:48:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561234abaa60 0x561234ad40e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a9151cb80 1 --2- 172.21.15.46:0/4072413017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561234abb380 0x561234ad4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a9151cb80 1 -- 172.21.15.46:0/4072413017 >> 172.21.15.46:0/4072413017 conn(0x561234a31220 msgr2=0x561234ac2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a9151cb80 1 -- 172.21.15.46:0/4072413017 shutdown_connections 2024-03-18T20:48:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a9151cb80 1 -- 172.21.15.46:0/4072413017 wait complete. 2024-03-18T20:48:04.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a9151cb80 20 client.29404 trim_cache size 0 max 16384 2024-03-18T20:48:04.725 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:48:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a85387700 1 -- 172.21.15.46:0/3813042972 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1a680a1290 msgr2=0x7f1a680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a85387700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1a680a1290 0x7f1a680a3750 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400b2e0 tx=0x7f1a74005960 comp rx=0 tx=0).stop 2024-03-18T20:48:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a85387700 1 -- 172.21.15.46:0/3813042972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800ffc40 msgr2=0x7f1a80197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a85387700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800ffc40 0x7f1a80197120 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000eb30 tx=0x7f1a7000ee40 comp rx=0 tx=0).stop 2024-03-18T20:48:04.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a85387700 1 -- 172.21.15.46:0/3813042972 shutdown_connections 2024-03-18T20:48:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a85387700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1a680a1290 0x7f1a680a3750 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a85387700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a800ffc40 0x7f1a80197120 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a85387700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a80105d00 0x7f1a8019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a85387700 1 --2- 172.21.15.46:0/3813042972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a80100600 0x7f1a80197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.716+0000 7f1a85387700 1 -- 172.21.15.46:0/3813042972 >> 172.21.15.46:0/3813042972 conn(0x7f1a80076de0 msgr2=0x7f1a80069570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.717+0000 7f1a85387700 1 -- 172.21.15.46:0/3813042972 shutdown_connections 2024-03-18T20:48:04.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.717+0000 7f1a85387700 1 -- 172.21.15.46:0/3813042972 wait complete. 2024-03-18T20:48:04.727 INFO:tasks.workunit.client.0.smithi046.stdout: 1 455730 3.73 MB/sec execute 539 sec latency 1024.359 ms 2024-03-18T20:48:04.727 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:04 smithi073 ceph-mon[47384]: pgmap v372: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.8 MiB/s wr, 206 op/s 2024-03-18T20:48:04.728 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:04 smithi073 ceph-mon[47384]: osdmap e93: 12 total, 12 up, 12 in 2024-03-18T20:48:04.728 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:04 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2453098973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:48:04.733 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:48:04.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:48:04.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.873+0000 7f2da3d1c700 1 -- 172.21.15.46:0/1877533714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d9c104f60 msgr2=0x7f2d9c10cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.873+0000 7f2da3d1c700 1 --2- 172.21.15.46:0/1877533714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d9c104f60 0x7f2d9c10cae0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400b0a0 tx=0x7f2d9400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.873+0000 7f2da3d1c700 1 -- 172.21.15.46:0/1877533714 shutdown_connections 2024-03-18T20:48:04.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.873+0000 7f2da3d1c700 1 --2- 172.21.15.46:0/1877533714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d9c104f60 0x7f2d9c10cae0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.873+0000 7f2da3d1c700 1 --2- 172.21.15.46:0/1877533714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9c0ffe50 0x7f2d9c100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.873+0000 7f2da3d1c700 1 --2- 172.21.15.46:0/1877533714 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d9c100800 0x7f2d9c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.873+0000 7f2da3d1c700 1 -- 172.21.15.46:0/1877533714 >> 172.21.15.46:0/1877533714 conn(0x7f2d9c076d40 msgr2=0x7f2d9c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.873+0000 7f2da3d1c700 1 -- 172.21.15.46:0/1877533714 shutdown_connections 2024-03-18T20:48:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.873+0000 7f2da3d1c700 1 -- 172.21.15.46:0/1877533714 wait complete. 2024-03-18T20:48:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.873+0000 7f2da3d1c700 1 Processor -- start 2024-03-18T20:48:04.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da3d1c700 1 -- start start 2024-03-18T20:48:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da3d1c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9c0ffe50 0x7f2d9c10d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da3d1c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d9c104f60 0x7f2d9c10d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da3d1c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d9c10dea0 0x7f2d9c1a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da3d1c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c1a5ae0 con 0x7f2d9c104f60 2024-03-18T20:48:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da3d1c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c1a5c50 con 0x7f2d9c0ffe50 2024-03-18T20:48:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da3d1c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c1a5e10 con 0x7f2d9c10dea0 2024-03-18T20:48:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da1ab8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9c0ffe50 0x7f2d9c10d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da1ab8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9c0ffe50 0x7f2d9c10d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:49458/0 (socket says 172.21.15.46:49458) 2024-03-18T20:48:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da12b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d9c104f60 0x7f2d9c10d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da1ab8700 1 -- 172.21.15.46:0/3408805656 learned_addr learned my addr 172.21.15.46:0/3408805656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da22b9700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d9c10dea0 0x7f2d9c1a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da12b7700 1 -- 172.21.15.46:0/3408805656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d9c10dea0 msgr2=0x7f2d9c1a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da12b7700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d9c10dea0 0x7f2d9c1a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da12b7700 1 -- 172.21.15.46:0/3408805656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9c0ffe50 msgr2=0x7f2d9c10d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da12b7700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9c0ffe50 0x7f2d9c10d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da12b7700 1 -- 172.21.15.46:0/3408805656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d94009d00 con 0x7f2d9c104f60 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da22b9700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d9c10dea0 0x7f2d9c1a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.874+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9c0ffe50 0x7f2d9c10d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.875+0000 7f2da12b7700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d9c104f60 0x7f2d9c10d960 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800e9c0 tx=0x7f2d9800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.875+0000 7f2d8effd700 1 -- 172.21.15.46:0/3408805656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9800c9c0 con 0x7f2d9c104f60 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.875+0000 7f2d8effd700 1 -- 172.21.15.46:0/3408805656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d98010430 con 0x7f2d9c104f60 2024-03-18T20:48:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.875+0000 7f2da3d1c700 1 -- 172.21.15.46:0/3408805656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9c1a60f0 con 0x7f2d9c104f60 2024-03-18T20:48:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.875+0000 7f2d8effd700 1 -- 172.21.15.46:0/3408805656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98018680 con 0x7f2d9c104f60 2024-03-18T20:48:04.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.875+0000 7f2da3d1c700 1 -- 172.21.15.46:0/3408805656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9c1a6650 con 0x7f2d9c104f60 2024-03-18T20:48:04.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.877+0000 7f2d8effd700 1 -- 172.21.15.46:0/3408805656 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2d9800cb20 con 0x7f2d9c104f60 2024-03-18T20:48:04.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.878+0000 7f2d8effd700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2d880a1310 0x7f2d880a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.878+0000 7f2d8effd700 1 -- 172.21.15.46:0/3408805656 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f2d98014070 con 0x7f2d9c104f60 2024-03-18T20:48:04.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.879+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2d880a1310 0x7f2d880a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.879+0000 7f2dafeb1b80 1 Processor -- start 2024-03-18T20:48:04.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.879+0000 7f2dafeb1b80 1 -- start start 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.879+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2d880a1310 0x7f2d880a37d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2d900104e0 tx=0x7f2d90007480 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.879+0000 7f2dafeb1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e0a60 0x562e385e0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.879+0000 7f2dafeb1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e1380 0x562e385ebce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2dafeb1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 0x562e385ee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2dafeb1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3853a990 con 0x562e385e1380 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2dafeb1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e38538350 con 0x562e385e0a60 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2dafeb1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e38533c40 con 0x562e385ec220 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2da12b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e1380 0x562e385ebce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2da1ab8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e0a60 0x562e385e0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2da12b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e1380 0x562e385ebce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44650/0 (socket says 172.21.15.46:44650) 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2da12b7700 1 -- 172.21.15.46:0/2541089405 learned_addr learned my addr 172.21.15.46:0/2541089405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2da22b9700 1 --2- 172.21.15.46:0/2541089405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 0x562e385ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2da12b7700 1 -- 172.21.15.46:0/2541089405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 msgr2=0x562e385ee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2da12b7700 1 --2- 172.21.15.46:0/2541089405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 0x562e385ee610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2da12b7700 1 -- 172.21.15.46:0/2541089405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e0a60 msgr2=0x562e385e0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2da12b7700 1 --2- 172.21.15.46:0/2541089405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e0a60 0x562e385e0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2da12b7700 1 -- 172.21.15.46:0/2541089405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e383b53b0 con 0x562e385e1380 2024-03-18T20:48:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/2541089405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e0a60 0x562e385e0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2da12b7700 1 --2- 172.21.15.46:0/2541089405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e1380 0x562e385ebce0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009230 tx=0x7f2d980999d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34ef99f9e9481e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2d87fff700 1 -- 172.21.15.46:0/2541089405 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98099e10 con 0x562e385e1380 2024-03-18T20:48:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2d87fff700 1 -- 172.21.15.46:0/2541089405 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d9809dc40 con 0x562e385e1380 2024-03-18T20:48:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.880+0000 7f2d87fff700 1 -- 172.21.15.46:0/2541089405 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9809a070 con 0x562e385e1380 2024-03-18T20:48:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.881+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2541089405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e1380 msgr2=0x562e385ebce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.881+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/2541089405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e1380 0x562e385ebce0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009230 tx=0x7f2d980999d0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.881+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2541089405 shutdown_connections 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.881+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/2541089405 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 0x562e385ee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.881+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/2541089405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e0a60 0x562e385e0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.881+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/2541089405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e1380 0x562e385ebce0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.881+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2541089405 >> 172.21.15.46:0/2541089405 conn(0x562e38557220 msgr2=0x562e385dd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.882+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2541089405 shutdown_connections 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.882+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2541089405 wait complete. 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.882+0000 7f2dafeb1b80 1 -- start start 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.882+0000 7f2dafeb1b80 1 Processor -- start 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2dafeb1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 0x562e385e8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2dafeb1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2dafeb1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 0x562e385e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2dafeb1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e384cfd80 con 0x562e385e0a60 2024-03-18T20:48:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2dafeb1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e384cfa20 con 0x562e385e1380 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2dafeb1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e383b76a0 con 0x562e385ec220 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2dafeb1b80 10 client.0 _ll_register_callbacks cb 0x562e3833b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2da1ab8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 0x562e385e8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2da1ab8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 0x562e385e8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44652/0 (socket says 172.21.15.46:44652) 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2da1ab8700 1 -- 172.21.15.46:0/2175263001 learned_addr learned my addr 172.21.15.46:0/2175263001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2da12b7700 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2da22b9700 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 0x562e385e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2da1ab8700 1 -- 172.21.15.46:0/2175263001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 msgr2=0x562e385e99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 0x562e385e99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2da1ab8700 1 -- 172.21.15.46:0/2175263001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 msgr2=0x562e385e94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385e94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2da1ab8700 1 -- 172.21.15.46:0/2175263001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d90011440 con 0x562e385e0a60 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2da12b7700 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 0x562e385e8f70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2d90011b20 tx=0x7f2d900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203341d0897f5c19 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.883+0000 7f2d857fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2d857fa700 1 -- 172.21.15.46:0/2175263001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9001e6c0 con 0x562e385e0a60 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2d857fa700 1 -- 172.21.15.46:0/2175263001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f2d90017070 con 0x562e385e0a60 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2175263001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e385d4ad0 con 0x562e385e0a60 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2d857fa700 1 -- 172.21.15.46:0/2175263001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d9001e820 con 0x562e385e0a60 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2dafeb1b80 10 client.19773 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2175263001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f2d9c1a6650 con 0x562e385e0a60 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2d857fa700 1 -- 172.21.15.46:0/2175263001 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e385d4ad0 con 0x562e385e0a60 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2d857fa700 1 -- 172.21.15.46:0/2175263001 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2d90032a50 con 0x562e385e0a60 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2dafeb1b80 10 client.19773 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2dafeb1b80 10 client.19773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2dafeb1b80 20 client.19773 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562e385fbe50 0x562e385fe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2dafeb1b80 4 client.19773 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.884+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2175263001 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e3852c700 con 0x562e385fbe50 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.885+0000 7f2da12b7700 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562e385fbe50 0x562e385fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.885+0000 7f2da12b7700 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562e385fbe50 0x562e385fe270 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80b4b765ae3ebab server_cookie=bf44c9827d02d89b in_seq=0 out_seq=0 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.885+0000 7f2d857fa700 10 client.19773 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2d857fa700 1 -- 172.21.15.46:0/2175263001 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562e3852c700 con 0x562e385fbe50 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2d857fa700 10 client.19773 handle_command_reply: tid=0 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2dafeb1b80 1 client.19773 shutdown 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2175263001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 msgr2=0x562e385e8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 0x562e385e8f70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2d90011b20 tx=0x7f2d900047c0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2175263001 shutdown_connections 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562e385fbe50 0x562e385fe270 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 0x562e385e8f70 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 0x562e385e99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/2175263001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2d857fa700 0 client.19773 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:04.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2175263001 >> 172.21.15.46:0/2175263001 conn(0x562e38557220 msgr2=0x562e385dcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2175263001 shutdown_connections 2024-03-18T20:48:04.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/2175263001 wait complete. 2024-03-18T20:48:04.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.886+0000 7f2dafeb1b80 20 client.19773 trim_cache size 0 max 16384 2024-03-18T20:48:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.901+0000 7f2dafeb1b80 1 Processor -- start 2024-03-18T20:48:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.901+0000 7f2dafeb1b80 1 -- start start 2024-03-18T20:48:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2dafeb1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385e0a60 0x562e385f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2dafeb1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385e9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2dafeb1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385ec220 0x562e385e42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2dafeb1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e385e1d30 con 0x562e385ec220 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2dafeb1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e385d95e0 con 0x562e385e1380 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2dafeb1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e385e1af0 con 0x562e385e0a60 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da22b9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385ec220 0x562e385e42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da1ab8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385e0a60 0x562e385f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da12b7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da1ab8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385e0a60 0x562e385f2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:35204/0 (socket says 172.21.15.46:35204) 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da1ab8700 1 -- 172.21.15.46:0/945542711 learned_addr learned my addr 172.21.15.46:0/945542711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da22b9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385ec220 0x562e385e42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44656/0 (socket says 172.21.15.46:44656) 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da1ab8700 1 -- 172.21.15.46:0/945542711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 msgr2=0x562e385e9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/945542711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385e9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da1ab8700 1 -- 172.21.15.46:0/945542711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385ec220 msgr2=0x562e385e42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/945542711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385ec220 0x562e385e42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da1ab8700 1 -- 172.21.15.46:0/945542711 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e38361b60 con 0x562e385e0a60 2024-03-18T20:48:04.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da22b9700 1 --2- 172.21.15.46:0/945542711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385ec220 0x562e385e42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.902+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/945542711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385e0a60 0x562e385f2df0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d90011640 tx=0x7f2d90010c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a0dc27ff2830b8f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.903+0000 7f2d8cff9700 1 -- 172.21.15.46:0/945542711 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90028790 con 0x562e385e0a60 2024-03-18T20:48:04.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.903+0000 7f2d8cff9700 1 -- 172.21.15.46:0/945542711 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d90023070 con 0x562e385e0a60 2024-03-18T20:48:04.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.903+0000 7f2d8cff9700 1 -- 172.21.15.46:0/945542711 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90032af0 con 0x562e385e0a60 2024-03-18T20:48:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.903+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/945542711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385e0a60 msgr2=0x562e385f2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.903+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/945542711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385e0a60 0x562e385f2df0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d90011640 tx=0x7f2d90010c50 comp rx=0 tx=0).stop 2024-03-18T20:48:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.903+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/945542711 shutdown_connections 2024-03-18T20:48:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.903+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/945542711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385ec220 0x562e385e42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.903+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/945542711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385e0a60 0x562e385f2df0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.903+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/945542711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385e9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.903+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/945542711 >> 172.21.15.46:0/945542711 conn(0x562e38557220 msgr2=0x562e385dcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.904+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/945542711 shutdown_connections 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.904+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/945542711 wait complete. 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.904+0000 7f2dafeb1b80 1 -- start start 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.904+0000 7f2dafeb1b80 1 Processor -- start 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.904+0000 7f2dafeb1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 0x562e385f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.904+0000 7f2dafeb1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385f0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.904+0000 7f2dafeb1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 0x562e3864ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.904+0000 7f2dafeb1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e385f1670 con 0x562e385e0a60 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.904+0000 7f2dafeb1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3864f1c0 con 0x562e385e1380 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.904+0000 7f2dafeb1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e3864f380 con 0x562e385ec220 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2dafeb1b80 10 client.0 _ll_register_callbacks cb 0x7f2d90011700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2da1ab8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 0x562e385f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2da12b7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385f0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2da1ab8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 0x562e385f09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44662/0 (socket says 172.21.15.46:44662) 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2da1ab8700 1 -- 172.21.15.46:0/463525073 learned_addr learned my addr 172.21.15.46:0/463525073 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2da22b9700 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 0x562e3864ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2da1ab8700 1 -- 172.21.15.46:0/463525073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 msgr2=0x562e3864ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 0x562e3864ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2da1ab8700 1 -- 172.21.15.46:0/463525073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 msgr2=0x562e385f0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385f0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2da1ab8700 1 -- 172.21.15.46:0/463525073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9c067df0 con 0x562e385e0a60 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2da12b7700 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385f0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2da1ab8700 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 0x562e385f09e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2d90005a60 tx=0x7f2d90032770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a5a34fcc759eaf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2d857fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.905+0000 7f2d857fa700 1 -- 172.21.15.46:0/463525073 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90044440 con 0x562e385e0a60 2024-03-18T20:48:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2d857fa700 1 -- 172.21.15.46:0/463525073 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f2d90017070 con 0x562e385e0a60 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/463525073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d9809dc40 con 0x562e385e0a60 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2d857fa700 1 -- 172.21.15.46:0/463525073 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d900445a0 con 0x562e385e0a60 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2dafeb1b80 10 client.19779 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/463525073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f2d9c1a6650 con 0x562e385e0a60 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2d857fa700 1 -- 172.21.15.46:0/463525073 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d9004f020 con 0x562e385e0a60 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2d857fa700 1 -- 172.21.15.46:0/463525073 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2d9001f8a0 con 0x562e385e0a60 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2dafeb1b80 10 client.19779 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2dafeb1b80 10 client.19779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2dafeb1b80 20 client.19779 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562e38650830 0x562e38652c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2dafeb1b80 4 client.19779 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/463525073 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562e38382cf0 con 0x562e38650830 2024-03-18T20:48:04.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.906+0000 7f2da12b7700 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562e38650830 0x562e38652c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2da12b7700 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562e38650830 0x562e38652c20 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60c5011f4886db77 server_cookie=a860bec42d982a8f in_seq=0 out_seq=0 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2d857fa700 10 client.19779 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2d857fa700 1 -- 172.21.15.46:0/463525073 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562e38382cf0 con 0x562e38650830 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2d857fa700 10 client.19779 handle_command_reply: tid=0 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2dafeb1b80 1 client.19779 shutdown 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/463525073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 msgr2=0x562e385f09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 0x562e385f09e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2d90005a60 tx=0x7f2d90032770 comp rx=0 tx=0).stop 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/463525073 shutdown_connections 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562e38650830 0x562e38652c20 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e385e0a60 0x562e385f09e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562e385ec220 0x562e3864ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2dafeb1b80 1 --2- 172.21.15.46:0/463525073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562e385e1380 0x562e385f0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2d857fa700 0 client.19779 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/463525073 >> 172.21.15.46:0/463525073 conn(0x562e38557220 msgr2=0x562e385e9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/463525073 shutdown_connections 2024-03-18T20:48:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2dafeb1b80 1 -- 172.21.15.46:0/463525073 wait complete. 2024-03-18T20:48:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.907+0000 7f2dafeb1b80 20 client.19779 trim_cache size 0 max 16384 2024-03-18T20:48:04.916 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:48:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.908+0000 7f2da3d1c700 1 -- 172.21.15.46:0/3408805656 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2d880a1310 msgr2=0x7f2d880a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.908+0000 7f2da3d1c700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2d880a1310 0x7f2d880a37d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2d900104e0 tx=0x7f2d90007480 comp rx=0 tx=0).stop 2024-03-18T20:48:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.908+0000 7f2da3d1c700 1 -- 172.21.15.46:0/3408805656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d9c104f60 msgr2=0x7f2d9c10d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.908+0000 7f2da3d1c700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d9c104f60 0x7f2d9c10d960 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800e9c0 tx=0x7f2d9800ecd0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.908+0000 7f2da3d1c700 1 -- 172.21.15.46:0/3408805656 shutdown_connections 2024-03-18T20:48:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.908+0000 7f2da3d1c700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2d880a1310 0x7f2d880a37d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.908+0000 7f2da3d1c700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d9c10dea0 0x7f2d9c1a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.908+0000 7f2da3d1c700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d9c0ffe50 0x7f2d9c10d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.908+0000 7f2da3d1c700 1 --2- 172.21.15.46:0/3408805656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d9c104f60 0x7f2d9c10d960 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.908+0000 7f2da3d1c700 1 -- 172.21.15.46:0/3408805656 >> 172.21.15.46:0/3408805656 conn(0x7f2d9c076d40 msgr2=0x7f2d9c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.908+0000 7f2da3d1c700 1 -- 172.21.15.46:0/3408805656 shutdown_connections 2024-03-18T20:48:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:04.908+0000 7f2da3d1c700 1 -- 172.21.15.46:0/3408805656 wait complete. 2024-03-18T20:48:05.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 457580 3.75 MB/sec execute 540 sec latency 1161.429 ms 2024-03-18T20:48:05.925 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:48:05.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:48:05.925 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:48:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.085+0000 7ff621312700 1 -- 172.21.15.46:0/1975259351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61c068ff0 msgr2=0x7ff61c0693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.085+0000 7ff621312700 1 --2- 172.21.15.46:0/1975259351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61c068ff0 0x7ff61c0693d0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009990 tx=0x7ff60c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.085+0000 7ff621312700 1 -- 172.21.15.46:0/1975259351 shutdown_connections 2024-03-18T20:48:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.085+0000 7ff621312700 1 --2- 172.21.15.46:0/1975259351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61c068ff0 0x7ff61c0693d0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.085+0000 7ff621312700 1 --2- 172.21.15.46:0/1975259351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61c10d750 0x7ff61c10fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.085+0000 7ff621312700 1 --2- 172.21.15.46:0/1975259351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61c069910 0x7ff61c10d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.085+0000 7ff621312700 1 -- 172.21.15.46:0/1975259351 >> 172.21.15.46:0/1975259351 conn(0x7ff61c076d80 msgr2=0x7ff61c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:06.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.085+0000 7ff621312700 1 -- 172.21.15.46:0/1975259351 shutdown_connections 2024-03-18T20:48:06.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.085+0000 7ff621312700 1 -- 172.21.15.46:0/1975259351 wait complete. 2024-03-18T20:48:06.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.086+0000 7ff621312700 1 Processor -- start 2024-03-18T20:48:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.087+0000 7ff621312700 1 -- start start 2024-03-18T20:48:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.087+0000 7ff621312700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61c068ff0 0x7ff61c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.087+0000 7ff621312700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61c069910 0x7ff61c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.087+0000 7ff621312700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61c10d750 0x7ff61c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.087+0000 7ff621312700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c19e2c0 con 0x7ff61c069910 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.087+0000 7ff621312700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c19e430 con 0x7ff61c068ff0 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.087+0000 7ff621312700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c19e5f0 con 0x7ff61c10d750 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff61a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61c069910 0x7ff61c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff61affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61c068ff0 0x7ff61c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff61a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61c069910 0x7ff61c1998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40668/0 (socket says 172.21.15.46:40668) 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff61a7fc700 1 -- 172.21.15.46:0/2141100028 learned_addr learned my addr 172.21.15.46:0/2141100028 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff61b7fe700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61c10d750 0x7ff61c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff61affd700 1 -- 172.21.15.46:0/2141100028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61c10d750 msgr2=0x7ff61c19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff61affd700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61c10d750 0x7ff61c19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff61affd700 1 -- 172.21.15.46:0/2141100028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61c069910 msgr2=0x7ff61c1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff61affd700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61c069910 0x7ff61c1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff61affd700 1 -- 172.21.15.46:0/2141100028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c0095f0 con 0x7ff61c068ff0 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff61a7fc700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61c069910 0x7ff61c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff61affd700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61c068ff0 0x7ff61c1993a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0059d0 tx=0x7ff60c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff603fff700 1 -- 172.21.15.46:0/2141100028 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c01d070 con 0x7ff61c068ff0 2024-03-18T20:48:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff621312700 1 -- 172.21.15.46:0/2141100028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c19e870 con 0x7ff61c068ff0 2024-03-18T20:48:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff603fff700 1 -- 172.21.15.46:0/2141100028 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff60c022470 con 0x7ff61c068ff0 2024-03-18T20:48:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.088+0000 7ff603fff700 1 -- 172.21.15.46:0/2141100028 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c017650 con 0x7ff61c068ff0 2024-03-18T20:48:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.089+0000 7ff621312700 1 -- 172.21.15.46:0/2141100028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c1a6110 con 0x7ff61c068ff0 2024-03-18T20:48:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.091+0000 7ff603fff700 1 -- 172.21.15.46:0/2141100028 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff60c0177b0 con 0x7ff61c068ff0 2024-03-18T20:48:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.091+0000 7ff603fff700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff6040a1290 0x7ff6040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.092+0000 7ff603fff700 1 -- 172.21.15.46:0/2141100028 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7ff60c0cddd0 con 0x7ff61c068ff0 2024-03-18T20:48:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.092+0000 7ff61a7fc700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff6040a1290 0x7ff6040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.092+0000 7ff61a7fc700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff6040a1290 0x7ff6040a3750 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff61c19a8f0 tx=0x7ff610009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.092+0000 7ff62d4a7b80 1 Processor -- start 2024-03-18T20:48:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff62d4a7b80 1 -- start start 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff62d4a7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72cae0 0x55f08c72cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff62d4a7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72d400 0x55f08c737d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff62d4a7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c7382a0 0x55f08c73a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff62d4a7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08c686a10 con 0x55f08c7382a0 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff62d4a7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08c6843d0 con 0x55f08c72d400 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff62d4a7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08c67fcc0 con 0x55f08c72cae0 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff61affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72cae0 0x55f08c72cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff61affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72cae0 0x55f08c72cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59208/0 (socket says 172.21.15.46:59208) 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff61affd700 1 -- 172.21.15.46:0/926475466 learned_addr learned my addr 172.21.15.46:0/926475466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff61b7fe700 1 --2- 172.21.15.46:0/926475466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c7382a0 0x55f08c73a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff61a7fc700 1 --2- 172.21.15.46:0/926475466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72d400 0x55f08c737d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff61affd700 1 -- 172.21.15.46:0/926475466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72d400 msgr2=0x55f08c737d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff61affd700 1 --2- 172.21.15.46:0/926475466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72d400 0x55f08c737d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff61affd700 1 -- 172.21.15.46:0/926475466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c7382a0 msgr2=0x55f08c73a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff61affd700 1 --2- 172.21.15.46:0/926475466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c7382a0 0x55f08c73a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff61affd700 1 -- 172.21.15.46:0/926475466 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f08c501430 con 0x55f08c72cae0 2024-03-18T20:48:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.093+0000 7ff61b7fe700 1 --2- 172.21.15.46:0/926475466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c7382a0 0x55f08c73a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.094+0000 7ff61affd700 1 --2- 172.21.15.46:0/926475466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72cae0 0x55f08c72cec0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff60c09b150 tx=0x7ff60c00b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f574e67bc609ff server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.094+0000 7ff6017fa700 1 -- 172.21.15.46:0/926475466 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0a1120 con 0x55f08c72cae0 2024-03-18T20:48:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.094+0000 7ff6017fa700 1 -- 172.21.15.46:0/926475466 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff60c09e070 con 0x55f08c72cae0 2024-03-18T20:48:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.094+0000 7ff6017fa700 1 -- 172.21.15.46:0/926475466 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c09a6b0 con 0x55f08c72cae0 2024-03-18T20:48:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.095+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/926475466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72cae0 msgr2=0x55f08c72cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.095+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/926475466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72cae0 0x55f08c72cec0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff60c09b150 tx=0x7ff60c00b6b0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.095+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/926475466 shutdown_connections 2024-03-18T20:48:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.095+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/926475466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c7382a0 0x55f08c73a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.095+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/926475466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72cae0 0x55f08c72cec0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.095+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/926475466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72d400 0x55f08c737d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.095+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/926475466 >> 172.21.15.46:0/926475466 conn(0x55f08c6a32a0 msgr2=0x55f08c7290a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.095+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/926475466 shutdown_connections 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.095+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/926475466 wait complete. 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.095+0000 7ff62d4a7b80 1 -- start start 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.095+0000 7ff62d4a7b80 1 Processor -- start 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff62d4a7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72cae0 0x55f08c738680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff62d4a7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 0x55f08c738bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff62d4a7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c7382a0 0x55f08c743700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff62d4a7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08c61be00 con 0x55f08c7382a0 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff62d4a7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08c61baa0 con 0x55f08c72cae0 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff62d4a7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08c503720 con 0x55f08c72d400 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff62d4a7b80 10 client.0 _ll_register_callbacks cb 0x55f08c487420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff61a7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 0x55f08c738bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff61b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c7382a0 0x55f08c743700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff61a7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 0x55f08c738bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59216/0 (socket says 172.21.15.46:59216) 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff61a7fc700 1 -- 172.21.15.46:0/2560392476 learned_addr learned my addr 172.21.15.46:0/2560392476 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff61affd700 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72cae0 0x55f08c738680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff61a7fc700 1 -- 172.21.15.46:0/2560392476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72cae0 msgr2=0x55f08c738680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff61a7fc700 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72cae0 0x55f08c738680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff61a7fc700 1 -- 172.21.15.46:0/2560392476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c7382a0 msgr2=0x55f08c743700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff61a7fc700 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c7382a0 0x55f08c743700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff61a7fc700 1 -- 172.21.15.46:0/2560392476 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff614011230 con 0x55f08c72d400 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff61b7fe700 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c7382a0 0x55f08c743700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff61a7fc700 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 0x55f08c738bc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff61000ed60 tx=0x7ff6100047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=652a7aac7b0f6b61 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.096+0000 7ff5feff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff5feff5700 1 -- 172.21.15.46:0/2560392476 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610022070 con 0x55f08c72d400 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff5feff5700 1 -- 172.21.15.46:0/2560392476 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7ff61001db00 con 0x55f08c72d400 2024-03-18T20:48:06.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/2560392476 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f08c720b50 con 0x55f08c72d400 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff5feff5700 1 -- 172.21.15.46:0/2560392476 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff61001dcc0 con 0x55f08c72d400 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff62d4a7b80 10 client.29434 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/2560392476 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7ff61c1a6110 con 0x55f08c72d400 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff5feff5700 1 -- 172.21.15.46:0/2560392476 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f08c720b50 con 0x55f08c72d400 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff5feff5700 1 -- 172.21.15.46:0/2560392476 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff61002e400 con 0x55f08c72d400 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff62d4a7b80 10 client.29434 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff62d4a7b80 10 client.29434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff62d4a7b80 20 client.29434 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f08c747b20 0x55f08c749f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff62d4a7b80 4 client.29434 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.097+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/2560392476 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f08c678780 con 0x55f08c747b20 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.098+0000 7ff61affd700 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f08c747b20 0x55f08c749f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.098+0000 7ff5feff5700 10 client.29434 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.098+0000 7ff61affd700 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f08c747b20 0x55f08c749f40 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e84490be3a57e9 server_cookie=e77ea55239387f4b in_seq=0 out_seq=0 2024-03-18T20:48:06.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.098+0000 7ff5feff5700 1 -- 172.21.15.46:0/2560392476 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f08c678780 con 0x55f08c747b20 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.098+0000 7ff5feff5700 10 client.29434 handle_command_reply: tid=0 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff62d4a7b80 1 client.29434 shutdown 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/2560392476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 msgr2=0x55f08c738bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 0x55f08c738bc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff61000ed60 tx=0x7ff6100047c0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/2560392476 shutdown_connections 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f08c747b20 0x55f08c749f40 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c7382a0 0x55f08c743700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72cae0 0x55f08c738680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/2560392476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 0x55f08c738bc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff5feff5700 0 client.29434 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/2560392476 >> 172.21.15.46:0/2560392476 conn(0x55f08c6a32a0 msgr2=0x55f08c728c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/2560392476 shutdown_connections 2024-03-18T20:48:06.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/2560392476 wait complete. 2024-03-18T20:48:06.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.099+0000 7ff62d4a7b80 20 client.29434 trim_cache size 0 max 16384 2024-03-18T20:48:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff62d4a7b80 1 Processor -- start 2024-03-18T20:48:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff62d4a7b80 1 -- start start 2024-03-18T20:48:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff62d4a7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 0x55f08c73d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff62d4a7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 0x55f08c735e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff62d4a7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c7382a0 0x55f08c736380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff62d4a7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08c72a500 con 0x55f08c72cae0 2024-03-18T20:48:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff62d4a7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08c744620 con 0x55f08c7382a0 2024-03-18T20:48:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff62d4a7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08c7368c0 con 0x55f08c72d400 2024-03-18T20:48:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff61affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 0x55f08c73d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff61affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 0x55f08c73d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40692/0 (socket says 172.21.15.46:40692) 2024-03-18T20:48:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff61a7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 0x55f08c735e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff61affd700 1 -- 172.21.15.46:0/3869231698 learned_addr learned my addr 172.21.15.46:0/3869231698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.118+0000 7ff61b7fe700 1 --2- 172.21.15.46:0/3869231698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c7382a0 0x55f08c736380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.119+0000 7ff61affd700 1 -- 172.21.15.46:0/3869231698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 msgr2=0x55f08c735e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.119+0000 7ff61affd700 1 --2- 172.21.15.46:0/3869231698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 0x55f08c735e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.119+0000 7ff61affd700 1 -- 172.21.15.46:0/3869231698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c7382a0 msgr2=0x55f08c736380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.119+0000 7ff61affd700 1 --2- 172.21.15.46:0/3869231698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c7382a0 0x55f08c736380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.119+0000 7ff61affd700 1 -- 172.21.15.46:0/3869231698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f08c4adbe0 con 0x55f08c72cae0 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.119+0000 7ff61a7fc700 1 --2- 172.21.15.46:0/3869231698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 0x55f08c735e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.119+0000 7ff61b7fe700 1 --2- 172.21.15.46:0/3869231698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c7382a0 0x55f08c736380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.119+0000 7ff61affd700 1 --2- 172.21.15.46:0/3869231698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 0x55f08c73d5a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0cbad0 tx=0x7ff60c022850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37ef4c5d9f6d88b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.119+0000 7ff601ffb700 1 -- 172.21.15.46:0/3869231698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0a3310 con 0x55f08c72cae0 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.119+0000 7ff601ffb700 1 -- 172.21.15.46:0/3869231698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff60c0228e0 con 0x55f08c72cae0 2024-03-18T20:48:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.119+0000 7ff601ffb700 1 -- 172.21.15.46:0/3869231698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c09a6b0 con 0x55f08c72cae0 2024-03-18T20:48:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.120+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/3869231698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 msgr2=0x55f08c73d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.120+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/3869231698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 0x55f08c73d5a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0cbad0 tx=0x7ff60c022850 comp rx=0 tx=0).stop 2024-03-18T20:48:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.120+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/3869231698 shutdown_connections 2024-03-18T20:48:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.120+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/3869231698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 0x55f08c73d5a0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.120+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/3869231698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c7382a0 0x55f08c736380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.120+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/3869231698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c72d400 0x55f08c735e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.120+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/3869231698 >> 172.21.15.46:0/3869231698 conn(0x55f08c6a32a0 msgr2=0x55f08c738680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.120+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/3869231698 shutdown_connections 2024-03-18T20:48:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.120+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/3869231698 wait complete. 2024-03-18T20:48:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.120+0000 7ff62d4a7b80 1 -- start start 2024-03-18T20:48:06.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.120+0000 7ff62d4a7b80 1 Processor -- start 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff62d4a7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 0x55f08c794ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff62d4a7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72d400 0x55f08c795410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff62d4a7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c7382a0 0x55f08c7903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff62d4a7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08c7909b0 con 0x55f08c72cae0 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff62d4a7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08c790af0 con 0x55f08c72d400 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff62d4a7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08c790cb0 con 0x55f08c7382a0 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff62d4a7b80 10 client.0 _ll_register_callbacks cb 0x55f08c487420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff61affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 0x55f08c794ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff61affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 0x55f08c794ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40694/0 (socket says 172.21.15.46:40694) 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff61affd700 1 -- 172.21.15.46:0/1079898577 learned_addr learned my addr 172.21.15.46:0/1079898577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff61b7fe700 1 --2- 172.21.15.46:0/1079898577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c7382a0 0x55f08c7903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff61affd700 1 -- 172.21.15.46:0/1079898577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c7382a0 msgr2=0x55f08c7903e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff61affd700 1 --2- 172.21.15.46:0/1079898577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c7382a0 0x55f08c7903e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff61affd700 1 -- 172.21.15.46:0/1079898577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72d400 msgr2=0x55f08c795410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff61affd700 1 --2- 172.21.15.46:0/1079898577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72d400 0x55f08c795410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff61affd700 1 -- 172.21.15.46:0/1079898577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c1a6110 con 0x55f08c72cae0 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.121+0000 7ff61affd700 1 --2- 172.21.15.46:0/1079898577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 0x55f08c794ed0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ff60c006010 tx=0x7ff60c0975b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13aca41c07e219ed server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff5feff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff5feff5700 1 -- 172.21.15.46:0/1079898577 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c09af90 con 0x55f08c72cae0 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff5feff5700 1 -- 172.21.15.46:0/1079898577 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7ff60c0a3310 con 0x55f08c72cae0 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/1079898577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff60c09e070 con 0x55f08c72cae0 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff5feff5700 1 -- 172.21.15.46:0/1079898577 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff60c0970d0 con 0x55f08c72cae0 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff62d4a7b80 10 client.19797 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/1079898577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7ff614007690 con 0x55f08c72cae0 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff5feff5700 1 -- 172.21.15.46:0/1079898577 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff60c09e070 con 0x55f08c72cae0 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff5feff5700 1 -- 172.21.15.46:0/1079898577 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff60c085190 con 0x55f08c72cae0 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff62d4a7b80 10 client.19797 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:06.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff62d4a7b80 10 client.19797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff62d4a7b80 20 client.19797 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/1079898577 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f08c79da10 0x55f08c79fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff62d4a7b80 4 client.19797 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.122+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/1079898577 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f08c4ced70 con 0x55f08c79da10 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.123+0000 7ff61a7fc700 1 --2- 172.21.15.46:0/1079898577 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f08c79da10 0x55f08c79fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.123+0000 7ff5feff5700 10 client.19797 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.123+0000 7ff61a7fc700 1 --2- 172.21.15.46:0/1079898577 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f08c79da10 0x55f08c79fe00 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe99470207d9e21 server_cookie=c1e89cd31580a562 in_seq=0 out_seq=0 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.124+0000 7ff5feff5700 1 -- 172.21.15.46:0/1079898577 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f08c4ced70 con 0x55f08c79da10 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.124+0000 7ff5feff5700 10 client.19797 handle_command_reply: tid=0 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.124+0000 7ff62d4a7b80 1 client.19797 shutdown 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.125+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/1079898577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 msgr2=0x55f08c794ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.125+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/1079898577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 0x55f08c794ed0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ff60c006010 tx=0x7ff60c0975b0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.125+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/1079898577 shutdown_connections 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.125+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/1079898577 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f08c79da10 0x55f08c79fe00 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.125+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/1079898577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f08c72cae0 0x55f08c794ed0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.125+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/1079898577 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f08c7382a0 0x55f08c7903e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.125+0000 7ff5feff5700 0 client.19797 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.125+0000 7ff62d4a7b80 1 --2- 172.21.15.46:0/1079898577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f08c72d400 0x55f08c795410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.125+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/1079898577 >> 172.21.15.46:0/1079898577 conn(0x55f08c6a32a0 msgr2=0x55f08c738680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.125+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/1079898577 shutdown_connections 2024-03-18T20:48:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.125+0000 7ff62d4a7b80 1 -- 172.21.15.46:0/1079898577 wait complete. 2024-03-18T20:48:06.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.125+0000 7ff62d4a7b80 20 client.19797 trim_cache size 0 max 16384 2024-03-18T20:48:06.132 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:48:06.132 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:48:06.132 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "9af2f6da-514d-4f95-b58b-88b7a569c89b", 2024-03-18T20:48:06.133 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:48:06.133 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:48:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.126+0000 7ff621312700 1 -- 172.21.15.46:0/2141100028 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff6040a1290 msgr2=0x7ff6040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.126+0000 7ff621312700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff6040a1290 0x7ff6040a3750 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff61c19a8f0 tx=0x7ff610009260 comp rx=0 tx=0).stop 2024-03-18T20:48:06.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.126+0000 7ff621312700 1 -- 172.21.15.46:0/2141100028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61c068ff0 msgr2=0x7ff61c1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.126+0000 7ff621312700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61c068ff0 0x7ff61c1993a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0059d0 tx=0x7ff60c004730 comp rx=0 tx=0).stop 2024-03-18T20:48:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.126+0000 7ff621312700 1 -- 172.21.15.46:0/2141100028 shutdown_connections 2024-03-18T20:48:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.126+0000 7ff621312700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff6040a1290 0x7ff6040a3750 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.126+0000 7ff621312700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61c10d750 0x7ff61c19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.126+0000 7ff621312700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff61c068ff0 0x7ff61c1993a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.126+0000 7ff621312700 1 --2- 172.21.15.46:0/2141100028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff61c069910 0x7ff61c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.126+0000 7ff621312700 1 -- 172.21.15.46:0/2141100028 >> 172.21.15.46:0/2141100028 conn(0x7ff61c076d80 msgr2=0x7ff61c10ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.126+0000 7ff621312700 1 -- 172.21.15.46:0/2141100028 shutdown_connections 2024-03-18T20:48:06.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.126+0000 7ff621312700 1 -- 172.21.15.46:0/2141100028 wait complete. 2024-03-18T20:48:06.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:48:06.146 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:48:06.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:06 smithi155 ceph-mon[48344]: pgmap v374: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-03-18T20:48:06.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:06 smithi046 ceph-mon[43495]: pgmap v374: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-03-18T20:48:06.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.297+0000 7f3cf8d15700 1 -- 172.21.15.46:0/2105616811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf4104f60 msgr2=0x7f3cf410cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.297+0000 7f3cf8d15700 1 --2- 172.21.15.46:0/2105616811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf4104f60 0x7f3cf410cae0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00b0a0 tx=0x7f3cec00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.297+0000 7f3cf8d15700 1 -- 172.21.15.46:0/2105616811 shutdown_connections 2024-03-18T20:48:06.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.297+0000 7f3cf8d15700 1 --2- 172.21.15.46:0/2105616811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf4104f60 0x7f3cf410cae0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.297+0000 7f3cf8d15700 1 --2- 172.21.15.46:0/2105616811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf40ffe50 0x7f3cf4100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.297+0000 7f3cf8d15700 1 --2- 172.21.15.46:0/2105616811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf4100800 0x7f3cf4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.297+0000 7f3cf8d15700 1 -- 172.21.15.46:0/2105616811 >> 172.21.15.46:0/2105616811 conn(0x7f3cf4076d40 msgr2=0x7f3cf4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:06.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.297+0000 7f3cf8d15700 1 -- 172.21.15.46:0/2105616811 shutdown_connections 2024-03-18T20:48:06.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.297+0000 7f3cf8d15700 1 -- 172.21.15.46:0/2105616811 wait complete. 2024-03-18T20:48:06.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.298+0000 7f3cf8d15700 1 Processor -- start 2024-03-18T20:48:06.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.298+0000 7f3cf8d15700 1 -- start start 2024-03-18T20:48:06.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.298+0000 7f3cf8d15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ffe50 0x7f3cf41972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.298+0000 7f3cf8d15700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf4104f60 0x7f3cf41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.298+0000 7f3cf8d15700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf419bb90 0x7f3cf419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.298+0000 7f3cf8d15700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf419c610 con 0x7f3cf40ffe50 2024-03-18T20:48:06.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.298+0000 7f3cf8d15700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf419c780 con 0x7f3cf4104f60 2024-03-18T20:48:06.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.298+0000 7f3cf8d15700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf419c940 con 0x7f3cf419bb90 2024-03-18T20:48:06.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.298+0000 7f3cf259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ffe50 0x7f3cf41972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3cf259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ffe50 0x7f3cf41972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40716/0 (socket says 172.21.15.46:40716) 2024-03-18T20:48:06.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3cf2d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf419bb90 0x7f3cf419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3cf259c700 1 -- 172.21.15.46:0/3351907491 learned_addr learned my addr 172.21.15.46:0/3351907491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:06.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3cf1d9b700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf4104f60 0x7f3cf41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3cf2d9d700 1 -- 172.21.15.46:0/3351907491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf4104f60 msgr2=0x7f3cf41977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3cf2d9d700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf4104f60 0x7f3cf41977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3cf2d9d700 1 -- 172.21.15.46:0/3351907491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ffe50 msgr2=0x7f3cf41972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3cf2d9d700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ffe50 0x7f3cf41972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3cf2d9d700 1 -- 172.21.15.46:0/3351907491 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec009d00 con 0x7f3cf419bb90 2024-03-18T20:48:06.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3cf259c700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ffe50 0x7f3cf41972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:06.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3cf2d9d700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf419bb90 0x7f3cf419c010 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0062a0 tx=0x7f3cec00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:06.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3ce37fe700 1 -- 172.21.15.46:0/3351907491 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec00e040 con 0x7f3cf419bb90 2024-03-18T20:48:06.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3cf8d15700 1 -- 172.21.15.46:0/3351907491 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf41a3e40 con 0x7f3cf419bb90 2024-03-18T20:48:06.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3ce37fe700 1 -- 172.21.15.46:0/3351907491 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cec01d480 con 0x7f3cf419bb90 2024-03-18T20:48:06.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.299+0000 7f3ce37fe700 1 -- 172.21.15.46:0/3351907491 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec01b6d0 con 0x7f3cf419bb90 2024-03-18T20:48:06.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.300+0000 7f3cf8d15700 1 -- 172.21.15.46:0/3351907491 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf41a4310 con 0x7f3cf419bb90 2024-03-18T20:48:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.302+0000 7f3ce37fe700 1 -- 172.21.15.46:0/3351907491 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3cec01d5f0 con 0x7f3cf419bb90 2024-03-18T20:48:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.303+0000 7f3ce37fe700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3cdc0a1290 0x7f3cdc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.303+0000 7f3ce37fe700 1 -- 172.21.15.46:0/3351907491 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f3cec0ce110 con 0x7f3cf419bb90 2024-03-18T20:48:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.303+0000 7f3cf259c700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3cdc0a1290 0x7f3cdc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.304+0000 7f3cf259c700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3cdc0a1290 0x7f3cdc0a3750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4011550 tx=0x7f3ce4009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.304+0000 7f3d04eaab80 1 Processor -- start 2024-03-18T20:48:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.304+0000 7f3d04eaab80 1 -- start start 2024-03-18T20:48:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.304+0000 7f3d04eaab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 0x557e28149e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.304+0000 7f3d04eaab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e2814a380 0x557e281590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.304+0000 7f3d04eaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e28159630 0x557e2815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.304+0000 7f3d04eaab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e280a3990 con 0x557e2814a380 2024-03-18T20:48:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.304+0000 7f3d04eaab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e280a1350 con 0x557e28149a60 2024-03-18T20:48:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.304+0000 7f3d04eaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2809cc40 con 0x557e28159630 2024-03-18T20:48:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3cf259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 0x557e28149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3cf1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e2814a380 0x557e281590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3cf259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 0x557e28149e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52722/0 (socket says 172.21.15.46:52722) 2024-03-18T20:48:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3cf259c700 1 -- 172.21.15.46:0/3786893294 learned_addr learned my addr 172.21.15.46:0/3786893294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:06.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3cf2d9d700 1 --2- 172.21.15.46:0/3786893294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e28159630 0x557e2815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3cf259c700 1 -- 172.21.15.46:0/3786893294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e28159630 msgr2=0x557e2815ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3cf259c700 1 --2- 172.21.15.46:0/3786893294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e28159630 0x557e2815ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3cf259c700 1 -- 172.21.15.46:0/3786893294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e2814a380 msgr2=0x557e281590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3cf259c700 1 --2- 172.21.15.46:0/3786893294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e2814a380 0x557e281590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3cf259c700 1 -- 172.21.15.46:0/3786893294 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e27f1e3b0 con 0x557e28149a60 2024-03-18T20:48:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3cf259c700 1 --2- 172.21.15.46:0/3786893294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 0x557e28149e40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009880 tx=0x7f3ce4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f25b6012d0b7daa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3ce0ff9700 1 -- 172.21.15.46:0/3786893294 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4026630 con 0x557e28149a60 2024-03-18T20:48:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3ce0ff9700 1 -- 172.21.15.46:0/3786893294 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ce4021070 con 0x557e28149a60 2024-03-18T20:48:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.305+0000 7f3ce0ff9700 1 -- 172.21.15.46:0/3786893294 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce401d650 con 0x557e28149a60 2024-03-18T20:48:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.306+0000 7f3d04eaab80 1 -- 172.21.15.46:0/3786893294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 msgr2=0x557e28149e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.306+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/3786893294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 0x557e28149e40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009880 tx=0x7f3ce4014000 comp rx=0 tx=0).stop 2024-03-18T20:48:06.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.306+0000 7f3d04eaab80 1 -- 172.21.15.46:0/3786893294 shutdown_connections 2024-03-18T20:48:06.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.306+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/3786893294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e28159630 0x557e2815ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.306+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/3786893294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 0x557e28149e40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.306+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/3786893294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e2814a380 0x557e281590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.306+0000 7f3d04eaab80 1 -- 172.21.15.46:0/3786893294 >> 172.21.15.46:0/3786893294 conn(0x557e280c0220 msgr2=0x557e28146020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:06.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.306+0000 7f3d04eaab80 1 -- 172.21.15.46:0/3786893294 shutdown_connections 2024-03-18T20:48:06.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.306+0000 7f3d04eaab80 1 -- 172.21.15.46:0/3786893294 wait complete. 2024-03-18T20:48:06.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3d04eaab80 1 -- start start 2024-03-18T20:48:06.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3d04eaab80 1 Processor -- start 2024-03-18T20:48:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3d04eaab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28149a60 0x557e281590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3d04eaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 0x557e281563c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3d04eaab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28159630 0x557e28156b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3d04eaab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e28038d80 con 0x557e28149a60 2024-03-18T20:48:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3d04eaab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e28038a20 con 0x557e28159630 2024-03-18T20:48:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3d04eaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e27f206a0 con 0x557e2814a380 2024-03-18T20:48:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3d04eaab80 10 client.0 _ll_register_callbacks cb 0x557e27ea43a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3cf259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28149a60 0x557e281590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3cf1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 0x557e281563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3cf259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28149a60 0x557e281590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40732/0 (socket says 172.21.15.46:40732) 2024-03-18T20:48:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3cf259c700 1 -- 172.21.15.46:0/2714987826 learned_addr learned my addr 172.21.15.46:0/2714987826 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.307+0000 7f3cf2d9d700 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28159630 0x557e28156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cf259c700 1 -- 172.21.15.46:0/2714987826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 msgr2=0x557e281563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cf259c700 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 0x557e281563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cf259c700 1 -- 172.21.15.46:0/2714987826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28159630 msgr2=0x557e28156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cf259c700 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28159630 0x557e28156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cf259c700 1 -- 172.21.15.46:0/2714987826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce4017070 con 0x557e28149a60 2024-03-18T20:48:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cf2d9d700 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28159630 0x557e28156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cf1d9b700 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 0x557e281563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cf259c700 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28149a60 0x557e281590d0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4011ca0 tx=0x7f3ce401cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9104bbe257859745 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cd9ffb700 1 -- 172.21.15.46:0/2714987826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce401c620 con 0x557e28149a60 2024-03-18T20:48:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3d04eaab80 1 -- 172.21.15.46:0/2714987826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e2813dad0 con 0x557e28149a60 2024-03-18T20:48:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cd9ffb700 1 -- 172.21.15.46:0/2714987826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f3ce401c780 con 0x557e28149a60 2024-03-18T20:48:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cd9ffb700 1 -- 172.21.15.46:0/2714987826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ce4005040 con 0x557e28149a60 2024-03-18T20:48:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3d04eaab80 10 client.19809 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3d04eaab80 1 -- 172.21.15.46:0/2714987826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x557e27ecab60 con 0x557e28149a60 2024-03-18T20:48:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.308+0000 7f3cd9ffb700 1 -- 172.21.15.46:0/2714987826 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e2813dad0 con 0x557e28149a60 2024-03-18T20:48:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.309+0000 7f3cd9ffb700 1 -- 172.21.15.46:0/2714987826 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3ce4004c10 con 0x557e28149a60 2024-03-18T20:48:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.309+0000 7f3d04eaab80 10 client.19809 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.309+0000 7f3d04eaab80 10 client.19809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.309+0000 7f3d04eaab80 20 client.19809 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.309+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e28169080 0x557e2816b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.309+0000 7f3d04eaab80 4 client.19809 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.309+0000 7f3d04eaab80 1 -- 172.21.15.46:0/2714987826 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e28095700 con 0x557e28169080 2024-03-18T20:48:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.309+0000 7f3cf1d9b700 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e28169080 0x557e2816b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.310+0000 7f3cd9ffb700 10 client.19809 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.310+0000 7f3cf1d9b700 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e28169080 0x557e2816b4a0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ded74f949802f08 server_cookie=8ce4f20a620bb29a in_seq=0 out_seq=0 2024-03-18T20:48:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3cd9ffb700 1 -- 172.21.15.46:0/2714987826 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557e28095700 con 0x557e28169080 2024-03-18T20:48:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3cd9ffb700 10 client.19809 handle_command_reply: tid=0 2024-03-18T20:48:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3d04eaab80 1 client.19809 shutdown 2024-03-18T20:48:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3d04eaab80 1 -- 172.21.15.46:0/2714987826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28149a60 msgr2=0x557e281590d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28149a60 0x557e281590d0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4011ca0 tx=0x7f3ce401cc50 comp rx=0 tx=0).stop 2024-03-18T20:48:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3d04eaab80 1 -- 172.21.15.46:0/2714987826 shutdown_connections 2024-03-18T20:48:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e28169080 0x557e2816b4a0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28149a60 0x557e281590d0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28159630 0x557e28156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/2714987826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 0x557e281563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3cd9ffb700 0 client.19809 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3d04eaab80 1 -- 172.21.15.46:0/2714987826 >> 172.21.15.46:0/2714987826 conn(0x557e280c0220 msgr2=0x557e28145c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3d04eaab80 1 -- 172.21.15.46:0/2714987826 shutdown_connections 2024-03-18T20:48:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3d04eaab80 1 -- 172.21.15.46:0/2714987826 wait complete. 2024-03-18T20:48:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.311+0000 7f3d04eaab80 20 client.19809 trim_cache size 0 max 16384 2024-03-18T20:48:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.329+0000 7f3d04eaab80 1 Processor -- start 2024-03-18T20:48:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.329+0000 7f3d04eaab80 1 -- start start 2024-03-18T20:48:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3d04eaab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 0x557e28160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3d04eaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 0x557e281649b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3d04eaab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 0x557e281516d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3d04eaab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2814a090 con 0x557e28159630 2024-03-18T20:48:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3d04eaab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e28110940 con 0x557e28149a60 2024-03-18T20:48:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3d04eaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2814a9d0 con 0x557e2814a380 2024-03-18T20:48:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 0x557e281649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 0x557e28160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 0x557e281516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf1d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 0x557e281649b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59294/0 (socket says 172.21.15.46:59294) 2024-03-18T20:48:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 0x557e28160260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52730/0 (socket says 172.21.15.46:52730) 2024-03-18T20:48:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf1d9b700 1 -- 172.21.15.46:0/1521844935 learned_addr learned my addr 172.21.15.46:0/1521844935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf2d9d700 1 -- 172.21.15.46:0/1521844935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 msgr2=0x557e281649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf2d9d700 1 --2- 172.21.15.46:0/1521844935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 0x557e281649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf2d9d700 1 -- 172.21.15.46:0/1521844935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 msgr2=0x557e28160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf2d9d700 1 --2- 172.21.15.46:0/1521844935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 0x557e28160260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf2d9d700 1 -- 172.21.15.46:0/1521844935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e28062ea0 con 0x557e28159630 2024-03-18T20:48:06.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf1d9b700 1 --2- 172.21.15.46:0/1521844935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 0x557e281649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:06.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.330+0000 7f3cf259c700 1 --2- 172.21.15.46:0/1521844935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 0x557e28160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:06.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.331+0000 7f3cf2d9d700 1 --2- 172.21.15.46:0/1521844935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 0x557e281516d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0cb7b0 tx=0x7f3cec009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2184bbc89efe0a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:06.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.331+0000 7f3ce17fa700 1 -- 172.21.15.46:0/1521844935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec09b900 con 0x557e28159630 2024-03-18T20:48:06.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.331+0000 7f3ce17fa700 1 -- 172.21.15.46:0/1521844935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cec09b200 con 0x557e28159630 2024-03-18T20:48:06.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.331+0000 7f3ce17fa700 1 -- 172.21.15.46:0/1521844935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec0a10b0 con 0x557e28159630 2024-03-18T20:48:06.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.332+0000 7f3d04eaab80 1 -- 172.21.15.46:0/1521844935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 msgr2=0x557e281516d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.332+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/1521844935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 0x557e281516d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0cb7b0 tx=0x7f3cec009040 comp rx=0 tx=0).stop 2024-03-18T20:48:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.332+0000 7f3d04eaab80 1 -- 172.21.15.46:0/1521844935 shutdown_connections 2024-03-18T20:48:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.332+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/1521844935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 0x557e281516d0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.332+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/1521844935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e28149a60 0x557e28160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.332+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/1521844935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e2814a380 0x557e281649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.332+0000 7f3d04eaab80 1 -- 172.21.15.46:0/1521844935 >> 172.21.15.46:0/1521844935 conn(0x557e280c0220 msgr2=0x557e28144750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.332+0000 7f3d04eaab80 1 -- 172.21.15.46:0/1521844935 shutdown_connections 2024-03-18T20:48:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.332+0000 7f3d04eaab80 1 -- 172.21.15.46:0/1521844935 wait complete. 2024-03-18T20:48:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.332+0000 7f3d04eaab80 1 -- start start 2024-03-18T20:48:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.332+0000 7f3d04eaab80 1 Processor -- start 2024-03-18T20:48:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3d04eaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e28149a60 0x557e2815fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3d04eaab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e2814a380 0x557e28160180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3d04eaab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 0x557e2815e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3d04eaab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2815e6e0 con 0x557e28159630 2024-03-18T20:48:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3d04eaab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2815e850 con 0x557e2814a380 2024-03-18T20:48:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3d04eaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e2815ea10 con 0x557e28149a60 2024-03-18T20:48:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3d04eaab80 10 client.0 _ll_register_callbacks cb 0x7f3ce8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cf2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 0x557e2815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cf259c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e28149a60 0x557e2815fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cf2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 0x557e2815e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40748/0 (socket says 172.21.15.46:40748) 2024-03-18T20:48:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cf2d9d700 1 -- 172.21.15.46:0/4000933608 learned_addr learned my addr 172.21.15.46:0/4000933608 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cf1d9b700 1 --2- 172.21.15.46:0/4000933608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e2814a380 0x557e28160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cf2d9d700 1 -- 172.21.15.46:0/4000933608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e28149a60 msgr2=0x557e2815fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cf2d9d700 1 --2- 172.21.15.46:0/4000933608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e28149a60 0x557e2815fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cf2d9d700 1 -- 172.21.15.46:0/4000933608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e2814a380 msgr2=0x557e28160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cf2d9d700 1 --2- 172.21.15.46:0/4000933608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e2814a380 0x557e28160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cf2d9d700 1 -- 172.21.15.46:0/4000933608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e27ecab60 con 0x557e28159630 2024-03-18T20:48:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cf2d9d700 1 --2- 172.21.15.46:0/4000933608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 0x557e2815e110 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00b780 tx=0x7f3cec09ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e77a1208624063 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cd9ffb700 1 -- 172.21.15.46:0/4000933608 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec0a12b0 con 0x557e28159630 2024-03-18T20:48:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3cd9ffb700 1 -- 172.21.15.46:0/4000933608 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f3cec0a35f0 con 0x557e28159630 2024-03-18T20:48:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.333+0000 7f3d04eaab80 1 -- 172.21.15.46:0/4000933608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ce4021070 con 0x557e28159630 2024-03-18T20:48:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.334+0000 7f3cd9ffb700 1 -- 172.21.15.46:0/4000933608 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cec01dae0 con 0x557e28159630 2024-03-18T20:48:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.334+0000 7f3d04eaab80 10 client.19815 fetch_fsmap learned FSMap version 51 2024-03-18T20:48:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.334+0000 7f3d04eaab80 1 -- 172.21.15.46:0/4000933608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f3ce8004040 con 0x557e28159630 2024-03-18T20:48:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.334+0000 7f3cd9ffb700 1 -- 172.21.15.46:0/4000933608 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ce4021070 con 0x557e28159630 2024-03-18T20:48:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.334+0000 7f3cd9ffb700 1 -- 172.21.15.46:0/4000933608 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3cec0b3d20 con 0x557e28159630 2024-03-18T20:48:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.334+0000 7f3d04eaab80 10 client.19815 fetch_fsmap finished waiting for FSMap version 51 2024-03-18T20:48:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.334+0000 7f3d04eaab80 10 client.19815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.334+0000 7f3d04eaab80 20 client.19815 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.334+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/4000933608 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e281bdce0 0x557e281c0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.334+0000 7f3d04eaab80 4 client.19815 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:48:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.334+0000 7f3d04eaab80 1 -- 172.21.15.46:0/4000933608 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557e27eebcf0 con 0x557e281bdce0 2024-03-18T20:48:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.334+0000 7f3cf259c700 1 --2- 172.21.15.46:0/4000933608 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e281bdce0 0x557e281c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3cf259c700 1 --2- 172.21.15.46:0/4000933608 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e281bdce0 0x557e281c0100 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=149119f8b23c43ba server_cookie=332a78fc8630e645 in_seq=0 out_seq=0 2024-03-18T20:48:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3cd9ffb700 10 client.19815 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3cd9ffb700 1 -- 172.21.15.46:0/4000933608 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557e27eebcf0 con 0x557e281bdce0 2024-03-18T20:48:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3cd9ffb700 10 client.19815 handle_command_reply: tid=0 2024-03-18T20:48:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3d04eaab80 1 client.19815 shutdown 2024-03-18T20:48:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3d04eaab80 1 -- 172.21.15.46:0/4000933608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 msgr2=0x557e2815e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/4000933608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 0x557e2815e110 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00b780 tx=0x7f3cec09ac20 comp rx=0 tx=0).stop 2024-03-18T20:48:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3d04eaab80 1 -- 172.21.15.46:0/4000933608 shutdown_connections 2024-03-18T20:48:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/4000933608 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557e281bdce0 0x557e281c0100 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/4000933608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e28159630 0x557e2815e110 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/4000933608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557e28149a60 0x557e2815fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3d04eaab80 1 --2- 172.21.15.46:0/4000933608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e2814a380 0x557e28160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3d04eaab80 1 -- 172.21.15.46:0/4000933608 >> 172.21.15.46:0/4000933608 conn(0x557e280c0220 msgr2=0x557e281516d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3cd9ffb700 0 client.19815 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3d04eaab80 1 -- 172.21.15.46:0/4000933608 shutdown_connections 2024-03-18T20:48:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3d04eaab80 1 -- 172.21.15.46:0/4000933608 wait complete. 2024-03-18T20:48:06.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.335+0000 7f3d04eaab80 20 client.19815 trim_cache size 0 max 16384 2024-03-18T20:48:06.350 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:48:06.351 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:48:06.351 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:48:06.351 INFO:teuthology.orchestra.run.smithi046.stdout: "9af2f6da-514d-4f95-b58b-88b7a569c89b": { 2024-03-18T20:48:06.351 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:48:06.351 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "9af2f6da-514d-4f95-b58b-88b7a569c89b", 2024-03-18T20:48:06.351 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:48:06.351 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:48:06.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:48:06.352 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:48:06.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.336+0000 7f3cf8d15700 1 -- 172.21.15.46:0/3351907491 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3cdc0a1290 msgr2=0x7f3cdc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.336+0000 7f3cf8d15700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3cdc0a1290 0x7f3cdc0a3750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4011550 tx=0x7f3ce4009260 comp rx=0 tx=0).stop 2024-03-18T20:48:06.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.336+0000 7f3cf8d15700 1 -- 172.21.15.46:0/3351907491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf419bb90 msgr2=0x7f3cf419c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:06.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.336+0000 7f3cf8d15700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf419bb90 0x7f3cf419c010 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0062a0 tx=0x7f3cec00bd70 comp rx=0 tx=0).stop 2024-03-18T20:48:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.336+0000 7f3cf8d15700 1 -- 172.21.15.46:0/3351907491 shutdown_connections 2024-03-18T20:48:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.336+0000 7f3cf8d15700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3cdc0a1290 0x7f3cdc0a3750 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.336+0000 7f3cf8d15700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cf40ffe50 0x7f3cf41972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.336+0000 7f3cf8d15700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3cf419bb90 0x7f3cf419c010 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.336+0000 7f3cf8d15700 1 --2- 172.21.15.46:0/3351907491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cf4104f60 0x7f3cf41977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.336+0000 7f3cf8d15700 1 -- 172.21.15.46:0/3351907491 >> 172.21.15.46:0/3351907491 conn(0x7f3cf4076d40 msgr2=0x7f3cf4068160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:06.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.336+0000 7f3cf8d15700 1 -- 172.21.15.46:0/3351907491 shutdown_connections 2024-03-18T20:48:06.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:06.336+0000 7f3cf8d15700 1 -- 172.21.15.46:0/3351907491 wait complete. 2024-03-18T20:48:06.356 INFO:tasks.cephfs.filesystem:scrub status for tag:9af2f6da-514d-4f95-b58b-88b7a569c89b - {'path': '/', 'tag': '9af2f6da-514d-4f95-b58b-88b7a569c89b', 'options': 'recursive,force'} 2024-03-18T20:48:06.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:06 smithi073 ceph-mon[47384]: pgmap v374: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-03-18T20:48:06.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 458344 3.75 MB/sec execute 541 sec latency 63.327 ms 2024-03-18T20:48:07.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:07.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:07.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:07.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:07.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:07.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:07.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:07.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:07 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:07.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:07.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:07.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:07.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:07.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:07.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:07.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:07 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:07.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:07.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:07.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:07.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:07.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:07.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:07.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:07 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:07.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 459380 3.78 MB/sec execute 542 sec latency 85.630 ms 2024-03-18T20:48:08.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:08 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:48:08.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:08 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:08.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:08 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:08.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:08 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:08.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:08 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:08.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:08 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:08.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:08 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:08.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:08 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:08.291 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:08 smithi155 ceph-mon[48344]: pgmap v375: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-03-18T20:48:08.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:08 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:48:08.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:08 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:08.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:08 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:08.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:08 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:08.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:08 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:08.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:08 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:08.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:08 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:08.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:08 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:08.297 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:08 smithi046 ceph-mon[43495]: pgmap v375: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-03-18T20:48:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:08 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:48:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:08 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:08 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:08 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:08 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:08 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:08 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:08 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:08.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:08 smithi073 ceph-mon[47384]: pgmap v375: 97 pgs: 97 active+clean; 177 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-03-18T20:48:08.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 460030 3.78 MB/sec execute 543 sec latency 604.622 ms 2024-03-18T20:48:09.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 460030 3.77 MB/sec execute 544 sec latency 1604.698 ms 2024-03-18T20:48:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:10 smithi155 ceph-mon[48344]: pgmap v376: 97 pgs: 97 active+clean; 179 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.8 MiB/s wr, 214 op/s 2024-03-18T20:48:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:10 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:10.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:10 smithi155 ceph-mon[48344]: osdmap e94: 12 total, 12 up, 12 in 2024-03-18T20:48:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:10 smithi046 ceph-mon[43495]: pgmap v376: 97 pgs: 97 active+clean; 179 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.8 MiB/s wr, 214 op/s 2024-03-18T20:48:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:10 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:10.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:10 smithi046 ceph-mon[43495]: osdmap e94: 12 total, 12 up, 12 in 2024-03-18T20:48:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:10 smithi073 ceph-mon[47384]: pgmap v376: 97 pgs: 97 active+clean; 179 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.8 MiB/s wr, 214 op/s 2024-03-18T20:48:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:10 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:10 smithi073 ceph-mon[47384]: osdmap e94: 12 total, 12 up, 12 in 2024-03-18T20:48:10.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 460834 3.77 MB/sec execute 545 sec latency 2227.521 ms 2024-03-18T20:48:11.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:11 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:11.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:11 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:11.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:11 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:11.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.78 MB/sec execute 546 sec latency 628.763 ms 2024-03-18T20:48:12.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:12 smithi155 ceph-mon[48344]: pgmap v378: 97 pgs: 97 active+clean; 179 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-03-18T20:48:12.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:12 smithi046 ceph-mon[43495]: pgmap v378: 97 pgs: 97 active+clean; 179 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-03-18T20:48:12.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:12 smithi073 ceph-mon[47384]: pgmap v378: 97 pgs: 97 active+clean; 179 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-03-18T20:48:12.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.77 MB/sec execute 547 sec latency 1628.830 ms 2024-03-18T20:48:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:13 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:48:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:13 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:48:13.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:13 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:48:13.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:13 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:48:13.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:13 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:48:13.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:13 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:48:13.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:13 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:48:13.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:13 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:48:13.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:13 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:48:13.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.76 MB/sec execute 548 sec latency 2628.909 ms 2024-03-18T20:48:14.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:14 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:48:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:14 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:48:14.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:14 smithi155 ceph-mon[48344]: pgmap v379: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-18T20:48:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:14 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:48:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:14 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:48:14.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:14 smithi046 ceph-mon[43495]: pgmap v379: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-18T20:48:14.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:14 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:48:14.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:14 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:48:14.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:14 smithi073 ceph-mon[47384]: pgmap v379: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.6 MiB/s wr, 152 op/s 2024-03-18T20:48:14.721 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.76 MB/sec execute 549 sec latency 3629.001 ms 2024-03-18T20:48:15.722 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.75 MB/sec execute 550 sec latency 4629.121 ms 2024-03-18T20:48:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.963+0000 7f38a4fb6700 1 -- 172.21.15.46:0/1976640997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38a00ffe00 msgr2=0x7f38a01001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.963+0000 7f38a4fb6700 1 --2- 172.21.15.46:0/1976640997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38a00ffe00 0x7f38a01001e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f3890009990 tx=0x7f3890009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.963+0000 7f38a4fb6700 1 -- 172.21.15.46:0/1976640997 shutdown_connections 2024-03-18T20:48:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.963+0000 7f38a4fb6700 1 --2- 172.21.15.46:0/1976640997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38a00ffe00 0x7f38a01001e0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.963+0000 7f38a4fb6700 1 --2- 172.21.15.46:0/1976640997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a0104f20 0x7f38a010caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.963+0000 7f38a4fb6700 1 --2- 172.21.15.46:0/1976640997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a01007b0 0x7f38a01047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.963+0000 7f38a4fb6700 1 -- 172.21.15.46:0/1976640997 >> 172.21.15.46:0/1976640997 conn(0x7f38a00fb680 msgr2=0x7f38a00fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.963+0000 7f38a4fb6700 1 -- 172.21.15.46:0/1976640997 shutdown_connections 2024-03-18T20:48:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.963+0000 7f38a4fb6700 1 -- 172.21.15.46:0/1976640997 wait complete. 2024-03-18T20:48:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f38a4fb6700 1 Processor -- start 2024-03-18T20:48:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f38a4fb6700 1 -- start start 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f38a4fb6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38a00ffe00 0x7f38a0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f38a4fb6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0104f20 0x7f38a01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f38a4fb6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a0198b00 0x7f38a01a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f38a4fb6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a01a3790 con 0x7f38a00ffe00 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f38a4fb6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a01a3900 con 0x7f38a0104f20 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f38a4fb6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a01a3ac0 con 0x7f38a0198b00 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f389e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38a00ffe00 0x7f38a0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f389e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38a00ffe00 0x7f38a0198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37082/0 (socket says 172.21.15.46:37082) 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f389e59c700 1 -- 172.21.15.46:0/2428207960 learned_addr learned my addr 172.21.15.46:0/2428207960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f389ed9d700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a0198b00 0x7f38a01a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f389dd9b700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0104f20 0x7f38a01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f389e59c700 1 -- 172.21.15.46:0/2428207960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a0198b00 msgr2=0x7f38a01a3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f389e59c700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a0198b00 0x7f38a01a3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f389e59c700 1 -- 172.21.15.46:0/2428207960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0104f20 msgr2=0x7f38a01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f389e59c700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0104f20 0x7f38a01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f389e59c700 1 -- 172.21.15.46:0/2428207960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38900095f0 con 0x7f38a00ffe00 2024-03-18T20:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f389dd9b700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0104f20 0x7f38a01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.964+0000 7f389e59c700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38a00ffe00 0x7f38a0198080 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f3890005910 tx=0x7f3890009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.965+0000 7f38977fe700 1 -- 172.21.15.46:0/2428207960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389001d070 con 0x7f38a00ffe00 2024-03-18T20:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.965+0000 7f38977fe700 1 -- 172.21.15.46:0/2428207960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3890022470 con 0x7f38a00ffe00 2024-03-18T20:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.965+0000 7f38a4fb6700 1 -- 172.21.15.46:0/2428207960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a01a3d40 con 0x7f38a00ffe00 2024-03-18T20:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.965+0000 7f38977fe700 1 -- 172.21.15.46:0/2428207960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890017650 con 0x7f38a00ffe00 2024-03-18T20:48:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.965+0000 7f38a4fb6700 1 -- 172.21.15.46:0/2428207960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a01a4160 con 0x7f38a00ffe00 2024-03-18T20:48:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.967+0000 7f38977fe700 1 -- 172.21.15.46:0/2428207960 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f38900225e0 con 0x7f38a00ffe00 2024-03-18T20:48:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.968+0000 7f38977fe700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f388c0a1290 0x7f388c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.968+0000 7f38977fe700 1 -- 172.21.15.46:0/2428207960 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f38900ce0f0 con 0x7f38a00ffe00 2024-03-18T20:48:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.968+0000 7f389dd9b700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f388c0a1290 0x7f388c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.968+0000 7f389dd9b700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f388c0a1290 0x7f388c0a3750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f38a0069750 tx=0x7f3888009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.969+0000 7f38b114bb80 1 Processor -- start 2024-03-18T20:48:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.969+0000 7f38b114bb80 1 -- start start 2024-03-18T20:48:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.969+0000 7f38b114bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a71950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.969+0000 7f38b114bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a71e90 0x561f50a7c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.969+0000 7f38b114bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a7cdb0 0x561f50a7f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.969+0000 7f38b114bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f509cb930 con 0x561f50a71570 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.969+0000 7f38b114bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f509c92f0 con 0x561f50a71e90 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f38b114bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f509c4be0 con 0x561f50a7cdb0 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f389e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a71950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f389e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a71950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37092/0 (socket says 172.21.15.46:37092) 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f389e59c700 1 -- 172.21.15.46:0/3320707475 learned_addr learned my addr 172.21.15.46:0/3320707475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f389ed9d700 1 --2- 172.21.15.46:0/3320707475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a7cdb0 0x561f50a7f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f389dd9b700 1 --2- 172.21.15.46:0/3320707475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a71e90 0x561f50a7c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f389e59c700 1 -- 172.21.15.46:0/3320707475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a7cdb0 msgr2=0x561f50a7f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f389e59c700 1 --2- 172.21.15.46:0/3320707475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a7cdb0 0x561f50a7f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f389e59c700 1 -- 172.21.15.46:0/3320707475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a71e90 msgr2=0x561f50a7c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f389e59c700 1 --2- 172.21.15.46:0/3320707475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a71e90 0x561f50a7c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f389e59c700 1 -- 172.21.15.46:0/3320707475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f50846350 con 0x561f50a71570 2024-03-18T20:48:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f389e59c700 1 --2- 172.21.15.46:0/3320707475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a71950 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f389009b3c0 tx=0x7f3890022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6958b02ed5d0046 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f3894ff9700 1 -- 172.21.15.46:0/3320707475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389009bb30 con 0x561f50a71570 2024-03-18T20:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f3894ff9700 1 -- 172.21.15.46:0/3320707475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f389009b430 con 0x561f50a71570 2024-03-18T20:48:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.970+0000 7f3894ff9700 1 -- 172.21.15.46:0/3320707475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389009b8d0 con 0x561f50a71570 2024-03-18T20:48:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.971+0000 7f38b114bb80 1 -- 172.21.15.46:0/3320707475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 msgr2=0x561f50a71950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.971+0000 7f38b114bb80 1 --2- 172.21.15.46:0/3320707475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a71950 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f389009b3c0 tx=0x7f3890022f90 comp rx=0 tx=0).stop 2024-03-18T20:48:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.971+0000 7f38b114bb80 1 -- 172.21.15.46:0/3320707475 shutdown_connections 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.971+0000 7f38b114bb80 1 --2- 172.21.15.46:0/3320707475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a71950 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.971+0000 7f38b114bb80 1 --2- 172.21.15.46:0/3320707475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a7cdb0 0x561f50a7f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.971+0000 7f38b114bb80 1 --2- 172.21.15.46:0/3320707475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a71e90 0x561f50a7c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.971+0000 7f38b114bb80 1 -- 172.21.15.46:0/3320707475 >> 172.21.15.46:0/3320707475 conn(0x561f509e81c0 msgr2=0x561f50a6da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.971+0000 7f38b114bb80 1 -- 172.21.15.46:0/3320707475 shutdown_connections 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.971+0000 7f38b114bb80 1 -- 172.21.15.46:0/3320707475 wait complete. 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f38b114bb80 1 -- start start 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f38b114bb80 1 Processor -- start 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f38b114bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a7c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f38b114bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a71e90 0x561f50a79ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f38b114bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a7cdb0 0x561f50a7a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f38b114bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f50960d20 con 0x561f50a71570 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f38b114bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f509609c0 con 0x561f50a7cdb0 2024-03-18T20:48:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f38b114bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f50848640 con 0x561f50a71e90 2024-03-18T20:48:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f38b114bb80 10 client.0 _ll_register_callbacks cb 0x561f507cc340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f389e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a7c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f389e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a7c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37108/0 (socket says 172.21.15.46:37108) 2024-03-18T20:48:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f389e59c700 1 -- 172.21.15.46:0/2210998632 learned_addr learned my addr 172.21.15.46:0/2210998632 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f389dd9b700 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a71e90 0x561f50a79ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.972+0000 7f389ed9d700 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a7cdb0 0x561f50a7a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f389e59c700 1 -- 172.21.15.46:0/2210998632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a71e90 msgr2=0x561f50a79ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f389e59c700 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a71e90 0x561f50a79ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f389e59c700 1 -- 172.21.15.46:0/2210998632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a7cdb0 msgr2=0x561f50a7a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f389e59c700 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a7cdb0 0x561f50a7a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f389e59c700 1 -- 172.21.15.46:0/2210998632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a01a4160 con 0x561f50a71570 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f389e59c700 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a7c7e0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f3890097930 tx=0x7f389009f170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bba667106575b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f3885ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f3885ffb700 1 -- 172.21.15.46:0/2210998632 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890065070 con 0x561f50a71570 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f3885ffb700 1 -- 172.21.15.46:0/2210998632 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f38900a3820 con 0x561f50a71570 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f38b114bb80 1 -- 172.21.15.46:0/2210998632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f50a656c0 con 0x561f50a71570 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f3885ffb700 1 -- 172.21.15.46:0/2210998632 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f389009bb30 con 0x561f50a71570 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f38b114bb80 10 client.19827 fetch_fsmap learned FSMap version 53 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f38b114bb80 1 -- 172.21.15.46:0/2210998632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x561f507f2b00 con 0x561f50a71570 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.973+0000 7f3885ffb700 1 -- 172.21.15.46:0/2210998632 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f50a656c0 con 0x561f50a71570 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.974+0000 7f3885ffb700 1 -- 172.21.15.46:0/2210998632 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 53) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f38900a3390 con 0x561f50a71570 2024-03-18T20:48:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.974+0000 7f38b114bb80 10 client.19827 fetch_fsmap finished waiting for FSMap version 53 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.974+0000 7f38b114bb80 10 client.19827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.974+0000 7f38b114bb80 20 client.19827 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.974+0000 7f38b114bb80 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f50a8c200 0x561f50a8e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.974+0000 7f38b114bb80 4 client.19827 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.974+0000 7f38b114bb80 1 -- 172.21.15.46:0/2210998632 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f509bd6a0 con 0x561f50a8c200 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.974+0000 7f389dd9b700 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f50a8c200 0x561f50a8e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.974+0000 7f3885ffb700 10 client.19827 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.974+0000 7f389dd9b700 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f50a8c200 0x561f50a8e5f0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a658c46b822e8165 server_cookie=5c2aa76df80314ef in_seq=0 out_seq=0 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f3885ffb700 1 -- 172.21.15.46:0/2210998632 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561f509bd6a0 con 0x561f50a8c200 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f3885ffb700 10 client.19827 handle_command_reply: tid=0 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f38b114bb80 1 client.19827 shutdown 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f38b114bb80 1 -- 172.21.15.46:0/2210998632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 msgr2=0x561f50a7c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f38b114bb80 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a7c7e0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f3890097930 tx=0x7f389009f170 comp rx=0 tx=0).stop 2024-03-18T20:48:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f38b114bb80 1 -- 172.21.15.46:0/2210998632 shutdown_connections 2024-03-18T20:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f38b114bb80 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f50a8c200 0x561f50a8e5f0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f38b114bb80 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a7c7e0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f38b114bb80 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a7cdb0 0x561f50a7a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f3885ffb700 0 client.19827 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f38b114bb80 1 --2- 172.21.15.46:0/2210998632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a71e90 0x561f50a79ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f38b114bb80 1 -- 172.21.15.46:0/2210998632 >> 172.21.15.46:0/2210998632 conn(0x561f509e81c0 msgr2=0x561f50a6d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f38b114bb80 1 -- 172.21.15.46:0/2210998632 shutdown_connections 2024-03-18T20:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f38b114bb80 1 -- 172.21.15.46:0/2210998632 wait complete. 2024-03-18T20:48:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.975+0000 7f38b114bb80 20 client.19827 trim_cache size 0 max 16384 2024-03-18T20:48:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.991+0000 7f38b114bb80 1 Processor -- start 2024-03-18T20:48:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.991+0000 7f38b114bb80 1 -- start start 2024-03-18T20:48:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.991+0000 7f38b114bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a71570 0x561f50a7a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.991+0000 7f38b114bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71e90 0x561f50a811c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.991+0000 7f38b114bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a7cdb0 0x561f50a7c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.991+0000 7f38b114bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f50a72850 con 0x561f50a71e90 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.991+0000 7f38b114bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f50a67c10 con 0x561f50a7cdb0 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.991+0000 7f38b114bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f50a75760 con 0x561f50a71570 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.991+0000 7f389dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71e90 0x561f50a811c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.991+0000 7f389dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71e90 0x561f50a811c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37116/0 (socket says 172.21.15.46:37116) 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.991+0000 7f389dd9b700 1 -- 172.21.15.46:0/2944324361 learned_addr learned my addr 172.21.15.46:0/2944324361 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.991+0000 7f389e59c700 1 --2- 172.21.15.46:0/2944324361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a71570 0x561f50a7a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.992+0000 7f389ed9d700 1 --2- 172.21.15.46:0/2944324361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a7cdb0 0x561f50a7c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.992+0000 7f389dd9b700 1 -- 172.21.15.46:0/2944324361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a71570 msgr2=0x561f50a7a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.992+0000 7f389dd9b700 1 --2- 172.21.15.46:0/2944324361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a71570 0x561f50a7a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.992+0000 7f389dd9b700 1 -- 172.21.15.46:0/2944324361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a7cdb0 msgr2=0x561f50a7c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.992+0000 7f389dd9b700 1 --2- 172.21.15.46:0/2944324361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a7cdb0 0x561f50a7c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.992+0000 7f389dd9b700 1 -- 172.21.15.46:0/2944324361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f5098ae40 con 0x561f50a71e90 2024-03-18T20:48:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.992+0000 7f389dd9b700 1 --2- 172.21.15.46:0/2944324361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71e90 0x561f50a811c0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f38a0067d10 tx=0x7f3888013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a59de63fada799 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.992+0000 7f38957fa700 1 -- 172.21.15.46:0/2944324361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888025040 con 0x561f50a71e90 2024-03-18T20:48:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.992+0000 7f38957fa700 1 -- 172.21.15.46:0/2944324361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38880044f0 con 0x561f50a71e90 2024-03-18T20:48:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.992+0000 7f38957fa700 1 -- 172.21.15.46:0/2944324361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888003da0 con 0x561f50a71e90 2024-03-18T20:48:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.992+0000 7f38b114bb80 1 -- 172.21.15.46:0/2944324361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71e90 msgr2=0x561f50a811c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 --2- 172.21.15.46:0/2944324361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71e90 0x561f50a811c0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f38a0067d10 tx=0x7f3888013000 comp rx=0 tx=0).stop 2024-03-18T20:48:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 -- 172.21.15.46:0/2944324361 shutdown_connections 2024-03-18T20:48:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 --2- 172.21.15.46:0/2944324361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a7cdb0 0x561f50a7c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 --2- 172.21.15.46:0/2944324361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a71570 0x561f50a7a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 --2- 172.21.15.46:0/2944324361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71e90 0x561f50a811c0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 -- 172.21.15.46:0/2944324361 >> 172.21.15.46:0/2944324361 conn(0x561f509e81c0 msgr2=0x561f50a92f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 -- 172.21.15.46:0/2944324361 shutdown_connections 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 -- 172.21.15.46:0/2944324361 wait complete. 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 -- start start 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 Processor -- start 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a7f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a71e90 0x561f50a7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a7cdb0 0x561f50adf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f50a80050 con 0x561f50a71570 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f50adf570 con 0x561f50a71e90 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f50adf730 con 0x561f50a7cdb0 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f38b114bb80 10 client.0 _ll_register_callbacks cb 0x7f3888016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:16.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f389e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a7f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f389e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a7f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37124/0 (socket says 172.21.15.46:37124) 2024-03-18T20:48:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.993+0000 7f389e59c700 1 -- 172.21.15.46:0/323954512 learned_addr learned my addr 172.21.15.46:0/323954512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f389ed9d700 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a7cdb0 0x561f50adf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f389dd9b700 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a71e90 0x561f50a7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f389e59c700 1 -- 172.21.15.46:0/323954512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a7cdb0 msgr2=0x561f50adf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f389e59c700 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a7cdb0 0x561f50adf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f389e59c700 1 -- 172.21.15.46:0/323954512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a71e90 msgr2=0x561f50a7f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f389e59c700 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a71e90 0x561f50a7f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f389e59c700 1 -- 172.21.15.46:0/323954512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38880109e0 con 0x561f50a71570 2024-03-18T20:48:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f389ed9d700 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a7cdb0 0x561f50adf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f389dd9b700 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a71e90 0x561f50a7f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:16.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f389e59c700 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a7f390 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f38900a1840 tx=0x7f38900a3650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f9b610cc5c6edb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f3885ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f3885ffb700 1 -- 172.21.15.46:0/323954512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38900a1310 con 0x561f50a71570 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f3885ffb700 1 -- 172.21.15.46:0/323954512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f389009bb30 con 0x561f50a71570 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f38b114bb80 1 -- 172.21.15.46:0/323954512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f389009b430 con 0x561f50a71570 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f3885ffb700 1 -- 172.21.15.46:0/323954512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f389009b200 con 0x561f50a71570 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f38b114bb80 10 client.19833 fetch_fsmap learned FSMap version 53 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f38b114bb80 1 -- 172.21.15.46:0/323954512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x561f5084ac70 con 0x561f50a71570 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.994+0000 7f3885ffb700 1 -- 172.21.15.46:0/323954512 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f389009b430 con 0x561f50a71570 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.995+0000 7f3885ffb700 1 -- 172.21.15.46:0/323954512 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 53) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3890097220 con 0x561f50a71570 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.995+0000 7f38b114bb80 10 client.19833 fetch_fsmap finished waiting for FSMap version 53 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.995+0000 7f38b114bb80 10 client.19833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.995+0000 7f38b114bb80 20 client.19833 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.995+0000 7f38b114bb80 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f50ae09f0 0x561f50ae2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.995+0000 7f38b114bb80 4 client.19833 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:48:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.995+0000 7f38b114bb80 1 -- 172.21.15.46:0/323954512 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f50813c90 con 0x561f50ae09f0 2024-03-18T20:48:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.995+0000 7f389dd9b700 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f50ae09f0 0x561f50ae2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.995+0000 7f3885ffb700 10 client.19833 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:15.995+0000 7f389dd9b700 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f50ae09f0 0x561f50ae2de0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2372ef040fc6d9c server_cookie=ff5b6ced992f1ede in_seq=0 out_seq=0 2024-03-18T20:48:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f3885ffb700 1 -- 172.21.15.46:0/323954512 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f50813c90 con 0x561f50ae09f0 2024-03-18T20:48:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f3885ffb700 10 client.19833 handle_command_reply: tid=0 2024-03-18T20:48:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f38b114bb80 1 client.19833 shutdown 2024-03-18T20:48:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f38b114bb80 1 -- 172.21.15.46:0/323954512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 msgr2=0x561f50a7f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f38b114bb80 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a7f390 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f38900a1840 tx=0x7f38900a3650 comp rx=0 tx=0).stop 2024-03-18T20:48:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f38b114bb80 1 -- 172.21.15.46:0/323954512 shutdown_connections 2024-03-18T20:48:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f38b114bb80 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f50ae09f0 0x561f50ae2de0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f38b114bb80 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f50a71570 0x561f50a7f390 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f38b114bb80 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f50a7cdb0 0x561f50adf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f3885ffb700 0 client.19833 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f38b114bb80 1 --2- 172.21.15.46:0/323954512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f50a71e90 0x561f50a7f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f38b114bb80 1 -- 172.21.15.46:0/323954512 >> 172.21.15.46:0/323954512 conn(0x561f509e81c0 msgr2=0x561f50a87920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f38b114bb80 1 -- 172.21.15.46:0/323954512 shutdown_connections 2024-03-18T20:48:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f38b114bb80 1 -- 172.21.15.46:0/323954512 wait complete. 2024-03-18T20:48:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.035+0000 7f38b114bb80 20 client.19833 trim_cache size 0 max 16384 2024-03-18T20:48:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:48:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:48:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:48:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:48:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.036+0000 7f38a4fb6700 1 -- 172.21.15.46:0/2428207960 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f388c0a1290 msgr2=0x7f388c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.036+0000 7f38a4fb6700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f388c0a1290 0x7f388c0a3750 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f38a0069750 tx=0x7f3888009260 comp rx=0 tx=0).stop 2024-03-18T20:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.036+0000 7f38a4fb6700 1 -- 172.21.15.46:0/2428207960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38a00ffe00 msgr2=0x7f38a0198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.036+0000 7f38a4fb6700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38a00ffe00 0x7f38a0198080 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f3890005910 tx=0x7f3890009570 comp rx=0 tx=0).stop 2024-03-18T20:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.036+0000 7f38a4fb6700 1 -- 172.21.15.46:0/2428207960 shutdown_connections 2024-03-18T20:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.036+0000 7f38a4fb6700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f388c0a1290 0x7f388c0a3750 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.036+0000 7f38a4fb6700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38a00ffe00 0x7f38a0198080 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.036+0000 7f38a4fb6700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38a0198b00 0x7f38a01a3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.036+0000 7f38a4fb6700 1 --2- 172.21.15.46:0/2428207960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38a0104f20 0x7f38a01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.036+0000 7f38a4fb6700 1 -- 172.21.15.46:0/2428207960 >> 172.21.15.46:0/2428207960 conn(0x7f38a00fb680 msgr2=0x7f38a00fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.036+0000 7f38a4fb6700 1 -- 172.21.15.46:0/2428207960 shutdown_connections 2024-03-18T20:48:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:16.036+0000 7f38a4fb6700 1 -- 172.21.15.46:0/2428207960 wait complete. 2024-03-18T20:48:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:16 smithi155 ceph-mon[48344]: pgmap v380: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-03-18T20:48:16.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:16 smithi046 ceph-mon[43495]: pgmap v380: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-03-18T20:48:16.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:16 smithi073 ceph-mon[47384]: pgmap v380: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-03-18T20:48:16.722 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.74 MB/sec execute 551 sec latency 5629.214 ms 2024-03-18T20:48:17.722 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.74 MB/sec execute 552 sec latency 6629.330 ms 2024-03-18T20:48:18.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:18 smithi155 ceph-mon[48344]: pgmap v381: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-03-18T20:48:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:18 smithi046 ceph-mon[43495]: pgmap v381: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-03-18T20:48:18.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:18 smithi073 ceph-mon[47384]: pgmap v381: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-03-18T20:48:18.722 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.73 MB/sec execute 553 sec latency 7629.434 ms 2024-03-18T20:48:19.722 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.72 MB/sec execute 554 sec latency 8629.558 ms 2024-03-18T20:48:20.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:20 smithi155 ceph-mon[48344]: pgmap v382: 97 pgs: 97 active+clean; 183 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.4 MiB/s wr, 85 op/s 2024-03-18T20:48:20.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:20 smithi046 ceph-mon[43495]: pgmap v382: 97 pgs: 97 active+clean; 183 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.4 MiB/s wr, 85 op/s 2024-03-18T20:48:20.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:20 smithi073 ceph-mon[47384]: pgmap v382: 97 pgs: 97 active+clean; 183 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.4 MiB/s wr, 85 op/s 2024-03-18T20:48:20.722 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.72 MB/sec execute 555 sec latency 9629.592 ms 2024-03-18T20:48:21.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:21 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:48:21.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:21 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:48:21.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:21 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:48:21.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:21 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:48:21.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:21 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:48:21.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:21 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:48:21.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:21 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:48:21.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:21 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:48:21.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:21 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:48:21.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:21 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:48:21.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:21 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:48:21.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:21 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:48:21.722 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.71 MB/sec execute 556 sec latency 10629.679 ms 2024-03-18T20:48:22.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:22 smithi155 ceph-mon[48344]: pgmap v383: 97 pgs: 97 active+clean; 183 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2024-03-18T20:48:22.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:22 smithi046 ceph-mon[43495]: pgmap v383: 97 pgs: 97 active+clean; 183 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2024-03-18T20:48:22.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:22 smithi073 ceph-mon[47384]: pgmap v383: 97 pgs: 97 active+clean; 183 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2024-03-18T20:48:22.722 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.70 MB/sec execute 557 sec latency 11629.773 ms 2024-03-18T20:48:23.723 INFO:tasks.workunit.client.0.smithi046.stdout: 1 461619 3.70 MB/sec execute 558 sec latency 12629.859 ms 2024-03-18T20:48:24.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:24 smithi155 ceph-mon[48344]: pgmap v384: 97 pgs: 97 active+clean; 181 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.5 MiB/s wr, 93 op/s 2024-03-18T20:48:24.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:24 smithi046 ceph-mon[43495]: pgmap v384: 97 pgs: 97 active+clean; 181 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.5 MiB/s wr, 93 op/s 2024-03-18T20:48:24.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:24 smithi073 ceph-mon[47384]: pgmap v384: 97 pgs: 97 active+clean; 181 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.5 MiB/s wr, 93 op/s 2024-03-18T20:48:24.722 INFO:tasks.workunit.client.0.smithi046.stdout: 1 462296 3.70 MB/sec execute 559 sec latency 13096.906 ms 2024-03-18T20:48:25.723 INFO:tasks.workunit.client.0.smithi046.stdout: 1 463595 3.70 MB/sec execute 560 sec latency 169.325 ms 2024-03-18T20:48:26.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:26 smithi155 ceph-mon[48344]: pgmap v385: 97 pgs: 97 active+clean; 181 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-18T20:48:26.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:26 smithi046 ceph-mon[43495]: pgmap v385: 97 pgs: 97 active+clean; 181 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-18T20:48:26.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:26 smithi073 ceph-mon[47384]: pgmap v385: 97 pgs: 97 active+clean; 181 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-18T20:48:26.723 INFO:tasks.workunit.client.0.smithi046.stdout: 1 463595 3.69 MB/sec execute 561 sec latency 1169.425 ms 2024-03-18T20:48:27.723 INFO:tasks.workunit.client.0.smithi046.stdout: 1 463595 3.69 MB/sec execute 562 sec latency 2169.541 ms 2024-03-18T20:48:28.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:28 smithi155 ceph-mon[48344]: pgmap v386: 97 pgs: 97 active+clean; 181 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-18T20:48:28.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:28 smithi046 ceph-mon[43495]: pgmap v386: 97 pgs: 97 active+clean; 181 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-18T20:48:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:28 smithi073 ceph-mon[47384]: pgmap v386: 97 pgs: 97 active+clean; 181 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-03-18T20:48:28.723 INFO:tasks.workunit.client.0.smithi046.stdout: 1 463595 3.68 MB/sec execute 563 sec latency 3169.632 ms 2024-03-18T20:48:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.163+0000 7fc08cd4c700 1 -- 172.21.15.46:0/3725196269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc088100570 msgr2=0x7fc08810ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.163+0000 7fc08cd4c700 1 --2- 172.21.15.46:0/3725196269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc088100570 0x7fc08810ca00 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc08000b0a0 tx=0x7fc08000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.163+0000 7fc08cd4c700 1 -- 172.21.15.46:0/3725196269 shutdown_connections 2024-03-18T20:48:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.163+0000 7fc08cd4c700 1 --2- 172.21.15.46:0/3725196269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc088100570 0x7fc08810ca00 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.163+0000 7fc08cd4c700 1 --2- 172.21.15.46:0/3725196269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc088105c60 0x7fc088106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.163+0000 7fc08cd4c700 1 --2- 172.21.15.46:0/3725196269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0880ffbb0 0x7fc088100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.163+0000 7fc08cd4c700 1 -- 172.21.15.46:0/3725196269 >> 172.21.15.46:0/3725196269 conn(0x7fc088076d50 msgr2=0x7fc088077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.163+0000 7fc08cd4c700 1 -- 172.21.15.46:0/3725196269 shutdown_connections 2024-03-18T20:48:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.163+0000 7fc08cd4c700 1 -- 172.21.15.46:0/3725196269 wait complete. 2024-03-18T20:48:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.164+0000 7fc08cd4c700 1 Processor -- start 2024-03-18T20:48:29.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.164+0000 7fc08cd4c700 1 -- start start 2024-03-18T20:48:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.164+0000 7fc08cd4c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0880ffbb0 0x7fc088101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.164+0000 7fc08cd4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc088100570 0x7fc088101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.164+0000 7fc08cd4c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc088105c60 0x7fc0881022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.164+0000 7fc08cd4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0881a56c0 con 0x7fc088100570 2024-03-18T20:48:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.164+0000 7fc08cd4c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0881a5830 con 0x7fc0880ffbb0 2024-03-18T20:48:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.164+0000 7fc08cd4c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0881a59f0 con 0x7fc088105c60 2024-03-18T20:48:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.164+0000 7fc085d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc088100570 0x7fc088101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.164+0000 7fc085d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc088100570 0x7fc088101d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40306/0 (socket says 172.21.15.46:40306) 2024-03-18T20:48:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.164+0000 7fc085d9b700 1 -- 172.21.15.46:0/1167906275 learned_addr learned my addr 172.21.15.46:0/1167906275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc086d9d700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc088105c60 0x7fc0881022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc08659c700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0880ffbb0 0x7fc088101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc085d9b700 1 -- 172.21.15.46:0/1167906275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc088105c60 msgr2=0x7fc0881022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc085d9b700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc088105c60 0x7fc0881022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc085d9b700 1 -- 172.21.15.46:0/1167906275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0880ffbb0 msgr2=0x7fc088101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc085d9b700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0880ffbb0 0x7fc088101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc085d9b700 1 -- 172.21.15.46:0/1167906275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080009d00 con 0x7fc088100570 2024-03-18T20:48:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc08659c700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0880ffbb0 0x7fc088101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc085d9b700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc088100570 0x7fc088101d90 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00d950 tx=0x7fc07c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc0777fe700 1 -- 172.21.15.46:0/1167906275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c009940 con 0x7fc088100570 2024-03-18T20:48:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc08cd4c700 1 -- 172.21.15.46:0/1167906275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0881a5cd0 con 0x7fc088100570 2024-03-18T20:48:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc0777fe700 1 -- 172.21.15.46:0/1167906275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c010460 con 0x7fc088100570 2024-03-18T20:48:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc0777fe700 1 -- 172.21.15.46:0/1167906275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00f620 con 0x7fc088100570 2024-03-18T20:48:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.165+0000 7fc08cd4c700 1 -- 172.21.15.46:0/1167906275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0881a61b0 con 0x7fc088100570 2024-03-18T20:48:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.168+0000 7fc0777fe700 1 -- 172.21.15.46:0/1167906275 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc07c0108d0 con 0x7fc088100570 2024-03-18T20:48:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.168+0000 7fc0777fe700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0700a1290 0x7fc0700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.168+0000 7fc0777fe700 1 -- 172.21.15.46:0/1167906275 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc07c0ccb70 con 0x7fc088100570 2024-03-18T20:48:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.168+0000 7fc08659c700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0700a1290 0x7fc0700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.169+0000 7fc08659c700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0700a1290 0x7fc0700a3750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc07800bb90 tx=0x7fc078005d80 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.169+0000 7fc098ee1b80 1 Processor -- start 2024-03-18T20:48:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc098ee1b80 1 -- start start 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc098ee1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8aced950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc098ee1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc098ee1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8acfb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc098ee1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8ac47930 con 0x55fe8aced570 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc098ee1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8ac452f0 con 0x55fe8acede90 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc098ee1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8ac40be0 con 0x55fe8acf8db0 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc08659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8aced950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc08659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8aced950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40314/0 (socket says 172.21.15.46:40314) 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc08659c700 1 -- 172.21.15.46:0/60220813 learned_addr learned my addr 172.21.15.46:0/60220813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc085d9b700 1 --2- 172.21.15.46:0/60220813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc086d9d700 1 --2- 172.21.15.46:0/60220813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8acfb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc08659c700 1 -- 172.21.15.46:0/60220813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 msgr2=0x55fe8acfb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc08659c700 1 --2- 172.21.15.46:0/60220813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8acfb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc08659c700 1 -- 172.21.15.46:0/60220813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 msgr2=0x55fe8acf8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc08659c700 1 --2- 172.21.15.46:0/60220813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc08659c700 1 -- 172.21.15.46:0/60220813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe8aac2350 con 0x55fe8aced570 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.170+0000 7fc08659c700 1 --2- 172.21.15.46:0/60220813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8aced950 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b970 tx=0x7fc078012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c43168b1fef5c79 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.171+0000 7fc074ff9700 1 -- 172.21.15.46:0/60220813 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078011ad0 con 0x55fe8aced570 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.171+0000 7fc074ff9700 1 -- 172.21.15.46:0/60220813 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc078011c30 con 0x55fe8aced570 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.171+0000 7fc074ff9700 1 -- 172.21.15.46:0/60220813 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07801d610 con 0x55fe8aced570 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.171+0000 7fc098ee1b80 1 -- 172.21.15.46:0/60220813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 msgr2=0x55fe8aced950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.171+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/60220813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8aced950 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b970 tx=0x7fc078012000 comp rx=0 tx=0).stop 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.172+0000 7fc098ee1b80 1 -- 172.21.15.46:0/60220813 shutdown_connections 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.172+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/60220813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8aced950 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.172+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/60220813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8acfb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.172+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/60220813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.172+0000 7fc098ee1b80 1 -- 172.21.15.46:0/60220813 >> 172.21.15.46:0/60220813 conn(0x55fe8ac641c0 msgr2=0x55fe8ace9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.172+0000 7fc098ee1b80 1 -- 172.21.15.46:0/60220813 shutdown_connections 2024-03-18T20:48:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.172+0000 7fc098ee1b80 1 -- 172.21.15.46:0/60220813 wait complete. 2024-03-18T20:48:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.172+0000 7fc098ee1b80 1 -- start start 2024-03-18T20:48:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.172+0000 7fc098ee1b80 1 Processor -- start 2024-03-18T20:48:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.172+0000 7fc098ee1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8acf8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.172+0000 7fc098ee1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc098ee1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8acf6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc098ee1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8abdcd20 con 0x55fe8aced570 2024-03-18T20:48:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc098ee1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8abdc9c0 con 0x55fe8acede90 2024-03-18T20:48:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc098ee1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8aac4640 con 0x55fe8acf8db0 2024-03-18T20:48:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc098ee1b80 10 client.0 _ll_register_callbacks cb 0x55fe8aa48340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc085d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc086d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8acf6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc085d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:46740/0 (socket says 172.21.15.46:46740) 2024-03-18T20:48:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc085d9b700 1 -- 172.21.15.46:0/68043533 learned_addr learned my addr 172.21.15.46:0/68043533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc085d9b700 1 -- 172.21.15.46:0/68043533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 msgr2=0x55fe8acf6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc085d9b700 1 --2- 172.21.15.46:0/68043533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8acf6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc085d9b700 1 -- 172.21.15.46:0/68043533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 msgr2=0x55fe8acf8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc085d9b700 1 --2- 172.21.15.46:0/68043533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8acf8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc085d9b700 1 -- 172.21.15.46:0/68043533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07800f9b0 con 0x55fe8acede90 2024-03-18T20:48:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc086d9d700 1 --2- 172.21.15.46:0/68043533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8acf6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc085d9b700 1 --2- 172.21.15.46:0/68043533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf5b10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc07c096160 tx=0x7fc07c099c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead266658b3b4cd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.173+0000 7fc06dffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc06dffb700 1 -- 172.21.15.46:0/68043533 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c09a0c0 con 0x55fe8acede90 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc06dffb700 1 -- 172.21.15.46:0/68043533 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc07c099150 con 0x55fe8acede90 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc098ee1b80 1 -- 172.21.15.46:0/68043533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe8ace16c0 con 0x55fe8acede90 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc06dffb700 1 -- 172.21.15.46:0/68043533 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc07c099310 con 0x55fe8acede90 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc098ee1b80 10 client.29453 fetch_fsmap learned FSMap version 54 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc098ee1b80 1 -- 172.21.15.46:0/68043533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55fe8aa6eb00 con 0x55fe8acede90 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc06dffb700 1 -- 172.21.15.46:0/68043533 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c0b8030 con 0x55fe8acede90 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc06dffb700 1 -- 172.21.15.46:0/68043533 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 54) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc07c095ae0 con 0x55fe8acede90 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc098ee1b80 10 client.29453 fetch_fsmap finished waiting for FSMap version 54 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc098ee1b80 10 client.29453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc098ee1b80 20 client.29453 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/68043533 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fe8ad08970 0x55fe8ad0ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc098ee1b80 4 client.29453 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.174+0000 7fc098ee1b80 1 -- 172.21.15.46:0/68043533 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe8ac396a0 con 0x55fe8ad08970 2024-03-18T20:48:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.175+0000 7fc08659c700 1 --2- 172.21.15.46:0/68043533 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fe8ad08970 0x55fe8ad0ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.175+0000 7fc06dffb700 10 client.29453 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.175+0000 7fc08659c700 1 --2- 172.21.15.46:0/68043533 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fe8ad08970 0x55fe8ad0ad90 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f95af351a14d58 server_cookie=14b673a859ecff1b in_seq=0 out_seq=0 2024-03-18T20:48:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc06dffb700 1 -- 172.21.15.46:0/68043533 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fe8ac396a0 con 0x55fe8ad08970 2024-03-18T20:48:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc06dffb700 10 client.29453 handle_command_reply: tid=0 2024-03-18T20:48:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc098ee1b80 1 client.29453 shutdown 2024-03-18T20:48:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc098ee1b80 1 -- 172.21.15.46:0/68043533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 msgr2=0x55fe8acf5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/68043533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf5b10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc07c096160 tx=0x7fc07c099c60 comp rx=0 tx=0).stop 2024-03-18T20:48:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc098ee1b80 1 -- 172.21.15.46:0/68043533 shutdown_connections 2024-03-18T20:48:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/68043533 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fe8ad08970 0x55fe8ad0ad90 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/68043533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8acf8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/68043533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8acf6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc06dffb700 0 client.29453 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:29.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/68043533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf5b10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc098ee1b80 1 -- 172.21.15.46:0/68043533 >> 172.21.15.46:0/68043533 conn(0x55fe8ac641c0 msgr2=0x55fe8ace9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc098ee1b80 1 -- 172.21.15.46:0/68043533 shutdown_connections 2024-03-18T20:48:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc098ee1b80 1 -- 172.21.15.46:0/68043533 wait complete. 2024-03-18T20:48:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.176+0000 7fc098ee1b80 20 client.29453 trim_cache size 0 max 16384 2024-03-18T20:48:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.194+0000 7fc098ee1b80 1 Processor -- start 2024-03-18T20:48:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.194+0000 7fc098ee1b80 1 -- start start 2024-03-18T20:48:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.194+0000 7fc098ee1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8acffa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.194+0000 7fc098ee1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.194+0000 7fc098ee1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8ad04190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.194+0000 7fc098ee1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8acf65b0 con 0x55fe8aced570 2024-03-18T20:48:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.194+0000 7fc098ee1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8acfff80 con 0x55fe8acede90 2024-03-18T20:48:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.194+0000 7fc098ee1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8acf1950 con 0x55fe8acf8db0 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.194+0000 7fc08659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8acffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.194+0000 7fc08659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8acffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40330/0 (socket says 172.21.15.46:40330) 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.194+0000 7fc08659c700 1 -- 172.21.15.46:0/257054137 learned_addr learned my addr 172.21.15.46:0/257054137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.195+0000 7fc086d9d700 1 --2- 172.21.15.46:0/257054137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8ad04190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.195+0000 7fc085d9b700 1 --2- 172.21.15.46:0/257054137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.195+0000 7fc08659c700 1 -- 172.21.15.46:0/257054137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 msgr2=0x55fe8ad04190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.195+0000 7fc08659c700 1 --2- 172.21.15.46:0/257054137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8ad04190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.195+0000 7fc08659c700 1 -- 172.21.15.46:0/257054137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 msgr2=0x55fe8acf8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.195+0000 7fc08659c700 1 --2- 172.21.15.46:0/257054137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.195+0000 7fc08659c700 1 -- 172.21.15.46:0/257054137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe8ac06e40 con 0x55fe8aced570 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.195+0000 7fc085d9b700 1 --2- 172.21.15.46:0/257054137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.195+0000 7fc08659c700 1 --2- 172.21.15.46:0/257054137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8acffa40 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fc07801ee30 tx=0x7fc07801edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218e24056f859e29 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.195+0000 7fc0757fa700 1 -- 172.21.15.46:0/257054137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07802d040 con 0x55fe8aced570 2024-03-18T20:48:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.195+0000 7fc0757fa700 1 -- 172.21.15.46:0/257054137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc078031d50 con 0x55fe8aced570 2024-03-18T20:48:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.195+0000 7fc0757fa700 1 -- 172.21.15.46:0/257054137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078031730 con 0x55fe8aced570 2024-03-18T20:48:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.196+0000 7fc098ee1b80 1 -- 172.21.15.46:0/257054137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 msgr2=0x55fe8acffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.196+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/257054137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8acffa40 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fc07801ee30 tx=0x7fc07801edc0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.196+0000 7fc098ee1b80 1 -- 172.21.15.46:0/257054137 shutdown_connections 2024-03-18T20:48:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.196+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/257054137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8aced570 0x55fe8acffa40 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.196+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/257054137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8acf8db0 0x55fe8ad04190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.196+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/257054137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acede90 0x55fe8acf8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.196+0000 7fc098ee1b80 1 -- 172.21.15.46:0/257054137 >> 172.21.15.46:0/257054137 conn(0x55fe8ac641c0 msgr2=0x55fe8ace5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.196+0000 7fc098ee1b80 1 -- 172.21.15.46:0/257054137 shutdown_connections 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.196+0000 7fc098ee1b80 1 -- 172.21.15.46:0/257054137 wait complete. 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.196+0000 7fc098ee1b80 1 -- start start 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc098ee1b80 1 Processor -- start 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc098ee1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8aced570 0x55fe8acfd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc098ee1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8acede90 0x55fe8acfdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc098ee1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acf8db0 0x55fe8ad5b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc098ee1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8acfe3c0 con 0x55fe8acede90 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc098ee1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8ad5bce0 con 0x55fe8acf8db0 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc098ee1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe8ad5bea0 con 0x55fe8aced570 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc098ee1b80 10 client.0 _ll_register_callbacks cb 0x7fc07c0995c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc085d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8acede90 0x55fe8acfdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc085d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8acede90 0x55fe8acfdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40346/0 (socket says 172.21.15.46:40346) 2024-03-18T20:48:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc085d9b700 1 -- 172.21.15.46:0/925515421 learned_addr learned my addr 172.21.15.46:0/925515421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc08659c700 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8aced570 0x55fe8acfd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.197+0000 7fc086d9d700 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acf8db0 0x55fe8ad5b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc085d9b700 1 -- 172.21.15.46:0/925515421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8aced570 msgr2=0x55fe8acfd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc085d9b700 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8aced570 0x55fe8acfd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc085d9b700 1 -- 172.21.15.46:0/925515421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acf8db0 msgr2=0x55fe8ad5b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc085d9b700 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acf8db0 0x55fe8ad5b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc085d9b700 1 -- 172.21.15.46:0/925515421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe8aac2350 con 0x55fe8acede90 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc085d9b700 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8acede90 0x55fe8acfdc70 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc07c097fb0 tx=0x7fc07c095c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed83bb3290e2b9dd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc06dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc06dffb700 1 -- 172.21.15.46:0/925515421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c098bc0 con 0x55fe8acede90 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc06dffb700 1 -- 172.21.15.46:0/925515421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc07c0a1c50 con 0x55fe8acede90 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc098ee1b80 1 -- 172.21.15.46:0/925515421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc078011c30 con 0x55fe8acede90 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc06dffb700 1 -- 172.21.15.46:0/925515421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc07c0a1e10 con 0x55fe8acede90 2024-03-18T20:48:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc098ee1b80 10 client.19848 fetch_fsmap learned FSMap version 54 2024-03-18T20:48:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc098ee1b80 1 -- 172.21.15.46:0/925515421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55fe8aac6c70 con 0x55fe8acede90 2024-03-18T20:48:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc06dffb700 1 -- 172.21.15.46:0/925515421 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc078011c30 con 0x55fe8acede90 2024-03-18T20:48:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc06dffb700 1 -- 172.21.15.46:0/925515421 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 54) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc07c05fc30 con 0x55fe8acede90 2024-03-18T20:48:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc098ee1b80 10 client.19848 fetch_fsmap finished waiting for FSMap version 54 2024-03-18T20:48:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc098ee1b80 10 client.19848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.198+0000 7fc098ee1b80 20 client.19848 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.199+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fe8ad5d2b0 0x55fe8ad5f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.199+0000 7fc098ee1b80 4 client.19848 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:48:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.199+0000 7fc098ee1b80 1 -- 172.21.15.46:0/925515421 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fe8aa8fc90 con 0x55fe8ad5d2b0 2024-03-18T20:48:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.199+0000 7fc08659c700 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fe8ad5d2b0 0x55fe8ad5f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.199+0000 7fc08659c700 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fe8ad5d2b0 0x55fe8ad5f6a0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e717f1260d593ba server_cookie=41d325cf136324f4 in_seq=0 out_seq=0 2024-03-18T20:48:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.199+0000 7fc06dffb700 10 client.19848 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc06dffb700 1 -- 172.21.15.46:0/925515421 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fe8aa8fc90 con 0x55fe8ad5d2b0 2024-03-18T20:48:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc06dffb700 10 client.19848 handle_command_reply: tid=0 2024-03-18T20:48:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc098ee1b80 1 client.19848 shutdown 2024-03-18T20:48:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc098ee1b80 1 -- 172.21.15.46:0/925515421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8acede90 msgr2=0x55fe8acfdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8acede90 0x55fe8acfdc70 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc07c097fb0 tx=0x7fc07c095c80 comp rx=0 tx=0).stop 2024-03-18T20:48:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc098ee1b80 1 -- 172.21.15.46:0/925515421 shutdown_connections 2024-03-18T20:48:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fe8ad5d2b0 0x55fe8ad5f6a0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fe8acf8db0 0x55fe8ad5b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc06dffb700 0 client.19848 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe8aced570 0x55fe8acfd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc098ee1b80 1 --2- 172.21.15.46:0/925515421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe8acede90 0x55fe8acfdc70 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc098ee1b80 1 -- 172.21.15.46:0/925515421 >> 172.21.15.46:0/925515421 conn(0x55fe8ac641c0 msgr2=0x55fe8acf43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc098ee1b80 1 -- 172.21.15.46:0/925515421 shutdown_connections 2024-03-18T20:48:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc098ee1b80 1 -- 172.21.15.46:0/925515421 wait complete. 2024-03-18T20:48:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.224+0000 7fc098ee1b80 20 client.19848 trim_cache size 0 max 16384 2024-03-18T20:48:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:48:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:48:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:48:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:48:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.225+0000 7fc08cd4c700 1 -- 172.21.15.46:0/1167906275 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0700a1290 msgr2=0x7fc0700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.225+0000 7fc08cd4c700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0700a1290 0x7fc0700a3750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc07800bb90 tx=0x7fc078005d80 comp rx=0 tx=0).stop 2024-03-18T20:48:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.225+0000 7fc08cd4c700 1 -- 172.21.15.46:0/1167906275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc088100570 msgr2=0x7fc088101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.225+0000 7fc08cd4c700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc088100570 0x7fc088101d90 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00d950 tx=0x7fc07c00dc60 comp rx=0 tx=0).stop 2024-03-18T20:48:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.225+0000 7fc08cd4c700 1 -- 172.21.15.46:0/1167906275 shutdown_connections 2024-03-18T20:48:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.225+0000 7fc08cd4c700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc0700a1290 0x7fc0700a3750 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.225+0000 7fc08cd4c700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc088105c60 0x7fc0881022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.225+0000 7fc08cd4c700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0880ffbb0 0x7fc088101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.225+0000 7fc08cd4c700 1 --2- 172.21.15.46:0/1167906275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc088100570 0x7fc088101d90 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.225+0000 7fc08cd4c700 1 -- 172.21.15.46:0/1167906275 >> 172.21.15.46:0/1167906275 conn(0x7fc088076d50 msgr2=0x7fc0880683a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.225+0000 7fc08cd4c700 1 -- 172.21.15.46:0/1167906275 shutdown_connections 2024-03-18T20:48:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:29.225+0000 7fc08cd4c700 1 -- 172.21.15.46:0/1167906275 wait complete. 2024-03-18T20:48:29.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:29 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:29.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:29 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:29.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:29 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:29.723 INFO:tasks.workunit.client.0.smithi046.stdout: 1 463595 3.67 MB/sec execute 564 sec latency 4169.680 ms 2024-03-18T20:48:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:30 smithi046 ceph-mon[43495]: pgmap v387: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-18T20:48:30.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:30 smithi073 ceph-mon[47384]: pgmap v387: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-18T20:48:30.723 INFO:tasks.workunit.client.0.smithi046.stdout: 1 464479 3.67 MB/sec execute 565 sec latency 4776.029 ms 2024-03-18T20:48:30.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:30 smithi155 ceph-mon[48344]: pgmap v387: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-18T20:48:31.723 INFO:tasks.workunit.client.0.smithi046.stdout: 1 466000 3.69 MB/sec execute 566 sec latency 12.366 ms 2024-03-18T20:48:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:32 smithi046 ceph-mon[43495]: pgmap v388: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 814 KiB/s wr, 64 op/s 2024-03-18T20:48:32.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:32 smithi073 ceph-mon[47384]: pgmap v388: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 814 KiB/s wr, 64 op/s 2024-03-18T20:48:32.724 INFO:tasks.workunit.client.0.smithi046.stdout: 1 467138 3.69 MB/sec execute 567 sec latency 272.566 ms 2024-03-18T20:48:32.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:32 smithi155 ceph-mon[48344]: pgmap v388: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 814 KiB/s wr, 64 op/s 2024-03-18T20:48:33.723 INFO:tasks.workunit.client.0.smithi046.stdout: 1 467138 3.68 MB/sec execute 568 sec latency 1272.686 ms 2024-03-18T20:48:34.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:34 smithi073 ceph-mon[47384]: pgmap v389: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-03-18T20:48:34.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:34 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:34.723 INFO:tasks.workunit.client.0.smithi046.stdout: 1 467138 3.68 MB/sec execute 569 sec latency 2272.779 ms 2024-03-18T20:48:34.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:34 smithi155 ceph-mon[48344]: pgmap v389: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-03-18T20:48:34.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:34 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:34.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:34 smithi046 ceph-mon[43495]: pgmap v389: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-03-18T20:48:34.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:34 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:35.724 INFO:tasks.workunit.client.0.smithi046.stdout: 1 468023 3.67 MB/sec execute 570 sec latency 2877.807 ms 2024-03-18T20:48:36.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:36 smithi155 ceph-mon[48344]: pgmap v390: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 917 KiB/s wr, 103 op/s 2024-03-18T20:48:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:36 smithi046 ceph-mon[43495]: pgmap v390: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 917 KiB/s wr, 103 op/s 2024-03-18T20:48:36.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:48:36.357 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:48:36.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:36 smithi073 ceph-mon[47384]: pgmap v390: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 917 KiB/s wr, 103 op/s 2024-03-18T20:48:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.511+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/368284393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcf0100f80 msgr2=0x7fbcf0101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.511+0000 7fbcf4ea8700 1 --2- 172.21.15.46:0/368284393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcf0100f80 0x7fbcf0101360 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009990 tx=0x7fbce0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.511+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/368284393 shutdown_connections 2024-03-18T20:48:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.511+0000 7fbcf4ea8700 1 --2- 172.21.15.46:0/368284393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcf0100f80 0x7fbcf0101360 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.511+0000 7fbcf4ea8700 1 --2- 172.21.15.46:0/368284393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf01060a0 0x7fbcf010dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.511+0000 7fbcf4ea8700 1 --2- 172.21.15.46:0/368284393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf0101930 0x7fbcf0105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.511+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/368284393 >> 172.21.15.46:0/368284393 conn(0x7fbcf00fc7e0 msgr2=0x7fbcf00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.511+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/368284393 shutdown_connections 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.511+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/368284393 wait complete. 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbcf4ea8700 1 Processor -- start 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbcf4ea8700 1 -- start start 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbcf4ea8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf0100f80 0x7fbcf0197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbcf4ea8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcf01060a0 0x7fbcf01978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbcf4ea8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf019bc70 0x7fbcf019c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbcf4ea8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf019c6f0 con 0x7fbcf01060a0 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbcf4ea8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf019c860 con 0x7fbcf0100f80 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbcf4ea8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf019ca20 con 0x7fbcf019bc70 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbcedd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcf01060a0 0x7fbcf01978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbcee59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf0100f80 0x7fbcf0197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbcedd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcf01060a0 0x7fbcf01978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54620/0 (socket says 172.21.15.46:54620) 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbcedd9b700 1 -- 172.21.15.46:0/1977739235 learned_addr learned my addr 172.21.15.46:0/1977739235 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.512+0000 7fbceed9d700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf019bc70 0x7fbcf019c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.513+0000 7fbcee59c700 1 -- 172.21.15.46:0/1977739235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf019bc70 msgr2=0x7fbcf019c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.513+0000 7fbcee59c700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf019bc70 0x7fbcf019c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.513+0000 7fbcee59c700 1 -- 172.21.15.46:0/1977739235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcf01060a0 msgr2=0x7fbcf01978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.513+0000 7fbcee59c700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcf01060a0 0x7fbcf01978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.513+0000 7fbcee59c700 1 -- 172.21.15.46:0/1977739235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce00095f0 con 0x7fbcf0100f80 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.513+0000 7fbcedd9b700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcf01060a0 0x7fbcf01978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.513+0000 7fbcee59c700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf0100f80 0x7fbcf0197390 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbce00098e0 tx=0x7fbce0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.513+0000 7fbcdf7fe700 1 -- 172.21.15.46:0/1977739235 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce001d070 con 0x7fbcf0100f80 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.513+0000 7fbcdf7fe700 1 -- 172.21.15.46:0/1977739235 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbce0022470 con 0x7fbcf0100f80 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.513+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/1977739235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf01a3f20 con 0x7fbcf0100f80 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.513+0000 7fbcdf7fe700 1 -- 172.21.15.46:0/1977739235 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0017650 con 0x7fbcf0100f80 2024-03-18T20:48:36.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.513+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/1977739235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf01a4340 con 0x7fbcf0100f80 2024-03-18T20:48:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.516+0000 7fbcdf7fe700 1 -- 172.21.15.46:0/1977739235 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fbce00177b0 con 0x7fbcf0100f80 2024-03-18T20:48:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.516+0000 7fbcdf7fe700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbcd80a1310 0x7fbcd80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.516+0000 7fbcedd9b700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbcd80a1310 0x7fbcd80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.516+0000 7fbcdf7fe700 1 -- 172.21.15.46:0/1977739235 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fbce00cdef0 con 0x7fbcf0100f80 2024-03-18T20:48:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.517+0000 7fbcedd9b700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbcd80a1310 0x7fbcd80a37d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbcf01988e0 tx=0x7fbce4009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.517+0000 7fbd0103db80 1 Processor -- start 2024-03-18T20:48:36.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.517+0000 7fbd0103db80 1 -- start start 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbd0103db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 0x55bb8656be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbd0103db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656c380 0x55bb86576ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbd0103db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb86577220 0x55bb86579610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbd0103db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb864c5990 con 0x55bb8656ba60 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbd0103db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb864c3350 con 0x55bb86577220 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbd0103db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb864bec40 con 0x55bb8656c380 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbcedd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656c380 0x55bb86576ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbcee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 0x55bb8656be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbceed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb86577220 0x55bb86579610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbcedd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656c380 0x55bb86576ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:32964/0 (socket says 172.21.15.46:32964) 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbcedd9b700 1 -- 172.21.15.46:0/208813475 learned_addr learned my addr 172.21.15.46:0/208813475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbcedd9b700 1 -- 172.21.15.46:0/208813475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb86577220 msgr2=0x55bb86579610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:36.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbcedd9b700 1 --2- 172.21.15.46:0/208813475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb86577220 0x55bb86579610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbcedd9b700 1 -- 172.21.15.46:0/208813475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 msgr2=0x55bb8656be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:36.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbcedd9b700 1 --2- 172.21.15.46:0/208813475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 0x55bb8656be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbcedd9b700 1 -- 172.21.15.46:0/208813475 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb863403b0 con 0x55bb8656c380 2024-03-18T20:48:36.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbcee59c700 1 --2- 172.21.15.46:0/208813475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 0x55bb8656be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:36.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbceed9d700 1 --2- 172.21.15.46:0/208813475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb86577220 0x55bb86579610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:36.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.518+0000 7fbcedd9b700 1 --2- 172.21.15.46:0/208813475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656c380 0x55bb86576ce0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbce40043d0 tx=0x7fbce4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c284abd21e49fc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:36.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.519+0000 7fbcdcff9700 1 -- 172.21.15.46:0/208813475 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce4003ce0 con 0x55bb8656c380 2024-03-18T20:48:36.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.519+0000 7fbcdcff9700 1 -- 172.21.15.46:0/208813475 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbce4003e40 con 0x55bb8656c380 2024-03-18T20:48:36.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.519+0000 7fbcdcff9700 1 -- 172.21.15.46:0/208813475 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce400e790 con 0x55bb8656c380 2024-03-18T20:48:36.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.519+0000 7fbd0103db80 1 -- 172.21.15.46:0/208813475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656c380 msgr2=0x55bb86576ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:36.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.519+0000 7fbd0103db80 1 --2- 172.21.15.46:0/208813475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656c380 0x55bb86576ce0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbce40043d0 tx=0x7fbce4014000 comp rx=0 tx=0).stop 2024-03-18T20:48:36.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.519+0000 7fbd0103db80 1 -- 172.21.15.46:0/208813475 shutdown_connections 2024-03-18T20:48:36.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.519+0000 7fbd0103db80 1 --2- 172.21.15.46:0/208813475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 0x55bb8656be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.519+0000 7fbd0103db80 1 --2- 172.21.15.46:0/208813475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb86577220 0x55bb86579610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.519+0000 7fbd0103db80 1 --2- 172.21.15.46:0/208813475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656c380 0x55bb86576ce0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.519+0000 7fbd0103db80 1 -- 172.21.15.46:0/208813475 >> 172.21.15.46:0/208813475 conn(0x55bb864e2220 msgr2=0x55bb86568020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:36.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.520+0000 7fbd0103db80 1 -- 172.21.15.46:0/208813475 shutdown_connections 2024-03-18T20:48:36.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.520+0000 7fbd0103db80 1 -- 172.21.15.46:0/208813475 wait complete. 2024-03-18T20:48:36.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.520+0000 7fbd0103db80 1 -- start start 2024-03-18T20:48:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.520+0000 7fbd0103db80 1 Processor -- start 2024-03-18T20:48:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.520+0000 7fbd0103db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656ba60 0x55bb86576ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.520+0000 7fbd0103db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 0x55bb86573fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.520+0000 7fbd0103db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb86577220 0x55bb865746d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.520+0000 7fbd0103db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8645ad80 con 0x55bb86577220 2024-03-18T20:48:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.520+0000 7fbd0103db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8645aa20 con 0x55bb8656c380 2024-03-18T20:48:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.520+0000 7fbd0103db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb863426a0 con 0x55bb8656ba60 2024-03-18T20:48:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.520+0000 7fbd0103db80 10 client.0 _ll_register_callbacks cb 0x55bb862c63a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbcee59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656ba60 0x55bb86576ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbcedd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 0x55bb86573fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbceed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb86577220 0x55bb865746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbcedd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 0x55bb86573fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51138/0 (socket says 172.21.15.46:51138) 2024-03-18T20:48:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbcee59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656ba60 0x55bb86576ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:32968/0 (socket says 172.21.15.46:32968) 2024-03-18T20:48:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbcedd9b700 1 -- 172.21.15.46:0/1121861755 learned_addr learned my addr 172.21.15.46:0/1121861755 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbceed9d700 1 -- 172.21.15.46:0/1121861755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656ba60 msgr2=0x55bb86576ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbceed9d700 1 --2- 172.21.15.46:0/1121861755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656ba60 0x55bb86576ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbceed9d700 1 -- 172.21.15.46:0/1121861755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 msgr2=0x55bb86573fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbceed9d700 1 --2- 172.21.15.46:0/1121861755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 0x55bb86573fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbceed9d700 1 -- 172.21.15.46:0/1121861755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce8011230 con 0x55bb86577220 2024-03-18T20:48:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbcedd9b700 1 --2- 172.21.15.46:0/1121861755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 0x55bb86573fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbceed9d700 1 --2- 172.21.15.46:0/1121861755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb86577220 0x55bb865746d0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fbce8007cb0 tx=0x7fbce80113c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a34fec1d1af5706 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbcd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbcd5ffb700 1 -- 172.21.15.46:0/1121861755 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce800e040 con 0x55bb86577220 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbcd5ffb700 1 -- 172.21.15.46:0/1121861755 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fbce8004db0 con 0x55bb86577220 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbd0103db80 1 -- 172.21.15.46:0/1121861755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb8655fad0 con 0x55bb86577220 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbcd5ffb700 1 -- 172.21.15.46:0/1121861755 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbce8011bd0 con 0x55bb86577220 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbd0103db80 10 client.19860 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.521+0000 7fbd0103db80 1 -- 172.21.15.46:0/1121861755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7fbce4017070 con 0x55bb86577220 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.522+0000 7fbcd5ffb700 1 -- 172.21.15.46:0/1121861755 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb8655fad0 con 0x55bb86577220 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.522+0000 7fbcd5ffb700 1 -- 172.21.15.46:0/1121861755 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbce8016400 con 0x55bb86577220 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.522+0000 7fbd0103db80 10 client.19860 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.522+0000 7fbd0103db80 10 client.19860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.522+0000 7fbd0103db80 20 client.19860 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.522+0000 7fbd0103db80 1 --2- 172.21.15.46:0/1121861755 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bb86587010 0x55bb86589430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.522+0000 7fbd0103db80 4 client.19860 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.522+0000 7fbd0103db80 1 -- 172.21.15.46:0/1121861755 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb864b7700 con 0x55bb86587010 2024-03-18T20:48:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.522+0000 7fbcee59c700 1 --2- 172.21.15.46:0/1121861755 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bb86587010 0x55bb86589430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.522+0000 7fbcee59c700 1 --2- 172.21.15.46:0/1121861755 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bb86587010 0x55bb86589430 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5357052d1ecee217 server_cookie=3874b4940a2635f in_seq=0 out_seq=0 2024-03-18T20:48:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.522+0000 7fbcd5ffb700 10 client.19860 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.523+0000 7fbcd5ffb700 1 -- 172.21.15.46:0/1121861755 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bb864b7700 con 0x55bb86587010 2024-03-18T20:48:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.523+0000 7fbcd5ffb700 10 client.19860 handle_command_reply: tid=0 2024-03-18T20:48:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbd0103db80 1 client.19860 shutdown 2024-03-18T20:48:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbd0103db80 1 -- 172.21.15.46:0/1121861755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb86577220 msgr2=0x55bb865746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbd0103db80 1 --2- 172.21.15.46:0/1121861755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb86577220 0x55bb865746d0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fbce8007cb0 tx=0x7fbce80113c0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbd0103db80 1 -- 172.21.15.46:0/1121861755 shutdown_connections 2024-03-18T20:48:36.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbd0103db80 1 --2- 172.21.15.46:0/1121861755 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bb86587010 0x55bb86589430 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbd0103db80 1 --2- 172.21.15.46:0/1121861755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb86577220 0x55bb865746d0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbd0103db80 1 --2- 172.21.15.46:0/1121861755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656ba60 0x55bb86576ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbd0103db80 1 --2- 172.21.15.46:0/1121861755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 0x55bb86573fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbd0103db80 1 -- 172.21.15.46:0/1121861755 >> 172.21.15.46:0/1121861755 conn(0x55bb864e2220 msgr2=0x55bb86567c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:36.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbcd5ffb700 0 client.19860 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:36.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbd0103db80 1 -- 172.21.15.46:0/1121861755 shutdown_connections 2024-03-18T20:48:36.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbd0103db80 1 -- 172.21.15.46:0/1121861755 wait complete. 2024-03-18T20:48:36.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.525+0000 7fbd0103db80 20 client.19860 trim_cache size 0 max 16384 2024-03-18T20:48:36.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.545+0000 7fbd0103db80 1 Processor -- start 2024-03-18T20:48:36.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.545+0000 7fbd0103db80 1 -- start start 2024-03-18T20:48:36.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbd0103db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656ba60 0x55bb8657de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbd0103db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656c380 0x55bb865728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbd0103db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb86577220 0x55bb86572e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbd0103db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8656c090 con 0x55bb8656c380 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbd0103db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb86532940 con 0x55bb86577220 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbd0103db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb86562210 con 0x55bb8656ba60 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbcedd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656c380 0x55bb865728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbcedd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656c380 0x55bb865728d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54640/0 (socket says 172.21.15.46:54640) 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbcedd9b700 1 -- 172.21.15.46:0/2719972556 learned_addr learned my addr 172.21.15.46:0/2719972556 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbceed9d700 1 --2- 172.21.15.46:0/2719972556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb86577220 0x55bb86572e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbcee59c700 1 --2- 172.21.15.46:0/2719972556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656ba60 0x55bb8657de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbceed9d700 1 -- 172.21.15.46:0/2719972556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656ba60 msgr2=0x55bb8657de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbceed9d700 1 --2- 172.21.15.46:0/2719972556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656ba60 0x55bb8657de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbceed9d700 1 -- 172.21.15.46:0/2719972556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656c380 msgr2=0x55bb865728d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbceed9d700 1 --2- 172.21.15.46:0/2719972556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656c380 0x55bb865728d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbceed9d700 1 -- 172.21.15.46:0/2719972556 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb862ecb60 con 0x55bb86577220 2024-03-18T20:48:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.546+0000 7fbceed9d700 1 --2- 172.21.15.46:0/2719972556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb86577220 0x55bb86572e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbce800b010 tx=0x7fbce8007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6225834b9109433e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.547+0000 7fbcdd7fa700 1 -- 172.21.15.46:0/2719972556 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce800e040 con 0x55bb86577220 2024-03-18T20:48:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.547+0000 7fbcdd7fa700 1 -- 172.21.15.46:0/2719972556 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbce800f040 con 0x55bb86577220 2024-03-18T20:48:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.547+0000 7fbcdd7fa700 1 -- 172.21.15.46:0/2719972556 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8011e10 con 0x55bb86577220 2024-03-18T20:48:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 -- 172.21.15.46:0/2719972556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb86577220 msgr2=0x55bb86572e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 --2- 172.21.15.46:0/2719972556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb86577220 0x55bb86572e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbce800b010 tx=0x7fbce8007500 comp rx=0 tx=0).stop 2024-03-18T20:48:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 -- 172.21.15.46:0/2719972556 shutdown_connections 2024-03-18T20:48:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 --2- 172.21.15.46:0/2719972556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb86577220 0x55bb86572e10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 --2- 172.21.15.46:0/2719972556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb8656ba60 0x55bb8657de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 --2- 172.21.15.46:0/2719972556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656c380 0x55bb865728d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 -- 172.21.15.46:0/2719972556 >> 172.21.15.46:0/2719972556 conn(0x55bb864e2220 msgr2=0x55bb8656f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 -- 172.21.15.46:0/2719972556 shutdown_connections 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 -- 172.21.15.46:0/2719972556 wait complete. 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 -- start start 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 Processor -- start 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 0x55bb8657be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 0x55bb8657c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb86577220 0x55bb8657c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb865d9f50 con 0x55bb8656ba60 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb865da0c0 con 0x55bb8656c380 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.548+0000 7fbd0103db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb865da280 con 0x55bb86577220 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbd0103db80 10 client.0 _ll_register_callbacks cb 0x7fbce80078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 0x55bb8657be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcee59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 0x55bb8657be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54642/0 (socket says 172.21.15.46:54642) 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcedd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 0x55bb8657c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcee59c700 1 -- 172.21.15.46:0/4256258555 learned_addr learned my addr 172.21.15.46:0/4256258555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbceed9d700 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb86577220 0x55bb8657c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcedd9b700 1 -- 172.21.15.46:0/4256258555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb86577220 msgr2=0x55bb8657c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcedd9b700 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb86577220 0x55bb8657c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcedd9b700 1 -- 172.21.15.46:0/4256258555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 msgr2=0x55bb8657be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcedd9b700 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 0x55bb8657be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcedd9b700 1 -- 172.21.15.46:0/4256258555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce4017070 con 0x55bb8656c380 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcee59c700 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 0x55bb8657be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcedd9b700 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 0x55bb8657c3a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55bb86569250 tx=0x7fbce4009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fffce3406789648 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcd5ffb700 1 -- 172.21.15.46:0/4256258555 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce402c040 con 0x55bb8656c380 2024-03-18T20:48:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbcd5ffb700 1 -- 172.21.15.46:0/4256258555 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fbce40048f0 con 0x55bb8656c380 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.549+0000 7fbd0103db80 1 -- 172.21.15.46:0/4256258555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbce4003e40 con 0x55bb8656c380 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbcd5ffb700 1 -- 172.21.15.46:0/4256258555 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbce4004ab0 con 0x55bb8656c380 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbd0103db80 10 client.29474 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbd0103db80 1 -- 172.21.15.46:0/4256258555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55bb86484ea0 con 0x55bb8656c380 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbcd5ffb700 1 -- 172.21.15.46:0/4256258555 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbce403e080 con 0x55bb8656c380 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbcd5ffb700 1 -- 172.21.15.46:0/4256258555 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbce4024020 con 0x55bb8656c380 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbd0103db80 10 client.29474 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbd0103db80 10 client.29474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbd0103db80 20 client.29474 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbd0103db80 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bb865db9a0 0x55bb865dddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbd0103db80 4 client.29474 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbd0103db80 1 -- 172.21.15.46:0/4256258555 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bb8630dcf0 con 0x55bb865db9a0 2024-03-18T20:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbcee59c700 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bb865db9a0 0x55bb865dddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbcee59c700 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bb865db9a0 0x55bb865dddc0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72f7613a5244c895 server_cookie=8793bf57cba3fcf in_seq=0 out_seq=0 2024-03-18T20:48:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.550+0000 7fbcd5ffb700 10 client.29474 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbcd5ffb700 1 -- 172.21.15.46:0/4256258555 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bb8630dcf0 con 0x55bb865db9a0 2024-03-18T20:48:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbcd5ffb700 10 client.29474 handle_command_reply: tid=0 2024-03-18T20:48:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbd0103db80 1 client.29474 shutdown 2024-03-18T20:48:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbd0103db80 1 -- 172.21.15.46:0/4256258555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 msgr2=0x55bb8657c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbd0103db80 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 0x55bb8657c3a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55bb86569250 tx=0x7fbce4009fb0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbd0103db80 1 -- 172.21.15.46:0/4256258555 shutdown_connections 2024-03-18T20:48:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbd0103db80 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bb865db9a0 0x55bb865dddc0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbd0103db80 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb8656ba60 0x55bb8657be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbd0103db80 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bb86577220 0x55bb8657c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbd0103db80 1 --2- 172.21.15.46:0/4256258555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb8656c380 0x55bb8657c3a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbd0103db80 1 -- 172.21.15.46:0/4256258555 >> 172.21.15.46:0/4256258555 conn(0x55bb864e2220 msgr2=0x55bb86572890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbcd5ffb700 0 client.29474 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbd0103db80 1 -- 172.21.15.46:0/4256258555 shutdown_connections 2024-03-18T20:48:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbd0103db80 1 -- 172.21.15.46:0/4256258555 wait complete. 2024-03-18T20:48:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.551+0000 7fbd0103db80 20 client.29474 trim_cache size 0 max 16384 2024-03-18T20:48:36.560 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:48:36.560 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:48:36.561 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:48:36.561 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:48:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.552+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/1977739235 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbcd80a1310 msgr2=0x7fbcd80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.552+0000 7fbcf4ea8700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbcd80a1310 0x7fbcd80a37d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbcf01988e0 tx=0x7fbce4009260 comp rx=0 tx=0).stop 2024-03-18T20:48:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.552+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/1977739235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf0100f80 msgr2=0x7fbcf0197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.552+0000 7fbcf4ea8700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf0100f80 0x7fbcf0197390 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbce00098e0 tx=0x7fbce0005740 comp rx=0 tx=0).stop 2024-03-18T20:48:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.552+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/1977739235 shutdown_connections 2024-03-18T20:48:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.552+0000 7fbcf4ea8700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbcd80a1310 0x7fbcd80a37d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.552+0000 7fbcf4ea8700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcf019bc70 0x7fbcf019c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.552+0000 7fbcf4ea8700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcf0100f80 0x7fbcf0197390 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.552+0000 7fbcf4ea8700 1 --2- 172.21.15.46:0/1977739235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcf01060a0 0x7fbcf01978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.552+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/1977739235 >> 172.21.15.46:0/1977739235 conn(0x7fbcf00fc7e0 msgr2=0x7fbcf00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:36.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.552+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/1977739235 shutdown_connections 2024-03-18T20:48:36.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.552+0000 7fbcf4ea8700 1 -- 172.21.15.46:0/1977739235 wait complete. 2024-03-18T20:48:36.568 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:48:36.568 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:48:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.710+0000 7fda3c3c0700 1 -- 172.21.15.46:0/999552412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda340ffa00 msgr2=0x7fda340ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.710+0000 7fda3c3c0700 1 --2- 172.21.15.46:0/999552412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda340ffa00 0x7fda340ffde0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fda30009990 tx=0x7fda30009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.711+0000 7fda3c3c0700 1 -- 172.21.15.46:0/999552412 shutdown_connections 2024-03-18T20:48:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.711+0000 7fda3c3c0700 1 --2- 172.21.15.46:0/999552412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda340ffa00 0x7fda340ffde0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.711+0000 7fda3c3c0700 1 --2- 172.21.15.46:0/999552412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda3410d5b0 0x7fda3410f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.711+0000 7fda3c3c0700 1 --2- 172.21.15.46:0/999552412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda341003b0 0x7fda3410cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.711+0000 7fda3c3c0700 1 -- 172.21.15.46:0/999552412 >> 172.21.15.46:0/999552412 conn(0x7fda340fb5f0 msgr2=0x7fda340fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.711+0000 7fda3c3c0700 1 -- 172.21.15.46:0/999552412 shutdown_connections 2024-03-18T20:48:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.711+0000 7fda3c3c0700 1 -- 172.21.15.46:0/999552412 wait complete. 2024-03-18T20:48:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.711+0000 7fda3c3c0700 1 Processor -- start 2024-03-18T20:48:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.711+0000 7fda3c3c0700 1 -- start start 2024-03-18T20:48:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3c3c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda340ffa00 0x7fda341993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3c3c0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda341003b0 0x7fda341998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3c3c0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda3410d5b0 0x7fda3419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3c3c0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419e280 con 0x7fda340ffa00 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3c3c0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419e3f0 con 0x7fda341003b0 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3c3c0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419e5b0 con 0x7fda3410d5b0 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3a15c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda340ffa00 0x7fda341993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3a95d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda3410d5b0 0x7fda3419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3a15c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda340ffa00 0x7fda341993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54662/0 (socket says 172.21.15.46:54662) 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3a15c700 1 -- 172.21.15.46:0/1688889950 learned_addr learned my addr 172.21.15.46:0/1688889950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3995b700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda341003b0 0x7fda341998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3a15c700 1 -- 172.21.15.46:0/1688889950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda3410d5b0 msgr2=0x7fda3419dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3a15c700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda3410d5b0 0x7fda3419dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3a15c700 1 -- 172.21.15.46:0/1688889950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda341003b0 msgr2=0x7fda341998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3a15c700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda341003b0 0x7fda341998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3a15c700 1 -- 172.21.15.46:0/1688889950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda24009520 con 0x7fda340ffa00 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.712+0000 7fda3a15c700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda340ffa00 0x7fda341993b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fda30005910 tx=0x7fda30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.713+0000 7fda2b7fe700 1 -- 172.21.15.46:0/1688889950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3001d070 con 0x7fda340ffa00 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.713+0000 7fda2b7fe700 1 -- 172.21.15.46:0/1688889950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda30022470 con 0x7fda340ffa00 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.713+0000 7fda3c3c0700 1 -- 172.21.15.46:0/1688889950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda300095f0 con 0x7fda340ffa00 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.713+0000 7fda2b7fe700 1 -- 172.21.15.46:0/1688889950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30017650 con 0x7fda340ffa00 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.713+0000 7fda3c3c0700 1 -- 172.21.15.46:0/1688889950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3419eac0 con 0x7fda340ffa00 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.714+0000 7fda3c3c0700 1 -- 172.21.15.46:0/1688889950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda34108740 con 0x7fda340ffa00 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.715+0000 7fda2b7fe700 1 -- 172.21.15.46:0/1688889950 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fda300225e0 con 0x7fda340ffa00 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.716+0000 7fda2b7fe700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fda200a1290 0x7fda200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.716+0000 7fda3995b700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fda200a1290 0x7fda200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.716+0000 7fda2b7fe700 1 -- 172.21.15.46:0/1688889950 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fda300cdf30 con 0x7fda340ffa00 2024-03-18T20:48:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.717+0000 7fda3995b700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fda200a1290 0x7fda200a3750 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fda3419a8b0 tx=0x7fda24009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:36.723 INFO:tasks.workunit.client.0.smithi046.stdout: 1 469468 3.69 MB/sec execute 571 sec latency 15.100 ms 2024-03-18T20:48:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:36.721+0000 7fda2b7fe700 1 -- 172.21.15.46:0/1688889950 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fda30097510 con 0x7fda340ffa00 2024-03-18T20:48:37.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.010+0000 7fda3c3c0700 1 -- 172.21.15.46:0/1688889950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda340698f0 con 0x7fda340ffa00 2024-03-18T20:48:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.011+0000 7fda2b7fe700 1 -- 172.21.15.46:0/1688889950 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+10918 (secure 0 0 0) 0x7fda3009ab30 con 0x7fda340ffa00 2024-03-18T20:48:37.014 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:48:37.014 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:48:33.291724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":5,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:48:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.014+0000 7fda3c3c0700 1 -- 172.21.15.46:0/1688889950 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fda200a1290 msgr2=0x7fda200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.014+0000 7fda3c3c0700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fda200a1290 0x7fda200a3750 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fda3419a8b0 tx=0x7fda24009260 comp rx=0 tx=0).stop 2024-03-18T20:48:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.014+0000 7fda3c3c0700 1 -- 172.21.15.46:0/1688889950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda340ffa00 msgr2=0x7fda341993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.014+0000 7fda3c3c0700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda340ffa00 0x7fda341993b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fda30005910 tx=0x7fda30004730 comp rx=0 tx=0).stop 2024-03-18T20:48:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.014+0000 7fda3c3c0700 1 -- 172.21.15.46:0/1688889950 shutdown_connections 2024-03-18T20:48:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.014+0000 7fda3c3c0700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fda200a1290 0x7fda200a3750 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.014+0000 7fda3c3c0700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda340ffa00 0x7fda341993b0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.014+0000 7fda3c3c0700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda3410d5b0 0x7fda3419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.014+0000 7fda3c3c0700 1 --2- 172.21.15.46:0/1688889950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda341003b0 0x7fda341998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.014+0000 7fda3c3c0700 1 -- 172.21.15.46:0/1688889950 >> 172.21.15.46:0/1688889950 conn(0x7fda340fb5f0 msgr2=0x7fda340fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.014+0000 7fda3c3c0700 1 -- 172.21.15.46:0/1688889950 shutdown_connections 2024-03-18T20:48:37.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.014+0000 7fda3c3c0700 1 -- 172.21.15.46:0/1688889950 wait complete. 2024-03-18T20:48:37.018 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 55 2024-03-18T20:48:37.035 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:48:37.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:48:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.184+0000 7fa9e162f700 1 -- 172.21.15.46:0/3079060734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060a0 msgr2=0x7fa9dc10dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.184+0000 7fa9e162f700 1 --2- 172.21.15.46:0/3079060734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060a0 0x7fa9dc10dc20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400b0d0 tx=0x7fa9d400b3e0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.184+0000 7fa9e162f700 1 -- 172.21.15.46:0/3079060734 shutdown_connections 2024-03-18T20:48:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.184+0000 7fa9e162f700 1 --2- 172.21.15.46:0/3079060734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060a0 0x7fa9dc10dc20 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.184+0000 7fa9e162f700 1 --2- 172.21.15.46:0/3079060734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069540 0x7fa9dc069920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.184+0000 7fa9e162f700 1 --2- 172.21.15.46:0/3079060734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc069ef0 0x7fa9dc105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.184+0000 7fa9e162f700 1 -- 172.21.15.46:0/3079060734 >> 172.21.15.46:0/3079060734 conn(0x7fa9dc076d50 msgr2=0x7fa9dc077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.184+0000 7fa9e162f700 1 -- 172.21.15.46:0/3079060734 shutdown_connections 2024-03-18T20:48:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.184+0000 7fa9e162f700 1 -- 172.21.15.46:0/3079060734 wait complete. 2024-03-18T20:48:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.185+0000 7fa9e162f700 1 Processor -- start 2024-03-18T20:48:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.185+0000 7fa9e162f700 1 -- start start 2024-03-18T20:48:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.185+0000 7fa9e162f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069540 0x7fa9dc1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.185+0000 7fa9e162f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060a0 0x7fa9dc197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.185+0000 7fa9e162f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc19bba0 0x7fa9dc19c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.185+0000 7fa9e162f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc19c620 con 0x7fa9dc1060a0 2024-03-18T20:48:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.185+0000 7fa9e162f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc19c790 con 0x7fa9dc069540 2024-03-18T20:48:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.185+0000 7fa9e162f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9dc19c950 con 0x7fa9dc19bba0 2024-03-18T20:48:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060a0 0x7fa9dc197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069540 0x7fa9dc1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060a0 0x7fa9dc197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54682/0 (socket says 172.21.15.46:54682) 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9da7fc700 1 -- 172.21.15.46:0/845402013 learned_addr learned my addr 172.21.15.46:0/845402013 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc19bba0 0x7fa9dc19c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9daffd700 1 -- 172.21.15.46:0/845402013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc19bba0 msgr2=0x7fa9dc19c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9daffd700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc19bba0 0x7fa9dc19c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9daffd700 1 -- 172.21.15.46:0/845402013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060a0 msgr2=0x7fa9dc197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9daffd700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060a0 0x7fa9dc197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9daffd700 1 -- 172.21.15.46:0/845402013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc009520 con 0x7fa9dc069540 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9daffd700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069540 0x7fa9dc1972c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00e9f0 tx=0x7fa9cc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9c3fff700 1 -- 172.21.15.46:0/845402013 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00cb70 con 0x7fa9dc069540 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9e162f700 1 -- 172.21.15.46:0/845402013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d4009d00 con 0x7fa9dc069540 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9c3fff700 1 -- 172.21.15.46:0/845402013 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9cc00ee90 con 0x7fa9dc069540 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9c3fff700 1 -- 172.21.15.46:0/845402013 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc003cf0 con 0x7fa9dc069540 2024-03-18T20:48:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.186+0000 7fa9e162f700 1 -- 172.21.15.46:0/845402013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9dc1a41f0 con 0x7fa9dc069540 2024-03-18T20:48:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.189+0000 7fa9c3fff700 1 -- 172.21.15.46:0/845402013 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa9cc00ccd0 con 0x7fa9dc069540 2024-03-18T20:48:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.190+0000 7fa9c3fff700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9c40a1290 0x7fa9c40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.190+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9c40a1290 0x7fa9c40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.190+0000 7fa9c3fff700 1 -- 172.21.15.46:0/845402013 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa9cc014070 con 0x7fa9dc069540 2024-03-18T20:48:37.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.190+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9c40a1290 0x7fa9c40a3750 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc198810 tx=0x7fa9d0009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9ed7c4b80 1 Processor -- start 2024-03-18T20:48:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9ed7c4b80 1 -- start start 2024-03-18T20:48:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9ed7c4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 0x5595dce67e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9ed7c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 0x5595dce72ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9ed7c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce75610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9ed7c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595dcdc1990 con 0x5595dce68380 2024-03-18T20:48:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9ed7c4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595dcdbf350 con 0x5595dce67a60 2024-03-18T20:48:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9ed7c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595dcdbac40 con 0x5595dce73220 2024-03-18T20:48:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 0x5595dce72ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 0x5595dce67e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9db7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce75610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 0x5595dce67e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51198/0 (socket says 172.21.15.46:51198) 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 0x5595dce72ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54684/0 (socket says 172.21.15.46:54684) 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.191+0000 7fa9daffd700 1 -- 172.21.15.46:0/3409613768 learned_addr learned my addr 172.21.15.46:0/3409613768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.192+0000 7fa9daffd700 1 -- 172.21.15.46:0/3409613768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 msgr2=0x5595dce75610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.192+0000 7fa9daffd700 1 --2- 172.21.15.46:0/3409613768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce75610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.192+0000 7fa9daffd700 1 -- 172.21.15.46:0/3409613768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 msgr2=0x5595dce72ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.192+0000 7fa9daffd700 1 --2- 172.21.15.46:0/3409613768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 0x5595dce72ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.192+0000 7fa9daffd700 1 -- 172.21.15.46:0/3409613768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595dcc3c3b0 con 0x5595dce67a60 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.192+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/3409613768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 0x5595dce72ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.192+0000 7fa9daffd700 1 --2- 172.21.15.46:0/3409613768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 0x5595dce67e40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc096ad0 tx=0x7fa9cc099a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=176630b90e791a21 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.192+0000 7fa9c17fa700 1 -- 172.21.15.46:0/3409613768 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc09de90 con 0x5595dce67a60 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.192+0000 7fa9c17fa700 1 -- 172.21.15.46:0/3409613768 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9cc0a24e0 con 0x5595dce67a60 2024-03-18T20:48:37.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.192+0000 7fa9c17fa700 1 -- 172.21.15.46:0/3409613768 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc099550 con 0x5595dce67a60 2024-03-18T20:48:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.193+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/3409613768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 msgr2=0x5595dce67e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.193+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/3409613768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 0x5595dce67e40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc096ad0 tx=0x7fa9cc099a40 comp rx=0 tx=0).stop 2024-03-18T20:48:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.193+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/3409613768 shutdown_connections 2024-03-18T20:48:37.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.193+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/3409613768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce75610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.193+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/3409613768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 0x5595dce67e40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.193+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/3409613768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 0x5595dce72ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.193+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/3409613768 >> 172.21.15.46:0/3409613768 conn(0x5595dcdde220 msgr2=0x5595dce64020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.193+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/3409613768 shutdown_connections 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.193+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/3409613768 wait complete. 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.193+0000 7fa9ed7c4b80 1 -- start start 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.193+0000 7fa9ed7c4b80 1 Processor -- start 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9ed7c4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 0x5595dce70110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9ed7c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 0x5595dce70650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9ed7c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce70b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9ed7c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595dcd56d80 con 0x5595dce68380 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9ed7c4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595dcd56a20 con 0x5595dce67a60 2024-03-18T20:48:37.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9ed7c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595dcc3e6a0 con 0x5595dce73220 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9ed7c4b80 10 client.0 _ll_register_callbacks cb 0x5595dcbc23a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 0x5595dce70650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 0x5595dce70110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 0x5595dce70650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54700/0 (socket says 172.21.15.46:54700) 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9da7fc700 1 -- 172.21.15.46:0/916223934 learned_addr learned my addr 172.21.15.46:0/916223934 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce70b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9da7fc700 1 -- 172.21.15.46:0/916223934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 msgr2=0x5595dce70b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce70b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9da7fc700 1 -- 172.21.15.46:0/916223934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 msgr2=0x5595dce70110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 0x5595dce70110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9da7fc700 1 -- 172.21.15.46:0/916223934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d0017070 con 0x5595dce68380 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9daffd700 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 0x5595dce70110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce70b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 0x5595dce70650 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0003900 tx=0x7fa9d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f233490a41d53f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.194+0000 7fa9beff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9beff5700 1 -- 172.21.15.46:0/916223934 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d001e800 con 0x5595dce68380 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9beff5700 1 -- 172.21.15.46:0/916223934 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa9d001e960 con 0x5595dce68380 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/916223934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595dce5bad0 con 0x5595dce68380 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9beff5700 1 -- 172.21.15.46:0/916223934 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9d001eb20 con 0x5595dce68380 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9ed7c4b80 10 client.19884 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/916223934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7fa9dc1a41f0 con 0x5595dce68380 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9beff5700 1 -- 172.21.15.46:0/916223934 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5595dce5bad0 con 0x5595dce68380 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9beff5700 1 -- 172.21.15.46:0/916223934 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa9d000e3b0 con 0x5595dce68380 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9ed7c4b80 10 client.19884 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9ed7c4b80 10 client.19884 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9ed7c4b80 20 client.19884 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5595dce82f00 0x5595dce85320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9ed7c4b80 4 client.19884 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:37.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.195+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/916223934 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595dcdb3700 con 0x5595dce82f00 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.196+0000 7fa9daffd700 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5595dce82f00 0x5595dce85320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.196+0000 7fa9daffd700 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5595dce82f00 0x5595dce85320 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=40cca35b72c0125e server_cookie=a66657f0f898768a in_seq=0 out_seq=0 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.196+0000 7fa9beff5700 10 client.19884 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.196+0000 7fa9beff5700 1 -- 172.21.15.46:0/916223934 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5595dcdb3700 con 0x5595dce82f00 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.196+0000 7fa9beff5700 10 client.19884 handle_command_reply: tid=0 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.196+0000 7fa9ed7c4b80 1 client.19884 shutdown 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.197+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/916223934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 msgr2=0x5595dce70650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.197+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 0x5595dce70650 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0003900 tx=0x7fa9d00047c0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.197+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/916223934 shutdown_connections 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.197+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5595dce82f00 0x5595dce85320 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.197+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce70b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.197+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce67a60 0x5595dce70110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.197+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/916223934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce68380 0x5595dce70650 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.197+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/916223934 >> 172.21.15.46:0/916223934 conn(0x5595dcdde220 msgr2=0x5595dce63c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.197+0000 7fa9beff5700 0 client.19884 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:48:37.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.197+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/916223934 shutdown_connections 2024-03-18T20:48:37.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.197+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/916223934 wait complete. 2024-03-18T20:48:37.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.197+0000 7fa9ed7c4b80 20 client.19884 trim_cache size 0 max 16384 2024-03-18T20:48:37.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.211+0000 7fa9ed7c4b80 1 Processor -- start 2024-03-18T20:48:37.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.211+0000 7fa9ed7c4b80 1 -- start start 2024-03-18T20:48:37.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9ed7c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 0x5595dce79e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9ed7c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce68380 0x5595dce709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9ed7c4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce73220 0x5595dce70f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9ed7c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595dce7a3c0 con 0x5595dce67a60 2024-03-18T20:48:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9ed7c4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595dce5b610 con 0x5595dce73220 2024-03-18T20:48:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9ed7c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595dce72b90 con 0x5595dce68380 2024-03-18T20:48:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 0x5595dce79e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce68380 0x5595dce709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9db7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce73220 0x5595dce70f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 0x5595dce79e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54714/0 (socket says 172.21.15.46:54714) 2024-03-18T20:48:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9da7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce68380 0x5595dce709d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33038/0 (socket says 172.21.15.46:33038) 2024-03-18T20:48:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9daffd700 1 -- 172.21.15.46:0/2356359811 learned_addr learned my addr 172.21.15.46:0/2356359811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9db7fe700 1 -- 172.21.15.46:0/2356359811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce68380 msgr2=0x5595dce709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/2356359811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce68380 0x5595dce709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9db7fe700 1 -- 172.21.15.46:0/2356359811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 msgr2=0x5595dce79e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/2356359811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 0x5595dce79e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9db7fe700 1 -- 172.21.15.46:0/2356359811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595dcbe8b60 con 0x5595dce73220 2024-03-18T20:48:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9daffd700 1 --2- 172.21.15.46:0/2356359811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 0x5595dce79e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/2356359811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce68380 0x5595dce709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.212+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/2356359811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce73220 0x5595dce70f10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009fd0 tx=0x7fa9d4008b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f8f245ddefe05ca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.213+0000 7fa9c1ffb700 1 -- 172.21.15.46:0/2356359811 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4021070 con 0x5595dce73220 2024-03-18T20:48:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.213+0000 7fa9c1ffb700 1 -- 172.21.15.46:0/2356359811 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9d4008c00 con 0x5595dce73220 2024-03-18T20:48:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.213+0000 7fa9c1ffb700 1 -- 172.21.15.46:0/2356359811 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d400e040 con 0x5595dce73220 2024-03-18T20:48:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.213+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/2356359811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce73220 msgr2=0x5595dce70f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.213+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/2356359811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce73220 0x5595dce70f10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009fd0 tx=0x7fa9d4008b10 comp rx=0 tx=0).stop 2024-03-18T20:48:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.213+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/2356359811 shutdown_connections 2024-03-18T20:48:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.213+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/2356359811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 0x5595dce79e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/2356359811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce73220 0x5595dce70f10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/2356359811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce68380 0x5595dce709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/2356359811 >> 172.21.15.46:0/2356359811 conn(0x5595dcdde220 msgr2=0x5595dce7e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/2356359811 shutdown_connections 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/2356359811 wait complete. 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 -- start start 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 Processor -- start 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 0x5595dcecf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce68380 0x5595dcecfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce79160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595dce79870 con 0x5595dce67a60 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595dce799e0 con 0x5595dce68380 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595dce79ba0 con 0x5595dce73220 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9ed7c4b80 10 client.0 _ll_register_callbacks cb 0x7fa9d0004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 0x5595dcecf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9daffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 0x5595dcecf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54724/0 (socket says 172.21.15.46:54724) 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9daffd700 1 -- 172.21.15.46:0/1855431817 learned_addr learned my addr 172.21.15.46:0/1855431817 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce68380 0x5595dcecfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.214+0000 7fa9db7fe700 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce79160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9daffd700 1 -- 172.21.15.46:0/1855431817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 msgr2=0x5595dce79160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9daffd700 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce79160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9daffd700 1 -- 172.21.15.46:0/1855431817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce68380 msgr2=0x5595dcecfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9daffd700 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce68380 0x5595dcecfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9daffd700 1 -- 172.21.15.46:0/1855431817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc0b6040 con 0x5595dce67a60 2024-03-18T20:48:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce68380 0x5595dcecfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9daffd700 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 0x5595dcecf670 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc096ad0 tx=0x7fa9cc099ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d86e84931e045f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9beff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9beff5700 1 -- 172.21.15.46:0/1855431817 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc098400 con 0x5595dce67a60 2024-03-18T20:48:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9beff5700 1 -- 172.21.15.46:0/1855431817 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa9cc01fde0 con 0x5595dce67a60 2024-03-18T20:48:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/1855431817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9cc0a24e0 con 0x5595dce67a60 2024-03-18T20:48:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9beff5700 1 -- 172.21.15.46:0/1855431817 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9cc0b82b0 con 0x5595dce67a60 2024-03-18T20:48:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9ed7c4b80 10 client.19890 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/1855431817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7fa9dc1a41f0 con 0x5595dce67a60 2024-03-18T20:48:37.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.215+0000 7fa9beff5700 1 -- 172.21.15.46:0/1855431817 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9cc0a6020 con 0x5595dce67a60 2024-03-18T20:48:37.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.216+0000 7fa9beff5700 1 -- 172.21.15.46:0/1855431817 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa9cc09a2c0 con 0x5595dce67a60 2024-03-18T20:48:37.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.216+0000 7fa9ed7c4b80 10 client.19890 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:37.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.216+0000 7fa9ed7c4b80 10 client.19890 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:48:37.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.216+0000 7fa9ed7c4b80 20 client.19890 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:37.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.216+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5595dced71d0 0x5595dced95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.216+0000 7fa9ed7c4b80 4 client.19890 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:48:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.216+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/1855431817 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5595dcc09cf0 con 0x5595dced71d0 2024-03-18T20:48:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.216+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5595dced71d0 0x5595dced95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.216+0000 7fa9beff5700 10 client.19890 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:48:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.216+0000 7fa9da7fc700 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5595dced71d0 0x5595dced95c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3fc92186dc4c8ac0 server_cookie=db7b3aa780a7e82 in_seq=0 out_seq=0 2024-03-18T20:48:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.216+0000 7fa9beff5700 1 -- 172.21.15.46:0/1855431817 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5595dcc09cf0 con 0x5595dced71d0 2024-03-18T20:48:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9beff5700 10 client.19890 handle_command_reply: tid=0 2024-03-18T20:48:37.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9ed7c4b80 1 client.19890 shutdown 2024-03-18T20:48:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/1855431817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 msgr2=0x5595dcecf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 0x5595dcecf670 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc096ad0 tx=0x7fa9cc099ec0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/1855431817 shutdown_connections 2024-03-18T20:48:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x5595dced71d0 0x5595dced95c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595dce67a60 0x5595dcecf670 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595dce73220 0x5595dce79160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9ed7c4b80 1 --2- 172.21.15.46:0/1855431817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595dce68380 0x5595dcecfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/1855431817 >> 172.21.15.46:0/1855431817 conn(0x5595dcdde220 msgr2=0x5595dce6b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9beff5700 0 client.19890 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:48:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/1855431817 shutdown_connections 2024-03-18T20:48:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9ed7c4b80 1 -- 172.21.15.46:0/1855431817 wait complete. 2024-03-18T20:48:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.217+0000 7fa9ed7c4b80 20 client.19890 trim_cache size 0 max 16384 2024-03-18T20:48:37.232 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:48:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.218+0000 7fa9e162f700 1 -- 172.21.15.46:0/845402013 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9c40a1290 msgr2=0x7fa9c40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.218+0000 7fa9e162f700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9c40a1290 0x7fa9c40a3750 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc198810 tx=0x7fa9d0009260 comp rx=0 tx=0).stop 2024-03-18T20:48:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.218+0000 7fa9e162f700 1 -- 172.21.15.46:0/845402013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069540 msgr2=0x7fa9dc1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.218+0000 7fa9e162f700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069540 0x7fa9dc1972c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00e9f0 tx=0x7fa9cc00ed00 comp rx=0 tx=0).stop 2024-03-18T20:48:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.218+0000 7fa9e162f700 1 -- 172.21.15.46:0/845402013 shutdown_connections 2024-03-18T20:48:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.218+0000 7fa9e162f700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa9c40a1290 0x7fa9c40a3750 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.218+0000 7fa9e162f700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9dc19bba0 0x7fa9dc19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.218+0000 7fa9e162f700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9dc069540 0x7fa9dc1972c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.218+0000 7fa9e162f700 1 --2- 172.21.15.46:0/845402013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9dc1060a0 0x7fa9dc197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.218+0000 7fa9e162f700 1 -- 172.21.15.46:0/845402013 >> 172.21.15.46:0/845402013 conn(0x7fa9dc076d50 msgr2=0x7fa9dc0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.218+0000 7fa9e162f700 1 -- 172.21.15.46:0/845402013 shutdown_connections 2024-03-18T20:48:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.218+0000 7fa9e162f700 1 -- 172.21.15.46:0/845402013 wait complete. 2024-03-18T20:48:37.234 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:48:37.235 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:48:37.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:37 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1688889950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:48:37.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:37 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1688889950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:48:37.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.398+0000 7f9267535700 1 -- 172.21.15.46:0/3004817088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9260068810 msgr2=0x7f926010db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.398+0000 7f9267535700 1 --2- 172.21.15.46:0/3004817088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9260068810 0x7f926010db00 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f925800b0a0 tx=0x7f925800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.398+0000 7f9267535700 1 -- 172.21.15.46:0/3004817088 shutdown_connections 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.398+0000 7f9267535700 1 --2- 172.21.15.46:0/3004817088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9260068810 0x7f926010db00 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.398+0000 7f9267535700 1 --2- 172.21.15.46:0/3004817088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260106e60 0x7f9260107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.398+0000 7f9267535700 1 --2- 172.21.15.46:0/3004817088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9260067e50 0x7f92600682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.398+0000 7f9267535700 1 -- 172.21.15.46:0/3004817088 >> 172.21.15.46:0/3004817088 conn(0x7f9260076d40 msgr2=0x7f9260077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.398+0000 7f9267535700 1 -- 172.21.15.46:0/3004817088 shutdown_connections 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.398+0000 7f9267535700 1 -- 172.21.15.46:0/3004817088 wait complete. 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.399+0000 7f9267535700 1 Processor -- start 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.399+0000 7f9267535700 1 -- start start 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.399+0000 7f9267535700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9260067e50 0x7f9260197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.399+0000 7f9267535700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260068810 0x7f92601977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.399+0000 7f9267535700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9260106e60 0x7f926019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.399+0000 7f9267535700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926019c160 con 0x7f9260067e50 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.399+0000 7f9267535700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926019c2d0 con 0x7f9260106e60 2024-03-18T20:48:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.399+0000 7f9267535700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926019c490 con 0x7f9260068810 2024-03-18T20:48:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.399+0000 7f9264ad0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260068810 0x7f92601977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f9264ad0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260068810 0x7f92601977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33056/0 (socket says 172.21.15.46:33056) 2024-03-18T20:48:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f9264ad0700 1 -- 172.21.15.46:0/867950338 learned_addr learned my addr 172.21.15.46:0/867950338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f9265ad2700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9260106e60 0x7f926019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f9264ad0700 1 -- 172.21.15.46:0/867950338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9260106e60 msgr2=0x7f926019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f9264ad0700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9260106e60 0x7f926019bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f9264ad0700 1 -- 172.21.15.46:0/867950338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9260067e50 msgr2=0x7f9260197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f9264ad0700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9260067e50 0x7f9260197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f9264ad0700 1 -- 172.21.15.46:0/867950338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9250009520 con 0x7f9260068810 2024-03-18T20:48:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f9264ad0700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260068810 0x7f92601977d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f925000eb30 tx=0x7f925000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f92567fc700 1 -- 172.21.15.46:0/867950338 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250009880 con 0x7f9260068810 2024-03-18T20:48:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f92567fc700 1 -- 172.21.15.46:0/867950338 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92500044e0 con 0x7f9260068810 2024-03-18T20:48:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f9267535700 1 -- 172.21.15.46:0/867950338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9258009d00 con 0x7f9260068810 2024-03-18T20:48:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f92567fc700 1 -- 172.21.15.46:0/867950338 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250010430 con 0x7f9260068810 2024-03-18T20:48:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.400+0000 7f9267535700 1 -- 172.21.15.46:0/867950338 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926019c960 con 0x7f9260068810 2024-03-18T20:48:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.403+0000 7f92567fc700 1 -- 172.21.15.46:0/867950338 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9250003ee0 con 0x7f9260068810 2024-03-18T20:48:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.404+0000 7f92567fc700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f924c0a1310 0x7f924c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.404+0000 7f92652d1700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f924c0a1310 0x7f924c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.404+0000 7f92567fc700 1 -- 172.21.15.46:0/867950338 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f9250014070 con 0x7f9260068810 2024-03-18T20:48:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.404+0000 7f92652d1700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f924c0a1310 0x7f924c0a37d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f925c005950 tx=0x7f925c0058e0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.405+0000 7f92736cab80 1 Processor -- start 2024-03-18T20:48:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.405+0000 7f92736cab80 1 -- start start 2024-03-18T20:48:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.405+0000 7f92736cab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb78a60 0x55cb6bb78e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.405+0000 7f92736cab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb79380 0x55cb6bb83ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.405+0000 7f92736cab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb84220 0x55cb6bb86610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.405+0000 7f92736cab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb6bad2990 con 0x55cb6bb79380 2024-03-18T20:48:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.405+0000 7f92736cab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb6bad0350 con 0x55cb6bb78a60 2024-03-18T20:48:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.405+0000 7f92736cab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb6bacbc40 con 0x55cb6bb84220 2024-03-18T20:48:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.405+0000 7f9264ad0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb79380 0x55cb6bb83ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.405+0000 7f92652d1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb78a60 0x55cb6bb78e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f92652d1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb78a60 0x55cb6bb78e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51242/0 (socket says 172.21.15.46:51242) 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f92652d1700 1 -- 172.21.15.46:0/61036156 learned_addr learned my addr 172.21.15.46:0/61036156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f9265ad2700 1 --2- 172.21.15.46:0/61036156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb84220 0x55cb6bb86610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f92652d1700 1 -- 172.21.15.46:0/61036156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb84220 msgr2=0x55cb6bb86610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f92652d1700 1 --2- 172.21.15.46:0/61036156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb84220 0x55cb6bb86610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f92652d1700 1 -- 172.21.15.46:0/61036156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb79380 msgr2=0x55cb6bb83ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f92652d1700 1 --2- 172.21.15.46:0/61036156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb79380 0x55cb6bb83ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f92652d1700 1 -- 172.21.15.46:0/61036156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb6b94d3b0 con 0x55cb6bb78a60 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f9264ad0700 1 --2- 172.21.15.46:0/61036156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb79380 0x55cb6bb83ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f92652d1700 1 --2- 172.21.15.46:0/61036156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb78a60 0x55cb6bb78e40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f925c00ebc0 tx=0x7f925c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88924e17d842f0ec server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f924b7fe700 1 -- 172.21.15.46:0/61036156 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c010ac0 con 0x55cb6bb78a60 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f924b7fe700 1 -- 172.21.15.46:0/61036156 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f925c009820 con 0x55cb6bb78a60 2024-03-18T20:48:37.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.406+0000 7f924b7fe700 1 -- 172.21.15.46:0/61036156 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c00f420 con 0x55cb6bb78a60 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.407+0000 7f92736cab80 1 -- 172.21.15.46:0/61036156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb78a60 msgr2=0x55cb6bb78e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.407+0000 7f92736cab80 1 --2- 172.21.15.46:0/61036156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb78a60 0x55cb6bb78e40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f925c00ebc0 tx=0x7f925c011000 comp rx=0 tx=0).stop 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.407+0000 7f92736cab80 1 -- 172.21.15.46:0/61036156 shutdown_connections 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.407+0000 7f92736cab80 1 --2- 172.21.15.46:0/61036156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb84220 0x55cb6bb86610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.407+0000 7f92736cab80 1 --2- 172.21.15.46:0/61036156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb78a60 0x55cb6bb78e40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.407+0000 7f92736cab80 1 --2- 172.21.15.46:0/61036156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb79380 0x55cb6bb83ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.407+0000 7f92736cab80 1 -- 172.21.15.46:0/61036156 >> 172.21.15.46:0/61036156 conn(0x55cb6baef220 msgr2=0x55cb6bb75020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.407+0000 7f92736cab80 1 -- 172.21.15.46:0/61036156 shutdown_connections 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.407+0000 7f92736cab80 1 -- 172.21.15.46:0/61036156 wait complete. 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f92736cab80 1 -- start start 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f92736cab80 1 Processor -- start 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f92736cab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb78a60 0x55cb6bb83c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f92736cab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb79380 0x55cb6bb80f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f92736cab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 0x55cb6bb816b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f92736cab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb6ba67d80 con 0x55cb6bb78a60 2024-03-18T20:48:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f92736cab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb6ba67a20 con 0x55cb6bb84220 2024-03-18T20:48:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f92736cab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb6b94f6a0 con 0x55cb6bb79380 2024-03-18T20:48:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f92736cab80 10 client.0 _ll_register_callbacks cb 0x55cb6b8d33a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f92652d1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb78a60 0x55cb6bb83c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f9264ad0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb79380 0x55cb6bb80f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f9265ad2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 0x55cb6bb816b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f92652d1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb78a60 0x55cb6bb83c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54754/0 (socket says 172.21.15.46:54754) 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f92652d1700 1 -- 172.21.15.46:0/3311204393 learned_addr learned my addr 172.21.15.46:0/3311204393 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.408+0000 7f9265ad2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 0x55cb6bb816b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51250/0 (socket says 172.21.15.46:51250) 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f9265ad2700 1 -- 172.21.15.46:0/3311204393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb79380 msgr2=0x55cb6bb80f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f9265ad2700 1 --2- 172.21.15.46:0/3311204393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb79380 0x55cb6bb80f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f9265ad2700 1 -- 172.21.15.46:0/3311204393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb78a60 msgr2=0x55cb6bb83c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f9265ad2700 1 --2- 172.21.15.46:0/3311204393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb78a60 0x55cb6bb83c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f9265ad2700 1 -- 172.21.15.46:0/3311204393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926019c960 con 0x55cb6bb84220 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f92652d1700 1 --2- 172.21.15.46:0/3311204393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb78a60 0x55cb6bb83c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f9265ad2700 1 --2- 172.21.15.46:0/3311204393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 0x55cb6bb816b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f925800b780 tx=0x7f9258007f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cda35bb20927fae0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f9248ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f9248ff9700 1 -- 172.21.15.46:0/3311204393 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925800e040 con 0x55cb6bb84220 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f9248ff9700 1 -- 172.21.15.46:0/3311204393 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f9258004a50 con 0x55cb6bb84220 2024-03-18T20:48:37.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f92736cab80 1 -- 172.21.15.46:0/3311204393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb6bac4700 con 0x55cb6bb84220 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f9248ff9700 1 -- 172.21.15.46:0/3311204393 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9258004c30 con 0x55cb6bb84220 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f92736cab80 10 client.29510 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.409+0000 7f92736cab80 1 -- 172.21.15.46:0/3311204393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f925c00e870 con 0x55cb6bb84220 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.410+0000 7f9248ff9700 1 -- 172.21.15.46:0/3311204393 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb6bac4700 con 0x55cb6bb84220 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.410+0000 7f9248ff9700 1 -- 172.21.15.46:0/3311204393 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9258024030 con 0x55cb6bb84220 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.410+0000 7f92736cab80 10 client.29510 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.410+0000 7f92736cab80 10 client.29510 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.410+0000 7f92736cab80 20 client.29510 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.410+0000 7f92736cab80 1 --2- 172.21.15.46:0/3311204393 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cb6bb93e10 0x55cb6bb96230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.410+0000 7f92736cab80 4 client.29510 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.410+0000 7f92736cab80 1 -- 172.21.15.46:0/3311204393 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb6b91acf0 con 0x55cb6bb93e10 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.410+0000 7f92652d1700 1 --2- 172.21.15.46:0/3311204393 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cb6bb93e10 0x55cb6bb96230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f9248ff9700 10 client.29510 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f92652d1700 1 --2- 172.21.15.46:0/3311204393 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cb6bb93e10 0x55cb6bb96230 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6f595474e60c0348 server_cookie=b451187f1efcd7dc in_seq=0 out_seq=0 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f9248ff9700 1 -- 172.21.15.46:0/3311204393 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cb6b91acf0 con 0x55cb6bb93e10 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f9248ff9700 10 client.29510 handle_command_reply: tid=0 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f92736cab80 1 client.29510 shutdown 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f92736cab80 1 -- 172.21.15.46:0/3311204393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 msgr2=0x55cb6bb816b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f92736cab80 1 --2- 172.21.15.46:0/3311204393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 0x55cb6bb816b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f925800b780 tx=0x7f9258007f30 comp rx=0 tx=0).stop 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f92736cab80 1 -- 172.21.15.46:0/3311204393 shutdown_connections 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f92736cab80 1 --2- 172.21.15.46:0/3311204393 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cb6bb93e10 0x55cb6bb96230 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f92736cab80 1 --2- 172.21.15.46:0/3311204393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb78a60 0x55cb6bb83c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f92736cab80 1 --2- 172.21.15.46:0/3311204393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 0x55cb6bb816b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f92736cab80 1 --2- 172.21.15.46:0/3311204393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb79380 0x55cb6bb80f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f9248ff9700 0 client.29510 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.411+0000 7f92736cab80 1 -- 172.21.15.46:0/3311204393 >> 172.21.15.46:0/3311204393 conn(0x55cb6baef220 msgr2=0x55cb6bb74c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.412+0000 7f92736cab80 1 -- 172.21.15.46:0/3311204393 shutdown_connections 2024-03-18T20:48:37.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.412+0000 7f92736cab80 1 -- 172.21.15.46:0/3311204393 wait complete. 2024-03-18T20:48:37.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.412+0000 7f92736cab80 20 client.29510 trim_cache size 0 max 16384 2024-03-18T20:48:37.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.428+0000 7f92736cab80 1 Processor -- start 2024-03-18T20:48:37.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.429+0000 7f92736cab80 1 -- start start 2024-03-18T20:48:37.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.429+0000 7f92736cab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb78a60 0x55cb6bb81610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.429+0000 7f92736cab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb79380 0x55cb6bb8ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.429+0000 7f92736cab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb84220 0x55cb6bb73750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.429+0000 7f92736cab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb6bb79d70 con 0x55cb6bb78a60 2024-03-18T20:48:37.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.429+0000 7f92736cab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb6bb812f0 con 0x55cb6bb79380 2024-03-18T20:48:37.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.429+0000 7f92736cab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb6bb7a3d0 con 0x55cb6bb84220 2024-03-18T20:48:37.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.429+0000 7f9264ad0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb79380 0x55cb6bb8ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.429+0000 7f9264ad0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb79380 0x55cb6bb8ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51266/0 (socket says 172.21.15.46:51266) 2024-03-18T20:48:37.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.429+0000 7f9264ad0700 1 -- 172.21.15.46:0/4208119759 learned_addr learned my addr 172.21.15.46:0/4208119759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.430+0000 7f92652d1700 1 --2- 172.21.15.46:0/4208119759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb78a60 0x55cb6bb81610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.430+0000 7f9264ad0700 1 -- 172.21.15.46:0/4208119759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb84220 msgr2=0x55cb6bb73750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:48:37.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.430+0000 7f9264ad0700 1 --2- 172.21.15.46:0/4208119759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb84220 0x55cb6bb73750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.430+0000 7f9264ad0700 1 -- 172.21.15.46:0/4208119759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb78a60 msgr2=0x55cb6bb81610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.430+0000 7f9264ad0700 1 --2- 172.21.15.46:0/4208119759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb78a60 0x55cb6bb81610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.430+0000 7f9264ad0700 1 -- 172.21.15.46:0/4208119759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb6b8f9b60 con 0x55cb6bb79380 2024-03-18T20:48:37.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.431+0000 7f9264ad0700 1 --2- 172.21.15.46:0/4208119759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb79380 0x55cb6bb8ae80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9250099f90 tx=0x7f9250095f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72b1fde5bc61d461 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.431+0000 7f924bfff700 1 -- 172.21.15.46:0/4208119759 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925009a290 con 0x55cb6bb79380 2024-03-18T20:48:37.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.431+0000 7f924bfff700 1 -- 172.21.15.46:0/4208119759 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f925009e0c0 con 0x55cb6bb79380 2024-03-18T20:48:37.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.431+0000 7f924bfff700 1 -- 172.21.15.46:0/4208119759 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92500cab50 con 0x55cb6bb79380 2024-03-18T20:48:37.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.432+0000 7f92736cab80 1 -- 172.21.15.46:0/4208119759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb79380 msgr2=0x55cb6bb8ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.432+0000 7f92736cab80 1 --2- 172.21.15.46:0/4208119759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb79380 0x55cb6bb8ae80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9250099f90 tx=0x7f9250095f60 comp rx=0 tx=0).stop 2024-03-18T20:48:37.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.432+0000 7f92736cab80 1 -- 172.21.15.46:0/4208119759 shutdown_connections 2024-03-18T20:48:37.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.432+0000 7f92736cab80 1 --2- 172.21.15.46:0/4208119759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb78a60 0x55cb6bb81610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.432+0000 7f92736cab80 1 --2- 172.21.15.46:0/4208119759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb84220 0x55cb6bb73750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.432+0000 7f92736cab80 1 --2- 172.21.15.46:0/4208119759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb79380 0x55cb6bb8ae80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.432+0000 7f92736cab80 1 -- 172.21.15.46:0/4208119759 >> 172.21.15.46:0/4208119759 conn(0x55cb6baef220 msgr2=0x55cb6bb70a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.432+0000 7f92736cab80 1 -- 172.21.15.46:0/4208119759 shutdown_connections 2024-03-18T20:48:37.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.432+0000 7f92736cab80 1 -- 172.21.15.46:0/4208119759 wait complete. 2024-03-18T20:48:37.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.433+0000 7f92736cab80 1 -- start start 2024-03-18T20:48:37.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.433+0000 7f92736cab80 1 Processor -- start 2024-03-18T20:48:37.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92736cab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb78a60 0x55cb6bbe1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92736cab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb79380 0x55cb6bbe1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92736cab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 0x55cb6bbdfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92736cab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb6bbe03b0 con 0x55cb6bb79380 2024-03-18T20:48:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92736cab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb6bbe0520 con 0x55cb6bb84220 2024-03-18T20:48:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92736cab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb6bbe06e0 con 0x55cb6bb78a60 2024-03-18T20:48:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92736cab80 10 client.0 _ll_register_callbacks cb 0x7f9258009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92652d1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb78a60 0x55cb6bbe1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92652d1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb78a60 0x55cb6bbe1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33098/0 (socket says 172.21.15.46:33098) 2024-03-18T20:48:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92652d1700 1 -- 172.21.15.46:0/2148344244 learned_addr learned my addr 172.21.15.46:0/2148344244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f9264ad0700 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb79380 0x55cb6bbe1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92652d1700 1 -- 172.21.15.46:0/2148344244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 msgr2=0x55cb6bbdfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f9265ad2700 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 0x55cb6bbdfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92652d1700 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 0x55cb6bbdfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92652d1700 1 -- 172.21.15.46:0/2148344244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb79380 msgr2=0x55cb6bbe1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.434+0000 7f92652d1700 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb79380 0x55cb6bbe1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.435+0000 7f92652d1700 1 -- 172.21.15.46:0/2148344244 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9258009d00 con 0x55cb6bb78a60 2024-03-18T20:48:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.435+0000 7f9264ad0700 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb79380 0x55cb6bbe1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.435+0000 7f9265ad2700 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 0x55cb6bbdfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:48:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.435+0000 7f92652d1700 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb78a60 0x55cb6bbe1300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f925c00ebc0 tx=0x7f925c00efb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34e641571dff21b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.435+0000 7f9248ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:48:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.435+0000 7f9248ff9700 1 -- 172.21.15.46:0/2148344244 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c02a040 con 0x55cb6bb78a60 2024-03-18T20:48:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.435+0000 7f9248ff9700 1 -- 172.21.15.46:0/2148344244 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f925c01c620 con 0x55cb6bb78a60 2024-03-18T20:48:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.435+0000 7f92736cab80 1 -- 172.21.15.46:0/2148344244 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb6bb6cad0 con 0x55cb6bb78a60 2024-03-18T20:48:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.435+0000 7f9248ff9700 1 -- 172.21.15.46:0/2148344244 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f925c01c7e0 con 0x55cb6bb78a60 2024-03-18T20:48:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.436+0000 7f92736cab80 10 client.29551 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.436+0000 7f92736cab80 1 -- 172.21.15.46:0/2148344244 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55cb6ba91ea0 con 0x55cb6bb78a60 2024-03-18T20:48:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.437+0000 7f9248ff9700 1 -- 172.21.15.46:0/2148344244 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb6bb6cad0 con 0x55cb6bb78a60 2024-03-18T20:48:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.437+0000 7f9248ff9700 1 -- 172.21.15.46:0/2148344244 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f925c018070 con 0x55cb6bb78a60 2024-03-18T20:48:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.437+0000 7f92736cab80 10 client.29551 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.437+0000 7f92736cab80 10 client.29551 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:48:37.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.437+0000 7f92736cab80 20 client.29551 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.437+0000 7f92736cab80 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cb6bbf17b0 0x55cb6bbf3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.437+0000 7f92736cab80 4 client.29551 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:48:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.437+0000 7f92736cab80 1 -- 172.21.15.46:0/2148344244 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb6b94fce0 con 0x55cb6bbf17b0 2024-03-18T20:48:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.437+0000 7f9264ad0700 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cb6bbf17b0 0x55cb6bbf3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.438+0000 7f9248ff9700 10 client.29551 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:48:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.438+0000 7f9264ad0700 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cb6bbf17b0 0x55cb6bbf3bd0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=67d66e8e3e9051cd server_cookie=831a5cd9e3247564 in_seq=0 out_seq=0 2024-03-18T20:48:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.438+0000 7f9248ff9700 1 -- 172.21.15.46:0/2148344244 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb6b94fce0 con 0x55cb6bbf17b0 2024-03-18T20:48:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.438+0000 7f9248ff9700 10 client.29551 handle_command_reply: tid=0 2024-03-18T20:48:37.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.438+0000 7f92736cab80 1 client.29551 shutdown 2024-03-18T20:48:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.438+0000 7f92736cab80 1 -- 172.21.15.46:0/2148344244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb78a60 msgr2=0x55cb6bbe1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.439+0000 7f92736cab80 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb78a60 0x55cb6bbe1300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f925c00ebc0 tx=0x7f925c00efb0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.439+0000 7f92736cab80 1 -- 172.21.15.46:0/2148344244 shutdown_connections 2024-03-18T20:48:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.439+0000 7f92736cab80 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb6bb84220 0x55cb6bbdfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.439+0000 7f92736cab80 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cb6bb78a60 0x55cb6bbe1300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.439+0000 7f92736cab80 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x55cb6bbf17b0 0x55cb6bbf3bd0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.439+0000 7f92736cab80 1 --2- 172.21.15.46:0/2148344244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cb6bb79380 0x55cb6bbe1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.439+0000 7f92736cab80 1 -- 172.21.15.46:0/2148344244 >> 172.21.15.46:0/2148344244 conn(0x55cb6baef220 msgr2=0x55cb6bb73750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.439+0000 7f9248ff9700 0 client.29551 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:48:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.439+0000 7f92736cab80 1 -- 172.21.15.46:0/2148344244 shutdown_connections 2024-03-18T20:48:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.439+0000 7f92736cab80 1 -- 172.21.15.46:0/2148344244 wait complete. 2024-03-18T20:48:37.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.440+0000 7f92736cab80 20 client.29551 trim_cache size 0 max 16384 2024-03-18T20:48:37.443 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:48:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.441+0000 7f9267535700 1 -- 172.21.15.46:0/867950338 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f924c0a1310 msgr2=0x7f924c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.441+0000 7f9267535700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f924c0a1310 0x7f924c0a37d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f925c005950 tx=0x7f925c0058e0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.441+0000 7f9267535700 1 -- 172.21.15.46:0/867950338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260068810 msgr2=0x7f92601977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.441+0000 7f9267535700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260068810 0x7f92601977d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f925000eb30 tx=0x7f925000eef0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.441+0000 7f9267535700 1 -- 172.21.15.46:0/867950338 shutdown_connections 2024-03-18T20:48:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.441+0000 7f9267535700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f924c0a1310 0x7f924c0a37d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.441+0000 7f9267535700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9260067e50 0x7f9260197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.441+0000 7f9267535700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9260106e60 0x7f926019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.441+0000 7f9267535700 1 --2- 172.21.15.46:0/867950338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9260068810 0x7f92601977d0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f925000eb30 tx=0x7f925000eef0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.441+0000 7f9267535700 1 -- 172.21.15.46:0/867950338 >> 172.21.15.46:0/867950338 conn(0x7f9260076d40 msgr2=0x7f92600fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.441+0000 7f9267535700 1 -- 172.21.15.46:0/867950338 shutdown_connections 2024-03-18T20:48:37.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.441+0000 7f9267535700 1 -- 172.21.15.46:0/867950338 wait complete. 2024-03-18T20:48:37.461 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:48:37.461 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:48:37.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:37 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1688889950' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:48:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.609+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/1439108758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb8103160 msgr2=0x7f2cb8103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.609+0000 7f2cbd9cf700 1 --2- 172.21.15.46:0/1439108758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb8103160 0x7f2cb8103540 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009990 tx=0x7f2ca8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.609+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/1439108758 shutdown_connections 2024-03-18T20:48:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.609+0000 7f2cbd9cf700 1 --2- 172.21.15.46:0/1439108758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb8103160 0x7f2cb8103540 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.609+0000 7f2cbd9cf700 1 --2- 172.21.15.46:0/1439108758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb8108280 0x7f2cb810fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.610+0000 7f2cbd9cf700 1 --2- 172.21.15.46:0/1439108758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb8103b10 0x7f2cb8107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.610+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/1439108758 >> 172.21.15.46:0/1439108758 conn(0x7f2cb80fe980 msgr2=0x7f2cb8100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.610+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/1439108758 shutdown_connections 2024-03-18T20:48:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.610+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/1439108758 wait complete. 2024-03-18T20:48:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.610+0000 7f2cbd9cf700 1 Processor -- start 2024-03-18T20:48:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.610+0000 7f2cbd9cf700 1 -- start start 2024-03-18T20:48:37.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cbd9cf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb8103160 0x7f2cb8199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cbd9cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb8108280 0x7f2cb8199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cbd9cf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb819dcb0 0x7f2cb819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cbd9cf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb819e730 con 0x7f2cb8108280 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cbd9cf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb819e8a0 con 0x7f2cb8103160 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cbd9cf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb819ea60 con 0x7f2cb819dcb0 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cae7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb8108280 0x7f2cb8199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb8103160 0x7f2cb8199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cae7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb8108280 0x7f2cb8199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54782/0 (socket says 172.21.15.46:54782) 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cae7fc700 1 -- 172.21.15.46:0/14342579 learned_addr learned my addr 172.21.15.46:0/14342579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cb77fe700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb819dcb0 0x7f2cb819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cb6ffd700 1 -- 172.21.15.46:0/14342579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb819dcb0 msgr2=0x7f2cb819e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb819dcb0 0x7f2cb819e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cb6ffd700 1 -- 172.21.15.46:0/14342579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb8108280 msgr2=0x7f2cb8199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb8108280 0x7f2cb8199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cb6ffd700 1 -- 172.21.15.46:0/14342579 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80095f0 con 0x7f2cb8103160 2024-03-18T20:48:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.611+0000 7f2cae7fc700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb8108280 0x7f2cb8199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.612+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb8103160 0x7f2cb8199420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80059d0 tx=0x7f2ca8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.612+0000 7f2cbc9cd700 1 -- 172.21.15.46:0/14342579 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca801d070 con 0x7f2cb8103160 2024-03-18T20:48:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.612+0000 7f2cbc9cd700 1 -- 172.21.15.46:0/14342579 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ca8022470 con 0x7f2cb8103160 2024-03-18T20:48:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.612+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/14342579 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb81a5f60 con 0x7f2cb8103160 2024-03-18T20:48:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.612+0000 7f2cbc9cd700 1 -- 172.21.15.46:0/14342579 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8017650 con 0x7f2cb8103160 2024-03-18T20:48:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.612+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/14342579 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb81a6320 con 0x7f2cb8103160 2024-03-18T20:48:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.614+0000 7f2cbc9cd700 1 -- 172.21.15.46:0/14342579 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2ca8022870 con 0x7f2cb8103160 2024-03-18T20:48:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.615+0000 7f2cbc9cd700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ca40a1290 0x7f2ca40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.615+0000 7f2cae7fc700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ca40a1290 0x7f2ca40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.615+0000 7f2cbc9cd700 1 -- 172.21.15.46:0/14342579 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f2ca80cdf40 con 0x7f2cb8103160 2024-03-18T20:48:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.616+0000 7f2cae7fc700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ca40a1290 0x7f2ca40a3750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2cb819a920 tx=0x7f2ca0007480 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.616+0000 7f2cc9b64b80 1 Processor -- start 2024-03-18T20:48:37.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cc9b64b80 1 -- start start 2024-03-18T20:48:37.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cc9b64b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7dba60 0x559b0a7dbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cc9b64b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a7e6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cc9b64b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7e7220 0x559b0a7e9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cc9b64b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0a735990 con 0x559b0a7e7220 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cc9b64b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0a733350 con 0x559b0a7dc380 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cc9b64b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0a72ec40 con 0x559b0a7dba60 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7dba60 0x559b0a7dbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cae7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a7e6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7dba60 0x559b0a7dbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33134/0 (socket says 172.21.15.46:33134) 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cb6ffd700 1 -- 172.21.15.46:0/2202430018 learned_addr learned my addr 172.21.15.46:0/2202430018 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cb77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7e7220 0x559b0a7e9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cb6ffd700 1 -- 172.21.15.46:0/2202430018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 msgr2=0x559b0a7e6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/2202430018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a7e6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cb6ffd700 1 -- 172.21.15.46:0/2202430018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7e7220 msgr2=0x559b0a7e9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/2202430018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7e7220 0x559b0a7e9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.617+0000 7f2cb6ffd700 1 -- 172.21.15.46:0/2202430018 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b0a5b03b0 con 0x559b0a7dba60 2024-03-18T20:48:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.618+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/2202430018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7dba60 0x559b0a7dbe40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80976e0 tx=0x7f2ca809ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=230065c8d233aca8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.618+0000 7f2cadffb700 1 -- 172.21.15.46:0/2202430018 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca809b1e0 con 0x559b0a7dba60 2024-03-18T20:48:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.618+0000 7f2cadffb700 1 -- 172.21.15.46:0/2202430018 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ca809f010 con 0x559b0a7dba60 2024-03-18T20:48:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.618+0000 7f2cadffb700 1 -- 172.21.15.46:0/2202430018 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80a1300 con 0x559b0a7dba60 2024-03-18T20:48:37.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.619+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/2202430018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7dba60 msgr2=0x559b0a7dbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.619+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/2202430018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7dba60 0x559b0a7dbe40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80976e0 tx=0x7f2ca809ad20 comp rx=0 tx=0).stop 2024-03-18T20:48:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.619+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/2202430018 shutdown_connections 2024-03-18T20:48:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.619+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/2202430018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7e7220 0x559b0a7e9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.619+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/2202430018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7dba60 0x559b0a7dbe40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.619+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/2202430018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a7e6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.619+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/2202430018 >> 172.21.15.46:0/2202430018 conn(0x559b0a752220 msgr2=0x559b0a7d8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.619+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/2202430018 shutdown_connections 2024-03-18T20:48:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.619+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/2202430018 wait complete. 2024-03-18T20:48:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.619+0000 7f2cc9b64b80 1 -- start start 2024-03-18T20:48:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cc9b64b80 1 Processor -- start 2024-03-18T20:48:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cc9b64b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a7e7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cc9b64b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a7e7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cc9b64b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7e7220 0x559b0a7f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cc9b64b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0a6cad80 con 0x559b0a7dba60 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cc9b64b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0a6caa20 con 0x559b0a7dc380 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cc9b64b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0a5b26a0 con 0x559b0a7e7220 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cc9b64b80 10 client.0 _ll_register_callbacks cb 0x559b0a5363a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a7e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a7e7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54804/0 (socket says 172.21.15.46:54804) 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cb6ffd700 1 -- 172.21.15.46:0/1732696352 learned_addr learned my addr 172.21.15.46:0/1732696352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cae7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a7e7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cb77fe700 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7e7220 0x559b0a7f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cb6ffd700 1 -- 172.21.15.46:0/1732696352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7e7220 msgr2=0x559b0a7f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7e7220 0x559b0a7f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cb6ffd700 1 -- 172.21.15.46:0/1732696352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 msgr2=0x559b0a7e7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a7e7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cb6ffd700 1 -- 172.21.15.46:0/1732696352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb81a6320 con 0x559b0a7dba60 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.620+0000 7f2cb77fe700 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7e7220 0x559b0a7f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.621+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a7e7600 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80cb720 tx=0x7f2ca8009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8902fbe821c6e125 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.621+0000 7f2c96ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.621+0000 7f2c96ffd700 1 -- 172.21.15.46:0/1732696352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80a35d0 con 0x559b0a7dba60 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.621+0000 7f2c96ffd700 1 -- 172.21.15.46:0/1732696352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f2ca80b9030 con 0x559b0a7dba60 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.621+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/1732696352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b0a7cfad0 con 0x559b0a7dba60 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.621+0000 7f2c96ffd700 1 -- 172.21.15.46:0/1732696352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca809b1e0 con 0x559b0a7dba60 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.621+0000 7f2cc9b64b80 10 client.19917 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.621+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/1732696352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x559b0a55cb60 con 0x559b0a7dba60 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.621+0000 7f2c96ffd700 1 -- 172.21.15.46:0/1732696352 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b0a7cfad0 con 0x559b0a7dba60 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.621+0000 7f2c96ffd700 1 -- 172.21.15.46:0/1732696352 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2ca80a3140 con 0x559b0a7dba60 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.621+0000 7f2cc9b64b80 10 client.19917 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.621+0000 7f2cc9b64b80 10 client.19917 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:48:37.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.622+0000 7f2cc9b64b80 20 client.19917 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.622+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559b0a7f65a0 0x559b0a7f8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.622+0000 7f2cc9b64b80 4 client.19917 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.622+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/1732696352 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b0a727700 con 0x559b0a7f65a0 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.622+0000 7f2cae7fc700 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559b0a7f65a0 0x559b0a7f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.622+0000 7f2cae7fc700 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559b0a7f65a0 0x559b0a7f8990 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd32d8a9b59574ff server_cookie=dc79b4fa82cdf28d in_seq=0 out_seq=0 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.622+0000 7f2c96ffd700 10 client.19917 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2c96ffd700 1 -- 172.21.15.46:0/1732696352 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559b0a727700 con 0x559b0a7f65a0 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2c96ffd700 10 client.19917 handle_command_reply: tid=0 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2cc9b64b80 1 client.19917 shutdown 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/1732696352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 msgr2=0x559b0a7e7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a7e7600 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80cb720 tx=0x7f2ca8009520 comp rx=0 tx=0).stop 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/1732696352 shutdown_connections 2024-03-18T20:48:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559b0a7f65a0 0x559b0a7f8990 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a7e7600 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7e7220 0x559b0a7f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/1732696352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a7e7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/1732696352 >> 172.21.15.46:0/1732696352 conn(0x559b0a752220 msgr2=0x559b0a7d7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2c96ffd700 0 client.19917 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:48:37.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/1732696352 shutdown_connections 2024-03-18T20:48:37.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.623+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/1732696352 wait complete. 2024-03-18T20:48:37.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.624+0000 7f2cc9b64b80 20 client.19917 trim_cache size 0 max 16384 2024-03-18T20:48:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.638+0000 7f2cc9b64b80 1 Processor -- start 2024-03-18T20:48:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.638+0000 7f2cc9b64b80 1 -- start start 2024-03-18T20:48:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.638+0000 7f2cc9b64b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a7e5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.638+0000 7f2cc9b64b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7dc380 0x559b0a7ec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.638+0000 7f2cc9b64b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7e7220 0x559b0a7e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.638+0000 7f2cc9b64b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0a7dd9b0 con 0x559b0a7dba60 2024-03-18T20:48:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cc9b64b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0a7a2940 con 0x559b0a7e7220 2024-03-18T20:48:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cc9b64b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0a7e6360 con 0x559b0a7dc380 2024-03-18T20:48:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a7e5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cb77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7e7220 0x559b0a7e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a7e5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54810/0 (socket says 172.21.15.46:54810) 2024-03-18T20:48:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cb6ffd700 1 -- 172.21.15.46:0/3213171379 learned_addr learned my addr 172.21.15.46:0/3213171379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cae7fc700 1 --2- 172.21.15.46:0/3213171379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7dc380 0x559b0a7ec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cb77fe700 1 -- 172.21.15.46:0/3213171379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7dc380 msgr2=0x559b0a7ec640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cb77fe700 1 --2- 172.21.15.46:0/3213171379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7dc380 0x559b0a7ec640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cb77fe700 1 -- 172.21.15.46:0/3213171379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 msgr2=0x559b0a7e5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cb77fe700 1 --2- 172.21.15.46:0/3213171379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a7e5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cb77fe700 1 -- 172.21.15.46:0/3213171379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b0a6f4ea0 con 0x559b0a7e7220 2024-03-18T20:48:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/3213171379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a7e5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2cb77fe700 1 --2- 172.21.15.46:0/3213171379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7e7220 0x559b0a7e37c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00093a0 tx=0x7f2cb0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=349ef2ad8c21a7ed server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.639+0000 7f2caeffd700 1 -- 172.21.15.46:0/3213171379 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb000e040 con 0x559b0a7e7220 2024-03-18T20:48:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.640+0000 7f2caeffd700 1 -- 172.21.15.46:0/3213171379 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cb000f040 con 0x559b0a7e7220 2024-03-18T20:48:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.640+0000 7f2caeffd700 1 -- 172.21.15.46:0/3213171379 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0017670 con 0x559b0a7e7220 2024-03-18T20:48:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.640+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3213171379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7e7220 msgr2=0x559b0a7e37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.640+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/3213171379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7e7220 0x559b0a7e37c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00093a0 tx=0x7f2cb0009920 comp rx=0 tx=0).stop 2024-03-18T20:48:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.640+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3213171379 shutdown_connections 2024-03-18T20:48:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.640+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/3213171379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a7e5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.640+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/3213171379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7e7220 0x559b0a7e37c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.640+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/3213171379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7dc380 0x559b0a7ec640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.640+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3213171379 >> 172.21.15.46:0/3213171379 conn(0x559b0a752220 msgr2=0x559b0a7e1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.640+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3213171379 shutdown_connections 2024-03-18T20:48:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.640+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3213171379 wait complete. 2024-03-18T20:48:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.641+0000 7f2cc9b64b80 1 -- start start 2024-03-18T20:48:37.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.641+0000 7f2cc9b64b80 1 Processor -- start 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.641+0000 7f2cc9b64b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a843880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.641+0000 7f2cc9b64b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a843dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.641+0000 7f2cc9b64b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7e7220 0x559b0a83e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.641+0000 7f2cc9b64b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0a83e8f0 con 0x559b0a7dba60 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.641+0000 7f2cc9b64b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0a83ea60 con 0x559b0a7dc380 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.641+0000 7f2cc9b64b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0a83ec20 con 0x559b0a7e7220 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cc9b64b80 10 client.0 _ll_register_callbacks cb 0x7f2ca80b1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cae7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a843dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a843880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cae7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a843dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51332/0 (socket says 172.21.15.46:51332) 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cb6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a843880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54822/0 (socket says 172.21.15.46:54822) 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cae7fc700 1 -- 172.21.15.46:0/3309489207 learned_addr learned my addr 172.21.15.46:0/3309489207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cb77fe700 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7e7220 0x559b0a83e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cae7fc700 1 -- 172.21.15.46:0/3309489207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7e7220 msgr2=0x559b0a83e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cae7fc700 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7e7220 0x559b0a83e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cae7fc700 1 -- 172.21.15.46:0/3309489207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 msgr2=0x559b0a843880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cae7fc700 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a843880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cae7fc700 1 -- 172.21.15.46:0/3309489207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b0a55cb60 con 0x559b0a7dc380 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a843880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cae7fc700 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a843dc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0007fd0 tx=0x7f2ca00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e8805bd6e42023f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2c96ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2c96ffd700 1 -- 172.21.15.46:0/3309489207 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0024040 con 0x559b0a7dc380 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2c96ffd700 1 -- 172.21.15.46:0/3309489207 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f2ca0017070 con 0x559b0a7dc380 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.642+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3309489207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ca809f010 con 0x559b0a7dc380 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.643+0000 7f2c96ffd700 1 -- 172.21.15.46:0/3309489207 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca0004440 con 0x559b0a7dc380 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.643+0000 7f2cc9b64b80 10 client.29531 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:37.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.643+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3309489207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f2cb0009050 con 0x559b0a7dc380 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.643+0000 7f2c96ffd700 1 -- 172.21.15.46:0/3309489207 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ca809f010 con 0x559b0a7dc380 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.643+0000 7f2c96ffd700 1 -- 172.21.15.46:0/3309489207 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2ca0003a70 con 0x559b0a7dc380 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.643+0000 7f2cc9b64b80 10 client.29531 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.643+0000 7f2cc9b64b80 10 client.29531 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.643+0000 7f2cc9b64b80 20 client.29531 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.643+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559b0a84b800 0x559b0a84dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.643+0000 7f2cc9b64b80 4 client.29531 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.643+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3309489207 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559b0a57dcf0 con 0x559b0a84b800 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.643+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559b0a84b800 0x559b0a84dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cb6ffd700 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559b0a84b800 0x559b0a84dc20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82a4faa22ff3aa61 server_cookie=2023e0e3a675dc2 in_seq=0 out_seq=0 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2c96ffd700 10 client.29531 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2c96ffd700 1 -- 172.21.15.46:0/3309489207 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559b0a57dcf0 con 0x559b0a84b800 2024-03-18T20:48:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2c96ffd700 10 client.29531 handle_command_reply: tid=0 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cc9b64b80 1 client.29531 shutdown 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3309489207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 msgr2=0x559b0a843dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a843dc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0007fd0 tx=0x7f2ca00047c0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3309489207 shutdown_connections 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x559b0a84b800 0x559b0a84dc20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b0a7dba60 0x559b0a843880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b0a7e7220 0x559b0a83e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cc9b64b80 1 --2- 172.21.15.46:0/3309489207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559b0a7dc380 0x559b0a843dc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2c96ffd700 0 client.29531 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3309489207 >> 172.21.15.46:0/3309489207 conn(0x559b0a752220 msgr2=0x559b0a7ddaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3309489207 shutdown_connections 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cc9b64b80 1 -- 172.21.15.46:0/3309489207 wait complete. 2024-03-18T20:48:37.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.644+0000 7f2cc9b64b80 20 client.29531 trim_cache size 0 max 16384 2024-03-18T20:48:37.653 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:48:37.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.645+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/14342579 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ca40a1290 msgr2=0x7f2ca40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.645+0000 7f2cbd9cf700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ca40a1290 0x7f2ca40a3750 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2cb819a920 tx=0x7f2ca0007480 comp rx=0 tx=0).stop 2024-03-18T20:48:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.645+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/14342579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb8103160 msgr2=0x7f2cb8199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.645+0000 7f2cbd9cf700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb8103160 0x7f2cb8199420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80059d0 tx=0x7f2ca8004730 comp rx=0 tx=0).stop 2024-03-18T20:48:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.645+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/14342579 shutdown_connections 2024-03-18T20:48:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.645+0000 7f2cbd9cf700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f2ca40a1290 0x7f2ca40a3750 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.645+0000 7f2cbd9cf700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb819dcb0 0x7f2cb819e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.645+0000 7f2cbd9cf700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb8103160 0x7f2cb8199420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.645+0000 7f2cbd9cf700 1 --2- 172.21.15.46:0/14342579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb8108280 0x7f2cb8199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.645+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/14342579 >> 172.21.15.46:0/14342579 conn(0x7f2cb80fe980 msgr2=0x7f2cb80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.645+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/14342579 shutdown_connections 2024-03-18T20:48:37.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.645+0000 7f2cbd9cf700 1 -- 172.21.15.46:0/14342579 wait complete. 2024-03-18T20:48:37.662 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:48:37.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:48:37.723 INFO:tasks.workunit.client.0.smithi046.stdout: 1 470685 3.69 MB/sec execute 572 sec latency 93.954 ms 2024-03-18T20:48:37.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.798+0000 7fc7cdf03700 1 -- 172.21.15.46:0/1423239387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8100800 msgr2=0x7fc7c8104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.798+0000 7fc7cdf03700 1 --2- 172.21.15.46:0/1423239387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8100800 0x7fc7c8104830 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009990 tx=0x7fc7bc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.798+0000 7fc7cdf03700 1 -- 172.21.15.46:0/1423239387 shutdown_connections 2024-03-18T20:48:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.798+0000 7fc7cdf03700 1 --2- 172.21.15.46:0/1423239387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8104f60 0x7fc7c810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.798+0000 7fc7cdf03700 1 --2- 172.21.15.46:0/1423239387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7c80ffe50 0x7fc7c8100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.798+0000 7fc7cdf03700 1 --2- 172.21.15.46:0/1423239387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8100800 0x7fc7c8104830 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.798+0000 7fc7cdf03700 1 -- 172.21.15.46:0/1423239387 >> 172.21.15.46:0/1423239387 conn(0x7fc7c8076d40 msgr2=0x7fc7c8077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.798+0000 7fc7cdf03700 1 -- 172.21.15.46:0/1423239387 shutdown_connections 2024-03-18T20:48:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.798+0000 7fc7cdf03700 1 -- 172.21.15.46:0/1423239387 wait complete. 2024-03-18T20:48:37.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.799+0000 7fc7cdf03700 1 Processor -- start 2024-03-18T20:48:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.799+0000 7fc7cdf03700 1 -- start start 2024-03-18T20:48:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.799+0000 7fc7cdf03700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c80ffe50 0x7fc7c81018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.799+0000 7fc7cdf03700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8104f60 0x7fc7c8101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7cdf03700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7c8102350 0x7fc7c81a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7cdf03700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c81a5a40 con 0x7fc7c8104f60 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7cdf03700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c81a5bb0 con 0x7fc7c8102350 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7cdf03700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c81a5d70 con 0x7fc7c80ffe50 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8104f60 0x7fc7c8101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8104f60 0x7fc7c8101e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54840/0 (socket says 172.21.15.46:54840) 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c6ffd700 1 -- 172.21.15.46:0/613233999 learned_addr learned my addr 172.21.15.46:0/613233999 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7c8102350 0x7fc7c81a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c77fe700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c80ffe50 0x7fc7c81018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c6ffd700 1 -- 172.21.15.46:0/613233999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c80ffe50 msgr2=0x7fc7c81018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c80ffe50 0x7fc7c81018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c6ffd700 1 -- 172.21.15.46:0/613233999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7c8102350 msgr2=0x7fc7c81a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7c8102350 0x7fc7c81a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c6ffd700 1 -- 172.21.15.46:0/613233999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc0095f0 con 0x7fc7c8104f60 2024-03-18T20:48:37.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7c8102350 0x7fc7c81a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8104f60 0x7fc7c8101e10 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc000c00 tx=0x7fc7bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c4ff9700 1 -- 172.21.15.46:0/613233999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc01d070 con 0x7fc7c8104f60 2024-03-18T20:48:37.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c4ff9700 1 -- 172.21.15.46:0/613233999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7bc021410 con 0x7fc7c8104f60 2024-03-18T20:48:37.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7cdf03700 1 -- 172.21.15.46:0/613233999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c81a5ff0 con 0x7fc7c8104f60 2024-03-18T20:48:37.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.800+0000 7fc7c4ff9700 1 -- 172.21.15.46:0/613233999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc017650 con 0x7fc7c8104f60 2024-03-18T20:48:37.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.801+0000 7fc7cdf03700 1 -- 172.21.15.46:0/613233999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c81a63b0 con 0x7fc7c8104f60 2024-03-18T20:48:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.803+0000 7fc7c4ff9700 1 -- 172.21.15.46:0/613233999 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc7bc021580 con 0x7fc7c8104f60 2024-03-18T20:48:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.804+0000 7fc7c4ff9700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7b00a1290 0x7fc7b00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.804+0000 7fc7c4ff9700 1 -- 172.21.15.46:0/613233999 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc7bc0cde70 con 0x7fc7c8104f60 2024-03-18T20:48:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.804+0000 7fc7c77fe700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7b00a1290 0x7fc7b00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.804+0000 7fc7da098b80 1 Processor -- start 2024-03-18T20:48:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.804+0000 7fc7da098b80 1 -- start start 2024-03-18T20:48:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.804+0000 7fc7c77fe700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7b00a1290 0x7fc7b00a3750 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009c60 tx=0x7fc7b8009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7da098b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1ba60 0x558ca0e1be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7da098b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1c380 0x558ca0e26ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7da098b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e27220 0x558ca0e29610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7da098b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca0d75990 con 0x558ca0e1c380 2024-03-18T20:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7da098b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca0d73350 con 0x558ca0e27220 2024-03-18T20:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7da098b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca0d6ec40 con 0x558ca0e1ba60 2024-03-18T20:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7c6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1c380 0x558ca0e26ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7c6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1c380 0x558ca0e26ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54846/0 (socket says 172.21.15.46:54846) 2024-03-18T20:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7c6ffd700 1 -- 172.21.15.46:0/3657604566 learned_addr learned my addr 172.21.15.46:0/3657604566 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7c77fe700 1 --2- 172.21.15.46:0/3657604566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1ba60 0x558ca0e1be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/3657604566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e27220 0x558ca0e29610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7c6ffd700 1 -- 172.21.15.46:0/3657604566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1ba60 msgr2=0x558ca0e1be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/3657604566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1ba60 0x558ca0e1be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7c6ffd700 1 -- 172.21.15.46:0/3657604566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e27220 msgr2=0x558ca0e29610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/3657604566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e27220 0x558ca0e29610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7c6ffd700 1 -- 172.21.15.46:0/3657604566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ca0bf03b0 con 0x558ca0e1c380 2024-03-18T20:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/3657604566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1c380 0x558ca0e26ce0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0974f0 tx=0x7fc7bc097400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789dccfe9d854108 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7adffb700 1 -- 172.21.15.46:0/3657604566 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc09a950 con 0x558ca0e1c380 2024-03-18T20:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7adffb700 1 -- 172.21.15.46:0/3657604566 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7bc09b6f0 con 0x558ca0e1c380 2024-03-18T20:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.805+0000 7fc7adffb700 1 -- 172.21.15.46:0/3657604566 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc09b250 con 0x558ca0e1c380 2024-03-18T20:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.806+0000 7fc7da098b80 1 -- 172.21.15.46:0/3657604566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1c380 msgr2=0x558ca0e26ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.806+0000 7fc7da098b80 1 --2- 172.21.15.46:0/3657604566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1c380 0x558ca0e26ce0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0974f0 tx=0x7fc7bc097400 comp rx=0 tx=0).stop 2024-03-18T20:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.806+0000 7fc7da098b80 1 -- 172.21.15.46:0/3657604566 shutdown_connections 2024-03-18T20:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.806+0000 7fc7da098b80 1 --2- 172.21.15.46:0/3657604566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e27220 0x558ca0e29610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.806+0000 7fc7da098b80 1 --2- 172.21.15.46:0/3657604566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1ba60 0x558ca0e1be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.806+0000 7fc7da098b80 1 --2- 172.21.15.46:0/3657604566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1c380 0x558ca0e26ce0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.806+0000 7fc7da098b80 1 -- 172.21.15.46:0/3657604566 >> 172.21.15.46:0/3657604566 conn(0x558ca0d92220 msgr2=0x558ca0e18020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.806+0000 7fc7da098b80 1 -- 172.21.15.46:0/3657604566 shutdown_connections 2024-03-18T20:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.806+0000 7fc7da098b80 1 -- 172.21.15.46:0/3657604566 wait complete. 2024-03-18T20:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7da098b80 1 -- start start 2024-03-18T20:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7da098b80 1 Processor -- start 2024-03-18T20:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7da098b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1ba60 0x558ca0e277f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7da098b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e1c380 0x558ca0e27d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7da098b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 0x558ca0e32680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7da098b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca0d0ad80 con 0x558ca0e27220 2024-03-18T20:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7da098b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca0d0aa20 con 0x558ca0e1c380 2024-03-18T20:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7da098b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca0bf26a0 con 0x558ca0e1ba60 2024-03-18T20:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7da098b80 10 client.0 _ll_register_callbacks cb 0x558ca0b763a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 0x558ca0e32680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 0x558ca0e32680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54856/0 (socket says 172.21.15.46:54856) 2024-03-18T20:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7c6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e1c380 0x558ca0e27d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7c77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1ba60 0x558ca0e277f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.807+0000 7fc7c7fff700 1 -- 172.21.15.46:0/1089389896 learned_addr learned my addr 172.21.15.46:0/1089389896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7c7fff700 1 -- 172.21.15.46:0/1089389896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1ba60 msgr2=0x558ca0e277f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/1089389896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1ba60 0x558ca0e277f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7c7fff700 1 -- 172.21.15.46:0/1089389896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e1c380 msgr2=0x558ca0e27d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/1089389896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e1c380 0x558ca0e27d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7c7fff700 1 -- 172.21.15.46:0/1089389896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c81a63b0 con 0x558ca0e27220 2024-03-18T20:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/1089389896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e1c380 0x558ca0e27d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/1089389896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 0x558ca0e32680 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00042c0 tx=0x7fc7c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca5851fdafb3da8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7a6ffd700 1 -- 172.21.15.46:0/1089389896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0004db0 con 0x558ca0e27220 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7a6ffd700 1 -- 172.21.15.46:0/1089389896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc7c0007500 con 0x558ca0e27220 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7da098b80 1 -- 172.21.15.46:0/1089389896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ca0e0fad0 con 0x558ca0e27220 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7a6ffd700 1 -- 172.21.15.46:0/1089389896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7c0007700 con 0x558ca0e27220 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7da098b80 10 client.19935 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.808+0000 7fc7da098b80 1 -- 172.21.15.46:0/1089389896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x558ca0b9cb60 con 0x558ca0e27220 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.809+0000 7fc7a6ffd700 1 -- 172.21.15.46:0/1089389896 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558ca0e0fad0 con 0x558ca0e27220 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.809+0000 7fc7a6ffd700 1 -- 172.21.15.46:0/1089389896 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc7c000ea50 con 0x558ca0e27220 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.809+0000 7fc7da098b80 10 client.19935 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.809+0000 7fc7da098b80 10 client.19935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.809+0000 7fc7da098b80 20 client.19935 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.809+0000 7fc7da098b80 1 --2- 172.21.15.46:0/1089389896 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x558ca0e36b60 0x558ca0e38f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.809+0000 7fc7da098b80 4 client.19935 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.809+0000 7fc7da098b80 1 -- 172.21.15.46:0/1089389896 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ca0d67700 con 0x558ca0e36b60 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.809+0000 7fc7c77fe700 1 --2- 172.21.15.46:0/1089389896 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x558ca0e36b60 0x558ca0e38f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.809+0000 7fc7a6ffd700 10 client.19935 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.809+0000 7fc7c77fe700 1 --2- 172.21.15.46:0/1089389896 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x558ca0e36b60 0x558ca0e38f80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5057a4bf12a4b538 server_cookie=c26aa66b460033d7 in_seq=0 out_seq=0 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7a6ffd700 1 -- 172.21.15.46:0/1089389896 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558ca0d67700 con 0x558ca0e36b60 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7a6ffd700 10 client.19935 handle_command_reply: tid=0 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7da098b80 1 client.19935 shutdown 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7da098b80 1 -- 172.21.15.46:0/1089389896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 msgr2=0x558ca0e32680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7da098b80 1 --2- 172.21.15.46:0/1089389896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 0x558ca0e32680 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00042c0 tx=0x7fc7c00042f0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7da098b80 1 -- 172.21.15.46:0/1089389896 shutdown_connections 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7da098b80 1 --2- 172.21.15.46:0/1089389896 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x558ca0e36b60 0x558ca0e38f80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7da098b80 1 --2- 172.21.15.46:0/1089389896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 0x558ca0e32680 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7da098b80 1 --2- 172.21.15.46:0/1089389896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1ba60 0x558ca0e277f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7a6ffd700 0 client.19935 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:48:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7da098b80 1 --2- 172.21.15.46:0/1089389896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e1c380 0x558ca0e27d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7da098b80 1 -- 172.21.15.46:0/1089389896 >> 172.21.15.46:0/1089389896 conn(0x558ca0d92220 msgr2=0x558ca0e17c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7da098b80 1 -- 172.21.15.46:0/1089389896 shutdown_connections 2024-03-18T20:48:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7da098b80 1 -- 172.21.15.46:0/1089389896 wait complete. 2024-03-18T20:48:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.810+0000 7fc7da098b80 20 client.19935 trim_cache size 0 max 16384 2024-03-18T20:48:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.824+0000 7fc7da098b80 1 Processor -- start 2024-03-18T20:48:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.824+0000 7fc7da098b80 1 -- start start 2024-03-18T20:48:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.824+0000 7fc7da098b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e1ba60 0x558ca0e2c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.824+0000 7fc7da098b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1c380 0x558ca0e25e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.824+0000 7fc7da098b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 0x558ca0e17900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.824+0000 7fc7da098b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca0e33020 con 0x558ca0e27220 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.824+0000 7fc7da098b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca0e1cb20 con 0x558ca0e1ba60 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.824+0000 7fc7da098b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca0e1d400 con 0x558ca0e1c380 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 0x558ca0e17900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 0x558ca0e17900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54870/0 (socket says 172.21.15.46:54870) 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7c7fff700 1 -- 172.21.15.46:0/1928244928 learned_addr learned my addr 172.21.15.46:0/1928244928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7c77fe700 1 --2- 172.21.15.46:0/1928244928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e1ba60 0x558ca0e2c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/1928244928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1c380 0x558ca0e25e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7c7fff700 1 -- 172.21.15.46:0/1928244928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1c380 msgr2=0x558ca0e25e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/1928244928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1c380 0x558ca0e25e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7c7fff700 1 -- 172.21.15.46:0/1928244928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e1ba60 msgr2=0x558ca0e2c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/1928244928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e1ba60 0x558ca0e2c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7c7fff700 1 -- 172.21.15.46:0/1928244928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ca0d34ea0 con 0x558ca0e27220 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/1928244928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 0x558ca0e17900 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009020 tx=0x7fc7c00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d9bcdab4961346 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7ae7fc700 1 -- 172.21.15.46:0/1928244928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00164d0 con 0x558ca0e27220 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7ae7fc700 1 -- 172.21.15.46:0/1928244928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7c000f040 con 0x558ca0e27220 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.825+0000 7fc7ae7fc700 1 -- 172.21.15.46:0/1928244928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0016dd0 con 0x558ca0e27220 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.826+0000 7fc7da098b80 1 -- 172.21.15.46:0/1928244928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 msgr2=0x558ca0e17900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.826+0000 7fc7da098b80 1 --2- 172.21.15.46:0/1928244928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 0x558ca0e17900 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009020 tx=0x7fc7c00047b0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.826+0000 7fc7da098b80 1 -- 172.21.15.46:0/1928244928 shutdown_connections 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.826+0000 7fc7da098b80 1 --2- 172.21.15.46:0/1928244928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e27220 0x558ca0e17900 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.826+0000 7fc7da098b80 1 --2- 172.21.15.46:0/1928244928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e1ba60 0x558ca0e2c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.826+0000 7fc7da098b80 1 --2- 172.21.15.46:0/1928244928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1c380 0x558ca0e25e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.826+0000 7fc7da098b80 1 -- 172.21.15.46:0/1928244928 >> 172.21.15.46:0/1928244928 conn(0x558ca0d92220 msgr2=0x558ca0e171f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.826+0000 7fc7da098b80 1 -- 172.21.15.46:0/1928244928 shutdown_connections 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.826+0000 7fc7da098b80 1 -- 172.21.15.46:0/1928244928 wait complete. 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.826+0000 7fc7da098b80 1 -- start start 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.826+0000 7fc7da098b80 1 Processor -- start 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7da098b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1ba60 0x558ca0e7eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7da098b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1c380 0x558ca0e7f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7da098b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e27220 0x558ca0e899a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7da098b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca0e7f7a0 con 0x558ca0e1ba60 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7da098b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca0e89ee0 con 0x558ca0e27220 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7da098b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca0e8a0a0 con 0x558ca0e1c380 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7da098b80 10 client.0 _ll_register_callbacks cb 0x7fc7c0003a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7c77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1ba60 0x558ca0e7eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7c77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1ba60 0x558ca0e7eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54886/0 (socket says 172.21.15.46:54886) 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7c77fe700 1 -- 172.21.15.46:0/3872216659 learned_addr learned my addr 172.21.15.46:0/3872216659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1c380 0x558ca0e7f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e27220 0x558ca0e899a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7c77fe700 1 -- 172.21.15.46:0/3872216659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1c380 msgr2=0x558ca0e7f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7c77fe700 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1c380 0x558ca0e7f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7c77fe700 1 -- 172.21.15.46:0/3872216659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e27220 msgr2=0x558ca0e899a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7c77fe700 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e27220 0x558ca0e899a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7c77fe700 1 -- 172.21.15.46:0/3872216659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ca0b9cb60 con 0x558ca0e1ba60 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7c7fff700 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e27220 0x558ca0e899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7c77fe700 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1ba60 0x558ca0e7eb10 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x558ca0e39860 tx=0x7fc7b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7642267e88d39d9d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.827+0000 7fc7a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7a6ffd700 1 -- 172.21.15.46:0/3872216659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8022040 con 0x558ca0e1ba60 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7da098b80 1 -- 172.21.15.46:0/3872216659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7bc09b6f0 con 0x558ca0e1ba60 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7a6ffd700 1 -- 172.21.15.46:0/3872216659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc7b802b380 con 0x558ca0e1ba60 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7a6ffd700 1 -- 172.21.15.46:0/3872216659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7b802b540 con 0x558ca0e1ba60 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7da098b80 10 client.19941 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:37.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7da098b80 1 -- 172.21.15.46:0/3872216659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x558ca0bf4cd0 con 0x558ca0e1ba60 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7a6ffd700 1 -- 172.21.15.46:0/3872216659 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7bc09b6f0 con 0x558ca0e1ba60 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7a6ffd700 1 -- 172.21.15.46:0/3872216659 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc7b8034870 con 0x558ca0e1ba60 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7da098b80 10 client.19941 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7da098b80 10 client.19941 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7da098b80 20 client.19941 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7da098b80 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x558ca0e8b900 0x558ca0e8dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7da098b80 4 client.19941 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7da098b80 1 -- 172.21.15.46:0/3872216659 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558ca0bbdcf0 con 0x558ca0e8b900 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.828+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x558ca0e8b900 0x558ca0e8dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7a6ffd700 10 client.19941 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7c6ffd700 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x558ca0e8b900 0x558ca0e8dd20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6049fd2c31f25dbb server_cookie=c1314cc1928a8b17 in_seq=0 out_seq=0 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7a6ffd700 1 -- 172.21.15.46:0/3872216659 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558ca0bbdcf0 con 0x558ca0e8b900 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7a6ffd700 10 client.19941 handle_command_reply: tid=0 2024-03-18T20:48:37.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7da098b80 1 client.19941 shutdown 2024-03-18T20:48:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7da098b80 1 -- 172.21.15.46:0/3872216659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1ba60 msgr2=0x558ca0e7eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7da098b80 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1ba60 0x558ca0e7eb10 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x558ca0e39860 tx=0x7fc7b80047c0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7da098b80 1 -- 172.21.15.46:0/3872216659 shutdown_connections 2024-03-18T20:48:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7da098b80 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x558ca0e8b900 0x558ca0e8dd20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7da098b80 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558ca0e1ba60 0x558ca0e7eb10 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7da098b80 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558ca0e27220 0x558ca0e899a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7a6ffd700 0 client.19941 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:48:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7da098b80 1 --2- 172.21.15.46:0/3872216659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558ca0e1c380 0x558ca0e7f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7da098b80 1 -- 172.21.15.46:0/3872216659 >> 172.21.15.46:0/3872216659 conn(0x558ca0d92220 msgr2=0x558ca0e17a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7da098b80 1 -- 172.21.15.46:0/3872216659 shutdown_connections 2024-03-18T20:48:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7da098b80 1 -- 172.21.15.46:0/3872216659 wait complete. 2024-03-18T20:48:37.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.829+0000 7fc7da098b80 20 client.19941 trim_cache size 0 max 16384 2024-03-18T20:48:37.839 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:48:37.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.830+0000 7fc7cdf03700 1 -- 172.21.15.46:0/613233999 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7b00a1290 msgr2=0x7fc7b00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.830+0000 7fc7cdf03700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7b00a1290 0x7fc7b00a3750 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009c60 tx=0x7fc7b8009260 comp rx=0 tx=0).stop 2024-03-18T20:48:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.830+0000 7fc7cdf03700 1 -- 172.21.15.46:0/613233999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8104f60 msgr2=0x7fc7c8101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.830+0000 7fc7cdf03700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8104f60 0x7fc7c8101e10 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc000c00 tx=0x7fc7bc004730 comp rx=0 tx=0).stop 2024-03-18T20:48:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.830+0000 7fc7cdf03700 1 -- 172.21.15.46:0/613233999 shutdown_connections 2024-03-18T20:48:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.830+0000 7fc7cdf03700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7b00a1290 0x7fc7b00a3750 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.830+0000 7fc7cdf03700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7c8102350 0x7fc7c81a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.830+0000 7fc7cdf03700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c80ffe50 0x7fc7c81018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.830+0000 7fc7cdf03700 1 --2- 172.21.15.46:0/613233999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7c8104f60 0x7fc7c8101e10 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.830+0000 7fc7cdf03700 1 -- 172.21.15.46:0/613233999 >> 172.21.15.46:0/613233999 conn(0x7fc7c8076d40 msgr2=0x7fc7c8068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.830+0000 7fc7cdf03700 1 -- 172.21.15.46:0/613233999 shutdown_connections 2024-03-18T20:48:37.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.830+0000 7fc7cdf03700 1 -- 172.21.15.46:0/613233999 wait complete. 2024-03-18T20:48:37.846 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:48:37.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:48:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.984+0000 7f8f06779700 1 -- 172.21.15.46:0/2798033345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00101900 msgr2=0x7f8f00105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.984+0000 7f8f06779700 1 --2- 172.21.15.46:0/2798033345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00101900 0x7f8f00105940 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009990 tx=0x7f8ee8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.984+0000 7f8f06779700 1 -- 172.21.15.46:0/2798033345 shutdown_connections 2024-03-18T20:48:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.984+0000 7f8f06779700 1 --2- 172.21.15.46:0/2798033345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f00106070 0x7f8f0010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.984+0000 7f8f06779700 1 --2- 172.21.15.46:0/2798033345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f00100f50 0x7f8f00101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.984+0000 7f8f06779700 1 --2- 172.21.15.46:0/2798033345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00101900 0x7f8f00105940 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.984+0000 7f8f06779700 1 -- 172.21.15.46:0/2798033345 >> 172.21.15.46:0/2798033345 conn(0x7f8f000fc7f0 msgr2=0x7f8f000fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.984+0000 7f8f06779700 1 -- 172.21.15.46:0/2798033345 shutdown_connections 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.984+0000 7f8f06779700 1 -- 172.21.15.46:0/2798033345 wait complete. 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.985+0000 7f8f06779700 1 Processor -- start 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.985+0000 7f8f06779700 1 -- start start 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.985+0000 7f8f06779700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f00100f50 0x7f8f00197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.985+0000 7f8f06779700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f00106070 0x7f8f00197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.985+0000 7f8f06779700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0019ba90 0x7f8f0019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.985+0000 7f8f06779700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0019c510 con 0x7f8f0019ba90 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.985+0000 7f8f06779700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0019c680 con 0x7f8f00100f50 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.985+0000 7f8f06779700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0019c840 con 0x7f8f00106070 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.985+0000 7f8f04d16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0019ba90 0x7f8f0019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8efffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f00100f50 0x7f8f00197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8f04d16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0019ba90 0x7f8f0019bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54912/0 (socket says 172.21.15.46:54912) 2024-03-18T20:48:37.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8f04d16700 1 -- 172.21.15.46:0/3971798975 learned_addr learned my addr 172.21.15.46:0/3971798975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8eff7fe700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f00106070 0x7f8f00197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8efffff700 1 -- 172.21.15.46:0/3971798975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f00106070 msgr2=0x7f8f00197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8efffff700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f00106070 0x7f8f00197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8efffff700 1 -- 172.21.15.46:0/3971798975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0019ba90 msgr2=0x7f8f0019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8efffff700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0019ba90 0x7f8f0019bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8efffff700 1 -- 172.21.15.46:0/3971798975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee80095f0 con 0x7f8f00100f50 2024-03-18T20:48:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8efffff700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f00100f50 0x7f8f00197200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000e9f0 tx=0x7f8ef000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8efd7fa700 1 -- 172.21.15.46:0/3971798975 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000c9c0 con 0x7f8f00100f50 2024-03-18T20:48:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8f06779700 1 -- 172.21.15.46:0/3971798975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f001a3da0 con 0x7f8f00100f50 2024-03-18T20:48:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8efd7fa700 1 -- 172.21.15.46:0/3971798975 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ef0010430 con 0x7f8f00100f50 2024-03-18T20:48:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8efd7fa700 1 -- 172.21.15.46:0/3971798975 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0018680 con 0x7f8f00100f50 2024-03-18T20:48:37.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.986+0000 7f8f06779700 1 -- 172.21.15.46:0/3971798975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f001a4220 con 0x7f8f00100f50 2024-03-18T20:48:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.988+0000 7f8efd7fa700 1 -- 172.21.15.46:0/3971798975 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8ef0010860 con 0x7f8f00100f50 2024-03-18T20:48:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.989+0000 7f8efd7fa700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8eec0a1290 0x7f8eec0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.989+0000 7f8efd7fa700 1 -- 172.21.15.46:0/3971798975 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8ef0014070 con 0x7f8f00100f50 2024-03-18T20:48:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.989+0000 7f8eff7fe700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8eec0a1290 0x7f8eec0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.990+0000 7f8eff7fe700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8eec0a1290 0x7f8eec0a3750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800b2e0 tx=0x7f8ee8005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.990+0000 7f8f1290eb80 1 Processor -- start 2024-03-18T20:48:37.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.990+0000 7f8f1290eb80 1 -- start start 2024-03-18T20:48:37.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.990+0000 7f8f1290eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb245e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.990+0000 7f8f1290eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb246380 0x55f1bb2550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.990+0000 7f8f1290eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 0x55f1bb257a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:37.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.990+0000 7f8f1290eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1bb19f990 con 0x55f1bb255630 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.990+0000 7f8f1290eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1bb19d350 con 0x55f1bb245a60 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.990+0000 7f8f1290eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1bb198c40 con 0x55f1bb246380 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8f04d16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 0x55f1bb257a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8efffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb245e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8f04d16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 0x55f1bb257a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54918/0 (socket says 172.21.15.46:54918) 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8f04d16700 1 -- 172.21.15.46:0/527573417 learned_addr learned my addr 172.21.15.46:0/527573417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8eff7fe700 1 --2- 172.21.15.46:0/527573417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb246380 0x55f1bb2550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8efffff700 1 -- 172.21.15.46:0/527573417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb246380 msgr2=0x55f1bb2550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8efffff700 1 --2- 172.21.15.46:0/527573417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb246380 0x55f1bb2550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8efffff700 1 -- 172.21.15.46:0/527573417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 msgr2=0x55f1bb257a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8efffff700 1 --2- 172.21.15.46:0/527573417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 0x55f1bb257a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8efffff700 1 -- 172.21.15.46:0/527573417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1bb01a3b0 con 0x55f1bb245a60 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8f04d16700 1 --2- 172.21.15.46:0/527573417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 0x55f1bb257a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8efffff700 1 --2- 172.21.15.46:0/527573417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb245e40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00ca540 tx=0x7f8ef0099930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c1c4a29da21af19 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:37.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8ef67fc700 1 -- 172.21.15.46:0/527573417 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef009dd30 con 0x55f1bb245a60 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8ef67fc700 1 -- 172.21.15.46:0/527573417 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ef00a2380 con 0x55f1bb245a60 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.991+0000 7f8ef67fc700 1 -- 172.21.15.46:0/527573417 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0099650 con 0x55f1bb245a60 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.992+0000 7f8f1290eb80 1 -- 172.21.15.46:0/527573417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 msgr2=0x55f1bb245e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.992+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/527573417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb245e40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00ca540 tx=0x7f8ef0099930 comp rx=0 tx=0).stop 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.992+0000 7f8f1290eb80 1 -- 172.21.15.46:0/527573417 shutdown_connections 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.992+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/527573417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 0x55f1bb257a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.992+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/527573417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb245e40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.992+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/527573417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb246380 0x55f1bb2550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.992+0000 7f8f1290eb80 1 -- 172.21.15.46:0/527573417 >> 172.21.15.46:0/527573417 conn(0x55f1bb1bc220 msgr2=0x55f1bb242020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.992+0000 7f8f1290eb80 1 -- 172.21.15.46:0/527573417 shutdown_connections 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.992+0000 7f8f1290eb80 1 -- 172.21.15.46:0/527573417 wait complete. 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.992+0000 7f8f1290eb80 1 -- start start 2024-03-18T20:48:37.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8f1290eb80 1 Processor -- start 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8f1290eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb2523e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8f1290eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 0x55f1bb252920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8f1290eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb255630 0x55f1bb252e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8f1290eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1bb134d80 con 0x55f1bb246380 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8f1290eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1bb134a20 con 0x55f1bb245a60 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8f1290eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1bb01c6a0 con 0x55f1bb255630 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8f1290eb80 10 client.0 _ll_register_callbacks cb 0x55f1bafa03a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8eff7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 0x55f1bb252920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8eff7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 0x55f1bb252920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54934/0 (socket says 172.21.15.46:54934) 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8eff7fe700 1 -- 172.21.15.46:0/3150366368 learned_addr learned my addr 172.21.15.46:0/3150366368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8f04d16700 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb255630 0x55f1bb252e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.993+0000 7f8efffff700 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb2523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8eff7fe700 1 -- 172.21.15.46:0/3150366368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb255630 msgr2=0x55f1bb252e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8eff7fe700 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb255630 0x55f1bb252e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8eff7fe700 1 -- 172.21.15.46:0/3150366368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 msgr2=0x55f1bb2523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8eff7fe700 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb2523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8eff7fe700 1 -- 172.21.15.46:0/3150366368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef800e370 con 0x55f1bb246380 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8eff7fe700 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 0x55f1bb252920 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8015000 tx=0x7f8ee801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48557d70a7eaf70 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8edf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8edf7fe700 1 -- 172.21.15.46:0/3150366368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800bee0 con 0x55f1bb246380 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8edf7fe700 1 -- 172.21.15.46:0/3150366368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8ee8004c70 con 0x55f1bb246380 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3150366368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1bb239ad0 con 0x55f1bb246380 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8edf7fe700 1 -- 172.21.15.46:0/3150366368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ee8004e30 con 0x55f1bb246380 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8f1290eb80 10 client.19953 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3150366368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f8f001a4220 con 0x55f1bb246380 2024-03-18T20:48:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.994+0000 7f8edf7fe700 1 -- 172.21.15.46:0/3150366368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ee803d020 con 0x55f1bb246380 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.995+0000 7f8edf7fe700 1 -- 172.21.15.46:0/3150366368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8ee8030540 con 0x55f1bb246380 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.995+0000 7f8f1290eb80 10 client.19953 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.995+0000 7f8f1290eb80 10 client.19953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.995+0000 7f8f1290eb80 20 client.19953 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.995+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f1bb265230 0x55f1bb267650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.995+0000 7f8f1290eb80 4 client.19953 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.995+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3150366368 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1bb191700 con 0x55f1bb265230 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.995+0000 7f8efffff700 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f1bb265230 0x55f1bb267650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.995+0000 7f8edf7fe700 10 client.19953 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.995+0000 7f8efffff700 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f1bb265230 0x55f1bb267650 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe664e1fe390cc56 server_cookie=c64e313d55cfce88 in_seq=0 out_seq=0 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8edf7fe700 1 -- 172.21.15.46:0/3150366368 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f1bb191700 con 0x55f1bb265230 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8edf7fe700 10 client.19953 handle_command_reply: tid=0 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8f1290eb80 1 client.19953 shutdown 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3150366368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 msgr2=0x55f1bb252920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 0x55f1bb252920 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8015000 tx=0x7f8ee801a050 comp rx=0 tx=0).stop 2024-03-18T20:48:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3150366368 shutdown_connections 2024-03-18T20:48:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f1bb265230 0x55f1bb267650 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb255630 0x55f1bb252e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb2523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8edf7fe700 0 client.19953 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/3150366368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 0x55f1bb252920 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3150366368 >> 172.21.15.46:0/3150366368 conn(0x55f1bb1bc220 msgr2=0x55f1bb241c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3150366368 shutdown_connections 2024-03-18T20:48:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3150366368 wait complete. 2024-03-18T20:48:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:37.996+0000 7f8f1290eb80 20 client.19953 trim_cache size 0 max 16384 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.010+0000 7f8f1290eb80 1 Processor -- start 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.010+0000 7f8f1290eb80 1 -- start start 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.010+0000 7f8f1290eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb2513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.010+0000 7f8f1290eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb246380 0x55f1bb252c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.010+0000 7f8f1290eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 0x55f1bb24f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.010+0000 7f8f1290eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1bb243440 con 0x55f1bb255630 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.010+0000 7f8f1290eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1bb253a00 con 0x55f1bb245a60 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.010+0000 7f8f1290eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1bb262890 con 0x55f1bb246380 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.010+0000 7f8f04d16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 0x55f1bb24f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.010+0000 7f8f04d16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 0x55f1bb24f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54946/0 (socket says 172.21.15.46:54946) 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.010+0000 7f8f04d16700 1 -- 172.21.15.46:0/3649401569 learned_addr learned my addr 172.21.15.46:0/3649401569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.010+0000 7f8eff7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb246380 0x55f1bb252c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.011+0000 7f8efffff700 1 --2- 172.21.15.46:0/3649401569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb2513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.011+0000 7f8f04d16700 1 -- 172.21.15.46:0/3649401569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb246380 msgr2=0x55f1bb252c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.011+0000 7f8f04d16700 1 --2- 172.21.15.46:0/3649401569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb246380 0x55f1bb252c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.011+0000 7f8f04d16700 1 -- 172.21.15.46:0/3649401569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 msgr2=0x55f1bb2513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.011+0000 7f8f04d16700 1 --2- 172.21.15.46:0/3649401569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb2513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.011+0000 7f8f04d16700 1 -- 172.21.15.46:0/3649401569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1bafc6b60 con 0x55f1bb255630 2024-03-18T20:48:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.011+0000 7f8f04d16700 1 --2- 172.21.15.46:0/3649401569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 0x55f1bb24f660 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800ea60 tx=0x7f8ef8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb69c1682a8de3ac server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.011+0000 7f8ef6ffd700 1 -- 172.21.15.46:0/3649401569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef800e040 con 0x55f1bb255630 2024-03-18T20:48:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.011+0000 7f8ef6ffd700 1 -- 172.21.15.46:0/3649401569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ef800f040 con 0x55f1bb255630 2024-03-18T20:48:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.011+0000 7f8ef6ffd700 1 -- 172.21.15.46:0/3649401569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80077f0 con 0x55f1bb255630 2024-03-18T20:48:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.012+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3649401569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 msgr2=0x55f1bb24f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.012+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/3649401569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 0x55f1bb24f660 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800ea60 tx=0x7f8ef8009920 comp rx=0 tx=0).stop 2024-03-18T20:48:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.012+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3649401569 shutdown_connections 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.012+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/3649401569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb255630 0x55f1bb24f660 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.012+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/3649401569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb2513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.012+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/3649401569 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb246380 0x55f1bb252c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.012+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3649401569 >> 172.21.15.46:0/3649401569 conn(0x55f1bb1bc220 msgr2=0x55f1bb250cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.012+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3649401569 shutdown_connections 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.012+0000 7f8f1290eb80 1 -- 172.21.15.46:0/3649401569 wait complete. 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.012+0000 7f8f1290eb80 1 -- start start 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.012+0000 7f8f1290eb80 1 Processor -- start 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8f1290eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb25a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8f1290eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 0x55f1bb25a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8f1290eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb255630 0x55f1bb2b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8f1290eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1bb2b8560 con 0x55f1bb246380 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8f1290eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1bb2b86a0 con 0x55f1bb245a60 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8f1290eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1bb2b8860 con 0x55f1bb255630 2024-03-18T20:48:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8f1290eb80 10 client.0 _ll_register_callbacks cb 0x7f8ee800ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8eff7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 0x55f1bb25a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8eff7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 0x55f1bb25a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54956/0 (socket says 172.21.15.46:54956) 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8eff7fe700 1 -- 172.21.15.46:0/2831376041 learned_addr learned my addr 172.21.15.46:0/2831376041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8efffff700 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb25a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8f04d16700 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb255630 0x55f1bb2b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8eff7fe700 1 -- 172.21.15.46:0/2831376041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb255630 msgr2=0x55f1bb2b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8eff7fe700 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb255630 0x55f1bb2b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8eff7fe700 1 -- 172.21.15.46:0/2831376041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 msgr2=0x55f1bb25a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8eff7fe700 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb25a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8eff7fe700 1 -- 172.21.15.46:0/2831376041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef800e710 con 0x55f1bb246380 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8eff7fe700 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 0x55f1bb25a540 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009990 tx=0x7f8ee8005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a73bb401e82a18 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8edf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8edf7fe700 1 -- 172.21.15.46:0/2831376041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8030630 con 0x55f1bb246380 2024-03-18T20:48:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.013+0000 7f8edf7fe700 1 -- 172.21.15.46:0/2831376041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8ee803e080 con 0x55f1bb246380 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8f1290eb80 1 -- 172.21.15.46:0/2831376041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ef00a2380 con 0x55f1bb246380 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8edf7fe700 1 -- 172.21.15.46:0/2831376041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ee8004ac0 con 0x55f1bb246380 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8f1290eb80 10 client.19959 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8f1290eb80 1 -- 172.21.15.46:0/2831376041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55f1bb15eea0 con 0x55f1bb246380 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8edf7fe700 1 -- 172.21.15.46:0/2831376041 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ef00a2380 con 0x55f1bb246380 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8edf7fe700 1 -- 172.21.15.46:0/2831376041 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8ee800fd20 con 0x55f1bb246380 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8f1290eb80 10 client.19959 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8f1290eb80 10 client.19959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8f1290eb80 20 client.19959 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f1bb2b9d20 0x55f1bb2bc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8f1290eb80 4 client.19959 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8f1290eb80 1 -- 172.21.15.46:0/2831376041 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f1bafe7cf0 con 0x55f1bb2b9d20 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.014+0000 7f8efffff700 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f1bb2b9d20 0x55f1bb2bc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8edf7fe700 10 client.19959 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:38.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8efffff700 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f1bb2b9d20 0x55f1bb2bc140 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea57f46fbf85793e server_cookie=6d654d74f5ecfb47 in_seq=0 out_seq=0 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8edf7fe700 1 -- 172.21.15.46:0/2831376041 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f1bafe7cf0 con 0x55f1bb2b9d20 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8edf7fe700 10 client.19959 handle_command_reply: tid=0 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8f1290eb80 1 client.19959 shutdown 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8f1290eb80 1 -- 172.21.15.46:0/2831376041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 msgr2=0x55f1bb25a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 0x55f1bb25a540 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009990 tx=0x7f8ee8005a40 comp rx=0 tx=0).stop 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8f1290eb80 1 -- 172.21.15.46:0/2831376041 shutdown_connections 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f1bb2b9d20 0x55f1bb2bc140 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f1bb255630 0x55f1bb2b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f1bb245a60 0x55f1bb25a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8edf7fe700 0 client.19959 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8f1290eb80 1 --2- 172.21.15.46:0/2831376041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1bb246380 0x55f1bb25a540 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8f1290eb80 1 -- 172.21.15.46:0/2831376041 >> 172.21.15.46:0/2831376041 conn(0x55f1bb1bc220 msgr2=0x55f1bb252cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8f1290eb80 1 -- 172.21.15.46:0/2831376041 shutdown_connections 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8f1290eb80 1 -- 172.21.15.46:0/2831376041 wait complete. 2024-03-18T20:48:38.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.015+0000 7f8f1290eb80 20 client.19959 trim_cache size 0 max 16384 2024-03-18T20:48:38.024 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:48:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.016+0000 7f8f06779700 1 -- 172.21.15.46:0/3971798975 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8eec0a1290 msgr2=0x7f8eec0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.016+0000 7f8f06779700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8eec0a1290 0x7f8eec0a3750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800b2e0 tx=0x7f8ee8005960 comp rx=0 tx=0).stop 2024-03-18T20:48:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.016+0000 7f8f06779700 1 -- 172.21.15.46:0/3971798975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f00100f50 msgr2=0x7f8f00197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.016+0000 7f8f06779700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f00100f50 0x7f8f00197200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000e9f0 tx=0x7f8ef000ed00 comp rx=0 tx=0).stop 2024-03-18T20:48:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.016+0000 7f8f06779700 1 -- 172.21.15.46:0/3971798975 shutdown_connections 2024-03-18T20:48:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.016+0000 7f8f06779700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8eec0a1290 0x7f8eec0a3750 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.016+0000 7f8f06779700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f0019ba90 0x7f8f0019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.016+0000 7f8f06779700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f00100f50 0x7f8f00197200 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.016+0000 7f8f06779700 1 --2- 172.21.15.46:0/3971798975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f00106070 0x7f8f00197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.016+0000 7f8f06779700 1 -- 172.21.15.46:0/3971798975 >> 172.21.15.46:0/3971798975 conn(0x7f8f000fc7f0 msgr2=0x7f8f000fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.016+0000 7f8f06779700 1 -- 172.21.15.46:0/3971798975 shutdown_connections 2024-03-18T20:48:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:38.016+0000 7f8f06779700 1 -- 172.21.15.46:0/3971798975 wait complete. 2024-03-18T20:48:38.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:38 smithi155 ceph-mon[48344]: pgmap v391: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 917 KiB/s wr, 103 op/s 2024-03-18T20:48:38.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:38 smithi046 ceph-mon[43495]: pgmap v391: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 917 KiB/s wr, 103 op/s 2024-03-18T20:48:38.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:38 smithi073 ceph-mon[47384]: pgmap v391: 97 pgs: 97 active+clean; 184 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 917 KiB/s wr, 103 op/s 2024-03-18T20:48:38.724 INFO:tasks.workunit.client.0.smithi046.stdout: 1 470685 3.68 MB/sec execute 573 sec latency 1094.065 ms 2024-03-18T20:48:39.035 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T20:48:39.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T20:48:39.035 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T20:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.197+0000 7f102b928700 1 -- 172.21.15.46:0/2177738336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1024101900 msgr2=0x7f1024105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.197+0000 7f102b928700 1 --2- 172.21.15.46:0/2177738336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1024101900 0x7f1024105940 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f1014009990 tx=0x7f1014009ca0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.197+0000 7f102b928700 1 -- 172.21.15.46:0/2177738336 shutdown_connections 2024-03-18T20:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.197+0000 7f102b928700 1 --2- 172.21.15.46:0/2177738336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1024106070 0x7f102410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.197+0000 7f102b928700 1 --2- 172.21.15.46:0/2177738336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1024100f50 0x7f1024101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.197+0000 7f102b928700 1 --2- 172.21.15.46:0/2177738336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1024101900 0x7f1024105940 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.197+0000 7f102b928700 1 -- 172.21.15.46:0/2177738336 >> 172.21.15.46:0/2177738336 conn(0x7f10240fc7f0 msgr2=0x7f10240fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.197+0000 7f102b928700 1 -- 172.21.15.46:0/2177738336 shutdown_connections 2024-03-18T20:48:39.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.197+0000 7f102b928700 1 -- 172.21.15.46:0/2177738336 wait complete. 2024-03-18T20:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f102b928700 1 Processor -- start 2024-03-18T20:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f102b928700 1 -- start start 2024-03-18T20:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f102b928700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1024100f50 0x7f1024197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f102b928700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1024106070 0x7f10241977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f102b928700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f102419baf0 0x7f102419bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f102b928700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102419c570 con 0x7f1024100f50 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f102b928700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102419c6e0 con 0x7f102419baf0 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f102b928700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102419c8a0 con 0x7f1024106070 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f10296c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1024100f50 0x7f1024197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f10296c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1024100f50 0x7f1024197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54972/0 (socket says 172.21.15.46:54972) 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f10296c4700 1 -- 172.21.15.46:0/3785883885 learned_addr learned my addr 172.21.15.46:0/3785883885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f1028ec3700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1024106070 0x7f10241977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.198+0000 7f1029ec5700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f102419baf0 0x7f102419bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.199+0000 7f10296c4700 1 -- 172.21.15.46:0/3785883885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1024106070 msgr2=0x7f10241977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.199+0000 7f10296c4700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1024106070 0x7f10241977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.199+0000 7f10296c4700 1 -- 172.21.15.46:0/3785883885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f102419baf0 msgr2=0x7f102419bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.199+0000 7f10296c4700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f102419baf0 0x7f102419bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.199+0000 7f10296c4700 1 -- 172.21.15.46:0/3785883885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10140095f0 con 0x7f1024100f50 2024-03-18T20:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.199+0000 7f10296c4700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1024100f50 0x7f1024197260 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f102000d980 tx=0x7f102000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.199+0000 7f101a7fc700 1 -- 172.21.15.46:0/3785883885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020004220 con 0x7f1024100f50 2024-03-18T20:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.199+0000 7f102b928700 1 -- 172.21.15.46:0/3785883885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10241a3e00 con 0x7f1024100f50 2024-03-18T20:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.199+0000 7f101a7fc700 1 -- 172.21.15.46:0/3785883885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1020007640 con 0x7f1024100f50 2024-03-18T20:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.199+0000 7f101a7fc700 1 -- 172.21.15.46:0/3785883885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020005690 con 0x7f1024100f50 2024-03-18T20:48:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.199+0000 7f102b928700 1 -- 172.21.15.46:0/3785883885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10241a4220 con 0x7f1024100f50 2024-03-18T20:48:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.201+0000 7f101a7fc700 1 -- 172.21.15.46:0/3785883885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1020004750 con 0x7f1024100f50 2024-03-18T20:48:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.202+0000 7f101a7fc700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f10100a1290 0x7f10100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.202+0000 7f1028ec3700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f10100a1290 0x7f10100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.202+0000 7f101a7fc700 1 -- 172.21.15.46:0/3785883885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f10200ccac0 con 0x7f1024100f50 2024-03-18T20:48:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.202+0000 7f1028ec3700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f10100a1290 0x7f10100a3750 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f101400b5c0 tx=0x7f1014005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.203+0000 7f1037abdb80 1 Processor -- start 2024-03-18T20:48:39.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.203+0000 7f1037abdb80 1 -- start start 2024-03-18T20:48:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.203+0000 7f1037abdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e69ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.203+0000 7f1037abdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e74d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.203+0000 7f1037abdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 0x55c1c0e77690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.203+0000 7f1037abdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c0dc3a10 con 0x55c1c0e69ae0 2024-03-18T20:48:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.203+0000 7f1037abdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c0dc13d0 con 0x55c1c0e6a400 2024-03-18T20:48:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.203+0000 7f1037abdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c0dbccc0 con 0x55c1c0e752a0 2024-03-18T20:48:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f10296c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e69ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f1028ec3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e74d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f10296c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e69ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54988/0 (socket says 172.21.15.46:54988) 2024-03-18T20:48:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f10296c4700 1 -- 172.21.15.46:0/3806432926 learned_addr learned my addr 172.21.15.46:0/3806432926 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:39.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f1028ec3700 1 --2- 172.21.15.46:0/3806432926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e74d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51480/0 (socket says 172.21.15.46:51480) 2024-03-18T20:48:39.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f1029ec5700 1 --2- 172.21.15.46:0/3806432926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 0x55c1c0e77690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f10296c4700 1 -- 172.21.15.46:0/3806432926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 msgr2=0x55c1c0e77690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f10296c4700 1 --2- 172.21.15.46:0/3806432926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 0x55c1c0e77690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f10296c4700 1 -- 172.21.15.46:0/3806432926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 msgr2=0x55c1c0e74d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f10296c4700 1 --2- 172.21.15.46:0/3806432926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e74d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f10296c4700 1 -- 172.21.15.46:0/3806432926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1c0c3e430 con 0x55c1c0e69ae0 2024-03-18T20:48:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f1028ec3700 1 --2- 172.21.15.46:0/3806432926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e74d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:39.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f10296c4700 1 --2- 172.21.15.46:0/3806432926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e69ec0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f1020099b40 tx=0x7f1020019f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248aa39101c723b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f100f7fe700 1 -- 172.21.15.46:0/3806432926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102009d9e0 con 0x55c1c0e69ae0 2024-03-18T20:48:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f100f7fe700 1 -- 172.21.15.46:0/3806432926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10200a2030 con 0x55c1c0e69ae0 2024-03-18T20:48:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.204+0000 7f100f7fe700 1 -- 172.21.15.46:0/3806432926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020099300 con 0x55c1c0e69ae0 2024-03-18T20:48:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.205+0000 7f1037abdb80 1 -- 172.21.15.46:0/3806432926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 msgr2=0x55c1c0e69ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.205+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3806432926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e69ec0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f1020099b40 tx=0x7f1020019f20 comp rx=0 tx=0).stop 2024-03-18T20:48:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.205+0000 7f1037abdb80 1 -- 172.21.15.46:0/3806432926 shutdown_connections 2024-03-18T20:48:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.205+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3806432926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e69ec0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.205+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3806432926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 0x55c1c0e77690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.205+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3806432926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e74d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.205+0000 7f1037abdb80 1 -- 172.21.15.46:0/3806432926 >> 172.21.15.46:0/3806432926 conn(0x55c1c0de02a0 msgr2=0x55c1c0e660a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.205+0000 7f1037abdb80 1 -- 172.21.15.46:0/3806432926 shutdown_connections 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.205+0000 7f1037abdb80 1 -- 172.21.15.46:0/3806432926 wait complete. 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f1037abdb80 1 -- start start 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f1037abdb80 1 Processor -- start 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f1037abdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e74cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f1037abdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e72040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f1037abdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e752a0 0x55c1c0e726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f1037abdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c0d58e00 con 0x55c1c0e752a0 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f1037abdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c0d58aa0 con 0x55c1c0e6a400 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f1037abdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c0c40720 con 0x55c1c0e69ae0 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f1037abdb80 10 client.0 _ll_register_callbacks cb 0x55c1c0bc4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:39.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f1029ec5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e752a0 0x55c1c0e726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f1029ec5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e752a0 0x55c1c0e726e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55004/0 (socket says 172.21.15.46:55004) 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f1029ec5700 1 -- 172.21.15.46:0/3262672652 learned_addr learned my addr 172.21.15.46:0/3262672652 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.206+0000 7f10296c4700 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e74cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f1028ec3700 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e72040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f1029ec5700 1 -- 172.21.15.46:0/3262672652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e69ae0 msgr2=0x55c1c0e74cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f1029ec5700 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e74cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f1029ec5700 1 -- 172.21.15.46:0/3262672652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 msgr2=0x55c1c0e72040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f1029ec5700 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e72040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f1029ec5700 1 -- 172.21.15.46:0/3262672652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102410e460 con 0x55c1c0e752a0 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f10296c4700 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e74cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f1029ec5700 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e752a0 0x55c1c0e726e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f101c009fb0 tx=0x7f101c004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7020726683ed4063 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f100cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f100cff9700 1 -- 172.21.15.46:0/3262672652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c007910 con 0x55c1c0e752a0 2024-03-18T20:48:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f100cff9700 1 -- 172.21.15.46:0/3262672652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f101c007a70 con 0x55c1c0e752a0 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f1037abdb80 1 -- 172.21.15.46:0/3262672652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1c0e5db50 con 0x55c1c0e752a0 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f100cff9700 1 -- 172.21.15.46:0/3262672652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f101c004920 con 0x55c1c0e752a0 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f1037abdb80 10 client.19971 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.207+0000 7f1037abdb80 1 -- 172.21.15.46:0/3262672652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x7f10241a4220 con 0x55c1c0e752a0 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.208+0000 7f100cff9700 1 -- 172.21.15.46:0/3262672652 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c1c0e5db50 con 0x55c1c0e752a0 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.208+0000 7f100cff9700 1 -- 172.21.15.46:0/3262672652 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f101c014040 con 0x55c1c0e752a0 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.208+0000 7f1037abdb80 10 client.19971 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.208+0000 7f1037abdb80 10 client.19971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.208+0000 7f1037abdb80 20 client.19971 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.208+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c1c0e84e10 0x55c1c0e87230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.208+0000 7f1037abdb80 4 client.19971 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.208+0000 7f1037abdb80 1 -- 172.21.15.46:0/3262672652 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1c0db5780 con 0x55c1c0e84e10 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.208+0000 7f10296c4700 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c1c0e84e10 0x55c1c0e87230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f100cff9700 10 client.19971 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f10296c4700 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c1c0e84e10 0x55c1c0e87230 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=710666c972748eed server_cookie=727a9d85560e96c3 in_seq=0 out_seq=0 2024-03-18T20:48:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f100cff9700 1 -- 172.21.15.46:0/3262672652 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c1c0db5780 con 0x55c1c0e84e10 2024-03-18T20:48:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f100cff9700 10 client.19971 handle_command_reply: tid=0 2024-03-18T20:48:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f1037abdb80 1 client.19971 shutdown 2024-03-18T20:48:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f1037abdb80 1 -- 172.21.15.46:0/3262672652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e752a0 msgr2=0x55c1c0e726e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e752a0 0x55c1c0e726e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f101c009fb0 tx=0x7f101c004320 comp rx=0 tx=0).stop 2024-03-18T20:48:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f1037abdb80 1 -- 172.21.15.46:0/3262672652 shutdown_connections 2024-03-18T20:48:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c1c0e84e10 0x55c1c0e87230 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e752a0 0x55c1c0e726e0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e74cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f100cff9700 0 client.19971 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3262672652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e72040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.209+0000 7f1037abdb80 1 -- 172.21.15.46:0/3262672652 >> 172.21.15.46:0/3262672652 conn(0x55c1c0de02a0 msgr2=0x55c1c0e65c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.210+0000 7f1037abdb80 1 -- 172.21.15.46:0/3262672652 shutdown_connections 2024-03-18T20:48:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.210+0000 7f1037abdb80 1 -- 172.21.15.46:0/3262672652 wait complete. 2024-03-18T20:48:39.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.210+0000 7f1037abdb80 20 client.19971 trim_cache size 0 max 16384 2024-03-18T20:48:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.227+0000 7f1037abdb80 1 Processor -- start 2024-03-18T20:48:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1037abdb80 1 -- start start 2024-03-18T20:48:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1037abdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e6ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1037abdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e6a400 0x55c1c0e7bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1037abdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 0x55c1c0e80bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1037abdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c0e6adf0 con 0x55c1c0e6a400 2024-03-18T20:48:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1037abdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c0e6b690 con 0x55c1c0e69ae0 2024-03-18T20:48:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1037abdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c0e6b4f0 con 0x55c1c0e752a0 2024-03-18T20:48:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1028ec3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e6a400 0x55c1c0e7bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1028ec3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e6a400 0x55c1c0e7bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55020/0 (socket says 172.21.15.46:55020) 2024-03-18T20:48:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1028ec3700 1 -- 172.21.15.46:0/3276708197 learned_addr learned my addr 172.21.15.46:0/3276708197 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1028ec3700 1 -- 172.21.15.46:0/3276708197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 msgr2=0x55c1c0e80bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1028ec3700 1 --2- 172.21.15.46:0/3276708197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 0x55c1c0e80bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1028ec3700 1 -- 172.21.15.46:0/3276708197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e69ae0 msgr2=0x55c1c0e6ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f10296c4700 1 --2- 172.21.15.46:0/3276708197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e6ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1028ec3700 1 --2- 172.21.15.46:0/3276708197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e6ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f1028ec3700 1 -- 172.21.15.46:0/3276708197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1c0beabe0 con 0x55c1c0e6a400 2024-03-18T20:48:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.228+0000 7f10296c4700 1 --2- 172.21.15.46:0/3276708197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e6ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:48:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.229+0000 7f1028ec3700 1 --2- 172.21.15.46:0/3276708197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e6a400 0x55c1c0e7bd50 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f1014009990 tx=0x7f1014020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35ce92be430b67c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.229+0000 7f100ffff700 1 -- 172.21.15.46:0/3276708197 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014004500 con 0x55c1c0e6a400 2024-03-18T20:48:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.229+0000 7f100ffff700 1 -- 172.21.15.46:0/3276708197 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f101401e040 con 0x55c1c0e6a400 2024-03-18T20:48:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.229+0000 7f100ffff700 1 -- 172.21.15.46:0/3276708197 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400f8f0 con 0x55c1c0e6a400 2024-03-18T20:48:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.229+0000 7f1037abdb80 1 -- 172.21.15.46:0/3276708197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e6a400 msgr2=0x55c1c0e7bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.229+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3276708197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e6a400 0x55c1c0e7bd50 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f1014009990 tx=0x7f1014020000 comp rx=0 tx=0).stop 2024-03-18T20:48:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.229+0000 7f1037abdb80 1 -- 172.21.15.46:0/3276708197 shutdown_connections 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.229+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3276708197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 0x55c1c0e80bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.229+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3276708197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e6ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.229+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3276708197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e6a400 0x55c1c0e7bd50 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.229+0000 7f1037abdb80 1 -- 172.21.15.46:0/3276708197 >> 172.21.15.46:0/3276708197 conn(0x55c1c0de02a0 msgr2=0x55c1c0e804a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.230+0000 7f1037abdb80 1 -- 172.21.15.46:0/3276708197 shutdown_connections 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.230+0000 7f1037abdb80 1 -- 172.21.15.46:0/3276708197 wait complete. 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.230+0000 7f1037abdb80 1 -- start start 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.230+0000 7f1037abdb80 1 Processor -- start 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.230+0000 7f1037abdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e7a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.230+0000 7f1037abdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e7acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.230+0000 7f1037abdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 0x55c1c0e7b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.230+0000 7f1037abdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c0ed8d40 con 0x55c1c0e69ae0 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.230+0000 7f1037abdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c0ed8eb0 con 0x55c1c0e6a400 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.230+0000 7f1037abdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c0ed9070 con 0x55c1c0e752a0 2024-03-18T20:48:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f1037abdb80 10 client.0 _ll_register_callbacks cb 0x7f101c009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f10296c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e7a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f10296c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e7a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55028/0 (socket says 172.21.15.46:55028) 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f10296c4700 1 -- 172.21.15.46:0/3578728770 learned_addr learned my addr 172.21.15.46:0/3578728770 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f1028ec3700 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e7acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f1029ec5700 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 0x55c1c0e7b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f10296c4700 1 -- 172.21.15.46:0/3578728770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 msgr2=0x55c1c0e7b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f10296c4700 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 0x55c1c0e7b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f10296c4700 1 -- 172.21.15.46:0/3578728770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 msgr2=0x55c1c0e7acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f10296c4700 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e7acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f10296c4700 1 -- 172.21.15.46:0/3578728770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1014005510 con 0x55c1c0e69ae0 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f10296c4700 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e7a790 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f10200ca770 tx=0x7f102009d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52117b56ab2c4548 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f100cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f100cff9700 1 -- 172.21.15.46:0/3578728770 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10200992a0 con 0x55c1c0e69ae0 2024-03-18T20:48:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f100cff9700 1 -- 172.21.15.46:0/3578728770 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f1020065330 con 0x55c1c0e69ae0 2024-03-18T20:48:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f1037abdb80 1 -- 172.21.15.46:0/3578728770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10200a2030 con 0x55c1c0e69ae0 2024-03-18T20:48:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f100cff9700 1 -- 172.21.15.46:0/3578728770 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1020099980 con 0x55c1c0e69ae0 2024-03-18T20:48:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f1037abdb80 10 client.19977 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.231+0000 7f1037abdb80 1 -- 172.21.15.46:0/3578728770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x55c1c0d82f20 con 0x55c1c0e69ae0 2024-03-18T20:48:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.232+0000 7f100cff9700 1 -- 172.21.15.46:0/3578728770 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10200a2030 con 0x55c1c0e69ae0 2024-03-18T20:48:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.232+0000 7f100cff9700 1 -- 172.21.15.46:0/3578728770 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f10200a1ba0 con 0x55c1c0e69ae0 2024-03-18T20:48:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.232+0000 7f1037abdb80 10 client.19977 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.232+0000 7f1037abdb80 10 client.19977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.232+0000 7f1037abdb80 20 client.19977 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.232+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c1c0eda530 0x55c1c0edc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.232+0000 7f1037abdb80 4 client.19977 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T20:48:39.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.232+0000 7f1037abdb80 1 -- 172.21.15.46:0/3578728770 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c1c0c0bd70 con 0x55c1c0eda530 2024-03-18T20:48:39.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.232+0000 7f1028ec3700 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c1c0eda530 0x55c1c0edc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.233+0000 7f100cff9700 10 client.19977 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:39.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.233+0000 7f1028ec3700 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c1c0eda530 0x55c1c0edc950 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70b4534f5ac37f55 server_cookie=1e7f2451e056d6a3 in_seq=0 out_seq=0 2024-03-18T20:48:39.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f100cff9700 1 -- 172.21.15.46:0/3578728770 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c1c0c0bd70 con 0x55c1c0eda530 2024-03-18T20:48:39.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f100cff9700 10 client.19977 handle_command_reply: tid=0 2024-03-18T20:48:39.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f1037abdb80 1 client.19977 shutdown 2024-03-18T20:48:39.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f1037abdb80 1 -- 172.21.15.46:0/3578728770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 msgr2=0x55c1c0e7a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e7a790 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f10200ca770 tx=0x7f102009d9e0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f1037abdb80 1 -- 172.21.15.46:0/3578728770 shutdown_connections 2024-03-18T20:48:39.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c1c0eda530 0x55c1c0edc950 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c1c0e69ae0 0x55c1c0e7a790 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1c0e752a0 0x55c1c0e7b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f100cff9700 0 client.19977 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:39.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f1037abdb80 1 --2- 172.21.15.46:0/3578728770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1c0e6a400 0x55c1c0e7acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f1037abdb80 1 -- 172.21.15.46:0/3578728770 >> 172.21.15.46:0/3578728770 conn(0x55c1c0de02a0 msgr2=0x55c1c0e70540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:39.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f1037abdb80 1 -- 172.21.15.46:0/3578728770 shutdown_connections 2024-03-18T20:48:39.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.234+0000 7f1037abdb80 1 -- 172.21.15.46:0/3578728770 wait complete. 2024-03-18T20:48:39.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f1037abdb80 20 client.19977 trim_cache size 0 max 16384 2024-03-18T20:48:39.250 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:48:39.250 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-18T20:48:39.250 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "6fc7c035-069c-4d2a-aa7e-c7baadd9251b", 2024-03-18T20:48:39.251 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-18T20:48:39.251 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:48:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f102b928700 1 -- 172.21.15.46:0/3785883885 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f10100a1290 msgr2=0x7f10100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f102b928700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f10100a1290 0x7f10100a3750 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f101400b5c0 tx=0x7f1014005960 comp rx=0 tx=0).stop 2024-03-18T20:48:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f102b928700 1 -- 172.21.15.46:0/3785883885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1024100f50 msgr2=0x7f1024197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f102b928700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1024100f50 0x7f1024197260 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f102000d980 tx=0x7f102000dc90 comp rx=0 tx=0).stop 2024-03-18T20:48:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f102b928700 1 -- 172.21.15.46:0/3785883885 shutdown_connections 2024-03-18T20:48:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f102b928700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f10100a1290 0x7f10100a3750 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f102b928700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1024100f50 0x7f1024197260 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f102b928700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f102419baf0 0x7f102419bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f102b928700 1 --2- 172.21.15.46:0/3785883885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1024106070 0x7f10241977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f102b928700 1 -- 172.21.15.46:0/3785883885 >> 172.21.15.46:0/3785883885 conn(0x7f10240fc7f0 msgr2=0x7f10240fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f102b928700 1 -- 172.21.15.46:0/3785883885 shutdown_connections 2024-03-18T20:48:39.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.235+0000 7f102b928700 1 -- 172.21.15.46:0/3785883885 wait complete. 2024-03-18T20:48:39.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:48:39.254 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:48:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.399+0000 7fdc9a95f700 1 -- 172.21.15.46:0/2133485758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc941060b0 msgr2=0x7fdc9410dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.399+0000 7fdc9a95f700 1 --2- 172.21.15.46:0/2133485758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc941060b0 0x7fdc9410dc30 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00b0d0 tx=0x7fdc8c00b3e0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.399+0000 7fdc9a95f700 1 -- 172.21.15.46:0/2133485758 shutdown_connections 2024-03-18T20:48:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.399+0000 7fdc9a95f700 1 --2- 172.21.15.46:0/2133485758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc941060b0 0x7fdc9410dc30 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.399+0000 7fdc9a95f700 1 --2- 172.21.15.46:0/2133485758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc94069280 0x7fdc94069660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.399+0000 7fdc9a95f700 1 --2- 172.21.15.46:0/2133485758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc94069c30 0x7fdc94105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.399+0000 7fdc9a95f700 1 -- 172.21.15.46:0/2133485758 >> 172.21.15.46:0/2133485758 conn(0x7fdc94076d80 msgr2=0x7fdc94077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.400+0000 7fdc9a95f700 1 -- 172.21.15.46:0/2133485758 shutdown_connections 2024-03-18T20:48:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.400+0000 7fdc9a95f700 1 -- 172.21.15.46:0/2133485758 wait complete. 2024-03-18T20:48:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.400+0000 7fdc9a95f700 1 Processor -- start 2024-03-18T20:48:39.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.400+0000 7fdc9a95f700 1 -- start start 2024-03-18T20:48:39.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.400+0000 7fdc9a95f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc94069280 0x7fdc940ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.400+0000 7fdc9a95f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc941060b0 0x7fdc941001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.400+0000 7fdc9a95f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc941006f0 0x7fdc941a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.400+0000 7fdc9a95f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc941a5a90 con 0x7fdc94069280 2024-03-18T20:48:39.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.400+0000 7fdc9a95f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc941a5c00 con 0x7fdc941060b0 2024-03-18T20:48:39.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.400+0000 7fdc9a95f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc941a5dc0 con 0x7fdc941006f0 2024-03-18T20:48:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc94069280 0x7fdc940ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc94069280 0x7fdc940ffc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55042/0 (socket says 172.21.15.46:55042) 2024-03-18T20:48:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc93fff700 1 -- 172.21.15.46:0/1909384587 learned_addr learned my addr 172.21.15.46:0/1909384587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc8bfff700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc941060b0 0x7fdc941001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc98efc700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc941006f0 0x7fdc941a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc93fff700 1 -- 172.21.15.46:0/1909384587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc941006f0 msgr2=0x7fdc941a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc93fff700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc941006f0 0x7fdc941a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc93fff700 1 -- 172.21.15.46:0/1909384587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc941060b0 msgr2=0x7fdc941001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc93fff700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc941060b0 0x7fdc941001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc93fff700 1 -- 172.21.15.46:0/1909384587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c009d00 con 0x7fdc94069280 2024-03-18T20:48:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc8bfff700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc941060b0 0x7fdc941001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc93fff700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc94069280 0x7fdc940ffc70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400e9a0 tx=0x7fdc8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc91ffb700 1 -- 172.21.15.46:0/1909384587 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84009790 con 0x7fdc94069280 2024-03-18T20:48:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc9a95f700 1 -- 172.21.15.46:0/1909384587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc941a60a0 con 0x7fdc94069280 2024-03-18T20:48:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc91ffb700 1 -- 172.21.15.46:0/1909384587 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8400cbd0 con 0x7fdc94069280 2024-03-18T20:48:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc91ffb700 1 -- 172.21.15.46:0/1909384587 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84018680 con 0x7fdc94069280 2024-03-18T20:48:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.401+0000 7fdc9a95f700 1 -- 172.21.15.46:0/1909384587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc941a65b0 con 0x7fdc94069280 2024-03-18T20:48:39.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.404+0000 7fdc91ffb700 1 -- 172.21.15.46:0/1909384587 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fdc8400cd40 con 0x7fdc94069280 2024-03-18T20:48:39.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.405+0000 7fdc91ffb700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdc800a1290 0x7fdc800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.405+0000 7fdc91ffb700 1 -- 172.21.15.46:0/1909384587 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fdc84014070 con 0x7fdc94069280 2024-03-18T20:48:39.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.405+0000 7fdc8bfff700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdc800a1290 0x7fdc800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.406+0000 7fdc8bfff700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdc800a1290 0x7fdc800a3750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdc941013a0 tx=0x7fdc7c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.406+0000 7fdca6af4b80 1 Processor -- start 2024-03-18T20:48:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.406+0000 7fdca6af4b80 1 -- start start 2024-03-18T20:48:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.406+0000 7fdca6af4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1117a60 0x557ac1117e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.406+0000 7fdca6af4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1118380 0x557ac1122ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.406+0000 7fdca6af4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1123220 0x557ac1125610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.406+0000 7fdca6af4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ac1071990 con 0x557ac1123220 2024-03-18T20:48:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.406+0000 7fdca6af4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ac106f350 con 0x557ac1118380 2024-03-18T20:48:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.406+0000 7fdca6af4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ac106ac40 con 0x557ac1117a60 2024-03-18T20:48:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1118380 0x557ac1122ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc98efc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1123220 0x557ac1125610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc98efc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1123220 0x557ac1125610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55056/0 (socket says 172.21.15.46:55056) 2024-03-18T20:48:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc98efc700 1 -- 172.21.15.46:0/2425692808 learned_addr learned my addr 172.21.15.46:0/2425692808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc93fff700 1 --2- 172.21.15.46:0/2425692808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1117a60 0x557ac1117e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc98efc700 1 -- 172.21.15.46:0/2425692808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1117a60 msgr2=0x557ac1117e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc98efc700 1 --2- 172.21.15.46:0/2425692808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1117a60 0x557ac1117e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc98efc700 1 -- 172.21.15.46:0/2425692808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1118380 msgr2=0x557ac1122ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc98efc700 1 --2- 172.21.15.46:0/2425692808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1118380 0x557ac1122ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc98efc700 1 -- 172.21.15.46:0/2425692808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ac0eec3b0 con 0x557ac1123220 2024-03-18T20:48:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc98efc700 1 --2- 172.21.15.46:0/2425692808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1123220 0x557ac1125610 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009c70 tx=0x7fdc8c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c61c703fd2240f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc8a7fc700 1 -- 172.21.15.46:0/2425692808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c00e040 con 0x557ac1123220 2024-03-18T20:48:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc8a7fc700 1 -- 172.21.15.46:0/2425692808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8c007d00 con 0x557ac1123220 2024-03-18T20:48:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.407+0000 7fdc8a7fc700 1 -- 172.21.15.46:0/2425692808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c007700 con 0x557ac1123220 2024-03-18T20:48:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.408+0000 7fdca6af4b80 1 -- 172.21.15.46:0/2425692808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1123220 msgr2=0x557ac1125610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.408+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/2425692808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1123220 0x557ac1125610 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009c70 tx=0x7fdc8c0092e0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.408+0000 7fdca6af4b80 1 -- 172.21.15.46:0/2425692808 shutdown_connections 2024-03-18T20:48:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.408+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/2425692808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1123220 0x557ac1125610 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.408+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/2425692808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1117a60 0x557ac1117e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.408+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/2425692808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1118380 0x557ac1122ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.408+0000 7fdca6af4b80 1 -- 172.21.15.46:0/2425692808 >> 172.21.15.46:0/2425692808 conn(0x557ac108e220 msgr2=0x557ac1114020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.408+0000 7fdca6af4b80 1 -- 172.21.15.46:0/2425692808 shutdown_connections 2024-03-18T20:48:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.408+0000 7fdca6af4b80 1 -- 172.21.15.46:0/2425692808 wait complete. 2024-03-18T20:48:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.409+0000 7fdca6af4b80 1 -- start start 2024-03-18T20:48:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.409+0000 7fdca6af4b80 1 Processor -- start 2024-03-18T20:48:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.409+0000 7fdca6af4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1117a60 0x557ac1122c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdca6af4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 0x557ac111ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdca6af4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1123220 0x557ac1120700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdca6af4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ac1006d80 con 0x557ac1118380 2024-03-18T20:48:39.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdca6af4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ac1006a20 con 0x557ac1123220 2024-03-18T20:48:39.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdca6af4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ac0eee6a0 con 0x557ac1117a60 2024-03-18T20:48:39.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdca6af4b80 10 client.0 _ll_register_callbacks cb 0x557ac0e723a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 0x557ac111ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 0x557ac111ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55064/0 (socket says 172.21.15.46:55064) 2024-03-18T20:48:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc8bfff700 1 -- 172.21.15.46:0/973253840 learned_addr learned my addr 172.21.15.46:0/973253840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc98efc700 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1123220 0x557ac1120700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc8bfff700 1 -- 172.21.15.46:0/973253840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1117a60 msgr2=0x557ac1122c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc93fff700 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1117a60 0x557ac1122c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc8bfff700 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1117a60 0x557ac1122c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc8bfff700 1 -- 172.21.15.46:0/973253840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1123220 msgr2=0x557ac1120700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc8bfff700 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1123220 0x557ac1120700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc8bfff700 1 -- 172.21.15.46:0/973253840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9410e470 con 0x557ac1118380 2024-03-18T20:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc93fff700 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1117a60 0x557ac1122c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc8bfff700 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 0x557ac111ffc0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00eb90 tx=0x7fdc7c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c6f53226fe2ba3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:39.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc737fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:39.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc737fe700 1 -- 172.21.15.46:0/973253840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c01fd60 con 0x557ac1118380 2024-03-18T20:48:39.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdc737fe700 1 -- 172.21.15.46:0/973253840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fdc7c0273e0 con 0x557ac1118380 2024-03-18T20:48:39.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.410+0000 7fdca6af4b80 1 -- 172.21.15.46:0/973253840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ac110bad0 con 0x557ac1118380 2024-03-18T20:48:39.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.411+0000 7fdc737fe700 1 -- 172.21.15.46:0/973253840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc7c0275a0 con 0x557ac1118380 2024-03-18T20:48:39.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.411+0000 7fdca6af4b80 10 client.19989 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.411+0000 7fdca6af4b80 1 -- 172.21.15.46:0/973253840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x557ac0e98b60 con 0x557ac1118380 2024-03-18T20:48:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.411+0000 7fdc737fe700 1 -- 172.21.15.46:0/973253840 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc7c034080 con 0x557ac1118380 2024-03-18T20:48:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.411+0000 7fdc737fe700 1 -- 172.21.15.46:0/973253840 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdc7c0287a0 con 0x557ac1118380 2024-03-18T20:48:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.411+0000 7fdca6af4b80 10 client.19989 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.411+0000 7fdca6af4b80 10 client.19989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.411+0000 7fdca6af4b80 20 client.19989 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.411+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557ac1132f10 0x557ac1135330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.411+0000 7fdca6af4b80 4 client.19989 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.411+0000 7fdca6af4b80 1 -- 172.21.15.46:0/973253840 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ac1063700 con 0x557ac1132f10 2024-03-18T20:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.411+0000 7fdc93fff700 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557ac1132f10 0x557ac1135330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.412+0000 7fdc737fe700 10 client.19989 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.412+0000 7fdc93fff700 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557ac1132f10 0x557ac1135330 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53a7383ec054af4 server_cookie=4431d61eb6faccdd in_seq=0 out_seq=0 2024-03-18T20:48:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.412+0000 7fdc737fe700 1 -- 172.21.15.46:0/973253840 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557ac1063700 con 0x557ac1132f10 2024-03-18T20:48:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.412+0000 7fdc737fe700 10 client.19989 handle_command_reply: tid=0 2024-03-18T20:48:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.412+0000 7fdca6af4b80 1 client.19989 shutdown 2024-03-18T20:48:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.413+0000 7fdca6af4b80 1 -- 172.21.15.46:0/973253840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 msgr2=0x557ac111ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.413+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 0x557ac111ffc0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00eb90 tx=0x7fdc7c0047c0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.413+0000 7fdca6af4b80 1 -- 172.21.15.46:0/973253840 shutdown_connections 2024-03-18T20:48:39.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.413+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557ac1132f10 0x557ac1135330 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.413+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1123220 0x557ac1120700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.413+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1117a60 0x557ac1122c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.413+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/973253840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 0x557ac111ffc0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.413+0000 7fdc737fe700 0 client.19989 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:39.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.413+0000 7fdca6af4b80 1 -- 172.21.15.46:0/973253840 >> 172.21.15.46:0/973253840 conn(0x557ac108e220 msgr2=0x557ac1113c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:39.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.413+0000 7fdca6af4b80 1 -- 172.21.15.46:0/973253840 shutdown_connections 2024-03-18T20:48:39.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.413+0000 7fdca6af4b80 1 -- 172.21.15.46:0/973253840 wait complete. 2024-03-18T20:48:39.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.413+0000 7fdca6af4b80 20 client.19989 trim_cache size 0 max 16384 2024-03-18T20:48:39.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.432+0000 7fdca6af4b80 1 Processor -- start 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.432+0000 7fdca6af4b80 1 -- start start 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdca6af4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1117a60 0x557ac1129df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdca6af4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 0x557ac111b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdca6af4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1123220 0x557ac111b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdca6af4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ac110e210 con 0x557ac1118380 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdca6af4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ac112f930 con 0x557ac1117a60 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdca6af4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ac11356f0 con 0x557ac1123220 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 0x557ac111b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 0x557ac111b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55078/0 (socket says 172.21.15.46:55078) 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc8bfff700 1 -- 172.21.15.46:0/3854089114 learned_addr learned my addr 172.21.15.46:0/3854089114 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc93fff700 1 --2- 172.21.15.46:0/3854089114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1117a60 0x557ac1129df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc98efc700 1 --2- 172.21.15.46:0/3854089114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1123220 0x557ac111b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc8bfff700 1 -- 172.21.15.46:0/3854089114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1123220 msgr2=0x557ac111b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc8bfff700 1 --2- 172.21.15.46:0/3854089114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1123220 0x557ac111b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc8bfff700 1 -- 172.21.15.46:0/3854089114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1117a60 msgr2=0x557ac1129df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc8bfff700 1 --2- 172.21.15.46:0/3854089114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1117a60 0x557ac1129df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc8bfff700 1 -- 172.21.15.46:0/3854089114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ac1030ea0 con 0x557ac1118380 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc93fff700 1 --2- 172.21.15.46:0/3854089114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1117a60 0x557ac1129df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc98efc700 1 --2- 172.21.15.46:0/3854089114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1123220 0x557ac111b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.433+0000 7fdc8bfff700 1 --2- 172.21.15.46:0/3854089114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 0x557ac111b320 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00ef90 tx=0x7fdc7c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5fd91515cb2edc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.434+0000 7fdc8affd700 1 -- 172.21.15.46:0/3854089114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c01fd60 con 0x557ac1118380 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.434+0000 7fdc8affd700 1 -- 172.21.15.46:0/3854089114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ac110bad0 con 0x557ac1118380 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.434+0000 7fdc8affd700 1 -- 172.21.15.46:0/3854089114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c030ad0 con 0x557ac1118380 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.434+0000 7fdca6af4b80 1 -- 172.21.15.46:0/3854089114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 msgr2=0x557ac111b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.434+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/3854089114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 0x557ac111b320 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c00ef90 tx=0x7fdc7c0046b0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.434+0000 7fdca6af4b80 1 -- 172.21.15.46:0/3854089114 shutdown_connections 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.434+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/3854089114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1123220 0x557ac111b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.434+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/3854089114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1117a60 0x557ac1129df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.434+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/3854089114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1118380 0x557ac111b320 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.434+0000 7fdca6af4b80 1 -- 172.21.15.46:0/3854089114 >> 172.21.15.46:0/3854089114 conn(0x557ac108e220 msgr2=0x557ac1122890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.435+0000 7fdca6af4b80 1 -- 172.21.15.46:0/3854089114 shutdown_connections 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.435+0000 7fdca6af4b80 1 -- 172.21.15.46:0/3854089114 wait complete. 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.435+0000 7fdca6af4b80 1 -- start start 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.435+0000 7fdca6af4b80 1 Processor -- start 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.435+0000 7fdca6af4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1117a60 0x557ac1129dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.435+0000 7fdca6af4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1118380 0x557ac11283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.435+0000 7fdca6af4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1123220 0x557ac1128910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.435+0000 7fdca6af4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ac1129050 con 0x557ac1117a60 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.435+0000 7fdca6af4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ac11291c0 con 0x557ac1118380 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.435+0000 7fdca6af4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ac1186d50 con 0x557ac1123220 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdca6af4b80 10 client.0 _ll_register_callbacks cb 0x7fdc7c00e980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:39.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1117a60 0x557ac1129dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1118380 0x557ac11283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1117a60 0x557ac1129dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55082/0 (socket says 172.21.15.46:55082) 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc93fff700 1 -- 172.21.15.46:0/1519339473 learned_addr learned my addr 172.21.15.46:0/1519339473 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc98efc700 1 --2- 172.21.15.46:0/1519339473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1123220 0x557ac1128910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc93fff700 1 -- 172.21.15.46:0/1519339473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1123220 msgr2=0x557ac1128910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc93fff700 1 --2- 172.21.15.46:0/1519339473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1123220 0x557ac1128910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc93fff700 1 -- 172.21.15.46:0/1519339473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1118380 msgr2=0x557ac11283d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc93fff700 1 --2- 172.21.15.46:0/1519339473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1118380 0x557ac11283d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc93fff700 1 -- 172.21.15.46:0/1519339473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc941a65b0 con 0x557ac1117a60 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc93fff700 1 --2- 172.21.15.46:0/1519339473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1117a60 0x557ac1129dd0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c5b0 tx=0x7fdc840963a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90875f418ce26d3a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc737fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc737fe700 1 -- 172.21.15.46:0/1519339473 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc840a24e0 con 0x557ac1117a60 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdc737fe700 1 -- 172.21.15.46:0/1519339473 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fdc840b8300 con 0x557ac1117a60 2024-03-18T20:48:39.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.436+0000 7fdca6af4b80 1 -- 172.21.15.46:0/1519339473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc8c007d00 con 0x557ac1117a60 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.437+0000 7fdc737fe700 1 -- 172.21.15.46:0/1519339473 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc8409a2c0 con 0x557ac1117a60 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.437+0000 7fdca6af4b80 10 client.19995 fetch_fsmap learned FSMap version 55 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.437+0000 7fdca6af4b80 1 -- 172.21.15.46:0/1519339473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x557ac0e98b60 con 0x557ac1117a60 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.437+0000 7fdc737fe700 1 -- 172.21.15.46:0/1519339473 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc8c007d00 con 0x557ac1117a60 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.437+0000 7fdc737fe700 1 -- 172.21.15.46:0/1519339473 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 55) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdc8407d020 con 0x557ac1117a60 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.437+0000 7fdca6af4b80 10 client.19995 fetch_fsmap finished waiting for FSMap version 55 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.437+0000 7fdca6af4b80 10 client.19995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.437+0000 7fdca6af4b80 20 client.19995 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.437+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/1519339473 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557ac1188590 0x557ac118a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.437+0000 7fdca6af4b80 4 client.19995 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.437+0000 7fdca6af4b80 1 -- 172.21.15.46:0/1519339473 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557ac0eb9cf0 con 0x557ac1188590 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.437+0000 7fdc8bfff700 1 --2- 172.21.15.46:0/1519339473 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557ac1188590 0x557ac118a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdc8bfff700 1 --2- 172.21.15.46:0/1519339473 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557ac1188590 0x557ac118a9b0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c19f4d667d0090 server_cookie=7edf1378537414b7 in_seq=0 out_seq=0 2024-03-18T20:48:39.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdc737fe700 10 client.19995 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdc737fe700 1 -- 172.21.15.46:0/1519339473 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557ac0eb9cf0 con 0x557ac1188590 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdc737fe700 10 client.19995 handle_command_reply: tid=0 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdca6af4b80 1 client.19995 shutdown 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdca6af4b80 1 -- 172.21.15.46:0/1519339473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1117a60 msgr2=0x557ac1129dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/1519339473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1117a60 0x557ac1129dd0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c5b0 tx=0x7fdc840963a0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdca6af4b80 1 -- 172.21.15.46:0/1519339473 shutdown_connections 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/1519339473 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557ac1188590 0x557ac118a9b0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/1519339473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557ac1117a60 0x557ac1129dd0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/1519339473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ac1123220 0x557ac1128910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdca6af4b80 1 --2- 172.21.15.46:0/1519339473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557ac1118380 0x557ac11283d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdc737fe700 0 client.19995 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdca6af4b80 1 -- 172.21.15.46:0/1519339473 >> 172.21.15.46:0/1519339473 conn(0x557ac108e220 msgr2=0x557ac111e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdca6af4b80 1 -- 172.21.15.46:0/1519339473 shutdown_connections 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdca6af4b80 1 -- 172.21.15.46:0/1519339473 wait complete. 2024-03-18T20:48:39.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.438+0000 7fdca6af4b80 20 client.19995 trim_cache size 0 max 16384 2024-03-18T20:48:39.452 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:48:39.452 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T20:48:39.452 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-18T20:48:39.452 INFO:teuthology.orchestra.run.smithi046.stdout: "6fc7c035-069c-4d2a-aa7e-c7baadd9251b": { 2024-03-18T20:48:39.452 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-18T20:48:39.452 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "6fc7c035-069c-4d2a-aa7e-c7baadd9251b", 2024-03-18T20:48:39.452 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-03-18T20:48:39.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:48:39.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:48:39.453 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:48:39.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.439+0000 7fdc9a95f700 1 -- 172.21.15.46:0/1909384587 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdc800a1290 msgr2=0x7fdc800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.439+0000 7fdc9a95f700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdc800a1290 0x7fdc800a3750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdc941013a0 tx=0x7fdc7c009260 comp rx=0 tx=0).stop 2024-03-18T20:48:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.439+0000 7fdc9a95f700 1 -- 172.21.15.46:0/1909384587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc94069280 msgr2=0x7fdc940ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.439+0000 7fdc9a95f700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc94069280 0x7fdc940ffc70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400e9a0 tx=0x7fdc8400ecb0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.439+0000 7fdc9a95f700 1 -- 172.21.15.46:0/1909384587 shutdown_connections 2024-03-18T20:48:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.439+0000 7fdc9a95f700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdc800a1290 0x7fdc800a3750 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.439+0000 7fdc9a95f700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc94069280 0x7fdc940ffc70 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.439+0000 7fdc9a95f700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc941006f0 0x7fdc941a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.439+0000 7fdc9a95f700 1 --2- 172.21.15.46:0/1909384587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc941060b0 0x7fdc941001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.439+0000 7fdc9a95f700 1 -- 172.21.15.46:0/1909384587 >> 172.21.15.46:0/1909384587 conn(0x7fdc94076d80 msgr2=0x7fdc940ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.439+0000 7fdc9a95f700 1 -- 172.21.15.46:0/1909384587 shutdown_connections 2024-03-18T20:48:39.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:48:39.441+0000 7fdc9a95f700 1 -- 172.21.15.46:0/1909384587 wait complete. 2024-03-18T20:48:39.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:39 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:39.457 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:39 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:39.457 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:39 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:39.457 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:39 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:39.458 INFO:tasks.cephfs.filesystem:scrub status for tag:6fc7c035-069c-4d2a-aa7e-c7baadd9251b - {'path': '/', 'tag': '6fc7c035-069c-4d2a-aa7e-c7baadd9251b', 'options': 'recursive,force'} 2024-03-18T20:48:39.540 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 20:48:39 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-i[85318]: 2024-03-18T20:48:39.248+0000 7f349ba7a700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000bb1 [14,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB auth{0=1} v7222 ap=1 s=260096 n(v0 rc2024-03-18T20:48:35.455475+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25108=0-4194304@13} caps={25108=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@53},l=25108 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x556200b03180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000bb1:[<0x10000000229/STUDENTS.DB v7112>,<0x10000000009/PARADOX v52302>,<0x10000000008/~dmtmp v37879>,<0x10000000007/client0 v1232>,<0x10000000006/clients v1181>,<0x10000000005/tmp v1169>,<0x100000001fb/client.0 v1166>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1180>,<0x10000000001/sv_1 v1200>,<0x10000000000/qa v1182>,<0x1/volumes v1150>]//[]","memoryvalue":"(3)0x10000000bb1:[<0x10000000229/STUDENTS.DB v7222>,<0x10000000009/PARADOX v53256>,<0x10000000008/~dmtmp v38782>,<0x10000000007/client0 v1272>,<0x10000000006/clients v1169>,<0x10000000005/tmp v1203>,<0x100000001fb/client.0 v1200>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1214>,<0x10000000001/sv_1 v1234>,<0x10000000000/qa v1216>,<0x1/volumes v1180>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-18T20:48:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:39 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:39 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:39 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:39.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:39 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:39.719 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: 2024-03-18T20:48:39.246+0000 7f60b88e0700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000240 [...14,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/SEED/ auth{1=1,4=1} v52310 ap=2 f(v4 m2024-03-18T20:48:07.458754+0000 3=3+0) n(v34 rc2024-03-18T20:48:07.458754+0000 4=3+1) old_inodes=6 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={25108=pAsLsXsFs/pAsLsXsFsxcral@33} mcw={1=AsLsXsFs,4=AsLsXsFs} | dirtyscattered=0 request=0 lock=3 importing=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55b6c38b7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000240:[<0x10000000009/SEED v120>,<0x10000000008/~dmtmp v53>,<0x10000000007/client0 v37>,<0x10000000006/clients v27>,<0x10000000005/tmp v27>,<0x100000001fb/client.0 v28>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v42>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v52>,<0x1/volumes v48>]//[]","memoryvalue":"(2)0x10000000240:[<0x10000000009/SEED v52310>,<0x10000000008/~dmtmp v38782>,<0x10000000007/client0 v1272>,<0x10000000006/clients v1215>,<0x10000000005/tmp v1203>,<0x100000001fb/client.0 v1200>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1214>,<0x10000000001/sv_1 v1234>,<0x10000000000/qa v1216>,<0x1/volumes v1180>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-18T20:48:07.448754+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-03-18T20:48:07.458754+0000 4=3+1)","memory_value.dirstat":"f(v4 m2024-03-18T20:48:07.458754+0000 3=3+0)","memory_value.rstat":"n(v34 rc2024-03-18T20:48:07.458754+0000 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-18T20:48:39.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:39.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:39.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:39.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:39.721 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: 2024-03-18T20:48:39.248+0000 7f42eee75700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x50000000510 [14,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03 auth v8200 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25108=0-4194304@13} caps={25108=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@53},l=25108 | request=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 authpin=1 tempexporting=0 scrubqueue=0 0x5585ab6a5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x50000000510:[<0x10000000229/ENROLL.X03 v8200>,<0x10000000009/PARADOX v52302>,<0x10000000008/~dmtmp v37879>,<0x10000000007/client0 v1232>,<0x10000000006/clients v1181>,<0x10000000005/tmp v1169>,<0x100000001fb/client.0 v1166>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1180>,<0x10000000001/sv_1 v1200>,<0x10000000000/qa v1182>,<0x1/volumes v1150>]//[]","memoryvalue":"(3)0x50000000510:[<0x10000000229/ENROLL.X03 v8200>,<0x10000000009/PARADOX v53256>,<0x10000000008/~dmtmp v38782>,<0x10000000007/client0 v1272>,<0x10000000006/clients v1169>,<0x10000000005/tmp v1203>,<0x100000001fb/client.0 v1200>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1214>,<0x10000000001/sv_1 v1234>,<0x10000000000/qa v1216>,<0x1/volumes v1180>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-18T20:48:39.721 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: 2024-03-18T20:48:39.249+0000 7f42eee75700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x50000000512 [14,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL auth v8204 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25108=0-4194304@13} caps={25108=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@53},l=25108 | request=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 authpin=1 tempexporting=0 scrubqueue=0 0x5585ab71c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x50000000512:[<0x10000000229/ENROLL.VAL v8204>,<0x10000000009/PARADOX v52302>,<0x10000000008/~dmtmp v37879>,<0x10000000007/client0 v1232>,<0x10000000006/clients v1181>,<0x10000000005/tmp v1169>,<0x100000001fb/client.0 v1166>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1180>,<0x10000000001/sv_1 v1200>,<0x10000000000/qa v1182>,<0x1/volumes v1150>]//[]","memoryvalue":"(3)0x50000000512:[<0x10000000229/ENROLL.VAL v8204>,<0x10000000009/PARADOX v53256>,<0x10000000008/~dmtmp v38782>,<0x10000000007/client0 v1272>,<0x10000000006/clients v1169>,<0x10000000005/tmp v1203>,<0x100000001fb/client.0 v1200>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1214>,<0x10000000001/sv_1 v1234>,<0x10000000000/qa v1216>,<0x1/volumes v1180>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-18T20:48:39.721 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: 2024-03-18T20:48:39.249+0000 7f42eee75700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x5000000050d [14,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB auth v8193 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25108=0-4194304@13} caps={25108=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@53},l=25108 | request=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 authpin=1 tempexporting=0 scrubqueue=0 0x5585ab06f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x5000000050d:[<0x10000000229/STUDENTS.MB v8193>,<0x10000000009/PARADOX v52302>,<0x10000000008/~dmtmp v37879>,<0x10000000007/client0 v1232>,<0x10000000006/clients v1181>,<0x10000000005/tmp v1169>,<0x100000001fb/client.0 v1166>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1180>,<0x10000000001/sv_1 v1200>,<0x10000000000/qa v1182>,<0x1/volumes v1150>]//[]","memoryvalue":"(3)0x5000000050d:[<0x10000000229/STUDENTS.MB v8193>,<0x10000000009/PARADOX v53256>,<0x10000000008/~dmtmp v38782>,<0x10000000007/client0 v1272>,<0x10000000006/clients v1169>,<0x10000000005/tmp v1203>,<0x100000001fb/client.0 v1200>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1214>,<0x10000000001/sv_1 v1234>,<0x10000000000/qa v1216>,<0x1/volumes v1180>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-18T20:48:39.721 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: 2024-03-18T20:48:39.249+0000 7f42eee75700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x5000000050e [14,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB auth v8196 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25108=0-4194304@13} caps={25108=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@53},l=25108 | request=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 authpin=1 tempexporting=0 scrubqueue=0 0x5585ab6a4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x5000000050e:[<0x10000000229/ENROLL.DB v8196>,<0x10000000009/PARADOX v52302>,<0x10000000008/~dmtmp v37879>,<0x10000000007/client0 v1232>,<0x10000000006/clients v1181>,<0x10000000005/tmp v1169>,<0x100000001fb/client.0 v1166>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1180>,<0x10000000001/sv_1 v1200>,<0x10000000000/qa v1182>,<0x1/volumes v1150>]//[]","memoryvalue":"(3)0x5000000050e:[<0x10000000229/ENROLL.DB v8196>,<0x10000000009/PARADOX v53256>,<0x10000000008/~dmtmp v38782>,<0x10000000007/client0 v1272>,<0x10000000006/clients v1169>,<0x10000000005/tmp v1203>,<0x100000001fb/client.0 v1200>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1214>,<0x10000000001/sv_1 v1234>,<0x10000000000/qa v1216>,<0x1/volumes v1180>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-18T20:48:39.721 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: 2024-03-18T20:48:39.249+0000 7f42eee75700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x5000000050b [14,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1 auth v8384 ap=1 s=260096 n(v0 rc2024-03-18T20:48:07.247756+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25108=0-4194304@13} caps={25108=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@53},l=25108 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=1 authpin=1 tempexporting=0 scrubqueue=0 0x5585aba3f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x5000000050b:[<0x10000000229/STUDENTS.XG1 v8189>,<0x10000000009/PARADOX v52302>,<0x10000000008/~dmtmp v37879>,<0x10000000007/client0 v1232>,<0x10000000006/clients v1181>,<0x10000000005/tmp v1169>,<0x100000001fb/client.0 v1166>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1180>,<0x10000000001/sv_1 v1200>,<0x10000000000/qa v1182>,<0x1/volumes v1150>]//[]","memoryvalue":"(3)0x5000000050b:[<0x10000000229/STUDENTS.XG1 v8384>,<0x10000000009/PARADOX v53256>,<0x10000000008/~dmtmp v38782>,<0x10000000007/client0 v1272>,<0x10000000006/clients v1169>,<0x10000000005/tmp v1203>,<0x100000001fb/client.0 v1200>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1214>,<0x10000000001/sv_1 v1234>,<0x10000000000/qa v1216>,<0x1/volumes v1180>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-18T20:48:39.721 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: 2024-03-18T20:48:39.249+0000 7f42eee75700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x50000000513 [14,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV auth v6922 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25108=0-4194304@13} caps={25108=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@53},l=25108 | request=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 authpin=1 tempexporting=0 scrubqueue=0 0x5585ab6a4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x50000000513:[<0x10000000229/STUDENTS.TV v6922>,<0x10000000009/PARADOX v52302>,<0x10000000008/~dmtmp v37879>,<0x10000000007/client0 v1232>,<0x10000000006/clients v1181>,<0x10000000005/tmp v1169>,<0x100000001fb/client.0 v1166>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1180>,<0x10000000001/sv_1 v1200>,<0x10000000000/qa v1182>,<0x1/volumes v1150>]//[]","memoryvalue":"(3)0x50000000513:[<0x10000000229/STUDENTS.TV v6922>,<0x10000000009/PARADOX v53256>,<0x10000000008/~dmtmp v38782>,<0x10000000007/client0 v1272>,<0x10000000006/clients v1169>,<0x10000000005/tmp v1203>,<0x100000001fb/client.0 v1200>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1214>,<0x10000000001/sv_1 v1234>,<0x10000000000/qa v1216>,<0x1/volumes v1180>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-18T20:48:39.721 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: 2024-03-18T20:48:39.250+0000 7f42eee75700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x50000000514 [14,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL auth v6925 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25108=0-4194304@13} caps={25108=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@53},l=25108 | request=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 authpin=1 tempexporting=0 scrubqueue=0 0x5585ab6a3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x50000000514:[<0x10000000229/STUDENTS.FSL v6925>,<0x10000000009/PARADOX v52302>,<0x10000000008/~dmtmp v37879>,<0x10000000007/client0 v1232>,<0x10000000006/clients v1181>,<0x10000000005/tmp v1169>,<0x100000001fb/client.0 v1166>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1180>,<0x10000000001/sv_1 v1200>,<0x10000000000/qa v1182>,<0x1/volumes v1150>]//[]","memoryvalue":"(3)0x50000000514:[<0x10000000229/STUDENTS.FSL v6925>,<0x10000000009/PARADOX v53256>,<0x10000000008/~dmtmp v38782>,<0x10000000007/client0 v1272>,<0x10000000006/clients v1169>,<0x10000000005/tmp v1203>,<0x100000001fb/client.0 v1200>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1214>,<0x10000000001/sv_1 v1234>,<0x10000000000/qa v1216>,<0x1/volumes v1180>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-18T20:48:39.721 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: 2024-03-18T20:48:39.250+0000 7f42eee75700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x5000000050c [14,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1 auth v7027 ap=1 s=260096 n(v0 rc2024-03-18T20:48:07.262756+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25108=0-4194304@13} caps={25108=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@53},l=25108 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=1 authpin=1 tempexporting=0 scrubqueue=0 0x5585ab71d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x5000000050c:[<0x10000000229/STUDENTS.YG1 v6911>,<0x10000000009/PARADOX v52302>,<0x10000000008/~dmtmp v37879>,<0x10000000007/client0 v1232>,<0x10000000006/clients v1181>,<0x10000000005/tmp v1169>,<0x100000001fb/client.0 v1166>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1180>,<0x10000000001/sv_1 v1200>,<0x10000000000/qa v1182>,<0x1/volumes v1150>]//[]","memoryvalue":"(3)0x5000000050c:[<0x10000000229/STUDENTS.YG1 v7027>,<0x10000000009/PARADOX v53256>,<0x10000000008/~dmtmp v38782>,<0x10000000007/client0 v1272>,<0x10000000006/clients v1169>,<0x10000000005/tmp v1203>,<0x100000001fb/client.0 v1200>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1214>,<0x10000000001/sv_1 v1234>,<0x10000000000/qa v1216>,<0x1/volumes v1180>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-18T20:48:39.721 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: 2024-03-18T20:48:39.250+0000 7f42eee75700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x50000000511 [14,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03 auth v6918 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25108=0-4194304@13} caps={25108=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@53},l=25108 | request=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 authpin=1 tempexporting=0 scrubqueue=0 0x5585ab398680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x50000000511:[<0x10000000229/ENROLL.Y03 v6918>,<0x10000000009/PARADOX v52302>,<0x10000000008/~dmtmp v37879>,<0x10000000007/client0 v1232>,<0x10000000006/clients v1181>,<0x10000000005/tmp v1169>,<0x100000001fb/client.0 v1166>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1180>,<0x10000000001/sv_1 v1200>,<0x10000000000/qa v1182>,<0x1/volumes v1150>]//[]","memoryvalue":"(3)0x50000000511:[<0x10000000229/ENROLL.Y03 v6918>,<0x10000000009/PARADOX v53256>,<0x10000000008/~dmtmp v38782>,<0x10000000007/client0 v1272>,<0x10000000006/clients v1169>,<0x10000000005/tmp v1203>,<0x100000001fb/client.0 v1200>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1214>,<0x10000000001/sv_1 v1234>,<0x10000000000/qa v1216>,<0x1/volumes v1180>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-18T20:48:39.721 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 20:48:39 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: 2024-03-18T20:48:39.251+0000 7f42eee75700 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x50000000515 [14,head] /volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL auth v6928 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={25108=0-4194304@13} caps={25108=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@53},l=25108 | request=0 importingcaps=0 caps=1 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 authpin=1 tempexporting=0 scrubqueue=0 0x5585ab398100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x50000000515:[<0x10000000229/COURSES.FSL v6928>,<0x10000000009/PARADOX v52302>,<0x10000000008/~dmtmp v37879>,<0x10000000007/client0 v1232>,<0x10000000006/clients v1181>,<0x10000000005/tmp v1169>,<0x100000001fb/client.0 v1166>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1180>,<0x10000000001/sv_1 v1200>,<0x10000000000/qa v1182>,<0x1/volumes v1150>]//[]","memoryvalue":"(3)0x50000000515:[<0x10000000229/COURSES.FSL v6928>,<0x10000000009/PARADOX v53256>,<0x10000000008/~dmtmp v38782>,<0x10000000007/client0 v1272>,<0x10000000006/clients v1169>,<0x10000000005/tmp v1203>,<0x100000001fb/client.0 v1200>,<0x100000001fa/67a95736-ebd5-44c9-a27b-9f804235a410 v1214>,<0x10000000001/sv_1 v1234>,<0x10000000000/qa v1216>,<0x1/volumes v1180>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-18T20:48:39.724 INFO:tasks.workunit.client.0.smithi046.stdout: 1 471599 3.68 MB/sec execute 574 sec latency 1633.298 ms 2024-03-18T20:48:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: pgmap v392: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.7 MiB/s wr, 158 op/s 2024-03-18T20:48:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub queued for path: / 2024-03-18T20:48:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.b log and `damage ls` output for details 2024-03-18T20:48:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: Scrub error on inode 0x10000000bb1 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.i log and `damage ls` output for details 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: Scrub error on inode 0x50000000510 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: Scrub error on inode 0x50000000512 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: Scrub error on inode 0x5000000050d (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: Scrub error on inode 0x5000000050e (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: Scrub error on inode 0x5000000050b (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: Scrub error on inode 0x50000000513 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: Scrub error on inode 0x50000000514 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: Scrub error on inode 0x5000000050c (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: Scrub error on inode 0x50000000511 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: Scrub error on inode 0x50000000515 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:40.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:40.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: active paths [/] 2024-03-18T20:48:40.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:40 smithi073 ceph-mon[47384]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.724 INFO:tasks.workunit.client.0.smithi046.stdout: 1 473131 3.69 MB/sec execute 575 sec latency 14.378 ms 2024-03-18T20:48:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: pgmap v392: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.7 MiB/s wr, 158 op/s 2024-03-18T20:48:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub queued for path: / 2024-03-18T20:48:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.b log and `damage ls` output for details 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: Scrub error on inode 0x10000000bb1 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.i log and `damage ls` output for details 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: Scrub error on inode 0x50000000510 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: Scrub error on inode 0x50000000512 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: Scrub error on inode 0x5000000050d (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: Scrub error on inode 0x5000000050e (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: Scrub error on inode 0x5000000050b (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: Scrub error on inode 0x50000000513 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: Scrub error on inode 0x50000000514 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: Scrub error on inode 0x5000000050c (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: Scrub error on inode 0x50000000511 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: Scrub error on inode 0x50000000515 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: active paths [/] 2024-03-18T20:48:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:40 smithi155 ceph-mon[48344]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: pgmap v392: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 2.7 MiB/s wr, 158 op/s 2024-03-18T20:48:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub queued for path: / 2024-03-18T20:48:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.b log and `damage ls` output for details 2024-03-18T20:48:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: Scrub error on inode 0x10000000bb1 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB) see mds.i log and `damage ls` output for details 2024-03-18T20:48:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: Scrub error on inode 0x50000000510 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: Scrub error on inode 0x50000000512 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: Scrub error on inode 0x5000000050d (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: Scrub error on inode 0x5000000050e (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: Scrub error on inode 0x5000000050b (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: Scrub error on inode 0x50000000513 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: Scrub error on inode 0x50000000514 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: Scrub error on inode 0x5000000050c (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: Scrub error on inode 0x50000000511 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: Scrub error on inode 0x50000000515 (/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL) see mds.k log and `damage ls` output for details 2024-03-18T20:48:40.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:40.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:40.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: active paths [/] 2024-03-18T20:48:40.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:40.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:40 smithi046 ceph-mon[43495]: scrub summary: idle+waiting paths [/] 2024-03-18T20:48:41.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:41 smithi073 ceph-mon[47384]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-18T20:48:41.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:41 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] up:active 2024-03-18T20:48:41.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:41 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:41.724 INFO:tasks.workunit.client.0.smithi046.stdout: 1 474231 3.70 MB/sec execute 576 sec latency 316.736 ms 2024-03-18T20:48:41.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:41 smithi155 ceph-mon[48344]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-18T20:48:41.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:41 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] up:active 2024-03-18T20:48:41.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:41 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:41.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:41 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a[43472]: 2024-03-18T20:48:41.313+0000 7fb26e3e2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-18T20:48:41.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:41 smithi046 ceph-mon[43495]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-18T20:48:41.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:41 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] up:active 2024-03-18T20:48:41.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:41 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.351+0000 7f7268e6e700 1 -- 172.21.15.46:0/2380501337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7264069df0 msgr2=0x7f726410db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.351+0000 7f7268e6e700 1 --2- 172.21.15.46:0/2380501337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7264069df0 0x7f726410db10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f725c00b0a0 tx=0x7f725c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.351+0000 7f7268e6e700 1 -- 172.21.15.46:0/2380501337 shutdown_connections 2024-03-18T20:48:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.351+0000 7f7268e6e700 1 --2- 172.21.15.46:0/2380501337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7264069df0 0x7f726410db10 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.351+0000 7f7268e6e700 1 --2- 172.21.15.46:0/2380501337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7264106d20 0x7f7264107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.351+0000 7f7268e6e700 1 --2- 172.21.15.46:0/2380501337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7264069430 0x7f72640698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.351+0000 7f7268e6e700 1 -- 172.21.15.46:0/2380501337 >> 172.21.15.46:0/2380501337 conn(0x7f7264076a80 msgr2=0x7f7264076e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.351+0000 7f7268e6e700 1 -- 172.21.15.46:0/2380501337 shutdown_connections 2024-03-18T20:48:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.351+0000 7f7268e6e700 1 -- 172.21.15.46:0/2380501337 wait complete. 2024-03-18T20:48:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.352+0000 7f7268e6e700 1 Processor -- start 2024-03-18T20:48:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.352+0000 7f7268e6e700 1 -- start start 2024-03-18T20:48:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7268e6e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7264069430 0x7f7264197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7268e6e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7264069df0 0x7f7264197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7268e6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7264106d20 0x7f726419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7268e6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726419c120 con 0x7f7264106d20 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7268e6e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726419c290 con 0x7f7264069df0 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7268e6e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726419c450 con 0x7f7264069430 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7262d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7264106d20 0x7f726419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7262d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7264106d20 0x7f726419bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55104/0 (socket says 172.21.15.46:55104) 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7262d9d700 1 -- 172.21.15.46:0/2944551281 learned_addr learned my addr 172.21.15.46:0/2944551281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f726259c700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7264069430 0x7f7264197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7261d9b700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7264069df0 0x7f7264197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7262d9d700 1 -- 172.21.15.46:0/2944551281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7264069430 msgr2=0x7f7264197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7262d9d700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7264069430 0x7f7264197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7262d9d700 1 -- 172.21.15.46:0/2944551281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7264069df0 msgr2=0x7f7264197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7262d9d700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7264069df0 0x7f7264197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7262d9d700 1 -- 172.21.15.46:0/2944551281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c009d00 con 0x7f7264106d20 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7261d9b700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7264069df0 0x7f7264197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7262d9d700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7264106d20 0x7f726419bae0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f725c00b070 tx=0x7f725c0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f725b7fe700 1 -- 172.21.15.46:0/2944551281 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c00e040 con 0x7f7264106d20 2024-03-18T20:48:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f725b7fe700 1 -- 172.21.15.46:0/2944551281 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f725c01d470 con 0x7f7264106d20 2024-03-18T20:48:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.353+0000 7f7268e6e700 1 -- 172.21.15.46:0/2944551281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726419c6d0 con 0x7f7264106d20 2024-03-18T20:48:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.354+0000 7f725b7fe700 1 -- 172.21.15.46:0/2944551281 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c0052a0 con 0x7f7264106d20 2024-03-18T20:48:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.354+0000 7f7268e6e700 1 -- 172.21.15.46:0/2944551281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7264100070 con 0x7f7264106d20 2024-03-18T20:48:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.356+0000 7f725b7fe700 1 -- 172.21.15.46:0/2944551281 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f725c01d850 con 0x7f7264106d20 2024-03-18T20:48:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.356+0000 7f725b7fe700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f72500a1290 0x7f72500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.356+0000 7f725b7fe700 1 -- 172.21.15.46:0/2944551281 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f725c0ce110 con 0x7f7264106d20 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.356+0000 7f726259c700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f72500a1290 0x7f72500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.357+0000 7f726259c700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f72500a1290 0x7f72500a3750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7254005950 tx=0x7f725400f820 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.357+0000 7f7275003b80 1 Processor -- start 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f7275003b80 1 -- start start 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f7275003b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fc6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f7275003b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6e90 0x558e11fd1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f7275003b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e11fd41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f7275003b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e11f20930 con 0x558e11fc6570 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f7275003b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e11f1e2f0 con 0x558e11fc6e90 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f7275003b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e11f19be0 con 0x558e11fd1db0 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f726259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fc6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f726259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fc6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55112/0 (socket says 172.21.15.46:55112) 2024-03-18T20:48:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f726259c700 1 -- 172.21.15.46:0/1686278606 learned_addr learned my addr 172.21.15.46:0/1686278606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f7262d9d700 1 --2- 172.21.15.46:0/1686278606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e11fd41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f7261d9b700 1 --2- 172.21.15.46:0/1686278606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6e90 0x558e11fd1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f726259c700 1 -- 172.21.15.46:0/1686278606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 msgr2=0x558e11fd41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f726259c700 1 --2- 172.21.15.46:0/1686278606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e11fd41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f726259c700 1 -- 172.21.15.46:0/1686278606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6e90 msgr2=0x558e11fd1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f726259c700 1 --2- 172.21.15.46:0/1686278606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6e90 0x558e11fd1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.358+0000 7f726259c700 1 -- 172.21.15.46:0/1686278606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e11d9b350 con 0x558e11fc6570 2024-03-18T20:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.359+0000 7f726259c700 1 --2- 172.21.15.46:0/1686278606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fc6950 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f725401b540 tx=0x7f7254017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77d5cff50d6bbd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.359+0000 7f7258ff9700 1 -- 172.21.15.46:0/1686278606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254026400 con 0x558e11fc6570 2024-03-18T20:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.359+0000 7f7258ff9700 1 -- 172.21.15.46:0/1686278606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7254015040 con 0x558e11fc6570 2024-03-18T20:48:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.359+0000 7f7258ff9700 1 -- 172.21.15.46:0/1686278606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254019720 con 0x558e11fc6570 2024-03-18T20:48:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.360+0000 7f7275003b80 1 -- 172.21.15.46:0/1686278606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 msgr2=0x558e11fc6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.360+0000 7f7275003b80 1 --2- 172.21.15.46:0/1686278606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fc6950 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f725401b540 tx=0x7f7254017000 comp rx=0 tx=0).stop 2024-03-18T20:48:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.360+0000 7f7275003b80 1 -- 172.21.15.46:0/1686278606 shutdown_connections 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.360+0000 7f7275003b80 1 --2- 172.21.15.46:0/1686278606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fc6950 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.360+0000 7f7275003b80 1 --2- 172.21.15.46:0/1686278606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e11fd41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.360+0000 7f7275003b80 1 --2- 172.21.15.46:0/1686278606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6e90 0x558e11fd1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.360+0000 7f7275003b80 1 -- 172.21.15.46:0/1686278606 >> 172.21.15.46:0/1686278606 conn(0x558e11f3d1c0 msgr2=0x558e11fc2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.360+0000 7f7275003b80 1 -- 172.21.15.46:0/1686278606 shutdown_connections 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.360+0000 7f7275003b80 1 -- 172.21.15.46:0/1686278606 wait complete. 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.360+0000 7f7275003b80 1 -- start start 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.360+0000 7f7275003b80 1 Processor -- start 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f7275003b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fd1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f7275003b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fc6e90 0x558e11fcea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f7275003b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fd1db0 0x558e11fcf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f7275003b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e11eb5d20 con 0x558e11fc6570 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f7275003b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e11eb59c0 con 0x558e11fd1db0 2024-03-18T20:48:42.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f7275003b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e11d9d640 con 0x558e11fc6e90 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f7275003b80 10 client.0 _ll_register_callbacks cb 0x558e11d21340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f726259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fd1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f726259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fd1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55126/0 (socket says 172.21.15.46:55126) 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f726259c700 1 -- 172.21.15.46:0/2331228678 learned_addr learned my addr 172.21.15.46:0/2331228678 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f7261d9b700 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fc6e90 0x558e11fcea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f7262d9d700 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fd1db0 0x558e11fcf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f726259c700 1 -- 172.21.15.46:0/2331228678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fc6e90 msgr2=0x558e11fcea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f726259c700 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fc6e90 0x558e11fcea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f726259c700 1 -- 172.21.15.46:0/2331228678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fd1db0 msgr2=0x558e11fcf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f726259c700 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fd1db0 0x558e11fcf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f726259c700 1 -- 172.21.15.46:0/2331228678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7254009dd0 con 0x558e11fc6570 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f726259c700 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fd1800 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f7254005d90 tx=0x7f7254005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ec2e4b87f9c147 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.361+0000 7f7249ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:42.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7249ffb700 1 -- 172.21.15.46:0/2331228678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254026810 con 0x558e11fc6570 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7249ffb700 1 -- 172.21.15.46:0/2331228678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7254026970 con 0x558e11fc6570 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7275003b80 1 -- 172.21.15.46:0/2331228678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e11fba6c0 con 0x558e11fc6570 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7249ffb700 1 -- 172.21.15.46:0/2331228678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7254026b30 con 0x558e11fc6570 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7275003b80 10 client.20007 fetch_fsmap learned FSMap version 56 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7275003b80 1 -- 172.21.15.46:0/2331228678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x558e11d47b00 con 0x558e11fc6570 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7249ffb700 1 -- 172.21.15.46:0/2331228678 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e11fba6c0 con 0x558e11fc6570 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7249ffb700 1 -- 172.21.15.46:0/2331228678 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 56) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f725402f810 con 0x558e11fc6570 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7275003b80 10 client.20007 fetch_fsmap finished waiting for FSMap version 56 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7275003b80 10 client.20007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7275003b80 20 client.20007 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7275003b80 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558e11fe1a40 0x558e11fe3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7275003b80 4 client.20007 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.362+0000 7f7275003b80 1 -- 172.21.15.46:0/2331228678 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e11f126a0 con 0x558e11fe1a40 2024-03-18T20:48:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.363+0000 7f7261d9b700 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558e11fe1a40 0x558e11fe3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.363+0000 7f7249ffb700 10 client.20007 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.363+0000 7f7261d9b700 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558e11fe1a40 0x558e11fe3e60 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19fa992f3ed3f288 server_cookie=18f8841fe372ec84 in_seq=0 out_seq=0 2024-03-18T20:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7249ffb700 1 -- 172.21.15.46:0/2331228678 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558e11f126a0 con 0x558e11fe1a40 2024-03-18T20:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7249ffb700 10 client.20007 handle_command_reply: tid=0 2024-03-18T20:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7275003b80 1 client.20007 shutdown 2024-03-18T20:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7275003b80 1 -- 172.21.15.46:0/2331228678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 msgr2=0x558e11fd1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7275003b80 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fd1800 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f7254005d90 tx=0x7f7254005710 comp rx=0 tx=0).stop 2024-03-18T20:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7275003b80 1 -- 172.21.15.46:0/2331228678 shutdown_connections 2024-03-18T20:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7275003b80 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558e11fe1a40 0x558e11fe3e60 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7275003b80 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fd1800 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7275003b80 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fd1db0 0x558e11fcf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7275003b80 1 --2- 172.21.15.46:0/2331228678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fc6e90 0x558e11fcea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7249ffb700 0 client.20007 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7275003b80 1 -- 172.21.15.46:0/2331228678 >> 172.21.15.46:0/2331228678 conn(0x558e11f3d1c0 msgr2=0x558e11fc2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7275003b80 1 -- 172.21.15.46:0/2331228678 shutdown_connections 2024-03-18T20:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7275003b80 1 -- 172.21.15.46:0/2331228678 wait complete. 2024-03-18T20:48:42.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.364+0000 7f7275003b80 20 client.20007 trim_cache size 0 max 16384 2024-03-18T20:48:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.380+0000 7f7275003b80 1 Processor -- start 2024-03-18T20:48:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.380+0000 7f7275003b80 1 -- start start 2024-03-18T20:48:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.380+0000 7f7275003b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fd8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.380+0000 7f7275003b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6e90 0x558e11fc9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.380+0000 7f7275003b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e11fca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.380+0000 7f7275003b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e11fbf1d0 con 0x558e11fc6570 2024-03-18T20:48:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.380+0000 7f7275003b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e11fcf7c0 con 0x558e11fc6e90 2024-03-18T20:48:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.380+0000 7f7275003b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e11fc8110 con 0x558e11fd1db0 2024-03-18T20:48:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.380+0000 7f726259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fd8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.380+0000 7f726259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fd8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55134/0 (socket says 172.21.15.46:55134) 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.380+0000 7f726259c700 1 -- 172.21.15.46:0/2385312463 learned_addr learned my addr 172.21.15.46:0/2385312463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.380+0000 7f7262d9d700 1 --2- 172.21.15.46:0/2385312463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e11fca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.381+0000 7f7261d9b700 1 --2- 172.21.15.46:0/2385312463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6e90 0x558e11fc9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.381+0000 7f726259c700 1 -- 172.21.15.46:0/2385312463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 msgr2=0x558e11fca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.381+0000 7f726259c700 1 --2- 172.21.15.46:0/2385312463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e11fca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.381+0000 7f726259c700 1 -- 172.21.15.46:0/2385312463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6e90 msgr2=0x558e11fc9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.381+0000 7f726259c700 1 --2- 172.21.15.46:0/2385312463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6e90 0x558e11fc9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.381+0000 7f726259c700 1 -- 172.21.15.46:0/2385312463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e11edfe40 con 0x558e11fc6570 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.381+0000 7f7261d9b700 1 --2- 172.21.15.46:0/2385312463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6e90 0x558e11fc9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.381+0000 7f7262d9d700 1 --2- 172.21.15.46:0/2385312463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e11fca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.381+0000 7f726259c700 1 --2- 172.21.15.46:0/2385312463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fd8a40 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f725402f040 tx=0x7f7254009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfb15bd12b7b36a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.381+0000 7f72597fa700 1 -- 172.21.15.46:0/2385312463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725401bb30 con 0x558e11fc6570 2024-03-18T20:48:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.381+0000 7f72597fa700 1 -- 172.21.15.46:0/2385312463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7254026610 con 0x558e11fc6570 2024-03-18T20:48:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.381+0000 7f72597fa700 1 -- 172.21.15.46:0/2385312463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72540373b0 con 0x558e11fc6570 2024-03-18T20:48:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.382+0000 7f7275003b80 1 -- 172.21.15.46:0/2385312463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 msgr2=0x558e11fd8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.382+0000 7f7275003b80 1 --2- 172.21.15.46:0/2385312463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fd8a40 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f725402f040 tx=0x7f7254009f60 comp rx=0 tx=0).stop 2024-03-18T20:48:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.382+0000 7f7275003b80 1 -- 172.21.15.46:0/2385312463 shutdown_connections 2024-03-18T20:48:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.382+0000 7f7275003b80 1 --2- 172.21.15.46:0/2385312463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6570 0x558e11fd8a40 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.382+0000 7f7275003b80 1 --2- 172.21.15.46:0/2385312463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e11fca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.382+0000 7f7275003b80 1 --2- 172.21.15.46:0/2385312463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6e90 0x558e11fc9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.382+0000 7f7275003b80 1 -- 172.21.15.46:0/2385312463 >> 172.21.15.46:0/2385312463 conn(0x558e11f3d1c0 msgr2=0x558e11fc2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.382+0000 7f7275003b80 1 -- 172.21.15.46:0/2385312463 shutdown_connections 2024-03-18T20:48:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.382+0000 7f7275003b80 1 -- 172.21.15.46:0/2385312463 wait complete. 2024-03-18T20:48:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.382+0000 7f7275003b80 1 -- start start 2024-03-18T20:48:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.382+0000 7f7275003b80 1 Processor -- start 2024-03-18T20:48:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7275003b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6570 0x558e11fd6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7275003b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6e90 0x558e11fd7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7275003b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e12034830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7275003b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e12034d70 con 0x558e11fc6e90 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7275003b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e12034eb0 con 0x558e11fc6570 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7275003b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e12034ff0 con 0x558e11fd1db0 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7275003b80 10 client.0 _ll_register_callbacks cb 0x7f725c009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7261d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6e90 0x558e11fd7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7261d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6e90 0x558e11fd7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55150/0 (socket says 172.21.15.46:55150) 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7261d9b700 1 -- 172.21.15.46:0/3928359054 learned_addr learned my addr 172.21.15.46:0/3928359054 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f726259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6570 0x558e11fd6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7262d9d700 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e12034830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7261d9b700 1 -- 172.21.15.46:0/3928359054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 msgr2=0x558e12034830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7261d9b700 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e12034830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7261d9b700 1 -- 172.21.15.46:0/3928359054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6570 msgr2=0x558e11fd6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7261d9b700 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6570 0x558e11fd6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f7261d9b700 1 -- 172.21.15.46:0/3928359054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e11d47b00 con 0x558e11fc6e90 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.383+0000 7f726259c700 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6570 0x558e11fd6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7261d9b700 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6e90 0x558e11fd7150 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f724c000c00 tx=0x7f724c01d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9988e913712001 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7249ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7249ffb700 1 -- 172.21.15.46:0/3928359054 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c018040 con 0x558e11fc6e90 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7249ffb700 1 -- 172.21.15.46:0/3928359054 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f724c004520 con 0x558e11fc6e90 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7275003b80 1 -- 172.21.15.46:0/3928359054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7254015040 con 0x558e11fc6e90 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7249ffb700 1 -- 172.21.15.46:0/3928359054 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f724c01d900 con 0x558e11fc6e90 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7275003b80 10 client.20013 fetch_fsmap learned FSMap version 56 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7275003b80 1 -- 172.21.15.46:0/3928359054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x558e11d9fc70 con 0x558e11fc6e90 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7249ffb700 1 -- 172.21.15.46:0/3928359054 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7254015040 con 0x558e11fc6e90 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7249ffb700 1 -- 172.21.15.46:0/3928359054 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 56) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f724c0037a0 con 0x558e11fc6e90 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7275003b80 10 client.20013 fetch_fsmap finished waiting for FSMap version 56 2024-03-18T20:48:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7275003b80 10 client.20013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7275003b80 20 client.20013 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7275003b80 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558e120368f0 0x558e12038d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7275003b80 4 client.20013 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:48:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.384+0000 7f7275003b80 1 -- 172.21.15.46:0/3928359054 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558e11d68c90 con 0x558e120368f0 2024-03-18T20:48:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.385+0000 7f726259c700 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558e120368f0 0x558e12038d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.385+0000 7f726259c700 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558e120368f0 0x558e12038d10 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa36407496ae11f7 server_cookie=59878da73ead0b5a in_seq=0 out_seq=0 2024-03-18T20:48:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.385+0000 7f7249ffb700 10 client.20013 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.439+0000 7f7249ffb700 1 -- 172.21.15.46:0/3928359054 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558e11d68c90 con 0x558e120368f0 2024-03-18T20:48:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.439+0000 7f7249ffb700 10 client.20013 handle_command_reply: tid=0 2024-03-18T20:48:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.439+0000 7f7275003b80 1 client.20013 shutdown 2024-03-18T20:48:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.440+0000 7f7275003b80 1 -- 172.21.15.46:0/3928359054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6e90 msgr2=0x558e11fd7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.440+0000 7f7275003b80 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6e90 0x558e11fd7150 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f724c000c00 tx=0x7f724c01d5f0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.441+0000 7f7275003b80 1 -- 172.21.15.46:0/3928359054 shutdown_connections 2024-03-18T20:48:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.441+0000 7f7275003b80 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x558e120368f0 0x558e12038d10 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.441+0000 7f7275003b80 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558e11fd1db0 0x558e12034830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.441+0000 7f7275003b80 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e11fc6570 0x558e11fd6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.441+0000 7f7275003b80 1 --2- 172.21.15.46:0/3928359054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e11fc6e90 0x558e11fd7150 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.441+0000 7f7275003b80 1 -- 172.21.15.46:0/3928359054 >> 172.21.15.46:0/3928359054 conn(0x558e11f3d1c0 msgr2=0x558e11fc2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.441+0000 7f7249ffb700 0 client.20013 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.441+0000 7f7275003b80 1 -- 172.21.15.46:0/3928359054 shutdown_connections 2024-03-18T20:48:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.441+0000 7f7275003b80 1 -- 172.21.15.46:0/3928359054 wait complete. 2024-03-18T20:48:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.441+0000 7f7275003b80 20 client.20013 trim_cache size 0 max 16384 2024-03-18T20:48:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:48:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:48:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:48:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:48:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.442+0000 7f7268e6e700 1 -- 172.21.15.46:0/2944551281 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f72500a1290 msgr2=0x7f72500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.442+0000 7f7268e6e700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f72500a1290 0x7f72500a3750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7254005950 tx=0x7f725400f820 comp rx=0 tx=0).stop 2024-03-18T20:48:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.442+0000 7f7268e6e700 1 -- 172.21.15.46:0/2944551281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7264106d20 msgr2=0x7f726419bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.442+0000 7f7268e6e700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7264106d20 0x7f726419bae0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f725c00b070 tx=0x7f725c0078c0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.442+0000 7f7268e6e700 1 -- 172.21.15.46:0/2944551281 shutdown_connections 2024-03-18T20:48:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.442+0000 7f7268e6e700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f72500a1290 0x7f72500a3750 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.442+0000 7f7268e6e700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7264106d20 0x7f726419bae0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.442+0000 7f7268e6e700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7264069430 0x7f7264197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.442+0000 7f7268e6e700 1 --2- 172.21.15.46:0/2944551281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7264069df0 0x7f7264197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.442+0000 7f7268e6e700 1 -- 172.21.15.46:0/2944551281 >> 172.21.15.46:0/2944551281 conn(0x7f7264076a80 msgr2=0x7f72640feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.442+0000 7f7268e6e700 1 -- 172.21.15.46:0/2944551281 shutdown_connections 2024-03-18T20:48:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:42.442+0000 7f7268e6e700 1 -- 172.21.15.46:0/2944551281 wait complete. 2024-03-18T20:48:42.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:42 smithi073 ceph-mon[47384]: pgmap v393: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.2 MiB/s wr, 116 op/s 2024-03-18T20:48:42.724 INFO:tasks.workunit.client.0.smithi046.stdout: 1 474231 3.69 MB/sec execute 577 sec latency 1316.816 ms 2024-03-18T20:48:42.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:42 smithi155 ceph-mon[48344]: pgmap v393: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.2 MiB/s wr, 116 op/s 2024-03-18T20:48:42.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:42 smithi046 ceph-mon[43495]: pgmap v393: 97 pgs: 97 active+clean; 205 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.2 MiB/s wr, 116 op/s 2024-03-18T20:48:43.724 INFO:tasks.workunit.client.0.smithi046.stdout: 1 474231 3.68 MB/sec execute 578 sec latency 2316.913 ms 2024-03-18T20:48:44.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:44 smithi073 ceph-mon[47384]: pgmap v394: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 3.2 MiB/s wr, 179 op/s 2024-03-18T20:48:44.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:44 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] up:active 2024-03-18T20:48:44.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:44 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:44.724 INFO:tasks.workunit.client.0.smithi046.stdout: 1 474231 3.68 MB/sec execute 579 sec latency 3317.041 ms 2024-03-18T20:48:44.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:44 smithi155 ceph-mon[48344]: pgmap v394: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 3.2 MiB/s wr, 179 op/s 2024-03-18T20:48:44.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:44 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] up:active 2024-03-18T20:48:44.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:44 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:44.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:44 smithi046 ceph-mon[43495]: pgmap v394: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 3.2 MiB/s wr, 179 op/s 2024-03-18T20:48:44.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:44 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] up:active 2024-03-18T20:48:44.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:44 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:45.724 INFO:tasks.workunit.client.0.smithi046.stdout: 1 476081 3.69 MB/sec execute 580 sec latency 3465.302 ms 2024-03-18T20:48:46.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:46 smithi155 ceph-mon[48344]: pgmap v395: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 2.8 MiB/s wr, 118 op/s 2024-03-18T20:48:46.295 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:46 smithi046 ceph-mon[43495]: pgmap v395: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 2.8 MiB/s wr, 118 op/s 2024-03-18T20:48:46.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:46 smithi073 ceph-mon[47384]: pgmap v395: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 2.8 MiB/s wr, 118 op/s 2024-03-18T20:48:46.724 INFO:tasks.workunit.client.0.smithi046.stdout: 1 477225 3.69 MB/sec execute 581 sec latency 17.384 ms 2024-03-18T20:48:47.724 INFO:tasks.workunit.client.0.smithi046.stdout: 1 477780 3.69 MB/sec execute 582 sec latency 658.712 ms 2024-03-18T20:48:48.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:48 smithi155 ceph-mon[48344]: pgmap v396: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 2.8 MiB/s wr, 118 op/s 2024-03-18T20:48:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:48 smithi046 ceph-mon[43495]: pgmap v396: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 2.8 MiB/s wr, 118 op/s 2024-03-18T20:48:48.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:48 smithi073 ceph-mon[47384]: pgmap v396: 97 pgs: 97 active+clean; 209 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 2.8 MiB/s wr, 118 op/s 2024-03-18T20:48:48.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 477780 3.69 MB/sec execute 583 sec latency 1658.837 ms 2024-03-18T20:48:49.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 477780 3.68 MB/sec execute 584 sec latency 2658.961 ms 2024-03-18T20:48:50.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:50 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a[43472]: 2024-03-18T20:48:50.017+0000 7fb26e3e2700 -1 log_channel(cluster) log [ERR] : Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-18T20:48:50.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:50 smithi155 ceph-mon[48344]: pgmap v397: 97 pgs: 97 active+clean; 210 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 3.2 MiB/s wr, 166 op/s 2024-03-18T20:48:50.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:50 smithi155 ceph-mon[48344]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-18T20:48:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:50 smithi046 ceph-mon[43495]: pgmap v397: 97 pgs: 97 active+clean; 210 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 3.2 MiB/s wr, 166 op/s 2024-03-18T20:48:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:50 smithi046 ceph-mon[43495]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-18T20:48:50.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:50 smithi073 ceph-mon[47384]: pgmap v397: 97 pgs: 97 active+clean; 210 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 3.2 MiB/s wr, 166 op/s 2024-03-18T20:48:50.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:50 smithi073 ceph-mon[47384]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-18T20:48:50.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 478723 3.68 MB/sec execute 585 sec latency 3264.627 ms 2024-03-18T20:48:51.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 480234 3.69 MB/sec execute 586 sec latency 16.341 ms 2024-03-18T20:48:52.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:52 smithi155 ceph-mon[48344]: pgmap v398: 97 pgs: 97 active+clean; 210 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-03-18T20:48:52.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:52 smithi046 ceph-mon[43495]: pgmap v398: 97 pgs: 97 active+clean; 210 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-03-18T20:48:52.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:52 smithi073 ceph-mon[47384]: pgmap v398: 97 pgs: 97 active+clean; 210 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-03-18T20:48:52.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 481360 3.69 MB/sec execute 587 sec latency 322.987 ms 2024-03-18T20:48:53.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:53 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:48:53.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:53 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:48:53.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:53 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:48:53.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:53 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:48:53.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:53 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:48:53.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:53 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:48:53.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 481360 3.69 MB/sec execute 588 sec latency 1323.025 ms 2024-03-18T20:48:53.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:53 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:48:53.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:53 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:48:53.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:53 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:48:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:54 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:48:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:54 smithi073 ceph-mon[47384]: scrub summary: idle 2024-03-18T20:48:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:54 smithi073 ceph-mon[47384]: pgmap v399: 97 pgs: 97 active+clean; 213 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-03-18T20:48:54.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 481360 3.68 MB/sec execute 589 sec latency 2323.134 ms 2024-03-18T20:48:54.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:54 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:48:54.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:54 smithi155 ceph-mon[48344]: scrub summary: idle 2024-03-18T20:48:54.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:54 smithi155 ceph-mon[48344]: pgmap v399: 97 pgs: 97 active+clean; 213 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-03-18T20:48:54.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:54 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:48:54.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:54 smithi046 ceph-mon[43495]: scrub summary: idle 2024-03-18T20:48:54.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:54 smithi046 ceph-mon[43495]: pgmap v399: 97 pgs: 97 active+clean; 213 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-03-18T20:48:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.583+0000 7f198190c700 1 -- 172.21.15.46:0/1755056075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c101ee0 msgr2=0x7f197c1022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.583+0000 7f198190c700 1 --2- 172.21.15.46:0/1755056075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c101ee0 0x7f197c1022c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f196c0098f0 tx=0x7f196c009c00 comp rx=0 tx=0).stop 2024-03-18T20:48:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.583+0000 7f198190c700 1 -- 172.21.15.46:0/1755056075 shutdown_connections 2024-03-18T20:48:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.583+0000 7f198190c700 1 --2- 172.21.15.46:0/1755056075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f197c10b2b0 0x7f197c10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.583+0000 7f198190c700 1 --2- 172.21.15.46:0/1755056075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c101ee0 0x7f197c1022c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.583+0000 7f198190c700 1 --2- 172.21.15.46:0/1755056075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f197c102800 0x7f197c10ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.583+0000 7f198190c700 1 -- 172.21.15.46:0/1755056075 >> 172.21.15.46:0/1755056075 conn(0x7f197c0fb680 msgr2=0x7f197c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.583+0000 7f198190c700 1 -- 172.21.15.46:0/1755056075 shutdown_connections 2024-03-18T20:48:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.583+0000 7f198190c700 1 -- 172.21.15.46:0/1755056075 wait complete. 2024-03-18T20:48:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.584+0000 7f198190c700 1 Processor -- start 2024-03-18T20:48:55.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.584+0000 7f198190c700 1 -- start start 2024-03-18T20:48:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.584+0000 7f198190c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f197c101ee0 0x7f197c197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f198190c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f197c102800 0x7f197c1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f198190c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c10b2b0 0x7f197c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f198190c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c19c0a0 con 0x7f197c101ee0 2024-03-18T20:48:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f198190c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c19c210 con 0x7f197c10b2b0 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f198190c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c19c3d0 con 0x7f197c102800 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f197affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f197c101ee0 0x7f197c197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f197a7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f197c102800 0x7f197c1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f197affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f197c101ee0 0x7f197c197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38088/0 (socket says 172.21.15.46:38088) 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f197affd700 1 -- 172.21.15.46:0/3145359566 learned_addr learned my addr 172.21.15.46:0/3145359566 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f197b7fe700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c10b2b0 0x7f197c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f197affd700 1 -- 172.21.15.46:0/3145359566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f197c102800 msgr2=0x7f197c1976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f197affd700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f197c102800 0x7f197c1976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f197affd700 1 -- 172.21.15.46:0/3145359566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c10b2b0 msgr2=0x7f197c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f197affd700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c10b2b0 0x7f197c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f197affd700 1 -- 172.21.15.46:0/3145359566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c009520 con 0x7f197c101ee0 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f197b7fe700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c10b2b0 0x7f197c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.585+0000 7f197affd700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f197c101ee0 0x7f197c197180 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f196c00fa00 tx=0x7f196c00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.586+0000 7f198090a700 1 -- 172.21.15.46:0/3145359566 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c01c070 con 0x7f197c101ee0 2024-03-18T20:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.586+0000 7f198190c700 1 -- 172.21.15.46:0/3145359566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c19c650 con 0x7f197c101ee0 2024-03-18T20:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.586+0000 7f198090a700 1 -- 172.21.15.46:0/3145359566 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f196c00bb20 con 0x7f197c101ee0 2024-03-18T20:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.586+0000 7f198090a700 1 -- 172.21.15.46:0/3145359566 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c004930 con 0x7f197c101ee0 2024-03-18T20:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.586+0000 7f198190c700 1 -- 172.21.15.46:0/3145359566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c1a3ef0 con 0x7f197c101ee0 2024-03-18T20:48:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.589+0000 7f198090a700 1 -- 172.21.15.46:0/3145359566 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f196c020070 con 0x7f197c101ee0 2024-03-18T20:48:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.589+0000 7f198090a700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f19680a1290 0x7f19680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f198daa1b80 1 Processor -- start 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f198daa1b80 1 -- start start 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f198090a700 1 -- 172.21.15.46:0/3145359566 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f196c0cd8d0 con 0x7f197c101ee0 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197a7fc700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f19680a1290 0x7f19680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f198daa1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a3a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f198daa1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a3af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f198daa1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a50840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f198daa1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c01994930 con 0x561c01a3c1c0 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f198daa1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c019922f0 con 0x561c01a3aab0 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f198daa1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c0198dbe0 con 0x561c01a4e450 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a3a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a3a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38098/0 (socket says 172.21.15.46:38098) 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197affd700 1 -- 172.21.15.46:0/251515268 learned_addr learned my addr 172.21.15.46:0/251515268 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197a7fc700 1 --2- 172.21.15.46:0/251515268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a3af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197b7fe700 1 --2- 172.21.15.46:0/251515268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a50840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197a7fc700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f19680a1290 0x7f19680a3750 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f197c1986d0 tx=0x7f1964011720 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197affd700 1 -- 172.21.15.46:0/251515268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 msgr2=0x561c01a50840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197affd700 1 --2- 172.21.15.46:0/251515268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a50840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197affd700 1 -- 172.21.15.46:0/251515268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 msgr2=0x561c01a3af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197affd700 1 --2- 172.21.15.46:0/251515268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a3af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197affd700 1 -- 172.21.15.46:0/251515268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c0180f350 con 0x561c01a3c1c0 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.590+0000 7f197a7fc700 1 --2- 172.21.15.46:0/251515268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a3af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.591+0000 7f197affd700 1 --2- 172.21.15.46:0/251515268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a3a570 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f196c0a28b0 tx=0x7f196c00bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c480ee80cd0f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.591+0000 7f1971ffb700 1 -- 172.21.15.46:0/251515268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c09a9f0 con 0x561c01a3c1c0 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.591+0000 7f1971ffb700 1 -- 172.21.15.46:0/251515268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f196c09e820 con 0x561c01a3c1c0 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.591+0000 7f1971ffb700 1 -- 172.21.15.46:0/251515268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c09ae50 con 0x561c01a3c1c0 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.592+0000 7f198daa1b80 1 -- 172.21.15.46:0/251515268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 msgr2=0x561c01a3a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.592+0000 7f198daa1b80 1 --2- 172.21.15.46:0/251515268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a3a570 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f196c0a28b0 tx=0x7f196c00bc90 comp rx=0 tx=0).stop 2024-03-18T20:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.592+0000 7f198daa1b80 1 -- 172.21.15.46:0/251515268 shutdown_connections 2024-03-18T20:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.592+0000 7f198daa1b80 1 --2- 172.21.15.46:0/251515268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a3a570 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.592+0000 7f198daa1b80 1 --2- 172.21.15.46:0/251515268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a50840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.592+0000 7f198daa1b80 1 --2- 172.21.15.46:0/251515268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a3af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.592+0000 7f198daa1b80 1 -- 172.21.15.46:0/251515268 >> 172.21.15.46:0/251515268 conn(0x561c019b11c0 msgr2=0x561c01a36a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.592+0000 7f198daa1b80 1 -- 172.21.15.46:0/251515268 shutdown_connections 2024-03-18T20:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.592+0000 7f198daa1b80 1 -- 172.21.15.46:0/251515268 wait complete. 2024-03-18T20:48:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.592+0000 7f198daa1b80 1 -- start start 2024-03-18T20:48:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f198daa1b80 1 Processor -- start 2024-03-18T20:48:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f198daa1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a412c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f198daa1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a3e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f198daa1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a3ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f198daa1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c01929d20 con 0x561c01a3c1c0 2024-03-18T20:48:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f198daa1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c019299c0 con 0x561c01a3aab0 2024-03-18T20:48:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f198daa1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c01811640 con 0x561c01a4e450 2024-03-18T20:48:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f198daa1b80 10 client.0 _ll_register_callbacks cb 0x561c01795340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f197a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a3e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f197a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a3e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38108/0 (socket says 172.21.15.46:38108) 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f197a7fc700 1 -- 172.21.15.46:0/1150268740 learned_addr learned my addr 172.21.15.46:0/1150268740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f197affd700 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.593+0000 7f197b7fe700 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a3ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f197affd700 1 -- 172.21.15.46:0/1150268740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 msgr2=0x561c01a3ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f197affd700 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a3ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f197affd700 1 -- 172.21.15.46:0/1150268740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 msgr2=0x561c01a3e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f197affd700 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a3e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f197affd700 1 -- 172.21.15.46:0/1150268740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1964011ca0 con 0x561c01a3aab0 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f197a7fc700 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a3e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f197affd700 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a412c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f196c098d40 tx=0x7f196c0caf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ccd69737057fbad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f195affd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f195affd700 1 -- 172.21.15.46:0/1150268740 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c09a9f0 con 0x561c01a3aab0 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f195affd700 1 -- 172.21.15.46:0/1150268740 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f196c09ac50 con 0x561c01a3aab0 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f198daa1b80 1 -- 172.21.15.46:0/1150268740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c01a2e6c0 con 0x561c01a3aab0 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f195affd700 1 -- 172.21.15.46:0/1150268740 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f196c09ae30 con 0x561c01a3aab0 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f198daa1b80 10 client.29627 fetch_fsmap learned FSMap version 57 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f198daa1b80 1 -- 172.21.15.46:0/1150268740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x7f197c1a3ef0 con 0x561c01a3aab0 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.594+0000 7f195affd700 1 -- 172.21.15.46:0/1150268740 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c01a2e6c0 con 0x561c01a3aab0 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.595+0000 7f195affd700 1 -- 172.21.15.46:0/1150268740 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 57) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f196c0b3070 con 0x561c01a3aab0 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.595+0000 7f198daa1b80 10 client.29627 fetch_fsmap finished waiting for FSMap version 57 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.595+0000 7f198daa1b80 10 client.29627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.595+0000 7f198daa1b80 20 client.29627 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.595+0000 7f198daa1b80 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561c01a5da50 0x561c01a5fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.595+0000 7f198daa1b80 4 client.29627 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.595+0000 7f198daa1b80 1 -- 172.21.15.46:0/1150268740 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c019866a0 con 0x561c01a5da50 2024-03-18T20:48:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.595+0000 7f197a7fc700 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561c01a5da50 0x561c01a5fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.595+0000 7f195affd700 10 client.29627 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.595+0000 7f197a7fc700 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561c01a5da50 0x561c01a5fe70 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80e7901b41afc892 server_cookie=4ee5999ccbfb15e9 in_seq=0 out_seq=0 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f195affd700 1 -- 172.21.15.46:0/1150268740 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561c019866a0 con 0x561c01a5da50 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f195affd700 10 client.29627 handle_command_reply: tid=0 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f198daa1b80 1 client.29627 shutdown 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f198daa1b80 1 -- 172.21.15.46:0/1150268740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 msgr2=0x561c01a412c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f198daa1b80 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a412c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f196c098d40 tx=0x7f196c0caf40 comp rx=0 tx=0).stop 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f198daa1b80 1 -- 172.21.15.46:0/1150268740 shutdown_connections 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f198daa1b80 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561c01a5da50 0x561c01a5fe70 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f198daa1b80 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a3ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f198daa1b80 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a412c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f195affd700 0 client.29627 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f198daa1b80 1 --2- 172.21.15.46:0/1150268740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a3e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f198daa1b80 1 -- 172.21.15.46:0/1150268740 >> 172.21.15.46:0/1150268740 conn(0x561c019b11c0 msgr2=0x561c01a36650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f198daa1b80 1 -- 172.21.15.46:0/1150268740 shutdown_connections 2024-03-18T20:48:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f198daa1b80 1 -- 172.21.15.46:0/1150268740 wait complete. 2024-03-18T20:48:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.596+0000 7f198daa1b80 20 client.29627 trim_cache size 0 max 16384 2024-03-18T20:48:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f198daa1b80 1 Processor -- start 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f198daa1b80 1 -- start start 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f198daa1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a3ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f198daa1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a3c1c0 0x561c01a40ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f198daa1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a4e450 0x561c01a4dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f198daa1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c01a30c10 con 0x561c01a4e450 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f198daa1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c01a4d730 con 0x561c01a3aab0 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f198daa1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c01a4b020 con 0x561c01a3c1c0 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f197b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a4e450 0x561c01a4dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f197affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a3ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f197affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a3ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55458/0 (socket says 172.21.15.46:55458) 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f197b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a4e450 0x561c01a4dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38114/0 (socket says 172.21.15.46:38114) 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f197affd700 1 -- 172.21.15.46:0/783709074 learned_addr learned my addr 172.21.15.46:0/783709074 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f197a7fc700 1 --2- 172.21.15.46:0/783709074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a3c1c0 0x561c01a40ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f197affd700 1 -- 172.21.15.46:0/783709074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a3c1c0 msgr2=0x561c01a40ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f197affd700 1 --2- 172.21.15.46:0/783709074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a3c1c0 0x561c01a40ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.613+0000 7f197affd700 1 -- 172.21.15.46:0/783709074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a4e450 msgr2=0x561c01a4dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.614+0000 7f197affd700 1 --2- 172.21.15.46:0/783709074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a4e450 0x561c01a4dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.614+0000 7f197affd700 1 -- 172.21.15.46:0/783709074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c017bbb00 con 0x561c01a3aab0 2024-03-18T20:48:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.614+0000 7f197affd700 1 --2- 172.21.15.46:0/783709074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a3ebf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f196c005bf0 tx=0x7f196c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f270ad73bf096b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.614+0000 7f19727fc700 1 -- 172.21.15.46:0/783709074 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c09deb0 con 0x561c01a3aab0 2024-03-18T20:48:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.614+0000 7f19727fc700 1 -- 172.21.15.46:0/783709074 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f196c09a9f0 con 0x561c01a3aab0 2024-03-18T20:48:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.614+0000 7f19727fc700 1 -- 172.21.15.46:0/783709074 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c09aeb0 con 0x561c01a3aab0 2024-03-18T20:48:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.615+0000 7f198daa1b80 1 -- 172.21.15.46:0/783709074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 msgr2=0x561c01a3ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.615+0000 7f198daa1b80 1 --2- 172.21.15.46:0/783709074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a3ebf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f196c005bf0 tx=0x7f196c009c00 comp rx=0 tx=0).stop 2024-03-18T20:48:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.615+0000 7f198daa1b80 1 -- 172.21.15.46:0/783709074 shutdown_connections 2024-03-18T20:48:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.615+0000 7f198daa1b80 1 --2- 172.21.15.46:0/783709074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a4e450 0x561c01a4dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.615+0000 7f198daa1b80 1 --2- 172.21.15.46:0/783709074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a3ebf0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.615+0000 7f198daa1b80 1 --2- 172.21.15.46:0/783709074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a3c1c0 0x561c01a40ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.615+0000 7f198daa1b80 1 -- 172.21.15.46:0/783709074 >> 172.21.15.46:0/783709074 conn(0x561c019b11c0 msgr2=0x561c01a35120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.615+0000 7f198daa1b80 1 -- 172.21.15.46:0/783709074 shutdown_connections 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.615+0000 7f198daa1b80 1 -- 172.21.15.46:0/783709074 wait complete. 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.615+0000 7f198daa1b80 1 -- start start 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f198daa1b80 1 Processor -- start 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f198daa1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a52820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f198daa1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a4fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f198daa1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a50120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f198daa1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c01a50800 con 0x561c01a3c1c0 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f198daa1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c01a50970 con 0x561c01a3aab0 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f198daa1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c01ab0880 con 0x561c01a4e450 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f198daa1b80 10 client.0 _ll_register_callbacks cb 0x7f1964009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:48:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a4fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a52820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a4fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38124/0 (socket says 172.21.15.46:38124) 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a52820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55460/0 (socket says 172.21.15.46:55460) 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197a7fc700 1 -- 172.21.15.46:0/198963353 learned_addr learned my addr 172.21.15.46:0/198963353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197b7fe700 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a50120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197affd700 1 -- 172.21.15.46:0/198963353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 msgr2=0x561c01a50120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197affd700 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a50120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197affd700 1 -- 172.21.15.46:0/198963353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 msgr2=0x561c01a4fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197affd700 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a4fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197affd700 1 -- 172.21.15.46:0/198963353 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c1a3ef0 con 0x561c01a3aab0 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197a7fc700 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a4fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.616+0000 7f197b7fe700 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a50120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f197affd700 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a52820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f196c098d40 tx=0x7f196c005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43604c56c0e7a036 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f195affd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:48:55.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f195affd700 1 -- 172.21.15.46:0/198963353 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c09deb0 con 0x561c01a3aab0 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f195affd700 1 -- 172.21.15.46:0/198963353 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f196c0a2950 con 0x561c01a3aab0 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f198daa1b80 1 -- 172.21.15.46:0/198963353 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f196c09e820 con 0x561c01a3aab0 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f195affd700 1 -- 172.21.15.46:0/198963353 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f196c066320 con 0x561c01a3aab0 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f198daa1b80 10 client.29633 fetch_fsmap learned FSMap version 57 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f198daa1b80 1 -- 172.21.15.46:0/198963353 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x561c01953e40 con 0x561c01a3aab0 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f195affd700 1 -- 172.21.15.46:0/198963353 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f196c09e820 con 0x561c01a3aab0 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f195affd700 1 -- 172.21.15.46:0/198963353 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 57) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f196c0969e0 con 0x561c01a3aab0 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f198daa1b80 10 client.29633 fetch_fsmap finished waiting for FSMap version 57 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f198daa1b80 10 client.29633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f198daa1b80 20 client.29633 populate_metadata read hostname 'smithi046' 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f198daa1b80 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561c01ab1a50 0x561c01ab3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f198daa1b80 4 client.29633 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.617+0000 7f198daa1b80 1 -- 172.21.15.46:0/198963353 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561c017dcc90 con 0x561c01ab1a50 2024-03-18T20:48:55.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.618+0000 7f197a7fc700 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561c01ab1a50 0x561c01ab3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:48:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.618+0000 7f195affd700 10 client.29633 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:55.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.618+0000 7f197a7fc700 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561c01ab1a50 0x561c01ab3e40 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23944d83ac895a4e server_cookie=d01c75c43880bc9d in_seq=0 out_seq=0 2024-03-18T20:48:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f195affd700 1 -- 172.21.15.46:0/198963353 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561c017dcc90 con 0x561c01ab1a50 2024-03-18T20:48:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f195affd700 10 client.29633 handle_command_reply: tid=0 2024-03-18T20:48:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f198daa1b80 1 client.29633 shutdown 2024-03-18T20:48:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f198daa1b80 1 -- 172.21.15.46:0/198963353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 msgr2=0x561c01a52820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f198daa1b80 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a52820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f196c098d40 tx=0x7f196c005960 comp rx=0 tx=0).stop 2024-03-18T20:48:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f198daa1b80 1 -- 172.21.15.46:0/198963353 shutdown_connections 2024-03-18T20:48:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f198daa1b80 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561c01ab1a50 0x561c01ab3e40 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f198daa1b80 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561c01a4e450 0x561c01a50120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f198daa1b80 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c01a3aab0 0x561c01a52820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f198daa1b80 1 --2- 172.21.15.46:0/198963353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c01a3c1c0 0x561c01a4fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f195affd700 0 client.29633 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:48:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f198daa1b80 1 -- 172.21.15.46:0/198963353 >> 172.21.15.46:0/198963353 conn(0x561c019b11c0 msgr2=0x561c01a4d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f198daa1b80 1 -- 172.21.15.46:0/198963353 shutdown_connections 2024-03-18T20:48:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f198daa1b80 1 -- 172.21.15.46:0/198963353 wait complete. 2024-03-18T20:48:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.644+0000 7f198daa1b80 20 client.29633 trim_cache size 0 max 16384 2024-03-18T20:48:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:48:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:48:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:48:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:48:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.645+0000 7f198190c700 1 -- 172.21.15.46:0/3145359566 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f19680a1290 msgr2=0x7f19680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.645+0000 7f198190c700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f19680a1290 0x7f19680a3750 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f197c1986d0 tx=0x7f1964011720 comp rx=0 tx=0).stop 2024-03-18T20:48:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.645+0000 7f198190c700 1 -- 172.21.15.46:0/3145359566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f197c101ee0 msgr2=0x7f197c197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:48:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.645+0000 7f198190c700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f197c101ee0 0x7f197c197180 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f196c00fa00 tx=0x7f196c00fa30 comp rx=0 tx=0).stop 2024-03-18T20:48:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.645+0000 7f198190c700 1 -- 172.21.15.46:0/3145359566 shutdown_connections 2024-03-18T20:48:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.645+0000 7f198190c700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f19680a1290 0x7f19680a3750 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.645+0000 7f198190c700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f197c101ee0 0x7f197c197180 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.645+0000 7f198190c700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c10b2b0 0x7f197c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.645+0000 7f198190c700 1 --2- 172.21.15.46:0/3145359566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f197c102800 0x7f197c1976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:48:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.645+0000 7f198190c700 1 -- 172.21.15.46:0/3145359566 >> 172.21.15.46:0/3145359566 conn(0x7f197c0fb680 msgr2=0x7f197c0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:48:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.645+0000 7f198190c700 1 -- 172.21.15.46:0/3145359566 shutdown_connections 2024-03-18T20:48:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:48:55.645+0000 7f198190c700 1 -- 172.21.15.46:0/3145359566 wait complete. 2024-03-18T20:48:55.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 482227 3.68 MB/sec execute 590 sec latency 2932.884 ms 2024-03-18T20:48:56.289 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:56 smithi155 ceph-mon[48344]: pgmap v400: 97 pgs: 97 active+clean; 213 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 711 KiB/s wr, 92 op/s 2024-03-18T20:48:56.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:56 smithi046 ceph-mon[43495]: pgmap v400: 97 pgs: 97 active+clean; 213 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 711 KiB/s wr, 92 op/s 2024-03-18T20:48:56.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:56 smithi073 ceph-mon[47384]: pgmap v400: 97 pgs: 97 active+clean; 213 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 711 KiB/s wr, 92 op/s 2024-03-18T20:48:56.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 483657 3.69 MB/sec execute 591 sec latency 17.218 ms 2024-03-18T20:48:57.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 484911 3.69 MB/sec execute 592 sec latency 224.545 ms 2024-03-18T20:48:58.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:58 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:58.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:48:58 smithi073 ceph-mon[47384]: pgmap v401: 97 pgs: 97 active+clean; 213 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 711 KiB/s wr, 92 op/s 2024-03-18T20:48:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:58 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:58.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:48:58 smithi155 ceph-mon[48344]: pgmap v401: 97 pgs: 97 active+clean; 213 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 711 KiB/s wr, 92 op/s 2024-03-18T20:48:58.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:58 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:48:58.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:48:58 smithi046 ceph-mon[43495]: pgmap v401: 97 pgs: 97 active+clean; 213 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 711 KiB/s wr, 92 op/s 2024-03-18T20:48:58.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 484911 3.69 MB/sec execute 593 sec latency 1224.663 ms 2024-03-18T20:48:59.726 INFO:tasks.workunit.client.0.smithi046.stdout: 1 484911 3.68 MB/sec execute 594 sec latency 2224.779 ms 2024-03-18T20:49:00.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:00 smithi155 ceph-mon[48344]: pgmap v402: 97 pgs: 97 active+clean; 214 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-03-18T20:49:00.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:00 smithi046 ceph-mon[43495]: pgmap v402: 97 pgs: 97 active+clean; 214 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-03-18T20:49:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:00 smithi073 ceph-mon[47384]: pgmap v402: 97 pgs: 97 active+clean; 214 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-03-18T20:49:00.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 485505 3.68 MB/sec execute 595 sec latency 2831.708 ms 2024-03-18T20:49:01.726 INFO:tasks.workunit.client.0.smithi046.stdout: 1 487023 3.69 MB/sec execute 596 sec latency 23.437 ms 2024-03-18T20:49:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:02 smithi046 ceph-mon[43495]: pgmap v403: 97 pgs: 97 active+clean; 214 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 758 KiB/s wr, 96 op/s 2024-03-18T20:49:02.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:02 smithi073 ceph-mon[47384]: pgmap v403: 97 pgs: 97 active+clean; 214 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 758 KiB/s wr, 96 op/s 2024-03-18T20:49:02.726 INFO:tasks.workunit.client.0.smithi046.stdout: 1 488420 3.70 MB/sec execute 597 sec latency 20.473 ms 2024-03-18T20:49:02.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:02 smithi155 ceph-mon[48344]: pgmap v403: 97 pgs: 97 active+clean; 214 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 758 KiB/s wr, 96 op/s 2024-03-18T20:49:03.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:03 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:03.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:03 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:03.725 INFO:tasks.workunit.client.0.smithi046.stdout: 1 488453 3.69 MB/sec execute 598 sec latency 979.757 ms 2024-03-18T20:49:03.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:03 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:04.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:04 smithi073 ceph-mon[47384]: pgmap v404: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-03-18T20:49:04.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:04 smithi073 ceph-mon[47384]: osdmap e95: 12 total, 12 up, 12 in 2024-03-18T20:49:04.726 INFO:tasks.workunit.client.0.smithi046.stdout: 1 489660 3.69 MB/sec execute 599 sec latency 1443.491 ms 2024-03-18T20:49:04.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:04 smithi155 ceph-mon[48344]: pgmap v404: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-03-18T20:49:04.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:04 smithi155 ceph-mon[48344]: osdmap e95: 12 total, 12 up, 12 in 2024-03-18T20:49:04.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:04 smithi046 ceph-mon[43495]: pgmap v404: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-03-18T20:49:04.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:04 smithi046 ceph-mon[43495]: osdmap e95: 12 total, 12 up, 12 in 2024-03-18T20:49:05.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:05 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:05.726 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 600 sec 2024-03-18T20:49:05.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:05 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:05.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:05 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:06.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:06 smithi073 ceph-mon[47384]: pgmap v406: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-03-18T20:49:06.726 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 601 sec 2024-03-18T20:49:06.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:06 smithi155 ceph-mon[48344]: pgmap v406: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-03-18T20:49:06.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:06 smithi046 ceph-mon[43495]: pgmap v406: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-03-18T20:49:07.726 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 602 sec 2024-03-18T20:49:08.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:08 smithi073 ceph-mon[47384]: pgmap v407: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-03-18T20:49:08.726 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 603 sec 2024-03-18T20:49:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.772+0000 7f29eebba700 1 -- 172.21.15.46:0/1072934230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e8068aa0 msgr2=0x7f29e810dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.772+0000 7f29eebba700 1 --2- 172.21.15.46:0/1072934230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e8068aa0 0x7f29e810dab0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f29e000b0a0 tx=0x7f29e000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.772+0000 7f29eebba700 1 -- 172.21.15.46:0/1072934230 shutdown_connections 2024-03-18T20:49:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.772+0000 7f29eebba700 1 --2- 172.21.15.46:0/1072934230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e8068aa0 0x7f29e810dab0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.772+0000 7f29eebba700 1 --2- 172.21.15.46:0/1072934230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e8106e10 0x7f29e81071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.772+0000 7f29eebba700 1 --2- 172.21.15.46:0/1072934230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29e80680e0 0x7f29e8068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.772+0000 7f29eebba700 1 -- 172.21.15.46:0/1072934230 >> 172.21.15.46:0/1072934230 conn(0x7f29e8076d50 msgr2=0x7f29e8077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.772+0000 7f29eebba700 1 -- 172.21.15.46:0/1072934230 shutdown_connections 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.772+0000 7f29eebba700 1 -- 172.21.15.46:0/1072934230 wait complete. 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.773+0000 7f29eebba700 1 Processor -- start 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.773+0000 7f29eebba700 1 -- start start 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.773+0000 7f29eebba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e80680e0 0x7f29e80ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.773+0000 7f29eebba700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29e8068aa0 0x7f29e8100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.773+0000 7f29eebba700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e8106e10 0x7f29e81006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.773+0000 7f29eebba700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e81a55c0 con 0x7f29e80680e0 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.773+0000 7f29eebba700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e81a5730 con 0x7f29e8068aa0 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.773+0000 7f29eebba700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e81a58f0 con 0x7f29e8106e10 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.773+0000 7f29ec956700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e80680e0 0x7f29e80ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.773+0000 7f29ec956700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e80680e0 0x7f29e80ffc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59238/0 (socket says 172.21.15.46:59238) 2024-03-18T20:49:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.773+0000 7f29ec956700 1 -- 172.21.15.46:0/1616587014 learned_addr learned my addr 172.21.15.46:0/1616587014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29ed157700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e8106e10 0x7f29e81006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29e7fff700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29e8068aa0 0x7f29e8100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29ec956700 1 -- 172.21.15.46:0/1616587014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e8106e10 msgr2=0x7f29e81006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29ec956700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e8106e10 0x7f29e81006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29ec956700 1 -- 172.21.15.46:0/1616587014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29e8068aa0 msgr2=0x7f29e8100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29ec956700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29e8068aa0 0x7f29e8100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29ec956700 1 -- 172.21.15.46:0/1616587014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e0009d00 con 0x7f29e80680e0 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29e7fff700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29e8068aa0 0x7f29e8100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29ec956700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e80680e0 0x7f29e80ffc20 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f29d800d950 tx=0x7f29d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29e5ffb700 1 -- 172.21.15.46:0/1616587014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8009940 con 0x7f29e80680e0 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29eebba700 1 -- 172.21.15.46:0/1616587014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e81a5bd0 con 0x7f29e80680e0 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29e5ffb700 1 -- 172.21.15.46:0/1616587014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29d8010460 con 0x7f29e80680e0 2024-03-18T20:49:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29e5ffb700 1 -- 172.21.15.46:0/1616587014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d800f620 con 0x7f29e80680e0 2024-03-18T20:49:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.774+0000 7f29eebba700 1 -- 172.21.15.46:0/1616587014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e81a60e0 con 0x7f29e80680e0 2024-03-18T20:49:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.777+0000 7f29e5ffb700 1 -- 172.21.15.46:0/1616587014 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f29d80108d0 con 0x7f29e80680e0 2024-03-18T20:49:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.777+0000 7f29e5ffb700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f29d40a1290 0x7f29d40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.778+0000 7f29e5ffb700 1 -- 172.21.15.46:0/1616587014 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f29d80ccb10 con 0x7f29e80680e0 2024-03-18T20:49:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.778+0000 7f29e7fff700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f29d40a1290 0x7f29d40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.778+0000 7f29e7fff700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f29d40a1290 0x7f29d40a3750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f29d0005950 tx=0x7f29d00058e0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.778+0000 7f29fad4fb80 1 Processor -- start 2024-03-18T20:49:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.778+0000 7f29fad4fb80 1 -- start start 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29fad4fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 0x55e42301d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29fad4fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301de90 0x55e423028870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29fad4fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e423028db0 0x55e42302b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29fad4fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e422f77930 con 0x55e42301d570 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29fad4fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e422f752f0 con 0x55e42301de90 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29fad4fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e422f70be0 con 0x55e423028db0 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29ec956700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 0x55e42301d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29ec956700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 0x55e42301d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59250/0 (socket says 172.21.15.46:59250) 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29ec956700 1 -- 172.21.15.46:0/33474306 learned_addr learned my addr 172.21.15.46:0/33474306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29ed157700 1 --2- 172.21.15.46:0/33474306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e423028db0 0x55e42302b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29e7fff700 1 --2- 172.21.15.46:0/33474306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301de90 0x55e423028870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29ec956700 1 -- 172.21.15.46:0/33474306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e423028db0 msgr2=0x55e42302b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29ec956700 1 --2- 172.21.15.46:0/33474306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e423028db0 0x55e42302b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29ec956700 1 -- 172.21.15.46:0/33474306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301de90 msgr2=0x55e423028870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29ec956700 1 --2- 172.21.15.46:0/33474306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301de90 0x55e423028870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29ec956700 1 -- 172.21.15.46:0/33474306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e422df2350 con 0x55e42301d570 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29e7fff700 1 --2- 172.21.15.46:0/33474306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301de90 0x55e423028870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.779+0000 7f29ec956700 1 --2- 172.21.15.46:0/33474306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 0x55e42301d950 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f29d80a1b20 tx=0x7f29d8099600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a7713ad95ab7ae server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.780+0000 7f29deffd700 1 -- 172.21.15.46:0/33474306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80a2050 con 0x55e42301d570 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.780+0000 7f29deffd700 1 -- 172.21.15.46:0/33474306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29d80962e0 con 0x55e42301d570 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.780+0000 7f29deffd700 1 -- 172.21.15.46:0/33474306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d809a100 con 0x55e42301d570 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.780+0000 7f29fad4fb80 1 -- 172.21.15.46:0/33474306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 msgr2=0x55e42301d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.780+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/33474306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 0x55e42301d950 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f29d80a1b20 tx=0x7f29d8099600 comp rx=0 tx=0).stop 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.780+0000 7f29fad4fb80 1 -- 172.21.15.46:0/33474306 shutdown_connections 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.780+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/33474306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 0x55e42301d950 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.780+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/33474306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e423028db0 0x55e42302b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.780+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/33474306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301de90 0x55e423028870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.780+0000 7f29fad4fb80 1 -- 172.21.15.46:0/33474306 >> 172.21.15.46:0/33474306 conn(0x55e422f941c0 msgr2=0x55e423019a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.780+0000 7f29fad4fb80 1 -- 172.21.15.46:0/33474306 shutdown_connections 2024-03-18T20:49:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.780+0000 7f29fad4fb80 1 -- 172.21.15.46:0/33474306 wait complete. 2024-03-18T20:49:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.781+0000 7f29fad4fb80 1 -- start start 2024-03-18T20:49:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.781+0000 7f29fad4fb80 1 Processor -- start 2024-03-18T20:49:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.781+0000 7f29fad4fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e42301d570 0x55e4230287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.781+0000 7f29fad4fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301de90 0x55e423025b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.781+0000 7f29fad4fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e423028db0 0x55e4230261d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.781+0000 7f29fad4fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e422f0cd20 con 0x55e42301de90 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.781+0000 7f29fad4fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e422f0c9c0 con 0x55e423028db0 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.781+0000 7f29fad4fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e422df4640 con 0x55e42301d570 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29fad4fb80 10 client.0 _ll_register_callbacks cb 0x55e422d78340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301de90 0x55e423025b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301de90 0x55e423025b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59266/0 (socket says 172.21.15.46:59266) 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29e7fff700 1 -- 172.21.15.46:0/157127680 learned_addr learned my addr 172.21.15.46:0/157127680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29ec956700 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e42301d570 0x55e4230287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29ed157700 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e423028db0 0x55e4230261d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29e7fff700 1 -- 172.21.15.46:0/157127680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e42301d570 msgr2=0x55e4230287a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29e7fff700 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e42301d570 0x55e4230287a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29e7fff700 1 -- 172.21.15.46:0/157127680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e423028db0 msgr2=0x55e4230261d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29e7fff700 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e423028db0 0x55e4230261d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29e7fff700 1 -- 172.21.15.46:0/157127680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e81a60e0 con 0x55e42301de90 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29ed157700 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e423028db0 0x55e4230261d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29ec956700 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e42301d570 0x55e4230287a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29e7fff700 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301de90 0x55e423025b50 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f29d000ff00 tx=0x7f29d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdca39bd58047fbd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29c7fff700 1 -- 172.21.15.46:0/157127680 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0017040 con 0x55e42301de90 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29c7fff700 1 -- 172.21.15.46:0/157127680 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f29d0003880 con 0x55e42301de90 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.782+0000 7f29fad4fb80 1 -- 172.21.15.46:0/157127680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4230116c0 con 0x55e42301de90 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.783+0000 7f29c7fff700 1 -- 172.21.15.46:0/157127680 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29d0003a40 con 0x55e42301de90 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.783+0000 7f29fad4fb80 10 client.20043 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.783+0000 7f29fad4fb80 1 -- 172.21.15.46:0/157127680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55e422d9eb00 con 0x55e42301de90 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.783+0000 7f29c7fff700 1 -- 172.21.15.46:0/157127680 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4230116c0 con 0x55e42301de90 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.783+0000 7f29c7fff700 1 -- 172.21.15.46:0/157127680 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f29d0011cc0 con 0x55e42301de90 2024-03-18T20:49:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.783+0000 7f29fad4fb80 10 client.20043 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.783+0000 7f29fad4fb80 10 client.20043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.783+0000 7f29fad4fb80 20 client.20043 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.783+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e423038a30 0x55e42303ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.783+0000 7f29fad4fb80 4 client.20043 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.783+0000 7f29fad4fb80 1 -- 172.21.15.46:0/157127680 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e422f696a0 con 0x55e423038a30 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.783+0000 7f29ec956700 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e423038a30 0x55e42303ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.784+0000 7f29c7fff700 10 client.20043 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.784+0000 7f29ec956700 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e423038a30 0x55e42303ae50 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1560b12a289c0bb8 server_cookie=e4ff72dfe9a35022 in_seq=0 out_seq=0 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.784+0000 7f29c7fff700 1 -- 172.21.15.46:0/157127680 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e422f696a0 con 0x55e423038a30 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.784+0000 7f29c7fff700 10 client.20043 handle_command_reply: tid=0 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29fad4fb80 1 client.20043 shutdown 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29fad4fb80 1 -- 172.21.15.46:0/157127680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301de90 msgr2=0x55e423025b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301de90 0x55e423025b50 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f29d000ff00 tx=0x7f29d0004750 comp rx=0 tx=0).stop 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29fad4fb80 1 -- 172.21.15.46:0/157127680 shutdown_connections 2024-03-18T20:49:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e423038a30 0x55e42303ae50 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e423028db0 0x55e4230261d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e42301d570 0x55e4230287a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29c7fff700 0 client.20043 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/157127680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301de90 0x55e423025b50 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29fad4fb80 1 -- 172.21.15.46:0/157127680 >> 172.21.15.46:0/157127680 conn(0x55e422f941c0 msgr2=0x55e423019650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29fad4fb80 1 -- 172.21.15.46:0/157127680 shutdown_connections 2024-03-18T20:49:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29fad4fb80 1 -- 172.21.15.46:0/157127680 wait complete. 2024-03-18T20:49:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.785+0000 7f29fad4fb80 20 client.20043 trim_cache size 0 max 16384 2024-03-18T20:49:08.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:08 smithi155 ceph-mon[48344]: pgmap v407: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-03-18T20:49:08.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:08 smithi046 ceph-mon[43495]: pgmap v407: 97 pgs: 97 active+clean; 220 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-03-18T20:49:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.800+0000 7f29fad4fb80 1 Processor -- start 2024-03-18T20:49:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.800+0000 7f29fad4fb80 1 -- start start 2024-03-18T20:49:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29fad4fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301d570 0x55e42302b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29fad4fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e42301de90 0x55e42302f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29fad4fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e423028db0 0x55e423020eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29fad4fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42302ff20 con 0x55e423028db0 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29fad4fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4230204e0 con 0x55e42301d570 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29fad4fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4230240e0 con 0x55e42301de90 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29ed157700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e423028db0 0x55e423020eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29ed157700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e423028db0 0x55e423020eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59272/0 (socket says 172.21.15.46:59272) 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29ed157700 1 -- 172.21.15.46:0/3781605821 learned_addr learned my addr 172.21.15.46:0/3781605821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29ec956700 1 --2- 172.21.15.46:0/3781605821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301d570 0x55e42302b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29e7fff700 1 --2- 172.21.15.46:0/3781605821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e42301de90 0x55e42302f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29ed157700 1 -- 172.21.15.46:0/3781605821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e42301de90 msgr2=0x55e42302f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29ed157700 1 --2- 172.21.15.46:0/3781605821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e42301de90 0x55e42302f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29ed157700 1 -- 172.21.15.46:0/3781605821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301d570 msgr2=0x55e42302b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29ed157700 1 --2- 172.21.15.46:0/3781605821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301d570 0x55e42302b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29ed157700 1 -- 172.21.15.46:0/3781605821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e422f36e40 con 0x55e423028db0 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.801+0000 7f29ed157700 1 --2- 172.21.15.46:0/3781605821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e423028db0 0x55e423020eb0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f29e0007c30 tx=0x7f29e0007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecc9359b68b1e44 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.802+0000 7f29df7fe700 1 -- 172.21.15.46:0/3781605821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e000e040 con 0x55e423028db0 2024-03-18T20:49:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.802+0000 7f29df7fe700 1 -- 172.21.15.46:0/3781605821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29e0021070 con 0x55e423028db0 2024-03-18T20:49:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.802+0000 7f29df7fe700 1 -- 172.21.15.46:0/3781605821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e0007940 con 0x55e423028db0 2024-03-18T20:49:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.802+0000 7f29fad4fb80 1 -- 172.21.15.46:0/3781605821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e423028db0 msgr2=0x55e423020eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.802+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/3781605821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e423028db0 0x55e423020eb0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f29e0007c30 tx=0x7f29e0007c60 comp rx=0 tx=0).stop 2024-03-18T20:49:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.802+0000 7f29fad4fb80 1 -- 172.21.15.46:0/3781605821 shutdown_connections 2024-03-18T20:49:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.802+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/3781605821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e423028db0 0x55e423020eb0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.802+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/3781605821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301d570 0x55e42302b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.802+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/3781605821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e42301de90 0x55e42302f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.802+0000 7f29fad4fb80 1 -- 172.21.15.46:0/3781605821 >> 172.21.15.46:0/3781605821 conn(0x55e422f941c0 msgr2=0x55e423015620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.802+0000 7f29fad4fb80 1 -- 172.21.15.46:0/3781605821 shutdown_connections 2024-03-18T20:49:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.802+0000 7f29fad4fb80 1 -- 172.21.15.46:0/3781605821 wait complete. 2024-03-18T20:49:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29fad4fb80 1 -- start start 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29fad4fb80 1 Processor -- start 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29fad4fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 0x55e42302d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29fad4fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301de90 0x55e42302df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29fad4fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e423028db0 0x55e42302e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29fad4fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42308b820 con 0x55e42301d570 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29fad4fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42308b990 con 0x55e42301de90 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29fad4fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e42308bb50 con 0x55e423028db0 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29fad4fb80 10 client.0 _ll_register_callbacks cb 0x7f29d000be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29ec956700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 0x55e42302d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29ec956700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 0x55e42302d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59278/0 (socket says 172.21.15.46:59278) 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29ec956700 1 -- 172.21.15.46:0/1126359861 learned_addr learned my addr 172.21.15.46:0/1126359861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29ed157700 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e423028db0 0x55e42302e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.803+0000 7f29e7fff700 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301de90 0x55e42302df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29ec956700 1 -- 172.21.15.46:0/1126359861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e423028db0 msgr2=0x55e42302e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29ec956700 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e423028db0 0x55e42302e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29ec956700 1 -- 172.21.15.46:0/1126359861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301de90 msgr2=0x55e42302df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29ec956700 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301de90 0x55e42302df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29ec956700 1 -- 172.21.15.46:0/1126359861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e810e440 con 0x55e42301d570 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29ec956700 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 0x55e42302d9c0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f29d8099b60 tx=0x7f29d8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef437272ad1ebcb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29c7fff700 1 -- 172.21.15.46:0/1126359861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8096450 con 0x55e42301d570 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29c7fff700 1 -- 172.21.15.46:0/1126359861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f29d80b2070 con 0x55e42301d570 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29fad4fb80 1 -- 172.21.15.46:0/1126359861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29d80962e0 con 0x55e42301d570 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29c7fff700 1 -- 172.21.15.46:0/1126359861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29d809adf0 con 0x55e42301d570 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29fad4fb80 10 client.20049 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29fad4fb80 1 -- 172.21.15.46:0/1126359861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55e422df6c70 con 0x55e42301d570 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.804+0000 7f29c7fff700 1 -- 172.21.15.46:0/1126359861 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29d80962e0 con 0x55e42301d570 2024-03-18T20:49:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.805+0000 7f29c7fff700 1 -- 172.21.15.46:0/1126359861 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f29d80a1bc0 con 0x55e42301d570 2024-03-18T20:49:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.805+0000 7f29fad4fb80 10 client.20049 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.805+0000 7f29fad4fb80 10 client.20049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:49:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.805+0000 7f29fad4fb80 20 client.20049 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.805+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e42308d070 0x55e42308f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.805+0000 7f29fad4fb80 4 client.20049 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:49:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.805+0000 7f29fad4fb80 1 -- 172.21.15.46:0/1126359861 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e422dbfc90 con 0x55e42308d070 2024-03-18T20:49:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.805+0000 7f29e7fff700 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e42308d070 0x55e42308f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.805+0000 7f29e7fff700 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e42308d070 0x55e42308f490 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06f577679f6c1c server_cookie=ec7570b472a2b65c in_seq=0 out_seq=0 2024-03-18T20:49:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.805+0000 7f29c7fff700 10 client.20049 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.837+0000 7f29c7fff700 1 -- 172.21.15.46:0/1126359861 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e422dbfc90 con 0x55e42308d070 2024-03-18T20:49:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.837+0000 7f29c7fff700 10 client.20049 handle_command_reply: tid=0 2024-03-18T20:49:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.837+0000 7f29fad4fb80 1 client.20049 shutdown 2024-03-18T20:49:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.837+0000 7f29fad4fb80 1 -- 172.21.15.46:0/1126359861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 msgr2=0x55e42302d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.837+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 0x55e42302d9c0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f29d8099b60 tx=0x7f29d8000bc0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.837+0000 7f29fad4fb80 1 -- 172.21.15.46:0/1126359861 shutdown_connections 2024-03-18T20:49:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.837+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e42308d070 0x55e42308f490 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.837+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e42301d570 0x55e42302d9c0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.837+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e423028db0 0x55e42302e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.837+0000 7f29c7fff700 0 client.20049 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.837+0000 7f29fad4fb80 1 --2- 172.21.15.46:0/1126359861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e42301de90 0x55e42302df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.837+0000 7f29fad4fb80 1 -- 172.21.15.46:0/1126359861 >> 172.21.15.46:0/1126359861 conn(0x55e422f941c0 msgr2=0x55e423024400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.838+0000 7f29fad4fb80 1 -- 172.21.15.46:0/1126359861 shutdown_connections 2024-03-18T20:49:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.838+0000 7f29fad4fb80 1 -- 172.21.15.46:0/1126359861 wait complete. 2024-03-18T20:49:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.838+0000 7f29fad4fb80 20 client.20049 trim_cache size 0 max 16384 2024-03-18T20:49:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:49:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:49:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:49:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:49:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.839+0000 7f29eebba700 1 -- 172.21.15.46:0/1616587014 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f29d40a1290 msgr2=0x7f29d40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.839+0000 7f29eebba700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f29d40a1290 0x7f29d40a3750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f29d0005950 tx=0x7f29d00058e0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.839+0000 7f29eebba700 1 -- 172.21.15.46:0/1616587014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e80680e0 msgr2=0x7f29e80ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.839+0000 7f29eebba700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e80680e0 0x7f29e80ffc20 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f29d800d950 tx=0x7f29d800dc60 comp rx=0 tx=0).stop 2024-03-18T20:49:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.839+0000 7f29eebba700 1 -- 172.21.15.46:0/1616587014 shutdown_connections 2024-03-18T20:49:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.839+0000 7f29eebba700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f29d40a1290 0x7f29d40a3750 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.839+0000 7f29eebba700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e80680e0 0x7f29e80ffc20 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.839+0000 7f29eebba700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29e8106e10 0x7f29e81006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.839+0000 7f29eebba700 1 --2- 172.21.15.46:0/1616587014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29e8068aa0 0x7f29e8100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.839+0000 7f29eebba700 1 -- 172.21.15.46:0/1616587014 >> 172.21.15.46:0/1616587014 conn(0x7f29e8076d50 msgr2=0x7f29e80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.839+0000 7f29eebba700 1 -- 172.21.15.46:0/1616587014 shutdown_connections 2024-03-18T20:49:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:08.839+0000 7f29eebba700 1 -- 172.21.15.46:0/1616587014 wait complete. 2024-03-18T20:49:09.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T20:49:09.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T20:49:09.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.604+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3611622607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94101900 msgr2=0x7f3d94105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.604+0000 7f3d9bb80700 1 --2- 172.21.15.46:0/3611622607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94101900 0x7f3d94105940 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009990 tx=0x7f3d88009ca0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.604+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3611622607 shutdown_connections 2024-03-18T20:49:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.605+0000 7f3d9bb80700 1 --2- 172.21.15.46:0/3611622607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d94106070 0x7f3d9410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.605+0000 7f3d9bb80700 1 --2- 172.21.15.46:0/3611622607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d94100f50 0x7f3d94101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.605+0000 7f3d9bb80700 1 --2- 172.21.15.46:0/3611622607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94101900 0x7f3d94105940 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.605+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3611622607 >> 172.21.15.46:0/3611622607 conn(0x7f3d940fc7f0 msgr2=0x7f3d940fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.605+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3611622607 shutdown_connections 2024-03-18T20:49:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.605+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3611622607 wait complete. 2024-03-18T20:49:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.605+0000 7f3d9bb80700 1 Processor -- start 2024-03-18T20:49:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9bb80700 1 -- start start 2024-03-18T20:49:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9bb80700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d94100f50 0x7f3d941971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9bb80700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94106070 0x7f3d941976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9bb80700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9419ba90 0x7f3d9419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9bb80700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9419c510 con 0x7f3d94106070 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9bb80700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9419c680 con 0x7f3d94100f50 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9bb80700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9419c840 con 0x7f3d9419ba90 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9911b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94106070 0x7f3d941976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9911b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94106070 0x7f3d941976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59306/0 (socket says 172.21.15.46:59306) 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9911b700 1 -- 172.21.15.46:0/3262920408 learned_addr learned my addr 172.21.15.46:0/3262920408 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9991c700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d94100f50 0x7f3d941971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9a11d700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9419ba90 0x7f3d9419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9911b700 1 -- 172.21.15.46:0/3262920408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9419ba90 msgr2=0x7f3d9419bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9911b700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9419ba90 0x7f3d9419bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9911b700 1 -- 172.21.15.46:0/3262920408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d94100f50 msgr2=0x7f3d941971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9911b700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d94100f50 0x7f3d941971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.606+0000 7f3d9911b700 1 -- 172.21.15.46:0/3262920408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d880095f0 con 0x7f3d94106070 2024-03-18T20:49:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.607+0000 7f3d9911b700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94106070 0x7f3d941976f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f3d88000c00 tx=0x7f3d88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.607+0000 7f3d86ffd700 1 -- 172.21.15.46:0/3262920408 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8801d070 con 0x7f3d94106070 2024-03-18T20:49:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.607+0000 7f3d86ffd700 1 -- 172.21.15.46:0/3262920408 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d88021410 con 0x7f3d94106070 2024-03-18T20:49:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.607+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3262920408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d941a3d40 con 0x7f3d94106070 2024-03-18T20:49:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.607+0000 7f3d86ffd700 1 -- 172.21.15.46:0/3262920408 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88017650 con 0x7f3d94106070 2024-03-18T20:49:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.607+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3262920408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d941a4160 con 0x7f3d94106070 2024-03-18T20:49:09.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.610+0000 7f3d86ffd700 1 -- 172.21.15.46:0/3262920408 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3d88021580 con 0x7f3d94106070 2024-03-18T20:49:09.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.611+0000 7f3d86ffd700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3d800a1290 0x7f3d800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.611+0000 7f3d86ffd700 1 -- 172.21.15.46:0/3262920408 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f3d880cdfd0 con 0x7f3d94106070 2024-03-18T20:49:09.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.611+0000 7f3d9991c700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3d800a1290 0x7f3d800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3da7d15b80 1 Processor -- start 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3d9991c700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3d800a1290 0x7f3d800a3750 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009c60 tx=0x7f3d90009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3da7d15b80 1 -- start start 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3da7d15b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6ea60 0x55578ea6ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3da7d15b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea7e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3da7d15b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea7e630 0x55578ea80a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3da7d15b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578e9c8990 con 0x55578ea6ea60 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3da7d15b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578e9c6350 con 0x55578ea6f380 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3da7d15b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578e9c1c40 con 0x55578ea7e630 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3d9991c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6ea60 0x55578ea6ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3d9991c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6ea60 0x55578ea6ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59318/0 (socket says 172.21.15.46:59318) 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3d9991c700 1 -- 172.21.15.46:0/2364079161 learned_addr learned my addr 172.21.15.46:0/2364079161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3d9a11d700 1 --2- 172.21.15.46:0/2364079161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea7e630 0x55578ea80a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3d9991c700 1 -- 172.21.15.46:0/2364079161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea7e630 msgr2=0x55578ea80a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3d9911b700 1 --2- 172.21.15.46:0/2364079161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea7e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.612+0000 7f3d9991c700 1 --2- 172.21.15.46:0/2364079161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea7e630 0x55578ea80a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.613+0000 7f3d9991c700 1 -- 172.21.15.46:0/2364079161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 msgr2=0x55578ea7e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.613+0000 7f3d9991c700 1 --2- 172.21.15.46:0/2364079161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea7e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.613+0000 7f3d9991c700 1 -- 172.21.15.46:0/2364079161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55578e8433b0 con 0x55578ea6ea60 2024-03-18T20:49:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.613+0000 7f3d9911b700 1 --2- 172.21.15.46:0/2364079161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea7e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:49:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.613+0000 7f3d9991c700 1 --2- 172.21.15.46:0/2364079161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6ea60 0x55578ea6ee40 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f3d90004170 tx=0x7f3d90014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc4e67f72e03b4b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.613+0000 7f3d7ffff700 1 -- 172.21.15.46:0/2364079161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90003d10 con 0x55578ea6ea60 2024-03-18T20:49:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.613+0000 7f3d7ffff700 1 -- 172.21.15.46:0/2364079161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d90003e70 con 0x55578ea6ea60 2024-03-18T20:49:09.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.613+0000 7f3d7ffff700 1 -- 172.21.15.46:0/2364079161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9001d610 con 0x55578ea6ea60 2024-03-18T20:49:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.614+0000 7f3da7d15b80 1 -- 172.21.15.46:0/2364079161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6ea60 msgr2=0x55578ea6ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.614+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/2364079161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6ea60 0x55578ea6ee40 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f3d90004170 tx=0x7f3d90014000 comp rx=0 tx=0).stop 2024-03-18T20:49:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.614+0000 7f3da7d15b80 1 -- 172.21.15.46:0/2364079161 shutdown_connections 2024-03-18T20:49:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.614+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/2364079161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6ea60 0x55578ea6ee40 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.614+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/2364079161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea7e630 0x55578ea80a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.614+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/2364079161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea7e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.614+0000 7f3da7d15b80 1 -- 172.21.15.46:0/2364079161 >> 172.21.15.46:0/2364079161 conn(0x55578e9e5220 msgr2=0x55578ea6b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.614+0000 7f3da7d15b80 1 -- 172.21.15.46:0/2364079161 shutdown_connections 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.614+0000 7f3da7d15b80 1 -- 172.21.15.46:0/2364079161 wait complete. 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.614+0000 7f3da7d15b80 1 -- start start 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.614+0000 7f3da7d15b80 1 Processor -- start 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3da7d15b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea7e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3da7d15b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea7b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3da7d15b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 0x55578ea7baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3da7d15b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578e95dd80 con 0x55578ea7e630 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3da7d15b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578e95da20 con 0x55578ea6f380 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3da7d15b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578e8456a0 con 0x55578ea6ea60 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3da7d15b80 10 client.0 _ll_register_callbacks cb 0x55578e7c93a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3d9a11d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 0x55578ea7baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3d9a11d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 0x55578ea7baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59332/0 (socket says 172.21.15.46:59332) 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3d9a11d700 1 -- 172.21.15.46:0/3930561416 learned_addr learned my addr 172.21.15.46:0/3930561416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3d9991c700 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea7e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3d9911b700 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea7b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3d9a11d700 1 -- 172.21.15.46:0/3930561416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 msgr2=0x55578ea7e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3d9a11d700 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea7e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3d9a11d700 1 -- 172.21.15.46:0/3930561416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 msgr2=0x55578ea7b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3d9a11d700 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea7b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.615+0000 7f3d9a11d700 1 -- 172.21.15.46:0/3930561416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d90017070 con 0x55578ea7e630 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3d9a11d700 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 0x55578ea7baa0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c011cc0 tx=0x7f3d8c00f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361ae870e8b9be76 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3d7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3d7d7fa700 1 -- 172.21.15.46:0/3930561416 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c011e60 con 0x55578ea7e630 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3d7d7fa700 1 -- 172.21.15.46:0/3930561416 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f3d8c00fcd0 con 0x55578ea7e630 2024-03-18T20:49:09.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3930561416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55578ea62ad0 con 0x55578ea7e630 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3d7d7fa700 1 -- 172.21.15.46:0/3930561416 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d8c00fed0 con 0x55578ea7e630 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3da7d15b80 10 client.20061 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3930561416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55578e7efb60 con 0x55578ea7e630 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3d7d7fa700 1 -- 172.21.15.46:0/3930561416 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55578ea62ad0 con 0x55578ea7e630 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3d7d7fa700 1 -- 172.21.15.46:0/3930561416 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f3d8c015550 con 0x55578ea7e630 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3da7d15b80 10 client.20061 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3da7d15b80 10 client.20061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3da7d15b80 20 client.20061 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55578ea8e300 0x55578ea90720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3da7d15b80 4 client.20061 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.616+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3930561416 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55578e9ba700 con 0x55578ea8e300 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.617+0000 7f3d9991c700 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55578ea8e300 0x55578ea90720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.617+0000 7f3d7d7fa700 10 client.20061 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.617+0000 7f3d9991c700 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55578ea8e300 0x55578ea90720 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fca1583b96e3d6c server_cookie=deeae965a55a1ea2 in_seq=0 out_seq=0 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3d7d7fa700 1 -- 172.21.15.46:0/3930561416 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55578e9ba700 con 0x55578ea8e300 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3d7d7fa700 10 client.20061 handle_command_reply: tid=0 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3da7d15b80 1 client.20061 shutdown 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3930561416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 msgr2=0x55578ea7baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 0x55578ea7baa0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c011cc0 tx=0x7f3d8c00f4b0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3930561416 shutdown_connections 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55578ea8e300 0x55578ea90720 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 0x55578ea7baa0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea7e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/3930561416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea7b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3d7d7fa700 0 client.20061 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3930561416 >> 172.21.15.46:0/3930561416 conn(0x55578e9e5220 msgr2=0x55578ea6ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3930561416 shutdown_connections 2024-03-18T20:49:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3930561416 wait complete. 2024-03-18T20:49:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.618+0000 7f3da7d15b80 20 client.20061 trim_cache size 0 max 16384 2024-03-18T20:49:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3da7d15b80 1 Processor -- start 2024-03-18T20:49:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3da7d15b80 1 -- start start 2024-03-18T20:49:09.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3da7d15b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea85260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3da7d15b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6f380 0x55578ea7dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3da7d15b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea7e630 0x55578ea6ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3da7d15b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578ea7c300 con 0x55578ea6f380 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3da7d15b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578ea65240 con 0x55578ea7e630 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3da7d15b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578ea675e0 con 0x55578ea6ea60 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3d9911b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6f380 0x55578ea7dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3d9911b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6f380 0x55578ea7dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59348/0 (socket says 172.21.15.46:59348) 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3d9911b700 1 -- 172.21.15.46:0/213037804 learned_addr learned my addr 172.21.15.46:0/213037804 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3d9991c700 1 --2- 172.21.15.46:0/213037804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea85260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3d9a11d700 1 --2- 172.21.15.46:0/213037804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea7e630 0x55578ea6ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3d9911b700 1 -- 172.21.15.46:0/213037804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 msgr2=0x55578ea85260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3d9911b700 1 --2- 172.21.15.46:0/213037804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea85260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3d9911b700 1 -- 172.21.15.46:0/213037804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea7e630 msgr2=0x55578ea6ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3d9911b700 1 --2- 172.21.15.46:0/213037804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea7e630 0x55578ea6ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3d9911b700 1 -- 172.21.15.46:0/213037804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55578e987ea0 con 0x55578ea6f380 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3d9991c700 1 --2- 172.21.15.46:0/213037804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea85260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.636+0000 7f3d9a11d700 1 --2- 172.21.15.46:0/213037804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea7e630 0x55578ea6ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.637+0000 7f3d9911b700 1 --2- 172.21.15.46:0/213037804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6f380 0x55578ea7dca0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f3d880cb6d0 tx=0x7f3d8809afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3284e8f49efd29 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.637+0000 7f3d84da8700 1 -- 172.21.15.46:0/213037804 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8809e070 con 0x55578ea6f380 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.637+0000 7f3d84da8700 1 -- 172.21.15.46:0/213037804 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d8809b0c0 con 0x55578ea6f380 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.637+0000 7f3d84da8700 1 -- 172.21.15.46:0/213037804 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8809b320 con 0x55578ea6f380 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.637+0000 7f3da7d15b80 1 -- 172.21.15.46:0/213037804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6f380 msgr2=0x55578ea7dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.637+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/213037804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6f380 0x55578ea7dca0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f3d880cb6d0 tx=0x7f3d8809afb0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.637+0000 7f3da7d15b80 1 -- 172.21.15.46:0/213037804 shutdown_connections 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.637+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/213037804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea7e630 0x55578ea6ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.637+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/213037804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea85260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.637+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/213037804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea6f380 0x55578ea7dca0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.637+0000 7f3da7d15b80 1 -- 172.21.15.46:0/213037804 >> 172.21.15.46:0/213037804 conn(0x55578e9e5220 msgr2=0x55578ea69750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:09.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.637+0000 7f3da7d15b80 1 -- 172.21.15.46:0/213037804 shutdown_connections 2024-03-18T20:49:09.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.638+0000 7f3da7d15b80 1 -- 172.21.15.46:0/213037804 wait complete. 2024-03-18T20:49:09.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.638+0000 7f3da7d15b80 1 -- start start 2024-03-18T20:49:09.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.638+0000 7f3da7d15b80 1 Processor -- start 2024-03-18T20:49:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.638+0000 7f3da7d15b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea83430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.638+0000 7f3da7d15b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea83970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.638+0000 7f3da7d15b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 0x55578eae10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.638+0000 7f3da7d15b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578eae1630 con 0x55578ea7e630 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.638+0000 7f3da7d15b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578eae1770 con 0x55578ea6f380 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.638+0000 7f3da7d15b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578eae18b0 con 0x55578ea6ea60 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.638+0000 7f3da7d15b80 10 client.0 _ll_register_callbacks cb 0x7f3d8c0115f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d9a11d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 0x55578eae10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d9a11d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 0x55578eae10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59360/0 (socket says 172.21.15.46:59360) 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d9a11d700 1 -- 172.21.15.46:0/3668488658 learned_addr learned my addr 172.21.15.46:0/3668488658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d9991c700 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea83430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d9911b700 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea83970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d9a11d700 1 -- 172.21.15.46:0/3668488658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 msgr2=0x55578ea83430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d9a11d700 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea83430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d9a11d700 1 -- 172.21.15.46:0/3668488658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 msgr2=0x55578ea83970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d9a11d700 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea83970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d9a11d700 1 -- 172.21.15.46:0/3668488658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d941a4160 con 0x55578ea7e630 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d9a11d700 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 0x55578eae10f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c000f80 tx=0x7f3d8c015d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9387a45f0f2790 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d7d7fa700 1 -- 172.21.15.46:0/3668488658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c00f8b0 con 0x55578ea7e630 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d7d7fa700 1 -- 172.21.15.46:0/3668488658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f3d8c011e60 con 0x55578ea7e630 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3668488658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d90003e70 con 0x55578ea7e630 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3d7d7fa700 1 -- 172.21.15.46:0/3668488658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d8c007480 con 0x55578ea7e630 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3da7d15b80 10 client.20067 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.639+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3668488658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55578e847cd0 con 0x55578ea7e630 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.640+0000 7f3d7d7fa700 1 -- 172.21.15.46:0/3668488658 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d90003e70 con 0x55578ea7e630 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.640+0000 7f3d7d7fa700 1 -- 172.21.15.46:0/3668488658 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f3d8c017940 con 0x55578ea7e630 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.640+0000 7f3da7d15b80 10 client.20067 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.640+0000 7f3da7d15b80 10 client.20067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.640+0000 7f3da7d15b80 20 client.20067 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.640+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55578eae3030 0x55578eae5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.640+0000 7f3da7d15b80 4 client.20067 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.640+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3668488658 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55578e810cf0 con 0x55578eae3030 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.640+0000 7f3d9991c700 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55578eae3030 0x55578eae5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.640+0000 7f3d7d7fa700 10 client.20067 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.640+0000 7f3d9991c700 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55578eae3030 0x55578eae5450 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd31dea475929b9 server_cookie=2de0cfa24d3cded5 in_seq=0 out_seq=0 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3d7d7fa700 1 -- 172.21.15.46:0/3668488658 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55578e810cf0 con 0x55578eae3030 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3d7d7fa700 10 client.20067 handle_command_reply: tid=0 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3da7d15b80 1 client.20067 shutdown 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3668488658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 msgr2=0x55578eae10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 0x55578eae10f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c000f80 tx=0x7f3d8c015d00 comp rx=0 tx=0).stop 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3668488658 shutdown_connections 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55578eae3030 0x55578eae5450 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578ea7e630 0x55578eae10f0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55578ea6ea60 0x55578ea83430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3da7d15b80 1 --2- 172.21.15.46:0/3668488658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55578ea6f380 0x55578ea83970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3d7d7fa700 0 client.20067 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3668488658 >> 172.21.15.46:0/3668488658 conn(0x55578e9e5220 msgr2=0x55578ea76690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3668488658 shutdown_connections 2024-03-18T20:49:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3da7d15b80 1 -- 172.21.15.46:0/3668488658 wait complete. 2024-03-18T20:49:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.641+0000 7f3da7d15b80 20 client.20067 trim_cache size 0 max 16384 2024-03-18T20:49:09.650 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T20:49:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-18T20:49:09.650 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-18T20:49:09.650 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T20:49:09.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.642+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3262920408 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3d800a1290 msgr2=0x7f3d800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:09.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.642+0000 7f3d9bb80700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3d800a1290 0x7f3d800a3750 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009c60 tx=0x7f3d90009260 comp rx=0 tx=0).stop 2024-03-18T20:49:09.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.642+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3262920408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94106070 msgr2=0x7f3d941976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:09.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.642+0000 7f3d9bb80700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94106070 0x7f3d941976f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f3d88000c00 tx=0x7f3d88004730 comp rx=0 tx=0).stop 2024-03-18T20:49:09.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.642+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3262920408 shutdown_connections 2024-03-18T20:49:09.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.642+0000 7f3d9bb80700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3d800a1290 0x7f3d800a3750 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.642+0000 7f3d9bb80700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9419ba90 0x7f3d9419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.642+0000 7f3d9bb80700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d94100f50 0x7f3d941971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.642+0000 7f3d9bb80700 1 --2- 172.21.15.46:0/3262920408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d94106070 0x7f3d941976f0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.642+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3262920408 >> 172.21.15.46:0/3262920408 conn(0x7f3d940fc7f0 msgr2=0x7f3d940feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:09.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.642+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3262920408 shutdown_connections 2024-03-18T20:49:09.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.642+0000 7f3d9bb80700 1 -- 172.21.15.46:0/3262920408 wait complete. 2024-03-18T20:49:09.657 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T20:49:09.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T20:49:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 604 sec 2024-03-18T20:49:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.793+0000 7f722848d700 1 -- 172.21.15.46:0/1486246672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72200ffb00 msgr2=0x7f72200fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.793+0000 7f722848d700 1 --2- 172.21.15.46:0/1486246672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72200ffb00 0x7f72200fff80 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f7210009990 tx=0x7f7210009ca0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.793+0000 7f722848d700 1 -- 172.21.15.46:0/1486246672 shutdown_connections 2024-03-18T20:49:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.793+0000 7f722848d700 1 --2- 172.21.15.46:0/1486246672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72201004c0 0x7f722010c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.793+0000 7f722848d700 1 --2- 172.21.15.46:0/1486246672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7220105ba0 0x7f7220105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.793+0000 7f722848d700 1 --2- 172.21.15.46:0/1486246672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72200ffb00 0x7f72200fff80 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.793+0000 7f722848d700 1 -- 172.21.15.46:0/1486246672 >> 172.21.15.46:0/1486246672 conn(0x7f72200fb5f0 msgr2=0x7f72200fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:09.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.793+0000 7f722848d700 1 -- 172.21.15.46:0/1486246672 shutdown_connections 2024-03-18T20:49:09.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.793+0000 7f722848d700 1 -- 172.21.15.46:0/1486246672 wait complete. 2024-03-18T20:49:09.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f722848d700 1 Processor -- start 2024-03-18T20:49:09.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f722848d700 1 -- start start 2024-03-18T20:49:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f722848d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72200ffb00 0x7f72200fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f722848d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72201004c0 0x7f72200fc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f722848d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7220105ba0 0x7f72200fc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f722848d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72201a5420 con 0x7f7220105ba0 2024-03-18T20:49:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f722848d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72201a5590 con 0x7f72201004c0 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f722848d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72201a5750 con 0x7f72200ffb00 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f7226a2a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7220105ba0 0x7f72200fc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f7226229700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72200ffb00 0x7f72200fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f7226a2a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7220105ba0 0x7f72200fc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59364/0 (socket says 172.21.15.46:59364) 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f7226a2a700 1 -- 172.21.15.46:0/265807015 learned_addr learned my addr 172.21.15.46:0/265807015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.794+0000 7f7225a28700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72201004c0 0x7f72200fc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.795+0000 7f7226a2a700 1 -- 172.21.15.46:0/265807015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72200ffb00 msgr2=0x7f72200fbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.795+0000 7f7226a2a700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72200ffb00 0x7f72200fbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.795+0000 7f7226a2a700 1 -- 172.21.15.46:0/265807015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72201004c0 msgr2=0x7f72200fc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.795+0000 7f7226a2a700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72201004c0 0x7f72200fc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.795+0000 7f7226a2a700 1 -- 172.21.15.46:0/265807015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72100095f0 con 0x7f7220105ba0 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.795+0000 7f7226a2a700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7220105ba0 0x7f72200fc730 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f7218007ee0 tx=0x7f721800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.795+0000 7f72177fe700 1 -- 172.21.15.46:0/265807015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721800dd20 con 0x7f7220105ba0 2024-03-18T20:49:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.795+0000 7f72177fe700 1 -- 172.21.15.46:0/265807015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f721800f040 con 0x7f7220105ba0 2024-03-18T20:49:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.795+0000 7f722848d700 1 -- 172.21.15.46:0/265807015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72201a5a30 con 0x7f7220105ba0 2024-03-18T20:49:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.795+0000 7f72177fe700 1 -- 172.21.15.46:0/265807015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72180048a0 con 0x7f7220105ba0 2024-03-18T20:49:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.795+0000 7f722848d700 1 -- 172.21.15.46:0/265807015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72201a5f10 con 0x7f7220105ba0 2024-03-18T20:49:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.796+0000 7f722848d700 1 -- 172.21.15.46:0/265807015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72201090c0 con 0x7f7220105ba0 2024-03-18T20:49:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.797+0000 7f72177fe700 1 -- 172.21.15.46:0/265807015 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7218003a00 con 0x7f7220105ba0 2024-03-18T20:49:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.798+0000 7f72177fe700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f720c0a1310 0x7f720c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.798+0000 7f72177fe700 1 -- 172.21.15.46:0/265807015 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f72180ccdb0 con 0x7f7220105ba0 2024-03-18T20:49:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.798+0000 7f7226229700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f720c0a1310 0x7f720c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.798+0000 7f7226229700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f720c0a1310 0x7f720c0a37d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f721c009be0 tx=0x7f721c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:09.802+0000 7f72177fe700 1 -- 172.21.15.46:0/265807015 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7218096150 con 0x7f7220105ba0 2024-03-18T20:49:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.077+0000 7f722848d700 1 -- 172.21.15.46:0/265807015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7220101200 con 0x7f7220105ba0 2024-03-18T20:49:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.078+0000 7f72177fe700 1 -- 172.21.15.46:0/265807015 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+10931 (secure 0 0 0) 0x7f7218023020 con 0x7f7220105ba0 2024-03-18T20:49:10.081 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T20:49:10.081 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T20:49:04.298382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":218,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":215,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[0,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25730":{"gid":25730,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/3087491981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":3087491981},{"type":"v1","addr":"172.21.15.155:6837","nonce":3087491981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T20:49:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.081+0000 7f722848d700 1 -- 172.21.15.46:0/265807015 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f720c0a1310 msgr2=0x7f720c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.081+0000 7f722848d700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f720c0a1310 0x7f720c0a37d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f721c009be0 tx=0x7f721c009260 comp rx=0 tx=0).stop 2024-03-18T20:49:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.081+0000 7f722848d700 1 -- 172.21.15.46:0/265807015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7220105ba0 msgr2=0x7f72200fc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.081+0000 7f722848d700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7220105ba0 0x7f72200fc730 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f7218007ee0 tx=0x7f721800d3b0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.081+0000 7f722848d700 1 -- 172.21.15.46:0/265807015 shutdown_connections 2024-03-18T20:49:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.081+0000 7f722848d700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f720c0a1310 0x7f720c0a37d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.081+0000 7f722848d700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7220105ba0 0x7f72200fc730 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.081+0000 7f722848d700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72200ffb00 0x7f72200fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.081+0000 7f722848d700 1 --2- 172.21.15.46:0/265807015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72201004c0 0x7f72200fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.081+0000 7f722848d700 1 -- 172.21.15.46:0/265807015 >> 172.21.15.46:0/265807015 conn(0x7f72200fb5f0 msgr2=0x7f72200691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.081+0000 7f722848d700 1 -- 172.21.15.46:0/265807015 shutdown_connections 2024-03-18T20:49:10.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.081+0000 7f722848d700 1 -- 172.21.15.46:0/265807015 wait complete. 2024-03-18T20:49:10.085 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 60 2024-03-18T20:49:10.097 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:49:10.097 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T20:49:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.240+0000 7f8af61af700 1 -- 172.21.15.46:0/264652645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af0106070 msgr2=0x7f8af010dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.240+0000 7f8af61af700 1 --2- 172.21.15.46:0/264652645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af0106070 0x7f8af010dbf0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800b0a0 tx=0x7f8ae800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.240+0000 7f8af61af700 1 -- 172.21.15.46:0/264652645 shutdown_connections 2024-03-18T20:49:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.240+0000 7f8af61af700 1 --2- 172.21.15.46:0/264652645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af0106070 0x7f8af010dbf0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.240+0000 7f8af61af700 1 --2- 172.21.15.46:0/264652645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af0100f50 0x7f8af0101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.240+0000 7f8af61af700 1 --2- 172.21.15.46:0/264652645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8af0101900 0x7f8af0105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.240+0000 7f8af61af700 1 -- 172.21.15.46:0/264652645 >> 172.21.15.46:0/264652645 conn(0x7f8af00fc7f0 msgr2=0x7f8af00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.240+0000 7f8af61af700 1 -- 172.21.15.46:0/264652645 shutdown_connections 2024-03-18T20:49:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.240+0000 7f8af61af700 1 -- 172.21.15.46:0/264652645 wait complete. 2024-03-18T20:49:10.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.241+0000 7f8af61af700 1 Processor -- start 2024-03-18T20:49:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.241+0000 7f8af61af700 1 -- start start 2024-03-18T20:49:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.241+0000 7f8af61af700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af0100f50 0x7f8af0197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.241+0000 7f8af61af700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8af0106070 0x7f8af0197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.241+0000 7f8af61af700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af019bc00 0x7f8af019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.241+0000 7f8af61af700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af019c680 con 0x7f8af0100f50 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.241+0000 7f8af61af700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af019c7f0 con 0x7f8af0106070 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.241+0000 7f8af61af700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af019c9b0 con 0x7f8af019bc00 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af0100f50 0x7f8af0197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af0100f50 0x7f8af0197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59376/0 (socket says 172.21.15.46:59376) 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aef7fe700 1 -- 172.21.15.46:0/2828163257 learned_addr learned my addr 172.21.15.46:0/2828163257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8af0106070 0x7f8af0197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aeffff700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af019bc00 0x7f8af019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aef7fe700 1 -- 172.21.15.46:0/2828163257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af019bc00 msgr2=0x7f8af019c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af019bc00 0x7f8af019c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aef7fe700 1 -- 172.21.15.46:0/2828163257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8af0106070 msgr2=0x7f8af0197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8af0106070 0x7f8af0197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aef7fe700 1 -- 172.21.15.46:0/2828163257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae8009d00 con 0x7f8af0100f50 2024-03-18T20:49:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aeffff700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af019bc00 0x7f8af019c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:49:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af0100f50 0x7f8af0197320 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800e9c0 tx=0x7f8ad800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aecff9700 1 -- 172.21.15.46:0/2828163257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80097b0 con 0x7f8af0100f50 2024-03-18T20:49:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aecff9700 1 -- 172.21.15.46:0/2828163257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ad8004cf0 con 0x7f8af0100f50 2024-03-18T20:49:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8af61af700 1 -- 172.21.15.46:0/2828163257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af01a3f10 con 0x7f8af0100f50 2024-03-18T20:49:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8aecff9700 1 -- 172.21.15.46:0/2828163257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8005600 con 0x7f8af0100f50 2024-03-18T20:49:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.242+0000 7f8af61af700 1 -- 172.21.15.46:0/2828163257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af01a4340 con 0x7f8af0100f50 2024-03-18T20:49:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.245+0000 7f8aecff9700 1 -- 172.21.15.46:0/2828163257 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8ad800cc60 con 0x7f8af0100f50 2024-03-18T20:49:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.245+0000 7f8aecff9700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8adc0a1290 0x7f8adc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.246+0000 7f8aecff9700 1 -- 172.21.15.46:0/2828163257 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8ad8014070 con 0x7f8af0100f50 2024-03-18T20:49:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.246+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8adc0a1290 0x7f8adc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.246+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8adc0a1290 0x7f8adc0a3750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8af0198870 tx=0x7f8ae0009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.246+0000 7f8b02344b80 1 Processor -- start 2024-03-18T20:49:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.246+0000 7f8b02344b80 1 -- start start 2024-03-18T20:49:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.246+0000 7f8b02344b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d58ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.246+0000 7f8b02344b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 0x563d4d595ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.246+0000 7f8b02344b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d598610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.246+0000 7f8b02344b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4d4e4990 con 0x563d4d58aa60 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.246+0000 7f8b02344b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4d4e2350 con 0x563d4d596220 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.246+0000 7f8b02344b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4d4ddc40 con 0x563d4d58b380 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.246+0000 7f8aef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d58ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8aef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d58ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59390/0 (socket says 172.21.15.46:59390) 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8aef7fe700 1 -- 172.21.15.46:0/3279860942 learned_addr learned my addr 172.21.15.46:0/3279860942 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8aeffff700 1 --2- 172.21.15.46:0/3279860942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d598610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/3279860942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 0x563d4d595ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8aef7fe700 1 -- 172.21.15.46:0/3279860942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 msgr2=0x563d4d595ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/3279860942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 0x563d4d595ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8aef7fe700 1 -- 172.21.15.46:0/3279860942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 msgr2=0x563d4d598610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/3279860942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d598610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8aef7fe700 1 -- 172.21.15.46:0/3279860942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d4d35f3b0 con 0x563d4d58aa60 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/3279860942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d58ae40 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80a1fe0 tx=0x7f8ad8099a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33620fed087f9983 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8ae5ffb700 1 -- 172.21.15.46:0/3279860942 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad809a090 con 0x563d4d58aa60 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8ae5ffb700 1 -- 172.21.15.46:0/3279860942 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ad809dec0 con 0x563d4d58aa60 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.247+0000 7f8ae5ffb700 1 -- 172.21.15.46:0/3279860942 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad809a4f0 con 0x563d4d58aa60 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.248+0000 7f8b02344b80 1 -- 172.21.15.46:0/3279860942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 msgr2=0x563d4d58ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.248+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3279860942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d58ae40 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80a1fe0 tx=0x7f8ad8099a70 comp rx=0 tx=0).stop 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.248+0000 7f8b02344b80 1 -- 172.21.15.46:0/3279860942 shutdown_connections 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.248+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3279860942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d58ae40 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.248+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3279860942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d598610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.248+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3279860942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 0x563d4d595ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.248+0000 7f8b02344b80 1 -- 172.21.15.46:0/3279860942 >> 172.21.15.46:0/3279860942 conn(0x563d4d501220 msgr2=0x563d4d587020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.248+0000 7f8b02344b80 1 -- 172.21.15.46:0/3279860942 shutdown_connections 2024-03-18T20:49:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.248+0000 7f8b02344b80 1 -- 172.21.15.46:0/3279860942 wait complete. 2024-03-18T20:49:10.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.248+0000 7f8b02344b80 1 -- start start 2024-03-18T20:49:10.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8b02344b80 1 Processor -- start 2024-03-18T20:49:10.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8b02344b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d595cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8b02344b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 0x563d4d592fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8b02344b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d5936e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8b02344b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4d479d80 con 0x563d4d58aa60 2024-03-18T20:49:10.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8b02344b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4d479a20 con 0x563d4d596220 2024-03-18T20:49:10.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8b02344b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4d3616a0 con 0x563d4d58b380 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8b02344b80 10 client.0 _ll_register_callbacks cb 0x563d4d2e53a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8aef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d595cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8aef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d595cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59400/0 (socket says 172.21.15.46:59400) 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8aef7fe700 1 -- 172.21.15.46:0/3270953988 learned_addr learned my addr 172.21.15.46:0/3270953988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 0x563d4d592fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8aeffff700 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d5936e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8aef7fe700 1 -- 172.21.15.46:0/3270953988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 msgr2=0x563d4d592fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 0x563d4d592fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8aef7fe700 1 -- 172.21.15.46:0/3270953988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 msgr2=0x563d4d5936e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.249+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d5936e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8aef7fe700 1 -- 172.21.15.46:0/3270953988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af01a4340 con 0x563d4d58aa60 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8aeffff700 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d5936e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:49:10.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d595cb0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8096590 tx=0x7f8ad8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590151d567434a1c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8aceffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8aceffd700 1 -- 172.21.15.46:0/3270953988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8098430 con 0x563d4d58aa60 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8aceffd700 1 -- 172.21.15.46:0/3270953988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8ad8099580 con 0x563d4d58aa60 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8b02344b80 1 -- 172.21.15.46:0/3270953988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d4d57ead0 con 0x563d4d58aa60 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8aceffd700 1 -- 172.21.15.46:0/3270953988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ad8099740 con 0x563d4d58aa60 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8b02344b80 10 client.20085 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8b02344b80 1 -- 172.21.15.46:0/3270953988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x563d4d30bb60 con 0x563d4d58aa60 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8aceffd700 1 -- 172.21.15.46:0/3270953988 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ad80a6080 con 0x563d4d58aa60 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8aceffd700 1 -- 172.21.15.46:0/3270953988 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f8ad8098ff0 con 0x563d4d58aa60 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8b02344b80 10 client.20085 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.250+0000 7f8b02344b80 10 client.20085 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.251+0000 7f8b02344b80 20 client.20085 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.251+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563d4d5a5b60 0x563d4d5a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.251+0000 7f8b02344b80 4 client.20085 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:49:10.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.251+0000 7f8b02344b80 1 -- 172.21.15.46:0/3270953988 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d4d4d6700 con 0x563d4d5a5b60 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.251+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563d4d5a5b60 0x563d4d5a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.251+0000 7f8aceffd700 10 client.20085 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.251+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563d4d5a5b60 0x563d4d5a7f80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=709a29a76fe5908 server_cookie=e708a5d5355fadb0 in_seq=0 out_seq=0 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8aceffd700 1 -- 172.21.15.46:0/3270953988 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563d4d4d6700 con 0x563d4d5a5b60 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8aceffd700 10 client.20085 handle_command_reply: tid=0 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8b02344b80 1 client.20085 shutdown 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8b02344b80 1 -- 172.21.15.46:0/3270953988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 msgr2=0x563d4d595cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d595cb0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8096590 tx=0x7f8ad8017000 comp rx=0 tx=0).stop 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8b02344b80 1 -- 172.21.15.46:0/3270953988 shutdown_connections 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563d4d5a5b60 0x563d4d5a7f80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d595cb0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d5936e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8aceffd700 0 client.20085 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:49:10.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3270953988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 0x563d4d592fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8b02344b80 1 -- 172.21.15.46:0/3270953988 >> 172.21.15.46:0/3270953988 conn(0x563d4d501220 msgr2=0x563d4d586c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8b02344b80 1 -- 172.21.15.46:0/3270953988 shutdown_connections 2024-03-18T20:49:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.252+0000 7f8b02344b80 1 -- 172.21.15.46:0/3270953988 wait complete. 2024-03-18T20:49:10.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.253+0000 7f8b02344b80 20 client.20085 trim_cache size 0 max 16384 2024-03-18T20:49:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.266+0000 7f8b02344b80 1 Processor -- start 2024-03-18T20:49:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.266+0000 7f8b02344b80 1 -- start start 2024-03-18T20:49:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.266+0000 7f8b02344b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d58aa60 0x563d4d5a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.266+0000 7f8b02344b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58b380 0x563d4d592880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.266+0000 7f8b02344b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d596220 0x563d4d592dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.266+0000 7f8b02344b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4d58bd30 con 0x563d4d58b380 2024-03-18T20:49:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.266+0000 7f8b02344b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4d5835e0 con 0x563d4d58aa60 2024-03-18T20:49:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.266+0000 7f8b02344b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4d58ba30 con 0x563d4d596220 2024-03-18T20:49:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8aeeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58b380 0x563d4d592880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8aeeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58b380 0x563d4d592880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59406/0 (socket says 172.21.15.46:59406) 2024-03-18T20:49:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8aeeffd700 1 -- 172.21.15.46:0/3695932922 learned_addr learned my addr 172.21.15.46:0/3695932922 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/3695932922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d58aa60 0x563d4d5a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8aeffff700 1 --2- 172.21.15.46:0/3695932922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d596220 0x563d4d592dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8aeeffd700 1 -- 172.21.15.46:0/3695932922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d596220 msgr2=0x563d4d592dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/3695932922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d596220 0x563d4d592dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8aeeffd700 1 -- 172.21.15.46:0/3695932922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d58aa60 msgr2=0x563d4d5a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/3695932922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d58aa60 0x563d4d5a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8aeeffd700 1 -- 172.21.15.46:0/3695932922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d4d4a3ea0 con 0x563d4d58b380 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/3695932922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58b380 0x563d4d592880 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f8af00679c0 tx=0x7f8ae0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392b83234aafd0b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8ae67fc700 1 -- 172.21.15.46:0/3695932922 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0024040 con 0x563d4d58b380 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8ae67fc700 1 -- 172.21.15.46:0/3695932922 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ae0011ae0 con 0x563d4d58b380 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.267+0000 7f8ae67fc700 1 -- 172.21.15.46:0/3695932922 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0028730 con 0x563d4d58b380 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.268+0000 7f8b02344b80 1 -- 172.21.15.46:0/3695932922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58b380 msgr2=0x563d4d592880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.268+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3695932922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58b380 0x563d4d592880 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f8af00679c0 tx=0x7f8ae0014000 comp rx=0 tx=0).stop 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.268+0000 7f8b02344b80 1 -- 172.21.15.46:0/3695932922 shutdown_connections 2024-03-18T20:49:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.268+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3695932922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d596220 0x563d4d592dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.268+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3695932922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d58aa60 0x563d4d5a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.268+0000 7f8b02344b80 1 --2- 172.21.15.46:0/3695932922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58b380 0x563d4d592880 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.268+0000 7f8b02344b80 1 -- 172.21.15.46:0/3695932922 >> 172.21.15.46:0/3695932922 conn(0x563d4d501220 msgr2=0x563d4d5a15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.268+0000 7f8b02344b80 1 -- 172.21.15.46:0/3695932922 shutdown_connections 2024-03-18T20:49:10.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.268+0000 7f8b02344b80 1 -- 172.21.15.46:0/3695932922 wait complete. 2024-03-18T20:49:10.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.268+0000 7f8b02344b80 1 -- start start 2024-03-18T20:49:10.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8b02344b80 1 Processor -- start 2024-03-18T20:49:10.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8b02344b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d59aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8b02344b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 0x563d4d59b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8b02344b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d5f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8b02344b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4d5f2f40 con 0x563d4d58aa60 2024-03-18T20:49:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8b02344b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4d5f30b0 con 0x563d4d596220 2024-03-18T20:49:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8b02344b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4d5f3270 con 0x563d4d58b380 2024-03-18T20:49:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8b02344b80 10 client.0 _ll_register_callbacks cb 0x563d4d2e53a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8aef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d59aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8aef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d59aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59410/0 (socket says 172.21.15.46:59410) 2024-03-18T20:49:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8aef7fe700 1 -- 172.21.15.46:0/155101248 learned_addr learned my addr 172.21.15.46:0/155101248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8aeffff700 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d5f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 0x563d4d59b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8aef7fe700 1 -- 172.21.15.46:0/155101248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 msgr2=0x563d4d59b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 0x563d4d59b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8aef7fe700 1 -- 172.21.15.46:0/155101248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 msgr2=0x563d4d5f29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d5f29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.269+0000 7f8aef7fe700 1 -- 172.21.15.46:0/155101248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae0017070 con 0x563d4d58aa60 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8aef7fe700 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d59aee0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009fd0 tx=0x7f8ad8099a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a171539da88e76d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8aceffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8aceffd700 1 -- 172.21.15.46:0/155101248 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80963f0 con 0x563d4d58aa60 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8aceffd700 1 -- 172.21.15.46:0/155101248 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8ad8004e60 con 0x563d4d58aa60 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8b02344b80 1 -- 172.21.15.46:0/155101248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ad809dec0 con 0x563d4d58aa60 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8aceffd700 1 -- 172.21.15.46:0/155101248 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ad80a2300 con 0x563d4d58aa60 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8b02344b80 10 client.20091 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8b02344b80 1 -- 172.21.15.46:0/155101248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x563d4d363cd0 con 0x563d4d58aa60 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8aceffd700 1 -- 172.21.15.46:0/155101248 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ad809dec0 con 0x563d4d58aa60 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8aceffd700 1 -- 172.21.15.46:0/155101248 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f8ad809ee30 con 0x563d4d58aa60 2024-03-18T20:49:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8b02344b80 10 client.20091 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8b02344b80 10 client.20091 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8b02344b80 20 client.20091 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8b02344b80 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563d4d5fa5e0 0x563d4d5fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8b02344b80 4 client.20091 mds_command: new command op to 24446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.270+0000 7f8b02344b80 1 -- 172.21.15.46:0/155101248 --> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563d4d32ccf0 con 0x563d4d5fa5e0 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563d4d5fa5e0 0x563d4d5fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8aceffd700 10 client.20091 ms_handle_connect on v2:172.21.15.73:6832/2440572005 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8aeeffd700 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563d4d5fa5e0 0x563d4d5fc9d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=550b1df2c58b5f13 server_cookie=9386e6efad2db299 in_seq=0 out_seq=0 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8aceffd700 1 -- 172.21.15.46:0/155101248 <== mds.4 v2:172.21.15.73:6832/2440572005 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+2329 (crc 0 0 0) 0x563d4d32ccf0 con 0x563d4d5fa5e0 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8aceffd700 10 client.20091 handle_command_reply: tid=0 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8b02344b80 1 client.20091 shutdown 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8b02344b80 1 -- 172.21.15.46:0/155101248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 msgr2=0x563d4d59aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8b02344b80 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d59aee0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009fd0 tx=0x7f8ad8099a70 comp rx=0 tx=0).stop 2024-03-18T20:49:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8b02344b80 1 -- 172.21.15.46:0/155101248 shutdown_connections 2024-03-18T20:49:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8b02344b80 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.73:6832/2440572005,v1:172.21.15.73:6833/2440572005] conn(0x563d4d5fa5e0 0x563d4d5fc9d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8b02344b80 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d4d58aa60 0x563d4d59aee0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8b02344b80 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563d4d596220 0x563d4d5f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8b02344b80 1 --2- 172.21.15.46:0/155101248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d4d58b380 0x563d4d59b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8aceffd700 0 client.20091 ms_handle_reset on v2:172.21.15.73:6832/2440572005 2024-03-18T20:49:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.271+0000 7f8b02344b80 1 -- 172.21.15.46:0/155101248 >> 172.21.15.46:0/155101248 conn(0x563d4d501220 msgr2=0x563d4d5a1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8b02344b80 1 -- 172.21.15.46:0/155101248 shutdown_connections 2024-03-18T20:49:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8b02344b80 1 -- 172.21.15.46:0/155101248 wait complete. 2024-03-18T20:49:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8b02344b80 20 client.20091 trim_cache size 0 max 16384 2024-03-18T20:49:10.280 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 900619346, 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 5497558140177, 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03" 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 958634982, 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 5497558140180, 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL" 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T20:49:10.281 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 1526672529, 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 5497558140178, 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL" 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 1750481800, 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 5497558140173, 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB" 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-18T20:49:10.282 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 1780457581, 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 5497558140171, 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1" 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 2057819981, 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 5497558140174, 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB" 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 2549199034, 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 5497558140176, 2024-03-18T20:49:10.283 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03" 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 2617804119, 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 5497558140179, 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV" 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 2792763337, 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 5497558140172, 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1" 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T20:49:10.284 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T20:49:10.285 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-18T20:49:10.285 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4001455076, 2024-03-18T20:49:10.285 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 5497558140181, 2024-03-18T20:49:10.285 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL" 2024-03-18T20:49:10.285 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:49:10.285 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-18T20:49:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8af61af700 1 -- 172.21.15.46:0/2828163257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8adc0a1290 msgr2=0x7f8adc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8af61af700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8adc0a1290 0x7f8adc0a3750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8af0198870 tx=0x7f8ae0009260 comp rx=0 tx=0).stop 2024-03-18T20:49:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8af61af700 1 -- 172.21.15.46:0/2828163257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af0100f50 msgr2=0x7f8af0197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8af61af700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af0100f50 0x7f8af0197320 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800e9c0 tx=0x7f8ad800ecd0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8af61af700 1 -- 172.21.15.46:0/2828163257 shutdown_connections 2024-03-18T20:49:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8af61af700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8adc0a1290 0x7f8adc0a3750 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8af61af700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8af0100f50 0x7f8af0197320 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8af61af700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8af019bc00 0x7f8af019c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8af61af700 1 --2- 172.21.15.46:0/2828163257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8af0106070 0x7f8af0197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8af61af700 1 -- 172.21.15.46:0/2828163257 >> 172.21.15.46:0/2828163257 conn(0x7f8af00fc7f0 msgr2=0x7f8af00feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8af61af700 1 -- 172.21.15.46:0/2828163257 shutdown_connections 2024-03-18T20:49:10.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.272+0000 7f8af61af700 1 -- 172.21.15.46:0/2828163257 wait complete. 2024-03-18T20:49:10.288 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:49:10.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T20:49:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.431+0000 7f24d7e97700 1 -- 172.21.15.46:0/4291202766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24d00ffe70 msgr2=0x7f24d0100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.431+0000 7f24d7e97700 1 --2- 172.21.15.46:0/4291202766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24d00ffe70 0x7f24d0100250 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009990 tx=0x7f24cc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.431+0000 7f24d7e97700 1 -- 172.21.15.46:0/4291202766 shutdown_connections 2024-03-18T20:49:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.431+0000 7f24d7e97700 1 --2- 172.21.15.46:0/4291202766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24d00ffe70 0x7f24d0100250 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.431+0000 7f24d7e97700 1 --2- 172.21.15.46:0/4291202766 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24d0104f70 0x7f24d010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.431+0000 7f24d7e97700 1 --2- 172.21.15.46:0/4291202766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24d0100820 0x7f24d0104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.431+0000 7f24d7e97700 1 -- 172.21.15.46:0/4291202766 >> 172.21.15.46:0/4291202766 conn(0x7f24d0076d50 msgr2=0x7f24d0077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.431+0000 7f24d7e97700 1 -- 172.21.15.46:0/4291202766 shutdown_connections 2024-03-18T20:49:10.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.431+0000 7f24d7e97700 1 -- 172.21.15.46:0/4291202766 wait complete. 2024-03-18T20:49:10.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.432+0000 7f24d7e97700 1 Processor -- start 2024-03-18T20:49:10.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.432+0000 7f24d7e97700 1 -- start start 2024-03-18T20:49:10.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.432+0000 7f24d7e97700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24d00ffe70 0x7f24d010d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.432+0000 7f24d7e97700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24d0104f70 0x7f24d010d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.432+0000 7f24d7e97700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24d010deb0 0x7f24d01a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.432+0000 7f24d7e97700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01a5970 con 0x7f24d010deb0 2024-03-18T20:49:10.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.432+0000 7f24d7e97700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01a5ae0 con 0x7f24d00ffe70 2024-03-18T20:49:10.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.432+0000 7f24d7e97700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01a5ca0 con 0x7f24d0104f70 2024-03-18T20:49:10.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.432+0000 7f24d6434700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24d010deb0 0x7f24d01a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.432+0000 7f24d6434700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24d010deb0 0x7f24d01a53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59434/0 (socket says 172.21.15.46:59434) 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.432+0000 7f24d6434700 1 -- 172.21.15.46:0/3339640611 learned_addr learned my addr 172.21.15.46:0/3339640611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24d5432700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24d0104f70 0x7f24d010d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24d5c33700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24d00ffe70 0x7f24d010d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24d6434700 1 -- 172.21.15.46:0/3339640611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24d0104f70 msgr2=0x7f24d010d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24d6434700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24d0104f70 0x7f24d010d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24d6434700 1 -- 172.21.15.46:0/3339640611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24d00ffe70 msgr2=0x7f24d010d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24d6434700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24d00ffe70 0x7f24d010d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24d6434700 1 -- 172.21.15.46:0/3339640611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc0095f0 con 0x7f24d010deb0 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24d5432700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24d0104f70 0x7f24d010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24d6434700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24d010deb0 0x7f24d01a53a0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f24c800ef30 tx=0x7f24c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24c2ffd700 1 -- 172.21.15.46:0/3339640611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8018070 con 0x7f24d010deb0 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24c2ffd700 1 -- 172.21.15.46:0/3339640611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24c800f040 con 0x7f24d010deb0 2024-03-18T20:49:10.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24d7e97700 1 -- 172.21.15.46:0/3339640611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d01a5f20 con 0x7f24d010deb0 2024-03-18T20:49:10.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24c2ffd700 1 -- 172.21.15.46:0/3339640611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c800cdb0 con 0x7f24d010deb0 2024-03-18T20:49:10.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.433+0000 7f24d7e97700 1 -- 172.21.15.46:0/3339640611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d01a63b0 con 0x7f24d010deb0 2024-03-18T20:49:10.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.436+0000 7f24c2ffd700 1 -- 172.21.15.46:0/3339640611 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f24c8003a00 con 0x7f24d010deb0 2024-03-18T20:49:10.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.436+0000 7f24c2ffd700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f24bc0a1290 0x7f24bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.436+0000 7f24c2ffd700 1 -- 172.21.15.46:0/3339640611 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f24c80cc8a0 con 0x7f24d010deb0 2024-03-18T20:49:10.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.436+0000 7f24d5c33700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f24bc0a1290 0x7f24bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.437+0000 7f24d5c33700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f24bc0a1290 0x7f24bc0a3750 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00b2e0 tx=0x7f24cc005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.437+0000 7f24e402cb80 1 Processor -- start 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.437+0000 7f24e402cb80 1 -- start start 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.437+0000 7f24e402cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b00a60 0x564c33b00e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24e402cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b01380 0x564c33b0bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24e402cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b0e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24e402cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c33a5a990 con 0x564c33b01380 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24e402cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c33a58350 con 0x564c33b00a60 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24e402cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c33a53c40 con 0x564c33b0c220 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24d5432700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b01380 0x564c33b0bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24d5432700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b01380 0x564c33b0bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59440/0 (socket says 172.21.15.46:59440) 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24d5432700 1 -- 172.21.15.46:0/1568676046 learned_addr learned my addr 172.21.15.46:0/1568676046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24d5c33700 1 --2- 172.21.15.46:0/1568676046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b00a60 0x564c33b00e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24d6434700 1 --2- 172.21.15.46:0/1568676046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b0e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24d5432700 1 -- 172.21.15.46:0/1568676046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 msgr2=0x564c33b0e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24d5432700 1 --2- 172.21.15.46:0/1568676046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b0e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24d5432700 1 -- 172.21.15.46:0/1568676046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b00a60 msgr2=0x564c33b00e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24d5432700 1 --2- 172.21.15.46:0/1568676046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b00a60 0x564c33b00e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24d5432700 1 -- 172.21.15.46:0/1568676046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c338d53b0 con 0x564c33b01380 2024-03-18T20:49:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24d5432700 1 --2- 172.21.15.46:0/1568676046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b01380 0x564c33b0bce0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f24c400bcd0 tx=0x7f24c400f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d019c559d7737ff7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24bbfff700 1 -- 172.21.15.46:0/1568676046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c400fbe0 con 0x564c33b01380 2024-03-18T20:49:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24bbfff700 1 -- 172.21.15.46:0/1568676046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24c400fd40 con 0x564c33b01380 2024-03-18T20:49:10.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.438+0000 7f24bbfff700 1 -- 172.21.15.46:0/1568676046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c40117c0 con 0x564c33b01380 2024-03-18T20:49:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.439+0000 7f24e402cb80 1 -- 172.21.15.46:0/1568676046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b01380 msgr2=0x564c33b0bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.439+0000 7f24e402cb80 1 --2- 172.21.15.46:0/1568676046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b01380 0x564c33b0bce0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f24c400bcd0 tx=0x7f24c400f360 comp rx=0 tx=0).stop 2024-03-18T20:49:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.439+0000 7f24e402cb80 1 -- 172.21.15.46:0/1568676046 shutdown_connections 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.439+0000 7f24e402cb80 1 --2- 172.21.15.46:0/1568676046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b0e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.439+0000 7f24e402cb80 1 --2- 172.21.15.46:0/1568676046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b00a60 0x564c33b00e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.439+0000 7f24e402cb80 1 --2- 172.21.15.46:0/1568676046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b01380 0x564c33b0bce0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.439+0000 7f24e402cb80 1 -- 172.21.15.46:0/1568676046 >> 172.21.15.46:0/1568676046 conn(0x564c33a77220 msgr2=0x564c33afd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.439+0000 7f24e402cb80 1 -- 172.21.15.46:0/1568676046 shutdown_connections 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.439+0000 7f24e402cb80 1 -- 172.21.15.46:0/1568676046 wait complete. 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.440+0000 7f24e402cb80 1 -- start start 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.440+0000 7f24e402cb80 1 Processor -- start 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.440+0000 7f24e402cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b00a60 0x564c33b0bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.440+0000 7f24e402cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b01380 0x564c33b08f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.440+0000 7f24e402cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b0c220 0x564c33b09580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.440+0000 7f24e402cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c339efd80 con 0x564c33b0c220 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.440+0000 7f24e402cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c339efa20 con 0x564c33b00a60 2024-03-18T20:49:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.440+0000 7f24e402cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c338d76a0 con 0x564c33b01380 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24e402cb80 10 client.0 _ll_register_callbacks cb 0x564c3385b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24d6434700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b0c220 0x564c33b09580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24d6434700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b0c220 0x564c33b09580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59450/0 (socket says 172.21.15.46:59450) 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24d6434700 1 -- 172.21.15.46:0/1864495789 learned_addr learned my addr 172.21.15.46:0/1864495789 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24d5c33700 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b00a60 0x564c33b0bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24d5432700 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b01380 0x564c33b08f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24d6434700 1 -- 172.21.15.46:0/1864495789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b01380 msgr2=0x564c33b08f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24d6434700 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b01380 0x564c33b08f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24d6434700 1 -- 172.21.15.46:0/1864495789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b00a60 msgr2=0x564c33b0bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24d6434700 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b00a60 0x564c33b0bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24d6434700 1 -- 172.21.15.46:0/1864495789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c400b900 con 0x564c33b0c220 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24d6434700 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b0c220 0x564c33b09580 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f24c8099720 tx=0x7f24c8099380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d05ca78da52901 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:10.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24b97fa700 1 -- 172.21.15.46:0/1864495789 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c801b320 con 0x564c33b0c220 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24b97fa700 1 -- 172.21.15.46:0/1864495789 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f24c809a120 con 0x564c33b0c220 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24e402cb80 1 -- 172.21.15.46:0/1864495789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c33af4ad0 con 0x564c33b0c220 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24b97fa700 1 -- 172.21.15.46:0/1864495789 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24c8099a20 con 0x564c33b0c220 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24e402cb80 10 client.20103 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.441+0000 7f24e402cb80 1 -- 172.21.15.46:0/1864495789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x564c33881b60 con 0x564c33b0c220 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.442+0000 7f24b97fa700 1 -- 172.21.15.46:0/1864495789 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24c80b6070 con 0x564c33b0c220 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.442+0000 7f24b97fa700 1 -- 172.21.15.46:0/1864495789 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f24c80a1980 con 0x564c33b0c220 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.442+0000 7f24e402cb80 10 client.20103 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.442+0000 7f24e402cb80 10 client.20103 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.442+0000 7f24e402cb80 20 client.20103 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.442+0000 7f24e402cb80 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564c33b1be00 0x564c33b1e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.442+0000 7f24e402cb80 4 client.20103 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.442+0000 7f24e402cb80 1 -- 172.21.15.46:0/1864495789 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c33a4c700 con 0x564c33b1be00 2024-03-18T20:49:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.442+0000 7f24d5c33700 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564c33b1be00 0x564c33b1e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.443+0000 7f24b97fa700 10 client.20103 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.443+0000 7f24d5c33700 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564c33b1be00 0x564c33b1e220 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b6f304a4bda28f8c server_cookie=fcc819ae76ff8d5e in_seq=0 out_seq=0 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.443+0000 7f24b97fa700 1 -- 172.21.15.46:0/1864495789 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564c33a4c700 con 0x564c33b1be00 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.443+0000 7f24b97fa700 10 client.20103 handle_command_reply: tid=0 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.443+0000 7f24e402cb80 1 client.20103 shutdown 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.443+0000 7f24e402cb80 1 -- 172.21.15.46:0/1864495789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b0c220 msgr2=0x564c33b09580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.443+0000 7f24e402cb80 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b0c220 0x564c33b09580 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f24c8099720 tx=0x7f24c8099380 comp rx=0 tx=0).stop 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.444+0000 7f24e402cb80 1 -- 172.21.15.46:0/1864495789 shutdown_connections 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.444+0000 7f24e402cb80 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564c33b1be00 0x564c33b1e220 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.444+0000 7f24e402cb80 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b0c220 0x564c33b09580 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.444+0000 7f24e402cb80 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b00a60 0x564c33b0bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.444+0000 7f24b97fa700 0 client.20103 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.444+0000 7f24e402cb80 1 --2- 172.21.15.46:0/1864495789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b01380 0x564c33b08f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.444+0000 7f24e402cb80 1 -- 172.21.15.46:0/1864495789 >> 172.21.15.46:0/1864495789 conn(0x564c33a77220 msgr2=0x564c33afcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.444+0000 7f24e402cb80 1 -- 172.21.15.46:0/1864495789 shutdown_connections 2024-03-18T20:49:10.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.444+0000 7f24e402cb80 1 -- 172.21.15.46:0/1864495789 wait complete. 2024-03-18T20:49:10.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.444+0000 7f24e402cb80 20 client.20103 trim_cache size 0 max 16384 2024-03-18T20:49:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.460+0000 7f24e402cb80 1 Processor -- start 2024-03-18T20:49:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.460+0000 7f24e402cb80 1 -- start start 2024-03-18T20:49:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.460+0000 7f24e402cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 0x564c33b09450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.460+0000 7f24e402cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b01380 0x564c33b12e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.460+0000 7f24e402cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b175a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.460+0000 7f24e402cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c33af7210 con 0x564c33b00a60 2024-03-18T20:49:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.460+0000 7f24e402cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c33b01d30 con 0x564c33b01380 2024-03-18T20:49:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.460+0000 7f24e402cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c33b02580 con 0x564c33b0c220 2024-03-18T20:49:10.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.460+0000 7f24d5c33700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 0x564c33b09450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24d5c33700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 0x564c33b09450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59460/0 (socket says 172.21.15.46:59460) 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24d5c33700 1 -- 172.21.15.46:0/240113855 learned_addr learned my addr 172.21.15.46:0/240113855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24d6434700 1 --2- 172.21.15.46:0/240113855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b175a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24d5432700 1 --2- 172.21.15.46:0/240113855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b01380 0x564c33b12e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24d6434700 1 -- 172.21.15.46:0/240113855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b01380 msgr2=0x564c33b12e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24d6434700 1 --2- 172.21.15.46:0/240113855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b01380 0x564c33b12e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24d6434700 1 -- 172.21.15.46:0/240113855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 msgr2=0x564c33b09450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24d6434700 1 --2- 172.21.15.46:0/240113855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 0x564c33b09450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24d6434700 1 -- 172.21.15.46:0/240113855 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c33a19ea0 con 0x564c33b0c220 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24d5c33700 1 --2- 172.21.15.46:0/240113855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 0x564c33b09450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24d6434700 1 --2- 172.21.15.46:0/240113855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b175a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f24c80090a0 tx=0x7f24c8004dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e8ab0c10ca549b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24c0ff9700 1 -- 172.21.15.46:0/240113855 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c809a120 con 0x564c33b0c220 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24c0ff9700 1 -- 172.21.15.46:0/240113855 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24c80b6020 con 0x564c33b0c220 2024-03-18T20:49:10.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.461+0000 7f24c0ff9700 1 -- 172.21.15.46:0/240113855 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8099a20 con 0x564c33b0c220 2024-03-18T20:49:10.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.462+0000 7f24e402cb80 1 -- 172.21.15.46:0/240113855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 msgr2=0x564c33b175a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.462+0000 7f24e402cb80 1 --2- 172.21.15.46:0/240113855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b175a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f24c80090a0 tx=0x7f24c8004dc0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.462+0000 7f24e402cb80 1 -- 172.21.15.46:0/240113855 shutdown_connections 2024-03-18T20:49:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.462+0000 7f24e402cb80 1 --2- 172.21.15.46:0/240113855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 0x564c33b09450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.462+0000 7f24e402cb80 1 --2- 172.21.15.46:0/240113855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b175a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.462+0000 7f24e402cb80 1 --2- 172.21.15.46:0/240113855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b01380 0x564c33b12e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.462+0000 7f24e402cb80 1 -- 172.21.15.46:0/240113855 >> 172.21.15.46:0/240113855 conn(0x564c33a77220 msgr2=0x564c33b077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.462+0000 7f24e402cb80 1 -- 172.21.15.46:0/240113855 shutdown_connections 2024-03-18T20:49:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.462+0000 7f24e402cb80 1 -- 172.21.15.46:0/240113855 wait complete. 2024-03-18T20:49:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24e402cb80 1 -- start start 2024-03-18T20:49:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24e402cb80 1 Processor -- start 2024-03-18T20:49:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24e402cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 0x564c33b063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24e402cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b01380 0x564c33b068f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24e402cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b05170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24e402cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c33b056b0 con 0x564c33b00a60 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24e402cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c33b05820 con 0x564c33b01380 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24e402cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c33b059e0 con 0x564c33b0c220 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24e402cb80 10 client.0 _ll_register_callbacks cb 0x7f24c80997f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24d5c33700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 0x564c33b063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24d5c33700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 0x564c33b063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59466/0 (socket says 172.21.15.46:59466) 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24d5c33700 1 -- 172.21.15.46:0/461093233 learned_addr learned my addr 172.21.15.46:0/461093233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.463+0000 7f24d6434700 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b05170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24d5432700 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b01380 0x564c33b068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24d5c33700 1 -- 172.21.15.46:0/461093233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 msgr2=0x564c33b05170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24d5c33700 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b05170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24d5c33700 1 -- 172.21.15.46:0/461093233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b01380 msgr2=0x564c33b068f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24d5c33700 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b01380 0x564c33b068f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24d5c33700 1 -- 172.21.15.46:0/461093233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d0069250 con 0x564c33b00a60 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24d5c33700 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 0x564c33b063b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f24cc006010 tx=0x7f24cc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30be6ffe8a5c505d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24b97fa700 1 -- 172.21.15.46:0/461093233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc036d10 con 0x564c33b00a60 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24b97fa700 1 -- 172.21.15.46:0/461093233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f24cc0360c0 con 0x564c33b00a60 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24e402cb80 1 -- 172.21.15.46:0/461093233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24c400fd40 con 0x564c33b00a60 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24b97fa700 1 -- 172.21.15.46:0/461093233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24cc036e70 con 0x564c33b00a60 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24e402cb80 10 client.20109 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.464+0000 7f24e402cb80 1 -- 172.21.15.46:0/461093233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x564c33881b60 con 0x564c33b00a60 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.465+0000 7f24b97fa700 1 -- 172.21.15.46:0/461093233 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24c400fd40 con 0x564c33b00a60 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.465+0000 7f24b97fa700 1 -- 172.21.15.46:0/461093233 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f24cc004bc0 con 0x564c33b00a60 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.465+0000 7f24e402cb80 10 client.20109 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.465+0000 7f24e402cb80 10 client.20109 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.465+0000 7f24e402cb80 20 client.20109 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.465+0000 7f24e402cb80 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564c33b70b60 0x564c33b72f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.465+0000 7f24e402cb80 4 client.20109 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.465+0000 7f24e402cb80 1 -- 172.21.15.46:0/461093233 --> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c338a2cf0 con 0x564c33b70b60 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.465+0000 7f24d5432700 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564c33b70b60 0x564c33b72f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.465+0000 7f24b97fa700 10 client.20109 ms_handle_connect on v2:172.21.15.73:6836/2773013996 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.465+0000 7f24d5432700 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564c33b70b60 0x564c33b72f80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c284c736ac55d145 server_cookie=d58f8843ae3b9b35 in_seq=0 out_seq=0 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24b97fa700 1 -- 172.21.15.46:0/461093233 <== mds.3 v2:172.21.15.73:6836/2773013996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c338a2cf0 con 0x564c33b70b60 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24b97fa700 10 client.20109 handle_command_reply: tid=0 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24e402cb80 1 client.20109 shutdown 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24e402cb80 1 -- 172.21.15.46:0/461093233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 msgr2=0x564c33b063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24e402cb80 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 0x564c33b063b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f24cc006010 tx=0x7f24cc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24e402cb80 1 -- 172.21.15.46:0/461093233 shutdown_connections 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24e402cb80 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.73:6836/2773013996,v1:172.21.15.73:6837/2773013996] conn(0x564c33b70b60 0x564c33b72f80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24e402cb80 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c33b00a60 0x564c33b063b0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24e402cb80 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c33b0c220 0x564c33b05170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24b97fa700 0 client.20109 ms_handle_reset on v2:172.21.15.73:6836/2773013996 2024-03-18T20:49:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24e402cb80 1 --2- 172.21.15.46:0/461093233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564c33b01380 0x564c33b068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24e402cb80 1 -- 172.21.15.46:0/461093233 >> 172.21.15.46:0/461093233 conn(0x564c33a77220 msgr2=0x564c33b17e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24e402cb80 1 -- 172.21.15.46:0/461093233 shutdown_connections 2024-03-18T20:49:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24e402cb80 1 -- 172.21.15.46:0/461093233 wait complete. 2024-03-18T20:49:10.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24e402cb80 20 client.20109 trim_cache size 0 max 16384 2024-03-18T20:49:10.474 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:49:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24d7e97700 1 -- 172.21.15.46:0/3339640611 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f24bc0a1290 msgr2=0x7f24bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24d7e97700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f24bc0a1290 0x7f24bc0a3750 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00b2e0 tx=0x7f24cc005960 comp rx=0 tx=0).stop 2024-03-18T20:49:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.466+0000 7f24d7e97700 1 -- 172.21.15.46:0/3339640611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24d010deb0 msgr2=0x7f24d01a53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.467+0000 7f24d7e97700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24d010deb0 0x7f24d01a53a0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f24c800ef30 tx=0x7f24c800c3a0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.467+0000 7f24d7e97700 1 -- 172.21.15.46:0/3339640611 shutdown_connections 2024-03-18T20:49:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.467+0000 7f24d7e97700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f24bc0a1290 0x7f24bc0a3750 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.467+0000 7f24d7e97700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24d010deb0 0x7f24d01a53a0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.467+0000 7f24d7e97700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24d00ffe70 0x7f24d010d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.467+0000 7f24d7e97700 1 --2- 172.21.15.46:0/3339640611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f24d0104f70 0x7f24d010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.467+0000 7f24d7e97700 1 -- 172.21.15.46:0/3339640611 >> 172.21.15.46:0/3339640611 conn(0x7f24d0076d50 msgr2=0x7f24d00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.467+0000 7f24d7e97700 1 -- 172.21.15.46:0/3339640611 shutdown_connections 2024-03-18T20:49:10.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.467+0000 7f24d7e97700 1 -- 172.21.15.46:0/3339640611 wait complete. 2024-03-18T20:49:10.481 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:10 smithi046 ceph-mon[43495]: pgmap v408: 97 pgs: 97 active+clean; 219 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-03-18T20:49:10.481 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:10 smithi046 ceph-mon[43495]: osdmap e96: 12 total, 12 up, 12 in 2024-03-18T20:49:10.481 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:10 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/265807015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:49:10.482 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:49:10.482 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T20:49:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.619+0000 7f7f56471700 1 -- 172.21.15.46:0/2696969086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50104f60 msgr2=0x7f7f5010cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.619+0000 7f7f56471700 1 --2- 172.21.15.46:0/2696969086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50104f60 0x7f7f5010cae0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800b0a0 tx=0x7f7f4800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.619+0000 7f7f56471700 1 -- 172.21.15.46:0/2696969086 shutdown_connections 2024-03-18T20:49:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.619+0000 7f7f56471700 1 --2- 172.21.15.46:0/2696969086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50104f60 0x7f7f5010cae0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.619+0000 7f7f56471700 1 --2- 172.21.15.46:0/2696969086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f500ffe50 0x7f7f50100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.619+0000 7f7f56471700 1 --2- 172.21.15.46:0/2696969086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f50100800 0x7f7f50104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.619+0000 7f7f56471700 1 -- 172.21.15.46:0/2696969086 >> 172.21.15.46:0/2696969086 conn(0x7f7f50076d40 msgr2=0x7f7f50077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.619+0000 7f7f56471700 1 -- 172.21.15.46:0/2696969086 shutdown_connections 2024-03-18T20:49:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.619+0000 7f7f56471700 1 -- 172.21.15.46:0/2696969086 wait complete. 2024-03-18T20:49:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.619+0000 7f7f56471700 1 Processor -- start 2024-03-18T20:49:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.619+0000 7f7f56471700 1 -- start start 2024-03-18T20:49:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f56471700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f500ffe50 0x7f7f501018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f56471700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50104f60 0x7f7f50101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f56471700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f50102350 0x7f7f501a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f56471700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f501a5a50 con 0x7f7f50104f60 2024-03-18T20:49:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f56471700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f501a5bc0 con 0x7f7f50102350 2024-03-18T20:49:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f56471700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f501a5d80 con 0x7f7f500ffe50 2024-03-18T20:49:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f4f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50104f60 0x7f7f50101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f4f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50104f60 0x7f7f50101e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59488/0 (socket says 172.21.15.46:59488) 2024-03-18T20:49:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f4f7fe700 1 -- 172.21.15.46:0/2479598425 learned_addr learned my addr 172.21.15.46:0/2479598425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f500ffe50 0x7f7f501018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f54a0e700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f50102350 0x7f7f501a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f4f7fe700 1 -- 172.21.15.46:0/2479598425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f500ffe50 msgr2=0x7f7f501018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f500ffe50 0x7f7f501018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f4f7fe700 1 -- 172.21.15.46:0/2479598425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f50102350 msgr2=0x7f7f501a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f50102350 0x7f7f501a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.620+0000 7f7f4f7fe700 1 -- 172.21.15.46:0/2479598425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f48009d00 con 0x7f7f50104f60 2024-03-18T20:49:10.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.621+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50104f60 0x7f7f50101e10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400d950 tx=0x7f7f4400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.621+0000 7f7f4d7fa700 1 -- 172.21.15.46:0/2479598425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f44004220 con 0x7f7f50104f60 2024-03-18T20:49:10.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.621+0000 7f7f4d7fa700 1 -- 172.21.15.46:0/2479598425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f44007620 con 0x7f7f50104f60 2024-03-18T20:49:10.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.621+0000 7f7f56471700 1 -- 172.21.15.46:0/2479598425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f501a6060 con 0x7f7f50104f60 2024-03-18T20:49:10.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.621+0000 7f7f4d7fa700 1 -- 172.21.15.46:0/2479598425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f44005690 con 0x7f7f50104f60 2024-03-18T20:49:10.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.621+0000 7f7f56471700 1 -- 172.21.15.46:0/2479598425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f501a65c0 con 0x7f7f50104f60 2024-03-18T20:49:10.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.623+0000 7f7f4d7fa700 1 -- 172.21.15.46:0/2479598425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7f44004750 con 0x7f7f50104f60 2024-03-18T20:49:10.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.624+0000 7f7f4d7fa700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7f380a1310 0x7f7f380a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.624+0000 7f7f4d7fa700 1 -- 172.21.15.46:0/2479598425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7f440cca00 con 0x7f7f50104f60 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.624+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7f380a1310 0x7f7f380a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7f380a1310 0x7f7f380a37d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009c60 tx=0x7f7f40009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f62606b80 1 Processor -- start 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f62606b80 1 -- start start 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f62606b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9dda60 0x55c39d9dde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f62606b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f62606b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9e9220 0x55c39d9eb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f62606b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d937990 con 0x55c39d9e9220 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f62606b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d935350 con 0x55c39d9dda60 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f62606b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d930c40 con 0x55c39d9de380 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f54a0e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9e9220 0x55c39d9eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f54a0e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9e9220 0x55c39d9eb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59496/0 (socket says 172.21.15.46:59496) 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f54a0e700 1 -- 172.21.15.46:0/74998988 learned_addr learned my addr 172.21.15.46:0/74998988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/74998988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/74998988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9dda60 0x55c39d9dde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f54a0e700 1 -- 172.21.15.46:0/74998988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 msgr2=0x55c39d9e8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f54a0e700 1 --2- 172.21.15.46:0/74998988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f54a0e700 1 -- 172.21.15.46:0/74998988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9dda60 msgr2=0x55c39d9dde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f54a0e700 1 --2- 172.21.15.46:0/74998988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9dda60 0x55c39d9dde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f54a0e700 1 -- 172.21.15.46:0/74998988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c39d7b23b0 con 0x55c39d9e9220 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.625+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/74998988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.626+0000 7f7f54a0e700 1 --2- 172.21.15.46:0/74998988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9e9220 0x55c39d9eb610 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009c80 tx=0x7f7f480092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdf3a607168b0ed server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.626+0000 7f7f367fc700 1 -- 172.21.15.46:0/74998988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800e050 con 0x55c39d9e9220 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.626+0000 7f7f367fc700 1 -- 172.21.15.46:0/74998988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f48007cc0 con 0x55c39d9e9220 2024-03-18T20:49:10.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.626+0000 7f7f367fc700 1 -- 172.21.15.46:0/74998988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f48004d60 con 0x55c39d9e9220 2024-03-18T20:49:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.627+0000 7f7f62606b80 1 -- 172.21.15.46:0/74998988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9e9220 msgr2=0x55c39d9eb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.627+0000 7f7f62606b80 1 --2- 172.21.15.46:0/74998988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9e9220 0x55c39d9eb610 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009c80 tx=0x7f7f480092e0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.627+0000 7f7f62606b80 1 -- 172.21.15.46:0/74998988 shutdown_connections 2024-03-18T20:49:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.627+0000 7f7f62606b80 1 --2- 172.21.15.46:0/74998988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9e9220 0x55c39d9eb610 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.627+0000 7f7f62606b80 1 --2- 172.21.15.46:0/74998988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9dda60 0x55c39d9dde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.627+0000 7f7f62606b80 1 --2- 172.21.15.46:0/74998988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.627+0000 7f7f62606b80 1 -- 172.21.15.46:0/74998988 >> 172.21.15.46:0/74998988 conn(0x55c39d954220 msgr2=0x55c39d9da020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.627+0000 7f7f62606b80 1 -- 172.21.15.46:0/74998988 shutdown_connections 2024-03-18T20:49:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.627+0000 7f7f62606b80 1 -- 172.21.15.46:0/74998988 wait complete. 2024-03-18T20:49:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.627+0000 7f7f62606b80 1 -- start start 2024-03-18T20:49:10.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.627+0000 7f7f62606b80 1 Processor -- start 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f62606b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f62606b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f62606b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9e9220 0x55c39d9e6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f62606b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d8ccd80 con 0x55c39d9dda60 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f62606b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d8cca20 con 0x55c39d9e9220 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f62606b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d7b46a0 con 0x55c39d9de380 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f62606b80 10 client.0 _ll_register_callbacks cb 0x55c39d7383a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f4ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f4ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9e8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59512/0 (socket says 172.21.15.46:59512) 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f4ffff700 1 -- 172.21.15.46:0/3680818812 learned_addr learned my addr 172.21.15.46:0/3680818812 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f54a0e700 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9e9220 0x55c39d9e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f4ffff700 1 -- 172.21.15.46:0/3680818812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 msgr2=0x55c39d9e5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f4ffff700 1 -- 172.21.15.46:0/3680818812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9e9220 msgr2=0x55c39d9e6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9e9220 0x55c39d9e6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f4ffff700 1 -- 172.21.15.46:0/3680818812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5010d350 con 0x55c39d9dda60 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f54a0e700 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9e9220 0x55c39d9e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9e8c00 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f7f4001ebe0 tx=0x7f7f400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9e1deec3065525 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.628+0000 7f7f2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f2f7fe700 1 -- 172.21.15.46:0/3680818812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40009e60 con 0x55c39d9dda60 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f2f7fe700 1 -- 172.21.15.46:0/3680818812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7f4000ea50 con 0x55c39d9dda60 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f62606b80 1 -- 172.21.15.46:0/3680818812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c39d9d1ad0 con 0x55c39d9dda60 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f2f7fe700 1 -- 172.21.15.46:0/3680818812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f4000ec10 con 0x55c39d9dda60 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f62606b80 10 client.20121 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:10.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f62606b80 1 -- 172.21.15.46:0/3680818812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55c39d75eb60 con 0x55c39d9dda60 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f2f7fe700 1 -- 172.21.15.46:0/3680818812 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f40033080 con 0x55c39d9dda60 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f2f7fe700 1 -- 172.21.15.46:0/3680818812 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f7f4002f420 con 0x55c39d9dda60 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f62606b80 10 client.20121 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f62606b80 10 client.20121 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f62606b80 20 client.20121 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f62606b80 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c39d9f8ec0 0x55c39d9fb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f62606b80 4 client.20121 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.629+0000 7f7f62606b80 1 -- 172.21.15.46:0/3680818812 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c39d929700 con 0x55c39d9f8ec0 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.630+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c39d9f8ec0 0x55c39d9fb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.630+0000 7f7f2f7fe700 10 client.20121 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.630+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c39d9f8ec0 0x55c39d9fb2e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=133ff43b4a776119 server_cookie=c37a68ba24fcccc4 in_seq=0 out_seq=0 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f2f7fe700 1 -- 172.21.15.46:0/3680818812 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c39d929700 con 0x55c39d9f8ec0 2024-03-18T20:49:10.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f2f7fe700 10 client.20121 handle_command_reply: tid=0 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f62606b80 1 client.20121 shutdown 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f62606b80 1 -- 172.21.15.46:0/3680818812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 msgr2=0x55c39d9e8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f62606b80 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9e8c00 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f7f4001ebe0 tx=0x7f7f400047c0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f62606b80 1 -- 172.21.15.46:0/3680818812 shutdown_connections 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f62606b80 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c39d9f8ec0 0x55c39d9fb2e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f62606b80 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9e8c00 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f62606b80 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9e9220 0x55c39d9e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f2f7fe700 0 client.20121 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f62606b80 1 --2- 172.21.15.46:0/3680818812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f62606b80 1 -- 172.21.15.46:0/3680818812 >> 172.21.15.46:0/3680818812 conn(0x55c39d954220 msgr2=0x55c39d9d9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f62606b80 1 -- 172.21.15.46:0/3680818812 shutdown_connections 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f62606b80 1 -- 172.21.15.46:0/3680818812 wait complete. 2024-03-18T20:49:10.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.631+0000 7f7f62606b80 20 client.20121 trim_cache size 0 max 16384 2024-03-18T20:49:10.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.645+0000 7f7f62606b80 1 Processor -- start 2024-03-18T20:49:10.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.645+0000 7f7f62606b80 1 -- start start 2024-03-18T20:49:10.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.645+0000 7f7f62606b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9e64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.645+0000 7f7f62606b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.645+0000 7f7f62606b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9e9220 0x55c39d9e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.645+0000 7f7f62606b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d9f6440 con 0x55c39d9dda60 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.645+0000 7f7f62606b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d9ded30 con 0x55c39d9e9220 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.645+0000 7f7f62606b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d9e5280 con 0x55c39d9de380 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.645+0000 7f7f4ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9e64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.645+0000 7f7f4ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9e64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59524/0 (socket says 172.21.15.46:59524) 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.645+0000 7f7f4ffff700 1 -- 172.21.15.46:0/2776941785 learned_addr learned my addr 172.21.15.46:0/2776941785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.645+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/2776941785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.646+0000 7f7f4ffff700 1 -- 172.21.15.46:0/2776941785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 msgr2=0x55c39d9e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.646+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/2776941785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.646+0000 7f7f54a0e700 1 --2- 172.21.15.46:0/2776941785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9e9220 0x55c39d9e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.646+0000 7f7f4ffff700 1 -- 172.21.15.46:0/2776941785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9e9220 msgr2=0x55c39d9e4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.646+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/2776941785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9e9220 0x55c39d9e4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.646+0000 7f7f4ffff700 1 -- 172.21.15.46:0/2776941785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c39d8f6ea0 con 0x55c39d9dda60 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.646+0000 7f7f54a0e700 1 --2- 172.21.15.46:0/2776941785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9e9220 0x55c39d9e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.646+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/2776941785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9e64f0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000c00 tx=0x7f7f40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d518fe0568dec server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.646+0000 7f7f36ffd700 1 -- 172.21.15.46:0/2776941785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40036070 con 0x55c39d9dda60 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.646+0000 7f7f36ffd700 1 -- 172.21.15.46:0/2776941785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c39d9d1ad0 con 0x55c39d9dda60 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.646+0000 7f7f36ffd700 1 -- 172.21.15.46:0/2776941785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40004030 con 0x55c39d9dda60 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.647+0000 7f7f62606b80 1 -- 172.21.15.46:0/2776941785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 msgr2=0x55c39d9e64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.647+0000 7f7f62606b80 1 --2- 172.21.15.46:0/2776941785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9e64f0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000c00 tx=0x7f7f40004750 comp rx=0 tx=0).stop 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.647+0000 7f7f62606b80 1 -- 172.21.15.46:0/2776941785 shutdown_connections 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.647+0000 7f7f62606b80 1 --2- 172.21.15.46:0/2776941785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9e64f0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.647+0000 7f7f62606b80 1 --2- 172.21.15.46:0/2776941785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9e9220 0x55c39d9e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.647+0000 7f7f62606b80 1 --2- 172.21.15.46:0/2776941785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9de380 0x55c39d9e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.647+0000 7f7f62606b80 1 -- 172.21.15.46:0/2776941785 >> 172.21.15.46:0/2776941785 conn(0x55c39d954220 msgr2=0x55c39d9e1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.647+0000 7f7f62606b80 1 -- 172.21.15.46:0/2776941785 shutdown_connections 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.647+0000 7f7f62606b80 1 -- 172.21.15.46:0/2776941785 wait complete. 2024-03-18T20:49:10.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.647+0000 7f7f62606b80 1 -- start start 2024-03-18T20:49:10.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.647+0000 7f7f62606b80 1 Processor -- start 2024-03-18T20:49:10.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f62606b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9edfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f62606b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9de380 0x55c39d9ee500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f62606b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9e9220 0x55c39da45d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f62606b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39da46270 con 0x55c39d9dda60 2024-03-18T20:49:10.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f62606b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39da463e0 con 0x55c39d9de380 2024-03-18T20:49:10.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f62606b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39da465a0 con 0x55c39d9e9220 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f62606b80 10 client.0 _ll_register_callbacks cb 0x7f7f4001e7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f4ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9edfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f4ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9edfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59536/0 (socket says 172.21.15.46:59536) 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f4ffff700 1 -- 172.21.15.46:0/1274104583 learned_addr learned my addr 172.21.15.46:0/1274104583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f54a0e700 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9e9220 0x55c39da45d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9de380 0x55c39d9ee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f4ffff700 1 -- 172.21.15.46:0/1274104583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9e9220 msgr2=0x55c39da45d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9e9220 0x55c39da45d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f4ffff700 1 -- 172.21.15.46:0/1274104583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9de380 msgr2=0x55c39d9ee500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9de380 0x55c39d9ee500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f4ffff700 1 -- 172.21.15.46:0/1274104583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c39d75eb60 con 0x55c39d9dda60 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9de380 0x55c39d9ee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f54a0e700 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9e9220 0x55c39da45d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:49:10.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f4ffff700 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9edfc0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f7f40005cb0 tx=0x7f7f40004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e618233bdd055fda server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.648+0000 7f7f2f7fe700 1 -- 172.21.15.46:0/1274104583 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40036070 con 0x55c39d9dda60 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f62606b80 1 -- 172.21.15.46:0/1274104583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f48007cc0 con 0x55c39d9dda60 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f2f7fe700 1 -- 172.21.15.46:0/1274104583 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7f40005040 con 0x55c39d9dda60 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f2f7fe700 1 -- 172.21.15.46:0/1274104583 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f40005200 con 0x55c39d9dda60 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f62606b80 10 client.20127 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f62606b80 1 -- 172.21.15.46:0/1274104583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55c39d7b6cd0 con 0x55c39d9dda60 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f2f7fe700 1 -- 172.21.15.46:0/1274104583 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f48007cc0 con 0x55c39d9dda60 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f2f7fe700 1 -- 172.21.15.46:0/1274104583 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f7f40027440 con 0x55c39d9dda60 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f62606b80 10 client.20127 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f62606b80 10 client.20127 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f62606b80 20 client.20127 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f62606b80 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c39da4d8d0 0x55c39da4fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f62606b80 4 client.20127 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:49:10.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f62606b80 1 -- 172.21.15.46:0/1274104583 --> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c39d77fcf0 con 0x55c39da4d8d0 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.649+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c39da4d8d0 0x55c39da4fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f2f7fe700 10 client.20127 ms_handle_connect on v2:172.21.15.155:6838/3116856560 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f4f7fe700 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c39da4d8d0 0x55c39da4fcc0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ebded75b36a77b5 server_cookie=85606d6ce5e07b37 in_seq=0 out_seq=0 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f2f7fe700 1 -- 172.21.15.46:0/1274104583 <== mds.2 v2:172.21.15.155:6838/3116856560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+236 (crc 0 0 0) 0x55c39d77fcf0 con 0x55c39da4d8d0 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f2f7fe700 10 client.20127 handle_command_reply: tid=0 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f62606b80 1 client.20127 shutdown 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f62606b80 1 -- 172.21.15.46:0/1274104583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 msgr2=0x55c39d9edfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f62606b80 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9edfc0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f7f40005cb0 tx=0x7f7f40004690 comp rx=0 tx=0).stop 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f62606b80 1 -- 172.21.15.46:0/1274104583 shutdown_connections 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f62606b80 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.155:6838/3116856560,v1:172.21.15.155:6839/3116856560] conn(0x55c39da4d8d0 0x55c39da4fcc0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f62606b80 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c39d9dda60 0x55c39d9edfc0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f62606b80 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c39d9e9220 0x55c39da45d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f2f7fe700 0 client.20127 ms_handle_reset on v2:172.21.15.155:6838/3116856560 2024-03-18T20:49:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f62606b80 1 --2- 172.21.15.46:0/1274104583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c39d9de380 0x55c39d9ee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f62606b80 1 -- 172.21.15.46:0/1274104583 >> 172.21.15.46:0/1274104583 conn(0x55c39d954220 msgr2=0x55c39d9e1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f62606b80 1 -- 172.21.15.46:0/1274104583 shutdown_connections 2024-03-18T20:49:10.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f62606b80 1 -- 172.21.15.46:0/1274104583 wait complete. 2024-03-18T20:49:10.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.650+0000 7f7f62606b80 20 client.20127 trim_cache size 0 max 16384 2024-03-18T20:49:10.659 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-18T20:49:10.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T20:49:10.659 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-18T20:49:10.659 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 3799837018, 2024-03-18T20:49:10.659 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511630769, 2024-03-18T20:49:10.659 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB" 2024-03-18T20:49:10.660 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T20:49:10.660 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-18T20:49:10.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.651+0000 7f7f56471700 1 -- 172.21.15.46:0/2479598425 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7f380a1310 msgr2=0x7f7f380a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.651+0000 7f7f56471700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7f380a1310 0x7f7f380a37d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009c60 tx=0x7f7f40009260 comp rx=0 tx=0).stop 2024-03-18T20:49:10.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.651+0000 7f7f56471700 1 -- 172.21.15.46:0/2479598425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50104f60 msgr2=0x7f7f50101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.651+0000 7f7f56471700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50104f60 0x7f7f50101e10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400d950 tx=0x7f7f4400dc60 comp rx=0 tx=0).stop 2024-03-18T20:49:10.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.651+0000 7f7f56471700 1 -- 172.21.15.46:0/2479598425 shutdown_connections 2024-03-18T20:49:10.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.651+0000 7f7f56471700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7f380a1310 0x7f7f380a37d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.651+0000 7f7f56471700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f50102350 0x7f7f501a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.651+0000 7f7f56471700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f500ffe50 0x7f7f501018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.651+0000 7f7f56471700 1 --2- 172.21.15.46:0/2479598425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f50104f60 0x7f7f50101e10 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.651+0000 7f7f56471700 1 -- 172.21.15.46:0/2479598425 >> 172.21.15.46:0/2479598425 conn(0x7f7f50076d40 msgr2=0x7f7f50068200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.651+0000 7f7f56471700 1 -- 172.21.15.46:0/2479598425 shutdown_connections 2024-03-18T20:49:10.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.651+0000 7f7f56471700 1 -- 172.21.15.46:0/2479598425 wait complete. 2024-03-18T20:49:10.667 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:49:10.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T20:49:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:10 smithi073 ceph-mon[47384]: pgmap v408: 97 pgs: 97 active+clean; 219 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-03-18T20:49:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:10 smithi073 ceph-mon[47384]: osdmap e96: 12 total, 12 up, 12 in 2024-03-18T20:49:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:10 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/265807015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:49:10.726 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 605 sec 2024-03-18T20:49:10.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:10 smithi155 ceph-mon[48344]: pgmap v408: 97 pgs: 97 active+clean; 219 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-03-18T20:49:10.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:10 smithi155 ceph-mon[48344]: osdmap e96: 12 total, 12 up, 12 in 2024-03-18T20:49:10.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:10 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/265807015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T20:49:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.803+0000 7fc1ce521700 1 -- 172.21.15.46:0/1025265182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c80ffe70 msgr2=0x7fc1c8100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.803+0000 7fc1ce521700 1 --2- 172.21.15.46:0/1025265182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c80ffe70 0x7fc1c8100250 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009990 tx=0x7fc1b8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.803+0000 7fc1ce521700 1 -- 172.21.15.46:0/1025265182 shutdown_connections 2024-03-18T20:49:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.803+0000 7fc1ce521700 1 --2- 172.21.15.46:0/1025265182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c80ffe70 0x7fc1c8100250 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.803+0000 7fc1ce521700 1 --2- 172.21.15.46:0/1025265182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1c8104f90 0x7fc1c810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.803+0000 7fc1ce521700 1 --2- 172.21.15.46:0/1025265182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c8100820 0x7fc1c8104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.803+0000 7fc1ce521700 1 -- 172.21.15.46:0/1025265182 >> 172.21.15.46:0/1025265182 conn(0x7fc1c8076d80 msgr2=0x7fc1c8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.803+0000 7fc1ce521700 1 -- 172.21.15.46:0/1025265182 shutdown_connections 2024-03-18T20:49:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.803+0000 7fc1ce521700 1 -- 172.21.15.46:0/1025265182 wait complete. 2024-03-18T20:49:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.804+0000 7fc1ce521700 1 Processor -- start 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.804+0000 7fc1ce521700 1 -- start start 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1ce521700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c80ffe70 0x7fc1c81960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1ce521700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c8104f90 0x7fc1c8196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1ce521700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1c819a9c0 0x7fc1c819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1ce521700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c819b440 con 0x7fc1c80ffe70 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1ce521700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c819b5b0 con 0x7fc1c8104f90 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1ce521700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c819b770 con 0x7fc1c819a9c0 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c80ffe70 0x7fc1c81960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c80ffe70 0x7fc1c81960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59564/0 (socket says 172.21.15.46:59564) 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c7fff700 1 -- 172.21.15.46:0/2513164821 learned_addr learned my addr 172.21.15.46:0/2513164821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c8104f90 0x7fc1c8196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1ccabe700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1c819a9c0 0x7fc1c819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c7fff700 1 -- 172.21.15.46:0/2513164821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1c819a9c0 msgr2=0x7fc1c819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c7fff700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1c819a9c0 0x7fc1c819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c7fff700 1 -- 172.21.15.46:0/2513164821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c8104f90 msgr2=0x7fc1c8196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c7fff700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c8104f90 0x7fc1c8196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c7fff700 1 -- 172.21.15.46:0/2513164821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b80095f0 con 0x7fc1c80ffe70 2024-03-18T20:49:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c8104f90 0x7fc1c8196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:49:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c7fff700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c80ffe70 0x7fc1c81960e0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8004760 tx=0x7fc1b8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c57fa700 1 -- 172.21.15.46:0/2513164821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b801d070 con 0x7fc1c80ffe70 2024-03-18T20:49:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1ce521700 1 -- 172.21.15.46:0/2513164821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c81a2c70 con 0x7fc1c80ffe70 2024-03-18T20:49:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c57fa700 1 -- 172.21.15.46:0/2513164821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1b800fbf0 con 0x7fc1c80ffe70 2024-03-18T20:49:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.805+0000 7fc1c57fa700 1 -- 172.21.15.46:0/2513164821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8022d50 con 0x7fc1c80ffe70 2024-03-18T20:49:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.806+0000 7fc1ce521700 1 -- 172.21.15.46:0/2513164821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c81a30f0 con 0x7fc1c80ffe70 2024-03-18T20:49:10.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.809+0000 7fc1c57fa700 1 -- 172.21.15.46:0/2513164821 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc1b8022470 con 0x7fc1c80ffe70 2024-03-18T20:49:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.809+0000 7fc1c57fa700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1b00a1290 0x7fc1b00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.809+0000 7fc1c57fa700 1 -- 172.21.15.46:0/2513164821 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc1b80cdda0 con 0x7fc1c80ffe70 2024-03-18T20:49:10.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.809+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1b00a1290 0x7fc1b00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1da6b6b80 1 Processor -- start 2024-03-18T20:49:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1da6b6b80 1 -- start start 2024-03-18T20:49:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1b00a1290 0x7fc1b00a3750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8197630 tx=0x7fc1bc0073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1da6b6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d1a60 0x56503d2d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1da6b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 0x56503d2dcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1da6b6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2dd220 0x56503d2df610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1da6b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503d22b990 con 0x56503d2d2380 2024-03-18T20:49:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1da6b6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503d229350 con 0x56503d2d1a60 2024-03-18T20:49:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1da6b6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503d224c40 con 0x56503d2dd220 2024-03-18T20:49:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1c77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 0x56503d2dcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1c77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 0x56503d2dcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59578/0 (socket says 172.21.15.46:59578) 2024-03-18T20:49:10.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1c77fe700 1 -- 172.21.15.46:0/3462908547 learned_addr learned my addr 172.21.15.46:0/3462908547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1c7fff700 1 --2- 172.21.15.46:0/3462908547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d1a60 0x56503d2d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.810+0000 7fc1ccabe700 1 --2- 172.21.15.46:0/3462908547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2dd220 0x56503d2df610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.811+0000 7fc1c77fe700 1 -- 172.21.15.46:0/3462908547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2dd220 msgr2=0x56503d2df610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.811+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/3462908547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2dd220 0x56503d2df610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.811+0000 7fc1c77fe700 1 -- 172.21.15.46:0/3462908547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d1a60 msgr2=0x56503d2d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.811+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/3462908547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d1a60 0x56503d2d1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.811+0000 7fc1c77fe700 1 -- 172.21.15.46:0/3462908547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56503d0a63b0 con 0x56503d2d2380 2024-03-18T20:49:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.811+0000 7fc1ccabe700 1 --2- 172.21.15.46:0/3462908547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2dd220 0x56503d2df610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:49:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.811+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/3462908547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 0x56503d2dcce0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc004380 tx=0x7fc1bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331c5804c73cb82c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.811+0000 7fc1ae7fc700 1 -- 172.21.15.46:0/3462908547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc003900 con 0x56503d2d2380 2024-03-18T20:49:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.811+0000 7fc1ae7fc700 1 -- 172.21.15.46:0/3462908547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1bc011440 con 0x56503d2d2380 2024-03-18T20:49:10.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.811+0000 7fc1ae7fc700 1 -- 172.21.15.46:0/3462908547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc01f690 con 0x56503d2d2380 2024-03-18T20:49:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.812+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/3462908547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 msgr2=0x56503d2dcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.812+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/3462908547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 0x56503d2dcce0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc004380 tx=0x7fc1bc013000 comp rx=0 tx=0).stop 2024-03-18T20:49:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.812+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/3462908547 shutdown_connections 2024-03-18T20:49:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.812+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/3462908547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2dd220 0x56503d2df610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.812+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/3462908547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d1a60 0x56503d2d1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.812+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/3462908547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 0x56503d2dcce0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.812+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/3462908547 >> 172.21.15.46:0/3462908547 conn(0x56503d248220 msgr2=0x56503d2ce020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.812+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/3462908547 shutdown_connections 2024-03-18T20:49:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.812+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/3462908547 wait complete. 2024-03-18T20:49:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.812+0000 7fc1da6b6b80 1 -- start start 2024-03-18T20:49:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1da6b6b80 1 Processor -- start 2024-03-18T20:49:10.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1da6b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d1a60 0x56503d2e4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1da6b6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2d2380 0x56503d2e5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1da6b6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2dd220 0x56503d2dc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1da6b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503d1c0d80 con 0x56503d2d1a60 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1da6b6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503d1c0a20 con 0x56503d2dd220 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1da6b6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503d0a86a0 con 0x56503d2d2380 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1da6b6b80 10 client.0 _ll_register_callbacks cb 0x56503d02c3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d1a60 0x56503d2e4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1c7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d1a60 0x56503d2e4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59582/0 (socket says 172.21.15.46:59582) 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1c7fff700 1 -- 172.21.15.46:0/2262995798 learned_addr learned my addr 172.21.15.46:0/2262995798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1ccabe700 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2dd220 0x56503d2dc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2d2380 0x56503d2e5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1c7fff700 1 -- 172.21.15.46:0/2262995798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2d2380 msgr2=0x56503d2e5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1c7fff700 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2d2380 0x56503d2e5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1c7fff700 1 -- 172.21.15.46:0/2262995798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2dd220 msgr2=0x56503d2dc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1c7fff700 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2dd220 0x56503d2dc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1c7fff700 1 -- 172.21.15.46:0/2262995798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc004030 con 0x56503d2d1a60 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.813+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2d2380 0x56503d2e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1c7fff700 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d1a60 0x56503d2e4b20 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80cb580 tx=0x7fc1b8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ae458c11f7686b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1a77fe700 1 -- 172.21.15.46:0/2262995798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b809b620 con 0x56503d2d1a60 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1a77fe700 1 -- 172.21.15.46:0/2262995798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc1b809af20 con 0x56503d2d1a60 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/2262995798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56503d2c5ad0 con 0x56503d2d1a60 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1a77fe700 1 -- 172.21.15.46:0/2262995798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1b809ed50 con 0x56503d2d1a60 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1da6b6b80 10 client.20139 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/2262995798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x56503d052b60 con 0x56503d2d1a60 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1a77fe700 1 -- 172.21.15.46:0/2262995798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1b80b7020 con 0x56503d2d1a60 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1a77fe700 1 -- 172.21.15.46:0/2262995798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fc1b8095c30 con 0x56503d2d1a60 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1da6b6b80 10 client.20139 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:10.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1da6b6b80 10 client.20139 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1da6b6b80 20 client.20139 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56503d2eec80 0x56503d2f1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1da6b6b80 4 client.20139 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/2262995798 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56503d21d700 con 0x56503d2eec80 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.814+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56503d2eec80 0x56503d2f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.815+0000 7fc1a77fe700 10 client.20139 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.815+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56503d2eec80 0x56503d2f1070 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31133ea4089ebd0b server_cookie=abd69ce9fe57d718 in_seq=0 out_seq=0 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.815+0000 7fc1a77fe700 1 -- 172.21.15.46:0/2262995798 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56503d21d700 con 0x56503d2eec80 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.815+0000 7fc1a77fe700 10 client.20139 handle_command_reply: tid=0 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.815+0000 7fc1da6b6b80 1 client.20139 shutdown 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.816+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/2262995798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d1a60 msgr2=0x56503d2e4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.816+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d1a60 0x56503d2e4b20 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80cb580 tx=0x7fc1b8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.816+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/2262995798 shutdown_connections 2024-03-18T20:49:10.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.816+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56503d2eec80 0x56503d2f1070 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.816+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d1a60 0x56503d2e4b20 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.816+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2dd220 0x56503d2dc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.816+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/2262995798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2d2380 0x56503d2e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.816+0000 7fc1a77fe700 0 client.20139 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:49:10.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.816+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/2262995798 >> 172.21.15.46:0/2262995798 conn(0x56503d248220 msgr2=0x56503d2cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.816+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/2262995798 shutdown_connections 2024-03-18T20:49:10.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.816+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/2262995798 wait complete. 2024-03-18T20:49:10.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.816+0000 7fc1da6b6b80 20 client.20139 trim_cache size 0 max 16384 2024-03-18T20:49:10.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.831+0000 7fc1da6b6b80 1 Processor -- start 2024-03-18T20:49:10.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.831+0000 7fc1da6b6b80 1 -- start start 2024-03-18T20:49:10.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.831+0000 7fc1da6b6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2d1a60 0x56503d2d9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.831+0000 7fc1da6b6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d2380 0x56503d2d88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.831+0000 7fc1da6b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2dd220 0x56503d2d8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.831+0000 7fc1da6b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503d2c8210 con 0x56503d2dd220 2024-03-18T20:49:10.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.831+0000 7fc1da6b6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503d2dfe20 con 0x56503d2d2380 2024-03-18T20:49:10.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.831+0000 7fc1da6b6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503d2d2d30 con 0x56503d2d1a60 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1ccabe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2dd220 0x56503d2d8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1ccabe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2dd220 0x56503d2d8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59590/0 (socket says 172.21.15.46:59590) 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1ccabe700 1 -- 172.21.15.46:0/1063695512 learned_addr learned my addr 172.21.15.46:0/1063695512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/1063695512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d2380 0x56503d2d88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1c7fff700 1 --2- 172.21.15.46:0/1063695512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2d1a60 0x56503d2d9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1ccabe700 1 -- 172.21.15.46:0/1063695512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2d1a60 msgr2=0x56503d2d9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1ccabe700 1 --2- 172.21.15.46:0/1063695512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2d1a60 0x56503d2d9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1ccabe700 1 -- 172.21.15.46:0/1063695512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d2380 msgr2=0x56503d2d88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1ccabe700 1 --2- 172.21.15.46:0/1063695512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d2380 0x56503d2d88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1ccabe700 1 -- 172.21.15.46:0/1063695512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56503d1eaea0 con 0x56503d2dd220 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/1063695512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d2380 0x56503d2d88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1ccabe700 1 --2- 172.21.15.46:0/1063695512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2dd220 0x56503d2d8de0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009db0 tx=0x7fc1c00077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8be01f7e1f1e880 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1aeffd700 1 -- 172.21.15.46:0/1063695512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0009900 con 0x56503d2dd220 2024-03-18T20:49:10.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1aeffd700 1 -- 172.21.15.46:0/1063695512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1c000f070 con 0x56503d2dd220 2024-03-18T20:49:10.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.832+0000 7fc1aeffd700 1 -- 172.21.15.46:0/1063695512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c00167c0 con 0x56503d2dd220 2024-03-18T20:49:10.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.833+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1063695512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2dd220 msgr2=0x56503d2d8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.833+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/1063695512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2dd220 0x56503d2d8de0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0009db0 tx=0x7fc1c00077e0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.833+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1063695512 shutdown_connections 2024-03-18T20:49:10.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.833+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/1063695512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2dd220 0x56503d2d8de0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.833+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/1063695512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2d1a60 0x56503d2d9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.833+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/1063695512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d2380 0x56503d2d88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.833+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1063695512 >> 172.21.15.46:0/1063695512 conn(0x56503d248220 msgr2=0x56503d2c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.833+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1063695512 shutdown_connections 2024-03-18T20:49:10.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.833+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1063695512 wait complete. 2024-03-18T20:49:10.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1da6b6b80 1 -- start start 2024-03-18T20:49:10.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1da6b6b80 1 Processor -- start 2024-03-18T20:49:10.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1da6b6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d1a60 0x56503d2eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1da6b6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 0x56503d2eb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1da6b6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2dd220 0x56503d2e93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1da6b6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503d2e9ac0 con 0x56503d2d2380 2024-03-18T20:49:10.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1da6b6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503d2e9c30 con 0x56503d2d1a60 2024-03-18T20:49:10.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1da6b6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503d2e9df0 con 0x56503d2dd220 2024-03-18T20:49:10.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1da6b6b80 10 client.0 _ll_register_callbacks cb 0x7fc1b809aa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:10.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1c77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 0x56503d2eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1c77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 0x56503d2eb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59602/0 (socket says 172.21.15.46:59602) 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1ccabe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2dd220 0x56503d2e93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1c77fe700 1 -- 172.21.15.46:0/1135028645 learned_addr learned my addr 172.21.15.46:0/1135028645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.834+0000 7fc1c7fff700 1 --2- 172.21.15.46:0/1135028645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d1a60 0x56503d2eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1c77fe700 1 -- 172.21.15.46:0/1135028645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2dd220 msgr2=0x56503d2e93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/1135028645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2dd220 0x56503d2e93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1c77fe700 1 -- 172.21.15.46:0/1135028645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d1a60 msgr2=0x56503d2eaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/1135028645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d1a60 0x56503d2eaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1c77fe700 1 -- 172.21.15.46:0/1135028645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56503d0a63b0 con 0x56503d2d2380 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1c77fe700 1 --2- 172.21.15.46:0/1135028645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 0x56503d2eb4c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc004350 tx=0x7fc1bc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9f31c0287b46b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1a77fe700 1 -- 172.21.15.46:0/1135028645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc032040 con 0x56503d2d2380 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1a77fe700 1 -- 172.21.15.46:0/1135028645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc1bc017070 con 0x56503d2d2380 2024-03-18T20:49:10.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1135028645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1bc011440 con 0x56503d2d2380 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1a77fe700 1 -- 172.21.15.46:0/1135028645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1bc011940 con 0x56503d2d2380 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1da6b6b80 10 client.20145 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.835+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1135028645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7fc1c0009a60 con 0x56503d2d2380 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.836+0000 7fc1a77fe700 1 -- 172.21.15.46:0/1135028645 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1bc043080 con 0x56503d2d2380 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.836+0000 7fc1a77fe700 1 -- 172.21.15.46:0/1135028645 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fc1bc004c90 con 0x56503d2d2380 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.836+0000 7fc1da6b6b80 10 client.20145 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.836+0000 7fc1da6b6b80 10 client.20145 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.836+0000 7fc1da6b6b80 20 client.20145 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.836+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/1135028645 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56503d343ca0 0x56503d3460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.836+0000 7fc1da6b6b80 4 client.20145 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.836+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1135028645 --> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56503d073cf0 con 0x56503d343ca0 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.836+0000 7fc1c7fff700 1 --2- 172.21.15.46:0/1135028645 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56503d343ca0 0x56503d3460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.836+0000 7fc1a77fe700 10 client.20145 ms_handle_connect on v2:172.21.15.46:6838/1267173621 2024-03-18T20:49:10.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.836+0000 7fc1c7fff700 1 --2- 172.21.15.46:0/1135028645 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56503d343ca0 0x56503d3460c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7d0c45cb6d7c7bf server_cookie=a54785135fc86a9 in_seq=0 out_seq=0 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1a77fe700 1 -- 172.21.15.46:0/1135028645 <== mds.1 v2:172.21.15.46:6838/1267173621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56503d073cf0 con 0x56503d343ca0 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1a77fe700 10 client.20145 handle_command_reply: tid=0 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1da6b6b80 1 client.20145 shutdown 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1135028645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 msgr2=0x56503d2eb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/1135028645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 0x56503d2eb4c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc004350 tx=0x7fc1bc011ba0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1135028645 shutdown_connections 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/1135028645 >> [v2:172.21.15.46:6838/1267173621,v1:172.21.15.46:6839/1267173621] conn(0x56503d343ca0 0x56503d3460c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/1135028645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56503d2dd220 0x56503d2e93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/1135028645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56503d2d1a60 0x56503d2eaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1da6b6b80 1 --2- 172.21.15.46:0/1135028645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56503d2d2380 0x56503d2eb4c0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1a77fe700 0 client.20145 ms_handle_reset on v2:172.21.15.46:6838/1267173621 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1135028645 >> 172.21.15.46:0/1135028645 conn(0x56503d248220 msgr2=0x56503d2c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1135028645 shutdown_connections 2024-03-18T20:49:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1da6b6b80 1 -- 172.21.15.46:0/1135028645 wait complete. 2024-03-18T20:49:10.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.837+0000 7fc1da6b6b80 20 client.20145 trim_cache size 0 max 16384 2024-03-18T20:49:10.846 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:49:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.838+0000 7fc1ce521700 1 -- 172.21.15.46:0/2513164821 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1b00a1290 msgr2=0x7fc1b00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.838+0000 7fc1ce521700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1b00a1290 0x7fc1b00a3750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8197630 tx=0x7fc1bc0073d0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.838+0000 7fc1ce521700 1 -- 172.21.15.46:0/2513164821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c80ffe70 msgr2=0x7fc1c81960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.838+0000 7fc1ce521700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c80ffe70 0x7fc1c81960e0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8004760 tx=0x7fc1b8004790 comp rx=0 tx=0).stop 2024-03-18T20:49:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.838+0000 7fc1ce521700 1 -- 172.21.15.46:0/2513164821 shutdown_connections 2024-03-18T20:49:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.838+0000 7fc1ce521700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc1b00a1290 0x7fc1b00a3750 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.838+0000 7fc1ce521700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c80ffe70 0x7fc1c81960e0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.838+0000 7fc1ce521700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1c819a9c0 0x7fc1c819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.838+0000 7fc1ce521700 1 --2- 172.21.15.46:0/2513164821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c8104f90 0x7fc1c8196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.838+0000 7fc1ce521700 1 -- 172.21.15.46:0/2513164821 >> 172.21.15.46:0/2513164821 conn(0x7fc1c8076d80 msgr2=0x7fc1c8106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.839+0000 7fc1ce521700 1 -- 172.21.15.46:0/2513164821 shutdown_connections 2024-03-18T20:49:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.839+0000 7fc1ce521700 1 -- 172.21.15.46:0/2513164821 wait complete. 2024-03-18T20:49:10.854 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-18T20:49:10.854 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T20:49:10.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.991+0000 7f465ea7b700 1 -- 172.21.15.46:0/3271651111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4658067e30 msgr2=0x7f46580682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.991+0000 7f465ea7b700 1 --2- 172.21.15.46:0/3271651111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4658067e30 0x7f46580682b0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4644009990 tx=0x7f4644009ca0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.991+0000 7f465ea7b700 1 -- 172.21.15.46:0/3271651111 shutdown_connections 2024-03-18T20:49:10.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.991+0000 7f465ea7b700 1 --2- 172.21.15.46:0/3271651111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46580687f0 0x7f465810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.991+0000 7f465ea7b700 1 --2- 172.21.15.46:0/3271651111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4658105ce0 0x7f46581060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.991+0000 7f465ea7b700 1 --2- 172.21.15.46:0/3271651111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4658067e30 0x7f46580682b0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.991+0000 7f465ea7b700 1 -- 172.21.15.46:0/3271651111 >> 172.21.15.46:0/3271651111 conn(0x7f4658076d40 msgr2=0x7f4658077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.992+0000 7f465ea7b700 1 -- 172.21.15.46:0/3271651111 shutdown_connections 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.992+0000 7f465ea7b700 1 -- 172.21.15.46:0/3271651111 wait complete. 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.992+0000 7f465ea7b700 1 Processor -- start 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.992+0000 7f465ea7b700 1 -- start start 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.992+0000 7f465ea7b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4658067e30 0x7f4658197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.992+0000 7f465ea7b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46580687f0 0x7f4658197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.992+0000 7f465ea7b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4658105ce0 0x7f465819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465ea7b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465819c130 con 0x7f4658067e30 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465ea7b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465819c2a0 con 0x7f4658105ce0 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465ea7b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465819c460 con 0x7f46580687f0 2024-03-18T20:49:10.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465c817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4658067e30 0x7f4658197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465c817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4658067e30 0x7f4658197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59624/0 (socket says 172.21.15.46:59624) 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465c817700 1 -- 172.21.15.46:0/2702434033 learned_addr learned my addr 172.21.15.46:0/2702434033 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465d018700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4658105ce0 0x7f465819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f4657fff700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46580687f0 0x7f4658197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465c817700 1 -- 172.21.15.46:0/2702434033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46580687f0 msgr2=0x7f4658197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465c817700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46580687f0 0x7f4658197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465c817700 1 -- 172.21.15.46:0/2702434033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4658105ce0 msgr2=0x7f465819baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465c817700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4658105ce0 0x7f465819baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465c817700 1 -- 172.21.15.46:0/2702434033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46440095f0 con 0x7f4658067e30 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465c817700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4658067e30 0x7f4658197210 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f464c00d980 tx=0x7f464c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f4655ffb700 1 -- 172.21.15.46:0/2702434033 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c004220 con 0x7f4658067e30 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f4655ffb700 1 -- 172.21.15.46:0/2702434033 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f464c007640 con 0x7f4658067e30 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f465ea7b700 1 -- 172.21.15.46:0/2702434033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465819c740 con 0x7f4658067e30 2024-03-18T20:49:10.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.993+0000 7f4655ffb700 1 -- 172.21.15.46:0/2702434033 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c005610 con 0x7f4658067e30 2024-03-18T20:49:10.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.994+0000 7f465ea7b700 1 -- 172.21.15.46:0/2702434033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46581a3f80 con 0x7f4658067e30 2024-03-18T20:49:10.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.996+0000 7f4655ffb700 1 -- 172.21.15.46:0/2702434033 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f464c004750 con 0x7f4658067e30 2024-03-18T20:49:10.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.996+0000 7f4655ffb700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f46480a1310 0x7f46480a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.997+0000 7f4655ffb700 1 -- 172.21.15.46:0/2702434033 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f464c0ccb10 con 0x7f4658067e30 2024-03-18T20:49:11.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.997+0000 7f4657fff700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f46480a1310 0x7f46480a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.997+0000 7f4657fff700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f46480a1310 0x7f46480a37d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4644006010 tx=0x7f464400b260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:11.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.997+0000 7f466ac10b80 1 Processor -- start 2024-03-18T20:49:11.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f466ac10b80 1 -- start start 2024-03-18T20:49:11.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f466ac10b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 0x55b99d642e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f466ac10b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d643380 0x55b99d64dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f466ac10b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d64e220 0x55b99d650610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f466ac10b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d59c990 con 0x55b99d642a60 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f466ac10b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d59a350 con 0x55b99d64e220 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f466ac10b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d595c40 con 0x55b99d643380 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f465c817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 0x55b99d642e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f465c817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 0x55b99d642e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59628/0 (socket says 172.21.15.46:59628) 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f465c817700 1 -- 172.21.15.46:0/2763243568 learned_addr learned my addr 172.21.15.46:0/2763243568 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f4657fff700 1 --2- 172.21.15.46:0/2763243568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d643380 0x55b99d64dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f465d018700 1 --2- 172.21.15.46:0/2763243568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d64e220 0x55b99d650610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f465c817700 1 -- 172.21.15.46:0/2763243568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d643380 msgr2=0x55b99d64dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f465c817700 1 --2- 172.21.15.46:0/2763243568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d643380 0x55b99d64dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f465c817700 1 -- 172.21.15.46:0/2763243568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d64e220 msgr2=0x55b99d650610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f465c817700 1 --2- 172.21.15.46:0/2763243568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d64e220 0x55b99d650610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f465c817700 1 -- 172.21.15.46:0/2763243568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b99d4173b0 con 0x55b99d642a60 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.998+0000 7f465d018700 1 --2- 172.21.15.46:0/2763243568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d64e220 0x55b99d650610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.999+0000 7f465c817700 1 --2- 172.21.15.46:0/2763243568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 0x55b99d642e40 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f464c0a1af0 tx=0x7f464c09a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424a59b33565d156 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.999+0000 7f4642ffd700 1 -- 172.21.15.46:0/2763243568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0a2020 con 0x55b99d642a60 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.999+0000 7f4642ffd700 1 -- 172.21.15.46:0/2763243568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f464c097f40 con 0x55b99d642a60 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.999+0000 7f4642ffd700 1 -- 172.21.15.46:0/2763243568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0992f0 con 0x55b99d642a60 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.999+0000 7f466ac10b80 1 -- 172.21.15.46:0/2763243568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 msgr2=0x55b99d642e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:10.999+0000 7f466ac10b80 1 --2- 172.21.15.46:0/2763243568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 0x55b99d642e40 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f464c0a1af0 tx=0x7f464c09a320 comp rx=0 tx=0).stop 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.000+0000 7f466ac10b80 1 -- 172.21.15.46:0/2763243568 shutdown_connections 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.000+0000 7f466ac10b80 1 --2- 172.21.15.46:0/2763243568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 0x55b99d642e40 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.000+0000 7f466ac10b80 1 --2- 172.21.15.46:0/2763243568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d64e220 0x55b99d650610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.000+0000 7f466ac10b80 1 --2- 172.21.15.46:0/2763243568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d643380 0x55b99d64dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.000+0000 7f466ac10b80 1 -- 172.21.15.46:0/2763243568 >> 172.21.15.46:0/2763243568 conn(0x55b99d5b9220 msgr2=0x55b99d63f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.000+0000 7f466ac10b80 1 -- 172.21.15.46:0/2763243568 shutdown_connections 2024-03-18T20:49:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.000+0000 7f466ac10b80 1 -- 172.21.15.46:0/2763243568 wait complete. 2024-03-18T20:49:11.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.000+0000 7f466ac10b80 1 -- start start 2024-03-18T20:49:11.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.000+0000 7f466ac10b80 1 Processor -- start 2024-03-18T20:49:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f466ac10b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 0x55b99d64dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f466ac10b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d643380 0x55b99d64b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f466ac10b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d64e220 0x55b99d64b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f466ac10b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d531d80 con 0x55b99d642a60 2024-03-18T20:49:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f466ac10b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d531a20 con 0x55b99d643380 2024-03-18T20:49:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f466ac10b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d4196a0 con 0x55b99d64e220 2024-03-18T20:49:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f466ac10b80 10 client.0 _ll_register_callbacks cb 0x55b99d39d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f465c817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 0x55b99d64dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f465c817700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 0x55b99d64dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59640/0 (socket says 172.21.15.46:59640) 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f465c817700 1 -- 172.21.15.46:0/456889933 learned_addr learned my addr 172.21.15.46:0/456889933 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f465d018700 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d64e220 0x55b99d64b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f4657fff700 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d643380 0x55b99d64b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f465c817700 1 -- 172.21.15.46:0/456889933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d64e220 msgr2=0x55b99d64b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f465c817700 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d64e220 0x55b99d64b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f465c817700 1 -- 172.21.15.46:0/456889933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d643380 msgr2=0x55b99d64b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f465c817700 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d643380 0x55b99d64b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f465c817700 1 -- 172.21.15.46:0/456889933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46581a3f80 con 0x55b99d642a60 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f465c817700 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 0x55b99d64dcd0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f464c0ca760 tx=0x7f464c0a20d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3c07cb9f5ac862 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f4637fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f4637fff700 1 -- 172.21.15.46:0/456889933 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c099e00 con 0x55b99d642a60 2024-03-18T20:49:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f4637fff700 1 -- 172.21.15.46:0/456889933 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f464c0b9050 con 0x55b99d642a60 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.001+0000 7f466ac10b80 1 -- 172.21.15.46:0/456889933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b99d636ad0 con 0x55b99d642a60 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.002+0000 7f4637fff700 1 -- 172.21.15.46:0/456889933 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f464c0830c0 con 0x55b99d642a60 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.002+0000 7f466ac10b80 10 client.20157 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.002+0000 7f466ac10b80 1 -- 172.21.15.46:0/456889933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55b99d3c3b60 con 0x55b99d642a60 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.002+0000 7f4637fff700 1 -- 172.21.15.46:0/456889933 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b99d636ad0 con 0x55b99d642a60 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.002+0000 7f4637fff700 1 -- 172.21.15.46:0/456889933 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f464c09abb0 con 0x55b99d642a60 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.002+0000 7f466ac10b80 10 client.20157 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.002+0000 7f466ac10b80 10 client.20157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.002+0000 7f466ac10b80 20 client.20157 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.002+0000 7f466ac10b80 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b99d65d8d0 0x55b99d65fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.002+0000 7f466ac10b80 4 client.20157 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.002+0000 7f466ac10b80 1 -- 172.21.15.46:0/456889933 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b99d58e700 con 0x55b99d65d8d0 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.002+0000 7f4657fff700 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b99d65d8d0 0x55b99d65fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.003+0000 7f4637fff700 10 client.20157 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.003+0000 7f4657fff700 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b99d65d8d0 0x55b99d65fcf0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11864d2dca512ced server_cookie=cf9ccca8c010c7a7 in_seq=0 out_seq=0 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.003+0000 7f4637fff700 1 -- 172.21.15.46:0/456889933 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b99d58e700 con 0x55b99d65d8d0 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.003+0000 7f4637fff700 10 client.20157 handle_command_reply: tid=0 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.003+0000 7f466ac10b80 1 client.20157 shutdown 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.004+0000 7f466ac10b80 1 -- 172.21.15.46:0/456889933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 msgr2=0x55b99d64dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.004+0000 7f466ac10b80 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 0x55b99d64dcd0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f464c0ca760 tx=0x7f464c0a20d0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.004+0000 7f466ac10b80 1 -- 172.21.15.46:0/456889933 shutdown_connections 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.004+0000 7f466ac10b80 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b99d65d8d0 0x55b99d65fcf0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.004+0000 7f466ac10b80 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d642a60 0x55b99d64dcd0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.004+0000 7f466ac10b80 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d64e220 0x55b99d64b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.004+0000 7f466ac10b80 1 --2- 172.21.15.46:0/456889933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d643380 0x55b99d64b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.004+0000 7f4637fff700 0 client.20157 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.004+0000 7f466ac10b80 1 -- 172.21.15.46:0/456889933 >> 172.21.15.46:0/456889933 conn(0x55b99d5b9220 msgr2=0x55b99d63ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.004+0000 7f466ac10b80 1 -- 172.21.15.46:0/456889933 shutdown_connections 2024-03-18T20:49:11.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.004+0000 7f466ac10b80 1 -- 172.21.15.46:0/456889933 wait complete. 2024-03-18T20:49:11.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.004+0000 7f466ac10b80 20 client.20157 trim_cache size 0 max 16384 2024-03-18T20:49:11.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.017+0000 7f466ac10b80 1 Processor -- start 2024-03-18T20:49:11.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f466ac10b80 1 -- start start 2024-03-18T20:49:11.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f466ac10b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d642a60 0x55b99d654d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f466ac10b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d643380 0x55b99d646380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f466ac10b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d64e220 0x55b99d6468c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f466ac10b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d64c5f0 con 0x55b99d643380 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f466ac10b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d64bfe0 con 0x55b99d642a60 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f466ac10b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d643d80 con 0x55b99d64e220 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f4657fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d643380 0x55b99d646380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f4657fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d643380 0x55b99d646380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59656/0 (socket says 172.21.15.46:59656) 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f4657fff700 1 -- 172.21.15.46:0/3600192840 learned_addr learned my addr 172.21.15.46:0/3600192840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f465c817700 1 --2- 172.21.15.46:0/3600192840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d642a60 0x55b99d654d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f465d018700 1 --2- 172.21.15.46:0/3600192840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d64e220 0x55b99d6468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f4657fff700 1 -- 172.21.15.46:0/3600192840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d64e220 msgr2=0x55b99d6468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f4657fff700 1 --2- 172.21.15.46:0/3600192840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d64e220 0x55b99d6468c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f4657fff700 1 -- 172.21.15.46:0/3600192840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d642a60 msgr2=0x55b99d654d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f4657fff700 1 --2- 172.21.15.46:0/3600192840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d642a60 0x55b99d654d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.018+0000 7f4657fff700 1 -- 172.21.15.46:0/3600192840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b99d55bea0 con 0x55b99d643380 2024-03-18T20:49:11.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f4657fff700 1 --2- 172.21.15.46:0/3600192840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d643380 0x55b99d646380 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4644005f50 tx=0x7f4644022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77933672656c177f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:11.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f46437fe700 1 -- 172.21.15.46:0/3600192840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644033030 con 0x55b99d643380 2024-03-18T20:49:11.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f46437fe700 1 -- 172.21.15.46:0/3600192840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f464402c050 con 0x55b99d643380 2024-03-18T20:49:11.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f46437fe700 1 -- 172.21.15.46:0/3600192840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644004760 con 0x55b99d643380 2024-03-18T20:49:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f466ac10b80 1 -- 172.21.15.46:0/3600192840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d643380 msgr2=0x55b99d646380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f466ac10b80 1 --2- 172.21.15.46:0/3600192840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d643380 0x55b99d646380 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f4644005f50 tx=0x7f4644022000 comp rx=0 tx=0).stop 2024-03-18T20:49:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f466ac10b80 1 -- 172.21.15.46:0/3600192840 shutdown_connections 2024-03-18T20:49:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f466ac10b80 1 --2- 172.21.15.46:0/3600192840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d64e220 0x55b99d6468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f466ac10b80 1 --2- 172.21.15.46:0/3600192840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d642a60 0x55b99d654d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f466ac10b80 1 --2- 172.21.15.46:0/3600192840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d643380 0x55b99d646380 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f466ac10b80 1 -- 172.21.15.46:0/3600192840 >> 172.21.15.46:0/3600192840 conn(0x55b99d5b9220 msgr2=0x55b99d659480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:11.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f466ac10b80 1 -- 172.21.15.46:0/3600192840 shutdown_connections 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.019+0000 7f466ac10b80 1 -- 172.21.15.46:0/3600192840 wait complete. 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.020+0000 7f466ac10b80 1 -- start start 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.020+0000 7f466ac10b80 1 Processor -- start 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.020+0000 7f466ac10b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d642a60 0x55b99d653f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.020+0000 7f466ac10b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d643380 0x55b99d654490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.020+0000 7f466ac10b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d64e220 0x55b99d6b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.020+0000 7f466ac10b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d654dd0 con 0x55b99d64e220 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.020+0000 7f466ac10b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d6b0c00 con 0x55b99d643380 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.020+0000 7f466ac10b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d6b0d40 con 0x55b99d642a60 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.020+0000 7f466ac10b80 10 client.0 _ll_register_callbacks cb 0x7f464c0995b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.020+0000 7f465d018700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d64e220 0x55b99d6b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f465d018700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d64e220 0x55b99d6b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59658/0 (socket says 172.21.15.46:59658) 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f465d018700 1 -- 172.21.15.46:0/2692895535 learned_addr learned my addr 172.21.15.46:0/2692895535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:11.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f4657fff700 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d643380 0x55b99d654490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f465c817700 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d642a60 0x55b99d653f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f465d018700 1 -- 172.21.15.46:0/2692895535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d642a60 msgr2=0x55b99d653f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f465d018700 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d642a60 0x55b99d653f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f465d018700 1 -- 172.21.15.46:0/2692895535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d643380 msgr2=0x55b99d654490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f465d018700 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d643380 0x55b99d654490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f465d018700 1 -- 172.21.15.46:0/2692895535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465810e550 con 0x55b99d64e220 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f4657fff700 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d643380 0x55b99d654490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f465d018700 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d64e220 0x55b99d6b06c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4650007ef0 tx=0x7f4650007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7923fe0eef6ae1d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f4637fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f4637fff700 1 -- 172.21.15.46:0/2692895535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465000e040 con 0x55b99d64e220 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f466ac10b80 1 -- 172.21.15.46:0/2692895535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f464c097f40 con 0x55b99d64e220 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f4637fff700 1 -- 172.21.15.46:0/2692895535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f4650004db0 con 0x55b99d64e220 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f4637fff700 1 -- 172.21.15.46:0/2692895535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46500159b0 con 0x55b99d64e220 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f466ac10b80 10 client.20163 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:11.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.021+0000 7f466ac10b80 1 -- 172.21.15.46:0/2692895535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55b99d41bcd0 con 0x55b99d64e220 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.022+0000 7f4637fff700 1 -- 172.21.15.46:0/2692895535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f464c097f40 con 0x55b99d64e220 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.022+0000 7f4637fff700 1 -- 172.21.15.46:0/2692895535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f46500057d0 con 0x55b99d64e220 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.022+0000 7f466ac10b80 10 client.20163 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.022+0000 7f466ac10b80 10 client.20163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.022+0000 7f466ac10b80 20 client.20163 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.022+0000 7f466ac10b80 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b99d6b2620 0x55b99d6b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.022+0000 7f466ac10b80 4 client.20163 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.022+0000 7f466ac10b80 1 -- 172.21.15.46:0/2692895535 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b99d3e4cf0 con 0x55b99d6b2620 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.022+0000 7f465c817700 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b99d6b2620 0x55b99d6b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f465c817700 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b99d6b2620 0x55b99d6b4a40 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6553d521b04d705d server_cookie=1254151b531b80e7 in_seq=0 out_seq=0 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f4637fff700 10 client.20163 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f4637fff700 1 -- 172.21.15.46:0/2692895535 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b99d3e4cf0 con 0x55b99d6b2620 2024-03-18T20:49:11.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f4637fff700 10 client.20163 handle_command_reply: tid=0 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f466ac10b80 1 client.20163 shutdown 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f466ac10b80 1 -- 172.21.15.46:0/2692895535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d64e220 msgr2=0x55b99d6b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f466ac10b80 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d64e220 0x55b99d6b06c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f4650007ef0 tx=0x7f4650007f20 comp rx=0 tx=0).stop 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f466ac10b80 1 -- 172.21.15.46:0/2692895535 shutdown_connections 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f466ac10b80 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b99d6b2620 0x55b99d6b4a40 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f466ac10b80 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b99d64e220 0x55b99d6b06c0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f466ac10b80 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b99d642a60 0x55b99d653f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f466ac10b80 1 --2- 172.21.15.46:0/2692895535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b99d643380 0x55b99d654490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f4637fff700 0 client.20163 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f466ac10b80 1 -- 172.21.15.46:0/2692895535 >> 172.21.15.46:0/2692895535 conn(0x55b99d5b9220 msgr2=0x55b99d646380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f466ac10b80 1 -- 172.21.15.46:0/2692895535 shutdown_connections 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f466ac10b80 1 -- 172.21.15.46:0/2692895535 wait complete. 2024-03-18T20:49:11.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.023+0000 7f466ac10b80 20 client.20163 trim_cache size 0 max 16384 2024-03-18T20:49:11.032 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-18T20:49:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.024+0000 7f465ea7b700 1 -- 172.21.15.46:0/2702434033 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f46480a1310 msgr2=0x7f46480a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:11.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.024+0000 7f465ea7b700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f46480a1310 0x7f46480a37d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4644006010 tx=0x7f464400b260 comp rx=0 tx=0).stop 2024-03-18T20:49:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.024+0000 7f465ea7b700 1 -- 172.21.15.46:0/2702434033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4658067e30 msgr2=0x7f4658197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.024+0000 7f465ea7b700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4658067e30 0x7f4658197210 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f464c00d980 tx=0x7f464c00dc90 comp rx=0 tx=0).stop 2024-03-18T20:49:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.024+0000 7f465ea7b700 1 -- 172.21.15.46:0/2702434033 shutdown_connections 2024-03-18T20:49:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.024+0000 7f465ea7b700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f46480a1310 0x7f46480a37d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.024+0000 7f465ea7b700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4658067e30 0x7f4658197210 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.024+0000 7f465ea7b700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4658105ce0 0x7f465819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.024+0000 7f465ea7b700 1 --2- 172.21.15.46:0/2702434033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46580687f0 0x7f4658197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.024+0000 7f465ea7b700 1 -- 172.21.15.46:0/2702434033 >> 172.21.15.46:0/2702434033 conn(0x7f4658076d40 msgr2=0x7f46580fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.024+0000 7f465ea7b700 1 -- 172.21.15.46:0/2702434033 shutdown_connections 2024-03-18T20:49:11.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T20:49:11.024+0000 7f465ea7b700 1 -- 172.21.15.46:0/2702434033 wait complete. 2024-03-18T20:49:11.040 ERROR:tasks.fwd_scrub:rank 4 damaged: [{'damage_type': 'backtrace', 'id': 900619346, 'ino': 5497558140177, 'path': '/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.Y03'}, {'damage_type': 'backtrace', 'id': 958634982, 'ino': 5497558140180, 'path': '/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.FSL'}, {'damage_type': 'backtrace', 'id': 1526672529, 'ino': 5497558140178, 'path': '/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.VAL'}, {'damage_type': 'backtrace', 'id': 1750481800, 'ino': 5497558140173, 'path': '/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.MB'}, {'damage_type': 'backtrace', 'id': 1780457581, 'ino': 5497558140171, 'path': '/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.XG1'}, {'damage_type': 'backtrace', 'id': 2057819981, 'ino': 5497558140174, 'path': '/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.DB'}, {'damage_type': 'backtrace', 'id': 2549199034, 'ino': 5497558140176, 'path': '/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/ENROLL.X03'}, {'damage_type': 'backtrace', 'id': 2617804119, 'ino': 5497558140179, 'path': '/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.TV'}, {'damage_type': 'backtrace', 'id': 2792763337, 'ino': 5497558140172, 'path': '/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.YG1'}, {'damage_type': 'backtrace', 'id': 4001455076, 'ino': 5497558140181, 'path': '/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/COURSES.FSL'}] 2024-03-18T20:49:11.040 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 3799837018, 'ino': 1099511630769, 'path': '/volumes/qa/sv_1/67a95736-ebd5-44c9-a27b-9f804235a410/client.0/tmp/clients/client0/~dmtmp/PARADOX/STUDENTS.DB'}] 2024-03-18T20:49:11.040 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_49bdc51142759622a281d3e5327eebd07cbb8bb3/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_49bdc51142759622a281d3e5327eebd07cbb8bb3/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_49bdc51142759622a281d3e5327eebd07cbb8bb3/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/github.com_ceph_ceph-c_49bdc51142759622a281d3e5327eebd07cbb8bb3/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-03-18T20:49:11.730 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 606 sec 2024-03-18T20:49:12.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:12 smithi073 ceph-mon[47384]: pgmap v410: 97 pgs: 97 active+clean; 219 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 771 KiB/s wr, 65 op/s 2024-03-18T20:49:12.726 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 607 sec 2024-03-18T20:49:12.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:12 smithi155 ceph-mon[48344]: pgmap v410: 97 pgs: 97 active+clean; 219 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 771 KiB/s wr, 65 op/s 2024-03-18T20:49:12.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:12 smithi046 ceph-mon[43495]: pgmap v410: 97 pgs: 97 active+clean; 219 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 771 KiB/s wr, 65 op/s 2024-03-18T20:49:13.727 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 608 sec 2024-03-18T20:49:14.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:14 smithi073 ceph-mon[47384]: pgmap v411: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 719 KiB/s wr, 82 op/s 2024-03-18T20:49:14.728 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 609 sec 2024-03-18T20:49:14.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:14 smithi155 ceph-mon[48344]: pgmap v411: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 719 KiB/s wr, 82 op/s 2024-03-18T20:49:14.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:14 smithi046 ceph-mon[43495]: pgmap v411: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 719 KiB/s wr, 82 op/s 2024-03-18T20:49:15.727 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 610 sec 2024-03-18T20:49:16.290 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:16 smithi155 ceph-mon[48344]: pgmap v412: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 715 KiB/s wr, 81 op/s 2024-03-18T20:49:16.295 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:16 smithi046 ceph-mon[43495]: pgmap v412: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 715 KiB/s wr, 81 op/s 2024-03-18T20:49:16.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:16 smithi073 ceph-mon[47384]: pgmap v412: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 715 KiB/s wr, 81 op/s 2024-03-18T20:49:16.727 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 611 sec 2024-03-18T20:49:17.727 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 612 sec 2024-03-18T20:49:18.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:18 smithi046 ceph-mon[43495]: pgmap v413: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 715 KiB/s wr, 81 op/s 2024-03-18T20:49:18.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:18 smithi073 ceph-mon[47384]: pgmap v413: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 715 KiB/s wr, 81 op/s 2024-03-18T20:49:18.727 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 613 sec 2024-03-18T20:49:18.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:18 smithi155 ceph-mon[48344]: pgmap v413: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 715 KiB/s wr, 81 op/s 2024-03-18T20:49:19.727 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 614 sec 2024-03-18T20:49:20.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:20 smithi073 ceph-mon[47384]: pgmap v414: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 459 KiB/s wr, 53 op/s 2024-03-18T20:49:20.727 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 615 sec 2024-03-18T20:49:20.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:20 smithi155 ceph-mon[48344]: pgmap v414: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 459 KiB/s wr, 53 op/s 2024-03-18T20:49:20.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:20 smithi046 ceph-mon[43495]: pgmap v414: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 459 KiB/s wr, 53 op/s 2024-03-18T20:49:21.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:21 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:49:21.727 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 616 sec 2024-03-18T20:49:21.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:21 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:49:21.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:21 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:49:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.966+0000 7f997d846700 1 -- 172.21.15.46:0/1127467434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9978100ee0 msgr2=0x7f99781012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.966+0000 7f997d846700 1 --2- 172.21.15.46:0/1127467434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9978100ee0 0x7f99781012c0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f9960009990 tx=0x7f9960009ca0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.966+0000 7f997d846700 1 -- 172.21.15.46:0/1127467434 shutdown_connections 2024-03-18T20:49:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.966+0000 7f997d846700 1 --2- 172.21.15.46:0/1127467434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9978100ee0 0x7f99781012c0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.966+0000 7f997d846700 1 --2- 172.21.15.46:0/1127467434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9978106000 0x7f997810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.966+0000 7f997d846700 1 --2- 172.21.15.46:0/1127467434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9978101890 0x7f99781058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.966+0000 7f997d846700 1 -- 172.21.15.46:0/1127467434 >> 172.21.15.46:0/1127467434 conn(0x7f99780fc780 msgr2=0x7f99780feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.966+0000 7f997d846700 1 -- 172.21.15.46:0/1127467434 shutdown_connections 2024-03-18T20:49:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.966+0000 7f997d846700 1 -- 172.21.15.46:0/1127467434 wait complete. 2024-03-18T20:49:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.967+0000 7f997d846700 1 Processor -- start 2024-03-18T20:49:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.967+0000 7f997d846700 1 -- start start 2024-03-18T20:49:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.967+0000 7f997d846700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9978100ee0 0x7f99780730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.967+0000 7f997d846700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9978106000 0x7f99780735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.967+0000 7f997d846700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9978073b30 0x7f99781a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.967+0000 7f997d846700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781a59c0 con 0x7f9978073b30 2024-03-18T20:49:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.967+0000 7f997d846700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781a5b30 con 0x7f9978100ee0 2024-03-18T20:49:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.967+0000 7f997d846700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781a5cf0 con 0x7f9978106000 2024-03-18T20:49:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.967+0000 7f99777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9978073b30 0x7f99781a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.967+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9978100ee0 0x7f99780730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f99777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9978073b30 0x7f99781a53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38908/0 (socket says 172.21.15.46:38908) 2024-03-18T20:49:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f99777fe700 1 -- 172.21.15.46:0/1624989846 learned_addr learned my addr 172.21.15.46:0/1624989846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f99767fc700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9978106000 0x7f99780735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f9976ffd700 1 -- 172.21.15.46:0/1624989846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9978106000 msgr2=0x7f99780735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f9976ffd700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9978106000 0x7f99780735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f9976ffd700 1 -- 172.21.15.46:0/1624989846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9978073b30 msgr2=0x7f99781a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f9976ffd700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9978073b30 0x7f99781a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f9976ffd700 1 -- 172.21.15.46:0/1624989846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968009520 con 0x7f9978100ee0 2024-03-18T20:49:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f99777fe700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9978073b30 0x7f99781a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:49:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f9976ffd700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9978100ee0 0x7f99780730b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f99600059d0 tx=0x7f9960004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f997c844700 1 -- 172.21.15.46:0/1624989846 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996001d070 con 0x7f9978100ee0 2024-03-18T20:49:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f997d846700 1 -- 172.21.15.46:0/1624989846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99600095f0 con 0x7f9978100ee0 2024-03-18T20:49:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f997d846700 1 -- 172.21.15.46:0/1624989846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99781a6200 con 0x7f9978100ee0 2024-03-18T20:49:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f997c844700 1 -- 172.21.15.46:0/1624989846 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9960022470 con 0x7f9978100ee0 2024-03-18T20:49:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.968+0000 7f997c844700 1 -- 172.21.15.46:0/1624989846 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960017650 con 0x7f9978100ee0 2024-03-18T20:49:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.971+0000 7f997c844700 1 -- 172.21.15.46:0/1624989846 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9960022850 con 0x7f9978100ee0 2024-03-18T20:49:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.971+0000 7f997c844700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f99640a1290 0x7f99640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.971+0000 7f997c844700 1 -- 172.21.15.46:0/1624989846 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f99600cde90 con 0x7f9978100ee0 2024-03-18T20:49:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.971+0000 7f99767fc700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f99640a1290 0x7f99640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.972+0000 7f99767fc700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f99640a1290 0x7f99640a3750 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9978074790 tx=0x7f9968009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.972+0000 7f99899dbb80 1 Processor -- start 2024-03-18T20:49:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.972+0000 7f99899dbb80 1 -- start start 2024-03-18T20:49:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f99899dbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad20950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f99899dbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 0x556a3ad2b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f99899dbb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad2bdb0 0x556a3ad2e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f99899dbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3ac7a930 con 0x556a3ad20570 2024-03-18T20:49:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f99899dbb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3ac782f0 con 0x556a3ad2bdb0 2024-03-18T20:49:21.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f99899dbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3ac73be0 con 0x556a3ad20e90 2024-03-18T20:49:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad20950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad20950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38918/0 (socket says 172.21.15.46:38918) 2024-03-18T20:49:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f9976ffd700 1 -- 172.21.15.46:0/243398083 learned_addr learned my addr 172.21.15.46:0/243398083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f99767fc700 1 --2- 172.21.15.46:0/243398083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 0x556a3ad2b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f99777fe700 1 --2- 172.21.15.46:0/243398083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad2bdb0 0x556a3ad2e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f9976ffd700 1 -- 172.21.15.46:0/243398083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 msgr2=0x556a3ad2b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f9976ffd700 1 --2- 172.21.15.46:0/243398083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 0x556a3ad2b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f9976ffd700 1 -- 172.21.15.46:0/243398083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad2bdb0 msgr2=0x556a3ad2e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f9976ffd700 1 --2- 172.21.15.46:0/243398083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad2bdb0 0x556a3ad2e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f9976ffd700 1 -- 172.21.15.46:0/243398083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a3aaf5350 con 0x556a3ad20570 2024-03-18T20:49:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f99777fe700 1 --2- 172.21.15.46:0/243398083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad2bdb0 0x556a3ad2e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:49:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f99767fc700 1 --2- 172.21.15.46:0/243398083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 0x556a3ad2b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:49:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.973+0000 7f9976ffd700 1 --2- 172.21.15.46:0/243398083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad20950 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f9960097510 tx=0x7f996009ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998e1b011ae7fbfd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.974+0000 7f996dffb700 1 -- 172.21.15.46:0/243398083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996009b010 con 0x556a3ad20570 2024-03-18T20:49:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.974+0000 7f996dffb700 1 -- 172.21.15.46:0/243398083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f996009ee40 con 0x556a3ad20570 2024-03-18T20:49:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.974+0000 7f996dffb700 1 -- 172.21.15.46:0/243398083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996009b4d0 con 0x556a3ad20570 2024-03-18T20:49:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.974+0000 7f99899dbb80 1 -- 172.21.15.46:0/243398083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 msgr2=0x556a3ad20950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.974+0000 7f99899dbb80 1 --2- 172.21.15.46:0/243398083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad20950 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f9960097510 tx=0x7f996009ab50 comp rx=0 tx=0).stop 2024-03-18T20:49:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.974+0000 7f99899dbb80 1 -- 172.21.15.46:0/243398083 shutdown_connections 2024-03-18T20:49:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.974+0000 7f99899dbb80 1 --2- 172.21.15.46:0/243398083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad20950 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.974+0000 7f99899dbb80 1 --2- 172.21.15.46:0/243398083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad2bdb0 0x556a3ad2e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.974+0000 7f99899dbb80 1 --2- 172.21.15.46:0/243398083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 0x556a3ad2b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.974+0000 7f99899dbb80 1 -- 172.21.15.46:0/243398083 >> 172.21.15.46:0/243398083 conn(0x556a3ac971c0 msgr2=0x556a3ad1ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.974+0000 7f99899dbb80 1 -- 172.21.15.46:0/243398083 shutdown_connections 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.975+0000 7f99899dbb80 1 -- 172.21.15.46:0/243398083 wait complete. 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.975+0000 7f99899dbb80 1 -- start start 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.975+0000 7f99899dbb80 1 Processor -- start 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f99899dbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad2b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f99899dbb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20e90 0x556a3ad28a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f99899dbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad2bdb0 0x556a3ad290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f99899dbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3ac0fd20 con 0x556a3ad20570 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f99899dbb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3ac0f9c0 con 0x556a3ad20e90 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f99899dbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3aaf7640 con 0x556a3ad2bdb0 2024-03-18T20:49:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f99899dbb80 10 client.0 _ll_register_callbacks cb 0x556a3aa7b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad2b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad2b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38926/0 (socket says 172.21.15.46:38926) 2024-03-18T20:49:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f9976ffd700 1 -- 172.21.15.46:0/844468977 learned_addr learned my addr 172.21.15.46:0/844468977 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f99777fe700 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad2bdb0 0x556a3ad290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f99767fc700 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20e90 0x556a3ad28a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f9976ffd700 1 -- 172.21.15.46:0/844468977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad2bdb0 msgr2=0x556a3ad290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f9976ffd700 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad2bdb0 0x556a3ad290a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f9976ffd700 1 -- 172.21.15.46:0/844468977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20e90 msgr2=0x556a3ad28a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f9976ffd700 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20e90 0x556a3ad28a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f9976ffd700 1 -- 172.21.15.46:0/844468977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9970009110 con 0x556a3ad20570 2024-03-18T20:49:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f99777fe700 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad2bdb0 0x556a3ad290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:49:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f99767fc700 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20e90 0x556a3ad28a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f9976ffd700 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad2b7a0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f9960000c00 tx=0x7f99600cb560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba93a5417c57e8c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f9956ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.976+0000 7f9956ffd700 1 -- 172.21.15.46:0/844468977 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996009b470 con 0x556a3ad20570 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f9956ffd700 1 -- 172.21.15.46:0/844468977 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f99600a3400 con 0x556a3ad20570 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f99899dbb80 1 -- 172.21.15.46:0/844468977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a3ad146c0 con 0x556a3ad20570 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f9956ffd700 1 -- 172.21.15.46:0/844468977 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99600a10c0 con 0x556a3ad20570 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f99899dbb80 10 client.20175 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f99899dbb80 1 -- 172.21.15.46:0/844468977 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7f99781a6200 con 0x556a3ad20570 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f9956ffd700 1 -- 172.21.15.46:0/844468977 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a3ad146c0 con 0x556a3ad20570 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f9956ffd700 1 -- 172.21.15.46:0/844468977 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f996000fad0 con 0x556a3ad20570 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f99899dbb80 10 client.20175 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f99899dbb80 10 client.20175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f99899dbb80 20 client.20175 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f99899dbb80 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a3ad3b2a0 0x556a3ad3d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f99899dbb80 4 client.20175 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f99899dbb80 1 -- 172.21.15.46:0/844468977 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a3ac6c6a0 con 0x556a3ad3b2a0 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.977+0000 7f99767fc700 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a3ad3b2a0 0x556a3ad3d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.978+0000 7f99767fc700 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a3ad3b2a0 0x556a3ad3d6c0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a542ff3d0e85a6 server_cookie=c1e494fb84531a93 in_seq=0 out_seq=0 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.978+0000 7f9956ffd700 10 client.20175 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.978+0000 7f9956ffd700 1 -- 172.21.15.46:0/844468977 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556a3ac6c6a0 con 0x556a3ad3b2a0 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.978+0000 7f9956ffd700 10 client.20175 handle_command_reply: tid=0 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.978+0000 7f99899dbb80 1 client.20175 shutdown 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.978+0000 7f99899dbb80 1 -- 172.21.15.46:0/844468977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 msgr2=0x556a3ad2b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.978+0000 7f99899dbb80 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad2b7a0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f9960000c00 tx=0x7f99600cb560 comp rx=0 tx=0).stop 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.979+0000 7f99899dbb80 1 -- 172.21.15.46:0/844468977 shutdown_connections 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.979+0000 7f99899dbb80 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a3ad3b2a0 0x556a3ad3d6c0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.979+0000 7f99899dbb80 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad2b7a0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.979+0000 7f99899dbb80 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad2bdb0 0x556a3ad290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.979+0000 7f9956ffd700 0 client.20175 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.979+0000 7f99899dbb80 1 --2- 172.21.15.46:0/844468977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20e90 0x556a3ad28a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.979+0000 7f99899dbb80 1 -- 172.21.15.46:0/844468977 >> 172.21.15.46:0/844468977 conn(0x556a3ac971c0 msgr2=0x556a3ad1c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.979+0000 7f99899dbb80 1 -- 172.21.15.46:0/844468977 shutdown_connections 2024-03-18T20:49:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.979+0000 7f99899dbb80 1 -- 172.21.15.46:0/844468977 wait complete. 2024-03-18T20:49:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.979+0000 7f99899dbb80 20 client.20175 trim_cache size 0 max 16384 2024-03-18T20:49:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99899dbb80 1 Processor -- start 2024-03-18T20:49:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99899dbb80 1 -- start start 2024-03-18T20:49:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99899dbb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20570 0x556a3ad1cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99899dbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 0x556a3ad2b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99899dbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad2bdb0 0x556a3ad28320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99899dbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3ad2a0c0 con 0x556a3ad2bdb0 2024-03-18T20:49:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99899dbb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3ad23680 con 0x556a3ad20570 2024-03-18T20:49:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99899dbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3ad19220 con 0x556a3ad20e90 2024-03-18T20:49:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad2bdb0 0x556a3ad28320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99767fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 0x556a3ad2b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99767fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 0x556a3ad2b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:50676/0 (socket says 172.21.15.46:50676) 2024-03-18T20:49:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad2bdb0 0x556a3ad28320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38928/0 (socket says 172.21.15.46:38928) 2024-03-18T20:49:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.994+0000 7f99767fc700 1 -- 172.21.15.46:0/1919296083 learned_addr learned my addr 172.21.15.46:0/1919296083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.995+0000 7f9976ffd700 1 --2- 172.21.15.46:0/1919296083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20570 0x556a3ad1cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.995+0000 7f99767fc700 1 -- 172.21.15.46:0/1919296083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20570 msgr2=0x556a3ad1cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.995+0000 7f99767fc700 1 --2- 172.21.15.46:0/1919296083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20570 0x556a3ad1cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.995+0000 7f99767fc700 1 -- 172.21.15.46:0/1919296083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad2bdb0 msgr2=0x556a3ad28320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.995+0000 7f99767fc700 1 --2- 172.21.15.46:0/1919296083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad2bdb0 0x556a3ad28320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.995+0000 7f99767fc700 1 -- 172.21.15.46:0/1919296083 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a3aaa1b00 con 0x556a3ad20e90 2024-03-18T20:49:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.995+0000 7f99777fe700 1 --2- 172.21.15.46:0/1919296083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad2bdb0 0x556a3ad28320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.995+0000 7f99767fc700 1 --2- 172.21.15.46:0/1919296083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 0x556a3ad2b2d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9978074b10 tx=0x7f9968014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a0906e9455a7ac server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.995+0000 7f996e7fc700 1 -- 172.21.15.46:0/1919296083 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968023040 con 0x556a3ad20e90 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.995+0000 7f996e7fc700 1 -- 172.21.15.46:0/1919296083 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99680283b0 con 0x556a3ad20e90 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.995+0000 7f996e7fc700 1 -- 172.21.15.46:0/1919296083 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968028ea0 con 0x556a3ad20e90 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.996+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919296083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 msgr2=0x556a3ad2b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.996+0000 7f99899dbb80 1 --2- 172.21.15.46:0/1919296083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 0x556a3ad2b2d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9978074b10 tx=0x7f9968014000 comp rx=0 tx=0).stop 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.996+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919296083 shutdown_connections 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.996+0000 7f99899dbb80 1 --2- 172.21.15.46:0/1919296083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad2bdb0 0x556a3ad28320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.996+0000 7f99899dbb80 1 --2- 172.21.15.46:0/1919296083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20570 0x556a3ad1cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.996+0000 7f99899dbb80 1 --2- 172.21.15.46:0/1919296083 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad20e90 0x556a3ad2b2d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.996+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919296083 >> 172.21.15.46:0/1919296083 conn(0x556a3ac971c0 msgr2=0x556a3ad1c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.996+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919296083 shutdown_connections 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.996+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919296083 wait complete. 2024-03-18T20:49:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.996+0000 7f99899dbb80 1 -- start start 2024-03-18T20:49:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.996+0000 7f99899dbb80 1 Processor -- start 2024-03-18T20:49:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f99899dbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad2e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f99899dbb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20e90 0x556a3ad2e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f99899dbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad2bdb0 0x556a3ad8e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f99899dbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3ad2ed60 con 0x556a3ad20570 2024-03-18T20:49:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f99899dbb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3ad8e5f0 con 0x556a3ad20e90 2024-03-18T20:49:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f99899dbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a3ad8e7b0 con 0x556a3ad2bdb0 2024-03-18T20:49:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f99899dbb80 10 client.0 _ll_register_callbacks cb 0x7f9970009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad2e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f9976ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad2e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38936/0 (socket says 172.21.15.46:38936) 2024-03-18T20:49:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f9976ffd700 1 -- 172.21.15.46:0/1919083700 learned_addr learned my addr 172.21.15.46:0/1919083700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f99777fe700 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad2bdb0 0x556a3ad8e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f99767fc700 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20e90 0x556a3ad2e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f9976ffd700 1 -- 172.21.15.46:0/1919083700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad2bdb0 msgr2=0x556a3ad8e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f9976ffd700 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad2bdb0 0x556a3ad8e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f9976ffd700 1 -- 172.21.15.46:0/1919083700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20e90 msgr2=0x556a3ad2e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f9976ffd700 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20e90 0x556a3ad2e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f9976ffd700 1 -- 172.21.15.46:0/1919083700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9970007500 con 0x556a3ad20570 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f9976ffd700 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad2e080 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f996000fe60 tx=0x7f99600092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffc29ec485a9383 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f9956ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f9956ffd700 1 -- 172.21.15.46:0/1919083700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996009ab50 con 0x556a3ad20570 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f9956ffd700 1 -- 172.21.15.46:0/1919083700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f99600a3400 con 0x556a3ad20570 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.997+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919083700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f996009ee40 con 0x556a3ad20570 2024-03-18T20:49:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.998+0000 7f9956ffd700 1 -- 172.21.15.46:0/1919083700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f996009b010 con 0x556a3ad20570 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.998+0000 7f99899dbb80 10 client.20181 fetch_fsmap learned FSMap version 60 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.998+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919083700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x7f9968004230 con 0x556a3ad20570 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.998+0000 7f9956ffd700 1 -- 172.21.15.46:0/1919083700 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f996009ee40 con 0x556a3ad20570 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.998+0000 7f9956ffd700 1 -- 172.21.15.46:0/1919083700 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 60) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f996009a110 con 0x556a3ad20570 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.998+0000 7f99899dbb80 10 client.20181 fetch_fsmap finished waiting for FSMap version 60 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.998+0000 7f99899dbb80 10 client.20181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.998+0000 7f99899dbb80 20 client.20181 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.998+0000 7f99899dbb80 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a3ad8fc20 0x556a3ad92040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.998+0000 7f99899dbb80 4 client.20181 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.998+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919083700 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556a3aac2c90 con 0x556a3ad8fc20 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.998+0000 7f99767fc700 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a3ad8fc20 0x556a3ad92040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.999+0000 7f9956ffd700 10 client.20181 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:21.999+0000 7f99767fc700 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a3ad8fc20 0x556a3ad92040 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a5f7d78dcf52ea server_cookie=34bbcdcd589a5f4b in_seq=0 out_seq=0 2024-03-18T20:49:22.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:22 smithi073 ceph-mon[47384]: pgmap v415: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 409 KiB/s wr, 48 op/s 2024-03-18T20:49:22.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:22 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:49:22.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:22 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:49:22.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:22 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:49:22.727 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 617 sec 2024-03-18T20:49:22.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:22 smithi155 ceph-mon[48344]: pgmap v415: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 409 KiB/s wr, 48 op/s 2024-03-18T20:49:22.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:22 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:49:22.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:22 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:49:22.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:22 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:49:22.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:22 smithi046 ceph-mon[43495]: pgmap v415: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 409 KiB/s wr, 48 op/s 2024-03-18T20:49:22.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:22 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:49:22.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:22 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:49:22.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:22 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:49:23.728 INFO:tasks.workunit.client.0.smithi046.stdout: 1 cleanup 618 sec 2024-03-18T20:49:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.189+0000 7f9956ffd700 1 -- 172.21.15.46:0/1919083700 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556a3aac2c90 con 0x556a3ad8fc20 2024-03-18T20:49:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.189+0000 7f9956ffd700 10 client.20181 handle_command_reply: tid=0 2024-03-18T20:49:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.189+0000 7f99899dbb80 1 client.20181 shutdown 2024-03-18T20:49:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.189+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919083700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 msgr2=0x556a3ad2e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.189+0000 7f99899dbb80 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad2e080 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f996000fe60 tx=0x7f99600092f0 comp rx=0 tx=0).stop 2024-03-18T20:49:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.189+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919083700 shutdown_connections 2024-03-18T20:49:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.189+0000 7f99899dbb80 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556a3ad8fc20 0x556a3ad92040 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.189+0000 7f99899dbb80 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556a3ad20570 0x556a3ad2e080 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.190+0000 7f99899dbb80 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556a3ad2bdb0 0x556a3ad8e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.190+0000 7f99899dbb80 1 --2- 172.21.15.46:0/1919083700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a3ad20e90 0x556a3ad2e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.190+0000 7f9956ffd700 0 client.20181 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.190+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919083700 >> 172.21.15.46:0/1919083700 conn(0x556a3ac971c0 msgr2=0x556a3ad27360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.190+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919083700 shutdown_connections 2024-03-18T20:49:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.190+0000 7f99899dbb80 1 -- 172.21.15.46:0/1919083700 wait complete. 2024-03-18T20:49:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.190+0000 7f99899dbb80 20 client.20181 trim_cache size 0 max 16384 2024-03-18T20:49:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:49:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:49:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:49:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:49:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.191+0000 7f997d846700 1 -- 172.21.15.46:0/1624989846 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f99640a1290 msgr2=0x7f99640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.191+0000 7f997d846700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f99640a1290 0x7f99640a3750 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9978074790 tx=0x7f9968009260 comp rx=0 tx=0).stop 2024-03-18T20:49:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.191+0000 7f997d846700 1 -- 172.21.15.46:0/1624989846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9978100ee0 msgr2=0x7f99780730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.191+0000 7f997d846700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9978100ee0 0x7f99780730b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f99600059d0 tx=0x7f9960004730 comp rx=0 tx=0).stop 2024-03-18T20:49:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.191+0000 7f997d846700 1 -- 172.21.15.46:0/1624989846 shutdown_connections 2024-03-18T20:49:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.191+0000 7f997d846700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f99640a1290 0x7f99640a3750 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.191+0000 7f997d846700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9978073b30 0x7f99781a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.191+0000 7f997d846700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9978100ee0 0x7f99780730b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.191+0000 7f997d846700 1 --2- 172.21.15.46:0/1624989846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9978106000 0x7f99780735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.191+0000 7f997d846700 1 -- 172.21.15.46:0/1624989846 >> 172.21.15.46:0/1624989846 conn(0x7f99780fc780 msgr2=0x7f99780fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.191+0000 7f997d846700 1 -- 172.21.15.46:0/1624989846 shutdown_connections 2024-03-18T20:49:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:24.191+0000 7f997d846700 1 -- 172.21.15.46:0/1624989846 wait complete. 2024-03-18T20:49:24.250 INFO:tasks.workunit.client.0.smithi046.stdout: 0 cleanup 619 sec 2024-03-18T20:49:24.251 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-03-18T20:49:24.251 INFO:tasks.workunit.client.0.smithi046.stdout: Operation Count AvgLat MaxLat 2024-03-18T20:49:24.251 INFO:tasks.workunit.client.0.smithi046.stdout: ---------------------------------------- 2024-03-18T20:49:24.251 INFO:tasks.workunit.client.0.smithi046.stdout: NTCreateX 70631 3.305 13096.898 2024-03-18T20:49:24.251 INFO:tasks.workunit.client.0.smithi046.stdout: Close 51884 0.003 0.312 2024-03-18T20:49:24.251 INFO:tasks.workunit.client.0.smithi046.stdout: Rename 2991 81.427 4776.016 2024-03-18T20:49:24.251 INFO:tasks.workunit.client.0.smithi046.stdout: Unlink 14259 2.611 13546.160 2024-03-18T20:49:24.251 INFO:tasks.workunit.client.0.smithi046.stdout: Deltree 2 103.029 120.438 2024-03-18T20:49:24.251 INFO:tasks.workunit.client.0.smithi046.stdout: Mkdir 1 0.007 0.007 2024-03-18T20:49:24.251 INFO:tasks.workunit.client.0.smithi046.stdout: Qpathinfo 64014 1.131 40.455 2024-03-18T20:49:24.251 INFO:tasks.workunit.client.0.smithi046.stdout: Qfileinfo 11227 0.002 0.034 2024-03-18T20:49:24.252 INFO:tasks.workunit.client.0.smithi046.stdout: Qfsinfo 11734 0.229 10.995 2024-03-18T20:49:24.252 INFO:tasks.workunit.client.0.smithi046.stdout: Sfileinfo 5752 0.016 6.492 2024-03-18T20:49:24.252 INFO:tasks.workunit.client.0.smithi046.stdout: Find 24746 0.209 27.995 2024-03-18T20:49:24.252 INFO:tasks.workunit.client.0.smithi046.stdout: WriteX 35258 0.107 36.077 2024-03-18T20:49:24.252 INFO:tasks.workunit.client.0.smithi046.stdout: ReadX 110697 0.010 11.646 2024-03-18T20:49:24.252 INFO:tasks.workunit.client.0.smithi046.stdout: LockX 230 0.808 2.392 2024-03-18T20:49:24.252 INFO:tasks.workunit.client.0.smithi046.stdout: UnlockX 230 0.754 1.733 2024-03-18T20:49:24.252 INFO:tasks.workunit.client.0.smithi046.stdout: Flush 4951 2.814 50.722 2024-03-18T20:49:24.252 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-03-18T20:49:24.252 INFO:tasks.workunit.client.0.smithi046.stdout:Throughput 3.69411 MB/sec 1 clients 1 procs max_latency=13096.906 ms 2024-03-18T20:49:24.253 INFO:tasks.workunit.client.0.smithi046.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-18T20:49:24.253 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-03-18T20:49:24.253 INFO:tasks.workunit.client.0.smithi046.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-18T20:49:24.253 INFO:tasks.workunit.client.0.smithi046.stdout:8 of 10 processes prepared for launch 0 sec 2024-03-18T20:49:24.356 INFO:tasks.workunit.client.0.smithi046.stdout:10 of 10 processes prepared for launch 0 sec 2024-03-18T20:49:24.356 INFO:tasks.workunit.client.0.smithi046.stdout:releasing clients 2024-03-18T20:49:24.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:24 smithi073 ceph-mon[47384]: pgmap v416: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 781 KiB/s wr, 73 op/s 2024-03-18T20:49:24.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:24 smithi155 ceph-mon[48344]: pgmap v416: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 781 KiB/s wr, 73 op/s 2024-03-18T20:49:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:24 smithi046 ceph-mon[43495]: pgmap v416: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 781 KiB/s wr, 73 op/s 2024-03-18T20:49:25.353 INFO:tasks.workunit.client.0.smithi046.stdout: 10 345 124.47 MB/sec warmup 1 sec latency 550.410 ms 2024-03-18T20:49:25.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:25 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:25.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:25 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:25.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:25 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:26.353 INFO:tasks.workunit.client.0.smithi046.stdout: 10 760 100.07 MB/sec warmup 2 sec latency 1550.479 ms 2024-03-18T20:49:26.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:26 smithi073 ceph-mon[47384]: pgmap v417: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 700 KiB/s wr, 49 op/s 2024-03-18T20:49:26.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:26 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:26.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:26 smithi155 ceph-mon[48344]: pgmap v417: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 700 KiB/s wr, 49 op/s 2024-03-18T20:49:26.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:26 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:26.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:26 smithi046 ceph-mon[43495]: pgmap v417: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 700 KiB/s wr, 49 op/s 2024-03-18T20:49:26.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:26 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:27.353 INFO:tasks.workunit.client.0.smithi046.stdout: 10 1304 81.01 MB/sec warmup 3 sec latency 1674.323 ms 2024-03-18T20:49:27.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:27 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:27.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:27 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:27.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:27 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:28.353 INFO:tasks.workunit.client.0.smithi046.stdout: 10 2245 67.24 MB/sec warmup 4 sec latency 133.562 ms 2024-03-18T20:49:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:28 smithi073 ceph-mon[47384]: pgmap v418: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 700 KiB/s wr, 49 op/s 2024-03-18T20:49:28.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:28 smithi155 ceph-mon[48344]: pgmap v418: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 700 KiB/s wr, 49 op/s 2024-03-18T20:49:28.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:28 smithi046 ceph-mon[43495]: pgmap v418: 97 pgs: 97 active+clean; 213 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 700 KiB/s wr, 49 op/s 2024-03-18T20:49:29.353 INFO:tasks.workunit.client.0.smithi046.stdout: 10 3613 76.24 MB/sec warmup 5 sec latency 41.653 ms 2024-03-18T20:49:30.353 INFO:tasks.workunit.client.0.smithi046.stdout: 10 4741 70.34 MB/sec warmup 6 sec latency 51.194 ms 2024-03-18T20:49:30.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:30 smithi073 ceph-mon[47384]: pgmap v419: 97 pgs: 97 active+clean; 222 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-03-18T20:49:30.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:30 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:30.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:30 smithi155 ceph-mon[48344]: pgmap v419: 97 pgs: 97 active+clean; 222 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-03-18T20:49:30.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:30 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:30.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:30 smithi046 ceph-mon[43495]: pgmap v419: 97 pgs: 97 active+clean; 222 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-03-18T20:49:30.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:30 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:31.353 INFO:tasks.workunit.client.0.smithi046.stdout: 10 6490 75.13 MB/sec warmup 7 sec latency 53.263 ms 2024-03-18T20:49:31.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:31 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:31.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:31 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:31.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:31 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:49:32.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 7415 72.36 MB/sec warmup 8 sec latency 52.019 ms 2024-03-18T20:49:32.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:32 smithi073 ceph-mon[47384]: pgmap v420: 97 pgs: 97 active+clean; 222 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-03-18T20:49:32.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:32 smithi155 ceph-mon[48344]: pgmap v420: 97 pgs: 97 active+clean; 222 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-03-18T20:49:32.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:32 smithi046 ceph-mon[43495]: pgmap v420: 97 pgs: 97 active+clean; 222 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 1.6 MiB/s wr, 144 op/s 2024-03-18T20:49:33.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 8852 71.68 MB/sec warmup 9 sec latency 63.786 ms 2024-03-18T20:49:34.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 10133 72.37 MB/sec warmup 10 sec latency 30.719 ms 2024-03-18T20:49:34.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:34 smithi073 ceph-mon[47384]: pgmap v421: 97 pgs: 97 active+clean; 261 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 482 op/s 2024-03-18T20:49:34.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:34 smithi155 ceph-mon[48344]: pgmap v421: 97 pgs: 97 active+clean; 261 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 482 op/s 2024-03-18T20:49:34.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:34 smithi046 ceph-mon[43495]: pgmap v421: 97 pgs: 97 active+clean; 261 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 482 op/s 2024-03-18T20:49:35.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 11164 70.39 MB/sec warmup 11 sec latency 47.596 ms 2024-03-18T20:49:36.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:36 smithi046 ceph-mon[43495]: pgmap v422: 97 pgs: 97 active+clean; 261 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 454 op/s 2024-03-18T20:49:36.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 12574 70.35 MB/sec warmup 12 sec latency 37.343 ms 2024-03-18T20:49:36.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:36 smithi073 ceph-mon[47384]: pgmap v422: 97 pgs: 97 active+clean; 261 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 454 op/s 2024-03-18T20:49:36.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:36 smithi155 ceph-mon[48344]: pgmap v422: 97 pgs: 97 active+clean; 261 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 454 op/s 2024-03-18T20:49:37.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 13890 71.47 MB/sec warmup 13 sec latency 61.841 ms 2024-03-18T20:49:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.375+0000 7fa56045c700 1 -- 172.21.15.46:0/2037438090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa558081930 msgr2=0x7fa558079710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.375+0000 7fa56045c700 1 --2- 172.21.15.46:0/2037438090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa558081930 0x7fa558079710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa554009990 tx=0x7fa554009ca0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.375+0000 7fa56045c700 1 -- 172.21.15.46:0/2037438090 shutdown_connections 2024-03-18T20:49:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.375+0000 7fa56045c700 1 --2- 172.21.15.46:0/2037438090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa55811a1d0 0x7fa55811c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.375+0000 7fa56045c700 1 --2- 172.21.15.46:0/2037438090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa558081930 0x7fa558079710 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.375+0000 7fa56045c700 1 --2- 172.21.15.46:0/2037438090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa558079c50 0x7fa55807a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.375+0000 7fa56045c700 1 -- 172.21.15.46:0/2037438090 >> 172.21.15.46:0/2037438090 conn(0x7fa558076ab0 msgr2=0x7fa558078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.376+0000 7fa56045c700 1 -- 172.21.15.46:0/2037438090 shutdown_connections 2024-03-18T20:49:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.376+0000 7fa56045c700 1 -- 172.21.15.46:0/2037438090 wait complete. 2024-03-18T20:49:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.377+0000 7fa56045c700 1 Processor -- start 2024-03-18T20:49:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.377+0000 7fa56045c700 1 -- start start 2024-03-18T20:49:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.377+0000 7fa56045c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa558079c50 0x7fa558111db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.377+0000 7fa56045c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa558081930 0x7fa5581122f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.377+0000 7fa56045c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa55811a1d0 0x7fa558119ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.377+0000 7fa56045c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55810cfe0 con 0x7fa558081930 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.377+0000 7fa56045c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55810d150 con 0x7fa55811a1d0 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.377+0000 7fa56045c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55810d310 con 0x7fa558079c50 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.377+0000 7fa55e9f9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa55811a1d0 0x7fa558119ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.377+0000 7fa55e9f9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa55811a1d0 0x7fa558119ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47694/0 (socket says 172.21.15.46:47694) 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.377+0000 7fa55e9f9700 1 -- 172.21.15.46:0/4012376916 learned_addr learned my addr 172.21.15.46:0/4012376916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.377+0000 7fa55e1f8700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa558079c50 0x7fa558111db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.378+0000 7fa55d9f7700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa558081930 0x7fa5581122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.378+0000 7fa55e9f9700 1 -- 172.21.15.46:0/4012376916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa558079c50 msgr2=0x7fa558111db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.378+0000 7fa55e9f9700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa558079c50 0x7fa558111db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.378+0000 7fa55e9f9700 1 -- 172.21.15.46:0/4012376916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa558081930 msgr2=0x7fa5581122f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.378+0000 7fa55e9f9700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa558081930 0x7fa5581122f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.378+0000 7fa55e9f9700 1 -- 172.21.15.46:0/4012376916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5540095f0 con 0x7fa55811a1d0 2024-03-18T20:49:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.378+0000 7fa55d9f7700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa558081930 0x7fa5581122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:49:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.378+0000 7fa55e9f9700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa55811a1d0 0x7fa558119ad0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa550007f10 tx=0x7fa55000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.379+0000 7fa54f7fe700 1 -- 172.21.15.46:0/4012376916 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550017070 con 0x7fa55811a1d0 2024-03-18T20:49:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.379+0000 7fa54f7fe700 1 -- 172.21.15.46:0/4012376916 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa55000f040 con 0x7fa55811a1d0 2024-03-18T20:49:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.379+0000 7fa54f7fe700 1 -- 172.21.15.46:0/4012376916 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550005740 con 0x7fa55811a1d0 2024-03-18T20:49:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.379+0000 7fa56045c700 1 -- 172.21.15.46:0/4012376916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55810d5f0 con 0x7fa55811a1d0 2024-03-18T20:49:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.379+0000 7fa56045c700 1 -- 172.21.15.46:0/4012376916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55810dab0 con 0x7fa55811a1d0 2024-03-18T20:49:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.383+0000 7fa56c5f1b80 1 Processor -- start 2024-03-18T20:49:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.384+0000 7fa54f7fe700 1 -- 172.21.15.46:0/4012376916 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa55000dc60 con 0x7fa55811a1d0 2024-03-18T20:49:37.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.384+0000 7fa56c5f1b80 1 -- start start 2024-03-18T20:49:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.384+0000 7fa56c5f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f8c570 0x5620b5f8c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.384+0000 7fa56c5f1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8ce90 0x5620b5f97870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.384+0000 7fa56c5f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f97db0 0x5620b5f9a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.384+0000 7fa56c5f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5ee6930 con 0x5620b5f97db0 2024-03-18T20:49:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.384+0000 7fa56c5f1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5ee42f0 con 0x5620b5f8ce90 2024-03-18T20:49:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.384+0000 7fa56c5f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5edfbe0 con 0x5620b5f8c570 2024-03-18T20:49:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.385+0000 7fa55d9f7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f8c570 0x5620b5f8c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.385+0000 7fa55d9f7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f8c570 0x5620b5f8c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:48706/0 (socket says 172.21.15.46:48706) 2024-03-18T20:49:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.385+0000 7fa55d9f7700 1 -- 172.21.15.46:0/1481698090 learned_addr learned my addr 172.21.15.46:0/1481698090 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.385+0000 7fa55d9f7700 1 --2- 172.21.15.46:0/1481698090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f97db0 0x5620b5f9a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.385+0000 7fa55d9f7700 1 -- 172.21.15.46:0/1481698090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8ce90 msgr2=0x5620b5f97870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:49:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.385+0000 7fa55d9f7700 1 --2- 172.21.15.46:0/1481698090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8ce90 0x5620b5f97870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.385+0000 7fa55d9f7700 1 -- 172.21.15.46:0/1481698090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f97db0 msgr2=0x5620b5f9a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.385+0000 7fa55d9f7700 1 --2- 172.21.15.46:0/1481698090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f97db0 0x5620b5f9a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.385+0000 7fa55d9f7700 1 -- 172.21.15.46:0/1481698090 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b5d61350 con 0x5620b5f8c570 2024-03-18T20:49:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.385+0000 7fa54f7fe700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa5440a1310 0x7fa5440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.385+0000 7fa54f7fe700 1 -- 172.21.15.46:0/4012376916 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa5500cc470 con 0x7fa55811a1d0 2024-03-18T20:49:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.386+0000 7fa55d9f7700 1 --2- 172.21.15.46:0/1481698090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f8c570 0x5620b5f8c950 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa548006fd0 tx=0x7fa54800c230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f8973a893eb5c44 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.386+0000 7fa54cda8700 1 -- 172.21.15.46:0/1481698090 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548015e30 con 0x5620b5f8c570 2024-03-18T20:49:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.386+0000 7fa54cda8700 1 -- 172.21.15.46:0/1481698090 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa548018930 con 0x5620b5f8c570 2024-03-18T20:49:37.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.386+0000 7fa54cda8700 1 -- 172.21.15.46:0/1481698090 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548016800 con 0x5620b5f8c570 2024-03-18T20:49:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.386+0000 7fa55e1f8700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa5440a1310 0x7fa5440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.387+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/1481698090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f8c570 msgr2=0x5620b5f8c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.387+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/1481698090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f8c570 0x5620b5f8c950 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa548006fd0 tx=0x7fa54800c230 comp rx=0 tx=0).stop 2024-03-18T20:49:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.387+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/1481698090 shutdown_connections 2024-03-18T20:49:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.387+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/1481698090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f97db0 0x5620b5f9a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.387+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/1481698090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f8c570 0x5620b5f8c950 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.387+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/1481698090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8ce90 0x5620b5f97870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.387+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/1481698090 >> 172.21.15.46:0/1481698090 conn(0x5620b5f031c0 msgr2=0x5620b5f88a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.387+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/1481698090 shutdown_connections 2024-03-18T20:49:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.387+0000 7fa55e1f8700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa5440a1310 0x7fa5440a37d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa554006010 tx=0x7fa55403b040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.387+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/1481698090 wait complete. 2024-03-18T20:49:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa56c5f1b80 1 -- start start 2024-03-18T20:49:37.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa56c5f1b80 1 Processor -- start 2024-03-18T20:49:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa56c5f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f8c570 0x5620b5f94c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa56c5f1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8ce90 0x5620b5f95150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa56c5f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f97db0 0x5620b5f95690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa56c5f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5e7bd20 con 0x5620b5f97db0 2024-03-18T20:49:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa56c5f1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5e7b9c0 con 0x5620b5f8ce90 2024-03-18T20:49:37.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa56c5f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5d63640 con 0x5620b5f8c570 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa56c5f1b80 10 client.0 _ll_register_callbacks cb 0x5620b5ce7340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa55e9f9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f97db0 0x5620b5f95690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa55e9f9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f97db0 0x5620b5f95690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58322/0 (socket says 172.21.15.46:58322) 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa55e9f9700 1 -- 172.21.15.46:0/3008577533 learned_addr learned my addr 172.21.15.46:0/3008577533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.388+0000 7fa55d9f7700 1 --2- 172.21.15.46:0/3008577533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8ce90 0x5620b5f95150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.389+0000 7fa55e9f9700 1 -- 172.21.15.46:0/3008577533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f8c570 msgr2=0x5620b5f94c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.389+0000 7fa55e9f9700 1 --2- 172.21.15.46:0/3008577533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f8c570 0x5620b5f94c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.389+0000 7fa55e9f9700 1 -- 172.21.15.46:0/3008577533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8ce90 msgr2=0x5620b5f95150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.389+0000 7fa55e9f9700 1 --2- 172.21.15.46:0/3008577533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8ce90 0x5620b5f95150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.389+0000 7fa55e9f9700 1 -- 172.21.15.46:0/3008577533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa548015320 con 0x5620b5f97db0 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.389+0000 7fa55e9f9700 1 --2- 172.21.15.46:0/3008577533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f97db0 0x5620b5f95690 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa550099450 tx=0x7fa550098e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3b613815edd9b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.389+0000 7fa541ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:37.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.389+0000 7fa541ffb700 1 -- 172.21.15.46:0/3008577533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550005aa0 con 0x5620b5f97db0 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.389+0000 7fa541ffb700 1 -- 172.21.15.46:0/3008577533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa55009d2f0 con 0x5620b5f97db0 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.389+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/3008577533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620b5f806c0 con 0x5620b5f97db0 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.390+0000 7fa541ffb700 1 -- 172.21.15.46:0/3008577533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5500a1940 con 0x5620b5f97db0 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.390+0000 7fa56c5f1b80 10 client.20187 fetch_fsmap learned FSMap version 65 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.390+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/3008577533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x5620b5d0db00 con 0x5620b5f97db0 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.390+0000 7fa541ffb700 1 -- 172.21.15.46:0/3008577533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620b5f806c0 con 0x5620b5f97db0 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.390+0000 7fa541ffb700 1 -- 172.21.15.46:0/3008577533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 65) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fa5500a14b0 con 0x5620b5f97db0 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.390+0000 7fa56c5f1b80 10 client.20187 fetch_fsmap finished waiting for FSMap version 65 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.390+0000 7fa56c5f1b80 10 client.20187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.390+0000 7fa56c5f1b80 20 client.20187 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.390+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/3008577533 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5620b5fa7a50 0x5620b5fa9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.390+0000 7fa56c5f1b80 4 client.20187 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:49:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.390+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/3008577533 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620b5ed86a0 con 0x5620b5fa7a50 2024-03-18T20:49:37.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.393+0000 7fa55e1f8700 1 --2- 172.21.15.46:0/3008577533 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5620b5fa7a50 0x5620b5fa9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.394+0000 7fa55e1f8700 1 --2- 172.21.15.46:0/3008577533 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5620b5fa7a50 0x5620b5fa9eb0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92697f47cea3048a server_cookie=d7ee3814d192630d in_seq=0 out_seq=0 2024-03-18T20:49:37.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.394+0000 7fa541ffb700 10 client.20187 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:37.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.394+0000 7fa541ffb700 1 -- 172.21.15.46:0/3008577533 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5620b5ed86a0 con 0x5620b5fa7a50 2024-03-18T20:49:37.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.394+0000 7fa541ffb700 10 client.20187 handle_command_reply: tid=0 2024-03-18T20:49:37.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.394+0000 7fa56c5f1b80 1 client.20187 shutdown 2024-03-18T20:49:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.395+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/3008577533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f97db0 msgr2=0x5620b5f95690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.395+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/3008577533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f97db0 0x5620b5f95690 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa550099450 tx=0x7fa550098e20 comp rx=0 tx=0).stop 2024-03-18T20:49:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.395+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/3008577533 shutdown_connections 2024-03-18T20:49:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.395+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/3008577533 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5620b5fa7a50 0x5620b5fa9eb0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.395+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/3008577533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f97db0 0x5620b5f95690 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.395+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/3008577533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f8c570 0x5620b5f94c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.395+0000 7fa541ffb700 0 client.20187 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.395+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/3008577533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8ce90 0x5620b5f95150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.395+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/3008577533 >> 172.21.15.46:0/3008577533 conn(0x5620b5f031c0 msgr2=0x5620b5f88650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.395+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/3008577533 shutdown_connections 2024-03-18T20:49:37.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.395+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/3008577533 wait complete. 2024-03-18T20:49:37.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.395+0000 7fa56c5f1b80 20 client.20187 trim_cache size 0 max 16384 2024-03-18T20:49:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.417+0000 7fa56c5f1b80 1 Processor -- start 2024-03-18T20:49:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa56c5f1b80 1 -- start start 2024-03-18T20:49:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa56c5f1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8c570 0x5620b5fa3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa56c5f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f8ce90 0x5620b5f88bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa56c5f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 0x5620b5f8ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa56c5f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5f961c0 con 0x5620b5f8ce90 2024-03-18T20:49:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa56c5f1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5f8e130 con 0x5620b5f8c570 2024-03-18T20:49:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa56c5f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5f8df60 con 0x5620b5f97db0 2024-03-18T20:49:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa55d9f7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f8ce90 0x5620b5f88bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa55e1f8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8c570 0x5620b5fa3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa55e9f9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 0x5620b5f8ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa55e1f8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8c570 0x5620b5fa3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47708/0 (socket says 172.21.15.46:47708) 2024-03-18T20:49:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa55e1f8700 1 -- 172.21.15.46:0/1369173896 learned_addr learned my addr 172.21.15.46:0/1369173896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.418+0000 7fa55d9f7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f8ce90 0x5620b5f88bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58338/0 (socket says 172.21.15.46:58338) 2024-03-18T20:49:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.419+0000 7fa55e9f9700 1 -- 172.21.15.46:0/1369173896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8c570 msgr2=0x5620b5fa3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.419+0000 7fa55e9f9700 1 --2- 172.21.15.46:0/1369173896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8c570 0x5620b5fa3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.419+0000 7fa55e9f9700 1 -- 172.21.15.46:0/1369173896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f8ce90 msgr2=0x5620b5f88bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.419+0000 7fa55e9f9700 1 --2- 172.21.15.46:0/1369173896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f8ce90 0x5620b5f88bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.419+0000 7fa55e9f9700 1 -- 172.21.15.46:0/1369173896 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620b5ea5e40 con 0x5620b5f97db0 2024-03-18T20:49:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.419+0000 7fa55e9f9700 1 --2- 172.21.15.46:0/1369173896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 0x5620b5f8ff50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa5500959c0 tx=0x7fa550099640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee19514bc7ef2a56 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.419+0000 7fa54d5a9700 1 -- 172.21.15.46:0/1369173896 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550009e20 con 0x5620b5f97db0 2024-03-18T20:49:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.419+0000 7fa54d5a9700 1 -- 172.21.15.46:0/1369173896 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa55009d2f0 con 0x5620b5f97db0 2024-03-18T20:49:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.419+0000 7fa54d5a9700 1 -- 172.21.15.46:0/1369173896 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5500058a0 con 0x5620b5f97db0 2024-03-18T20:49:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.420+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/1369173896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 msgr2=0x5620b5f8ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.420+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/1369173896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 0x5620b5f8ff50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa5500959c0 tx=0x7fa550099640 comp rx=0 tx=0).stop 2024-03-18T20:49:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.421+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/1369173896 shutdown_connections 2024-03-18T20:49:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.421+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/1369173896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 0x5620b5f8ff50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.421+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/1369173896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8c570 0x5620b5fa3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.421+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/1369173896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f8ce90 0x5620b5f88bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.421+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/1369173896 >> 172.21.15.46:0/1369173896 conn(0x5620b5f031c0 msgr2=0x5620b5f944a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.421+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/1369173896 shutdown_connections 2024-03-18T20:49:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.421+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/1369173896 wait complete. 2024-03-18T20:49:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.421+0000 7fa56c5f1b80 1 -- start start 2024-03-18T20:49:37.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.421+0000 7fa56c5f1b80 1 Processor -- start 2024-03-18T20:49:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa56c5f1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8c570 0x5620b5f9ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa56c5f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f8ce90 0x5620b5f9bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa56c5f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 0x5620b5f9c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa56c5f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5f9cb20 con 0x5620b5f8ce90 2024-03-18T20:49:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa56c5f1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5f9cc90 con 0x5620b5f8c570 2024-03-18T20:49:37.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa56c5f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620b5ffa900 con 0x5620b5f97db0 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa56c5f1b80 10 client.0 _ll_register_callbacks cb 0x7fa5500992d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa55e9f9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 0x5620b5f9c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa55e9f9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 0x5620b5f9c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:48728/0 (socket says 172.21.15.46:48728) 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa55e9f9700 1 -- 172.21.15.46:0/680533865 learned_addr learned my addr 172.21.15.46:0/680533865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa55e1f8700 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8c570 0x5620b5f9ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa55d9f7700 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f8ce90 0x5620b5f9bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa55e9f9700 1 -- 172.21.15.46:0/680533865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8c570 msgr2=0x5620b5f9ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa55e9f9700 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8c570 0x5620b5f9ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa55e9f9700 1 -- 172.21.15.46:0/680533865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f8ce90 msgr2=0x5620b5f9bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa55e9f9700 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f8ce90 0x5620b5f9bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa55e9f9700 1 -- 172.21.15.46:0/680533865 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620b5d0db00 con 0x5620b5f97db0 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.422+0000 7fa55d9f7700 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f8ce90 0x5620b5f9bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.423+0000 7fa55e9f9700 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 0x5620b5f9c440 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa5500a1410 tx=0x7fa5500a16e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc255d8841ad5543 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:37.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.423+0000 7fa541ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.423+0000 7fa541ffb700 1 -- 172.21.15.46:0/680533865 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550009e20 con 0x5620b5f97db0 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.423+0000 7fa541ffb700 1 -- 172.21.15.46:0/680533865 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa5500a1940 con 0x5620b5f97db0 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.423+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/680533865 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa548018930 con 0x5620b5f97db0 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.423+0000 7fa541ffb700 1 -- 172.21.15.46:0/680533865 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa550005ab0 con 0x5620b5f97db0 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.423+0000 7fa56c5f1b80 10 client.29830 fetch_fsmap learned FSMap version 65 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.423+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/680533865 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x5620b5d65c70 con 0x5620b5f97db0 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.423+0000 7fa541ffb700 1 -- 172.21.15.46:0/680533865 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa548018930 con 0x5620b5f97db0 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.424+0000 7fa541ffb700 1 -- 172.21.15.46:0/680533865 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 65) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fa5500b7070 con 0x5620b5f97db0 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.424+0000 7fa56c5f1b80 10 client.29830 fetch_fsmap finished waiting for FSMap version 65 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.424+0000 7fa56c5f1b80 10 client.29830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.424+0000 7fa56c5f1b80 20 client.29830 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.424+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5620b5ffbb50 0x5620b5ffdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.424+0000 7fa56c5f1b80 4 client.29830 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:49:37.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.424+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/680533865 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5620b5d2ec90 con 0x5620b5ffbb50 2024-03-18T20:49:37.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.424+0000 7fa55e1f8700 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5620b5ffbb50 0x5620b5ffdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.432+0000 7fa541ffb700 10 client.29830 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.432+0000 7fa55e1f8700 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5620b5ffbb50 0x5620b5ffdf40 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae5ba436cb885cf0 server_cookie=945d8a4440cd6908 in_seq=0 out_seq=0 2024-03-18T20:49:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.680+0000 7fa541ffb700 1 -- 172.21.15.46:0/680533865 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5620b5d2ec90 con 0x5620b5ffbb50 2024-03-18T20:49:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.680+0000 7fa541ffb700 10 client.29830 handle_command_reply: tid=0 2024-03-18T20:49:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.683+0000 7fa56c5f1b80 1 client.29830 shutdown 2024-03-18T20:49:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.688+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/680533865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 msgr2=0x5620b5f9c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.688+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 0x5620b5f9c440 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa5500a1410 tx=0x7fa5500a16e0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.688+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/680533865 shutdown_connections 2024-03-18T20:49:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.688+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5620b5ffbb50 0x5620b5ffdf40 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.688+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5620b5f97db0 0x5620b5f9c440 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.688+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5620b5f8c570 0x5620b5f9ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.688+0000 7fa56c5f1b80 1 --2- 172.21.15.46:0/680533865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5620b5f8ce90 0x5620b5f9bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.688+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/680533865 >> 172.21.15.46:0/680533865 conn(0x5620b5f031c0 msgr2=0x5620b5f93480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.688+0000 7fa541ffb700 0 client.29830 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.690+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/680533865 shutdown_connections 2024-03-18T20:49:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.690+0000 7fa56c5f1b80 1 -- 172.21.15.46:0/680533865 wait complete. 2024-03-18T20:49:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:49:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:49:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:49:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:49:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.691+0000 7fa56c5f1b80 20 client.29830 trim_cache size 0 max 16384 2024-03-18T20:49:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.701+0000 7fa56045c700 1 -- 172.21.15.46:0/4012376916 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa5440a1310 msgr2=0x7fa5440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.701+0000 7fa56045c700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa5440a1310 0x7fa5440a37d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa554006010 tx=0x7fa55403b040 comp rx=0 tx=0).stop 2024-03-18T20:49:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.701+0000 7fa56045c700 1 -- 172.21.15.46:0/4012376916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa55811a1d0 msgr2=0x7fa558119ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.701+0000 7fa56045c700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa55811a1d0 0x7fa558119ad0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa550007f10 tx=0x7fa55000d3b0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.701+0000 7fa56045c700 1 -- 172.21.15.46:0/4012376916 shutdown_connections 2024-03-18T20:49:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.701+0000 7fa56045c700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa5440a1310 0x7fa5440a37d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.701+0000 7fa56045c700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa55811a1d0 0x7fa558119ad0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.701+0000 7fa56045c700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa558079c50 0x7fa558111db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.701+0000 7fa56045c700 1 --2- 172.21.15.46:0/4012376916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa558081930 0x7fa5581122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.701+0000 7fa56045c700 1 -- 172.21.15.46:0/4012376916 >> 172.21.15.46:0/4012376916 conn(0x7fa558076ab0 msgr2=0x7fa558078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.704+0000 7fa56045c700 1 -- 172.21.15.46:0/4012376916 shutdown_connections 2024-03-18T20:49:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:37.705+0000 7fa56045c700 1 -- 172.21.15.46:0/4012376916 wait complete. 2024-03-18T20:49:38.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 14843 69.83 MB/sec warmup 14 sec latency 124.336 ms 2024-03-18T20:49:38.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:38 smithi073 ceph-mon[47384]: pgmap v423: 97 pgs: 97 active+clean; 261 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 454 op/s 2024-03-18T20:49:38.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:38 smithi155 ceph-mon[48344]: pgmap v423: 97 pgs: 97 active+clean; 261 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 454 op/s 2024-03-18T20:49:38.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:38 smithi046 ceph-mon[43495]: pgmap v423: 97 pgs: 97 active+clean; 261 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 454 op/s 2024-03-18T20:49:39.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 16067 69.35 MB/sec warmup 15 sec latency 52.414 ms 2024-03-18T20:49:40.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 17388 69.33 MB/sec warmup 16 sec latency 32.350 ms 2024-03-18T20:49:40.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:40 smithi073 ceph-mon[47384]: pgmap v424: 97 pgs: 97 active+clean; 283 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.6 MiB/s wr, 912 op/s 2024-03-18T20:49:40.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:40 smithi155 ceph-mon[48344]: pgmap v424: 97 pgs: 97 active+clean; 283 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.6 MiB/s wr, 912 op/s 2024-03-18T20:49:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:40 smithi046 ceph-mon[43495]: pgmap v424: 97 pgs: 97 active+clean; 283 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.6 MiB/s wr, 912 op/s 2024-03-18T20:49:41.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 18734 69.76 MB/sec warmup 17 sec latency 45.434 ms 2024-03-18T20:49:42.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 19981 69.48 MB/sec warmup 18 sec latency 42.860 ms 2024-03-18T20:49:42.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:42 smithi073 ceph-mon[47384]: pgmap v425: 97 pgs: 97 active+clean; 283 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.5 MiB/s wr, 797 op/s 2024-03-18T20:49:42.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:42 smithi155 ceph-mon[48344]: pgmap v425: 97 pgs: 97 active+clean; 283 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.5 MiB/s wr, 797 op/s 2024-03-18T20:49:42.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:42 smithi046 ceph-mon[43495]: pgmap v425: 97 pgs: 97 active+clean; 283 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.5 MiB/s wr, 797 op/s 2024-03-18T20:49:43.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 21235 69.88 MB/sec warmup 19 sec latency 38.867 ms 2024-03-18T20:49:44.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 21633 66.89 MB/sec warmup 20 sec latency 499.676 ms 2024-03-18T20:49:44.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:44 smithi073 ceph-mon[47384]: pgmap v426: 97 pgs: 97 active+clean; 318 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-03-18T20:49:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:44 smithi155 ceph-mon[48344]: pgmap v426: 97 pgs: 97 active+clean; 318 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-03-18T20:49:44.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:44 smithi046 ceph-mon[43495]: pgmap v426: 97 pgs: 97 active+clean; 318 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-03-18T20:49:45.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 21974 64.58 MB/sec warmup 21 sec latency 655.440 ms 2024-03-18T20:49:46.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 22574 63.26 MB/sec warmup 22 sec latency 94.797 ms 2024-03-18T20:49:46.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:46 smithi073 ceph-mon[47384]: pgmap v427: 97 pgs: 97 active+clean; 318 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.1 MiB/s wr, 802 op/s 2024-03-18T20:49:46.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:46 smithi155 ceph-mon[48344]: pgmap v427: 97 pgs: 97 active+clean; 318 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.1 MiB/s wr, 802 op/s 2024-03-18T20:49:46.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:46 smithi046 ceph-mon[43495]: pgmap v427: 97 pgs: 97 active+clean; 318 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.1 MiB/s wr, 802 op/s 2024-03-18T20:49:47.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 23766 63.07 MB/sec warmup 23 sec latency 61.952 ms 2024-03-18T20:49:48.356 INFO:tasks.workunit.client.0.smithi046.stdout: 10 24861 62.93 MB/sec warmup 24 sec latency 36.662 ms 2024-03-18T20:49:48.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:48 smithi073 ceph-mon[47384]: pgmap v428: 97 pgs: 97 active+clean; 318 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.1 MiB/s wr, 802 op/s 2024-03-18T20:49:48.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:48 smithi155 ceph-mon[48344]: pgmap v428: 97 pgs: 97 active+clean; 318 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.1 MiB/s wr, 802 op/s 2024-03-18T20:49:48.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:48 smithi046 ceph-mon[43495]: pgmap v428: 97 pgs: 97 active+clean; 318 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.1 MiB/s wr, 802 op/s 2024-03-18T20:49:49.357 INFO:tasks.workunit.client.0.smithi046.stdout: 10 25912 62.84 MB/sec warmup 25 sec latency 76.788 ms 2024-03-18T20:49:50.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 27015 62.62 MB/sec warmup 26 sec latency 76.200 ms 2024-03-18T20:49:50.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:50 smithi073 ceph-mon[47384]: pgmap v429: 97 pgs: 97 active+clean; 518 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 40 MiB/s wr, 1.57k op/s 2024-03-18T20:49:50.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:50 smithi155 ceph-mon[48344]: pgmap v429: 97 pgs: 97 active+clean; 518 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 40 MiB/s wr, 1.57k op/s 2024-03-18T20:49:50.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:50 smithi046 ceph-mon[43495]: pgmap v429: 97 pgs: 97 active+clean; 518 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 40 MiB/s wr, 1.57k op/s 2024-03-18T20:49:50.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.879+0000 7f89fcfbb700 1 -- 172.21.15.46:0/984012413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f8100570 msgr2=0x7f89f810ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:50.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.879+0000 7f89fcfbb700 1 --2- 172.21.15.46:0/984012413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f8100570 0x7f89f810ca00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009f90 tx=0x7f89f000b330 comp rx=0 tx=0).stop 2024-03-18T20:49:50.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.879+0000 7f89fcfbb700 1 -- 172.21.15.46:0/984012413 shutdown_connections 2024-03-18T20:49:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.879+0000 7f89fcfbb700 1 --2- 172.21.15.46:0/984012413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f8100570 0x7f89f810ca00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.879+0000 7f89fcfbb700 1 --2- 172.21.15.46:0/984012413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89f8105c60 0x7f89f8106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.879+0000 7f89fcfbb700 1 --2- 172.21.15.46:0/984012413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f80ffbb0 0x7f89f8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.879+0000 7f89fcfbb700 1 -- 172.21.15.46:0/984012413 >> 172.21.15.46:0/984012413 conn(0x7f89f8076d50 msgr2=0x7f89f8077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.879+0000 7f89fcfbb700 1 -- 172.21.15.46:0/984012413 shutdown_connections 2024-03-18T20:49:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.879+0000 7f89fcfbb700 1 -- 172.21.15.46:0/984012413 wait complete. 2024-03-18T20:49:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.880+0000 7f89fcfbb700 1 Processor -- start 2024-03-18T20:49:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.881+0000 7f89fcfbb700 1 -- start start 2024-03-18T20:49:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.881+0000 7f89fcfbb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89f80ffbb0 0x7f89f8101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.881+0000 7f89fcfbb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f8100570 0x7f89f8101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.881+0000 7f89fcfbb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f8105c60 0x7f89f81022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.881+0000 7f89fcfbb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f81a5650 con 0x7f89f8100570 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.881+0000 7f89fcfbb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f81a57c0 con 0x7f89f80ffbb0 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.881+0000 7f89fcfbb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f81a5980 con 0x7f89f8105c60 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.881+0000 7f89effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f8100570 0x7f89f8101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.881+0000 7f89f6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f8105c60 0x7f89f81022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.881+0000 7f89effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f8100570 0x7f89f8101d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34986/0 (socket says 172.21.15.46:34986) 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.881+0000 7f89effff700 1 -- 172.21.15.46:0/80761979 learned_addr learned my addr 172.21.15.46:0/80761979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.881+0000 7f89f659c700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89f80ffbb0 0x7f89f8101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.882+0000 7f89effff700 1 -- 172.21.15.46:0/80761979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f8105c60 msgr2=0x7f89f81022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.882+0000 7f89effff700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f8105c60 0x7f89f81022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.882+0000 7f89effff700 1 -- 172.21.15.46:0/80761979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89f80ffbb0 msgr2=0x7f89f8101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.882+0000 7f89effff700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89f80ffbb0 0x7f89f8101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.882+0000 7f89effff700 1 -- 172.21.15.46:0/80761979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f0009c40 con 0x7f89f8100570 2024-03-18T20:49:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.882+0000 7f89effff700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f8100570 0x7f89f8101d90 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f89e000e9c0 tx=0x7f89e000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.882+0000 7f89ef7fe700 1 -- 172.21.15.46:0/80761979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0004a30 con 0x7f89f8100570 2024-03-18T20:49:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.882+0000 7f89ef7fe700 1 -- 172.21.15.46:0/80761979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89e000cc80 con 0x7f89f8100570 2024-03-18T20:49:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.882+0000 7f89ef7fe700 1 -- 172.21.15.46:0/80761979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e00054e0 con 0x7f89f8100570 2024-03-18T20:49:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.882+0000 7f89fcfbb700 1 -- 172.21.15.46:0/80761979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f81a5c60 con 0x7f89f8100570 2024-03-18T20:49:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.883+0000 7f89fcfbb700 1 -- 172.21.15.46:0/80761979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f81a60f0 con 0x7f89f8100570 2024-03-18T20:49:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.887+0000 7f89ef7fe700 1 -- 172.21.15.46:0/80761979 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f89e0004b90 con 0x7f89f8100570 2024-03-18T20:49:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.888+0000 7f8a09150b80 1 Processor -- start 2024-03-18T20:49:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.888+0000 7f8a09150b80 1 -- start start 2024-03-18T20:49:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.889+0000 7f8a09150b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a67570 0x5564c5a67950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.889+0000 7f8a09150b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67e90 0x5564c5a7b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.889+0000 7f8a09150b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a7d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.889+0000 7f8a09150b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564c59c1930 con 0x5564c5a67e90 2024-03-18T20:49:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.889+0000 7f8a09150b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564c59bf2f0 con 0x5564c5a7b5c0 2024-03-18T20:49:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.889+0000 7f8a09150b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564c59babe0 con 0x5564c5a67570 2024-03-18T20:49:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.889+0000 7f89ef7fe700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f89e40a1310 0x7f89e40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.889+0000 7f89ef7fe700 1 -- 172.21.15.46:0/80761979 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f89e0014070 con 0x7f89f8100570 2024-03-18T20:49:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.889+0000 7f89f659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67e90 0x5564c5a7b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.889+0000 7f89f659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67e90 0x5564c5a7b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34994/0 (socket says 172.21.15.46:34994) 2024-03-18T20:49:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.889+0000 7f89f659c700 1 -- 172.21.15.46:0/4096865932 learned_addr learned my addr 172.21.15.46:0/4096865932 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.890+0000 7f89f6d9d700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f89e40a1310 0x7f89e40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.890+0000 7f89f6d9d700 1 --2- 172.21.15.46:0/4096865932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a67570 0x5564c5a67950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.890+0000 7f89effff700 1 --2- 172.21.15.46:0/4096865932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a7d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.890+0000 7f89f6d9d700 1 -- 172.21.15.46:0/4096865932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 msgr2=0x5564c5a7d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.890+0000 7f89f6d9d700 1 --2- 172.21.15.46:0/4096865932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a7d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.890+0000 7f89f6d9d700 1 -- 172.21.15.46:0/4096865932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67e90 msgr2=0x5564c5a7b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.890+0000 7f89f6d9d700 1 --2- 172.21.15.46:0/4096865932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67e90 0x5564c5a7b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.890+0000 7f89f6d9d700 1 -- 172.21.15.46:0/4096865932 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564c583c350 con 0x5564c5a67570 2024-03-18T20:49:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.890+0000 7f89effff700 1 --2- 172.21.15.46:0/4096865932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a7d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:49:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.890+0000 7f89f6d9d700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f89e40a1310 0x7f89e40a37d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f89f00093d0 tx=0x7f89f0007420 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.891+0000 7f89f6d9d700 1 --2- 172.21.15.46:0/4096865932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a67570 0x5564c5a67950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f89f8102830 tx=0x7f89f0036000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd214f0eb7195d1f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.893+0000 7f89ecff9700 1 -- 172.21.15.46:0/4096865932 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0036070 con 0x5564c5a67570 2024-03-18T20:49:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.893+0000 7f89ecff9700 1 -- 172.21.15.46:0/4096865932 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89f0007e90 con 0x5564c5a67570 2024-03-18T20:49:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.893+0000 7f89ecff9700 1 -- 172.21.15.46:0/4096865932 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0031a10 con 0x5564c5a67570 2024-03-18T20:49:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.894+0000 7f8a09150b80 1 -- 172.21.15.46:0/4096865932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a67570 msgr2=0x5564c5a67950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.894+0000 7f8a09150b80 1 --2- 172.21.15.46:0/4096865932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a67570 0x5564c5a67950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f89f8102830 tx=0x7f89f0036000 comp rx=0 tx=0).stop 2024-03-18T20:49:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.895+0000 7f8a09150b80 1 -- 172.21.15.46:0/4096865932 shutdown_connections 2024-03-18T20:49:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.895+0000 7f8a09150b80 1 --2- 172.21.15.46:0/4096865932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a7d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.895+0000 7f8a09150b80 1 --2- 172.21.15.46:0/4096865932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a67570 0x5564c5a67950 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.895+0000 7f8a09150b80 1 --2- 172.21.15.46:0/4096865932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67e90 0x5564c5a7b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.895+0000 7f8a09150b80 1 -- 172.21.15.46:0/4096865932 >> 172.21.15.46:0/4096865932 conn(0x5564c59de1c0 msgr2=0x5564c5a63a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.895+0000 7f8a09150b80 1 -- 172.21.15.46:0/4096865932 shutdown_connections 2024-03-18T20:49:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.895+0000 7f8a09150b80 1 -- 172.21.15.46:0/4096865932 wait complete. 2024-03-18T20:49:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.895+0000 7f8a09150b80 1 -- start start 2024-03-18T20:49:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.895+0000 7f8a09150b80 1 Processor -- start 2024-03-18T20:49:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.896+0000 7f8a09150b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a67570 0x5564c5a7af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.896+0000 7f8a09150b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67e90 0x5564c5a782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.896+0000 7f8a09150b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.896+0000 7f8a09150b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564c5956d20 con 0x5564c5a67e90 2024-03-18T20:49:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.896+0000 7f8a09150b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564c59569c0 con 0x5564c5a7b5c0 2024-03-18T20:49:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.896+0000 7f8a09150b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564c583e640 con 0x5564c5a67570 2024-03-18T20:49:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.896+0000 7f8a09150b80 10 client.0 _ll_register_callbacks cb 0x5564c57c2340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.896+0000 7f89f659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a67570 0x5564c5a7af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.896+0000 7f89f659c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a67570 0x5564c5a7af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43946/0 (socket says 172.21.15.46:43946) 2024-03-18T20:49:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.896+0000 7f89f659c700 1 -- 172.21.15.46:0/3539403868 learned_addr learned my addr 172.21.15.46:0/3539403868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.896+0000 7f89effff700 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67e90 0x5564c5a782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.896+0000 7f89f6d9d700 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f89effff700 1 -- 172.21.15.46:0/3539403868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a67570 msgr2=0x5564c5a7af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f89effff700 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a67570 0x5564c5a7af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f89effff700 1 -- 172.21.15.46:0/3539403868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 msgr2=0x5564c5a788e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f89effff700 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a788e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f89effff700 1 -- 172.21.15.46:0/3539403868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f81a60f0 con 0x5564c5a67e90 2024-03-18T20:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f89effff700 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67e90 0x5564c5a782b0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f89e00ca500 tx=0x7f89e000ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c0dbcf243bc602 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f89ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f89ddffb700 1 -- 172.21.15.46:0/3539403868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e009dcf0 con 0x5564c5a67e90 2024-03-18T20:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f89ddffb700 1 -- 172.21.15.46:0/3539403868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f89e00a2340 con 0x5564c5a67e90 2024-03-18T20:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f8a09150b80 1 -- 172.21.15.46:0/3539403868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564c5a5b6c0 con 0x5564c5a67e90 2024-03-18T20:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f89ddffb700 1 -- 172.21.15.46:0/3539403868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89e009a120 con 0x5564c5a67e90 2024-03-18T20:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f8a09150b80 10 client.20205 fetch_fsmap learned FSMap version 65 2024-03-18T20:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.897+0000 7f8a09150b80 1 -- 172.21.15.46:0/3539403868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x7f89f810d2f0 con 0x5564c5a67e90 2024-03-18T20:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.898+0000 7f89ddffb700 1 -- 172.21.15.46:0/3539403868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564c5a5b6c0 con 0x5564c5a67e90 2024-03-18T20:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.898+0000 7f89ddffb700 1 -- 172.21.15.46:0/3539403868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 65) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f89e00a1eb0 con 0x5564c5a67e90 2024-03-18T20:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.898+0000 7f8a09150b80 10 client.20205 fetch_fsmap finished waiting for FSMap version 65 2024-03-18T20:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.898+0000 7f8a09150b80 10 client.20205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.898+0000 7f8a09150b80 20 client.20205 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.898+0000 7f8a09150b80 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564c5a6e760 0x5564c5a70b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.898+0000 7f8a09150b80 4 client.20205 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.898+0000 7f8a09150b80 1 -- 172.21.15.46:0/3539403868 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564c59b36a0 con 0x5564c5a6e760 2024-03-18T20:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.899+0000 7f89f659c700 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564c5a6e760 0x5564c5a70b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.900+0000 7f89f659c700 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564c5a6e760 0x5564c5a70b80 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db27380f6b099444 server_cookie=25c5a4d28e7a0664 in_seq=0 out_seq=0 2024-03-18T20:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.900+0000 7f89ddffb700 10 client.20205 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.902+0000 7f89ddffb700 1 -- 172.21.15.46:0/3539403868 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5564c59b36a0 con 0x5564c5a6e760 2024-03-18T20:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.902+0000 7f89ddffb700 10 client.20205 handle_command_reply: tid=0 2024-03-18T20:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.902+0000 7f8a09150b80 1 client.20205 shutdown 2024-03-18T20:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.903+0000 7f8a09150b80 1 -- 172.21.15.46:0/3539403868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67e90 msgr2=0x5564c5a782b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.903+0000 7f8a09150b80 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67e90 0x5564c5a782b0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f89e00ca500 tx=0x7f89e000ce70 comp rx=0 tx=0).stop 2024-03-18T20:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.903+0000 7f8a09150b80 1 -- 172.21.15.46:0/3539403868 shutdown_connections 2024-03-18T20:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.903+0000 7f8a09150b80 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564c5a6e760 0x5564c5a70b80 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.903+0000 7f8a09150b80 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.903+0000 7f8a09150b80 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a67570 0x5564c5a7af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.903+0000 7f8a09150b80 1 --2- 172.21.15.46:0/3539403868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67e90 0x5564c5a782b0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.903+0000 7f8a09150b80 1 -- 172.21.15.46:0/3539403868 >> 172.21.15.46:0/3539403868 conn(0x5564c59de1c0 msgr2=0x5564c5a63650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.903+0000 7f89ddffb700 0 client.20205 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.903+0000 7f8a09150b80 1 -- 172.21.15.46:0/3539403868 shutdown_connections 2024-03-18T20:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.903+0000 7f8a09150b80 1 -- 172.21.15.46:0/3539403868 wait complete. 2024-03-18T20:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.904+0000 7f8a09150b80 20 client.20205 trim_cache size 0 max 16384 2024-03-18T20:49:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.924+0000 7f8a09150b80 1 Processor -- start 2024-03-18T20:49:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.924+0000 7f8a09150b80 1 -- start start 2024-03-18T20:49:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.924+0000 7f8a09150b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a67570 0x5564c5a6a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.924+0000 7f8a09150b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a7b5c0 0x5564c5a735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f8a09150b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a9c2d0 0x5564c5a73b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f8a09150b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564c5a68e20 con 0x5564c5a9c2d0 2024-03-18T20:49:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f8a09150b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564c5a64f70 con 0x5564c5a67570 2024-03-18T20:49:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f8a09150b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564c5a5dc10 con 0x5564c5a7b5c0 2024-03-18T20:49:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f89effff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a7b5c0 0x5564c5a735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f89f659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a67570 0x5564c5a6a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f89effff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a7b5c0 0x5564c5a735c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43960/0 (socket says 172.21.15.46:43960) 2024-03-18T20:49:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f89effff700 1 -- 172.21.15.46:0/3847583937 learned_addr learned my addr 172.21.15.46:0/3847583937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f89f659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a67570 0x5564c5a6a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:42804/0 (socket says 172.21.15.46:42804) 2024-03-18T20:49:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f89f6d9d700 1 --2- 172.21.15.46:0/3847583937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a9c2d0 0x5564c5a73b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f89effff700 1 -- 172.21.15.46:0/3847583937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a67570 msgr2=0x5564c5a6a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f89effff700 1 --2- 172.21.15.46:0/3847583937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a67570 0x5564c5a6a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f89effff700 1 -- 172.21.15.46:0/3847583937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a9c2d0 msgr2=0x5564c5a73b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f89effff700 1 --2- 172.21.15.46:0/3847583937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a9c2d0 0x5564c5a73b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f89effff700 1 -- 172.21.15.46:0/3847583937 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564c5a5f8f0 con 0x5564c5a7b5c0 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.925+0000 7f89effff700 1 --2- 172.21.15.46:0/3847583937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a7b5c0 0x5564c5a735c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f89e0005950 tx=0x7f89e009a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=143faf56712248ed server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.926+0000 7f89ed7fa700 1 -- 172.21.15.46:0/3847583937 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e00a2340 con 0x5564c5a7b5c0 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.926+0000 7f89ed7fa700 1 -- 172.21.15.46:0/3847583937 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89e009dcf0 con 0x5564c5a7b5c0 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.926+0000 7f89ed7fa700 1 -- 172.21.15.46:0/3847583937 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0099610 con 0x5564c5a7b5c0 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.927+0000 7f8a09150b80 1 -- 172.21.15.46:0/3847583937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a7b5c0 msgr2=0x5564c5a735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.927+0000 7f8a09150b80 1 --2- 172.21.15.46:0/3847583937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a7b5c0 0x5564c5a735c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f89e0005950 tx=0x7f89e009a040 comp rx=0 tx=0).stop 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.927+0000 7f8a09150b80 1 -- 172.21.15.46:0/3847583937 shutdown_connections 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.927+0000 7f8a09150b80 1 --2- 172.21.15.46:0/3847583937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a9c2d0 0x5564c5a73b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.927+0000 7f8a09150b80 1 --2- 172.21.15.46:0/3847583937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a67570 0x5564c5a6a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.927+0000 7f8a09150b80 1 --2- 172.21.15.46:0/3847583937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a7b5c0 0x5564c5a735c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.927+0000 7f8a09150b80 1 -- 172.21.15.46:0/3847583937 >> 172.21.15.46:0/3847583937 conn(0x5564c59de1c0 msgr2=0x5564c5a63500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.927+0000 7f8a09150b80 1 -- 172.21.15.46:0/3847583937 shutdown_connections 2024-03-18T20:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.927+0000 7f8a09150b80 1 -- 172.21.15.46:0/3847583937 wait complete. 2024-03-18T20:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.928+0000 7f8a09150b80 1 -- start start 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.928+0000 7f8a09150b80 1 Processor -- start 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.928+0000 7f8a09150b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67570 0x5564c5a821f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.928+0000 7f8a09150b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a74950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.928+0000 7f8a09150b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a9c2d0 0x5564c5a74e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.928+0000 7f8a09150b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564c5a75420 con 0x5564c5a67570 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.928+0000 7f8a09150b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564c5a75590 con 0x5564c5a7b5c0 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.928+0000 7f8a09150b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564c5a75750 con 0x5564c5a9c2d0 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.928+0000 7f8a09150b80 10 client.0 _ll_register_callbacks cb 0x7f89e0099820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.929+0000 7f89f6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a9c2d0 0x5564c5a74e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.929+0000 7f89f6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a9c2d0 0x5564c5a74e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43972/0 (socket says 172.21.15.46:43972) 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.929+0000 7f89f6d9d700 1 -- 172.21.15.46:0/2935391407 learned_addr learned my addr 172.21.15.46:0/2935391407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.929+0000 7f89f659c700 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67570 0x5564c5a821f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.929+0000 7f89effff700 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a74950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.929+0000 7f89f6d9d700 1 -- 172.21.15.46:0/2935391407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 msgr2=0x5564c5a74950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.929+0000 7f89f6d9d700 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a74950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.929+0000 7f89f6d9d700 1 -- 172.21.15.46:0/2935391407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67570 msgr2=0x5564c5a821f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.929+0000 7f89f6d9d700 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67570 0x5564c5a821f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.929+0000 7f89f6d9d700 1 -- 172.21.15.46:0/2935391407 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f810d2f0 con 0x5564c5a9c2d0 2024-03-18T20:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.930+0000 7f89f6d9d700 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a9c2d0 0x5564c5a74e90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f89f000b6c0 tx=0x7f89f0007dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c118fef39bb9896 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.930+0000 7f89ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.930+0000 7f89ddffb700 1 -- 172.21.15.46:0/2935391407 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f00319f0 con 0x5564c5a9c2d0 2024-03-18T20:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.930+0000 7f89ddffb700 1 -- 172.21.15.46:0/2935391407 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f89f0009440 con 0x5564c5a9c2d0 2024-03-18T20:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.931+0000 7f8a09150b80 1 -- 172.21.15.46:0/2935391407 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89f0007e90 con 0x5564c5a9c2d0 2024-03-18T20:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.932+0000 7f89ddffb700 1 -- 172.21.15.46:0/2935391407 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89f0031b50 con 0x5564c5a9c2d0 2024-03-18T20:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.932+0000 7f8a09150b80 10 client.29848 fetch_fsmap learned FSMap version 65 2024-03-18T20:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.932+0000 7f8a09150b80 1 -- 172.21.15.46:0/2935391407 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x5564c57e8b00 con 0x5564c5a9c2d0 2024-03-18T20:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.932+0000 7f89ddffb700 1 -- 172.21.15.46:0/2935391407 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89f0007e90 con 0x5564c5a9c2d0 2024-03-18T20:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.933+0000 7f89ddffb700 1 -- 172.21.15.46:0/2935391407 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 65) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f89f003cd20 con 0x5564c5a9c2d0 2024-03-18T20:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.933+0000 7f8a09150b80 10 client.29848 fetch_fsmap finished waiting for FSMap version 65 2024-03-18T20:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.933+0000 7f8a09150b80 10 client.29848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.933+0000 7f8a09150b80 20 client.29848 populate_metadata read hostname 'smithi046' 2024-03-18T20:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.933+0000 7f8a09150b80 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564c5ae8650 0x5564c5aeaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.933+0000 7f8a09150b80 4 client.29848 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:49:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.933+0000 7f8a09150b80 1 -- 172.21.15.46:0/2935391407 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5564c5809c90 con 0x5564c5ae8650 2024-03-18T20:49:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.934+0000 7f89f659c700 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564c5ae8650 0x5564c5aeaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:49:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.934+0000 7f89ddffb700 10 client.29848 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:50.934+0000 7f89f659c700 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564c5ae8650 0x5564c5aeaa70 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adf270e2d7db8a3c server_cookie=8f01c0753bffdf0d in_seq=0 out_seq=0 2024-03-18T20:49:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.073+0000 7f89ddffb700 1 -- 172.21.15.46:0/2935391407 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5564c5809c90 con 0x5564c5ae8650 2024-03-18T20:49:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.073+0000 7f89ddffb700 10 client.29848 handle_command_reply: tid=0 2024-03-18T20:49:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.073+0000 7f8a09150b80 1 client.29848 shutdown 2024-03-18T20:49:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.073+0000 7f8a09150b80 1 -- 172.21.15.46:0/2935391407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a9c2d0 msgr2=0x5564c5a74e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.073+0000 7f8a09150b80 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a9c2d0 0x5564c5a74e90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f89f000b6c0 tx=0x7f89f0007dd0 comp rx=0 tx=0).stop 2024-03-18T20:49:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.073+0000 7f8a09150b80 1 -- 172.21.15.46:0/2935391407 shutdown_connections 2024-03-18T20:49:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.073+0000 7f8a09150b80 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5564c5ae8650 0x5564c5aeaa70 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.073+0000 7f8a09150b80 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564c5a67570 0x5564c5a821f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.073+0000 7f8a09150b80 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5564c5a9c2d0 0x5564c5a74e90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.073+0000 7f8a09150b80 1 --2- 172.21.15.46:0/2935391407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564c5a7b5c0 0x5564c5a74950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.073+0000 7f8a09150b80 1 -- 172.21.15.46:0/2935391407 >> 172.21.15.46:0/2935391407 conn(0x5564c59de1c0 msgr2=0x5564c5a63500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.073+0000 7f89ddffb700 0 client.29848 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:49:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.074+0000 7f8a09150b80 1 -- 172.21.15.46:0/2935391407 shutdown_connections 2024-03-18T20:49:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.074+0000 7f8a09150b80 1 -- 172.21.15.46:0/2935391407 wait complete. 2024-03-18T20:49:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.074+0000 7f8a09150b80 20 client.29848 trim_cache size 0 max 16384 2024-03-18T20:49:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:49:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:49:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:49:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:49:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.075+0000 7f89fcfbb700 1 -- 172.21.15.46:0/80761979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f89e40a1310 msgr2=0x7f89e40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.075+0000 7f89fcfbb700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f89e40a1310 0x7f89e40a37d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f89f00093d0 tx=0x7f89f0007420 comp rx=0 tx=0).stop 2024-03-18T20:49:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.075+0000 7f89fcfbb700 1 -- 172.21.15.46:0/80761979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f8100570 msgr2=0x7f89f8101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:49:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.075+0000 7f89fcfbb700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f8100570 0x7f89f8101d90 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f89e000e9c0 tx=0x7f89e000ecd0 comp rx=0 tx=0).stop 2024-03-18T20:49:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.075+0000 7f89fcfbb700 1 -- 172.21.15.46:0/80761979 shutdown_connections 2024-03-18T20:49:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.075+0000 7f89fcfbb700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f89e40a1310 0x7f89e40a37d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.075+0000 7f89fcfbb700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89f8105c60 0x7f89f81022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.075+0000 7f89fcfbb700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89f80ffbb0 0x7f89f8101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.075+0000 7f89fcfbb700 1 --2- 172.21.15.46:0/80761979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f8100570 0x7f89f8101d90 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:49:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.075+0000 7f89fcfbb700 1 -- 172.21.15.46:0/80761979 >> 172.21.15.46:0/80761979 conn(0x7f89f8076d50 msgr2=0x7f89f80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:49:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.075+0000 7f89fcfbb700 1 -- 172.21.15.46:0/80761979 shutdown_connections 2024-03-18T20:49:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:49:51.075+0000 7f89fcfbb700 1 -- 172.21.15.46:0/80761979 wait complete. 2024-03-18T20:49:51.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 28046 62.37 MB/sec warmup 27 sec latency 98.604 ms 2024-03-18T20:49:52.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 29179 62.46 MB/sec warmup 28 sec latency 43.636 ms 2024-03-18T20:49:52.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:52 smithi073 ceph-mon[47384]: pgmap v430: 97 pgs: 97 active+clean; 518 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 37 MiB/s wr, 1.12k op/s 2024-03-18T20:49:52.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:52 smithi155 ceph-mon[48344]: pgmap v430: 97 pgs: 97 active+clean; 518 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 37 MiB/s wr, 1.12k op/s 2024-03-18T20:49:52.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:52 smithi046 ceph-mon[43495]: pgmap v430: 97 pgs: 97 active+clean; 518 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 37 MiB/s wr, 1.12k op/s 2024-03-18T20:49:53.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 30315 62.40 MB/sec warmup 29 sec latency 48.029 ms 2024-03-18T20:49:54.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 31481 62.51 MB/sec warmup 30 sec latency 42.855 ms 2024-03-18T20:49:54.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:54 smithi073 ceph-mon[47384]: pgmap v431: 97 pgs: 97 active+clean; 532 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 40 MiB/s wr, 1.53k op/s 2024-03-18T20:49:54.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:54 smithi155 ceph-mon[48344]: pgmap v431: 97 pgs: 97 active+clean; 532 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 40 MiB/s wr, 1.53k op/s 2024-03-18T20:49:54.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:54 smithi046 ceph-mon[43495]: pgmap v431: 97 pgs: 97 active+clean; 532 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 40 MiB/s wr, 1.53k op/s 2024-03-18T20:49:55.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 32310 61.95 MB/sec warmup 31 sec latency 41.203 ms 2024-03-18T20:49:56.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 33355 61.26 MB/sec warmup 32 sec latency 75.939 ms 2024-03-18T20:49:56.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:56 smithi073 ceph-mon[47384]: pgmap v432: 97 pgs: 97 active+clean; 532 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 36 MiB/s wr, 1.18k op/s 2024-03-18T20:49:56.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:56 smithi155 ceph-mon[48344]: pgmap v432: 97 pgs: 97 active+clean; 532 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 36 MiB/s wr, 1.18k op/s 2024-03-18T20:49:56.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:56 smithi046 ceph-mon[43495]: pgmap v432: 97 pgs: 97 active+clean; 532 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 36 MiB/s wr, 1.18k op/s 2024-03-18T20:49:57.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 34746 62.01 MB/sec warmup 33 sec latency 33.999 ms 2024-03-18T20:49:58.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 35720 62.06 MB/sec warmup 34 sec latency 50.548 ms 2024-03-18T20:49:58.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:49:58 smithi073 ceph-mon[47384]: pgmap v433: 97 pgs: 97 active+clean; 532 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 36 MiB/s wr, 1.18k op/s 2024-03-18T20:49:58.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:49:58 smithi155 ceph-mon[48344]: pgmap v433: 97 pgs: 97 active+clean; 532 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 36 MiB/s wr, 1.18k op/s 2024-03-18T20:49:58.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:49:58 smithi046 ceph-mon[43495]: pgmap v433: 97 pgs: 97 active+clean; 532 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 36 MiB/s wr, 1.18k op/s 2024-03-18T20:49:59.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 36639 61.31 MB/sec warmup 35 sec latency 45.947 ms 2024-03-18T20:50:00.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a[43472]: 2024-03-18T20:49:59.999+0000 7fb26e3e2700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-03-18T20:50:00.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a[43472]: 2024-03-18T20:49:59.999+0000 7fb26e3e2700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-03-18T20:50:00.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a[43472]: 2024-03-18T20:49:59.999+0000 7fb26e3e2700 -1 log_channel(cluster) log [ERR] : mds.k(mds.4): Metadata damage detected 2024-03-18T20:50:00.296 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a[43472]: 2024-03-18T20:49:59.999+0000 7fb26e3e2700 -1 log_channel(cluster) log [ERR] : mds.i(mds.2): Metadata damage detected 2024-03-18T20:50:00.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 37967 61.45 MB/sec warmup 36 sec latency 31.231 ms 2024-03-18T20:50:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:00 smithi073 ceph-mon[47384]: pgmap v434: 97 pgs: 97 active+clean; 562 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 39 MiB/s wr, 1.51k op/s 2024-03-18T20:50:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:00 smithi073 ceph-mon[47384]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-03-18T20:50:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:00 smithi073 ceph-mon[47384]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-03-18T20:50:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:00 smithi073 ceph-mon[47384]: mds.k(mds.4): Metadata damage detected 2024-03-18T20:50:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:00 smithi073 ceph-mon[47384]: mds.i(mds.2): Metadata damage detected 2024-03-18T20:50:00.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:00 smithi155 ceph-mon[48344]: pgmap v434: 97 pgs: 97 active+clean; 562 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 39 MiB/s wr, 1.51k op/s 2024-03-18T20:50:00.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:00 smithi155 ceph-mon[48344]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-03-18T20:50:00.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:00 smithi155 ceph-mon[48344]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-03-18T20:50:00.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:00 smithi155 ceph-mon[48344]: mds.k(mds.4): Metadata damage detected 2024-03-18T20:50:00.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:00 smithi155 ceph-mon[48344]: mds.i(mds.2): Metadata damage detected 2024-03-18T20:50:00.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:00 smithi046 ceph-mon[43495]: pgmap v434: 97 pgs: 97 active+clean; 562 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 39 MiB/s wr, 1.51k op/s 2024-03-18T20:50:00.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:00 smithi046 ceph-mon[43495]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2024-03-18T20:50:00.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:00 smithi046 ceph-mon[43495]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2024-03-18T20:50:00.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:00 smithi046 ceph-mon[43495]: mds.k(mds.4): Metadata damage detected 2024-03-18T20:50:00.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:00 smithi046 ceph-mon[43495]: mds.i(mds.2): Metadata damage detected 2024-03-18T20:50:01.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 38542 60.72 MB/sec warmup 37 sec latency 85.847 ms 2024-03-18T20:50:02.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 39418 60.56 MB/sec warmup 38 sec latency 78.328 ms 2024-03-18T20:50:02.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:02 smithi073 ceph-mon[47384]: pgmap v435: 97 pgs: 97 active+clean; 562 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.2 MiB/s wr, 736 op/s 2024-03-18T20:50:02.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:02 smithi155 ceph-mon[48344]: pgmap v435: 97 pgs: 97 active+clean; 562 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.2 MiB/s wr, 736 op/s 2024-03-18T20:50:02.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:02 smithi046 ceph-mon[43495]: pgmap v435: 97 pgs: 97 active+clean; 562 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.2 MiB/s wr, 736 op/s 2024-03-18T20:50:03.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 40289 60.12 MB/sec warmup 39 sec latency 62.416 ms 2024-03-18T20:50:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.260+0000 7f9c54a8b700 1 -- 172.21.15.46:0/2186194462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50105c40 msgr2=0x7f9c501036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.260+0000 7f9c54a8b700 1 --2- 172.21.15.46:0/2186194462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50105c40 0x7f9c501036c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009990 tx=0x7f9c44009ca0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.262+0000 7f9c54a8b700 1 -- 172.21.15.46:0/2186194462 shutdown_connections 2024-03-18T20:50:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.262+0000 7f9c54a8b700 1 --2- 172.21.15.46:0/2186194462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c500680e0 0x7f9c50068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.262+0000 7f9c54a8b700 1 --2- 172.21.15.46:0/2186194462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50105c40 0x7f9c501036c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.262+0000 7f9c54a8b700 1 --2- 172.21.15.46:0/2186194462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c50103c00 0x7f9c50104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.262+0000 7f9c54a8b700 1 -- 172.21.15.46:0/2186194462 >> 172.21.15.46:0/2186194462 conn(0x7f9c50076d50 msgr2=0x7f9c50077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.265+0000 7f9c54a8b700 1 -- 172.21.15.46:0/2186194462 shutdown_connections 2024-03-18T20:50:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.268+0000 7f9c54a8b700 1 -- 172.21.15.46:0/2186194462 wait complete. 2024-03-18T20:50:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.269+0000 7f9c54a8b700 1 Processor -- start 2024-03-18T20:50:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.269+0000 7f9c54a8b700 1 -- start start 2024-03-18T20:50:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.270+0000 7f9c54a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c500680e0 0x7f9c5019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.270+0000 7f9c54a8b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c50103c00 0x7f9c5019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.270+0000 7f9c54a8b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50105c40 0x7f9c501a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.270+0000 7f9c54a8b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c501a2660 con 0x7f9c500680e0 2024-03-18T20:50:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.270+0000 7f9c54a8b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c501a27d0 con 0x7f9c50105c40 2024-03-18T20:50:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.270+0000 7f9c54a8b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c501a2990 con 0x7f9c50103c00 2024-03-18T20:50:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.270+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c50103c00 0x7f9c5019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.270+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c50103c00 0x7f9c5019dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:44938/0 (socket says 172.21.15.46:44938) 2024-03-18T20:50:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.270+0000 7f9c4dd9b700 1 -- 172.21.15.46:0/3331327122 learned_addr learned my addr 172.21.15.46:0/3331327122 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.270+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c500680e0 0x7f9c5019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.270+0000 7f9c4ed9d700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50105c40 0x7f9c501a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.271+0000 7f9c4e59c700 1 -- 172.21.15.46:0/3331327122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c50103c00 msgr2=0x7f9c5019dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.271+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c50103c00 0x7f9c5019dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.271+0000 7f9c4e59c700 1 -- 172.21.15.46:0/3331327122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50105c40 msgr2=0x7f9c501a2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.271+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50105c40 0x7f9c501a2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.271+0000 7f9c4e59c700 1 -- 172.21.15.46:0/3331327122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c440095f0 con 0x7f9c500680e0 2024-03-18T20:50:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.271+0000 7f9c4ed9d700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50105c40 0x7f9c501a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:50:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.271+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c500680e0 0x7f9c5019d740 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f9c44000c00 tx=0x7f9c44009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.271+0000 7f9c3f7fe700 1 -- 172.21.15.46:0/3331327122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4401d070 con 0x7f9c500680e0 2024-03-18T20:50:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.271+0000 7f9c3f7fe700 1 -- 172.21.15.46:0/3331327122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c44021410 con 0x7f9c500680e0 2024-03-18T20:50:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.271+0000 7f9c3f7fe700 1 -- 172.21.15.46:0/3331327122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44017650 con 0x7f9c500680e0 2024-03-18T20:50:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.272+0000 7f9c54a8b700 1 -- 172.21.15.46:0/3331327122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c501a2c10 con 0x7f9c500680e0 2024-03-18T20:50:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.272+0000 7f9c54a8b700 1 -- 172.21.15.46:0/3331327122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c501a2e90 con 0x7f9c500680e0 2024-03-18T20:50:04.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.277+0000 7f9c60c20b80 1 Processor -- start 2024-03-18T20:50:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.278+0000 7f9c3f7fe700 1 -- 172.21.15.46:0/3331327122 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9c44021580 con 0x7f9c500680e0 2024-03-18T20:50:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.279+0000 7f9c60c20b80 1 -- start start 2024-03-18T20:50:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.279+0000 7f9c60c20b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a627b570 0x5600a627b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.279+0000 7f9c60c20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a6286870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.279+0000 7f9c60c20b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a6286db0 0x5600a62891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.279+0000 7f9c60c20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a61d5930 con 0x5600a627be90 2024-03-18T20:50:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.279+0000 7f9c60c20b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a61d32f0 con 0x5600a627b570 2024-03-18T20:50:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.279+0000 7f9c60c20b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a61cebe0 con 0x5600a6286db0 2024-03-18T20:50:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.280+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a6286db0 0x5600a62891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.280+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a6286db0 0x5600a62891a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:44946/0 (socket says 172.21.15.46:44946) 2024-03-18T20:50:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.280+0000 7f9c4dd9b700 1 -- 172.21.15.46:0/3863777290 learned_addr learned my addr 172.21.15.46:0/3863777290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.280+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/3863777290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a6286870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.280+0000 7f9c3f7fe700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9c380a1310 0x7f9c380a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.280+0000 7f9c4ed9d700 1 --2- 172.21.15.46:0/3863777290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a627b570 0x5600a627b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.280+0000 7f9c4ed9d700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9c380a1310 0x7f9c380a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.280+0000 7f9c3f7fe700 1 -- 172.21.15.46:0/3331327122 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f9c440cddc0 con 0x7f9c500680e0 2024-03-18T20:50:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.281+0000 7f9c4ed9d700 1 -- 172.21.15.46:0/3863777290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a6286db0 msgr2=0x5600a62891a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.281+0000 7f9c4ed9d700 1 --2- 172.21.15.46:0/3863777290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a6286db0 0x5600a62891a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.281+0000 7f9c4ed9d700 1 -- 172.21.15.46:0/3863777290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 msgr2=0x5600a6286870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.281+0000 7f9c4ed9d700 1 --2- 172.21.15.46:0/3863777290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a6286870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.281+0000 7f9c4ed9d700 1 -- 172.21.15.46:0/3863777290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a6050350 con 0x5600a627b570 2024-03-18T20:50:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.281+0000 7f9c4ed9d700 1 --2- 172.21.15.46:0/3863777290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a627b570 0x5600a627b950 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009ec0 tx=0x7f9c48008c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf74aba6aba6f5e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.281+0000 7f9c3cff9700 1 -- 172.21.15.46:0/3863777290 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4800ee70 con 0x5600a627b570 2024-03-18T20:50:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.282+0000 7f9c4ed9d700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9c380a1310 0x7f9c380a37d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9c48008fd0 tx=0x7f9c4801e000 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.282+0000 7f9c3cff9700 1 -- 172.21.15.46:0/3863777290 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4800f040 con 0x5600a627b570 2024-03-18T20:50:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.282+0000 7f9c3cff9700 1 -- 172.21.15.46:0/3863777290 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48022c80 con 0x5600a627b570 2024-03-18T20:50:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.283+0000 7f9c60c20b80 1 -- 172.21.15.46:0/3863777290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a627b570 msgr2=0x5600a627b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.283+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/3863777290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a627b570 0x5600a627b950 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009ec0 tx=0x7f9c48008c80 comp rx=0 tx=0).stop 2024-03-18T20:50:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.283+0000 7f9c60c20b80 1 -- 172.21.15.46:0/3863777290 shutdown_connections 2024-03-18T20:50:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.283+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/3863777290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a6286db0 0x5600a62891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.283+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/3863777290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a627b570 0x5600a627b950 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.283+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/3863777290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a6286870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.283+0000 7f9c60c20b80 1 -- 172.21.15.46:0/3863777290 >> 172.21.15.46:0/3863777290 conn(0x5600a61f21c0 msgr2=0x5600a6277a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.284+0000 7f9c60c20b80 1 -- 172.21.15.46:0/3863777290 shutdown_connections 2024-03-18T20:50:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.285+0000 7f9c60c20b80 1 -- 172.21.15.46:0/3863777290 wait complete. 2024-03-18T20:50:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.285+0000 7f9c60c20b80 1 -- start start 2024-03-18T20:50:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.285+0000 7f9c60c20b80 1 Processor -- start 2024-03-18T20:50:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.285+0000 7f9c60c20b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a627b570 0x5600a6283b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.285+0000 7f9c60c20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a62840c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.285+0000 7f9c60c20b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a6286db0 0x5600a6284600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.285+0000 7f9c60c20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a616ad20 con 0x5600a627be90 2024-03-18T20:50:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.285+0000 7f9c60c20b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a616a9c0 con 0x5600a627b570 2024-03-18T20:50:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.285+0000 7f9c60c20b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a6052640 con 0x5600a6286db0 2024-03-18T20:50:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.285+0000 7f9c60c20b80 10 client.0 _ll_register_callbacks cb 0x5600a5fd6340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:50:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.286+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a62840c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.286+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a62840c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55380/0 (socket says 172.21.15.46:55380) 2024-03-18T20:50:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.286+0000 7f9c4dd9b700 1 -- 172.21.15.46:0/167248281 learned_addr learned my addr 172.21.15.46:0/167248281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.286+0000 7f9c4ed9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a6286db0 0x5600a6284600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.286+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/167248281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a627b570 0x5600a6283b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.286+0000 7f9c4e59c700 1 -- 172.21.15.46:0/167248281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a6286db0 msgr2=0x5600a6284600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.286+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/167248281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a6286db0 0x5600a6284600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.286+0000 7f9c4e59c700 1 -- 172.21.15.46:0/167248281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 msgr2=0x5600a62840c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.286+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/167248281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a62840c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.286+0000 7f9c4e59c700 1 -- 172.21.15.46:0/167248281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c48008600 con 0x5600a627b570 2024-03-18T20:50:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.287+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/167248281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a627b570 0x5600a6283b80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c440cb4c0 tx=0x7f9c44004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc5767e04b6f08d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.288+0000 7f9c35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.288+0000 7f9c35ffb700 1 -- 172.21.15.46:0/167248281 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44009290 con 0x5600a627b570 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.288+0000 7f9c35ffb700 1 -- 172.21.15.46:0/167248281 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f9c4409af10 con 0x5600a627b570 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.288+0000 7f9c60c20b80 1 -- 172.21.15.46:0/167248281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600a626f6c0 con 0x5600a627b570 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.288+0000 7f9c35ffb700 1 -- 172.21.15.46:0/167248281 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c4409a870 con 0x5600a627b570 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.289+0000 7f9c60c20b80 10 client.29801 fetch_fsmap learned FSMap version 65 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.289+0000 7f9c60c20b80 1 -- 172.21.15.46:0/167248281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x5600a5ffcb00 con 0x5600a627b570 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.290+0000 7f9c35ffb700 1 -- 172.21.15.46:0/167248281 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600a626f6c0 con 0x5600a627b570 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.290+0000 7f9c35ffb700 1 -- 172.21.15.46:0/167248281 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f9c440a2e70 con 0x5600a627b570 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.290+0000 7f9c60c20b80 10 client.29801 fetch_fsmap finished waiting for FSMap version 65 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.290+0000 7f9c60c20b80 10 client.29801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.290+0000 7f9c60c20b80 20 client.29801 populate_metadata read hostname 'smithi046' 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.290+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/167248281 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5600a6296760 0x5600a6298b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.290+0000 7f9c60c20b80 4 client.29801 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:50:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.290+0000 7f9c60c20b80 1 -- 172.21.15.46:0/167248281 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600a61c76a0 con 0x5600a6296760 2024-03-18T20:50:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.290+0000 7f9c4dd9b700 1 --2- 172.21.15.46:0/167248281 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5600a6296760 0x5600a6298b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.291+0000 7f9c35ffb700 10 client.29801 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.291+0000 7f9c4dd9b700 1 --2- 172.21.15.46:0/167248281 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5600a6296760 0x5600a6298b80 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53f3ba03fce2b98 server_cookie=17ff221cfe350da6 in_seq=0 out_seq=0 2024-03-18T20:50:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.292+0000 7f9c35ffb700 1 -- 172.21.15.46:0/167248281 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5600a61c76a0 con 0x5600a6296760 2024-03-18T20:50:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.292+0000 7f9c35ffb700 10 client.29801 handle_command_reply: tid=0 2024-03-18T20:50:04.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.293+0000 7f9c60c20b80 1 client.29801 shutdown 2024-03-18T20:50:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.293+0000 7f9c60c20b80 1 -- 172.21.15.46:0/167248281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a627b570 msgr2=0x5600a6283b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.293+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/167248281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a627b570 0x5600a6283b80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9c440cb4c0 tx=0x7f9c44004730 comp rx=0 tx=0).stop 2024-03-18T20:50:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.293+0000 7f9c60c20b80 1 -- 172.21.15.46:0/167248281 shutdown_connections 2024-03-18T20:50:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.293+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/167248281 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5600a6296760 0x5600a6298b80 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.293+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/167248281 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a6286db0 0x5600a6284600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.293+0000 7f9c35ffb700 0 client.29801 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.293+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/167248281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a627b570 0x5600a6283b80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.293+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/167248281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a62840c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.293+0000 7f9c60c20b80 1 -- 172.21.15.46:0/167248281 >> 172.21.15.46:0/167248281 conn(0x5600a61f21c0 msgr2=0x5600a6277650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.293+0000 7f9c60c20b80 1 -- 172.21.15.46:0/167248281 shutdown_connections 2024-03-18T20:50:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.293+0000 7f9c60c20b80 1 -- 172.21.15.46:0/167248281 wait complete. 2024-03-18T20:50:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.294+0000 7f9c60c20b80 20 client.29801 trim_cache size 0 max 16384 2024-03-18T20:50:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.319+0000 7f9c60c20b80 1 Processor -- start 2024-03-18T20:50:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.319+0000 7f9c60c20b80 1 -- start start 2024-03-18T20:50:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c60c20b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627b570 0x5600a62891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c60c20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a62834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c60c20b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a6286db0 0x5600a627eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c60c20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a62866f0 con 0x5600a627be90 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c60c20b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a62741d0 con 0x5600a6286db0 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c60c20b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a627d130 con 0x5600a627b570 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a62834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a62834a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55384/0 (socket says 172.21.15.46:55384) 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c4dd9b700 1 -- 172.21.15.46:0/1493611897 learned_addr learned my addr 172.21.15.46:0/1493611897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/1493611897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627b570 0x5600a62891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c4ed9d700 1 --2- 172.21.15.46:0/1493611897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a6286db0 0x5600a627eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c4dd9b700 1 -- 172.21.15.46:0/1493611897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627b570 msgr2=0x5600a62891a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c4dd9b700 1 --2- 172.21.15.46:0/1493611897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627b570 0x5600a62891a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c4dd9b700 1 -- 172.21.15.46:0/1493611897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a6286db0 msgr2=0x5600a627eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c4dd9b700 1 --2- 172.21.15.46:0/1493611897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a6286db0 0x5600a627eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c4dd9b700 1 -- 172.21.15.46:0/1493611897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a6194e40 con 0x5600a627be90 2024-03-18T20:50:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c4ed9d700 1 --2- 172.21.15.46:0/1493611897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a6286db0 0x5600a627eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:50:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.320+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/1493611897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627b570 0x5600a62891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:50:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.321+0000 7f9c4dd9b700 1 --2- 172.21.15.46:0/1493611897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a62834a0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f9c400076a0 tx=0x7f9c4001ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c8ece0dd4c5682 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.321+0000 7f9c3d7fa700 1 -- 172.21.15.46:0/1493611897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40017040 con 0x5600a627be90 2024-03-18T20:50:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.321+0000 7f9c3d7fa700 1 -- 172.21.15.46:0/1493611897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4001ce90 con 0x5600a627be90 2024-03-18T20:50:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.321+0000 7f9c3d7fa700 1 -- 172.21.15.46:0/1493611897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40013a20 con 0x5600a627be90 2024-03-18T20:50:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.322+0000 7f9c60c20b80 1 -- 172.21.15.46:0/1493611897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 msgr2=0x5600a62834a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.322+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/1493611897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a62834a0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f9c400076a0 tx=0x7f9c4001ce20 comp rx=0 tx=0).stop 2024-03-18T20:50:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.322+0000 7f9c60c20b80 1 -- 172.21.15.46:0/1493611897 shutdown_connections 2024-03-18T20:50:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.322+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/1493611897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a6286db0 0x5600a627eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.322+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/1493611897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627b570 0x5600a62891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.322+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/1493611897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627be90 0x5600a62834a0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.322+0000 7f9c60c20b80 1 -- 172.21.15.46:0/1493611897 >> 172.21.15.46:0/1493611897 conn(0x5600a61f21c0 msgr2=0x5600a6276bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.322+0000 7f9c60c20b80 1 -- 172.21.15.46:0/1493611897 shutdown_connections 2024-03-18T20:50:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.322+0000 7f9c60c20b80 1 -- 172.21.15.46:0/1493611897 wait complete. 2024-03-18T20:50:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.322+0000 7f9c60c20b80 1 -- start start 2024-03-18T20:50:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.322+0000 7f9c60c20b80 1 Processor -- start 2024-03-18T20:50:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.323+0000 7f9c60c20b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627b570 0x5600a628bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.323+0000 7f9c60c20b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627be90 0x5600a628c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.323+0000 7f9c60c20b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a6286db0 0x5600a62e9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.323+0000 7f9c60c20b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a628c7d0 con 0x5600a627b570 2024-03-18T20:50:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.323+0000 7f9c60c20b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a62e9a90 con 0x5600a6286db0 2024-03-18T20:50:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.323+0000 7f9c60c20b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a62e9bd0 con 0x5600a627be90 2024-03-18T20:50:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.323+0000 7f9c60c20b80 10 client.0 _ll_register_callbacks cb 0x5600a5fd6340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:50:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.323+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627be90 0x5600a628c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.323+0000 7f9c4dd9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627be90 0x5600a628c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:44976/0 (socket says 172.21.15.46:44976) 2024-03-18T20:50:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.323+0000 7f9c4dd9b700 1 -- 172.21.15.46:0/2446264949 learned_addr learned my addr 172.21.15.46:0/2446264949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.323+0000 7f9c4ed9d700 1 --2- 172.21.15.46:0/2446264949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a6286db0 0x5600a62e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.324+0000 7f9c4dd9b700 1 -- 172.21.15.46:0/2446264949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a6286db0 msgr2=0x5600a62e9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.324+0000 7f9c4dd9b700 1 --2- 172.21.15.46:0/2446264949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a6286db0 0x5600a62e9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.324+0000 7f9c4dd9b700 1 -- 172.21.15.46:0/2446264949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627b570 msgr2=0x5600a628bbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:50:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.324+0000 7f9c4dd9b700 1 --2- 172.21.15.46:0/2446264949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627b570 0x5600a628bbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.324+0000 7f9c4dd9b700 1 -- 172.21.15.46:0/2446264949 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600a5ffcb00 con 0x5600a627be90 2024-03-18T20:50:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.325+0000 7f9c4dd9b700 1 --2- 172.21.15.46:0/2446264949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627be90 0x5600a628c0f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9c40007dc0 tx=0x7f9c40013330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efc459b348ac9077 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.325+0000 7f9c35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:50:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.326+0000 7f9c35ffb700 1 -- 172.21.15.46:0/2446264949 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40017040 con 0x5600a627be90 2024-03-18T20:50:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.326+0000 7f9c35ffb700 1 -- 172.21.15.46:0/2446264949 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f9c4001c530 con 0x5600a627be90 2024-03-18T20:50:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.326+0000 7f9c60c20b80 1 -- 172.21.15.46:0/2446264949 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c4800f040 con 0x5600a627be90 2024-03-18T20:50:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.326+0000 7f9c35ffb700 1 -- 172.21.15.46:0/2446264949 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c400139d0 con 0x5600a627be90 2024-03-18T20:50:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.326+0000 7f9c60c20b80 10 client.29866 fetch_fsmap learned FSMap version 65 2024-03-18T20:50:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.326+0000 7f9c60c20b80 1 -- 172.21.15.46:0/2446264949 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x7f9c4001c900 con 0x5600a627be90 2024-03-18T20:50:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.326+0000 7f9c35ffb700 1 -- 172.21.15.46:0/2446264949 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4800f040 con 0x5600a627be90 2024-03-18T20:50:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.326+0000 7f9c35ffb700 1 -- 172.21.15.46:0/2446264949 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 65) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f9c40011440 con 0x5600a627be90 2024-03-18T20:50:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.327+0000 7f9c60c20b80 10 client.29866 fetch_fsmap finished waiting for FSMap version 65 2024-03-18T20:50:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.327+0000 7f9c60c20b80 10 client.29866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:50:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.327+0000 7f9c60c20b80 20 client.29866 populate_metadata read hostname 'smithi046' 2024-03-18T20:50:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.327+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/2446264949 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5600a62eb350 0x5600a62ed770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.327+0000 7f9c60c20b80 4 client.29866 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:50:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.327+0000 7f9c60c20b80 1 -- 172.21.15.46:0/2446264949 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5600a601dc90 con 0x5600a62eb350 2024-03-18T20:50:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.327+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/2446264949 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5600a62eb350 0x5600a62ed770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.328+0000 7f9c4e59c700 1 --2- 172.21.15.46:0/2446264949 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5600a62eb350 0x5600a62ed770 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bade3cf9bd745b23 server_cookie=76805c1df03f7ae in_seq=0 out_seq=0 2024-03-18T20:50:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.328+0000 7f9c35ffb700 10 client.29866 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:04.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 41351 59.85 MB/sec warmup 40 sec latency 51.152 ms 2024-03-18T20:50:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c35ffb700 1 -- 172.21.15.46:0/2446264949 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5600a601dc90 con 0x5600a62eb350 2024-03-18T20:50:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c35ffb700 10 client.29866 handle_command_reply: tid=0 2024-03-18T20:50:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c60c20b80 1 client.29866 shutdown 2024-03-18T20:50:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c60c20b80 1 -- 172.21.15.46:0/2446264949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627be90 msgr2=0x5600a628c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/2446264949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627be90 0x5600a628c0f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9c40007dc0 tx=0x7f9c40013330 comp rx=0 tx=0).stop 2024-03-18T20:50:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c60c20b80 1 -- 172.21.15.46:0/2446264949 shutdown_connections 2024-03-18T20:50:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/2446264949 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5600a62eb350 0x5600a62ed770 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/2446264949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a627b570 0x5600a628bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/2446264949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5600a6286db0 0x5600a62e9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c60c20b80 1 --2- 172.21.15.46:0/2446264949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a627be90 0x5600a628c0f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c60c20b80 1 -- 172.21.15.46:0/2446264949 >> 172.21.15.46:0/2446264949 conn(0x5600a61f21c0 msgr2=0x5600a6285420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c35ffb700 0 client.29866 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c60c20b80 1 -- 172.21.15.46:0/2446264949 shutdown_connections 2024-03-18T20:50:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.504+0000 7f9c60c20b80 1 -- 172.21.15.46:0/2446264949 wait complete. 2024-03-18T20:50:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.505+0000 7f9c60c20b80 20 client.29866 trim_cache size 0 max 16384 2024-03-18T20:50:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:50:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:50:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:50:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:50:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.506+0000 7f9c54a8b700 1 -- 172.21.15.46:0/3331327122 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9c380a1310 msgr2=0x7f9c380a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.506+0000 7f9c54a8b700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9c380a1310 0x7f9c380a37d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9c48008fd0 tx=0x7f9c4801e000 comp rx=0 tx=0).stop 2024-03-18T20:50:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.506+0000 7f9c54a8b700 1 -- 172.21.15.46:0/3331327122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c500680e0 msgr2=0x7f9c5019d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.506+0000 7f9c54a8b700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c500680e0 0x7f9c5019d740 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f9c44000c00 tx=0x7f9c44009580 comp rx=0 tx=0).stop 2024-03-18T20:50:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.506+0000 7f9c54a8b700 1 -- 172.21.15.46:0/3331327122 shutdown_connections 2024-03-18T20:50:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.506+0000 7f9c54a8b700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9c380a1310 0x7f9c380a37d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.506+0000 7f9c54a8b700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c500680e0 0x7f9c5019d740 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.506+0000 7f9c54a8b700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50105c40 0x7f9c501a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.506+0000 7f9c54a8b700 1 --2- 172.21.15.46:0/3331327122 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c50103c00 0x7f9c5019dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.506+0000 7f9c54a8b700 1 -- 172.21.15.46:0/3331327122 >> 172.21.15.46:0/3331327122 conn(0x7f9c50076d50 msgr2=0x7f9c500fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.508+0000 7f9c54a8b700 1 -- 172.21.15.46:0/3331327122 shutdown_connections 2024-03-18T20:50:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:04.508+0000 7f9c54a8b700 1 -- 172.21.15.46:0/3331327122 wait complete. 2024-03-18T20:50:04.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:04 smithi073 ceph-mon[47384]: pgmap v436: 97 pgs: 97 active+clean; 592 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.8 MiB/s wr, 1.24k op/s 2024-03-18T20:50:04.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:04 smithi073 ceph-mon[47384]: osdmap e97: 12 total, 12 up, 12 in 2024-03-18T20:50:04.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:04 smithi155 ceph-mon[48344]: pgmap v436: 97 pgs: 97 active+clean; 592 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.8 MiB/s wr, 1.24k op/s 2024-03-18T20:50:04.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:04 smithi155 ceph-mon[48344]: osdmap e97: 12 total, 12 up, 12 in 2024-03-18T20:50:04.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:04 smithi046 ceph-mon[43495]: pgmap v436: 97 pgs: 97 active+clean; 592 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.8 MiB/s wr, 1.24k op/s 2024-03-18T20:50:04.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:04 smithi046 ceph-mon[43495]: osdmap e97: 12 total, 12 up, 12 in 2024-03-18T20:50:05.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 42561 60.34 MB/sec warmup 41 sec latency 106.312 ms 2024-03-18T20:50:06.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 43400 59.97 MB/sec warmup 42 sec latency 60.976 ms 2024-03-18T20:50:06.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:06 smithi073 ceph-mon[47384]: pgmap v438: 97 pgs: 97 active+clean; 592 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.7 MiB/s wr, 1k op/s 2024-03-18T20:50:06.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:06 smithi155 ceph-mon[48344]: pgmap v438: 97 pgs: 97 active+clean; 592 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.7 MiB/s wr, 1k op/s 2024-03-18T20:50:06.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:06 smithi046 ceph-mon[43495]: pgmap v438: 97 pgs: 97 active+clean; 592 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.7 MiB/s wr, 1k op/s 2024-03-18T20:50:07.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 44466 59.68 MB/sec warmup 43 sec latency 55.615 ms 2024-03-18T20:50:08.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 45713 60.12 MB/sec warmup 44 sec latency 45.530 ms 2024-03-18T20:50:08.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:08 smithi073 ceph-mon[47384]: pgmap v439: 97 pgs: 97 active+clean; 592 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.7 MiB/s wr, 1k op/s 2024-03-18T20:50:08.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:08 smithi155 ceph-mon[48344]: pgmap v439: 97 pgs: 97 active+clean; 592 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.7 MiB/s wr, 1k op/s 2024-03-18T20:50:08.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:08 smithi046 ceph-mon[43495]: pgmap v439: 97 pgs: 97 active+clean; 592 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.7 MiB/s wr, 1k op/s 2024-03-18T20:50:09.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 46646 59.82 MB/sec warmup 45 sec latency 39.791 ms 2024-03-18T20:50:10.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 46789 58.60 MB/sec warmup 46 sec latency 565.273 ms 2024-03-18T20:50:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:10 smithi073 ceph-mon[47384]: pgmap v440: 97 pgs: 97 active+clean; 614 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.4 MiB/s wr, 1.14k op/s 2024-03-18T20:50:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:10 smithi073 ceph-mon[47384]: osdmap e98: 12 total, 12 up, 12 in 2024-03-18T20:50:10.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:10 smithi155 ceph-mon[48344]: pgmap v440: 97 pgs: 97 active+clean; 614 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.4 MiB/s wr, 1.14k op/s 2024-03-18T20:50:10.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:10 smithi155 ceph-mon[48344]: osdmap e98: 12 total, 12 up, 12 in 2024-03-18T20:50:10.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:10 smithi046 ceph-mon[43495]: pgmap v440: 97 pgs: 97 active+clean; 614 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.4 MiB/s wr, 1.14k op/s 2024-03-18T20:50:10.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:10 smithi046 ceph-mon[43495]: osdmap e98: 12 total, 12 up, 12 in 2024-03-18T20:50:11.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 47252 57.87 MB/sec warmup 47 sec latency 528.682 ms 2024-03-18T20:50:12.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 48096 57.66 MB/sec warmup 48 sec latency 57.396 ms 2024-03-18T20:50:12.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:12 smithi073 ceph-mon[47384]: pgmap v442: 97 pgs: 97 active+clean; 614 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 660 op/s 2024-03-18T20:50:12.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:12 smithi155 ceph-mon[48344]: pgmap v442: 97 pgs: 97 active+clean; 614 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 660 op/s 2024-03-18T20:50:12.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:12 smithi046 ceph-mon[43495]: pgmap v442: 97 pgs: 97 active+clean; 614 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 660 op/s 2024-03-18T20:50:13.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 49155 57.52 MB/sec warmup 49 sec latency 55.043 ms 2024-03-18T20:50:14.359 INFO:tasks.workunit.client.0.smithi046.stdout: 10 50354 57.73 MB/sec warmup 50 sec latency 36.414 ms 2024-03-18T20:50:14.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:14 smithi073 ceph-mon[47384]: pgmap v443: 97 pgs: 97 active+clean; 641 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 24 MiB/s wr, 1.34k op/s 2024-03-18T20:50:14.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:14 smithi155 ceph-mon[48344]: pgmap v443: 97 pgs: 97 active+clean; 641 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 24 MiB/s wr, 1.34k op/s 2024-03-18T20:50:14.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:14 smithi046 ceph-mon[43495]: pgmap v443: 97 pgs: 97 active+clean; 641 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 24 MiB/s wr, 1.34k op/s 2024-03-18T20:50:15.358 INFO:tasks.workunit.client.0.smithi046.stdout: 10 51087 57.47 MB/sec warmup 51 sec latency 292.800 ms 2024-03-18T20:50:16.359 INFO:tasks.workunit.client.0.smithi046.stdout: 10 51790 57.07 MB/sec warmup 52 sec latency 76.518 ms 2024-03-18T20:50:16.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:16 smithi073 ceph-mon[47384]: pgmap v444: 97 pgs: 97 active+clean; 641 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 23 MiB/s wr, 1.33k op/s 2024-03-18T20:50:16.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:16 smithi155 ceph-mon[48344]: pgmap v444: 97 pgs: 97 active+clean; 641 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 23 MiB/s wr, 1.33k op/s 2024-03-18T20:50:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:16 smithi046 ceph-mon[43495]: pgmap v444: 97 pgs: 97 active+clean; 641 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 23 MiB/s wr, 1.33k op/s 2024-03-18T20:50:17.359 INFO:tasks.workunit.client.0.smithi046.stdout: 10 52876 57.00 MB/sec warmup 53 sec latency 48.820 ms 2024-03-18T20:50:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.692+0000 7fb051b06700 1 -- 172.21.15.46:0/4048420932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb04c0ffbb0 msgr2=0x7fb04c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.692+0000 7fb051b06700 1 --2- 172.21.15.46:0/4048420932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb04c0ffbb0 0x7fb04c100030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb040009990 tx=0x7fb040009ca0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.693+0000 7fb051b06700 1 -- 172.21.15.46:0/4048420932 shutdown_connections 2024-03-18T20:50:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.693+0000 7fb051b06700 1 --2- 172.21.15.46:0/4048420932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c100570 0x7fb04c10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.693+0000 7fb051b06700 1 --2- 172.21.15.46:0/4048420932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04c105c60 0x7fb04c106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.693+0000 7fb051b06700 1 --2- 172.21.15.46:0/4048420932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb04c0ffbb0 0x7fb04c100030 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.693+0000 7fb051b06700 1 -- 172.21.15.46:0/4048420932 >> 172.21.15.46:0/4048420932 conn(0x7fb04c076d50 msgr2=0x7fb04c077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.693+0000 7fb051b06700 1 -- 172.21.15.46:0/4048420932 shutdown_connections 2024-03-18T20:50:17.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.694+0000 7fb051b06700 1 -- 172.21.15.46:0/4048420932 wait complete. 2024-03-18T20:50:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.695+0000 7fb051b06700 1 Processor -- start 2024-03-18T20:50:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.695+0000 7fb051b06700 1 -- start start 2024-03-18T20:50:17.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.695+0000 7fb051b06700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04c0ffbb0 0x7fb04c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.695+0000 7fb051b06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c100570 0x7fb04c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.695+0000 7fb051b06700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb04c105c60 0x7fb04c1a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.695+0000 7fb051b06700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c198ca0 con 0x7fb04c100570 2024-03-18T20:50:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.695+0000 7fb051b06700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c1a34b0 con 0x7fb04c0ffbb0 2024-03-18T20:50:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.695+0000 7fb051b06700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c1a35f0 con 0x7fb04c105c60 2024-03-18T20:50:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.695+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04c0ffbb0 0x7fb04c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.695+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04c0ffbb0 0x7fb04c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:44106/0 (socket says 172.21.15.46:44106) 2024-03-18T20:50:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.695+0000 7fb04b7fe700 1 -- 172.21.15.46:0/1465776606 learned_addr learned my addr 172.21.15.46:0/1465776606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:17.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.695+0000 7fb04bfff700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb04c105c60 0x7fb04c1a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.696+0000 7fb04b7fe700 1 -- 172.21.15.46:0/1465776606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb04c105c60 msgr2=0x7fb04c1a2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.696+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb04c105c60 0x7fb04c1a2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.696+0000 7fb04b7fe700 1 -- 172.21.15.46:0/1465776606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c100570 msgr2=0x7fb04c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.696+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c100570 0x7fb04c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.696+0000 7fb04b7fe700 1 -- 172.21.15.46:0/1465776606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0400095f0 con 0x7fb04c0ffbb0 2024-03-18T20:50:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.696+0000 7fb04bfff700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb04c105c60 0x7fb04c1a2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:50:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.696+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04c0ffbb0 0x7fb04c197ef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00e9a0 tx=0x7fb03c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.696+0000 7fb048ff9700 1 -- 172.21.15.46:0/1465776606 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c009790 con 0x7fb04c0ffbb0 2024-03-18T20:50:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.696+0000 7fb051b06700 1 -- 172.21.15.46:0/1465776606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c1a3880 con 0x7fb04c0ffbb0 2024-03-18T20:50:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.696+0000 7fb051b06700 1 -- 172.21.15.46:0/1465776606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c1a3de0 con 0x7fb04c0ffbb0 2024-03-18T20:50:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.697+0000 7fb048ff9700 1 -- 172.21.15.46:0/1465776606 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb03c00cbd0 con 0x7fb04c0ffbb0 2024-03-18T20:50:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.697+0000 7fb048ff9700 1 -- 172.21.15.46:0/1465776606 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c018680 con 0x7fb04c0ffbb0 2024-03-18T20:50:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.700+0000 7fb048ff9700 1 -- 172.21.15.46:0/1465776606 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fb03c00cd40 con 0x7fb04c0ffbb0 2024-03-18T20:50:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.702+0000 7fb05dc9bb80 1 Processor -- start 2024-03-18T20:50:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.703+0000 7fb048ff9700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb0340a1390 0x7fb0340a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.703+0000 7fb048ff9700 1 -- 172.21.15.46:0/1465776606 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fb03c014070 con 0x7fb04c0ffbb0 2024-03-18T20:50:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.706+0000 7fb05dc9bb80 1 -- start start 2024-03-18T20:50:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.706+0000 7fb05dc9bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277d3f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.706+0000 7fb05dc9bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 0x561277d30a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.706+0000 7fb05dc9bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30f40 0x561277d45b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.706+0000 7fb05dc9bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561277c89930 con 0x561277d30580 2024-03-18T20:50:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.706+0000 7fb05dc9bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561277c872f0 con 0x561277d3edd0 2024-03-18T20:50:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.706+0000 7fb05dc9bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561277c82be0 con 0x561277d30f40 2024-03-18T20:50:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.706+0000 7fb04bfff700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb0340a1390 0x7fb0340a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.706+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277d3f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.706+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277d3f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:44116/0 (socket says 172.21.15.46:44116) 2024-03-18T20:50:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.706+0000 7fb04affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 0x561277d30a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.706+0000 7fb04b7fe700 1 -- 172.21.15.46:0/3397095621 learned_addr learned my addr 172.21.15.46:0/3397095621 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.707+0000 7fb04bfff700 1 --2- 172.21.15.46:0/3397095621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30f40 0x561277d45b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.707+0000 7fb04affd700 1 -- 172.21.15.46:0/3397095621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30f40 msgr2=0x561277d45b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.707+0000 7fb04affd700 1 --2- 172.21.15.46:0/3397095621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30f40 0x561277d45b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.707+0000 7fb04affd700 1 -- 172.21.15.46:0/3397095621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 msgr2=0x561277d3f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.707+0000 7fb04affd700 1 --2- 172.21.15.46:0/3397095621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277d3f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.707+0000 7fb04affd700 1 -- 172.21.15.46:0/3397095621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561277b04350 con 0x561277d30580 2024-03-18T20:50:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.707+0000 7fb04affd700 1 --2- 172.21.15.46:0/3397095621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 0x561277d30a00 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb04c068890 tx=0x7fb040005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e4ef35d09735b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.707+0000 7fb04bfff700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb0340a1390 0x7fb0340a3850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb044004230 tx=0x7fb0440045f0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.708+0000 7fb031ffb700 1 -- 172.21.15.46:0/3397095621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04001d070 con 0x561277d30580 2024-03-18T20:50:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.708+0000 7fb031ffb700 1 -- 172.21.15.46:0/3397095621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb04000bc10 con 0x561277d30580 2024-03-18T20:50:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.708+0000 7fb031ffb700 1 -- 172.21.15.46:0/3397095621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04000fbc0 con 0x561277d30580 2024-03-18T20:50:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.708+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/3397095621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 msgr2=0x561277d30a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.708+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/3397095621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 0x561277d30a00 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb04c068890 tx=0x7fb040005e80 comp rx=0 tx=0).stop 2024-03-18T20:50:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.709+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/3397095621 shutdown_connections 2024-03-18T20:50:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.709+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/3397095621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30f40 0x561277d45b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.709+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/3397095621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277d3f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.709+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/3397095621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 0x561277d30a00 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.709+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/3397095621 >> 172.21.15.46:0/3397095621 conn(0x561277ca61c0 msgr2=0x561277d2b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.709+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/3397095621 shutdown_connections 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.709+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/3397095621 wait complete. 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.709+0000 7fb05dc9bb80 1 -- start start 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.709+0000 7fb05dc9bb80 1 Processor -- start 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.710+0000 7fb05dc9bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 0x561277d45ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.710+0000 7fb05dc9bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30f40 0x561277d42db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.710+0000 7fb05dc9bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277d433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.710+0000 7fb05dc9bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561277c1ed20 con 0x561277d30580 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.710+0000 7fb05dc9bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561277c1e9c0 con 0x561277d3edd0 2024-03-18T20:50:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.710+0000 7fb05dc9bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561277b06640 con 0x561277d30f40 2024-03-18T20:50:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.710+0000 7fb05dc9bb80 10 client.0 _ll_register_callbacks cb 0x561277a8a340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:50:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.710+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 0x561277d45ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.710+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 0x561277d45ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54118/0 (socket says 172.21.15.46:54118) 2024-03-18T20:50:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.710+0000 7fb04b7fe700 1 -- 172.21.15.46:0/307855386 learned_addr learned my addr 172.21.15.46:0/307855386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.710+0000 7fb04affd700 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30f40 0x561277d42db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.710+0000 7fb04bfff700 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277d433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.711+0000 7fb04b7fe700 1 -- 172.21.15.46:0/307855386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30f40 msgr2=0x561277d42db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.711+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30f40 0x561277d42db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.711+0000 7fb04b7fe700 1 -- 172.21.15.46:0/307855386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 msgr2=0x561277d433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.711+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277d433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.711+0000 7fb04b7fe700 1 -- 172.21.15.46:0/307855386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c10d2f0 con 0x561277d30580 2024-03-18T20:50:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.711+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 0x561277d45ac0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fb03c099c80 tx=0x7fb03c0cab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa53b66d32a9b8a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.711+0000 7fb02affd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:50:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.711+0000 7fb02affd700 1 -- 172.21.15.46:0/307855386 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c09ddb0 con 0x561277d30580 2024-03-18T20:50:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.711+0000 7fb02affd700 1 -- 172.21.15.46:0/307855386 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fb03c0a2400 con 0x561277d30580 2024-03-18T20:50:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.712+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/307855386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561277d236c0 con 0x561277d30580 2024-03-18T20:50:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.713+0000 7fb02affd700 1 -- 172.21.15.46:0/307855386 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb03c0b8280 con 0x561277d30580 2024-03-18T20:50:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.713+0000 7fb05dc9bb80 10 client.20235 fetch_fsmap learned FSMap version 65 2024-03-18T20:50:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.713+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/307855386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x561277ab0b00 con 0x561277d30580 2024-03-18T20:50:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.714+0000 7fb02affd700 1 -- 172.21.15.46:0/307855386 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561277d236c0 con 0x561277d30580 2024-03-18T20:50:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.714+0000 7fb02affd700 1 -- 172.21.15.46:0/307855386 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 65) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fb03c095e70 con 0x561277d30580 2024-03-18T20:50:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.714+0000 7fb05dc9bb80 10 client.20235 fetch_fsmap finished waiting for FSMap version 65 2024-03-18T20:50:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.714+0000 7fb05dc9bb80 10 client.20235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:50:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.714+0000 7fb05dc9bb80 20 client.20235 populate_metadata read hostname 'smithi046' 2024-03-18T20:50:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.714+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561277d53170 0x561277d55590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.714+0000 7fb05dc9bb80 4 client.20235 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:50:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.714+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/307855386 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561277c7b6a0 con 0x561277d53170 2024-03-18T20:50:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.715+0000 7fb04affd700 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561277d53170 0x561277d55590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.717+0000 7fb02affd700 10 client.20235 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:17.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.717+0000 7fb04affd700 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561277d53170 0x561277d55590 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f80f425d83caf6 server_cookie=8504bdff609abcb2 in_seq=0 out_seq=0 2024-03-18T20:50:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.718+0000 7fb02affd700 1 -- 172.21.15.46:0/307855386 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561277c7b6a0 con 0x561277d53170 2024-03-18T20:50:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.718+0000 7fb02affd700 10 client.20235 handle_command_reply: tid=0 2024-03-18T20:50:17.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.718+0000 7fb05dc9bb80 1 client.20235 shutdown 2024-03-18T20:50:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.718+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/307855386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 msgr2=0x561277d45ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.718+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 0x561277d45ac0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fb03c099c80 tx=0x7fb03c0cab20 comp rx=0 tx=0).stop 2024-03-18T20:50:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.718+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/307855386 shutdown_connections 2024-03-18T20:50:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.718+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561277d53170 0x561277d55590 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.718+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30580 0x561277d45ac0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.718+0000 7fb02affd700 0 client.20235 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.718+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277d433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.718+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/307855386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30f40 0x561277d42db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.718+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/307855386 >> 172.21.15.46:0/307855386 conn(0x561277ca61c0 msgr2=0x561277d2b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.719+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/307855386 shutdown_connections 2024-03-18T20:50:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.719+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/307855386 wait complete. 2024-03-18T20:50:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.719+0000 7fb05dc9bb80 20 client.20235 trim_cache size 0 max 16384 2024-03-18T20:50:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.745+0000 7fb05dc9bb80 1 Processor -- start 2024-03-18T20:50:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.745+0000 7fb05dc9bb80 1 -- start start 2024-03-18T20:50:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.745+0000 7fb05dc9bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 0x561277d3f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.745+0000 7fb05dc9bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d30f40 0x561277d4a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.745+0000 7fb05dc9bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d3edd0 0x561277d455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.745+0000 7fb05dc9bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561277d31540 con 0x561277d3edd0 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.745+0000 7fb05dc9bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561277d506e0 con 0x561277d30f40 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.745+0000 7fb05dc9bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561277d4a820 con 0x561277d30580 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d30f40 0x561277d4a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 0x561277d3f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d30f40 0x561277d4a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:44134/0 (socket says 172.21.15.46:44134) 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04affd700 1 -- 172.21.15.46:0/135153329 learned_addr learned my addr 172.21.15.46:0/135153329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 0x561277d3f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33398/0 (socket says 172.21.15.46:33398) 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04bfff700 1 --2- 172.21.15.46:0/135153329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d3edd0 0x561277d455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04b7fe700 1 -- 172.21.15.46:0/135153329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d30f40 msgr2=0x561277d4a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/135153329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d30f40 0x561277d4a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04b7fe700 1 -- 172.21.15.46:0/135153329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d3edd0 msgr2=0x561277d455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/135153329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d3edd0 0x561277d455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04b7fe700 1 -- 172.21.15.46:0/135153329 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561277c48e40 con 0x561277d30580 2024-03-18T20:50:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04affd700 1 --2- 172.21.15.46:0/135153329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d30f40 0x561277d4a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:50:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/135153329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 0x561277d3f1b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0969f0 tx=0x7fb03c096370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f49960f630f7f7ca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.746+0000 7fb0327fc700 1 -- 172.21.15.46:0/135153329 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0985e0 con 0x561277d30580 2024-03-18T20:50:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.747+0000 7fb0327fc700 1 -- 172.21.15.46:0/135153329 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb03c0a2400 con 0x561277d30580 2024-03-18T20:50:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.747+0000 7fb0327fc700 1 -- 172.21.15.46:0/135153329 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0a1f70 con 0x561277d30580 2024-03-18T20:50:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.748+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/135153329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 msgr2=0x561277d3f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.748+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/135153329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 0x561277d3f1b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0969f0 tx=0x7fb03c096370 comp rx=0 tx=0).stop 2024-03-18T20:50:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.748+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/135153329 shutdown_connections 2024-03-18T20:50:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.748+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/135153329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d3edd0 0x561277d455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.748+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/135153329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 0x561277d3f1b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.748+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/135153329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d30f40 0x561277d4a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.748+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/135153329 >> 172.21.15.46:0/135153329 conn(0x561277ca61c0 msgr2=0x561277d2abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.748+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/135153329 shutdown_connections 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.748+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/135153329 wait complete. 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.748+0000 7fb05dc9bb80 1 -- start start 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.749+0000 7fb05dc9bb80 1 Processor -- start 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.749+0000 7fb05dc9bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 0x561277da1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.749+0000 7fb05dc9bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30f40 0x561277da2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.749+0000 7fb05dc9bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277da29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.749+0000 7fb05dc9bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561277d482c0 con 0x561277d30f40 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.749+0000 7fb05dc9bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561277d48400 con 0x561277d3edd0 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.749+0000 7fb05dc9bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561277d485c0 con 0x561277d30580 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.749+0000 7fb05dc9bb80 10 client.0 _ll_register_callbacks cb 0x561277a8a340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.749+0000 7fb04bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277da29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.749+0000 7fb04bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277da29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:44150/0 (socket says 172.21.15.46:44150) 2024-03-18T20:50:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.749+0000 7fb04bfff700 1 -- 172.21.15.46:0/2038329215 learned_addr learned my addr 172.21.15.46:0/2038329215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.750+0000 7fb04affd700 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30f40 0x561277da2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.750+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 0x561277da1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.750+0000 7fb04bfff700 1 -- 172.21.15.46:0/2038329215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 msgr2=0x561277da1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.750+0000 7fb04bfff700 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 0x561277da1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.750+0000 7fb04bfff700 1 -- 172.21.15.46:0/2038329215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30f40 msgr2=0x561277da2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.750+0000 7fb04bfff700 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30f40 0x561277da2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.750+0000 7fb04bfff700 1 -- 172.21.15.46:0/2038329215 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561277b04350 con 0x561277d3edd0 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.750+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 0x561277da1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.750+0000 7fb04bfff700 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277da29c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb044003f10 tx=0x7fb044004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5be658fbf02011b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.750+0000 7fb02affd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.750+0000 7fb02affd700 1 -- 172.21.15.46:0/2038329215 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044030430 con 0x561277d3edd0 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.750+0000 7fb02affd700 1 -- 172.21.15.46:0/2038329215 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fb044018450 con 0x561277d3edd0 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.751+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/2038329215 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb04000bc10 con 0x561277d3edd0 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.751+0000 7fb02affd700 1 -- 172.21.15.46:0/2038329215 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb044018610 con 0x561277d3edd0 2024-03-18T20:50:17.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.751+0000 7fb05dc9bb80 10 client.29825 fetch_fsmap learned FSMap version 65 2024-03-18T20:50:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.751+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/2038329215 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x561277ab0b00 con 0x561277d3edd0 2024-03-18T20:50:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.751+0000 7fb02affd700 1 -- 172.21.15.46:0/2038329215 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb04000bc10 con 0x561277d3edd0 2024-03-18T20:50:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.751+0000 7fb02affd700 1 -- 172.21.15.46:0/2038329215 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 65) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fb044030a80 con 0x561277d3edd0 2024-03-18T20:50:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.753+0000 7fb05dc9bb80 10 client.29825 fetch_fsmap finished waiting for FSMap version 65 2024-03-18T20:50:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.753+0000 7fb05dc9bb80 10 client.29825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:50:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.753+0000 7fb05dc9bb80 20 client.29825 populate_metadata read hostname 'smithi046' 2024-03-18T20:50:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.753+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561277db8a00 0x561277dbae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.753+0000 7fb05dc9bb80 4 client.29825 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:50:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.753+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/2038329215 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561277ad1c90 con 0x561277db8a00 2024-03-18T20:50:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.756+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561277db8a00 0x561277dbae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.757+0000 7fb04b7fe700 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561277db8a00 0x561277dbae20 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc3e0db0cb2833e server_cookie=98faad03ff01779e in_seq=0 out_seq=0 2024-03-18T20:50:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.757+0000 7fb02affd700 10 client.29825 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb02affd700 1 -- 172.21.15.46:0/2038329215 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561277ad1c90 con 0x561277db8a00 2024-03-18T20:50:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb02affd700 10 client.29825 handle_command_reply: tid=0 2024-03-18T20:50:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb05dc9bb80 1 client.29825 shutdown 2024-03-18T20:50:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/2038329215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 msgr2=0x561277da29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277da29c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb044003f10 tx=0x7fb044004b40 comp rx=0 tx=0).stop 2024-03-18T20:50:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/2038329215 shutdown_connections 2024-03-18T20:50:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561277db8a00 0x561277dbae20 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561277d3edd0 0x561277da29c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561277d30580 0x561277da1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb05dc9bb80 1 --2- 172.21.15.46:0/2038329215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561277d30f40 0x561277da2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/2038329215 >> 172.21.15.46:0/2038329215 conn(0x561277ca61c0 msgr2=0x561277d3cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb02affd700 0 client.29825 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/2038329215 shutdown_connections 2024-03-18T20:50:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.938+0000 7fb05dc9bb80 1 -- 172.21.15.46:0/2038329215 wait complete. 2024-03-18T20:50:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.939+0000 7fb05dc9bb80 20 client.29825 trim_cache size 0 max 16384 2024-03-18T20:50:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:50:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:50:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:50:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:50:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.939+0000 7fb051b06700 1 -- 172.21.15.46:0/1465776606 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb0340a1390 msgr2=0x7fb0340a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.939+0000 7fb051b06700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb0340a1390 0x7fb0340a3850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb044004230 tx=0x7fb0440045f0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.939+0000 7fb051b06700 1 -- 172.21.15.46:0/1465776606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04c0ffbb0 msgr2=0x7fb04c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.939+0000 7fb051b06700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04c0ffbb0 0x7fb04c197ef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00e9a0 tx=0x7fb03c00ecb0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.939+0000 7fb051b06700 1 -- 172.21.15.46:0/1465776606 shutdown_connections 2024-03-18T20:50:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.939+0000 7fb051b06700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb0340a1390 0x7fb0340a3850 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.939+0000 7fb051b06700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb04c105c60 0x7fb04c1a2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.939+0000 7fb051b06700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04c0ffbb0 0x7fb04c197ef0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.939+0000 7fb051b06700 1 --2- 172.21.15.46:0/1465776606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04c100570 0x7fb04c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.939+0000 7fb051b06700 1 -- 172.21.15.46:0/1465776606 >> 172.21.15.46:0/1465776606 conn(0x7fb04c076d50 msgr2=0x7fb04c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.940+0000 7fb051b06700 1 -- 172.21.15.46:0/1465776606 shutdown_connections 2024-03-18T20:50:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:17.940+0000 7fb051b06700 1 -- 172.21.15.46:0/1465776606 wait complete. 2024-03-18T20:50:18.359 INFO:tasks.workunit.client.0.smithi046.stdout: 10 53863 57.04 MB/sec warmup 54 sec latency 113.506 ms 2024-03-18T20:50:18.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:18 smithi073 ceph-mon[47384]: pgmap v445: 97 pgs: 97 active+clean; 641 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 23 MiB/s wr, 1.33k op/s 2024-03-18T20:50:18.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:18 smithi155 ceph-mon[48344]: pgmap v445: 97 pgs: 97 active+clean; 641 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 23 MiB/s wr, 1.33k op/s 2024-03-18T20:50:18.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:18 smithi046 ceph-mon[43495]: pgmap v445: 97 pgs: 97 active+clean; 641 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 23 MiB/s wr, 1.33k op/s 2024-03-18T20:50:19.359 INFO:tasks.workunit.client.0.smithi046.stdout: 10 54968 56.98 MB/sec warmup 55 sec latency 63.725 ms 2024-03-18T20:50:20.359 INFO:tasks.workunit.client.0.smithi046.stdout: 10 56030 57.02 MB/sec warmup 56 sec latency 52.256 ms 2024-03-18T20:50:20.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:20 smithi073 ceph-mon[47384]: pgmap v446: 97 pgs: 97 active+clean; 654 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 29 MiB/s wr, 1.43k op/s 2024-03-18T20:50:20.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:20 smithi155 ceph-mon[48344]: pgmap v446: 97 pgs: 97 active+clean; 654 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 29 MiB/s wr, 1.43k op/s 2024-03-18T20:50:20.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:20 smithi046 ceph-mon[43495]: pgmap v446: 97 pgs: 97 active+clean; 654 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 29 MiB/s wr, 1.43k op/s 2024-03-18T20:50:21.359 INFO:tasks.workunit.client.0.smithi046.stdout: 10 56978 57.04 MB/sec warmup 57 sec latency 63.988 ms 2024-03-18T20:50:22.321 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:22 smithi073 ceph-mon[47384]: pgmap v447: 97 pgs: 97 active+clean; 654 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 26 MiB/s wr, 1.28k op/s 2024-03-18T20:50:22.322 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:22 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:50:22.322 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:22 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.322 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:22 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.322 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:22 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.322 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:22 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.322 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:22 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.323 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:22 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.359 INFO:tasks.workunit.client.0.smithi046.stdout: 10 57833 56.70 MB/sec warmup 58 sec latency 65.949 ms 2024-03-18T20:50:22.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:22 smithi155 ceph-mon[48344]: pgmap v447: 97 pgs: 97 active+clean; 654 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 26 MiB/s wr, 1.28k op/s 2024-03-18T20:50:22.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:22 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:50:22.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:22 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:22 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:22 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:22 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:22 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:22 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:22 smithi046 ceph-mon[43495]: pgmap v447: 97 pgs: 97 active+clean; 654 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 26 MiB/s wr, 1.28k op/s 2024-03-18T20:50:22.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:22 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:50:22.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:22 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:22 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:22 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:22 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:22 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:22.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:22 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:23.359 INFO:tasks.workunit.client.0.smithi046.stdout: 10 58967 56.87 MB/sec warmup 59 sec latency 79.906 ms 2024-03-18T20:50:23.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:23 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:50:23.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:23 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:50:23.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:23 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:23.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:23 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:50:23.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:23 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:50:23.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:23 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:23.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:23 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:50:23.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:23 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:50:23.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:23 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:50:24.359 INFO:tasks.workunit.client.0.smithi046.stdout: 10 59986 56.74 MB/sec warmup 60 sec latency 55.544 ms 2024-03-18T20:50:24.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:24 smithi073 ceph-mon[47384]: pgmap v448: 97 pgs: 97 active+clean; 688 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 1.53k op/s 2024-03-18T20:50:24.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:24 smithi155 ceph-mon[48344]: pgmap v448: 97 pgs: 97 active+clean; 688 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 1.53k op/s 2024-03-18T20:50:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:24 smithi046 ceph-mon[43495]: pgmap v448: 97 pgs: 97 active+clean; 688 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 1.53k op/s 2024-03-18T20:50:25.359 INFO:tasks.workunit.client.0.smithi046.stdout: 10 61144 56.84 MB/sec warmup 61 sec latency 46.460 ms 2024-03-18T20:50:26.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 62278 57.01 MB/sec warmup 62 sec latency 38.231 ms 2024-03-18T20:50:26.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:26 smithi073 ceph-mon[47384]: pgmap v449: 97 pgs: 97 active+clean; 688 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 860 op/s 2024-03-18T20:50:26.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:26 smithi155 ceph-mon[48344]: pgmap v449: 97 pgs: 97 active+clean; 688 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 860 op/s 2024-03-18T20:50:26.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:26 smithi046 ceph-mon[43495]: pgmap v449: 97 pgs: 97 active+clean; 688 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 860 op/s 2024-03-18T20:50:27.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 63254 56.89 MB/sec warmup 63 sec latency 46.230 ms 2024-03-18T20:50:28.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 63775 56.38 MB/sec warmup 64 sec latency 388.954 ms 2024-03-18T20:50:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:28 smithi073 ceph-mon[47384]: pgmap v450: 97 pgs: 97 active+clean; 688 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 860 op/s 2024-03-18T20:50:28.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:28 smithi155 ceph-mon[48344]: pgmap v450: 97 pgs: 97 active+clean; 688 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 860 op/s 2024-03-18T20:50:28.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:28 smithi046 ceph-mon[43495]: pgmap v450: 97 pgs: 97 active+clean; 688 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 860 op/s 2024-03-18T20:50:29.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 64937 56.44 MB/sec warmup 65 sec latency 42.376 ms 2024-03-18T20:50:30.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 65923 56.52 MB/sec warmup 66 sec latency 45.947 ms 2024-03-18T20:50:30.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:30 smithi073 ceph-mon[47384]: pgmap v451: 97 pgs: 97 active+clean; 731 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2024-03-18T20:50:30.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:30 smithi155 ceph-mon[48344]: pgmap v451: 97 pgs: 97 active+clean; 731 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2024-03-18T20:50:30.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:30 smithi046 ceph-mon[43495]: pgmap v451: 97 pgs: 97 active+clean; 731 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.18k op/s 2024-03-18T20:50:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.086+0000 7f37b253a700 1 -- 172.21.15.46:0/1319144917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37ac0ffbb0 msgr2=0x7f37ac100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.086+0000 7f37b253a700 1 --2- 172.21.15.46:0/1319144917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37ac0ffbb0 0x7f37ac100030 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009990 tx=0x7f37a0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.086+0000 7f37b253a700 1 -- 172.21.15.46:0/1319144917 shutdown_connections 2024-03-18T20:50:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.086+0000 7f37b253a700 1 --2- 172.21.15.46:0/1319144917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37ac105c60 0x7f37ac106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.086+0000 7f37b253a700 1 --2- 172.21.15.46:0/1319144917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac100570 0x7f37ac10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.086+0000 7f37b253a700 1 --2- 172.21.15.46:0/1319144917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37ac0ffbb0 0x7f37ac100030 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.086+0000 7f37b253a700 1 -- 172.21.15.46:0/1319144917 >> 172.21.15.46:0/1319144917 conn(0x7f37ac076d50 msgr2=0x7f37ac077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.086+0000 7f37b253a700 1 -- 172.21.15.46:0/1319144917 shutdown_connections 2024-03-18T20:50:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.086+0000 7f37b253a700 1 -- 172.21.15.46:0/1319144917 wait complete. 2024-03-18T20:50:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.087+0000 7f37b253a700 1 Processor -- start 2024-03-18T20:50:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.087+0000 7f37b253a700 1 -- start start 2024-03-18T20:50:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37b253a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac0ffbb0 0x7f37ac194ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37b253a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37ac100570 0x7f37ac1953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37b253a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37ac105c60 0x7f37ac199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37b253a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac199dc0 con 0x7f37ac100570 2024-03-18T20:50:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37b253a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac199f30 con 0x7f37ac105c60 2024-03-18T20:50:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37b253a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac19a0f0 con 0x7f37ac0ffbb0 2024-03-18T20:50:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37abfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac0ffbb0 0x7f37ac194ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37abfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac0ffbb0 0x7f37ac194ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:50924/0 (socket says 172.21.15.46:50924) 2024-03-18T20:50:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37abfff700 1 -- 172.21.15.46:0/592963290 learned_addr learned my addr 172.21.15.46:0/592963290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37ab7fe700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37ac100570 0x7f37ac1953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37abfff700 1 -- 172.21.15.46:0/592963290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37ac105c60 msgr2=0x7f37ac199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37abfff700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37ac105c60 0x7f37ac199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37abfff700 1 -- 172.21.15.46:0/592963290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37ac100570 msgr2=0x7f37ac1953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37abfff700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37ac100570 0x7f37ac1953e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.088+0000 7f37abfff700 1 -- 172.21.15.46:0/592963290 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a00095f0 con 0x7f37ac0ffbb0 2024-03-18T20:50:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.089+0000 7f37abfff700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac0ffbb0 0x7f37ac194ea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f379c007a00 tx=0x7f379c007d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.089+0000 7f37a97fa700 1 -- 172.21.15.46:0/592963290 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c010070 con 0x7f37ac0ffbb0 2024-03-18T20:50:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.089+0000 7f37b253a700 1 -- 172.21.15.46:0/592963290 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac19a3d0 con 0x7f37ac0ffbb0 2024-03-18T20:50:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.089+0000 7f37b253a700 1 -- 172.21.15.46:0/592963290 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac1a1c10 con 0x7f37ac0ffbb0 2024-03-18T20:50:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.090+0000 7f37a97fa700 1 -- 172.21.15.46:0/592963290 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f379c00b660 con 0x7f37ac0ffbb0 2024-03-18T20:50:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.090+0000 7f37a97fa700 1 -- 172.21.15.46:0/592963290 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c003920 con 0x7f37ac0ffbb0 2024-03-18T20:50:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.092+0000 7f37a97fa700 1 -- 172.21.15.46:0/592963290 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f379c003b60 con 0x7f37ac0ffbb0 2024-03-18T20:50:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.092+0000 7f37a97fa700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37940a1310 0x7f37940a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.093+0000 7f37a97fa700 1 -- 172.21.15.46:0/592963290 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f379c0cc140 con 0x7f37ac0ffbb0 2024-03-18T20:50:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.093+0000 7f37ab7fe700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37940a1310 0x7f37940a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.093+0000 7f37ab7fe700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37940a1310 0x7f37940a37d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f37a000b2e0 tx=0x7f37a0005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.093+0000 7f37be6cfb80 1 Processor -- start 2024-03-18T20:50:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.093+0000 7f37be6cfb80 1 -- start start 2024-03-18T20:50:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.093+0000 7f37be6cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5570 0x55b3380f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37be6cfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5e90 0x55b338100870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37be6cfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b338100db0 0x55b3381031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37be6cfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33804f930 con 0x55b3380f5e90 2024-03-18T20:50:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37be6cfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33804d2f0 con 0x55b338100db0 2024-03-18T20:50:31.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37be6cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b338048be0 con 0x55b3380f5570 2024-03-18T20:50:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37ab7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5e90 0x55b338100870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37ab7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5e90 0x55b338100870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50148/0 (socket says 172.21.15.46:50148) 2024-03-18T20:50:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37ab7fe700 1 -- 172.21.15.46:0/216606809 learned_addr learned my addr 172.21.15.46:0/216606809 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37abfff700 1 --2- 172.21.15.46:0/216606809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5570 0x55b3380f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37ab7fe700 1 -- 172.21.15.46:0/216606809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5570 msgr2=0x55b3380f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37ab7fe700 1 --2- 172.21.15.46:0/216606809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5570 0x55b3380f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37ab7fe700 1 -- 172.21.15.46:0/216606809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b338100db0 msgr2=0x55b3381031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37ab7fe700 1 --2- 172.21.15.46:0/216606809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b338100db0 0x55b3381031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37ab7fe700 1 -- 172.21.15.46:0/216606809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b337eca350 con 0x55b3380f5e90 2024-03-18T20:50:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37ab7fe700 1 --2- 172.21.15.46:0/216606809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5e90 0x55b338100870 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009990 tx=0x7f37a0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fb394328eedfdf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.094+0000 7f37927fc700 1 -- 172.21.15.46:0/216606809 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0034070 con 0x55b3380f5e90 2024-03-18T20:50:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.095+0000 7f37927fc700 1 -- 172.21.15.46:0/216606809 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37a001e040 con 0x55b3380f5e90 2024-03-18T20:50:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.095+0000 7f37927fc700 1 -- 172.21.15.46:0/216606809 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a002f620 con 0x55b3380f5e90 2024-03-18T20:50:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.095+0000 7f37be6cfb80 1 -- 172.21.15.46:0/216606809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5e90 msgr2=0x55b338100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.095+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/216606809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5e90 0x55b338100870 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009990 tx=0x7f37a0020000 comp rx=0 tx=0).stop 2024-03-18T20:50:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.095+0000 7f37be6cfb80 1 -- 172.21.15.46:0/216606809 shutdown_connections 2024-03-18T20:50:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.095+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/216606809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b338100db0 0x55b3381031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.095+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/216606809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5570 0x55b3380f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.095+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/216606809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5e90 0x55b338100870 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.095+0000 7f37be6cfb80 1 -- 172.21.15.46:0/216606809 >> 172.21.15.46:0/216606809 conn(0x55b33806c1c0 msgr2=0x55b3380f1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.095+0000 7f37be6cfb80 1 -- 172.21.15.46:0/216606809 shutdown_connections 2024-03-18T20:50:31.103 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.095+0000 7f37be6cfb80 1 -- 172.21.15.46:0/216606809 wait complete. 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.096+0000 7f37be6cfb80 1 -- start start 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.096+0000 7f37be6cfb80 1 Processor -- start 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37be6cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5570 0x55b338100760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37be6cfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3380f5e90 0x55b3380fdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37be6cfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b338100db0 0x55b3380fe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37be6cfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b337fe4d20 con 0x55b338100db0 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37be6cfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b337fe49c0 con 0x55b3380f5e90 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37be6cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b337ecc640 con 0x55b3380f5570 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37be6cfb80 10 client.0 _ll_register_callbacks cb 0x55b337e50340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37b0ad7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b338100db0 0x55b3380fe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37b0ad7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b338100db0 0x55b3380fe0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50164/0 (socket says 172.21.15.46:50164) 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37b0ad7700 1 -- 172.21.15.46:0/4184324003 learned_addr learned my addr 172.21.15.46:0/4184324003 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37abfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5570 0x55b338100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37ab7fe700 1 --2- 172.21.15.46:0/4184324003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3380f5e90 0x55b3380fdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37b0ad7700 1 -- 172.21.15.46:0/4184324003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5570 msgr2=0x55b338100760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37b0ad7700 1 --2- 172.21.15.46:0/4184324003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5570 0x55b338100760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37b0ad7700 1 -- 172.21.15.46:0/4184324003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3380f5e90 msgr2=0x55b3380fdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37b0ad7700 1 --2- 172.21.15.46:0/4184324003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3380f5e90 0x55b3380fdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37b0ad7700 1 -- 172.21.15.46:0/4184324003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac10d2f0 con 0x55b338100db0 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f37b0ad7700 1 --2- 172.21.15.46:0/4184324003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b338100db0 0x55b3380fe0f0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f37a4007e90 tx=0x7f37a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66031c2c1452599a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.097+0000 7f378b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f378b7fe700 1 -- 172.21.15.46:0/4184324003 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4003a00 con 0x55b338100db0 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f378b7fe700 1 -- 172.21.15.46:0/4184324003 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f37a4003b60 con 0x55b338100db0 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f37be6cfb80 1 -- 172.21.15.46:0/4184324003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3380e96c0 con 0x55b338100db0 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f378b7fe700 1 -- 172.21.15.46:0/4184324003 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37a4003d60 con 0x55b338100db0 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f37be6cfb80 10 client.20253 fetch_fsmap learned FSMap version 65 2024-03-18T20:50:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f37be6cfb80 1 -- 172.21.15.46:0/4184324003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x55b337e76b00 con 0x55b338100db0 2024-03-18T20:50:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f378b7fe700 1 -- 172.21.15.46:0/4184324003 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b3380e96c0 con 0x55b338100db0 2024-03-18T20:50:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f378b7fe700 1 -- 172.21.15.46:0/4184324003 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 65) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f37a4004410 con 0x55b338100db0 2024-03-18T20:50:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f37be6cfb80 10 client.20253 fetch_fsmap finished waiting for FSMap version 65 2024-03-18T20:50:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f37be6cfb80 10 client.20253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:50:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f37be6cfb80 20 client.20253 populate_metadata read hostname 'smithi046' 2024-03-18T20:50:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/4184324003 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b338110ab0 0x55b338112ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f37be6cfb80 4 client.20253 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:50:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f37be6cfb80 1 -- 172.21.15.46:0/4184324003 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3380416a0 con 0x55b338110ab0 2024-03-18T20:50:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.098+0000 7f37abfff700 1 --2- 172.21.15.46:0/4184324003 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b338110ab0 0x55b338112ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.099+0000 7f37abfff700 1 --2- 172.21.15.46:0/4184324003 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b338110ab0 0x55b338112ed0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f00a17988e4dfd server_cookie=ceec517d3e05e9df in_seq=0 out_seq=0 2024-03-18T20:50:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.099+0000 7f378b7fe700 10 client.20253 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.099+0000 7f378b7fe700 1 -- 172.21.15.46:0/4184324003 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b3380416a0 con 0x55b338110ab0 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.099+0000 7f378b7fe700 10 client.20253 handle_command_reply: tid=0 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f37be6cfb80 1 client.20253 shutdown 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f37be6cfb80 1 -- 172.21.15.46:0/4184324003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b338100db0 msgr2=0x55b3380fe0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/4184324003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b338100db0 0x55b3380fe0f0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f37a4007e90 tx=0x7f37a400d3b0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f37be6cfb80 1 -- 172.21.15.46:0/4184324003 shutdown_connections 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/4184324003 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b338110ab0 0x55b338112ed0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/4184324003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b338100db0 0x55b3380fe0f0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/4184324003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5570 0x55b338100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/4184324003 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3380f5e90 0x55b3380fdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f378b7fe700 0 client.20253 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f37be6cfb80 1 -- 172.21.15.46:0/4184324003 >> 172.21.15.46:0/4184324003 conn(0x55b33806c1c0 msgr2=0x55b3380f1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f37be6cfb80 1 -- 172.21.15.46:0/4184324003 shutdown_connections 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f37be6cfb80 1 -- 172.21.15.46:0/4184324003 wait complete. 2024-03-18T20:50:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.100+0000 7f37be6cfb80 20 client.20253 trim_cache size 0 max 16384 2024-03-18T20:50:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.122+0000 7f37be6cfb80 1 Processor -- start 2024-03-18T20:50:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.122+0000 7f37be6cfb80 1 -- start start 2024-03-18T20:50:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37be6cfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5570 0x55b3381031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37be6cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5e90 0x55b3380f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37be6cfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b338100db0 0x55b3381079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37be6cfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b338107f20 con 0x55b3380f5570 2024-03-18T20:50:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37be6cfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33810df70 con 0x55b338100db0 2024-03-18T20:50:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37be6cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3380fc0e0 con 0x55b3380f5e90 2024-03-18T20:50:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37abfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5570 0x55b3381031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37abfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5570 0x55b3381031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50166/0 (socket says 172.21.15.46:50166) 2024-03-18T20:50:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37abfff700 1 -- 172.21.15.46:0/3076925077 learned_addr learned my addr 172.21.15.46:0/3076925077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37ab7fe700 1 --2- 172.21.15.46:0/3076925077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5e90 0x55b3380f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37b0ad7700 1 --2- 172.21.15.46:0/3076925077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b338100db0 0x55b3381079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37abfff700 1 -- 172.21.15.46:0/3076925077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5e90 msgr2=0x55b3380f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37abfff700 1 --2- 172.21.15.46:0/3076925077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5e90 0x55b3380f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37abfff700 1 -- 172.21.15.46:0/3076925077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b338100db0 msgr2=0x55b3381079e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37abfff700 1 --2- 172.21.15.46:0/3076925077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b338100db0 0x55b3381079e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37abfff700 1 -- 172.21.15.46:0/3076925077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b33800ee40 con 0x55b3380f5570 2024-03-18T20:50:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.123+0000 7f37abfff700 1 --2- 172.21.15.46:0/3076925077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5570 0x55b3381031a0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f379c0058f0 tx=0x7f379c0cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fe395a68d96f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.124+0000 7f3792ffd700 1 -- 172.21.15.46:0/3076925077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0b6070 con 0x55b3380f5570 2024-03-18T20:50:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.124+0000 7f3792ffd700 1 -- 172.21.15.46:0/3076925077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f379c098240 con 0x55b3380f5570 2024-03-18T20:50:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.124+0000 7f3792ffd700 1 -- 172.21.15.46:0/3076925077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c09a360 con 0x55b3380f5570 2024-03-18T20:50:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.124+0000 7f37be6cfb80 1 -- 172.21.15.46:0/3076925077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5570 msgr2=0x55b3381031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.124+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/3076925077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5570 0x55b3381031a0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f379c0058f0 tx=0x7f379c0cb040 comp rx=0 tx=0).stop 2024-03-18T20:50:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.124+0000 7f37be6cfb80 1 -- 172.21.15.46:0/3076925077 shutdown_connections 2024-03-18T20:50:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.124+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/3076925077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5570 0x55b3381031a0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.124+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/3076925077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b338100db0 0x55b3381079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.125+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/3076925077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3380f5e90 0x55b3380f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.125+0000 7f37be6cfb80 1 -- 172.21.15.46:0/3076925077 >> 172.21.15.46:0/3076925077 conn(0x55b33806c1c0 msgr2=0x55b338100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.125+0000 7f37be6cfb80 1 -- 172.21.15.46:0/3076925077 shutdown_connections 2024-03-18T20:50:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.125+0000 7f37be6cfb80 1 -- 172.21.15.46:0/3076925077 wait complete. 2024-03-18T20:50:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.125+0000 7f37be6cfb80 1 -- start start 2024-03-18T20:50:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.125+0000 7f37be6cfb80 1 Processor -- start 2024-03-18T20:50:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37be6cfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3380f5570 0x55b3381059c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37be6cfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5e90 0x55b338105f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37be6cfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b338100db0 0x55b3381638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37be6cfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3381066a0 con 0x55b3380f5e90 2024-03-18T20:50:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37be6cfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b338106810 con 0x55b3380f5570 2024-03-18T20:50:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37be6cfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b338163e20 con 0x55b338100db0 2024-03-18T20:50:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37be6cfb80 10 client.0 _ll_register_callbacks cb 0x7f37a4007780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:50:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37b0ad7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b338100db0 0x55b3381638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37b0ad7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b338100db0 0x55b3381638e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:50948/0 (socket says 172.21.15.46:50948) 2024-03-18T20:50:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37b0ad7700 1 -- 172.21.15.46:0/386578648 learned_addr learned my addr 172.21.15.46:0/386578648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37ab7fe700 1 --2- 172.21.15.46:0/386578648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5e90 0x55b338105f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37b0ad7700 1 -- 172.21.15.46:0/386578648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3380f5570 msgr2=0x55b3381059c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37b0ad7700 1 --2- 172.21.15.46:0/386578648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3380f5570 0x55b3381059c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37b0ad7700 1 -- 172.21.15.46:0/386578648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5e90 msgr2=0x55b338105f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37b0ad7700 1 --2- 172.21.15.46:0/386578648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5e90 0x55b338105f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.126+0000 7f37b0ad7700 1 -- 172.21.15.46:0/386578648 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac1a1c10 con 0x55b338100db0 2024-03-18T20:50:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.127+0000 7f37b0ad7700 1 --2- 172.21.15.46:0/386578648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b338100db0 0x55b3381638e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000f80 tx=0x7f37a4004a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d457eec8e09f4b67 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.127+0000 7f378b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:50:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.129+0000 7f378b7fe700 1 -- 172.21.15.46:0/386578648 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4004bb0 con 0x55b338100db0 2024-03-18T20:50:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.129+0000 7f378b7fe700 1 -- 172.21.15.46:0/386578648 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f37a4004d10 con 0x55b338100db0 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.129+0000 7f37be6cfb80 1 -- 172.21.15.46:0/386578648 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37a001e040 con 0x55b338100db0 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.130+0000 7f378b7fe700 1 -- 172.21.15.46:0/386578648 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37a400d8c0 con 0x55b338100db0 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.130+0000 7f37be6cfb80 10 client.29902 fetch_fsmap learned FSMap version 65 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.130+0000 7f37be6cfb80 1 -- 172.21.15.46:0/386578648 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x55b337ecec70 con 0x55b338100db0 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.130+0000 7f378b7fe700 1 -- 172.21.15.46:0/386578648 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37a4036020 con 0x55b338100db0 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.130+0000 7f378b7fe700 1 -- 172.21.15.46:0/386578648 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 65) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f37a4018b30 con 0x55b338100db0 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.130+0000 7f37be6cfb80 10 client.29902 fetch_fsmap finished waiting for FSMap version 65 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.130+0000 7f37be6cfb80 10 client.29902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.130+0000 7f37be6cfb80 20 client.29902 populate_metadata read hostname 'smithi046' 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.130+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/386578648 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b338165650 0x55b338167a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.130+0000 7f37be6cfb80 4 client.29902 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.130+0000 7f37be6cfb80 1 -- 172.21.15.46:0/386578648 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b337e97c90 con 0x55b338165650 2024-03-18T20:50:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.130+0000 7f37abfff700 1 --2- 172.21.15.46:0/386578648 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b338165650 0x55b338167a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.131+0000 7f378b7fe700 10 client.29902 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.131+0000 7f37abfff700 1 --2- 172.21.15.46:0/386578648 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b338165650 0x55b338167a70 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc66d84e966dc248 server_cookie=5f3c93ad87162296 in_seq=0 out_seq=0 2024-03-18T20:50:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.344+0000 7f378b7fe700 1 -- 172.21.15.46:0/386578648 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b337e97c90 con 0x55b338165650 2024-03-18T20:50:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.344+0000 7f378b7fe700 10 client.29902 handle_command_reply: tid=0 2024-03-18T20:50:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.346+0000 7f37be6cfb80 1 client.29902 shutdown 2024-03-18T20:50:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.346+0000 7f37be6cfb80 1 -- 172.21.15.46:0/386578648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b338100db0 msgr2=0x55b3381638e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.346+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/386578648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b338100db0 0x55b3381638e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000f80 tx=0x7f37a4004a10 comp rx=0 tx=0).stop 2024-03-18T20:50:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.346+0000 7f37be6cfb80 1 -- 172.21.15.46:0/386578648 shutdown_connections 2024-03-18T20:50:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.346+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/386578648 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b338165650 0x55b338167a70 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.346+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/386578648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b338100db0 0x55b3381638e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.346+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/386578648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3380f5570 0x55b3381059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.346+0000 7f37be6cfb80 1 --2- 172.21.15.46:0/386578648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3380f5e90 0x55b338105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.346+0000 7f37be6cfb80 1 -- 172.21.15.46:0/386578648 >> 172.21.15.46:0/386578648 conn(0x55b33806c1c0 msgr2=0x55b3380fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.346+0000 7f378b7fe700 0 client.29902 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.346+0000 7f37be6cfb80 1 -- 172.21.15.46:0/386578648 shutdown_connections 2024-03-18T20:50:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.346+0000 7f37be6cfb80 1 -- 172.21.15.46:0/386578648 wait complete. 2024-03-18T20:50:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.347+0000 7f37be6cfb80 20 client.29902 trim_cache size 0 max 16384 2024-03-18T20:50:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:50:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:50:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:50:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:50:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.349+0000 7f37b253a700 1 -- 172.21.15.46:0/592963290 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37940a1310 msgr2=0x7f37940a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.349+0000 7f37b253a700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37940a1310 0x7f37940a37d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f37a000b2e0 tx=0x7f37a0005960 comp rx=0 tx=0).stop 2024-03-18T20:50:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.349+0000 7f37b253a700 1 -- 172.21.15.46:0/592963290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac0ffbb0 msgr2=0x7f37ac194ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.349+0000 7f37b253a700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac0ffbb0 0x7f37ac194ea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f379c007a00 tx=0x7f379c007d10 comp rx=0 tx=0).stop 2024-03-18T20:50:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.350+0000 7f37b253a700 1 -- 172.21.15.46:0/592963290 shutdown_connections 2024-03-18T20:50:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.350+0000 7f37b253a700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f37940a1310 0x7f37940a37d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.350+0000 7f37b253a700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37ac105c60 0x7f37ac199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.350+0000 7f37b253a700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac0ffbb0 0x7f37ac194ea0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.350+0000 7f37b253a700 1 --2- 172.21.15.46:0/592963290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37ac100570 0x7f37ac1953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.350+0000 7f37b253a700 1 -- 172.21.15.46:0/592963290 >> 172.21.15.46:0/592963290 conn(0x7f37ac076d50 msgr2=0x7f37ac0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.350+0000 7f37b253a700 1 -- 172.21.15.46:0/592963290 shutdown_connections 2024-03-18T20:50:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:31.351+0000 7f37b253a700 1 -- 172.21.15.46:0/592963290 wait complete. 2024-03-18T20:50:31.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 66717 56.28 MB/sec warmup 67 sec latency 109.982 ms 2024-03-18T20:50:32.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 67800 56.29 MB/sec warmup 68 sec latency 36.207 ms 2024-03-18T20:50:32.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:32 smithi073 ceph-mon[47384]: pgmap v452: 97 pgs: 97 active+clean; 731 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.0 MiB/s wr, 657 op/s 2024-03-18T20:50:32.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:32 smithi155 ceph-mon[48344]: pgmap v452: 97 pgs: 97 active+clean; 731 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.0 MiB/s wr, 657 op/s 2024-03-18T20:50:32.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:32 smithi046 ceph-mon[43495]: pgmap v452: 97 pgs: 97 active+clean; 731 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.0 MiB/s wr, 657 op/s 2024-03-18T20:50:33.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 68726 56.26 MB/sec warmup 69 sec latency 88.283 ms 2024-03-18T20:50:34.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 69570 56.04 MB/sec warmup 70 sec latency 52.908 ms 2024-03-18T20:50:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:34 smithi073 ceph-mon[47384]: pgmap v453: 97 pgs: 97 active+clean; 740 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-03-18T20:50:34.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:34 smithi155 ceph-mon[48344]: pgmap v453: 97 pgs: 97 active+clean; 740 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-03-18T20:50:34.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:34 smithi046 ceph-mon[43495]: pgmap v453: 97 pgs: 97 active+clean; 740 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-03-18T20:50:35.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 70586 56.06 MB/sec warmup 71 sec latency 37.679 ms 2024-03-18T20:50:36.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 71848 56.19 MB/sec warmup 72 sec latency 28.469 ms 2024-03-18T20:50:36.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:36 smithi073 ceph-mon[47384]: pgmap v454: 97 pgs: 97 active+clean; 740 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.6 MiB/s wr, 757 op/s 2024-03-18T20:50:36.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:36 smithi155 ceph-mon[48344]: pgmap v454: 97 pgs: 97 active+clean; 740 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.6 MiB/s wr, 757 op/s 2024-03-18T20:50:36.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:36 smithi046 ceph-mon[43495]: pgmap v454: 97 pgs: 97 active+clean; 740 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.6 MiB/s wr, 757 op/s 2024-03-18T20:50:37.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 72850 56.21 MB/sec warmup 73 sec latency 46.710 ms 2024-03-18T20:50:38.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 73719 56.11 MB/sec warmup 74 sec latency 62.420 ms 2024-03-18T20:50:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:38 smithi073 ceph-mon[47384]: pgmap v455: 97 pgs: 97 active+clean; 740 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.6 MiB/s wr, 757 op/s 2024-03-18T20:50:38.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:38 smithi155 ceph-mon[48344]: pgmap v455: 97 pgs: 97 active+clean; 740 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.6 MiB/s wr, 757 op/s 2024-03-18T20:50:38.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:38 smithi046 ceph-mon[43495]: pgmap v455: 97 pgs: 97 active+clean; 740 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.6 MiB/s wr, 757 op/s 2024-03-18T20:50:39.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 74761 56.01 MB/sec warmup 75 sec latency 47.368 ms 2024-03-18T20:50:40.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 75928 56.16 MB/sec warmup 76 sec latency 49.929 ms 2024-03-18T20:50:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:40 smithi073 ceph-mon[47384]: pgmap v456: 97 pgs: 97 active+clean; 773 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-03-18T20:50:40.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:40 smithi155 ceph-mon[48344]: pgmap v456: 97 pgs: 97 active+clean; 773 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-03-18T20:50:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:40 smithi046 ceph-mon[43495]: pgmap v456: 97 pgs: 97 active+clean; 773 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2024-03-18T20:50:41.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 76950 56.17 MB/sec warmup 77 sec latency 39.827 ms 2024-03-18T20:50:42.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 77993 56.16 MB/sec warmup 78 sec latency 40.326 ms 2024-03-18T20:50:42.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:42 smithi073 ceph-mon[47384]: pgmap v457: 97 pgs: 97 active+clean; 773 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.5 MiB/s wr, 769 op/s 2024-03-18T20:50:42.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:42 smithi155 ceph-mon[48344]: pgmap v457: 97 pgs: 97 active+clean; 773 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.5 MiB/s wr, 769 op/s 2024-03-18T20:50:42.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:42 smithi046 ceph-mon[43495]: pgmap v457: 97 pgs: 97 active+clean; 773 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.5 MiB/s wr, 769 op/s 2024-03-18T20:50:43.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 78972 56.05 MB/sec warmup 79 sec latency 38.792 ms 2024-03-18T20:50:44.361 INFO:tasks.workunit.client.0.smithi046.stdout: 10 80020 56.00 MB/sec warmup 80 sec latency 109.307 ms 2024-03-18T20:50:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.499+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/811745426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d480ffe00 msgr2=0x7f9d481001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.499+0000 7f9d4f4a6700 1 --2- 172.21.15.46:0/811745426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d480ffe00 0x7f9d481001e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9d440098f0 tx=0x7f9d44009c00 comp rx=0 tx=0).stop 2024-03-18T20:50:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.499+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/811745426 shutdown_connections 2024-03-18T20:50:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.499+0000 7f9d4f4a6700 1 --2- 172.21.15.46:0/811745426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d48104f20 0x7f9d4810caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.499+0000 7f9d4f4a6700 1 --2- 172.21.15.46:0/811745426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d480ffe00 0x7f9d481001e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.499+0000 7f9d4f4a6700 1 --2- 172.21.15.46:0/811745426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d481007b0 0x7f9d481047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.499+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/811745426 >> 172.21.15.46:0/811745426 conn(0x7f9d480fb680 msgr2=0x7f9d480fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.499+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/811745426 shutdown_connections 2024-03-18T20:50:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.499+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/811745426 wait complete. 2024-03-18T20:50:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.500+0000 7f9d4f4a6700 1 Processor -- start 2024-03-18T20:50:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.500+0000 7f9d4f4a6700 1 -- start start 2024-03-18T20:50:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.500+0000 7f9d4f4a6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d480ffe00 0x7f9d481971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.500+0000 7f9d4f4a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d48104f20 0x7f9d48197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.500+0000 7f9d4f4a6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4819bab0 0x7f9d4819bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.500+0000 7f9d4f4a6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4819c530 con 0x7f9d48104f20 2024-03-18T20:50:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.500+0000 7f9d4f4a6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4819c6a0 con 0x7f9d480ffe00 2024-03-18T20:50:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.500+0000 7f9d4f4a6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4819c860 con 0x7f9d4819bab0 2024-03-18T20:50:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.501+0000 7f9d4d242700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d480ffe00 0x7f9d481971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.501+0000 7f9d4d242700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d480ffe00 0x7f9d481971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:39182/0 (socket says 172.21.15.46:39182) 2024-03-18T20:50:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.501+0000 7f9d4d242700 1 -- 172.21.15.46:0/757259868 learned_addr learned my addr 172.21.15.46:0/757259868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.501+0000 7f9d4da43700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4819bab0 0x7f9d4819bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.501+0000 7f9d4ca41700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d48104f20 0x7f9d48197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.501+0000 7f9d4d242700 1 -- 172.21.15.46:0/757259868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4819bab0 msgr2=0x7f9d4819bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.501+0000 7f9d4d242700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4819bab0 0x7f9d4819bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.501+0000 7f9d4d242700 1 -- 172.21.15.46:0/757259868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d48104f20 msgr2=0x7f9d48197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.501+0000 7f9d4d242700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d48104f20 0x7f9d48197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.501+0000 7f9d4d242700 1 -- 172.21.15.46:0/757259868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d380095f0 con 0x7f9d480ffe00 2024-03-18T20:50:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.501+0000 7f9d4ca41700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d48104f20 0x7f9d48197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:50:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.501+0000 7f9d4d242700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d480ffe00 0x7f9d481971d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400fa00 tx=0x7f9d44010f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.502+0000 7f9d3e7fc700 1 -- 172.21.15.46:0/757259868 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4401c070 con 0x7f9d480ffe00 2024-03-18T20:50:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.502+0000 7f9d3e7fc700 1 -- 172.21.15.46:0/757259868 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4400bac0 con 0x7f9d480ffe00 2024-03-18T20:50:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.502+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/757259868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d44009520 con 0x7f9d480ffe00 2024-03-18T20:50:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.502+0000 7f9d3e7fc700 1 -- 172.21.15.46:0/757259868 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44004930 con 0x7f9d480ffe00 2024-03-18T20:50:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.502+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/757259868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d481a4050 con 0x7f9d480ffe00 2024-03-18T20:50:44.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.504+0000 7f9d3e7fc700 1 -- 172.21.15.46:0/757259868 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9d44020070 con 0x7f9d480ffe00 2024-03-18T20:50:44.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.505+0000 7f9d3e7fc700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9d340a1310 0x7f9d340a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.505+0000 7f9d4ca41700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9d340a1310 0x7f9d340a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.505+0000 7f9d3e7fc700 1 -- 172.21.15.46:0/757259868 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f9d440cd920 con 0x7f9d480ffe00 2024-03-18T20:50:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.506+0000 7f9d4ca41700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9d340a1310 0x7f9d340a37d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9d48198720 tx=0x7f9d38009310 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d5b63bb80 1 Processor -- start 2024-03-18T20:50:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d5b63bb80 1 -- start start 2024-03-18T20:50:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d5b63bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fa2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d5b63bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 0x55b033fad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d5b63bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 0x55b033fb01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d5b63bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b033efc930 con 0x55b033fa2e90 2024-03-18T20:50:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d5b63bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b033efa2f0 con 0x55b033faddb0 2024-03-18T20:50:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d5b63bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b033ef5be0 con 0x55b033fa2570 2024-03-18T20:50:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d4d242700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fa2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d4d242700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fa2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:47128/0 (socket says 172.21.15.46:47128) 2024-03-18T20:50:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d4d242700 1 -- 172.21.15.46:0/1010341621 learned_addr learned my addr 172.21.15.46:0/1010341621 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d4da43700 1 --2- 172.21.15.46:0/1010341621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 0x55b033fb01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d4ca41700 1 --2- 172.21.15.46:0/1010341621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 0x55b033fad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d4d242700 1 -- 172.21.15.46:0/1010341621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 msgr2=0x55b033fb01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d4d242700 1 --2- 172.21.15.46:0/1010341621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 0x55b033fb01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d4d242700 1 -- 172.21.15.46:0/1010341621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 msgr2=0x55b033fad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.507+0000 7f9d4d242700 1 --2- 172.21.15.46:0/1010341621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 0x55b033fad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.508+0000 7f9d4d242700 1 -- 172.21.15.46:0/1010341621 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b033d77350 con 0x55b033fa2570 2024-03-18T20:50:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.508+0000 7f9d4da43700 1 --2- 172.21.15.46:0/1010341621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 0x55b033fb01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:50:44.514 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.508+0000 7f9d4ca41700 1 --2- 172.21.15.46:0/1010341621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 0x55b033fad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.508+0000 7f9d4d242700 1 --2- 172.21.15.46:0/1010341621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fa2950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9d440974b0 tx=0x7f9d44005960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44269df275fe0c1a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.508+0000 7f9d337fe700 1 -- 172.21.15.46:0/1010341621 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4409aa40 con 0x55b033fa2570 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.508+0000 7f9d337fe700 1 -- 172.21.15.46:0/1010341621 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4409e870 con 0x55b033fa2570 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.508+0000 7f9d337fe700 1 -- 172.21.15.46:0/1010341621 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4409aea0 con 0x55b033fa2570 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.509+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1010341621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 msgr2=0x55b033fa2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.509+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/1010341621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fa2950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9d440974b0 tx=0x7f9d44005960 comp rx=0 tx=0).stop 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.509+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1010341621 shutdown_connections 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.509+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/1010341621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 0x55b033fb01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.509+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/1010341621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fa2950 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.509+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/1010341621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 0x55b033fad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.509+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1010341621 >> 172.21.15.46:0/1010341621 conn(0x55b033f191c0 msgr2=0x55b033f9ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.509+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1010341621 shutdown_connections 2024-03-18T20:50:44.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.509+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1010341621 wait complete. 2024-03-18T20:50:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.510+0000 7f9d5b63bb80 1 -- start start 2024-03-18T20:50:44.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.510+0000 7f9d5b63bb80 1 Processor -- start 2024-03-18T20:50:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d5b63bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d5b63bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 0x55b033faaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d5b63bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 0x55b033fab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d5b63bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b033e91d20 con 0x55b033fa2e90 2024-03-18T20:50:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d5b63bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b033e919c0 con 0x55b033faddb0 2024-03-18T20:50:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d5b63bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b033d79640 con 0x55b033fa2570 2024-03-18T20:50:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d5b63bb80 10 client.0 _ll_register_callbacks cb 0x55b033cfd340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:50:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d4d242700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d4d242700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:47142/0 (socket says 172.21.15.46:47142) 2024-03-18T20:50:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d4d242700 1 -- 172.21.15.46:0/1088138403 learned_addr learned my addr 172.21.15.46:0/1088138403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d4da43700 1 --2- 172.21.15.46:0/1088138403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 0x55b033fab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d4d242700 1 -- 172.21.15.46:0/1088138403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 msgr2=0x55b033fab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d4d242700 1 --2- 172.21.15.46:0/1088138403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 0x55b033fab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d4d242700 1 -- 172.21.15.46:0/1088138403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 msgr2=0x55b033faaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d4d242700 1 --2- 172.21.15.46:0/1088138403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 0x55b033faaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.511+0000 7f9d4d242700 1 -- 172.21.15.46:0/1088138403 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d481a4050 con 0x55b033fa2570 2024-03-18T20:50:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.512+0000 7f9d4d242700 1 --2- 172.21.15.46:0/1088138403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fad730 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9d440cd040 tx=0x7f9d44011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2154b4dbfb098115 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.512+0000 7f9d30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:50:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.512+0000 7f9d30ff9700 1 -- 172.21.15.46:0/1088138403 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4409aa40 con 0x55b033fa2570 2024-03-18T20:50:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.512+0000 7f9d30ff9700 1 -- 172.21.15.46:0/1088138403 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f9d4409a140 con 0x55b033fa2570 2024-03-18T20:50:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.512+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1088138403 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b033f966c0 con 0x55b033fa2570 2024-03-18T20:50:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.512+0000 7f9d30ff9700 1 -- 172.21.15.46:0/1088138403 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d440b3220 con 0x55b033fa2570 2024-03-18T20:50:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.512+0000 7f9d5b63bb80 10 client.29914 fetch_fsmap learned FSMap version 65 2024-03-18T20:50:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.512+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1088138403 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x55b033d23b00 con 0x55b033fa2570 2024-03-18T20:50:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.512+0000 7f9d30ff9700 1 -- 172.21.15.46:0/1088138403 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4407d020 con 0x55b033fa2570 2024-03-18T20:50:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.512+0000 7f9d30ff9700 1 -- 172.21.15.46:0/1088138403 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 65) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f9d440a2aa0 con 0x55b033fa2570 2024-03-18T20:50:44.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.513+0000 7f9d5b63bb80 10 client.29914 fetch_fsmap finished waiting for FSMap version 65 2024-03-18T20:50:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.513+0000 7f9d5b63bb80 10 client.29914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:50:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.513+0000 7f9d5b63bb80 20 client.29914 populate_metadata read hostname 'smithi046' 2024-03-18T20:50:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.513+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/1088138403 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b033fce270 0x55b033fd0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.513+0000 7f9d5b63bb80 4 client.29914 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:50:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.513+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1088138403 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b033eee6a0 con 0x55b033fce270 2024-03-18T20:50:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.513+0000 7f9d4ca41700 1 --2- 172.21.15.46:0/1088138403 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b033fce270 0x55b033fd0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.513+0000 7f9d30ff9700 10 client.29914 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.513+0000 7f9d4ca41700 1 --2- 172.21.15.46:0/1088138403 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b033fce270 0x55b033fd0690 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d6539824bb0c707 server_cookie=885988263a8f2463 in_seq=0 out_seq=0 2024-03-18T20:50:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.516+0000 7f9d30ff9700 1 -- 172.21.15.46:0/1088138403 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b033eee6a0 con 0x55b033fce270 2024-03-18T20:50:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.516+0000 7f9d30ff9700 10 client.29914 handle_command_reply: tid=0 2024-03-18T20:50:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.516+0000 7f9d5b63bb80 1 client.29914 shutdown 2024-03-18T20:50:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.517+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1088138403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 msgr2=0x55b033fad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.517+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/1088138403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fad730 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9d440cd040 tx=0x7f9d44011000 comp rx=0 tx=0).stop 2024-03-18T20:50:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.517+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1088138403 shutdown_connections 2024-03-18T20:50:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.517+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/1088138403 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b033fce270 0x55b033fd0690 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.517+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/1088138403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 0x55b033fab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.517+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/1088138403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fad730 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.517+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/1088138403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 0x55b033faaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.517+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1088138403 >> 172.21.15.46:0/1088138403 conn(0x55b033f191c0 msgr2=0x55b033f9e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.517+0000 7f9d30ff9700 0 client.29914 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.517+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1088138403 shutdown_connections 2024-03-18T20:50:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.517+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/1088138403 wait complete. 2024-03-18T20:50:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.517+0000 7f9d5b63bb80 20 client.29914 trim_cache size 0 max 16384 2024-03-18T20:50:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.542+0000 7f9d5b63bb80 1 Processor -- start 2024-03-18T20:50:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.542+0000 7f9d5b63bb80 1 -- start start 2024-03-18T20:50:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.542+0000 7f9d5b63bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fa54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.542+0000 7f9d5b63bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 0x55b033fa93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.542+0000 7f9d5b63bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 0x55b033fa98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.542+0000 7f9d5b63bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b033fa4130 con 0x55b033fa2e90 2024-03-18T20:50:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.542+0000 7f9d5b63bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b033fa9e30 con 0x55b033faddb0 2024-03-18T20:50:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.542+0000 7f9d5b63bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b033f98c10 con 0x55b033fa2570 2024-03-18T20:50:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.542+0000 7f9d4d242700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fa54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.542+0000 7f9d4ca41700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 0x55b033fa93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.542+0000 7f9d4d242700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fa54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:47148/0 (socket says 172.21.15.46:47148) 2024-03-18T20:50:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.542+0000 7f9d4d242700 1 -- 172.21.15.46:0/3633104784 learned_addr learned my addr 172.21.15.46:0/3633104784 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.543+0000 7f9d4d242700 1 -- 172.21.15.46:0/3633104784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 msgr2=0x55b033fa98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.543+0000 7f9d4d242700 1 --2- 172.21.15.46:0/3633104784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 0x55b033fa98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.543+0000 7f9d4d242700 1 -- 172.21.15.46:0/3633104784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 msgr2=0x55b033fa93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.543+0000 7f9d4d242700 1 --2- 172.21.15.46:0/3633104784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 0x55b033fa93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.543+0000 7f9d4d242700 1 -- 172.21.15.46:0/3633104784 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b033ebbe40 con 0x55b033fa2570 2024-03-18T20:50:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.543+0000 7f9d4d242700 1 --2- 172.21.15.46:0/3633104784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fa54e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9d440974b0 tx=0x7f9d44005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cabcf2434abfa4d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.543+0000 7f9d33fff700 1 -- 172.21.15.46:0/3633104784 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44096bc0 con 0x55b033fa2570 2024-03-18T20:50:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.543+0000 7f9d33fff700 1 -- 172.21.15.46:0/3633104784 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4407d070 con 0x55b033fa2570 2024-03-18T20:50:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.543+0000 7f9d33fff700 1 -- 172.21.15.46:0/3633104784 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4409aab0 con 0x55b033fa2570 2024-03-18T20:50:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.544+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3633104784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 msgr2=0x55b033fa54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.544+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/3633104784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fa54e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9d440974b0 tx=0x7f9d44005b40 comp rx=0 tx=0).stop 2024-03-18T20:50:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.544+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3633104784 shutdown_connections 2024-03-18T20:50:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.544+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/3633104784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033faddb0 0x55b033fa98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.544+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/3633104784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033fa2570 0x55b033fa54e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.544+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/3633104784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2e90 0x55b033fa93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.544+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3633104784 >> 172.21.15.46:0/3633104784 conn(0x55b033f191c0 msgr2=0x55b033fa5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.544+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3633104784 shutdown_connections 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.545+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3633104784 wait complete. 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.545+0000 7f9d5b63bb80 1 -- start start 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.545+0000 7f9d5b63bb80 1 Processor -- start 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.545+0000 7f9d5b63bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2570 0x55b033fb23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.545+0000 7f9d5b63bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033fa2e90 0x55b033fb01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.545+0000 7f9d5b63bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033faddb0 0x55b033fb08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.545+0000 7f9d5b63bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b033fb0ff0 con 0x55b033fa2570 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.545+0000 7f9d5b63bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b034021060 con 0x55b033fa2e90 2024-03-18T20:50:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.545+0000 7f9d5b63bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0340211a0 con 0x55b033faddb0 2024-03-18T20:50:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.545+0000 7f9d5b63bb80 10 client.0 _ll_register_callbacks cb 0x7f9d4409a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:50:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.546+0000 7f9d4d242700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2570 0x55b033fb23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.546+0000 7f9d4d242700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2570 0x55b033fb23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33598/0 (socket says 172.21.15.46:33598) 2024-03-18T20:50:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.546+0000 7f9d4d242700 1 -- 172.21.15.46:0/3850333915 learned_addr learned my addr 172.21.15.46:0/3850333915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.546+0000 7f9d4da43700 1 --2- 172.21.15.46:0/3850333915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033faddb0 0x55b033fb08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.546+0000 7f9d4d242700 1 -- 172.21.15.46:0/3850333915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033faddb0 msgr2=0x55b033fb08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.546+0000 7f9d4d242700 1 --2- 172.21.15.46:0/3850333915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033faddb0 0x55b033fb08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.546+0000 7f9d4d242700 1 -- 172.21.15.46:0/3850333915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033fa2e90 msgr2=0x55b033fb01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.546+0000 7f9d4d242700 1 --2- 172.21.15.46:0/3850333915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033fa2e90 0x55b033fb01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.546+0000 7f9d4d242700 1 -- 172.21.15.46:0/3850333915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b033d23b00 con 0x55b033fa2570 2024-03-18T20:50:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.546+0000 7f9d4d242700 1 --2- 172.21.15.46:0/3850333915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2570 0x55b033fb23b0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9d440a2900 tx=0x7f9d4400bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e3b879fb3a6b6e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.546+0000 7f9d30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.547+0000 7f9d30ff9700 1 -- 172.21.15.46:0/3850333915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4409a140 con 0x55b033fa2570 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.547+0000 7f9d30ff9700 1 -- 172.21.15.46:0/3850333915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f9d4407e080 con 0x55b033fa2570 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.547+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3850333915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d4409e870 con 0x55b033fa2570 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.547+0000 7f9d30ff9700 1 -- 172.21.15.46:0/3850333915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d44096bc0 con 0x55b033fa2570 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.547+0000 7f9d5b63bb80 10 client.20268 fetch_fsmap learned FSMap version 65 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.547+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3850333915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x55b033d7bc70 con 0x55b033fa2570 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.547+0000 7f9d30ff9700 1 -- 172.21.15.46:0/3850333915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4409e870 con 0x55b033fa2570 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.547+0000 7f9d30ff9700 1 -- 172.21.15.46:0/3850333915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 65) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f9d4400b580 con 0x55b033fa2570 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.547+0000 7f9d5b63bb80 10 client.20268 fetch_fsmap finished waiting for FSMap version 65 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.547+0000 7f9d5b63bb80 10 client.20268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.547+0000 7f9d5b63bb80 20 client.20268 populate_metadata read hostname 'smithi046' 2024-03-18T20:50:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.548+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/3850333915 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b034022470 0x55b034024860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.548+0000 7f9d5b63bb80 4 client.20268 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:50:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.548+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3850333915 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b033d44c90 con 0x55b034022470 2024-03-18T20:50:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.549+0000 7f9d4ca41700 1 --2- 172.21.15.46:0/3850333915 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b034022470 0x55b034024860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.549+0000 7f9d4ca41700 1 --2- 172.21.15.46:0/3850333915 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b034022470 0x55b034024860 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd9033e2d63f9c60 server_cookie=88119b69579ffe77 in_seq=0 out_seq=0 2024-03-18T20:50:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.549+0000 7f9d30ff9700 10 client.20268 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:44.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:44 smithi073 ceph-mon[47384]: pgmap v458: 97 pgs: 97 active+clean; 808 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 1.10k op/s 2024-03-18T20:50:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.755+0000 7f9d30ff9700 1 -- 172.21.15.46:0/3850333915 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b033d44c90 con 0x55b034022470 2024-03-18T20:50:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.755+0000 7f9d30ff9700 10 client.20268 handle_command_reply: tid=0 2024-03-18T20:50:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.755+0000 7f9d5b63bb80 1 client.20268 shutdown 2024-03-18T20:50:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.755+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3850333915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2570 msgr2=0x55b033fb23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.755+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/3850333915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2570 0x55b033fb23b0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9d440a2900 tx=0x7f9d4400bc30 comp rx=0 tx=0).stop 2024-03-18T20:50:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.755+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3850333915 shutdown_connections 2024-03-18T20:50:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.755+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/3850333915 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b034022470 0x55b034024860 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.755+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/3850333915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b033fa2570 0x55b033fb23b0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.755+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/3850333915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b033faddb0 0x55b033fb08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.755+0000 7f9d30ff9700 0 client.20268 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.755+0000 7f9d5b63bb80 1 --2- 172.21.15.46:0/3850333915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b033fa2e90 0x55b033fb01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.755+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3850333915 >> 172.21.15.46:0/3850333915 conn(0x55b033f191c0 msgr2=0x55b033fa9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.756+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3850333915 shutdown_connections 2024-03-18T20:50:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.756+0000 7f9d5b63bb80 1 -- 172.21.15.46:0/3850333915 wait complete. 2024-03-18T20:50:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.756+0000 7f9d5b63bb80 20 client.20268 trim_cache size 0 max 16384 2024-03-18T20:50:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:50:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:50:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:50:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:50:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.756+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/757259868 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9d340a1310 msgr2=0x7f9d340a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.756+0000 7f9d4f4a6700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9d340a1310 0x7f9d340a37d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9d48198720 tx=0x7f9d38009310 comp rx=0 tx=0).stop 2024-03-18T20:50:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.756+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/757259868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d480ffe00 msgr2=0x7f9d481971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.756+0000 7f9d4f4a6700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d480ffe00 0x7f9d481971d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400fa00 tx=0x7f9d44010f90 comp rx=0 tx=0).stop 2024-03-18T20:50:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.757+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/757259868 shutdown_connections 2024-03-18T20:50:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.757+0000 7f9d4f4a6700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9d340a1310 0x7f9d340a37d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.757+0000 7f9d4f4a6700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d4819bab0 0x7f9d4819bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.757+0000 7f9d4f4a6700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d480ffe00 0x7f9d481971d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.757+0000 7f9d4f4a6700 1 --2- 172.21.15.46:0/757259868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d48104f20 0x7f9d48197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.757+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/757259868 >> 172.21.15.46:0/757259868 conn(0x7f9d480fb680 msgr2=0x7f9d480fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.758+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/757259868 shutdown_connections 2024-03-18T20:50:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:44.758+0000 7f9d4f4a6700 1 -- 172.21.15.46:0/757259868 wait complete. 2024-03-18T20:50:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:44 smithi155 ceph-mon[48344]: pgmap v458: 97 pgs: 97 active+clean; 808 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 1.10k op/s 2024-03-18T20:50:44.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:44 smithi046 ceph-mon[43495]: pgmap v458: 97 pgs: 97 active+clean; 808 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 1.10k op/s 2024-03-18T20:50:45.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 80750 55.85 MB/sec warmup 81 sec latency 254.062 ms 2024-03-18T20:50:46.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 80767 55.17 MB/sec warmup 82 sec latency 946.859 ms 2024-03-18T20:50:46.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:46 smithi073 ceph-mon[47384]: pgmap v459: 97 pgs: 97 active+clean; 808 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.4 MiB/s wr, 665 op/s 2024-03-18T20:50:46.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:46 smithi155 ceph-mon[48344]: pgmap v459: 97 pgs: 97 active+clean; 808 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.4 MiB/s wr, 665 op/s 2024-03-18T20:50:46.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:46 smithi046 ceph-mon[43495]: pgmap v459: 97 pgs: 97 active+clean; 808 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.4 MiB/s wr, 665 op/s 2024-03-18T20:50:47.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 81165 54.89 MB/sec warmup 83 sec latency 1222.321 ms 2024-03-18T20:50:48.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 81575 54.49 MB/sec warmup 84 sec latency 69.712 ms 2024-03-18T20:50:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:48 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:50:48.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:48 smithi073 ceph-mon[47384]: pgmap v460: 97 pgs: 97 active+clean; 808 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.4 MiB/s wr, 665 op/s 2024-03-18T20:50:48.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:48 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:50:48.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:48 smithi155 ceph-mon[48344]: pgmap v460: 97 pgs: 97 active+clean; 808 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.4 MiB/s wr, 665 op/s 2024-03-18T20:50:48.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:48 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:50:48.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:48 smithi046 ceph-mon[43495]: pgmap v460: 97 pgs: 97 active+clean; 808 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.4 MiB/s wr, 665 op/s 2024-03-18T20:50:49.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 82494 54.35 MB/sec warmup 85 sec latency 39.925 ms 2024-03-18T20:50:50.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 83750 54.42 MB/sec warmup 86 sec latency 58.717 ms 2024-03-18T20:50:50.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:50 smithi073 ceph-mon[47384]: pgmap v461: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 28 MiB/s wr, 1.53k op/s 2024-03-18T20:50:50.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:50 smithi155 ceph-mon[48344]: pgmap v461: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 28 MiB/s wr, 1.53k op/s 2024-03-18T20:50:50.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:50 smithi046 ceph-mon[43495]: pgmap v461: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 28 MiB/s wr, 1.53k op/s 2024-03-18T20:50:51.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 84287 54.30 MB/sec warmup 87 sec latency 316.096 ms 2024-03-18T20:50:52.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 84935 54.00 MB/sec warmup 88 sec latency 89.612 ms 2024-03-18T20:50:52.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:52 smithi073 ceph-mon[47384]: pgmap v462: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 25 MiB/s wr, 1.20k op/s 2024-03-18T20:50:52.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:52 smithi155 ceph-mon[48344]: pgmap v462: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 25 MiB/s wr, 1.20k op/s 2024-03-18T20:50:52.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:52 smithi046 ceph-mon[43495]: pgmap v462: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 25 MiB/s wr, 1.20k op/s 2024-03-18T20:50:53.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 85847 53.92 MB/sec warmup 89 sec latency 95.184 ms 2024-03-18T20:50:54.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 86925 53.99 MB/sec warmup 90 sec latency 52.585 ms 2024-03-18T20:50:54.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:54 smithi073 ceph-mon[47384]: pgmap v463: 97 pgs: 97 active+clean; 813 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 1.73k op/s 2024-03-18T20:50:54.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:54 smithi155 ceph-mon[48344]: pgmap v463: 97 pgs: 97 active+clean; 813 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 1.73k op/s 2024-03-18T20:50:54.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:54 smithi046 ceph-mon[43495]: pgmap v463: 97 pgs: 97 active+clean; 813 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 1.73k op/s 2024-03-18T20:50:55.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 87942 54.04 MB/sec warmup 91 sec latency 86.559 ms 2024-03-18T20:50:56.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 88983 54.09 MB/sec warmup 92 sec latency 98.244 ms 2024-03-18T20:50:56.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:56 smithi073 ceph-mon[47384]: pgmap v464: 97 pgs: 97 active+clean; 813 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 30 MiB/s wr, 1.40k op/s 2024-03-18T20:50:56.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:56 smithi155 ceph-mon[48344]: pgmap v464: 97 pgs: 97 active+clean; 813 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 30 MiB/s wr, 1.40k op/s 2024-03-18T20:50:56.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:56 smithi046 ceph-mon[43495]: pgmap v464: 97 pgs: 97 active+clean; 813 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 30 MiB/s wr, 1.40k op/s 2024-03-18T20:50:57.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 89895 53.94 MB/sec warmup 93 sec latency 69.661 ms 2024-03-18T20:50:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.943+0000 7f8dcba8d700 1 -- 172.21.15.46:0/3040806778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc410f940 msgr2=0x7f8dc41082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.943+0000 7f8dcba8d700 1 --2- 172.21.15.46:0/3040806778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc410f940 0x7f8dc41082f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8db40098f0 tx=0x7f8db4009c00 comp rx=0 tx=0).stop 2024-03-18T20:50:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.946+0000 7f8dcba8d700 1 -- 172.21.15.46:0/3040806778 shutdown_connections 2024-03-18T20:50:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.946+0000 7f8dcba8d700 1 --2- 172.21.15.46:0/3040806778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc411a590 0x7f8dc411c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.946+0000 7f8dcba8d700 1 --2- 172.21.15.46:0/3040806778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dc410f4d0 0x7f8dc4107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.946+0000 7f8dcba8d700 1 --2- 172.21.15.46:0/3040806778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc410f940 0x7f8dc41082f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.946+0000 7f8dcba8d700 1 -- 172.21.15.46:0/3040806778 >> 172.21.15.46:0/3040806778 conn(0x7f8dc406bd80 msgr2=0x7f8dc406c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.946+0000 7f8dcba8d700 1 -- 172.21.15.46:0/3040806778 shutdown_connections 2024-03-18T20:50:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.946+0000 7f8dcba8d700 1 -- 172.21.15.46:0/3040806778 wait complete. 2024-03-18T20:50:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.947+0000 7f8dcba8d700 1 Processor -- start 2024-03-18T20:50:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.947+0000 7f8dcba8d700 1 -- start start 2024-03-18T20:50:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.947+0000 7f8dcba8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc410f4d0 0x7f8dc4115e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.947+0000 7f8dcba8d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc410f940 0x7f8dc4116350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.947+0000 7f8dcba8d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dc411a590 0x7f8dc41a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.947+0000 7f8dcba8d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc41a2900 con 0x7f8dc410f4d0 2024-03-18T20:50:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.947+0000 7f8dcba8d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc41a2a70 con 0x7f8dc410f940 2024-03-18T20:50:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.947+0000 7f8dcba8d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc41a2c30 con 0x7f8dc411a590 2024-03-18T20:50:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.948+0000 7f8dc9028700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc410f940 0x7f8dc4116350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.948+0000 7f8dca02a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dc411a590 0x7f8dc41a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.948+0000 7f8dca02a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dc411a590 0x7f8dc41a2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33038/0 (socket says 172.21.15.46:33038) 2024-03-18T20:50:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.948+0000 7f8dca02a700 1 -- 172.21.15.46:0/1348084558 learned_addr learned my addr 172.21.15.46:0/1348084558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.948+0000 7f8dc9829700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc410f4d0 0x7f8dc4115e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.948+0000 7f8dc9028700 1 -- 172.21.15.46:0/1348084558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dc411a590 msgr2=0x7f8dc41a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.948+0000 7f8dc9028700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dc411a590 0x7f8dc41a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.948+0000 7f8dc9028700 1 -- 172.21.15.46:0/1348084558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc410f4d0 msgr2=0x7f8dc4115e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.948+0000 7f8dc9028700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc410f4d0 0x7f8dc4115e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.948+0000 7f8dc9028700 1 -- 172.21.15.46:0/1348084558 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc01ad10 con 0x7f8dc410f940 2024-03-18T20:50:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.949+0000 7f8dc9028700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc410f940 0x7f8dc4116350 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8dc410fd40 tx=0x7f8db400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.949+0000 7f8dbaffd700 1 -- 172.21.15.46:0/1348084558 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db401c070 con 0x7f8dc410f940 2024-03-18T20:50:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.949+0000 7f8dcba8d700 1 -- 172.21.15.46:0/1348084558 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db4009520 con 0x7f8dc410f940 2024-03-18T20:50:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.949+0000 7f8dbaffd700 1 -- 172.21.15.46:0/1348084558 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8db400ba20 con 0x7f8dc410f940 2024-03-18T20:50:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.949+0000 7f8dbaffd700 1 -- 172.21.15.46:0/1348084558 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db40038e0 con 0x7f8dc410f940 2024-03-18T20:50:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.949+0000 7f8dcba8d700 1 -- 172.21.15.46:0/1348084558 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc4079010 con 0x7f8dc410f940 2024-03-18T20:50:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.953+0000 7f8dbaffd700 1 -- 172.21.15.46:0/1348084558 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8db4020030 con 0x7f8dc410f940 2024-03-18T20:50:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.953+0000 7f8dd7c22b80 1 Processor -- start 2024-03-18T20:50:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.954+0000 7f8dbaffd700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8db00a1290 0x7f8db00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.954+0000 7f8dbaffd700 1 -- 172.21.15.46:0/1348084558 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8db40cd9d0 con 0x7f8dc410f940 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.954+0000 7f8dd7c22b80 1 -- start start 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.954+0000 7f8dd7c22b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be090570 0x5613be090950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.954+0000 7f8dd7c22b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 0x5613be09b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.954+0000 7f8dd7c22b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be09bdb0 0x5613be09e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dd7c22b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bdfea930 con 0x5613be090570 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dd7c22b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bdfe82f0 con 0x5613be09bdb0 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dd7c22b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bdfe3be0 con 0x5613be090e90 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dc9028700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 0x5613be09b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dc9028700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 0x5613be09b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33048/0 (socket says 172.21.15.46:33048) 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dc9028700 1 -- 172.21.15.46:0/3121995456 learned_addr learned my addr 172.21.15.46:0/3121995456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dc9028700 1 -- 172.21.15.46:0/3121995456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be09bdb0 msgr2=0x5613be09e1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dc9028700 1 --2- 172.21.15.46:0/3121995456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be09bdb0 0x5613be09e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dc9028700 1 -- 172.21.15.46:0/3121995456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be090570 msgr2=0x5613be090950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:50:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dc9028700 1 --2- 172.21.15.46:0/3121995456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be090570 0x5613be090950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dc9028700 1 -- 172.21.15.46:0/3121995456 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613bde65350 con 0x5613be090e90 2024-03-18T20:50:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dc9028700 1 --2- 172.21.15.46:0/3121995456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 0x5613be09b870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8db40a2a10 tx=0x7f8db409a7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56fd9ab66b7b7c15 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.955+0000 7f8dca02a700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8db00a1290 0x7f8db00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.956+0000 7f8dca02a700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8db00a1290 0x7f8db00a3750 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc01ace0 tx=0x7f8dbc01c040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.957+0000 7f8daffff700 1 -- 172.21.15.46:0/3121995456 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db409b250 con 0x5613be090e90 2024-03-18T20:50:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.957+0000 7f8daffff700 1 -- 172.21.15.46:0/3121995456 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8db409ab50 con 0x5613be090e90 2024-03-18T20:50:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.957+0000 7f8daffff700 1 -- 172.21.15.46:0/3121995456 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db4097260 con 0x5613be090e90 2024-03-18T20:50:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.958+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/3121995456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 msgr2=0x5613be09b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.958+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/3121995456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 0x5613be09b870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8db40a2a10 tx=0x7f8db409a7c0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.958+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/3121995456 shutdown_connections 2024-03-18T20:50:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.958+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/3121995456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be090570 0x5613be090950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.958+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/3121995456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be09bdb0 0x5613be09e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.958+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/3121995456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 0x5613be09b870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.958+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/3121995456 >> 172.21.15.46:0/3121995456 conn(0x5613be0071c0 msgr2=0x5613be08ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.958+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/3121995456 shutdown_connections 2024-03-18T20:50:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.958+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/3121995456 wait complete. 2024-03-18T20:50:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.959+0000 7f8dd7c22b80 1 -- start start 2024-03-18T20:50:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.959+0000 7f8dd7c22b80 1 Processor -- start 2024-03-18T20:50:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.959+0000 7f8dd7c22b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090570 0x5613be09b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.959+0000 7f8dd7c22b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be090e90 0x5613be098b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.959+0000 7f8dd7c22b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be09bdb0 0x5613be099140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.959+0000 7f8dd7c22b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bdf7fd20 con 0x5613be09bdb0 2024-03-18T20:50:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.959+0000 7f8dd7c22b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bdf7f9c0 con 0x5613be090e90 2024-03-18T20:50:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.959+0000 7f8dd7c22b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613bde67640 con 0x5613be090570 2024-03-18T20:50:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.959+0000 7f8dd7c22b80 10 client.0 _ll_register_callbacks cb 0x5613bddeb340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.960+0000 7f8dc9028700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be090e90 0x5613be098b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.960+0000 7f8dc9028700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be090e90 0x5613be098b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:39540/0 (socket says 172.21.15.46:39540) 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.960+0000 7f8dc9028700 1 -- 172.21.15.46:0/95192823 learned_addr learned my addr 172.21.15.46:0/95192823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.960+0000 7f8dc9829700 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090570 0x5613be09b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.960+0000 7f8dc9028700 1 -- 172.21.15.46:0/95192823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090570 msgr2=0x5613be09b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.960+0000 7f8dc9028700 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090570 0x5613be09b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.960+0000 7f8dca02a700 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be09bdb0 0x5613be099140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.960+0000 7f8dc9028700 1 -- 172.21.15.46:0/95192823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be09bdb0 msgr2=0x5613be099140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.960+0000 7f8dc9028700 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be09bdb0 0x5613be099140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.961+0000 7f8dc9028700 1 -- 172.21.15.46:0/95192823 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc4079010 con 0x5613be090e90 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.961+0000 7f8dca02a700 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be09bdb0 0x5613be099140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.961+0000 7f8dc9028700 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be090e90 0x5613be098b10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8db40975c0 tx=0x7f8db409a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d411f8f27f99b95 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.961+0000 7f8dad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:50:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.961+0000 7f8dad7fa700 1 -- 172.21.15.46:0/95192823 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db40cb380 con 0x5613be090e90 2024-03-18T20:50:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.961+0000 7f8dad7fa700 1 -- 172.21.15.46:0/95192823 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8db40cb4e0 con 0x5613be090e90 2024-03-18T20:50:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.961+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/95192823 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613be0846c0 con 0x5613be090e90 2024-03-18T20:50:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.962+0000 7f8dad7fa700 1 -- 172.21.15.46:0/95192823 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8db4097260 con 0x5613be090e90 2024-03-18T20:50:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.962+0000 7f8dd7c22b80 10 client.29855 fetch_fsmap learned FSMap version 66 2024-03-18T20:50:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.962+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/95192823 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x5613bde11b00 con 0x5613be090e90 2024-03-18T20:50:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.963+0000 7f8dad7fa700 1 -- 172.21.15.46:0/95192823 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5613be0846c0 con 0x5613be090e90 2024-03-18T20:50:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.963+0000 7f8dad7fa700 1 -- 172.21.15.46:0/95192823 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 66) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f8db409adb0 con 0x5613be090e90 2024-03-18T20:50:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.963+0000 7f8dd7c22b80 10 client.29855 fetch_fsmap finished waiting for FSMap version 66 2024-03-18T20:50:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.963+0000 7f8dd7c22b80 10 client.29855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:50:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.963+0000 7f8dd7c22b80 20 client.29855 populate_metadata read hostname 'smithi046' 2024-03-18T20:50:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.963+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5613be0ab140 0x5613be0ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.963+0000 7f8dd7c22b80 4 client.29855 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:50:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.963+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/95192823 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613bdfdc6a0 con 0x5613be0ab140 2024-03-18T20:50:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.963+0000 7f8dc9829700 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5613be0ab140 0x5613be0ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.964+0000 7f8dc9829700 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5613be0ab140 0x5613be0ad530 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733f097512115947 server_cookie=9cb99a7047b0a253 in_seq=0 out_seq=0 2024-03-18T20:50:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.964+0000 7f8dad7fa700 10 client.29855 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dad7fa700 1 -- 172.21.15.46:0/95192823 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5613bdfdc6a0 con 0x5613be0ab140 2024-03-18T20:50:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dad7fa700 10 client.29855 handle_command_reply: tid=0 2024-03-18T20:50:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dd7c22b80 1 client.29855 shutdown 2024-03-18T20:50:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/95192823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be090e90 msgr2=0x5613be098b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be090e90 0x5613be098b10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8db40975c0 tx=0x7f8db409a5d0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/95192823 shutdown_connections 2024-03-18T20:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5613be0ab140 0x5613be0ad530 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be09bdb0 0x5613be099140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090570 0x5613be09b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/95192823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be090e90 0x5613be098b10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/95192823 >> 172.21.15.46:0/95192823 conn(0x5613be0071c0 msgr2=0x5613be08c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dad7fa700 0 client.29855 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/95192823 shutdown_connections 2024-03-18T20:50:57.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/95192823 wait complete. 2024-03-18T20:50:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.966+0000 7f8dd7c22b80 20 client.29855 trim_cache size 0 max 16384 2024-03-18T20:50:57.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.994+0000 7f8dd7c22b80 1 Processor -- start 2024-03-18T20:50:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.994+0000 7f8dd7c22b80 1 -- start start 2024-03-18T20:50:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.994+0000 7f8dd7c22b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be090570 0x5613be099100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.994+0000 7f8dd7c22b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 0x5613be0a2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.994+0000 7f8dd7c22b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be09bdb0 0x5613be0973e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.994+0000 7f8dd7c22b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613be091850 con 0x5613be09bdb0 2024-03-18T20:50:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.994+0000 7f8dd7c22b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613be0891d0 con 0x5613be090570 2024-03-18T20:50:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.994+0000 7f8dd7c22b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613be0934e0 con 0x5613be090e90 2024-03-18T20:50:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.995+0000 7f8dc9028700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 0x5613be0a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.995+0000 7f8dc9028700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 0x5613be0a2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33068/0 (socket says 172.21.15.46:33068) 2024-03-18T20:50:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.995+0000 7f8dc9028700 1 -- 172.21.15.46:0/4017298918 learned_addr learned my addr 172.21.15.46:0/4017298918 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.995+0000 7f8dc9829700 1 --2- 172.21.15.46:0/4017298918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be090570 0x5613be099100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.995+0000 7f8dc9028700 1 -- 172.21.15.46:0/4017298918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be090570 msgr2=0x5613be099100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.995+0000 7f8dc9028700 1 --2- 172.21.15.46:0/4017298918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be090570 0x5613be099100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.995+0000 7f8dc9028700 1 -- 172.21.15.46:0/4017298918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be09bdb0 msgr2=0x5613be0973e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:50:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.995+0000 7f8dc9028700 1 --2- 172.21.15.46:0/4017298918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be09bdb0 0x5613be0973e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.995+0000 7f8dc9028700 1 -- 172.21.15.46:0/4017298918 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613bdfa9e40 con 0x5613be090e90 2024-03-18T20:50:57.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.996+0000 7f8dc9028700 1 --2- 172.21.15.46:0/4017298918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 0x5613be0a2770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8db4098ea0 tx=0x7f8db4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a16549f97c2c313 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.996+0000 7f8db8da8700 1 -- 172.21.15.46:0/4017298918 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db409ea00 con 0x5613be090e90 2024-03-18T20:50:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.996+0000 7f8db8da8700 1 -- 172.21.15.46:0/4017298918 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8db40a2f40 con 0x5613be090e90 2024-03-18T20:50:58.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.996+0000 7f8db8da8700 1 -- 172.21.15.46:0/4017298918 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db409a110 con 0x5613be090e90 2024-03-18T20:50:58.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.997+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/4017298918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 msgr2=0x5613be0a2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.997+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/4017298918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 0x5613be0a2770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8db4098ea0 tx=0x7f8db4009c00 comp rx=0 tx=0).stop 2024-03-18T20:50:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.998+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/4017298918 shutdown_connections 2024-03-18T20:50:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.998+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/4017298918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be09bdb0 0x5613be0973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.998+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/4017298918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be090570 0x5613be099100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.998+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/4017298918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be090e90 0x5613be0a2770 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8db4098ea0 tx=0x7f8db4009c00 comp rx=0 tx=0).stop 2024-03-18T20:50:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.998+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/4017298918 >> 172.21.15.46:0/4017298918 conn(0x5613be0071c0 msgr2=0x5613be08b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.999+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/4017298918 shutdown_connections 2024-03-18T20:50:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:57.999+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/4017298918 wait complete. 2024-03-18T20:50:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.000+0000 7f8dd7c22b80 1 -- start start 2024-03-18T20:50:58.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.000+0000 7f8dd7c22b80 1 Processor -- start 2024-03-18T20:50:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.000+0000 7f8dd7c22b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be090570 0x5613be0a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.000+0000 7f8dd7c22b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be09bdb0 0x5613be0a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.000+0000 7f8dd7c22b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be0fbe90 0x5613be0fe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.000+0000 7f8dd7c22b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613be0a10f0 con 0x5613be090570 2024-03-18T20:50:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.000+0000 7f8dd7c22b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613be0fe7c0 con 0x5613be0fbe90 2024-03-18T20:50:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.000+0000 7f8dd7c22b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613be0fe900 con 0x5613be09bdb0 2024-03-18T20:50:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.000+0000 7f8dd7c22b80 10 client.0 _ll_register_callbacks cb 0x7f8db409a690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:50:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.001+0000 7f8dc9028700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be09bdb0 0x5613be0a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.001+0000 7f8dc9028700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be09bdb0 0x5613be0a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33078/0 (socket says 172.21.15.46:33078) 2024-03-18T20:50:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.001+0000 7f8dc9028700 1 -- 172.21.15.46:0/2623976998 learned_addr learned my addr 172.21.15.46:0/2623976998 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:50:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.001+0000 7f8dc9829700 1 --2- 172.21.15.46:0/2623976998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be090570 0x5613be0a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:58.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.001+0000 7f8dc9028700 1 -- 172.21.15.46:0/2623976998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be0fbe90 msgr2=0x5613be0fe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.001+0000 7f8dc9028700 1 --2- 172.21.15.46:0/2623976998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be0fbe90 0x5613be0fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.001+0000 7f8dc9028700 1 -- 172.21.15.46:0/2623976998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be090570 msgr2=0x5613be0a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:50:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.001+0000 7f8dc9028700 1 --2- 172.21.15.46:0/2623976998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be090570 0x5613be0a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:58.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.001+0000 7f8dc9028700 1 -- 172.21.15.46:0/2623976998 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613bde11b00 con 0x5613be09bdb0 2024-03-18T20:50:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.001+0000 7f8dc9028700 1 --2- 172.21.15.46:0/2623976998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be09bdb0 0x5613be0a0a30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8db407b040 tx=0x7f8db4005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69da1462b17a00cb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:50:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.001+0000 7f8dad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:50:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.002+0000 7f8dad7fa700 1 -- 172.21.15.46:0/2623976998 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db409e980 con 0x5613be09bdb0 2024-03-18T20:50:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.002+0000 7f8dad7fa700 1 -- 172.21.15.46:0/2623976998 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8db40a4020 con 0x5613be09bdb0 2024-03-18T20:50:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.002+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/2623976998 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8db409ab50 con 0x5613be09bdb0 2024-03-18T20:50:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.002+0000 7f8dad7fa700 1 -- 172.21.15.46:0/2623976998 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8db40840e0 con 0x5613be09bdb0 2024-03-18T20:50:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.002+0000 7f8dd7c22b80 10 client.29938 fetch_fsmap learned FSMap version 66 2024-03-18T20:50:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.002+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/2623976998 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x5613bde69c70 con 0x5613be09bdb0 2024-03-18T20:50:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.003+0000 7f8dad7fa700 1 -- 172.21.15.46:0/2623976998 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8db409ab50 con 0x5613be09bdb0 2024-03-18T20:50:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.003+0000 7f8dad7fa700 1 -- 172.21.15.46:0/2623976998 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 66) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f8db4005040 con 0x5613be09bdb0 2024-03-18T20:50:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.003+0000 7f8dd7c22b80 10 client.29938 fetch_fsmap finished waiting for FSMap version 66 2024-03-18T20:50:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.003+0000 7f8dd7c22b80 10 client.29938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:50:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.003+0000 7f8dd7c22b80 20 client.29938 populate_metadata read hostname 'smithi046' 2024-03-18T20:50:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.003+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/2623976998 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5613be0ffb50 0x5613be101f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:50:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.003+0000 7f8dd7c22b80 4 client.29938 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:50:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.003+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/2623976998 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5613bde32c90 con 0x5613be0ffb50 2024-03-18T20:50:58.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.006+0000 7f8dc9829700 1 --2- 172.21.15.46:0/2623976998 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5613be0ffb50 0x5613be101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:50:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.010+0000 7f8dc9829700 1 --2- 172.21.15.46:0/2623976998 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5613be0ffb50 0x5613be101f40 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c05114ab1506d030 server_cookie=2c71f528a9654818 in_seq=0 out_seq=0 2024-03-18T20:50:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.010+0000 7f8dad7fa700 10 client.29938 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.190+0000 7f8dad7fa700 1 -- 172.21.15.46:0/2623976998 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5613bde32c90 con 0x5613be0ffb50 2024-03-18T20:50:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.190+0000 7f8dad7fa700 10 client.29938 handle_command_reply: tid=0 2024-03-18T20:50:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.193+0000 7f8dd7c22b80 1 client.29938 shutdown 2024-03-18T20:50:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.194+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/2623976998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be09bdb0 msgr2=0x5613be0a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.194+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/2623976998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be09bdb0 0x5613be0a0a30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8db407b040 tx=0x7f8db4005b40 comp rx=0 tx=0).stop 2024-03-18T20:50:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.196+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/2623976998 shutdown_connections 2024-03-18T20:50:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.196+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/2623976998 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5613be0ffb50 0x5613be101f40 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.196+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/2623976998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613be090570 0x5613be0a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.196+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/2623976998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613be0fbe90 0x5613be0fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.196+0000 7f8dd7c22b80 1 --2- 172.21.15.46:0/2623976998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613be09bdb0 0x5613be0a0a30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.196+0000 7f8dad7fa700 0 client.29938 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:50:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.196+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/2623976998 >> 172.21.15.46:0/2623976998 conn(0x5613be0071c0 msgr2=0x5613be0973a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.196+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/2623976998 shutdown_connections 2024-03-18T20:50:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.196+0000 7f8dd7c22b80 1 -- 172.21.15.46:0/2623976998 wait complete. 2024-03-18T20:50:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.197+0000 7f8dd7c22b80 20 client.29938 trim_cache size 0 max 16384 2024-03-18T20:50:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:50:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:50:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:50:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:50:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.198+0000 7f8dcba8d700 1 -- 172.21.15.46:0/1348084558 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8db00a1290 msgr2=0x7f8db00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.198+0000 7f8dcba8d700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8db00a1290 0x7f8db00a3750 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc01ace0 tx=0x7f8dbc01c040 comp rx=0 tx=0).stop 2024-03-18T20:50:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.198+0000 7f8dcba8d700 1 -- 172.21.15.46:0/1348084558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc410f940 msgr2=0x7f8dc4116350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:50:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.198+0000 7f8dcba8d700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc410f940 0x7f8dc4116350 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8dc410fd40 tx=0x7f8db400fa00 comp rx=0 tx=0).stop 2024-03-18T20:50:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.199+0000 7f8dcba8d700 1 -- 172.21.15.46:0/1348084558 shutdown_connections 2024-03-18T20:50:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.199+0000 7f8dcba8d700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8db00a1290 0x7f8db00a3750 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.199+0000 7f8dcba8d700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc410f4d0 0x7f8dc4115e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.199+0000 7f8dcba8d700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8dc411a590 0x7f8dc41a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:50:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.199+0000 7f8dcba8d700 1 --2- 172.21.15.46:0/1348084558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc410f940 0x7f8dc4116350 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8dc410fd40 tx=0x7f8db400fa00 comp rx=0 tx=0).stop 2024-03-18T20:50:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.199+0000 7f8dcba8d700 1 -- 172.21.15.46:0/1348084558 >> 172.21.15.46:0/1348084558 conn(0x7f8dc406bd80 msgr2=0x7f8dc410c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:50:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.201+0000 7f8dcba8d700 1 -- 172.21.15.46:0/1348084558 shutdown_connections 2024-03-18T20:50:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:50:58.201+0000 7f8dcba8d700 1 -- 172.21.15.46:0/1348084558 wait complete. 2024-03-18T20:50:58.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 90911 54.00 MB/sec warmup 94 sec latency 133.240 ms 2024-03-18T20:50:58.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:50:58 smithi073 ceph-mon[47384]: pgmap v465: 97 pgs: 97 active+clean; 813 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 30 MiB/s wr, 1.40k op/s 2024-03-18T20:50:58.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:50:58 smithi155 ceph-mon[48344]: pgmap v465: 97 pgs: 97 active+clean; 813 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 30 MiB/s wr, 1.40k op/s 2024-03-18T20:50:58.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:50:58 smithi046 ceph-mon[43495]: pgmap v465: 97 pgs: 97 active+clean; 813 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 30 MiB/s wr, 1.40k op/s 2024-03-18T20:50:59.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 92009 54.03 MB/sec warmup 95 sec latency 29.487 ms 2024-03-18T20:51:00.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 92942 53.99 MB/sec warmup 96 sec latency 83.059 ms 2024-03-18T20:51:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:00 smithi073 ceph-mon[47384]: pgmap v466: 97 pgs: 97 active+clean; 848 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 1.77k op/s 2024-03-18T20:51:00.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:00 smithi155 ceph-mon[48344]: pgmap v466: 97 pgs: 97 active+clean; 848 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 1.77k op/s 2024-03-18T20:51:00.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:00 smithi046 ceph-mon[43495]: pgmap v466: 97 pgs: 97 active+clean; 848 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 33 MiB/s wr, 1.77k op/s 2024-03-18T20:51:01.362 INFO:tasks.workunit.client.0.smithi046.stdout: 10 93212 53.64 MB/sec warmup 97 sec latency 747.308 ms 2024-03-18T20:51:02.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 93302 53.12 MB/sec warmup 98 sec latency 1465.394 ms 2024-03-18T20:51:02.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:02 smithi073 ceph-mon[47384]: pgmap v467: 97 pgs: 97 active+clean; 848 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 901 op/s 2024-03-18T20:51:02.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:02 smithi155 ceph-mon[48344]: pgmap v467: 97 pgs: 97 active+clean; 848 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 901 op/s 2024-03-18T20:51:02.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:02 smithi046 ceph-mon[43495]: pgmap v467: 97 pgs: 97 active+clean; 848 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 11 MiB/s wr, 901 op/s 2024-03-18T20:51:03.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 93302 52.59 MB/sec warmup 99 sec latency 1170.733 ms 2024-03-18T20:51:04.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 93302 52.06 MB/sec warmup 100 sec latency 2170.817 ms 2024-03-18T20:51:04.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:04 smithi073 ceph-mon[47384]: pgmap v468: 97 pgs: 97 active+clean; 853 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 1.12k op/s 2024-03-18T20:51:04.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:04 smithi155 ceph-mon[48344]: pgmap v468: 97 pgs: 97 active+clean; 853 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 1.12k op/s 2024-03-18T20:51:04.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:04 smithi046 ceph-mon[43495]: pgmap v468: 97 pgs: 97 active+clean; 853 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 13 MiB/s wr, 1.12k op/s 2024-03-18T20:51:05.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 93302 51.55 MB/sec warmup 101 sec latency 3170.900 ms 2024-03-18T20:51:06.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 93302 51.04 MB/sec warmup 102 sec latency 4170.957 ms 2024-03-18T20:51:06.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:06 smithi073 ceph-mon[47384]: pgmap v469: 97 pgs: 97 active+clean; 853 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 586 op/s 2024-03-18T20:51:06.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:06 smithi155 ceph-mon[48344]: pgmap v469: 97 pgs: 97 active+clean; 853 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 586 op/s 2024-03-18T20:51:06.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:06 smithi046 ceph-mon[43495]: pgmap v469: 97 pgs: 97 active+clean; 853 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 586 op/s 2024-03-18T20:51:07.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 93814 50.78 MB/sec warmup 103 sec latency 4318.529 ms 2024-03-18T20:51:08.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 94559 50.67 MB/sec warmup 104 sec latency 56.933 ms 2024-03-18T20:51:08.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:08 smithi073 ceph-mon[47384]: pgmap v470: 97 pgs: 97 active+clean; 853 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 586 op/s 2024-03-18T20:51:08.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:08 smithi155 ceph-mon[48344]: pgmap v470: 97 pgs: 97 active+clean; 853 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 586 op/s 2024-03-18T20:51:08.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:08 smithi046 ceph-mon[43495]: pgmap v470: 97 pgs: 97 active+clean; 853 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 586 op/s 2024-03-18T20:51:09.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 95239 50.47 MB/sec warmup 105 sec latency 70.971 ms 2024-03-18T20:51:09.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:09 smithi073 ceph-mon[47384]: osdmap e99: 12 total, 12 up, 12 in 2024-03-18T20:51:09.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:09 smithi155 ceph-mon[48344]: osdmap e99: 12 total, 12 up, 12 in 2024-03-18T20:51:09.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:09 smithi046 ceph-mon[43495]: osdmap e99: 12 total, 12 up, 12 in 2024-03-18T20:51:10.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 96324 50.59 MB/sec warmup 106 sec latency 53.870 ms 2024-03-18T20:51:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:10 smithi073 ceph-mon[47384]: pgmap v472: 97 pgs: 97 active+clean; 872 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 727 op/s 2024-03-18T20:51:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:10 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:10.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:10 smithi155 ceph-mon[48344]: pgmap v472: 97 pgs: 97 active+clean; 872 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 727 op/s 2024-03-18T20:51:10.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:10 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:10.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:10 smithi046 ceph-mon[43495]: pgmap v472: 97 pgs: 97 active+clean; 872 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 727 op/s 2024-03-18T20:51:10.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:10 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:11.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 97268 50.71 MB/sec warmup 107 sec latency 43.434 ms 2024-03-18T20:51:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.373+0000 7f6d1a304700 1 -- 172.21.15.46:0/3534384544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d14107cb0 msgr2=0x7f6d141141a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.373+0000 7f6d1a304700 1 --2- 172.21.15.46:0/3534384544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d14107cb0 0x7f6d141141a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c01c0d0 tx=0x7f6d0c01c3e0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.374+0000 7f6d1a304700 1 -- 172.21.15.46:0/3534384544 shutdown_connections 2024-03-18T20:51:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.374+0000 7f6d1a304700 1 --2- 172.21.15.46:0/3534384544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d1410d370 0x7f6d1410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.374+0000 7f6d1a304700 1 --2- 172.21.15.46:0/3534384544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d14107cb0 0x7f6d141141a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.374+0000 7f6d1a304700 1 --2- 172.21.15.46:0/3534384544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d141072f0 0x7f6d14107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.374+0000 7f6d1a304700 1 -- 172.21.15.46:0/3534384544 >> 172.21.15.46:0/3534384544 conn(0x7f6d14076bd0 msgr2=0x7f6d14105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.374+0000 7f6d1a304700 1 -- 172.21.15.46:0/3534384544 shutdown_connections 2024-03-18T20:51:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.374+0000 7f6d1a304700 1 -- 172.21.15.46:0/3534384544 wait complete. 2024-03-18T20:51:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.374+0000 7f6d1a304700 1 Processor -- start 2024-03-18T20:51:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.374+0000 7f6d1a304700 1 -- start start 2024-03-18T20:51:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.375+0000 7f6d1a304700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d141072f0 0x7f6d14113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.375+0000 7f6d1a304700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d14107cb0 0x7f6d141140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.375+0000 7f6d1a304700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d1410d370 0x7f6d1410fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.375+0000 7f6d1a304700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d14110560 con 0x7f6d141072f0 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.375+0000 7f6d1a304700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d141106d0 con 0x7f6d1410d370 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.375+0000 7f6d1a304700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d14110890 con 0x7f6d14107cb0 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.375+0000 7f6d13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d141072f0 0x7f6d14113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.375+0000 7f6d13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d141072f0 0x7f6d14113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45606/0 (socket says 172.21.15.46:45606) 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.375+0000 7f6d13fff700 1 -- 172.21.15.46:0/1329410390 learned_addr learned my addr 172.21.15.46:0/1329410390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.375+0000 7f6d137fe700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d14107cb0 0x7f6d141140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d188a1700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d1410d370 0x7f6d1410fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d137fe700 1 -- 172.21.15.46:0/1329410390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d1410d370 msgr2=0x7f6d1410fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d137fe700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d1410d370 0x7f6d1410fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d137fe700 1 -- 172.21.15.46:0/1329410390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d141072f0 msgr2=0x7f6d14113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d137fe700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d141072f0 0x7f6d14113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d137fe700 1 -- 172.21.15.46:0/1329410390 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c01ad10 con 0x7f6d14107cb0 2024-03-18T20:51:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d137fe700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d14107cb0 0x7f6d141140d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800d950 tx=0x7f6d0800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d117fa700 1 -- 172.21.15.46:0/1329410390 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08009930 con 0x7f6d14107cb0 2024-03-18T20:51:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d117fa700 1 -- 172.21.15.46:0/1329410390 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d08010460 con 0x7f6d14107cb0 2024-03-18T20:51:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d117fa700 1 -- 172.21.15.46:0/1329410390 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800b5d0 con 0x7f6d14107cb0 2024-03-18T20:51:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d1a304700 1 -- 172.21.15.46:0/1329410390 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d141a1a50 con 0x7f6d14107cb0 2024-03-18T20:51:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d1a304700 1 -- 172.21.15.46:0/1329410390 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d141a1e20 con 0x7f6d14107cb0 2024-03-18T20:51:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.376+0000 7f6d13fff700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d141072f0 0x7f6d14113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:51:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.377+0000 7f6d188a1700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d1410d370 0x7f6d1410fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:51:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.379+0000 7f6d117fa700 1 -- 172.21.15.46:0/1329410390 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6d08010840 con 0x7f6d14107cb0 2024-03-18T20:51:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.380+0000 7f6d117fa700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6cfc0a1310 0x7f6cfc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.380+0000 7f6d117fa700 1 -- 172.21.15.46:0/1329410390 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f6d080cc880 con 0x7f6d14107cb0 2024-03-18T20:51:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.381+0000 7f6d13fff700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6cfc0a1310 0x7f6cfc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.381+0000 7f6d13fff700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6cfc0a1310 0x7f6cfc0a37d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6d04011550 tx=0x7f6d04009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.382+0000 7f6d26499b80 1 Processor -- start 2024-03-18T20:51:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.382+0000 7f6d26499b80 1 -- start start 2024-03-18T20:51:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.382+0000 7f6d26499b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1570 0x55d4f4be1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.382+0000 7f6d26499b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4bec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.383+0000 7f6d26499b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4bef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.383+0000 7f6d26499b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f4b3b930 con 0x55d4f4be1e90 2024-03-18T20:51:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.383+0000 7f6d26499b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f4b392f0 con 0x55d4f4becdb0 2024-03-18T20:51:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.383+0000 7f6d26499b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f4b34be0 con 0x55d4f4be1570 2024-03-18T20:51:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.383+0000 7f6d188a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4bef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.383+0000 7f6d188a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4bef1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:44894/0 (socket says 172.21.15.46:44894) 2024-03-18T20:51:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.383+0000 7f6d188a1700 1 -- 172.21.15.46:0/1312248079 learned_addr learned my addr 172.21.15.46:0/1312248079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.384+0000 7f6d188a1700 1 -- 172.21.15.46:0/1312248079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1570 msgr2=0x55d4f4be1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.384+0000 7f6d188a1700 1 --2- 172.21.15.46:0/1312248079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1570 0x55d4f4be1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.384+0000 7f6d188a1700 1 -- 172.21.15.46:0/1312248079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 msgr2=0x55d4f4bec870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.384+0000 7f6d188a1700 1 --2- 172.21.15.46:0/1312248079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4bec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.384+0000 7f6d188a1700 1 -- 172.21.15.46:0/1312248079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4f49b6350 con 0x55d4f4becdb0 2024-03-18T20:51:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.386+0000 7f6d188a1700 1 --2- 172.21.15.46:0/1312248079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4bef1a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c01a350 tx=0x7f6d0c01a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce6d24618534b97 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.386+0000 7f6cfa7fc700 1 -- 172.21.15.46:0/1312248079 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c01f040 con 0x55d4f4becdb0 2024-03-18T20:51:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.386+0000 7f6cfa7fc700 1 -- 172.21.15.46:0/1312248079 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d0c018be0 con 0x55d4f4becdb0 2024-03-18T20:51:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.386+0000 7f6cfa7fc700 1 -- 172.21.15.46:0/1312248079 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c018750 con 0x55d4f4becdb0 2024-03-18T20:51:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.387+0000 7f6d26499b80 1 -- 172.21.15.46:0/1312248079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 msgr2=0x55d4f4bef1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.387+0000 7f6d26499b80 1 --2- 172.21.15.46:0/1312248079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4bef1a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c01a350 tx=0x7f6d0c01a380 comp rx=0 tx=0).stop 2024-03-18T20:51:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.387+0000 7f6d26499b80 1 -- 172.21.15.46:0/1312248079 shutdown_connections 2024-03-18T20:51:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.387+0000 7f6d26499b80 1 --2- 172.21.15.46:0/1312248079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4bef1a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.387+0000 7f6d26499b80 1 --2- 172.21.15.46:0/1312248079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1570 0x55d4f4be1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.387+0000 7f6d26499b80 1 --2- 172.21.15.46:0/1312248079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4bec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.387+0000 7f6d26499b80 1 -- 172.21.15.46:0/1312248079 >> 172.21.15.46:0/1312248079 conn(0x55d4f4b581c0 msgr2=0x55d4f4bdda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.387+0000 7f6d26499b80 1 -- 172.21.15.46:0/1312248079 shutdown_connections 2024-03-18T20:51:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.387+0000 7f6d26499b80 1 -- 172.21.15.46:0/1312248079 wait complete. 2024-03-18T20:51:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.388+0000 7f6d26499b80 1 -- start start 2024-03-18T20:51:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.388+0000 7f6d26499b80 1 Processor -- start 2024-03-18T20:51:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.388+0000 7f6d26499b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1570 0x55d4f4be9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.388+0000 7f6d26499b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1e90 0x55d4f4bea1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.388+0000 7f6d26499b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4bea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.388+0000 7f6d26499b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f4ad0d20 con 0x55d4f4be1570 2024-03-18T20:51:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.388+0000 7f6d26499b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f4ad09c0 con 0x55d4f4becdb0 2024-03-18T20:51:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.388+0000 7f6d26499b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f49b8640 con 0x55d4f4be1e90 2024-03-18T20:51:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.389+0000 7f6d26499b80 10 client.0 _ll_register_callbacks cb 0x55d4f493c340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:51:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.389+0000 7f6d13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1570 0x55d4f4be9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.389+0000 7f6d188a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4bea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.389+0000 7f6d13fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1570 0x55d4f4be9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45634/0 (socket says 172.21.15.46:45634) 2024-03-18T20:51:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.389+0000 7f6d188a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4bea700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:44910/0 (socket says 172.21.15.46:44910) 2024-03-18T20:51:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.389+0000 7f6d13fff700 1 -- 172.21.15.46:0/2145285666 learned_addr learned my addr 172.21.15.46:0/2145285666 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.389+0000 7f6d13fff700 1 -- 172.21.15.46:0/2145285666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1e90 msgr2=0x55d4f4bea1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:51:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.389+0000 7f6d13fff700 1 --2- 172.21.15.46:0/2145285666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1e90 0x55d4f4bea1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.389+0000 7f6d13fff700 1 -- 172.21.15.46:0/2145285666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 msgr2=0x55d4f4bea700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.389+0000 7f6d13fff700 1 --2- 172.21.15.46:0/2145285666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4bea700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.389+0000 7f6d13fff700 1 -- 172.21.15.46:0/2145285666 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d14114aa0 con 0x55d4f4be1570 2024-03-18T20:51:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.389+0000 7f6d188a1700 1 --2- 172.21.15.46:0/2145285666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4bea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:51:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.390+0000 7f6d13fff700 1 --2- 172.21.15.46:0/2145285666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1570 0x55d4f4be9c80 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400ed10 tx=0x7f6d04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba83b53bcfd2e3da server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.390+0000 7f6cf37fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:51:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.390+0000 7f6cf37fe700 1 -- 172.21.15.46:0/2145285666 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0400fb60 con 0x55d4f4be1570 2024-03-18T20:51:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.390+0000 7f6cf37fe700 1 -- 172.21.15.46:0/2145285666 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f6d0400fcc0 con 0x55d4f4be1570 2024-03-18T20:51:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.390+0000 7f6d26499b80 1 -- 172.21.15.46:0/2145285666 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4f4bd56c0 con 0x55d4f4be1570 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.390+0000 7f6cf37fe700 1 -- 172.21.15.46:0/2145285666 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d0400fe80 con 0x55d4f4be1570 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.391+0000 7f6d26499b80 10 client.20286 fetch_fsmap learned FSMap version 67 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.391+0000 7f6d26499b80 1 -- 172.21.15.46:0/2145285666 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55d4f4962b00 con 0x55d4f4be1570 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.391+0000 7f6cf37fe700 1 -- 172.21.15.46:0/2145285666 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d4f4bd56c0 con 0x55d4f4be1570 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.391+0000 7f6cf37fe700 1 -- 172.21.15.46:0/2145285666 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 67) v1 ==== 7522+0+0 (secure 0 0 0) 0x7f6d04024020 con 0x55d4f4be1570 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.391+0000 7f6d26499b80 10 client.20286 fetch_fsmap finished waiting for FSMap version 67 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.391+0000 7f6d26499b80 10 client.20286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.391+0000 7f6d26499b80 20 client.20286 populate_metadata read hostname 'smithi046' 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.391+0000 7f6d26499b80 1 --2- 172.21.15.46:0/2145285666 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4f4bfcc10 0x55d4f4bff030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.391+0000 7f6d26499b80 4 client.20286 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.391+0000 7f6d26499b80 1 -- 172.21.15.46:0/2145285666 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4f4b2d6a0 con 0x55d4f4bfcc10 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.391+0000 7f6d137fe700 1 --2- 172.21.15.46:0/2145285666 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4f4bfcc10 0x55d4f4bff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.392+0000 7f6cf37fe700 10 client.20286 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.392+0000 7f6d137fe700 1 --2- 172.21.15.46:0/2145285666 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4f4bfcc10 0x55d4f4bff030 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83244844f97a8502 server_cookie=7b0aa58b83cc0347 in_seq=0 out_seq=0 2024-03-18T20:51:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6cf37fe700 1 -- 172.21.15.46:0/2145285666 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d4f4b2d6a0 con 0x55d4f4bfcc10 2024-03-18T20:51:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6cf37fe700 10 client.20286 handle_command_reply: tid=0 2024-03-18T20:51:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6d26499b80 1 client.20286 shutdown 2024-03-18T20:51:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6d26499b80 1 -- 172.21.15.46:0/2145285666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1570 msgr2=0x55d4f4be9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6d26499b80 1 --2- 172.21.15.46:0/2145285666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1570 0x55d4f4be9c80 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400ed10 tx=0x7f6d04004750 comp rx=0 tx=0).stop 2024-03-18T20:51:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6d26499b80 1 -- 172.21.15.46:0/2145285666 shutdown_connections 2024-03-18T20:51:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6d26499b80 1 --2- 172.21.15.46:0/2145285666 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4f4bfcc10 0x55d4f4bff030 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6d26499b80 1 --2- 172.21.15.46:0/2145285666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1570 0x55d4f4be9c80 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6d26499b80 1 --2- 172.21.15.46:0/2145285666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4bea700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6cf37fe700 0 client.20286 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6d26499b80 1 --2- 172.21.15.46:0/2145285666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1e90 0x55d4f4bea1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6d26499b80 1 -- 172.21.15.46:0/2145285666 >> 172.21.15.46:0/2145285666 conn(0x55d4f4b581c0 msgr2=0x55d4f4bdd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6d26499b80 1 -- 172.21.15.46:0/2145285666 shutdown_connections 2024-03-18T20:51:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.405+0000 7f6d26499b80 1 -- 172.21.15.46:0/2145285666 wait complete. 2024-03-18T20:51:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.406+0000 7f6d26499b80 20 client.20286 trim_cache size 0 max 16384 2024-03-18T20:51:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d26499b80 1 Processor -- start 2024-03-18T20:51:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d26499b80 1 -- start start 2024-03-18T20:51:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d26499b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1570 0x55d4f4bf3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d26499b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4be4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d26499b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4be5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d26499b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f4be2610 con 0x55d4f4be1e90 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d26499b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f4bd7c10 con 0x55d4f4becdb0 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d26499b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f4bdef70 con 0x55d4f4be1570 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4be4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4be4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45646/0 (socket says 172.21.15.46:45646) 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d137fe700 1 -- 172.21.15.46:0/2844738324 learned_addr learned my addr 172.21.15.46:0/2844738324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d188a1700 1 --2- 172.21.15.46:0/2844738324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4be5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.435+0000 7f6d13fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1570 0x55d4f4bf3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.436+0000 7f6d137fe700 1 -- 172.21.15.46:0/2844738324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1570 msgr2=0x55d4f4bf3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.436+0000 7f6d137fe700 1 --2- 172.21.15.46:0/2844738324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1570 0x55d4f4bf3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.436+0000 7f6d137fe700 1 -- 172.21.15.46:0/2844738324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 msgr2=0x55d4f4be5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.436+0000 7f6d137fe700 1 --2- 172.21.15.46:0/2844738324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4be5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.436+0000 7f6d137fe700 1 -- 172.21.15.46:0/2844738324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4f4afae40 con 0x55d4f4be1e90 2024-03-18T20:51:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.436+0000 7f6d137fe700 1 --2- 172.21.15.46:0/2844738324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4be4f50 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f6d08005950 tx=0x7f6d08095d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f90a5e5e4b56afc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.436+0000 7f6cfaffd700 1 -- 172.21.15.46:0/2844738324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0809d770 con 0x55d4f4be1e90 2024-03-18T20:51:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.436+0000 7f6cfaffd700 1 -- 172.21.15.46:0/2844738324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d08084040 con 0x55d4f4be1e90 2024-03-18T20:51:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.436+0000 7f6cfaffd700 1 -- 172.21.15.46:0/2844738324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08099e00 con 0x55d4f4be1e90 2024-03-18T20:51:11.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.437+0000 7f6d26499b80 1 -- 172.21.15.46:0/2844738324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 msgr2=0x55d4f4be4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.437+0000 7f6d26499b80 1 --2- 172.21.15.46:0/2844738324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4be4f50 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f6d08005950 tx=0x7f6d08095d10 comp rx=0 tx=0).stop 2024-03-18T20:51:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.437+0000 7f6d26499b80 1 -- 172.21.15.46:0/2844738324 shutdown_connections 2024-03-18T20:51:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.437+0000 7f6d26499b80 1 --2- 172.21.15.46:0/2844738324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4becdb0 0x55d4f4be5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.437+0000 7f6d26499b80 1 --2- 172.21.15.46:0/2844738324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4be1570 0x55d4f4bf3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.437+0000 7f6d26499b80 1 --2- 172.21.15.46:0/2844738324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4be4f50 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.437+0000 7f6d26499b80 1 -- 172.21.15.46:0/2844738324 >> 172.21.15.46:0/2844738324 conn(0x55d4f4b581c0 msgr2=0x55d4f4bdc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.437+0000 7f6d26499b80 1 -- 172.21.15.46:0/2844738324 shutdown_connections 2024-03-18T20:51:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.437+0000 7f6d26499b80 1 -- 172.21.15.46:0/2844738324 wait complete. 2024-03-18T20:51:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.438+0000 7f6d26499b80 1 -- start start 2024-03-18T20:51:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.438+0000 7f6d26499b80 1 Processor -- start 2024-03-18T20:51:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.438+0000 7f6d26499b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4be1570 0x55d4f4bf1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.438+0000 7f6d26499b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4bf1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d26499b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4becdb0 0x55d4f4c4fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d26499b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f4bf20e0 con 0x55d4f4be1e90 2024-03-18T20:51:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d26499b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f4bf2250 con 0x55d4f4be1570 2024-03-18T20:51:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d26499b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4f4c4ff80 con 0x55d4f4becdb0 2024-03-18T20:51:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d26499b80 10 client.0 _ll_register_callbacks cb 0x7f6d0400e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:51:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d13fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4be1570 0x55d4f4bf1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d13fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4be1570 0x55d4f4bf1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:44936/0 (socket says 172.21.15.46:44936) 2024-03-18T20:51:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d13fff700 1 -- 172.21.15.46:0/3339165877 learned_addr learned my addr 172.21.15.46:0/3339165877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d137fe700 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4bf1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d188a1700 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4becdb0 0x55d4f4c4fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d188a1700 1 -- 172.21.15.46:0/3339165877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4be1570 msgr2=0x55d4f4bf1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d188a1700 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4be1570 0x55d4f4bf1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d188a1700 1 -- 172.21.15.46:0/3339165877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 msgr2=0x55d4f4bf1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d188a1700 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4bf1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d188a1700 1 -- 172.21.15.46:0/3339165877 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d141a1e20 con 0x55d4f4becdb0 2024-03-18T20:51:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d13fff700 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4be1570 0x55d4f4bf1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:51:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.439+0000 7f6d137fe700 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4bf1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:51:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.440+0000 7f6d188a1700 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4becdb0 0x55d4f4c4fa40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c01c0d0 tx=0x7f6d0c018740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ad561d072bd5b8a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.440+0000 7f6cf37fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:51:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.440+0000 7f6cf37fe700 1 -- 172.21.15.46:0/3339165877 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c01f040 con 0x55d4f4becdb0 2024-03-18T20:51:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.440+0000 7f6cf37fe700 1 -- 172.21.15.46:0/3339165877 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f6d0c0050c0 con 0x55d4f4becdb0 2024-03-18T20:51:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.440+0000 7f6d26499b80 1 -- 172.21.15.46:0/3339165877 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d0c018be0 con 0x55d4f4becdb0 2024-03-18T20:51:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.440+0000 7f6cf37fe700 1 -- 172.21.15.46:0/3339165877 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d0c005280 con 0x55d4f4becdb0 2024-03-18T20:51:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.440+0000 7f6d26499b80 10 client.29950 fetch_fsmap learned FSMap version 67 2024-03-18T20:51:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.440+0000 7f6d26499b80 1 -- 172.21.15.46:0/3339165877 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55d4f49bac70 con 0x55d4f4becdb0 2024-03-18T20:51:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.440+0000 7f6cf37fe700 1 -- 172.21.15.46:0/3339165877 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d0c018be0 con 0x55d4f4becdb0 2024-03-18T20:51:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.441+0000 7f6cf37fe700 1 -- 172.21.15.46:0/3339165877 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 67) v1 ==== 7522+0+0 (secure 0 0 0) 0x7f6d0c02a040 con 0x55d4f4becdb0 2024-03-18T20:51:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.441+0000 7f6d26499b80 10 client.29950 fetch_fsmap finished waiting for FSMap version 67 2024-03-18T20:51:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.441+0000 7f6d26499b80 10 client.29950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:51:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.441+0000 7f6d26499b80 20 client.29950 populate_metadata read hostname 'smithi046' 2024-03-18T20:51:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.441+0000 7f6d26499b80 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4f4c515e0 0x55d4f4c53a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.441+0000 7f6d26499b80 4 client.29950 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:51:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.441+0000 7f6d26499b80 1 -- 172.21.15.46:0/3339165877 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d4f4983c90 con 0x55d4f4c515e0 2024-03-18T20:51:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.441+0000 7f6d13fff700 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4f4c515e0 0x55d4f4c53a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.441+0000 7f6d13fff700 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4f4c515e0 0x55d4f4c53a00 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4932086dc2c7c9d4 server_cookie=68bc48e59fbc3684 in_seq=0 out_seq=0 2024-03-18T20:51:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.441+0000 7f6cf37fe700 10 client.29950 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.604+0000 7f6cf37fe700 1 -- 172.21.15.46:0/3339165877 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d4f4983c90 con 0x55d4f4c515e0 2024-03-18T20:51:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.604+0000 7f6cf37fe700 10 client.29950 handle_command_reply: tid=0 2024-03-18T20:51:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.604+0000 7f6d26499b80 1 client.29950 shutdown 2024-03-18T20:51:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.605+0000 7f6d26499b80 1 -- 172.21.15.46:0/3339165877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4becdb0 msgr2=0x55d4f4c4fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.605+0000 7f6d26499b80 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4becdb0 0x55d4f4c4fa40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c01c0d0 tx=0x7f6d0c018740 comp rx=0 tx=0).stop 2024-03-18T20:51:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.605+0000 7f6d26499b80 1 -- 172.21.15.46:0/3339165877 shutdown_connections 2024-03-18T20:51:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.605+0000 7f6d26499b80 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d4f4c515e0 0x55d4f4c53a00 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.605+0000 7f6d26499b80 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d4f4becdb0 0x55d4f4c4fa40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.605+0000 7f6d26499b80 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d4f4be1570 0x55d4f4bf1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.605+0000 7f6d26499b80 1 --2- 172.21.15.46:0/3339165877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4f4be1e90 0x55d4f4bf1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.605+0000 7f6d26499b80 1 -- 172.21.15.46:0/3339165877 >> 172.21.15.46:0/3339165877 conn(0x55d4f4b581c0 msgr2=0x55d4f4be4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.605+0000 7f6cf37fe700 0 client.29950 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.610+0000 7f6d26499b80 1 -- 172.21.15.46:0/3339165877 shutdown_connections 2024-03-18T20:51:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:51:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:51:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:51:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:51:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.610+0000 7f6d26499b80 1 -- 172.21.15.46:0/3339165877 wait complete. 2024-03-18T20:51:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.611+0000 7f6d26499b80 20 client.29950 trim_cache size 0 max 16384 2024-03-18T20:51:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.611+0000 7f6d1a304700 1 -- 172.21.15.46:0/1329410390 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6cfc0a1310 msgr2=0x7f6cfc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.611+0000 7f6d1a304700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6cfc0a1310 0x7f6cfc0a37d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6d04011550 tx=0x7f6d04009260 comp rx=0 tx=0).stop 2024-03-18T20:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.611+0000 7f6d1a304700 1 -- 172.21.15.46:0/1329410390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d14107cb0 msgr2=0x7f6d141140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.611+0000 7f6d1a304700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d14107cb0 0x7f6d141140d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800d950 tx=0x7f6d0800dc60 comp rx=0 tx=0).stop 2024-03-18T20:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.612+0000 7f6d1a304700 1 -- 172.21.15.46:0/1329410390 shutdown_connections 2024-03-18T20:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.612+0000 7f6d1a304700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6cfc0a1310 0x7f6cfc0a37d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.612+0000 7f6d1a304700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d141072f0 0x7f6d14113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.612+0000 7f6d1a304700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d1410d370 0x7f6d1410fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.612+0000 7f6d1a304700 1 --2- 172.21.15.46:0/1329410390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d14107cb0 0x7f6d141140d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.612+0000 7f6d1a304700 1 -- 172.21.15.46:0/1329410390 >> 172.21.15.46:0/1329410390 conn(0x7f6d14076bd0 msgr2=0x7f6d14077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.612+0000 7f6d1a304700 1 -- 172.21.15.46:0/1329410390 shutdown_connections 2024-03-18T20:51:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:11.613+0000 7f6d1a304700 1 -- 172.21.15.46:0/1329410390 wait complete. 2024-03-18T20:51:12.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 97949 50.47 MB/sec warmup 108 sec latency 100.083 ms 2024-03-18T20:51:12.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:12 smithi073 ceph-mon[47384]: pgmap v473: 97 pgs: 97 active+clean; 872 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 727 op/s 2024-03-18T20:51:12.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:12 smithi155 ceph-mon[48344]: pgmap v473: 97 pgs: 97 active+clean; 872 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 727 op/s 2024-03-18T20:51:12.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:12 smithi046 ceph-mon[43495]: pgmap v473: 97 pgs: 97 active+clean; 872 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 727 op/s 2024-03-18T20:51:13.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 99152 50.61 MB/sec warmup 109 sec latency 47.348 ms 2024-03-18T20:51:14.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 100222 50.74 MB/sec warmup 110 sec latency 36.979 ms 2024-03-18T20:51:14.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:14 smithi073 ceph-mon[47384]: pgmap v474: 97 pgs: 97 active+clean; 899 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 17 MiB/s wr, 918 op/s 2024-03-18T20:51:14.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:14 smithi155 ceph-mon[48344]: pgmap v474: 97 pgs: 97 active+clean; 899 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 17 MiB/s wr, 918 op/s 2024-03-18T20:51:14.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:14 smithi046 ceph-mon[43495]: pgmap v474: 97 pgs: 97 active+clean; 899 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 17 MiB/s wr, 918 op/s 2024-03-18T20:51:15.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 101084 50.69 MB/sec warmup 111 sec latency 61.653 ms 2024-03-18T20:51:15.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:15 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:15.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:15 smithi073 ceph-mon[47384]: osdmap e100: 12 total, 12 up, 12 in 2024-03-18T20:51:15.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:15 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:15.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:15 smithi155 ceph-mon[48344]: osdmap e100: 12 total, 12 up, 12 in 2024-03-18T20:51:15.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:15 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:15.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:15 smithi046 ceph-mon[43495]: osdmap e100: 12 total, 12 up, 12 in 2024-03-18T20:51:16.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 102159 50.69 MB/sec warmup 112 sec latency 41.932 ms 2024-03-18T20:51:16.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:16 smithi073 ceph-mon[47384]: pgmap v476: 97 pgs: 97 active+clean; 899 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 22 MiB/s wr, 1.15k op/s 2024-03-18T20:51:16.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:16 smithi155 ceph-mon[48344]: pgmap v476: 97 pgs: 97 active+clean; 899 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 22 MiB/s wr, 1.15k op/s 2024-03-18T20:51:16.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:16 smithi046 ceph-mon[43495]: pgmap v476: 97 pgs: 97 active+clean; 899 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 22 MiB/s wr, 1.15k op/s 2024-03-18T20:51:17.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 103153 50.76 MB/sec warmup 113 sec latency 49.952 ms 2024-03-18T20:51:18.363 INFO:tasks.workunit.client.0.smithi046.stdout: 10 104125 50.79 MB/sec warmup 114 sec latency 97.553 ms 2024-03-18T20:51:18.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:18 smithi073 ceph-mon[47384]: pgmap v477: 97 pgs: 97 active+clean; 899 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.6 MiB/s wr, 502 op/s 2024-03-18T20:51:18.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:18 smithi155 ceph-mon[48344]: pgmap v477: 97 pgs: 97 active+clean; 899 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.6 MiB/s wr, 502 op/s 2024-03-18T20:51:18.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:18 smithi046 ceph-mon[43495]: pgmap v477: 97 pgs: 97 active+clean; 899 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.6 MiB/s wr, 502 op/s 2024-03-18T20:51:19.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 105015 50.76 MB/sec warmup 115 sec latency 62.443 ms 2024-03-18T20:51:20.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 105939 50.75 MB/sec warmup 116 sec latency 59.613 ms 2024-03-18T20:51:20.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:20 smithi073 ceph-mon[47384]: pgmap v478: 97 pgs: 97 active+clean; 938 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.4 MiB/s wr, 844 op/s 2024-03-18T20:51:20.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:20 smithi155 ceph-mon[48344]: pgmap v478: 97 pgs: 97 active+clean; 938 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.4 MiB/s wr, 844 op/s 2024-03-18T20:51:20.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:20 smithi046 ceph-mon[43495]: pgmap v478: 97 pgs: 97 active+clean; 938 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.4 MiB/s wr, 844 op/s 2024-03-18T20:51:21.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 107092 50.75 MB/sec warmup 117 sec latency 79.934 ms 2024-03-18T20:51:21.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:21 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:21.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:21 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:21.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:21 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:22.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 108288 51.02 MB/sec warmup 118 sec latency 36.163 ms 2024-03-18T20:51:22.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:22 smithi073 ceph-mon[47384]: pgmap v479: 97 pgs: 97 active+clean; 938 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.4 MiB/s wr, 844 op/s 2024-03-18T20:51:22.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:22 smithi155 ceph-mon[48344]: pgmap v479: 97 pgs: 97 active+clean; 938 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.4 MiB/s wr, 844 op/s 2024-03-18T20:51:22.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:22 smithi046 ceph-mon[43495]: pgmap v479: 97 pgs: 97 active+clean; 938 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.4 MiB/s wr, 844 op/s 2024-03-18T20:51:23.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 108876 50.81 MB/sec warmup 119 sec latency 69.260 ms 2024-03-18T20:51:23.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:23 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:51:23.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:23 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:51:23.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:23 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:51:24.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:24 smithi073 ceph-mon[47384]: pgmap v480: 97 pgs: 97 active+clean; 976 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.6 MiB/s wr, 771 op/s 2024-03-18T20:51:24.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:24 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:51:24.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:24 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:51:24.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:24 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:51:24.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:24 smithi155 ceph-mon[48344]: pgmap v480: 97 pgs: 97 active+clean; 976 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.6 MiB/s wr, 771 op/s 2024-03-18T20:51:24.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:24 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:51:24.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:24 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:51:24.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:24 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:51:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:24 smithi046 ceph-mon[43495]: pgmap v480: 97 pgs: 97 active+clean; 976 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 8.6 MiB/s wr, 771 op/s 2024-03-18T20:51:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:24 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:51:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:24 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:51:24.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:24 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:51:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.798+0000 7f7585902700 1 -- 172.21.15.46:0/778133626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f758011a590 msgr2=0x7f758011c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.798+0000 7f7585902700 1 --2- 172.21.15.46:0/778133626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f758011a590 0x7f758011c980 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f757801c0d0 tx=0x7f757801c3e0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.799+0000 7f7585902700 1 -- 172.21.15.46:0/778133626 shutdown_connections 2024-03-18T20:51:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.799+0000 7f7585902700 1 --2- 172.21.15.46:0/778133626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758010f4d0 0x7f7580107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.799+0000 7f7585902700 1 --2- 172.21.15.46:0/778133626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f758011a590 0x7f758011c980 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.799+0000 7f7585902700 1 --2- 172.21.15.46:0/778133626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f758010f940 0x7f75801082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.799+0000 7f7585902700 1 -- 172.21.15.46:0/778133626 >> 172.21.15.46:0/778133626 conn(0x7f758006bd80 msgr2=0x7f758006c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.799+0000 7f7585902700 1 -- 172.21.15.46:0/778133626 shutdown_connections 2024-03-18T20:51:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.799+0000 7f7585902700 1 -- 172.21.15.46:0/778133626 wait complete. 2024-03-18T20:51:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.800+0000 7f7585902700 1 Processor -- start 2024-03-18T20:51:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.800+0000 7f7585902700 1 -- start start 2024-03-18T20:51:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.801+0000 7f7585902700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758010f4d0 0x7f7580119a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.801+0000 7f7585902700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f758010f940 0x7f7580119fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.801+0000 7f7585902700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f758011a590 0x7f7580115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.801+0000 7f7585902700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7580116410 con 0x7f758010f4d0 2024-03-18T20:51:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.801+0000 7f7585902700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7580116580 con 0x7f758011a590 2024-03-18T20:51:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.801+0000 7f7585902700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7580116740 con 0x7f758010f940 2024-03-18T20:51:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.801+0000 7f757e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f758010f940 0x7f7580119fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.801+0000 7f757e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f758010f940 0x7f7580119fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40458/0 (socket says 172.21.15.46:40458) 2024-03-18T20:51:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.801+0000 7f757e7fc700 1 -- 172.21.15.46:0/1301471672 learned_addr learned my addr 172.21.15.46:0/1301471672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.801+0000 7f757e7fc700 1 -- 172.21.15.46:0/1301471672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f758011a590 msgr2=0x7f7580115cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:51:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.801+0000 7f757e7fc700 1 --2- 172.21.15.46:0/1301471672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f758011a590 0x7f7580115cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.802+0000 7f757e7fc700 1 -- 172.21.15.46:0/1301471672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758010f4d0 msgr2=0x7f7580119a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.802+0000 7f757e7fc700 1 --2- 172.21.15.46:0/1301471672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758010f4d0 0x7f7580119a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.802+0000 7f757e7fc700 1 -- 172.21.15.46:0/1301471672 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757801ad10 con 0x7f758010f940 2024-03-18T20:51:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.802+0000 7f757e7fc700 1 --2- 172.21.15.46:0/1301471672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f758010f940 0x7f7580119fc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f757000d980 tx=0x7f757000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.802+0000 7f7584900700 1 -- 172.21.15.46:0/1301471672 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570009b40 con 0x7f758010f940 2024-03-18T20:51:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.802+0000 7f7584900700 1 -- 172.21.15.46:0/1301471672 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f757000be20 con 0x7f758010f940 2024-03-18T20:51:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.802+0000 7f7584900700 1 -- 172.21.15.46:0/1301471672 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757000f620 con 0x7f758010f940 2024-03-18T20:51:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.802+0000 7f7585902700 1 -- 172.21.15.46:0/1301471672 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75801aa1c0 con 0x7f758010f940 2024-03-18T20:51:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.802+0000 7f7585902700 1 -- 172.21.15.46:0/1301471672 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75801aa590 con 0x7f758010f940 2024-03-18T20:51:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.805+0000 7f7584900700 1 -- 172.21.15.46:0/1301471672 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7570010460 con 0x7f758010f940 2024-03-18T20:51:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.807+0000 7f7584900700 1 --2- 172.21.15.46:0/1301471672 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f75680a1350 0x7f75680a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.807+0000 7f7584900700 1 -- 172.21.15.46:0/1301471672 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f75700cc900 con 0x7f758010f940 2024-03-18T20:51:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.807+0000 7f757effd700 1 --2- 172.21.15.46:0/1301471672 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f75680a1350 0x7f75680a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.810+0000 7f757effd700 1 --2- 172.21.15.46:0/1301471672 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f75680a1350 0x7f75680a3810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7574005950 tx=0x7f75740058e0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.811+0000 7f7591a97b80 1 Processor -- start 2024-03-18T20:51:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.811+0000 7f7591a97b80 1 -- start start 2024-03-18T20:51:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.811+0000 7f7591a97b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52570 0x55de4ee52950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.811+0000 7f7591a97b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee52e90 0x55de4ee5d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.811+0000 7f7591a97b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee5ddb0 0x55de4ee601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.811+0000 7f7591a97b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4edac930 con 0x55de4ee52e90 2024-03-18T20:51:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.811+0000 7f7591a97b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4edaa2f0 con 0x55de4ee52570 2024-03-18T20:51:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.811+0000 7f7591a97b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4eda5be0 con 0x55de4ee5ddb0 2024-03-18T20:51:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.812+0000 7f757f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee5ddb0 0x55de4ee601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.812+0000 7f757f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee5ddb0 0x55de4ee601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40466/0 (socket says 172.21.15.46:40466) 2024-03-18T20:51:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.812+0000 7f757f7fe700 1 -- 172.21.15.46:0/2222290097 learned_addr learned my addr 172.21.15.46:0/2222290097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.812+0000 7f757f7fe700 1 -- 172.21.15.46:0/2222290097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52570 msgr2=0x55de4ee52950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:51:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.812+0000 7f757f7fe700 1 --2- 172.21.15.46:0/2222290097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52570 0x55de4ee52950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.812+0000 7f757f7fe700 1 -- 172.21.15.46:0/2222290097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee52e90 msgr2=0x55de4ee5d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.812+0000 7f757f7fe700 1 --2- 172.21.15.46:0/2222290097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee52e90 0x55de4ee5d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.812+0000 7f757f7fe700 1 -- 172.21.15.46:0/2222290097 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de4ec27350 con 0x55de4ee5ddb0 2024-03-18T20:51:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.813+0000 7f757f7fe700 1 --2- 172.21.15.46:0/2222290097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee5ddb0 0x55de4ee601a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f757801ac90 tx=0x7f7578019bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdd1ac4b29ca46d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.813+0000 7f7565ffb700 1 -- 172.21.15.46:0/2222290097 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757801f040 con 0x55de4ee5ddb0 2024-03-18T20:51:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.813+0000 7f7565ffb700 1 -- 172.21.15.46:0/2222290097 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7578019e20 con 0x55de4ee5ddb0 2024-03-18T20:51:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.813+0000 7f7565ffb700 1 -- 172.21.15.46:0/2222290097 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75780188b0 con 0x55de4ee5ddb0 2024-03-18T20:51:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.814+0000 7f7591a97b80 1 -- 172.21.15.46:0/2222290097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee5ddb0 msgr2=0x55de4ee601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.814+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2222290097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee5ddb0 0x55de4ee601a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f757801ac90 tx=0x7f7578019bd0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.814+0000 7f7591a97b80 1 -- 172.21.15.46:0/2222290097 shutdown_connections 2024-03-18T20:51:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.814+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2222290097 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee5ddb0 0x55de4ee601a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.814+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2222290097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52570 0x55de4ee52950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.814+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2222290097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee52e90 0x55de4ee5d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.814+0000 7f7591a97b80 1 -- 172.21.15.46:0/2222290097 >> 172.21.15.46:0/2222290097 conn(0x55de4edc91c0 msgr2=0x55de4ee4ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.814+0000 7f7591a97b80 1 -- 172.21.15.46:0/2222290097 shutdown_connections 2024-03-18T20:51:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.814+0000 7f7591a97b80 1 -- 172.21.15.46:0/2222290097 wait complete. 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.814+0000 7f7591a97b80 1 -- start start 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.815+0000 7f7591a97b80 1 Processor -- start 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.815+0000 7f7591a97b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee5d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.815+0000 7f7591a97b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee52e90 0x55de4ee5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.815+0000 7f7591a97b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee5ddb0 0x55de4ee5b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.815+0000 7f7591a97b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4ed41d20 con 0x55de4ee52e90 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.815+0000 7f7591a97b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4ed419c0 con 0x55de4ee5ddb0 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.815+0000 7f7591a97b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4ec29640 con 0x55de4ee52570 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.815+0000 7f7591a97b80 10 client.0 _ll_register_callbacks cb 0x55de4ebad340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.816+0000 7f757f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee5ddb0 0x55de4ee5b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.816+0000 7f757f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee5ddb0 0x55de4ee5b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51628/0 (socket says 172.21.15.46:51628) 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.816+0000 7f757f7fe700 1 -- 172.21.15.46:0/2669979308 learned_addr learned my addr 172.21.15.46:0/2669979308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.816+0000 7f757f7fe700 1 -- 172.21.15.46:0/2669979308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 msgr2=0x55de4ee5d810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.816+0000 7f757f7fe700 1 --2- 172.21.15.46:0/2669979308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee5d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.816+0000 7f757f7fe700 1 -- 172.21.15.46:0/2669979308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee52e90 msgr2=0x55de4ee5ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.816+0000 7f757f7fe700 1 --2- 172.21.15.46:0/2669979308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee52e90 0x55de4ee5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.816+0000 7f757f7fe700 1 -- 172.21.15.46:0/2669979308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758011d210 con 0x55de4ee5ddb0 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.816+0000 7f757f7fe700 1 --2- 172.21.15.46:0/2669979308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee5ddb0 0x55de4ee5b240 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f757801c420 tx=0x7f7578003df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76050ea033f41d33 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.816+0000 7f755effd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.817+0000 7f755effd700 1 -- 172.21.15.46:0/2669979308 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75780076f0 con 0x55de4ee5ddb0 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.817+0000 7f755effd700 1 -- 172.21.15.46:0/2669979308 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7578007850 con 0x55de4ee5ddb0 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.817+0000 7f7591a97b80 1 -- 172.21.15.46:0/2669979308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de4ee466c0 con 0x55de4ee5ddb0 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.817+0000 7f755effd700 1 -- 172.21.15.46:0/2669979308 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7578007a10 con 0x55de4ee5ddb0 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.817+0000 7f7591a97b80 10 client.29879 fetch_fsmap learned FSMap version 69 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.817+0000 7f7591a97b80 1 -- 172.21.15.46:0/2669979308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x55de4ebd3b00 con 0x55de4ee5ddb0 2024-03-18T20:51:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.817+0000 7f755effd700 1 -- 172.21.15.46:0/2669979308 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de4ee466c0 con 0x55de4ee5ddb0 2024-03-18T20:51:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.817+0000 7f755effd700 1 -- 172.21.15.46:0/2669979308 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 69) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f757802a040 con 0x55de4ee5ddb0 2024-03-18T20:51:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.817+0000 7f7591a97b80 10 client.29879 fetch_fsmap finished waiting for FSMap version 69 2024-03-18T20:51:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.817+0000 7f7591a97b80 10 client.29879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:51:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.817+0000 7f7591a97b80 20 client.29879 populate_metadata read hostname 'smithi046' 2024-03-18T20:51:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.818+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2669979308 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4ee6d6b0 0x55de4ee6fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.818+0000 7f7591a97b80 4 client.29879 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:51:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.818+0000 7f7591a97b80 1 -- 172.21.15.46:0/2669979308 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de4ed9e6a0 con 0x55de4ee6d6b0 2024-03-18T20:51:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.818+0000 7f757effd700 1 --2- 172.21.15.46:0/2669979308 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4ee6d6b0 0x55de4ee6fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.818+0000 7f755effd700 10 client.29879 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:24.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.818+0000 7f757effd700 1 --2- 172.21.15.46:0/2669979308 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4ee6d6b0 0x55de4ee6fad0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92309dbd22c7b121 server_cookie=35912df3c67bdec7 in_seq=0 out_seq=0 2024-03-18T20:51:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.821+0000 7f755effd700 1 -- 172.21.15.46:0/2669979308 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55de4ed9e6a0 con 0x55de4ee6d6b0 2024-03-18T20:51:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.821+0000 7f755effd700 10 client.29879 handle_command_reply: tid=0 2024-03-18T20:51:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.821+0000 7f7591a97b80 1 client.29879 shutdown 2024-03-18T20:51:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.821+0000 7f7591a97b80 1 -- 172.21.15.46:0/2669979308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee5ddb0 msgr2=0x55de4ee5b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.821+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2669979308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee5ddb0 0x55de4ee5b240 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f757801c420 tx=0x7f7578003df0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.824+0000 7f7591a97b80 1 -- 172.21.15.46:0/2669979308 shutdown_connections 2024-03-18T20:51:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.824+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2669979308 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4ee6d6b0 0x55de4ee6fad0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.824+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2669979308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee5ddb0 0x55de4ee5b240 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.824+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2669979308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee5d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.824+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2669979308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee52e90 0x55de4ee5ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.824+0000 7f7591a97b80 1 -- 172.21.15.46:0/2669979308 >> 172.21.15.46:0/2669979308 conn(0x55de4edc91c0 msgr2=0x55de4ee4e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.825+0000 7f755effd700 0 client.29879 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.825+0000 7f7591a97b80 1 -- 172.21.15.46:0/2669979308 shutdown_connections 2024-03-18T20:51:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.825+0000 7f7591a97b80 1 -- 172.21.15.46:0/2669979308 wait complete. 2024-03-18T20:51:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.828+0000 7f7591a97b80 20 client.29879 trim_cache size 0 max 16384 2024-03-18T20:51:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.848+0000 7f7591a97b80 1 Processor -- start 2024-03-18T20:51:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.848+0000 7f7591a97b80 1 -- start start 2024-03-18T20:51:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.848+0000 7f7591a97b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee64a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.848+0000 7f7591a97b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 0x55de4ee5d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.848+0000 7f7591a97b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee5ddb0 0x55de4ee5a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.848+0000 7f7591a97b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4ee48c10 con 0x55de4ee5ddb0 2024-03-18T20:51:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.848+0000 7f7591a97b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4ee5b820 con 0x55de4ee52e90 2024-03-18T20:51:24.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.848+0000 7f7591a97b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4ee4f760 con 0x55de4ee52570 2024-03-18T20:51:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee64a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 0x55de4ee5d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:24.852 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 0x55de4ee5d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51632/0 (socket says 172.21.15.46:51632) 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee64a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40480/0 (socket says 172.21.15.46:40480) 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757e7fc700 1 -- 172.21.15.46:0/2117793841 learned_addr learned my addr 172.21.15.46:0/2117793841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757f7fe700 1 --2- 172.21.15.46:0/2117793841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee5ddb0 0x55de4ee5a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757effd700 1 -- 172.21.15.46:0/2117793841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 msgr2=0x55de4ee5d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757effd700 1 --2- 172.21.15.46:0/2117793841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 0x55de4ee5d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757effd700 1 -- 172.21.15.46:0/2117793841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee5ddb0 msgr2=0x55de4ee5a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757effd700 1 --2- 172.21.15.46:0/2117793841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee5ddb0 0x55de4ee5a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757effd700 1 -- 172.21.15.46:0/2117793841 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de4ed6be40 con 0x55de4ee52570 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757e7fc700 1 --2- 172.21.15.46:0/2117793841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 0x55de4ee5d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f757effd700 1 --2- 172.21.15.46:0/2117793841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee64a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55de4ee70340 tx=0x7f7574010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcd6a2076af77b8f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f75667fc700 1 -- 172.21.15.46:0/2117793841 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574005b20 con 0x55de4ee52570 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.849+0000 7f75667fc700 1 -- 172.21.15.46:0/2117793841 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7574022e60 con 0x55de4ee52570 2024-03-18T20:51:24.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.850+0000 7f75667fc700 1 -- 172.21.15.46:0/2117793841 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757401e450 con 0x55de4ee52570 2024-03-18T20:51:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.850+0000 7f7591a97b80 1 -- 172.21.15.46:0/2117793841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 msgr2=0x55de4ee64a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.850+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2117793841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee64a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55de4ee70340 tx=0x7f7574010000 comp rx=0 tx=0).stop 2024-03-18T20:51:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.852+0000 7f7591a97b80 1 -- 172.21.15.46:0/2117793841 shutdown_connections 2024-03-18T20:51:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.852+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2117793841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee5ddb0 0x55de4ee5a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.852+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2117793841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee64a10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.852+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2117793841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 0x55de4ee5d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.852+0000 7f7591a97b80 1 -- 172.21.15.46:0/2117793841 >> 172.21.15.46:0/2117793841 conn(0x55de4edc91c0 msgr2=0x55de4ee55dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.852+0000 7f7591a97b80 1 -- 172.21.15.46:0/2117793841 shutdown_connections 2024-03-18T20:51:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.852+0000 7f7591a97b80 1 -- 172.21.15.46:0/2117793841 wait complete. 2024-03-18T20:51:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.852+0000 7f7591a97b80 1 -- start start 2024-03-18T20:51:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.853+0000 7f7591a97b80 1 Processor -- start 2024-03-18T20:51:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.855+0000 7f7591a97b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee62560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.855+0000 7f7591a97b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 0x55de4ee62aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.855+0000 7f7591a97b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee5ddb0 0x55de4eec04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.855+0000 7f7591a97b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4ee63250 con 0x55de4ee5ddb0 2024-03-18T20:51:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.855+0000 7f7591a97b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4ee633c0 con 0x55de4ee52e90 2024-03-18T20:51:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.855+0000 7f7591a97b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4eec0a10 con 0x55de4ee52570 2024-03-18T20:51:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.855+0000 7f7591a97b80 10 client.0 _ll_register_callbacks cb 0x7f757801af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:51:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.855+0000 7f757e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 0x55de4ee62aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.855+0000 7f757e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 0x55de4ee62aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51640/0 (socket says 172.21.15.46:51640) 2024-03-18T20:51:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.855+0000 7f757e7fc700 1 -- 172.21.15.46:0/2705407269 learned_addr learned my addr 172.21.15.46:0/2705407269 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:24.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.855+0000 7f757effd700 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee62560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.856+0000 7f757e7fc700 1 -- 172.21.15.46:0/2705407269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 msgr2=0x55de4ee62560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.856+0000 7f757e7fc700 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee62560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.856+0000 7f757e7fc700 1 -- 172.21.15.46:0/2705407269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee5ddb0 msgr2=0x55de4eec04a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:51:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.856+0000 7f757e7fc700 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee5ddb0 0x55de4eec04a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.856+0000 7f757e7fc700 1 -- 172.21.15.46:0/2705407269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75801aa590 con 0x55de4ee52e90 2024-03-18T20:51:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.856+0000 7f757effd700 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee62560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:51:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.856+0000 7f757e7fc700 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 0x55de4ee62aa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f75700ca630 tx=0x7f7570099530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b59800374629248 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.856+0000 7f755effd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:51:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.856+0000 7f755effd700 1 -- 172.21.15.46:0/2705407269 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757009a0f0 con 0x55de4ee52e90 2024-03-18T20:51:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.856+0000 7f755effd700 1 -- 172.21.15.46:0/2705407269 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f75700999f0 con 0x55de4ee52e90 2024-03-18T20:51:24.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.856+0000 7f7591a97b80 1 -- 172.21.15.46:0/2705407269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7578019e20 con 0x55de4ee52e90 2024-03-18T20:51:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.859+0000 7f755effd700 1 -- 172.21.15.46:0/2705407269 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f757009d820 con 0x55de4ee52e90 2024-03-18T20:51:24.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.859+0000 7f7591a97b80 10 client.29885 fetch_fsmap learned FSMap version 69 2024-03-18T20:51:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.859+0000 7f7591a97b80 1 -- 172.21.15.46:0/2705407269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x7f7574022b40 con 0x55de4ee52e90 2024-03-18T20:51:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.861+0000 7f755effd700 1 -- 172.21.15.46:0/2705407269 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7578019e20 con 0x55de4ee52e90 2024-03-18T20:51:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.862+0000 7f755effd700 1 -- 172.21.15.46:0/2705407269 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 69) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f7570098ee0 con 0x55de4ee52e90 2024-03-18T20:51:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.862+0000 7f7591a97b80 10 client.29885 fetch_fsmap finished waiting for FSMap version 69 2024-03-18T20:51:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.862+0000 7f7591a97b80 10 client.29885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:51:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.862+0000 7f7591a97b80 20 client.29885 populate_metadata read hostname 'smithi046' 2024-03-18T20:51:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.862+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4eec2310 0x55de4eec4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.862+0000 7f7591a97b80 4 client.29885 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:51:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.862+0000 7f7591a97b80 1 -- 172.21.15.46:0/2705407269 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55de4ebf4c90 con 0x55de4eec2310 2024-03-18T20:51:24.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.863+0000 7f757effd700 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4eec2310 0x55de4eec4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.863+0000 7f757effd700 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4eec2310 0x55de4eec4730 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a7442a103e9cd server_cookie=558d51080b51e2aa in_seq=0 out_seq=0 2024-03-18T20:51:24.866 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:24.864+0000 7f755effd700 10 client.29885 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.086+0000 7f755effd700 1 -- 172.21.15.46:0/2705407269 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55de4ebf4c90 con 0x55de4eec2310 2024-03-18T20:51:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.086+0000 7f755effd700 10 client.29885 handle_command_reply: tid=0 2024-03-18T20:51:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.086+0000 7f7591a97b80 1 client.29885 shutdown 2024-03-18T20:51:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.086+0000 7f7591a97b80 1 -- 172.21.15.46:0/2705407269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 msgr2=0x55de4ee62aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.086+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 0x55de4ee62aa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f75700ca630 tx=0x7f7570099530 comp rx=0 tx=0).stop 2024-03-18T20:51:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.086+0000 7f7591a97b80 1 -- 172.21.15.46:0/2705407269 shutdown_connections 2024-03-18T20:51:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.086+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55de4eec2310 0x55de4eec4730 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.086+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de4ee5ddb0 0x55de4eec04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.086+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55de4ee52570 0x55de4ee62560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.086+0000 7f7591a97b80 1 --2- 172.21.15.46:0/2705407269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de4ee52e90 0x55de4ee62aa0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.086+0000 7f7591a97b80 1 -- 172.21.15.46:0/2705407269 >> 172.21.15.46:0/2705407269 conn(0x55de4edc91c0 msgr2=0x55de4ee5a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.086+0000 7f755effd700 0 client.29885 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.087+0000 7f7591a97b80 1 -- 172.21.15.46:0/2705407269 shutdown_connections 2024-03-18T20:51:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.087+0000 7f7591a97b80 1 -- 172.21.15.46:0/2705407269 wait complete. 2024-03-18T20:51:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.087+0000 7f7591a97b80 20 client.29885 trim_cache size 0 max 16384 2024-03-18T20:51:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:51:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:51:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:51:25.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:51:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.088+0000 7f7585902700 1 -- 172.21.15.46:0/1301471672 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f75680a1350 msgr2=0x7f75680a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.088+0000 7f7585902700 1 --2- 172.21.15.46:0/1301471672 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f75680a1350 0x7f75680a3810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7574005950 tx=0x7f75740058e0 comp rx=0 tx=0).stop 2024-03-18T20:51:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.088+0000 7f7585902700 1 -- 172.21.15.46:0/1301471672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f758010f940 msgr2=0x7f7580119fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.088+0000 7f7585902700 1 --2- 172.21.15.46:0/1301471672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f758010f940 0x7f7580119fc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f757000d980 tx=0x7f757000dd40 comp rx=0 tx=0).stop 2024-03-18T20:51:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.088+0000 7f7585902700 1 -- 172.21.15.46:0/1301471672 shutdown_connections 2024-03-18T20:51:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.088+0000 7f7585902700 1 --2- 172.21.15.46:0/1301471672 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f75680a1350 0x7f75680a3810 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.088+0000 7f7585902700 1 --2- 172.21.15.46:0/1301471672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758010f4d0 0x7f7580119a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.088+0000 7f7585902700 1 --2- 172.21.15.46:0/1301471672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f758011a590 0x7f7580115cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.088+0000 7f7585902700 1 --2- 172.21.15.46:0/1301471672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f758010f940 0x7f7580119fc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.088+0000 7f7585902700 1 -- 172.21.15.46:0/1301471672 >> 172.21.15.46:0/1301471672 conn(0x7f758006bd80 msgr2=0x7f758011a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.088+0000 7f7585902700 1 -- 172.21.15.46:0/1301471672 shutdown_connections 2024-03-18T20:51:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:25.088+0000 7f7585902700 1 -- 172.21.15.46:0/1301471672 wait complete. 2024-03-18T20:51:25.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 110692 59.09 MB/sec execute 1 sec latency 116.315 ms 2024-03-18T20:51:26.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 111601 53.46 MB/sec execute 2 sec latency 50.154 ms 2024-03-18T20:51:26.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:26 smithi073 ceph-mon[47384]: pgmap v481: 97 pgs: 97 active+clean; 976 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.4 MiB/s wr, 756 op/s 2024-03-18T20:51:26.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:26 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:26.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:26 smithi155 ceph-mon[48344]: pgmap v481: 97 pgs: 97 active+clean; 976 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.4 MiB/s wr, 756 op/s 2024-03-18T20:51:26.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:26 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:26.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:26 smithi046 ceph-mon[43495]: pgmap v481: 97 pgs: 97 active+clean; 976 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.4 MiB/s wr, 756 op/s 2024-03-18T20:51:26.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:26 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:27.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 112611 53.50 MB/sec execute 3 sec latency 26.715 ms 2024-03-18T20:51:28.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 113598 54.69 MB/sec execute 4 sec latency 80.753 ms 2024-03-18T20:51:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:28 smithi073 ceph-mon[47384]: pgmap v482: 97 pgs: 97 active+clean; 976 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.1 MiB/s wr, 643 op/s 2024-03-18T20:51:28.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:28 smithi155 ceph-mon[48344]: pgmap v482: 97 pgs: 97 active+clean; 976 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.1 MiB/s wr, 643 op/s 2024-03-18T20:51:28.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:28 smithi046 ceph-mon[43495]: pgmap v482: 97 pgs: 97 active+clean; 976 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.1 MiB/s wr, 643 op/s 2024-03-18T20:51:29.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 114316 49.75 MB/sec execute 5 sec latency 296.154 ms 2024-03-18T20:51:30.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 115359 52.02 MB/sec execute 6 sec latency 69.162 ms 2024-03-18T20:51:30.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:30 smithi073 ceph-mon[47384]: pgmap v483: 97 pgs: 97 active+clean; 1000 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2024-03-18T20:51:30.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:30 smithi155 ceph-mon[48344]: pgmap v483: 97 pgs: 97 active+clean; 1000 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2024-03-18T20:51:30.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:30 smithi046 ceph-mon[43495]: pgmap v483: 97 pgs: 97 active+clean; 1000 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2024-03-18T20:51:31.364 INFO:tasks.workunit.client.0.smithi046.stdout: 10 116441 52.51 MB/sec execute 7 sec latency 48.759 ms 2024-03-18T20:51:32.365 INFO:tasks.workunit.client.0.smithi046.stdout: 10 117404 52.38 MB/sec execute 8 sec latency 41.170 ms 2024-03-18T20:51:32.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:32 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:32.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:32 smithi073 ceph-mon[47384]: pgmap v484: 97 pgs: 97 active+clean; 1000 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.9 MiB/s wr, 684 op/s 2024-03-18T20:51:32.545 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:32 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:32.545 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:32 smithi155 ceph-mon[48344]: pgmap v484: 97 pgs: 97 active+clean; 1000 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.9 MiB/s wr, 684 op/s 2024-03-18T20:51:32.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:32 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:32.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:32 smithi046 ceph-mon[43495]: pgmap v484: 97 pgs: 97 active+clean; 1000 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.9 MiB/s wr, 684 op/s 2024-03-18T20:51:33.365 INFO:tasks.workunit.client.0.smithi046.stdout: 10 118392 52.75 MB/sec execute 9 sec latency 61.246 ms 2024-03-18T20:51:34.365 INFO:tasks.workunit.client.0.smithi046.stdout: 10 119322 52.78 MB/sec execute 10 sec latency 51.946 ms 2024-03-18T20:51:34.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:34 smithi073 ceph-mon[47384]: pgmap v485: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 997 op/s 2024-03-18T20:51:34.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:34 smithi155 ceph-mon[48344]: pgmap v485: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 997 op/s 2024-03-18T20:51:34.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:34 smithi046 ceph-mon[43495]: pgmap v485: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 997 op/s 2024-03-18T20:51:35.365 INFO:tasks.workunit.client.0.smithi046.stdout: 10 120330 52.85 MB/sec execute 11 sec latency 44.347 ms 2024-03-18T20:51:35.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:35 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:35.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:35 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:35.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:35 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:36.365 INFO:tasks.workunit.client.0.smithi046.stdout: 10 121266 51.84 MB/sec execute 12 sec latency 103.940 ms 2024-03-18T20:51:36.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:36 smithi073 ceph-mon[47384]: pgmap v486: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 683 op/s 2024-03-18T20:51:36.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:36 smithi155 ceph-mon[48344]: pgmap v486: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 683 op/s 2024-03-18T20:51:36.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:36 smithi046 ceph-mon[43495]: pgmap v486: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 683 op/s 2024-03-18T20:51:37.365 INFO:tasks.workunit.client.0.smithi046.stdout: 10 122231 52.42 MB/sec execute 13 sec latency 66.849 ms 2024-03-18T20:51:37.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:37 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:37.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:37 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:37.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:37 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.321+0000 7f075c163700 1 -- 172.21.15.46:0/3118541006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0754112d00 msgr2=0x7f07541150f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.321+0000 7f075c163700 1 --2- 172.21.15.46:0/3118541006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0754112d00 0x7f07541150f0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f0750009990 tx=0x7f0750009ca0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.321+0000 7f075c163700 1 -- 172.21.15.46:0/3118541006 shutdown_connections 2024-03-18T20:51:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.321+0000 7f075c163700 1 --2- 172.21.15.46:0/3118541006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0754112d00 0x7f07541150f0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.321+0000 7f075c163700 1 --2- 172.21.15.46:0/3118541006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0754109750 0x7f0754109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.321+0000 7f075c163700 1 --2- 172.21.15.46:0/3118541006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f075410a070 0x7f07541127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.321+0000 7f075c163700 1 -- 172.21.15.46:0/3118541006 >> 172.21.15.46:0/3118541006 conn(0x7f075406bd80 msgr2=0x7f075406c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.329+0000 7f075c163700 1 -- 172.21.15.46:0/3118541006 shutdown_connections 2024-03-18T20:51:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.334+0000 7f075c163700 1 -- 172.21.15.46:0/3118541006 wait complete. 2024-03-18T20:51:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.335+0000 7f075c163700 1 Processor -- start 2024-03-18T20:51:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.337+0000 7f075c163700 1 -- start start 2024-03-18T20:51:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.337+0000 7f075c163700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0754109750 0x7f07541b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.337+0000 7f075c163700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0754112d00 0x7f07541b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.337+0000 7f075c163700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07541ae9e0 0x7f07541aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.337+0000 7f075c163700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07541af460 con 0x7f0754112d00 2024-03-18T20:51:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.337+0000 7f075c163700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07541af5d0 con 0x7f0754109750 2024-03-18T20:51:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.337+0000 7f075c163700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07541af790 con 0x7f07541ae9e0 2024-03-18T20:51:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.339+0000 7f07596fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0754112d00 0x7f07541b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.340+0000 7f07596fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0754112d00 0x7f07541b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50284/0 (socket says 172.21.15.46:50284) 2024-03-18T20:51:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.340+0000 7f07596fe700 1 -- 172.21.15.46:0/1276961990 learned_addr learned my addr 172.21.15.46:0/1276961990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.341+0000 7f07596fe700 1 -- 172.21.15.46:0/1276961990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07541ae9e0 msgr2=0x7f07541aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.341+0000 7f07596fe700 1 --2- 172.21.15.46:0/1276961990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07541ae9e0 0x7f07541aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.341+0000 7f07596fe700 1 -- 172.21.15.46:0/1276961990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0754109750 msgr2=0x7f07541b48b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-18T20:51:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.341+0000 7f07596fe700 1 --2- 172.21.15.46:0/1276961990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0754109750 0x7f07541b48b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.341+0000 7f07596fe700 1 -- 172.21.15.46:0/1276961990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07500095f0 con 0x7f0754112d00 2024-03-18T20:51:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.342+0000 7f07596fe700 1 --2- 172.21.15.46:0/1276961990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0754112d00 0x7f07541b4df0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f074c00b9b0 tx=0x7f074c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.342+0000 7f074affd700 1 -- 172.21.15.46:0/1276961990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c00ca30 con 0x7f0754112d00 2024-03-18T20:51:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.342+0000 7f074affd700 1 -- 172.21.15.46:0/1276961990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f074c00cb90 con 0x7f0754112d00 2024-03-18T20:51:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.342+0000 7f074affd700 1 -- 172.21.15.46:0/1276961990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c0125a0 con 0x7f0754112d00 2024-03-18T20:51:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.346+0000 7f075c163700 1 -- 172.21.15.46:0/1276961990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07541bb780 con 0x7f0754112d00 2024-03-18T20:51:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.346+0000 7f075c163700 1 -- 172.21.15.46:0/1276961990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07541bbc10 con 0x7f0754112d00 2024-03-18T20:51:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.353+0000 7f07682f8b80 1 Processor -- start 2024-03-18T20:51:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.357+0000 7f07682f8b80 1 -- start start 2024-03-18T20:51:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.357+0000 7f07682f8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a48890570 0x564a48890950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.357+0000 7f07682f8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890e90 0x564a4889b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.357+0000 7f07682f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a4889bdb0 0x564a4889e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.357+0000 7f07682f8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a487ea930 con 0x564a48890e90 2024-03-18T20:51:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.357+0000 7f07682f8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a487e82f0 con 0x564a48890570 2024-03-18T20:51:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.357+0000 7f07682f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a487e3be0 con 0x564a4889bdb0 2024-03-18T20:51:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.357+0000 7f074affd700 1 -- 172.21.15.46:0/1276961990 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f074c002020 con 0x7f0754112d00 2024-03-18T20:51:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.358+0000 7f074affd700 1 --2- 172.21.15.46:0/1276961990 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f07400a14a0 0x7f07400a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.359+0000 7f074affd700 1 -- 172.21.15.46:0/1276961990 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f074c0cc140 con 0x7f0754112d00 2024-03-18T20:51:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.359+0000 7f07596fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a4889bdb0 0x564a4889e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.359+0000 7f07596fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a4889bdb0 0x564a4889e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:47808/0 (socket says 172.21.15.46:47808) 2024-03-18T20:51:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.359+0000 7f07596fe700 1 -- 172.21.15.46:0/2873993168 learned_addr learned my addr 172.21.15.46:0/2873993168 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.361+0000 7f075a700700 1 --2- 172.21.15.46:0/2873993168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a48890570 0x564a48890950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.361+0000 7f07596fe700 1 -- 172.21.15.46:0/2873993168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a48890570 msgr2=0x564a48890950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.361+0000 7f07596fe700 1 --2- 172.21.15.46:0/2873993168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a48890570 0x564a48890950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.361+0000 7f07596fe700 1 -- 172.21.15.46:0/2873993168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890e90 msgr2=0x564a4889b870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:51:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.361+0000 7f07596fe700 1 --2- 172.21.15.46:0/2873993168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890e90 0x564a4889b870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.361+0000 7f07596fe700 1 -- 172.21.15.46:0/2873993168 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a48665350 con 0x564a4889bdb0 2024-03-18T20:51:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.361+0000 7f07596fe700 1 --2- 172.21.15.46:0/2873993168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a4889bdb0 0x564a4889e1a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f074c00be90 tx=0x7f074c098cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2b43d83c3c8867 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:38.366 INFO:tasks.workunit.client.0.smithi046.stdout: 10 122348 49.09 MB/sec execute 14 sec latency 857.115 ms 2024-03-18T20:51:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.365+0000 7f073ffff700 1 -- 172.21.15.46:0/2873993168 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c09d110 con 0x564a4889bdb0 2024-03-18T20:51:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.365+0000 7f073ffff700 1 -- 172.21.15.46:0/2873993168 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f074c0b1e90 con 0x564a4889bdb0 2024-03-18T20:51:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.365+0000 7f073ffff700 1 -- 172.21.15.46:0/2873993168 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c0987d0 con 0x564a4889bdb0 2024-03-18T20:51:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.368+0000 7f07682f8b80 1 -- 172.21.15.46:0/2873993168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a4889bdb0 msgr2=0x564a4889e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.368+0000 7f07682f8b80 1 --2- 172.21.15.46:0/2873993168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a4889bdb0 0x564a4889e1a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f074c00be90 tx=0x7f074c098cc0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.368+0000 7f07682f8b80 1 -- 172.21.15.46:0/2873993168 shutdown_connections 2024-03-18T20:51:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.368+0000 7f07682f8b80 1 --2- 172.21.15.46:0/2873993168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a4889bdb0 0x564a4889e1a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.368+0000 7f07682f8b80 1 --2- 172.21.15.46:0/2873993168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a48890570 0x564a48890950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.368+0000 7f07682f8b80 1 --2- 172.21.15.46:0/2873993168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890e90 0x564a4889b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.368+0000 7f07682f8b80 1 -- 172.21.15.46:0/2873993168 >> 172.21.15.46:0/2873993168 conn(0x564a488071c0 msgr2=0x564a4888ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.368+0000 7f07682f8b80 1 -- 172.21.15.46:0/2873993168 shutdown_connections 2024-03-18T20:51:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.368+0000 7f07682f8b80 1 -- 172.21.15.46:0/2873993168 wait complete. 2024-03-18T20:51:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.369+0000 7f07682f8b80 1 -- start start 2024-03-18T20:51:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.369+0000 7f07682f8b80 1 Processor -- start 2024-03-18T20:51:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.370+0000 7f07682f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890570 0x564a4889b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.370+0000 7f07682f8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890e90 0x564a48898ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.370+0000 7f07682f8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a4889bdb0 0x564a48899210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.370+0000 7f07682f8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4877fd20 con 0x564a48890e90 2024-03-18T20:51:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.370+0000 7f07682f8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a4877f9c0 con 0x564a4889bdb0 2024-03-18T20:51:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.370+0000 7f07682f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a48667640 con 0x564a48890570 2024-03-18T20:51:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.370+0000 7f07682f8b80 10 client.0 _ll_register_callbacks cb 0x564a485eb340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:51:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.370+0000 7f0759eff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890570 0x564a4889b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.370+0000 7f0759eff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890570 0x564a4889b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:47822/0 (socket says 172.21.15.46:47822) 2024-03-18T20:51:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.370+0000 7f0759eff700 1 -- 172.21.15.46:0/770698442 learned_addr learned my addr 172.21.15.46:0/770698442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.372+0000 7f0759eff700 1 -- 172.21.15.46:0/770698442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a4889bdb0 msgr2=0x564a48899210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.372+0000 7f0759eff700 1 --2- 172.21.15.46:0/770698442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a4889bdb0 0x564a48899210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.372+0000 7f0759eff700 1 -- 172.21.15.46:0/770698442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890e90 msgr2=0x564a48898ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.372+0000 7f0759eff700 1 --2- 172.21.15.46:0/770698442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890e90 0x564a48898ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.372+0000 7f0759eff700 1 -- 172.21.15.46:0/770698442 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07541bbc10 con 0x564a48890570 2024-03-18T20:51:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.372+0000 7f075a700700 1 --2- 172.21.15.46:0/1276961990 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f07400a14a0 0x7f07400a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.372+0000 7f0759eff700 1 --2- 172.21.15.46:0/770698442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890570 0x564a4889b850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0744009ea0 tx=0x7f074400b3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1df42c689baa7f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.372+0000 7f073d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:51:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.373+0000 7f073d7fa700 1 -- 172.21.15.46:0/770698442 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744011420 con 0x564a48890570 2024-03-18T20:51:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.373+0000 7f073d7fa700 1 -- 172.21.15.46:0/770698442 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f0744011580 con 0x564a48890570 2024-03-18T20:51:38.375 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.373+0000 7f07682f8b80 1 -- 172.21.15.46:0/770698442 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a488846c0 con 0x564a48890570 2024-03-18T20:51:38.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.374+0000 7f075a700700 1 --2- 172.21.15.46:0/1276961990 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f07400a14a0 0x7f07400a3960 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0750005b40 tx=0x7f0750005ab0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.392+0000 7f073d7fa700 1 -- 172.21.15.46:0/770698442 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0744011740 con 0x564a48890570 2024-03-18T20:51:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.399+0000 7f07682f8b80 10 client.29971 fetch_fsmap learned FSMap version 73 2024-03-18T20:51:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.399+0000 7f07682f8b80 1 -- 172.21.15.46:0/770698442 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x564a48611b00 con 0x564a48890570 2024-03-18T20:51:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.404+0000 7f073d7fa700 1 -- 172.21.15.46:0/770698442 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a488846c0 con 0x564a48890570 2024-03-18T20:51:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.404+0000 7f073d7fa700 1 -- 172.21.15.46:0/770698442 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 73) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f074401aa10 con 0x564a48890570 2024-03-18T20:51:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.405+0000 7f07682f8b80 10 client.29971 fetch_fsmap finished waiting for FSMap version 73 2024-03-18T20:51:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.405+0000 7f07682f8b80 10 client.29971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:51:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.405+0000 7f07682f8b80 20 client.29971 populate_metadata read hostname 'smithi046' 2024-03-18T20:51:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.405+0000 7f07682f8b80 1 --2- 172.21.15.46:0/770698442 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a488abba0 0x564a488adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.405+0000 7f07682f8b80 4 client.29971 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:51:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.405+0000 7f07682f8b80 1 -- 172.21.15.46:0/770698442 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a487dc6a0 con 0x564a488abba0 2024-03-18T20:51:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.411+0000 7f07596fe700 1 --2- 172.21.15.46:0/770698442 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a488abba0 0x564a488adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.433+0000 7f07596fe700 1 --2- 172.21.15.46:0/770698442 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a488abba0 0x564a488adfc0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13084d7dc00e3599 server_cookie=8f2f9add4004394c in_seq=0 out_seq=0 2024-03-18T20:51:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.441+0000 7f073d7fa700 10 client.29971 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.448+0000 7f073d7fa700 1 -- 172.21.15.46:0/770698442 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564a487dc6a0 con 0x564a488abba0 2024-03-18T20:51:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.448+0000 7f073d7fa700 10 client.29971 handle_command_reply: tid=0 2024-03-18T20:51:38.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.448+0000 7f07682f8b80 1 client.29971 shutdown 2024-03-18T20:51:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.448+0000 7f07682f8b80 1 -- 172.21.15.46:0/770698442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890570 msgr2=0x564a4889b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.448+0000 7f07682f8b80 1 --2- 172.21.15.46:0/770698442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890570 0x564a4889b850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0744009ea0 tx=0x7f074400b3c0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.449+0000 7f07682f8b80 1 -- 172.21.15.46:0/770698442 shutdown_connections 2024-03-18T20:51:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.449+0000 7f07682f8b80 1 --2- 172.21.15.46:0/770698442 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a488abba0 0x564a488adfc0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.449+0000 7f07682f8b80 1 --2- 172.21.15.46:0/770698442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a4889bdb0 0x564a48899210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.449+0000 7f07682f8b80 1 --2- 172.21.15.46:0/770698442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890570 0x564a4889b850 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.449+0000 7f07682f8b80 1 --2- 172.21.15.46:0/770698442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890e90 0x564a48898ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.449+0000 7f07682f8b80 1 -- 172.21.15.46:0/770698442 >> 172.21.15.46:0/770698442 conn(0x564a488071c0 msgr2=0x564a4888c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.449+0000 7f073d7fa700 0 client.29971 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.449+0000 7f07682f8b80 1 -- 172.21.15.46:0/770698442 shutdown_connections 2024-03-18T20:51:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.449+0000 7f07682f8b80 1 -- 172.21.15.46:0/770698442 wait complete. 2024-03-18T20:51:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.453+0000 7f07682f8b80 20 client.29971 trim_cache size 0 max 16384 2024-03-18T20:51:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.491+0000 7f07682f8b80 1 Processor -- start 2024-03-18T20:51:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.491+0000 7f07682f8b80 1 -- start start 2024-03-18T20:51:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.491+0000 7f07682f8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890570 0x564a488adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.491+0000 7f07682f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890e90 0x564a488a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.491+0000 7f07682f8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a4889bdb0 0x564a4889b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.491+0000 7f07682f8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a48899ea0 con 0x564a48890570 2024-03-18T20:51:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.491+0000 7f07682f8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a488a9100 con 0x564a4889bdb0 2024-03-18T20:51:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.491+0000 7f07682f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a488a2f80 con 0x564a48890e90 2024-03-18T20:51:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.492+0000 7f0759eff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890570 0x564a488adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.492+0000 7f075a700700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a4889bdb0 0x564a4889b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.492+0000 7f0759eff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890570 0x564a488adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50300/0 (socket says 172.21.15.46:50300) 2024-03-18T20:51:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.492+0000 7f0759eff700 1 -- 172.21.15.46:0/630034927 learned_addr learned my addr 172.21.15.46:0/630034927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.492+0000 7f07596fe700 1 --2- 172.21.15.46:0/630034927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890e90 0x564a488a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.492+0000 7f07596fe700 1 -- 172.21.15.46:0/630034927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a4889bdb0 msgr2=0x564a4889b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.492+0000 7f07596fe700 1 --2- 172.21.15.46:0/630034927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a4889bdb0 0x564a4889b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.492+0000 7f07596fe700 1 -- 172.21.15.46:0/630034927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890570 msgr2=0x564a488adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.492+0000 7f07596fe700 1 --2- 172.21.15.46:0/630034927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890570 0x564a488adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.492+0000 7f07596fe700 1 -- 172.21.15.46:0/630034927 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a487a9e40 con 0x564a48890e90 2024-03-18T20:51:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.493+0000 7f07596fe700 1 --2- 172.21.15.46:0/630034927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890e90 0x564a488a2a40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x564a48898330 tx=0x7f074c00e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f09645a4713c056 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.493+0000 7f0748ff9700 1 -- 172.21.15.46:0/630034927 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c0b5070 con 0x564a48890e90 2024-03-18T20:51:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.493+0000 7f0748ff9700 1 -- 172.21.15.46:0/630034927 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f074c097680 con 0x564a48890e90 2024-03-18T20:51:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.493+0000 7f0748ff9700 1 -- 172.21.15.46:0/630034927 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c0a12d0 con 0x564a48890e90 2024-03-18T20:51:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.494+0000 7f07682f8b80 1 -- 172.21.15.46:0/630034927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890e90 msgr2=0x564a488a2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.494+0000 7f07682f8b80 1 --2- 172.21.15.46:0/630034927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890e90 0x564a488a2a40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x564a48898330 tx=0x7f074c00e040 comp rx=0 tx=0).stop 2024-03-18T20:51:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.494+0000 7f07682f8b80 1 -- 172.21.15.46:0/630034927 shutdown_connections 2024-03-18T20:51:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.494+0000 7f07682f8b80 1 --2- 172.21.15.46:0/630034927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a48890570 0x564a488adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.494+0000 7f07682f8b80 1 --2- 172.21.15.46:0/630034927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a4889bdb0 0x564a4889b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.494+0000 7f07682f8b80 1 --2- 172.21.15.46:0/630034927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a48890e90 0x564a488a2a40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.494+0000 7f07682f8b80 1 -- 172.21.15.46:0/630034927 >> 172.21.15.46:0/630034927 conn(0x564a488071c0 msgr2=0x564a48888620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.498+0000 7f07682f8b80 1 -- 172.21.15.46:0/630034927 shutdown_connections 2024-03-18T20:51:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.501+0000 7f07682f8b80 1 -- 172.21.15.46:0/630034927 wait complete. 2024-03-18T20:51:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.502+0000 7f07682f8b80 1 -- start start 2024-03-18T20:51:38.506 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.502+0000 7f07682f8b80 1 Processor -- start 2024-03-18T20:51:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.503+0000 7f07682f8b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a48890e90 0x564a488a0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.503+0000 7f07682f8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a4889bdb0 0x564a488a0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.503+0000 7f07682f8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a488a0dd0 0x564a488fe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.503+0000 7f07682f8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a488feff0 con 0x564a488a0dd0 2024-03-18T20:51:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.503+0000 7f07682f8b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a488ff160 con 0x564a48890e90 2024-03-18T20:51:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.503+0000 7f07682f8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a488ff320 con 0x564a4889bdb0 2024-03-18T20:51:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.503+0000 7f07682f8b80 10 client.0 _ll_register_callbacks cb 0x564a485eb340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:51:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.504+0000 7f0759eff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a48890e90 0x564a488a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.504+0000 7f0759eff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a48890e90 0x564a488a0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:60234/0 (socket says 172.21.15.46:60234) 2024-03-18T20:51:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.504+0000 7f0759eff700 1 -- 172.21.15.46:0/2645555185 learned_addr learned my addr 172.21.15.46:0/2645555185 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.504+0000 7f07596fe700 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a4889bdb0 0x564a488a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.507+0000 7f075a700700 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a488a0dd0 0x564a488fe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.511+0000 7f0759eff700 1 -- 172.21.15.46:0/2645555185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a4889bdb0 msgr2=0x564a488a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.511+0000 7f0759eff700 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a4889bdb0 0x564a488a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.511+0000 7f0759eff700 1 -- 172.21.15.46:0/2645555185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a488a0dd0 msgr2=0x564a488fe9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.511+0000 7f0759eff700 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a488a0dd0 0x564a488fe9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.511+0000 7f0759eff700 1 -- 172.21.15.46:0/2645555185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a48665350 con 0x564a48890e90 2024-03-18T20:51:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.529+0000 7f0759eff700 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a48890e90 0x564a488a0300 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f07440097d0 tx=0x7f074400bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=717a6416ee9a704b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.530+0000 7f073d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:51:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.539+0000 7f073d7fa700 1 -- 172.21.15.46:0/2645555185 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744004590 con 0x564a48890e90 2024-03-18T20:51:38.542 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.539+0000 7f073d7fa700 1 -- 172.21.15.46:0/2645555185 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f0744003740 con 0x564a48890e90 2024-03-18T20:51:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.541+0000 7f07682f8b80 1 -- 172.21.15.46:0/2645555185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f074c0b1e90 con 0x564a48890e90 2024-03-18T20:51:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.548+0000 7f073d7fa700 1 -- 172.21.15.46:0/2645555185 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07440046f0 con 0x564a48890e90 2024-03-18T20:51:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.559+0000 7f07682f8b80 10 client.29894 fetch_fsmap learned FSMap version 73 2024-03-18T20:51:38.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.559+0000 7f07682f8b80 1 -- 172.21.15.46:0/2645555185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x564a48669c70 con 0x564a48890e90 2024-03-18T20:51:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.575+0000 7f073d7fa700 1 -- 172.21.15.46:0/2645555185 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f074c0b1e90 con 0x564a48890e90 2024-03-18T20:51:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.575+0000 7f073d7fa700 1 -- 172.21.15.46:0/2645555185 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 73) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0744036420 con 0x564a48890e90 2024-03-18T20:51:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.575+0000 7f07682f8b80 10 client.29894 fetch_fsmap finished waiting for FSMap version 73 2024-03-18T20:51:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.575+0000 7f07682f8b80 10 client.29894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:51:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.575+0000 7f07682f8b80 20 client.29894 populate_metadata read hostname 'smithi046' 2024-03-18T20:51:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.575+0000 7f07682f8b80 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a48900b30 0x564a48902f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.575+0000 7f07682f8b80 4 client.29894 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:51:38.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.575+0000 7f07682f8b80 1 -- 172.21.15.46:0/2645555185 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564a48632c90 con 0x564a48900b30 2024-03-18T20:51:38.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.594+0000 7f07596fe700 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a48900b30 0x564a48902f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.602+0000 7f07596fe700 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a48900b30 0x564a48902f50 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c3ceca416b65cbc server_cookie=ee15c9d87899e86f in_seq=0 out_seq=0 2024-03-18T20:51:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:38.602+0000 7f073d7fa700 10 client.29894 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:38 smithi073 ceph-mon[47384]: pgmap v487: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 683 op/s 2024-03-18T20:51:38.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:38 smithi155 ceph-mon[48344]: pgmap v487: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 683 op/s 2024-03-18T20:51:38.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:38 smithi046 ceph-mon[43495]: pgmap v487: 97 pgs: 97 active+clean; 1.0 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.5 MiB/s wr, 683 op/s 2024-03-18T20:51:39.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 122397 45.92 MB/sec execute 15 sec latency 1298.823 ms 2024-03-18T20:51:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f073d7fa700 1 -- 172.21.15.46:0/2645555185 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564a48632c90 con 0x564a48900b30 2024-03-18T20:51:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f073d7fa700 10 client.29894 handle_command_reply: tid=0 2024-03-18T20:51:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f07682f8b80 1 client.29894 shutdown 2024-03-18T20:51:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f07682f8b80 1 -- 172.21.15.46:0/2645555185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a48890e90 msgr2=0x564a488a0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f07682f8b80 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a48890e90 0x564a488a0300 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f07440097d0 tx=0x7f074400bf30 comp rx=0 tx=0).stop 2024-03-18T20:51:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f07682f8b80 1 -- 172.21.15.46:0/2645555185 shutdown_connections 2024-03-18T20:51:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f07682f8b80 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x564a48900b30 0x564a48902f50 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f07682f8b80 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564a488a0dd0 0x564a488fe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f07682f8b80 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564a48890e90 0x564a488a0300 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f07682f8b80 1 --2- 172.21.15.46:0/2645555185 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564a4889bdb0 0x564a488a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f07682f8b80 1 -- 172.21.15.46:0/2645555185 >> 172.21.15.46:0/2645555185 conn(0x564a488071c0 msgr2=0x564a48893db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f073d7fa700 0 client.29894 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f07682f8b80 1 -- 172.21.15.46:0/2645555185 shutdown_connections 2024-03-18T20:51:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.521+0000 7f07682f8b80 1 -- 172.21.15.46:0/2645555185 wait complete. 2024-03-18T20:51:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.523+0000 7f07682f8b80 20 client.29894 trim_cache size 0 max 16384 2024-03-18T20:51:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:51:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:51:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:51:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:51:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.526+0000 7f075c163700 1 -- 172.21.15.46:0/1276961990 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f07400a14a0 msgr2=0x7f07400a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.526+0000 7f075c163700 1 --2- 172.21.15.46:0/1276961990 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f07400a14a0 0x7f07400a3960 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0750005b40 tx=0x7f0750005ab0 comp rx=0 tx=0).stop 2024-03-18T20:51:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.526+0000 7f075c163700 1 -- 172.21.15.46:0/1276961990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0754112d00 msgr2=0x7f07541b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.526+0000 7f075c163700 1 --2- 172.21.15.46:0/1276961990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0754112d00 0x7f07541b4df0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f074c00b9b0 tx=0x7f074c00bd70 comp rx=0 tx=0).stop 2024-03-18T20:51:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.527+0000 7f075c163700 1 -- 172.21.15.46:0/1276961990 shutdown_connections 2024-03-18T20:51:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.527+0000 7f075c163700 1 --2- 172.21.15.46:0/1276961990 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f07400a14a0 0x7f07400a3960 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.527+0000 7f075c163700 1 --2- 172.21.15.46:0/1276961990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07541ae9e0 0x7f07541aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.527+0000 7f075c163700 1 --2- 172.21.15.46:0/1276961990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0754109750 0x7f07541b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.527+0000 7f075c163700 1 --2- 172.21.15.46:0/1276961990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0754112d00 0x7f07541b4df0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.527+0000 7f075c163700 1 -- 172.21.15.46:0/1276961990 >> 172.21.15.46:0/1276961990 conn(0x7f075406bd80 msgr2=0x7f075406fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.527+0000 7f075c163700 1 -- 172.21.15.46:0/1276961990 shutdown_connections 2024-03-18T20:51:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:39.528+0000 7f075c163700 1 -- 172.21.15.46:0/1276961990 wait complete. 2024-03-18T20:51:40.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 122826 44.68 MB/sec execute 16 sec latency 163.276 ms 2024-03-18T20:51:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:40 smithi073 ceph-mon[47384]: pgmap v488: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.22k op/s 2024-03-18T20:51:40.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:40 smithi155 ceph-mon[48344]: pgmap v488: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.22k op/s 2024-03-18T20:51:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:40 smithi046 ceph-mon[43495]: pgmap v488: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.22k op/s 2024-03-18T20:51:41.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 123344 43.63 MB/sec execute 17 sec latency 79.962 ms 2024-03-18T20:51:42.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 124279 44.14 MB/sec execute 18 sec latency 72.610 ms 2024-03-18T20:51:42.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:42 smithi073 ceph-mon[47384]: pgmap v489: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 22 MiB/s wr, 847 op/s 2024-03-18T20:51:42.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:42 smithi155 ceph-mon[48344]: pgmap v489: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 22 MiB/s wr, 847 op/s 2024-03-18T20:51:42.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:42 smithi046 ceph-mon[43495]: pgmap v489: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 22 MiB/s wr, 847 op/s 2024-03-18T20:51:43.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 125264 44.37 MB/sec execute 19 sec latency 45.358 ms 2024-03-18T20:51:44.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 126271 44.83 MB/sec execute 20 sec latency 67.808 ms 2024-03-18T20:51:44.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:44 smithi073 ceph-mon[47384]: pgmap v490: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 25 MiB/s wr, 1.03k op/s 2024-03-18T20:51:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:44 smithi155 ceph-mon[48344]: pgmap v490: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 25 MiB/s wr, 1.03k op/s 2024-03-18T20:51:44.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:44 smithi046 ceph-mon[43495]: pgmap v490: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 25 MiB/s wr, 1.03k op/s 2024-03-18T20:51:45.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 127192 45.48 MB/sec execute 21 sec latency 43.519 ms 2024-03-18T20:51:46.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 128174 45.36 MB/sec execute 22 sec latency 39.166 ms 2024-03-18T20:51:46.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:46 smithi073 ceph-mon[47384]: pgmap v491: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 27 MiB/s wr, 1.25k op/s 2024-03-18T20:51:46.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:46 smithi155 ceph-mon[48344]: pgmap v491: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 27 MiB/s wr, 1.25k op/s 2024-03-18T20:51:46.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:46 smithi046 ceph-mon[43495]: pgmap v491: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 27 MiB/s wr, 1.25k op/s 2024-03-18T20:51:47.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 129247 45.90 MB/sec execute 23 sec latency 73.746 ms 2024-03-18T20:51:48.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 130099 46.40 MB/sec execute 24 sec latency 64.882 ms 2024-03-18T20:51:48.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:48 smithi073 ceph-mon[47384]: pgmap v492: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 27 MiB/s wr, 1.25k op/s 2024-03-18T20:51:48.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:48 smithi155 ceph-mon[48344]: pgmap v492: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 27 MiB/s wr, 1.25k op/s 2024-03-18T20:51:48.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:48 smithi046 ceph-mon[43495]: pgmap v492: 97 pgs: 97 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 27 MiB/s wr, 1.25k op/s 2024-03-18T20:51:49.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 130732 45.72 MB/sec execute 25 sec latency 144.521 ms 2024-03-18T20:51:50.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 131744 45.60 MB/sec execute 26 sec latency 47.646 ms 2024-03-18T20:51:50.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:50 smithi073 ceph-mon[47384]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 1.57k op/s 2024-03-18T20:51:50.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:50 smithi155 ceph-mon[48344]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 1.57k op/s 2024-03-18T20:51:50.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:50 smithi046 ceph-mon[43495]: pgmap v493: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 30 MiB/s wr, 1.57k op/s 2024-03-18T20:51:51.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 132960 46.69 MB/sec execute 27 sec latency 44.412 ms 2024-03-18T20:51:52.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 133804 46.91 MB/sec execute 28 sec latency 42.319 ms 2024-03-18T20:51:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.706+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/1702401636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e54100540 msgr2=0x7f4e54104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.706+0000 7f4e5b8a5700 1 --2- 172.21.15.46:0/1702401636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e54100540 0x7f4e54104a00 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009990 tx=0x7f4e50009ca0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.707+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/1702401636 shutdown_connections 2024-03-18T20:51:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.707+0000 7f4e5b8a5700 1 --2- 172.21.15.46:0/1702401636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e54105020 0x7f4e541054d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.707+0000 7f4e5b8a5700 1 --2- 172.21.15.46:0/1702401636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e540ffc20 0x7f4e54100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.707+0000 7f4e5b8a5700 1 --2- 172.21.15.46:0/1702401636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e54100540 0x7f4e54104a00 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.707+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/1702401636 >> 172.21.15.46:0/1702401636 conn(0x7f4e540fb630 msgr2=0x7f4e540fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.707+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/1702401636 shutdown_connections 2024-03-18T20:51:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.707+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/1702401636 wait complete. 2024-03-18T20:51:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.707+0000 7f4e5b8a5700 1 Processor -- start 2024-03-18T20:51:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.707+0000 7f4e5b8a5700 1 -- start start 2024-03-18T20:51:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e5b8a5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e540ffc20 0x7f4e5419d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e5b8a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e54100540 0x7f4e5419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e5b8a5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e54105020 0x7f4e541a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e5b8a5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e541a15a0 con 0x7f4e54100540 2024-03-18T20:51:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e5b8a5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e541a1710 con 0x7f4e540ffc20 2024-03-18T20:51:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e5b8a5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e541a18d0 con 0x7f4e54105020 2024-03-18T20:51:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e59641700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e540ffc20 0x7f4e5419d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e59641700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e540ffc20 0x7f4e5419d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:48480/0 (socket says 172.21.15.46:48480) 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e59641700 1 -- 172.21.15.46:0/3024334973 learned_addr learned my addr 172.21.15.46:0/3024334973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e59e42700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e54105020 0x7f4e541a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e58e40700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e54100540 0x7f4e5419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e59641700 1 -- 172.21.15.46:0/3024334973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e54105020 msgr2=0x7f4e541a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e59641700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e54105020 0x7f4e541a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e59641700 1 -- 172.21.15.46:0/3024334973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e54100540 msgr2=0x7f4e5419dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e59641700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e54100540 0x7f4e5419dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e59641700 1 -- 172.21.15.46:0/3024334973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e500095f0 con 0x7f4e540ffc20 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.708+0000 7f4e58e40700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e54100540 0x7f4e5419dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.709+0000 7f4e59641700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e540ffc20 0x7f4e5419d770 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e9a0 tx=0x7f4e4800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.709+0000 7f4e467fc700 1 -- 172.21.15.46:0/3024334973 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48009790 con 0x7f4e540ffc20 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.709+0000 7f4e467fc700 1 -- 172.21.15.46:0/3024334973 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e4800cbd0 con 0x7f4e540ffc20 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.709+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/3024334973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e541a1b50 con 0x7f4e540ffc20 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.709+0000 7f4e467fc700 1 -- 172.21.15.46:0/3024334973 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48018680 con 0x7f4e540ffc20 2024-03-18T20:51:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.709+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/3024334973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5406a9b0 con 0x7f4e540ffc20 2024-03-18T20:51:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.712+0000 7f4e467fc700 1 -- 172.21.15.46:0/3024334973 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f4e480187e0 con 0x7f4e540ffc20 2024-03-18T20:51:52.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:52 smithi073 ceph-mon[47384]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-03-18T20:51:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.714+0000 7f4e67a3ab80 1 Processor -- start 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.714+0000 7f4e467fc700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e400a1350 0x7f4e400a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e467fc700 1 -- 172.21.15.46:0/3024334973 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f4e48014070 con 0x7f4e540ffc20 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e67a3ab80 1 -- start start 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e67a3ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 0x55ccabe18950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e67a3ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe18e90 0x55ccabe23870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e67a3ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe23db0 0x55ccabe261a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e67a3ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccabd72930 con 0x55ccabe23db0 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e67a3ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccabd702f0 con 0x55ccabe18570 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e67a3ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccabd6bbe0 con 0x55ccabe18e90 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e59641700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 0x55ccabe18950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e59641700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 0x55ccabe18950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:48494/0 (socket says 172.21.15.46:48494) 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e59641700 1 -- 172.21.15.46:0/2005359371 learned_addr learned my addr 172.21.15.46:0/2005359371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e58e40700 1 --2- 172.21.15.46:0/2005359371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe18e90 0x55ccabe23870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.715+0000 7f4e59e42700 1 --2- 172.21.15.46:0/2005359371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe23db0 0x55ccabe261a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.716+0000 7f4e59641700 1 -- 172.21.15.46:0/2005359371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe18e90 msgr2=0x55ccabe23870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.716+0000 7f4e59641700 1 --2- 172.21.15.46:0/2005359371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe18e90 0x55ccabe23870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.716+0000 7f4e59641700 1 -- 172.21.15.46:0/2005359371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe23db0 msgr2=0x55ccabe261a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.716+0000 7f4e59641700 1 --2- 172.21.15.46:0/2005359371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe23db0 0x55ccabe261a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.716+0000 7f4e59641700 1 -- 172.21.15.46:0/2005359371 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccabbed350 con 0x55ccabe18570 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.716+0000 7f4e59641700 1 --2- 172.21.15.46:0/2005359371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 0x55ccabe18950 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4e48096580 tx=0x7f4e4800ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=151395271cfcab4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.716+0000 7f4e3f7fe700 1 -- 172.21.15.46:0/2005359371 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4809deb0 con 0x55ccabe18570 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.716+0000 7f4e3f7fe700 1 -- 172.21.15.46:0/2005359371 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e480a2500 con 0x55ccabe18570 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.716+0000 7f4e3f7fe700 1 -- 172.21.15.46:0/2005359371 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480a2070 con 0x55ccabe18570 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.717+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/2005359371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 msgr2=0x55ccabe18950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.717+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/2005359371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 0x55ccabe18950 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4e48096580 tx=0x7f4e4800ce60 comp rx=0 tx=0).stop 2024-03-18T20:51:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.717+0000 7f4e59e42700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e400a1350 0x7f4e400a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.717+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/2005359371 shutdown_connections 2024-03-18T20:51:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.717+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/2005359371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe23db0 0x55ccabe261a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.717+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/2005359371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 0x55ccabe18950 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.717+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/2005359371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe18e90 0x55ccabe23870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.717+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/2005359371 >> 172.21.15.46:0/2005359371 conn(0x55ccabd8f1c0 msgr2=0x55ccabe14a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.718+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/2005359371 shutdown_connections 2024-03-18T20:51:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.718+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/2005359371 wait complete. 2024-03-18T20:51:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.718+0000 7f4e59e42700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e400a1350 0x7f4e400a3810 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009fd0 tx=0x7f4e4c004b40 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.718+0000 7f4e67a3ab80 1 -- start start 2024-03-18T20:51:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.718+0000 7f4e67a3ab80 1 Processor -- start 2024-03-18T20:51:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.719+0000 7f4e67a3ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe18570 0x55ccabe20c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.719+0000 7f4e67a3ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18e90 0x55ccabe21180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.719+0000 7f4e67a3ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe23db0 0x55ccabe216c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.719+0000 7f4e67a3ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccabd07d20 con 0x55ccabe23db0 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.719+0000 7f4e67a3ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccabd079c0 con 0x55ccabe18e90 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.719+0000 7f4e67a3ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccabbef640 con 0x55ccabe18570 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.719+0000 7f4e67a3ab80 10 client.0 _ll_register_callbacks cb 0x55ccabb73340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.719+0000 7f4e59641700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe18570 0x55ccabe20c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.719+0000 7f4e59641700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe18570 0x55ccabe20c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:45208/0 (socket says 172.21.15.46:45208) 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.719+0000 7f4e59641700 1 -- 172.21.15.46:0/1853831505 learned_addr learned my addr 172.21.15.46:0/1853831505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.719+0000 7f4e59e42700 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe23db0 0x55ccabe216c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e59641700 1 -- 172.21.15.46:0/1853831505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18e90 msgr2=0x55ccabe21180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e59641700 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18e90 0x55ccabe21180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e59641700 1 -- 172.21.15.46:0/1853831505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe23db0 msgr2=0x55ccabe216c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e59641700 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe23db0 0x55ccabe216c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e59641700 1 -- 172.21.15.46:0/1853831505 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5406a9b0 con 0x55ccabe18570 2024-03-18T20:51:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e59e42700 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe23db0 0x55ccabe216c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:51:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e59641700 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe18570 0x55ccabe20c40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e480a1fd0 tx=0x7f4e480997a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bde8bb327d13c416 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:51:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e3cff9700 1 -- 172.21.15.46:0/1853831505 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4809deb0 con 0x55ccabe18570 2024-03-18T20:51:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e3cff9700 1 -- 172.21.15.46:0/1853831505 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f4e480a2280 con 0x55ccabe18570 2024-03-18T20:51:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1853831505 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ccabe0c6c0 con 0x55ccabe18570 2024-03-18T20:51:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e3cff9700 1 -- 172.21.15.46:0/1853831505 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e480b8120 con 0x55ccabe18570 2024-03-18T20:51:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e67a3ab80 10 client.29989 fetch_fsmap learned FSMap version 73 2024-03-18T20:51:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.720+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1853831505 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x55ccabb99b00 con 0x55ccabe18570 2024-03-18T20:51:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.721+0000 7f4e3cff9700 1 -- 172.21.15.46:0/1853831505 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e48082030 con 0x55ccabe18570 2024-03-18T20:51:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.721+0000 7f4e3cff9700 1 -- 172.21.15.46:0/1853831505 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 73) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f4e48095e70 con 0x55ccabe18570 2024-03-18T20:51:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.721+0000 7f4e67a3ab80 10 client.29989 fetch_fsmap finished waiting for FSMap version 73 2024-03-18T20:51:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.721+0000 7f4e67a3ab80 10 client.29989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:51:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.721+0000 7f4e67a3ab80 20 client.29989 populate_metadata read hostname 'smithi046' 2024-03-18T20:51:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.721+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ccabe446e0 0x55ccabe46b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.722+0000 7f4e67a3ab80 4 client.29989 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:51:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.722+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1853831505 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccabd646a0 con 0x55ccabe446e0 2024-03-18T20:51:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.722+0000 7f4e58e40700 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ccabe446e0 0x55ccabe46b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.722+0000 7f4e3cff9700 10 client.29989 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.722+0000 7f4e58e40700 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ccabe446e0 0x55ccabe46b00 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c22210ed8c8fc6 server_cookie=75b14139b99f49bc in_seq=0 out_seq=0 2024-03-18T20:51:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.726+0000 7f4e3cff9700 1 -- 172.21.15.46:0/1853831505 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ccabd646a0 con 0x55ccabe446e0 2024-03-18T20:51:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.726+0000 7f4e3cff9700 10 client.29989 handle_command_reply: tid=0 2024-03-18T20:51:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.727+0000 7f4e67a3ab80 1 client.29989 shutdown 2024-03-18T20:51:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.727+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1853831505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe18570 msgr2=0x55ccabe20c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.727+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe18570 0x55ccabe20c40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e480a1fd0 tx=0x7f4e480997a0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.728+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1853831505 shutdown_connections 2024-03-18T20:51:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.728+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ccabe446e0 0x55ccabe46b00 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.728+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe23db0 0x55ccabe216c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.728+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe18570 0x55ccabe20c40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.728+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/1853831505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18e90 0x55ccabe21180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.728+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1853831505 >> 172.21.15.46:0/1853831505 conn(0x55ccabd8f1c0 msgr2=0x55ccabe14650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.728+0000 7f4e3cff9700 0 client.29989 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.730+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1853831505 shutdown_connections 2024-03-18T20:51:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.730+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1853831505 wait complete. 2024-03-18T20:51:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.730+0000 7f4e67a3ab80 20 client.29989 trim_cache size 0 max 16384 2024-03-18T20:51:52.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.754+0000 7f4e67a3ab80 1 Processor -- start 2024-03-18T20:51:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.754+0000 7f4e67a3ab80 1 -- start start 2024-03-18T20:51:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.754+0000 7f4e67a3ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe18570 0x55ccabe14bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.754+0000 7f4e67a3ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18e90 0x55ccabe1b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.754+0000 7f4e67a3ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe23db0 0x55ccabe1f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.754+0000 7f4e67a3ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccabe1ba20 con 0x55ccabe18570 2024-03-18T20:51:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.754+0000 7f4e67a3ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccabe1f190 con 0x55ccabe18e90 2024-03-18T20:51:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.754+0000 7f4e67a3ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccabe1ca80 con 0x55ccabe23db0 2024-03-18T20:51:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.755+0000 7f4e58e40700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18e90 0x55ccabe1b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.755+0000 7f4e58e40700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18e90 0x55ccabe1b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:48506/0 (socket says 172.21.15.46:48506) 2024-03-18T20:51:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.755+0000 7f4e58e40700 1 -- 172.21.15.46:0/160578381 learned_addr learned my addr 172.21.15.46:0/160578381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.755+0000 7f4e59e42700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe23db0 0x55ccabe1f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.755+0000 7f4e59641700 1 --2- 172.21.15.46:0/160578381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe18570 0x55ccabe14bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.755+0000 7f4e58e40700 1 -- 172.21.15.46:0/160578381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe23db0 msgr2=0x55ccabe1f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.755+0000 7f4e58e40700 1 --2- 172.21.15.46:0/160578381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe23db0 0x55ccabe1f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.755+0000 7f4e58e40700 1 -- 172.21.15.46:0/160578381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe18570 msgr2=0x55ccabe14bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.755+0000 7f4e58e40700 1 --2- 172.21.15.46:0/160578381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe18570 0x55ccabe14bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.755+0000 7f4e58e40700 1 -- 172.21.15.46:0/160578381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccabd31e40 con 0x55ccabe18e90 2024-03-18T20:51:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.755+0000 7f4e58e40700 1 --2- 172.21.15.46:0/160578381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18e90 0x55ccabe1b4e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009990 tx=0x7f4e5003d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff928e55df36ff26 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.756+0000 7f4e3ffff700 1 -- 172.21.15.46:0/160578381 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50042040 con 0x55ccabe18e90 2024-03-18T20:51:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.756+0000 7f4e3ffff700 1 -- 172.21.15.46:0/160578381 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e50048970 con 0x55ccabe18e90 2024-03-18T20:51:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.756+0000 7f4e3ffff700 1 -- 172.21.15.46:0/160578381 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5000f6c0 con 0x55ccabe18e90 2024-03-18T20:51:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.757+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/160578381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18e90 msgr2=0x55ccabe1b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.757+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/160578381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18e90 0x55ccabe1b4e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009990 tx=0x7f4e5003d040 comp rx=0 tx=0).stop 2024-03-18T20:51:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.757+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/160578381 shutdown_connections 2024-03-18T20:51:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.757+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/160578381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe18570 0x55ccabe14bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.757+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/160578381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe23db0 0x55ccabe1f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.757+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/160578381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18e90 0x55ccabe1b4e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.757+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/160578381 >> 172.21.15.46:0/160578381 conn(0x55ccabd8f1c0 msgr2=0x55ccabe1bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.757+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/160578381 shutdown_connections 2024-03-18T20:51:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.758+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/160578381 wait complete. 2024-03-18T20:51:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.758+0000 7f4e67a3ab80 1 -- start start 2024-03-18T20:51:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.758+0000 7f4e67a3ab80 1 Processor -- start 2024-03-18T20:51:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.758+0000 7f4e67a3ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 0x55ccabe2a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.758+0000 7f4e67a3ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe18e90 0x55ccabe2a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.758+0000 7f4e67a3ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe23db0 0x55ccabe289d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.758+0000 7f4e67a3ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccabe28fa0 con 0x55ccabe18e90 2024-03-18T20:51:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.758+0000 7f4e67a3ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccabe290e0 con 0x55ccabe18570 2024-03-18T20:51:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.758+0000 7f4e67a3ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccabe292a0 con 0x55ccabe23db0 2024-03-18T20:51:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.759+0000 7f4e67a3ab80 10 client.0 _ll_register_callbacks cb 0x7f4e4c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:51:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.759+0000 7f4e59641700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 0x55ccabe2a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.759+0000 7f4e59641700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 0x55ccabe2a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:48518/0 (socket says 172.21.15.46:48518) 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.759+0000 7f4e59641700 1 -- 172.21.15.46:0/1577514200 learned_addr learned my addr 172.21.15.46:0/1577514200 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.759+0000 7f4e59e42700 1 --2- 172.21.15.46:0/1577514200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe23db0 0x55ccabe289d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.759+0000 7f4e59641700 1 -- 172.21.15.46:0/1577514200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe23db0 msgr2=0x55ccabe289d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.759+0000 7f4e59641700 1 --2- 172.21.15.46:0/1577514200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe23db0 0x55ccabe289d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.759+0000 7f4e59641700 1 -- 172.21.15.46:0/1577514200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe18e90 msgr2=0x55ccabe2a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.759+0000 7f4e59641700 1 --2- 172.21.15.46:0/1577514200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe18e90 0x55ccabe2a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.759+0000 7f4e59641700 1 -- 172.21.15.46:0/1577514200 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e541149d0 con 0x55ccabe18570 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.759+0000 7f4e59641700 1 --2- 172.21.15.46:0/1577514200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 0x55ccabe2a420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000c00 tx=0x7f4e4809a670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b304ac7a239f0e15 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.759+0000 7f4e3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e3cff9700 1 -- 172.21.15.46:0/1577514200 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4800ce60 con 0x55ccabe18570 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1577514200 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e480a2500 con 0x55ccabe18570 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e3cff9700 1 -- 172.21.15.46:0/1577514200 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f4e4809deb0 con 0x55ccabe18570 2024-03-18T20:51:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e3cff9700 1 -- 172.21.15.46:0/1577514200 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e48099130 con 0x55ccabe18570 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e67a3ab80 10 client.29909 fetch_fsmap learned FSMap version 73 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1577514200 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x55ccabbf1c70 con 0x55ccabe18570 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e3cff9700 1 -- 172.21.15.46:0/1577514200 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e480a2500 con 0x55ccabe18570 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e3cff9700 1 -- 172.21.15.46:0/1577514200 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 73) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f4e4809f040 con 0x55ccabe18570 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e67a3ab80 10 client.29909 fetch_fsmap finished waiting for FSMap version 73 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e67a3ab80 10 client.29909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e67a3ab80 20 client.29909 populate_metadata read hostname 'smithi046' 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/1577514200 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ccabe992b0 0x55ccabe9b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e67a3ab80 4 client.29909 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.760+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1577514200 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ccabbbac90 con 0x55ccabe992b0 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.762+0000 7f4e58e40700 1 --2- 172.21.15.46:0/1577514200 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ccabe992b0 0x55ccabe9b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.763+0000 7f4e58e40700 1 --2- 172.21.15.46:0/1577514200 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ccabe992b0 0x55ccabe9b6d0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef654cf409ace23a server_cookie=4642458d38c0906c in_seq=0 out_seq=0 2024-03-18T20:51:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:52.763+0000 7f4e3cff9700 10 client.29909 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:52.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:52 smithi155 ceph-mon[48344]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-03-18T20:51:52.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:52 smithi046 ceph-mon[43495]: pgmap v494: 97 pgs: 97 active+clean; 1.1 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-03-18T20:51:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e3cff9700 1 -- 172.21.15.46:0/1577514200 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ccabbbac90 con 0x55ccabe992b0 2024-03-18T20:51:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e3cff9700 10 client.29909 handle_command_reply: tid=0 2024-03-18T20:51:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e67a3ab80 1 client.29909 shutdown 2024-03-18T20:51:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1577514200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 msgr2=0x55ccabe2a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/1577514200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 0x55ccabe2a420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000c00 tx=0x7f4e4809a670 comp rx=0 tx=0).stop 2024-03-18T20:51:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1577514200 shutdown_connections 2024-03-18T20:51:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/1577514200 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ccabe992b0 0x55ccabe9b6d0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/1577514200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ccabe23db0 0x55ccabe289d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/1577514200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ccabe18570 0x55ccabe2a420 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e3cff9700 0 client.29909 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:51:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e67a3ab80 1 --2- 172.21.15.46:0/1577514200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccabe18e90 0x55ccabe2a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1577514200 >> 172.21.15.46:0/1577514200 conn(0x55ccabd8f1c0 msgr2=0x55ccabe1f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.013+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1577514200 shutdown_connections 2024-03-18T20:51:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.014+0000 7f4e67a3ab80 1 -- 172.21.15.46:0/1577514200 wait complete. 2024-03-18T20:51:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.014+0000 7f4e67a3ab80 20 client.29909 trim_cache size 0 max 16384 2024-03-18T20:51:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:51:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:51:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:51:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:51:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.015+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/3024334973 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e400a1350 msgr2=0x7f4e400a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.015+0000 7f4e5b8a5700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e400a1350 0x7f4e400a3810 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009fd0 tx=0x7f4e4c004b40 comp rx=0 tx=0).stop 2024-03-18T20:51:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.015+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/3024334973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e540ffc20 msgr2=0x7f4e5419d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:51:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.015+0000 7f4e5b8a5700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e540ffc20 0x7f4e5419d770 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e9a0 tx=0x7f4e4800ed60 comp rx=0 tx=0).stop 2024-03-18T20:51:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.015+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/3024334973 shutdown_connections 2024-03-18T20:51:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.015+0000 7f4e5b8a5700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f4e400a1350 0x7f4e400a3810 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.015+0000 7f4e5b8a5700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e54105020 0x7f4e541a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.015+0000 7f4e5b8a5700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e540ffc20 0x7f4e5419d770 secure :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e9a0 tx=0x7f4e4800ed60 comp rx=0 tx=0).stop 2024-03-18T20:51:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.015+0000 7f4e5b8a5700 1 --2- 172.21.15.46:0/3024334973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e54100540 0x7f4e5419dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:51:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.015+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/3024334973 >> 172.21.15.46:0/3024334973 conn(0x7f4e540fb630 msgr2=0x7f4e540fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:51:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.015+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/3024334973 shutdown_connections 2024-03-18T20:51:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:51:53.015+0000 7f4e5b8a5700 1 -- 172.21.15.46:0/3024334973 wait complete. 2024-03-18T20:51:53.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 134477 46.30 MB/sec execute 29 sec latency 224.972 ms 2024-03-18T20:51:54.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 135508 46.36 MB/sec execute 30 sec latency 38.010 ms 2024-03-18T20:51:54.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:54 smithi073 ceph-mon[47384]: pgmap v495: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-03-18T20:51:54.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:54 smithi155 ceph-mon[48344]: pgmap v495: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-03-18T20:51:54.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:54 smithi046 ceph-mon[43495]: pgmap v495: 97 pgs: 97 active+clean; 1.1 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-03-18T20:51:55.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 136627 47.07 MB/sec execute 31 sec latency 54.706 ms 2024-03-18T20:51:56.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 137426 46.95 MB/sec execute 32 sec latency 47.614 ms 2024-03-18T20:51:56.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:56 smithi073 ceph-mon[47384]: pgmap v496: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 1.22k op/s 2024-03-18T20:51:56.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:56 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:56.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:56 smithi155 ceph-mon[48344]: pgmap v496: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 1.22k op/s 2024-03-18T20:51:56.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:56 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:56.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:56 smithi046 ceph-mon[43495]: pgmap v496: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 1.22k op/s 2024-03-18T20:51:56.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:56 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:57.367 INFO:tasks.workunit.client.0.smithi046.stdout: 10 138146 46.73 MB/sec execute 33 sec latency 53.181 ms 2024-03-18T20:51:58.368 INFO:tasks.workunit.client.0.smithi046.stdout: 10 139189 46.76 MB/sec execute 34 sec latency 58.590 ms 2024-03-18T20:51:58.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:58 smithi073 ceph-mon[47384]: pgmap v497: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.7 MiB/s wr, 686 op/s 2024-03-18T20:51:58.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:58 smithi155 ceph-mon[48344]: pgmap v497: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.7 MiB/s wr, 686 op/s 2024-03-18T20:51:58.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:58 smithi046 ceph-mon[43495]: pgmap v497: 97 pgs: 97 active+clean; 1.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.7 MiB/s wr, 686 op/s 2024-03-18T20:51:59.368 INFO:tasks.workunit.client.0.smithi046.stdout: 10 140266 47.31 MB/sec execute 35 sec latency 33.894 ms 2024-03-18T20:51:59.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:51:59 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:59.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:51:59 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:51:59.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:51:59 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:52:00.368 INFO:tasks.workunit.client.0.smithi046.stdout: 10 140935 47.01 MB/sec execute 36 sec latency 72.220 ms 2024-03-18T20:52:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:00 smithi073 ceph-mon[47384]: pgmap v498: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 993 op/s 2024-03-18T20:52:00.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:00 smithi155 ceph-mon[48344]: pgmap v498: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 993 op/s 2024-03-18T20:52:00.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:00 smithi046 ceph-mon[43495]: pgmap v498: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 993 op/s 2024-03-18T20:52:01.368 INFO:tasks.workunit.client.0.smithi046.stdout: 10 141031 45.84 MB/sec execute 37 sec latency 870.972 ms 2024-03-18T20:52:02.368 INFO:tasks.workunit.client.0.smithi046.stdout: 10 141225 44.85 MB/sec execute 38 sec latency 1610.523 ms 2024-03-18T20:52:02.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:02 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:52:02.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:02 smithi073 ceph-mon[47384]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 680 op/s 2024-03-18T20:52:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:02 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:52:02.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:02 smithi155 ceph-mon[48344]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 680 op/s 2024-03-18T20:52:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:02 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:52:02.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:02 smithi046 ceph-mon[43495]: pgmap v499: 97 pgs: 97 active+clean; 1.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.8 MiB/s wr, 680 op/s 2024-03-18T20:52:03.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 141630 44.05 MB/sec execute 39 sec latency 271.270 ms 2024-03-18T20:52:04.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 141754 43.09 MB/sec execute 40 sec latency 345.288 ms 2024-03-18T20:52:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:04 smithi073 ceph-mon[47384]: osdmap e101: 12 total, 12 up, 12 in 2024-03-18T20:52:04.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:04 smithi073 ceph-mon[47384]: pgmap v501: 97 pgs: 8 active+clean+snaptrim, 83 active+clean, 6 active+clean+snaptrim_wait; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.6 MiB/s wr, 767 op/s 2024-03-18T20:52:04.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:04 smithi155 ceph-mon[48344]: osdmap e101: 12 total, 12 up, 12 in 2024-03-18T20:52:04.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:04 smithi155 ceph-mon[48344]: pgmap v501: 97 pgs: 8 active+clean+snaptrim, 83 active+clean, 6 active+clean+snaptrim_wait; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.6 MiB/s wr, 767 op/s 2024-03-18T20:52:04.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:04 smithi046 ceph-mon[43495]: osdmap e101: 12 total, 12 up, 12 in 2024-03-18T20:52:04.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:04 smithi046 ceph-mon[43495]: pgmap v501: 97 pgs: 8 active+clean+snaptrim, 83 active+clean, 6 active+clean+snaptrim_wait; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.6 MiB/s wr, 767 op/s 2024-03-18T20:52:05.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 142685 43.32 MB/sec execute 41 sec latency 114.337 ms 2024-03-18T20:52:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.191+0000 7f38c47b0700 1 -- 172.21.15.46:0/3571282139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc0ffbe0 msgr2=0x7f38bc10dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.191+0000 7f38c47b0700 1 --2- 172.21.15.46:0/3571282139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc0ffbe0 0x7f38bc10dbf0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009fc0 tx=0x7f38b400b330 comp rx=0 tx=0).stop 2024-03-18T20:52:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.191+0000 7f38c47b0700 1 -- 172.21.15.46:0/3571282139 shutdown_connections 2024-03-18T20:52:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.191+0000 7f38c47b0700 1 --2- 172.21.15.46:0/3571282139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc0ffbe0 0x7f38bc10dbf0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.191+0000 7f38c47b0700 1 --2- 172.21.15.46:0/3571282139 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38bc106e30 0x7f38bc107210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.191+0000 7f38c47b0700 1 --2- 172.21.15.46:0/3571282139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38bc0ff220 0x7f38bc0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.191+0000 7f38c47b0700 1 -- 172.21.15.46:0/3571282139 >> 172.21.15.46:0/3571282139 conn(0x7f38bc0761b0 msgr2=0x7f38bc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.191+0000 7f38c47b0700 1 -- 172.21.15.46:0/3571282139 shutdown_connections 2024-03-18T20:52:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.191+0000 7f38c47b0700 1 -- 172.21.15.46:0/3571282139 wait complete. 2024-03-18T20:52:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.192+0000 7f38c47b0700 1 Processor -- start 2024-03-18T20:52:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.192+0000 7f38c47b0700 1 -- start start 2024-03-18T20:52:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.192+0000 7f38c47b0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38bc0ff220 0x7f38bc1a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.192+0000 7f38c47b0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc0ffbe0 0x7f38bc1a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.192+0000 7f38c47b0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38bc106e30 0x7f38bc10ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.192+0000 7f38c47b0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc10d5d0 con 0x7f38bc0ffbe0 2024-03-18T20:52:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.193+0000 7f38c47b0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc10d710 con 0x7f38bc0ff220 2024-03-18T20:52:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.193+0000 7f38c47b0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc10d8d0 con 0x7f38bc106e30 2024-03-18T20:52:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.193+0000 7f38c254c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38bc0ff220 0x7f38bc1a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.193+0000 7f38c254c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38bc0ff220 0x7f38bc1a3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56996/0 (socket says 172.21.15.46:56996) 2024-03-18T20:52:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.193+0000 7f38c254c700 1 -- 172.21.15.46:0/3911095300 learned_addr learned my addr 172.21.15.46:0/3911095300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.194+0000 7f38c254c700 1 -- 172.21.15.46:0/3911095300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38bc106e30 msgr2=0x7f38bc10ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.194+0000 7f38c254c700 1 --2- 172.21.15.46:0/3911095300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38bc106e30 0x7f38bc10ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.194+0000 7f38c254c700 1 -- 172.21.15.46:0/3911095300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc0ffbe0 msgr2=0x7f38bc1a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.194+0000 7f38c254c700 1 --2- 172.21.15.46:0/3911095300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc0ffbe0 0x7f38bc1a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.194+0000 7f38c254c700 1 -- 172.21.15.46:0/3911095300 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b4009c40 con 0x7f38bc0ff220 2024-03-18T20:52:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.194+0000 7f38c254c700 1 --2- 172.21.15.46:0/3911095300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38bc0ff220 0x7f38bc1a3710 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00e9a0 tx=0x7f38ac00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.195+0000 7f38b37fe700 1 -- 172.21.15.46:0/3911095300 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00cbd0 con 0x7f38bc0ff220 2024-03-18T20:52:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.195+0000 7f38b37fe700 1 -- 172.21.15.46:0/3911095300 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38ac00cd30 con 0x7f38bc0ff220 2024-03-18T20:52:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.195+0000 7f38b37fe700 1 -- 172.21.15.46:0/3911095300 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac018680 con 0x7f38bc0ff220 2024-03-18T20:52:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.195+0000 7f38c47b0700 1 -- 172.21.15.46:0/3911095300 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38bc10dbb0 con 0x7f38bc0ff220 2024-03-18T20:52:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.195+0000 7f38c47b0700 1 -- 172.21.15.46:0/3911095300 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38bc06a9b0 con 0x7f38bc0ff220 2024-03-18T20:52:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.200+0000 7f38d0945b80 1 Processor -- start 2024-03-18T20:52:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.201+0000 7f38b37fe700 1 -- 172.21.15.46:0/3911095300 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f38ac010430 con 0x7f38bc0ff220 2024-03-18T20:52:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.202+0000 7f38d0945b80 1 -- start start 2024-03-18T20:52:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.202+0000 7f38d0945b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12570 0x5647c6e12950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.203+0000 7f38d0945b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12e90 0x5647c6e1d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.203+0000 7f38d0945b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1ddb0 0x5647c6e201a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.203+0000 7f38d0945b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6d6c930 con 0x5647c6e1ddb0 2024-03-18T20:52:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.203+0000 7f38d0945b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6d6a2f0 con 0x5647c6e12570 2024-03-18T20:52:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.203+0000 7f38d0945b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6d65be0 con 0x5647c6e12e90 2024-03-18T20:52:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.203+0000 7f38b37fe700 1 --2- 172.21.15.46:0/3911095300 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38a80a13e0 0x7f38a80a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.203+0000 7f38b37fe700 1 -- 172.21.15.46:0/3911095300 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f38ac014070 con 0x7f38bc0ff220 2024-03-18T20:52:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.205+0000 7f38c1d4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1ddb0 0x5647c6e201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.205+0000 7f38c1d4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1ddb0 0x5647c6e201a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39454/0 (socket says 172.21.15.46:39454) 2024-03-18T20:52:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.205+0000 7f38c1d4b700 1 -- 172.21.15.46:0/1910154997 learned_addr learned my addr 172.21.15.46:0/1910154997 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.206+0000 7f38c1d4b700 1 -- 172.21.15.46:0/1910154997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12e90 msgr2=0x5647c6e1d870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:52:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.206+0000 7f38c1d4b700 1 --2- 172.21.15.46:0/1910154997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12e90 0x5647c6e1d870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.206+0000 7f38c1d4b700 1 -- 172.21.15.46:0/1910154997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12570 msgr2=0x5647c6e12950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:52:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.206+0000 7f38c1d4b700 1 --2- 172.21.15.46:0/1910154997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12570 0x5647c6e12950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.206+0000 7f38c1d4b700 1 -- 172.21.15.46:0/1910154997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647c6be7350 con 0x5647c6e1ddb0 2024-03-18T20:52:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.206+0000 7f38c1d4b700 1 --2- 172.21.15.46:0/1910154997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1ddb0 0x5647c6e201a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009b70 tx=0x7f38b8009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758b23902d6cfa16 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.207+0000 7f38b0ff9700 1 -- 172.21.15.46:0/1910154997 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8004220 con 0x5647c6e1ddb0 2024-03-18T20:52:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.207+0000 7f38b0ff9700 1 -- 172.21.15.46:0/1910154997 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38b800edc0 con 0x5647c6e1ddb0 2024-03-18T20:52:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.207+0000 7f38b0ff9700 1 -- 172.21.15.46:0/1910154997 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8003db0 con 0x5647c6e1ddb0 2024-03-18T20:52:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.208+0000 7f38c2d4d700 1 --2- 172.21.15.46:0/3911095300 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38a80a13e0 0x7f38a80a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.208+0000 7f38d0945b80 1 -- 172.21.15.46:0/1910154997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1ddb0 msgr2=0x5647c6e201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.208+0000 7f38d0945b80 1 --2- 172.21.15.46:0/1910154997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1ddb0 0x5647c6e201a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009b70 tx=0x7f38b8009e80 comp rx=0 tx=0).stop 2024-03-18T20:52:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.208+0000 7f38d0945b80 1 -- 172.21.15.46:0/1910154997 shutdown_connections 2024-03-18T20:52:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.208+0000 7f38d0945b80 1 --2- 172.21.15.46:0/1910154997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1ddb0 0x5647c6e201a0 secure :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009b70 tx=0x7f38b8009e80 comp rx=0 tx=0).stop 2024-03-18T20:52:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.208+0000 7f38d0945b80 1 --2- 172.21.15.46:0/1910154997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12570 0x5647c6e12950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.208+0000 7f38d0945b80 1 --2- 172.21.15.46:0/1910154997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12e90 0x5647c6e1d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.208+0000 7f38d0945b80 1 -- 172.21.15.46:0/1910154997 >> 172.21.15.46:0/1910154997 conn(0x5647c6d891c0 msgr2=0x5647c6e0ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:06.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.208+0000 7f38d0945b80 1 -- 172.21.15.46:0/1910154997 shutdown_connections 2024-03-18T20:52:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.208+0000 7f38d0945b80 1 -- 172.21.15.46:0/1910154997 wait complete. 2024-03-18T20:52:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.209+0000 7f38d0945b80 1 -- start start 2024-03-18T20:52:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.209+0000 7f38d0945b80 1 Processor -- start 2024-03-18T20:52:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.210+0000 7f38c2d4d700 1 --2- 172.21.15.46:0/3911095300 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38a80a13e0 0x7f38a80a38a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f38b4015000 tx=0x7f38b4009f50 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.210+0000 7f38d0945b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e12570 0x5647c6e1ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.210+0000 7f38d0945b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12e90 0x5647c6e1b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.210+0000 7f38d0945b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e1b810 0x5647c6e29af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.210+0000 7f38d0945b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6d01d20 con 0x5647c6e12570 2024-03-18T20:52:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.210+0000 7f38d0945b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6d019c0 con 0x5647c6e12e90 2024-03-18T20:52:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.210+0000 7f38d0945b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6be9640 con 0x5647c6e1b810 2024-03-18T20:52:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.210+0000 7f38d0945b80 10 client.0 _ll_register_callbacks cb 0x5647c6b6d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:52:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.210+0000 7f38c254c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e12570 0x5647c6e1ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.210+0000 7f38c254c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e12570 0x5647c6e1ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39468/0 (socket says 172.21.15.46:39468) 2024-03-18T20:52:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.210+0000 7f38c2d4d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e1b810 0x5647c6e29af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.210+0000 7f38c254c700 1 -- 172.21.15.46:0/895513137 learned_addr learned my addr 172.21.15.46:0/895513137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.211+0000 7f38c254c700 1 -- 172.21.15.46:0/895513137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e1b810 msgr2=0x5647c6e29af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.211+0000 7f38c254c700 1 --2- 172.21.15.46:0/895513137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e1b810 0x5647c6e29af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.211+0000 7f38c254c700 1 -- 172.21.15.46:0/895513137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12e90 msgr2=0x5647c6e1b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.211+0000 7f38c254c700 1 --2- 172.21.15.46:0/895513137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12e90 0x5647c6e1b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.211+0000 7f38c254c700 1 -- 172.21.15.46:0/895513137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b8009820 con 0x5647c6e12570 2024-03-18T20:52:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.211+0000 7f38c254c700 1 --2- 172.21.15.46:0/895513137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e12570 0x5647c6e1ad90 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00f010 tx=0x7f38ac09a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681a0b92eb754485 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.211+0000 7f38a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:52:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.211+0000 7f38a5ffb700 1 -- 172.21.15.46:0/895513137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0a2470 con 0x5647c6e12570 2024-03-18T20:52:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.211+0000 7f38a5ffb700 1 -- 172.21.15.46:0/895513137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f38ac0a1fe0 con 0x5647c6e12570 2024-03-18T20:52:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.211+0000 7f38d0945b80 1 -- 172.21.15.46:0/895513137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647c6e066c0 con 0x5647c6e12570 2024-03-18T20:52:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.212+0000 7f38a5ffb700 1 -- 172.21.15.46:0/895513137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38ac0a21c0 con 0x5647c6e12570 2024-03-18T20:52:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.212+0000 7f38d0945b80 10 client.20331 fetch_fsmap learned FSMap version 76 2024-03-18T20:52:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.212+0000 7f38d0945b80 1 -- 172.21.15.46:0/895513137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5647c6b93b00 con 0x5647c6e12570 2024-03-18T20:52:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.212+0000 7f38a5ffb700 1 -- 172.21.15.46:0/895513137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647c6e066c0 con 0x5647c6e12570 2024-03-18T20:52:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.212+0000 7f38a5ffb700 1 -- 172.21.15.46:0/895513137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 76) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f38ac010840 con 0x5647c6e12570 2024-03-18T20:52:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.213+0000 7f38d0945b80 10 client.20331 fetch_fsmap finished waiting for FSMap version 76 2024-03-18T20:52:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.213+0000 7f38d0945b80 10 client.20331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:52:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.213+0000 7f38d0945b80 20 client.20331 populate_metadata read hostname 'smithi046' 2024-03-18T20:52:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.213+0000 7f38d0945b80 1 --2- 172.21.15.46:0/895513137 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647c6e2df80 0x5647c6e303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.213+0000 7f38d0945b80 4 client.20331 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:52:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.213+0000 7f38d0945b80 1 -- 172.21.15.46:0/895513137 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647c6d5e6a0 con 0x5647c6e2df80 2024-03-18T20:52:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.213+0000 7f38c1d4b700 1 --2- 172.21.15.46:0/895513137 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647c6e2df80 0x5647c6e303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.214+0000 7f38c1d4b700 1 --2- 172.21.15.46:0/895513137 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647c6e2df80 0x5647c6e303c0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5bdf1eadbb6489b server_cookie=23389001200acab7 in_seq=0 out_seq=0 2024-03-18T20:52:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.214+0000 7f38a5ffb700 10 client.20331 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.215+0000 7f38a5ffb700 1 -- 172.21.15.46:0/895513137 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5647c6d5e6a0 con 0x5647c6e2df80 2024-03-18T20:52:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.215+0000 7f38a5ffb700 10 client.20331 handle_command_reply: tid=0 2024-03-18T20:52:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.215+0000 7f38d0945b80 1 client.20331 shutdown 2024-03-18T20:52:06.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.216+0000 7f38d0945b80 1 -- 172.21.15.46:0/895513137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e12570 msgr2=0x5647c6e1ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:06.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.216+0000 7f38d0945b80 1 --2- 172.21.15.46:0/895513137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e12570 0x5647c6e1ad90 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00f010 tx=0x7f38ac09a820 comp rx=0 tx=0).stop 2024-03-18T20:52:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.216+0000 7f38d0945b80 1 -- 172.21.15.46:0/895513137 shutdown_connections 2024-03-18T20:52:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.216+0000 7f38d0945b80 1 --2- 172.21.15.46:0/895513137 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647c6e2df80 0x5647c6e303c0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.216+0000 7f38d0945b80 1 --2- 172.21.15.46:0/895513137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e12570 0x5647c6e1ad90 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.216+0000 7f38d0945b80 1 --2- 172.21.15.46:0/895513137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e1b810 0x5647c6e29af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.216+0000 7f38d0945b80 1 --2- 172.21.15.46:0/895513137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12e90 0x5647c6e1b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.216+0000 7f38d0945b80 1 -- 172.21.15.46:0/895513137 >> 172.21.15.46:0/895513137 conn(0x5647c6d891c0 msgr2=0x5647c6e0e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.216+0000 7f38a5ffb700 0 client.20331 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.216+0000 7f38d0945b80 1 -- 172.21.15.46:0/895513137 shutdown_connections 2024-03-18T20:52:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.216+0000 7f38d0945b80 1 -- 172.21.15.46:0/895513137 wait complete. 2024-03-18T20:52:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.216+0000 7f38d0945b80 20 client.20331 trim_cache size 0 max 16384 2024-03-18T20:52:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38d0945b80 1 Processor -- start 2024-03-18T20:52:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38d0945b80 1 -- start start 2024-03-18T20:52:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38d0945b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 0x5647c6e29190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38d0945b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e12e90 0x5647c6e296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38d0945b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e1b810 0x5647c6e18050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38d0945b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6e08c50 con 0x5647c6e12e90 2024-03-18T20:52:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38d0945b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6e2aad0 con 0x5647c6e1b810 2024-03-18T20:52:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38d0945b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6e1c2f0 con 0x5647c6e12570 2024-03-18T20:52:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38c254c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 0x5647c6e29190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38c254c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 0x5647c6e29190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:54718/0 (socket says 172.21.15.46:54718) 2024-03-18T20:52:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38c254c700 1 -- 172.21.15.46:0/1266802208 learned_addr learned my addr 172.21.15.46:0/1266802208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38c2d4d700 1 --2- 172.21.15.46:0/1266802208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e1b810 0x5647c6e18050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.239+0000 7f38c1d4b700 1 --2- 172.21.15.46:0/1266802208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e12e90 0x5647c6e296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.240+0000 7f38c254c700 1 -- 172.21.15.46:0/1266802208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e1b810 msgr2=0x5647c6e18050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.240+0000 7f38c254c700 1 --2- 172.21.15.46:0/1266802208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e1b810 0x5647c6e18050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.240+0000 7f38c254c700 1 -- 172.21.15.46:0/1266802208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e12e90 msgr2=0x5647c6e296d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.240+0000 7f38c254c700 1 --2- 172.21.15.46:0/1266802208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e12e90 0x5647c6e296d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.240+0000 7f38c254c700 1 -- 172.21.15.46:0/1266802208 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647c6d2be40 con 0x5647c6e12570 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.240+0000 7f38c254c700 1 --2- 172.21.15.46:0/1266802208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 0x5647c6e29190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0cac30 tx=0x7f38ac096460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10a7065648866b3d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.240+0000 7f38b17fa700 1 -- 172.21.15.46:0/1266802208 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0a2470 con 0x5647c6e12570 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.240+0000 7f38b17fa700 1 -- 172.21.15.46:0/1266802208 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38ac0b82c0 con 0x5647c6e12570 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.240+0000 7f38b17fa700 1 -- 172.21.15.46:0/1266802208 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0a1fe0 con 0x5647c6e12570 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.241+0000 7f38d0945b80 1 -- 172.21.15.46:0/1266802208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 msgr2=0x5647c6e29190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.241+0000 7f38d0945b80 1 --2- 172.21.15.46:0/1266802208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 0x5647c6e29190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0cac30 tx=0x7f38ac096460 comp rx=0 tx=0).stop 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.241+0000 7f38d0945b80 1 -- 172.21.15.46:0/1266802208 shutdown_connections 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.241+0000 7f38d0945b80 1 --2- 172.21.15.46:0/1266802208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e1b810 0x5647c6e18050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.242+0000 7f38d0945b80 1 --2- 172.21.15.46:0/1266802208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 0x5647c6e29190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.242+0000 7f38d0945b80 1 --2- 172.21.15.46:0/1266802208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e12e90 0x5647c6e296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.242+0000 7f38d0945b80 1 -- 172.21.15.46:0/1266802208 >> 172.21.15.46:0/1266802208 conn(0x5647c6d891c0 msgr2=0x5647c6e196c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.242+0000 7f38d0945b80 1 -- 172.21.15.46:0/1266802208 shutdown_connections 2024-03-18T20:52:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.242+0000 7f38d0945b80 1 -- 172.21.15.46:0/1266802208 wait complete. 2024-03-18T20:52:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.242+0000 7f38d0945b80 1 -- start start 2024-03-18T20:52:06.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.242+0000 7f38d0945b80 1 Processor -- start 2024-03-18T20:52:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38d0945b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 0x5647c6e22230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38d0945b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12e90 0x5647c6e22770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38d0945b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1b810 0x5647c6e22cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38d0945b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6e80e10 con 0x5647c6e1b810 2024-03-18T20:52:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38d0945b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6e80f50 con 0x5647c6e12e90 2024-03-18T20:52:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38d0945b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c6e81090 con 0x5647c6e12570 2024-03-18T20:52:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38d0945b80 10 client.0 _ll_register_callbacks cb 0x5647c6b6d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:52:06.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38c254c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 0x5647c6e22230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38c254c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 0x5647c6e22230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:54734/0 (socket says 172.21.15.46:54734) 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38c254c700 1 -- 172.21.15.46:0/2098448611 learned_addr learned my addr 172.21.15.46:0/2098448611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38c2d4d700 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1b810 0x5647c6e22cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38c254c700 1 -- 172.21.15.46:0/2098448611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12e90 msgr2=0x5647c6e22770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38c1d4b700 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12e90 0x5647c6e22770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38c254c700 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12e90 0x5647c6e22770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38c254c700 1 -- 172.21.15.46:0/2098448611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1b810 msgr2=0x5647c6e22cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38c254c700 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1b810 0x5647c6e22cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38c254c700 1 -- 172.21.15.46:0/2098448611 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647c6b93b00 con 0x5647c6e12570 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38c254c700 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 0x5647c6e22230 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0a1f40 tx=0x7f38ac00c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96125dadc74183c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.243+0000 7f38a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38c1d4b700 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12e90 0x5647c6e22770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:52:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38c2d4d700 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1b810 0x5647c6e22cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38a5ffb700 1 -- 172.21.15.46:0/2098448611 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0a21e0 con 0x5647c6e12570 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38d0945b80 1 -- 172.21.15.46:0/2098448611 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38b800edc0 con 0x5647c6e12570 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38a5ffb700 1 -- 172.21.15.46:0/2098448611 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f38ac0a2470 con 0x5647c6e12570 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38a5ffb700 1 -- 172.21.15.46:0/2098448611 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38ac0b2ba0 con 0x5647c6e12570 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38d0945b80 10 client.30007 fetch_fsmap learned FSMap version 76 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38d0945b80 1 -- 172.21.15.46:0/2098448611 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) v3 -- 0x5647c6bebc70 con 0x5647c6e12570 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38a5ffb700 1 -- 172.21.15.46:0/2098448611 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38b800edc0 con 0x5647c6e12570 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38a5ffb700 1 -- 172.21.15.46:0/2098448611 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 76) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f38ac010880 con 0x5647c6e12570 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38d0945b80 10 client.30007 fetch_fsmap finished waiting for FSMap version 76 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38d0945b80 10 client.30007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38d0945b80 20 client.30007 populate_metadata read hostname 'smithi046' 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38d0945b80 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647c6e82360 0x5647c6e84750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38d0945b80 4 client.30007 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:52:06.250 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.244+0000 7f38d0945b80 1 -- 172.21.15.46:0/2098448611 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5647c6bb4c90 con 0x5647c6e82360 2024-03-18T20:52:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.245+0000 7f38c1d4b700 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647c6e82360 0x5647c6e84750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.245+0000 7f38a5ffb700 10 client.30007 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.245+0000 7f38c1d4b700 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647c6e82360 0x5647c6e84750 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55419ec13938956 server_cookie=3cd049a929e7c92d in_seq=0 out_seq=0 2024-03-18T20:52:06.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 143689 43.92 MB/sec execute 42 sec latency 70.210 ms 2024-03-18T20:52:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.460+0000 7f38a5ffb700 1 -- 172.21.15.46:0/2098448611 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5647c6bb4c90 con 0x5647c6e82360 2024-03-18T20:52:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.460+0000 7f38a5ffb700 10 client.30007 handle_command_reply: tid=0 2024-03-18T20:52:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.460+0000 7f38d0945b80 1 client.30007 shutdown 2024-03-18T20:52:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.460+0000 7f38d0945b80 1 -- 172.21.15.46:0/2098448611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 msgr2=0x5647c6e22230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.460+0000 7f38d0945b80 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 0x5647c6e22230 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0a1f40 tx=0x7f38ac00c6e0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.462+0000 7f38d0945b80 1 -- 172.21.15.46:0/2098448611 shutdown_connections 2024-03-18T20:52:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.462+0000 7f38d0945b80 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5647c6e82360 0x5647c6e84750 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.462+0000 7f38d0945b80 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647c6e1b810 0x5647c6e22cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.462+0000 7f38d0945b80 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5647c6e12570 0x5647c6e22230 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.462+0000 7f38d0945b80 1 --2- 172.21.15.46:0/2098448611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647c6e12e90 0x5647c6e22770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.462+0000 7f38d0945b80 1 -- 172.21.15.46:0/2098448611 >> 172.21.15.46:0/2098448611 conn(0x5647c6d891c0 msgr2=0x5647c6e29190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.462+0000 7f38a5ffb700 0 client.30007 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.462+0000 7f38d0945b80 1 -- 172.21.15.46:0/2098448611 shutdown_connections 2024-03-18T20:52:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.462+0000 7f38d0945b80 1 -- 172.21.15.46:0/2098448611 wait complete. 2024-03-18T20:52:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.462+0000 7f38d0945b80 20 client.30007 trim_cache size 0 max 16384 2024-03-18T20:52:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:52:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:52:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:52:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:52:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.463+0000 7f38c47b0700 1 -- 172.21.15.46:0/3911095300 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38a80a13e0 msgr2=0x7f38a80a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.463+0000 7f38c47b0700 1 --2- 172.21.15.46:0/3911095300 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38a80a13e0 0x7f38a80a38a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f38b4015000 tx=0x7f38b4009f50 comp rx=0 tx=0).stop 2024-03-18T20:52:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.463+0000 7f38c47b0700 1 -- 172.21.15.46:0/3911095300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38bc0ff220 msgr2=0x7f38bc1a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.463+0000 7f38c47b0700 1 --2- 172.21.15.46:0/3911095300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38bc0ff220 0x7f38bc1a3710 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00e9a0 tx=0x7f38ac00ed60 comp rx=0 tx=0).stop 2024-03-18T20:52:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.463+0000 7f38c47b0700 1 -- 172.21.15.46:0/3911095300 shutdown_connections 2024-03-18T20:52:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.463+0000 7f38c47b0700 1 --2- 172.21.15.46:0/3911095300 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38a80a13e0 0x7f38a80a38a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.463+0000 7f38c47b0700 1 --2- 172.21.15.46:0/3911095300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38bc106e30 0x7f38bc10ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.463+0000 7f38c47b0700 1 --2- 172.21.15.46:0/3911095300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38bc0ff220 0x7f38bc1a3710 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.463+0000 7f38c47b0700 1 --2- 172.21.15.46:0/3911095300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38bc0ffbe0 0x7f38bc1a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.463+0000 7f38c47b0700 1 -- 172.21.15.46:0/3911095300 >> 172.21.15.46:0/3911095300 conn(0x7f38bc0761b0 msgr2=0x7f38bc0fc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.463+0000 7f38c47b0700 1 -- 172.21.15.46:0/3911095300 shutdown_connections 2024-03-18T20:52:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:06.464+0000 7f38c47b0700 1 -- 172.21.15.46:0/3911095300 wait complete. 2024-03-18T20:52:06.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:06 smithi073 ceph-mon[47384]: pgmap v502: 97 pgs: 24 active+clean+snaptrim, 33 active+clean, 40 active+clean+snaptrim_wait; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 741 op/s 2024-03-18T20:52:06.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:06 smithi155 ceph-mon[48344]: pgmap v502: 97 pgs: 24 active+clean+snaptrim, 33 active+clean, 40 active+clean+snaptrim_wait; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 741 op/s 2024-03-18T20:52:06.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:06 smithi046 ceph-mon[43495]: pgmap v502: 97 pgs: 24 active+clean+snaptrim, 33 active+clean, 40 active+clean+snaptrim_wait; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 741 op/s 2024-03-18T20:52:07.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 144314 43.55 MB/sec execute 43 sec latency 129.654 ms 2024-03-18T20:52:07.545 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 20:52:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T20:52:07.097+0000 7ff59139e700 -1 calc_pg_upmaps abort due to max <= 0 2024-03-18T20:52:08.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 145288 43.71 MB/sec execute 44 sec latency 47.042 ms 2024-03-18T20:52:08.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:08 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:52:08.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:08 smithi073 ceph-mon[47384]: pgmap v503: 97 pgs: 24 active+clean+snaptrim, 33 active+clean, 40 active+clean+snaptrim_wait; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 741 op/s 2024-03-18T20:52:08.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:08 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:52:08.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:08 smithi155 ceph-mon[48344]: pgmap v503: 97 pgs: 24 active+clean+snaptrim, 33 active+clean, 40 active+clean+snaptrim_wait; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 741 op/s 2024-03-18T20:52:08.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:08 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:52:08.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:08 smithi046 ceph-mon[43495]: pgmap v503: 97 pgs: 24 active+clean+snaptrim, 33 active+clean, 40 active+clean+snaptrim_wait; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 741 op/s 2024-03-18T20:52:09.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 146267 43.96 MB/sec execute 45 sec latency 81.974 ms 2024-03-18T20:52:10.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 147148 43.96 MB/sec execute 46 sec latency 37.527 ms 2024-03-18T20:52:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:10 smithi073 ceph-mon[47384]: pgmap v504: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.4 MiB/s wr, 868 op/s 2024-03-18T20:52:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:10 smithi073 ceph-mon[47384]: osdmap e102: 12 total, 12 up, 12 in 2024-03-18T20:52:10.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:10 smithi155 ceph-mon[48344]: pgmap v504: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.4 MiB/s wr, 868 op/s 2024-03-18T20:52:10.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:10 smithi155 ceph-mon[48344]: osdmap e102: 12 total, 12 up, 12 in 2024-03-18T20:52:10.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:10 smithi046 ceph-mon[43495]: pgmap v504: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.4 MiB/s wr, 868 op/s 2024-03-18T20:52:10.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:10 smithi046 ceph-mon[43495]: osdmap e102: 12 total, 12 up, 12 in 2024-03-18T20:52:11.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 148182 44.19 MB/sec execute 47 sec latency 40.881 ms 2024-03-18T20:52:12.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 149140 44.42 MB/sec execute 48 sec latency 69.202 ms 2024-03-18T20:52:12.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:12 smithi073 ceph-mon[47384]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.0 MiB/s wr, 1.06k op/s 2024-03-18T20:52:12.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:12 smithi155 ceph-mon[48344]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.0 MiB/s wr, 1.06k op/s 2024-03-18T20:52:12.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:12 smithi046 ceph-mon[43495]: pgmap v506: 97 pgs: 97 active+clean; 1.2 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.0 MiB/s wr, 1.06k op/s 2024-03-18T20:52:13.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 149769 44.28 MB/sec execute 49 sec latency 199.147 ms 2024-03-18T20:52:14.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 150391 43.99 MB/sec execute 50 sec latency 233.278 ms 2024-03-18T20:52:14.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:14 smithi073 ceph-mon[47384]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.5 MiB/s wr, 899 op/s 2024-03-18T20:52:14.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:14 smithi155 ceph-mon[48344]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.5 MiB/s wr, 899 op/s 2024-03-18T20:52:14.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:14 smithi046 ceph-mon[43495]: pgmap v507: 97 pgs: 97 active+clean; 1.2 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.5 MiB/s wr, 899 op/s 2024-03-18T20:52:15.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 151236 43.98 MB/sec execute 51 sec latency 92.324 ms 2024-03-18T20:52:16.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 152175 44.13 MB/sec execute 52 sec latency 57.168 ms 2024-03-18T20:52:16.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:16 smithi073 ceph-mon[47384]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 853 op/s 2024-03-18T20:52:16.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:16 smithi155 ceph-mon[48344]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 853 op/s 2024-03-18T20:52:16.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:16 smithi046 ceph-mon[43495]: pgmap v508: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 853 op/s 2024-03-18T20:52:17.369 INFO:tasks.workunit.client.0.smithi046.stdout: 10 153133 44.29 MB/sec execute 53 sec latency 97.174 ms 2024-03-18T20:52:18.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 153944 44.37 MB/sec execute 54 sec latency 90.444 ms 2024-03-18T20:52:18.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:18 smithi073 ceph-mon[47384]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 853 op/s 2024-03-18T20:52:18.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:18 smithi155 ceph-mon[48344]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 853 op/s 2024-03-18T20:52:18.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:18 smithi046 ceph-mon[43495]: pgmap v509: 97 pgs: 97 active+clean; 1.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.2 MiB/s wr, 853 op/s 2024-03-18T20:52:19.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 154460 44.01 MB/sec execute 55 sec latency 196.057 ms 2024-03-18T20:52:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.650+0000 7fd30f8b7700 1 -- 172.21.15.46:0/181643200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3080680e0 msgr2=0x7fd308068560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.650+0000 7fd30f8b7700 1 --2- 172.21.15.46:0/181643200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3080680e0 0x7fd308068560 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009990 tx=0x7fd2f8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.650+0000 7fd30f8b7700 1 -- 172.21.15.46:0/181643200 shutdown_connections 2024-03-18T20:52:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.650+0000 7fd30f8b7700 1 --2- 172.21.15.46:0/181643200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308068aa0 0x7fd30810dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.650+0000 7fd30f8b7700 1 --2- 172.21.15.46:0/181643200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd308106e10 0x7fd3081071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.650+0000 7fd30f8b7700 1 --2- 172.21.15.46:0/181643200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3080680e0 0x7fd308068560 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.650+0000 7fd30f8b7700 1 -- 172.21.15.46:0/181643200 >> 172.21.15.46:0/181643200 conn(0x7fd308076d50 msgr2=0x7fd308077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.651+0000 7fd30f8b7700 1 -- 172.21.15.46:0/181643200 shutdown_connections 2024-03-18T20:52:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.651+0000 7fd30f8b7700 1 -- 172.21.15.46:0/181643200 wait complete. 2024-03-18T20:52:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.651+0000 7fd30f8b7700 1 Processor -- start 2024-03-18T20:52:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.652+0000 7fd30f8b7700 1 -- start start 2024-03-18T20:52:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.652+0000 7fd30f8b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3080680e0 0x7fd308114f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.652+0000 7fd30f8b7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308068aa0 0x7fd30811ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.652+0000 7fd30f8b7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd308106e10 0x7fd3081100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.652+0000 7fd30f8b7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3081159a0 con 0x7fd3080680e0 2024-03-18T20:52:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.652+0000 7fd30f8b7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd308110620 con 0x7fd308106e10 2024-03-18T20:52:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.652+0000 7fd30f8b7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3081107e0 con 0x7fd308068aa0 2024-03-18T20:52:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.652+0000 7fd30d653700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3080680e0 0x7fd308114f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.652+0000 7fd30de54700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd308106e10 0x7fd3081100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.652+0000 7fd30d653700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3080680e0 0x7fd308114f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44822/0 (socket says 172.21.15.46:44822) 2024-03-18T20:52:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.652+0000 7fd30d653700 1 -- 172.21.15.46:0/418480039 learned_addr learned my addr 172.21.15.46:0/418480039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.652+0000 7fd30ce52700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308068aa0 0x7fd30811ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.653+0000 7fd30ce52700 1 -- 172.21.15.46:0/418480039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd308106e10 msgr2=0x7fd3081100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.653+0000 7fd30ce52700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd308106e10 0x7fd3081100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.653+0000 7fd30ce52700 1 -- 172.21.15.46:0/418480039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3080680e0 msgr2=0x7fd308114f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.653+0000 7fd30ce52700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3080680e0 0x7fd308114f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.653+0000 7fd30ce52700 1 -- 172.21.15.46:0/418480039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f80095f0 con 0x7fd308068aa0 2024-03-18T20:52:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.654+0000 7fd30ce52700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308068aa0 0x7fd30811ad00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8000c00 tx=0x7fd2f8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.655+0000 7fd2fe7fc700 1 -- 172.21.15.46:0/418480039 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f801d070 con 0x7fd308068aa0 2024-03-18T20:52:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.655+0000 7fd2fe7fc700 1 -- 172.21.15.46:0/418480039 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2f8022470 con 0x7fd308068aa0 2024-03-18T20:52:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.655+0000 7fd2fe7fc700 1 -- 172.21.15.46:0/418480039 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8017650 con 0x7fd308068aa0 2024-03-18T20:52:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.655+0000 7fd30f8b7700 1 -- 172.21.15.46:0/418480039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd308110a60 con 0x7fd308068aa0 2024-03-18T20:52:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.655+0000 7fd30f8b7700 1 -- 172.21.15.46:0/418480039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd308110ec0 con 0x7fd308068aa0 2024-03-18T20:52:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.660+0000 7fd2fe7fc700 1 -- 172.21.15.46:0/418480039 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd2f80225e0 con 0x7fd308068aa0 2024-03-18T20:52:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd2fe7fc700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2f40a13e0 0x7fd2f40a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd31ba4cb80 1 Processor -- start 2024-03-18T20:52:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd2fe7fc700 1 -- 172.21.15.46:0/418480039 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd2f80cddb0 con 0x7fd308068aa0 2024-03-18T20:52:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd31ba4cb80 1 -- start start 2024-03-18T20:52:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd31ba4cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cb570 0x5595d21cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd31ba4cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cbe90 0x5595d21d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd31ba4cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 0x5595d21d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd31ba4cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d2125930 con 0x5595d21cbe90 2024-03-18T20:52:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd31ba4cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d21232f0 con 0x5595d21d6db0 2024-03-18T20:52:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd31ba4cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d211ebe0 con 0x5595d21cb570 2024-03-18T20:52:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd30d653700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cb570 0x5595d21cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd30d653700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cb570 0x5595d21cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:46000/0 (socket says 172.21.15.46:46000) 2024-03-18T20:52:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.661+0000 7fd30d653700 1 -- 172.21.15.46:0/771372742 learned_addr learned my addr 172.21.15.46:0/771372742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.662+0000 7fd30ce52700 1 --2- 172.21.15.46:0/771372742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cbe90 0x5595d21d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.662+0000 7fd30de54700 1 --2- 172.21.15.46:0/771372742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 0x5595d21d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.662+0000 7fd30d653700 1 -- 172.21.15.46:0/771372742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 msgr2=0x5595d21d91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.662+0000 7fd30d653700 1 --2- 172.21.15.46:0/771372742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 0x5595d21d91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.662+0000 7fd30d653700 1 -- 172.21.15.46:0/771372742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cbe90 msgr2=0x5595d21d6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.662+0000 7fd30d653700 1 --2- 172.21.15.46:0/771372742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cbe90 0x5595d21d6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.662+0000 7fd30d653700 1 -- 172.21.15.46:0/771372742 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595d1fa0350 con 0x5595d21cb570 2024-03-18T20:52:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.662+0000 7fd30d653700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2f40a13e0 0x7fd2f40a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.663+0000 7fd30d653700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2f40a13e0 0x7fd2f40a38a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd304018000 tx=0x7fd30401a000 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.664+0000 7fd30d653700 1 --2- 172.21.15.46:0/771372742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cb570 0x5595d21cb950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd304009f20 tx=0x7fd304013740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efa27a7a69a7deda server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.665+0000 7fd2f37fe700 1 -- 172.21.15.46:0/771372742 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304011640 con 0x5595d21cb570 2024-03-18T20:52:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.665+0000 7fd2f37fe700 1 -- 172.21.15.46:0/771372742 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd30401d940 con 0x5595d21cb570 2024-03-18T20:52:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.665+0000 7fd2f37fe700 1 -- 172.21.15.46:0/771372742 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30400e760 con 0x5595d21cb570 2024-03-18T20:52:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.666+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/771372742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cb570 msgr2=0x5595d21cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.666+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/771372742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cb570 0x5595d21cb950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd304009f20 tx=0x7fd304013740 comp rx=0 tx=0).stop 2024-03-18T20:52:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.667+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/771372742 shutdown_connections 2024-03-18T20:52:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.667+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/771372742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 0x5595d21d91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.667+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/771372742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cb570 0x5595d21cb950 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.667+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/771372742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cbe90 0x5595d21d6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.667+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/771372742 >> 172.21.15.46:0/771372742 conn(0x5595d21421c0 msgr2=0x5595d21c7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.667+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/771372742 shutdown_connections 2024-03-18T20:52:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.667+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/771372742 wait complete. 2024-03-18T20:52:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.667+0000 7fd31ba4cb80 1 -- start start 2024-03-18T20:52:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.667+0000 7fd31ba4cb80 1 Processor -- start 2024-03-18T20:52:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.668+0000 7fd31ba4cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21cb570 0x5595d21d6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.668+0000 7fd31ba4cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.668+0000 7fd31ba4cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21d6db0 0x5595d21d40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.668+0000 7fd31ba4cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d20bad20 con 0x5595d21d6db0 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.668+0000 7fd31ba4cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d20ba9c0 con 0x5595d21cb570 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.668+0000 7fd31ba4cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d1fa2640 con 0x5595d21cbe90 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.668+0000 7fd31ba4cb80 10 client.0 _ll_register_callbacks cb 0x5595d1f26340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.668+0000 7fd30de54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21d6db0 0x5595d21d40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.668+0000 7fd30de54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21d6db0 0x5595d21d40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44838/0 (socket says 172.21.15.46:44838) 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.668+0000 7fd30de54700 1 -- 172.21.15.46:0/2066177749 learned_addr learned my addr 172.21.15.46:0/2066177749 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.668+0000 7fd30ce52700 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.669+0000 7fd30d653700 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21cb570 0x5595d21d6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.669+0000 7fd30ce52700 1 -- 172.21.15.46:0/2066177749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21cb570 msgr2=0x5595d21d6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.669+0000 7fd30ce52700 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21cb570 0x5595d21d6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.669+0000 7fd30ce52700 1 -- 172.21.15.46:0/2066177749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21d6db0 msgr2=0x5595d21d40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.669+0000 7fd30ce52700 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21d6db0 0x5595d21d40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.669+0000 7fd30ce52700 1 -- 172.21.15.46:0/2066177749 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd304009b50 con 0x5595d21cbe90 2024-03-18T20:52:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.669+0000 7fd30ce52700 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d3a70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80cb4e0 tx=0x7fd2f8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7761202cb58c16b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.669+0000 7fd2f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:52:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.669+0000 7fd2f0ff9700 1 -- 172.21.15.46:0/2066177749 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f809aa70 con 0x5595d21cbe90 2024-03-18T20:52:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.669+0000 7fd2f0ff9700 1 -- 172.21.15.46:0/2066177749 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd2f809b630 con 0x5595d21cbe90 2024-03-18T20:52:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.670+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/2066177749 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595d21bf6c0 con 0x5595d21cbe90 2024-03-18T20:52:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.670+0000 7fd2f0ff9700 1 -- 172.21.15.46:0/2066177749 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2f809a890 con 0x5595d21cbe90 2024-03-18T20:52:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.670+0000 7fd31ba4cb80 10 client.30019 fetch_fsmap learned FSMap version 77 2024-03-18T20:52:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.670+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/2066177749 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x5595d1f4cb00 con 0x5595d21cbe90 2024-03-18T20:52:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.670+0000 7fd2f0ff9700 1 -- 172.21.15.46:0/2066177749 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5595d21bf6c0 con 0x5595d21cbe90 2024-03-18T20:52:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.671+0000 7fd2f0ff9700 1 -- 172.21.15.46:0/2066177749 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 77) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd2f809a4b0 con 0x5595d21cbe90 2024-03-18T20:52:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.671+0000 7fd31ba4cb80 10 client.30019 fetch_fsmap finished waiting for FSMap version 77 2024-03-18T20:52:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.671+0000 7fd31ba4cb80 10 client.30019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:52:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.671+0000 7fd31ba4cb80 20 client.30019 populate_metadata read hostname 'smithi046' 2024-03-18T20:52:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.671+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5595d21e6630 0x5595d21e8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.671+0000 7fd31ba4cb80 4 client.30019 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:52:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.671+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/2066177749 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595d21176a0 con 0x5595d21e6630 2024-03-18T20:52:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.671+0000 7fd30d653700 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5595d21e6630 0x5595d21e8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.672+0000 7fd30d653700 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5595d21e6630 0x5595d21e8a20 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26b913ffe6bc7305 server_cookie=4a771020f3acdf85 in_seq=0 out_seq=0 2024-03-18T20:52:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.672+0000 7fd2f0ff9700 10 client.30019 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.672+0000 7fd2f0ff9700 1 -- 172.21.15.46:0/2066177749 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5595d21176a0 con 0x5595d21e6630 2024-03-18T20:52:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.672+0000 7fd2f0ff9700 10 client.30019 handle_command_reply: tid=0 2024-03-18T20:52:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.673+0000 7fd31ba4cb80 1 client.30019 shutdown 2024-03-18T20:52:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.673+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/2066177749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 msgr2=0x5595d21d3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.673+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d3a70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80cb4e0 tx=0x7fd2f8005c00 comp rx=0 tx=0).stop 2024-03-18T20:52:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.673+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/2066177749 shutdown_connections 2024-03-18T20:52:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.673+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5595d21e6630 0x5595d21e8a20 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.673+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21d6db0 0x5595d21d40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.673+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21cb570 0x5595d21d6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.673+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/2066177749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d3a70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.673+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/2066177749 >> 172.21.15.46:0/2066177749 conn(0x5595d21421c0 msgr2=0x5595d21c7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.673+0000 7fd2f0ff9700 0 client.30019 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.673+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/2066177749 shutdown_connections 2024-03-18T20:52:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.673+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/2066177749 wait complete. 2024-03-18T20:52:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.674+0000 7fd31ba4cb80 20 client.30019 trim_cache size 0 max 16384 2024-03-18T20:52:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.697+0000 7fd31ba4cb80 1 Processor -- start 2024-03-18T20:52:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.697+0000 7fd31ba4cb80 1 -- start start 2024-03-18T20:52:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.697+0000 7fd31ba4cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 0x5595d21c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.697+0000 7fd31ba4cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.697+0000 7fd31ba4cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 0x5595d21d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.697+0000 7fd31ba4cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d21d3ed0 con 0x5595d21cb570 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.697+0000 7fd31ba4cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d21d49d0 con 0x5595d21d6db0 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.697+0000 7fd31ba4cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d21e34f0 con 0x5595d21cbe90 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.697+0000 7fd30d653700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 0x5595d21c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.697+0000 7fd30d653700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 0x5595d21c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44840/0 (socket says 172.21.15.46:44840) 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.697+0000 7fd30d653700 1 -- 172.21.15.46:0/3030687112 learned_addr learned my addr 172.21.15.46:0/3030687112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.697+0000 7fd30ce52700 1 --2- 172.21.15.46:0/3030687112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.698+0000 7fd30ce52700 1 -- 172.21.15.46:0/3030687112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 msgr2=0x5595d21d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.698+0000 7fd30ce52700 1 --2- 172.21.15.46:0/3030687112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 0x5595d21d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.698+0000 7fd30ce52700 1 -- 172.21.15.46:0/3030687112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 msgr2=0x5595d21c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.698+0000 7fd30ce52700 1 --2- 172.21.15.46:0/3030687112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 0x5595d21c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.698+0000 7fd30ce52700 1 -- 172.21.15.46:0/3030687112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595d20e4e40 con 0x5595d21cbe90 2024-03-18T20:52:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.698+0000 7fd30ce52700 1 --2- 172.21.15.46:0/3030687112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d2340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8099280 tx=0x7fd2f8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=468a3bbda0888c3b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.698+0000 7fd2f3fff700 1 -- 172.21.15.46:0/3030687112 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80b92a0 con 0x5595d21cbe90 2024-03-18T20:52:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.698+0000 7fd2f3fff700 1 -- 172.21.15.46:0/3030687112 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2f809a890 con 0x5595d21cbe90 2024-03-18T20:52:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.698+0000 7fd2f3fff700 1 -- 172.21.15.46:0/3030687112 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f809af30 con 0x5595d21cbe90 2024-03-18T20:52:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.699+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3030687112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 msgr2=0x5595d21d2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.699+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/3030687112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d2340 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8099280 tx=0x7fd2f8009ca0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.700+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3030687112 shutdown_connections 2024-03-18T20:52:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.700+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/3030687112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 0x5595d21c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.700+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/3030687112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 0x5595d21d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.700+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/3030687112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d2340 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.700+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3030687112 >> 172.21.15.46:0/3030687112 conn(0x5595d21421c0 msgr2=0x5595d21ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.703+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3030687112 shutdown_connections 2024-03-18T20:52:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.703+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3030687112 wait complete. 2024-03-18T20:52:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.704+0000 7fd31ba4cb80 1 -- start start 2024-03-18T20:52:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.704+0000 7fd31ba4cb80 1 Processor -- start 2024-03-18T20:52:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.704+0000 7fd31ba4cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 0x5595d21d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.704+0000 7fd31ba4cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.704+0000 7fd31ba4cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 0x5595d224a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.704+0000 7fd31ba4cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d21da6f0 con 0x5595d21cb570 2024-03-18T20:52:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.704+0000 7fd31ba4cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d21da860 con 0x5595d21d6db0 2024-03-18T20:52:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.704+0000 7fd31ba4cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595d224a970 con 0x5595d21cbe90 2024-03-18T20:52:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.704+0000 7fd31ba4cb80 10 client.0 _ll_register_callbacks cb 0x7fd2f80a10b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:52:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.705+0000 7fd30d653700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 0x5595d21d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.705+0000 7fd30d653700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 0x5595d21d9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44850/0 (socket says 172.21.15.46:44850) 2024-03-18T20:52:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.705+0000 7fd30d653700 1 -- 172.21.15.46:0/3104955336 learned_addr learned my addr 172.21.15.46:0/3104955336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.705+0000 7fd30ce52700 1 --2- 172.21.15.46:0/3104955336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.706+0000 7fd30d653700 1 -- 172.21.15.46:0/3104955336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 msgr2=0x5595d21d9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.706+0000 7fd30d653700 1 --2- 172.21.15.46:0/3104955336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.706+0000 7fd30d653700 1 -- 172.21.15.46:0/3104955336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 msgr2=0x5595d224a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.706+0000 7fd30d653700 1 --2- 172.21.15.46:0/3104955336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 0x5595d224a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.706+0000 7fd30d653700 1 -- 172.21.15.46:0/3104955336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595d1f4cb00 con 0x5595d21cb570 2024-03-18T20:52:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.706+0000 7fd30d653700 1 --2- 172.21.15.46:0/3104955336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 0x5595d21d9a10 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fd304011fe0 tx=0x7fd304009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d565bc22c153b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.706+0000 7fd2f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:52:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.706+0000 7fd2f0ff9700 1 -- 172.21.15.46:0/3104955336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3040333f0 con 0x5595d21cb570 2024-03-18T20:52:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.706+0000 7fd2f0ff9700 1 -- 172.21.15.46:0/3104955336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd30402b040 con 0x5595d21cb570 2024-03-18T20:52:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.707+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3104955336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd30401d940 con 0x5595d21cb570 2024-03-18T20:52:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.709+0000 7fd2f0ff9700 1 -- 172.21.15.46:0/3104955336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd304033550 con 0x5595d21cb570 2024-03-18T20:52:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.709+0000 7fd31ba4cb80 10 client.20355 fetch_fsmap learned FSMap version 77 2024-03-18T20:52:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.709+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3104955336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x5595d1fa4c70 con 0x5595d21cb570 2024-03-18T20:52:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.709+0000 7fd2f0ff9700 1 -- 172.21.15.46:0/3104955336 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd30401d940 con 0x5595d21cb570 2024-03-18T20:52:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.709+0000 7fd2f0ff9700 1 -- 172.21.15.46:0/3104955336 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 77) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd304039020 con 0x5595d21cb570 2024-03-18T20:52:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.709+0000 7fd31ba4cb80 10 client.20355 fetch_fsmap finished waiting for FSMap version 77 2024-03-18T20:52:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.709+0000 7fd31ba4cb80 10 client.20355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:52:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.709+0000 7fd31ba4cb80 20 client.20355 populate_metadata read hostname 'smithi046' 2024-03-18T20:52:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.709+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/3104955336 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5595d224c0c0 0x5595d224e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.709+0000 7fd31ba4cb80 4 client.20355 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:52:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.709+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3104955336 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5595d1f6dc90 con 0x5595d224c0c0 2024-03-18T20:52:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.710+0000 7fd30ce52700 1 --2- 172.21.15.46:0/3104955336 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5595d224c0c0 0x5595d224e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.710+0000 7fd2f0ff9700 10 client.20355 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.710+0000 7fd30ce52700 1 --2- 172.21.15.46:0/3104955336 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5595d224c0c0 0x5595d224e4e0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7acec4b6ce70481d server_cookie=3f0d588b0e2bbecd in_seq=0 out_seq=0 2024-03-18T20:52:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.978+0000 7fd2f0ff9700 1 -- 172.21.15.46:0/3104955336 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5595d1f6dc90 con 0x5595d224c0c0 2024-03-18T20:52:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.978+0000 7fd2f0ff9700 10 client.20355 handle_command_reply: tid=0 2024-03-18T20:52:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.978+0000 7fd31ba4cb80 1 client.20355 shutdown 2024-03-18T20:52:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.979+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3104955336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 msgr2=0x5595d21d9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.979+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/3104955336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 0x5595d21d9a10 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fd304011fe0 tx=0x7fd304009040 comp rx=0 tx=0).stop 2024-03-18T20:52:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.979+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3104955336 shutdown_connections 2024-03-18T20:52:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.979+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/3104955336 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5595d224c0c0 0x5595d224e4e0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.979+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/3104955336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5595d21cb570 0x5595d21d9a10 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.979+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/3104955336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595d21d6db0 0x5595d224a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.979+0000 7fd31ba4cb80 1 --2- 172.21.15.46:0/3104955336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5595d21cbe90 0x5595d21d9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.979+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3104955336 >> 172.21.15.46:0/3104955336 conn(0x5595d21421c0 msgr2=0x5595d21d2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.979+0000 7fd2f0ff9700 0 client.20355 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.979+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3104955336 shutdown_connections 2024-03-18T20:52:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.979+0000 7fd31ba4cb80 1 -- 172.21.15.46:0/3104955336 wait complete. 2024-03-18T20:52:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.979+0000 7fd31ba4cb80 20 client.20355 trim_cache size 0 max 16384 2024-03-18T20:52:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:52:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:52:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:52:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:52:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.981+0000 7fd30f8b7700 1 -- 172.21.15.46:0/418480039 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2f40a13e0 msgr2=0x7fd2f40a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.981+0000 7fd30f8b7700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2f40a13e0 0x7fd2f40a38a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd304018000 tx=0x7fd30401a000 comp rx=0 tx=0).stop 2024-03-18T20:52:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.981+0000 7fd30f8b7700 1 -- 172.21.15.46:0/418480039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308068aa0 msgr2=0x7fd30811ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.981+0000 7fd30f8b7700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308068aa0 0x7fd30811ad00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8000c00 tx=0x7fd2f8005740 comp rx=0 tx=0).stop 2024-03-18T20:52:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.981+0000 7fd30f8b7700 1 -- 172.21.15.46:0/418480039 shutdown_connections 2024-03-18T20:52:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.981+0000 7fd30f8b7700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd2f40a13e0 0x7fd2f40a38a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.981+0000 7fd30f8b7700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3080680e0 0x7fd308114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.981+0000 7fd30f8b7700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd308106e10 0x7fd3081100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.981+0000 7fd30f8b7700 1 --2- 172.21.15.46:0/418480039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd308068aa0 0x7fd30811ad00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.981+0000 7fd30f8b7700 1 -- 172.21.15.46:0/418480039 >> 172.21.15.46:0/418480039 conn(0x7fd308076d50 msgr2=0x7fd3080fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.981+0000 7fd30f8b7700 1 -- 172.21.15.46:0/418480039 shutdown_connections 2024-03-18T20:52:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:19.981+0000 7fd30f8b7700 1 -- 172.21.15.46:0/418480039 wait complete. 2024-03-18T20:52:20.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 155109 43.80 MB/sec execute 56 sec latency 130.904 ms 2024-03-18T20:52:20.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:20 smithi073 ceph-mon[47384]: pgmap v510: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-03-18T20:52:20.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:20 smithi155 ceph-mon[48344]: pgmap v510: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-03-18T20:52:20.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:20 smithi046 ceph-mon[43495]: pgmap v510: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-03-18T20:52:21.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 156154 43.85 MB/sec execute 57 sec latency 63.060 ms 2024-03-18T20:52:22.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 157092 44.21 MB/sec execute 58 sec latency 40.743 ms 2024-03-18T20:52:22.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:22 smithi073 ceph-mon[47384]: pgmap v511: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 743 op/s 2024-03-18T20:52:22.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:22 smithi155 ceph-mon[48344]: pgmap v511: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 743 op/s 2024-03-18T20:52:22.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:22 smithi046 ceph-mon[43495]: pgmap v511: 97 pgs: 97 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 743 op/s 2024-03-18T20:52:23.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 157827 44.10 MB/sec execute 59 sec latency 40.218 ms 2024-03-18T20:52:24.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 158535 43.97 MB/sec execute 60 sec latency 265.230 ms 2024-03-18T20:52:24.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:24 smithi073 ceph-mon[47384]: pgmap v512: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 873 op/s 2024-03-18T20:52:24.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:24 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:52:24.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:24 smithi155 ceph-mon[48344]: pgmap v512: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 873 op/s 2024-03-18T20:52:24.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:24 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:52:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:24 smithi046 ceph-mon[43495]: pgmap v512: 97 pgs: 97 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 873 op/s 2024-03-18T20:52:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:24 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:52:25.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 158542 43.25 MB/sec execute 61 sec latency 1236.854 ms 2024-03-18T20:52:26.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 158810 42.80 MB/sec execute 62 sec latency 1346.220 ms 2024-03-18T20:52:26.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:26 smithi073 ceph-mon[47384]: pgmap v513: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-03-18T20:52:26.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:26 smithi155 ceph-mon[48344]: pgmap v513: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-03-18T20:52:26.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:26 smithi046 ceph-mon[43495]: pgmap v513: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-03-18T20:52:27.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 159334 42.51 MB/sec execute 63 sec latency 136.570 ms 2024-03-18T20:52:27.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:27 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:52:27.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:27 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:52:27.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:27 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:52:27.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:27 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:52:27.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:27 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:52:27.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:27 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:52:27.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:27 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:52:27.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:27 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:52:27.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:27 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:52:28.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 159881 42.30 MB/sec execute 64 sec latency 88.011 ms 2024-03-18T20:52:28.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:28 smithi073 ceph-mon[47384]: pgmap v514: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 15 MiB/s wr, 860 op/s 2024-03-18T20:52:28.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:28 smithi155 ceph-mon[48344]: pgmap v514: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 15 MiB/s wr, 860 op/s 2024-03-18T20:52:28.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:28 smithi046 ceph-mon[43495]: pgmap v514: 97 pgs: 97 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 15 MiB/s wr, 860 op/s 2024-03-18T20:52:29.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 160714 42.48 MB/sec execute 65 sec latency 52.260 ms 2024-03-18T20:52:30.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 161364 42.26 MB/sec execute 66 sec latency 114.107 ms 2024-03-18T20:52:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:30 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:52:30.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:30 smithi046 ceph-mon[43495]: pgmap v515: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 36 MiB/s wr, 1.64k op/s 2024-03-18T20:52:30.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:30 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:52:30.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:30 smithi073 ceph-mon[47384]: pgmap v515: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 36 MiB/s wr, 1.64k op/s 2024-03-18T20:52:30.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:30 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:52:30.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:30 smithi155 ceph-mon[48344]: pgmap v515: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 36 MiB/s wr, 1.64k op/s 2024-03-18T20:52:31.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 162350 42.34 MB/sec execute 67 sec latency 44.014 ms 2024-03-18T20:52:32.370 INFO:tasks.workunit.client.0.smithi046.stdout: 10 163510 42.70 MB/sec execute 68 sec latency 63.059 ms 2024-03-18T20:52:32.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:32 smithi073 ceph-mon[47384]: pgmap v516: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 28 MiB/s wr, 1.25k op/s 2024-03-18T20:52:32.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:32 smithi155 ceph-mon[48344]: pgmap v516: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 28 MiB/s wr, 1.25k op/s 2024-03-18T20:52:32.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:32 smithi046 ceph-mon[43495]: pgmap v516: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 28 MiB/s wr, 1.25k op/s 2024-03-18T20:52:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.157+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/4262164749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e4100100 msgr2=0x7fe3e41004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.157+0000 7fe3e9aaa700 1 --2- 172.21.15.46:0/4262164749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e4100100 0x7fe3e41004e0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009990 tx=0x7fe3d4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.157+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/4262164749 shutdown_connections 2024-03-18T20:52:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.157+0000 7fe3e9aaa700 1 --2- 172.21.15.46:0/4262164749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e4100100 0x7fe3e41004e0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.157+0000 7fe3e9aaa700 1 --2- 172.21.15.46:0/4262164749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3e4104f40 0x7fe3e4069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.157+0000 7fe3e9aaa700 1 --2- 172.21.15.46:0/4262164749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e4100ab0 0x7fe3e4104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.157+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/4262164749 >> 172.21.15.46:0/4262164749 conn(0x7fe3e4076a80 msgr2=0x7fe3e4076e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.157+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/4262164749 shutdown_connections 2024-03-18T20:52:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.157+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/4262164749 wait complete. 2024-03-18T20:52:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.158+0000 7fe3e9aaa700 1 Processor -- start 2024-03-18T20:52:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.158+0000 7fe3e9aaa700 1 -- start start 2024-03-18T20:52:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e9aaa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e4100100 0x7fe3e4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e9aaa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e4104f40 0x7fe3e4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e9aaa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3e419dce0 0x7fe3e419e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e9aaa700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e419e760 con 0x7fe3e4100100 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e9aaa700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e419e8d0 con 0x7fe3e419dce0 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e9aaa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e419ea90 con 0x7fe3e4104f40 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e4100100 0x7fe3e4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e4100100 0x7fe3e4199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35998/0 (socket says 172.21.15.46:35998) 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e37fe700 1 -- 172.21.15.46:0/2945521529 learned_addr learned my addr 172.21.15.46:0/2945521529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e4104f40 0x7fe3e4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e3fff700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3e419dce0 0x7fe3e419e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e37fe700 1 -- 172.21.15.46:0/2945521529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e4104f40 msgr2=0x7fe3e4199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e37fe700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e4104f40 0x7fe3e4199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e37fe700 1 -- 172.21.15.46:0/2945521529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3e419dce0 msgr2=0x7fe3e419e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e37fe700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3e419dce0 0x7fe3e419e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.159+0000 7fe3e37fe700 1 -- 172.21.15.46:0/2945521529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d40095f0 con 0x7fe3e4100100 2024-03-18T20:52:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.160+0000 7fe3e37fe700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e4100100 0x7fe3e4199400 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005910 tx=0x7fe3d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.160+0000 7fe3e0ff9700 1 -- 172.21.15.46:0/2945521529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d401d070 con 0x7fe3e4100100 2024-03-18T20:52:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.160+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/2945521529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e41a5f90 con 0x7fe3e4100100 2024-03-18T20:52:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.160+0000 7fe3e0ff9700 1 -- 172.21.15.46:0/2945521529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d4022470 con 0x7fe3e4100100 2024-03-18T20:52:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.160+0000 7fe3e0ff9700 1 -- 172.21.15.46:0/2945521529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4017650 con 0x7fe3e4100100 2024-03-18T20:52:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.160+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/2945521529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e41a63b0 con 0x7fe3e4100100 2024-03-18T20:52:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.163+0000 7fe3e0ff9700 1 -- 172.21.15.46:0/2945521529 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe3d40225e0 con 0x7fe3e4100100 2024-03-18T20:52:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.164+0000 7fe3e0ff9700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe3cc0a1290 0x7fe3cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.164+0000 7fe3e0ff9700 1 -- 172.21.15.46:0/2945521529 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe3d40d01e0 con 0x7fe3e4100100 2024-03-18T20:52:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.165+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe3cc0a1290 0x7fe3cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.167+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe3cc0a1290 0x7fe3cc0a3750 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe3e419a950 tx=0x7fe3d8009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.168+0000 7fe3f5c3fb80 1 Processor -- start 2024-03-18T20:52:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3f5c3fb80 1 -- start start 2024-03-18T20:52:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3f5c3fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24d3570 0x55e8a24d3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3f5c3fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 0x55e8a24de870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3f5c3fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24dedb0 0x55e8a24e11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3f5c3fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a242d930 con 0x55e8a24dedb0 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3f5c3fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a242b2f0 con 0x55e8a24d3570 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3f5c3fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a2426be0 con 0x55e8a24d3e90 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3e2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 0x55e8a24de870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24d3570 0x55e8a24d3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3e2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 0x55e8a24de870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:36564/0 (socket says 172.21.15.46:36564) 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3e2ffd700 1 -- 172.21.15.46:0/3476143345 learned_addr learned my addr 172.21.15.46:0/3476143345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3e3fff700 1 --2- 172.21.15.46:0/3476143345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24dedb0 0x55e8a24e11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3e2ffd700 1 -- 172.21.15.46:0/3476143345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24d3570 msgr2=0x55e8a24d3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/3476143345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24d3570 0x55e8a24d3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3e2ffd700 1 -- 172.21.15.46:0/3476143345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24dedb0 msgr2=0x55e8a24e11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/3476143345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24dedb0 0x55e8a24e11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3e2ffd700 1 -- 172.21.15.46:0/3476143345 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8a22a8350 con 0x55e8a24d3e90 2024-03-18T20:52:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.169+0000 7fe3e3fff700 1 --2- 172.21.15.46:0/3476143345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24dedb0 0x55e8a24e11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:52:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.170+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/3476143345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 0x55e8a24de870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8011cf0 tx=0x7fe3d8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=825014072567b03c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.170+0000 7fe3c9ffb700 1 -- 172.21.15.46:0/3476143345 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800ea40 con 0x55e8a24d3e90 2024-03-18T20:52:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.170+0000 7fe3c9ffb700 1 -- 172.21.15.46:0/3476143345 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d8021070 con 0x55e8a24d3e90 2024-03-18T20:52:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.170+0000 7fe3c9ffb700 1 -- 172.21.15.46:0/3476143345 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d801d410 con 0x55e8a24d3e90 2024-03-18T20:52:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.171+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/3476143345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 msgr2=0x55e8a24de870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.171+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/3476143345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 0x55e8a24de870 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8011cf0 tx=0x7fe3d8014000 comp rx=0 tx=0).stop 2024-03-18T20:52:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.171+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/3476143345 shutdown_connections 2024-03-18T20:52:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.171+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/3476143345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24dedb0 0x55e8a24e11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.171+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/3476143345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24d3570 0x55e8a24d3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.171+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/3476143345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 0x55e8a24de870 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.171+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/3476143345 >> 172.21.15.46:0/3476143345 conn(0x55e8a244a1c0 msgr2=0x55e8a24cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.172+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/3476143345 shutdown_connections 2024-03-18T20:52:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.172+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/3476143345 wait complete. 2024-03-18T20:52:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.172+0000 7fe3f5c3fb80 1 -- start start 2024-03-18T20:52:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.172+0000 7fe3f5c3fb80 1 Processor -- start 2024-03-18T20:52:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3f5c3fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3570 0x55e8a24de7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3f5c3fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 0x55e8a24dbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3f5c3fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24dedb0 0x55e8a24dc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3f5c3fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a23c2d20 con 0x55e8a24d3570 2024-03-18T20:52:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3f5c3fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a23c29c0 con 0x55e8a24dedb0 2024-03-18T20:52:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3f5c3fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a22aa640 con 0x55e8a24d3e90 2024-03-18T20:52:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3f5c3fb80 10 client.0 _ll_register_callbacks cb 0x55e8a222e340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:52:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3e2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 0x55e8a24dbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3570 0x55e8a24de7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3570 0x55e8a24de7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36022/0 (socket says 172.21.15.46:36022) 2024-03-18T20:52:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3e2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 0x55e8a24dbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:36574/0 (socket says 172.21.15.46:36574) 2024-03-18T20:52:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3e37fe700 1 -- 172.21.15.46:0/4140411994 learned_addr learned my addr 172.21.15.46:0/4140411994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.173+0000 7fe3e3fff700 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24dedb0 0x55e8a24dc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.174+0000 7fe3e37fe700 1 -- 172.21.15.46:0/4140411994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 msgr2=0x55e8a24dbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.174+0000 7fe3e37fe700 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 0x55e8a24dbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.174+0000 7fe3e37fe700 1 -- 172.21.15.46:0/4140411994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24dedb0 msgr2=0x55e8a24dc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.174+0000 7fe3e37fe700 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24dedb0 0x55e8a24dc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.174+0000 7fe3e37fe700 1 -- 172.21.15.46:0/4140411994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d8017070 con 0x55e8a24d3570 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.174+0000 7fe3e3fff700 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24dedb0 0x55e8a24dc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.174+0000 7fe3e37fe700 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3570 0x55e8a24de7d0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fe3d409ae70 tx=0x7fe3d409aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62802edb91098b3b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.174+0000 7fe3c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.174+0000 7fe3c2ffd700 1 -- 172.21.15.46:0/4140411994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d409b5e0 con 0x55e8a24d3570 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.174+0000 7fe3c2ffd700 1 -- 172.21.15.46:0/4140411994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe3d409aee0 con 0x55e8a24d3570 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.174+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/4140411994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8a24c76c0 con 0x55e8a24d3570 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.175+0000 7fe3c2ffd700 1 -- 172.21.15.46:0/4140411994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3d409ed10 con 0x55e8a24d3570 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.175+0000 7fe3f5c3fb80 10 client.20367 fetch_fsmap learned FSMap version 78 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.175+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/4140411994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55e8a2254b00 con 0x55e8a24d3570 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.175+0000 7fe3c2ffd700 1 -- 172.21.15.46:0/4140411994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d4064080 con 0x55e8a24d3570 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.175+0000 7fe3c2ffd700 1 -- 172.21.15.46:0/4140411994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe3d40aed20 con 0x55e8a24d3570 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.175+0000 7fe3f5c3fb80 10 client.20367 fetch_fsmap finished waiting for FSMap version 78 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.175+0000 7fe3f5c3fb80 10 client.20367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:52:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.175+0000 7fe3f5c3fb80 20 client.20367 populate_metadata read hostname 'smithi046' 2024-03-18T20:52:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.175+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8a2507d30 0x55e8a250a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.175+0000 7fe3f5c3fb80 4 client.20367 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:52:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.175+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/4140411994 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8a241f6a0 con 0x55e8a2507d30 2024-03-18T20:52:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.176+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8a2507d30 0x55e8a250a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.176+0000 7fe3c2ffd700 10 client.20367 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.176+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8a2507d30 0x55e8a250a120 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a68180fa668ea6e server_cookie=358f7b63679e4f80 in_seq=0 out_seq=0 2024-03-18T20:52:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.178+0000 7fe3c2ffd700 1 -- 172.21.15.46:0/4140411994 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e8a241f6a0 con 0x55e8a2507d30 2024-03-18T20:52:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.178+0000 7fe3c2ffd700 10 client.20367 handle_command_reply: tid=0 2024-03-18T20:52:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.178+0000 7fe3f5c3fb80 1 client.20367 shutdown 2024-03-18T20:52:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.179+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/4140411994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3570 msgr2=0x55e8a24de7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.179+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3570 0x55e8a24de7d0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fe3d409ae70 tx=0x7fe3d409aa20 comp rx=0 tx=0).stop 2024-03-18T20:52:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.179+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/4140411994 shutdown_connections 2024-03-18T20:52:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.179+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8a2507d30 0x55e8a250a120 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.179+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3570 0x55e8a24de7d0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.179+0000 7fe3c2ffd700 0 client.20367 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.179+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24dedb0 0x55e8a24dc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.179+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/4140411994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3e90 0x55e8a24dbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.179+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/4140411994 >> 172.21.15.46:0/4140411994 conn(0x55e8a244a1c0 msgr2=0x55e8a24cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.179+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/4140411994 shutdown_connections 2024-03-18T20:52:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.179+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/4140411994 wait complete. 2024-03-18T20:52:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.179+0000 7fe3f5c3fb80 20 client.20367 trim_cache size 0 max 16384 2024-03-18T20:52:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.200+0000 7fe3f5c3fb80 1 Processor -- start 2024-03-18T20:52:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.200+0000 7fe3f5c3fb80 1 -- start start 2024-03-18T20:52:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3f5c3fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24d3570 0x55e8a24d65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3f5c3fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3e90 0x55e8a24ee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3f5c3fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24dedb0 0x55e8a24de360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3f5c3fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a2504d00 con 0x55e8a24d3e90 2024-03-18T20:52:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3f5c3fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a24d4e50 con 0x55e8a24d3570 2024-03-18T20:52:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3f5c3fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a24c9c90 con 0x55e8a24dedb0 2024-03-18T20:52:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24d3570 0x55e8a24d65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24d3570 0x55e8a24d65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:60224/0 (socket says 172.21.15.46:60224) 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3e37fe700 1 -- 172.21.15.46:0/3702538188 learned_addr learned my addr 172.21.15.46:0/3702538188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/3702538188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3e90 0x55e8a24ee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3e3fff700 1 --2- 172.21.15.46:0/3702538188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24dedb0 0x55e8a24de360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3e37fe700 1 -- 172.21.15.46:0/3702538188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24dedb0 msgr2=0x55e8a24de360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3e37fe700 1 --2- 172.21.15.46:0/3702538188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24dedb0 0x55e8a24de360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3e37fe700 1 -- 172.21.15.46:0/3702538188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3e90 msgr2=0x55e8a24ee2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3e37fe700 1 --2- 172.21.15.46:0/3702538188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3e90 0x55e8a24ee2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3e37fe700 1 -- 172.21.15.46:0/3702538188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8a23ece40 con 0x55e8a24d3570 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3e3fff700 1 --2- 172.21.15.46:0/3702538188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24dedb0 0x55e8a24de360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.201+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/3702538188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3e90 0x55e8a24ee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.202+0000 7fe3e37fe700 1 --2- 172.21.15.46:0/3702538188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24d3570 0x55e8a24d65e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe3d409abe0 tx=0x7fe3d40cbf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12d2fa0e66f208a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.202+0000 7fe3ca7fc700 1 -- 172.21.15.46:0/3702538188 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d409b5e0 con 0x55e8a24d3570 2024-03-18T20:52:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.202+0000 7fe3ca7fc700 1 -- 172.21.15.46:0/3702538188 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e8a24c76c0 con 0x55e8a24d3570 2024-03-18T20:52:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.202+0000 7fe3ca7fc700 1 -- 172.21.15.46:0/3702538188 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40a32d0 con 0x55e8a24d3570 2024-03-18T20:52:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.203+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/3702538188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24d3570 msgr2=0x55e8a24d65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.203+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/3702538188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24d3570 0x55e8a24d65e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe3d409abe0 tx=0x7fe3d40cbf90 comp rx=0 tx=0).stop 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.203+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/3702538188 shutdown_connections 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.203+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/3702538188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24dedb0 0x55e8a24de360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.203+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/3702538188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24d3570 0x55e8a24d65e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.203+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/3702538188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3e90 0x55e8a24ee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.203+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/3702538188 >> 172.21.15.46:0/3702538188 conn(0x55e8a244a1c0 msgr2=0x55e8a24ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.203+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/3702538188 shutdown_connections 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.203+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/3702538188 wait complete. 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.204+0000 7fe3f5c3fb80 1 -- start start 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.204+0000 7fe3f5c3fb80 1 Processor -- start 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.204+0000 7fe3f5c3fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3570 0x55e8a24ec2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.204+0000 7fe3f5c3fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3e90 0x55e8a24ec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.204+0000 7fe3f5c3fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24dedb0 0x55e8a255ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.204+0000 7fe3f5c3fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a24ecf80 con 0x55e8a24d3e90 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.204+0000 7fe3f5c3fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a24ed0f0 con 0x55e8a24dedb0 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.204+0000 7fe3f5c3fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8a255b040 con 0x55e8a24d3570 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.204+0000 7fe3f5c3fb80 10 client.0 _ll_register_callbacks cb 0x7fe3d409acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.205+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3570 0x55e8a24ec2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.205+0000 7fe3e37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3570 0x55e8a24ec2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:36588/0 (socket says 172.21.15.46:36588) 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.205+0000 7fe3e37fe700 1 -- 172.21.15.46:0/844969013 learned_addr learned my addr 172.21.15.46:0/844969013 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.205+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3e90 0x55e8a24ec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.205+0000 7fe3e3fff700 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24dedb0 0x55e8a255ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.205+0000 7fe3e37fe700 1 -- 172.21.15.46:0/844969013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24dedb0 msgr2=0x55e8a255ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.205+0000 7fe3e37fe700 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24dedb0 0x55e8a255ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.205+0000 7fe3e37fe700 1 -- 172.21.15.46:0/844969013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3e90 msgr2=0x55e8a24ec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.205+0000 7fe3e37fe700 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3e90 0x55e8a24ec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.205+0000 7fe3e37fe700 1 -- 172.21.15.46:0/844969013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d802a040 con 0x55e8a24d3570 2024-03-18T20:52:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.205+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3e90 0x55e8a24ec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:52:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.208+0000 7fe3e37fe700 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3570 0x55e8a24ec2b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe3d409ae70 tx=0x7fe3d400b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=223d4a7ac2d4dd54 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.208+0000 7fe3c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:52:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.208+0000 7fe3c2ffd700 1 -- 172.21.15.46:0/844969013 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40cbe30 con 0x55e8a24d3570 2024-03-18T20:52:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.208+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/844969013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3d8021070 con 0x55e8a24d3570 2024-03-18T20:52:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.208+0000 7fe3c2ffd700 1 -- 172.21.15.46:0/844969013 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe3d409b5e0 con 0x55e8a24d3570 2024-03-18T20:52:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.208+0000 7fe3c2ffd700 1 -- 172.21.15.46:0/844969013 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3d409aa20 con 0x55e8a24d3570 2024-03-18T20:52:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.208+0000 7fe3f5c3fb80 10 client.30043 fetch_fsmap learned FSMap version 78 2024-03-18T20:52:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.208+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/844969013 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55e8a2254b00 con 0x55e8a24d3570 2024-03-18T20:52:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.209+0000 7fe3c2ffd700 1 -- 172.21.15.46:0/844969013 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3d8021070 con 0x55e8a24d3570 2024-03-18T20:52:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.209+0000 7fe3c2ffd700 1 -- 172.21.15.46:0/844969013 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe3d40aed20 con 0x55e8a24d3570 2024-03-18T20:52:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.209+0000 7fe3f5c3fb80 10 client.30043 fetch_fsmap finished waiting for FSMap version 78 2024-03-18T20:52:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.209+0000 7fe3f5c3fb80 10 client.30043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:52:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.209+0000 7fe3f5c3fb80 20 client.30043 populate_metadata read hostname 'smithi046' 2024-03-18T20:52:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.209+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8a255c340 0x55e8a255e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.209+0000 7fe3f5c3fb80 4 client.30043 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:52:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.209+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/844969013 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e8a2275c90 con 0x55e8a255c340 2024-03-18T20:52:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.209+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8a255c340 0x55e8a255e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.209+0000 7fe3c2ffd700 10 client.30043 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.209+0000 7fe3e2ffd700 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8a255c340 0x55e8a255e730 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c95fb14e2d678998 server_cookie=9ad7162d52881c06 in_seq=0 out_seq=0 2024-03-18T20:52:33.371 INFO:tasks.workunit.client.0.smithi046.stdout: 10 164374 42.89 MB/sec execute 69 sec latency 150.588 ms 2024-03-18T20:52:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.399+0000 7fe3c2ffd700 1 -- 172.21.15.46:0/844969013 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e8a2275c90 con 0x55e8a255c340 2024-03-18T20:52:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.399+0000 7fe3c2ffd700 10 client.30043 handle_command_reply: tid=0 2024-03-18T20:52:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3f5c3fb80 1 client.30043 shutdown 2024-03-18T20:52:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/844969013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3570 msgr2=0x55e8a24ec2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3570 0x55e8a24ec2b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe3d409ae70 tx=0x7fe3d400b610 comp rx=0 tx=0).stop 2024-03-18T20:52:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/844969013 shutdown_connections 2024-03-18T20:52:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8a255c340 0x55e8a255e730 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8a24dedb0 0x55e8a255ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8a24d3570 0x55e8a24ec2b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3f5c3fb80 1 --2- 172.21.15.46:0/844969013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8a24d3e90 0x55e8a24ec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3c2ffd700 0 client.30043 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/844969013 >> 172.21.15.46:0/844969013 conn(0x55e8a244a1c0 msgr2=0x55e8a24d6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/844969013 shutdown_connections 2024-03-18T20:52:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3f5c3fb80 1 -- 172.21.15.46:0/844969013 wait complete. 2024-03-18T20:52:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.400+0000 7fe3f5c3fb80 20 client.30043 trim_cache size 0 max 16384 2024-03-18T20:52:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:52:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:52:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:52:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:52:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.401+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/2945521529 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe3cc0a1290 msgr2=0x7fe3cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.401+0000 7fe3e9aaa700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe3cc0a1290 0x7fe3cc0a3750 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe3e419a950 tx=0x7fe3d8009260 comp rx=0 tx=0).stop 2024-03-18T20:52:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.401+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/2945521529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e4100100 msgr2=0x7fe3e4199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.401+0000 7fe3e9aaa700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e4100100 0x7fe3e4199400 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005910 tx=0x7fe3d4004730 comp rx=0 tx=0).stop 2024-03-18T20:52:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.401+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/2945521529 shutdown_connections 2024-03-18T20:52:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.401+0000 7fe3e9aaa700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe3cc0a1290 0x7fe3cc0a3750 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.401+0000 7fe3e9aaa700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3e4100100 0x7fe3e4199400 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.401+0000 7fe3e9aaa700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3e419dce0 0x7fe3e419e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.401+0000 7fe3e9aaa700 1 --2- 172.21.15.46:0/2945521529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3e4104f40 0x7fe3e4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.401+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/2945521529 >> 172.21.15.46:0/2945521529 conn(0x7fe3e4076a80 msgr2=0x7fe3e40fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.402+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/2945521529 shutdown_connections 2024-03-18T20:52:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:33.403+0000 7fe3e9aaa700 1 -- 172.21.15.46:0/2945521529 wait complete. 2024-03-18T20:52:34.371 INFO:tasks.workunit.client.0.smithi046.stdout: 10 165134 42.81 MB/sec execute 70 sec latency 56.637 ms 2024-03-18T20:52:34.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:34 smithi073 ceph-mon[47384]: pgmap v517: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 30 MiB/s wr, 1.35k op/s 2024-03-18T20:52:34.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:34 smithi155 ceph-mon[48344]: pgmap v517: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 30 MiB/s wr, 1.35k op/s 2024-03-18T20:52:34.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:34 smithi046 ceph-mon[43495]: pgmap v517: 97 pgs: 97 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 30 MiB/s wr, 1.35k op/s 2024-03-18T20:52:35.371 INFO:tasks.workunit.client.0.smithi046.stdout: 10 165802 42.65 MB/sec execute 71 sec latency 183.764 ms 2024-03-18T20:52:36.371 INFO:tasks.workunit.client.0.smithi046.stdout: 10 166803 42.74 MB/sec execute 72 sec latency 54.153 ms 2024-03-18T20:52:36.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:36 smithi073 ceph-mon[47384]: pgmap v518: 97 pgs: 97 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 29 MiB/s wr, 1.41k op/s 2024-03-18T20:52:36.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:36 smithi155 ceph-mon[48344]: pgmap v518: 97 pgs: 97 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 29 MiB/s wr, 1.41k op/s 2024-03-18T20:52:36.607 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:36 smithi046 ceph-mon[43495]: pgmap v518: 97 pgs: 97 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 29 MiB/s wr, 1.41k op/s 2024-03-18T20:52:37.371 INFO:tasks.workunit.client.0.smithi046.stdout: 10 167803 43.01 MB/sec execute 73 sec latency 32.744 ms 2024-03-18T20:52:38.371 INFO:tasks.workunit.client.0.smithi046.stdout: 10 168618 42.99 MB/sec execute 74 sec latency 51.568 ms 2024-03-18T20:52:38.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:38 smithi073 ceph-mon[47384]: pgmap v519: 97 pgs: 97 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 25 MiB/s wr, 1.12k op/s 2024-03-18T20:52:38.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:38 smithi155 ceph-mon[48344]: pgmap v519: 97 pgs: 97 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 25 MiB/s wr, 1.12k op/s 2024-03-18T20:52:38.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:38 smithi046 ceph-mon[43495]: pgmap v519: 97 pgs: 97 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 25 MiB/s wr, 1.12k op/s 2024-03-18T20:52:39.371 INFO:tasks.workunit.client.0.smithi046.stdout: 10 169525 43.05 MB/sec execute 75 sec latency 61.742 ms 2024-03-18T20:52:40.371 INFO:tasks.workunit.client.0.smithi046.stdout: 10 170324 43.10 MB/sec execute 76 sec latency 106.874 ms 2024-03-18T20:52:40.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:40 smithi073 ceph-mon[47384]: pgmap v520: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 1.46k op/s 2024-03-18T20:52:40.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:40 smithi155 ceph-mon[48344]: pgmap v520: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 1.46k op/s 2024-03-18T20:52:40.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:40 smithi046 ceph-mon[43495]: pgmap v520: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 29 MiB/s wr, 1.46k op/s 2024-03-18T20:52:41.371 INFO:tasks.workunit.client.0.smithi046.stdout: 10 171255 43.11 MB/sec execute 77 sec latency 47.506 ms 2024-03-18T20:52:42.371 INFO:tasks.workunit.client.0.smithi046.stdout: 10 172211 43.24 MB/sec execute 78 sec latency 41.579 ms 2024-03-18T20:52:42.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:42 smithi073 ceph-mon[47384]: pgmap v521: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.4 MiB/s wr, 680 op/s 2024-03-18T20:52:42.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:42 smithi155 ceph-mon[48344]: pgmap v521: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.4 MiB/s wr, 680 op/s 2024-03-18T20:52:42.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:42 smithi046 ceph-mon[43495]: pgmap v521: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.4 MiB/s wr, 680 op/s 2024-03-18T20:52:43.372 INFO:tasks.workunit.client.0.smithi046.stdout: 10 173264 43.53 MB/sec execute 79 sec latency 51.345 ms 2024-03-18T20:52:44.371 INFO:tasks.workunit.client.0.smithi046.stdout: 10 173975 43.40 MB/sec execute 80 sec latency 69.998 ms 2024-03-18T20:52:44.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:44 smithi073 ceph-mon[47384]: pgmap v522: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.9 MiB/s wr, 785 op/s 2024-03-18T20:52:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:44 smithi155 ceph-mon[48344]: pgmap v522: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.9 MiB/s wr, 785 op/s 2024-03-18T20:52:44.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:44 smithi046 ceph-mon[43495]: pgmap v522: 97 pgs: 97 active+clean; 1.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.9 MiB/s wr, 785 op/s 2024-03-18T20:52:45.372 INFO:tasks.workunit.client.0.smithi046.stdout: 10 174868 43.41 MB/sec execute 81 sec latency 41.584 ms 2024-03-18T20:52:46.372 INFO:tasks.workunit.client.0.smithi046.stdout: 10 175932 43.55 MB/sec execute 82 sec latency 45.007 ms 2024-03-18T20:52:46.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:46 smithi073 ceph-mon[47384]: pgmap v523: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 873 op/s 2024-03-18T20:52:46.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:46 smithi155 ceph-mon[48344]: pgmap v523: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 873 op/s 2024-03-18T20:52:46.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:46 smithi046 ceph-mon[43495]: pgmap v523: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 873 op/s 2024-03-18T20:52:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.554+0000 7ff19c4de700 1 -- 172.21.15.46:0/2141684669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1940ff220 msgr2=0x7ff1940ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.554+0000 7ff19c4de700 1 --2- 172.21.15.46:0/2141684669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1940ff220 0x7ff1940ff6a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7ff1840099c0 tx=0x7ff184009cd0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.554+0000 7ff19c4de700 1 -- 172.21.15.46:0/2141684669 shutdown_connections 2024-03-18T20:52:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.554+0000 7ff19c4de700 1 --2- 172.21.15.46:0/2141684669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1940ffbe0 0x7ff19410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.554+0000 7ff19c4de700 1 --2- 172.21.15.46:0/2141684669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff194106e30 0x7ff194107210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.554+0000 7ff19c4de700 1 --2- 172.21.15.46:0/2141684669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1940ff220 0x7ff1940ff6a0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.554+0000 7ff19c4de700 1 -- 172.21.15.46:0/2141684669 >> 172.21.15.46:0/2141684669 conn(0x7ff1940761b0 msgr2=0x7ff1940fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.554+0000 7ff19c4de700 1 -- 172.21.15.46:0/2141684669 shutdown_connections 2024-03-18T20:52:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.554+0000 7ff19c4de700 1 -- 172.21.15.46:0/2141684669 wait complete. 2024-03-18T20:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff19c4de700 1 Processor -- start 2024-03-18T20:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff19c4de700 1 -- start start 2024-03-18T20:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff19c4de700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1940ff220 0x7ff1941a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff19c4de700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1940ffbe0 0x7ff1941a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff19c4de700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff194106e30 0x7ff19410cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff19c4de700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19410d380 con 0x7ff1940ff220 2024-03-18T20:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff19c4de700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19410d4f0 con 0x7ff1940ffbe0 2024-03-18T20:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff19c4de700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19410d6b0 con 0x7ff194106e30 2024-03-18T20:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff199a79700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1940ffbe0 0x7ff1941a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff19a27a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1940ff220 0x7ff1941a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff199a79700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1940ffbe0 0x7ff1941a3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51182/0 (socket says 172.21.15.46:51182) 2024-03-18T20:52:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff199a79700 1 -- 172.21.15.46:0/55465770 learned_addr learned my addr 172.21.15.46:0/55465770 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.555+0000 7ff19aa7b700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff194106e30 0x7ff19410cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.556+0000 7ff19a27a700 1 -- 172.21.15.46:0/55465770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff194106e30 msgr2=0x7ff19410cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.556+0000 7ff19a27a700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff194106e30 0x7ff19410cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.556+0000 7ff19a27a700 1 -- 172.21.15.46:0/55465770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1940ffbe0 msgr2=0x7ff1941a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.556+0000 7ff19a27a700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1940ffbe0 0x7ff1941a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.556+0000 7ff19a27a700 1 -- 172.21.15.46:0/55465770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1840095f0 con 0x7ff1940ff220 2024-03-18T20:52:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.556+0000 7ff19a27a700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1940ff220 0x7ff1941a3580 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7ff19000e9f0 tx=0x7ff19000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.556+0000 7ff18b7fe700 1 -- 172.21.15.46:0/55465770 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000c9c0 con 0x7ff1940ff220 2024-03-18T20:52:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.556+0000 7ff18b7fe700 1 -- 172.21.15.46:0/55465770 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff190010430 con 0x7ff1940ff220 2024-03-18T20:52:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.556+0000 7ff18b7fe700 1 -- 172.21.15.46:0/55465770 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190018680 con 0x7ff1940ff220 2024-03-18T20:52:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.557+0000 7ff19c4de700 1 -- 172.21.15.46:0/55465770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19410d990 con 0x7ff1940ff220 2024-03-18T20:52:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.557+0000 7ff19c4de700 1 -- 172.21.15.46:0/55465770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1941aa500 con 0x7ff1940ff220 2024-03-18T20:52:46.564 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.562+0000 7ff1a8673b80 1 Processor -- start 2024-03-18T20:52:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.562+0000 7ff18b7fe700 1 -- 172.21.15.46:0/55465770 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff190010860 con 0x7ff1940ff220 2024-03-18T20:52:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.562+0000 7ff1a8673b80 1 -- start start 2024-03-18T20:52:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff1a8673b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102ea570 0x55fc102ea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff1a8673b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102eae90 0x55fc102f5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff1a8673b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102f5db0 0x55fc102f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff1a8673b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc10244930 con 0x55fc102eae90 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff1a8673b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc102422f0 con 0x55fc102f5db0 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff1a8673b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1023dbe0 con 0x55fc102ea570 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff19aa7b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102ea570 0x55fc102ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff19aa7b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102ea570 0x55fc102ea950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:41734/0 (socket says 172.21.15.46:41734) 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff19aa7b700 1 -- 172.21.15.46:0/2574314944 learned_addr learned my addr 172.21.15.46:0/2574314944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff19a27a700 1 --2- 172.21.15.46:0/2574314944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102eae90 0x55fc102f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff19aa7b700 1 -- 172.21.15.46:0/2574314944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102f5db0 msgr2=0x55fc102f81a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff19aa7b700 1 --2- 172.21.15.46:0/2574314944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102f5db0 0x55fc102f81a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff19aa7b700 1 -- 172.21.15.46:0/2574314944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102eae90 msgr2=0x55fc102f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff19aa7b700 1 --2- 172.21.15.46:0/2574314944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102eae90 0x55fc102f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff19aa7b700 1 -- 172.21.15.46:0/2574314944 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc100bf350 con 0x55fc102ea570 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.563+0000 7ff19aa7b700 1 --2- 172.21.15.46:0/2574314944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102ea570 0x55fc102ea950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff18c0115f0 tx=0x7ff18c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56d4069318a87f27 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.564+0000 7ff188ff9700 1 -- 172.21.15.46:0/2574314944 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c00e040 con 0x55fc102ea570 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.564+0000 7ff188ff9700 1 -- 172.21.15.46:0/2574314944 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff18c00f040 con 0x55fc102ea570 2024-03-18T20:52:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.564+0000 7ff188ff9700 1 -- 172.21.15.46:0/2574314944 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c0048e0 con 0x55fc102ea570 2024-03-18T20:52:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.564+0000 7ff18b7fe700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff1800a1310 0x7ff1800a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.564+0000 7ff19a27a700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff1800a1310 0x7ff1800a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.564+0000 7ff18b7fe700 1 -- 172.21.15.46:0/55465770 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff190014070 con 0x7ff1940ff220 2024-03-18T20:52:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.564+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2574314944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102ea570 msgr2=0x55fc102ea950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.564+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/2574314944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102ea570 0x55fc102ea950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff18c0115f0 tx=0x7ff18c009920 comp rx=0 tx=0).stop 2024-03-18T20:52:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.565+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2574314944 shutdown_connections 2024-03-18T20:52:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.565+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/2574314944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102f5db0 0x55fc102f81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.565+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/2574314944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102ea570 0x55fc102ea950 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.565+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/2574314944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102eae90 0x55fc102f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.565+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2574314944 >> 172.21.15.46:0/2574314944 conn(0x55fc102611c0 msgr2=0x55fc102e6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.565+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2574314944 shutdown_connections 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.565+0000 7ff19a27a700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff1800a1310 0x7ff1800a37d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff1900ca6b0 tx=0x7ff190010c60 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.565+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2574314944 wait complete. 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.565+0000 7ff1a8673b80 1 -- start start 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.565+0000 7ff1a8673b80 1 Processor -- start 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff1a8673b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 0x55fc102f5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff1a8673b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102eae90 0x55fc102f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff1a8673b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102f5db0 0x55fc102f3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff1a8673b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc101d9d20 con 0x55fc102eae90 2024-03-18T20:52:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff1a8673b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc101d99c0 con 0x55fc102ea570 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff1a8673b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc100c1640 con 0x55fc102f5db0 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff1a8673b80 10 client.0 _ll_register_callbacks cb 0x55fc10045340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff19aa7b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 0x55fc102f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff19aa7b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102f5db0 0x55fc102f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff19aa7b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 0x55fc102f5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51208/0 (socket says 172.21.15.46:51208) 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff19aa7b700 1 -- 172.21.15.46:0/1439216991 learned_addr learned my addr 172.21.15.46:0/1439216991 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff199a79700 1 --2- 172.21.15.46:0/1439216991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102eae90 0x55fc102f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff19aa7b700 1 -- 172.21.15.46:0/1439216991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 msgr2=0x55fc102f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff19aa7b700 1 --2- 172.21.15.46:0/1439216991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 0x55fc102f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff19aa7b700 1 -- 172.21.15.46:0/1439216991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102eae90 msgr2=0x55fc102f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.566+0000 7ff19aa7b700 1 --2- 172.21.15.46:0/1439216991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102eae90 0x55fc102f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.567+0000 7ff19aa7b700 1 -- 172.21.15.46:0/1439216991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18c0112d0 con 0x55fc102f5db0 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.567+0000 7ff19aa7b700 1 --2- 172.21.15.46:0/1439216991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102f5db0 0x55fc102f3050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff18c011c90 tx=0x7ff18c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f1d7efe3683630c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.567+0000 7ff17dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:52:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.567+0000 7ff17dffb700 1 -- 172.21.15.46:0/1439216991 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c003e40 con 0x55fc102f5db0 2024-03-18T20:52:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.567+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1439216991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc102de6c0 con 0x55fc102f5db0 2024-03-18T20:52:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.567+0000 7ff17dffb700 1 -- 172.21.15.46:0/1439216991 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff18c007500 con 0x55fc102f5db0 2024-03-18T20:52:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.567+0000 7ff17dffb700 1 -- 172.21.15.46:0/1439216991 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff18c0076c0 con 0x55fc102f5db0 2024-03-18T20:52:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.567+0000 7ff1a8673b80 10 client.30055 fetch_fsmap learned FSMap version 78 2024-03-18T20:52:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.567+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1439216991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55fc1006bb00 con 0x55fc102f5db0 2024-03-18T20:52:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.568+0000 7ff17dffb700 1 -- 172.21.15.46:0/1439216991 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc102de6c0 con 0x55fc102f5db0 2024-03-18T20:52:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.568+0000 7ff17dffb700 1 -- 172.21.15.46:0/1439216991 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff18c015890 con 0x55fc102f5db0 2024-03-18T20:52:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.568+0000 7ff1a8673b80 10 client.30055 fetch_fsmap finished waiting for FSMap version 78 2024-03-18T20:52:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.568+0000 7ff1a8673b80 10 client.30055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:52:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.568+0000 7ff1a8673b80 20 client.30055 populate_metadata read hostname 'smithi046' 2024-03-18T20:52:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.568+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/1439216991 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fc10316890 0x55fc10318cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.568+0000 7ff1a8673b80 4 client.30055 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:52:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.568+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1439216991 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc102366a0 con 0x55fc10316890 2024-03-18T20:52:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.568+0000 7ff199a79700 1 --2- 172.21.15.46:0/1439216991 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fc10316890 0x55fc10318cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.569+0000 7ff199a79700 1 --2- 172.21.15.46:0/1439216991 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fc10316890 0x55fc10318cb0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37428a3985f81d1a server_cookie=f5413cc046f5deda in_seq=0 out_seq=0 2024-03-18T20:52:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.569+0000 7ff17dffb700 10 client.30055 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.573+0000 7ff17dffb700 1 -- 172.21.15.46:0/1439216991 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fc102366a0 con 0x55fc10316890 2024-03-18T20:52:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.573+0000 7ff17dffb700 10 client.30055 handle_command_reply: tid=0 2024-03-18T20:52:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.573+0000 7ff1a8673b80 1 client.30055 shutdown 2024-03-18T20:52:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.577+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1439216991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102f5db0 msgr2=0x55fc102f3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:46.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.577+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/1439216991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102f5db0 0x55fc102f3050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff18c011c90 tx=0x7ff18c00b040 comp rx=0 tx=0).stop 2024-03-18T20:52:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.578+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1439216991 shutdown_connections 2024-03-18T20:52:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.578+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/1439216991 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fc10316890 0x55fc10318cb0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.578+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/1439216991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102f5db0 0x55fc102f3050 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.578+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/1439216991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 0x55fc102f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.578+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/1439216991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102eae90 0x55fc102f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.578+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1439216991 >> 172.21.15.46:0/1439216991 conn(0x55fc102611c0 msgr2=0x55fc102e6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.578+0000 7ff17dffb700 0 client.30055 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.578+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1439216991 shutdown_connections 2024-03-18T20:52:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.578+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1439216991 wait complete. 2024-03-18T20:52:46.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.578+0000 7ff1a8673b80 20 client.30055 trim_cache size 0 max 16384 2024-03-18T20:52:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.599+0000 7ff1a8673b80 1 Processor -- start 2024-03-18T20:52:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.599+0000 7ff1a8673b80 1 -- start start 2024-03-18T20:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff1a8673b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 0x55fc102fca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff1a8673b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 0x55fc103011c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff1a8673b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102f5db0 0x55fc102f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff1a8673b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc102ebe50 con 0x55fc102f5db0 2024-03-18T20:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff1a8673b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc102f3b30 con 0x55fc102ea570 2024-03-18T20:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff1a8673b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc102f87d0 con 0x55fc102eae90 2024-03-18T20:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff199a79700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 0x55fc103011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff19aa7b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102f5db0 0x55fc102f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff199a79700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 0x55fc103011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:41754/0 (socket says 172.21.15.46:41754) 2024-03-18T20:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff199a79700 1 -- 172.21.15.46:0/1319943803 learned_addr learned my addr 172.21.15.46:0/1319943803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:46.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff19aa7b700 1 --2- 172.21.15.46:0/1319943803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 0x55fc102fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff199a79700 1 -- 172.21.15.46:0/1319943803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 msgr2=0x55fc102fca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff199a79700 1 --2- 172.21.15.46:0/1319943803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 0x55fc102fca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff199a79700 1 -- 172.21.15.46:0/1319943803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102f5db0 msgr2=0x55fc102f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff199a79700 1 --2- 172.21.15.46:0/1319943803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102f5db0 0x55fc102f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.600+0000 7ff199a79700 1 -- 172.21.15.46:0/1319943803 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc10203e40 con 0x55fc102eae90 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.601+0000 7ff199a79700 1 --2- 172.21.15.46:0/1319943803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 0x55fc103011c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff1840058e0 tx=0x7ff184005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9db94ac168d30dc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.601+0000 7ff1897fa700 1 -- 172.21.15.46:0/1319943803 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18401c040 con 0x55fc102eae90 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.601+0000 7ff1897fa700 1 -- 172.21.15.46:0/1319943803 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff18401d040 con 0x55fc102eae90 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.601+0000 7ff1897fa700 1 -- 172.21.15.46:0/1319943803 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18400b790 con 0x55fc102eae90 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.602+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1319943803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 msgr2=0x55fc103011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.602+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/1319943803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 0x55fc103011c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff1840058e0 tx=0x7ff184005fb0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.602+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1319943803 shutdown_connections 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.602+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/1319943803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102f5db0 0x55fc102f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.602+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/1319943803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 0x55fc102fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.602+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/1319943803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 0x55fc103011c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.602+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1319943803 >> 172.21.15.46:0/1319943803 conn(0x55fc102611c0 msgr2=0x55fc103188a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.602+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1319943803 shutdown_connections 2024-03-18T20:52:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.602+0000 7ff1a8673b80 1 -- 172.21.15.46:0/1319943803 wait complete. 2024-03-18T20:52:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.603+0000 7ff1a8673b80 1 -- start start 2024-03-18T20:52:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.603+0000 7ff1a8673b80 1 Processor -- start 2024-03-18T20:52:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.603+0000 7ff1a8673b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 0x55fc102f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.603+0000 7ff1a8673b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 0x55fc102eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.603+0000 7ff1a8673b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102f5db0 0x55fc102ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.603+0000 7ff1a8673b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc102efb00 con 0x55fc102f5db0 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.603+0000 7ff1a8673b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc102efc70 con 0x55fc102ea570 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.603+0000 7ff1a8673b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc102efe30 con 0x55fc102eae90 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.603+0000 7ff1a8673b80 10 client.0 _ll_register_callbacks cb 0x7ff18c004870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.604+0000 7ff199a79700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 0x55fc102eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.604+0000 7ff199a79700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 0x55fc102eecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:41764/0 (socket says 172.21.15.46:41764) 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.604+0000 7ff199a79700 1 -- 172.21.15.46:0/2648908039 learned_addr learned my addr 172.21.15.46:0/2648908039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.604+0000 7ff19aa7b700 1 --2- 172.21.15.46:0/2648908039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102f5db0 0x55fc102ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.604+0000 7ff199a79700 1 -- 172.21.15.46:0/2648908039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 msgr2=0x55fc102f7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.604+0000 7ff199a79700 1 --2- 172.21.15.46:0/2648908039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 0x55fc102f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.604+0000 7ff199a79700 1 -- 172.21.15.46:0/2648908039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102f5db0 msgr2=0x55fc102ef3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.604+0000 7ff199a79700 1 --2- 172.21.15.46:0/2648908039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102f5db0 0x55fc102ef3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.604+0000 7ff199a79700 1 -- 172.21.15.46:0/2648908039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19410e4f0 con 0x55fc102eae90 2024-03-18T20:52:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.604+0000 7ff199a79700 1 --2- 172.21.15.46:0/2648908039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 0x55fc102eecb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff184009990 tx=0x7ff184009340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31952687a7d52fd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:52:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.604+0000 7ff17dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:52:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.605+0000 7ff17dffb700 1 -- 172.21.15.46:0/2648908039 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18401c040 con 0x55fc102eae90 2024-03-18T20:52:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.605+0000 7ff17dffb700 1 -- 172.21.15.46:0/2648908039 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff184027de0 con 0x55fc102eae90 2024-03-18T20:52:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.605+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2648908039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff18c00f040 con 0x55fc102eae90 2024-03-18T20:52:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.605+0000 7ff17dffb700 1 -- 172.21.15.46:0/2648908039 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1840052d0 con 0x55fc102eae90 2024-03-18T20:52:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.608+0000 7ff1a8673b80 10 client.30061 fetch_fsmap learned FSMap version 78 2024-03-18T20:52:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.608+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2648908039 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55fc100c3c70 con 0x55fc102eae90 2024-03-18T20:52:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.608+0000 7ff17dffb700 1 -- 172.21.15.46:0/2648908039 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff18c00f040 con 0x55fc102eae90 2024-03-18T20:52:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.608+0000 7ff17dffb700 1 -- 172.21.15.46:0/2648908039 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff18400b630 con 0x55fc102eae90 2024-03-18T20:52:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.608+0000 7ff1a8673b80 10 client.30061 fetch_fsmap finished waiting for FSMap version 78 2024-03-18T20:52:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.608+0000 7ff1a8673b80 10 client.30061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:52:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.608+0000 7ff1a8673b80 20 client.30061 populate_metadata read hostname 'smithi046' 2024-03-18T20:52:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.608+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/2648908039 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fc1036aab0 0x55fc1036cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:52:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.608+0000 7ff1a8673b80 4 client.30061 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:52:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.608+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2648908039 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc1008cc90 con 0x55fc1036aab0 2024-03-18T20:52:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.610+0000 7ff19aa7b700 1 --2- 172.21.15.46:0/2648908039 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fc1036aab0 0x55fc1036cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:52:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.611+0000 7ff17dffb700 10 client.30061 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.611+0000 7ff19aa7b700 1 --2- 172.21.15.46:0/2648908039 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fc1036aab0 0x55fc1036cea0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d517626e90246a82 server_cookie=6a225168bd95b6d1 in_seq=0 out_seq=0 2024-03-18T20:52:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.852+0000 7ff17dffb700 1 -- 172.21.15.46:0/2648908039 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc1008cc90 con 0x55fc1036aab0 2024-03-18T20:52:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.852+0000 7ff17dffb700 10 client.30061 handle_command_reply: tid=0 2024-03-18T20:52:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.852+0000 7ff1a8673b80 1 client.30061 shutdown 2024-03-18T20:52:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.853+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2648908039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 msgr2=0x55fc102eecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.853+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/2648908039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 0x55fc102eecb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff184009990 tx=0x7ff184009340 comp rx=0 tx=0).stop 2024-03-18T20:52:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.853+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2648908039 shutdown_connections 2024-03-18T20:52:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.853+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/2648908039 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55fc1036aab0 0x55fc1036cea0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.853+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/2648908039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc102f5db0 0x55fc102ef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.853+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/2648908039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc102ea570 0x55fc102f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.853+0000 7ff1a8673b80 1 --2- 172.21.15.46:0/2648908039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fc102eae90 0x55fc102eecb0 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff184009990 tx=0x7ff184009340 comp rx=0 tx=0).stop 2024-03-18T20:52:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.853+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2648908039 >> 172.21.15.46:0/2648908039 conn(0x55fc102611c0 msgr2=0x55fc102f1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.853+0000 7ff17dffb700 0 client.30061 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:52:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.855+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2648908039 shutdown_connections 2024-03-18T20:52:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.855+0000 7ff1a8673b80 1 -- 172.21.15.46:0/2648908039 wait complete. 2024-03-18T20:52:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.856+0000 7ff1a8673b80 20 client.30061 trim_cache size 0 max 16384 2024-03-18T20:52:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:52:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:52:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:52:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:52:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.860+0000 7ff19c4de700 1 -- 172.21.15.46:0/55465770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff1800a1310 msgr2=0x7ff1800a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.861+0000 7ff19c4de700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff1800a1310 0x7ff1800a37d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff1900ca6b0 tx=0x7ff190010c60 comp rx=0 tx=0).stop 2024-03-18T20:52:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.861+0000 7ff19c4de700 1 -- 172.21.15.46:0/55465770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1940ff220 msgr2=0x7ff1941a3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:52:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.861+0000 7ff19c4de700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1940ff220 0x7ff1941a3580 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7ff19000e9f0 tx=0x7ff19000ed00 comp rx=0 tx=0).stop 2024-03-18T20:52:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.861+0000 7ff19c4de700 1 -- 172.21.15.46:0/55465770 shutdown_connections 2024-03-18T20:52:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.861+0000 7ff19c4de700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff1800a1310 0x7ff1800a37d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.861+0000 7ff19c4de700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1940ff220 0x7ff1941a3580 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.861+0000 7ff19c4de700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff194106e30 0x7ff19410cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.861+0000 7ff19c4de700 1 --2- 172.21.15.46:0/55465770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1940ffbe0 0x7ff1941a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:52:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.861+0000 7ff19c4de700 1 -- 172.21.15.46:0/55465770 >> 172.21.15.46:0/55465770 conn(0x7ff1940761b0 msgr2=0x7ff194103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:52:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.861+0000 7ff19c4de700 1 -- 172.21.15.46:0/55465770 shutdown_connections 2024-03-18T20:52:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:52:46.861+0000 7ff19c4de700 1 -- 172.21.15.46:0/55465770 wait complete. 2024-03-18T20:52:47.372 INFO:tasks.workunit.client.0.smithi046.stdout: 10 176685 43.64 MB/sec execute 83 sec latency 161.279 ms 2024-03-18T20:52:48.372 INFO:tasks.workunit.client.0.smithi046.stdout: 10 177394 43.52 MB/sec execute 84 sec latency 58.667 ms 2024-03-18T20:52:48.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:48 smithi073 ceph-mon[47384]: pgmap v524: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.6 MiB/s wr, 635 op/s 2024-03-18T20:52:48.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:48 smithi155 ceph-mon[48344]: pgmap v524: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.6 MiB/s wr, 635 op/s 2024-03-18T20:52:48.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:48 smithi046 ceph-mon[43495]: pgmap v524: 97 pgs: 97 active+clean; 1.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.6 MiB/s wr, 635 op/s 2024-03-18T20:52:49.372 INFO:tasks.workunit.client.0.smithi046.stdout: 10 178192 43.43 MB/sec execute 85 sec latency 73.565 ms 2024-03-18T20:52:50.375 INFO:tasks.workunit.client.0.smithi046.stdout: 10 179106 43.59 MB/sec execute 86 sec latency 65.154 ms 2024-03-18T20:52:50.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:50 smithi073 ceph-mon[47384]: pgmap v525: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-03-18T20:52:50.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:50 smithi155 ceph-mon[48344]: pgmap v525: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-03-18T20:52:50.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:50 smithi046 ceph-mon[43495]: pgmap v525: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-03-18T20:52:51.374 INFO:tasks.workunit.client.0.smithi046.stdout: 10 179894 43.54 MB/sec execute 87 sec latency 76.080 ms 2024-03-18T20:52:52.373 INFO:tasks.workunit.client.0.smithi046.stdout: 10 180717 43.59 MB/sec execute 88 sec latency 52.671 ms 2024-03-18T20:52:52.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:52 smithi073 ceph-mon[47384]: pgmap v526: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 667 op/s 2024-03-18T20:52:52.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:52 smithi155 ceph-mon[48344]: pgmap v526: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 667 op/s 2024-03-18T20:52:52.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:52 smithi046 ceph-mon[43495]: pgmap v526: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 667 op/s 2024-03-18T20:52:53.376 INFO:tasks.workunit.client.0.smithi046.stdout: 10 181712 43.63 MB/sec execute 89 sec latency 60.736 ms 2024-03-18T20:52:54.376 INFO:tasks.workunit.client.0.smithi046.stdout: 10 182654 43.89 MB/sec execute 90 sec latency 60.607 ms 2024-03-18T20:52:54.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:54 smithi073 ceph-mon[47384]: pgmap v527: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.5 MiB/s wr, 769 op/s 2024-03-18T20:52:54.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:54 smithi155 ceph-mon[48344]: pgmap v527: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.5 MiB/s wr, 769 op/s 2024-03-18T20:52:54.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:54 smithi046 ceph-mon[43495]: pgmap v527: 97 pgs: 97 active+clean; 1.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.5 MiB/s wr, 769 op/s 2024-03-18T20:52:55.376 INFO:tasks.workunit.client.0.smithi046.stdout: 10 183413 43.74 MB/sec execute 91 sec latency 38.646 ms 2024-03-18T20:52:56.376 INFO:tasks.workunit.client.0.smithi046.stdout: 10 184365 43.82 MB/sec execute 92 sec latency 33.995 ms 2024-03-18T20:52:56.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:56 smithi073 ceph-mon[47384]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.1 MiB/s wr, 848 op/s 2024-03-18T20:52:56.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:56 smithi155 ceph-mon[48344]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.1 MiB/s wr, 848 op/s 2024-03-18T20:52:56.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:56 smithi046 ceph-mon[43495]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.1 MiB/s wr, 848 op/s 2024-03-18T20:52:57.376 INFO:tasks.workunit.client.0.smithi046.stdout: 10 185312 43.87 MB/sec execute 93 sec latency 62.925 ms 2024-03-18T20:52:58.376 INFO:tasks.workunit.client.0.smithi046.stdout: 10 186189 43.96 MB/sec execute 94 sec latency 52.144 ms 2024-03-18T20:52:58.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:52:58 smithi073 ceph-mon[47384]: pgmap v529: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.9 MiB/s wr, 658 op/s 2024-03-18T20:52:58.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:52:58 smithi155 ceph-mon[48344]: pgmap v529: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.9 MiB/s wr, 658 op/s 2024-03-18T20:52:58.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:52:58 smithi046 ceph-mon[43495]: pgmap v529: 97 pgs: 97 active+clean; 1.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.9 MiB/s wr, 658 op/s 2024-03-18T20:52:59.376 INFO:tasks.workunit.client.0.smithi046.stdout: 10 187054 43.91 MB/sec execute 95 sec latency 63.013 ms 2024-03-18T20:53:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.101+0000 7f16e2b55700 1 -- 172.21.15.46:0/9463224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc10f4d0 msgr2=0x7f16dc107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.101+0000 7f16e2b55700 1 --2- 172.21.15.46:0/9463224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc10f4d0 0x7f16dc107db0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f16d400b0d0 tx=0x7f16d400b3e0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.105+0000 7f16e2b55700 1 -- 172.21.15.46:0/9463224 shutdown_connections 2024-03-18T20:53:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.105+0000 7f16e2b55700 1 --2- 172.21.15.46:0/9463224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc10f4d0 0x7f16dc107db0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.105+0000 7f16e2b55700 1 --2- 172.21.15.46:0/9463224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc11a590 0x7f16dc11c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.105+0000 7f16e2b55700 1 --2- 172.21.15.46:0/9463224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc10f940 0x7f16dc1082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.105+0000 7f16e2b55700 1 -- 172.21.15.46:0/9463224 >> 172.21.15.46:0/9463224 conn(0x7f16dc06bd80 msgr2=0x7f16dc06c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.119+0000 7f16e2b55700 1 -- 172.21.15.46:0/9463224 shutdown_connections 2024-03-18T20:53:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.128+0000 7f16e2b55700 1 -- 172.21.15.46:0/9463224 wait complete. 2024-03-18T20:53:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.128+0000 7f16e2b55700 1 Processor -- start 2024-03-18T20:53:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.129+0000 7f16e2b55700 1 -- start start 2024-03-18T20:53:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.129+0000 7f16e2b55700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc10f4d0 0x7f16dc1199f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.129+0000 7f16e2b55700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc10f940 0x7f16dc119f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.129+0000 7f16e2b55700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc11a590 0x7f16dc113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.129+0000 7f16e2b55700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc114380 con 0x7f16dc11a590 2024-03-18T20:53:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.129+0000 7f16e2b55700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc1144f0 con 0x7f16dc10f4d0 2024-03-18T20:53:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.129+0000 7f16e2b55700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc1146b0 con 0x7f16dc10f940 2024-03-18T20:53:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.132+0000 7f16dbfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc10f940 0x7f16dc119f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.132+0000 7f16dbfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc10f940 0x7f16dc119f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:60298/0 (socket says 172.21.15.46:60298) 2024-03-18T20:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.132+0000 7f16dbfff700 1 -- 172.21.15.46:0/1425754027 learned_addr learned my addr 172.21.15.46:0/1425754027 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.132+0000 7f16e10f2700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc11a590 0x7f16dc113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.137+0000 7f16dbfff700 1 -- 172.21.15.46:0/1425754027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc10f4d0 msgr2=0x7f16dc1199f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.137+0000 7f16dbfff700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc10f4d0 0x7f16dc1199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.137+0000 7f16dbfff700 1 -- 172.21.15.46:0/1425754027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc11a590 msgr2=0x7f16dc113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.137+0000 7f16dbfff700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc11a590 0x7f16dc113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.137+0000 7f16dbfff700 1 -- 172.21.15.46:0/1425754027 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d4009d00 con 0x7f16dc10f940 2024-03-18T20:53:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.145+0000 7f16dbfff700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc10f940 0x7f16dc119f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f16d000d980 tx=0x7f16d000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.148+0000 7f16d9ffb700 1 -- 172.21.15.46:0/1425754027 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0009b40 con 0x7f16dc10f940 2024-03-18T20:53:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.148+0000 7f16d9ffb700 1 -- 172.21.15.46:0/1425754027 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d000be20 con 0x7f16dc10f940 2024-03-18T20:53:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.148+0000 7f16d9ffb700 1 -- 172.21.15.46:0/1425754027 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d000f620 con 0x7f16dc10f940 2024-03-18T20:53:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.148+0000 7f16e2b55700 1 -- 172.21.15.46:0/1425754027 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16dc079010 con 0x7f16dc10f940 2024-03-18T20:53:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.149+0000 7f16e2b55700 1 -- 172.21.15.46:0/1425754027 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc0794d0 con 0x7f16dc10f940 2024-03-18T20:53:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.154+0000 7f16d9ffb700 1 -- 172.21.15.46:0/1425754027 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f16d000f780 con 0x7f16dc10f940 2024-03-18T20:53:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.155+0000 7f16d9ffb700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f16c40a14a0 0x7f16c40a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.159 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.155+0000 7f16d9ffb700 1 -- 172.21.15.46:0/1425754027 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f16d00cd880 con 0x7f16dc10f940 2024-03-18T20:53:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.156+0000 7f16eeceab80 1 Processor -- start 2024-03-18T20:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.156+0000 7f16eeceab80 1 -- start start 2024-03-18T20:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.156+0000 7f16eeceab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc83570 0x56403bc83950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.156+0000 7f16eeceab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 0x56403bc8e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.156+0000 7f16eeceab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc8edb0 0x56403bc911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.156+0000 7f16eeceab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bbdd930 con 0x56403bc83e90 2024-03-18T20:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.156+0000 7f16eeceab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bbdb2f0 con 0x56403bc8edb0 2024-03-18T20:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.156+0000 7f16eeceab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bbd6be0 con 0x56403bc83570 2024-03-18T20:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.158+0000 7f16dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 0x56403bc8e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.159+0000 7f16dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 0x56403bc8e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58146/0 (socket says 172.21.15.46:58146) 2024-03-18T20:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.159+0000 7f16dbfff700 1 -- 172.21.15.46:0/2232056043 learned_addr learned my addr 172.21.15.46:0/2232056043 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.164+0000 7f16e08f1700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f16c40a14a0 0x7f16c40a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.167+0000 7f16e10f2700 1 --2- 172.21.15.46:0/2232056043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc8edb0 0x56403bc911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.175+0000 7f16dbfff700 1 -- 172.21.15.46:0/2232056043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc83570 msgr2=0x56403bc83950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.175+0000 7f16dbfff700 1 --2- 172.21.15.46:0/2232056043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc83570 0x56403bc83950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.175+0000 7f16dbfff700 1 -- 172.21.15.46:0/2232056043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc8edb0 msgr2=0x56403bc911a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.175+0000 7f16dbfff700 1 --2- 172.21.15.46:0/2232056043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc8edb0 0x56403bc911a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.175+0000 7f16dbfff700 1 -- 172.21.15.46:0/2232056043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56403ba58350 con 0x56403bc83e90 2024-03-18T20:53:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.178+0000 7f16dbfff700 1 --2- 172.21.15.46:0/2232056043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 0x56403bc8e870 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f16d0010c40 tx=0x7f16d009a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f1cf143ce0c979 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.185+0000 7f16c2ffd700 1 -- 172.21.15.46:0/2232056043 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d009e740 con 0x56403bc83e90 2024-03-18T20:53:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.185+0000 7f16c2ffd700 1 -- 172.21.15.46:0/2232056043 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d00a2d90 con 0x56403bc83e90 2024-03-18T20:53:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.185+0000 7f16c2ffd700 1 -- 172.21.15.46:0/2232056043 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d009a000 con 0x56403bc83e90 2024-03-18T20:53:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.186+0000 7f16e08f1700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f16c40a14a0 0x7f16c40a3960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f16d400b0d0 tx=0x7f16d4000f40 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.187+0000 7f16eeceab80 1 -- 172.21.15.46:0/2232056043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 msgr2=0x56403bc8e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.187+0000 7f16eeceab80 1 --2- 172.21.15.46:0/2232056043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 0x56403bc8e870 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f16d0010c40 tx=0x7f16d009a2f0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.192+0000 7f16eeceab80 1 -- 172.21.15.46:0/2232056043 shutdown_connections 2024-03-18T20:53:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.192+0000 7f16eeceab80 1 --2- 172.21.15.46:0/2232056043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc8edb0 0x56403bc911a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.192+0000 7f16eeceab80 1 --2- 172.21.15.46:0/2232056043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc83570 0x56403bc83950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.192+0000 7f16eeceab80 1 --2- 172.21.15.46:0/2232056043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 0x56403bc8e870 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.192+0000 7f16eeceab80 1 -- 172.21.15.46:0/2232056043 >> 172.21.15.46:0/2232056043 conn(0x56403bbfa1c0 msgr2=0x56403bc7fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.205+0000 7f16eeceab80 1 -- 172.21.15.46:0/2232056043 shutdown_connections 2024-03-18T20:53:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.206+0000 7f16eeceab80 1 -- 172.21.15.46:0/2232056043 wait complete. 2024-03-18T20:53:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.206+0000 7f16eeceab80 1 -- start start 2024-03-18T20:53:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.206+0000 7f16eeceab80 1 Processor -- start 2024-03-18T20:53:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.207+0000 7f16eeceab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc83570 0x56403bc8e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.207+0000 7f16eeceab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 0x56403bc8bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.207+0000 7f16eeceab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc8edb0 0x56403bc8c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.207+0000 7f16eeceab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bb72d20 con 0x56403bc83e90 2024-03-18T20:53:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.207+0000 7f16eeceab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bb729c0 con 0x56403bc8edb0 2024-03-18T20:53:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.207+0000 7f16eeceab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403ba5a640 con 0x56403bc83570 2024-03-18T20:53:00.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.207+0000 7f16eeceab80 10 client.0 _ll_register_callbacks cb 0x56403b9de340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:53:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.216+0000 7f16dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 0x56403bc8bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.216+0000 7f16dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 0x56403bc8bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58150/0 (socket says 172.21.15.46:58150) 2024-03-18T20:53:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.216+0000 7f16dbfff700 1 -- 172.21.15.46:0/561717241 learned_addr learned my addr 172.21.15.46:0/561717241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.217+0000 7f16e10f2700 1 --2- 172.21.15.46:0/561717241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc8edb0 0x56403bc8c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.225+0000 7f16dbfff700 1 -- 172.21.15.46:0/561717241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc83570 msgr2=0x56403bc8e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.225+0000 7f16dbfff700 1 --2- 172.21.15.46:0/561717241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc83570 0x56403bc8e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.225+0000 7f16dbfff700 1 -- 172.21.15.46:0/561717241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc8edb0 msgr2=0x56403bc8c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.225+0000 7f16dbfff700 1 --2- 172.21.15.46:0/561717241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc8edb0 0x56403bc8c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.225+0000 7f16dbfff700 1 -- 172.21.15.46:0/561717241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc0794d0 con 0x56403bc83e90 2024-03-18T20:53:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.236+0000 7f16dbfff700 1 --2- 172.21.15.46:0/561717241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 0x56403bc8bb60 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f16d0012040 tx=0x7f16d000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8397140fd8f20d80 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.236+0000 7f16bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:53:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.242+0000 7f16bbfff700 1 -- 172.21.15.46:0/561717241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d009e740 con 0x56403bc83e90 2024-03-18T20:53:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.242+0000 7f16bbfff700 1 -- 172.21.15.46:0/561717241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f16d009a010 con 0x56403bc83e90 2024-03-18T20:53:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.244+0000 7f16eeceab80 1 -- 172.21.15.46:0/561717241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56403bc776c0 con 0x56403bc83e90 2024-03-18T20:53:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.256+0000 7f16bbfff700 1 -- 172.21.15.46:0/561717241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d00b3120 con 0x56403bc83e90 2024-03-18T20:53:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.257+0000 7f16eeceab80 10 client.20403 fetch_fsmap learned FSMap version 78 2024-03-18T20:53:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.257+0000 7f16eeceab80 1 -- 172.21.15.46:0/561717241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x56403ba04b00 con 0x56403bc83e90 2024-03-18T20:53:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.267+0000 7f16bbfff700 1 -- 172.21.15.46:0/561717241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56403bc776c0 con 0x56403bc83e90 2024-03-18T20:53:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.267+0000 7f16bbfff700 1 -- 172.21.15.46:0/561717241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f16d00ba0b0 con 0x56403bc83e90 2024-03-18T20:53:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.271+0000 7f16eeceab80 10 client.20403 fetch_fsmap finished waiting for FSMap version 78 2024-03-18T20:53:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.271+0000 7f16eeceab80 10 client.20403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:53:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.271+0000 7f16eeceab80 20 client.20403 populate_metadata read hostname 'smithi046' 2024-03-18T20:53:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.271+0000 7f16eeceab80 1 --2- 172.21.15.46:0/561717241 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56403bc9e4c0 0x56403bca08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.271+0000 7f16eeceab80 4 client.20403 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:53:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.271+0000 7f16eeceab80 1 -- 172.21.15.46:0/561717241 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56403bbcf6a0 con 0x56403bc9e4c0 2024-03-18T20:53:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.278+0000 7f16e08f1700 1 --2- 172.21.15.46:0/561717241 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56403bc9e4c0 0x56403bca08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.320+0000 7f16e08f1700 1 --2- 172.21.15.46:0/561717241 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56403bc9e4c0 0x56403bca08b0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5d82bf24efce561 server_cookie=27a749e2112aad4d in_seq=0 out_seq=0 2024-03-18T20:53:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.321+0000 7f16bbfff700 10 client.20403 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.355+0000 7f16bbfff700 1 -- 172.21.15.46:0/561717241 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56403bbcf6a0 con 0x56403bc9e4c0 2024-03-18T20:53:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.355+0000 7f16bbfff700 10 client.20403 handle_command_reply: tid=0 2024-03-18T20:53:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.355+0000 7f16eeceab80 1 client.20403 shutdown 2024-03-18T20:53:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.356+0000 7f16eeceab80 1 -- 172.21.15.46:0/561717241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 msgr2=0x56403bc8bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.356+0000 7f16eeceab80 1 --2- 172.21.15.46:0/561717241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 0x56403bc8bb60 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f16d0012040 tx=0x7f16d000bf90 comp rx=0 tx=0).stop 2024-03-18T20:53:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.356+0000 7f16eeceab80 1 -- 172.21.15.46:0/561717241 shutdown_connections 2024-03-18T20:53:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.356+0000 7f16eeceab80 1 --2- 172.21.15.46:0/561717241 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56403bc9e4c0 0x56403bca08b0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.356+0000 7f16eeceab80 1 --2- 172.21.15.46:0/561717241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc8edb0 0x56403bc8c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.356+0000 7f16eeceab80 1 --2- 172.21.15.46:0/561717241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc83570 0x56403bc8e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.356+0000 7f16eeceab80 1 --2- 172.21.15.46:0/561717241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83e90 0x56403bc8bb60 secure :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f16d0012040 tx=0x7f16d000bf90 comp rx=0 tx=0).stop 2024-03-18T20:53:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.356+0000 7f16eeceab80 1 -- 172.21.15.46:0/561717241 >> 172.21.15.46:0/561717241 conn(0x56403bbfa1c0 msgr2=0x56403bc7f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:00.364 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.360+0000 7f16bbfff700 0 client.20403 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:00.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 187664 43.84 MB/sec execute 96 sec latency 424.935 ms 2024-03-18T20:53:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.378+0000 7f16eeceab80 1 -- 172.21.15.46:0/561717241 shutdown_connections 2024-03-18T20:53:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.381+0000 7f16eeceab80 1 -- 172.21.15.46:0/561717241 wait complete. 2024-03-18T20:53:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.385+0000 7f16eeceab80 20 client.20403 trim_cache size 0 max 16384 2024-03-18T20:53:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.416+0000 7f16eeceab80 1 Processor -- start 2024-03-18T20:53:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.419+0000 7f16eeceab80 1 -- start start 2024-03-18T20:53:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.419+0000 7f16eeceab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc83570 0x56403bc911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.419+0000 7f16eeceab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc8edb0 0x56403bc864e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.419+0000 7f16eeceab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc95a10 0x56403bc8e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.419+0000 7f16eeceab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bc9b520 con 0x56403bc95a10 2024-03-18T20:53:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.419+0000 7f16eeceab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bc79c40 con 0x56403bc83570 2024-03-18T20:53:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.419+0000 7f16eeceab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bc879f0 con 0x56403bc8edb0 2024-03-18T20:53:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.422+0000 7f16e10f2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc95a10 0x56403bc8e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.422+0000 7f16e10f2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc95a10 0x56403bc8e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58166/0 (socket says 172.21.15.46:58166) 2024-03-18T20:53:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.422+0000 7f16e10f2700 1 -- 172.21.15.46:0/766865304 learned_addr learned my addr 172.21.15.46:0/766865304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.426+0000 7f16e10f2700 1 -- 172.21.15.46:0/766865304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc8edb0 msgr2=0x56403bc864e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:53:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.426+0000 7f16e10f2700 1 --2- 172.21.15.46:0/766865304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc8edb0 0x56403bc864e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.426+0000 7f16e10f2700 1 -- 172.21.15.46:0/766865304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc83570 msgr2=0x56403bc911a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.426+0000 7f16e10f2700 1 --2- 172.21.15.46:0/766865304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc83570 0x56403bc911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.426+0000 7f16e10f2700 1 -- 172.21.15.46:0/766865304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56403bb9ce40 con 0x56403bc95a10 2024-03-18T20:53:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.435+0000 7f16e10f2700 1 --2- 172.21.15.46:0/766865304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc95a10 0x56403bc8e3c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00b840 tx=0x7f16cc00bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f513810752adf031 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.446+0000 7f16c37fe700 1 -- 172.21.15.46:0/766865304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc0156f0 con 0x56403bc95a10 2024-03-18T20:53:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.446+0000 7f16c37fe700 1 -- 172.21.15.46:0/766865304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16cc015d10 con 0x56403bc95a10 2024-03-18T20:53:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.446+0000 7f16c37fe700 1 -- 172.21.15.46:0/766865304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc013ea0 con 0x56403bc95a10 2024-03-18T20:53:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.447+0000 7f16eeceab80 1 -- 172.21.15.46:0/766865304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc95a10 msgr2=0x56403bc8e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.447+0000 7f16eeceab80 1 --2- 172.21.15.46:0/766865304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc95a10 0x56403bc8e3c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00b840 tx=0x7f16cc00bb50 comp rx=0 tx=0).stop 2024-03-18T20:53:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.453+0000 7f16eeceab80 1 -- 172.21.15.46:0/766865304 shutdown_connections 2024-03-18T20:53:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.453+0000 7f16eeceab80 1 --2- 172.21.15.46:0/766865304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc95a10 0x56403bc8e3c0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.453+0000 7f16eeceab80 1 --2- 172.21.15.46:0/766865304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc83570 0x56403bc911a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.453+0000 7f16eeceab80 1 --2- 172.21.15.46:0/766865304 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc8edb0 0x56403bc864e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.453+0000 7f16eeceab80 1 -- 172.21.15.46:0/766865304 >> 172.21.15.46:0/766865304 conn(0x56403bbfa1c0 msgr2=0x56403bc7e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.459+0000 7f16eeceab80 1 -- 172.21.15.46:0/766865304 shutdown_connections 2024-03-18T20:53:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.466+0000 7f16eeceab80 1 -- 172.21.15.46:0/766865304 wait complete. 2024-03-18T20:53:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.466+0000 7f16eeceab80 1 -- start start 2024-03-18T20:53:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.467+0000 7f16eeceab80 1 Processor -- start 2024-03-18T20:53:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.467+0000 7f16eeceab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83570 0x56403bc93be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.467+0000 7f16eeceab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc8edb0 0x56403bc94120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.467+0000 7f16eeceab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc95a10 0x56403bcf12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.467+0000 7f16eeceab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bcf17e0 con 0x56403bc83570 2024-03-18T20:53:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.467+0000 7f16eeceab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bcf1920 con 0x56403bc95a10 2024-03-18T20:53:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.467+0000 7f16eeceab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56403bcf1a90 con 0x56403bc8edb0 2024-03-18T20:53:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.467+0000 7f16eeceab80 10 client.0 _ll_register_callbacks cb 0x7f16cc00b1a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:53:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.471+0000 7f16dbfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc8edb0 0x56403bc94120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.472+0000 7f16dbfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc8edb0 0x56403bc94120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:60334/0 (socket says 172.21.15.46:60334) 2024-03-18T20:53:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.472+0000 7f16dbfff700 1 -- 172.21.15.46:0/3036088443 learned_addr learned my addr 172.21.15.46:0/3036088443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.475+0000 7f16e10f2700 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc95a10 0x56403bcf12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.476+0000 7f16e08f1700 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83570 0x56403bc93be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.479+0000 7f16e08f1700 1 -- 172.21.15.46:0/3036088443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc8edb0 msgr2=0x56403bc94120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.479+0000 7f16e08f1700 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc8edb0 0x56403bc94120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.479+0000 7f16e08f1700 1 -- 172.21.15.46:0/3036088443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc95a10 msgr2=0x56403bcf12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.479+0000 7f16e08f1700 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc95a10 0x56403bcf12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.479+0000 7f16e08f1700 1 -- 172.21.15.46:0/3036088443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16cc00e040 con 0x56403bc83570 2024-03-18T20:53:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.484+0000 7f16e08f1700 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83570 0x56403bc93be0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x56403bca1140 tx=0x7f16d40090d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6e90da961963ee server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.492+0000 7f16bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:53:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.492+0000 7f16bbfff700 1 -- 172.21.15.46:0/3036088443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4030030 con 0x56403bc83570 2024-03-18T20:53:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.493+0000 7f16bbfff700 1 -- 172.21.15.46:0/3036088443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f16d4031070 con 0x56403bc83570 2024-03-18T20:53:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.493+0000 7f16eeceab80 1 -- 172.21.15.46:0/3036088443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16d00a2d90 con 0x56403bc83570 2024-03-18T20:53:00.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.504+0000 7f16bbfff700 1 -- 172.21.15.46:0/3036088443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d402cea0 con 0x56403bc83570 2024-03-18T20:53:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.514+0000 7f16eeceab80 10 client.20409 fetch_fsmap learned FSMap version 78 2024-03-18T20:53:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.514+0000 7f16eeceab80 1 -- 172.21.15.46:0/3036088443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x56403ba5cc70 con 0x56403bc83570 2024-03-18T20:53:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.521+0000 7f16bbfff700 1 -- 172.21.15.46:0/3036088443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d00a2d90 con 0x56403bc83570 2024-03-18T20:53:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.521+0000 7f16bbfff700 1 -- 172.21.15.46:0/3036088443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f16d401d830 con 0x56403bc83570 2024-03-18T20:53:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.525+0000 7f16eeceab80 10 client.20409 fetch_fsmap finished waiting for FSMap version 78 2024-03-18T20:53:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.525+0000 7f16eeceab80 10 client.20409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:53:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.525+0000 7f16eeceab80 20 client.20409 populate_metadata read hostname 'smithi046' 2024-03-18T20:53:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.525+0000 7f16eeceab80 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56403bcf3210 0x56403bcf5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.525+0000 7f16eeceab80 4 client.20409 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:53:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.525+0000 7f16eeceab80 1 -- 172.21.15.46:0/3036088443 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56403ba25c90 con 0x56403bcf3210 2024-03-18T20:53:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.532+0000 7f16dbfff700 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56403bcf3210 0x56403bcf5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.545+0000 7f16dbfff700 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56403bcf3210 0x56403bcf5630 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a98fead5a8c7dd55 server_cookie=7e58106695db3c9e in_seq=0 out_seq=0 2024-03-18T20:53:00.552 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:00.549+0000 7f16bbfff700 10 client.20409 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:00 smithi073 ceph-mon[47384]: pgmap v530: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 943 op/s 2024-03-18T20:53:00.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:00 smithi155 ceph-mon[48344]: pgmap v530: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 943 op/s 2024-03-18T20:53:00.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:00 smithi046 ceph-mon[43495]: pgmap v530: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 943 op/s 2024-03-18T20:53:01.382 INFO:tasks.workunit.client.0.smithi046.stdout: 10 187683 43.41 MB/sec execute 97 sec latency 1413.834 ms 2024-03-18T20:53:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.028+0000 7f16bbfff700 1 -- 172.21.15.46:0/3036088443 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56403ba25c90 con 0x56403bcf3210 2024-03-18T20:53:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.028+0000 7f16bbfff700 10 client.20409 handle_command_reply: tid=0 2024-03-18T20:53:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.028+0000 7f16eeceab80 1 client.20409 shutdown 2024-03-18T20:53:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.029+0000 7f16eeceab80 1 -- 172.21.15.46:0/3036088443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83570 msgr2=0x56403bc93be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.029+0000 7f16eeceab80 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83570 0x56403bc93be0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x56403bca1140 tx=0x7f16d40090d0 comp rx=0 tx=0).stop 2024-03-18T20:53:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.032+0000 7f16eeceab80 1 -- 172.21.15.46:0/3036088443 shutdown_connections 2024-03-18T20:53:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.032+0000 7f16eeceab80 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56403bcf3210 0x56403bcf5630 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.032+0000 7f16eeceab80 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56403bc83570 0x56403bc93be0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.032+0000 7f16eeceab80 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56403bc95a10 0x56403bcf12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.032+0000 7f16eeceab80 1 --2- 172.21.15.46:0/3036088443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56403bc8edb0 0x56403bc94120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.032+0000 7f16eeceab80 1 -- 172.21.15.46:0/3036088443 >> 172.21.15.46:0/3036088443 conn(0x56403bbfa1c0 msgr2=0x56403bc86e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.032+0000 7f16bbfff700 0 client.20409 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.033+0000 7f16eeceab80 1 -- 172.21.15.46:0/3036088443 shutdown_connections 2024-03-18T20:53:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.033+0000 7f16eeceab80 1 -- 172.21.15.46:0/3036088443 wait complete. 2024-03-18T20:53:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:53:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:53:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:53:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:53:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.033+0000 7f16eeceab80 20 client.20409 trim_cache size 0 max 16384 2024-03-18T20:53:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.035+0000 7f16e2b55700 1 -- 172.21.15.46:0/1425754027 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f16c40a14a0 msgr2=0x7f16c40a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.035+0000 7f16e2b55700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f16c40a14a0 0x7f16c40a3960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f16d400b0d0 tx=0x7f16d4000f40 comp rx=0 tx=0).stop 2024-03-18T20:53:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.035+0000 7f16e2b55700 1 -- 172.21.15.46:0/1425754027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc10f940 msgr2=0x7f16dc119f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.035+0000 7f16e2b55700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc10f940 0x7f16dc119f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f16d000d980 tx=0x7f16d000dd40 comp rx=0 tx=0).stop 2024-03-18T20:53:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.038+0000 7f16e2b55700 1 -- 172.21.15.46:0/1425754027 shutdown_connections 2024-03-18T20:53:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.038+0000 7f16e2b55700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f16c40a14a0 0x7f16c40a3960 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.038+0000 7f16e2b55700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16dc11a590 0x7f16dc113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.038+0000 7f16e2b55700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16dc10f4d0 0x7f16dc1199f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.038+0000 7f16e2b55700 1 --2- 172.21.15.46:0/1425754027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16dc10f940 0x7f16dc119f30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:02.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.038+0000 7f16e2b55700 1 -- 172.21.15.46:0/1425754027 >> 172.21.15.46:0/1425754027 conn(0x7f16dc06bd80 msgr2=0x7f16dc11a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.038+0000 7f16e2b55700 1 -- 172.21.15.46:0/1425754027 shutdown_connections 2024-03-18T20:53:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:02.038+0000 7f16e2b55700 1 -- 172.21.15.46:0/1425754027 wait complete. 2024-03-18T20:53:02.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 187900 43.09 MB/sec execute 98 sec latency 2027.117 ms 2024-03-18T20:53:02.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:02 smithi073 ceph-mon[47384]: pgmap v531: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.1 MiB/s wr, 572 op/s 2024-03-18T20:53:02.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:02 smithi155 ceph-mon[48344]: pgmap v531: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.1 MiB/s wr, 572 op/s 2024-03-18T20:53:02.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:02 smithi046 ceph-mon[43495]: pgmap v531: 97 pgs: 97 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.1 MiB/s wr, 572 op/s 2024-03-18T20:53:03.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 187992 42.71 MB/sec execute 99 sec latency 789.164 ms 2024-03-18T20:53:04.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 188545 42.51 MB/sec execute 100 sec latency 220.797 ms 2024-03-18T20:53:04.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:04 smithi073 ceph-mon[47384]: pgmap v532: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 12 MiB/s wr, 768 op/s 2024-03-18T20:53:04.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:04 smithi155 ceph-mon[48344]: pgmap v532: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 12 MiB/s wr, 768 op/s 2024-03-18T20:53:04.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:04 smithi046 ceph-mon[43495]: pgmap v532: 97 pgs: 97 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 12 MiB/s wr, 768 op/s 2024-03-18T20:53:05.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 189016 42.39 MB/sec execute 101 sec latency 227.580 ms 2024-03-18T20:53:06.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 189642 42.35 MB/sec execute 102 sec latency 190.359 ms 2024-03-18T20:53:06.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:06 smithi073 ceph-mon[47384]: pgmap v533: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 1.27k op/s 2024-03-18T20:53:06.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:06 smithi155 ceph-mon[48344]: pgmap v533: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 1.27k op/s 2024-03-18T20:53:06.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:06 smithi046 ceph-mon[43495]: pgmap v533: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 28 MiB/s wr, 1.27k op/s 2024-03-18T20:53:07.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 190388 42.31 MB/sec execute 103 sec latency 84.487 ms 2024-03-18T20:53:08.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 191269 42.33 MB/sec execute 104 sec latency 1014.427 ms 2024-03-18T20:53:08.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:08 smithi073 ceph-mon[47384]: pgmap v534: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 26 MiB/s wr, 1.08k op/s 2024-03-18T20:53:08.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:08 smithi155 ceph-mon[48344]: pgmap v534: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 26 MiB/s wr, 1.08k op/s 2024-03-18T20:53:08.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:08 smithi046 ceph-mon[43495]: pgmap v534: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 26 MiB/s wr, 1.08k op/s 2024-03-18T20:53:09.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 191694 42.16 MB/sec execute 105 sec latency 2014.470 ms 2024-03-18T20:53:09.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:09 smithi073 ceph-mon[47384]: osdmap e103: 12 total, 12 up, 12 in 2024-03-18T20:53:09.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:09 smithi155 ceph-mon[48344]: osdmap e103: 12 total, 12 up, 12 in 2024-03-18T20:53:09.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:09 smithi046 ceph-mon[43495]: osdmap e103: 12 total, 12 up, 12 in 2024-03-18T20:53:10.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 192260 42.03 MB/sec execute 106 sec latency 3014.509 ms 2024-03-18T20:53:10.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:10 smithi073 ceph-mon[47384]: pgmap v536: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 32 MiB/s wr, 1.52k op/s 2024-03-18T20:53:10.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:10 smithi155 ceph-mon[48344]: pgmap v536: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 32 MiB/s wr, 1.52k op/s 2024-03-18T20:53:10.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:10 smithi046 ceph-mon[43495]: pgmap v536: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 32 MiB/s wr, 1.52k op/s 2024-03-18T20:53:11.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 193123 42.14 MB/sec execute 107 sec latency 4014.542 ms 2024-03-18T20:53:12.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 193536 41.93 MB/sec execute 108 sec latency 5014.579 ms 2024-03-18T20:53:12.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:12 smithi073 ceph-mon[47384]: pgmap v537: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 32 MiB/s wr, 1.52k op/s 2024-03-18T20:53:12.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:12 smithi155 ceph-mon[48344]: pgmap v537: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 32 MiB/s wr, 1.52k op/s 2024-03-18T20:53:12.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:12 smithi046 ceph-mon[43495]: pgmap v537: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 32 MiB/s wr, 1.52k op/s 2024-03-18T20:53:13.380 INFO:tasks.workunit.client.0.smithi046.stdout: 10 194213 41.84 MB/sec execute 109 sec latency 6014.631 ms 2024-03-18T20:53:14.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 194916 41.81 MB/sec execute 110 sec latency 7014.671 ms 2024-03-18T20:53:14.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:14 smithi073 ceph-mon[47384]: pgmap v538: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 27 MiB/s wr, 1.39k op/s 2024-03-18T20:53:14.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:14 smithi155 ceph-mon[48344]: pgmap v538: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 27 MiB/s wr, 1.39k op/s 2024-03-18T20:53:14.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:14 smithi046 ceph-mon[43495]: pgmap v538: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 27 MiB/s wr, 1.39k op/s 2024-03-18T20:53:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.194+0000 7fe168ff0700 1 -- 172.21.15.46:0/1020861726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe164105d50 msgr2=0x7fe164106130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.194+0000 7fe168ff0700 1 --2- 172.21.15.46:0/1020861726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe164105d50 0x7fe164106130 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe154009990 tx=0x7fe154009ca0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.194+0000 7fe168ff0700 1 -- 172.21.15.46:0/1020861726 shutdown_connections 2024-03-18T20:53:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.194+0000 7fe168ff0700 1 --2- 172.21.15.46:0/1020861726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe164100a70 0x7fe16410c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.194+0000 7fe168ff0700 1 --2- 172.21.15.46:0/1020861726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe164105d50 0x7fe164106130 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.194+0000 7fe168ff0700 1 --2- 172.21.15.46:0/1020861726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1641000b0 0x7fe164100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.194+0000 7fe168ff0700 1 -- 172.21.15.46:0/1020861726 >> 172.21.15.46:0/1020861726 conn(0x7fe164076a20 msgr2=0x7fe164076e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.194+0000 7fe168ff0700 1 -- 172.21.15.46:0/1020861726 shutdown_connections 2024-03-18T20:53:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.194+0000 7fe168ff0700 1 -- 172.21.15.46:0/1020861726 wait complete. 2024-03-18T20:53:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.195+0000 7fe168ff0700 1 Processor -- start 2024-03-18T20:53:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.195+0000 7fe168ff0700 1 -- start start 2024-03-18T20:53:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe168ff0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1641000b0 0x7fe164195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe168ff0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe164100a70 0x7fe1641964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe168ff0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe164105d50 0x7fe16419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe168ff0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16419ae80 con 0x7fe1641000b0 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe168ff0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16419aff0 con 0x7fe164100a70 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe168ff0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16419b1b0 con 0x7fe164105d50 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe161d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe164100a70 0x7fe1641964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe16259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1641000b0 0x7fe164195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe161d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe164100a70 0x7fe1641964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33636/0 (socket says 172.21.15.46:33636) 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe161d9b700 1 -- 172.21.15.46:0/3940105308 learned_addr learned my addr 172.21.15.46:0/3940105308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe162d9d700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe164105d50 0x7fe16419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe161d9b700 1 -- 172.21.15.46:0/3940105308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe164105d50 msgr2=0x7fe16419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe161d9b700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe164105d50 0x7fe16419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe161d9b700 1 -- 172.21.15.46:0/3940105308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1641000b0 msgr2=0x7fe164195f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe161d9b700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1641000b0 0x7fe164195f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe161d9b700 1 -- 172.21.15.46:0/3940105308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1540095f0 con 0x7fe164100a70 2024-03-18T20:53:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.196+0000 7fe161d9b700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe164100a70 0x7fe1641964a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe15800e9a0 tx=0x7fe15800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.197+0000 7fe1537fe700 1 -- 172.21.15.46:0/3940105308 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158009790 con 0x7fe164100a70 2024-03-18T20:53:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.197+0000 7fe1537fe700 1 -- 172.21.15.46:0/3940105308 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe158010430 con 0x7fe164100a70 2024-03-18T20:53:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.197+0000 7fe168ff0700 1 -- 172.21.15.46:0/3940105308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16419b490 con 0x7fe164100a70 2024-03-18T20:53:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.197+0000 7fe1537fe700 1 -- 172.21.15.46:0/3940105308 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158018680 con 0x7fe164100a70 2024-03-18T20:53:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.197+0000 7fe168ff0700 1 -- 172.21.15.46:0/3940105308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1641a2cd0 con 0x7fe164100a70 2024-03-18T20:53:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.200+0000 7fe1537fe700 1 -- 172.21.15.46:0/3940105308 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe1580187e0 con 0x7fe164100a70 2024-03-18T20:53:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.201+0000 7fe1537fe700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe14c0a1390 0x7fe14c0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.201+0000 7fe1537fe700 1 -- 172.21.15.46:0/3940105308 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fe158014070 con 0x7fe164100a70 2024-03-18T20:53:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.201+0000 7fe16259c700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe14c0a1390 0x7fe14c0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.202+0000 7fe175185b80 1 Processor -- start 2024-03-18T20:53:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.202+0000 7fe175185b80 1 -- start start 2024-03-18T20:53:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.202+0000 7fe16259c700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe14c0a1390 0x7fe14c0a3850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe154006010 tx=0x7fe15400b260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.202+0000 7fe175185b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0eb570 0x56238f0eb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.202+0000 7fe175185b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0ebe90 0x56238f0f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.202+0000 7fe175185b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0f8fc0 0x56238f0fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.202+0000 7fe175185b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238f045930 con 0x56238f0ebe90 2024-03-18T20:53:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.202+0000 7fe175185b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238f0432f0 con 0x56238f0eb570 2024-03-18T20:53:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.202+0000 7fe175185b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238f03ebe0 con 0x56238f0f8fc0 2024-03-18T20:53:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.202+0000 7fe161d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0ebe90 0x56238f0f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.203+0000 7fe162d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0f8fc0 0x56238f0fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.203+0000 7fe162d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0f8fc0 0x56238f0fb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:48718/0 (socket says 172.21.15.46:48718) 2024-03-18T20:53:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.203+0000 7fe16259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0eb570 0x56238f0eb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.203+0000 7fe162d9d700 1 -- 172.21.15.46:0/622823762 learned_addr learned my addr 172.21.15.46:0/622823762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.203+0000 7fe161d9b700 1 -- 172.21.15.46:0/622823762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0f8fc0 msgr2=0x56238f0fb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.203+0000 7fe161d9b700 1 --2- 172.21.15.46:0/622823762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0f8fc0 0x56238f0fb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.203+0000 7fe161d9b700 1 -- 172.21.15.46:0/622823762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0eb570 msgr2=0x56238f0eb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.203+0000 7fe161d9b700 1 --2- 172.21.15.46:0/622823762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0eb570 0x56238f0eb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.203+0000 7fe161d9b700 1 -- 172.21.15.46:0/622823762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56238eec0350 con 0x56238f0ebe90 2024-03-18T20:53:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.203+0000 7fe162d9d700 1 --2- 172.21.15.46:0/622823762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0f8fc0 0x56238f0fb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:53:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.203+0000 7fe161d9b700 1 --2- 172.21.15.46:0/622823762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0ebe90 0x56238f0f8a80 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe1580ca550 tx=0x7fe158099960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b3f1a358abf5c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.203+0000 7fe150ff9700 1 -- 172.21.15.46:0/622823762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15809dd40 con 0x56238f0ebe90 2024-03-18T20:53:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.204+0000 7fe150ff9700 1 -- 172.21.15.46:0/622823762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1580a2390 con 0x56238f0ebe90 2024-03-18T20:53:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.204+0000 7fe150ff9700 1 -- 172.21.15.46:0/622823762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15809a3e0 con 0x56238f0ebe90 2024-03-18T20:53:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.204+0000 7fe175185b80 1 -- 172.21.15.46:0/622823762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0ebe90 msgr2=0x56238f0f8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.204+0000 7fe175185b80 1 --2- 172.21.15.46:0/622823762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0ebe90 0x56238f0f8a80 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe1580ca550 tx=0x7fe158099960 comp rx=0 tx=0).stop 2024-03-18T20:53:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.204+0000 7fe175185b80 1 -- 172.21.15.46:0/622823762 shutdown_connections 2024-03-18T20:53:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.204+0000 7fe175185b80 1 --2- 172.21.15.46:0/622823762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0f8fc0 0x56238f0fb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.204+0000 7fe175185b80 1 --2- 172.21.15.46:0/622823762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0eb570 0x56238f0eb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.204+0000 7fe175185b80 1 --2- 172.21.15.46:0/622823762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0ebe90 0x56238f0f8a80 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.205+0000 7fe175185b80 1 -- 172.21.15.46:0/622823762 >> 172.21.15.46:0/622823762 conn(0x56238f0621c0 msgr2=0x56238f0e7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.205+0000 7fe175185b80 1 -- 172.21.15.46:0/622823762 shutdown_connections 2024-03-18T20:53:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.205+0000 7fe175185b80 1 -- 172.21.15.46:0/622823762 wait complete. 2024-03-18T20:53:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe175185b80 1 -- start start 2024-03-18T20:53:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe175185b80 1 Processor -- start 2024-03-18T20:53:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe175185b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0eb570 0x56238f0f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe175185b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0ebe90 0x56238f0f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe175185b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0f8fc0 0x56238f0f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe175185b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238efdad20 con 0x56238f0f8fc0 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe175185b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238efda9c0 con 0x56238f0ebe90 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe175185b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238eec2640 con 0x56238f0eb570 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe175185b80 10 client.0 _ll_register_callbacks cb 0x56238ee46340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe161d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0ebe90 0x56238f0f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe161d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0ebe90 0x56238f0f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33652/0 (socket says 172.21.15.46:33652) 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe161d9b700 1 -- 172.21.15.46:0/2244279659 learned_addr learned my addr 172.21.15.46:0/2244279659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe16259c700 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0eb570 0x56238f0f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe161d9b700 1 -- 172.21.15.46:0/2244279659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0eb570 msgr2=0x56238f0f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe161d9b700 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0eb570 0x56238f0f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe161d9b700 1 -- 172.21.15.46:0/2244279659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0f8fc0 msgr2=0x56238f0f40c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe161d9b700 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0f8fc0 0x56238f0f40c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.206+0000 7fe161d9b700 1 -- 172.21.15.46:0/2244279659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c009c60 con 0x56238f0ebe90 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe16259c700 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0eb570 0x56238f0f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:53:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe161d9b700 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0ebe90 0x56238f0f3b80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe158099ea0 tx=0x7fe1580107d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f285429e4ba9639 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe149ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe149ffb700 1 -- 172.21.15.46:0/2244279659 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1580b82a0 con 0x56238f0ebe90 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe149ffb700 1 -- 172.21.15.46:0/2244279659 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fe15809a170 con 0x56238f0ebe90 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe175185b80 1 -- 172.21.15.46:0/2244279659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56238f0df6c0 con 0x56238f0ebe90 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe149ffb700 1 -- 172.21.15.46:0/2244279659 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe15809a330 con 0x56238f0ebe90 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe175185b80 10 client.29984 fetch_fsmap learned FSMap version 78 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe175185b80 1 -- 172.21.15.46:0/2244279659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7fe1641a2cd0 con 0x56238f0ebe90 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe149ffb700 1 -- 172.21.15.46:0/2244279659 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56238f0df6c0 con 0x56238f0ebe90 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe149ffb700 1 -- 172.21.15.46:0/2244279659 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe158095d00 con 0x56238f0ebe90 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe175185b80 10 client.29984 fetch_fsmap finished waiting for FSMap version 78 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe175185b80 10 client.29984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.207+0000 7fe175185b80 20 client.29984 populate_metadata read hostname 'smithi046' 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.208+0000 7fe175185b80 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56238f10ef20 0x56238f111340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.208+0000 7fe175185b80 4 client.29984 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:53:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.208+0000 7fe175185b80 1 -- 172.21.15.46:0/2244279659 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56238f0376a0 con 0x56238f10ef20 2024-03-18T20:53:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.208+0000 7fe16259c700 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56238f10ef20 0x56238f111340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.208+0000 7fe149ffb700 10 client.29984 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.208+0000 7fe16259c700 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56238f10ef20 0x56238f111340 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=947bb3b238b8acf3 server_cookie=2e8ea43ff348ed8e in_seq=0 out_seq=0 2024-03-18T20:53:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.210+0000 7fe149ffb700 1 -- 172.21.15.46:0/2244279659 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56238f0376a0 con 0x56238f10ef20 2024-03-18T20:53:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.210+0000 7fe149ffb700 10 client.29984 handle_command_reply: tid=0 2024-03-18T20:53:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.210+0000 7fe175185b80 1 client.29984 shutdown 2024-03-18T20:53:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.210+0000 7fe175185b80 1 -- 172.21.15.46:0/2244279659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0ebe90 msgr2=0x56238f0f3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.210+0000 7fe175185b80 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0ebe90 0x56238f0f3b80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe158099ea0 tx=0x7fe1580107d0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.210+0000 7fe175185b80 1 -- 172.21.15.46:0/2244279659 shutdown_connections 2024-03-18T20:53:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.210+0000 7fe175185b80 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56238f10ef20 0x56238f111340 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.210+0000 7fe175185b80 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0f8fc0 0x56238f0f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.210+0000 7fe175185b80 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0eb570 0x56238f0f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.211+0000 7fe149ffb700 0 client.29984 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.211+0000 7fe175185b80 1 --2- 172.21.15.46:0/2244279659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0ebe90 0x56238f0f3b80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.211+0000 7fe175185b80 1 -- 172.21.15.46:0/2244279659 >> 172.21.15.46:0/2244279659 conn(0x56238f0621c0 msgr2=0x56238f0e7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.211+0000 7fe175185b80 1 -- 172.21.15.46:0/2244279659 shutdown_connections 2024-03-18T20:53:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.211+0000 7fe175185b80 1 -- 172.21.15.46:0/2244279659 wait complete. 2024-03-18T20:53:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.211+0000 7fe175185b80 20 client.29984 trim_cache size 0 max 16384 2024-03-18T20:53:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.228+0000 7fe175185b80 1 Processor -- start 2024-03-18T20:53:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.228+0000 7fe175185b80 1 -- start start 2024-03-18T20:53:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.228+0000 7fe175185b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0eb570 0x56238f0ee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.228+0000 7fe175185b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0ebe90 0x56238f1084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.228+0000 7fe175185b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0f8fc0 0x56238f0eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe175185b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238f0ec850 con 0x56238f0f8fc0 2024-03-18T20:53:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe175185b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238f0e8f70 con 0x56238f0eb570 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe175185b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238f0ece20 con 0x56238f0ebe90 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe162d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0f8fc0 0x56238f0eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe16259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0eb570 0x56238f0ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe162d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0f8fc0 0x56238f0eee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59318/0 (socket says 172.21.15.46:59318) 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe162d9d700 1 -- 172.21.15.46:0/1599379575 learned_addr learned my addr 172.21.15.46:0/1599379575 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe161d9b700 1 --2- 172.21.15.46:0/1599379575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0ebe90 0x56238f1084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe162d9d700 1 -- 172.21.15.46:0/1599379575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0ebe90 msgr2=0x56238f1084a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe162d9d700 1 --2- 172.21.15.46:0/1599379575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0ebe90 0x56238f1084a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe162d9d700 1 -- 172.21.15.46:0/1599379575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0eb570 msgr2=0x56238f0ee540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe162d9d700 1 --2- 172.21.15.46:0/1599379575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0eb570 0x56238f0ee540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe162d9d700 1 -- 172.21.15.46:0/1599379575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56238ee6cb00 con 0x56238f0f8fc0 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe16259c700 1 --2- 172.21.15.46:0/1599379575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0eb570 0x56238f0ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe162d9d700 1 --2- 172.21.15.46:0/1599379575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0f8fc0 0x56238f0eee50 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00a5c0 tx=0x7fe15c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94215abd631596c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.229+0000 7fe1517fa700 1 -- 172.21.15.46:0/1599379575 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c010040 con 0x56238f0f8fc0 2024-03-18T20:53:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.230+0000 7fe1517fa700 1 -- 172.21.15.46:0/1599379575 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe15c016b30 con 0x56238f0f8fc0 2024-03-18T20:53:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.230+0000 7fe1517fa700 1 -- 172.21.15.46:0/1599379575 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c015ca0 con 0x56238f0f8fc0 2024-03-18T20:53:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.230+0000 7fe175185b80 1 -- 172.21.15.46:0/1599379575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0f8fc0 msgr2=0x56238f0eee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.230+0000 7fe175185b80 1 --2- 172.21.15.46:0/1599379575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0f8fc0 0x56238f0eee50 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00a5c0 tx=0x7fe15c009650 comp rx=0 tx=0).stop 2024-03-18T20:53:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.230+0000 7fe175185b80 1 -- 172.21.15.46:0/1599379575 shutdown_connections 2024-03-18T20:53:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.230+0000 7fe175185b80 1 --2- 172.21.15.46:0/1599379575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0f8fc0 0x56238f0eee50 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.230+0000 7fe175185b80 1 --2- 172.21.15.46:0/1599379575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0eb570 0x56238f0ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.230+0000 7fe175185b80 1 --2- 172.21.15.46:0/1599379575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0ebe90 0x56238f1084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.230+0000 7fe175185b80 1 -- 172.21.15.46:0/1599379575 >> 172.21.15.46:0/1599379575 conn(0x56238f0621c0 msgr2=0x56238f0f3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.230+0000 7fe175185b80 1 -- 172.21.15.46:0/1599379575 shutdown_connections 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.230+0000 7fe175185b80 1 -- 172.21.15.46:0/1599379575 wait complete. 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe175185b80 1 -- start start 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe175185b80 1 Processor -- start 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe175185b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0eb570 0x56238f1060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe175185b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0ebe90 0x56238f106630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe175185b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0f8fc0 0x56238f161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe175185b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238f106d10 con 0x56238f0eb570 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe175185b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238f162290 con 0x56238f0ebe90 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe175185b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56238f1623d0 con 0x56238f0f8fc0 2024-03-18T20:53:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe175185b80 10 client.0 _ll_register_callbacks cb 0x7fe154005be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe16259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0eb570 0x56238f1060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe16259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0eb570 0x56238f1060f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59330/0 (socket says 172.21.15.46:59330) 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe16259c700 1 -- 172.21.15.46:0/3272715885 learned_addr learned my addr 172.21.15.46:0/3272715885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.231+0000 7fe161d9b700 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0ebe90 0x56238f106630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe162d9d700 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0f8fc0 0x56238f161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe16259c700 1 -- 172.21.15.46:0/3272715885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0f8fc0 msgr2=0x56238f161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe16259c700 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0f8fc0 0x56238f161d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe16259c700 1 -- 172.21.15.46:0/3272715885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0ebe90 msgr2=0x56238f106630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe16259c700 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0ebe90 0x56238f106630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe16259c700 1 -- 172.21.15.46:0/3272715885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16410d320 con 0x56238f0eb570 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe16259c700 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0eb570 0x56238f1060f0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fe154009990 tx=0x7fe154009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c97bbd90910c7c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe149ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe149ffb700 1 -- 172.21.15.46:0/3272715885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154035d60 con 0x56238f0eb570 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe149ffb700 1 -- 172.21.15.46:0/3272715885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fe154035040 con 0x56238f0eb570 2024-03-18T20:53:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe175185b80 1 -- 172.21.15.46:0/3272715885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1580a2390 con 0x56238f0eb570 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe149ffb700 1 -- 172.21.15.46:0/3272715885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe154035200 con 0x56238f0eb570 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe175185b80 10 client.20421 fetch_fsmap learned FSMap version 78 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.232+0000 7fe175185b80 1 -- 172.21.15.46:0/3272715885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7fe15c009e50 con 0x56238f0eb570 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.233+0000 7fe149ffb700 1 -- 172.21.15.46:0/3272715885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1580a2390 con 0x56238f0eb570 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.233+0000 7fe149ffb700 1 -- 172.21.15.46:0/3272715885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 78) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe15403b020 con 0x56238f0eb570 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.233+0000 7fe175185b80 10 client.20421 fetch_fsmap finished waiting for FSMap version 78 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.233+0000 7fe175185b80 10 client.20421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.233+0000 7fe175185b80 20 client.20421 populate_metadata read hostname 'smithi046' 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.233+0000 7fe175185b80 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56238f163ae0 0x56238f165f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.233+0000 7fe175185b80 4 client.20421 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.233+0000 7fe175185b80 1 -- 172.21.15.46:0/3272715885 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56238ee8dc90 con 0x56238f163ae0 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.233+0000 7fe161d9b700 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56238f163ae0 0x56238f165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.235+0000 7fe161d9b700 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56238f163ae0 0x56238f165f00 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d617f9cc27dd21 server_cookie=3ae9b1159d2c3217 in_seq=0 out_seq=0 2024-03-18T20:53:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:15.235+0000 7fe149ffb700 10 client.20421 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:15.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 195364 41.70 MB/sec execute 111 sec latency 8014.709 ms 2024-03-18T20:53:16.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 195870 41.47 MB/sec execute 112 sec latency 8116.482 ms 2024-03-18T20:53:16.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:16 smithi073 ceph-mon[47384]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.6 MiB/s wr, 795 op/s 2024-03-18T20:53:16.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:16 smithi155 ceph-mon[48344]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.6 MiB/s wr, 795 op/s 2024-03-18T20:53:16.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:16 smithi046 ceph-mon[43495]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.6 MiB/s wr, 795 op/s 2024-03-18T20:53:17.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 196641 41.52 MB/sec execute 113 sec latency 7625.846 ms 2024-03-18T20:53:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.085+0000 7fe149ffb700 1 -- 172.21.15.46:0/3272715885 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56238ee8dc90 con 0x56238f163ae0 2024-03-18T20:53:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.085+0000 7fe149ffb700 10 client.20421 handle_command_reply: tid=0 2024-03-18T20:53:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.085+0000 7fe175185b80 1 client.20421 shutdown 2024-03-18T20:53:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.086+0000 7fe175185b80 1 -- 172.21.15.46:0/3272715885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0eb570 msgr2=0x56238f1060f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.086+0000 7fe175185b80 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0eb570 0x56238f1060f0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fe154009990 tx=0x7fe154009ca0 comp rx=0 tx=0).stop 2024-03-18T20:53:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.086+0000 7fe175185b80 1 -- 172.21.15.46:0/3272715885 shutdown_connections 2024-03-18T20:53:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.086+0000 7fe175185b80 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56238f163ae0 0x56238f165f00 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.086+0000 7fe175185b80 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56238f0eb570 0x56238f1060f0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.086+0000 7fe175185b80 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56238f0f8fc0 0x56238f161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.086+0000 7fe175185b80 1 --2- 172.21.15.46:0/3272715885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56238f0ebe90 0x56238f106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.086+0000 7fe175185b80 1 -- 172.21.15.46:0/3272715885 >> 172.21.15.46:0/3272715885 conn(0x56238f0621c0 msgr2=0x56238f0eee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.086+0000 7fe149ffb700 0 client.20421 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.088+0000 7fe175185b80 1 -- 172.21.15.46:0/3272715885 shutdown_connections 2024-03-18T20:53:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.088+0000 7fe175185b80 1 -- 172.21.15.46:0/3272715885 wait complete. 2024-03-18T20:53:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.089+0000 7fe175185b80 20 client.20421 trim_cache size 0 max 16384 2024-03-18T20:53:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:53:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:53:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:53:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.089+0000 7fe168ff0700 1 -- 172.21.15.46:0/3940105308 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe14c0a1390 msgr2=0x7fe14c0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.089+0000 7fe168ff0700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe14c0a1390 0x7fe14c0a3850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe154006010 tx=0x7fe15400b260 comp rx=0 tx=0).stop 2024-03-18T20:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.089+0000 7fe168ff0700 1 -- 172.21.15.46:0/3940105308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe164100a70 msgr2=0x7fe1641964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.089+0000 7fe168ff0700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe164100a70 0x7fe1641964a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe15800e9a0 tx=0x7fe15800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.092+0000 7fe168ff0700 1 -- 172.21.15.46:0/3940105308 shutdown_connections 2024-03-18T20:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.092+0000 7fe168ff0700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe14c0a1390 0x7fe14c0a3850 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.092+0000 7fe168ff0700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1641000b0 0x7fe164195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.092+0000 7fe168ff0700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe164105d50 0x7fe16419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.092+0000 7fe168ff0700 1 --2- 172.21.15.46:0/3940105308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe164100a70 0x7fe1641964a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.092+0000 7fe168ff0700 1 -- 172.21.15.46:0/3940105308 >> 172.21.15.46:0/3940105308 conn(0x7fe164076a20 msgr2=0x7fe1640fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.092+0000 7fe168ff0700 1 -- 172.21.15.46:0/3940105308 shutdown_connections 2024-03-18T20:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:18.092+0000 7fe168ff0700 1 -- 172.21.15.46:0/3940105308 wait complete. 2024-03-18T20:53:18.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 197243 41.43 MB/sec execute 114 sec latency 8625.887 ms 2024-03-18T20:53:18.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:18 smithi073 ceph-mon[47384]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.6 MiB/s wr, 795 op/s 2024-03-18T20:53:18.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:18 smithi155 ceph-mon[48344]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.6 MiB/s wr, 795 op/s 2024-03-18T20:53:18.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:18 smithi046 ceph-mon[43495]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.6 MiB/s wr, 795 op/s 2024-03-18T20:53:19.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 197906 41.39 MB/sec execute 115 sec latency 9533.957 ms 2024-03-18T20:53:20.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 198755 41.42 MB/sec execute 116 sec latency 77.964 ms 2024-03-18T20:53:20.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:20 smithi073 ceph-mon[47384]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.0 MiB/s wr, 755 op/s 2024-03-18T20:53:20.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:20 smithi073 ceph-mon[47384]: osdmap e104: 12 total, 12 up, 12 in 2024-03-18T20:53:20.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:20 smithi155 ceph-mon[48344]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.0 MiB/s wr, 755 op/s 2024-03-18T20:53:20.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:20 smithi155 ceph-mon[48344]: osdmap e104: 12 total, 12 up, 12 in 2024-03-18T20:53:20.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:20 smithi046 ceph-mon[43495]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 8.0 MiB/s wr, 755 op/s 2024-03-18T20:53:20.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:20 smithi046 ceph-mon[43495]: osdmap e104: 12 total, 12 up, 12 in 2024-03-18T20:53:21.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 199394 41.37 MB/sec execute 117 sec latency 88.903 ms 2024-03-18T20:53:22.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 200085 41.35 MB/sec execute 118 sec latency 97.792 ms 2024-03-18T20:53:22.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:22 smithi073 ceph-mon[47384]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 588 op/s 2024-03-18T20:53:22.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:22 smithi155 ceph-mon[48344]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 588 op/s 2024-03-18T20:53:22.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:22 smithi046 ceph-mon[43495]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 588 op/s 2024-03-18T20:53:23.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 200645 41.25 MB/sec execute 119 sec latency 104.745 ms 2024-03-18T20:53:24.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 201379 41.19 MB/sec execute 120 sec latency 123.740 ms 2024-03-18T20:53:24.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:24 smithi073 ceph-mon[47384]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 536 op/s 2024-03-18T20:53:24.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:24 smithi155 ceph-mon[48344]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 536 op/s 2024-03-18T20:53:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:24 smithi046 ceph-mon[43495]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.5 MiB/s wr, 536 op/s 2024-03-18T20:53:25.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 202252 41.24 MB/sec execute 121 sec latency 70.052 ms 2024-03-18T20:53:26.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 202996 41.28 MB/sec execute 122 sec latency 85.876 ms 2024-03-18T20:53:26.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:26 smithi073 ceph-mon[47384]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 642 op/s 2024-03-18T20:53:26.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:26 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:26.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:26 smithi155 ceph-mon[48344]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 642 op/s 2024-03-18T20:53:26.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:26 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:26.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:26 smithi046 ceph-mon[43495]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 642 op/s 2024-03-18T20:53:26.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:26 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:27.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 203594 41.21 MB/sec execute 123 sec latency 72.585 ms 2024-03-18T20:53:27.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:27 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:53:27.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:27 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:53:27.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:27 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:53:28.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 204317 41.21 MB/sec execute 124 sec latency 116.805 ms 2024-03-18T20:53:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:28 smithi073 ceph-mon[47384]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 642 op/s 2024-03-18T20:53:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:28 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:53:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:28 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:53:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:28 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:53:28.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:28 smithi155 ceph-mon[48344]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 642 op/s 2024-03-18T20:53:28.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:28 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:53:28.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:28 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:53:28.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:28 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:53:28.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:28 smithi046 ceph-mon[43495]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 642 op/s 2024-03-18T20:53:28.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:28 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:53:28.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:28 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:53:28.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:28 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:53:29.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 205039 41.13 MB/sec execute 125 sec latency 78.606 ms 2024-03-18T20:53:30.381 INFO:tasks.workunit.client.0.smithi046.stdout: 10 205802 41.14 MB/sec execute 126 sec latency 87.650 ms 2024-03-18T20:53:30.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:30 smithi073 ceph-mon[47384]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.6 MiB/s wr, 563 op/s 2024-03-18T20:53:30.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:30 smithi155 ceph-mon[48344]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.6 MiB/s wr, 563 op/s 2024-03-18T20:53:30.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:30 smithi046 ceph-mon[43495]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.6 MiB/s wr, 563 op/s 2024-03-18T20:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.274+0000 7f84c13ca700 1 -- 172.21.15.46:0/1592400211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc1007d0 msgr2=0x7f84bc104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.274+0000 7f84c13ca700 1 --2- 172.21.15.46:0/1592400211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc1007d0 0x7f84bc104810 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009990 tx=0x7f84ac009ca0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.274+0000 7f84c13ca700 1 -- 172.21.15.46:0/1592400211 shutdown_connections 2024-03-18T20:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.274+0000 7f84c13ca700 1 --2- 172.21.15.46:0/1592400211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84bc0ffe20 0x7f84bc100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.274+0000 7f84c13ca700 1 --2- 172.21.15.46:0/1592400211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84bc104f40 0x7f84bc10cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.274+0000 7f84c13ca700 1 --2- 172.21.15.46:0/1592400211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc1007d0 0x7f84bc104810 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.274+0000 7f84c13ca700 1 -- 172.21.15.46:0/1592400211 >> 172.21.15.46:0/1592400211 conn(0x7f84bc076d50 msgr2=0x7f84bc077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.274+0000 7f84c13ca700 1 -- 172.21.15.46:0/1592400211 shutdown_connections 2024-03-18T20:53:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.274+0000 7f84c13ca700 1 -- 172.21.15.46:0/1592400211 wait complete. 2024-03-18T20:53:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.276+0000 7f84c13ca700 1 Processor -- start 2024-03-18T20:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.276+0000 7f84c13ca700 1 -- start start 2024-03-18T20:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.276+0000 7f84c13ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84bc0ffe20 0x7f84bc197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.276+0000 7f84c13ca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84bc104f40 0x7f84bc197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.276+0000 7f84c13ca700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc19bb20 0x7f84bc19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.276+0000 7f84c13ca700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc19c5a0 con 0x7f84bc0ffe20 2024-03-18T20:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.276+0000 7f84c13ca700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc19c6e0 con 0x7f84bc104f40 2024-03-18T20:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.276+0000 7f84c13ca700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc19c850 con 0x7f84bc19bb20 2024-03-18T20:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.277+0000 7f84ba7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84bc104f40 0x7f84bc197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.277+0000 7f84ba7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84bc104f40 0x7f84bc197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:39810/0 (socket says 172.21.15.46:39810) 2024-03-18T20:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.277+0000 7f84ba7fc700 1 -- 172.21.15.46:0/595962802 learned_addr learned my addr 172.21.15.46:0/595962802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.277+0000 7f84bb7fe700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc19bb20 0x7f84bc19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.277+0000 7f84ba7fc700 1 -- 172.21.15.46:0/595962802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc19bb20 msgr2=0x7f84bc19bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.277+0000 7f84ba7fc700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc19bb20 0x7f84bc19bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.277+0000 7f84ba7fc700 1 -- 172.21.15.46:0/595962802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84bc0ffe20 msgr2=0x7f84bc197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.277+0000 7f84ba7fc700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84bc0ffe20 0x7f84bc197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.277+0000 7f84ba7fc700 1 -- 172.21.15.46:0/595962802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac0095f0 con 0x7f84bc104f40 2024-03-18T20:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.277+0000 7f84ba7fc700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84bc104f40 0x7f84bc197780 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f84ac000c00 tx=0x7f84ac004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.278+0000 7f84a3fff700 1 -- 172.21.15.46:0/595962802 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac01d070 con 0x7f84bc104f40 2024-03-18T20:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.278+0000 7f84a3fff700 1 -- 172.21.15.46:0/595962802 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ac005030 con 0x7f84bc104f40 2024-03-18T20:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.278+0000 7f84a3fff700 1 -- 172.21.15.46:0/595962802 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac004a40 con 0x7f84bc104f40 2024-03-18T20:53:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.278+0000 7f84c13ca700 1 -- 172.21.15.46:0/595962802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc06a7d0 con 0x7f84bc104f40 2024-03-18T20:53:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.278+0000 7f84c13ca700 1 -- 172.21.15.46:0/595962802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc06ab70 con 0x7f84bc104f40 2024-03-18T20:53:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.281+0000 7f84a3fff700 1 -- 172.21.15.46:0/595962802 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f84ac004c20 con 0x7f84bc104f40 2024-03-18T20:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.282+0000 7f84a3fff700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84a40a1310 0x7f84a40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.283+0000 7f84a3fff700 1 -- 172.21.15.46:0/595962802 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f84ac0cde40 con 0x7f84bc104f40 2024-03-18T20:53:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.283+0000 7f84baffd700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84a40a1310 0x7f84a40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.283+0000 7f84cd55fb80 1 Processor -- start 2024-03-18T20:53:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.284+0000 7f84cd55fb80 1 -- start start 2024-03-18T20:53:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.284+0000 7f84cd55fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff2f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.284+0000 7f84cd55fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff2ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.284+0000 7f84cd55fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff45840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.284+0000 7f84cd55fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5fe89930 con 0x55ae5ff2fab0 2024-03-18T20:53:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.284+0000 7f84cd55fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5fe872f0 con 0x55ae5ff311c0 2024-03-18T20:53:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.284+0000 7f84cd55fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5fe82be0 con 0x55ae5ff43450 2024-03-18T20:53:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.284+0000 7f84bb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff45840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.284+0000 7f84bb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff45840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:42992/0 (socket says 172.21.15.46:42992) 2024-03-18T20:53:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.284+0000 7f84bb7fe700 1 -- 172.21.15.46:0/4111756723 learned_addr learned my addr 172.21.15.46:0/4111756723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.285+0000 7f84ba7fc700 1 --2- 172.21.15.46:0/4111756723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff2ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.285+0000 7f84baffd700 1 --2- 172.21.15.46:0/4111756723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff2f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.285+0000 7f84bb7fe700 1 -- 172.21.15.46:0/4111756723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 msgr2=0x55ae5ff2f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.285+0000 7f84bb7fe700 1 --2- 172.21.15.46:0/4111756723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff2f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.285+0000 7f84bb7fe700 1 -- 172.21.15.46:0/4111756723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 msgr2=0x55ae5ff2ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.285+0000 7f84bb7fe700 1 --2- 172.21.15.46:0/4111756723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff2ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.285+0000 7f84bb7fe700 1 -- 172.21.15.46:0/4111756723 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae5fd04350 con 0x55ae5ff43450 2024-03-18T20:53:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.285+0000 7f84baffd700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84a40a1310 0x7f84a40a37d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f84b00097b0 tx=0x7f84b0006d20 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.285+0000 7f84bb7fe700 1 --2- 172.21.15.46:0/4111756723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff45840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f84b4011680 tx=0x7f84b4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43e293909136f43 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.285+0000 7f84a17fa700 1 -- 172.21.15.46:0/4111756723 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b400e040 con 0x55ae5ff43450 2024-03-18T20:53:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.286+0000 7f84a17fa700 1 -- 172.21.15.46:0/4111756723 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84b400f040 con 0x55ae5ff43450 2024-03-18T20:53:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.286+0000 7f84a17fa700 1 -- 172.21.15.46:0/4111756723 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b40048c0 con 0x55ae5ff43450 2024-03-18T20:53:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.286+0000 7f84cd55fb80 1 -- 172.21.15.46:0/4111756723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 msgr2=0x55ae5ff45840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.286+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/4111756723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff45840 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f84b4011680 tx=0x7f84b4009920 comp rx=0 tx=0).stop 2024-03-18T20:53:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.287+0000 7f84cd55fb80 1 -- 172.21.15.46:0/4111756723 shutdown_connections 2024-03-18T20:53:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.287+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/4111756723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff45840 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.287+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/4111756723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff2f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.287+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/4111756723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff2ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.287+0000 7f84cd55fb80 1 -- 172.21.15.46:0/4111756723 >> 172.21.15.46:0/4111756723 conn(0x55ae5fea61c0 msgr2=0x55ae5ff2ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.287+0000 7f84cd55fb80 1 -- 172.21.15.46:0/4111756723 shutdown_connections 2024-03-18T20:53:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.287+0000 7f84cd55fb80 1 -- 172.21.15.46:0/4111756723 wait complete. 2024-03-18T20:53:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.287+0000 7f84cd55fb80 1 -- start start 2024-03-18T20:53:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.287+0000 7f84cd55fb80 1 Processor -- start 2024-03-18T20:53:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.288+0000 7f84cd55fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff3cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.288+0000 7f84cd55fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff3d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.288+0000 7f84cd55fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff42890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.288+0000 7f84cd55fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5fe1ed20 con 0x55ae5ff2fab0 2024-03-18T20:53:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.288+0000 7f84cd55fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5fe1e9c0 con 0x55ae5ff311c0 2024-03-18T20:53:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.288+0000 7f84cd55fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5fd06640 con 0x55ae5ff43450 2024-03-18T20:53:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.288+0000 7f84cd55fb80 10 client.0 _ll_register_callbacks cb 0x55ae5fc8a340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.288+0000 7f84bb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff42890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.288+0000 7f84bb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff42890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43006/0 (socket says 172.21.15.46:43006) 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.288+0000 7f84bb7fe700 1 -- 172.21.15.46:0/291905641 learned_addr learned my addr 172.21.15.46:0/291905641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.289+0000 7f84bb7fe700 1 -- 172.21.15.46:0/291905641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 msgr2=0x55ae5ff3d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.289+0000 7f84bb7fe700 1 --2- 172.21.15.46:0/291905641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff3d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.289+0000 7f84bb7fe700 1 -- 172.21.15.46:0/291905641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 msgr2=0x55ae5ff3cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.289+0000 7f84bb7fe700 1 --2- 172.21.15.46:0/291905641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff3cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.289+0000 7f84bb7fe700 1 -- 172.21.15.46:0/291905641 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b4011330 con 0x55ae5ff43450 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.289+0000 7f84bb7fe700 1 --2- 172.21.15.46:0/291905641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff42890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f84b4012010 tx=0x7f84b4003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa1b1fc70d1cc033 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.289+0000 7f849eff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.289+0000 7f849eff5700 1 -- 172.21.15.46:0/291905641 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b400e030 con 0x55ae5ff43450 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.289+0000 7f849eff5700 1 -- 172.21.15.46:0/291905641 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f84b40048c0 con 0x55ae5ff43450 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.289+0000 7f84cd55fb80 1 -- 172.21.15.46:0/291905641 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae5ff236c0 con 0x55ae5ff43450 2024-03-18T20:53:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.289+0000 7f849eff5700 1 -- 172.21.15.46:0/291905641 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84b40041e0 con 0x55ae5ff43450 2024-03-18T20:53:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.290+0000 7f84cd55fb80 10 client.30097 fetch_fsmap learned FSMap version 80 2024-03-18T20:53:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.290+0000 7f84cd55fb80 1 -- 172.21.15.46:0/291905641 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55ae5fcb0b00 con 0x55ae5ff43450 2024-03-18T20:53:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.290+0000 7f849eff5700 1 -- 172.21.15.46:0/291905641 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae5ff236c0 con 0x55ae5ff43450 2024-03-18T20:53:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.290+0000 7f849eff5700 1 -- 172.21.15.46:0/291905641 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f84b4024020 con 0x55ae5ff43450 2024-03-18T20:53:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.290+0000 7f84cd55fb80 10 client.30097 fetch_fsmap finished waiting for FSMap version 80 2024-03-18T20:53:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.290+0000 7f84cd55fb80 10 client.30097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.290+0000 7f84cd55fb80 20 client.30097 populate_metadata read hostname 'smithi046' 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.290+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/291905641 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae5ff5b8d0 0x55ae5ff5dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.290+0000 7f84cd55fb80 4 client.30097 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.290+0000 7f84cd55fb80 1 -- 172.21.15.46:0/291905641 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae5fe7b6a0 con 0x55ae5ff5b8d0 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.291+0000 7f84baffd700 1 --2- 172.21.15.46:0/291905641 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae5ff5b8d0 0x55ae5ff5dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.291+0000 7f84baffd700 1 --2- 172.21.15.46:0/291905641 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae5ff5b8d0 0x55ae5ff5dcc0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e09ee15e3b4c6e2 server_cookie=d7075ef2edbb384c in_seq=0 out_seq=0 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.291+0000 7f849eff5700 10 client.30097 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.292+0000 7f849eff5700 1 -- 172.21.15.46:0/291905641 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ae5fe7b6a0 con 0x55ae5ff5b8d0 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.292+0000 7f849eff5700 10 client.30097 handle_command_reply: tid=0 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.292+0000 7f84cd55fb80 1 client.30097 shutdown 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.292+0000 7f84cd55fb80 1 -- 172.21.15.46:0/291905641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 msgr2=0x55ae5ff42890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.292+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/291905641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff42890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f84b4012010 tx=0x7f84b4003ce0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.292+0000 7f84cd55fb80 1 -- 172.21.15.46:0/291905641 shutdown_connections 2024-03-18T20:53:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.292+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/291905641 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae5ff5b8d0 0x55ae5ff5dcc0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.292+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/291905641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff3cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.292+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/291905641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff42890 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.292+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/291905641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff3d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.292+0000 7f84cd55fb80 1 -- 172.21.15.46:0/291905641 >> 172.21.15.46:0/291905641 conn(0x55ae5fea61c0 msgr2=0x55ae5ff2b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.293+0000 7f849eff5700 0 client.30097 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.293+0000 7f84cd55fb80 1 -- 172.21.15.46:0/291905641 shutdown_connections 2024-03-18T20:53:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.293+0000 7f84cd55fb80 1 -- 172.21.15.46:0/291905641 wait complete. 2024-03-18T20:53:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.293+0000 7f84cd55fb80 20 client.30097 trim_cache size 0 max 16384 2024-03-18T20:53:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.313+0000 7f84cd55fb80 1 Processor -- start 2024-03-18T20:53:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.313+0000 7f84cd55fb80 1 -- start start 2024-03-18T20:53:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.313+0000 7f84cd55fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff5dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.313+0000 7f84cd55fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff377b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.313+0000 7f84cd55fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff35f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.313+0000 7f84cd55fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5ff37cf0 con 0x55ae5ff2fab0 2024-03-18T20:53:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.313+0000 7f84cd55fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5ff2c760 con 0x55ae5ff311c0 2024-03-18T20:53:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.313+0000 7f84cd55fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5ff35c10 con 0x55ae5ff43450 2024-03-18T20:53:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.313+0000 7f84ba7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.314+0000 7f84bb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff35f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.314+0000 7f84bb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff35f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43010/0 (socket says 172.21.15.46:43010) 2024-03-18T20:53:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.314+0000 7f84bb7fe700 1 -- 172.21.15.46:0/349497337 learned_addr learned my addr 172.21.15.46:0/349497337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.314+0000 7f84baffd700 1 --2- 172.21.15.46:0/349497337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff5dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.314+0000 7f84ba7fc700 1 -- 172.21.15.46:0/349497337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 msgr2=0x55ae5ff35f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.314+0000 7f84ba7fc700 1 --2- 172.21.15.46:0/349497337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff35f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.314+0000 7f84ba7fc700 1 -- 172.21.15.46:0/349497337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 msgr2=0x55ae5ff5dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.314+0000 7f84ba7fc700 1 --2- 172.21.15.46:0/349497337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff5dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.314+0000 7f84ba7fc700 1 -- 172.21.15.46:0/349497337 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae5fe48e40 con 0x55ae5ff311c0 2024-03-18T20:53:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.315+0000 7f84ba7fc700 1 --2- 172.21.15.46:0/349497337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff377b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0cb570 tx=0x7f84ac09ab00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30db450bd3b7cee2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.315+0000 7f84a1ffb700 1 -- 172.21.15.46:0/349497337 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac0a10c0 con 0x55ae5ff311c0 2024-03-18T20:53:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.315+0000 7f84a1ffb700 1 -- 172.21.15.46:0/349497337 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ac09b6c0 con 0x55ae5ff311c0 2024-03-18T20:53:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.315+0000 7f84a1ffb700 1 -- 172.21.15.46:0/349497337 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac09a4b0 con 0x55ae5ff311c0 2024-03-18T20:53:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.316+0000 7f84cd55fb80 1 -- 172.21.15.46:0/349497337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 msgr2=0x55ae5ff377b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.316+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/349497337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff377b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0cb570 tx=0x7f84ac09ab00 comp rx=0 tx=0).stop 2024-03-18T20:53:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.316+0000 7f84cd55fb80 1 -- 172.21.15.46:0/349497337 shutdown_connections 2024-03-18T20:53:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.316+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/349497337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff5dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.316+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/349497337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff35f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.316+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/349497337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff377b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.316+0000 7f84cd55fb80 1 -- 172.21.15.46:0/349497337 >> 172.21.15.46:0/349497337 conn(0x55ae5fea61c0 msgr2=0x55ae5ff62660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.316+0000 7f84cd55fb80 1 -- 172.21.15.46:0/349497337 shutdown_connections 2024-03-18T20:53:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.316+0000 7f84cd55fb80 1 -- 172.21.15.46:0/349497337 wait complete. 2024-03-18T20:53:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.316+0000 7f84cd55fb80 1 -- start start 2024-03-18T20:53:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.317+0000 7f84cd55fb80 1 Processor -- start 2024-03-18T20:53:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.317+0000 7f84cd55fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff40090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.317+0000 7f84cd55fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.317+0000 7f84cd55fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff40b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.317+0000 7f84cd55fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5ffae930 con 0x55ae5ff2fab0 2024-03-18T20:53:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.317+0000 7f84cd55fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5ffaeaa0 con 0x55ae5ff311c0 2024-03-18T20:53:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.317+0000 7f84cd55fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5ffaec60 con 0x55ae5ff43450 2024-03-18T20:53:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.317+0000 7f84cd55fb80 10 client.0 _ll_register_callbacks cb 0x7f84b4011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:53:31.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.317+0000 7f84bb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff40b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.317+0000 7f84bb7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff40b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43026/0 (socket says 172.21.15.46:43026) 2024-03-18T20:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.317+0000 7f84bb7fe700 1 -- 172.21.15.46:0/3047237343 learned_addr learned my addr 172.21.15.46:0/3047237343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.317+0000 7f84ba7fc700 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.318+0000 7f84baffd700 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff40090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.318+0000 7f84ba7fc700 1 -- 172.21.15.46:0/3047237343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 msgr2=0x55ae5ff40b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.318+0000 7f84ba7fc700 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff40b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.318+0000 7f84ba7fc700 1 -- 172.21.15.46:0/3047237343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 msgr2=0x55ae5ff40090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.318+0000 7f84ba7fc700 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff40090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.318+0000 7f84ba7fc700 1 -- 172.21.15.46:0/3047237343 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc06ab70 con 0x55ae5ff311c0 2024-03-18T20:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.318+0000 7f84ba7fc700 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff405d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0058f0 tx=0x7f84ac009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32c7b66360105aa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:31.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.318+0000 7f849eff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.319+0000 7f849eff5700 1 -- 172.21.15.46:0/3047237343 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac09afc0 con 0x55ae5ff311c0 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.319+0000 7f849eff5700 1 -- 172.21.15.46:0/3047237343 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f84ac09b220 con 0x55ae5ff311c0 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.319+0000 7f84cd55fb80 1 -- 172.21.15.46:0/3047237343 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84b400f040 con 0x55ae5ff311c0 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.320+0000 7f849eff5700 1 -- 172.21.15.46:0/3047237343 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84ac0b92d0 con 0x55ae5ff311c0 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.320+0000 7f84cd55fb80 10 client.30005 fetch_fsmap learned FSMap version 80 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.320+0000 7f84cd55fb80 1 -- 172.21.15.46:0/3047237343 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) v3 -- 0x55ae5fd08c70 con 0x55ae5ff311c0 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.320+0000 7f849eff5700 1 -- 172.21.15.46:0/3047237343 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84b400f040 con 0x55ae5ff311c0 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.320+0000 7f849eff5700 1 -- 172.21.15.46:0/3047237343 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 80) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f84ac0a2f20 con 0x55ae5ff311c0 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.320+0000 7f84cd55fb80 10 client.30005 fetch_fsmap finished waiting for FSMap version 80 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.320+0000 7f84cd55fb80 10 client.30005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.320+0000 7f84cd55fb80 20 client.30005 populate_metadata read hostname 'smithi046' 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.320+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae5ffaff00 0x55ae5ffb22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:31.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.320+0000 7f84cd55fb80 4 client.30005 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:53:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.320+0000 7f84cd55fb80 1 -- 172.21.15.46:0/3047237343 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ae5fcd1c90 con 0x55ae5ffaff00 2024-03-18T20:53:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.321+0000 7f84baffd700 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae5ffaff00 0x55ae5ffb22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.321+0000 7f849eff5700 10 client.30005 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:31.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.322+0000 7f84baffd700 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae5ffaff00 0x55ae5ffb22f0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f203b345d9d4bd8 server_cookie=7ed03657b40087c3 in_seq=0 out_seq=0 2024-03-18T20:53:31.384 INFO:tasks.workunit.client.0.smithi046.stdout: 10 206457 41.11 MB/sec execute 127 sec latency 118.613 ms 2024-03-18T20:53:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f849eff5700 1 -- 172.21.15.46:0/3047237343 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ae5fcd1c90 con 0x55ae5ffaff00 2024-03-18T20:53:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f849eff5700 10 client.30005 handle_command_reply: tid=0 2024-03-18T20:53:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f84cd55fb80 1 client.30005 shutdown 2024-03-18T20:53:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f84cd55fb80 1 -- 172.21.15.46:0/3047237343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 msgr2=0x55ae5ff405d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff405d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0058f0 tx=0x7f84ac009ca0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f84cd55fb80 1 -- 172.21.15.46:0/3047237343 shutdown_connections 2024-03-18T20:53:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae5ffaff00 0x55ae5ffb22f0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae5ff2fab0 0x55ae5ff40090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae5ff43450 0x55ae5ff40b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f84cd55fb80 1 --2- 172.21.15.46:0/3047237343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae5ff311c0 0x55ae5ff405d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f84cd55fb80 1 -- 172.21.15.46:0/3047237343 >> 172.21.15.46:0/3047237343 conn(0x55ae5fea61c0 msgr2=0x55ae5ff35eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f849eff5700 0 client.30005 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f84cd55fb80 1 -- 172.21.15.46:0/3047237343 shutdown_connections 2024-03-18T20:53:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.597+0000 7f84cd55fb80 1 -- 172.21.15.46:0/3047237343 wait complete. 2024-03-18T20:53:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.599+0000 7f84cd55fb80 20 client.30005 trim_cache size 0 max 16384 2024-03-18T20:53:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:53:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:53:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:53:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:53:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.601+0000 7f84c13ca700 1 -- 172.21.15.46:0/595962802 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84a40a1310 msgr2=0x7f84a40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.601+0000 7f84c13ca700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84a40a1310 0x7f84a40a37d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f84b00097b0 tx=0x7f84b0006d20 comp rx=0 tx=0).stop 2024-03-18T20:53:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.601+0000 7f84c13ca700 1 -- 172.21.15.46:0/595962802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84bc104f40 msgr2=0x7f84bc197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.601+0000 7f84c13ca700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84bc104f40 0x7f84bc197780 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f84ac000c00 tx=0x7f84ac004730 comp rx=0 tx=0).stop 2024-03-18T20:53:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.601+0000 7f84c13ca700 1 -- 172.21.15.46:0/595962802 shutdown_connections 2024-03-18T20:53:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.601+0000 7f84c13ca700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84a40a1310 0x7f84a40a37d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.601+0000 7f84c13ca700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84bc0ffe20 0x7f84bc197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.601+0000 7f84c13ca700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84bc19bb20 0x7f84bc19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.601+0000 7f84c13ca700 1 --2- 172.21.15.46:0/595962802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84bc104f40 0x7f84bc197780 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.601+0000 7f84c13ca700 1 -- 172.21.15.46:0/595962802 >> 172.21.15.46:0/595962802 conn(0x7f84bc076d50 msgr2=0x7f84bc1060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.601+0000 7f84c13ca700 1 -- 172.21.15.46:0/595962802 shutdown_connections 2024-03-18T20:53:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:31.601+0000 7f84c13ca700 1 -- 172.21.15.46:0/595962802 wait complete. 2024-03-18T20:53:31.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:31 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:31.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:31 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:31.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:31 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:32.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 207009 41.04 MB/sec execute 128 sec latency 208.822 ms 2024-03-18T20:53:32.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:32 smithi155 ceph-mon[48344]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.8 MiB/s wr, 503 op/s 2024-03-18T20:53:32.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:32 smithi046 ceph-mon[43495]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.8 MiB/s wr, 503 op/s 2024-03-18T20:53:32.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:32 smithi073 ceph-mon[47384]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.8 MiB/s wr, 503 op/s 2024-03-18T20:53:33.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 207686 40.96 MB/sec execute 129 sec latency 86.266 ms 2024-03-18T20:53:33.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:33 smithi155 ceph-mon[48344]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.9 MiB/s wr, 615 op/s 2024-03-18T20:53:33.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:33 smithi046 ceph-mon[43495]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.9 MiB/s wr, 615 op/s 2024-03-18T20:53:33.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:33 smithi073 ceph-mon[47384]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.9 MiB/s wr, 615 op/s 2024-03-18T20:53:34.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 208383 40.96 MB/sec execute 130 sec latency 76.017 ms 2024-03-18T20:53:35.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 209044 40.89 MB/sec execute 131 sec latency 155.637 ms 2024-03-18T20:53:36.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 209294 40.66 MB/sec execute 132 sec latency 561.164 ms 2024-03-18T20:53:36.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:36 smithi073 ceph-mon[47384]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.9 MiB/s wr, 747 op/s 2024-03-18T20:53:36.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:36 smithi155 ceph-mon[48344]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.9 MiB/s wr, 747 op/s 2024-03-18T20:53:36.556 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:36 smithi046 ceph-mon[43495]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.9 MiB/s wr, 747 op/s 2024-03-18T20:53:37.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 209362 40.39 MB/sec execute 133 sec latency 1278.101 ms 2024-03-18T20:53:38.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 209683 40.25 MB/sec execute 134 sec latency 157.909 ms 2024-03-18T20:53:38.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:38 smithi073 ceph-mon[47384]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 555 op/s 2024-03-18T20:53:38.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:38 smithi155 ceph-mon[48344]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 555 op/s 2024-03-18T20:53:38.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:38 smithi046 ceph-mon[43495]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.4 MiB/s wr, 555 op/s 2024-03-18T20:53:39.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 210135 40.19 MB/sec execute 135 sec latency 107.965 ms 2024-03-18T20:53:40.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 210703 40.08 MB/sec execute 136 sec latency 83.494 ms 2024-03-18T20:53:40.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:40 smithi073 ceph-mon[47384]: pgmap v552: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 25 MiB/s wr, 1.12k op/s 2024-03-18T20:53:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:40 smithi155 ceph-mon[48344]: pgmap v552: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 25 MiB/s wr, 1.12k op/s 2024-03-18T20:53:40.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:40 smithi046 ceph-mon[43495]: pgmap v552: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 25 MiB/s wr, 1.12k op/s 2024-03-18T20:53:41.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 211352 39.99 MB/sec execute 137 sec latency 104.172 ms 2024-03-18T20:53:42.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 212134 40.06 MB/sec execute 138 sec latency 121.023 ms 2024-03-18T20:53:42.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:42 smithi073 ceph-mon[47384]: pgmap v553: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 22 MiB/s wr, 889 op/s 2024-03-18T20:53:42.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:42 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:42.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:42 smithi155 ceph-mon[48344]: pgmap v553: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 22 MiB/s wr, 889 op/s 2024-03-18T20:53:42.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:42 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:42.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:42 smithi046 ceph-mon[43495]: pgmap v553: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 22 MiB/s wr, 889 op/s 2024-03-18T20:53:42.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:42 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:43.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 212895 40.03 MB/sec execute 139 sec latency 91.108 ms 2024-03-18T20:53:44.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 213718 40.15 MB/sec execute 140 sec latency 53.442 ms 2024-03-18T20:53:44.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:44 smithi073 ceph-mon[47384]: pgmap v554: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 23 MiB/s wr, 1.00k op/s 2024-03-18T20:53:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.775+0000 7fd065f89700 1 -- 172.21.15.46:0/4189797286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0601007d0 msgr2=0x7fd060104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.775+0000 7fd065f89700 1 --2- 172.21.15.46:0/4189797286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0601007d0 0x7fd060104810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd050009990 tx=0x7fd050009ca0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.775+0000 7fd065f89700 1 -- 172.21.15.46:0/4189797286 shutdown_connections 2024-03-18T20:53:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.775+0000 7fd065f89700 1 --2- 172.21.15.46:0/4189797286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd060104f40 0x7fd06010cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.775+0000 7fd065f89700 1 --2- 172.21.15.46:0/4189797286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd0600ffe20 0x7fd060100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.775+0000 7fd065f89700 1 --2- 172.21.15.46:0/4189797286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0601007d0 0x7fd060104810 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.775+0000 7fd065f89700 1 -- 172.21.15.46:0/4189797286 >> 172.21.15.46:0/4189797286 conn(0x7fd0600fb680 msgr2=0x7fd0600fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.775+0000 7fd065f89700 1 -- 172.21.15.46:0/4189797286 shutdown_connections 2024-03-18T20:53:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.775+0000 7fd065f89700 1 -- 172.21.15.46:0/4189797286 wait complete. 2024-03-18T20:53:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.776+0000 7fd065f89700 1 Processor -- start 2024-03-18T20:53:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.777+0000 7fd065f89700 1 -- start start 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.777+0000 7fd065f89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0600ffe20 0x7fd060194fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.777+0000 7fd065f89700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060104f40 0x7fd060195500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.777+0000 7fd065f89700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd060199850 0x7fd060199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.777+0000 7fd065f89700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06019a2d0 con 0x7fd0600ffe20 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.777+0000 7fd065f89700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06019a440 con 0x7fd060199850 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.777+0000 7fd065f89700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06019a600 con 0x7fd060104f40 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.777+0000 7fd05effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060104f40 0x7fd060195500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.777+0000 7fd05effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060104f40 0x7fd060195500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:39538/0 (socket says 172.21.15.46:39538) 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.777+0000 7fd05effd700 1 -- 172.21.15.46:0/4049780381 learned_addr learned my addr 172.21.15.46:0/4049780381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd05ffff700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd060199850 0x7fd060199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd05f7fe700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0600ffe20 0x7fd060194fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd05effd700 1 -- 172.21.15.46:0/4049780381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd060199850 msgr2=0x7fd060199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd05effd700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd060199850 0x7fd060199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd05effd700 1 -- 172.21.15.46:0/4049780381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0600ffe20 msgr2=0x7fd060194fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd05effd700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0600ffe20 0x7fd060194fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd05effd700 1 -- 172.21.15.46:0/4049780381 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0500095f0 con 0x7fd060104f40 2024-03-18T20:53:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd05f7fe700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0600ffe20 0x7fd060194fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:53:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd05effd700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060104f40 0x7fd060195500 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd05000b2e0 tx=0x7fd05000b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd05cff9700 1 -- 172.21.15.46:0/4049780381 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05001d070 con 0x7fd060104f40 2024-03-18T20:53:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd05cff9700 1 -- 172.21.15.46:0/4049780381 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd05000fb80 con 0x7fd060104f40 2024-03-18T20:53:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd05cff9700 1 -- 172.21.15.46:0/4049780381 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500173f0 con 0x7fd060104f40 2024-03-18T20:53:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd065f89700 1 -- 172.21.15.46:0/4049780381 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0601a1b00 con 0x7fd060104f40 2024-03-18T20:53:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.778+0000 7fd065f89700 1 -- 172.21.15.46:0/4049780381 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0601a1ed0 con 0x7fd060104f40 2024-03-18T20:53:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.781+0000 7fd05cff9700 1 -- 172.21.15.46:0/4049780381 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd05000fcf0 con 0x7fd060104f40 2024-03-18T20:53:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.782+0000 7fd05cff9700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0480a1290 0x7fd0480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.782+0000 7fd05cff9700 1 -- 172.21.15.46:0/4049780381 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd0500cdf70 con 0x7fd060104f40 2024-03-18T20:53:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.782+0000 7fd05f7fe700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0480a1290 0x7fd0480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:44 smithi155 ceph-mon[48344]: pgmap v554: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 23 MiB/s wr, 1.00k op/s 2024-03-18T20:53:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.790+0000 7fd05f7fe700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0480a1290 0x7fd0480a3750 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd054005fd0 tx=0x7fd054005dc0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.792+0000 7fd07211eb80 1 Processor -- start 2024-03-18T20:53:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.792+0000 7fd07211eb80 1 -- start start 2024-03-18T20:53:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.792+0000 7fd07211eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626f570 0x55bc9626f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.792+0000 7fd07211eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626fe90 0x55bc9627a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.792+0000 7fd07211eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 0x55bc9627d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.792+0000 7fd07211eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc961c9930 con 0x55bc9626f570 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.792+0000 7fd07211eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc961c72f0 con 0x55bc9626fe90 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.792+0000 7fd07211eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc961c2be0 con 0x55bc9627adb0 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.792+0000 7fd05ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 0x55bc9627d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.792+0000 7fd05ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 0x55bc9627d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:39552/0 (socket says 172.21.15.46:39552) 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.793+0000 7fd05ffff700 1 -- 172.21.15.46:0/516509881 learned_addr learned my addr 172.21.15.46:0/516509881 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.793+0000 7fd05effd700 1 --2- 172.21.15.46:0/516509881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626fe90 0x55bc9627a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.793+0000 7fd05f7fe700 1 --2- 172.21.15.46:0/516509881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626f570 0x55bc9626f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.793+0000 7fd05ffff700 1 -- 172.21.15.46:0/516509881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626fe90 msgr2=0x55bc9627a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.793+0000 7fd05ffff700 1 --2- 172.21.15.46:0/516509881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626fe90 0x55bc9627a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.793+0000 7fd05ffff700 1 -- 172.21.15.46:0/516509881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626f570 msgr2=0x55bc9626f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.793+0000 7fd05ffff700 1 --2- 172.21.15.46:0/516509881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626f570 0x55bc9626f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.793+0000 7fd05ffff700 1 -- 172.21.15.46:0/516509881 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc96044350 con 0x55bc9627adb0 2024-03-18T20:53:44.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:44 smithi046 ceph-mon[43495]: pgmap v554: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 23 MiB/s wr, 1.00k op/s 2024-03-18T20:53:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.794+0000 7fd05ffff700 1 --2- 172.21.15.46:0/516509881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 0x55bc9627d1a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd058009390 tx=0x7fd058009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9f053fc3a764f00 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.794+0000 7fd045ffb700 1 -- 172.21.15.46:0/516509881 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058014070 con 0x55bc9627adb0 2024-03-18T20:53:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.794+0000 7fd045ffb700 1 -- 172.21.15.46:0/516509881 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd058015040 con 0x55bc9627adb0 2024-03-18T20:53:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.794+0000 7fd045ffb700 1 -- 172.21.15.46:0/516509881 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058019cd0 con 0x55bc9627adb0 2024-03-18T20:53:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.795+0000 7fd07211eb80 1 -- 172.21.15.46:0/516509881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 msgr2=0x55bc9627d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.795+0000 7fd07211eb80 1 --2- 172.21.15.46:0/516509881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 0x55bc9627d1a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd058009390 tx=0x7fd058009920 comp rx=0 tx=0).stop 2024-03-18T20:53:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.795+0000 7fd07211eb80 1 -- 172.21.15.46:0/516509881 shutdown_connections 2024-03-18T20:53:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.795+0000 7fd07211eb80 1 --2- 172.21.15.46:0/516509881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626f570 0x55bc9626f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.795+0000 7fd07211eb80 1 --2- 172.21.15.46:0/516509881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 0x55bc9627d1a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.795+0000 7fd07211eb80 1 --2- 172.21.15.46:0/516509881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626fe90 0x55bc9627a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.795+0000 7fd07211eb80 1 -- 172.21.15.46:0/516509881 >> 172.21.15.46:0/516509881 conn(0x55bc961e61c0 msgr2=0x55bc9626ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.795+0000 7fd07211eb80 1 -- 172.21.15.46:0/516509881 shutdown_connections 2024-03-18T20:53:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.795+0000 7fd07211eb80 1 -- 172.21.15.46:0/516509881 wait complete. 2024-03-18T20:53:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.795+0000 7fd07211eb80 1 -- start start 2024-03-18T20:53:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.796+0000 7fd07211eb80 1 Processor -- start 2024-03-18T20:53:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.796+0000 7fd07211eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626f570 0x55bc9627a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.796+0000 7fd07211eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9626fe90 0x55bc96277ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.796+0000 7fd07211eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9627adb0 0x55bc962780e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.796+0000 7fd07211eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9615ed20 con 0x55bc9626f570 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.796+0000 7fd07211eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9615e9c0 con 0x55bc9627adb0 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.796+0000 7fd07211eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc96046640 con 0x55bc9626fe90 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.796+0000 7fd07211eb80 10 client.0 _ll_register_callbacks cb 0x55bc95fca340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.797+0000 7fd05f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626f570 0x55bc9627a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.797+0000 7fd05f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626f570 0x55bc9627a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46188/0 (socket says 172.21.15.46:46188) 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.797+0000 7fd05f7fe700 1 -- 172.21.15.46:0/3208061511 learned_addr learned my addr 172.21.15.46:0/3208061511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.797+0000 7fd05ffff700 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9627adb0 0x55bc962780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.797+0000 7fd05effd700 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9626fe90 0x55bc96277ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.798+0000 7fd05ffff700 1 -- 172.21.15.46:0/3208061511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9626fe90 msgr2=0x55bc96277ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.798+0000 7fd05ffff700 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9626fe90 0x55bc96277ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.798+0000 7fd05ffff700 1 -- 172.21.15.46:0/3208061511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626f570 msgr2=0x55bc9627a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.798+0000 7fd05ffff700 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626f570 0x55bc9627a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.798+0000 7fd05ffff700 1 -- 172.21.15.46:0/3208061511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd058009070 con 0x55bc9627adb0 2024-03-18T20:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.798+0000 7fd05ffff700 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9627adb0 0x55bc962780e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd058009390 tx=0x7fd05801ab10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eade1fe08e88db72 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.798+0000 7fd03effd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:53:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd03effd700 1 -- 172.21.15.46:0/3208061511 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058014040 con 0x55bc9627adb0 2024-03-18T20:53:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd03effd700 1 -- 172.21.15.46:0/3208061511 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd058019400 con 0x55bc9627adb0 2024-03-18T20:53:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd07211eb80 1 -- 172.21.15.46:0/3208061511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc962636c0 con 0x55bc9627adb0 2024-03-18T20:53:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd03effd700 1 -- 172.21.15.46:0/3208061511 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd05801af10 con 0x55bc9627adb0 2024-03-18T20:53:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd07211eb80 10 client.30017 fetch_fsmap learned FSMap version 81 2024-03-18T20:53:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd07211eb80 1 -- 172.21.15.46:0/3208061511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x55bc95ff0b00 con 0x55bc9627adb0 2024-03-18T20:53:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd03effd700 1 -- 172.21.15.46:0/3208061511 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc962636c0 con 0x55bc9627adb0 2024-03-18T20:53:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd03effd700 1 -- 172.21.15.46:0/3208061511 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd05802b530 con 0x55bc9627adb0 2024-03-18T20:53:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd07211eb80 10 client.30017 fetch_fsmap finished waiting for FSMap version 81 2024-03-18T20:53:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd07211eb80 10 client.30017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:53:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd07211eb80 20 client.30017 populate_metadata read hostname 'smithi046' 2024-03-18T20:53:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd07211eb80 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bc9628a8b0 0x55bc9628cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd07211eb80 4 client.30017 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:53:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.799+0000 7fd07211eb80 1 -- 172.21.15.46:0/3208061511 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc961bb6a0 con 0x55bc9628a8b0 2024-03-18T20:53:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.800+0000 7fd05f7fe700 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bc9628a8b0 0x55bc9628cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.800+0000 7fd03effd700 10 client.30017 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.800+0000 7fd05f7fe700 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bc9628a8b0 0x55bc9628cca0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6488fc12a8980437 server_cookie=f28fd387b6d58842 in_seq=0 out_seq=0 2024-03-18T20:53:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.802+0000 7fd03effd700 1 -- 172.21.15.46:0/3208061511 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bc961bb6a0 con 0x55bc9628a8b0 2024-03-18T20:53:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.802+0000 7fd03effd700 10 client.30017 handle_command_reply: tid=0 2024-03-18T20:53:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.802+0000 7fd07211eb80 1 client.30017 shutdown 2024-03-18T20:53:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.802+0000 7fd07211eb80 1 -- 172.21.15.46:0/3208061511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9627adb0 msgr2=0x55bc962780e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.802+0000 7fd07211eb80 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9627adb0 0x55bc962780e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd058009390 tx=0x7fd05801ab10 comp rx=0 tx=0).stop 2024-03-18T20:53:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.802+0000 7fd07211eb80 1 -- 172.21.15.46:0/3208061511 shutdown_connections 2024-03-18T20:53:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.802+0000 7fd07211eb80 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bc9628a8b0 0x55bc9628cca0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.802+0000 7fd07211eb80 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626f570 0x55bc9627a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.802+0000 7fd07211eb80 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9627adb0 0x55bc962780e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.802+0000 7fd07211eb80 1 --2- 172.21.15.46:0/3208061511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9626fe90 0x55bc96277ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.802+0000 7fd07211eb80 1 -- 172.21.15.46:0/3208061511 >> 172.21.15.46:0/3208061511 conn(0x55bc961e61c0 msgr2=0x55bc9626b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.802+0000 7fd03effd700 0 client.30017 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.803+0000 7fd07211eb80 1 -- 172.21.15.46:0/3208061511 shutdown_connections 2024-03-18T20:53:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.803+0000 7fd07211eb80 1 -- 172.21.15.46:0/3208061511 wait complete. 2024-03-18T20:53:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.803+0000 7fd07211eb80 20 client.30017 trim_cache size 0 max 16384 2024-03-18T20:53:44.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.825+0000 7fd07211eb80 1 Processor -- start 2024-03-18T20:53:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.825+0000 7fd07211eb80 1 -- start start 2024-03-18T20:53:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.825+0000 7fd07211eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626f570 0x55bc962724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.825+0000 7fd07211eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626fe90 0x55bc96281a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.825+0000 7fd07211eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 0x55bc96277a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.825+0000 7fd07211eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc96287500 con 0x55bc9626fe90 2024-03-18T20:53:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.825+0000 7fd07211eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc962778f0 con 0x55bc9626f570 2024-03-18T20:53:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.825+0000 7fd07211eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc96270850 con 0x55bc9627adb0 2024-03-18T20:53:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.826+0000 7fd05f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626f570 0x55bc962724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.826+0000 7fd05ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 0x55bc96277a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.826+0000 7fd05ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 0x55bc96277a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:39562/0 (socket says 172.21.15.46:39562) 2024-03-18T20:53:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.826+0000 7fd05f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626f570 0x55bc962724e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51360/0 (socket says 172.21.15.46:51360) 2024-03-18T20:53:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.826+0000 7fd05ffff700 1 -- 172.21.15.46:0/2329132662 learned_addr learned my addr 172.21.15.46:0/2329132662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.826+0000 7fd05effd700 1 --2- 172.21.15.46:0/2329132662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626fe90 0x55bc96281a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.826+0000 7fd05ffff700 1 -- 172.21.15.46:0/2329132662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626f570 msgr2=0x55bc962724e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.826+0000 7fd05ffff700 1 --2- 172.21.15.46:0/2329132662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626f570 0x55bc962724e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.826+0000 7fd05ffff700 1 -- 172.21.15.46:0/2329132662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626fe90 msgr2=0x55bc96281a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.826+0000 7fd05ffff700 1 --2- 172.21.15.46:0/2329132662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626fe90 0x55bc96281a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.826+0000 7fd05ffff700 1 -- 172.21.15.46:0/2329132662 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc96188e40 con 0x55bc9627adb0 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.826+0000 7fd05ffff700 1 --2- 172.21.15.46:0/2329132662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 0x55bc96277a70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd058009360 tx=0x7fd058009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3b23d5846ed1ca9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.827+0000 7fd0467fc700 1 -- 172.21.15.46:0/2329132662 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058019600 con 0x55bc9627adb0 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.827+0000 7fd0467fc700 1 -- 172.21.15.46:0/2329132662 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd05802bb80 con 0x55bc9627adb0 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.827+0000 7fd0467fc700 1 -- 172.21.15.46:0/2329132662 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058022e10 con 0x55bc9627adb0 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.827+0000 7fd07211eb80 1 -- 172.21.15.46:0/2329132662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 msgr2=0x55bc96277a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.827+0000 7fd07211eb80 1 --2- 172.21.15.46:0/2329132662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 0x55bc96277a70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd058009360 tx=0x7fd058009920 comp rx=0 tx=0).stop 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.827+0000 7fd07211eb80 1 -- 172.21.15.46:0/2329132662 shutdown_connections 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.827+0000 7fd07211eb80 1 --2- 172.21.15.46:0/2329132662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9627adb0 0x55bc96277a70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.827+0000 7fd07211eb80 1 --2- 172.21.15.46:0/2329132662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626f570 0x55bc962724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.827+0000 7fd07211eb80 1 --2- 172.21.15.46:0/2329132662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9626fe90 0x55bc96281a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.827+0000 7fd07211eb80 1 -- 172.21.15.46:0/2329132662 >> 172.21.15.46:0/2329132662 conn(0x55bc961e61c0 msgr2=0x55bc96272e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.828+0000 7fd07211eb80 1 -- 172.21.15.46:0/2329132662 shutdown_connections 2024-03-18T20:53:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.828+0000 7fd07211eb80 1 -- 172.21.15.46:0/2329132662 wait complete. 2024-03-18T20:53:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.828+0000 7fd07211eb80 1 -- start start 2024-03-18T20:53:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.828+0000 7fd07211eb80 1 Processor -- start 2024-03-18T20:53:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd07211eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626f570 0x55bc9627fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd07211eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9626fe90 0x55bc96280180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd07211eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9627adb0 0x55bc962dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd07211eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc96280860 con 0x55bc9627adb0 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd07211eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc962ddbf0 con 0x55bc9626f570 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd07211eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc962ddd30 con 0x55bc9626fe90 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd07211eb80 10 client.0 _ll_register_callbacks cb 0x7fd050097450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd05effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9626fe90 0x55bc96280180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd05effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9626fe90 0x55bc96280180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:39576/0 (socket says 172.21.15.46:39576) 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd05effd700 1 -- 172.21.15.46:0/4186373790 learned_addr learned my addr 172.21.15.46:0/4186373790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd05f7fe700 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626f570 0x55bc9627fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd05ffff700 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9627adb0 0x55bc962dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd05effd700 1 -- 172.21.15.46:0/4186373790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626f570 msgr2=0x55bc9627fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd05effd700 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626f570 0x55bc9627fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd05effd700 1 -- 172.21.15.46:0/4186373790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9627adb0 msgr2=0x55bc962dd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd05effd700 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9627adb0 0x55bc962dd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.829+0000 7fd05effd700 1 -- 172.21.15.46:0/4186373790 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc95ff0b00 con 0x55bc9626fe90 2024-03-18T20:53:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.830+0000 7fd05effd700 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9626fe90 0x55bc96280180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd050097b00 tx=0x7fd05000bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2cf5c414c28f40a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.830+0000 7fd03effd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:53:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.830+0000 7fd03effd700 1 -- 172.21.15.46:0/4186373790 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500a1170 con 0x55bc9626fe90 2024-03-18T20:53:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.830+0000 7fd03effd700 1 -- 172.21.15.46:0/4186373790 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fd05009a790 con 0x55bc9626fe90 2024-03-18T20:53:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.830+0000 7fd07211eb80 1 -- 172.21.15.46:0/4186373790 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd058015040 con 0x55bc9626fe90 2024-03-18T20:53:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.830+0000 7fd03effd700 1 -- 172.21.15.46:0/4186373790 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0500b9250 con 0x55bc9626fe90 2024-03-18T20:53:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.830+0000 7fd07211eb80 10 client.30118 fetch_fsmap learned FSMap version 81 2024-03-18T20:53:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.830+0000 7fd07211eb80 1 -- 172.21.15.46:0/4186373790 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x55bc96048c70 con 0x55bc9626fe90 2024-03-18T20:53:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.830+0000 7fd03effd700 1 -- 172.21.15.46:0/4186373790 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd058015040 con 0x55bc9626fe90 2024-03-18T20:53:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.830+0000 7fd03effd700 1 -- 172.21.15.46:0/4186373790 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd0500b3cb0 con 0x55bc9626fe90 2024-03-18T20:53:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.831+0000 7fd07211eb80 10 client.30118 fetch_fsmap finished waiting for FSMap version 81 2024-03-18T20:53:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.831+0000 7fd07211eb80 10 client.30118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:53:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.831+0000 7fd07211eb80 20 client.30118 populate_metadata read hostname 'smithi046' 2024-03-18T20:53:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.831+0000 7fd07211eb80 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bc962df270 0x55bc962e1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.831+0000 7fd07211eb80 4 client.30118 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:53:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.831+0000 7fd07211eb80 1 -- 172.21.15.46:0/4186373790 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bc96011c90 con 0x55bc962df270 2024-03-18T20:53:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.831+0000 7fd05f7fe700 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bc962df270 0x55bc962e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.831+0000 7fd03effd700 10 client.30118 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:44.832+0000 7fd05f7fe700 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bc962df270 0x55bc962e1690 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=905bdf06bdb12aff server_cookie=9293b3897393a424 in_seq=0 out_seq=0 2024-03-18T20:53:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.034+0000 7fd03effd700 1 -- 172.21.15.46:0/4186373790 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bc96011c90 con 0x55bc962df270 2024-03-18T20:53:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.034+0000 7fd03effd700 10 client.30118 handle_command_reply: tid=0 2024-03-18T20:53:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.034+0000 7fd07211eb80 1 client.30118 shutdown 2024-03-18T20:53:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.035+0000 7fd07211eb80 1 -- 172.21.15.46:0/4186373790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9626fe90 msgr2=0x55bc96280180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.035+0000 7fd07211eb80 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9626fe90 0x55bc96280180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd050097b00 tx=0x7fd05000bfa0 comp rx=0 tx=0).stop 2024-03-18T20:53:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.035+0000 7fd07211eb80 1 -- 172.21.15.46:0/4186373790 shutdown_connections 2024-03-18T20:53:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.035+0000 7fd07211eb80 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bc962df270 0x55bc962e1690 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.035+0000 7fd07211eb80 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc9627adb0 0x55bc962dd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.035+0000 7fd07211eb80 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bc9626f570 0x55bc9627fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.035+0000 7fd07211eb80 1 --2- 172.21.15.46:0/4186373790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc9626fe90 0x55bc96280180 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.035+0000 7fd07211eb80 1 -- 172.21.15.46:0/4186373790 >> 172.21.15.46:0/4186373790 conn(0x55bc961e61c0 msgr2=0x55bc96277a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.035+0000 7fd03effd700 0 client.30118 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.035+0000 7fd07211eb80 1 -- 172.21.15.46:0/4186373790 shutdown_connections 2024-03-18T20:53:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.035+0000 7fd07211eb80 1 -- 172.21.15.46:0/4186373790 wait complete. 2024-03-18T20:53:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.035+0000 7fd07211eb80 20 client.30118 trim_cache size 0 max 16384 2024-03-18T20:53:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:53:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:53:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:53:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:53:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.036+0000 7fd065f89700 1 -- 172.21.15.46:0/4049780381 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0480a1290 msgr2=0x7fd0480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.036+0000 7fd065f89700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0480a1290 0x7fd0480a3750 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd054005fd0 tx=0x7fd054005dc0 comp rx=0 tx=0).stop 2024-03-18T20:53:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.036+0000 7fd065f89700 1 -- 172.21.15.46:0/4049780381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060104f40 msgr2=0x7fd060195500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.036+0000 7fd065f89700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060104f40 0x7fd060195500 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd05000b2e0 tx=0x7fd05000b730 comp rx=0 tx=0).stop 2024-03-18T20:53:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.037+0000 7fd065f89700 1 -- 172.21.15.46:0/4049780381 shutdown_connections 2024-03-18T20:53:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.037+0000 7fd065f89700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0480a1290 0x7fd0480a3750 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.037+0000 7fd065f89700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0600ffe20 0x7fd060194fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.037+0000 7fd065f89700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd060199850 0x7fd060199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.037+0000 7fd065f89700 1 --2- 172.21.15.46:0/4049780381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060104f40 0x7fd060195500 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.037+0000 7fd065f89700 1 -- 172.21.15.46:0/4049780381 >> 172.21.15.46:0/4049780381 conn(0x7fd0600fb680 msgr2=0x7fd0600fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.038+0000 7fd065f89700 1 -- 172.21.15.46:0/4049780381 shutdown_connections 2024-03-18T20:53:45.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:45.038+0000 7fd065f89700 1 -- 172.21.15.46:0/4049780381 wait complete. 2024-03-18T20:53:45.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 214159 39.99 MB/sec execute 141 sec latency 196.220 ms 2024-03-18T20:53:46.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 214710 39.85 MB/sec execute 142 sec latency 154.451 ms 2024-03-18T20:53:46.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:46 smithi073 ceph-mon[47384]: pgmap v555: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 23 MiB/s wr, 1.10k op/s 2024-03-18T20:53:46.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:46 smithi155 ceph-mon[48344]: pgmap v555: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 23 MiB/s wr, 1.10k op/s 2024-03-18T20:53:46.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:46 smithi046 ceph-mon[43495]: pgmap v555: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 23 MiB/s wr, 1.10k op/s 2024-03-18T20:53:47.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 215416 39.84 MB/sec execute 143 sec latency 149.055 ms 2024-03-18T20:53:47.471 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:47 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:47.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:47 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:47.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:47 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:48.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 216272 39.96 MB/sec execute 144 sec latency 100.151 ms 2024-03-18T20:53:48.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:48 smithi073 ceph-mon[47384]: pgmap v556: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 918 op/s 2024-03-18T20:53:48.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:48 smithi155 ceph-mon[48344]: pgmap v556: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 918 op/s 2024-03-18T20:53:48.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:48 smithi046 ceph-mon[43495]: pgmap v556: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 918 op/s 2024-03-18T20:53:49.386 INFO:tasks.workunit.client.0.smithi046.stdout: 10 216919 39.91 MB/sec execute 145 sec latency 76.966 ms 2024-03-18T20:53:50.386 INFO:tasks.workunit.client.0.smithi046.stdout: 10 217570 39.90 MB/sec execute 146 sec latency 76.592 ms 2024-03-18T20:53:50.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:50 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:50.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:50 smithi155 ceph-mon[48344]: pgmap v557: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.23k op/s 2024-03-18T20:53:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:50 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:50.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:50 smithi046 ceph-mon[43495]: pgmap v557: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.23k op/s 2024-03-18T20:53:50.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:50 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:53:50.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:50 smithi073 ceph-mon[47384]: pgmap v557: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 1.23k op/s 2024-03-18T20:53:51.386 INFO:tasks.workunit.client.0.smithi046.stdout: 10 218116 39.78 MB/sec execute 147 sec latency 91.770 ms 2024-03-18T20:53:52.385 INFO:tasks.workunit.client.0.smithi046.stdout: 10 218934 39.76 MB/sec execute 148 sec latency 106.881 ms 2024-03-18T20:53:52.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:52 smithi073 ceph-mon[47384]: pgmap v558: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.3 MiB/s wr, 670 op/s 2024-03-18T20:53:52.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:52 smithi155 ceph-mon[48344]: pgmap v558: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.3 MiB/s wr, 670 op/s 2024-03-18T20:53:52.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:52 smithi046 ceph-mon[43495]: pgmap v558: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.3 MiB/s wr, 670 op/s 2024-03-18T20:53:53.386 INFO:tasks.workunit.client.0.smithi046.stdout: 10 219863 39.92 MB/sec execute 149 sec latency 83.854 ms 2024-03-18T20:53:54.386 INFO:tasks.workunit.client.0.smithi046.stdout: 10 220463 39.84 MB/sec execute 150 sec latency 64.068 ms 2024-03-18T20:53:54.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:54 smithi073 ceph-mon[47384]: pgmap v559: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.7 MiB/s wr, 775 op/s 2024-03-18T20:53:54.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:54 smithi155 ceph-mon[48344]: pgmap v559: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.7 MiB/s wr, 775 op/s 2024-03-18T20:53:54.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:54 smithi046 ceph-mon[43495]: pgmap v559: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.7 MiB/s wr, 775 op/s 2024-03-18T20:53:55.386 INFO:tasks.workunit.client.0.smithi046.stdout: 10 221205 39.85 MB/sec execute 151 sec latency 79.125 ms 2024-03-18T20:53:56.386 INFO:tasks.workunit.client.0.smithi046.stdout: 10 221826 39.82 MB/sec execute 152 sec latency 101.050 ms 2024-03-18T20:53:56.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:56 smithi073 ceph-mon[47384]: pgmap v560: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.9 MiB/s wr, 818 op/s 2024-03-18T20:53:56.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:56 smithi155 ceph-mon[48344]: pgmap v560: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.9 MiB/s wr, 818 op/s 2024-03-18T20:53:56.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:56 smithi046 ceph-mon[43495]: pgmap v560: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 8.9 MiB/s wr, 818 op/s 2024-03-18T20:53:57.386 INFO:tasks.workunit.client.0.smithi046.stdout: 10 222398 39.71 MB/sec execute 153 sec latency 141.108 ms 2024-03-18T20:53:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.201+0000 7f04b57bb700 1 -- 172.21.15.46:0/3543738863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04b00ff220 msgr2=0x7f04b00ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.201+0000 7f04b57bb700 1 --2- 172.21.15.46:0/3543738863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04b00ff220 0x7f04b00ff6a0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009990 tx=0x7f04a4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.201+0000 7f04b57bb700 1 -- 172.21.15.46:0/3543738863 shutdown_connections 2024-03-18T20:53:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.201+0000 7f04b57bb700 1 --2- 172.21.15.46:0/3543738863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04b00ffbe0 0x7f04b01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.201+0000 7f04b57bb700 1 --2- 172.21.15.46:0/3543738863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b0105be0 0x7f04b0105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.201+0000 7f04b57bb700 1 --2- 172.21.15.46:0/3543738863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04b00ff220 0x7f04b00ff6a0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.201+0000 7f04b57bb700 1 -- 172.21.15.46:0/3543738863 >> 172.21.15.46:0/3543738863 conn(0x7f04b00761b0 msgr2=0x7f04b00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.201+0000 7f04b57bb700 1 -- 172.21.15.46:0/3543738863 shutdown_connections 2024-03-18T20:53:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.202+0000 7f04b57bb700 1 -- 172.21.15.46:0/3543738863 wait complete. 2024-03-18T20:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.202+0000 7f04b57bb700 1 Processor -- start 2024-03-18T20:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.203+0000 7f04b57bb700 1 -- start start 2024-03-18T20:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.203+0000 7f04b57bb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04b00ff220 0x7f04b019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.203+0000 7f04b57bb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b00ffbe0 0x7f04b019dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.203+0000 7f04b57bb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04b0105be0 0x7f04b01a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.203+0000 7f04b57bb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b01a2640 con 0x7f04b00ff220 2024-03-18T20:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.203+0000 7f04b57bb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b01a27b0 con 0x7f04b0105be0 2024-03-18T20:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.203+0000 7f04b57bb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b01a2970 con 0x7f04b00ffbe0 2024-03-18T20:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.203+0000 7f04aeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04b00ff220 0x7f04b019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.203+0000 7f04aeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04b00ff220 0x7f04b019d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37126/0 (socket says 172.21.15.46:37126) 2024-03-18T20:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.203+0000 7f04aeffd700 1 -- 172.21.15.46:0/2282642966 learned_addr learned my addr 172.21.15.46:0/2282642966 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.203+0000 7f04af7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04b0105be0 0x7f04b01a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.204+0000 7f04ae7fc700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b00ffbe0 0x7f04b019dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.204+0000 7f04af7fe700 1 -- 172.21.15.46:0/2282642966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b00ffbe0 msgr2=0x7f04b019dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.204+0000 7f04af7fe700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b00ffbe0 0x7f04b019dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.204+0000 7f04af7fe700 1 -- 172.21.15.46:0/2282642966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04b00ff220 msgr2=0x7f04b019d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.204+0000 7f04af7fe700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04b00ff220 0x7f04b019d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.204+0000 7f04af7fe700 1 -- 172.21.15.46:0/2282642966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a40095f0 con 0x7f04b0105be0 2024-03-18T20:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.204+0000 7f04ae7fc700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b00ffbe0 0x7f04b019dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.205+0000 7f04af7fe700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04b0105be0 0x7f04b01a2000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f04a800efc0 tx=0x7f04a800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.205+0000 7f0497fff700 1 -- 172.21.15.46:0/2282642966 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8018070 con 0x7f04b0105be0 2024-03-18T20:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.205+0000 7f04b57bb700 1 -- 172.21.15.46:0/2282642966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b01a2bf0 con 0x7f04b0105be0 2024-03-18T20:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.205+0000 7f0497fff700 1 -- 172.21.15.46:0/2282642966 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04a800f040 con 0x7f04b0105be0 2024-03-18T20:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.205+0000 7f0497fff700 1 -- 172.21.15.46:0/2282642966 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a800cc80 con 0x7f04b0105be0 2024-03-18T20:53:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.205+0000 7f04b57bb700 1 -- 172.21.15.46:0/2282642966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b01aa490 con 0x7f04b0105be0 2024-03-18T20:53:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.210+0000 7f04c1950b80 1 Processor -- start 2024-03-18T20:53:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.211+0000 7f0497fff700 1 -- 172.21.15.46:0/2282642966 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f04a8003a00 con 0x7f04b0105be0 2024-03-18T20:53:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.211+0000 7f04c1950b80 1 -- start start 2024-03-18T20:53:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.212+0000 7f04c1950b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2570 0x55f8a85d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.212+0000 7f04c1950b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85d2e90 0x55f8a85dd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.212+0000 7f04c1950b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85dddb0 0x55f8a85e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.212+0000 7f04c1950b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a852c930 con 0x55f8a85dddb0 2024-03-18T20:53:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.212+0000 7f04c1950b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a852a2f0 con 0x55f8a85d2e90 2024-03-18T20:53:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.212+0000 7f04c1950b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a8525be0 con 0x55f8a85d2570 2024-03-18T20:53:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.212+0000 7f04aeffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85d2e90 0x55f8a85dd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.212+0000 7f04aeffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85d2e90 0x55f8a85dd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:60776/0 (socket says 172.21.15.46:60776) 2024-03-18T20:53:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.212+0000 7f04aeffd700 1 -- 172.21.15.46:0/3691060125 learned_addr learned my addr 172.21.15.46:0/3691060125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.212+0000 7f04ae7fc700 1 --2- 172.21.15.46:0/3691060125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2570 0x55f8a85d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.213+0000 7f0497fff700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f04980a1390 0x7f04980a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.213+0000 7f0497fff700 1 -- 172.21.15.46:0/2282642966 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f04a80cc8d0 con 0x7f04b0105be0 2024-03-18T20:53:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.213+0000 7f04aeffd700 1 -- 172.21.15.46:0/3691060125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2570 msgr2=0x55f8a85d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.213+0000 7f04aeffd700 1 --2- 172.21.15.46:0/3691060125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2570 0x55f8a85d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.213+0000 7f04aeffd700 1 -- 172.21.15.46:0/3691060125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85dddb0 msgr2=0x55f8a85e01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.213+0000 7f04aeffd700 1 --2- 172.21.15.46:0/3691060125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85dddb0 0x55f8a85e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.213+0000 7f04aeffd700 1 -- 172.21.15.46:0/3691060125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8a83a7350 con 0x55f8a85d2e90 2024-03-18T20:53:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.213+0000 7f04af7fe700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f04980a1390 0x7f04980a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.214+0000 7f04aeffd700 1 --2- 172.21.15.46:0/3691060125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85d2e90 0x55f8a85dd870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009dd0 tx=0x7f04a0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38c6995a0a1c142 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.214+0000 7f04af7fe700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f04980a1390 0x7f04980a3850 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009320 tx=0x7f04a8010870 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.214+0000 7f04957fa700 1 -- 172.21.15.46:0/3691060125 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0004a30 con 0x55f8a85d2e90 2024-03-18T20:53:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.214+0000 7f04957fa700 1 -- 172.21.15.46:0/3691060125 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04a00054e0 con 0x55f8a85d2e90 2024-03-18T20:53:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.214+0000 7f04957fa700 1 -- 172.21.15.46:0/3691060125 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0019890 con 0x55f8a85d2e90 2024-03-18T20:53:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.215+0000 7f04c1950b80 1 -- 172.21.15.46:0/3691060125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85d2e90 msgr2=0x55f8a85dd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.215+0000 7f04c1950b80 1 --2- 172.21.15.46:0/3691060125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85d2e90 0x55f8a85dd870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009dd0 tx=0x7f04a0011720 comp rx=0 tx=0).stop 2024-03-18T20:53:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.215+0000 7f04c1950b80 1 -- 172.21.15.46:0/3691060125 shutdown_connections 2024-03-18T20:53:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.215+0000 7f04c1950b80 1 --2- 172.21.15.46:0/3691060125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85dddb0 0x55f8a85e01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.215+0000 7f04c1950b80 1 --2- 172.21.15.46:0/3691060125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2570 0x55f8a85d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.215+0000 7f04c1950b80 1 --2- 172.21.15.46:0/3691060125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85d2e90 0x55f8a85dd870 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.215+0000 7f04c1950b80 1 -- 172.21.15.46:0/3691060125 >> 172.21.15.46:0/3691060125 conn(0x55f8a85491c0 msgr2=0x55f8a85cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.217+0000 7f04c1950b80 1 -- 172.21.15.46:0/3691060125 shutdown_connections 2024-03-18T20:53:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.218+0000 7f04c1950b80 1 -- 172.21.15.46:0/3691060125 wait complete. 2024-03-18T20:53:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.218+0000 7f04c1950b80 1 -- start start 2024-03-18T20:53:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.218+0000 7f04c1950b80 1 Processor -- start 2024-03-18T20:53:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.218+0000 7f04c1950b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85d2570 0x55f8a85dd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.218+0000 7f04c1950b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85daaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.218+0000 7f04c1950b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85dddb0 0x55f8a85db120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.218+0000 7f04c1950b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a84c1d20 con 0x55f8a85d2570 2024-03-18T20:53:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.218+0000 7f04c1950b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a84c19c0 con 0x55f8a85dddb0 2024-03-18T20:53:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.218+0000 7f04c1950b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a83a9640 con 0x55f8a85d2e90 2024-03-18T20:53:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.218+0000 7f04c1950b80 10 client.0 _ll_register_callbacks cb 0x55f8a832d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:53:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.219+0000 7f04ae7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85d2570 0x55f8a85dd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.219+0000 7f04ae7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85d2570 0x55f8a85dd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37156/0 (socket says 172.21.15.46:37156) 2024-03-18T20:53:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.219+0000 7f04ae7fc700 1 -- 172.21.15.46:0/1460731434 learned_addr learned my addr 172.21.15.46:0/1460731434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.219+0000 7f04ae7fc700 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85dddb0 0x55f8a85db120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.219+0000 7f04aeffd700 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85daaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.219+0000 7f04ae7fc700 1 -- 172.21.15.46:0/1460731434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 msgr2=0x55f8a85daaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.219+0000 7f04ae7fc700 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85daaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.224 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.219+0000 7f04ae7fc700 1 -- 172.21.15.46:0/1460731434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85d2570 msgr2=0x55f8a85dd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.219+0000 7f04ae7fc700 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85d2570 0x55f8a85dd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.219+0000 7f04ae7fc700 1 -- 172.21.15.46:0/1460731434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a0009a30 con 0x55f8a85dddb0 2024-03-18T20:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.219+0000 7f04ae7fc700 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85dddb0 0x55f8a85db120 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009990 tx=0x7f04a401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb523ee11bf7138e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.219+0000 7f0492ff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.220+0000 7f0492ff5700 1 -- 172.21.15.46:0/1460731434 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a4005260 con 0x55f8a85dddb0 2024-03-18T20:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.220+0000 7f0492ff5700 1 -- 172.21.15.46:0/1460731434 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f04a40053c0 con 0x55f8a85dddb0 2024-03-18T20:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.220+0000 7f04c1950b80 1 -- 172.21.15.46:0/1460731434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8a85c66c0 con 0x55f8a85dddb0 2024-03-18T20:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.220+0000 7f0492ff5700 1 -- 172.21.15.46:0/1460731434 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04a400f800 con 0x55f8a85dddb0 2024-03-18T20:53:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.220+0000 7f04c1950b80 10 client.30035 fetch_fsmap learned FSMap version 83 2024-03-18T20:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.220+0000 7f04c1950b80 1 -- 172.21.15.46:0/1460731434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55f8a8353b00 con 0x55f8a85dddb0 2024-03-18T20:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.220+0000 7f0492ff5700 1 -- 172.21.15.46:0/1460731434 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8a85c66c0 con 0x55f8a85dddb0 2024-03-18T20:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.221+0000 7f0492ff5700 1 -- 172.21.15.46:0/1460731434 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f04a4024070 con 0x55f8a85dddb0 2024-03-18T20:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.221+0000 7f04c1950b80 10 client.30035 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.221+0000 7f04c1950b80 10 client.30035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.221+0000 7f04c1950b80 20 client.30035 populate_metadata read hostname 'smithi046' 2024-03-18T20:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.221+0000 7f04c1950b80 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f8a85fe9a0 0x55f8a8600dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.221+0000 7f04c1950b80 4 client.30035 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.221+0000 7f04c1950b80 1 -- 172.21.15.46:0/1460731434 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8a851e6a0 con 0x55f8a85fe9a0 2024-03-18T20:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.221+0000 7f04aeffd700 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f8a85fe9a0 0x55f8a8600dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.221+0000 7f0492ff5700 10 client.30035 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.221+0000 7f04aeffd700 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f8a85fe9a0 0x55f8a8600dc0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=371565b45f3b7f7a server_cookie=e4620a422f9a797a in_seq=0 out_seq=0 2024-03-18T20:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f0492ff5700 1 -- 172.21.15.46:0/1460731434 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f8a851e6a0 con 0x55f8a85fe9a0 2024-03-18T20:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f0492ff5700 10 client.30035 handle_command_reply: tid=0 2024-03-18T20:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f04c1950b80 1 client.30035 shutdown 2024-03-18T20:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f04c1950b80 1 -- 172.21.15.46:0/1460731434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85dddb0 msgr2=0x55f8a85db120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f04c1950b80 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85dddb0 0x55f8a85db120 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009990 tx=0x7f04a401a040 comp rx=0 tx=0).stop 2024-03-18T20:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f04c1950b80 1 -- 172.21.15.46:0/1460731434 shutdown_connections 2024-03-18T20:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f04c1950b80 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f8a85fe9a0 0x55f8a8600dc0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f04c1950b80 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85d2570 0x55f8a85dd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f04c1950b80 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85dddb0 0x55f8a85db120 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f04c1950b80 1 --2- 172.21.15.46:0/1460731434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85daaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f04c1950b80 1 -- 172.21.15.46:0/1460731434 >> 172.21.15.46:0/1460731434 conn(0x55f8a85491c0 msgr2=0x55f8a85ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f0492ff5700 0 client.30035 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f04c1950b80 1 -- 172.21.15.46:0/1460731434 shutdown_connections 2024-03-18T20:53:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f04c1950b80 1 -- 172.21.15.46:0/1460731434 wait complete. 2024-03-18T20:53:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.226+0000 7f04c1950b80 20 client.30035 trim_cache size 0 max 16384 2024-03-18T20:53:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.254+0000 7f04c1950b80 1 Processor -- start 2024-03-18T20:53:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.254+0000 7f04c1950b80 1 -- start start 2024-03-18T20:53:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.254+0000 7f04c1950b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85d2570 0x55f8a85e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.254+0000 7f04c1950b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85d9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.254+0000 7f04c1950b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85dddb0 0x55f8a85ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.254+0000 7f04c1950b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a85cb1d0 con 0x55f8a85dddb0 2024-03-18T20:53:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.254+0000 7f04c1950b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a85c8c10 con 0x55f8a85d2570 2024-03-18T20:53:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.254+0000 7f04c1950b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a85e4f80 con 0x55f8a85d2e90 2024-03-18T20:53:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.254+0000 7f04aeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85d9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.254+0000 7f04aeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85d9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:53702/0 (socket says 172.21.15.46:53702) 2024-03-18T20:53:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.254+0000 7f04aeffd700 1 -- 172.21.15.46:0/2683298877 learned_addr learned my addr 172.21.15.46:0/2683298877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.255+0000 7f04aeffd700 1 -- 172.21.15.46:0/2683298877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85d2570 msgr2=0x55f8a85e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.255+0000 7f04aeffd700 1 --2- 172.21.15.46:0/2683298877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85d2570 0x55f8a85e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.255+0000 7f04aeffd700 1 -- 172.21.15.46:0/2683298877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85dddb0 msgr2=0x55f8a85ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.255+0000 7f04aeffd700 1 --2- 172.21.15.46:0/2683298877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85dddb0 0x55f8a85ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.255+0000 7f04aeffd700 1 -- 172.21.15.46:0/2683298877 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8a84ebe40 con 0x55f8a85d2e90 2024-03-18T20:53:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.255+0000 7f04aeffd700 1 --2- 172.21.15.46:0/2683298877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85d9fe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f04a00096b0 tx=0x7f04a0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49a4a6308aa42da8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.255+0000 7f0495ffb700 1 -- 172.21.15.46:0/2683298877 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a002a2c0 con 0x55f8a85d2e90 2024-03-18T20:53:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.255+0000 7f0495ffb700 1 -- 172.21.15.46:0/2683298877 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04a002a420 con 0x55f8a85d2e90 2024-03-18T20:53:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.255+0000 7f0495ffb700 1 -- 172.21.15.46:0/2683298877 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0003680 con 0x55f8a85d2e90 2024-03-18T20:53:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.256+0000 7f04c1950b80 1 -- 172.21.15.46:0/2683298877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 msgr2=0x55f8a85d9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.256+0000 7f04c1950b80 1 --2- 172.21.15.46:0/2683298877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85d9fe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f04a00096b0 tx=0x7f04a0011720 comp rx=0 tx=0).stop 2024-03-18T20:53:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.256+0000 7f04c1950b80 1 -- 172.21.15.46:0/2683298877 shutdown_connections 2024-03-18T20:53:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.256+0000 7f04c1950b80 1 --2- 172.21.15.46:0/2683298877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85dddb0 0x55f8a85ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.256+0000 7f04c1950b80 1 --2- 172.21.15.46:0/2683298877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85d2570 0x55f8a85e4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.256+0000 7f04c1950b80 1 --2- 172.21.15.46:0/2683298877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85d9fe0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.256+0000 7f04c1950b80 1 -- 172.21.15.46:0/2683298877 >> 172.21.15.46:0/2683298877 conn(0x55f8a85491c0 msgr2=0x55f8a86009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.256+0000 7f04c1950b80 1 -- 172.21.15.46:0/2683298877 shutdown_connections 2024-03-18T20:53:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.256+0000 7f04c1950b80 1 -- 172.21.15.46:0/2683298877 wait complete. 2024-03-18T20:53:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.257+0000 7f04c1950b80 1 -- start start 2024-03-18T20:53:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.257+0000 7f04c1950b80 1 Processor -- start 2024-03-18T20:53:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.257+0000 7f04c1950b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85d2570 0x55f8a85e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.257+0000 7f04c1950b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85e4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.257+0000 7f04c1950b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85dddb0 0x55f8a85e2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.257+0000 7f04c1950b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a85e2e10 con 0x55f8a85d2570 2024-03-18T20:53:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.257+0000 7f04c1950b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a85e2f80 con 0x55f8a85dddb0 2024-03-18T20:53:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.257+0000 7f04c1950b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8a85e30f0 con 0x55f8a85d2e90 2024-03-18T20:53:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.257+0000 7f04c1950b80 10 client.0 _ll_register_callbacks cb 0x7f04a400be10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:53:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.257+0000 7f04aeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85e4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.257+0000 7f04aeffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85e4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:53714/0 (socket says 172.21.15.46:53714) 2024-03-18T20:53:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.257+0000 7f04aeffd700 1 -- 172.21.15.46:0/2506276636 learned_addr learned my addr 172.21.15.46:0/2506276636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:53:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.258+0000 7f04aeffd700 1 -- 172.21.15.46:0/2506276636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85dddb0 msgr2=0x55f8a85e2700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:53:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.258+0000 7f04aeffd700 1 --2- 172.21.15.46:0/2506276636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85dddb0 0x55f8a85e2700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.258+0000 7f04aeffd700 1 -- 172.21.15.46:0/2506276636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85d2570 msgr2=0x55f8a85e42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:53:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.258+0000 7f04aeffd700 1 --2- 172.21.15.46:0/2506276636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85d2570 0x55f8a85e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.258+0000 7f04aeffd700 1 -- 172.21.15.46:0/2506276636 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8a83a7350 con 0x55f8a85d2e90 2024-03-18T20:53:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.258+0000 7f04aeffd700 1 --2- 172.21.15.46:0/2506276636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85e4830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f04a0005bd0 tx=0x7f04a0005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef5f928f19a35202 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:53:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.258+0000 7f0492ff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:53:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.258+0000 7f0492ff5700 1 -- 172.21.15.46:0/2506276636 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a003a070 con 0x55f8a85d2e90 2024-03-18T20:53:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.258+0000 7f0492ff5700 1 -- 172.21.15.46:0/2506276636 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f04a00050e0 con 0x55f8a85d2e90 2024-03-18T20:53:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.259+0000 7f04c1950b80 1 -- 172.21.15.46:0/2506276636 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04a00054e0 con 0x55f8a85d2e90 2024-03-18T20:53:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.259+0000 7f0492ff5700 1 -- 172.21.15.46:0/2506276636 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04a00052a0 con 0x55f8a85d2e90 2024-03-18T20:53:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.259+0000 7f04c1950b80 10 client.30136 fetch_fsmap learned FSMap version 83 2024-03-18T20:53:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.259+0000 7f04c1950b80 1 -- 172.21.15.46:0/2506276636 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55f8a83abc70 con 0x55f8a85d2e90 2024-03-18T20:53:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.259+0000 7f0492ff5700 1 -- 172.21.15.46:0/2506276636 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04a00054e0 con 0x55f8a85d2e90 2024-03-18T20:53:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.259+0000 7f0492ff5700 1 -- 172.21.15.46:0/2506276636 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f04a003d030 con 0x55f8a85d2e90 2024-03-18T20:53:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.259+0000 7f04c1950b80 10 client.30136 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:53:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.259+0000 7f04c1950b80 10 client.30136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:53:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.259+0000 7f04c1950b80 20 client.30136 populate_metadata read hostname 'smithi046' 2024-03-18T20:53:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.259+0000 7f04c1950b80 1 --2- 172.21.15.46:0/2506276636 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f8a8664650 0x55f8a8666a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:53:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.260+0000 7f04c1950b80 4 client.30136 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:53:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.260+0000 7f04c1950b80 1 -- 172.21.15.46:0/2506276636 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f8a8374c90 con 0x55f8a8664650 2024-03-18T20:53:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.260+0000 7f04ae7fc700 1 --2- 172.21.15.46:0/2506276636 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f8a8664650 0x55f8a8666a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:53:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.260+0000 7f0492ff5700 10 client.30136 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.260+0000 7f04ae7fc700 1 --2- 172.21.15.46:0/2506276636 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f8a8664650 0x55f8a8666a70 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=138f0beb5a734a4a server_cookie=8bb080dfb2f9f9ef in_seq=0 out_seq=0 2024-03-18T20:53:58.394 INFO:tasks.workunit.client.0.smithi046.stdout: 10 223000 39.69 MB/sec execute 154 sec latency 286.880 ms 2024-03-18T20:53:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.506+0000 7f0492ff5700 1 -- 172.21.15.46:0/2506276636 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f8a8374c90 con 0x55f8a8664650 2024-03-18T20:53:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.506+0000 7f0492ff5700 10 client.30136 handle_command_reply: tid=0 2024-03-18T20:53:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.506+0000 7f04c1950b80 1 client.30136 shutdown 2024-03-18T20:53:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.506+0000 7f04c1950b80 1 -- 172.21.15.46:0/2506276636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 msgr2=0x55f8a85e4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.506+0000 7f04c1950b80 1 --2- 172.21.15.46:0/2506276636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85e4830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f04a0005bd0 tx=0x7f04a0005700 comp rx=0 tx=0).stop 2024-03-18T20:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.508+0000 7f04c1950b80 1 -- 172.21.15.46:0/2506276636 shutdown_connections 2024-03-18T20:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.508+0000 7f04c1950b80 1 --2- 172.21.15.46:0/2506276636 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55f8a8664650 0x55f8a8666a70 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.509+0000 7f04c1950b80 1 --2- 172.21.15.46:0/2506276636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8a85d2570 0x55f8a85e42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.509+0000 7f04c1950b80 1 --2- 172.21.15.46:0/2506276636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8a85dddb0 0x55f8a85e2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.509+0000 7f04c1950b80 1 --2- 172.21.15.46:0/2506276636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f8a85d2e90 0x55f8a85e4830 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.509+0000 7f04c1950b80 1 -- 172.21.15.46:0/2506276636 >> 172.21.15.46:0/2506276636 conn(0x55f8a85491c0 msgr2=0x55f8a85d9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.509+0000 7f0492ff5700 0 client.30136 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:53:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.509+0000 7f04c1950b80 1 -- 172.21.15.46:0/2506276636 shutdown_connections 2024-03-18T20:53:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.509+0000 7f04c1950b80 1 -- 172.21.15.46:0/2506276636 wait complete. 2024-03-18T20:53:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.509+0000 7f04c1950b80 20 client.30136 trim_cache size 0 max 16384 2024-03-18T20:53:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:53:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:53:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:53:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:53:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.512+0000 7f04b57bb700 1 -- 172.21.15.46:0/2282642966 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f04980a1390 msgr2=0x7f04980a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.512+0000 7f04b57bb700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f04980a1390 0x7f04980a3850 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009320 tx=0x7f04a8010870 comp rx=0 tx=0).stop 2024-03-18T20:53:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.512+0000 7f04b57bb700 1 -- 172.21.15.46:0/2282642966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04b0105be0 msgr2=0x7f04b01a2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:53:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.512+0000 7f04b57bb700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04b0105be0 0x7f04b01a2000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f04a800efc0 tx=0x7f04a800c3a0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.512+0000 7f04b57bb700 1 -- 172.21.15.46:0/2282642966 shutdown_connections 2024-03-18T20:53:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.512+0000 7f04b57bb700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f04980a1390 0x7f04980a3850 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.512+0000 7f04b57bb700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04b00ff220 0x7f04b019d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.512+0000 7f04b57bb700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04b0105be0 0x7f04b01a2000 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.512+0000 7f04b57bb700 1 --2- 172.21.15.46:0/2282642966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b00ffbe0 0x7f04b019dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:53:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.512+0000 7f04b57bb700 1 -- 172.21.15.46:0/2282642966 >> 172.21.15.46:0/2282642966 conn(0x7f04b00761b0 msgr2=0x7f04b0101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:53:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.512+0000 7f04b57bb700 1 -- 172.21.15.46:0/2282642966 shutdown_connections 2024-03-18T20:53:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:53:58.512+0000 7f04b57bb700 1 -- 172.21.15.46:0/2282642966 wait complete. 2024-03-18T20:53:58.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:53:58 smithi073 ceph-mon[47384]: pgmap v561: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.0 MiB/s wr, 574 op/s 2024-03-18T20:53:58.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:53:58 smithi155 ceph-mon[48344]: pgmap v561: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.0 MiB/s wr, 574 op/s 2024-03-18T20:53:58.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:53:58 smithi046 ceph-mon[43495]: pgmap v561: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.0 MiB/s wr, 574 op/s 2024-03-18T20:53:59.391 INFO:tasks.workunit.client.0.smithi046.stdout: 10 223650 39.68 MB/sec execute 155 sec latency 190.519 ms 2024-03-18T20:54:00.391 INFO:tasks.workunit.client.0.smithi046.stdout: 10 224336 39.66 MB/sec execute 156 sec latency 83.668 ms 2024-03-18T20:54:00.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:00 smithi073 ceph-mon[47384]: pgmap v562: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 847 op/s 2024-03-18T20:54:00.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:00 smithi155 ceph-mon[48344]: pgmap v562: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 847 op/s 2024-03-18T20:54:00.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:00 smithi046 ceph-mon[43495]: pgmap v562: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 847 op/s 2024-03-18T20:54:01.391 INFO:tasks.workunit.client.0.smithi046.stdout: 10 224336 39.40 MB/sec execute 157 sec latency 1083.826 ms 2024-03-18T20:54:02.391 INFO:tasks.workunit.client.0.smithi046.stdout: 10 224336 39.15 MB/sec execute 158 sec latency 2083.951 ms 2024-03-18T20:54:02.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:02 smithi073 ceph-mon[47384]: pgmap v563: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 532 op/s 2024-03-18T20:54:02.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:02 smithi155 ceph-mon[48344]: pgmap v563: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 532 op/s 2024-03-18T20:54:02.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:02 smithi046 ceph-mon[43495]: pgmap v563: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.1 MiB/s wr, 532 op/s 2024-03-18T20:54:03.391 INFO:tasks.workunit.client.0.smithi046.stdout: 10 224336 38.91 MB/sec execute 159 sec latency 3084.023 ms 2024-03-18T20:54:04.391 INFO:tasks.workunit.client.0.smithi046.stdout: 10 224418 38.69 MB/sec execute 160 sec latency 3988.720 ms 2024-03-18T20:54:04.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:04 smithi073 ceph-mon[47384]: pgmap v564: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.8 MiB/s wr, 605 op/s 2024-03-18T20:54:04.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:04 smithi155 ceph-mon[48344]: pgmap v564: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.8 MiB/s wr, 605 op/s 2024-03-18T20:54:04.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:04 smithi046 ceph-mon[43495]: pgmap v564: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.8 MiB/s wr, 605 op/s 2024-03-18T20:54:05.391 INFO:tasks.workunit.client.0.smithi046.stdout: 10 224708 38.52 MB/sec execute 161 sec latency 126.611 ms 2024-03-18T20:54:06.391 INFO:tasks.workunit.client.0.smithi046.stdout: 10 225136 38.41 MB/sec execute 162 sec latency 146.486 ms 2024-03-18T20:54:06.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:06 smithi073 ceph-mon[47384]: pgmap v565: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.0 MiB/s wr, 562 op/s 2024-03-18T20:54:06.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:06 smithi155 ceph-mon[48344]: pgmap v565: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.0 MiB/s wr, 562 op/s 2024-03-18T20:54:06.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:06 smithi046 ceph-mon[43495]: pgmap v565: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.0 MiB/s wr, 562 op/s 2024-03-18T20:54:07.391 INFO:tasks.workunit.client.0.smithi046.stdout: 10 225458 38.27 MB/sec execute 163 sec latency 167.762 ms 2024-03-18T20:54:08.391 INFO:tasks.workunit.client.0.smithi046.stdout: 10 226089 38.27 MB/sec execute 164 sec latency 112.104 ms 2024-03-18T20:54:08.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:08 smithi073 ceph-mon[47384]: pgmap v566: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.8 MiB/s wr, 407 op/s 2024-03-18T20:54:08.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:08 smithi155 ceph-mon[48344]: pgmap v566: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.8 MiB/s wr, 407 op/s 2024-03-18T20:54:08.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:08 smithi046 ceph-mon[43495]: pgmap v566: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.8 MiB/s wr, 407 op/s 2024-03-18T20:54:09.392 INFO:tasks.workunit.client.0.smithi046.stdout: 10 226407 38.12 MB/sec execute 165 sec latency 206.807 ms 2024-03-18T20:54:09.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:09 smithi073 ceph-mon[47384]: osdmap e105: 12 total, 12 up, 12 in 2024-03-18T20:54:09.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:09 smithi155 ceph-mon[48344]: osdmap e105: 12 total, 12 up, 12 in 2024-03-18T20:54:09.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:09 smithi046 ceph-mon[43495]: osdmap e105: 12 total, 12 up, 12 in 2024-03-18T20:54:10.396 INFO:tasks.workunit.client.0.smithi046.stdout: 10 226765 38.01 MB/sec execute 166 sec latency 110.692 ms 2024-03-18T20:54:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:10 smithi073 ceph-mon[47384]: pgmap v568: 97 pgs: 16 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 47 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 582 op/s 2024-03-18T20:54:10.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:10 smithi155 ceph-mon[48344]: pgmap v568: 97 pgs: 16 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 47 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 582 op/s 2024-03-18T20:54:10.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:10 smithi046 ceph-mon[43495]: pgmap v568: 97 pgs: 16 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 47 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 582 op/s 2024-03-18T20:54:11.396 INFO:tasks.workunit.client.0.smithi046.stdout: 10 227531 38.09 MB/sec execute 167 sec latency 79.363 ms 2024-03-18T20:54:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.726+0000 7f081add1700 1 -- 172.21.15.46:0/1234245726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081410f940 msgr2=0x7f08141082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.726+0000 7f081add1700 1 --2- 172.21.15.46:0/1234245726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081410f940 0x7f08141082f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f080401a9d0 tx=0x7f080401ace0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.737+0000 7f081add1700 1 -- 172.21.15.46:0/1234245726 shutdown_connections 2024-03-18T20:54:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.737+0000 7f081add1700 1 --2- 172.21.15.46:0/1234245726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081411a590 0x7f081411c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.737+0000 7f081add1700 1 --2- 172.21.15.46:0/1234245726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081410f4d0 0x7f0814107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.737+0000 7f081add1700 1 --2- 172.21.15.46:0/1234245726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081410f940 0x7f08141082f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.737+0000 7f081add1700 1 -- 172.21.15.46:0/1234245726 >> 172.21.15.46:0/1234245726 conn(0x7f081406bd80 msgr2=0x7f081406c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.737+0000 7f081add1700 1 -- 172.21.15.46:0/1234245726 shutdown_connections 2024-03-18T20:54:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.737+0000 7f081add1700 1 -- 172.21.15.46:0/1234245726 wait complete. 2024-03-18T20:54:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.738+0000 7f081add1700 1 Processor -- start 2024-03-18T20:54:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.738+0000 7f081add1700 1 -- start start 2024-03-18T20:54:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.738+0000 7f081add1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081410f4d0 0x7f0814115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.738+0000 7f081add1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081410f940 0x7f08141161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.738+0000 7f081add1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081411a590 0x7f0814119580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.738+0000 7f081add1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814119bc0 con 0x7f081410f4d0 2024-03-18T20:54:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.738+0000 7f081add1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814119d30 con 0x7f081410f940 2024-03-18T20:54:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.738+0000 7f081add1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814119ef0 con 0x7f081411a590 2024-03-18T20:54:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.740+0000 7f0818b6d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081410f4d0 0x7f0814115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.741+0000 7f0818b6d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081410f4d0 0x7f0814115cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36592/0 (socket says 172.21.15.46:36592) 2024-03-18T20:54:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.741+0000 7f0818b6d700 1 -- 172.21.15.46:0/4093145664 learned_addr learned my addr 172.21.15.46:0/4093145664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.742+0000 7f081936e700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081411a590 0x7f0814119580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.743+0000 7f081936e700 1 -- 172.21.15.46:0/4093145664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081410f940 msgr2=0x7f08141161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.743+0000 7f081936e700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081410f940 0x7f08141161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.743+0000 7f081936e700 1 -- 172.21.15.46:0/4093145664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081410f4d0 msgr2=0x7f0814115cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.743+0000 7f081936e700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081410f4d0 0x7f0814115cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.743+0000 7f081936e700 1 -- 172.21.15.46:0/4093145664 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080401a600 con 0x7f081411a590 2024-03-18T20:54:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.743+0000 7f081936e700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081411a590 0x7f0814119580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f081406cdd0 tx=0x7f080800b750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.748+0000 7f0811ffb700 1 -- 172.21.15.46:0/4093145664 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808011830 con 0x7f081411a590 2024-03-18T20:54:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.748+0000 7f0811ffb700 1 -- 172.21.15.46:0/4093145664 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0808011e50 con 0x7f081411a590 2024-03-18T20:54:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.748+0000 7f0811ffb700 1 -- 172.21.15.46:0/4093145664 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808010e50 con 0x7f081411a590 2024-03-18T20:54:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.748+0000 7f081add1700 1 -- 172.21.15.46:0/4093145664 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081411a1d0 con 0x7f081411a590 2024-03-18T20:54:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.748+0000 7f081add1700 1 -- 172.21.15.46:0/4093145664 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08141aa600 con 0x7f081411a590 2024-03-18T20:54:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.753+0000 7f0826f66b80 1 Processor -- start 2024-03-18T20:54:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.756+0000 7f0826f66b80 1 -- start start 2024-03-18T20:54:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.757+0000 7f0826f66b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762b570 0x562ce762b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.757+0000 7f0826f66b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce762be90 0x562ce7636870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.757+0000 7f0826f66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce7636db0 0x562ce76391a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.757+0000 7f0826f66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce7585930 con 0x562ce7636db0 2024-03-18T20:54:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.757+0000 7f0826f66b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce75832f0 con 0x562ce762b570 2024-03-18T20:54:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.757+0000 7f0826f66b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce757ebe0 con 0x562ce762be90 2024-03-18T20:54:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.758+0000 7f0818b6d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce762be90 0x562ce7636870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.758+0000 7f0813fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce7636db0 0x562ce76391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.758+0000 7f0813fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce7636db0 0x562ce76391a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36594/0 (socket says 172.21.15.46:36594) 2024-03-18T20:54:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.758+0000 7f0813fff700 1 -- 172.21.15.46:0/196346507 learned_addr learned my addr 172.21.15.46:0/196346507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.758+0000 7f0813fff700 1 --2- 172.21.15.46:0/196346507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762b570 0x562ce762b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.758+0000 7f0811ffb700 1 -- 172.21.15.46:0/4093145664 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0808011990 con 0x7f081411a590 2024-03-18T20:54:11.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.759+0000 7f0811ffb700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f07fc0a14a0 0x7f07fc0a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.760+0000 7f0811ffb700 1 -- 172.21.15.46:0/4093145664 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f08080cc4d0 con 0x7f081411a590 2024-03-18T20:54:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.761+0000 7f081936e700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f07fc0a14a0 0x7f07fc0a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.761+0000 7f0818b6d700 1 -- 172.21.15.46:0/196346507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762b570 msgr2=0x562ce762b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.761+0000 7f0818b6d700 1 --2- 172.21.15.46:0/196346507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762b570 0x562ce762b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.761+0000 7f0818b6d700 1 -- 172.21.15.46:0/196346507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce7636db0 msgr2=0x562ce76391a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.761+0000 7f0818b6d700 1 --2- 172.21.15.46:0/196346507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce7636db0 0x562ce76391a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.761+0000 7f0818b6d700 1 -- 172.21.15.46:0/196346507 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ce7400350 con 0x562ce762be90 2024-03-18T20:54:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.761+0000 7f0818b6d700 1 --2- 172.21.15.46:0/196346507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce762be90 0x562ce7636870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f080c009f90 tx=0x7f080c007650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=581e23d5e50e3ed8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.761+0000 7f081936e700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f07fc0a14a0 0x7f07fc0a3960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0808095b40 tx=0x7f08080b9040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.776+0000 7f07faffd700 1 -- 172.21.15.46:0/196346507 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c014600 con 0x562ce762be90 2024-03-18T20:54:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.776+0000 7f07faffd700 1 -- 172.21.15.46:0/196346507 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f080c014c20 con 0x562ce762be90 2024-03-18T20:54:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.776+0000 7f07faffd700 1 -- 172.21.15.46:0/196346507 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c013af0 con 0x562ce762be90 2024-03-18T20:54:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.777+0000 7f0826f66b80 1 -- 172.21.15.46:0/196346507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce762be90 msgr2=0x562ce7636870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.777+0000 7f0826f66b80 1 --2- 172.21.15.46:0/196346507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce762be90 0x562ce7636870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f080c009f90 tx=0x7f080c007650 comp rx=0 tx=0).stop 2024-03-18T20:54:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.781+0000 7f0826f66b80 1 -- 172.21.15.46:0/196346507 shutdown_connections 2024-03-18T20:54:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.781+0000 7f0826f66b80 1 --2- 172.21.15.46:0/196346507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce7636db0 0x562ce76391a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.781+0000 7f0826f66b80 1 --2- 172.21.15.46:0/196346507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762b570 0x562ce762b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.781+0000 7f0826f66b80 1 --2- 172.21.15.46:0/196346507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce762be90 0x562ce7636870 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.781+0000 7f0826f66b80 1 -- 172.21.15.46:0/196346507 >> 172.21.15.46:0/196346507 conn(0x562ce75a21c0 msgr2=0x562ce7627a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.781+0000 7f0826f66b80 1 -- 172.21.15.46:0/196346507 shutdown_connections 2024-03-18T20:54:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.781+0000 7f0826f66b80 1 -- 172.21.15.46:0/196346507 wait complete. 2024-03-18T20:54:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.782+0000 7f0826f66b80 1 -- start start 2024-03-18T20:54:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.782+0000 7f0826f66b80 1 Processor -- start 2024-03-18T20:54:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.782+0000 7f0826f66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762b570 0x562ce76367a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.782+0000 7f0826f66b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762be90 0x562ce7633a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.782+0000 7f0826f66b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 0x562ce76340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.782+0000 7f0826f66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce751ad20 con 0x562ce762b570 2024-03-18T20:54:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.783+0000 7f0826f66b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce751a9c0 con 0x562ce762be90 2024-03-18T20:54:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.783+0000 7f0826f66b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce7402640 con 0x562ce7636db0 2024-03-18T20:54:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.783+0000 7f0826f66b80 10 client.0 _ll_register_callbacks cb 0x562ce7386340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:54:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.790+0000 7f0818b6d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762be90 0x562ce7633a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.790+0000 7f0818b6d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762be90 0x562ce7633a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55100/0 (socket says 172.21.15.46:55100) 2024-03-18T20:54:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.790+0000 7f0818b6d700 1 -- 172.21.15.46:0/1992862740 learned_addr learned my addr 172.21.15.46:0/1992862740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.790+0000 7f0813fff700 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 0x562ce76340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.790+0000 7f0813fff700 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762b570 0x562ce76367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.790+0000 7f0818b6d700 1 -- 172.21.15.46:0/1992862740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 msgr2=0x562ce76340a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.790+0000 7f0818b6d700 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 0x562ce76340a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.790+0000 7f0818b6d700 1 -- 172.21.15.46:0/1992862740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762b570 msgr2=0x562ce76367a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.790+0000 7f0818b6d700 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762b570 0x562ce76367a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.790+0000 7f0818b6d700 1 -- 172.21.15.46:0/1992862740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080c009bc0 con 0x562ce762be90 2024-03-18T20:54:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.791+0000 7f0818b6d700 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762be90 0x562ce7633a90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f080c004c20 tx=0x7f080c014ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5934152fefe711f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.791+0000 7f07f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:54:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.795+0000 7f07f3fff700 1 -- 172.21.15.46:0/1992862740 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c007ee0 con 0x562ce762be90 2024-03-18T20:54:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.795+0000 7f07f3fff700 1 -- 172.21.15.46:0/1992862740 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f080c006e40 con 0x562ce762be90 2024-03-18T20:54:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.796+0000 7f0826f66b80 1 -- 172.21.15.46:0/1992862740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ce761f6c0 con 0x562ce762be90 2024-03-18T20:54:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.796+0000 7f07f3fff700 1 -- 172.21.15.46:0/1992862740 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f080c013b80 con 0x562ce762be90 2024-03-18T20:54:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.796+0000 7f0826f66b80 10 client.30041 fetch_fsmap learned FSMap version 83 2024-03-18T20:54:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.796+0000 7f0826f66b80 1 -- 172.21.15.46:0/1992862740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x562ce73acb00 con 0x562ce762be90 2024-03-18T20:54:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.796+0000 7f0813fff700 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 0x562ce76340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:54:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.799+0000 7f07f3fff700 1 -- 172.21.15.46:0/1992862740 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ce761f6c0 con 0x562ce762be90 2024-03-18T20:54:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.799+0000 7f07f3fff700 1 -- 172.21.15.46:0/1992862740 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f080c01f420 con 0x562ce762be90 2024-03-18T20:54:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.799+0000 7f0826f66b80 10 client.30041 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:54:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.799+0000 7f0826f66b80 10 client.30041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:54:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.799+0000 7f0826f66b80 20 client.30041 populate_metadata read hostname 'smithi046' 2024-03-18T20:54:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.799+0000 7f0826f66b80 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ce7646820 0x562ce7648c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.799+0000 7f0826f66b80 4 client.30041 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:54:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.799+0000 7f0826f66b80 1 -- 172.21.15.46:0/1992862740 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ce75776a0 con 0x562ce7646820 2024-03-18T20:54:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.812+0000 7f0813fff700 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ce7646820 0x562ce7648c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.837+0000 7f0813fff700 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ce7646820 0x562ce7648c40 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1696e8cfc0c214f3 server_cookie=e16c00546d66cda4 in_seq=0 out_seq=0 2024-03-18T20:54:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.837+0000 7f07f3fff700 10 client.30041 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.855+0000 7f07f3fff700 1 -- 172.21.15.46:0/1992862740 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562ce75776a0 con 0x562ce7646820 2024-03-18T20:54:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.855+0000 7f07f3fff700 10 client.30041 handle_command_reply: tid=0 2024-03-18T20:54:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.855+0000 7f0826f66b80 1 client.30041 shutdown 2024-03-18T20:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.864+0000 7f0826f66b80 1 -- 172.21.15.46:0/1992862740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762be90 msgr2=0x562ce7633a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.864+0000 7f0826f66b80 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762be90 0x562ce7633a90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f080c004c20 tx=0x7f080c014ba0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.866+0000 7f0826f66b80 1 -- 172.21.15.46:0/1992862740 shutdown_connections 2024-03-18T20:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.866+0000 7f0826f66b80 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ce7646820 0x562ce7648c40 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.866+0000 7f0826f66b80 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762b570 0x562ce76367a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.866+0000 7f0826f66b80 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 0x562ce76340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.866+0000 7f0826f66b80 1 --2- 172.21.15.46:0/1992862740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762be90 0x562ce7633a90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.866+0000 7f0826f66b80 1 -- 172.21.15.46:0/1992862740 >> 172.21.15.46:0/1992862740 conn(0x562ce75a21c0 msgr2=0x562ce7627650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.867+0000 7f07f3fff700 0 client.30041 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.875+0000 7f0826f66b80 1 -- 172.21.15.46:0/1992862740 shutdown_connections 2024-03-18T20:54:11.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.875+0000 7f0826f66b80 1 -- 172.21.15.46:0/1992862740 wait complete. 2024-03-18T20:54:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.878+0000 7f0826f66b80 20 client.30041 trim_cache size 0 max 16384 2024-03-18T20:54:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.912+0000 7f0826f66b80 1 Processor -- start 2024-03-18T20:54:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.912+0000 7f0826f66b80 1 -- start start 2024-03-18T20:54:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.912+0000 7f0826f66b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762b570 0x562ce76340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.912+0000 7f0826f66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762be90 0x562ce763daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.912+0000 7f0826f66b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 0x562ce762ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.912+0000 7f0826f66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce763dfe0 con 0x562ce762be90 2024-03-18T20:54:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.912+0000 7f0826f66b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce7631fe0 con 0x562ce762b570 2024-03-18T20:54:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.912+0000 7f0826f66b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce762e5e0 con 0x562ce7636db0 2024-03-18T20:54:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.916+0000 7f0818b6d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762be90 0x562ce763daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.916+0000 7f0818b6d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762be90 0x562ce763daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36620/0 (socket says 172.21.15.46:36620) 2024-03-18T20:54:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.916+0000 7f0818b6d700 1 -- 172.21.15.46:0/2560065745 learned_addr learned my addr 172.21.15.46:0/2560065745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.918+0000 7f0818b6d700 1 -- 172.21.15.46:0/2560065745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 msgr2=0x562ce762ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.918+0000 7f0818b6d700 1 --2- 172.21.15.46:0/2560065745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 0x562ce762ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.918+0000 7f0818b6d700 1 -- 172.21.15.46:0/2560065745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762b570 msgr2=0x562ce76340a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.918+0000 7f0818b6d700 1 --2- 172.21.15.46:0/2560065745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762b570 0x562ce76340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.918+0000 7f0818b6d700 1 -- 172.21.15.46:0/2560065745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ce7544e40 con 0x562ce762be90 2024-03-18T20:54:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.926+0000 7f0818b6d700 1 --2- 172.21.15.46:0/2560065745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762be90 0x562ce763daa0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f080c002230 tx=0x7f080c009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55714e992dfaa31d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.931+0000 7f07fb7fe700 1 -- 172.21.15.46:0/2560065745 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c013600 con 0x562ce762be90 2024-03-18T20:54:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.931+0000 7f07fb7fe700 1 -- 172.21.15.46:0/2560065745 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f080c00d030 con 0x562ce762be90 2024-03-18T20:54:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.931+0000 7f07fb7fe700 1 -- 172.21.15.46:0/2560065745 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c013980 con 0x562ce762be90 2024-03-18T20:54:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.933+0000 7f0826f66b80 1 -- 172.21.15.46:0/2560065745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762be90 msgr2=0x562ce763daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.933+0000 7f0826f66b80 1 --2- 172.21.15.46:0/2560065745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762be90 0x562ce763daa0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f080c002230 tx=0x7f080c009930 comp rx=0 tx=0).stop 2024-03-18T20:54:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.941+0000 7f0826f66b80 1 -- 172.21.15.46:0/2560065745 shutdown_connections 2024-03-18T20:54:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.941+0000 7f0826f66b80 1 --2- 172.21.15.46:0/2560065745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 0x562ce762ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.941+0000 7f0826f66b80 1 --2- 172.21.15.46:0/2560065745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762b570 0x562ce76340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.941+0000 7f0826f66b80 1 --2- 172.21.15.46:0/2560065745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762be90 0x562ce763daa0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.941+0000 7f0826f66b80 1 -- 172.21.15.46:0/2560065745 >> 172.21.15.46:0/2560065745 conn(0x562ce75a21c0 msgr2=0x562ce7648830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.942+0000 7f0826f66b80 1 -- 172.21.15.46:0/2560065745 shutdown_connections 2024-03-18T20:54:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.947+0000 7f0826f66b80 1 -- 172.21.15.46:0/2560065745 wait complete. 2024-03-18T20:54:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.948+0000 7f0826f66b80 1 -- start start 2024-03-18T20:54:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.948+0000 7f0826f66b80 1 Processor -- start 2024-03-18T20:54:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.948+0000 7f0826f66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762b570 0x562ce7639a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.948+0000 7f0826f66b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762be90 0x562ce7639fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.948+0000 7f0826f66b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 0x562ce763a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.948+0000 7f0826f66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce763de80 con 0x562ce762b570 2024-03-18T20:54:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.948+0000 7f0826f66b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce763dff0 con 0x562ce762be90 2024-03-18T20:54:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.949+0000 7f0826f66b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce76996e0 con 0x562ce7636db0 2024-03-18T20:54:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.949+0000 7f0826f66b80 10 client.0 _ll_register_callbacks cb 0x7f080401ad50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:54:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.949+0000 7f0813fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762b570 0x562ce7639a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.949+0000 7f0813fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762b570 0x562ce7639a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36628/0 (socket says 172.21.15.46:36628) 2024-03-18T20:54:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.949+0000 7f0813fff700 1 -- 172.21.15.46:0/3917819376 learned_addr learned my addr 172.21.15.46:0/3917819376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.949+0000 7f0813fff700 1 --2- 172.21.15.46:0/3917819376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 0x562ce763a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.954+0000 7f0813fff700 1 -- 172.21.15.46:0/3917819376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 msgr2=0x562ce763a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.954+0000 7f0813fff700 1 --2- 172.21.15.46:0/3917819376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 0x562ce763a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.954+0000 7f0813fff700 1 -- 172.21.15.46:0/3917819376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762be90 msgr2=0x562ce7639fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:54:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.954+0000 7f0813fff700 1 --2- 172.21.15.46:0/3917819376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762be90 0x562ce7639fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.954+0000 7f0813fff700 1 -- 172.21.15.46:0/3917819376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ce73acb00 con 0x562ce762b570 2024-03-18T20:54:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.956+0000 7f0813fff700 1 --2- 172.21.15.46:0/3917819376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762b570 0x562ce7639a70 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f0804000c00 tx=0x7f080402a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767f054f4070f946 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.956+0000 7f07f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:54:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.958+0000 7f07f3fff700 1 -- 172.21.15.46:0/3917819376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080402f040 con 0x562ce762b570 2024-03-18T20:54:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.958+0000 7f07f3fff700 1 -- 172.21.15.46:0/3917819376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f080403b030 con 0x562ce762b570 2024-03-18T20:54:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.962+0000 7f0826f66b80 1 -- 172.21.15.46:0/3917819376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f080c014c20 con 0x562ce762b570 2024-03-18T20:54:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.963+0000 7f07f3fff700 1 -- 172.21.15.46:0/3917819376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f080403b1f0 con 0x562ce762b570 2024-03-18T20:54:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.963+0000 7f0826f66b80 10 client.20469 fetch_fsmap learned FSMap version 83 2024-03-18T20:54:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.963+0000 7f0826f66b80 1 -- 172.21.15.46:0/3917819376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x562ce7404c70 con 0x562ce762b570 2024-03-18T20:54:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.963+0000 7f07f3fff700 1 -- 172.21.15.46:0/3917819376 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f080c014c20 con 0x562ce762b570 2024-03-18T20:54:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.963+0000 7f07f3fff700 1 -- 172.21.15.46:0/3917819376 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0804045070 con 0x562ce762b570 2024-03-18T20:54:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.963+0000 7f0826f66b80 10 client.20469 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:54:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.963+0000 7f0826f66b80 10 client.20469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:54:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.963+0000 7f0826f66b80 20 client.20469 populate_metadata read hostname 'smithi046' 2024-03-18T20:54:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.964+0000 7f0826f66b80 1 --2- 172.21.15.46:0/3917819376 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ce769ae60 0x562ce769d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.964+0000 7f0826f66b80 4 client.20469 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:54:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.964+0000 7f0826f66b80 1 -- 172.21.15.46:0/3917819376 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ce73cdc90 con 0x562ce769ae60 2024-03-18T20:54:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.964+0000 7f0818b6d700 1 --2- 172.21.15.46:0/3917819376 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ce769ae60 0x562ce769d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.977+0000 7f0818b6d700 1 --2- 172.21.15.46:0/3917819376 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ce769ae60 0x562ce769d280 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f943221567efa69 server_cookie=df1766eaf6717641 in_seq=0 out_seq=0 2024-03-18T20:54:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:11.978+0000 7f07f3fff700 10 client.20469 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:12.396 INFO:tasks.workunit.client.0.smithi046.stdout: 10 227681 37.92 MB/sec execute 168 sec latency 764.368 ms 2024-03-18T20:54:12.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:12 smithi155 ceph-mon[48344]: pgmap v569: 97 pgs: 16 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 47 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 582 op/s 2024-03-18T20:54:12.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:12 smithi046 ceph-mon[43495]: pgmap v569: 97 pgs: 16 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 47 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 582 op/s 2024-03-18T20:54:12.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:12 smithi073 ceph-mon[47384]: pgmap v569: 97 pgs: 16 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 47 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 582 op/s 2024-03-18T20:54:13.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.228+0000 7f07f3fff700 1 -- 172.21.15.46:0/3917819376 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ce73cdc90 con 0x562ce769ae60 2024-03-18T20:54:13.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.228+0000 7f07f3fff700 10 client.20469 handle_command_reply: tid=0 2024-03-18T20:54:13.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.229+0000 7f0826f66b80 1 client.20469 shutdown 2024-03-18T20:54:13.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.229+0000 7f0826f66b80 1 -- 172.21.15.46:0/3917819376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762b570 msgr2=0x562ce7639a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:13.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.229+0000 7f0826f66b80 1 --2- 172.21.15.46:0/3917819376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762b570 0x562ce7639a70 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f0804000c00 tx=0x7f080402a040 comp rx=0 tx=0).stop 2024-03-18T20:54:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.231+0000 7f0826f66b80 1 -- 172.21.15.46:0/3917819376 shutdown_connections 2024-03-18T20:54:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.231+0000 7f0826f66b80 1 --2- 172.21.15.46:0/3917819376 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562ce769ae60 0x562ce769d280 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.231+0000 7f0826f66b80 1 --2- 172.21.15.46:0/3917819376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562ce762b570 0x562ce7639a70 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.231+0000 7f0826f66b80 1 --2- 172.21.15.46:0/3917819376 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562ce7636db0 0x562ce763a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.231+0000 7f0826f66b80 1 --2- 172.21.15.46:0/3917819376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ce762be90 0x562ce7639fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.231+0000 7f0826f66b80 1 -- 172.21.15.46:0/3917819376 >> 172.21.15.46:0/3917819376 conn(0x562ce75a21c0 msgr2=0x562ce762ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.231+0000 7f07f3fff700 0 client.20469 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.232+0000 7f0826f66b80 1 -- 172.21.15.46:0/3917819376 shutdown_connections 2024-03-18T20:54:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:54:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:54:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:54:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:54:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.233+0000 7f0826f66b80 1 -- 172.21.15.46:0/3917819376 wait complete. 2024-03-18T20:54:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.233+0000 7f0826f66b80 20 client.20469 trim_cache size 0 max 16384 2024-03-18T20:54:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.235+0000 7f081add1700 1 -- 172.21.15.46:0/4093145664 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f07fc0a14a0 msgr2=0x7f07fc0a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.235+0000 7f081add1700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f07fc0a14a0 0x7f07fc0a3960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0808095b40 tx=0x7f08080b9040 comp rx=0 tx=0).stop 2024-03-18T20:54:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.235+0000 7f081add1700 1 -- 172.21.15.46:0/4093145664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081411a590 msgr2=0x7f0814119580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.235+0000 7f081add1700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081411a590 0x7f0814119580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f081406cdd0 tx=0x7f080800b750 comp rx=0 tx=0).stop 2024-03-18T20:54:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.239+0000 7f081add1700 1 -- 172.21.15.46:0/4093145664 shutdown_connections 2024-03-18T20:54:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.239+0000 7f081add1700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f07fc0a14a0 0x7f07fc0a3960 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.239+0000 7f081add1700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081410f4d0 0x7f0814115cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.239+0000 7f081add1700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f081411a590 0x7f0814119580 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.239+0000 7f081add1700 1 --2- 172.21.15.46:0/4093145664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f081410f940 0x7f08141161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.239+0000 7f081add1700 1 -- 172.21.15.46:0/4093145664 >> 172.21.15.46:0/4093145664 conn(0x7f081406bd80 msgr2=0x7f081410bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.244+0000 7f081add1700 1 -- 172.21.15.46:0/4093145664 shutdown_connections 2024-03-18T20:54:13.248 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:13.246+0000 7f081add1700 1 -- 172.21.15.46:0/4093145664 wait complete. 2024-03-18T20:54:13.396 INFO:tasks.workunit.client.0.smithi046.stdout: 10 227812 37.75 MB/sec execute 169 sec latency 1059.144 ms 2024-03-18T20:54:13.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:13 smithi155 ceph-mon[48344]: pgmap v570: 97 pgs: 16 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 47 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.8 MiB/s wr, 640 op/s 2024-03-18T20:54:13.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:13 smithi046 ceph-mon[43495]: pgmap v570: 97 pgs: 16 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 47 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.8 MiB/s wr, 640 op/s 2024-03-18T20:54:13.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:13 smithi073 ceph-mon[47384]: pgmap v570: 97 pgs: 16 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 47 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.8 MiB/s wr, 640 op/s 2024-03-18T20:54:14.396 INFO:tasks.workunit.client.0.smithi046.stdout: 10 228081 37.58 MB/sec execute 170 sec latency 155.356 ms 2024-03-18T20:54:15.396 INFO:tasks.workunit.client.0.smithi046.stdout: 10 228484 37.45 MB/sec execute 171 sec latency 145.760 ms 2024-03-18T20:54:16.397 INFO:tasks.workunit.client.0.smithi046.stdout: 10 229376 37.52 MB/sec execute 172 sec latency 77.353 ms 2024-03-18T20:54:16.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:16 smithi073 ceph-mon[47384]: pgmap v571: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 22 MiB/s wr, 940 op/s 2024-03-18T20:54:16.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:16 smithi155 ceph-mon[48344]: pgmap v571: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 22 MiB/s wr, 940 op/s 2024-03-18T20:54:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:16 smithi046 ceph-mon[43495]: pgmap v571: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 22 MiB/s wr, 940 op/s 2024-03-18T20:54:17.397 INFO:tasks.workunit.client.0.smithi046.stdout: 10 230240 37.58 MB/sec execute 173 sec latency 73.608 ms 2024-03-18T20:54:18.398 INFO:tasks.workunit.client.0.smithi046.stdout: 10 230988 37.62 MB/sec execute 174 sec latency 85.524 ms 2024-03-18T20:54:18.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:18 smithi073 ceph-mon[47384]: pgmap v572: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 22 MiB/s wr, 940 op/s 2024-03-18T20:54:18.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:18 smithi155 ceph-mon[48344]: pgmap v572: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 22 MiB/s wr, 940 op/s 2024-03-18T20:54:18.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:18 smithi046 ceph-mon[43495]: pgmap v572: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 22 MiB/s wr, 940 op/s 2024-03-18T20:54:19.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 231637 37.59 MB/sec execute 175 sec latency 109.669 ms 2024-03-18T20:54:20.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 232282 37.56 MB/sec execute 176 sec latency 94.892 ms 2024-03-18T20:54:20.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:20 smithi073 ceph-mon[47384]: pgmap v573: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 23 MiB/s wr, 1.19k op/s 2024-03-18T20:54:20.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:20 smithi073 ceph-mon[47384]: osdmap e106: 12 total, 12 up, 12 in 2024-03-18T20:54:20.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:20 smithi155 ceph-mon[48344]: pgmap v573: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 23 MiB/s wr, 1.19k op/s 2024-03-18T20:54:20.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:20 smithi155 ceph-mon[48344]: osdmap e106: 12 total, 12 up, 12 in 2024-03-18T20:54:20.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:20 smithi046 ceph-mon[43495]: pgmap v573: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 23 MiB/s wr, 1.19k op/s 2024-03-18T20:54:20.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:20 smithi046 ceph-mon[43495]: osdmap e106: 12 total, 12 up, 12 in 2024-03-18T20:54:21.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 233182 37.61 MB/sec execute 177 sec latency 128.304 ms 2024-03-18T20:54:22.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 233875 37.65 MB/sec execute 178 sec latency 113.893 ms 2024-03-18T20:54:22.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:22 smithi073 ceph-mon[47384]: pgmap v575: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 23 MiB/s wr, 1.01k op/s 2024-03-18T20:54:22.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:22 smithi155 ceph-mon[48344]: pgmap v575: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 23 MiB/s wr, 1.01k op/s 2024-03-18T20:54:22.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:22 smithi046 ceph-mon[43495]: pgmap v575: 97 pgs: 97 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 23 MiB/s wr, 1.01k op/s 2024-03-18T20:54:23.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 234562 37.63 MB/sec execute 179 sec latency 80.460 ms 2024-03-18T20:54:24.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 235236 37.61 MB/sec execute 180 sec latency 100.427 ms 2024-03-18T20:54:24.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:24 smithi073 ceph-mon[47384]: pgmap v576: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 20 MiB/s wr, 996 op/s 2024-03-18T20:54:24.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:24 smithi155 ceph-mon[48344]: pgmap v576: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 20 MiB/s wr, 996 op/s 2024-03-18T20:54:24.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:24 smithi046 ceph-mon[43495]: pgmap v576: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 20 MiB/s wr, 996 op/s 2024-03-18T20:54:25.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 235830 37.60 MB/sec execute 181 sec latency 190.905 ms 2024-03-18T20:54:26.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 236583 37.59 MB/sec execute 182 sec latency 89.177 ms 2024-03-18T20:54:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.428+0000 7efeabfbf700 1 -- 172.21.15.46:0/2716119365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4107530 msgr2=0x7efea4107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.428+0000 7efeabfbf700 1 --2- 172.21.15.46:0/2716119365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4107530 0x7efea4107910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efea0009990 tx=0x7efea0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.428+0000 7efeabfbf700 1 -- 172.21.15.46:0/2716119365 shutdown_connections 2024-03-18T20:54:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.428+0000 7efeabfbf700 1 --2- 172.21.15.46:0/2716119365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efea410c650 0x7efea41141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.428+0000 7efeabfbf700 1 --2- 172.21.15.46:0/2716119365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4107530 0x7efea4107910 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.428+0000 7efeabfbf700 1 --2- 172.21.15.46:0/2716119365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4107ee0 0x7efea410bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.428+0000 7efeabfbf700 1 -- 172.21.15.46:0/2716119365 >> 172.21.15.46:0/2716119365 conn(0x7efea4076ab0 msgr2=0x7efea4078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.428+0000 7efeabfbf700 1 -- 172.21.15.46:0/2716119365 shutdown_connections 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.428+0000 7efeabfbf700 1 -- 172.21.15.46:0/2716119365 wait complete. 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.429+0000 7efeabfbf700 1 Processor -- start 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.429+0000 7efeabfbf700 1 -- start start 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.429+0000 7efeabfbf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4107530 0x7efea4081220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.429+0000 7efeabfbf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efea410c650 0x7efea40817a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.429+0000 7efeabfbf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea407a340 0x7efea407a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.429+0000 7efeabfbf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea407afa0 con 0x7efea407a340 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.429+0000 7efeabfbf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea407b110 con 0x7efea4107530 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.429+0000 7efeabfbf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea407edb0 con 0x7efea410c650 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.429+0000 7efea9d5b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4107530 0x7efea4081220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.429+0000 7efea9d5b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4107530 0x7efea4081220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47058/0 (socket says 172.21.15.46:47058) 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.429+0000 7efea9d5b700 1 -- 172.21.15.46:0/1238913861 learned_addr learned my addr 172.21.15.46:0/1238913861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.429+0000 7efea955a700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efea410c650 0x7efea40817a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.430+0000 7efeaa55c700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea407a340 0x7efea407a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.430+0000 7efea9d5b700 1 -- 172.21.15.46:0/1238913861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efea410c650 msgr2=0x7efea40817a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.430+0000 7efea9d5b700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efea410c650 0x7efea40817a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.430+0000 7efea9d5b700 1 -- 172.21.15.46:0/1238913861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea407a340 msgr2=0x7efea407a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.430+0000 7efea9d5b700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea407a340 0x7efea407a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.430+0000 7efea9d5b700 1 -- 172.21.15.46:0/1238913861 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00095f0 con 0x7efea4107530 2024-03-18T20:54:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.430+0000 7efeaa55c700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea407a340 0x7efea407a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:54:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.430+0000 7efea9d5b700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4107530 0x7efea4081220 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efea0005910 tx=0x7efea0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.430+0000 7efe9affd700 1 -- 172.21.15.46:0/1238913861 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea001d070 con 0x7efea4107530 2024-03-18T20:54:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.430+0000 7efe9affd700 1 -- 172.21.15.46:0/1238913861 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea0022470 con 0x7efea4107530 2024-03-18T20:54:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.430+0000 7efe9affd700 1 -- 172.21.15.46:0/1238913861 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0017650 con 0x7efea4107530 2024-03-18T20:54:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.431+0000 7efeabfbf700 1 -- 172.21.15.46:0/1238913861 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea407f000 con 0x7efea4107530 2024-03-18T20:54:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.431+0000 7efeabfbf700 1 -- 172.21.15.46:0/1238913861 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea407f490 con 0x7efea4107530 2024-03-18T20:54:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.434+0000 7efe9affd700 1 -- 172.21.15.46:0/1238913861 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7efea00225e0 con 0x7efea4107530 2024-03-18T20:54:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.435+0000 7efe9affd700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efe900a1290 0x7efe900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.435+0000 7efe9affd700 1 -- 172.21.15.46:0/1238913861 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8556+0+0 (secure 0 0 0) 0x7efea00cd9f0 con 0x7efea4107530 2024-03-18T20:54:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.436+0000 7efeb8154b80 1 Processor -- start 2024-03-18T20:54:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.436+0000 7efea955a700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efe900a1290 0x7efe900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.437+0000 7efeb8154b80 1 -- start start 2024-03-18T20:54:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.437+0000 7efeb8154b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 0x55c265a23950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.437+0000 7efeb8154b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a23e90 0x55c265a2e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.437+0000 7efeb8154b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a2edb0 0x55c265a311a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.437+0000 7efeb8154b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26597d930 con 0x55c265a2edb0 2024-03-18T20:54:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.437+0000 7efeb8154b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c26597b2f0 con 0x55c265a23570 2024-03-18T20:54:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.437+0000 7efeb8154b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c265976be0 con 0x55c265a23e90 2024-03-18T20:54:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efea9d5b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 0x55c265a23950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efea955a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a23e90 0x55c265a2e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efea9d5b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 0x55c265a23950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47068/0 (socket says 172.21.15.46:47068) 2024-03-18T20:54:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efea9d5b700 1 -- 172.21.15.46:0/514396265 learned_addr learned my addr 172.21.15.46:0/514396265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efeaa55c700 1 --2- 172.21.15.46:0/514396265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a2edb0 0x55c265a311a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efea9d5b700 1 -- 172.21.15.46:0/514396265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a23e90 msgr2=0x55c265a2e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efea9d5b700 1 --2- 172.21.15.46:0/514396265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a23e90 0x55c265a2e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efea9d5b700 1 -- 172.21.15.46:0/514396265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a2edb0 msgr2=0x55c265a311a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efea9d5b700 1 --2- 172.21.15.46:0/514396265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a2edb0 0x55c265a311a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efea955a700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efe900a1290 0x7efe900a3750 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efea407ba20 tx=0x7efe94009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efea9d5b700 1 -- 172.21.15.46:0/514396265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2657f8350 con 0x55c265a23570 2024-03-18T20:54:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efea9d5b700 1 --2- 172.21.15.46:0/514396265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 0x55c265a23950 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efea00975e0 tx=0x7efea009aa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb989d13b28225ef server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efe8ffff700 1 -- 172.21.15.46:0/514396265 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea009b270 con 0x55c265a23570 2024-03-18T20:54:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.438+0000 7efe8ffff700 1 -- 172.21.15.46:0/514396265 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea009ab70 con 0x55c265a23570 2024-03-18T20:54:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.439+0000 7efe8ffff700 1 -- 172.21.15.46:0/514396265 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea009add0 con 0x55c265a23570 2024-03-18T20:54:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.439+0000 7efeb8154b80 1 -- 172.21.15.46:0/514396265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 msgr2=0x55c265a23950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.439+0000 7efeb8154b80 1 --2- 172.21.15.46:0/514396265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 0x55c265a23950 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efea00975e0 tx=0x7efea009aa10 comp rx=0 tx=0).stop 2024-03-18T20:54:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.440+0000 7efeb8154b80 1 -- 172.21.15.46:0/514396265 shutdown_connections 2024-03-18T20:54:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.440+0000 7efeb8154b80 1 --2- 172.21.15.46:0/514396265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a2edb0 0x55c265a311a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.440+0000 7efeb8154b80 1 --2- 172.21.15.46:0/514396265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 0x55c265a23950 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.440+0000 7efeb8154b80 1 --2- 172.21.15.46:0/514396265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a23e90 0x55c265a2e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.440+0000 7efeb8154b80 1 -- 172.21.15.46:0/514396265 >> 172.21.15.46:0/514396265 conn(0x55c26599a1c0 msgr2=0x55c265a1fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.440+0000 7efeb8154b80 1 -- 172.21.15.46:0/514396265 shutdown_connections 2024-03-18T20:54:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.440+0000 7efeb8154b80 1 -- 172.21.15.46:0/514396265 wait complete. 2024-03-18T20:54:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.440+0000 7efeb8154b80 1 -- start start 2024-03-18T20:54:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.441+0000 7efeb8154b80 1 Processor -- start 2024-03-18T20:54:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.441+0000 7efeb8154b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 0x55c265a2e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.441+0000 7efeb8154b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23e90 0x55c265a2ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.441+0000 7efeb8154b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a2edb0 0x55c265a2bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.441+0000 7efeb8154b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c265912d20 con 0x55c265a23570 2024-03-18T20:54:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.441+0000 7efeb8154b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2659129c0 con 0x55c265a23e90 2024-03-18T20:54:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.441+0000 7efeb8154b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2657fa640 con 0x55c265a2edb0 2024-03-18T20:54:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.441+0000 7efeb8154b80 10 client.0 _ll_register_callbacks cb 0x55c26577e340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:54:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.442+0000 7efea9d5b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 0x55c265a2e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.442+0000 7efea9d5b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 0x55c265a2e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59624/0 (socket says 172.21.15.46:59624) 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.442+0000 7efea9d5b700 1 -- 172.21.15.46:0/4238388920 learned_addr learned my addr 172.21.15.46:0/4238388920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.442+0000 7efeaa55c700 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a2edb0 0x55c265a2bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.442+0000 7efea955a700 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23e90 0x55c265a2ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.442+0000 7efeaa55c700 1 -- 172.21.15.46:0/4238388920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23e90 msgr2=0x55c265a2ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.442+0000 7efeaa55c700 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23e90 0x55c265a2ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.442+0000 7efeaa55c700 1 -- 172.21.15.46:0/4238388920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 msgr2=0x55c265a2e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.442+0000 7efeaa55c700 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 0x55c265a2e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.443+0000 7efeaa55c700 1 -- 172.21.15.46:0/4238388920 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea407f490 con 0x55c265a2edb0 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.443+0000 7efeaa55c700 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a2edb0 0x55c265a2bfe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efe9c007a80 tx=0x7efe9c007e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69f9f9818cf118e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.443+0000 7efe8d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.443+0000 7efe8d7fa700 1 -- 172.21.15.46:0/4238388920 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c011930 con 0x55c265a2edb0 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.443+0000 7efe8d7fa700 1 -- 172.21.15.46:0/4238388920 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8556+0+0 (secure 0 0 0) 0x7efe9c011a90 con 0x55c265a2edb0 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.443+0000 7efeb8154b80 1 -- 172.21.15.46:0/4238388920 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c265a176c0 con 0x55c265a2edb0 2024-03-18T20:54:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.443+0000 7efe8d7fa700 1 -- 172.21.15.46:0/4238388920 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe9c011c50 con 0x55c265a2edb0 2024-03-18T20:54:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.443+0000 7efeb8154b80 10 client.30163 fetch_fsmap learned FSMap version 83 2024-03-18T20:54:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.443+0000 7efeb8154b80 1 -- 172.21.15.46:0/4238388920 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55c2657a4b00 con 0x55c265a2edb0 2024-03-18T20:54:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.444+0000 7efe8d7fa700 1 -- 172.21.15.46:0/4238388920 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c265a176c0 con 0x55c265a2edb0 2024-03-18T20:54:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.444+0000 7efe8d7fa700 1 -- 172.21.15.46:0/4238388920 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7efe9c016070 con 0x55c265a2edb0 2024-03-18T20:54:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.444+0000 7efeb8154b80 10 client.30163 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:54:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.444+0000 7efeb8154b80 10 client.30163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:54:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.444+0000 7efeb8154b80 20 client.30163 populate_metadata read hostname 'smithi046' 2024-03-18T20:54:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.444+0000 7efeb8154b80 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c265a3e710 0x55c265a40b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.444+0000 7efeb8154b80 4 client.30163 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:54:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.444+0000 7efeb8154b80 1 -- 172.21.15.46:0/4238388920 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c26596f6a0 con 0x55c265a3e710 2024-03-18T20:54:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.444+0000 7efea9d5b700 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c265a3e710 0x55c265a40b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.445+0000 7efea9d5b700 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c265a3e710 0x55c265a40b30 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ccd7e557b0eb199 server_cookie=2f42eb5a67c80b70 in_seq=0 out_seq=0 2024-03-18T20:54:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.445+0000 7efe8d7fa700 10 client.30163 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.445+0000 7efe8d7fa700 1 -- 172.21.15.46:0/4238388920 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c26596f6a0 con 0x55c265a3e710 2024-03-18T20:54:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.445+0000 7efe8d7fa700 10 client.30163 handle_command_reply: tid=0 2024-03-18T20:54:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.445+0000 7efeb8154b80 1 client.30163 shutdown 2024-03-18T20:54:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.446+0000 7efeb8154b80 1 -- 172.21.15.46:0/4238388920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a2edb0 msgr2=0x55c265a2bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.446+0000 7efeb8154b80 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a2edb0 0x55c265a2bfe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efe9c007a80 tx=0x7efe9c007e40 comp rx=0 tx=0).stop 2024-03-18T20:54:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.446+0000 7efeb8154b80 1 -- 172.21.15.46:0/4238388920 shutdown_connections 2024-03-18T20:54:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.446+0000 7efeb8154b80 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c265a3e710 0x55c265a40b30 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.446+0000 7efeb8154b80 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 0x55c265a2e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.446+0000 7efeb8154b80 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a2edb0 0x55c265a2bfe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.446+0000 7efeb8154b80 1 --2- 172.21.15.46:0/4238388920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23e90 0x55c265a2ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.446+0000 7efeb8154b80 1 -- 172.21.15.46:0/4238388920 >> 172.21.15.46:0/4238388920 conn(0x55c26599a1c0 msgr2=0x55c265a1f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.446+0000 7efe8d7fa700 0 client.30163 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.446+0000 7efeb8154b80 1 -- 172.21.15.46:0/4238388920 shutdown_connections 2024-03-18T20:54:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.447+0000 7efeb8154b80 1 -- 172.21.15.46:0/4238388920 wait complete. 2024-03-18T20:54:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.447+0000 7efeb8154b80 20 client.30163 trim_cache size 0 max 16384 2024-03-18T20:54:26.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:26 smithi073 ceph-mon[47384]: pgmap v577: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.3 MiB/s wr, 798 op/s 2024-03-18T20:54:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.468+0000 7efeb8154b80 1 Processor -- start 2024-03-18T20:54:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.468+0000 7efeb8154b80 1 -- start start 2024-03-18T20:54:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efeb8154b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 0x55c265a1fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efeb8154b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a23e90 0x55c265a2b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efeb8154b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a2edb0 0x55c265a2bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efeb8154b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c265a2cd50 con 0x55c265a2edb0 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efeb8154b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c265a20550 con 0x55c265a23570 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efeb8154b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c265a1e7a0 con 0x55c265a23e90 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efea9d5b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 0x55c265a1fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efea955a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a23e90 0x55c265a2b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efea9d5b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 0x55c265a1fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47084/0 (socket says 172.21.15.46:47084) 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efea9d5b700 1 -- 172.21.15.46:0/3091552541 learned_addr learned my addr 172.21.15.46:0/3091552541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efea955a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a23e90 0x55c265a2b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:42570/0 (socket says 172.21.15.46:42570) 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efea9d5b700 1 -- 172.21.15.46:0/3091552541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a23e90 msgr2=0x55c265a2b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efea9d5b700 1 --2- 172.21.15.46:0/3091552541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a23e90 0x55c265a2b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efea9d5b700 1 -- 172.21.15.46:0/3091552541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a2edb0 msgr2=0x55c265a2bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efea9d5b700 1 --2- 172.21.15.46:0/3091552541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a2edb0 0x55c265a2bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.469+0000 7efea9d5b700 1 -- 172.21.15.46:0/3091552541 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c26593ce40 con 0x55c265a23570 2024-03-18T20:54:26.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.470+0000 7efea9d5b700 1 --2- 172.21.15.46:0/3091552541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 0x55c265a1fbd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efea0098ec0 tx=0x7efea001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d28dd8c9f2f370db server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.471+0000 7efe98ff9700 1 -- 172.21.15.46:0/3091552541 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0064070 con 0x55c265a23570 2024-03-18T20:54:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.471+0000 7efe98ff9700 1 -- 172.21.15.46:0/3091552541 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea00b7070 con 0x55c265a23570 2024-03-18T20:54:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.471+0000 7efe98ff9700 1 -- 172.21.15.46:0/3091552541 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea009a060 con 0x55c265a23570 2024-03-18T20:54:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.473+0000 7efeb8154b80 1 -- 172.21.15.46:0/3091552541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 msgr2=0x55c265a1fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:26.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.473+0000 7efeb8154b80 1 --2- 172.21.15.46:0/3091552541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 0x55c265a1fbd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efea0098ec0 tx=0x7efea001a040 comp rx=0 tx=0).stop 2024-03-18T20:54:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.474+0000 7efeb8154b80 1 -- 172.21.15.46:0/3091552541 shutdown_connections 2024-03-18T20:54:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.474+0000 7efeb8154b80 1 --2- 172.21.15.46:0/3091552541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a2edb0 0x55c265a2bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.474+0000 7efeb8154b80 1 --2- 172.21.15.46:0/3091552541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23570 0x55c265a1fbd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.474+0000 7efeb8154b80 1 --2- 172.21.15.46:0/3091552541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a23e90 0x55c265a2b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.474+0000 7efeb8154b80 1 -- 172.21.15.46:0/3091552541 >> 172.21.15.46:0/3091552541 conn(0x55c26599a1c0 msgr2=0x55c265a26d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.475+0000 7efeb8154b80 1 -- 172.21.15.46:0/3091552541 shutdown_connections 2024-03-18T20:54:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.475+0000 7efeb8154b80 1 -- 172.21.15.46:0/3091552541 wait complete. 2024-03-18T20:54:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.475+0000 7efeb8154b80 1 -- start start 2024-03-18T20:54:26.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.475+0000 7efeb8154b80 1 Processor -- start 2024-03-18T20:54:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efeb8154b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 0x55c265a28f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efeb8154b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23e90 0x55c265a294d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efeb8154b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a2edb0 0x55c265a27cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efeb8154b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c265a281f0 con 0x55c265a23570 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efeb8154b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c265a28360 con 0x55c265a23e90 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efeb8154b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c265a28520 con 0x55c265a2edb0 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efeb8154b80 10 client.0 _ll_register_callbacks cb 0x7efe9c009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efea9d5b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 0x55c265a28f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efea9d5b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 0x55c265a28f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59642/0 (socket says 172.21.15.46:59642) 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efea9d5b700 1 -- 172.21.15.46:0/3250259734 learned_addr learned my addr 172.21.15.46:0/3250259734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efeaa55c700 1 --2- 172.21.15.46:0/3250259734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a2edb0 0x55c265a27cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efea9d5b700 1 -- 172.21.15.46:0/3250259734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a2edb0 msgr2=0x55c265a27cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.476+0000 7efea9d5b700 1 --2- 172.21.15.46:0/3250259734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a2edb0 0x55c265a27cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.477+0000 7efea9d5b700 1 -- 172.21.15.46:0/3250259734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23e90 msgr2=0x55c265a294d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.477+0000 7efea9d5b700 1 --2- 172.21.15.46:0/3250259734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23e90 0x55c265a294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.477+0000 7efea9d5b700 1 -- 172.21.15.46:0/3250259734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2657f8350 con 0x55c265a23570 2024-03-18T20:54:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.477+0000 7efea9d5b700 1 --2- 172.21.15.46:0/3250259734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 0x55c265a28f90 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7efea009a960 tx=0x7efea0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e771ea4631dbc03 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.477+0000 7efe8d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.477+0000 7efe8d7fa700 1 -- 172.21.15.46:0/3250259734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0064030 con 0x55c265a23570 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.477+0000 7efe8d7fa700 1 -- 172.21.15.46:0/3250259734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8556+0+0 (secure 0 0 0) 0x7efea0065070 con 0x55c265a23570 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.477+0000 7efeb8154b80 1 -- 172.21.15.46:0/3250259734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efea009ab70 con 0x55c265a23570 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.477+0000 7efe8d7fa700 1 -- 172.21.15.46:0/3250259734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7efea0097280 con 0x55c265a23570 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.477+0000 7efeb8154b80 10 client.20481 fetch_fsmap learned FSMap version 83 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.477+0000 7efeb8154b80 1 -- 172.21.15.46:0/3250259734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55c2657fcc70 con 0x55c265a23570 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.478+0000 7efe8d7fa700 1 -- 172.21.15.46:0/3250259734 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea009ab70 con 0x55c265a23570 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.478+0000 7efe8d7fa700 1 -- 172.21.15.46:0/3250259734 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7efea009a060 con 0x55c265a23570 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.478+0000 7efeb8154b80 10 client.20481 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.478+0000 7efeb8154b80 10 client.20481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.478+0000 7efeb8154b80 20 client.20481 populate_metadata read hostname 'smithi046' 2024-03-18T20:54:26.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.478+0000 7efeb8154b80 1 --2- 172.21.15.46:0/3250259734 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c265a92f40 0x55c265a95330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.478+0000 7efeb8154b80 4 client.20481 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:54:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.478+0000 7efeb8154b80 1 -- 172.21.15.46:0/3250259734 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2657c5c90 con 0x55c265a92f40 2024-03-18T20:54:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.479+0000 7efea955a700 1 --2- 172.21.15.46:0/3250259734 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c265a92f40 0x55c265a95330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.480+0000 7efe8d7fa700 10 client.20481 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.480+0000 7efea955a700 1 --2- 172.21.15.46:0/3250259734 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c265a92f40 0x55c265a95330 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d22bd6552f76808 server_cookie=b7f97c8d58dd334e in_seq=0 out_seq=0 2024-03-18T20:54:26.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:26 smithi155 ceph-mon[48344]: pgmap v577: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.3 MiB/s wr, 798 op/s 2024-03-18T20:54:26.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:26 smithi046 ceph-mon[43495]: pgmap v577: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.3 MiB/s wr, 798 op/s 2024-03-18T20:54:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:54:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:54:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:54:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:54:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.744+0000 7efe8d7fa700 1 -- 172.21.15.46:0/3250259734 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2657c5c90 con 0x55c265a92f40 2024-03-18T20:54:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.744+0000 7efe8d7fa700 10 client.20481 handle_command_reply: tid=0 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.744+0000 7efeb8154b80 1 client.20481 shutdown 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.744+0000 7efeb8154b80 1 -- 172.21.15.46:0/3250259734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 msgr2=0x55c265a28f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.744+0000 7efeb8154b80 1 --2- 172.21.15.46:0/3250259734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 0x55c265a28f90 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7efea009a960 tx=0x7efea0009ca0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.745+0000 7efeb8154b80 1 -- 172.21.15.46:0/3250259734 shutdown_connections 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.745+0000 7efeb8154b80 1 --2- 172.21.15.46:0/3250259734 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c265a92f40 0x55c265a95330 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.745+0000 7efeb8154b80 1 --2- 172.21.15.46:0/3250259734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c265a23570 0x55c265a28f90 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.745+0000 7efeb8154b80 1 --2- 172.21.15.46:0/3250259734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c265a2edb0 0x55c265a27cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.745+0000 7efeb8154b80 1 --2- 172.21.15.46:0/3250259734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c265a23e90 0x55c265a294d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.745+0000 7efeb8154b80 1 -- 172.21.15.46:0/3250259734 >> 172.21.15.46:0/3250259734 conn(0x55c26599a1c0 msgr2=0x55c265a26d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.745+0000 7efe8d7fa700 0 client.20481 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.745+0000 7efeb8154b80 1 -- 172.21.15.46:0/3250259734 shutdown_connections 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.745+0000 7efeb8154b80 1 -- 172.21.15.46:0/3250259734 wait complete. 2024-03-18T20:54:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.745+0000 7efeb8154b80 20 client.20481 trim_cache size 0 max 16384 2024-03-18T20:54:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.746+0000 7efeabfbf700 1 -- 172.21.15.46:0/1238913861 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efe900a1290 msgr2=0x7efe900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.746+0000 7efeabfbf700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efe900a1290 0x7efe900a3750 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7efea407ba20 tx=0x7efe94009260 comp rx=0 tx=0).stop 2024-03-18T20:54:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.746+0000 7efeabfbf700 1 -- 172.21.15.46:0/1238913861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4107530 msgr2=0x7efea4081220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.746+0000 7efeabfbf700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4107530 0x7efea4081220 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efea0005910 tx=0x7efea0004730 comp rx=0 tx=0).stop 2024-03-18T20:54:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.750+0000 7efeabfbf700 1 -- 172.21.15.46:0/1238913861 shutdown_connections 2024-03-18T20:54:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.750+0000 7efeabfbf700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efe900a1290 0x7efe900a3750 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.750+0000 7efeabfbf700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea407a340 0x7efea407a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.750+0000 7efeabfbf700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4107530 0x7efea4081220 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.750+0000 7efeabfbf700 1 --2- 172.21.15.46:0/1238913861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efea410c650 0x7efea40817a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.750+0000 7efeabfbf700 1 -- 172.21.15.46:0/1238913861 >> 172.21.15.46:0/1238913861 conn(0x7efea4076ab0 msgr2=0x7efea4077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.750+0000 7efeabfbf700 1 -- 172.21.15.46:0/1238913861 shutdown_connections 2024-03-18T20:54:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:26.750+0000 7efeabfbf700 1 -- 172.21.15.46:0/1238913861 wait complete. 2024-03-18T20:54:27.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 237221 37.60 MB/sec execute 183 sec latency 148.319 ms 2024-03-18T20:54:28.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 237861 37.55 MB/sec execute 184 sec latency 89.270 ms 2024-03-18T20:54:28.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:28 smithi073 ceph-mon[47384]: pgmap v578: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.3 MiB/s wr, 798 op/s 2024-03-18T20:54:28.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:28 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:54:28.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:28 smithi155 ceph-mon[48344]: pgmap v578: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.3 MiB/s wr, 798 op/s 2024-03-18T20:54:28.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:28 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:54:28.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:28 smithi046 ceph-mon[43495]: pgmap v578: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.3 MiB/s wr, 798 op/s 2024-03-18T20:54:28.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:28 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:54:29.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 238609 37.59 MB/sec execute 185 sec latency 84.900 ms 2024-03-18T20:54:29.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:29 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:54:29.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:29 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:54:29.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:29 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:54:29.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:29 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:54:29.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:29 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:54:29.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:29 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:54:29.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:29 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:54:29.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:29 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:54:29.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:29 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:54:30.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 239311 37.60 MB/sec execute 186 sec latency 84.581 ms 2024-03-18T20:54:30.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:30 smithi073 ceph-mon[47384]: pgmap v579: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.5 MiB/s wr, 694 op/s 2024-03-18T20:54:30.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:30 smithi155 ceph-mon[48344]: pgmap v579: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.5 MiB/s wr, 694 op/s 2024-03-18T20:54:30.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:30 smithi046 ceph-mon[43495]: pgmap v579: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.5 MiB/s wr, 694 op/s 2024-03-18T20:54:31.400 INFO:tasks.workunit.client.0.smithi046.stdout: 10 240019 37.59 MB/sec execute 187 sec latency 90.528 ms 2024-03-18T20:54:32.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 240746 37.61 MB/sec execute 188 sec latency 77.145 ms 2024-03-18T20:54:32.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:32 smithi073 ceph-mon[47384]: pgmap v580: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 620 op/s 2024-03-18T20:54:32.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:32 smithi155 ceph-mon[48344]: pgmap v580: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 620 op/s 2024-03-18T20:54:32.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:32 smithi046 ceph-mon[43495]: pgmap v580: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 620 op/s 2024-03-18T20:54:33.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 241499 37.61 MB/sec execute 189 sec latency 67.516 ms 2024-03-18T20:54:34.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 242258 37.66 MB/sec execute 190 sec latency 81.093 ms 2024-03-18T20:54:34.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:34 smithi073 ceph-mon[47384]: pgmap v581: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.3 MiB/s wr, 671 op/s 2024-03-18T20:54:34.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:34 smithi155 ceph-mon[48344]: pgmap v581: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.3 MiB/s wr, 671 op/s 2024-03-18T20:54:34.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:34 smithi046 ceph-mon[43495]: pgmap v581: 97 pgs: 97 active+clean; 1.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.3 MiB/s wr, 671 op/s 2024-03-18T20:54:35.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 242933 37.59 MB/sec execute 191 sec latency 119.198 ms 2024-03-18T20:54:36.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 243615 37.64 MB/sec execute 192 sec latency 85.042 ms 2024-03-18T20:54:36.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:36 smithi073 ceph-mon[47384]: pgmap v582: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.7 MiB/s wr, 708 op/s 2024-03-18T20:54:36.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:36 smithi155 ceph-mon[48344]: pgmap v582: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.7 MiB/s wr, 708 op/s 2024-03-18T20:54:36.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:36 smithi046 ceph-mon[43495]: pgmap v582: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.7 MiB/s wr, 708 op/s 2024-03-18T20:54:37.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 244192 37.58 MB/sec execute 193 sec latency 74.857 ms 2024-03-18T20:54:38.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 244887 37.56 MB/sec execute 194 sec latency 141.957 ms 2024-03-18T20:54:38.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:38 smithi073 ceph-mon[47384]: pgmap v583: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.4 MiB/s wr, 561 op/s 2024-03-18T20:54:38.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:38 smithi155 ceph-mon[48344]: pgmap v583: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.4 MiB/s wr, 561 op/s 2024-03-18T20:54:38.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:38 smithi046 ceph-mon[43495]: pgmap v583: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.4 MiB/s wr, 561 op/s 2024-03-18T20:54:39.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 245673 37.62 MB/sec execute 195 sec latency 80.026 ms 2024-03-18T20:54:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.920+0000 7fe6646f1700 1 -- 172.21.15.46:0/798171035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe65c100ab0 msgr2=0x7fe65c104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.920+0000 7fe6646f1700 1 --2- 172.21.15.46:0/798171035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe65c100ab0 0x7fe65c104860 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0098f0 tx=0x7fe64c009c00 comp rx=0 tx=0).stop 2024-03-18T20:54:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.921+0000 7fe6646f1700 1 -- 172.21.15.46:0/798171035 shutdown_connections 2024-03-18T20:54:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.921+0000 7fe6646f1700 1 --2- 172.21.15.46:0/798171035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe65c104f40 0x7fe65c069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.921+0000 7fe6646f1700 1 --2- 172.21.15.46:0/798171035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe65c100100 0x7fe65c1004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.921+0000 7fe6646f1700 1 --2- 172.21.15.46:0/798171035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe65c100ab0 0x7fe65c104860 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.921+0000 7fe6646f1700 1 -- 172.21.15.46:0/798171035 >> 172.21.15.46:0/798171035 conn(0x7fe65c076a80 msgr2=0x7fe65c076e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.921+0000 7fe6646f1700 1 -- 172.21.15.46:0/798171035 shutdown_connections 2024-03-18T20:54:39.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.921+0000 7fe6646f1700 1 -- 172.21.15.46:0/798171035 wait complete. 2024-03-18T20:54:39.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.921+0000 7fe6646f1700 1 Processor -- start 2024-03-18T20:54:39.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.922+0000 7fe6646f1700 1 -- start start 2024-03-18T20:54:39.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.922+0000 7fe6646f1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe65c100100 0x7fe65c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.922+0000 7fe6646f1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe65c104f40 0x7fe65c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.922+0000 7fe6646f1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe65c19dc30 0x7fe65c19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.922+0000 7fe6646f1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c19e6b0 con 0x7fe65c100100 2024-03-18T20:54:39.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.922+0000 7fe6646f1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c19e820 con 0x7fe65c104f40 2024-03-18T20:54:39.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.922+0000 7fe6646f1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c19e9e0 con 0x7fe65c19dc30 2024-03-18T20:54:39.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.922+0000 7fe66248d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe65c100100 0x7fe65c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.922+0000 7fe66248d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe65c100100 0x7fe65c199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43990/0 (socket says 172.21.15.46:43990) 2024-03-18T20:54:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.922+0000 7fe66248d700 1 -- 172.21.15.46:0/1505515149 learned_addr learned my addr 172.21.15.46:0/1505515149 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.922+0000 7fe662c8e700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe65c19dc30 0x7fe65c19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.922+0000 7fe661c8c700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe65c104f40 0x7fe65c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.923+0000 7fe661c8c700 1 -- 172.21.15.46:0/1505515149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe65c19dc30 msgr2=0x7fe65c19e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.923+0000 7fe661c8c700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe65c19dc30 0x7fe65c19e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.923+0000 7fe661c8c700 1 -- 172.21.15.46:0/1505515149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe65c100100 msgr2=0x7fe65c199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.923+0000 7fe661c8c700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe65c100100 0x7fe65c199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.923+0000 7fe661c8c700 1 -- 172.21.15.46:0/1505515149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c009520 con 0x7fe65c104f40 2024-03-18T20:54:39.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.923+0000 7fe661c8c700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe65c104f40 0x7fe65c199890 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009230 tx=0x7fe64c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:39.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.923+0000 7fe6537fe700 1 -- 172.21.15.46:0/1505515149 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c02d070 con 0x7fe65c104f40 2024-03-18T20:54:39.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.923+0000 7fe6537fe700 1 -- 172.21.15.46:0/1505515149 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe64c00bb20 con 0x7fe65c104f40 2024-03-18T20:54:39.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.923+0000 7fe6537fe700 1 -- 172.21.15.46:0/1505515149 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c003900 con 0x7fe65c104f40 2024-03-18T20:54:39.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.923+0000 7fe6646f1700 1 -- 172.21.15.46:0/1505515149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c06a7d0 con 0x7fe65c104f40 2024-03-18T20:54:39.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.923+0000 7fe6646f1700 1 -- 172.21.15.46:0/1505515149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c06ac30 con 0x7fe65c104f40 2024-03-18T20:54:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.929+0000 7fe670886b80 1 Processor -- start 2024-03-18T20:54:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.929+0000 7fe6537fe700 1 -- 172.21.15.46:0/1505515149 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe64c031070 con 0x7fe65c104f40 2024-03-18T20:54:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.930+0000 7fe6537fe700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe6480a1390 0x7fe6480a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.930+0000 7fe6537fe700 1 -- 172.21.15.46:0/1505515149 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe64c0df510 con 0x7fe65c104f40 2024-03-18T20:54:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.931+0000 7fe670886b80 1 -- start start 2024-03-18T20:54:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.931+0000 7fe670886b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803570 0x55b557803950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.931+0000 7fe670886b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b55780e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.931+0000 7fe670886b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b5578111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.931+0000 7fe670886b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55775d930 con 0x55b557803570 2024-03-18T20:54:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.931+0000 7fe670886b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55775b2f0 con 0x55b55780edb0 2024-03-18T20:54:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.931+0000 7fe670886b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b557756be0 con 0x55b557803e90 2024-03-18T20:54:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.932+0000 7fe661c8c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b55780e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.932+0000 7fe662c8e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b5578111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.932+0000 7fe661c8c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b55780e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:38696/0 (socket says 172.21.15.46:38696) 2024-03-18T20:54:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.932+0000 7fe661c8c700 1 -- 172.21.15.46:0/3206939495 learned_addr learned my addr 172.21.15.46:0/3206939495 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.932+0000 7fe661c8c700 1 -- 172.21.15.46:0/3206939495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 msgr2=0x55b5578111a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.932+0000 7fe661c8c700 1 --2- 172.21.15.46:0/3206939495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b5578111a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.932+0000 7fe661c8c700 1 -- 172.21.15.46:0/3206939495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803570 msgr2=0x55b557803950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.932+0000 7fe661c8c700 1 --2- 172.21.15.46:0/3206939495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803570 0x55b557803950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.932+0000 7fe661c8c700 1 -- 172.21.15.46:0/3206939495 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5575d8350 con 0x55b557803e90 2024-03-18T20:54:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.932+0000 7fe661c8c700 1 --2- 172.21.15.46:0/3206939495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b55780e870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0b44f0 tx=0x7fe64c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea5b8720949f7a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.933+0000 7fe650ff9700 1 -- 172.21.15.46:0/3206939495 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0acd30 con 0x55b557803e90 2024-03-18T20:54:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.933+0000 7fe650ff9700 1 -- 172.21.15.46:0/3206939495 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe64c0ac630 con 0x55b557803e90 2024-03-18T20:54:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.933+0000 7fe650ff9700 1 -- 172.21.15.46:0/3206939495 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0b0120 con 0x55b557803e90 2024-03-18T20:54:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.934+0000 7fe662c8e700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe6480a1390 0x7fe6480a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.934+0000 7fe670886b80 1 -- 172.21.15.46:0/3206939495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 msgr2=0x55b55780e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.934+0000 7fe670886b80 1 --2- 172.21.15.46:0/3206939495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b55780e870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe64c0b44f0 tx=0x7fe64c009c00 comp rx=0 tx=0).stop 2024-03-18T20:54:39.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.935+0000 7fe670886b80 1 -- 172.21.15.46:0/3206939495 shutdown_connections 2024-03-18T20:54:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.935+0000 7fe670886b80 1 --2- 172.21.15.46:0/3206939495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803570 0x55b557803950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.935+0000 7fe670886b80 1 --2- 172.21.15.46:0/3206939495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b5578111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.935+0000 7fe670886b80 1 --2- 172.21.15.46:0/3206939495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b55780e870 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.935+0000 7fe670886b80 1 -- 172.21.15.46:0/3206939495 >> 172.21.15.46:0/3206939495 conn(0x55b55777a1c0 msgr2=0x55b5577ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.935+0000 7fe670886b80 1 -- 172.21.15.46:0/3206939495 shutdown_connections 2024-03-18T20:54:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.935+0000 7fe670886b80 1 -- 172.21.15.46:0/3206939495 wait complete. 2024-03-18T20:54:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.935+0000 7fe670886b80 1 -- start start 2024-03-18T20:54:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.935+0000 7fe670886b80 1 Processor -- start 2024-03-18T20:54:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.936+0000 7fe670886b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803570 0x55b55780e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.936+0000 7fe670886b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803e90 0x55b55780ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.936+0000 7fe670886b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b55780c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.936+0000 7fe670886b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5576f2d20 con 0x55b557803e90 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.936+0000 7fe670886b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5576f29c0 con 0x55b55780edb0 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.936+0000 7fe670886b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5575da640 con 0x55b557803570 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.936+0000 7fe670886b80 10 client.0 _ll_register_callbacks cb 0x55b55755e340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.936+0000 7fe662c8e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b55780c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.936+0000 7fe662c8e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b55780c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47560/0 (socket says 172.21.15.46:47560) 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.936+0000 7fe662c8e700 1 -- 172.21.15.46:0/2508385720 learned_addr learned my addr 172.21.15.46:0/2508385720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.936+0000 7fe66248d700 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803570 0x55b55780e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.937+0000 7fe661c8c700 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803e90 0x55b55780ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.937+0000 7fe662c8e700 1 -- 172.21.15.46:0/2508385720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803570 msgr2=0x55b55780e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.937+0000 7fe662c8e700 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803570 0x55b55780e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.937+0000 7fe662c8e700 1 -- 172.21.15.46:0/2508385720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803e90 msgr2=0x55b55780ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.937+0000 7fe662c8e700 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803e90 0x55b55780ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.937+0000 7fe662c8e700 1 -- 172.21.15.46:0/2508385720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c06ac30 con 0x55b55780edb0 2024-03-18T20:54:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.937+0000 7fe661c8c700 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803e90 0x55b55780ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:54:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.937+0000 7fe662c8e700 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b55780c080 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe65401a590 tx=0x7fe65400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79668aaa909b198e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.937+0000 7fe645ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:54:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.937+0000 7fe645ffb700 1 -- 172.21.15.46:0/2508385720 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654004bb0 con 0x55b55780edb0 2024-03-18T20:54:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.937+0000 7fe645ffb700 1 -- 172.21.15.46:0/2508385720 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe654004d10 con 0x55b55780edb0 2024-03-18T20:54:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.938+0000 7fe670886b80 1 -- 172.21.15.46:0/2508385720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5577f76c0 con 0x55b55780edb0 2024-03-18T20:54:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.938+0000 7fe645ffb700 1 -- 172.21.15.46:0/2508385720 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe654005020 con 0x55b55780edb0 2024-03-18T20:54:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.938+0000 7fe670886b80 10 client.30068 fetch_fsmap learned FSMap version 83 2024-03-18T20:54:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.938+0000 7fe670886b80 1 -- 172.21.15.46:0/2508385720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55b557584b00 con 0x55b55780edb0 2024-03-18T20:54:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.938+0000 7fe645ffb700 1 -- 172.21.15.46:0/2508385720 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5577f76c0 con 0x55b55780edb0 2024-03-18T20:54:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.939+0000 7fe645ffb700 1 -- 172.21.15.46:0/2508385720 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fe654026400 con 0x55b55780edb0 2024-03-18T20:54:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.939+0000 7fe670886b80 10 client.30068 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:54:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.939+0000 7fe670886b80 10 client.30068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:54:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.939+0000 7fe670886b80 20 client.30068 populate_metadata read hostname 'smithi046' 2024-03-18T20:54:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.939+0000 7fe670886b80 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b557826eb0 0x55b5578292d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.939+0000 7fe670886b80 4 client.30068 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:54:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.939+0000 7fe670886b80 1 -- 172.21.15.46:0/2508385720 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b55774f6a0 con 0x55b557826eb0 2024-03-18T20:54:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.940+0000 7fe66248d700 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b557826eb0 0x55b5578292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.940+0000 7fe662c8e700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe6480a1390 0x7fe6480a3850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe654009610 tx=0x7fe6540093a0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.944+0000 7fe66248d700 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b557826eb0 0x55b5578292d0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0e6f38f1167fb1b server_cookie=3931f5e1994435a6 in_seq=0 out_seq=0 2024-03-18T20:54:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.944+0000 7fe645ffb700 10 client.30068 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.945+0000 7fe645ffb700 1 -- 172.21.15.46:0/2508385720 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b55774f6a0 con 0x55b557826eb0 2024-03-18T20:54:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.945+0000 7fe645ffb700 10 client.30068 handle_command_reply: tid=0 2024-03-18T20:54:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.945+0000 7fe670886b80 1 client.30068 shutdown 2024-03-18T20:54:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.947+0000 7fe670886b80 1 -- 172.21.15.46:0/2508385720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 msgr2=0x55b55780c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.947+0000 7fe670886b80 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b55780c080 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe65401a590 tx=0x7fe65400b040 comp rx=0 tx=0).stop 2024-03-18T20:54:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.949+0000 7fe670886b80 1 -- 172.21.15.46:0/2508385720 shutdown_connections 2024-03-18T20:54:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.949+0000 7fe670886b80 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b557826eb0 0x55b5578292d0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.949+0000 7fe670886b80 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b55780c080 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.949+0000 7fe670886b80 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803570 0x55b55780e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.949+0000 7fe670886b80 1 --2- 172.21.15.46:0/2508385720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803e90 0x55b55780ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.949+0000 7fe645ffb700 0 client.30068 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.949+0000 7fe670886b80 1 -- 172.21.15.46:0/2508385720 >> 172.21.15.46:0/2508385720 conn(0x55b55777a1c0 msgr2=0x55b5577ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.949+0000 7fe670886b80 1 -- 172.21.15.46:0/2508385720 shutdown_connections 2024-03-18T20:54:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.949+0000 7fe670886b80 1 -- 172.21.15.46:0/2508385720 wait complete. 2024-03-18T20:54:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.950+0000 7fe670886b80 20 client.30068 trim_cache size 0 max 16384 2024-03-18T20:54:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.971+0000 7fe670886b80 1 Processor -- start 2024-03-18T20:54:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.973+0000 7fe670886b80 1 -- start start 2024-03-18T20:54:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.973+0000 7fe670886b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b557803570 0x55b5577ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.973+0000 7fe670886b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b55780a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.973+0000 7fe670886b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b55780edb0 0x55b5578150b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.973+0000 7fe670886b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b557804500 con 0x55b55780edb0 2024-03-18T20:54:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.973+0000 7fe670886b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b557804850 con 0x55b557803570 2024-03-18T20:54:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.973+0000 7fe670886b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5577fc200 con 0x55b557803e90 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.973+0000 7fe661c8c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b55780a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.973+0000 7fe662c8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b55780edb0 0x55b5578150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.974+0000 7fe661c8c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b55780a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:38702/0 (socket says 172.21.15.46:38702) 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.974+0000 7fe661c8c700 1 -- 172.21.15.46:0/3374869118 learned_addr learned my addr 172.21.15.46:0/3374869118 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.974+0000 7fe661c8c700 1 -- 172.21.15.46:0/3374869118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b557803570 msgr2=0x55b5577ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.974+0000 7fe661c8c700 1 --2- 172.21.15.46:0/3374869118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b557803570 0x55b5577ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.974+0000 7fe661c8c700 1 -- 172.21.15.46:0/3374869118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b55780edb0 msgr2=0x55b5578150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.974+0000 7fe661c8c700 1 --2- 172.21.15.46:0/3374869118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b55780edb0 0x55b5578150b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.974+0000 7fe661c8c700 1 -- 172.21.15.46:0/3374869118 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b55782df00 con 0x55b557803e90 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.974+0000 7fe662c8e700 1 --2- 172.21.15.46:0/3374869118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b55780edb0 0x55b5578150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.974+0000 7fe661c8c700 1 --2- 172.21.15.46:0/3374869118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b55780a9a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009810 tx=0x7fe64c0ac7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec6da393726987ca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.974+0000 7fe6517fa700 1 -- 172.21.15.46:0/3374869118 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0ac400 con 0x55b557803e90 2024-03-18T20:54:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.975+0000 7fe6517fa700 1 -- 172.21.15.46:0/3374869118 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe64c0b0460 con 0x55b557803e90 2024-03-18T20:54:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.975+0000 7fe6517fa700 1 -- 172.21.15.46:0/3374869118 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0dce60 con 0x55b557803e90 2024-03-18T20:54:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.975+0000 7fe670886b80 1 -- 172.21.15.46:0/3374869118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 msgr2=0x55b55780a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.975+0000 7fe670886b80 1 --2- 172.21.15.46:0/3374869118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b55780a9a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009810 tx=0x7fe64c0ac7a0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.976+0000 7fe670886b80 1 -- 172.21.15.46:0/3374869118 shutdown_connections 2024-03-18T20:54:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.976+0000 7fe670886b80 1 --2- 172.21.15.46:0/3374869118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b55780edb0 0x55b5578150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.976+0000 7fe670886b80 1 --2- 172.21.15.46:0/3374869118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b557803570 0x55b5577ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.976+0000 7fe670886b80 1 --2- 172.21.15.46:0/3374869118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b55780a9a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.976+0000 7fe670886b80 1 -- 172.21.15.46:0/3374869118 >> 172.21.15.46:0/3374869118 conn(0x55b55777a1c0 msgr2=0x55b55780b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.976+0000 7fe670886b80 1 -- 172.21.15.46:0/3374869118 shutdown_connections 2024-03-18T20:54:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.976+0000 7fe670886b80 1 -- 172.21.15.46:0/3374869118 wait complete. 2024-03-18T20:54:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.976+0000 7fe670886b80 1 -- start start 2024-03-18T20:54:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.976+0000 7fe670886b80 1 Processor -- start 2024-03-18T20:54:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.980+0000 7fe670886b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803570 0x55b5578111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.980+0000 7fe670886b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b5578116e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.980+0000 7fe670886b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b557879cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.980+0000 7fe670886b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b557811ff0 con 0x55b557803570 2024-03-18T20:54:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.980+0000 7fe670886b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55787a210 con 0x55b55780edb0 2024-03-18T20:54:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.980+0000 7fe670886b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55787a350 con 0x55b557803e90 2024-03-18T20:54:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.980+0000 7fe670886b80 10 client.0 _ll_register_callbacks cb 0x7fe65401cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:54:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe662c8e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b557879cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe661c8c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b5578116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe661c8c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b5578116e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:38708/0 (socket says 172.21.15.46:38708) 2024-03-18T20:54:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe661c8c700 1 -- 172.21.15.46:0/392500875 learned_addr learned my addr 172.21.15.46:0/392500875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe66248d700 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803570 0x55b5578111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe662c8e700 1 -- 172.21.15.46:0/392500875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 msgr2=0x55b5578116e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe662c8e700 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b5578116e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe662c8e700 1 -- 172.21.15.46:0/392500875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803570 msgr2=0x55b5578111a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe662c8e700 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803570 0x55b5578111a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe662c8e700 1 -- 172.21.15.46:0/392500875 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b557584b00 con 0x55b55780edb0 2024-03-18T20:54:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe661c8c700 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b5578116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:54:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe662c8e700 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b557879cd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe654007730 tx=0x7fe654027640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bfa98f0f2b43913 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.981+0000 7fe645ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:54:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.983+0000 7fe645ffb700 1 -- 172.21.15.46:0/392500875 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654004ea0 con 0x55b55780edb0 2024-03-18T20:54:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.983+0000 7fe645ffb700 1 -- 172.21.15.46:0/392500875 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fe654003a00 con 0x55b55780edb0 2024-03-18T20:54:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.983+0000 7fe670886b80 1 -- 172.21.15.46:0/392500875 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe64c0ac630 con 0x55b55780edb0 2024-03-18T20:54:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.983+0000 7fe645ffb700 1 -- 172.21.15.46:0/392500875 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6540099e0 con 0x55b55780edb0 2024-03-18T20:54:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.983+0000 7fe670886b80 10 client.30071 fetch_fsmap learned FSMap version 83 2024-03-18T20:54:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.983+0000 7fe670886b80 1 -- 172.21.15.46:0/392500875 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55b5575d8350 con 0x55b55780edb0 2024-03-18T20:54:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.984+0000 7fe645ffb700 1 -- 172.21.15.46:0/392500875 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe64c0ac630 con 0x55b55780edb0 2024-03-18T20:54:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.984+0000 7fe645ffb700 1 -- 172.21.15.46:0/392500875 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fe654004030 con 0x55b55780edb0 2024-03-18T20:54:39.988 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.984+0000 7fe670886b80 10 client.30071 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:54:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.984+0000 7fe670886b80 10 client.30071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:54:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.984+0000 7fe670886b80 20 client.30071 populate_metadata read hostname 'smithi046' 2024-03-18T20:54:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.984+0000 7fe670886b80 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b55787ba20 0x55b55787de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.984+0000 7fe670886b80 4 client.30071 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:54:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.984+0000 7fe670886b80 1 -- 172.21.15.46:0/392500875 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b5575a5c90 con 0x55b55787ba20 2024-03-18T20:54:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.984+0000 7fe66248d700 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b55787ba20 0x55b55787de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.984+0000 7fe645ffb700 10 client.30071 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:39.989 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:39.985+0000 7fe66248d700 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b55787ba20 0x55b55787de40 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b57f8e3cbcbb024 server_cookie=acaea60d49776513 in_seq=0 out_seq=0 2024-03-18T20:54:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.275+0000 7fe645ffb700 1 -- 172.21.15.46:0/392500875 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b5575a5c90 con 0x55b55787ba20 2024-03-18T20:54:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.275+0000 7fe645ffb700 10 client.30071 handle_command_reply: tid=0 2024-03-18T20:54:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.275+0000 7fe670886b80 1 client.30071 shutdown 2024-03-18T20:54:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.276+0000 7fe670886b80 1 -- 172.21.15.46:0/392500875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 msgr2=0x55b557879cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.276+0000 7fe670886b80 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b557879cd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe654007730 tx=0x7fe654027640 comp rx=0 tx=0).stop 2024-03-18T20:54:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.276+0000 7fe670886b80 1 -- 172.21.15.46:0/392500875 shutdown_connections 2024-03-18T20:54:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.276+0000 7fe670886b80 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b55787ba20 0x55b55787de40 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.276+0000 7fe670886b80 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b557803570 0x55b5578111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.276+0000 7fe670886b80 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b55780edb0 0x55b557879cd0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.276+0000 7fe645ffb700 0 client.30071 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.276+0000 7fe670886b80 1 --2- 172.21.15.46:0/392500875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b557803e90 0x55b5578116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.276+0000 7fe670886b80 1 -- 172.21.15.46:0/392500875 >> 172.21.15.46:0/392500875 conn(0x55b55777a1c0 msgr2=0x55b5577febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.276+0000 7fe670886b80 1 -- 172.21.15.46:0/392500875 shutdown_connections 2024-03-18T20:54:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.276+0000 7fe670886b80 1 -- 172.21.15.46:0/392500875 wait complete. 2024-03-18T20:54:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.276+0000 7fe670886b80 20 client.30071 trim_cache size 0 max 16384 2024-03-18T20:54:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:54:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:54:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:54:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:54:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.277+0000 7fe6646f1700 1 -- 172.21.15.46:0/1505515149 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe6480a1390 msgr2=0x7fe6480a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.277+0000 7fe6646f1700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe6480a1390 0x7fe6480a3850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe654009610 tx=0x7fe6540093a0 comp rx=0 tx=0).stop 2024-03-18T20:54:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.277+0000 7fe6646f1700 1 -- 172.21.15.46:0/1505515149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe65c104f40 msgr2=0x7fe65c199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.277+0000 7fe6646f1700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe65c104f40 0x7fe65c199890 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009230 tx=0x7fe64c00fa00 comp rx=0 tx=0).stop 2024-03-18T20:54:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.277+0000 7fe6646f1700 1 -- 172.21.15.46:0/1505515149 shutdown_connections 2024-03-18T20:54:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.277+0000 7fe6646f1700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fe6480a1390 0x7fe6480a3850 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.277+0000 7fe6646f1700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe65c100100 0x7fe65c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.277+0000 7fe6646f1700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe65c19dc30 0x7fe65c19e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.277+0000 7fe6646f1700 1 --2- 172.21.15.46:0/1505515149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe65c104f40 0x7fe65c199890 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.277+0000 7fe6646f1700 1 -- 172.21.15.46:0/1505515149 >> 172.21.15.46:0/1505515149 conn(0x7fe65c076a80 msgr2=0x7fe65c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.277+0000 7fe6646f1700 1 -- 172.21.15.46:0/1505515149 shutdown_connections 2024-03-18T20:54:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:40.277+0000 7fe6646f1700 1 -- 172.21.15.46:0/1505515149 wait complete. 2024-03-18T20:54:40.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 246225 37.55 MB/sec execute 196 sec latency 257.385 ms 2024-03-18T20:54:40.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:40 smithi073 ceph-mon[47384]: pgmap v584: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 800 op/s 2024-03-18T20:54:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:40 smithi155 ceph-mon[48344]: pgmap v584: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 800 op/s 2024-03-18T20:54:40.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:40 smithi046 ceph-mon[43495]: pgmap v584: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 800 op/s 2024-03-18T20:54:41.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 246885 37.57 MB/sec execute 197 sec latency 122.416 ms 2024-03-18T20:54:42.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 247251 37.47 MB/sec execute 198 sec latency 304.895 ms 2024-03-18T20:54:42.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:42 smithi073 ceph-mon[47384]: pgmap v585: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.5 MiB/s wr, 478 op/s 2024-03-18T20:54:42.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:42 smithi155 ceph-mon[48344]: pgmap v585: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.5 MiB/s wr, 478 op/s 2024-03-18T20:54:42.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:42 smithi046 ceph-mon[43495]: pgmap v585: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.5 MiB/s wr, 478 op/s 2024-03-18T20:54:43.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 247286 37.29 MB/sec execute 199 sec latency 1058.317 ms 2024-03-18T20:54:44.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 247575 37.17 MB/sec execute 200 sec latency 1402.897 ms 2024-03-18T20:54:44.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:44 smithi155 ceph-mon[48344]: pgmap v586: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.5 MiB/s wr, 591 op/s 2024-03-18T20:54:44.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:44 smithi046 ceph-mon[43495]: pgmap v586: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.5 MiB/s wr, 591 op/s 2024-03-18T20:54:44.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:44 smithi073 ceph-mon[47384]: pgmap v586: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.5 MiB/s wr, 591 op/s 2024-03-18T20:54:45.401 INFO:tasks.workunit.client.0.smithi046.stdout: 10 248017 37.14 MB/sec execute 201 sec latency 105.238 ms 2024-03-18T20:54:46.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 248411 37.05 MB/sec execute 202 sec latency 152.582 ms 2024-03-18T20:54:46.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:46 smithi073 ceph-mon[47384]: pgmap v587: 97 pgs: 97 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 21 MiB/s wr, 887 op/s 2024-03-18T20:54:46.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:46 smithi155 ceph-mon[48344]: pgmap v587: 97 pgs: 97 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 21 MiB/s wr, 887 op/s 2024-03-18T20:54:46.548 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:46 smithi046 ceph-mon[43495]: pgmap v587: 97 pgs: 97 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 21 MiB/s wr, 887 op/s 2024-03-18T20:54:47.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 249067 37.01 MB/sec execute 203 sec latency 107.027 ms 2024-03-18T20:54:48.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 249820 37.06 MB/sec execute 204 sec latency 130.856 ms 2024-03-18T20:54:48.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:48 smithi073 ceph-mon[47384]: pgmap v588: 97 pgs: 97 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 740 op/s 2024-03-18T20:54:48.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:48 smithi155 ceph-mon[48344]: pgmap v588: 97 pgs: 97 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 740 op/s 2024-03-18T20:54:48.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:48 smithi046 ceph-mon[43495]: pgmap v588: 97 pgs: 97 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 19 MiB/s wr, 740 op/s 2024-03-18T20:54:49.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 250524 37.04 MB/sec execute 205 sec latency 108.882 ms 2024-03-18T20:54:50.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 251332 37.09 MB/sec execute 206 sec latency 64.353 ms 2024-03-18T20:54:50.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:50 smithi073 ceph-mon[47384]: pgmap v589: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 24 MiB/s wr, 1.17k op/s 2024-03-18T20:54:50.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:50 smithi155 ceph-mon[48344]: pgmap v589: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 24 MiB/s wr, 1.17k op/s 2024-03-18T20:54:50.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:50 smithi046 ceph-mon[43495]: pgmap v589: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 24 MiB/s wr, 1.17k op/s 2024-03-18T20:54:51.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 251946 37.08 MB/sec execute 207 sec latency 79.324 ms 2024-03-18T20:54:52.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 252473 37.00 MB/sec execute 208 sec latency 173.252 ms 2024-03-18T20:54:52.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:52 smithi073 ceph-mon[47384]: pgmap v590: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 933 op/s 2024-03-18T20:54:52.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:52 smithi155 ceph-mon[48344]: pgmap v590: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 933 op/s 2024-03-18T20:54:52.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:52 smithi046 ceph-mon[43495]: pgmap v590: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 933 op/s 2024-03-18T20:54:53.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 253083 36.99 MB/sec execute 209 sec latency 82.846 ms 2024-03-18T20:54:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.453+0000 7f9f75f0c700 1 -- 172.21.15.46:0/53145675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70104f20 msgr2=0x7f9f7010caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.453+0000 7f9f75f0c700 1 --2- 172.21.15.46:0/53145675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70104f20 0x7f9f7010caa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9f680098f0 tx=0x7f9f68009c00 comp rx=0 tx=0).stop 2024-03-18T20:54:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.459+0000 7f9f75f0c700 1 -- 172.21.15.46:0/53145675 shutdown_connections 2024-03-18T20:54:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.459+0000 7f9f75f0c700 1 --2- 172.21.15.46:0/53145675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70104f20 0x7f9f7010caa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.459+0000 7f9f75f0c700 1 --2- 172.21.15.46:0/53145675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f700ffe00 0x7f9f701001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.459+0000 7f9f75f0c700 1 --2- 172.21.15.46:0/53145675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f701007b0 0x7f9f701047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.459+0000 7f9f75f0c700 1 -- 172.21.15.46:0/53145675 >> 172.21.15.46:0/53145675 conn(0x7f9f700fb680 msgr2=0x7f9f700fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.459+0000 7f9f75f0c700 1 -- 172.21.15.46:0/53145675 shutdown_connections 2024-03-18T20:54:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.459+0000 7f9f75f0c700 1 -- 172.21.15.46:0/53145675 wait complete. 2024-03-18T20:54:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.460+0000 7f9f75f0c700 1 Processor -- start 2024-03-18T20:54:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.460+0000 7f9f75f0c700 1 -- start start 2024-03-18T20:54:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.460+0000 7f9f75f0c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700ffe00 0x7f9f7010d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.460+0000 7f9f75f0c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70104f20 0x7f9f7010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.460+0000 7f9f75f0c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f7010f050 0x7f9f7010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.460+0000 7f9f75f0c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f701a9d80 con 0x7f9f700ffe00 2024-03-18T20:54:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.460+0000 7f9f75f0c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f701a9ef0 con 0x7f9f7010f050 2024-03-18T20:54:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.460+0000 7f9f75f0c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f701aa0b0 con 0x7f9f70104f20 2024-03-18T20:54:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.461+0000 7f9f6f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700ffe00 0x7f9f7010d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.461+0000 7f9f6f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700ffe00 0x7f9f7010d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51156/0 (socket says 172.21.15.46:51156) 2024-03-18T20:54:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.461+0000 7f9f6f7fe700 1 -- 172.21.15.46:0/126225516 learned_addr learned my addr 172.21.15.46:0/126225516 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.461+0000 7f9f66ffd700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70104f20 0x7f9f7010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.461+0000 7f9f66ffd700 1 -- 172.21.15.46:0/126225516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f7010f050 msgr2=0x7f9f7010de60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:54:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.461+0000 7f9f66ffd700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f7010f050 0x7f9f7010de60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.461+0000 7f9f66ffd700 1 -- 172.21.15.46:0/126225516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700ffe00 msgr2=0x7f9f7010d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.461+0000 7f9f66ffd700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700ffe00 0x7f9f7010d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.461+0000 7f9f66ffd700 1 -- 172.21.15.46:0/126225516 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f580095f0 con 0x7f9f70104f20 2024-03-18T20:54:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.462+0000 7f9f66ffd700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70104f20 0x7f9f7010d920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9f58005fd0 tx=0x7f9f5800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.462+0000 7f9f6d7fa700 1 -- 172.21.15.46:0/126225516 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800cc90 con 0x7f9f70104f20 2024-03-18T20:54:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.462+0000 7f9f6d7fa700 1 -- 172.21.15.46:0/126225516 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f580044e0 con 0x7f9f70104f20 2024-03-18T20:54:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.462+0000 7f9f6d7fa700 1 -- 172.21.15.46:0/126225516 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58010430 con 0x7f9f70104f20 2024-03-18T20:54:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.462+0000 7f9f75f0c700 1 -- 172.21.15.46:0/126225516 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f68009520 con 0x7f9f70104f20 2024-03-18T20:54:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.462+0000 7f9f75f0c700 1 -- 172.21.15.46:0/126225516 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f701aa6c0 con 0x7f9f70104f20 2024-03-18T20:54:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.465+0000 7f9f6d7fa700 1 -- 172.21.15.46:0/126225516 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9f58003f40 con 0x7f9f70104f20 2024-03-18T20:54:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.465+0000 7f9f6d7fa700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9f5c0a1390 0x7f9f5c0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.466+0000 7f9f6d7fa700 1 -- 172.21.15.46:0/126225516 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f9f58014070 con 0x7f9f70104f20 2024-03-18T20:54:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.466+0000 7f9f6f7fe700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9f5c0a1390 0x7f9f5c0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.467+0000 7f9f6f7fe700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9f5c0a1390 0x7f9f5c0a3850 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009c10 tx=0x7f9f6000b040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.468+0000 7f9f820a1b80 1 Processor -- start 2024-03-18T20:54:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f820a1b80 1 -- start start 2024-03-18T20:54:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f820a1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6570 0x559bc4ad6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f820a1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ad6e90 0x559bc4ae1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f820a1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae1db0 0x559bc4ae41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f820a1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bc4a30930 con 0x559bc4ae1db0 2024-03-18T20:54:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f820a1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bc4a2e2f0 con 0x559bc4ad6e90 2024-03-18T20:54:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f820a1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bc4a29be0 con 0x559bc4ad6570 2024-03-18T20:54:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f66ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ad6e90 0x559bc4ae1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f66ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ad6e90 0x559bc4ae1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55298/0 (socket says 172.21.15.46:55298) 2024-03-18T20:54:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f66ffd700 1 -- 172.21.15.46:0/898576944 learned_addr learned my addr 172.21.15.46:0/898576944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f6f7fe700 1 --2- 172.21.15.46:0/898576944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6570 0x559bc4ad6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f6ffff700 1 --2- 172.21.15.46:0/898576944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae1db0 0x559bc4ae41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f66ffd700 1 -- 172.21.15.46:0/898576944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6570 msgr2=0x559bc4ad6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f66ffd700 1 --2- 172.21.15.46:0/898576944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6570 0x559bc4ad6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f66ffd700 1 -- 172.21.15.46:0/898576944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae1db0 msgr2=0x559bc4ae41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f66ffd700 1 --2- 172.21.15.46:0/898576944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae1db0 0x559bc4ae41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f66ffd700 1 -- 172.21.15.46:0/898576944 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bc48ab350 con 0x559bc4ad6e90 2024-03-18T20:54:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.469+0000 7f9f66ffd700 1 --2- 172.21.15.46:0/898576944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ad6e90 0x559bc4ae1870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f58096cc0 tx=0x7f9f580966c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=814861a2e8e4b0a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.470+0000 7f9f65ffb700 1 -- 172.21.15.46:0/898576944 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5809a950 con 0x559bc4ad6e90 2024-03-18T20:54:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.470+0000 7f9f65ffb700 1 -- 172.21.15.46:0/898576944 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f5809a250 con 0x559bc4ad6e90 2024-03-18T20:54:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.470+0000 7f9f65ffb700 1 -- 172.21.15.46:0/898576944 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58098700 con 0x559bc4ad6e90 2024-03-18T20:54:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.471+0000 7f9f820a1b80 1 -- 172.21.15.46:0/898576944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ad6e90 msgr2=0x559bc4ae1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.471+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/898576944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ad6e90 0x559bc4ae1870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f58096cc0 tx=0x7f9f580966c0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.471+0000 7f9f820a1b80 1 -- 172.21.15.46:0/898576944 shutdown_connections 2024-03-18T20:54:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.471+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/898576944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae1db0 0x559bc4ae41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.471+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/898576944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6570 0x559bc4ad6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.471+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/898576944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ad6e90 0x559bc4ae1870 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.471+0000 7f9f820a1b80 1 -- 172.21.15.46:0/898576944 >> 172.21.15.46:0/898576944 conn(0x559bc4a4d1c0 msgr2=0x559bc4ad2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.471+0000 7f9f820a1b80 1 -- 172.21.15.46:0/898576944 shutdown_connections 2024-03-18T20:54:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.471+0000 7f9f820a1b80 1 -- 172.21.15.46:0/898576944 wait complete. 2024-03-18T20:54:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.471+0000 7f9f820a1b80 1 -- start start 2024-03-18T20:54:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.472+0000 7f9f820a1b80 1 Processor -- start 2024-03-18T20:54:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.472+0000 7f9f820a1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6570 0x559bc4ae1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.472+0000 7f9f820a1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ad6e90 0x559bc4adeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.472+0000 7f9f820a1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae1db0 0x559bc4adf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.472+0000 7f9f820a1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bc49c5d20 con 0x559bc4ae1db0 2024-03-18T20:54:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.472+0000 7f9f820a1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bc49c59c0 con 0x559bc4ad6e90 2024-03-18T20:54:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.472+0000 7f9f820a1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bc48ad640 con 0x559bc4ad6570 2024-03-18T20:54:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.472+0000 7f9f820a1b80 10 client.0 _ll_register_callbacks cb 0x559bc4831340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:54:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.472+0000 7f9f6f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6570 0x559bc4ae1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.472+0000 7f9f6f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6570 0x559bc4ae1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:56782/0 (socket says 172.21.15.46:56782) 2024-03-18T20:54:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.472+0000 7f9f6f7fe700 1 -- 172.21.15.46:0/2129477029 learned_addr learned my addr 172.21.15.46:0/2129477029 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.473+0000 7f9f6f7fe700 1 -- 172.21.15.46:0/2129477029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ad6e90 msgr2=0x559bc4adeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.473+0000 7f9f6f7fe700 1 --2- 172.21.15.46:0/2129477029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ad6e90 0x559bc4adeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.473+0000 7f9f6f7fe700 1 -- 172.21.15.46:0/2129477029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae1db0 msgr2=0x559bc4adf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.473+0000 7f9f6f7fe700 1 --2- 172.21.15.46:0/2129477029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae1db0 0x559bc4adf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.473+0000 7f9f6f7fe700 1 -- 172.21.15.46:0/2129477029 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f701aa6c0 con 0x559bc4ad6570 2024-03-18T20:54:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.473+0000 7f9f6f7fe700 1 --2- 172.21.15.46:0/2129477029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6570 0x559bc4ae1850 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9f600045f0 tx=0x7f9f60004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84240dbd257602b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.473+0000 7f9f4effd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:54:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.474+0000 7f9f4effd700 1 -- 172.21.15.46:0/2129477029 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6000e040 con 0x559bc4ad6570 2024-03-18T20:54:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.474+0000 7f9f4effd700 1 -- 172.21.15.46:0/2129477029 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f9f60017aa0 con 0x559bc4ad6570 2024-03-18T20:54:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.474+0000 7f9f820a1b80 1 -- 172.21.15.46:0/2129477029 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bc4aca6c0 con 0x559bc4ad6570 2024-03-18T20:54:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.474+0000 7f9f4effd700 1 -- 172.21.15.46:0/2129477029 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f6000f2d0 con 0x559bc4ad6570 2024-03-18T20:54:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.474+0000 7f9f820a1b80 10 client.30190 fetch_fsmap learned FSMap version 83 2024-03-18T20:54:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.474+0000 7f9f820a1b80 1 -- 172.21.15.46:0/2129477029 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x559bc4857b00 con 0x559bc4ad6570 2024-03-18T20:54:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.475+0000 7f9f4effd700 1 -- 172.21.15.46:0/2129477029 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559bc4aca6c0 con 0x559bc4ad6570 2024-03-18T20:54:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.475+0000 7f9f4effd700 1 -- 172.21.15.46:0/2129477029 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f9f60015410 con 0x559bc4ad6570 2024-03-18T20:54:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.475+0000 7f9f820a1b80 10 client.30190 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:54:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.475+0000 7f9f820a1b80 10 client.30190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:54:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.475+0000 7f9f820a1b80 20 client.30190 populate_metadata read hostname 'smithi046' 2024-03-18T20:54:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.475+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/2129477029 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559bc4b02ab0 0x559bc4b04ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.475+0000 7f9f820a1b80 4 client.30190 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:54:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.475+0000 7f9f820a1b80 1 -- 172.21.15.46:0/2129477029 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bc4a226a0 con 0x559bc4b02ab0 2024-03-18T20:54:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.475+0000 7f9f66ffd700 1 --2- 172.21.15.46:0/2129477029 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559bc4b02ab0 0x559bc4b04ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.476+0000 7f9f66ffd700 1 --2- 172.21.15.46:0/2129477029 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559bc4b02ab0 0x559bc4b04ed0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7501afc5afa803f server_cookie=903ff7d2ee50fb80 in_seq=0 out_seq=0 2024-03-18T20:54:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.476+0000 7f9f4effd700 10 client.30190 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.476+0000 7f9f4effd700 1 -- 172.21.15.46:0/2129477029 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559bc4a226a0 con 0x559bc4b02ab0 2024-03-18T20:54:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.476+0000 7f9f4effd700 10 client.30190 handle_command_reply: tid=0 2024-03-18T20:54:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.477+0000 7f9f820a1b80 1 client.30190 shutdown 2024-03-18T20:54:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.477+0000 7f9f820a1b80 1 -- 172.21.15.46:0/2129477029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6570 msgr2=0x559bc4ae1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.477+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/2129477029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6570 0x559bc4ae1850 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9f600045f0 tx=0x7f9f60004ad0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.480+0000 7f9f820a1b80 1 -- 172.21.15.46:0/2129477029 shutdown_connections 2024-03-18T20:54:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.480+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/2129477029 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559bc4b02ab0 0x559bc4b04ed0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.480+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/2129477029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae1db0 0x559bc4adf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.480+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/2129477029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6570 0x559bc4ae1850 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.480+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/2129477029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ad6e90 0x559bc4adeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.480+0000 7f9f820a1b80 1 -- 172.21.15.46:0/2129477029 >> 172.21.15.46:0/2129477029 conn(0x559bc4a4d1c0 msgr2=0x559bc4ad2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.480+0000 7f9f4effd700 0 client.30190 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.480+0000 7f9f820a1b80 1 -- 172.21.15.46:0/2129477029 shutdown_connections 2024-03-18T20:54:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.480+0000 7f9f820a1b80 1 -- 172.21.15.46:0/2129477029 wait complete. 2024-03-18T20:54:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.480+0000 7f9f820a1b80 20 client.30190 trim_cache size 0 max 16384 2024-03-18T20:54:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.505+0000 7f9f820a1b80 1 Processor -- start 2024-03-18T20:54:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.505+0000 7f9f820a1b80 1 -- start start 2024-03-18T20:54:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f820a1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ad6570 0x559bc4ad6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f820a1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6e90 0x559bc4ade3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f820a1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ae1db0 0x559bc4ae8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f820a1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bc4ae8f80 con 0x559bc4ad6570 2024-03-18T20:54:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f820a1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bc4add090 con 0x559bc4ae1db0 2024-03-18T20:54:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f820a1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bc4ada980 con 0x559bc4ad6e90 2024-03-18T20:54:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f6ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ae1db0 0x559bc4ae8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f6ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ae1db0 0x559bc4ae8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55318/0 (socket says 172.21.15.46:55318) 2024-03-18T20:54:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f6ffff700 1 -- 172.21.15.46:0/676950979 learned_addr learned my addr 172.21.15.46:0/676950979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f6ffff700 1 -- 172.21.15.46:0/676950979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6e90 msgr2=0x559bc4ade3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:54:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f6ffff700 1 --2- 172.21.15.46:0/676950979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6e90 0x559bc4ade3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f6ffff700 1 -- 172.21.15.46:0/676950979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ad6570 msgr2=0x559bc4ad6950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:54:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f6ffff700 1 --2- 172.21.15.46:0/676950979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ad6570 0x559bc4ad6950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.506+0000 7f9f6ffff700 1 -- 172.21.15.46:0/676950979 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bc49efe40 con 0x559bc4ae1db0 2024-03-18T20:54:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.507+0000 7f9f6ffff700 1 --2- 172.21.15.46:0/676950979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ae1db0 0x559bc4ae8a40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9f680098f0 tx=0x7f9f68005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58150789b8f12cd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.507+0000 7f9f667fc700 1 -- 172.21.15.46:0/676950979 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6801c040 con 0x559bc4ae1db0 2024-03-18T20:54:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.507+0000 7f9f667fc700 1 -- 172.21.15.46:0/676950979 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f6800fcb0 con 0x559bc4ae1db0 2024-03-18T20:54:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.507+0000 7f9f667fc700 1 -- 172.21.15.46:0/676950979 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6800bb90 con 0x559bc4ae1db0 2024-03-18T20:54:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.508+0000 7f9f820a1b80 1 -- 172.21.15.46:0/676950979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ae1db0 msgr2=0x559bc4ae8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.508+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/676950979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ae1db0 0x559bc4ae8a40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9f680098f0 tx=0x7f9f68005b40 comp rx=0 tx=0).stop 2024-03-18T20:54:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.508+0000 7f9f820a1b80 1 -- 172.21.15.46:0/676950979 shutdown_connections 2024-03-18T20:54:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.508+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/676950979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ad6570 0x559bc4ad6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.508+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/676950979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ae1db0 0x559bc4ae8a40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.508+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/676950979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ad6e90 0x559bc4ade3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.508+0000 7f9f820a1b80 1 -- 172.21.15.46:0/676950979 >> 172.21.15.46:0/676950979 conn(0x559bc4a4d1c0 msgr2=0x559bc4b09870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.510+0000 7f9f820a1b80 1 -- 172.21.15.46:0/676950979 shutdown_connections 2024-03-18T20:54:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.513+0000 7f9f820a1b80 1 -- 172.21.15.46:0/676950979 wait complete. 2024-03-18T20:54:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.513+0000 7f9f820a1b80 1 -- start start 2024-03-18T20:54:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.514+0000 7f9f820a1b80 1 Processor -- start 2024-03-18T20:54:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.514+0000 7f9f820a1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ae1db0 0x559bc4ae8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.514+0000 7f9f820a1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae86c0 0x559bc4ae6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.514+0000 7f9f820a1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ae6b60 0x559bc4ae6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.514+0000 7f9f820a1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bc4ae8c50 con 0x559bc4ae86c0 2024-03-18T20:54:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.514+0000 7f9f820a1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bc4b50900 con 0x559bc4ae1db0 2024-03-18T20:54:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.514+0000 7f9f820a1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bc4b50ac0 con 0x559bc4ae6b60 2024-03-18T20:54:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.514+0000 7f9f820a1b80 10 client.0 _ll_register_callbacks cb 0x7f9f60003da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:54:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.515+0000 7f9f66ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae86c0 0x559bc4ae6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.515+0000 7f9f66ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae86c0 0x559bc4ae6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51190/0 (socket says 172.21.15.46:51190) 2024-03-18T20:54:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.515+0000 7f9f66ffd700 1 -- 172.21.15.46:0/3473814665 learned_addr learned my addr 172.21.15.46:0/3473814665 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:54:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.515+0000 7f9f6ffff700 1 --2- 172.21.15.46:0/3473814665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ae6b60 0x559bc4ae6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.515+0000 7f9f66ffd700 1 -- 172.21.15.46:0/3473814665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ae6b60 msgr2=0x559bc4ae6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:54:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.515+0000 7f9f66ffd700 1 --2- 172.21.15.46:0/3473814665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ae6b60 0x559bc4ae6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.515+0000 7f9f66ffd700 1 -- 172.21.15.46:0/3473814665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ae1db0 msgr2=0x559bc4ae8180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:54:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.515+0000 7f9f66ffd700 1 --2- 172.21.15.46:0/3473814665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ae1db0 0x559bc4ae8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.515+0000 7f9f66ffd700 1 -- 172.21.15.46:0/3473814665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f70068f90 con 0x559bc4ae86c0 2024-03-18T20:54:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.515+0000 7f9f66ffd700 1 --2- 172.21.15.46:0/3473814665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae86c0 0x559bc4ae6620 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800c5b0 tx=0x7f9f5800eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c434932b9e8b0cc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:54:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.515+0000 7f9f4effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:54:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.516+0000 7f9f4effd700 1 -- 172.21.15.46:0/3473814665 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f580a26d0 con 0x559bc4ae86c0 2024-03-18T20:54:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.516+0000 7f9f4effd700 1 -- 172.21.15.46:0/3473814665 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f9f580985f0 con 0x559bc4ae86c0 2024-03-18T20:54:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.516+0000 7f9f820a1b80 1 -- 172.21.15.46:0/3473814665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f5809a250 con 0x559bc4ae86c0 2024-03-18T20:54:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.516+0000 7f9f4effd700 1 -- 172.21.15.46:0/3473814665 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f580cab10 con 0x559bc4ae86c0 2024-03-18T20:54:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.516+0000 7f9f820a1b80 10 client.20505 fetch_fsmap learned FSMap version 83 2024-03-18T20:54:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.516+0000 7f9f820a1b80 1 -- 172.21.15.46:0/3473814665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x559bc48afc70 con 0x559bc4ae86c0 2024-03-18T20:54:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.523+0000 7f9f4effd700 1 -- 172.21.15.46:0/3473814665 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f5809a250 con 0x559bc4ae86c0 2024-03-18T20:54:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.523+0000 7f9f4effd700 1 -- 172.21.15.46:0/3473814665 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f9f58099740 con 0x559bc4ae86c0 2024-03-18T20:54:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.523+0000 7f9f820a1b80 10 client.20505 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:54:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.523+0000 7f9f820a1b80 10 client.20505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:54:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.523+0000 7f9f820a1b80 20 client.20505 populate_metadata read hostname 'smithi046' 2024-03-18T20:54:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.523+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/3473814665 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559bc4b51e90 0x559bc4b522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:54:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.523+0000 7f9f820a1b80 4 client.20505 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:54:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.523+0000 7f9f820a1b80 1 -- 172.21.15.46:0/3473814665 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559bc4878c90 con 0x559bc4b51e90 2024-03-18T20:54:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.525+0000 7f9f6f7fe700 1 --2- 172.21.15.46:0/3473814665 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559bc4b51e90 0x559bc4b522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:54:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.525+0000 7f9f4effd700 10 client.20505 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.525+0000 7f9f6f7fe700 1 --2- 172.21.15.46:0/3473814665 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559bc4b51e90 0x559bc4b522a0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4412672d18570dd server_cookie=c811aaecdf95cb92 in_seq=0 out_seq=0 2024-03-18T20:54:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.748+0000 7f9f4effd700 1 -- 172.21.15.46:0/3473814665 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559bc4878c90 con 0x559bc4b51e90 2024-03-18T20:54:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.748+0000 7f9f4effd700 10 client.20505 handle_command_reply: tid=0 2024-03-18T20:54:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.748+0000 7f9f820a1b80 1 client.20505 shutdown 2024-03-18T20:54:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.748+0000 7f9f820a1b80 1 -- 172.21.15.46:0/3473814665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae86c0 msgr2=0x559bc4ae6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.748+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/3473814665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae86c0 0x559bc4ae6620 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800c5b0 tx=0x7f9f5800eb10 comp rx=0 tx=0).stop 2024-03-18T20:54:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.748+0000 7f9f820a1b80 1 -- 172.21.15.46:0/3473814665 shutdown_connections 2024-03-18T20:54:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.748+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/3473814665 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x559bc4b51e90 0x559bc4b522a0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.748+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/3473814665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559bc4ae6b60 0x559bc4ae6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.748+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/3473814665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559bc4ae1db0 0x559bc4ae8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.748+0000 7f9f820a1b80 1 --2- 172.21.15.46:0/3473814665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559bc4ae86c0 0x559bc4ae6620 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.748+0000 7f9f4effd700 0 client.20505 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:54:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.748+0000 7f9f820a1b80 1 -- 172.21.15.46:0/3473814665 >> 172.21.15.46:0/3473814665 conn(0x559bc4a4d1c0 msgr2=0x559bc4ad1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.749+0000 7f9f820a1b80 1 -- 172.21.15.46:0/3473814665 shutdown_connections 2024-03-18T20:54:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.749+0000 7f9f820a1b80 1 -- 172.21.15.46:0/3473814665 wait complete. 2024-03-18T20:54:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.749+0000 7f9f820a1b80 20 client.20505 trim_cache size 0 max 16384 2024-03-18T20:54:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:54:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:54:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:54:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:54:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.749+0000 7f9f75f0c700 1 -- 172.21.15.46:0/126225516 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9f5c0a1390 msgr2=0x7f9f5c0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.749+0000 7f9f75f0c700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9f5c0a1390 0x7f9f5c0a3850 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009c10 tx=0x7f9f6000b040 comp rx=0 tx=0).stop 2024-03-18T20:54:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.749+0000 7f9f75f0c700 1 -- 172.21.15.46:0/126225516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70104f20 msgr2=0x7f9f7010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:54:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.749+0000 7f9f75f0c700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70104f20 0x7f9f7010d920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9f58005fd0 tx=0x7f9f5800ee70 comp rx=0 tx=0).stop 2024-03-18T20:54:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.751+0000 7f9f75f0c700 1 -- 172.21.15.46:0/126225516 shutdown_connections 2024-03-18T20:54:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.751+0000 7f9f75f0c700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9f5c0a1390 0x7f9f5c0a3850 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.751+0000 7f9f75f0c700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700ffe00 0x7f9f7010d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.751+0000 7f9f75f0c700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f7010f050 0x7f9f7010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.751+0000 7f9f75f0c700 1 --2- 172.21.15.46:0/126225516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f70104f20 0x7f9f7010d920 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:54:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.751+0000 7f9f75f0c700 1 -- 172.21.15.46:0/126225516 >> 172.21.15.46:0/126225516 conn(0x7f9f700fb680 msgr2=0x7f9f700fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:54:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.751+0000 7f9f75f0c700 1 -- 172.21.15.46:0/126225516 shutdown_connections 2024-03-18T20:54:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:54:53.752+0000 7f9f75f0c700 1 -- 172.21.15.46:0/126225516 wait complete. 2024-03-18T20:54:54.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 253730 36.97 MB/sec execute 210 sec latency 172.054 ms 2024-03-18T20:54:54.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:54 smithi073 ceph-mon[47384]: pgmap v591: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 21 MiB/s wr, 992 op/s 2024-03-18T20:54:54.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:54 smithi155 ceph-mon[48344]: pgmap v591: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 21 MiB/s wr, 992 op/s 2024-03-18T20:54:54.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:54 smithi046 ceph-mon[43495]: pgmap v591: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 21 MiB/s wr, 992 op/s 2024-03-18T20:54:55.405 INFO:tasks.workunit.client.0.smithi046.stdout: 10 254475 37.03 MB/sec execute 211 sec latency 57.384 ms 2024-03-18T20:54:56.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 255043 36.99 MB/sec execute 212 sec latency 75.525 ms 2024-03-18T20:54:56.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:56 smithi073 ceph-mon[47384]: pgmap v592: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 1.07k op/s 2024-03-18T20:54:56.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:56 smithi155 ceph-mon[48344]: pgmap v592: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 1.07k op/s 2024-03-18T20:54:56.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:56 smithi046 ceph-mon[43495]: pgmap v592: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 21 MiB/s wr, 1.07k op/s 2024-03-18T20:54:57.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 255713 36.97 MB/sec execute 213 sec latency 75.204 ms 2024-03-18T20:54:58.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 255973 36.85 MB/sec execute 214 sec latency 700.471 ms 2024-03-18T20:54:58.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:54:58 smithi073 ceph-mon[47384]: pgmap v593: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.8 MiB/s wr, 686 op/s 2024-03-18T20:54:58.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:54:58 smithi155 ceph-mon[48344]: pgmap v593: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.8 MiB/s wr, 686 op/s 2024-03-18T20:54:58.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:54:58 smithi046 ceph-mon[43495]: pgmap v593: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.8 MiB/s wr, 686 op/s 2024-03-18T20:54:59.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 256290 36.75 MB/sec execute 215 sec latency 706.648 ms 2024-03-18T20:55:00.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 257145 36.81 MB/sec execute 216 sec latency 115.376 ms 2024-03-18T20:55:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:00 smithi073 ceph-mon[47384]: pgmap v594: 97 pgs: 97 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-03-18T20:55:00.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:00 smithi155 ceph-mon[48344]: pgmap v594: 97 pgs: 97 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-03-18T20:55:00.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:00 smithi046 ceph-mon[43495]: pgmap v594: 97 pgs: 97 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-03-18T20:55:01.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 257745 36.77 MB/sec execute 217 sec latency 125.154 ms 2024-03-18T20:55:02.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 257857 36.64 MB/sec execute 218 sec latency 198.328 ms 2024-03-18T20:55:02.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:02 smithi073 ceph-mon[47384]: pgmap v595: 97 pgs: 97 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 12 MiB/s wr, 605 op/s 2024-03-18T20:55:02.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:02 smithi155 ceph-mon[48344]: pgmap v595: 97 pgs: 97 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 12 MiB/s wr, 605 op/s 2024-03-18T20:55:02.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:02 smithi046 ceph-mon[43495]: pgmap v595: 97 pgs: 97 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 12 MiB/s wr, 605 op/s 2024-03-18T20:55:03.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 258152 36.57 MB/sec execute 219 sec latency 167.120 ms 2024-03-18T20:55:04.402 INFO:tasks.workunit.client.0.smithi046.stdout: 10 258352 36.45 MB/sec execute 220 sec latency 484.207 ms 2024-03-18T20:55:04.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:04 smithi073 ceph-mon[47384]: pgmap v596: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 757 op/s 2024-03-18T20:55:04.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:04 smithi073 ceph-mon[47384]: osdmap e107: 12 total, 12 up, 12 in 2024-03-18T20:55:04.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:04 smithi155 ceph-mon[48344]: pgmap v596: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 757 op/s 2024-03-18T20:55:04.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:04 smithi155 ceph-mon[48344]: osdmap e107: 12 total, 12 up, 12 in 2024-03-18T20:55:04.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:04 smithi046 ceph-mon[43495]: pgmap v596: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 14 MiB/s wr, 757 op/s 2024-03-18T20:55:04.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:04 smithi046 ceph-mon[43495]: osdmap e107: 12 total, 12 up, 12 in 2024-03-18T20:55:05.403 INFO:tasks.workunit.client.0.smithi046.stdout: 10 258604 36.35 MB/sec execute 221 sec latency 448.479 ms 2024-03-18T20:55:06.406 INFO:tasks.workunit.client.0.smithi046.stdout: 10 259011 36.27 MB/sec execute 222 sec latency 146.541 ms 2024-03-18T20:55:06.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:06 smithi073 ceph-mon[47384]: pgmap v598: 97 pgs: 32 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 51 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 957 op/s 2024-03-18T20:55:06.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:06 smithi155 ceph-mon[48344]: pgmap v598: 97 pgs: 32 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 51 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 957 op/s 2024-03-18T20:55:06.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:06 smithi046 ceph-mon[43495]: pgmap v598: 97 pgs: 32 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 51 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 957 op/s 2024-03-18T20:55:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.926+0000 7fd89e2f0700 1 -- 172.21.15.46:0/3771699532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd898106e30 msgr2=0x7fd898107210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.926+0000 7fd89e2f0700 1 --2- 172.21.15.46:0/3771699532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd898106e30 0x7fd898107210 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd8900099c0 tx=0x7fd890009cd0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.926+0000 7fd89e2f0700 1 -- 172.21.15.46:0/3771699532 shutdown_connections 2024-03-18T20:55:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.926+0000 7fd89e2f0700 1 --2- 172.21.15.46:0/3771699532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8980ffbe0 0x7fd89810dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.926+0000 7fd89e2f0700 1 --2- 172.21.15.46:0/3771699532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd898106e30 0x7fd898107210 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.926+0000 7fd89e2f0700 1 --2- 172.21.15.46:0/3771699532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8980ff220 0x7fd8980ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.926+0000 7fd89e2f0700 1 -- 172.21.15.46:0/3771699532 >> 172.21.15.46:0/3771699532 conn(0x7fd8980761b0 msgr2=0x7fd8980fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.927+0000 7fd89e2f0700 1 -- 172.21.15.46:0/3771699532 shutdown_connections 2024-03-18T20:55:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.927+0000 7fd89e2f0700 1 -- 172.21.15.46:0/3771699532 wait complete. 2024-03-18T20:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.928+0000 7fd89e2f0700 1 Processor -- start 2024-03-18T20:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.928+0000 7fd89e2f0700 1 -- start start 2024-03-18T20:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.928+0000 7fd89e2f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8980ff220 0x7fd898199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.928+0000 7fd89e2f0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8980ffbe0 0x7fd89819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.928+0000 7fd89e2f0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd898106e30 0x7fd89810cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.928+0000 7fd89e2f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89810d3c0 con 0x7fd8980ff220 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.928+0000 7fd89e2f0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89810d530 con 0x7fd898106e30 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.928+0000 7fd89e2f0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89810d6f0 con 0x7fd8980ffbe0 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8980ffbe0 0x7fd89819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8980ffbe0 0x7fd89819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37174/0 (socket says 172.21.15.46:37174) 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd8977fe700 1 -- 172.21.15.46:0/21541162 learned_addr learned my addr 172.21.15.46:0/21541162 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd89c88d700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd898106e30 0x7fd89810cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd8977fe700 1 -- 172.21.15.46:0/21541162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd898106e30 msgr2=0x7fd89810cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd8977fe700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd898106e30 0x7fd89810cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd8977fe700 1 -- 172.21.15.46:0/21541162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8980ff220 msgr2=0x7fd898199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd8977fe700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8980ff220 0x7fd898199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd8977fe700 1 -- 172.21.15.46:0/21541162 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8900095f0 con 0x7fd8980ffbe0 2024-03-18T20:55:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd8977fe700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8980ffbe0 0x7fd89819a3e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd88800ba50 tx=0x7fd88800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd8957fa700 1 -- 172.21.15.46:0/21541162 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88800c9b0 con 0x7fd8980ffbe0 2024-03-18T20:55:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd89e2f0700 1 -- 172.21.15.46:0/21541162 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89810d9d0 con 0x7fd8980ffbe0 2024-03-18T20:55:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.929+0000 7fd8957fa700 1 -- 172.21.15.46:0/21541162 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd888014440 con 0x7fd8980ffbe0 2024-03-18T20:55:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.930+0000 7fd8957fa700 1 -- 172.21.15.46:0/21541162 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8880075d0 con 0x7fd8980ffbe0 2024-03-18T20:55:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.930+0000 7fd89e2f0700 1 -- 172.21.15.46:0/21541162 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89819dec0 con 0x7fd8980ffbe0 2024-03-18T20:55:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.933+0000 7fd8957fa700 1 -- 172.21.15.46:0/21541162 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd88800cb10 con 0x7fd8980ffbe0 2024-03-18T20:55:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.933+0000 7fd8957fa700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd8840a1310 0x7fd8840a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.934+0000 7fd8957fa700 1 -- 172.21.15.46:0/21541162 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fd888006e40 con 0x7fd8980ffbe0 2024-03-18T20:55:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.934+0000 7fd897fff700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd8840a1310 0x7fd8840a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.934+0000 7fd897fff700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd8840a1310 0x7fd8840a37d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd8900098e0 tx=0x7fd890009f10 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.934+0000 7fd8aa485b80 1 Processor -- start 2024-03-18T20:55:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd8aa485b80 1 -- start start 2024-03-18T20:55:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd8aa485b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5570 0x560974cd5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd8aa485b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974cd5e90 0x560974ce0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd8aa485b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974ce0db0 0x560974ce31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd8aa485b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974c2f930 con 0x560974ce0db0 2024-03-18T20:55:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd8aa485b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974c2d2f0 con 0x560974cd5570 2024-03-18T20:55:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd8aa485b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974c28be0 con 0x560974cd5e90 2024-03-18T20:55:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd89c88d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974ce0db0 0x560974ce31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd89c88d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974ce0db0 0x560974ce31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44926/0 (socket says 172.21.15.46:44926) 2024-03-18T20:55:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd89c88d700 1 -- 172.21.15.46:0/3348803089 learned_addr learned my addr 172.21.15.46:0/3348803089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd897fff700 1 --2- 172.21.15.46:0/3348803089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5570 0x560974cd5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd8977fe700 1 --2- 172.21.15.46:0/3348803089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974cd5e90 0x560974ce0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd89c88d700 1 -- 172.21.15.46:0/3348803089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974cd5e90 msgr2=0x560974ce0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd89c88d700 1 --2- 172.21.15.46:0/3348803089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974cd5e90 0x560974ce0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd89c88d700 1 -- 172.21.15.46:0/3348803089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5570 msgr2=0x560974cd5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.935+0000 7fd89c88d700 1 --2- 172.21.15.46:0/3348803089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5570 0x560974cd5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.936+0000 7fd89c88d700 1 -- 172.21.15.46:0/3348803089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560974aaa350 con 0x560974ce0db0 2024-03-18T20:55:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.936+0000 7fd89c88d700 1 --2- 172.21.15.46:0/3348803089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974ce0db0 0x560974ce31a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fd880009a30 tx=0x7fd880009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d7a3f05f3d633 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.936+0000 7fd88e7fc700 1 -- 172.21.15.46:0/3348803089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880009500 con 0x560974ce0db0 2024-03-18T20:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.936+0000 7fd88e7fc700 1 -- 172.21.15.46:0/3348803089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd88000ed90 con 0x560974ce0db0 2024-03-18T20:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.936+0000 7fd88e7fc700 1 -- 172.21.15.46:0/3348803089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8800123d0 con 0x560974ce0db0 2024-03-18T20:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.937+0000 7fd8aa485b80 1 -- 172.21.15.46:0/3348803089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974ce0db0 msgr2=0x560974ce31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.937+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/3348803089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974ce0db0 0x560974ce31a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fd880009a30 tx=0x7fd880009d40 comp rx=0 tx=0).stop 2024-03-18T20:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.937+0000 7fd8aa485b80 1 -- 172.21.15.46:0/3348803089 shutdown_connections 2024-03-18T20:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.937+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/3348803089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974ce0db0 0x560974ce31a0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.937+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/3348803089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5570 0x560974cd5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.937+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/3348803089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974cd5e90 0x560974ce0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.937+0000 7fd8aa485b80 1 -- 172.21.15.46:0/3348803089 >> 172.21.15.46:0/3348803089 conn(0x560974c4c1c0 msgr2=0x560974cd1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.937+0000 7fd8aa485b80 1 -- 172.21.15.46:0/3348803089 shutdown_connections 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.937+0000 7fd8aa485b80 1 -- 172.21.15.46:0/3348803089 wait complete. 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.938+0000 7fd8aa485b80 1 -- start start 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.938+0000 7fd8aa485b80 1 Processor -- start 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.938+0000 7fd8aa485b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5570 0x560974ce0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.938+0000 7fd8aa485b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5e90 0x560974cdda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.938+0000 7fd8aa485b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974ce0db0 0x560974cde0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.938+0000 7fd8aa485b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974bc4d20 con 0x560974cd5570 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.938+0000 7fd8aa485b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974bc49c0 con 0x560974cd5e90 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.938+0000 7fd8aa485b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974aac640 con 0x560974ce0db0 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.938+0000 7fd8aa485b80 10 client.0 _ll_register_callbacks cb 0x560974a30340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.938+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5e90 0x560974cdda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.939+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5e90 0x560974cdda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:46226/0 (socket says 172.21.15.46:46226) 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.939+0000 7fd8977fe700 1 -- 172.21.15.46:0/1497843437 learned_addr learned my addr 172.21.15.46:0/1497843437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.939+0000 7fd89c88d700 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974ce0db0 0x560974cde0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.939+0000 7fd897fff700 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5570 0x560974ce0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.939+0000 7fd8977fe700 1 -- 172.21.15.46:0/1497843437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974ce0db0 msgr2=0x560974cde0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.939+0000 7fd8977fe700 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974ce0db0 0x560974cde0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.939+0000 7fd8977fe700 1 -- 172.21.15.46:0/1497843437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5570 msgr2=0x560974ce0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.939+0000 7fd8977fe700 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5570 0x560974ce0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.939+0000 7fd8977fe700 1 -- 172.21.15.46:0/1497843437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd880009660 con 0x560974cd5e90 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.939+0000 7fd897fff700 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5570 0x560974ce0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.939+0000 7fd8977fe700 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5e90 0x560974cdda70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd8880c9c00 tx=0x7fd888084330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7752dd13e5ac68 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.939+0000 7fd8777fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8777fe700 1 -- 172.21.15.46:0/1497843437 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88809d3f0 con 0x560974cd5e90 2024-03-18T20:55:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8aa485b80 1 -- 172.21.15.46:0/1497843437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560974cc96c0 con 0x560974cd5e90 2024-03-18T20:55:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8777fe700 1 -- 172.21.15.46:0/1497843437 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fd8880a19e0 con 0x560974cd5e90 2024-03-18T20:55:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8777fe700 1 -- 172.21.15.46:0/1497843437 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8880997c0 con 0x560974cd5e90 2024-03-18T20:55:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8aa485b80 10 client.30092 fetch_fsmap learned FSMap version 83 2024-03-18T20:55:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8aa485b80 1 -- 172.21.15.46:0/1497843437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x560974a56b00 con 0x560974cd5e90 2024-03-18T20:55:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8777fe700 1 -- 172.21.15.46:0/1497843437 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560974cc96c0 con 0x560974cd5e90 2024-03-18T20:55:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8777fe700 1 -- 172.21.15.46:0/1497843437 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fd8880a1550 con 0x560974cd5e90 2024-03-18T20:55:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8aa485b80 10 client.30092 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:55:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8aa485b80 10 client.30092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:55:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8aa485b80 20 client.30092 populate_metadata read hostname 'smithi046' 2024-03-18T20:55:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560974cf0980 0x560974cf2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8aa485b80 4 client.30092 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:55:06.946 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.940+0000 7fd8aa485b80 1 -- 172.21.15.46:0/1497843437 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560974c216a0 con 0x560974cf0980 2024-03-18T20:55:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.941+0000 7fd897fff700 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560974cf0980 0x560974cf2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.942+0000 7fd8777fe700 10 client.30092 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.942+0000 7fd897fff700 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560974cf0980 0x560974cf2da0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5107a35454cacda4 server_cookie=cfa300526e4bb4d2 in_seq=0 out_seq=0 2024-03-18T20:55:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8777fe700 1 -- 172.21.15.46:0/1497843437 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560974c216a0 con 0x560974cf0980 2024-03-18T20:55:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8777fe700 10 client.30092 handle_command_reply: tid=0 2024-03-18T20:55:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8aa485b80 1 client.30092 shutdown 2024-03-18T20:55:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8aa485b80 1 -- 172.21.15.46:0/1497843437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5e90 msgr2=0x560974cdda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5e90 0x560974cdda70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd8880c9c00 tx=0x7fd888084330 comp rx=0 tx=0).stop 2024-03-18T20:55:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8aa485b80 1 -- 172.21.15.46:0/1497843437 shutdown_connections 2024-03-18T20:55:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560974cf0980 0x560974cf2da0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5570 0x560974ce0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974ce0db0 0x560974cde0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/1497843437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5e90 0x560974cdda70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8aa485b80 1 -- 172.21.15.46:0/1497843437 >> 172.21.15.46:0/1497843437 conn(0x560974c4c1c0 msgr2=0x560974cd1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8777fe700 0 client.30092 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8aa485b80 1 -- 172.21.15.46:0/1497843437 shutdown_connections 2024-03-18T20:55:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8aa485b80 1 -- 172.21.15.46:0/1497843437 wait complete. 2024-03-18T20:55:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.944+0000 7fd8aa485b80 20 client.30092 trim_cache size 0 max 16384 2024-03-18T20:55:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.966+0000 7fd8aa485b80 1 Processor -- start 2024-03-18T20:55:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.966+0000 7fd8aa485b80 1 -- start start 2024-03-18T20:55:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.966+0000 7fd8aa485b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5570 0x560974ce3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.966+0000 7fd8aa485b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974cd5e90 0x560974cf2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.966+0000 7fd8aa485b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974ce0db0 0x560974cd8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.966+0000 7fd8aa485b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974cdbfc0 con 0x560974cd5570 2024-03-18T20:55:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.966+0000 7fd8aa485b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974cd98b0 con 0x560974ce0db0 2024-03-18T20:55:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.966+0000 7fd8aa485b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974cd2640 con 0x560974cd5e90 2024-03-18T20:55:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.966+0000 7fd89c88d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974ce0db0 0x560974cd8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.966+0000 7fd89c88d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974ce0db0 0x560974cd8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:46238/0 (socket says 172.21.15.46:46238) 2024-03-18T20:55:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.966+0000 7fd89c88d700 1 -- 172.21.15.46:0/2697341978 learned_addr learned my addr 172.21.15.46:0/2697341978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.967+0000 7fd89c88d700 1 -- 172.21.15.46:0/2697341978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974cd5e90 msgr2=0x560974cf2990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:55:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.967+0000 7fd89c88d700 1 --2- 172.21.15.46:0/2697341978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974cd5e90 0x560974cf2990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.967+0000 7fd89c88d700 1 -- 172.21.15.46:0/2697341978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5570 msgr2=0x560974ce3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.967+0000 7fd89c88d700 1 --2- 172.21.15.46:0/2697341978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5570 0x560974ce3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.967+0000 7fd89c88d700 1 -- 172.21.15.46:0/2697341978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560974beee40 con 0x560974ce0db0 2024-03-18T20:55:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.967+0000 7fd89c88d700 1 --2- 172.21.15.46:0/2697341978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974ce0db0 0x560974cd8d70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd88000e4b0 tx=0x7fd880013ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f00b2d579b72aa27 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.967+0000 7fd88effd700 1 -- 172.21.15.46:0/2697341978 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880021070 con 0x560974ce0db0 2024-03-18T20:55:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.967+0000 7fd88effd700 1 -- 172.21.15.46:0/2697341978 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd880013b20 con 0x560974ce0db0 2024-03-18T20:55:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.967+0000 7fd88effd700 1 -- 172.21.15.46:0/2697341978 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88001c6b0 con 0x560974ce0db0 2024-03-18T20:55:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.968+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2697341978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974ce0db0 msgr2=0x560974cd8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.968+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/2697341978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974ce0db0 0x560974cd8d70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd88000e4b0 tx=0x7fd880013ab0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.969+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2697341978 shutdown_connections 2024-03-18T20:55:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.969+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/2697341978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5570 0x560974ce3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.969+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/2697341978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974ce0db0 0x560974cd8d70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.969+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/2697341978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974cd5e90 0x560974cf2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.969+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2697341978 >> 172.21.15.46:0/2697341978 conn(0x560974c4c1c0 msgr2=0x560974cd0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.969+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2697341978 shutdown_connections 2024-03-18T20:55:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.969+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2697341978 wait complete. 2024-03-18T20:55:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.969+0000 7fd8aa485b80 1 -- start start 2024-03-18T20:55:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd8aa485b80 1 Processor -- start 2024-03-18T20:55:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd8aa485b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5570 0x560974ce3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd8aa485b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5e90 0x560974ce41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd8aa485b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974ce0db0 0x560974d437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd8aa485b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974ce4880 con 0x560974cd5e90 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd8aa485b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974d43ce0 con 0x560974cd5570 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd8aa485b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974d43e20 con 0x560974ce0db0 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd8aa485b80 10 client.0 _ll_register_callbacks cb 0x560974a30340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd897fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5570 0x560974ce3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd897fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5570 0x560974ce3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:46254/0 (socket says 172.21.15.46:46254) 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd897fff700 1 -- 172.21.15.46:0/2196442839 learned_addr learned my addr 172.21.15.46:0/2196442839 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd89c88d700 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974ce0db0 0x560974d437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd8977fe700 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5e90 0x560974ce41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd897fff700 1 -- 172.21.15.46:0/2196442839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974ce0db0 msgr2=0x560974d437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd897fff700 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974ce0db0 0x560974d437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd897fff700 1 -- 172.21.15.46:0/2196442839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5e90 msgr2=0x560974ce41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd897fff700 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5e90 0x560974ce41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.970+0000 7fd897fff700 1 -- 172.21.15.46:0/2196442839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560974aaa350 con 0x560974cd5570 2024-03-18T20:55:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.971+0000 7fd8977fe700 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5e90 0x560974ce41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:55:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.971+0000 7fd897fff700 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5570 0x560974ce3c60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd890006010 tx=0x7fd890000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ad4472901c73c15 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.971+0000 7fd8777fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:55:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.971+0000 7fd8777fe700 1 -- 172.21.15.46:0/2196442839 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89001d070 con 0x560974cd5570 2024-03-18T20:55:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.971+0000 7fd8777fe700 1 -- 172.21.15.46:0/2196442839 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fd89000bb70 con 0x560974cd5570 2024-03-18T20:55:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.971+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2196442839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd88000ed90 con 0x560974cd5570 2024-03-18T20:55:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.971+0000 7fd8777fe700 1 -- 172.21.15.46:0/2196442839 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd89000bd30 con 0x560974cd5570 2024-03-18T20:55:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.971+0000 7fd8aa485b80 10 client.30098 fetch_fsmap learned FSMap version 83 2024-03-18T20:55:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.971+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2196442839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x560974aaec70 con 0x560974cd5570 2024-03-18T20:55:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.972+0000 7fd8777fe700 1 -- 172.21.15.46:0/2196442839 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd88000ed90 con 0x560974cd5570 2024-03-18T20:55:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.972+0000 7fd8777fe700 1 -- 172.21.15.46:0/2196442839 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fd89000f910 con 0x560974cd5570 2024-03-18T20:55:06.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.972+0000 7fd8aa485b80 10 client.30098 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:55:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.972+0000 7fd8aa485b80 10 client.30098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:55:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.972+0000 7fd8aa485b80 20 client.30098 populate_metadata read hostname 'smithi046' 2024-03-18T20:55:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.972+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560974d45660 0x560974d47a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.972+0000 7fd8aa485b80 4 client.30098 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:55:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.972+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2196442839 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560974a77c90 con 0x560974d45660 2024-03-18T20:55:06.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.976+0000 7fd8977fe700 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560974d45660 0x560974d47a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.977+0000 7fd8777fe700 10 client.30098 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:06.977+0000 7fd8977fe700 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560974d45660 0x560974d47a80 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8c82cafd8becc33 server_cookie=30d6c7da865b4a0e in_seq=0 out_seq=0 2024-03-18T20:55:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.164+0000 7fd8777fe700 1 -- 172.21.15.46:0/2196442839 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560974a77c90 con 0x560974d45660 2024-03-18T20:55:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.164+0000 7fd8777fe700 10 client.30098 handle_command_reply: tid=0 2024-03-18T20:55:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.164+0000 7fd8aa485b80 1 client.30098 shutdown 2024-03-18T20:55:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.165+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2196442839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5570 msgr2=0x560974ce3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.165+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5570 0x560974ce3c60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd890006010 tx=0x7fd890000bc0 comp rx=0 tx=0).stop 2024-03-18T20:55:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.165+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2196442839 shutdown_connections 2024-03-18T20:55:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.165+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x560974d45660 0x560974d47a80 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.165+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560974ce0db0 0x560974d437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.165+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560974cd5570 0x560974ce3c60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.165+0000 7fd8aa485b80 1 --2- 172.21.15.46:0/2196442839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560974cd5e90 0x560974ce41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.165+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2196442839 >> 172.21.15.46:0/2196442839 conn(0x560974c4c1c0 msgr2=0x560974cd8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.165+0000 7fd8777fe700 0 client.30098 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:07.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.165+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2196442839 shutdown_connections 2024-03-18T20:55:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.165+0000 7fd8aa485b80 1 -- 172.21.15.46:0/2196442839 wait complete. 2024-03-18T20:55:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.165+0000 7fd8aa485b80 20 client.30098 trim_cache size 0 max 16384 2024-03-18T20:55:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:55:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:55:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:55:07.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:55:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.166+0000 7fd89e2f0700 1 -- 172.21.15.46:0/21541162 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd8840a1310 msgr2=0x7fd8840a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.166+0000 7fd89e2f0700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd8840a1310 0x7fd8840a37d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd8900098e0 tx=0x7fd890009f10 comp rx=0 tx=0).stop 2024-03-18T20:55:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.166+0000 7fd89e2f0700 1 -- 172.21.15.46:0/21541162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8980ffbe0 msgr2=0x7fd89819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.166+0000 7fd89e2f0700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8980ffbe0 0x7fd89819a3e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd88800ba50 tx=0x7fd88800bd60 comp rx=0 tx=0).stop 2024-03-18T20:55:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.166+0000 7fd89e2f0700 1 -- 172.21.15.46:0/21541162 shutdown_connections 2024-03-18T20:55:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.166+0000 7fd89e2f0700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd8840a1310 0x7fd8840a37d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.166+0000 7fd89e2f0700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8980ff220 0x7fd898199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.166+0000 7fd89e2f0700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd898106e30 0x7fd89810cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.166+0000 7fd89e2f0700 1 --2- 172.21.15.46:0/21541162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8980ffbe0 0x7fd89819a3e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.166+0000 7fd89e2f0700 1 -- 172.21.15.46:0/21541162 >> 172.21.15.46:0/21541162 conn(0x7fd8980761b0 msgr2=0x7fd898103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.166+0000 7fd89e2f0700 1 -- 172.21.15.46:0/21541162 shutdown_connections 2024-03-18T20:55:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:07.166+0000 7fd89e2f0700 1 -- 172.21.15.46:0/21541162 wait complete. 2024-03-18T20:55:07.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 259554 36.24 MB/sec execute 223 sec latency 156.661 ms 2024-03-18T20:55:08.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 260325 36.27 MB/sec execute 224 sec latency 80.511 ms 2024-03-18T20:55:08.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:08 smithi073 ceph-mon[47384]: pgmap v599: 97 pgs: 32 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 51 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 957 op/s 2024-03-18T20:55:08.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:08 smithi155 ceph-mon[48344]: pgmap v599: 97 pgs: 32 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 51 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 957 op/s 2024-03-18T20:55:08.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:08 smithi046 ceph-mon[43495]: pgmap v599: 97 pgs: 32 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 51 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 15 MiB/s wr, 957 op/s 2024-03-18T20:55:09.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 260925 36.27 MB/sec execute 225 sec latency 125.174 ms 2024-03-18T20:55:10.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 261584 36.25 MB/sec execute 226 sec latency 96.064 ms 2024-03-18T20:55:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:10 smithi073 ceph-mon[47384]: pgmap v600: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.5 MiB/s wr, 795 op/s 2024-03-18T20:55:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:10 smithi073 ceph-mon[47384]: osdmap e108: 12 total, 12 up, 12 in 2024-03-18T20:55:10.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:10 smithi046 ceph-mon[43495]: pgmap v600: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.5 MiB/s wr, 795 op/s 2024-03-18T20:55:10.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:10 smithi046 ceph-mon[43495]: osdmap e108: 12 total, 12 up, 12 in 2024-03-18T20:55:10.863 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:10 smithi155 ceph-mon[48344]: pgmap v600: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 7.5 MiB/s wr, 795 op/s 2024-03-18T20:55:10.863 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:10 smithi155 ceph-mon[48344]: osdmap e108: 12 total, 12 up, 12 in 2024-03-18T20:55:11.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 262178 36.23 MB/sec execute 227 sec latency 93.929 ms 2024-03-18T20:55:12.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 262946 36.26 MB/sec execute 228 sec latency 119.605 ms 2024-03-18T20:55:12.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:12 smithi073 ceph-mon[47384]: pgmap v602: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.1 MiB/s wr, 766 op/s 2024-03-18T20:55:12.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:12 smithi046 ceph-mon[43495]: pgmap v602: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.1 MiB/s wr, 766 op/s 2024-03-18T20:55:13.328 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:12 smithi155 ceph-mon[48344]: pgmap v602: 97 pgs: 97 active+clean; 1.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.1 MiB/s wr, 766 op/s 2024-03-18T20:55:13.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 263730 36.31 MB/sec execute 229 sec latency 61.453 ms 2024-03-18T20:55:14.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 264241 36.25 MB/sec execute 230 sec latency 108.686 ms 2024-03-18T20:55:14.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:14 smithi073 ceph-mon[47384]: pgmap v603: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.6 MiB/s wr, 685 op/s 2024-03-18T20:55:14.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:14 smithi155 ceph-mon[48344]: pgmap v603: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.6 MiB/s wr, 685 op/s 2024-03-18T20:55:14.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:14 smithi046 ceph-mon[43495]: pgmap v603: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.6 MiB/s wr, 685 op/s 2024-03-18T20:55:15.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 264950 36.26 MB/sec execute 231 sec latency 78.962 ms 2024-03-18T20:55:16.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 265577 36.22 MB/sec execute 232 sec latency 169.541 ms 2024-03-18T20:55:16.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:16 smithi073 ceph-mon[47384]: pgmap v604: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 531 op/s 2024-03-18T20:55:16.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:16 smithi155 ceph-mon[48344]: pgmap v604: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 531 op/s 2024-03-18T20:55:16.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:16 smithi046 ceph-mon[43495]: pgmap v604: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 531 op/s 2024-03-18T20:55:17.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 266434 36.26 MB/sec execute 233 sec latency 67.191 ms 2024-03-18T20:55:18.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 267192 36.35 MB/sec execute 234 sec latency 59.204 ms 2024-03-18T20:55:18.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:18 smithi073 ceph-mon[47384]: pgmap v605: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 531 op/s 2024-03-18T20:55:18.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:18 smithi155 ceph-mon[48344]: pgmap v605: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 531 op/s 2024-03-18T20:55:18.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:18 smithi046 ceph-mon[43495]: pgmap v605: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.9 MiB/s wr, 531 op/s 2024-03-18T20:55:19.404 INFO:tasks.workunit.client.0.smithi046.stdout: 10 267625 36.27 MB/sec execute 235 sec latency 102.078 ms 2024-03-18T20:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.355+0000 7f36aba74700 1 -- 172.21.15.46:0/2411825301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a4101a00 msgr2=0x7f36a41058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.355+0000 7f36aba74700 1 --2- 172.21.15.46:0/2411825301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a4101a00 0x7f36a41058e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3694009990 tx=0x7f3694009ca0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.356+0000 7f36aba74700 1 -- 172.21.15.46:0/2411825301 shutdown_connections 2024-03-18T20:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.356+0000 7f36aba74700 1 --2- 172.21.15.46:0/2411825301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36a4101050 0x7f36a4101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.356+0000 7f36aba74700 1 --2- 172.21.15.46:0/2411825301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a4106010 0x7f36a410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.356+0000 7f36aba74700 1 --2- 172.21.15.46:0/2411825301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a4101a00 0x7f36a41058e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.356+0000 7f36aba74700 1 -- 172.21.15.46:0/2411825301 >> 172.21.15.46:0/2411825301 conn(0x7f36a4075ac0 msgr2=0x7f36a40ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.358+0000 7f36aba74700 1 -- 172.21.15.46:0/2411825301 shutdown_connections 2024-03-18T20:55:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.363+0000 7f36aba74700 1 -- 172.21.15.46:0/2411825301 wait complete. 2024-03-18T20:55:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.364+0000 7f36aba74700 1 Processor -- start 2024-03-18T20:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.365+0000 7f36aba74700 1 -- start start 2024-03-18T20:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.365+0000 7f36aba74700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36a4101050 0x7f36a41971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.365+0000 7f36aba74700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a4106010 0x7f36a4197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.365+0000 7f36aba74700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a419bac0 0x7f36a419bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.365+0000 7f36aba74700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a419c540 con 0x7f36a4101050 2024-03-18T20:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.365+0000 7f36aba74700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a419c6b0 con 0x7f36a4106010 2024-03-18T20:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.365+0000 7f36aba74700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a419c870 con 0x7f36a419bac0 2024-03-18T20:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.366+0000 7f36a900f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a4106010 0x7f36a4197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.366+0000 7f36aa011700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a419bac0 0x7f36a419bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.366+0000 7f36aa011700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a419bac0 0x7f36a419bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:49978/0 (socket says 172.21.15.46:49978) 2024-03-18T20:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.366+0000 7f36a900f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a4106010 0x7f36a4197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47864/0 (socket says 172.21.15.46:47864) 2024-03-18T20:55:20.368 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.366+0000 7f36aa011700 1 -- 172.21.15.46:0/2424567785 learned_addr learned my addr 172.21.15.46:0/2424567785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.366+0000 7f36a900f700 1 -- 172.21.15.46:0/2424567785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a419bac0 msgr2=0x7f36a419bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.366+0000 7f36a900f700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a419bac0 0x7f36a419bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.366+0000 7f36a9810700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36a4101050 0x7f36a41971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.366+0000 7f36a900f700 1 -- 172.21.15.46:0/2424567785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36a4101050 msgr2=0x7f36a41971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.366+0000 7f36a900f700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36a4101050 0x7f36a41971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.366+0000 7f36a900f700 1 -- 172.21.15.46:0/2424567785 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36940095f0 con 0x7f36a4106010 2024-03-18T20:55:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.366+0000 7f36a9810700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36a4101050 0x7f36a41971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.367+0000 7f36a900f700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a4106010 0x7f36a4197720 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3694000c00 tx=0x7f3694005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.367+0000 7f369affd700 1 -- 172.21.15.46:0/2424567785 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369401d070 con 0x7f36a4106010 2024-03-18T20:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.367+0000 7f36aba74700 1 -- 172.21.15.46:0/2424567785 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a4079120 con 0x7f36a4106010 2024-03-18T20:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.367+0000 7f369affd700 1 -- 172.21.15.46:0/2424567785 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3694021410 con 0x7f36a4106010 2024-03-18T20:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.367+0000 7f369affd700 1 -- 172.21.15.46:0/2424567785 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694017650 con 0x7f36a4106010 2024-03-18T20:55:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.367+0000 7f36aba74700 1 -- 172.21.15.46:0/2424567785 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a4079680 con 0x7f36a4106010 2024-03-18T20:55:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.370+0000 7f369affd700 1 -- 172.21.15.46:0/2424567785 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f36940177b0 con 0x7f36a4106010 2024-03-18T20:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.371+0000 7f369affd700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36900a9be0 0x7f36900ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.371+0000 7f369affd700 1 -- 172.21.15.46:0/2424567785 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f36940cecd0 con 0x7f36a4106010 2024-03-18T20:55:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.371+0000 7f36a9810700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36900a9be0 0x7f36900ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.373+0000 7f36b7c09b80 1 Processor -- start 2024-03-18T20:55:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.376+0000 7f36b7c09b80 1 -- start start 2024-03-18T20:55:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.376+0000 7f36b7c09b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762e7570 0x55d9762e7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.376+0000 7f36b7c09b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7e90 0x55d9762f2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.376+0000 7f36b7c09b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 0x55d9762f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.376+0000 7f36b7c09b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d976241930 con 0x55d9762e7570 2024-03-18T20:55:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.376+0000 7f36b7c09b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97623f2f0 con 0x55d9762e7e90 2024-03-18T20:55:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.376+0000 7f36b7c09b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d97623abe0 con 0x55d9762f2db0 2024-03-18T20:55:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.377+0000 7f36aa011700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 0x55d9762f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.377+0000 7f36aa011700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 0x55d9762f51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:49994/0 (socket says 172.21.15.46:49994) 2024-03-18T20:55:20.384 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.377+0000 7f36aa011700 1 -- 172.21.15.46:0/719435060 learned_addr learned my addr 172.21.15.46:0/719435060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.377+0000 7f36a900f700 1 --2- 172.21.15.46:0/719435060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7e90 0x55d9762f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.378+0000 7f36aa011700 1 -- 172.21.15.46:0/719435060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7e90 msgr2=0x55d9762f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.378+0000 7f36aa011700 1 --2- 172.21.15.46:0/719435060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7e90 0x55d9762f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.378+0000 7f36aa011700 1 -- 172.21.15.46:0/719435060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762e7570 msgr2=0x55d9762e7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.378+0000 7f36aa011700 1 --2- 172.21.15.46:0/719435060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762e7570 0x55d9762e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.378+0000 7f36aa011700 1 -- 172.21.15.46:0/719435060 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9760bc350 con 0x55d9762f2db0 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.378+0000 7f36aa011700 1 --2- 172.21.15.46:0/719435060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 0x55d9762f51a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f369c00e690 tx=0x7f369c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f031a1ced4297b5a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.378+0000 7f368ffff700 1 -- 172.21.15.46:0/719435060 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c00e040 con 0x55d9762f2db0 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.378+0000 7f368ffff700 1 -- 172.21.15.46:0/719435060 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f369c00f040 con 0x55d9762f2db0 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.378+0000 7f368ffff700 1 -- 172.21.15.46:0/719435060 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c004790 con 0x55d9762f2db0 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.379+0000 7f36b7c09b80 1 -- 172.21.15.46:0/719435060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 msgr2=0x55d9762f51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.379+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/719435060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 0x55d9762f51a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f369c00e690 tx=0x7f369c009920 comp rx=0 tx=0).stop 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.379+0000 7f36a9810700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36900a9be0 0x7f36900ac0a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f36a0007900 tx=0x7f36a0008040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.379+0000 7f36b7c09b80 1 -- 172.21.15.46:0/719435060 shutdown_connections 2024-03-18T20:55:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.379+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/719435060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762e7570 0x55d9762e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.379+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/719435060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 0x55d9762f51a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.379+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/719435060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7e90 0x55d9762f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.379+0000 7f36b7c09b80 1 -- 172.21.15.46:0/719435060 >> 172.21.15.46:0/719435060 conn(0x55d97625e1c0 msgr2=0x55d9762e3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.381+0000 7f36b7c09b80 1 -- 172.21.15.46:0/719435060 shutdown_connections 2024-03-18T20:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.383+0000 7f36b7c09b80 1 -- 172.21.15.46:0/719435060 wait complete. 2024-03-18T20:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.383+0000 7f36b7c09b80 1 -- start start 2024-03-18T20:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.383+0000 7f36b7c09b80 1 Processor -- start 2024-03-18T20:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.384+0000 7f36b7c09b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7570 0x55d9762f2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.384+0000 7f36b7c09b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762e7e90 0x55d9762efb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.384+0000 7f36b7c09b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 0x55d9762f0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.384+0000 7f36b7c09b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9761d6d20 con 0x55d9762e7e90 2024-03-18T20:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.384+0000 7f36b7c09b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9761d69c0 con 0x55d9762e7570 2024-03-18T20:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.384+0000 7f36b7c09b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9760be640 con 0x55d9762f2db0 2024-03-18T20:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.384+0000 7f36b7c09b80 10 client.0 _ll_register_callbacks cb 0x55d976042340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.384+0000 7f36a9810700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7570 0x55d9762f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.384+0000 7f36a9810700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7570 0x55d9762f2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47880/0 (socket says 172.21.15.46:47880) 2024-03-18T20:55:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.384+0000 7f36a9810700 1 -- 172.21.15.46:0/1666600063 learned_addr learned my addr 172.21.15.46:0/1666600063 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.384+0000 7f36aa011700 1 --2- 172.21.15.46:0/1666600063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 0x55d9762f0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.385+0000 7f36aa011700 1 -- 172.21.15.46:0/1666600063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7570 msgr2=0x55d9762f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.385+0000 7f36aa011700 1 --2- 172.21.15.46:0/1666600063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7570 0x55d9762f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.385+0000 7f36aa011700 1 -- 172.21.15.46:0/1666600063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762e7e90 msgr2=0x55d9762efb50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:55:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.385+0000 7f36aa011700 1 --2- 172.21.15.46:0/1666600063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762e7e90 0x55d9762efb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.385+0000 7f36aa011700 1 -- 172.21.15.46:0/1666600063 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c00e370 con 0x55d9762f2db0 2024-03-18T20:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.385+0000 7f36aa011700 1 --2- 172.21.15.46:0/1666600063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 0x55d9762f0290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f369c00ec80 tx=0x7f369c003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8281b45218eea9f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.385+0000 7f368d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.385+0000 7f368d7fa700 1 -- 172.21.15.46:0/1666600063 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c00e040 con 0x55d9762f2db0 2024-03-18T20:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.385+0000 7f368d7fa700 1 -- 172.21.15.46:0/1666600063 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f369c0092e0 con 0x55d9762f2db0 2024-03-18T20:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.385+0000 7f36b7c09b80 1 -- 172.21.15.46:0/1666600063 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9762db6c0 con 0x55d9762f2db0 2024-03-18T20:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.386+0000 7f368d7fa700 1 -- 172.21.15.46:0/1666600063 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f369c004130 con 0x55d9762f2db0 2024-03-18T20:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.386+0000 7f36b7c09b80 10 client.30217 fetch_fsmap learned FSMap version 83 2024-03-18T20:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.386+0000 7f36b7c09b80 1 -- 172.21.15.46:0/1666600063 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55d976068b00 con 0x55d9762f2db0 2024-03-18T20:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.386+0000 7f368d7fa700 1 -- 172.21.15.46:0/1666600063 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d9762db6c0 con 0x55d9762f2db0 2024-03-18T20:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.386+0000 7f368d7fa700 1 -- 172.21.15.46:0/1666600063 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f369c007a40 con 0x55d9762f2db0 2024-03-18T20:55:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.386+0000 7f36b7c09b80 10 client.30217 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.386+0000 7f36b7c09b80 10 client.30217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.386+0000 7f36b7c09b80 20 client.30217 populate_metadata read hostname 'smithi046' 2024-03-18T20:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.386+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/1666600063 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d976313a00 0x55d976315e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.386+0000 7f36b7c09b80 4 client.30217 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.386+0000 7f36b7c09b80 1 -- 172.21.15.46:0/1666600063 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9762336a0 con 0x55d976313a00 2024-03-18T20:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.387+0000 7f36a9810700 1 --2- 172.21.15.46:0/1666600063 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d976313a00 0x55d976315e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.388+0000 7f36a9810700 1 --2- 172.21.15.46:0/1666600063 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d976313a00 0x55d976315e20 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae4f6edd08b35fa server_cookie=26aa1275b49c216c in_seq=0 out_seq=0 2024-03-18T20:55:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.388+0000 7f368d7fa700 10 client.30217 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.389+0000 7f368d7fa700 1 -- 172.21.15.46:0/1666600063 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d9762336a0 con 0x55d976313a00 2024-03-18T20:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.389+0000 7f368d7fa700 10 client.30217 handle_command_reply: tid=0 2024-03-18T20:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.389+0000 7f36b7c09b80 1 client.30217 shutdown 2024-03-18T20:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.390+0000 7f36b7c09b80 1 -- 172.21.15.46:0/1666600063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 msgr2=0x55d9762f0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.390+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/1666600063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 0x55d9762f0290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f369c00ec80 tx=0x7f369c003c30 comp rx=0 tx=0).stop 2024-03-18T20:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.390+0000 7f36b7c09b80 1 -- 172.21.15.46:0/1666600063 shutdown_connections 2024-03-18T20:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.390+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/1666600063 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d976313a00 0x55d976315e20 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.390+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/1666600063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762f2db0 0x55d9762f0290 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.390+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/1666600063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7570 0x55d9762f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.390+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/1666600063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762e7e90 0x55d9762efb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.390+0000 7f36b7c09b80 1 -- 172.21.15.46:0/1666600063 >> 172.21.15.46:0/1666600063 conn(0x55d97625e1c0 msgr2=0x55d9762e3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.393+0000 7f368d7fa700 0 client.30217 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.398+0000 7f36b7c09b80 1 -- 172.21.15.46:0/1666600063 shutdown_connections 2024-03-18T20:55:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.398+0000 7f36b7c09b80 1 -- 172.21.15.46:0/1666600063 wait complete. 2024-03-18T20:55:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.400+0000 7f36b7c09b80 20 client.30217 trim_cache size 0 max 16384 2024-03-18T20:55:20.410 INFO:tasks.workunit.client.0.smithi046.stdout: 10 268109 36.19 MB/sec execute 236 sec latency 108.346 ms 2024-03-18T20:55:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.428+0000 7f36b7c09b80 1 Processor -- start 2024-03-18T20:55:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.428+0000 7f36b7c09b80 1 -- start start 2024-03-18T20:55:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.428+0000 7f36b7c09b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762e7570 0x55d9762f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.428+0000 7f36b7c09b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7e90 0x55d9762ef470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.428+0000 7f36b7c09b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762f2db0 0x55d9762e2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.428+0000 7f36b7c09b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9762eff40 con 0x55d9762f2db0 2024-03-18T20:55:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.429+0000 7f36b7c09b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9762ddc40 con 0x55d9762e7e90 2024-03-18T20:55:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.429+0000 7f36b7c09b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9762ee110 con 0x55d9762e7570 2024-03-18T20:55:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.429+0000 7f36a900f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7e90 0x55d9762ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.429+0000 7f36a900f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7e90 0x55d9762ef470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47892/0 (socket says 172.21.15.46:47892) 2024-03-18T20:55:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.429+0000 7f36a900f700 1 -- 172.21.15.46:0/85353339 learned_addr learned my addr 172.21.15.46:0/85353339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.429+0000 7f36a900f700 1 -- 172.21.15.46:0/85353339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762e7570 msgr2=0x55d9762f01f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:55:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.429+0000 7f36a900f700 1 --2- 172.21.15.46:0/85353339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762e7570 0x55d9762f01f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.429+0000 7f36a900f700 1 -- 172.21.15.46:0/85353339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762f2db0 msgr2=0x55d9762e2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.429+0000 7f36a900f700 1 --2- 172.21.15.46:0/85353339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762f2db0 0x55d9762e2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.429+0000 7f36a900f700 1 -- 172.21.15.46:0/85353339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d976200e40 con 0x55d9762e7e90 2024-03-18T20:55:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.430+0000 7f36a900f700 1 --2- 172.21.15.46:0/85353339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7e90 0x55d9762ef470 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f369409bd80 tx=0x7f3694009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb907b69c715c9d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.430+0000 7f3698da8700 1 -- 172.21.15.46:0/85353339 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369409bdf0 con 0x55d9762e7e90 2024-03-18T20:55:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.430+0000 7f3698da8700 1 -- 172.21.15.46:0/85353339 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f369409fc20 con 0x55d9762e7e90 2024-03-18T20:55:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.430+0000 7f3698da8700 1 -- 172.21.15.46:0/85353339 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369409c2b0 con 0x55d9762e7e90 2024-03-18T20:55:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.431+0000 7f36b7c09b80 1 -- 172.21.15.46:0/85353339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7e90 msgr2=0x55d9762ef470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.431+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/85353339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7e90 0x55d9762ef470 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f369409bd80 tx=0x7f3694009380 comp rx=0 tx=0).stop 2024-03-18T20:55:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.433+0000 7f36b7c09b80 1 -- 172.21.15.46:0/85353339 shutdown_connections 2024-03-18T20:55:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.433+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/85353339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762f2db0 0x55d9762e2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.433+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/85353339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762e7570 0x55d9762f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.433+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/85353339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762e7e90 0x55d9762ef470 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.433+0000 7f36b7c09b80 1 -- 172.21.15.46:0/85353339 >> 172.21.15.46:0/85353339 conn(0x55d97625e1c0 msgr2=0x55d97631a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.435+0000 7f36b7c09b80 1 -- 172.21.15.46:0/85353339 shutdown_connections 2024-03-18T20:55:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.435+0000 7f36b7c09b80 1 -- 172.21.15.46:0/85353339 wait complete. 2024-03-18T20:55:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.435+0000 7f36b7c09b80 1 -- start start 2024-03-18T20:55:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.435+0000 7f36b7c09b80 1 Processor -- start 2024-03-18T20:55:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.436+0000 7f36b7c09b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762e7570 0x55d97630a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.436+0000 7f36b7c09b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762e7e90 0x55d97630a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.436+0000 7f36b7c09b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762f2db0 0x55d976308ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.436+0000 7f36b7c09b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d976309050 con 0x55d9762e7e90 2024-03-18T20:55:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.436+0000 7f36b7c09b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9763091c0 con 0x55d9762f2db0 2024-03-18T20:55:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.436+0000 7f36b7c09b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d976309380 con 0x55d9762e7570 2024-03-18T20:55:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.436+0000 7f36b7c09b80 10 client.0 _ll_register_callbacks cb 0x7f3694009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:55:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.436+0000 7f36aa011700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762f2db0 0x55d976308ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.436+0000 7f36aa011700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762f2db0 0x55d976308ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47894/0 (socket says 172.21.15.46:47894) 2024-03-18T20:55:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.436+0000 7f36aa011700 1 -- 172.21.15.46:0/3038837535 learned_addr learned my addr 172.21.15.46:0/3038837535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.436+0000 7f36a9810700 1 --2- 172.21.15.46:0/3038837535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762e7570 0x55d97630a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.437+0000 7f36aa011700 1 -- 172.21.15.46:0/3038837535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762e7570 msgr2=0x55d97630a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.437+0000 7f36aa011700 1 --2- 172.21.15.46:0/3038837535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762e7570 0x55d97630a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.437+0000 7f36aa011700 1 -- 172.21.15.46:0/3038837535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762e7e90 msgr2=0x55d97630a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.437+0000 7f36aa011700 1 --2- 172.21.15.46:0/3038837535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762e7e90 0x55d97630a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.437+0000 7f36aa011700 1 -- 172.21.15.46:0/3038837535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a4079680 con 0x55d9762f2db0 2024-03-18T20:55:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.437+0000 7f36aa011700 1 --2- 172.21.15.46:0/3038837535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762f2db0 0x55d976308ad0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f369c00e690 tx=0x7f369c009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b19424c223a5b01 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.437+0000 7f368d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:55:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.437+0000 7f368d7fa700 1 -- 172.21.15.46:0/3038837535 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c005120 con 0x55d9762f2db0 2024-03-18T20:55:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.437+0000 7f368d7fa700 1 -- 172.21.15.46:0/3038837535 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f369c0092e0 con 0x55d9762f2db0 2024-03-18T20:55:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.442+0000 7f36b7c09b80 1 -- 172.21.15.46:0/3038837535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f369c00f040 con 0x55d9762f2db0 2024-03-18T20:55:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.442+0000 7f368d7fa700 1 -- 172.21.15.46:0/3038837535 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f369c004270 con 0x55d9762f2db0 2024-03-18T20:55:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.442+0000 7f36b7c09b80 10 client.30116 fetch_fsmap learned FSMap version 83 2024-03-18T20:55:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.442+0000 7f36b7c09b80 1 -- 172.21.15.46:0/3038837535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=4+}) v3 -- 0x55d9760c0c70 con 0x55d9762f2db0 2024-03-18T20:55:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.442+0000 7f368d7fa700 1 -- 172.21.15.46:0/3038837535 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f369c00f040 con 0x55d9762f2db0 2024-03-18T20:55:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.442+0000 7f368d7fa700 1 -- 172.21.15.46:0/3038837535 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 83) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f369c01c360 con 0x55d9762f2db0 2024-03-18T20:55:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.442+0000 7f36b7c09b80 10 client.30116 fetch_fsmap finished waiting for FSMap version 83 2024-03-18T20:55:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.442+0000 7f36b7c09b80 10 client.30116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:55:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.442+0000 7f36b7c09b80 20 client.30116 populate_metadata read hostname 'smithi046' 2024-03-18T20:55:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.442+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/3038837535 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d976368610 0x55d97636aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.442+0000 7f36b7c09b80 4 client.30116 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:55:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.442+0000 7f36b7c09b80 1 -- 172.21.15.46:0/3038837535 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d976089c90 con 0x55d976368610 2024-03-18T20:55:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.443+0000 7f36a9810700 1 --2- 172.21.15.46:0/3038837535 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d976368610 0x55d97636aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.443+0000 7f36a9810700 1 --2- 172.21.15.46:0/3038837535 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d976368610 0x55d97636aa30 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a26b2839c8aebb3 server_cookie=a7dcda7689a7555 in_seq=0 out_seq=0 2024-03-18T20:55:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.443+0000 7f368d7fa700 10 client.30116 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:20.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:20 smithi073 ceph-mon[47384]: pgmap v606: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.9 MiB/s wr, 634 op/s 2024-03-18T20:55:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f368d7fa700 1 -- 172.21.15.46:0/3038837535 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d976089c90 con 0x55d976368610 2024-03-18T20:55:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f368d7fa700 10 client.30116 handle_command_reply: tid=0 2024-03-18T20:55:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f36b7c09b80 1 client.30116 shutdown 2024-03-18T20:55:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f36b7c09b80 1 -- 172.21.15.46:0/3038837535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762f2db0 msgr2=0x55d976308ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/3038837535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762f2db0 0x55d976308ad0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f369c00e690 tx=0x7f369c009ea0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f36b7c09b80 1 -- 172.21.15.46:0/3038837535 shutdown_connections 2024-03-18T20:55:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/3038837535 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d976368610 0x55d97636aa30 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/3038837535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d9762f2db0 0x55d976308ad0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/3038837535 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d9762e7570 0x55d97630a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f36b7c09b80 1 --2- 172.21.15.46:0/3038837535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d9762e7e90 0x55d97630a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f36b7c09b80 1 -- 172.21.15.46:0/3038837535 >> 172.21.15.46:0/3038837535 conn(0x55d97625e1c0 msgr2=0x55d9762eaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f368d7fa700 0 client.30116 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f36b7c09b80 1 -- 172.21.15.46:0/3038837535 shutdown_connections 2024-03-18T20:55:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f36b7c09b80 1 -- 172.21.15.46:0/3038837535 wait complete. 2024-03-18T20:55:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.743+0000 7f36b7c09b80 20 client.30116 trim_cache size 0 max 16384 2024-03-18T20:55:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:55:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:55:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:55:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:55:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.744+0000 7f36aba74700 1 -- 172.21.15.46:0/2424567785 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36900a9be0 msgr2=0x7f36900ac0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.744+0000 7f36aba74700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36900a9be0 0x7f36900ac0a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f36a0007900 tx=0x7f36a0008040 comp rx=0 tx=0).stop 2024-03-18T20:55:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.744+0000 7f36aba74700 1 -- 172.21.15.46:0/2424567785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a4106010 msgr2=0x7f36a4197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.744+0000 7f36aba74700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a4106010 0x7f36a4197720 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3694000c00 tx=0x7f3694005740 comp rx=0 tx=0).stop 2024-03-18T20:55:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.744+0000 7f36aba74700 1 -- 172.21.15.46:0/2424567785 shutdown_connections 2024-03-18T20:55:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.744+0000 7f36aba74700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f36900a9be0 0x7f36900ac0a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.744+0000 7f36aba74700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36a4101050 0x7f36a41971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.744+0000 7f36aba74700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f36a419bac0 0x7f36a419bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.744+0000 7f36aba74700 1 --2- 172.21.15.46:0/2424567785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a4106010 0x7f36a4197720 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.744+0000 7f36aba74700 1 -- 172.21.15.46:0/2424567785 >> 172.21.15.46:0/2424567785 conn(0x7f36a4075ac0 msgr2=0x7f36a4076180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.744+0000 7f36aba74700 1 -- 172.21.15.46:0/2424567785 shutdown_connections 2024-03-18T20:55:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:20.744+0000 7f36aba74700 1 -- 172.21.15.46:0/2424567785 wait complete. 2024-03-18T20:55:20.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:20 smithi155 ceph-mon[48344]: pgmap v606: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.9 MiB/s wr, 634 op/s 2024-03-18T20:55:20.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:20 smithi046 ceph-mon[43495]: pgmap v606: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.9 MiB/s wr, 634 op/s 2024-03-18T20:55:21.411 INFO:tasks.workunit.client.0.smithi046.stdout: 10 268653 36.19 MB/sec execute 237 sec latency 193.384 ms 2024-03-18T20:55:22.411 INFO:tasks.workunit.client.0.smithi046.stdout: 10 269255 36.12 MB/sec execute 238 sec latency 121.008 ms 2024-03-18T20:55:22.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:22 smithi073 ceph-mon[47384]: pgmap v607: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.0 MiB/s wr, 566 op/s 2024-03-18T20:55:22.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:22 smithi155 ceph-mon[48344]: pgmap v607: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.0 MiB/s wr, 566 op/s 2024-03-18T20:55:22.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:22 smithi046 ceph-mon[43495]: pgmap v607: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.0 MiB/s wr, 566 op/s 2024-03-18T20:55:23.411 INFO:tasks.workunit.client.0.smithi046.stdout: 10 270096 36.18 MB/sec execute 239 sec latency 66.032 ms 2024-03-18T20:55:24.411 INFO:tasks.workunit.client.0.smithi046.stdout: 10 270837 36.24 MB/sec execute 240 sec latency 71.447 ms 2024-03-18T20:55:24.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:24 smithi073 ceph-mon[47384]: pgmap v608: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.8 MiB/s wr, 573 op/s 2024-03-18T20:55:24.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:24 smithi155 ceph-mon[48344]: pgmap v608: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.8 MiB/s wr, 573 op/s 2024-03-18T20:55:24.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:24 smithi046 ceph-mon[43495]: pgmap v608: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.8 MiB/s wr, 573 op/s 2024-03-18T20:55:25.411 INFO:tasks.workunit.client.0.smithi046.stdout: 10 271365 36.19 MB/sec execute 241 sec latency 122.282 ms 2024-03-18T20:55:26.412 INFO:tasks.workunit.client.0.smithi046.stdout: 10 271974 36.15 MB/sec execute 242 sec latency 81.580 ms 2024-03-18T20:55:26.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:26 smithi073 ceph-mon[47384]: pgmap v609: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.2 MiB/s wr, 700 op/s 2024-03-18T20:55:26.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:26 smithi155 ceph-mon[48344]: pgmap v609: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.2 MiB/s wr, 700 op/s 2024-03-18T20:55:26.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:26 smithi046 ceph-mon[43495]: pgmap v609: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.2 MiB/s wr, 700 op/s 2024-03-18T20:55:27.412 INFO:tasks.workunit.client.0.smithi046.stdout: 10 272701 36.17 MB/sec execute 243 sec latency 93.386 ms 2024-03-18T20:55:28.412 INFO:tasks.workunit.client.0.smithi046.stdout: 10 273506 36.19 MB/sec execute 244 sec latency 79.053 ms 2024-03-18T20:55:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:28 smithi073 ceph-mon[47384]: pgmap v610: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.5 MiB/s wr, 574 op/s 2024-03-18T20:55:28.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:28 smithi155 ceph-mon[48344]: pgmap v610: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.5 MiB/s wr, 574 op/s 2024-03-18T20:55:28.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:28 smithi046 ceph-mon[43495]: pgmap v610: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.5 MiB/s wr, 574 op/s 2024-03-18T20:55:29.412 INFO:tasks.workunit.client.0.smithi046.stdout: 10 274183 36.22 MB/sec execute 245 sec latency 79.773 ms 2024-03-18T20:55:29.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:29 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:55:29.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:29 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:55:29.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:29 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:55:29.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:29 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:55:29.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:29 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:55:29.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:29 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:55:30.415 INFO:tasks.workunit.client.0.smithi046.stdout: 10 274766 36.18 MB/sec execute 246 sec latency 144.203 ms 2024-03-18T20:55:30.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:30 smithi073 ceph-mon[47384]: pgmap v611: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 797 op/s 2024-03-18T20:55:30.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:30 smithi155 ceph-mon[48344]: pgmap v611: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 797 op/s 2024-03-18T20:55:30.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:30 smithi046 ceph-mon[43495]: pgmap v611: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 797 op/s 2024-03-18T20:55:31.415 INFO:tasks.workunit.client.0.smithi046.stdout: 10 275360 36.18 MB/sec execute 247 sec latency 92.380 ms 2024-03-18T20:55:31.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:31 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:55:31.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:31 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:55:31.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:31 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:55:32.415 INFO:tasks.workunit.client.0.smithi046.stdout: 10 275964 36.15 MB/sec execute 248 sec latency 123.098 ms 2024-03-18T20:55:32.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:32 smithi073 ceph-mon[47384]: pgmap v612: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 496 op/s 2024-03-18T20:55:32.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:32 smithi155 ceph-mon[48344]: pgmap v612: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 496 op/s 2024-03-18T20:55:32.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:32 smithi046 ceph-mon[43495]: pgmap v612: 97 pgs: 97 active+clean; 2.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 496 op/s 2024-03-18T20:55:33.415 INFO:tasks.workunit.client.0.smithi046.stdout: 10 276716 36.17 MB/sec execute 249 sec latency 109.337 ms 2024-03-18T20:55:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.982+0000 7f05820f0700 1 -- 172.21.15.46:0/78946050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f057c10f940 msgr2=0x7f057c1082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.982+0000 7f05820f0700 1 --2- 172.21.15.46:0/78946050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f057c10f940 0x7f057c1082f0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f05700099c0 tx=0x7f0570009cd0 comp rx=0 tx=0).stop 2024-03-18T20:55:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.982+0000 7f05820f0700 1 -- 172.21.15.46:0/78946050 shutdown_connections 2024-03-18T20:55:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.982+0000 7f05820f0700 1 --2- 172.21.15.46:0/78946050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f057c11a590 0x7f057c11c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.982+0000 7f05820f0700 1 --2- 172.21.15.46:0/78946050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f057c10f4d0 0x7f057c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.982+0000 7f05820f0700 1 --2- 172.21.15.46:0/78946050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f057c10f940 0x7f057c1082f0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.982+0000 7f05820f0700 1 -- 172.21.15.46:0/78946050 >> 172.21.15.46:0/78946050 conn(0x7f057c06bd80 msgr2=0x7f057c06c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.982+0000 7f05820f0700 1 -- 172.21.15.46:0/78946050 shutdown_connections 2024-03-18T20:55:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.996+0000 7f05820f0700 1 -- 172.21.15.46:0/78946050 wait complete. 2024-03-18T20:55:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.997+0000 7f05820f0700 1 Processor -- start 2024-03-18T20:55:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.998+0000 7f05820f0700 1 -- start start 2024-03-18T20:55:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.998+0000 7f05820f0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f057c10f4d0 0x7f057c118a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.998+0000 7f05820f0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f057c10f940 0x7f057c118f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.998+0000 7f05820f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f057c11a590 0x7f057c111bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.998+0000 7f05820f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c1123c0 con 0x7f057c11a590 2024-03-18T20:55:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.998+0000 7f05820f0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c112530 con 0x7f057c10f940 2024-03-18T20:55:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.998+0000 7f05820f0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c1126f0 con 0x7f057c10f4d0 2024-03-18T20:55:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.999+0000 7f057affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f057c10f940 0x7f057c118f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.999+0000 7f057affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f057c10f940 0x7f057c118f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:60450/0 (socket says 172.21.15.46:60450) 2024-03-18T20:55:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.999+0000 7f057affd700 1 -- 172.21.15.46:0/1649801295 learned_addr learned my addr 172.21.15.46:0/1649801295 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.999+0000 7f057b7fe700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f057c10f4d0 0x7f057c118a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:33.999+0000 7f057bfff700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f057c11a590 0x7f057c111bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.017+0000 7f057affd700 1 -- 172.21.15.46:0/1649801295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f057c10f4d0 msgr2=0x7f057c118a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.017+0000 7f057affd700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f057c10f4d0 0x7f057c118a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.017+0000 7f057affd700 1 -- 172.21.15.46:0/1649801295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f057c11a590 msgr2=0x7f057c111bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.017+0000 7f057affd700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f057c11a590 0x7f057c111bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.017+0000 7f057affd700 1 -- 172.21.15.46:0/1649801295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05700095f0 con 0x7f057c10f940 2024-03-18T20:55:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.025+0000 7f057affd700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f057c10f940 0x7f057c118f80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f057000b580 tx=0x7f0570005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.026+0000 7f0578ff9700 1 -- 172.21.15.46:0/1649801295 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057001d070 con 0x7f057c10f940 2024-03-18T20:55:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.026+0000 7f0578ff9700 1 -- 172.21.15.46:0/1649801295 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0570003f30 con 0x7f057c10f940 2024-03-18T20:55:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.026+0000 7f0578ff9700 1 -- 172.21.15.46:0/1649801295 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057000fc80 con 0x7f057c10f940 2024-03-18T20:55:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.026+0000 7f05820f0700 1 -- 172.21.15.46:0/1649801295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c116590 con 0x7f057c10f940 2024-03-18T20:55:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.026+0000 7f05820f0700 1 -- 172.21.15.46:0/1649801295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c1169f0 con 0x7f057c10f940 2024-03-18T20:55:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.032+0000 7f058e285b80 1 Processor -- start 2024-03-18T20:55:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.033+0000 7f058e285b80 1 -- start start 2024-03-18T20:55:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.033+0000 7f058e285b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26570 0x55d888a26950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.033+0000 7f058e285b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26e90 0x55d888a31870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.033+0000 7f058e285b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a31db0 0x55d888a341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.033+0000 7f058e285b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d888980930 con 0x55d888a26e90 2024-03-18T20:55:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.033+0000 7f058e285b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d88897e2f0 con 0x55d888a31db0 2024-03-18T20:55:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.033+0000 7f058e285b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d888979be0 con 0x55d888a26570 2024-03-18T20:55:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.033+0000 7f057bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26570 0x55d888a26950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.033+0000 7f057bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26570 0x55d888a26950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40936/0 (socket says 172.21.15.46:40936) 2024-03-18T20:55:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.033+0000 7f057bfff700 1 -- 172.21.15.46:0/733592866 learned_addr learned my addr 172.21.15.46:0/733592866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.034+0000 7f057b7fe700 1 --2- 172.21.15.46:0/733592866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26e90 0x55d888a31870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.034+0000 7f057b7fe700 1 -- 172.21.15.46:0/733592866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26570 msgr2=0x55d888a26950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.034+0000 7f057b7fe700 1 --2- 172.21.15.46:0/733592866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26570 0x55d888a26950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.034+0000 7f057b7fe700 1 -- 172.21.15.46:0/733592866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a31db0 msgr2=0x55d888a341a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:55:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.034+0000 7f057b7fe700 1 --2- 172.21.15.46:0/733592866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a31db0 0x55d888a341a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.034+0000 7f057b7fe700 1 -- 172.21.15.46:0/733592866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8887fb350 con 0x55d888a26e90 2024-03-18T20:55:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.035+0000 7f057b7fe700 1 --2- 172.21.15.46:0/733592866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26e90 0x55d888a31870 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0574009f20 tx=0x7f0574009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0a8d53175fa690 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.036+0000 7f0561ffb700 1 -- 172.21.15.46:0/733592866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574011070 con 0x55d888a26e90 2024-03-18T20:55:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.036+0000 7f0561ffb700 1 -- 172.21.15.46:0/733592866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f057400c0d0 con 0x55d888a26e90 2024-03-18T20:55:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.036+0000 7f0561ffb700 1 -- 172.21.15.46:0/733592866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574015cf0 con 0x55d888a26e90 2024-03-18T20:55:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.037+0000 7f058e285b80 1 -- 172.21.15.46:0/733592866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26e90 msgr2=0x55d888a31870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.037+0000 7f058e285b80 1 --2- 172.21.15.46:0/733592866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26e90 0x55d888a31870 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0574009f20 tx=0x7f0574009f50 comp rx=0 tx=0).stop 2024-03-18T20:55:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.037+0000 7f058e285b80 1 -- 172.21.15.46:0/733592866 shutdown_connections 2024-03-18T20:55:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.037+0000 7f058e285b80 1 --2- 172.21.15.46:0/733592866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a31db0 0x55d888a341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.037+0000 7f058e285b80 1 --2- 172.21.15.46:0/733592866 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26570 0x55d888a26950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.037+0000 7f058e285b80 1 --2- 172.21.15.46:0/733592866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26e90 0x55d888a31870 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.037+0000 7f058e285b80 1 -- 172.21.15.46:0/733592866 >> 172.21.15.46:0/733592866 conn(0x55d88899d1c0 msgr2=0x55d888a22a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.042+0000 7f0578ff9700 1 -- 172.21.15.46:0/1649801295 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f057000f460 con 0x7f057c10f940 2024-03-18T20:55:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.042+0000 7f0578ff9700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f05640a14a0 0x7f05640a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.043+0000 7f0578ff9700 1 -- 172.21.15.46:0/1649801295 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f05700cdad0 con 0x7f057c10f940 2024-03-18T20:55:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.044+0000 7f057b7fe700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f05640a14a0 0x7f05640a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.044+0000 7f058e285b80 1 -- 172.21.15.46:0/733592866 shutdown_connections 2024-03-18T20:55:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.046+0000 7f058e285b80 1 -- 172.21.15.46:0/733592866 wait complete. 2024-03-18T20:55:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.047+0000 7f058e285b80 1 -- start start 2024-03-18T20:55:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.047+0000 7f058e285b80 1 Processor -- start 2024-03-18T20:55:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.048+0000 7f057b7fe700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f05640a14a0 0x7f05640a3960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0574015530 tx=0x7f05740157b0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:34.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.051+0000 7f058e285b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26570 0x55d888a2ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.051+0000 7f058e285b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26e90 0x55d888a2f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.051+0000 7f058e285b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a2f6f0 0x55d888a3db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.051+0000 7f058e285b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d888915d20 con 0x55d888a26e90 2024-03-18T20:55:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.051+0000 7f058e285b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8889159c0 con 0x55d888a26570 2024-03-18T20:55:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.051+0000 7f058e285b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8887fd640 con 0x55d888a2f6f0 2024-03-18T20:55:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.052+0000 7f058e285b80 10 client.0 _ll_register_callbacks cb 0x55d888781340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:55:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.052+0000 7f057b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26570 0x55d888a2ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.058+0000 7f057bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a2f6f0 0x55d888a3db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.058+0000 7f057bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a2f6f0 0x55d888a3db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40938/0 (socket says 172.21.15.46:40938) 2024-03-18T20:55:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.058+0000 7f057bfff700 1 -- 172.21.15.46:0/2401536367 learned_addr learned my addr 172.21.15.46:0/2401536367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.065+0000 7f057affd700 1 --2- 172.21.15.46:0/2401536367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26e90 0x55d888a2f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.067+0000 7f057b7fe700 1 -- 172.21.15.46:0/2401536367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a2f6f0 msgr2=0x55d888a3db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.067+0000 7f057b7fe700 1 --2- 172.21.15.46:0/2401536367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a2f6f0 0x55d888a3db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.067+0000 7f057b7fe700 1 -- 172.21.15.46:0/2401536367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26e90 msgr2=0x55d888a2f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.067+0000 7f057b7fe700 1 --2- 172.21.15.46:0/2401536367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26e90 0x55d888a2f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.067+0000 7f057b7fe700 1 -- 172.21.15.46:0/2401536367 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0574009bd0 con 0x55d888a26570 2024-03-18T20:55:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.068+0000 7f057b7fe700 1 --2- 172.21.15.46:0/2401536367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26570 0x55d888a2ec70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0574000f80 tx=0x7f0574009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c905200e3ae5ccbb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.073+0000 7f055affd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:55:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.073+0000 7f055affd700 1 -- 172.21.15.46:0/2401536367 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057400ce40 con 0x55d888a26570 2024-03-18T20:55:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.073+0000 7f055affd700 1 -- 172.21.15.46:0/2401536367 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f0574004050 con 0x55d888a26570 2024-03-18T20:55:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.073+0000 7f058e285b80 1 -- 172.21.15.46:0/2401536367 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d888a1a6c0 con 0x55d888a26570 2024-03-18T20:55:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.084+0000 7f055affd700 1 -- 172.21.15.46:0/2401536367 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0574015b20 con 0x55d888a26570 2024-03-18T20:55:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.084+0000 7f058e285b80 10 client.30122 fetch_fsmap learned FSMap version 85 2024-03-18T20:55:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.084+0000 7f058e285b80 1 -- 172.21.15.46:0/2401536367 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=4+}) v3 -- 0x55d8887a7b00 con 0x55d888a26570 2024-03-18T20:55:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.094+0000 7f055affd700 1 -- 172.21.15.46:0/2401536367 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d888a1a6c0 con 0x55d888a26570 2024-03-18T20:55:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.094+0000 7f055affd700 1 -- 172.21.15.46:0/2401536367 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 85) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f0574032070 con 0x55d888a26570 2024-03-18T20:55:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.095+0000 7f058e285b80 10 client.30122 fetch_fsmap finished waiting for FSMap version 85 2024-03-18T20:55:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.095+0000 7f058e285b80 10 client.30122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:55:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.096+0000 7f058e285b80 20 client.30122 populate_metadata read hostname 'smithi046' 2024-03-18T20:55:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.096+0000 7f058e285b80 1 --2- 172.21.15.46:0/2401536367 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d888a41d90 0x55d888a441b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.096+0000 7f058e285b80 4 client.30122 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:55:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.096+0000 7f058e285b80 1 -- 172.21.15.46:0/2401536367 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8889726a0 con 0x55d888a41d90 2024-03-18T20:55:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.106+0000 7f057affd700 1 --2- 172.21.15.46:0/2401536367 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d888a41d90 0x55d888a441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.141+0000 7f057affd700 1 --2- 172.21.15.46:0/2401536367 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d888a41d90 0x55d888a441b0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df155e21eb5c1125 server_cookie=75d59ba95e687c9c in_seq=0 out_seq=0 2024-03-18T20:55:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.142+0000 7f055affd700 10 client.30122 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.159+0000 7f055affd700 1 -- 172.21.15.46:0/2401536367 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d8889726a0 con 0x55d888a41d90 2024-03-18T20:55:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.159+0000 7f055affd700 10 client.30122 handle_command_reply: tid=0 2024-03-18T20:55:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.159+0000 7f058e285b80 1 client.30122 shutdown 2024-03-18T20:55:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.177+0000 7f058e285b80 1 -- 172.21.15.46:0/2401536367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26570 msgr2=0x55d888a2ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.177+0000 7f058e285b80 1 --2- 172.21.15.46:0/2401536367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26570 0x55d888a2ec70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0574000f80 tx=0x7f0574009f50 comp rx=0 tx=0).stop 2024-03-18T20:55:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.189+0000 7f058e285b80 1 -- 172.21.15.46:0/2401536367 shutdown_connections 2024-03-18T20:55:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.189+0000 7f058e285b80 1 --2- 172.21.15.46:0/2401536367 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d888a41d90 0x55d888a441b0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.189+0000 7f058e285b80 1 --2- 172.21.15.46:0/2401536367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a2f6f0 0x55d888a3db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.189+0000 7f058e285b80 1 --2- 172.21.15.46:0/2401536367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26570 0x55d888a2ec70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.189+0000 7f058e285b80 1 --2- 172.21.15.46:0/2401536367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26e90 0x55d888a2f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.189+0000 7f058e285b80 1 -- 172.21.15.46:0/2401536367 >> 172.21.15.46:0/2401536367 conn(0x55d88899d1c0 msgr2=0x55d888a22650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.189+0000 7f055affd700 0 client.30122 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.201+0000 7f058e285b80 1 -- 172.21.15.46:0/2401536367 shutdown_connections 2024-03-18T20:55:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.207+0000 7f058e285b80 1 -- 172.21.15.46:0/2401536367 wait complete. 2024-03-18T20:55:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.207+0000 7f058e285b80 20 client.30122 trim_cache size 0 max 16384 2024-03-18T20:55:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.237+0000 7f058e285b80 1 Processor -- start 2024-03-18T20:55:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.240+0000 7f058e285b80 1 -- start start 2024-03-18T20:55:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.240+0000 7f058e285b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26570 0x55d888a295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.240+0000 7f058e285b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26e90 0x55d888a2d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.240+0000 7f058e285b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a2f6f0 0x55d888a2daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.240+0000 7f058e285b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d888a27850 con 0x55d888a26570 2024-03-18T20:55:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.240+0000 7f058e285b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d888a2dfe0 con 0x55d888a26e90 2024-03-18T20:55:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.240+0000 7f058e285b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d888a1cc10 con 0x55d888a2f6f0 2024-03-18T20:55:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.242+0000 7f057affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26e90 0x55d888a2d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.242+0000 7f057affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26e90 0x55d888a2d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:60460/0 (socket says 172.21.15.46:60460) 2024-03-18T20:55:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.242+0000 7f057affd700 1 -- 172.21.15.46:0/919936062 learned_addr learned my addr 172.21.15.46:0/919936062 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.249+0000 7f057bfff700 1 --2- 172.21.15.46:0/919936062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a2f6f0 0x55d888a2daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.257+0000 7f057b7fe700 1 --2- 172.21.15.46:0/919936062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26570 0x55d888a295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.257+0000 7f057b7fe700 1 -- 172.21.15.46:0/919936062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a2f6f0 msgr2=0x55d888a2daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.257+0000 7f057b7fe700 1 --2- 172.21.15.46:0/919936062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a2f6f0 0x55d888a2daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.257+0000 7f057b7fe700 1 -- 172.21.15.46:0/919936062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26e90 msgr2=0x55d888a2d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.257+0000 7f057b7fe700 1 --2- 172.21.15.46:0/919936062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26e90 0x55d888a2d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.258+0000 7f057b7fe700 1 -- 172.21.15.46:0/919936062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d88893fe40 con 0x55d888a26570 2024-03-18T20:55:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.258+0000 7f057b7fe700 1 --2- 172.21.15.46:0/919936062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26570 0x55d888a295c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0574033c60 tx=0x7f0574015840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c71fad78c6a566 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.259+0000 7f05627fc700 1 -- 172.21.15.46:0/919936062 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574015650 con 0x55d888a26570 2024-03-18T20:55:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.259+0000 7f05627fc700 1 -- 172.21.15.46:0/919936062 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0574012040 con 0x55d888a26570 2024-03-18T20:55:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.259+0000 7f05627fc700 1 -- 172.21.15.46:0/919936062 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574034c00 con 0x55d888a26570 2024-03-18T20:55:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.260+0000 7f058e285b80 1 -- 172.21.15.46:0/919936062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26570 msgr2=0x55d888a295c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.260+0000 7f058e285b80 1 --2- 172.21.15.46:0/919936062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26570 0x55d888a295c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f0574033c60 tx=0x7f0574015840 comp rx=0 tx=0).stop 2024-03-18T20:55:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.260+0000 7f058e285b80 1 -- 172.21.15.46:0/919936062 shutdown_connections 2024-03-18T20:55:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.260+0000 7f058e285b80 1 --2- 172.21.15.46:0/919936062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a26570 0x55d888a295c0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.260+0000 7f058e285b80 1 --2- 172.21.15.46:0/919936062 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a2f6f0 0x55d888a2daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.260+0000 7f058e285b80 1 --2- 172.21.15.46:0/919936062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26e90 0x55d888a2d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.260+0000 7f058e285b80 1 -- 172.21.15.46:0/919936062 >> 172.21.15.46:0/919936062 conn(0x55d88899d1c0 msgr2=0x55d888a29f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.265+0000 7f058e285b80 1 -- 172.21.15.46:0/919936062 shutdown_connections 2024-03-18T20:55:34.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.266+0000 7f058e285b80 1 -- 172.21.15.46:0/919936062 wait complete. 2024-03-18T20:55:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.266+0000 7f058e285b80 1 -- start start 2024-03-18T20:55:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.266+0000 7f058e285b80 1 Processor -- start 2024-03-18T20:55:34.269 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.267+0000 7f058e285b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26570 0x55d888a36a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.267+0000 7f058e285b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26e90 0x55d888a36fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.267+0000 7f058e285b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a2f6f0 0x55d888a37500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.267+0000 7f058e285b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d888a94bc0 con 0x55d888a2f6f0 2024-03-18T20:55:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.267+0000 7f058e285b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d888a94d30 con 0x55d888a26570 2024-03-18T20:55:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.267+0000 7f058e285b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d888a94ef0 con 0x55d888a26e90 2024-03-18T20:55:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.267+0000 7f058e285b80 10 client.0 _ll_register_callbacks cb 0x7f057400c320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:55:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.273+0000 7f057affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26e90 0x55d888a36fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.273+0000 7f057affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26e90 0x55d888a36fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40950/0 (socket says 172.21.15.46:40950) 2024-03-18T20:55:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.273+0000 7f057affd700 1 -- 172.21.15.46:0/2083675127 learned_addr learned my addr 172.21.15.46:0/2083675127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.283+0000 7f057bfff700 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a2f6f0 0x55d888a37500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.284+0000 7f057b7fe700 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26570 0x55d888a36a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.285+0000 7f057affd700 1 -- 172.21.15.46:0/2083675127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26570 msgr2=0x55d888a36a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.285+0000 7f057affd700 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26570 0x55d888a36a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.285+0000 7f057affd700 1 -- 172.21.15.46:0/2083675127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a2f6f0 msgr2=0x55d888a37500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.285+0000 7f057affd700 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a2f6f0 0x55d888a37500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.285+0000 7f057affd700 1 -- 172.21.15.46:0/2083675127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8887a7b00 con 0x55d888a26e90 2024-03-18T20:55:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.293+0000 7f057affd700 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26e90 0x55d888a36fc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f057009a8f0 tx=0x7f0570096f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f6ffd66e50b56ac server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.302+0000 7f055affd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:55:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.302+0000 7f055affd700 1 -- 172.21.15.46:0/2083675127 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057009ea20 con 0x55d888a26e90 2024-03-18T20:55:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.302+0000 7f055affd700 1 -- 172.21.15.46:0/2083675127 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f05700a2fe0 con 0x55d888a26e90 2024-03-18T20:55:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.302+0000 7f058e285b80 1 -- 172.21.15.46:0/2083675127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f057400c0d0 con 0x55d888a26e90 2024-03-18T20:55:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.314+0000 7f055affd700 1 -- 172.21.15.46:0/2083675127 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0570098f90 con 0x55d888a26e90 2024-03-18T20:55:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.318+0000 7f058e285b80 10 client.30235 fetch_fsmap learned FSMap version 85 2024-03-18T20:55:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.318+0000 7f058e285b80 1 -- 172.21.15.46:0/2083675127 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=4+}) v3 -- 0x55d8887ffc70 con 0x55d888a26e90 2024-03-18T20:55:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.335+0000 7f055affd700 1 -- 172.21.15.46:0/2083675127 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f057400c0d0 con 0x55d888a26e90 2024-03-18T20:55:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.335+0000 7f055affd700 1 -- 172.21.15.46:0/2083675127 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 85) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f05700a2b50 con 0x55d888a26e90 2024-03-18T20:55:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.335+0000 7f058e285b80 10 client.30235 fetch_fsmap finished waiting for FSMap version 85 2024-03-18T20:55:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.335+0000 7f058e285b80 10 client.30235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:55:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.335+0000 7f058e285b80 20 client.30235 populate_metadata read hostname 'smithi046' 2024-03-18T20:55:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.335+0000 7f058e285b80 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d888a96430 0x55d888a98850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.335+0000 7f058e285b80 4 client.30235 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:55:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.335+0000 7f058e285b80 1 -- 172.21.15.46:0/2083675127 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d8887c8c90 con 0x55d888a96430 2024-03-18T20:55:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.349+0000 7f057b7fe700 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d888a96430 0x55d888a98850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:34.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.393+0000 7f057b7fe700 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d888a96430 0x55d888a98850 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=babc1c83bcfa96e4 server_cookie=7a5d0108f66dfc4 in_seq=0 out_seq=0 2024-03-18T20:55:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:34.404+0000 7f055affd700 10 client.30235 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:34.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 276848 36.06 MB/sec execute 250 sec latency 883.285 ms 2024-03-18T20:55:34.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:34 smithi046 ceph-mon[43495]: pgmap v613: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.3 MiB/s wr, 615 op/s 2024-03-18T20:55:34.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:34 smithi073 ceph-mon[47384]: pgmap v613: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.3 MiB/s wr, 615 op/s 2024-03-18T20:55:35.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:34 smithi155 ceph-mon[48344]: pgmap v613: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.3 MiB/s wr, 615 op/s 2024-03-18T20:55:35.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 276950 35.95 MB/sec execute 251 sec latency 1466.527 ms 2024-03-18T20:55:35.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.734+0000 7f055affd700 1 -- 172.21.15.46:0/2083675127 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d8887c8c90 con 0x55d888a96430 2024-03-18T20:55:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.734+0000 7f055affd700 10 client.30235 handle_command_reply: tid=0 2024-03-18T20:55:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.734+0000 7f058e285b80 1 client.30235 shutdown 2024-03-18T20:55:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.735+0000 7f058e285b80 1 -- 172.21.15.46:0/2083675127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26e90 msgr2=0x55d888a36fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:35.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.735+0000 7f058e285b80 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26e90 0x55d888a36fc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f057009a8f0 tx=0x7f0570096f30 comp rx=0 tx=0).stop 2024-03-18T20:55:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.737+0000 7f058e285b80 1 -- 172.21.15.46:0/2083675127 shutdown_connections 2024-03-18T20:55:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.737+0000 7f058e285b80 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55d888a96430 0x55d888a98850 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.737+0000 7f058e285b80 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d888a2f6f0 0x55d888a37500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.737+0000 7f058e285b80 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d888a26570 0x55d888a36a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.737+0000 7f058e285b80 1 --2- 172.21.15.46:0/2083675127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d888a26e90 0x55d888a36fc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.737+0000 7f058e285b80 1 -- 172.21.15.46:0/2083675127 >> 172.21.15.46:0/2083675127 conn(0x55d88899d1c0 msgr2=0x55d888a2d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.737+0000 7f055affd700 0 client.30235 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.737+0000 7f058e285b80 1 -- 172.21.15.46:0/2083675127 shutdown_connections 2024-03-18T20:55:35.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.737+0000 7f058e285b80 1 -- 172.21.15.46:0/2083675127 wait complete. 2024-03-18T20:55:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.741+0000 7f058e285b80 20 client.30235 trim_cache size 0 max 16384 2024-03-18T20:55:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:55:35.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:55:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:55:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:55:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.745+0000 7f05820f0700 1 -- 172.21.15.46:0/1649801295 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f05640a14a0 msgr2=0x7f05640a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.745+0000 7f05820f0700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f05640a14a0 0x7f05640a3960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0574015530 tx=0x7f05740157b0 comp rx=0 tx=0).stop 2024-03-18T20:55:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.745+0000 7f05820f0700 1 -- 172.21.15.46:0/1649801295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f057c10f940 msgr2=0x7f057c118f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:35.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.745+0000 7f05820f0700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f057c10f940 0x7f057c118f80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f057000b580 tx=0x7f0570005740 comp rx=0 tx=0).stop 2024-03-18T20:55:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.746+0000 7f05820f0700 1 -- 172.21.15.46:0/1649801295 shutdown_connections 2024-03-18T20:55:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.746+0000 7f05820f0700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f05640a14a0 0x7f05640a3960 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.746+0000 7f05820f0700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f057c11a590 0x7f057c111bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.746+0000 7f05820f0700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f057c10f4d0 0x7f057c118a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.746+0000 7f05820f0700 1 --2- 172.21.15.46:0/1649801295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f057c10f940 0x7f057c118f80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.746+0000 7f05820f0700 1 -- 172.21.15.46:0/1649801295 >> 172.21.15.46:0/1649801295 conn(0x7f057c06bd80 msgr2=0x7f057c06f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.748+0000 7f05820f0700 1 -- 172.21.15.46:0/1649801295 shutdown_connections 2024-03-18T20:55:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:35.754+0000 7f05820f0700 1 -- 172.21.15.46:0/1649801295 wait complete. 2024-03-18T20:55:35.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:35 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:55:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:35 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:55:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:35 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:55:35.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:35 smithi073 ceph-mon[47384]: pgmap v614: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 786 op/s 2024-03-18T20:55:36.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:35 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:55:36.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:35 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:55:36.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:35 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:55:36.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:35 smithi155 ceph-mon[48344]: pgmap v614: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 786 op/s 2024-03-18T20:55:36.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:35 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:55:36.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:35 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:55:36.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:35 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:55:36.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:35 smithi046 ceph-mon[43495]: pgmap v614: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 786 op/s 2024-03-18T20:55:36.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 277185 35.85 MB/sec execute 252 sec latency 506.770 ms 2024-03-18T20:55:37.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 277483 35.74 MB/sec execute 253 sec latency 160.345 ms 2024-03-18T20:55:38.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 278194 35.76 MB/sec execute 254 sec latency 99.264 ms 2024-03-18T20:55:38.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:38 smithi073 ceph-mon[47384]: pgmap v615: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 10 MiB/s wr, 558 op/s 2024-03-18T20:55:38.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:38 smithi155 ceph-mon[48344]: pgmap v615: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 10 MiB/s wr, 558 op/s 2024-03-18T20:55:38.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:38 smithi046 ceph-mon[43495]: pgmap v615: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 10 MiB/s wr, 558 op/s 2024-03-18T20:55:39.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 278831 35.80 MB/sec execute 255 sec latency 118.135 ms 2024-03-18T20:55:40.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 279449 35.75 MB/sec execute 256 sec latency 127.366 ms 2024-03-18T20:55:40.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:40 smithi073 ceph-mon[47384]: pgmap v616: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 22 MiB/s wr, 1.02k op/s 2024-03-18T20:55:40.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:40 smithi155 ceph-mon[48344]: pgmap v616: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 22 MiB/s wr, 1.02k op/s 2024-03-18T20:55:40.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:40 smithi046 ceph-mon[43495]: pgmap v616: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 22 MiB/s wr, 1.02k op/s 2024-03-18T20:55:41.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 280248 35.81 MB/sec execute 257 sec latency 65.229 ms 2024-03-18T20:55:42.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 280808 35.76 MB/sec execute 258 sec latency 88.742 ms 2024-03-18T20:55:42.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:42 smithi073 ceph-mon[47384]: pgmap v617: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 20 MiB/s wr, 919 op/s 2024-03-18T20:55:42.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:42 smithi155 ceph-mon[48344]: pgmap v617: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 20 MiB/s wr, 919 op/s 2024-03-18T20:55:42.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:42 smithi046 ceph-mon[43495]: pgmap v617: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 20 MiB/s wr, 919 op/s 2024-03-18T20:55:43.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 281447 35.77 MB/sec execute 259 sec latency 108.541 ms 2024-03-18T20:55:44.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 282081 35.75 MB/sec execute 260 sec latency 78.585 ms 2024-03-18T20:55:44.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:44 smithi073 ceph-mon[47384]: pgmap v618: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-03-18T20:55:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:44 smithi155 ceph-mon[48344]: pgmap v618: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-03-18T20:55:44.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:44 smithi046 ceph-mon[43495]: pgmap v618: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-03-18T20:55:45.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 282829 35.77 MB/sec execute 261 sec latency 104.558 ms 2024-03-18T20:55:46.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 283555 35.81 MB/sec execute 262 sec latency 85.042 ms 2024-03-18T20:55:46.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:46 smithi073 ceph-mon[47384]: pgmap v619: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 21 MiB/s wr, 1.04k op/s 2024-03-18T20:55:46.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:46 smithi155 ceph-mon[48344]: pgmap v619: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 21 MiB/s wr, 1.04k op/s 2024-03-18T20:55:46.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:46 smithi046 ceph-mon[43495]: pgmap v619: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 21 MiB/s wr, 1.04k op/s 2024-03-18T20:55:47.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 284070 35.75 MB/sec execute 263 sec latency 117.171 ms 2024-03-18T20:55:47.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:47 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:55:47.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:47 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:55:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:47 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:55:48.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 284568 35.71 MB/sec execute 264 sec latency 204.076 ms 2024-03-18T20:55:48.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:48 smithi073 ceph-mon[47384]: pgmap v620: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 16 MiB/s wr, 829 op/s 2024-03-18T20:55:48.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:48 smithi155 ceph-mon[48344]: pgmap v620: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 16 MiB/s wr, 829 op/s 2024-03-18T20:55:48.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:48 smithi046 ceph-mon[43495]: pgmap v620: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 16 MiB/s wr, 829 op/s 2024-03-18T20:55:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.001+0000 7fb81b730700 1 -- 172.21.15.46:0/4215447032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8141007d0 msgr2=0x7fb814104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.001+0000 7fb81b730700 1 --2- 172.21.15.46:0/4215447032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8141007d0 0x7fb814104810 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb80401a9a0 tx=0x7fb80401acb0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.002+0000 7fb81b730700 1 -- 172.21.15.46:0/4215447032 shutdown_connections 2024-03-18T20:55:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.002+0000 7fb81b730700 1 --2- 172.21.15.46:0/4215447032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb814104f40 0x7fb81410cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.002+0000 7fb81b730700 1 --2- 172.21.15.46:0/4215447032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8140ffe20 0x7fb814100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.002+0000 7fb81b730700 1 --2- 172.21.15.46:0/4215447032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8141007d0 0x7fb814104810 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.002+0000 7fb81b730700 1 -- 172.21.15.46:0/4215447032 >> 172.21.15.46:0/4215447032 conn(0x7fb8140fb680 msgr2=0x7fb8140fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.002+0000 7fb81b730700 1 -- 172.21.15.46:0/4215447032 shutdown_connections 2024-03-18T20:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.002+0000 7fb81b730700 1 -- 172.21.15.46:0/4215447032 wait complete. 2024-03-18T20:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.003+0000 7fb81b730700 1 Processor -- start 2024-03-18T20:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.003+0000 7fb81b730700 1 -- start start 2024-03-18T20:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.003+0000 7fb81b730700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8140ffe20 0x7fb8140691b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.003+0000 7fb81b730700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb814104f40 0x7fb8140696f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.003+0000 7fb81b730700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8141b52b0 0x7fb8141b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.003+0000 7fb81b730700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8141b5d30 con 0x7fb8140ffe20 2024-03-18T20:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.003+0000 7fb81b730700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8141b5ea0 con 0x7fb814104f40 2024-03-18T20:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.003+0000 7fb81b730700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8141b6060 con 0x7fb8141b52b0 2024-03-18T20:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.003+0000 7fb8194cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8140ffe20 0x7fb8140691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.003+0000 7fb8194cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8140ffe20 0x7fb8140691b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60514/0 (socket says 172.21.15.46:60514) 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.003+0000 7fb8194cc700 1 -- 172.21.15.46:0/1547844770 learned_addr learned my addr 172.21.15.46:0/1547844770 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.003+0000 7fb818ccb700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb814104f40 0x7fb8140696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.004+0000 7fb819ccd700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8141b52b0 0x7fb8141b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.004+0000 7fb8194cc700 1 -- 172.21.15.46:0/1547844770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8141b52b0 msgr2=0x7fb8141b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.004+0000 7fb8194cc700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8141b52b0 0x7fb8141b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.004+0000 7fb8194cc700 1 -- 172.21.15.46:0/1547844770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb814104f40 msgr2=0x7fb8140696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.004+0000 7fb8194cc700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb814104f40 0x7fb8140696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.004+0000 7fb8194cc700 1 -- 172.21.15.46:0/1547844770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80401a600 con 0x7fb8140ffe20 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.004+0000 7fb8194cc700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8140ffe20 0x7fb8140691b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fb81000d950 tx=0x7fb81000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.004+0000 7fb80a7fc700 1 -- 172.21.15.46:0/1547844770 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81000f890 con 0x7fb8140ffe20 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.004+0000 7fb81b730700 1 -- 172.21.15.46:0/1547844770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb814108420 con 0x7fb8140ffe20 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.004+0000 7fb80a7fc700 1 -- 172.21.15.46:0/1547844770 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb810010460 con 0x7fb8140ffe20 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.004+0000 7fb80a7fc700 1 -- 172.21.15.46:0/1547844770 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81000e610 con 0x7fb8140ffe20 2024-03-18T20:55:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.004+0000 7fb81b730700 1 -- 172.21.15.46:0/1547844770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb814108900 con 0x7fb8140ffe20 2024-03-18T20:55:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.009+0000 7fb8278c5b80 1 Processor -- start 2024-03-18T20:55:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.010+0000 7fb80a7fc700 1 -- 172.21.15.46:0/1547844770 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fb8100108d0 con 0x7fb8140ffe20 2024-03-18T20:55:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.010+0000 7fb8278c5b80 1 -- start start 2024-03-18T20:55:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.010+0000 7fb8278c5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25ad80 0x56263f25b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.010+0000 7fb8278c5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25b6a0 0x56263f266010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.010+0000 7fb8278c5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f266550 0x56263f268940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.010+0000 7fb8278c5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263f1ac930 con 0x56263f25b6a0 2024-03-18T20:55:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.010+0000 7fb8278c5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263f1aa2f0 con 0x56263f25ad80 2024-03-18T20:55:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.010+0000 7fb8278c5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263f1a5be0 con 0x56263f266550 2024-03-18T20:55:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.010+0000 7fb819ccd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25ad80 0x56263f25b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.010+0000 7fb819ccd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25ad80 0x56263f25b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:58688/0 (socket says 172.21.15.46:58688) 2024-03-18T20:55:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.010+0000 7fb819ccd700 1 -- 172.21.15.46:0/2785051884 learned_addr learned my addr 172.21.15.46:0/2785051884 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.011+0000 7fb80a7fc700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb8000a1290 0x7fb8000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.011+0000 7fb819ccd700 1 -- 172.21.15.46:0/2785051884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f266550 msgr2=0x56263f268940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:55:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.011+0000 7fb819ccd700 1 --2- 172.21.15.46:0/2785051884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f266550 0x56263f268940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.011+0000 7fb819ccd700 1 -- 172.21.15.46:0/2785051884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25b6a0 msgr2=0x56263f266010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.011+0000 7fb819ccd700 1 --2- 172.21.15.46:0/2785051884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25b6a0 0x56263f266010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.011+0000 7fb819ccd700 1 -- 172.21.15.46:0/2785051884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56263f027350 con 0x56263f25ad80 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.011+0000 7fb80a7fc700 1 -- 172.21.15.46:0/1547844770 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fb8100cca30 con 0x7fb8140ffe20 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.011+0000 7fb819ccd700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb8000a1290 0x7fb8000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.011+0000 7fb819ccd700 1 --2- 172.21.15.46:0/2785051884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25ad80 0x56263f25b160 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e620 tx=0x7fb80c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41ea43496e27184f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.011+0000 7fb7ff7fe700 1 -- 172.21.15.46:0/2785051884 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c007700 con 0x56263f25ad80 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.011+0000 7fb7ff7fe700 1 -- 172.21.15.46:0/2785051884 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb80c011030 con 0x56263f25ad80 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.011+0000 7fb7ff7fe700 1 -- 172.21.15.46:0/2785051884 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c010050 con 0x56263f25ad80 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.012+0000 7fb819ccd700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb8000a1290 0x7fb8000a3750 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0094f0 tx=0x7fb80c01b000 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.012+0000 7fb8278c5b80 1 -- 172.21.15.46:0/2785051884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25ad80 msgr2=0x56263f25b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.012+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/2785051884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25ad80 0x56263f25b160 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e620 tx=0x7fb80c00b040 comp rx=0 tx=0).stop 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.013+0000 7fb8278c5b80 1 -- 172.21.15.46:0/2785051884 shutdown_connections 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.013+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/2785051884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f266550 0x56263f268940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.013+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/2785051884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25ad80 0x56263f25b160 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.013+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/2785051884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25b6a0 0x56263f266010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.013+0000 7fb8278c5b80 1 -- 172.21.15.46:0/2785051884 >> 172.21.15.46:0/2785051884 conn(0x56263f1c91c0 msgr2=0x56263f24e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.013+0000 7fb8278c5b80 1 -- 172.21.15.46:0/2785051884 shutdown_connections 2024-03-18T20:55:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.013+0000 7fb8278c5b80 1 -- 172.21.15.46:0/2785051884 wait complete. 2024-03-18T20:55:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.014+0000 7fb8278c5b80 1 -- start start 2024-03-18T20:55:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.014+0000 7fb8278c5b80 1 Processor -- start 2024-03-18T20:55:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.014+0000 7fb8278c5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25ad80 0x56263f265fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.014+0000 7fb8278c5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f25b6a0 0x56263f263350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.014+0000 7fb8278c5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f2639f0 0x56263f2722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.014+0000 7fb8278c5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263f141d20 con 0x56263f25ad80 2024-03-18T20:55:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.014+0000 7fb8278c5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263f1419c0 con 0x56263f2639f0 2024-03-18T20:55:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.014+0000 7fb8278c5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263f029640 con 0x56263f25b6a0 2024-03-18T20:55:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.014+0000 7fb8278c5b80 10 client.0 _ll_register_callbacks cb 0x56263efad340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:55:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.015+0000 7fb818ccb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f25b6a0 0x56263f263350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.015+0000 7fb818ccb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f25b6a0 0x56263f263350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:55618/0 (socket says 172.21.15.46:55618) 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.015+0000 7fb818ccb700 1 -- 172.21.15.46:0/1778214745 learned_addr learned my addr 172.21.15.46:0/1778214745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.015+0000 7fb818ccb700 1 -- 172.21.15.46:0/1778214745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f2639f0 msgr2=0x56263f2722a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.015+0000 7fb818ccb700 1 --2- 172.21.15.46:0/1778214745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f2639f0 0x56263f2722a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.015+0000 7fb818ccb700 1 -- 172.21.15.46:0/1778214745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25ad80 msgr2=0x56263f265fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.015+0000 7fb818ccb700 1 --2- 172.21.15.46:0/1778214745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25ad80 0x56263f265fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.015+0000 7fb818ccb700 1 -- 172.21.15.46:0/1778214745 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80c00e210 con 0x56263f25b6a0 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.015+0000 7fb818ccb700 1 --2- 172.21.15.46:0/1778214745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f25b6a0 0x56263f263350 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb80401a9a0 tx=0x7fb80401cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3d87cdc047c1562 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.015+0000 7fb7fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.016+0000 7fb7fcff9700 1 -- 172.21.15.46:0/1778214745 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8040054d0 con 0x56263f25b6a0 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.016+0000 7fb7fcff9700 1 -- 172.21.15.46:0/1778214745 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fb804005630 con 0x56263f25b6a0 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.016+0000 7fb8278c5b80 1 -- 172.21.15.46:0/1778214745 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56263f2466c0 con 0x56263f25b6a0 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.016+0000 7fb7fcff9700 1 -- 172.21.15.46:0/1778214745 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb80401ceb0 con 0x56263f25b6a0 2024-03-18T20:55:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.016+0000 7fb8278c5b80 10 client.30244 fetch_fsmap learned FSMap version 86 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.016+0000 7fb8278c5b80 1 -- 172.21.15.46:0/1778214745 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x56263efd3b00 con 0x56263f25b6a0 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.017+0000 7fb7fcff9700 1 -- 172.21.15.46:0/1778214745 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56263f2466c0 con 0x56263f25b6a0 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.017+0000 7fb7fcff9700 1 -- 172.21.15.46:0/1778214745 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 86) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb804014020 con 0x56263f25b6a0 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.017+0000 7fb8278c5b80 10 client.30244 fetch_fsmap finished waiting for FSMap version 86 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.017+0000 7fb8278c5b80 10 client.30244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.017+0000 7fb8278c5b80 20 client.30244 populate_metadata read hostname 'smithi046' 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.017+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/1778214745 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56263f2764e0 0x56263f278900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.017+0000 7fb8278c5b80 4 client.30244 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.017+0000 7fb8278c5b80 1 -- 172.21.15.46:0/1778214745 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56263f19e6a0 con 0x56263f2764e0 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.017+0000 7fb8194cc700 1 --2- 172.21.15.46:0/1778214745 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56263f2764e0 0x56263f278900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.018+0000 7fb8194cc700 1 --2- 172.21.15.46:0/1778214745 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56263f2764e0 0x56263f278900 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c1e63cacddcb92 server_cookie=5471fd86bdb4526 in_seq=0 out_seq=0 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.018+0000 7fb7fcff9700 10 client.30244 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb7fcff9700 1 -- 172.21.15.46:0/1778214745 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56263f19e6a0 con 0x56263f2764e0 2024-03-18T20:55:49.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb7fcff9700 10 client.30244 handle_command_reply: tid=0 2024-03-18T20:55:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb8278c5b80 1 client.30244 shutdown 2024-03-18T20:55:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb8278c5b80 1 -- 172.21.15.46:0/1778214745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f25b6a0 msgr2=0x56263f263350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/1778214745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f25b6a0 0x56263f263350 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb80401a9a0 tx=0x7fb80401cc90 comp rx=0 tx=0).stop 2024-03-18T20:55:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb8278c5b80 1 -- 172.21.15.46:0/1778214745 shutdown_connections 2024-03-18T20:55:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/1778214745 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56263f2764e0 0x56263f278900 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/1778214745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25ad80 0x56263f265fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/1778214745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f2639f0 0x56263f2722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/1778214745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f25b6a0 0x56263f263350 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb8278c5b80 1 -- 172.21.15.46:0/1778214745 >> 172.21.15.46:0/1778214745 conn(0x56263f1c91c0 msgr2=0x56263f24e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb7fcff9700 0 client.30244 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb8278c5b80 1 -- 172.21.15.46:0/1778214745 shutdown_connections 2024-03-18T20:55:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb8278c5b80 1 -- 172.21.15.46:0/1778214745 wait complete. 2024-03-18T20:55:49.026 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.019+0000 7fb8278c5b80 20 client.30244 trim_cache size 0 max 16384 2024-03-18T20:55:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.044+0000 7fb8278c5b80 1 Processor -- start 2024-03-18T20:55:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb8278c5b80 1 -- start start 2024-03-18T20:55:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb8278c5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f25ad80 0x56263f25dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb8278c5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 0x56263f26d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb8278c5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f2639f0 0x56263f2605b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb8278c5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263f248ca0 con 0x56263f2639f0 2024-03-18T20:55:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb8278c5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263f26d750 con 0x56263f25b6a0 2024-03-18T20:55:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb8278c5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263f261880 con 0x56263f25ad80 2024-03-18T20:55:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb818ccb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 0x56263f26d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb818ccb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 0x56263f26d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:58702/0 (socket says 172.21.15.46:58702) 2024-03-18T20:55:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb818ccb700 1 -- 172.21.15.46:0/4281232033 learned_addr learned my addr 172.21.15.46:0/4281232033 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb819ccd700 1 --2- 172.21.15.46:0/4281232033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f2639f0 0x56263f2605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb818ccb700 1 -- 172.21.15.46:0/4281232033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f25ad80 msgr2=0x56263f25dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.045+0000 7fb818ccb700 1 --2- 172.21.15.46:0/4281232033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f25ad80 0x56263f25dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.046+0000 7fb818ccb700 1 -- 172.21.15.46:0/4281232033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f2639f0 msgr2=0x56263f2605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.046+0000 7fb818ccb700 1 --2- 172.21.15.46:0/4281232033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f2639f0 0x56263f2605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.046+0000 7fb818ccb700 1 -- 172.21.15.46:0/4281232033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56263f24a8f0 con 0x56263f25b6a0 2024-03-18T20:55:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.046+0000 7fb819ccd700 1 --2- 172.21.15.46:0/4281232033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f2639f0 0x56263f2605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:55:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.046+0000 7fb818ccb700 1 --2- 172.21.15.46:0/4281232033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 0x56263f26d1e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb804005a60 tx=0x7fb80400ea10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d321dd505c0d93d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.046+0000 7fb7fffff700 1 -- 172.21.15.46:0/4281232033 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80400e410 con 0x56263f25b6a0 2024-03-18T20:55:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.046+0000 7fb7fffff700 1 -- 172.21.15.46:0/4281232033 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb80400e570 con 0x56263f25b6a0 2024-03-18T20:55:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.046+0000 7fb7fffff700 1 -- 172.21.15.46:0/4281232033 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804016400 con 0x56263f25b6a0 2024-03-18T20:55:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.047+0000 7fb8278c5b80 1 -- 172.21.15.46:0/4281232033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 msgr2=0x56263f26d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.047+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/4281232033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 0x56263f26d1e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb804005a60 tx=0x7fb80400ea10 comp rx=0 tx=0).stop 2024-03-18T20:55:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.047+0000 7fb8278c5b80 1 -- 172.21.15.46:0/4281232033 shutdown_connections 2024-03-18T20:55:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.047+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/4281232033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f2639f0 0x56263f2605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.047+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/4281232033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f25ad80 0x56263f25dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.047+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/4281232033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 0x56263f26d1e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.047+0000 7fb8278c5b80 1 -- 172.21.15.46:0/4281232033 >> 172.21.15.46:0/4281232033 conn(0x56263f1c91c0 msgr2=0x56263f261c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.048+0000 7fb8278c5b80 1 -- 172.21.15.46:0/4281232033 shutdown_connections 2024-03-18T20:55:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.048+0000 7fb8278c5b80 1 -- 172.21.15.46:0/4281232033 wait complete. 2024-03-18T20:55:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.048+0000 7fb8278c5b80 1 -- start start 2024-03-18T20:55:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.048+0000 7fb8278c5b80 1 Processor -- start 2024-03-18T20:55:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8278c5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25ad80 0x56263f26d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8278c5b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 0x56263f26b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8278c5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f2639f0 0x56263f26b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8278c5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263f26bff0 con 0x56263f25ad80 2024-03-18T20:55:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8278c5b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263f2c92d0 con 0x56263f25b6a0 2024-03-18T20:55:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8278c5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56263f2c9410 con 0x56263f2639f0 2024-03-18T20:55:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8278c5b80 10 client.0 _ll_register_callbacks cb 0x7fb804005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:55:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb818ccb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 0x56263f26b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8194cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25ad80 0x56263f26d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb818ccb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 0x56263f26b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:58716/0 (socket says 172.21.15.46:58716) 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb818ccb700 1 -- 172.21.15.46:0/3000947060 learned_addr learned my addr 172.21.15.46:0/3000947060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8194cc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25ad80 0x56263f26d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60540/0 (socket says 172.21.15.46:60540) 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb819ccd700 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f2639f0 0x56263f26b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8194cc700 1 -- 172.21.15.46:0/3000947060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f2639f0 msgr2=0x56263f26b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8194cc700 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f2639f0 0x56263f26b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8194cc700 1 -- 172.21.15.46:0/3000947060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 msgr2=0x56263f26b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8194cc700 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 0x56263f26b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb8194cc700 1 -- 172.21.15.46:0/3000947060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56263f027350 con 0x56263f25ad80 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.049+0000 7fb818ccb700 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 0x56263f26b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.050+0000 7fb8194cc700 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25ad80 0x56263f26d3e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fb8100ca160 tx=0x7fb810099580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787498a57bd29424 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.050+0000 7fb7fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:55:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.050+0000 7fb7fcff9700 1 -- 172.21.15.46:0/3000947060 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8100ccea0 con 0x56263f25ad80 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.050+0000 7fb7fcff9700 1 -- 172.21.15.46:0/3000947060 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fb8100a1fa0 con 0x56263f25ad80 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.050+0000 7fb8278c5b80 1 -- 172.21.15.46:0/3000947060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb80c011030 con 0x56263f25ad80 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.050+0000 7fb7fcff9700 1 -- 172.21.15.46:0/3000947060 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb810099d80 con 0x56263f25ad80 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.050+0000 7fb8278c5b80 10 client.20541 fetch_fsmap learned FSMap version 86 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.050+0000 7fb8278c5b80 1 -- 172.21.15.46:0/3000947060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=4+}) v3 -- 0x56263efd3b00 con 0x56263f25ad80 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.050+0000 7fb7fcff9700 1 -- 172.21.15.46:0/3000947060 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb80c011030 con 0x56263f25ad80 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.050+0000 7fb7fcff9700 1 -- 172.21.15.46:0/3000947060 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 86) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb8100a1b10 con 0x56263f25ad80 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.051+0000 7fb8278c5b80 10 client.20541 fetch_fsmap finished waiting for FSMap version 86 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.051+0000 7fb8278c5b80 10 client.20541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.051+0000 7fb8278c5b80 20 client.20541 populate_metadata read hostname 'smithi046' 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.051+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56263f2caa00 0x56263f2cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.051+0000 7fb8278c5b80 4 client.20541 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:55:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.051+0000 7fb8278c5b80 1 -- 172.21.15.46:0/3000947060 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56263eff4c90 con 0x56263f2caa00 2024-03-18T20:55:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.052+0000 7fb818ccb700 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56263f2caa00 0x56263f2cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:55:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.054+0000 7fb818ccb700 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56263f2caa00 0x56263f2cce20 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12b70a691371a43 server_cookie=2acbb5a29cd557ab in_seq=0 out_seq=0 2024-03-18T20:55:49.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.054+0000 7fb7fcff9700 10 client.20541 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.284+0000 7fb7fcff9700 1 -- 172.21.15.46:0/3000947060 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56263eff4c90 con 0x56263f2caa00 2024-03-18T20:55:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.284+0000 7fb7fcff9700 10 client.20541 handle_command_reply: tid=0 2024-03-18T20:55:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.284+0000 7fb8278c5b80 1 client.20541 shutdown 2024-03-18T20:55:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.285+0000 7fb8278c5b80 1 -- 172.21.15.46:0/3000947060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25ad80 msgr2=0x56263f26d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.285+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25ad80 0x56263f26d3e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fb8100ca160 tx=0x7fb810099580 comp rx=0 tx=0).stop 2024-03-18T20:55:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.285+0000 7fb8278c5b80 1 -- 172.21.15.46:0/3000947060 shutdown_connections 2024-03-18T20:55:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.285+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56263f2caa00 0x56263f2cce20 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.285+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56263f25ad80 0x56263f26d3e0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.285+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56263f2639f0 0x56263f26b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.285+0000 7fb8278c5b80 1 --2- 172.21.15.46:0/3000947060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56263f25b6a0 0x56263f26b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.285+0000 7fb7fcff9700 0 client.20541 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:55:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.285+0000 7fb8278c5b80 1 -- 172.21.15.46:0/3000947060 >> 172.21.15.46:0/3000947060 conn(0x56263f1c91c0 msgr2=0x56263f262870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.285+0000 7fb8278c5b80 1 -- 172.21.15.46:0/3000947060 shutdown_connections 2024-03-18T20:55:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.285+0000 7fb8278c5b80 1 -- 172.21.15.46:0/3000947060 wait complete. 2024-03-18T20:55:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.285+0000 7fb8278c5b80 20 client.20541 trim_cache size 0 max 16384 2024-03-18T20:55:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:55:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:55:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:55:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:55:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.287+0000 7fb81b730700 1 -- 172.21.15.46:0/1547844770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb8000a1290 msgr2=0x7fb8000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.287+0000 7fb81b730700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb8000a1290 0x7fb8000a3750 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0094f0 tx=0x7fb80c01b000 comp rx=0 tx=0).stop 2024-03-18T20:55:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.287+0000 7fb81b730700 1 -- 172.21.15.46:0/1547844770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8140ffe20 msgr2=0x7fb8140691b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:55:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.287+0000 7fb81b730700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8140ffe20 0x7fb8140691b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fb81000d950 tx=0x7fb81000dc60 comp rx=0 tx=0).stop 2024-03-18T20:55:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.287+0000 7fb81b730700 1 -- 172.21.15.46:0/1547844770 shutdown_connections 2024-03-18T20:55:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.287+0000 7fb81b730700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fb8000a1290 0x7fb8000a3750 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.287+0000 7fb81b730700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8140ffe20 0x7fb8140691b0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.287+0000 7fb81b730700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8141b52b0 0x7fb8141b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.287+0000 7fb81b730700 1 --2- 172.21.15.46:0/1547844770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb814104f40 0x7fb8140696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:55:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.287+0000 7fb81b730700 1 -- 172.21.15.46:0/1547844770 >> 172.21.15.46:0/1547844770 conn(0x7fb8140fb680 msgr2=0x7fb8140fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:55:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.287+0000 7fb81b730700 1 -- 172.21.15.46:0/1547844770 shutdown_connections 2024-03-18T20:55:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:55:49.287+0000 7fb81b730700 1 -- 172.21.15.46:0/1547844770 wait complete. 2024-03-18T20:55:49.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 285120 35.69 MB/sec execute 265 sec latency 187.006 ms 2024-03-18T20:55:50.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 285869 35.71 MB/sec execute 266 sec latency 84.474 ms 2024-03-18T20:55:50.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:50 smithi073 ceph-mon[47384]: pgmap v621: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 1.06k op/s 2024-03-18T20:55:50.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:50 smithi155 ceph-mon[48344]: pgmap v621: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 1.06k op/s 2024-03-18T20:55:50.987 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:50 smithi046 ceph-mon[43495]: pgmap v621: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 1.06k op/s 2024-03-18T20:55:51.421 INFO:tasks.workunit.client.0.smithi046.stdout: 10 286207 35.67 MB/sec execute 267 sec latency 677.665 ms 2024-03-18T20:55:52.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 286502 35.61 MB/sec execute 268 sec latency 722.663 ms 2024-03-18T20:55:52.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:52 smithi073 ceph-mon[47384]: pgmap v622: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.6 MiB/s wr, 626 op/s 2024-03-18T20:55:52.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:52 smithi155 ceph-mon[48344]: pgmap v622: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.6 MiB/s wr, 626 op/s 2024-03-18T20:55:52.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:52 smithi046 ceph-mon[43495]: pgmap v622: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.6 MiB/s wr, 626 op/s 2024-03-18T20:55:53.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 286930 35.54 MB/sec execute 269 sec latency 99.855 ms 2024-03-18T20:55:54.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 287526 35.54 MB/sec execute 270 sec latency 115.001 ms 2024-03-18T20:55:54.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:54 smithi073 ceph-mon[47384]: pgmap v623: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 640 op/s 2024-03-18T20:55:54.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:54 smithi155 ceph-mon[48344]: pgmap v623: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 640 op/s 2024-03-18T20:55:54.995 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:54 smithi046 ceph-mon[43495]: pgmap v623: 97 pgs: 97 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 640 op/s 2024-03-18T20:55:55.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 288066 35.48 MB/sec execute 271 sec latency 123.557 ms 2024-03-18T20:55:56.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 288866 35.48 MB/sec execute 272 sec latency 71.950 ms 2024-03-18T20:55:56.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:56 smithi073 ceph-mon[47384]: pgmap v624: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 16 MiB/s wr, 849 op/s 2024-03-18T20:55:56.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:56 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:55:56.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:56 smithi155 ceph-mon[48344]: pgmap v624: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 16 MiB/s wr, 849 op/s 2024-03-18T20:55:56.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:56 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:55:56.982 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:56 smithi046 ceph-mon[43495]: pgmap v624: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 16 MiB/s wr, 849 op/s 2024-03-18T20:55:56.982 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:56 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:55:57.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 289676 35.57 MB/sec execute 273 sec latency 95.454 ms 2024-03-18T20:55:58.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 290212 35.55 MB/sec execute 274 sec latency 106.584 ms 2024-03-18T20:55:58.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:55:58 smithi073 ceph-mon[47384]: pgmap v625: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 14 MiB/s wr, 696 op/s 2024-03-18T20:55:58.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:55:58 smithi046 ceph-mon[43495]: pgmap v625: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 14 MiB/s wr, 696 op/s 2024-03-18T20:55:58.796 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:55:58 smithi155 ceph-mon[48344]: pgmap v625: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 14 MiB/s wr, 696 op/s 2024-03-18T20:55:59.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 290809 35.51 MB/sec execute 275 sec latency 111.108 ms 2024-03-18T20:56:00.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 291406 35.50 MB/sec execute 276 sec latency 87.901 ms 2024-03-18T20:56:00.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:00 smithi073 ceph-mon[47384]: pgmap v626: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 17 MiB/s wr, 919 op/s 2024-03-18T20:56:00.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:00 smithi155 ceph-mon[48344]: pgmap v626: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 17 MiB/s wr, 919 op/s 2024-03-18T20:56:00.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:00 smithi046 ceph-mon[43495]: pgmap v626: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 17 MiB/s wr, 919 op/s 2024-03-18T20:56:01.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 291892 35.43 MB/sec execute 277 sec latency 240.493 ms 2024-03-18T20:56:02.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 291893 35.31 MB/sec execute 278 sec latency 1240.559 ms 2024-03-18T20:56:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.698+0000 7f9f247d3700 1 -- 172.21.15.46:0/3412174592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f1c0ffbb0 msgr2=0x7f9f1c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.698+0000 7f9f247d3700 1 --2- 172.21.15.46:0/3412174592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f1c0ffbb0 0x7f9f1c100030 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009990 tx=0x7f9f0c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.698+0000 7f9f247d3700 1 -- 172.21.15.46:0/3412174592 shutdown_connections 2024-03-18T20:56:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.698+0000 7f9f247d3700 1 --2- 172.21.15.46:0/3412174592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f1c100570 0x7f9f1c10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.698+0000 7f9f247d3700 1 --2- 172.21.15.46:0/3412174592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f1c105c60 0x7f9f1c106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.698+0000 7f9f247d3700 1 --2- 172.21.15.46:0/3412174592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f1c0ffbb0 0x7f9f1c100030 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.698+0000 7f9f247d3700 1 -- 172.21.15.46:0/3412174592 >> 172.21.15.46:0/3412174592 conn(0x7f9f1c076d50 msgr2=0x7f9f1c077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.698+0000 7f9f247d3700 1 -- 172.21.15.46:0/3412174592 shutdown_connections 2024-03-18T20:56:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.698+0000 7f9f247d3700 1 -- 172.21.15.46:0/3412174592 wait complete. 2024-03-18T20:56:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f247d3700 1 Processor -- start 2024-03-18T20:56:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f247d3700 1 -- start start 2024-03-18T20:56:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f247d3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f1c0ffbb0 0x7f9f1c1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f247d3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f1c100570 0x7f9f1c197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f247d3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f1c105c60 0x7f9f1c19b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f247d3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c19c010 con 0x7f9f1c0ffbb0 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f247d3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c19c180 con 0x7f9f1c105c60 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f247d3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c19c340 con 0x7f9f1c100570 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f2256f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f1c0ffbb0 0x7f9f1c1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f2256f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f1c0ffbb0 0x7f9f1c1970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47798/0 (socket says 172.21.15.46:47798) 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f2256f700 1 -- 172.21.15.46:0/1422079791 learned_addr learned my addr 172.21.15.46:0/1422079791 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f22d70700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f1c105c60 0x7f9f1c19b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.699+0000 7f9f21d6e700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f1c100570 0x7f9f1c197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.700+0000 7f9f2256f700 1 -- 172.21.15.46:0/1422079791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f1c100570 msgr2=0x7f9f1c197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.700+0000 7f9f2256f700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f1c100570 0x7f9f1c197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.700+0000 7f9f2256f700 1 -- 172.21.15.46:0/1422079791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f1c105c60 msgr2=0x7f9f1c19b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.700+0000 7f9f2256f700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f1c105c60 0x7f9f1c19b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.700+0000 7f9f2256f700 1 -- 172.21.15.46:0/1422079791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c0095f0 con 0x7f9f1c0ffbb0 2024-03-18T20:56:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.700+0000 7f9f2256f700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f1c0ffbb0 0x7f9f1c1970f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800e9a0 tx=0x7f9f1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.700+0000 7f9f137fe700 1 -- 172.21.15.46:0/1422079791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18009790 con 0x7f9f1c0ffbb0 2024-03-18T20:56:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.700+0000 7f9f137fe700 1 -- 172.21.15.46:0/1422079791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f1800cbd0 con 0x7f9f1c0ffbb0 2024-03-18T20:56:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.700+0000 7f9f247d3700 1 -- 172.21.15.46:0/1422079791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1c19c620 con 0x7f9f1c0ffbb0 2024-03-18T20:56:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.700+0000 7f9f137fe700 1 -- 172.21.15.46:0/1422079791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18018680 con 0x7f9f1c0ffbb0 2024-03-18T20:56:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.700+0000 7f9f247d3700 1 -- 172.21.15.46:0/1422079791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c1a3e60 con 0x7f9f1c0ffbb0 2024-03-18T20:56:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.703+0000 7f9f137fe700 1 -- 172.21.15.46:0/1422079791 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9f1800cd40 con 0x7f9f1c0ffbb0 2024-03-18T20:56:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.704+0000 7f9f137fe700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9f080a1310 0x7f9f080a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.704+0000 7f9f137fe700 1 -- 172.21.15.46:0/1422079791 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f9f18014070 con 0x7f9f1c0ffbb0 2024-03-18T20:56:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.704+0000 7f9f21d6e700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9f080a1310 0x7f9f080a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.705+0000 7f9f21d6e700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9f080a1310 0x7f9f080a37d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00b2e0 tx=0x7f9f0c005fd0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.705+0000 7f9f30968b80 1 Processor -- start 2024-03-18T20:56:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.705+0000 7f9f30968b80 1 -- start start 2024-03-18T20:56:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.705+0000 7f9f30968b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae1d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.705+0000 7f9f30968b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 0x562eaae28870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.705+0000 7f9f30968b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 0x562eaae2b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.705+0000 7f9f30968b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eaad77930 con 0x562eaae28db0 2024-03-18T20:56:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.705+0000 7f9f30968b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eaad752f0 con 0x562eaae1d570 2024-03-18T20:56:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.705+0000 7f9f30968b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eaad70be0 con 0x562eaae1de90 2024-03-18T20:56:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.705+0000 7f9f22d70700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 0x562eaae2b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f22d70700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 0x562eaae2b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47804/0 (socket says 172.21.15.46:47804) 2024-03-18T20:56:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f22d70700 1 -- 172.21.15.46:0/379803924 learned_addr learned my addr 172.21.15.46:0/379803924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f2256f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae1d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f21d6e700 1 --2- 172.21.15.46:0/379803924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 0x562eaae28870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f2256f700 1 -- 172.21.15.46:0/379803924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 msgr2=0x562eaae28870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f2256f700 1 --2- 172.21.15.46:0/379803924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 0x562eaae28870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f2256f700 1 -- 172.21.15.46:0/379803924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 msgr2=0x562eaae2b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f2256f700 1 --2- 172.21.15.46:0/379803924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 0x562eaae2b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f2256f700 1 -- 172.21.15.46:0/379803924 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eaabf2350 con 0x562eaae1d570 2024-03-18T20:56:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f2256f700 1 --2- 172.21.15.46:0/379803924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae1d950 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9f180ca630 tx=0x7f9f18099950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49ff3e6b00bba86 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f10ff9700 1 -- 172.21.15.46:0/379803924 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1809de20 con 0x562eaae1d570 2024-03-18T20:56:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f10ff9700 1 -- 172.21.15.46:0/379803924 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f180a2470 con 0x562eaae1d570 2024-03-18T20:56:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.706+0000 7f9f10ff9700 1 -- 172.21.15.46:0/379803924 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f180a1fe0 con 0x562eaae1d570 2024-03-18T20:56:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.707+0000 7f9f30968b80 1 -- 172.21.15.46:0/379803924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 msgr2=0x562eaae1d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.707+0000 7f9f30968b80 1 --2- 172.21.15.46:0/379803924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae1d950 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9f180ca630 tx=0x7f9f18099950 comp rx=0 tx=0).stop 2024-03-18T20:56:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.707+0000 7f9f30968b80 1 -- 172.21.15.46:0/379803924 shutdown_connections 2024-03-18T20:56:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.707+0000 7f9f30968b80 1 --2- 172.21.15.46:0/379803924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 0x562eaae2b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.707+0000 7f9f30968b80 1 --2- 172.21.15.46:0/379803924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae1d950 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.707+0000 7f9f30968b80 1 --2- 172.21.15.46:0/379803924 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 0x562eaae28870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.707+0000 7f9f30968b80 1 -- 172.21.15.46:0/379803924 >> 172.21.15.46:0/379803924 conn(0x562eaad941c0 msgr2=0x562eaae19a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.707+0000 7f9f30968b80 1 -- 172.21.15.46:0/379803924 shutdown_connections 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.707+0000 7f9f30968b80 1 -- 172.21.15.46:0/379803924 wait complete. 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f30968b80 1 -- start start 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f30968b80 1 Processor -- start 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f30968b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae28820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f30968b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1de90 0x562eaae25b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f30968b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae28db0 0x562eaae26270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f30968b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eaad0cd20 con 0x562eaae1de90 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f30968b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eaad0c9c0 con 0x562eaae1d570 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f30968b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eaabf4640 con 0x562eaae28db0 2024-03-18T20:56:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f30968b80 10 client.0 _ll_register_callbacks cb 0x562eaab78340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f21d6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1de90 0x562eaae25b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f21d6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1de90 0x562eaae25b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47810/0 (socket says 172.21.15.46:47810) 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f21d6e700 1 -- 172.21.15.46:0/1308574735 learned_addr learned my addr 172.21.15.46:0/1308574735 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f22d70700 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae28db0 0x562eaae26270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f2256f700 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae28820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f21d6e700 1 -- 172.21.15.46:0/1308574735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae28db0 msgr2=0x562eaae26270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f21d6e700 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae28db0 0x562eaae26270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.708+0000 7f9f21d6e700 1 -- 172.21.15.46:0/1308574735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 msgr2=0x562eaae28820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.709+0000 7f9f21d6e700 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae28820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.709+0000 7f9f21d6e700 1 -- 172.21.15.46:0/1308574735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c1a3e60 con 0x562eaae1de90 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.709+0000 7f9f21d6e700 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1de90 0x562eaae25b80 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c015000 tx=0x7f9f0c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407470f73778551c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.709+0000 7f9f05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.709+0000 7f9f05ffb700 1 -- 172.21.15.46:0/1308574735 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c02d040 con 0x562eaae1de90 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.709+0000 7f9f05ffb700 1 -- 172.21.15.46:0/1308574735 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f9f0c004a90 con 0x562eaae1de90 2024-03-18T20:56:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.709+0000 7f9f30968b80 1 -- 172.21.15.46:0/1308574735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562eaae116c0 con 0x562eaae1de90 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.709+0000 7f9f05ffb700 1 -- 172.21.15.46:0/1308574735 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f0c004c50 con 0x562eaae1de90 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.709+0000 7f9f30968b80 10 client.20553 fetch_fsmap learned FSMap version 87 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.709+0000 7f9f30968b80 1 -- 172.21.15.46:0/1308574735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x562eaab9eb00 con 0x562eaae1de90 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.709+0000 7f9f05ffb700 1 -- 172.21.15.46:0/1308574735 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562eaae116c0 con 0x562eaae1de90 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.710+0000 7f9f05ffb700 1 -- 172.21.15.46:0/1308574735 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 87) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f0c00f460 con 0x562eaae1de90 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.710+0000 7f9f30968b80 10 client.20553 fetch_fsmap finished waiting for FSMap version 87 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.710+0000 7f9f30968b80 10 client.20553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.710+0000 7f9f30968b80 20 client.20553 populate_metadata read hostname 'smithi046' 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.710+0000 7f9f30968b80 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562eaae3ce70 0x562eaae3f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.710+0000 7f9f30968b80 4 client.20553 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.710+0000 7f9f30968b80 1 -- 172.21.15.46:0/1308574735 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562eaad696a0 con 0x562eaae3ce70 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.710+0000 7f9f2256f700 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562eaae3ce70 0x562eaae3f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.710+0000 7f9f05ffb700 10 client.20553 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.710+0000 7f9f2256f700 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562eaae3ce70 0x562eaae3f290 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410ddc95e5e38d0a server_cookie=2ced82d9069d8f1c in_seq=0 out_seq=0 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f05ffb700 1 -- 172.21.15.46:0/1308574735 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562eaad696a0 con 0x562eaae3ce70 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f05ffb700 10 client.20553 handle_command_reply: tid=0 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f30968b80 1 client.20553 shutdown 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f30968b80 1 -- 172.21.15.46:0/1308574735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1de90 msgr2=0x562eaae25b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f30968b80 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1de90 0x562eaae25b80 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c015000 tx=0x7f9f0c01f000 comp rx=0 tx=0).stop 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f30968b80 1 -- 172.21.15.46:0/1308574735 shutdown_connections 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f30968b80 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562eaae3ce70 0x562eaae3f290 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f30968b80 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae28db0 0x562eaae26270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f30968b80 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae28820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f30968b80 1 --2- 172.21.15.46:0/1308574735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1de90 0x562eaae25b80 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f05ffb700 0 client.20553 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f30968b80 1 -- 172.21.15.46:0/1308574735 >> 172.21.15.46:0/1308574735 conn(0x562eaad941c0 msgr2=0x562eaae19650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f30968b80 1 -- 172.21.15.46:0/1308574735 shutdown_connections 2024-03-18T20:56:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.711+0000 7f9f30968b80 1 -- 172.21.15.46:0/1308574735 wait complete. 2024-03-18T20:56:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.712+0000 7f9f30968b80 20 client.20553 trim_cache size 0 max 16384 2024-03-18T20:56:02.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:02 smithi073 ceph-mon[47384]: pgmap v627: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 13 MiB/s wr, 701 op/s 2024-03-18T20:56:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.727+0000 7f9f30968b80 1 Processor -- start 2024-03-18T20:56:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.727+0000 7f9f30968b80 1 -- start start 2024-03-18T20:56:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.727+0000 7f9f30968b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae26230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.727+0000 7f9f30968b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 0x562eaae33e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.727+0000 7f9f30968b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 0x562eaae20e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.727+0000 7f9f30968b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eaae1e850 con 0x562eaae28db0 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.727+0000 7f9f30968b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eaae1ad60 con 0x562eaae1d570 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.727+0000 7f9f30968b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eaae13c10 con 0x562eaae1de90 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.727+0000 7f9f22d70700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 0x562eaae20e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.727+0000 7f9f22d70700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 0x562eaae20e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47820/0 (socket says 172.21.15.46:47820) 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.727+0000 7f9f22d70700 1 -- 172.21.15.46:0/4175345331 learned_addr learned my addr 172.21.15.46:0/4175345331 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.728+0000 7f9f2256f700 1 --2- 172.21.15.46:0/4175345331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae26230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.728+0000 7f9f21d6e700 1 --2- 172.21.15.46:0/4175345331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 0x562eaae33e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.728+0000 7f9f22d70700 1 -- 172.21.15.46:0/4175345331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 msgr2=0x562eaae33e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.728+0000 7f9f22d70700 1 --2- 172.21.15.46:0/4175345331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 0x562eaae33e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.728+0000 7f9f22d70700 1 -- 172.21.15.46:0/4175345331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 msgr2=0x562eaae26230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.728+0000 7f9f22d70700 1 --2- 172.21.15.46:0/4175345331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae26230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.728+0000 7f9f22d70700 1 -- 172.21.15.46:0/4175345331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562eaad36e40 con 0x562eaae28db0 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.728+0000 7f9f2256f700 1 --2- 172.21.15.46:0/4175345331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae26230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.728+0000 7f9f22d70700 1 --2- 172.21.15.46:0/4175345331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 0x562eaae20e50 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f9f140041a0 tx=0x7f9f14009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7097934a9c316ec5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.728+0000 7f9f117fa700 1 -- 172.21.15.46:0/4175345331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400e040 con 0x562eaae28db0 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.728+0000 7f9f117fa700 1 -- 172.21.15.46:0/4175345331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f1400f040 con 0x562eaae28db0 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.728+0000 7f9f117fa700 1 -- 172.21.15.46:0/4175345331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14016970 con 0x562eaae28db0 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.729+0000 7f9f30968b80 1 -- 172.21.15.46:0/4175345331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 msgr2=0x562eaae20e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.729+0000 7f9f30968b80 1 --2- 172.21.15.46:0/4175345331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 0x562eaae20e50 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f9f140041a0 tx=0x7f9f14009920 comp rx=0 tx=0).stop 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.729+0000 7f9f30968b80 1 -- 172.21.15.46:0/4175345331 shutdown_connections 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.729+0000 7f9f30968b80 1 --2- 172.21.15.46:0/4175345331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae28db0 0x562eaae20e50 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.729+0000 7f9f30968b80 1 --2- 172.21.15.46:0/4175345331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae1d570 0x562eaae26230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.729+0000 7f9f30968b80 1 --2- 172.21.15.46:0/4175345331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 0x562eaae33e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.729+0000 7f9f30968b80 1 -- 172.21.15.46:0/4175345331 >> 172.21.15.46:0/4175345331 conn(0x562eaad941c0 msgr2=0x562eaae18120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.729+0000 7f9f30968b80 1 -- 172.21.15.46:0/4175345331 shutdown_connections 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.729+0000 7f9f30968b80 1 -- 172.21.15.46:0/4175345331 wait complete. 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.729+0000 7f9f30968b80 1 -- start start 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.729+0000 7f9f30968b80 1 Processor -- start 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f30968b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1d570 0x562eaae31a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f30968b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 0x562eaae31fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f30968b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae28db0 0x562eaae8fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f30968b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eaae326f0 con 0x562eaae1d570 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f30968b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eaae901a0 con 0x562eaae28db0 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f30968b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562eaae90330 con 0x562eaae1de90 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f30968b80 10 client.0 _ll_register_callbacks cb 0x7f9f0c00bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f2256f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1d570 0x562eaae31a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f2256f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1d570 0x562eaae31a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47828/0 (socket says 172.21.15.46:47828) 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f2256f700 1 -- 172.21.15.46:0/3901001547 learned_addr learned my addr 172.21.15.46:0/3901001547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f21d6e700 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 0x562eaae31fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f22d70700 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae28db0 0x562eaae8fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f2256f700 1 -- 172.21.15.46:0/3901001547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 msgr2=0x562eaae31fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f2256f700 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 0x562eaae31fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f2256f700 1 -- 172.21.15.46:0/3901001547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae28db0 msgr2=0x562eaae8fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f2256f700 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae28db0 0x562eaae8fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f2256f700 1 -- 172.21.15.46:0/3901001547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f14003d50 con 0x562eaae1d570 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f22d70700 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae28db0 0x562eaae8fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f2256f700 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1d570 0x562eaae31a80 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f9f18099ac0 tx=0x7f9f180cab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c78dd73b442cecd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.730+0000 7f9f05ffb700 1 -- 172.21.15.46:0/3901001547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1809dea0 con 0x562eaae1d570 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f05ffb700 1 -- 172.21.15.46:0/3901001547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f9f180b81b0 con 0x562eaae1d570 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f30968b80 1 -- 172.21.15.46:0/3901001547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f180a2470 con 0x562eaae1d570 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f05ffb700 1 -- 172.21.15.46:0/3901001547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f180b8370 con 0x562eaae1d570 2024-03-18T20:56:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f30968b80 10 client.20559 fetch_fsmap learned FSMap version 87 2024-03-18T20:56:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f30968b80 1 -- 172.21.15.46:0/3901001547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x562eaabf6c70 con 0x562eaae1d570 2024-03-18T20:56:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f05ffb700 1 -- 172.21.15.46:0/3901001547 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f18083080 con 0x562eaae1d570 2024-03-18T20:56:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f05ffb700 1 -- 172.21.15.46:0/3901001547 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 87) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f180a1fe0 con 0x562eaae1d570 2024-03-18T20:56:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f30968b80 10 client.20559 fetch_fsmap finished waiting for FSMap version 87 2024-03-18T20:56:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f30968b80 10 client.20559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:56:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f30968b80 20 client.20559 populate_metadata read hostname 'smithi046' 2024-03-18T20:56:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f30968b80 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562eaae91750 0x562eaae93b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f30968b80 4 client.20559 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:56:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f30968b80 1 -- 172.21.15.46:0/3901001547 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562eaabbfc90 con 0x562eaae91750 2024-03-18T20:56:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.731+0000 7f9f21d6e700 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562eaae91750 0x562eaae93b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.732+0000 7f9f21d6e700 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562eaae91750 0x562eaae93b70 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b31f38d94120804c server_cookie=9506fd25a791c1bf in_seq=0 out_seq=0 2024-03-18T20:56:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.732+0000 7f9f05ffb700 10 client.20559 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:02.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:02 smithi155 ceph-mon[48344]: pgmap v627: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 13 MiB/s wr, 701 op/s 2024-03-18T20:56:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:02 smithi046 ceph-mon[43495]: pgmap v627: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 13 MiB/s wr, 701 op/s 2024-03-18T20:56:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.920+0000 7f9f05ffb700 1 -- 172.21.15.46:0/3901001547 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562eaabbfc90 con 0x562eaae91750 2024-03-18T20:56:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.920+0000 7f9f05ffb700 10 client.20559 handle_command_reply: tid=0 2024-03-18T20:56:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f30968b80 1 client.20559 shutdown 2024-03-18T20:56:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f30968b80 1 -- 172.21.15.46:0/3901001547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1d570 msgr2=0x562eaae31a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f30968b80 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1d570 0x562eaae31a80 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f9f18099ac0 tx=0x7f9f180cab90 comp rx=0 tx=0).stop 2024-03-18T20:56:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f30968b80 1 -- 172.21.15.46:0/3901001547 shutdown_connections 2024-03-18T20:56:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f30968b80 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562eaae91750 0x562eaae93b70 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f30968b80 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562eaae1d570 0x562eaae31a80 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f30968b80 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562eaae28db0 0x562eaae8fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f30968b80 1 --2- 172.21.15.46:0/3901001547 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562eaae1de90 0x562eaae31fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f05ffb700 0 client.20559 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f30968b80 1 -- 172.21.15.46:0/3901001547 >> 172.21.15.46:0/3901001547 conn(0x562eaad941c0 msgr2=0x562eaae243e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f30968b80 1 -- 172.21.15.46:0/3901001547 shutdown_connections 2024-03-18T20:56:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f30968b80 1 -- 172.21.15.46:0/3901001547 wait complete. 2024-03-18T20:56:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.921+0000 7f9f30968b80 20 client.20559 trim_cache size 0 max 16384 2024-03-18T20:56:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:56:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:56:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:56:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:56:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.922+0000 7f9f247d3700 1 -- 172.21.15.46:0/1422079791 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9f080a1310 msgr2=0x7f9f080a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.922+0000 7f9f247d3700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9f080a1310 0x7f9f080a37d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00b2e0 tx=0x7f9f0c005fd0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.922+0000 7f9f247d3700 1 -- 172.21.15.46:0/1422079791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f1c0ffbb0 msgr2=0x7f9f1c1970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.922+0000 7f9f247d3700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f1c0ffbb0 0x7f9f1c1970f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800e9a0 tx=0x7f9f1800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.924+0000 7f9f247d3700 1 -- 172.21.15.46:0/1422079791 shutdown_connections 2024-03-18T20:56:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.924+0000 7f9f247d3700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f9f080a1310 0x7f9f080a37d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.924+0000 7f9f247d3700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f1c0ffbb0 0x7f9f1c1970f0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.924+0000 7f9f247d3700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f1c105c60 0x7f9f1c19b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.924+0000 7f9f247d3700 1 --2- 172.21.15.46:0/1422079791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f1c100570 0x7f9f1c197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.924+0000 7f9f247d3700 1 -- 172.21.15.46:0/1422079791 >> 172.21.15.46:0/1422079791 conn(0x7f9f1c076d50 msgr2=0x7f9f1c0682f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.924+0000 7f9f247d3700 1 -- 172.21.15.46:0/1422079791 shutdown_connections 2024-03-18T20:56:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:02.924+0000 7f9f247d3700 1 -- 172.21.15.46:0/1422079791 wait complete. 2024-03-18T20:56:03.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 292230 35.25 MB/sec execute 279 sec latency 1762.051 ms 2024-03-18T20:56:04.422 INFO:tasks.workunit.client.0.smithi046.stdout: 10 292327 35.14 MB/sec execute 280 sec latency 723.160 ms 2024-03-18T20:56:04.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:04 smithi155 ceph-mon[48344]: pgmap v628: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 763 op/s 2024-03-18T20:56:04.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:04 smithi155 ceph-mon[48344]: osdmap e109: 12 total, 12 up, 12 in 2024-03-18T20:56:04.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:04 smithi046 ceph-mon[43495]: pgmap v628: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 763 op/s 2024-03-18T20:56:04.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:04 smithi046 ceph-mon[43495]: osdmap e109: 12 total, 12 up, 12 in 2024-03-18T20:56:04.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:04 smithi073 ceph-mon[47384]: pgmap v628: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 763 op/s 2024-03-18T20:56:04.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:04 smithi073 ceph-mon[47384]: osdmap e109: 12 total, 12 up, 12 in 2024-03-18T20:56:05.423 INFO:tasks.workunit.client.0.smithi046.stdout: 10 292408 35.03 MB/sec execute 281 sec latency 840.519 ms 2024-03-18T20:56:06.423 INFO:tasks.workunit.client.0.smithi046.stdout: 10 292654 34.96 MB/sec execute 282 sec latency 293.687 ms 2024-03-18T20:56:06.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:06 smithi155 ceph-mon[48344]: pgmap v630: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 595 op/s 2024-03-18T20:56:06.547 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:06 smithi046 ceph-mon[43495]: pgmap v630: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 595 op/s 2024-03-18T20:56:06.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:06 smithi073 ceph-mon[47384]: pgmap v630: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 595 op/s 2024-03-18T20:56:07.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 292964 34.92 MB/sec execute 283 sec latency 151.297 ms 2024-03-18T20:56:08.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 293580 34.94 MB/sec execute 284 sec latency 116.572 ms 2024-03-18T20:56:08.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:08 smithi073 ceph-mon[47384]: pgmap v631: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 595 op/s 2024-03-18T20:56:08.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:08 smithi155 ceph-mon[48344]: pgmap v631: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 595 op/s 2024-03-18T20:56:08.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:08 smithi046 ceph-mon[43495]: pgmap v631: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.2 MiB/s wr, 595 op/s 2024-03-18T20:56:09.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 294052 34.88 MB/sec execute 285 sec latency 119.771 ms 2024-03-18T20:56:10.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 294752 34.88 MB/sec execute 286 sec latency 90.174 ms 2024-03-18T20:56:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:10 smithi073 ceph-mon[47384]: pgmap v632: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.4 MiB/s wr, 581 op/s 2024-03-18T20:56:10.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:10 smithi155 ceph-mon[48344]: pgmap v632: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.4 MiB/s wr, 581 op/s 2024-03-18T20:56:10.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:10 smithi046 ceph-mon[43495]: pgmap v632: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 9.4 MiB/s wr, 581 op/s 2024-03-18T20:56:11.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 295461 34.89 MB/sec execute 287 sec latency 83.340 ms 2024-03-18T20:56:12.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 296250 34.91 MB/sec execute 288 sec latency 98.570 ms 2024-03-18T20:56:12.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:12 smithi073 ceph-mon[47384]: pgmap v633: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 645 op/s 2024-03-18T20:56:12.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:12 smithi155 ceph-mon[48344]: pgmap v633: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 645 op/s 2024-03-18T20:56:12.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:12 smithi046 ceph-mon[43495]: pgmap v633: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 645 op/s 2024-03-18T20:56:13.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 296991 34.98 MB/sec execute 289 sec latency 61.663 ms 2024-03-18T20:56:14.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 297466 34.92 MB/sec execute 290 sec latency 152.507 ms 2024-03-18T20:56:14.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:14 smithi073 ceph-mon[47384]: pgmap v634: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 604 op/s 2024-03-18T20:56:14.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:14 smithi155 ceph-mon[48344]: pgmap v634: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 604 op/s 2024-03-18T20:56:14.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:14 smithi046 ceph-mon[43495]: pgmap v634: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 604 op/s 2024-03-18T20:56:15.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 298006 34.88 MB/sec execute 291 sec latency 137.846 ms 2024-03-18T20:56:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.128+0000 7f17511ad700 1 -- 172.21.15.46:0/1353439731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174c0ffbb0 msgr2=0x7f174c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:16.130 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.128+0000 7f17511ad700 1 --2- 172.21.15.46:0/1353439731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174c0ffbb0 0x7f174c100030 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f17400098f0 tx=0x7f1740009c00 comp rx=0 tx=0).stop 2024-03-18T20:56:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.128+0000 7f17511ad700 1 -- 172.21.15.46:0/1353439731 shutdown_connections 2024-03-18T20:56:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.128+0000 7f17511ad700 1 --2- 172.21.15.46:0/1353439731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c105c60 0x7f174c106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.128+0000 7f17511ad700 1 --2- 172.21.15.46:0/1353439731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f174c100570 0x7f174c10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.128+0000 7f17511ad700 1 --2- 172.21.15.46:0/1353439731 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174c0ffbb0 0x7f174c100030 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.128+0000 7f17511ad700 1 -- 172.21.15.46:0/1353439731 >> 172.21.15.46:0/1353439731 conn(0x7f174c076d50 msgr2=0x7f174c077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.128+0000 7f17511ad700 1 -- 172.21.15.46:0/1353439731 shutdown_connections 2024-03-18T20:56:16.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.128+0000 7f17511ad700 1 -- 172.21.15.46:0/1353439731 wait complete. 2024-03-18T20:56:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.129+0000 7f17511ad700 1 Processor -- start 2024-03-18T20:56:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.129+0000 7f17511ad700 1 -- start start 2024-03-18T20:56:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.129+0000 7f17511ad700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c0ffbb0 0x7f174c072430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f17511ad700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f174c100570 0x7f174c072970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f17511ad700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174c105c60 0x7f174c06c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f17511ad700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c06cdf0 con 0x7f174c0ffbb0 2024-03-18T20:56:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f17511ad700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c06cf60 con 0x7f174c100570 2024-03-18T20:56:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f17511ad700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c06d120 con 0x7f174c105c60 2024-03-18T20:56:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f174b59e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174c105c60 0x7f174c06c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f174b59e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174c105c60 0x7f174c06c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40212/0 (socket says 172.21.15.46:40212) 2024-03-18T20:56:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f174b59e700 1 -- 172.21.15.46:0/3751649939 learned_addr learned my addr 172.21.15.46:0/3751649939 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f174a59c700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f174c100570 0x7f174c072970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f174ad9d700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c0ffbb0 0x7f174c072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f174b59e700 1 -- 172.21.15.46:0/3751649939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f174c100570 msgr2=0x7f174c072970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f174b59e700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f174c100570 0x7f174c072970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f174b59e700 1 -- 172.21.15.46:0/3751649939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c0ffbb0 msgr2=0x7f174c072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f174b59e700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c0ffbb0 0x7f174c072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.130+0000 7f174b59e700 1 -- 172.21.15.46:0/3751649939 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1740009520 con 0x7f174c105c60 2024-03-18T20:56:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.131+0000 7f174b59e700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174c105c60 0x7f174c06c620 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1744007ee0 tx=0x7f174400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.131+0000 7f1733fff700 1 -- 172.21.15.46:0/3751649939 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174400ddd0 con 0x7f174c105c60 2024-03-18T20:56:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.131+0000 7f1733fff700 1 -- 172.21.15.46:0/3751649939 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f174400f040 con 0x7f174c105c60 2024-03-18T20:56:16.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.131+0000 7f17511ad700 1 -- 172.21.15.46:0/3751649939 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174c1aa0e0 con 0x7f174c105c60 2024-03-18T20:56:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.131+0000 7f17511ad700 1 -- 172.21.15.46:0/3751649939 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c1aa5c0 con 0x7f174c105c60 2024-03-18T20:56:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.131+0000 7f1733fff700 1 -- 172.21.15.46:0/3751649939 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17440048e0 con 0x7f174c105c60 2024-03-18T20:56:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.134+0000 7f1733fff700 1 -- 172.21.15.46:0/3751649939 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1744015460 con 0x7f174c105c60 2024-03-18T20:56:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.135+0000 7f1733fff700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f17340a13e0 0x7f17340a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.135+0000 7f1733fff700 1 -- 172.21.15.46:0/3751649939 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f17440cc4b0 con 0x7f174c105c60 2024-03-18T20:56:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.135+0000 7f175d342b80 1 Processor -- start 2024-03-18T20:56:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.135+0000 7f175d342b80 1 -- start start 2024-03-18T20:56:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.135+0000 7f175d342b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7f570 0x5574a6c7f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.135+0000 7f175d342b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7fe90 0x5574a6c8a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.135+0000 7f174ad9d700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f17340a13e0 0x7f17340a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.135+0000 7f175d342b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c8adb0 0x5574a6c8d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.135+0000 7f175d342b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a6bd9930 con 0x5574a6c7fe90 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.135+0000 7f175d342b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a6bd72f0 con 0x5574a6c7f570 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.135+0000 7f175d342b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a6bd2be0 con 0x5574a6c8adb0 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f174a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7fe90 0x5574a6c8a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f174a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7fe90 0x5574a6c8a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50480/0 (socket says 172.21.15.46:50480) 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f174a59c700 1 -- 172.21.15.46:0/3902179438 learned_addr learned my addr 172.21.15.46:0/3902179438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f174b59e700 1 --2- 172.21.15.46:0/3902179438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c8adb0 0x5574a6c8d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f174ad9d700 1 --2- 172.21.15.46:0/3902179438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7f570 0x5574a6c7f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f174b59e700 1 -- 172.21.15.46:0/3902179438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7f570 msgr2=0x5574a6c7f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f174b59e700 1 --2- 172.21.15.46:0/3902179438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7f570 0x5574a6c7f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f174b59e700 1 -- 172.21.15.46:0/3902179438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7fe90 msgr2=0x5574a6c8a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f174b59e700 1 --2- 172.21.15.46:0/3902179438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7fe90 0x5574a6c8a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f174b59e700 1 -- 172.21.15.46:0/3902179438 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574a6a54350 con 0x5574a6c8adb0 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f174b59e700 1 --2- 172.21.15.46:0/3902179438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c8adb0 0x5574a6c8d1a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f17440c9be0 tx=0x7f1744007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86c27795776a677d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f174ad9d700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f17340a13e0 0x7f17340a38a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f173c0092e0 tx=0x7f173c013400 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f17317fa700 1 -- 172.21.15.46:0/3902179438 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1744099ca0 con 0x5574a6c8adb0 2024-03-18T20:56:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f17317fa700 1 -- 172.21.15.46:0/3902179438 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17440995a0 con 0x5574a6c8adb0 2024-03-18T20:56:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.136+0000 7f17317fa700 1 -- 172.21.15.46:0/3902179438 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1744004020 con 0x5574a6c8adb0 2024-03-18T20:56:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.137+0000 7f175d342b80 1 -- 172.21.15.46:0/3902179438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c8adb0 msgr2=0x5574a6c8d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.137+0000 7f175d342b80 1 --2- 172.21.15.46:0/3902179438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c8adb0 0x5574a6c8d1a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f17440c9be0 tx=0x7f1744007f90 comp rx=0 tx=0).stop 2024-03-18T20:56:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.137+0000 7f175d342b80 1 -- 172.21.15.46:0/3902179438 shutdown_connections 2024-03-18T20:56:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.137+0000 7f175d342b80 1 --2- 172.21.15.46:0/3902179438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c8adb0 0x5574a6c8d1a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.137+0000 7f175d342b80 1 --2- 172.21.15.46:0/3902179438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7f570 0x5574a6c7f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.137+0000 7f175d342b80 1 --2- 172.21.15.46:0/3902179438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7fe90 0x5574a6c8a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.137+0000 7f175d342b80 1 -- 172.21.15.46:0/3902179438 >> 172.21.15.46:0/3902179438 conn(0x5574a6bf61c0 msgr2=0x5574a6c7ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.137+0000 7f175d342b80 1 -- 172.21.15.46:0/3902179438 shutdown_connections 2024-03-18T20:56:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.137+0000 7f175d342b80 1 -- 172.21.15.46:0/3902179438 wait complete. 2024-03-18T20:56:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f175d342b80 1 -- start start 2024-03-18T20:56:16.143 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f175d342b80 1 Processor -- start 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f175d342b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7f570 0x5574a6c8a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f175d342b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7fe90 0x5574a6c87b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f175d342b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c8adb0 0x5574a6c881e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f175d342b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a6b6ed20 con 0x5574a6c7f570 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f175d342b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a6b6e9c0 con 0x5574a6c7fe90 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f175d342b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a6a56640 con 0x5574a6c8adb0 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f175d342b80 10 client.0 _ll_register_callbacks cb 0x5574a69da340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f174a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7fe90 0x5574a6c87b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f174a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7fe90 0x5574a6c87b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:57756/0 (socket says 172.21.15.46:57756) 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f174a59c700 1 -- 172.21.15.46:0/2579482503 learned_addr learned my addr 172.21.15.46:0/2579482503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.138+0000 7f174ad9d700 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7f570 0x5574a6c8a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.139+0000 7f174b59e700 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c8adb0 0x5574a6c881e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.139+0000 7f174a59c700 1 -- 172.21.15.46:0/2579482503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c8adb0 msgr2=0x5574a6c881e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.139+0000 7f174a59c700 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c8adb0 0x5574a6c881e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.139+0000 7f174a59c700 1 -- 172.21.15.46:0/2579482503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7f570 msgr2=0x5574a6c8a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.139+0000 7f174a59c700 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7f570 0x5574a6c8a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.139+0000 7f174a59c700 1 -- 172.21.15.46:0/2579482503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c1aa5c0 con 0x5574a6c7fe90 2024-03-18T20:56:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.139+0000 7f174b59e700 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c8adb0 0x5574a6c881e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:56:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.139+0000 7f174a59c700 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7fe90 0x5574a6c87b60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f17400098c0 tx=0x7f1740005d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11c5bdb6d33dcfc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.144+0000 7f172eff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:56:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.144+0000 7f172eff5700 1 -- 172.21.15.46:0/2579482503 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17400054c0 con 0x5574a6c7fe90 2024-03-18T20:56:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.144+0000 7f172eff5700 1 -- 172.21.15.46:0/2579482503 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f1740016950 con 0x5574a6c7fe90 2024-03-18T20:56:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.144+0000 7f175d342b80 1 -- 172.21.15.46:0/2579482503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574a6c736c0 con 0x5574a6c7fe90 2024-03-18T20:56:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.144+0000 7f172eff5700 1 -- 172.21.15.46:0/2579482503 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1740005620 con 0x5574a6c7fe90 2024-03-18T20:56:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.144+0000 7f175d342b80 10 client.30173 fetch_fsmap learned FSMap version 87 2024-03-18T20:56:16.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.144+0000 7f175d342b80 1 -- 172.21.15.46:0/2579482503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x5574a6a00b00 con 0x5574a6c7fe90 2024-03-18T20:56:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.146+0000 7f172eff5700 1 -- 172.21.15.46:0/2579482503 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5574a6c736c0 con 0x5574a6c7fe90 2024-03-18T20:56:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.146+0000 7f172eff5700 1 -- 172.21.15.46:0/2579482503 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 87) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1740003ac0 con 0x5574a6c7fe90 2024-03-18T20:56:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.146+0000 7f175d342b80 10 client.30173 fetch_fsmap finished waiting for FSMap version 87 2024-03-18T20:56:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.146+0000 7f175d342b80 10 client.30173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:56:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.146+0000 7f175d342b80 20 client.30173 populate_metadata read hostname 'smithi046' 2024-03-18T20:56:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.146+0000 7f175d342b80 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5574a6c9ab00 0x5574a6c9cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.146+0000 7f175d342b80 4 client.30173 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:56:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.146+0000 7f175d342b80 1 -- 172.21.15.46:0/2579482503 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574a6bcb6a0 con 0x5574a6c9ab00 2024-03-18T20:56:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.146+0000 7f174ad9d700 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5574a6c9ab00 0x5574a6c9cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.147+0000 7f174ad9d700 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5574a6c9ab00 0x5574a6c9cf20 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e216f5dd1a76e2ee server_cookie=8dee3ad7eefbe791 in_seq=0 out_seq=0 2024-03-18T20:56:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.147+0000 7f172eff5700 10 client.30173 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.148+0000 7f172eff5700 1 -- 172.21.15.46:0/2579482503 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5574a6bcb6a0 con 0x5574a6c9ab00 2024-03-18T20:56:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.148+0000 7f172eff5700 10 client.30173 handle_command_reply: tid=0 2024-03-18T20:56:16.151 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f175d342b80 1 client.30173 shutdown 2024-03-18T20:56:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f175d342b80 1 -- 172.21.15.46:0/2579482503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7fe90 msgr2=0x5574a6c87b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f175d342b80 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7fe90 0x5574a6c87b60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f17400098c0 tx=0x7f1740005d00 comp rx=0 tx=0).stop 2024-03-18T20:56:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f175d342b80 1 -- 172.21.15.46:0/2579482503 shutdown_connections 2024-03-18T20:56:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f175d342b80 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5574a6c9ab00 0x5574a6c9cf20 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f175d342b80 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7f570 0x5574a6c8a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f175d342b80 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c8adb0 0x5574a6c881e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f175d342b80 1 --2- 172.21.15.46:0/2579482503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7fe90 0x5574a6c87b60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f175d342b80 1 -- 172.21.15.46:0/2579482503 >> 172.21.15.46:0/2579482503 conn(0x5574a6bf61c0 msgr2=0x5574a6c7b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f172eff5700 0 client.30173 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f175d342b80 1 -- 172.21.15.46:0/2579482503 shutdown_connections 2024-03-18T20:56:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f175d342b80 1 -- 172.21.15.46:0/2579482503 wait complete. 2024-03-18T20:56:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.149+0000 7f175d342b80 20 client.30173 trim_cache size 0 max 16384 2024-03-18T20:56:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.176+0000 7f175d342b80 1 Processor -- start 2024-03-18T20:56:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.176+0000 7f175d342b80 1 -- start start 2024-03-18T20:56:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f175d342b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 0x5574a6c82540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f175d342b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7fe90 0x5574a6c86430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f175d342b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c8adb0 0x5574a6c86970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f175d342b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a6c86eb0 con 0x5574a6c7fe90 2024-03-18T20:56:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f175d342b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a6c75c10 con 0x5574a6c8adb0 2024-03-18T20:56:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f175d342b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a6c839c0 con 0x5574a6c7f570 2024-03-18T20:56:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f174ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 0x5574a6c82540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f174a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7fe90 0x5574a6c86430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f174b59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c8adb0 0x5574a6c86970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f174ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 0x5574a6c82540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40236/0 (socket says 172.21.15.46:40236) 2024-03-18T20:56:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f174ad9d700 1 -- 172.21.15.46:0/4036698202 learned_addr learned my addr 172.21.15.46:0/4036698202 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f174ad9d700 1 -- 172.21.15.46:0/4036698202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c8adb0 msgr2=0x5574a6c86970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f174ad9d700 1 --2- 172.21.15.46:0/4036698202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c8adb0 0x5574a6c86970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f174ad9d700 1 -- 172.21.15.46:0/4036698202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7fe90 msgr2=0x5574a6c86430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f174ad9d700 1 --2- 172.21.15.46:0/4036698202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7fe90 0x5574a6c86430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f174ad9d700 1 -- 172.21.15.46:0/4036698202 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574a6b98e40 con 0x5574a6c7f570 2024-03-18T20:56:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.177+0000 7f174b59e700 1 --2- 172.21.15.46:0/4036698202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c8adb0 0x5574a6c86970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:56:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.178+0000 7f174ad9d700 1 --2- 172.21.15.46:0/4036698202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 0x5574a6c82540 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f173c009930 tx=0x7f173c021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea231b3aa51890b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.178+0000 7f1731ffb700 1 -- 172.21.15.46:0/4036698202 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c026070 con 0x5574a6c7f570 2024-03-18T20:56:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.178+0000 7f1731ffb700 1 -- 172.21.15.46:0/4036698202 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f173c009cb0 con 0x5574a6c7f570 2024-03-18T20:56:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.178+0000 7f1731ffb700 1 -- 172.21.15.46:0/4036698202 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c0285a0 con 0x5574a6c7f570 2024-03-18T20:56:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.179+0000 7f175d342b80 1 -- 172.21.15.46:0/4036698202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 msgr2=0x5574a6c82540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.179+0000 7f175d342b80 1 --2- 172.21.15.46:0/4036698202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 0x5574a6c82540 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f173c009930 tx=0x7f173c021000 comp rx=0 tx=0).stop 2024-03-18T20:56:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.179+0000 7f175d342b80 1 -- 172.21.15.46:0/4036698202 shutdown_connections 2024-03-18T20:56:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.179+0000 7f175d342b80 1 --2- 172.21.15.46:0/4036698202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c8adb0 0x5574a6c86970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.179+0000 7f175d342b80 1 --2- 172.21.15.46:0/4036698202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 0x5574a6c82540 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.179+0000 7f175d342b80 1 --2- 172.21.15.46:0/4036698202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c7fe90 0x5574a6c86430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.179+0000 7f175d342b80 1 -- 172.21.15.46:0/4036698202 >> 172.21.15.46:0/4036698202 conn(0x5574a6bf61c0 msgr2=0x5574a6c82e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.179+0000 7f175d342b80 1 -- 172.21.15.46:0/4036698202 shutdown_connections 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.179+0000 7f175d342b80 1 -- 172.21.15.46:0/4036698202 wait complete. 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.179+0000 7f175d342b80 1 -- start start 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.179+0000 7f175d342b80 1 Processor -- start 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f175d342b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 0x5574a6c8f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f175d342b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7fe90 0x5574a6c8ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f175d342b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c8adb0 0x5574a6c90470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f175d342b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a6ced8f0 con 0x5574a6c8adb0 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f175d342b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a6ceda30 con 0x5574a6c7fe90 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f175d342b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a6cedb70 con 0x5574a6c7f570 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f175d342b80 10 client.0 _ll_register_callbacks cb 0x7f1740016e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:56:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f174ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 0x5574a6c8f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f174ad9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 0x5574a6c8f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40250/0 (socket says 172.21.15.46:40250) 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f174ad9d700 1 -- 172.21.15.46:0/4279267886 learned_addr learned my addr 172.21.15.46:0/4279267886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f174b59e700 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c8adb0 0x5574a6c90470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f174a59c700 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7fe90 0x5574a6c8ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f174ad9d700 1 -- 172.21.15.46:0/4279267886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7fe90 msgr2=0x5574a6c8ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f174ad9d700 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7fe90 0x5574a6c8ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f174ad9d700 1 -- 172.21.15.46:0/4279267886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c8adb0 msgr2=0x5574a6c90470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.180+0000 7f174ad9d700 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c8adb0 0x5574a6c90470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.181+0000 7f174ad9d700 1 -- 172.21.15.46:0/4279267886 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c01b040 con 0x5574a6c7f570 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.181+0000 7f174ad9d700 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 0x5574a6c8f9f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f173c009770 tx=0x7f173c00fe30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e76601c6166894ba server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.181+0000 7f172eff5700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.181+0000 7f172eff5700 1 -- 172.21.15.46:0/4279267886 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c026040 con 0x5574a6c7f570 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.181+0000 7f172eff5700 1 -- 172.21.15.46:0/4279267886 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f173c0216e0 con 0x5574a6c7f570 2024-03-18T20:56:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.181+0000 7f175d342b80 1 -- 172.21.15.46:0/4279267886 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17440995a0 con 0x5574a6c7f570 2024-03-18T20:56:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.182+0000 7f172eff5700 1 -- 172.21.15.46:0/4279267886 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f173c0218a0 con 0x5574a6c7f570 2024-03-18T20:56:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.184+0000 7f175d342b80 10 client.30280 fetch_fsmap learned FSMap version 87 2024-03-18T20:56:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.184+0000 7f175d342b80 1 -- 172.21.15.46:0/4279267886 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) v3 -- 0x5574a6a58c70 con 0x5574a6c7f570 2024-03-18T20:56:16.186 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.184+0000 7f172eff5700 1 -- 172.21.15.46:0/4279267886 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17440995a0 con 0x5574a6c7f570 2024-03-18T20:56:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.185+0000 7f172eff5700 1 -- 172.21.15.46:0/4279267886 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 87) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f173c03aaa0 con 0x5574a6c7f570 2024-03-18T20:56:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.185+0000 7f175d342b80 10 client.30280 fetch_fsmap finished waiting for FSMap version 87 2024-03-18T20:56:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.185+0000 7f175d342b80 10 client.30280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:56:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.185+0000 7f175d342b80 20 client.30280 populate_metadata read hostname 'smithi046' 2024-03-18T20:56:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.185+0000 7f175d342b80 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5574a6cef280 0x5574a6cf16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.185+0000 7f175d342b80 4 client.30280 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:56:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.185+0000 7f175d342b80 1 -- 172.21.15.46:0/4279267886 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5574a6a21c90 con 0x5574a6cef280 2024-03-18T20:56:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.185+0000 7f174a59c700 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5574a6cef280 0x5574a6cf16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.186+0000 7f172eff5700 10 client.30280 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.186+0000 7f174a59c700 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5574a6cef280 0x5574a6cf16a0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc13d829855d8d33 server_cookie=6ca2347267bb7846 in_seq=0 out_seq=0 2024-03-18T20:56:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.403+0000 7f172eff5700 1 -- 172.21.15.46:0/4279267886 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5574a6a21c90 con 0x5574a6cef280 2024-03-18T20:56:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.403+0000 7f172eff5700 10 client.30280 handle_command_reply: tid=0 2024-03-18T20:56:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.403+0000 7f175d342b80 1 client.30280 shutdown 2024-03-18T20:56:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.404+0000 7f175d342b80 1 -- 172.21.15.46:0/4279267886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 msgr2=0x5574a6c8f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.404+0000 7f175d342b80 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 0x5574a6c8f9f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f173c009770 tx=0x7f173c00fe30 comp rx=0 tx=0).stop 2024-03-18T20:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.404+0000 7f175d342b80 1 -- 172.21.15.46:0/4279267886 shutdown_connections 2024-03-18T20:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.404+0000 7f175d342b80 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x5574a6cef280 0x5574a6cf16a0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.404+0000 7f175d342b80 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574a6c8adb0 0x5574a6c90470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.404+0000 7f175d342b80 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574a6c7f570 0x5574a6c8f9f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.404+0000 7f172eff5700 0 client.30280 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.404+0000 7f175d342b80 1 --2- 172.21.15.46:0/4279267886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5574a6c7fe90 0x5574a6c8ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.404+0000 7f175d342b80 1 -- 172.21.15.46:0/4279267886 >> 172.21.15.46:0/4279267886 conn(0x5574a6bf61c0 msgr2=0x5574a6c862c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.404+0000 7f175d342b80 1 -- 172.21.15.46:0/4279267886 shutdown_connections 2024-03-18T20:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.404+0000 7f175d342b80 1 -- 172.21.15.46:0/4279267886 wait complete. 2024-03-18T20:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.404+0000 7f175d342b80 20 client.30280 trim_cache size 0 max 16384 2024-03-18T20:56:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:56:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:56:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:56:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:56:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.409+0000 7f17511ad700 1 -- 172.21.15.46:0/3751649939 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f17340a13e0 msgr2=0x7f17340a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.409+0000 7f17511ad700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f17340a13e0 0x7f17340a38a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f173c0092e0 tx=0x7f173c013400 comp rx=0 tx=0).stop 2024-03-18T20:56:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.409+0000 7f17511ad700 1 -- 172.21.15.46:0/3751649939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174c105c60 msgr2=0x7f174c06c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.409+0000 7f17511ad700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174c105c60 0x7f174c06c620 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1744007ee0 tx=0x7f174400d3b0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.409+0000 7f17511ad700 1 -- 172.21.15.46:0/3751649939 shutdown_connections 2024-03-18T20:56:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.409+0000 7f17511ad700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f17340a13e0 0x7f17340a38a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.409+0000 7f17511ad700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f174c0ffbb0 0x7f174c072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.409+0000 7f17511ad700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174c105c60 0x7f174c06c620 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.409+0000 7f17511ad700 1 --2- 172.21.15.46:0/3751649939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f174c100570 0x7f174c072970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.409+0000 7f17511ad700 1 -- 172.21.15.46:0/3751649939 >> 172.21.15.46:0/3751649939 conn(0x7f174c076d50 msgr2=0x7f174c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.410+0000 7f17511ad700 1 -- 172.21.15.46:0/3751649939 shutdown_connections 2024-03-18T20:56:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:16.410+0000 7f17511ad700 1 -- 172.21.15.46:0/3751649939 wait complete. 2024-03-18T20:56:16.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 298647 34.90 MB/sec execute 292 sec latency 157.967 ms 2024-03-18T20:56:16.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:16 smithi073 ceph-mon[47384]: osdmap e110: 12 total, 12 up, 12 in 2024-03-18T20:56:16.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:16 smithi073 ceph-mon[47384]: pgmap v636: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 623 op/s 2024-03-18T20:56:16.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:16 smithi155 ceph-mon[48344]: osdmap e110: 12 total, 12 up, 12 in 2024-03-18T20:56:16.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:16 smithi155 ceph-mon[48344]: pgmap v636: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 623 op/s 2024-03-18T20:56:16.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:16 smithi046 ceph-mon[43495]: osdmap e110: 12 total, 12 up, 12 in 2024-03-18T20:56:16.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:16 smithi046 ceph-mon[43495]: pgmap v636: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 623 op/s 2024-03-18T20:56:17.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 299362 34.89 MB/sec execute 293 sec latency 118.889 ms 2024-03-18T20:56:18.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 300145 34.94 MB/sec execute 294 sec latency 83.357 ms 2024-03-18T20:56:18.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:18 smithi073 ceph-mon[47384]: pgmap v637: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 623 op/s 2024-03-18T20:56:18.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:18 smithi155 ceph-mon[48344]: pgmap v637: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 623 op/s 2024-03-18T20:56:18.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:18 smithi046 ceph-mon[43495]: pgmap v637: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 623 op/s 2024-03-18T20:56:19.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 300782 34.95 MB/sec execute 295 sec latency 78.182 ms 2024-03-18T20:56:20.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 301388 34.93 MB/sec execute 296 sec latency 99.923 ms 2024-03-18T20:56:20.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:20 smithi073 ceph-mon[47384]: pgmap v638: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 10 MiB/s wr, 663 op/s 2024-03-18T20:56:20.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:20 smithi155 ceph-mon[48344]: pgmap v638: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 10 MiB/s wr, 663 op/s 2024-03-18T20:56:20.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:20 smithi046 ceph-mon[43495]: pgmap v638: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 10 MiB/s wr, 663 op/s 2024-03-18T20:56:21.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 302033 34.91 MB/sec execute 297 sec latency 85.647 ms 2024-03-18T20:56:22.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 302875 34.95 MB/sec execute 298 sec latency 86.449 ms 2024-03-18T20:56:22.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:22 smithi073 ceph-mon[47384]: pgmap v639: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.7 MiB/s wr, 614 op/s 2024-03-18T20:56:22.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:22 smithi155 ceph-mon[48344]: pgmap v639: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.7 MiB/s wr, 614 op/s 2024-03-18T20:56:22.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:22 smithi046 ceph-mon[43495]: pgmap v639: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.7 MiB/s wr, 614 op/s 2024-03-18T20:56:23.425 INFO:tasks.workunit.client.0.smithi046.stdout: 10 303563 34.96 MB/sec execute 299 sec latency 79.631 ms 2024-03-18T20:56:24.426 INFO:tasks.workunit.client.0.smithi046.stdout: 10 304290 34.98 MB/sec execute 300 sec latency 84.251 ms 2024-03-18T20:56:24.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:24 smithi073 ceph-mon[47384]: pgmap v640: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.0 MiB/s wr, 650 op/s 2024-03-18T20:56:24.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:24 smithi155 ceph-mon[48344]: pgmap v640: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.0 MiB/s wr, 650 op/s 2024-03-18T20:56:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:24 smithi046 ceph-mon[43495]: pgmap v640: 97 pgs: 97 active+clean; 2.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.0 MiB/s wr, 650 op/s 2024-03-18T20:56:25.426 INFO:tasks.workunit.client.0.smithi046.stdout: 10 304928 34.99 MB/sec execute 301 sec latency 118.460 ms 2024-03-18T20:56:25.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:25 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:56:25.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:25 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:56:25.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:25 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:56:26.426 INFO:tasks.workunit.client.0.smithi046.stdout: 10 305572 34.95 MB/sec execute 302 sec latency 83.189 ms 2024-03-18T20:56:26.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:26 smithi073 ceph-mon[47384]: pgmap v641: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 603 op/s 2024-03-18T20:56:26.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:26 smithi155 ceph-mon[48344]: pgmap v641: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 603 op/s 2024-03-18T20:56:26.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:26 smithi046 ceph-mon[43495]: pgmap v641: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 603 op/s 2024-03-18T20:56:27.426 INFO:tasks.workunit.client.0.smithi046.stdout: 10 306304 34.99 MB/sec execute 303 sec latency 92.019 ms 2024-03-18T20:56:28.426 INFO:tasks.workunit.client.0.smithi046.stdout: 10 306990 35.00 MB/sec execute 304 sec latency 99.502 ms 2024-03-18T20:56:28.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:28 smithi073 ceph-mon[47384]: pgmap v642: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 510 op/s 2024-03-18T20:56:28.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:28 smithi155 ceph-mon[48344]: pgmap v642: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 510 op/s 2024-03-18T20:56:28.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:28 smithi046 ceph-mon[43495]: pgmap v642: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.1 MiB/s wr, 510 op/s 2024-03-18T20:56:29.426 INFO:tasks.workunit.client.0.smithi046.stdout: 10 307699 35.00 MB/sec execute 305 sec latency 90.744 ms 2024-03-18T20:56:30.426 INFO:tasks.workunit.client.0.smithi046.stdout: 10 308450 35.05 MB/sec execute 306 sec latency 85.159 ms 2024-03-18T20:56:30.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:30 smithi073 ceph-mon[47384]: pgmap v643: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 726 op/s 2024-03-18T20:56:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.726+0000 7f74d1186700 1 -- 172.21.15.46:0/3027192363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc105c60 msgr2=0x7f74cc106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.726+0000 7f74d1186700 1 --2- 172.21.15.46:0/3027192363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc105c60 0x7f74cc106040 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009990 tx=0x7f74bc009ca0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.726+0000 7f74d1186700 1 -- 172.21.15.46:0/3027192363 shutdown_connections 2024-03-18T20:56:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.726+0000 7f74d1186700 1 --2- 172.21.15.46:0/3027192363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc105c60 0x7f74cc106040 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.726+0000 7f74d1186700 1 --2- 172.21.15.46:0/3027192363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc100570 0x7f74cc10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.726+0000 7f74d1186700 1 --2- 172.21.15.46:0/3027192363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74cc0ffbb0 0x7f74cc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.726+0000 7f74d1186700 1 -- 172.21.15.46:0/3027192363 >> 172.21.15.46:0/3027192363 conn(0x7f74cc076d50 msgr2=0x7f74cc077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.726+0000 7f74d1186700 1 -- 172.21.15.46:0/3027192363 shutdown_connections 2024-03-18T20:56:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.726+0000 7f74d1186700 1 -- 172.21.15.46:0/3027192363 wait complete. 2024-03-18T20:56:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.727+0000 7f74d1186700 1 Processor -- start 2024-03-18T20:56:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.728+0000 7f74d1186700 1 -- start start 2024-03-18T20:56:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.728+0000 7f74d1186700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc0ffbb0 0x7f74cc197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.728+0000 7f74d1186700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74cc100570 0x7f74cc1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.728+0000 7f74d1186700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc105c60 0x7f74cc1a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.728+0000 7f74d1186700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc198cf0 con 0x7f74cc105c60 2024-03-18T20:56:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.728+0000 7f74d1186700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc1a3520 con 0x7f74cc0ffbb0 2024-03-18T20:56:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.728+0000 7f74d1186700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc1a3660 con 0x7f74cc100570 2024-03-18T20:56:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.729+0000 7f74ca59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74cc100570 0x7f74cc1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.729+0000 7f74ca59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74cc100570 0x7f74cc1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:41134/0 (socket says 172.21.15.46:41134) 2024-03-18T20:56:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.729+0000 7f74ca59c700 1 -- 172.21.15.46:0/1125588564 learned_addr learned my addr 172.21.15.46:0/1125588564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.729+0000 7f74cb59e700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc105c60 0x7f74cc1a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.729+0000 7f74ca59c700 1 -- 172.21.15.46:0/1125588564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc0ffbb0 msgr2=0x7f74cc197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.729+0000 7f74ca59c700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc0ffbb0 0x7f74cc197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.729+0000 7f74ca59c700 1 -- 172.21.15.46:0/1125588564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc105c60 msgr2=0x7f74cc1a2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.729+0000 7f74ca59c700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc105c60 0x7f74cc1a2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.729+0000 7f74ca59c700 1 -- 172.21.15.46:0/1125588564 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc0095f0 con 0x7f74cc100570 2024-03-18T20:56:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.729+0000 7f74ca59c700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74cc100570 0x7f74cc1984a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f74c000e9a0 tx=0x7f74c000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.730+0000 7f74b3fff700 1 -- 172.21.15.46:0/1125588564 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0009790 con 0x7f74cc100570 2024-03-18T20:56:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.730+0000 7f74b3fff700 1 -- 172.21.15.46:0/1125588564 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74c000cbd0 con 0x7f74cc100570 2024-03-18T20:56:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.730+0000 7f74d1186700 1 -- 172.21.15.46:0/1125588564 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc1a38c0 con 0x7f74cc100570 2024-03-18T20:56:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.730+0000 7f74d1186700 1 -- 172.21.15.46:0/1125588564 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc1a3e20 con 0x7f74cc100570 2024-03-18T20:56:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.731+0000 7f74b3fff700 1 -- 172.21.15.46:0/1125588564 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c0018680 con 0x7f74cc100570 2024-03-18T20:56:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.735+0000 7f74dd31bb80 1 Processor -- start 2024-03-18T20:56:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.735+0000 7f74dd31bb80 1 -- start start 2024-03-18T20:56:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.735+0000 7f74dd31bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cee570 0x557470cee950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.735+0000 7f74dd31bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.735+0000 7f74b3fff700 1 -- 172.21.15.46:0/1125588564 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f74c00187e0 con 0x7f74cc100570 2024-03-18T20:56:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.735+0000 7f74cad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.735+0000 7f74cad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:48210/0 (socket says 172.21.15.46:48210) 2024-03-18T20:56:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.735+0000 7f74cad9d700 1 -- 172.21.15.46:0/2615824143 learned_addr learned my addr 172.21.15.46:0/2615824143 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.736+0000 7f74b3fff700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f74b40a5720 0x7f74b40a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.736+0000 7f74b3fff700 1 -- 172.21.15.46:0/1125588564 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f74c0014070 con 0x7f74cc100570 2024-03-18T20:56:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.737+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/2615824143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470cfc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.737+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2615824143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557470c48930 con 0x557470cee570 2024-03-18T20:56:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.737+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2615824143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557470c462f0 con 0x557470ceee90 2024-03-18T20:56:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.737+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2615824143 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557470c41be0 con 0x557470cf9db0 2024-03-18T20:56:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.737+0000 7f74cad9d700 1 -- 172.21.15.46:0/2615824143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 msgr2=0x557470cfc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.737+0000 7f74cb59e700 1 --2- 172.21.15.46:0/2615824143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470cfc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.737+0000 7f74cad9d700 1 --2- 172.21.15.46:0/2615824143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470cfc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.737+0000 7f74cad9d700 1 -- 172.21.15.46:0/2615824143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cee570 msgr2=0x557470cee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.737+0000 7f74cad9d700 1 --2- 172.21.15.46:0/2615824143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cee570 0x557470cee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.737+0000 7f74cad9d700 1 -- 172.21.15.46:0/2615824143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557470ac3350 con 0x557470ceee90 2024-03-18T20:56:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.737+0000 7f74cb59e700 1 --2- 172.21.15.46:0/2615824143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470cfc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:56:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.737+0000 7f74cad9d700 1 --2- 172.21.15.46:0/2615824143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf9870 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f74bc005b40 tx=0x7f74bc005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de27ea1ca46439b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.738+0000 7f74b17fa700 1 -- 172.21.15.46:0/2615824143 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc01d070 con 0x557470ceee90 2024-03-18T20:56:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.738+0000 7f74b17fa700 1 -- 172.21.15.46:0/2615824143 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74bc004cf0 con 0x557470ceee90 2024-03-18T20:56:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.738+0000 7f74b17fa700 1 -- 172.21.15.46:0/2615824143 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc00b7b0 con 0x557470ceee90 2024-03-18T20:56:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.739+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2615824143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 msgr2=0x557470cf9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.739+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/2615824143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf9870 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f74bc005b40 tx=0x7f74bc005ea0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.739+0000 7f74ca59c700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f74b40a5720 0x7f74b40a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.740+0000 7f74ca59c700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f74b40a5720 0x7f74b40a7be0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f74c000e970 tx=0x7f74c000eef0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.746+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2615824143 shutdown_connections 2024-03-18T20:56:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.746+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/2615824143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cee570 0x557470cee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.746+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/2615824143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470cfc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.746+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/2615824143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf9870 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.746+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2615824143 >> 172.21.15.46:0/2615824143 conn(0x557470c651c0 msgr2=0x557470ceaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.746+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2615824143 shutdown_connections 2024-03-18T20:56:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.746+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2615824143 wait complete. 2024-03-18T20:56:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.746+0000 7f74dd31bb80 1 -- start start 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.746+0000 7f74dd31bb80 1 Processor -- start 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.747+0000 7f74dd31bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cee570 0x557470cf9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.747+0000 7f74dd31bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.747+0000 7f74dd31bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470cf7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.747+0000 7f74dd31bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557470bddd20 con 0x557470cee570 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.747+0000 7f74dd31bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557470bdd9c0 con 0x557470ceee90 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.747+0000 7f74dd31bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557470ac5640 con 0x557470cf9db0 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.747+0000 7f74dd31bb80 10 client.0 _ll_register_callbacks cb 0x557470a49340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.747+0000 7f74cad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.747+0000 7f74cad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:48224/0 (socket says 172.21.15.46:48224) 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.747+0000 7f74cad9d700 1 -- 172.21.15.46:0/4006884570 learned_addr learned my addr 172.21.15.46:0/4006884570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.747+0000 7f74cb59e700 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470cf7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.747+0000 7f74cb59e700 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cee570 0x557470cf9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.748+0000 7f74cb59e700 1 -- 172.21.15.46:0/4006884570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 msgr2=0x557470cf7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.748+0000 7f74cb59e700 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470cf7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.748+0000 7f74cb59e700 1 -- 172.21.15.46:0/4006884570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 msgr2=0x557470cf6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.748+0000 7f74cb59e700 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.748+0000 7f74cb59e700 1 -- 172.21.15.46:0/4006884570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc10d2f0 con 0x557470cee570 2024-03-18T20:56:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.748+0000 7f74cb59e700 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cee570 0x557470cf9790 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009ec0 tx=0x7f74c4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f3809747337c2c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.748+0000 7f74aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:56:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.749+0000 7f74aeff5700 1 -- 172.21.15.46:0/4006884570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c400e040 con 0x557470cee570 2024-03-18T20:56:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.749+0000 7f74aeff5700 1 -- 172.21.15.46:0/4006884570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f74c4017730 con 0x557470cee570 2024-03-18T20:56:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.749+0000 7f74dd31bb80 1 -- 172.21.15.46:0/4006884570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557470ce26c0 con 0x557470cee570 2024-03-18T20:56:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.749+0000 7f74aeff5700 1 -- 172.21.15.46:0/4006884570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74c40178f0 con 0x557470cee570 2024-03-18T20:56:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.749+0000 7f74dd31bb80 10 client.20586 fetch_fsmap learned FSMap version 88 2024-03-18T20:56:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.749+0000 7f74dd31bb80 1 -- 172.21.15.46:0/4006884570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x557470a6fb00 con 0x557470cee570 2024-03-18T20:56:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.749+0000 7f74aeff5700 1 -- 172.21.15.46:0/4006884570 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74c4029020 con 0x557470cee570 2024-03-18T20:56:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.749+0000 7f74aeff5700 1 -- 172.21.15.46:0/4006884570 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 88) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f74c4019950 con 0x557470cee570 2024-03-18T20:56:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.749+0000 7f74dd31bb80 10 client.20586 fetch_fsmap finished waiting for FSMap version 88 2024-03-18T20:56:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.749+0000 7f74dd31bb80 10 client.20586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:56:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.749+0000 7f74dd31bb80 20 client.20586 populate_metadata read hostname 'smithi046' 2024-03-18T20:56:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.749+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557470d008d0 0x557470d02cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.750+0000 7f74dd31bb80 4 client.20586 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:56:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.750+0000 7f74dd31bb80 1 -- 172.21.15.46:0/4006884570 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557470c3a6a0 con 0x557470d008d0 2024-03-18T20:56:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.750+0000 7f74cad9d700 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557470d008d0 0x557470d02cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.750+0000 7f74cad9d700 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557470d008d0 0x557470d02cf0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ff49d48c967bcb server_cookie=ab5a7ab954f72314 in_seq=0 out_seq=0 2024-03-18T20:56:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.750+0000 7f74aeff5700 10 client.20586 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74aeff5700 1 -- 172.21.15.46:0/4006884570 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557470c3a6a0 con 0x557470d008d0 2024-03-18T20:56:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74aeff5700 10 client.20586 handle_command_reply: tid=0 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74dd31bb80 1 client.20586 shutdown 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74dd31bb80 1 -- 172.21.15.46:0/4006884570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cee570 msgr2=0x557470cf9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cee570 0x557470cf9790 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009ec0 tx=0x7f74c4009920 comp rx=0 tx=0).stop 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74dd31bb80 1 -- 172.21.15.46:0/4006884570 shutdown_connections 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557470d008d0 0x557470d02cf0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cee570 0x557470cf9790 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470cf7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74aeff5700 0 client.20586 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/4006884570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74dd31bb80 1 -- 172.21.15.46:0/4006884570 >> 172.21.15.46:0/4006884570 conn(0x557470c651c0 msgr2=0x557470cea650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74dd31bb80 1 -- 172.21.15.46:0/4006884570 shutdown_connections 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74dd31bb80 1 -- 172.21.15.46:0/4006884570 wait complete. 2024-03-18T20:56:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.755+0000 7f74dd31bb80 20 client.20586 trim_cache size 0 max 16384 2024-03-18T20:56:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74dd31bb80 1 Processor -- start 2024-03-18T20:56:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74dd31bb80 1 -- start start 2024-03-18T20:56:30.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74dd31bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470cee570 0x557470cf6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74dd31bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470ceee90 0x557470d104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74dd31bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470ce9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74dd31bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557470cef5f0 con 0x557470ceee90 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74dd31bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557470d10a10 con 0x557470cee570 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74dd31bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557470cf28b0 con 0x557470cf9db0 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74cad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470ceee90 0x557470d104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74cad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470ceee90 0x557470d104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58978/0 (socket says 172.21.15.46:58978) 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74cad9d700 1 -- 172.21.15.46:0/1053847291 learned_addr learned my addr 172.21.15.46:0/1053847291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74cb59e700 1 --2- 172.21.15.46:0/1053847291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470cee570 0x557470cf6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.781+0000 7f74cb59e700 1 --2- 172.21.15.46:0/1053847291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470ce9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.782+0000 7f74cad9d700 1 -- 172.21.15.46:0/1053847291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 msgr2=0x557470ce9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.782+0000 7f74cad9d700 1 --2- 172.21.15.46:0/1053847291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470ce9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.782+0000 7f74cad9d700 1 -- 172.21.15.46:0/1053847291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470cee570 msgr2=0x557470cf6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.782+0000 7f74cad9d700 1 --2- 172.21.15.46:0/1053847291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470cee570 0x557470cf6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.782+0000 7f74cad9d700 1 -- 172.21.15.46:0/1053847291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557470c07e40 con 0x557470ceee90 2024-03-18T20:56:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.782+0000 7f74cb59e700 1 --2- 172.21.15.46:0/1053847291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470cee570 0x557470cf6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:56:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.782+0000 7f74cad9d700 1 --2- 172.21.15.46:0/1053847291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470ceee90 0x557470d104d0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f74bc005460 tx=0x7f74bc005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554e9f67e450be0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.783+0000 7f74b1ffb700 1 -- 172.21.15.46:0/1053847291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc01d040 con 0x557470ceee90 2024-03-18T20:56:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.783+0000 7f74b1ffb700 1 -- 172.21.15.46:0/1053847291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74bc0049e0 con 0x557470ceee90 2024-03-18T20:56:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.783+0000 7f74b1ffb700 1 -- 172.21.15.46:0/1053847291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc02e350 con 0x557470ceee90 2024-03-18T20:56:30.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.784+0000 7f74dd31bb80 1 -- 172.21.15.46:0/1053847291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470ceee90 msgr2=0x557470d104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.784+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/1053847291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470ceee90 0x557470d104d0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f74bc005460 tx=0x7f74bc005c60 comp rx=0 tx=0).stop 2024-03-18T20:56:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.784+0000 7f74dd31bb80 1 -- 172.21.15.46:0/1053847291 shutdown_connections 2024-03-18T20:56:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.784+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/1053847291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cf9db0 0x557470ce9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.784+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/1053847291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470cee570 0x557470cf6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.784+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/1053847291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470ceee90 0x557470d104d0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.784+0000 7f74dd31bb80 1 -- 172.21.15.46:0/1053847291 >> 172.21.15.46:0/1053847291 conn(0x557470c651c0 msgr2=0x557470d07650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.786+0000 7f74dd31bb80 1 -- 172.21.15.46:0/1053847291 shutdown_connections 2024-03-18T20:56:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.786+0000 7f74dd31bb80 1 -- 172.21.15.46:0/1053847291 wait complete. 2024-03-18T20:56:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.786+0000 7f74dd31bb80 1 -- start start 2024-03-18T20:56:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.786+0000 7f74dd31bb80 1 Processor -- start 2024-03-18T20:56:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.787+0000 7f74dd31bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cee570 0x557470cfbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.787+0000 7f74dd31bb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.787+0000 7f74dd31bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cf9db0 0x557470cf4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.787+0000 7f74dd31bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557470cf4980 con 0x557470cf9db0 2024-03-18T20:56:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.787+0000 7f74dd31bb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557470cf4af0 con 0x557470ceee90 2024-03-18T20:56:30.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.787+0000 7f74dd31bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557470cfa5c0 con 0x557470cee570 2024-03-18T20:56:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.787+0000 7f74dd31bb80 10 client.0 _ll_register_callbacks cb 0x7f74c4008bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:56:30.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:30 smithi155 ceph-mon[48344]: pgmap v643: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 726 op/s 2024-03-18T20:56:30.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.787+0000 7f74cad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.787+0000 7f74cb59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cf9db0 0x557470cf4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.787+0000 7f74cad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:48242/0 (socket says 172.21.15.46:48242) 2024-03-18T20:56:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.787+0000 7f74cad9d700 1 -- 172.21.15.46:0/2752952558 learned_addr learned my addr 172.21.15.46:0/2752952558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:30.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.787+0000 7f74cb59e700 1 --2- 172.21.15.46:0/2752952558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cee570 0x557470cfbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.788+0000 7f74cb59e700 1 -- 172.21.15.46:0/2752952558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cee570 msgr2=0x557470cfbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.788+0000 7f74cb59e700 1 --2- 172.21.15.46:0/2752952558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cee570 0x557470cfbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.788+0000 7f74cb59e700 1 -- 172.21.15.46:0/2752952558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 msgr2=0x557470cf3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.788+0000 7f74cb59e700 1 --2- 172.21.15.46:0/2752952558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.788+0000 7f74cb59e700 1 -- 172.21.15.46:0/2752952558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557470a6fb00 con 0x557470cf9db0 2024-03-18T20:56:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.788+0000 7f74cb59e700 1 --2- 172.21.15.46:0/2752952558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cf9db0 0x557470cf4270 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009010 tx=0x7f74c4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f731c968ee78c30c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.788+0000 7f74aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:56:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.788+0000 7f74aeff5700 1 -- 172.21.15.46:0/2752952558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c400e070 con 0x557470cf9db0 2024-03-18T20:56:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.788+0000 7f74aeff5700 1 -- 172.21.15.46:0/2752952558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f74c402a080 con 0x557470cf9db0 2024-03-18T20:56:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.788+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2752952558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74bc004cf0 con 0x557470cf9db0 2024-03-18T20:56:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.789+0000 7f74aeff5700 1 -- 172.21.15.46:0/2752952558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74c40093d0 con 0x557470cf9db0 2024-03-18T20:56:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.790+0000 7f74dd31bb80 10 client.20592 fetch_fsmap learned FSMap version 88 2024-03-18T20:56:30.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.790+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2752952558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x557470ac3350 con 0x557470cf9db0 2024-03-18T20:56:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.791+0000 7f74aeff5700 1 -- 172.21.15.46:0/2752952558 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74bc004cf0 con 0x557470cf9db0 2024-03-18T20:56:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.791+0000 7f74aeff5700 1 -- 172.21.15.46:0/2752952558 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 88) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f74c401b740 con 0x557470cf9db0 2024-03-18T20:56:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.792+0000 7f74dd31bb80 10 client.20592 fetch_fsmap finished waiting for FSMap version 88 2024-03-18T20:56:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.792+0000 7f74dd31bb80 10 client.20592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:56:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.792+0000 7f74dd31bb80 20 client.20592 populate_metadata read hostname 'smithi046' 2024-03-18T20:56:30.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.792+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/2752952558 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557470d803e0 0x557470d82800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.792+0000 7f74dd31bb80 4 client.20592 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:56:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.792+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2752952558 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557470a90c90 con 0x557470d803e0 2024-03-18T20:56:30.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.792+0000 7f74cad9d700 1 --2- 172.21.15.46:0/2752952558 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557470d803e0 0x557470d82800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:30.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:30 smithi046 ceph-mon[43495]: pgmap v643: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 726 op/s 2024-03-18T20:56:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.794+0000 7f74cad9d700 1 --2- 172.21.15.46:0/2752952558 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557470d803e0 0x557470d82800 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a87fb3a80df5457 server_cookie=59199b6d5bf15825 in_seq=0 out_seq=0 2024-03-18T20:56:30.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:30.794+0000 7f74aeff5700 10 client.20592 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:31.427 INFO:tasks.workunit.client.0.smithi046.stdout: 10 308683 34.97 MB/sec execute 307 sec latency 494.504 ms 2024-03-18T20:56:32.428 INFO:tasks.workunit.client.0.smithi046.stdout: 10 308719 34.86 MB/sec execute 308 sec latency 1305.266 ms 2024-03-18T20:56:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.439+0000 7f74aeff5700 1 -- 172.21.15.46:0/2752952558 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557470a90c90 con 0x557470d803e0 2024-03-18T20:56:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.439+0000 7f74aeff5700 10 client.20592 handle_command_reply: tid=0 2024-03-18T20:56:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.439+0000 7f74dd31bb80 1 client.20592 shutdown 2024-03-18T20:56:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.439+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2752952558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cf9db0 msgr2=0x557470cf4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.439+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/2752952558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cf9db0 0x557470cf4270 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009010 tx=0x7f74c4009360 comp rx=0 tx=0).stop 2024-03-18T20:56:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.439+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2752952558 shutdown_connections 2024-03-18T20:56:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.439+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/2752952558 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557470d803e0 0x557470d82800 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.439+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/2752952558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557470cf9db0 0x557470cf4270 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.439+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/2752952558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557470cee570 0x557470cfbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.439+0000 7f74dd31bb80 1 --2- 172.21.15.46:0/2752952558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557470ceee90 0x557470cf3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.439+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2752952558 >> 172.21.15.46:0/2752952558 conn(0x557470c651c0 msgr2=0x557470ce9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.439+0000 7f74aeff5700 0 client.20592 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.440+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2752952558 shutdown_connections 2024-03-18T20:56:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.440+0000 7f74dd31bb80 1 -- 172.21.15.46:0/2752952558 wait complete. 2024-03-18T20:56:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.441+0000 7f74dd31bb80 20 client.20592 trim_cache size 0 max 16384 2024-03-18T20:56:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:56:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:56:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:56:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:56:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.451+0000 7f74d1186700 1 -- 172.21.15.46:0/1125588564 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f74b40a5720 msgr2=0x7f74b40a7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.451+0000 7f74d1186700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f74b40a5720 0x7f74b40a7be0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f74c000e970 tx=0x7f74c000eef0 comp rx=0 tx=0).stop 2024-03-18T20:56:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.451+0000 7f74d1186700 1 -- 172.21.15.46:0/1125588564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74cc100570 msgr2=0x7f74cc1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.451+0000 7f74d1186700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74cc100570 0x7f74cc1984a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f74c000e9a0 tx=0x7f74c000ecb0 comp rx=0 tx=0).stop 2024-03-18T20:56:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.451+0000 7f74d1186700 1 -- 172.21.15.46:0/1125588564 shutdown_connections 2024-03-18T20:56:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.451+0000 7f74d1186700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f74b40a5720 0x7f74b40a7be0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.451+0000 7f74d1186700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74cc105c60 0x7f74cc1a2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.451+0000 7f74d1186700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc0ffbb0 0x7f74cc197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.451+0000 7f74d1186700 1 --2- 172.21.15.46:0/1125588564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74cc100570 0x7f74cc1984a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.451+0000 7f74d1186700 1 -- 172.21.15.46:0/1125588564 >> 172.21.15.46:0/1125588564 conn(0x7f74cc076d50 msgr2=0x7f74cc0fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.451+0000 7f74d1186700 1 -- 172.21.15.46:0/1125588564 shutdown_connections 2024-03-18T20:56:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:32.451+0000 7f74d1186700 1 -- 172.21.15.46:0/1125588564 wait complete. 2024-03-18T20:56:32.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:32 smithi155 ceph-mon[48344]: pgmap v644: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.0 MiB/s wr, 493 op/s 2024-03-18T20:56:32.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:32 smithi046 ceph-mon[43495]: pgmap v644: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.0 MiB/s wr, 493 op/s 2024-03-18T20:56:32.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:32 smithi073 ceph-mon[47384]: pgmap v644: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.0 MiB/s wr, 493 op/s 2024-03-18T20:56:33.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 309027 34.78 MB/sec execute 309 sec latency 1351.621 ms 2024-03-18T20:56:33.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:33 smithi073 ceph-mon[47384]: pgmap v645: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 609 op/s 2024-03-18T20:56:34.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:33 smithi155 ceph-mon[48344]: pgmap v645: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 609 op/s 2024-03-18T20:56:34.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:33 smithi046 ceph-mon[43495]: pgmap v645: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 609 op/s 2024-03-18T20:56:34.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 309365 34.73 MB/sec execute 310 sec latency 262.620 ms 2024-03-18T20:56:35.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 309989 34.73 MB/sec execute 311 sec latency 96.006 ms 2024-03-18T20:56:36.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 310626 34.73 MB/sec execute 312 sec latency 81.428 ms 2024-03-18T20:56:36.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:36 smithi155 ceph-mon[48344]: pgmap v646: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 21 MiB/s wr, 923 op/s 2024-03-18T20:56:36.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:36 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:56:36.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:36 smithi046 ceph-mon[43495]: pgmap v646: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 21 MiB/s wr, 923 op/s 2024-03-18T20:56:36.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:36 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:56:36.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:36 smithi073 ceph-mon[47384]: pgmap v646: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 21 MiB/s wr, 923 op/s 2024-03-18T20:56:36.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:36 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:56:37.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 311130 34.71 MB/sec execute 313 sec latency 110.642 ms 2024-03-18T20:56:38.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 311812 34.71 MB/sec execute 314 sec latency 85.553 ms 2024-03-18T20:56:38.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:38 smithi073 ceph-mon[47384]: pgmap v647: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 18 MiB/s wr, 769 op/s 2024-03-18T20:56:38.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:38 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:38 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:38 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:38 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:38 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:38 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:38 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:38 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:38 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:38 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:38 smithi155 ceph-mon[48344]: pgmap v647: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 18 MiB/s wr, 769 op/s 2024-03-18T20:56:38.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:38 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:38.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:38 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:38.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:38 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:38 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:38 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:38 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:38 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:38 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:38 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:38 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:38 smithi046 ceph-mon[43495]: pgmap v647: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 18 MiB/s wr, 769 op/s 2024-03-18T20:56:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:38 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:38 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:38 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:38 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:38 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:38 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:38 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:38 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:38 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:38 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:39.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 312417 34.73 MB/sec execute 315 sec latency 77.538 ms 2024-03-18T20:56:39.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:39 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi046 to 183.5M 2024-03-18T20:56:39.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:39 smithi073 ceph-mon[47384]: Unable to set osd_memory_target on smithi046 to 192483737: error parsing value: Value '192483737' is below minimum 939524096 2024-03-18T20:56:39.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:39 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:39.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:39 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:39.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:39 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:39.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:39 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:39.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:39 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi046 to 183.5M 2024-03-18T20:56:39.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:39 smithi155 ceph-mon[48344]: Unable to set osd_memory_target on smithi046 to 192483737: error parsing value: Value '192483737' is below minimum 939524096 2024-03-18T20:56:39.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:39 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:39.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:39 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:39.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:39 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:39.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:39 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:39.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:39 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi046 to 183.5M 2024-03-18T20:56:39.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:39 smithi046 ceph-mon[43495]: Unable to set osd_memory_target on smithi046 to 192483737: error parsing value: Value '192483737' is below minimum 939524096 2024-03-18T20:56:39.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:39 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:39.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:39 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:39.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:39 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:39.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:39 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 312952 34.67 MB/sec execute 316 sec latency 88.155 ms 2024-03-18T20:56:40.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:40 smithi073 ceph-mon[47384]: pgmap v648: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 21 MiB/s wr, 1.03k op/s 2024-03-18T20:56:40.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:40 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:40 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:40 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:40 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:40 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:40 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:40 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi073 to 182.4M 2024-03-18T20:56:40.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:40 smithi073 ceph-mon[47384]: Unable to set osd_memory_target on smithi073 to 191350476: error parsing value: Value '191350476' is below minimum 939524096 2024-03-18T20:56:40.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:40 smithi155 ceph-mon[48344]: pgmap v648: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 21 MiB/s wr, 1.03k op/s 2024-03-18T20:56:40.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:40 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:40 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:40 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:40 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:40 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:40 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:40 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi073 to 182.4M 2024-03-18T20:56:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:40 smithi155 ceph-mon[48344]: Unable to set osd_memory_target on smithi073 to 191350476: error parsing value: Value '191350476' is below minimum 939524096 2024-03-18T20:56:40.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:40 smithi046 ceph-mon[43495]: pgmap v648: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 21 MiB/s wr, 1.03k op/s 2024-03-18T20:56:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:40 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:40 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:40 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:40 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:40 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:40 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:40 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi073 to 182.4M 2024-03-18T20:56:40.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:40 smithi046 ceph-mon[43495]: Unable to set osd_memory_target on smithi073 to 191350476: error parsing value: Value '191350476' is below minimum 939524096 2024-03-18T20:56:41.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 313700 34.70 MB/sec execute 317 sec latency 78.439 ms 2024-03-18T20:56:41.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:41.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:41.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:41.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:41.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:41.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:41.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:41.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:41.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:41.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:41.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-03-18T20:56:41.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-03-18T20:56:41.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:56:41.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:56:41.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:41.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:41 smithi073 ceph-mon[47384]: pgmap v649: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 18 MiB/s wr, 826 op/s 2024-03-18T20:56:42.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:42.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:42.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:41 smithi155 ceph-mon[48344]: pgmap v649: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 18 MiB/s wr, 826 op/s 2024-03-18T20:56:42.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:42.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:42.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T20:56:42.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-03-18T20:56:42.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-03-18T20:56:42.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:56:42.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:56:42.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:56:42.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:41 smithi046 ceph-mon[43495]: pgmap v649: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 18 MiB/s wr, 826 op/s 2024-03-18T20:56:42.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 313946 34.64 MB/sec execute 318 sec latency 422.452 ms 2024-03-18T20:56:43.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 314496 34.62 MB/sec execute 319 sec latency 132.948 ms 2024-03-18T20:56:44.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 315185 34.62 MB/sec execute 320 sec latency 70.253 ms 2024-03-18T20:56:44.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:44 smithi073 ceph-mon[47384]: pgmap v650: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 916 op/s 2024-03-18T20:56:44.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:44 smithi155 ceph-mon[48344]: pgmap v650: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 916 op/s 2024-03-18T20:56:44.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:44 smithi046 ceph-mon[43495]: pgmap v650: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 20 MiB/s wr, 916 op/s 2024-03-18T20:56:45.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 315869 34.64 MB/sec execute 321 sec latency 69.962 ms 2024-03-18T20:56:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.747+0000 7fbc4df4a700 1 -- 172.21.15.46:0/1759073477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc48105be0 msgr2=0x7fbc48069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.747+0000 7fbc4df4a700 1 --2- 172.21.15.46:0/1759073477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc48105be0 0x7fbc48069430 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009990 tx=0x7fbc38009ca0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.747+0000 7fbc4df4a700 1 -- 172.21.15.46:0/1759073477 shutdown_connections 2024-03-18T20:56:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.747+0000 7fbc4df4a700 1 --2- 172.21.15.46:0/1759073477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc4806a050 0x7fbc48103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.747+0000 7fbc4df4a700 1 --2- 172.21.15.46:0/1759073477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc48105be0 0x7fbc48069430 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.747+0000 7fbc4df4a700 1 --2- 172.21.15.46:0/1759073477 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc48106050 0x7fbc48069970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.747+0000 7fbc4df4a700 1 -- 172.21.15.46:0/1759073477 >> 172.21.15.46:0/1759073477 conn(0x7fbc48076a80 msgr2=0x7fbc48076e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.747+0000 7fbc4df4a700 1 -- 172.21.15.46:0/1759073477 shutdown_connections 2024-03-18T20:56:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.747+0000 7fbc4df4a700 1 -- 172.21.15.46:0/1759073477 wait complete. 2024-03-18T20:56:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.748+0000 7fbc4df4a700 1 Processor -- start 2024-03-18T20:56:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.748+0000 7fbc4df4a700 1 -- start start 2024-03-18T20:56:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc4df4a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc48105be0 0x7fbc481a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc4df4a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc48106050 0x7fbc481a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc4df4a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc4819d6a0 0x7fbc4819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc4df4a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4819e1a0 con 0x7fbc48106050 2024-03-18T20:56:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc4df4a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4819e310 con 0x7fbc48105be0 2024-03-18T20:56:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc4df4a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4819e4d0 con 0x7fbc4819d6a0 2024-03-18T20:56:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc477fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc48105be0 0x7fbc481a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc477fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc48105be0 0x7fbc481a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:35184/0 (socket says 172.21.15.46:35184) 2024-03-18T20:56:45.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc477fe700 1 -- 172.21.15.46:0/137621748 learned_addr learned my addr 172.21.15.46:0/137621748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc46ffd700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc48106050 0x7fbc481a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc47fff700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc4819d6a0 0x7fbc4819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc477fe700 1 -- 172.21.15.46:0/137621748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc4819d6a0 msgr2=0x7fbc4819db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc477fe700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc4819d6a0 0x7fbc4819db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc477fe700 1 -- 172.21.15.46:0/137621748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc48106050 msgr2=0x7fbc481a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc477fe700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc48106050 0x7fbc481a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc477fe700 1 -- 172.21.15.46:0/137621748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc380095f0 con 0x7fbc48105be0 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.749+0000 7fbc47fff700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc4819d6a0 0x7fbc4819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.750+0000 7fbc477fe700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc48105be0 0x7fbc481a3600 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbc380095c0 tx=0x7fbc3800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.750+0000 7fbc44ff9700 1 -- 172.21.15.46:0/137621748 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3801d070 con 0x7fbc48105be0 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.750+0000 7fbc44ff9700 1 -- 172.21.15.46:0/137621748 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc380054f0 con 0x7fbc48105be0 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.750+0000 7fbc4df4a700 1 -- 172.21.15.46:0/137621748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc481aa400 con 0x7fbc48105be0 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.750+0000 7fbc44ff9700 1 -- 172.21.15.46:0/137621748 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38004bd0 con 0x7fbc48105be0 2024-03-18T20:56:45.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.750+0000 7fbc4df4a700 1 -- 172.21.15.46:0/137621748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc481aa880 con 0x7fbc48105be0 2024-03-18T20:56:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.753+0000 7fbc44ff9700 1 -- 172.21.15.46:0/137621748 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fbc3800fb80 con 0x7fbc48105be0 2024-03-18T20:56:45.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.753+0000 7fbc44ff9700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbc300a1290 0x7fbc300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.754+0000 7fbc44ff9700 1 -- 172.21.15.46:0/137621748 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fbc380cdad0 con 0x7fbc48105be0 2024-03-18T20:56:45.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.754+0000 7fbc46ffd700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbc300a1290 0x7fbc300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.754+0000 7fbc46ffd700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbc300a1290 0x7fbc300a3750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbc4819ee80 tx=0x7fbc3c00b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.770+0000 7fbc5a0dfb80 1 Processor -- start 2024-03-18T20:56:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.772+0000 7fbc5a0dfb80 1 -- start start 2024-03-18T20:56:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc5a0dfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce57b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc5a0dfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 0x55b9ce586870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc5a0dfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce586db0 0x55b9ce5891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc5a0dfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ce4d5930 con 0x55b9ce57b570 2024-03-18T20:56:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc5a0dfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ce4d32f0 con 0x55b9ce586db0 2024-03-18T20:56:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc5a0dfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ce4cebe0 con 0x55b9ce57be90 2024-03-18T20:56:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce57b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce57b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58232/0 (socket says 172.21.15.46:58232) 2024-03-18T20:56:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc477fe700 1 -- 172.21.15.46:0/3300040790 learned_addr learned my addr 172.21.15.46:0/3300040790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc47fff700 1 --2- 172.21.15.46:0/3300040790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce586db0 0x55b9ce5891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc477fe700 1 -- 172.21.15.46:0/3300040790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 msgr2=0x55b9ce586870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:56:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc477fe700 1 --2- 172.21.15.46:0/3300040790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 0x55b9ce586870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc477fe700 1 -- 172.21.15.46:0/3300040790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce586db0 msgr2=0x55b9ce5891a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc477fe700 1 --2- 172.21.15.46:0/3300040790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce586db0 0x55b9ce5891a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.773+0000 7fbc477fe700 1 -- 172.21.15.46:0/3300040790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ce350350 con 0x55b9ce57b570 2024-03-18T20:56:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.774+0000 7fbc477fe700 1 --2- 172.21.15.46:0/3300040790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce57b950 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fbc380970f0 tx=0x7fbc38009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bd986e1bcf0f17 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.774+0000 7fbc2dffb700 1 -- 172.21.15.46:0/3300040790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3809abf0 con 0x55b9ce57b570 2024-03-18T20:56:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.774+0000 7fbc2dffb700 1 -- 172.21.15.46:0/3300040790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc3809ea20 con 0x55b9ce57b570 2024-03-18T20:56:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.774+0000 7fbc2dffb700 1 -- 172.21.15.46:0/3300040790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3809a300 con 0x55b9ce57b570 2024-03-18T20:56:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.775+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3300040790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 msgr2=0x55b9ce57b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.775+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/3300040790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce57b950 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fbc380970f0 tx=0x7fbc38009310 comp rx=0 tx=0).stop 2024-03-18T20:56:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.775+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3300040790 shutdown_connections 2024-03-18T20:56:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.775+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/3300040790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce57b950 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.775+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/3300040790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce586db0 0x55b9ce5891a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.775+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/3300040790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 0x55b9ce586870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.775+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3300040790 >> 172.21.15.46:0/3300040790 conn(0x55b9ce4f21c0 msgr2=0x55b9ce577a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.777+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3300040790 shutdown_connections 2024-03-18T20:56:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.778+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3300040790 wait complete. 2024-03-18T20:56:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.778+0000 7fbc5a0dfb80 1 -- start start 2024-03-18T20:56:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.779+0000 7fbc5a0dfb80 1 Processor -- start 2024-03-18T20:56:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.779+0000 7fbc5a0dfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce57b570 0x55b9ce586770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.779+0000 7fbc5a0dfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 0x55b9ce583a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.779+0000 7fbc5a0dfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce586db0 0x55b9ce5840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.779+0000 7fbc5a0dfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ce46ad20 con 0x55b9ce586db0 2024-03-18T20:56:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.779+0000 7fbc5a0dfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ce46a9c0 con 0x55b9ce57b570 2024-03-18T20:56:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.779+0000 7fbc5a0dfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ce352640 con 0x55b9ce57be90 2024-03-18T20:56:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.779+0000 7fbc5a0dfb80 10 client.0 _ll_register_callbacks cb 0x55b9ce2d6340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:56:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.780+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce586db0 0x55b9ce5840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.780+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce586db0 0x55b9ce5840e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58244/0 (socket says 172.21.15.46:58244) 2024-03-18T20:56:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.780+0000 7fbc47fff700 1 -- 172.21.15.46:0/2524008101 learned_addr learned my addr 172.21.15.46:0/2524008101 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.781+0000 7fbc47fff700 1 -- 172.21.15.46:0/2524008101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 msgr2=0x55b9ce583a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:56:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.781+0000 7fbc47fff700 1 --2- 172.21.15.46:0/2524008101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 0x55b9ce583a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.781+0000 7fbc47fff700 1 -- 172.21.15.46:0/2524008101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce57b570 msgr2=0x55b9ce586770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:56:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.781+0000 7fbc47fff700 1 --2- 172.21.15.46:0/2524008101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce57b570 0x55b9ce586770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.781+0000 7fbc47fff700 1 -- 172.21.15.46:0/2524008101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc481aa880 con 0x55b9ce586db0 2024-03-18T20:56:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.781+0000 7fbc47fff700 1 --2- 172.21.15.46:0/2524008101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce586db0 0x55b9ce5840e0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000ac50 tx=0x7fbc4000ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de4c8ec76f3dca6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.782+0000 7fbc26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:56:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.782+0000 7fbc26ffd700 1 -- 172.21.15.46:0/2524008101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40007bb0 con 0x55b9ce586db0 2024-03-18T20:56:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.782+0000 7fbc26ffd700 1 -- 172.21.15.46:0/2524008101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fbc40006e40 con 0x55b9ce586db0 2024-03-18T20:56:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.782+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/2524008101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9ce56f6c0 con 0x55b9ce586db0 2024-03-18T20:56:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.782+0000 7fbc26ffd700 1 -- 172.21.15.46:0/2524008101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc40007d10 con 0x55b9ce586db0 2024-03-18T20:56:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.782+0000 7fbc5a0dfb80 10 client.20601 fetch_fsmap learned FSMap version 88 2024-03-18T20:56:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.782+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/2524008101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x55b9ce2fcb00 con 0x55b9ce586db0 2024-03-18T20:56:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.783+0000 7fbc26ffd700 1 -- 172.21.15.46:0/2524008101 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9ce56f6c0 con 0x55b9ce586db0 2024-03-18T20:56:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.783+0000 7fbc26ffd700 1 -- 172.21.15.46:0/2524008101 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 88) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fbc4001e070 con 0x55b9ce586db0 2024-03-18T20:56:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.783+0000 7fbc5a0dfb80 10 client.20601 fetch_fsmap finished waiting for FSMap version 88 2024-03-18T20:56:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.783+0000 7fbc5a0dfb80 10 client.20601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:56:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.783+0000 7fbc5a0dfb80 20 client.20601 populate_metadata read hostname 'smithi046' 2024-03-18T20:56:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.783+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/2524008101 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b9ce5a7710 0x55b9ce5a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.783+0000 7fbc5a0dfb80 4 client.20601 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:56:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.783+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/2524008101 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9ce4c76a0 con 0x55b9ce5a7710 2024-03-18T20:56:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.783+0000 7fbc477fe700 1 --2- 172.21.15.46:0/2524008101 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b9ce5a7710 0x55b9ce5a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.784+0000 7fbc26ffd700 10 client.20601 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.784+0000 7fbc477fe700 1 --2- 172.21.15.46:0/2524008101 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b9ce5a7710 0x55b9ce5a9b30 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b7568c96491c157 server_cookie=ab2b3b0019b2cac4 in_seq=0 out_seq=0 2024-03-18T20:56:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.786+0000 7fbc26ffd700 1 -- 172.21.15.46:0/2524008101 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b9ce4c76a0 con 0x55b9ce5a7710 2024-03-18T20:56:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.786+0000 7fbc26ffd700 10 client.20601 handle_command_reply: tid=0 2024-03-18T20:56:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.786+0000 7fbc5a0dfb80 1 client.20601 shutdown 2024-03-18T20:56:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.786+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/2524008101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce586db0 msgr2=0x55b9ce5840e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.786+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/2524008101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce586db0 0x55b9ce5840e0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000ac50 tx=0x7fbc4000ad30 comp rx=0 tx=0).stop 2024-03-18T20:56:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.786+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/2524008101 shutdown_connections 2024-03-18T20:56:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.786+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/2524008101 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b9ce5a7710 0x55b9ce5a9b30 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.786+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/2524008101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce586db0 0x55b9ce5840e0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.786+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/2524008101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce57b570 0x55b9ce586770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.786+0000 7fbc26ffd700 0 client.20601 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.786+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/2524008101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 0x55b9ce583a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.786+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/2524008101 >> 172.21.15.46:0/2524008101 conn(0x55b9ce4f21c0 msgr2=0x55b9ce577650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.787+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/2524008101 shutdown_connections 2024-03-18T20:56:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.787+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/2524008101 wait complete. 2024-03-18T20:56:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.787+0000 7fbc5a0dfb80 20 client.20601 trim_cache size 0 max 16384 2024-03-18T20:56:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.806+0000 7fbc5a0dfb80 1 Processor -- start 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.806+0000 7fbc5a0dfb80 1 -- start start 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc5a0dfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce583fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc5a0dfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 0x55b9ce577500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc5a0dfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce586db0 0x55b9ce577a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc5a0dfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ce5741d0 con 0x55b9ce57b570 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc5a0dfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ce577f80 con 0x55b9ce586db0 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc5a0dfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ce56f200 con 0x55b9ce57be90 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce583fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc46ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 0x55b9ce577500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce583fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58256/0 (socket says 172.21.15.46:58256) 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc477fe700 1 -- 172.21.15.46:0/958496541 learned_addr learned my addr 172.21.15.46:0/958496541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc47fff700 1 --2- 172.21.15.46:0/958496541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce586db0 0x55b9ce577a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc477fe700 1 -- 172.21.15.46:0/958496541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 msgr2=0x55b9ce577500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc477fe700 1 --2- 172.21.15.46:0/958496541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 0x55b9ce577500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc477fe700 1 -- 172.21.15.46:0/958496541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce586db0 msgr2=0x55b9ce577a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc477fe700 1 --2- 172.21.15.46:0/958496541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce586db0 0x55b9ce577a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc477fe700 1 -- 172.21.15.46:0/958496541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9ce494e40 con 0x55b9ce57b570 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.807+0000 7fbc477fe700 1 --2- 172.21.15.46:0/958496541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce583fb0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fbc38005cd0 tx=0x7fbc3809a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4f98f09e47006a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.808+0000 7fbc2e7fc700 1 -- 172.21.15.46:0/958496541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3809abf0 con 0x55b9ce57b570 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.808+0000 7fbc2e7fc700 1 -- 172.21.15.46:0/958496541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc380b2070 con 0x55b9ce57b570 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.808+0000 7fbc2e7fc700 1 -- 172.21.15.46:0/958496541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3809b090 con 0x55b9ce57b570 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.809+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/958496541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 msgr2=0x55b9ce583fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.809+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/958496541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce583fb0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fbc38005cd0 tx=0x7fbc3809a680 comp rx=0 tx=0).stop 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.809+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/958496541 shutdown_connections 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.809+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/958496541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce583fb0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.809+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/958496541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce586db0 0x55b9ce577a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.809+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/958496541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce57be90 0x55b9ce577500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.809+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/958496541 >> 172.21.15.46:0/958496541 conn(0x55b9ce4f21c0 msgr2=0x55b9ce582330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.809+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/958496541 shutdown_connections 2024-03-18T20:56:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.809+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/958496541 wait complete. 2024-03-18T20:56:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.809+0000 7fbc5a0dfb80 1 -- start start 2024-03-18T20:56:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc5a0dfb80 1 Processor -- start 2024-03-18T20:56:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc5a0dfb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce59a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc5a0dfb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce57be90 0x55b9ce59ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc5a0dfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce586db0 0x55b9ce5f4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc5a0dfb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ce5f4ab0 con 0x55b9ce57b570 2024-03-18T20:56:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc5a0dfb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ce5f4c20 con 0x55b9ce57be90 2024-03-18T20:56:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc5a0dfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9ce5f4d90 con 0x55b9ce586db0 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc5a0dfb80 10 client.0 _ll_register_callbacks cb 0x7fbc4000a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce59a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce59a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58262/0 (socket says 172.21.15.46:58262) 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce586db0 0x55b9ce5f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc46ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce57be90 0x55b9ce59ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc477fe700 1 -- 172.21.15.46:0/3777313499 learned_addr learned my addr 172.21.15.46:0/3777313499 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc477fe700 1 -- 172.21.15.46:0/3777313499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce586db0 msgr2=0x55b9ce5f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc477fe700 1 --2- 172.21.15.46:0/3777313499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce586db0 0x55b9ce5f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc477fe700 1 -- 172.21.15.46:0/3777313499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce57be90 msgr2=0x55b9ce59ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc477fe700 1 --2- 172.21.15.46:0/3777313499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce57be90 0x55b9ce59ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.810+0000 7fbc477fe700 1 -- 172.21.15.46:0/3777313499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9ce350350 con 0x55b9ce57b570 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.811+0000 7fbc46ffd700 1 --2- 172.21.15.46:0/3777313499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce57be90 0x55b9ce59ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.811+0000 7fbc477fe700 1 --2- 172.21.15.46:0/3777313499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce59a940 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbc38098f40 tx=0x7fbc38009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae513c3ff5873ad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.811+0000 7fbc26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.811+0000 7fbc26ffd700 1 -- 172.21.15.46:0/3777313499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3809a3a0 con 0x55b9ce57b570 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.811+0000 7fbc26ffd700 1 -- 172.21.15.46:0/3777313499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fbc3809abf0 con 0x55b9ce57b570 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.811+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3777313499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc3809ea20 con 0x55b9ce57b570 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.811+0000 7fbc26ffd700 1 -- 172.21.15.46:0/3777313499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc3809b060 con 0x55b9ce57b570 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.812+0000 7fbc5a0dfb80 10 client.20607 fetch_fsmap learned FSMap version 88 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.812+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3777313499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x55b9ce354c70 con 0x55b9ce57b570 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.812+0000 7fbc26ffd700 1 -- 172.21.15.46:0/3777313499 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc3809ea20 con 0x55b9ce57b570 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.812+0000 7fbc26ffd700 1 -- 172.21.15.46:0/3777313499 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 88) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fbc380969e0 con 0x55b9ce57b570 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.812+0000 7fbc5a0dfb80 10 client.20607 fetch_fsmap finished waiting for FSMap version 88 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.812+0000 7fbc5a0dfb80 10 client.20607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.812+0000 7fbc5a0dfb80 20 client.20607 populate_metadata read hostname 'smithi046' 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.812+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/3777313499 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b9ce5fc070 0x55b9ce5fe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.812+0000 7fbc5a0dfb80 4 client.20607 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.812+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3777313499 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b9ce31dc90 con 0x55b9ce5fc070 2024-03-18T20:56:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.812+0000 7fbc46ffd700 1 --2- 172.21.15.46:0/3777313499 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b9ce5fc070 0x55b9ce5fe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.813+0000 7fbc26ffd700 10 client.20607 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:45.813+0000 7fbc46ffd700 1 --2- 172.21.15.46:0/3777313499 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b9ce5fc070 0x55b9ce5fe460 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eea125b770feb60 server_cookie=947dc195c66c1b38 in_seq=0 out_seq=0 2024-03-18T20:56:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.029+0000 7fbc26ffd700 1 -- 172.21.15.46:0/3777313499 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b9ce31dc90 con 0x55b9ce5fc070 2024-03-18T20:56:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.029+0000 7fbc26ffd700 10 client.20607 handle_command_reply: tid=0 2024-03-18T20:56:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.029+0000 7fbc5a0dfb80 1 client.20607 shutdown 2024-03-18T20:56:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.030+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3777313499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 msgr2=0x55b9ce59a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.030+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/3777313499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce59a940 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbc38098f40 tx=0x7fbc38009310 comp rx=0 tx=0).stop 2024-03-18T20:56:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.030+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3777313499 shutdown_connections 2024-03-18T20:56:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.030+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/3777313499 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b9ce5fc070 0x55b9ce5fe460 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.030+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/3777313499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b9ce57b570 0x55b9ce59a940 secure :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbc38098f40 tx=0x7fbc38009310 comp rx=0 tx=0).stop 2024-03-18T20:56:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.030+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/3777313499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b9ce586db0 0x55b9ce5f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.030+0000 7fbc5a0dfb80 1 --2- 172.21.15.46:0/3777313499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b9ce57be90 0x55b9ce59ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.030+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3777313499 >> 172.21.15.46:0/3777313499 conn(0x55b9ce4f21c0 msgr2=0x55b9ce582280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.030+0000 7fbc26ffd700 0 client.20607 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.034+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3777313499 shutdown_connections 2024-03-18T20:56:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.035+0000 7fbc5a0dfb80 1 -- 172.21.15.46:0/3777313499 wait complete. 2024-03-18T20:56:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.036+0000 7fbc5a0dfb80 20 client.20607 trim_cache size 0 max 16384 2024-03-18T20:56:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:56:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:56:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:56:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:56:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.038+0000 7fbc4df4a700 1 -- 172.21.15.46:0/137621748 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbc300a1290 msgr2=0x7fbc300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:46.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.038+0000 7fbc4df4a700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbc300a1290 0x7fbc300a3750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbc4819ee80 tx=0x7fbc3c00b410 comp rx=0 tx=0).stop 2024-03-18T20:56:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.038+0000 7fbc4df4a700 1 -- 172.21.15.46:0/137621748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc48105be0 msgr2=0x7fbc481a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.038+0000 7fbc4df4a700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc48105be0 0x7fbc481a3600 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbc380095c0 tx=0x7fbc3800f690 comp rx=0 tx=0).stop 2024-03-18T20:56:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.042+0000 7fbc4df4a700 1 -- 172.21.15.46:0/137621748 shutdown_connections 2024-03-18T20:56:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.042+0000 7fbc4df4a700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fbc300a1290 0x7fbc300a3750 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.042+0000 7fbc4df4a700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc4819d6a0 0x7fbc4819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.042+0000 7fbc4df4a700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc48105be0 0x7fbc481a3600 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.042+0000 7fbc4df4a700 1 --2- 172.21.15.46:0/137621748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc48106050 0x7fbc481a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.042+0000 7fbc4df4a700 1 -- 172.21.15.46:0/137621748 >> 172.21.15.46:0/137621748 conn(0x7fbc48076a80 msgr2=0x7fbc481025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.042+0000 7fbc4df4a700 1 -- 172.21.15.46:0/137621748 shutdown_connections 2024-03-18T20:56:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:46.042+0000 7fbc4df4a700 1 -- 172.21.15.46:0/137621748 wait complete. 2024-03-18T20:56:46.431 INFO:tasks.workunit.client.0.smithi046.stdout: 10 316368 34.62 MB/sec execute 322 sec latency 245.924 ms 2024-03-18T20:56:46.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:46 smithi155 ceph-mon[48344]: pgmap v651: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 21 MiB/s wr, 987 op/s 2024-03-18T20:56:46.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:46 smithi046 ceph-mon[43495]: pgmap v651: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 21 MiB/s wr, 987 op/s 2024-03-18T20:56:46.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:46 smithi073 ceph-mon[47384]: pgmap v651: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 21 MiB/s wr, 987 op/s 2024-03-18T20:56:47.432 INFO:tasks.workunit.client.0.smithi046.stdout: 10 316972 34.61 MB/sec execute 323 sec latency 88.184 ms 2024-03-18T20:56:48.432 INFO:tasks.workunit.client.0.smithi046.stdout: 10 317567 34.58 MB/sec execute 324 sec latency 111.991 ms 2024-03-18T20:56:48.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:48 smithi073 ceph-mon[47384]: pgmap v652: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 10 MiB/s wr, 588 op/s 2024-03-18T20:56:48.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:48 smithi155 ceph-mon[48344]: pgmap v652: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 10 MiB/s wr, 588 op/s 2024-03-18T20:56:48.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:48 smithi046 ceph-mon[43495]: pgmap v652: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 10 MiB/s wr, 588 op/s 2024-03-18T20:56:49.432 INFO:tasks.workunit.client.0.smithi046.stdout: 10 318308 34.61 MB/sec execute 325 sec latency 106.691 ms 2024-03-18T20:56:50.432 INFO:tasks.workunit.client.0.smithi046.stdout: 10 319007 34.61 MB/sec execute 326 sec latency 87.412 ms 2024-03-18T20:56:50.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:50 smithi073 ceph-mon[47384]: pgmap v653: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 818 op/s 2024-03-18T20:56:50.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:50 smithi155 ceph-mon[48344]: pgmap v653: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 818 op/s 2024-03-18T20:56:50.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:50 smithi046 ceph-mon[43495]: pgmap v653: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 13 MiB/s wr, 818 op/s 2024-03-18T20:56:51.432 INFO:tasks.workunit.client.0.smithi046.stdout: 10 319789 34.65 MB/sec execute 327 sec latency 140.242 ms 2024-03-18T20:56:52.432 INFO:tasks.workunit.client.0.smithi046.stdout: 10 320377 34.65 MB/sec execute 328 sec latency 91.006 ms 2024-03-18T20:56:52.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:52 smithi073 ceph-mon[47384]: pgmap v654: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 11 MiB/s wr, 607 op/s 2024-03-18T20:56:52.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:52 smithi155 ceph-mon[48344]: pgmap v654: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 11 MiB/s wr, 607 op/s 2024-03-18T20:56:52.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:52 smithi046 ceph-mon[43495]: pgmap v654: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 11 MiB/s wr, 607 op/s 2024-03-18T20:56:53.432 INFO:tasks.workunit.client.0.smithi046.stdout: 10 320891 34.61 MB/sec execute 329 sec latency 92.037 ms 2024-03-18T20:56:54.433 INFO:tasks.workunit.client.0.smithi046.stdout: 10 321514 34.60 MB/sec execute 330 sec latency 109.835 ms 2024-03-18T20:56:54.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:54 smithi073 ceph-mon[47384]: pgmap v655: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 650 op/s 2024-03-18T20:56:54.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:54 smithi155 ceph-mon[48344]: pgmap v655: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 650 op/s 2024-03-18T20:56:54.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:54 smithi046 ceph-mon[43495]: pgmap v655: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 650 op/s 2024-03-18T20:56:55.432 INFO:tasks.workunit.client.0.smithi046.stdout: 10 322249 34.63 MB/sec execute 331 sec latency 76.918 ms 2024-03-18T20:56:56.434 INFO:tasks.workunit.client.0.smithi046.stdout: 10 322934 34.63 MB/sec execute 332 sec latency 148.219 ms 2024-03-18T20:56:56.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:56 smithi155 ceph-mon[48344]: pgmap v656: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 742 op/s 2024-03-18T20:56:56.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:56 smithi046 ceph-mon[43495]: pgmap v656: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 742 op/s 2024-03-18T20:56:56.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:56 smithi073 ceph-mon[47384]: pgmap v656: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 742 op/s 2024-03-18T20:56:57.433 INFO:tasks.workunit.client.0.smithi046.stdout: 10 323587 34.67 MB/sec execute 333 sec latency 77.338 ms 2024-03-18T20:56:58.433 INFO:tasks.workunit.client.0.smithi046.stdout: 10 324019 34.61 MB/sec execute 334 sec latency 95.690 ms 2024-03-18T20:56:58.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:56:58 smithi073 ceph-mon[47384]: pgmap v657: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.6 MiB/s wr, 528 op/s 2024-03-18T20:56:58.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:56:58 smithi155 ceph-mon[48344]: pgmap v657: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.6 MiB/s wr, 528 op/s 2024-03-18T20:56:58.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:56:58 smithi046 ceph-mon[43495]: pgmap v657: 97 pgs: 97 active+clean; 2.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.6 MiB/s wr, 528 op/s 2024-03-18T20:56:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.256+0000 7f384fd46700 1 -- 172.21.15.46:0/3039029608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3848103ad0 msgr2=0x7f3848107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.256+0000 7f384fd46700 1 --2- 172.21.15.46:0/3039029608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3848103ad0 0x7f3848107b10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f38380098f0 tx=0x7f3838009c00 comp rx=0 tx=0).stop 2024-03-18T20:56:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.256+0000 7f384fd46700 1 -- 172.21.15.46:0/3039029608 shutdown_connections 2024-03-18T20:56:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.256+0000 7f384fd46700 1 --2- 172.21.15.46:0/3039029608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3848108240 0x7f384810fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.256+0000 7f384fd46700 1 --2- 172.21.15.46:0/3039029608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3848103120 0x7f3848103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.256+0000 7f384fd46700 1 --2- 172.21.15.46:0/3039029608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3848103ad0 0x7f3848107b10 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.256+0000 7f384fd46700 1 -- 172.21.15.46:0/3039029608 >> 172.21.15.46:0/3039029608 conn(0x7f38480fe960 msgr2=0x7f3848100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.258+0000 7f384fd46700 1 -- 172.21.15.46:0/3039029608 shutdown_connections 2024-03-18T20:56:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.259+0000 7f384fd46700 1 -- 172.21.15.46:0/3039029608 wait complete. 2024-03-18T20:56:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.260+0000 7f384fd46700 1 Processor -- start 2024-03-18T20:56:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.260+0000 7f384fd46700 1 -- start start 2024-03-18T20:56:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.260+0000 7f384fd46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3848103120 0x7f384810e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.260+0000 7f384fd46700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3848108240 0x7f384810ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.260+0000 7f384fd46700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3848071ea0 0x7f3848072350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.260+0000 7f384fd46700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3848072920 con 0x7f3848071ea0 2024-03-18T20:56:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.260+0000 7f384fd46700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3848072a90 con 0x7f3848103120 2024-03-18T20:56:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.260+0000 7f384fd46700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3848072c50 con 0x7f3848108240 2024-03-18T20:56:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.263+0000 7f384d2e1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3848108240 0x7f384810ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.263+0000 7f384d2e1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3848108240 0x7f384810ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40568/0 (socket says 172.21.15.46:40568) 2024-03-18T20:56:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.263+0000 7f384d2e1700 1 -- 172.21.15.46:0/2294034270 learned_addr learned my addr 172.21.15.46:0/2294034270 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.263+0000 7f384e2e3700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3848071ea0 0x7f3848072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.263+0000 7f384dae2700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3848103120 0x7f384810e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.263+0000 7f384d2e1700 1 -- 172.21.15.46:0/2294034270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3848103120 msgr2=0x7f384810e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.263+0000 7f384d2e1700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3848103120 0x7f384810e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.263+0000 7f384d2e1700 1 -- 172.21.15.46:0/2294034270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3848071ea0 msgr2=0x7f3848072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.263+0000 7f384d2e1700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3848071ea0 0x7f3848072350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.263+0000 7f384d2e1700 1 -- 172.21.15.46:0/2294034270 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3838009520 con 0x7f3848108240 2024-03-18T20:56:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.263+0000 7f384d2e1700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3848108240 0x7f384810ed10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3838009810 tx=0x7f383800bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.264+0000 7f383effd700 1 -- 172.21.15.46:0/2294034270 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383801c070 con 0x7f3848108240 2024-03-18T20:56:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.264+0000 7f384fd46700 1 -- 172.21.15.46:0/2294034270 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38481148a0 con 0x7f3848108240 2024-03-18T20:56:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.264+0000 7f384fd46700 1 -- 172.21.15.46:0/2294034270 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3848114da0 con 0x7f3848108240 2024-03-18T20:56:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.265+0000 7f383effd700 1 -- 172.21.15.46:0/2294034270 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3838005500 con 0x7f3848108240 2024-03-18T20:56:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.265+0000 7f383effd700 1 -- 172.21.15.46:0/2294034270 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838004a70 con 0x7f3848108240 2024-03-18T20:56:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.271+0000 7f385bedbb80 1 Processor -- start 2024-03-18T20:56:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.272+0000 7f383effd700 1 -- 172.21.15.46:0/2294034270 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f383800b8c0 con 0x7f3848108240 2024-03-18T20:56:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.272+0000 7f383effd700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38340a1350 0x7f38340a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.273+0000 7f383effd700 1 -- 172.21.15.46:0/2294034270 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f38380cdf10 con 0x7f3848108240 2024-03-18T20:56:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.273+0000 7f385bedbb80 1 -- start start 2024-03-18T20:56:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.273+0000 7f384e2e3700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38340a1350 0x7f38340a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.274+0000 7f385bedbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 0x55e8738b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.274+0000 7f385bedbb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3e90 0x55e8738be870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.274+0000 7f385bedbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738bedb0 0x55e8738c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.274+0000 7f385bedbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87380d930 con 0x55e8738bedb0 2024-03-18T20:56:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.274+0000 7f385bedbb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87380b2f0 con 0x55e8738b3e90 2024-03-18T20:56:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.274+0000 7f385bedbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e873806be0 con 0x55e8738b3570 2024-03-18T20:56:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.274+0000 7f384dae2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 0x55e8738b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.274+0000 7f384dae2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 0x55e8738b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40574/0 (socket says 172.21.15.46:40574) 2024-03-18T20:56:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.274+0000 7f384dae2700 1 -- 172.21.15.46:0/2663287432 learned_addr learned my addr 172.21.15.46:0/2663287432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.274+0000 7f384e2e3700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38340a1350 0x7f38340a3810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3840009ef0 tx=0x7f384000c420 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.275+0000 7f384dae2700 1 -- 172.21.15.46:0/2663287432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3e90 msgr2=0x55e8738be870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:56:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.275+0000 7f384dae2700 1 --2- 172.21.15.46:0/2663287432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3e90 0x55e8738be870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.275+0000 7f384dae2700 1 -- 172.21.15.46:0/2663287432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738bedb0 msgr2=0x55e8738c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.275+0000 7f384dae2700 1 --2- 172.21.15.46:0/2663287432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738bedb0 0x55e8738c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.275+0000 7f384dae2700 1 -- 172.21.15.46:0/2663287432 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e873688350 con 0x55e8738b3570 2024-03-18T20:56:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.275+0000 7f384dae2700 1 --2- 172.21.15.46:0/2663287432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 0x55e8738b3950 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f384401acf0 tx=0x7f38440223f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=705852c5abe4b92 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.276+0000 7f3833fff700 1 -- 172.21.15.46:0/2663287432 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844022d30 con 0x55e8738b3570 2024-03-18T20:56:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.276+0000 7f3833fff700 1 -- 172.21.15.46:0/2663287432 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3844022e90 con 0x55e8738b3570 2024-03-18T20:56:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.276+0000 7f3833fff700 1 -- 172.21.15.46:0/2663287432 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384402a8a0 con 0x55e8738b3570 2024-03-18T20:56:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.276+0000 7f385bedbb80 1 -- 172.21.15.46:0/2663287432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 msgr2=0x55e8738b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.276+0000 7f385bedbb80 1 --2- 172.21.15.46:0/2663287432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 0x55e8738b3950 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f384401acf0 tx=0x7f38440223f0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.277+0000 7f385bedbb80 1 -- 172.21.15.46:0/2663287432 shutdown_connections 2024-03-18T20:56:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.277+0000 7f385bedbb80 1 --2- 172.21.15.46:0/2663287432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738bedb0 0x55e8738c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.277+0000 7f385bedbb80 1 --2- 172.21.15.46:0/2663287432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 0x55e8738b3950 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.277+0000 7f385bedbb80 1 --2- 172.21.15.46:0/2663287432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3e90 0x55e8738be870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.277+0000 7f385bedbb80 1 -- 172.21.15.46:0/2663287432 >> 172.21.15.46:0/2663287432 conn(0x55e87382a1c0 msgr2=0x55e8738afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.277+0000 7f385bedbb80 1 -- 172.21.15.46:0/2663287432 shutdown_connections 2024-03-18T20:56:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.277+0000 7f385bedbb80 1 -- 172.21.15.46:0/2663287432 wait complete. 2024-03-18T20:56:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.277+0000 7f385bedbb80 1 -- start start 2024-03-18T20:56:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.277+0000 7f385bedbb80 1 Processor -- start 2024-03-18T20:56:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.278+0000 7f385bedbb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3570 0x55e8738be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.278+0000 7f385bedbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 0x55e8738bbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.278+0000 7f385bedbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 0x55e8738bc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.278+0000 7f385bedbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8737a2d20 con 0x55e8738b3e90 2024-03-18T20:56:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.278+0000 7f385bedbb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8737a29c0 con 0x55e8738b3570 2024-03-18T20:56:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.278+0000 7f385bedbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87368a640 con 0x55e8738bedb0 2024-03-18T20:56:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.278+0000 7f385bedbb80 10 client.0 _ll_register_callbacks cb 0x55e87360e340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:56:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.279+0000 7f384e2e3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 0x55e8738bc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.279+0000 7f384e2e3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 0x55e8738bc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40576/0 (socket says 172.21.15.46:40576) 2024-03-18T20:56:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.279+0000 7f384e2e3700 1 -- 172.21.15.46:0/1761500980 learned_addr learned my addr 172.21.15.46:0/1761500980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.281+0000 7f384dae2700 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3570 0x55e8738be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.281+0000 7f384d2e1700 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 0x55e8738bbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.281+0000 7f384e2e3700 1 -- 172.21.15.46:0/1761500980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3570 msgr2=0x55e8738be830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.281+0000 7f384e2e3700 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3570 0x55e8738be830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.281+0000 7f384e2e3700 1 -- 172.21.15.46:0/1761500980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 msgr2=0x55e8738bbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.281+0000 7f384e2e3700 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 0x55e8738bbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.281+0000 7f384e2e3700 1 -- 172.21.15.46:0/1761500980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384401a950 con 0x55e8738bedb0 2024-03-18T20:56:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.282+0000 7f384e2e3700 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 0x55e8738bc210 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3840019a30 tx=0x7f3840004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99c0c41816136031 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.282+0000 7f38317fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:56:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.282+0000 7f38317fa700 1 -- 172.21.15.46:0/1761500980 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840011040 con 0x55e8738bedb0 2024-03-18T20:56:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.282+0000 7f38317fa700 1 -- 172.21.15.46:0/1761500980 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3840006e40 con 0x55e8738bedb0 2024-03-18T20:56:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.282+0000 7f385bedbb80 1 -- 172.21.15.46:0/1761500980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8738a76c0 con 0x55e8738bedb0 2024-03-18T20:56:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.283+0000 7f38317fa700 1 -- 172.21.15.46:0/1761500980 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3840019ea0 con 0x55e8738bedb0 2024-03-18T20:56:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.283+0000 7f385bedbb80 10 client.30319 fetch_fsmap learned FSMap version 88 2024-03-18T20:56:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.283+0000 7f385bedbb80 1 -- 172.21.15.46:0/1761500980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x55e873634b00 con 0x55e8738bedb0 2024-03-18T20:56:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.283+0000 7f38317fa700 1 -- 172.21.15.46:0/1761500980 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e8738a76c0 con 0x55e8738bedb0 2024-03-18T20:56:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.283+0000 7f38317fa700 1 -- 172.21.15.46:0/1761500980 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 88) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f384002b020 con 0x55e8738bedb0 2024-03-18T20:56:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.284+0000 7f385bedbb80 10 client.30319 fetch_fsmap finished waiting for FSMap version 88 2024-03-18T20:56:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.284+0000 7f385bedbb80 10 client.30319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:56:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.284+0000 7f385bedbb80 20 client.30319 populate_metadata read hostname 'smithi046' 2024-03-18T20:56:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.284+0000 7f385bedbb80 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8738ceb70 0x55e8738d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.284+0000 7f385bedbb80 4 client.30319 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:56:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.284+0000 7f385bedbb80 1 -- 172.21.15.46:0/1761500980 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8737ff6a0 con 0x55e8738ceb70 2024-03-18T20:56:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.284+0000 7f384dae2700 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8738ceb70 0x55e8738d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.285+0000 7f384dae2700 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8738ceb70 0x55e8738d0f90 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6429900038ea644 server_cookie=19a22a0d6466730 in_seq=0 out_seq=0 2024-03-18T20:56:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.285+0000 7f38317fa700 10 client.30319 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.290+0000 7f38317fa700 1 -- 172.21.15.46:0/1761500980 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e8737ff6a0 con 0x55e8738ceb70 2024-03-18T20:56:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.290+0000 7f38317fa700 10 client.30319 handle_command_reply: tid=0 2024-03-18T20:56:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.290+0000 7f385bedbb80 1 client.30319 shutdown 2024-03-18T20:56:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.290+0000 7f385bedbb80 1 -- 172.21.15.46:0/1761500980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 msgr2=0x55e8738bc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.290+0000 7f385bedbb80 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 0x55e8738bc210 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3840019a30 tx=0x7f3840004be0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.290+0000 7f385bedbb80 1 -- 172.21.15.46:0/1761500980 shutdown_connections 2024-03-18T20:56:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.290+0000 7f385bedbb80 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e8738ceb70 0x55e8738d0f90 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.290+0000 7f385bedbb80 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 0x55e8738bc210 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.290+0000 7f385bedbb80 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3570 0x55e8738be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.290+0000 7f385bedbb80 1 --2- 172.21.15.46:0/1761500980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 0x55e8738bbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.290+0000 7f385bedbb80 1 -- 172.21.15.46:0/1761500980 >> 172.21.15.46:0/1761500980 conn(0x55e87382a1c0 msgr2=0x55e8738af650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.290+0000 7f38317fa700 0 client.30319 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.291+0000 7f385bedbb80 1 -- 172.21.15.46:0/1761500980 shutdown_connections 2024-03-18T20:56:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.291+0000 7f385bedbb80 1 -- 172.21.15.46:0/1761500980 wait complete. 2024-03-18T20:56:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.292+0000 7f385bedbb80 20 client.30319 trim_cache size 0 max 16384 2024-03-18T20:56:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.312+0000 7f385bedbb80 1 Processor -- start 2024-03-18T20:56:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.312+0000 7f385bedbb80 1 -- start start 2024-03-18T20:56:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.312+0000 7f385bedbb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3570 0x55e8738c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.312+0000 7f385bedbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 0x55e8738be360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.312+0000 7f385bedbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 0x55e8738b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.312+0000 7f385bedbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8738c5fe0 con 0x55e8738b3e90 2024-03-18T20:56:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.312+0000 7f385bedbb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8738b0fb0 con 0x55e8738b3570 2024-03-18T20:56:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.312+0000 7f385bedbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8738b7960 con 0x55e8738bedb0 2024-03-18T20:56:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.314+0000 7f384dae2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3570 0x55e8738c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.314+0000 7f384e2e3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 0x55e8738b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.314+0000 7f384e2e3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 0x55e8738b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:40578/0 (socket says 172.21.15.46:40578) 2024-03-18T20:56:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.314+0000 7f384e2e3700 1 -- 172.21.15.46:0/1683973912 learned_addr learned my addr 172.21.15.46:0/1683973912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.314+0000 7f384d2e1700 1 --2- 172.21.15.46:0/1683973912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 0x55e8738be360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.315+0000 7f384d2e1700 1 -- 172.21.15.46:0/1683973912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 msgr2=0x55e8738b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.315+0000 7f384d2e1700 1 --2- 172.21.15.46:0/1683973912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 0x55e8738b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.315+0000 7f384d2e1700 1 -- 172.21.15.46:0/1683973912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3570 msgr2=0x55e8738c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.315+0000 7f384d2e1700 1 --2- 172.21.15.46:0/1683973912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3570 0x55e8738c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.315+0000 7f384d2e1700 1 -- 172.21.15.46:0/1683973912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8737cce40 con 0x55e8738b3e90 2024-03-18T20:56:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.315+0000 7f384d2e1700 1 --2- 172.21.15.46:0/1683973912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 0x55e8738be360 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f38380cb5e0 tx=0x7f383809a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa7b4e1a60b6354 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.319+0000 7f383cff9700 1 -- 172.21.15.46:0/1683973912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383809ee60 con 0x55e8738b3e90 2024-03-18T20:56:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.319+0000 7f383cff9700 1 -- 172.21.15.46:0/1683973912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38380a3420 con 0x55e8738b3e90 2024-03-18T20:56:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.319+0000 7f383cff9700 1 -- 172.21.15.46:0/1683973912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38380cb860 con 0x55e8738b3e90 2024-03-18T20:56:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.320+0000 7f385bedbb80 1 -- 172.21.15.46:0/1683973912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 msgr2=0x55e8738be360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.320+0000 7f385bedbb80 1 --2- 172.21.15.46:0/1683973912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 0x55e8738be360 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f38380cb5e0 tx=0x7f383809a990 comp rx=0 tx=0).stop 2024-03-18T20:56:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.322+0000 7f385bedbb80 1 -- 172.21.15.46:0/1683973912 shutdown_connections 2024-03-18T20:56:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.322+0000 7f385bedbb80 1 --2- 172.21.15.46:0/1683973912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738bedb0 0x55e8738b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.322+0000 7f385bedbb80 1 --2- 172.21.15.46:0/1683973912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738b3570 0x55e8738c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.322+0000 7f385bedbb80 1 --2- 172.21.15.46:0/1683973912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 0x55e8738be360 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.322+0000 7f385bedbb80 1 -- 172.21.15.46:0/1683973912 >> 172.21.15.46:0/1683973912 conn(0x55e87382a1c0 msgr2=0x55e8738ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.325+0000 7f385bedbb80 1 -- 172.21.15.46:0/1683973912 shutdown_connections 2024-03-18T20:56:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.325+0000 7f385bedbb80 1 -- 172.21.15.46:0/1683973912 wait complete. 2024-03-18T20:56:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.325+0000 7f385bedbb80 1 -- start start 2024-03-18T20:56:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.325+0000 7f385bedbb80 1 Processor -- start 2024-03-18T20:56:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.326+0000 7f385bedbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 0x55e8738c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.326+0000 7f385bedbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 0x55e8738c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.326+0000 7f385bedbb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738bedb0 0x55e8738c3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.326+0000 7f385bedbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8738c3fc0 con 0x55e8738b3e90 2024-03-18T20:56:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.326+0000 7f385bedbb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8738c4130 con 0x55e8738bedb0 2024-03-18T20:56:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.326+0000 7f385bedbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8738c42f0 con 0x55e8738b3570 2024-03-18T20:56:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.326+0000 7f385bedbb80 10 client.0 _ll_register_callbacks cb 0x7f38400193d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:56:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.326+0000 7f384e2e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738bedb0 0x55e8738c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.326+0000 7f384e2e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738bedb0 0x55e8738c3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:48882/0 (socket says 172.21.15.46:48882) 2024-03-18T20:56:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.326+0000 7f384e2e3700 1 -- 172.21.15.46:0/167449110 learned_addr learned my addr 172.21.15.46:0/167449110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:56:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.327+0000 7f384dae2700 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 0x55e8738c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.327+0000 7f384e2e3700 1 -- 172.21.15.46:0/167449110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 msgr2=0x55e8738c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.327+0000 7f384e2e3700 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 0x55e8738c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.327+0000 7f384e2e3700 1 -- 172.21.15.46:0/167449110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 msgr2=0x55e8738c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:56:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.327+0000 7f384e2e3700 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 0x55e8738c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.327+0000 7f384e2e3700 1 -- 172.21.15.46:0/167449110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3848114da0 con 0x55e8738bedb0 2024-03-18T20:56:59.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.327+0000 7f384dae2700 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 0x55e8738c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:56:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.327+0000 7f384e2e3700 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738bedb0 0x55e8738c3970 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f384001ace0 tx=0x7f3840024400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad00802a43bd48c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:56:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.327+0000 7f38317fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:56:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.327+0000 7f38317fa700 1 -- 172.21.15.46:0/167449110 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840011040 con 0x55e8738bedb0 2024-03-18T20:56:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.327+0000 7f38317fa700 1 -- 172.21.15.46:0/167449110 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f384001a440 con 0x55e8738bedb0 2024-03-18T20:56:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.331+0000 7f385bedbb80 1 -- 172.21.15.46:0/167449110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3844022e90 con 0x55e8738bedb0 2024-03-18T20:56:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.332+0000 7f38317fa700 1 -- 172.21.15.46:0/167449110 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f384001a600 con 0x55e8738bedb0 2024-03-18T20:56:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.332+0000 7f385bedbb80 10 client.30218 fetch_fsmap learned FSMap version 88 2024-03-18T20:56:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.332+0000 7f385bedbb80 1 -- 172.21.15.46:0/167449110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=4+}) v3 -- 0x55e87368cc70 con 0x55e8738bedb0 2024-03-18T20:56:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.332+0000 7f38317fa700 1 -- 172.21.15.46:0/167449110 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3844022e90 con 0x55e8738bedb0 2024-03-18T20:56:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.332+0000 7f38317fa700 1 -- 172.21.15.46:0/167449110 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 88) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3840019b10 con 0x55e8738bedb0 2024-03-18T20:56:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.333+0000 7f385bedbb80 10 client.30218 fetch_fsmap finished waiting for FSMap version 88 2024-03-18T20:56:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.333+0000 7f385bedbb80 10 client.30218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:56:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.333+0000 7f385bedbb80 20 client.30218 populate_metadata read hostname 'smithi046' 2024-03-18T20:56:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.333+0000 7f385bedbb80 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e873923ac0 0x55e873925ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:56:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.333+0000 7f385bedbb80 4 client.30218 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:56:59.335 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.333+0000 7f385bedbb80 1 -- 172.21.15.46:0/167449110 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e873655c90 con 0x55e873923ac0 2024-03-18T20:56:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.333+0000 7f384dae2700 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e873923ac0 0x55e873925ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:56:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.335+0000 7f384dae2700 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e873923ac0 0x55e873925ee0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81bb78896ee14337 server_cookie=cd386cf2b212f897 in_seq=0 out_seq=0 2024-03-18T20:56:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.335+0000 7f38317fa700 10 client.30218 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:59.432 INFO:tasks.workunit.client.0.smithi046.stdout: 10 324427 34.57 MB/sec execute 335 sec latency 137.051 ms 2024-03-18T20:56:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.643+0000 7f38317fa700 1 -- 172.21.15.46:0/167449110 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e873655c90 con 0x55e873923ac0 2024-03-18T20:56:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.643+0000 7f38317fa700 10 client.30218 handle_command_reply: tid=0 2024-03-18T20:56:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.643+0000 7f385bedbb80 1 client.30218 shutdown 2024-03-18T20:56:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.644+0000 7f385bedbb80 1 -- 172.21.15.46:0/167449110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738bedb0 msgr2=0x55e8738c3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.644+0000 7f385bedbb80 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738bedb0 0x55e8738c3970 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f384001ace0 tx=0x7f3840024400 comp rx=0 tx=0).stop 2024-03-18T20:56:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.644+0000 7f385bedbb80 1 -- 172.21.15.46:0/167449110 shutdown_connections 2024-03-18T20:56:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.644+0000 7f385bedbb80 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e873923ac0 0x55e873925ee0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.644+0000 7f385bedbb80 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e8738bedb0 0x55e8738c3970 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.644+0000 7f385bedbb80 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8738b3570 0x55e8738c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.644+0000 7f385bedbb80 1 --2- 172.21.15.46:0/167449110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e8738b3e90 0x55e8738c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.644+0000 7f385bedbb80 1 -- 172.21.15.46:0/167449110 >> 172.21.15.46:0/167449110 conn(0x55e87382a1c0 msgr2=0x55e8738b6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.644+0000 7f38317fa700 0 client.30218 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:56:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.644+0000 7f385bedbb80 1 -- 172.21.15.46:0/167449110 shutdown_connections 2024-03-18T20:56:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.644+0000 7f385bedbb80 1 -- 172.21.15.46:0/167449110 wait complete. 2024-03-18T20:56:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.644+0000 7f385bedbb80 20 client.30218 trim_cache size 0 max 16384 2024-03-18T20:56:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:56:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:56:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:56:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:56:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.645+0000 7f384fd46700 1 -- 172.21.15.46:0/2294034270 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38340a1350 msgr2=0x7f38340a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.645+0000 7f384fd46700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38340a1350 0x7f38340a3810 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3840009ef0 tx=0x7f384000c420 comp rx=0 tx=0).stop 2024-03-18T20:56:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.645+0000 7f384fd46700 1 -- 172.21.15.46:0/2294034270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3848108240 msgr2=0x7f384810ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:56:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.645+0000 7f384fd46700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3848108240 0x7f384810ed10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3838009810 tx=0x7f383800bfa0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.648+0000 7f384fd46700 1 -- 172.21.15.46:0/2294034270 shutdown_connections 2024-03-18T20:56:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.648+0000 7f384fd46700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f38340a1350 0x7f38340a3810 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.648+0000 7f384fd46700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3848071ea0 0x7f3848072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.648+0000 7f384fd46700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3848103120 0x7f384810e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.648+0000 7f384fd46700 1 --2- 172.21.15.46:0/2294034270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3848108240 0x7f384810ed10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:56:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.648+0000 7f384fd46700 1 -- 172.21.15.46:0/2294034270 >> 172.21.15.46:0/2294034270 conn(0x7f38480fe960 msgr2=0x7f38480ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:56:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.648+0000 7f384fd46700 1 -- 172.21.15.46:0/2294034270 shutdown_connections 2024-03-18T20:56:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:56:59.648+0000 7f384fd46700 1 -- 172.21.15.46:0/2294034270 wait complete. 2024-03-18T20:57:00.432 INFO:tasks.workunit.client.0.smithi046.stdout: 10 324934 34.53 MB/sec execute 336 sec latency 226.200 ms 2024-03-18T20:57:00.488 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:00 smithi155 ceph-mon[48344]: pgmap v658: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.5 MiB/s wr, 727 op/s 2024-03-18T20:57:00.489 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:00 smithi155 ceph-mon[48344]: daemon mds.c restarted 2024-03-18T20:57:00.489 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:00 smithi155 ceph-mon[48344]: osdmap e111: 12 total, 12 up, 12 in 2024-03-18T20:57:00.489 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:00 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6836/73154035,v1:172.21.15.155:6837/73154035] up:boot 2024-03-18T20:57:00.489 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:57:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: ignoring --setuser ceph since I am not root 2024-03-18T20:57:00.489 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:57:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: ignoring --setgroup ceph since I am not root 2024-03-18T20:57:00.489 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:57:00 smithi155 ceph-mds[84998]: -- 172.21.15.155:0/1360244350 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5625c19301a0 con 0x5625c1906c00 2024-03-18T20:57:00.489 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 20:57:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: starting mds.c at 2024-03-18T20:57:00.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:00 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:57:00.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:00 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T20:57:00.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:00 smithi046 ceph-mon[43495]: pgmap v658: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.5 MiB/s wr, 727 op/s 2024-03-18T20:57:00.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:00 smithi046 ceph-mon[43495]: daemon mds.c restarted 2024-03-18T20:57:00.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:00 smithi046 ceph-mon[43495]: osdmap e111: 12 total, 12 up, 12 in 2024-03-18T20:57:00.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:00 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6836/73154035,v1:172.21.15.155:6837/73154035] up:boot 2024-03-18T20:57:00.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:00 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:57:00.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:00 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T20:57:00.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:00 smithi073 ceph-mon[47384]: pgmap v658: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.5 MiB/s wr, 727 op/s 2024-03-18T20:57:00.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:00 smithi073 ceph-mon[47384]: daemon mds.c restarted 2024-03-18T20:57:00.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:00 smithi073 ceph-mon[47384]: osdmap e111: 12 total, 12 up, 12 in 2024-03-18T20:57:00.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:00 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6836/73154035,v1:172.21.15.155:6837/73154035] up:boot 2024-03-18T20:57:00.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:00 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:57:00.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:00 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T20:57:01.436 INFO:tasks.workunit.client.0.smithi046.stdout: 10 325203 34.46 MB/sec execute 337 sec latency 292.746 ms 2024-03-18T20:57:02.435 INFO:tasks.workunit.client.0.smithi046.stdout: 10 325271 34.38 MB/sec execute 338 sec latency 1052.391 ms 2024-03-18T20:57:02.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:02 smithi155 ceph-mon[48344]: pgmap v660: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.8 MiB/s wr, 562 op/s 2024-03-18T20:57:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:02 smithi046 ceph-mon[43495]: pgmap v660: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.8 MiB/s wr, 562 op/s 2024-03-18T20:57:02.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:02 smithi073 ceph-mon[47384]: pgmap v660: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 7.8 MiB/s wr, 562 op/s 2024-03-18T20:57:03.436 INFO:tasks.workunit.client.0.smithi046.stdout: 10 325716 34.36 MB/sec execute 339 sec latency 1087.844 ms 2024-03-18T20:57:04.437 INFO:tasks.workunit.client.0.smithi046.stdout: 10 325890 34.29 MB/sec execute 340 sec latency 226.977 ms 2024-03-18T20:57:04.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:04 smithi046 ceph-mon[43495]: pgmap v661: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 646 op/s 2024-03-18T20:57:04.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:04 smithi046 ceph-mon[43495]: osdmap e112: 12 total, 12 up, 12 in 2024-03-18T20:57:04.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:04 smithi073 ceph-mon[47384]: pgmap v661: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 646 op/s 2024-03-18T20:57:04.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:04 smithi073 ceph-mon[47384]: osdmap e112: 12 total, 12 up, 12 in 2024-03-18T20:57:05.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:04 smithi155 ceph-mon[48344]: pgmap v661: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 646 op/s 2024-03-18T20:57:05.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:04 smithi155 ceph-mon[48344]: osdmap e112: 12 total, 12 up, 12 in 2024-03-18T20:57:05.437 INFO:tasks.workunit.client.0.smithi046.stdout: 10 326115 34.21 MB/sec execute 341 sec latency 647.767 ms 2024-03-18T20:57:06.438 INFO:tasks.workunit.client.0.smithi046.stdout: 10 326474 34.16 MB/sec execute 342 sec latency 1648.643 ms 2024-03-18T20:57:06.540 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:06 smithi155 ceph-mon[48344]: pgmap v663: 97 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 70 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-03-18T20:57:06.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:06 smithi046 ceph-mon[43495]: pgmap v663: 97 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 70 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-03-18T20:57:06.724 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:06 smithi073 ceph-mon[47384]: pgmap v663: 97 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 70 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 22 MiB/s wr, 1.03k op/s 2024-03-18T20:57:07.439 INFO:tasks.workunit.client.0.smithi046.stdout: 10 326937 34.17 MB/sec execute 343 sec latency 2648.683 ms 2024-03-18T20:57:08.438 INFO:tasks.workunit.client.0.smithi046.stdout: 10 327427 34.15 MB/sec execute 344 sec latency 3648.726 ms 2024-03-18T20:57:08.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:08 smithi073 ceph-mon[47384]: pgmap v664: 97 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 70 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 17 MiB/s wr, 733 op/s 2024-03-18T20:57:08.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:08 smithi155 ceph-mon[48344]: pgmap v664: 97 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 70 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 17 MiB/s wr, 733 op/s 2024-03-18T20:57:08.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:08 smithi046 ceph-mon[43495]: pgmap v664: 97 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 70 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 17 MiB/s wr, 733 op/s 2024-03-18T20:57:09.441 INFO:tasks.workunit.client.0.smithi046.stdout: 10 327928 34.13 MB/sec execute 345 sec latency 4651.213 ms 2024-03-18T20:57:10.441 INFO:tasks.workunit.client.0.smithi046.stdout: 10 328435 34.10 MB/sec execute 346 sec latency 5651.267 ms 2024-03-18T20:57:10.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:10 smithi073 ceph-mon[47384]: pgmap v665: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 17 MiB/s wr, 807 op/s 2024-03-18T20:57:10.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:10 smithi155 ceph-mon[48344]: pgmap v665: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 17 MiB/s wr, 807 op/s 2024-03-18T20:57:10.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:10 smithi046 ceph-mon[43495]: pgmap v665: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 17 MiB/s wr, 807 op/s 2024-03-18T20:57:11.441 INFO:tasks.workunit.client.0.smithi046.stdout: 10 329085 34.10 MB/sec execute 347 sec latency 6651.311 ms 2024-03-18T20:57:12.441 INFO:tasks.workunit.client.0.smithi046.stdout: 10 329736 34.10 MB/sec execute 348 sec latency 7651.349 ms 2024-03-18T20:57:12.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:12 smithi073 ceph-mon[47384]: pgmap v666: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 777 op/s 2024-03-18T20:57:12.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:12 smithi155 ceph-mon[48344]: pgmap v666: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 777 op/s 2024-03-18T20:57:12.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:12 smithi046 ceph-mon[43495]: pgmap v666: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 16 MiB/s wr, 777 op/s 2024-03-18T20:57:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.817+0000 7fcd92134700 1 -- 172.21.15.46:0/229211651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd8c10d590 msgr2=0x7fcd8c10f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.817+0000 7fcd92134700 1 --2- 172.21.15.46:0/229211651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd8c10d590 0x7fcd8c10f980 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400b0a0 tx=0x7fcd8400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.817+0000 7fcd92134700 1 -- 172.21.15.46:0/229211651 shutdown_connections 2024-03-18T20:57:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.817+0000 7fcd92134700 1 --2- 172.21.15.46:0/229211651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd8c10d590 0x7fcd8c10f980 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.817+0000 7fcd92134700 1 --2- 172.21.15.46:0/229211651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c101bc0 0x7fcd8c069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.817+0000 7fcd92134700 1 --2- 172.21.15.46:0/229211651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd8c069970 0x7fcd8c069df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.817+0000 7fcd92134700 1 -- 172.21.15.46:0/229211651 >> 172.21.15.46:0/229211651 conn(0x7fcd8c076a80 msgr2=0x7fcd8c076e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.818+0000 7fcd92134700 1 -- 172.21.15.46:0/229211651 shutdown_connections 2024-03-18T20:57:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.818+0000 7fcd92134700 1 -- 172.21.15.46:0/229211651 wait complete. 2024-03-18T20:57:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.819+0000 7fcd92134700 1 Processor -- start 2024-03-18T20:57:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.820+0000 7fcd92134700 1 -- start start 2024-03-18T20:57:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.820+0000 7fcd92134700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd8c069970 0x7fcd8c199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.820+0000 7fcd92134700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c101bc0 0x7fcd8c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.820+0000 7fcd92134700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd8c10d590 0x7fcd8c19dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.820+0000 7fcd92134700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c19e3a0 con 0x7fcd8c10d590 2024-03-18T20:57:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.820+0000 7fcd92134700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c19e510 con 0x7fcd8c069970 2024-03-18T20:57:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.820+0000 7fcd92134700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c19e6d0 con 0x7fcd8c101bc0 2024-03-18T20:57:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.820+0000 7fcd8b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd8c069970 0x7fcd8c199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.820+0000 7fcd8b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd8c069970 0x7fcd8c199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:39632/0 (socket says 172.21.15.46:39632) 2024-03-18T20:57:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.820+0000 7fcd8b7fe700 1 -- 172.21.15.46:0/3445127725 learned_addr learned my addr 172.21.15.46:0/3445127725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.820+0000 7fcd8affd700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c101bc0 0x7fcd8c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.820+0000 7fcd8bfff700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd8c10d590 0x7fcd8c19dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.821+0000 7fcd8b7fe700 1 -- 172.21.15.46:0/3445127725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c101bc0 msgr2=0x7fcd8c1999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.821+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c101bc0 0x7fcd8c1999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.821+0000 7fcd8b7fe700 1 -- 172.21.15.46:0/3445127725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd8c10d590 msgr2=0x7fcd8c19dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.821+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd8c10d590 0x7fcd8c19dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.821+0000 7fcd8b7fe700 1 -- 172.21.15.46:0/3445127725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd84009d00 con 0x7fcd8c069970 2024-03-18T20:57:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.821+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd8c069970 0x7fcd8c199480 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00e9a0 tx=0x7fcd7c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.821+0000 7fcd88ff9700 1 -- 172.21.15.46:0/3445127725 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c009790 con 0x7fcd8c069970 2024-03-18T20:57:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.821+0000 7fcd92134700 1 -- 172.21.15.46:0/3445127725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c19e9b0 con 0x7fcd8c069970 2024-03-18T20:57:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.821+0000 7fcd88ff9700 1 -- 172.21.15.46:0/3445127725 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd7c010430 con 0x7fcd8c069970 2024-03-18T20:57:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.821+0000 7fcd88ff9700 1 -- 172.21.15.46:0/3445127725 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c018680 con 0x7fcd8c069970 2024-03-18T20:57:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.821+0000 7fcd92134700 1 -- 172.21.15.46:0/3445127725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c1a61f0 con 0x7fcd8c069970 2024-03-18T20:57:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.824+0000 7fcd88ff9700 1 -- 172.21.15.46:0/3445127725 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fcd7c0105a0 con 0x7fcd8c069970 2024-03-18T20:57:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.825+0000 7fcd88ff9700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcd740a1290 0x7fcd740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.825+0000 7fcd8affd700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcd740a1290 0x7fcd740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.827 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.825+0000 7fcd88ff9700 1 -- 172.21.15.46:0/3445127725 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fcd7c014070 con 0x7fcd8c069970 2024-03-18T20:57:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.825+0000 7fcd8affd700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcd740a1290 0x7fcd740a3750 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c19a9d0 tx=0x7fcd80009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.830+0000 7fcd9e2c9b80 1 Processor -- start 2024-03-18T20:57:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.830+0000 7fcd9e2c9b80 1 -- start start 2024-03-18T20:57:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.830+0000 7fcd9e2c9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb0930570 0x55edb0930950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.830+0000 7fcd9e2c9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930e90 0x55edb0944080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.830+0000 7fcd9e2c9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb09445c0 0x55edb09469b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.830+0000 7fcd9e2c9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb088a930 con 0x55edb0930570 2024-03-18T20:57:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.830+0000 7fcd9e2c9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb08882f0 con 0x55edb0930e90 2024-03-18T20:57:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.830+0000 7fcd9e2c9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb0883be0 con 0x55edb09445c0 2024-03-18T20:57:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930e90 0x55edb0944080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb09445c0 0x55edb09469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930e90 0x55edb0944080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:39638/0 (socket says 172.21.15.46:39638) 2024-03-18T20:57:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8affd700 1 -- 172.21.15.46:0/3310673911 learned_addr learned my addr 172.21.15.46:0/3310673911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb09445c0 0x55edb09469b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:34006/0 (socket says 172.21.15.46:34006) 2024-03-18T20:57:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/3310673911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb0930570 0x55edb0930950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8bfff700 1 -- 172.21.15.46:0/3310673911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930e90 msgr2=0x55edb0944080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8bfff700 1 --2- 172.21.15.46:0/3310673911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930e90 0x55edb0944080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8bfff700 1 -- 172.21.15.46:0/3310673911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb0930570 msgr2=0x55edb0930950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8bfff700 1 --2- 172.21.15.46:0/3310673911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb0930570 0x55edb0930950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8bfff700 1 -- 172.21.15.46:0/3310673911 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edb0705350 con 0x55edb09445c0 2024-03-18T20:57:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/3310673911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb0930570 0x55edb0930950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:57:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8affd700 1 --2- 172.21.15.46:0/3310673911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930e90 0x55edb0944080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:57:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.831+0000 7fcd8bfff700 1 --2- 172.21.15.46:0/3310673911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb09445c0 0x55edb09469b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400b070 tx=0x7fcd84009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f319740acbc0f7ad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.832+0000 7fcd71ffb700 1 -- 172.21.15.46:0/3310673911 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8400e040 con 0x55edb09445c0 2024-03-18T20:57:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.832+0000 7fcd71ffb700 1 -- 172.21.15.46:0/3310673911 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd84006000 con 0x55edb09445c0 2024-03-18T20:57:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.832+0000 7fcd71ffb700 1 -- 172.21.15.46:0/3310673911 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84007740 con 0x55edb09445c0 2024-03-18T20:57:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.833+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3310673911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb09445c0 msgr2=0x55edb09469b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.833+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/3310673911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb09445c0 0x55edb09469b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400b070 tx=0x7fcd84009c60 comp rx=0 tx=0).stop 2024-03-18T20:57:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.833+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3310673911 shutdown_connections 2024-03-18T20:57:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.833+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/3310673911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb0930570 0x55edb0930950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.833+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/3310673911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb09445c0 0x55edb09469b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.833+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/3310673911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930e90 0x55edb0944080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.833+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3310673911 >> 172.21.15.46:0/3310673911 conn(0x55edb08a71c0 msgr2=0x55edb092ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.833+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3310673911 shutdown_connections 2024-03-18T20:57:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.833+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3310673911 wait complete. 2024-03-18T20:57:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.834+0000 7fcd9e2c9b80 1 -- start start 2024-03-18T20:57:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.834+0000 7fcd9e2c9b80 1 Processor -- start 2024-03-18T20:57:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.834+0000 7fcd9e2c9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930570 0x55edb093da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.834+0000 7fcd9e2c9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb093dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.834+0000 7fcd9e2c9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb09445c0 0x55edb09437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.834+0000 7fcd9e2c9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb081fd20 con 0x55edb09445c0 2024-03-18T20:57:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.834+0000 7fcd9e2c9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb081f9c0 con 0x55edb0930570 2024-03-18T20:57:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.834+0000 7fcd9e2c9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb0707640 con 0x55edb0930e90 2024-03-18T20:57:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd9e2c9b80 10 client.0 _ll_register_callbacks cb 0x55edb068b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:57:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb093dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb093dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:34008/0 (socket says 172.21.15.46:34008) 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8affd700 1 -- 172.21.15.46:0/3490458190 learned_addr learned my addr 172.21.15.46:0/3490458190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930570 0x55edb093da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8bfff700 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb09445c0 0x55edb09437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8b7fe700 1 -- 172.21.15.46:0/3490458190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 msgr2=0x55edb093dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb093dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8b7fe700 1 -- 172.21.15.46:0/3490458190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb09445c0 msgr2=0x55edb09437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb09445c0 0x55edb09437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8b7fe700 1 -- 172.21.15.46:0/3490458190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd80017070 con 0x55edb0930570 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8affd700 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb093dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8bfff700 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb09445c0 0x55edb09437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930570 0x55edb093da80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0ca590 tx=0x7fcd7c099930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a90681b73068420 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.835+0000 7fcd6affd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:57:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.836+0000 7fcd6affd700 1 -- 172.21.15.46:0/3490458190 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0b1070 con 0x55edb0930570 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.836+0000 7fcd6affd700 1 -- 172.21.15.46:0/3490458190 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fcd7c09dd80 con 0x55edb0930570 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.836+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3490458190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edb09246c0 con 0x55edb0930570 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.836+0000 7fcd6affd700 1 -- 172.21.15.46:0/3490458190 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd7c0a23d0 con 0x55edb0930570 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.836+0000 7fcd9e2c9b80 10 client.30236 fetch_fsmap learned FSMap version 89 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.836+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3490458190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x7fcd8c110620 con 0x55edb0930570 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.836+0000 7fcd6affd700 1 -- 172.21.15.46:0/3490458190 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55edb09246c0 con 0x55edb0930570 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.837+0000 7fcd6affd700 1 -- 172.21.15.46:0/3490458190 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 89) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fcd7c0a1fd0 con 0x55edb0930570 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.837+0000 7fcd9e2c9b80 10 client.30236 fetch_fsmap finished waiting for FSMap version 89 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.837+0000 7fcd9e2c9b80 10 client.30236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.837+0000 7fcd9e2c9b80 20 client.30236 populate_metadata read hostname 'smithi046' 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.837+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edb095c9e0 0x55edb095ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.837+0000 7fcd9e2c9b80 4 client.30236 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.837+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3490458190 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edb087c6a0 con 0x55edb095c9e0 2024-03-18T20:57:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.837+0000 7fcd8affd700 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edb095c9e0 0x55edb095ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.838+0000 7fcd8affd700 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edb095c9e0 0x55edb095ee00 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87ea4806ac967c56 server_cookie=ae07f54c60d5cd51 in_seq=0 out_seq=0 2024-03-18T20:57:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.838+0000 7fcd6affd700 10 client.30236 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.840+0000 7fcd6affd700 1 -- 172.21.15.46:0/3490458190 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55edb087c6a0 con 0x55edb095c9e0 2024-03-18T20:57:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.840+0000 7fcd6affd700 10 client.30236 handle_command_reply: tid=0 2024-03-18T20:57:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.840+0000 7fcd9e2c9b80 1 client.30236 shutdown 2024-03-18T20:57:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.841+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3490458190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930570 msgr2=0x55edb093da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.841+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930570 0x55edb093da80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0ca590 tx=0x7fcd7c099930 comp rx=0 tx=0).stop 2024-03-18T20:57:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.841+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3490458190 shutdown_connections 2024-03-18T20:57:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.841+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edb095c9e0 0x55edb095ee00 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.841+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb09445c0 0x55edb09437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.841+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930570 0x55edb093da80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.841+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/3490458190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb093dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.841+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3490458190 >> 172.21.15.46:0/3490458190 conn(0x55edb08a71c0 msgr2=0x55edb092c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.841+0000 7fcd6affd700 0 client.30236 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.841+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3490458190 shutdown_connections 2024-03-18T20:57:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.841+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/3490458190 wait complete. 2024-03-18T20:57:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.841+0000 7fcd9e2c9b80 20 client.30236 trim_cache size 0 max 16384 2024-03-18T20:57:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.865+0000 7fcd9e2c9b80 1 Processor -- start 2024-03-18T20:57:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.865+0000 7fcd9e2c9b80 1 -- start start 2024-03-18T20:57:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.865+0000 7fcd9e2c9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb0930570 0x55edb09436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.865+0000 7fcd9e2c9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb0937640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.865+0000 7fcd9e2c9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb09445c0 0x55edb0937b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.865+0000 7fcd9e2c9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb09291d0 con 0x55edb0930570 2024-03-18T20:57:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.865+0000 7fcd9e2c9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb092df00 con 0x55edb09445c0 2024-03-18T20:57:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.865+0000 7fcd9e2c9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb09380c0 con 0x55edb0930e90 2024-03-18T20:57:12.868 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.866+0000 7fcd8affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb0937640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.866+0000 7fcd8affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb0937640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:34024/0 (socket says 172.21.15.46:34024) 2024-03-18T20:57:12.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.866+0000 7fcd8affd700 1 -- 172.21.15.46:0/1470528076 learned_addr learned my addr 172.21.15.46:0/1470528076 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.866+0000 7fcd8bfff700 1 --2- 172.21.15.46:0/1470528076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb09445c0 0x55edb0937b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.866+0000 7fcd8affd700 1 -- 172.21.15.46:0/1470528076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb09445c0 msgr2=0x55edb0937b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.866+0000 7fcd8affd700 1 --2- 172.21.15.46:0/1470528076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb09445c0 0x55edb0937b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.866+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/1470528076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb0930570 0x55edb09436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.866+0000 7fcd8affd700 1 -- 172.21.15.46:0/1470528076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb0930570 msgr2=0x55edb09436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.866+0000 7fcd8affd700 1 --2- 172.21.15.46:0/1470528076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb0930570 0x55edb09436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.866+0000 7fcd8affd700 1 -- 172.21.15.46:0/1470528076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edb06b1b00 con 0x55edb0930e90 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.866+0000 7fcd8bfff700 1 --2- 172.21.15.46:0/1470528076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb09445c0 0x55edb0937b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.866+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/1470528076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb0930570 0x55edb09436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.867+0000 7fcd8affd700 1 --2- 172.21.15.46:0/1470528076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb0937640 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcd800117d0 tx=0x7fcd80014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3270145deee3a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.867+0000 7fcd727fc700 1 -- 172.21.15.46:0/1470528076 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80003e70 con 0x55edb0930e90 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.867+0000 7fcd727fc700 1 -- 172.21.15.46:0/1470528076 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd80023040 con 0x55edb0930e90 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.867+0000 7fcd727fc700 1 -- 172.21.15.46:0/1470528076 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8001db80 con 0x55edb0930e90 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.868+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/1470528076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 msgr2=0x55edb0937640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.868+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/1470528076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb0937640 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcd800117d0 tx=0x7fcd80014000 comp rx=0 tx=0).stop 2024-03-18T20:57:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.868+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/1470528076 shutdown_connections 2024-03-18T20:57:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.868+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/1470528076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb0930570 0x55edb09436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.868+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/1470528076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb09445c0 0x55edb0937b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.868+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/1470528076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb0937640 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.868+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/1470528076 >> 172.21.15.46:0/1470528076 conn(0x55edb08a71c0 msgr2=0x55edb092b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.868+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/1470528076 shutdown_connections 2024-03-18T20:57:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.868+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/1470528076 wait complete. 2024-03-18T20:57:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.868+0000 7fcd9e2c9b80 1 -- start start 2024-03-18T20:57:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.869+0000 7fcd9e2c9b80 1 Processor -- start 2024-03-18T20:57:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.869+0000 7fcd9e2c9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930570 0x55edb093ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.869+0000 7fcd9e2c9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb093f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.869+0000 7fcd9e2c9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb09445c0 0x55edb093f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.869+0000 7fcd9e2c9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb09af800 con 0x55edb09445c0 2024-03-18T20:57:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.869+0000 7fcd9e2c9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb09af940 con 0x55edb0930570 2024-03-18T20:57:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.869+0000 7fcd9e2c9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edb09c0a90 con 0x55edb0930e90 2024-03-18T20:57:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.869+0000 7fcd9e2c9b80 10 client.0 _ll_register_callbacks cb 0x7fcd7c096380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:57:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.869+0000 7fcd8b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930570 0x55edb093ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.869+0000 7fcd8b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930570 0x55edb093ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:39676/0 (socket says 172.21.15.46:39676) 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.869+0000 7fcd8b7fe700 1 -- 172.21.15.46:0/2424917871 learned_addr learned my addr 172.21.15.46:0/2424917871 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.870+0000 7fcd8affd700 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb093f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.870+0000 7fcd8bfff700 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb09445c0 0x55edb093f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.871+0000 7fcd8affd700 1 -- 172.21.15.46:0/2424917871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930570 msgr2=0x55edb093ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.871+0000 7fcd8affd700 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930570 0x55edb093ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.871+0000 7fcd8affd700 1 -- 172.21.15.46:0/2424917871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb09445c0 msgr2=0x55edb093f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.871+0000 7fcd8affd700 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb09445c0 0x55edb093f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.871+0000 7fcd8affd700 1 -- 172.21.15.46:0/2424917871 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd80004550 con 0x55edb0930e90 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.871+0000 7fcd8affd700 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb093f280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55edb0926ee0 tx=0x7fcd800044a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=795f10383ac95e93 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.871+0000 7fcd6affd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.871+0000 7fcd6affd700 1 -- 172.21.15.46:0/2424917871 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80029980 con 0x55edb0930e90 2024-03-18T20:57:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.871+0000 7fcd6affd700 1 -- 172.21.15.46:0/2424917871 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fcd80029ae0 con 0x55edb0930e90 2024-03-18T20:57:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.872+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/2424917871 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd84006000 con 0x55edb0930e90 2024-03-18T20:57:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.872+0000 7fcd6affd700 1 -- 172.21.15.46:0/2424917871 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd80029ca0 con 0x55edb0930e90 2024-03-18T20:57:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.872+0000 7fcd9e2c9b80 10 client.30349 fetch_fsmap learned FSMap version 89 2024-03-18T20:57:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.872+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/2424917871 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x55edb0849e40 con 0x55edb0930e90 2024-03-18T20:57:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.884+0000 7fcd6affd700 1 -- 172.21.15.46:0/2424917871 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd8003f030 con 0x55edb0930e90 2024-03-18T20:57:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.884+0000 7fcd6affd700 1 -- 172.21.15.46:0/2424917871 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 89) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fcd80004a60 con 0x55edb0930e90 2024-03-18T20:57:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.889+0000 7fcd9e2c9b80 10 client.30349 fetch_fsmap finished waiting for FSMap version 89 2024-03-18T20:57:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.889+0000 7fcd9e2c9b80 10 client.30349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:57:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.889+0000 7fcd9e2c9b80 20 client.30349 populate_metadata read hostname 'smithi046' 2024-03-18T20:57:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.889+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edb09c21c0 0x55edb09c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.889+0000 7fcd9e2c9b80 4 client.30349 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:57:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.889+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/2424917871 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55edb06d2c90 con 0x55edb09c21c0 2024-03-18T20:57:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.909+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edb09c21c0 0x55edb09c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.947+0000 7fcd6affd700 10 client.30349 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:12.947+0000 7fcd8b7fe700 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edb09c21c0 0x55edb09c45e0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3abc5ce15dfafd3b server_cookie=6d9c399920a3a9e7 in_seq=0 out_seq=0 2024-03-18T20:57:13.441 INFO:tasks.workunit.client.0.smithi046.stdout: 10 330039 34.05 MB/sec execute 349 sec latency 8651.392 ms 2024-03-18T20:57:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.653+0000 7fcd6affd700 1 -- 172.21.15.46:0/2424917871 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55edb06d2c90 con 0x55edb09c21c0 2024-03-18T20:57:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.653+0000 7fcd6affd700 10 client.30349 handle_command_reply: tid=0 2024-03-18T20:57:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.653+0000 7fcd9e2c9b80 1 client.30349 shutdown 2024-03-18T20:57:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.653+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/2424917871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 msgr2=0x55edb093f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.653+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb093f280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55edb0926ee0 tx=0x7fcd800044a0 comp rx=0 tx=0).stop 2024-03-18T20:57:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.653+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/2424917871 shutdown_connections 2024-03-18T20:57:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.653+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55edb09c21c0 0x55edb09c45e0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.654+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55edb09445c0 0x55edb093f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.654+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55edb0930570 0x55edb093ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.654+0000 7fcd9e2c9b80 1 --2- 172.21.15.46:0/2424917871 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55edb0930e90 0x55edb093f280 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.654+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/2424917871 >> 172.21.15.46:0/2424917871 conn(0x55edb08a71c0 msgr2=0x55edb0934050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.654+0000 7fcd6affd700 0 client.30349 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.654+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/2424917871 shutdown_connections 2024-03-18T20:57:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.654+0000 7fcd9e2c9b80 1 -- 172.21.15.46:0/2424917871 wait complete. 2024-03-18T20:57:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.654+0000 7fcd9e2c9b80 20 client.30349 trim_cache size 0 max 16384 2024-03-18T20:57:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:57:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:57:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:57:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:57:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.655+0000 7fcd92134700 1 -- 172.21.15.46:0/3445127725 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcd740a1290 msgr2=0x7fcd740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.655+0000 7fcd92134700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcd740a1290 0x7fcd740a3750 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c19a9d0 tx=0x7fcd80009260 comp rx=0 tx=0).stop 2024-03-18T20:57:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.655+0000 7fcd92134700 1 -- 172.21.15.46:0/3445127725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd8c069970 msgr2=0x7fcd8c199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.655+0000 7fcd92134700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd8c069970 0x7fcd8c199480 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00e9a0 tx=0x7fcd7c00ecb0 comp rx=0 tx=0).stop 2024-03-18T20:57:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.655+0000 7fcd92134700 1 -- 172.21.15.46:0/3445127725 shutdown_connections 2024-03-18T20:57:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.655+0000 7fcd92134700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fcd740a1290 0x7fcd740a3750 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.655+0000 7fcd92134700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd8c10d590 0x7fcd8c19dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.655+0000 7fcd92134700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd8c069970 0x7fcd8c199480 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.655+0000 7fcd92134700 1 --2- 172.21.15.46:0/3445127725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8c101bc0 0x7fcd8c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.655+0000 7fcd92134700 1 -- 172.21.15.46:0/3445127725 >> 172.21.15.46:0/3445127725 conn(0x7fcd8c076a80 msgr2=0x7fcd8c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.655+0000 7fcd92134700 1 -- 172.21.15.46:0/3445127725 shutdown_connections 2024-03-18T20:57:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:13.655+0000 7fcd92134700 1 -- 172.21.15.46:0/3445127725 wait complete. 2024-03-18T20:57:14.441 INFO:tasks.workunit.client.0.smithi046.stdout: 10 330405 34.01 MB/sec execute 350 sec latency 9651.434 ms 2024-03-18T20:57:14.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:14 smithi155 ceph-mon[48344]: pgmap v667: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 689 op/s 2024-03-18T20:57:14.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:14 smithi046 ceph-mon[43495]: pgmap v667: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 689 op/s 2024-03-18T20:57:14.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:14 smithi073 ceph-mon[47384]: pgmap v667: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 14 MiB/s wr, 689 op/s 2024-03-18T20:57:15.441 INFO:tasks.workunit.client.0.smithi046.stdout: 10 330893 34.00 MB/sec execute 351 sec latency 10651.483 ms 2024-03-18T20:57:16.441 INFO:tasks.workunit.client.0.smithi046.stdout: 10 331380 33.96 MB/sec execute 352 sec latency 10783.927 ms 2024-03-18T20:57:16.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:16 smithi155 ceph-mon[48344]: osdmap e113: 12 total, 12 up, 12 in 2024-03-18T20:57:16.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:16 smithi155 ceph-mon[48344]: pgmap v669: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 11 MiB/s wr, 556 op/s 2024-03-18T20:57:16.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:16 smithi046 ceph-mon[43495]: osdmap e113: 12 total, 12 up, 12 in 2024-03-18T20:57:16.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:16 smithi046 ceph-mon[43495]: pgmap v669: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 11 MiB/s wr, 556 op/s 2024-03-18T20:57:16.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:16 smithi073 ceph-mon[47384]: osdmap e113: 12 total, 12 up, 12 in 2024-03-18T20:57:16.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:16 smithi073 ceph-mon[47384]: pgmap v669: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 11 MiB/s wr, 556 op/s 2024-03-18T20:57:17.441 INFO:tasks.workunit.client.0.smithi046.stdout: 10 332025 33.97 MB/sec execute 353 sec latency 96.329 ms 2024-03-18T20:57:18.441 INFO:tasks.workunit.client.0.smithi046.stdout: 10 332580 33.96 MB/sec execute 354 sec latency 94.399 ms 2024-03-18T20:57:18.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:18 smithi073 ceph-mon[47384]: pgmap v670: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 11 MiB/s wr, 556 op/s 2024-03-18T20:57:18.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:18 smithi155 ceph-mon[48344]: pgmap v670: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 11 MiB/s wr, 556 op/s 2024-03-18T20:57:18.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:18 smithi046 ceph-mon[43495]: pgmap v670: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 11 MiB/s wr, 556 op/s 2024-03-18T20:57:19.445 INFO:tasks.workunit.client.0.smithi046.stdout: 10 333144 33.95 MB/sec execute 355 sec latency 98.333 ms 2024-03-18T20:57:20.443 INFO:tasks.workunit.client.0.smithi046.stdout: 10 333670 33.92 MB/sec execute 356 sec latency 85.171 ms 2024-03-18T20:57:20.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:20 smithi073 ceph-mon[47384]: pgmap v671: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 647 op/s 2024-03-18T20:57:20.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:20 smithi155 ceph-mon[48344]: pgmap v671: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 647 op/s 2024-03-18T20:57:20.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:20 smithi046 ceph-mon[43495]: pgmap v671: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 13 MiB/s wr, 647 op/s 2024-03-18T20:57:21.443 INFO:tasks.workunit.client.0.smithi046.stdout: 10 334190 33.90 MB/sec execute 357 sec latency 105.303 ms 2024-03-18T20:57:22.443 INFO:tasks.workunit.client.0.smithi046.stdout: 10 334794 33.90 MB/sec execute 358 sec latency 104.828 ms 2024-03-18T20:57:22.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:22 smithi073 ceph-mon[47384]: pgmap v672: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 643 op/s 2024-03-18T20:57:22.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:22 smithi155 ceph-mon[48344]: pgmap v672: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 643 op/s 2024-03-18T20:57:22.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:22 smithi046 ceph-mon[43495]: pgmap v672: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 643 op/s 2024-03-18T20:57:23.447 INFO:tasks.workunit.client.0.smithi046.stdout: 10 335526 33.92 MB/sec execute 359 sec latency 80.401 ms 2024-03-18T20:57:24.443 INFO:tasks.workunit.client.0.smithi046.stdout: 10 336081 33.94 MB/sec execute 360 sec latency 107.379 ms 2024-03-18T20:57:24.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:24 smithi073 ceph-mon[47384]: pgmap v673: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 656 op/s 2024-03-18T20:57:24.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:24 smithi155 ceph-mon[48344]: pgmap v673: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 656 op/s 2024-03-18T20:57:24.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:24 smithi046 ceph-mon[43495]: pgmap v673: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 656 op/s 2024-03-18T20:57:25.443 INFO:tasks.workunit.client.0.smithi046.stdout: 10 336642 33.91 MB/sec execute 361 sec latency 116.766 ms 2024-03-18T20:57:26.443 INFO:tasks.workunit.client.0.smithi046.stdout: 10 337207 33.89 MB/sec execute 362 sec latency 93.351 ms 2024-03-18T20:57:26.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:26 smithi155 ceph-mon[48344]: pgmap v674: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 542 op/s 2024-03-18T20:57:26.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:26 smithi046 ceph-mon[43495]: pgmap v674: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 542 op/s 2024-03-18T20:57:26.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:26 smithi073 ceph-mon[47384]: pgmap v674: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.3 MiB/s wr, 542 op/s 2024-03-18T20:57:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.883+0000 7fa29761d700 1 -- 172.21.15.46:0/1697578240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa290106e30 msgr2=0x7fa290107210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.883+0000 7fa29761d700 1 --2- 172.21.15.46:0/1697578240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa290106e30 0x7fa290107210 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009990 tx=0x7fa28c009ca0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.883+0000 7fa29761d700 1 -- 172.21.15.46:0/1697578240 shutdown_connections 2024-03-18T20:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.883+0000 7fa29761d700 1 --2- 172.21.15.46:0/1697578240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2900ffbe0 0x7fa29010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.883+0000 7fa29761d700 1 --2- 172.21.15.46:0/1697578240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa290106e30 0x7fa290107210 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.883+0000 7fa29761d700 1 --2- 172.21.15.46:0/1697578240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2900ff220 0x7fa2900ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.883+0000 7fa29761d700 1 -- 172.21.15.46:0/1697578240 >> 172.21.15.46:0/1697578240 conn(0x7fa2900761b0 msgr2=0x7fa2900fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.883+0000 7fa29761d700 1 -- 172.21.15.46:0/1697578240 shutdown_connections 2024-03-18T20:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.883+0000 7fa29761d700 1 -- 172.21.15.46:0/1697578240 wait complete. 2024-03-18T20:57:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.884+0000 7fa29761d700 1 Processor -- start 2024-03-18T20:57:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.884+0000 7fa29761d700 1 -- start start 2024-03-18T20:57:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.884+0000 7fa29761d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2900ff220 0x7fa290195130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.884+0000 7fa29761d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2900ffbe0 0x7fa290195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.884+0000 7fa29761d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa290106e30 0x7fa290199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.884+0000 7fa29761d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29019a050 con 0x7fa290106e30 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.884+0000 7fa29761d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29019a1c0 con 0x7fa2900ff220 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.884+0000 7fa29761d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29019a380 con 0x7fa2900ffbe0 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.884+0000 7fa295bba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa290106e30 0x7fa290199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.884+0000 7fa2953b9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2900ff220 0x7fa290195130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.885+0000 7fa295bba700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa290106e30 0x7fa290199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60734/0 (socket says 172.21.15.46:60734) 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.885+0000 7fa295bba700 1 -- 172.21.15.46:0/972478332 learned_addr learned my addr 172.21.15.46:0/972478332 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.885+0000 7fa294bb8700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2900ffbe0 0x7fa290195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.885+0000 7fa295bba700 1 -- 172.21.15.46:0/972478332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2900ffbe0 msgr2=0x7fa290195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.885+0000 7fa295bba700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2900ffbe0 0x7fa290195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.885+0000 7fa295bba700 1 -- 172.21.15.46:0/972478332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2900ff220 msgr2=0x7fa290195130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.885+0000 7fa295bba700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2900ff220 0x7fa290195130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.885+0000 7fa295bba700 1 -- 172.21.15.46:0/972478332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c0095f0 con 0x7fa290106e30 2024-03-18T20:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.885+0000 7fa2953b9700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2900ff220 0x7fa290195130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.885+0000 7fa295bba700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa290106e30 0x7fa290199a10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fa28800b9e0 tx=0x7fa28800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.885+0000 7fa2867fc700 1 -- 172.21.15.46:0/972478332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28800d820 con 0x7fa290106e30 2024-03-18T20:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.886+0000 7fa2867fc700 1 -- 172.21.15.46:0/972478332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa28800de40 con 0x7fa290106e30 2024-03-18T20:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.886+0000 7fa29761d700 1 -- 172.21.15.46:0/972478332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29019a660 con 0x7fa290106e30 2024-03-18T20:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.886+0000 7fa29761d700 1 -- 172.21.15.46:0/972478332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2901a1ea0 con 0x7fa290106e30 2024-03-18T20:57:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.888+0000 7fa2867fc700 1 -- 172.21.15.46:0/972478332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2880143d0 con 0x7fa290106e30 2024-03-18T20:57:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.891+0000 7fa2867fc700 1 -- 172.21.15.46:0/972478332 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa288014530 con 0x7fa290106e30 2024-03-18T20:57:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.891+0000 7fa2a37b2b80 1 Processor -- start 2024-03-18T20:57:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.891+0000 7fa2a37b2b80 1 -- start start 2024-03-18T20:57:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.891+0000 7fa2a37b2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f55ed80 0x55988f55f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.891+0000 7fa2a37b2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55f6a0 0x55988f55c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.891+0000 7fa2a37b2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f572fc0 0x55988f5753b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.892+0000 7fa2a37b2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988f4b0930 con 0x55988f55f6a0 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.892+0000 7fa2a37b2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988f4ae2f0 con 0x55988f572fc0 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.892+0000 7fa2a37b2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988f4a9be0 con 0x55988f55ed80 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.892+0000 7fa2867fc700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa27c0a1460 0x7fa27c0a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.892+0000 7fa294bb8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f572fc0 0x55988f5753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.892+0000 7fa295bba700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa27c0a1460 0x7fa27c0a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.892+0000 7fa294bb8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f572fc0 0x55988f5753b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:42622/0 (socket says 172.21.15.46:42622) 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.892+0000 7fa294bb8700 1 -- 172.21.15.46:0/2957412657 learned_addr learned my addr 172.21.15.46:0/2957412657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.892+0000 7fa294bb8700 1 --2- 172.21.15.46:0/2957412657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f55ed80 0x55988f55f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.892+0000 7fa2867fc700 1 -- 172.21.15.46:0/972478332 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa288011070 con 0x7fa290106e30 2024-03-18T20:57:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.893+0000 7fa295bba700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa27c0a1460 0x7fa27c0a3920 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa2880a2420 tx=0x7fa2880ba040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.893+0000 7fa294bb8700 1 -- 172.21.15.46:0/2957412657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f55ed80 msgr2=0x55988f55f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.893+0000 7fa294bb8700 1 --2- 172.21.15.46:0/2957412657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f55ed80 0x55988f55f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.893+0000 7fa294bb8700 1 -- 172.21.15.46:0/2957412657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55f6a0 msgr2=0x55988f55c550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:57:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.893+0000 7fa294bb8700 1 --2- 172.21.15.46:0/2957412657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55f6a0 0x55988f55c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.893+0000 7fa294bb8700 1 -- 172.21.15.46:0/2957412657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55988f32b350 con 0x55988f572fc0 2024-03-18T20:57:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.893+0000 7fa294bb8700 1 --2- 172.21.15.46:0/2957412657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f572fc0 0x55988f5753b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa280039610 tx=0x7fa2800095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ce2855f43bcd893 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.894+0000 7fa27b7fe700 1 -- 172.21.15.46:0/2957412657 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280039c30 con 0x55988f572fc0 2024-03-18T20:57:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.894+0000 7fa27b7fe700 1 -- 172.21.15.46:0/2957412657 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa28003d070 con 0x55988f572fc0 2024-03-18T20:57:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.894+0000 7fa27b7fe700 1 -- 172.21.15.46:0/2957412657 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2800373d0 con 0x55988f572fc0 2024-03-18T20:57:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.895+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2957412657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f572fc0 msgr2=0x55988f5753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.895+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/2957412657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f572fc0 0x55988f5753b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa280039610 tx=0x7fa2800095a0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.895+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2957412657 shutdown_connections 2024-03-18T20:57:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.895+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/2957412657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f572fc0 0x55988f5753b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.895+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/2957412657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f55ed80 0x55988f55f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.895+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/2957412657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55f6a0 0x55988f55c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.895+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2957412657 >> 172.21.15.46:0/2957412657 conn(0x55988f4cd1c0 msgr2=0x55988f5529b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.897+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2957412657 shutdown_connections 2024-03-18T20:57:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.899+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2957412657 wait complete. 2024-03-18T20:57:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.899+0000 7fa2a37b2b80 1 -- start start 2024-03-18T20:57:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.899+0000 7fa2a37b2b80 1 Processor -- start 2024-03-18T20:57:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.899+0000 7fa2a37b2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f55ed80 0x55988f562160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa2a37b2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55f6a0 0x55988f5626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa2a37b2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f572fc0 0x55988f56d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa2a37b2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988f445d20 con 0x55988f55f6a0 2024-03-18T20:57:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa2a37b2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988f4459c0 con 0x55988f55ed80 2024-03-18T20:57:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa2a37b2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988f32d640 con 0x55988f572fc0 2024-03-18T20:57:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa2a37b2b80 10 client.0 _ll_register_callbacks cb 0x55988f2b1340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:57:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa294bb8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f55ed80 0x55988f562160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa294bb8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f55ed80 0x55988f562160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:42624/0 (socket says 172.21.15.46:42624) 2024-03-18T20:57:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa294bb8700 1 -- 172.21.15.46:0/2624384925 learned_addr learned my addr 172.21.15.46:0/2624384925 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa294bb8700 1 -- 172.21.15.46:0/2624384925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f572fc0 msgr2=0x55988f56d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa294bb8700 1 --2- 172.21.15.46:0/2624384925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f572fc0 0x55988f56d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa294bb8700 1 -- 172.21.15.46:0/2624384925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55f6a0 msgr2=0x55988f5626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa294bb8700 1 --2- 172.21.15.46:0/2624384925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55f6a0 0x55988f5626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.900+0000 7fa294bb8700 1 -- 172.21.15.46:0/2624384925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28002f040 con 0x55988f55ed80 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.901+0000 7fa294bb8700 1 --2- 172.21.15.46:0/2624384925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f55ed80 0x55988f562160 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa280009370 tx=0x7fa280009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5474a44019f9e81 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.901+0000 7fa278ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.901+0000 7fa278ff9700 1 -- 172.21.15.46:0/2624384925 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280039e10 con 0x55988f55ed80 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.901+0000 7fa278ff9700 1 -- 172.21.15.46:0/2624384925 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa280000e40 con 0x55988f55ed80 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.901+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2624384925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55988f54a6c0 con 0x55988f55ed80 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.901+0000 7fa278ff9700 1 -- 172.21.15.46:0/2624384925 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa280039030 con 0x55988f55ed80 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.902+0000 7fa2a37b2b80 10 client.30248 fetch_fsmap learned FSMap version 89 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.902+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2624384925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x55988f2d7b00 con 0x55988f55ed80 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.903+0000 7fa278ff9700 1 -- 172.21.15.46:0/2624384925 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55988f54a6c0 con 0x55988f55ed80 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.903+0000 7fa278ff9700 1 -- 172.21.15.46:0/2624384925 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 89) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa28004b860 con 0x55988f55ed80 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.903+0000 7fa2a37b2b80 10 client.30248 fetch_fsmap finished waiting for FSMap version 89 2024-03-18T20:57:26.907 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.903+0000 7fa2a37b2b80 10 client.30248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:57:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.903+0000 7fa2a37b2b80 20 client.30248 populate_metadata read hostname 'smithi046' 2024-03-18T20:57:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.903+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/2624384925 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55988f5714f0 0x55988f571900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.903+0000 7fa2a37b2b80 4 client.30248 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:57:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.903+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2624384925 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55988f4a26a0 con 0x55988f5714f0 2024-03-18T20:57:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.904+0000 7fa2953b9700 1 --2- 172.21.15.46:0/2624384925 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55988f5714f0 0x55988f571900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.907+0000 7fa2953b9700 1 --2- 172.21.15.46:0/2624384925 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55988f5714f0 0x55988f571900 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=699a0f41dc58e3a5 server_cookie=311c4361a8a7145b in_seq=0 out_seq=0 2024-03-18T20:57:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.908+0000 7fa278ff9700 10 client.30248 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.908+0000 7fa278ff9700 1 -- 172.21.15.46:0/2624384925 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55988f4a26a0 con 0x55988f5714f0 2024-03-18T20:57:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.908+0000 7fa278ff9700 10 client.30248 handle_command_reply: tid=0 2024-03-18T20:57:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.908+0000 7fa2a37b2b80 1 client.30248 shutdown 2024-03-18T20:57:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.917+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2624384925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f55ed80 msgr2=0x55988f562160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.917+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/2624384925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f55ed80 0x55988f562160 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa280009370 tx=0x7fa280009b10 comp rx=0 tx=0).stop 2024-03-18T20:57:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.917+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2624384925 shutdown_connections 2024-03-18T20:57:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.917+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/2624384925 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55988f5714f0 0x55988f571900 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.917+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/2624384925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f572fc0 0x55988f56d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.917+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/2624384925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f55ed80 0x55988f562160 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.917+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/2624384925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55f6a0 0x55988f5626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.917+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2624384925 >> 172.21.15.46:0/2624384925 conn(0x55988f4cd1c0 msgr2=0x55988f5525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.918+0000 7fa278ff9700 0 client.30248 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.918+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2624384925 shutdown_connections 2024-03-18T20:57:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.918+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/2624384925 wait complete. 2024-03-18T20:57:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.920+0000 7fa2a37b2b80 20 client.30248 trim_cache size 0 max 16384 2024-03-18T20:57:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.957+0000 7fa2a37b2b80 1 Processor -- start 2024-03-18T20:57:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.968+0000 7fa2a37b2b80 1 -- start start 2024-03-18T20:57:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.968+0000 7fa2a37b2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55ed80 0x55988f55c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.968+0000 7fa2a37b2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f55f6a0 0x55988f5793f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.968+0000 7fa2a37b2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f572fc0 0x55988f5714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.968+0000 7fa2a37b2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988f579980 con 0x55988f55ed80 2024-03-18T20:57:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.968+0000 7fa2a37b2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988f562eb0 con 0x55988f55f6a0 2024-03-18T20:57:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.968+0000 7fa2a37b2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988f557b40 con 0x55988f572fc0 2024-03-18T20:57:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.969+0000 7fa294bb8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55ed80 0x55988f55c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.969+0000 7fa294bb8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55ed80 0x55988f55c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60752/0 (socket says 172.21.15.46:60752) 2024-03-18T20:57:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.969+0000 7fa294bb8700 1 -- 172.21.15.46:0/1564931383 learned_addr learned my addr 172.21.15.46:0/1564931383 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.969+0000 7fa294bb8700 1 -- 172.21.15.46:0/1564931383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f572fc0 msgr2=0x55988f5714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.969+0000 7fa294bb8700 1 --2- 172.21.15.46:0/1564931383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f572fc0 0x55988f5714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.969+0000 7fa294bb8700 1 -- 172.21.15.46:0/1564931383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f55f6a0 msgr2=0x55988f5793f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:57:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.969+0000 7fa294bb8700 1 --2- 172.21.15.46:0/1564931383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f55f6a0 0x55988f5793f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.969+0000 7fa294bb8700 1 -- 172.21.15.46:0/1564931383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55988f46fe40 con 0x55988f55ed80 2024-03-18T20:57:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.970+0000 7fa294bb8700 1 --2- 172.21.15.46:0/1564931383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55ed80 0x55988f55c540 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fa280039610 tx=0x7fa2800046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9dc5bc9fcaeb8d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.974+0000 7fa27bfff700 1 -- 172.21.15.46:0/1564931383 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28004cea0 con 0x55988f55ed80 2024-03-18T20:57:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.974+0000 7fa27bfff700 1 -- 172.21.15.46:0/1564931383 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2800094d0 con 0x55988f55ed80 2024-03-18T20:57:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.974+0000 7fa27bfff700 1 -- 172.21.15.46:0/1564931383 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28004b7b0 con 0x55988f55ed80 2024-03-18T20:57:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.975+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/1564931383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55ed80 msgr2=0x55988f55c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.975+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/1564931383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55ed80 0x55988f55c540 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fa280039610 tx=0x7fa2800046d0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.976+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/1564931383 shutdown_connections 2024-03-18T20:57:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.976+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/1564931383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55ed80 0x55988f55c540 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.976+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/1564931383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f572fc0 0x55988f5714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.976+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/1564931383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f55f6a0 0x55988f5793f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.976+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/1564931383 >> 172.21.15.46:0/1564931383 conn(0x55988f4cd1c0 msgr2=0x55988f5524b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.982+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/1564931383 shutdown_connections 2024-03-18T20:57:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.996+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/1564931383 wait complete. 2024-03-18T20:57:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.996+0000 7fa2a37b2b80 1 -- start start 2024-03-18T20:57:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.996+0000 7fa2a37b2b80 1 Processor -- start 2024-03-18T20:57:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.997+0000 7fa2a37b2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f55ed80 0x55988f5c14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.997+0000 7fa2a37b2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55f6a0 0x55988f5c19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.997+0000 7fa2a37b2b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f572fc0 0x55988f5bf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.997+0000 7fa2a37b2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988f5c20f0 con 0x55988f55f6a0 2024-03-18T20:57:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.997+0000 7fa2a37b2b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988f5c2260 con 0x55988f572fc0 2024-03-18T20:57:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.997+0000 7fa2a37b2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55988f5bf9c0 con 0x55988f55ed80 2024-03-18T20:57:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:26.997+0000 7fa2a37b2b80 10 client.0 _ll_register_callbacks cb 0x55988f2b1340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:57:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.003+0000 7fa294bb8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f55ed80 0x55988f5c14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.003+0000 7fa294bb8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f55ed80 0x55988f5c14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43512/0 (socket says 172.21.15.46:43512) 2024-03-18T20:57:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.003+0000 7fa294bb8700 1 -- 172.21.15.46:0/614952213 learned_addr learned my addr 172.21.15.46:0/614952213 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.007+0000 7fa294bb8700 1 --2- 172.21.15.46:0/614952213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f572fc0 0x55988f5bf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.007+0000 7fa294bb8700 1 -- 172.21.15.46:0/614952213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f572fc0 msgr2=0x55988f5bf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.007+0000 7fa294bb8700 1 --2- 172.21.15.46:0/614952213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f572fc0 0x55988f5bf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.007+0000 7fa294bb8700 1 -- 172.21.15.46:0/614952213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55f6a0 msgr2=0x55988f5c19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.007+0000 7fa294bb8700 1 --2- 172.21.15.46:0/614952213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55f6a0 0x55988f5c19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.007+0000 7fa294bb8700 1 -- 172.21.15.46:0/614952213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55988f2d7b00 con 0x55988f55ed80 2024-03-18T20:57:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.022+0000 7fa294bb8700 1 --2- 172.21.15.46:0/614952213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f55ed80 0x55988f5c14a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa2800358a0 tx=0x7fa28004e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=271a08ce2008a0f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.022+0000 7fa278ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:57:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.032+0000 7fa278ff9700 1 -- 172.21.15.46:0/614952213 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28004b890 con 0x55988f55ed80 2024-03-18T20:57:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.032+0000 7fa278ff9700 1 -- 172.21.15.46:0/614952213 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa280000e40 con 0x55988f55ed80 2024-03-18T20:57:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.034+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/614952213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa28003d070 con 0x55988f55ed80 2024-03-18T20:57:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.038+0000 7fa278ff9700 1 -- 172.21.15.46:0/614952213 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa28004b9f0 con 0x55988f55ed80 2024-03-18T20:57:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.038+0000 7fa2a37b2b80 10 client.30361 fetch_fsmap learned FSMap version 89 2024-03-18T20:57:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.038+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/614952213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) v3 -- 0x55988f32fc70 con 0x55988f55ed80 2024-03-18T20:57:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.040+0000 7fa278ff9700 1 -- 172.21.15.46:0/614952213 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa28003d070 con 0x55988f55ed80 2024-03-18T20:57:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.040+0000 7fa278ff9700 1 -- 172.21.15.46:0/614952213 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 89) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa280039640 con 0x55988f55ed80 2024-03-18T20:57:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.040+0000 7fa2a37b2b80 10 client.30361 fetch_fsmap finished waiting for FSMap version 89 2024-03-18T20:57:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.040+0000 7fa2a37b2b80 10 client.30361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:57:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.040+0000 7fa2a37b2b80 20 client.30361 populate_metadata read hostname 'smithi046' 2024-03-18T20:57:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.040+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/614952213 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55988f578b90 0x55988f578fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.040+0000 7fa2a37b2b80 4 client.30361 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:57:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.040+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/614952213 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55988f2f8c90 con 0x55988f578b90 2024-03-18T20:57:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.050+0000 7fa2953b9700 1 --2- 172.21.15.46:0/614952213 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55988f578b90 0x55988f578fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.093+0000 7fa2953b9700 1 --2- 172.21.15.46:0/614952213 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55988f578b90 0x55988f578fa0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef9b8b58db489aa4 server_cookie=9ec077bd60eb03a9 in_seq=0 out_seq=0 2024-03-18T20:57:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:27.093+0000 7fa278ff9700 10 client.30361 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:27.445 INFO:tasks.workunit.client.0.smithi046.stdout: 10 337472 33.84 MB/sec execute 363 sec latency 551.949 ms 2024-03-18T20:57:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.078+0000 7fa278ff9700 1 -- 172.21.15.46:0/614952213 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55988f2f8c90 con 0x55988f578b90 2024-03-18T20:57:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.078+0000 7fa278ff9700 10 client.30361 handle_command_reply: tid=0 2024-03-18T20:57:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.078+0000 7fa2a37b2b80 1 client.30361 shutdown 2024-03-18T20:57:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.078+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/614952213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f55ed80 msgr2=0x55988f5c14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.078+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/614952213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f55ed80 0x55988f5c14a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa2800358a0 tx=0x7fa28004e040 comp rx=0 tx=0).stop 2024-03-18T20:57:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.078+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/614952213 shutdown_connections 2024-03-18T20:57:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.078+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/614952213 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55988f578b90 0x55988f578fa0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.078+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/614952213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55988f572fc0 0x55988f5bf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.078+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/614952213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55988f55ed80 0x55988f5c14a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.078+0000 7fa2a37b2b80 1 --2- 172.21.15.46:0/614952213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55988f55f6a0 0x55988f5c19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.078+0000 7fa278ff9700 0 client.30361 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.078+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/614952213 >> 172.21.15.46:0/614952213 conn(0x55988f4cd1c0 msgr2=0x55988f5714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.079+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/614952213 shutdown_connections 2024-03-18T20:57:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.079+0000 7fa2a37b2b80 1 -- 172.21.15.46:0/614952213 wait complete. 2024-03-18T20:57:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.079+0000 7fa2a37b2b80 20 client.30361 trim_cache size 0 max 16384 2024-03-18T20:57:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:57:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:57:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:57:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:57:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.080+0000 7fa29761d700 1 -- 172.21.15.46:0/972478332 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa27c0a1460 msgr2=0x7fa27c0a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.080+0000 7fa29761d700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa27c0a1460 0x7fa27c0a3920 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa2880a2420 tx=0x7fa2880ba040 comp rx=0 tx=0).stop 2024-03-18T20:57:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.080+0000 7fa29761d700 1 -- 172.21.15.46:0/972478332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa290106e30 msgr2=0x7fa290199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.080+0000 7fa29761d700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa290106e30 0x7fa290199a10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fa28800b9e0 tx=0x7fa28800bcf0 comp rx=0 tx=0).stop 2024-03-18T20:57:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.081+0000 7fa29761d700 1 -- 172.21.15.46:0/972478332 shutdown_connections 2024-03-18T20:57:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.081+0000 7fa29761d700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa27c0a1460 0x7fa27c0a3920 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.081+0000 7fa29761d700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa290106e30 0x7fa290199a10 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.081+0000 7fa29761d700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2900ff220 0x7fa290195130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.081+0000 7fa29761d700 1 --2- 172.21.15.46:0/972478332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2900ffbe0 0x7fa290195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.081+0000 7fa29761d700 1 -- 172.21.15.46:0/972478332 >> 172.21.15.46:0/972478332 conn(0x7fa2900761b0 msgr2=0x7fa290103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.081+0000 7fa29761d700 1 -- 172.21.15.46:0/972478332 shutdown_connections 2024-03-18T20:57:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:28.081+0000 7fa29761d700 1 -- 172.21.15.46:0/972478332 wait complete. 2024-03-18T20:57:28.445 INFO:tasks.workunit.client.0.smithi046.stdout: 10 337684 33.77 MB/sec execute 364 sec latency 851.320 ms 2024-03-18T20:57:28.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:28 smithi155 ceph-mon[48344]: pgmap v675: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 458 op/s 2024-03-18T20:57:28.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:28 smithi046 ceph-mon[43495]: pgmap v675: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 458 op/s 2024-03-18T20:57:28.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:28 smithi073 ceph-mon[47384]: pgmap v675: 97 pgs: 97 active+clean; 2.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.2 MiB/s wr, 458 op/s 2024-03-18T20:57:29.446 INFO:tasks.workunit.client.0.smithi046.stdout: 10 338219 33.74 MB/sec execute 365 sec latency 136.792 ms 2024-03-18T20:57:30.446 INFO:tasks.workunit.client.0.smithi046.stdout: 10 338940 33.78 MB/sec execute 366 sec latency 99.073 ms 2024-03-18T20:57:30.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:30 smithi155 ceph-mon[48344]: pgmap v676: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 761 op/s 2024-03-18T20:57:30.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:30 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:57:30.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:30 smithi046 ceph-mon[43495]: pgmap v676: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 761 op/s 2024-03-18T20:57:30.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:30 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:57:30.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:30 smithi073 ceph-mon[47384]: pgmap v676: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 761 op/s 2024-03-18T20:57:30.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:30 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:57:31.446 INFO:tasks.workunit.client.0.smithi046.stdout: 10 339447 33.78 MB/sec execute 367 sec latency 75.470 ms 2024-03-18T20:57:32.446 INFO:tasks.workunit.client.0.smithi046.stdout: 10 339866 33.74 MB/sec execute 368 sec latency 120.346 ms 2024-03-18T20:57:32.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:32 smithi155 ceph-mon[48344]: pgmap v677: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 593 op/s 2024-03-18T20:57:32.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:32 smithi046 ceph-mon[43495]: pgmap v677: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 593 op/s 2024-03-18T20:57:32.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:32 smithi073 ceph-mon[47384]: pgmap v677: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 593 op/s 2024-03-18T20:57:33.446 INFO:tasks.workunit.client.0.smithi046.stdout: 10 340405 33.71 MB/sec execute 369 sec latency 79.033 ms 2024-03-18T20:57:34.446 INFO:tasks.workunit.client.0.smithi046.stdout: 10 340994 33.70 MB/sec execute 370 sec latency 146.027 ms 2024-03-18T20:57:34.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:34 smithi155 ceph-mon[48344]: pgmap v678: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 664 op/s 2024-03-18T20:57:34.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:34 smithi046 ceph-mon[43495]: pgmap v678: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 664 op/s 2024-03-18T20:57:34.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:34 smithi073 ceph-mon[47384]: pgmap v678: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 13 MiB/s wr, 664 op/s 2024-03-18T20:57:35.446 INFO:tasks.workunit.client.0.smithi046.stdout: 10 341638 33.70 MB/sec execute 371 sec latency 96.356 ms 2024-03-18T20:57:35.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:35 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:57:35.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:35 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:57:35.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:35 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:57:36.446 INFO:tasks.workunit.client.0.smithi046.stdout: 10 342411 33.73 MB/sec execute 372 sec latency 67.033 ms 2024-03-18T20:57:36.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:36 smithi155 ceph-mon[48344]: pgmap v679: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 741 op/s 2024-03-18T20:57:36.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:36 smithi046 ceph-mon[43495]: pgmap v679: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 741 op/s 2024-03-18T20:57:36.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:36 smithi073 ceph-mon[47384]: pgmap v679: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 741 op/s 2024-03-18T20:57:37.446 INFO:tasks.workunit.client.0.smithi046.stdout: 10 342900 33.73 MB/sec execute 373 sec latency 74.222 ms 2024-03-18T20:57:37.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:37 smithi073 ceph-mon[47384]: pgmap v680: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 596 op/s 2024-03-18T20:57:38.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:37 smithi155 ceph-mon[48344]: pgmap v680: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 596 op/s 2024-03-18T20:57:38.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:37 smithi046 ceph-mon[43495]: pgmap v680: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 12 MiB/s wr, 596 op/s 2024-03-18T20:57:38.446 INFO:tasks.workunit.client.0.smithi046.stdout: 10 343357 33.70 MB/sec execute 374 sec latency 93.976 ms 2024-03-18T20:57:39.446 INFO:tasks.workunit.client.0.smithi046.stdout: 10 343864 33.67 MB/sec execute 375 sec latency 127.058 ms 2024-03-18T20:57:40.446 INFO:tasks.workunit.client.0.smithi046.stdout: 10 344514 33.67 MB/sec execute 376 sec latency 108.303 ms 2024-03-18T20:57:40.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:40 smithi073 ceph-mon[47384]: pgmap v681: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 14 MiB/s wr, 771 op/s 2024-03-18T20:57:40.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:40 smithi155 ceph-mon[48344]: pgmap v681: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 14 MiB/s wr, 771 op/s 2024-03-18T20:57:40.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:40 smithi046 ceph-mon[43495]: pgmap v681: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 14 MiB/s wr, 771 op/s 2024-03-18T20:57:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.275+0000 7f802a1db700 1 -- 172.21.15.46:0/846946658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8024100ee0 msgr2=0x7f80241012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.275+0000 7f802a1db700 1 --2- 172.21.15.46:0/846946658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8024100ee0 0x7f80241012c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8014009990 tx=0x7f8014009ca0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.276+0000 7f802a1db700 1 -- 172.21.15.46:0/846946658 shutdown_connections 2024-03-18T20:57:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.276+0000 7f802a1db700 1 --2- 172.21.15.46:0/846946658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8024100ee0 0x7f80241012c0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.276+0000 7f802a1db700 1 --2- 172.21.15.46:0/846946658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8024106000 0x7f802410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.276+0000 7f802a1db700 1 --2- 172.21.15.46:0/846946658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8024101890 0x7f80241058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.276+0000 7f802a1db700 1 -- 172.21.15.46:0/846946658 >> 172.21.15.46:0/846946658 conn(0x7f80240fc780 msgr2=0x7f80240feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.276+0000 7f802a1db700 1 -- 172.21.15.46:0/846946658 shutdown_connections 2024-03-18T20:57:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.276+0000 7f802a1db700 1 -- 172.21.15.46:0/846946658 wait complete. 2024-03-18T20:57:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.277+0000 7f802a1db700 1 Processor -- start 2024-03-18T20:57:41.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.279+0000 7f802a1db700 1 -- start start 2024-03-18T20:57:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f802a1db700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8024100ee0 0x7f8024199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f802a1db700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8024106000 0x7f8024199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f802a1db700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f802419dc60 0x7f802419e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f802a1db700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802419e6e0 con 0x7f8024106000 2024-03-18T20:57:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f802a1db700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802419e850 con 0x7f802419dc60 2024-03-18T20:57:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f802a1db700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802419ea10 con 0x7f8024100ee0 2024-03-18T20:57:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f80237fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8024100ee0 0x7f8024199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f80237fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8024100ee0 0x7f8024199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43380/0 (socket says 172.21.15.46:43380) 2024-03-18T20:57:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f80237fe700 1 -- 172.21.15.46:0/3960053037 learned_addr learned my addr 172.21.15.46:0/3960053037 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f80237fe700 1 -- 172.21.15.46:0/3960053037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f802419dc60 msgr2=0x7f802419e0e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-18T20:57:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f80237fe700 1 --2- 172.21.15.46:0/3960053037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f802419dc60 0x7f802419e0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f80237fe700 1 -- 172.21.15.46:0/3960053037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8024106000 msgr2=0x7f8024199960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T20:57:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f80237fe700 1 --2- 172.21.15.46:0/3960053037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8024106000 0x7f8024199960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.280+0000 7f80237fe700 1 -- 172.21.15.46:0/3960053037 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80140095f0 con 0x7f8024100ee0 2024-03-18T20:57:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.281+0000 7f80237fe700 1 --2- 172.21.15.46:0/3960053037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8024100ee0 0x7f8024199420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8014005770 tx=0x7f8014005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.281+0000 7f8020ff9700 1 -- 172.21.15.46:0/3960053037 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801401d070 con 0x7f8024100ee0 2024-03-18T20:57:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.281+0000 7f802a1db700 1 -- 172.21.15.46:0/3960053037 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80241a5f10 con 0x7f8024100ee0 2024-03-18T20:57:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.281+0000 7f802a1db700 1 -- 172.21.15.46:0/3960053037 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80241a6330 con 0x7f8024100ee0 2024-03-18T20:57:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.282+0000 7f8020ff9700 1 -- 172.21.15.46:0/3960053037 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8014022470 con 0x7f8024100ee0 2024-03-18T20:57:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.282+0000 7f8020ff9700 1 -- 172.21.15.46:0/3960053037 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014022df0 con 0x7f8024100ee0 2024-03-18T20:57:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.285+0000 7f8020ff9700 1 -- 172.21.15.46:0/3960053037 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f80140173f0 con 0x7f8024100ee0 2024-03-18T20:57:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.286+0000 7f8020ff9700 1 --2- 172.21.15.46:0/3960053037 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f800c0a1390 0x7f800c0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.286+0000 7f8020ff9700 1 -- 172.21.15.46:0/3960053037 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f80140cee80 con 0x7f8024100ee0 2024-03-18T20:57:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.287+0000 7f8022ffd700 1 --2- 172.21.15.46:0/3960053037 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f800c0a1390 0x7f800c0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.288+0000 7f8022ffd700 1 --2- 172.21.15.46:0/3960053037 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f800c0a1390 0x7f800c0a3850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f802419a970 tx=0x7f8018009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.288+0000 7f8036370b80 1 Processor -- start 2024-03-18T20:57:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.289+0000 7f8036370b80 1 -- start start 2024-03-18T20:57:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.289+0000 7f8036370b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74570 0x561f30a74950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.289+0000 7f8036370b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74e90 0x561f30a7f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.289+0000 7f8036370b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30a821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.289+0000 7f8036370b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f309ce930 con 0x561f30a74e90 2024-03-18T20:57:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.289+0000 7f8036370b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f309cc2f0 con 0x561f30a74570 2024-03-18T20:57:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.289+0000 7f8036370b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f309c7be0 con 0x561f30a7fdb0 2024-03-18T20:57:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.289+0000 7f8023fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30a821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.289+0000 7f8022ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74e90 0x561f30a7f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.289+0000 7f8023fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30a821a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43390/0 (socket says 172.21.15.46:43390) 2024-03-18T20:57:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.289+0000 7f8023fff700 1 -- 172.21.15.46:0/2349073663 learned_addr learned my addr 172.21.15.46:0/2349073663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.289+0000 7f80237fe700 1 --2- 172.21.15.46:0/2349073663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74570 0x561f30a74950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.290+0000 7f8023fff700 1 -- 172.21.15.46:0/2349073663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74570 msgr2=0x561f30a74950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.290+0000 7f8023fff700 1 --2- 172.21.15.46:0/2349073663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74570 0x561f30a74950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.290+0000 7f8023fff700 1 -- 172.21.15.46:0/2349073663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74e90 msgr2=0x561f30a7f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.290+0000 7f8023fff700 1 --2- 172.21.15.46:0/2349073663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74e90 0x561f30a7f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.290+0000 7f8023fff700 1 -- 172.21.15.46:0/2349073663 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f30849350 con 0x561f30a7fdb0 2024-03-18T20:57:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.291+0000 7f8023fff700 1 --2- 172.21.15.46:0/2349073663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30a821a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f801c00e380 tx=0x7f801c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6541ce5c71a0ac59 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.294+0000 7f8009ffb700 1 -- 172.21.15.46:0/2349073663 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c018070 con 0x561f30a7fdb0 2024-03-18T20:57:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.294+0000 7f8009ffb700 1 -- 172.21.15.46:0/2349073663 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f801c00ee30 con 0x561f30a7fdb0 2024-03-18T20:57:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.294+0000 7f8009ffb700 1 -- 172.21.15.46:0/2349073663 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c010040 con 0x561f30a7fdb0 2024-03-18T20:57:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.295+0000 7f8036370b80 1 -- 172.21.15.46:0/2349073663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 msgr2=0x561f30a821a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.295+0000 7f8036370b80 1 --2- 172.21.15.46:0/2349073663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30a821a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f801c00e380 tx=0x7f801c00b040 comp rx=0 tx=0).stop 2024-03-18T20:57:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.298+0000 7f8036370b80 1 -- 172.21.15.46:0/2349073663 shutdown_connections 2024-03-18T20:57:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.298+0000 7f8036370b80 1 --2- 172.21.15.46:0/2349073663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30a821a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.298+0000 7f8036370b80 1 --2- 172.21.15.46:0/2349073663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74570 0x561f30a74950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.298+0000 7f8036370b80 1 --2- 172.21.15.46:0/2349073663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74e90 0x561f30a7f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.298+0000 7f8036370b80 1 -- 172.21.15.46:0/2349073663 >> 172.21.15.46:0/2349073663 conn(0x561f309eb1c0 msgr2=0x561f30a70a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.300+0000 7f8036370b80 1 -- 172.21.15.46:0/2349073663 shutdown_connections 2024-03-18T20:57:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.301+0000 7f8036370b80 1 -- 172.21.15.46:0/2349073663 wait complete. 2024-03-18T20:57:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.301+0000 7f8036370b80 1 -- start start 2024-03-18T20:57:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.301+0000 7f8036370b80 1 Processor -- start 2024-03-18T20:57:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f8036370b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74570 0x561f30a7f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f8036370b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74e90 0x561f30a7ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f8036370b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30a7d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f8036370b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f30963d20 con 0x561f30a74e90 2024-03-18T20:57:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f8036370b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f309639c0 con 0x561f30a74570 2024-03-18T20:57:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f8036370b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f3084b640 con 0x561f30a7fdb0 2024-03-18T20:57:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f8036370b80 10 client.0 _ll_register_callbacks cb 0x561f307cf340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:57:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f80237fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74570 0x561f30a7f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f80237fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74570 0x561f30a7f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:50456/0 (socket says 172.21.15.46:50456) 2024-03-18T20:57:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f80237fe700 1 -- 172.21.15.46:0/1555134510 learned_addr learned my addr 172.21.15.46:0/1555134510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f8022ffd700 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74e90 0x561f30a7ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f8023fff700 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30a7d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f80237fe700 1 -- 172.21.15.46:0/1555134510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 msgr2=0x561f30a7d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f80237fe700 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30a7d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f80237fe700 1 -- 172.21.15.46:0/1555134510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74e90 msgr2=0x561f30a7ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f80237fe700 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74e90 0x561f30a7ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.302+0000 7f80237fe700 1 -- 172.21.15.46:0/1555134510 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c00e0c0 con 0x561f30a74570 2024-03-18T20:57:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.303+0000 7f80237fe700 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74570 0x561f30a7f750 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8014006010 tx=0x7f801409c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8585dae117b4bc8b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.303+0000 7f8002ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:57:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.303+0000 7f8002ffd700 1 -- 172.21.15.46:0/1555134510 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140a4390 con 0x561f30a74570 2024-03-18T20:57:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.303+0000 7f8036370b80 1 -- 172.21.15.46:0/1555134510 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f30a686c0 con 0x561f30a74570 2024-03-18T20:57:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.303+0000 7f8002ffd700 1 -- 172.21.15.46:0/1555134510 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f801409b490 con 0x561f30a74570 2024-03-18T20:57:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.303+0000 7f8002ffd700 1 -- 172.21.15.46:0/1555134510 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f801409b650 con 0x561f30a74570 2024-03-18T20:57:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.305+0000 7f8036370b80 10 client.30260 fetch_fsmap learned FSMap version 91 2024-03-18T20:57:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.305+0000 7f8036370b80 1 -- 172.21.15.46:0/1555134510 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x561f307f5b00 con 0x561f30a74570 2024-03-18T20:57:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.307+0000 7f8002ffd700 1 -- 172.21.15.46:0/1555134510 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f30a686c0 con 0x561f30a74570 2024-03-18T20:57:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.308+0000 7f8002ffd700 1 -- 172.21.15.46:0/1555134510 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 91) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f801400fad0 con 0x561f30a74570 2024-03-18T20:57:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.308+0000 7f8036370b80 10 client.30260 fetch_fsmap finished waiting for FSMap version 91 2024-03-18T20:57:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.308+0000 7f8036370b80 10 client.30260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:57:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.308+0000 7f8036370b80 20 client.30260 populate_metadata read hostname 'smithi046' 2024-03-18T20:57:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.308+0000 7f8036370b80 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f30a97e10 0x561f30a9a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.308+0000 7f8036370b80 4 client.30260 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:57:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.308+0000 7f8036370b80 1 -- 172.21.15.46:0/1555134510 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f309c06a0 con 0x561f30a97e10 2024-03-18T20:57:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.309+0000 7f8022ffd700 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f30a97e10 0x561f30a9a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.310+0000 7f8002ffd700 10 client.30260 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.310+0000 7f8022ffd700 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f30a97e10 0x561f30a9a230 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff677dd0c9502988 server_cookie=548655f32cd18e8a in_seq=0 out_seq=0 2024-03-18T20:57:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.310+0000 7f8002ffd700 1 -- 172.21.15.46:0/1555134510 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561f309c06a0 con 0x561f30a97e10 2024-03-18T20:57:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.310+0000 7f8002ffd700 10 client.30260 handle_command_reply: tid=0 2024-03-18T20:57:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.310+0000 7f8036370b80 1 client.30260 shutdown 2024-03-18T20:57:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.311+0000 7f8036370b80 1 -- 172.21.15.46:0/1555134510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74570 msgr2=0x561f30a7f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.311+0000 7f8036370b80 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74570 0x561f30a7f750 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8014006010 tx=0x7f801409c750 comp rx=0 tx=0).stop 2024-03-18T20:57:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.311+0000 7f8036370b80 1 -- 172.21.15.46:0/1555134510 shutdown_connections 2024-03-18T20:57:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.311+0000 7f8036370b80 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f30a97e10 0x561f30a9a230 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.311+0000 7f8036370b80 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30a7d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.311+0000 7f8002ffd700 0 client.30260 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.311+0000 7f8036370b80 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74570 0x561f30a7f750 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.311+0000 7f8036370b80 1 --2- 172.21.15.46:0/1555134510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74e90 0x561f30a7ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.311+0000 7f8036370b80 1 -- 172.21.15.46:0/1555134510 >> 172.21.15.46:0/1555134510 conn(0x561f309eb1c0 msgr2=0x561f30a70650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.311+0000 7f8036370b80 1 -- 172.21.15.46:0/1555134510 shutdown_connections 2024-03-18T20:57:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.311+0000 7f8036370b80 1 -- 172.21.15.46:0/1555134510 wait complete. 2024-03-18T20:57:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.311+0000 7f8036370b80 20 client.30260 trim_cache size 0 max 16384 2024-03-18T20:57:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.342+0000 7f8036370b80 1 Processor -- start 2024-03-18T20:57:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.342+0000 7f8036370b80 1 -- start start 2024-03-18T20:57:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.342+0000 7f8036370b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 0x561f30a821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.342+0000 7f8036370b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a74e90 0x561f30a8f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.342+0000 7f8036370b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a7fdb0 0x561f30a70500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f8036370b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f30a6d200 con 0x561f30a74570 2024-03-18T20:57:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f8036370b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f30a71f70 con 0x561f30a7fdb0 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f8036370b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f30a8f810 con 0x561f30a74e90 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f80237fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 0x561f30a821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f8022ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a74e90 0x561f30a8f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f80237fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 0x561f30a821a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44774/0 (socket says 172.21.15.46:44774) 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f80237fe700 1 -- 172.21.15.46:0/468901250 learned_addr learned my addr 172.21.15.46:0/468901250 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f8022ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a74e90 0x561f30a8f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:43398/0 (socket says 172.21.15.46:43398) 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f8023fff700 1 --2- 172.21.15.46:0/468901250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a7fdb0 0x561f30a70500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f80237fe700 1 -- 172.21.15.46:0/468901250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a74e90 msgr2=0x561f30a8f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f80237fe700 1 --2- 172.21.15.46:0/468901250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a74e90 0x561f30a8f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f80237fe700 1 -- 172.21.15.46:0/468901250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a7fdb0 msgr2=0x561f30a70500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f80237fe700 1 --2- 172.21.15.46:0/468901250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a7fdb0 0x561f30a70500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f80237fe700 1 -- 172.21.15.46:0/468901250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f3098de40 con 0x561f30a74570 2024-03-18T20:57:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.343+0000 7f80237fe700 1 --2- 172.21.15.46:0/468901250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 0x561f30a821a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f8014098a10 tx=0x7f8014009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899105e079efed0c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.344+0000 7f800a7fc700 1 -- 172.21.15.46:0/468901250 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801409bd70 con 0x561f30a74570 2024-03-18T20:57:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.344+0000 7f800a7fc700 1 -- 172.21.15.46:0/468901250 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80140ba2a0 con 0x561f30a74570 2024-03-18T20:57:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.344+0000 7f800a7fc700 1 -- 172.21.15.46:0/468901250 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140a4160 con 0x561f30a74570 2024-03-18T20:57:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.345+0000 7f8036370b80 1 -- 172.21.15.46:0/468901250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 msgr2=0x561f30a821a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.345+0000 7f8036370b80 1 --2- 172.21.15.46:0/468901250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 0x561f30a821a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f8014098a10 tx=0x7f8014009310 comp rx=0 tx=0).stop 2024-03-18T20:57:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.348+0000 7f8036370b80 1 -- 172.21.15.46:0/468901250 shutdown_connections 2024-03-18T20:57:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.348+0000 7f8036370b80 1 --2- 172.21.15.46:0/468901250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 0x561f30a821a0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.348+0000 7f8036370b80 1 --2- 172.21.15.46:0/468901250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a7fdb0 0x561f30a70500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.348+0000 7f8036370b80 1 --2- 172.21.15.46:0/468901250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a74e90 0x561f30a8f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.348+0000 7f8036370b80 1 -- 172.21.15.46:0/468901250 >> 172.21.15.46:0/468901250 conn(0x561f309eb1c0 msgr2=0x561f30a6f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.352+0000 7f8036370b80 1 -- 172.21.15.46:0/468901250 shutdown_connections 2024-03-18T20:57:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.352+0000 7f8036370b80 1 -- 172.21.15.46:0/468901250 wait complete. 2024-03-18T20:57:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.352+0000 7f8036370b80 1 -- start start 2024-03-18T20:57:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.352+0000 7f8036370b80 1 Processor -- start 2024-03-18T20:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f8036370b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 0x561f30a8d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f8036370b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74e90 0x561f30a8d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f8036370b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30aeac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f8036370b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f30a8e040 con 0x561f30a74570 2024-03-18T20:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f8036370b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f30aeb180 con 0x561f30a74e90 2024-03-18T20:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f8036370b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f30aeb2c0 con 0x561f30a7fdb0 2024-03-18T20:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f8036370b80 10 client.0 _ll_register_callbacks cb 0x7f801409bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f80237fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 0x561f30a8d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f80237fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 0x561f30a8d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44778/0 (socket says 172.21.15.46:44778) 2024-03-18T20:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f80237fe700 1 -- 172.21.15.46:0/608716963 learned_addr learned my addr 172.21.15.46:0/608716963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f8022ffd700 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74e90 0x561f30a8d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f8023fff700 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30aeac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f80237fe700 1 -- 172.21.15.46:0/608716963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 msgr2=0x561f30aeac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f80237fe700 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30aeac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f80237fe700 1 -- 172.21.15.46:0/608716963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74e90 msgr2=0x561f30a8d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f80237fe700 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74e90 0x561f30a8d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.353+0000 7f80237fe700 1 -- 172.21.15.46:0/608716963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f307f5b00 con 0x561f30a74570 2024-03-18T20:57:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.354+0000 7f80237fe700 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 0x561f30a8d2b0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f801409a590 tx=0x7f8014005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c7d95757fd064d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.354+0000 7f8002ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:57:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.354+0000 7f8002ffd700 1 -- 172.21.15.46:0/608716963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801409b690 con 0x561f30a74570 2024-03-18T20:57:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.354+0000 7f8002ffd700 1 -- 172.21.15.46:0/608716963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f80140a4390 con 0x561f30a74570 2024-03-18T20:57:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.354+0000 7f8036370b80 1 -- 172.21.15.46:0/608716963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f801c00ee30 con 0x561f30a74570 2024-03-18T20:57:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.354+0000 7f8002ffd700 1 -- 172.21.15.46:0/608716963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f801409bd70 con 0x561f30a74570 2024-03-18T20:57:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.354+0000 7f8036370b80 10 client.20658 fetch_fsmap learned FSMap version 91 2024-03-18T20:57:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.354+0000 7f8036370b80 1 -- 172.21.15.46:0/608716963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) v3 -- 0x561f3084dc70 con 0x561f30a74570 2024-03-18T20:57:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.355+0000 7f8002ffd700 1 -- 172.21.15.46:0/608716963 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f801c00ee30 con 0x561f30a74570 2024-03-18T20:57:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.355+0000 7f8002ffd700 1 -- 172.21.15.46:0/608716963 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 91) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f801409af00 con 0x561f30a74570 2024-03-18T20:57:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.355+0000 7f8036370b80 10 client.20658 fetch_fsmap finished waiting for FSMap version 91 2024-03-18T20:57:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.355+0000 7f8036370b80 10 client.20658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:57:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.355+0000 7f8036370b80 20 client.20658 populate_metadata read hostname 'smithi046' 2024-03-18T20:57:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.355+0000 7f8036370b80 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f30aec520 0x561f30aee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.355+0000 7f8036370b80 4 client.20658 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:57:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.356+0000 7f8036370b80 1 -- 172.21.15.46:0/608716963 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561f30816c90 con 0x561f30aec520 2024-03-18T20:57:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.356+0000 7f8022ffd700 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f30aec520 0x561f30aee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.356+0000 7f8022ffd700 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f30aec520 0x561f30aee910 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474a95f5464d078b server_cookie=6896fbabfe6f773f in_seq=0 out_seq=0 2024-03-18T20:57:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.356+0000 7f8002ffd700 10 client.20658 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:41.453 INFO:tasks.workunit.client.0.smithi046.stdout: 10 345193 33.68 MB/sec execute 377 sec latency 73.975 ms 2024-03-18T20:57:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.667+0000 7f8002ffd700 1 -- 172.21.15.46:0/608716963 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561f30816c90 con 0x561f30aec520 2024-03-18T20:57:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.667+0000 7f8002ffd700 10 client.20658 handle_command_reply: tid=0 2024-03-18T20:57:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.667+0000 7f8036370b80 1 client.20658 shutdown 2024-03-18T20:57:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.673+0000 7f8036370b80 1 -- 172.21.15.46:0/608716963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 msgr2=0x561f30a8d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.673+0000 7f8036370b80 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 0x561f30a8d2b0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f801409a590 tx=0x7f8014005850 comp rx=0 tx=0).stop 2024-03-18T20:57:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.677+0000 7f8036370b80 1 -- 172.21.15.46:0/608716963 shutdown_connections 2024-03-18T20:57:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.677+0000 7f8036370b80 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561f30aec520 0x561f30aee910 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.677+0000 7f8036370b80 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f30a74570 0x561f30a8d2b0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.677+0000 7f8036370b80 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561f30a7fdb0 0x561f30aeac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.677+0000 7f8036370b80 1 --2- 172.21.15.46:0/608716963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f30a74e90 0x561f30a8d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.677+0000 7f8036370b80 1 -- 172.21.15.46:0/608716963 >> 172.21.15.46:0/608716963 conn(0x561f309eb1c0 msgr2=0x561f30a7b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.677+0000 7f8002ffd700 0 client.20658 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.677+0000 7f8036370b80 1 -- 172.21.15.46:0/608716963 shutdown_connections 2024-03-18T20:57:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.677+0000 7f8036370b80 1 -- 172.21.15.46:0/608716963 wait complete. 2024-03-18T20:57:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.677+0000 7f8036370b80 20 client.20658 trim_cache size 0 max 16384 2024-03-18T20:57:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:57:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:57:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:57:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.681+0000 7f802a1db700 1 -- 172.21.15.46:0/3960053037 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f800c0a1390 msgr2=0x7f800c0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.681+0000 7f802a1db700 1 --2- 172.21.15.46:0/3960053037 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f800c0a1390 0x7f800c0a3850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f802419a970 tx=0x7f8018009260 comp rx=0 tx=0).stop 2024-03-18T20:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.681+0000 7f802a1db700 1 -- 172.21.15.46:0/3960053037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8024100ee0 msgr2=0x7f8024199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.681+0000 7f802a1db700 1 --2- 172.21.15.46:0/3960053037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8024100ee0 0x7f8024199420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8014005770 tx=0x7f8014005e80 comp rx=0 tx=0).stop 2024-03-18T20:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.683+0000 7f802a1db700 1 -- 172.21.15.46:0/3960053037 shutdown_connections 2024-03-18T20:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.683+0000 7f802a1db700 1 --2- 172.21.15.46:0/3960053037 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f800c0a1390 0x7f800c0a3850 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.683+0000 7f802a1db700 1 --2- 172.21.15.46:0/3960053037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f802419dc60 0x7f802419e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.683+0000 7f802a1db700 1 --2- 172.21.15.46:0/3960053037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8024100ee0 0x7f8024199420 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.683+0000 7f802a1db700 1 --2- 172.21.15.46:0/3960053037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8024106000 0x7f8024199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.683+0000 7f802a1db700 1 -- 172.21.15.46:0/3960053037 >> 172.21.15.46:0/3960053037 conn(0x7f80240fc780 msgr2=0x7f80240fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.683+0000 7f802a1db700 1 -- 172.21.15.46:0/3960053037 shutdown_connections 2024-03-18T20:57:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:41.683+0000 7f802a1db700 1 -- 172.21.15.46:0/3960053037 wait complete. 2024-03-18T20:57:41.984 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:41 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:57:42.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:41 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:57:42.051 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:41 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:57:42.449 INFO:tasks.workunit.client.0.smithi046.stdout: 10 345305 33.61 MB/sec execute 378 sec latency 615.524 ms 2024-03-18T20:57:42.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:42 smithi073 ceph-mon[47384]: pgmap v682: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.5 MiB/s wr, 484 op/s 2024-03-18T20:57:43.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:42 smithi155 ceph-mon[48344]: pgmap v682: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.5 MiB/s wr, 484 op/s 2024-03-18T20:57:43.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:42 smithi046 ceph-mon[43495]: pgmap v682: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.5 MiB/s wr, 484 op/s 2024-03-18T20:57:43.449 INFO:tasks.workunit.client.0.smithi046.stdout: 10 345602 33.56 MB/sec execute 379 sec latency 705.045 ms 2024-03-18T20:57:43.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:43 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:57:43.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:43 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:57:43.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:43 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:57:43.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:43 smithi073 ceph-mon[47384]: pgmap v683: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.0 MiB/s wr, 498 op/s 2024-03-18T20:57:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:43 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:57:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:43 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:57:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:43 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:57:44.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:43 smithi155 ceph-mon[48344]: pgmap v683: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.0 MiB/s wr, 498 op/s 2024-03-18T20:57:44.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:43 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:57:44.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:43 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:57:44.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:43 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:57:44.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:43 smithi046 ceph-mon[43495]: pgmap v683: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 8.0 MiB/s wr, 498 op/s 2024-03-18T20:57:44.449 INFO:tasks.workunit.client.0.smithi046.stdout: 10 345984 33.53 MB/sec execute 380 sec latency 143.709 ms 2024-03-18T20:57:45.449 INFO:tasks.workunit.client.0.smithi046.stdout: 10 346557 33.53 MB/sec execute 381 sec latency 85.898 ms 2024-03-18T20:57:46.450 INFO:tasks.workunit.client.0.smithi046.stdout: 10 347080 33.50 MB/sec execute 382 sec latency 105.804 ms 2024-03-18T20:57:46.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:46 smithi155 ceph-mon[48344]: pgmap v684: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 704 op/s 2024-03-18T20:57:46.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:46 smithi046 ceph-mon[43495]: pgmap v684: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 704 op/s 2024-03-18T20:57:46.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:46 smithi073 ceph-mon[47384]: pgmap v684: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 14 MiB/s wr, 704 op/s 2024-03-18T20:57:47.450 INFO:tasks.workunit.client.0.smithi046.stdout: 10 347745 33.53 MB/sec execute 383 sec latency 74.292 ms 2024-03-18T20:57:48.450 INFO:tasks.workunit.client.0.smithi046.stdout: 10 348285 33.51 MB/sec execute 384 sec latency 79.231 ms 2024-03-18T20:57:48.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:48 smithi155 ceph-mon[48344]: pgmap v685: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 565 op/s 2024-03-18T20:57:48.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:48 smithi046 ceph-mon[43495]: pgmap v685: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 565 op/s 2024-03-18T20:57:48.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:48 smithi073 ceph-mon[47384]: pgmap v685: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 565 op/s 2024-03-18T20:57:49.450 INFO:tasks.workunit.client.0.smithi046.stdout: 10 348879 33.50 MB/sec execute 385 sec latency 106.202 ms 2024-03-18T20:57:49.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:49 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:57:49.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:49 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:57:49.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:49 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:57:50.450 INFO:tasks.workunit.client.0.smithi046.stdout: 10 349547 33.51 MB/sec execute 386 sec latency 116.785 ms 2024-03-18T20:57:50.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:50 smithi155 ceph-mon[48344]: pgmap v686: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 14 MiB/s wr, 747 op/s 2024-03-18T20:57:50.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:50 smithi046 ceph-mon[43495]: pgmap v686: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 14 MiB/s wr, 747 op/s 2024-03-18T20:57:50.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:50 smithi073 ceph-mon[47384]: pgmap v686: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 14 MiB/s wr, 747 op/s 2024-03-18T20:57:51.450 INFO:tasks.workunit.client.0.smithi046.stdout: 10 350074 33.50 MB/sec execute 387 sec latency 89.756 ms 2024-03-18T20:57:52.450 INFO:tasks.workunit.client.0.smithi046.stdout: 10 350695 33.50 MB/sec execute 388 sec latency 90.785 ms 2024-03-18T20:57:52.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:52 smithi155 ceph-mon[48344]: pgmap v687: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 624 op/s 2024-03-18T20:57:52.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:52 smithi046 ceph-mon[43495]: pgmap v687: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 624 op/s 2024-03-18T20:57:52.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:52 smithi073 ceph-mon[47384]: pgmap v687: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 12 MiB/s wr, 624 op/s 2024-03-18T20:57:53.450 INFO:tasks.workunit.client.0.smithi046.stdout: 10 351280 33.51 MB/sec execute 389 sec latency 83.592 ms 2024-03-18T20:57:54.450 INFO:tasks.workunit.client.0.smithi046.stdout: 10 351756 33.47 MB/sec execute 390 sec latency 138.012 ms 2024-03-18T20:57:54.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:54 smithi155 ceph-mon[48344]: pgmap v688: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 694 op/s 2024-03-18T20:57:54.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:54 smithi046 ceph-mon[43495]: pgmap v688: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 694 op/s 2024-03-18T20:57:54.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.873+0000 7f8992701700 1 -- 172.21.15.46:0/39368502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f898c0ffa90 msgr2=0x7f898c0ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.873+0000 7f8992701700 1 --2- 172.21.15.46:0/39368502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f898c0ffa90 0x7f898c0ffe70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f897c00b0a0 tx=0x7f897c00b3b0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.873+0000 7f8992701700 1 -- 172.21.15.46:0/39368502 shutdown_connections 2024-03-18T20:57:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.873+0000 7f8992701700 1 --2- 172.21.15.46:0/39368502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f898c10d640 0x7f898c10fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.873+0000 7f8992701700 1 --2- 172.21.15.46:0/39368502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f898c0ffa90 0x7f898c0ffe70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.873+0000 7f8992701700 1 --2- 172.21.15.46:0/39368502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898c100440 0x7f898c10d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.873+0000 7f8992701700 1 -- 172.21.15.46:0/39368502 >> 172.21.15.46:0/39368502 conn(0x7f898c0fb680 msgr2=0x7f898c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.873+0000 7f8992701700 1 -- 172.21.15.46:0/39368502 shutdown_connections 2024-03-18T20:57:54.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.873+0000 7f8992701700 1 -- 172.21.15.46:0/39368502 wait complete. 2024-03-18T20:57:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f8992701700 1 Processor -- start 2024-03-18T20:57:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f8992701700 1 -- start start 2024-03-18T20:57:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f8992701700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f898c0ffa90 0x7f898c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f8992701700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f898c100440 0x7f898c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f8992701700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898c10d640 0x7f898c19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f8992701700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c19e4c0 con 0x7f898c10d640 2024-03-18T20:57:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f8992701700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c19e630 con 0x7f898c0ffa90 2024-03-18T20:57:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f8992701700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c19e7f0 con 0x7f898c100440 2024-03-18T20:57:54.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f8990c9e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898c10d640 0x7f898c19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f898bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f898c0ffa90 0x7f898c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f8990c9e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898c10d640 0x7f898c19de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35582/0 (socket says 172.21.15.46:35582) 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f898bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f898c0ffa90 0x7f898c1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33002/0 (socket says 172.21.15.46:33002) 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.874+0000 7f8990c9e700 1 -- 172.21.15.46:0/3898062118 learned_addr learned my addr 172.21.15.46:0/3898062118 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.875+0000 7f898bfff700 1 -- 172.21.15.46:0/3898062118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f898c100440 msgr2=0x7f898c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.875+0000 7f898bfff700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f898c100440 0x7f898c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.875+0000 7f898bfff700 1 -- 172.21.15.46:0/3898062118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898c10d640 msgr2=0x7f898c19de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.875+0000 7f898bfff700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898c10d640 0x7f898c19de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.875+0000 7f898bfff700 1 -- 172.21.15.46:0/3898062118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c009d00 con 0x7f898c0ffa90 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.875+0000 7f8990c9e700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898c10d640 0x7f898c19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.875+0000 7f898bfff700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f898c0ffa90 0x7f898c1995f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f897c00b070 tx=0x7f897c0078e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.875+0000 7f89897fa700 1 -- 172.21.15.46:0/3898062118 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c00e040 con 0x7f898c0ffa90 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.875+0000 7f89897fa700 1 -- 172.21.15.46:0/3898062118 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f897c01d470 con 0x7f898c0ffa90 2024-03-18T20:57:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.875+0000 7f8992701700 1 -- 172.21.15.46:0/3898062118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c19ea70 con 0x7f898c0ffa90 2024-03-18T20:57:54.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.875+0000 7f89897fa700 1 -- 172.21.15.46:0/3898062118 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c0052a0 con 0x7f898c0ffa90 2024-03-18T20:57:54.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.875+0000 7f8992701700 1 -- 172.21.15.46:0/3898062118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c1a6310 con 0x7f898c0ffa90 2024-03-18T20:57:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.878+0000 7f89897fa700 1 -- 172.21.15.46:0/3898062118 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f897c01d5e0 con 0x7f898c0ffa90 2024-03-18T20:57:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.879+0000 7f89897fa700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f89780a1290 0x7f89780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.880+0000 7f898b7fe700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f89780a1290 0x7f89780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.880+0000 7f89897fa700 1 -- 172.21.15.46:0/3898062118 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f897c0cdfb0 con 0x7f898c0ffa90 2024-03-18T20:57:54.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.881+0000 7f898b7fe700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f89780a1290 0x7f89780a3750 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8974005950 tx=0x7f89740058e0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.881+0000 7f899e896b80 1 Processor -- start 2024-03-18T20:57:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.881+0000 7f899e896b80 1 -- start start 2024-03-18T20:57:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.881+0000 7f899e896b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36570 0x56420bd36950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.881+0000 7f899e896b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36e90 0x56420bd41870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.881+0000 7f899e896b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd41db0 0x56420bd441a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f899e896b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bc90930 con 0x56420bd41db0 2024-03-18T20:57:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f899e896b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bc8e2f0 con 0x56420bd36570 2024-03-18T20:57:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f899e896b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bc89be0 con 0x56420bd36e90 2024-03-18T20:57:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f898b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36e90 0x56420bd41870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f898b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36e90 0x56420bd41870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37534/0 (socket says 172.21.15.46:37534) 2024-03-18T20:57:54.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f898b7fe700 1 -- 172.21.15.46:0/2756793650 learned_addr learned my addr 172.21.15.46:0/2756793650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f898bfff700 1 --2- 172.21.15.46:0/2756793650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36570 0x56420bd36950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f898bfff700 1 -- 172.21.15.46:0/2756793650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36e90 msgr2=0x56420bd41870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f898bfff700 1 --2- 172.21.15.46:0/2756793650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36e90 0x56420bd41870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f8990c9e700 1 --2- 172.21.15.46:0/2756793650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd41db0 0x56420bd441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f898bfff700 1 -- 172.21.15.46:0/2756793650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd41db0 msgr2=0x56420bd441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f898bfff700 1 --2- 172.21.15.46:0/2756793650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd41db0 0x56420bd441a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f898bfff700 1 -- 172.21.15.46:0/2756793650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56420bb0b350 con 0x56420bd36570 2024-03-18T20:57:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.882+0000 7f8990c9e700 1 --2- 172.21.15.46:0/2756793650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd41db0 0x56420bd441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:57:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.883+0000 7f898bfff700 1 --2- 172.21.15.46:0/2756793650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36570 0x56420bd36950 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f897c0cb6e0 tx=0x7f897c09aa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c76df28368e4c191 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.883+0000 7f89827fc700 1 -- 172.21.15.46:0/2756793650 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c09ef60 con 0x56420bd36570 2024-03-18T20:57:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.883+0000 7f89827fc700 1 -- 172.21.15.46:0/2756793650 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f897c09d030 con 0x56420bd36570 2024-03-18T20:57:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.883+0000 7f89827fc700 1 -- 172.21.15.46:0/2756793650 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c01da90 con 0x56420bd36570 2024-03-18T20:57:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.884+0000 7f899e896b80 1 -- 172.21.15.46:0/2756793650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36570 msgr2=0x56420bd36950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.884+0000 7f899e896b80 1 --2- 172.21.15.46:0/2756793650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36570 0x56420bd36950 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f897c0cb6e0 tx=0x7f897c09aa90 comp rx=0 tx=0).stop 2024-03-18T20:57:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.884+0000 7f899e896b80 1 -- 172.21.15.46:0/2756793650 shutdown_connections 2024-03-18T20:57:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.884+0000 7f899e896b80 1 --2- 172.21.15.46:0/2756793650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd41db0 0x56420bd441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.884+0000 7f899e896b80 1 --2- 172.21.15.46:0/2756793650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36570 0x56420bd36950 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.884+0000 7f899e896b80 1 --2- 172.21.15.46:0/2756793650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36e90 0x56420bd41870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.884+0000 7f899e896b80 1 -- 172.21.15.46:0/2756793650 >> 172.21.15.46:0/2756793650 conn(0x56420bcad1c0 msgr2=0x56420bd32a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.884+0000 7f899e896b80 1 -- 172.21.15.46:0/2756793650 shutdown_connections 2024-03-18T20:57:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.884+0000 7f899e896b80 1 -- 172.21.15.46:0/2756793650 wait complete. 2024-03-18T20:57:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.885+0000 7f899e896b80 1 -- start start 2024-03-18T20:57:54.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.885+0000 7f899e896b80 1 Processor -- start 2024-03-18T20:57:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f899e896b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36570 0x56420bd3eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f899e896b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36e90 0x56420bd3f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f899e896b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd41db0 0x56420bd3f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f899e896b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bc25d20 con 0x56420bd41db0 2024-03-18T20:57:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f899e896b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bc259c0 con 0x56420bd36e90 2024-03-18T20:57:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f899e896b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bb0d640 con 0x56420bd36570 2024-03-18T20:57:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f899e896b80 10 client.0 _ll_register_callbacks cb 0x56420ba91340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:57:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f898b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36e90 0x56420bd3f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f898b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36e90 0x56420bd3f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33034/0 (socket says 172.21.15.46:33034) 2024-03-18T20:57:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f898bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36570 0x56420bd3eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f898b7fe700 1 -- 172.21.15.46:0/1716680896 learned_addr learned my addr 172.21.15.46:0/1716680896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f898b7fe700 1 -- 172.21.15.46:0/1716680896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36570 msgr2=0x56420bd3eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f898b7fe700 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36570 0x56420bd3eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f898b7fe700 1 -- 172.21.15.46:0/1716680896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd41db0 msgr2=0x56420bd3f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f8990c9e700 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd41db0 0x56420bd3f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f898b7fe700 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd41db0 0x56420bd3f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f898b7fe700 1 -- 172.21.15.46:0/1716680896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c1a6310 con 0x56420bd36e90 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.886+0000 7f8990c9e700 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd41db0 0x56420bd3f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.887+0000 7f898b7fe700 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36e90 0x56420bd3f0b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f897401d6e0 tx=0x7f8974004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81e33ac0d2ead5b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.887+0000 7f896b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.887+0000 7f896b7fe700 1 -- 172.21.15.46:0/1716680896 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897400e8a0 con 0x56420bd36e90 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.887+0000 7f896b7fe700 1 -- 172.21.15.46:0/1716680896 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8974004a30 con 0x56420bd36e90 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.887+0000 7f899e896b80 1 -- 172.21.15.46:0/1716680896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56420bd2a6c0 con 0x56420bd36e90 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.887+0000 7f896b7fe700 1 -- 172.21.15.46:0/1716680896 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8974004bf0 con 0x56420bd36e90 2024-03-18T20:57:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.887+0000 7f899e896b80 10 client.30278 fetch_fsmap learned FSMap version 92 2024-03-18T20:57:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.887+0000 7f899e896b80 1 -- 172.21.15.46:0/1716680896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x56420bab7b00 con 0x56420bd36e90 2024-03-18T20:57:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.887+0000 7f896b7fe700 1 -- 172.21.15.46:0/1716680896 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8974032020 con 0x56420bd36e90 2024-03-18T20:57:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.888+0000 7f896b7fe700 1 -- 172.21.15.46:0/1716680896 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 92) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8974018070 con 0x56420bd36e90 2024-03-18T20:57:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.888+0000 7f899e896b80 10 client.30278 fetch_fsmap finished waiting for FSMap version 92 2024-03-18T20:57:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.888+0000 7f899e896b80 10 client.30278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:57:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.888+0000 7f899e896b80 20 client.30278 populate_metadata read hostname 'smithi046' 2024-03-18T20:57:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.888+0000 7f899e896b80 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56420bd51890 0x56420bd53cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.888+0000 7f899e896b80 4 client.30278 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:57:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.888+0000 7f899e896b80 1 -- 172.21.15.46:0/1716680896 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56420bc826a0 con 0x56420bd51890 2024-03-18T20:57:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.888+0000 7f898bfff700 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56420bd51890 0x56420bd53cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.888+0000 7f896b7fe700 10 client.30278 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.888+0000 7f898bfff700 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56420bd51890 0x56420bd53cb0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5805eced79f5660a server_cookie=1d966c5c86f0923 in_seq=0 out_seq=0 2024-03-18T20:57:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.890+0000 7f896b7fe700 1 -- 172.21.15.46:0/1716680896 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56420bc826a0 con 0x56420bd51890 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.890+0000 7f896b7fe700 10 client.30278 handle_command_reply: tid=0 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.890+0000 7f899e896b80 1 client.30278 shutdown 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.891+0000 7f899e896b80 1 -- 172.21.15.46:0/1716680896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36e90 msgr2=0x56420bd3f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.891+0000 7f899e896b80 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36e90 0x56420bd3f0b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f897401d6e0 tx=0x7f8974004750 comp rx=0 tx=0).stop 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.891+0000 7f899e896b80 1 -- 172.21.15.46:0/1716680896 shutdown_connections 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.891+0000 7f899e896b80 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56420bd51890 0x56420bd53cb0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.891+0000 7f899e896b80 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd41db0 0x56420bd3f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.891+0000 7f896b7fe700 0 client.30278 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.891+0000 7f899e896b80 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36570 0x56420bd3eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.891+0000 7f899e896b80 1 --2- 172.21.15.46:0/1716680896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd36e90 0x56420bd3f0b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.891+0000 7f899e896b80 1 -- 172.21.15.46:0/1716680896 >> 172.21.15.46:0/1716680896 conn(0x56420bcad1c0 msgr2=0x56420bd32650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.891+0000 7f899e896b80 1 -- 172.21.15.46:0/1716680896 shutdown_connections 2024-03-18T20:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.891+0000 7f899e896b80 1 -- 172.21.15.46:0/1716680896 wait complete. 2024-03-18T20:57:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.891+0000 7f899e896b80 20 client.30278 trim_cache size 0 max 16384 2024-03-18T20:57:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.922+0000 7f899e896b80 1 Processor -- start 2024-03-18T20:57:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.922+0000 7f899e896b80 1 -- start start 2024-03-18T20:57:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.922+0000 7f899e896b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd36570 0x56420bd53c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.922+0000 7f899e896b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36e90 0x56420bd32bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.922+0000 7f899e896b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 0x56420bd31120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.922+0000 7f899e896b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bd37e70 con 0x56420bd36570 2024-03-18T20:57:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.922+0000 7f899e896b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bd36bd0 con 0x56420bd41db0 2024-03-18T20:57:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.922+0000 7f899e896b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bd33de0 con 0x56420bd36e90 2024-03-18T20:57:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.923+0000 7f8990c9e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 0x56420bd31120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.923+0000 7f8990c9e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 0x56420bd31120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33048/0 (socket says 172.21.15.46:33048) 2024-03-18T20:57:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.923+0000 7f8990c9e700 1 -- 172.21.15.46:0/416737457 learned_addr learned my addr 172.21.15.46:0/416737457 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.923+0000 7f8990c9e700 1 -- 172.21.15.46:0/416737457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36e90 msgr2=0x56420bd32bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:57:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.923+0000 7f8990c9e700 1 --2- 172.21.15.46:0/416737457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36e90 0x56420bd32bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.923+0000 7f8990c9e700 1 -- 172.21.15.46:0/416737457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd36570 msgr2=0x56420bd53c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:57:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.923+0000 7f8990c9e700 1 --2- 172.21.15.46:0/416737457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd36570 0x56420bd53c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.923+0000 7f8990c9e700 1 -- 172.21.15.46:0/416737457 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56420bc4fe40 con 0x56420bd41db0 2024-03-18T20:57:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.924+0000 7f8990c9e700 1 --2- 172.21.15.46:0/416737457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 0x56420bd31120 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8984009840 tx=0x7f8984033830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2382ff924ccfc90 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.924+0000 7f8982ffd700 1 -- 172.21.15.46:0/416737457 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984038070 con 0x56420bd41db0 2024-03-18T20:57:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.924+0000 7f8982ffd700 1 -- 172.21.15.46:0/416737457 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f898403c8b0 con 0x56420bd41db0 2024-03-18T20:57:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.924+0000 7f8982ffd700 1 -- 172.21.15.46:0/416737457 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898400fa00 con 0x56420bd41db0 2024-03-18T20:57:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.925+0000 7f899e896b80 1 -- 172.21.15.46:0/416737457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 msgr2=0x56420bd31120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.925+0000 7f899e896b80 1 --2- 172.21.15.46:0/416737457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 0x56420bd31120 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8984009840 tx=0x7f8984033830 comp rx=0 tx=0).stop 2024-03-18T20:57:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.925+0000 7f899e896b80 1 -- 172.21.15.46:0/416737457 shutdown_connections 2024-03-18T20:57:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.925+0000 7f899e896b80 1 --2- 172.21.15.46:0/416737457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd36570 0x56420bd53c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.925+0000 7f899e896b80 1 --2- 172.21.15.46:0/416737457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 0x56420bd31120 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.925+0000 7f899e896b80 1 --2- 172.21.15.46:0/416737457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bd36e90 0x56420bd32bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.925+0000 7f899e896b80 1 -- 172.21.15.46:0/416737457 >> 172.21.15.46:0/416737457 conn(0x56420bcad1c0 msgr2=0x56420bd3e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.926+0000 7f899e896b80 1 -- 172.21.15.46:0/416737457 shutdown_connections 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.926+0000 7f899e896b80 1 -- 172.21.15.46:0/416737457 wait complete. 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.926+0000 7f899e896b80 1 -- start start 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.926+0000 7f899e896b80 1 Processor -- start 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.927+0000 7f899e896b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd36570 0x56420bd469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.927+0000 7f899e896b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 0x56420bd46f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.927+0000 7f899e896b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bda26b0 0x56420bda4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.927+0000 7f899e896b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bda4fe0 con 0x56420bd36570 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.927+0000 7f899e896b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bda5120 con 0x56420bd41db0 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.927+0000 7f899e896b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56420bda52e0 con 0x56420bda26b0 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.927+0000 7f899e896b80 10 client.0 _ll_register_callbacks cb 0x7f897400eda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.927+0000 7f898b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 0x56420bd46f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.927+0000 7f898b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 0x56420bd46f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33056/0 (socket says 172.21.15.46:33056) 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.927+0000 7f898b7fe700 1 -- 172.21.15.46:0/2648537412 learned_addr learned my addr 172.21.15.46:0/2648537412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.927+0000 7f8990c9e700 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bda26b0 0x56420bda4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.928+0000 7f898bfff700 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd36570 0x56420bd469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.928+0000 7f8990c9e700 1 -- 172.21.15.46:0/2648537412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 msgr2=0x56420bd46f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.928+0000 7f8990c9e700 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 0x56420bd46f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.928+0000 7f8990c9e700 1 -- 172.21.15.46:0/2648537412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd36570 msgr2=0x56420bd469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.928+0000 7f8990c9e700 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd36570 0x56420bd469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.928+0000 7f8990c9e700 1 -- 172.21.15.46:0/2648537412 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8984009d60 con 0x56420bda26b0 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.928+0000 7f8990c9e700 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bda26b0 0x56420bda4aa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f89840094f0 tx=0x7f898403c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6e0876e6237cc48 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.928+0000 7f896b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.928+0000 7f896b7fe700 1 -- 172.21.15.46:0/2648537412 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984038040 con 0x56420bda26b0 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.928+0000 7f896b7fe700 1 -- 172.21.15.46:0/2648537412 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8984005510 con 0x56420bda26b0 2024-03-18T20:57:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.928+0000 7f899e896b80 1 -- 172.21.15.46:0/2648537412 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f897c09d030 con 0x56420bda26b0 2024-03-18T20:57:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.929+0000 7f896b7fe700 1 -- 172.21.15.46:0/2648537412 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89840056d0 con 0x56420bda26b0 2024-03-18T20:57:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.929+0000 7f899e896b80 10 client.30391 fetch_fsmap learned FSMap version 92 2024-03-18T20:57:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.929+0000 7f899e896b80 1 -- 172.21.15.46:0/2648537412 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x56420bb0fc70 con 0x56420bda26b0 2024-03-18T20:57:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.929+0000 7f896b7fe700 1 -- 172.21.15.46:0/2648537412 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f897c09d030 con 0x56420bda26b0 2024-03-18T20:57:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.930+0000 7f896b7fe700 1 -- 172.21.15.46:0/2648537412 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 92) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8984043070 con 0x56420bda26b0 2024-03-18T20:57:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.931+0000 7f899e896b80 10 client.30391 fetch_fsmap finished waiting for FSMap version 92 2024-03-18T20:57:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.931+0000 7f899e896b80 10 client.30391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:57:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.931+0000 7f899e896b80 20 client.30391 populate_metadata read hostname 'smithi046' 2024-03-18T20:57:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.931+0000 7f899e896b80 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56420bda6a60 0x56420bda8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:57:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.931+0000 7f899e896b80 4 client.30391 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:57:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.931+0000 7f899e896b80 1 -- 172.21.15.46:0/2648537412 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56420bad8c90 con 0x56420bda6a60 2024-03-18T20:57:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.932+0000 7f898bfff700 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56420bda6a60 0x56420bda8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:57:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.932+0000 7f896b7fe700 10 client.30391 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:54.932+0000 7f898bfff700 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56420bda6a60 0x56420bda8e80 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee020d29d98a98 server_cookie=1b44da399d932064 in_seq=0 out_seq=0 2024-03-18T20:57:54.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:54 smithi073 ceph-mon[47384]: pgmap v688: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 694 op/s 2024-03-18T20:57:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.213+0000 7f896b7fe700 1 -- 172.21.15.46:0/2648537412 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56420bad8c90 con 0x56420bda6a60 2024-03-18T20:57:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.213+0000 7f896b7fe700 10 client.30391 handle_command_reply: tid=0 2024-03-18T20:57:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.213+0000 7f899e896b80 1 client.30391 shutdown 2024-03-18T20:57:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.215+0000 7f899e896b80 1 -- 172.21.15.46:0/2648537412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bda26b0 msgr2=0x56420bda4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.215+0000 7f899e896b80 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bda26b0 0x56420bda4aa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f89840094f0 tx=0x7f898403c620 comp rx=0 tx=0).stop 2024-03-18T20:57:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.219+0000 7f899e896b80 1 -- 172.21.15.46:0/2648537412 shutdown_connections 2024-03-18T20:57:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.219+0000 7f899e896b80 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x56420bda6a60 0x56420bda8e80 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.219+0000 7f899e896b80 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56420bd36570 0x56420bd469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.219+0000 7f899e896b80 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56420bda26b0 0x56420bda4aa0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.219+0000 7f899e896b80 1 --2- 172.21.15.46:0/2648537412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56420bd41db0 0x56420bd46f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.219+0000 7f896b7fe700 0 client.30391 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:57:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.219+0000 7f899e896b80 1 -- 172.21.15.46:0/2648537412 >> 172.21.15.46:0/2648537412 conn(0x56420bcad1c0 msgr2=0x56420bd3d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.220+0000 7f899e896b80 1 -- 172.21.15.46:0/2648537412 shutdown_connections 2024-03-18T20:57:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.220+0000 7f899e896b80 1 -- 172.21.15.46:0/2648537412 wait complete. 2024-03-18T20:57:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:57:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:57:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:57:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:57:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.221+0000 7f899e896b80 20 client.30391 trim_cache size 0 max 16384 2024-03-18T20:57:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.223+0000 7f8992701700 1 -- 172.21.15.46:0/3898062118 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f89780a1290 msgr2=0x7f89780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.223+0000 7f8992701700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f89780a1290 0x7f89780a3750 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8974005950 tx=0x7f89740058e0 comp rx=0 tx=0).stop 2024-03-18T20:57:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.223+0000 7f8992701700 1 -- 172.21.15.46:0/3898062118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f898c0ffa90 msgr2=0x7f898c1995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:57:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.223+0000 7f8992701700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f898c0ffa90 0x7f898c1995f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f897c00b070 tx=0x7f897c0078e0 comp rx=0 tx=0).stop 2024-03-18T20:57:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.224+0000 7f8992701700 1 -- 172.21.15.46:0/3898062118 shutdown_connections 2024-03-18T20:57:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.224+0000 7f8992701700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f89780a1290 0x7f89780a3750 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.224+0000 7f8992701700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f898c10d640 0x7f898c19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.224+0000 7f8992701700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f898c0ffa90 0x7f898c1995f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.224+0000 7f8992701700 1 --2- 172.21.15.46:0/3898062118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f898c100440 0x7f898c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:57:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.224+0000 7f8992701700 1 -- 172.21.15.46:0/3898062118 >> 172.21.15.46:0/3898062118 conn(0x7f898c0fb680 msgr2=0x7f898c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:57:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.224+0000 7f8992701700 1 -- 172.21.15.46:0/3898062118 shutdown_connections 2024-03-18T20:57:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:57:55.224+0000 7f8992701700 1 -- 172.21.15.46:0/3898062118 wait complete. 2024-03-18T20:57:55.450 INFO:tasks.workunit.client.0.smithi046.stdout: 10 352089 33.41 MB/sec execute 391 sec latency 250.633 ms 2024-03-18T20:57:56.450 INFO:tasks.workunit.client.0.smithi046.stdout: 10 352810 33.43 MB/sec execute 392 sec latency 102.021 ms 2024-03-18T20:57:56.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:56 smithi155 ceph-mon[48344]: pgmap v689: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 723 op/s 2024-03-18T20:57:56.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:56 smithi046 ceph-mon[43495]: pgmap v689: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 723 op/s 2024-03-18T20:57:56.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:56 smithi073 ceph-mon[47384]: pgmap v689: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 723 op/s 2024-03-18T20:57:57.452 INFO:tasks.workunit.client.0.smithi046.stdout: 10 353556 33.46 MB/sec execute 393 sec latency 84.788 ms 2024-03-18T20:57:58.452 INFO:tasks.workunit.client.0.smithi046.stdout: 10 354189 33.47 MB/sec execute 394 sec latency 68.625 ms 2024-03-18T20:57:58.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:57:58 smithi155 ceph-mon[48344]: pgmap v690: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 429 op/s 2024-03-18T20:57:58.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:57:58 smithi046 ceph-mon[43495]: pgmap v690: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 429 op/s 2024-03-18T20:57:58.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:57:58 smithi073 ceph-mon[47384]: pgmap v690: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 429 op/s 2024-03-18T20:57:59.453 INFO:tasks.workunit.client.0.smithi046.stdout: 10 354672 33.45 MB/sec execute 395 sec latency 122.467 ms 2024-03-18T20:58:00.454 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355136 33.44 MB/sec execute 396 sec latency 141.009 ms 2024-03-18T20:58:00.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:00 smithi155 ceph-mon[48344]: pgmap v691: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.7 MiB/s wr, 661 op/s 2024-03-18T20:58:00.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:00 smithi046 ceph-mon[43495]: pgmap v691: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.7 MiB/s wr, 661 op/s 2024-03-18T20:58:00.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:00 smithi073 ceph-mon[47384]: pgmap v691: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 8.7 MiB/s wr, 661 op/s 2024-03-18T20:58:01.454 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355176 33.35 MB/sec execute 397 sec latency 929.736 ms 2024-03-18T20:58:02.454 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355176 33.27 MB/sec execute 398 sec latency 1929.862 ms 2024-03-18T20:58:02.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:02 smithi155 ceph-mon[48344]: pgmap v692: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.8 MiB/s wr, 502 op/s 2024-03-18T20:58:02.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:02 smithi046 ceph-mon[43495]: pgmap v692: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.8 MiB/s wr, 502 op/s 2024-03-18T20:58:02.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:02 smithi073 ceph-mon[47384]: pgmap v692: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.8 MiB/s wr, 502 op/s 2024-03-18T20:58:03.454 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355176 33.19 MB/sec execute 399 sec latency 2929.921 ms 2024-03-18T20:58:04.454 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355176 33.10 MB/sec execute 400 sec latency 3930.005 ms 2024-03-18T20:58:04.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:04 smithi155 ceph-mon[48344]: pgmap v693: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.8 MiB/s wr, 487 op/s 2024-03-18T20:58:04.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:04 smithi046 ceph-mon[43495]: pgmap v693: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.8 MiB/s wr, 487 op/s 2024-03-18T20:58:04.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:04 smithi073 ceph-mon[47384]: pgmap v693: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 6.8 MiB/s wr, 487 op/s 2024-03-18T20:58:05.454 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355176 33.02 MB/sec execute 401 sec latency 4930.067 ms 2024-03-18T20:58:06.454 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355305 32.96 MB/sec execute 402 sec latency 4999.747 ms 2024-03-18T20:58:06.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:06 smithi155 ceph-mon[48344]: pgmap v694: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.3 MiB/s wr, 454 op/s 2024-03-18T20:58:06.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:06 smithi046 ceph-mon[43495]: pgmap v694: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.3 MiB/s wr, 454 op/s 2024-03-18T20:58:06.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:06 smithi073 ceph-mon[47384]: pgmap v694: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.3 MiB/s wr, 454 op/s 2024-03-18T20:58:07.454 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355305 32.88 MB/sec execute 403 sec latency 1734.181 ms 2024-03-18T20:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.359+0000 7f118ccb2700 1 -- 172.21.15.46:0/2682997791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11880ffbe0 msgr2=0x7f118810dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.359+0000 7f118ccb2700 1 --2- 172.21.15.46:0/2682997791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11880ffbe0 0x7f118810dbf0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f118000b0d0 tx=0x7f118000b3e0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.359+0000 7f118ccb2700 1 -- 172.21.15.46:0/2682997791 shutdown_connections 2024-03-18T20:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.359+0000 7f118ccb2700 1 --2- 172.21.15.46:0/2682997791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11880ffbe0 0x7f118810dbf0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.359+0000 7f118ccb2700 1 --2- 172.21.15.46:0/2682997791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1188106e30 0x7f1188107210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.359+0000 7f118ccb2700 1 --2- 172.21.15.46:0/2682997791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11880ff220 0x7f11880ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.359+0000 7f118ccb2700 1 -- 172.21.15.46:0/2682997791 >> 172.21.15.46:0/2682997791 conn(0x7f11880761b0 msgr2=0x7f11880fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.359+0000 7f118ccb2700 1 -- 172.21.15.46:0/2682997791 shutdown_connections 2024-03-18T20:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.359+0000 7f118ccb2700 1 -- 172.21.15.46:0/2682997791 wait complete. 2024-03-18T20:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.360+0000 7f118ccb2700 1 Processor -- start 2024-03-18T20:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.360+0000 7f118ccb2700 1 -- start start 2024-03-18T20:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.360+0000 7f118ccb2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11880ff220 0x7f1188197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.360+0000 7f118ccb2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11880ffbe0 0x7f1188197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.360+0000 7f118ccb2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1188106e30 0x7f118819bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.360+0000 7f118ccb2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118819c260 con 0x7f1188106e30 2024-03-18T20:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.360+0000 7f118ccb2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118819c3d0 con 0x7f11880ff220 2024-03-18T20:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.360+0000 7f118ccb2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118819c590 con 0x7f11880ffbe0 2024-03-18T20:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.360+0000 7f1186d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1188106e30 0x7f118819bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f1186d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1188106e30 0x7f118819bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45108/0 (socket says 172.21.15.46:45108) 2024-03-18T20:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f1186d9d700 1 -- 172.21.15.46:0/2684040075 learned_addr learned my addr 172.21.15.46:0/2684040075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f1185d9b700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11880ffbe0 0x7f1188197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f118659c700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11880ff220 0x7f1188197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f1186d9d700 1 -- 172.21.15.46:0/2684040075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11880ffbe0 msgr2=0x7f1188197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f1186d9d700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11880ffbe0 0x7f1188197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f1186d9d700 1 -- 172.21.15.46:0/2684040075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11880ff220 msgr2=0x7f1188197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f1186d9d700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11880ff220 0x7f1188197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f1186d9d700 1 -- 172.21.15.46:0/2684040075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1180009d00 con 0x7f1188106e30 2024-03-18T20:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f1186d9d700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1188106e30 0x7f118819bc20 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f118000bcd0 tx=0x7f118000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f11777fe700 1 -- 172.21.15.46:0/2684040075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118000e040 con 0x7f1188106e30 2024-03-18T20:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f11777fe700 1 -- 172.21.15.46:0/2684040075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1180005070 con 0x7f1188106e30 2024-03-18T20:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f11777fe700 1 -- 172.21.15.46:0/2684040075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118001b6a0 con 0x7f1188106e30 2024-03-18T20:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f118ccb2700 1 -- 172.21.15.46:0/2684040075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118819c810 con 0x7f1188106e30 2024-03-18T20:58:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.361+0000 7f118ccb2700 1 -- 172.21.15.46:0/2684040075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118819ca90 con 0x7f1188106e30 2024-03-18T20:58:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.364+0000 7f11777fe700 1 -- 172.21.15.46:0/2684040075 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f11800051e0 con 0x7f1188106e30 2024-03-18T20:58:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.364+0000 7f11777fe700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f11700a1310 0x7f11700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.364+0000 7f118659c700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f11700a1310 0x7f11700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.364+0000 7f11777fe700 1 -- 172.21.15.46:0/2684040075 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f11800cd5b0 con 0x7f1188106e30 2024-03-18T20:58:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.365+0000 7f118659c700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f11700a1310 0x7f11700a37d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1178009be0 tx=0x7f1178009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.365+0000 7f1198e47b80 1 Processor -- start 2024-03-18T20:58:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.365+0000 7f1198e47b80 1 -- start start 2024-03-18T20:58:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.365+0000 7f1198e47b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495570 0x561b51495950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.365+0000 7f1198e47b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495e90 0x561b514a0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.365+0000 7f1198e47b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b514a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.365+0000 7f1198e47b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b513ef930 con 0x561b51495e90 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.365+0000 7f1198e47b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b513ed2f0 con 0x561b514a0db0 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.365+0000 7f1198e47b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b513e8be0 con 0x561b51495570 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1185d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495e90 0x561b514a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1185d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495e90 0x561b514a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45116/0 (socket says 172.21.15.46:45116) 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1185d9b700 1 -- 172.21.15.46:0/2577339463 learned_addr learned my addr 172.21.15.46:0/2577339463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1186d9d700 1 --2- 172.21.15.46:0/2577339463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b514a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f118659c700 1 --2- 172.21.15.46:0/2577339463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495570 0x561b51495950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1185d9b700 1 -- 172.21.15.46:0/2577339463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495570 msgr2=0x561b51495950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1185d9b700 1 --2- 172.21.15.46:0/2577339463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495570 0x561b51495950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1185d9b700 1 -- 172.21.15.46:0/2577339463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 msgr2=0x561b514a31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1185d9b700 1 --2- 172.21.15.46:0/2577339463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b514a31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1185d9b700 1 -- 172.21.15.46:0/2577339463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b5126a350 con 0x561b51495e90 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1186d9d700 1 --2- 172.21.15.46:0/2577339463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b514a31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:58:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1185d9b700 1 --2- 172.21.15.46:0/2577339463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495e90 0x561b514a0870 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f117c00bd30 tx=0x7f117c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd41f492b579f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1174ff9700 1 -- 172.21.15.46:0/2577339463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c010e10 con 0x561b51495e90 2024-03-18T20:58:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1174ff9700 1 -- 172.21.15.46:0/2577339463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f117c015070 con 0x561b51495e90 2024-03-18T20:58:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.366+0000 7f1174ff9700 1 -- 172.21.15.46:0/2577339463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c003e50 con 0x561b51495e90 2024-03-18T20:58:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.367+0000 7f1198e47b80 1 -- 172.21.15.46:0/2577339463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495e90 msgr2=0x561b514a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.367+0000 7f1198e47b80 1 --2- 172.21.15.46:0/2577339463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495e90 0x561b514a0870 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f117c00bd30 tx=0x7f117c0106b0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.367+0000 7f1198e47b80 1 -- 172.21.15.46:0/2577339463 shutdown_connections 2024-03-18T20:58:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.367+0000 7f1198e47b80 1 --2- 172.21.15.46:0/2577339463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b514a31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.367+0000 7f1198e47b80 1 --2- 172.21.15.46:0/2577339463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495570 0x561b51495950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.367+0000 7f1198e47b80 1 --2- 172.21.15.46:0/2577339463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495e90 0x561b514a0870 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.367+0000 7f1198e47b80 1 -- 172.21.15.46:0/2577339463 >> 172.21.15.46:0/2577339463 conn(0x561b5140c1c0 msgr2=0x561b51491a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.367+0000 7f1198e47b80 1 -- 172.21.15.46:0/2577339463 shutdown_connections 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.367+0000 7f1198e47b80 1 -- 172.21.15.46:0/2577339463 wait complete. 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.367+0000 7f1198e47b80 1 -- start start 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f1198e47b80 1 Processor -- start 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f1198e47b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 0x561b5149db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f1198e47b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 0x561b5149e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f1198e47b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b5149e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f1198e47b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b51384d20 con 0x561b51495570 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f1198e47b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b513849c0 con 0x561b514a0db0 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f1198e47b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5126c640 con 0x561b51495e90 2024-03-18T20:58:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f1198e47b80 10 client.0 _ll_register_callbacks cb 0x561b511f0340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f118659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 0x561b5149db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f118659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 0x561b5149db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45126/0 (socket says 172.21.15.46:45126) 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f118659c700 1 -- 172.21.15.46:0/170237382 learned_addr learned my addr 172.21.15.46:0/170237382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f1185d9b700 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 0x561b5149e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f1186d9d700 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b5149e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f118659c700 1 -- 172.21.15.46:0/170237382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 msgr2=0x561b5149e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f118659c700 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 0x561b5149e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f118659c700 1 -- 172.21.15.46:0/170237382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 msgr2=0x561b5149e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f118659c700 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b5149e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f118659c700 1 -- 172.21.15.46:0/170237382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1180029070 con 0x561b51495570 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.368+0000 7f1186d9d700 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b5149e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f118659c700 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 0x561b5149db90 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f117801e8a0 tx=0x7f11780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd50aa8fa0c4e53 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f116dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:58:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f116dffb700 1 -- 172.21.15.46:0/170237382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178004020 con 0x561b51495570 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f116dffb700 1 -- 172.21.15.46:0/170237382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f1178004180 con 0x561b51495570 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f1198e47b80 1 -- 172.21.15.46:0/170237382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b514896c0 con 0x561b51495570 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f116dffb700 1 -- 172.21.15.46:0/170237382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1178004340 con 0x561b51495570 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f1198e47b80 10 client.20679 fetch_fsmap learned FSMap version 92 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f1198e47b80 1 -- 172.21.15.46:0/170237382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x7f117c00b960 con 0x561b51495570 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f116dffb700 1 -- 172.21.15.46:0/170237382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b514896c0 con 0x561b51495570 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f116dffb700 1 -- 172.21.15.46:0/170237382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 92) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f117800e710 con 0x561b51495570 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f1198e47b80 10 client.20679 fetch_fsmap finished waiting for FSMap version 92 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f1198e47b80 10 client.20679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f1198e47b80 20 client.20679 populate_metadata read hostname 'smithi046' 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f1198e47b80 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561b514b0a40 0x561b514b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f1198e47b80 4 client.20679 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:58:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.369+0000 7f1198e47b80 1 -- 172.21.15.46:0/170237382 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b513e16a0 con 0x561b514b0a40 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.370+0000 7f1185d9b700 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561b514b0a40 0x561b514b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.370+0000 7f1185d9b700 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561b514b0a40 0x561b514b2e60 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8ed170b475656b4 server_cookie=a4688e80072766d5 in_seq=0 out_seq=0 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.370+0000 7f116dffb700 10 client.20679 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.371+0000 7f116dffb700 1 -- 172.21.15.46:0/170237382 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561b513e16a0 con 0x561b514b0a40 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.371+0000 7f116dffb700 10 client.20679 handle_command_reply: tid=0 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.371+0000 7f1198e47b80 1 client.20679 shutdown 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.371+0000 7f1198e47b80 1 -- 172.21.15.46:0/170237382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 msgr2=0x561b5149db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.371+0000 7f1198e47b80 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 0x561b5149db90 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f117801e8a0 tx=0x7f11780047c0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.372+0000 7f1198e47b80 1 -- 172.21.15.46:0/170237382 shutdown_connections 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.372+0000 7f1198e47b80 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561b514b0a40 0x561b514b2e60 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.372+0000 7f1198e47b80 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 0x561b5149db90 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.372+0000 7f1198e47b80 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b5149e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.372+0000 7f116dffb700 0 client.20679 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.372+0000 7f1198e47b80 1 --2- 172.21.15.46:0/170237382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 0x561b5149e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.372+0000 7f1198e47b80 1 -- 172.21.15.46:0/170237382 >> 172.21.15.46:0/170237382 conn(0x561b5140c1c0 msgr2=0x561b51491650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.372+0000 7f1198e47b80 1 -- 172.21.15.46:0/170237382 shutdown_connections 2024-03-18T20:58:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.372+0000 7f1198e47b80 1 -- 172.21.15.46:0/170237382 wait complete. 2024-03-18T20:58:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.372+0000 7f1198e47b80 20 client.20679 trim_cache size 0 max 16384 2024-03-18T20:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.387+0000 7f1198e47b80 1 Processor -- start 2024-03-18T20:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.387+0000 7f1198e47b80 1 -- start start 2024-03-18T20:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1198e47b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b51495570 0x561b514acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1198e47b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 0x561b514a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1198e47b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b514a0db0 0x561b5149e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1198e47b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b51492d60 con 0x561b514a0db0 2024-03-18T20:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1198e47b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5148bc10 con 0x561b51495570 2024-03-18T20:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1198e47b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b51498a40 con 0x561b51495e90 2024-03-18T20:58:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1186d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b514a0db0 0x561b5149e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1186d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b514a0db0 0x561b5149e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45130/0 (socket says 172.21.15.46:45130) 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1186d9d700 1 -- 172.21.15.46:0/113558432 learned_addr learned my addr 172.21.15.46:0/113558432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f118659c700 1 --2- 172.21.15.46:0/113558432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b51495570 0x561b514acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1185d9b700 1 --2- 172.21.15.46:0/113558432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 0x561b514a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1186d9d700 1 -- 172.21.15.46:0/113558432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 msgr2=0x561b514a7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1186d9d700 1 --2- 172.21.15.46:0/113558432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 0x561b514a7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1186d9d700 1 -- 172.21.15.46:0/113558432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b51495570 msgr2=0x561b514acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1186d9d700 1 --2- 172.21.15.46:0/113558432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b51495570 0x561b514acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f1186d9d700 1 -- 172.21.15.46:0/113558432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b51216b00 con 0x561b514a0db0 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.388+0000 7f118659c700 1 --2- 172.21.15.46:0/113558432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b51495570 0x561b514acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f1186d9d700 1 --2- 172.21.15.46:0/113558432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b514a0db0 0x561b5149e450 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f1180096b90 tx=0x7f118000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c4b093f339dd68 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f11757fa700 1 -- 172.21.15.46:0/113558432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118009ad90 con 0x561b514a0db0 2024-03-18T20:58:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f11757fa700 1 -- 172.21.15.46:0/113558432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1180083e90 con 0x561b514a0db0 2024-03-18T20:58:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f11757fa700 1 -- 172.21.15.46:0/113558432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11800a2b10 con 0x561b514a0db0 2024-03-18T20:58:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f1198e47b80 1 -- 172.21.15.46:0/113558432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b514a0db0 msgr2=0x561b5149e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f1198e47b80 1 --2- 172.21.15.46:0/113558432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b514a0db0 0x561b5149e450 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f1180096b90 tx=0x7f118000b3e0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f1198e47b80 1 -- 172.21.15.46:0/113558432 shutdown_connections 2024-03-18T20:58:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f1198e47b80 1 --2- 172.21.15.46:0/113558432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b514a0db0 0x561b5149e450 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f1198e47b80 1 --2- 172.21.15.46:0/113558432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b51495570 0x561b514acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f1198e47b80 1 --2- 172.21.15.46:0/113558432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 0x561b514a7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f1198e47b80 1 -- 172.21.15.46:0/113558432 >> 172.21.15.46:0/113558432 conn(0x561b5140c1c0 msgr2=0x561b51490120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f1198e47b80 1 -- 172.21.15.46:0/113558432 shutdown_connections 2024-03-18T20:58:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.389+0000 7f1198e47b80 1 -- 172.21.15.46:0/113558432 wait complete. 2024-03-18T20:58:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.390+0000 7f1198e47b80 1 -- start start 2024-03-18T20:58:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.390+0000 7f1198e47b80 1 Processor -- start 2024-03-18T20:58:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.390+0000 7f1198e47b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 0x561b514a5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.390+0000 7f1198e47b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 0x561b514a5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.390+0000 7f1198e47b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b514a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.390+0000 7f1198e47b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b51503950 con 0x561b51495570 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.390+0000 7f1198e47b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b51503ac0 con 0x561b514a0db0 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.390+0000 7f1198e47b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b51503c80 con 0x561b51495e90 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.390+0000 7f1198e47b80 10 client.0 _ll_register_callbacks cb 0x7f117801e3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.390+0000 7f118659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 0x561b514a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.390+0000 7f118659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 0x561b514a5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45132/0 (socket says 172.21.15.46:45132) 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.390+0000 7f118659c700 1 -- 172.21.15.46:0/1715570724 learned_addr learned my addr 172.21.15.46:0/1715570724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f1186d9d700 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b514a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f1185d9b700 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 0x561b514a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f118659c700 1 -- 172.21.15.46:0/1715570724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 msgr2=0x561b514a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f118659c700 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 0x561b514a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f118659c700 1 -- 172.21.15.46:0/1715570724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 msgr2=0x561b514a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f118659c700 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b514a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f118659c700 1 -- 172.21.15.46:0/1715570724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c00b960 con 0x561b51495570 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f118659c700 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 0x561b514a5340 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1178005950 tx=0x7f117800e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e560db4ade4aaa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f116dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f116dffb700 1 -- 172.21.15.46:0/1715570724 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178009950 con 0x561b51495570 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f116dffb700 1 -- 172.21.15.46:0/1715570724 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f1178036030 con 0x561b51495570 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f1198e47b80 1 -- 172.21.15.46:0/1715570724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f117c015070 con 0x561b51495570 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f116dffb700 1 -- 172.21.15.46:0/1715570724 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f117800e850 con 0x561b51495570 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f1198e47b80 10 client.20685 fetch_fsmap learned FSMap version 92 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.391+0000 7f1198e47b80 1 -- 172.21.15.46:0/1715570724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x7f118819c810 con 0x561b51495570 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.392+0000 7f116dffb700 1 -- 172.21.15.46:0/1715570724 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f117c015070 con 0x561b51495570 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.392+0000 7f116dffb700 1 -- 172.21.15.46:0/1715570724 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 92) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1178028930 con 0x561b51495570 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.392+0000 7f1198e47b80 10 client.20685 fetch_fsmap finished waiting for FSMap version 92 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.392+0000 7f1198e47b80 10 client.20685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.392+0000 7f1198e47b80 20 client.20685 populate_metadata read hostname 'smithi046' 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.392+0000 7f1198e47b80 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561b51505380 0x561b515077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.392+0000 7f1198e47b80 4 client.20685 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:58:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.392+0000 7f1198e47b80 1 -- 172.21.15.46:0/1715570724 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561b51237c90 con 0x561b51505380 2024-03-18T20:58:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.392+0000 7f1185d9b700 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561b51505380 0x561b515077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.392+0000 7f116dffb700 10 client.20685 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.392+0000 7f1185d9b700 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561b51505380 0x561b515077a0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=839e036b676f107a server_cookie=f8f7d43f4943aa5f in_seq=0 out_seq=0 2024-03-18T20:58:08.454 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355305 32.80 MB/sec execute 404 sec latency 2734.240 ms 2024-03-18T20:58:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f116dffb700 1 -- 172.21.15.46:0/1715570724 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561b51237c90 con 0x561b51505380 2024-03-18T20:58:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f116dffb700 10 client.20685 handle_command_reply: tid=0 2024-03-18T20:58:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f1198e47b80 1 client.20685 shutdown 2024-03-18T20:58:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f1198e47b80 1 -- 172.21.15.46:0/1715570724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 msgr2=0x561b514a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f1198e47b80 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 0x561b514a5340 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1178005950 tx=0x7f117800e660 comp rx=0 tx=0).stop 2024-03-18T20:58:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f1198e47b80 1 -- 172.21.15.46:0/1715570724 shutdown_connections 2024-03-18T20:58:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f1198e47b80 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x561b51505380 0x561b515077a0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f1198e47b80 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b51495570 0x561b514a5340 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f1198e47b80 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561b514a0db0 0x561b514a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f1198e47b80 1 --2- 172.21.15.46:0/1715570724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561b51495e90 0x561b514a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f1198e47b80 1 -- 172.21.15.46:0/1715570724 >> 172.21.15.46:0/1715570724 conn(0x561b5140c1c0 msgr2=0x561b5149e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f116dffb700 0 client.20685 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f1198e47b80 1 -- 172.21.15.46:0/1715570724 shutdown_connections 2024-03-18T20:58:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f1198e47b80 1 -- 172.21.15.46:0/1715570724 wait complete. 2024-03-18T20:58:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f1198e47b80 20 client.20685 trim_cache size 0 max 16384 2024-03-18T20:58:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:58:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:58:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:58:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:58:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f118ccb2700 1 -- 172.21.15.46:0/2684040075 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f11700a1310 msgr2=0x7f11700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f118ccb2700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f11700a1310 0x7f11700a37d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1178009be0 tx=0x7f1178009260 comp rx=0 tx=0).stop 2024-03-18T20:58:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f118ccb2700 1 -- 172.21.15.46:0/2684040075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1188106e30 msgr2=0x7f118819bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f118ccb2700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1188106e30 0x7f118819bc20 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f118000bcd0 tx=0x7f118000bd00 comp rx=0 tx=0).stop 2024-03-18T20:58:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f118ccb2700 1 -- 172.21.15.46:0/2684040075 shutdown_connections 2024-03-18T20:58:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f118ccb2700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f11700a1310 0x7f11700a37d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f118ccb2700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1188106e30 0x7f118819bc20 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f118ccb2700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11880ff220 0x7f1188197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f118ccb2700 1 --2- 172.21.15.46:0/2684040075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11880ffbe0 0x7f1188197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f118ccb2700 1 -- 172.21.15.46:0/2684040075 >> 172.21.15.46:0/2684040075 conn(0x7f11880761b0 msgr2=0x7f1188103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f118ccb2700 1 -- 172.21.15.46:0/2684040075 shutdown_connections 2024-03-18T20:58:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:08.531+0000 7f118ccb2700 1 -- 172.21.15.46:0/2684040075 wait complete. 2024-03-18T20:58:08.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:08 smithi155 ceph-mon[48344]: pgmap v695: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 344 op/s 2024-03-18T20:58:08.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:08 smithi046 ceph-mon[43495]: pgmap v695: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 344 op/s 2024-03-18T20:58:08.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:08 smithi073 ceph-mon[47384]: pgmap v695: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 344 op/s 2024-03-18T20:58:09.454 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355305 32.72 MB/sec execute 405 sec latency 3734.297 ms 2024-03-18T20:58:10.454 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355462 32.64 MB/sec execute 406 sec latency 4325.119 ms 2024-03-18T20:58:10.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:10 smithi155 ceph-mon[48344]: pgmap v696: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 391 op/s 2024-03-18T20:58:10.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:10 smithi046 ceph-mon[43495]: pgmap v696: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 391 op/s 2024-03-18T20:58:10.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:10 smithi073 ceph-mon[47384]: pgmap v696: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.1 MiB/s wr, 391 op/s 2024-03-18T20:58:11.458 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355839 32.58 MB/sec execute 407 sec latency 184.562 ms 2024-03-18T20:58:12.458 INFO:tasks.workunit.client.0.smithi046.stdout: 10 355951 32.51 MB/sec execute 408 sec latency 228.146 ms 2024-03-18T20:58:12.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:12 smithi155 ceph-mon[48344]: pgmap v697: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-03-18T20:58:12.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:12 smithi046 ceph-mon[43495]: pgmap v697: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-03-18T20:58:12.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:12 smithi073 ceph-mon[47384]: pgmap v697: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-03-18T20:58:13.458 INFO:tasks.workunit.client.0.smithi046.stdout: 10 356253 32.50 MB/sec execute 409 sec latency 225.119 ms 2024-03-18T20:58:14.460 INFO:tasks.workunit.client.0.smithi046.stdout: 10 356410 32.44 MB/sec execute 410 sec latency 435.804 ms 2024-03-18T20:58:14.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:14 smithi155 ceph-mon[48344]: pgmap v698: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-03-18T20:58:14.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:14 smithi155 ceph-mon[48344]: osdmap e114: 12 total, 12 up, 12 in 2024-03-18T20:58:14.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:14 smithi046 ceph-mon[43495]: pgmap v698: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-03-18T20:58:14.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:14 smithi046 ceph-mon[43495]: osdmap e114: 12 total, 12 up, 12 in 2024-03-18T20:58:14.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:14 smithi073 ceph-mon[47384]: pgmap v698: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-03-18T20:58:14.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:14 smithi073 ceph-mon[47384]: osdmap e114: 12 total, 12 up, 12 in 2024-03-18T20:58:15.461 INFO:tasks.workunit.client.0.smithi046.stdout: 10 356641 32.40 MB/sec execute 411 sec latency 575.387 ms 2024-03-18T20:58:16.462 INFO:tasks.workunit.client.0.smithi046.stdout: 10 357047 32.37 MB/sec execute 412 sec latency 111.940 ms 2024-03-18T20:58:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:16 smithi155 ceph-mon[48344]: pgmap v700: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 390 op/s 2024-03-18T20:58:16.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:16 smithi046 ceph-mon[43495]: pgmap v700: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 390 op/s 2024-03-18T20:58:16.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:16 smithi073 ceph-mon[47384]: pgmap v700: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 390 op/s 2024-03-18T20:58:17.462 INFO:tasks.workunit.client.0.smithi046.stdout: 10 357648 32.37 MB/sec execute 413 sec latency 89.023 ms 2024-03-18T20:58:18.462 INFO:tasks.workunit.client.0.smithi046.stdout: 10 357922 32.34 MB/sec execute 414 sec latency 547.981 ms 2024-03-18T20:58:18.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:18 smithi155 ceph-mon[48344]: pgmap v701: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 390 op/s 2024-03-18T20:58:18.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:18 smithi046 ceph-mon[43495]: pgmap v701: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 390 op/s 2024-03-18T20:58:18.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:18 smithi073 ceph-mon[47384]: pgmap v701: 97 pgs: 12 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 57 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 390 op/s 2024-03-18T20:58:19.462 INFO:tasks.workunit.client.0.smithi046.stdout: 10 358019 32.28 MB/sec execute 415 sec latency 1071.590 ms 2024-03-18T20:58:20.462 INFO:tasks.workunit.client.0.smithi046.stdout: 10 358234 32.23 MB/sec execute 416 sec latency 231.296 ms 2024-03-18T20:58:20.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:20 smithi155 ceph-mon[48344]: pgmap v702: 97 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 78 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 555 op/s 2024-03-18T20:58:20.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:20 smithi046 ceph-mon[43495]: pgmap v702: 97 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 78 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 555 op/s 2024-03-18T20:58:20.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:20 smithi073 ceph-mon[47384]: pgmap v702: 97 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 78 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 555 op/s 2024-03-18T20:58:21.462 INFO:tasks.workunit.client.0.smithi046.stdout: 10 358586 32.19 MB/sec execute 417 sec latency 245.236 ms 2024-03-18T20:58:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.693+0000 7f4795956700 1 -- 172.21.15.46:0/2785614926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4790103c00 msgr2=0x7f4790104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.693+0000 7f4795956700 1 --2- 172.21.15.46:0/2785614926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4790103c00 0x7f4790104080 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4778009990 tx=0x7f4778009ca0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.693+0000 7f4795956700 1 -- 172.21.15.46:0/2785614926 shutdown_connections 2024-03-18T20:58:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.693+0000 7f4795956700 1 --2- 172.21.15.46:0/2785614926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47900680e0 0x7f4790068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.693+0000 7f4795956700 1 --2- 172.21.15.46:0/2785614926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4790105c40 0x7f47901036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.693+0000 7f4795956700 1 --2- 172.21.15.46:0/2785614926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4790103c00 0x7f4790104080 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.693+0000 7f4795956700 1 -- 172.21.15.46:0/2785614926 >> 172.21.15.46:0/2785614926 conn(0x7f4790076d50 msgr2=0x7f4790077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.693+0000 7f4795956700 1 -- 172.21.15.46:0/2785614926 shutdown_connections 2024-03-18T20:58:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.693+0000 7f4795956700 1 -- 172.21.15.46:0/2785614926 wait complete. 2024-03-18T20:58:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.694+0000 7f4795956700 1 Processor -- start 2024-03-18T20:58:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.694+0000 7f4795956700 1 -- start start 2024-03-18T20:58:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.694+0000 7f4795956700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47900680e0 0x7f4790113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.694+0000 7f4795956700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4790103c00 0x7f4790114360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.694+0000 7f4795956700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4790105c40 0x7f4790111a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.694+0000 7f4795956700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4790112210 con 0x7f4790105c40 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.694+0000 7f4795956700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4790112380 con 0x7f4790103c00 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.694+0000 7f4795956700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4790112540 con 0x7f47900680e0 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.694+0000 7f478effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47900680e0 0x7f4790113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.694+0000 7f478effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47900680e0 0x7f4790113e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:49164/0 (socket says 172.21.15.46:49164) 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.694+0000 7f478effd700 1 -- 172.21.15.46:0/2872138713 learned_addr learned my addr 172.21.15.46:0/2872138713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.694+0000 7f478e7fc700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4790103c00 0x7f4790114360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.695+0000 7f478e7fc700 1 -- 172.21.15.46:0/2872138713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47900680e0 msgr2=0x7f4790113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.695+0000 7f478e7fc700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47900680e0 0x7f4790113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.695+0000 7f478e7fc700 1 -- 172.21.15.46:0/2872138713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4790105c40 msgr2=0x7f4790111a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.695+0000 7f478e7fc700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4790105c40 0x7f4790111a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.695+0000 7f478e7fc700 1 -- 172.21.15.46:0/2872138713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47780095f0 con 0x7f4790103c00 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.695+0000 7f478effd700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47900680e0 0x7f4790113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:58:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.695+0000 7f478e7fc700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4790103c00 0x7f4790114360 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f477800b2e0 tx=0x7f4778005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.695+0000 7f4794954700 1 -- 172.21.15.46:0/2872138713 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477801d070 con 0x7f4790103c00 2024-03-18T20:58:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.695+0000 7f4795956700 1 -- 172.21.15.46:0/2872138713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47901127c0 con 0x7f4790103c00 2024-03-18T20:58:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.695+0000 7f4794954700 1 -- 172.21.15.46:0/2872138713 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47780173f0 con 0x7f4790103c00 2024-03-18T20:58:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.695+0000 7f4794954700 1 -- 172.21.15.46:0/2872138713 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477800e640 con 0x7f4790103c00 2024-03-18T20:58:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.695+0000 7f4795956700 1 -- 172.21.15.46:0/2872138713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479010d260 con 0x7f4790103c00 2024-03-18T20:58:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.699+0000 7f4794954700 1 -- 172.21.15.46:0/2872138713 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f4778017560 con 0x7f4790103c00 2024-03-18T20:58:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.699+0000 7f4794954700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f477c0a1290 0x7f477c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.700+0000 7f4794954700 1 -- 172.21.15.46:0/2872138713 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f47780ce090 con 0x7f4790103c00 2024-03-18T20:58:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.700+0000 7f478effd700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f477c0a1290 0x7f477c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.701+0000 7f478effd700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f477c0a1290 0x7f477c0a3750 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4790103b60 tx=0x7f4780009290 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.701+0000 7f47a1aebb80 1 Processor -- start 2024-03-18T20:58:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.702+0000 7f47a1aebb80 1 -- start start 2024-03-18T20:58:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.702+0000 7f47a1aebb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3645570 0x55bbc3645950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.702+0000 7f47a1aebb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645e90 0x55bbc3650870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.702+0000 7f47a1aebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3650db0 0x55bbc36531a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.702+0000 7f47a1aebb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc359f930 con 0x55bbc3645570 2024-03-18T20:58:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.702+0000 7f47a1aebb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc359d2f0 con 0x55bbc3645e90 2024-03-18T20:58:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.702+0000 7f47a1aebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc3598be0 con 0x55bbc3650db0 2024-03-18T20:58:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f478effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3645570 0x55bbc3645950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f478e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645e90 0x55bbc3650870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f478e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645e90 0x55bbc3650870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:34140/0 (socket says 172.21.15.46:34140) 2024-03-18T20:58:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f478e7fc700 1 -- 172.21.15.46:0/4184159857 learned_addr learned my addr 172.21.15.46:0/4184159857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f478e7fc700 1 -- 172.21.15.46:0/4184159857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3650db0 msgr2=0x55bbc36531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f478e7fc700 1 --2- 172.21.15.46:0/4184159857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3650db0 0x55bbc36531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f478e7fc700 1 -- 172.21.15.46:0/4184159857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3645570 msgr2=0x55bbc3645950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f478e7fc700 1 --2- 172.21.15.46:0/4184159857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3645570 0x55bbc3645950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f478e7fc700 1 -- 172.21.15.46:0/4184159857 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbc341a350 con 0x55bbc3645e90 2024-03-18T20:58:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f478e7fc700 1 --2- 172.21.15.46:0/4184159857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645e90 0x55bbc3650870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f47780976e0 tx=0x7f477809add0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3481450153ae7017 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f4785ffb700 1 -- 172.21.15.46:0/4184159857 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477809b8e0 con 0x55bbc3645e90 2024-03-18T20:58:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f4785ffb700 1 -- 172.21.15.46:0/4184159857 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477809b1e0 con 0x55bbc3645e90 2024-03-18T20:58:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.703+0000 7f4785ffb700 1 -- 172.21.15.46:0/4184159857 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778017880 con 0x55bbc3645e90 2024-03-18T20:58:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.704+0000 7f47a1aebb80 1 -- 172.21.15.46:0/4184159857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645e90 msgr2=0x55bbc3650870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.704+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/4184159857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645e90 0x55bbc3650870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f47780976e0 tx=0x7f477809add0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.704+0000 7f47a1aebb80 1 -- 172.21.15.46:0/4184159857 shutdown_connections 2024-03-18T20:58:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.704+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/4184159857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3645570 0x55bbc3645950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.704+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/4184159857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3650db0 0x55bbc36531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.704+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/4184159857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645e90 0x55bbc3650870 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.704+0000 7f47a1aebb80 1 -- 172.21.15.46:0/4184159857 >> 172.21.15.46:0/4184159857 conn(0x55bbc35bc1c0 msgr2=0x55bbc3641a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.705+0000 7f47a1aebb80 1 -- 172.21.15.46:0/4184159857 shutdown_connections 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.705+0000 7f47a1aebb80 1 -- 172.21.15.46:0/4184159857 wait complete. 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.705+0000 7f47a1aebb80 1 -- start start 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.705+0000 7f47a1aebb80 1 Processor -- start 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f47a1aebb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3645570 0x55bbc3650820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f47a1aebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc364db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f47a1aebb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3650db0 0x55bbc364e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f47a1aebb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc3534d20 con 0x55bbc3645570 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f47a1aebb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc35349c0 con 0x55bbc3650db0 2024-03-18T20:58:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f47a1aebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc341c640 con 0x55bbc3645e90 2024-03-18T20:58:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f47a1aebb80 10 client.0 _ll_register_callbacks cb 0x55bbc33a0340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:58:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f478e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc364db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f478e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc364db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:49176/0 (socket says 172.21.15.46:49176) 2024-03-18T20:58:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f478e7fc700 1 -- 172.21.15.46:0/3381796902 learned_addr learned my addr 172.21.15.46:0/3381796902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f478f7fe700 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3650db0 0x55bbc364e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f478effd700 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3645570 0x55bbc3650820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f478e7fc700 1 -- 172.21.15.46:0/3381796902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3650db0 msgr2=0x55bbc364e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f478e7fc700 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3650db0 0x55bbc364e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f478e7fc700 1 -- 172.21.15.46:0/3381796902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3645570 msgr2=0x55bbc3650820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f478e7fc700 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3645570 0x55bbc3650820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.706+0000 7f478e7fc700 1 -- 172.21.15.46:0/3381796902 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479010d260 con 0x55bbc3645e90 2024-03-18T20:58:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.707+0000 7f478e7fc700 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc364db40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f47780ce370 tx=0x7f47780cb700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=495021d7b27c6097 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.707+0000 7f476effd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:58:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.707+0000 7f476effd700 1 -- 172.21.15.46:0/3381796902 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778017a80 con 0x55bbc3645e90 2024-03-18T20:58:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.707+0000 7f47a1aebb80 1 -- 172.21.15.46:0/3381796902 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bbc36396c0 con 0x55bbc3645e90 2024-03-18T20:58:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.707+0000 7f476effd700 1 -- 172.21.15.46:0/3381796902 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f477809b8e0 con 0x55bbc3645e90 2024-03-18T20:58:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.707+0000 7f476effd700 1 -- 172.21.15.46:0/3381796902 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47780a10b0 con 0x55bbc3645e90 2024-03-18T20:58:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.708+0000 7f47a1aebb80 10 client.30418 fetch_fsmap learned FSMap version 92 2024-03-18T20:58:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.708+0000 7f47a1aebb80 1 -- 172.21.15.46:0/3381796902 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x55bbc33c6b00 con 0x55bbc3645e90 2024-03-18T20:58:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.708+0000 7f476effd700 1 -- 172.21.15.46:0/3381796902 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bbc36396c0 con 0x55bbc3645e90 2024-03-18T20:58:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.708+0000 7f476effd700 1 -- 172.21.15.46:0/3381796902 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 92) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f477809a760 con 0x55bbc3645e90 2024-03-18T20:58:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.708+0000 7f47a1aebb80 10 client.30418 fetch_fsmap finished waiting for FSMap version 92 2024-03-18T20:58:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.708+0000 7f47a1aebb80 10 client.30418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:58:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.708+0000 7f47a1aebb80 20 client.30418 populate_metadata read hostname 'smithi046' 2024-03-18T20:58:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.708+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bbc36601a0 0x55bbc3662590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.708+0000 7f47a1aebb80 4 client.30418 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:58:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.708+0000 7f47a1aebb80 1 -- 172.21.15.46:0/3381796902 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbc35916a0 con 0x55bbc36601a0 2024-03-18T20:58:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.708+0000 7f478effd700 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bbc36601a0 0x55bbc3662590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.709+0000 7f478effd700 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bbc36601a0 0x55bbc3662590 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aec8ff7f60f3b9c server_cookie=e6236c2fa69d526e in_seq=0 out_seq=0 2024-03-18T20:58:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.709+0000 7f476effd700 10 client.30418 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f476effd700 1 -- 172.21.15.46:0/3381796902 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bbc35916a0 con 0x55bbc36601a0 2024-03-18T20:58:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f476effd700 10 client.30418 handle_command_reply: tid=0 2024-03-18T20:58:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f47a1aebb80 1 client.30418 shutdown 2024-03-18T20:58:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f47a1aebb80 1 -- 172.21.15.46:0/3381796902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 msgr2=0x55bbc364db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc364db40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f47780ce370 tx=0x7f47780cb700 comp rx=0 tx=0).stop 2024-03-18T20:58:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f47a1aebb80 1 -- 172.21.15.46:0/3381796902 shutdown_connections 2024-03-18T20:58:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bbc36601a0 0x55bbc3662590 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3645570 0x55bbc3650820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3650db0 0x55bbc364e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/3381796902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc364db40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f47a1aebb80 1 -- 172.21.15.46:0/3381796902 >> 172.21.15.46:0/3381796902 conn(0x55bbc35bc1c0 msgr2=0x55bbc3641650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f476effd700 0 client.30418 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f47a1aebb80 1 -- 172.21.15.46:0/3381796902 shutdown_connections 2024-03-18T20:58:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f47a1aebb80 1 -- 172.21.15.46:0/3381796902 wait complete. 2024-03-18T20:58:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.711+0000 7f47a1aebb80 20 client.30418 trim_cache size 0 max 16384 2024-03-18T20:58:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f47a1aebb80 1 Processor -- start 2024-03-18T20:58:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f47a1aebb80 1 -- start start 2024-03-18T20:58:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f47a1aebb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645570 0x55bbc364cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f47a1aebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc36551c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f47a1aebb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3650db0 0x55bbc3650350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f47a1aebb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc3643060 con 0x55bbc3650db0 2024-03-18T20:58:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f47a1aebb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc3646640 con 0x55bbc3645570 2024-03-18T20:58:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f47a1aebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc364d210 con 0x55bbc3645e90 2024-03-18T20:58:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f478effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645570 0x55bbc364cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f478e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc36551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f478effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645570 0x55bbc364cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:34162/0 (socket says 172.21.15.46:34162) 2024-03-18T20:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f478effd700 1 -- 172.21.15.46:0/2625652157 learned_addr learned my addr 172.21.15.46:0/2625652157 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.731+0000 7f478f7fe700 1 --2- 172.21.15.46:0/2625652157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3650db0 0x55bbc3650350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.732+0000 7f478e7fc700 1 -- 172.21.15.46:0/2625652157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645570 msgr2=0x55bbc364cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.732+0000 7f478e7fc700 1 --2- 172.21.15.46:0/2625652157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645570 0x55bbc364cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.732+0000 7f478e7fc700 1 -- 172.21.15.46:0/2625652157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3650db0 msgr2=0x55bbc3650350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.732+0000 7f478e7fc700 1 --2- 172.21.15.46:0/2625652157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3650db0 0x55bbc3650350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.732+0000 7f478e7fc700 1 -- 172.21.15.46:0/2625652157 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbc355ee40 con 0x55bbc3645e90 2024-03-18T20:58:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.732+0000 7f478e7fc700 1 --2- 172.21.15.46:0/2625652157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc36551c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f477809aee0 tx=0x7f4778009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8941d5d05dad496 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.732+0000 7f47867fc700 1 -- 172.21.15.46:0/2625652157 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778017a80 con 0x55bbc3645e90 2024-03-18T20:58:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.732+0000 7f47867fc700 1 -- 172.21.15.46:0/2625652157 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47780a32b0 con 0x55bbc3645e90 2024-03-18T20:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.732+0000 7f47867fc700 1 -- 172.21.15.46:0/2625652157 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477809b6a0 con 0x55bbc3645e90 2024-03-18T20:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.733+0000 7f47a1aebb80 1 -- 172.21.15.46:0/2625652157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 msgr2=0x55bbc36551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.733+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/2625652157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc36551c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f477809aee0 tx=0x7f4778009310 comp rx=0 tx=0).stop 2024-03-18T20:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.734+0000 7f47a1aebb80 1 -- 172.21.15.46:0/2625652157 shutdown_connections 2024-03-18T20:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.734+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/2625652157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3650db0 0x55bbc3650350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.734+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/2625652157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645570 0x55bbc364cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.734+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/2625652157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc36551c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.734+0000 7f47a1aebb80 1 -- 172.21.15.46:0/2625652157 >> 172.21.15.46:0/2625652157 conn(0x55bbc35bc1c0 msgr2=0x55bbc3647530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.734+0000 7f47a1aebb80 1 -- 172.21.15.46:0/2625652157 shutdown_connections 2024-03-18T20:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.734+0000 7f47a1aebb80 1 -- 172.21.15.46:0/2625652157 wait complete. 2024-03-18T20:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.734+0000 7f47a1aebb80 1 -- start start 2024-03-18T20:58:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.734+0000 7f47a1aebb80 1 Processor -- start 2024-03-18T20:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f47a1aebb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645570 0x55bbc36aef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f47a1aebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc36af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f47a1aebb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3650db0 0x55bbc3652c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f47a1aebb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc36afc00 con 0x55bbc3650db0 2024-03-18T20:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f47a1aebb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc36afd70 con 0x55bbc3645570 2024-03-18T20:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f47a1aebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbc36531a0 con 0x55bbc3645e90 2024-03-18T20:58:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f47a1aebb80 10 client.0 _ll_register_callbacks cb 0x7f4780009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f478e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc36af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f478e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc36af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:49194/0 (socket says 172.21.15.46:49194) 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f478e7fc700 1 -- 172.21.15.46:0/1182959243 learned_addr learned my addr 172.21.15.46:0/1182959243 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f478e7fc700 1 -- 172.21.15.46:0/1182959243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645570 msgr2=0x55bbc36aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f478e7fc700 1 --2- 172.21.15.46:0/1182959243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645570 0x55bbc36aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f478e7fc700 1 -- 172.21.15.46:0/1182959243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3650db0 msgr2=0x55bbc3652c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f478e7fc700 1 --2- 172.21.15.46:0/1182959243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3650db0 0x55bbc3652c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.735+0000 7f478e7fc700 1 -- 172.21.15.46:0/1182959243 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbc33c6b00 con 0x55bbc3645e90 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f478e7fc700 1 --2- 172.21.15.46:0/1182959243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc36af490 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4778000c00 tx=0x7f4778099790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3df8b0d92c632e3d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f476effd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f476effd700 1 -- 172.21.15.46:0/1182959243 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477809b8e0 con 0x55bbc3645e90 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f476effd700 1 -- 172.21.15.46:0/1182959243 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8600+0+0 (secure 0 0 0) 0x7f477809a6d0 con 0x55bbc3645e90 2024-03-18T20:58:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f47a1aebb80 1 -- 172.21.15.46:0/1182959243 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f477809b1e0 con 0x55bbc3645e90 2024-03-18T20:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f476effd700 1 -- 172.21.15.46:0/1182959243 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f477809a890 con 0x55bbc3645e90 2024-03-18T20:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f47a1aebb80 10 client.30424 fetch_fsmap learned FSMap version 92 2024-03-18T20:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f47a1aebb80 1 -- 172.21.15.46:0/1182959243 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) v3 -- 0x55bbc341ec70 con 0x55bbc3645e90 2024-03-18T20:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f476effd700 1 -- 172.21.15.46:0/1182959243 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477809b1e0 con 0x55bbc3645e90 2024-03-18T20:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f476effd700 1 -- 172.21.15.46:0/1182959243 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 92) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4778083030 con 0x55bbc3645e90 2024-03-18T20:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f47a1aebb80 10 client.30424 fetch_fsmap finished waiting for FSMap version 92 2024-03-18T20:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f47a1aebb80 10 client.30424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f47a1aebb80 20 client.30424 populate_metadata read hostname 'smithi046' 2024-03-18T20:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/1182959243 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bbc36c57a0 0x55bbc36c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f47a1aebb80 4 client.30424 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.736+0000 7f47a1aebb80 1 -- 172.21.15.46:0/1182959243 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bbc33e7c90 con 0x55bbc36c57a0 2024-03-18T20:58:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.737+0000 7f478effd700 1 --2- 172.21.15.46:0/1182959243 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bbc36c57a0 0x55bbc36c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.737+0000 7f478effd700 1 --2- 172.21.15.46:0/1182959243 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bbc36c57a0 0x55bbc36c7b90 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2716d54ffbecd63 server_cookie=4882237e0dc90d65 in_seq=0 out_seq=0 2024-03-18T20:58:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.737+0000 7f476effd700 10 client.30424 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.928+0000 7f476effd700 1 -- 172.21.15.46:0/1182959243 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bbc33e7c90 con 0x55bbc36c57a0 2024-03-18T20:58:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.928+0000 7f476effd700 10 client.30424 handle_command_reply: tid=0 2024-03-18T20:58:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.928+0000 7f47a1aebb80 1 client.30424 shutdown 2024-03-18T20:58:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.928+0000 7f47a1aebb80 1 -- 172.21.15.46:0/1182959243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 msgr2=0x55bbc36af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.928+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/1182959243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc36af490 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4778000c00 tx=0x7f4778099790 comp rx=0 tx=0).stop 2024-03-18T20:58:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.928+0000 7f47a1aebb80 1 -- 172.21.15.46:0/1182959243 shutdown_connections 2024-03-18T20:58:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.928+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/1182959243 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bbc36c57a0 0x55bbc36c7b90 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.928+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/1182959243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bbc3650db0 0x55bbc3652c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.928+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/1182959243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bbc3645570 0x55bbc36aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.928+0000 7f47a1aebb80 1 --2- 172.21.15.46:0/1182959243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bbc3645e90 0x55bbc36af490 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.928+0000 7f47a1aebb80 1 -- 172.21.15.46:0/1182959243 >> 172.21.15.46:0/1182959243 conn(0x55bbc35bc1c0 msgr2=0x55bbc364cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.928+0000 7f476effd700 0 client.30424 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.929+0000 7f47a1aebb80 1 -- 172.21.15.46:0/1182959243 shutdown_connections 2024-03-18T20:58:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.929+0000 7f47a1aebb80 1 -- 172.21.15.46:0/1182959243 wait complete. 2024-03-18T20:58:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.929+0000 7f47a1aebb80 20 client.30424 trim_cache size 0 max 16384 2024-03-18T20:58:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:58:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:58:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:58:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:58:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.929+0000 7f4795956700 1 -- 172.21.15.46:0/2872138713 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f477c0a1290 msgr2=0x7f477c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.929+0000 7f4795956700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f477c0a1290 0x7f477c0a3750 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4790103b60 tx=0x7f4780009290 comp rx=0 tx=0).stop 2024-03-18T20:58:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.929+0000 7f4795956700 1 -- 172.21.15.46:0/2872138713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4790103c00 msgr2=0x7f4790114360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.929+0000 7f4795956700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4790103c00 0x7f4790114360 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f477800b2e0 tx=0x7f4778005740 comp rx=0 tx=0).stop 2024-03-18T20:58:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.930+0000 7f4795956700 1 -- 172.21.15.46:0/2872138713 shutdown_connections 2024-03-18T20:58:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.930+0000 7f4795956700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f477c0a1290 0x7f477c0a3750 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.930+0000 7f4795956700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4790105c40 0x7f4790111a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.930+0000 7f4795956700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47900680e0 0x7f4790113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.930+0000 7f4795956700 1 --2- 172.21.15.46:0/2872138713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4790103c00 0x7f4790114360 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.930+0000 7f4795956700 1 -- 172.21.15.46:0/2872138713 >> 172.21.15.46:0/2872138713 conn(0x7f4790076d50 msgr2=0x7f47900fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.930+0000 7f4795956700 1 -- 172.21.15.46:0/2872138713 shutdown_connections 2024-03-18T20:58:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:21.930+0000 7f4795956700 1 -- 172.21.15.46:0/2872138713 wait complete. 2024-03-18T20:58:22.462 INFO:tasks.workunit.client.0.smithi046.stdout: 10 358967 32.15 MB/sec execute 418 sec latency 118.801 ms 2024-03-18T20:58:22.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:22 smithi155 ceph-mon[48344]: pgmap v703: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 14 MiB/s wr, 641 op/s 2024-03-18T20:58:22.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:22 smithi046 ceph-mon[43495]: pgmap v703: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 14 MiB/s wr, 641 op/s 2024-03-18T20:58:22.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:22 smithi073 ceph-mon[47384]: pgmap v703: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 14 MiB/s wr, 641 op/s 2024-03-18T20:58:23.464 INFO:tasks.workunit.client.0.smithi046.stdout: 10 359605 32.15 MB/sec execute 419 sec latency 93.262 ms 2024-03-18T20:58:24.464 INFO:tasks.workunit.client.0.smithi046.stdout: 10 360344 32.19 MB/sec execute 420 sec latency 95.651 ms 2024-03-18T20:58:24.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:24 smithi155 ceph-mon[48344]: pgmap v704: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 15 MiB/s wr, 650 op/s 2024-03-18T20:58:24.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:24 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:58:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:24 smithi046 ceph-mon[43495]: pgmap v704: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 15 MiB/s wr, 650 op/s 2024-03-18T20:58:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:24 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:58:24.974 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:24 smithi073 ceph-mon[47384]: pgmap v704: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 15 MiB/s wr, 650 op/s 2024-03-18T20:58:24.975 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:24 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:58:25.464 INFO:tasks.workunit.client.0.smithi046.stdout: 10 360880 32.17 MB/sec execute 421 sec latency 945.276 ms 2024-03-18T20:58:26.464 INFO:tasks.workunit.client.0.smithi046.stdout: 10 361392 32.18 MB/sec execute 422 sec latency 1945.315 ms 2024-03-18T20:58:26.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:26 smithi155 ceph-mon[48344]: osdmap e115: 12 total, 12 up, 12 in 2024-03-18T20:58:26.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:26 smithi155 ceph-mon[48344]: pgmap v706: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 717 op/s 2024-03-18T20:58:26.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:26 smithi046 ceph-mon[43495]: osdmap e115: 12 total, 12 up, 12 in 2024-03-18T20:58:26.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:26 smithi046 ceph-mon[43495]: pgmap v706: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 717 op/s 2024-03-18T20:58:26.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:26 smithi073 ceph-mon[47384]: osdmap e115: 12 total, 12 up, 12 in 2024-03-18T20:58:26.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:26 smithi073 ceph-mon[47384]: pgmap v706: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 717 op/s 2024-03-18T20:58:27.465 INFO:tasks.workunit.client.0.smithi046.stdout: 10 361803 32.16 MB/sec execute 423 sec latency 2945.349 ms 2024-03-18T20:58:28.464 INFO:tasks.workunit.client.0.smithi046.stdout: 10 362263 32.13 MB/sec execute 424 sec latency 3945.381 ms 2024-03-18T20:58:28.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:28 smithi155 ceph-mon[48344]: pgmap v707: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 717 op/s 2024-03-18T20:58:28.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:28 smithi046 ceph-mon[43495]: pgmap v707: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 717 op/s 2024-03-18T20:58:28.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:28 smithi073 ceph-mon[47384]: pgmap v707: 97 pgs: 97 active+clean; 2.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 18 MiB/s wr, 717 op/s 2024-03-18T20:58:29.464 INFO:tasks.workunit.client.0.smithi046.stdout: 10 362774 32.12 MB/sec execute 425 sec latency 4945.784 ms 2024-03-18T20:58:29.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:29 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:58:29.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:29 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:58:29.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:29 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:58:30.464 INFO:tasks.workunit.client.0.smithi046.stdout: 10 363299 32.10 MB/sec execute 426 sec latency 4982.192 ms 2024-03-18T20:58:30.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:30 smithi155 ceph-mon[48344]: pgmap v708: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 703 op/s 2024-03-18T20:58:30.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:30 smithi046 ceph-mon[43495]: pgmap v708: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 703 op/s 2024-03-18T20:58:30.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:30 smithi073 ceph-mon[47384]: pgmap v708: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 703 op/s 2024-03-18T20:58:31.464 INFO:tasks.workunit.client.0.smithi046.stdout: 10 363879 32.10 MB/sec execute 427 sec latency 5503.407 ms 2024-03-18T20:58:32.465 INFO:tasks.workunit.client.0.smithi046.stdout: 10 364486 32.10 MB/sec execute 428 sec latency 6503.443 ms 2024-03-18T20:58:32.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:32 smithi155 ceph-mon[48344]: pgmap v709: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.5 MiB/s wr, 626 op/s 2024-03-18T20:58:32.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:32 smithi046 ceph-mon[43495]: pgmap v709: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.5 MiB/s wr, 626 op/s 2024-03-18T20:58:32.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:32 smithi073 ceph-mon[47384]: pgmap v709: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.5 MiB/s wr, 626 op/s 2024-03-18T20:58:33.465 INFO:tasks.workunit.client.0.smithi046.stdout: 10 365026 32.11 MB/sec execute 429 sec latency 7254.908 ms 2024-03-18T20:58:34.465 INFO:tasks.workunit.client.0.smithi046.stdout: 10 365496 32.09 MB/sec execute 430 sec latency 75.989 ms 2024-03-18T20:58:34.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:34 smithi155 ceph-mon[48344]: pgmap v710: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.6 MiB/s wr, 584 op/s 2024-03-18T20:58:34.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:34 smithi046 ceph-mon[43495]: pgmap v710: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.6 MiB/s wr, 584 op/s 2024-03-18T20:58:34.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:34 smithi073 ceph-mon[47384]: pgmap v710: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.6 MiB/s wr, 584 op/s 2024-03-18T20:58:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.114+0000 7f5b265d3700 1 -- 172.21.15.46:0/3719224876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b20100cf0 msgr2=0x7f5b2010ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.114+0000 7f5b265d3700 1 --2- 172.21.15.46:0/3719224876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b20100cf0 0x7f5b2010ca00 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b0a0 tx=0x7f5b1800b3b0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.114+0000 7f5b265d3700 1 -- 172.21.15.46:0/3719224876 shutdown_connections 2024-03-18T20:58:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.114+0000 7f5b265d3700 1 --2- 172.21.15.46:0/3719224876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b20100cf0 0x7f5b2010ca00 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.114+0000 7f5b265d3700 1 --2- 172.21.15.46:0/3719224876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b20104b90 0x7f5b20104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.114+0000 7f5b265d3700 1 --2- 172.21.15.46:0/3719224876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b20100330 0x7f5b201007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.114+0000 7f5b265d3700 1 -- 172.21.15.46:0/3719224876 >> 172.21.15.46:0/3719224876 conn(0x7f5b200761b0 msgr2=0x7f5b200fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.114+0000 7f5b265d3700 1 -- 172.21.15.46:0/3719224876 shutdown_connections 2024-03-18T20:58:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.114+0000 7f5b265d3700 1 -- 172.21.15.46:0/3719224876 wait complete. 2024-03-18T20:58:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.115+0000 7f5b265d3700 1 Processor -- start 2024-03-18T20:58:35.119 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.115+0000 7f5b265d3700 1 -- start start 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b265d3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b20100330 0x7f5b201972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b265d3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b20100cf0 0x7f5b20197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b265d3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b20104b90 0x7f5b2019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b265d3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019c210 con 0x7f5b20100cf0 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b265d3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019c380 con 0x7f5b20104b90 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b265d3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019c540 con 0x7f5b20100330 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b1ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b20100330 0x7f5b201972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b1f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b20100cf0 0x7f5b20197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b1ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b20100330 0x7f5b201972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37232/0 (socket says 172.21.15.46:37232) 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b1ffff700 1 -- 172.21.15.46:0/1028529862 learned_addr learned my addr 172.21.15.46:0/1028529862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b24b70700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b20104b90 0x7f5b2019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b1ffff700 1 -- 172.21.15.46:0/1028529862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b20104b90 msgr2=0x7f5b2019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b1ffff700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b20104b90 0x7f5b2019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b1ffff700 1 -- 172.21.15.46:0/1028529862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b20100cf0 msgr2=0x7f5b20197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b1ffff700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b20100cf0 0x7f5b20197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b1ffff700 1 -- 172.21.15.46:0/1028529862 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b10009520 con 0x7f5b20100330 2024-03-18T20:58:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.116+0000 7f5b24b70700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b20104b90 0x7f5b2019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:58:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.117+0000 7f5b1ffff700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b20100330 0x7f5b201972f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000eb30 tx=0x7f5b1000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.117+0000 7f5b1f7fe700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b20100cf0 0x7f5b20197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:58:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.117+0000 7f5b1d7fa700 1 -- 172.21.15.46:0/1028529862 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10009880 con 0x7f5b20100330 2024-03-18T20:58:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.117+0000 7f5b1d7fa700 1 -- 172.21.15.46:0/1028529862 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b100044e0 con 0x7f5b20100330 2024-03-18T20:58:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.117+0000 7f5b1d7fa700 1 -- 172.21.15.46:0/1028529862 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10003d50 con 0x7f5b20100330 2024-03-18T20:58:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.118+0000 7f5b265d3700 1 -- 172.21.15.46:0/1028529862 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b18009d00 con 0x7f5b20100330 2024-03-18T20:58:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.118+0000 7f5b265d3700 1 -- 172.21.15.46:0/1028529862 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b201a3e80 con 0x7f5b20100330 2024-03-18T20:58:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.121+0000 7f5b1d7fa700 1 -- 172.21.15.46:0/1028529862 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5b10003eb0 con 0x7f5b20100330 2024-03-18T20:58:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.121+0000 7f5b1d7fa700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5b080a1390 0x7f5b080a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.122+0000 7f5b1d7fa700 1 -- 172.21.15.46:0/1028529862 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5b10014070 con 0x7f5b20100330 2024-03-18T20:58:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.123+0000 7f5b32768b80 1 Processor -- start 2024-03-18T20:58:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.125+0000 7f5b1f7fe700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5b080a1390 0x7f5b080a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.125+0000 7f5b32768b80 1 -- start start 2024-03-18T20:58:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.125+0000 7f5b32768b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9570 0x55b23f6e9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.125+0000 7f5b32768b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6f4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.125+0000 7f5b32768b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f6f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.125+0000 7f5b32768b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23f643930 con 0x55b23f6e9e90 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.125+0000 7f5b32768b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23f6412f0 con 0x55b23f6f4db0 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.125+0000 7f5b32768b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23f63cbe0 con 0x55b23f6e9570 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b24b70700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f6f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b24b70700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f6f71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33188/0 (socket says 172.21.15.46:33188) 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b24b70700 1 -- 172.21.15.46:0/2320044998 learned_addr learned my addr 172.21.15.46:0/2320044998 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b1ffff700 1 --2- 172.21.15.46:0/2320044998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9570 0x55b23f6e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b24b70700 1 -- 172.21.15.46:0/2320044998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9570 msgr2=0x55b23f6e9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b24b70700 1 --2- 172.21.15.46:0/2320044998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9570 0x55b23f6e9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b24b70700 1 -- 172.21.15.46:0/2320044998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 msgr2=0x55b23f6f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b24b70700 1 --2- 172.21.15.46:0/2320044998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b24b70700 1 -- 172.21.15.46:0/2320044998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b23f4be350 con 0x55b23f6f4db0 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b1ffff700 1 --2- 172.21.15.46:0/2320044998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9570 0x55b23f6e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:58:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b24b70700 1 --2- 172.21.15.46:0/2320044998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f6f71a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5b180092e0 tx=0x7f5b18009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8004befb6f541d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b1f7fe700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5b080a1390 0x7f5b080a3850 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5b20198840 tx=0x7f5b1400e4b0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.126+0000 7f5b067fc700 1 -- 172.21.15.46:0/2320044998 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800e040 con 0x55b23f6f4db0 2024-03-18T20:58:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.127+0000 7f5b067fc700 1 -- 172.21.15.46:0/2320044998 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b18009b80 con 0x55b23f6f4db0 2024-03-18T20:58:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.127+0000 7f5b067fc700 1 -- 172.21.15.46:0/2320044998 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b180052a0 con 0x55b23f6f4db0 2024-03-18T20:58:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.128+0000 7f5b32768b80 1 -- 172.21.15.46:0/2320044998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 msgr2=0x55b23f6f71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.128+0000 7f5b32768b80 1 --2- 172.21.15.46:0/2320044998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f6f71a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5b180092e0 tx=0x7f5b18009310 comp rx=0 tx=0).stop 2024-03-18T20:58:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.128+0000 7f5b32768b80 1 -- 172.21.15.46:0/2320044998 shutdown_connections 2024-03-18T20:58:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.128+0000 7f5b32768b80 1 --2- 172.21.15.46:0/2320044998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f6f71a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.128+0000 7f5b32768b80 1 --2- 172.21.15.46:0/2320044998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9570 0x55b23f6e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.128+0000 7f5b32768b80 1 --2- 172.21.15.46:0/2320044998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.128+0000 7f5b32768b80 1 -- 172.21.15.46:0/2320044998 >> 172.21.15.46:0/2320044998 conn(0x55b23f6601c0 msgr2=0x55b23f6e5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.128+0000 7f5b32768b80 1 -- 172.21.15.46:0/2320044998 shutdown_connections 2024-03-18T20:58:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.128+0000 7f5b32768b80 1 -- 172.21.15.46:0/2320044998 wait complete. 2024-03-18T20:58:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.129+0000 7f5b32768b80 1 -- start start 2024-03-18T20:58:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.129+0000 7f5b32768b80 1 Processor -- start 2024-03-18T20:58:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.129+0000 7f5b32768b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9570 0x55b23f6f4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.129+0000 7f5b32768b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9e90 0x55b23f6f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.129+0000 7f5b32768b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f6f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.129+0000 7f5b32768b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23f5d8d20 con 0x55b23f6e9570 2024-03-18T20:58:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.129+0000 7f5b32768b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23f5d89c0 con 0x55b23f6f4db0 2024-03-18T20:58:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.129+0000 7f5b32768b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23f4c0640 con 0x55b23f6e9e90 2024-03-18T20:58:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.129+0000 7f5b32768b80 10 client.0 _ll_register_callbacks cb 0x55b23f444340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:58:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.130+0000 7f5b1f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9e90 0x55b23f6f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.130+0000 7f5b1f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9e90 0x55b23f6f1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:37236/0 (socket says 172.21.15.46:37236) 2024-03-18T20:58:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.130+0000 7f5b1f7fe700 1 -- 172.21.15.46:0/971630277 learned_addr learned my addr 172.21.15.46:0/971630277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.130+0000 7f5b24b70700 1 --2- 172.21.15.46:0/971630277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f6f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.130+0000 7f5b24b70700 1 -- 172.21.15.46:0/971630277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9e90 msgr2=0x55b23f6f1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.130+0000 7f5b24b70700 1 --2- 172.21.15.46:0/971630277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9e90 0x55b23f6f1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.130+0000 7f5b24b70700 1 -- 172.21.15.46:0/971630277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9570 msgr2=0x55b23f6f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.130+0000 7f5b24b70700 1 --2- 172.21.15.46:0/971630277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9570 0x55b23f6f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.130+0000 7f5b24b70700 1 -- 172.21.15.46:0/971630277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b201a3e80 con 0x55b23f6f4db0 2024-03-18T20:58:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.131+0000 7f5b24b70700 1 --2- 172.21.15.46:0/971630277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f6f1fe0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b3f0 tx=0x7f5b18007a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8774769040584c1c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.131+0000 7f5aff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.131+0000 7f5aff7fe700 1 -- 172.21.15.46:0/971630277 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800e070 con 0x55b23f6f4db0 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.131+0000 7f5aff7fe700 1 -- 172.21.15.46:0/971630277 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5b180050b0 con 0x55b23f6f4db0 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.131+0000 7f5b32768b80 1 -- 172.21.15.46:0/971630277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b23f6356a0 con 0x55b23f6f4db0 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.131+0000 7f5aff7fe700 1 -- 172.21.15.46:0/971630277 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b18005270 con 0x55b23f6f4db0 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.131+0000 7f5b32768b80 10 client.30320 fetch_fsmap learned FSMap version 94 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.131+0000 7f5b32768b80 1 -- 172.21.15.46:0/971630277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=4+}) v3 -- 0x7f5b200ff220 con 0x55b23f6f4db0 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.132+0000 7f5aff7fe700 1 -- 172.21.15.46:0/971630277 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b18036080 con 0x55b23f6f4db0 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.132+0000 7f5aff7fe700 1 -- 172.21.15.46:0/971630277 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 94) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5b1800b0d0 con 0x55b23f6f4db0 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.132+0000 7f5b32768b80 10 client.30320 fetch_fsmap finished waiting for FSMap version 94 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.132+0000 7f5b32768b80 10 client.30320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.132+0000 7f5b32768b80 20 client.30320 populate_metadata read hostname 'smithi046' 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.132+0000 7f5b32768b80 1 --2- 172.21.15.46:0/971630277 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b23f7156d0 0x55b23f717af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.132+0000 7f5b32768b80 4 client.30320 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:58:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.132+0000 7f5b32768b80 1 -- 172.21.15.46:0/971630277 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b23f48bc90 con 0x55b23f7156d0 2024-03-18T20:58:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.132+0000 7f5b1ffff700 1 --2- 172.21.15.46:0/971630277 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b23f7156d0 0x55b23f717af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.132+0000 7f5aff7fe700 10 client.30320 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.133+0000 7f5b1ffff700 1 --2- 172.21.15.46:0/971630277 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b23f7156d0 0x55b23f717af0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a0bed0a01ddf65 server_cookie=926833a5a48f000f in_seq=0 out_seq=0 2024-03-18T20:58:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.134+0000 7f5aff7fe700 1 -- 172.21.15.46:0/971630277 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b23f48bc90 con 0x55b23f7156d0 2024-03-18T20:58:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.134+0000 7f5aff7fe700 10 client.30320 handle_command_reply: tid=0 2024-03-18T20:58:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5b32768b80 1 client.30320 shutdown 2024-03-18T20:58:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5b32768b80 1 -- 172.21.15.46:0/971630277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 msgr2=0x55b23f6f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5b32768b80 1 --2- 172.21.15.46:0/971630277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f6f1fe0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b3f0 tx=0x7f5b18007a90 comp rx=0 tx=0).stop 2024-03-18T20:58:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5b32768b80 1 -- 172.21.15.46:0/971630277 shutdown_connections 2024-03-18T20:58:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5b32768b80 1 --2- 172.21.15.46:0/971630277 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b23f7156d0 0x55b23f717af0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5b32768b80 1 --2- 172.21.15.46:0/971630277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9570 0x55b23f6f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5b32768b80 1 --2- 172.21.15.46:0/971630277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f6f1fe0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5b32768b80 1 --2- 172.21.15.46:0/971630277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9e90 0x55b23f6f1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5b32768b80 1 -- 172.21.15.46:0/971630277 >> 172.21.15.46:0/971630277 conn(0x55b23f6601c0 msgr2=0x55b23f6e5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5aff7fe700 0 client.30320 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5b32768b80 1 -- 172.21.15.46:0/971630277 shutdown_connections 2024-03-18T20:58:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5b32768b80 1 -- 172.21.15.46:0/971630277 wait complete. 2024-03-18T20:58:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.135+0000 7f5b32768b80 20 client.30320 trim_cache size 0 max 16384 2024-03-18T20:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.160+0000 7f5b32768b80 1 Processor -- start 2024-03-18T20:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.160+0000 7f5b32768b80 1 -- start start 2024-03-18T20:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.160+0000 7f5b32768b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9570 0x55b23f6e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.160+0000 7f5b32768b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.160+0000 7f5b32768b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f70ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.160+0000 7f5b32768b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23f6e2200 con 0x55b23f6e9e90 2024-03-18T20:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.160+0000 7f5b32768b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23f6e67d0 con 0x55b23f6f4db0 2024-03-18T20:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.160+0000 7f5b32768b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23f6dfc60 con 0x55b23f6e9570 2024-03-18T20:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.161+0000 7f5b1f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.164 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.161+0000 7f5b1f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51544/0 (socket says 172.21.15.46:51544) 2024-03-18T20:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.161+0000 7f5b24b70700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f70ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.161+0000 7f5b1f7fe700 1 -- 172.21.15.46:0/854218037 learned_addr learned my addr 172.21.15.46:0/854218037 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.161+0000 7f5b1f7fe700 1 -- 172.21.15.46:0/854218037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9570 msgr2=0x55b23f6e5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.161+0000 7f5b1f7fe700 1 --2- 172.21.15.46:0/854218037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9570 0x55b23f6e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.161+0000 7f5b1f7fe700 1 -- 172.21.15.46:0/854218037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 msgr2=0x55b23f70ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.161+0000 7f5b1f7fe700 1 --2- 172.21.15.46:0/854218037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f70ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.161+0000 7f5b1f7fe700 1 -- 172.21.15.46:0/854218037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b23f46ab00 con 0x55b23f6e9e90 2024-03-18T20:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.161+0000 7f5b24b70700 1 --2- 172.21.15.46:0/854218037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f70ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:58:35.165 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.162+0000 7f5b1f7fe700 1 --2- 172.21.15.46:0/854218037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6f1f70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400edf0 tx=0x7f5b14013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd18f1c06cee324d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.162+0000 7f5b06ffd700 1 -- 172.21.15.46:0/854218037 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14015be0 con 0x55b23f6e9e90 2024-03-18T20:58:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.162+0000 7f5b06ffd700 1 -- 172.21.15.46:0/854218037 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b14015d40 con 0x55b23f6e9e90 2024-03-18T20:58:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.162+0000 7f5b06ffd700 1 -- 172.21.15.46:0/854218037 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b140167b0 con 0x55b23f6e9e90 2024-03-18T20:58:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.163+0000 7f5b32768b80 1 -- 172.21.15.46:0/854218037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 msgr2=0x55b23f6f1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.163+0000 7f5b32768b80 1 --2- 172.21.15.46:0/854218037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6f1f70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400edf0 tx=0x7f5b14013000 comp rx=0 tx=0).stop 2024-03-18T20:58:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.163+0000 7f5b32768b80 1 -- 172.21.15.46:0/854218037 shutdown_connections 2024-03-18T20:58:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.163+0000 7f5b32768b80 1 --2- 172.21.15.46:0/854218037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6f4db0 0x55b23f70ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.163+0000 7f5b32768b80 1 --2- 172.21.15.46:0/854218037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6e9570 0x55b23f6e5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.163+0000 7f5b32768b80 1 --2- 172.21.15.46:0/854218037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6f1f70 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.163+0000 7f5b32768b80 1 -- 172.21.15.46:0/854218037 >> 172.21.15.46:0/854218037 conn(0x55b23f6601c0 msgr2=0x55b23f6ecd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.163+0000 7f5b32768b80 1 -- 172.21.15.46:0/854218037 shutdown_connections 2024-03-18T20:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.164+0000 7f5b32768b80 1 -- 172.21.15.46:0/854218037 wait complete. 2024-03-18T20:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.165+0000 7f5b32768b80 1 -- start start 2024-03-18T20:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.165+0000 7f5b32768b80 1 Processor -- start 2024-03-18T20:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.165+0000 7f5b32768b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6e9570 0x55b23f6eed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.165+0000 7f5b32768b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.165+0000 7f5b32768b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6f4db0 0x55b23f6edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.165+0000 7f5b32768b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23f6ee1d0 con 0x55b23f6e9e90 2024-03-18T20:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.165+0000 7f5b32768b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23f6ee340 con 0x55b23f6e9570 2024-03-18T20:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.165+0000 7f5b32768b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23f6ee500 con 0x55b23f6f4db0 2024-03-18T20:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.165+0000 7f5b32768b80 10 client.0 _ll_register_callbacks cb 0x7f5b1000fe70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.165+0000 7f5b1ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6e9570 0x55b23f6eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.166+0000 7f5b1ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6e9570 0x55b23f6eed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33206/0 (socket says 172.21.15.46:33206) 2024-03-18T20:58:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.166+0000 7f5b1ffff700 1 -- 172.21.15.46:0/3304036486 learned_addr learned my addr 172.21.15.46:0/3304036486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.166+0000 7f5b1f7fe700 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.165+0000 7f5b24b70700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6f4db0 0x55b23f6edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.166+0000 7f5b24b70700 1 -- 172.21.15.46:0/3304036486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6e9570 msgr2=0x55b23f6eed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.166+0000 7f5b24b70700 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6e9570 0x55b23f6eed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.166+0000 7f5b24b70700 1 -- 172.21.15.46:0/3304036486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 msgr2=0x55b23f6ef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.166+0000 7f5b24b70700 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6ef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.166+0000 7f5b24b70700 1 -- 172.21.15.46:0/3304036486 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b200ff220 con 0x55b23f6f4db0 2024-03-18T20:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.166+0000 7f5b1f7fe700 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.166+0000 7f5b24b70700 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6f4db0 0x55b23f6edc90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009380 tx=0x7f5b18032660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ad7a74b5d103b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.166+0000 7f5aff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.167+0000 7f5aff7fe700 1 -- 172.21.15.46:0/3304036486 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800e030 con 0x55b23f6f4db0 2024-03-18T20:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.167+0000 7f5aff7fe700 1 -- 172.21.15.46:0/3304036486 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5b18032dd0 con 0x55b23f6f4db0 2024-03-18T20:58:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.167+0000 7f5b32768b80 1 -- 172.21.15.46:0/3304036486 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b23f6dd6c0 con 0x55b23f6f4db0 2024-03-18T20:58:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.168+0000 7f5aff7fe700 1 -- 172.21.15.46:0/3304036486 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b180076b0 con 0x55b23f6f4db0 2024-03-18T20:58:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.169+0000 7f5b32768b80 10 client.30439 fetch_fsmap learned FSMap version 94 2024-03-18T20:58:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.169+0000 7f5b32768b80 1 -- 172.21.15.46:0/3304036486 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=4+}) v3 -- 0x7f5b1400eaa0 con 0x55b23f6f4db0 2024-03-18T20:58:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.169+0000 7f5aff7fe700 1 -- 172.21.15.46:0/3304036486 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b23f6dd6c0 con 0x55b23f6f4db0 2024-03-18T20:58:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.169+0000 7f5aff7fe700 1 -- 172.21.15.46:0/3304036486 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 94) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5b1803c070 con 0x55b23f6f4db0 2024-03-18T20:58:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.169+0000 7f5b32768b80 10 client.30439 fetch_fsmap finished waiting for FSMap version 94 2024-03-18T20:58:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.169+0000 7f5b32768b80 10 client.30439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:58:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.169+0000 7f5b32768b80 20 client.30439 populate_metadata read hostname 'smithi046' 2024-03-18T20:58:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.169+0000 7f5b32768b80 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b23f769f40 0x55b23f76c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.169+0000 7f5b32768b80 4 client.30439 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:58:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.169+0000 7f5b32768b80 1 -- 172.21.15.46:0/3304036486 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b23f4c0c80 con 0x55b23f769f40 2024-03-18T20:58:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.173+0000 7f5b1ffff700 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b23f769f40 0x55b23f76c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.174+0000 7f5aff7fe700 10 client.30439 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.174+0000 7f5b1ffff700 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b23f769f40 0x55b23f76c330 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ce031b6f244f71 server_cookie=197893ae106d1613 in_seq=0 out_seq=0 2024-03-18T20:58:35.465 INFO:tasks.workunit.client.0.smithi046.stdout: 10 365873 32.05 MB/sec execute 431 sec latency 237.624 ms 2024-03-18T20:58:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.477+0000 7f5aff7fe700 1 -- 172.21.15.46:0/3304036486 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b23f4c0c80 con 0x55b23f769f40 2024-03-18T20:58:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.477+0000 7f5aff7fe700 10 client.30439 handle_command_reply: tid=0 2024-03-18T20:58:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.477+0000 7f5b32768b80 1 client.30439 shutdown 2024-03-18T20:58:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.477+0000 7f5b32768b80 1 -- 172.21.15.46:0/3304036486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6f4db0 msgr2=0x55b23f6edc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.477+0000 7f5b32768b80 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6f4db0 0x55b23f6edc90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009380 tx=0x7f5b18032660 comp rx=0 tx=0).stop 2024-03-18T20:58:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.478+0000 7f5b32768b80 1 -- 172.21.15.46:0/3304036486 shutdown_connections 2024-03-18T20:58:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.478+0000 7f5b32768b80 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b23f769f40 0x55b23f76c330 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.478+0000 7f5b32768b80 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b23f6f4db0 0x55b23f6edc90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.478+0000 7f5b32768b80 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b23f6e9570 0x55b23f6eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.478+0000 7f5b32768b80 1 --2- 172.21.15.46:0/3304036486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b23f6e9e90 0x55b23f6ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.478+0000 7f5b32768b80 1 -- 172.21.15.46:0/3304036486 >> 172.21.15.46:0/3304036486 conn(0x55b23f6601c0 msgr2=0x55b23f6f0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:35.481 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.478+0000 7f5aff7fe700 0 client.30439 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.482+0000 7f5b32768b80 1 -- 172.21.15.46:0/3304036486 shutdown_connections 2024-03-18T20:58:35.484 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.482+0000 7f5b32768b80 1 -- 172.21.15.46:0/3304036486 wait complete. 2024-03-18T20:58:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:58:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:58:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:58:35.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:58:35.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.484+0000 7f5b32768b80 20 client.30439 trim_cache size 0 max 16384 2024-03-18T20:58:35.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.487+0000 7f5b265d3700 1 -- 172.21.15.46:0/1028529862 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5b080a1390 msgr2=0x7f5b080a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:35.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.487+0000 7f5b265d3700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5b080a1390 0x7f5b080a3850 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5b20198840 tx=0x7f5b1400e4b0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.487+0000 7f5b265d3700 1 -- 172.21.15.46:0/1028529862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b20100330 msgr2=0x7f5b201972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:35.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.487+0000 7f5b265d3700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b20100330 0x7f5b201972f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000eb30 tx=0x7f5b1000ee40 comp rx=0 tx=0).stop 2024-03-18T20:58:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.491+0000 7f5b265d3700 1 -- 172.21.15.46:0/1028529862 shutdown_connections 2024-03-18T20:58:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.491+0000 7f5b265d3700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5b080a1390 0x7f5b080a3850 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.491+0000 7f5b265d3700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b20104b90 0x7f5b2019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.491+0000 7f5b265d3700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b20100330 0x7f5b201972f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.491+0000 7f5b265d3700 1 --2- 172.21.15.46:0/1028529862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b20100cf0 0x7f5b20197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.491+0000 7f5b265d3700 1 -- 172.21.15.46:0/1028529862 >> 172.21.15.46:0/1028529862 conn(0x7f5b200761b0 msgr2=0x7f5b20102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.496+0000 7f5b265d3700 1 -- 172.21.15.46:0/1028529862 shutdown_connections 2024-03-18T20:58:35.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:35.496+0000 7f5b265d3700 1 -- 172.21.15.46:0/1028529862 wait complete. 2024-03-18T20:58:36.465 INFO:tasks.workunit.client.0.smithi046.stdout: 10 366544 32.04 MB/sec execute 432 sec latency 118.710 ms 2024-03-18T20:58:36.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:36 smithi073 ceph-mon[47384]: pgmap v711: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.2 MiB/s wr, 468 op/s 2024-03-18T20:58:36.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:36 smithi155 ceph-mon[48344]: pgmap v711: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.2 MiB/s wr, 468 op/s 2024-03-18T20:58:36.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:36 smithi046 ceph-mon[43495]: pgmap v711: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.2 MiB/s wr, 468 op/s 2024-03-18T20:58:37.465 INFO:tasks.workunit.client.0.smithi046.stdout: 10 367238 32.08 MB/sec execute 433 sec latency 110.430 ms 2024-03-18T20:58:38.465 INFO:tasks.workunit.client.0.smithi046.stdout: 10 367691 32.06 MB/sec execute 434 sec latency 138.334 ms 2024-03-18T20:58:38.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:38 smithi155 ceph-mon[48344]: pgmap v712: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.0 MiB/s wr, 396 op/s 2024-03-18T20:58:38.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:38 smithi046 ceph-mon[43495]: pgmap v712: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.0 MiB/s wr, 396 op/s 2024-03-18T20:58:38.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:38 smithi073 ceph-mon[47384]: pgmap v712: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.0 MiB/s wr, 396 op/s 2024-03-18T20:58:39.466 INFO:tasks.workunit.client.0.smithi046.stdout: 10 368130 32.03 MB/sec execute 435 sec latency 245.681 ms 2024-03-18T20:58:40.466 INFO:tasks.workunit.client.0.smithi046.stdout: 10 368745 32.04 MB/sec execute 436 sec latency 70.305 ms 2024-03-18T20:58:40.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:40 smithi155 ceph-mon[48344]: pgmap v713: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.1 MiB/s wr, 577 op/s 2024-03-18T20:58:40.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:40 smithi046 ceph-mon[43495]: pgmap v713: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.1 MiB/s wr, 577 op/s 2024-03-18T20:58:40.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:40 smithi073 ceph-mon[47384]: pgmap v713: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.1 MiB/s wr, 577 op/s 2024-03-18T20:58:41.466 INFO:tasks.workunit.client.0.smithi046.stdout: 10 369229 32.03 MB/sec execute 437 sec latency 104.960 ms 2024-03-18T20:58:42.466 INFO:tasks.workunit.client.0.smithi046.stdout: 10 369835 32.02 MB/sec execute 438 sec latency 131.731 ms 2024-03-18T20:58:42.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:42 smithi155 ceph-mon[48344]: pgmap v714: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.3 MiB/s wr, 470 op/s 2024-03-18T20:58:42.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:42 smithi046 ceph-mon[43495]: pgmap v714: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.3 MiB/s wr, 470 op/s 2024-03-18T20:58:42.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:42 smithi073 ceph-mon[47384]: pgmap v714: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 7.3 MiB/s wr, 470 op/s 2024-03-18T20:58:43.466 INFO:tasks.workunit.client.0.smithi046.stdout: 10 370373 32.02 MB/sec execute 439 sec latency 96.650 ms 2024-03-18T20:58:43.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:43 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:58:43.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:43 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:58:43.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:43 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:58:44.466 INFO:tasks.workunit.client.0.smithi046.stdout: 10 370810 32.00 MB/sec execute 440 sec latency 166.438 ms 2024-03-18T20:58:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:44 smithi155 ceph-mon[48344]: pgmap v715: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.8 MiB/s wr, 491 op/s 2024-03-18T20:58:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:44 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:58:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:44 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:58:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:44 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:58:44.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:44 smithi046 ceph-mon[43495]: pgmap v715: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.8 MiB/s wr, 491 op/s 2024-03-18T20:58:44.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:44 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:58:44.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:44 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:58:44.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:44 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:58:44.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:44 smithi073 ceph-mon[47384]: pgmap v715: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.8 MiB/s wr, 491 op/s 2024-03-18T20:58:44.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:44 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:58:44.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:44 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:58:44.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:44 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:58:45.466 INFO:tasks.workunit.client.0.smithi046.stdout: 10 371304 31.98 MB/sec execute 441 sec latency 97.839 ms 2024-03-18T20:58:46.466 INFO:tasks.workunit.client.0.smithi046.stdout: 10 371925 31.99 MB/sec execute 442 sec latency 111.393 ms 2024-03-18T20:58:46.543 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:46 smithi155 ceph-mon[48344]: pgmap v716: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 623 op/s 2024-03-18T20:58:46.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:46 smithi046 ceph-mon[43495]: pgmap v716: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 623 op/s 2024-03-18T20:58:46.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:46 smithi073 ceph-mon[47384]: pgmap v716: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 623 op/s 2024-03-18T20:58:47.466 INFO:tasks.workunit.client.0.smithi046.stdout: 10 372458 31.98 MB/sec execute 443 sec latency 75.706 ms 2024-03-18T20:58:48.466 INFO:tasks.workunit.client.0.smithi046.stdout: 10 373130 31.98 MB/sec execute 444 sec latency 110.496 ms 2024-03-18T20:58:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:48 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:58:48.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:48 smithi046 ceph-mon[43495]: pgmap v717: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 442 op/s 2024-03-18T20:58:48.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:48 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:58:48.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:48 smithi073 ceph-mon[47384]: pgmap v717: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 442 op/s 2024-03-18T20:58:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.779+0000 7fab9351c700 1 -- 172.21.15.46:0/1229390445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c100570 msgr2=0x7fab8c10ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.779+0000 7fab9351c700 1 --2- 172.21.15.46:0/1229390445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c100570 0x7fab8c10ca00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fab8400b0a0 tx=0x7fab8400b3b0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.780+0000 7fab9351c700 1 -- 172.21.15.46:0/1229390445 shutdown_connections 2024-03-18T20:58:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.780+0000 7fab9351c700 1 --2- 172.21.15.46:0/1229390445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c100570 0x7fab8c10ca00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.780+0000 7fab9351c700 1 --2- 172.21.15.46:0/1229390445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c105c60 0x7fab8c106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.780+0000 7fab9351c700 1 --2- 172.21.15.46:0/1229390445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c0ffbb0 0x7fab8c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.780+0000 7fab9351c700 1 -- 172.21.15.46:0/1229390445 >> 172.21.15.46:0/1229390445 conn(0x7fab8c076d50 msgr2=0x7fab8c077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.780+0000 7fab9351c700 1 -- 172.21.15.46:0/1229390445 shutdown_connections 2024-03-18T20:58:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.780+0000 7fab9351c700 1 -- 172.21.15.46:0/1229390445 wait complete. 2024-03-18T20:58:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.781+0000 7fab9351c700 1 Processor -- start 2024-03-18T20:58:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.781+0000 7fab9351c700 1 -- start start 2024-03-18T20:58:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab9351c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c0ffbb0 0x7fab8c072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab9351c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c100570 0x7fab8c06b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab9351c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c105c60 0x7fab8c06bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab9351c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c06c0c0 con 0x7fab8c100570 2024-03-18T20:58:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab9351c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c06c200 con 0x7fab8c105c60 2024-03-18T20:58:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab9351c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c06c3c0 con 0x7fab8c0ffbb0 2024-03-18T20:58:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab912b8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c0ffbb0 0x7fab8c072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab912b8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c0ffbb0 0x7fab8c072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:54094/0 (socket says 172.21.15.46:54094) 2024-03-18T20:58:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab912b8700 1 -- 172.21.15.46:0/3770755792 learned_addr learned my addr 172.21.15.46:0/3770755792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab91ab9700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c105c60 0x7fab8c06bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab912b8700 1 -- 172.21.15.46:0/3770755792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c105c60 msgr2=0x7fab8c06bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab912b8700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c105c60 0x7fab8c06bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab912b8700 1 -- 172.21.15.46:0/3770755792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c100570 msgr2=0x7fab8c06b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab912b8700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c100570 0x7fab8c06b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab912b8700 1 -- 172.21.15.46:0/3770755792 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab84009d00 con 0x7fab8c0ffbb0 2024-03-18T20:58:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.782+0000 7fab91ab9700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c105c60 0x7fab8c06bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:58:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.783+0000 7fab912b8700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c0ffbb0 0x7fab8c072570 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fab8800b530 tx=0x7fab8800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.783+0000 7fab827fc700 1 -- 172.21.15.46:0/3770755792 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88011830 con 0x7fab8c0ffbb0 2024-03-18T20:58:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.783+0000 7fab827fc700 1 -- 172.21.15.46:0/3770755792 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab88011e50 con 0x7fab8c0ffbb0 2024-03-18T20:58:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.783+0000 7fab9351c700 1 -- 172.21.15.46:0/3770755792 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8c070160 con 0x7fab8c0ffbb0 2024-03-18T20:58:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.783+0000 7fab827fc700 1 -- 172.21.15.46:0/3770755792 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8800f2e0 con 0x7fab8c0ffbb0 2024-03-18T20:58:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.783+0000 7fab9351c700 1 -- 172.21.15.46:0/3770755792 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8c0706c0 con 0x7fab8c0ffbb0 2024-03-18T20:58:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.786+0000 7fab827fc700 1 -- 172.21.15.46:0/3770755792 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fab8800f440 con 0x7fab8c0ffbb0 2024-03-18T20:58:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.787+0000 7fab827fc700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fab780a1350 0x7fab780a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.787+0000 7fab827fc700 1 -- 172.21.15.46:0/3770755792 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fab880cc750 con 0x7fab8c0ffbb0 2024-03-18T20:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.788+0000 7fab90ab7700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fab780a1350 0x7fab780a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.788+0000 7fab9f6b1b80 1 Processor -- start 2024-03-18T20:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.788+0000 7fab9f6b1b80 1 -- start start 2024-03-18T20:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.788+0000 7fab90ab7700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fab780a1350 0x7fab780a3810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fab8c06cd70 tx=0x7fab7c006d20 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.788+0000 7fab9f6b1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b090570 0x562b5b090950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.788+0000 7fab9f6b1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090e90 0x562b5b09b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.788+0000 7fab9f6b1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b09bdb0 0x562b5b09e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.788+0000 7fab9f6b1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5afea930 con 0x562b5b090570 2024-03-18T20:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.788+0000 7fab9f6b1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5afe82f0 con 0x562b5b090e90 2024-03-18T20:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.788+0000 7fab9f6b1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5afe3be0 con 0x562b5b09bdb0 2024-03-18T20:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.789+0000 7fab91ab9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b09bdb0 0x562b5b09e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.789+0000 7fab91ab9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b09bdb0 0x562b5b09e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:54104/0 (socket says 172.21.15.46:54104) 2024-03-18T20:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.789+0000 7fab91ab9700 1 -- 172.21.15.46:0/1036581170 learned_addr learned my addr 172.21.15.46:0/1036581170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.789+0000 7fab912b8700 1 --2- 172.21.15.46:0/1036581170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b090570 0x562b5b090950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.789+0000 7fab90ab7700 1 --2- 172.21.15.46:0/1036581170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090e90 0x562b5b09b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.789+0000 7fab91ab9700 1 -- 172.21.15.46:0/1036581170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090e90 msgr2=0x562b5b09b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.789+0000 7fab91ab9700 1 --2- 172.21.15.46:0/1036581170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090e90 0x562b5b09b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.789+0000 7fab91ab9700 1 -- 172.21.15.46:0/1036581170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b090570 msgr2=0x562b5b090950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.789+0000 7fab91ab9700 1 --2- 172.21.15.46:0/1036581170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b090570 0x562b5b090950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.789+0000 7fab91ab9700 1 -- 172.21.15.46:0/1036581170 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b5ae65350 con 0x562b5b09bdb0 2024-03-18T20:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.789+0000 7fab91ab9700 1 --2- 172.21.15.46:0/1036581170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b09bdb0 0x562b5b09e1a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fab8400b830 tx=0x7fab84007c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b86f9d37edb9e490 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.790+0000 7fab777fe700 1 -- 172.21.15.46:0/1036581170 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8400e050 con 0x562b5b09bdb0 2024-03-18T20:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.790+0000 7fab777fe700 1 -- 172.21.15.46:0/1036581170 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab840093b0 con 0x562b5b09bdb0 2024-03-18T20:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.790+0000 7fab777fe700 1 -- 172.21.15.46:0/1036581170 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab840079f0 con 0x562b5b09bdb0 2024-03-18T20:58:48.795 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:48 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:58:48.795 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:48 smithi155 ceph-mon[48344]: pgmap v717: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.7 MiB/s wr, 442 op/s 2024-03-18T20:58:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.791+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/1036581170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b09bdb0 msgr2=0x562b5b09e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.791+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/1036581170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b09bdb0 0x562b5b09e1a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fab8400b830 tx=0x7fab84007c80 comp rx=0 tx=0).stop 2024-03-18T20:58:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.796+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/1036581170 shutdown_connections 2024-03-18T20:58:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.796+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/1036581170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b090570 0x562b5b090950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.796+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/1036581170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b09bdb0 0x562b5b09e1a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.796+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/1036581170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090e90 0x562b5b09b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.796+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/1036581170 >> 172.21.15.46:0/1036581170 conn(0x562b5b0071c0 msgr2=0x562b5b08ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.796+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/1036581170 shutdown_connections 2024-03-18T20:58:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.803+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/1036581170 wait complete. 2024-03-18T20:58:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.803+0000 7fab9f6b1b80 1 -- start start 2024-03-18T20:58:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.803+0000 7fab9f6b1b80 1 Processor -- start 2024-03-18T20:58:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab9f6b1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 0x562b5b09b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab9f6b1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b090e90 0x562b5b098ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab9f6b1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b09bdb0 0x562b5b099210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab9f6b1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5af7fd20 con 0x562b5b09bdb0 2024-03-18T20:58:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab9f6b1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5af7f9c0 con 0x562b5b090570 2024-03-18T20:58:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab9f6b1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5ae67640 con 0x562b5b090e90 2024-03-18T20:58:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab9f6b1b80 10 client.0 _ll_register_callbacks cb 0x562b5adeb340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:58:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab912b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 0x562b5b09b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab912b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 0x562b5b09b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:43816/0 (socket says 172.21.15.46:43816) 2024-03-18T20:58:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab912b8700 1 -- 172.21.15.46:0/774554404 learned_addr learned my addr 172.21.15.46:0/774554404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab90ab7700 1 --2- 172.21.15.46:0/774554404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b090e90 0x562b5b098ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab912b8700 1 -- 172.21.15.46:0/774554404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b090e90 msgr2=0x562b5b098ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab912b8700 1 --2- 172.21.15.46:0/774554404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b090e90 0x562b5b098ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab912b8700 1 -- 172.21.15.46:0/774554404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b09bdb0 msgr2=0x562b5b099210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab912b8700 1 --2- 172.21.15.46:0/774554404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b09bdb0 0x562b5b099210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.804+0000 7fab912b8700 1 -- 172.21.15.46:0/774554404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8c068240 con 0x562b5b090570 2024-03-18T20:58:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.805+0000 7fab912b8700 1 --2- 172.21.15.46:0/774554404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 0x562b5b09b7e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fab88095e00 tx=0x7fab8809a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10713d51b31cc093 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.805+0000 7fab74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:58:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.805+0000 7fab74ff9700 1 -- 172.21.15.46:0/774554404 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab880b1070 con 0x562b5b090570 2024-03-18T20:58:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.805+0000 7fab74ff9700 1 -- 172.21.15.46:0/774554404 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fab8809d780 con 0x562b5b090570 2024-03-18T20:58:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.805+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/774554404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b5b0846c0 con 0x562b5b090570 2024-03-18T20:58:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.805+0000 7fab74ff9700 1 -- 172.21.15.46:0/774554404 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab880a1dd0 con 0x562b5b090570 2024-03-18T20:58:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.805+0000 7fab9f6b1b80 10 client.30335 fetch_fsmap learned FSMap version 95 2024-03-18T20:58:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.805+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/774554404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) v3 -- 0x562b5ae11b00 con 0x562b5b090570 2024-03-18T20:58:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.806+0000 7fab74ff9700 1 -- 172.21.15.46:0/774554404 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562b5b0846c0 con 0x562b5b090570 2024-03-18T20:58:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.806+0000 7fab74ff9700 1 -- 172.21.15.46:0/774554404 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 95) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fab88099bb0 con 0x562b5b090570 2024-03-18T20:58:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.806+0000 7fab9f6b1b80 10 client.30335 fetch_fsmap finished waiting for FSMap version 95 2024-03-18T20:58:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.806+0000 7fab9f6b1b80 10 client.30335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:58:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.806+0000 7fab9f6b1b80 20 client.30335 populate_metadata read hostname 'smithi046' 2024-03-18T20:58:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.806+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/774554404 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562b5b0ab9b0 0x562b5b0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.806+0000 7fab9f6b1b80 4 client.30335 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:58:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.806+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/774554404 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b5afdc6a0 con 0x562b5b0ab9b0 2024-03-18T20:58:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.806+0000 7fab90ab7700 1 --2- 172.21.15.46:0/774554404 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562b5b0ab9b0 0x562b5b0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.806+0000 7fab74ff9700 10 client.30335 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.807+0000 7fab90ab7700 1 --2- 172.21.15.46:0/774554404 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562b5b0ab9b0 0x562b5b0addd0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f9df3427723bace server_cookie=ef6ae86e69db8329 in_seq=0 out_seq=0 2024-03-18T20:58:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.808+0000 7fab74ff9700 1 -- 172.21.15.46:0/774554404 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562b5afdc6a0 con 0x562b5b0ab9b0 2024-03-18T20:58:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.808+0000 7fab74ff9700 10 client.30335 handle_command_reply: tid=0 2024-03-18T20:58:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.808+0000 7fab9f6b1b80 1 client.30335 shutdown 2024-03-18T20:58:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.810+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/774554404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 msgr2=0x562b5b09b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.810+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/774554404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 0x562b5b09b7e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fab88095e00 tx=0x7fab8809a0d0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.812+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/774554404 shutdown_connections 2024-03-18T20:58:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.812+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/774554404 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562b5b0ab9b0 0x562b5b0addd0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.812+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/774554404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b09bdb0 0x562b5b099210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.812+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/774554404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 0x562b5b09b7e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.812+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/774554404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b090e90 0x562b5b098ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.812+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/774554404 >> 172.21.15.46:0/774554404 conn(0x562b5b0071c0 msgr2=0x562b5b08c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.816+0000 7fab74ff9700 0 client.30335 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.818+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/774554404 shutdown_connections 2024-03-18T20:58:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.819+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/774554404 wait complete. 2024-03-18T20:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.820+0000 7fab9f6b1b80 20 client.30335 trim_cache size 0 max 16384 2024-03-18T20:58:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.846+0000 7fab9f6b1b80 1 Processor -- start 2024-03-18T20:58:48.849 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.846+0000 7fab9f6b1b80 1 -- start start 2024-03-18T20:58:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.846+0000 7fab9f6b1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 0x562b5b09b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.846+0000 7fab9f6b1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b090e90 0x562b5b09b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.846+0000 7fab9f6b1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b09bdb0 0x562b5b098360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.846+0000 7fab9f6b1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5b092140 con 0x562b5b090e90 2024-03-18T20:58:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.846+0000 7fab9f6b1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5b0891d0 con 0x562b5b090570 2024-03-18T20:58:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.846+0000 7fab9f6b1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5b086c10 con 0x562b5b09bdb0 2024-03-18T20:58:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.846+0000 7fab912b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 0x562b5b09b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.847+0000 7fab912b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 0x562b5b09b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:43832/0 (socket says 172.21.15.46:43832) 2024-03-18T20:58:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.847+0000 7fab912b8700 1 -- 172.21.15.46:0/3179975084 learned_addr learned my addr 172.21.15.46:0/3179975084 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.847+0000 7fab91ab9700 1 --2- 172.21.15.46:0/3179975084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b09bdb0 0x562b5b098360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.847+0000 7fab912b8700 1 -- 172.21.15.46:0/3179975084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b09bdb0 msgr2=0x562b5b098360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.847+0000 7fab912b8700 1 --2- 172.21.15.46:0/3179975084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b09bdb0 0x562b5b098360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.847+0000 7fab912b8700 1 -- 172.21.15.46:0/3179975084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b090e90 msgr2=0x562b5b09b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.847+0000 7fab912b8700 1 --2- 172.21.15.46:0/3179975084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b090e90 0x562b5b09b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.847+0000 7fab912b8700 1 -- 172.21.15.46:0/3179975084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b5afa9e40 con 0x562b5b090570 2024-03-18T20:58:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.847+0000 7fab912b8700 1 --2- 172.21.15.46:0/3179975084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 0x562b5b09b310 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fab88099650 tx=0x7fab88010790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b23562ddd508f229 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.847+0000 7fab77fff700 1 -- 172.21.15.46:0/3179975084 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8800bea0 con 0x562b5b090570 2024-03-18T20:58:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.851+0000 7fab77fff700 1 -- 172.21.15.46:0/3179975084 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab8800d040 con 0x562b5b090570 2024-03-18T20:58:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.851+0000 7fab77fff700 1 -- 172.21.15.46:0/3179975084 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab880a1dd0 con 0x562b5b090570 2024-03-18T20:58:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.852+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3179975084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 msgr2=0x562b5b09b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.852+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/3179975084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 0x562b5b09b310 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fab88099650 tx=0x7fab88010790 comp rx=0 tx=0).stop 2024-03-18T20:58:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.852+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3179975084 shutdown_connections 2024-03-18T20:58:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.852+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/3179975084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b09bdb0 0x562b5b098360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.852+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/3179975084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090570 0x562b5b09b310 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.852+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/3179975084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b090e90 0x562b5b09b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.852+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3179975084 >> 172.21.15.46:0/3179975084 conn(0x562b5b0071c0 msgr2=0x562b5b093e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.852+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3179975084 shutdown_connections 2024-03-18T20:58:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.852+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3179975084 wait complete. 2024-03-18T20:58:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.853+0000 7fab9f6b1b80 1 -- start start 2024-03-18T20:58:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.853+0000 7fab9f6b1b80 1 Processor -- start 2024-03-18T20:58:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.854+0000 7fab9f6b1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b090570 0x562b5b09e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.854+0000 7fab9f6b1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090e90 0x562b5b09eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.854+0000 7fab9f6b1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b09bdb0 0x562b5b09f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.854+0000 7fab9f6b1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5b0a2dc0 con 0x562b5b09bdb0 2024-03-18T20:58:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.854+0000 7fab9f6b1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5b0a2f30 con 0x562b5b090e90 2024-03-18T20:58:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.854+0000 7fab9f6b1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5b0f7790 con 0x562b5b090570 2024-03-18T20:58:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.854+0000 7fab9f6b1b80 10 client.0 _ll_register_callbacks cb 0x7fab880992b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:58:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.854+0000 7fab912b8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b090570 0x562b5b09e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.854+0000 7fab912b8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b090570 0x562b5b09e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:54144/0 (socket says 172.21.15.46:54144) 2024-03-18T20:58:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.854+0000 7fab912b8700 1 -- 172.21.15.46:0/3893044469 learned_addr learned my addr 172.21.15.46:0/3893044469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:58:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab912b8700 1 -- 172.21.15.46:0/3893044469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090e90 msgr2=0x562b5b09eef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:58:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab912b8700 1 --2- 172.21.15.46:0/3893044469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090e90 0x562b5b09eef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab912b8700 1 -- 172.21.15.46:0/3893044469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b09bdb0 msgr2=0x562b5b09f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:58:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab912b8700 1 --2- 172.21.15.46:0/3893044469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b09bdb0 0x562b5b09f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab912b8700 1 -- 172.21.15.46:0/3893044469 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b5ae11b00 con 0x562b5b090570 2024-03-18T20:58:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab912b8700 1 --2- 172.21.15.46:0/3893044469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b090570 0x562b5b09e9b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fab880a18a0 tx=0x7fab88099a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcf442451c888e32 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:58:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:58:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab74ff9700 1 -- 172.21.15.46:0/3893044469 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab880b1070 con 0x562b5b090570 2024-03-18T20:58:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3893044469 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b5ae32c90 con 0x562b5b090570 2024-03-18T20:58:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab74ff9700 1 -- 172.21.15.46:0/3893044469 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fab880a1dd0 con 0x562b5b090570 2024-03-18T20:58:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab74ff9700 1 -- 172.21.15.46:0/3893044469 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab880a1b50 con 0x562b5b090570 2024-03-18T20:58:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab9f6b1b80 10 client.30457 fetch_fsmap learned FSMap version 95 2024-03-18T20:58:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.855+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3893044469 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) v3 -- 0x562b5ae69c70 con 0x562b5b090570 2024-03-18T20:58:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.856+0000 7fab74ff9700 1 -- 172.21.15.46:0/3893044469 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562b5ae32c90 con 0x562b5b090570 2024-03-18T20:58:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.856+0000 7fab74ff9700 1 -- 172.21.15.46:0/3893044469 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 95) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fab880b9080 con 0x562b5b090570 2024-03-18T20:58:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.856+0000 7fab9f6b1b80 10 client.30457 fetch_fsmap finished waiting for FSMap version 95 2024-03-18T20:58:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.856+0000 7fab9f6b1b80 10 client.30457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:58:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.856+0000 7fab9f6b1b80 20 client.30457 populate_metadata read hostname 'smithi046' 2024-03-18T20:58:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.856+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/3893044469 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562b5b0f8b10 0x562b5b0faf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:58:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.856+0000 7fab9f6b1b80 4 client.30457 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:58:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.856+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3893044469 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562b5ae67c80 con 0x562b5b0f8b10 2024-03-18T20:58:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.856+0000 7fab90ab7700 1 --2- 172.21.15.46:0/3893044469 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562b5b0f8b10 0x562b5b0faf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:58:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.857+0000 7fab90ab7700 1 --2- 172.21.15.46:0/3893044469 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562b5b0f8b10 0x562b5b0faf30 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=891d0cbc7fec7c2d server_cookie=25a0b6d11e02754f in_seq=0 out_seq=0 2024-03-18T20:58:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:48.857+0000 7fab74ff9700 10 client.30457 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.195+0000 7fab74ff9700 1 -- 172.21.15.46:0/3893044469 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562b5ae67c80 con 0x562b5b0f8b10 2024-03-18T20:58:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.195+0000 7fab74ff9700 10 client.30457 handle_command_reply: tid=0 2024-03-18T20:58:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.195+0000 7fab9f6b1b80 1 client.30457 shutdown 2024-03-18T20:58:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.195+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3893044469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b090570 msgr2=0x562b5b09e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.195+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/3893044469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b090570 0x562b5b09e9b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fab880a18a0 tx=0x7fab88099a10 comp rx=0 tx=0).stop 2024-03-18T20:58:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.196+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3893044469 shutdown_connections 2024-03-18T20:58:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.196+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/3893044469 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x562b5b0f8b10 0x562b5b0faf30 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.196+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/3893044469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562b5b09bdb0 0x562b5b09f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.196+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/3893044469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562b5b090570 0x562b5b09e9b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.196+0000 7fab9f6b1b80 1 --2- 172.21.15.46:0/3893044469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b5b090e90 0x562b5b09eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.196+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3893044469 >> 172.21.15.46:0/3893044469 conn(0x562b5b0071c0 msgr2=0x562b5b092640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.196+0000 7fab74ff9700 0 client.30457 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:58:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.197+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3893044469 shutdown_connections 2024-03-18T20:58:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.197+0000 7fab9f6b1b80 1 -- 172.21.15.46:0/3893044469 wait complete. 2024-03-18T20:58:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.198+0000 7fab9f6b1b80 20 client.30457 trim_cache size 0 max 16384 2024-03-18T20:58:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:58:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:58:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:58:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:58:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.199+0000 7fab9351c700 1 -- 172.21.15.46:0/3770755792 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fab780a1350 msgr2=0x7fab780a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.199+0000 7fab9351c700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fab780a1350 0x7fab780a3810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fab8c06cd70 tx=0x7fab7c006d20 comp rx=0 tx=0).stop 2024-03-18T20:58:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.199+0000 7fab9351c700 1 -- 172.21.15.46:0/3770755792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c0ffbb0 msgr2=0x7fab8c072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:58:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.199+0000 7fab9351c700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c0ffbb0 0x7fab8c072570 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fab8800b530 tx=0x7fab8800b8f0 comp rx=0 tx=0).stop 2024-03-18T20:58:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.199+0000 7fab9351c700 1 -- 172.21.15.46:0/3770755792 shutdown_connections 2024-03-18T20:58:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.199+0000 7fab9351c700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fab780a1350 0x7fab780a3810 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.199+0000 7fab9351c700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c105c60 0x7fab8c06bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.199+0000 7fab9351c700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab8c0ffbb0 0x7fab8c072570 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.199+0000 7fab9351c700 1 --2- 172.21.15.46:0/3770755792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab8c100570 0x7fab8c06b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:58:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.199+0000 7fab9351c700 1 -- 172.21.15.46:0/3770755792 >> 172.21.15.46:0/3770755792 conn(0x7fab8c076d50 msgr2=0x7fab8c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:58:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.199+0000 7fab9351c700 1 -- 172.21.15.46:0/3770755792 shutdown_connections 2024-03-18T20:58:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:58:49.199+0000 7fab9351c700 1 -- 172.21.15.46:0/3770755792 wait complete. 2024-03-18T20:58:49.469 INFO:tasks.workunit.client.0.smithi046.stdout: 10 373595 31.98 MB/sec execute 445 sec latency 250.871 ms 2024-03-18T20:58:50.469 INFO:tasks.workunit.client.0.smithi046.stdout: 10 374113 31.99 MB/sec execute 446 sec latency 99.275 ms 2024-03-18T20:58:50.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:50 smithi155 ceph-mon[48344]: pgmap v718: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.3 MiB/s wr, 613 op/s 2024-03-18T20:58:50.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:50 smithi046 ceph-mon[43495]: pgmap v718: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.3 MiB/s wr, 613 op/s 2024-03-18T20:58:50.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:50 smithi073 ceph-mon[47384]: pgmap v718: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.3 MiB/s wr, 613 op/s 2024-03-18T20:58:51.469 INFO:tasks.workunit.client.0.smithi046.stdout: 10 374576 31.95 MB/sec execute 447 sec latency 84.578 ms 2024-03-18T20:58:52.469 INFO:tasks.workunit.client.0.smithi046.stdout: 10 375182 31.95 MB/sec execute 448 sec latency 131.549 ms 2024-03-18T20:58:52.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:52 smithi155 ceph-mon[48344]: pgmap v719: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 483 op/s 2024-03-18T20:58:52.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:52 smithi046 ceph-mon[43495]: pgmap v719: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 483 op/s 2024-03-18T20:58:52.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:52 smithi073 ceph-mon[47384]: pgmap v719: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 483 op/s 2024-03-18T20:58:53.469 INFO:tasks.workunit.client.0.smithi046.stdout: 10 375767 31.96 MB/sec execute 449 sec latency 100.166 ms 2024-03-18T20:58:53.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:53 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:58:53.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:53 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:58:53.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:53 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:58:54.476 INFO:tasks.workunit.client.0.smithi046.stdout: 10 375930 31.91 MB/sec execute 450 sec latency 748.513 ms 2024-03-18T20:58:54.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:54 smithi073 ceph-mon[47384]: pgmap v720: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 441 op/s 2024-03-18T20:58:55.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:54 smithi155 ceph-mon[48344]: pgmap v720: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 441 op/s 2024-03-18T20:58:55.053 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:54 smithi046 ceph-mon[43495]: pgmap v720: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 441 op/s 2024-03-18T20:58:55.476 INFO:tasks.workunit.client.0.smithi046.stdout: 10 375971 31.84 MB/sec execute 451 sec latency 1246.237 ms 2024-03-18T20:58:56.477 INFO:tasks.workunit.client.0.smithi046.stdout: 10 376180 31.79 MB/sec execute 452 sec latency 765.355 ms 2024-03-18T20:58:56.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:56 smithi155 ceph-mon[48344]: pgmap v721: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 14 MiB/s wr, 704 op/s 2024-03-18T20:58:56.551 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:56 smithi046 ceph-mon[43495]: pgmap v721: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 14 MiB/s wr, 704 op/s 2024-03-18T20:58:56.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:56 smithi073 ceph-mon[47384]: pgmap v721: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 14 MiB/s wr, 704 op/s 2024-03-18T20:58:57.477 INFO:tasks.workunit.client.0.smithi046.stdout: 10 376391 31.75 MB/sec execute 453 sec latency 205.354 ms 2024-03-18T20:58:58.477 INFO:tasks.workunit.client.0.smithi046.stdout: 10 376926 31.73 MB/sec execute 454 sec latency 117.027 ms 2024-03-18T20:58:58.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:58:58 smithi155 ceph-mon[48344]: pgmap v722: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 546 op/s 2024-03-18T20:58:58.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:58:58 smithi046 ceph-mon[43495]: pgmap v722: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 546 op/s 2024-03-18T20:58:58.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:58:58 smithi073 ceph-mon[47384]: pgmap v722: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 546 op/s 2024-03-18T20:58:59.477 INFO:tasks.workunit.client.0.smithi046.stdout: 10 377598 31.75 MB/sec execute 455 sec latency 82.143 ms 2024-03-18T20:59:00.477 INFO:tasks.workunit.client.0.smithi046.stdout: 10 378104 31.75 MB/sec execute 456 sec latency 128.652 ms 2024-03-18T20:59:00.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:00 smithi155 ceph-mon[48344]: pgmap v723: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 20 MiB/s wr, 845 op/s 2024-03-18T20:59:00.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:00 smithi046 ceph-mon[43495]: pgmap v723: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 20 MiB/s wr, 845 op/s 2024-03-18T20:59:00.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:00 smithi073 ceph-mon[47384]: pgmap v723: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 20 MiB/s wr, 845 op/s 2024-03-18T20:59:01.477 INFO:tasks.workunit.client.0.smithi046.stdout: 10 378359 31.71 MB/sec execute 457 sec latency 173.280 ms 2024-03-18T20:59:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.413+0000 7f400bf27700 1 -- 172.21.15.46:0/2723779039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4004101890 msgr2=0x7f40041058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.413+0000 7f400bf27700 1 --2- 172.21.15.46:0/2723779039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4004101890 0x7f40041058d0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009990 tx=0x7f3ff4009ca0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.414+0000 7f400bf27700 1 -- 172.21.15.46:0/2723779039 shutdown_connections 2024-03-18T20:59:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.414+0000 7f400bf27700 1 --2- 172.21.15.46:0/2723779039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4004106000 0x7f400410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.414+0000 7f400bf27700 1 --2- 172.21.15.46:0/2723779039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4004100ee0 0x7f40041012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.414+0000 7f400bf27700 1 --2- 172.21.15.46:0/2723779039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4004101890 0x7f40041058d0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.414+0000 7f400bf27700 1 -- 172.21.15.46:0/2723779039 >> 172.21.15.46:0/2723779039 conn(0x7f40040fc780 msgr2=0x7f40040feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.414+0000 7f400bf27700 1 -- 172.21.15.46:0/2723779039 shutdown_connections 2024-03-18T20:59:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.414+0000 7f400bf27700 1 -- 172.21.15.46:0/2723779039 wait complete. 2024-03-18T20:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.414+0000 7f400bf27700 1 Processor -- start 2024-03-18T20:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.415+0000 7f400bf27700 1 -- start start 2024-03-18T20:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.415+0000 7f400bf27700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4004100ee0 0x7f40040730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.415+0000 7f400bf27700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4004106000 0x7f40040735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.415+0000 7f400bf27700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4004073b30 0x7f40041a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.415+0000 7f400bf27700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40041a5a20 con 0x7f4004100ee0 2024-03-18T20:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.415+0000 7f400bf27700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40041a5b60 con 0x7f4004106000 2024-03-18T20:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.415+0000 7f400bf27700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40041a5d20 con 0x7f4004073b30 2024-03-18T20:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.415+0000 7f40094c2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4004106000 0x7f40040735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.415+0000 7f4009cc3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4004100ee0 0x7f40040730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.415+0000 7f40094c2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4004106000 0x7f40040735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40026/0 (socket says 172.21.15.46:40026) 2024-03-18T20:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.415+0000 7f40094c2700 1 -- 172.21.15.46:0/2281563342 learned_addr learned my addr 172.21.15.46:0/2281563342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.415+0000 7f400a4c4700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4004073b30 0x7f40041a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.416+0000 7f40094c2700 1 -- 172.21.15.46:0/2281563342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4004073b30 msgr2=0x7f40041a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.416+0000 7f40094c2700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4004073b30 0x7f40041a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.416+0000 7f40094c2700 1 -- 172.21.15.46:0/2281563342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4004100ee0 msgr2=0x7f40040730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.416+0000 7f40094c2700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4004100ee0 0x7f40040730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.416+0000 7f40094c2700 1 -- 172.21.15.46:0/2281563342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff40095f0 con 0x7f4004106000 2024-03-18T20:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.416+0000 7f40094c2700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4004106000 0x7f40040735f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4000c00 tx=0x7f3ff4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.416+0000 7f3ffaffd700 1 -- 172.21.15.46:0/2281563342 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff401d070 con 0x7f4004106000 2024-03-18T20:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.416+0000 7f400bf27700 1 -- 172.21.15.46:0/2281563342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40041a5fa0 con 0x7f4004106000 2024-03-18T20:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.416+0000 7f3ffaffd700 1 -- 172.21.15.46:0/2281563342 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ff4022470 con 0x7f4004106000 2024-03-18T20:59:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.416+0000 7f3ffaffd700 1 -- 172.21.15.46:0/2281563342 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4017650 con 0x7f4004106000 2024-03-18T20:59:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.416+0000 7f400bf27700 1 -- 172.21.15.46:0/2281563342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40041a6360 con 0x7f4004106000 2024-03-18T20:59:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.421+0000 7f3ffaffd700 1 -- 172.21.15.46:0/2281563342 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3ff40177b0 con 0x7f4004106000 2024-03-18T20:59:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.422+0000 7f40180bcb80 1 Processor -- start 2024-03-18T20:59:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.422+0000 7f3ffaffd700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ff00a1310 0x7f3ff00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.423+0000 7f3ffaffd700 1 -- 172.21.15.46:0/2281563342 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3ff40ce570 con 0x7f4004106000 2024-03-18T20:59:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.423+0000 7f40180bcb80 1 -- start start 2024-03-18T20:59:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.423+0000 7f400a4c4700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ff00a1310 0x7f3ff00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.423+0000 7f40180bcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c0d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.423+0000 7f40180bcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c21080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.423+0000 7f40180bcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.423+0000 7f40180bcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e021b67930 con 0x55e021c215c0 2024-03-18T20:59:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.423+0000 7f40180bcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e021b652f0 con 0x55e021c0de90 2024-03-18T20:59:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.423+0000 7f40180bcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e021b60be0 con 0x55e021c0d570 2024-03-18T20:59:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.424+0000 7f4009cc3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c0d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.424+0000 7f4009cc3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c0d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:46902/0 (socket says 172.21.15.46:46902) 2024-03-18T20:59:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.424+0000 7f4009cc3700 1 -- 172.21.15.46:0/3224109104 learned_addr learned my addr 172.21.15.46:0/3224109104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.424+0000 7f40094c2700 1 --2- 172.21.15.46:0/3224109104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c21080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.424+0000 7f4009cc3700 1 -- 172.21.15.46:0/3224109104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 msgr2=0x55e021c21080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.424+0000 7f4009cc3700 1 --2- 172.21.15.46:0/3224109104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c21080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.424+0000 7f4009cc3700 1 -- 172.21.15.46:0/3224109104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 msgr2=0x55e021c239b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.424+0000 7f400a4c4700 1 --2- 172.21.15.46:0/3224109104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.424+0000 7f4009cc3700 1 --2- 172.21.15.46:0/3224109104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c239b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.424+0000 7f4009cc3700 1 -- 172.21.15.46:0/3224109104 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0219e2350 con 0x55e021c0d570 2024-03-18T20:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.425+0000 7f400a4c4700 1 --2- 172.21.15.46:0/3224109104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c239b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.425+0000 7f4009cc3700 1 --2- 172.21.15.46:0/3224109104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c0d950 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4000009af0 tx=0x7f4000009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d93de82ff9b91d33 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.425+0000 7f400a4c4700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ff00a1310 0x7f3ff00a37d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00f540 tx=0x7f3ffc00b040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.429+0000 7f3feffff700 1 -- 172.21.15.46:0/3224109104 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40000095f0 con 0x55e021c0d570 2024-03-18T20:59:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.430+0000 7f3feffff700 1 -- 172.21.15.46:0/3224109104 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400000ee50 con 0x55e021c0d570 2024-03-18T20:59:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.430+0000 7f3feffff700 1 -- 172.21.15.46:0/3224109104 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000012630 con 0x55e021c0d570 2024-03-18T20:59:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.430+0000 7f40180bcb80 1 -- 172.21.15.46:0/3224109104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 msgr2=0x55e021c0d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.430+0000 7f40180bcb80 1 --2- 172.21.15.46:0/3224109104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c0d950 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4000009af0 tx=0x7f4000009e00 comp rx=0 tx=0).stop 2024-03-18T20:59:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.433+0000 7f40180bcb80 1 -- 172.21.15.46:0/3224109104 shutdown_connections 2024-03-18T20:59:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.433+0000 7f40180bcb80 1 --2- 172.21.15.46:0/3224109104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c239b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.433+0000 7f40180bcb80 1 --2- 172.21.15.46:0/3224109104 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c0d950 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.433+0000 7f40180bcb80 1 --2- 172.21.15.46:0/3224109104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c21080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.433+0000 7f40180bcb80 1 -- 172.21.15.46:0/3224109104 >> 172.21.15.46:0/3224109104 conn(0x55e021b841c0 msgr2=0x55e021c09a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.433+0000 7f40180bcb80 1 -- 172.21.15.46:0/3224109104 shutdown_connections 2024-03-18T20:59:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.433+0000 7f40180bcb80 1 -- 172.21.15.46:0/3224109104 wait complete. 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.433+0000 7f40180bcb80 1 -- start start 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f40180bcb80 1 Processor -- start 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f40180bcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0d570 0x55e021c20ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f40180bcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c0de90 0x55e021c1e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f40180bcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c215c0 0x55e021c1e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f40180bcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e021afcd20 con 0x55e021c0de90 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f40180bcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e021afc9c0 con 0x55e021c0d570 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f40180bcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0219e4640 con 0x55e021c215c0 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f40180bcb80 10 client.0 _ll_register_callbacks cb 0x55e021968340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f4009cc3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0d570 0x55e021c20ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f4009cc3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0d570 0x55e021c20ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40040/0 (socket says 172.21.15.46:40040) 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f4009cc3700 1 -- 172.21.15.46:0/2699173838 learned_addr learned my addr 172.21.15.46:0/2699173838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f40094c2700 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c0de90 0x55e021c1e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.434+0000 7f400a4c4700 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c215c0 0x55e021c1e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.435+0000 7f4009cc3700 1 -- 172.21.15.46:0/2699173838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c215c0 msgr2=0x55e021c1e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.435+0000 7f4009cc3700 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c215c0 0x55e021c1e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.435+0000 7f4009cc3700 1 -- 172.21.15.46:0/2699173838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c0de90 msgr2=0x55e021c1e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.435+0000 7f4009cc3700 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c0de90 0x55e021c1e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.435+0000 7f4009cc3700 1 -- 172.21.15.46:0/2699173838 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4000009750 con 0x55e021c0d570 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.435+0000 7f400a4c4700 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c215c0 0x55e021c1e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.435+0000 7f4009cc3700 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0d570 0x55e021c20ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4000004790 tx=0x7f400000ebc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ce9b86af4ee6f3d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.435+0000 7f3fed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.435+0000 7f3fed7fa700 1 -- 172.21.15.46:0/2699173838 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40000095d0 con 0x55e021c0d570 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.435+0000 7f3fed7fa700 1 -- 172.21.15.46:0/2699173838 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f400001c660 con 0x55e021c0d570 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.435+0000 7f40180bcb80 1 -- 172.21.15.46:0/2699173838 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e021c016c0 con 0x55e021c0d570 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.436+0000 7f3fed7fa700 1 -- 172.21.15.46:0/2699173838 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f400001c820 con 0x55e021c0d570 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.436+0000 7f40180bcb80 10 client.30344 fetch_fsmap learned FSMap version 96 2024-03-18T20:59:02.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.436+0000 7f40180bcb80 1 -- 172.21.15.46:0/2699173838 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x55e02198eb00 con 0x55e021c0d570 2024-03-18T20:59:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.436+0000 7f3fed7fa700 1 -- 172.21.15.46:0/2699173838 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e021c016c0 con 0x55e021c0d570 2024-03-18T20:59:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.436+0000 7f3fed7fa700 1 -- 172.21.15.46:0/2699173838 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 96) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f400002c070 con 0x55e021c0d570 2024-03-18T20:59:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.436+0000 7f40180bcb80 10 client.30344 fetch_fsmap finished waiting for FSMap version 96 2024-03-18T20:59:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.436+0000 7f40180bcb80 10 client.30344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:59:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.436+0000 7f40180bcb80 20 client.30344 populate_metadata read hostname 'smithi046' 2024-03-18T20:59:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.436+0000 7f40180bcb80 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e021c144c0 0x55e021c168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.436+0000 7f40180bcb80 4 client.30344 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:59:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.436+0000 7f40180bcb80 1 -- 172.21.15.46:0/2699173838 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e021b596a0 con 0x55e021c144c0 2024-03-18T20:59:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.436+0000 7f40094c2700 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e021c144c0 0x55e021c168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.437+0000 7f40094c2700 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e021c144c0 0x55e021c168e0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6749fd3ab82988 server_cookie=6c9c97b186e56576 in_seq=0 out_seq=0 2024-03-18T20:59:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.437+0000 7f3fed7fa700 10 client.30344 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.440+0000 7f3fed7fa700 1 -- 172.21.15.46:0/2699173838 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e021b596a0 con 0x55e021c144c0 2024-03-18T20:59:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.440+0000 7f3fed7fa700 10 client.30344 handle_command_reply: tid=0 2024-03-18T20:59:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.440+0000 7f40180bcb80 1 client.30344 shutdown 2024-03-18T20:59:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.441+0000 7f40180bcb80 1 -- 172.21.15.46:0/2699173838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0d570 msgr2=0x55e021c20ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.441+0000 7f40180bcb80 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0d570 0x55e021c20ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4000004790 tx=0x7f400000ebc0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.441+0000 7f40180bcb80 1 -- 172.21.15.46:0/2699173838 shutdown_connections 2024-03-18T20:59:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.441+0000 7f40180bcb80 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e021c144c0 0x55e021c168e0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.441+0000 7f40180bcb80 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c215c0 0x55e021c1e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.441+0000 7f40180bcb80 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0d570 0x55e021c20ff0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.441+0000 7f40180bcb80 1 --2- 172.21.15.46:0/2699173838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c0de90 0x55e021c1e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.441+0000 7f3fed7fa700 0 client.30344 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.441+0000 7f40180bcb80 1 -- 172.21.15.46:0/2699173838 >> 172.21.15.46:0/2699173838 conn(0x55e021b841c0 msgr2=0x55e021c09650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.441+0000 7f40180bcb80 1 -- 172.21.15.46:0/2699173838 shutdown_connections 2024-03-18T20:59:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.442+0000 7f40180bcb80 1 -- 172.21.15.46:0/2699173838 wait complete. 2024-03-18T20:59:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.445+0000 7f40180bcb80 20 client.30344 trim_cache size 0 max 16384 2024-03-18T20:59:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.468+0000 7f40180bcb80 1 Processor -- start 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.468+0000 7f40180bcb80 1 -- start start 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.468+0000 7f40180bcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.468+0000 7f40180bcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c09bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.468+0000 7f40180bcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c20b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f40180bcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e021c1a0e0 con 0x55e021c215c0 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f40180bcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e021c1f840 con 0x55e021c0de90 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f40180bcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e021c23c60 con 0x55e021c0d570 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f4009cc3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f400a4c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c20b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f400a4c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c20b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56276/0 (socket says 172.21.15.46:56276) 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f4009cc3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c168b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:46916/0 (socket says 172.21.15.46:46916) 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f400a4c4700 1 -- 172.21.15.46:0/2075201928 learned_addr learned my addr 172.21.15.46:0/2075201928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f40094c2700 1 --2- 172.21.15.46:0/2075201928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c09bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f400a4c4700 1 -- 172.21.15.46:0/2075201928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 msgr2=0x55e021c168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f400a4c4700 1 --2- 172.21.15.46:0/2075201928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f400a4c4700 1 -- 172.21.15.46:0/2075201928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 msgr2=0x55e021c09bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f400a4c4700 1 --2- 172.21.15.46:0/2075201928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c09bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f400a4c4700 1 -- 172.21.15.46:0/2075201928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e021b26e40 con 0x55e021c215c0 2024-03-18T20:59:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.469+0000 7f400a4c4700 1 --2- 172.21.15.46:0/2075201928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c20b20 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009470 tx=0x7f3ffc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8641d7c38073be45 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.470+0000 7f3ff8ff9700 1 -- 172.21.15.46:0/2075201928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc00e040 con 0x55e021c215c0 2024-03-18T20:59:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.470+0000 7f3ff8ff9700 1 -- 172.21.15.46:0/2075201928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ffc00f040 con 0x55e021c215c0 2024-03-18T20:59:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.470+0000 7f3ff8ff9700 1 -- 172.21.15.46:0/2075201928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc0176a0 con 0x55e021c215c0 2024-03-18T20:59:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.471+0000 7f40180bcb80 1 -- 172.21.15.46:0/2075201928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 msgr2=0x55e021c20b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.471+0000 7f40180bcb80 1 --2- 172.21.15.46:0/2075201928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c20b20 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009470 tx=0x7f3ffc019000 comp rx=0 tx=0).stop 2024-03-18T20:59:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.471+0000 7f40180bcb80 1 -- 172.21.15.46:0/2075201928 shutdown_connections 2024-03-18T20:59:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.471+0000 7f40180bcb80 1 --2- 172.21.15.46:0/2075201928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c20b20 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.471+0000 7f40180bcb80 1 --2- 172.21.15.46:0/2075201928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.471+0000 7f40180bcb80 1 --2- 172.21.15.46:0/2075201928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c09bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.471+0000 7f40180bcb80 1 -- 172.21.15.46:0/2075201928 >> 172.21.15.46:0/2075201928 conn(0x55e021b841c0 msgr2=0x55e021c42360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.471+0000 7f40180bcb80 1 -- 172.21.15.46:0/2075201928 shutdown_connections 2024-03-18T20:59:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.471+0000 7f40180bcb80 1 -- 172.21.15.46:0/2075201928 wait complete. 2024-03-18T20:59:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.471+0000 7f40180bcb80 1 -- start start 2024-03-18T20:59:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f40180bcb80 1 Processor -- start 2024-03-18T20:59:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f40180bcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c27f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f40180bcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c28450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f40180bcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c24510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f40180bcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e021c24c20 con 0x55e021c215c0 2024-03-18T20:59:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f40180bcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e021c24d90 con 0x55e021c0de90 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f40180bcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e021c24f50 con 0x55e021c0d570 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f40180bcb80 10 client.0 _ll_register_callbacks cb 0x7f4000009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f400a4c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c24510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f400a4c4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c24510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56286/0 (socket says 172.21.15.46:56286) 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f400a4c4700 1 -- 172.21.15.46:0/3271898870 learned_addr learned my addr 172.21.15.46:0/3271898870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f4009cc3700 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c27f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.472+0000 7f40094c2700 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c28450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.473+0000 7f400a4c4700 1 -- 172.21.15.46:0/3271898870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 msgr2=0x55e021c27f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.473+0000 7f400a4c4700 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c27f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.473+0000 7f400a4c4700 1 -- 172.21.15.46:0/3271898870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 msgr2=0x55e021c28450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.473+0000 7f400a4c4700 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c28450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.473+0000 7f400a4c4700 1 -- 172.21.15.46:0/3271898870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ffc0090a0 con 0x55e021c215c0 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.473+0000 7f40094c2700 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c28450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:59:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.473+0000 7f400a4c4700 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c24510 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc008ad0 tx=0x7f3ffc0099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53fa134d3e633f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.473+0000 7f3fed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:59:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.473+0000 7f3fed7fa700 1 -- 172.21.15.46:0/3271898870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc00e040 con 0x55e021c215c0 2024-03-18T20:59:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.473+0000 7f3fed7fa700 1 -- 172.21.15.46:0/3271898870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3ffc053070 con 0x55e021c215c0 2024-03-18T20:59:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.473+0000 7f40180bcb80 1 -- 172.21.15.46:0/3271898870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f400000ee50 con 0x55e021c215c0 2024-03-18T20:59:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.474+0000 7f3fed7fa700 1 -- 172.21.15.46:0/3271898870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ffc008e90 con 0x55e021c215c0 2024-03-18T20:59:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.474+0000 7f40180bcb80 10 client.20727 fetch_fsmap learned FSMap version 96 2024-03-18T20:59:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.474+0000 7f40180bcb80 1 -- 172.21.15.46:0/3271898870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x55e0219e6c70 con 0x55e021c215c0 2024-03-18T20:59:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.474+0000 7f3fed7fa700 1 -- 172.21.15.46:0/3271898870 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400000ee50 con 0x55e021c215c0 2024-03-18T20:59:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.474+0000 7f3fed7fa700 1 -- 172.21.15.46:0/3271898870 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 96) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f3ffc004020 con 0x55e021c215c0 2024-03-18T20:59:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.475+0000 7f40180bcb80 10 client.20727 fetch_fsmap finished waiting for FSMap version 96 2024-03-18T20:59:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.475+0000 7f40180bcb80 10 client.20727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:59:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.475+0000 7f40180bcb80 20 client.20727 populate_metadata read hostname 'smithi046' 2024-03-18T20:59:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.475+0000 7f40180bcb80 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e021c2d670 0x55e021c2fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.475+0000 7f40180bcb80 4 client.20727 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:59:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.475+0000 7f40180bcb80 1 -- 172.21.15.46:0/3271898870 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e0219afc90 con 0x55e021c2d670 2024-03-18T20:59:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.475+0000 7f4009cc3700 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e021c2d670 0x55e021c2fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:02.480 INFO:tasks.workunit.client.0.smithi046.stdout: 10 378547 31.65 MB/sec execute 458 sec latency 194.931 ms 2024-03-18T20:59:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.476+0000 7f4009cc3700 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e021c2d670 0x55e021c2fa90 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b576e8404f1cc5 server_cookie=4a66d3e4a0288077 in_seq=0 out_seq=0 2024-03-18T20:59:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.476+0000 7f3fed7fa700 10 client.20727 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:02.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.720+0000 7f3fed7fa700 1 -- 172.21.15.46:0/3271898870 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e0219afc90 con 0x55e021c2d670 2024-03-18T20:59:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.720+0000 7f3fed7fa700 10 client.20727 handle_command_reply: tid=0 2024-03-18T20:59:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.721+0000 7f40180bcb80 1 client.20727 shutdown 2024-03-18T20:59:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.721+0000 7f40180bcb80 1 -- 172.21.15.46:0/3271898870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 msgr2=0x55e021c24510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.721+0000 7f40180bcb80 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c24510 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc008ad0 tx=0x7f3ffc0099a0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.721+0000 7f40180bcb80 1 -- 172.21.15.46:0/3271898870 shutdown_connections 2024-03-18T20:59:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.721+0000 7f40180bcb80 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55e021c2d670 0x55e021c2fa90 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.721+0000 7f40180bcb80 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e021c215c0 0x55e021c24510 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.721+0000 7f40180bcb80 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e021c0d570 0x55e021c27f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.721+0000 7f40180bcb80 1 --2- 172.21.15.46:0/3271898870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e021c0de90 0x55e021c28450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.721+0000 7f40180bcb80 1 -- 172.21.15.46:0/3271898870 >> 172.21.15.46:0/3271898870 conn(0x55e021b841c0 msgr2=0x55e021c19540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.721+0000 7f3fed7fa700 0 client.20727 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.722+0000 7f40180bcb80 1 -- 172.21.15.46:0/3271898870 shutdown_connections 2024-03-18T20:59:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.722+0000 7f40180bcb80 1 -- 172.21.15.46:0/3271898870 wait complete. 2024-03-18T20:59:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.724+0000 7f40180bcb80 20 client.20727 trim_cache size 0 max 16384 2024-03-18T20:59:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:59:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:59:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:59:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:59:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.725+0000 7f400bf27700 1 -- 172.21.15.46:0/2281563342 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ff00a1310 msgr2=0x7f3ff00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.725+0000 7f400bf27700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ff00a1310 0x7f3ff00a37d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00f540 tx=0x7f3ffc00b040 comp rx=0 tx=0).stop 2024-03-18T20:59:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.725+0000 7f400bf27700 1 -- 172.21.15.46:0/2281563342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4004106000 msgr2=0x7f40040735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.725+0000 7f400bf27700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4004106000 0x7f40040735f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4000c00 tx=0x7f3ff4005740 comp rx=0 tx=0).stop 2024-03-18T20:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.727+0000 7f400bf27700 1 -- 172.21.15.46:0/2281563342 shutdown_connections 2024-03-18T20:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.727+0000 7f400bf27700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3ff00a1310 0x7f3ff00a37d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.727+0000 7f400bf27700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4004100ee0 0x7f40040730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.727+0000 7f400bf27700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4004073b30 0x7f40041a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.727+0000 7f400bf27700 1 --2- 172.21.15.46:0/2281563342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4004106000 0x7f40040735f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.727+0000 7f400bf27700 1 -- 172.21.15.46:0/2281563342 >> 172.21.15.46:0/2281563342 conn(0x7f40040fc780 msgr2=0x7f40040fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.728+0000 7f400bf27700 1 -- 172.21.15.46:0/2281563342 shutdown_connections 2024-03-18T20:59:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:02.728+0000 7f400bf27700 1 -- 172.21.15.46:0/2281563342 wait complete. 2024-03-18T20:59:02.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:02 smithi155 ceph-mon[48344]: pgmap v724: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 815 op/s 2024-03-18T20:59:02.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:02 smithi046 ceph-mon[43495]: pgmap v724: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 815 op/s 2024-03-18T20:59:02.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:02 smithi073 ceph-mon[47384]: pgmap v724: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 815 op/s 2024-03-18T20:59:03.478 INFO:tasks.workunit.client.0.smithi046.stdout: 10 378644 31.60 MB/sec execute 459 sec latency 310.054 ms 2024-03-18T20:59:04.480 INFO:tasks.workunit.client.0.smithi046.stdout: 10 378736 31.54 MB/sec execute 460 sec latency 721.891 ms 2024-03-18T20:59:04.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:04 smithi073 ceph-mon[47384]: pgmap v725: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 789 op/s 2024-03-18T20:59:04.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:04 smithi073 ceph-mon[47384]: osdmap e116: 12 total, 12 up, 12 in 2024-03-18T20:59:05.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:04 smithi155 ceph-mon[48344]: pgmap v725: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 789 op/s 2024-03-18T20:59:05.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:04 smithi155 ceph-mon[48344]: osdmap e116: 12 total, 12 up, 12 in 2024-03-18T20:59:05.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:04 smithi046 ceph-mon[43495]: pgmap v725: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 789 op/s 2024-03-18T20:59:05.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:04 smithi046 ceph-mon[43495]: osdmap e116: 12 total, 12 up, 12 in 2024-03-18T20:59:05.484 INFO:tasks.workunit.client.0.smithi046.stdout: 10 378875 31.48 MB/sec execute 461 sec latency 655.153 ms 2024-03-18T20:59:06.486 INFO:tasks.workunit.client.0.smithi046.stdout: 10 379074 31.43 MB/sec execute 462 sec latency 503.703 ms 2024-03-18T20:59:06.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:06 smithi073 ceph-mon[47384]: pgmap v727: 97 pgs: 35 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 42 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 15 MiB/s wr, 929 op/s 2024-03-18T20:59:06.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:06 smithi155 ceph-mon[48344]: pgmap v727: 97 pgs: 35 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 42 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 15 MiB/s wr, 929 op/s 2024-03-18T20:59:06.799 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:06 smithi046 ceph-mon[43495]: pgmap v727: 97 pgs: 35 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 42 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 15 MiB/s wr, 929 op/s 2024-03-18T20:59:07.486 INFO:tasks.workunit.client.0.smithi046.stdout: 10 379605 31.42 MB/sec execute 463 sec latency 228.003 ms 2024-03-18T20:59:08.486 INFO:tasks.workunit.client.0.smithi046.stdout: 10 380207 31.44 MB/sec execute 464 sec latency 80.486 ms 2024-03-18T20:59:08.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:08 smithi155 ceph-mon[48344]: pgmap v728: 97 pgs: 35 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 42 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 15 MiB/s wr, 929 op/s 2024-03-18T20:59:08.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:08 smithi046 ceph-mon[43495]: pgmap v728: 97 pgs: 35 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 42 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 15 MiB/s wr, 929 op/s 2024-03-18T20:59:08.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:08 smithi073 ceph-mon[47384]: pgmap v728: 97 pgs: 35 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 42 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 15 MiB/s wr, 929 op/s 2024-03-18T20:59:09.486 INFO:tasks.workunit.client.0.smithi046.stdout: 10 380825 31.42 MB/sec execute 465 sec latency 96.454 ms 2024-03-18T20:59:10.486 INFO:tasks.workunit.client.0.smithi046.stdout: 10 381443 31.46 MB/sec execute 466 sec latency 101.358 ms 2024-03-18T20:59:10.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:10 smithi155 ceph-mon[48344]: pgmap v729: 97 pgs: 19 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 70 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.8 MiB/s wr, 745 op/s 2024-03-18T20:59:10.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:10 smithi046 ceph-mon[43495]: pgmap v729: 97 pgs: 19 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 70 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.8 MiB/s wr, 745 op/s 2024-03-18T20:59:10.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:10 smithi073 ceph-mon[47384]: pgmap v729: 97 pgs: 19 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 70 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.8 MiB/s wr, 745 op/s 2024-03-18T20:59:11.487 INFO:tasks.workunit.client.0.smithi046.stdout: 10 381947 31.43 MB/sec execute 467 sec latency 100.493 ms 2024-03-18T20:59:12.487 INFO:tasks.workunit.client.0.smithi046.stdout: 10 382475 31.44 MB/sec execute 468 sec latency 95.252 ms 2024-03-18T20:59:12.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:12 smithi155 ceph-mon[48344]: pgmap v730: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.0 MiB/s wr, 619 op/s 2024-03-18T20:59:12.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:12 smithi046 ceph-mon[43495]: pgmap v730: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.0 MiB/s wr, 619 op/s 2024-03-18T20:59:12.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:12 smithi073 ceph-mon[47384]: pgmap v730: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.0 MiB/s wr, 619 op/s 2024-03-18T20:59:13.487 INFO:tasks.workunit.client.0.smithi046.stdout: 10 382953 31.42 MB/sec execute 469 sec latency 85.534 ms 2024-03-18T20:59:14.487 INFO:tasks.workunit.client.0.smithi046.stdout: 10 383466 31.42 MB/sec execute 470 sec latency 157.002 ms 2024-03-18T20:59:14.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:14 smithi155 ceph-mon[48344]: pgmap v731: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 625 op/s 2024-03-18T20:59:14.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:14 smithi046 ceph-mon[43495]: pgmap v731: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 625 op/s 2024-03-18T20:59:14.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:14 smithi073 ceph-mon[47384]: pgmap v731: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.4 MiB/s wr, 625 op/s 2024-03-18T20:59:15.487 INFO:tasks.workunit.client.0.smithi046.stdout: 10 384106 31.46 MB/sec execute 471 sec latency 83.045 ms 2024-03-18T20:59:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.876+0000 7fc91ef0b700 1 -- 172.21.15.46:0/701077191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9180680e0 msgr2=0x7fc918068560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.876+0000 7fc91ef0b700 1 --2- 172.21.15.46:0/701077191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9180680e0 0x7fc918068560 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc904009990 tx=0x7fc904009ca0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.876+0000 7fc91ef0b700 1 -- 172.21.15.46:0/701077191 shutdown_connections 2024-03-18T20:59:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.876+0000 7fc91ef0b700 1 --2- 172.21.15.46:0/701077191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc918106e10 0x7fc9181071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.876+0000 7fc91ef0b700 1 --2- 172.21.15.46:0/701077191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc918068aa0 0x7fc91810dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.876+0000 7fc91ef0b700 1 --2- 172.21.15.46:0/701077191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9180680e0 0x7fc918068560 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.876+0000 7fc91ef0b700 1 -- 172.21.15.46:0/701077191 >> 172.21.15.46:0/701077191 conn(0x7fc918076d50 msgr2=0x7fc918077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.876+0000 7fc91ef0b700 1 -- 172.21.15.46:0/701077191 shutdown_connections 2024-03-18T20:59:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.876+0000 7fc91ef0b700 1 -- 172.21.15.46:0/701077191 wait complete. 2024-03-18T20:59:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.877+0000 7fc91ef0b700 1 Processor -- start 2024-03-18T20:59:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.877+0000 7fc91ef0b700 1 -- start start 2024-03-18T20:59:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.877+0000 7fc91ef0b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9180680e0 0x7fc9180723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.877+0000 7fc91ef0b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc918068aa0 0x7fc918072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.877+0000 7fc91ef0b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc918106e10 0x7fc91806b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.877+0000 7fc91ef0b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91806bcf0 con 0x7fc918068aa0 2024-03-18T20:59:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.877+0000 7fc91ef0b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91806be60 con 0x7fc9180680e0 2024-03-18T20:59:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.877+0000 7fc91ef0b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91806c020 con 0x7fc918106e10 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.877+0000 7fc91cca7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9180680e0 0x7fc9180723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.877+0000 7fc917fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc918068aa0 0x7fc918072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.877+0000 7fc91cca7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9180680e0 0x7fc9180723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:41856/0 (socket says 172.21.15.46:41856) 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.877+0000 7fc91cca7700 1 -- 172.21.15.46:0/2075059042 learned_addr learned my addr 172.21.15.46:0/2075059042 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.878+0000 7fc91d4a8700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc918106e10 0x7fc91806b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.878+0000 7fc91cca7700 1 -- 172.21.15.46:0/2075059042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc918106e10 msgr2=0x7fc91806b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.878+0000 7fc91cca7700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc918106e10 0x7fc91806b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.878+0000 7fc91cca7700 1 -- 172.21.15.46:0/2075059042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc918068aa0 msgr2=0x7fc918072920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.878+0000 7fc91cca7700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc918068aa0 0x7fc918072920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.878+0000 7fc91cca7700 1 -- 172.21.15.46:0/2075059042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c009520 con 0x7fc9180680e0 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.878+0000 7fc91cca7700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9180680e0 0x7fc9180723a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc90c00eb30 tx=0x7fc90c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.878+0000 7fc915ffb700 1 -- 172.21.15.46:0/2075059042 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c009880 con 0x7fc9180680e0 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.878+0000 7fc915ffb700 1 -- 172.21.15.46:0/2075059042 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc90c0044e0 con 0x7fc9180680e0 2024-03-18T20:59:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.878+0000 7fc915ffb700 1 -- 172.21.15.46:0/2075059042 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c003d80 con 0x7fc9180680e0 2024-03-18T20:59:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.878+0000 7fc91ef0b700 1 -- 172.21.15.46:0/2075059042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9040095f0 con 0x7fc9180680e0 2024-03-18T20:59:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.878+0000 7fc91ef0b700 1 -- 172.21.15.46:0/2075059042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc918070160 con 0x7fc9180680e0 2024-03-18T20:59:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.882+0000 7fc915ffb700 1 -- 172.21.15.46:0/2075059042 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc90c018430 con 0x7fc9180680e0 2024-03-18T20:59:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.883+0000 7fc92b0a0b80 1 Processor -- start 2024-03-18T20:59:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.883+0000 7fc92b0a0b80 1 -- start start 2024-03-18T20:59:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.883+0000 7fc915ffb700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc9080a1310 0x7fc9080a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.884+0000 7fc92b0a0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e97486b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.884+0000 7fc92b0a0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 0x556e974837b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.884+0000 7fc92b0a0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e9747d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.884+0000 7fc92b0a0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e973d6930 con 0x556e97486760 2024-03-18T20:59:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.884+0000 7fc92b0a0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e973d42f0 con 0x556e97483330 2024-03-18T20:59:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.884+0000 7fc92b0a0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e973cfbe0 con 0x556e97483cf0 2024-03-18T20:59:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.884+0000 7fc915ffb700 1 -- 172.21.15.46:0/2075059042 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc90c014070 con 0x7fc9180680e0 2024-03-18T20:59:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.884+0000 7fc91d4a8700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc9080a1310 0x7fc9080a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.884+0000 7fc917fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 0x556e974837b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.884+0000 7fc917fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 0x556e974837b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:41862/0 (socket says 172.21.15.46:41862) 2024-03-18T20:59:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.884+0000 7fc917fff700 1 -- 172.21.15.46:0/1324002266 learned_addr learned my addr 172.21.15.46:0/1324002266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.885+0000 7fc917fff700 1 -- 172.21.15.46:0/1324002266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 msgr2=0x556e9747d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.885+0000 7fc917fff700 1 --2- 172.21.15.46:0/1324002266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e9747d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.885+0000 7fc917fff700 1 -- 172.21.15.46:0/1324002266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 msgr2=0x556e97486b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.885+0000 7fc917fff700 1 --2- 172.21.15.46:0/1324002266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e97486b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.885+0000 7fc917fff700 1 -- 172.21.15.46:0/1324002266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e97251350 con 0x556e97483330 2024-03-18T20:59:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.885+0000 7fc917fff700 1 --2- 172.21.15.46:0/1324002266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 0x556e974837b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc904015000 tx=0x7fc904005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=796e28f2f8cdf40b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.885+0000 7fc91d4a8700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc9080a1310 0x7fc9080a37d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc910009ec0 tx=0x7fc910009920 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.890+0000 7fc902ffd700 1 -- 172.21.15.46:0/1324002266 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904061070 con 0x556e97483330 2024-03-18T20:59:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.890+0000 7fc902ffd700 1 -- 172.21.15.46:0/1324002266 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc904062040 con 0x556e97483330 2024-03-18T20:59:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.890+0000 7fc902ffd700 1 -- 172.21.15.46:0/1324002266 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904004880 con 0x556e97483330 2024-03-18T20:59:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.891+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/1324002266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 msgr2=0x556e974837b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.891+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/1324002266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 0x556e974837b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc904015000 tx=0x7fc904005ec0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.891+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/1324002266 shutdown_connections 2024-03-18T20:59:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.891+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/1324002266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e97486b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.891+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/1324002266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e9747d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.891+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/1324002266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 0x556e974837b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.891+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/1324002266 >> 172.21.15.46:0/1324002266 conn(0x556e973f31c0 msgr2=0x556e97478870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.891+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/1324002266 shutdown_connections 2024-03-18T20:59:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.891+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/1324002266 wait complete. 2024-03-18T20:59:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.892+0000 7fc92b0a0b80 1 -- start start 2024-03-18T20:59:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.892+0000 7fc92b0a0b80 1 Processor -- start 2024-03-18T20:59:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.892+0000 7fc92b0a0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483330 0x556e9749e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.892+0000 7fc92b0a0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483cf0 0x556e9749ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.892+0000 7fc92b0a0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e974988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc92b0a0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9736bd20 con 0x556e97486760 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc92b0a0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9736b9c0 con 0x556e97483cf0 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc92b0a0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e97253640 con 0x556e97483330 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc92b0a0b80 10 client.0 _ll_register_callbacks cb 0x556e971d7340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc91cca7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483330 0x556e9749e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc91cca7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483330 0x556e9749e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:49270/0 (socket says 172.21.15.46:49270) 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc91cca7700 1 -- 172.21.15.46:0/69318124 learned_addr learned my addr 172.21.15.46:0/69318124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc91d4a8700 1 --2- 172.21.15.46:0/69318124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e974988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc91d4a8700 1 -- 172.21.15.46:0/69318124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483330 msgr2=0x556e9749e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc91d4a8700 1 --2- 172.21.15.46:0/69318124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483330 0x556e9749e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc91d4a8700 1 -- 172.21.15.46:0/69318124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483cf0 msgr2=0x556e9749ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc91d4a8700 1 --2- 172.21.15.46:0/69318124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483cf0 0x556e9749ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc91d4a8700 1 -- 172.21.15.46:0/69318124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91810e440 con 0x556e97486760 2024-03-18T20:59:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc91d4a8700 1 --2- 172.21.15.46:0/69318124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e974988c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc91000fca0 tx=0x7fc910004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9813ab810674fdd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.893+0000 7fc8f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:59:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.894+0000 7fc8f7fff700 1 -- 172.21.15.46:0/69318124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910026070 con 0x556e97486760 2024-03-18T20:59:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.894+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/69318124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e974706c0 con 0x556e97486760 2024-03-18T20:59:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.894+0000 7fc8f7fff700 1 -- 172.21.15.46:0/69318124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc910019e30 con 0x556e97486760 2024-03-18T20:59:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.894+0000 7fc8f7fff700 1 -- 172.21.15.46:0/69318124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc91000e040 con 0x556e97486760 2024-03-18T20:59:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.894+0000 7fc92b0a0b80 10 client.20736 fetch_fsmap learned FSMap version 96 2024-03-18T20:59:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.894+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/69318124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x556e971fdb00 con 0x556e97486760 2024-03-18T20:59:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.894+0000 7fc8f7fff700 1 -- 172.21.15.46:0/69318124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc91002e020 con 0x556e97486760 2024-03-18T20:59:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.895+0000 7fc8f7fff700 1 -- 172.21.15.46:0/69318124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 96) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fc9100040b0 con 0x556e97486760 2024-03-18T20:59:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.895+0000 7fc92b0a0b80 10 client.20736 fetch_fsmap finished waiting for FSMap version 96 2024-03-18T20:59:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.895+0000 7fc92b0a0b80 10 client.20736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:59:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.895+0000 7fc92b0a0b80 20 client.20736 populate_metadata read hostname 'smithi046' 2024-03-18T20:59:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.895+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/69318124 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556e974a8be0 0x556e974ab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.895+0000 7fc92b0a0b80 4 client.20736 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:59:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.895+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/69318124 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e973c86a0 con 0x556e974a8be0 2024-03-18T20:59:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.896+0000 7fc91cca7700 1 --2- 172.21.15.46:0/69318124 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556e974a8be0 0x556e974ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.897+0000 7fc8f7fff700 10 client.20736 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.897+0000 7fc91cca7700 1 --2- 172.21.15.46:0/69318124 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556e974a8be0 0x556e974ab000 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78eefc6d4042c472 server_cookie=c985dec4c1b45180 in_seq=0 out_seq=0 2024-03-18T20:59:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.898+0000 7fc8f7fff700 1 -- 172.21.15.46:0/69318124 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556e973c86a0 con 0x556e974a8be0 2024-03-18T20:59:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.898+0000 7fc8f7fff700 10 client.20736 handle_command_reply: tid=0 2024-03-18T20:59:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc92b0a0b80 1 client.20736 shutdown 2024-03-18T20:59:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/69318124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 msgr2=0x556e974988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/69318124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e974988c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc91000fca0 tx=0x7fc910004ad0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/69318124 shutdown_connections 2024-03-18T20:59:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/69318124 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556e974a8be0 0x556e974ab000 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/69318124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e974988c0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/69318124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483330 0x556e9749e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/69318124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483cf0 0x556e9749ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/69318124 >> 172.21.15.46:0/69318124 conn(0x556e973f31c0 msgr2=0x556e97477bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc8f7fff700 0 client.20736 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/69318124 shutdown_connections 2024-03-18T20:59:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/69318124 wait complete. 2024-03-18T20:59:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.899+0000 7fc92b0a0b80 20 client.20736 trim_cache size 0 max 16384 2024-03-18T20:59:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc92b0a0b80 1 Processor -- start 2024-03-18T20:59:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc92b0a0b80 1 -- start start 2024-03-18T20:59:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc92b0a0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97483330 0x556e9747ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc92b0a0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e97482190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc92b0a0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97486760 0x556e974826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc92b0a0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e974a55a0 con 0x556e97483330 2024-03-18T20:59:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc92b0a0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e97479790 con 0x556e97486760 2024-03-18T20:59:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc92b0a0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e974a6130 con 0x556e97483cf0 2024-03-18T20:59:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc917fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e97482190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc91cca7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97483330 0x556e9747ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc91cca7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97483330 0x556e9747ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37328/0 (socket says 172.21.15.46:37328) 2024-03-18T20:59:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc917fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e97482190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:49276/0 (socket says 172.21.15.46:49276) 2024-03-18T20:59:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.924+0000 7fc91cca7700 1 -- 172.21.15.46:0/1376426765 learned_addr learned my addr 172.21.15.46:0/1376426765 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.925+0000 7fc91d4a8700 1 --2- 172.21.15.46:0/1376426765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97486760 0x556e974826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.925+0000 7fc917fff700 1 -- 172.21.15.46:0/1376426765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97486760 msgr2=0x556e974826d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.925+0000 7fc917fff700 1 --2- 172.21.15.46:0/1376426765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97486760 0x556e974826d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.925+0000 7fc917fff700 1 -- 172.21.15.46:0/1376426765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97483330 msgr2=0x556e9747ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.925+0000 7fc917fff700 1 --2- 172.21.15.46:0/1376426765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97483330 0x556e9747ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.925+0000 7fc917fff700 1 -- 172.21.15.46:0/1376426765 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e97395e40 con 0x556e97483cf0 2024-03-18T20:59:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.925+0000 7fc91cca7700 1 --2- 172.21.15.46:0/1376426765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97483330 0x556e9747ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:59:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.925+0000 7fc91d4a8700 1 --2- 172.21.15.46:0/1376426765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97486760 0x556e974826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:59:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.925+0000 7fc917fff700 1 --2- 172.21.15.46:0/1376426765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e97482190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x556e974702f0 tx=0x7fc904009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89c81126349dbfdb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.925+0000 7fc9037fe700 1 -- 172.21.15.46:0/1376426765 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904061070 con 0x556e97483cf0 2024-03-18T20:59:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.925+0000 7fc9037fe700 1 -- 172.21.15.46:0/1376426765 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc904005650 con 0x556e97483cf0 2024-03-18T20:59:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.926+0000 7fc9037fe700 1 -- 172.21.15.46:0/1376426765 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904004b40 con 0x556e97483cf0 2024-03-18T20:59:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.927+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/1376426765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 msgr2=0x556e97482190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.927+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/1376426765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e97482190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x556e974702f0 tx=0x7fc904009ca0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.927+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/1376426765 shutdown_connections 2024-03-18T20:59:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.927+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/1376426765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97483330 0x556e9747ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.927+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/1376426765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97486760 0x556e974826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.927+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/1376426765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e97482190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.927+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/1376426765 >> 172.21.15.46:0/1376426765 conn(0x556e973f31c0 msgr2=0x556e9747ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.930+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/1376426765 shutdown_connections 2024-03-18T20:59:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.930+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/1376426765 wait complete. 2024-03-18T20:59:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.931+0000 7fc92b0a0b80 1 -- start start 2024-03-18T20:59:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.932+0000 7fc92b0a0b80 1 Processor -- start 2024-03-18T20:59:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.932+0000 7fc92b0a0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 0x556e974a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.932+0000 7fc92b0a0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e974a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.932+0000 7fc92b0a0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e974a4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.932+0000 7fc92b0a0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e974a49a0 con 0x556e97486760 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.932+0000 7fc92b0a0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e974a4b10 con 0x556e97483330 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.932+0000 7fc92b0a0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e974fba60 con 0x556e97483cf0 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.932+0000 7fc92b0a0b80 10 client.0 _ll_register_callbacks cb 0x7fc910019800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.932+0000 7fc91cca7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 0x556e974a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc91cca7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 0x556e974a5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:41902/0 (socket says 172.21.15.46:41902) 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc91cca7700 1 -- 172.21.15.46:0/2005822196 learned_addr learned my addr 172.21.15.46:0/2005822196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc917fff700 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e974a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc91d4a8700 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e974a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc91cca7700 1 -- 172.21.15.46:0/2005822196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 msgr2=0x556e974a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc91cca7700 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e974a3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc91cca7700 1 -- 172.21.15.46:0/2005822196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 msgr2=0x556e974a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc91cca7700 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e974a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc91cca7700 1 -- 172.21.15.46:0/2005822196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c0af040 con 0x556e97483330 2024-03-18T20:59:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc91d4a8700 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e974a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T20:59:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc91cca7700 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 0x556e974a5110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc90c096790 tx=0x7fc90c09a990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=975af5414bcdd03b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc8f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc8f7fff700 1 -- 172.21.15.46:0/2005822196 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c09e0c0 con 0x556e97483330 2024-03-18T20:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/2005822196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc904062040 con 0x556e97483330 2024-03-18T20:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.933+0000 7fc8f7fff700 1 -- 172.21.15.46:0/2005822196 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc90c0b2e40 con 0x556e97483330 2024-03-18T20:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc8f7fff700 1 -- 172.21.15.46:0/2005822196 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc90c0a2710 con 0x556e97483330 2024-03-18T20:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc92b0a0b80 10 client.30365 fetch_fsmap learned FSMap version 96 2024-03-18T20:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/2005822196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=4+}) v3 -- 0x556e971fdb00 con 0x556e97483330 2024-03-18T20:59:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc8f7fff700 1 -- 172.21.15.46:0/2005822196 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc904062040 con 0x556e97483330 2024-03-18T20:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc8f7fff700 1 -- 172.21.15.46:0/2005822196 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 96) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fc90c0b2040 con 0x556e97483330 2024-03-18T20:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc92b0a0b80 10 client.30365 fetch_fsmap finished waiting for FSMap version 96 2024-03-18T20:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc92b0a0b80 10 client.30365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc92b0a0b80 20 client.30365 populate_metadata read hostname 'smithi046' 2024-03-18T20:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556e974fd310 0x556e974ff730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc92b0a0b80 4 client.30365 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/2005822196 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556e9721ec90 con 0x556e974fd310 2024-03-18T20:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.934+0000 7fc917fff700 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556e974fd310 0x556e974ff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.935+0000 7fc917fff700 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556e974fd310 0x556e974ff730 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520ee40b57012898 server_cookie=9e2c1153459d2a6d in_seq=0 out_seq=0 2024-03-18T20:59:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:15.935+0000 7fc8f7fff700 10 client.30365 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.244+0000 7fc8f7fff700 1 -- 172.21.15.46:0/2005822196 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556e9721ec90 con 0x556e974fd310 2024-03-18T20:59:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.244+0000 7fc8f7fff700 10 client.30365 handle_command_reply: tid=0 2024-03-18T20:59:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.244+0000 7fc92b0a0b80 1 client.30365 shutdown 2024-03-18T20:59:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.249+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/2005822196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 msgr2=0x556e974a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.249+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 0x556e974a5110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc90c096790 tx=0x7fc90c09a990 comp rx=0 tx=0).stop 2024-03-18T20:59:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.249+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/2005822196 shutdown_connections 2024-03-18T20:59:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.249+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x556e974fd310 0x556e974ff730 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.249+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e97486760 0x556e974a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.249+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556e97483330 0x556e974a5110 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.249+0000 7fc92b0a0b80 1 --2- 172.21.15.46:0/2005822196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556e97483cf0 0x556e974a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.249+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/2005822196 >> 172.21.15.46:0/2005822196 conn(0x556e973f31c0 msgr2=0x556e9747c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.249+0000 7fc8f7fff700 0 client.30365 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.249+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/2005822196 shutdown_connections 2024-03-18T20:59:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.251+0000 7fc92b0a0b80 1 -- 172.21.15.46:0/2005822196 wait complete. 2024-03-18T20:59:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.253+0000 7fc92b0a0b80 20 client.30365 trim_cache size 0 max 16384 2024-03-18T20:59:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:59:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:59:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:59:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:59:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.254+0000 7fc91ef0b700 1 -- 172.21.15.46:0/2075059042 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc9080a1310 msgr2=0x7fc9080a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.254+0000 7fc91ef0b700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc9080a1310 0x7fc9080a37d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc910009ec0 tx=0x7fc910009920 comp rx=0 tx=0).stop 2024-03-18T20:59:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.254+0000 7fc91ef0b700 1 -- 172.21.15.46:0/2075059042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9180680e0 msgr2=0x7fc9180723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.254+0000 7fc91ef0b700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9180680e0 0x7fc9180723a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc90c00eb30 tx=0x7fc90c00ee40 comp rx=0 tx=0).stop 2024-03-18T20:59:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.254+0000 7fc91ef0b700 1 -- 172.21.15.46:0/2075059042 shutdown_connections 2024-03-18T20:59:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.254+0000 7fc91ef0b700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc9080a1310 0x7fc9080a37d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.254+0000 7fc91ef0b700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc918106e10 0x7fc91806b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.254+0000 7fc91ef0b700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9180680e0 0x7fc9180723a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.254+0000 7fc91ef0b700 1 --2- 172.21.15.46:0/2075059042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc918068aa0 0x7fc918072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.254+0000 7fc91ef0b700 1 -- 172.21.15.46:0/2075059042 >> 172.21.15.46:0/2075059042 conn(0x7fc918076d50 msgr2=0x7fc9180feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.254+0000 7fc91ef0b700 1 -- 172.21.15.46:0/2075059042 shutdown_connections 2024-03-18T20:59:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:16.254+0000 7fc91ef0b700 1 -- 172.21.15.46:0/2075059042 wait complete. 2024-03-18T20:59:16.487 INFO:tasks.workunit.client.0.smithi046.stdout: 10 384436 31.42 MB/sec execute 472 sec latency 226.878 ms 2024-03-18T20:59:16.541 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:16 smithi155 ceph-mon[48344]: osdmap e117: 12 total, 12 up, 12 in 2024-03-18T20:59:16.542 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:16 smithi155 ceph-mon[48344]: pgmap v733: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.9 MiB/s wr, 436 op/s 2024-03-18T20:59:16.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:16 smithi046 ceph-mon[43495]: osdmap e117: 12 total, 12 up, 12 in 2024-03-18T20:59:16.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:16 smithi046 ceph-mon[43495]: pgmap v733: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.9 MiB/s wr, 436 op/s 2024-03-18T20:59:16.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:16 smithi073 ceph-mon[47384]: osdmap e117: 12 total, 12 up, 12 in 2024-03-18T20:59:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:16 smithi073 ceph-mon[47384]: pgmap v733: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.9 MiB/s wr, 436 op/s 2024-03-18T20:59:17.492 INFO:tasks.workunit.client.0.smithi046.stdout: 10 385031 31.42 MB/sec execute 473 sec latency 219.744 ms 2024-03-18T20:59:18.488 INFO:tasks.workunit.client.0.smithi046.stdout: 10 385617 31.43 MB/sec execute 474 sec latency 126.353 ms 2024-03-18T20:59:18.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:18 smithi155 ceph-mon[48344]: pgmap v734: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.9 MiB/s wr, 436 op/s 2024-03-18T20:59:18.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:18 smithi046 ceph-mon[43495]: pgmap v734: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.9 MiB/s wr, 436 op/s 2024-03-18T20:59:18.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:18 smithi073 ceph-mon[47384]: pgmap v734: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.9 MiB/s wr, 436 op/s 2024-03-18T20:59:19.488 INFO:tasks.workunit.client.0.smithi046.stdout: 10 386183 31.43 MB/sec execute 475 sec latency 110.569 ms 2024-03-18T20:59:20.491 INFO:tasks.workunit.client.0.smithi046.stdout: 10 386718 31.42 MB/sec execute 476 sec latency 93.721 ms 2024-03-18T20:59:20.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:20 smithi155 ceph-mon[48344]: pgmap v735: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.8 MiB/s wr, 513 op/s 2024-03-18T20:59:20.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:20 smithi046 ceph-mon[43495]: pgmap v735: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.8 MiB/s wr, 513 op/s 2024-03-18T20:59:20.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:20 smithi073 ceph-mon[47384]: pgmap v735: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.8 MiB/s wr, 513 op/s 2024-03-18T20:59:21.491 INFO:tasks.workunit.client.0.smithi046.stdout: 10 387218 31.41 MB/sec execute 477 sec latency 151.756 ms 2024-03-18T20:59:22.491 INFO:tasks.workunit.client.0.smithi046.stdout: 10 387717 31.39 MB/sec execute 478 sec latency 108.028 ms 2024-03-18T20:59:22.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:22 smithi155 ceph-mon[48344]: pgmap v736: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.0 MiB/s wr, 531 op/s 2024-03-18T20:59:22.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:22 smithi046 ceph-mon[43495]: pgmap v736: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.0 MiB/s wr, 531 op/s 2024-03-18T20:59:22.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:22 smithi073 ceph-mon[47384]: pgmap v736: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.0 MiB/s wr, 531 op/s 2024-03-18T20:59:23.491 INFO:tasks.workunit.client.0.smithi046.stdout: 10 388280 31.39 MB/sec execute 479 sec latency 120.448 ms 2024-03-18T20:59:24.491 INFO:tasks.workunit.client.0.smithi046.stdout: 10 389006 31.41 MB/sec execute 480 sec latency 72.061 ms 2024-03-18T20:59:24.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:24 smithi155 ceph-mon[48344]: pgmap v737: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.0 MiB/s wr, 526 op/s 2024-03-18T20:59:24.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:24 smithi046 ceph-mon[43495]: pgmap v737: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.0 MiB/s wr, 526 op/s 2024-03-18T20:59:24.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:24 smithi073 ceph-mon[47384]: pgmap v737: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.0 MiB/s wr, 526 op/s 2024-03-18T20:59:25.491 INFO:tasks.workunit.client.0.smithi046.stdout: 10 389592 31.43 MB/sec execute 481 sec latency 94.087 ms 2024-03-18T20:59:26.491 INFO:tasks.workunit.client.0.smithi046.stdout: 10 390043 31.40 MB/sec execute 482 sec latency 78.279 ms 2024-03-18T20:59:26.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:26 smithi073 ceph-mon[47384]: pgmap v738: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 10 MiB/s wr, 559 op/s 2024-03-18T20:59:26.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:26 smithi155 ceph-mon[48344]: pgmap v738: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 10 MiB/s wr, 559 op/s 2024-03-18T20:59:26.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:26 smithi046 ceph-mon[43495]: pgmap v738: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 10 MiB/s wr, 559 op/s 2024-03-18T20:59:27.491 INFO:tasks.workunit.client.0.smithi046.stdout: 10 390584 31.39 MB/sec execute 483 sec latency 82.438 ms 2024-03-18T20:59:28.491 INFO:tasks.workunit.client.0.smithi046.stdout: 10 390978 31.37 MB/sec execute 484 sec latency 110.420 ms 2024-03-18T20:59:28.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:28 smithi155 ceph-mon[48344]: pgmap v739: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.5 MiB/s wr, 473 op/s 2024-03-18T20:59:28.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:28 smithi046 ceph-mon[43495]: pgmap v739: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.5 MiB/s wr, 473 op/s 2024-03-18T20:59:28.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:28 smithi073 ceph-mon[47384]: pgmap v739: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.5 MiB/s wr, 473 op/s 2024-03-18T20:59:29.491 INFO:tasks.workunit.client.0.smithi046.stdout: 10 391477 31.35 MB/sec execute 485 sec latency 222.685 ms 2024-03-18T20:59:29.577 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.575+0000 7f66286c1700 1 -- 172.21.15.46:0/135025910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66200ffe20 msgr2=0x7f6620100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.575+0000 7f66286c1700 1 --2- 172.21.15.46:0/135025910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66200ffe20 0x7f6620100200 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f66140098f0 tx=0x7f6614009c00 comp rx=0 tx=0).stop 2024-03-18T20:59:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.575+0000 7f66286c1700 1 -- 172.21.15.46:0/135025910 shutdown_connections 2024-03-18T20:59:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.575+0000 7f66286c1700 1 --2- 172.21.15.46:0/135025910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6620104f40 0x7f662010cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.575+0000 7f66286c1700 1 --2- 172.21.15.46:0/135025910 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66200ffe20 0x7f6620100200 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.575+0000 7f66286c1700 1 --2- 172.21.15.46:0/135025910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66201007d0 0x7f6620104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.575+0000 7f66286c1700 1 -- 172.21.15.46:0/135025910 >> 172.21.15.46:0/135025910 conn(0x7f66200fb680 msgr2=0x7f66200fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.575+0000 7f66286c1700 1 -- 172.21.15.46:0/135025910 shutdown_connections 2024-03-18T20:59:29.578 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.575+0000 7f66286c1700 1 -- 172.21.15.46:0/135025910 wait complete. 2024-03-18T20:59:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.576+0000 7f66286c1700 1 Processor -- start 2024-03-18T20:59:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.576+0000 7f66286c1700 1 -- start start 2024-03-18T20:59:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.576+0000 7f66286c1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66200ffe20 0x7f6620115060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.576+0000 7f66286c1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6620104f40 0x7f66201155a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.576+0000 7f66286c1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f662010f260 0x7f662010f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.576+0000 7f66286c1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662010fee0 con 0x7f6620104f40 2024-03-18T20:59:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.576+0000 7f66286c1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66201b28b0 con 0x7f66200ffe20 2024-03-18T20:59:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.576+0000 7f66286c1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66201b29f0 con 0x7f662010f260 2024-03-18T20:59:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f662645d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66200ffe20 0x7f6620115060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f662645d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66200ffe20 0x7f6620115060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47406/0 (socket says 172.21.15.46:47406) 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f662645d700 1 -- 172.21.15.46:0/3070428443 learned_addr learned my addr 172.21.15.46:0/3070428443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f6626c5e700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f662010f260 0x7f662010f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f6625c5c700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6620104f40 0x7f66201155a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f662645d700 1 -- 172.21.15.46:0/3070428443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f662010f260 msgr2=0x7f662010f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f662645d700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f662010f260 0x7f662010f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f662645d700 1 -- 172.21.15.46:0/3070428443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6620104f40 msgr2=0x7f66201155a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f662645d700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6620104f40 0x7f66201155a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f662645d700 1 -- 172.21.15.46:0/3070428443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614009520 con 0x7f66200ffe20 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f662645d700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66200ffe20 0x7f6620115060 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6614005e90 tx=0x7f661400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f66137fe700 1 -- 172.21.15.46:0/3070428443 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661401c070 con 0x7f66200ffe20 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f66286c1700 1 -- 172.21.15.46:0/3070428443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66201b2bf0 con 0x7f66200ffe20 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f66137fe700 1 -- 172.21.15.46:0/3070428443 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6614005040 con 0x7f66200ffe20 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f66137fe700 1 -- 172.21.15.46:0/3070428443 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614003a20 con 0x7f66200ffe20 2024-03-18T20:59:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.577+0000 7f66286c1700 1 -- 172.21.15.46:0/3070428443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66201b3010 con 0x7f66200ffe20 2024-03-18T20:59:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.582+0000 7f6634856b80 1 Processor -- start 2024-03-18T20:59:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.582+0000 7f66137fe700 1 -- 172.21.15.46:0/3070428443 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6614003b80 con 0x7f66200ffe20 2024-03-18T20:59:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.584+0000 7f66137fe700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f660c0a13c0 0x7f660c0a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.584+0000 7f66137fe700 1 -- 172.21.15.46:0/3070428443 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f66140ce860 con 0x7f66200ffe20 2024-03-18T20:59:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.584+0000 7f6634856b80 1 -- start start 2024-03-18T20:59:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.584+0000 7f6634856b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439279570 0x55b439279950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.584+0000 7f6634856b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b439279e90 0x55b439284870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.584+0000 7f6634856b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439284db0 0x55b4392871a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.584+0000 7f6634856b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4391d3930 con 0x55b439279570 2024-03-18T20:59:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.584+0000 7f6634856b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4391d12f0 con 0x55b439279e90 2024-03-18T20:59:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.584+0000 7f6634856b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4391ccbe0 con 0x55b439284db0 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.585+0000 7f6626c5e700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f660c0a13c0 0x7f660c0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.586+0000 7f6626c5e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439284db0 0x55b4392871a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.586+0000 7f6626c5e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439284db0 0x55b4392871a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59620/0 (socket says 172.21.15.46:59620) 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.586+0000 7f6626c5e700 1 -- 172.21.15.46:0/3906650676 learned_addr learned my addr 172.21.15.46:0/3906650676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.586+0000 7f6626c5e700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f660c0a13c0 0x7f660c0a3880 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f661800f930 tx=0x7f661800fcf0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.586+0000 7f662645d700 1 --2- 172.21.15.46:0/3906650676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439279570 0x55b439279950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.586+0000 7f662645d700 1 -- 172.21.15.46:0/3906650676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439284db0 msgr2=0x55b4392871a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.586+0000 7f662645d700 1 --2- 172.21.15.46:0/3906650676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439284db0 0x55b4392871a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.586+0000 7f662645d700 1 -- 172.21.15.46:0/3906650676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b439279e90 msgr2=0x55b439284870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.586+0000 7f6625c5c700 1 --2- 172.21.15.46:0/3906650676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b439279e90 0x55b439284870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.587+0000 7f662645d700 1 --2- 172.21.15.46:0/3906650676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b439279e90 0x55b439284870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.587+0000 7f662645d700 1 -- 172.21.15.46:0/3906650676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b43904e350 con 0x55b439279570 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.587+0000 7f6625c5c700 1 --2- 172.21.15.46:0/3906650676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b439279e90 0x55b439284870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T20:59:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.587+0000 7f662645d700 1 --2- 172.21.15.46:0/3906650676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439279570 0x55b439279950 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f66140cbf30 tx=0x7f661409b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d412fe4505c3266 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.588+0000 7f6610ff9700 1 -- 172.21.15.46:0/3906650676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661409b980 con 0x55b439279570 2024-03-18T20:59:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.588+0000 7f6610ff9700 1 -- 172.21.15.46:0/3906650676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f661409f7b0 con 0x55b439279570 2024-03-18T20:59:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.588+0000 7f6610ff9700 1 -- 172.21.15.46:0/3906650676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661409ae70 con 0x55b439279570 2024-03-18T20:59:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.590+0000 7f6634856b80 1 -- 172.21.15.46:0/3906650676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439279570 msgr2=0x55b439279950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.590+0000 7f6634856b80 1 --2- 172.21.15.46:0/3906650676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439279570 0x55b439279950 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f66140cbf30 tx=0x7f661409b330 comp rx=0 tx=0).stop 2024-03-18T20:59:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.591+0000 7f6634856b80 1 -- 172.21.15.46:0/3906650676 shutdown_connections 2024-03-18T20:59:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.591+0000 7f6634856b80 1 --2- 172.21.15.46:0/3906650676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439279570 0x55b439279950 secure :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f66140cbf30 tx=0x7f661409b330 comp rx=0 tx=0).stop 2024-03-18T20:59:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.591+0000 7f6634856b80 1 --2- 172.21.15.46:0/3906650676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439284db0 0x55b4392871a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.591+0000 7f6634856b80 1 --2- 172.21.15.46:0/3906650676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b439279e90 0x55b439284870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.591+0000 7f6634856b80 1 -- 172.21.15.46:0/3906650676 >> 172.21.15.46:0/3906650676 conn(0x55b4391f01c0 msgr2=0x55b439275a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.602+0000 7f6634856b80 1 -- 172.21.15.46:0/3906650676 shutdown_connections 2024-03-18T20:59:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.602+0000 7f6634856b80 1 -- 172.21.15.46:0/3906650676 wait complete. 2024-03-18T20:59:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.602+0000 7f6634856b80 1 -- start start 2024-03-18T20:59:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.603+0000 7f6634856b80 1 Processor -- start 2024-03-18T20:59:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.603+0000 7f6634856b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b439281d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.603+0000 7f6634856b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 0x55b439282290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.603+0000 7f6634856b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4392827d0 0x55b4392907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.603+0000 7f6634856b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b439168d20 con 0x55b439284db0 2024-03-18T20:59:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.603+0000 7f6634856b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4391689c0 con 0x55b4392827d0 2024-03-18T20:59:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.603+0000 7f6634856b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b439050640 con 0x55b439279e90 2024-03-18T20:59:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.603+0000 7f6634856b80 10 client.0 _ll_register_callbacks cb 0x55b438fd4340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:59:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.611+0000 7f6626c5e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4392827d0 0x55b4392907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.611+0000 7f662645d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b439281d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.611+0000 7f6625c5c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 0x55b439282290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.611+0000 7f6625c5c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 0x55b439282290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47498/0 (socket says 172.21.15.46:47498) 2024-03-18T20:59:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.611+0000 7f6625c5c700 1 -- 172.21.15.46:0/3693678478 learned_addr learned my addr 172.21.15.46:0/3693678478 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.621+0000 7f6625c5c700 1 -- 172.21.15.46:0/3693678478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 msgr2=0x55b439281d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.621+0000 7f6625c5c700 1 --2- 172.21.15.46:0/3693678478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b439281d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.621+0000 7f6625c5c700 1 -- 172.21.15.46:0/3693678478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4392827d0 msgr2=0x55b4392907a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.621+0000 7f6625c5c700 1 --2- 172.21.15.46:0/3693678478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4392827d0 0x55b4392907a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.621+0000 7f6625c5c700 1 -- 172.21.15.46:0/3693678478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66201b3010 con 0x55b439284db0 2024-03-18T20:59:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.622+0000 7f6625c5c700 1 --2- 172.21.15.46:0/3693678478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 0x55b439282290 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f661c0126b0 tx=0x7f661c012a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedf08072d97ae2e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.622+0000 7f6609ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:59:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.632+0000 7f6609ffb700 1 -- 172.21.15.46:0/3693678478 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c0106c0 con 0x55b439284db0 2024-03-18T20:59:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.632+0000 7f6609ffb700 1 -- 172.21.15.46:0/3693678478 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f661c010820 con 0x55b439284db0 2024-03-18T20:59:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.632+0000 7f6634856b80 1 -- 172.21.15.46:0/3693678478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b43926d6c0 con 0x55b439284db0 2024-03-18T20:59:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.633+0000 7f6609ffb700 1 -- 172.21.15.46:0/3693678478 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7f661c0109e0 con 0x55b439284db0 2024-03-18T20:59:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.636+0000 7f6634856b80 10 client.20754 fetch_fsmap learned FSMap version 97 2024-03-18T20:59:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.636+0000 7f6634856b80 1 -- 172.21.15.46:0/3693678478 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=4+}) v3 -- 0x55b438ffab00 con 0x55b439284db0 2024-03-18T20:59:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.644+0000 7f6609ffb700 1 -- 172.21.15.46:0/3693678478 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b43926d6c0 con 0x55b439284db0 2024-03-18T20:59:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.644+0000 7f6609ffb700 1 -- 172.21.15.46:0/3693678478 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 97) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f661c022070 con 0x55b439284db0 2024-03-18T20:59:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.645+0000 7f6634856b80 10 client.20754 fetch_fsmap finished waiting for FSMap version 97 2024-03-18T20:59:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.645+0000 7f6634856b80 10 client.20754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:59:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.645+0000 7f6634856b80 20 client.20754 populate_metadata read hostname 'smithi046' 2024-03-18T20:59:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.645+0000 7f6634856b80 1 --2- 172.21.15.46:0/3693678478 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b439294cd0 0x55b4392970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.645+0000 7f6634856b80 4 client.20754 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:59:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.645+0000 7f6634856b80 1 -- 172.21.15.46:0/3693678478 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4391c56a0 con 0x55b439294cd0 2024-03-18T20:59:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.653+0000 7f662645d700 1 --2- 172.21.15.46:0/3693678478 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b439294cd0 0x55b4392970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.679+0000 7f662645d700 1 --2- 172.21.15.46:0/3693678478 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b439294cd0 0x55b4392970f0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce1c071cebc1494a server_cookie=ebb62c51db5a899f in_seq=0 out_seq=0 2024-03-18T20:59:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.679+0000 7f6609ffb700 10 client.20754 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.687+0000 7f6609ffb700 1 -- 172.21.15.46:0/3693678478 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b4391c56a0 con 0x55b439294cd0 2024-03-18T20:59:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.687+0000 7f6609ffb700 10 client.20754 handle_command_reply: tid=0 2024-03-18T20:59:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.690+0000 7f6634856b80 1 client.20754 shutdown 2024-03-18T20:59:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.696+0000 7f6634856b80 1 -- 172.21.15.46:0/3693678478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 msgr2=0x55b439282290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.696+0000 7f6634856b80 1 --2- 172.21.15.46:0/3693678478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 0x55b439282290 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f661c0126b0 tx=0x7f661c012a70 comp rx=0 tx=0).stop 2024-03-18T20:59:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.696+0000 7f6634856b80 1 -- 172.21.15.46:0/3693678478 shutdown_connections 2024-03-18T20:59:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.696+0000 7f6634856b80 1 --2- 172.21.15.46:0/3693678478 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b439294cd0 0x55b4392970f0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.696+0000 7f6634856b80 1 --2- 172.21.15.46:0/3693678478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4392827d0 0x55b4392907a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.696+0000 7f6634856b80 1 --2- 172.21.15.46:0/3693678478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b439281d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.696+0000 7f6634856b80 1 --2- 172.21.15.46:0/3693678478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 0x55b439282290 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.696+0000 7f6634856b80 1 -- 172.21.15.46:0/3693678478 >> 172.21.15.46:0/3693678478 conn(0x55b4391f01c0 msgr2=0x55b439275650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.696+0000 7f6609ffb700 0 client.20754 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.698+0000 7f6634856b80 1 -- 172.21.15.46:0/3693678478 shutdown_connections 2024-03-18T20:59:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.708+0000 7f6634856b80 1 -- 172.21.15.46:0/3693678478 wait complete. 2024-03-18T20:59:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.713+0000 7f6634856b80 20 client.20754 trim_cache size 0 max 16384 2024-03-18T20:59:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.741+0000 7f6634856b80 1 Processor -- start 2024-03-18T20:59:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.744+0000 7f6634856b80 1 -- start start 2024-03-18T20:59:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.744+0000 7f6634856b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b439281610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.744+0000 7f6634856b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4392827d0 0x55b439290930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.744+0000 7f6634856b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b439284db0 0x55b439283bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.744+0000 7f6634856b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b439292230 con 0x55b4392827d0 2024-03-18T20:59:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.744+0000 7f6634856b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4392847c0 con 0x55b439284db0 2024-03-18T20:59:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.744+0000 7f6634856b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43927daf0 con 0x55b439279e90 2024-03-18T20:59:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.745+0000 7f662645d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b439281610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.745+0000 7f662645d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b439281610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:59636/0 (socket says 172.21.15.46:59636) 2024-03-18T20:59:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.745+0000 7f662645d700 1 -- 172.21.15.46:0/2900623275 learned_addr learned my addr 172.21.15.46:0/2900623275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.754+0000 7f662645d700 1 -- 172.21.15.46:0/2900623275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b439284db0 msgr2=0x55b439283bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.754+0000 7f662645d700 1 --2- 172.21.15.46:0/2900623275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b439284db0 0x55b439283bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.754+0000 7f662645d700 1 -- 172.21.15.46:0/2900623275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4392827d0 msgr2=0x55b439290930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.754+0000 7f662645d700 1 --2- 172.21.15.46:0/2900623275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4392827d0 0x55b439290930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.754+0000 7f662645d700 1 -- 172.21.15.46:0/2900623275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4392718f0 con 0x55b439279e90 2024-03-18T20:59:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.766+0000 7f662645d700 1 --2- 172.21.15.46:0/2900623275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b439281610 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f661409b680 tx=0x7f661400b2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e5b1b88a57815d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.775+0000 7f66117fa700 1 -- 172.21.15.46:0/2900623275 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66140a3d70 con 0x55b439279e90 2024-03-18T20:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.775+0000 7f66117fa700 1 -- 172.21.15.46:0/2900623275 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f661409b980 con 0x55b439279e90 2024-03-18T20:59:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.775+0000 7f66117fa700 1 -- 172.21.15.46:0/2900623275 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661409b0d0 con 0x55b439279e90 2024-03-18T20:59:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.780+0000 7f6634856b80 1 -- 172.21.15.46:0/2900623275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 msgr2=0x55b439281610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.780+0000 7f6634856b80 1 --2- 172.21.15.46:0/2900623275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b439281610 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f661409b680 tx=0x7f661400b2e0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.780+0000 7f6634856b80 1 -- 172.21.15.46:0/2900623275 shutdown_connections 2024-03-18T20:59:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.780+0000 7f6634856b80 1 --2- 172.21.15.46:0/2900623275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b439284db0 0x55b439283bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.780+0000 7f6634856b80 1 --2- 172.21.15.46:0/2900623275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b439281610 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.780+0000 7f6634856b80 1 --2- 172.21.15.46:0/2900623275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b4392827d0 0x55b439290930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.780+0000 7f6634856b80 1 -- 172.21.15.46:0/2900623275 >> 172.21.15.46:0/2900623275 conn(0x55b4391f01c0 msgr2=0x55b4392834e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.787+0000 7f6634856b80 1 -- 172.21.15.46:0/2900623275 shutdown_connections 2024-03-18T20:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.787+0000 7f6634856b80 1 -- 172.21.15.46:0/2900623275 wait complete. 2024-03-18T20:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.787+0000 7f6634856b80 1 -- start start 2024-03-18T20:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.787+0000 7f6634856b80 1 Processor -- start 2024-03-18T20:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.788+0000 7f6634856b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b43927f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.788+0000 7f6634856b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4392827d0 0x55b4392871a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.788+0000 7f6634856b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 0x55b4392876e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.788+0000 7f6634856b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b439287c20 con 0x55b439284db0 2024-03-18T20:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.788+0000 7f6634856b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b439287d60 con 0x55b4392827d0 2024-03-18T20:59:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.788+0000 7f6634856b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b439287f20 con 0x55b439279e90 2024-03-18T20:59:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.788+0000 7f6634856b80 10 client.0 _ll_register_callbacks cb 0x7f661c009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:59:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.789+0000 7f6626c5e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 0x55b4392876e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.789+0000 7f6626c5e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 0x55b4392876e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47520/0 (socket says 172.21.15.46:47520) 2024-03-18T20:59:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.789+0000 7f6626c5e700 1 -- 172.21.15.46:0/1587324426 learned_addr learned my addr 172.21.15.46:0/1587324426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.790+0000 7f6626c5e700 1 -- 172.21.15.46:0/1587324426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 msgr2=0x55b43927f830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:59:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.790+0000 7f6626c5e700 1 --2- 172.21.15.46:0/1587324426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b43927f830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.790+0000 7f6626c5e700 1 -- 172.21.15.46:0/1587324426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4392827d0 msgr2=0x55b4392871a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:59:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.790+0000 7f6626c5e700 1 --2- 172.21.15.46:0/1587324426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4392827d0 0x55b4392871a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.790+0000 7f6626c5e700 1 -- 172.21.15.46:0/1587324426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b43904e350 con 0x55b439284db0 2024-03-18T20:59:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.790+0000 7f6626c5e700 1 --2- 172.21.15.46:0/1587324426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 0x55b4392876e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f6618018dd0 tx=0x7f6618004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939414c8ef734006 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.798+0000 7f6609ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T20:59:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.798+0000 7f6609ffb700 1 -- 172.21.15.46:0/1587324426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661800e040 con 0x55b439284db0 2024-03-18T20:59:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.798+0000 7f6609ffb700 1 -- 172.21.15.46:0/1587324426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f6618009400 con 0x55b439284db0 2024-03-18T20:59:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.798+0000 7f6634856b80 1 -- 172.21.15.46:0/1587324426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f661409f7b0 con 0x55b439284db0 2024-03-18T20:59:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.801+0000 7f6609ffb700 1 -- 172.21.15.46:0/1587324426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7f661800fe70 con 0x55b439284db0 2024-03-18T20:59:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.801+0000 7f6634856b80 10 client.20757 fetch_fsmap learned FSMap version 97 2024-03-18T20:59:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.801+0000 7f6634856b80 1 -- 172.21.15.46:0/1587324426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=4+}) v3 -- 0x55b439192e40 con 0x55b439284db0 2024-03-18T20:59:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.808+0000 7f6609ffb700 1 -- 172.21.15.46:0/1587324426 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f661409f7b0 con 0x55b439284db0 2024-03-18T20:59:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.808+0000 7f6609ffb700 1 -- 172.21.15.46:0/1587324426 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 97) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f661802a030 con 0x55b439284db0 2024-03-18T20:59:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.808+0000 7f6634856b80 10 client.20757 fetch_fsmap finished waiting for FSMap version 97 2024-03-18T20:59:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.808+0000 7f6634856b80 10 client.20757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:59:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.808+0000 7f6634856b80 20 client.20757 populate_metadata read hostname 'smithi046' 2024-03-18T20:59:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.808+0000 7f6634856b80 1 --2- 172.21.15.46:0/1587324426 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b4392df350 0x55b4392e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.808+0000 7f6634856b80 4 client.20757 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:59:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.808+0000 7f6634856b80 1 -- 172.21.15.46:0/1587324426 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b43901bc90 con 0x55b4392df350 2024-03-18T20:59:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.818+0000 7f662645d700 1 --2- 172.21.15.46:0/1587324426 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b4392df350 0x55b4392e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.849+0000 7f662645d700 1 --2- 172.21.15.46:0/1587324426 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b4392df350 0x55b4392e1770 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ed85fece3b9566 server_cookie=67ddbe2afe6aac37 in_seq=0 out_seq=0 2024-03-18T20:59:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:29.849+0000 7f6609ffb700 10 client.20757 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:30.492 INFO:tasks.workunit.client.0.smithi046.stdout: 10 391565 31.30 MB/sec execute 486 sec latency 828.433 ms 2024-03-18T20:59:30.802 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:30 smithi046 ceph-mon[43495]: pgmap v740: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 607 op/s 2024-03-18T20:59:30.802 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:30 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:59:30.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:30 smithi073 ceph-mon[47384]: pgmap v740: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 607 op/s 2024-03-18T20:59:30.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:30 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:59:31.049 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:30 smithi155 ceph-mon[48344]: pgmap v740: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 607 op/s 2024-03-18T20:59:31.049 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:30 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:59:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.427+0000 7f6609ffb700 1 -- 172.21.15.46:0/1587324426 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b43901bc90 con 0x55b4392df350 2024-03-18T20:59:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.427+0000 7f6609ffb700 10 client.20757 handle_command_reply: tid=0 2024-03-18T20:59:31.429 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.427+0000 7f6634856b80 1 client.20757 shutdown 2024-03-18T20:59:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.430+0000 7f6634856b80 1 -- 172.21.15.46:0/1587324426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 msgr2=0x55b4392876e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.430+0000 7f6634856b80 1 --2- 172.21.15.46:0/1587324426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 0x55b4392876e0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f6618018dd0 tx=0x7f6618004ad0 comp rx=0 tx=0).stop 2024-03-18T20:59:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.433+0000 7f6634856b80 1 -- 172.21.15.46:0/1587324426 shutdown_connections 2024-03-18T20:59:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.433+0000 7f6634856b80 1 --2- 172.21.15.46:0/1587324426 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55b4392df350 0x55b4392e1770 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.433+0000 7f6634856b80 1 --2- 172.21.15.46:0/1587324426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b439284db0 0x55b4392876e0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.433+0000 7f6634856b80 1 --2- 172.21.15.46:0/1587324426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b439279e90 0x55b43927f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.433+0000 7f6634856b80 1 --2- 172.21.15.46:0/1587324426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4392827d0 0x55b4392871a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.433+0000 7f6634856b80 1 -- 172.21.15.46:0/1587324426 >> 172.21.15.46:0/1587324426 conn(0x55b4391f01c0 msgr2=0x55b439275450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.433+0000 7f6609ffb700 0 client.20757 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.435+0000 7f6634856b80 1 -- 172.21.15.46:0/1587324426 shutdown_connections 2024-03-18T20:59:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.435+0000 7f6634856b80 1 -- 172.21.15.46:0/1587324426 wait complete. 2024-03-18T20:59:31.438 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.436+0000 7f6634856b80 20 client.20757 trim_cache size 0 max 16384 2024-03-18T20:59:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:59:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:59:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:59:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:59:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.442+0000 7f66286c1700 1 -- 172.21.15.46:0/3070428443 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f660c0a13c0 msgr2=0x7f660c0a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.442+0000 7f66286c1700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f660c0a13c0 0x7f660c0a3880 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f661800f930 tx=0x7f661800fcf0 comp rx=0 tx=0).stop 2024-03-18T20:59:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.442+0000 7f66286c1700 1 -- 172.21.15.46:0/3070428443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66200ffe20 msgr2=0x7f6620115060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:31.445 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.442+0000 7f66286c1700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66200ffe20 0x7f6620115060 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6614005e90 tx=0x7f661400fa00 comp rx=0 tx=0).stop 2024-03-18T20:59:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.449+0000 7f66286c1700 1 -- 172.21.15.46:0/3070428443 shutdown_connections 2024-03-18T20:59:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.449+0000 7f66286c1700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f660c0a13c0 0x7f660c0a3880 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.449+0000 7f66286c1700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f662010f260 0x7f662010f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.449+0000 7f66286c1700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66200ffe20 0x7f6620115060 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.449+0000 7f66286c1700 1 --2- 172.21.15.46:0/3070428443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6620104f40 0x7f66201155a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.449+0000 7f66286c1700 1 -- 172.21.15.46:0/3070428443 >> 172.21.15.46:0/3070428443 conn(0x7f66200fb680 msgr2=0x7f66200fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.449+0000 7f66286c1700 1 -- 172.21.15.46:0/3070428443 shutdown_connections 2024-03-18T20:59:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:31.449+0000 7f66286c1700 1 -- 172.21.15.46:0/3070428443 wait complete. 2024-03-18T20:59:31.492 INFO:tasks.workunit.client.0.smithi046.stdout: 10 391638 31.24 MB/sec execute 487 sec latency 1205.887 ms 2024-03-18T20:59:32.495 INFO:tasks.workunit.client.0.smithi046.stdout: 10 391891 31.21 MB/sec execute 488 sec latency 209.731 ms 2024-03-18T20:59:32.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:32 smithi073 ceph-mon[47384]: pgmap v741: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 513 op/s 2024-03-18T20:59:33.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:32 smithi155 ceph-mon[48344]: pgmap v741: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 513 op/s 2024-03-18T20:59:33.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:32 smithi046 ceph-mon[43495]: pgmap v741: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 513 op/s 2024-03-18T20:59:33.495 INFO:tasks.workunit.client.0.smithi046.stdout: 10 392262 31.19 MB/sec execute 489 sec latency 179.155 ms 2024-03-18T20:59:34.494 INFO:tasks.workunit.client.0.smithi046.stdout: 10 392904 31.20 MB/sec execute 490 sec latency 82.461 ms 2024-03-18T20:59:34.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:34 smithi073 ceph-mon[47384]: pgmap v742: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 12 MiB/s wr, 504 op/s 2024-03-18T20:59:35.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:34 smithi155 ceph-mon[48344]: pgmap v742: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 12 MiB/s wr, 504 op/s 2024-03-18T20:59:35.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:34 smithi046 ceph-mon[43495]: pgmap v742: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 12 MiB/s wr, 504 op/s 2024-03-18T20:59:35.495 INFO:tasks.workunit.client.0.smithi046.stdout: 10 393449 31.20 MB/sec execute 491 sec latency 98.172 ms 2024-03-18T20:59:36.495 INFO:tasks.workunit.client.0.smithi046.stdout: 10 393962 31.18 MB/sec execute 492 sec latency 114.000 ms 2024-03-18T20:59:36.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:36 smithi073 ceph-mon[47384]: pgmap v743: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 20 MiB/s wr, 781 op/s 2024-03-18T20:59:36.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:36 smithi155 ceph-mon[48344]: pgmap v743: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 20 MiB/s wr, 781 op/s 2024-03-18T20:59:36.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:36 smithi046 ceph-mon[43495]: pgmap v743: 97 pgs: 97 active+clean; 2.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 20 MiB/s wr, 781 op/s 2024-03-18T20:59:37.495 INFO:tasks.workunit.client.0.smithi046.stdout: 10 394506 31.19 MB/sec execute 493 sec latency 107.875 ms 2024-03-18T20:59:38.495 INFO:tasks.workunit.client.0.smithi046.stdout: 10 395007 31.19 MB/sec execute 494 sec latency 104.350 ms 2024-03-18T20:59:38.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:38 smithi155 ceph-mon[48344]: pgmap v744: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 697 op/s 2024-03-18T20:59:38.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:38 smithi046 ceph-mon[43495]: pgmap v744: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 697 op/s 2024-03-18T20:59:38.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:38 smithi073 ceph-mon[47384]: pgmap v744: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 18 MiB/s wr, 697 op/s 2024-03-18T20:59:39.495 INFO:tasks.workunit.client.0.smithi046.stdout: 10 395452 31.15 MB/sec execute 495 sec latency 191.000 ms 2024-03-18T20:59:40.500 INFO:tasks.workunit.client.0.smithi046.stdout: 10 396007 31.15 MB/sec execute 496 sec latency 140.669 ms 2024-03-18T20:59:40.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:40 smithi155 ceph-mon[48344]: pgmap v745: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 20 MiB/s wr, 844 op/s 2024-03-18T20:59:40.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:40 smithi046 ceph-mon[43495]: pgmap v745: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 20 MiB/s wr, 844 op/s 2024-03-18T20:59:40.973 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:40 smithi073 ceph-mon[47384]: pgmap v745: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 20 MiB/s wr, 844 op/s 2024-03-18T20:59:41.496 INFO:tasks.workunit.client.0.smithi046.stdout: 10 396668 31.17 MB/sec execute 497 sec latency 92.484 ms 2024-03-18T20:59:42.496 INFO:tasks.workunit.client.0.smithi046.stdout: 10 397279 31.16 MB/sec execute 498 sec latency 103.080 ms 2024-03-18T20:59:42.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:42 smithi155 ceph-mon[48344]: pgmap v746: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 766 op/s 2024-03-18T20:59:42.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:42 smithi046 ceph-mon[43495]: pgmap v746: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 766 op/s 2024-03-18T20:59:42.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:42 smithi073 ceph-mon[47384]: pgmap v746: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 19 MiB/s wr, 766 op/s 2024-03-18T20:59:43.496 INFO:tasks.workunit.client.0.smithi046.stdout: 10 397833 31.17 MB/sec execute 499 sec latency 100.961 ms 2024-03-18T20:59:44.496 INFO:tasks.workunit.client.0.smithi046.stdout: 10 398253 31.16 MB/sec execute 500 sec latency 153.306 ms 2024-03-18T20:59:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.628+0000 7fc810912700 1 -- 172.21.15.46:0/705906341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c1027a0 msgr2=0x7fc80c10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.628+0000 7fc810912700 1 --2- 172.21.15.46:0/705906341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c1027a0 0x7fc80c10ac80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc8000098f0 tx=0x7fc800009c00 comp rx=0 tx=0).stop 2024-03-18T20:59:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.629+0000 7fc810912700 1 -- 172.21.15.46:0/705906341 shutdown_connections 2024-03-18T20:59:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.629+0000 7fc810912700 1 --2- 172.21.15.46:0/705906341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c101e80 0x7fc80c102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.629+0000 7fc810912700 1 --2- 172.21.15.46:0/705906341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc80c10b2a0 0x7fc80c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.629+0000 7fc810912700 1 --2- 172.21.15.46:0/705906341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c1027a0 0x7fc80c10ac80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.629+0000 7fc810912700 1 -- 172.21.15.46:0/705906341 >> 172.21.15.46:0/705906341 conn(0x7fc80c0fb680 msgr2=0x7fc80c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.630+0000 7fc810912700 1 -- 172.21.15.46:0/705906341 shutdown_connections 2024-03-18T20:59:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.630+0000 7fc810912700 1 -- 172.21.15.46:0/705906341 wait complete. 2024-03-18T20:59:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc810912700 1 Processor -- start 2024-03-18T20:59:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc810912700 1 -- start start 2024-03-18T20:59:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc810912700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc80c101e80 0x7fc80c114fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc810912700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c1027a0 0x7fc80c115540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc810912700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c10b2a0 0x7fc80c110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc810912700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c110930 con 0x7fc80c1027a0 2024-03-18T20:59:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc810912700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c110aa0 con 0x7fc80c101e80 2024-03-18T20:59:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc810912700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c110c60 con 0x7fc80c10b2a0 2024-03-18T20:59:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc80a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc80c101e80 0x7fc80c114fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc80a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc80c101e80 0x7fc80c114fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56594/0 (socket says 172.21.15.46:56594) 2024-03-18T20:59:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc80a59c700 1 -- 172.21.15.46:0/3119854339 learned_addr learned my addr 172.21.15.46:0/3119854339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc809d9b700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c1027a0 0x7fc80c115540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.631+0000 7fc80ad9d700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c10b2a0 0x7fc80c110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.632+0000 7fc80a59c700 1 -- 172.21.15.46:0/3119854339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c10b2a0 msgr2=0x7fc80c110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.632+0000 7fc80a59c700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c10b2a0 0x7fc80c110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.632+0000 7fc80a59c700 1 -- 172.21.15.46:0/3119854339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c1027a0 msgr2=0x7fc80c115540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.632+0000 7fc80a59c700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c1027a0 0x7fc80c115540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.632+0000 7fc80a59c700 1 -- 172.21.15.46:0/3119854339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc800009520 con 0x7fc80c101e80 2024-03-18T20:59:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.632+0000 7fc809d9b700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c1027a0 0x7fc80c115540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:59:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.632+0000 7fc80a59c700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc80c101e80 0x7fc80c114fe0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00e9c0 tx=0x7fc7fc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.633+0000 7fc7fb7fe700 1 -- 172.21.15.46:0/3119854339 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00cbd0 con 0x7fc80c101e80 2024-03-18T20:59:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.633+0000 7fc7fb7fe700 1 -- 172.21.15.46:0/3119854339 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7fc00eeb0 con 0x7fc80c101e80 2024-03-18T20:59:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.633+0000 7fc7fb7fe700 1 -- 172.21.15.46:0/3119854339 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc005580 con 0x7fc80c101e80 2024-03-18T20:59:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.634+0000 7fc810912700 1 -- 172.21.15.46:0/3119854339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80c110f40 con 0x7fc80c101e80 2024-03-18T20:59:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.634+0000 7fc810912700 1 -- 172.21.15.46:0/3119854339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80c11b000 con 0x7fc80c101e80 2024-03-18T20:59:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.639+0000 7fc81caa7b80 1 Processor -- start 2024-03-18T20:59:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.639+0000 7fc7fb7fe700 1 -- 172.21.15.46:0/3119854339 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc7fc004750 con 0x7fc80c101e80 2024-03-18T20:59:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.640+0000 7fc7fb7fe700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7f40a1350 0x7fc7f40a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.641+0000 7fc7fb7fe700 1 -- 172.21.15.46:0/3119854339 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc7fc014070 con 0x7fc80c101e80 2024-03-18T20:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.641+0000 7fc81caa7b80 1 -- start start 2024-03-18T20:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.642+0000 7fc81caa7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9570 0x555d916a9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.642+0000 7fc81caa7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9e90 0x555d916b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.642+0000 7fc81caa7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916b4db0 0x555d916b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.642+0000 7fc81caa7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d91603930 con 0x555d916a9570 2024-03-18T20:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.642+0000 7fc81caa7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d916012f0 con 0x555d916b4db0 2024-03-18T20:59:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.642+0000 7fc81caa7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d915fcbe0 con 0x555d916a9e90 2024-03-18T20:59:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.642+0000 7fc80a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9570 0x555d916a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.642+0000 7fc80a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9570 0x555d916a9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57458/0 (socket says 172.21.15.46:57458) 2024-03-18T20:59:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.642+0000 7fc80a59c700 1 -- 172.21.15.46:0/2260436664 learned_addr learned my addr 172.21.15.46:0/2260436664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.642+0000 7fc80ad9d700 1 --2- 172.21.15.46:0/2260436664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916b4db0 0x555d916b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.642+0000 7fc80ad9d700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7f40a1350 0x7fc7f40a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.643+0000 7fc809d9b700 1 --2- 172.21.15.46:0/2260436664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9e90 0x555d916b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.643+0000 7fc80ad9d700 1 -- 172.21.15.46:0/2260436664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9e90 msgr2=0x555d916b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.643+0000 7fc80ad9d700 1 --2- 172.21.15.46:0/2260436664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9e90 0x555d916b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.643+0000 7fc80ad9d700 1 -- 172.21.15.46:0/2260436664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9570 msgr2=0x555d916a9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.643+0000 7fc80ad9d700 1 --2- 172.21.15.46:0/2260436664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9570 0x555d916a9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.643+0000 7fc80ad9d700 1 -- 172.21.15.46:0/2260436664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d9147e350 con 0x555d916b4db0 2024-03-18T20:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.643+0000 7fc80ad9d700 1 --2- 172.21.15.46:0/2260436664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916b4db0 0x555d916b71a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc80c115c70 tx=0x7fc804018640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=250eb22d3b6d3151 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.643+0000 7fc80ad9d700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7f40a1350 0x7fc7f40a3810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc804018a40 tx=0x7fc80400b000 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.646+0000 7fc7f8ff9700 1 -- 172.21.15.46:0/2260436664 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80400e070 con 0x555d916b4db0 2024-03-18T20:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.647+0000 7fc7f8ff9700 1 -- 172.21.15.46:0/2260436664 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc80400f040 con 0x555d916b4db0 2024-03-18T20:59:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.647+0000 7fc7f8ff9700 1 -- 172.21.15.46:0/2260436664 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804005020 con 0x555d916b4db0 2024-03-18T20:59:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.648+0000 7fc81caa7b80 1 -- 172.21.15.46:0/2260436664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916b4db0 msgr2=0x555d916b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.648+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/2260436664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916b4db0 0x555d916b71a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc80c115c70 tx=0x7fc804018640 comp rx=0 tx=0).stop 2024-03-18T20:59:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.651+0000 7fc81caa7b80 1 -- 172.21.15.46:0/2260436664 shutdown_connections 2024-03-18T20:59:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.651+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/2260436664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9570 0x555d916a9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.651+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/2260436664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916b4db0 0x555d916b71a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.651+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/2260436664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9e90 0x555d916b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.651+0000 7fc81caa7b80 1 -- 172.21.15.46:0/2260436664 >> 172.21.15.46:0/2260436664 conn(0x555d916201c0 msgr2=0x555d916a5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:44.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.658+0000 7fc81caa7b80 1 -- 172.21.15.46:0/2260436664 shutdown_connections 2024-03-18T20:59:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.658+0000 7fc81caa7b80 1 -- 172.21.15.46:0/2260436664 wait complete. 2024-03-18T20:59:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.658+0000 7fc81caa7b80 1 -- start start 2024-03-18T20:59:44.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.658+0000 7fc81caa7b80 1 Processor -- start 2024-03-18T20:59:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.659+0000 7fc81caa7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9570 0x555d916b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.659+0000 7fc81caa7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9e90 0x555d916b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.659+0000 7fc81caa7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916b4db0 0x555d916b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.659+0000 7fc81caa7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d91598d20 con 0x555d916b4db0 2024-03-18T20:59:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.659+0000 7fc81caa7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d915989c0 con 0x555d916a9570 2024-03-18T20:59:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.659+0000 7fc81caa7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d91480640 con 0x555d916a9e90 2024-03-18T20:59:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.659+0000 7fc81caa7b80 10 client.0 _ll_register_callbacks cb 0x555d91404340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:59:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.659+0000 7fc80a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9570 0x555d916b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.660+0000 7fc80a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9570 0x555d916b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56606/0 (socket says 172.21.15.46:56606) 2024-03-18T20:59:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.660+0000 7fc80a59c700 1 -- 172.21.15.46:0/616447356 learned_addr learned my addr 172.21.15.46:0/616447356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.660+0000 7fc809d9b700 1 --2- 172.21.15.46:0/616447356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9e90 0x555d916b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.660+0000 7fc80a59c700 1 -- 172.21.15.46:0/616447356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9e90 msgr2=0x555d916b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.660+0000 7fc80a59c700 1 --2- 172.21.15.46:0/616447356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9e90 0x555d916b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.660+0000 7fc80a59c700 1 -- 172.21.15.46:0/616447356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916b4db0 msgr2=0x555d916b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.660+0000 7fc80a59c700 1 --2- 172.21.15.46:0/616447356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916b4db0 0x555d916b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.660+0000 7fc80a59c700 1 -- 172.21.15.46:0/616447356 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc804007970 con 0x555d916a9570 2024-03-18T20:59:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.660+0000 7fc80a59c700 1 --2- 172.21.15.46:0/616447356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9570 0x555d916b1b00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc096520 tx=0x7fc7fc00cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=326a0cded73e8c79 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.662+0000 7fc7f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T20:59:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.662+0000 7fc7f1ffb700 1 -- 172.21.15.46:0/616447356 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc01f550 con 0x555d916a9570 2024-03-18T20:59:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.662+0000 7fc7f1ffb700 1 -- 172.21.15.46:0/616447356 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc7fc09de50 con 0x555d916a9570 2024-03-18T20:59:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.662+0000 7fc81caa7b80 1 -- 172.21.15.46:0/616447356 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d9169d6c0 con 0x555d916a9570 2024-03-18T20:59:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.665+0000 7fc7f1ffb700 1 -- 172.21.15.46:0/616447356 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7fc0a24a0 con 0x555d916a9570 2024-03-18T20:59:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.666+0000 7fc81caa7b80 10 client.30386 fetch_fsmap learned FSMap version 97 2024-03-18T20:59:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.666+0000 7fc81caa7b80 1 -- 172.21.15.46:0/616447356 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=4+}) v3 -- 0x555d9142ab00 con 0x555d916a9570 2024-03-18T20:59:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.668+0000 7fc7f1ffb700 1 -- 172.21.15.46:0/616447356 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d9169d6c0 con 0x555d916a9570 2024-03-18T20:59:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.668+0000 7fc7f1ffb700 1 -- 172.21.15.46:0/616447356 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 97) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fc7fc099510 con 0x555d916a9570 2024-03-18T20:59:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.668+0000 7fc81caa7b80 10 client.30386 fetch_fsmap finished waiting for FSMap version 97 2024-03-18T20:59:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.668+0000 7fc81caa7b80 10 client.30386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:59:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.668+0000 7fc81caa7b80 20 client.30386 populate_metadata read hostname 'smithi046' 2024-03-18T20:59:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.668+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/616447356 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d916d5a00 0x555d916d7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.668+0000 7fc81caa7b80 4 client.30386 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:59:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.668+0000 7fc81caa7b80 1 -- 172.21.15.46:0/616447356 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d915f56a0 con 0x555d916d5a00 2024-03-18T20:59:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.670+0000 7fc809d9b700 1 --2- 172.21.15.46:0/616447356 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d916d5a00 0x555d916d7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.671+0000 7fc809d9b700 1 --2- 172.21.15.46:0/616447356 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d916d5a00 0x555d916d7e20 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124b4f9ed383b561 server_cookie=97bea0776939f03d in_seq=0 out_seq=0 2024-03-18T20:59:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.671+0000 7fc7f1ffb700 10 client.30386 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.673+0000 7fc7f1ffb700 1 -- 172.21.15.46:0/616447356 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555d915f56a0 con 0x555d916d5a00 2024-03-18T20:59:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.673+0000 7fc7f1ffb700 10 client.30386 handle_command_reply: tid=0 2024-03-18T20:59:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.673+0000 7fc81caa7b80 1 client.30386 shutdown 2024-03-18T20:59:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.680+0000 7fc81caa7b80 1 -- 172.21.15.46:0/616447356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9570 msgr2=0x555d916b1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.680+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/616447356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9570 0x555d916b1b00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc096520 tx=0x7fc7fc00cd30 comp rx=0 tx=0).stop 2024-03-18T20:59:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.687+0000 7fc81caa7b80 1 -- 172.21.15.46:0/616447356 shutdown_connections 2024-03-18T20:59:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.687+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/616447356 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d916d5a00 0x555d916d7e20 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.687+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/616447356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916b4db0 0x555d916b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.687+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/616447356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9570 0x555d916b1b00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.687+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/616447356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9e90 0x555d916b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.687+0000 7fc81caa7b80 1 -- 172.21.15.46:0/616447356 >> 172.21.15.46:0/616447356 conn(0x555d916201c0 msgr2=0x555d916a5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.690+0000 7fc7f1ffb700 0 client.30386 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.690+0000 7fc81caa7b80 1 -- 172.21.15.46:0/616447356 shutdown_connections 2024-03-18T20:59:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.691+0000 7fc81caa7b80 1 -- 172.21.15.46:0/616447356 wait complete. 2024-03-18T20:59:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.694+0000 7fc81caa7b80 20 client.30386 trim_cache size 0 max 16384 2024-03-18T20:59:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.723+0000 7fc81caa7b80 1 Processor -- start 2024-03-18T20:59:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.726+0000 7fc81caa7b80 1 -- start start 2024-03-18T20:59:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.726+0000 7fc81caa7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9570 0x555d916bb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.726+0000 7fc81caa7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9e90 0x555d916b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.726+0000 7fc81caa7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916b4db0 0x555d916ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.726+0000 7fc81caa7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d916ac950 con 0x555d916a9570 2024-03-18T20:59:44.729 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.726+0000 7fc81caa7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d916a6f00 con 0x555d916a9e90 2024-03-18T20:59:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.726+0000 7fc81caa7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d9169fc10 con 0x555d916b4db0 2024-03-18T20:59:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.729+0000 7fc809d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9e90 0x555d916b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.729+0000 7fc809d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9e90 0x555d916b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56614/0 (socket says 172.21.15.46:56614) 2024-03-18T20:59:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.729+0000 7fc809d9b700 1 -- 172.21.15.46:0/3249195194 learned_addr learned my addr 172.21.15.46:0/3249195194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.730+0000 7fc809d9b700 1 -- 172.21.15.46:0/3249195194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916b4db0 msgr2=0x555d916ace10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T20:59:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.730+0000 7fc809d9b700 1 --2- 172.21.15.46:0/3249195194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916b4db0 0x555d916ace10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.730+0000 7fc809d9b700 1 -- 172.21.15.46:0/3249195194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9570 msgr2=0x555d916bb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.730+0000 7fc809d9b700 1 --2- 172.21.15.46:0/3249195194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9570 0x555d916bb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.733 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.730+0000 7fc809d9b700 1 -- 172.21.15.46:0/3249195194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d915c2e40 con 0x555d916a9e90 2024-03-18T20:59:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.731+0000 7fc809d9b700 1 --2- 172.21.15.46:0/3249195194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9e90 0x555d916b23a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc8000098f0 tx=0x7fc800009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=627bd301d5060a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.731+0000 7fc7f97fa700 1 -- 172.21.15.46:0/3249195194 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80001c040 con 0x555d916a9e90 2024-03-18T20:59:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.731+0000 7fc7f97fa700 1 -- 172.21.15.46:0/3249195194 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc80001d040 con 0x555d916a9e90 2024-03-18T20:59:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.731+0000 7fc7f97fa700 1 -- 172.21.15.46:0/3249195194 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800025440 con 0x555d916a9e90 2024-03-18T20:59:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.736+0000 7fc81caa7b80 1 -- 172.21.15.46:0/3249195194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9e90 msgr2=0x555d916b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.736+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/3249195194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9e90 0x555d916b23a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc8000098f0 tx=0x7fc800009f70 comp rx=0 tx=0).stop 2024-03-18T20:59:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.743+0000 7fc81caa7b80 1 -- 172.21.15.46:0/3249195194 shutdown_connections 2024-03-18T20:59:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.743+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/3249195194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9570 0x555d916bb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.743+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/3249195194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916b4db0 0x555d916ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.743+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/3249195194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916a9e90 0x555d916b23a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.743+0000 7fc81caa7b80 1 -- 172.21.15.46:0/3249195194 >> 172.21.15.46:0/3249195194 conn(0x555d916201c0 msgr2=0x555d916c0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.748+0000 7fc81caa7b80 1 -- 172.21.15.46:0/3249195194 shutdown_connections 2024-03-18T20:59:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.748+0000 7fc81caa7b80 1 -- 172.21.15.46:0/3249195194 wait complete. 2024-03-18T20:59:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.749+0000 7fc81caa7b80 1 -- start start 2024-03-18T20:59:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.749+0000 7fc81caa7b80 1 Processor -- start 2024-03-18T20:59:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.749+0000 7fc81caa7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9570 0x555d916b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.749+0000 7fc81caa7b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9e90 0x555d916ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.749+0000 7fc81caa7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916b4db0 0x555d91728830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.749+0000 7fc81caa7b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d916ba7a0 con 0x555d916a9e90 2024-03-18T20:59:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.749+0000 7fc81caa7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d91728d70 con 0x555d916b4db0 2024-03-18T20:59:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.749+0000 7fc81caa7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d91728eb0 con 0x555d916a9570 2024-03-18T20:59:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.749+0000 7fc81caa7b80 10 client.0 _ll_register_callbacks cb 0x7fc7fc099ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:59:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.751+0000 7fc80a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9570 0x555d916b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.751+0000 7fc809d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9e90 0x555d916ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.751+0000 7fc80a59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9570 0x555d916b9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:46836/0 (socket says 172.21.15.46:46836) 2024-03-18T20:59:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.751+0000 7fc809d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9e90 0x555d916ba0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57488/0 (socket says 172.21.15.46:57488) 2024-03-18T20:59:44.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.751+0000 7fc80a59c700 1 -- 172.21.15.46:0/365303140 learned_addr learned my addr 172.21.15.46:0/365303140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.753+0000 7fc80a59c700 1 -- 172.21.15.46:0/365303140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916b4db0 msgr2=0x555d91728830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.753+0000 7fc80a59c700 1 --2- 172.21.15.46:0/365303140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916b4db0 0x555d91728830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.753+0000 7fc80a59c700 1 -- 172.21.15.46:0/365303140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9e90 msgr2=0x555d916ba0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.753+0000 7fc80a59c700 1 --2- 172.21.15.46:0/365303140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9e90 0x555d916ba0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.753+0000 7fc80a59c700 1 -- 172.21.15.46:0/365303140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80c10e420 con 0x555d916a9570 2024-03-18T20:59:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.753+0000 7fc80a59c700 1 --2- 172.21.15.46:0/365303140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9570 0x555d916b9b80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0ca710 tx=0x7fc7fc09a0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6461c8bf0e29c37 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.753+0000 7fc7f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:59:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.753+0000 7fc7f1ffb700 1 -- 172.21.15.46:0/365303140 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc01f550 con 0x555d916a9570 2024-03-18T20:59:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.753+0000 7fc7f1ffb700 1 -- 172.21.15.46:0/365303140 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc7fc09de50 con 0x555d916a9570 2024-03-18T20:59:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.755+0000 7fc81caa7b80 1 -- 172.21.15.46:0/365303140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc80400f040 con 0x555d916a9570 2024-03-18T20:59:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.755+0000 7fc7f1ffb700 1 -- 172.21.15.46:0/365303140 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7fc099720 con 0x555d916a9570 2024-03-18T20:59:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.755+0000 7fc81caa7b80 10 client.30517 fetch_fsmap learned FSMap version 97 2024-03-18T20:59:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.755+0000 7fc81caa7b80 1 -- 172.21.15.46:0/365303140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=4+}) v3 -- 0x555d91482c70 con 0x555d916a9570 2024-03-18T20:59:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.756+0000 7fc7f1ffb700 1 -- 172.21.15.46:0/365303140 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc80400f040 con 0x555d916a9570 2024-03-18T20:59:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.756+0000 7fc7f1ffb700 1 -- 172.21.15.46:0/365303140 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 97) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fc7fc0a2080 con 0x555d916a9570 2024-03-18T20:59:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.756+0000 7fc81caa7b80 10 client.30517 fetch_fsmap finished waiting for FSMap version 97 2024-03-18T20:59:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.756+0000 7fc81caa7b80 10 client.30517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:59:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.756+0000 7fc81caa7b80 20 client.30517 populate_metadata read hostname 'smithi046' 2024-03-18T20:59:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.756+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/365303140 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d9172a230 0x555d9172c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:44.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.756+0000 7fc81caa7b80 4 client.30517 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:59:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.756+0000 7fc81caa7b80 1 -- 172.21.15.46:0/365303140 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555d9144bc90 con 0x555d9172a230 2024-03-18T20:59:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.758+0000 7fc809d9b700 1 --2- 172.21.15.46:0/365303140 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d9172a230 0x555d9172c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.759+0000 7fc809d9b700 1 --2- 172.21.15.46:0/365303140 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d9172a230 0x555d9172c620 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d327fa717eb9e264 server_cookie=c7e8a137d3c15e16 in_seq=0 out_seq=0 2024-03-18T20:59:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:44.759+0000 7fc7f1ffb700 10 client.30517 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:44 smithi155 ceph-mon[48344]: pgmap v747: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 674 op/s 2024-03-18T20:59:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:44 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:59:44.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:44 smithi046 ceph-mon[43495]: pgmap v747: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 674 op/s 2024-03-18T20:59:44.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:44 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:59:45.054 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:44 smithi073 ceph-mon[47384]: pgmap v747: 97 pgs: 97 active+clean; 2.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 674 op/s 2024-03-18T20:59:45.054 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:44 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T20:59:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.091+0000 7fc7f1ffb700 1 -- 172.21.15.46:0/365303140 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555d9144bc90 con 0x555d9172a230 2024-03-18T20:59:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.092+0000 7fc7f1ffb700 10 client.30517 handle_command_reply: tid=0 2024-03-18T20:59:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.092+0000 7fc81caa7b80 1 client.30517 shutdown 2024-03-18T20:59:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.097+0000 7fc81caa7b80 1 -- 172.21.15.46:0/365303140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9570 msgr2=0x555d916b9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.097+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/365303140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9570 0x555d916b9b80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0ca710 tx=0x7fc7fc09a0e0 comp rx=0 tx=0).stop 2024-03-18T20:59:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.097+0000 7fc81caa7b80 1 -- 172.21.15.46:0/365303140 shutdown_connections 2024-03-18T20:59:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.097+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/365303140 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555d9172a230 0x555d9172c620 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.097+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/365303140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555d916b4db0 0x555d91728830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.097+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/365303140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d916a9570 0x555d916b9b80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.097+0000 7fc81caa7b80 1 --2- 172.21.15.46:0/365303140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d916a9e90 0x555d916ba0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.097+0000 7fc81caa7b80 1 -- 172.21.15.46:0/365303140 >> 172.21.15.46:0/365303140 conn(0x555d916201c0 msgr2=0x555d916ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.097+0000 7fc7f1ffb700 0 client.30517 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.103+0000 7fc81caa7b80 1 -- 172.21.15.46:0/365303140 shutdown_connections 2024-03-18T20:59:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.103+0000 7fc81caa7b80 1 -- 172.21.15.46:0/365303140 wait complete. 2024-03-18T20:59:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.103+0000 7fc81caa7b80 20 client.30517 trim_cache size 0 max 16384 2024-03-18T20:59:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:59:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:59:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:59:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:59:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.110+0000 7fc810912700 1 -- 172.21.15.46:0/3119854339 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7f40a1350 msgr2=0x7fc7f40a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.110+0000 7fc810912700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7f40a1350 0x7fc7f40a3810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc804018a40 tx=0x7fc80400b000 comp rx=0 tx=0).stop 2024-03-18T20:59:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.110+0000 7fc810912700 1 -- 172.21.15.46:0/3119854339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc80c101e80 msgr2=0x7fc80c114fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.110+0000 7fc810912700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc80c101e80 0x7fc80c114fe0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00e9c0 tx=0x7fc7fc00ecd0 comp rx=0 tx=0).stop 2024-03-18T20:59:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.110+0000 7fc810912700 1 -- 172.21.15.46:0/3119854339 shutdown_connections 2024-03-18T20:59:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.110+0000 7fc810912700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc7f40a1350 0x7fc7f40a3810 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.110+0000 7fc810912700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc80c10b2a0 0x7fc80c110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.110+0000 7fc810912700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc80c101e80 0x7fc80c114fe0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.110+0000 7fc810912700 1 --2- 172.21.15.46:0/3119854339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80c1027a0 0x7fc80c115540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:45.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.110+0000 7fc810912700 1 -- 172.21.15.46:0/3119854339 >> 172.21.15.46:0/3119854339 conn(0x7fc80c0fb680 msgr2=0x7fc80c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.115+0000 7fc810912700 1 -- 172.21.15.46:0/3119854339 shutdown_connections 2024-03-18T20:59:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:45.115+0000 7fc810912700 1 -- 172.21.15.46:0/3119854339 wait complete. 2024-03-18T20:59:45.496 INFO:tasks.workunit.client.0.smithi046.stdout: 10 398504 31.12 MB/sec execute 501 sec latency 235.938 ms 2024-03-18T20:59:45.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:45 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:59:45.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:45 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:59:45.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:45 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:59:45.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:45 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:59:45.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:45 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:59:45.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:45 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:59:45.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:45 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T20:59:45.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:45 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T20:59:45.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:45 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T20:59:46.497 INFO:tasks.workunit.client.0.smithi046.stdout: 10 399025 31.11 MB/sec execute 502 sec latency 101.021 ms 2024-03-18T20:59:46.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:46 smithi155 ceph-mon[48344]: pgmap v748: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 17 MiB/s wr, 779 op/s 2024-03-18T20:59:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:46 smithi046 ceph-mon[43495]: pgmap v748: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 17 MiB/s wr, 779 op/s 2024-03-18T20:59:47.088 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:46 smithi073 ceph-mon[47384]: pgmap v748: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 17 MiB/s wr, 779 op/s 2024-03-18T20:59:47.496 INFO:tasks.workunit.client.0.smithi046.stdout: 10 399623 31.11 MB/sec execute 503 sec latency 115.317 ms 2024-03-18T20:59:48.497 INFO:tasks.workunit.client.0.smithi046.stdout: 10 400276 31.12 MB/sec execute 504 sec latency 282.597 ms 2024-03-18T20:59:48.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:48 smithi046 ceph-mon[43495]: pgmap v749: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.6 MiB/s wr, 529 op/s 2024-03-18T20:59:48.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:48 smithi073 ceph-mon[47384]: pgmap v749: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.6 MiB/s wr, 529 op/s 2024-03-18T20:59:49.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:48 smithi155 ceph-mon[48344]: pgmap v749: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 8.6 MiB/s wr, 529 op/s 2024-03-18T20:59:49.496 INFO:tasks.workunit.client.0.smithi046.stdout: 10 400840 31.12 MB/sec execute 505 sec latency 82.467 ms 2024-03-18T20:59:50.497 INFO:tasks.workunit.client.0.smithi046.stdout: 10 401405 31.12 MB/sec execute 506 sec latency 172.805 ms 2024-03-18T20:59:50.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:50 smithi073 ceph-mon[47384]: pgmap v750: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.5 MiB/s wr, 557 op/s 2024-03-18T20:59:51.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:50 smithi155 ceph-mon[48344]: pgmap v750: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.5 MiB/s wr, 557 op/s 2024-03-18T20:59:51.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:50 smithi046 ceph-mon[43495]: pgmap v750: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.5 MiB/s wr, 557 op/s 2024-03-18T20:59:51.497 INFO:tasks.workunit.client.0.smithi046.stdout: 10 402050 31.13 MB/sec execute 507 sec latency 84.786 ms 2024-03-18T20:59:52.497 INFO:tasks.workunit.client.0.smithi046.stdout: 10 402605 31.13 MB/sec execute 508 sec latency 78.922 ms 2024-03-18T20:59:52.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:52 smithi073 ceph-mon[47384]: pgmap v751: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.3 MiB/s wr, 486 op/s 2024-03-18T20:59:53.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:52 smithi155 ceph-mon[48344]: pgmap v751: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.3 MiB/s wr, 486 op/s 2024-03-18T20:59:53.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:52 smithi046 ceph-mon[43495]: pgmap v751: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 8.3 MiB/s wr, 486 op/s 2024-03-18T20:59:53.497 INFO:tasks.workunit.client.0.smithi046.stdout: 10 403177 31.13 MB/sec execute 509 sec latency 133.509 ms 2024-03-18T20:59:54.497 INFO:tasks.workunit.client.0.smithi046.stdout: 10 403693 31.13 MB/sec execute 510 sec latency 109.988 ms 2024-03-18T20:59:54.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:54 smithi073 ceph-mon[47384]: pgmap v752: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.5 MiB/s wr, 451 op/s 2024-03-18T20:59:55.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:54 smithi155 ceph-mon[48344]: pgmap v752: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.5 MiB/s wr, 451 op/s 2024-03-18T20:59:55.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:54 smithi046 ceph-mon[43495]: pgmap v752: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.5 MiB/s wr, 451 op/s 2024-03-18T20:59:55.497 INFO:tasks.workunit.client.0.smithi046.stdout: 10 404107 31.10 MB/sec execute 511 sec latency 98.746 ms 2024-03-18T20:59:55.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:55 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:59:56.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:55 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:59:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:55 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T20:59:56.497 INFO:tasks.workunit.client.0.smithi046.stdout: 10 404717 31.08 MB/sec execute 512 sec latency 95.311 ms 2024-03-18T20:59:56.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:56 smithi073 ceph-mon[47384]: pgmap v753: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.4 MiB/s wr, 564 op/s 2024-03-18T20:59:57.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:56 smithi155 ceph-mon[48344]: pgmap v753: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.4 MiB/s wr, 564 op/s 2024-03-18T20:59:57.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:56 smithi046 ceph-mon[43495]: pgmap v753: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.4 MiB/s wr, 564 op/s 2024-03-18T20:59:57.497 INFO:tasks.workunit.client.0.smithi046.stdout: 10 405385 31.11 MB/sec execute 513 sec latency 95.689 ms 2024-03-18T20:59:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.295+0000 7fdaacbce700 1 -- 172.21.15.46:0/2011392839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa8104f20 msgr2=0x7fdaa810caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.295+0000 7fdaacbce700 1 --2- 172.21.15.46:0/2011392839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa8104f20 0x7fdaa810caa0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000b0a0 tx=0x7fdaa000b3b0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.295+0000 7fdaacbce700 1 -- 172.21.15.46:0/2011392839 shutdown_connections 2024-03-18T20:59:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.295+0000 7fdaacbce700 1 --2- 172.21.15.46:0/2011392839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa8104f20 0x7fdaa810caa0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.295+0000 7fdaacbce700 1 --2- 172.21.15.46:0/2011392839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa80ffe00 0x7fdaa81001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.295+0000 7fdaacbce700 1 --2- 172.21.15.46:0/2011392839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaa81007b0 0x7fdaa81047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.295+0000 7fdaacbce700 1 -- 172.21.15.46:0/2011392839 >> 172.21.15.46:0/2011392839 conn(0x7fdaa80fb680 msgr2=0x7fdaa80fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.295+0000 7fdaacbce700 1 -- 172.21.15.46:0/2011392839 shutdown_connections 2024-03-18T20:59:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.295+0000 7fdaacbce700 1 -- 172.21.15.46:0/2011392839 wait complete. 2024-03-18T20:59:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.296+0000 7fdaacbce700 1 Processor -- start 2024-03-18T20:59:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.296+0000 7fdaacbce700 1 -- start start 2024-03-18T20:59:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.296+0000 7fdaacbce700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa80ffe00 0x7fdaa8199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.296+0000 7fdaacbce700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaa8104f20 0x7fdaa8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.296+0000 7fdaacbce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa819dd90 0x7fdaa819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.296+0000 7fdaacbce700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa819e810 con 0x7fdaa819dd90 2024-03-18T20:59:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.296+0000 7fdaacbce700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa819e980 con 0x7fdaa80ffe00 2024-03-18T20:59:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.296+0000 7fdaacbce700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa819eb40 con 0x7fdaa8104f20 2024-03-18T20:59:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.296+0000 7fdaa659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa80ffe00 0x7fdaa8199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.296+0000 7fdaa6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa819dd90 0x7fdaa819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fdaa659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa80ffe00 0x7fdaa8199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37752/0 (socket says 172.21.15.46:37752) 2024-03-18T20:59:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fdaa6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa819dd90 0x7fdaa819e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33352/0 (socket says 172.21.15.46:33352) 2024-03-18T20:59:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fdaa659c700 1 -- 172.21.15.46:0/1092416468 learned_addr learned my addr 172.21.15.46:0/1092416468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fdaa5d9b700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaa8104f20 0x7fdaa8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fdaa659c700 1 -- 172.21.15.46:0/1092416468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaa8104f20 msgr2=0x7fdaa8199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fdaa659c700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaa8104f20 0x7fdaa8199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fdaa659c700 1 -- 172.21.15.46:0/1092416468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa819dd90 msgr2=0x7fdaa819e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fdaa659c700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa819dd90 0x7fdaa819e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fdaa659c700 1 -- 172.21.15.46:0/1092416468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa0009d00 con 0x7fdaa80ffe00 2024-03-18T20:59:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fdaa659c700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa80ffe00 0x7fdaa8199500 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fda9800e9a0 tx=0x7fda9800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fda9f7fe700 1 -- 172.21.15.46:0/1092416468 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98009790 con 0x7fdaa80ffe00 2024-03-18T20:59:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fdaacbce700 1 -- 172.21.15.46:0/1092416468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa81a60a0 con 0x7fdaa80ffe00 2024-03-18T20:59:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fda9f7fe700 1 -- 172.21.15.46:0/1092416468 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda98010430 con 0x7fdaa80ffe00 2024-03-18T20:59:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.297+0000 7fda9f7fe700 1 -- 172.21.15.46:0/1092416468 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98018680 con 0x7fdaa80ffe00 2024-03-18T20:59:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.298+0000 7fdaacbce700 1 -- 172.21.15.46:0/1092416468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa8194e40 con 0x7fdaa80ffe00 2024-03-18T20:59:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.302+0000 7fdab8d63b80 1 Processor -- start 2024-03-18T20:59:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.302+0000 7fdab8d63b80 1 -- start start 2024-03-18T20:59:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.302+0000 7fdab8d63b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdd2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.302+0000 7fdab8d63b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 0x555c4cdee880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.302+0000 7fdab8d63b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdeedc0 0x555c4cdf11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.302+0000 7fdab8d63b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4cd2c930 con 0x555c4cdd2e90 2024-03-18T20:59:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.302+0000 7fdab8d63b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4cd2a2f0 con 0x555c4cdeedc0 2024-03-18T20:59:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.302+0000 7fdab8d63b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4cd25be0 con 0x555c4cdd2570 2024-03-18T20:59:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.303+0000 7fdaa6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdd2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.303+0000 7fdaa6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdd2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:32906/0 (socket says 172.21.15.46:32906) 2024-03-18T20:59:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.303+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/702698870 learned_addr learned my addr 172.21.15.46:0/702698870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.303+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/702698870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdeedc0 msgr2=0x555c4cdf11b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:59:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.303+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/702698870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdeedc0 0x555c4cdf11b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.303+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/702698870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 msgr2=0x555c4cdee880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T20:59:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.303+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/702698870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 0x555c4cdee880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.303+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/702698870 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c4cba7350 con 0x555c4cdd2570 2024-03-18T20:59:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.303+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/702698870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdd2950 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000b0a0 tx=0x7fdaa0009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbecf3bf333c93e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.304+0000 7fda9cff9700 1 -- 172.21.15.46:0/702698870 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000e040 con 0x555c4cdd2570 2024-03-18T20:59:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.304+0000 7fda9f7fe700 1 -- 172.21.15.46:0/1092416468 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fda980105a0 con 0x7fdaa80ffe00 2024-03-18T20:59:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.304+0000 7fda9cff9700 1 -- 172.21.15.46:0/702698870 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaa0007cc0 con 0x555c4cdd2570 2024-03-18T20:59:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.304+0000 7fda9cff9700 1 -- 172.21.15.46:0/702698870 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0003c60 con 0x555c4cdd2570 2024-03-18T20:59:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.304+0000 7fda9f7fe700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fda940a9a90 0x7fda940abf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.305+0000 7fdab8d63b80 1 -- 172.21.15.46:0/702698870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 msgr2=0x555c4cdd2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.305+0000 7fda9f7fe700 1 -- 172.21.15.46:0/1092416468 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fda98014070 con 0x7fdaa80ffe00 2024-03-18T20:59:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.305+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/702698870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdd2950 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000b0a0 tx=0x7fdaa0009c60 comp rx=0 tx=0).stop 2024-03-18T20:59:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.305+0000 7fdaa659c700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fda940a9a90 0x7fda940abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.305+0000 7fdab8d63b80 1 -- 172.21.15.46:0/702698870 shutdown_connections 2024-03-18T20:59:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.305+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/702698870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdeedc0 0x555c4cdf11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.305+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/702698870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdd2950 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.305+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/702698870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 0x555c4cdee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.305+0000 7fdab8d63b80 1 -- 172.21.15.46:0/702698870 >> 172.21.15.46:0/702698870 conn(0x555c4cd491c0 msgr2=0x555c4cdcea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.305+0000 7fdaa659c700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fda940a9a90 0x7fda940abf50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fda980964c0 tx=0x7fda98096390 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.309+0000 7fdab8d63b80 1 -- 172.21.15.46:0/702698870 shutdown_connections 2024-03-18T20:59:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.309+0000 7fdab8d63b80 1 -- 172.21.15.46:0/702698870 wait complete. 2024-03-18T20:59:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.309+0000 7fdab8d63b80 1 -- start start 2024-03-18T20:59:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.310+0000 7fdab8d63b80 1 Processor -- start 2024-03-18T20:59:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.311+0000 7fdab8d63b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.311+0000 7fdab8d63b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 0x555c4cdec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.311+0000 7fdab8d63b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdeedc0 0x555c4cdec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.311+0000 7fdab8d63b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4ccc1d20 con 0x555c4cdd2e90 2024-03-18T20:59:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.311+0000 7fdab8d63b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4ccc19c0 con 0x555c4cdeedc0 2024-03-18T20:59:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.311+0000 7fdab8d63b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4cba9640 con 0x555c4cdd2570 2024-03-18T20:59:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.311+0000 7fdab8d63b80 10 client.0 _ll_register_callbacks cb 0x555c4cb2d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:59:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.311+0000 7fdaa6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.311+0000 7fdaa6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdebe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:32908/0 (socket says 172.21.15.46:32908) 2024-03-18T20:59:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.311+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/2731573154 learned_addr learned my addr 172.21.15.46:0/2731573154 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.311+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdeedc0 0x555c4cdec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.311+0000 7fdaa5d9b700 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 0x555c4cdec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.312+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/2731573154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdeedc0 msgr2=0x555c4cdec8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.312+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdeedc0 0x555c4cdec8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.312+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/2731573154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 msgr2=0x555c4cdec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.312+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 0x555c4cdec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.312+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/2731573154 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa810d310 con 0x555c4cdd2570 2024-03-18T20:59:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.313+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdebe60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000b800 tx=0x7fdaa0006160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5576b320722493e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.313+0000 7fda8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:59:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.313+0000 7fda8dffb700 1 -- 172.21.15.46:0/2731573154 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000e070 con 0x555c4cdd2570 2024-03-18T20:59:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.313+0000 7fdab8d63b80 1 -- 172.21.15.46:0/2731573154 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c4cdc66c0 con 0x555c4cdd2570 2024-03-18T20:59:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.313+0000 7fda8dffb700 1 -- 172.21.15.46:0/2731573154 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fdaa0003e40 con 0x555c4cdd2570 2024-03-18T20:59:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.313+0000 7fda8dffb700 1 -- 172.21.15.46:0/2731573154 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaa00078a0 con 0x555c4cdd2570 2024-03-18T20:59:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.313+0000 7fdab8d63b80 10 client.30529 fetch_fsmap learned FSMap version 98 2024-03-18T20:59:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.313+0000 7fdab8d63b80 1 -- 172.21.15.46:0/2731573154 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x555c4cb53b00 con 0x555c4cdd2570 2024-03-18T20:59:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.314+0000 7fda8dffb700 1 -- 172.21.15.46:0/2731573154 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaa005a080 con 0x555c4cdd2570 2024-03-18T20:59:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.314+0000 7fda8dffb700 1 -- 172.21.15.46:0/2731573154 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 98) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdaa003e720 con 0x555c4cdd2570 2024-03-18T20:59:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.314+0000 7fdab8d63b80 10 client.30529 fetch_fsmap finished waiting for FSMap version 98 2024-03-18T20:59:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.314+0000 7fdab8d63b80 10 client.30529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:59:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.314+0000 7fdab8d63b80 20 client.30529 populate_metadata read hostname 'smithi046' 2024-03-18T20:59:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.314+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555c4cdfe720 0x555c4ce00b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.314+0000 7fdab8d63b80 4 client.30529 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T20:59:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.314+0000 7fdab8d63b80 1 -- 172.21.15.46:0/2731573154 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c4cd1e6a0 con 0x555c4cdfe720 2024-03-18T20:59:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.314+0000 7fdaa5d9b700 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555c4cdfe720 0x555c4ce00b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.314+0000 7fda8dffb700 10 client.30529 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.315+0000 7fdaa5d9b700 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555c4cdfe720 0x555c4ce00b40 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61ca3a39c041edac server_cookie=49d227be61174ae3 in_seq=0 out_seq=0 2024-03-18T20:59:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.315+0000 7fda8dffb700 1 -- 172.21.15.46:0/2731573154 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555c4cd1e6a0 con 0x555c4cdfe720 2024-03-18T20:59:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.315+0000 7fda8dffb700 10 client.30529 handle_command_reply: tid=0 2024-03-18T20:59:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.315+0000 7fdab8d63b80 1 client.30529 shutdown 2024-03-18T20:59:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.320+0000 7fdab8d63b80 1 -- 172.21.15.46:0/2731573154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 msgr2=0x555c4cdebe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.320+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdebe60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000b800 tx=0x7fdaa0006160 comp rx=0 tx=0).stop 2024-03-18T20:59:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.320+0000 7fdab8d63b80 1 -- 172.21.15.46:0/2731573154 shutdown_connections 2024-03-18T20:59:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.320+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555c4cdfe720 0x555c4ce00b40 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.320+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdeedc0 0x555c4cdec8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.320+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdebe60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.320+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/2731573154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 0x555c4cdec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.320+0000 7fdab8d63b80 1 -- 172.21.15.46:0/2731573154 >> 172.21.15.46:0/2731573154 conn(0x555c4cd491c0 msgr2=0x555c4cdce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.320+0000 7fda8dffb700 0 client.30529 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.320+0000 7fdab8d63b80 1 -- 172.21.15.46:0/2731573154 shutdown_connections 2024-03-18T20:59:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.320+0000 7fdab8d63b80 1 -- 172.21.15.46:0/2731573154 wait complete. 2024-03-18T20:59:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.321+0000 7fdab8d63b80 20 client.30529 trim_cache size 0 max 16384 2024-03-18T20:59:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdab8d63b80 1 Processor -- start 2024-03-18T20:59:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdab8d63b80 1 -- start start 2024-03-18T20:59:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdab8d63b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdd2570 0x555c4cdd2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdab8d63b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 0x555c4cdcebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdab8d63b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdeedc0 0x555c4cdf5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdab8d63b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4cdfa880 con 0x555c4cdd2e90 2024-03-18T20:59:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdab8d63b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4cdc8c40 con 0x555c4cdd2570 2024-03-18T20:59:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdab8d63b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4cdf5770 con 0x555c4cdeedc0 2024-03-18T20:59:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdaa6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdd2570 0x555c4cdd2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdaa5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 0x555c4cdcebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdaa6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdd2570 0x555c4cdd2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37762/0 (socket says 172.21.15.46:37762) 2024-03-18T20:59:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/3947581452 learned_addr learned my addr 172.21.15.46:0/3947581452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.341+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/3947581452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdeedc0 0x555c4cdf5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.342+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/3947581452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdeedc0 msgr2=0x555c4cdf5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.342+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/3947581452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdeedc0 0x555c4cdf5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.342+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/3947581452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 msgr2=0x555c4cdcebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.342+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/3947581452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 0x555c4cdcebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.342+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/3947581452 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c4ccebe40 con 0x555c4cdd2570 2024-03-18T20:59:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.342+0000 7fdaa5d9b700 1 --2- 172.21.15.46:0/3947581452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 0x555c4cdcebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T20:59:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.342+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/3947581452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdd2570 0x555c4cdd2950 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000d010 tx=0x7fdaa0007ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab22d6284bab1c4b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.342+0000 7fda9d7fa700 1 -- 172.21.15.46:0/3947581452 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000e040 con 0x555c4cdd2570 2024-03-18T20:59:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.342+0000 7fda9d7fa700 1 -- 172.21.15.46:0/3947581452 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c4cdc66c0 con 0x555c4cdd2570 2024-03-18T20:59:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.342+0000 7fda9d7fa700 1 -- 172.21.15.46:0/3947581452 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0054890 con 0x555c4cdd2570 2024-03-18T20:59:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.343+0000 7fdab8d63b80 1 -- 172.21.15.46:0/3947581452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdd2570 msgr2=0x555c4cdd2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.343+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/3947581452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdd2570 0x555c4cdd2950 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000d010 tx=0x7fdaa0007ac0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.343+0000 7fdab8d63b80 1 -- 172.21.15.46:0/3947581452 shutdown_connections 2024-03-18T20:59:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.343+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/3947581452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdeedc0 0x555c4cdf5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.343+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/3947581452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdd2570 0x555c4cdd2950 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.343+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/3947581452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdd2e90 0x555c4cdcebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.343+0000 7fdab8d63b80 1 -- 172.21.15.46:0/3947581452 >> 172.21.15.46:0/3947581452 conn(0x555c4cd491c0 msgr2=0x555c4cdcd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.343+0000 7fdab8d63b80 1 -- 172.21.15.46:0/3947581452 shutdown_connections 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.343+0000 7fdab8d63b80 1 -- 172.21.15.46:0/3947581452 wait complete. 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.343+0000 7fdab8d63b80 1 -- start start 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdab8d63b80 1 Processor -- start 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdab8d63b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdd8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdab8d63b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdd2e90 0x555c4cdf11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdab8d63b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdeedc0 0x555c4cdf16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdab8d63b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4cdf1c30 con 0x555c4cdeedc0 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdab8d63b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4cdf1da0 con 0x555c4cdd2e90 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdab8d63b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4cdf1f60 con 0x555c4cdd2570 2024-03-18T20:59:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdab8d63b80 10 client.0 _ll_register_callbacks cb 0x7fdaa000bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdaa6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdd8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdaa6d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdd8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:32930/0 (socket says 172.21.15.46:32930) 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/92895811 learned_addr learned my addr 172.21.15.46:0/92895811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdeedc0 0x555c4cdf16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdaa5d9b700 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdd2e90 0x555c4cdf11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/92895811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdd2e90 msgr2=0x555c4cdf11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.344+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdd2e90 0x555c4cdf11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.345+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/92895811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdeedc0 msgr2=0x555c4cdf16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.345+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdeedc0 0x555c4cdf16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.345+0000 7fdaa6d9d700 1 -- 172.21.15.46:0/92895811 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda90009c30 con 0x555c4cdd2570 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.345+0000 7fdaa6d9d700 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdd8850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009fd0 tx=0x7fdaa0054470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7774acb9bc1ede8b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.345+0000 7fda8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.345+0000 7fda8dffb700 1 -- 172.21.15.46:0/92895811 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000e070 con 0x555c4cdd2570 2024-03-18T20:59:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.345+0000 7fda8dffb700 1 -- 172.21.15.46:0/92895811 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fdaa0003e40 con 0x555c4cdd2570 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.345+0000 7fdab8d63b80 1 -- 172.21.15.46:0/92895811 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdaa0007cc0 con 0x555c4cdd2570 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.345+0000 7fda8dffb700 1 -- 172.21.15.46:0/92895811 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaa003e650 con 0x555c4cdd2570 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.345+0000 7fdab8d63b80 10 client.30535 fetch_fsmap learned FSMap version 98 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.345+0000 7fdab8d63b80 1 -- 172.21.15.46:0/92895811 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x555c4cb53b00 con 0x555c4cdd2570 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.345+0000 7fda8dffb700 1 -- 172.21.15.46:0/92895811 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaa006b070 con 0x555c4cdd2570 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.346+0000 7fda8dffb700 1 -- 172.21.15.46:0/92895811 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 98) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdaa0047c10 con 0x555c4cdd2570 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.346+0000 7fdab8d63b80 10 client.30535 fetch_fsmap finished waiting for FSMap version 98 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.346+0000 7fdab8d63b80 10 client.30535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.346+0000 7fdab8d63b80 20 client.30535 populate_metadata read hostname 'smithi046' 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.346+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555c4ce526b0 0x555c4ce54ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.346+0000 7fdab8d63b80 4 client.30535 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.346+0000 7fdab8d63b80 1 -- 172.21.15.46:0/92895811 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555c4cb74c90 con 0x555c4ce526b0 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.346+0000 7fdaa5d9b700 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555c4ce526b0 0x555c4ce54ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T20:59:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.346+0000 7fdaa5d9b700 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555c4ce526b0 0x555c4ce54ad0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ffead59927ddc8 server_cookie=d83c141abfab0f19 in_seq=0 out_seq=0 2024-03-18T20:59:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.346+0000 7fda8dffb700 10 client.30535 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:58.499 INFO:tasks.workunit.client.0.smithi046.stdout: 10 405854 31.10 MB/sec execute 514 sec latency 150.661 ms 2024-03-18T20:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fda8dffb700 1 -- 172.21.15.46:0/92895811 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555c4cb74c90 con 0x555c4ce526b0 2024-03-18T20:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fda8dffb700 10 client.30535 handle_command_reply: tid=0 2024-03-18T20:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fdab8d63b80 1 client.30535 shutdown 2024-03-18T20:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fdab8d63b80 1 -- 172.21.15.46:0/92895811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 msgr2=0x555c4cdd8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdd8850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009fd0 tx=0x7fdaa0054470 comp rx=0 tx=0).stop 2024-03-18T20:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fdab8d63b80 1 -- 172.21.15.46:0/92895811 shutdown_connections 2024-03-18T20:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x555c4ce526b0 0x555c4ce54ad0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555c4cdeedc0 0x555c4cdf16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c4cdd2570 0x555c4cdd8850 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fdab8d63b80 1 --2- 172.21.15.46:0/92895811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555c4cdd2e90 0x555c4cdf11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fdab8d63b80 1 -- 172.21.15.46:0/92895811 >> 172.21.15.46:0/92895811 conn(0x555c4cd491c0 msgr2=0x555c4cdd95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fda8dffb700 0 client.30535 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T20:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fdab8d63b80 1 -- 172.21.15.46:0/92895811 shutdown_connections 2024-03-18T20:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fdab8d63b80 1 -- 172.21.15.46:0/92895811 wait complete. 2024-03-18T20:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.602+0000 7fdab8d63b80 20 client.30535 trim_cache size 0 max 16384 2024-03-18T20:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T20:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T20:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T20:59:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T20:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.604+0000 7fdaacbce700 1 -- 172.21.15.46:0/1092416468 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fda940a9a90 msgr2=0x7fda940abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.604+0000 7fdaacbce700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fda940a9a90 0x7fda940abf50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fda980964c0 tx=0x7fda98096390 comp rx=0 tx=0).stop 2024-03-18T20:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.604+0000 7fdaacbce700 1 -- 172.21.15.46:0/1092416468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa80ffe00 msgr2=0x7fdaa8199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T20:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.604+0000 7fdaacbce700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa80ffe00 0x7fdaa8199500 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fda9800e9a0 tx=0x7fda9800ecb0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.604+0000 7fdaacbce700 1 -- 172.21.15.46:0/1092416468 shutdown_connections 2024-03-18T20:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.604+0000 7fdaacbce700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fda940a9a90 0x7fda940abf50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.604+0000 7fdaacbce700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa819dd90 0x7fdaa819e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.604+0000 7fdaacbce700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa80ffe00 0x7fdaa8199500 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.604+0000 7fdaacbce700 1 --2- 172.21.15.46:0/1092416468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdaa8104f20 0x7fdaa8199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T20:59:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.604+0000 7fdaacbce700 1 -- 172.21.15.46:0/1092416468 >> 172.21.15.46:0/1092416468 conn(0x7fdaa80fb680 msgr2=0x7fdaa8103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T20:59:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.606+0000 7fdaacbce700 1 -- 172.21.15.46:0/1092416468 shutdown_connections 2024-03-18T20:59:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T20:59:58.608+0000 7fdaacbce700 1 -- 172.21.15.46:0/1092416468 wait complete. 2024-03-18T20:59:58.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 20:59:58 smithi073 ceph-mon[47384]: pgmap v754: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.2 MiB/s wr, 440 op/s 2024-03-18T20:59:59.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 20:59:58 smithi155 ceph-mon[48344]: pgmap v754: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.2 MiB/s wr, 440 op/s 2024-03-18T20:59:59.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 20:59:58 smithi046 ceph-mon[43495]: pgmap v754: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.2 MiB/s wr, 440 op/s 2024-03-18T20:59:59.499 INFO:tasks.workunit.client.0.smithi046.stdout: 10 406308 31.08 MB/sec execute 515 sec latency 237.568 ms 2024-03-18T21:00:00.295 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a[43472]: 2024-03-18T20:59:59.999+0000 7fb26e3e2700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 2 MDSs report damaged metadata 2024-03-18T21:00:00.499 INFO:tasks.workunit.client.0.smithi046.stdout: 10 406652 31.07 MB/sec execute 516 sec latency 476.993 ms 2024-03-18T21:00:00.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:00 smithi073 ceph-mon[47384]: pgmap v755: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.1 MiB/s wr, 569 op/s 2024-03-18T21:00:00.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:00 smithi073 ceph-mon[47384]: overall HEALTH_ERR 2 MDSs report damaged metadata 2024-03-18T21:00:01.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:00 smithi155 ceph-mon[48344]: pgmap v755: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.1 MiB/s wr, 569 op/s 2024-03-18T21:00:01.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:00 smithi155 ceph-mon[48344]: overall HEALTH_ERR 2 MDSs report damaged metadata 2024-03-18T21:00:01.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:00 smithi046 ceph-mon[43495]: pgmap v755: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 9.1 MiB/s wr, 569 op/s 2024-03-18T21:00:01.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:00 smithi046 ceph-mon[43495]: overall HEALTH_ERR 2 MDSs report damaged metadata 2024-03-18T21:00:01.499 INFO:tasks.workunit.client.0.smithi046.stdout: 10 406652 31.01 MB/sec execute 517 sec latency 1477.086 ms 2024-03-18T21:00:02.499 INFO:tasks.workunit.client.0.smithi046.stdout: 10 406762 30.96 MB/sec execute 518 sec latency 2230.086 ms 2024-03-18T21:00:02.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:02 smithi073 ceph-mon[47384]: pgmap v756: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.2 MiB/s wr, 481 op/s 2024-03-18T21:00:03.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:02 smithi155 ceph-mon[48344]: pgmap v756: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.2 MiB/s wr, 481 op/s 2024-03-18T21:00:03.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:02 smithi046 ceph-mon[43495]: pgmap v756: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.2 MiB/s wr, 481 op/s 2024-03-18T21:00:03.500 INFO:tasks.workunit.client.0.smithi046.stdout: 10 406789 30.90 MB/sec execute 519 sec latency 967.083 ms 2024-03-18T21:00:04.500 INFO:tasks.workunit.client.0.smithi046.stdout: 10 406789 30.84 MB/sec execute 520 sec latency 1967.197 ms 2024-03-18T21:00:04.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:04 smithi073 ceph-mon[47384]: pgmap v757: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.4 MiB/s wr, 439 op/s 2024-03-18T21:00:05.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:04 smithi155 ceph-mon[48344]: pgmap v757: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.4 MiB/s wr, 439 op/s 2024-03-18T21:00:05.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:04 smithi046 ceph-mon[43495]: pgmap v757: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.4 MiB/s wr, 439 op/s 2024-03-18T21:00:05.504 INFO:tasks.workunit.client.0.smithi046.stdout: 10 406941 30.80 MB/sec execute 521 sec latency 2607.458 ms 2024-03-18T21:00:06.502 INFO:tasks.workunit.client.0.smithi046.stdout: 10 407311 30.78 MB/sec execute 522 sec latency 310.182 ms 2024-03-18T21:00:06.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:06 smithi073 ceph-mon[47384]: pgmap v758: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.0 MiB/s wr, 506 op/s 2024-03-18T21:00:06.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:06 smithi155 ceph-mon[48344]: pgmap v758: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.0 MiB/s wr, 506 op/s 2024-03-18T21:00:06.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:06 smithi046 ceph-mon[43495]: pgmap v758: 97 pgs: 97 active+clean; 2.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.0 MiB/s wr, 506 op/s 2024-03-18T21:00:07.502 INFO:tasks.workunit.client.0.smithi046.stdout: 10 407710 30.77 MB/sec execute 523 sec latency 140.165 ms 2024-03-18T21:00:08.502 INFO:tasks.workunit.client.0.smithi046.stdout: 10 408280 30.75 MB/sec execute 524 sec latency 86.780 ms 2024-03-18T21:00:08.791 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:08 smithi155 ceph-mon[48344]: pgmap v759: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 400 op/s 2024-03-18T21:00:08.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:08 smithi046 ceph-mon[43495]: pgmap v759: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 400 op/s 2024-03-18T21:00:08.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:08 smithi073 ceph-mon[47384]: pgmap v759: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.4 MiB/s wr, 400 op/s 2024-03-18T21:00:09.503 INFO:tasks.workunit.client.0.smithi046.stdout: 10 408465 30.71 MB/sec execute 525 sec latency 205.139 ms 2024-03-18T21:00:09.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:09 smithi073 ceph-mon[47384]: osdmap e118: 12 total, 12 up, 12 in 2024-03-18T21:00:10.046 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:09 smithi155 ceph-mon[48344]: osdmap e118: 12 total, 12 up, 12 in 2024-03-18T21:00:10.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:09 smithi046 ceph-mon[43495]: osdmap e118: 12 total, 12 up, 12 in 2024-03-18T21:00:10.505 INFO:tasks.workunit.client.0.smithi046.stdout: 10 408719 30.67 MB/sec execute 526 sec latency 240.881 ms 2024-03-18T21:00:10.976 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:10 smithi073 ceph-mon[47384]: pgmap v761: 97 pgs: 12 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 71 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.7 MiB/s wr, 396 op/s 2024-03-18T21:00:11.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:10 smithi155 ceph-mon[48344]: pgmap v761: 97 pgs: 12 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 71 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.7 MiB/s wr, 396 op/s 2024-03-18T21:00:11.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:10 smithi046 ceph-mon[43495]: pgmap v761: 97 pgs: 12 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 71 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.7 MiB/s wr, 396 op/s 2024-03-18T21:00:11.505 INFO:tasks.workunit.client.0.smithi046.stdout: 10 408768 30.62 MB/sec execute 527 sec latency 950.034 ms 2024-03-18T21:00:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.916+0000 7fdc0458b700 1 -- 172.21.15.46:0/2227665070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbfc10f940 msgr2=0x7fdbfc1082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.916+0000 7fdc0458b700 1 --2- 172.21.15.46:0/2227665070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbfc10f940 0x7fdbfc1082f0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80099c0 tx=0x7fdbf8009cd0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.922+0000 7fdc0458b700 1 -- 172.21.15.46:0/2227665070 shutdown_connections 2024-03-18T21:00:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.922+0000 7fdc0458b700 1 --2- 172.21.15.46:0/2227665070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbfc11a590 0x7fdbfc11c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.922+0000 7fdc0458b700 1 --2- 172.21.15.46:0/2227665070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbfc10f4d0 0x7fdbfc107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.922+0000 7fdc0458b700 1 --2- 172.21.15.46:0/2227665070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbfc10f940 0x7fdbfc1082f0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.922+0000 7fdc0458b700 1 -- 172.21.15.46:0/2227665070 >> 172.21.15.46:0/2227665070 conn(0x7fdbfc06bd80 msgr2=0x7fdbfc06c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.928+0000 7fdc0458b700 1 -- 172.21.15.46:0/2227665070 shutdown_connections 2024-03-18T21:00:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.929+0000 7fdc0458b700 1 -- 172.21.15.46:0/2227665070 wait complete. 2024-03-18T21:00:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.930+0000 7fdc0458b700 1 Processor -- start 2024-03-18T21:00:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.930+0000 7fdc0458b700 1 -- start start 2024-03-18T21:00:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.930+0000 7fdc0458b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbfc10f4d0 0x7fdbfc1a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.930+0000 7fdc0458b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbfc10f940 0x7fdbfc1a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.930+0000 7fdc0458b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbfc11a590 0x7fdbfc1aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.930+0000 7fdc0458b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1aafe0 con 0x7fdbfc10f940 2024-03-18T21:00:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.930+0000 7fdc0458b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1ab150 con 0x7fdbfc10f4d0 2024-03-18T21:00:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.930+0000 7fdc0458b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1ab310 con 0x7fdbfc11a590 2024-03-18T21:00:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.931+0000 7fdc02b28700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbfc11a590 0x7fdbfc1aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.931+0000 7fdc02b28700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbfc11a590 0x7fdbfc1aa9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:51790/0 (socket says 172.21.15.46:51790) 2024-03-18T21:00:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.931+0000 7fdc02b28700 1 -- 172.21.15.46:0/2205214759 learned_addr learned my addr 172.21.15.46:0/2205214759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.932+0000 7fdc01b26700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbfc10f940 0x7fdbfc1a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.932+0000 7fdc01b26700 1 -- 172.21.15.46:0/2205214759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbfc11a590 msgr2=0x7fdbfc1aa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.932+0000 7fdc01b26700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbfc11a590 0x7fdbfc1aa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.932+0000 7fdc01b26700 1 -- 172.21.15.46:0/2205214759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbfc10f4d0 msgr2=0x7fdbfc1a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.932+0000 7fdc01b26700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbfc10f4d0 0x7fdbfc1a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.932+0000 7fdc01b26700 1 -- 172.21.15.46:0/2205214759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf80095f0 con 0x7fdbfc10f940 2024-03-18T21:00:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.933+0000 7fdc01b26700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbfc10f940 0x7fdbfc1a6600 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc10fd40 tx=0x7fdbf800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.933+0000 7fdbf37fe700 1 -- 172.21.15.46:0/2205214759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf801d070 con 0x7fdbfc10f940 2024-03-18T21:00:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.933+0000 7fdbf37fe700 1 -- 172.21.15.46:0/2205214759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf8005510 con 0x7fdbfc10f940 2024-03-18T21:00:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.933+0000 7fdbf37fe700 1 -- 172.21.15.46:0/2205214759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8004ad0 con 0x7fdbfc10f940 2024-03-18T21:00:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.942+0000 7fdc0458b700 1 -- 172.21.15.46:0/2205214759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc1ab590 con 0x7fdbfc10f940 2024-03-18T21:00:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.942+0000 7fdc0458b700 1 -- 172.21.15.46:0/2205214759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc1ab7e0 con 0x7fdbfc10f940 2024-03-18T21:00:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.945+0000 7fdbf37fe700 1 -- 172.21.15.46:0/2205214759 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fdbf8004c30 con 0x7fdbfc10f940 2024-03-18T21:00:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.945+0000 7fdbf37fe700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdbe80a1390 0x7fdbe80a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.946+0000 7fdbf37fe700 1 -- 172.21.15.46:0/2205214759 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fdbf80cd660 con 0x7fdbfc10f940 2024-03-18T21:00:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.951+0000 7fdc10720b80 1 Processor -- start 2024-03-18T21:00:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.952+0000 7fdc10720b80 1 -- start start 2024-03-18T21:00:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.953+0000 7fdc10720b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 0x55bde902c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.953+0000 7fdc10720b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 0x55bde9037870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.953+0000 7fdc10720b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 0x55bde903a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.953+0000 7fdc10720b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde8f86930 con 0x55bde902ce90 2024-03-18T21:00:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.953+0000 7fdc10720b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde8f842f0 con 0x55bde9037db0 2024-03-18T21:00:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.953+0000 7fdc10720b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde8f7fbe0 con 0x55bde902c570 2024-03-18T21:00:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.953+0000 7fdc02327700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdbe80a1390 0x7fdbe80a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.955+0000 7fdc02b28700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 0x55bde903a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.955+0000 7fdc02b28700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 0x55bde903a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:53698/0 (socket says 172.21.15.46:53698) 2024-03-18T21:00:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.955+0000 7fdc02b28700 1 -- 172.21.15.46:0/3516554324 learned_addr learned my addr 172.21.15.46:0/3516554324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.956+0000 7fdc02327700 1 --2- 172.21.15.46:0/3516554324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 0x55bde902c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.957+0000 7fdc02327700 1 -- 172.21.15.46:0/3516554324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 msgr2=0x55bde903a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.957+0000 7fdc02327700 1 --2- 172.21.15.46:0/3516554324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 0x55bde903a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.957+0000 7fdc02327700 1 -- 172.21.15.46:0/3516554324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 msgr2=0x55bde9037870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.957+0000 7fdc02327700 1 --2- 172.21.15.46:0/3516554324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 0x55bde9037870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.957+0000 7fdc02327700 1 -- 172.21.15.46:0/3516554324 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bde8e01350 con 0x55bde902c570 2024-03-18T21:00:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.958+0000 7fdc02327700 1 --2- 172.21.15.46:0/3516554324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 0x55bde902c950 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4010c40 tx=0x7fdbf4010f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20f0813697a4e257 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.958+0000 7fdc02327700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdbe80a1390 0x7fdbe80a3850 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40175d0 tx=0x7fdbf401b000 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.958+0000 7fdbf0ff9700 1 -- 172.21.15.46:0/3516554324 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400e070 con 0x55bde902c570 2024-03-18T21:00:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.958+0000 7fdbf0ff9700 1 -- 172.21.15.46:0/3516554324 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf4017d40 con 0x55bde902c570 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.958+0000 7fdbf0ff9700 1 -- 172.21.15.46:0/3516554324 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4004e10 con 0x55bde902c570 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.959+0000 7fdc10720b80 1 -- 172.21.15.46:0/3516554324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 msgr2=0x55bde902c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.959+0000 7fdc10720b80 1 --2- 172.21.15.46:0/3516554324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 0x55bde902c950 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4010c40 tx=0x7fdbf4010f50 comp rx=0 tx=0).stop 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.960+0000 7fdc10720b80 1 -- 172.21.15.46:0/3516554324 shutdown_connections 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.960+0000 7fdc10720b80 1 --2- 172.21.15.46:0/3516554324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 0x55bde903a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.960+0000 7fdc10720b80 1 --2- 172.21.15.46:0/3516554324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 0x55bde902c950 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.960+0000 7fdc10720b80 1 --2- 172.21.15.46:0/3516554324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 0x55bde9037870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.960+0000 7fdc10720b80 1 -- 172.21.15.46:0/3516554324 >> 172.21.15.46:0/3516554324 conn(0x55bde8fa31c0 msgr2=0x55bde9028a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.960+0000 7fdc10720b80 1 -- 172.21.15.46:0/3516554324 shutdown_connections 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.960+0000 7fdc10720b80 1 -- 172.21.15.46:0/3516554324 wait complete. 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.960+0000 7fdc10720b80 1 -- start start 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.961+0000 7fdc10720b80 1 Processor -- start 2024-03-18T21:00:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.961+0000 7fdc10720b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 0x55bde9034cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.961+0000 7fdc10720b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 0x55bde90351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.961+0000 7fdc10720b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 0x55bde9035730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.961+0000 7fdc10720b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde8f1bd20 con 0x55bde902ce90 2024-03-18T21:00:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.961+0000 7fdc10720b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde8f1b9c0 con 0x55bde9037db0 2024-03-18T21:00:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.961+0000 7fdc10720b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde8e03640 con 0x55bde902c570 2024-03-18T21:00:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.961+0000 7fdc10720b80 10 client.0 _ll_register_callbacks cb 0x55bde8d87340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T21:00:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.962+0000 7fdc02327700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 0x55bde9034cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.962+0000 7fdc02327700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 0x55bde9034cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:51806/0 (socket says 172.21.15.46:51806) 2024-03-18T21:00:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.962+0000 7fdc02327700 1 -- 172.21.15.46:0/4003559045 learned_addr learned my addr 172.21.15.46:0/4003559045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.965+0000 7fdc01b26700 1 --2- 172.21.15.46:0/4003559045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 0x55bde90351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.966+0000 7fdc02327700 1 -- 172.21.15.46:0/4003559045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 msgr2=0x55bde9035730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.966+0000 7fdc02327700 1 --2- 172.21.15.46:0/4003559045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 0x55bde9035730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.966+0000 7fdc02327700 1 -- 172.21.15.46:0/4003559045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 msgr2=0x55bde90351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.966+0000 7fdc02327700 1 --2- 172.21.15.46:0/4003559045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 0x55bde90351f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.966+0000 7fdc02327700 1 -- 172.21.15.46:0/4003559045 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf4011040 con 0x55bde902c570 2024-03-18T21:00:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.966+0000 7fdc02327700 1 --2- 172.21.15.46:0/4003559045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 0x55bde9034cb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdbf401aa10 tx=0x7fdbf401aa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea71962be4b1af56 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.966+0000 7fdbe5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T21:00:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.967+0000 7fdbe5ffb700 1 -- 172.21.15.46:0/4003559045 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400e040 con 0x55bde902c570 2024-03-18T21:00:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.967+0000 7fdbe5ffb700 1 -- 172.21.15.46:0/4003559045 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fdbf4020070 con 0x55bde902c570 2024-03-18T21:00:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.967+0000 7fdc10720b80 1 -- 172.21.15.46:0/4003559045 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bde90206c0 con 0x55bde902c570 2024-03-18T21:00:11.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:11 smithi073 ceph-mon[47384]: pgmap v762: 97 pgs: 18 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 49 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.2 MiB/s wr, 431 op/s 2024-03-18T21:00:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.972+0000 7fdbe5ffb700 1 -- 172.21.15.46:0/4003559045 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbf401adc0 con 0x55bde902c570 2024-03-18T21:00:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.974+0000 7fdc10720b80 10 client.30547 fetch_fsmap learned FSMap version 98 2024-03-18T21:00:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.974+0000 7fdc10720b80 1 -- 172.21.15.46:0/4003559045 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x55bde8dadb00 con 0x55bde902c570 2024-03-18T21:00:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.975+0000 7fdbe5ffb700 1 -- 172.21.15.46:0/4003559045 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bde90206c0 con 0x55bde902c570 2024-03-18T21:00:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.975+0000 7fdbe5ffb700 1 -- 172.21.15.46:0/4003559045 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 98) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdbf4024420 con 0x55bde902c570 2024-03-18T21:00:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.976+0000 7fdc10720b80 10 client.30547 fetch_fsmap finished waiting for FSMap version 98 2024-03-18T21:00:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.976+0000 7fdc10720b80 10 client.30547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T21:00:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.976+0000 7fdc10720b80 20 client.30547 populate_metadata read hostname 'smithi046' 2024-03-18T21:00:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.976+0000 7fdc10720b80 1 --2- 172.21.15.46:0/4003559045 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bde9047790 0x55bde9049bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.976+0000 7fdc10720b80 4 client.30547 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T21:00:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.976+0000 7fdc10720b80 1 -- 172.21.15.46:0/4003559045 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bde8f786a0 con 0x55bde9047790 2024-03-18T21:00:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.976+0000 7fdc01b26700 1 --2- 172.21.15.46:0/4003559045 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bde9047790 0x55bde9049bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.978+0000 7fdc01b26700 1 --2- 172.21.15.46:0/4003559045 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bde9047790 0x55bde9049bb0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d6524c3210dbf5a server_cookie=aeb5e818165c1f56 in_seq=0 out_seq=0 2024-03-18T21:00:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:11.978+0000 7fdbe5ffb700 10 client.30547 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.000+0000 7fdbe5ffb700 1 -- 172.21.15.46:0/4003559045 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bde8f786a0 con 0x55bde9047790 2024-03-18T21:00:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.001+0000 7fdbe5ffb700 10 client.30547 handle_command_reply: tid=0 2024-03-18T21:00:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.001+0000 7fdc10720b80 1 client.30547 shutdown 2024-03-18T21:00:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.002+0000 7fdc10720b80 1 -- 172.21.15.46:0/4003559045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 msgr2=0x55bde9034cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.002+0000 7fdc10720b80 1 --2- 172.21.15.46:0/4003559045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 0x55bde9034cb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdbf401aa10 tx=0x7fdbf401aa40 comp rx=0 tx=0).stop 2024-03-18T21:00:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.002+0000 7fdc10720b80 1 -- 172.21.15.46:0/4003559045 shutdown_connections 2024-03-18T21:00:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.002+0000 7fdc10720b80 1 --2- 172.21.15.46:0/4003559045 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bde9047790 0x55bde9049bb0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.002+0000 7fdc10720b80 1 --2- 172.21.15.46:0/4003559045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 0x55bde9035730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.002+0000 7fdc10720b80 1 --2- 172.21.15.46:0/4003559045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902c570 0x55bde9034cb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.002+0000 7fdc10720b80 1 --2- 172.21.15.46:0/4003559045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 0x55bde90351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.002+0000 7fdc10720b80 1 -- 172.21.15.46:0/4003559045 >> 172.21.15.46:0/4003559045 conn(0x55bde8fa31c0 msgr2=0x55bde9028650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.002+0000 7fdbe5ffb700 0 client.30547 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.002+0000 7fdc10720b80 1 -- 172.21.15.46:0/4003559045 shutdown_connections 2024-03-18T21:00:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.002+0000 7fdc10720b80 1 -- 172.21.15.46:0/4003559045 wait complete. 2024-03-18T21:00:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.007+0000 7fdc10720b80 20 client.30547 trim_cache size 0 max 16384 2024-03-18T21:00:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.035+0000 7fdc10720b80 1 Processor -- start 2024-03-18T21:00:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.035+0000 7fdc10720b80 1 -- start start 2024-03-18T21:00:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.035+0000 7fdc10720b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde902c570 0x55bde9035550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.035+0000 7fdc10720b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 0x55bde902ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.035+0000 7fdc10720b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde9037db0 0x55bde90304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.035+0000 7fdc10720b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde90362b0 con 0x55bde902ce90 2024-03-18T21:00:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.035+0000 7fdc10720b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde90297a0 con 0x55bde902c570 2024-03-18T21:00:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.035+0000 7fdc10720b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde9022c40 con 0x55bde9037db0 2024-03-18T21:00:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.036+0000 7fdc02327700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde902c570 0x55bde9035550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.036+0000 7fdc02b28700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde9037db0 0x55bde90304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.036+0000 7fdc02b28700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde9037db0 0x55bde90304f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:51810/0 (socket says 172.21.15.46:51810) 2024-03-18T21:00:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.036+0000 7fdc02b28700 1 -- 172.21.15.46:0/1744628586 learned_addr learned my addr 172.21.15.46:0/1744628586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:12.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:11 smithi155 ceph-mon[48344]: pgmap v762: 97 pgs: 18 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 49 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.2 MiB/s wr, 431 op/s 2024-03-18T21:00:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.039+0000 7fdc02b28700 1 -- 172.21.15.46:0/1744628586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde902c570 msgr2=0x55bde9035550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.039+0000 7fdc02b28700 1 --2- 172.21.15.46:0/1744628586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde902c570 0x55bde9035550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.039+0000 7fdc02b28700 1 -- 172.21.15.46:0/1744628586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 msgr2=0x55bde902ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.039+0000 7fdc02b28700 1 --2- 172.21.15.46:0/1744628586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 0x55bde902ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.039+0000 7fdc02b28700 1 -- 172.21.15.46:0/1744628586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bde8f45e40 con 0x55bde9037db0 2024-03-18T21:00:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.042+0000 7fdc02b28700 1 --2- 172.21.15.46:0/1744628586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde9037db0 0x55bde90304f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdbec005950 tx=0x7fdbec011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2946972f8dc5afde server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:12.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.043+0000 7fdbf17fa700 1 -- 172.21.15.46:0/1744628586 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec016070 con 0x55bde9037db0 2024-03-18T21:00:12.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.043+0000 7fdbf17fa700 1 -- 172.21.15.46:0/1744628586 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbec011bd0 con 0x55bde9037db0 2024-03-18T21:00:12.047 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.043+0000 7fdbf17fa700 1 -- 172.21.15.46:0/1744628586 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec010470 con 0x55bde9037db0 2024-03-18T21:00:12.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:11 smithi046 ceph-mon[43495]: pgmap v762: 97 pgs: 18 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 49 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.2 MiB/s wr, 431 op/s 2024-03-18T21:00:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.046+0000 7fdc10720b80 1 -- 172.21.15.46:0/1744628586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde9037db0 msgr2=0x55bde90304f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.046+0000 7fdc10720b80 1 --2- 172.21.15.46:0/1744628586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde9037db0 0x55bde90304f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdbec005950 tx=0x7fdbec011720 comp rx=0 tx=0).stop 2024-03-18T21:00:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.049+0000 7fdc10720b80 1 -- 172.21.15.46:0/1744628586 shutdown_connections 2024-03-18T21:00:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.049+0000 7fdc10720b80 1 --2- 172.21.15.46:0/1744628586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde9037db0 0x55bde90304f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.049+0000 7fdc10720b80 1 --2- 172.21.15.46:0/1744628586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde902c570 0x55bde9035550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.049+0000 7fdc10720b80 1 --2- 172.21.15.46:0/1744628586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902ce90 0x55bde902ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.049+0000 7fdc10720b80 1 -- 172.21.15.46:0/1744628586 >> 172.21.15.46:0/1744628586 conn(0x55bde8fa31c0 msgr2=0x55bde90497a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.050+0000 7fdc10720b80 1 -- 172.21.15.46:0/1744628586 shutdown_connections 2024-03-18T21:00:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.050+0000 7fdc10720b80 1 -- 172.21.15.46:0/1744628586 wait complete. 2024-03-18T21:00:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.050+0000 7fdc10720b80 1 -- start start 2024-03-18T21:00:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.050+0000 7fdc10720b80 1 Processor -- start 2024-03-18T21:00:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.051+0000 7fdc10720b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902c570 0x55bde9038a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.051+0000 7fdc10720b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902ce90 0x55bde9038f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.051+0000 7fdc10720b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 0x55bde9039490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.051+0000 7fdc10720b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde90bcad0 con 0x55bde902c570 2024-03-18T21:00:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.051+0000 7fdc10720b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde90bcc10 con 0x55bde9037db0 2024-03-18T21:00:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.051+0000 7fdc10720b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde90bcd50 con 0x55bde902ce90 2024-03-18T21:00:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.051+0000 7fdc10720b80 10 client.0 _ll_register_callbacks cb 0x7fdbec009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T21:00:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.053+0000 7fdc02b28700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 0x55bde9039490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.054+0000 7fdc01b26700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902ce90 0x55bde9038f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.054+0000 7fdc01b26700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902ce90 0x55bde9038f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:51824/0 (socket says 172.21.15.46:51824) 2024-03-18T21:00:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.054+0000 7fdc01b26700 1 -- 172.21.15.46:0/3223620787 learned_addr learned my addr 172.21.15.46:0/3223620787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.054+0000 7fdc02327700 1 --2- 172.21.15.46:0/3223620787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902c570 0x55bde9038a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:12.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.054+0000 7fdc01b26700 1 -- 172.21.15.46:0/3223620787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 msgr2=0x55bde9039490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:12.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.054+0000 7fdc01b26700 1 --2- 172.21.15.46:0/3223620787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 0x55bde9039490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.054+0000 7fdc01b26700 1 -- 172.21.15.46:0/3223620787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902c570 msgr2=0x55bde9038a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:12.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.054+0000 7fdc01b26700 1 --2- 172.21.15.46:0/3223620787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902c570 0x55bde9038a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.054+0000 7fdc01b26700 1 -- 172.21.15.46:0/3223620787 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbec009ab0 con 0x55bde902ce90 2024-03-18T21:00:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.056+0000 7fdc01b26700 1 --2- 172.21.15.46:0/3223620787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902ce90 0x55bde9038f50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdbf809a6e0 tx=0x7fdbf8003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91e4c0f53adfc8a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.056+0000 7fdbe5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T21:00:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.056+0000 7fdbe5ffb700 1 -- 172.21.15.46:0/3223620787 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8005680 con 0x55bde902ce90 2024-03-18T21:00:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.056+0000 7fdbe5ffb700 1 -- 172.21.15.46:0/3223620787 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8600+0+0 (secure 0 0 0) 0x7fdbf809a750 con 0x55bde902ce90 2024-03-18T21:00:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.056+0000 7fdc10720b80 1 -- 172.21.15.46:0/3223620787 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbf4017d40 con 0x55bde902ce90 2024-03-18T21:00:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.057+0000 7fdbe5ffb700 1 -- 172.21.15.46:0/3223620787 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbf809e580 con 0x55bde902ce90 2024-03-18T21:00:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.057+0000 7fdc10720b80 10 client.30553 fetch_fsmap learned FSMap version 98 2024-03-18T21:00:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.057+0000 7fdc10720b80 1 -- 172.21.15.46:0/3223620787 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) v3 -- 0x55bde8e05c70 con 0x55bde902ce90 2024-03-18T21:00:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.057+0000 7fdbe5ffb700 1 -- 172.21.15.46:0/3223620787 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf4017d40 con 0x55bde902ce90 2024-03-18T21:00:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.057+0000 7fdbe5ffb700 1 -- 172.21.15.46:0/3223620787 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 98) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdbf809aa40 con 0x55bde902ce90 2024-03-18T21:00:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.057+0000 7fdc10720b80 10 client.30553 fetch_fsmap finished waiting for FSMap version 98 2024-03-18T21:00:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.057+0000 7fdc10720b80 10 client.30553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T21:00:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.057+0000 7fdc10720b80 20 client.30553 populate_metadata read hostname 'smithi046' 2024-03-18T21:00:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.057+0000 7fdc10720b80 1 --2- 172.21.15.46:0/3223620787 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bde90be310 0x55bde90c0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.057+0000 7fdc10720b80 4 client.30553 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T21:00:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.057+0000 7fdc10720b80 1 -- 172.21.15.46:0/3223620787 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bde8dcec90 con 0x55bde90be310 2024-03-18T21:00:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.073+0000 7fdc02327700 1 --2- 172.21.15.46:0/3223620787 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bde90be310 0x55bde90c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.095+0000 7fdc02327700 1 --2- 172.21.15.46:0/3223620787 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bde90be310 0x55bde90c0730 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26c8f0100918d16 server_cookie=402bb1ce689f17f7 in_seq=0 out_seq=0 2024-03-18T21:00:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.095+0000 7fdbe5ffb700 10 client.30553 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:12.505 INFO:tasks.workunit.client.0.smithi046.stdout: 10 408843 30.57 MB/sec execute 528 sec latency 1295.171 ms 2024-03-18T21:00:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.635+0000 7fdbe5ffb700 1 -- 172.21.15.46:0/3223620787 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bde8dcec90 con 0x55bde90be310 2024-03-18T21:00:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.635+0000 7fdbe5ffb700 10 client.30553 handle_command_reply: tid=0 2024-03-18T21:00:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.635+0000 7fdc10720b80 1 client.30553 shutdown 2024-03-18T21:00:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.639+0000 7fdc10720b80 1 -- 172.21.15.46:0/3223620787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902ce90 msgr2=0x55bde9038f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.639+0000 7fdc10720b80 1 --2- 172.21.15.46:0/3223620787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902ce90 0x55bde9038f50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdbf809a6e0 tx=0x7fdbf8003680 comp rx=0 tx=0).stop 2024-03-18T21:00:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc10720b80 1 -- 172.21.15.46:0/3223620787 shutdown_connections 2024-03-18T21:00:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc10720b80 1 --2- 172.21.15.46:0/3223620787 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55bde90be310 0x55bde90c0730 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc10720b80 1 --2- 172.21.15.46:0/3223620787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bde902c570 0x55bde9038a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc10720b80 1 --2- 172.21.15.46:0/3223620787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bde9037db0 0x55bde9039490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc10720b80 1 --2- 172.21.15.46:0/3223620787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bde902ce90 0x55bde9038f50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc10720b80 1 -- 172.21.15.46:0/3223620787 >> 172.21.15.46:0/3223620787 conn(0x55bde8fa31c0 msgr2=0x55bde902ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdbe5ffb700 0 client.30553 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc10720b80 1 -- 172.21.15.46:0/3223620787 shutdown_connections 2024-03-18T21:00:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc10720b80 1 -- 172.21.15.46:0/3223620787 wait complete. 2024-03-18T21:00:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc10720b80 20 client.30553 trim_cache size 0 max 16384 2024-03-18T21:00:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T21:00:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T21:00:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T21:00:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T21:00:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc0458b700 1 -- 172.21.15.46:0/2205214759 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdbe80a1390 msgr2=0x7fdbe80a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc0458b700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdbe80a1390 0x7fdbe80a3850 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40175d0 tx=0x7fdbf401b000 comp rx=0 tx=0).stop 2024-03-18T21:00:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc0458b700 1 -- 172.21.15.46:0/2205214759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbfc10f940 msgr2=0x7fdbfc1a6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.640+0000 7fdc0458b700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbfc10f940 0x7fdbfc1a6600 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc10fd40 tx=0x7fdbf800f740 comp rx=0 tx=0).stop 2024-03-18T21:00:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.641+0000 7fdc0458b700 1 -- 172.21.15.46:0/2205214759 shutdown_connections 2024-03-18T21:00:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.641+0000 7fdc0458b700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fdbe80a1390 0x7fdbe80a3850 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.641+0000 7fdc0458b700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbfc11a590 0x7fdbfc1aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.641+0000 7fdc0458b700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbfc10f4d0 0x7fdbfc1a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.641+0000 7fdc0458b700 1 --2- 172.21.15.46:0/2205214759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbfc10f940 0x7fdbfc1a6600 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.641+0000 7fdc0458b700 1 -- 172.21.15.46:0/2205214759 >> 172.21.15.46:0/2205214759 conn(0x7fdbfc06bd80 msgr2=0x7fdbfc118580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.641+0000 7fdc0458b700 1 -- 172.21.15.46:0/2205214759 shutdown_connections 2024-03-18T21:00:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:12.641+0000 7fdc0458b700 1 -- 172.21.15.46:0/2205214759 wait complete. 2024-03-18T21:00:13.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 409044 30.53 MB/sec execute 529 sec latency 389.427 ms 2024-03-18T21:00:13.789 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 21:00:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: ignoring --setuser ceph since I am not root 2024-03-18T21:00:13.789 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 21:00:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: ignoring --setgroup ceph since I am not root 2024-03-18T21:00:13.789 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 21:00:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: starting mds.c at 2024-03-18T21:00:13.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:13 smithi155 ceph-mon[48344]: daemon mds.c restarted 2024-03-18T21:00:13.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:13 smithi155 ceph-mon[48344]: osdmap e119: 12 total, 12 up, 12 in 2024-03-18T21:00:13.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:13 smithi073 ceph-mon[47384]: daemon mds.c restarted 2024-03-18T21:00:13.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:13 smithi073 ceph-mon[47384]: osdmap e119: 12 total, 12 up, 12 in 2024-03-18T21:00:14.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:13 smithi046 ceph-mon[43495]: daemon mds.c restarted 2024-03-18T21:00:14.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:13 smithi046 ceph-mon[43495]: osdmap e119: 12 total, 12 up, 12 in 2024-03-18T21:00:14.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 409284 30.50 MB/sec execute 530 sec latency 171.748 ms 2024-03-18T21:00:14.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:14 smithi073 ceph-mon[47384]: pgmap v763: 97 pgs: 20 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 42 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 12 MiB/s wr, 476 op/s 2024-03-18T21:00:14.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:14 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6836/1436755958,v1:172.21.15.155:6837/1436755958] up:boot 2024-03-18T21:00:14.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:14 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T21:00:14.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:14 smithi073 ceph-mon[47384]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T21:00:14.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:14 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:15.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:14 smithi155 ceph-mon[48344]: pgmap v763: 97 pgs: 20 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 42 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 12 MiB/s wr, 476 op/s 2024-03-18T21:00:15.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:14 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6836/1436755958,v1:172.21.15.155:6837/1436755958] up:boot 2024-03-18T21:00:15.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:14 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T21:00:15.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:14 smithi155 ceph-mon[48344]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T21:00:15.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:14 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:15.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:14 smithi046 ceph-mon[43495]: pgmap v763: 97 pgs: 20 active+clean+snaptrim, 35 active+clean+snaptrim_wait, 42 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 12 MiB/s wr, 476 op/s 2024-03-18T21:00:15.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:14 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6836/1436755958,v1:172.21.15.155:6837/1436755958] up:boot 2024-03-18T21:00:15.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:14 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T21:00:15.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:14 smithi046 ceph-mon[43495]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T21:00:15.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:14 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:15.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 409877 30.51 MB/sec execute 531 sec latency 142.296 ms 2024-03-18T21:00:16.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 410535 30.54 MB/sec execute 532 sec latency 69.913 ms 2024-03-18T21:00:16.720 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:16 smithi073 ceph-mon[47384]: pgmap v765: 97 pgs: 6 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 74 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 24 MiB/s wr, 764 op/s 2024-03-18T21:00:16.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:16 smithi155 ceph-mon[48344]: pgmap v765: 97 pgs: 6 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 74 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 24 MiB/s wr, 764 op/s 2024-03-18T21:00:16.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:16 smithi046 ceph-mon[43495]: pgmap v765: 97 pgs: 6 active+clean+snaptrim, 17 active+clean+snaptrim_wait, 74 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 24 MiB/s wr, 764 op/s 2024-03-18T21:00:17.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 410968 30.53 MB/sec execute 533 sec latency 86.698 ms 2024-03-18T21:00:18.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 411315 30.50 MB/sec execute 534 sec latency 97.196 ms 2024-03-18T21:00:18.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:18 smithi155 ceph-mon[48344]: pgmap v766: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 22 MiB/s wr, 768 op/s 2024-03-18T21:00:18.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:18 smithi046 ceph-mon[43495]: pgmap v766: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 22 MiB/s wr, 768 op/s 2024-03-18T21:00:18.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:18 smithi073 ceph-mon[47384]: pgmap v766: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 22 MiB/s wr, 768 op/s 2024-03-18T21:00:19.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 411796 30.48 MB/sec execute 535 sec latency 136.429 ms 2024-03-18T21:00:19.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:19 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:19.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:19 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:19.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:19 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:20.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 412315 30.47 MB/sec execute 536 sec latency 80.087 ms 2024-03-18T21:00:20.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:20 smithi073 ceph-mon[47384]: pgmap v767: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 20 MiB/s wr, 782 op/s 2024-03-18T21:00:20.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:20 smithi073 ceph-mon[47384]: osdmap e120: 12 total, 12 up, 12 in 2024-03-18T21:00:21.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:20 smithi155 ceph-mon[48344]: pgmap v767: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 20 MiB/s wr, 782 op/s 2024-03-18T21:00:21.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:20 smithi155 ceph-mon[48344]: osdmap e120: 12 total, 12 up, 12 in 2024-03-18T21:00:21.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:20 smithi046 ceph-mon[43495]: pgmap v767: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 20 MiB/s wr, 782 op/s 2024-03-18T21:00:21.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:20 smithi046 ceph-mon[43495]: osdmap e120: 12 total, 12 up, 12 in 2024-03-18T21:00:21.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 412930 30.46 MB/sec execute 537 sec latency 114.280 ms 2024-03-18T21:00:22.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 413670 30.49 MB/sec execute 538 sec latency 91.573 ms 2024-03-18T21:00:22.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:22 smithi073 ceph-mon[47384]: pgmap v769: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 19 MiB/s wr, 842 op/s 2024-03-18T21:00:23.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:22 smithi155 ceph-mon[48344]: pgmap v769: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 19 MiB/s wr, 842 op/s 2024-03-18T21:00:23.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:22 smithi046 ceph-mon[43495]: pgmap v769: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 19 MiB/s wr, 842 op/s 2024-03-18T21:00:23.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 414221 30.51 MB/sec execute 539 sec latency 174.415 ms 2024-03-18T21:00:24.508 INFO:tasks.workunit.client.0.smithi046.stdout: 10 414565 30.49 MB/sec execute 540 sec latency 130.688 ms 2024-03-18T21:00:24.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:24 smithi073 ceph-mon[47384]: pgmap v770: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 718 op/s 2024-03-18T21:00:25.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:24 smithi155 ceph-mon[48344]: pgmap v770: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 718 op/s 2024-03-18T21:00:25.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:24 smithi046 ceph-mon[43495]: pgmap v770: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 718 op/s 2024-03-18T21:00:25.507 INFO:tasks.workunit.client.0.smithi046.stdout: 10 415015 30.46 MB/sec execute 541 sec latency 106.671 ms 2024-03-18T21:00:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.842+0000 7ff99e5a9700 1 -- 172.21.15.46:0/248647685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff998100590 msgr2=0x7ff998104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.842+0000 7ff99e5a9700 1 --2- 172.21.15.46:0/248647685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff998100590 0x7ff998104a50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff980009990 tx=0x7ff980009ca0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.843+0000 7ff99e5a9700 1 -- 172.21.15.46:0/248647685 shutdown_connections 2024-03-18T21:00:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.843+0000 7ff99e5a9700 1 --2- 172.21.15.46:0/248647685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff998105070 0x7ff998105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.843+0000 7ff99e5a9700 1 --2- 172.21.15.46:0/248647685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9980ffc70 0x7ff998100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.843+0000 7ff99e5a9700 1 --2- 172.21.15.46:0/248647685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff998100590 0x7ff998104a50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.843+0000 7ff99e5a9700 1 -- 172.21.15.46:0/248647685 >> 172.21.15.46:0/248647685 conn(0x7ff9980fb680 msgr2=0x7ff9980fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.854+0000 7ff99e5a9700 1 -- 172.21.15.46:0/248647685 shutdown_connections 2024-03-18T21:00:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.854+0000 7ff99e5a9700 1 -- 172.21.15.46:0/248647685 wait complete. 2024-03-18T21:00:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.855+0000 7ff99e5a9700 1 Processor -- start 2024-03-18T21:00:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.855+0000 7ff99e5a9700 1 -- start start 2024-03-18T21:00:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.855+0000 7ff99e5a9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9980ffc70 0x7ff998070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.855+0000 7ff99e5a9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff998100590 0x7ff998070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.855+0000 7ff99e5a9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff998105070 0x7ff99810a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.855+0000 7ff99e5a9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99810a770 con 0x7ff998100590 2024-03-18T21:00:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.855+0000 7ff99e5a9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99810a8e0 con 0x7ff998105070 2024-03-18T21:00:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.855+0000 7ff99e5a9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99810aaa0 con 0x7ff9980ffc70 2024-03-18T21:00:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.856+0000 7ff997fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9980ffc70 0x7ff998070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.856+0000 7ff997fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9980ffc70 0x7ff998070730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:38050/0 (socket says 172.21.15.46:38050) 2024-03-18T21:00:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.856+0000 7ff997fff700 1 -- 172.21.15.46:0/3492374252 learned_addr learned my addr 172.21.15.46:0/3492374252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.856+0000 7ff997fff700 1 -- 172.21.15.46:0/3492374252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff998105070 msgr2=0x7ff99810a1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-18T21:00:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.856+0000 7ff997fff700 1 --2- 172.21.15.46:0/3492374252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff998105070 0x7ff99810a1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.856+0000 7ff997fff700 1 -- 172.21.15.46:0/3492374252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff998100590 msgr2=0x7ff998070c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.856+0000 7ff997fff700 1 --2- 172.21.15.46:0/3492374252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff998100590 0x7ff998070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.856+0000 7ff997fff700 1 -- 172.21.15.46:0/3492374252 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800095f0 con 0x7ff9980ffc70 2024-03-18T21:00:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.856+0000 7ff997fff700 1 --2- 172.21.15.46:0/3492374252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9980ffc70 0x7ff998070730 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff98800b840 tx=0x7ff98800bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.860+0000 7ff9957fa700 1 -- 172.21.15.46:0/3492374252 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98800d890 con 0x7ff9980ffc70 2024-03-18T21:00:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.860+0000 7ff99e5a9700 1 -- 172.21.15.46:0/3492374252 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99810ad80 con 0x7ff9980ffc70 2024-03-18T21:00:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.860+0000 7ff9957fa700 1 -- 172.21.15.46:0/3492374252 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff98800e430 con 0x7ff9980ffc70 2024-03-18T21:00:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.860+0000 7ff9957fa700 1 -- 172.21.15.46:0/3492374252 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988017660 con 0x7ff9980ffc70 2024-03-18T21:00:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.860+0000 7ff99e5a9700 1 -- 172.21.15.46:0/3492374252 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff998112630 con 0x7ff9980ffc70 2024-03-18T21:00:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.864+0000 7ff9aa73eb80 1 Processor -- start 2024-03-18T21:00:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.865+0000 7ff9aa73eb80 1 -- start start 2024-03-18T21:00:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.865+0000 7ff9aa73eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 0x55913b97f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.865+0000 7ff9aa73eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97fe90 0x55913b98a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.865+0000 7ff9aa73eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b98adb0 0x55913b98d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.865+0000 7ff9aa73eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913b8d9930 con 0x55913b98adb0 2024-03-18T21:00:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.865+0000 7ff9aa73eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913b8d72f0 con 0x55913b97f570 2024-03-18T21:00:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.865+0000 7ff9aa73eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913b8d2be0 con 0x55913b97fe90 2024-03-18T21:00:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.866+0000 7ff99cb46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 0x55913b97f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.866+0000 7ff99cb46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 0x55913b97f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:43360/0 (socket says 172.21.15.46:43360) 2024-03-18T21:00:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.866+0000 7ff99cb46700 1 -- 172.21.15.46:0/295327744 learned_addr learned my addr 172.21.15.46:0/295327744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.867+0000 7ff9957fa700 1 -- 172.21.15.46:0/3492374252 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff9880203f0 con 0x7ff9980ffc70 2024-03-18T21:00:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.867+0000 7ff9977fe700 1 --2- 172.21.15.46:0/295327744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b98adb0 0x55913b98d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.867+0000 7ff9957fa700 1 --2- 172.21.15.46:0/3492374252 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9840a1390 0x7ff9840a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.868+0000 7ff997fff700 1 --2- 172.21.15.46:0/295327744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97fe90 0x55913b98a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.868+0000 7ff99cb46700 1 -- 172.21.15.46:0/295327744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97fe90 msgr2=0x55913b98a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.868+0000 7ff99cb46700 1 --2- 172.21.15.46:0/295327744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97fe90 0x55913b98a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.868+0000 7ff99cb46700 1 -- 172.21.15.46:0/295327744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b98adb0 msgr2=0x55913b98d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.868+0000 7ff99cb46700 1 --2- 172.21.15.46:0/295327744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b98adb0 0x55913b98d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.868+0000 7ff9957fa700 1 -- 172.21.15.46:0/3492374252 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff9880cd830 con 0x7ff9980ffc70 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.869+0000 7ff99cb46700 1 -- 172.21.15.46:0/295327744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55913b754350 con 0x55913b97f570 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.869+0000 7ff99cb46700 1 --2- 172.21.15.46:0/295327744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 0x55913b97f950 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff99002b830 tx=0x7ff99002bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32ca3474274cc6e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.869+0000 7ff99cb46700 1 --2- 172.21.15.46:0/3492374252 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9840a1390 0x7ff9840a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.869+0000 7ff98e7fc700 1 -- 172.21.15.46:0/295327744 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99002d780 con 0x55913b97f570 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.869+0000 7ff98e7fc700 1 -- 172.21.15.46:0/295327744 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff990033430 con 0x55913b97f570 2024-03-18T21:00:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.869+0000 7ff98e7fc700 1 -- 172.21.15.46:0/295327744 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99003c660 con 0x55913b97f570 2024-03-18T21:00:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.870+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/295327744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 msgr2=0x55913b97f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.870+0000 7ff99cb46700 1 --2- 172.21.15.46:0/3492374252 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9840a1390 0x7ff9840a3850 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff99002b040 tx=0x7ff99003cc50 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.870+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/295327744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 0x55913b97f950 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff99002b830 tx=0x7ff99002bb40 comp rx=0 tx=0).stop 2024-03-18T21:00:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.870+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/295327744 shutdown_connections 2024-03-18T21:00:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.870+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/295327744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b98adb0 0x55913b98d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.870+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/295327744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 0x55913b97f950 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.870+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/295327744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97fe90 0x55913b98a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.870+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/295327744 >> 172.21.15.46:0/295327744 conn(0x55913b8f61c0 msgr2=0x55913b97ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.871+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/295327744 shutdown_connections 2024-03-18T21:00:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.871+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/295327744 wait complete. 2024-03-18T21:00:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.871+0000 7ff9aa73eb80 1 -- start start 2024-03-18T21:00:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.872+0000 7ff9aa73eb80 1 Processor -- start 2024-03-18T21:00:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.872+0000 7ff9aa73eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 0x55913b98a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.872+0000 7ff9aa73eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b97fe90 0x55913b987a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.872+0000 7ff9aa73eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b98adb0 0x55913b987fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.872+0000 7ff9aa73eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913b86ed20 con 0x55913b97fe90 2024-03-18T21:00:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.872+0000 7ff9aa73eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913b86e9c0 con 0x55913b97f570 2024-03-18T21:00:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.872+0000 7ff9aa73eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913b756640 con 0x55913b98adb0 2024-03-18T21:00:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.872+0000 7ff9aa73eb80 10 client.0 _ll_register_callbacks cb 0x55913b6da340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T21:00:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.872+0000 7ff997fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 0x55913b98a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.872+0000 7ff997fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 0x55913b98a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:43368/0 (socket says 172.21.15.46:43368) 2024-03-18T21:00:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.872+0000 7ff997fff700 1 -- 172.21.15.46:0/1293212654 learned_addr learned my addr 172.21.15.46:0/1293212654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.874+0000 7ff997fff700 1 -- 172.21.15.46:0/1293212654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b98adb0 msgr2=0x55913b987fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T21:00:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.874+0000 7ff997fff700 1 --2- 172.21.15.46:0/1293212654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b98adb0 0x55913b987fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.874+0000 7ff997fff700 1 -- 172.21.15.46:0/1293212654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b97fe90 msgr2=0x55913b987a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T21:00:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.874+0000 7ff997fff700 1 --2- 172.21.15.46:0/1293212654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b97fe90 0x55913b987a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.874+0000 7ff997fff700 1 -- 172.21.15.46:0/1293212654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99002b460 con 0x55913b97f570 2024-03-18T21:00:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.875+0000 7ff997fff700 1 --2- 172.21.15.46:0/1293212654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 0x55913b98a710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff988096dc0 tx=0x7ff98809a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47e250e216d057a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.875+0000 7ff9777fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T21:00:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.875+0000 7ff9777fe700 1 -- 172.21.15.46:0/1293212654 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988016560 con 0x55913b97f570 2024-03-18T21:00:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.875+0000 7ff9777fe700 1 -- 172.21.15.46:0/1293212654 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff9880a2d40 con 0x55913b97f570 2024-03-18T21:00:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.875+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/1293212654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55913b9736c0 con 0x55913b97f570 2024-03-18T21:00:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.876+0000 7ff9777fe700 1 -- 172.21.15.46:0/1293212654 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9880b30f0 con 0x55913b97f570 2024-03-18T21:00:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.876+0000 7ff9aa73eb80 10 client.30419 fetch_fsmap learned FSMap version 101 2024-03-18T21:00:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.876+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/1293212654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) v3 -- 0x55913b700b00 con 0x55913b97f570 2024-03-18T21:00:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.876+0000 7ff9777fe700 1 -- 172.21.15.46:0/1293212654 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55913b9736c0 con 0x55913b97f570 2024-03-18T21:00:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.876+0000 7ff9777fe700 1 -- 172.21.15.46:0/1293212654 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 101) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff9880ba0e0 con 0x55913b97f570 2024-03-18T21:00:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.876+0000 7ff9aa73eb80 10 client.30419 fetch_fsmap finished waiting for FSMap version 101 2024-03-18T21:00:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.876+0000 7ff9aa73eb80 10 client.30419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T21:00:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.876+0000 7ff9aa73eb80 20 client.30419 populate_metadata read hostname 'smithi046' 2024-03-18T21:00:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.876+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/1293212654 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55913b99aa70 0x55913b99ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.876+0000 7ff9aa73eb80 4 client.30419 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T21:00:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.876+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/1293212654 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55913b8cb6a0 con 0x55913b99aa70 2024-03-18T21:00:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.877+0000 7ff9977fe700 1 --2- 172.21.15.46:0/1293212654 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55913b99aa70 0x55913b99ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.877+0000 7ff9977fe700 1 --2- 172.21.15.46:0/1293212654 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55913b99aa70 0x55913b99ced0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1898dfa4d6a26f94 server_cookie=3d4c1eceabc59703 in_seq=0 out_seq=0 2024-03-18T21:00:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.877+0000 7ff9777fe700 10 client.30419 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.882+0000 7ff9777fe700 1 -- 172.21.15.46:0/1293212654 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55913b8cb6a0 con 0x55913b99aa70 2024-03-18T21:00:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.882+0000 7ff9777fe700 10 client.30419 handle_command_reply: tid=0 2024-03-18T21:00:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.882+0000 7ff9aa73eb80 1 client.30419 shutdown 2024-03-18T21:00:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.882+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/1293212654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 msgr2=0x55913b98a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.882+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/1293212654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 0x55913b98a710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff988096dc0 tx=0x7ff98809a8c0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.882+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/1293212654 shutdown_connections 2024-03-18T21:00:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.882+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/1293212654 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55913b99aa70 0x55913b99ced0 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.882+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/1293212654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b98adb0 0x55913b987fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.882+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/1293212654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97f570 0x55913b98a710 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.882+0000 7ff9777fe700 0 client.30419 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.882+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/1293212654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b97fe90 0x55913b987a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.882+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/1293212654 >> 172.21.15.46:0/1293212654 conn(0x55913b8f61c0 msgr2=0x55913b97b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.883+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/1293212654 shutdown_connections 2024-03-18T21:00:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.883+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/1293212654 wait complete. 2024-03-18T21:00:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.883+0000 7ff9aa73eb80 20 client.30419 trim_cache size 0 max 16384 2024-03-18T21:00:25.918 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.913+0000 7ff9aa73eb80 1 Processor -- start 2024-03-18T21:00:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.916+0000 7ff9aa73eb80 1 -- start start 2024-03-18T21:00:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.916+0000 7ff9aa73eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97f570 0x55913b9879c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.916+0000 7ff9aa73eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97fe90 0x55913b991ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.916+0000 7ff9aa73eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b98adb0 0x55913b97a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.916+0000 7ff9aa73eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913b992010 con 0x55913b98adb0 2024-03-18T21:00:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.916+0000 7ff9aa73eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913b988320 con 0x55913b97fe90 2024-03-18T21:00:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.916+0000 7ff9aa73eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913b97a660 con 0x55913b97f570 2024-03-18T21:00:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.917+0000 7ff997fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97f570 0x55913b9879c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.917+0000 7ff997fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97f570 0x55913b9879c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:38074/0 (socket says 172.21.15.46:38074) 2024-03-18T21:00:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.917+0000 7ff997fff700 1 -- 172.21.15.46:0/4174928159 learned_addr learned my addr 172.21.15.46:0/4174928159 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.917+0000 7ff9977fe700 1 --2- 172.21.15.46:0/4174928159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97fe90 0x55913b991ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.920 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.917+0000 7ff997fff700 1 -- 172.21.15.46:0/4174928159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97fe90 msgr2=0x55913b991ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.917+0000 7ff997fff700 1 --2- 172.21.15.46:0/4174928159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97fe90 0x55913b991ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.917+0000 7ff997fff700 1 -- 172.21.15.46:0/4174928159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b98adb0 msgr2=0x55913b97a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.917+0000 7ff997fff700 1 --2- 172.21.15.46:0/4174928159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b98adb0 0x55913b97a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.917+0000 7ff997fff700 1 -- 172.21.15.46:0/4174928159 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55913b898e40 con 0x55913b97f570 2024-03-18T21:00:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.917+0000 7ff997fff700 1 --2- 172.21.15.46:0/4174928159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97f570 0x55913b9879c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff98809e010 tx=0x7ff988009a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac24f50dac177c2b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.918+0000 7ff98effd700 1 -- 172.21.15.46:0/4174928159 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880a28b0 con 0x55913b97f570 2024-03-18T21:00:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.918+0000 7ff98effd700 1 -- 172.21.15.46:0/4174928159 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff98809a690 con 0x55913b97f570 2024-03-18T21:00:25.921 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.918+0000 7ff98effd700 1 -- 172.21.15.46:0/4174928159 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98800d040 con 0x55913b97f570 2024-03-18T21:00:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.919+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/4174928159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97f570 msgr2=0x55913b9879c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.919+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/4174928159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97f570 0x55913b9879c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff98809e010 tx=0x7ff988009a40 comp rx=0 tx=0).stop 2024-03-18T21:00:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.919+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/4174928159 shutdown_connections 2024-03-18T21:00:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.919+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/4174928159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b98adb0 0x55913b97a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.919+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/4174928159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97f570 0x55913b9879c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.919+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/4174928159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b97fe90 0x55913b991ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.919+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/4174928159 >> 172.21.15.46:0/4174928159 conn(0x55913b8f61c0 msgr2=0x55913b9a1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.924+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/4174928159 shutdown_connections 2024-03-18T21:00:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.924+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/4174928159 wait complete. 2024-03-18T21:00:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.925+0000 7ff9aa73eb80 1 -- start start 2024-03-18T21:00:25.927 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.925+0000 7ff9aa73eb80 1 Processor -- start 2024-03-18T21:00:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.925+0000 7ff9aa73eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b97f570 0x55913b98daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.925+0000 7ff9aa73eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97fe90 0x55913b98dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.925+0000 7ff9aa73eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b98adb0 0x55913b98e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.925+0000 7ff9aa73eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913b9ed910 con 0x55913b97f570 2024-03-18T21:00:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.925+0000 7ff9aa73eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913b9eda50 con 0x55913b98adb0 2024-03-18T21:00:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.925+0000 7ff9aa73eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913b9edb90 con 0x55913b97fe90 2024-03-18T21:00:25.928 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.925+0000 7ff9aa73eb80 10 client.0 _ll_register_callbacks cb 0x55913b6da340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T21:00:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.926+0000 7ff9977fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97fe90 0x55913b98dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.926+0000 7ff9977fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97fe90 0x55913b98dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:38080/0 (socket says 172.21.15.46:38080) 2024-03-18T21:00:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.926+0000 7ff9977fe700 1 -- 172.21.15.46:0/3650487326 learned_addr learned my addr 172.21.15.46:0/3650487326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.926+0000 7ff997fff700 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b97f570 0x55913b98daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.927+0000 7ff99cb46700 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b98adb0 0x55913b98e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.927+0000 7ff9977fe700 1 -- 172.21.15.46:0/3650487326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b98adb0 msgr2=0x55913b98e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.927+0000 7ff9977fe700 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b98adb0 0x55913b98e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.927+0000 7ff9977fe700 1 -- 172.21.15.46:0/3650487326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b97f570 msgr2=0x55913b98daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.927+0000 7ff9977fe700 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b97f570 0x55913b98daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.927+0000 7ff9977fe700 1 -- 172.21.15.46:0/3650487326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55913b700b00 con 0x55913b97fe90 2024-03-18T21:00:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.928+0000 7ff9977fe700 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97fe90 0x55913b98dfe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff998107150 tx=0x7ff980005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43256a6b0e46cf67 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.928+0000 7ff9777fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T21:00:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.928+0000 7ff9777fe700 1 -- 172.21.15.46:0/3650487326 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98001d040 con 0x55913b97fe90 2024-03-18T21:00:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.928+0000 7ff9777fe700 1 -- 172.21.15.46:0/3650487326 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8556+0+0 (secure 0 0 0) 0x7ff980024b80 con 0x55913b97fe90 2024-03-18T21:00:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.928+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/3650487326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff990033430 con 0x55913b97fe90 2024-03-18T21:00:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.929+0000 7ff9777fe700 1 -- 172.21.15.46:0/3650487326 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff98000bea0 con 0x55913b97fe90 2024-03-18T21:00:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.932+0000 7ff9aa73eb80 10 client.30571 fetch_fsmap learned FSMap version 101 2024-03-18T21:00:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.932+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/3650487326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=4+}) v3 -- 0x55913b758c70 con 0x55913b97fe90 2024-03-18T21:00:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.932+0000 7ff9777fe700 1 -- 172.21.15.46:0/3650487326 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff990033430 con 0x55913b97fe90 2024-03-18T21:00:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.932+0000 7ff9777fe700 1 -- 172.21.15.46:0/3650487326 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 101) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff980005190 con 0x55913b97fe90 2024-03-18T21:00:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.932+0000 7ff9aa73eb80 10 client.30571 fetch_fsmap finished waiting for FSMap version 101 2024-03-18T21:00:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.932+0000 7ff9aa73eb80 10 client.30571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T21:00:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.932+0000 7ff9aa73eb80 20 client.30571 populate_metadata read hostname 'smithi046' 2024-03-18T21:00:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.933+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55913b9ef2f0 0x55913b9f1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.933+0000 7ff9aa73eb80 4 client.30571 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T21:00:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.933+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/3650487326 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55913b721c90 con 0x55913b9ef2f0 2024-03-18T21:00:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.933+0000 7ff997fff700 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55913b9ef2f0 0x55913b9f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.933+0000 7ff997fff700 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55913b9ef2f0 0x55913b9f1710 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d8eed3caeb72a9b server_cookie=912694ff3367ef24 in_seq=0 out_seq=0 2024-03-18T21:00:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:25.933+0000 7ff9777fe700 10 client.30571 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:26.511 INFO:tasks.workunit.client.0.smithi046.stdout: 10 415206 30.41 MB/sec execute 542 sec latency 572.333 ms 2024-03-18T21:00:26.726 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:26 smithi073 ceph-mon[47384]: pgmap v771: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.9 MiB/s wr, 590 op/s 2024-03-18T21:00:26.792 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:26 smithi155 ceph-mon[48344]: pgmap v771: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.9 MiB/s wr, 590 op/s 2024-03-18T21:00:26.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:26 smithi046 ceph-mon[43495]: pgmap v771: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.9 MiB/s wr, 590 op/s 2024-03-18T21:00:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.963+0000 7ff9777fe700 1 -- 172.21.15.46:0/3650487326 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55913b721c90 con 0x55913b9ef2f0 2024-03-18T21:00:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.963+0000 7ff9777fe700 10 client.30571 handle_command_reply: tid=0 2024-03-18T21:00:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.963+0000 7ff9aa73eb80 1 client.30571 shutdown 2024-03-18T21:00:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.964+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/3650487326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97fe90 msgr2=0x55913b98dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.964+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97fe90 0x55913b98dfe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff998107150 tx=0x7ff980005ab0 comp rx=0 tx=0).stop 2024-03-18T21:00:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.966+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/3650487326 shutdown_connections 2024-03-18T21:00:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.966+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55913b9ef2f0 0x55913b9f1710 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.966+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55913b97f570 0x55913b98daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.966+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55913b98adb0 0x55913b98e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.966+0000 7ff9aa73eb80 1 --2- 172.21.15.46:0/3650487326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55913b97fe90 0x55913b98dfe0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.966+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/3650487326 >> 172.21.15.46:0/3650487326 conn(0x55913b8f61c0 msgr2=0x55913b97b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.966+0000 7ff9777fe700 0 client.30571 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.966+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/3650487326 shutdown_connections 2024-03-18T21:00:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.966+0000 7ff9aa73eb80 1 -- 172.21.15.46:0/3650487326 wait complete. 2024-03-18T21:00:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.966+0000 7ff9aa73eb80 20 client.30571 trim_cache size 0 max 16384 2024-03-18T21:00:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T21:00:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T21:00:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T21:00:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T21:00:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.967+0000 7ff99e5a9700 1 -- 172.21.15.46:0/3492374252 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9840a1390 msgr2=0x7ff9840a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.967+0000 7ff99e5a9700 1 --2- 172.21.15.46:0/3492374252 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9840a1390 0x7ff9840a3850 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff99002b040 tx=0x7ff99003cc50 comp rx=0 tx=0).stop 2024-03-18T21:00:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.967+0000 7ff99e5a9700 1 -- 172.21.15.46:0/3492374252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9980ffc70 msgr2=0x7ff998070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.967+0000 7ff99e5a9700 1 --2- 172.21.15.46:0/3492374252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9980ffc70 0x7ff998070730 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff98800b840 tx=0x7ff98800bc00 comp rx=0 tx=0).stop 2024-03-18T21:00:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.967+0000 7ff99e5a9700 1 -- 172.21.15.46:0/3492374252 shutdown_connections 2024-03-18T21:00:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.967+0000 7ff99e5a9700 1 --2- 172.21.15.46:0/3492374252 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7ff9840a1390 0x7ff9840a3850 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.967+0000 7ff99e5a9700 1 --2- 172.21.15.46:0/3492374252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff998105070 0x7ff99810a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.967+0000 7ff99e5a9700 1 --2- 172.21.15.46:0/3492374252 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9980ffc70 0x7ff998070730 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.967+0000 7ff99e5a9700 1 --2- 172.21.15.46:0/3492374252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff998100590 0x7ff998070c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.967+0000 7ff99e5a9700 1 -- 172.21.15.46:0/3492374252 >> 172.21.15.46:0/3492374252 conn(0x7ff9980fb680 msgr2=0x7ff9980fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.969+0000 7ff99e5a9700 1 -- 172.21.15.46:0/3492374252 shutdown_connections 2024-03-18T21:00:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:26.969+0000 7ff99e5a9700 1 -- 172.21.15.46:0/3492374252 wait complete. 2024-03-18T21:00:27.511 INFO:tasks.workunit.client.0.smithi046.stdout: 10 415523 30.39 MB/sec execute 543 sec latency 828.727 ms 2024-03-18T21:00:28.511 INFO:tasks.workunit.client.0.smithi046.stdout: 10 415930 30.39 MB/sec execute 544 sec latency 184.928 ms 2024-03-18T21:00:28.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:28 smithi046 ceph-mon[43495]: pgmap v772: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.3 MiB/s wr, 530 op/s 2024-03-18T21:00:28.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:28 smithi073 ceph-mon[47384]: pgmap v772: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.3 MiB/s wr, 530 op/s 2024-03-18T21:00:29.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:28 smithi155 ceph-mon[48344]: pgmap v772: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 9.3 MiB/s wr, 530 op/s 2024-03-18T21:00:29.511 INFO:tasks.workunit.client.0.smithi046.stdout: 10 416407 30.36 MB/sec execute 545 sec latency 109.421 ms 2024-03-18T21:00:29.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:29 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:30.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:29 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:30.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:29 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:30.511 INFO:tasks.workunit.client.0.smithi046.stdout: 10 417029 30.37 MB/sec execute 546 sec latency 135.576 ms 2024-03-18T21:00:30.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:30 smithi073 ceph-mon[47384]: pgmap v773: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 13 MiB/s wr, 603 op/s 2024-03-18T21:00:31.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:30 smithi155 ceph-mon[48344]: pgmap v773: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 13 MiB/s wr, 603 op/s 2024-03-18T21:00:31.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:30 smithi046 ceph-mon[43495]: pgmap v773: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 13 MiB/s wr, 603 op/s 2024-03-18T21:00:31.511 INFO:tasks.workunit.client.0.smithi046.stdout: 10 417660 30.40 MB/sec execute 547 sec latency 104.420 ms 2024-03-18T21:00:32.511 INFO:tasks.workunit.client.0.smithi046.stdout: 10 418158 30.42 MB/sec execute 548 sec latency 95.199 ms 2024-03-18T21:00:32.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:32 smithi073 ceph-mon[47384]: pgmap v774: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 572 op/s 2024-03-18T21:00:33.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:32 smithi155 ceph-mon[48344]: pgmap v774: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 572 op/s 2024-03-18T21:00:33.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:32 smithi046 ceph-mon[43495]: pgmap v774: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 572 op/s 2024-03-18T21:00:33.511 INFO:tasks.workunit.client.0.smithi046.stdout: 10 418511 30.40 MB/sec execute 549 sec latency 156.581 ms 2024-03-18T21:00:34.511 INFO:tasks.workunit.client.0.smithi046.stdout: 10 418874 30.38 MB/sec execute 550 sec latency 112.370 ms 2024-03-18T21:00:34.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:34 smithi073 ceph-mon[47384]: pgmap v775: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 604 op/s 2024-03-18T21:00:35.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:34 smithi155 ceph-mon[48344]: pgmap v775: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 604 op/s 2024-03-18T21:00:35.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:34 smithi046 ceph-mon[43495]: pgmap v775: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 604 op/s 2024-03-18T21:00:35.511 INFO:tasks.workunit.client.0.smithi046.stdout: 10 419282 30.35 MB/sec execute 551 sec latency 140.588 ms 2024-03-18T21:00:36.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 419846 30.35 MB/sec execute 552 sec latency 87.309 ms 2024-03-18T21:00:36.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:36 smithi073 ceph-mon[47384]: pgmap v776: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-03-18T21:00:36.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:36 smithi155 ceph-mon[48344]: pgmap v776: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-03-18T21:00:36.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:36 smithi046 ceph-mon[43495]: pgmap v776: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-03-18T21:00:37.511 INFO:tasks.workunit.client.0.smithi046.stdout: 10 420455 30.36 MB/sec execute 553 sec latency 76.121 ms 2024-03-18T21:00:37.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:37 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:37.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:37 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:37.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:37 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:38.511 INFO:tasks.workunit.client.0.smithi046.stdout: 10 421056 30.37 MB/sec execute 554 sec latency 155.718 ms 2024-03-18T21:00:38.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:38 smithi073 ceph-mon[47384]: pgmap v777: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 584 op/s 2024-03-18T21:00:38.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:38 smithi155 ceph-mon[48344]: pgmap v777: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 584 op/s 2024-03-18T21:00:38.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:38 smithi046 ceph-mon[43495]: pgmap v777: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 584 op/s 2024-03-18T21:00:39.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 421553 30.37 MB/sec execute 555 sec latency 104.435 ms 2024-03-18T21:00:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.179+0000 7f904fb3b700 1 -- 172.21.15.46:0/24915638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90480696a0 msgr2=0x7f9048069a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.179+0000 7f904fb3b700 1 --2- 172.21.15.46:0/24915638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90480696a0 0x7f9048069a80 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f9044009990 tx=0x7f9044009ca0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.180+0000 7f904fb3b700 1 -- 172.21.15.46:0/24915638 shutdown_connections 2024-03-18T21:00:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.180+0000 7f904fb3b700 1 --2- 172.21.15.46:0/24915638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90480696a0 0x7f9048069a80 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.180+0000 7f904fb3b700 1 --2- 172.21.15.46:0/24915638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9048106040 0x7f904810dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.180+0000 7f904fb3b700 1 --2- 172.21.15.46:0/24915638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f904806a050 0x7f9048105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.180+0000 7f904fb3b700 1 -- 172.21.15.46:0/24915638 >> 172.21.15.46:0/24915638 conn(0x7f9048076a80 msgr2=0x7f9048076e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.180+0000 7f904fb3b700 1 -- 172.21.15.46:0/24915638 shutdown_connections 2024-03-18T21:00:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.180+0000 7f904fb3b700 1 -- 172.21.15.46:0/24915638 wait complete. 2024-03-18T21:00:40.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.181+0000 7f904fb3b700 1 Processor -- start 2024-03-18T21:00:40.185 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.181+0000 7f904fb3b700 1 -- start start 2024-03-18T21:00:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.181+0000 7f904fb3b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90480696a0 0x7f90480fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.181+0000 7f904fb3b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9048106040 0x7f90481004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.181+0000 7f904fb3b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90481009e0 0x7f90481a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.181+0000 7f904fb3b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90481a5990 con 0x7f90480696a0 2024-03-18T21:00:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.181+0000 7f904fb3b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90481a5b00 con 0x7f90481009e0 2024-03-18T21:00:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.181+0000 7f904fb3b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90481a5cc0 con 0x7f9048106040 2024-03-18T21:00:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.182+0000 7f904d8d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90480696a0 0x7f90480fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.182+0000 7f904d8d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90480696a0 0x7f90480fff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59604/0 (socket says 172.21.15.46:59604) 2024-03-18T21:00:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.182+0000 7f904d8d7700 1 -- 172.21.15.46:0/1982298123 learned_addr learned my addr 172.21.15.46:0/1982298123 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.182+0000 7f904e0d8700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90481009e0 0x7f90481a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.182+0000 7f904e0d8700 1 -- 172.21.15.46:0/1982298123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9048106040 msgr2=0x7f90481004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.182+0000 7f904e0d8700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9048106040 0x7f90481004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.182+0000 7f904e0d8700 1 -- 172.21.15.46:0/1982298123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90480696a0 msgr2=0x7f90480fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.182+0000 7f904e0d8700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90480696a0 0x7f90480fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.182+0000 7f904e0d8700 1 -- 172.21.15.46:0/1982298123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040009c40 con 0x7f90481009e0 2024-03-18T21:00:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.182+0000 7f904e0d8700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90481009e0 0x7f90481a53c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f904000efc0 tx=0x7f904000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.183+0000 7f903affd700 1 -- 172.21.15.46:0/1982298123 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040018070 con 0x7f90481009e0 2024-03-18T21:00:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.183+0000 7f903affd700 1 -- 172.21.15.46:0/1982298123 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f904000f040 con 0x7f90481009e0 2024-03-18T21:00:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.183+0000 7f904fb3b700 1 -- 172.21.15.46:0/1982298123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90440095f0 con 0x7f90481009e0 2024-03-18T21:00:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.183+0000 7f904fb3b700 1 -- 172.21.15.46:0/1982298123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90481a62c0 con 0x7f90481009e0 2024-03-18T21:00:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.183+0000 7f903affd700 1 -- 172.21.15.46:0/1982298123 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040004260 con 0x7f90481009e0 2024-03-18T21:00:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.187+0000 7f903affd700 1 -- 172.21.15.46:0/1982298123 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9040003a00 con 0x7f90481009e0 2024-03-18T21:00:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.188+0000 7f903affd700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90340a1390 0x7f90340a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.188+0000 7f903affd700 1 -- 172.21.15.46:0/1982298123 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f90400cc5b0 con 0x7f90481009e0 2024-03-18T21:00:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.189+0000 7f904d8d7700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90340a1390 0x7f90340a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.189+0000 7f905bcd0b80 1 Processor -- start 2024-03-18T21:00:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.189+0000 7f905bcd0b80 1 -- start start 2024-03-18T21:00:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f904d8d7700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90340a1390 0x7f90340a3850 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9044006010 tx=0x7f904400b260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f905bcd0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0ccfc570 0x55ae0ccfc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f905bcd0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 0x55ae0cd07870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f905bcd0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0cd07db0 0x55ae0cd0a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f905bcd0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae0cc56930 con 0x55ae0ccfce90 2024-03-18T21:00:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f905bcd0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae0cc542f0 con 0x55ae0cd07db0 2024-03-18T21:00:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f905bcd0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae0cc4fbe0 con 0x55ae0ccfc570 2024-03-18T21:00:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f904d0d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 0x55ae0cd07870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f904d8d7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0ccfc570 0x55ae0ccfc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.195 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f904d0d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 0x55ae0cd07870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59620/0 (socket says 172.21.15.46:59620) 2024-03-18T21:00:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f904d0d6700 1 -- 172.21.15.46:0/1351711470 learned_addr learned my addr 172.21.15.46:0/1351711470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f904e0d8700 1 --2- 172.21.15.46:0/1351711470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0cd07db0 0x55ae0cd0a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f904d0d6700 1 -- 172.21.15.46:0/1351711470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0ccfc570 msgr2=0x55ae0ccfc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.190+0000 7f904d0d6700 1 --2- 172.21.15.46:0/1351711470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0ccfc570 0x55ae0ccfc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.191+0000 7f904d0d6700 1 -- 172.21.15.46:0/1351711470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0cd07db0 msgr2=0x55ae0cd0a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.191+0000 7f904d0d6700 1 --2- 172.21.15.46:0/1351711470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0cd07db0 0x55ae0cd0a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.191+0000 7f904d0d6700 1 -- 172.21.15.46:0/1351711470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae0cad1350 con 0x55ae0ccfce90 2024-03-18T21:00:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.191+0000 7f904d0d6700 1 --2- 172.21.15.46:0/1351711470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 0x55ae0cd07870 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f903c009c40 tx=0x7f903c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616c530197cb6f1c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.191+0000 7f9033fff700 1 -- 172.21.15.46:0/1351711470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c0097c0 con 0x55ae0ccfce90 2024-03-18T21:00:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.191+0000 7f9033fff700 1 -- 172.21.15.46:0/1351711470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f903c004830 con 0x55ae0ccfce90 2024-03-18T21:00:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.191+0000 7f9033fff700 1 -- 172.21.15.46:0/1351711470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c005650 con 0x55ae0ccfce90 2024-03-18T21:00:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.192+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1351711470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 msgr2=0x55ae0cd07870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.192+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/1351711470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 0x55ae0cd07870 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f903c009c40 tx=0x7f903c009f50 comp rx=0 tx=0).stop 2024-03-18T21:00:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.192+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1351711470 shutdown_connections 2024-03-18T21:00:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.192+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/1351711470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0cd07db0 0x55ae0cd0a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.192+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/1351711470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0ccfc570 0x55ae0ccfc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.192+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/1351711470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 0x55ae0cd07870 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.192+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1351711470 >> 172.21.15.46:0/1351711470 conn(0x55ae0cc731c0 msgr2=0x55ae0ccf8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.192+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1351711470 shutdown_connections 2024-03-18T21:00:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.192+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1351711470 wait complete. 2024-03-18T21:00:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.193+0000 7f905bcd0b80 1 -- start start 2024-03-18T21:00:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.193+0000 7f905bcd0b80 1 Processor -- start 2024-03-18T21:00:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.193+0000 7f905bcd0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0ccfc570 0x55ae0cd07730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.193+0000 7f905bcd0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 0x55ae0cd04ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.193+0000 7f905bcd0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0cd07db0 0x55ae0cd050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.193+0000 7f905bcd0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae0cbebd20 con 0x55ae0ccfce90 2024-03-18T21:00:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.193+0000 7f905bcd0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae0cbeb9c0 con 0x55ae0ccfc570 2024-03-18T21:00:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.193+0000 7f905bcd0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae0cad3640 con 0x55ae0cd07db0 2024-03-18T21:00:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.193+0000 7f905bcd0b80 10 client.0 _ll_register_callbacks cb 0x55ae0ca57340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T21:00:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.193+0000 7f904d0d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 0x55ae0cd04ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.193+0000 7f904d0d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 0x55ae0cd04ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59624/0 (socket says 172.21.15.46:59624) 2024-03-18T21:00:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.194+0000 7f904d0d6700 1 -- 172.21.15.46:0/2776398609 learned_addr learned my addr 172.21.15.46:0/2776398609 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.194+0000 7f904d8d7700 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0ccfc570 0x55ae0cd07730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.194+0000 7f904e0d8700 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0cd07db0 0x55ae0cd050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.194+0000 7f904d0d6700 1 -- 172.21.15.46:0/2776398609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0cd07db0 msgr2=0x55ae0cd050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.194+0000 7f904d0d6700 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0cd07db0 0x55ae0cd050f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.194+0000 7f904d0d6700 1 -- 172.21.15.46:0/2776398609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0ccfc570 msgr2=0x55ae0cd07730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.194+0000 7f904d0d6700 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0ccfc570 0x55ae0cd07730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.194+0000 7f904d0d6700 1 -- 172.21.15.46:0/2776398609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903c009920 con 0x55ae0ccfce90 2024-03-18T21:00:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.194+0000 7f904d8d7700 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0ccfc570 0x55ae0cd07730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T21:00:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.194+0000 7f904d0d6700 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 0x55ae0cd04ac0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f903c005500 tx=0x7f903c0055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151c9d25beb0364b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.194+0000 7f90317fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T21:00:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.194+0000 7f90317fa700 1 -- 172.21.15.46:0/2776398609 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c0097c0 con 0x55ae0ccfce90 2024-03-18T21:00:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.195+0000 7f90317fa700 1 -- 172.21.15.46:0/2776398609 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f903c014070 con 0x55ae0ccfce90 2024-03-18T21:00:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.195+0000 7f905bcd0b80 1 -- 172.21.15.46:0/2776398609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae0ccf06c0 con 0x55ae0ccfce90 2024-03-18T21:00:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.195+0000 7f90317fa700 1 -- 172.21.15.46:0/2776398609 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f903c005020 con 0x55ae0ccfce90 2024-03-18T21:00:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.195+0000 7f905bcd0b80 10 client.20820 fetch_fsmap learned FSMap version 103 2024-03-18T21:00:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.195+0000 7f905bcd0b80 1 -- 172.21.15.46:0/2776398609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x55ae0ca7db00 con 0x55ae0ccfce90 2024-03-18T21:00:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.195+0000 7f90317fa700 1 -- 172.21.15.46:0/2776398609 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae0ccf06c0 con 0x55ae0ccfce90 2024-03-18T21:00:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.195+0000 7f90317fa700 1 -- 172.21.15.46:0/2776398609 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 103) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f903c018a80 con 0x55ae0ccfce90 2024-03-18T21:00:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.195+0000 7f905bcd0b80 10 client.20820 fetch_fsmap finished waiting for FSMap version 103 2024-03-18T21:00:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.195+0000 7f905bcd0b80 10 client.20820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T21:00:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.196+0000 7f905bcd0b80 20 client.20820 populate_metadata read hostname 'smithi046' 2024-03-18T21:00:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.196+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae0cd28810 0x55ae0cd2ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.196+0000 7f905bcd0b80 4 client.20820 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T21:00:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.196+0000 7f905bcd0b80 1 -- 172.21.15.46:0/2776398609 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae0cc486a0 con 0x55ae0cd28810 2024-03-18T21:00:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.196+0000 7f904d8d7700 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae0cd28810 0x55ae0cd2ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.196+0000 7f90317fa700 10 client.20820 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.196+0000 7f904d8d7700 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae0cd28810 0x55ae0cd2ac30 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f8cd3638f15a73 server_cookie=d848b6053a2cdc6f in_seq=0 out_seq=0 2024-03-18T21:00:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.197+0000 7f90317fa700 1 -- 172.21.15.46:0/2776398609 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ae0cc486a0 con 0x55ae0cd28810 2024-03-18T21:00:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.197+0000 7f90317fa700 10 client.20820 handle_command_reply: tid=0 2024-03-18T21:00:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.197+0000 7f905bcd0b80 1 client.20820 shutdown 2024-03-18T21:00:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.197+0000 7f905bcd0b80 1 -- 172.21.15.46:0/2776398609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 msgr2=0x55ae0cd04ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.197+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 0x55ae0cd04ac0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f903c005500 tx=0x7f903c0055e0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.197+0000 7f905bcd0b80 1 -- 172.21.15.46:0/2776398609 shutdown_connections 2024-03-18T21:00:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.198+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae0cd28810 0x55ae0cd2ac30 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.198+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0cd07db0 0x55ae0cd050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.198+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0ccfc570 0x55ae0cd07730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.198+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/2776398609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfce90 0x55ae0cd04ac0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.198+0000 7f905bcd0b80 1 -- 172.21.15.46:0/2776398609 >> 172.21.15.46:0/2776398609 conn(0x55ae0cc731c0 msgr2=0x55ae0ccf8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.198+0000 7f90317fa700 0 client.20820 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.198+0000 7f905bcd0b80 1 -- 172.21.15.46:0/2776398609 shutdown_connections 2024-03-18T21:00:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.198+0000 7f905bcd0b80 1 -- 172.21.15.46:0/2776398609 wait complete. 2024-03-18T21:00:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.198+0000 7f905bcd0b80 20 client.20820 trim_cache size 0 max 16384 2024-03-18T21:00:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.218+0000 7f905bcd0b80 1 Processor -- start 2024-03-18T21:00:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f905bcd0b80 1 -- start start 2024-03-18T21:00:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f905bcd0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfc570 0x55ae0ccf8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f905bcd0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0ccfce90 0x55ae0cd1fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f905bcd0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0cd07db0 0x55ae0ccffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f905bcd0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae0cd04890 con 0x55ae0ccfc570 2024-03-18T21:00:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f905bcd0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae0cd059d0 con 0x55ae0ccfce90 2024-03-18T21:00:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f905bcd0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae0cd1ffa0 con 0x55ae0cd07db0 2024-03-18T21:00:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f904d8d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfc570 0x55ae0ccf8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f904d8d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfc570 0x55ae0ccf8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59632/0 (socket says 172.21.15.46:59632) 2024-03-18T21:00:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f904d8d7700 1 -- 172.21.15.46:0/394339277 learned_addr learned my addr 172.21.15.46:0/394339277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f904d0d6700 1 --2- 172.21.15.46:0/394339277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0ccfce90 0x55ae0cd1fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f904e0d8700 1 --2- 172.21.15.46:0/394339277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0cd07db0 0x55ae0ccffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f904d8d7700 1 -- 172.21.15.46:0/394339277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0cd07db0 msgr2=0x55ae0ccffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f904d8d7700 1 --2- 172.21.15.46:0/394339277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0cd07db0 0x55ae0ccffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f904d8d7700 1 -- 172.21.15.46:0/394339277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0ccfce90 msgr2=0x55ae0cd1fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f904d8d7700 1 --2- 172.21.15.46:0/394339277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0ccfce90 0x55ae0cd1fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.219+0000 7f904d8d7700 1 -- 172.21.15.46:0/394339277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae0cc15e40 con 0x55ae0ccfc570 2024-03-18T21:00:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.220+0000 7f904d8d7700 1 --2- 172.21.15.46:0/394339277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfc570 0x55ae0ccf8bd0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f904400b690 tx=0x7f9044028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6cadd6ca64a32 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.220+0000 7f9038da8700 1 -- 172.21.15.46:0/394339277 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044028070 con 0x55ae0ccfc570 2024-03-18T21:00:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.220+0000 7f9038da8700 1 -- 172.21.15.46:0/394339277 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9044037040 con 0x55ae0ccfc570 2024-03-18T21:00:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.220+0000 7f9038da8700 1 -- 172.21.15.46:0/394339277 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904403ce70 con 0x55ae0ccfc570 2024-03-18T21:00:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.221+0000 7f905bcd0b80 1 -- 172.21.15.46:0/394339277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfc570 msgr2=0x55ae0ccf8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.221+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/394339277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfc570 0x55ae0ccf8bd0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f904400b690 tx=0x7f9044028000 comp rx=0 tx=0).stop 2024-03-18T21:00:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.221+0000 7f905bcd0b80 1 -- 172.21.15.46:0/394339277 shutdown_connections 2024-03-18T21:00:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.221+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/394339277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0ccfc570 0x55ae0ccf8bd0 secure :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f904400b690 tx=0x7f9044028000 comp rx=0 tx=0).stop 2024-03-18T21:00:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.221+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/394339277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0cd07db0 0x55ae0ccffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.221+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/394339277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0ccfce90 0x55ae0cd1fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.221+0000 7f905bcd0b80 1 -- 172.21.15.46:0/394339277 >> 172.21.15.46:0/394339277 conn(0x55ae0cc731c0 msgr2=0x55ae0ccf7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.221+0000 7f905bcd0b80 1 -- 172.21.15.46:0/394339277 shutdown_connections 2024-03-18T21:00:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.221+0000 7f905bcd0b80 1 -- 172.21.15.46:0/394339277 wait complete. 2024-03-18T21:00:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.221+0000 7f905bcd0b80 1 -- start start 2024-03-18T21:00:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.221+0000 7f905bcd0b80 1 Processor -- start 2024-03-18T21:00:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f905bcd0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0ccfce90 0x55ae0cd1fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f905bcd0b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0cd07db0 0x55ae0cd1de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f905bcd0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0cd1e4f0 0x55ae0cd7c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f905bcd0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae0cd1eb40 con 0x55ae0cd1e4f0 2024-03-18T21:00:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f905bcd0b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae0cd1ecb0 con 0x55ae0cd07db0 2024-03-18T21:00:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f905bcd0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae0cd7cb90 con 0x55ae0ccfce90 2024-03-18T21:00:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f905bcd0b80 10 client.0 _ll_register_callbacks cb 0x7f90400094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T21:00:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f904d8d7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0ccfce90 0x55ae0cd1fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f904d8d7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0ccfce90 0x55ae0cd1fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:33664/0 (socket says 172.21.15.46:33664) 2024-03-18T21:00:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f904d8d7700 1 -- 172.21.15.46:0/1826807881 learned_addr learned my addr 172.21.15.46:0/1826807881 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f904e0d8700 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0cd1e4f0 0x55ae0cd7c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f904d0d6700 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0cd07db0 0x55ae0cd1de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f904d8d7700 1 -- 172.21.15.46:0/1826807881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0cd07db0 msgr2=0x55ae0cd1de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f904d8d7700 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0cd07db0 0x55ae0cd1de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f904d8d7700 1 -- 172.21.15.46:0/1826807881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0cd1e4f0 msgr2=0x55ae0cd7c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f904d8d7700 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0cd1e4f0 0x55ae0cd7c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f904d8d7700 1 -- 172.21.15.46:0/1826807881 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904810e430 con 0x55ae0ccfce90 2024-03-18T21:00:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.222+0000 7f904d0d6700 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0cd07db0 0x55ae0cd1de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T21:00:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f904d8d7700 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0ccfce90 0x55ae0cd1fa00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9044023e20 tx=0x7f9044009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db542660f5fa2c81 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f90317fa700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-03-18T21:00:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f90317fa700 1 -- 172.21.15.46:0/1826807881 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904400bad0 con 0x55ae0ccfce90 2024-03-18T21:00:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f90317fa700 1 -- 172.21.15.46:0/1826807881 <== mon.1 v2:172.21.15.155:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f904400bc30 con 0x55ae0ccfce90 2024-03-18T21:00:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1826807881 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f903c004830 con 0x55ae0ccfce90 2024-03-18T21:00:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f90317fa700 1 -- 172.21.15.46:0/1826807881 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f904400bdf0 con 0x55ae0ccfce90 2024-03-18T21:00:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f905bcd0b80 10 client.30583 fetch_fsmap learned FSMap version 103 2024-03-18T21:00:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1826807881 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) v3 -- 0x55ae0cad5c70 con 0x55ae0ccfce90 2024-03-18T21:00:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f90317fa700 1 -- 172.21.15.46:0/1826807881 <== mon.1 v2:172.21.15.155:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f903c004830 con 0x55ae0ccfce90 2024-03-18T21:00:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f90317fa700 1 -- 172.21.15.46:0/1826807881 <== mon.1 v2:172.21.15.155:3300/0 5 ==== fsmap(e 103) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9044004b50 con 0x55ae0ccfce90 2024-03-18T21:00:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f905bcd0b80 10 client.30583 fetch_fsmap finished waiting for FSMap version 103 2024-03-18T21:00:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f905bcd0b80 10 client.30583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T21:00:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.223+0000 7f905bcd0b80 20 client.30583 populate_metadata read hostname 'smithi046' 2024-03-18T21:00:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.224+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae0cd7dd90 0x55ae0cd80180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.224+0000 7f905bcd0b80 4 client.30583 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T21:00:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.224+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1826807881 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ae0ca9ec90 con 0x55ae0cd7dd90 2024-03-18T21:00:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.224+0000 7f904d0d6700 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae0cd7dd90 0x55ae0cd80180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.224+0000 7f904d0d6700 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae0cd7dd90 0x55ae0cd80180 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3f2e13b4d9f8a server_cookie=24dfc17328ac6d5d in_seq=0 out_seq=0 2024-03-18T21:00:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.224+0000 7f90317fa700 10 client.30583 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:40.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 421846 30.33 MB/sec execute 556 sec latency 212.490 ms 2024-03-18T21:00:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f90317fa700 1 -- 172.21.15.46:0/1826807881 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ae0ca9ec90 con 0x55ae0cd7dd90 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f90317fa700 10 client.30583 handle_command_reply: tid=0 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f905bcd0b80 1 client.30583 shutdown 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1826807881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0ccfce90 msgr2=0x55ae0cd1fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0ccfce90 0x55ae0cd1fa00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9044023e20 tx=0x7f9044009290 comp rx=0 tx=0).stop 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1826807881 shutdown_connections 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55ae0cd7dd90 0x55ae0cd80180 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae0cd1e4f0 0x55ae0cd7c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ae0ccfce90 0x55ae0cd1fa00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f90317fa700 0 client.30583 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f905bcd0b80 1 --2- 172.21.15.46:0/1826807881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae0cd07db0 0x55ae0cd1de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1826807881 >> 172.21.15.46:0/1826807881 conn(0x55ae0cc731c0 msgr2=0x55ae0ccffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1826807881 shutdown_connections 2024-03-18T21:00:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f905bcd0b80 1 -- 172.21.15.46:0/1826807881 wait complete. 2024-03-18T21:00:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.557+0000 7f905bcd0b80 20 client.30583 trim_cache size 0 max 16384 2024-03-18T21:00:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T21:00:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T21:00:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T21:00:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T21:00:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.558+0000 7f904fb3b700 1 -- 172.21.15.46:0/1982298123 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90340a1390 msgr2=0x7f90340a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.558+0000 7f904fb3b700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90340a1390 0x7f90340a3850 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9044006010 tx=0x7f904400b260 comp rx=0 tx=0).stop 2024-03-18T21:00:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.558+0000 7f904fb3b700 1 -- 172.21.15.46:0/1982298123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90481009e0 msgr2=0x7f90481a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.558+0000 7f904fb3b700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90481009e0 0x7f90481a53c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f904000efc0 tx=0x7f904000c3a0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.558+0000 7f904fb3b700 1 -- 172.21.15.46:0/1982298123 shutdown_connections 2024-03-18T21:00:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.558+0000 7f904fb3b700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f90340a1390 0x7f90340a3850 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.558+0000 7f904fb3b700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90480696a0 0x7f90480fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.558+0000 7f904fb3b700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90481009e0 0x7f90481a53c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.558+0000 7f904fb3b700 1 --2- 172.21.15.46:0/1982298123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9048106040 0x7f90481004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.558+0000 7f904fb3b700 1 -- 172.21.15.46:0/1982298123 >> 172.21.15.46:0/1982298123 conn(0x7f9048076a80 msgr2=0x7f90481071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.558+0000 7f904fb3b700 1 -- 172.21.15.46:0/1982298123 shutdown_connections 2024-03-18T21:00:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:40.558+0000 7f904fb3b700 1 -- 172.21.15.46:0/1982298123 wait complete. 2024-03-18T21:00:40.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:40 smithi046 ceph-mon[43495]: pgmap v778: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 646 op/s 2024-03-18T21:00:40.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:40 smithi073 ceph-mon[47384]: pgmap v778: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 646 op/s 2024-03-18T21:00:41.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:40 smithi155 ceph-mon[48344]: pgmap v778: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 646 op/s 2024-03-18T21:00:41.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 422408 30.32 MB/sec execute 557 sec latency 220.742 ms 2024-03-18T21:00:42.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 422993 30.32 MB/sec execute 558 sec latency 131.120 ms 2024-03-18T21:00:42.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:42 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:42.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:42 smithi073 ceph-mon[47384]: pgmap v779: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 448 op/s 2024-03-18T21:00:42.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:42 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:42.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:42 smithi155 ceph-mon[48344]: pgmap v779: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 448 op/s 2024-03-18T21:00:42.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:42 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:42.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:42 smithi046 ceph-mon[43495]: pgmap v779: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 448 op/s 2024-03-18T21:00:43.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 423589 30.36 MB/sec execute 559 sec latency 177.738 ms 2024-03-18T21:00:44.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 423989 30.35 MB/sec execute 560 sec latency 90.516 ms 2024-03-18T21:00:44.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:44 smithi155 ceph-mon[48344]: pgmap v780: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.0 MiB/s wr, 497 op/s 2024-03-18T21:00:44.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:44 smithi046 ceph-mon[43495]: pgmap v780: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.0 MiB/s wr, 497 op/s 2024-03-18T21:00:44.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:44 smithi073 ceph-mon[47384]: pgmap v780: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.0 MiB/s wr, 497 op/s 2024-03-18T21:00:45.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 424426 30.34 MB/sec execute 561 sec latency 109.650 ms 2024-03-18T21:00:46.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 424974 30.34 MB/sec execute 562 sec latency 68.217 ms 2024-03-18T21:00:46.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:46 smithi073 ceph-mon[47384]: pgmap v781: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.1 MiB/s wr, 534 op/s 2024-03-18T21:00:46.721 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:46 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T21:00:46.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:46 smithi155 ceph-mon[48344]: pgmap v781: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.1 MiB/s wr, 534 op/s 2024-03-18T21:00:46.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:46 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T21:00:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:46 smithi046 ceph-mon[43495]: pgmap v781: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.1 MiB/s wr, 534 op/s 2024-03-18T21:00:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:46 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T21:00:47.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 425431 30.32 MB/sec execute 563 sec latency 113.579 ms 2024-03-18T21:00:47.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:47 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:47.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:47 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:47.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:47 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:47.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:47 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:47.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:47 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:47.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:47 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:47 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:47 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:47 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:47 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:47 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:47 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:47 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:47 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:47 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:47 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:47 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:47 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:48.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 426101 30.33 MB/sec execute 564 sec latency 96.383 ms 2024-03-18T21:00:48.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:48 smithi073 ceph-mon[47384]: pgmap v782: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.1 MiB/s wr, 459 op/s 2024-03-18T21:00:48.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:48 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T21:00:48.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:48 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T21:00:48.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:48 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:49.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:48 smithi155 ceph-mon[48344]: pgmap v782: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.1 MiB/s wr, 459 op/s 2024-03-18T21:00:49.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:48 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T21:00:49.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:48 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T21:00:49.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:48 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:49.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:48 smithi046 ceph-mon[43495]: pgmap v782: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.1 MiB/s wr, 459 op/s 2024-03-18T21:00:49.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:48 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T21:00:49.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:48 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T21:00:49.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:48 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:00:49.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 426831 30.34 MB/sec execute 565 sec latency 91.218 ms 2024-03-18T21:00:50.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 427384 30.36 MB/sec execute 566 sec latency 82.030 ms 2024-03-18T21:00:50.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:50 smithi073 ceph-mon[47384]: pgmap v783: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.0 MiB/s wr, 529 op/s 2024-03-18T21:00:51.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:50 smithi155 ceph-mon[48344]: pgmap v783: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.0 MiB/s wr, 529 op/s 2024-03-18T21:00:51.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:50 smithi046 ceph-mon[43495]: pgmap v783: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.0 MiB/s wr, 529 op/s 2024-03-18T21:00:51.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 427805 30.34 MB/sec execute 567 sec latency 111.168 ms 2024-03-18T21:00:52.514 INFO:tasks.workunit.client.0.smithi046.stdout: 10 428390 30.34 MB/sec execute 568 sec latency 68.485 ms 2024-03-18T21:00:52.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:52 smithi073 ceph-mon[47384]: pgmap v784: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.8 MiB/s wr, 453 op/s 2024-03-18T21:00:53.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:52 smithi155 ceph-mon[48344]: pgmap v784: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.8 MiB/s wr, 453 op/s 2024-03-18T21:00:53.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:52 smithi046 ceph-mon[43495]: pgmap v784: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.8 MiB/s wr, 453 op/s 2024-03-18T21:00:53.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 428939 30.34 MB/sec execute 569 sec latency 270.808 ms 2024-03-18T21:00:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.732+0000 7f84b3b28700 1 -- 172.21.15.46:0/3673083973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ac104f40 msgr2=0x7f84ac10cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.732+0000 7f84b3b28700 1 --2- 172.21.15.46:0/3673083973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ac104f40 0x7f84ac10cac0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f84a400b0a0 tx=0x7f84a400b3b0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.733+0000 7f84b3b28700 1 -- 172.21.15.46:0/3673083973 shutdown_connections 2024-03-18T21:00:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.733+0000 7f84b3b28700 1 --2- 172.21.15.46:0/3673083973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ac104f40 0x7f84ac10cac0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.733+0000 7f84b3b28700 1 --2- 172.21.15.46:0/3673083973 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0ffe20 0x7f84ac100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.733+0000 7f84b3b28700 1 --2- 172.21.15.46:0/3673083973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84ac1007d0 0x7f84ac104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.733+0000 7f84b3b28700 1 -- 172.21.15.46:0/3673083973 >> 172.21.15.46:0/3673083973 conn(0x7f84ac0fb680 msgr2=0x7f84ac0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.733+0000 7f84b3b28700 1 -- 172.21.15.46:0/3673083973 shutdown_connections 2024-03-18T21:00:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.733+0000 7f84b3b28700 1 -- 172.21.15.46:0/3673083973 wait complete. 2024-03-18T21:00:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.734+0000 7f84b3b28700 1 Processor -- start 2024-03-18T21:00:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.736+0000 7f84b3b28700 1 -- start start 2024-03-18T21:00:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.736+0000 7f84b3b28700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0ffe20 0x7f84ac072580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.736+0000 7f84b3b28700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ac104f40 0x7f84ac06b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.736+0000 7f84b3b28700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84ac06bb90 0x7f84ac06c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.736+0000 7f84b3b28700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac072e30 con 0x7f84ac104f40 2024-03-18T21:00:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.736+0000 7f84b3b28700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac070110 con 0x7f84ac06bb90 2024-03-18T21:00:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.736+0000 7f84b3b28700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ac0702d0 con 0x7f84ac0ffe20 2024-03-18T21:00:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.738+0000 7f84b18c4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0ffe20 0x7f84ac072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.739+0000 7f84b18c4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0ffe20 0x7f84ac072580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:41132/0 (socket says 172.21.15.46:41132) 2024-03-18T21:00:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.739+0000 7f84b18c4700 1 -- 172.21.15.46:0/1767884028 learned_addr learned my addr 172.21.15.46:0/1767884028 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.739+0000 7f84b18c4700 1 -- 172.21.15.46:0/1767884028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84ac06bb90 msgr2=0x7f84ac06c040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-18T21:00:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.739+0000 7f84b18c4700 1 --2- 172.21.15.46:0/1767884028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84ac06bb90 0x7f84ac06c040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.739+0000 7f84b18c4700 1 -- 172.21.15.46:0/1767884028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ac104f40 msgr2=0x7f84ac06b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.739+0000 7f84b18c4700 1 --2- 172.21.15.46:0/1767884028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ac104f40 0x7f84ac06b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.739+0000 7f84b18c4700 1 -- 172.21.15.46:0/1767884028 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c009520 con 0x7f84ac0ffe20 2024-03-18T21:00:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.739+0000 7f84b18c4700 1 --2- 172.21.15.46:0/1767884028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0ffe20 0x7f84ac072580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b530 tx=0x7f84a800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.740+0000 7f84a2ffd700 1 -- 172.21.15.46:0/1767884028 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8011830 con 0x7f84ac0ffe20 2024-03-18T21:00:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.740+0000 7f84a2ffd700 1 -- 172.21.15.46:0/1767884028 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a8011e50 con 0x7f84ac0ffe20 2024-03-18T21:00:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.740+0000 7f84b3b28700 1 -- 172.21.15.46:0/1767884028 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a4009d00 con 0x7f84ac0ffe20 2024-03-18T21:00:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.740+0000 7f84a2ffd700 1 -- 172.21.15.46:0/1767884028 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a800f2e0 con 0x7f84ac0ffe20 2024-03-18T21:00:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.740+0000 7f84b3b28700 1 -- 172.21.15.46:0/1767884028 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ac070930 con 0x7f84ac0ffe20 2024-03-18T21:00:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.743+0000 7f84a2ffd700 1 -- 172.21.15.46:0/1767884028 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f84a800f440 con 0x7f84ac0ffe20 2024-03-18T21:00:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.744+0000 7f84a2ffd700 1 --2- 172.21.15.46:0/1767884028 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84980a13e0 0x7f84980a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.744+0000 7f84a2ffd700 1 -- 172.21.15.46:0/1767884028 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f84a80cd630 con 0x7f84ac0ffe20 2024-03-18T21:00:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.745+0000 7f84b10c3700 1 --2- 172.21.15.46:0/1767884028 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84980a13e0 0x7f84980a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.746+0000 7f84bfcbdb80 1 Processor -- start 2024-03-18T21:00:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.746+0000 7f84b10c3700 1 --2- 172.21.15.46:0/1767884028 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84980a13e0 0x7f84980a38a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f849c005d90 tx=0x7f849c005ce0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.746+0000 7f84bfcbdb80 1 -- start start 2024-03-18T21:00:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.746+0000 7f84bfcbdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f8a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.746+0000 7f84bfcbdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178f87550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.746+0000 7f84bfcbdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178fa03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.746+0000 7f84bfcbdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557178edb930 con 0x557178f8a6a0 2024-03-18T21:00:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.746+0000 7f84bfcbdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557178ed92f0 con 0x557178f89d80 2024-03-18T21:00:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.746+0000 7f84bfcbdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557178ed4be0 con 0x557178f9dfc0 2024-03-18T21:00:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.746+0000 7f84b20c5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178fa03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.747+0000 7f84b20c5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178fa03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:41136/0 (socket says 172.21.15.46:41136) 2024-03-18T21:00:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.747+0000 7f84b20c5700 1 -- 172.21.15.46:0/2444841735 learned_addr learned my addr 172.21.15.46:0/2444841735 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.747+0000 7f84b18c4700 1 --2- 172.21.15.46:0/2444841735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f8a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.747+0000 7f84b20c5700 1 -- 172.21.15.46:0/2444841735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 msgr2=0x557178f8a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.747+0000 7f84b20c5700 1 --2- 172.21.15.46:0/2444841735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f8a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.747+0000 7f84b20c5700 1 -- 172.21.15.46:0/2444841735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 msgr2=0x557178f87550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.747+0000 7f84b20c5700 1 --2- 172.21.15.46:0/2444841735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178f87550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.747+0000 7f84b20c5700 1 -- 172.21.15.46:0/2444841735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557178d56350 con 0x557178f9dfc0 2024-03-18T21:00:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.747+0000 7f84b20c5700 1 --2- 172.21.15.46:0/2444841735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178fa03b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009bc0 tx=0x7f84a4009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77a23b2b192dde63 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.748+0000 7f8497fff700 1 -- 172.21.15.46:0/2444841735 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a400e070 con 0x557178f9dfc0 2024-03-18T21:00:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.748+0000 7f8497fff700 1 -- 172.21.15.46:0/2444841735 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a4007da0 con 0x557178f9dfc0 2024-03-18T21:00:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.748+0000 7f8497fff700 1 -- 172.21.15.46:0/2444841735 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a40078a0 con 0x557178f9dfc0 2024-03-18T21:00:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.749+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/2444841735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 msgr2=0x557178fa03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.749+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/2444841735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178fa03b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009bc0 tx=0x7f84a4009bf0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.749+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/2444841735 shutdown_connections 2024-03-18T21:00:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.749+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/2444841735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178fa03b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.749+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/2444841735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f8a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.749+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/2444841735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178f87550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.749+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/2444841735 >> 172.21.15.46:0/2444841735 conn(0x557178ef81c0 msgr2=0x557178f7d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.749+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/2444841735 shutdown_connections 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.749+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/2444841735 wait complete. 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.749+0000 7f84bfcbdb80 1 -- start start 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.750+0000 7f84bfcbdb80 1 Processor -- start 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.750+0000 7f84bfcbdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f8d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.750+0000 7f84bfcbdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178f8d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.750+0000 7f84bfcbdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178f8dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.750+0000 7f84bfcbdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557178e70d20 con 0x557178f8a6a0 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.750+0000 7f84bfcbdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557178e709c0 con 0x557178f89d80 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.750+0000 7f84bfcbdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557178d58640 con 0x557178f9dfc0 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.750+0000 7f84bfcbdb80 10 client.0 _ll_register_callbacks cb 0x557178cdc340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.750+0000 7f84b18c4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f8d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.750+0000 7f84b18c4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f8d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:42258/0 (socket says 172.21.15.46:42258) 2024-03-18T21:00:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.750+0000 7f84b18c4700 1 -- 172.21.15.46:0/670153491 learned_addr learned my addr 172.21.15.46:0/670153491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.750+0000 7f84b10c3700 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178f8d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.751+0000 7f84b20c5700 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178f8dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.751+0000 7f84b10c3700 1 -- 172.21.15.46:0/670153491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 msgr2=0x557178f8dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.751+0000 7f84b10c3700 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178f8dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.751+0000 7f84b10c3700 1 -- 172.21.15.46:0/670153491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 msgr2=0x557178f8d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.751+0000 7f84b10c3700 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f8d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.751+0000 7f84b10c3700 1 -- 172.21.15.46:0/670153491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ac0692d0 con 0x557178f8a6a0 2024-03-18T21:00:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.751+0000 7f84b10c3700 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178f8d6a0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f84ac06cd80 tx=0x7f849c010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669148d1fe45ffc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.751+0000 7f84957fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T21:00:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.751+0000 7f84957fa700 1 -- 172.21.15.46:0/670153491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c010b60 con 0x557178f8a6a0 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84957fa700 1 -- 172.21.15.46:0/670153491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f849c010cc0 con 0x557178f8a6a0 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/670153491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557178f756c0 con 0x557178f8a6a0 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84957fa700 1 -- 172.21.15.46:0/670153491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f849c010e80 con 0x557178f8a6a0 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84bfcbdb80 10 client.20832 fetch_fsmap learned FSMap version 106 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/670153491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=4+}) v3 -- 0x557178d02b00 con 0x557178f8a6a0 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84957fa700 1 -- 172.21.15.46:0/670153491 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557178f756c0 con 0x557178f8a6a0 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84957fa700 1 -- 172.21.15.46:0/670153491 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 106) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f849c0256d0 con 0x557178f8a6a0 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84bfcbdb80 10 client.20832 fetch_fsmap finished waiting for FSMap version 106 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84bfcbdb80 10 client.20832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84bfcbdb80 20 client.20832 populate_metadata read hostname 'smithi046' 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557178f9c590 0x557178f9c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84bfcbdb80 4 client.20832 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.752+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/670153491 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557178ecd6a0 con 0x557178f9c590 2024-03-18T21:00:53.757 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.753+0000 7f84b18c4700 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557178f9c590 0x557178f9c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.756+0000 7f84b18c4700 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557178f9c590 0x557178f9c9a0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b363e1aa775c3ffc server_cookie=bd6da6175db2d8ca in_seq=0 out_seq=0 2024-03-18T21:00:53.758 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.756+0000 7f84957fa700 10 client.20832 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.758+0000 7f84957fa700 1 -- 172.21.15.46:0/670153491 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557178ecd6a0 con 0x557178f9c590 2024-03-18T21:00:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.758+0000 7f84957fa700 10 client.20832 handle_command_reply: tid=0 2024-03-18T21:00:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.759+0000 7f84bfcbdb80 1 client.20832 shutdown 2024-03-18T21:00:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.760+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/670153491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 msgr2=0x557178f8d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.760+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178f8d6a0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f84ac06cd80 tx=0x7f849c010710 comp rx=0 tx=0).stop 2024-03-18T21:00:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.760+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/670153491 shutdown_connections 2024-03-18T21:00:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.760+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557178f9c590 0x557178f9c9a0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.760+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178f8dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.760+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f8d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.760+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/670153491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178f8d6a0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.760+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/670153491 >> 172.21.15.46:0/670153491 conn(0x557178ef81c0 msgr2=0x557178f7d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.760+0000 7f84957fa700 0 client.20832 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.760+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/670153491 shutdown_connections 2024-03-18T21:00:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.760+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/670153491 wait complete. 2024-03-18T21:00:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.760+0000 7f84bfcbdb80 20 client.20832 trim_cache size 0 max 16384 2024-03-18T21:00:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.783+0000 7f84bfcbdb80 1 Processor -- start 2024-03-18T21:00:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.785+0000 7f84bfcbdb80 1 -- start start 2024-03-18T21:00:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.785+0000 7f84bfcbdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f81fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.785+0000 7f84bfcbdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178f824e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.785+0000 7f84bfcbdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178f861c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.785+0000 7f84bfcbdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557178f8b000 con 0x557178f8a6a0 2024-03-18T21:00:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.785+0000 7f84bfcbdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557178f99bb0 con 0x557178f89d80 2024-03-18T21:00:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.785+0000 7f84bfcbdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557178f89bf0 con 0x557178f9dfc0 2024-03-18T21:00:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.785+0000 7f84b18c4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f81fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.785+0000 7f84b18c4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f81fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:42266/0 (socket says 172.21.15.46:42266) 2024-03-18T21:00:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.785+0000 7f84b18c4700 1 -- 172.21.15.46:0/4147065980 learned_addr learned my addr 172.21.15.46:0/4147065980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.785+0000 7f84b20c5700 1 --2- 172.21.15.46:0/4147065980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178f861c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.786+0000 7f84b18c4700 1 -- 172.21.15.46:0/4147065980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 msgr2=0x557178f861c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.786+0000 7f84b18c4700 1 --2- 172.21.15.46:0/4147065980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178f861c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.786+0000 7f84b18c4700 1 -- 172.21.15.46:0/4147065980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 msgr2=0x557178f824e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T21:00:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.786+0000 7f84b18c4700 1 --2- 172.21.15.46:0/4147065980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178f824e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.786+0000 7f84b18c4700 1 -- 172.21.15.46:0/4147065980 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557178e9ae40 con 0x557178f89d80 2024-03-18T21:00:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.786+0000 7f84b20c5700 1 --2- 172.21.15.46:0/4147065980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178f861c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T21:00:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.786+0000 7f84b18c4700 1 --2- 172.21.15.46:0/4147065980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f81fa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f84a8014040 tx=0x7f84a8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb955df7a846e5e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.786+0000 7f84a0da8700 1 -- 172.21.15.46:0/4147065980 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a80b3170 con 0x557178f89d80 2024-03-18T21:00:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.786+0000 7f84a0da8700 1 -- 172.21.15.46:0/4147065980 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a80b2070 con 0x557178f89d80 2024-03-18T21:00:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.786+0000 7f84a0da8700 1 -- 172.21.15.46:0/4147065980 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8018570 con 0x557178f89d80 2024-03-18T21:00:53.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.787+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4147065980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 msgr2=0x557178f81fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.787+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/4147065980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f81fa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f84a8014040 tx=0x7f84a8000bc0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.787+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4147065980 shutdown_connections 2024-03-18T21:00:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.787+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/4147065980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f9dfc0 0x557178f861c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.787+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/4147065980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f89d80 0x557178f81fa0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.787+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/4147065980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178f824e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.787+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4147065980 >> 172.21.15.46:0/4147065980 conn(0x557178ef81c0 msgr2=0x557178f9c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.788+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4147065980 shutdown_connections 2024-03-18T21:00:53.792 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.791+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4147065980 wait complete. 2024-03-18T21:00:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.791+0000 7f84bfcbdb80 1 -- start start 2024-03-18T21:00:53.793 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.791+0000 7f84bfcbdb80 1 Processor -- start 2024-03-18T21:00:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84bfcbdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f89d80 0x557178fa4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84bfcbdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178fa1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84bfcbdb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f9dfc0 0x557178fa1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84bfcbdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557178fa2420 con 0x557178f8a6a0 2024-03-18T21:00:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84bfcbdb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557178fa2590 con 0x557178f9dfc0 2024-03-18T21:00:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84bfcbdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557178ff0480 con 0x557178f89d80 2024-03-18T21:00:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84bfcbdb80 10 client.0 _ll_register_callbacks cb 0x7f849c0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T21:00:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84b20c5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f9dfc0 0x557178fa1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:53.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84b20c5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f9dfc0 0x557178fa1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:42278/0 (socket says 172.21.15.46:42278) 2024-03-18T21:00:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84b20c5700 1 -- 172.21.15.46:0/4111781266 learned_addr learned my addr 172.21.15.46:0/4111781266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:00:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84b20c5700 1 -- 172.21.15.46:0/4111781266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f89d80 msgr2=0x557178fa4320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T21:00:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84b20c5700 1 --2- 172.21.15.46:0/4111781266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f89d80 0x557178fa4320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84b20c5700 1 -- 172.21.15.46:0/4111781266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 msgr2=0x557178fa1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:00:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84b20c5700 1 --2- 172.21.15.46:0/4111781266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178fa1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.792+0000 7f84b20c5700 1 -- 172.21.15.46:0/4111781266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557178d56350 con 0x557178f9dfc0 2024-03-18T21:00:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.793+0000 7f84b20c5700 1 --2- 172.21.15.46:0/4111781266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f9dfc0 0x557178fa1d40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f84a4006130 tx=0x7f84a40060c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61ea142915fc7b88 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:00:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.793+0000 7f84957fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T21:00:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.793+0000 7f84957fa700 1 -- 172.21.15.46:0/4111781266 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a400e040 con 0x557178f9dfc0 2024-03-18T21:00:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.794+0000 7f84957fa700 1 -- 172.21.15.46:0/4111781266 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f84a40078b0 con 0x557178f9dfc0 2024-03-18T21:00:53.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.794+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4111781266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84a4007da0 con 0x557178f9dfc0 2024-03-18T21:00:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.794+0000 7f84957fa700 1 -- 172.21.15.46:0/4111781266 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84a4003e90 con 0x557178f9dfc0 2024-03-18T21:00:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.794+0000 7f84bfcbdb80 10 client.30455 fetch_fsmap learned FSMap version 106 2024-03-18T21:00:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.794+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4111781266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=4+}) v3 -- 0x7f84ac070930 con 0x557178f9dfc0 2024-03-18T21:00:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.795+0000 7f84957fa700 1 -- 172.21.15.46:0/4111781266 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a4007da0 con 0x557178f9dfc0 2024-03-18T21:00:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.795+0000 7f84957fa700 1 -- 172.21.15.46:0/4111781266 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 106) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f84a4044070 con 0x557178f9dfc0 2024-03-18T21:00:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.795+0000 7f84bfcbdb80 10 client.30455 fetch_fsmap finished waiting for FSMap version 106 2024-03-18T21:00:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.795+0000 7f84bfcbdb80 10 client.30455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T21:00:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.795+0000 7f84bfcbdb80 20 client.30455 populate_metadata read hostname 'smithi046' 2024-03-18T21:00:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.795+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/4111781266 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557178ff1620 0x557178ff3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:00:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.795+0000 7f84bfcbdb80 4 client.30455 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T21:00:53.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.795+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4111781266 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557178d23c90 con 0x557178ff1620 2024-03-18T21:00:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.798+0000 7f84b18c4700 1 --2- 172.21.15.46:0/4111781266 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557178ff1620 0x557178ff3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:00:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.799+0000 7f84957fa700 10 client.30455 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:53.801 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:53.799+0000 7f84b18c4700 1 --2- 172.21.15.46:0/4111781266 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557178ff1620 0x557178ff3a40 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1988302c122d92b server_cookie=5a56d6728cd7f7dc in_seq=0 out_seq=0 2024-03-18T21:00:53.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:53 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:54.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:53 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:54.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:53 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.141+0000 7f84957fa700 1 -- 172.21.15.46:0/4111781266 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557178d23c90 con 0x557178ff1620 2024-03-18T21:00:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.141+0000 7f84957fa700 10 client.30455 handle_command_reply: tid=0 2024-03-18T21:00:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.141+0000 7f84bfcbdb80 1 client.30455 shutdown 2024-03-18T21:00:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.142+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4111781266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f9dfc0 msgr2=0x557178fa1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.142+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/4111781266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f9dfc0 0x557178fa1d40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f84a4006130 tx=0x7f84a40060c0 comp rx=0 tx=0).stop 2024-03-18T21:00:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.143+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4111781266 shutdown_connections 2024-03-18T21:00:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.143+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/4111781266 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x557178ff1620 0x557178ff3a40 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.143+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/4111781266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557178f9dfc0 0x557178fa1d40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.143+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/4111781266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557178f89d80 0x557178fa4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.143+0000 7f84bfcbdb80 1 --2- 172.21.15.46:0/4111781266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557178f8a6a0 0x557178fa1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.143+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4111781266 >> 172.21.15.46:0/4111781266 conn(0x557178ef81c0 msgr2=0x557178f81fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.143+0000 7f84957fa700 0 client.30455 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T21:00:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.143+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4111781266 shutdown_connections 2024-03-18T21:00:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.143+0000 7f84bfcbdb80 1 -- 172.21.15.46:0/4111781266 wait complete. 2024-03-18T21:00:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.143+0000 7f84bfcbdb80 20 client.30455 trim_cache size 0 max 16384 2024-03-18T21:00:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T21:00:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T21:00:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T21:00:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T21:00:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.144+0000 7f84b3b28700 1 -- 172.21.15.46:0/1767884028 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84980a13e0 msgr2=0x7f84980a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.144+0000 7f84b3b28700 1 --2- 172.21.15.46:0/1767884028 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84980a13e0 0x7f84980a38a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f849c005d90 tx=0x7f849c005ce0 comp rx=0 tx=0).stop 2024-03-18T21:00:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.144+0000 7f84b3b28700 1 -- 172.21.15.46:0/1767884028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0ffe20 msgr2=0x7f84ac072580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:00:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.144+0000 7f84b3b28700 1 --2- 172.21.15.46:0/1767884028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0ffe20 0x7f84ac072580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b530 tx=0x7f84a800b8f0 comp rx=0 tx=0).stop 2024-03-18T21:00:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.144+0000 7f84b3b28700 1 -- 172.21.15.46:0/1767884028 shutdown_connections 2024-03-18T21:00:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.144+0000 7f84b3b28700 1 --2- 172.21.15.46:0/1767884028 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f84980a13e0 0x7f84980a38a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.144+0000 7f84b3b28700 1 --2- 172.21.15.46:0/1767884028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84ac06bb90 0x7f84ac06c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.144+0000 7f84b3b28700 1 --2- 172.21.15.46:0/1767884028 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84ac0ffe20 0x7f84ac072580 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.144+0000 7f84b3b28700 1 --2- 172.21.15.46:0/1767884028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ac104f40 0x7f84ac06b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:00:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.144+0000 7f84b3b28700 1 -- 172.21.15.46:0/1767884028 >> 172.21.15.46:0/1767884028 conn(0x7f84ac0fb680 msgr2=0x7f84ac0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:00:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.144+0000 7f84b3b28700 1 -- 172.21.15.46:0/1767884028 shutdown_connections 2024-03-18T21:00:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:00:54.144+0000 7f84b3b28700 1 -- 172.21.15.46:0/1767884028 wait complete. 2024-03-18T21:00:54.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 429363 30.33 MB/sec execute 570 sec latency 183.871 ms 2024-03-18T21:00:54.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:54 smithi073 ceph-mon[47384]: pgmap v785: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.7 MiB/s wr, 499 op/s 2024-03-18T21:00:54.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:54 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:55.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:54 smithi155 ceph-mon[48344]: pgmap v785: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.7 MiB/s wr, 499 op/s 2024-03-18T21:00:55.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:54 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:55.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:54 smithi046 ceph-mon[43495]: pgmap v785: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.7 MiB/s wr, 499 op/s 2024-03-18T21:00:55.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:54 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:00:55.513 INFO:tasks.workunit.client.0.smithi046.stdout: 10 429877 30.32 MB/sec execute 571 sec latency 122.437 ms 2024-03-18T21:00:56.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 430415 30.30 MB/sec execute 572 sec latency 96.609 ms 2024-03-18T21:00:56.722 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:56 smithi073 ceph-mon[47384]: pgmap v786: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.2 MiB/s wr, 508 op/s 2024-03-18T21:00:56.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:56 smithi155 ceph-mon[48344]: pgmap v786: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.2 MiB/s wr, 508 op/s 2024-03-18T21:00:56.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:56 smithi046 ceph-mon[43495]: pgmap v786: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.2 MiB/s wr, 508 op/s 2024-03-18T21:00:57.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 431077 30.33 MB/sec execute 573 sec latency 68.708 ms 2024-03-18T21:00:58.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 431711 30.34 MB/sec execute 574 sec latency 84.592 ms 2024-03-18T21:00:58.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:00:58 smithi155 ceph-mon[48344]: pgmap v787: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.5 MiB/s wr, 458 op/s 2024-03-18T21:00:58.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:00:58 smithi046 ceph-mon[43495]: pgmap v787: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.5 MiB/s wr, 458 op/s 2024-03-18T21:00:58.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:00:58 smithi073 ceph-mon[47384]: pgmap v787: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.5 MiB/s wr, 458 op/s 2024-03-18T21:00:59.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 432149 30.33 MB/sec execute 575 sec latency 104.065 ms 2024-03-18T21:01:00.512 INFO:tasks.workunit.client.0.smithi046.stdout: 10 432722 30.33 MB/sec execute 576 sec latency 96.785 ms 2024-03-18T21:01:00.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:00 smithi046 ceph-mon[43495]: pgmap v788: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.6 MiB/s wr, 547 op/s 2024-03-18T21:01:00.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:00 smithi073 ceph-mon[47384]: pgmap v788: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.6 MiB/s wr, 547 op/s 2024-03-18T21:01:01.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:00 smithi155 ceph-mon[48344]: pgmap v788: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 8.6 MiB/s wr, 547 op/s 2024-03-18T21:01:01.513 INFO:tasks.workunit.client.0.smithi046.stdout: 10 432739 30.28 MB/sec execute 577 sec latency 998.827 ms 2024-03-18T21:01:02.513 INFO:tasks.workunit.client.0.smithi046.stdout: 10 432739 30.23 MB/sec execute 578 sec latency 1999.648 ms 2024-03-18T21:01:02.625 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:02 smithi155 ceph-mon[48344]: pgmap v789: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.8 MiB/s wr, 486 op/s 2024-03-18T21:01:02.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:02 smithi046 ceph-mon[43495]: pgmap v789: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.8 MiB/s wr, 486 op/s 2024-03-18T21:01:02.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:02 smithi073 ceph-mon[47384]: pgmap v789: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 7.8 MiB/s wr, 486 op/s 2024-03-18T21:01:03.513 INFO:tasks.workunit.client.0.smithi046.stdout: 10 432739 30.18 MB/sec execute 579 sec latency 2999.698 ms 2024-03-18T21:01:04.513 INFO:tasks.workunit.client.0.smithi046.stdout: 10 432885 30.14 MB/sec execute 580 sec latency 3366.409 ms 2024-03-18T21:01:04.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:04 smithi073 ceph-mon[47384]: pgmap v790: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.1 MiB/s wr, 492 op/s 2024-03-18T21:01:05.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:04 smithi155 ceph-mon[48344]: pgmap v790: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.1 MiB/s wr, 492 op/s 2024-03-18T21:01:05.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:04 smithi046 ceph-mon[43495]: pgmap v790: 97 pgs: 97 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.1 MiB/s wr, 492 op/s 2024-03-18T21:01:05.513 INFO:tasks.workunit.client.0.smithi046.stdout: 10 432888 30.09 MB/sec execute 581 sec latency 1109.060 ms 2024-03-18T21:01:06.514 INFO:tasks.workunit.client.0.smithi046.stdout: 10 432888 30.04 MB/sec execute 582 sec latency 2109.164 ms 2024-03-18T21:01:06.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:06 smithi073 ceph-mon[47384]: pgmap v791: 97 pgs: 97 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 18 MiB/s wr, 627 op/s 2024-03-18T21:01:06.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:06 smithi155 ceph-mon[48344]: pgmap v791: 97 pgs: 97 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 18 MiB/s wr, 627 op/s 2024-03-18T21:01:06.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:06 smithi046 ceph-mon[43495]: pgmap v791: 97 pgs: 97 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 18 MiB/s wr, 627 op/s 2024-03-18T21:01:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.279+0000 7f8ae006c700 1 -- 172.21.15.46:0/752964666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad8100570 msgr2=0x7f8ad810ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.279+0000 7f8ae006c700 1 --2- 172.21.15.46:0/752964666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad8100570 0x7f8ad810ca00 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000b0a0 tx=0x7f8ad000b3b0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.279+0000 7f8ae006c700 1 -- 172.21.15.46:0/752964666 shutdown_connections 2024-03-18T21:01:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.279+0000 7f8ae006c700 1 --2- 172.21.15.46:0/752964666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad8100570 0x7f8ad810ca00 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.279+0000 7f8ae006c700 1 --2- 172.21.15.46:0/752964666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ad8105c60 0x7f8ad8106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.279+0000 7f8ae006c700 1 --2- 172.21.15.46:0/752964666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad80ffbb0 0x7f8ad8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.279+0000 7f8ae006c700 1 -- 172.21.15.46:0/752964666 >> 172.21.15.46:0/752964666 conn(0x7f8ad8076d50 msgr2=0x7f8ad8077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.279+0000 7f8ae006c700 1 -- 172.21.15.46:0/752964666 shutdown_connections 2024-03-18T21:01:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.279+0000 7f8ae006c700 1 -- 172.21.15.46:0/752964666 wait complete. 2024-03-18T21:01:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8ae006c700 1 Processor -- start 2024-03-18T21:01:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8ae006c700 1 -- start start 2024-03-18T21:01:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8ae006c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80ffbb0 0x7f8ad8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8ae006c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ad8100570 0x7f8ad81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8ae006c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad8105c60 0x7f8ad81a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8ae006c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad8198e00 con 0x7f8ad80ffbb0 2024-03-18T21:01:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8ae006c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81a3630 con 0x7f8ad8105c60 2024-03-18T21:01:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8ae006c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad81a3770 con 0x7f8ad8100570 2024-03-18T21:01:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8adde08700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80ffbb0 0x7f8ad8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8adde08700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80ffbb0 0x7f8ad8198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52774/0 (socket says 172.21.15.46:52774) 2024-03-18T21:01:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8adde08700 1 -- 172.21.15.46:0/1118352898 learned_addr learned my addr 172.21.15.46:0/1118352898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8ade609700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad8105c60 0x7f8ad81a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8add607700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ad8100570 0x7f8ad81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8adde08700 1 -- 172.21.15.46:0/1118352898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ad8100570 msgr2=0x7f8ad81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8adde08700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ad8100570 0x7f8ad81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8adde08700 1 -- 172.21.15.46:0/1118352898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad8105c60 msgr2=0x7f8ad81a30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8adde08700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad8105c60 0x7f8ad81a30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.280+0000 7f8adde08700 1 -- 172.21.15.46:0/1118352898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad0009d00 con 0x7f8ad80ffbb0 2024-03-18T21:01:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.281+0000 7f8adde08700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80ffbb0 0x7f8ad8198070 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400e9d0 tx=0x7f8ad400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.281+0000 7f8aceffd700 1 -- 172.21.15.46:0/1118352898 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4009790 con 0x7f8ad80ffbb0 2024-03-18T21:01:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.281+0000 7f8ae006c700 1 -- 172.21.15.46:0/1118352898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad81a39d0 con 0x7f8ad80ffbb0 2024-03-18T21:01:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.281+0000 7f8aceffd700 1 -- 172.21.15.46:0/1118352898 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ad400ee70 con 0x7f8ad80ffbb0 2024-03-18T21:01:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.281+0000 7f8aceffd700 1 -- 172.21.15.46:0/1118352898 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4010ea0 con 0x7f8ad80ffbb0 2024-03-18T21:01:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.281+0000 7f8ae006c700 1 -- 172.21.15.46:0/1118352898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad81a3f30 con 0x7f8ad80ffbb0 2024-03-18T21:01:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.284+0000 7f8aceffd700 1 -- 172.21.15.46:0/1118352898 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8ad400cbd0 con 0x7f8ad80ffbb0 2024-03-18T21:01:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.284+0000 7f8aceffd700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8ac40a1290 0x7f8ac40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.285+0000 7f8add607700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8ac40a1290 0x7f8ac40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.285+0000 7f8aceffd700 1 -- 172.21.15.46:0/1118352898 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8ad4014070 con 0x7f8ad80ffbb0 2024-03-18T21:01:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.285+0000 7f8aec201b80 1 Processor -- start 2024-03-18T21:01:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.285+0000 7f8add607700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8ac40a1290 0x7f8ac40a3750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8068870 tx=0x7f8ac80073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.285+0000 7f8aec201b80 1 -- start start 2024-03-18T21:01:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.285+0000 7f8aec201b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c210749950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.285+0000 7f8aec201b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c210749e90 0x55c210758c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.285+0000 7f8aec201b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 0x55c21075b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.285+0000 7f8aec201b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2106a3930 con 0x55c2107591c0 2024-03-18T21:01:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.285+0000 7f8aec201b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2106a12f0 con 0x55c210749570 2024-03-18T21:01:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.285+0000 7f8aec201b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21069cbe0 con 0x55c210749e90 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8ade609700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 0x55c21075b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8adde08700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c210749950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8ade609700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 0x55c21075b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52788/0 (socket says 172.21.15.46:52788) 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8ade609700 1 -- 172.21.15.46:0/3069706398 learned_addr learned my addr 172.21.15.46:0/3069706398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8add607700 1 --2- 172.21.15.46:0/3069706398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c210749e90 0x55c210758c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8ade609700 1 -- 172.21.15.46:0/3069706398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c210749e90 msgr2=0x55c210758c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8ade609700 1 --2- 172.21.15.46:0/3069706398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c210749e90 0x55c210758c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8ade609700 1 -- 172.21.15.46:0/3069706398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 msgr2=0x55c210749950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8ade609700 1 --2- 172.21.15.46:0/3069706398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c210749950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8ade609700 1 -- 172.21.15.46:0/3069706398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c21051e350 con 0x55c2107591c0 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8adde08700 1 --2- 172.21.15.46:0/3069706398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c210749950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8ade609700 1 --2- 172.21.15.46:0/3069706398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 0x55c21075b5b0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009390 tx=0x7f8ad0009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f408b667035f7d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8ac3fff700 1 -- 172.21.15.46:0/3069706398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000e040 con 0x55c2107591c0 2024-03-18T21:01:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8ac3fff700 1 -- 172.21.15.46:0/3069706398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ad0007d30 con 0x55c2107591c0 2024-03-18T21:01:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.286+0000 7f8ac3fff700 1 -- 172.21.15.46:0/3069706398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0007780 con 0x55c2107591c0 2024-03-18T21:01:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.287+0000 7f8aec201b80 1 -- 172.21.15.46:0/3069706398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 msgr2=0x55c21075b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.287+0000 7f8aec201b80 1 --2- 172.21.15.46:0/3069706398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 0x55c21075b5b0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009390 tx=0x7f8ad0009c90 comp rx=0 tx=0).stop 2024-03-18T21:01:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.287+0000 7f8aec201b80 1 -- 172.21.15.46:0/3069706398 shutdown_connections 2024-03-18T21:01:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.287+0000 7f8aec201b80 1 --2- 172.21.15.46:0/3069706398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 0x55c21075b5b0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.287+0000 7f8aec201b80 1 --2- 172.21.15.46:0/3069706398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c210749950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.287+0000 7f8aec201b80 1 --2- 172.21.15.46:0/3069706398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c210749e90 0x55c210758c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.287+0000 7f8aec201b80 1 -- 172.21.15.46:0/3069706398 >> 172.21.15.46:0/3069706398 conn(0x55c2106c01c0 msgr2=0x55c210745a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.287+0000 7f8aec201b80 1 -- 172.21.15.46:0/3069706398 shutdown_connections 2024-03-18T21:01:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.287+0000 7f8aec201b80 1 -- 172.21.15.46:0/3069706398 wait complete. 2024-03-18T21:01:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.287+0000 7f8aec201b80 1 -- start start 2024-03-18T21:01:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8aec201b80 1 Processor -- start 2024-03-18T21:01:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8aec201b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749570 0x55c210758b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8aec201b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749e90 0x55c210755e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8aec201b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2107591c0 0x55c2107563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8aec201b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c210638d20 con 0x55c210749570 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8aec201b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2106389c0 con 0x55c210749e90 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8aec201b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c210520640 con 0x55c2107591c0 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8aec201b80 10 client.0 _ll_register_callbacks cb 0x55c2104a4340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8adde08700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749570 0x55c210758b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8adde08700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749570 0x55c210758b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52800/0 (socket says 172.21.15.46:52800) 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8adde08700 1 -- 172.21.15.46:0/3373407060 learned_addr learned my addr 172.21.15.46:0/3373407060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8ade609700 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2107591c0 0x55c2107563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8add607700 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749e90 0x55c210755e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8adde08700 1 -- 172.21.15.46:0/3373407060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2107591c0 msgr2=0x55c2107563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8adde08700 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2107591c0 0x55c2107563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8adde08700 1 -- 172.21.15.46:0/3373407060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749e90 msgr2=0x55c210755e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8adde08700 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749e90 0x55c210755e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.288+0000 7f8adde08700 1 -- 172.21.15.46:0/3373407060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad81a3f30 con 0x55c210749570 2024-03-18T21:01:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8adde08700 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749570 0x55c210758b50 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40969c0 tx=0x7f8ad409a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dd5a33f725fd4b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8ac17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8ac17fa700 1 -- 172.21.15.46:0/3373407060 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad409dd80 con 0x55c210749570 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8ac17fa700 1 -- 172.21.15.46:0/3373407060 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8ad40a23d0 con 0x55c210749570 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8aec201b80 1 -- 172.21.15.46:0/3373407060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c21073d6c0 con 0x55c210749570 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8ac17fa700 1 -- 172.21.15.46:0/3373407060 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ad40b8170 con 0x55c210749570 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8aec201b80 10 client.20844 fetch_fsmap learned FSMap version 106 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8aec201b80 1 -- 172.21.15.46:0/3373407060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=4+}) v3 -- 0x7f8ad810d2f0 con 0x55c210749570 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8ac17fa700 1 -- 172.21.15.46:0/3373407060 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ad4063080 con 0x55c210749570 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8ac17fa700 1 -- 172.21.15.46:0/3373407060 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 106) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8ad4094bd0 con 0x55c210749570 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8aec201b80 10 client.20844 fetch_fsmap finished waiting for FSMap version 106 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8aec201b80 10 client.20844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8aec201b80 20 client.20844 populate_metadata read hostname 'smithi046' 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8aec201b80 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c210768d20 0x55c21076b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.298 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8aec201b80 4 client.20844 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.289+0000 7f8aec201b80 1 -- 172.21.15.46:0/3373407060 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2106956a0 con 0x55c210768d20 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.290+0000 7f8add607700 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c210768d20 0x55c21076b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.290+0000 7f8ac17fa700 10 client.20844 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.290+0000 7f8add607700 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c210768d20 0x55c21076b140 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd475723b8396bc0 server_cookie=6d3c51b97143446a in_seq=0 out_seq=0 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8ac17fa700 1 -- 172.21.15.46:0/3373407060 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c2106956a0 con 0x55c210768d20 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8ac17fa700 10 client.20844 handle_command_reply: tid=0 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8aec201b80 1 client.20844 shutdown 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8aec201b80 1 -- 172.21.15.46:0/3373407060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749570 msgr2=0x55c210758b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8aec201b80 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749570 0x55c210758b50 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40969c0 tx=0x7f8ad409a740 comp rx=0 tx=0).stop 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8aec201b80 1 -- 172.21.15.46:0/3373407060 shutdown_connections 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8aec201b80 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c210768d20 0x55c21076b140 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8aec201b80 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749570 0x55c210758b50 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8aec201b80 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2107591c0 0x55c2107563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8aec201b80 1 --2- 172.21.15.46:0/3373407060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749e90 0x55c210755e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8ac17fa700 0 client.20844 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T21:01:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8aec201b80 1 -- 172.21.15.46:0/3373407060 >> 172.21.15.46:0/3373407060 conn(0x55c2106c01c0 msgr2=0x55c210745650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8aec201b80 1 -- 172.21.15.46:0/3373407060 shutdown_connections 2024-03-18T21:01:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8aec201b80 1 -- 172.21.15.46:0/3373407060 wait complete. 2024-03-18T21:01:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.291+0000 7f8aec201b80 20 client.20844 trim_cache size 0 max 16384 2024-03-18T21:01:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8aec201b80 1 Processor -- start 2024-03-18T21:01:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8aec201b80 1 -- start start 2024-03-18T21:01:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8aec201b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c210749950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8aec201b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c210749e90 0x55c210745bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8aec201b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 0x55c21075fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8aec201b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c210760350 con 0x55c2107591c0 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8aec201b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2107543f0 con 0x55c210749570 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8aec201b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c210751d00 con 0x55c210749e90 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8ade609700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 0x55c21075fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8ade609700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 0x55c21075fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52814/0 (socket says 172.21.15.46:52814) 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8ade609700 1 -- 172.21.15.46:0/1906590266 learned_addr learned my addr 172.21.15.46:0/1906590266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8add607700 1 --2- 172.21.15.46:0/1906590266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c210749e90 0x55c210745bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8adde08700 1 --2- 172.21.15.46:0/1906590266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c210749950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.307+0000 7f8ade609700 1 -- 172.21.15.46:0/1906590266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c210749e90 msgr2=0x55c210745bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8ade609700 1 --2- 172.21.15.46:0/1906590266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c210749e90 0x55c210745bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8ade609700 1 -- 172.21.15.46:0/1906590266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 msgr2=0x55c210749950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8ade609700 1 --2- 172.21.15.46:0/1906590266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c210749950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8ade609700 1 -- 172.21.15.46:0/1906590266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2104cab00 con 0x55c2107591c0 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8ade609700 1 --2- 172.21.15.46:0/1906590266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 0x55c21075fe10 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000b3f0 tx=0x7f8ad0003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd428dc2e3cb2667 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8accff9700 1 -- 172.21.15.46:0/1906590266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000e040 con 0x55c2107591c0 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8accff9700 1 -- 172.21.15.46:0/1906590266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ad0003a00 con 0x55c2107591c0 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8accff9700 1 -- 172.21.15.46:0/1906590266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0005320 con 0x55c2107591c0 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8aec201b80 1 -- 172.21.15.46:0/1906590266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 msgr2=0x55c21075fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8aec201b80 1 --2- 172.21.15.46:0/1906590266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 0x55c21075fe10 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000b3f0 tx=0x7f8ad0003f80 comp rx=0 tx=0).stop 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8aec201b80 1 -- 172.21.15.46:0/1906590266 shutdown_connections 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8aec201b80 1 --2- 172.21.15.46:0/1906590266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2107591c0 0x55c21075fe10 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8aec201b80 1 --2- 172.21.15.46:0/1906590266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c210749950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8aec201b80 1 --2- 172.21.15.46:0/1906590266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c210749e90 0x55c210745bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.308+0000 7f8aec201b80 1 -- 172.21.15.46:0/1906590266 >> 172.21.15.46:0/1906590266 conn(0x55c2106c01c0 msgr2=0x55c210744120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8aec201b80 1 -- 172.21.15.46:0/1906590266 shutdown_connections 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8aec201b80 1 -- 172.21.15.46:0/1906590266 wait complete. 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8aec201b80 1 -- start start 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8aec201b80 1 Processor -- start 2024-03-18T21:01:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8aec201b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c21075bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8aec201b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749e90 0x55c21075c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8aec201b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2107591c0 0x55c2107bbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8aec201b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21075cc00 con 0x55c210749e90 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8aec201b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2107bc080 con 0x55c210749570 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8aec201b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2107bc1c0 con 0x55c2107591c0 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8aec201b80 10 client.0 _ll_register_callbacks cb 0x7f8ad4096380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8add607700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749e90 0x55c21075c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8add607700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749e90 0x55c21075c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52822/0 (socket says 172.21.15.46:52822) 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8add607700 1 -- 172.21.15.46:0/2030675256 learned_addr learned my addr 172.21.15.46:0/2030675256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8adde08700 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c21075bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.309+0000 7f8ade609700 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2107591c0 0x55c2107bbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8add607700 1 -- 172.21.15.46:0/2030675256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2107591c0 msgr2=0x55c2107bbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8add607700 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2107591c0 0x55c2107bbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8add607700 1 -- 172.21.15.46:0/2030675256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 msgr2=0x55c21075bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8add607700 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c21075bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8add607700 1 -- 172.21.15.46:0/2030675256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c21051e350 con 0x55c210749e90 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8adde08700 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c21075bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8add607700 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749e90 0x55c21075c520 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x55c210757410 tx=0x7f8ac80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2238ff4512bb0f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8ac17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8ac17fa700 1 -- 172.21.15.46:0/2030675256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8004230 con 0x55c210749e90 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8ac17fa700 1 -- 172.21.15.46:0/2030675256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8ac8017040 con 0x55c210749e90 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8aec201b80 1 -- 172.21.15.46:0/2030675256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ad0007d30 con 0x55c210749e90 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8ac17fa700 1 -- 172.21.15.46:0/2030675256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ac8004390 con 0x55c210749e90 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8aec201b80 10 client.20850 fetch_fsmap learned FSMap version 106 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.310+0000 7f8aec201b80 1 -- 172.21.15.46:0/2030675256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=4+}) v3 -- 0x55c210662e40 con 0x55c210749e90 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.311+0000 7f8ac17fa700 1 -- 172.21.15.46:0/2030675256 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ad0007d30 con 0x55c210749e90 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.311+0000 7f8ac17fa700 1 -- 172.21.15.46:0/2030675256 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 106) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8ac8003880 con 0x55c210749e90 2024-03-18T21:01:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.311+0000 7f8aec201b80 10 client.20850 fetch_fsmap finished waiting for FSMap version 106 2024-03-18T21:01:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.311+0000 7f8aec201b80 10 client.20850 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T21:01:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.311+0000 7f8aec201b80 20 client.20850 populate_metadata read hostname 'smithi046' 2024-03-18T21:01:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.311+0000 7f8aec201b80 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c2107bda60 0x55c2107bfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.311+0000 7f8aec201b80 4 client.20850 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T21:01:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.311+0000 7f8aec201b80 1 -- 172.21.15.46:0/2030675256 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c2104ebc90 con 0x55c2107bda60 2024-03-18T21:01:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.311+0000 7f8adde08700 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c2107bda60 0x55c2107bfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.311+0000 7f8ac17fa700 10 client.20850 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T21:01:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.311+0000 7f8adde08700 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c2107bda60 0x55c2107bfe80 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee24a0c39503ba2 server_cookie=9ef99563ab99c985 in_seq=0 out_seq=0 2024-03-18T21:01:07.514 INFO:tasks.workunit.client.0.smithi046.stdout: 10 432923 29.99 MB/sec execute 583 sec latency 3069.484 ms 2024-03-18T21:01:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.517+0000 7f8ac17fa700 1 -- 172.21.15.46:0/2030675256 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c2104ebc90 con 0x55c2107bda60 2024-03-18T21:01:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.517+0000 7f8ac17fa700 10 client.20850 handle_command_reply: tid=0 2024-03-18T21:01:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.517+0000 7f8aec201b80 1 client.20850 shutdown 2024-03-18T21:01:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.517+0000 7f8aec201b80 1 -- 172.21.15.46:0/2030675256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749e90 msgr2=0x55c21075c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.517+0000 7f8aec201b80 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749e90 0x55c21075c520 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x55c210757410 tx=0x7f8ac80047c0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.518+0000 7f8aec201b80 1 -- 172.21.15.46:0/2030675256 shutdown_connections 2024-03-18T21:01:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.518+0000 7f8aec201b80 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c2107bda60 0x55c2107bfe80 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.518+0000 7f8aec201b80 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c2107591c0 0x55c2107bbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.518+0000 7f8aec201b80 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c210749570 0x55c21075bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.518+0000 7f8aec201b80 1 --2- 172.21.15.46:0/2030675256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c210749e90 0x55c21075c520 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.518+0000 7f8aec201b80 1 -- 172.21.15.46:0/2030675256 >> 172.21.15.46:0/2030675256 conn(0x55c2106c01c0 msgr2=0x55c210764570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.518+0000 7f8ac17fa700 0 client.20850 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T21:01:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.518+0000 7f8aec201b80 1 -- 172.21.15.46:0/2030675256 shutdown_connections 2024-03-18T21:01:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.519+0000 7f8aec201b80 1 -- 172.21.15.46:0/2030675256 wait complete. 2024-03-18T21:01:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.519+0000 7f8aec201b80 20 client.20850 trim_cache size 0 max 16384 2024-03-18T21:01:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T21:01:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T21:01:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T21:01:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T21:01:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.521+0000 7f8ae006c700 1 -- 172.21.15.46:0/1118352898 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8ac40a1290 msgr2=0x7f8ac40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.521+0000 7f8ae006c700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8ac40a1290 0x7f8ac40a3750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8068870 tx=0x7f8ac80073d0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.521+0000 7f8ae006c700 1 -- 172.21.15.46:0/1118352898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80ffbb0 msgr2=0x7f8ad8198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.521+0000 7f8ae006c700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80ffbb0 0x7f8ad8198070 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400e9d0 tx=0x7f8ad400ece0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.521+0000 7f8ae006c700 1 -- 172.21.15.46:0/1118352898 shutdown_connections 2024-03-18T21:01:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.521+0000 7f8ae006c700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8ac40a1290 0x7f8ac40a3750 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.521+0000 7f8ae006c700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ad80ffbb0 0x7f8ad8198070 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.521+0000 7f8ae006c700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad8105c60 0x7f8ad81a30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.521+0000 7f8ae006c700 1 --2- 172.21.15.46:0/1118352898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ad8100570 0x7f8ad81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.521+0000 7f8ae006c700 1 -- 172.21.15.46:0/1118352898 >> 172.21.15.46:0/1118352898 conn(0x7f8ad8076d50 msgr2=0x7f8ad80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.521+0000 7f8ae006c700 1 -- 172.21.15.46:0/1118352898 shutdown_connections 2024-03-18T21:01:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:07.521+0000 7f8ae006c700 1 -- 172.21.15.46:0/1118352898 wait complete. 2024-03-18T21:01:08.514 INFO:tasks.workunit.client.0.smithi046.stdout: 10 433390 29.97 MB/sec execute 584 sec latency 103.129 ms 2024-03-18T21:01:08.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:08 smithi073 ceph-mon[47384]: pgmap v792: 97 pgs: 97 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 23 MiB/s wr, 673 op/s 2024-03-18T21:01:09.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:08 smithi155 ceph-mon[48344]: pgmap v792: 97 pgs: 97 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 23 MiB/s wr, 673 op/s 2024-03-18T21:01:09.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:08 smithi046 ceph-mon[43495]: pgmap v792: 97 pgs: 97 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 23 MiB/s wr, 673 op/s 2024-03-18T21:01:09.514 INFO:tasks.workunit.client.0.smithi046.stdout: 10 433590 29.93 MB/sec execute 585 sec latency 250.431 ms 2024-03-18T21:01:09.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:09 smithi073 ceph-mon[47384]: osdmap e121: 12 total, 12 up, 12 in 2024-03-18T21:01:10.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:09 smithi155 ceph-mon[48344]: osdmap e121: 12 total, 12 up, 12 in 2024-03-18T21:01:10.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:09 smithi046 ceph-mon[43495]: osdmap e121: 12 total, 12 up, 12 in 2024-03-18T21:01:10.514 INFO:tasks.workunit.client.0.smithi046.stdout: 10 433838 29.90 MB/sec execute 586 sec latency 243.137 ms 2024-03-18T21:01:10.972 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:10 smithi073 ceph-mon[47384]: pgmap v794: 97 pgs: 14 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 71 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 25 MiB/s wr, 746 op/s 2024-03-18T21:01:11.097 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:10 smithi046 ceph-mon[43495]: pgmap v794: 97 pgs: 14 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 71 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 25 MiB/s wr, 746 op/s 2024-03-18T21:01:11.097 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:10 smithi155 ceph-mon[48344]: pgmap v794: 97 pgs: 14 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 71 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 25 MiB/s wr, 746 op/s 2024-03-18T21:01:11.514 INFO:tasks.workunit.client.0.smithi046.stdout: 10 434205 29.89 MB/sec execute 587 sec latency 332.648 ms 2024-03-18T21:01:12.514 INFO:tasks.workunit.client.0.smithi046.stdout: 10 434691 29.88 MB/sec execute 588 sec latency 173.198 ms 2024-03-18T21:01:12.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:12 smithi073 ceph-mon[47384]: pgmap v795: 97 pgs: 18 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 64 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 25 MiB/s wr, 751 op/s 2024-03-18T21:01:13.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:12 smithi155 ceph-mon[48344]: pgmap v795: 97 pgs: 18 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 64 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 25 MiB/s wr, 751 op/s 2024-03-18T21:01:13.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:12 smithi046 ceph-mon[43495]: pgmap v795: 97 pgs: 18 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 64 active+clean; 2.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 25 MiB/s wr, 751 op/s 2024-03-18T21:01:13.514 INFO:tasks.workunit.client.0.smithi046.stdout: 10 435201 29.88 MB/sec execute 589 sec latency 119.063 ms 2024-03-18T21:01:14.514 INFO:tasks.workunit.client.0.smithi046.stdout: 10 435788 29.89 MB/sec execute 590 sec latency 78.182 ms 2024-03-18T21:01:14.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:14 smithi073 ceph-mon[47384]: pgmap v796: 97 pgs: 18 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 64 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 23 MiB/s wr, 755 op/s 2024-03-18T21:01:15.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:14 smithi155 ceph-mon[48344]: pgmap v796: 97 pgs: 18 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 64 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 23 MiB/s wr, 755 op/s 2024-03-18T21:01:15.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:14 smithi046 ceph-mon[43495]: pgmap v796: 97 pgs: 18 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 64 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 23 MiB/s wr, 755 op/s 2024-03-18T21:01:15.514 INFO:tasks.workunit.client.0.smithi046.stdout: 10 436456 29.90 MB/sec execute 591 sec latency 77.679 ms 2024-03-18T21:01:16.515 INFO:tasks.workunit.client.0.smithi046.stdout: 10 437014 29.90 MB/sec execute 592 sec latency 84.338 ms 2024-03-18T21:01:16.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:16 smithi073 ceph-mon[47384]: pgmap v797: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 621 op/s 2024-03-18T21:01:16.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:16 smithi155 ceph-mon[48344]: pgmap v797: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 621 op/s 2024-03-18T21:01:16.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:16 smithi046 ceph-mon[43495]: pgmap v797: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 621 op/s 2024-03-18T21:01:17.514 INFO:tasks.workunit.client.0.smithi046.stdout: 10 437456 29.89 MB/sec execute 593 sec latency 80.770 ms 2024-03-18T21:01:18.515 INFO:tasks.workunit.client.0.smithi046.stdout: 10 437911 29.89 MB/sec execute 594 sec latency 90.396 ms 2024-03-18T21:01:18.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:18 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:18.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:18 smithi073 ceph-mon[47384]: pgmap v798: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.7 MiB/s wr, 483 op/s 2024-03-18T21:01:18.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:18 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:18.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:18 smithi155 ceph-mon[48344]: pgmap v798: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.7 MiB/s wr, 483 op/s 2024-03-18T21:01:18.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:18 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:18.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:18 smithi046 ceph-mon[43495]: pgmap v798: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.7 MiB/s wr, 483 op/s 2024-03-18T21:01:19.515 INFO:tasks.workunit.client.0.smithi046.stdout: 10 438405 29.87 MB/sec execute 595 sec latency 129.960 ms 2024-03-18T21:01:20.518 INFO:tasks.workunit.client.0.smithi046.stdout: 10 439013 29.87 MB/sec execute 596 sec latency 91.128 ms 2024-03-18T21:01:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.688+0000 7f5e08384700 1 -- 172.21.15.46:0/50736069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00105c60 msgr2=0x7f5e00106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.688+0000 7f5e08384700 1 --2- 172.21.15.46:0/50736069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00105c60 0x7f5e00106040 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009990 tx=0x7f5dfc009ca0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.689+0000 7f5e08384700 1 -- 172.21.15.46:0/50736069 shutdown_connections 2024-03-18T21:01:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.689+0000 7f5e08384700 1 --2- 172.21.15.46:0/50736069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00105c60 0x7f5e00106040 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.689+0000 7f5e08384700 1 --2- 172.21.15.46:0/50736069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e00100570 0x7f5e0010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.689+0000 7f5e08384700 1 --2- 172.21.15.46:0/50736069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e000ffbb0 0x7f5e00100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.689+0000 7f5e08384700 1 -- 172.21.15.46:0/50736069 >> 172.21.15.46:0/50736069 conn(0x7f5e00076d50 msgr2=0x7f5e00077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.689+0000 7f5e08384700 1 -- 172.21.15.46:0/50736069 shutdown_connections 2024-03-18T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.689+0000 7f5e08384700 1 -- 172.21.15.46:0/50736069 wait complete. 2024-03-18T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.690+0000 7f5e08384700 1 Processor -- start 2024-03-18T21:01:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.690+0000 7f5e08384700 1 -- start start 2024-03-18T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.690+0000 7f5e08384700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e000ffbb0 0x7f5e00106a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.690+0000 7f5e08384700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00100570 0x7f5e00106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.690+0000 7f5e08384700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e00105c60 0x7f5e00107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.690+0000 7f5e08384700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e001a98a0 con 0x7f5e00100570 2024-03-18T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.690+0000 7f5e08384700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e00198fb0 con 0x7f5e000ffbb0 2024-03-18T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.690+0000 7f5e08384700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e00199120 con 0x7f5e00105c60 2024-03-18T21:01:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.691+0000 7f5e06921700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e00105c60 0x7f5e00107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.691+0000 7f5e06921700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e00105c60 0x7f5e00107490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.46:51686/0 (socket says 172.21.15.46:51686) 2024-03-18T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.691+0000 7f5e06921700 1 -- 172.21.15.46:0/445820221 learned_addr learned my addr 172.21.15.46:0/445820221 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.691+0000 7f5e06120700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e000ffbb0 0x7f5e00106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.693+0000 7f5e06120700 1 -- 172.21.15.46:0/445820221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e00105c60 msgr2=0x7f5e00107490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.693+0000 7f5e06120700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e00105c60 0x7f5e00107490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.693+0000 7f5e06120700 1 -- 172.21.15.46:0/445820221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00100570 msgr2=0x7f5e00106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.693+0000 7f5e06120700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00100570 0x7f5e00106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.693+0000 7f5e06120700 1 -- 172.21.15.46:0/445820221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc0095f0 con 0x7f5e000ffbb0 2024-03-18T21:01:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.694+0000 7f5e06120700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e000ffbb0 0x7f5e00106a10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc006010 tx=0x7f5dfc0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.694+0000 7f5df77fe700 1 -- 172.21.15.46:0/445820221 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc01d070 con 0x7f5e000ffbb0 2024-03-18T21:01:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.694+0000 7f5e08384700 1 -- 172.21.15.46:0/445820221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e001993a0 con 0x7f5e000ffbb0 2024-03-18T21:01:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.694+0000 7f5e08384700 1 -- 172.21.15.46:0/445820221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e001998a0 con 0x7f5e000ffbb0 2024-03-18T21:01:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.694+0000 7f5df77fe700 1 -- 172.21.15.46:0/445820221 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dfc021410 con 0x7f5e000ffbb0 2024-03-18T21:01:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.694+0000 7f5df77fe700 1 -- 172.21.15.46:0/445820221 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc017650 con 0x7f5e000ffbb0 2024-03-18T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.698+0000 7f5df77fe700 1 -- 172.21.15.46:0/445820221 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5dfc00fad0 con 0x7f5e000ffbb0 2024-03-18T21:01:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.698+0000 7f5df77fe700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5dec0a1310 0x7f5dec0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.699+0000 7f5df77fe700 1 -- 172.21.15.46:0/445820221 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5dfc0cdd10 con 0x7f5e000ffbb0 2024-03-18T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.699+0000 7f5e0591f700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5dec0a1310 0x7f5dec0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.699+0000 7f5e14519b80 1 Processor -- start 2024-03-18T21:01:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.699+0000 7f5e14519b80 1 -- start start 2024-03-18T21:01:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.699+0000 7f5e14519b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abd570 0x55c403abd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.699+0000 7f5e14519b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403abde90 0x55c403ac8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.699+0000 7f5e14519b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403ac8db0 0x55c403acb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.699+0000 7f5e14519b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c403a17930 con 0x55c403abd570 2024-03-18T21:01:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.699+0000 7f5e14519b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c403a152f0 con 0x55c403abde90 2024-03-18T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.699+0000 7f5e14519b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c403a10be0 con 0x55c403ac8db0 2024-03-18T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.700+0000 7f5e06120700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abd570 0x55c403abd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.700+0000 7f5e06120700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abd570 0x55c403abd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60076/0 (socket says 172.21.15.46:60076) 2024-03-18T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.700+0000 7f5e06120700 1 -- 172.21.15.46:0/3117360574 learned_addr learned my addr 172.21.15.46:0/3117360574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.700+0000 7f5e06921700 1 --2- 172.21.15.46:0/3117360574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403ac8db0 0x55c403acb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.700+0000 7f5e0591f700 1 --2- 172.21.15.46:0/3117360574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403abde90 0x55c403ac8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.700+0000 7f5e06921700 1 -- 172.21.15.46:0/3117360574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403abde90 msgr2=0x55c403ac8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.700+0000 7f5e06921700 1 --2- 172.21.15.46:0/3117360574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403abde90 0x55c403ac8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.701+0000 7f5e06921700 1 -- 172.21.15.46:0/3117360574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abd570 msgr2=0x55c403abd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.701+0000 7f5e06921700 1 --2- 172.21.15.46:0/3117360574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abd570 0x55c403abd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.701+0000 7f5e06921700 1 -- 172.21.15.46:0/3117360574 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c403892350 con 0x55c403ac8db0 2024-03-18T21:01:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.701+0000 7f5e06921700 1 --2- 172.21.15.46:0/3117360574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403ac8db0 0x55c403acb1a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009c10 tx=0x7f5df8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=815e7c443ee907e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.701+0000 7f5df4da8700 1 -- 172.21.15.46:0/3117360574 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df800e040 con 0x55c403ac8db0 2024-03-18T21:01:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.701+0000 7f5df4da8700 1 -- 172.21.15.46:0/3117360574 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5df800f040 con 0x55c403ac8db0 2024-03-18T21:01:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.701+0000 7f5df4da8700 1 -- 172.21.15.46:0/3117360574 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df80047a0 con 0x55c403ac8db0 2024-03-18T21:01:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.702+0000 7f5e14519b80 1 -- 172.21.15.46:0/3117360574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403ac8db0 msgr2=0x55c403acb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.702+0000 7f5e14519b80 1 --2- 172.21.15.46:0/3117360574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403ac8db0 0x55c403acb1a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009c10 tx=0x7f5df8009920 comp rx=0 tx=0).stop 2024-03-18T21:01:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.702+0000 7f5e0591f700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5dec0a1310 0x7f5dec0a37d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5df0005950 tx=0x7f5df00058e0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.702+0000 7f5e14519b80 1 -- 172.21.15.46:0/3117360574 shutdown_connections 2024-03-18T21:01:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.702+0000 7f5e14519b80 1 --2- 172.21.15.46:0/3117360574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abd570 0x55c403abd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.702+0000 7f5e14519b80 1 --2- 172.21.15.46:0/3117360574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403ac8db0 0x55c403acb1a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.702+0000 7f5e14519b80 1 --2- 172.21.15.46:0/3117360574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403abde90 0x55c403ac8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.702+0000 7f5e14519b80 1 -- 172.21.15.46:0/3117360574 >> 172.21.15.46:0/3117360574 conn(0x55c403a341c0 msgr2=0x55c403ab9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.702+0000 7f5e14519b80 1 -- 172.21.15.46:0/3117360574 shutdown_connections 2024-03-18T21:01:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.702+0000 7f5e14519b80 1 -- 172.21.15.46:0/3117360574 wait complete. 2024-03-18T21:01:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.703+0000 7f5e14519b80 1 -- start start 2024-03-18T21:01:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.703+0000 7f5e14519b80 1 Processor -- start 2024-03-18T21:01:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.703+0000 7f5e14519b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403abd570 0x55c403ac8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.703+0000 7f5e14519b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abde90 0x55c403ac59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.703+0000 7f5e14519b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403ac8db0 0x55c403ac5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.703+0000 7f5e14519b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4039acd20 con 0x55c403ac8db0 2024-03-18T21:01:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.703+0000 7f5e14519b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4039ac9c0 con 0x55c403abd570 2024-03-18T21:01:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.703+0000 7f5e14519b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c403894640 con 0x55c403abde90 2024-03-18T21:01:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.703+0000 7f5e14519b80 10 client.0 _ll_register_callbacks cb 0x55c403818340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T21:01:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.704+0000 7f5e06921700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403ac8db0 0x55c403ac5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.704+0000 7f5e06921700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403ac8db0 0x55c403ac5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60082/0 (socket says 172.21.15.46:60082) 2024-03-18T21:01:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.704+0000 7f5e06921700 1 -- 172.21.15.46:0/3540229347 learned_addr learned my addr 172.21.15.46:0/3540229347 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.704+0000 7f5e06120700 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403abd570 0x55c403ac8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.704+0000 7f5e0591f700 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abde90 0x55c403ac59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.704+0000 7f5e06120700 1 -- 172.21.15.46:0/3540229347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abde90 msgr2=0x55c403ac59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.704+0000 7f5e06120700 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abde90 0x55c403ac59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.704+0000 7f5e06120700 1 -- 172.21.15.46:0/3540229347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403ac8db0 msgr2=0x55c403ac5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.704+0000 7f5e06120700 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403ac8db0 0x55c403ac5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.704+0000 7f5e06120700 1 -- 172.21.15.46:0/3540229347 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df800e330 con 0x55c403abd570 2024-03-18T21:01:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.704+0000 7f5e06120700 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403abd570 0x55c403ac8680 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009990 tx=0x7f5dfc09adf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2da4f8249cd35d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.704+0000 7f5de9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T21:01:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.705+0000 7f5de9ffb700 1 -- 172.21.15.46:0/3540229347 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc09b400 con 0x55c403abd570 2024-03-18T21:01:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.705+0000 7f5de9ffb700 1 -- 172.21.15.46:0/3540229347 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5dfc09b6a0 con 0x55c403abd570 2024-03-18T21:01:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.705+0000 7f5e14519b80 1 -- 172.21.15.46:0/3540229347 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c403ab16c0 con 0x55c403abd570 2024-03-18T21:01:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.706+0000 7f5de9ffb700 1 -- 172.21.15.46:0/3540229347 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dfc0a10b0 con 0x55c403abd570 2024-03-18T21:01:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.709+0000 7f5e14519b80 10 client.30485 fetch_fsmap learned FSMap version 107 2024-03-18T21:01:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.709+0000 7f5e14519b80 1 -- 172.21.15.46:0/3540229347 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x55c40383eb00 con 0x55c403abd570 2024-03-18T21:01:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.709+0000 7f5de9ffb700 1 -- 172.21.15.46:0/3540229347 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c403ab16c0 con 0x55c403abd570 2024-03-18T21:01:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.709+0000 7f5de9ffb700 1 -- 172.21.15.46:0/3540229347 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 107) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5dfc09a490 con 0x55c403abd570 2024-03-18T21:01:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.709+0000 7f5e14519b80 10 client.30485 fetch_fsmap finished waiting for FSMap version 107 2024-03-18T21:01:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.709+0000 7f5e14519b80 10 client.30485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T21:01:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.709+0000 7f5e14519b80 20 client.30485 populate_metadata read hostname 'smithi046' 2024-03-18T21:01:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.710+0000 7f5e14519b80 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c403ad8600 0x55c403adaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.710+0000 7f5e14519b80 4 client.30485 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T21:01:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.710+0000 7f5e14519b80 1 -- 172.21.15.46:0/3540229347 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c403a096a0 con 0x55c403ad8600 2024-03-18T21:01:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.710+0000 7f5e0591f700 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c403ad8600 0x55c403adaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.711+0000 7f5e0591f700 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c403ad8600 0x55c403adaa20 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84aa9763a4ca00e3 server_cookie=870cfe462fe1a84e in_seq=0 out_seq=0 2024-03-18T21:01:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.711+0000 7f5de9ffb700 10 client.30485 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T21:01:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5de9ffb700 1 -- 172.21.15.46:0/3540229347 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c403a096a0 con 0x55c403ad8600 2024-03-18T21:01:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5de9ffb700 10 client.30485 handle_command_reply: tid=0 2024-03-18T21:01:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5e14519b80 1 client.30485 shutdown 2024-03-18T21:01:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5e14519b80 1 -- 172.21.15.46:0/3540229347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403abd570 msgr2=0x55c403ac8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5e14519b80 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403abd570 0x55c403ac8680 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009990 tx=0x7f5dfc09adf0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5e14519b80 1 -- 172.21.15.46:0/3540229347 shutdown_connections 2024-03-18T21:01:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5e14519b80 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c403ad8600 0x55c403adaa20 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5e14519b80 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403ac8db0 0x55c403ac5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5e14519b80 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403abd570 0x55c403ac8680 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5de9ffb700 0 client.30485 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T21:01:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5e14519b80 1 --2- 172.21.15.46:0/3540229347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abde90 0x55c403ac59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5e14519b80 1 -- 172.21.15.46:0/3540229347 >> 172.21.15.46:0/3540229347 conn(0x55c403a341c0 msgr2=0x55c403ab9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5e14519b80 1 -- 172.21.15.46:0/3540229347 shutdown_connections 2024-03-18T21:01:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.713+0000 7f5e14519b80 1 -- 172.21.15.46:0/3540229347 wait complete. 2024-03-18T21:01:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.714+0000 7f5e14519b80 20 client.30485 trim_cache size 0 max 16384 2024-03-18T21:01:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e14519b80 1 Processor -- start 2024-03-18T21:01:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e14519b80 1 -- start start 2024-03-18T21:01:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e14519b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abd570 0x55c403ab9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e14519b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 0x55c403ac42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e14519b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 0x55c403ac47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e14519b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c403ac07f0 con 0x55c403abde90 2024-03-18T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e14519b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c403ac1840 con 0x55c403ac8db0 2024-03-18T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e14519b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c403ab1200 con 0x55c403abd570 2024-03-18T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e0591f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 0x55c403ac42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e0591f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 0x55c403ac42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60096/0 (socket says 172.21.15.46:60096) 2024-03-18T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e0591f700 1 -- 172.21.15.46:0/2399665071 learned_addr learned my addr 172.21.15.46:0/2399665071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e06120700 1 --2- 172.21.15.46:0/2399665071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abd570 0x55c403ab9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.733+0000 7f5e06921700 1 --2- 172.21.15.46:0/2399665071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 0x55c403ac47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.734+0000 7f5e0591f700 1 -- 172.21.15.46:0/2399665071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abd570 msgr2=0x55c403ab9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.734+0000 7f5e0591f700 1 --2- 172.21.15.46:0/2399665071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abd570 0x55c403ab9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.734+0000 7f5e0591f700 1 -- 172.21.15.46:0/2399665071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 msgr2=0x55c403ac47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.734+0000 7f5e0591f700 1 --2- 172.21.15.46:0/2399665071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 0x55c403ac47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.734+0000 7f5e0591f700 1 -- 172.21.15.46:0/2399665071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4039d6e40 con 0x55c403abde90 2024-03-18T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.734+0000 7f5e06921700 1 --2- 172.21.15.46:0/2399665071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 0x55c403ac47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.734+0000 7f5e06120700 1 --2- 172.21.15.46:0/2399665071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abd570 0x55c403ab9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.734+0000 7f5e0591f700 1 --2- 172.21.15.46:0/2399665071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 0x55c403ac42b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x55c403ac6eb0 tx=0x7f5df0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7aa6a510bad9c1f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.734+0000 7f5df55a9700 1 -- 172.21.15.46:0/2399665071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0021040 con 0x55c403abde90 2024-03-18T21:01:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.734+0000 7f5df55a9700 1 -- 172.21.15.46:0/2399665071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5df0027e30 con 0x55c403abde90 2024-03-18T21:01:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.734+0000 7f5df55a9700 1 -- 172.21.15.46:0/2399665071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0004c10 con 0x55c403abde90 2024-03-18T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.735+0000 7f5e14519b80 1 -- 172.21.15.46:0/2399665071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 msgr2=0x55c403ac42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.735+0000 7f5e14519b80 1 --2- 172.21.15.46:0/2399665071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 0x55c403ac42b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x55c403ac6eb0 tx=0x7f5df0015000 comp rx=0 tx=0).stop 2024-03-18T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 -- 172.21.15.46:0/2399665071 shutdown_connections 2024-03-18T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 --2- 172.21.15.46:0/2399665071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 0x55c403ac47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 --2- 172.21.15.46:0/2399665071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abd570 0x55c403ab9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 --2- 172.21.15.46:0/2399665071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 0x55c403ac42b0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 -- 172.21.15.46:0/2399665071 >> 172.21.15.46:0/2399665071 conn(0x55c403a341c0 msgr2=0x55c403ab8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 -- 172.21.15.46:0/2399665071 shutdown_connections 2024-03-18T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 -- 172.21.15.46:0/2399665071 wait complete. 2024-03-18T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 -- start start 2024-03-18T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 Processor -- start 2024-03-18T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abd570 0x55c403acf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 0x55c403acf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 0x55c403acba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c403acc1b0 con 0x55c403abde90 2024-03-18T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c403acc320 con 0x55c403ac8db0 2024-03-18T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.736+0000 7f5e14519b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c403acc4e0 con 0x55c403abd570 2024-03-18T21:01:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e14519b80 10 client.0 _ll_register_callbacks cb 0x55c403818340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e06921700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 0x55c403acba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e06921700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 0x55c403acba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:34546/0 (socket says 172.21.15.46:34546) 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e06921700 1 -- 172.21.15.46:0/1239246386 learned_addr learned my addr 172.21.15.46:0/1239246386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e0591f700 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 0x55c403acf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e06921700 1 -- 172.21.15.46:0/1239246386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abd570 msgr2=0x55c403acf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e06120700 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abd570 0x55c403acf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e06921700 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abd570 0x55c403acf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e06921700 1 -- 172.21.15.46:0/1239246386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 msgr2=0x55c403acf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e06921700 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 0x55c403acf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e06921700 1 -- 172.21.15.46:0/1239246386 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c40383eb00 con 0x55c403ac8db0 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e0591f700 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 0x55c403acf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.737+0000 7f5e06120700 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abd570 0x55c403acf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.738+0000 7f5e06921700 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 0x55c403acba70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009e80 tx=0x7f5df8005b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac355c0215845c69 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.738+0000 7f5de9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-18T21:01:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.738+0000 7f5de9ffb700 1 -- 172.21.15.46:0/1239246386 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df800e040 con 0x55c403ac8db0 2024-03-18T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.738+0000 7f5de9ffb700 1 -- 172.21.15.46:0/1239246386 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f5df8003cd0 con 0x55c403ac8db0 2024-03-18T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.738+0000 7f5e14519b80 1 -- 172.21.15.46:0/1239246386 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5df800f040 con 0x55c403ac8db0 2024-03-18T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.738+0000 7f5de9ffb700 1 -- 172.21.15.46:0/1239246386 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5df8003e90 con 0x55c403ac8db0 2024-03-18T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.738+0000 7f5e14519b80 10 client.30491 fetch_fsmap learned FSMap version 107 2024-03-18T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.738+0000 7f5e14519b80 1 -- 172.21.15.46:0/1239246386 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=4+}) v3 -- 0x55c403892350 con 0x55c403ac8db0 2024-03-18T21:01:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.739+0000 7f5de9ffb700 1 -- 172.21.15.46:0/1239246386 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5df800f040 con 0x55c403ac8db0 2024-03-18T21:01:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.739+0000 7f5de9ffb700 1 -- 172.21.15.46:0/1239246386 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 107) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5df8028450 con 0x55c403ac8db0 2024-03-18T21:01:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.739+0000 7f5e14519b80 10 client.30491 fetch_fsmap finished waiting for FSMap version 107 2024-03-18T21:01:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.739+0000 7f5e14519b80 10 client.30491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-18T21:01:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.739+0000 7f5e14519b80 20 client.30491 populate_metadata read hostname 'smithi046' 2024-03-18T21:01:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.739+0000 7f5e14519b80 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c403b2d5b0 0x55c403b2f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.739+0000 7f5e14519b80 4 client.30491 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T21:01:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.739+0000 7f5e14519b80 1 -- 172.21.15.46:0/1239246386 --> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c40385fc90 con 0x55c403b2d5b0 2024-03-18T21:01:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.739+0000 7f5e06120700 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c403b2d5b0 0x55c403b2f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.739+0000 7f5de9ffb700 10 client.30491 ms_handle_connect on v2:172.21.15.73:6838/2377859422 2024-03-18T21:01:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:20.739+0000 7f5e06120700 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c403b2d5b0 0x55c403b2f9d0 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8f7a666b1bf99b2 server_cookie=7172f7f4201f4525 in_seq=0 out_seq=0 2024-03-18T21:01:20.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:20 smithi046 ceph-mon[43495]: pgmap v799: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.5 MiB/s wr, 535 op/s 2024-03-18T21:01:20.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:20 smithi046 ceph-mon[43495]: osdmap e122: 12 total, 12 up, 12 in 2024-03-18T21:01:20.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:20 smithi073 ceph-mon[47384]: pgmap v799: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.5 MiB/s wr, 535 op/s 2024-03-18T21:01:20.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:20 smithi073 ceph-mon[47384]: osdmap e122: 12 total, 12 up, 12 in 2024-03-18T21:01:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.010+0000 7f5de9ffb700 1 -- 172.21.15.46:0/1239246386 <== mds.0 v2:172.21.15.73:6838/2377859422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c40385fc90 con 0x55c403b2d5b0 2024-03-18T21:01:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.010+0000 7f5de9ffb700 10 client.30491 handle_command_reply: tid=0 2024-03-18T21:01:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.010+0000 7f5e14519b80 1 client.30491 shutdown 2024-03-18T21:01:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.014+0000 7f5e14519b80 1 -- 172.21.15.46:0/1239246386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 msgr2=0x55c403acba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.014+0000 7f5e14519b80 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 0x55c403acba70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009e80 tx=0x7f5df8005b30 comp rx=0 tx=0).stop 2024-03-18T21:01:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.014+0000 7f5e14519b80 1 -- 172.21.15.46:0/1239246386 shutdown_connections 2024-03-18T21:01:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.014+0000 7f5e14519b80 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.73:6838/2377859422,v1:172.21.15.73:6839/2377859422] conn(0x55c403b2d5b0 0x55c403b2f9d0 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.014+0000 7f5e14519b80 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c403ac8db0 0x55c403acba70 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.014+0000 7f5e14519b80 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c403abd570 0x55c403acf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.014+0000 7f5e14519b80 1 --2- 172.21.15.46:0/1239246386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c403abde90 0x55c403acf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.014+0000 7f5de9ffb700 0 client.30491 ms_handle_reset on v2:172.21.15.73:6838/2377859422 2024-03-18T21:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.014+0000 7f5e14519b80 1 -- 172.21.15.46:0/1239246386 >> 172.21.15.46:0/1239246386 conn(0x55c403a341c0 msgr2=0x55c403ac4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.014+0000 7f5e14519b80 1 -- 172.21.15.46:0/1239246386 shutdown_connections 2024-03-18T21:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.014+0000 7f5e14519b80 1 -- 172.21.15.46:0/1239246386 wait complete. 2024-03-18T21:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.014+0000 7f5e14519b80 20 client.30491 trim_cache size 0 max 16384 2024-03-18T21:01:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-03-18T21:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "message": "", 2024-03-18T21:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-03-18T21:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-03-18T21:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.015+0000 7f5e08384700 1 -- 172.21.15.46:0/445820221 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5dec0a1310 msgr2=0x7f5dec0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.015+0000 7f5e08384700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5dec0a1310 0x7f5dec0a37d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5df0005950 tx=0x7f5df00058e0 comp rx=0 tx=0).stop 2024-03-18T21:01:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.015+0000 7f5e08384700 1 -- 172.21.15.46:0/445820221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e000ffbb0 msgr2=0x7f5e00106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.015+0000 7f5e08384700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e000ffbb0 0x7f5e00106a10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc006010 tx=0x7f5dfc0057c0 comp rx=0 tx=0).stop 2024-03-18T21:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.016+0000 7f5e08384700 1 -- 172.21.15.46:0/445820221 shutdown_connections 2024-03-18T21:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.016+0000 7f5e08384700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f5dec0a1310 0x7f5dec0a37d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.016+0000 7f5e08384700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e00105c60 0x7f5e00107490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.016+0000 7f5e08384700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e000ffbb0 0x7f5e00106a10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.016+0000 7f5e08384700 1 --2- 172.21.15.46:0/445820221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00100570 0x7f5e00106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.016+0000 7f5e08384700 1 -- 172.21.15.46:0/445820221 >> 172.21.15.46:0/445820221 conn(0x7f5e00076d50 msgr2=0x7f5e000680b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.016+0000 7f5e08384700 1 -- 172.21.15.46:0/445820221 shutdown_connections 2024-03-18T21:01:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-03-18T21:01:21.021+0000 7f5e08384700 1 -- 172.21.15.46:0/445820221 wait complete. 2024-03-18T21:01:21.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:20 smithi155 ceph-mon[48344]: pgmap v799: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.5 MiB/s wr, 535 op/s 2024-03-18T21:01:21.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:20 smithi155 ceph-mon[48344]: osdmap e122: 12 total, 12 up, 12 in 2024-03-18T21:01:21.515 INFO:tasks.workunit.client.0.smithi046.stdout: 10 439526 29.87 MB/sec execute 597 sec latency 231.041 ms 2024-03-18T21:01:22.515 INFO:tasks.workunit.client.0.smithi046.stdout: 10 440177 29.88 MB/sec execute 598 sec latency 110.620 ms 2024-03-18T21:01:22.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:22 smithi046 ceph-mon[43495]: pgmap v801: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 413 op/s 2024-03-18T21:01:22.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:22 smithi073 ceph-mon[47384]: pgmap v801: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 413 op/s 2024-03-18T21:01:23.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:22 smithi155 ceph-mon[48344]: pgmap v801: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.4 MiB/s wr, 413 op/s 2024-03-18T21:01:23.517 INFO:tasks.workunit.client.0.smithi046.stdout: 10 440759 29.89 MB/sec execute 599 sec latency 107.046 ms 2024-03-18T21:01:24.517 INFO:tasks.workunit.client.0.smithi046.stdout: 10 cleanup 600 sec 2024-03-18T21:01:24.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:24 smithi073 ceph-mon[47384]: pgmap v802: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.3 MiB/s wr, 389 op/s 2024-03-18T21:01:25.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:24 smithi155 ceph-mon[48344]: pgmap v802: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.3 MiB/s wr, 389 op/s 2024-03-18T21:01:25.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:24 smithi046 ceph-mon[43495]: pgmap v802: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.3 MiB/s wr, 389 op/s 2024-03-18T21:01:25.352 INFO:tasks.workunit.client.0.smithi046.stdout: 0 cleanup 601 sec 2024-03-18T21:01:25.352 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-03-18T21:01:25.352 INFO:tasks.workunit.client.0.smithi046.stdout: Operation Count AvgLat MaxLat 2024-03-18T21:01:25.352 INFO:tasks.workunit.client.0.smithi046.stdout: ---------------------------------------- 2024-03-18T21:01:25.352 INFO:tasks.workunit.client.0.smithi046.stdout: NTCreateX 572649 3.970 9533.954 2024-03-18T21:01:25.352 INFO:tasks.workunit.client.0.smithi046.stdout: Close 420619 0.003 0.784 2024-03-18T21:01:25.353 INFO:tasks.workunit.client.0.smithi046.stdout: Rename 24277 16.636 3920.476 2024-03-18T21:01:25.353 INFO:tasks.workunit.client.0.smithi046.stdout: Unlink 115692 4.247 4275.441 2024-03-18T21:01:25.353 INFO:tasks.workunit.client.0.smithi046.stdout: Deltree 6 188.497 380.472 2024-03-18T21:01:25.353 INFO:tasks.workunit.client.0.smithi046.stdout: Mkdir 3 0.008 0.009 2024-03-18T21:01:25.353 INFO:tasks.workunit.client.0.smithi046.stdout: Qpathinfo 519390 3.670 10783.921 2024-03-18T21:01:25.353 INFO:tasks.workunit.client.0.smithi046.stdout: Qfileinfo 90778 0.002 0.035 2024-03-18T21:01:25.353 INFO:tasks.workunit.client.0.smithi046.stdout: Qfsinfo 95288 0.722 129.200 2024-03-18T21:01:25.353 INFO:tasks.workunit.client.0.smithi046.stdout: Sfileinfo 46654 0.206 1032.041 2024-03-18T21:01:25.354 INFO:tasks.workunit.client.0.smithi046.stdout: Find 200873 0.537 8411.471 2024-03-18T21:01:25.354 INFO:tasks.workunit.client.0.smithi046.stdout: WriteX 284090 0.623 1351.616 2024-03-18T21:01:25.354 INFO:tasks.workunit.client.0.smithi046.stdout: ReadX 899475 0.042 1340.959 2024-03-18T21:01:25.354 INFO:tasks.workunit.client.0.smithi046.stdout: LockX 1872 2.589 44.426 2024-03-18T21:01:25.354 INFO:tasks.workunit.client.0.smithi046.stdout: UnlockX 1872 2.291 94.904 2024-03-18T21:01:25.354 INFO:tasks.workunit.client.0.smithi046.stdout: Flush 40132 12.606 1466.519 2024-03-18T21:01:25.355 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-03-18T21:01:25.355 INFO:tasks.workunit.client.0.smithi046.stdout:Throughput 29.8934 MB/sec 10 clients 10 procs max_latency=10783.927 ms 2024-03-18T21:01:25.362 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-18T21:01:25.362 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-18T21:01:26.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:26 smithi073 ceph-mon[47384]: pgmap v803: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.6 MiB/s wr, 512 op/s 2024-03-18T21:01:26.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:26 smithi155 ceph-mon[48344]: pgmap v803: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.6 MiB/s wr, 512 op/s 2024-03-18T21:01:26.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:26 smithi046 ceph-mon[43495]: pgmap v803: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.6 MiB/s wr, 512 op/s 2024-03-18T21:01:27.262 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-03-18T21:01:27.263 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-18T21:01:27.680 DEBUG:teuthology.parallel:result is None 2024-03-18T21:01:27.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-18T21:01:27.725 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-18T21:01:27.725 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-18T21:01:27.735 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi046.front.sepia.ceph.com 2024-03-18T21:01:27.775 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-18T21:01:27.789 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-18T21:01:27.789 INFO:teuthology.task.exec:Executing custom commands... 2024-03-18T21:01:27.789 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-03-18T21:01:27.789 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-18T21:01:27.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-18T21:01:27.897 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-18T21:01:28.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.082+0000 7f7b14790700 1 -- 172.21.15.46:0/2326895545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c102840 msgr2=0x7f7b0c10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.082+0000 7f7b14790700 1 --2- 172.21.15.46:0/2326895545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c102840 0x7f7b0c10ad20 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f7afc009990 tx=0x7f7afc009ca0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.082+0000 7f7b14790700 1 -- 172.21.15.46:0/2326895545 shutdown_connections 2024-03-18T21:01:28.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.082+0000 7f7b14790700 1 --2- 172.21.15.46:0/2326895545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b0c10b2f0 0x7f7b0c10d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.082+0000 7f7b14790700 1 --2- 172.21.15.46:0/2326895545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b0c101f20 0x7f7b0c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.082+0000 7f7b14790700 1 --2- 172.21.15.46:0/2326895545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c102840 0x7f7b0c10ad20 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.082+0000 7f7b14790700 1 -- 172.21.15.46:0/2326895545 >> 172.21.15.46:0/2326895545 conn(0x7f7b0c0fb680 msgr2=0x7f7b0c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:28.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.082+0000 7f7b14790700 1 -- 172.21.15.46:0/2326895545 shutdown_connections 2024-03-18T21:01:28.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.082+0000 7f7b14790700 1 -- 172.21.15.46:0/2326895545 wait complete. 2024-03-18T21:01:28.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b14790700 1 Processor -- start 2024-03-18T21:01:28.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b14790700 1 -- start start 2024-03-18T21:01:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b14790700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b0c101f20 0x7f7b0c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b14790700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c102840 0x7f7b0c1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b14790700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b0c10b2f0 0x7f7b0c19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b14790700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c19c080 con 0x7f7b0c102840 2024-03-18T21:01:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b14790700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c19c1f0 con 0x7f7b0c10b2f0 2024-03-18T21:01:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b14790700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c19c3b0 con 0x7f7b0c101f20 2024-03-18T21:01:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b11d2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c102840 0x7f7b0c1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b11d2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c102840 0x7f7b0c1976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43812/0 (socket says 172.21.15.46:43812) 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b11d2b700 1 -- 172.21.15.46:0/3612612738 learned_addr learned my addr 172.21.15.46:0/3612612738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b1252c700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b0c101f20 0x7f7b0c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.083+0000 7f7b12d2d700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b0c10b2f0 0x7f7b0c19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.084+0000 7f7b11d2b700 1 -- 172.21.15.46:0/3612612738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b0c101f20 msgr2=0x7f7b0c197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.084+0000 7f7b11d2b700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b0c101f20 0x7f7b0c197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.084+0000 7f7b11d2b700 1 -- 172.21.15.46:0/3612612738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b0c10b2f0 msgr2=0x7f7b0c19ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.084+0000 7f7b11d2b700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b0c10b2f0 0x7f7b0c19ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.084+0000 7f7b11d2b700 1 -- 172.21.15.46:0/3612612738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc0095f0 con 0x7f7b0c102840 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.084+0000 7f7b11d2b700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c102840 0x7f7b0c1976a0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f7afc000c00 tx=0x7f7afc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.084+0000 7f7b037fe700 1 -- 172.21.15.46:0/3612612738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc01d070 con 0x7f7b0c102840 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.084+0000 7f7b037fe700 1 -- 172.21.15.46:0/3612612738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7afc022470 con 0x7f7b0c102840 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.084+0000 7f7b14790700 1 -- 172.21.15.46:0/3612612738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c19c630 con 0x7f7b0c102840 2024-03-18T21:01:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.084+0000 7f7b037fe700 1 -- 172.21.15.46:0/3612612738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc017650 con 0x7f7b0c102840 2024-03-18T21:01:28.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.084+0000 7f7b14790700 1 -- 172.21.15.46:0/3612612738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0c1a3ed0 con 0x7f7b0c102840 2024-03-18T21:01:28.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.085+0000 7f7b14790700 1 -- 172.21.15.46:0/3612612738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c106410 con 0x7f7b0c102840 2024-03-18T21:01:28.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.087+0000 7f7b037fe700 1 -- 172.21.15.46:0/3612612738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7afc0225e0 con 0x7f7b0c102840 2024-03-18T21:01:28.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.088+0000 7f7b037fe700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7af80a1310 0x7f7af80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:28.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.088+0000 7f7b1252c700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7af80a1310 0x7f7af80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:28.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.088+0000 7f7b037fe700 1 -- 172.21.15.46:0/3612612738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f7afc0ce380 con 0x7f7b0c102840 2024-03-18T21:01:28.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.088+0000 7f7b1252c700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7af80a1310 0x7f7af80a37d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009c60 tx=0x7f7b08009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:28.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.092+0000 7f7b037fe700 1 -- 172.21.15.46:0/3612612738 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7afc0978c0 con 0x7f7b0c102840 2024-03-18T21:01:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.339+0000 7f7b14790700 1 -- 172.21.15.46:0/3612612738 --> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b0c198460 con 0x7f7af80a1310 2024-03-18T21:01:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.354+0000 7f7b037fe700 1 -- 172.21.15.46:0/3612612738 <== mgr.15498 v2:172.21.15.46:6800/805586292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f7b0c198460 con 0x7f7af80a1310 2024-03-18T21:01:28.357 INFO:teuthology.orchestra.run.smithi046.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-18T00:00:00", "created": "2024-03-18T20:36:08", "first": "2024-03-18T20:37:00", "last": "2024-03-18T21:01:00", "last_pruned": "2024-03-18T21:01:04", "created_count": 25, "pruned_count": 19, "active": true} 2024-03-18T21:01:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.357+0000 7f7b14790700 1 -- 172.21.15.46:0/3612612738 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7af80a1310 msgr2=0x7f7af80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.357+0000 7f7b14790700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7af80a1310 0x7f7af80a37d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009c60 tx=0x7f7b08009260 comp rx=0 tx=0).stop 2024-03-18T21:01:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.357+0000 7f7b14790700 1 -- 172.21.15.46:0/3612612738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c102840 msgr2=0x7f7b0c1976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.357+0000 7f7b14790700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c102840 0x7f7b0c1976a0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f7afc000c00 tx=0x7f7afc004730 comp rx=0 tx=0).stop 2024-03-18T21:01:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.357+0000 7f7b14790700 1 -- 172.21.15.46:0/3612612738 shutdown_connections 2024-03-18T21:01:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.357+0000 7f7b14790700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f7af80a1310 0x7f7af80a37d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.357+0000 7f7b14790700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b0c10b2f0 0x7f7b0c19ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.357+0000 7f7b14790700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b0c101f20 0x7f7b0c197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.357+0000 7f7b14790700 1 --2- 172.21.15.46:0/3612612738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c102840 0x7f7b0c1976a0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.357+0000 7f7b14790700 1 -- 172.21.15.46:0/3612612738 >> 172.21.15.46:0/3612612738 conn(0x7f7b0c0fb680 msgr2=0x7f7b0c0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.357+0000 7f7b14790700 1 -- 172.21.15.46:0/3612612738 shutdown_connections 2024-03-18T21:01:28.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.357+0000 7f7b14790700 1 -- 172.21.15.46:0/3612612738 wait complete. 2024-03-18T21:01:28.373 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-18T21:01:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.509+0000 7f001ea16700 1 -- 172.21.15.46:0/3768990295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00180ffe00 msgr2=0x7f00181001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.509+0000 7f001ea16700 1 --2- 172.21.15.46:0/3768990295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00180ffe00 0x7f00181001e0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f0008009990 tx=0x7f0008009ca0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.509+0000 7f001ea16700 1 -- 172.21.15.46:0/3768990295 shutdown_connections 2024-03-18T21:01:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.509+0000 7f001ea16700 1 --2- 172.21.15.46:0/3768990295 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00180ffe00 0x7f00181001e0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.509+0000 7f001ea16700 1 --2- 172.21.15.46:0/3768990295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0018104f20 0x7f001810caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.509+0000 7f001ea16700 1 --2- 172.21.15.46:0/3768990295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00181007b0 0x7f00181047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.509+0000 7f001ea16700 1 -- 172.21.15.46:0/3768990295 >> 172.21.15.46:0/3768990295 conn(0x7f00180fb680 msgr2=0x7f00180fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.509+0000 7f001ea16700 1 -- 172.21.15.46:0/3768990295 shutdown_connections 2024-03-18T21:01:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.509+0000 7f001ea16700 1 -- 172.21.15.46:0/3768990295 wait complete. 2024-03-18T21:01:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.510+0000 7f001ea16700 1 Processor -- start 2024-03-18T21:01:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.510+0000 7f001ea16700 1 -- start start 2024-03-18T21:01:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.510+0000 7f001ea16700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00180ffe00 0x7f0018101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.510+0000 7f001ea16700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0018104f20 0x7f0018101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.510+0000 7f001ea16700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0018102310 0x7f00181a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.510+0000 7f001ea16700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00181a5a20 con 0x7f0018102310 2024-03-18T21:01:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.510+0000 7f001ea16700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00181a5b90 con 0x7f00180ffe00 2024-03-18T21:01:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.510+0000 7f001ea16700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00181a5d50 con 0x7f0018104f20 2024-03-18T21:01:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f001cfb3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0018102310 0x7f00181a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f001cfb3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0018102310 0x7f00181a5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43832/0 (socket says 172.21.15.46:43832) 2024-03-18T21:01:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f001cfb3700 1 -- 172.21.15.46:0/4110694046 learned_addr learned my addr 172.21.15.46:0/4110694046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f0017fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00180ffe00 0x7f0018101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f00177fe700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0018104f20 0x7f0018101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f001cfb3700 1 -- 172.21.15.46:0/4110694046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0018104f20 msgr2=0x7f0018101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f001cfb3700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0018104f20 0x7f0018101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f001cfb3700 1 -- 172.21.15.46:0/4110694046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00180ffe00 msgr2=0x7f0018101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f001cfb3700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00180ffe00 0x7f0018101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f001cfb3700 1 -- 172.21.15.46:0/4110694046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00080095f0 con 0x7f0018102310 2024-03-18T21:01:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f0017fff700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00180ffe00 0x7f0018101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T21:01:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f001cfb3700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0018102310 0x7f00181a5450 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f001000ef30 tx=0x7f001000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f00157fa700 1 -- 172.21.15.46:0/4110694046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010018070 con 0x7f0018102310 2024-03-18T21:01:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f00157fa700 1 -- 172.21.15.46:0/4110694046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001000f040 con 0x7f0018102310 2024-03-18T21:01:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f001ea16700 1 -- 172.21.15.46:0/4110694046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00181a5fd0 con 0x7f0018102310 2024-03-18T21:01:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f00157fa700 1 -- 172.21.15.46:0/4110694046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001000cd00 con 0x7f0018102310 2024-03-18T21:01:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.511+0000 7f001ea16700 1 -- 172.21.15.46:0/4110694046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00181a64b0 con 0x7f0018102310 2024-03-18T21:01:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.512+0000 7f001ea16700 1 -- 172.21.15.46:0/4110694046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00181091a0 con 0x7f0018102310 2024-03-18T21:01:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.514+0000 7f00157fa700 1 -- 172.21.15.46:0/4110694046 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0010003a00 con 0x7f0018102310 2024-03-18T21:01:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.514+0000 7f00157fa700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f00040a1310 0x7f00040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:28.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.514+0000 7f00157fa700 1 -- 172.21.15.46:0/4110694046 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f00100cd0c0 con 0x7f0018102310 2024-03-18T21:01:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.514+0000 7f0017fff700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f00040a1310 0x7f00040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.515+0000 7f0017fff700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f00040a1310 0x7f00040a37d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f000800b2e0 tx=0x7f0008005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.518+0000 7f00157fa700 1 -- 172.21.15.46:0/4110694046 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0010096580 con 0x7f0018102310 2024-03-18T21:01:28.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.762+0000 7f001ea16700 1 -- 172.21.15.46:0/4110694046 --> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0018102d70 con 0x7f00040a1310 2024-03-18T21:01:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.777+0000 7f00157fa700 1 -- 172.21.15.46:0/4110694046 <== mgr.15498 v2:172.21.15.46:6800/805586292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f0018102d70 con 0x7f00040a1310 2024-03-18T21:01:28.780 INFO:teuthology.orchestra.run.smithi046.stdout:/ 1m 6m3h 2024-03-18T21:01:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.780+0000 7f001ea16700 1 -- 172.21.15.46:0/4110694046 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f00040a1310 msgr2=0x7f00040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.780+0000 7f001ea16700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f00040a1310 0x7f00040a37d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f000800b2e0 tx=0x7f0008005960 comp rx=0 tx=0).stop 2024-03-18T21:01:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.780+0000 7f001ea16700 1 -- 172.21.15.46:0/4110694046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0018102310 msgr2=0x7f00181a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.780+0000 7f001ea16700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0018102310 0x7f00181a5450 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f001000ef30 tx=0x7f001000c3a0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.780+0000 7f001ea16700 1 -- 172.21.15.46:0/4110694046 shutdown_connections 2024-03-18T21:01:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.780+0000 7f001ea16700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f00040a1310 0x7f00040a37d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.780+0000 7f001ea16700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0018102310 0x7f00181a5450 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.780+0000 7f001ea16700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00180ffe00 0x7f0018101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.780+0000 7f001ea16700 1 --2- 172.21.15.46:0/4110694046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0018104f20 0x7f0018101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.780+0000 7f001ea16700 1 -- 172.21.15.46:0/4110694046 >> 172.21.15.46:0/4110694046 conn(0x7f00180fb680 msgr2=0x7f0018069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.780+0000 7f001ea16700 1 -- 172.21.15.46:0/4110694046 shutdown_connections 2024-03-18T21:01:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.780+0000 7f001ea16700 1 -- 172.21.15.46:0/4110694046 wait complete. 2024-03-18T21:01:28.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:28 smithi046 ceph-mon[43495]: pgmap v804: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.2 MiB/s wr, 491 op/s 2024-03-18T21:01:28.796 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-18T21:01:28.806 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-18T21:01:28.806 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_49bdc51142759622a281d3e5327eebd07cbb8bb3/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph-c_49bdc51142759622a281d3e5327eebd07cbb8bb3/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-18T21:01:28.807 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-18T21:01:28.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T21:01:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.954+0000 7f6a816d0700 1 -- 172.21.15.46:0/4157658039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c101e30 msgr2=0x7f6a7c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.954+0000 7f6a816d0700 1 --2- 172.21.15.46:0/4157658039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c101e30 0x7f6a7c102210 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009990 tx=0x7f6a64009ca0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.954+0000 7f6a816d0700 1 -- 172.21.15.46:0/4157658039 shutdown_connections 2024-03-18T21:01:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.954+0000 7f6a816d0700 1 --2- 172.21.15.46:0/4157658039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c101e30 0x7f6a7c102210 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.954+0000 7f6a816d0700 1 --2- 172.21.15.46:0/4157658039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a7c10b200 0x7f6a7c10d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.954+0000 7f6a816d0700 1 --2- 172.21.15.46:0/4157658039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a7c102750 0x7f6a7c10ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.954+0000 7f6a816d0700 1 -- 172.21.15.46:0/4157658039 >> 172.21.15.46:0/4157658039 conn(0x7f6a7c0fb5f0 msgr2=0x7f6a7c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.954+0000 7f6a816d0700 1 -- 172.21.15.46:0/4157658039 shutdown_connections 2024-03-18T21:01:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.954+0000 7f6a816d0700 1 -- 172.21.15.46:0/4157658039 wait complete. 2024-03-18T21:01:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.955+0000 7f6a816d0700 1 Processor -- start 2024-03-18T21:01:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.955+0000 7f6a816d0700 1 -- start start 2024-03-18T21:01:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.955+0000 7f6a816d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a7c101e30 0x7f6a7c0ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.955+0000 7f6a816d0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c102750 0x7f6a7c0fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.955+0000 7f6a816d0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a7c10b200 0x7f6a7c100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.955+0000 7f6a816d0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1a5480 con 0x7f6a7c102750 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.955+0000 7f6a816d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1a55f0 con 0x7f6a7c101e30 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.955+0000 7f6a816d0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1a57b0 con 0x7f6a7c10b200 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.955+0000 7f6a7a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c102750 0x7f6a7c0fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.955+0000 7f6a7a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c102750 0x7f6a7c0fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43838/0 (socket says 172.21.15.46:43838) 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.955+0000 7f6a7a7fc700 1 -- 172.21.15.46:0/1817045554 learned_addr learned my addr 172.21.15.46:0/1817045554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.955+0000 7f6a7affd700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a7c101e30 0x7f6a7c0ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.956+0000 7f6a7b7fe700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a7c10b200 0x7f6a7c100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.956+0000 7f6a7a7fc700 1 -- 172.21.15.46:0/1817045554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a7c10b200 msgr2=0x7f6a7c100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.956+0000 7f6a7a7fc700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a7c10b200 0x7f6a7c100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.956+0000 7f6a7a7fc700 1 -- 172.21.15.46:0/1817045554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a7c101e30 msgr2=0x7f6a7c0ffa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.956+0000 7f6a7a7fc700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a7c101e30 0x7f6a7c0ffa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.956+0000 7f6a7a7fc700 1 -- 172.21.15.46:0/1817045554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a640095f0 con 0x7f6a7c102750 2024-03-18T21:01:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.956+0000 7f6a7a7fc700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c102750 0x7f6a7c0fffc0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00d950 tx=0x7f6a6c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.956+0000 7f6a73fff700 1 -- 172.21.15.46:0/1817045554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c009940 con 0x7f6a7c102750 2024-03-18T21:01:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.956+0000 7f6a816d0700 1 -- 172.21.15.46:0/1817045554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c1a5a90 con 0x7f6a7c102750 2024-03-18T21:01:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.956+0000 7f6a73fff700 1 -- 172.21.15.46:0/1817045554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a6c010460 con 0x7f6a7c102750 2024-03-18T21:01:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.956+0000 7f6a73fff700 1 -- 172.21.15.46:0/1817045554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00f620 con 0x7f6a7c102750 2024-03-18T21:01:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.956+0000 7f6a816d0700 1 -- 172.21.15.46:0/1817045554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c1a5ff0 con 0x7f6a7c102750 2024-03-18T21:01:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.957+0000 7f6a816d0700 1 -- 172.21.15.46:0/1817045554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c106380 con 0x7f6a7c102750 2024-03-18T21:01:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.958+0000 7f6a73fff700 1 -- 172.21.15.46:0/1817045554 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6a6c0108d0 con 0x7f6a7c102750 2024-03-18T21:01:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.959+0000 7f6a73fff700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6a680a1290 0x7f6a680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.959+0000 7f6a73fff700 1 -- 172.21.15.46:0/1817045554 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f6a6c0ccbb0 con 0x7f6a7c102750 2024-03-18T21:01:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.959+0000 7f6a7affd700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6a680a1290 0x7f6a680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.960+0000 7f6a7affd700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6a680a1290 0x7f6a680a3750 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400b5c0 tx=0x7f6a64005fd0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:28.962+0000 7f6a73fff700 1 -- 172.21.15.46:0/1817045554 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6a6c095f80 con 0x7f6a7c102750 2024-03-18T21:01:28.968 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:28 smithi073 ceph-mon[47384]: pgmap v804: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.2 MiB/s wr, 491 op/s 2024-03-18T21:01:29.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:28 smithi155 ceph-mon[48344]: pgmap v804: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.2 MiB/s wr, 491 op/s 2024-03-18T21:01:29.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.192+0000 7f6a816d0700 1 -- 172.21.15.46:0/1817045554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a7c100f60 con 0x7f6a7c102750 2024-03-18T21:01:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.194+0000 7f6a73fff700 1 -- 172.21.15.46:0/1817045554 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+23036 (secure 0 0 0) 0x7f6a6c016090 con 0x7f6a7c102750 2024-03-18T21:01:29.196 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T21:01:29.196 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":122,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T21:01:20.339788+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/3400106473":"2024-03-19T20:36:06.973937+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2587067898":"2024-03-19T20:36:06.973937+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/3248305722":"2024-03-19T20:36:06.973937+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/604593037":"2024-03-19T20:36:06.973937+0000","172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1017695887":"2024-03-19T20:36:06.973937+0000","172.21.15.46:0/2204627135":"2024-03-19T20:36:06.973937+0000","172.21.15.46:6801/3400106473":"2024-03-19T20:36:06.973937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T21:01:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.196+0000 7f6a816d0700 1 -- 172.21.15.46:0/1817045554 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6a680a1290 msgr2=0x7f6a680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.196+0000 7f6a816d0700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6a680a1290 0x7f6a680a3750 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400b5c0 tx=0x7f6a64005fd0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.196+0000 7f6a816d0700 1 -- 172.21.15.46:0/1817045554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c102750 msgr2=0x7f6a7c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.196+0000 7f6a816d0700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c102750 0x7f6a7c0fffc0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00d950 tx=0x7f6a6c00dc60 comp rx=0 tx=0).stop 2024-03-18T21:01:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.196+0000 7f6a816d0700 1 -- 172.21.15.46:0/1817045554 shutdown_connections 2024-03-18T21:01:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.196+0000 7f6a816d0700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f6a680a1290 0x7f6a680a3750 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.196+0000 7f6a816d0700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a7c10b200 0x7f6a7c100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.196+0000 7f6a816d0700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a7c101e30 0x7f6a7c0ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.196+0000 7f6a816d0700 1 --2- 172.21.15.46:0/1817045554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7c102750 0x7f6a7c0fffc0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.196+0000 7f6a816d0700 1 -- 172.21.15.46:0/1817045554 >> 172.21.15.46:0/1817045554 conn(0x7f6a7c0fb5f0 msgr2=0x7f6a7c0fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:29.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.197+0000 7f6a816d0700 1 -- 172.21.15.46:0/1817045554 shutdown_connections 2024-03-18T21:01:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.197+0000 7f6a816d0700 1 -- 172.21.15.46:0/1817045554 wait complete. 2024-03-18T21:01:29.213 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T20:31:27.488318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T20:34:37.213560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T20:34:38.162692+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 121, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T21:01:29.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T21:01:29.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.358+0000 7f3fa41d0700 1 -- 172.21.15.46:0/3170928836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f9c102790 msgr2=0x7f3f9c10ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:29.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.358+0000 7f3fa41d0700 1 --2- 172.21.15.46:0/3170928836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f9c102790 0x7f3f9c10ac70 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009990 tx=0x7f3f8c009ca0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.358+0000 7f3fa41d0700 1 -- 172.21.15.46:0/3170928836 shutdown_connections 2024-03-18T21:01:29.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.358+0000 7f3fa41d0700 1 --2- 172.21.15.46:0/3170928836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f9c10b240 0x7f3f9c10d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.358+0000 7f3fa41d0700 1 --2- 172.21.15.46:0/3170928836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f9c101e70 0x7f3f9c102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.358+0000 7f3fa41d0700 1 --2- 172.21.15.46:0/3170928836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f9c102790 0x7f3f9c10ac70 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.358+0000 7f3fa41d0700 1 -- 172.21.15.46:0/3170928836 >> 172.21.15.46:0/3170928836 conn(0x7f3f9c0fb5b0 msgr2=0x7f3f9c0fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:29.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.358+0000 7f3fa41d0700 1 -- 172.21.15.46:0/3170928836 shutdown_connections 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.358+0000 7f3fa41d0700 1 -- 172.21.15.46:0/3170928836 wait complete. 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.358+0000 7f3fa41d0700 1 Processor -- start 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa41d0700 1 -- start start 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa41d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f9c101e70 0x7f3f9c197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa41d0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f9c102790 0x7f3f9c197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa41d0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f9c10b240 0x7f3f9c19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa41d0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c19bf70 con 0x7f3f9c10b240 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa41d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c19c0e0 con 0x7f3f9c101e70 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa41d0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c19c2a0 con 0x7f3f9c102790 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa276d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f9c10b240 0x7f3f9c19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa276d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f9c10b240 0x7f3f9c19b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43856/0 (socket says 172.21.15.46:43856) 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa276d700 1 -- 172.21.15.46:0/488012348 learned_addr learned my addr 172.21.15.46:0/488012348 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:29.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa1f6c700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f9c101e70 0x7f3f9c197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa176b700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f9c102790 0x7f3f9c197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa276d700 1 -- 172.21.15.46:0/488012348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f9c102790 msgr2=0x7f3f9c197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa276d700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f9c102790 0x7f3f9c197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa276d700 1 -- 172.21.15.46:0/488012348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f9c101e70 msgr2=0x7f3f9c197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa276d700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f9c101e70 0x7f3f9c197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.359+0000 7f3fa276d700 1 -- 172.21.15.46:0/488012348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8c0095f0 con 0x7f3f9c10b240 2024-03-18T21:01:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.360+0000 7f3fa276d700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f9c10b240 0x7f3f9c19b930 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f3f94007ee0 tx=0x7f3f9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.360+0000 7f3f92ffd700 1 -- 172.21.15.46:0/488012348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94017070 con 0x7f3f9c10b240 2024-03-18T21:01:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.360+0000 7f3f92ffd700 1 -- 172.21.15.46:0/488012348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f9400f040 con 0x7f3f9c10b240 2024-03-18T21:01:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.360+0000 7f3fa41d0700 1 -- 172.21.15.46:0/488012348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9c19c580 con 0x7f3f9c10b240 2024-03-18T21:01:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.360+0000 7f3f92ffd700 1 -- 172.21.15.46:0/488012348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94005550 con 0x7f3f9c10b240 2024-03-18T21:01:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.360+0000 7f3fa41d0700 1 -- 172.21.15.46:0/488012348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9c1a3dc0 con 0x7f3f9c10b240 2024-03-18T21:01:29.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.361+0000 7f3fa41d0700 1 -- 172.21.15.46:0/488012348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f9c106360 con 0x7f3f9c10b240 2024-03-18T21:01:29.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.362+0000 7f3f92ffd700 1 -- 172.21.15.46:0/488012348 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3f9400db20 con 0x7f3f9c10b240 2024-03-18T21:01:29.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.363+0000 7f3f92ffd700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f880a1300 0x7f3f880a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:29.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.363+0000 7f3fa1f6c700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f880a1300 0x7f3f880a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:29.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.363+0000 7f3f92ffd700 1 -- 172.21.15.46:0/488012348 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3f940cc390 con 0x7f3f9c10b240 2024-03-18T21:01:29.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.363+0000 7f3fa1f6c700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f880a1300 0x7f3f880a37c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800ac50 tx=0x7f3f9800a380 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:29.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.367+0000 7f3f92ffd700 1 -- 172.21.15.46:0/488012348 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3f94095760 con 0x7f3f9c10b240 2024-03-18T21:01:29.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.629+0000 7f3fa41d0700 1 -- 172.21.15.46:0/488012348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3f9c198350 con 0x7f3f9c10b240 2024-03-18T21:01:29.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.629+0000 7f3f92ffd700 1 -- 172.21.15.46:0/488012348 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3f94098d80 con 0x7f3f9c10b240 2024-03-18T21:01:29.633 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-18T21:01:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.631+0000 7f3fa41d0700 1 -- 172.21.15.46:0/488012348 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f880a1300 msgr2=0x7f3f880a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.631+0000 7f3fa41d0700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f880a1300 0x7f3f880a37c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800ac50 tx=0x7f3f9800a380 comp rx=0 tx=0).stop 2024-03-18T21:01:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.631+0000 7f3fa41d0700 1 -- 172.21.15.46:0/488012348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f9c10b240 msgr2=0x7f3f9c19b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.631+0000 7f3fa41d0700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f9c10b240 0x7f3f9c19b930 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f3f94007ee0 tx=0x7f3f9400d3b0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.631+0000 7f3fa41d0700 1 -- 172.21.15.46:0/488012348 shutdown_connections 2024-03-18T21:01:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.631+0000 7f3fa41d0700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3f880a1300 0x7f3f880a37c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.631+0000 7f3fa41d0700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f9c10b240 0x7f3f9c19b930 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.631+0000 7f3fa41d0700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f9c101e70 0x7f3f9c197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.631+0000 7f3fa41d0700 1 --2- 172.21.15.46:0/488012348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3f9c102790 0x7f3f9c197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.631+0000 7f3fa41d0700 1 -- 172.21.15.46:0/488012348 >> 172.21.15.46:0/488012348 conn(0x7f3f9c0fb5b0 msgr2=0x7f3f9c0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:29.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.632+0000 7f3fa41d0700 1 -- 172.21.15.46:0/488012348 shutdown_connections 2024-03-18T21:01:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.632+0000 7f3fa41d0700 1 -- 172.21.15.46:0/488012348 wait complete. 2024-03-18T21:01:29.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T21:01:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.794+0000 7f252fa4c700 1 -- 172.21.15.46:0/34065378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25280731e0 msgr2=0x7f25280735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.794+0000 7f252fa4c700 1 --2- 172.21.15.46:0/34065378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25280731e0 0x7f25280735c0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f252000b0a0 tx=0x7f252000b3b0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.794+0000 7f252fa4c700 1 -- 172.21.15.46:0/34065378 shutdown_connections 2024-03-18T21:01:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.794+0000 7f252fa4c700 1 --2- 172.21.15.46:0/34065378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25280731e0 0x7f25280735c0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.794+0000 7f252fa4c700 1 --2- 172.21.15.46:0/34065378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f252810d530 0x7f252810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.794+0000 7f252fa4c700 1 --2- 172.21.15.46:0/34065378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2528073b00 0x7f252810cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.794+0000 7f252fa4c700 1 -- 172.21.15.46:0/34065378 >> 172.21.15.46:0/34065378 conn(0x7f25280fc7c0 msgr2=0x7f25280febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.795+0000 7f252fa4c700 1 -- 172.21.15.46:0/34065378 shutdown_connections 2024-03-18T21:01:29.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.795+0000 7f252fa4c700 1 -- 172.21.15.46:0/34065378 wait complete. 2024-03-18T21:01:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.795+0000 7f252fa4c700 1 Processor -- start 2024-03-18T21:01:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.795+0000 7f252fa4c700 1 -- start start 2024-03-18T21:01:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.795+0000 7f252fa4c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25280731e0 0x7f2528199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.795+0000 7f252fa4c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2528073b00 0x7f2528199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.795+0000 7f252fa4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f252810d530 0x7f252819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.795+0000 7f252fa4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252819e460 con 0x7f252810d530 2024-03-18T21:01:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.795+0000 7f252fa4c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252819e5d0 con 0x7f25280731e0 2024-03-18T21:01:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.795+0000 7f252fa4c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252819e790 con 0x7f2528073b00 2024-03-18T21:01:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f252dfe9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f252810d530 0x7f252819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f252cfe7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2528073b00 0x7f2528199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f252dfe9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f252810d530 0x7f252819de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43880/0 (socket says 172.21.15.46:43880) 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f252d7e8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25280731e0 0x7f2528199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f252dfe9700 1 -- 172.21.15.46:0/1867709400 learned_addr learned my addr 172.21.15.46:0/1867709400 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f252dfe9700 1 -- 172.21.15.46:0/1867709400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2528073b00 msgr2=0x7f2528199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f252dfe9700 1 --2- 172.21.15.46:0/1867709400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2528073b00 0x7f2528199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f252dfe9700 1 -- 172.21.15.46:0/1867709400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25280731e0 msgr2=0x7f2528199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f252dfe9700 1 --2- 172.21.15.46:0/1867709400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25280731e0 0x7f2528199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f252dfe9700 1 -- 172.21.15.46:0/1867709400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2520009d00 con 0x7f252810d530 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f252dfe9700 1 --2- 172.21.15.46:0/1867709400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f252810d530 0x7f252819de20 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f252400d980 tx=0x7f252400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f251e7fc700 1 -- 172.21.15.46:0/1867709400 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524004a30 con 0x7f252810d530 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f251e7fc700 1 -- 172.21.15.46:0/1867709400 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25240054e0 con 0x7f252810d530 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f251e7fc700 1 -- 172.21.15.46:0/1867709400 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252400fbc0 con 0x7f252810d530 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.796+0000 7f252fa4c700 1 -- 172.21.15.46:0/1867709400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252819ea70 con 0x7f252810d530 2024-03-18T21:01:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.797+0000 7f252fa4c700 1 -- 172.21.15.46:0/1867709400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25281a62b0 con 0x7f252810d530 2024-03-18T21:01:29.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.797+0000 7f252fa4c700 1 -- 172.21.15.46:0/1867709400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2528108740 con 0x7f252810d530 2024-03-18T21:01:29.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.799+0000 7f251e7fc700 1 -- 172.21.15.46:0/1867709400 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2524004c20 con 0x7f252810d530 2024-03-18T21:01:29.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.799+0000 7f251e7fc700 1 --2- 172.21.15.46:0/1867709400 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f25140a1310 0x7f25140a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:29.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.800+0000 7f251e7fc700 1 -- 172.21.15.46:0/1867709400 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f25240ccd90 con 0x7f252810d530 2024-03-18T21:01:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.800+0000 7f252d7e8700 1 --2- 172.21.15.46:0/1867709400 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f25140a1310 0x7f25140a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.800+0000 7f252d7e8700 1 --2- 172.21.15.46:0/1867709400 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f25140a1310 0x7f25140a37d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f252000b830 tx=0x7f2520006210 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:29.803+0000 7f251e7fc700 1 -- 172.21.15.46:0/1867709400 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2524096240 con 0x7f252810d530 2024-03-18T21:01:29.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:29 smithi073 ceph-mon[47384]: from='client.20871 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T21:01:29.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:29 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:29.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:29 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1817045554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T21:01:30.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.025+0000 7f252fa4c700 1 -- 172.21.15.46:0/1867709400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f252819a840 con 0x7f252810d530 2024-03-18T21:01:30.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.026+0000 7f251e7fc700 1 -- 172.21.15.46:0/1867709400 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25240170a0 con 0x7f252810d530 2024-03-18T21:01:30.029 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-18T21:01:30.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.028+0000 7f252fa4c700 1 -- 172.21.15.46:0/1867709400 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f25140a1310 msgr2=0x7f25140a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:30.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.028+0000 7f252fa4c700 1 --2- 172.21.15.46:0/1867709400 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f25140a1310 0x7f25140a37d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f252000b830 tx=0x7f2520006210 comp rx=0 tx=0).stop 2024-03-18T21:01:30.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.028+0000 7f252fa4c700 1 -- 172.21.15.46:0/1867709400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f252810d530 msgr2=0x7f252819de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:30.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.028+0000 7f252fa4c700 1 --2- 172.21.15.46:0/1867709400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f252810d530 0x7f252819de20 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f252400d980 tx=0x7f252400dc90 comp rx=0 tx=0).stop 2024-03-18T21:01:30.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.028+0000 7f252fa4c700 1 -- 172.21.15.46:0/1867709400 shutdown_connections 2024-03-18T21:01:30.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.028+0000 7f252fa4c700 1 --2- 172.21.15.46:0/1867709400 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f25140a1310 0x7f25140a37d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.028+0000 7f252fa4c700 1 --2- 172.21.15.46:0/1867709400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f252810d530 0x7f252819de20 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.028+0000 7f252fa4c700 1 --2- 172.21.15.46:0/1867709400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25280731e0 0x7f2528199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.028+0000 7f252fa4c700 1 --2- 172.21.15.46:0/1867709400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2528073b00 0x7f2528199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.028+0000 7f252fa4c700 1 -- 172.21.15.46:0/1867709400 >> 172.21.15.46:0/1867709400 conn(0x7f25280fc7c0 msgr2=0x7f25280fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:30.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.028+0000 7f252fa4c700 1 -- 172.21.15.46:0/1867709400 shutdown_connections 2024-03-18T21:01:30.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.028+0000 7f252fa4c700 1 -- 172.21.15.46:0/1867709400 wait complete. 2024-03-18T21:01:30.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:29 smithi155 ceph-mon[48344]: from='client.20871 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T21:01:30.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:29 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:30.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:29 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1817045554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T21:01:30.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:29 smithi046 ceph-mon[43495]: from='client.20871 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T21:01:30.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:29 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:30.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:29 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1817045554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T21:01:30.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T21:01:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.192+0000 7f8df78f1700 1 -- 172.21.15.46:0/3608570734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df0100800 msgr2=0x7f8df0104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.192+0000 7f8df78f1700 1 --2- 172.21.15.46:0/3608570734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df0100800 0x7f8df0104830 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009990 tx=0x7f8de4009ca0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.192+0000 7f8df78f1700 1 -- 172.21.15.46:0/3608570734 shutdown_connections 2024-03-18T21:01:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.192+0000 7f8df78f1700 1 --2- 172.21.15.46:0/3608570734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8df0104f60 0x7f8df010cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.192+0000 7f8df78f1700 1 --2- 172.21.15.46:0/3608570734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df00ffe50 0x7f8df0100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.192+0000 7f8df78f1700 1 --2- 172.21.15.46:0/3608570734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df0100800 0x7f8df0104830 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.192+0000 7f8df78f1700 1 -- 172.21.15.46:0/3608570734 >> 172.21.15.46:0/3608570734 conn(0x7f8df0076d40 msgr2=0x7f8df0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.192+0000 7f8df78f1700 1 -- 172.21.15.46:0/3608570734 shutdown_connections 2024-03-18T21:01:30.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.192+0000 7f8df78f1700 1 -- 172.21.15.46:0/3608570734 wait complete. 2024-03-18T21:01:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.193+0000 7f8df78f1700 1 Processor -- start 2024-03-18T21:01:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.193+0000 7f8df78f1700 1 -- start start 2024-03-18T21:01:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.193+0000 7f8df78f1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df00ffe50 0x7f8df0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.193+0000 7f8df78f1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8df0104f60 0x7f8df0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.193+0000 7f8df78f1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df0198a90 0x7f8df01a3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df78f1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df01a3730 con 0x7f8df00ffe50 2024-03-18T21:01:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df78f1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df01a38a0 con 0x7f8df0104f60 2024-03-18T21:01:30.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df78f1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df01a3a60 con 0x7f8df0198a90 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df568d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df00ffe50 0x7f8df0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df568d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df00ffe50 0x7f8df0198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43890/0 (socket says 172.21.15.46:43890) 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df568d700 1 -- 172.21.15.46:0/2373433836 learned_addr learned my addr 172.21.15.46:0/2373433836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df4e8c700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8df0104f60 0x7f8df0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df5e8e700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df0198a90 0x7f8df01a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df568d700 1 -- 172.21.15.46:0/2373433836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df0198a90 msgr2=0x7f8df01a3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df568d700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df0198a90 0x7f8df01a3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df568d700 1 -- 172.21.15.46:0/2373433836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8df0104f60 msgr2=0x7f8df0198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df568d700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8df0104f60 0x7f8df0198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df568d700 1 -- 172.21.15.46:0/2373433836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de40095f0 con 0x7f8df00ffe50 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df5e8e700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df0198a90 0x7f8df01a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df568d700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df00ffe50 0x7f8df0198010 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00e9a0 tx=0x7f8dec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:30.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8de27fc700 1 -- 172.21.15.46:0/2373433836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec009790 con 0x7f8df00ffe50 2024-03-18T21:01:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8de27fc700 1 -- 172.21.15.46:0/2373433836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dec00cbd0 con 0x7f8df00ffe50 2024-03-18T21:01:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8de27fc700 1 -- 172.21.15.46:0/2373433836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec018680 con 0x7f8df00ffe50 2024-03-18T21:01:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df78f1700 1 -- 172.21.15.46:0/2373433836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df01a3d40 con 0x7f8df00ffe50 2024-03-18T21:01:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.194+0000 7f8df78f1700 1 -- 172.21.15.46:0/2373433836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df01a41d0 con 0x7f8df00ffe50 2024-03-18T21:01:30.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.195+0000 7f8df78f1700 1 -- 172.21.15.46:0/2373433836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df0108230 con 0x7f8df00ffe50 2024-03-18T21:01:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.197+0000 7f8de27fc700 1 -- 172.21.15.46:0/2373433836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8dec00cd40 con 0x7f8df00ffe50 2024-03-18T21:01:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.197+0000 7f8de27fc700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8ddc0a1310 0x7f8ddc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.197+0000 7f8de27fc700 1 -- 172.21.15.46:0/2373433836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f8dec014070 con 0x7f8df00ffe50 2024-03-18T21:01:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.197+0000 7f8df4e8c700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8ddc0a1310 0x7f8ddc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:30.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.198+0000 7f8df4e8c700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8ddc0a1310 0x7f8ddc0a37d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8de400b2e0 tx=0x7f8de4005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:30.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.201+0000 7f8de27fc700 1 -- 172.21.15.46:0/2373433836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8dec096640 con 0x7f8df00ffe50 2024-03-18T21:01:30.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.419+0000 7f8df78f1700 1 -- 172.21.15.46:0/2373433836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8df0068390 con 0x7f8df00ffe50 2024-03-18T21:01:30.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.419+0000 7f8de27fc700 1 -- 172.21.15.46:0/2373433836 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8dec0260a0 con 0x7f8df00ffe50 2024-03-18T21:01:30.423 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-18T21:01:30.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.422+0000 7f8df78f1700 1 -- 172.21.15.46:0/2373433836 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8ddc0a1310 msgr2=0x7f8ddc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:30.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.422+0000 7f8df78f1700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8ddc0a1310 0x7f8ddc0a37d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8de400b2e0 tx=0x7f8de4005960 comp rx=0 tx=0).stop 2024-03-18T21:01:30.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.422+0000 7f8df78f1700 1 -- 172.21.15.46:0/2373433836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df00ffe50 msgr2=0x7f8df0198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:30.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.422+0000 7f8df78f1700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df00ffe50 0x7f8df0198010 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00e9a0 tx=0x7f8dec00ecb0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.422+0000 7f8df78f1700 1 -- 172.21.15.46:0/2373433836 shutdown_connections 2024-03-18T21:01:30.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.422+0000 7f8df78f1700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f8ddc0a1310 0x7f8ddc0a37d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.422+0000 7f8df78f1700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8df00ffe50 0x7f8df0198010 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.422+0000 7f8df78f1700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8df0198a90 0x7f8df01a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.422+0000 7f8df78f1700 1 --2- 172.21.15.46:0/2373433836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8df0104f60 0x7f8df0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.422+0000 7f8df78f1700 1 -- 172.21.15.46:0/2373433836 >> 172.21.15.46:0/2373433836 conn(0x7f8df0076d40 msgr2=0x7f8df00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:30.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.422+0000 7f8df78f1700 1 -- 172.21.15.46:0/2373433836 shutdown_connections 2024-03-18T21:01:30.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.422+0000 7f8df78f1700 1 -- 172.21.15.46:0/2373433836 wait complete. 2024-03-18T21:01:30.440 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-18T21:01:30.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.588+0000 7f762eed9700 1 -- 172.21.15.46:0/2962513575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280ffae0 msgr2=0x7f76280fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:30.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.588+0000 7f762eed9700 1 --2- 172.21.15.46:0/2962513575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280ffae0 0x7f76280fff60 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f7610009990 tx=0x7f7610009ca0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.589+0000 7f762eed9700 1 -- 172.21.15.46:0/2962513575 shutdown_connections 2024-03-18T21:01:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.589+0000 7f762eed9700 1 --2- 172.21.15.46:0/2962513575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76281004a0 0x7f762810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.589+0000 7f762eed9700 1 --2- 172.21.15.46:0/2962513575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628105b80 0x7f7628105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.589+0000 7f762eed9700 1 --2- 172.21.15.46:0/2962513575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280ffae0 0x7f76280fff60 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.589+0000 7f762eed9700 1 -- 172.21.15.46:0/2962513575 >> 172.21.15.46:0/2962513575 conn(0x7f76280fb5f0 msgr2=0x7f76280fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.589+0000 7f762eed9700 1 -- 172.21.15.46:0/2962513575 shutdown_connections 2024-03-18T21:01:30.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.589+0000 7f762eed9700 1 -- 172.21.15.46:0/2962513575 wait complete. 2024-03-18T21:01:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.589+0000 7f762eed9700 1 Processor -- start 2024-03-18T21:01:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762eed9700 1 -- start start 2024-03-18T21:01:30.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762eed9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280ffae0 0x7f7628197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762eed9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76281004a0 0x7f76281983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762eed9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628105b80 0x7f76281a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762eed9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7628198c10 con 0x7f76280ffae0 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762eed9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76281a3440 con 0x7f7628105b80 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762eed9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76281a3580 con 0x7f76281004a0 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762cc75700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280ffae0 0x7f7628197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762d476700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628105b80 0x7f76281a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762d476700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628105b80 0x7f76281a2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56486/0 (socket says 172.21.15.46:56486) 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762cc75700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280ffae0 0x7f7628197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43916/0 (socket says 172.21.15.46:43916) 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762d476700 1 -- 172.21.15.46:0/1742474251 learned_addr learned my addr 172.21.15.46:0/1742474251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f7627fff700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76281004a0 0x7f76281983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762d476700 1 -- 172.21.15.46:0/1742474251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76281004a0 msgr2=0x7f76281983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762d476700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76281004a0 0x7f76281983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762d476700 1 -- 172.21.15.46:0/1742474251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280ffae0 msgr2=0x7f7628197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762d476700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280ffae0 0x7f7628197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.590+0000 7f762d476700 1 -- 172.21.15.46:0/1742474251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76100095f0 con 0x7f7628105b80 2024-03-18T21:01:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.591+0000 7f7627fff700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76281004a0 0x7f76281983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T21:01:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.591+0000 7f762d476700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628105b80 0x7f76281a2f00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f762000ef60 tx=0x7f762000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.591+0000 7f7625ffb700 1 -- 172.21.15.46:0/1742474251 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7620018070 con 0x7f7628105b80 2024-03-18T21:01:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.591+0000 7f7625ffb700 1 -- 172.21.15.46:0/1742474251 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f762000f040 con 0x7f7628105b80 2024-03-18T21:01:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.591+0000 7f7625ffb700 1 -- 172.21.15.46:0/1742474251 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762000cd70 con 0x7f7628105b80 2024-03-18T21:01:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.591+0000 7f762eed9700 1 -- 172.21.15.46:0/1742474251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76281a3780 con 0x7f7628105b80 2024-03-18T21:01:30.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.591+0000 7f762eed9700 1 -- 172.21.15.46:0/1742474251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76281a3c40 con 0x7f7628105b80 2024-03-18T21:01:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.593+0000 7f762eed9700 1 -- 172.21.15.46:0/1742474251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7628108100 con 0x7f7628105b80 2024-03-18T21:01:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.594+0000 7f7625ffb700 1 -- 172.21.15.46:0/1742474251 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7620003a00 con 0x7f7628105b80 2024-03-18T21:01:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.594+0000 7f7625ffb700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f76140a1290 0x7f76140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.595+0000 7f7625ffb700 1 -- 172.21.15.46:0/1742474251 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f76200cc890 con 0x7f7628105b80 2024-03-18T21:01:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.595+0000 7f762cc75700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f76140a1290 0x7f76140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:30.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.595+0000 7f762cc75700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f76140a1290 0x7f76140a3750 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7618009be0 tx=0x7f7618009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:30.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.674+0000 7f7625ffb700 1 -- 172.21.15.46:0/1742474251 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7620095d50 con 0x7f7628105b80 2024-03-18T21:01:30.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:30 smithi046 ceph-mon[43495]: from='client.20877 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T21:01:30.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:30 smithi046 ceph-mon[43495]: pgmap v805: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.2 MiB/s wr, 433 op/s 2024-03-18T21:01:30.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:30 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:30.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:30 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/488012348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T21:01:30.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:30 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1867709400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T21:01:30.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:30 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2373433836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T21:01:30.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:30 smithi073 ceph-mon[47384]: from='client.20877 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T21:01:30.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:30 smithi073 ceph-mon[47384]: pgmap v805: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.2 MiB/s wr, 433 op/s 2024-03-18T21:01:30.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:30 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:30.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:30 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/488012348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T21:01:30.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:30 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1867709400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T21:01:30.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:30 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2373433836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T21:01:30.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.966+0000 7f762eed9700 1 -- 172.21.15.46:0/1742474251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f7628068390 con 0x7f7628105b80 2024-03-18T21:01:30.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.975+0000 7f7625ffb700 1 -- 172.21.15.46:0/1742474251 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+733921 (secure 0 0 0) 0x7f7620099370 con 0x7f7628105b80 2024-03-18T21:01:30.988 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T21:01:30.988 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T21:01:30.988 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 25, 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: "active_gid": 15498, 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: "active_name": "x", 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addrs": { 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6800", 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 805586292 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6801", 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 805586292 2024-03-18T21:01:30.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addr": "172.21.15.46:6801/805586292", 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: "active_change": "2024-03-18T20:36:06.974156+0000", 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: "active_mgr_features": 4540138322906710015, 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: "standbys": [ 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 15492, 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "z", 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540138322906710015, 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-03-18T21:01:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:30.991 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:30.992 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:30.993 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:30.994 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:30.995 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:30.996 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-03-18T21:01:30.997 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:30.998 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:30.999 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.000 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-03-18T21:01:31.001 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-03-18T21:01:31.002 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.003 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.004 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-03-18T21:01:31.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-03-18T21:01:31.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-03-18T21:01:31.007 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-03-18T21:01:31.008 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-03-18T21:01:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-03-18T21:01:31.010 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-03-18T21:01:31.011 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-03-18T21:01:31.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.014 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.015 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.016 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.017 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-03-18T21:01:31.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-03-18T21:01:31.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap" 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.020 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.022 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.023 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.024 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-03-18T21:01:31.025 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.026 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.027 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-03-18T21:01:31.028 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.028 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.028 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.028 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-18T21:01:31.028 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.028 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.028 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.029 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-03-18T21:01:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-03-18T21:01:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-18T21:01:31.030 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.031 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.032 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-18T21:01:31.033 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-03-18T21:01:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-03-18T21:01:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.034 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.035 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.036 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-03-18T21:01:31.037 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.038 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-03-18T21:01:31.040 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.041 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.042 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.044 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.045 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.046 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.047 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.048 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.049 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.050 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.051 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-03-18T21:01:31.052 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.053 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.056 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.060 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.063 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.064 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.065 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.067 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.068 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.069 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.070 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.071 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-03-18T21:01:31.072 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.073 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.074 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.075 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.076 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.077 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.078 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.079 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.080 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.081 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.082 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.083 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.084 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.085 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.086 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.087 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.088 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.089 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.090 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-03-18T21:01:31.091 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-03-18T21:01:31.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.093 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.094 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.095 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.096 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.097 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.098 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.099 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.100 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.101 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.102 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.103 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.105 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.106 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.107 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.108 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.109 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.110 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.112 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.114 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.117 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.118 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.120 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.122 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-18T21:01:31.123 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.124 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-18T21:01:31.125 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-18T21:01:31.126 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-18T21:01:31.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-18T21:01:31.128 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.129 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.130 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.131 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-03-18T21:01:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-18T21:01:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.136 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.137 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.138 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.141 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.142 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.143 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.148 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.152 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.153 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.154 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.155 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-03-18T21:01:31.156 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.157 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.159 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.160 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.161 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.162 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.163 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.164 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.165 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.166 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.167 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.168 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.169 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-03-18T21:01:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.173 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.174 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.175 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.176 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.177 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.178 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.179 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.180 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.181 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-03-18T21:01:31.182 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.183 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.184 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-03-18T21:01:31.185 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.186 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.188 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.189 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.190 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.191 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.192 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.193 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-03-18T21:01:31.194 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.195 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.196 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.198 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.199 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.200 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.201 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.202 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.203 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.204 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.205 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.206 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.207 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.208 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.209 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.210 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.211 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.212 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.213 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.216 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.217 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.220 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.221 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.222 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.223 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.224 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.225 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-03-18T21:01:31.226 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.227 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.228 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.229 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.230 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.231 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.233 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.234 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.236 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.237 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.238 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.239 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.240 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.241 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.242 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.243 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.244 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.245 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.247 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.248 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.249 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.250 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.251 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.252 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.253 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.254 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.255 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.256 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.257 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.258 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.259 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.260 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.262 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.262 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.262 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.262 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.262 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.262 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.262 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.262 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.262 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.262 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.263 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.264 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.265 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.266 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.267 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.268 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.269 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.270 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.273 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.274 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.275 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.278 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.279 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-03-18T21:01:31.280 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-03-18T21:01:31.281 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.285 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.286 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.287 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.288 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.289 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.290 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.291 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-03-18T21:01:31.292 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-03-18T21:01:31.293 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-03-18T21:01:31.294 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-03-18T21:01:31.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.297 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.298 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.301 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.302 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.303 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.304 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.305 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-03-18T21:01:31.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.307 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.308 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.310 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.311 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.311 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.311 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.311 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.311 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.311 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.311 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.311 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.312 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.313 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.314 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.315 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.316 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.318 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.319 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.320 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.321 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.323 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.324 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.327 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.328 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.329 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.332 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.337 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-03-18T21:01:31.338 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.339 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.340 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-03-18T21:01:31.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-03-18T21:01:31.342 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-03-18T21:01:31.343 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.344 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.345 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-03-18T21:01:31.346 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.347 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.348 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.349 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.351 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.352 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.353 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.354 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.355 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-03-18T21:01:31.357 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.358 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.360 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.362 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.363 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-03-18T21:01:31.364 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.365 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.368 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.369 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.370 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.371 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.372 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-03-18T21:01:31.373 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.374 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-03-18T21:01:31.375 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-03-18T21:01:31.376 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-03-18T21:01:31.377 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.378 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.379 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.380 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.381 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.382 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.383 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.384 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.385 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.387 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.389 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.390 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.391 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.392 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.395 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.396 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.397 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.398 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.399 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.400 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.402 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.403 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.404 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.405 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.406 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.407 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:31.408 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.409 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.411 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.412 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.413 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.414 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.415 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.416 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 25262, 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "y", 2024-03-18T21:01:31.417 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540138322906710015, 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.418 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.419 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.420 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.422 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.423 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-03-18T21:01:31.425 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.426 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.428 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.429 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.430 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.431 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-03-18T21:01:31.432 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-03-18T21:01:31.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-03-18T21:01:31.434 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-03-18T21:01:31.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-03-18T21:01:31.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-03-18T21:01:31.438 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.439 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.441 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.443 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.444 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.445 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.446 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap" 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.447 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.448 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-03-18T21:01:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-03-18T21:01:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.452 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.453 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.454 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.455 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-18T21:01:31.456 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-18T21:01:31.457 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-18T21:01:31.458 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.459 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.460 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.461 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.462 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.463 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.464 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.465 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.466 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.467 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.468 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.469 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.470 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.471 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.472 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.473 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.474 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.475 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.476 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-18T21:01:31.477 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.478 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-18T21:01:31.479 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-03-18T21:01:31.480 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.483 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.484 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.485 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.486 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.487 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.488 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.489 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.490 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.493 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.494 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.496 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-03-18T21:01:31.497 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.498 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.499 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.500 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.501 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.506 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.507 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.508 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.509 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.510 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.511 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.513 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.514 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.515 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.517 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.518 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.519 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.520 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.521 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.522 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.523 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.527 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.528 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.529 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.530 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.531 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-18T21:01:31.532 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-03-18T21:01:31.533 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-18T21:01:31.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.534 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.534 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-03-18T21:01:31.534 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.534 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.534 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.534 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.534 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.534 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.535 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.536 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.537 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.542 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-18T21:01:31.542 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-18T21:01:31.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.542 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.542 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.542 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.543 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.544 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.545 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.546 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.547 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.548 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.549 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.550 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.551 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.552 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.553 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.554 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.555 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.556 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.558 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.559 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.562 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.564 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-18T21:01:31.565 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-03-18T21:01:31.566 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-03-18T21:01:31.567 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-03-18T21:01:31.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.569 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.570 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.574 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.575 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.576 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.577 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.578 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.579 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.580 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.581 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.582 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-03-18T21:01:31.583 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.584 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-03-18T21:01:31.585 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.586 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.587 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.588 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.589 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.589 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.589 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.589 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.589 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.589 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.589 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.589 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.590 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.590 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.590 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.590 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.590 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.590 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.590 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.591 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.591 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.591 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.591 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.591 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.591 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.592 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.592 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.592 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.592 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.592 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.592 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.592 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.592 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.593 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.593 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.593 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.593 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-03-18T21:01:31.593 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-03-18T21:01:31.593 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.593 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.594 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-03-18T21:01:31.594 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.594 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.594 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.594 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-18T21:01:31.594 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.594 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.599 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.601 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.602 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.603 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.604 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.606 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.607 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.613 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.614 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.616 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.617 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.618 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-03-18T21:01:31.619 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.620 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-03-18T21:01:31.621 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-03-18T21:01:31.622 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.622 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-18T21:01:31.622 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.622 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.622 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.622 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-03-18T21:01:31.622 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-03-18T21:01:31.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.622 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.623 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.624 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.625 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.626 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.628 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.629 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.630 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.631 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.633 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.634 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.635 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.636 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.637 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.638 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.639 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.640 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.641 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.643 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.644 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.645 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.647 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.648 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-03-18T21:01:31.649 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-03-18T21:01:31.650 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.652 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.653 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.654 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.656 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.658 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.658 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.658 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.658 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.658 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.658 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.658 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.658 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.658 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-03-18T21:01:31.659 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.660 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.661 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.662 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.663 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.664 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.665 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.668 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.669 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-03-18T21:01:31.670 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.671 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.672 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.673 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.674 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.675 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-03-18T21:01:31.676 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.677 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.680 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.681 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.682 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.683 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.684 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.685 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.686 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.687 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.688 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.689 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.690 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.691 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.692 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.693 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.694 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.695 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-03-18T21:01:31.696 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.697 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.698 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.699 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.700 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.701 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-03-18T21:01:31.703 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.704 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.705 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.706 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-18T21:01:31.707 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.708 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.709 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.710 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.711 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.715 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.716 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.718 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.719 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.720 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.721 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.724 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.725 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.726 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.727 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.728 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.729 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.730 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.731 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.732 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.734 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.735 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.736 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.739 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.740 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.741 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.742 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.742 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.742 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.742 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.742 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.742 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.748 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.749 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.750 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.751 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.753 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.754 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.755 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.756 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.757 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.760 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.764 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.769 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.770 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.771 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.772 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.773 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.774 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.778 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.779 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.780 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.781 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.782 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.784 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.785 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.788 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.789 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.790 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.791 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.793 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.794 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.795 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.796 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.798 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.799 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.801 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.802 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.803 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.804 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.805 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.806 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-03-18T21:01:31.807 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.808 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.809 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.810 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.811 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.812 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.813 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.814 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.815 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.816 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.817 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-18T21:01:31.818 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.819 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.820 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.821 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.822 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.827 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.828 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-03-18T21:01:31.830 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.831 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.832 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.833 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.836 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.837 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.838 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.839 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.840 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.841 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-03-18T21:01:31.842 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-03-18T21:01:31.842 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.842 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.842 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.842 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.842 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.842 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.842 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.842 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.842 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.842 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.843 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.844 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard", 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: "restful", 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_schedule" 2024-03-18T21:01:31.845 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.846 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.847 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.848 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.849 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.850 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.851 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.852 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.853 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.854 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.855 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.856 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.857 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.858 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.859 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-03-18T21:01:31.860 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-03-18T21:01:31.861 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-03-18T21:01:31.862 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-03-18T21:01:31.863 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-03-18T21:01:31.864 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-03-18T21:01:31.865 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-03-18T21:01:31.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-03-18T21:01:31.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.868 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.871 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.872 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap" 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.879 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.880 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.881 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.882 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.883 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.884 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.885 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.886 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.887 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.888 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.889 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.890 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.891 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.892 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.893 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.894 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.895 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.896 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.897 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.898 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.899 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.900 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-03-18T21:01:31.901 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.902 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.903 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.904 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.905 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.906 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.907 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.908 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.909 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.910 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.911 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-18T21:01:31.912 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-18T21:01:31.913 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.914 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.915 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.916 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.917 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.918 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.919 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.920 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-18T21:01:31.921 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-03-18T21:01:31.922 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-18T21:01:31.923 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-03-18T21:01:31.924 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-03-18T21:01:31.925 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.926 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.927 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.928 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.929 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.930 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.931 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.932 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-03-18T21:01:31.933 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.934 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-18T21:01:31.935 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-18T21:01:31.936 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.938 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.939 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.940 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.943 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.944 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.945 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.946 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-18T21:01:31.947 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.948 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.949 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.950 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.951 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.952 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.956 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.957 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.958 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.959 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-18T21:01:31.960 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-18T21:01:31.961 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-03-18T21:01:31.962 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-18T21:01:31.963 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-03-18T21:01:31.964 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.965 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-18T21:01:31.966 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.967 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.969 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.971 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.972 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.973 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.974 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-18T21:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-18T21:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.975 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.977 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.978 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.979 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.980 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.981 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.982 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.983 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.987 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.990 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-18T21:01:31.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-03-18T21:01:31.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-03-18T21:01:31.993 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-03-18T21:01:31.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-03-18T21:01:31.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.996 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:31.997 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.998 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:31.999 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.000 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.001 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.002 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.003 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-03-18T21:01:32.009 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.010 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.011 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.012 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.013 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.014 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.015 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.016 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-03-18T21:01:32.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-03-18T21:01:32.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-03-18T21:01:32.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-03-18T21:01:32.020 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.022 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.023 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.024 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.025 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.026 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.027 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.029 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.033 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.034 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.035 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.036 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.037 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.040 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.042 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.042 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-03-18T21:01:32.042 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-03-18T21:01:32.042 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.042 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.042 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.042 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.042 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.042 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.042 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.043 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.044 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-18T21:01:32.045 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-03-18T21:01:32.046 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T21:01:32.047 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.048 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.049 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.050 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.051 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.054 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.055 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.056 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.057 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.058 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-03-18T21:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-03-18T21:01:32.060 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.061 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.063 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.064 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.065 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.066 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.067 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.068 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.069 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.070 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.071 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-03-18T21:01:32.072 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-03-18T21:01:32.073 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-03-18T21:01:32.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-03-18T21:01:32.075 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-03-18T21:01:32.076 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.077 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.078 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.079 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.080 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.081 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.082 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.083 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.084 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.086 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.087 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.088 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.089 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.090 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.093 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.094 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.096 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.097 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.099 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-03-18T21:01:32.100 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.101 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.102 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.103 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.104 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.105 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.106 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.107 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.108 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.109 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.110 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.112 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.114 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.115 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.118 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-03-18T21:01:32.119 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.120 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.121 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.122 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.123 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.124 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.125 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.127 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.128 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.130 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.131 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.132 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.133 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.134 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.136 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.137 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.138 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.139 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.140 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-03-18T21:01:32.141 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-03-18T21:01:32.142 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.143 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-03-18T21:01:32.144 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.145 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.146 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.148 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.149 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.150 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.151 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.152 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.152 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.152 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-03-18T21:01:32.152 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-03-18T21:01:32.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.152 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.153 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.154 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.157 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.158 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.159 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.160 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.163 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.164 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.165 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.166 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.167 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.168 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.171 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.172 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.172 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.172 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.172 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.172 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.172 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.172 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.172 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.173 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.173 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.173 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.173 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.173 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.173 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.173 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.174 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.174 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.174 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.174 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.174 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.175 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.175 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.175 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.175 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.175 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.176 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.176 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.176 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.176 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.176 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.176 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.177 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.177 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-03-18T21:01:32.177 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-03-18T21:01:32.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.177 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.177 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-03-18T21:01:32.178 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.178 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.178 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.178 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-18T21:01:32.178 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.178 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.178 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.178 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.178 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.179 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.180 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.181 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.182 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.183 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.184 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.185 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.186 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.187 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.188 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.189 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.190 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.191 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.192 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.196 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.198 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.201 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.202 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.203 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.204 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.205 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.206 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.207 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.208 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.209 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.210 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.211 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.213 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-18T21:01:32.214 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-03-18T21:01:32.215 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.216 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.217 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.218 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.219 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.220 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-03-18T21:01:32.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.222 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.223 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.225 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.226 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.227 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.228 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.229 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.230 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.231 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.232 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.233 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.234 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.235 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.236 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.237 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.238 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.239 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.241 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.242 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.243 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.245 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.246 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.247 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.248 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.249 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.250 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.252 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.253 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.254 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.255 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.256 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.258 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.260 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-03-18T21:01:32.262 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-03-18T21:01:32.263 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-03-18T21:01:32.264 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-03-18T21:01:32.265 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-18T21:01:32.266 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: "services": { 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard": "https://172.21.15.46:8443/" 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: "always_on_modules": { 2024-03-18T21:01:32.267 INFO:teuthology.orchestra.run.smithi046.stdout: "octopus": [ 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "pacific": [ 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-03-18T21:01:32.268 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "quincy": [ 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-03-18T21:01:32.269 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: "reef": [ 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-03-18T21:01:32.270 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: "last_failure_osd_epoch": 81, 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: "active_clients": [ 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-03-18T21:01:32.271 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 945995638 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "libcephsqlite", 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1706013017 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.272 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2774247633 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-03-18T21:01:32.273 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2384784583 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 199279252 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:32.274 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.275 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-03-18T21:01:32.275 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-03-18T21:01:32.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-18T21:01:32.275 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-03-18T21:01:32.275 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-03-18T21:01:32.275 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2711660229 2024-03-18T21:01:32.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.275 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-18T21:01:32.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:32.275 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-18T21:01:32.275 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T21:01:32.277 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:31 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:32.277 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:31 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1742474251' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-18T21:01:32.277 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:30 smithi155 ceph-mon[48344]: from='client.20877 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T21:01:32.277 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:30 smithi155 ceph-mon[48344]: pgmap v805: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.2 MiB/s wr, 433 op/s 2024-03-18T21:01:32.277 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:30 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:32.278 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:30 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/488012348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T21:01:32.278 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:30 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1867709400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T21:01:32.278 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:30 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2373433836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T21:01:32.278 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:31 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:32.278 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:31 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1742474251' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-18T21:01:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.982+0000 7f762eed9700 1 -- 172.21.15.46:0/1742474251 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f76140a1290 msgr2=0x7f76140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.982+0000 7f762eed9700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f76140a1290 0x7f76140a3750 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7618009be0 tx=0x7f7618009260 comp rx=0 tx=0).stop 2024-03-18T21:01:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.982+0000 7f762eed9700 1 -- 172.21.15.46:0/1742474251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628105b80 msgr2=0x7f76281a2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.982+0000 7f762eed9700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628105b80 0x7f76281a2f00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f762000ef60 tx=0x7f762000ef90 comp rx=0 tx=0).stop 2024-03-18T21:01:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.982+0000 7f762eed9700 1 -- 172.21.15.46:0/1742474251 shutdown_connections 2024-03-18T21:01:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.982+0000 7f762eed9700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f76140a1290 0x7f76140a3750 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.982+0000 7f762eed9700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76280ffae0 0x7f7628197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.982+0000 7f762eed9700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7628105b80 0x7f76281a2f00 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.982+0000 7f762eed9700 1 --2- 172.21.15.46:0/1742474251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76281004a0 0x7f76281983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.982+0000 7f762eed9700 1 -- 172.21.15.46:0/1742474251 >> 172.21.15.46:0/1742474251 conn(0x7f76280fb5f0 msgr2=0x7f76280fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.982+0000 7f762eed9700 1 -- 172.21.15.46:0/1742474251 shutdown_connections 2024-03-18T21:01:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:30.982+0000 7f762eed9700 1 -- 172.21.15.46:0/1742474251 wait complete. 2024-03-18T21:01:32.285 DEBUG:tasks.check_counter:Getting stats from a 2024-03-18T21:01:32.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-18T21:01:32.285 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:31 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:32.285 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:31 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1742474251' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-18T21:01:32.790 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T21:01:32.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:32 smithi046 ceph-mon[43495]: pgmap v806: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.6 MiB/s wr, 363 op/s 2024-03-18T21:01:32.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:32 smithi073 ceph-mon[47384]: pgmap v806: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.6 MiB/s wr, 363 op/s 2024-03-18T21:01:33.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:32 smithi155 ceph-mon[48344]: pgmap v806: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 6.6 MiB/s wr, 363 op/s 2024-03-18T21:01:33.945 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:33 smithi046 ceph-mon[43495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:33.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:33 smithi073 ceph-mon[47384]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:34.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:33 smithi155 ceph-mon[48344]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T21:01:34.604 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:34 smithi046 ceph-mon[43495]: pgmap v807: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 355 op/s 2024-03-18T21:01:34.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:34 smithi073 ceph-mon[47384]: pgmap v807: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 355 op/s 2024-03-18T21:01:35.046 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:34 smithi155 ceph-mon[48344]: pgmap v807: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 355 op/s 2024-03-18T21:01:35.998 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2886, 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2888, 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 328025353, 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 897547, 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 9, 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 4, 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.484214939, 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.095966503, 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.269485292, 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.090796822, 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2886, 2024-03-18T21:01:35.999 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.250996666, 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000086970 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 3001, 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2904, 2024-03-18T21:01:36.000 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 252661333, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 1009596, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 5, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.461219221, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.080891921, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.292009867, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.079682126, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2904, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.224091628, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000077166 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.001 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 2045344, 2024-03-18T21:01:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 256608 2024-03-18T21:01:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T21:01:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 1005, 2024-03-18T21:01:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 1332, 2024-03-18T21:01:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 339995614, 2024-03-18T21:01:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 1128257, 2024-03-18T21:01:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 9, 2024-03-18T21:01:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 6, 2024-03-18T21:01:36.002 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.328691955, 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.039345778, 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.244835782, 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.031623463, 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1329, 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.148769453, 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000111940 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.003 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 816624 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 1, 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-MDSRank": { 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 10426, 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.315123449, 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000030224 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-PurgeQueue": { 2024-03-18T21:01:36.004 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "request": 0, 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "reply": 0, 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "reply_latency": { 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.005 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_reply": 0, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "forward": 0, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_complete": 0, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_keys": 0, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_commit": 0, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_split": 0, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_merge": 0, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes": 35711, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_top": 12840, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_bottom": 22871, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pin_tail": 0, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pinned": 17193, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_expired": 0, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_with_caps": 0, 2024-03-18T21:01:36.006 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": 0, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "subtrees": 6, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse": 0, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_hit": 0, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_forward": 0, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_discover": 0, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_dir_fetch": 0, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_remote_ino": 0, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_lock": 0, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "load_cent": 0, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "q": 0, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "exported": 0, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "exported_inodes": 0, 2024-03-18T21:01:36.007 INFO:teuthology.orchestra.run.smithi046.stdout: "imported": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "imported_inodes": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_dir_fetch": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_backtrace_fetch": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_peer_discover": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rfiles": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rbytes": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rsnaps": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_set_tag": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_inotable_repaired": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_inodes": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T21:01:36.008 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_file_inodes": 0, 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_inode_file_caps": 0, 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_grant": 0, 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps": 0, 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps_dirty": 0, 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_cap_release": 0, 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "process_request_cap_release": 0 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.009 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache": { 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update_receipt": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_try_discover": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_send_discover": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_handle_discover": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_delayed": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_enqueuing": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_created": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_enqueued": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_reintegrated": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_migrated": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_processing": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_enqueued": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_prioritized": 0, 2024-03-18T21:01:36.010 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_started": 0, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_completed": 0, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_exportdir": 0, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_flush": 0, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragmentdir": 0, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragstats": 0, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_inodestats": 0 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log": { 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "evadd": 0, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "evex": 0, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "evtrm": 0, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "ev": 160408, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "evexg": 0, 2024-03-18T21:01:36.011 INFO:teuthology.orchestra.run.smithi046.stdout: "evexd": 0, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "segadd": 0, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "segex": 0, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "segtrm": 0, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "seg": 129, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "segexg": 0, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "segexd": 0, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "expos": 385891954, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "wrpos": 921083467, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "rdpos": 921054166, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "jlat": { 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "replayed": 160408 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.012 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mem": { 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1740, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "ino+": 35794, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "ino-": 34054, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "dir": 52, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "dir+": 164, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "dir-": 112, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "dn": 35608, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "dn+": 36223, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "dn-": 615, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "cap": 0, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "cap+": 0, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "cap-": 0, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "rss": 91656, 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: "heap": 207156 2024-03-18T21:01:36.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server": { 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_client_request": 0, 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_server_request": 0, 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_request": 0, 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_session": 0, 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_peer_request": 0, 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "req_create_latency": { 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getattr_latency": { 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.014 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getfilelock_latency": { 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "req_link_latency": { 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookup_latency": { 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.015 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookuphash_latency": { 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupino_latency": { 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupname_latency": { 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.016 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupparent_latency": { 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupsnap_latency": { 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lssnap_latency": { 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.017 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mkdir_latency": { 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mknod_latency": { 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mksnap_latency": { 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.018 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "req_open_latency": { 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "req_readdir_latency": { 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.019 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rename_latency": { 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "req_renamesnap_latency": { 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "req_snapdiff_latency": { 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.020 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmdir_latency": { 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmsnap_latency": { 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmxattr_latency": { 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setattr_latency": { 2024-03-18T21:01:36.021 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setdirlayout_latency": { 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setfilelock_latency": { 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setlayout_latency": { 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setxattr_latency": { 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "req_symlink_latency": { 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "req_unlink_latency": { 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_revoke_eviction": 0, 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_acquisition_throttle": 0, 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getvxattr_latency": { 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessions": { 2024-03-18T21:01:36.024 INFO:teuthology.orchestra.run.smithi046.stdout: "session_count": 1, 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "session_add": 3, 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "session_remove": 2, 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_open": 3, 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_stale": 0, 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "total_load": 0, 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "average_load": 0, 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "avg_session_uptime": 5709, 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "mdthresh_evicted": 0 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T21:01:36.025 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-03-18T21:01:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T21:01:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-03-18T21:01:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T21:01:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T21:01:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T21:01:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T21:01:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T21:01:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-03-18T21:01:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T21:01:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T21:01:36.026 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T21:01:36.027 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 307819, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 2111, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-03-18T21:01:36.028 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 8256, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 188, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 29694287, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 221295, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-03-18T21:01:36.029 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter": { 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "op_active": 0, 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "op_laggy": 0, 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send": 6309, 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send_bytes": 21, 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "op_resend": 3, 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "op_reply": 6309, 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "op_latency": { 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 6309, 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 21.418793596, 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003394958 2024-03-18T21:01:36.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "op_inflight": 0, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "oplen_avg": { 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 6309, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 6310 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "op": 6309, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "op_r": 6309, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "op_w": 0, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "op_rmw": 0, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "op_pg": 0, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_stat": 3453, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_create": 0, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_read": 2854, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_write": 0, 2024-03-18T21:01:36.031 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writefull": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writesame": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_append": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_zero": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_truncate": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_delete": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_mapext": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_sparse_read": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_clonerange": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_getxattr": 1, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_setxattr": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_cmpxattr": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_rmxattr": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_resetxattrs": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_call": 0, 2024-03-18T21:01:36.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_watch": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_notify": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls_filter": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_other": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_active": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_send": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_resend": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_ping": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_active": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_send": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_resend": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_active": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_send": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_resend": 0, 2024-03-18T21:01:36.033 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_active": 0, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_send": 0, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_resend": 0, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "command_active": 0, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "command_send": 0, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "command_resend": 0, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "map_epoch": 122, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "map_full": 0, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "map_inc": 44, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_sessions": 11, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_open": 12, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_close": 1, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_laggy": 0, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_wr": 0, 2024-03-18T21:01:36.034 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_rd": 2, 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_del": 0 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "oft": { 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_objs": 0, 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_kv_pairs": 0, 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_updates": 0, 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_removes": 0 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "osd-slow-ops": { 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_ops_count": 0 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "purge_queue": { 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops": 0, 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T21:01:36.035 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing": 0, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_high_water": 0, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed": 0, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_item_in_journal": 0 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 6888, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 920030337, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 6888, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 6888, 2024-03-18T21:01:36.036 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 920030337, 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_bytes": { 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:36.037 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 6309, 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 916902798, 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 6309, 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 916902798, 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_ops": { 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-03-18T21:01:36.038 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1024, 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 6309, 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 6309, 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 6309, 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 6309, 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle": { 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 0, 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 0, 2024-03-18T21:01:36.040 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle-0x561d25d065a0": { 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-03-18T21:01:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-03-18T21:01:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 0, 2024-03-18T21:01:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 0, 2024-03-18T21:01:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-03-18T21:01:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:36.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:36.042 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T21:01:36.526 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-18T21:01:36.527 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-18T21:01:36.527 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-03-18T21:01:36.527 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-18T21:01:36.527 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T21:01:36.527 DEBUG:tasks.check_counter:Getting stats from d 2024-03-18T21:01:36.527 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-18T21:01:36.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:36 smithi073 ceph-mon[47384]: pgmap v808: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.8 MiB/s wr, 343 op/s 2024-03-18T21:01:36.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:36 smithi155 ceph-mon[48344]: pgmap v808: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.8 MiB/s wr, 343 op/s 2024-03-18T21:01:36.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:36 smithi046 ceph-mon[43495]: pgmap v808: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.8 MiB/s wr, 343 op/s 2024-03-18T21:01:36.796 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T21:01:37.881 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T21:01:37.881 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T21:01:37.881 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 4, 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 4, 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 2944, 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 448, 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.002544422, 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000069614, 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000896526, 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3, 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.001318783, 2024-03-18T21:01:37.882 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000439594 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 3, 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 333, 2024-03-18T21:01:37.883 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 480, 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 286992, 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.025279441, 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.018011455, 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.003176001, 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 330, 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.037512982, 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000113675 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:37.884 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:37.885 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:37.885 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:37.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:37.885 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-18T21:01:37.885 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 286992 2024-03-18T21:01:37.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:37.885 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T21:01:37.885 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 445, 2024-03-18T21:01:37.885 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 435, 2024-03-18T21:01:37.885 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1487568, 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 249360, 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.073139129, 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.026347734, 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.040269694, 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.010455138, 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 434, 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.032913736, 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000075838 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:37.886 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1487568, 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 249360 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-03-18T21:01:37.887 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-03-18T21:01:37.888 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-03-18T21:01:37.889 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 149745, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 31, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-03-18T21:01:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:37.891 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 448, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1424720, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 448, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 448, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1424720, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:37.892 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:37.893 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:37.893 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:37.893 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T21:01:38.206 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-18T21:01:38.207 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-18T21:01:38.207 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-18T21:01:38.207 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-18T21:01:38.207 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-18T21:01:38.207 DEBUG:tasks.check_counter:Getting stats from g 2024-03-18T21:01:38.207 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-18T21:01:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T21:01:38.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:38 smithi046 ceph-mon[43495]: pgmap v809: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.9 MiB/s wr, 215 op/s 2024-03-18T21:01:38.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:38 smithi073 ceph-mon[47384]: pgmap v809: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.9 MiB/s wr, 215 op/s 2024-03-18T21:01:39.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:38 smithi155 ceph-mon[48344]: pgmap v809: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.9 MiB/s wr, 215 op/s 2024-03-18T21:01:39.460 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T21:01:39.460 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T21:01:39.460 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 555710, 2024-03-18T21:01:39.461 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 766813, 2024-03-18T21:01:39.461 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 185589351, 2024-03-18T21:01:39.461 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 797503503, 2024-03-18T21:01:39.461 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 17, 2024-03-18T21:01:39.461 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 6, 2024-03-18T21:01:39.461 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 49.439366772, 2024-03-18T21:01:39.461 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 24.410048399, 2024-03-18T21:01:39.461 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 33.737961050, 2024-03-18T21:01:39.461 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 1.290455311, 2024-03-18T21:01:39.461 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:39.461 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 766810, 2024-03-18T21:01:39.461 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 123.603895230, 2024-03-18T21:01:39.462 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000161192 2024-03-18T21:01:39.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.462 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:39.462 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1127941, 2024-03-18T21:01:39.462 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.308523437, 2024-03-18T21:01:39.462 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000273 2024-03-18T21:01:39.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.462 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-03-18T21:01:39.462 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 402000 2024-03-18T21:01:39.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.462 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T21:01:39.463 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 372126, 2024-03-18T21:01:39.463 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 387298, 2024-03-18T21:01:39.463 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 126920614, 2024-03-18T21:01:39.463 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 382466158, 2024-03-18T21:01:39.463 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 59, 2024-03-18T21:01:39.463 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 6, 2024-03-18T21:01:39.463 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 24.790421655, 2024-03-18T21:01:39.463 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 10.216961540, 2024-03-18T21:01:39.463 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 17.198626620, 2024-03-18T21:01:39.463 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.956653975, 2024-03-18T21:01:39.463 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:39.463 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 387297, 2024-03-18T21:01:39.464 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 55.859268341, 2024-03-18T21:01:39.464 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000144228 2024-03-18T21:01:39.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.464 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:39.464 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 676235, 2024-03-18T21:01:39.464 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.141376505, 2024-03-18T21:01:39.464 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000209 2024-03-18T21:01:39.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.464 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 5136, 2024-03-18T21:01:39.464 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 436784 2024-03-18T21:01:39.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.464 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T21:01:39.465 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 185289, 2024-03-18T21:01:39.465 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 199553, 2024-03-18T21:01:39.465 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 72763511, 2024-03-18T21:01:39.465 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 539970155, 2024-03-18T21:01:39.465 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 11, 2024-03-18T21:01:39.465 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 7, 2024-03-18T21:01:39.465 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 14.195446300, 2024-03-18T21:01:39.465 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 5.830542183, 2024-03-18T21:01:39.465 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 7.701836741, 2024-03-18T21:01:39.465 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 1.807550059, 2024-03-18T21:01:39.465 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:39.465 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 199552, 2024-03-18T21:01:39.466 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 37.429772168, 2024-03-18T21:01:39.466 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000187569 2024-03-18T21:01:39.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.466 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:39.466 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 279493, 2024-03-18T21:01:39.466 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.060488644, 2024-03-18T21:01:39.466 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000216 2024-03-18T21:01:39.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.466 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 2080528, 2024-03-18T21:01:39.466 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 288432 2024-03-18T21:01:39.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.467 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-03-18T21:01:39.467 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 1, 2024-03-18T21:01:39.467 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-03-18T21:01:39.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.467 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-MDSRank": { 2024-03-18T21:01:39.467 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-03-18T21:01:39.467 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-03-18T21:01:39.467 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 97398, 2024-03-18T21:01:39.467 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 120.777298961, 2024-03-18T21:01:39.467 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001240038 2024-03-18T21:01:39.467 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: "finisher-PurgeQueue": { 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: "queue_len": 0, 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: "complete_latency": { 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 60977, 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 10.245454608, 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000168021 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: "request": 262910, 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: "reply": 246508, 2024-03-18T21:01:39.468 INFO:teuthology.orchestra.run.smithi046.stdout: "reply_latency": { 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 246508, 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 1212.243178983, 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.004917662 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_reply": 0, 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: "forward": 16402, 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_complete": 1, 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_fetch_keys": 0, 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_commit": 156, 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_split": 12, 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_merge": 9, 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes": 2830, 2024-03-18T21:01:39.469 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_top": 1188, 2024-03-18T21:01:39.470 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_bottom": 1306, 2024-03-18T21:01:39.470 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pin_tail": 336, 2024-03-18T21:01:39.470 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_pinned": 1132, 2024-03-18T21:01:39.470 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_expired": 924, 2024-03-18T21:01:39.470 INFO:teuthology.orchestra.run.smithi046.stdout: "inodes_with_caps": 3, 2024-03-18T21:01:39.470 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": 3, 2024-03-18T21:01:39.470 INFO:teuthology.orchestra.run.smithi046.stdout: "subtrees": 6, 2024-03-18T21:01:39.470 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse": 380891, 2024-03-18T21:01:39.470 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_hit": 227886, 2024-03-18T21:01:39.470 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_forward": 16391, 2024-03-18T21:01:39.470 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_discover": 113, 2024-03-18T21:01:39.471 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_dir_fetch": 0, 2024-03-18T21:01:39.471 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_remote_ino": 0, 2024-03-18T21:01:39.471 INFO:teuthology.orchestra.run.smithi046.stdout: "traverse_lock": 3476, 2024-03-18T21:01:39.471 INFO:teuthology.orchestra.run.smithi046.stdout: "load_cent": 25255, 2024-03-18T21:01:39.471 INFO:teuthology.orchestra.run.smithi046.stdout: "q": 0, 2024-03-18T21:01:39.471 INFO:teuthology.orchestra.run.smithi046.stdout: "exported": 19, 2024-03-18T21:01:39.471 INFO:teuthology.orchestra.run.smithi046.stdout: "exported_inodes": 574, 2024-03-18T21:01:39.471 INFO:teuthology.orchestra.run.smithi046.stdout: "imported": 12, 2024-03-18T21:01:39.471 INFO:teuthology.orchestra.run.smithi046.stdout: "imported_inodes": 642, 2024-03-18T21:01:39.471 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_dir_fetch": 0, 2024-03-18T21:01:39.471 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_backtrace_fetch": 434, 2024-03-18T21:01:39.472 INFO:teuthology.orchestra.run.smithi046.stdout: "openino_peer_discover": 0, 2024-03-18T21:01:39.472 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rfiles": 4, 2024-03-18T21:01:39.472 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rbytes": 234, 2024-03-18T21:01:39.472 INFO:teuthology.orchestra.run.smithi046.stdout: "root_rsnaps": 6, 2024-03-18T21:01:39.472 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_fetch": 434, 2024-03-18T21:01:39.472 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_set_tag": 0, 2024-03-18T21:01:39.472 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T21:01:39.473 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_inotable_repaired": 0, 2024-03-18T21:01:39.473 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_inodes": 0, 2024-03-18T21:01:39.473 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T21:01:39.473 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T21:01:39.473 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_file_inodes": 434, 2024-03-18T21:01:39.473 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_inode_file_caps": 12, 2024-03-18T21:01:39.473 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_revoke": 54776, 2024-03-18T21:01:39.473 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_grant": 151810, 2024-03-18T21:01:39.473 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T21:01:39.474 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flushsnap_ack": 1877, 2024-03-18T21:01:39.474 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T21:01:39.474 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps": 102339, 2024-03-18T21:01:39.474 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_caps_dirty": 29964, 2024-03-18T21:01:39.474 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_cap_release": 6465, 2024-03-18T21:01:39.474 INFO:teuthology.orchestra.run.smithi046.stdout: "process_request_cap_release": 175925 2024-03-18T21:01:39.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.474 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_cache": { 2024-03-18T21:01:39.474 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update": 27, 2024-03-18T21:01:39.474 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_update_receipt": 63, 2024-03-18T21:01:39.474 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_try_discover": 45, 2024-03-18T21:01:39.475 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_send_discover": 143, 2024-03-18T21:01:39.475 INFO:teuthology.orchestra.run.smithi046.stdout: "dir_handle_discover": 128, 2024-03-18T21:01:39.475 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays": 91, 2024-03-18T21:01:39.475 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_delayed": 71, 2024-03-18T21:01:39.475 INFO:teuthology.orchestra.run.smithi046.stdout: "num_strays_enqueuing": 0, 2024-03-18T21:01:39.475 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_created": 34255, 2024-03-18T21:01:39.475 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_enqueued": 34164, 2024-03-18T21:01:39.475 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_reintegrated": 0, 2024-03-18T21:01:39.475 INFO:teuthology.orchestra.run.smithi046.stdout: "strays_migrated": 0, 2024-03-18T21:01:39.475 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_processing": 0, 2024-03-18T21:01:39.475 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_enqueued": 0, 2024-03-18T21:01:39.475 INFO:teuthology.orchestra.run.smithi046.stdout: "num_recovering_prioritized": 0, 2024-03-18T21:01:39.476 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_started": 0, 2024-03-18T21:01:39.476 INFO:teuthology.orchestra.run.smithi046.stdout: "recovery_completed": 0, 2024-03-18T21:01:39.476 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T21:01:39.476 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_exportdir": 21, 2024-03-18T21:01:39.476 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_flush": 0, 2024-03-18T21:01:39.476 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragmentdir": 21, 2024-03-18T21:01:39.476 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_fragstats": 0, 2024-03-18T21:01:39.476 INFO:teuthology.orchestra.run.smithi046.stdout: "ireq_inodestats": 0 2024-03-18T21:01:39.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.476 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_log": { 2024-03-18T21:01:39.476 INFO:teuthology.orchestra.run.smithi046.stdout: "evadd": 168985, 2024-03-18T21:01:39.476 INFO:teuthology.orchestra.run.smithi046.stdout: "evex": 103122, 2024-03-18T21:01:39.477 INFO:teuthology.orchestra.run.smithi046.stdout: "evtrm": 103122, 2024-03-18T21:01:39.477 INFO:teuthology.orchestra.run.smithi046.stdout: "ev": 66416, 2024-03-18T21:01:39.477 INFO:teuthology.orchestra.run.smithi046.stdout: "evexg": 0, 2024-03-18T21:01:39.477 INFO:teuthology.orchestra.run.smithi046.stdout: "evexd": 553, 2024-03-18T21:01:39.477 INFO:teuthology.orchestra.run.smithi046.stdout: "segadd": 275, 2024-03-18T21:01:39.477 INFO:teuthology.orchestra.run.smithi046.stdout: "segex": 147, 2024-03-18T21:01:39.477 INFO:teuthology.orchestra.run.smithi046.stdout: "segtrm": 147, 2024-03-18T21:01:39.477 INFO:teuthology.orchestra.run.smithi046.stdout: "seg": 129, 2024-03-18T21:01:39.477 INFO:teuthology.orchestra.run.smithi046.stdout: "segexg": 0, 2024-03-18T21:01:39.477 INFO:teuthology.orchestra.run.smithi046.stdout: "segexd": 1, 2024-03-18T21:01:39.477 INFO:teuthology.orchestra.run.smithi046.stdout: "expos": 432023380, 2024-03-18T21:01:39.478 INFO:teuthology.orchestra.run.smithi046.stdout: "wrpos": 961458720, 2024-03-18T21:01:39.478 INFO:teuthology.orchestra.run.smithi046.stdout: "rdpos": 0, 2024-03-18T21:01:39.478 INFO:teuthology.orchestra.run.smithi046.stdout: "jlat": { 2024-03-18T21:01:39.478 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 54113, 2024-03-18T21:01:39.478 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 519.634249660, 2024-03-18T21:01:39.478 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.009602761 2024-03-18T21:01:39.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.478 INFO:teuthology.orchestra.run.smithi046.stdout: "replayed": 0 2024-03-18T21:01:39.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.478 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_mem": { 2024-03-18T21:01:39.478 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1757, 2024-03-18T21:01:39.478 INFO:teuthology.orchestra.run.smithi046.stdout: "ino+": 36352, 2024-03-18T21:01:39.479 INFO:teuthology.orchestra.run.smithi046.stdout: "ino-": 34595, 2024-03-18T21:01:39.479 INFO:teuthology.orchestra.run.smithi046.stdout: "dir": 95, 2024-03-18T21:01:39.479 INFO:teuthology.orchestra.run.smithi046.stdout: "dir+": 331, 2024-03-18T21:01:39.479 INFO:teuthology.orchestra.run.smithi046.stdout: "dir-": 236, 2024-03-18T21:01:39.479 INFO:teuthology.orchestra.run.smithi046.stdout: "dn": 2857, 2024-03-18T21:01:39.479 INFO:teuthology.orchestra.run.smithi046.stdout: "dn+": 70405, 2024-03-18T21:01:39.479 INFO:teuthology.orchestra.run.smithi046.stdout: "dn-": 67548, 2024-03-18T21:01:39.479 INFO:teuthology.orchestra.run.smithi046.stdout: "cap": 3, 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: "cap+": 38926, 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: "cap-": 38923, 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: "rss": 87520, 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: "heap": 198964 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_server": { 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_client_request": 382655, 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: "dispatch_server_request": 3010, 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_request": 262910, 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_client_session": 4567, 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: "handle_peer_request": 11591, 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: "req_create_latency": { 2024-03-18T21:01:39.480 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 34091, 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 129.654712639, 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.003803194 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getattr_latency": { 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2462, 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 53.336708689, 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.021663975 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getfilelock_latency": { 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.482 INFO:teuthology.orchestra.run.smithi046.stdout: "req_link_latency": { 2024-03-18T21:01:39.482 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.482 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.482 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.482 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookup_latency": { 2024-03-18T21:01:39.482 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 71710, 2024-03-18T21:01:39.482 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 133.185039150, 2024-03-18T21:01:39.482 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001857272 2024-03-18T21:01:39.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.482 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookuphash_latency": { 2024-03-18T21:01:39.482 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.483 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.483 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.483 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupino_latency": { 2024-03-18T21:01:39.483 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 2, 2024-03-18T21:01:39.483 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000300704, 2024-03-18T21:01:39.483 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000150352 2024-03-18T21:01:39.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.483 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupname_latency": { 2024-03-18T21:01:39.483 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.483 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.483 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.484 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupparent_latency": { 2024-03-18T21:01:39.484 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.484 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.484 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.484 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lookupsnap_latency": { 2024-03-18T21:01:39.484 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.484 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.484 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.484 INFO:teuthology.orchestra.run.smithi046.stdout: "req_lssnap_latency": { 2024-03-18T21:01:39.485 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.485 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.485 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.485 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mkdir_latency": { 2024-03-18T21:01:39.485 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.485 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.485 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mknod_latency": { 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: "req_mksnap_latency": { 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: "req_open_latency": { 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.486 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: "req_readdir_latency": { 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 95543, 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 676.862750927, 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.007084378 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rename_latency": { 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 7194, 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 119.907662608, 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.016667731 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: "req_renamesnap_latency": { 2024-03-18T21:01:39.487 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: "req_snapdiff_latency": { 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmdir_latency": { 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 20, 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.160988817, 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.008049440 2024-03-18T21:01:39.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmsnap_latency": { 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: "req_rmxattr_latency": { 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setattr_latency": { 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 67, 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 2.675326980, 2024-03-18T21:01:39.489 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.039930253 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setdirlayout_latency": { 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setfilelock_latency": { 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 1184, 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 1.483538137, 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.001252988 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setlayout_latency": { 2024-03-18T21:01:39.490 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: "req_setxattr_latency": { 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: "req_symlink_latency": { 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: "req_unlink_latency": { 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 34235, 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 94.976150332, 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.002774241 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_revoke_eviction": 0, 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: "cap_acquisition_throttle": 0, 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: "req_getvxattr_latency": { 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.492 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_sessions": { 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "session_count": 2, 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "session_add": 3, 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "session_remove": 1, 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_open": 2, 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "sessions_stale": 0, 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "total_load": 23907, 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "average_load": 11953, 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "avg_session_uptime": 1513, 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "mdthresh_evicted": 0 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-03-18T21:01:39.493 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-03-18T21:01:39.494 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-03-18T21:01:39.495 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 3735189, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 1680, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 352, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 4, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-03-18T21:01:39.496 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 8256, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 188, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 6798903, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 92166, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-03-18T21:01:39.497 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: "objecter": { 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: "op_active": 0, 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: "op_laggy": 0, 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send": 93350, 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: "op_send_bytes": 964811348, 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: "op_resend": 26, 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: "op_reply": 93350, 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: "op_latency": { 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 93350, 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 953.439792577, 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.010213602 2024-03-18T21:01:39.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "op_inflight": 0, 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "oplen_avg": { 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 93350, 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 94430 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "op": 93350, 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "op_r": 1825, 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "op_w": 91525, 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "op_rmw": 0, 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "op_pg": 0, 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_stat": 18, 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_create": 150, 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_read": 1390, 2024-03-18T21:01:39.499 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_write": 55504, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writefull": 839, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_writesame": 0, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_append": 0, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_zero": 0, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_truncate": 12, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_delete": 34564, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_mapext": 0, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_sparse_read": 0, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_clonerange": 0, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_getxattr": 434, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_setxattr": 300, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_cmpxattr": 0, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_rmxattr": 0, 2024-03-18T21:01:39.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_resetxattrs": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_call": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_watch": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_notify": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_pgls_filter": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "osdop_other": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_active": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_send": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_resend": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "linger_ping": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_active": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_send": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "poolop_resend": 0, 2024-03-18T21:01:39.501 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_active": 0, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_send": 0, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "poolstat_resend": 0, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_active": 0, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_send": 0, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "statfs_resend": 0, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "command_active": 0, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "command_send": 0, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "command_resend": 0, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "map_epoch": 122, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "map_full": 0, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "map_inc": 44, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_sessions": 12, 2024-03-18T21:01:39.502 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_open": 12, 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_session_close": 0, 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_laggy": 0, 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_wr": 886, 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_rd": 2, 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_del": 331 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: "oft": { 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_objs": 1, 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_kv_pairs": 0, 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_updates": 0, 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: "omap_total_removes": 0 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: "osd-slow-ops": { 2024-03-18T21:01:39.503 INFO:teuthology.orchestra.run.smithi046.stdout: "slow_ops_count": 265436 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: "purge_queue": { 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops": 0, 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_ops_high_water": 6, 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing": 0, 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executing_high_water": 6, 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_executed": 34164, 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: "pq_item_in_journal": 0 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-03-18T21:01:39.504 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 1113121, 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 285056218, 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 1113121, 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 1113121, 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 285056218, 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.505 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_bytes": { 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 93350, 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 968271304, 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 93350, 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 968271304, 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-03-18T21:01:39.506 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-objecter_ops": { 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 1024, 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 0, 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 0, 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 93350, 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 93350, 2024-03-18T21:01:39.507 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 93350, 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 93350, 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle": { 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 34164, 2024-03-18T21:01:39.508 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 3459972, 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 34164, 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 1391, 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 3459972, 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-write_buf_throttle-0x55902c1aa0a0": { 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-03-18T21:01:39.509 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 3758096384, 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 168985, 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 957264416, 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 168985, 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 54113, 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 957264416, 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:39.510 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:39.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:39.511 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T21:01:39.826 INFO:tasks.check_counter:Daemon mds.g mds.exported=19 2024-03-18T21:01:39.826 INFO:tasks.check_counter:Daemon mds.g mds.imported=12 2024-03-18T21:01:39.826 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-03-18T21:01:39.826 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-03-18T21:01:39.826 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T21:01:39.826 DEBUG:tasks.check_counter:Getting stats from j 2024-03-18T21:01:39.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-18T21:01:40.121 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T21:01:40.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:40 smithi073 ceph-mon[47384]: pgmap v810: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.3 MiB/s wr, 292 op/s 2024-03-18T21:01:40.983 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:40 smithi046 ceph-mon[43495]: pgmap v810: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.3 MiB/s wr, 292 op/s 2024-03-18T21:01:41.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:40 smithi155 ceph-mon[48344]: pgmap v810: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.3 MiB/s wr, 292 op/s 2024-03-18T21:01:41.187 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-18T21:01:41.187 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.001828688, 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000548719, 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 3, 2024-03-18T21:01:41.189 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 334, 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 480, 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 287168, 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 6, 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.025685008, 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.018736132, 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002857055, 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 331, 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.043332128, 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000130912 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:41.190 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 287168 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 451, 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 439, 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1491056, 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 250992, 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-03-18T21:01:41.191 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.074617346, 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.027158381, 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.041379567, 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.010506378, 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 439, 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.042986225, 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000097918 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:41.192 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1491056, 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 250992 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-03-18T21:01:41.193 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-03-18T21:01:41.194 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-03-18T21:01:41.195 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 148731, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 31, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-03-18T21:01:41.196 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-03-18T21:01:41.197 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-03-18T21:01:41.198 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-03-18T21:01:41.198 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 450, 2024-03-18T21:01:41.198 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1424980, 2024-03-18T21:01:41.198 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:41.198 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 450, 2024-03-18T21:01:41.198 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-03-18T21:01:41.198 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-03-18T21:01:41.198 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 450, 2024-03-18T21:01:41.198 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1424980, 2024-03-18T21:01:41.198 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-03-18T21:01:41.199 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-03-18T21:01:41.199 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-03-18T21:01:41.199 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-03-18T21:01:41.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:41.199 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-18T21:01:41.199 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-18T21:01:41.577 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-18T21:01:41.577 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-18T21:01:41.577 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-18T21:01:41.577 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-18T21:01:41.577 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-18T21:01:41.577 DEBUG:tasks.check_counter:Getting stats from b 2024-03-18T21:01:41.577 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-18T21:01:42.162 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T21:01:42.726 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:42 smithi073 ceph-mon[47384]: pgmap v811: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.4 MiB/s wr, 253 op/s 2024-03-18T21:01:43.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:42 smithi155 ceph-mon[48344]: pgmap v811: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.4 MiB/s wr, 253 op/s 2024-03-18T21:01:43.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:42 smithi046 ceph-mon[43495]: pgmap v811: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.4 MiB/s wr, 253 op/s 2024-03-18T21:01:43.800 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-03-18T21:01:43.800 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T21:01:43.800 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 696123, 2024-03-18T21:01:43.800 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 877580, 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 228957152, 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 845190976, 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 31, 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 3, 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 36.963781910, 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 15.589670676, 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 27.477570949, 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 1.803434954, 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 877580, 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 242.637031378, 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000276484 2024-03-18T21:01:43.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1265919, 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.226081179, 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000178 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 2887616, 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 563184 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 58002, 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 58183, 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 19110515, 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 329023071, 2024-03-18T21:01:43.802 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 415, 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 4.667543586, 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 1.417700275, 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.946361553, 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 2.040436244, 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 58180, 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 56.441310593, 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000970115 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 684, 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000078978, 2024-03-18T21:01:43.803 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000115 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 94592, 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 780688 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 561003, 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 569457, 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 215459279, 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 416697938, 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 9, 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 7, 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 29.433739426, 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 12.794403576, 2024-03-18T21:01:43.804 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 19.874271487, 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 1.327292003, 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 569455, 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 110.394873913, 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000193860 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1024861, 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.158627926, 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000154 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T21:01:43.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 137198, 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 149.059876787, 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001086458 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-03-18T21:01:43.806 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 60563, 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 12.188346546, 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000201250 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 310773, 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 277194, 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 277194, 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1652.037981580, 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.005959861 2024-03-18T21:01:43.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 33579, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 30, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 2, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 4008, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 12, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 9, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 2797, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 1372, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 1157, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 268, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 819, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 2231, 2024-03-18T21:01:43.808 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 2, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 2, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 6, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 474146, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 294383, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 33564, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 143, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 4460, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 130, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 33, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 1442, 2024-03-18T21:01:43.809 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 33, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 1045, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 855, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 4, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 234, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 6, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 855, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 444, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 24, 2024-03-18T21:01:43.810 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 420, 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 411, 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 55, 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 49662, 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 130483, 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 1517, 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 90890, 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 28895, 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 4694, 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 200852 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-03-18T21:01:43.811 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 18, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 40, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 26, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 159, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 195, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 33, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 35527, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 35494, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-03-18T21:01:43.812 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 24, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 36, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 21, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 186276, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 184543, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 184543, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 1733, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-03-18T21:01:43.813 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 1683, 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 364, 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 361, 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 361, 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 3, 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 4, 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 794432477, 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 800905420, 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 0, 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 74163, 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 669.845273785, 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.009032068 2024-03-18T21:01:43.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 0 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1688, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 38741, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 37053, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 112, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 401, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 289, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 2807, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 72733, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 69926, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 2, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 38519, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 38517, 2024-03-18T21:01:43.815 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 87156, 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 207156 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 447379, 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 19986, 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 310773, 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 4828, 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 46943, 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 36021, 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 204.095164187, 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.005666004 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.816 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 36653, 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 310.319635607, 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.008466418 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.817 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 59066, 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 327.132407592, 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.005538421 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 25, 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.032631009, 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001305240 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 119, 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.694346705, 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.005834846 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 25, 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 18.435525076, 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.737421003 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 9, 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.014268630, 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001585403 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 101589, 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 446.617150682, 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.004396314 2024-03-18T21:01:43.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6805, 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 167.569613317, 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.024624483 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.822 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 44, 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.636377723, 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.014463130 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 19, 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 20.070359380, 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 1.056334704 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.823 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 58, 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1.397101934, 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.024087964 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1260, 2024-03-18T21:01:43.824 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 2.673653636, 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.002121947 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 7, 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.019111137, 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.002730162 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.825 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.826 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.826 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-03-18T21:01:43.826 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 35494, 2024-03-18T21:01:43.826 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 152.330634965, 2024-03-18T21:01:43.826 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.004291729 2024-03-18T21:01:43.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.826 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-03-18T21:01:43.826 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-03-18T21:01:43.826 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 2, 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 4, 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 2, 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 2, 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 23080, 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 11540, 2024-03-18T21:01:43.827 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 1517, 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 240, 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 240, 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T21:01:43.828 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-03-18T21:01:43.829 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 935462, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 1350, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 264, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 3, 2024-03-18T21:01:43.830 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 8256, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 188, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 6416672, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 63987, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-03-18T21:01:43.831 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 135465, 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 834300507, 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 37, 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 135465, 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 135465, 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 3372.380824760, 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.024894849 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-03-18T21:01:43.832 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 135465, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 177936 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 135465, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 2002, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 133463, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 3857, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 16176, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 1091, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 75230, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 1067, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-03-18T21:01:43.833 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 12, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 36435, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 891, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 32352, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-03-18T21:01:43.834 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-03-18T21:01:43.835 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-03-18T21:01:43.836 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-03-18T21:01:43.836 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-03-18T21:01:43.836 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-03-18T21:01:43.836 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-03-18T21:01:43.836 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 122, 2024-03-18T21:01:43.836 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-03-18T21:01:43.836 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 45, 2024-03-18T21:01:43.836 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 12, 2024-03-18T21:01:43.836 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 12, 2024-03-18T21:01:43.836 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-03-18T21:01:43.836 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-03-18T21:01:43.836 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 8311, 2024-03-18T21:01:43.837 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 64, 2024-03-18T21:01:43.837 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 2450 2024-03-18T21:01:43.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.837 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-03-18T21:01:43.837 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 1, 2024-03-18T21:01:43.837 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-03-18T21:01:43.837 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-03-18T21:01:43.837 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-03-18T21:01:43.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.837 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-03-18T21:01:43.837 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 330908 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 6, 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 6, 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 35494, 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-03-18T21:01:43.838 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 1315124, 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 345035452, 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 1315124, 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 1315124, 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 345035452, 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.839 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 135465, 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 837894313, 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 135465, 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 837894313, 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.840 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 135465, 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 135465, 2024-03-18T21:01:43.841 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 135465, 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 135465, 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 35494, 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 3594294, 2024-03-18T21:01:43.842 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 35494, 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 1067, 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 3594294, 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x55b6c337c0a0": { 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-03-18T21:01:43.843 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 186276, 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 796711116, 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 186276, 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 74163, 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 796711116, 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:43.844 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-03-18T21:01:44.163 INFO:tasks.check_counter:Daemon mds.b mds.exported=33 2024-03-18T21:01:44.164 INFO:tasks.check_counter:Daemon mds.b mds.imported=33 2024-03-18T21:01:44.164 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-18T21:01:44.164 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=25 2024-03-18T21:01:44.164 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=19 2024-03-18T21:01:44.164 DEBUG:tasks.check_counter:Getting stats from e 2024-03-18T21:01:44.164 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-18T21:01:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T21:01:44.841 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:44 smithi073 ceph-mon[47384]: pgmap v812: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.4 MiB/s wr, 251 op/s 2024-03-18T21:01:45.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:44 smithi155 ceph-mon[48344]: pgmap v812: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.4 MiB/s wr, 251 op/s 2024-03-18T21:01:45.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:44 smithi046 ceph-mon[43495]: pgmap v812: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.4 MiB/s wr, 251 op/s 2024-03-18T21:01:45.416 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-03-18T21:01:45.416 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T21:01:45.416 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 1619, 2024-03-18T21:01:45.416 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 1621, 2024-03-18T21:01:45.416 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 369527750, 2024-03-18T21:01:45.416 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 503573, 2024-03-18T21:01:45.416 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 9, 2024-03-18T21:01:45.416 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 4, 2024-03-18T21:01:45.416 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.307750118, 2024-03-18T21:01:45.416 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.036266003, 2024-03-18T21:01:45.416 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.215075890, 2024-03-18T21:01:45.416 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.043433346, 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1619, 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.266905358, 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000164858 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.417 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T21:01:45.418 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 2047, 2024-03-18T21:01:45.418 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 2378, 2024-03-18T21:01:45.418 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 401279931, 2024-03-18T21:01:45.418 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 1371040, 2024-03-18T21:01:45.418 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 9, 2024-03-18T21:01:45.418 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 7, 2024-03-18T21:01:45.418 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.418363976, 2024-03-18T21:01:45.418 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.048822979, 2024-03-18T21:01:45.418 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.286074461, 2024-03-18T21:01:45.418 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.064909196, 2024-03-18T21:01:45.418 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:45.418 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2375, 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.262421077, 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000110493 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 735040 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T21:01:45.419 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 3258, 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 3158, 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 258723731, 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 1089938, 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 5, 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 4, 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.449969321, 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.083814870, 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.251260140, 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.102507070, 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 3158, 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.318401159, 2024-03-18T21:01:45.420 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000100823 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 2062672, 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 259568 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.421 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 10365, 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.306929778, 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000029612 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.422 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 0, 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 0, 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 0, 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-03-18T21:01:45.423 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 0, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 0, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 37689, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 16476, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 21213, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 0, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 14045, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 0, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 0, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 0, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 1, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 0, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 0, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 0, 2024-03-18T21:01:45.424 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 0, 2024-03-18T21:01:45.425 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T21:01:45.426 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 0, 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 0, 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 0, 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 0 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 0, 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 0, 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 0, 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 0, 2024-03-18T21:01:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T21:01:45.428 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 0, 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 0, 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 0, 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 176326, 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-03-18T21:01:45.429 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 0, 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 129, 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 499132193, 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 1029889292, 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 1029879372, 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.430 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 176326 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1389, 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 37726, 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 36337, 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 52, 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 146, 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 94, 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 37689, 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 38194, 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 505, 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 0, 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 0, 2024-03-18T21:01:45.431 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 0, 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 91004, 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 207156 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 0, 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 0, 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 0, 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 0, 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 0, 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.433 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.434 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.435 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-03-18T21:01:45.436 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.437 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.437 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.437 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-03-18T21:01:45.437 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.437 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.437 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.437 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-03-18T21:01:45.437 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.437 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.437 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.438 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-03-18T21:01:45.438 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.438 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.438 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.438 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-03-18T21:01:45.438 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.438 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.438 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.438 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.439 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.440 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.441 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.441 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-03-18T21:01:45.441 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.441 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.441 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.441 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-03-18T21:01:45.441 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.441 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.442 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.443 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 1, 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 3, 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 2, 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 2, 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 0, 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 0, 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 4340, 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.444 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T21:01:45.445 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T21:01:45.446 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T21:01:45.447 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-03-18T21:01:45.447 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-03-18T21:01:45.447 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-03-18T21:01:45.447 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-03-18T21:01:45.447 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T21:01:45.447 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-03-18T21:01:45.447 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T21:01:45.447 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-03-18T21:01:45.447 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 294671, 2024-03-18T21:01:45.447 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 1619, 2024-03-18T21:01:45.447 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-03-18T21:01:45.448 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-03-18T21:01:45.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-03-18T21:01:45.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-03-18T21:01:45.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-03-18T21:01:45.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-03-18T21:01:45.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-03-18T21:01:45.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-03-18T21:01:45.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 8256, 2024-03-18T21:01:45.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 188, 2024-03-18T21:01:45.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T21:01:45.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-03-18T21:01:45.448 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 30058865, 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 187317, 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 6334, 2024-03-18T21:01:45.449 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 21, 2024-03-18T21:01:45.450 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 3, 2024-03-18T21:01:45.450 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 6334, 2024-03-18T21:01:45.450 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-03-18T21:01:45.450 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6334, 2024-03-18T21:01:45.450 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 22.345501393, 2024-03-18T21:01:45.450 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.003527865 2024-03-18T21:01:45.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.450 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-03-18T21:01:45.450 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-03-18T21:01:45.450 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6334, 2024-03-18T21:01:45.450 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 6335 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 6334, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 6334, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 0, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 3509, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 0, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 2823, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 0, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 0, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-03-18T21:01:45.451 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 1, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T21:01:45.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-03-18T21:01:45.453 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 122, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 44, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 11, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 12, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 1, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 0, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 2, 2024-03-18T21:01:45.454 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 0 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 0, 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 0 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T21:01:45.455 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 0, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 0, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 6920, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 1028876201, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 6920, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-03-18T21:01:45.456 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 6920, 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 1028876201, 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:45.457 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:45.458 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 6334, 2024-03-18T21:01:45.458 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 1025726016, 2024-03-18T21:01:45.458 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 6334, 2024-03-18T21:01:45.458 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 1025726016, 2024-03-18T21:01:45.458 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:45.458 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.458 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.458 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.458 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:45.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.458 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-03-18T21:01:45.458 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 6334, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 6334, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 6334, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 6334, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.459 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-03-18T21:01:45.460 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x5582a6a125a0": { 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-03-18T21:01:45.461 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-03-18T21:01:45.462 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-03-18T21:01:45.462 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-03-18T21:01:45.462 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:45.462 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:45.462 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:45.462 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:45.462 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:45.462 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:45.462 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-03-18T21:01:45.789 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-18T21:01:45.789 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-18T21:01:45.789 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-03-18T21:01:45.789 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-18T21:01:45.789 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T21:01:45.789 DEBUG:tasks.check_counter:Getting stats from h 2024-03-18T21:01:45.789 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-18T21:01:46.014 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T21:01:46.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:46 smithi073 ceph-mon[47384]: pgmap v813: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.5 MiB/s wr, 252 op/s 2024-03-18T21:01:46.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:46 smithi155 ceph-mon[48344]: pgmap v813: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.5 MiB/s wr, 252 op/s 2024-03-18T21:01:46.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:46 smithi046 ceph-mon[43495]: pgmap v813: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.5 MiB/s wr, 252 op/s 2024-03-18T21:01:46.998 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-03-18T21:01:46.998 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T21:01:46.998 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 201115, 2024-03-18T21:01:46.998 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 189191, 2024-03-18T21:01:46.999 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 73290094, 2024-03-18T21:01:46.999 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 225345668, 2024-03-18T21:01:46.999 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 53, 2024-03-18T21:01:46.999 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 4, 2024-03-18T21:01:46.999 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 12.147923879, 2024-03-18T21:01:46.999 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 5.423695599, 2024-03-18T21:01:46.999 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 7.587619586, 2024-03-18T21:01:46.999 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.709640139, 2024-03-18T21:01:46.999 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:46.999 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 189192, 2024-03-18T21:01:47.000 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 35.301128742, 2024-03-18T21:01:47.000 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000186588 2024-03-18T21:01:47.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.000 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:47.000 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 333346, 2024-03-18T21:01:47.000 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.059896505, 2024-03-18T21:01:47.000 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000179 2024-03-18T21:01:47.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.000 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 2078352, 2024-03-18T21:01:47.000 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 323232 2024-03-18T21:01:47.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.001 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T21:01:47.001 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 653105, 2024-03-18T21:01:47.001 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 844254, 2024-03-18T21:01:47.001 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 227737255, 2024-03-18T21:01:47.001 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 883399643, 2024-03-18T21:01:47.001 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 12, 2024-03-18T21:01:47.001 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 6, 2024-03-18T21:01:47.001 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 34.776348991, 2024-03-18T21:01:47.001 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 15.315982417, 2024-03-18T21:01:47.001 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 26.745140805, 2024-03-18T21:01:47.002 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.963894347, 2024-03-18T21:01:47.002 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:47.002 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 844255, 2024-03-18T21:01:47.002 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 224.430292605, 2024-03-18T21:01:47.002 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000265832 2024-03-18T21:01:47.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.002 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:47.002 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1219351, 2024-03-18T21:01:47.002 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.240447478, 2024-03-18T21:01:47.003 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000197 2024-03-18T21:01:47.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.003 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-03-18T21:01:47.003 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 737568 2024-03-18T21:01:47.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.003 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T21:01:47.003 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 200403, 2024-03-18T21:01:47.003 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 186060, 2024-03-18T21:01:47.003 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 82799012, 2024-03-18T21:01:47.003 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 503563235, 2024-03-18T21:01:47.004 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 22, 2024-03-18T21:01:47.004 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 8, 2024-03-18T21:01:47.004 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 11.338335684, 2024-03-18T21:01:47.004 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 4.301617734, 2024-03-18T21:01:47.004 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 6.612005708, 2024-03-18T21:01:47.004 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 1.631349359, 2024-03-18T21:01:47.004 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:47.004 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 186053, 2024-03-18T21:01:47.004 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 54.416824568, 2024-03-18T21:01:47.005 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000292480 2024-03-18T21:01:47.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.005 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:47.005 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 286071, 2024-03-18T21:01:47.005 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.051489236, 2024-03-18T21:01:47.005 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000179 2024-03-18T21:01:47.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.005 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 1840, 2024-03-18T21:01:47.005 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 2832 2024-03-18T21:01:47.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.006 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-03-18T21:01:47.006 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-03-18T21:01:47.006 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-03-18T21:01:47.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.006 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-03-18T21:01:47.006 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-03-18T21:01:47.006 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-03-18T21:01:47.006 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 94335, 2024-03-18T21:01:47.007 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 113.203867367, 2024-03-18T21:01:47.007 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001200019 2024-03-18T21:01:47.007 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:47.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.007 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-03-18T21:01:47.007 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-03-18T21:01:47.007 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-03-18T21:01:47.007 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 54980, 2024-03-18T21:01:47.008 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 9.884629019, 2024-03-18T21:01:47.008 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000179785 2024-03-18T21:01:47.008 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:47.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.008 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-03-18T21:01:47.008 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 252604, 2024-03-18T21:01:47.008 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 234797, 2024-03-18T21:01:47.008 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-03-18T21:01:47.008 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 234797, 2024-03-18T21:01:47.008 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1012.262314277, 2024-03-18T21:01:47.008 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.004311223 2024-03-18T21:01:47.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.009 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-03-18T21:01:47.009 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 17807, 2024-03-18T21:01:47.009 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-03-18T21:01:47.009 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-03-18T21:01:47.009 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 141, 2024-03-18T21:01:47.009 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 10, 2024-03-18T21:01:47.009 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 8, 2024-03-18T21:01:47.009 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 2499, 2024-03-18T21:01:47.009 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 1101, 2024-03-18T21:01:47.009 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 1325, 2024-03-18T21:01:47.009 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 73, 2024-03-18T21:01:47.010 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 914, 2024-03-18T21:01:47.010 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 838, 2024-03-18T21:01:47.010 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 1, 2024-03-18T21:01:47.010 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 1, 2024-03-18T21:01:47.010 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 6, 2024-03-18T21:01:47.010 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 365099, 2024-03-18T21:01:47.010 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 227137, 2024-03-18T21:01:47.010 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 17800, 2024-03-18T21:01:47.010 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 132, 2024-03-18T21:01:47.010 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-03-18T21:01:47.010 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-03-18T21:01:47.011 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 3528, 2024-03-18T21:01:47.011 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-03-18T21:01:47.011 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-03-18T21:01:47.011 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 12, 2024-03-18T21:01:47.011 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 402, 2024-03-18T21:01:47.011 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 12, 2024-03-18T21:01:47.011 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 440, 2024-03-18T21:01:47.011 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-03-18T21:01:47.011 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 244, 2024-03-18T21:01:47.011 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-03-18T21:01:47.011 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 4, 2024-03-18T21:01:47.012 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 234, 2024-03-18T21:01:47.012 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 6, 2024-03-18T21:01:47.012 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 244, 2024-03-18T21:01:47.012 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-03-18T21:01:47.012 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T21:01:47.012 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-03-18T21:01:47.012 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 20, 2024-03-18T21:01:47.012 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T21:01:47.012 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 20, 2024-03-18T21:01:47.012 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 224, 2024-03-18T21:01:47.013 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 8, 2024-03-18T21:01:47.013 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 47844, 2024-03-18T21:01:47.013 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 143436, 2024-03-18T21:01:47.013 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T21:01:47.013 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 2028, 2024-03-18T21:01:47.013 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T21:01:47.013 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 94455, 2024-03-18T21:01:47.013 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 31342, 2024-03-18T21:01:47.013 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 4996, 2024-03-18T21:01:47.014 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 170199 2024-03-18T21:01:47.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.014 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-03-18T21:01:47.014 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 17, 2024-03-18T21:01:47.014 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 70, 2024-03-18T21:01:47.014 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 49, 2024-03-18T21:01:47.014 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 169, 2024-03-18T21:01:47.014 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 164, 2024-03-18T21:01:47.014 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 20, 2024-03-18T21:01:47.015 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-03-18T21:01:47.015 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-03-18T21:01:47.015 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 32225, 2024-03-18T21:01:47.015 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 32205, 2024-03-18T21:01:47.015 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-03-18T21:01:47.015 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-03-18T21:01:47.015 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-03-18T21:01:47.015 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-03-18T21:01:47.015 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-03-18T21:01:47.016 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-03-18T21:01:47.016 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-03-18T21:01:47.016 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T21:01:47.016 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 12, 2024-03-18T21:01:47.016 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-03-18T21:01:47.016 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 18, 2024-03-18T21:01:47.016 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-03-18T21:01:47.016 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-03-18T21:01:47.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.016 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-03-18T21:01:47.017 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 164697, 2024-03-18T21:01:47.017 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 99821, 2024-03-18T21:01:47.017 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 99821, 2024-03-18T21:01:47.017 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 64876, 2024-03-18T21:01:47.017 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-03-18T21:01:47.017 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 554, 2024-03-18T21:01:47.017 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 267, 2024-03-18T21:01:47.017 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 139, 2024-03-18T21:01:47.017 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 139, 2024-03-18T21:01:47.017 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 129, 2024-03-18T21:01:47.017 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-03-18T21:01:47.018 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 1, 2024-03-18T21:01:47.018 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 415256236, 2024-03-18T21:01:47.018 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 951512246, 2024-03-18T21:01:47.018 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 0, 2024-03-18T21:01:47.018 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-03-18T21:01:47.018 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 52346, 2024-03-18T21:01:47.018 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 477.406163066, 2024-03-18T21:01:47.018 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.009120203 2024-03-18T21:01:47.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.018 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 0 2024-03-18T21:01:47.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.019 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-03-18T21:01:47.019 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1522, 2024-03-18T21:01:47.019 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 34241, 2024-03-18T21:01:47.019 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 32719, 2024-03-18T21:01:47.019 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 90, 2024-03-18T21:01:47.019 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 350, 2024-03-18T21:01:47.019 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 260, 2024-03-18T21:01:47.019 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 2513, 2024-03-18T21:01:47.019 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 64787, 2024-03-18T21:01:47.020 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 62274, 2024-03-18T21:01:47.020 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 1, 2024-03-18T21:01:47.020 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 35873, 2024-03-18T21:01:47.020 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 35872, 2024-03-18T21:01:47.020 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 81392, 2024-03-18T21:01:47.020 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 198964 2024-03-18T21:01:47.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.020 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-03-18T21:01:47.020 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 355610, 2024-03-18T21:01:47.020 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 4195, 2024-03-18T21:01:47.021 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 252604, 2024-03-18T21:01:47.021 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 4798, 2024-03-18T21:01:47.021 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 11942, 2024-03-18T21:01:47.021 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-03-18T21:01:47.021 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 32175, 2024-03-18T21:01:47.021 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 99.271123883, 2024-03-18T21:01:47.021 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.003085349 2024-03-18T21:01:47.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.021 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-03-18T21:01:47.021 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 5245, 2024-03-18T21:01:47.022 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 105.391169750, 2024-03-18T21:01:47.022 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.020093645 2024-03-18T21:01:47.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.022 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-03-18T21:01:47.022 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.022 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.022 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.022 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-03-18T21:01:47.022 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.023 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.023 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.023 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-03-18T21:01:47.023 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 59496, 2024-03-18T21:01:47.023 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 132.446045448, 2024-03-18T21:01:47.023 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.002226133 2024-03-18T21:01:47.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.023 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-03-18T21:01:47.023 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.024 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.024 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.024 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-03-18T21:01:47.024 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.024 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.024 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.024 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-03-18T21:01:47.024 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.024 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.024 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.025 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-03-18T21:01:47.025 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.025 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.025 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.025 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-03-18T21:01:47.025 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.025 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.025 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.026 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-03-18T21:01:47.026 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.026 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.026 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.026 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-03-18T21:01:47.026 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.026 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.026 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.026 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-03-18T21:01:47.027 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.027 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.027 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.027 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-03-18T21:01:47.027 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.027 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.027 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.027 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-03-18T21:01:47.027 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.027 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.028 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.028 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-03-18T21:01:47.028 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 96577, 2024-03-18T21:01:47.028 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 561.292073326, 2024-03-18T21:01:47.028 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.005811860 2024-03-18T21:01:47.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.028 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-03-18T21:01:47.028 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 8014, 2024-03-18T21:01:47.028 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 23.341102413, 2024-03-18T21:01:47.028 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.002912540 2024-03-18T21:01:47.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.029 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-03-18T21:01:47.029 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.029 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.029 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.029 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-03-18T21:01:47.029 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.029 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.029 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.029 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-03-18T21:01:47.030 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 20, 2024-03-18T21:01:47.030 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.117659627, 2024-03-18T21:01:47.030 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.005882981 2024-03-18T21:01:47.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.030 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-03-18T21:01:47.030 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.030 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.030 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.030 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-03-18T21:01:47.030 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.031 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.031 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.031 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-03-18T21:01:47.031 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 81, 2024-03-18T21:01:47.031 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1.652561517, 2024-03-18T21:01:47.031 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.020401994 2024-03-18T21:01:47.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.031 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-03-18T21:01:47.031 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.031 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.031 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.032 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-03-18T21:01:47.032 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 984, 2024-03-18T21:01:47.032 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1.209717607, 2024-03-18T21:01:47.032 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001229387 2024-03-18T21:01:47.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.032 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-03-18T21:01:47.032 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.032 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.032 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.032 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-03-18T21:01:47.033 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.033 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.033 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.033 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-03-18T21:01:47.033 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.033 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.033 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.033 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-03-18T21:01:47.033 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 32205, 2024-03-18T21:01:47.034 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 87.540860706, 2024-03-18T21:01:47.034 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.002718238 2024-03-18T21:01:47.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.034 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-03-18T21:01:47.034 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-03-18T21:01:47.034 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-03-18T21:01:47.034 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.034 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.034 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.035 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:47.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.035 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-03-18T21:01:47.035 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 2, 2024-03-18T21:01:47.035 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 4, 2024-03-18T21:01:47.035 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 2, 2024-03-18T21:01:47.035 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 2, 2024-03-18T21:01:47.035 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-03-18T21:01:47.035 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 20856, 2024-03-18T21:01:47.035 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 10428, 2024-03-18T21:01:47.036 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 1481, 2024-03-18T21:01:47.036 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-03-18T21:01:47.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.036 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-03-18T21:01:47.036 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-03-18T21:01:47.036 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-03-18T21:01:47.036 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T21:01:47.036 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-03-18T21:01:47.036 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T21:01:47.036 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-03-18T21:01:47.037 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T21:01:47.037 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T21:01:47.037 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T21:01:47.037 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T21:01:47.037 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T21:01:47.037 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-03-18T21:01:47.037 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T21:01:47.037 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T21:01:47.037 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-03-18T21:01:47.037 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-03-18T21:01:47.037 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-03-18T21:01:47.038 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-03-18T21:01:47.038 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T21:01:47.038 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T21:01:47.038 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T21:01:47.038 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T21:01:47.038 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T21:01:47.038 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-03-18T21:01:47.038 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-03-18T21:01:47.038 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-03-18T21:01:47.039 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T21:01:47.039 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T21:01:47.039 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-03-18T21:01:47.039 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-03-18T21:01:47.039 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-03-18T21:01:47.039 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-03-18T21:01:47.039 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T21:01:47.039 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-03-18T21:01:47.039 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T21:01:47.039 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-03-18T21:01:47.039 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 494425, 2024-03-18T21:01:47.040 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 754, 2024-03-18T21:01:47.040 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 176, 2024-03-18T21:01:47.040 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 2, 2024-03-18T21:01:47.040 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-03-18T21:01:47.040 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-03-18T21:01:47.040 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-03-18T21:01:47.040 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-03-18T21:01:47.040 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-03-18T21:01:47.040 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-03-18T21:01:47.041 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 8256, 2024-03-18T21:01:47.041 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 188, 2024-03-18T21:01:47.041 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T21:01:47.041 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-03-18T21:01:47.041 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-03-18T21:01:47.041 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-03-18T21:01:47.041 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 5820178, 2024-03-18T21:01:47.041 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 45838, 2024-03-18T21:01:47.041 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-03-18T21:01:47.041 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-03-18T21:01:47.042 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-03-18T21:01:47.042 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-03-18T21:01:47.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.042 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-03-18T21:01:47.042 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-03-18T21:01:47.042 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-03-18T21:01:47.042 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 89124, 2024-03-18T21:01:47.042 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 954387511, 2024-03-18T21:01:47.042 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 27, 2024-03-18T21:01:47.042 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 89124, 2024-03-18T21:01:47.043 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-03-18T21:01:47.043 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 89124, 2024-03-18T21:01:47.043 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 912.623671414, 2024-03-18T21:01:47.043 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.010239931 2024-03-18T21:01:47.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.043 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-03-18T21:01:47.043 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-03-18T21:01:47.043 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 89124, 2024-03-18T21:01:47.043 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 90176 2024-03-18T21:01:47.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.044 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 89124, 2024-03-18T21:01:47.044 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 1526, 2024-03-18T21:01:47.044 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 87598, 2024-03-18T21:01:47.044 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-03-18T21:01:47.044 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-03-18T21:01:47.044 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 21, 2024-03-18T21:01:47.044 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 150, 2024-03-18T21:01:47.044 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 1282, 2024-03-18T21:01:47.044 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 53628, 2024-03-18T21:01:47.044 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 784, 2024-03-18T21:01:47.045 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-03-18T21:01:47.045 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-03-18T21:01:47.045 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-03-18T21:01:47.045 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 10, 2024-03-18T21:01:47.045 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 32590, 2024-03-18T21:01:47.045 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-03-18T21:01:47.045 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-03-18T21:01:47.045 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-03-18T21:01:47.045 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 244, 2024-03-18T21:01:47.045 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 300, 2024-03-18T21:01:47.046 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-03-18T21:01:47.046 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-03-18T21:01:47.046 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-03-18T21:01:47.046 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-03-18T21:01:47.046 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-03-18T21:01:47.046 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-03-18T21:01:47.046 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T21:01:47.046 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-03-18T21:01:47.046 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-03-18T21:01:47.047 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-03-18T21:01:47.047 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-03-18T21:01:47.047 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-03-18T21:01:47.047 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-03-18T21:01:47.047 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-03-18T21:01:47.047 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-03-18T21:01:47.047 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-03-18T21:01:47.047 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-03-18T21:01:47.047 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-03-18T21:01:47.047 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-03-18T21:01:47.047 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-03-18T21:01:47.048 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-03-18T21:01:47.048 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-03-18T21:01:47.048 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-03-18T21:01:47.048 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-03-18T21:01:47.048 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-03-18T21:01:47.048 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-03-18T21:01:47.048 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 122, 2024-03-18T21:01:47.048 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-03-18T21:01:47.048 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 44, 2024-03-18T21:01:47.048 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 12, 2024-03-18T21:01:47.049 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 12, 2024-03-18T21:01:47.049 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-03-18T21:01:47.049 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-03-18T21:01:47.049 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 848, 2024-03-18T21:01:47.049 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 0, 2024-03-18T21:01:47.049 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 319 2024-03-18T21:01:47.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.049 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-03-18T21:01:47.049 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 1, 2024-03-18T21:01:47.049 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-03-18T21:01:47.049 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-03-18T21:01:47.050 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-03-18T21:01:47.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.050 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-03-18T21:01:47.050 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 255191 2024-03-18T21:01:47.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.050 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-03-18T21:01:47.050 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-03-18T21:01:47.050 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 6, 2024-03-18T21:01:47.050 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-03-18T21:01:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 6, 2024-03-18T21:01:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 32205, 2024-03-18T21:01:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-03-18T21:01:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T21:01:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-03-18T21:01:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 1054619, 2024-03-18T21:01:47.051 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 288873966, 2024-03-18T21:01:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 1054619, 2024-03-18T21:01:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-03-18T21:01:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-03-18T21:01:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 1054619, 2024-03-18T21:01:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 288873966, 2024-03-18T21:01:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-03-18T21:01:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-03-18T21:01:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-03-18T21:01:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-03-18T21:01:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 89124, 2024-03-18T21:01:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 957650216, 2024-03-18T21:01:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 89124, 2024-03-18T21:01:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 957650216, 2024-03-18T21:01:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-03-18T21:01:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-03-18T21:01:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:47.054 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-03-18T21:01:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-03-18T21:01:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 89124, 2024-03-18T21:01:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 89124, 2024-03-18T21:01:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 89124, 2024-03-18T21:01:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 89124, 2024-03-18T21:01:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.055 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-03-18T21:01:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-03-18T21:01:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 32205, 2024-03-18T21:01:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 3262705, 2024-03-18T21:01:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 32205, 2024-03-18T21:01:47.056 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-03-18T21:01:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-03-18T21:01:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 1282, 2024-03-18T21:01:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 3262705, 2024-03-18T21:01:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x55dda1f9c0a0": { 2024-03-18T21:01:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:47.057 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-03-18T21:01:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 164697, 2024-03-18T21:01:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 947317942, 2024-03-18T21:01:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 164697, 2024-03-18T21:01:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-03-18T21:01:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-03-18T21:01:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 52346, 2024-03-18T21:01:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 947317942, 2024-03-18T21:01:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:47.058 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:47.059 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:47.059 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-03-18T21:01:47.371 INFO:tasks.check_counter:Daemon mds.h mds.exported=12 2024-03-18T21:01:47.371 INFO:tasks.check_counter:Daemon mds.h mds.imported=12 2024-03-18T21:01:47.371 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-03-18T21:01:47.371 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-18T21:01:47.371 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T21:01:47.371 DEBUG:tasks.check_counter:Getting stats from k 2024-03-18T21:01:47.371 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-18T21:01:47.598 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T21:01:48.768 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 654896, 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 821733, 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 242806463, 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 782413012, 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 20, 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 34.664480341, 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 15.114997262, 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 26.814673428, 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.986636448, 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:48.769 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 821730, 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 228.228675742, 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000277741 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1210437, 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.221534030, 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000183 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T21:01:48.770 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 187132, 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 190099, 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 63325643, 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 407047985, 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 16, 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 12.541112821, 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 5.478995376, 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 6.852093586, 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 1.402969349, 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 190098, 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 39.409352484, 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000207310 2024-03-18T21:01:48.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 289464, 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.051206118, 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000176 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 1984, 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 114496 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 151372, 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 141652, 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 66486638, 2024-03-18T21:01:48.772 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 368711368, 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 50, 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 6, 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 8.529575897, 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 3.269173569, 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 5.067438778, 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 1.094034700, 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 141651, 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 43.745440645, 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000308825 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 230640, 2024-03-18T21:01:48.773 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.039085344, 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000169 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 2066992, 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 924384 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 85627, 2024-03-18T21:01:48.774 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 118.031459959, 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001378437 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 58091, 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 10.420832364, 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000179388 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 233440, 2024-03-18T21:01:48.775 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 231559, 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 231559, 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1032.056832540, 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.004456992 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 1881, 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 3, 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 137, 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 10, 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 9, 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 2637, 2024-03-18T21:01:48.776 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 1213, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 1343, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 81, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 898, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 853, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 0, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 0, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 6, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 341414, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 207649, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 1863, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 99, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-03-18T21:01:48.777 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 2047, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 14, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 568, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 21, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 710, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 240, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 4, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 234, 2024-03-18T21:01:48.778 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 6, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 240, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 0, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 240, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 10, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 44699, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 132522, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 1786, 2024-03-18T21:01:48.779 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T21:01:48.780 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 87320, 2024-03-18T21:01:48.780 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 28214, 2024-03-18T21:01:48.780 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 4574, 2024-03-18T21:01:48.780 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 163594 2024-03-18T21:01:48.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.780 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-03-18T21:01:48.780 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 16, 2024-03-18T21:01:48.780 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 66, 2024-03-18T21:01:48.780 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 47, 2024-03-18T21:01:48.780 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 129, 2024-03-18T21:01:48.780 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 112, 2024-03-18T21:01:48.781 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 20, 2024-03-18T21:01:48.781 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-03-18T21:01:48.781 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-03-18T21:01:48.781 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 33858, 2024-03-18T21:01:48.781 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 33838, 2024-03-18T21:01:48.781 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-03-18T21:01:48.781 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-03-18T21:01:48.781 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-03-18T21:01:48.781 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-03-18T21:01:48.781 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 15, 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 19, 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 160857, 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 89775, 2024-03-18T21:01:48.782 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 89775, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 71082, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 597, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 255, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 127, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 127, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 129, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 1, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 390079698, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 922868496, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 0, 2024-03-18T21:01:48.783 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 48055, 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 493.446888822, 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.010268377 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 0 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1502, 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 35877, 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 34375, 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 71, 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 323, 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 252, 2024-03-18T21:01:48.784 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 2646, 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 69796, 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 67150, 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 0, 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 37430, 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 37430, 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 81256, 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 198964 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 336579, 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 553, 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 233440, 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 4303, 2024-03-18T21:01:48.785 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 5234, 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 33621, 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 132.939109419, 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.003954049 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 3027, 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 53.472916472, 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.017665317 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 63691, 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 121.623332710, 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001909584 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.787 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.788 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.789 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 89839, 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 564.488071446, 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.006283329 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6458, 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 30.176264423, 2024-03-18T21:01:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.004672695 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 20, 2024-03-18T21:01:48.792 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.103649738, 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.005182486 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-03-18T21:01:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 65, 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1.585408962, 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.024390907 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1000, 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1.427388314, 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001427388 2024-03-18T21:01:48.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.795 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.796 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-03-18T21:01:48.796 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 33838, 2024-03-18T21:01:48.796 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 126.240691056, 2024-03-18T21:01:48.796 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.003730737 2024-03-18T21:01:48.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.796 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 1, 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 3, 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 2, 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 1, 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-03-18T21:01:48.797 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 22342, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 22342, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 1537, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T21:01:48.798 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T21:01:48.799 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T21:01:48.799 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-03-18T21:01:48.799 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T21:01:48.799 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T21:01:48.799 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-03-18T21:01:48.799 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-03-18T21:01:48.799 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-03-18T21:01:48.799 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-03-18T21:01:48.799 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T21:01:48.799 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-03-18T21:01:48.800 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T21:01:48.801 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-03-18T21:01:48.801 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 765908, 2024-03-18T21:01:48.801 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 979, 2024-03-18T21:01:48.801 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 440, 2024-03-18T21:01:48.801 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 5, 2024-03-18T21:01:48.801 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-03-18T21:01:48.801 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-03-18T21:01:48.801 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-03-18T21:01:48.801 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-03-18T21:01:48.801 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-03-18T21:01:48.801 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-03-18T21:01:48.801 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 8256, 2024-03-18T21:01:48.802 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 188, 2024-03-18T21:01:48.802 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T21:01:48.802 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-03-18T21:01:48.802 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-03-18T21:01:48.802 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-03-18T21:01:48.802 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 5788586, 2024-03-18T21:01:48.802 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 55275, 2024-03-18T21:01:48.802 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-03-18T21:01:48.802 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-03-18T21:01:48.802 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 86129, 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 925698864, 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 33, 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 86129, 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 86129, 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 953.255770249, 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.011067767 2024-03-18T21:01:48.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 86129, 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 87117 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 86129, 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 1400, 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 84729, 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 15, 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 125, 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 1157, 2024-03-18T21:01:48.804 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 49212, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 725, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 10, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 34226, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 240, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 250, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-03-18T21:01:48.805 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-03-18T21:01:48.806 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 122, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 44, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 12, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 12, 2024-03-18T21:01:48.807 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 836, 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 6, 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 315 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 1, 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 234021 2024-03-18T21:01:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 6, 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 6, 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 33838, 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 993396, 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 283177162, 2024-03-18T21:01:48.809 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 993396, 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 993396, 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 283177162, 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:48.810 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 86129, 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 929126022, 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 86129, 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 929126022, 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:48.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 86129, 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 86129, 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 86129, 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 86129, 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:48.812 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 33838, 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 3427206, 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 33838, 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-03-18T21:01:48.813 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 1157, 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 3427206, 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x5585ab2640a0": { 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 14975, 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 160857, 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 918674192, 2024-03-18T21:01:48.814 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:48.815 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 160857, 2024-03-18T21:01:48.815 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-03-18T21:01:48.815 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-03-18T21:01:48.815 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 48055, 2024-03-18T21:01:48.815 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 918659217, 2024-03-18T21:01:48.815 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-03-18T21:01:48.815 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-03-18T21:01:48.815 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-03-18T21:01:48.815 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-03-18T21:01:48.815 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:48.815 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-18T21:01:48.815 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-03-18T21:01:48.816 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:48 smithi073 ceph-mon[47384]: pgmap v814: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.4 MiB/s wr, 225 op/s 2024-03-18T21:01:48.816 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:48 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T21:01:49.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:48 smithi155 ceph-mon[48344]: pgmap v814: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.4 MiB/s wr, 225 op/s 2024-03-18T21:01:49.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:48 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T21:01:49.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:48 smithi046 ceph-mon[43495]: pgmap v814: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.4 MiB/s wr, 225 op/s 2024-03-18T21:01:49.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:48 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T21:01:49.195 INFO:tasks.check_counter:Daemon mds.k mds.exported=14 2024-03-18T21:01:49.195 INFO:tasks.check_counter:Daemon mds.k mds.imported=21 2024-03-18T21:01:49.195 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-03-18T21:01:49.195 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-18T21:01:49.195 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T21:01:49.195 DEBUG:tasks.check_counter:Getting stats from c 2024-03-18T21:01:49.195 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-18T21:01:49.527 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T21:01:49.636 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:49 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T21:01:49.636 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:49 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T21:01:49.636 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:49 smithi155 ceph-mon[48344]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:01:49.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:49 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T21:01:49.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:49 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T21:01:49.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:49 smithi073 ceph-mon[47384]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:01:50.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:49 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T21:01:50.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:49 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T21:01:50.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:49 smithi046 ceph-mon[43495]: from='mgr.15498 ' entity='mgr.x' 2024-03-18T21:01:50.874 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-03-18T21:01:50.874 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T21:01:50.874 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 19, 2024-03-18T21:01:50.874 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 20, 2024-03-18T21:01:50.874 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 4201587, 2024-03-18T21:01:50.874 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 5279, 2024-03-18T21:01:50.874 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 7, 2024-03-18T21:01:50.874 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-03-18T21:01:50.874 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.007607071, 2024-03-18T21:01:50.874 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.000276640, 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.005681899, 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000408002, 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 18, 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.024432040, 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001357335 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.875 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 2608, 2024-03-18T21:01:50.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 288 2024-03-18T21:01:50.876 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.876 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T21:01:50.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 256, 2024-03-18T21:01:50.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 244, 2024-03-18T21:01:50.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 16882306, 2024-03-18T21:01:50.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 81258, 2024-03-18T21:01:50.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 5, 2024-03-18T21:01:50.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-03-18T21:01:50.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.032319151, 2024-03-18T21:01:50.876 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.006105087, 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.019768526, 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.006489969, 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 243, 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.073888096, 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000304066 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.877 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 231120, 2024-03-18T21:01:50.878 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 18064 2024-03-18T21:01:50.878 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.878 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T21:01:50.878 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 189, 2024-03-18T21:01:50.878 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 211, 2024-03-18T21:01:50.878 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 32407516, 2024-03-18T21:01:50.878 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 178773, 2024-03-18T21:01:50.878 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 6, 2024-03-18T21:01:50.878 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-03-18T21:01:50.879 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.032496741, 2024-03-18T21:01:50.879 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.004494135, 2024-03-18T21:01:50.879 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.021612550, 2024-03-18T21:01:50.879 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.004932849, 2024-03-18T21:01:50.879 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:50.879 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 209, 2024-03-18T21:01:50.879 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.049722658, 2024-03-18T21:01:50.879 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000237907 2024-03-18T21:01:50.879 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.879 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:50.879 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T21:01:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 120304 2024-03-18T21:01:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-03-18T21:01:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-03-18T21:01:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-03-18T21:01:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-03-18T21:01:50.880 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-18T21:01:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-18T21:01:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 656, 2024-03-18T21:01:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.038525006, 2024-03-18T21:01:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000058727 2024-03-18T21:01:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-03-18T21:01:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-18T21:01:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-18T21:01:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.881 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.882 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:50.882 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.882 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-03-18T21:01:50.882 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-03-18T21:01:50.882 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-03-18T21:01:50.882 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-03-18T21:01:50.882 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.882 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.882 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.882 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.882 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-03-18T21:01:50.883 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-03-18T21:01:50.883 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-03-18T21:01:50.883 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-03-18T21:01:50.883 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2024-03-18T21:01:50.883 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-03-18T21:01:50.883 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-03-18T21:01:50.883 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 3425, 2024-03-18T21:01:50.883 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 1700, 2024-03-18T21:01:50.883 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 1725, 2024-03-18T21:01:50.883 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-03-18T21:01:50.883 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 997, 2024-03-18T21:01:50.883 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-03-18T21:01:50.884 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-03-18T21:01:50.884 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-03-18T21:01:50.884 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 2, 2024-03-18T21:01:50.884 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2024-03-18T21:01:50.884 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2024-03-18T21:01:50.884 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-03-18T21:01:50.884 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-03-18T21:01:50.884 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-03-18T21:01:50.884 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-03-18T21:01:50.884 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-03-18T21:01:50.884 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-03-18T21:01:50.884 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-03-18T21:01:50.885 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-03-18T21:01:50.885 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-03-18T21:01:50.885 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-03-18T21:01:50.885 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-03-18T21:01:50.885 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-03-18T21:01:50.885 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-03-18T21:01:50.885 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-03-18T21:01:50.885 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 4, 2024-03-18T21:01:50.885 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 234, 2024-03-18T21:01:50.885 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2024-03-18T21:01:50.885 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T21:01:50.886 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-03-18T21:01:50.886 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T21:01:50.886 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-03-18T21:01:50.886 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-03-18T21:01:50.886 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T21:01:50.886 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T21:01:50.886 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-03-18T21:01:50.886 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-03-18T21:01:50.886 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T21:01:50.886 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-03-18T21:01:50.886 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T21:01:50.886 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T21:01:50.887 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T21:01:50.887 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-03-18T21:01:50.887 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-03-18T21:01:50.887 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-03-18T21:01:50.887 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-03-18T21:01:50.887 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.887 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-03-18T21:01:50.887 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-03-18T21:01:50.887 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-03-18T21:01:50.887 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-03-18T21:01:50.887 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-03-18T21:01:50.887 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-03-18T21:01:50.888 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T21:01:50.889 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-03-18T21:01:50.889 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-03-18T21:01:50.889 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-03-18T21:01:50.889 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-03-18T21:01:50.889 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-03-18T21:01:50.889 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.889 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-03-18T21:01:50.889 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2024-03-18T21:01:50.889 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-03-18T21:01:50.889 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-03-18T21:01:50.889 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 11408, 2024-03-18T21:01:50.889 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-03-18T21:01:50.890 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-03-18T21:01:50.890 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2024-03-18T21:01:50.890 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-03-18T21:01:50.890 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-03-18T21:01:50.890 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 3, 2024-03-18T21:01:50.890 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-03-18T21:01:50.890 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-03-18T21:01:50.890 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 794432477, 2024-03-18T21:01:50.890 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 800920684, 2024-03-18T21:01:50.890 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 800905420, 2024-03-18T21:01:50.890 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-03-18T21:01:50.890 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.891 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.891 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.891 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.891 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 11408 2024-03-18T21:01:50.891 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.891 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-03-18T21:01:50.891 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 300, 2024-03-18T21:01:50.891 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 2740, 2024-03-18T21:01:50.891 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 2440, 2024-03-18T21:01:50.891 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 87, 2024-03-18T21:01:50.891 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 93, 2024-03-18T21:01:50.891 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 6, 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 3425, 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 3440, 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 15, 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 52820, 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 207132 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-03-18T21:01:50.892 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-03-18T21:01:50.893 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2024-03-18T21:01:50.893 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2024-03-18T21:01:50.893 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-03-18T21:01:50.893 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.893 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.893 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.893 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.893 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-03-18T21:01:50.893 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.893 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.893 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-03-18T21:01:50.894 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.895 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.895 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.895 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.895 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-03-18T21:01:50.895 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.895 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.895 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.895 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.895 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-03-18T21:01:50.895 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.895 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.895 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-03-18T21:01:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-03-18T21:01:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.896 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.897 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.898 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.898 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-03-18T21:01:50.898 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.898 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.898 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.898 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.898 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-03-18T21:01:50.898 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.898 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.898 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.898 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.899 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-03-18T21:01:50.899 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.899 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.899 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.899 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.899 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-03-18T21:01:50.899 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.899 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.899 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.899 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.899 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-03-18T21:01:50.899 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.900 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.900 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.900 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.900 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-03-18T21:01:50.900 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.900 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.900 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.900 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.900 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-03-18T21:01:50.900 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.900 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.900 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.901 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.901 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-03-18T21:01:50.901 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.901 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.901 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.901 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.901 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-03-18T21:01:50.901 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.901 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.901 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.901 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.901 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-03-18T21:01:50.902 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.902 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.902 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.902 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.902 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-03-18T21:01:50.902 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.902 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.902 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.902 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.902 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-03-18T21:01:50.902 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.902 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-03-18T21:01:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-03-18T21:01:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.903 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.904 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-03-18T21:01:50.904 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.904 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.904 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.904 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.904 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-03-18T21:01:50.904 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.904 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.904 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.904 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.904 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-03-18T21:01:50.905 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.905 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.905 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.905 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.905 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-03-18T21:01:50.905 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-03-18T21:01:50.905 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-03-18T21:01:50.905 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.905 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.905 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.905 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:50.906 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.906 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-03-18T21:01:50.906 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2024-03-18T21:01:50.906 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 2, 2024-03-18T21:01:50.906 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 0, 2024-03-18T21:01:50.906 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 0, 2024-03-18T21:01:50.906 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-03-18T21:01:50.906 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-03-18T21:01:50.906 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-03-18T21:01:50.906 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 2928, 2024-03-18T21:01:50.906 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-03-18T21:01:50.906 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-03-18T21:01:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-03-18T21:01:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-03-18T21:01:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T21:01:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-03-18T21:01:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T21:01:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-03-18T21:01:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T21:01:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T21:01:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T21:01:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T21:01:50.907 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T21:01:50.908 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-03-18T21:01:50.908 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T21:01:50.908 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T21:01:50.908 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-03-18T21:01:50.908 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-03-18T21:01:50.908 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-03-18T21:01:50.908 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-03-18T21:01:50.908 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T21:01:50.908 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T21:01:50.908 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T21:01:50.908 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T21:01:50.909 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T21:01:50.909 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-03-18T21:01:50.909 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-03-18T21:01:50.909 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-03-18T21:01:50.909 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T21:01:50.909 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T21:01:50.909 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-03-18T21:01:50.909 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-03-18T21:01:50.909 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-03-18T21:01:50.909 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-03-18T21:01:50.909 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T21:01:50.910 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-03-18T21:01:50.910 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T21:01:50.910 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-03-18T21:01:50.910 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 237081, 2024-03-18T21:01:50.910 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 541, 2024-03-18T21:01:50.910 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-03-18T21:01:50.910 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-03-18T21:01:50.910 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-03-18T21:01:50.910 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-03-18T21:01:50.910 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-03-18T21:01:50.910 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-03-18T21:01:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-03-18T21:01:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-03-18T21:01:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8256, 2024-03-18T21:01:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 188, 2024-03-18T21:01:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T21:01:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-03-18T21:01:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-03-18T21:01:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-03-18T21:01:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 3403343, 2024-03-18T21:01:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 27002, 2024-03-18T21:01:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-03-18T21:01:50.911 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 407, 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 26, 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 4, 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 407, 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 407, 2024-03-18T21:01:50.912 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3.121941973, 2024-03-18T21:01:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.007670619 2024-03-18T21:01:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-03-18T21:01:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-03-18T21:01:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 407, 2024-03-18T21:01:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 408 2024-03-18T21:01:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 407, 2024-03-18T21:01:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 407, 2024-03-18T21:01:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2024-03-18T21:01:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-03-18T21:01:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 214, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 190, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-03-18T21:01:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-03-18T21:01:50.915 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 2, 2024-03-18T21:01:50.915 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2024-03-18T21:01:50.915 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-03-18T21:01:50.915 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-03-18T21:01:50.915 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-03-18T21:01:50.915 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-03-18T21:01:50.915 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-03-18T21:01:50.915 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-03-18T21:01:50.916 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T21:01:50.916 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-03-18T21:01:50.916 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-03-18T21:01:50.916 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-03-18T21:01:50.916 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-03-18T21:01:50.916 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-03-18T21:01:50.916 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-03-18T21:01:50.916 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-03-18T21:01:50.916 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-03-18T21:01:50.916 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-03-18T21:01:50.916 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-03-18T21:01:50.916 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-03-18T21:01:50.917 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-03-18T21:01:50.917 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-03-18T21:01:50.917 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-03-18T21:01:50.917 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-03-18T21:01:50.917 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-03-18T21:01:50.917 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-03-18T21:01:50.917 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-03-18T21:01:50.917 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-03-18T21:01:50.917 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 122, 2024-03-18T21:01:50.917 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-03-18T21:01:50.917 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 3, 2024-03-18T21:01:50.918 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 9, 2024-03-18T21:01:50.918 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 9, 2024-03-18T21:01:50.918 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-03-18T21:01:50.918 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-03-18T21:01:50.918 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2024-03-18T21:01:50.918 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2024-03-18T21:01:50.918 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2024-03-18T21:01:50.918 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.918 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-03-18T21:01:50.918 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2024-03-18T21:01:50.918 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-03-18T21:01:50.919 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-03-18T21:01:50.919 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-03-18T21:01:50.919 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.919 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-03-18T21:01:50.919 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-03-18T21:01:50.919 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.919 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-03-18T21:01:50.919 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-03-18T21:01:50.919 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T21:01:50.920 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-03-18T21:01:50.920 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-03-18T21:01:50.920 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-03-18T21:01:50.920 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-03-18T21:01:50.920 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.920 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T21:01:50.920 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:50.920 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-18T21:01:50.920 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:50.920 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 461, 2024-03-18T21:01:50.920 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 53444674, 2024-03-18T21:01:50.920 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:50.921 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 461, 2024-03-18T21:01:50.921 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-18T21:01:50.921 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-18T21:01:50.921 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 461, 2024-03-18T21:01:50.921 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 53444674, 2024-03-18T21:01:50.921 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:50.921 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.921 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.921 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.921 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:50.921 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.921 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-03-18T21:01:50.922 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:50.922 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-18T21:01:50.922 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:50.922 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:50.922 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:50.922 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:50.922 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:50.922 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 407, 2024-03-18T21:01:50.922 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 53138921, 2024-03-18T21:01:50.922 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 407, 2024-03-18T21:01:50.922 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 53138921, 2024-03-18T21:01:50.922 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:50.923 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.923 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.923 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.923 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:50.923 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.923 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-03-18T21:01:50.923 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:50.923 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-03-18T21:01:50.923 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:50.923 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:50.923 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:50.923 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 407, 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 407, 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 407, 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 407, 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-03-18T21:01:50.924 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:50.925 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-18T21:01:50.925 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:50.925 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:50.925 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:50.925 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:50.925 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:50.925 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-18T21:01:50.925 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-18T21:01:50.925 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-03-18T21:01:50.925 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-03-18T21:01:50.925 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:50.925 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.926 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.926 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.926 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:50.926 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:50.926 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55764a3be5a0": { 2024-03-18T21:01:50.926 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:50.926 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-18T21:01:50.926 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:50.926 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:50.926 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:50.926 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:50.926 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:50.927 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-18T21:01:50.927 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-18T21:01:50.927 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-03-18T21:01:50.927 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-03-18T21:01:50.927 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:50.927 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:50.927 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:50.927 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:50.927 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:50.927 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:50.927 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-03-18T21:01:50.928 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:50 smithi155 ceph-mon[48344]: pgmap v815: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 157 op/s 2024-03-18T21:01:50.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:50 smithi073 ceph-mon[47384]: pgmap v815: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 157 op/s 2024-03-18T21:01:51.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:50 smithi046 ceph-mon[43495]: pgmap v815: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 157 op/s 2024-03-18T21:01:51.288 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-18T21:01:51.288 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-18T21:01:51.288 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-03-18T21:01:51.288 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-18T21:01:51.288 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T21:01:51.288 DEBUG:tasks.check_counter:Getting stats from f 2024-03-18T21:01:51.288 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-18T21:01:51.511 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T21:01:52.530 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-03-18T21:01:52.530 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T21:01:52.530 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 3935, 2024-03-18T21:01:52.530 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 3935, 2024-03-18T21:01:52.530 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 340884565, 2024-03-18T21:01:52.530 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1222996, 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 8, 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.475710124, 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.090131590, 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.249187877, 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.108162951, 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3934, 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.244573012, 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000062169 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.531 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.532 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.532 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-03-18T21:01:52.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T21:01:52.532 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.532 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T21:01:52.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1408, 2024-03-18T21:01:52.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1740, 2024-03-18T21:01:52.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 303385717, 2024-03-18T21:01:52.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1174112, 2024-03-18T21:01:52.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 9, 2024-03-18T21:01:52.532 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 7, 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.278854230, 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.038081652, 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.190298960, 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.036922488, 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1737, 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.150223692, 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000086484 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.533 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 736784 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1676, 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1578, 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 316948284, 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 599013, 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 6, 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.321120238, 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.045370623, 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.234310541, 2024-03-18T21:01:52.534 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.041835050, 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1577, 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.117627286, 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000074589 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 2060000, 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 259712 2024-03-18T21:01:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 10627, 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.293384269, 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000027607 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.536 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.537 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.538 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.538 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-03-18T21:01:52.538 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-03-18T21:01:52.538 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-03-18T21:01:52.538 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-03-18T21:01:52.538 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2024-03-18T21:01:52.538 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-03-18T21:01:52.538 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-03-18T21:01:52.538 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 36011, 2024-03-18T21:01:52.538 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 14735, 2024-03-18T21:01:52.538 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 21276, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 14885, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 1, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-03-18T21:01:52.539 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T21:01:52.540 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-03-18T21:01:52.541 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-03-18T21:01:52.542 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-03-18T21:01:52.543 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 169058, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 128, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 432023380, 2024-03-18T21:01:52.544 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 961528866, 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 961518000, 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 169058 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 1471, 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 36240, 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 34769, 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 59, 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 172, 2024-03-18T21:01:52.545 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 113, 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 36011, 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 36621, 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 610, 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 89320, 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 207156 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-03-18T21:01:52.546 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.547 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.548 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.549 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.550 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.551 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-03-18T21:01:52.552 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-03-18T21:01:52.553 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.554 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.555 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.556 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:52.557 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.558 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-03-18T21:01:52.558 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2024-03-18T21:01:52.558 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 3, 2024-03-18T21:01:52.558 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 1, 2024-03-18T21:01:52.558 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 3, 2024-03-18T21:01:52.558 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-03-18T21:01:52.558 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-03-18T21:01:52.558 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-03-18T21:01:52.558 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 3626, 2024-03-18T21:01:52.558 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-03-18T21:01:52.558 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.558 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T21:01:52.559 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T21:01:52.560 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 297415, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 1742, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-03-18T21:01:52.561 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8256, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 188, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 29098129, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 183391, 2024-03-18T21:01:52.562 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-03-18T21:01:52.563 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-03-18T21:01:52.563 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-03-18T21:01:52.563 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-03-18T21:01:52.563 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.563 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-03-18T21:01:52.563 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-03-18T21:01:52.563 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-03-18T21:01:52.563 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 6428, 2024-03-18T21:01:52.563 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 21, 2024-03-18T21:01:52.563 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 3, 2024-03-18T21:01:52.563 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 6428, 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6428, 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 19.255632382, 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002995586 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6428, 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 6429 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 6428, 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 6428, 2024-03-18T21:01:52.564 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 3518, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 2908, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-03-18T21:01:52.565 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 1, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-03-18T21:01:52.566 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-03-18T21:01:52.567 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 122, 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 44, 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 11, 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 1, 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2024-03-18T21:01:52.568 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-03-18T21:01:52.569 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 7015, 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 960554753, 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 7015, 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 7015, 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 960554753, 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:52.570 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 6428, 2024-03-18T21:01:52.571 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 957388572, 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 6428, 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 957388572, 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:52.572 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 6428, 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 6428, 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 6428, 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 6428, 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-03-18T21:01:52.573 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.574 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55c6365b25a0": { 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-03-18T21:01:52.575 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:52.576 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:52.576 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:52.576 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:52.576 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:52.576 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:52.576 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-03-18T21:01:52.673 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:52 smithi155 ceph-mon[48344]: pgmap v816: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 584 KiB/s wr, 58 op/s 2024-03-18T21:01:52.879 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-18T21:01:52.879 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-18T21:01:52.879 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-18T21:01:52.879 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-18T21:01:52.879 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T21:01:52.879 DEBUG:tasks.check_counter:Getting stats from i 2024-03-18T21:01:52.879 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-18T21:01:52.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:52 smithi073 ceph-mon[47384]: pgmap v816: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 584 KiB/s wr, 58 op/s 2024-03-18T21:01:53.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:52 smithi046 ceph-mon[43495]: pgmap v816: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 584 KiB/s wr, 58 op/s 2024-03-18T21:01:53.105 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T21:01:54.055 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-03-18T21:01:54.055 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T21:01:54.055 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 245335, 2024-03-18T21:01:54.055 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 249180, 2024-03-18T21:01:54.055 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 79762532, 2024-03-18T21:01:54.055 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 435183989, 2024-03-18T21:01:54.055 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 14, 2024-03-18T21:01:54.055 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-03-18T21:01:54.055 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 13.143354872, 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 5.356006940, 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 8.270712505, 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.995267423, 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 249176, 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 20.359173925, 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000081705 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 421640, 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.063778987, 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000151 2024-03-18T21:01:54.056 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 2077952, 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 923616 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 707246, 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 910285, 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 252799499, 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 873704490, 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 18, 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 34.173224155, 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 14.621684465, 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 25.459833358, 2024-03-18T21:01:54.057 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 1.179552246, 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 910285, 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 108.857854419, 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000119586 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1344488, 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.206248279, 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000153 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 112288 2024-03-18T21:01:54.058 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 158172, 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 168120, 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 59848620, 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 441906698, 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 54, 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 9.376521742, 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 3.492066074, 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 4.705949435, 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 1.752620321, 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:54.059 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 168119, 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 19.583476347, 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000116485 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 226388, 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.033911942, 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000149 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 5264, 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 2608 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-03-18T21:01:54.060 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 111205, 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 100.832723742, 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000906728 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 66520, 2024-03-18T21:01:54.061 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 7.010989389, 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000105396 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 265289, 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 246109, 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 246109, 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 988.025547778, 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004014585 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-03-18T21:01:54.062 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 19180, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 2, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 131, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 9, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 10, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 2679, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 1134, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 1268, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 277, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 1044, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 785, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-03-18T21:01:54.063 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 6, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 387245, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 227451, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 19172, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 69, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 2567, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 13, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 496, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 13, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 645, 2024-03-18T21:01:54.064 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 211, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 4, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 234, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 211, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 211, 2024-03-18T21:01:54.065 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 14, 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 56571, 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 158478, 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 1910, 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 104882, 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 33012, 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 5990, 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 177165 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 21, 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 101, 2024-03-18T21:01:54.066 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 74, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 112, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 113, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 20, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 35984, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 35964, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-03-18T21:01:54.067 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 14, 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 19, 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 176327, 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 114797, 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 114797, 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 61530, 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 498, 2024-03-18T21:01:54.068 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 288, 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 160, 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 160, 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 129, 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 1, 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 499132193, 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 1029916020, 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 61227, 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 533.990723384, 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.008721490 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2024-03-18T21:01:54.069 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 1408, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 37906, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 36498, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 72, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 322, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 250, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 2693, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 71184, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 68491, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 39637, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 39637, 2024-03-18T21:01:54.070 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 84664, 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 198964 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 388049, 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 4921, 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 265289, 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 5268, 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 17584, 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 35832, 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 97.746069454, 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002727898 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.071 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 7032, 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 145.564306593, 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.020700271 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.072 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 65162, 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 87.845273275, 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001348105 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.073 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.074 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.075 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000752084, 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000752084 2024-03-18T21:01:54.076 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 93546, 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 546.007681754, 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.005836782 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 7408, 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 28.463296286, 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.003842237 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.077 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 20, 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.126011272, 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.006300563 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.078 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 56, 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.774837351, 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.013836381 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.079 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1088, 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.066245501, 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000980005 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-03-18T21:01:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 35964, 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 80.431074208, 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002236432 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.081 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 1, 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 3, 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 2, 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 1, 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-03-18T21:01:54.082 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 23030, 2024-03-18T21:01:54.083 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 23030, 2024-03-18T21:01:54.083 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 1483, 2024-03-18T21:01:54.083 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-03-18T21:01:54.083 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.083 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-03-18T21:01:54.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-03-18T21:01:54.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-03-18T21:01:54.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T21:01:54.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-03-18T21:01:54.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T21:01:54.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-03-18T21:01:54.083 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T21:01:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T21:01:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T21:01:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T21:01:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T21:01:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-03-18T21:01:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T21:01:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T21:01:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-03-18T21:01:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-03-18T21:01:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-03-18T21:01:54.084 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-03-18T21:01:54.085 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 472512, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 808, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 176, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 2, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-03-18T21:01:54.086 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-03-18T21:01:54.087 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8256, 2024-03-18T21:01:54.087 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 188, 2024-03-18T21:01:54.087 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T21:01:54.087 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-03-18T21:01:54.087 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-03-18T21:01:54.087 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-03-18T21:01:54.087 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 5588266, 2024-03-18T21:01:54.087 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 47523, 2024-03-18T21:01:54.087 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-03-18T21:01:54.087 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-03-18T21:01:54.087 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-03-18T21:01:54.087 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 101929, 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 1033540408, 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 27, 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 101929, 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 101929, 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 996.018067030, 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.009771684 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-03-18T21:01:54.088 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 101929, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 102964 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 101929, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 1560, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 100369, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 18, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 139, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 1347, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 62574, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 831, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-03-18T21:01:54.089 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 9, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 36390, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 213, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 278, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-03-18T21:01:54.090 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-03-18T21:01:54.091 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 122, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 44, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 12, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 832, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 4, 2024-03-18T21:01:54.092 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 329 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 270228 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 6, 2024-03-18T21:01:54.093 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 6, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 35964, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 1110749, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 292407462, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 1110749, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-18T21:01:54.094 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 1110749, 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 292407462, 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:54.095 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 101929, 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 1037181876, 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 101929, 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1037181876, 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-03-18T21:01:54.096 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 101929, 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 101929, 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 101929, 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 101929, 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.097 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 35964, 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 3641500, 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 35964, 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 1347, 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 3641500, 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.098 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x5562003700a0": { 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 176327, 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 1025721716, 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 176327, 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-18T21:01:54.099 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 61227, 2024-03-18T21:01:54.100 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1025721716, 2024-03-18T21:01:54.100 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:54.100 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:54.100 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:54.100 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:54.100 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:54.100 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:54.100 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-03-18T21:01:54.469 INFO:tasks.check_counter:Daemon mds.i mds.exported=13 2024-03-18T21:01:54.469 INFO:tasks.check_counter:Daemon mds.i mds.imported=13 2024-03-18T21:01:54.469 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-03-18T21:01:54.469 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-18T21:01:54.469 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T21:01:54.469 DEBUG:tasks.check_counter:Getting stats from l 2024-03-18T21:01:54.470 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:49bdc51142759622a281d3e5327eebd07cbb8bb3 shell --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-18T21:01:54.694 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T21:01:54.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:54 smithi155 ceph-mon[48344]: pgmap v817: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 36 op/s 2024-03-18T21:01:54.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:54 smithi073 ceph-mon[47384]: pgmap v817: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 36 op/s 2024-03-18T21:01:55.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:54 smithi046 ceph-mon[43495]: pgmap v817: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 274 KiB/s wr, 36 op/s 2024-03-18T21:01:55.714 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-03-18T21:01:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T21:01:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 3080, 2024-03-18T21:01:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2982, 2024-03-18T21:01:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 270665986, 2024-03-18T21:01:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1036500, 2024-03-18T21:01:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 7, 2024-03-18T21:01:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-03-18T21:01:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.453029043, 2024-03-18T21:01:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.080426464, 2024-03-18T21:01:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.273437270, 2024-03-18T21:01:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.085445663, 2024-03-18T21:01:55.715 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2981, 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.216957197, 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000072780 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 2060816, 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 261488 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.716 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2124, 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2459, 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 369666821, 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 1398890, 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 9, 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 7, 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.327000893, 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.060821499, 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.196721149, 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.053059158, 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2456, 2024-03-18T21:01:55.717 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.212041605, 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000086336 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 738880 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1827, 2024-03-18T21:01:55.718 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1827, 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 310855802, 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 567402, 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 6, 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.311033497, 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.039920919, 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.212093568, 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.044935824, 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1826, 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.132429492, 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000072524 2024-03-18T21:01:55.719 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-03-18T21:01:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-03-18T21:01:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T21:01:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-03-18T21:01:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-03-18T21:01:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-03-18T21:01:55.720 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 10643, 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.295325757, 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000027748 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.721 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-03-18T21:01:55.722 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 34010, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 13858, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 20152, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 14212, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-03-18T21:01:55.723 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-03-18T21:01:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 1, 2024-03-18T21:01:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2024-03-18T21:01:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2024-03-18T21:01:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-03-18T21:01:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-03-18T21:01:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-03-18T21:01:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-03-18T21:01:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-03-18T21:01:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-03-18T21:01:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-03-18T21:01:55.724 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-03-18T21:01:55.725 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-03-18T21:01:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T21:01:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T21:01:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-03-18T21:01:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-03-18T21:01:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T21:01:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-03-18T21:01:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T21:01:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T21:01:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T21:01:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-03-18T21:01:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-03-18T21:01:55.726 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-03-18T21:01:55.727 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-03-18T21:01:55.728 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 164697, 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 129, 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-03-18T21:01:55.729 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 415256236, 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 951512246, 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 951486937, 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 164697 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 1492, 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 34120, 2024-03-18T21:01:55.730 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 32628, 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 59, 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 152, 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 93, 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 34010, 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 34406, 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 396, 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 88320, 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 207156 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-03-18T21:01:55.731 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.732 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.733 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.734 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.735 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.736 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.737 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.738 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.739 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.740 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 4, 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 2, 2024-03-18T21:01:55.743 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 4, 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 4321, 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-03-18T21:01:55.744 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T21:01:55.745 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-03-18T21:01:55.746 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 299185, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 1773, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-03-18T21:01:55.747 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 8256, 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 188, 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 27751246, 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 176329, 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-03-18T21:01:55.748 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 6437, 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 21, 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 3, 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 6437, 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6437, 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 17.835659751, 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002770803 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6437, 2024-03-18T21:01:55.749 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 6438 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 6437, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 6437, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 3515, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 2920, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-03-18T21:01:55.750 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 1, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-03-18T21:01:55.751 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-03-18T21:01:55.752 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 122, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 44, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 11, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 11, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2024-03-18T21:01:55.753 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T21:01:55.754 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 7027, 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 950523560, 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 7027, 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-18T21:01:55.755 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 7027, 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 950523560, 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:55.757 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:55.757 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:55.757 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 6437, 2024-03-18T21:01:55.757 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 947355615, 2024-03-18T21:01:55.757 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 6437, 2024-03-18T21:01:55.757 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 947355615, 2024-03-18T21:01:55.757 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:55.757 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.757 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.757 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 6437, 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 6437, 2024-03-18T21:01:55.758 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 6437, 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 6437, 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:55.759 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x559ecde08500": { 2024-03-18T21:01:55.760 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-03-18T21:01:55.761 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-03-18T21:01:55.762 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:55.762 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-03-18T21:01:55.762 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-03-18T21:01:56.077 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-18T21:01:56.077 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-18T21:01:56.077 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-03-18T21:01:56.077 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-18T21:01:56.077 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-18T21:01:56.077 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-18T21:01:56.089 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-18T21:01:56.089 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T21:01:56.089 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T21:01:56.130 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T21:01:56.131 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T21:01:56.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T21:01:56.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.384+0000 7f3c8bd36700 1 -- 172.21.15.46:0/1209474579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84100e80 msgr2=0x7f3c84101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:56.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.384+0000 7f3c8bd36700 1 --2- 172.21.15.46:0/1209474579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84100e80 0x7f3c84101260 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00b0a0 tx=0x7f3c7c00b3b0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.384+0000 7f3c8bd36700 1 -- 172.21.15.46:0/1209474579 shutdown_connections 2024-03-18T21:01:56.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.384+0000 7f3c8bd36700 1 --2- 172.21.15.46:0/1209474579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84100e80 0x7f3c84101260 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.384+0000 7f3c8bd36700 1 --2- 172.21.15.46:0/1209474579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c84105fa0 0x7f3c8410db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.384+0000 7f3c8bd36700 1 --2- 172.21.15.46:0/1209474579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c84101830 0x7f3c84105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.384+0000 7f3c8bd36700 1 -- 172.21.15.46:0/1209474579 >> 172.21.15.46:0/1209474579 conn(0x7f3c840fc720 msgr2=0x7f3c840feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:56.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.384+0000 7f3c8bd36700 1 -- 172.21.15.46:0/1209474579 shutdown_connections 2024-03-18T21:01:56.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.384+0000 7f3c8bd36700 1 -- 172.21.15.46:0/1209474579 wait complete. 2024-03-18T21:01:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.385+0000 7f3c8bd36700 1 Processor -- start 2024-03-18T21:01:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.385+0000 7f3c8bd36700 1 -- start start 2024-03-18T21:01:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c8bd36700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84100e80 0x7f3c840730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c8bd36700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c84105fa0 0x7f3c840735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c8bd36700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c84073b30 0x7f3c841a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c8bd36700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c841a5b30 con 0x7f3c84100e80 2024-03-18T21:01:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c8bd36700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c841a5c70 con 0x7f3c84105fa0 2024-03-18T21:01:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c8bd36700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c841a5e30 con 0x7f3c84073b30 2024-03-18T21:01:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c89ad2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84100e80 0x7f3c840730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c89ad2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84100e80 0x7f3c840730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52378/0 (socket says 172.21.15.46:52378) 2024-03-18T21:01:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c89ad2700 1 -- 172.21.15.46:0/2949617461 learned_addr learned my addr 172.21.15.46:0/2949617461 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c8a2d3700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c84073b30 0x7f3c841a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c892d1700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c84105fa0 0x7f3c840735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c89ad2700 1 -- 172.21.15.46:0/2949617461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c84073b30 msgr2=0x7f3c841a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c89ad2700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c84073b30 0x7f3c841a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c89ad2700 1 -- 172.21.15.46:0/2949617461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c84105fa0 msgr2=0x7f3c840735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c89ad2700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c84105fa0 0x7f3c840735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c89ad2700 1 -- 172.21.15.46:0/2949617461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c009d00 con 0x7f3c84100e80 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c8a2d3700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c84073b30 0x7f3c841a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c892d1700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c84105fa0 0x7f3c840735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.386+0000 7f3c89ad2700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84100e80 0x7f3c840730b0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00d010 tx=0x7f3c7c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.387+0000 7f3c82ffd700 1 -- 172.21.15.46:0/2949617461 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c00e040 con 0x7f3c84100e80 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.387+0000 7f3c82ffd700 1 -- 172.21.15.46:0/2949617461 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c7c004c30 con 0x7f3c84100e80 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.387+0000 7f3c8bd36700 1 -- 172.21.15.46:0/2949617461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c841a60b0 con 0x7f3c84100e80 2024-03-18T21:01:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.387+0000 7f3c82ffd700 1 -- 172.21.15.46:0/2949617461 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c01b690 con 0x7f3c84100e80 2024-03-18T21:01:56.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.387+0000 7f3c8bd36700 1 -- 172.21.15.46:0/2949617461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c841a6530 con 0x7f3c84100e80 2024-03-18T21:01:56.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.388+0000 7f3c8bd36700 1 -- 172.21.15.46:0/2949617461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c84109210 con 0x7f3c84100e80 2024-03-18T21:01:56.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.389+0000 7f3c82ffd700 1 -- 172.21.15.46:0/2949617461 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3c7c01d480 con 0x7f3c84100e80 2024-03-18T21:01:56.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.389+0000 7f3c82ffd700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3c6c0a1290 0x7f3c6c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:56.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.390+0000 7f3c892d1700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3c6c0a1290 0x7f3c6c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:56.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.390+0000 7f3c82ffd700 1 -- 172.21.15.46:0/2949617461 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f3c7c0cea50 con 0x7f3c84100e80 2024-03-18T21:01:56.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.390+0000 7f3c892d1700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3c6c0a1290 0x7f3c6c0a3750 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3c84074790 tx=0x7f3c78009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:56.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.393+0000 7f3c82ffd700 1 -- 172.21.15.46:0/2949617461 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3c7c097f10 con 0x7f3c84100e80 2024-03-18T21:01:56.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:56 smithi046 ceph-mon[43495]: pgmap v818: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 255 KiB/s wr, 33 op/s 2024-03-18T21:01:56.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.630+0000 7f3c8bd36700 1 -- 172.21.15.46:0/2949617461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c840671b0 con 0x7f3c84100e80 2024-03-18T21:01:56.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.631+0000 7f3c82ffd700 1 -- 172.21.15.46:0/2949617461 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+23036 (secure 0 0 0) 0x7f3c7c025e90 con 0x7f3c84100e80 2024-03-18T21:01:56.634 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T21:01:56.634 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":122,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T21:01:20.339788+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/3400106473":"2024-03-19T20:36:06.973937+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2587067898":"2024-03-19T20:36:06.973937+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/3248305722":"2024-03-19T20:36:06.973937+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/604593037":"2024-03-19T20:36:06.973937+0000","172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1017695887":"2024-03-19T20:36:06.973937+0000","172.21.15.46:0/2204627135":"2024-03-19T20:36:06.973937+0000","172.21.15.46:6801/3400106473":"2024-03-19T20:36:06.973937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T21:01:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.634+0000 7f3c8bd36700 1 -- 172.21.15.46:0/2949617461 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3c6c0a1290 msgr2=0x7f3c6c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:56.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.634+0000 7f3c8bd36700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3c6c0a1290 0x7f3c6c0a3750 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3c84074790 tx=0x7f3c78009260 comp rx=0 tx=0).stop 2024-03-18T21:01:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.634+0000 7f3c8bd36700 1 -- 172.21.15.46:0/2949617461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84100e80 msgr2=0x7f3c840730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.634+0000 7f3c8bd36700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84100e80 0x7f3c840730b0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00d010 tx=0x7f3c7c00bf10 comp rx=0 tx=0).stop 2024-03-18T21:01:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.634+0000 7f3c8bd36700 1 -- 172.21.15.46:0/2949617461 shutdown_connections 2024-03-18T21:01:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.634+0000 7f3c8bd36700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f3c6c0a1290 0x7f3c6c0a3750 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.634+0000 7f3c8bd36700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c84100e80 0x7f3c840730b0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.634+0000 7f3c8bd36700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c84073b30 0x7f3c841a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.634+0000 7f3c8bd36700 1 --2- 172.21.15.46:0/2949617461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c84105fa0 0x7f3c840735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.634+0000 7f3c8bd36700 1 -- 172.21.15.46:0/2949617461 >> 172.21.15.46:0/2949617461 conn(0x7f3c840fc720 msgr2=0x7f3c840fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.634+0000 7f3c8bd36700 1 -- 172.21.15.46:0/2949617461 shutdown_connections 2024-03-18T21:01:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.634+0000 7f3c8bd36700 1 -- 172.21.15.46:0/2949617461 wait complete. 2024-03-18T21:01:56.651 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T20:31:27.488318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T20:34:37.213560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T20:34:38.162692+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 121, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T21:01:56.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T21:01:56.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:56 smithi073 ceph-mon[47384]: pgmap v818: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 255 KiB/s wr, 33 op/s 2024-03-18T21:01:56.789 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:56 smithi155 ceph-mon[48344]: pgmap v818: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 255 KiB/s wr, 33 op/s 2024-03-18T21:01:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.790+0000 7f969a300700 1 -- 172.21.15.46:0/1526651789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96940ff4b0 msgr2=0x7f96940ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.790+0000 7f969a300700 1 --2- 172.21.15.46:0/1526651789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96940ff4b0 0x7f96940ff890 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f9684009990 tx=0x7f9684009ca0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.790+0000 7f969a300700 1 -- 172.21.15.46:0/1526651789 shutdown_connections 2024-03-18T21:01:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.790+0000 7f969a300700 1 --2- 172.21.15.46:0/1526651789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96940ff4b0 0x7f96940ff890 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.790+0000 7f969a300700 1 --2- 172.21.15.46:0/1526651789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96941060c0 0x7f969410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.790+0000 7f969a300700 1 --2- 172.21.15.46:0/1526651789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96940ffe60 0x7f9694105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.790+0000 7f969a300700 1 -- 172.21.15.46:0/1526651789 >> 172.21.15.46:0/1526651789 conn(0x7f96940761e0 msgr2=0x7f96940fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.790+0000 7f969a300700 1 -- 172.21.15.46:0/1526651789 shutdown_connections 2024-03-18T21:01:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.790+0000 7f969a300700 1 -- 172.21.15.46:0/1526651789 wait complete. 2024-03-18T21:01:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.791+0000 7f969a300700 1 Processor -- start 2024-03-18T21:01:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.791+0000 7f969a300700 1 -- start start 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.791+0000 7f969a300700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96940ff4b0 0x7f9694197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.791+0000 7f969a300700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96941060c0 0x7f96941976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.791+0000 7f969a300700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f969419ba50 0x7f969419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.791+0000 7f969a300700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969419c4d0 con 0x7f969419ba50 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.791+0000 7f969a300700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969419c640 con 0x7f96941060c0 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.791+0000 7f969a300700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969419c800 con 0x7f96940ff4b0 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.791+0000 7f969889d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f969419ba50 0x7f969419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.791+0000 7f969889d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f969419ba50 0x7f969419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52398/0 (socket says 172.21.15.46:52398) 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f969889d700 1 -- 172.21.15.46:0/2852919951 learned_addr learned my addr 172.21.15.46:0/2852919951 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f96937fe700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96941060c0 0x7f96941976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f9693fff700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96940ff4b0 0x7f9694197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f96937fe700 1 -- 172.21.15.46:0/2852919951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96940ff4b0 msgr2=0x7f9694197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:56.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f96937fe700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96940ff4b0 0x7f9694197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f96937fe700 1 -- 172.21.15.46:0/2852919951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f969419ba50 msgr2=0x7f969419bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f96937fe700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f969419ba50 0x7f969419bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f96937fe700 1 -- 172.21.15.46:0/2852919951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96840095f0 con 0x7f96941060c0 2024-03-18T21:01:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f969889d700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f969419ba50 0x7f969419bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T21:01:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f96937fe700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96941060c0 0x7f96941976b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f96940734a0 tx=0x7f968800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f96917fa700 1 -- 172.21.15.46:0/2852919951 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968800f890 con 0x7f96941060c0 2024-03-18T21:01:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f96917fa700 1 -- 172.21.15.46:0/2852919951 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9688010460 con 0x7f96941060c0 2024-03-18T21:01:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f969a300700 1 -- 172.21.15.46:0/2852919951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96941a3d60 con 0x7f96941060c0 2024-03-18T21:01:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f96917fa700 1 -- 172.21.15.46:0/2852919951 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968800e610 con 0x7f96941060c0 2024-03-18T21:01:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.792+0000 7f969a300700 1 -- 172.21.15.46:0/2852919951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96941a41e0 con 0x7f96941060c0 2024-03-18T21:01:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.793+0000 7f969a300700 1 -- 172.21.15.46:0/2852919951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9694109320 con 0x7f96941060c0 2024-03-18T21:01:56.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.795+0000 7f96917fa700 1 -- 172.21.15.46:0/2852919951 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9688010840 con 0x7f96941060c0 2024-03-18T21:01:56.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.795+0000 7f96917fa700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f967c0a1290 0x7f967c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:56.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.796+0000 7f9693fff700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f967c0a1290 0x7f967c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:56.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.796+0000 7f96917fa700 1 -- 172.21.15.46:0/2852919951 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f96880ccdf0 con 0x7f96941060c0 2024-03-18T21:01:56.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.796+0000 7f9693fff700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f967c0a1290 0x7f967c0a3750 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9684006010 tx=0x7f9684005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:56.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:56.799+0000 7f96917fa700 1 -- 172.21.15.46:0/2852919951 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f96880961c0 con 0x7f96941060c0 2024-03-18T21:01:57.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.017+0000 7f969a300700 1 -- 172.21.15.46:0/2852919951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9694198470 con 0x7f96941060c0 2024-03-18T21:01:57.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.018+0000 7f96917fa700 1 -- 172.21.15.46:0/2852919951 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96880200a0 con 0x7f96941060c0 2024-03-18T21:01:57.020 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-18T21:01:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.020+0000 7f969a300700 1 -- 172.21.15.46:0/2852919951 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f967c0a1290 msgr2=0x7f967c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.020+0000 7f969a300700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f967c0a1290 0x7f967c0a3750 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9684006010 tx=0x7f9684005960 comp rx=0 tx=0).stop 2024-03-18T21:01:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.020+0000 7f969a300700 1 -- 172.21.15.46:0/2852919951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96941060c0 msgr2=0x7f96941976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.020+0000 7f969a300700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96941060c0 0x7f96941976b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f96940734a0 tx=0x7f968800dc30 comp rx=0 tx=0).stop 2024-03-18T21:01:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.021+0000 7f969a300700 1 -- 172.21.15.46:0/2852919951 shutdown_connections 2024-03-18T21:01:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.021+0000 7f969a300700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f967c0a1290 0x7f967c0a3750 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.021+0000 7f969a300700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f969419ba50 0x7f969419bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.021+0000 7f969a300700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96940ff4b0 0x7f9694197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.021+0000 7f969a300700 1 --2- 172.21.15.46:0/2852919951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96941060c0 0x7f96941976b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.021+0000 7f969a300700 1 -- 172.21.15.46:0/2852919951 >> 172.21.15.46:0/2852919951 conn(0x7f96940761e0 msgr2=0x7f9694104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.021+0000 7f969a300700 1 -- 172.21.15.46:0/2852919951 shutdown_connections 2024-03-18T21:01:57.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.021+0000 7f969a300700 1 -- 172.21.15.46:0/2852919951 wait complete. 2024-03-18T21:01:57.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T21:01:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.176+0000 7f022ebc6700 1 -- 172.21.15.46:0/21461213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0228105ce0 msgr2=0x7f02281060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.176+0000 7f022ebc6700 1 --2- 172.21.15.46:0/21461213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0228105ce0 0x7f02281060c0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f021c009990 tx=0x7f021c009ca0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.176+0000 7f022ebc6700 1 -- 172.21.15.46:0/21461213 shutdown_connections 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.176+0000 7f022ebc6700 1 --2- 172.21.15.46:0/21461213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0228105ce0 0x7f02281060c0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.176+0000 7f022ebc6700 1 --2- 172.21.15.46:0/21461213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02280687f0 0x7f022810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.176+0000 7f022ebc6700 1 --2- 172.21.15.46:0/21461213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0228067e30 0x7f02280682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.176+0000 7f022ebc6700 1 -- 172.21.15.46:0/21461213 >> 172.21.15.46:0/21461213 conn(0x7f0228076d40 msgr2=0x7f0228077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.176+0000 7f022ebc6700 1 -- 172.21.15.46:0/21461213 shutdown_connections 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.176+0000 7f022ebc6700 1 -- 172.21.15.46:0/21461213 wait complete. 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f022ebc6700 1 Processor -- start 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f022ebc6700 1 -- start start 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f022ebc6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0228067e30 0x7f0228197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f022ebc6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02280687f0 0x7f02281977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f022ebc6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105ce0 0x7f022819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f022ebc6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022819c190 con 0x7f02280687f0 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f022ebc6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022819c300 con 0x7f0228067e30 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f022ebc6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022819c4c0 con 0x7f0228105ce0 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f0227fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02280687f0 0x7f02281977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f0227fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02280687f0 0x7f02281977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52418/0 (socket says 172.21.15.46:52418) 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f0227fff700 1 -- 172.21.15.46:0/2353619877 learned_addr learned my addr 172.21.15.46:0/2353619877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f022c962700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0228067e30 0x7f0228197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f022d163700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105ce0 0x7f022819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f0227fff700 1 -- 172.21.15.46:0/2353619877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105ce0 msgr2=0x7f022819bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f0227fff700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105ce0 0x7f022819bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f0227fff700 1 -- 172.21.15.46:0/2353619877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0228067e30 msgr2=0x7f0228197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f0227fff700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0228067e30 0x7f0228197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f0227fff700 1 -- 172.21.15.46:0/2353619877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0220009c40 con 0x7f02280687f0 2024-03-18T21:01:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.177+0000 7f022d163700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105ce0 0x7f022819bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T21:01:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.178+0000 7f0227fff700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02280687f0 0x7f02281977b0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f021400e9a0 tx=0x7f021400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.178+0000 7f0225ffb700 1 -- 172.21.15.46:0/2353619877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214009790 con 0x7f02280687f0 2024-03-18T21:01:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.178+0000 7f0225ffb700 1 -- 172.21.15.46:0/2353619877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f021400cbd0 con 0x7f02280687f0 2024-03-18T21:01:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.178+0000 7f022ebc6700 1 -- 172.21.15.46:0/2353619877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021c0095f0 con 0x7f02280687f0 2024-03-18T21:01:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.178+0000 7f0225ffb700 1 -- 172.21.15.46:0/2353619877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214018680 con 0x7f02280687f0 2024-03-18T21:01:57.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.178+0000 7f022ebc6700 1 -- 172.21.15.46:0/2353619877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022819c990 con 0x7f02280687f0 2024-03-18T21:01:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.179+0000 7f022ebc6700 1 -- 172.21.15.46:0/2353619877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0228109280 con 0x7f02280687f0 2024-03-18T21:01:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.180+0000 7f0225ffb700 1 -- 172.21.15.46:0/2353619877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f021400cd40 con 0x7f02280687f0 2024-03-18T21:01:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.181+0000 7f0225ffb700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f02180a1310 0x7f02180a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.181+0000 7f022c962700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f02180a1310 0x7f02180a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.181+0000 7f0225ffb700 1 -- 172.21.15.46:0/2353619877 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f0214014070 con 0x7f02280687f0 2024-03-18T21:01:57.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.182+0000 7f022c962700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f02180a1310 0x7f02180a37d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f021c0098e0 tx=0x7f021c005960 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:57.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.185+0000 7f0225ffb700 1 -- 172.21.15.46:0/2353619877 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0214096650 con 0x7f02280687f0 2024-03-18T21:01:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.403+0000 7f022ebc6700 1 -- 172.21.15.46:0/2353619877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0228198570 con 0x7f02280687f0 2024-03-18T21:01:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.404+0000 7f0225ffb700 1 -- 172.21.15.46:0/2353619877 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0214099c70 con 0x7f02280687f0 2024-03-18T21:01:57.406 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-18T21:01:57.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.406+0000 7f022ebc6700 1 -- 172.21.15.46:0/2353619877 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f02180a1310 msgr2=0x7f02180a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.406+0000 7f022ebc6700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f02180a1310 0x7f02180a37d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f021c0098e0 tx=0x7f021c005960 comp rx=0 tx=0).stop 2024-03-18T21:01:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.406+0000 7f022ebc6700 1 -- 172.21.15.46:0/2353619877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02280687f0 msgr2=0x7f02281977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.406+0000 7f022ebc6700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02280687f0 0x7f02281977b0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f021400e9a0 tx=0x7f021400ecb0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.406+0000 7f022ebc6700 1 -- 172.21.15.46:0/2353619877 shutdown_connections 2024-03-18T21:01:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.406+0000 7f022ebc6700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f02180a1310 0x7f02180a37d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.406+0000 7f022ebc6700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0228105ce0 0x7f022819bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.406+0000 7f022ebc6700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0228067e30 0x7f0228197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.406+0000 7f022ebc6700 1 --2- 172.21.15.46:0/2353619877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02280687f0 0x7f02281977b0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.406+0000 7f022ebc6700 1 -- 172.21.15.46:0/2353619877 >> 172.21.15.46:0/2353619877 conn(0x7f0228076d40 msgr2=0x7f02280fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.406+0000 7f022ebc6700 1 -- 172.21.15.46:0/2353619877 shutdown_connections 2024-03-18T21:01:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.406+0000 7f022ebc6700 1 -- 172.21.15.46:0/2353619877 wait complete. 2024-03-18T21:01:57.422 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T21:01:57.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.561+0000 7fa624820700 1 -- 172.21.15.46:0/2376503502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6200ffe70 msgr2=0x7fa620100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.561+0000 7fa624820700 1 --2- 172.21.15.46:0/2376503502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6200ffe70 0x7fa620100250 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fa610009990 tx=0x7fa610009ca0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.561+0000 7fa624820700 1 -- 172.21.15.46:0/2376503502 shutdown_connections 2024-03-18T21:01:57.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.561+0000 7fa624820700 1 --2- 172.21.15.46:0/2376503502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6200ffe70 0x7fa620100250 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.561+0000 7fa624820700 1 --2- 172.21.15.46:0/2376503502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa620104f90 0x7fa62010cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.561+0000 7fa624820700 1 --2- 172.21.15.46:0/2376503502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa620100820 0x7fa620104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.561+0000 7fa624820700 1 -- 172.21.15.46:0/2376503502 >> 172.21.15.46:0/2376503502 conn(0x7fa620076d80 msgr2=0x7fa620077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:57.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.561+0000 7fa624820700 1 -- 172.21.15.46:0/2376503502 shutdown_connections 2024-03-18T21:01:57.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.561+0000 7fa624820700 1 -- 172.21.15.46:0/2376503502 wait complete. 2024-03-18T21:01:57.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.562+0000 7fa624820700 1 Processor -- start 2024-03-18T21:01:57.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.562+0000 7fa624820700 1 -- start start 2024-03-18T21:01:57.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.562+0000 7fa624820700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6200ffe70 0x7fa6201980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:57.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa624820700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa620104f90 0x7fa6201985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:57.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa624820700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa620198b20 0x7fa6201a3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa624820700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6201a3800 con 0x7fa6200ffe70 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa624820700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6201a3970 con 0x7fa620198b20 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa624820700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6201a3b30 con 0x7fa620104f90 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa61e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6200ffe70 0x7fa6201980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa61e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6200ffe70 0x7fa6201980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52448/0 (socket says 172.21.15.46:52448) 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa61e59c700 1 -- 172.21.15.46:0/1532285644 learned_addr learned my addr 172.21.15.46:0/1532285644 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa61dd9b700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa620104f90 0x7fa6201985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa61ed9d700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa620198b20 0x7fa6201a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa61e59c700 1 -- 172.21.15.46:0/1532285644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa620104f90 msgr2=0x7fa6201985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa61e59c700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa620104f90 0x7fa6201985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa61e59c700 1 -- 172.21.15.46:0/1532285644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa620198b20 msgr2=0x7fa6201a3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa61e59c700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa620198b20 0x7fa6201a3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa61e59c700 1 -- 172.21.15.46:0/1532285644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100095f0 con 0x7fa6200ffe70 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa61ed9d700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa620198b20 0x7fa6201a3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T21:01:57.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa61e59c700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6200ffe70 0x7fa6201980a0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa610004760 tx=0x7fa610009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:57.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa60f7fe700 1 -- 172.21.15.46:0/1532285644 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61001d070 con 0x7fa6200ffe70 2024-03-18T21:01:57.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa624820700 1 -- 172.21.15.46:0/1532285644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6201a3db0 con 0x7fa6200ffe70 2024-03-18T21:01:57.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa60f7fe700 1 -- 172.21.15.46:0/1532285644 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa61000fc00 con 0x7fa6200ffe70 2024-03-18T21:01:57.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa60f7fe700 1 -- 172.21.15.46:0/1532285644 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610017650 con 0x7fa6200ffe70 2024-03-18T21:01:57.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.563+0000 7fa624820700 1 -- 172.21.15.46:0/1532285644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6201a41d0 con 0x7fa6200ffe70 2024-03-18T21:01:57.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.565+0000 7fa624820700 1 -- 172.21.15.46:0/1532285644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa620108200 con 0x7fa6200ffe70 2024-03-18T21:01:57.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.566+0000 7fa60f7fe700 1 -- 172.21.15.46:0/1532285644 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa610004830 con 0x7fa6200ffe70 2024-03-18T21:01:57.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.567+0000 7fa60f7fe700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa6080a1290 0x7fa6080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:57.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.567+0000 7fa61dd9b700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa6080a1290 0x7fa6080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:57.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.567+0000 7fa60f7fe700 1 -- 172.21.15.46:0/1532285644 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fa6100ce610 con 0x7fa6200ffe70 2024-03-18T21:01:57.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.567+0000 7fa61dd9b700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa6080a1290 0x7fa6080a3750 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa614005fd0 tx=0x7fa614005f60 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:57.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.571+0000 7fa60f7fe700 1 -- 172.21.15.46:0/1532285644 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa610097bf0 con 0x7fa6200ffe70 2024-03-18T21:01:57.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:57 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2949617461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T21:01:57.719 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:57 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2852919951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T21:01:57.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.783+0000 7fa624820700 1 -- 172.21.15.46:0/1532285644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa620069530 con 0x7fa6200ffe70 2024-03-18T21:01:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.784+0000 7fa60f7fe700 1 -- 172.21.15.46:0/1532285644 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa610027020 con 0x7fa6200ffe70 2024-03-18T21:01:57.786 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-18T21:01:57.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.786+0000 7fa624820700 1 -- 172.21.15.46:0/1532285644 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa6080a1290 msgr2=0x7fa6080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.786+0000 7fa624820700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa6080a1290 0x7fa6080a3750 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa614005fd0 tx=0x7fa614005f60 comp rx=0 tx=0).stop 2024-03-18T21:01:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.786+0000 7fa624820700 1 -- 172.21.15.46:0/1532285644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6200ffe70 msgr2=0x7fa6201980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.786+0000 7fa624820700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6200ffe70 0x7fa6201980a0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fa610004760 tx=0x7fa610009570 comp rx=0 tx=0).stop 2024-03-18T21:01:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.786+0000 7fa624820700 1 -- 172.21.15.46:0/1532285644 shutdown_connections 2024-03-18T21:01:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.786+0000 7fa624820700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fa6080a1290 0x7fa6080a3750 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.786+0000 7fa624820700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6200ffe70 0x7fa6201980a0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.786+0000 7fa624820700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa620198b20 0x7fa6201a3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.786+0000 7fa624820700 1 --2- 172.21.15.46:0/1532285644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa620104f90 0x7fa6201985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.786+0000 7fa624820700 1 -- 172.21.15.46:0/1532285644 >> 172.21.15.46:0/1532285644 conn(0x7fa620076d80 msgr2=0x7fa620106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.786+0000 7fa624820700 1 -- 172.21.15.46:0/1532285644 shutdown_connections 2024-03-18T21:01:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.786+0000 7fa624820700 1 -- 172.21.15.46:0/1532285644 wait complete. 2024-03-18T21:01:57.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:57 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2949617461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T21:01:57.790 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:57 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2852919951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T21:01:57.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:57 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2949617461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T21:01:57.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:57 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2852919951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T21:01:57.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-18T21:01:57.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.939+0000 7f23a58d5700 1 -- 172.21.15.46:0/2714246125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a010d5b0 msgr2=0x7f23a010f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.939+0000 7f23a58d5700 1 --2- 172.21.15.46:0/2714246125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a010d5b0 0x7f23a010f9e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f239800b0a0 tx=0x7f239800b3b0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.939+0000 7f23a58d5700 1 -- 172.21.15.46:0/2714246125 shutdown_connections 2024-03-18T21:01:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.939+0000 7f23a58d5700 1 --2- 172.21.15.46:0/2714246125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a010d5b0 0x7f23a010f9e0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.939+0000 7f23a58d5700 1 --2- 172.21.15.46:0/2714246125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23a0073180 0x7f23a0073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.939+0000 7f23a58d5700 1 --2- 172.21.15.46:0/2714246125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23a0073aa0 0x7f23a010d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.939+0000 7f23a58d5700 1 -- 172.21.15.46:0/2714246125 >> 172.21.15.46:0/2714246125 conn(0x7f23a00fc7e0 msgr2=0x7f23a00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.940+0000 7f23a58d5700 1 -- 172.21.15.46:0/2714246125 shutdown_connections 2024-03-18T21:01:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.940+0000 7f23a58d5700 1 -- 172.21.15.46:0/2714246125 wait complete. 2024-03-18T21:01:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.940+0000 7f23a58d5700 1 Processor -- start 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.940+0000 7f23a58d5700 1 -- start start 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f23a58d5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a0073180 0x7f23a0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f23a58d5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23a0073aa0 0x7f23a0199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f23a58d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23a010d5b0 0x7f23a019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f23a58d5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a019e460 con 0x7f23a0073180 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f23a58d5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a019e5d0 con 0x7f23a0073aa0 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f23a58d5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a019e790 con 0x7f23a010d5b0 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f239effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a0073180 0x7f23a0199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f239effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a0073180 0x7f23a0199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52460/0 (socket says 172.21.15.46:52460) 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f239effd700 1 -- 172.21.15.46:0/994640561 learned_addr learned my addr 172.21.15.46:0/994640561 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f23967fc700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23a0073aa0 0x7f23a0199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f239f7fe700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23a010d5b0 0x7f23a019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f239effd700 1 -- 172.21.15.46:0/994640561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23a010d5b0 msgr2=0x7f23a019de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f239effd700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23a010d5b0 0x7f23a019de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f239effd700 1 -- 172.21.15.46:0/994640561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23a0073aa0 msgr2=0x7f23a0199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f239effd700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23a0073aa0 0x7f23a0199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f239effd700 1 -- 172.21.15.46:0/994640561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2398009d00 con 0x7f23a0073180 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f23967fc700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23a0073aa0 0x7f23a0199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f239f7fe700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23a010d5b0 0x7f23a019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.941+0000 7f239effd700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a0073180 0x7f23a0199540 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f239000d950 tx=0x7f239000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.942+0000 7f23a48d3700 1 -- 172.21.15.46:0/994640561 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390009940 con 0x7f23a0073180 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.942+0000 7f23a58d5700 1 -- 172.21.15.46:0/994640561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a019ea70 con 0x7f23a0073180 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.942+0000 7f23a48d3700 1 -- 172.21.15.46:0/994640561 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2390010460 con 0x7f23a0073180 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.942+0000 7f23a48d3700 1 -- 172.21.15.46:0/994640561 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000f620 con 0x7f23a0073180 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.942+0000 7f23a58d5700 1 -- 172.21.15.46:0/994640561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a01a62b0 con 0x7f23a0073180 2024-03-18T21:01:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.943+0000 7f23a58d5700 1 -- 172.21.15.46:0/994640561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a01087c0 con 0x7f23a0073180 2024-03-18T21:01:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.944+0000 7f23a48d3700 1 -- 172.21.15.46:0/994640561 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f23900108d0 con 0x7f23a0073180 2024-03-18T21:01:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.944+0000 7f23a48d3700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f238c0a1290 0x7f238c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.945+0000 7f23967fc700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f238c0a1290 0x7f238c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:57.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.945+0000 7f23a48d3700 1 -- 172.21.15.46:0/994640561 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f23900ccb50 con 0x7f23a0073180 2024-03-18T21:01:57.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.945+0000 7f23967fc700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f238c0a1290 0x7f238c0a3750 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f23a019aa90 tx=0x7f23880073d0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:57.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:57.948+0000 7f23a48d3700 1 -- 172.21.15.46:0/994640561 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2390095f80 con 0x7f23a0073180 2024-03-18T21:01:58.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.197+0000 7f23a58d5700 1 -- 172.21.15.46:0/994640561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f23a0067f70 con 0x7f23a0073180 2024-03-18T21:01:58.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.198+0000 7f23a48d3700 1 -- 172.21.15.46:0/994640561 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v122) v1 ==== 81+0+1001 (secure 0 0 0) 0x7f2390020400 con 0x7f23a0073180 2024-03-18T21:01:58.200 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/1385957034 2024-03-19T20:29:14.346231+0000 2024-03-18T21:01:58.200 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/3400106473 2024-03-19T20:36:06.973937+0000 2024-03-18T21:01:58.200 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/1385957034 2024-03-19T20:29:14.346231+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2440783396 2024-03-19T20:29:14.346231+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/699745793 2024-03-19T20:29:14.346231+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2587067898 2024-03-19T20:36:06.973937+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/606003730 2024-03-19T20:29:14.346231+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/2753187643 2024-03-19T20:29:36.953241+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3248305722 2024-03-19T20:36:06.973937+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/2753187643 2024-03-19T20:29:36.953241+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1997677402 2024-03-19T20:29:36.953241+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3646752058 2024-03-19T20:29:36.953241+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/604593037 2024-03-19T20:36:06.973937+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2471573219 2024-03-19T20:29:36.953241+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1017695887 2024-03-19T20:36:06.973937+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2204627135 2024-03-19T20:36:06.973937+0000 2024-03-18T21:01:58.201 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/3400106473 2024-03-19T20:36:06.973937+0000 2024-03-18T21:01:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.200+0000 7f23a58d5700 1 -- 172.21.15.46:0/994640561 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f238c0a1290 msgr2=0x7f238c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.200+0000 7f23a58d5700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f238c0a1290 0x7f238c0a3750 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f23a019aa90 tx=0x7f23880073d0 comp rx=0 tx=0).stop 2024-03-18T21:01:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.200+0000 7f23a58d5700 1 -- 172.21.15.46:0/994640561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a0073180 msgr2=0x7f23a0199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.200+0000 7f23a58d5700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a0073180 0x7f23a0199540 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f239000d950 tx=0x7f239000dc60 comp rx=0 tx=0).stop 2024-03-18T21:01:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.200+0000 7f23a58d5700 1 -- 172.21.15.46:0/994640561 shutdown_connections 2024-03-18T21:01:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.200+0000 7f23a58d5700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f238c0a1290 0x7f238c0a3750 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.200+0000 7f23a58d5700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a0073180 0x7f23a0199540 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.200+0000 7f23a58d5700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23a010d5b0 0x7f23a019de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.200+0000 7f23a58d5700 1 --2- 172.21.15.46:0/994640561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23a0073aa0 0x7f23a0199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.200+0000 7f23a58d5700 1 -- 172.21.15.46:0/994640561 >> 172.21.15.46:0/994640561 conn(0x7f23a00fc7e0 msgr2=0x7f23a00fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.200+0000 7f23a58d5700 1 -- 172.21.15.46:0/994640561 shutdown_connections 2024-03-18T21:01:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:58.200+0000 7f23a58d5700 1 -- 172.21.15.46:0/994640561 wait complete. 2024-03-18T21:01:58.204 INFO:teuthology.orchestra.run.smithi046.stderr:listed 17 entries 2024-03-18T21:01:58.217 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-18T21:01:58.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T21:01:58.217 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-18T21:01:58.475 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-18T21:01:58.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T21:01:58.475 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-18T21:01:58.476 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:58 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2353619877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T21:01:58.476 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:58 smithi046 ceph-mon[43495]: pgmap v819: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-03-18T21:01:58.476 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:58 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1532285644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T21:01:58.476 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:01:58 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/994640561' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T21:01:58.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T21:01:58.661 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-18T21:01:58.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:58 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2353619877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T21:01:58.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:58 smithi155 ceph-mon[48344]: pgmap v819: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-03-18T21:01:58.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:58 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1532285644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T21:01:58.793 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:01:58 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/994640561' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T21:01:58.881 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-18T21:01:58.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T21:01:58.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T21:01:58.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:58 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2353619877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T21:01:58.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:58 smithi073 ceph-mon[47384]: pgmap v819: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-03-18T21:01:58.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:58 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1532285644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T21:01:58.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:01:58 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/994640561' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T21:01:59.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T21:01:59.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-03-18T21:01:59.438 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-18T21:01:59.439 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-18T21:01:59.439 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T21:01:59.492 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16247276k,nr_inodes=4061819,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1781 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/cb6724668267beb81f0f01585ea921ad54f12ec704bf0f45fbff13a30c808824/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZDPMJNZ7USVGBRIGEIXPJD7OCL:/var/lib/containers/storage/overlay/l/GRGK4CX7UQVEUSZK5HFTC2QHV6,upperdir=/var/lib/containers/storage/overlay/cb6724668267beb81f0f01585ea921ad54f12ec704bf0f45fbff13a30c808824/diff,workdir=/var/lib/containers/storage/overlay/cb6724668267beb81f0f01585ea921ad54f12ec704bf0f45fbff13a30c808824/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2611cea0f6d7fdd9f1675a63cdf18da9514e3fed5aef1077296539bbd40a2f5d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZDPMJNZ7USVGBRIGEIXPJD7OCL:/var/lib/containers/storage/overlay/l/GRGK4CX7UQVEUSZK5HFTC2QHV6,upperdir=/var/lib/containers/storage/overlay/2611cea0f6d7fdd9f1675a63cdf18da9514e3fed5aef1077296539bbd40a2f5d/diff,workdir=/var/lib/containers/storage/overlay/2611cea0f6d7fdd9f1675a63cdf18da9514e3fed5aef1077296539bbd40a2f5d/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3253140k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/f615bf7fecc4048e9296830253ea42f24b89bd6232afcb38d2eacb83b1ae628e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZDPMJNZ7USVGBRIGEIXPJD7OCL:/var/lib/containers/storage/overlay/l/GRGK4CX7UQVEUSZK5HFTC2QHV6,upperdir=/var/lib/containers/storage/overlay/f615bf7fecc4048e9296830253ea42f24b89bd6232afcb38d2eacb83b1ae628e/diff,workdir=/var/lib/containers/storage/overlay/f615bf7fecc4048e9296830253ea42f24b89bd6232afcb38d2eacb83b1ae628e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8be810eedf9aee647d14fb696db3e765cd8c12327607cd6e9a346e68df38f5ad/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZDPMJNZ7USVGBRIGEIXPJD7OCL:/var/lib/containers/storage/overlay/l/GRGK4CX7UQVEUSZK5HFTC2QHV6,upperdir=/var/lib/containers/storage/overlay/8be810eedf9aee647d14fb696db3e765cd8c12327607cd6e9a346e68df38f5ad/diff,workdir=/var/lib/containers/storage/overlay/8be810eedf9aee647d14fb696db3e765cd8c12327607cd6e9a346e68df38f5ad/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6353949aa33478cf8e77d60a837e9dd11762a3176a37770ba758ed0a64f87d5d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZDPMJNZ7USVGBRIGEIXPJD7OCL:/var/lib/containers/storage/overlay/l/GRGK4CX7UQVEUSZK5HFTC2QHV6,upperdir=/var/lib/containers/storage/overlay/6353949aa33478cf8e77d60a837e9dd11762a3176a37770ba758ed0a64f87d5d/diff,workdir=/var/lib/containers/storage/overlay/6353949aa33478cf8e77d60a837e9dd11762a3176a37770ba758ed0a64f87d5d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ddcb6a8ca0b8fc67c901e7d03366914ee903d42068738cf3ef64703fa38301a7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZDPMJNZ7USVGBRIGEIXPJD7OCL:/var/lib/containers/storage/overlay/l/GRGK4CX7UQVEUSZK5HFTC2QHV6,upperdir=/var/lib/containers/storage/overlay/ddcb6a8ca0b8fc67c901e7d03366914ee903d42068738cf3ef64703fa38301a7/diff,workdir=/var/lib/containers/storage/overlay/ddcb6a8ca0b8fc67c901e7d03366914ee903d42068738cf3ef64703fa38301a7/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6eab8f8442477e290a5b0174c47939ec502eb2e0c4ef2a65448cc10cdec99749/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZDPMJNZ7USVGBRIGEIXPJD7OCL:/var/lib/containers/storage/overlay/l/GRGK4CX7UQVEUSZK5HFTC2QHV6,upperdir=/var/lib/containers/storage/overlay/6eab8f8442477e290a5b0174c47939ec502eb2e0c4ef2a65448cc10cdec99749/diff,workdir=/var/lib/containers/storage/overlay/6eab8f8442477e290a5b0174c47939ec502eb2e0c4ef2a65448cc10cdec99749/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6d408ec57ffcfbfbcb81e83096d06f7337664abb5598df86b2307a8cba5fa78b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZDPMJNZ7USVGBRIGEIXPJD7OCL:/var/lib/containers/storage/overlay/l/GRGK4CX7UQVEUSZK5HFTC2QHV6,upperdir=/var/lib/containers/storage/overlay/6d408ec57ffcfbfbcb81e83096d06f7337664abb5598df86b2307a8cba5fa78b/diff,workdir=/var/lib/containers/storage/overlay/6d408ec57ffcfbfbcb81e83096d06f7337664abb5598df86b2307a8cba5fa78b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5caf9955a09e35be1af066eba4ce942eecf0852da0a3ec12d66fce8431200b1f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZDPMJNZ7USVGBRIGEIXPJD7OCL:/var/lib/containers/storage/overlay/l/GRGK4CX7UQVEUSZK5HFTC2QHV6,upperdir=/var/lib/containers/storage/overlay/5caf9955a09e35be1af066eba4ce942eecf0852da0a3ec12d66fce8431200b1f/diff,workdir=/var/lib/containers/storage/overlay/5caf9955a09e35be1af066eba4ce942eecf0852da0a3ec12d66fce8431200b1f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/94a67a3d8a20159e364dffc71d30fa4e8f8f8ae753bba9164422c7d00d37b674/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/ZDPMJNZ7USVGBRIGEIXPJD7OCL:/var/lib/containers/storage/overlay/l/GRGK4CX7UQVEUSZK5HFTC2QHV6,upperdir=/var/lib/containers/storage/overlay/94a67a3d8a20159e364dffc71d30fa4e8f8f8ae753bba9164422c7d00d37b674/diff,workdir=/var/lib/containers/storage/overlay/94a67a3d8a20159e364dffc71d30fa4e8f8f8ae753bba9164422c7d00d37b674/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2024-03-18T21:01:59.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T21:01:59.492 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-18T21:01:59.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T21:01:59.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-03-18T21:01:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-03-18T21:01:59.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:01:59.668 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-18T21:01:59.679 INFO:tasks.cephadm:Teardown begin 2024-03-18T21:01:59.680 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-18T21:01:59.680 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T21:01:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.818+0000 7fc322198700 1 -- 172.21.15.46:0/3630158815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc31c102770 msgr2=0x7fc31c10ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.818+0000 7fc322198700 1 --2- 172.21.15.46:0/3630158815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc31c102770 0x7fc31c10ac50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc3100098f0 tx=0x7fc310009c00 comp rx=0 tx=0).stop 2024-03-18T21:01:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.818+0000 7fc322198700 1 -- 172.21.15.46:0/3630158815 shutdown_connections 2024-03-18T21:01:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.818+0000 7fc322198700 1 --2- 172.21.15.46:0/3630158815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc31c10b220 0x7fc31c10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.818+0000 7fc322198700 1 --2- 172.21.15.46:0/3630158815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc31c101e50 0x7fc31c102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.818+0000 7fc322198700 1 --2- 172.21.15.46:0/3630158815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc31c102770 0x7fc31c10ac50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.818+0000 7fc322198700 1 -- 172.21.15.46:0/3630158815 >> 172.21.15.46:0/3630158815 conn(0x7fc31c0fb5f0 msgr2=0x7fc31c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:01:59.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.818+0000 7fc322198700 1 -- 172.21.15.46:0/3630158815 shutdown_connections 2024-03-18T21:01:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.818+0000 7fc322198700 1 -- 172.21.15.46:0/3630158815 wait complete. 2024-03-18T21:01:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.818+0000 7fc322198700 1 Processor -- start 2024-03-18T21:01:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc322198700 1 -- start start 2024-03-18T21:01:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc322198700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc31c101e50 0x7fc31c0ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc322198700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc31c102770 0x7fc31c0fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc322198700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc31c10b220 0x7fc31c100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc322198700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c1a5540 con 0x7fc31c101e50 2024-03-18T21:01:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc322198700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c1a56b0 con 0x7fc31c10b220 2024-03-18T21:01:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc322198700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c1a5870 con 0x7fc31c102770 2024-03-18T21:01:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc31b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc31c101e50 0x7fc31c0ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:59.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc31b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc31c101e50 0x7fc31c0ffaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52488/0 (socket says 172.21.15.46:52488) 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc31b7fe700 1 -- 172.21.15.46:0/3780428184 learned_addr learned my addr 172.21.15.46:0/3780428184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc31bfff700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc31c10b220 0x7fc31c100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc31affd700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc31c102770 0x7fc31c0fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc31b7fe700 1 -- 172.21.15.46:0/3780428184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc31c102770 msgr2=0x7fc31c0fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc31b7fe700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc31c102770 0x7fc31c0fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc31b7fe700 1 -- 172.21.15.46:0/3780428184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc31c10b220 msgr2=0x7fc31c100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc31b7fe700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc31c10b220 0x7fc31c100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.819+0000 7fc31b7fe700 1 -- 172.21.15.46:0/3780428184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc314009d00 con 0x7fc31c101e50 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.820+0000 7fc31b7fe700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc31c101e50 0x7fc31c0ffaa0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00d950 tx=0x7fc30c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.820+0000 7fc318ff9700 1 -- 172.21.15.46:0/3780428184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00f890 con 0x7fc31c101e50 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.820+0000 7fc318ff9700 1 -- 172.21.15.46:0/3780428184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc30c010460 con 0x7fc31c101e50 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.820+0000 7fc322198700 1 -- 172.21.15.46:0/3780428184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc310009520 con 0x7fc31c101e50 2024-03-18T21:01:59.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.820+0000 7fc318ff9700 1 -- 172.21.15.46:0/3780428184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00e610 con 0x7fc31c101e50 2024-03-18T21:01:59.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.820+0000 7fc322198700 1 -- 172.21.15.46:0/3780428184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31c1a5ea0 con 0x7fc31c101e50 2024-03-18T21:01:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.821+0000 7fc322198700 1 -- 172.21.15.46:0/3780428184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc31c106340 con 0x7fc31c101e50 2024-03-18T21:01:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.822+0000 7fc318ff9700 1 -- 172.21.15.46:0/3780428184 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc30c0108d0 con 0x7fc31c101e50 2024-03-18T21:01:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.823+0000 7fc318ff9700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc3040a1290 0x7fc3040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:01:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.823+0000 7fc31affd700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc3040a1290 0x7fc3040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:01:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.823+0000 7fc318ff9700 1 -- 172.21.15.46:0/3780428184 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7fc30c0ccab0 con 0x7fc31c101e50 2024-03-18T21:01:59.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.824+0000 7fc31affd700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc3040a1290 0x7fc3040a3750 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc3100098c0 tx=0x7fc310019040 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:01:59.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:01:59.827+0000 7fc318ff9700 1 -- 172.21.15.46:0/3780428184 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc30c095ee0 con 0x7fc31c101e50 2024-03-18T21:02:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.090+0000 7fc322198700 1 -- 172.21.15.46:0/3780428184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc31c068630 con 0x7fc31c101e50 2024-03-18T21:02:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.091+0000 7fc318ff9700 1 -- 172.21.15.46:0/3780428184 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+10928 (secure 0 0 0) 0x7fc30c016090 con 0x7fc31c101e50 2024-03-18T21:02:00.093 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T21:02:00.094 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":111,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T21:01:32.566934+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":218,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[0,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":215,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_30559":{"gid":30559,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1436755958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1436755958},{"type":"v1","addr":"172.21.15.155:6837","nonce":1436755958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T21:02:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.093+0000 7fc322198700 1 -- 172.21.15.46:0/3780428184 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc3040a1290 msgr2=0x7fc3040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.093+0000 7fc322198700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc3040a1290 0x7fc3040a3750 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc3100098c0 tx=0x7fc310019040 comp rx=0 tx=0).stop 2024-03-18T21:02:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.094+0000 7fc322198700 1 -- 172.21.15.46:0/3780428184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc31c101e50 msgr2=0x7fc31c0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.094+0000 7fc322198700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc31c101e50 0x7fc31c0ffaa0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00d950 tx=0x7fc30c00dc60 comp rx=0 tx=0).stop 2024-03-18T21:02:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.094+0000 7fc322198700 1 -- 172.21.15.46:0/3780428184 shutdown_connections 2024-03-18T21:02:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.094+0000 7fc322198700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fc3040a1290 0x7fc3040a3750 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.094+0000 7fc322198700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc31c101e50 0x7fc31c0ffaa0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.094+0000 7fc322198700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc31c10b220 0x7fc31c100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.094+0000 7fc322198700 1 --2- 172.21.15.46:0/3780428184 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc31c102770 0x7fc31c0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.094+0000 7fc322198700 1 -- 172.21.15.46:0/3780428184 >> 172.21.15.46:0/3780428184 conn(0x7fc31c0fb5f0 msgr2=0x7fc31c0fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.094+0000 7fc322198700 1 -- 172.21.15.46:0/3780428184 shutdown_connections 2024-03-18T21:02:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.094+0000 7fc322198700 1 -- 172.21.15.46:0/3780428184 wait complete. 2024-03-18T21:02:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 111 2024-03-18T21:02:00.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T21:02:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.246+0000 7f281c56e700 1 -- 172.21.15.46:0/112018754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f281410b220 msgr2=0x7f281410d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.246+0000 7f281c56e700 1 --2- 172.21.15.46:0/112018754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f281410b220 0x7f281410d6c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f280c00b0a0 tx=0x7f280c00b3b0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.246+0000 7f281c56e700 1 -- 172.21.15.46:0/112018754 shutdown_connections 2024-03-18T21:02:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.246+0000 7f281c56e700 1 --2- 172.21.15.46:0/112018754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f281410b220 0x7f281410d6c0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.246+0000 7f281c56e700 1 --2- 172.21.15.46:0/112018754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2814101e50 0x7f2814102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.246+0000 7f281c56e700 1 --2- 172.21.15.46:0/112018754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2814102770 0x7f281410ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.246+0000 7f281c56e700 1 -- 172.21.15.46:0/112018754 >> 172.21.15.46:0/112018754 conn(0x7f28140fb5f0 msgr2=0x7f28140fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.246+0000 7f281c56e700 1 -- 172.21.15.46:0/112018754 shutdown_connections 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.246+0000 7f281c56e700 1 -- 172.21.15.46:0/112018754 wait complete. 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.246+0000 7f281c56e700 1 Processor -- start 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.246+0000 7f281c56e700 1 -- start start 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f281c56e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2814101e50 0x7f2814197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f281c56e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2814102770 0x7f2814197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f281c56e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281410b220 0x7f281419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f281c56e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281419c170 con 0x7f2814102770 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f281c56e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281419c2e0 con 0x7f281410b220 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f281c56e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281419c4a0 con 0x7f2814101e50 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f2819b09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2814102770 0x7f2814197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f2819b09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2814102770 0x7f2814197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52502/0 (socket says 172.21.15.46:52502) 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f2819b09700 1 -- 172.21.15.46:0/1949073919 learned_addr learned my addr 172.21.15.46:0/1949073919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:02:00.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f281a30a700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2814101e50 0x7f2814197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f281ab0b700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281410b220 0x7f281419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f2819b09700 1 -- 172.21.15.46:0/1949073919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2814101e50 msgr2=0x7f2814197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f2819b09700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2814101e50 0x7f2814197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f2819b09700 1 -- 172.21.15.46:0/1949073919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281410b220 msgr2=0x7f281419bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f2819b09700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281410b220 0x7f281419bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f2819b09700 1 -- 172.21.15.46:0/1949073919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c009d00 con 0x7f2814102770 2024-03-18T21:02:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f2819b09700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2814102770 0x7f2814197790 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f280800d950 tx=0x7f280800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:02:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.247+0000 7f28077fe700 1 -- 172.21.15.46:0/1949073919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280800f890 con 0x7f2814102770 2024-03-18T21:02:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.248+0000 7f28077fe700 1 -- 172.21.15.46:0/1949073919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2808010460 con 0x7f2814102770 2024-03-18T21:02:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.248+0000 7f281c56e700 1 -- 172.21.15.46:0/1949073919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281419c780 con 0x7f2814102770 2024-03-18T21:02:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.248+0000 7f28077fe700 1 -- 172.21.15.46:0/1949073919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280800e610 con 0x7f2814102770 2024-03-18T21:02:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.248+0000 7f281c56e700 1 -- 172.21.15.46:0/1949073919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28141a3fc0 con 0x7f2814102770 2024-03-18T21:02:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.248+0000 7f281c56e700 1 -- 172.21.15.46:0/1949073919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28141063a0 con 0x7f2814102770 2024-03-18T21:02:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.250+0000 7f28077fe700 1 -- 172.21.15.46:0/1949073919 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f28080108d0 con 0x7f2814102770 2024-03-18T21:02:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.250+0000 7f28077fe700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f28000a1290 0x7f28000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:00.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.251+0000 7f281a30a700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f28000a1290 0x7f28000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.251+0000 7f28077fe700 1 -- 172.21.15.46:0/1949073919 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f28080cca90 con 0x7f2814102770 2024-03-18T21:02:00.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.251+0000 7f281a30a700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f28000a1290 0x7f28000a3750 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2810005950 tx=0x7f281000b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:02:00.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.254+0000 7f28077fe700 1 -- 172.21.15.46:0/1949073919 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2808095ec0 con 0x7f2814102770 2024-03-18T21:02:00.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.533+0000 7f281c56e700 1 -- 172.21.15.46:0/1949073919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2814198550 con 0x7f2814102770 2024-03-18T21:02:00.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.533+0000 7f28077fe700 1 -- 172.21.15.46:0/1949073919 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+10928 (secure 0 0 0) 0x7f2808016090 con 0x7f2814102770 2024-03-18T21:02:00.536 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T21:02:00.536 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/231081000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":231081000},{"type":"v1","addr":"172.21.15.46:6835","nonce":231081000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/114990598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":114990598},{"type":"v1","addr":"172.21.15.46:6837","nonce":114990598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":111,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T20:34:38.785094+0000","modified":"2024-03-18T21:01:32.566934+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24536,"mds_1":24521,"mds_2":24514,"mds_3":24506,"mds_4":24446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.46:6841/3470614253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":3470614253},{"type":"v1","addr":"172.21.15.46:6841","nonce":3470614253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24446":{"gid":24446,"name":"k","rank":4,"incarnation":24,"state":"up:active","state_seq":218,"addr":"172.21.15.73:6833/2440572005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2440572005},{"type":"v1","addr":"172.21.15.73:6833","nonce":2440572005}]},"join_fscid":-1,"export_targets":[0,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24448":{"gid":24448,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6833/4207499009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4207499009},{"type":"v1","addr":"172.21.15.155:6833","nonce":4207499009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24479":{"gid":24479,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6835/2049469125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2049469125},{"type":"v1","addr":"172.21.15.73:6835","nonce":2049469125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/2573112997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2573112997},{"type":"v1","addr":"172.21.15.155:6835","nonce":2573112997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.73:6837/2773013996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2773013996},{"type":"v1","addr":"172.21.15.73:6837","nonce":2773013996}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":215,"addr":"172.21.15.155:6839/3116856560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":3116856560},{"type":"v1","addr":"172.21.15.155:6839","nonce":3116856560}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"g","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.46:6839/1267173621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":1267173621},{"type":"v1","addr":"172.21.15.46:6839","nonce":1267173621}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6839/2377859422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2377859422},{"type":"v1","addr":"172.21.15.73:6839","nonce":2377859422}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_30559":{"gid":30559,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1436755958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1436755958},{"type":"v1","addr":"172.21.15.155:6837","nonce":1436755958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-18T21:02:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.536+0000 7f281c56e700 1 -- 172.21.15.46:0/1949073919 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f28000a1290 msgr2=0x7f28000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.536+0000 7f281c56e700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f28000a1290 0x7f28000a3750 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2810005950 tx=0x7f281000b410 comp rx=0 tx=0).stop 2024-03-18T21:02:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.536+0000 7f281c56e700 1 -- 172.21.15.46:0/1949073919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2814102770 msgr2=0x7f2814197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.536+0000 7f281c56e700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2814102770 0x7f2814197790 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f280800d950 tx=0x7f280800dc60 comp rx=0 tx=0).stop 2024-03-18T21:02:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.536+0000 7f281c56e700 1 -- 172.21.15.46:0/1949073919 shutdown_connections 2024-03-18T21:02:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.536+0000 7f281c56e700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f28000a1290 0x7f28000a3750 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.536+0000 7f281c56e700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f281410b220 0x7f281419bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.536+0000 7f281c56e700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2814101e50 0x7f2814197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.536+0000 7f281c56e700 1 --2- 172.21.15.46:0/1949073919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2814102770 0x7f2814197790 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.536+0000 7f281c56e700 1 -- 172.21.15.46:0/1949073919 >> 172.21.15.46:0/1949073919 conn(0x7f28140fb5f0 msgr2=0x7f28140fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.537+0000 7f281c56e700 1 -- 172.21.15.46:0/1949073919 shutdown_connections 2024-03-18T21:02:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.537+0000 7f281c56e700 1 -- 172.21.15.46:0/1949073919 wait complete. 2024-03-18T21:02:00.540 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 111 2024-03-18T21:02:00.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T21:02:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.700+0000 7efd5e3a5700 1 -- 172.21.15.46:0/989441268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd5810b1d0 msgr2=0x7efd5810d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.700+0000 7efd5e3a5700 1 --2- 172.21.15.46:0/989441268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd5810b1d0 0x7efd5810d690 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7efd5000b0a0 tx=0x7efd5000b3b0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.700+0000 7efd5e3a5700 1 -- 172.21.15.46:0/989441268 shutdown_connections 2024-03-18T21:02:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.700+0000 7efd5e3a5700 1 --2- 172.21.15.46:0/989441268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd5810b1d0 0x7efd5810d690 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.700+0000 7efd5e3a5700 1 --2- 172.21.15.46:0/989441268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58101db0 0x7efd58102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.700+0000 7efd5e3a5700 1 --2- 172.21.15.46:0/989441268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd581026d0 0x7efd5810abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.700+0000 7efd5e3a5700 1 -- 172.21.15.46:0/989441268 >> 172.21.15.46:0/989441268 conn(0x7efd580fb5f0 msgr2=0x7efd580fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.700+0000 7efd5e3a5700 1 -- 172.21.15.46:0/989441268 shutdown_connections 2024-03-18T21:02:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.700+0000 7efd5e3a5700 1 -- 172.21.15.46:0/989441268 wait complete. 2024-03-18T21:02:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.701+0000 7efd5e3a5700 1 Processor -- start 2024-03-18T21:02:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.701+0000 7efd5e3a5700 1 -- start start 2024-03-18T21:02:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd5e3a5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58101db0 0x7efd580ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd5e3a5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd581026d0 0x7efd580fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd5e3a5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5810b1d0 0x7efd58100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd5e3a5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd581a5580 con 0x7efd581026d0 2024-03-18T21:02:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd5e3a5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd581a56f0 con 0x7efd5810b1d0 2024-03-18T21:02:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd5e3a5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd581a58b0 con 0x7efd58101db0 2024-03-18T21:02:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd577fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd581026d0 0x7efd580fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd577fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd581026d0 0x7efd580fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52508/0 (socket says 172.21.15.46:52508) 2024-03-18T21:02:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd577fe700 1 -- 172.21.15.46:0/3267885378 learned_addr learned my addr 172.21.15.46:0/3267885378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:02:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd57fff700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58101db0 0x7efd580ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd5c942700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5810b1d0 0x7efd58100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd577fe700 1 -- 172.21.15.46:0/3267885378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58101db0 msgr2=0x7efd580ffa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd577fe700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58101db0 0x7efd580ffa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd577fe700 1 -- 172.21.15.46:0/3267885378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5810b1d0 msgr2=0x7efd58100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd577fe700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5810b1d0 0x7efd58100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd577fe700 1 -- 172.21.15.46:0/3267885378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd50009d00 con 0x7efd581026d0 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd57fff700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58101db0 0x7efd580ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd5c942700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5810b1d0 0x7efd58100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd577fe700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd581026d0 0x7efd580fff40 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00d750 tx=0x7efd4c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd557fa700 1 -- 172.21.15.46:0/3267885378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c004220 con 0x7efd581026d0 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd557fa700 1 -- 172.21.15.46:0/3267885378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd4c004cf0 con 0x7efd581026d0 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd5e3a5700 1 -- 172.21.15.46:0/3267885378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd581a5b90 con 0x7efd581026d0 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.702+0000 7efd557fa700 1 -- 172.21.15.46:0/3267885378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c0056a0 con 0x7efd581026d0 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.703+0000 7efd5e3a5700 1 -- 172.21.15.46:0/3267885378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd581a60a0 con 0x7efd581026d0 2024-03-18T21:02:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.704+0000 7efd5e3a5700 1 -- 172.21.15.46:0/3267885378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd58106300 con 0x7efd581026d0 2024-03-18T21:02:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.705+0000 7efd557fa700 1 -- 172.21.15.46:0/3267885378 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7efd4c004380 con 0x7efd581026d0 2024-03-18T21:02:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.706+0000 7efd557fa700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efd400a1290 0x7efd400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.706+0000 7efd57fff700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efd400a1290 0x7efd400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.706+0000 7efd557fa700 1 -- 172.21.15.46:0/3267885378 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7efd4c0ccec0 con 0x7efd581026d0 2024-03-18T21:02:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.706+0000 7efd57fff700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efd400a1290 0x7efd400a3750 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efd48011490 tx=0x7efd48009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:02:00.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.710+0000 7efd557fa700 1 -- 172.21.15.46:0/3267885378 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efd4c0962f0 con 0x7efd581026d0 2024-03-18T21:02:00.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.939+0000 7efd5e3a5700 1 -- 172.21.15.46:0/3267885378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd580686d0 con 0x7efd581026d0 2024-03-18T21:02:00.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.941+0000 7efd557fa700 1 -- 172.21.15.46:0/3267885378 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+23036 (secure 0 0 0) 0x7efd4c016020 con 0x7efd581026d0 2024-03-18T21:02:00.943 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-18T21:02:00.943 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":122,"fsid":"04a8ae18-e566-11ee-95c9-87774f69a715","created":"2024-03-18T20:28:46.137332+0000","modified":"2024-03-18T21:01:20.339788+0000","last_up_change":"2024-03-18T20:34:05.244017+0000","last_in_change":"2024-03-18T20:33:49.133808+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T20:31:27.488318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T20:34:37.213560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T20:34:38.162692+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"303e69be-0b32-4cff-b91e-ce6665f3a087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6803","nonce":2908504490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6805","nonce":2908504490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6809","nonce":2908504490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2908504490},{"type":"v1","addr":"172.21.15.46:6807","nonce":2908504490}]},"public_addr":"172.21.15.46:6803/2908504490","cluster_addr":"172.21.15.46:6805/2908504490","heartbeat_back_addr":"172.21.15.46:6809/2908504490","heartbeat_front_addr":"172.21.15.46:6807/2908504490","state":["exists","up"]},{"osd":1,"uuid":"e7b934a7-0679-4158-bc78-6f8cc7d107c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6801","nonce":1334268822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6803","nonce":1334268822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6807","nonce":1334268822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":1334268822},{"type":"v1","addr":"172.21.15.73:6805","nonce":1334268822}]},"public_addr":"172.21.15.73:6801/1334268822","cluster_addr":"172.21.15.73:6803/1334268822","heartbeat_back_addr":"172.21.15.73:6807/1334268822","heartbeat_front_addr":"172.21.15.73:6805/1334268822","state":["exists","up"]},{"osd":2,"uuid":"10f51beb-1c27-4b01-9648-f296eddd43af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6801","nonce":772117296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6803","nonce":772117296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6807","nonce":772117296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":772117296},{"type":"v1","addr":"172.21.15.155:6805","nonce":772117296}]},"public_addr":"172.21.15.155:6801/772117296","cluster_addr":"172.21.15.155:6803/772117296","heartbeat_back_addr":"172.21.15.155:6807/772117296","heartbeat_front_addr":"172.21.15.155:6805/772117296","state":["exists","up"]},{"osd":3,"uuid":"e36074d6-5d67-4411-998e-42bfd7f85a98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6811","nonce":1070659456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6813","nonce":1070659456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6817","nonce":1070659456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1070659456},{"type":"v1","addr":"172.21.15.46:6815","nonce":1070659456}]},"public_addr":"172.21.15.46:6811/1070659456","cluster_addr":"172.21.15.46:6813/1070659456","heartbeat_back_addr":"172.21.15.46:6817/1070659456","heartbeat_front_addr":"172.21.15.46:6815/1070659456","state":["exists","up"]},{"osd":4,"uuid":"73dd25b5-ca0d-4ca4-b079-34487065b2c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6809","nonce":1954145749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6811","nonce":1954145749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6815","nonce":1954145749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1954145749},{"type":"v1","addr":"172.21.15.73:6813","nonce":1954145749}]},"public_addr":"172.21.15.73:6809/1954145749","cluster_addr":"172.21.15.73:6811/1954145749","heartbeat_back_addr":"172.21.15.73:6815/1954145749","heartbeat_front_addr":"172.21.15.73:6813/1954145749","state":["exists","up"]},{"osd":5,"uuid":"83fcbac8-8a5c-4ddf-a8d3-2c6757f1c697","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6809","nonce":2840490215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6811","nonce":2840490215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6815","nonce":2840490215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2840490215},{"type":"v1","addr":"172.21.15.155:6813","nonce":2840490215}]},"public_addr":"172.21.15.155:6809/2840490215","cluster_addr":"172.21.15.155:6811/2840490215","heartbeat_back_addr":"172.21.15.155:6815/2840490215","heartbeat_front_addr":"172.21.15.155:6813/2840490215","state":["exists","up"]},{"osd":6,"uuid":"cb44585c-e16c-421c-981a-a7cb8b9ba58d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6819","nonce":2670405640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6821","nonce":2670405640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6825","nonce":2670405640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2670405640},{"type":"v1","addr":"172.21.15.46:6823","nonce":2670405640}]},"public_addr":"172.21.15.46:6819/2670405640","cluster_addr":"172.21.15.46:6821/2670405640","heartbeat_back_addr":"172.21.15.46:6825/2670405640","heartbeat_front_addr":"172.21.15.46:6823/2670405640","state":["exists","up"]},{"osd":7,"uuid":"85b71a49-23d9-4769-b084-7066e7bb38a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6817","nonce":1220123190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6819","nonce":1220123190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6823","nonce":1220123190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1220123190},{"type":"v1","addr":"172.21.15.73:6821","nonce":1220123190}]},"public_addr":"172.21.15.73:6817/1220123190","cluster_addr":"172.21.15.73:6819/1220123190","heartbeat_back_addr":"172.21.15.73:6823/1220123190","heartbeat_front_addr":"172.21.15.73:6821/1220123190","state":["exists","up"]},{"osd":8,"uuid":"45ae5d47-1129-43f7-a5b5-d7e3336ae755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6817","nonce":962631462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6819","nonce":962631462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6823","nonce":962631462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":962631462},{"type":"v1","addr":"172.21.15.155:6821","nonce":962631462}]},"public_addr":"172.21.15.155:6817/962631462","cluster_addr":"172.21.15.155:6819/962631462","heartbeat_back_addr":"172.21.15.155:6823/962631462","heartbeat_front_addr":"172.21.15.155:6821/962631462","state":["exists","up"]},{"osd":9,"uuid":"a8f8c910-b6f1-496a-b5ca-ef6f68fef718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6827","nonce":167112015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6829","nonce":167112015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6833","nonce":167112015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":167112015},{"type":"v1","addr":"172.21.15.46:6831","nonce":167112015}]},"public_addr":"172.21.15.46:6827/167112015","cluster_addr":"172.21.15.46:6829/167112015","heartbeat_back_addr":"172.21.15.46:6833/167112015","heartbeat_front_addr":"172.21.15.46:6831/167112015","state":["exists","up"]},{"osd":10,"uuid":"b9975d16-8806-494b-926e-2ba60d76ac1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6825","nonce":2313090025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6827","nonce":2313090025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6831","nonce":2313090025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2313090025},{"type":"v1","addr":"172.21.15.73:6829","nonce":2313090025}]},"public_addr":"172.21.15.73:6825/2313090025","cluster_addr":"172.21.15.73:6827/2313090025","heartbeat_back_addr":"172.21.15.73:6831/2313090025","heartbeat_front_addr":"172.21.15.73:6829/2313090025","state":["exists","up"]},{"osd":11,"uuid":"ef1f614e-b094-4a41-a132-8c4db3d34b3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6825","nonce":2159697764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6827","nonce":2159697764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6831","nonce":2159697764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":2159697764},{"type":"v1","addr":"172.21.15.155:6829","nonce":2159697764}]},"public_addr":"172.21.15.155:6825/2159697764","cluster_addr":"172.21.15.155:6827/2159697764","heartbeat_back_addr":"172.21.15.155:6831/2159697764","heartbeat_front_addr":"172.21.15.155:6829/2159697764","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:30:49.834070+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:07.274208+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:24.614252+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:31:42.758236+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:00.160549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:17.309427+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:35.228523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:32:53.001892+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:10.490011+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:28.407286+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:33:46.423347+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T20:34:03.807433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/3400106473":"2024-03-19T20:36:06.973937+0000","172.21.15.46:6800/1385957034":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2440783396":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/699745793":"2024-03-19T20:29:14.346231+0000","172.21.15.46:0/2587067898":"2024-03-19T20:36:06.973937+0000","172.21.15.46:0/606003730":"2024-03-19T20:29:14.346231+0000","172.21.15.46:6800/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/3248305722":"2024-03-19T20:36:06.973937+0000","172.21.15.46:6801/2753187643":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1997677402":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/3646752058":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/604593037":"2024-03-19T20:36:06.973937+0000","172.21.15.46:0/2471573219":"2024-03-19T20:29:36.953241+0000","172.21.15.46:0/1017695887":"2024-03-19T20:36:06.973937+0000","172.21.15.46:0/2204627135":"2024-03-19T20:36:06.973937+0000","172.21.15.46:6801/3400106473":"2024-03-19T20:36:06.973937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T21:02:00.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.943+0000 7efd5e3a5700 1 -- 172.21.15.46:0/3267885378 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efd400a1290 msgr2=0x7efd400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.943+0000 7efd5e3a5700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efd400a1290 0x7efd400a3750 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efd48011490 tx=0x7efd48009260 comp rx=0 tx=0).stop 2024-03-18T21:02:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.943+0000 7efd5e3a5700 1 -- 172.21.15.46:0/3267885378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd581026d0 msgr2=0x7efd580fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.943+0000 7efd5e3a5700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd581026d0 0x7efd580fff40 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00d750 tx=0x7efd4c00da60 comp rx=0 tx=0).stop 2024-03-18T21:02:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.943+0000 7efd5e3a5700 1 -- 172.21.15.46:0/3267885378 shutdown_connections 2024-03-18T21:02:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.943+0000 7efd5e3a5700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7efd400a1290 0x7efd400a3750 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.943+0000 7efd5e3a5700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5810b1d0 0x7efd58100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.943+0000 7efd5e3a5700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58101db0 0x7efd580ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.943+0000 7efd5e3a5700 1 --2- 172.21.15.46:0/3267885378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd581026d0 0x7efd580fff40 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.943+0000 7efd5e3a5700 1 -- 172.21.15.46:0/3267885378 >> 172.21.15.46:0/3267885378 conn(0x7efd580fb5f0 msgr2=0x7efd580fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.943+0000 7efd5e3a5700 1 -- 172.21.15.46:0/3267885378 shutdown_connections 2024-03-18T21:02:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:00.943+0000 7efd5e3a5700 1 -- 172.21.15.46:0/3267885378 wait complete. 2024-03-18T21:02:00.958 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:00 smithi046 ceph-mon[43495]: pgmap v820: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T21:02:00.958 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:00 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3780428184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T21:02:00.958 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:00 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/1949073919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T21:02:00.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-18T21:02:00.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:00 smithi073 ceph-mon[47384]: pgmap v820: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T21:02:00.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:00 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3780428184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T21:02:00.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:00 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/1949073919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T21:02:01.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:00 smithi155 ceph-mon[48344]: pgmap v820: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-03-18T21:02:01.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:00 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3780428184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T21:02:01.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:00 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/1949073919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T21:02:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.096+0000 7f1aaff0d700 1 -- 172.21.15.46:0/197375251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aa8073b00 msgr2=0x7f1aa810cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.096+0000 7f1aaff0d700 1 --2- 172.21.15.46:0/197375251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aa8073b00 0x7f1aa810cff0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009990 tx=0x7f1a9c009ca0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.096+0000 7f1aaff0d700 1 -- 172.21.15.46:0/197375251 shutdown_connections 2024-03-18T21:02:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.096+0000 7f1aaff0d700 1 --2- 172.21.15.46:0/197375251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa810d530 0x7f1aa810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.096+0000 7f1aaff0d700 1 --2- 172.21.15.46:0/197375251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1aa80731e0 0x7f1aa80735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.096+0000 7f1aaff0d700 1 --2- 172.21.15.46:0/197375251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aa8073b00 0x7f1aa810cff0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.096+0000 7f1aaff0d700 1 -- 172.21.15.46:0/197375251 >> 172.21.15.46:0/197375251 conn(0x7f1aa80fc7c0 msgr2=0x7f1aa80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.097+0000 7f1aaff0d700 1 -- 172.21.15.46:0/197375251 shutdown_connections 2024-03-18T21:02:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.097+0000 7f1aaff0d700 1 -- 172.21.15.46:0/197375251 wait complete. 2024-03-18T21:02:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.097+0000 7f1aaff0d700 1 Processor -- start 2024-03-18T21:02:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.097+0000 7f1aaff0d700 1 -- start start 2024-03-18T21:02:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.097+0000 7f1aaff0d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa80731e0 0x7f1aa8199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aaff0d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aa8073b00 0x7f1aa8199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aaff0d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1aa810d530 0x7f1aa819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aaff0d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa819e340 con 0x7f1aa8073b00 2024-03-18T21:02:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aaff0d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa819e4b0 con 0x7f1aa80731e0 2024-03-18T21:02:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aaff0d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa819e670 con 0x7f1aa810d530 2024-03-18T21:02:01.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aad4a8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aa8073b00 0x7f1aa8199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aad4a8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aa8073b00 0x7f1aa8199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52528/0 (socket says 172.21.15.46:52528) 2024-03-18T21:02:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aadca9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa80731e0 0x7f1aa8199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aad4a8700 1 -- 172.21.15.46:0/3242982329 learned_addr learned my addr 172.21.15.46:0/3242982329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:02:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aae4aa700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1aa810d530 0x7f1aa819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aad4a8700 1 -- 172.21.15.46:0/3242982329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1aa810d530 msgr2=0x7f1aa819dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aad4a8700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1aa810d530 0x7f1aa819dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aad4a8700 1 -- 172.21.15.46:0/3242982329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa80731e0 msgr2=0x7f1aa8199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aad4a8700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa80731e0 0x7f1aa8199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aad4a8700 1 -- 172.21.15.46:0/3242982329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c0095f0 con 0x7f1aa8073b00 2024-03-18T21:02:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aae4aa700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1aa810d530 0x7f1aa819dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T21:02:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aad4a8700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aa8073b00 0x7f1aa8199960 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c000c00 tx=0x7f1a9c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:02:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aa6ffd700 1 -- 172.21.15.46:0/3242982329 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c01d070 con 0x7f1aa8073b00 2024-03-18T21:02:01.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aaff0d700 1 -- 172.21.15.46:0/3242982329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa819e8f0 con 0x7f1aa8073b00 2024-03-18T21:02:01.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aa6ffd700 1 -- 172.21.15.46:0/3242982329 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a9c022470 con 0x7f1aa8073b00 2024-03-18T21:02:01.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aa6ffd700 1 -- 172.21.15.46:0/3242982329 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c017650 con 0x7f1aa8073b00 2024-03-18T21:02:01.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.098+0000 7f1aaff0d700 1 -- 172.21.15.46:0/3242982329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa81a6190 con 0x7f1aa8073b00 2024-03-18T21:02:01.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.099+0000 7f1aaff0d700 1 -- 172.21.15.46:0/3242982329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa8108740 con 0x7f1aa8073b00 2024-03-18T21:02:01.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.101+0000 7f1aa6ffd700 1 -- 172.21.15.46:0/3242982329 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1a9c0225e0 con 0x7f1aa8073b00 2024-03-18T21:02:01.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.102+0000 7f1aa6ffd700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1a900a1290 0x7f1a900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:01.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.102+0000 7f1aadca9700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1a900a1290 0x7f1a900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:01.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.102+0000 7f1aa6ffd700 1 -- 172.21.15.46:0/3242982329 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8556+0+0 (secure 0 0 0) 0x7f1a9c0ce400 con 0x7f1aa8073b00 2024-03-18T21:02:01.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.102+0000 7f1aadca9700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1a900a1290 0x7f1a900a3750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009550 tx=0x7f1a980091b0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:02:01.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.106+0000 7f1aa6ffd700 1 -- 172.21.15.46:0/3242982329 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1a9c0978c0 con 0x7f1aa8073b00 2024-03-18T21:02:01.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.372+0000 7f1aaff0d700 1 -- 172.21.15.46:0/3242982329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1aa8074890 con 0x7f1aa8073b00 2024-03-18T21:02:01.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.384+0000 7f1aa6ffd700 1 -- 172.21.15.46:0/3242982329 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v112) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1a9c09aee0 con 0x7f1aa8073b00 2024-03-18T21:02:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.386+0000 7f1aaff0d700 1 -- 172.21.15.46:0/3242982329 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1a900a1290 msgr2=0x7f1a900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.387+0000 7f1aaff0d700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1a900a1290 0x7f1a900a3750 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009550 tx=0x7f1a980091b0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.387+0000 7f1aaff0d700 1 -- 172.21.15.46:0/3242982329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aa8073b00 msgr2=0x7f1aa8199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.387+0000 7f1aaff0d700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aa8073b00 0x7f1aa8199960 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c000c00 tx=0x7f1a9c005740 comp rx=0 tx=0).stop 2024-03-18T21:02:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.387+0000 7f1aaff0d700 1 -- 172.21.15.46:0/3242982329 shutdown_connections 2024-03-18T21:02:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.387+0000 7f1aaff0d700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f1a900a1290 0x7f1a900a3750 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.387+0000 7f1aaff0d700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1aa810d530 0x7f1aa819dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.387+0000 7f1aaff0d700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1aa80731e0 0x7f1aa8199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.387+0000 7f1aaff0d700 1 --2- 172.21.15.46:0/3242982329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1aa8073b00 0x7f1aa8199960 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.387+0000 7f1aaff0d700 1 -- 172.21.15.46:0/3242982329 >> 172.21.15.46:0/3242982329 conn(0x7f1aa80fc7c0 msgr2=0x7f1aa80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:01.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.387+0000 7f1aaff0d700 1 -- 172.21.15.46:0/3242982329 shutdown_connections 2024-03-18T21:02:01.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.387+0000 7f1aaff0d700 1 -- 172.21.15.46:0/3242982329 wait complete. 2024-03-18T21:02:01.390 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-18T21:02:01.405 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-18T21:02:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.549+0000 7fd064cbe700 1 -- 172.21.15.46:0/529502886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd060101900 msgr2=0x7fd060105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:01.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.549+0000 7fd064cbe700 1 --2- 172.21.15.46:0/529502886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd060101900 0x7fd060105940 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fd054009990 tx=0x7fd054009ca0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.549+0000 7fd064cbe700 1 -- 172.21.15.46:0/529502886 shutdown_connections 2024-03-18T21:02:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.549+0000 7fd064cbe700 1 --2- 172.21.15.46:0/529502886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060106070 0x7fd06010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.549+0000 7fd064cbe700 1 --2- 172.21.15.46:0/529502886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd060100f50 0x7fd060101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.549+0000 7fd064cbe700 1 --2- 172.21.15.46:0/529502886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd060101900 0x7fd060105940 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.549+0000 7fd064cbe700 1 -- 172.21.15.46:0/529502886 >> 172.21.15.46:0/529502886 conn(0x7fd0600fc7f0 msgr2=0x7fd0600fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.549+0000 7fd064cbe700 1 -- 172.21.15.46:0/529502886 shutdown_connections 2024-03-18T21:02:01.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.549+0000 7fd064cbe700 1 -- 172.21.15.46:0/529502886 wait complete. 2024-03-18T21:02:01.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd064cbe700 1 Processor -- start 2024-03-18T21:02:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd064cbe700 1 -- start start 2024-03-18T21:02:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd064cbe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd060100f50 0x7fd0600730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd064cbe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd060106070 0x7fd0600735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd064cbe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060073b30 0x7fd0601a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd064cbe700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601a59d0 con 0x7fd060106070 2024-03-18T21:02:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd064cbe700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601a5b40 con 0x7fd060100f50 2024-03-18T21:02:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd064cbe700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601a5d00 con 0x7fd060073b30 2024-03-18T21:02:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd05dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd060106070 0x7fd0600735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd05dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd060106070 0x7fd0600735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52572/0 (socket says 172.21.15.46:52572) 2024-03-18T21:02:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd05dd9b700 1 -- 172.21.15.46:0/4201252755 learned_addr learned my addr 172.21.15.46:0/4201252755 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:02:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd05e59c700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd060100f50 0x7fd0600730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.550+0000 7fd05ed9d700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060073b30 0x7fd0601a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.551+0000 7fd05dd9b700 1 -- 172.21.15.46:0/4201252755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060073b30 msgr2=0x7fd0601a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.551+0000 7fd05dd9b700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060073b30 0x7fd0601a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.551+0000 7fd05dd9b700 1 -- 172.21.15.46:0/4201252755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd060100f50 msgr2=0x7fd0600730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.551+0000 7fd05dd9b700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd060100f50 0x7fd0600730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.551+0000 7fd05dd9b700 1 -- 172.21.15.46:0/4201252755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0540095f0 con 0x7fd060106070 2024-03-18T21:02:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.551+0000 7fd05dd9b700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd060106070 0x7fd0600735f0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fd0540059d0 tx=0x7fd054009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:02:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.551+0000 7fd04f7fe700 1 -- 172.21.15.46:0/4201252755 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05401d070 con 0x7fd060106070 2024-03-18T21:02:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.551+0000 7fd04f7fe700 1 -- 172.21.15.46:0/4201252755 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd05400fb30 con 0x7fd060106070 2024-03-18T21:02:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.551+0000 7fd064cbe700 1 -- 172.21.15.46:0/4201252755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0601a5f80 con 0x7fd060106070 2024-03-18T21:02:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.551+0000 7fd04f7fe700 1 -- 172.21.15.46:0/4201252755 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054022bf0 con 0x7fd060106070 2024-03-18T21:02:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.551+0000 7fd064cbe700 1 -- 172.21.15.46:0/4201252755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0601a6340 con 0x7fd060106070 2024-03-18T21:02:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.552+0000 7fd064cbe700 1 -- 172.21.15.46:0/4201252755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd060109340 con 0x7fd060106070 2024-03-18T21:02:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.553+0000 7fd04f7fe700 1 -- 172.21.15.46:0/4201252755 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd05400fca0 con 0x7fd060106070 2024-03-18T21:02:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.554+0000 7fd04f7fe700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0480a1290 0x7fd0480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.554+0000 7fd05e59c700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0480a1290 0x7fd0480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.554+0000 7fd04f7fe700 1 -- 172.21.15.46:0/4201252755 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8942+0+0 (secure 0 0 0) 0x7fd0540ce270 con 0x7fd060106070 2024-03-18T21:02:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.554+0000 7fd05e59c700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0480a1290 0x7fd0480a3750 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd050005950 tx=0x7fd05000b410 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:02:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.559+0000 7fd04f7fe700 1 -- 172.21.15.46:0/4201252755 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd054097550 con 0x7fd060106070 2024-03-18T21:02:01.719 INFO:journalctl@ceph.mds.e.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-e[83127]: ignoring --setuser ceph since I am not root 2024-03-18T21:02:01.719 INFO:journalctl@ceph.mds.e.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-e[83127]: ignoring --setgroup ceph since I am not root 2024-03-18T21:02:01.719 INFO:journalctl@ceph.mds.e.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-e[83127]: starting mds.e at 2024-03-18T21:02:01.720 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: ignoring --setuser ceph since I am not root 2024-03-18T21:02:01.720 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: ignoring --setgroup ceph since I am not root 2024-03-18T21:02:01.721 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-mds[82810]: -- 172.21.15.73:0/2506919561 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5555ff59a1a0 con 0x5555ff571000 2024-03-18T21:02:01.721 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: starting mds.k at 2024-03-18T21:02:01.721 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: ignoring --setuser ceph since I am not root 2024-03-18T21:02:01.721 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: ignoring --setgroup ceph since I am not root 2024-03-18T21:02:01.722 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-mds[83832]: -- 172.21.15.73:0/1495416465 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56016e8861a0 con 0x56016e85cc00 2024-03-18T21:02:01.722 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: starting mds.b at 2024-03-18T21:02:01.722 INFO:journalctl@ceph.mds.h.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-h[83470]: ignoring --setuser ceph since I am not root 2024-03-18T21:02:01.722 INFO:journalctl@ceph.mds.h.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-h[83470]: ignoring --setgroup ceph since I am not root 2024-03-18T21:02:01.723 INFO:journalctl@ceph.mds.h.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-h[83470]: starting mds.h at 2024-03-18T21:02:01.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3267885378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T21:02:01.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3242982329' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-18T21:02:01.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-mon[47384]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T21:02:01.723 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-mon[47384]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T21:02:01.724 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-mon[47384]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-03-18T21:02:01.724 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-mon[47384]: osdmap e123: 12 total, 12 up, 12 in 2024-03-18T21:02:01.724 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/3242982329' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-18T21:02:01.724 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:01 smithi073 ceph-mon[47384]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-18T21:02:01.795 INFO:journalctl@ceph.mds.a.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-a[93799]: ignoring --setuser ceph since I am not root 2024-03-18T21:02:01.796 INFO:journalctl@ceph.mds.a.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-a[93799]: ignoring --setgroup ceph since I am not root 2024-03-18T21:02:01.796 INFO:journalctl@ceph.mds.a.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-a[93799]: starting mds.a at 2024-03-18T21:02:01.796 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-g[93409]: ignoring --setuser ceph since I am not root 2024-03-18T21:02:01.796 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-g[93409]: ignoring --setgroup ceph since I am not root 2024-03-18T21:02:01.796 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-mds[93432]: -- 172.21.15.46:0/1040517672 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562df351a1a0 con 0x562df34f0800 2024-03-18T21:02:01.796 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-g[93409]: starting mds.g at 2024-03-18T21:02:01.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a[43472]: 2024-03-18T21:02:01.373+0000 7fb26bbdd700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T21:02:01.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3267885378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T21:02:01.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3242982329' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-18T21:02:01.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-mon[43495]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T21:02:01.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-mon[43495]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T21:02:01.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-mon[43495]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-03-18T21:02:01.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-mon[43495]: osdmap e123: 12 total, 12 up, 12 in 2024-03-18T21:02:01.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/3242982329' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-18T21:02:01.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:01 smithi046 ceph-mon[43495]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-18T21:02:01.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:01.826+0000 7fd064cbe700 1 -- 172.21.15.46:0/4201252755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd060074540 con 0x7fd060106070 2024-03-18T21:02:02.039 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: ignoring --setuser ceph since I am not root 2024-03-18T21:02:02.039 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: ignoring --setgroup ceph since I am not root 2024-03-18T21:02:02.039 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-mds[84998]: -- 172.21.15.155:0/1945945630 <== mon.1 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a333d2c1a0 con 0x55a333d03000 2024-03-18T21:02:02.039 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: starting mds.c at 2024-03-18T21:02:02.040 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-f[84635]: ignoring --setuser ceph since I am not root 2024-03-18T21:02:02.040 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-f[84635]: ignoring --setgroup ceph since I am not root 2024-03-18T21:02:02.040 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-f[84635]: starting mds.f at 2024-03-18T21:02:02.040 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-i[85318]: ignoring --setuser ceph since I am not root 2024-03-18T21:02:02.040 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-i[85318]: ignoring --setgroup ceph since I am not root 2024-03-18T21:02:02.040 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-i[85318]: starting mds.i at 2024-03-18T21:02:02.040 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-l[84292]: ignoring --setuser ceph since I am not root 2024-03-18T21:02:02.041 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-l[84292]: ignoring --setgroup ceph since I am not root 2024-03-18T21:02:02.041 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-l[84292]: starting mds.l at 2024-03-18T21:02:02.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3267885378' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T21:02:02.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3242982329' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-18T21:02:02.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-mon[48344]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T21:02:02.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-mon[48344]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T21:02:02.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-mon[48344]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2024-03-18T21:02:02.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-mon[48344]: osdmap e123: 12 total, 12 up, 12 in 2024-03-18T21:02:02.042 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/3242982329' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-18T21:02:02.043 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:01 smithi155 ceph-mon[48344]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-18T21:02:02.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.391+0000 7fd04f7fe700 1 -- 172.21.15.46:0/4201252755 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v113) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd05409ab70 con 0x7fd060106070 2024-03-18T21:02:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.394+0000 7fd064cbe700 1 -- 172.21.15.46:0/4201252755 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0480a1290 msgr2=0x7fd0480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.394+0000 7fd064cbe700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0480a1290 0x7fd0480a3750 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd050005950 tx=0x7fd05000b410 comp rx=0 tx=0).stop 2024-03-18T21:02:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.394+0000 7fd064cbe700 1 -- 172.21.15.46:0/4201252755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd060106070 msgr2=0x7fd0600735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.394+0000 7fd064cbe700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd060106070 0x7fd0600735f0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fd0540059d0 tx=0x7fd054009580 comp rx=0 tx=0).stop 2024-03-18T21:02:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.394+0000 7fd064cbe700 1 -- 172.21.15.46:0/4201252755 shutdown_connections 2024-03-18T21:02:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.394+0000 7fd064cbe700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fd0480a1290 0x7fd0480a3750 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.394+0000 7fd064cbe700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd060073b30 0x7fd0601a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.394+0000 7fd064cbe700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd060100f50 0x7fd0600730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.394+0000 7fd064cbe700 1 --2- 172.21.15.46:0/4201252755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd060106070 0x7fd0600735f0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.394+0000 7fd064cbe700 1 -- 172.21.15.46:0/4201252755 >> 172.21.15.46:0/4201252755 conn(0x7fd0600fc7f0 msgr2=0x7fd0600fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:02.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.394+0000 7fd064cbe700 1 -- 172.21.15.46:0/4201252755 shutdown_connections 2024-03-18T21:02:02.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.394+0000 7fd064cbe700 1 -- 172.21.15.46:0/4201252755 wait complete. 2024-03-18T21:02:02.414 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-18T21:02:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.559+0000 7f983f88a700 1 -- 172.21.15.46:0/3101796902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98380ffe50 msgr2=0x7f9838100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.559+0000 7f983f88a700 1 --2- 172.21.15.46:0/3101796902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98380ffe50 0x7f9838100230 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f9834009990 tx=0x7f9834009ca0 comp rx=0 tx=0).stop 2024-03-18T21:02:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.559+0000 7f983f88a700 1 -- 172.21.15.46:0/3101796902 shutdown_connections 2024-03-18T21:02:02.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.559+0000 7f983f88a700 1 --2- 172.21.15.46:0/3101796902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98380ffe50 0x7f9838100230 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:02.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.559+0000 7f983f88a700 1 --2- 172.21.15.46:0/3101796902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9838104f60 0x7f983810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:02.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.559+0000 7f983f88a700 1 --2- 172.21.15.46:0/3101796902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9838100800 0x7f9838104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:02.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.559+0000 7f983f88a700 1 -- 172.21.15.46:0/3101796902 >> 172.21.15.46:0/3101796902 conn(0x7f9838076d40 msgr2=0x7f9838077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:02.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.559+0000 7f983f88a700 1 -- 172.21.15.46:0/3101796902 shutdown_connections 2024-03-18T21:02:02.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.559+0000 7f983f88a700 1 -- 172.21.15.46:0/3101796902 wait complete. 2024-03-18T21:02:02.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983f88a700 1 Processor -- start 2024-03-18T21:02:02.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983f88a700 1 -- start start 2024-03-18T21:02:02.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983f88a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98380ffe50 0x7f9838195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:02.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983f88a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9838104f60 0x7f9838196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:02.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983f88a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98381968d0 0x7f98381a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:02.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983f88a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98381a5a10 con 0x7f9838104f60 2024-03-18T21:02:02.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983f88a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98381a5b50 con 0x7f98380ffe50 2024-03-18T21:02:02.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983f88a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98381a5cc0 con 0x7f98381968d0 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983ce25700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9838104f60 0x7f9838196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983ce25700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9838104f60 0x7f9838196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52600/0 (socket says 172.21.15.46:52600) 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983ce25700 1 -- 172.21.15.46:0/2500728842 learned_addr learned my addr 172.21.15.46:0/2500728842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983de27700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98381968d0 0x7f98381a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983d626700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98380ffe50 0x7f9838195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983ce25700 1 -- 172.21.15.46:0/2500728842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98381968d0 msgr2=0x7f98381a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983ce25700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98381968d0 0x7f98381a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983ce25700 1 -- 172.21.15.46:0/2500728842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98380ffe50 msgr2=0x7f9838195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983ce25700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98380ffe50 0x7f9838195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983ce25700 1 -- 172.21.15.46:0/2500728842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340095f0 con 0x7f9838104f60 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.560+0000 7f983ce25700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9838104f60 0x7f9838196390 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f982800e970 tx=0x7f982800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.561+0000 7f982e7fc700 1 -- 172.21.15.46:0/2500728842 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828009790 con 0x7f9838104f60 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.561+0000 7f982e7fc700 1 -- 172.21.15.46:0/2500728842 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9828004cf0 con 0x7f9838104f60 2024-03-18T21:02:02.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.561+0000 7f983f88a700 1 -- 172.21.15.46:0/2500728842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98381a5fa0 con 0x7f9838104f60 2024-03-18T21:02:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.561+0000 7f982e7fc700 1 -- 172.21.15.46:0/2500728842 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828005680 con 0x7f9838104f60 2024-03-18T21:02:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.561+0000 7f983f88a700 1 -- 172.21.15.46:0/2500728842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98381a6420 con 0x7f9838104f60 2024-03-18T21:02:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.562+0000 7f983f88a700 1 -- 172.21.15.46:0/2500728842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9838108180 con 0x7f9838104f60 2024-03-18T21:02:02.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.563+0000 7f982e7fc700 1 -- 172.21.15.46:0/2500728842 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9828004750 con 0x7f9838104f60 2024-03-18T21:02:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.564+0000 7f982e7fc700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f98240a1290 0x7f98240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.564+0000 7f983d626700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f98240a1290 0x7f98240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.564+0000 7f982e7fc700 1 -- 172.21.15.46:0/2500728842 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 8942+0+0 (secure 0 0 0) 0x7f9828014070 con 0x7f9838104f60 2024-03-18T21:02:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.564+0000 7f983d626700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f98240a1290 0x7f98240a3750 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f983400b2e0 tx=0x7f9834000bc0 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:02:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.568+0000 7f982e7fc700 1 -- 172.21.15.46:0/2500728842 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9828096410 con 0x7f9838104f60 2024-03-18T21:02:02.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:02.788+0000 7f983f88a700 1 -- 172.21.15.46:0/2500728842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9838068390 con 0x7f9838104f60 2024-03-18T21:02:02.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: pgmap v822: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 KiB/s wr, 13 op/s 2024-03-18T21:02:02.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/4201252755' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: Cluster is now healthy 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.46:6839/839493213,v1:172.21.15.46:6841/839493213] up:boot 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.46:6838/2625752731,v1:172.21.15.46:6840/2625752731] up:boot 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.73:6834/3971844583,v1:172.21.15.73:6836/3971844583] up:boot 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.73:6837/603700667,v1:172.21.15.73:6839/603700667] up:boot 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.73:6832/1112402549,v1:172.21.15.73:6835/1112402549] up:boot 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.73:6833/2828663317,v1:172.21.15.73:6838/2828663317] up:boot 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6835/3706508435,v1:172.21.15.155:6837/3706508435] up:boot 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6838/4268361182,v1:172.21.15.155:6839/4268361182] up:boot 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6832/325756966,v1:172.21.15.155:6834/325756966] up:boot 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: mds.? [v2:172.21.15.155:6833/3828856536,v1:172.21.15.155:6836/3828856536] up:boot 2024-03-18T21:02:02.796 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/4201252755' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-18T21:02:02.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: fsmap 12 up:standby 2024-03-18T21:02:02.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T21:02:02.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T21:02:02.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T21:02:02.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T21:02:02.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T21:02:02.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T21:02:02.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T21:02:02.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T21:02:02.797 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T21:02:02.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:02 smithi046 ceph-mon[43495]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T21:02:02.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: pgmap v822: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 KiB/s wr, 13 op/s 2024-03-18T21:02:02.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/4201252755' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T21:02:02.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T21:02:02.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T21:02:02.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: Cluster is now healthy 2024-03-18T21:02:02.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.46:6839/839493213,v1:172.21.15.46:6841/839493213] up:boot 2024-03-18T21:02:02.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.46:6838/2625752731,v1:172.21.15.46:6840/2625752731] up:boot 2024-03-18T21:02:02.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.73:6834/3971844583,v1:172.21.15.73:6836/3971844583] up:boot 2024-03-18T21:02:02.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.73:6837/603700667,v1:172.21.15.73:6839/603700667] up:boot 2024-03-18T21:02:02.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.73:6832/1112402549,v1:172.21.15.73:6835/1112402549] up:boot 2024-03-18T21:02:02.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.73:6833/2828663317,v1:172.21.15.73:6838/2828663317] up:boot 2024-03-18T21:02:02.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6835/3706508435,v1:172.21.15.155:6837/3706508435] up:boot 2024-03-18T21:02:02.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6838/4268361182,v1:172.21.15.155:6839/4268361182] up:boot 2024-03-18T21:02:02.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6832/325756966,v1:172.21.15.155:6834/325756966] up:boot 2024-03-18T21:02:02.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: mds.? [v2:172.21.15.155:6833/3828856536,v1:172.21.15.155:6836/3828856536] up:boot 2024-03-18T21:02:02.970 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/4201252755' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-18T21:02:02.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: fsmap 12 up:standby 2024-03-18T21:02:02.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T21:02:02.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T21:02:02.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T21:02:02.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T21:02:02.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T21:02:02.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T21:02:02.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T21:02:02.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T21:02:02.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T21:02:02.971 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:02 smithi073 ceph-mon[47384]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T21:02:03.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: pgmap v822: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 KiB/s wr, 13 op/s 2024-03-18T21:02:03.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/4201252755' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T21:02:03.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: Cluster is now healthy 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.46:6839/839493213,v1:172.21.15.46:6841/839493213] up:boot 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.46:6838/2625752731,v1:172.21.15.46:6840/2625752731] up:boot 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.73:6834/3971844583,v1:172.21.15.73:6836/3971844583] up:boot 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.73:6837/603700667,v1:172.21.15.73:6839/603700667] up:boot 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.73:6832/1112402549,v1:172.21.15.73:6835/1112402549] up:boot 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.73:6833/2828663317,v1:172.21.15.73:6838/2828663317] up:boot 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6835/3706508435,v1:172.21.15.155:6837/3706508435] up:boot 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6838/4268361182,v1:172.21.15.155:6839/4268361182] up:boot 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6832/325756966,v1:172.21.15.155:6834/325756966] up:boot 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: mds.? [v2:172.21.15.155:6833/3828856536,v1:172.21.15.155:6836/3828856536] up:boot 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/4201252755' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: fsmap 12 up:standby 2024-03-18T21:02:03.040 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T21:02:03.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T21:02:03.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T21:02:03.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T21:02:03.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T21:02:03.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T21:02:03.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T21:02:03.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T21:02:03.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T21:02:03.041 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:02 smithi155 ceph-mon[48344]: from='mgr.15498 172.21.15.46:0/948164533' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T21:02:03.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.578+0000 7f982e7fc700 1 -- 172.21.15.46:0/2500728842 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v124) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9828099a30 con 0x7f9838104f60 2024-03-18T21:02:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.582+0000 7f983f88a700 1 -- 172.21.15.46:0/2500728842 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f98240a1290 msgr2=0x7f98240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.582+0000 7f983f88a700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f98240a1290 0x7f98240a3750 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f983400b2e0 tx=0x7f9834000bc0 comp rx=0 tx=0).stop 2024-03-18T21:02:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.582+0000 7f983f88a700 1 -- 172.21.15.46:0/2500728842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9838104f60 msgr2=0x7f9838196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.582+0000 7f983f88a700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9838104f60 0x7f9838196390 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f982800e970 tx=0x7f982800ed30 comp rx=0 tx=0).stop 2024-03-18T21:02:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.583+0000 7f983f88a700 1 -- 172.21.15.46:0/2500728842 shutdown_connections 2024-03-18T21:02:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.583+0000 7f983f88a700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7f98240a1290 0x7f98240a3750 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f983400b2e0 tx=0x7f9834000bc0 comp rx=0 tx=0).stop 2024-03-18T21:02:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.583+0000 7f983f88a700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98381968d0 0x7f98381a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.583+0000 7f983f88a700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98380ffe50 0x7f9838195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.583+0000 7f983f88a700 1 --2- 172.21.15.46:0/2500728842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9838104f60 0x7f9838196390 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.583+0000 7f983f88a700 1 -- 172.21.15.46:0/2500728842 >> 172.21.15.46:0/2500728842 conn(0x7f9838076d40 msgr2=0x7f98380fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.583+0000 7f983f88a700 1 -- 172.21.15.46:0/2500728842 shutdown_connections 2024-03-18T21:02:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.583+0000 7f983f88a700 1 -- 172.21.15.46:0/2500728842 wait complete. 2024-03-18T21:02:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-03-18T21:02:03.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-18T21:02:03.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.768+0000 7fed177e5700 1 -- 172.21.15.46:0/39073425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10101030 msgr2=0x7fed1010d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:03.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.768+0000 7fed177e5700 1 --2- 172.21.15.46:0/39073425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10101030 0x7fed1010d210 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fed00009990 tx=0x7fed00009ca0 comp rx=0 tx=0).stop 2024-03-18T21:02:03.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.768+0000 7fed177e5700 1 -- 172.21.15.46:0/39073425 shutdown_connections 2024-03-18T21:02:03.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.768+0000 7fed177e5700 1 --2- 172.21.15.46:0/39073425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed1010d750 0x7fed1010fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:03.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.768+0000 7fed177e5700 1 --2- 172.21.15.46:0/39073425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed10100710 0x7fed10100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:03.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.768+0000 7fed177e5700 1 --2- 172.21.15.46:0/39073425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10101030 0x7fed1010d210 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:03.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.768+0000 7fed177e5700 1 -- 172.21.15.46:0/39073425 >> 172.21.15.46:0/39073425 conn(0x7fed10076d40 msgr2=0x7fed10077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:03.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.768+0000 7fed177e5700 1 -- 172.21.15.46:0/39073425 shutdown_connections 2024-03-18T21:02:03.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.768+0000 7fed177e5700 1 -- 172.21.15.46:0/39073425 wait complete. 2024-03-18T21:02:03.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.768+0000 7fed177e5700 1 Processor -- start 2024-03-18T21:02:03.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed177e5700 1 -- start start 2024-03-18T21:02:03.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed177e5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10100710 0x7fed101994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:03.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed177e5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed10101030 0x7fed10199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:03.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed177e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed1010d750 0x7fed1019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:03.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed177e5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1019e410 con 0x7fed1010d750 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed177e5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1019e580 con 0x7fed10101030 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed177e5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1019e740 con 0x7fed10100710 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed15d82700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed1010d750 0x7fed1019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed15d82700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed1010d750 0x7fed1019ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52610/0 (socket says 172.21.15.46:52610) 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed15d82700 1 -- 172.21.15.46:0/2808723838 learned_addr learned my addr 172.21.15.46:0/2808723838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed15581700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10100710 0x7fed101994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed14d80700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed10101030 0x7fed10199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed15d82700 1 -- 172.21.15.46:0/2808723838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10100710 msgr2=0x7fed101994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed15d82700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10100710 0x7fed101994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed15d82700 1 -- 172.21.15.46:0/2808723838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed10101030 msgr2=0x7fed10199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed15d82700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed10101030 0x7fed10199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.769+0000 7fed15d82700 1 -- 172.21.15.46:0/2808723838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed000095f0 con 0x7fed1010d750 2024-03-18T21:02:03.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.770+0000 7fed15d82700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed1010d750 0x7fed1019ddd0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fed08009fd0 tx=0x7fed0800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:02:03.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.770+0000 7fed067fc700 1 -- 172.21.15.46:0/2808723838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed08018070 con 0x7fed1010d750 2024-03-18T21:02:03.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.770+0000 7fed177e5700 1 -- 172.21.15.46:0/2808723838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1019e9c0 con 0x7fed1010d750 2024-03-18T21:02:03.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.770+0000 7fed067fc700 1 -- 172.21.15.46:0/2808723838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed0800f040 con 0x7fed1010d750 2024-03-18T21:02:03.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.770+0000 7fed067fc700 1 -- 172.21.15.46:0/2808723838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0800cd00 con 0x7fed1010d750 2024-03-18T21:02:03.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.770+0000 7fed177e5700 1 -- 172.21.15.46:0/2808723838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed101a6260 con 0x7fed1010d750 2024-03-18T21:02:03.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.771+0000 7fed177e5700 1 -- 172.21.15.46:0/2808723838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed10108970 con 0x7fed1010d750 2024-03-18T21:02:03.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.772+0000 7fed067fc700 1 -- 172.21.15.46:0/2808723838 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fed08004ad0 con 0x7fed1010d750 2024-03-18T21:02:03.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.773+0000 7fed067fc700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fecfc0a1290 0x7fecfc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T21:02:03.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.773+0000 7fed067fc700 1 -- 172.21.15.46:0/2808723838 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8446+0+0 (secure 0 0 0) 0x7fed080cc700 con 0x7fed1010d750 2024-03-18T21:02:03.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.773+0000 7fed15581700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fecfc0a1290 0x7fecfc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T21:02:03.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.773+0000 7fed15581700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fecfc0a1290 0x7fecfc0a3750 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009c60 tx=0x7fed0c009260 comp rx=0 tx=0).ready entity=mgr.15498 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T21:02:03.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:03.777+0000 7fed067fc700 1 -- 172.21.15.46:0/2808723838 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fed08095c30 con 0x7fed1010d750 2024-03-18T21:02:03.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:03 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2500728842' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T21:02:04.039 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:03 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2500728842' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T21:02:04.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:03 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2500728842' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T21:02:04.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.122+0000 7fed177e5700 1 -- 172.21.15.46:0/2808723838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fed10067af0 con 0x7fed1010d750 2024-03-18T21:02:04.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.583+0000 7fed067fc700 1 -- 172.21.15.46:0/2808723838 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v125) v1 ==== 169+0+0 (secure 0 0 0) 0x7fed08099250 con 0x7fed1010d750 2024-03-18T21:02:04.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.592+0000 7fed177e5700 1 -- 172.21.15.46:0/2808723838 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fecfc0a1290 msgr2=0x7fecfc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:04.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.592+0000 7fed177e5700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fecfc0a1290 0x7fecfc0a3750 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009c60 tx=0x7fed0c009260 comp rx=0 tx=0).stop 2024-03-18T21:02:04.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.592+0000 7fed177e5700 1 -- 172.21.15.46:0/2808723838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed1010d750 msgr2=0x7fed1019ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T21:02:04.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.592+0000 7fed177e5700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed1010d750 0x7fed1019ddd0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fed08009fd0 tx=0x7fed0800ef90 comp rx=0 tx=0).stop 2024-03-18T21:02:04.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.592+0000 7fed177e5700 1 -- 172.21.15.46:0/2808723838 shutdown_connections 2024-03-18T21:02:04.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.592+0000 7fed177e5700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.46:6800/805586292,v1:172.21.15.46:6801/805586292] conn(0x7fecfc0a1290 0x7fecfc0a3750 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:04.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.592+0000 7fed177e5700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed1010d750 0x7fed1019ddd0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:04.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.592+0000 7fed177e5700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed10100710 0x7fed101994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:04.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.592+0000 7fed177e5700 1 --2- 172.21.15.46:0/2808723838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed10101030 0x7fed10199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T21:02:04.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.592+0000 7fed177e5700 1 -- 172.21.15.46:0/2808723838 >> 172.21.15.46:0/2808723838 conn(0x7fed10076d40 msgr2=0x7fed1010ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T21:02:04.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.592+0000 7fed177e5700 1 -- 172.21.15.46:0/2808723838 shutdown_connections 2024-03-18T21:02:04.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-18T21:02:04.592+0000 7fed177e5700 1 -- 172.21.15.46:0/2808723838 wait complete. 2024-03-18T21:02:04.596 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-03-18T21:02:04.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T21:02:04.697 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T21:02:04.787 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T21:02:04.861 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:04 smithi155 ceph-mon[48344]: pgmap v823: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-03-18T21:02:04.861 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:04 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2500728842' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-18T21:02:04.861 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:04 smithi155 ceph-mon[48344]: osdmap e124: 12 total, 12 up, 12 in 2024-03-18T21:02:04.862 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:04 smithi155 ceph-mon[48344]: from='client.? 172.21.15.46:0/2808723838' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T21:02:04.866 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-18T21:02:04.866 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T21:02:04.894 DEBUG:teuthology.orchestra.run.smithi073:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T21:02:04.910 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T21:02:04.936 INFO:tasks.cephadm:Stopping all daemons... 2024-03-18T21:02:04.937 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-18T21:02:04.937 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.a 2024-03-18T21:02:04.964 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:04 smithi046 ceph-mon[43495]: pgmap v823: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-03-18T21:02:04.964 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:04 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2500728842' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-18T21:02:04.964 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:04 smithi046 ceph-mon[43495]: osdmap e124: 12 total, 12 up, 12 in 2024-03-18T21:02:04.964 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:04 smithi046 ceph-mon[43495]: from='client.? 172.21.15.46:0/2808723838' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T21:02:04.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:04 smithi073 ceph-mon[47384]: pgmap v823: 97 pgs: 97 active+clean; 2.4 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 13 op/s 2024-03-18T21:02:04.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:04 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2500728842' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-18T21:02:04.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:04 smithi073 ceph-mon[47384]: osdmap e124: 12 total, 12 up, 12 in 2024-03-18T21:02:04.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:04 smithi073 ceph-mon[47384]: from='client.? 172.21.15.46:0/2808723838' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T21:02:05.222 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:04 smithi046 systemd[1]: Stopping Ceph mon.a for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:05.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:05 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a[43472]: 2024-03-18T21:02:05.220+0000 7fb2713e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:02:05.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:05 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a[43472]: 2024-03-18T21:02:05.220+0000 7fb2713e8700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-18T21:02:06.795 INFO:journalctl@ceph.mon.a.smithi046.stdout:Mar 18 21:02:06 smithi046 bash[137121]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-a 2024-03-18T21:02:07.486 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.a.service' 2024-03-18T21:02:07.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:02:07.586 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-18T21:02:07.586 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-18T21:02:07.586 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.b 2024-03-18T21:02:07.892 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:07 smithi073 systemd[1]: Stopping Ceph mon.b for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:08.219 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:07 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-b[47361]: 2024-03-18T21:02:07.889+0000 7f2c31009700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:02:08.219 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:07 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-b[47361]: 2024-03-18T21:02:07.889+0000 7f2c31009700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-18T21:02:08.565 INFO:journalctl@ceph.mon.b.smithi073.stdout:Mar 18 21:02:08 smithi073 bash[92925]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-b 2024-03-18T21:02:08.571 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.b.service' 2024-03-18T21:02:08.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:02:08.615 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-18T21:02:08.615 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-18T21:02:08.615 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.c 2024-03-18T21:02:08.946 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:08 smithi155 systemd[1]: Stopping Ceph mon.c for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:09.206 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:08 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-c[48322]: 2024-03-18T21:02:08.942+0000 7fe78cf40700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:02:09.206 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:08 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-c[48322]: 2024-03-18T21:02:08.942+0000 7fe78cf40700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-18T21:02:09.539 INFO:journalctl@ceph.mon.c.smithi155.stdout:Mar 18 21:02:09 smithi155 bash[97021]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mon-c 2024-03-18T21:02:09.547 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mon.c.service' 2024-03-18T21:02:09.606 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:02:09.606 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-18T21:02:09.606 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-18T21:02:09.607 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.x 2024-03-18T21:02:09.914 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 21:02:09 smithi046 systemd[1]: Stopping Ceph mgr.x for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:09.915 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 21:02:09 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T21:02:09.910+0000 7ff5f61c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:02:09.915 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 21:02:09 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x[43744]: 2024-03-18T21:02:09.910+0000 7ff5f61c8700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-18T21:02:10.295 INFO:journalctl@ceph.mgr.x.smithi046.stdout:Mar 18 21:02:10 smithi046 bash[137262]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-x 2024-03-18T21:02:10.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.x.service' 2024-03-18T21:02:10.532 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:02:10.533 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-18T21:02:10.533 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-18T21:02:10.533 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.y 2024-03-18T21:02:10.926 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 21:02:10 smithi073 systemd[1]: Stopping Ceph mgr.y for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:11.219 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 21:02:10 smithi073 bash[93064]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-y 2024-03-18T21:02:11.219 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Mar 18 21:02:11 smithi073 systemd[1]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-18T21:02:11.314 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.y.service' 2024-03-18T21:02:11.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:02:11.352 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-18T21:02:11.352 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-18T21:02:11.353 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.z 2024-03-18T21:02:11.696 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 21:02:11 smithi155 systemd[1]: Stopping Ceph mgr.z for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:12.020 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 21:02:11 smithi155 bash[97167]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mgr-z 2024-03-18T21:02:12.020 INFO:journalctl@ceph.mgr.z.smithi155.stdout:Mar 18 21:02:11 smithi155 systemd[1]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-18T21:02:12.026 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mgr.z.service' 2024-03-18T21:02:12.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:02:12.065 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-18T21:02:12.065 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-18T21:02:12.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.0 2024-03-18T21:02:12.545 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 21:02:12 smithi046 systemd[1]: Stopping Ceph osd.0 for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:12.546 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 21:02:12 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-0[61199]: 2024-03-18T21:02:12.319+0000 7fc43b79e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:02:12.546 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 21:02:12 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-0[61199]: 2024-03-18T21:02:12.319+0000 7fc43b79e700 -1 osd.0 125 *** Got signal Terminated *** 2024-03-18T21:02:12.546 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 21:02:12 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-0[61199]: 2024-03-18T21:02:12.319+0000 7fc43b79e700 -1 osd.0 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T21:02:18.295 INFO:journalctl@ceph.osd.0.smithi046.stdout:Mar 18 21:02:18 smithi046 bash[137403]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-0 2024-03-18T21:02:19.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.0.service' 2024-03-18T21:02:19.728 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:02:19.729 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-18T21:02:19.729 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-18T21:02:19.729 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.1 2024-03-18T21:02:20.010 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 21:02:19 smithi073 systemd[1]: Stopping Ceph osd.1 for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:20.010 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 21:02:20 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-1[52371]: 2024-03-18T21:02:20.004+0000 7ff872570700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:02:20.010 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 21:02:20 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-1[52371]: 2024-03-18T21:02:20.004+0000 7ff872570700 -1 osd.1 125 *** Got signal Terminated *** 2024-03-18T21:02:20.010 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 21:02:20 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-1[52371]: 2024-03-18T21:02:20.004+0000 7ff872570700 -1 osd.1 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T21:02:26.219 INFO:journalctl@ceph.osd.1.smithi073.stdout:Mar 18 21:02:25 smithi073 bash[93204]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-1 2024-03-18T21:02:27.227 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.1.service' 2024-03-18T21:02:27.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:02:27.271 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-18T21:02:27.271 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-18T21:02:27.271 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.2 2024-03-18T21:02:27.789 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 21:02:27 smithi155 systemd[1]: Stopping Ceph osd.2 for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:27.789 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 21:02:27 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-2[54140]: 2024-03-18T21:02:27.512+0000 7fbfe0148700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:02:27.789 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 21:02:27 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-2[54140]: 2024-03-18T21:02:27.512+0000 7fbfe0148700 -1 osd.2 125 *** Got signal Terminated *** 2024-03-18T21:02:27.789 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 21:02:27 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-2[54140]: 2024-03-18T21:02:27.512+0000 7fbfe0148700 -1 osd.2 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T21:02:33.539 INFO:journalctl@ceph.osd.2.smithi155.stdout:Mar 18 21:02:33 smithi155 bash[97314]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-2 2024-03-18T21:02:34.659 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.2.service' 2024-03-18T21:02:34.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:02:34.702 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-18T21:02:34.702 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-18T21:02:34.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.3 2024-03-18T21:02:35.045 INFO:journalctl@ceph.osd.3.smithi046.stdout:Mar 18 21:02:34 smithi046 systemd[1]: Stopping Ceph osd.3 for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:35.046 INFO:journalctl@ceph.osd.3.smithi046.stdout:Mar 18 21:02:34 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-3[69745]: 2024-03-18T21:02:34.957+0000 7f97f519b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:02:35.046 INFO:journalctl@ceph.osd.3.smithi046.stdout:Mar 18 21:02:34 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-3[69745]: 2024-03-18T21:02:34.957+0000 7f97f519b700 -1 osd.3 125 *** Got signal Terminated *** 2024-03-18T21:02:35.046 INFO:journalctl@ceph.osd.3.smithi046.stdout:Mar 18 21:02:34 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-3[69745]: 2024-03-18T21:02:34.957+0000 7f97f519b700 -1 osd.3 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T21:02:38.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:38 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:38.266+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:39.539 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:39 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:39.060+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:39.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:39 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:39.234+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:40.471 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:39 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:39.976+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:40.539 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:40 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:40.109+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:40.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:40 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:40.180+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:40.546 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:40 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:40.232+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:40.966 INFO:journalctl@ceph.osd.3.smithi046.stdout:Mar 18 21:02:40 smithi046 bash[137667]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-3 2024-03-18T21:02:41.281 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:40 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:40.963+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:41.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:41 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:41.005+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:41.539 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:41 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:41.129+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:41.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:41 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:41.141+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:41.542 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:41 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:41.278+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:41.891 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.3.service' 2024-03-18T21:02:41.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:02:41.946 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-18T21:02:41.946 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-18T21:02:41.946 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.4 2024-03-18T21:02:42.287 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:41 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:41.926+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:42.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:41 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:41.987+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:42.469 INFO:journalctl@ceph.osd.4.smithi073.stdout:Mar 18 21:02:41 smithi073 systemd[1]: Stopping Ceph osd.4 for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:42.469 INFO:journalctl@ceph.osd.4.smithi073.stdout:Mar 18 21:02:42 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-4[60911]: 2024-03-18T21:02:42.176+0000 7f7521801700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:02:42.469 INFO:journalctl@ceph.osd.4.smithi073.stdout:Mar 18 21:02:42 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-4[60911]: 2024-03-18T21:02:42.176+0000 7f7521801700 -1 osd.4 125 *** Got signal Terminated *** 2024-03-18T21:02:42.469 INFO:journalctl@ceph.osd.4.smithi073.stdout:Mar 18 21:02:42 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-4[60911]: 2024-03-18T21:02:42.176+0000 7f7521801700 -1 osd.4 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T21:02:42.539 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:42 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:42.136+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:42.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:42 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:42.095+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:42.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:42 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:42.285+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:43.272 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:42 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:42.942+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:43.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:43 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:43.016+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:43.539 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:43 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:43.180+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:43.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:43 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:43.112+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:43.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:43 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:43.247+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:43.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:43 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:43.270+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:44.264 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:43 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:43.924+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:44.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:44 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:44.016+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:44.539 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:44 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:44.166+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:44.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:44 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:44.064+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:44.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:44 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:44.289+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:44.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:44 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:44.262+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:45.274 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:44 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:44.915+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:45.468 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:45 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:45.011+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:45.539 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:45 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:45.130+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:45.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:45 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:45.040+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:45.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:45 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:45.269+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:45.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:45 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:45.273+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:46.166 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:46 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:46.007+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:46.263 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:45 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:45.880+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:46.289 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:46 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:46.164+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:46.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:46 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:46.019+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:46.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:46 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:46.261+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:46.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:46 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:46.261+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:46.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:46 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:46.303+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:47.241 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:46 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:46.901+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:47.289 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:47 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:47.126+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:47.289 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:47 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:47.126+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:24.487745+0000 front 2024-03-18T21:02:24.487790+0000 (oldest deadline 2024-03-18T21:02:46.787876+0000) 2024-03-18T21:02:47.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:47 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:47.016+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:47.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:47 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:46.999+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:47.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:47 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:47.239+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:47.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:47 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:47.239+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:47.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:47 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:47.305+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:48.219 INFO:journalctl@ceph.osd.4.smithi073.stdout:Mar 18 21:02:47 smithi073 bash[93468]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-4 2024-03-18T21:02:48.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:47 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:47.966+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:48.267 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:47 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:47.929+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:48.539 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:48 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:48.091+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:48.539 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:48 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:48.091+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:24.487745+0000 front 2024-03-18T21:02:24.487790+0000 (oldest deadline 2024-03-18T21:02:46.787876+0000) 2024-03-18T21:02:48.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:48 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:48.288+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:48.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:48 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:48.047+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:48.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:48 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:48.047+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:02:48.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:48 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:48.265+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:48.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:48 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:48.265+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:48.660 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:48 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:48.228+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:02:48.998 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.4.service' 2024-03-18T21:02:49.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:02:49.039 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-18T21:02:49.039 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-18T21:02:49.039 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.5 2024-03-18T21:02:49.184 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:49 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:49.181+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:02:49.184 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:48 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:48.944+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:49.184 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:48 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:48.944+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:02:49.285 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:48 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:48.916+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:49.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:49 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:49.076+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:49.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:49 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:49.076+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:02:49.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:49 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:49.257+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:49.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:49 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:49.258+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:02:49.540 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:49 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:49.056+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:49.540 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:49 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:49.056+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:24.487745+0000 front 2024-03-18T21:02:24.487790+0000 (oldest deadline 2024-03-18T21:02:46.787876+0000) 2024-03-18T21:02:49.540 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:49 smithi155 systemd[1]: Stopping Ceph osd.5 for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:49.540 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:49 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:49.283+0000 7fa6366ff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:02:49.540 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:49 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:49.283+0000 7fa6366ff700 -1 osd.5 125 *** Got signal Terminated *** 2024-03-18T21:02:49.541 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:49 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:49.283+0000 7fa6366ff700 -1 osd.5 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T21:02:49.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:49 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:49.283+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:49.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:49 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:49.283+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:50.219 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:50 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:50.189+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:02:50.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:49 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:49.957+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:50.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:49 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:49.957+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:02:50.239 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:49 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:49.925+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:50.239 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:49 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:49.925+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:02:50.539 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:50 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:50.042+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:50.539 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:50 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:50.042+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:24.487745+0000 front 2024-03-18T21:02:24.487790+0000 (oldest deadline 2024-03-18T21:02:46.787876+0000) 2024-03-18T21:02:50.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:50 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:50.264+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:50.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:50 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:50.264+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:02:50.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:50 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:50.084+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:50.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:50 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:50.084+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:02:50.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:50 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:50.237+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:50.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:50 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:50.237+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:51.229 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:50 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:50.946+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:51.229 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:50 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:50.946+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:02:51.285 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:50 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:50.994+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:51.285 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:50 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:50.994+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:24.487745+0000 front 2024-03-18T21:02:24.487790+0000 (oldest deadline 2024-03-18T21:02:46.787876+0000) 2024-03-18T21:02:51.286 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:51 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:51.119+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:51.286 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:51 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:51.119+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:02:51.469 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:51 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:51.230+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:02:51.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:50 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:50.988+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:51.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:50 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:50.988+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:02:51.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:51 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:51.284+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:51.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:51 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:51.284+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:02:51.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:51 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:51.227+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:51.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:51 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:51.227+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:52.220 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:51 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:51.963+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:52.220 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:51 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:51.963+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:02:52.248 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:51 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:51.960+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:52.248 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:51 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:51.960+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:02:52.289 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:52 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:52.001+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:52.289 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:52 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:52.001+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:24.487745+0000 front 2024-03-18T21:02:24.487790+0000 (oldest deadline 2024-03-18T21:02:46.787876+0000) 2024-03-18T21:02:52.290 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:52 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:52.235+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:52.290 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:52 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:52.235+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:02:52.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:52 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:52.117+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:52.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:52 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:52.117+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:02:52.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:52 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:52.246+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:52.546 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:52 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:52.246+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:52.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:52 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:52.271+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:02:53.234 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:52 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:52.948+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:53.234 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:52 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:52.948+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:02:53.289 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:53 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:53.193+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:53.289 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:53 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:53.193+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:02:53.289 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:53 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:53.030+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:53.289 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:53 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:53.030+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:24.487745+0000 front 2024-03-18T21:02:24.487790+0000 (oldest deadline 2024-03-18T21:02:46.787876+0000) 2024-03-18T21:02:53.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:53 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:53.106+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:53.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:53 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:53.106+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:02:53.469 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:53 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:53.224+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:02:53.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:52 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:52.976+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:53.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:52 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:52.976+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:02:53.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:53 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:53.232+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:53.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:53 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:53.232+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:54.218 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:53 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:53.951+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:54.218 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:53 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:53.951+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:02:54.249 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:53 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:53.970+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:54.249 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:53 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:53.970+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:02:54.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:54 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:54.163+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:54.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:54 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:54.163+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:02:54.540 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:54 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:54.041+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.084870+0000 front 2024-03-18T21:02:17.084763+0000 (oldest deadline 2024-03-18T21:02:38.785011+0000) 2024-03-18T21:02:54.540 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:54 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:54.041+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:24.487745+0000 front 2024-03-18T21:02:24.487790+0000 (oldest deadline 2024-03-18T21:02:46.787876+0000) 2024-03-18T21:02:54.540 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:54 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5[62581]: 2024-03-18T21:02:54.041+0000 7fa631cf3700 -1 osd.5 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:27.789303+0000 front 2024-03-18T21:02:27.789318+0000 (oldest deadline 2024-03-18T21:02:53.689582+0000) 2024-03-18T21:02:54.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:54 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:54.104+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:54.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:54 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:54.104+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:02:54.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:54 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:54.216+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:54.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:54 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:54.216+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:54.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:54 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:54.216+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:27.430614+0000 front 2024-03-18T21:02:27.430643+0000 (oldest deadline 2024-03-18T21:02:53.330702+0000) 2024-03-18T21:02:54.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:54 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:54.247+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:02:55.262 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:55 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:55.003+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:55.262 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:55 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:55.003+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:02:55.262 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:55 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:55.003+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:02:55.289 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 18 21:02:55 smithi155 bash[97568]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-5 2024-03-18T21:02:55.290 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:55 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:55.180+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:55.290 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:55 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:55.180+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:02:55.290 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:55 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:55.180+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:02:55.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:55 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:55.140+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:55.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:55 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:55.140+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:02:55.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:54 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:54.984+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:55.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:54 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:54.984+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:02:55.295 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:55 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:55.184+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:55.295 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:55 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:55.184+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:55.295 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:55 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:55.184+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:27.430614+0000 front 2024-03-18T21:02:27.430643+0000 (oldest deadline 2024-03-18T21:02:53.330702+0000) 2024-03-18T21:02:55.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:55 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:55.260+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:02:56.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:56 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:56.123+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:56.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:56 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:56.123+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:02:56.290 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:56 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:56.208+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:56.290 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:56 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:56.208+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:02:56.290 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:56 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:56.208+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:02:56.290 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:56 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:56.038+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:56.290 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:56 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:56.038+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:02:56.290 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:56 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:56.038+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:02:56.295 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:56 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:56.177+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:56.295 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:56 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:56.177+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:56.295 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:56 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:56.177+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:27.430614+0000 front 2024-03-18T21:02:27.430643+0000 (oldest deadline 2024-03-18T21:02:53.330702+0000) 2024-03-18T21:02:56.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:55 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:55.984+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:56.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:55 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:55.984+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:02:56.330 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.5.service' 2024-03-18T21:02:56.371 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:02:56.371 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-18T21:02:56.371 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-18T21:02:56.371 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.6 2024-03-18T21:02:56.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:56 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:56.288+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:02:56.758 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:56 smithi046 systemd[1]: Stopping Ceph osd.6 for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:02:57.045 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:56 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:56.756+0000 7f3017d4f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:02:57.046 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:56 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:56.756+0000 7f3017d4f700 -1 osd.6 125 *** Got signal Terminated *** 2024-03-18T21:02:57.046 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:56 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:56.756+0000 7f3017d4f700 -1 osd.6 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T21:02:57.046 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:56 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:56.948+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:57.046 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:56 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:56.948+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:02:57.046 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:56 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:56.948+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:02:57.469 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:57 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:57.299+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:02:57.469 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:57 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:57.299+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.344816+0000 front 2024-03-18T21:02:31.344937+0000 (oldest deadline 2024-03-18T21:02:56.644827+0000) 2024-03-18T21:02:57.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:57 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:57.058+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:57.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:57 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:57.058+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:02:57.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:57 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:57.058+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:02:57.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:57 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:57.082+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:57.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:57 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:57.082+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:02:57.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:57 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:57.082+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:02:57.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:57 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:57.237+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:57.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:57 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:57.237+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:02:57.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:57 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:57.237+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:02:57.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:57 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:57.206+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:57.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:57 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:57.206+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:57.545 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:57 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:57.206+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:27.430614+0000 front 2024-03-18T21:02:27.430643+0000 (oldest deadline 2024-03-18T21:02:53.330702+0000) 2024-03-18T21:02:58.295 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:58 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:58.162+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:58.295 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:58 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:58.162+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:58.296 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:58 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:58.162+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:27.430614+0000 front 2024-03-18T21:02:27.430643+0000 (oldest deadline 2024-03-18T21:02:53.330702+0000) 2024-03-18T21:02:58.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:57 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:57.994+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:58.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:57 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:57.994+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:02:58.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:57 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:57.994+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:02:58.325 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:58 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:58.052+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:58.325 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:58 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:58.052+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:02:58.325 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:58 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:58.052+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:02:58.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:58 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:58.238+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:58.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:58 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:58.238+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:02:58.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:58 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:58.238+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:02:58.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:58 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:58.053+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:58.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:58 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:58.053+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:02:58.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:58 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:58.053+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:02:58.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:58 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:58.324+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:02:58.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:58 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:58.324+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.344816+0000 front 2024-03-18T21:02:31.344937+0000 (oldest deadline 2024-03-18T21:02:56.644827+0000) 2024-03-18T21:02:59.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:59 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:58.999+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:02:59.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:59 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:58.999+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:02:59.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:02:59 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:02:58.999+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:02:59.297 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:59 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:59.172+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:02:59.297 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:59 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:59.172+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:02:59.297 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:02:59 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:02:59.172+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:27.430614+0000 front 2024-03-18T21:02:27.430643+0000 (oldest deadline 2024-03-18T21:02:53.330702+0000) 2024-03-18T21:02:59.469 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:59 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:59.298+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:02:59.469 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:02:59 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:02:59.298+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.344816+0000 front 2024-03-18T21:02:31.344937+0000 (oldest deadline 2024-03-18T21:02:56.644827+0000) 2024-03-18T21:02:59.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:59 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:59.058+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:02:59.470 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:59 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:59.058+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:02:59.470 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:02:59 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:02:59.058+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:02:59.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:59 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:59.213+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:02:59.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:59 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:59.213+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:02:59.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:02:59 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:02:59.213+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:02:59.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:59 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:59.044+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:02:59.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:59 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:59.044+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:02:59.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:02:59 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:02:59.044+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:00.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:00.042+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:00.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:00.042+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:00.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:00.042+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:00.296 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:03:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:03:00.130+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:03:00.296 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:03:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:03:00.130+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:03:00.296 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:03:00 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:03:00.130+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:27.430614+0000 front 2024-03-18T21:02:27.430643+0000 (oldest deadline 2024-03-18T21:02:53.330702+0000) 2024-03-18T21:03:00.327 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:00 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:00.051+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:00.328 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:00 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:00.051+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:00.328 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:00 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:00.051+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:00.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:00.202+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:00.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:00.202+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:00.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:00.202+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:00.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:00.093+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:00.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:00.093+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:00.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:00 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:00.093+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:00.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:00 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:00.323+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:03:00.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:00 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:00.323+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.344816+0000 front 2024-03-18T21:02:31.344937+0000 (oldest deadline 2024-03-18T21:02:56.644827+0000) 2024-03-18T21:03:01.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:00.999+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:01.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:00.999+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:01.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:00.999+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:01.296 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:03:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:03:01.137+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.427634+0000 front 2024-03-18T21:02:16.427655+0000 (oldest deadline 2024-03-18T21:02:38.127984+0000) 2024-03-18T21:03:01.296 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:03:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:03:01.137+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:20.429351+0000 front 2024-03-18T21:02:20.429461+0000 (oldest deadline 2024-03-18T21:02:45.729473+0000) 2024-03-18T21:03:01.296 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:03:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6[77731]: 2024-03-18T21:03:01.137+0000 7f3014b46700 -1 osd.6 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:27.430614+0000 front 2024-03-18T21:02:27.430643+0000 (oldest deadline 2024-03-18T21:02:53.330702+0000) 2024-03-18T21:03:01.366 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:01.016+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:01.366 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:01.016+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:01.366 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:01.016+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:01.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:01.103+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:01.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:01.103+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:01.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:01.103+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:01.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:01.223+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:01.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:01.223+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:01.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:01.223+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:01.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:01 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:01.223+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:01.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:01.364+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:03:01.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:01 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:01.364+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.344816+0000 front 2024-03-18T21:02:31.344937+0000 (oldest deadline 2024-03-18T21:02:56.644827+0000) 2024-03-18T21:03:02.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:01.981+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:02.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:01.981+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:02.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:01 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:01.981+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:02.349 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:02 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:02.061+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:02.349 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:02 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:02.061+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:02.349 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:02 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:02.061+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:02.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:02 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:02.123+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:02.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:02 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:02.123+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:02.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:02 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:02.123+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:02.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:02 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:02.202+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:02.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:02 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:02.202+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:02.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:02 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:02.202+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:02.540 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:02 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:02.202+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:02.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:02 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:02.346+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:03:02.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:02 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:02.346+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.344816+0000 front 2024-03-18T21:02:31.344937+0000 (oldest deadline 2024-03-18T21:02:56.644827+0000) 2024-03-18T21:03:02.796 INFO:journalctl@ceph.osd.6.smithi046.stdout:Mar 18 21:03:02 smithi046 bash[137927]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-6 2024-03-18T21:03:03.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:03 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:03.023+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:03.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:03 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:03.023+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:03.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:03 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:03.023+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:03.369 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:03 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:03.050+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:03.369 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:03 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:03.050+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:03.369 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:03 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:03.050+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:03.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:03 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:03.168+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:03.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:03 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:03.168+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:03.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:03 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:03.168+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:03.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:03 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:03.168+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:03.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:03 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:03.106+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:03.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:03 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:03.106+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:03.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:03 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:03.106+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:03.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:03 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:03.106+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:03.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.6.service' 2024-03-18T21:03:03.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:03 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:03.367+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:03:03.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:03 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:03.367+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.344816+0000 front 2024-03-18T21:02:31.344937+0000 (oldest deadline 2024-03-18T21:02:56.644827+0000) 2024-03-18T21:03:03.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:03:03.725 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-18T21:03:03.725 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-18T21:03:03.726 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.7 2024-03-18T21:03:04.024 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:03 smithi073 systemd[1]: Stopping Ceph osd.7 for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:03:04.024 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:03 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:03.955+0000 7fd7b9c58700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:03:04.024 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:03 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:03.955+0000 7fd7b9c58700 -1 osd.7 125 *** Got signal Terminated *** 2024-03-18T21:03:04.024 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:03 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:03.955+0000 7fd7b9c58700 -1 osd.7 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T21:03:04.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:03 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:03.992+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:04.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:03 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:03.992+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:04.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:03 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:03.992+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:04.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:03 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:03.992+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:04.328 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:04 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:04.021+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:04.329 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:04 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:04.021+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:04.329 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:04 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:04.021+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:04.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:04.208+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:04.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:04.208+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:04.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:04.208+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:04.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:04.208+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:04.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:04.097+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:04.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:04.097+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:04.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:04.097+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:04.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:04 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:04.097+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:04.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:04 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:04.326+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:03:04.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:04 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:04.326+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.344816+0000 front 2024-03-18T21:02:31.344937+0000 (oldest deadline 2024-03-18T21:02:56.644827+0000) 2024-03-18T21:03:04.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:04 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:04.326+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:39.446526+0000 front 2024-03-18T21:02:39.446563+0000 (oldest deadline 2024-03-18T21:03:04.146625+0000) 2024-03-18T21:03:05.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:04 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:04.982+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:05.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:04 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:04.982+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:05.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:04 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:04.982+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:05.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:04 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:04.982+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:05.297 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:05.008+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:05.297 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:05.008+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:05.297 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:05.008+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:05.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:05 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:05.215+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:05.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:05 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:05.215+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:05.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:05 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:05.215+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:05.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:05 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:05.215+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:05.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:05 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:05.051+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:05.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:05 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:05.051+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:05.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:05 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:05.051+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:05.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:05 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:05.051+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:05.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:05.295+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:03:05.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:05.295+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.344816+0000 front 2024-03-18T21:02:31.344937+0000 (oldest deadline 2024-03-18T21:02:56.644827+0000) 2024-03-18T21:03:05.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:05.295+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:39.446526+0000 front 2024-03-18T21:02:39.446563+0000 (oldest deadline 2024-03-18T21:03:04.146625+0000) 2024-03-18T21:03:06.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:05 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:05.990+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:06.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:05 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:05.990+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:06.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:05 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:05.990+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:06.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:05 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:05.990+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:06.331 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:05.981+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:06.331 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:05.981+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:06.331 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:05.981+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:06.331 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:05 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:05.981+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:06.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:06 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:06.263+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:06.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:06 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:06.263+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:06.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:06 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:06.263+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:06.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:06 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:06.263+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:06.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:06 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:06.055+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:06.539 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:06 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:06.055+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:06.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:06 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:06.055+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:06.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:06 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:06.055+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:06.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:06 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:06.329+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:03:06.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:06 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:06.329+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.344816+0000 front 2024-03-18T21:02:31.344937+0000 (oldest deadline 2024-03-18T21:02:56.644827+0000) 2024-03-18T21:03:06.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:06 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:06.329+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:39.446526+0000 front 2024-03-18T21:02:39.446563+0000 (oldest deadline 2024-03-18T21:03:04.146625+0000) 2024-03-18T21:03:07.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:06 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:06.962+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:07.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:06 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:06.962+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:07.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:06 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:06.962+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:07.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:06 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:06.962+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:07.286 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:07 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:07.010+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:07.286 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:07 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:07.010+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:07.286 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:07 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:07.010+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:07.286 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:07 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:07.010+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:07.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:06 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:06.955+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:07.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:06 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:06.955+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:07.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:06 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:06.955+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:07.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:06 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:06.955+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:07.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:07 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:07.284+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:07.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:07 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:07.284+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:07.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:07 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:07.284+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:07.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:07 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:07.284+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:07.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:07 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:07.356+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:03:07.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:07 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:07.356+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.344816+0000 front 2024-03-18T21:02:31.344937+0000 (oldest deadline 2024-03-18T21:02:56.644827+0000) 2024-03-18T21:03:07.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:07 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:07.356+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:39.446526+0000 front 2024-03-18T21:02:39.446563+0000 (oldest deadline 2024-03-18T21:03:04.146625+0000) 2024-03-18T21:03:08.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:07.963+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:08.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:07.963+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:08.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:07.963+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:08.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:07 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:07.963+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:08.378 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:07 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:07.988+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:08.378 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:07 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:07.988+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:08.378 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:07 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:07.988+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:08.378 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:07 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:07.988+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:08.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:08 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:08.288+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:08.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:08 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:08.288+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:08.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:08 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:08.288+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:08.539 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:08 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:08.288+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:08.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:08 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:08.045+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:08.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:08 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:08.045+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:08.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:08 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:08.045+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:08.540 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:08 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:08.045+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:08.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:08 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:08.376+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:22.042700+0000 front 2024-03-18T21:02:22.042727+0000 (oldest deadline 2024-03-18T21:02:47.942874+0000) 2024-03-18T21:03:08.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:08 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:08.376+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.344816+0000 front 2024-03-18T21:02:31.344937+0000 (oldest deadline 2024-03-18T21:02:56.644827+0000) 2024-03-18T21:03:08.719 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:08 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7[68877]: 2024-03-18T21:03:08.376+0000 7fd7b2a47700 -1 osd.7 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:39.446526+0000 front 2024-03-18T21:02:39.446563+0000 (oldest deadline 2024-03-18T21:03:04.146625+0000) 2024-03-18T21:03:09.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:08 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:08.997+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:09.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:08 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:08.997+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:09.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:08 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:08.997+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:09.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:08 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:08.997+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:09.338 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:09 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:09.055+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:09.339 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:09 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:09.055+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:09.339 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:09 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:09.055+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:09.339 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:09 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:09.055+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:09.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:08 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:08.966+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:09.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:08 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:08.966+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:09.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:08 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:08.966+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:09.470 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:08 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:08.966+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:09.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:09 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:09.336+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:09.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:09 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:09.336+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:09.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:09 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:09.336+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:09.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:09 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:09.336+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:09.969 INFO:journalctl@ceph.osd.7.smithi073.stdout:Mar 18 21:03:09 smithi073 bash[93725]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-7 2024-03-18T21:03:10.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:10 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:10.030+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:10.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:10 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:10.030+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:10.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:10 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:10.030+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:10.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:10 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:10.030+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:10.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:10 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:10.030+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:10.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:09 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:09.972+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:10.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:09 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:09.972+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:10.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:09 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:09.972+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:10.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:09 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:09.972+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:10.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:10 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:10.009+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:10.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:10 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:10.009+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:10.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:10 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:10.009+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:10.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:10 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:10.009+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:10.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:10 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:10.322+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:10.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:10 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:10.322+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:10.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:10 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:10.322+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:10.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:10 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:10.322+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:10.878 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.7.service' 2024-03-18T21:03:10.920 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:03:10.920 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-18T21:03:10.920 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-18T21:03:10.921 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.8 2024-03-18T21:03:11.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:11 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:11.005+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:11.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:11 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:11.005+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:11.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:11 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:11.005+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:11.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:11 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:11.005+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:11.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:11 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:11.005+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:11.289 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:10 smithi155 systemd[1]: Stopping Ceph osd.8 for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:03:11.289 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:11.155+0000 7f0f858f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:03:11.289 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:11.155+0000 7f0f858f4700 -1 osd.8 125 *** Got signal Terminated *** 2024-03-18T21:03:11.290 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:11.155+0000 7f0f858f4700 -1 osd.8 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T21:03:11.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:10.999+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:11.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:10.999+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:11.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:10.999+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:11.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:10.999+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:11.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:10.999+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:11.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:10 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:10.983+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:11.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:10 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:10.983+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:11.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:10 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:10.983+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:11.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:10 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:10.983+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:11.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:11.355+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:11.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:11.355+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:11.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:11.355+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:11.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:11 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:11.355+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:12.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:12.005+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:12.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:12.005+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:12.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:12.005+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:12.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:12.005+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:12.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:12.005+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:12.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:11 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:11.974+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:12.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:11 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:11.974+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:12.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:11 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:11.974+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:12.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:11 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:11.974+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:12.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:11 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:11.974+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.942587+0000 front 2024-03-18T21:02:45.942471+0000 (oldest deadline 2024-03-18T21:03:11.242455+0000) 2024-03-18T21:03:12.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:11 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:11.991+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:12.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:11 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:11.991+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:12.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:11 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:11.991+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:12.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:11 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:11.991+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:12.470 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:11 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:11.991+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:12.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:12.320+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:12.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:12.321+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:12.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:12.321+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:12.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:12.321+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:13.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:12 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:12.945+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:13.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:12 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:12.945+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:13.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:12 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:12.945+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:13.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:12 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:12.945+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:13.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:12 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:12.945+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:13.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:12.955+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:13.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:12.955+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:13.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:12.955+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:13.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:12.956+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:13.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:12 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:12.956+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:13.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:12 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:12.948+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:13.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:12 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:12.948+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:13.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:12 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:12.948+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:13.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:12 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:12.948+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:13.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:12 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:12.948+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.942587+0000 front 2024-03-18T21:02:45.942471+0000 (oldest deadline 2024-03-18T21:03:11.242455+0000) 2024-03-18T21:03:13.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:13.305+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:13.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:13.305+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:13.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:13.305+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:13.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:13.305+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:14.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:13 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:13.945+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:14.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:13 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:13.945+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:14.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:13 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:13.945+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:14.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:13 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:13.945+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:14.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:13 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:13.945+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:14.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:13.991+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:14.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:13.991+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:14.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:13.991+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:14.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:13.991+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:14.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:13 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:13.991+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:14.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:13 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:13.970+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:14.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:13 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:13.970+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:14.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:13 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:13.970+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:14.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:13 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:13.970+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:14.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:13 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:13.970+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.942587+0000 front 2024-03-18T21:02:45.942471+0000 (oldest deadline 2024-03-18T21:03:11.242455+0000) 2024-03-18T21:03:14.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:14 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:14.349+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:14.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:14 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:14.349+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:14.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:14 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:14.349+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:14.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:14 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:14.349+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:15.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:14 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:14.976+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:15.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:14 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:14.976+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:15.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:14 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:14.976+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:15.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:14 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:14.976+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:15.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:14 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:14.976+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:15.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:14 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:14.993+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:15.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:14 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:14.993+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:15.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:14 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:14.993+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:15.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:14 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:14.993+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:15.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:14 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:14.993+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.942587+0000 front 2024-03-18T21:02:45.942471+0000 (oldest deadline 2024-03-18T21:03:11.242455+0000) 2024-03-18T21:03:15.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:14 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:14.993+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:15.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:14 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:14.993+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:15.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:14 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:14.993+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:15.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:14 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:14.993+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:15.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:14 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:14.993+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:15.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:15 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:15.335+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.609782+0000 front 2024-03-18T21:02:16.609713+0000 (oldest deadline 2024-03-18T21:02:43.010412+0000) 2024-03-18T21:03:15.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:15 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:15.335+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.010996+0000 front 2024-03-18T21:02:23.011063+0000 (oldest deadline 2024-03-18T21:02:48.911153+0000) 2024-03-18T21:03:15.789 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:15 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:15.335+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.812231+0000 front 2024-03-18T21:02:31.812251+0000 (oldest deadline 2024-03-18T21:02:54.712370+0000) 2024-03-18T21:03:15.790 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:15 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8[70704]: 2024-03-18T21:03:15.335+0000 7f0f7f6e5700 -1 osd.8 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:37.613542+0000 front 2024-03-18T21:02:37.613477+0000 (oldest deadline 2024-03-18T21:03:00.513593+0000) 2024-03-18T21:03:16.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:15 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:15.957+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:16.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:15 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:15.957+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:16.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:15 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:15.957+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:16.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:15 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:15.957+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:16.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:15 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:15.957+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:16.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:16 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:16.031+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:16.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:16 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:16.031+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:16.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:16 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:16.031+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:16.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:16 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:16.031+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:16.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:16 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:16.031+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.942587+0000 front 2024-03-18T21:02:45.942471+0000 (oldest deadline 2024-03-18T21:03:11.242455+0000) 2024-03-18T21:03:16.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:16 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:16.002+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:16.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:16 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:16.002+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:16.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:16 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:16.002+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:16.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:16 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:16.002+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:16.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:16 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:16.002+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:17.289 INFO:journalctl@ceph.osd.8.smithi155.stdout:Mar 18 21:03:16 smithi155 bash[97833]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-8 2024-03-18T21:03:17.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:16 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:16.952+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:17.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:16 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:16.952+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:17.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:16 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:16.952+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:17.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:16 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:16.952+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:17.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:16 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:16.952+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:17.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:17 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:17.021+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:17.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:17 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:17.021+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:17.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:17 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:17.021+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:17.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:17 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:17.021+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:17.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:17 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:17.021+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.942587+0000 front 2024-03-18T21:02:45.942471+0000 (oldest deadline 2024-03-18T21:03:11.242455+0000) 2024-03-18T21:03:17.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:17 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:17.014+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:17.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:17 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:17.014+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:17.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:17 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:17.014+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:17.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:17 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:17.014+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:17.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:17 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:17.014+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:18.039 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:17 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:17.980+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:18.039 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:17 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:17.980+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:18.039 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:17 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:17.980+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:18.039 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:17 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:17.980+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:18.040 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:17 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:17.980+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:18.102 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.8.service' 2024-03-18T21:03:18.143 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:03:18.143 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-18T21:03:18.144 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-18T21:03:18.144 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.9 2024-03-18T21:03:18.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:18.019+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:18.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:18.019+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:18.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:18.019+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:18.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:18.019+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:18.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:18.019+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.942587+0000 front 2024-03-18T21:02:45.942471+0000 (oldest deadline 2024-03-18T21:03:11.242455+0000) 2024-03-18T21:03:18.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:18 smithi046 systemd[1]: Stopping Ceph osd.9 for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:03:18.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:18 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:18.011+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:18.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:18 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:18.011+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:18.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:18 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:18.011+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:18.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:18 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:18.011+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:18.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:18 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:18.011+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:18.795 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:18.402+0000 7f4cf7167700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:03:18.795 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:18.402+0000 7f4cf7167700 -1 osd.9 125 *** Got signal Terminated *** 2024-03-18T21:03:18.795 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:18 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:18.402+0000 7f4cf7167700 -1 osd.9 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T21:03:19.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:18 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:18.989+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:19.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:18 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:18.989+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:19.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:18 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:18.989+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:19.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:18 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:18.989+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:19.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:18 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:18.989+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:19.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:18 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:18.989+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:19.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:19 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:19.010+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:19.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:19 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:19.010+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:19.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:19 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:19.010+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:19.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:19 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:19.010+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:19.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:19 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:19.010+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.942587+0000 front 2024-03-18T21:02:45.942471+0000 (oldest deadline 2024-03-18T21:03:11.242455+0000) 2024-03-18T21:03:19.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:19 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:19.026+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:19.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:19 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:19.026+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:19.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:19 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:19.026+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:19.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:19 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:19.026+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:19.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:19 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:19.026+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:20.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:19 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:19.948+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:20.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:19 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:19.948+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:20.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:19 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:19.948+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:20.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:19 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:19.948+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:20.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:19 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:19.948+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:20.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:19 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:19.948+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:20.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:20 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:20.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:20.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:20 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:20.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:20.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:20 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:20.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:20.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:20 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:20.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:20.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:20 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:20.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.942587+0000 front 2024-03-18T21:02:45.942471+0000 (oldest deadline 2024-03-18T21:03:11.242455+0000) 2024-03-18T21:03:20.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:20 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:20.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:54.043908+0000 front 2024-03-18T21:02:54.044016+0000 (oldest deadline 2024-03-18T21:03:19.343955+0000) 2024-03-18T21:03:20.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:20 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:20.050+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:20.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:20 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:20.050+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:20.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:20 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:20.050+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:20.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:20 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:20.050+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:20.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:20 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:20.050+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:21.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:20 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:20.986+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:21.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:20 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:20.986+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:21.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:20 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:20.986+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:21.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:20 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:20.986+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:21.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:20 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:20.986+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:21.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:20 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:20.986+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:21.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:21 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:21.032+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:21.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:21 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:21.032+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:21.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:21 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:21.032+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:21.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:21 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:21.032+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:21.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:21 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:21.032+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:21.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:21 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:21.032+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:50.293288+0000 front 2024-03-18T21:02:50.293274+0000 (oldest deadline 2024-03-18T21:03:20.194356+0000) 2024-03-18T21:03:21.545 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:21 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:21.064+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:21.545 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:21 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:21.064+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:21.545 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:21 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:21.064+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:21.545 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:21 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:21.064+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:21.546 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:21 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:21.064+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.942587+0000 front 2024-03-18T21:02:45.942471+0000 (oldest deadline 2024-03-18T21:03:11.242455+0000) 2024-03-18T21:03:21.546 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:21 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:21.064+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:54.043908+0000 front 2024-03-18T21:02:54.044016+0000 (oldest deadline 2024-03-18T21:03:19.343955+0000) 2024-03-18T21:03:22.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:21 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:21.979+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:22.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:21 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:21.979+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:22.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:21 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:21.979+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:22.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:21 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:21.979+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:22.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:21 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:21.979+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:22.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:21 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:21.979+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:22.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:22 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:22.021+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:22.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:22 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:22.021+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:22.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:22 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:22.021+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:22.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:22 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:22.021+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:22.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:22 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:22.021+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.942587+0000 front 2024-03-18T21:02:45.942471+0000 (oldest deadline 2024-03-18T21:03:11.242455+0000) 2024-03-18T21:03:22.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:22 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:22.021+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:54.043908+0000 front 2024-03-18T21:02:54.044016+0000 (oldest deadline 2024-03-18T21:03:19.343955+0000) 2024-03-18T21:03:22.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:22 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:22.036+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:22.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:22 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:22.036+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:22.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:22 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:22.036+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:22.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:22 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:22.036+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:22.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:22 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:22.036+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:22.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:22 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:22.036+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:50.293288+0000 front 2024-03-18T21:02:50.293274+0000 (oldest deadline 2024-03-18T21:03:20.194356+0000) 2024-03-18T21:03:23.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:22 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:22.931+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:23.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:22 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:22.931+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:23.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:22 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:22.931+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:23.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:22 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:22.931+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:23.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:22 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:22.931+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:23.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:22 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:22.931+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:23.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:23 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:23.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:16.335593+0000 front 2024-03-18T21:02:16.335794+0000 (oldest deadline 2024-03-18T21:02:40.435761+0000) 2024-03-18T21:03:23.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:23 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:23.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.937342+0000 front 2024-03-18T21:02:23.937110+0000 (oldest deadline 2024-03-18T21:02:49.837154+0000) 2024-03-18T21:03:23.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:23 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:23.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.538653+0000 front 2024-03-18T21:02:31.538689+0000 (oldest deadline 2024-03-18T21:02:56.238642+0000) 2024-03-18T21:03:23.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:23 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:23.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.440574+0000 front 2024-03-18T21:02:38.440598+0000 (oldest deadline 2024-03-18T21:03:03.641403+0000) 2024-03-18T21:03:23.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:23 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:23.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.942587+0000 front 2024-03-18T21:02:45.942471+0000 (oldest deadline 2024-03-18T21:03:11.242455+0000) 2024-03-18T21:03:23.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:23 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:23.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:54.043908+0000 front 2024-03-18T21:02:54.044016+0000 (oldest deadline 2024-03-18T21:03:19.343955+0000) 2024-03-18T21:03:23.296 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:23 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9[85899]: 2024-03-18T21:03:23.014+0000 7f4cf0f58700 -1 osd.9 125 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-03-18T21:03:00.444849+0000 front 2024-03-18T21:03:00.444905+0000 (oldest deadline 2024-03-18T21:03:22.744984+0000) 2024-03-18T21:03:23.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:23 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:23.014+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:23.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:23 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:23.014+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:23.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:23 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:23.014+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:23.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:23 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:23.014+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:23.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:23 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:23.014+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:23.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:23 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:23.014+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:50.293288+0000 front 2024-03-18T21:02:50.293274+0000 (oldest deadline 2024-03-18T21:03:20.194356+0000) 2024-03-18T21:03:24.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:23 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:23.905+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:24.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:23 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:23.905+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:24.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:23 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:23.905+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:24.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:23 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:23.905+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:24.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:23 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:23.905+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:24.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:23 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:23.905+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:24.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:24 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:24.018+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:24.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:24 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:24.018+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:24.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:24 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:24.018+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:24.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:24 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:24.018+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:24.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:24 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:24.018+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:24.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:24 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:24.018+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:50.293288+0000 front 2024-03-18T21:02:50.293274+0000 (oldest deadline 2024-03-18T21:03:20.194356+0000) 2024-03-18T21:03:24.545 INFO:journalctl@ceph.osd.9.smithi046.stdout:Mar 18 21:03:24 smithi046 bash[138180]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-9 2024-03-18T21:03:25.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:24 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:24.935+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:25.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:24 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:24.935+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:25.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:24 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:24.935+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:25.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:24 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:24.935+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:25.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:24 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:24.935+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:25.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:24 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:24.935+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:25.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.9.service' 2024-03-18T21:03:25.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.012+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:25.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.012+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:25.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.012+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:25.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.012+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:25.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.012+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:25.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.012+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:50.293288+0000 front 2024-03-18T21:02:50.293274+0000 (oldest deadline 2024-03-18T21:03:20.194356+0000) 2024-03-18T21:03:25.504 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:03:25.504 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-18T21:03:25.504 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-18T21:03:25.504 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.10 2024-03-18T21:03:25.969 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 systemd[1]: Stopping Ceph osd.10 for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:03:25.970 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.735+0000 7f85a2ab2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:03:25.970 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.735+0000 7f85a2ab2700 -1 osd.10 125 *** Got signal Terminated *** 2024-03-18T21:03:25.970 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.735+0000 7f85a2ab2700 -1 osd.10 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T21:03:26.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:25 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:25.913+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:26.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:25 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:25.913+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:26.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:25 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:25.913+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:26.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:25 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:25.913+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:26.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:25 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:25.913+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:26.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:25 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:25.913+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:26.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.970+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:26.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.970+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:26.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.970+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:26.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.970+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:26.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.970+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:26.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.970+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:50.293288+0000 front 2024-03-18T21:02:50.293274+0000 (oldest deadline 2024-03-18T21:03:20.194356+0000) 2024-03-18T21:03:26.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:25 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:25.970+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-03-18T21:03:00.194964+0000 front 2024-03-18T21:03:00.195060+0000 (oldest deadline 2024-03-18T21:03:25.494942+0000) 2024-03-18T21:03:27.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:26 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:26.929+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:27.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:26 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:26.929+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:27.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:26 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:26.929+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:27.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:26 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:26.929+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:27.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:26 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:26.929+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:27.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:26 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:26.929+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:50.293288+0000 front 2024-03-18T21:02:50.293274+0000 (oldest deadline 2024-03-18T21:03:20.194356+0000) 2024-03-18T21:03:27.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:26 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:26.929+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-03-18T21:03:00.194964+0000 front 2024-03-18T21:03:00.195060+0000 (oldest deadline 2024-03-18T21:03:25.494942+0000) 2024-03-18T21:03:27.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:26 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:26.864+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:27.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:26 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:26.864+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:27.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:26 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:26.864+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:27.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:26 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:26.864+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:27.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:26 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:26.864+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:27.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:26 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:26.864+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:28.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:27 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:27.937+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:28.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:27 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:27.937+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:28.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:27 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:27.937+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:28.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:27 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:27.937+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:28.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:27 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:27.937+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:28.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:27 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:27.937+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:50.293288+0000 front 2024-03-18T21:02:50.293274+0000 (oldest deadline 2024-03-18T21:03:20.194356+0000) 2024-03-18T21:03:28.219 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:27 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:27.937+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-03-18T21:03:00.194964+0000 front 2024-03-18T21:03:00.195060+0000 (oldest deadline 2024-03-18T21:03:25.494942+0000) 2024-03-18T21:03:28.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:27 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:27.894+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:28.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:27 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:27.894+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:28.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:27 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:27.894+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:28.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:27 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:27.894+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:28.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:27 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:27.894+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:28.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:27 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:27.894+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:29.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:28 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:28.934+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:29.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:28 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:28.934+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:29.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:28 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:28.934+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:29.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:28 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:28.934+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:29.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:28 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:28.934+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:29.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:28 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:28.934+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:29.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:28 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:28.979+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:29.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:28 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:28.979+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:29.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:28 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:28.979+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:29.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:28 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:28.979+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:29.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:28 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:28.979+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:29.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:28 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:28.979+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:50.293288+0000 front 2024-03-18T21:02:50.293274+0000 (oldest deadline 2024-03-18T21:03:20.194356+0000) 2024-03-18T21:03:29.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:28 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:28.979+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-03-18T21:03:00.194964+0000 front 2024-03-18T21:03:00.195060+0000 (oldest deadline 2024-03-18T21:03:25.494942+0000) 2024-03-18T21:03:30.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:29 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:29.956+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:30.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:29 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:29.956+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:30.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:29 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:29.956+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:30.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:29 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:29.956+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:30.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:29 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:29.956+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:30.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:29 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:29.956+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:30.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:30 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:30.022+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:17.087586+0000 front 2024-03-18T21:02:17.087623+0000 (oldest deadline 2024-03-18T21:02:39.387535+0000) 2024-03-18T21:03:30.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:30 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:30.022+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:23.989217+0000 front 2024-03-18T21:02:23.989241+0000 (oldest deadline 2024-03-18T21:02:48.689386+0000) 2024-03-18T21:03:30.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:30 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:30.022+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:30.390367+0000 front 2024-03-18T21:02:30.390458+0000 (oldest deadline 2024-03-18T21:02:54.490474+0000) 2024-03-18T21:03:30.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:30 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:30.022+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:34.491085+0000 front 2024-03-18T21:02:34.490956+0000 (oldest deadline 2024-03-18T21:03:05.691526+0000) 2024-03-18T21:03:30.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:30 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:30.022+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:45.691902+0000 front 2024-03-18T21:02:45.691924+0000 (oldest deadline 2024-03-18T21:03:10.292669+0000) 2024-03-18T21:03:30.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:30 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:30.022+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:50.293288+0000 front 2024-03-18T21:02:50.293274+0000 (oldest deadline 2024-03-18T21:03:20.194356+0000) 2024-03-18T21:03:30.469 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:30 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10[77534]: 2024-03-18T21:03:30.022+0000 7f859c8a3700 -1 osd.10 125 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-03-18T21:03:00.194964+0000 front 2024-03-18T21:03:00.195060+0000 (oldest deadline 2024-03-18T21:03:25.494942+0000) 2024-03-18T21:03:31.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:30 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:30.957+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:31.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:30 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:30.957+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:31.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:30 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:30.957+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:31.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:30 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:30.957+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:31.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:30 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:30.957+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:31.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:30 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:30.957+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:31.719 INFO:journalctl@ceph.osd.10.smithi073.stdout:Mar 18 21:03:31 smithi073 bash[93989]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-10 2024-03-18T21:03:32.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:31 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:31.981+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:32.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:31 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:31.981+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:32.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:31 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:31.981+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:32.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:31 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:31.981+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:32.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:31 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:31.981+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:32.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:31 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:31.981+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:32.616 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.10.service' 2024-03-18T21:03:32.658 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:03:32.658 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-18T21:03:32.658 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-18T21:03:32.658 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.11 2024-03-18T21:03:33.039 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:32 smithi155 systemd[1]: Stopping Ceph osd.11 for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:03:33.039 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:32.919+0000 7f677c1b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:03:33.039 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:32.919+0000 7f677c1b8700 -1 osd.11 125 *** Got signal Terminated *** 2024-03-18T21:03:33.039 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:32.919+0000 7f677c1b8700 -1 osd.11 125 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T21:03:33.040 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:32.932+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:33.040 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:32.932+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:33.040 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:32.932+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:33.040 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:32.932+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:33.040 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:32.932+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:33.040 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:32.932+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:33.040 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:32 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:32.932+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-03-18T21:03:08.885266+0000 front 2024-03-18T21:03:08.885260+0000 (oldest deadline 2024-03-18T21:03:32.885771+0000) 2024-03-18T21:03:34.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:33 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:33.951+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:34.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:33 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:33.951+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:34.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:33 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:33.951+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:34.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:33 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:33.951+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:34.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:33 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:33.951+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:34.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:33 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:33.951+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:34.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:33 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:33.951+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-03-18T21:03:08.885266+0000 front 2024-03-18T21:03:08.885260+0000 (oldest deadline 2024-03-18T21:03:32.885771+0000) 2024-03-18T21:03:35.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:34 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:34.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:35.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:34 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:34.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:35.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:34 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:34.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:35.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:34 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:34.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:35.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:34 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:34.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:35.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:34 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:34.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:35.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:34 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:34.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-03-18T21:03:08.885266+0000 front 2024-03-18T21:03:08.885260+0000 (oldest deadline 2024-03-18T21:03:32.885771+0000) 2024-03-18T21:03:36.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:35 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:35.938+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:36.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:35 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:35.938+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:36.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:35 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:35.938+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:36.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:35 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:35.938+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:36.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:35 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:35.938+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:36.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:35 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:35.938+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:36.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:35 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:35.938+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-03-18T21:03:08.885266+0000 front 2024-03-18T21:03:08.885260+0000 (oldest deadline 2024-03-18T21:03:32.885771+0000) 2024-03-18T21:03:37.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:36 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:36.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:37.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:36 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:36.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:37.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:36 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:36.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:37.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:36 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:36.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:37.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:36 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:36.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:37.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:36 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:36.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:37.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:36 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:36.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-03-18T21:03:08.885266+0000 front 2024-03-18T21:03:08.885260+0000 (oldest deadline 2024-03-18T21:03:32.885771+0000) 2024-03-18T21:03:37.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:36 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:36.928+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6820 osd.8 since back 2024-03-18T21:03:12.886341+0000 front 2024-03-18T21:03:12.886364+0000 (oldest deadline 2024-03-18T21:03:36.887071+0000) 2024-03-18T21:03:38.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:37 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:37.900+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-03-18T21:02:14.775587+0000 front 2024-03-18T21:02:14.775615+0000 (oldest deadline 2024-03-18T21:02:39.475653+0000) 2024-03-18T21:03:38.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:37 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:37.900+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-03-18T21:02:21.776795+0000 front 2024-03-18T21:02:21.776607+0000 (oldest deadline 2024-03-18T21:02:47.076722+0000) 2024-03-18T21:03:38.289 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:37 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:37.900+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6804 osd.2 since back 2024-03-18T21:02:31.678303+0000 front 2024-03-18T21:02:31.678330+0000 (oldest deadline 2024-03-18T21:02:56.378469+0000) 2024-03-18T21:03:38.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:37 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:37.900+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-03-18T21:02:38.079738+0000 front 2024-03-18T21:02:38.079808+0000 (oldest deadline 2024-03-18T21:03:02.179725+0000) 2024-03-18T21:03:38.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:37 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:37.900+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-03-18T21:02:46.281041+0000 front 2024-03-18T21:02:46.281204+0000 (oldest deadline 2024-03-18T21:03:09.781029+0000) 2024-03-18T21:03:38.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:37 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:37.900+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6812 osd.5 since back 2024-03-18T21:02:53.881937+0000 front 2024-03-18T21:02:53.881964+0000 (oldest deadline 2024-03-18T21:03:17.982034+0000) 2024-03-18T21:03:38.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:37 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:37.900+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-03-18T21:03:08.885266+0000 front 2024-03-18T21:03:08.885260+0000 (oldest deadline 2024-03-18T21:03:32.885771+0000) 2024-03-18T21:03:38.290 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:37 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11[78743]: 2024-03-18T21:03:37.900+0000 7f6776fab700 -1 osd.11 125 heartbeat_check: no reply from 172.21.15.155:6820 osd.8 since back 2024-03-18T21:03:12.886341+0000 front 2024-03-18T21:03:12.886364+0000 (oldest deadline 2024-03-18T21:03:36.887071+0000) 2024-03-18T21:03:38.789 INFO:journalctl@ceph.osd.11.smithi155.stdout:Mar 18 21:03:38 smithi155 bash[98089]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-osd-11 2024-03-18T21:03:39.749 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@osd.11.service' 2024-03-18T21:03:39.790 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:03:39.790 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-18T21:03:39.791 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-18T21:03:39.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.a 2024-03-18T21:03:40.295 INFO:journalctl@ceph.mds.a.smithi046.stdout:Mar 18 21:03:39 smithi046 systemd[1]: Stopping Ceph mds.a for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:03:40.296 INFO:journalctl@ceph.mds.a.smithi046.stdout:Mar 18 21:03:40 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-a[93799]: 2024-03-18T21:03:40.041+0000 7fa8870c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:03:40.296 INFO:journalctl@ceph.mds.a.smithi046.stdout:Mar 18 21:03:40 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-a[93799]: 2024-03-18T21:03:40.041+0000 7fa8870c2700 -1 mds.a *** got signal Terminated *** 2024-03-18T21:03:42.045 INFO:journalctl@ceph.mds.a.smithi046.stdout:Mar 18 21:03:41 smithi046 bash[138434]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-a 2024-03-18T21:03:42.093 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.a.service' 2024-03-18T21:03:42.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:03:42.138 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-18T21:03:42.138 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-18T21:03:42.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.d 2024-03-18T21:03:42.419 INFO:journalctl@ceph.mds.d.smithi046.stdout:Mar 18 21:03:42 smithi046 systemd[1]: Stopping Ceph mds.d for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:03:42.795 INFO:journalctl@ceph.mds.d.smithi046.stdout:Mar 18 21:03:42 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-d[92887]: 2024-03-18T21:03:42.416+0000 7f3c9714c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:03:42.795 INFO:journalctl@ceph.mds.d.smithi046.stdout:Mar 18 21:03:42 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-d[92887]: 2024-03-18T21:03:42.416+0000 7f3c9714c700 -1 mds.d *** got signal Terminated *** 2024-03-18T21:03:46.977 INFO:journalctl@ceph.mds.d.smithi046.stdout:Mar 18 21:03:46 smithi046 bash[138580]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-d 2024-03-18T21:03:46.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.d.service' 2024-03-18T21:03:47.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:03:47.023 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-18T21:03:47.023 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-18T21:03:47.023 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.g 2024-03-18T21:03:47.295 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 21:03:47 smithi046 systemd[1]: Stopping Ceph mds.g for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:03:47.795 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 21:03:47 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-g[93409]: 2024-03-18T21:03:47.313+0000 7f6d0b23e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:03:47.795 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 21:03:47 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-g[93409]: 2024-03-18T21:03:47.313+0000 7f6d0b23e700 -1 mds.g *** got signal Terminated *** 2024-03-18T21:03:49.962 INFO:journalctl@ceph.mds.g.smithi046.stdout:Mar 18 21:03:49 smithi046 bash[138718]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-g 2024-03-18T21:03:49.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.g.service' 2024-03-18T21:03:50.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:03:50.007 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-18T21:03:50.007 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-18T21:03:50.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.j 2024-03-18T21:03:50.283 INFO:journalctl@ceph.mds.j.smithi046.stdout:Mar 18 21:03:50 smithi046 systemd[1]: Stopping Ceph mds.j for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:03:50.545 INFO:journalctl@ceph.mds.j.smithi046.stdout:Mar 18 21:03:50 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-j[92498]: 2024-03-18T21:03:50.281+0000 7f5134263700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:03:50.545 INFO:journalctl@ceph.mds.j.smithi046.stdout:Mar 18 21:03:50 smithi046 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-j[92498]: 2024-03-18T21:03:50.281+0000 7f5134263700 -1 mds.j *** got signal Terminated *** 2024-03-18T21:03:53.212 INFO:journalctl@ceph.mds.j.smithi046.stdout:Mar 18 21:03:52 smithi046 bash[138859]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-j 2024-03-18T21:03:53.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.j.service' 2024-03-18T21:03:53.258 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:03:53.258 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-18T21:03:53.258 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-18T21:03:53.259 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.b 2024-03-18T21:03:53.719 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 21:03:53 smithi073 systemd[1]: Stopping Ceph mds.b for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:03:53.719 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 21:03:53 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: 2024-03-18T21:03:53.498+0000 7f9d8f161700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:03:53.719 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 21:03:53 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b[83809]: 2024-03-18T21:03:53.498+0000 7f9d8f161700 -1 mds.b *** got signal Terminated *** 2024-03-18T21:03:57.958 INFO:journalctl@ceph.mds.b.smithi073.stdout:Mar 18 21:03:57 smithi073 bash[94246]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-b 2024-03-18T21:03:57.964 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.b.service' 2024-03-18T21:03:58.000 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:03:58.000 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-18T21:03:58.000 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-18T21:03:58.001 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.e 2024-03-18T21:03:58.219 INFO:journalctl@ceph.mds.e.smithi073.stdout:Mar 18 21:03:58 smithi073 systemd[1]: Stopping Ceph mds.e for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:03:58.718 INFO:journalctl@ceph.mds.e.smithi073.stdout:Mar 18 21:03:58 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-e[83127]: 2024-03-18T21:03:58.290+0000 7f8e29609700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:03:58.719 INFO:journalctl@ceph.mds.e.smithi073.stdout:Mar 18 21:03:58 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-e[83127]: 2024-03-18T21:03:58.290+0000 7f8e29609700 -1 mds.e *** got signal Terminated *** 2024-03-18T21:04:02.161 INFO:journalctl@ceph.mds.e.smithi073.stdout:Mar 18 21:04:01 smithi073 bash[94385]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-e 2024-03-18T21:04:02.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.e.service' 2024-03-18T21:04:02.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:04:02.200 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-18T21:04:02.200 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-18T21:04:02.201 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.h 2024-03-18T21:04:02.465 INFO:journalctl@ceph.mds.h.smithi073.stdout:Mar 18 21:04:02 smithi073 systemd[1]: Stopping Ceph mds.h for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:04:02.719 INFO:journalctl@ceph.mds.h.smithi073.stdout:Mar 18 21:04:02 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-h[83470]: 2024-03-18T21:04:02.462+0000 7f7a9af3b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:04:02.719 INFO:journalctl@ceph.mds.h.smithi073.stdout:Mar 18 21:04:02 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-h[83470]: 2024-03-18T21:04:02.462+0000 7f7a9af3b700 -1 mds.h *** got signal Terminated *** 2024-03-18T21:04:06.040 INFO:journalctl@ceph.mds.h.smithi073.stdout:Mar 18 21:04:05 smithi073 bash[94524]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-h 2024-03-18T21:04:06.046 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.h.service' 2024-03-18T21:04:06.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:04:06.083 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-18T21:04:06.083 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-18T21:04:06.083 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.k 2024-03-18T21:04:06.357 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 21:04:06 smithi073 systemd[1]: Stopping Ceph mds.k for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:04:06.719 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 21:04:06 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: 2024-03-18T21:04:06.354+0000 7f4a71e77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:04:06.719 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 21:04:06 smithi073 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k[82787]: 2024-03-18T21:04:06.354+0000 7f4a71e77700 -1 mds.k *** got signal Terminated *** 2024-03-18T21:04:09.970 INFO:journalctl@ceph.mds.k.smithi073.stdout:Mar 18 21:04:09 smithi073 bash[94671]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-k 2024-03-18T21:04:09.986 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.k.service' 2024-03-18T21:04:10.021 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:04:10.021 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-18T21:04:10.021 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-18T21:04:10.021 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.c 2024-03-18T21:04:10.539 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 21:04:10 smithi155 systemd[1]: Stopping Ceph mds.c for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:04:10.539 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 21:04:10 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: 2024-03-18T21:04:10.255+0000 7f8a0eeaf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:04:10.539 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 21:04:10 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c[84975]: 2024-03-18T21:04:10.255+0000 7f8a0eeaf700 -1 mds.c *** got signal Terminated *** 2024-03-18T21:04:14.039 INFO:journalctl@ceph.mds.c.smithi155.stdout:Mar 18 21:04:13 smithi155 bash[98350]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-c 2024-03-18T21:04:14.067 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.c.service' 2024-03-18T21:04:14.105 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:04:14.105 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-18T21:04:14.105 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-18T21:04:14.106 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.f 2024-03-18T21:04:14.364 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 18 21:04:14 smithi155 systemd[1]: Stopping Ceph mds.f for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:04:14.789 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 18 21:04:14 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-f[84635]: 2024-03-18T21:04:14.362+0000 7fd0c685f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:04:14.789 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 18 21:04:14 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-f[84635]: 2024-03-18T21:04:14.362+0000 7fd0c685f700 -1 mds.f *** got signal Terminated *** 2024-03-18T21:04:18.039 INFO:journalctl@ceph.mds.f.smithi155.stdout:Mar 18 21:04:17 smithi155 bash[98489]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-f 2024-03-18T21:04:18.111 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.f.service' 2024-03-18T21:04:18.145 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:04:18.145 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-18T21:04:18.145 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-18T21:04:18.146 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.i 2024-03-18T21:04:18.413 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 21:04:18 smithi155 systemd[1]: Stopping Ceph mds.i for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:04:18.789 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 21:04:18 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-i[85318]: 2024-03-18T21:04:18.411+0000 7f9d727d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:04:18.789 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 21:04:18 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-i[85318]: 2024-03-18T21:04:18.411+0000 7f9d727d9700 -1 mds.i *** got signal Terminated *** 2024-03-18T21:04:22.039 INFO:journalctl@ceph.mds.i.smithi155.stdout:Mar 18 21:04:21 smithi155 bash[98629]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-i 2024-03-18T21:04:22.107 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.i.service' 2024-03-18T21:04:22.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:04:22.141 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-18T21:04:22.141 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-18T21:04:22.141 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.l 2024-03-18T21:04:22.412 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 18 21:04:22 smithi155 systemd[1]: Stopping Ceph mds.l for 04a8ae18-e566-11ee-95c9-87774f69a715... 2024-03-18T21:04:22.789 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 18 21:04:22 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-l[84292]: 2024-03-18T21:04:22.410+0000 7fd5ba4bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T21:04:22.789 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 18 21:04:22 smithi155 ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-l[84292]: 2024-03-18T21:04:22.410+0000 7fd5ba4bc700 -1 mds.l *** got signal Terminated *** 2024-03-18T21:04:26.039 INFO:journalctl@ceph.mds.l.smithi155.stdout:Mar 18 21:04:25 smithi155 bash[98771]: ceph-04a8ae18-e566-11ee-95c9-87774f69a715-mds-l 2024-03-18T21:04:26.095 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-04a8ae18-e566-11ee-95c9-87774f69a715@mds.l.service' 2024-03-18T21:04:26.129 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T21:04:26.129 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-18T21:04:26.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T21:04:26.329 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T21:04:28.002 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T21:04:28.193 INFO:teuthology.orchestra.run.smithi073.stdout:Deleting cluster with fsid: 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T21:04:29.646 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T21:04:29.845 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T21:04:31.311 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T21:04:31.350 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T21:04:31.386 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T21:04:31.417 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-18T21:04:31.419 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382/remote/smithi046/crash 2024-03-18T21:04:31.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/crash -- . 2024-03-18T21:04:31.486 INFO:teuthology.orchestra.run.smithi046.stderr:tar: /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T21:04:31.486 INFO:teuthology.orchestra.run.smithi046.stderr:tar: Error is not recoverable: exiting now 2024-03-18T21:04:31.488 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382/remote/smithi073/crash 2024-03-18T21:04:31.489 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/crash -- . 2024-03-18T21:04:31.536 INFO:teuthology.orchestra.run.smithi073.stderr:tar: /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T21:04:31.536 INFO:teuthology.orchestra.run.smithi073.stderr:tar: Error is not recoverable: exiting now 2024-03-18T21:04:31.538 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382/remote/smithi155/crash 2024-03-18T21:04:31.539 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/crash -- . 2024-03-18T21:04:31.605 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T21:04:31.605 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2024-03-18T21:04:31.606 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-18T21:04:31.606 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-18T21:04:31.648 INFO:teuthology.orchestra.run.smithi046.stderr:grep: /var/log/ceph/04a8ae18-e566-11ee-95c9-87774f69a715/ceph.log: No such file or directory 2024-03-18T21:04:31.650 INFO:tasks.cephadm:Compressing logs... 2024-03-18T21:04:31.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T21:04:31.694 DEBUG:teuthology.orchestra.run.smithi073:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T21:04:31.696 DEBUG:teuthology.orchestra.run.smithi155:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T21:04:31.734 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-18T21:04:31.736 INFO:teuthology.orchestra.run.smithi073.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-18T21:04:31.739 INFO:teuthology.orchestra.run.smithi046.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-18T21:18:50.435 INFO:tasks.cephadm:Archiving logs... 2024-03-18T21:18:50.458 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382/remote/smithi046/log 2024-03-18T21:18:50.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T21:19:32.880 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382/remote/smithi073/log 2024-03-18T21:19:32.882 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T21:21:16.119 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382/remote/smithi155/log 2024-03-18T21:21:16.123 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T21:22:01.704 INFO:tasks.cephadm:Removing cluster... 2024-03-18T21:22:01.705 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 --force 2024-03-18T21:22:02.017 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T21:22:02.685 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 --force 2024-03-18T21:22:03.043 INFO:teuthology.orchestra.run.smithi073.stdout:Deleting cluster with fsid: 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T21:22:03.867 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 04a8ae18-e566-11ee-95c9-87774f69a715 --force 2024-03-18T21:22:04.122 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 04a8ae18-e566-11ee-95c9-87774f69a715 2024-03-18T21:22:04.725 INFO:tasks.cephadm:Removing cephadm ... 2024-03-18T21:22:04.725 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T21:22:04.748 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T21:22:04.783 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T21:22:04.805 INFO:tasks.cephadm:Teardown complete 2024-03-18T21:22:04.805 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-18T21:22:04.820 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-18T21:22:04.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T21:22:04.824 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T21:22:04.826 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T21:22:04.952 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-18T21:22:04.963 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-18T21:22:04.963 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T21:22:04.965 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T21:22:04.968 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T21:22:04.983 INFO:teuthology.orchestra.run.smithi073.stderr:bash: ntpq: command not found 2024-03-18T21:22:04.984 INFO:teuthology.orchestra.run.smithi046.stderr:bash: ntpq: command not found 2024-03-18T21:22:04.984 INFO:teuthology.orchestra.run.smithi155.stderr:bash: ntpq: command not found 2024-03-18T21:22:05.002 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T21:22:05.002 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-03-18T21:22:05.002 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 +2968us[+2968us] +/- 108ms 2024-03-18T21:22:05.002 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 -1786us[-1740us] +/- 61ms 2024-03-18T21:22:05.003 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 +457us[ +502us] +/- 83ms 2024-03-18T21:22:05.003 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T21:22:05.026 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T21:22:05.026 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-03-18T21:22:05.026 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 +2944us[+2959us] +/- 108ms 2024-03-18T21:22:05.026 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -1826us[-1809us] +/- 61ms 2024-03-18T21:22:05.026 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 +433us[ +447us] +/- 83ms 2024-03-18T21:22:05.026 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T21:22:05.028 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T21:22:05.028 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-03-18T21:22:05.028 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 +2985us[+2990us] +/- 108ms 2024-03-18T21:22:05.028 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -1795us[-1790us] +/- 61ms 2024-03-18T21:22:05.028 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +452us[ +452us] +/- 83ms 2024-03-18T21:22:05.028 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T21:22:05.028 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-18T21:22:05.039 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-18T21:22:05.040 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-18T21:22:05.058 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T21:22:05.211 DEBUG:teuthology.orchestra.run.smithi073:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T21:22:05.337 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T21:22:05.456 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-18T21:22:05.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:22:05.496 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-18T21:22:05.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:22:05.532 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-18T21:22:05.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:22:05.562 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-18T21:22:05.573 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-18T21:22:05.590 INFO:teuthology.task.internal:Duration was 4082.020383 seconds 2024-03-18T21:22:05.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-18T21:22:05.605 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-18T21:22:05.605 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T21:22:05.607 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T21:22:05.610 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T21:22:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T21:22:05.704 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T21:22:05.713 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T21:22:06.069 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-18T21:22:06.069 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-03-18T21:22:06.069 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T21:22:06.107 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2024-03-18T21:22:06.108 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T21:22:06.140 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-03-18T21:22:06.140 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T21:22:06.165 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-18T21:22:06.166 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T21:22:06.168 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T21:22:06.182 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T21:22:06.292 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-18T21:22:06.292 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T21:22:06.295 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T21:22:06.297 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T21:22:06.429 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-18T21:22:06.441 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-18T21:22:06.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T21:22:06.499 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T21:22:06.550 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T21:22:06.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-18T21:22:06.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T21:22:06.619 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T21:22:06.621 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T21:22:06.655 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-03-18T21:22:06.659 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2024-03-18T21:22:06.680 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-03-18T21:22:06.700 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T21:22:06.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:22:06.720 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T21:22:06.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:22:06.740 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T21:22:06.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T21:22:06.756 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-18T21:22:06.769 INFO:teuthology.task.internal:Transferring archived files... 2024-03-18T21:22:06.770 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382/remote/smithi046 2024-03-18T21:22:06.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T21:22:06.822 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382/remote/smithi073 2024-03-18T21:22:06.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T21:22:06.863 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382/remote/smithi155 2024-03-18T21:22:06.863 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T21:22:06.905 INFO:teuthology.task.internal:Removing archive directory... 2024-03-18T21:22:06.905 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T21:22:06.907 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T21:22:06.910 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T21:22:06.955 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-18T21:22:06.969 INFO:teuthology.task.internal:Not uploading archives. 2024-03-18T21:22:06.970 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-18T21:22:06.981 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-18T21:22:06.982 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T21:22:06.984 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T21:22:06.987 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T21:22:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 21:22 /home/ubuntu/cephtest 2024-03-18T21:22:07.003 INFO:teuthology.orchestra.run.smithi046.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 21:22 /home/ubuntu/cephtest 2024-03-18T21:22:07.011 INFO:teuthology.orchestra.run.smithi155.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 21:22 /home/ubuntu/cephtest 2024-03-18T21:22:07.012 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-18T21:22:07.026 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-18T21:22:07.071 INFO:teuthology.nuke:Checking targets against current locks 2024-03-18T21:22:07.098 DEBUG:teuthology.nuke:shortname: smithi046 2024-03-18T21:22:07.098 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T21:22:07.118 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 20:08:32.955827', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJjbaIubNluIMJ5WYE2UsGKZYMdPrlAlHELCOZgn5CAEzD7NPU+mrkUwcL48e7pZz1x+aY1v/0ku468yC0+qBXU='} 2024-03-18T21:22:07.123 DEBUG:teuthology.nuke:shortname: smithi073 2024-03-18T21:22:07.123 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T21:22:07.146 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 20:08:32.957238', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFEsrw1C5H2lCkAnrn/V52JlpCJU99u6LHS4CKEwe7In5E93NlG7KT7uJI4l+G5YLeA6Jsm/yMh9TwxiRdXb5+s='} 2024-03-18T21:22:07.149 DEBUG:teuthology.nuke:shortname: smithi155 2024-03-18T21:22:07.149 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T21:22:07.172 INFO:teuthology.orchestra.console.smithi046:Power off 2024-03-18T21:22:07.173 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T21:22:07.185 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-03-16_15:03:17-fs-wip-yuri10-testing-2024-03-15-1653-reef-distro-default-smithi/7606382', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 20:08:32.954227', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH3GcrcPA+A6q8OGeEMHtFZt1ALZ47FuwpBd94qmQw+xP7LNWuj5zeeEHZBfgI+nZzGsrtajzFUEjRTnGRf2qO8='} 2024-03-18T21:22:07.196 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off 2024-03-18T21:22:07.196 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T21:22:07.218 INFO:teuthology.orchestra.console.smithi073:Power off 2024-03-18T21:22:07.219 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T21:22:07.227 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2024-03-18T21:22:07.240 DEBUG:teuthology.orchestra.console.smithi073:power off output: Chassis Power Control: Down/Off 2024-03-18T21:22:07.240 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T21:22:07.257 INFO:teuthology.orchestra.console.smithi155:Power off 2024-03-18T21:22:07.257 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T21:22:07.266 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2024-03-18T21:22:07.280 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2024-03-18T21:22:07.281 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T21:22:07.308 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-03-18T21:22:11.228 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T21:22:11.267 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T21:22:11.308 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T21:22:11.338 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2024-03-18T21:22:11.376 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2024-03-18T21:22:11.417 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-03-18T21:22:15.342 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T21:22:15.377 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T21:22:15.418 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T21:22:15.452 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2024-03-18T21:22:15.487 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2024-03-18T21:22:15.527 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-03-18T21:22:15.553 INFO:teuthology.orchestra.console.smithi046:Power off completed 2024-03-18T21:22:19.492 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T21:22:19.529 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T21:22:19.602 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is off 2024-03-18T21:22:19.639 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2024-03-18T21:22:19.703 INFO:teuthology.orchestra.console.smithi073:Power off completed 2024-03-18T21:22:19.740 INFO:teuthology.orchestra.console.smithi155:Power off completed 2024-03-18T21:22:19.845 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 4082.0203828811646 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-03-18T21:22:19.846 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T21:22:19.921 INFO:teuthology.run:FAIL